Apache Kafka has become one of the most popular distributed event streaming platforms for building real-time applications, data pipelines, and event-driven architectures. With the introduction of KRaft (Kafka Raft Metadata Mode), Kafka no longer requires ZooKeeper, making installation and management much simpler. In this guide, you’ll learn how to set up Apache Kafka in KRaft mode on Windows for local development, configure the broker, initialize metadata storage, create topics, and test message streaming using Kafka producers and consumers.

What Is KRaft Mode in Apache Kafka?

Prior to Apache Kafka 3.x, every Kafka cluster depended on Apache ZooKeeper to manage metadata and coordinate brokers.

Modern versions of Kafka introduce KRaft (Kafka Raft Metadata Mode), which replaces ZooKeeper with Kafka’s own consensus mechanism based on the Raft protocol.

Benefits of KRaft Mode

• No ZooKeeper installation required
• Easier cluster management
• Faster setup for local development
• Reduced infrastructure complexity
• Recommended architecture for new Kafka deployments

If you’re starting a new Kafka project, KRaft mode is the preferred approach.

Prerequisites

Before installing Kafka, make sure your system meets the following requirements:

• Windows 10 or Windows 11
• Java Development Kit (JDK) 11 or later
• Command Prompt or PowerShell
• Apache Kafka binary package

Verify Java installation by running:

java -version

If Java is installed correctly, you’ll see the installed JDK version.

Step 1: Download Apache Kafka

Download the latest Apache Kafka binary package from the Official Apache Kafka Website.

After downloading:

1. Move the ZIP file to a convenient location such as:

C:\

2. Extract the archive.
3. Rename the extracted folder to something simple like:

C:\Kafka

Using a short directory path makes Kafka commands easier to execute throughout development.

Step 2: Verify the Installation

Open the Kafka folder and confirm that important directories are present.

Typical folders include:

• bin
• config
• libs
• logs (created later)

If these folders exist, the installation was extracted successfully.

Step 3: Configure Kafka

Navigate to the configuration directory:

C:\Kafka\config

Open the server configuration file:

server.properties

Modify the configuration by adding or updating settings for:

• Kafka log directory
• Server host (localhost)
• Default listening port

These settings determine where Kafka stores its data and how clients connect to the broker.

After making the changes, save the file.

Step 4: Initialize Kafka in KRaft Mode

Unlike older Kafka versions, KRaft mode requires initializing metadata storage before the broker starts.

The process consists of three steps.

Generate a Cluster ID

Generate a random UUID that will serve as the Kafka cluster ID.

Example:

kafka-storage.bat random-uuid

Copy the generated UUID.

Format the Metadata Storage

Initialize Kafka’s metadata storage using the generated Cluster ID.

Example:

kafka-storage.bat format -t <Cluster-ID> -c config\server.properties

This creates the metadata required for Kafka to operate in KRaft mode.

Start the Kafka Broker

After formatting the storage, start Kafka.

Example:

kafka-server-start.bat config\server.properties

//In powershell
PS C:\kafka>  .\bin\windows\kafka-server-start.bat config\server.properties

Leave this terminal window running.

This terminal represents your Kafka server.

Step 5: Create a Kafka Topic

Open a new Command Prompt window.

Create your first topic.

Example:

kafka-topics.bat --create --topic test-topic

A topic acts as a channel where producers publish events and consumers read them.

Step 6: Start a Kafka Producer

Open another terminal window.

Launch the producer for your topic.

Example:

kafka-console-producer.bat --topic test-topic

Anything typed into this terminal is published to Kafka.

Example messages:

Hello KafkaFirst EventLearning Kafka Streaming

Step 7: Start a Kafka Consumer

Open one final terminal.

Start the consumer.

Example:

kafka-console-consumer.bat --topic test-topic --from-beginning

The consumer continuously listens for new messages published by the producer.

Whenever you send a message from the producer terminal, it immediately appears in the consumer terminal.

This demonstrates Kafka’s event streaming capabilities in real time.

Step 8: Stop the Kafka server

When the learning is finished open the new terminal or powershell

in powershell from the correct windows directory enter kafka-server-stop.bat

PS C:\kafka> .\bin\windows\kafka-server-stop.bat
Deleting instance \\RT\ROOT\CIMV2:Win32_Process.Handle="20124"
Instance deletion successful.

Understanding the Kafka Workflow

The complete local workflow looks like this:

1. Install Java
2. Download Apache Kafka
3. Configure server.properties
4. Generate a Cluster ID
5. Initialize KRaft metadata
6. Start Kafka broker
7. Create a topic
8. Start a producer
9. Start a consumer
10. Exchange messages in real time

This simple setup helps developers understand the core concepts behind Kafka’s publish-subscribe messaging model.

Why Use Kafka in KRaft Mode?

KRaft mode offers several advantages over the traditional ZooKeeper-based architecture.

Simpler Architecture

Everything is managed directly by Kafka.

Easier Local Development

No need to install or configure ZooKeeper.

Better Performance

Metadata management is more efficient and scalable.

Future-Proof

ZooKeeper has been deprecated for new Kafka deployments, making KRaft the recommended choice moving forward.

Common Issues During Setup

If Kafka doesn’t start correctly, check the following:

• Java version is 11 or higher.
• The metadata storage has been formatted before starting the broker.
• The Cluster ID was generated successfully.
• The configured log directory exists.
• No other application is using Kafka’s configured port.
• The broker is running before creating topics or starting producers and consumers.

Most startup issues are caused by skipped initialization steps or configuration errors.

Best Practices for Local Kafka Development

For a smoother development experience:

• Keep Kafka in a short directory path like C:\Kafka.
• Use descriptive topic names.
• Keep separate terminals for the broker, producer, and consumer.
• Verify configuration changes before starting Kafka.
• Test message flow with small sample messages before integrating applications.

These practices make troubleshooting much easier as you begin building Kafka-based applications.

Conclusion

Setting up Apache Kafka in KRaft mode on Windows is significantly easier than older ZooKeeper-based installations. By generating a Cluster ID, initializing metadata storage, configuring the broker, and testing producers and consumers, you can quickly build a local Kafka environment for learning, development, and experimentation. As KRaft becomes the standard architecture for Apache Kafka, mastering this setup is an essential skill for developers working with event-driven systems, real-time data streaming, and modern microservices architectures.

Frequently Asked Questions (FAQ)

Do I still need ZooKeeper for Apache Kafka?

No. Modern versions of Apache Kafka support KRaft mode, which eliminates the need for ZooKeeper and is the recommended deployment option for new Kafka clusters.

What Java version is required for Kafka?

Most recent Kafka releases require Java 11 or later. Always verify the compatibility with the Kafka version you download.

Why do I need to generate a Cluster ID?

The Cluster ID uniquely identifies your Kafka cluster and is required to initialize the metadata storage used by KRaft mode.

Can I run Kafka locally on Windows?

Yes. Kafka runs well on Windows for development and testing. You can create topics, publish messages, and consume events using Command Prompt or PowerShell.

How many terminal windows do I need?

For a basic local setup, you’ll typically use four terminal windows:

• Kafka broker
• Topic management (optional)
• Producer
• Consumer

This setup allows you to simulate real-time message streaming on your local machine.

Related Links

Kafka Testing Page