How To

Install Apache Kafka on Ubuntu 24.04 / 22.04

Kibet John Updated Mar 22, 2026 · 8 min read

Apache Kafka is a distributed event streaming platform built for high-throughput, fault-tolerant data pipelines and real-time analytics. It handles millions of events per second and is used by companies like LinkedIn, Netflix, and Uber for log aggregation, stream processing, and messaging between microservices.

This guide walks through installing Apache Kafka 4.2 on Ubuntu 24.04 and 22.04 using KRaft mode – the new consensus protocol that replaces ZooKeeper entirely. Starting with Kafka 4.0, ZooKeeper support was removed, making KRaft the only deployment mode. We cover Java 21 installation, Kafka setup, systemd service configuration, topic management, producer/consumer testing, firewall rules, and multi-broker basics.

Prerequisites

  • Ubuntu 24.04 or 22.04 LTS server with at least 2GB RAM and 2 CPU cores
  • Root or sudo access
  • Java 17 or newer (Java 21 LTS recommended – we install it in Step 1)
  • Port 9092 (TCP) open for Kafka broker traffic
  • Port 9093 (TCP) open for KRaft controller communication in multi-broker setups

Step 1: Install Java 21 on Ubuntu

Kafka brokers require Java 17 or higher. We install OpenJDK 21 LTS since it is the current long-term support release. Update the package index and install OpenJDK 21:

sudo apt update
sudo apt install -y openjdk-21-jdk-headless

Confirm Java is installed and reports version 21:

java -version

The output should show the OpenJDK 21 version string:

openjdk version "21.0.6" 2025-01-21
OpenJDK Runtime Environment (build 21.0.6+7-Ubuntu-124.04.1)
OpenJDK 64-Bit Server VM (build 21.0.6+7-Ubuntu-124.04.1, mixed mode, sharing)

Set the JAVA_HOME environment variable so Kafka can locate your Java installation:

echo 'export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64' | sudo tee /etc/profile.d/java.sh
source /etc/profile.d/java.sh

For a more detailed Java setup, see our guide on installing Java 21 LTS on Ubuntu 24.04 / Debian 13.

Step 2: Download and Extract Apache Kafka

Create a dedicated system user for running Kafka. Running it as a non-root user is a basic security practice:

sudo useradd -r -m -s /usr/sbin/nologin kafka

Download the latest Kafka 4.2.0 binary from the Apache Kafka project and extract it to /opt/kafka:

KAFKA_VERSION="4.2.0"
SCALA_VERSION="2.13"
wget https://downloads.apache.org/kafka/${KAFKA_VERSION}/kafka_${SCALA_VERSION}-${KAFKA_VERSION}.tgz -O /tmp/kafka.tgz
sudo tar -xzf /tmp/kafka.tgz -C /opt
sudo mv /opt/kafka_${SCALA_VERSION}-${KAFKA_VERSION} /opt/kafka
sudo chown -R kafka:kafka /opt/kafka
rm -f /tmp/kafka.tgz

Verify the Kafka directory structure is in place:

ls /opt/kafka/

You should see the standard Kafka directories including bin, config, libs, and licenses:

LICENSE  NOTICE  bin  config  libs  licenses  site-docs

Step 3: Configure Kafka in KRaft Mode

Kafka 4.x runs exclusively in KRaft (Kafka Raft) mode. There is no ZooKeeper dependency. KRaft handles metadata management and leader election internally, which simplifies deployment and reduces the number of components to manage.

Create the Kafka data directory:

sudo mkdir -p /var/kafka-logs
sudo chown kafka:kafka /var/kafka-logs

Edit the Kafka server configuration file:

sudo vi /opt/kafka/config/server.properties

Replace the contents with the following KRaft configuration. This sets up a combined controller and broker on a single node:

# KRaft Mode - Combined Controller + Broker
# Node ID - must be unique across the cluster
node.id=1

# Process roles: 'broker' handles client traffic, 'controller' manages metadata
process.roles=broker,controller

# Controller quorum voters: node_id@host:controller_port
controller.quorum.voters=1@localhost:9093

# Listener configuration
listeners=PLAINTEXT://:9092,CONTROLLER://:9093
advertised.listeners=PLAINTEXT://YOUR_SERVER_IP:9092
listener.security.protocol.map=CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT

# Controller listener name (must match one of the listener names above)
controller.listener.names=CONTROLLER
inter.broker.listener.name=PLAINTEXT

# Data directory for logs and metadata
log.dirs=/var/kafka-logs

# Number of partitions for auto-created topics
num.partitions=3

# Replication factor for internal topics (set to 1 for single node)
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1

# Log retention - keep data for 7 days
log.retention.hours=168
log.retention.bytes=1073741824

# Log segment size (1GB)
log.segment.bytes=1073741824

# Frequency of log cleanup checks
log.retention.check.interval.ms=300000

Replace YOUR_SERVER_IP with your server’s actual IP address (e.g., 192.168.1.10). If running Kafka only for local development, you can use localhost instead.

Step 4: Generate Cluster ID and Format Storage

KRaft mode requires a cluster ID to identify the Kafka cluster. Generate a unique cluster ID:

KAFKA_CLUSTER_ID=$(/opt/kafka/bin/kafka-storage.sh random-uuid)
echo $KAFKA_CLUSTER_ID

The command outputs a UUID string that identifies your cluster:

dTy4Xz1bQhKL3FJpk8mN7A

Format the storage directory with this cluster ID. The --standalone flag is used for single-node deployments:

sudo -u kafka /opt/kafka/bin/kafka-storage.sh format --standalone \
  -t $KAFKA_CLUSTER_ID \
  -c /opt/kafka/config/server.properties

The format command initializes the log directory and confirms the metadata version:

Formatting /var/kafka-logs with metadata.version 4.2-IV0.

Step 5: Create Kafka Systemd Service

Create a systemd unit file so Kafka starts automatically on boot and can be managed with standard service commands. If you are new to running Java applications as services, our guide on running Java applications with Systemd covers the fundamentals.

sudo vi /etc/systemd/system/kafka.service

Add the following service definition:

[Unit]
Description=Apache Kafka Server (KRaft Mode)
Documentation=https://kafka.apache.org/documentation/
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=kafka
Group=kafka
Environment="JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64"
Environment="KAFKA_HEAP_OPTS=-Xmx1G -Xms1G"
ExecStart=/opt/kafka/bin/kafka-server-start.sh /opt/kafka/config/server.properties
ExecStop=/opt/kafka/bin/kafka-server-stop.sh
Restart=on-failure
RestartSec=10
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

The KAFKA_HEAP_OPTS setting allocates 1GB of heap memory. Adjust this based on your server’s available RAM – production brokers typically need 4-6GB.

Reload systemd, enable the service to start on boot, and start Kafka:

sudo systemctl daemon-reload
sudo systemctl enable --now kafka

Verify the Kafka service is running without errors:

sudo systemctl status kafka

The service should show active (running) status with the Kafka startup logs:

● kafka.service - Apache Kafka Server (KRaft Mode)
     Loaded: loaded (/etc/systemd/system/kafka.service; enabled; preset: enabled)
     Active: active (running) since Sun 2026-03-22 10:15:32 UTC; 5s ago
       Docs: https://kafka.apache.org/documentation/
   Main PID: 12345 (java)
      Tasks: 72 (limit: 4567)
     Memory: 412.3M
        CPU: 8.521s
     CGroup: /system.slice/kafka.service
             └─12345 java -Xmx1G -Xms1G ...

Confirm Kafka is listening on port 9092:

ss -tlnp | grep 9092

You should see Kafka’s Java process listening on port 9092:

LISTEN  0  50  *:9092  *:*  users:(("java",pid=12345,fd=142))

Step 6: Configure Firewall for Kafka (Port 9092)

If UFW is enabled on your Ubuntu server, open port 9092 for Kafka client connections and port 9093 for controller communication:

sudo ufw allow 9092/tcp comment "Kafka broker"
sudo ufw allow 9093/tcp comment "Kafka controller"
sudo ufw reload

Verify the firewall rules are active:

sudo ufw status verbose

The output should list both Kafka ports as allowed:

9092/tcp                   ALLOW IN    Anywhere                   # Kafka broker
9093/tcp                   ALLOW IN    Anywhere                   # Kafka controller

Step 7: Create Kafka Topics and Test Messaging

With Kafka running, create a test topic to verify everything works. Create a topic named test-events with 3 partitions:

/opt/kafka/bin/kafka-topics.sh --create \
  --topic test-events \
  --partitions 3 \
  --replication-factor 1 \
  --bootstrap-server localhost:9092

Kafka confirms the topic was created:

Created topic test-events.

List all topics to verify:

/opt/kafka/bin/kafka-topics.sh --list --bootstrap-server localhost:9092

The output shows your newly created topic:

test-events

Get detailed information about the topic including partition assignments:

/opt/kafka/bin/kafka-topics.sh --describe \
  --topic test-events \
  --bootstrap-server localhost:9092

The describe output shows topic configuration, partition count, and replica placement:

Topic: test-events	TopicId: AbC123dEfGhIjKlMnO	PartitionCount: 3	ReplicationFactor: 1	Configs: segment.bytes=1073741824
	Topic: test-events	Partition: 0	Leader: 1	Replicas: 1	Isr: 1	Elr: 	LastKnownElr:
	Topic: test-events	Partition: 1	Leader: 1	Replicas: 1	Isr: 1	Elr: 	LastKnownElr:
	Topic: test-events	Partition: 2	Leader: 1	Replicas: 1	Isr: 1	Elr: 	LastKnownElr:

Produce Test Messages

Start a console producer to send messages to the test-events topic. Type each message and press Enter. Press Ctrl+C when done:

/opt/kafka/bin/kafka-console-producer.sh \
  --topic test-events \
  --bootstrap-server localhost:9092

Type a few test messages at the prompt:

>Hello from Kafka on Ubuntu
>This is a test event
>Kafka 4.2 with KRaft mode works

Consume Test Messages

In a separate terminal, start a console consumer to read messages from the beginning of the topic:

/opt/kafka/bin/kafka-console-consumer.sh \
  --topic test-events \
  --from-beginning \
  --bootstrap-server localhost:9092

The consumer outputs all messages that were produced to the topic:

Hello from Kafka on Ubuntu
This is a test event
Kafka 4.2 with KRaft mode works

Press Ctrl+C to stop the consumer. If you see all three messages, your Kafka installation is working correctly.

Step 8: Multi-Broker Kafka Cluster Basics

A single Kafka broker works for development but production deployments need multiple brokers for fault tolerance and higher throughput. Here are the key configuration changes for a 3-node KRaft cluster.

Each broker in the cluster needs a unique node.id and the controller.quorum.voters must list all controller nodes. For a 3-node cluster where each node runs both broker and controller roles, configure each node’s server.properties as follows:

Node 1 (192.168.1.10)

node.id=1
process.roles=broker,controller
[email protected]:9093,[email protected]:9093,[email protected]:9093
listeners=PLAINTEXT://:9092,CONTROLLER://:9093
advertised.listeners=PLAINTEXT://192.168.1.10:9092

Node 2 (192.168.1.11)

node.id=2
process.roles=broker,controller
[email protected]:9093,[email protected]:9093,[email protected]:9093
listeners=PLAINTEXT://:9092,CONTROLLER://:9093
advertised.listeners=PLAINTEXT://192.168.1.11:9092

Node 3 (192.168.1.12)

node.id=3
process.roles=broker,controller
[email protected]:9093,[email protected]:9093,[email protected]:9093
listeners=PLAINTEXT://:9092,CONTROLLER://:9093
advertised.listeners=PLAINTEXT://192.168.1.12:9092

Important multi-broker considerations:

  • All nodes must use the same cluster ID when formatting storage
  • Set offsets.topic.replication.factor=3 and transaction.state.log.replication.factor=3 for proper redundancy
  • Use an odd number of controller nodes (3 or 5) for proper quorum
  • Each node formats its storage with the same cluster ID but without the --standalone flag
  • Port 9093 must be open between all controller nodes for Raft consensus

For a related message queue system, see our guide on installing RabbitMQ Server on Ubuntu.

Useful Kafka Management Commands

Here are common commands for managing your Kafka installation:

TaskCommand
List all topics/opt/kafka/bin/kafka-topics.sh --list --bootstrap-server localhost:9092
Delete a topic/opt/kafka/bin/kafka-topics.sh --delete --topic TOPIC_NAME --bootstrap-server localhost:9092
Check consumer group lag/opt/kafka/bin/kafka-consumer-groups.sh --describe --group GROUP_NAME --bootstrap-server localhost:9092
View broker metadata/opt/kafka/bin/kafka-metadata.sh --snapshot /var/kafka-logs/__cluster_metadata-0/00000000000000000000.log --cluster-id CLUSTER_ID
Increase topic partitions/opt/kafka/bin/kafka-topics.sh --alter --topic TOPIC_NAME --partitions 6 --bootstrap-server localhost:9092

Monitor Kafka’s resource usage and integrate it with a monitoring stack. Our guide on installing Prometheus on Ubuntu covers setting up the monitoring infrastructure that pairs well with Kafka’s JMX metrics exporter.

Kafka on Ubuntu – Key Configuration Reference

PropertyDescription
node.idUnique integer ID for each broker/controller node in the cluster
process.rolesNode role – broker, controller, or broker,controller for combined mode
controller.quorum.votersComma-separated list of controller nodes in format id@host:port
listenersNetwork interfaces and ports Kafka binds to
advertised.listenersAddress clients use to connect – set to external/routable IP
log.dirsDirectory for Kafka data and metadata storage
num.partitionsDefault partition count for auto-created topics
log.retention.hoursHow long to keep messages before deletion (default 168 = 7 days)

If you are running Kafka alongside other services in a containerized setup, check our guide on Apache Kafka with CMAK for cluster management UI options.

Conclusion

You now have Apache Kafka 4.2 running on Ubuntu in KRaft mode with a systemd service, tested topic creation, and verified producer/consumer message flow. The KRaft consensus protocol eliminates the ZooKeeper dependency that older Kafka versions required, reducing operational overhead.

For production deployments, enable TLS encryption on broker listeners, configure SASL authentication, set up a multi-broker cluster with replication factor 3, and monitor broker health with Prometheus and JMX metrics. Regular backups of the /var/kafka-logs directory protect against data loss.

Kibet John

The power of technology can be blatantly perceived by everyone in the world today and its sway did not spare me. Kibet is a tech enthusiast, ComputingforGeeks writer, and an ardent lover of knowledge and new skills that make the world brighter. His interests lie in Storage systems, High Availability, Routing and Switching, Automation, Monitoring, and Arts.

Related Articles

Ubuntu How To Install Mastodon on Ubuntu with Let’s Encrypt SSL Debian Install SVN Server on Ubuntu 20.04|18.04|16.04 & Debian 10|9 Ubuntu How To Install Monica CRM on Ubuntu 22.04|20.04|18.04 Ubuntu Install Chatwoot on Ubuntu 24.04 with Let’s Encrypt SSL

Press ESC to close