Installation and setup of Kafka

Installation of Kafka:

Pre-requisite:
• Install Java if not present

sudo apt update
sudo apt install default-jdk

• Check Java version to ensure java is installed on the machine

java --version

Steps:
1. Download latest Apache Kafka

wget https://downloads.apache.org/kafka/3.4.0/kafka_2.12-3.4.0.tgz

2. Extract the archive file

tar xzf kafka_2.12-3.4.0.tgz 

sudo mv kafka_2.12-3.4.0 /usr/local/kafka

3. Create System Unit Files for Zookeeper and Kafka

1. System Unit File for Zookeeper
   1. Create the file

      nano /etc/systemd/system/zookeeper.service

   2. Add the following content

      [Unit]
      Description=Apache Zookeeper server
      Documentation=http://zookeeper.apache.org
      Requires=network.target remote-fs.target
      After=network.target remote-fs.target
      
      [Service]
      Type=simple
      ExecStart=/usr/local/kafka/bin/zookeeper-server-start.sh /usr/local/kafka/config/zookeeper.properties
      ExecStop=/usr/local/kafka/bin/zookeeper-server-stop.sh
      Restart=on-abnormal
      
      [Install]
      WantedBy=multi-user.target

   3. Save the file and close it
2. System Unit File for Kafka
   1. Create the file

      nano /etc/systemd/system/kafka.service

   2. Add the following content

      [Unit]
      Description=Apache Kafka Server
      Documentation=http://kafka.apache.org/documentation.html
      Requires=zookeeper.service
      
      [Service]
      Type=simple
      Environment="JAVA_HOME=/usr/lib/jvm/java-1.11.0-openjdk-amd64"
      ExecStart=/usr/local/kafka/bin/kafka-server-start.sh /usr/local/kafka/config/server.properties
      ExecStop=/usr/local/kafka/bin/kafka-server-stop.sh
      
      [Install]
      WantedBy=multi-user.target

   3. Save the file and close it

4. Reload the systemd daemon to apply new changes

systemctl daemon-reload

5. Start Zookeeper service

sudo systemctl start zookeeper

6. Start Kafka service

sudo systemctl start kafka

7. Check the status of Zookeeper and Kafka

sudo systemctl status zookeeper
sudo systemctl status kafka

Setup of Kafka:

Steps:
1. Create Topic in Kafka

cd /usr/local/kafka
bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic test

Note: This Topic is named “test” with a single partition with a single replica
2. List the Topics in Kafka

bin/kafka-topics.sh--list--bootstrap-server localhost:9092

In case if you have any queries please get us at support@helicaltech.com

Thank You
Sai Kavya
Helical IT Solutions

Best Open Source Business Intelligence Software Helical Insight is Here

A Business Intelligence Framework

Free
Download