Setup First Kafka Message Stream From Command Line

June 30, 2018

Hello!

This article explains how to set-up apache Kafka and zookeeper and start your first streaming of messages from command-line.

Download Apache Kafka binaries from Mirror site. Official Documentation and downloads available at https://kafka.apache.org/downloads

I have downloaded Kafka and decompressed the file in Documents folder. Now open the command prompt / terminal and let’s dive deep.

Start Zookeeper server

Rajivs-Air:Documents rjrajivjha$ cd kafka_2.12-1.1.0/

Rajivs-Air:kafka_2.12-1.1.0 rjrajivjha$ ./bin/zookeeper-server-start.sh config/zookeeper.properties

Start Kafka Server

Last login: Sat Jun 30 01:52:17 on ttys003

Rajivs-Air:~ rjrajivjha$ cd Documents/kafka_2.12-1.1.0

Rajivs-Air:kafka_2.12-1.1.0 rjrajivjha$ ./bin/kafka-server-start.sh config/server.properties

Create Topic

Rajivs-Air:~ rjrajivjha$ cd Documents/kafka_2.12-1.1.0

Rajivs-Air:kafka_2.12-1.1.0 rjrajivjha$ ./bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

Created topic "test".

Run Producer

Rajivs-Air:Documents rjrajivjha$ cd kafka_2.12-1.1.0

Rajivs-Air:kafka_2.12-1.1.0 rjrajivjha$ ./bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test

>my first message

>my second message

>my third message

>wow

>Type your next message here to be produced at consumer end

Run Consumer

Last login: Sat Jun 30 02:06:16 on ttys005

Rajivs-Air:~ rjrajivjha$ cd Documents/kafka_2.12-1.1.0

Rajivs-Air:kafka_2.12-1.1.0 rjrajivjha$ ./bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning

my first message

my second message

my third message

wow

Keep tuned for more blogs from my “ Kafka-Zookeeper and Pyspark “ series.

Happy Learning!

Rajiv Jha :)

Rajiv Jha

Rajiv Jha

My name is Rajiv Jha. I am Senior Engineering student at Guru Gobind Singh Indraprastha University.