In the third part of Apache Kafka, we continue with more commands. We will use the same environment that we set up in this post
Let us restart Zookeeper and Kafka we have done in previous post with below commands:
Start Zookeeper:
Open first terminal:
F:\>
F:\>cd kafka_2.12-1.0.0
F:\kafka_2.12-1.0.0>bin\windows\zookeeper-server-start.bat config\zookeeper.properties
Start Kafka:
Next, open another terminal and start Kafka as shown below:
F:\>
F:\>cd kafka_2.12-1.0.0
F:\kafka_2.12-1.0.0>bin\windows\kafka-server-start.bat config\server.properties
Create Topic:
In a third terminal, let us now proceed to create a topic called topic1 having a replication factor as 1 and number of partitions as 3:
F:\>
F:\>cd kafka_2.12-1.0.0
F:\kafka_2.12-1.0.0>bin\windows\kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 3 --topic topic1
Start Producer:
In a fourth terminal, let us start a producer with below command as shown:
F:\>
F:\>cd kafka_2.12-1.0.0
F:\kafka_2.12-1.0.0>bin\windows\kafka-console-producer.bat --broker-list localhost:9092 --topic topic1
In a fifth terminal, increase the retention time (in milliseconds) of messages as shown below:
F:\>
F:\>cd kafka_2.12-1.0.0
F:\kafka_2.12-1.0.0>bin\windows\kafka-configs --zookeeper localhost:2181 --entity-type topics --entity-name topic1 --alter --add-config retention.ms=1800000
In the fourth terminal, add the following messages:
>Message 1
>Message 2
>Message 3
>Message 4
>Message 5
>Message 6
>Message 7
>Message 8
>Message 9
>Message 10
>Message 11
>Messgae 12
>Message 13
Start Consumer:
On a sixth terminal, start a console consumer as follows:
bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic topic1 --from-beginning
Output is shown below:
F:\kafka_2.12-1.0.0>bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic topic1 --from-beginning
Message 1
Message 4
Message 7
Message 10
Message 13
Message 2
Message 5
Message 8
Message 11
Message 3
Message 6
Message 9
Messgae 12
Note that the order of messages is not the same. You can see the messages through the command we ran in the post:
F:\kafka_2.12-1.0.0>bin\windows\kafka-console-consumer.bat --zookeeper localhost:2181 --topic topic1 --from-beginning
Using the ConsoleConsumer with old consumer is deprecated and will be removed in a future major release. Consider using the new consumer by passing [bootstrap-server] instead of [zookeeper].
Message 2
Message 5
Message 8
Message 11
Message 3
Message 6
Message 9
Messgae 12
Message 1
Message 4
Message 7
Message 10
Message 13
To see the messages in a partition, run below command:
bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic topic1 --partition 0 --from-beginning
Output is shown below:
F:\kafka_2.12-1.0.0>bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic topic1 --partition 0 --from-beginning
Message 2
Message 5
Message 8
Message 11
Note that the messages within a partition are ordered
This concludes the third post on Apache Kafka
Let us restart Zookeeper and Kafka we have done in previous post with below commands:
Start Zookeeper:
Open first terminal:
F:\>
F:\>cd kafka_2.12-1.0.0
F:\kafka_2.12-1.0.0>bin\windows\zookeeper-server-start.bat config\zookeeper.properties
Start Kafka:
Next, open another terminal and start Kafka as shown below:
F:\>
F:\>cd kafka_2.12-1.0.0
F:\kafka_2.12-1.0.0>bin\windows\kafka-server-start.bat config\server.properties
Create Topic:
In a third terminal, let us now proceed to create a topic called topic1 having a replication factor as 1 and number of partitions as 3:
F:\>
F:\>cd kafka_2.12-1.0.0
F:\kafka_2.12-1.0.0>bin\windows\kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 3 --topic topic1
Start Producer:
In a fourth terminal, let us start a producer with below command as shown:
F:\>
F:\>cd kafka_2.12-1.0.0
F:\kafka_2.12-1.0.0>bin\windows\kafka-console-producer.bat --broker-list localhost:9092 --topic topic1
In a fifth terminal, increase the retention time (in milliseconds) of messages as shown below:
F:\>
F:\>cd kafka_2.12-1.0.0
F:\kafka_2.12-1.0.0>bin\windows\kafka-configs --zookeeper localhost:2181 --entity-type topics --entity-name topic1 --alter --add-config retention.ms=1800000
In the fourth terminal, add the following messages:
>Message 1
>Message 2
>Message 3
>Message 4
>Message 5
>Message 6
>Message 7
>Message 8
>Message 9
>Message 10
>Message 11
>Messgae 12
>Message 13
Start Consumer:
On a sixth terminal, start a console consumer as follows:
bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic topic1 --from-beginning
Output is shown below:
F:\kafka_2.12-1.0.0>bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic topic1 --from-beginning
Message 1
Message 4
Message 7
Message 10
Message 13
Message 2
Message 5
Message 8
Message 11
Message 3
Message 6
Message 9
Messgae 12
Note that the order of messages is not the same. You can see the messages through the command we ran in the post:
F:\kafka_2.12-1.0.0>bin\windows\kafka-console-consumer.bat --zookeeper localhost:2181 --topic topic1 --from-beginning
Using the ConsoleConsumer with old consumer is deprecated and will be removed in a future major release. Consider using the new consumer by passing [bootstrap-server] instead of [zookeeper].
Message 2
Message 5
Message 8
Message 11
Message 3
Message 6
Message 9
Messgae 12
Message 1
Message 4
Message 7
Message 10
Message 13
To see the messages in a partition, run below command:
bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic topic1 --partition 0 --from-beginning
Output is shown below:
F:\kafka_2.12-1.0.0>bin\windows\kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic topic1 --partition 0 --from-beginning
Message 2
Message 5
Message 8
Message 11
Note that the messages within a partition are ordered
This concludes the third post on Apache Kafka
