You can pipe it in: kafka-console-producer.sh –broker-list localhost:9092 –topic my_topic –new-producer < my_file.txt Found here. From 0.9.0: kafka-console-producer.sh –broker-list localhost:9092 –topic my_topic < my_file.txt
Does it mean if i want to have more than 1 consumer (from the same group) reading from one topic I need to have more than 1 partition? Let’s see the following properties of kafka: each partition is consumed by exactly one consumer in the group one consumer in the group can consume more than … Read more
All Kafka brokers must be assigned a broker.id. On startup a broker will create an ephemeral node in Zookeeper with a path of /broker/ids/$id. As the node is ephemeral it will be removed as soon as the broker disconnects, e.g. by shutting down. You can view the list of the ephemeral broker nodes like so: … Read more
Keys are mostly useful/necessary if you require strong order for a key and are developing something like a state machine. If you require that messages with the same key (for instance, a unique id) are always seen in the correct order, attaching a key to messages will ensure messages with the same key always go … Read more
This post already has answers, but I am adding my view with a few pictures from Kafka Definitive Guide Before answering the questions, let’s look at an overview of producer components: 1. When a producer is producing a message – It will specify the topic it wants to send the message to, is that right? … Read more