Kafka Consumer Groups
Kafka consumer groups provide a way to parallelize message processing for a topic. Multiple consumers can join a consumer group and each consumer in the group will only consume a subset of the messages from the subscribed topic. This enables load balancing and fault tolerance as multiple consumers can work together to process messages.
Creating a Consumer Group
To create a consumer group in Kafka, you first need to set the group.id property in the consumer's configuration. This property specifies the name of the consumer group that the consumer belongs to.
TEXT/X-JAVA
1Properties properties = new Properties();
2properties.put("group.id", "my-consumer-group");xxxxxxxxxx28
class Main { public static void main(String[] args) { // replace with your Java logic here // Kafka Consumer Group String consumerGroup = "my-consumer-group"; // Create properties for the Kafka consumer Properties properties = new Properties(); properties.put("bootstrap.servers", "localhost:9092"); properties.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer"); properties.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer"); properties.put("group.id", consumerGroup); // Create a KafkaConsumer instance KafkaConsumer<String, String> consumer = new KafkaConsumer<>(properties); // Subscribe to the topic consumer.subscribe(Arrays.asList("my-topic")); // Start consuming messages while (true) { ConsumerRecords<String, String> records = consumer.poll(100); for (ConsumerRecord<String, String> record : records) { System.out.println("Received message: " + record.value()); } } }}OUTPUT
:001 > Cmd/Ctrl-Enter to run, Cmd/Ctrl-/ to comment
