Getting Started with Apache Kafka

To get started with Apache Kafka, you first need to set it up in your development environment. Here are the steps to follow:

1. Download Apache Kafka: Visit the Apache Kafka website and download the latest stable version of Apache Kafka.

2. Extract the Files: Once the download is complete, extract the contents of the downloaded package to a directory of your choice.

3. Start the Kafka Server: Open a terminal or command prompt and navigate to the directory where you extracted the Kafka files. Start the Kafka server by running the following command:

   SNIPPET

   1bin/kafka-server-start.sh config/server.properties

4. Create a Kafka Topic: Kafka uses topics to organize data. You can create a topic by running the following command:

   SNIPPET

   1bin/kafka-topics.sh --create --topic my-topic --bootstrap-server localhost:9092 --partitions 1 --replication-factor 1

   In the above command, replace my-topic with the name of your desired topic.

5. Start a Kafka Producer: A Kafka producer is responsible for producing and sending messages to Kafka topics. You can create a simple Kafka producer using the following Java code:

   TEXT/X-JAVA

   1// Set up the producer properties
   2Properties properties = new Properties();
   3properties.put("bootstrap.servers", "localhost:9092");
   4properties.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
   5properties.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
   6
   7// Create the Kafka producer
   8Producer<String, String> producer = new KafkaProducer<>(properties);
   9
   10// Create and send a Kafka record
   11String topic = "my-topic";
   12String key = "key";
   13String value = "Hello, Kafka!";
   14producer.send(new ProducerRecord<>(topic, key, value));
   15
   16// Close the producer
   17producer.close();

   In the above code, we set up the producer properties, create a Kafka producer, and send a Kafka record with a specified topic, key, and value.

6. Start a Kafka Consumer: A Kafka consumer is responsible for consuming and processing messages from Kafka topics. You can create a simple Kafka consumer using the following Java code:

   TEXT/X-JAVA

   1// Set up the consumer properties
   2Properties properties = new Properties();
   3properties.put("bootstrap.servers", "localhost:9092");
   4properties.put("group.id", "my-group");
   5properties.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
   6properties.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
   7
   8// Create the Kafka consumer
   9Consumer<String, String> consumer = new KafkaConsumer<>(properties);
   10
   11// Subscribe to a Kafka topic
   12String topic = "my-topic";
   13consumer.subscribe(Collections.singletonList(topic));
   14
   15// Start consuming records
   16while (true) {
   17    ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(100));
   18    for (ConsumerRecord<String, String> record : records) {
   19        String key = record.key();
   20        String value = record.value();
   21        System.out.println("Received message: key = " + key + ", value = " + value);
   22    }
   23}

   In the above code, we set up the consumer properties, create a Kafka consumer, and subscribe to a Kafka topic. We then start consuming records and print each received message.

By following these steps, you can get started with Apache Kafka and create a simple producer and consumer to publish and consume messages from Kafka topics.

xxxxxxxxxx

import org.apache.kafka.clients.producer.Producer;

import org.apache.kafka.clients.producer.ProducerRecord;

import org.apache.kafka.clients.producer.KafkaProducer;

import java.util.Properties;

​

public class SimpleProducer {

    public static void main(String[] args) {

        // Set up the producer properties

        Properties properties = new Properties();

        properties.put("bootstrap.servers", "localhost:9092");

        properties.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");

        properties.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");

​

        // Create the Kafka producer

        Producer<String, String> producer = new KafkaProducer<>(properties);

​

        // Create and send a Kafka record

        String topic = "my-topic";

        String key = "key";

        String value = "Hello, Kafka!";

        producer.send(new ProducerRecord<>(topic, key, value));

​

        // Close the producer

        producer.close();

    }

}