This is a summary of the Kafka white paper, which is available at this link.
1. Abstract
- Kafka, a distributed messaging system that is developed for collecting and delivering high volumes of log data with low latency.
- Kafka has superior performance when compared to two popular messaging systems.
2. Introduction
A large amount of log data is generated at any sizable internet company which includes user activity, operational metrics.
- User activity
- It contains events corresponding to logins, page views, clicks, comments, and search queries.
- Operational metrics
- It contains service call stack, call latency, errors, and system metrics
- system metrics include CPU, memory, network, or disk utilization on each machine.
- User activity
Activity data
- search relevance
- recommendations
- ad targeting and reporting
- security
- Kafka provides an API similar to a messaging system and allows applications to consume log events in real-time.
