This Kafka Introduction course prepares Developers and Architects in applying Kafka to their integrations and micro-services.
Introduction
- Introduction to Event Streaming
- Discuss use cases for Kafka
- Introduce Apache Kafka
- Introduce Event Sourcing
- Understand the concept of journaling
- Understand Kafka’s Architecture
- Push vs Pull architectures
- Introduce main concepts ( event , producers , partitions , topics , brokers , logs )
- Compare Kafka to MQ/JMS (traditional Message Oriented Middleware)
- Discuss alternatives
- Overview of the different APIs in Kafka
Introduction to Event Streaming
Discuss use cases for Kafka
Introduce Apache Kafka
Introduce Event Sourcing
Understand the concept of journaling
Understand Kafka’s Architecture
Push vs Pull architectures
Introduce main concepts ( event , producers , partitions , topics , brokers , logs )
Compare Kafka to MQ/JMS (traditional Message Oriented Middleware)
Discuss alternatives
Overview of the different APIs in Kafka
Installing & Running Kafka Cluster
- Planning hardware for kafka
- Explore the installation directory
- Understand the current role of Apache Zookeeper
- Configure kafka brokers
- Define producer/consumer quotas
- Discuss best practices in cluster size
- Partitions and brokers
- Building a kafka cluster
Producing Messages
- Producing messages using the Producer API
- Configure producers (latency, throughput, timeouts)
- Appreciate the batching of messages
- Configure acknowledgments
- Use message compression
- Build resilient producers
- Idempotent messages and exactly-once delivery semantics
- Using ProducerInterceptor
Messages (Partitioning, Serialization, …)
- Recap message structure
- Using message headers
- List available Serializers/Deserializers (e.g., Protobuf )
- Defining extensible schemas using Apache Avro
- Planning schemas for change
- Using Schema Registries with Avro
- Building Custom Serializers / Deserializers
- Using different Partitioning strategies (e.g., key-based, round-robin, …)
- Building a custom Partitioner
- Introduction to Kafka Connect
Consuming Messages
- Producing messages using the Consumer API
- Determine the off-set to control the Consumer Position
- Partition-based scaling using consumer groups
- Configuring PartitionAssignor strategies
- Introduce Static membership
- Understand semantic guarantees (At most/least/exactly once)
- Managing the offset (e.g., committing )
- Partition ownership and rebalancing
- Using ConsumerRebalanceListener
Basic Stream Processing
- Introduce Kafka Streams
- Introduce the Streams API
- Stream architecture ( topology , source and sink processors)
- Understand key concepts streams and tables
- Time and windowing
- Discuss different **use-cases and patterns**
