Tag: message-queues
All the articles with the tag "message-queues".
Kafka (Part 5): Consumer Callbacks, Scheduled Retries, and Rebalancing
This largely completes a messaging system built with Kafka. The key code has been explained across the articles; see the source repository for the complete design and implementation. The next Kafka article is planned to cover synchronizing Oracle data to PostgreSQL with Kafka Connect.
Kafka (Part 4): Consuming JSON, Sharing a Deserializer, and Improving Throughput
This article implements Kafka consumers for JSON messages, including shared deserialization, deduplication, retries, and offset commits. Load tests inform partition counts and consumer settings, and the template method pattern reuses the consumption flow for emails of different priorities.
Kafka (Part 2): Designing a Messaging System to Decouple Email Delivery
Uses email delivery to explain a design that decouples a business system from an email system through Kafka. It discusses responsibility boundaries, a unified message format, sending and callback flows, and a fallback when switching between old and new systems, providing background for subsequent producer and consumer implementations.
Kafka (Part 1): A Single-Node KRaft Setup with Docker Compose, Kafka UI, and Prometheus JMX Exporter
Builds a single-node Kafka environment in KRaft mode with Docker Compose and integrates Kafka UI and Prometheus JMX Exporter. It explains listeners, roles, memory, and monitoring configuration, then covers startup verification, JMX port conflicts, and resource-related troubleshooting.