Design A Kafka-like Distributed Message Queue

Picture a replicated, partitioned, append‑only log that lives between producers and consumers, designed for durable messaging. Producers write messages to a topic, and the cluster persists them on disk across multiple brokers. Consumers can read at their own pace using offsets, often organized into consumer groups that split the partition ownership. Because the log retains every record for a configurable window instead of deleting on acknowledgment, separate consumer…