Back to problems

Design a WAL Log Enrichment Pipeline

System Design · Netflix · Hard

Build a WAL-Based Data Enrichment Pipeline System DesignSoftware Engineer Create a system that reads Write-Ahead Log (WAL) records from a source database, adds relevant context to those records, and writes the enriched results into a destination database. It must sustain 1 million writes each second with consistent data and very low delay. At this volume, assume the source uses shards or several primary databases, each producing an independent WAL feed. This discussion is…

Checking your access…