Design a Kafka-Based File Watcher for Millions of Files

You are tasked with architecting a service that monitors a filesystem containing millions of files and delivers change events to downstream consumers via Apache Kafka. The system must support initial file discovery, ongoing real-time change streaming, event ordering, failure recovery, and detection of events that the native filesystem notification API may have dropped. Consider the following context and requirements: The files may be stored on local disks, distributed across…