Back to problems

Job Scheduler / ETL Pipeline System Design

System Design · Microsoft · Hard

Requirements Functional Allow users to create jobs with cron-style schedules and prerequisites on other jobs. Start each due execution once for every schedule occurrence, or permit at-least-once delivery when consumers are idempotent. Maintain execution states: pending, running, succeeded, failed, and retrying. Expose retry configuration at the individual-job level. Offer an administrative UI or API for viewing jobs and launching runs manually. Non-functional Support 10,000…

Checking your access…