Back to problems

Design idempotent daily loads with deduping

SQL · Amazon · Medium

Your nightly job copies seven days' worth of orders from an untidy staging feed into a large fact table. Treat the run date as 2025-07-15, so the batch window spans 2025-07-09 through 2025-07-15 inclusive. The feed is messy: the same order_id can appear several times, a correction may land days after the original record, and the same source file may be delivered twice. Re-executing the job must leave the fact table exactly as one clean execution would, and a record that is…

Checking your access…