Back to problems

Resumable Batch Dataloader with Weighted Sampling

Algorithm · Microsoft · Medium

Requirements Build a BatchDataLoader that combines multiple iterable datasets (source_a, source_b, …) and emits batches containing exactly B items. Part 1 — Integer-weight batching Assign each source an integer weight w_i. Each batch must draw a w_i / sum(w) share of its items from source i. When the weights divide the batch size evenly—for example, B=8, w=[1,3], which requires 2 items from A and 6 from B—select the items from each source deterministically. Part 2 —…

Checking your access…