Back to problems

Large-Scale Media Transfer to ML Compute

System Design · Apple · Hard

Large-Scale Dataset Transfer to Cloud GPU Cluster Hard · System Design · Cloud Architecture · Networking · Data Pipeline · Amazon · Google · Facebook · Microsoft · Hints An organization needs to move a 200 TB dataset from its on‑premises data centre to a cloud‑hosted GPU cluster so that machine‑learning models can be trained on it. The corpus contains a mix of high‑resolution video recordings (several GB each) and millions of small image snapshots (a few KB each). The whole…

Checking your access…