Fixing Slow File Deduplication in a Shared Upload Service

System Design · Karat · Medium

Consider a file storage service that is open to the public. To avoid keeping multiple copies of identical content, it handles each incoming upload as follows: it looks up every already-stored file whose size matches the new file, compares the new file against each of them byte by byte, and, when an exact match is found, links the upload to the existing copy instead of storing it again. Your task has three parts: Determine what is flawed about this deduplication design.…

Loading the rest of the question…