System Design · Stackadapt · Medium
Web Crawler Scheduler Medium · Topics · Company Tags · Hints You are building a web crawler that must respect two rules: Rate limiting: No domain may be crawled more than once per second. Priority: Each crawl task has an integer priority. Higher numbers indicate higher priority. The crawler should always pick the highest‑priority task that is eligible to be crawled at the current time. The crawler starts at time 0 seconds. In each second, it may crawl any number of domains,…
Checking your access…