Object-Oriented Programming · Snowflake · Medium
Build a rate limiter that runs inside one process and keeps all of its state in memory, serving many tenants at once. Every caller is named by a unique client_id, and may be attached to a named tier such as free, pro, or enterprise. Throttling follows the Token Bucket scheme. A tier is described by two numbers: capacity: the largest number of tokens a bucket can hold. refill_rate: how many tokens are added to a bucket every second. Each client owns a separate bucket. At a…