Code Review: Thread Safety of a Python Compute-and-Cache Function

You are asked to review a simplified Python memoization helper:

_memo = {}
def get(key):
    if key in _memo:
        return _memo[key]
    value = expensive_call(key)
    _memo[key] = value
    return value

expensive_call represents a slow deterministic operation. Assume get will be invoked by several threads at the same time inside one process. The discussion proceeds through four stages.

Before commenting, clarify:

• Does expensive_call have side effects, so that calling it twice with the same argument would be incorrect rather than merely wasteful?
• Are all callers threads inside one process, or can calls also come from other processes or machines?
• If expensive_call raises, should that failure be stored in the cache, or should the next caller retry?
• Is the cache allowed to grow without bound?
• Is the runtime standard CPython with the GIL, or a free-threaded build?

Example 1:

Input: _memo = {}, call get("alpha")
Output: expensive_call("alpha") is invoked once; its result is stored and returned

Explanation: On a cold cache, the helper computes, caches, and returns the value.

Example 2:

Input: _memo = {"alpha": 42}, call get("alpha")
Output: 42

Explanation: A cached key returns the stored value and skips expensive_call.

Part 1 — First-pass review comments

What feedback would you leave on this code assuming it is called concurrently? Describe the exact interleaving that causes a problem, and explain its impact on both cost and correctness. Mention any other review comments that are not directly about concurrency.

Part 2 — Make one computation per key

Add locking so that concurrent callers for the same key cause expensive_call to run only once. Walk through why your code is correct when several threads miss at almost the same instant.

Part 3 — Place the lock precisely

Which lines should the lock protect? What is the downside of acquiring it too broadly, and what is the risk of acquiring it too narrowly? Suggest a refinement that prevents unrelated keys from blocking one another.

Part 4 — Concurrent dictionary access in Python

How does a Python dict behave when multiple threads read and write individual entries at the same time? Can a reader observe an old value, a new value, or a partially updated value? State what a single dictionary lookup or assignment guarantees, and explain why a check-then-act sequence remains unsafe.

Follow-up questions

1. If expensive_call raises for a key while other threads are waiting for that key, what should those waiters observe, and when should the key be retried?
2. How would you write a reproducible test that shows duplicated work before the fix and no duplicated work after it?
3. What changes in your design if the cache is relocated to a shared store accessed by multiple processes or machines?
4. What happens if a non-reentrant lock is used and expensive_call recursively invokes get for a smaller key?

Constraints:

• Several threads may invoke get concurrently in the same process.
• expensive_call may be slow and may have side effects or raise exceptions.
• The cache has no built-in eviction policy.