Design a Highly Available Conversational AI Service

You are asked to design a conversational AI product that streams generated text back to a user. First produce a deliberately small, stateless version that has no stored conversation history. Then extend that base design so it can handle much larger traffic, remain available during partial failures, tolerate faults, and expose useful operational signals. Be explicit about which later capabilities cannot be implemented without persistence; do not add durable state to the…