System Design · Disney · Medium
This discussion covers two separate scenarios. In the first, a newly built inference model has to be proven out before it is allowed into production. Explain how you would roll it out, how you would design an A/B test to evaluate it, which metrics you would track, and how you would measure the lift it delivers. In the second, a different model that consumes more than 2000 features has become expensive to run, and the goal is to bring its operating cost down. Address how you…