Compute precision/recall from a flaky top-k API

Key Idea Ground truth gives us a set of file IDs that are actually dogs. Let: $$D$$ = the set of true dog file IDs. $$A$$ = the set of file IDs returned by searchDogs(k), after any necessary cleaning. Precision and recall are then: $$ P = \frac{ D \cap A }{ A } $$ $$ R = \frac{ D \cap A }{ D } $$ Precision answers: of the files the API returned, how many are actually dogs? Recall answers: of all the real dog files, how many did the API manage to return? If the API returns…