How detect duplicate card records?

You receive a credit card transaction dataset and believe some entries may be repeated. Discuss:

• Which real-world scenarios could create records that look like duplicates.
• How you would decide what counts as a true duplicate, especially when two records are very similar but not exactly identical.
• How you would identify, measure, and extract duplicate records from the dataset in a reliable way. Your answer should cover both exact duplicates and near-duplicates arising from ingestion problems, retries, formatting differences, or multiple processing stages. Overview: This question evaluates a candidate's ability in duplicate detection, record linkage, entity resolution, and data quality assessment for credit card transaction datasets.