The Problem of Duplicate Patient Records

Resolving duplicate records by Patient Matching

Why is it important?

• These difficulties hinder health information interchange and care coordination, leading to medical error deaths.
• Problems sharing health information and coordinating care can lead to medical errors and patient deaths.
• Despite best practices for patient access and medical record management, duplicate records still occur.

Why is patient matching difficult?

The following can compound these challenges:

1. Non-integrated systems: If different parts of the health system are built on different environments, there may not be a single identifier that works in all of them. So, any matching between systems will need keys that aren't tied to the platform itself.
2. Lack of universal identifiers: When a country or health system has a unique universal identifier, like a National ID or a National Health ID, it gives all systems a key. Many countries don't have a single form of ID, so they need extra proof of identity.
3. Inadequate data capture skills and incentives: For a health site administrator to keep a reliable patient record system, they must capture important information every time a patient enters the system. Health care administrators should:
1. Check if person is in system
2. Update records if they've changed
3. Duplicate records must be manually fixed.
4. Privacy concerns: Health records are high-risk information where privacy rules exist. The health system must:
1. restrict access to key information
2. Limit unnecessary security-risking system integrations
3. Record only necessary information
4. Reduce record-keeping time.

However, there may now not be enough relevant data for systems to use probabilistic matches.

5. Changing data fields: Identification information can change. Names, contact info, geolocations, and jobs change. The longer it's been since records were gathered and compared, the more probable the underlying information has changed, reducing their relevance.

Generic matching process diagram

Steps:

1. Database A: This database includes patient demographic data from different source systems.
2. Pre-processing: Data is pre-processed to improve the data quality and make sure that records captured in different systems can be compared to each other. This includes standardising date formats and coded value sets, transforming fields, removing special characters, etc.
3. Blocking: Blocking is the process of identifying pairs that are plausible matches and discarding the others. This is necessary to reduce the total number of possible matches due to processing power limitations. Matching algorithms are used to do the comparisons. It is a constant process of checking and adjusting the blocking criteria and algorithms used to minimise false negative rates whilst trying to maximise efficiency. It is important to block carefully, only using identifiers that are likely to be quite stable across all sources.
4. Comparisons: After blocking, all the remaining records are compared against each other. This process can be very resource intensive. Algorithms are used to compare each record identifier and the outcome of each comparison is aggregated to generate a record pair matching score. The way the matching score is calculated will depend on the patient matching approach used.
5. Classification: Based on a pre-defined threshold that is obtained in the Evaluation step, the record pairs are classified as matches (very high probability that they are identical), non matches (very unlikely to not be the same individual) , and potential matches (probability falls in a band of uncertainty).
6. Clerical review: For record pairs that are potential matches, they may be reviewed manually and become a match or non-match or remain as potential matches.
7. Evaluation: The degree to which the classification algorithm correctly identifies matches are measured and optimised for the expected data. In particular, the system must minimise the ratios of the potential errors that may occur. The critical measures considered are recall (measure of the capability of the matching algorithm to correctly classify two records that refer to the same patient as true matches) and precision (measure of the fraction of pairs that the matching algorithm classified accurately as matches). These accuracy measurements are described in detail below.

Patient matching approaches

1. Deterministic matching: Rules-based or deterministic matching uses a one-to-one comparison approach for record linking. Deterministic matching compares exact character-by-character values in fields to determine whether two records should be linked. This strict algorithm does not account for spelling variations or differences in using abbreviations, such as St. versus Street. It should be noted that many consumer-facing applications use deterministic matching over probabilistic, as it is necessary to only return one exact consumer record to that specific record. On the other hand, provider-facing applications can return numerous records, for the provider to then choose the appropriate patient record. It should be noted that even the first academic paper on record linkage recognized that strict deterministic matching is inadequate.
2. Probabilistic matching: also referred to as fuzzy matching, uses statistical analysis to determine the overall likelihood or probability that two records match. It is generally accepted that probabilistic matching is a more sophisticated approach to record linking than deterministic matching. Probabilistic matching can take phonetic sounds and nicknames into account as well as “edit distances” and “string similarities” when there is variation in fields. For example, probabilistic matching will recognize that records with a first name of Tim and Timothy may belong to the same individual and rank higher probability for similar sounding names like White and Wight. Finally, the probabilistic algorithm will often give a higher probability to 123 Main Street vs. 163 Main Street, recognizing the edit distance between the two as one-character difference.
3. Machine learning: There are two broad approaches to patient matching with machine learning. The first is known as “supervised” machine learning. Supervision refers to the role of the human operator: in supervised machine learning, the operator classifies a modest number of pairs as matches or not, using his or her judgement. This is known as “training” the algorithm.