A patient's EHR contains two fundamentally different categories of clinical information, and the distinction matters enormously for how eligibility matching works. Structured data lives in defined fields: diagnosis codes in the problem list, medication names in the MAR, lab results with values and timestamps, vital signs organized into tables. Unstructured data lives in free text: progress notes, consultation summaries, radiology reads, pathology reports, operative notes, discharge summaries.
The promise of EHR-based patient matching, the idea that eligibility can be evaluated automatically against the record, depends entirely on where the eligibility criteria live relative to that structured/unstructured boundary. And in our experience, that boundary falls in an uncomfortable place for most protocol criteria: the majority of what protocols actually ask lives in free text.
What structured data covers well
Structured data in an EHR is genuinely useful for a subset of eligibility criteria. Age range is in the demographics table. Primary diagnosis codes are in the problem list, though coding practices vary enough to require careful vocabulary mapping. Current medications are in the medication list. Recent lab values with timestamps are in the lab results table. Active enrollment in another trial may or may not be in the system depending on site practices, but when it is, it is findable.
For a typical Phase 2 protocol with 30 eligibility criteria, structured data can confidently address 8 to 12 of them. That is meaningful. A structured-data first pass that rules out candidates who fail on age, diagnosis, or clearly documented contraindicated medications is worth doing before touching notes. It reduces the note-reading workload for the remaining candidates.
The problem is the assumption that builds on this: that structured data can be extended to cover more criteria through better EHR configuration, better coding, or better query design. For certain types of clinical information, that is true. For most of what protocol eligibility criteria actually ask, it is not.
What lives irreducibly in free text
Protocol eligibility criteria are written by clinical scientists to capture clinically relevant patient characteristics. Clinical science describes patients in language, not in database fields. The result is a persistent gap between what EHR structured fields record and what eligibility criteria require.
Prior therapy lines and dates: the medication list records current prescriptions and some historical ones, but prior therapy for a given indication, the specific sequence, dates of initiation and discontinuation, and reason for stopping, is most reliably documented in oncology progress notes, treatment summaries, and consultation letters. The MAR shows what was administered. The notes explain why and when in the clinical context that eligibility criteria care about.
Disease stage and assessment: staging information may appear as a discrete field in some cancer registries integrated with the EHR, but current disease status, response assessments, and staging at specific time points relative to prior therapies live in radiology reports, pathology reports, and oncology progress notes. A query for "metastatic disease" returns diagnosis codes; a requirement for "documented disease progression on most recent line of therapy" requires reading the notes that describe what the imaging and pathology showed.
Toxicity and adverse event history: structured EHR fields for adverse events are frequently incomplete. Clinicians document adverse events in notes using CTCAE grading terminology when they remember to, and using plain language when they do not. A criterion that excludes patients with prior Grade 3 or higher hepatotoxicity requires reading notes for liver function narratives, not querying a structured toxicity field.
Surgical and procedure history: the EHR procedure list records scheduled and completed procedures, but the clinical detail, what was found, what was removed, what the pathology showed, lives in operative notes, pathology reports, and post-procedure summaries. A criterion that excludes patients who have had a specific procedure type with specific findings requires note reading.
The 30/70 approximation
A rough working approximation: structured data reliably addresses around 25 to 35 percent of what a typical complex protocol's eligibility criteria require. The remaining 65 to 75 percent requires reading clinical notes to assess accurately. The exact ratio varies by indication and protocol design, but the direction is consistent across the protocols we have worked with.
This approximation has a practical implication. Any patient-matching approach that relies primarily on structured data is operating on a partial picture of each candidate's eligibility. The candidates it identifies as eligible based on structured fields may not be eligible once note content is reviewed. The candidates it identifies as ineligible may have disqualifying information that a structured query missed, or qualifying information in notes that a structured query did not see.
False negatives from structured-only matching, patients who qualify but are not identified, are the more costly error type in enrollment terms. A candidate who is incorrectly excluded from the pre-screen list never gets the chance to be reviewed. That patient is lost to the trial without anyone knowing they were missed.
How matching accuracy changes when notes are included
When we add free-text note reading to the matching process, the yield improvement is not uniform across criteria types. For criteria that are well-served by structured data, note reading adds little. For criteria in the categories described above, the difference in candidate identification rate is substantial.
In one set of protocols we tested internally, running a structured-data-only screen identified a candidate list that, when cross-checked against full chart review, captured around 60 to 65 percent of the genuinely eligible population within a site's records. When note reading was added, that coverage rose to 85 to 90 percent, with the remaining gap attributable to documentation completeness issues at the individual site level rather than matching methodology.
The false positive rate is also affected. Structured-data screens that pass candidates on incomplete evidence generate a higher false positive rate, meaning more candidates who appear eligible but fail when notes are reviewed. These candidates advance into coordinator review time that could have been avoided.
What this means for how we build
We do not treat structured and unstructured data as alternatives. The matching pipeline we run starts with structured data, because it is fast and deterministic on the criteria it covers, and then adds note reading for the criteria that require it. The order matters: structured-data disqualification is a faster operation than note reading, and removing candidates early on the criteria where structured data is reliable reduces the note-reading volume for the remaining candidates.
The note-reading component is where the engineering work is concentrated, not because free-text is harder to build against in principle, but because production clinical notes are inconsistent, abbreviated, abbreviation-heavy, and written under time pressure. The same clinical fact appears in different forms across different sites, different EHR systems, and different clinician writing styles. Building a note-reading pipeline that works reliably in that environment is an ongoing calibration challenge, not a solved problem.
We are not claiming we have solved it. What we do say is that any team building patient-matching tools who is not addressing the unstructured data layer is building against the minority of the problem and calling it done.