Protocol eligibility criteria look, at first pass, like structured logic. A list of conditions, each one expressible as a Boolean check. A patient has or does not have a given diagnosis. Their creatinine is above or below a threshold. They have or have not received a particular drug class. Feed the protocol to a rule-extraction pipeline, map each criterion to a SNOMED or ICD code, run the query against the EHR. Done.
The problem is that real protocol language is not structured logic. It is natural language that happens to describe logical intent, and the gap between those two things is where most automated criteria-matching pipelines break. We have built against this gap long enough to have a detailed picture of where it falls apart and why.
Negation: the most common failure mode
Consider a standard exclusion criterion: "No prior treatment with a checkpoint inhibitor within 6 months of enrollment." A naive rule-extraction pipeline identifies "checkpoint inhibitor" as the entity and creates a query for patients who have received one. It will catch the positive case. What it tends to miss is the temporal qualifier and the negation scope.
Clinical notes handle negation inconsistently. A note might read: "Patient has not received immunotherapy." Another note for the same patient three weeks later might read: "Started on pembrolizumab in Q3 2024, discontinued due to pneumonitis." A pipeline that flags the second note without resolving the temporal relationship to enrollment date and without reconciling it against the first note's negation statement will produce incorrect eligibility signals in both directions: false positives if it misses the treatment history, false negatives if it sees the prior treatment but cannot place it relative to the 6-month window.
Negation is handled inconsistently across EHR systems, across note types, and across individual clinicians' documentation styles. Negation detection in clinical NLP is a solved-ish problem in research settings with well-curated corpora. It is a much harder problem in production, against real notes written by clinicians under time pressure in a system that does not enforce documentation standards.
Temporal qualifiers and the window problem
Protocol criteria almost always have temporal dimensions. "Within 3 months of screening." "At least 4 weeks since last dose." "History of X within the past 2 years." These windows require the system to anchor dates, which requires finding the relevant dates in note text, and that requires reading notes rather than querying structured fields.
Lab values in structured fields have timestamps. Medication administration events in the MAR have dates. But many of the clinical events that eligibility criteria care about live only in notes: the date a biopsy was performed per a pathology read, the start of a specific therapy line as described in an oncology consultation note, the resolution of a toxicity documented in a follow-up visit summary.
Extracting those dates and placing them relative to a screening anchor requires a pipeline that understands relative time expressions ("three months prior," "following completion of second-line therapy"), recognizes date formats across the inconsistent patterns clinicians use, and correctly resolves ambiguous references like "at her last visit" or "when she was started on treatment."
Most rule-extraction systems that work well against structured data fail here because the temporal layer requires a fundamentally different kind of reading than entity recognition. It requires understanding sentences in context.
Compound conditions and criteria that reference prior criteria
Protocol criteria often compose. "Patients with advanced disease who have received at least two prior lines of therapy and have documented disease progression on the most recent line." That is three conditions in one criterion, and they are not independent: the disease progression qualifier applies specifically to the most recent therapy line, not to any prior therapy.
A pipeline that extracts entities and checks them independently will find the diagnosis, will find two or more prior lines, and will find a progression event somewhere in the record. It will pass the patient. But if the progression event documented in the record predates the most recent therapy line, that patient may not actually meet the criterion. The conjunction is logical, but the logic is sequential and referential.
We see this failure mode particularly in oncology trials with complex prior therapy requirements, and in trials where exclusion criteria are written with carve-outs: "Any prior malignancy except adequately treated basal cell carcinoma or in-situ disease." The "except" clause requires the pipeline to both identify the exclusion and evaluate the exception, and to pull the information needed to evaluate that exception from chart history.
How we approach the problem differently
Rather than converting criteria to a query language and running the query, Enrollvue treats each criterion as a reading task. The system takes the criterion text, identifies the logical conditions and their modifiers (temporal qualifiers, negation scope, conjunction type), and then reads chart notes for evidence against each of those conditions specifically.
The output is evidence-linked: for each criterion, the system returns whether supporting evidence was found, where it was found, and what the note text said. This is important because it lets a coordinator verify the system's read rather than trusting a verdict. If the system finds evidence of prior checkpoint inhibitor use, it shows the coordinator the specific note passage that mentions it, including the date context.
We are not claiming this eliminates the hard cases. Criteria that require clinical judgment, ambiguous documentation, or rare entity types are flagged for coordinator review rather than auto-decided. The goal is not full automation. The goal is to move the easy-to-resolve majority of criteria off the coordinator's plate so that they spend their limited review time on the cases that actually require their expertise.
What this means for matching accuracy
In internal testing against a set of manually reviewed patient-protocol pairs across several therapeutic areas, the gap between a structured-query approach and a note-reading approach is substantial on criteria that involve negation, temporal qualifiers, or compound conditions. On simple positive-inclusion criteria (diagnosis present, age range met, structured lab value), the two approaches perform similarly. On the harder linguistic cases, structured queries miss a meaningful share of disqualifying evidence in the record and generate false eligibility signals.
The downstream effect of those false signals is not just inaccurate candidate lists. It is coordinator time spent reviewing patients who would have been ruled out earlier, or in some cases, screening failures at later stages when the exclusion evidence is eventually found. Screening failure is expensive in time and sometimes in patient welfare when a patient has been consented for a trial they ultimately cannot participate in.
The criteria-matching problem is a language problem first and a database problem second. Building against that reality is what makes the difference between a pipeline that works in demos and one that works against production notes at a real site.