AutoTrace
From patches to triggers — agentic interprocedural exploration that localizes the exact statement where a vulnerability fires, then turns verified source-to-sink chains into a benchmark.
Overview
Given a vulnerability-fixing commit, trigger localization asks which specific statement turns the vulnerable program state into a concrete unsafe operation. The question is harder than binary detection because the answer demands interprocedural, causal reasoning: in a substantial fraction of real-world CVEs the triggering statement lies several call layers outside the patched function, beyond the reach of static rule sets and pattern-matching language models alike.
AutoTrace is an agentic pipeline that localizes triggers by exploring a code property graph layer by layer, with LLM agents deciding where to look next and deterministic admissibility gates deciding what evidence is required before a trigger can be reported. Agents never accept a trigger on their own authority; every reported trigger is backed by explicit graph evidence. On the full InterPVD benchmark AutoTrace reaches 75.0% VulnHit and 80.8% FuncHit, surpassing the prior state of the art on the same corpus. Building on the same machinery, we construct SinkTrace-Bench, a dataset that exposes each vulnerability as a source-to-sink causal chain drawn from matched vulnerable and patched program states — and find that even frontier LLMs struggle to separate the matched pairs.
The trigger is not where the patch is
cf2_interpT2CharString, yet the trigger — the sink where the corrupted index is read, stack->buffer[idx] — lies three call layers away in another file. Patch-line and single-function methods point at the guard; only statement-level interprocedural localization reaches the sink.How AutoTrace works
Two principles govern the design. First, LLM agents act as semantic reasoners that interpret code and decide where the search moves next, but never bypass the hard admissibility checks: every candidate must survive deterministic gates that test it against explicit code-property-graph evidence. The model explores, the evidence decides. Second, interprocedural analysis proceeds through repeated local slicing rather than one whole-program slice — each function is sliced independently and evidence is stitched across call edges, bounding both analysis and prompt context to a single function however deep the chain goes.
On the CVEs both cover, AutoTrace localizes the exact trigger statement almost twice as often (71.0% vs. 37.8% VulnHit). The baseline names the right function but misses the triggering statement; its higher function-level reach is a coverage artifact, since a plain LLM always emits some answer while AutoTrace's verifier abstains rather than guess.
75.0% VulnHit on the full InterPVD corpus, against VulTrigger's 69.8% under the same line-level inclusion metric — while also reporting triggers several call layers outside the patched function that single-function methods cannot reach.
$0.65 and 23.2 LLM calls per CVE on average. Wall-clock is dominated by CPG construction and slicing, not LLM inference — and the graph is built once per repository, so corpus-scale runs amortize the dominant cost.
From verified chains to a benchmark
Every verifier-confirmed trigger forks into a matched pair: a vulnerable half from the pre-fix snapshot and a safe half from the post-fix snapshot, each carrying the full source-to-sink slice, the trigger statement, and the critical variable. Prior datasets label what is vulnerable; SinkTrace-Bench annotates why — the causal chain is a first-class label. The release holds 771 matched pairs (1,542 samples) across 16 CWE classes. We manually annotated all 1,542 samples against a fixed rubric: every sample carries an agreed vulnerable/safe label (100% label fidelity), and the annotated trigger genuinely causes the vulnerability in 83.8% of vulnerable samples.
stack->buffer[idx]), so a model that keys on the sink alone cannot separate them. The sole difference is a guard two call hops upstream that bounds the value before it propagates — assigning opposite labels demands interprocedural reasoning along the source-to-sink chain, not patch-diff pattern matching.Can frontier models tell the twins apart?
Top zero-shot accuracy is 59.0%, only nine points above the 50% chance floor on the balanced set — and even that comes from defaulting to a vulnerable verdict.
The strongest backbones reach 87–95% true-positive rate but collapse to 7–31% on the safe half: they predict vulnerable whenever the sink is present. Separating a matched pair requires reasoning about the upstream fix, which is exactly what sink pattern matching cannot do.
A model recalling “this CVE is vulnerable” would not explain the systematic failure on the safe halves of the same chains; the spread tracks code-reasoning ability rather than mere exposure.
Why this matters
The model explores, the evidence decides
Separating agent proposals from deterministic acceptance means no reported trigger rests on unverified model judgment — a successful prompt injection can at most yield a rejected candidate, never an unsupported report.
Statement-level answers are what practitioners consume
A function-level label says a defect is somewhere in a long body; a patch hunk often marks a guard upstream of the real problem. Only the trigger statement pins the exact operation that patch review, triage, and test construction actually need.
Causal labels, not surface patterns
Datasets built from patch syntax teach models to imitate changed-code patterns. SinkTrace-Bench labels causal structure — matched pairs that differ only by the fix — so progress on it requires the reasoning the task actually demands.
Toward leak-free discovery
Because every reported trigger is graph-gated rather than a model guess, the same machinery extends to repository snapshots newer than any model's training cutoff — evidence-backed candidates in live code, where no public record can leak the answer.