LLMKernelBench: Benchmarking Large Language Models on Software Vulnerability Detection in Linux Kernel
IEEE Transactions on Reliability · pp. 1–15 · 2026
I’m a Ph.D. researcher working on verification and reward design for long-horizon software agents. My work spans large language models, autonomous agents, and program analysis, applied to the open problem of checking whether a model’s claim about a security flaw is actually true.
Probing how large language models reason about code, exploits, and patches, from zero-shot detection to chain-of-thought patch synthesis.
trace.llm — reasoning · zero-shot · patching ActiveAgent loops that read, instrument, and probe codebases. They plan experiments, execute tools, and propose exploit hypotheses without human prompting.
trace.agents — planner · tools · self-eval In-progressCheckers that ground a security claim in real program evidence rather than a persuasive argument. What each one covers, and what it costs when they can be fooled.
trace.verify — coverage · gaming · reward ActiveStatic and dynamic analysis married to neural models: benchmarks, datasets, and adversarial stress tests that ground evaluation in real CVEs.
trace.analysis — benchmarks · cwe · safetyI am a Ph.D. student in Computer Science at UNC Charlotte, advised by Prof. Marco Vieira. I work on verification and reward design for long-horizon software agents, using vulnerability reasoning as the testbed. I build systems that read code and reason about security, and I care just as much about whether those systems can be checked when they get it wrong.
Two of my papers were accepted to the ISSRE 2026 Main Research Track. AutoTrace is an agentic pipeline that localizes vulnerability triggers by exploring a code property graph layer by layer, with every reported trigger backed by explicit graph evidence rather than ungrounded model judgment. CWE-Trace diagnoses what fine-tuning actually buys an LLM on systems software, and finds calibration without comprehension: output distributions adapt to the training data while the underlying security reasoning stays absent.
My earlier work focuses on large language models for vulnerability detection, automated program repair, and benchmark development, including VulnSage on zero-shot reasoning, VulnLLMEval on evaluation, and an ICSE 2026 workshop paper comparing diverse LLMs on detection and repair.
My current work asks what I think is the harder question. How do you check whether a model’s security claim is actually true? Training against verifiable rewards works beautifully in math and code, where an answer either passes the test or it doesn’t. Security breaks that assumption. A sanitizer gives near-perfect ground truth for bugs that crash, but says nothing about the logic and authorization flaws that never crash at all. The usual fallback, asking another model to grade the answer, covers everything but can be talked into agreeing with a confident wrong one.
I am studying a third option: program-analysis verifiers that demand a real dependency path through the code rather than a persuasive paragraph. Execution-free like a learned judge, but not something a model can argue its way past. What interests me is the tradeoff between them. What can each kind of checker actually score, and how often does each one get fooled when a model optimizes against it?
I love connecting with other researchers and practitioners. If you’re interested in collaboration, have a question about the work, or are building in this space, please reach out.
IEEE Transactions on Reliability · pp. 1–15 · 2026
37th IEEE Int’l Symposium on Software Reliability Engineering (ISSRE) · Main Research Track · 2026
37th IEEE Int’l Symposium on Software Reliability Engineering (ISSRE) · Main Research Track · 2026
3rd International Workshop on LLMs for Code (LLM4Code) · co-located with ICSE 2026
arXiv preprint arXiv:2503.17885 · 2025
Journal paper published in IEEE Transactions on Reliability: LLMKernelBench: Benchmarking Large Language Models on Software Vulnerability Detection in Linux Kernel.
Two papers accepted at ISSRE 2026, the 37th IEEE International Symposium on Software Reliability Engineering, Main Research Track: 🎉
Attended the 2026 NSF CyberTraining Summer Camp on LLM Cybersecurity at Penn State University (May 18–28), covering AI safety and alignment, LLM security, and safeguards for AI systems, with a hands-on group project.
New paper accepted: Diverse LLMs vs. Vulnerabilities: Who Detects and Fixes Them Better? at the 3rd LLM4Code Workshop, co-located with ICSE 2026.
New preprint: Reasoning with LLMs for Zero-Shot Vulnerability Detection.
Released VulnLLMEval, a benchmark for evaluating LLMs on software vulnerability detection & patching.
Published a comprehensive survey on Smart Grid security.
Started Ph.D. at UNC Charlotte, advised by Prof. Marco Vieira.
Research collaboration, a question about a paper, PhD mentorship, or a role you think I’d fit — tell me a little and I’ll follow up with times that work.