The core problem: Context vs. rules
Traditional SAST tools, as we know, are rule-bound; they inspect code, bytecode, or binaries for patterns that match known security flaws. While effective, they often fail when it comes to contextual understanding, missing vulnerabilities in complex logical flaws, multi-file dependencies, or hard-to-track code paths. This gap is why their precision rates and the percentage of true vulnerabilities among all reported findings remain low. In our empirical study, the widely used SAST tool, Semgrep, reported a precision of just 35.7%.
Our LLM-SAST mashup is designed to bridge this gap. LLMs, pre-trained on massive code datasets, possess pattern recognition capabilities for code behavior and a knowledge of dependencies that deterministic rules lack. This allows them to reason about the code’s behavior in the context of the surrounding code, relevant files, and the entire code base.
A two-stage pipeline for intelligent triage
Our framework operates as a two-stage pipeline, leveraging a SAST core (in our case, Semgrep) to identify potential risks and then feeding that information into an LLM-powered layer for intelligent analysis and validation.
