The War on False Positives

Early Release: The following content is an early preview and should be considered a work in progress. Information may be missing or incomplete until the final release

False positives are an unavoidable reality of security scanning, but that doesn’t mean we should accept the current state of affairs as good enough. Every security tool since the dawn of automated vulnerability detection has been prone to finding issues that look like security vulnerabilities but aren’t actually applicable in your specific environment. The only reliable way to identify false positives has traditionally been manual review; a time-consuming process that doesn’t scale with the volume of findings modern security tools generate.

This manual review bottleneck has plagued security teams for years. You can’t just ignore findings because some might be genuine vulnerabilities, but investigating every alert is unsustainable when you’re dealing with thousands of potential issues across multiple applications and teams. Not to mention in many large organizations that developers outnumber security 100:1 or more.

The question becomes, how can we handle false positives using more modern and automated approaches that reduces manual overhead while maintaining security effectiveness?

False Positive Detected!

The first step in building better false positive management is understanding what you’re actually dealing with. Not all false positives are the same, and different types require different handling approaches.

Detection issues are findings where the vendor signature is simply off-base. The can engine flagged something as a vulnerability when there’s no actual security issue present. This might happen when static analysis tools misunderstand framework behavior or when signature-based detection produces funky matches.

Valid detection with custom controls represent cases where the scan engine correctly identifies a potentially vulnerable code pattern, but your organization has custom security controls in place that mitigate the risk. The detection logic is working correctly, but it can’t see the compensating controls that make the finding irrelevant in your environment.

Acceptable risks aren’t false positives in the technical sense, but they function like false positives in your workflow. An example might be a low-severity vulnerability in a package that can’t be upgraded due to compatibility constraints or technical debt. The vulnerability is real, but the risk is acceptable given business and technical constraints.

A comprehensive approach is crucial when designing system workflows to manage various types of false positives. This necessity arises from the distinct remediation strategies and varying degrees of documentation and approval required for each type.

Two Key Actions: Suppressing vs. Ignoring

Once you understand what type of false positive you’re dealing with, there are two primary actions you can take, each serving different purposes in your security workflow.

Suppressing Issues

For findings where you have custom security controls in place, you can write custom rules to suppress similar findings automatically. This approach recognizes that your organization has implemented security measures that the scan engine can’t understand; something like custom sanitization functions, proprietary security libraries, or architectural patterns that prevent exploitation.

This suppression approach requires deep knowledge of your organization’s internal coding practices, security architecture, and development preferences. Historically, a human security expert would have had to make all these connections manually, reviewing each finding against organizational knowledge to determine if custom controls make it safe.

This is where AI can significantly improve the process. Modern AI systems can use Model Context Protocol (MCP) servers to “wire together” information from your internal knowledge bases, coding standards, and security documentation to generate custom rules more rapidly and accurately than manual review.

Ignoring Issues

For issues that truly need to be taken out of scope, you need to create formal “ignore” classifications. This applies to two distinct categories; True False Positives and Accepted Risks. Both categories require careful documentation and audit trails, but for different reasons. False positives need documentation to improve tool accuracy and prevent similar issues. Accepted risks need documentation to ensure the risk acceptance decision was made with appropriate authority and justification.

Finding Balance in Process

The challenge in false positive management isn’t just technical, it’s organizational. You need to balance making issue management usable for developers while maintaining appropriate accountability for security decisions.

If ignoring security issues becomes too easy, developers will start ignoring everything (and I’ve seen this happen more times than I care to admit). The path of least resistance becomes “mark it as a false positive and move on” rather than taking the time to properly analyze and address real security concerns.

Conversely, if security teams don’t proactively create suppressions and custom rules for obvious false positives, developers become overwhelmed by noise and start ignoring security findings entirely because the signal-to-noise ratio makes the (security) tools more frustrating than helpful.

The solution requires finding the right balance between usability and accountability, supported by comprehensive audit trails that track every decision without creating bureaucratic overhead that discourages proper security hygiene.

Audit Trails and Accountability

So how can we design a system to handle ignores? Well, every classification decision needs a clear audit trail that documents who made the decision, when it was made, what reasoning was used, and what evidence supported the conclusion. This audit capability serves multiple critical functions beyond simple compliance.

For each ignored or suppressed finding, there should be a clear link to:

• The original finding with full technical details
• The specific reason it was ignored or classified as acceptable risk
• Who made the initial decision to ignore it
• Who approved or declined the ignore request (if approval workflows are required)
• Any supporting evidence or analysis that justified the decision

This audit information needs to stem from an easy-to-use workflow that doesn’t create friction for legitimate security decisions, combined with comprehensive audit logs for reporting and analysis purposes. The key to success in this process is ensuring that the workflow quite literally “works” from the lens of a developer and the security team. One without the other quickly creates imbalance.

Additionally, audit trails enable continuous improvement by allowing you to analyze patterns in your classification decisions and identify opportunities for better automated detection or process refinement. They provide accountability for security decisions and ensure that risk acceptance decisions are made with appropriate authority and documentation.

AI as a Force Multiplier for Accuracy

AI is beginning to change this dynamic for the better, offering an initial automated assessment that can dramatically reduce the manual effort required for false positive identification. While it’s not possible to eliminate false positives entirely, AI is getting us closer to significantly reducing the amount of noise they generate.

The real breakthrough comes when AI systems can access your organization’s specific context and learn from your historical decisions, transforming from generic recommendations into organization specific intelligence systems.

Leveraging Internal Knowledge for Context

The first major advantage AI brings is the ability to pull in your internal knowledge base to help identify organization specific patterns that generic security rules can’t understand. This includes custom sanitization functions, internal libraries, coding standards, and architectural patterns that your organization uses to prevent security vulnerabilities.

As AI gets smarter, it can start to understand larger codebases, internal knowledge hubs, and engineering practices with increasing sophistication. When an AI system understands that your organization uses a particular input validation library, it can automatically recognize when code that looks vulnerable to a generic scanner is actually properly protected. When it knows about your custom authentication frameworks, it can distinguish between code that’s genuinely missing security controls and code that relies on framework-provided protections.

This contextual awareness transforms AI from a generic pattern-matching tool into something that understands your specific technical environment and can make more nuanced judgments about what constitutes a real security issue versus a false alarm.

Learning from Historical Patterns

The second breakthrough area involves training AI systems on your organization’s historical false positive patterns and the reasoning behind those classifications. An internal dataset of common engineering patterns, previously identified false positives, and the explanations for why they were marked as non-issues allows you to build models that understand your organization’s specific context and decision-making patterns.

This approach goes beyond simple pattern matching to understand the reasoning behind security decisions. When previous reviewers determined that certain code patterns are safe in your environment because of specific mitigating factors, AI systems can learn to recognize similar patterns and apply similar reasoning to new findings.

The Next Frontier, Custom Internal Models

The next step in this advancement is training your own internal model on your organization’s specific codebase(s) and decision patterns. These models will be able to detect false positives or issues that should be classified as acceptable risks with far greater accuracy than generic AI systems. The better your organization becomes at this internal model training, the more you can reduce your false positive noise entirely. In some anecdotal conversations, a few companies I’ve spoken to have reported seeing a greater than 50% reduction in false positives through custom model training. That’s huge.

These custom models represent a significant evolution beyond using commercial models. By training on your specific codebase, historical decisions, architectural patterns, and risk tolerance, internal models can make classification decisions that closely match what your expert security reviewers would conclude.