Survey: Security teams growing wary of fully automated pentesting tools

Security professionals appear to be losing confidence in fully automated penetration testing, according to a new report from offensive security firm Cobalt.

The company’s 2026 State of Pentesting survey suggests that many teams have become frustrated with tools that promise broad coverage but miss serious issues in practice. Cobalt said 78% of respondents reported “critical false negatives” from automated scanners, particularly when testing environments that include AI and large language model components.

Why the skepticism is increasing

According to Cobalt, traditional scanners remain useful for finding known, signature-based weaknesses. The problem, the company argues, is that these tools often struggle with flaws that require more context, chained interactions, or human-like probing.

That matters more in AI-driven systems, where security issues such as prompt injection or excessive agent autonomy may only surface after multiple steps of interaction. Cobalt said those kinds of logic flaws are unlikely to be exposed by single-pass automated checks.

• Only 9% of respondents said they were open to fully autonomous pentesting, down from 29% a year earlier.
• 78% said they had seen critical false negatives from automated tools.
• In traditional environments, about 12% of detected vulnerabilities were rated high or critical.
• In AI and LLM environments, that figure rose to 32% and has remained at that level for two years, Cobalt said.

Hybrid testing gains favor

Cobalt argues that the shift away from full automation is not necessarily a setback for the industry. Instead, it says organizations are increasingly looking for assurance rather than simple scan coverage, and that the most sensitive systems still need human judgment.

The report’s sample size was relatively small, with 450 respondents, but the findings echo a wider concern in the sector: AI-assisted development is creating more security work than many teams can comfortably absorb.

Not everyone is as cautious. Amazon security chief CJ Moses has said AI-based pentesting has improved team efficiency, though he has also stressed that humans still need to remain involved in decision-making and oversight.

For now, the trend appears to favor a mixed approach: automated tools for scale, and human testers for the hardest problems.