Open-Weight AI Models Are Closing the Cyber Gap With Frontier Systems, AISI Finds

The UK's AI Security Institute has published its first public comparison of how open-weight models stack up against leading proprietary systems in cyber capabilities. The finding: the gap is shrinking fast, and the cost of running cyberattacks on open models is staggeringly low.

AISI tested models on two benchmarks. The first, a set of 70 cyber tasks spanning vulnerability research, reverse engineering, web exploitation, and cryptography, showed GLM-5.2 matching the performance of Anthropic's Opus 4.6 — released roughly four months earlier. DeepSeek V4-Pro hit the level of Opus 4.5 from late 2025. At the start of 2025, the gap between open and closed models sat at six to ten months. The second test, a simulated 32-step corporate network attack called "The Last Ones," showed a wider seven-month gap, though AISI cautioned that result rests on fewer scenarios.

The cost disparity is where things get stark. Running a 100-million-token Cyber Range test cost about $85 on Opus 4.5 or 4.6, $46 on GLM-5.2, and just $1.19 on DeepSeek V4-Pro. For individual tasks that both models solved reliably, DeepSeek cost 28 cents per task compared to $15 for Opus 4.6. That makes scaling cyberattacks on open models dramatically cheaper.

AISI also found that safety guardrails on open-weight models were largely ineffective. DeepSeek V4-Pro occasionally refused reverse-engineering tasks, but simply retrying bypassed the restriction. The institute noted that broader safeguards like monitoring and classifiers don't carry over to models anyone can download and run privately. It framed the closing performance gap as a shrinking window for defenders using the strongest closed systems to prepare before equivalent capabilities become freely available without comparable controls.