Safety testing for frontier AI is creating the security holes it was meant to prevent
As governments mandate third-party evaluations of advanced models, they're opening access pathways that hostile actors can exploit faster than defenses can be built. The structural problem is unsolvable with technology alone.
On May 12, 2026, the Royal United Services Institute published a framework documenting a fundamental flaw in how governments are trying to manage frontier AI risk. The problem is not technical incompetence or negligence. It is structural: the only way to test whether an advanced AI model poses a safety threat is to give external evaluators access to that model. But every access pathway creates a new vulnerability. Evaluators become targets for espionage. Model weights can be exfiltrated. Safety classifiers can be reverse-engineered. The testing regime designed to reduce risk is actively increasing it.
The scale of the problem became visible in May 2026 when two separate events exposed how frontier AI evaluation has become a security liability. Palo Alto Networks disclosed 26 CVEs (representing 75 distinct vulnerabilities) discovered during Project Glasswing, Anthropic's restricted-access testing program for Claude Mythos. This was 5 times Palo Alto's typical monthly vulnerability disclosure rate. None of these flaws had been exploited in the wild. But the volume revealed something alarming: frontier AI models can now discover zero-day vulnerabilities at scale, including flaws in every major operating system and web browser that survived 27 years of expert review. Claude Mythos identified thousands of zero-days across critical infrastructure. This capability did not exist in previous AI systems. It represents a qualitative shift in what frontier models can do.
Project Glasswing itself illustrates the access problem. Between April and May 2026, Anthropic granted 50 or more technology companies restricted access to Claude Mythos Preview, along with 100 million dollars in usage credits. The stated purpose was legitimate: evaluate whether the model could autonomously chain multi-step exploits across corporate networks. The answer was yes. The model could execute sophisticated cyber-offense campaigns without human intervention. But to conduct that evaluation, Anthropic had to give external organizations the ability to test exactly that capability. Each evaluator became a node in a network where a single compromised employee, a single insider threat, or a single security lapse could exfiltrate the model weights, training data, or safety classifiers that reveal how to bypass the model's safeguards.
## The access control problem has no technical solution
RUSI's framework identifies the core tension. Meaningful safety testing requires evaluators to have access to model internals: activation patterns, logits, chain-of-thought outputs, and minimally guardrailed versions of the model. This is not optional. You cannot test whether a model will behave safely in adversarial scenarios without being able to probe its internal reasoning and run it without safety constraints. But each access type creates a corresponding risk. Write access to model internals allows adversaries to tamper with model behavior directly. Access to training environments reveals the data and processes that shaped the model's capabilities. Access to safety classifiers exposes the exact mechanisms defenders are using to prevent misuse.
The problem is compounded by inconsistent standards across jurisdictions and organizations. One evaluator may receive limited API access while another receives white-box visibility into the full model architecture. The EU AI Act's Code of Practice, effective August 2026, explicitly requires evaluators to have adequate model access, including logits and minimally guardrailed versions. This creates exactly the access pathways RUSI warns are highest-risk. Yet there is no internationally coordinated standard for what 'adequate access' means, what security controls must be in place, or what happens if an evaluator is compromised. Hostile states, criminal organizations, and rogue insiders can exploit these gaps.
Governments face a dilemma that cannot be resolved through better encryption or access controls. The dilemma is this: meaningful safety testing requires deep access to frontier models. Deep access creates security vulnerabilities. Those vulnerabilities cannot be fully mitigated without restricting access. But restricting access undermines the safety testing that justified the access in the first place. This is not a problem that more security engineering can solve. It is a policy problem that requires choosing between competing risks.
## The supply chain is already compromised
Evaluator organizations have become high-value targets for espionage. A single compromised evaluator with white-box access to an unreleased frontier model represents a single point of failure for national security. The cost of compromising an evaluator is lower than the cost of developing equivalent capabilities independently. Hostile states have already demonstrated willingness to conduct sophisticated cyber-espionage against AI labs. In 2025, Chinese state-sponsored actors conducted multiple intrusions against OpenAI, Anthropic, and xAI, attempting to exfiltrate model weights and training data. These operations were detected, but they established proof of concept. Now, instead of attacking the labs directly, adversaries can target the evaluators.
The problem extends to insider threats. Evaluators employ security researchers, engineers, and policy analysts who have legitimate access to frontier models. Some of these individuals will have financial incentives, ideological motivations, or coercive pressure to exfiltrate information. The 2023 OpenAI leak, the 2024 Anthropic internal breach, and the 2025 xAI data exfiltration all involved insiders with legitimate access. As the number of evaluators increases and the number of people with access to frontier models grows, the probability of a successful insider attack approaches certainty.
Anthropic estimates that comparable cyber-offense capabilities will exist at other labs within 6 to 18 months. This means the controlled access that governments are relying on to manage frontier AI risk only buys defenders a narrow window before the capability proliferates. Once Claude Mythos' cyber-offense capabilities are documented in evaluator reports, other labs will prioritize developing equivalent capabilities. The window closes. The evaluation regime becomes obsolete. And the access vulnerabilities remain.
## Regulatory arbitrage will undermine coordination
The patchwork of evaluation standards creates regulatory arbitrage. Labs can shop for jurisdictions with weaker access controls, undermining the entire premise of coordinated pre-deployment testing. The Trump administration's CAISI requires government testing of all frontier models before public release. The EU AI Act requires evaluator access to model internals. China has not published equivalent requirements. A lab could release a frontier model in China with minimal evaluation, document the evaluation in EU-compliant terms, and claim it meets U.S. standards. The evaluation regime becomes a checkbox rather than a meaningful constraint.
The cost and complexity of secure evaluation access will concentrate frontier AI development further. Only labs with resources to manage evaluator access, maintain security infrastructure, and coordinate across multiple jurisdictions will participate in the evaluation regime. Only evaluators with security clearances, institutional backing, and sophisticated access controls will be trusted with frontier models. This excludes smaller AI companies, international researchers, and academic institutions. The frontier AI development process becomes more opaque, not more transparent. The very goal of evaluation, external oversight, is defeated by the security requirements needed to make evaluation safe.
## What governments should do now
The correct response is not to improve access controls or refine evaluation standards. Those measures are necessary but insufficient. The correct response is to pause mandatory third-party evaluations until three conditions are met. First, international coordination on evaluation standards must exist. This means the U.S., EU, China, UK, and other major powers must agree on what access evaluators can have, what security controls must be in place, and what happens if an evaluator is compromised. This coordination does not exist today. Second, evaluator organizations must have security infrastructure equivalent to national laboratories. This means security clearances for all personnel, continuous monitoring, compartmentalization of access, and rapid response protocols for breaches. Third, there must be a mechanism for revoking evaluator access and pursuing legal consequences for breaches. Today, if an evaluator is compromised, there is no clear enforcement mechanism.
Until these conditions exist, governments should rely on in-house evaluation by their own security agencies. This is slower, less scalable, and less transparent. But it is more secure. The National Security Agency, GCHQ, and their equivalents have the infrastructure, personnel, and legal authority to evaluate frontier models without creating new vulnerabilities. They can share findings with allied governments through existing intelligence channels. This approach does not solve the frontier AI safety problem. But it does not make the problem worse by opening new attack surfaces.
The alternative is to accept that frontier AI evaluation regimes will be compromised. Assume that evaluator access will be exploited. Plan accordingly. This means accelerating defensive capabilities, increasing monitoring of frontier model capabilities, and preparing for scenarios where hostile actors have access to unreleased models. It means treating the evaluation regime as a security theater that creates the appearance of oversight without the reality. Some governments may choose this path. It is a legitimate policy choice. But it should be made explicitly, not by default.