AI Can Find Risk Faster Than You Can Retire It

The UK's Government Cyber Coordination Centre, working with the National Cyber Security Centre and AI Security Institute, ran weekly in-person hackathons across nine government organisations. Teams scanned public code repositories, experimented with different model-assisted methods and reused the approaches that proved most useful. The government reports a model-token cost of £13,000 for the month. ¹

Participants reported 407 findings, including weaknesses associated with authentication bypass, data exposure and remote code execution. Some findings were already understood and protected by compensating controls; others were previously unknown. The official case says all critical weaknesses were remediated and that no evidence of exploitation was found. ¹

One reported example involved a legacy automation workflow in which a specially constructed public comment could trigger code execution and potentially expose repository credentials. The important operating result was not that a model described the issue. Specialists re-verified the exposure, assessed exploitability and moved the confirmed weakness into remediation.

Teams did not rely on one prompt or one autonomous agent. One approach used six stages that challenged and refined earlier findings. Another began with conventional security scanners, then used models to investigate how separate weaknesses might combine. A third converted a repeatable multi-service audit into reusable, tightly scoped skills. ¹

Human expertise stayed at the consequential boundary. Every material escalation was checked, exposure was re-verified and false positives were handled before remediation. Existing vulnerability-management frameworks determined what was prioritised and patched. The model expanded search and connected context; it did not own risk acceptance or production change.

This matters beyond cybersecurity. AI implementation performs best when the work is decomposed, the interface to existing systems is explicit, an accountable person decides at the point of consequence and the output enters a process capable of acting on it. The model may accelerate one stage, but the operating result belongs to the complete chain.

The case is reported by the organisations that ran it. It does not publish the total number of repositories scanned, a severity distribution, false-positive rate, analyst hours, a conventional-scanner comparison or the time and cost required to remediate each class of issue. The £13,000 figure covers model tokens, not the full programme cost.

The public-code setting also matters. Code that had already passed publication review could be shared with model providers under conditions that may not apply to confidential systems, regulated data, proprietary software or markets with different disclosure laws. The planned second phase will extend to closed-source estates, so the operating and legal conditions may change. ¹

Leaders should therefore treat the case as proof that a structured trial can produce useful discoveries—not as a universal productivity ratio, a vendor comparison or evidence that autonomous remediation is ready. The relevant test is whether the organisation can reproduce the full workflow and measure net risk reduction in its own environment.

Three new arXiv preprints point in the same direction without turning this case into a technical paper digest. QuoteBench shows that an agent can generate the right command yet fail when an interface serialises and reparses it; similar headline scores can conceal very different failures along the execution path. Beyond Final Scores finds that long-running agents with similar outcomes can have different process bottlenecks, inconsistent runs and experience that sometimes harms later decisions. ⁵⁶

Practice Makes Unsafe examines systems that convert successful past behaviour into reusable procedures. In its experimental setting, unsafe success could persist into later tasks after the original trigger disappeared. The authors' proposed control governs both what is written into persistent skill state and what later work is allowed to reuse. It is a new preprint, not production evidence. ⁷

The plain-language implication is consistent: evaluate the whole operating path and lifecycle. A model benchmark cannot tell a leader whether information survived an interface, whether a procedure accumulated unsafe behaviour, whether review caught the problem or whether the organisation could act on the result.

The core pattern travels across markets: scope the discovery task, constrain access, validate consequential outputs and connect them to an owned remediation process. What changes is the code that can be shared, the legal jurisdiction of hosted models, the maturity of vulnerability disclosure, the availability of security specialists and the capacity to patch legacy estates.

European organisations must layer sector, data-protection, cybersecurity and AI Act duties onto the same workflow. Organisations in markets with limited specialist capacity may need a narrower discovery scope because verification—not model access—is the scarce resource.

The practice to change now is simple: set an intake limit. Approve the next repository, system or business unit only when the current queue meets agreed targets for validation time, critical-finding ownership, remediation time, false-positive load and recurrence. If discovery grows faster than closure, pause expansion and strengthen the pathway that retires risk.