Ask What the AI Safety Audit Actually Tested

A preprint newly listed on arXiv on 30 September reconstructs selected agent failure behaviours in simplified environments. Its authors found that eliciting different known behaviours required very different amounts of testing. Their automated method targeted behaviours described in advance; it did not solve the harder problem of discovering risks nobody had anticipated. ²

The work is a controlled research exercise, not a census of deployed failures. Its simplified setup and simulated interactions limit what can be inferred about production systems. It nevertheless illustrates a purchasing problem: two reports with no observed failure may represent very different searches.

Ask the evaluator how the test cases were chosen, which conditions were varied and what stopped the search. Was the budget exhausted, a coverage target reached, or an agreed procedure completed? A clean result without that context can be overread. More effort is not automatically better, but the amount and purpose of the effort should be visible.

Keep ordinary task quality separate from the ability to resist misuse or unintended action. Passing one evaluation does not answer every other question. The report should say which property was examined rather than inviting the reader to infer that the whole service is safe.

Start with a short description of the proposed use. Name the model version, surrounding software, available tools, permissions, data types and relevant languages. Record whether the evaluator tested that configuration or a different one. A supplier’s model-level assessment can be valuable without covering your connected application.

Next, require a method and effort statement. It should describe the test population, number of attempts, repeated runs, human review and exclusions. Where results are sampled, ask how uncertainty is represented. Do not demand an attractive percentage unless its denominator and meaning are clear.

The findings should distinguish an observed failure, a suspected weakness and an untested possibility. Ask for severity in terms of the business consequence, with supporting evidence available through an appropriate confidential channel. A public summary may protect sensitive details; confidentiality should not prevent the accountable decision-maker from understanding the remaining risk.

Finally, request the remediation trail: what changed, who checked the change, which tests were repeated and what remains unresolved. Finding a problem is not the same as closing it. If the proposed fix changes a capability or introduces a new dependency, that trade-off belongs in the approval record too.

The US accord is a voluntary commitment by its signatories. Do not treat it as a new worldwide duty, an exemption from existing law or evidence that a particular product passed an assessment. Nor should an enterprise buyer assume that a frontier-model commitment covers every downstream application it builds. ¹

In the European Union, the AI Act imposes additional duties on providers of general-purpose AI models with systemic risk. Commission guidance describes model evaluation, systemic-risk assessment, serious-incident reporting and cybersecurity obligations. It also distinguishes models placed on the market before 2 August 2025, for which the relevant compliance transition runs to 2 August 2027. These are scoped legal duties, not an interchangeable version of the US pledge. ³

Singapore’s updated Model AI Governance Framework for Agentic AI offers deployment guidance. It is a governance framework, not a universal legal certification. ⁴ A company can use such guidance to improve its tests while separately determining what the law requires in each market.

For the UK and other markets, do not copy the EU provider classification into a local compliance checklist without review. For operations across Asia-Pacific, Africa, the Middle East and Latin America, identify applicable sector, privacy and contractual requirements and test the actual languages and infrastructure used. This briefing does not establish equivalent laws or comparable audit capacity across those regions.

An external evaluator can challenge assumptions that an implementation team misses. Ask about relevant expertise, access to the system and any financial or implementation interests. But independence alone does not establish adequate coverage. A technically strong review can still answer a narrower question than the business is asking.

Resist turning audit effort into a competition over the largest number of tests. Repeating easy cases can inflate activity without improving coverage of consequential failures. Conversely, a small, well-designed test may reveal a reason to stop. Ask which decision changed because of the work and what contrary evidence was considered.

Where local language, connectivity or institutional practice differs from the tested setting, record the gap. Use a bounded local trial or additional review to close it before expanding the claim. The absence of comparable evidence is a limit on the conclusion, not evidence that a region is unsafe or that every deployment must wait indefinitely.

For the next material AI approval, have the business owner and risk lead agree on the decision first. Then commission the evidence needed for it. Attach unresolved findings to the approval, with a named owner and a clear condition for escalation.

Set retest triggers for meaningful changes to the model, tools, permissions or intended use. The old report can remain useful history without being treated as current assurance. Record which changes require a fresh assessment and which can be handled through routine monitoring.

The aim is not another document that says AI was reviewed. It is an approval whose limits are understandable: this configuration, for this use, on this evidence, with these issues still open. That gives a board something it can question—and management something it can act on.