Approve the AI and Its Operating Environment Together

A high-stakes AI system does not arrive alone. It enters local data, professional practice, user interfaces, access controls, staffing, escalation paths and reporting systems. Each can change performance or determine whether a weak result is caught before it matters.

The commission therefore recommends that manufacturers specify the operating conditions needed for safe use, including cybersecurity, user training and institutional readiness. It also calls for clearer responsibility, stronger implementation governance and monitoring throughout the product lifecycle. ¹

The transferable point is not medical-device regulation. It is that product risk and operating risk cannot be approved in separate rooms. If procurement approves the capability while operations, risk and the workforce discover the conditions later, the organisation has approved only part of the system.

First, approve the product boundary: the intended job, population, prohibited use and material limitations. Second, approve the host conditions: data quality, user roles, integration points, training and cyber controls. Third, approve the learning loop: what is monitored, who can report a concern and which changes trigger reassessment. Fourth, approve the exit: who can pause the system, how service continues and how affected people obtain review or redress.

These four decisions belong in one paper because they fail together. A technically capable model with an untrained user is not the approved system. A validated model connected to changed data is not the same system. A useful pilot without monitoring or withdrawal is not ready to become an operating dependency.

Use the gate at initial deployment and whenever the model, data, workflow, user group, population or level of authority changes.

The UK is extending the approach. London Region I is designed to deploy up to 10 AI-enabled medical devices in controlled, real-world NHS settings under regulatory oversight. Its stated purpose is to generate additional information about benefit, safety and effectiveness before wider adoption. Results are not yet available. ⁴

That distinction matters. A sandbox is not a badge of success and a recommendation is not a deployment outcome. Their value is the learning architecture: limited scope, named partners, live observation and a path to adjust the rules as the system meets reality.

A board can borrow that architecture without creating a formal regulatory sandbox. Limit the first population, define the permitted workflow, observe the complete service and expand only when the product and environment pass together.

The joint approval principle travels across markets and sectors. The exact conditions do not. In health, professional duties, clinical governance, patient communication and redress vary. In finance, credit, conduct and model-risk rules take their place. In public services, administrative law, accessibility and citizen challenge matter.

The European Union adds AI Act and sector-specific obligations. An approval copied from one country may therefore preserve the product description while missing the local operating system.

Trust also depends on context. UK public research published this month found support for healthcare AI was conditional on strong safeguards, accountability and human oversight. Those findings should shape the UK case; they should not be silently converted into a global percentage. ⁵

Choose the next AI proposal that can affect a customer, employee, patient, citizen or material financial decision. Add one page titled “Operating environment.” Name the required data, workflow, skills, controls, monitoring, incident route and exit. Put an accountable owner beside every condition.

Then ask what would make this a different system. A model upgrade, a new population, a new data source or added authority should not inherit approval automatically. Set the reapproval trigger before the pilot begins.

The goal is not to make every deployment slow. It is to stop speed being purchased by leaving half the system unowned. Approve the product and its operating environment together, then let both earn the right to scale.