Decide Where AI Work Is Allowed to Run

PAIR can direct a request to a compatible computer on the same local network. It does not combine several computers’ graphics memory or split one request across their processors. The requested model must be available on an eligible node. That distinction matters when a purchasing case assumes that several smaller devices become one larger machine. ²

Local-network routing also differs from choosing between a local model and a cloud service. NVIDIA’s announcement discusses both, including its Switchyard routing library. Treat them as separate capabilities with separate release dates and information boundaries, not a single promise that every application will keep every job local. ¹

GitHub provides a useful, specific warning. Its 7 October Copilot CLI announcement says choosing a local model does not enable offline mode or disable telemetry. It also warns that a remote provider can still receive prompts and code context even when offline mode is enabled. Its intelligent local routing is announced with availability still forthcoming. This is evidence about that product, not every AI assistant. ³

Use three simple categories for a pilot: this device, an approved local-network machine, or an approved cloud service. These are proposed management categories, not assurances supplied by a router. Record which categories each task may use, what information may travel and which person owns the decision.

Then specify what happens when the approved route cannot serve the request. A model might be missing, a machine busy or a connection unavailable. For sensitive work, the right behaviour may be to stop and explain why. For another task, an approved alternative may be acceptable. Silent fallback should not make the policy decision for you.

Follow the whole job, not only the model call. An assistant can run inference locally and still use a remote search service, repository or business application. Review those connections and the permissions to change records or send information. The useful boundary is the workflow the customer or employee actually uses.

Ask for an observable record of the destination. PAIR’s documentation explicitly notes that receiving a reply is not enough to prove a request used the intended remote node. Apply that discipline more broadly: a successful answer is evidence of completion, not evidence of where processing happened. ²

Recent research reinforces the need to test changing conditions. RATED studies tool selection that uses both task fit and execution reliability. Its authors report improved results in a benchmark with injected API failures. That suggests a useful design direction: selection should respond to whether a service can actually perform the job. It does not establish an enterprise outage rate or a privacy guarantee. ⁴

DualQuant explores reducing the memory and computation needed for language models. Its reported improvements use particular models, tests and hardware, including an RTX 4090 implementation. Smaller local deployments may become more practical, but a research speed-up is not a complete business case. Neither study proves that your workload should move from cloud to device. ⁵

Both papers were first submitted on 9 October and were visible in the latest release batch inspected, dated 12 October. DualQuant is a preprint. RATED’s authors report conference acceptance; that status should not be confused with an independent replication of their results.

Cross-border organisations should check the actual processing destinations, supplier terms and applicable obligations for each deployment. An office in Europe, North America or Asia-Pacific may use the same assistant but a different provider configuration. Physical proximity alone does not establish permission to process data or a contractual assurance.

The economics also change by location and institution. Device supply, reliable power, connectivity, support capacity and performance in the working language belong in the trial. These are variables to measure, not reasons to assume every market in Africa, the Middle East or Latin America has the same constraint.

The World Bank’s recent Small AI event frames affordable, locally adapted applications as a development opportunity. That is a useful counterweight to treating high-end hardware as the universal starting point. It is an event framework, not measured proof of job creation or local-AI returns. Comparable independent fleet economics across regions remain missing. ⁶

Choose three real tasks with different information sensitivity and service needs. Have the business owner, IT, security and finance agree the permitted routes and failure response. Test a busy machine, a missing model and a lost connection as well as the normal path. Check destinations directly rather than relying on the answer’s quality.

Compare the complete service with the current alternative: accepted output, waiting time, human review, support effort, energy and device cost. Retain failed jobs. Expand only where the evidence supports a repeatable operating arrangement. The next purchase should follow a demonstrated way of working, not precede the decision about where work may go.