When the Frontier Was Jagged

By early 2023, leaders could see that general-purpose language systems could produce useful text, analysis and code across many tasks. What they could not see was a mature distribution of enterprise outcomes. Demonstrations established possibility; they did not establish reliability, economics, control effectiveness or the organisational changes required for durable value.

NIST released AI RMF 1.0 in January 2023 around four continuous functions—govern, map, measure and manage. Its existence is important historical evidence: the need for contextual mapping, measurement and lifecycle governance was not an objection added after deployment problems appeared. It was part of the available operating record from the start. ³

Economic analysis during 2023 also warned against treating automation as a one-directional labour story. The Turing Transformation argued that task automation could change skill premiums and expand who can perform complementary work. It was a theoretical mechanism, not a forecast that every organisation would realise those benefits, but it made the outcome focus explicit. ⁴

A reasonable 2023 position was therefore neither ‘ignore this’ nor ‘transform everything’. It was to build an evidence portfolio: identify candidate tasks, measure outcomes, map risk, and preserve the ability to learn as capabilities and work practices changed.

The most influential enterprise experiment studied 758 consultants performing 18 realistic tasks. On tasks judged to sit inside the tested system’s capability frontier, access increased completion, speed and human-rated quality. The experimental design and preregistration made this far stronger than product anecdotes. ¹²

The result was economically meaningful but bounded. The participants were consultants from one firm; the work was a selected set of knowledge tasks; the system and prompting approach reflected a particular point in model development; and the outcome measures were task performance rather than realised client or firm economics. Those are not defects. They define what the study can support.

The study also found different interaction patterns. Some participants divided work between human and system; others integrated AI throughout the task. That variation matters because ‘AI access’ was not a single treatment in practice. People built different configurations, and those configurations changed how capability entered the work. ¹

The defensible 2023 conclusion was conditional: for some professionally relevant tasks, some workers using a capable system could produce more or better output quickly. It was not evidence that every workflow should be automated, that quality would remain high beyond the task boundary, or that the surrounding organisation would capture the gain.

The same research programme supplied the warning that later headlines often removed. On a task outside the tested capability frontier, participants with AI were less likely to reach the correct answer. A system could make work feel fluent and complete while moving the user away from the right result. ¹²

This creates an asymmetric management problem. Gains inside the frontier are visible as faster production. Failures outside it can be hidden by plausible language, especially when reviewers lack the expertise or time to reconstruct the reasoning. The risk is not simply a hallucinated sentence; it is misplaced organisational confidence.

The frontier is also dynamic. Model updates can move it, but changing tools do not remove the need to identify it. A task that works in one context can fail after data, instructions, users, integrations or consequence change. Evaluation therefore belongs to the deployed configuration and must be repeated after material change.

A board could already have asked in 2023: which tasks have been tested, where did performance deteriorate, who can recognise failure, and what happens when the system is uncertain? Those questions remain more useful than a general claim that the next model is better.

Market discussion converted early evidence through four leaps. It moved from selected tasks to whole jobs; from participants to all workers; from output measures to financial value; and from a temporary model configuration to a stable technological rule. Each leap might become true in a particular organisation, but none followed automatically from the experiment.

The confusion was reinforced by the word productivity. In a study it can mean tasks completed, issues resolved or time spent. At enterprise level it may mean value added relative to labour and capital. A faster draft does not establish higher firm productivity if review grows, demand is fixed, saved capacity is not redeployed or errors create downstream cost.

The generalisation error also obscured heterogeneity. Workers differed in baseline experience, task approach and reliance on the system. Later customer-service evidence found gains concentrated among less experienced workers, with far smaller speed gains and small quality deterioration among the most skilled. ¹¹

The Institute’s retrospective judgment is that 2023 evidence was strong enough to justify structured experimentation and too weak to justify universal operating claims. The error was not optimism. It was failing to preserve denominators, task boundaries and outcome definitions while translating research into strategy.

Representative US surveys later showed how quickly individual use spread. By late 2024, nearly 40% of adults aged 18–64 reported using generative AI, 23% of employed respondents had used it for work in the previous week and 9% used it every workday. The same research estimated only 1–5% of work hours were assisted. ⁵

Those figures resolve an apparent contradiction: a technology can diffuse rapidly through workers while remaining shallow in total work. Adoption counted as people touched, weekly users or hours assisted produces different answers. None measures whether a controlled workflow changed or whether the employer realised value.

This was an unmanaged adoption wave because the unit of diffusion was often the individual account. Employees could change drafting, research and analysis behaviour without a use-case register, process baseline, data boundary or shared evaluation. Formal transformation programmes were therefore observing only part of the change.

The historical lesson is that visibility must precede confident scaling. Organisations need ways to discover embedded AI features and employee-built practices without criminalising exploration. Otherwise the official portfolio measures only centrally funded projects while material work changes elsewhere.

NIST AI RMF 1.0 framed governance as continuous and cross-cutting, supported by mapping, measurement and management. It emphasised context, affected people, human roles and lifecycle change. That structure undercuts the later idea that governance necessarily arrived as paperwork after innovation. ³

The framework did not prescribe one control set or prove that implementation would create financial value. It offered a vocabulary and outcome structure. Organisations still needed to translate it into owners, inventories, evaluation methods, monitoring, incident processes and decisions about acceptable risk.

Subsequent guidance made those operational artefacts more explicit. NIST’s AI Resource Center and ARIA pilot extended the operational focus toward contextual testing, evaluation, verification and validation; the generative AI profile added technology-specific risks and actions. ⁹¹⁰¹⁹

The retrospective conclusion is not that every firm should have implemented a mature control plane in 2023. It is that experimental velocity and governance did not need to be sequential. A lightweight register, named owner, scoped data boundary, task evaluation and stop rule were already compatible with rapid learning.

Early labour analysis attempted to map which occupations and tasks could be affected by generative systems. Exposure measures are useful for identifying where work may change, but they are not observed job-loss statistics. They generally do not include adoption cost, demand response, organisational redesign, regulation or the creation of complementary tasks. ⁶⁸

The Turing Transformation offered a second mechanism: automation of some tasks can raise the value of complementary skills and expand the pool of people able to perform other work. That possibility makes job-level forecasts dependent on how tasks are recombined, how demand changes and whether workers can move into the complementary work. ⁴

The ILO’s later refined index retained the transformation emphasis. It estimated one in four workers was in an occupation with some exposure, while 3.3% of global employment sat in the highest exposure category. Its method combined almost 30,000 task definitions, worker assessments, expert deliberation and model predictions; it measured technical exposure, not realised displacement. Stanford administrative payroll analysis later found a relative decline among early-career workers in highly exposed occupations, but remained observational and sensitive to timing and controls. ¹⁴¹⁷

A responsible workforce strategy should therefore separate four questions: which tasks are technically exposed, which are economically attractive to change, which changes are organisationally feasible, and what actually happens to jobs and people. Collapsing them produces false precision and weak decisions.

A customer-support deployment across 5,172 agents later reported a 15% average increase in successful issue resolution per hour. Gains were concentrated among less experienced workers, consistent with AI transferring elements of stronger practice. The setting, system vintage and workflow remained specific, but the operational dataset was far richer than a laboratory task. ¹¹

A randomised six-month experiment involving 7,137 knowledge workers across 66 firms found active users spent about two fewer hours on email each week. It did not detect broader shifts in task quantity or composition from individual provision. That result supports a distinction between personal efficiency and coordinated workflow change. ¹²

An RCT with experienced open-source developers reported that participants took longer when AI tools were available despite expecting the opposite. The sample was small and specialised, and a later methodology update warned that newer-tool estimates were unreliable because of selection and measurement problems. The correct lesson is not that AI slows software work universally; it is that task and worker context can reverse the sign of an effect. ¹³

Together these studies strengthen the 2023 frontier thesis. AI can create material gains, but the size and direction depend on task fit, experience, use pattern and surrounding process. The evidence base became broader without becoming a universal coefficient.

The expectation that model capability would improve rapidly held, as later AI Index synthesis documented across multiple capability measures. The expectation that rapid capability would automatically create deep firm adoption did not. Official indicators through 2025 continued to show large gaps by firm size, sector and digital maturity. OECD analysis emphasised connectivity, data, skills and finance as complementary enablers for SMEs. ⁷¹⁵

The expectation that knowledge work would change held, but the pathway was not a clean substitution curve. Field evidence showed augmentation, compressed learning, personal time savings, no detected coordination change, and negative effects in a specialised setting. The distribution mattered more than the average.

The expectation that governance would need to become operational also held. Later standards and guidance converged on named accountability, impact assessment, testing, monitoring and response. ¹⁹

The revised Institute thesis is that model capability is an input; the value-producing unit is a configuration of task, people, model, context, controls and capacity allocation. Two organisations using the same model can produce different outcomes because they select different tasks, supply different context, distribute authority differently and act differently on saved capacity.

This framing preserves the jagged frontier but makes it operational. Task fit asks whether the system performs under representative conditions. Human fit asks who benefits, who is deskilled and who can detect failure. Workflow fit asks how inputs, hand-offs, review and demand change. Control fit asks whether consequence is bounded and visible. Economic fit asks whether realised outcomes exceed full cost.

The configuration should be versioned. A model update, new data source, changed prompt, expanded tool right, different worker population or new consequence can invalidate earlier evidence. Approval should therefore attach to a defined configuration and evidence window rather than to an abstract product name.

This is a stricter interpretation of the 2023 evidence, not a rejection of it. The early experiment remains valuable precisely because it showed both gains and a boundary. Later studies make the boundary more important, not less.

Start with a named outcome and a representative work sample. Record the baseline distribution, including cycle time, quality, exceptions, rework and consequence—not only the mean. Predefine the minimum improvement, quality floor and stop conditions before teams see the result.

Test at least three configurations where practical: current work, assisted work and redesigned workflow. Segment results by experience and task type. Observe how people actually use the system, because a licence assignment does not specify the human–AI treatment.

Carry the result into operations with a time-limited authority boundary. Monitor the same outcome and quality measures, plus incidents, review burden and capacity destination. Reassess after model, data, prompt, integration, population or workflow change.

For portfolio decisions, distinguish explore, embed, expand, hold and retire. Exploration needs learning value. Embedding needs repeatable workflow evidence. Expansion needs control and outcome evidence. Holding is appropriate when uncertainty remains material. Retirement is a positive decision when economics or consequence do not justify further work.

Ask which outcome is expected to change, which population and work are in scope, and what current performance distribution forms the baseline. Ask what evidence shows the task sits inside the tested frontier and what conditions place it outside.

Ask who benefits, who reviews, who can recognise failure and how learning pathways change. Ask what data, model, prompts, tools and authority define the approved configuration, and which changes trigger re-evaluation.

Ask where saved capacity goes, how full cost is measured and which quality or risk limits cannot be traded away. Ask what the strongest counterevidence says and whether the team tested the conditions under which the result may reverse.

Finally, ask what would cause the organisation to stop. A portfolio without hold and retire decisions is accumulating belief, not evidence. The enduring lesson from 2023 is that speed of capability raises the value of disciplined learning.