Accuracy Alone Cannot Decide: Recovery Time for Production AI

Imagine a Friday afternoon review meeting debating whether to move a generative AI system with “92% average accuracy” into production. The number looks good enough. But no one has decided who will find the remaining 8% of errors, how long correction takes, or how many people’s decisions one missed error could affect.

Approving production in this state is premature. A generative AI pass criterion should depend not only on average accuracy but also on the time needed to “detect, contain, and restore” after an error. This article calls that period “recovery time.”

After expanding use comes operationalizing evaluation

NTT DATA announced that it provides an internal generative AI service for searching company information to all employees; a post-deployment survey found an average time saving of about 60 minutes per week and an 89% intention to continue using it. These are survey results published by the company, not independently verified effects. [1] Japan’s Digital Agency has operated its internal “Gennai” environment since May 2025. In fiscal 2026, it plans a large-scale trial across ministries for about 180,000 people to verify effects and issues, leading toward full deployment from fiscal 2027. [2]

The two cases point to a shift from asking whether a trial is possible to understanding benefits and issues while use continues. However, the existence of individual cases does not prove that your own business process will see the same effect. What is needed is an evaluation system that reflects your company’s decisions and exceptions.

On July 7, 2026, Japan’s AI Safety Institute (AISI) updated the “Guide to Evaluation Perspectives for AI Safety” to Version 1.20, adding “observation and control” for AI agents. The guide presents an approach to checking safety from development through operation and retirement. [3] Version 2.0 of the Digital Agency’s procurement and use guidelines also covers AI system logs, system audits, and responses to serious risk cases. [4]

Three costs average accuracy hides

The same accuracy can have different business value. Measure these three factors separately first.

Detection time — how long until an error is noticed

A response with a broken evidence link may be noticed immediately. A wrong condition in a proposal sent to a customer might not surface until a reply the next day. The later an error is detected, the more likely it is to contaminate subsequent decisions and deliverables. Evaluation sets should record not only right or wrong but also “who can notice and from what signal.”

Impact scope — how far has the error spread by discovery?

An error in one draft differs from one entered into shared knowledge. For external sending, publication, ledger updates, or triggering downstream processing, count the number of cases, affected people, and reversibility separately. If these are collapsed into one average score, rare but high-loss failures disappear.

Recovery time — how long to restore a correct state

Time to write a correction is not enough. Recovery includes finding the original input, verifying facts, identifying affected destinations, reapproval, resending, and correcting records. NIST’s AI Risk Management Framework includes incident response time among safety evaluation metrics, alongside continuous testing before and during operation. [5]

The decision formula need not be complex. For each error type, calculate “number of occurrences × recovery time per occurrence × required expert rate,” and review missed-error losses separately. Even if accuracy improves, operating costs do not fall if verification work is concentrated on expensive specialists.

Make the evaluation set a “decision ledger,” not just a question list

A weak approach is to list only clean questions and model answers. Production includes missing inputs, notation variations, old attachments, insufficient permissions, and exceptional customer conditions. An evaluation set built only by a technical team tends to miss boundary cases that business staff handle daily.

OpenAI recommends defining the objective, critical decision points, desired outputs, and outputs to avoid with experts for business-specific evaluations, then measuring under conditions close to real situations. It also describes continuously evaluating outputs from real inputs after launch and routing ambiguous or high-cost cases to experts. This is provider guidance, not a performance guarantee. [6]

A decision ledger should retain the input, expected handling, acceptable variation, error type, detector, detection time, affected destinations, recovery procedure, and recovery time. If a large language model (LLM) assists with scoring, business experts should still periodically audit serious failures and boundary cases. Automated scoring alone can miss defects in the scoring criteria themselves.

Use FDE to make production decisions in three stages

In Forward Deployed Engineering (FDE), evaluation is not left until the end of development. The user department and development team review the same decision ledger and expand scope in this order.

Stage one is shadow operation: do not apply AI outputs to the business process; compare them with human handling. Here, verifying that error types and recovery steps can be recorded matters more than accuracy. Stage two is limited operation: narrow the department, volume, and input formats, and set limits for detection and recovery time. Stage three updates the evaluation set from real logs and moves only errors recoverable within the limit to automated handling.

Having a person check every case is useful at the beginning, but with growing volume, the review queue becomes a new bottleneck. Random sampling alone misses failures that are infrequent but costly. A practical approach combines sample audits for ordinary cases with full review of high-loss conditions.

A sophisticated automated evaluation platform is unsuitable when case volume is low, error impact is limited, and staff can quickly check every case as part of normal work. A shared ledger and regular review are enough. Conversely, for legal, medical, hiring, or credit judgments—where errors have serious consequences and individual decisions by qualified experts cannot be removed—AI should not automatically finalize decisions, even if recovery time is short.

Practical implications

Recommendation 1: Add three time measures to the production decision table. Alongside accuracy, record time to detect, time to contain, and time to complete recovery.

Recommendation 2: Separate high-loss failures from the average. Actions involving external communication, publication, money, contracts, or personnel need separate pass criteria even when infrequent.

Recommendation 3: Feed real logs back into the evaluation set. Exceptions found in limited operation must be included in the next release decision. Do not update only the model while leaving the evaluation set unchanged.

Putting generative AI into production does not mean waiting for errors to disappear. It means proving that errors can be detected, prevented from spreading, and returned to a correct business state within an operationally manageable time. Average accuracy is the entry point. What should be approved is the business process, including a recoverable system.

References

1. NTT DATA, “Developed and deployed internally at NTT DATA! What is the internal generative AI service that boosts employee productivity?” (2025-01-06)
2. Japan Digital Agency, “Government AI ‘Gennai’” (checked 2026-09-21)
3. AI Safety Institute, “Guide to Evaluation Perspectives for AI Safety (Version 1.20)” (2026-07-07)
4. Japan Digital Agency, “Guidelines for Procuring and Using Generative AI to Advance and Innovate Government (Version 2.0)” (2026-06-12)
5. NIST, “AI Risk Management Framework Core” (AI RMF 1.0; checked 2026-09-21)
6. OpenAI, “How evals drive the next chapter in AI for businesses” (2025-11-19)

This article was created with AI assistance and automatically reviewed before publication. Information checked on: 2026-09-21.