Skip to content
From AI proof of concept to production: five gates that matter
← Lab · Strategy · Production

From AI proof of conceptto productionfive gates that matter

Journey
5 decision gates
Evidence
Lifecycle TEVV
Decision
Go · Conditional · No-go
Operations
Owner + rollback

An AI proof of concept shows that an interaction can work. Production requires stronger evidence: a bounded purpose, representative conditions, measurable thresholds, an operating owner and a safe way to stop. These five decision gates turn enthusiasm into an accountable deployment choice.

In brief

  • 01A polished demo validates a possibility; it does not prove that the system is ready for real users, real data or real failure modes.
  • 02Each gate needs observable evidence, an accountable owner and a decision threshold defined before the result is known.
  • 03Production readiness includes monitoring, human takeover, incident response, rollback and retirement — not only model quality.
Do not ask whether the demo is impressive. Ask what evidence would make deployment responsible.
Production principle
A production decision compares traceable evidence, assumptions and exceptions rather than relying on one polished interface.
A production decision compares traceable evidence, assumptions and exceptions rather than relying on one polished interface.

A demo, a pilot and a production service answer different questions

A proof of concept can deliberately simplify data, integrations and edge cases. That is useful when the learning objective is explicit. The problem begins when this limited evidence is treated as evidence about an operating service.

  1. 01
    Proof of concept

    Can the proposed interaction or technical mechanism work under controlled conditions?

  2. 02
    Pilot

    Does it remain useful with representative users, integrations, permissions and operating constraints?

  3. 03
    Production

    Can the organisation operate, monitor, recover, improve and eventually retire it within agreed risk and cost limits?

Five gates between a POC and production

FIVE DECISION GATES
  1. 011. Value & boundary ›Name one operational journey, the people affected, the intended outcome and what remains out of scope. Assign a business owner and define the baseline the system must improve.
  2. 022. Data & integration ›Test representative inputs, identity, permissions, source freshness and downstream systems. Include dependency failures, realistic latency and expected volume instead of assuming every API will respond correctly.
  3. 033. Quality & risk ›Set acceptance thresholds for task completion, known failure detection, unsupported answers, human transfer, privacy, security and accessibility. Exercise ambiguity and misuse, not only happy paths.
  4. 044. Operations & economics ›Name the service owner, support path and budget. Confirm monitoring, incident handling, supplier changes, model or prompt updates, recurring costs and the procedure for degraded operation.
  5. 055. Decision evidence ›Assemble results, unresolved risks, mitigations and ownership in one decision record. Choose go, conditional go or no-go against criteria agreed before the final review.
Exceptions and warnings are decision evidence; hiding them behind an average score weakens the production case.
Exceptions and warnings are decision evidence; hiding them behind an average score weakens the production case.

Measure the service, not only the model

There is no universal readiness score. Thresholds depend on the journey, the consequence of failure and the organisation’s risk tolerance. The useful metric set connects technical behaviour to the work people actually need to complete.

  1. 01
    Task success

    Did the user reach the verified outcome, rather than merely receive a fluent response?

  2. 02
    Failure detection

    Did the system recognise missing evidence, stale data, low confidence or an unavailable tool before acting?

  3. 03
    Handoff quality

    Was the case transferred at the right moment with usable context and without forcing the person to restart?

  4. 04
    Service performance

    Track latency, availability, error rate, recovery time and cost per completed journey under representative conditions.

  5. 05
    Operational effect

    Measure operator workload, rework, escalations and the agreed business outcome alongside user experience.

TEVV must resemble the deployment context

Test, evaluation, verification and validation — TEVV — is not a single test at the end of development. NIST describes TEVV tasks across design, development, deployment and operation. For a production decision, the scenario set must represent the intended environment and be repeated after material changes.

  • 01Normal journeys, incomplete requests, conflicting information and out-of-scope attempts.
  • 02Unavailable tools, stale sources, revoked permissions, slow dependencies and interrupted sessions.
  • 03Variation in language, phrasing, noise, device, accessibility needs and user behaviour where relevant.
  • 04Production-like load, integration state, retention rules and human handoff conditions.
  • 05Regression tests after changes to models, prompts, knowledge, policies, tools or suppliers.
A system is not production-ready because it passed once. It needs a repeatable way to show that the relevant controls still work.
Evidence principle

Ownership, rollback and shutdown are product features

  1. 01
    Named owner

    One accountable role accepts the service outcome and coordinates technical, operational, legal and domain contributors.

  2. 02
    Observable operation

    Dashboards, traces and alerts reveal task failures, policy breaches, handoff problems, cost changes and dependency health.

  3. 03
    Controlled change

    Material updates have versioning, review, regression evidence and a documented release decision.

  4. 04
    Rollback & continuity

    The team can reduce autonomy, restore a known configuration, switch to a tested fallback or pause the service without improvisation.

  5. 05
    Retirement

    A shutdown path covers data, access, integrations, records, user communication and remaining obligations.

Human review connects technical evidence to an accountable release, restriction or stop decision.
Human review connects technical evidence to an accountable release, restriction or stop decision.

A practical go/no-go record

DECISION RECORD
  1. 01GO ›The agreed thresholds are met, residual risks have owners and monitoring, support and rollback are ready for the approved scope.
  2. 02CONDITIONAL GO ›A bounded release can proceed with explicit restrictions, additional supervision, an expiry date and evidence required before expansion.
  3. 03NO-GO ›A critical threshold is missed, an essential control has no owner, or the remaining uncertainty is incompatible with the intended use.
  4. 04NEXT REVIEW ›Record what will be tested, who owns it, which evidence is expected and when the decision will be revisited.

How the NIST references fit

The five-gate model is a Banana Navy delivery pattern, not a certification scheme. It translates three useful NIST ideas into a compact production decision: govern, map, measure and manage risk across the lifecycle; test generative AI before deployment and monitor it afterwards; and assign deployment, operation, monitoring and TEVV tasks to explicit actors.

Continue the production journey

Engineering guidance, not a certification, security audit or legal opinion. NIST references checked on 10 November 2026; acceptance thresholds and controls must be adapted to the intended use, applicable obligations and risk tolerance.

Turn the next demo into an evidence-based production decision.

Next article
Voice AI in healthcare: useful scope, hard limits and human handoff →

Let's talk

Email
marc@banana-navy.com
Phone
+32 495 27 70 44
Studio
Banana Navy SRL
Rue Antoine de Saint-Exupéry 2
6041 Charleroi (Gosselies), BelgiumVAT BE 0745.575.058
Call our Voicebots