
An AI proof of concept shows that an interaction can work. Production requires stronger evidence: a bounded purpose, representative conditions, measurable thresholds, an operating owner and a safe way to stop. These five decision gates turn enthusiasm into an accountable deployment choice.
In brief
- 01A polished demo validates a possibility; it does not prove that the system is ready for real users, real data or real failure modes.
- 02Each gate needs observable evidence, an accountable owner and a decision threshold defined before the result is known.
- 03Production readiness includes monitoring, human takeover, incident response, rollback and retirement — not only model quality.
Do not ask whether the demo is impressive. Ask what evidence would make deployment responsible.Production principle

A demo, a pilot and a production service answer different questions
A proof of concept can deliberately simplify data, integrations and edge cases. That is useful when the learning objective is explicit. The problem begins when this limited evidence is treated as evidence about an operating service.
- 01Proof of concept
Can the proposed interaction or technical mechanism work under controlled conditions?
- 02Pilot
Does it remain useful with representative users, integrations, permissions and operating constraints?
- 03Production
Can the organisation operate, monitor, recover, improve and eventually retire it within agreed risk and cost limits?
Five gates between a POC and production
- 011. Value & boundary ›Name one operational journey, the people affected, the intended outcome and what remains out of scope. Assign a business owner and define the baseline the system must improve.
- 022. Data & integration ›Test representative inputs, identity, permissions, source freshness and downstream systems. Include dependency failures, realistic latency and expected volume instead of assuming every API will respond correctly.
- 033. Quality & risk ›Set acceptance thresholds for task completion, known failure detection, unsupported answers, human transfer, privacy, security and accessibility. Exercise ambiguity and misuse, not only happy paths.
- 044. Operations & economics ›Name the service owner, support path and budget. Confirm monitoring, incident handling, supplier changes, model or prompt updates, recurring costs and the procedure for degraded operation.
- 055. Decision evidence ›Assemble results, unresolved risks, mitigations and ownership in one decision record. Choose go, conditional go or no-go against criteria agreed before the final review.

Measure the service, not only the model
There is no universal readiness score. Thresholds depend on the journey, the consequence of failure and the organisation’s risk tolerance. The useful metric set connects technical behaviour to the work people actually need to complete.
- 01Task success
Did the user reach the verified outcome, rather than merely receive a fluent response?
- 02Failure detection
Did the system recognise missing evidence, stale data, low confidence or an unavailable tool before acting?
- 03Handoff quality
Was the case transferred at the right moment with usable context and without forcing the person to restart?
- 04Service performance
Track latency, availability, error rate, recovery time and cost per completed journey under representative conditions.
- 05Operational effect
Measure operator workload, rework, escalations and the agreed business outcome alongside user experience.
TEVV must resemble the deployment context
Test, evaluation, verification and validation — TEVV — is not a single test at the end of development. NIST describes TEVV tasks across design, development, deployment and operation. For a production decision, the scenario set must represent the intended environment and be repeated after material changes.
- 01Normal journeys, incomplete requests, conflicting information and out-of-scope attempts.
- 02Unavailable tools, stale sources, revoked permissions, slow dependencies and interrupted sessions.
- 03Variation in language, phrasing, noise, device, accessibility needs and user behaviour where relevant.
- 04Production-like load, integration state, retention rules and human handoff conditions.
- 05Regression tests after changes to models, prompts, knowledge, policies, tools or suppliers.
A system is not production-ready because it passed once. It needs a repeatable way to show that the relevant controls still work.Evidence principle
Ownership, rollback and shutdown are product features
- 01Named owner
One accountable role accepts the service outcome and coordinates technical, operational, legal and domain contributors.
- 02Observable operation
Dashboards, traces and alerts reveal task failures, policy breaches, handoff problems, cost changes and dependency health.
- 03Controlled change
Material updates have versioning, review, regression evidence and a documented release decision.
- 04Rollback & continuity
The team can reduce autonomy, restore a known configuration, switch to a tested fallback or pause the service without improvisation.
- 05Retirement
A shutdown path covers data, access, integrations, records, user communication and remaining obligations.

A practical go/no-go record
- 01GO ›The agreed thresholds are met, residual risks have owners and monitoring, support and rollback are ready for the approved scope.
- 02CONDITIONAL GO ›A bounded release can proceed with explicit restrictions, additional supervision, an expiry date and evidence required before expansion.
- 03NO-GO ›A critical threshold is missed, an essential control has no owner, or the remaining uncertainty is incompatible with the intended use.
- 04NEXT REVIEW ›Record what will be tested, who owns it, which evidence is expected and when the decision will be revisited.
How the NIST references fit
The five-gate model is a Banana Navy delivery pattern, not a certification scheme. It translates three useful NIST ideas into a compact production decision: govern, map, measure and manage risk across the lifecycle; test generative AI before deployment and monitor it afterwards; and assign deployment, operation, monitoring and TEVV tasks to explicit actors.
Continue the production journey
Engineering guidance, not a certification, security audit or legal opinion. NIST references checked on 10 November 2026; acceptance thresholds and controls must be adapted to the intended use, applicable obligations and risk tolerance.
Turn the next demo into an evidence-based production decision.
Let's talk
- marc@banana-navy.com
- Phone
- +32 495 27 70 44
- Studio
- Banana Navy SRL
Rue Antoine de Saint-Exupéry 2
6041 Charleroi (Gosselies), BelgiumVAT BE 0745.575.058