From AI prototype to production: a checklist for Québec teams

1. Confirm the product decision before improving the model

State who will use the capability, what decision or task it changes, and what observable improvement would make the system worthwhile. “Use generative AI” is not a product outcome. “Reduce the time required to find, compare, and act on approved client documents while preserving source citations” is testable.

Compare the AI-enabled workflow with simpler options. Better search, deterministic rules, process redesign, an existing product, or a smaller integration may solve the problem with lower operating risk. A production plan should explain why AI is justified and what would cause the team to change course.

• Named user and job to be done
• Current workflow and failure points
• Target behavior and measurable usefulness
• Non-AI and buy alternatives considered
• Adoption owner and operational change required

2. Draw the data and permission boundary

List every data source the system reads, generates, stores, or sends to a model provider. Classify who may access each source, how tenant or client boundaries are enforced, what leaves the organization, and how long prompts, outputs, logs, and evaluation records remain available.

In Québec, systems involving personal information may trigger privacy-policy, governance, consent, or privacy-impact assessment obligations depending on the facts. Resolve the technical data flow early enough for qualified privacy and legal review to influence the design—not after tracking and model calls are already enabled.

• Source, owner, sensitivity, and permitted use
• Identity, role, workspace, and tenant enforcement
• Provider retention and training settings
• Regional processing and subcontractor path
• Deletion, correction, export, and audit behavior
• Explicit prohibition on sending inquiry or client data to unapproved AI services

3. Build an evaluation around the real workflow

A handful of impressive prompts is not an evaluation. Collect representative tasks, difficult cases, expected source material, unacceptable outcomes, and the conditions under which a human must review or take over. Separate model quality from retrieval quality, tool execution, application logic, and interface behavior so a failure can be diagnosed.

The useful metric depends on the job. It may include groundedness, citation accuracy, classification precision and recall, completion rate, review time, escalation rate, or task success. Define the minimum acceptable threshold and the test owner before release.

4. Design failure as a normal state

Models time out, providers throttle requests, source material becomes stale, retrieval returns the wrong context, tools fail, and outputs can be confident but wrong. The product needs explicit behavior for each condition: retry, fallback, partial result, refusal, human review, or a safe deterministic path.

Do not hide uncertainty behind polished prose. Show sources where the answer depends on controlled knowledge. Make destructive or consequential actions confirmable. Limit tool permissions. Preserve an inspectable record when a business process requires one.

5. Integrate the surrounding product system

Production value usually depends more on integration than on the prompt. Identity, permissions, workflow state, business rules, notifications, data writes, audit history, accessibility, and mobile behavior determine whether people can use the capability responsibly.

Treat the AI feature as one component in an application architecture. Document the contracts around it. Keep deterministic validation after generation when the output must meet a schema or business rule. Use asynchronous processing when latency would otherwise block a workflow.

6. Establish the operating budget

Estimate cost from expected volume, model mix, token or media size, retrieval calls, tool execution, retries, caching, and evaluation. Add service limits and alerts before traffic arrives. A capability that is affordable in a demo can become uneconomic when every user action creates several model and tool calls.

Track latency, success, provider errors, fallback, evaluation results, and cost per useful task. Avoid storing prompts or outputs by default merely because an observability tool can accept them; logging must respect the data boundary.

7. Assign production ownership and release deliberately

Name the owner for model and provider changes, evaluation maintenance, incidents, data-source quality, support, privacy questions, and user feedback. Document how the capability is disabled safely and who can make that decision.

Begin with a controlled group whose workflow and feedback can be observed. Compare against the prior process. Expand only when the evidence supports it. The milestone is not “the feature shipped”; it is “the system has a responsible operating loop.”

• Release and rollback plan
• Monitoring and alert thresholds
• Named technical and product owners
• Support and escalation path
• Evaluation regression schedule
• User feedback and change process
• Decision date for expansion, redesign, or retirement

The production gate

A team should be able to answer four questions before launch: What useful outcome does the system create? What data and authority can it access? How do we know when it succeeds or fails? Who operates and improves it after release?

If those answers remain vague, the initiative is still a prototype—regardless of how convincing the demonstration looks.