If your health organization is considering AI-assisted development for a digital health tool, a faster build does not automatically mean a faster route to launch. Privacy, security, and procurement reviews still need evidence, and that evidence does not assemble itself because the code was written quickly.
There is a version of this conversation happening in health organizations across Canada right now. A program manager or clinical lead has been told to build something, a patient portal, a referral intake form, a care coordination tool, a decision aid, and their vendor or internal team has mentioned that they use AI to build faster. The timeline sounds reasonable. The scope sounds manageable. What the pitch often leaves out is that the code is only one part of getting a digital health tool into production. The privacy impact assessment, the vendor agreements, the data residency review, and the IT security sign-off: none of those are created by the build. A system that is technically ready, but governance-incomplete does not move through procurement any faster than one that is still being written. It just arrives at the problem sooner.
The speed claim is probably accurate. Accountability hasn’t changed.
AI-assisted development can meaningfully compress timelines for certain types of digital health tools. A vendor who offers a compressed timeline compared to a traditional build often describing the code-writing phase accurately.
What does not compress is the governance work that sits around the code.
Before any new system that handles patient data can go live at a Canadian health organization, the same questions apply whether the code was written by AI or by hand. Where does personal health information live and travel? Who has agreed, in writing, to handle it appropriately? How are changes to the system reviewed and controlled? Is there a privacy impact assessment?
These are not new requirements invented for AI. They are the existing requirements that apply to any new digital health system, and they do not complete themselves just because the build phase was fast.
The practical risk is this: a vendor delivers quickly, the team is pleased with the result, and the system moves toward procurement or launch, and the privacy office or IT security team runs
their standard review and finds that the governance documentation was never assembled. At that point the speed advantage has become a problem. The code is ready, and the evidence is not.
Three questions to ask a vendor who says they build with AI
The following questions are not a test designed to trip a vendor up. They are questions that a health system privacy office will eventually ask, and the answers need to exist before procurement, not during it.
1. Where does data go during the build?
AI coding assistants send code and surrounding context to a model as part of how they work. That context often includes test data, database structure, and code comments. If a developer working on a patient intake form has a real patient record in their development environment when the AI tool is running, that data has traveled somewhere outside your organization.
Ask the vendor directly: what AI tools are used in development, what data reaches those tools, and how do they ensure that personal health information from your organization never enters a development or test environment? The right answer is specific and covers two things. First, the vendor uses an enterprise agreement with their AI vendor that includes zero data retention and no model training on customer inputs. Second, the development environment uses synthetic or de-identified data so that no real patient records exist there in the first place. A vendor who cannot describe both of those controls is describing a gap, not a process.
2. What does the agreement cover?
Under Canadian health privacy legislation, including PHIPA in Ontario, HIA in Alberta, and PIPEDA where provincial legislation does not apply, accountability for personal health information stays with the custodian, or trustee depending on your province, even after the data is handled by a vendor. The written instrument that governs the vendor relationship is typically an information manager agreement or a data sharing agreement, depending on your province.
The written instrument that governs the vendor relationship varies by province. In Ontario under PHIPA, the applicable instrument is typically an agreement with an agent, an electronic service provider, or a health information network provider, depending on the vendor’s role. In Alberta under HIA, it is an information manager agreement; where a vendor operates outside Alberta, that agreement must carry prescribed safeguards, which is directly relevant to the data residency point below. In other jurisdictions, equivalent instruments apply.
A vendor’s standard service terms and a vendor’s privacy policy are not the same thing as the applicable provincial instrument. Ask which agreement covers the engagement, confirm it names the applicable legislation, and confirm it prohibits the vendor from using your data for model training.
Also ask where data is stored. Hospital procurement policies and health privacy legislation in several provinces include data residency expectations. In Quebec, Law 25 requires a privacy impact assessment before personal information is communicated outside the province, which means the PIA has to be underway before any vendor arrangement is finalized, not after. A vendor whose subprocessors store data outside Canada, or outside Quebec for organizations subject to Law 25, can quietly fail a procurement review even if the product itself is well built.
3. Who is on the review record?
When a health system asks how changes to a patient-data system are controlled and who approved them, the answer needs to be a named person, not a process.
Ask the vendor how AI-generated code is reviewed before it is deployed. The answer you are looking for includes a named engineer who approved every change, a record that can be produced, and stricter controls for code that touches patient data or access management. An answer that amounts to “we reviewed it as a team” or “the AI checks for errors” will not satisfy a health authority’s IT security review.
What “AI-assisted” means in a regulated context
It helps to understand what the term actually covers, because vendors use it to mean different things, and the distinction matters when you are evaluating a proposal.
In a well-run AI-assisted build, engineers use AI tools to accelerate specific parts of the work. Writing test scaffolding and boilerplate code is one area where AI tooling is genuinely useful: generating a first draft of unit tests, producing repetitive data-handling functions, and drafting standard API integrations can all move faster with AI assistance than without it. Drafting initial documentation, summarizing requirements, and producing code comments are other areas where the tools add speed without adding risk, provided no real patient data is in the environment.
What AI tooling does not replace is the judgment that a project requires. Architecture decisions, access control design, integration planning for systems that hold patient data, and the logic that governs what a user can see or do in a clinical context: these require engineers who understand the specific system, the specific data, and the specific regulatory environment. AI can produce a plausible-looking answer to any of these, but plausible and correct are not the same thing in a regulated build, and the difference is the engineer’s responsibility to catch.
The accountability structure in a well-run AI-assisted build is that engineers own the architecture, review every change before it is deployed, and are answerable for what ships. The AI accelerates the work; it does not replace the judgment or accountability.
What this is not: prompting an application into existence and shipping the output. A system assembled in that way is unlikely to meet the security, documentation, and change-control standards a health authority review process applies.
It also differs from building an AI product, a system where a language model is what the product delivers, such as a clinical decision support tool or a documentation assistant. That category involves additional regulatory considerations, including Health Canada’s guidance on AI and ML-enabled software as a medical device, that fall outside the scope of this piece.
When a vendor says “AI-assisted,” it is reasonable to ask what the phrase means for their process specifically: which tools, what oversight, who is accountable for the review record.
The procurement gap
There is a pattern worth naming explicitly because it is predictable and avoidable.
A digital health tool can be technically functional, working features, reasonable performance, no obvious defects, and still fail a health authority’s IT security review or a certification process like OntarioMD because the governance documentation was not assembled during the build.
The procurement gap is the distance between “the code works” and “the evidence exists.” It includes the privacy impact assessment, the information manager agreements, the data flow documentation, the change control records, and the test environment policy. None of these are created automatically by an AI tool. All of them take time that should be planned alongside the build, not discovered after delivery.
A vendor who offers a fast timeline without a corresponding governance plan is not offering a complete solution. The missing part will surface during procurement, and fixing it after the build is more expensive than including it from the start.
A 30-day readiness checklist before commissioning an AI-assisted build
Before signing a statement of work for a digital health tool that involves personal health information, a health system team should be able to work through the following:
- Data maps drafted. You have documented what personal health information the system will hold, where it will travel, in which province or country it will be stored, and which vendors will touch it. This document is the foundation of the privacy impact assessment.
- AI tools in the build are identified. The vendor has named which AI tools will be used in development and confirmed the process for keeping real patient data out of those tools.
- Agreements reviewed. You have confirmed which agreement governs for each vendor that will touch patient data, verified that each agreement names the applicable provincial health privacy legislation, and confirmed data residency requirements are met.
- Review process confirmed. The vendor has described how AI-generated code is reviewed before deployment, who approves changes, and what records will exist. This information should be written.
- Privacy impact assessments started. Your privacy office has been engaged, and the PIA is underway as a living document, not scheduled for after the build is complete. Most provincial health privacy frameworks expect a PIA before a new system goes live or enters procurement. In Alberta, the obligation is more specific: a custodian must prepare a PIA and submit it to the Office of the Information and Privacy Commissioner before implementing a new information system.
- IT security and procurement requirements are confirmed. You have spoken with your IT security team and confirmed what review the finished system will need to pass. If OntarioMD EMR certification (which applies where the tool connects to provincial EHR services such as OLIS or HRM), a provincial clinical viewer integration, or a health authority security assessment is in scope, those requirements should be known at the start of the project, not at the end.
Conclusion
AI-assisted development offers real benefits for digital health projects, and there is no governance reason to avoid it. There is a governance reason to ask the right questions before the build starts.
The questions are not new. They are the questions that have always applied to new digital health systems in Canada: where does patient data go, who has agreed to handle it appropriately, and how are changes reviewed and documented. What changes with AI-assisted builds is the pace, and the pace creates a risk that governance work falls behind the code.
Before signing a statement of work, the right people to have in the room are the project lead, the privacy office, IT security, and whoever owns procurement sign-off at your organization. Not after the vendor has delivered. Before the engagement begins. Those four conversations
will surface most of what the checklist above is asking for and having them early is the practical difference between a system that clears procurement and one that stalls there.
For a more detailed look at the compliance decisions that arise during a health product build, including vendor agreement requirements, data leak patterns, and code review practices, see Cabot’s guide to AI-assisted MVP development.