Artificial intelligence may already be influencing regulated processes within your organization, even where a system or function is not explicitly described as AI. Understanding where it is used, how it works and which decisions it supports is an essential first step toward applying the right controls.
AI and the changing GMP landscape
The publication of draft Annex 11 together with draft Annex 22 has put AI at the center of many compliance discussions. Model training, data quality, human oversight, and ongoing performance monitoring are areas that traditional computerized system validation did not address with the same model-specific focus.
Draft Annex 11 continues to address the wider computerized-system lifecycle, while draft Annex 22 introduces additional expectations for certain AI and machine-learning models used in critical GMP applications. Understanding how the two work together is important. But first, organizations need to understand what kind of technology they are using.
Written logic or trained model?
But there is a distinction that matters.
Life sciences companies have used automated decision-making systems for many years. Some systems called “AI” today would have been called advanced analytics, expert systems, chemometrics or automated decision logic in the past.
In traditionally programmed systems, the logic is written by a person. It can be read, locked and validated with the methods we already have. These systems generally remain subject to established computerized-system controls under Annex 11 rather than the model-specific requirements introduced in draft Annex 22.
A trained machine-learning model is different. Its logic is learned from data rather than written. Its behavior can degrade as the data it sees changes. Its performance therefore depends not only on the system and its configuration, but also on the training data, test data, intended use and operating environment.
Under the current draft, Annex 22 applies to data-trained, static models with deterministic outputs when they are used in critical GMP applications. Dynamic models, probabilistic models, generative AI and large language models fall outside their scope and should not be used in critical GMP applications.
It also asks for controls such as independent test data, defined acceptance criteria and confidence thresholds, and explainability. The model’s performance must also be monitored during operation to identify degradation or changes in the input data.
How Annex 11 and Annex 22 work together
Annex 11 covers questions related to lifecycle, data integrity, access, audit trails and supplier oversight which apply to AI as part of the computerized system in which it operates. Draft Annex 22 adds the controls needed for the trained model itself.
This means that validating an AI-enabled solution is not only about testing the model. Organizations must consider the complete system, its intended use, the process in which it operates and the decisions its outputs may influence.
The applicable validation approach should therefore reflect:
- The intended use and GMP criticality
- Whether the functionality is explicitly programmed or learned from data
- Whether the model is static or continues to adapt
- Whether identical inputs produce identical outputs
- The role of human review and decision-making
- The supplier’s responsibility for model development, testing and updates
- How performance and data drift will be monitored during operation
A practical first step: map where AI is already used
A practical first step is to create an inventory of where AI or AI-like functionality is already used across your GMP processes.
This review should include both internally developed solutions and functionality embedded within third-party software. A familiar user interface may conceal a model that has been introduced or updated by the supplier.
Ask:
- Where do we already use AI or AI-like functionality?
- Where do vendors use AI within tools my organization relies on?
- Where do automated outputs influence GMP decisions?
- Which systems use trained models rather than written logic, and therefore may require Annex 22 controls?
- Who reviews the outputs, and how is that documented?
- How are model changes, performance degradation, and data drift identified?
The inventory should record more than the name of the tool. It should capture the intended use, system owner, supplier, model type, version, GMP relevance, and the decisions supported or automated by the system.
Once this information is available, each system can be classified according to its risk and the controls it requires. This helps organizations focus validation effort on the applications with the greatest potential impact on patient safety, product quality, and data integrity.
The real risk is not always visible
The biggest risk is not AI itself. It is relying on automated outputs without understanding what produced them and which controls should apply.
AI functionality may be introduced through a software update, added to an existing platform or described by a supplier using broad terms such as intelligent automation or advanced analytics. The name of the technology is less important than how it works and how its output is used.
Organizations therefore need visibility across the complete decision pathway: the data entering the system, the model or logic processing it, the resulting output and the person or process responsible for the final GMP decision.
How KVALITO can support
At KVALITO, we support life sciences organizations in assessing and validating AI-enabled solutions for regulated environments. This includes identifying the applicable risks, defining the right controls, and building validation approaches that reflect how the system works in practice and the context in which it is used.
Our support can include:
- AI system and use-case inventories
- GMP impact and risk assessments
- AI readiness and gap assessments
- Supplier and embedded-AI evaluations
- Definition of intended use and acceptance criteria
- Risk-based validation strategies
- Lifecycle governance and performance-monitoring approaches
What comes next
This article is the first in our AI series. In the next articles, we will explore AI validation, the practical implications of draft Annex 11 and Annex 22, and AI use cases across life sciences.
Do you know where AI is already influencing GMP decisions in your organization? KVALITO can help you map your current landscape, understand the applicable risks, and define an appropriate validation approach.
References
- European Commission: Draft Annex 11—Computerised Systems
- European Commission: Draft Annex 22—Artificial Intelligence
- ISPE, GAMP® 5: A Risk-Based Approach to Compliant GxP Computerized Systems, Second Edition.
