KVALITO recently supported its pharmaceutical client in implementing and validating their Bayesian Multilevel Monitoring System. The Bayesian Multilevel Monitoring system monitors the safety of clinical trials in Oncology Biometrics and Data Management, Global Oncology Development, and Drug Safety and Epidemiology.
The system offers an efficient approach to monitoring safety, with the ability to detect potential signals of safety issues early in the development process.
The Bayesian Multilevel Monitoring System operates within our client’s modelling and simulation environment. The modelling and simulation environment provides the necessary infrastructure to support the Bayesian Multilevel Monitoring System.
The modelling and simulation environment is a specialised computing environment specifically designed for high-performance computing. This environment is used to conduct statistical modelling and simulation and execute models. It is a qualified and controlled environment that is dedicated to this purpose.
Challenge
The challenge is validating the Bayesian Multilevel Monitoring System and ensuring that the system meets the necessary regulatory requirements and performs as intended while also being able to detect potential safety signals promptly.
What we did
To validate the Bayesian Multilevel Monitoring System, our lead consultant performed an initial Risk Assessment to determine the relevant regulatory requirements and identify any potential risks associated with the system. Based on the results of the Risk Assessment, a validation strategy and plan were developed that outlined the testing and documentation requirements for the system. The validation activities included the creation of a User Requirements Specification (URS), a business process and functional risk assessment, a code review, functional and acceptance testing, and a validation report. Specific controls were established to comply with 21 CRF Part 11 and EU GMP Annex 11. Additionally, a system and application management handbook, business procedures (SOPs), and end-user training were established to ensure the continued maintenance and compliance of the system.
People, Processes and Tools
Project Roles
The validation project was led by our Lead Life Science Consultant, Computerized System Validation Expert, with participation from client representatives such as but not limited to the business process owner, system owner, quality assurance, statistical subject matter experts and developers.
Business Processes and Intended System Usage
The system analyses efficacy and safety data from clinical trials, supporting research, analysis, monitoring and prediction. Data for the system can be sourced from either the modelling and simulation environment or the client Global Programming and Statistics business unit. Results generated by the system can be used for internal business decision-making or supporting regulatory submissions such as periodic safety update reports, annual safety reports, clinical trial reports and other reports.
The Bayesian Multilevel Monitoring System is intended for analysis only and does not handle the creation, modification, maintenance, archival, retrieval, or transmission of electronic records required by GxP regulations. However, the system’s output can be used for GxP purposes. Additionally, the system does not create or maintain any GxP records. It’s worth noting that the system may be subject to inspections or audits by regulatory bodies such as the FDA, EMEA, MHRA, or other national health authorities.
The business process/process steps and data flow steps of the Bayesian Multilevel Monitoring System are performed as follows:
In step A, the Bayesian Multilevel Model is used in batch processing mode, where the user initiates execution of the system through a shell script program that is part of the computing environment.
In step B, the user inputs a data path and parameters into a file, which includes:
- “Mandatory” parameters, such as the system’s version number and a path to the data residing in the modelling and simulation environment the user has prepared. This data can come from the modelling and simulation environment or the Global Programming and Statistics business unit. Before the analysis begins, a compliance checker will run to ensure that the input data is in the correct format and meets relevant business procedures (SOPs)
- “Optional” parameters, such as an alternative prior distribution for the mean reference parameter for both Poisson and Binomial models.
Step C represents the “Parameter File,” which contains the input data prepared by the user, as well as the “mandatory” and “optional” parameters that have been defined to control the execution of the system and its non-graphical user interface (GUI).
In Step D, the input data from clinical studies are analysed using multilevel statistical models, either Poisson (Step D.1) or Binomial (Step D.2), depending on what has been defined by the user in the “Parameter File.”
Step E produces an output file containing statistical summary data, including Poisson (Step E.1) or Binomial (Step E.2) statistical summaries, which can be processed further outside the system to obtain tables, listings, and graphs.
Finally, in Step F, after the analysis of the input data from clinical studies is complete, all results are obtained in output files containing Statistical Summaries (collection of Poisson or Binomial outputs) and Meta Summaries for each collection of output. These output files can be further processed in a Standard Clinical Reporting System.
Diagram 1: High-Level Overview of Business Process steps and Data Flow, Copyright KVALITO Consulting Group 2023
Project Processes
The system development methodology used for the project was an in-house development process. The system validation process followed GAMP5 and relevant “Validation of Computerized Systems” processes (SOPs, Policies) from the client. The software development methodology, validation strategy, and plan provided a clear and structured design, development, and testing approach. They ensured that all key deliverables were developed and maintained in a controlled manner.
Tools and Programming Software Used During the Project
Bayesian Multilevel Monitoring system was created using R and WinBUGS.
SharePoint, Documentum, and other tools were used during the project.
Value Delivered
The successful validation of the system has ensured that the system meets all relevant regulatory requirements and can effectively detect potential safety signals in clinical trials. The development and implementation of system and application management procedures will also ensure the continued maintenance and compliance of the system. Overall, the system provides a valuable tool for monitoring the safety of clinical trials, which helps to protect patients and ensure the development of safe and effective medications.
Clients/References
- Novartis
