Section 1: Test Plan Development and Acceptance Criteria

10.1.1 The “Why”: Translating Verification into a Formal Strategy

In your entire professional life as a pharmacist, the concept of verification has been an ever-present, almost subconscious, part of your workflow. It is the silent pause you take before handing a medication to a patient, the rapid mental checklist you run when a new prescription appears on your screen, the “gut feeling” that something about a dose is not quite right. This constant, low-level hum of critical analysis is what makes you a safe and effective practitioner. It is a process honed over tens of thousands of repetitions, built on a deep foundation of clinical knowledge, pattern recognition, and a profound sense of responsibility.

The challenge and opportunity of pharmacy informatics is to take this deeply ingrained, almost intuitive, process and translate it into a formal, documented, and repeatable strategy. A Test Plan is the document that accomplishes this. It is the formal manifestation of a pharmacist’s skepticism. It is not merely a “to-do” list of tasks; it is a strategic document that outlines the entire philosophy, scope, and methodology for proving that a complex system is safe for patient care. It is your ultimate risk mitigation tool. While a pharmacist at the bench prevents errors one prescription at a time, a pharmacist authoring a test plan prevents entire classes of errors for thousands of patients simultaneously.

If a project’s implementation plan is its roadmap, the test plan is its comprehensive safety inspection protocol. It forces the entire project team—from developers and analysts to clinical end-users—to stop and ask the most critical questions before a single line of code is released into the wild:

• What, precisely, are we trying to achieve with this new technology?
• How will we objectively know if we have succeeded?
• What are all the ways this could fail, and how will we proactively try to make it fail in a controlled environment?
• Who is responsible for what, and what resources do we need to do this job properly?
• What are the absolute, non-negotiable conditions that must be met before we allow this system to touch a real patient?

This section will provide the master framework for answering these questions. We will deconstruct the formal components of a test plan and, more importantly, master the art of writing Acceptance Criteria—the clear, unambiguous, and testable statements that define success. This is where your clinical expertise becomes the bedrock of technical quality assurance.

10.1.2 The Anatomy of a Master Test Plan: Your Strategic Blueprint

A formal test plan is not an informal memo; it is a structured, comprehensive document that follows a standardized format, often based on industry standards like IEEE 829. While the exact template may vary between organizations, the core components are universal. As a pharmacy informatics analyst, you will be responsible for authoring, reviewing, and contributing to these documents. Understanding each section is essential to ensuring a thorough and defensible validation strategy. Let’s dissect a master test plan, section by section, from a pharmacist’s perspective.

Masterclass Table: Deconstructing the Test Plan

Test Plan Section	Purpose & Description	Pharmacist’s Clinical Application & Key Questions
1.0 Test Plan Identifier	A unique name or number for the document (e.g., “ADC_Firmware_Upgrade_v2.5_TestPlan_2025-10-18”). This is for version control and traceability.	Is this linked to a specific project or change request number? If I need to find this document in five years during an audit, can I?
2.0 Introduction & Scope	A high-level overview of the project and its goals. Crucially, it defines the boundaries of the testing effort. It explicitly states what is IN SCOPE (will be tested) and what is OUT OF SCOPE (will not be tested).	This is the most important political and safety section. In Scope: We are testing the new barcode verification for chemotherapy IVs. Out of Scope: We are NOT re-testing the entire IV workflow system, barcode scanning for oral meds, or the interface to the billing system. Question: Have we clearly drawn the line to prevent scope creep, while ensuring all safety-critical components are included?
3.0 Test Items	A list of the specific software, hardware, and documents that are the subject of the test. This includes version numbers, build numbers, and hardware models.	Software: Epic 2023.1, Pyxis ES v1.6.2, Codonics Safe Label System v5.2. Hardware: Zebra ZD410 label printers, Honeywell 1900 scanners. Documents: Clinical workflow diagrams, functional specification documents. Question: Are we certain we are testing against the exact version of the software that will be going live?
4.0 Features to Be Tested	A detailed, granular list of the new features or functionalities that will be validated. This is a breakdown of the “In Scope” items.	Think like a clinician using the system. Ability to place a CPOE order for a weight-based heparin drip. Correct calculation of the initial bolus and starting infusion rate based on the protocol. Generation of appropriate nursing tasks for PTT monitoring. Correct display of the MAR entry for the nurse.
5.0 Features Not to Be Tested	An explicit list of functionalities that will be intentionally excluded from this testing cycle, along with the rationale. This is a breakdown of the “Out of Scope” items.	This protects you and the team. Reasoning: “The billing output of the heparin order is not being tested because this functionality was extensively validated during the last revenue cycle upgrade and no code changes have been made to that interface.” Question: Is the justification for excluding something sound and defensible? Have we accepted a risk by not testing something?
6.0 Approach / Strategy	The core methodology of the test. It describes the types of testing to be performed (e.g., Unit, Integration, Regression, User Acceptance Testing) and the overall flow. It mentions who will be testing and at a high level, how they will be testing.	This is your battle plan. “An integrated testing cycle will be conducted over two weeks by a team of 3 pharmacists and 3 nurses. Testers will execute scripted test cases that simulate the entire medication process from order to administration. A separate day will be dedicated to unscripted, ‘break-the-system’ negative testing.” Question: Does our strategy reflect the complexity and risk of the change? Are we involving the right people?
7.0 Item Pass/Fail Criteria	The high-level definition of success for the entire testing effort. It defines when a specific test case passes or fails, and when the overall project is considered ready for go-live.	This is the “go/no-go” decision point. Test Case Pass: All steps executed, and actual results match expected results. Overall Project Pass: 100% of high-priority test cases must pass. No more than 5% of medium-priority test cases may be open. No “Blocker” or “Critical” severity defects are open. Question: Are our criteria for success stringent enough for a patient care system?
8.0 Suspension & Resumption Criteria	Defines the conditions under which testing will be halted entirely (e.g., discovery of a critical “showstopper” bug that prevents further testing) and the criteria for restarting once the issue is fixed.	This is your emergency stop button. Suspend if: “The testing environment becomes unstable, or a defect is found that prevents more than 25% of the remaining test cases from being executed (e.g., login failure).” Resume when: “A patch has been applied to the test environment, and the blocking defect has been successfully re-tested and closed by the QA team.”
9.0 Test Deliverables	A list of all the documents and artifacts that will be created as part of the testing process.	This is your evidence locker for audits. Test Cases Test Scripts Defect/Bug Reports Test Execution Logs Test Summary Report Validation Sign-off Sheet
10.0 Environmental Needs	A detailed specification of the necessary hardware, software, data, interfaces, and security access needed to perform the testing.	This is your setup for success. Hardware: 5 PCs, 2 Zebra printers, 3 Honeywell scanners, 1 Pyxis MedStation. Software: Access to the TEST environment of the EHR, credentials for 3 test pharmacists and 3 test nurses. Data: At least 20 de-identified patient profiles that mirror common clinical scenarios (e.g., renal failure, pediatric, ICU). Interfaces: A functional, tested interface between the EHR and the ADC in the test environment.
11.0 Staffing & Training Needs	Identifies the key personnel needed for testing and any specific training they might require before the testing cycle begins.	You can’t test what you don’t understand. Staff: John Smith (Pharmacist Lead), Jane Doe (Nurse Lead), 2 additional pharmacists, 2 additional nurses. Training: “All testers must attend a 1-hour training session on the new heparin protocol functionality and a 30-minute session on how to use the Jira bug tracking system.”
12.0 Responsibilities	Clearly defines who is responsible for what. Often uses a RACI (Responsible, Accountable, Consulted, Informed) chart to assign roles for every major task in the testing process.	This prevents finger-pointing later. Authoring Test Cases: Pharmacist/Nurse Leads (Responsible), Project Manager (Accountable). Executing Tests: Staff Pharmacists/Nurses (Responsible). Triaging Defects: Analyst Team (Responsible). Fixing Defects: Development Team (Responsible).
13.0 Schedule	A detailed timeline of all testing activities, including deadlines for test case writing, environment setup, execution cycles, and defect resolution.	This is your project timeline. It should include buffer time. A 2-week testing cycle might have 3-4 days dedicated to re-testing fixed bugs and regression testing.
14.0 Risks & Contingencies	Identifies potential problems that could derail the testing effort and outlines a proactive plan to mitigate them.	Thinking ahead about what could go wrong. Risk: “The key pharmacist subject matter expert may be pulled away for urgent clinical duties.” Contingency: “A secondary pharmacist has been identified and trained as a backup.” Risk: “The development team may not be able to fix critical defects within the testing window.” Contingency: “A daily defect triage meeting with the development lead will be held to prioritize bugs. The go-live date may be postponed if critical defects remain.”
15.0 Approvals	Specifies who must review and sign the test plan before testing can begin. This signifies that key stakeholders agree with the strategy and scope.	This is the formal sign-off. Signatories often include the Project Manager, the Director of Pharmacy, a key Physician champion, and the lead IT Quality Assurance manager.

10.1.3 Masterclass on Writing Acceptance Criteria (AC)

If the test plan is the strategic blueprint, then the Acceptance Criteria (AC) are the detailed, legally-binding construction specifications. They are short, clear, testable statements that describe the required behavior of a feature from an end-user’s perspective. They are the ultimate definition of “done.” An AC is a contract: if the system meets all of its acceptance criteria, the feature is considered complete and correct. If it fails even one, it has a defect. Your ability as a pharmacist to write clear, comprehensive, and unambiguous AC is perhaps the single most important skill in clinical system validation.

Poorly written AC are the leading cause of failed projects, software that doesn’t meet clinical needs, and patient safety risks. Vague statements like “The system should be easy to use” or “The order should process quickly” are useless because they are not testable. How do you measure “easy”? How do you define “quickly”? Writing effective AC removes all ambiguity and subjectivity, replacing it with a clear, binary (pass/fail) condition.

The Gherkin Framework: A Universal Language for Behavior

One of the most effective and widely adopted formats for writing AC is the Gherkin syntax, often used in Behavior-Driven Development (BDD). It provides a simple, structured “Given-When-Then” framework that anyone—from a developer to a clinician—can understand. It tells a story about how a user interacts with the system.

From Weak to World-Class: Transforming Acceptance Criteria

Let’s see this in action. We will take common pharmacy scenarios and transform vague, untestable requirements into strong, Gherkin-formatted acceptance criteria. This transformation is your core function as a clinical quality gatekeeper.

Scenario	Weak / Vague Requirement	Strong / Testable Acceptance Criteria
Ordering a Renally Dosed Drug	“The system should warn me about kidney function.”	Given a patient with a documented Creatinine Clearance of 25 mL/min is in the chart And the pharmacist is placing a new CPOE order for enoxaparin 1 mg/kg Q12H When the pharmacist clicks “Sign Order” Then a blocking alert should fire with the title “Renal Dosing Alert” And the alert text must state “Patient’s CrCl is 25 mL/min. Standard dose is not recommended. The recommended dose is 1 mg/kg Q24H.” And the alert must present two options: “Cancel Order” and “Override with Justification”.
Barcode Medication Administration (BCMA)	“Scanning the wrong drug should cause an error.”	Given a patient’s electronic MAR (eMAR) has an active order for Lisinopril 10 mg And the nurse is in the patient’s room and has scanned the patient’s wristband successfully When the nurse scans the manufacturer barcode on a package of Losartan 50 mg Then the eMAR should display a red banner with the text “Wrong Medication Scanned. Expected Lisinopril 10 mg.” And an audible, high-pitched error tone must play through the workstation speakers And the “Administer” button must remain disabled.
Automated Dispensing Cabinet (ADC) Stocking	“The ADC needs to be refilled correctly.”	Given the ADC’s inventory level for hydromorphone 2mg tabs is below the par level And a “Stock Out” report has been generated When the pharmacy technician loads the “Stock Out” report on the ADC screen Then the specific pocket for hydromorphone 2mg tabs should pop open And the screen must prompt “Scan medication to refill.” And When the tech scans a valid hydromorphone 2mg tab barcode Then the screen must prompt “Enter quantity” And When the tech enters “20” and closes the pocket Then the system inventory for that medication must be incremented by 20.
IV Workflow Management System	“The system should take a picture of the ingredients.”	Given a technician is assigned to prepare a vancomycin 1.5g IV piggyback in the clean room And the technician is at the “Ingredient Verification” step in the IV workflow software When the technician scans the barcode on a vial of Vancomycin 1g powder And the technician scans the barcode on a vial of Vancomycin 500mg powder And the technician scans the barcode on a 250 mL bag of D5W Then the system must capture and store a high-resolution image of the three scanned items together And the system must advance the status of the preparation to “Ready for Pharmacist Verification”.

10.1.4 Integrating Risk Analysis into Test Planning

You cannot test everything. In a complex health IT system, the number of possible user pathways and data combinations is virtually infinite. A brute-force approach to testing is inefficient and often ineffective. The key to a successful testing strategy is to be intelligent and risk-based. As a pharmacist, you are already an expert at risk stratification. You instinctively pay more attention to a new warfarin prescription than a refill for ibuprofen. Your test plan must reflect this same clinical pragmatism. You must focus your limited time and resources on the areas of the system that pose the greatest risk to patient safety.

A powerful, proactive tool for achieving this is the Failure Mode and Effects Analysis (FMEA). FMEA is a systematic, team-based activity that involves brainstorming all the potential ways a process or system could fail (Failure Modes), understanding the consequences of those failures (Effects), and identifying their potential causes. Crucially, it involves a scoring system to prioritize which risks are the most severe, allowing you to tailor your testing strategy to be tougher on the high-risk areas.

The FMEA Scoring System: Calculating the Risk Priority Number (RPN)

The core of the FMEA is calculating a Risk Priority Number (RPN) for each potential failure mode. This number helps to objectively rank the risks. The formula is:

$$ \text{RPN} = \text{Severity (S)} \times \text{Occurrence (O)} \times \text{Detection (D)} $$

Each factor is typically scored on a scale of 1 to 10. The higher the RPN, the higher the risk, and the more intense your testing for that scenario should be.

Factor	Description	Example Scoring (1 = Low, 10 = High)
Severity (S)	How catastrophic would the consequences be if this failure occurred?	10: Patient death or catastrophic harm (e.g., wrong chemo dose). 7: Significant harm requiring intervention (e.g., missed dose of anticoagulant). 4: Minor inconvenience, no patient harm (e.g., label prints in wrong font). 1: No impact.
Occurrence (O)	How likely is this failure to happen?	10: Almost certain to happen frequently. 7: Could happen occasionally. 4: Unlikely, but possible. 1: Extremely remote, has never happened before.
Detection (D)	How likely are we to detect the failure *before* it reaches the patient?	10: Impossible to detect; the failure is silent (e.g., incorrect data sent to a background interface). 7: A very astute clinician might notice, but there’s no system check. 4: A standard nursing double-check would likely catch it. 1: The system has a built-in hard stop that makes the failure obvious.

Masterclass Example: Applying FMEA to a New CPOE Feature

Let’s apply this to a real-world scenario: implementing a new order set for Patient-Controlled Analgesia (PCA) pumps.

Failure Mode (What could go wrong?)	Potential Effect (What is the consequence?)	Severity (S)	Occurrence (O)	Detection (D)	RPN (S x O x D)	Resulting Test Strategy
The order set defaults to mcg instead of mg for morphine concentration.	Patient receives a 1000-fold overdose, leading to respiratory arrest and death.	10	3	6	180	Maximum Intensity: Multiple, scripted test cases using different users and patient types. Negative testing to ensure the unit cannot be changed. Independent double-check of all build configurations.
The order does not correctly transmit the “4-hour limit” to the pharmacy system.	The pharmacy might prepare the bag without the clinical context, but the pump itself has a hard limit, preventing overdose. Risk of confusion.	5	5	4	100	High Intensity: Dedicated integration test cases to verify data flow between CPOE and pharmacy system. Test across multiple orders to confirm consistency.
The printed MAR from the order set is poorly formatted and hard to read.	Nurse frustration. Potential for misinterpretation, but likely to be caught during standard verification. Low risk of direct harm.	3	7	3	63	Medium Intensity: A single test case to verify MAR appearance. Solicit feedback from nurse testers. Considered a “usability” bug, not a safety-critical one.

By performing this FMEA *before* you even start writing test cases, you have already identified your highest-risk areas. The PCA concentration error (RPN=180) demands far more rigorous testing than the MAR formatting issue (RPN=63). Your test plan should explicitly state that the intensity of testing will be proportional to the RPN scores identified in the project’s risk analysis. This is how you build a defensible, efficient, and patient-safe validation strategy.