CPIA Module 9, Section 5: Testing & Quality Assurance
MODULE 9: THE SOFTWARE DEVELOPMENT LIFE CYCLE (SDLC)

Section 5: Testing & Quality Assurance

Your most critical patient safety role in the SDLC. Learn the different types of testing, how to write effective test scripts, and master the art of User Acceptance Testing (UAT) to ensure software is safe and effective.

SECTION 9.5

The Final Check: Your Ultimate Patient Safety Mandate

This is where your professional identity as a pharmacist is most directly translated into the world of informatics.

9.5.1 The “Why”: The Last Line of Defense

In your entire career as a pharmacist, your most sacred and non-negotiable duty has been the final verification. No matter how many automated systems are in place, no matter how skilled the technician is, the medication does not reach the patient until you, the licensed professional, have performed a final, meticulous check. You verify the drug, the dose, the patient, the directions, and the clinical appropriateness against the original prescription. This act is not a mere technicality; it is a profound professional and ethical responsibility. It is the last line of defense between a potential error and a potential patient tragedy.

The Testing and Quality Assurance (QA) phase of the SDLC is the direct digital equivalent of this sacred duty. The development team has built the application, the servers are configured, and the code has been compiled. But the software does not touch a real patient or a real clinician until it has passed a rigorous series of tests designed to find and exterminate bugs, errors, and workflow flaws. As an informatics pharmacist, you are the lead clinical architect of this testing process. You are no longer just checking a single prescription; you are stress-testing an entire system that will process millions of prescriptions.

A failure in this phase is a direct threat to patient safety on a massive scale. A single bug—a flawed dose calculation, a missing allergy alert, a confusing screen layout—will be replicated thousands of times per day across the entire health system. The potential for harm is magnified exponentially. This is why the testing phase is not a technical afterthought; it is, for the informatics pharmacist, the most critical patient safety function you will perform. Your ability to think like an adversary, to anticipate every possible way a user could make a mistake or the system could fail, and to design tests that expose these weaknesses is the ultimate expression of your commitment to the patient.

Retail Pharmacist Analogy: The Multi-Layered Final Verification

A patient brings in a new prescription for dabigatran. A technician enters and fills the prescription. It now arrives at your station for the final verification. You don’t just glance at it and bag it. You perform a multi-layered testing protocol that directly mirrors the phases of software QA.

1. Unit Testing (Checking the Label): First, you look only at the vial in your hand. You read the label you just printed. Is the patient’s name spelled correctly? Does the drug name on the label match the manufacturer’s bottle? Is the dose “150 mg” and not “15.0 mg”? Are the directions clear? You are testing the smallest “unit” of work—the data entry and label generation.

2. Integration Testing (Checking the Profile): Next, you integrate this new prescription with the rest of the patient’s data. You pull up their profile on your computer.

  • Allergy Check: Does the patient have a documented allergy to this medication?
  • Duplicate Therapy Check: Is the patient already on warfarin or another anticoagulant from a different doctor? (Checking integration with the patient’s history).
  • Drug Interaction Check: Is the patient also taking dronedarone, creating a severe interaction? (Checking integration with their other active meds).

3. System Testing (The Clinical Appropriateness Check): Now you test the prescription against the “system” of the patient’s overall health. You look at their profile information. The patient is 88 years old and their last recorded creatinine was 1.8 mg/dL, giving them a CrCl of approximately 25 mL/min. The standard dose of 150 mg BID is contraindicated. The system works as a whole, but it’s the wrong plan for this specific patient. Your clinical judgment is the final system test.

4. User Acceptance Testing (The Consultation): You have corrected the dose to 75 mg BID after consulting the prescriber. You now bring the medication to the patient. This is the final, most important test. You ask the user, “The doctor has prescribed Pradaxa 75mg for you, to be taken twice a day to prevent blood clots. Have you taken this or a similar medication before?” You show them the capsule. The patient might say, “Oh, my kidney doctor told me I should never take that kind of medicine.” This is user feedback that no computer could ever find. You have just completed User Acceptance Testing, and in doing so, you have prevented a potentially fatal bleed.

9.5.2 The Testing Pyramid: A Framework for Quality

Effective software testing is not a single activity but a multi-layered strategy. A popular model for visualizing this strategy is the “Testing Pyramid.” The concept is simple: you should have many more small, fast, low-level tests at the bottom of the pyramid, and fewer large, slow, end-to-end tests at the top. As an informatics pharmacist, your involvement increases as you move up the pyramid, culminating in your leadership of the UAT phase at the very peak.

USER ACCEPTANCE TESTING (UAT)

“Does this solve the user’s problem in a real-world workflow?”

System Testing

“Does the whole system work together as expected?”

Integration Testing

“Do the individual modules work together?”

Unit Testing

“Does this single piece of code do what I expect?”

The Testing Pyramid illustrates the ideal distribution of testing efforts.

Level 1: Unit Tests (The Foundation)

Who: Performed by developers.
What: Testing the smallest possible piece of code—a single function or method—in isolation from the rest of the system.
Example: A developer writes a function called `calculateCrCl(age, weight, creatinine, gender)`. They will then write a series of unit tests to verify its logic: “If I pass in the data for an 80-year-old male, does it return the correct value of 56?” “If I pass in a negative number for age, does it correctly throw an error?”
Your Role: While you don’t write these tests, you are the source of the clinical logic they are based on. You provide the developers with the acceptance criteria and edge cases they need to test for. “When you test the CrCl calculator, make sure you have a test case for an elderly female with a very low creatinine, where the value needs to be rounded up to 0.8 before calculating. That’s a critical safety rule.”

Level 2: Integration Tests

Who: Performed by developers and QA engineers.
What: Testing how two or more separate “units” or modules of the system work together.
Example: After the CPOE module and the Pharmacy module are built, an integration test would verify that when a physician enters an order in CPOE (Unit A), it correctly transmits and appears in the pharmacist’s verification queue in the Pharmacy module (Unit B). This tests the API and the data transfer between the modules.
Your Role: You help design the integration test cases by defining the expected outcome of cross-module workflows. “When a nurse documents the administration of a dose of vancomycin in the BCMA module, an integration test must confirm that a message is sent to the pharmacy module to schedule the correct trough draw time in the system.”

Level 3: System Tests (End-to-End)

Who: Performed by a dedicated QA team, often with informatics pharmacists’ input.
What: Testing the entire, fully integrated application from start to finish, simulating a complete clinical workflow.
Example: A system test would involve a tester logging in as a physician, placing a complex medication order, logging out, logging in as a pharmacist, verifying the order, logging out, logging in as a nurse, administering the medication via barcode scanning, and finally checking that the billing module correctly charged for the dose.
Your Role: You are the architect of these end-to-end clinical scenarios. You provide the QA team with complex, real-world workflows that will push the system to its limits. “Let’s create a test scenario for a patient on TPN who is admitted, has their TPN order modified by the dietician, verified by the pharmacist, and then transferred to another unit. We need to ensure the order follows the patient correctly at every step.”

Level 4: User Acceptance Testing (UAT) (The Peak)

Who: Performed by actual end-users (pharmacists, nurses, physicians) under the guidance of the informatics team.
What: The final phase of testing where the intended users of the software use it to execute real-world tasks in a controlled test environment. The goal is not just to find bugs, but to answer the question: “Is this software acceptable for use in our clinical environment?”
Your Role: You are the leader, planner, and facilitator of UAT. This is your ultimate responsibility before go-live, and we will cover it in a dedicated masterclass section below.

9.5.3 The Tester’s Craft: Writing Scripts and Reporting Defects

Saying “Go test the new CPOE system” is not a strategy. Rigorous, effective testing requires discipline and structure. Your ability to write clear, reproducible test scripts and document defects in a precise, unambiguous way is essential for a successful testing phase.

Masterclass in Writing Test Scripts

A good test script is like a well-written recipe. It provides all the necessary ingredients (prerequisites and test data) and a step-by-step set of instructions so that anyone can execute the test and achieve the same result. The best practice is to use the Gherkin “Given-When-Then” format we introduced in the requirements section.

Masterclass Table: From Bad to Great Test Scripts
Poorly Written Test Script Well-Written Test Script Why It’s Better

Test Name: Allergy Alert Test

Steps:

  1. Log in as a doctor.
  2. Order penicillin for a patient with a penicillin allergy.
  3. See if an alert fires.

Test Case ID: UAT-ALL-001

Title: Verify that a severe allergy alert fires and blocks an order.

Given I am logged in as Dr. Jones (a physician role).

And I have opened the chart for patient “John Smith,” who has a documented “severe” allergy to Penicillin (manifestation: anaphylaxis).

When I create a new medication order for “Amoxicillin 500mg PO TID”.

And I click “Sign Order”.

Then the system must display a red, blocking “Severe Allergy Alert”.

And the alert text must state “Patient has a severe allergy to Penicillins.”

And the “Sign Order” button must be disabled until I provide an override reason.

 It is specific (which user, which patient, which drug). It is unambiguous (defines the exact type and text of the alert). It is verifiable (the expected outcomes are clear, testable statements). Anyone can run this test and know exactly what to look for.

Masterclass in Reporting Defects

When a test fails, you have found a defect (or “bug”). Your job now is to report it in a way that gives the developer all the information they need to find, understand, and fix the problem. A great bug report is a gift to a developer; a bad one is a source of frustration and delay.

The Anatomy of a Perfect Bug Report

Every bug tracking system (like Jira or Azure DevOps) has a standard format. Your report must always include:

  1. A Clear, Concise Title: “Allergy alert does not fire for cross-reactive cephalosporin.” (Good) vs. “Allergy broken” (Bad).
  2. Steps to Reproduce: A numbered, step-by-step list of the exact actions you took. This is the most important part. If the developer can’t reproduce the bug, they can’t fix it.
  3. Expected Result: What you predicted would happen based on the test script. “The system was expected to fire a cross-sensitivity warning.”
  4. Actual Result: What actually happened. “No warning was displayed, and the order was accepted by the system.”
  5. Severity Level: You must assess the clinical impact of the bug. This is a critical pharmacist skill.
    • Blocker/Critical: Patient safety issue, data loss, system crash. Must be fixed before go-live.
    • Major: Major loss of functionality, no workaround exists.
    • Minor: Minor loss of functionality, or a workaround is available.
    • Trivial: Cosmetic issue, like a typo or a misaligned button.
  6. Attachments: A screenshot is worth a thousand words. Always attach a screenshot of the error message or the incorrect screen. If possible, a short screen recording is even better.

9.5.4 Masterclass in User Acceptance Testing (UAT)

UAT is the final gatekeeper. It is your formal process for confirming that the system is ready for the real world. This is not informal “playing around” with the system. It is a structured, planned, and documented phase of the project that you will lead.

Phase 1: Planning and Preparation

  • Recruit Your Testers: Do not just pick the “super-users” who love technology. You need a representative sample. Include the tech-savvy pharmacist, the tech-skeptical pharmacist, the brand-new nurse, and the charge nurse with 20 years of experience. A variety of perspectives is key to finding real-world issues.
  • Build the Test Environment: Work with the technical team to ensure you have a dedicated UAT environment. This should be a copy of the production environment with the new code installed. It must be stable and populated with realistic (but de-identified) patient data.
  • Develop Scenarios and Scripts: Write the detailed test scripts that your testers will follow. Don’t just test individual functions; create day-in-the-life scenarios. “A patient is admitted from the ED, has their home meds reconciled, receives three new IV medications, and is then transferred to the ICU. Go.”
  • Schedule Everything: UAT is a major logistical challenge. You need to book conference rooms, schedule time for your testers (which often means arranging for backfill on their units), and ensure technical support is available during the sessions.

Phase 2: Execution and Facilitation

Your role during the UAT session is that of a neutral facilitator. Your goal is to observe and record, not to teach or defend the system.

The UAT Facilitator’s Playbook
  1. Set the Stage: “Thank you for being here. Today we are testing a new version of our CPOE system. Please remember, we are testing the software, not you. You cannot make a mistake. Please think out loud and tell me what you are trying to do and what you expect to happen.”
  2. Provide the Scenario: Give the user their script, but encourage them to use their own judgment. “Your task is to discontinue the patient’s oral metoprolol and order the IV equivalent.”
  3. Observe and Be Quiet: This is the hardest part. Watch their clicks. Listen to their thought process. Do not jump in to help them if they are struggling. Let them struggle. Their struggle is the most valuable data you can collect. Where do they hesitate? Where do they click in the wrong place first? This reveals flaws in the UX.
  4. Use Probing Questions: If they get stuck, ask open-ended questions. “What were you expecting to see on this screen?” or “What are you looking for right now?”
  5. Document Everything: Have a dedicated scribe or use a screen recording tool. Capture every observation, quote, and bug.

Phase 3: The Go/No-Go Decision

After UAT is complete, you will have a list of defects. Your final, critical task is to lead the clinical stakeholders in a “Go/No-Go” meeting. You will present the results of UAT: the number of test cases passed and failed, and a detailed review of all outstanding critical and major bugs. Based on this data, you must make a formal recommendation:

  • Go: The system is safe and effective. All critical defects have been resolved. We are ready to move to implementation.
  • Go with Workarounds: The system is largely stable, but there are some outstanding major defects. We have developed and trained users on safe, temporary workarounds for these issues and we can proceed with the go-live while these are being fixed.
  • No-Go: We have discovered significant patient safety issues or workflow-blocking defects. It would be irresponsible to release this software in its current state. We must postpone the go-live until these critical issues are resolved and re-tested.
This recommendation is one of the most important you will make in your informatics career. It requires courage, data-driven analysis, and an unwavering commitment to patient safety over project deadlines.

9.5.5 Conclusion: The Pharmacist as the System’s Immune Response

If the SDLC is the life cycle of a living organism, the Testing and Quality Assurance phase is its immune system. It is the complex, multi-layered process that actively seeks out, identifies, and neutralizes threats before they can harm the host—the patient. Your role as an informatics pharmacist is to be the intelligent, driving force of that immune system. You bring a unique clinical awareness that allows you to identify pathogens (bugs) that a purely technical test might miss.

Your entire professional training has been a preparation for this role. The meticulous verification of a prescription, the critical evaluation of a patient’s profile, the constant vigilance for potential interactions and contraindications—these are the very skills required to design and execute a world-class testing strategy. By embracing this phase, you are not just ensuring software quality. You are upholding your pharmacist’s oath on a systemic scale, guaranteeing that the powerful technological tools you help to create are worthy of the trust that patients and clinicians place in them.