CASP Module 2, Section 5: Internal Auditing and Continuous Improvement
Module 2: Regulatory, Accreditation & Quality Frameworks

Section 2.5: Internal Auditing and Continuous Improvement

Utilizing internal audits, root cause analysis (RCA), and Plan-Do-Study-Act (PDSA) cycles to identify deficiencies, implement corrective actions, and foster ongoing quality enhancement.

SECTION 2.5

Internal Auditing and Continuous Improvement

From Reactive Problem-Solving to Proactive Excellence Engineering.

2.5.1 The “Why”: Closing the Loop – The Engine of the QMS

In Section 2.4, we established the Quality Management System (QMS) as the pharmacy’s central nervous system—the framework of policies, procedures, data (KPIs), and documentation that defines how the organization functions. However, a nervous system that only senses and processes information without acting on it is inert. This section focuses on the action components of the QMS: the tools and methodologies used to proactively find weaknesses, systematically fix problems, and continuously drive the organization towards higher levels of performance. These are Internal Auditing and Continuous Quality Improvement (CQI).

Think of it this way:

  • Regulations, Accreditation, Contracts (2.1-2.3): Define the rules of the road and the destination (compliance, LDD access).
  • QMS & Documentation (2.4): Build the vehicle (the processes, P&Ps) and the dashboard (KPIs, documentation) to navigate the road.
  • Internal Auditing & CQI (2.5): Provide the engine and the steering wheel. Internal audits are the engine’s sensors, detecting when the vehicle is veering off course or malfunctioning. CQI methodologies (like RCA and PDSA/PDCA) are the steering wheel and accelerator/brake, allowing the Quality Management Committee (the driver) to correct the course and optimize performance.

In your community practice, investigating a dispensing error might have been a reactive event, perhaps focused on individual blame. In specialty pharmacy, under the umbrella of a QMS, this process is formalized, proactive, and system-focused. Internal audits are not “gotcha” exercises; they are planned check-ups designed to find potential problems before an external auditor (URAC, FDA, Manufacturer) does. CQI is not about assigning blame; it is about dissecting why an error occurred (Root Cause Analysis – RCA) and implementing robust system changes to prevent recurrence (Plan-Do-Study-Act – PDSA).

Accreditation bodies don’t just ask, “Do you have a QMS?” They demand proof: “Show me your internal audit schedule and reports. Show me your RCA documentation for the last dispensing error. Show me your active Performance Improvement Projects (PIPs) based on your KPI data.” Mastering these tools is not optional; it is a core competency required by URAC, ACHC, and essential for maintaining LDD contracts. As a CASP, you will be involved in all aspects of this cycle – potentially participating in audits, leading RCAs for clinical errors, and implementing corrective actions within your team.

Pharmacist Analogy: From Ad Hoc Fixes to Formula 1 Pit Crew & Engineering Team

Imagine your community pharmacy encountered a problem, like realizing you were storing vaccines at the wrong temperature. The response might have been: buy a new thermometer, maybe talk to the staff – an ad hoc fix.

In specialty pharmacy, the approach is like that of a Formula 1 Racing Team.

  • KPIs (The Telemetry): The car is covered in sensors (KPIs) constantly feeding data back to the pits (the QMC) – tire pressure (cold chain), engine temp (TAT), fuel consumption (adherence rates).
  • Internal Audit (The Pre-Race Inspection & Pit Stop Practice): Before every race (and regularly between them), the team performs meticulous inspections (internal audits). They don’t just look for obvious broken parts; they check tolerances, simulate race conditions, and practice pit stops, proactively looking for any potential weakness or inefficiency against their established procedures (P&Ps).
  • The Incident (The Spin-Out or Slow Pit Stop): During the race, something goes wrong. The car spins out (a patient complaint), or a pit stop takes 5 seconds instead of 2 (a missed TAT metric).
  • Root Cause Analysis (The Post-Race Engineering Debrief): After the race, the team doesn’t just blame the driver (the pharmacist) or the tire changer (the tech). The engineers (the QMC and PIP team) gather all the telemetry data (KPIs), review video footage (documentation, error reports), and conduct a deep-dive analysis (RCA). Was it driver error? Faulty equipment (a system issue)? A flawed procedure (P&P)? Poor communication (training gap)? They use tools like the “5 Whys” or a “Fishbone Diagram” to get to the fundamental underlying system failure.
  • PDSA/PDCA Cycle (The Engineering Fix & Testing): Based on the RCA, the engineers propose a specific change (PLAN – e.g., redesign a wheel nut, change the pit stop choreography). They implement the change on a small scale or in practice (DO). They collect data on the new process (STUDY/CHECK – did it fix the problem without causing new ones?). If successful, they implement it across the team and update the official procedures (ACT).

This constant cycle of proactive inspection (audit), deep analysis of failures (RCA), and systematic implementation of improvements (PDSA) is what allows F1 teams to operate at the absolute peak of performance and safety. It’s the same methodology that high-performing specialty pharmacies use to manage complexity and drive excellence. Your role shifts from simply fixing the flat tire to being part of the engineering team that prevents flats from happening in the first place.

2.5.2 Masterclass on Internal Auditing: The Proactive Check-Up

Internal auditing is a cornerstone of any effective QMS and a specific requirement of accreditation standards (e.g., URAC QM 4.0). It is a formal, systematic, and documented process for objectively evaluating whether your pharmacy’s activities and related results comply with planned arrangements (P&Ps, regulations, standards) and whether these arrangements are implemented effectively and are suitable to achieve objectives.

Think of it as giving yourself an accreditation survey or a manufacturer audit before they do. It’s about finding and fixing your own problems proactively.

Types of Internal Audits in Specialty Pharmacy

Internal audits can take several forms, focusing on different aspects of the operation:

  • Process Audits: Evaluate the effectiveness and compliance of a specific workflow or P&P. (e.g., “Let’s audit our Cold Chain Shipping Process from pack-out to delivery confirmation.”)
  • Compliance Audits: Focus on adherence to specific external requirements. (e.g., “Let’s audit our last 30 REMS dispenses for proper authorization documentation.” or “Let’s audit our DSCSA receiving logs for the past month.”)
  • Patient Chart Audits (Tracer Audits): Follow the journey of several randomly selected patients through your system, evaluating compliance at each step (intake, BV, clinical assessment, counseling, dispensing, shipping, follow-up). This mimics the accreditation tracer methodology.
  • Departmental Audits: Review the overall compliance and performance of a specific team (e.g., Clinical Pharmacists, Intake Specialists).

The Internal Audit Process: A Step-by-Step Tutorial

A formal internal audit follows a structured process, typically overseen by the Quality department or QMC.

1
PLAN: The Audit Schedule & Plan

The QMC develops an annual Internal Audit Schedule, deciding which processes/areas will be audited and when (often quarterly). For each specific audit, a detailed Audit Plan is created:

  • Objective: What is the purpose of this audit? (e.g., “Verify compliance with P&P CLN-05: Initial Patient Assessment”).
  • Scope: What specific activities, dates, and patient records will be included? (e.g., “Review 15 randomly selected new patient charts initiated between Jan 1 – Mar 31”).
  • Criteria: What standards are we auditing against? (e.g., “P&P CLN-05, URAC SP Std 2.1”).
  • Auditor(s): Who will conduct the audit? (Should be independent of the area being audited if possible).
  • Methodology: How will the audit be done? (e.g., Chart review, staff interviews, observation).

2
DO: Conduct the Audit & Gather Evidence

The assigned auditor executes the plan. They gather objective evidence – reviewing documents, observing processes, interviewing staff. They compare what they find (the actual practice) against the criteria (the required standard/P&P). All findings, both compliant and non-compliant, are documented meticulously on audit worksheets.

3
STUDY/CHECK: Analyze Findings & Report

The auditor analyzes the evidence. Non-conformities (where practice does not meet the standard) are identified and categorized (e.g., minor, major). A formal Internal Audit Report is written, summarizing:

  • Audit scope, objectives, criteria.
  • Summary of findings (strengths and weaknesses).
  • Detailed list of all non-conformities identified, citing specific evidence.
  • Overall conclusion and recommendations.
This report is presented to the management of the audited area and to the QMC.

4
ACT: Corrective Actions & Follow-Up

This is where the audit connects to CQI. For each non-conformity, the management of the audited area must:

  • Perform an RCA (see 2.5.3) to understand why the failure occurred.
  • Develop a Corrective and Preventive Action (CAPA) Plan detailing the specific steps they will take to fix the problem and prevent recurrence, with timelines and responsible individuals.
  • Implement the CAPA plan.
The auditor (or QA department) then performs a Follow-Up Audit later to verify that the CAPA was implemented and effective. This closure is reported back to the QMC.

Your Role as a Pharmacist in Internal Audits

You might wear several hats:

  • The Auditee: An auditor might review your patient charts, observe your counseling technique, or interview you about a process. Your role is to be transparent, provide accurate information, and demonstrate your knowledge of the relevant P&Ps.
  • The Subject Matter Expert (SME): You might be asked to help develop audit criteria or review findings related to clinical processes.
  • The Auditor: With appropriate training, you might be asked to conduct audits of other departments or processes (especially clinical ones), bringing your expert eye to the evaluation.
  • The Problem Solver: If an audit identifies a non-conformity in your area, you will likely be involved in the RCA and the development/implementation of the CAPA plan.

2.5.3 Masterclass on Root Cause Analysis (RCA): Digging for “Why”

When something goes wrong – a dispensing error, a patient complaint, a missed REMS requirement, a poor KPI result – the first instinct can be to blame an individual or implement a quick fix. CQI demands a deeper approach: Root Cause Analysis (RCA). RCA is a systematic problem-solving method aimed at identifying the underlying system factors that caused the event, not just the surface symptoms.

The core principle: Errors are rarely caused by just one thing or one person. They are usually the result of multiple contributing factors within the system (e.g., flawed processes, inadequate training, poor communication, environmental factors, technology issues). Fixing only the surface problem (e.g., retraining one person) doesn’t prevent the same error from happening again with someone else if the underlying system flaw remains.

When is an RCA Needed?

Accreditation standards require RCAs for significant events, but a good QMS uses them more broadly:

  • Sentinel Events: Unexpected occurrences involving death or serious physical or psychological injury (required by TJC, URAC, ACHC).
  • Significant Medication Errors: Errors reaching the patient that cause harm or have the potential to cause significant harm (NCC MERP Categories E-I).
  • Repeated Errors/Near Misses: A pattern of the same type of error occurring, even if minor.
  • Significant Compliance Failures: Major findings on internal or external audits (e.g., a pattern of missed REMS authorizations).
  • Persistent Poor KPI Performance: When a key metric remains below target despite initial efforts.

Common RCA Tools: 5 Whys & Fishbone Diagram

RCA is typically conducted by a small, multidisciplinary team (including frontline staff involved). Two common, simple tools are often used:

Tool 1: The 5 Whys (Tutorial)

This technique involves asking “Why?” repeatedly (often around 5 times) to drill down from the surface problem to the underlying root cause(s).

Playbook: 5 Whys Example – Wrong Drug Dispensed

Problem: Patient received Drug B instead of Drug A (Look-Alike/Sound-Alike Error).

  1. Why #1? Why did the pharmacist verify Drug B when the Rx was for Drug A?
    Answer: The pharmacist visually misidentified the bottle pulled by the technician.
  2. Why #2? Why did the technician pull Drug B?
    Answer: The bottles for Drug A and Drug B were stored right next to each other on the shelf, and their labels look similar.
  3. Why #3? Why are two look-alike drugs stored adjacently?
    Answer: The pharmacy shelves are organized purely alphabetically by generic name.
  4. Why #4? Why haven’t we implemented other safety strategies for LASA drugs?
    Answer: Although discussed, we haven’t formally updated our P&P or implemented strategies like Tall Man lettering or shelf separators for these specific drugs.
  5. Why #5? Why wasn’t this risk identified and addressed sooner?
    Answer: Our process for proactively identifying and mitigating LASA risks based on ISMP lists and internal stocking is not robust.

Root Causes Identified: Poor shelf organization strategy (alphabetical only), lack of specific LASA mitigation strategies (Tall Man, separators), inadequate proactive risk assessment process.

Resulting CAPA Plan: (1) Revise shelving P&P to separate identified LASA pairs. (2) Apply Tall Man lettering labels to Drugs A & B. (3) Implement shelf separators. (4) Assign QMC to develop a proactive LASA risk review process.

Tool 2: Fishbone Diagram (Ishikawa Diagram) (Tutorial)

This visual tool helps brainstorm potential causes by categorizing them into common domains. The “head” of the fish is the problem, and the “bones” are categories of potential causes.

Playbook: Fishbone Example – Delay in New Rx TAT

Problem (Head): Average New Rx TAT = 55 hours (Goal < 48 hours).

Inadequate staffing in BV team?
Insufficient training on PA process?
Pharmacist clinical review backlog?
Process
Workflow for handling incomplete referrals unclear?
No process for prioritizing urgent PAs?
Multiple handoffs between teams?
Technology
EHR/Dispensing system slow?
Fax machine unreliable for PAs?
PA submission portal often down?
Materials/Info
Referrals often missing clinical documentation?
Incorrect insurance information provided?
Environment
High noise level impacting phone calls?
Frequent interruptions?
Delay in New Rx TAT (>48h)

Potential Root Causes Identified: The diagram helps the team brainstorm. Further investigation might reveal the biggest delays are due to missing clinicals on referrals (Materials/Info) and an unclear process for handling them (Process).

Resulting CAPA Plan: (1) Develop a checklist for intake staff for required referral elements. (2) Create a P&P for handling incomplete referrals, including standardized communication back to the prescriber. (3) Track impact on TAT.

The Goal: System Fixes, Not Blame

A successful RCA focuses on identifying and fixing the system vulnerabilities that allowed the error or problem to occur. It should lead to concrete, actionable improvements to processes, training, technology, or the environment.

Your Role as a Pharmacist in RCA: If an error involves clinical judgment or the dispensing process, you are a crucial SME. Participate honestly. Focus on the process, not the person. Help identify system weaknesses you see in your daily work.

2.5.4 Masterclass on PDSA/PDCA Cycles: Implementing & Testing Change

Identifying a root cause (RCA) or an opportunity for improvement (from KPIs or audits) is only the first step. The Plan-Do-Study-Act (PDSA) cycle (also commonly called Plan-Do-Check-Act or PDCA) is the scientific method applied to quality improvement. It provides a structured way to implement changes, test their effectiveness, and standardize improvements.

This cycle is the engine of your Performance Improvement Projects (PIPs) – the formal projects launched by the QMC to address specific problems or goals.

The PDSA Cycle Explained (Tutorial)

We introduced this briefly in 2.4.6. Let’s break down each phase in the context of a PIP.

P
PLAN

Define & Plan the Change.

  • Objective: What specific, measurable goal are we trying to achieve? (e.g., “Reduce average New Rx TAT from 55h to <48h within 3 months").
  • Current State Analysis: What does the data show? What did the RCA reveal?
  • Proposed Change (Hypothesis): What specific change are we proposing? (e.g., “Implement a new P&P for handling incomplete referrals”).
  • Action Plan: Who will do what, by when? (e.g., “Pharmacist Lead to draft P&P by Oct 31; IT to update system field by Nov 15; Training for Intake team Nov 20”).
  • Prediction: What outcome do we expect? (e.g., “We predict this will reduce holds for missing info by 50%”).
  • Data Collection Plan: How will we measure the impact? (e.g., “Track TAT daily; track reason codes for holds”).
D
DO

Implement the Change (Often on a Small Scale).

  • Execute the action plan.
  • Implement the change (e.g., train the team, roll out the new P&P).
  • Pilot Testing: It’s often wise to test the change with one team, one drug, or for a limited time first, rather than a full rollout.
  • Document any problems or deviations encountered during implementation.
  • Begin collecting data according to the plan.
S / C
STUDY / CHECK

Analyze the Data & Compare to Predictions.

  • Aggregate the data collected during the “Do” phase.
  • Analyze the results: Did we achieve the objective? (e.g., “Average TAT decreased to 46 hours”).
  • Compare results to predictions: Was the impact as expected? Were there unexpected consequences (positive or negative)? (e.g., “Holds decreased by 60%, exceeding prediction. However, Intake team reports increased workload.”).
  • Summarize the learnings.
A
ACT

Standardize, Adjust, or Abandon.

  • Adopt/Standardize: If the change was successful, formally adopt it. Update P&Ps, implement full rollout, provide ongoing training. Continue monitoring the relevant KPIs.
  • Adjust/Adapt: If the change was partially successful or had negative consequences, modify the plan based on the learnings and run another PDSA cycle. (e.g., “Refine the incomplete referral P&P to balance workload and re-pilot”).
  • Abandon: If the change was ineffective or detrimental, discard it and return to the “Plan” phase to develop a different approach based on the RCA.

Documenting the PIP

Accreditation requires that these PIPs are formally documented. This often involves a PIP Charter or Worksheet that includes:

  • Project Title & Goal Statement (SMART objective)
  • Team Members
  • Problem Analysis / RCA Summary
  • Proposed Intervention(s)
  • Action Plan (Tasks, Owners, Dates)
  • Data Collection Plan & Measures
  • Results / Analysis (Charts, Graphs)
  • Conclusions & Next Steps (Adopt, Adjust, Abandon)
  • Dates Reviewed by QMC

This documentation provides the evidence that your pharmacy has a functioning CQI process.

Your Role as a Pharmacist in PDSA/PIPs

As with RCA, your involvement is crucial:

  • Team Member: You may be assigned to a PIP team focused on improving a clinical or operational process you’re involved in.
  • Process Owner: You might lead the implementation (“Do” phase) of changes within the pharmacy workflow.
  • Data Collector: You might be responsible for tracking specific data points related to the PIP (e.g., number of interventions made using a new tool).
  • Feedback Provider: Your observations during the “Do” and “Study” phases are critical for determining if the change is working in the real world.

2.5.5 Conclusion: Embedding Quality into Daily Practice

Internal auditing and continuous quality improvement are not separate, occasional activities performed only by the “Quality Department.” They are the dynamic, action-oriented components of the QMS that must be embedded in the daily work and culture of the entire specialty pharmacy.

This section has provided you with the tools and mindset:

  • You understand that Internal Audits are proactive health checks, essential for finding risks before they become major problems, and you know your role as both potential auditee and auditor.
  • You can utilize Root Cause Analysis (RCA) tools like the 5 Whys and Fishbone Diagrams to move beyond surface symptoms and identify underlying system failures when things go wrong.
  • You grasp the iterative PDSA/PDCA cycle as the structured methodology for planning, testing, implementing, and standardizing improvements through formal Performance Improvement Projects (PIPs).

As a CASP, you are a leader in this process. You demonstrate quality through your meticulous documentation, your adherence to P&Ps, and your vigilance in identifying potential risks. You contribute to quality by participating actively and honestly in audits and RCAs. And you drive quality by embracing change, implementing corrective actions, and always seeking ways to improve patient safety, efficiency, and outcomes through the CQI process.

By integrating these principles into your daily practice, you move beyond simply reacting to problems and become an engineer of proactive excellence, ensuring your specialty pharmacy not only meets the complex demands of today but continuously adapts and improves for the future.