R.C. McDowall is the principle of McDowall Consulting and director of R.D. McDowall Limited, and "Questions of Quality" column editor for LCGC Europe, Spectroscopy's sister magazine.

R.D. McDowall

The need to calibrate and qualify analytical instruments and systems is well recognized in laboratories. However, calibrating and qualifying analytical instruments and systems are seen as discrete tasks. There is an increasing regulatory expectation that a risk-based life cycle approach is necessary to ensure “fitness for purpose” as part of an analytical procedure. The calibration and qualification lifecycle of analytical instruments and systems is a journey and not a sequence of disconnected events.

This article describes the principal stages of the lifecycle in both GXP or ISO/IEC 17025 laboratories that ensure that any analytical instrument and system is “fit for its intended use” as part of an analytical procedure, whether that is calibration or qualification. However, the tasks typically associated with these stages are often seen as discrete activities, but they should be integrated to form a continuum. In other words, the calibration and qualification lifecycle of analytical instruments and systems is a continuous journey, not a sequence of disconnected events. It is timely to consider a lifecycle approach because the United States Pharmacopoeia (USP) proposed a new General Chapter <1220> in September 2020 for analytical procedure lifecycle management (APLM) (1). There are no novel elements in this approach, but there is a structured process linking these activities that is described here.

Our starting point is the U.S. Good Manufacturing Practice (GMP) regulation for equipment design, size, and location in 21 

Equipment used in the manufacture, processing, packing, or holding of a drug product shall be of appropriate design, adequate size, and suitably located to facilitate operations for its intended use and for its cleaning and maintenance (2,3).

What does the phrase “intended use” mean and how should “fitness for intended use or purpose” be demonstrated in a regulated laboratory? Using our analytical instruments for analysis, we reviewed the high level process flow for the analytical procedure lifecycle from the proposed USP <1220> shown in Figure 1 (1). Notice that the key part of the process is stage 1 (Procedure Design, Development and Understanding). Method development is critical to the success of the whole process, and the draft USP <1220> covers an activity that the current International Council on Harmonization (ICH) Q2(R1) does not (4). Consequently, analysts developing methods must ensure that their analytical instruments are qualified and calibrated correctly and that the analytical systems are validated to ensure the smooth development and transfer of methods to the next stage of the lifecycle, which is called procedure performance qualification (PPQ). Qualification and calibration of analytical instruments is particularly important when analytical procedures are developed and validated in Research and Development (R&D) and later transferred to quality control laboratories.

Figure 1 shows that analytical procedure validation is no longer a discrete activity but a journey. As part of that journey, the first aspect of stage 2 PPQ requires qualification and calibration of instruments and systems as a prerequisite.

Analytical Process: Where Does Qualification and Calibration Fit? 

The analytical process, conducted within a pharmaceutical quality management system, provides reliable, traceable, and reportable values that are secure. The basic analytical process is shown in Figure 2 together with the relationship to stage 2 of the APLM highlighted in blue.

If we cannot demonstrate “fitness for purpose” of our instruments and systems, then outputs from our analytical process are unreliable. The current regulatory focus on data integrity and data governance requires us to put in place technical and procedure controls throughout the analytical process to ensure that data and records are trustworthy and reliable, and that quality decisions can be made in confidence.

The data governance and data integrity requirements and controls for a proposed analytical instrument and system (AIS) lifecycle that is defensible from a regulatory and technical perspective in Figure 2 can be mapped to the Foundation and Level 1 of a Data Integrity Model (5-8), shown in Figure S1, online. The Foundation ensures the right culture, ethos, and training for data governance and Level 1 ensures that instruments and computerized systems are calibrated, qualified and where appropriate validated for their intended use.

Analytical Instrument and System Lifecycle

The overall purpose of the AIS lifecycle is to demonstrate that not only is the instrument or system calibrated or qualified, but that any software is suitably validated and is operating in a controlled state. It is only then may we claim that the instrument or system is “fit for intended purpose” so that metrological data integrity is ensured. This article focuses on level 1 in this model concerning metrological integrity. The AIS lifecycle is shown in Figure 3. Data quality is only ensured if all components of data integrity and data governance are in place.

The levels of the AIS, with a specific focus on analytical instruments and systems, are presented at the bottom of Figure 3. Levels 1 and 2 are broken down into four elements each to provide metrological and procedural integrity, respectively. Levels 1 and 2 also overlay the culture and ethos of the foundation level. The quality oversight and some of the foundation level elements of the model are combined under data governance. Together these provide the essential requirements for the right reportable result at level 3 with associated data quality to ensure the correct decision is made.

The top portion of Figure 3 is based on the requirements of USP <1058>, analytical instrument qualification (9)that requires the following:

1) A laboratory user requirements specification (URS) to select the right instrument for the job (design qualification)

2) Following purchase of the instrument, a supplier IQ/OQ is per- formed to demonstrate that the requirements in the laboratory URS have been met. If this is not the case, the laboratory needs to conduct supplementary qualification tests to match the operating ranges specified.

3) Performance qualification (PQ) tests to demonstrate that the laboratory specifications are being met during operation of the instrument must be devised along with a plan for requalification of the instrument both after repair and maintenance but also periodically. 

4) The PQ test performance should be trended to detect any changes as well as maintaining deviation management and change control to demonstrate that the instrument is in a state of control.

The authors have published a risk assessment to help classify analytical instruments and systems into the USP <1058> groups and subtypes and suggest the extent of computerized system validation to be undertaken (10). Combining all the activities shown in Figure 3 ensures that analytical instruments and systems generate data and reportable results with appropriate integrity and quality.

Recently in Pharmacopoeial Forum, a stimulus to the revision process article was published discussing a lifecycle approach to both calibration and qualification (11). However, before we discuss this article, we need to consider the applicable FDA GMP regulations, by going back to the future.

Since the promulgation of the GMP regulations in 1978, approaches to ensuring that analytical instruments are “fit for intended use” have changed along with the terminology. Analytical instruments have changed and software, either in the form of firmware or a separate data system, now comprises a major component of modern analytical instrumentation. However, the wording of the regulations has not changed but the interpretation of them has. In 21 

 211.160(b) there are requirements that any laboratory control mechanisms must be scientifically sound and in 211.160(b)(4) we have the following requirement for calibration (3):

The calibration of instruments, apparatus, gauges, and recording devices at suitable intervals in accordance with an established written program containing specific directions, schedules, limits for accuracy and precision, and provisions for remedial action in the event accuracy and/or precision limits are not met. Instruments, apparatus, gauges, and recording devices not meeting established specifications shall not be used.

The records for such calibrations are defined in 211.194 (d) (3):

Complete records shall be maintained of the periodic calibration of laboratory instruments, apparatus, gauges, and recording devices required by 211.160(b)(4).

The problem is that the U.S. GMP regulations only mention calibration and therefore citations in 483 observations and warning letters will only cite deficiencies in instrument calibration when a qualification lapse could be the cause of the noncompliance, resulting in potential confusion. However, as our understanding of how to demonstrate fitness for intended use has evolved, so has the terminology.

The terms used within this section are one of the sources of great confusion between laboratories. For example, 

 can sometimes be used as a general “descriptive label” or have a specific meaning within the context of use. In addition to calibration, we now have the terms 

 also in common use. USP general chapter <1058> on analytical instrument qualification (9) provides an integrated approach to qualification of the analytical instrument and validation of the software. This general chapter mentions the terms calibration, qualification, verification, and validation. What do these terms mean? Are they linked? And where are we going in the future? All organizations, including regulatory authorities, need a technical glossary. There is no technical glossary that ensures a consistent meaning and use of terms throughout the pharmacopeia and other regulatory documents.

It is generally accepted that calibration forms part of the overall qualification process of an instrument or system. Calibration is all about metrology. Calibration is that part of qualification relating to measurement integrity of the ordinate response and abscissa functions, that is, all measurements from the instrument or system are within defined acceptance limits of a true or certified value.

The foundation for assurance of metrological “fitness for purpose” lies with the correct use of either a certified reference material (CRM) or a reference standard (RS). In the pharmacopeia, calibration is a comparison between a known or accepted value of a CRM or RS with its measured value on a specific laboratory analytical instrument or system being used for a pharmacopeial purpose the difference between these two values is compared with an acceptance criterion that may be an acceptable range or tolerance interval. For example, the measured value of a certified standard value of a wavelength standard must lie within the acceptance criteria of ±2 nm over the spectral range 200 to 400 nm. For some analytical instruments, calibration may all that is required (9).

Qualification is about overall “fitness for purpose” of an instrument or system, not only about good numbers. Hence the pharmacopeias require qualified instruments and systems. 

 <1058> is the only pharmacopoeial general chapter to describe the overall process of analytical instrument qualification (9).

 relates to the “fitness for purpose” of analytical procedures as required by the General Chapter on Verification of Analytical Procedures <1226> (12). However, verification is also used in the American National Standards Institute (ANSI) for software to refer to demonstrating that each individual step is mathematically or procedurally correct. GAMP 5 describes the testing phases of a computerized system lifecycle to demonstrate meeting user specifications (13). Therefore, some aspects of modular calibration (see later section) may be regarded as a verification process. The current version of 

 <1058> on analytical instrument qualification (AIQ) does not use 

Currently, validation relates either to the establishment of analytical procedures as required by General Chapter <1225>, Validation of Analytical Procedures (14) or particularly in General Chapter <1058>, Analytical Instrument Qualification for “fitness for purpose” (9). For example, in the “Instrument Control, Acquisition and Processing Software” section of <1058>, it states:

At the user site, integrated qualification of the instrument, in conjunction with validation of the software, involves the entire system. This is more efficient than separating instrument qualification from validation of the software (9).

However, there is a diversity of definitions for calibration, qualification, verification, and validation within the USP General Chapters that need to be harmonized to assist understanding and prevent user confusion. This is particularly true for the differentiation of holistic and modular calibration standards and the expression of acceptance criteria for establishing “fitness for purpose” (15,16).

The perennial question of “who does what and when” is answered in the lifecycle approach to calibration shown in Figure 4. The solid lines denote the primary process flows for establishing and confirming “fitness for purpose.” Dotted or dashed lines indicate either feedback loops triggered by data generated during the ongoing performance phase or, in the case of the URS, supported by the vendor.

The generic process of calibration and qualification should be designed to allow confirmation of “fitness for purpose” at all stages in the lifecycle including assurance of preventative maintenance and change control.

Users need to define their own control strategy to specify ongoing “fitness for purpose” by establishing the frequency of calibration, standards and acceptance criteria that will be employed and trend analysis of the ongoing performance data from the instrument.

One important aspect of the calibration and qualification lifecycle is the presence of change control feedback loops as part of the lifecycle. For example, if the qualified operational wavelength range for a UV spectrometer was originally 240 to 600 nm and the requirement is to extend this to 210 to 600 nm, then a new qualification standard is required to confirm “fitness for purpose” (17–19). In addition, the laboratory URS needs to be reviewed to ensure that the instrument was correctly specified originally to allow the revision of operational range.

Do You Specify Exactly What You Need the Instrument or System to Do?

 <1058> (9) is defined in a user specification. These are laboratory requirements for any analytical instrument with scientifically sound acceptance criteria. This does not mean simply copying the supplier’s specification, as discussed in an earlier column (18) but defining what is actually required. For example, if a UV-vis spectrometer has a supplier’s wavelength specification between 190 and 800 nm, what do you actually use? For example, if you only require measurement between 210 and 300 nm, then this should be your operating range. The requirement for wavelength accuracy is contained in 

 <857> (20) and this should be quoted in the specification document for the instrument. Testing wavelength accuracy using a CRM (for example, a holmium perchlorate solution or a glass filter) should demonstrate that this user requirement is met and that the instrument performs within scientifically sound acceptance criteria. Similar approaches need to be undertaken for the other technical requirements for the spectrometer.

Best Practices for Instrument Calibration

The best practice is to use the most stable artifact with the smallest uncertainty budget. For example, for routine monitoring of UV-vis and near infrared spectrometers, solid or liquid filters traceable to a national measurement institution (NMI), for both wavelength and absorbance accuracy qualification provide the most reliable option for the user.

For example, Figure 5 shows an overview of available standards for UV-vis-NIR over the overall spectral range available from one manufacturer. For example, to cover an operational wavelength range of 200 to 795 nm, two references would be required.

Articles

The calibration and qualification of analytical systems is a journey, not an event.

Are You Ready for a Life Cycle Approach for Analytical Instruments and Systems?

English is hopeless as there is no word for it. Spanish? Nope. French? Forget it. Chinese? Not a clue. In contrast, German has the word for it. What is it? Schadenfreude. Huh? Schadenfreude is defined by Wikipedia as the experience of pleasure, joy, or self-satisfaction that comes from learning of or witnessing the troubles, failures, or humiliation of another.Has this column turned into a language lesson? No. Rather, my goal is to teach you that it is better to be compliant than not.

Previously, my focus in previous “Focus on Quality” columns has been mainly on instances of noncompliance found in FDA Form 483 observations and warning letters, such as a recent review of cases of infrared (IR) spectrometer noncompliance (1). However, in this installment, I discuss the extensive and expensive remediation activities required by the FDA in two July 2020 warning letters issued for data integrity violations by two U.S. companies, Stason Pharmaceuticals and Tender Corporation (2,3).

What is the relationship of this to schadenfreude, you may ask? Wait until you see the scope and detail of the work required by the FDA to remediate the noncompliance by these two companies. It looks like the FDA is losing patience with the pharmaceutical industry and is sending an unmistakable message to companies to get their data integrity act together.

Use schadenfreude as described here to serve as a warning to check how compliant your laboratory processes and systems are now. Being inspection-ready is always preferable to having an inspector uncover noncompliant bodies that have been buried. Before we get into the miasma of these noncompliances, we need first to understand and discuss the costs of compliance and noncompliance.


The problem in the pharmaceutical industry is that many analytical chemists in the industry have never read the regulations they have to comply with, because the regulations have been interpreted in the past by the organization. However, reading, understanding, and interpreting regulations is critical. Take for example the U.S. Good Manufacturing Practices (GMP) regulations for equipment in 21 

 211.63. We can boil down the 35 words of this clause into three simple requirements, that instruments:

be suitably located for intended use (4).

That’s all it says. However, these words need to be interpreted in combination with 

A specification for an analytical instrument and any software must be documented in a user requirements specification (URS). The URS should contain statements about adequate size (for example, capacity, such as maximum sample numbers in a run) of the instrument or number of concurrent users if a networked system.

Design qualification (DQ) is required to demonstrate that the right instrument or system has been selected.

The instrument and any associated software must be installed correctly (installation qualification).

The instrument and any software must undergo an operational qualification and user acceptance testing against the URS to confirm fitness for intended use. This phase can include an appropriate mixture of calibrating, qualifying, and testing the whole system while establishing a baseline against which operation of the instrument can be monitored.

Monitoring the operation of the instrument over time against the baseline and user requirements (performance qualification) is required.

Given that the regulations state only the minimum required elements, these regulations have spawned an industry of writers to interpret these regulations through regulatory guidance documents, a pharmacopeial general chapter, publications from scientific societies, and articles by individuals like me (5–10).

Even with all these publications, there is still the choice of how to implement these regulations in an individual laboratory for each analytical instrument and computerized system. Interpretation of pharmaceutical regulations is always a balance between the cost of compliance against the cost of noncompliance. Remember also that any work in a GMP-regulated laboratory must be scientifically sound under 21 

 211.160(b) (4). Figure 1 illustrates the balance between the costs of compliance and noncompliance.

Cost of Compliance vs. Cost of Noncompliance

Risk management is one of the requirements for the pharmaceutical industry following the publication of the FDA’s 

 document and the ICH Q9 guideline on quality risk management (11,12). The work required in a regulated laboratory is dependent on a justified and documented risk assessment that must also be scientifically sound. This discussion can be summarized as the balance between the cost of compliance and the cost of noncompliance. Each laboratory makes decisions along a spectrum that ranges from doing nothing to doing everything possible, and this decision determines how much regulatory and business risk a company wishes to mitigate or carry as well as how much money the company wishes to spend.

The left-hand vertical axis of Figure 1 is the cost of noncompliance and the right-hand vertical axis shows the cost of compliance. You will note that the cost of noncompliance axis is much bigger than the cost of compliance axis. One viewpoint is that one axis is logarithmic, and the other is linear. Guess which one is linear? This is one of the balances you need to consider: The right-hand side shows the cost of doing it right the first time, and the left-hand side is the cost of getting caught. Fixing a regulatory problem that has been identified in a warning letter is always more expensive than doing the right thing the first time, or even of finding a problem and fixing it yourself. If anyone is in doubt about the cost of noncompliance for data integrity violations, I suggest that you read a consent decree, such as that for Ranbaxy (14). In that consent decree, the cost of noncompliance can be quantified as hundreds of millions of dollars.

In Figure 1, the horizontal axis is the percentage of compliance from 0 to 100%. The only fixed points are at the ends of the scale where 0% indicates no control of the process or system and 100% is where anything that can be compliant is compliant. In between 0 and 100% is a relative scale of compliance. The major point to note is that this scale is not fixed but moves as indicated by the arrow at the bottom of the figure. However, the direction of movement is only one way and that is to the right! To understand this point, consider the situation with data integrity. The FDA GMP regulations have changed little since 1978 and have always contained data integrity requirements. However, since the Able Laboratories fraud case in 2005 and the discovery of industrial-scale data falsification and poor data management practices, we have seen regulatory authorities issue guidance documents on the topic and enforce the regulations more strictly (15–20). Because of the proactive monitoring by regulatory authorities, the compliance scale has moved to the right.


Because data integrity has been a major compliance topic for the past 15 years, you would think that organizations would have taken the hint and started assessment and remediation projects already. Apparently, this is not the case. There have been many past warning letters highlighting the issues that are common in the two warning letters that we review in this column. Notwithstanding the availability of those letters on the FDA’s website, apparently little if anything was been done at Stason Pharmaceuticals or Tender Corporation to even assess if there were any problems; such an effort was solely the responsibility of senior management. Also, little if any effort had been made at these companies to assess the compliance landscape, see the trends in inspections, and plan ahead. Then again, even if there was the knowledge of compliance trends at those companies, would either organization have done anything different? One has to wonder. For those working in organizations that are aware of the data integrity issues and are working to remediate the problems, you can have a little schadenfreude smirk.

We are now ready to consider the cost of noncompliance of two warning letter remediation plans mandated by the FDA (2,3). These are as follows:

Stason Pharmaceuticals Inc. received two citations against 21 

 211.68(b). There were major problems with out of specification (OOS) investigations that resulted in a major list of remediation tasks. Our interest is in the second citation for failure to exercise appropriate controls over computerized systems so that only authorized individuals can make changes to records.

Tender Corporation has three citations, under 21 

 211.192. Here the citations were for failure to have complete data available from testing by deleting data, although there is also mention of common login and password for access, which is a 211.168(b) requirement. There was also a failure to have scientifically sound test methods and assessment of method problems. The final citation was for a failure to have a procedure for compliant handling and inadequate complaint investigations.

The main citations from both warning letters are shown in Figure 2. Interestingly, both organizations are U.S. companies and not located in India or China, reinforcing that data integrity is a global problem. Both warning letters were issued by the FDA’s Division of Pharmaceutical Quality Operations.

What is interesting is that Stason is cited under 21 

 211.68(b) for failing to control computer systems and Tender is cited under 21 

 211.160(b) for a failure of laboratory controls. However, each company is required to submit a comprehensive remediation plan for data integrity but from different clauses of the U.S. GMP regulations. So what? The what is a nearly identical list of remediation activities contained in both FDA warning letters that support my contention that the cost of non-compliance is significantly larger than the cost of compliance.

Both warning letters require comprehensive corrective action and preventive action (CAPA) plans, shown at the bottom of Figure 2, and the wording is identical:

A comprehensive, independent assessment and corrective action and preventive action (CAPA) plan for computer system security and integrity. Include a report that identifies vulnerabilities in the design and controls. Also include appropriate remediations for each of your laboratory computer systems. This should include but not be limited to…

Let us analyze this single paragraph in some detail:

 An extremely detailed and extensive report is required. Some of the words and phrases used to describe the remedial action required include 

describe in detail, strict, on-going control, enhanced, 

. This plan is not going to be written on the back of an envelope. Nor will it be cheap.

The work must be led and conducted by an external consulting company with extensive knowledge of data integrity, regulations, and computerized systems. It will also require much input from laboratory staff, which will slow down analytical work while trying to assess and remediate processes, systems, and documentation. A consulting opportunity.

Each computerized system in the laboratory needs to be assessed from the perspectives of unique user identities, password use, user roles with associated access privileges that avoid conflicts of interest, and application configuration to ensure protection of electronic records.

 Data process mapping needs to be conducted on all systems. For each workflow in a system the records generated need to be identified, and the controls in place (assuming that there are some) and the data vulnerabilities need to be noted. Data process mapping is an iterative process involving a facilitator and subject-matter experts from the laboratory and, where appropriate, the IT group. It does not state how many instruments or computerized systems are in either laboratory, but for a medium-sized laboratory, there might be 10–25 instruments.

The list cites both procedural controls and training for staff but also mentions technical controls (and the associated validation costs) for long-term resolution of the data integrity problems.

At the end of every warning letter is the following paragraph:

The violations cited in this letter are not intended to be an all-inclusive list of violations that exist at your facility. Inspections and audits sample and only identify problems seen during the visit.

Inspections and audits sample and only identify problems seen during the visit. The regulatory expectation is that a systems approach is taken and if there is a problem with user account management seen with one computerized system, then an assessment of all systems should be undertaken. In addition, during the assessment of processes and systems a new non-compliance is identified, fix it! This is in contrast to the traditional QA mentality to just answer the question asked and not to think wider.

From a single paragraph in each warning letter you can sense that this remediation plan will be extensive and expensive. This program of work is not a two-minute job and with external involvement in all aspects of the preparation, this work will not be cheap. Remember the sentence at the end of the paragraph: “This should include and not be limited to…more work may be required than listed in the warning letter.” This is a consulting opportunity by invitation of the FDA. My spectroscopic schadenfreude-o-meter is starting its inexorable rise into the red zone and warning lights are flashing down at quality control!

It is interesting that many organizations claim that they never have any money available for improvements or time to do compliance work. It is strange that following the receipt of a 483 and especially a warning letter, money and resources flow like water over Niagara Falls. Now you can begin to see how the cost of non-compliance is much more than the cost of compliance. With the former, you have the boot of a regulatory agency inserted in places it should not be inserted, combined with an urgent timetable from the company to resolve the non-compliance. In comparison, a well documented risk management approach that is defendable has a fraction of the cost. Not convinced yet that the cost of compliance is less than the cost of noncompliance? Before we see the evidence to support my contention, as we enter into the valley of noncompliance death, we need to discuss my data integrity model. This is necessary to understand the scope of data integrity within the context of a pharmaceutical quality system.

The analytical portion of my data integrity model has featured in this column before and the full model in other publications of mine (10,13,21–23). It consists of four layers analogous to building a house plus quality oversight and is presented in Figure 3. This model is important because we map the citations in the warning letters against the four levels of the model. The analogy with building a house is that if there is no foundation, the house collapses. This analogy is important to remember when we review the warning letters and analyze the remediation. Although the full model shows three levels of production and quality control (QC) analysis, we will only consider the foundation, analytical levels, and quality assurance (QA) oversight in the analysis of the warning letters. The layers of the model are:

Foundation (F): Data governance, which includes management leadership, quality culture, ethos, and training for data integrity including good documentation practices.

Level 1 (L1): The right analytical instrument and computerized system qualified and validated for intended use.

Level 2 (L2): The right analytical procedure for verification or validation under actual conditions of use.

Level 3 (L3): Executing the validated procedure using qualified instruments and validated computerized systems with staff trained in data integrity operating in an open culture.

Quality oversight (QA): Checks, audits, and investigations to ensure work at all levels has been performed correctly.

The whole of this model must exist within a company’s pharmaceutical quality system. Each part of the FDA-required remediation plan will be mapped to one or more parts of this model using the annotations in parenthesis in the bullet list above.

Here’s where we go into detail for the remediation CAPA plan. If you think the overview discussed above was bad, wait until you see what is coming. Figure 3 shows the minimum elements of the data integrity remediation required by the FDA of Stason Pharmaceutical and Tender Corporation. Item 12 in Figure 3 is exclusive to the latter company. Visualizing each plan is essential to have a better understanding of the scope and detail of the remediation required, which are extensive. Please note that the elements of the plan in Figure 3 have had the text edited so that the figure is manageable.

In contrast, Table I presents the data integrity remediation tasks in both the Stason Pharmaceuticals and Tender Corporation warning letters, and the one additional requirement required of Tender Corporation is added at the bottom of this table. Table I consists of three columns:

The left-hand column lists the verbatim requirement copied from the
warning letters.

The middle column shows the requirement mapped against the data
integrity model.

The right-hand column shows my comments on the remediation requirements

We discussed the main features of a data integrity model earlier and each of the remediation requirements of the FDA in Table I are mapped against the layers of this model. Figure 4 shows the required remediation activities mapped visually against the model. This mapping is illuminating because both organizations have failed to implement basic GMP requirements in terms of data governance and associated training at the foundation and failed totally at level 1 to ensure that analytical instruments and computerized systems are fit for their intended use.

This figure also reinforces a key point that has been made since the inception of this model: If you don’t get the lower levels of the data integrity model correct, then whatever data integrity efforts are implemented at the upper layers become worthless. Of interest is the fact that level 2 is not mentioned at all in the remediation requirements because of the scope and depth of the non-compliances cited in Table I were such a shambles that the inspectors did not need to look closely at how analytical procedures were verified or validated.

The issues mapped to level 3 are, in part, a direct result of failures at the foundation and level 1. For example, because audit trails have not been implemented and validated at level 1, there is a consequential requirement for a procedure and training on how to review the entries during batch analysis at level 3. Security and access control appear conspicuous by their absence before the inspection as there is a major remediation effort to attribute work to individuals and avoid conflicts of interest when accessing systems, another level 1 failure. The failure to meet the requirement for complete data to be generated during testing is a fundamental failure in training staff in good documentation practices and data integrity including the meaning of 

Continuing with omissions in the foundation, the spotlight now turns full beam on management. Management is not treated very kindly as there appears to be an abject failure of executive leadership with respect to data integrity and data governance. This is a fully deserved citation as data integrity begins and ends with executive management. Management is responsible for the pharmaceutical quality system and this is made crystal clear in GMP and good laboratory practices (GLP) regulations (24–26) as well as in regulatory guidance documents on data integrity (16–20). Any data integrity program requires resources, time, and funding for the assessment and remediation of processes and systems including updating or replacing inadequate systems. Executive management bears sole responsibility for this work.

The QA departments in these firms also appear to be inept, given that FDA’s remediation has the explicit requirement for quality staff with IT knowledge. It seems likely that the focus of quality oversight was only on paper records, possibly with paper printouts from the computerized systems being defined as the “raw data.” This is despite the fact that the FDA has had a level 2 guidance on why electronic records take priority over paper printouts available since 2010, which is publicly available on the agency’s web site (27). This position was reiterated in the 2018 data integrity guidance from the agency (19).

There is an initial remediation emphasis on procedural controls but the longer-term focus is directed by the FDA to technical controls. Overall, there should be elimination of paper by interfacing instruments, such as pH meters, and analytical balances to a laboratory information management system (LIMS). This is a crucial approach as procedural controls are operated by humans but consistency and identifying errors are not a given. Technical controls that have been validated can be operated consistently and enforce a procedure and hence GXP compliance, this is a far better approach compared to procedural controls.

Sometimes I get a feeling that an independent assessment of the current working practices will open one or more cans of worms for the company, and my schadenfreude-o-meter is ready to tick up a notch or two.

The focus of the FDA’s remediation requirements at level 1 of the model appears to indicate that both organizations did little more than install all analytical systems and then operate them in default mode. The systems themselves were either poorly designed (no database or audit trails were never turned on) or badly implemented, because it was possible to delete data. Because there are numerous citations for data deletions or failing to report all data, the remediations will be very expensive, coupled with the two independent assessments of systems and working practices.

In addition to the work required under the comprehensive and independent CAPA plan listed in Table I, there is a second major data integrity remediation requirement for Stason:

A complete assessment of documentation systems used throughout your manufacturing and laboratory operations to determine where documentation practices are insufficient. Include a detailed CAPA plan that comprehensively remediates your firm’s documentation practices to ensure you retain attributable, legible, complete, original, accurate, contemporaneous records throughout your operation (2).

How best to summarize this paragraph? Your quality management system is terrible. Fix it. This is a massive project in addition to all the other remediation work. Here the whole organization’s quality management system (QMS) and all methods of recording regulated activities, both in the laboratory and in production, have been brought into question. If the agency cannot trust the data, what hope is there for the company other than a complete reassessment of policies and procedures within the QMS? This work must run in parallel with the data integrity remediation discussed above. There will be multiple interactions between the two projects to ensure attributable, legible, contemporaneous, original, and accurate (ALCOA) principles are applied while completing data and record integration into the foundation of what the company does. This work applies not just to paper but also to hybrid and electronic records. This extensive remediation project can be mapped to the foundation level of the data integrity model.

Noncompliance Can Seriously Damage Your Wealth

When a company receives an FDA warning letter, there are several consequences, many of which will result in additional costs that we have not covered yet.

Reactive Compliance Approach: Laboratories that have a reactive compliance approach, summarized as “wait until a noncompliance is found and then fix it,” will find that the exponential cost of remediation action makes this approach extremely expensive, dangerous and foolish as seen in Figure 1. A proactive compliance approach of “do it right first time” is the only way to work now.

Regulatory Credibility: A laboratory’s regulatory credibility is lost, not just with the FDA, but with global regulators as inspection information is shared under mutual recognition agreements. For example, when FDA places a foreign company on import alert, Health Canada frequently follows suit even if they have not been involved with the inspection. The result will be increased scrutiny during subsequent inspections from all authorities. This means ensuring that the organization is in a state of constant inspection readiness. For serial offenders, there is an option of a consent decree of permanent injunction that binds the company to be compliant with the regulations in perpetuity and that also may be associated with large fines (14).

Informing Your Sponsors: This entire situation is even worse for a contract research organization (CRO) or contract manufacturing organization (CMO). Here one lucky individual draws the short straw to phone all sponsors to inform them of their regulatory failings. This task is guaranteed to raise any individual’s blood pressure and may lead to a loss of business with existing and potential sponsors. In some cases, work supporting a regulated product may need to be repeated, as sponsors found with studies carried out by Cetero Research (28,29).

Share Price Impact: Information about a warning letter or even the possibility of one can result in a fall in the company share price. For example, on February 15, 2019, the price of shares of Dr. Reddy’s Laboratories fell nearly 29% in a morning on the Bombay Stock Exchange before ending the day more than 4% lower than its previous close. This was because of the fear of a further FDA warning letter for the company. In a situation like this, management needs to think about whether their stock options are worth anything.

We have discussed how to interpret pharmaceutical regulations with the careful balance between the cost of compliance with the cost of non-compliance. The two warning letters discussed here clearly demonstrate that it is easier and cheaper to be compliant rather than face the consequences of remediating noncompliance. Will organizations listen, eliminate paper, and move to automated processes with validated technical controls to ensure compliance and data integrity? That is the $64,000 question (cost of compliance) or $64,000,000 question (cost of non-compliance). If your organization is compliant, you can experience schadenfreude.

I would like to thank John English whose work of posting FDA 483 observations and warning letters on LinkedIn led to the genesis of this article, and Chris Burgess, Paul Smith, and Kevin Roberson for helpful comments during preparation.

Warning Letter Stason Pharmaceuticals, Inc.

“Current Good Manufacturing Practice for Finished Pharmaceutical Products,“ in 21 

 211 (Food and Drug Administration, Sliver Spring, Maryland, 2008).

General Chapter <1058> “Analytical Instrument Qualification,” in 

 (United States Pharmacopeial Convention, Rockville, Maryland, 2002).

Guidance for Industry General Principles of Software Validation

Good Automated Manufacturing Practice (GAMP) Guide 

(International Society for Pharmaceutical Engineering, Tampa, Florida, 2nd ed., 2012).

Good Automated Manufacturing Practice (GAMP) Guide

 (International Society for Pharmaceutical Engineering, Tampa, Florida, 2008).

Two recent warning letters show that the US FDA is substantially increasing the amount of remediation work it requires for companies to correct data integrity noncompliance. That work can be very expensive—far exceeding the cost of ensuring compliance in the first place.

Do You Really Understand the Cost of Noncompliance?

Universally available on desktop computers, easy to use, and universally loved by analytical chemists, spreadsheets are used to collate lists of instruments and perform a multitude of calculations in many regulated laboratories. But spreadsheets also present an auditing and inspecting opportunity to find abundant instances of noncompliance and can lead to problems and unnecessary risk. Why is the use of spreadsheets heavily discouraged in a regulated laboratory?

If you turn on any workstationin your laboratory, you will probably find a shortcut to a spreadsheet program. When presented with this icon of temptation and seduction, a willing analyst is happy to open Pandora’s spreadsheet box and navigate between Scylla and Charybdis. Charybdis is the whirlpool of quality assurance where you have the challenge of specifying, building, and validating each spreadsheet template. Alternatively, steer the course to Scylla, the six-headed monster of regulatory noncompliance. With Scylla, if the unvalidated spreadsheet you have written and used is found, you are dead.


Spreadsheets are often used to perform GMP–related calculations, trend data, plan calibration and qualification work, and manage laboratory assets. I have even seen spreadsheets used to document the test steps for validating software—and the least said about that the better. This article discusses why spreadsheets should be eliminated as much as possible from performing GXP work to increase business efficiency and reduce regulatory risk. Note that business efficiency came before reducing regulatory risk in the last sentence. This column is dedicated to all spreadsheet lovers, or in the words of that great analytical chemist, William Shakespeare, “I come to bury spreadsheets, not to praise them.”

One of many companies that have sailed into Scylla and been cited for noncompliant spreadsheets is Tismore Health, as seen in a Federal Drug Administration (FDA) warning letter in December 2020 (1). I hope this does not happen in your laboratory. Citation 3 is against 21 

211.160(b) (2), although I would have raised the citation against 211.68(b) for failing to verify formulas. However, that is the problem with the FDA’s last century GMP regulations:

Your firm failed to establish laboratory controls that include scientifically sound and appropriate specifications, standards, sampling plans and test procedures designed to assure that components, drug products conform to appropriate standards of identity, strength, quality, and purity (21 CFR 211.160(b)).

Your firm failed to validate the spreadsheet used to perform the assay calculation for your <redacted>. Your procedures lacked guidance on how to check and manually verify the calculation sheets. During the inspection, our investigator identified a calculation error within the spreadsheet. The incorrect formula for averaging the Internal Standard peak area was used (1).

If you really want to descend into the abyss of compliance hell, let an inspector find an error in a spreadsheet. You will receive a comment like this: “There is no assurance that the associated assay results recorded are reliable and accurate.”

It is not the best situation having an inspector that thinks that all of your data are garbage. Also, if there is one calculation problem in unvalidated spreadsheets, then there are bound to be more, because there is a total lack of control when using spreadsheets.

During the review, you identified another error within your spreadsheets. The assay test result for <redacted> batch <redacted> was incorrect due to a transcription entry error for active peak area. Your firm used a new spreadsheet and entered the correct active peak area. The result was recalculated, and the final result was reported. The product had already been released with test results using the incorrect calculation although the recalculated test result was still within specification. You have committed to manually check calculations until the spreadsheet has been validated (1).

In the last sentence, we see a way to resolve the problem: You should manually check all calculations until the spreadsheet is validated. How useful are the unvalidated spreadsheets now? Read the full warning letter to see the extent of the remediation required by FDA (1). However, the approach taken by the company to manually check calculations (which is error prone in itself) misses the point about spreadsheets: They are a hybrid system that has major business process efficiency issues as well as regulatory compliance problems, as we shall see later in this column.

Here goes a list of the spreadsheet screw-ups. That well-known pharmaceutical company, Enron Corporation, used spreadsheets for a number of activities, and after the company went bankrupt, more than 15,000 spreadsheets were analyzed from the archive of 65,000 e-mails by two academics, Felienne Hermans and Emerson Murphy-Hill (3). Their main conclusions were:

Out of all the spreadsheets used, 76% of them used the same 15 functions.

In regards to spreadsheets with at least one formula, 24% of them contained an error.

Spreadsheets were a frequent topic of e-mail conversation with 10% of emails either referring to or sending spreadsheets and frequently discussing errors in and updates to spreadsheets.

Just like in your laboratory! In fact, if spreadsheets are well embedded into your culture, a spreadsheet developed in the morning could be the department standard by the afternoon. If so, I’m assuming you are steering a course for Scylla rather than Charybdis.

In a study that had a major impact on governments, two eminent economists, Carmen Reinhart and Kenneth Rogoff, published an article “Growth in a Time of Debt

which concluded that counties with a high amount of debt to Gross Domestic Product (GDP) (>90%) have slow economic growth. This resulted in austere policies to reduce debt and simulate growth in some countries, including the United Kingdom. Unfortunately for the two intrepid economists, they published in a non-peer reviewed part of a journal and used a spreadsheet to perform their calculations. And guess what happened? There were a few problems with the spreadsheet and the major conclusion was reversed (see Wikipedia for more details).

If you really want to understand the scope of spreadsheet disasters, take a look at Ray Panko’s website (www.panko.com). Panko is a University of Hawaii academic who has studied spreadsheets and errors associated with them for about 25 years. On his website, there is a page on spreadsheet errors that states the following (4):

Humans are very accurate when we work. When we create a spreadsheet formula, we are correct 95 to 99% of the time.

Unfortunately, we are not as good at finding errors. When we look diligently for errors, we only find 40 to 90%, and finding 90% is rare.

We would like to know how effective we are at finding errors when we inspect a spreadsheet. This is not just an “academic” concern. If our spreadsheets have as many errors as they seem to have, then we must test them more. One way to do this is to inspect them cell by cell. Many people believe that if we are careful enough, and if we spend enough time, we can find all of the errors in a spreadsheet. However, this ability is not realistic. If detection rates are too low, then a team inspection is necessary to catch 80 to 90% of the errors (4).

The inability to find and resolve spreadsheet errors is a worrying problem. How should we resolve it? The obvious one is that more resources need to be put into specifying the spreadsheet and how the formulas are built and tested. As Panko’s website suggests, more effort should be placed in checking the formulas. However, there is a simpler way, which is explained later on in this article.

In a section guaranteed to ensure that many readers will want to stick large pins into my effigy, I am advocating that spreadsheets be exterminated from use in a regulatory laboratory. This is not because a spreadsheet application is bad per se, but in a regulated environment it is a recipe for potential disaster, for many reasons, as outlined in Table I.

I’m not interested in your personal life, but look at your business process in the laboratory when you are using a spreadsheet. Figure 1 shows the tasks of a general quantitative analytical process in the middle in green. If you use a spreadsheet, you’ll follow the left hand side of Figure 1: Prepare the sample, analyze it along with standards, interpret the data in the spectrometer data system, print the results, manually type data into your (hopefully) validated spreadsheet, print the results out, and then type the result into a laboratory information management system (LIMS) or similar informatics application. But just look at the mess of the process and the number of records that you have created. You are a laboratory masochist.

Job done? No, enter the poor sap who is the second person reviewer. Just look at the mess this person has to review: the number of transfers between computerized systems, and the need to check for transcription errors each time data are entered manually into a different computerized system with checks of both electronic records and paper printouts. Welcome to death by compliance: transcription error checking by the reviewer, which is also an error-prone process. If you are really unlucky, the second person review takes longer than the analysis (10)!

Guess what? Instead of the mess on the left-hand side of Figure 1, many calculations you have spent time incorporating into a spreadsheet can be done by the instrument data system—except few people bother to read the manual. You can avoid the need to print and manually enter data into a spreadsheet followed by the consequential transcription error checks. Take the time to specify and validate the calculations, and you can save much time and effort in performing the analysis and the subsequent second person review. Look at the right-hand side of Figure 1 and compare with the left-hand side and immediately you’ll see my point. You only have the sample preparation records, the manual input of data into the data system, and the electronic records in the data system. Add electronic signatures and your process is sped up immeasurably, and you only have a single location to check records. Why would you want to do anything differently?

As mentioned earlier, let us discuss two of the main problems with spreadsheets in more detail, which are hybrid systems and single accounts and passwords.

A spreadsheet macro or template is a hybrid system—an electronic record of the saved and completed file with a hand-signed paper printout. Poor design means that data are typically input manually, and there is a large transcription error checking burden placed on both the analyst and the second person reviewer. One typical omission in laboratories I have audited is the lack of record signature linking between the paper printout and the electronic record that generated it. It is not an option to state that the paper is the GXP record; the spreadsheet file is a key component of the record set of any analysis. Please read question 12 of the FDA data integrity guidance to understand what is required:

12. When does electronic data become a CGMP record?

When generated to satisfy a CGMP requirement, all data become a CGMP record. You must document, or save, the data at the time of performance to create a record in compliance with CGMP requirements, including, but not limited to, §§ 211.100(b) and 211.160(a) (11).

This is very easy to understand and follow: Save the spreadsheet file or book an appointment with Scylla.

One of the issues with spreadsheet security is that there is only a single account and password for a template. Once the developer has locked the spreadsheet (this can be individual worksheets as well as the whole template) with one or more passwords, what are you going to do? This is one case when you will write the password down and follow the steps below:

The developer needs to write the spreadsheet name and version along with the passwords, and place the paper into an opaque envelope, write the spreadsheet and version name on the front, seal the envelope, then sign and date over the seal. This is to indicate if the envelope has been opened.

The envelope should be placed in a secure location, typically a fireproof safe, until the spreadsheet needs to be updated.

The envelope is unsealed, the spreadsheet maintained and revalidated, and the new passwords start the process over again.

If you find this a bit of a pain, then get rid of the spreadsheet.

Having spent all this time saying that you should eliminate spreadsheets, I must acknowledge that there are some exceptions to this rule. There are situations where calculations across analyses cannot be performed by instrument data systems and, in the absence of a LIMS, a spreadsheet may be the only option for calculating results. However, do you want to input data manually, and what about tracking changes to data? Ideally, a validated spreadsheet should be used within an electronic lab oratory notebook (ELN) or equivalent environment. Here, data should be imported and loaded electronically with a validated process to eliminate transcription error checking, and any changes are monitored by the audit trail of the informatics application. Calculated results should be transferred electronically to the next part of the process for speed and to eliminate transcription error checking.

When it comes to persuading laboratories to use alternative ways of calculating other than spreadsheets, I feel like King Cnut, an 11th Century King of England and Denmark, who commanded the sea to retreat. This was an obvious exercise in abject failure. However, never mind regulatory risk. Look at your business processes and see what they look like; it may not be a pretty sight. Incorporate calculations into instrument data systems or use informatics applications such as ELN or LIMS to move away from spreadsheets into an electronic way of working. Slaughter spreadsheets, please. You know it makes sense.

Warning Letter Tismore Health and Wellness Pty Limited

, Part 211 (FDA, Silver Spring, Maryland, 2008).

F. Hermans and E. Murphy-Hill, “Enron’s Spreadsheets and Related Emails: A Dataset and Analysis,” paper presented at the IEEE/ACM 37th IEEE International Conference on Software Engineering, Florence, Italy, 2015.

WHO Technical Report Series No.996 Annex 5 Guidance on Good Data and Records Management Practices (World Health Organisation: Geneva, Switzerland, 2016).

, Part 21.11 (Food and Drug Administration, Washington, DC, 1997).

European Commission Health and Consumers Directorate-General, 

The Rules Governing Medicinal Porudtcs in the European Union

Good Manufacturing Practice (GMP) Guidelines, Annex 11: Computerised Systems 

Guidance for Industry Data Integrity and Compliance With Drug CGMP Questions and Answers

 is the director of R.D. McDowall Limited and the editor of the “Questions of Quality” column for LCGC Europe, Spectroscopy’s sister magazine. Direct correspondence to: 

Spreadsheets are often used to perform GMP-related calculations, but can lead to serious problems and unnecessary risk. We explain why the use of spreadsheets is heavily discouraged in a regulated laboratory environment.

Spreadsheets: A Sound Foundation for a Lack of Data Integrity?

The U.S. Food and Drug Administration (FDA) recently issued an updated version of its Compliance Policy Guide for Pre-Approval Inspections (PAI). If you are not aware of the impact on analytical procedures and data included in regulatory submissions, then please read on.

	One of the U.S. FDA’s Compliance Program Guides (CPG), numbered 7346.832, is entitled Pre-Approval Inspections. This document is used to guide and inform inspectors conducting this type of inspection before granting a license to market an active pharmaceutical ingredient (API) or medicinal product for the U.S. market. The Able Laboratories fraud case in 2005 was a turning point for this document. The Agency had conducted seven PAIs of Able and not found any evidence of falsification. A whistle-blower in the quality-control (QC) laboratories at Able then informed the local FDA field office of the data integrity issues in the company. The following day, a posse of inspectors rode into town and camped at the Able facility looking at electronic data and comparing these records with the printouts that had been the focus of earlier inspections. Their findings are summarized in the 483 observations issued in July 2005 that resulted in the company’s going bankrupt. The Able 483 is still available on the FDA web site (1).

	The FDA realized that their inspection approach of paper review was inadequate, and that inspections had to focus on the electronic records in laboratory computerized systems. As a result, paper records became incidental to the inspection of computerized systems. This required rewriting the existing CPG 7346.832 (yes, there is a “Star Trek” fan working at the FDA!), and the new version was issued in May 2010 and became fully effective in May 2012 (2). This gap allowed time for industry to understand the new inspection approach, as well as time to train inspectors on ways for inspecting electronic records within the various data systems used in regulated GMP laboratories.

	The key changes in the rewritten version were the three inspection objectives (2):

		Objective 1: Readiness for Commercial Manufacturing

		Objective 2: Conformance to the Application

	Reading this list of objectives, one could be forgiven for  assuming that the only data integrity focus was objective 3. However, you would be mistaken; data integrity runs throughout all three objectives. This inspection approach, coupled with how to inspect electronic systems, has been used since 2012, and as a result there have been an increased number of 483 observations and warning letters issued to laboratories concerning data integrity.

Updating CPG 7346.832, Take 2: September 2019

The Food and Drug Administration (FDA) has issued an updated version of its Compliance Policy Guide for Pre-Approval Inspections (PAI). This article will guide you on this policy’s impact on analytical procedures and data included within regulatory submissions.

R.D. McDowall

Articles

Are You Ready for a Life Cycle Approach for Analytical Instruments and Systems?

Do You Really Understand the Cost of Noncompliance?

Spreadsheets: A Sound Foundation for a Lack of Data Integrity?

Do You Understand the FDA’s Updated Approach to Pre-Approval Inspections?