How Effective Is Your Analyst Training?

Training is essential before letting any analyst loose to work in a spectroscopy lab. But how effective is the training? Is training a one-off activity or an ongoing process?

What is the best way to train analytical chemists to follow standard operating and analytical procedures? Good Manufacturing Practice (GMP) ¹ and Good Laboratory Practice (GLP) regulations^2,3 state that to analyze a sample using a spectrometer, an analyst needs to have a combination of education, training, and experience to do the job. There is no set formula—merely the sum of these three elements. However, ISO 17025 - 2017, the quality standard for test and calibration laboratories ⁴, requires in section 6.2 the addition of qualification (education?), technical knowledge, skills, and competence. Laboratories need trained and competent analysts; this takes time.

Although the focus of this column is on training to execute standard operating procedures (SOPs) and analytical procedures, don’t forget that these topics are closely linked with data integrity⁵, data governance, and open culture.^6,7 You also need to understand the cost of noncompliance, as prevention through effective training is always cheaper than the cure after an inspection or audit.⁸

Overview of Training

The Principle of EU GMP Chapter 4 on Documentation states:

Good documentation constitutes an essential part of the quality assurance system and is key to operating in compliance with GMP requirements. …

There are two primary types of documentation used to manage and record GMP compliance: instructions (directions, requirements) and records/reports.⁹

The basic elements of this approach can be seen at the bottom of Figure 1: Define what must be done, confirm what was actually done, and document and record it. Corollaries are that if an activity has not been documented, it didn’t happen, and if the records are not attributable, they are graffiti.

Figure 1 also shows the scope of this column:

• Training plans and training records for each analyst
• SOPs and the options for training
• Training for analytical procedures

We will not discuss good documentation practices (for example, do not use a laboratory coat sleeve for recording data), although this is a key requirement for working in any laboratory. My focus here is on how analysts are trained to perform and record their work.

Training Plan With Training Records

Before discussing training methods, we need to get the basics right first. Everyone needs a training plan with associated training records, including evidence of any assessments of understanding and competence (see Figure 1). An input into the plan would be each individual’s education and prior experience. The training plan should be reviewed annually during each individual’s performance appraisal, especially if an individual’s role has changed. The plan would include any annual GXP refresher training required, including any changes to the regulations and their interpretations.^1-3

This is shown at the top of Figure 1. A key area of concern is training to ensure consistent interpretation and understanding of any procedure, and the methods for achieving this.

SOP Problems?

This all assumes that a laboratory has SOPs for specific tasks. In many cases, this is a gross assumption, as FDA 483 observations since 2006 have consistently shown that Procedures not in writing, fully followed (21 CFR 211.22(d)) and Absence of Written Procedures (21 CFR 211.100(a)) are usually in the top three of all noncompliances found in pharmaceutical inspections.¹⁰ Be warned!

• How well are your SOPs written?
  An earlier FOQ column discussed incorporating diagrams in an SOP to illustrate the activities, responsibilities, and data to be collected and interpreted.¹¹ What a novel thought!
• Are your SOPs sufficiently detailed?
  If not, there will be a cottage industry of individual interpretations.
• Are your SOPs too detailed?
  The converse of the point above would be writing the procedure for beginners, not trained users. Striking the balance between too much and too little detail can be difficult.
• Are SOPs only updated at the end of the review period?
  Or can user feedback be used to update a procedure easily at any time? This is a prerequisite for ensuring that an SOP is current. Contrast that with a bureaucratic and slow revision system where changes are just collated until the time for review. There are now gaps between procedure and practice.

SOP Training: The Good, the Bad, and the Ugly

As part of an onboarding process, a new analyst may be presented with an arm’s-length list of SOPs for the laboratory. Are they all relevant for this person? Does the person need to be trained on everything? Of course, this never happens in your laboratory. With the caveat that you have an adequate SOP, how will you train analysts to follow it?

• The Good: Instructor-Led Training (ILT)
  This can be done face-to-face or, more commonly, via video using a learning management system. The latter does not provide trainees with the opportunity to ask interactive questions. ILT provides a consistent message as a starter for laboratory consistency. Provide copies of the procedure and the training material so that trainees can make notes to enhance their understanding of what is required. Video clips or virtual reality can be used to illustrate all or parts of the procedure to enhance understanding. A proviso here is that the instructor knows and understands the material, does not just read the words on the slides, and engages with the trainees.
• The Bad: Read and Understand 1
  If you want a uniform understanding of how to execute an SOP throughout any laboratory, this is not the way to do it. It leaves interpretation of the procedure up to the individual reader, which is the start of analytical mayhem. With this option, you can check understanding with a multiple-choice questionnaire that we will discuss in the next section.
• The Ugly: Read and Understand 2
  The worst option is to read and understand without a check of understanding. It is simply a tick-box approach to training that will lead to problems later. Management prefers the read-and-understand approach because it is cheap. Let me give you some free consulting: DON’T!

Checking Understanding and Competence

The problem with any training approach is how you check understanding and competence.

• Multiple-Choice Assessment
  A very common approach is a multiple-choice questionnaire with 10-20 questions covering the scope of the training. One problem is determining how low the pass mark is. Does a 60% pass mean you can tolerate 40% of rubbish?
  What happens if you fail an assessment? Nothing? Or do you retake the training again? Are the wrong answers ever discussed with trainees? This approach may check understanding or a lucky guess of the answer, but does it confirm competence?
• Execute the Procedure
  To ensure understanding and competence, the best way is for each trainee to execute the SOP. Successful execution will demonstrate an individual’s competence far better than any questionnaire and will also highlight any areas that require further explanation. However, will management allow this?

Perhaps the best approach is to classify each SOP by risk, importance, or both. Execution is reserved for the most important procedures, with questionnaires for the remainder.

Analytical Procedure Training

This is the heart of an analytical laboratory. How well do analysts execute analytical procedures? It requires technical knowledge and expertise (for example, use of analytical apparatus, instruments, and systems); acquisition, interpretation, and recording of data; and competence in the procedure itself. Although GXP regulations require a combination of education, training, and experience, some of the best advice for analyst training is found in the FDA guidance on investigating out-of-specification (OOS) test results under Responsibility of the Analyst ¹² (this is also the same text in the 2006 version of the guidance):

• The first responsibility for achieving accurate laboratory testing results lies with the analyst who is performing the test.
• The analyst should be aware of potential problems that could occur during the testing process and should watch for problems that could create inaccurate results. …

The key point is making analysts aware of potential problems that could occur through knowledge management:

• Do you show an analyst the development and validation reports to read, or just the procedure? The development report should highlight the key parameters to control in the procedure and any impact of their variation.
• How many of your analytical procedures include descriptions of potential problems and how to avoid or resolve them?
• How are newly discovered problems documented, analyzed, and managed? Update the analytical procedure!

These are all important aspects of ensuring analysts are aware of problems, with feedback into a possible update of the procedure and analyst retraining as necessary.

Analytical Procedure Training

The best (only?) way to train an analyst is a three-phase approach. This can be used for a trainee but also for method transfer:

• Run 1: Experienced Analyst Demonstrates
  First, the trainee should read the procedure along with the development and validation reports to understand the aim of the procedure. Then, they should identify the key stages, apparatus, analytical instruments, and systems to be used. Is the analyst competent to use the apparatus, instrumentation, and systems? Videos or virtual reality could be used to help the trainee understand and execute the method, while also helping them identify potential problems and interpret data. Next, discuss the procedure and any problems with the trainer. The trainer should demonstrate the procedure to the trainee, especially highlighting potential problems and how to avoid them.
• Run 2: Trainee Executes With Trainer Oversight
  Next, the trainee executes the analytical procedures under the eye of the trainer. This allows the trainee to ask questions as the procedure is performed and the results are interpreted.
• Run 3: Trainee Executes Alone
  Working alone, the trainee executes the procedure with unknown spiked samples, and the results are compared to see how accurately the trainee has measured the analyte(s). After a debrief, and if the results are consistent, the trainee is signed off as competent to work on this procedure.

Evidence of the training runs, plus electronic records in any computerized systems used, should be maintained in separate directories or folders. Always use spiked samples, not production batch material, for training.

Problems will occur during operational testing, and the OOS guidance has the following advice:

• If errors are obvious, such as the spilling of a sample solution or the incomplete transfer of a sample composite, the analyst should immediately document what happened.
• Analysts should not knowingly continue an analysis they expect to invalidate at a later time for an assignable cause (that is, analyses should not be completed for the sole purpose of seeing what results can be obtained when obvious errors are known).¹²

This is critical advice for both personal and data integrity. If you have made a mistake, stop, document it, and go back to the stage where you can restart the procedure. The data acquired up to the mistake cannot be deleted or ignored because they are part of the complete data and subject to review under 21 CFR 211.194(a).¹ What should also happen is an investigation into why the mistake occurred. This is an opportunity to improve. Is the procedure adequate? Were there distractions in the laboratory or instrument problems? This is a key part of an open, no-blame culture within an overall data governance and data integrity program. Retraining may also be required, which leads us to the question.

Is Training a One-Off Activity?

No, is the simplest answer. As EU GMP Chapter 2.11 states: Continuing training should also be given, and its practical effectiveness should be periodically assessed.¹³ You can test analysts by providing them with blinded spiked samples that are both within and outside acceptable limits.

Quality Oversight

I want to focus here on GLP and ISO 17025 rather than GMP. The GLP Quality Assurance Unit (QAU) is mandated to carry out internal inspections of studies in 21 CFR 58.35(3)², OECD GLP No 1 section 2.2(c)³, and OECD GLP No 23.¹⁴ Section 7.4.1 of the latter states:

The selection of the experimental phases that should be subjected to an inspection is made by the QA personnel. The focus of the inspection should be placed on activities which may present the greatest risk for the compliance of the study.¹⁴

This means that a member from the QAU observes the work being carried out and it complies with the procedure. OECD 23 also states that a video inspection is a last resort. As a former ISO 17025 assessor, I would view the test procedures during a laboratory assessment to ensure that the assessment correctly executed and complied with the procedures. My presence could be disruptive, and I needed to observe from a distance, not like a parrot on an analyst’s shoulders.

Summary

We need trained and competent staff in our laboratories. Training for SOPs and analytical procedures is critical for the correct operation of any laboratory. This takes time, and management needs to recognize it as a long-term investment in staff and not a one-off event. Reading and understanding SOPs will result in different interpretations in practice, and instructor-led training is essential. A three-phase approach for training analysts to perform analytical procedures is suggested. Quality also has a role in ensuring that staff are trained by observing work being executed in practice.

Acknowledgments

I thank Monika Andraos and Chris Burgess for their constructive review comments during the preparation of this column.

References

(1) 21 CFR 211 Current Good Manufacturing Practice for Finished Pharmaceutical Products. Food and Drug Administration: Silver Spring, MD, 2008.

(2) 21 CFR 58 Good Laboratory Practice for Non-Clinical Laboratory Studies. Food and Drug Administration: Washington, DC, 1978.

(3) OECD Series on Principles of Good Laboratory Practice and Compliance Monitoring Number 1, OECD Principles on Good Laboratory Practice. Organization for Economic Co-operation and Development: Paris, 1998.

(4) ISO/IEC 17025-2017 General Requirements for the Competence of Testing and Calibration Laboratories; Geneva, 2017.

(5) McDowall, R. D. What is Data Integrity Training? Spectroscopy 2015, 30 (11), 34–41. Available at: https://www.spectroscopyonline.com/view/what-data-integrity-training

(6) GAMP Guide Records and Data integrity; International Society for Pharmaceutical Engineering, 2017.

(7) McDowall, R. D. Data Integrity and Data Governance: Practical Implementation in Regulated Laboratories; Royal Society of Chemistry, 2019.

(8) McDowall, R. D. Do You Really Understand the Cost of Noncompliance? Spectroscopy 2020, 35 (11), 13-22. https://www.spectroscopyonline.com/view/do-you-really-understand-the-cost-of-noncompliance-

(9) EudraLex - Volume 4 Good Manufacturing Practice (GMP) Guidelines, Chapter 4 Documentation. European Commission: Brussels, 2011.

(10) Inspection Observations. Food and Drug Administration, https://www.fda.gov/inspections-compliance-enforcement-and-criminal-investigations/inspection-references/inspection-observations (accessed.

(11) McDowall, R. D. The Write Stuff? Spectroscopy 2012, 29 (9). Available at: https://www.spectroscopyonline.com/view/write-stuff

(12) FDA Guidance for Industry, Investigating Out-of-Specification (OOS) Test Results for Pharmaceutical Production. Food and Drug Administration: Silver Spring. MD, 2022.

(13) EudraLex - Volume 4 Good Manufacturing Practice (GMP) Guidelines, Chapter 2 Personnel. European Commission: Brussels, 2014.

(14) OECD No 23 Advisory Document of the Working Party on Good Laboratory Practice on Quality Assurance and GLP. Organisation of Economic Cooperation and Development (OECD): Paris, 2022.