Validation of Spectrometry Software: Applying an Integrated Approach to Equipment Qualification and Computerized System Validation

In the last column (1), I outlined a suggested process for the integration of equipment qualification and computerized system validation for a spectrometer. In this column I would like to describe how this process should be used in practice with a near-infrared (NIR) spectrometer and computer system for the identification of raw materials in a manufacturing environment.

R.D. McDowall

Let's start from the beginning through to the release of the system for operational use to understand the process in more detail. I did not cover the need to specify the defined use of the instrument and associated software and computer system in my last column, so we'll start the discussion from this point. (I'm also assuming that you have not already purchased the instrument and been using it for some time.) As always, you'll need to modify and adapt this process to your specific situation.

In the Beginning . . .

The first part of the integrated analytical instrument qualification–computerized system validation journey is writing the user requirements for both the instrument and the software to document the system's intended use. Therefore, you must define the following items and document in the specification document for the system. (Note: I'm not using the word "instrument" quite deliberately for many of the items below and in the rest of the column.)

• The intended use of the system: In our NIR spectrometer example, the system will be used for the identification of raw materials in a goods inwards facility; therefore, we will consider a qualitative mode of operation only.

• Instrument operating parameters: Some of the considerations are to define the operating range of the instrument in region of 750–2500 nm. What resolution will be required of the instrument to be able to discriminate between different samples that might have similar structures? How will the spectra be acquired: in the instrument or using a probe? Automatic sample carousel or manual measurement?

• Sample presentation: How will samples be presented to the instrument? This needs to be discussed and debated before the system is purchased. Document in the user requirements specification (URS) the sample type (solid, liquid, or semisolid), container type, and measurement mode (reflectance, transmittance, diffuse reflection, or transflectance). Will measurements be made manually, or will there be automation? The decision will depend on the expected numbers of samples to be identified.

• Spectral acquisition, review, and storage: Define what is required here for the use of the system. How will the sample spectra be acquired? How will you store and manage the spectra? For identification, you might not require the sophisticated chemometric tools that a quantitative system would need; therefore, a more basic set of requirements should suffice — but you need to define them.

• Library: The instrument will need to acquire and manage spectra used to identify the raw materials. Will there be a single library, or will you need more than one library depending on the discrimination of the instrument or the type of products made within the facility?

• System positioning requirements: Will the instrument be located in the laboratory or will it be placed in the warehouse?

• Security and access control: Who will be using the system? The needs of the various users will require appropriate access controls. The main users of the system will need more basic user privileges compared with the system manager and a user who can import spectra into the library. Security might not just be around the computerized system, but might also pertain to physical security if the system is placed in a warehouse. Does the system need to be locked in a specific room?

• 21 CFR 11 requirements such as backup and recovery of the library, sample spectra, application software, and the operating system: Will electronic signatures be used by the operator and the reviewer of the results?

• Interfacing requirements: Will the system be standalone or be connected to the company network? Will data be transferred from the spectrometer to another computerized system such as a laboratory information management system (LIMS)?

• Regulatory requirements for NIR spectrometers: The European Pharmacopoeia (2) and United States Pharmacopoeia (3) need to be taken into account both in the initial specification of the system as well as in the qualification tests for the instrument. The two chapters are too long to summarize here, so I will only refer to them as needed, but please keep them in mind as we go through the overall process. However, there is one gem in the European Pharmacopoeia on data storage: "store the electronic NIR, libraries and data according to the current regulations." As you can see, this is very specific (2).

Caveat Emptor

Let's face facts — if you get this part of the life cycle wrong, you will be buying the wrong system. No discussion and no debate. But most people will ignore this advice and just purchase the system. However, if you have not adequately defined your initial requirements correctly, you'll waste money. Also, if you have not thought the problem through well enough, there might be issues when it comes to operating the system. For example, in the outline specification above, the system is being used for a qualitative function. What would happen if you wanted to use it in a quantitative mode as well? Would the same system be suitable? Think things through first and document the requirements in the URS.

The Great Day Arrives

Fast forward to when the purchased system arrives at your laboratory. OK, I've missed the boring bits like initial vendor selection, the delights of reading misunderstood responses to your URS, hands-on evaluation of the short listed systems, and justification of the final system choice; but the emphasis of this column is the integration of instrument qualification with computerized system validation.

You'll recall from my last column (1) that the process that I outlined consisted of the following five steps:

• Component installation: Installing the instrument components and the hardware and software for control, data acquisition, and analysis.

• Component integration: Integrating the components into the system and showing that they communicate together.

• Vendor commissioning: Confirming that the whole system works correctly, including a holistic test of the overall system.

• Defining user ways of working: Calibrating and qualifying the instrument over the operating parameters specified for the laboratory, configuring the software for the laboratory and user base, and where necessary, customizing software for specific working practices such as macros.

• User acceptance: Conducting system tests to demonstrate the intended purpose, specific software tests, and an overall capacity test if appropriate. These tests should be based upon and traceable back to the documented user requirements.

Component Installation and Integration

I'll deal with these two stages as if they were one to save some repetition. These are typically the responsibility of the vendor and must be carried out by them if your warranty and service contract is to be valid; however, it also means you are not wasting your time trying to get the system to work. The process starts from unpacking the instrument and installing it in the location where it will operate. To save time, money, and wasted effort, you will of course install the instrument where it will be used operationally, won't you?

The vendor service engineer or representative will unpack the components and check that each one is undamaged. The components will be installed by following a written plan provided by the vendor, and the engineer will confirm that each portion has been adequately completed by completing the document. In the event of errors, the engineer will resolve these before completing the installation of a component. For the instrument, the unit will be taken out of the box and positioned on the work bench where it will operate. The instrument should be connected to the power and checked to make sure that it turns on OK. If a sampling probe or autosampler is part of the overall configuration, these items will also be installed by the engineer with appropriate documentation.

After the instrument is installed, the computer is installed followed by the application software. For ease of discussion, I am assuming that only one application is required. The installation of this must also be adequately documented by the vendor. If the workstation is supplied by the vendor, it might come preinstalled with the operating system, typically Windows. Alternatively, if the system is to be connected to the company network, the company IT department might supply a workstation with the appropriately configured operating system preinstalled. Regardless of the source of the workstation, it can be connected to the network and given an Internet Protocol (IP) address and other information for it to connect and store data on a network drive. Regardless of who does the work, the fact that it was done must be recorded in a compliant manner. Then the application software will be installed by the engineer; again, this will be performed and documented in a procedure that will be completed on site together with problem resolution.

When completed, the instrument and the computer system will be connected and checked out to see that the two communicate as expected by the vendor. In outline, this completes what should be done in the first two stages of the work. Most, if not all, will be performed by the vendor's engineer and will be adequately documented. It is also the laboratory's responsibility to ensure that the work is recorded in a compliant way and that the work is of a suitable scientific soundness to comply with current good manufacturing practices (GMP) regulations (4). The system owner will review this work and approve it.

Vendor Commissioning

This stage can either be a separate activity after the component installation and integration or it can be combined with them in a continuous session to make it simpler for the engineer to complete. If the latter approach is taken, the process will be from unpacking the individual components to showing that the complete system works and is acceptable from the vendor's perspective. As mentioned above, the instrument and software should work and there should be a holistic test for the overall system.

Here you should be vigilant and work smarter rather than harder. What is the vendor doing here? Are there tests that are carried out and that can be used instead of you doing the work in later stages of the process, or are there holes in the package? In this instance, you need to do more work later. Therefore, review critically what the vendor proposes to do here and cross check against the system requirements in your URS. Where the vendor's material matches your requirements, then accept this and do no further work. The problem is that you may be reviewing a standard document produced by the vendor to meet most uses of the system. Therefore, in our example of an NIR spectrometer used for qualitative work, what would your reaction be if the vendor proposed to execute tests for quantitative analysis? These tests for the way the spectrometer is intended to be used are probably as much use as a chocolate teapot.

An example that the vendor could undertake here to confirm instrument operation could be to run some of the certified or standard reference materials to demonstrate that the instrument is correctly calibrated for its intended operation. Some of the parameters include wavelength accuracy, photometric linearity, and noise. It is important that the noise of the instrument is included either here or in the user testing because the instrument performance tends to degrade over time, and therefore it is vital to monitor the overall performance.

If these tests are not performed here, the users should undertake them during the next phase of the integrated qualification and validation. For example, if a wavelength calibration standard used by a vendor does not offer sufficient resolution or wavelength range, then alternative sources should be used in the next phase by the laboratory. To illustrate this point, Figure 1 shows an example of a wavelength standard over the convention NIR range; more information about traceable standards in NIR will be found in an article to be published in Spectroscopy by Burgess and Hammond (5).

Figure 1: NIR wavelength accuracy standard for 890–2540 nm (5,10).

Review the proposed tests for the software that the vendor proposes to execute as well. What areas of the software do they propose to check out, and do the tests they will run match your working practices? Will the tests be performed on the unconfigured application, or will the vendor configure this to meet your needs? Look at these carefully. If there is extensive testing on the basic installation, this might have little value if you subsequently configure or customize the software in the next phase of work. On the other hand, if these tests match your requirements, then you'll have less to do later because you have already tested some of your requirements. However, in my experience, the tests performed by the vendor typically will be on the unconfigured software and you have to make a judgment on how useful these will be against your requirements.

In addition to the instrument and the software, a vendor could provide a standard holistic test to show that the whole system is working as expected. This should exercise both the instrument and the software but should not be exhaustive. It is merely used to demonstrate that everything works as expected.

Defining User Ways of Working

Here is where the existing guidance from AAPS Analytical Instrument Qualification (6), USP <1058> (7), and GAMP Forum systems good practice guide for validation of computerized laboratory systems (8) all fall down — they only concentrate on one side or the other of the qualification–validation equation. As I have been advocating (1,9), we need an integrated and systems-based approach to the whole issue or regulatory holes appear in our work. To focus solely on analytical instrument qualification to the exclusion of computerized system validation or vice versa is essentially wrong.

In this portion of the work, define and configure how you will use the whole system. This may consist of one or more of the following items:

Qualify the instrument for the operating parameters specified in your URS. Reiterating my earlier point, if the vendor has done this work already and it covers your operating parameters adequately, why repeat this? However, if your ways of working differ from the standard approach, then this is where you will plug the gap and qualify the instrument for your laboratory's specific operating parameters. This is risk management in practice — albeit in an ad hoc form.

Calibrate the instrument. Again, this takes the same point as above. If it has been done to an acceptable standard over the operating ranges that you intend to use the system, do not do any more work because you'll be wasting time. Alternatively, you know the drill by now.

Configure the software and firmware. We'll focus here on the software for our qualitative use of the NIR. The main work will be configuring the security and access privileges and setting up the library.

At a minimum, you'll need to define the user types and the access privileges associated with each type (this will be documented because you'll be testing this in the next phase of the work). Therefore, not all users will be system administrators — unless the user base is very small (for example, two users). In addition, each user will need to have a user identity and password to access the system; every user needs a unique identity so that they can be identified within the system including the audit trails. Most importantly, any default user identities should be disabled and default passwords changed.

For our qualitative use of sample identification, we will need to establish a spectral library. Therefore, we need to know the following. What compounds will you be sampling? Do you want to organize one library or more? How will you acquire spectra (consideration of wavelength range, number of data points per spectrum, and spectroscopic technique) and how will you treat the spectra before you import them into the library? Who will be able to create entries in the library? You'll need about 30 spectra from different batches of the same material to create the library and the samples should not be too old for inclusion in the library. Much of this work will need a standard operating procedure (SOP) to document how to do this, along with the associated training. The library might need to be prototyped before the building of the library; perhaps only a proportion of the substances to be identified will be entered before the user acceptance testing prior to operational release. In this case the library will continue to be built after the system is live under the control of the library SOP discussed above.

Customizing the software can include writing macros to reduce the amount of input work a user needs to perform to custom software for interfacing the NIR system to another application such as a LIMS if a standard interface does not exist. For the purposes of this discussion, there will be no customization of the software. This has several advantages: it is faster (no additional work to do), cheaper (same reason), and lower risk (you are not writing unique code that you will have to maintain).

User Acceptance

The focus of this phase of work is at a system level and brings together all work done at the earlier stages of the life cycle. Here you will demonstrate that the system as a whole meets intended use requirements. Testing here will be traced to the requirements defined in the URS. There will be functional tests of the system: identification of substances using the library as well as nonfunctional tests such as security and access control and possibly performance if that is a key requirement. For this column, I'll focus on the functional testing of the library and identification of substances because this is a specific European Pharmacopoeia requirement for NIR systems (2), but it is not explicitly covered in the corresponding chapter of the USP (3).

You'll select a number of substances for the user acceptance testing. These should be representative of the compounds in the library so that positive identification by the system can be established by comparison with the reference spectra in the library. You must always include in this process compounds that have similar NIR spectra to test the discriminatory power of the instrument and the library software — this is a critical system test that combines elements of analytical instrument qualification and computerized systems validation. Specifically, the European Pharmacopoeia (2) mandates a failure test using compounds that have similar appearance, chemical structure, or name. There is also a specific requirement to use different batches of compound (such as those not used to build the library) for positive testing. Also consider testing for robustness of the identification process (2). That is, you do not just perform one identification per compound. Consider the impact of different operators, environmental conditions on the sample, sample positioning in the reader, and sample presentation differences.

Release for Operational Use

Once all this has been done and any issues have been resolved, the system can be released for operational use. However, the control does not end there. You'll be adding new compounds to the library, so there must be procedures for this but no revalidation. This is the normal way that you will use an NIR system. Consider also the European Pharmacopoeia requirement to check when there is a replacement of instrument parts or sampling presentation devices (2). You need to consider which part of the overall process you should repeat to confirm that the system still operates as intended.

Summary

The key point is that there can never be a standard formula that everyone follows because the dividing lines cannot be drawn exactly between the spectrometer system vendor and the laboratory users. Different organizations do different things and have different approaches. However, it is a matter of being aware of the overall process and ensuring that, from the perspective of the system owner, work is not duplicated wastefully but an effective qualification of the instrument and validation of the software plus testing of the overall system is performed.

R.D. McDowall is principal of McDowall Consulting and director of R.D. McDowall Limited, and "Questions of Quality" column editor for LCGC Europe, Spectroscopy's sister magazine. Address correspondence to him at 73 Murray Avenue, Bromley, Kent, BR1 3DJ, UK.

References

(1) R.D. McDowall, Spectroscopy 21(11), 18–23 (2006).

(2) European Pharmacopoeia version 5.0, Chapter 2.2.40 Near-Infrared Spectrophotometry, 2005.

(3) United States Pharmacopoeia 28, General Chapter <1119> Near-Infrared Spectrophotometry, 2005.

(4) Current Good Manufacturing Practice 21 CFR 211 §160 (Food and Drug Administration, Rockville, Maryland).

(5) C. Burgess and J. Hammond, Spectroscopy, in press.

(6) S.K. Bansal, T. Layloff, E.D. Bush, M. Hamilton, E.A. Hankinson, J.S. Landy, S. Lowes, M.M. Nasr, P.A. St. Jean, and V.P. Shah, Qualification of Analytical Instruments for Use in the Pharmaceutical Industry: A Scientific Approach (American Association of Pharmaceutical Scientists, Arlington, Virginia, 2004).

(7) Pharmacopoeial Forum, <1058> Analytical Equipment Qualification, 32(2), (2006).

(8) GAMP Forum Good Practice Guide – Validation of Laboratory Computerized Systems (International Society for Pharmaceutical Engineering, Tampa, Florida, 2005).

(9) R.D. McDowall, Spectroscopy 21(4), 14–30 (2006).

(10)C. Burgess, personal communication (for further information see www.burgessconsultancy.co.uk)