How Do I Leverage a Supplier’s Software Development in my Spectrometer Validation?

Tired of writing user acceptance test scripts? Want to reduce the amount of testing you do? Do you want to see how a supplier can help you? Are you lazy?

Our focus here is on a supplier assessment at the end of the selection phase within a validation project. We will discuss how a regulated laboratory can leverage supplier software development and testing to reduce User Acceptance Testing (UAT) of a configured spectrometer software (USP <1058> Group C2 system; GAMP software category 4).^1,2

Suppliers can use different software development models. Our focus is on iterative/incremental Agile development for an instrument control application. However, we will not cover its integration with the spectrometer and its firmware.

Don’t forget that selection is finding the right analytical system for the right job which will be defined in a generic User Requirements Specification (URS).³ You’ll want to avoid purchasing a spectrometer that has compliance problems as shown by Smith and McDowall⁴ where 37% of regulatory citations could have been prevented by selecting a more compliant system.

Select the system first based on the fit to the laboratory’s URS, then assess the chosen supplier’s Quality Management System (QMS) via a questionnaire followed by an audit of how the software was developed.

Is Commercial Software Validated?

No! This is the laboratory’s responsibility. See 21 CFR 11 1997 Federal Register 62, 13430: Part 11 Preamble Comment 65⁵

One comment … argued that validation of commercially available software is not necessary because such software has already been thoroughly validated.

The agency disagrees with the comment’s claim that all commercial software has been validated. The agency believes that commercial availability is no guarantee that software has undergone “thorough validation” and is unaware of any regulatory entity that has jurisdiction over general purpose software producers. The agency notes that, in general, commercial software packages are accompanied not by statements of suitability or compliance with established standards, but rather by disclaimers as to their fitness for use.

Regulatory Pick and Mix?

Regulations and industry guidance documents of Computerized System Validation (CSV) discuss supplier assessment; however, they provide insufficient detail on how to carry out such assessment. Annex 11 clause 4.5 states:

The regulated user should take all reasonable steps, to ensure that the system has been developed in accordance with an appropriate quality management system. The supplier should be assessed appropriately.⁶

The revised Annex 11⁷ provides generic content for a service provider qualification in section 7.2 rather than being specific about how to perform assessment of software development. Likewise, OECD GLP No. 17 lacks detail on HOW to perform supplier assessment.⁸

Computer Software Assurance (CSA) Final Guidance contains an example of a Learning Management System (LMS)and considers evaluating suppliers but it says absolutely nothing about HOW to perform it.⁹

GAMP 5 SE explains an overall supplier assessment process in Appendix M2.¹⁰ Agile Software Development and its supporting tools are discussed throughout Appendices D8 and D9, respectively. The latter considers risk factors of using tools that provide help to assess a supplier yet there are no details about how to assess Agile software development to reduce laboratory testing.

Appendix D9 states… “Such tools and systems supporting system life cycles… managed by routine company assessment and assurance practices and good IT practices.”¹⁰ This is consistent with OECD GLP No. 17 clause 36 that mentions… “Suppliers do not need to conform to GLP regulations, but must operate to a documented quality system ….”⁸. In contrast, the ISPE GPG on digital validation contradicts this: “Validation of the DVT (Digital Validation Tool) platform is a key step to ensure that the system’s functionality aligns with organizational goals and meets quality and compliance standards.”¹¹

System and Requirement Categorization

GAMP software categories can be used to classify not only the whole system but also individual user requirements. For example, the entire spectroscopic system is GAMP category 4 (configurable components). Requirements can also be classified into both standard components (GAMP category 3) and configurable components (GAMP category 4) ¹⁰ as shown in Figure 1.

Let's elaborate on the difference in requirements classification by sharing a few examples from Figure 1.

1. GAMP Category 3 components: … cannot change the business process.¹⁰ The software does exactly what it says in the user manual nothing more, nothing less. Category 3 requirements can be parameterized. For example, if the wavelength is set at 220 nm and then changed to 280 nm, the instrument is still measuring the absorbance of an analyte regardless of wavelength, meaning the business process is unchanged.¹² Configuration of user roles and access privileges, as well as reports, is possible, but the business process is unchanged.
2. GAMP Category 4 components: … enable configuration of user-specific business processes into one or more workflows….¹⁰. For example, enabling electronic signatures changes the business process and moves a spectrometer from a hybrid to a digitalized system.

User Acceptance Testing (UAT)/Performance Qualification (PQ)

Potentially the longest part of a validation is deciding what well-formed user requirements³ to test; collating requirements into workflows reflecting working practices and writing the test scripts then executing them.

How can we reduce the amount of testing? Standard functions (GAMP software category 3) can be considered tested by the supplier as part of your validation project, SUBJECT ONLY to an adequate audit of a supplier’s software development. Regression testing and continuous integration testing are fundamental aspects to be covered during the supplier audit. This should demonstrate that GAMP software category 3 requirements can be accepted in lieu of being tested by a laboratory. Instead, focus testing the configured requirements of your intended use which will be a combination of category 3 and 4 user requirements.

Agile Software Development

Many suppliers use an Agile development model with the aim to produce useable code at the end of a 2–3-week sprint. To leverage a supplier’s software development methodology into a laboratory’s validation, the life cycle model used for software development needs to be understood. You don’t need to be a programmer to do this, but you should have a reasonable understanding of software development. To provide a useful overview of Agile development in a regulated environment, the Association for the Advancement of Medical Instrumentation (AAMI) have published technical information report TIR45 on Guidance on the use of agile practices in the development of medical device software.¹³

Agile development employs user stories or epics which are larger grouping of similar user stories. Coding/programming and testing proceeds in an iterative or emerging manner. We refer to user stories throughout this column.

This is a different approach. Don’t expect to see specification documents or paper testing documents at a supplier. Instead, all specification elements and results are electronically captured and managed (kept under control) in tools. These tools should be configured to enforce workflows and user roles etc.

Some of the automated test output may be unintelligible to non-software testers. However, test management processes should be well-formalised, followed and monitored.

It is possible to release incremental software updates of an application frequently, but it is not the best option for a regulated user as this results in a validation treadmill throughout the application lifecycle. Annual application updates are the best option as well as critical bug fixes as necessary.

Software Development Tools

Several software tools can be used:

1. Project management: This controls the overall software development. Each user story or bug fix is given a unique identification number or ticket. Every user has a unique identity to enable traceability of actions to an individual (this is not an audit trail). Software defects or bugs can be tracked within the project management tool or use a specific defect tracking tool. The tools can be configured for user roles and have gates so that work cannot proceed until specific criteria have been met (for example, you must have a code review before testing).

• Source code repository: Used for managing and controlling the software configuration during a sprint and of the final release version of software.
• Automated testing: Used for regression testing to see if a new module has impacted the rest of the application. Passing manual tests for new modules are input into a test tool to expend the scope of automated testing for the next software module under development.
• Source code review: Some suppliers use tools to automate code reviews for a peer developer to verify the output for the developer to correct.
• Wiki: A common area to help communication that is interfaced with the project management tool for storage of task lists, reports and documents. Records are version controlled and changes are traceable to individuals.
• Deployment tool (for Cloud systems): Ensures that the software that is actually used is in fact controlled, however Cloud is out of scope here.

As shown in Figure 2, the Agile lifecycle starts with entering user stories into the backlog within a project management tool. The backlog contains enhancements and improvement ideas in the form of user stories as well as software defects.

Below is an abbreviated description of Agile development; more information can be found in references [10] and [14].^10,14

• Groom Backlog: Repository of enhancements, improvement ideas and software defects for resolution for an application. The product owner will groom the backlog or, in plain English, select the items that can be implemented in a sprint (for example, new features or defects). During grooming user stories may be refined.
• Sprint: This is a 2–3-week period where a team of developers and testers turn selected user stories or defects into tested code for inclusion in the application.
• User Story: Written from a user perspective, this is a concise description of what a user wants to do. For example,

As an analyst [User] I want [an end goal, not a feature] to automatically compare a sample spectrum with that of the reference standard [Reason].

This is not a user requirement. A user story will be decomposed into detailed requirements within the project management tool.

• Definition of Done (DoD): A user story is linked to the DoD. It describes acceptance criteria to determine when development of a user story is complete. The DoD may be updated as the user story is better understood.
• Iterative Program Development: Using the requirements from the user story, a coder will write and test software iteratively until the DoD is met. No records are created apart from the final source code. This is not a GXP activity.
• Code Review: Peer review of the module source code against the supplier’s coding standards and user story requirements should be undertaken. Approaches to code review vary between suppliers: Best practice is for all modules to be peer reviewed as it can find defects which are easy and cheap to fix rather than wait until testing. Other suppliers only code review critical user stories. Which approach is better?
• Module Testing: New code is informally and manually tested against DoD. When passed, the module is integrated into the existing code and automated regression testing is executed on the new software build. Regression testing of each software build should be undertaken to ensure that new modules do not introduce software bugs.
• Defect Management: Any bugs are reviewed and sent to the developer for fixing, then returned for testing. Depending on the tool used, an identified bug will receive a ticket to track the issue through to its resolution.
• Application Release Testing: Carried out on the application and user interface to ensure that the finished product has no critical or major defects. Known bugs can be included in a release if they are minor and a risk assessment indicates they should not impact the main functionality of the product.

Supplier Assessment

In accordance with the computerised system category, a regulated user should take a risk-based approach to assess a supplier before purchasing the selected product Annex 11 clauses 1 and 4.3, and Chapter 7.^6,15 For a USP Group C1 system (GAMP category 3 software), it is not required to conduct a supplier software assessment, but for a Group C2 system (GAMP category 4 software) one should be performed.¹⁰ After a supplier agrees to allow an audit (either on-site or remote) and you have signed a non-disclosure agreement, you need to plan what you want to audit. Remember, accountability remains with the regulated organization.

Supplier Quality Management System (QMS) Assessment

The QMS should be covered with a questionnaire prior to the audit. QMS review includes the scope of certification, quality manual, organization, procedures, training (including GxP awareness to comply with 21 CFR 11.10(i)⁵), and software development process.

If the supplier is ISO 9001 or 90003 certified,^16,17 ask for the supplier’s certificate and assess the scope of services covered. A certificate covers the whole scope of design, development, production, installation and servicing of a product or service. The scope of certification determines how much of the standard is applicable to an organization. That is why it is important to see the ISO certificate when the supplier assessment is conducted and note the scope of certification.¹⁸

An ISO certificate is inadequate for category 4 as stated by PIC/S PI-011-3:

11.5 … an assessment of the supplier’s Quality Management System (QMS) and recognized certification alone is unlikely to be the final arbiter for critical systems. The certification may very well be inadequate, or inappropriate.¹⁹

However, the adequacy and efficiency of the selected engineering approach are not fully challenged or verified. A need for an on-site or remote audit of software development is, in our view, required for GAMP software category 4 IF you want to reduce the extent of UAT by the laboratory.

If you want to reduce the extent of UAT by the laboratory for GAMP software category 4, then, in our view, an on-site or remote audit of software development is required.

Supplier Software Development Audit

How will you perform an audit of an Agile software development and testing? You need a scope and agenda with approximate timings, and the first part could be to verify some of the questionnaire’s answers. However, the main audit focus will be on how software is specified, coded and tested. Regression testing and continuous integration testing are fundamental aspects to be covered during the supplier audit. We are not covering metrics or scrum meeting notes during the assessment unless required.

• Overview: The supplier should provide experienced staff who can speak to, operate, and introduce the various software tools effectively, and explain what is happening to answer your questions during the audit. A user audit team should consist of personnel who are knowledgeable and experienced with software development and may include a member from the laboratory who was involved with the selection of the system.

Our suggestion is to trace the works as outlined below:

• Software Development: Start with the project management tool. What data are generated/stored by the tool?Where are the data stored? What cybersecurity controls are in place, including penetration testing?
• Follow a Story or Ticket: Select a user story or ticket in a sprint. One possible option is a regulatory compliance feature. See how it is decomposed into requirements. How detailed are they? How many are there? How do they relate to the user story/ticket? If the auditor doesn’t have the experience, ask the person who is operating the project management tool to provide explanations. There should be an assessment of the impact or risk of each user story/ticket on the application to determine the extent of requirements and testing. See Figure 2.
• Code Review: Check to see if there is a gate configured to ensure that the code of all modules is reviewed. If it is not, ask for the procedure and process that determine if code is peer reviewed or not. If changes were requested in the review, is there evidence of this in the project management tool, identifying who did this and when? Also ask a developer how the code complies with the coding standard used.
• Source Code Management: Confirm the vendor has a defined and controlled process for retrieving, modifying, and uploading source code during updates, with safeguards to prevent inadvertent or unauthorized changes. For example:
  • Version control (repository-based, with branching/merge controls)
  • Access restrictions (authorized developers only)
  • Change traceability (upload linked to an approved change request/user story)
  • Integrity check (review gate before merge)
• Module Testing: Manual testing of modules has little or no documentation. Much of the testing is automated and the output may need to be explained by the supplier’s staff. Ask to see the applicable procedure and how detailed the testing is against the requirements generated from the user story. How confident are you that the module has been tested adequately? What regression testing has been consistently applied to each build, and have any defects been documented, resolved, and retested?
• Release Application Testing: How is a release candidate tested before release and deployment? What is the scope of testing: some or all of the application?

Examine some of the tests and, where necessary, ask the supplier’s staff to help you:

• How extensive are they?
• Is there testing to pass as well as testing to fail?
• Are all areas of the software testing the same or do some areas have targeted testing as they are more critical?
• Is there a list of tests available to the laboratory to justify reduced UAT testing?
• How do they document the release of the application?

Writing the Audit Report

The audit conclusion and observed findings must be documented as part of the validation documentation. Any white papers or ISO standard certification from the supplier can be used to supplement the audit report. The assessment report must be referenced in the validation documentation and should state whether the supplier software development and testing is adequate to leverage into your UAT/PQ.

Depending on the approved report, you need to decide whether to proceed with the system purchase or explore other options that are better fitted to your intended use or whether there is a concern about their compliance with the QMS and software development process. If the identified risks are low and can be resolved in a timely manner, an alternative is to ask the supplier to initiate corrective actions (CAPA) rather than rejecting the system outright.

Summary

At the start of this column, we asked how a supplier can help you reduce the amount of UAT in a validation.

To answer the question, assess the overall software development process: the backlog, the management and traceability of user stories/tickets, and regression and continuous integration testing.

If the audit determines that a supplier’s software development and testing are satisfactory, then you can conclude that standard components can be leveraged into your validation. This is a risk-based and sensible resolution that is justified in the supplier audit report.

The next column will discuss how the test scripts should be written and the extent of documented evidence required.

Acknowledgements

We would like to thank, in alphabetical order, Jim Henderson, Rina Nir and Yves Samson for their thorough and very rapid review of our article which has improved it.

References

1. USP General Chapter <1058> Analytical Instrument Qualification; United States Pharmacopeial Convention: Rockville, MD.
2. USP <1058> Analytical Instrument and System Qualification, in process revision. Pharmacopeial Forum 2025, 51 (2).
3. Andraos, M.; McDowall, R. D. We Know What We Want! Why Write Requirements? Spectroscopy Online, 2026. https://www.spectroscopyonline.com/view/we-know-what-we-want-why-write-requirements- (accessed July 9, 2026).
4. Smith, P. A.; McDowall, R. D. Analysis of FDA Infra-Red 483 Citations – Have You a Data Integrity Problem? Spectroscopy 2019, 34 (9), 22–28.
5. 21 CFR Part 11; Electronic Records; Electronic Signatures, Final Rule. Federal Register 1997, 62 (54), 13430–13466.
6. EudraLex – Volume 4 Good Manufacturing Practice (GMP) Guidelines, Annex 11: Computerised Systems; European Commission: Brussels, 2011.
7. Stakeholders’ Consultation on EudraLex Volume 4 – Good Manufacturing Practice Guidelines: Chapter 4, Annex 11 and New Annex 22; European Medicines Agency: Amsterdam, 2025. https://health.ec.europa.eu/consultations/stakeholders-consultation-eudralex-volume-4-good-manufacturing-practice-guidelines-chapter-4-annex_en (accessed July 7, 2025).
8. OECD Series on Principles of Good Laboratory Practice and Compliance Monitoring, Number 17: Application of GLP Principles to Computerised Systems; Organisation for Economic Co-operation and Development: Paris, 2022.
9. FDA Guidance for Industry: Computer Software Assurance for Production and Quality System Software; Food and Drug Administration: Silver Spring, MD, 2026.
10. GAMP 5: A Risk-Based Approach to Compliant GxP Computerized Systems, 2nd ed.; International Society for Pharmaceutical Engineering: Tampa, FL, 2022.
11. ISPE Good Practice Guide: Digital Validation; International Society for Pharmaceutical Engineering: Tampa, FL, 2025.
12. McDowall, R. D. Simple Spectrometer System, Simple Validation? Spectroscopy 2023, 38 (11), 16–19. DOI: https://doi.org/10.56530/spectroscopy.tc3777t8.
13. AAMI TIR45:2023; Guidance on the Use of Agile Practices in the Development of Medical Device Software; Association for the Advancement of Medical Instrumentation: Arlington, VA, 2023.
14. What Is Agile? Understanding the Agile Methodology; Atlassian, 2026. https://www.atlassian.com/agile (accessed July 9, 2026).
15. EudraLex – Volume 4 Good Manufacturing Practice (GMP) Guidelines, Chapter 7: Outsourced Activities; European Commission: Brussels, 2013.
16. ISO 9001:2015 Quality Management Systems – Requirements; International Organization for Standardization: Geneva, 2015.
17. ISO/IEC/IEEE 90003:2018 Software Engineering – Guidelines for the Application of ISO 9001:2015 to Computer Software; International Organization for Standardization: Geneva, 2018.
18. McDowall, R. D. Validation of Chromatography Data Systems: Ensuring Data Integrity, Meeting Business and Regulatory Requirements; Royal Society of Chemistry: Cambridge, U.K., 2017.
19. PIC/S Good Practices for Computerised Systems in Regulated “GXP” Environments (PI 011-3); Pharmaceutical Inspection Convention/Pharmaceutical Inspection Co-operation Scheme (PIC/S): Geneva, 2007.