The development of an expert calibration system (ECS) for spectroscopic-based process analytical chemistry would be a significant advancement aimed at automating the creation of high-quality calibration models for standard zero-order and first-order calibrations as well as multidimensional imaging applications. The concept of ECS would seek to reduce the reliance on users’ extensive knowledge of chemometrics all the while leveraging their domain knowledge and understanding of specific sample physical and chemical properties. By providing automated tools and guidance, an ECS would aim to streamline the calibration process, improve calibration transfer, enhance operator efficiency, and improve the overall consistency and reliability of analytical results produced using advanced chemometrics and machine earning techniques. In this article, we resurrect the discussion from nearly three decades ago.
At the 1995 Instrument Society of America (ISA) Conference in Toronto, Canada, there was a paper presented and published entitled, “The Design of an Expert Calibration System for Spectroscopic Based Process Analytical Chemistry,” authored by J. Workman, B. Kowalski, and P. Mobley (1). This presentation suggested that an automated expert calibration system (ECS) would be of keen interest to those analysts using spectroscopic analytical techniques, especially for process analysis applications. Based on the then state-of-the-art computer-based technology, an overview was described for such a system. The paper explained a complete ECS program that would include data transformations, data preprocessing, linear and non-linear calibration methods, model output validation, calibration performance monitoring, and model transfer optimization. The paper further suggested that additional features for an ideal ECS would include online help screens, computational time estimates, automatic report generation (with a description of pertinent details), and multiple user levels based upon operator expertise and sophistication (1). Because of the age of this presentation, we review its content here with the intention of expanding upon these concepts in future articles.
Process analytical chemistry plays a crucial role in various industries, facilitating the optimization of processes, improving productivity, ensuring product quality, and minimizing waste. The adoption of optical spectroscopic methods, such as ultraviolet (UV), visible (vis), near-infrared (NIR), mid-infrared (MIR), and Raman spectroscopy, has become increasingly common because of their speed, precision, non-destructive nature, and ability to perform multicomponent analyses. These methods rely on multivariate quantitative analysis techniques to establish relationships between instrument responses (absorbance, reflection, transmittance, and intensity) and analyte parameter concentrations.
Process analytical chemistry has witnessed remarkable progress, driven by the demand for rapid, accurate, and reliable analytical information in industrial settings. Optical spectroscopic methods have emerged as the preferred techniques for process analysis because of their versatility and efficiency. These methods allow for real-time monitoring of various parameters, enabling proactive process control and optimization.
The traditional approach to calibration involves establishing relationships between instrument responses and analyte concentrations using chemometric techniques, such as Beer’s law, combined with regression analysis. However, this process requires considerable expertise in chemometrics, which may not be readily available to all users. Additionally, manual calibration procedures are time-consuming and prone to errors, leading to suboptimal results.
The ECS concept aims to bridge the gap between user expertise and calibration requirements by providing automated, user-friendly tools for calibration model development and utilization. Unlike traditional methods that rely heavily on manual intervention and expert judgment, the ECS concept utilizes a knowledge-based approach to guide users through the calibration process. By incorporating user input regarding sample properties and application requirements, the ECS generates calibration models tailored to specific analytical needs.
Would modern artificial intelligence (AI) tools, including various new machine learning (ML) tools, offer some remedies for the challenges associated with calibration modeling, transfer, and maintenance? We have begun to ask these questions once again and to explore the options and precautions of moving forward in this direction (2–5). However, what are the key requirements of an ECS that need to be addressed irrespective of the software tools utilized?
The ECS concept comprises several essential components, including steps for basic spectroscopic model generation and optimization. These steps include: data preprocessing, model building, validation, monitoring, and model transfer optimization (Table I and Figure 1). These components work together to ensure the creation of robust and reliable calibration models (6). Data preprocessing involves various techniques such as spectral correction, baseline correction, and noise reduction to enhance the quality of input data. Model building encompasses the selection of appropriate calibration algorithms, optimization of model parameters, and validation of model performance. In addition, successful model building requires making provisions for seamless calibration transfer across instrumentation platforms. For imaging applications, a similar set of steps is required for creation, analysis, and validation of image generated analytical results (Figure 2). For these processes a set of well-constructed heuristic rules is required for ECS design. Table II shows the basic types of rules required for creating such a software functionality. If AI is involved, such knowledge is also required as input parameters and functional structure for the AI system.
Validation and monitoring are critical aspects of the calibration process, ensuring the integrity and reliability of calibration models. Validation involves assessing the accuracy and precision of calibration models using independent data sets or reference methods. Monitoring, on the other hand, involves ongoing evaluation of model performance to detect any deviations or drifts over time. By continuously monitoring model performance, the ECS can alert users to potential issues and facilitate timely recalibration or adjustments.
Model transfer optimization is essential when deploying calibration models across different instruments or environments. Variations in instrument characteristics or sample matrices may affect the performance of calibration models, necessitating adjustments or fine-tuning. The ECS employs advanced algorithms and decision-making criteria to optimize model transferability, ensuring consistent and reliable results across diverse operational conditions.
The development and implementation of the ECS presents several challenges, including data integration, algorithm selection, and user interface design. Addressing these challenges requires a multidisciplinary approach involving expertise in chemometrics, data science, and software engineering. By leveraging state-of-the-art computational techniques and knowledge engineering principles, the ECS can overcome these challenges and deliver robust, user-friendly solutions.
The development of an ECS would represent a significant step forward in process analytical chemistry, offering automated tools and guidance for calibration model development. By streamlining the calibration process and reducing reliance on user expertise, the ECS would enhance the efficiency, reliability, and usability of spectroscopic-based process analyzers. With ongoing advancements in technology and methodology, the ECS holds promise for revolutionizing process analytical chemistry and driving innovation across various industries. The implementation of machine learning and other AI software tools will still require sound experimental design and domain knowledge to create a useful system able to make calibration modeling, calibration transfer, and calibration maintenance more efficient, but also to minimize errors in these processes (2–5). Furthermore, machine learning and deep learning AI programming will be more effective for spectroscopic methods if the spectral data is of high quality and is properly aligned for X- and Y-axis registrations, such that the signals in the spectral data are comparable or standardized.
(1) Workman, J.: Kowalski, B.; Mobley, P. The Design of an Expert Calibration System for Spectroscopic Based Process Analytical Chemistry. In Proc. Instrument Society of America (ISA) Conference, Toronto, Canada, 1995, April, 97–106.
(2) Analytically Speaking Podcast Page. Ep. 9: Automating Advanced Chemometric Methods for Data Processing. February 1, 2023. https://www.spectroscopyonline.com/view/ep-9-automating-advanced-chemometric-methods-for-data-processing (accessed 2024-03-08)
(3) Workman, Jr., J; Mark, H. Artificial Intelligence in Analytical Spectroscopy, Part I: Basic Concepts and Discussion. Spectroscopy 2023, 38 (2), 13–22. DOI: 10.56530/spectroscopy.og4284z8
(4) Workman, Jr., J.; Mark, H. Artificial Intelligence in Analytical Spectroscopy, Part II: Examples in Spectroscopy. Spectroscopy 2023, 38 (6), 10–15. DOI: 10.56530/spectroscopy.js8781e3
(5) Workman, Jr., J. An Interview with AI About Its Potential Role in Vibrational and Atomic Spectroscopy, May 17, 2023 Online. https://www.spectroscopyonline.com/view/an-interview-with-ai-about-its-potential-role-in-vibrational-and-atomic-spectroscopy (accessed 2024-03-08)
(6) ASTM E1655-17 Standard Practices for Infrared Multivariate Quantitative Analysis Book of Standards Volume: 03.06, Developed by Subcommittee: E13.11 DOI: 10.1520/E1655-17
(7) Workman, Jr. J. Exploring the Spectrum of Analytical Techniques for Material Characterization. Spectroscopy 2023, 38 (s11), 6–13. DOI: 10.56530/spectroscopy.on6669u1
AI, Deep Learning, and Machine Learning in the Dynamic World of Spectroscopy
December 2nd 2024Over the past two years Spectroscopy Magazine has increased our coverage of artificial intelligence (AI), deep learning (DL), and machine learning (ML) and the mathematical approaches relevant to the AI topic. In this article we summarize AI coverage and provide the reference links for a series of selected articles specifically examining these subjects. The resources highlighted in this overview article include those from the Analytically Speaking podcasts, the Chemometrics in Spectroscopy column, and various feature articles and news stories published in Spectroscopy. Here, we provide active links to each of the full articles or podcasts resident on the Spectroscopy website.
Diffuse Reflectance Spectroscopy to Advance Tree-Level NSC Analysis
November 28th 2024Researchers have developed a novel method combining near-infrared (NIR) and mid-infrared (MIR) diffuse reflectance spectroscopy with advanced data fusion techniques to improve the accuracy of non-structural carbohydrate estimation in diverse tree tissues, advancing carbon cycle research.
Young Scientist Awardee Uses Spectrophotometry and AI for Pesticide Detection Tool
November 11th 2024Sirish Subash is the winner of the Young Scientist Award, presented by 3M and Discovery education. His work incorporates spectrophotometry, a nondestructive method that measures the light of various wavelengths that is reflected off fruits and vegetables.
Emerging Leader Highlights Innovations in Machine Learning, Chemometrics at SciX Awards Session
October 23rd 2024Five invited speakers joined Joseph Smith, the 2024 Emerging Leader in Molecular Spectroscopy, on stage to speak about trends in hyperspectral imaging, FT-IR, surface enhanced Raman spectroscopy (SERS), and more during the conference in Raleigh.
Advancing Forensic Science with Chemometrics: New Tools for Objective Evidence Analysis
October 22nd 2024A review by researchers from Curtin University comprehensively explores how chemometrics can revolutionize forensic science by offering objective and statistically validated methods to interpret evidence. The chemometrics approach seeks to enhance the accuracy and reliability of forensic analyses, mitigating human bias and improving courtroom confidence in forensic conclusions.