Data Analytics, Statistics, Chemometrics, and Artificial Intelligence

Karl Booksh, a Professor of Chemistry at the University of Delaware, has won the 2025 EAS Award for Outstanding Achievements in Chemometrics.

This mini-tutorial explores how data preprocessing (DP) transforms raw FT-IR ATR spectra into meaningful, reliable inputs for chemometric modeling. Readers will learn about key DP methods: normalization, scatter correction, centering, scaling, and baseline correction, and how proper selection of these techniques improves accuracy, reproducibility, and interpretability in infrared spectroscopic analysis.

Fourier transform infrared (FT-IR) spectroscopy is a versatile, non-destructive analytical tool used to characterize molecular structures, monitor chemical reactions, and quantify analytes in diverse materials. This mini-tutorial reviews fundamental principles, key operational modes, and practical examples across environmental, biomedical, and industrial applications. Readers will review and learn how to optimize FT-IR methods, interpret spectra, and avoid common pitfalls in data collection and processing.

The miniaturization of spectroscopic instruments has reached a remarkable milestone: wearable vibrational spectroscopy. Techniques such as Raman, surface-enhanced Raman scattering (SERS), infrared (IR), and functional near-infrared (fNIRS) spectroscopy are no longer confined to the laboratory bench—they now fit on our bodies, into household devices, and onto industrial equipment. These wearable devices promise continuous, real-time monitoring, offering molecular-level insights for personal health, household management, clinical care, and industrial applications.

Phil C. Williams (1933–2025) was an internationally recognized pioneer in near-infrared spectroscopy whose visionary work transformed grain analysis from chemical assays to rapid, environmentally responsible spectroscopic methods. His lifelong commitment to scientific rigor, mentorship, and practical innovation has left an enduring legacy that continues to shape industrial spectroscopy for grain analysis that impacts the global economy.

This second part of the Recent Research in Chemometrics and AI for Spectroscopy article surveys current and emerging applications of artificial intelligence (AI) in spectroscopy, highlighting explainable AI (XAI), deep learning, and generative AI frameworks.

This first article in a two-part series introduces the foundations and terminology of AI as applied to chemometrics, defines key algorithmic approaches, and explores their growing role in spectral data analysis, model quantitative calibration, classification, and interpretability.

Using optical and near-infrared spectroscopy, researchers have identified crystalline water ice as the likely driver of explosive outbursts on comet 12P/Pons-Brooks. Their findings link this dramatic cometary activity to a process once observed in comets 17P/Holmes and 332P/Ikeya–Murakami.

A recent study presented a novel deep learning model that could improve the prediction of fuel properties in the petroleum industry.

A new international review highlights how hyperspectral imaging (HSI) is revolutionizing diverse fields—from counterfeit detection and agriculture to cancer diagnostics—by capturing unprecedented spectral detail invisible to traditional cameras. The study identifies major advances, challenges, and the growing role of artificial intelligence in real-time HSI applications.

Researchers have developed a new method combining unmanned aerial vehicle (UAV) hyperspectral imaging with satellite data to monitor chlorophyll-a (Chla) and total nitrogen (TN) concentrations in coastal wetland waters. Their approach enhances the precision and scalability of water quality assessments, providing a model for managing eutrophication in fragile ecosystems.

Researchers at the European Space Research and Technology Centre (ESTEC) have developed a new framework for onboard hyperspectral image processing that uses deep learning to analyze massive volumes of spectral data in real time. Their review highlights lightweight neural networks, generative models, and hardware accelerators as key technologies shaping the next generation of spaceborne Earth observation.

A sweeping review, “Surface-Enhanced Raman Spectroscopy: A Half-Century Historical Perspective” traces the evolution of SERS from its foundational experiments in the 1970s through the nanoscience era and modern high-spatial-resolution techniques.

Here are ten main unsolved problems in vibrational and atomic spectroscopy, each accompanied by a tutorial-style synopsis suitable for advanced practitioners or graduate-level students. Each of these tutorials, spanning advanced spectroscopy modeling, chemometrics, machine learning (ML) interpretability, and standardization, consists of a descriptive article. Each piece is well-referenced (with detailed matrix equations, radiative transfer models, chemometric derivations, and so forth), and includes the following. • Special focus on each topic—including mathematical derivations in matrix notation. • Conservative, verifiable content anchored to established reference sources. • Appropriate tutorial article structure: Title, Summary, Abstract, Introduction, Theory with equations, Examples, Discussion & Future Research, and References.

This curated collection of recent Spectroscopy magazine mini-tutorials highlights the latest analytical and data-driven innovations in vibrational spectroscopy. Covering NIR, Raman, O-PTIR, and related optical methods, the series emphasizes practical workflows, emerging machine learning integrations, and advanced chemometric techniques for real-world laboratory applications—from food and environmental monitoring to biomedical analysis and nanoscale imaging.

A new perspective from researchers at the Karlsruhe Institute of Technology explores the evolving relationship between human expertise and artificial intelligence in polymer chemistry.

Scientists have developed IR-Bot, an autonomous robotic platform that combines infrared spectroscopy, machine learning, and quantum chemistry to perform real-time analysis of chemical mixtures. The system promises to transform autonomous experimentation by delivering rapid, accurate feedback to guide chemical reactions without human oversight.

This tutorial explains how baseline drift and multiplicative scatter distort spectroscopic data, reviews correction techniques such as MSC, SNV, EMSC, wavelet-based detrending, and AsLS baseline estimation with matrix-based derivations, and explores emerging data-driven scatter modeling strategies and future research directions.

A recent review by Jhonatan Contreras and Thomas Bocklitz from Friedrich Schiller University Jena and the Leibniz Institute of Photonic Technology delves into the emerging field of explainable artificial intelligence (XAI) in spectroscopy.

This tutorial explores the challenges posed by nonlinearities in spectroscopic calibration models, including physical origins, detection strategies, and correction approaches. Linear regression methods such as partial least squares (PLS) dominate chemometrics, but real-world data often violate linear assumptions due to Beer–Lambert law deviations, scattering, and instrumental artifacts. We examine extensions beyond linearity, including polynomial regression, kernel partial least squares (K-PLS), Gaussian process regression (GPR), and artificial neural networks (ANNs). Equations are provided in full matrix notation for clarity. Practical applications across near-infrared (NIR), mid-infrared (MIR), Raman, and atomic spectroscopies are discussed, and future research directions are outlined with emphasis on hybrid models that integrate physical and statistical knowledge.

This Chemometrics in Spectroscopy column traces the historical and technical development of these methods, emphasizing their application in calibrating spectrophotometers for prediction of measured sample chemical or physical properties and explores how AI and deep learning are reshaping the spectroscopic landscape.

This tutorial explores the motivation, mathematical underpinnings, and practical approaches to fusing spectral data, with emphasis on early, intermediate, and late fusion strategies.

Near-infrared (NIR) spectroscopy combined with aquaphotomics shows potential for a rapid, non-invasive approach to detect subtle biochemical changes in biofluids and agricultural products. By monitoring water molecular structures through water matrix coordinates (WAMACs) and visualizing water absorption spectrum patterns (WASPs) via aquagrams, researchers can identify disease biomarkers, food contaminants, and other analytes with high accuracy. This tutorial introduces the principles, practical workflow, and applications of NIR aquaphotomics for everyday laboratory use.

This tutorial provides an in-depth discussion of methods to make machine learning (ML) models interpretable in the context of spectroscopic data analysis. As atomic and molecular spectroscopy increasingly incorporates advanced ML techniques, the black-box nature of these models can limit their utility in scientific research and practical applications. We present explainable artificial intelligence (XAI) approaches such as SHAP, LIME, and saliency maps, demonstrating how they can help identify chemically meaningful spectral features. This tutorial also explores the trade-off between model complexity and interpretability.

Researchers have demonstrated a non-invasive method using milk and near-infrared spectroscopy combined with Aquaphotomics to accurately detect Paratuberculosis in dairy cattle. The technique offers faster, more sensitive diagnosis than traditional methods.