Data Analytics, Statistics, Chemometrics, and Artificial Intelligence

In this Spectroscopy blog post, Yang Liu, a Scientific Liaison for General Chapters at the United States Pharmacopeia (USP), discusses how data-driven methodologies are set to transform spectroscopy in 2026 and beyond, particularly in pharmaceutical analysis.

In 2025, the vibrational-spectroscopy community saw a convergence of deep learning, advanced simulation, and portable instrumentation that materially changed how spectra are interpreted and applied. Breakthroughs in spectrum-to-structure models, machine learning (ML)-accelerated molecular dynamics, and field-deployable classic Raman, near-infrared (NIR), and surface-enhanced Raman spectroscopy (SERS) sensors pushed vibrational techniques from complex laboratory characterization toward automated structure elucidation, rapid analysis, and real-world sample sensing (1–6,9). This summary article highlights key 2025 contributions and their implications for the year of discovery.

A proposed update to United States Pharmacopoeia (USP) <1029> was published in July 2025 for industry comment. What’s changed and are the changes significant?

Top articles published this week include interviews with Paolo Oliveri of the University of Genoa (Italy) and Maryam Shakiba and Santiago Marin of the University of Colorado Boulder, and an inside look at vibrational spectroscopy trends.

A new study from Heilongjiang Bayi Agricultural University introduces a high-accuracy, explainable deep learning model that significantly improves nondestructive nitrogen and chlorophyll estimation in maize canopies using hyperspectral data.

In this short feature, we highlight how the Sharjah Academy of Astronomy, Space Sciences, and Technology (SAASST) are using techniques such as XRF, XRD, SEM-EDS, and Raman tests to determine the mineral composition of meteorites.

This review article highlights how a new review by Da-Wen Sun demonstrates that integrating spectroscopy with chemometric techniques can significantly improve cold chain food quality monitoring, authentication, and overall system efficiency.

A new review by researchers from the University of Waterloo, Sanofi, and McGill University highlights how vibrational and fluorescence spectroscopy are reshaping real-time monitoring of pharmaceutical bioprocesses. The authors detail recent advances in UV-Vis, NIR-MIR, Raman, and fluorescence sensing, supported by modern chemometrics and AI tools.

A recent study demonstrates that near-infrared (NIR) spectroscopy is a fast, cost-effective, and reliable tool for assessing soil and tree ecological traits, offering major potential for large-scale forest conservation and monitoring.

A new perspective article by Anna de Juan and Rodrigo Rocha de Oliveira highlights how hyperspectral imaging (HSI), paired with advanced chemometrics, is redefining process analytical technology (PAT) by coupling chemical specificity with full-field spatial resolution. Their work outlines how HSI surpasses classical spectroscopic PAT tools and enables quantitative, qualitative, and mechanistic insight into chemical processes in real time.

A recent study demonstrated that deep learning combined with ICP spectroscopy can accurately and affordably predict multiple soil properties.

The article highlights how rapidly advancing, non-destructive spectroscopic and imaging technologies, supported by AI, multi-modal data integration, and increasing instrument portability.

A research team has developed the first short synthetic peptide-based biosensor for real-time tracking of the disease-related protease matrix metalloproteinase-9 (MMP-9), using multi-parametric surface plasmon resonance spectroscopy (MP-SPR).

This tutorial introduces spectroscopy professionals to the operational principles, practical workflows, and laboratory applications of biosensors. It covers core definitions, biosensor types, transduction methods, nanomaterials-enabled strategies, and optical/electrochemical approaches relevant to spectroscopic analysis. Readers will learn how biosensors integrate biological recognition with physicochemical detection, how to implement them in real-world measurement tasks, and how to avoid common technical pitfalls when translating biosensor theory into laboratory practice.

A new study published in Ecological Informatics finds that hyperspectral remote sensing combined with machine learning can accurately predict grassland forage quality across global biomes.

Karl Booksh, a Professor of Chemistry at the University of Delaware, has won the 2025 EAS Award for Outstanding Achievements in Chemometrics.

This mini-tutorial explores how data preprocessing (DP) transforms raw FT-IR ATR spectra into meaningful, reliable inputs for chemometric modeling. Readers will learn about key DP methods: normalization, scatter correction, centering, scaling, and baseline correction, and how proper selection of these techniques improves accuracy, reproducibility, and interpretability in infrared spectroscopic analysis.

Fourier transform infrared (FT-IR) spectroscopy is a versatile, non-destructive analytical tool used to characterize molecular structures, monitor chemical reactions, and quantify analytes in diverse materials. This mini-tutorial reviews fundamental principles, key operational modes, and practical examples across environmental, biomedical, and industrial applications. Readers will review and learn how to optimize FT-IR methods, interpret spectra, and avoid common pitfalls in data collection and processing.

The miniaturization of spectroscopic instruments has reached a remarkable milestone: wearable vibrational spectroscopy. Techniques such as Raman, surface-enhanced Raman scattering (SERS), infrared (IR), and functional near-infrared (fNIRS) spectroscopy are no longer confined to the laboratory bench—they now fit on our bodies, into household devices, and onto industrial equipment. These wearable devices promise continuous, real-time monitoring, offering molecular-level insights for personal health, household management, clinical care, and industrial applications.

Phil C. Williams (1933–2025) was an internationally recognized pioneer in near-infrared spectroscopy whose visionary work transformed grain analysis from chemical assays to rapid, environmentally responsible spectroscopic methods. His lifelong commitment to scientific rigor, mentorship, and practical innovation has left an enduring legacy that continues to shape industrial spectroscopy for grain analysis that impacts the global economy.

This second part of the Recent Research in Chemometrics and AI for Spectroscopy article surveys current and emerging applications of artificial intelligence (AI) in spectroscopy, highlighting explainable AI (XAI), deep learning, and generative AI frameworks.

This first article in a two-part series introduces the foundations and terminology of AI as applied to chemometrics, defines key algorithmic approaches, and explores their growing role in spectral data analysis, model quantitative calibration, classification, and interpretability.

Using optical and near-infrared spectroscopy, researchers have identified crystalline water ice as the likely driver of explosive outbursts on comet 12P/Pons-Brooks. Their findings link this dramatic cometary activity to a process once observed in comets 17P/Holmes and 332P/Ikeya–Murakami.

A recent study presented a novel deep learning model that could improve the prediction of fuel properties in the petroleum industry.

A new international review highlights how hyperspectral imaging (HSI) is revolutionizing diverse fields—from counterfeit detection and agriculture to cancer diagnostics—by capturing unprecedented spectral detail invisible to traditional cameras. The study identifies major advances, challenges, and the growing role of artificial intelligence in real-time HSI applications.