Columns | Column: Unsolved Problems in Spectroscopy

This tutorial contrasts classical analytical error propagation with modern Bayesian and resampling approaches, including bootstrapping and jackknifing. Uncertainty estimation in multivariate calibration remains an unsolved problem in spectroscopy, as traditional, Bayesian, and resampling approaches yield differing error bars for chemometric models like PLS and PCR, highlighting the need for deeper theoretical and practical solutions.

This tutorial investigates the persistent issue of sample heterogeneity—chemical and physical—during spectroscopic analysis. Focus will be placed on understanding how spatial variation, surface texture, and particle interactions influence spectral features. Imaging spectroscopy, localized sampling strategies, and adaptive averaging algorithms will be reviewed as tools to manage this problem, as one of the remaining unsolved problems in spectroscopy.

Inter-instrument variability is a major obstacle in multivariate spectroscopic analysis, affecting the reliability and portability of calibration models. This tutorial addresses the theoretical and practical challenges of model transfer across instruments. It covers spectral variability sources—such as wavelength shifts, resolution differences, and line shape variations—and presents key standardization techniques including direct standardization (DS), piecewise direct standardization (PDS), and external parameter orthogonalization (EPO). We discuss the underlying mathematics of these approaches using matrix notation and highlight limitations that must be considered for reliable universal calibration.

This tutorial addresses the critical issue of analyte specificity in multivariate spectroscopy using the concept of Net Analyte Signal (NAS). NAS allows chemometricians to isolate the portion of the signal that is unique to the analyte of interest, thereby enhancing model interpretability and robustness in the presence of interfering species. While this tutorial introduces the foundational concepts for beginners, it also includes selected advanced topics to bridge toward expert-level applications and future research. The tutorial covers the mathematical foundation of NAS, its application in regression models like partial least squares (PLS), and emerging methods to optimize specificity and variable selection. Applications in pharmaceuticals, clinical diagnostics, and industrial process control are also discussed.

This tutorial examines the modeling of diffuse reflectance (DR) in complex particulate samples, such as powders and granular solids. Traditional theoretical frameworks like empirical absorbance, Kubelka-Munk, radiative transfer theory (RTT), and the Hapke model are presented in standard and matrix notation where applicable. Their advantages and limitations are highlighted, particularly for heterogeneous particle size distributions and real-world variations in the optical properties of particulate samples. Hybrid and emerging computational strategies, including Monte Carlo methods, full-wave numerical solvers, and machine learning (ML) models, are evaluated for their potential to produce more generalizable prediction models.