Howard Mark

This “Chemometrics in Spectroscopy” column traces the historical and technical development of these methods, emphasizing their application in calibrating spectrophotometers for predicting measured sample chemical or physical properties—particularly in near-infrared (NIR), infrared (IR), Raman, and atomic spectroscopy—and explores how AI and deep learning are reshaping the spectroscopic landscape.

This column is the continuation of our previous column that describes and explains some algorithms and data transforms beyond those most commonly used. We present and discuss algorithms that are rarely, if ever, seen or used in practice, despite that they have been proposed and described in the literature.

By providing automated tools and guidance, an ECS would aim to streamline the calibration process, improve calibration transfer, enhance operator efficiency, and improve the overall consistency and reliability of analytical results produced using advanced chemometrics and machine earning techniques.

In this column and its successor, we describe and explain some algorithms and data transforms beyond those commonly used. We present and discuss algorithms that are rarely, if ever, used in practice, despite having been described in the literature. These comprise algorithms used in conjunction with continuous spectra, as well as those used with discrete spectra.

There is a variation of the MLR calibration algorithm that can reduce sensitivity to repacked sample measurements. We explore that MLR method here in detail.

A sample library of selected references discussing the application of artificial intelligence (AI) in analytical chemistry and molecular spectroscopy is presented.

Are you intrigued by artificial intelligence, but unsure what it really means for analytical chemistry? Read on.

The past decision to use binary representation in computer architectures affects the results of chemometric-based outputs, especially if different data values are used.

We examine variations of the multiple linear regression (MLR) algorithm confer special properties on the model that the algorithm produces and critique the use of derivatives in calibration models.

Raw data produced by an NIR instrument undergoes some sort of processing, or transformation, to make them easier to use. In this series, we explore options for that data transformation, starting with multiple linear regression (MLR).

The second in a two-part series highlighting key explanatory or tutorial references for each of 29 chemometric methods.

The carefully selected literature references in this curated set describe the application of 29 major chemometric methods used for analyzing molecular spectroscopy data.

Our annual review of new products for atomic and molecular spectroscopy, including details by category and highlights of overarching trends.

Mathematics is a formal logic system, perhaps the ultimate formal logic system. Here we describe the elegance of the foundations of the mathematics that chemometrics is based on.

We explore how different algorithms and different numbers of factors affect the results.

We provide a scorecard of chemometric techniques used in spectroscopy. The tables and lists of reference sources given here provide an indispensable resource for anyone seeking guidance on understanding chemometric methods or choosing the most suitable approach for a given analysis problem.

In celebration of Spectroscopy’s 35th Anniversary, leading experts discuss important issues and challenges in analytical spectroscopy.

Our annual review of products introduced at Pittcon or during the previous year.

A previous analysis of data is compared to the results achieved using classical least squares and principal component analysis. What did we learn?

Our annual review of products introduced at Pittcon or during the previous year

The use of reference materials to align or test the wavelength–wavenumber axis for optical spectroscopy is essential for quantitative and qualitative methods. This article provides details for using reference materials with ultraviolet, visible, near-infrared, infrared, and Raman spectroscopy methods.

What are the steps to take once an outlier is discovered? There are several options.

Our annual review of spectroscopy products, broken down into the following categories: Atomic Spectroscopy, Florescence, Mass Spectrometry, Mid-IR, NIR, NMR and ESR, Raman, UV-vis, X-ray, Software, Accessories, Components

How can you detect the presence of an outlier when it is mixed with multiple other, similar, samples?

Outliers are fundamentally a very fuzzy notion. Here, we try to clear up what outliers are and how they affect your data.

Our annual review of new spectroscopy products introduced at Pittcon, or during the previous year.

As we have previously discussed, the most time consuming and bothersome issue associated with calibration modeling and the routine use of multivariate models for quantitative analysis in spectroscopy are the constant intercept (bias) or slope adjustments. These adjustments must be routinely performed for every product and each constituent model. For transfer and maintenance of multivariate calibrations this procedure must be continuously implemented to maintain calibration prediction accuracy over time. Sample composition, reference values, within and between instrument drift, and operator differences may be the cause of variation over time. When calibration transfer is attempted using instruments of somewhat different vintage or design type the problem is amplified. In this discussion of the problem we continue to delve into the issues causing prediction error, bias and slope changes for quantitative calibrations using spectroscopy.

Part III of this series discusses the principle of least squares

This column addresses the issue of degrees of freedom (df) for regression models. The use of smaller degrees of freedom (df) (e.g., n or n-1) underestimates the size of the standard error; and possibly the larger df (e.g., n-k-1) overestimates the size of the standard deviation. It seems one should use the same df for both SEE and SECV, but what is a clear statistical explanation for selecting the appropriate df? It is a good time to raise this question once again and it seems there is some confusion among experts about the use of df for the various calibration and prediction situations - the standard error parameters should be comparable and are related to the total independent samples, data channels containing information (i.e., wavelengths or wavenumbers), and number of factors or terms in the regression. By convention everyone could just choose a definition but is there a more correct one that should be verified and discussed for each case? The problem with this subject is in computing the standard deviation using different df without a more rigorous explanation and then putting an over emphasis on the actual number derived for SEE and SECV, rather than on using properly computed confidence intervals. Note that confidence limit computations for standard error have been discussed previously and are routinely derived in standard statistical texts (4).

Our annual review of products introduced at Pittcon or during the previous year, broken down by the following categories: accessories, atomic spectroscopy, components, imaging, mass spectrometry, mid-IR, NIR, NMR, Raman, software, UV-vis, and X-ray.