Columns | Column: Chemometrics in Spectroscopy

This column addresses the issue of degrees of freedom (df) for regression models. The use of smaller degrees of freedom (df) (e.g., n or n-1) underestimates the size of the standard error; and possibly the larger df (e.g., n-k-1) overestimates the size of the standard deviation. It seems one should use the same df for both SEE and SECV, but what is a clear statistical explanation for selecting the appropriate df? It is a good time to raise this question once again and it seems there is some confusion among experts about the use of df for the various calibration and prediction situations - the standard error parameters should be comparable and are related to the total independent samples, data channels containing information (i.e., wavelengths or wavenumbers), and number of factors or terms in the regression. By convention everyone could just choose a definition but is there a more correct one that should be verified and discussed for each case? The problem with this subject is in computing the standard deviation using different df without a more rigorous explanation and then putting an over emphasis on the actual number derived for SEE and SECV, rather than on using properly computed confidence intervals. Note that confidence limit computations for standard error have been discussed previously and are routinely derived in standard statistical texts (4).

This column is the continuation of our discussion in part I dealing with statistics.

We present the first of a short set of columns dealing with the subject of statistics. This current series is organized as a “top down” view of the subject, as opposed to the usual literature (and our own previous) approach of giving “bottom up” description of the multitude of equations that are encountered. We hope this different approach will succeed in giving our readers a more coherent view of the subject, as well as persuading them to undertake further study of the field.

The archnemesis of calibration modeling and the routine use of multivariate models for quantitative analysis in spectroscopy is the confounded bias or slope adjustments that must be continually implemented to maintain calibration prediction accuracy over time. A perfectly developed calibration model that predicted well on day one suddenly has to be bias adjusted on a regular basis to pass a simple bias test when predicted values are compared to reference values at a later date. Why does this problem continue to plague researchers and users of chemometrics and spectroscopy?

When using any regression technique, either linear or nonlinear, there is a rational process that allows the researcher to select the best model.

What is it that we thought we knew that we have learned "ain't so" from the work reported in this series of columns?Volume 30 Number 2Pages 24-33What is it that we thought we knew that we have learned "ain't so" from the work reported in this series of columns?  

How do current commercial instruments vary with respect to photometric accuracy and precision over time? What are potential solutions to this challenge?

Now that we have shown the relationships between different units for concentration, we continue by demonstrating their effects on the data we collected and used for our examples. What are the ramifications and consequences of these findings?

What are the techniques and mathematics used to compute uncertainty, and the optimum methods for maintaining wavelength accuracy within instrumentation over time, when considering measurement condition changes?

The data show that different units of measurement have different relationships to the spectral values, for reasons having nothing to do with the spectroscopy. This finding disproves the assumption that different measures of concentration are equivalent except, perhaps, for a constant scaling factor.

The statistical methods used for evaluating the agreement between two or more instruments (or methods) for reported analytical results are discussed, with an emphasis on acceptable analytical accuracy and confidence levels using two standard approaches, standard uncertainty or relative standard uncertainty, and Bland-Altman "limits of agreement."

Calibration transfer is a series of techniques used to apply a single spectral database, and the calibration model developed using that database, to two or more instruments. Here, we review the mathematical approaches and issues related to the calibration transfer process.

Part II of this series surveys the issues related to instrument measurement differences associated with the calibration transfer problem.

A definition for calibration transfer is proposed, along with a method for evaluating it, based on recent discoveries about the nature of light absorbance in spectroscopic analysis.

In the final installment of this series, the main problem is solved using the CLS algorithm to find that the spectroscopy is sensitive to the volume fractions of the various components in a mixture.

The results from the experiment in the second laboratory are calculated and examined.

Here, the results are examined after repeating the original experiment in another laboratory.

The series on classical least squares continues with a comparison of experimental results and theoretical expectations.

Continued discussion of the classical least squares approach to calibration, with a focus on the reconstruction of mixtures

The detailed examination of the spectral behavior of three-component mixtures continues.

In the last four columns we described the theory of what should happen when we perform classical least squares calculations on mixtures when Beer's law applies. In this column we take our first look at what actually does happen.

The connection between the mathematics of classical least squares and the graphical displays used to present it is examined in further detail.

The authors continue their ongoing discussion of classical least squares with a look at spectroscopic theory.

The authors continue their discussion of the classical least squares approach to calibration.

In this month's installment of "Chemometrics in Spectroscopy," the authors begin a new subseries with the goal of explaining the classical least squares algorithm.

This installment of "Chemometrics in Spectroscopy" illustrates the various graphical ways used to observe and interpret comparative clinical quantitative measurement methods.

In this installment, columnists Jerome Workman and Howard Mark describe the statistical underpinnings related to computation and interpretation of chemometric methods and statistics for reporting clinical quantitative measurement methods.

This article describes the application of chemometric methods and statistics for reporting clinical quantitative measurement methods. The equations and terminology are consistent with the Clinical and Laboratory Standards Institute (CLSI) guidelines. These chemometric and statistical methods describe the accuracy and precision of a test method compared to a reference method for a single analyte determination. Part I will introduce these concepts and Part II will discuss the statistical underpinnings in greater detail.

Columnists Howard Mark and Jerome Workman, Jr. take a final look at the topic of principal components, which has been the subject of six previous installments.

This column is a continuation of the set we have been working on to explain and derive the equations behind principal components (1–5). As we usually do, when we continue the discussion of a topic through more than one column, we continue the numbering of equations from where we left off.