Jerome Workman Jr.

In our previous column, we introduced a set of data suitable for more extensive examination than was permitted in our previous discussion of the classical least squares (CLS) algorithm. In this column, we start to analyze the data using two methods (CLS and principal component analysis [PCA]) that do not involve the concentrations of the components of the mixtures. We begin our expansion of the analysis by examining the data from the 70-sample data set the same way we did for the previous 15-sample data set. First, we examine the spectral results, and then apply the CLS algorithm to verify the previous findings. Neither of these types of analysis requires the use of external constituent information. We then further expand the analysis by calculating principal components. 

	Although the goal of this column is to examine the effects of using volume fractions vs. weight fractions as the constituent concentrations for principal component regression (PCR) and partial least squares (PLS) calibrations, there is much to be learned from the spectroscopic data itself, without reference to any constituent compositions. Therefore, our initial inspection of the data parallels the initial examination of the smaller, 15-sample set of mixtures, to verify that those computed values did not require concentration values, or involve the application of advanced chemometric calibration algorithms. Instead, our initial foray into looking at the results from the statistical experimental design described in the previous column (1) was to examine the behavior of the spectral data alone, without involving the use of chemometrics. The application of chemometrics to the data will be described later, and for reasons that will become clear, may not be described fully here. The advantage of analyzing the spectral data without bringing in the constituent values is that the expectations of their behavior are better known, and more easily described. This allows us to compare our results with the theoretical expectations, so that when we introduce the constituent values, we will have some foreknowledge of how we might expect the data to behave. Some of these behaviors have already been listed qualitatively in Tables I and II (see the previous column [1]), where we compared current near-infrared (NIR) spectroscopy practice to theoretical expectations.

	While we have spectra of the pure materials that comprise the mixtures, they were not taken as part of the calibration set. Therefore, for comparison and demonstration purposes, we use the five mixtures containing each of those materials at 60% (the maximum value they can attain in these samples) with each of the other mixture components at their lowest value (10%) as surrogates for the pure materials. The plots in Figure 1 present these spectra, for reference.

	In keeping with our desire for initial analysis to not use the reference values, our next activity is to present the results of performing classical least squares (CLS) analysis on the mixture data. One aspect of CLS gives us the ability to reconstruct the spectrum of a sample from the spectra of the pure components. Thus the validity of the data analysis, as well as the quality of the data, can be captured by CLS analysis by the agreement between a pure-component spectrum computed from the mixture spectra and the measured spectrum of that chemical component. These plots are shown in Figure 2, for the mixtures containing the maximum amount (60 wt%) of each of the ingredients respectively, and for each mixture the minimum amount (10 wt%) of all the other ingredients. It is clear that the agreement between the theoretical reconstructed spectrum and the actual spectrum of each ingredient is almost, although not quite, as good as the agreement between duplicate spectra of the same material. This an important observation, since later on we will be comparing spectra to each other, and knowing the amount of deviation expected between actual measurements and computed approximations will be important in evaluating the computed results.

	In Figure 3, we have plotted, for each of the five ingredients, the values of the CLS-calculated concentrations vs. the known weight percent and volume percent of the ingredients. Not surprisingly, we note the following characteristics of the plots in Figure 3:

	For the plots where the abscissa variable represents the weight fraction, the points corresponding to a given weight fraction of the ingredient lies on a vertical line. The explanation is simple-the samples were made up gravimetrically to prescribed values, thus the actual and the nominal values of weight fraction are all the same (within experimental error) for a given fraction of the specified ingredient. The spectral values plotted on the ordinate direction varied due to the actual changing composition, reflected in the spectral response, hence the data points corresponding to samples of the nominal composition are spread out in only the 

	For the plots where the abscissa variable represents the volume fraction, the points corresponding to a given volume fraction of the ingredient lie along a line essentially parallel to the direction of the line through all the samples of varying composition. This reflects the improvement of the performance attainable through the use of volume fractions. As we’ve seen previously with the more limited sample set (2), when volume fractions are used as the measure of concentration, the change in value of the constituent concentration is mirrored by an equivalent change in the value of the concentration as determined by the CLS algorithm (2). The calculated values of the concentration measure within each group of data points then follows the trend of the data very nicely, instead of them all lying vertically as in the weight percent values of Figure 3.

	The difference between the two cases is that when weight fraction is used for the constituent concentration, the value used to represent concentration of the sample is constant, because the samples were made up that way-to contain a pre-specified amount of the ingredient. In this case, only the spectroscopically calculated value changes (due to the variations in the other components), and therefore the resulting data points fall on a vertical line. When the volume fraction is used as the constituent value, then both the spectroscopically calculated value and known reference value change, and change concurrently, so that the data points for each sample lie on a sloping line that reflects the corresponding changes in the two values.

	Another characteristic of the data can also be seen in some (although not all) of the plots in Figure 3. Some of the clusters of data points for the plots of spectral values versus volume fractions show an additional effect. This effect is the spreading of the data points away from the line defining the main direction of the points in each cluster, in this case perpendicular to the line. In some cases (and this is particularly noteworthy for the butanol plots, methanol plots, and to a lesser extent, for dichloropropane plots), the clusters of points are not randomly spaced away from the line defining the direction of the data, but form a pattern. Albeit imperfect, the pattern shows lines of data points within the cluster, each line containing a different number of data points: first one isolated point, then two on a line, then three, then four. The net effect is that for each cluster, the points appear to be contained in a triangular envelope, mimicking, to some extent, the experimental design from which the plots were created.

	The cause of this phenomenon is not entirely clear at this time, although suspicion falls on the potential residual interactions between the various chemical components of the samples. The possibility of hydrogen bonding between the two alcohols and the highly polar -C-Cl bonds in the chlorine-containing molecules also seem likely sources of this sort of interaction.

	CLS is not the only type of data analysis that can be applied to the spectral data without needing constituent values. Principal component analysis (PCA) is a data analysis method that can be applied to the spectra alone, without needing auxiliary constituent information. We have shown in earlier columns (3–8) that a principal component is a least-square estimator of the underlying data, and that, at each stage, it accounts for the maximum amount of variance remaining in the data after all previous components have been used to explain their respective contributions to the measured data values.

	This being the case, if the theoretical assumptions underlying the data are valid, it should not be necessary to compute more principal components than there are independent sources of variation affecting the data set being examined. This is effectively the same as saying this reflects the number of degrees of freedom in the data, ignoring the innumerable degrees of freedom that represent the noise content of the spectroscopic data.

	In the case of five-component mixtures such as we are dealing with here, you might think that there would be five degrees of freedom in the spectral data. However, since the mixture components must add up to 100% (called 

 of the dataset), the concentration of any four components determines the concentration of the fifth; therefore it is not “free” to vary independently. The restriction that the mixture ingredients must add up to 100% removes a degree of freedom from the data. Therefore the spectral data should contain 5 - 1 = 4 degrees of freedom. Ideally, therefore, a five-component mixture should be completely represented by four principal components, ignoring the noise. What we should observe then, upon performing PCA, is that the variance remaining in the spectral data should decrease as each new principal component is computed and the effect of that component is removed from the data set. Furthermore, the early components will account for more variance than later components; the first principal component that is calculated will account for the most variance of the data, while each of the later components will remove less and less. The effect is cumulative, and successive components will remove more and more variance from the data until the first four components are found, at which point the remaining variance should represent the random noise remaining in the data, and that noise should not change by any appreciable amount if more principal components are computed. The remaining variance will then remain almost constant as successive principal components are calculated. Plotting this residual variance as a function of the number of components should therefore approach a very small almost-constant value, once all the non-noise variations of the data are removed by this process. The actual behavior of this data set is shown in Figure 4.

	What we actually find in Figure 4, distinct from the theoretical expectation, however, is that there is a continual drop-off of the remaining variance in the data up to at least the sixth principal component. In conformance with theory, the earlier principal components do indeed account for more variance than later ones. Nevertheless, the number of principal components needed to account for or “explain” the spectral data is more than the four degrees of freedom, due to the five individual chemical components that we know are present in the data set (while accounting for the closure of the dataset). The reason for this is also not entirely clear, although it is consistent with the finding, from the examination of the CLS results, of possible interactions between some of the ingredients in the mixtures.

Conclusions (From This and the Previous Column)

	CLS analysis of this new data set, which is appreciably larger than the previous data set we were able to use, confirms the results we obtained from the previous data set. The CLS algorithm enabled the reconstruction of the spectra of mixtures from the spectra of the components of each mixture. For several of the constituents (methanol is a good example here), the plot of calculated vs. measured concentration is a straight line when the measured concentration is expressed as volume fraction, but exhibits visible curvature when the concentration is expressed as weight fraction.

	Principal component analysis shows that, contrary to theoretical expectation, it requires the calculation and use of more (>6) principal components to account for all the variance in the spectra than theoretical evaluations (four principal components) of the data would suggest (see Figure 4 for the actual amounts of variance explained). Initial explanations for this behavior, based on previous understanding, cast suspicion on more of the “usual suspects”-mainly interactions between ingredients in the mixtures list.

	Further, and closer, examination of the results revealed that, while interactions cannot be completely exonerated as a cause of the observed behavior, another unexpected phenomenon showed up, which changes our understanding of how apportionment of variance among several competing effects influences the final results obtained. This phenomenon will be described in a subsequent column.

		H. Mark, R. Rubinovitz, D. Heaps, P. Gemperline, D. Dahm, and K. Dahm, 

Jerome Workman Jr. serves on the Editorial Advisory Board of 

. He is also a Certified Core Adjunct Professor at U.S. National University in La Jolla, California. He was formerly the Executive Vice President of Research and Engineering for Unity Scientific and Process Sensors Corporation.

Howard Mark serves on the Editorial Advisory Board of 

, and runs a consulting service, Mark Electronics, in Suffern, New York. Direct correspondence to: 

Articles

A previous analysis of data is compared to the results achieved using classical least squares and principal component analysis. What did we learn?

More About CLS, Part 2: Spectral Results (Not Needing Constituent Values) and CLS

We continue with our review of the methods and mathematics involved with the subject of calibration transfer. Calibration transfer involves several strategies and mathematical techniques for applying a single calibration database consisting of samples, reference data, and calibration equations to two or more instruments. In this installment, we continue by providing a review of the chemometric and tactical strategies used for the calibration transfer process.

	The basic methods that have been published related to calibration transfer mathematics and the techniques involved in this process are discussed as a continuation of our previous column (1).

	The use of the successive projections algorithm (VSPA) to select variables for building robust transferable multiple linear regression (MLR) models has been demonstrated (2). Robust MLR models have been compared to partial least squares (PLS) models using piecewise direct standardization (PDS) to correct the secondary or child instrument spectra. In data sets tested, the mean prediction errors at the child instrument for the robust VSPA-MLR and the PLS-PDS models were comparable, and slightly better for VSPA-MLR. The VSPA approach appears to be a valid alternative to the commonly used PLS-PDS technique.

	An example of the VSPA for selecting variables for building robust transferable MLR models has also been demonstrated. The transfer sample set is selected using a Kennard–Stone (KS) algorithm or modified VSPA that is applied on the rows of the instrument response matrix. In published examples, the mean prediction errors of the secondary or child instrument are similar for VSPA-MLR as compared to PLS-PDS models. For some applications, the VSPA-MLR method produces lower prediction errors.

	Direct standardization in the compressed wavelet domain is termed 

. Calibration transfer in the compressed wavelet domain permits a reduction in processing time for the analysis of large spectral data sets with some success in calibration transfer (3).

	In a published example, wavelet transform methods such as denoising, compression, and multiscale analysis have been developed for near-infrared (NIR) calibration transfer. Combined methods have been empirically tested to improve the precision of PDS. Methods compared have included

		wavelet denoising and piecewise direct standardization (WDPDS);

		wavelet compression and piecewise direct standardization (WCPDS); and

		wavelet multiscale and piecewise direct standardization (WMPDS).

	Optimal results have been reported for these combined methods on various data (4).

	A method based on canonical correlation analysis (CCA) for calibration model transfer was developed for calibration transfer (5). In this work, two real NIR data sets were tested in a comparative study between CCA and PDS. A comparison of methods was most positive for the CCA technique.

	Another approach to calibration transfer has been developed based on a revised factorization technique called 

 (PMF). PMF was developed and initially applied to environment data analysis (6). It has important differences from PCA. Because it is a least squares approach to solving the factor analysis problem, it can use subjective weights for individual data points and thereby allows the inclusion of uncertain data points-for example, missing data or analysis values below the detection limit.

	A calibration transfer method for NIR spectra based on spectral regression has been examined (7). A comparative study of a spectral method and PDS for standardization on two benchmark NIR data sets was demonstrated with the results using spectral regression similar to those obtained using PDS with background correction.

	A procedure for the transfer of the regression equation in NIR spectroscopy, from a first (parent) instrument to a second (child) instrument, is presented in an example paper (8). The procedure uses PLS regression twice: in the first step to compute the relationship between the spectra of transfer samples of the two instruments, and in the second step to compute the regression equation (relationship between chemical variables and spectral variables) of the first instrument. In the example paper, spectra were recorded with four different instruments, in four different laboratories with transfer results given, and reported as standard error of prediction. The spectral regression method prediction results for the transfer instrument were reported to be comparable to the original calibration.

Wavelet Packet Transform Standardization 

wavelet packet transform standardization (WPTS)

, based on a wavelet packet transform (WPT) and entropy criteria, was proposed to transfer spectra between two NIR spectrometers (9). By comparison, the example WPTS produces a predicted value transfer performance comparable to wavelet transform standardization 1 (WTS1), which is considered one of the best existing transfer methods, and better than wavelet transform standardization 2 (WTS2) combined with standard PDS (WTS2-PDS). The WPTS method also has a low computational requirement as an added benefit.

	A paper demonstrated the use of stacked partial least-squares (SPLS) regression and stacked dual-domain (SDD) regression analysis combined with commonly used techniques for calibration transfer to improve predictive performance from transferred multivariate calibration models (10). The paper reported that the predictive performance for gasoline samples was improved when combined with three conventional calibration transfer methods-namely, PDS, orthogonal signal correction (OSC), and model updating (MUP). These methods require transfer standards to be measured on both instruments.

Multiplicative Scatter (or Signal) Correction 

	An applications paper compares two multiplicative signal correction (MSC) algorithms for the standardization of data from two NIR spectrometers (11). Absorbance spectra were measured in the 1000–2200 nm range for a set of 45 jet fuel samples. Data from one instrument were standardized to match data from a second instrument using windowed MSC (W-MSC) and moving window MSC (MW-MSC). For W-MSC user-defined windows were selected and for MW-MSC the window size was optimized based on a two-step procedure: assigning a cut off window to avoid over-processing and selection of a specific window size based on sample leverage.

	Multiplicative scatter (or signal) correction provides some mitigation of the scattering phenomenon by regressing the individual spectrum to the mean spectrum of a collection of spectra, for both calibration and prediction datasets (12–15). The identical algorithm must be completed for both calibration and prediction sets. When conditions or datasets are changed the algorithm is recomputed for the entire set. The algorithm assumes that the ideal spectrum is the mean spectrum, which is an oversimplification, but it provides a basis for making some correction. Light scatter and spectral measurements are complex and poorly understood so correction algorithms, although useful, do not present an ideal theoretical correction for the scattering phenomenon, but only an approximation.

	For the MSC algorithm, each spectrum in a set of spectra represents the dependent variable (as a vector) in a least squares regression case where the mean spectrum is the independent variable (also a vector). From this computation, both the slope and intercept MSC variables are computed. So, the general least squares formula in standard notation is as shown in equation 1:

	For this least squares linear regression equation, 

 is the mean spectrum of the set of spectra, 

	The estimated or corrected MSC spectrum is computed using the computed slope (

	MSC approximates a regression estimate of the mean spectrum with the remaining unexplained residual spectrum representing the spectral differences between the spectra within a set. This residual difference is assumed to contain much of the chemical information differences between samples in the spectrum set.

	The standard normal variate (SNV) data preprocessing method partially compensates for some spectral slope and particle size differences within a series of spectra measured using diffuse reflection (16–19). The algorithm is applied individually to each spectrum for both calibration and prediction spectra. This processing step is applied to each data point for each spectrum using the mean and standard deviation of the spectrum in absorbance space (units). The computation is straightforward as the mean absorbance difference for each data point to the mean absorbance of the spectrum ratioed to the standard deviation of the absorbance for the spectrum. The computation formula is as shown in equation 3:

 is the computed standard normal variate for the datapoint (element) 

 is the number of data points or elements in the spectrum.

is a general term used to describe the removal of a basic trend from data to more closely determine the signal attributes associated with analyte concentration or basic information content in a signal (16). To remove the trend (that is, detrend) data in a time series a linear least squares fit is determined for the data and then subtracted from the data. For unusual shapes, such as spectra, a polynomial function of a selected order is fit to all the data points in the spectrum and subtracted from the spectrum. The polynomial is used to fit all data points in the spectrum. This technique will remove large background interference, or specific trending variation, and is often used in combination with the SNV preprocessing function. A detrend algorithm consists of a polynomial or linear fit to each spectrum and a subtraction function of the fit function from the spectrum.

	Spectrometer instruments are characterized by classifying their spectra into previously defined clusters. The spectra are mapped to the clusters and a classification is made based on the similarity of extracted spectral features to one of the previously defined clusters. Calibration models for each cluster are provided to compensate for within or between instrumental variation. A simplified method of calibration transfer maps clusters to each other, so that a calibration transferred between clusters models only the difference between the two clusters, substantially reducing the complexity of the models used for transfer (20).

	A spectrum simulation method is described for use in the development and transfer of multivariate calibration models from NIR spectra (21). By use of previously measured molar absorptivities and solvent displacement factors, synthetic calibration spectra are computed using only background spectra collected with the spectrometer for which a calibration model is desired. The resulting synthetic calibration set is used with PLS regression to form the calibration model. This methodology is demonstrated for use in the analysis of physiological levels of glucose (1–30 mM) in an aqueous matrix containing variable levels of alanine, ascorbate, lactate, urea, and triacetin.

	NIR calibration models have been developed for the determination of content uniformity in pharmaceutical tablets (22). Data used in one study were acquired from five NIR instruments manufactured by two different vendors. One of these spectrometers was a dispersive-based NIR system while the other four were Fourier transform (FT) based. The transferability of the optimized PLS calibration models developed on the primary instrument (A) located in a research facility was evaluated using spectral data acquired from secondary instruments B, C, D, and E. Instruments B and E were located in the same research facility as spectrometer A while instruments C and D were located in a production facility 35 miles away. The same set of tablet samples was used to acquire spectral data from all instruments. To minimize the effect of instrument variability, calibration transfer techniques such as PDS and wavelet hybrid direct standardization (WHDS) were used.

	A principal component regression calibration model developed on a scanning NIR spectrophotometer (with a standard cuvette) was transferred to a Fourier transform NIR spectrophotometer (with a fiber-optic probe) for the determination of sodium chloride in aqueous solutions by using the piecewise direct standardization method (23). Only two transfer samples were needed in obtaining a good calibration transfer.

	The objective of one study was to assess the potential of calibration transfer from the Foss NIRSystem 6500 instrument to the Polychromix Phazir instrument (24). The results show that good calibration models were obtained for various feed properties (that is, fat, fiber, protein, and starch) developed on a Foss NIRSystem 6500 instrument, based on a spectral database of 9164 samples transferred to a microelectromechanical (MEMS)-based Polychromix Phazir handheld spectrometer.

	Calibration transfer between spectra measured at different temperatures was investigated for NIR spectroscopic determination of sodium chloride in aqueous solutions (25). A PCR calibration model developed at 23.0 °C was transferred to a model at 28.5 °C by using a PDS method.

	A method for calibrating two-dimensional (2D) responses measured on multiple instruments or on a single instrument under different operating conditions has been demonstrated (26). From computer simulation and experimental data, it was found that the design of the standard sample spectral profile is crucial for the parameter estimations and response standardization. This is because of the phenomenon that instrument response is ample dependent.

Compensating for Extraneous Phenomena (or Unmodeled Variation)

	The analysis accuracy and precision of any multivariate calibration method will be severely degraded if unmodeled sources of spectral variation are present in the unknown sample spectra. A synthetic generalized method was developed and published for correcting errors generated by the presence of unmodeled components or spectral variation (27). This method requires that the spectrum of the unmodeled component be simulated or measured and mathematically added in variable amounts to the original calibration spectra. After this adjustment, a revised calibration model is generated. This new calibration accommodates the unmodeled source of spectral variation. This method minimizes the effect of long-term drift on prediction errors and using the synthetic method eliminates the need for the expensive generation of new calibration samples.

prediction-augmented classical least-squares 

), and does not require that all interfering spectral characteristics be known during the calibration process. PACLS can correct calibration models if unmodeled spectral components can be identified, simulated, and added during a PACLS prediction step (28). The addition of a classical least-squares–partial least-squares (CLS–PLS) hybrid algorithm, and a prediction-augmented classical least-squares–partial least-squares (PACLS–PLS) hybrid algorithm improves the calibration process even in the presence of unmodeled sources of spectral variation (29,30).

	An additional prediction-augmented classical least-squares–partial least-squares (PACLS–PLS) hybrid algorithm has been demonstrated for use in calibration transfer between instruments (31). Dramatic spectral differences caused by deviations in instrument response are incorporated into the algorithm through the use of sample spectra measured on both parent and child spectrometers.

Creating Transferable Calibrations in Mid-IR

	Sensitivity studies showed that detector nonlinearity, frequency accuracy, incident angle, and variation in purge gases are important parameters to obtain transferable calibrations for Fourier transform infrared (FT-IR) calibrations (32). As the complexity of the modeled variables and calibration increases, calibration transfer becomes more challenging. Outlier detection methods are useful to determine instrument similarity and goodness of fit for a specific spectrum to a calibration database. Calibrations that maintained limited variance in instrument factors, subtracted purge gas spectra, and used data from specific robust frequencies were successfully transferred, and were also robust against spectrometer drift.

	This column completes the overview of calibration transfer mathematical approaches. As new techniques are developed and published we hope to provide updates on this topic.

	(2) F.A. Honorato, R.K.H. Galvão, M.F. Pimentel, B. de Barros Neto, M.C.U. Araújo, and F.R. de Carvalho, 

	(5) W. Fan, Y. Liang, D. Yuan, and J. Wang, 

	(7) J. Peng, S. Peng, A. Jiang, and J. Tan, 

	(8) M. Forina, G. Drava, C. Armanino, R. Boggia, S. Lanteri, R. Leardi, and R. Bigoni, 

	(11) K.E. Kramer, R.E. Morris, and S.L. Rose-Pehrsson, 

	(13) M.S. Dhanoa, S.J. Lister, R. Sanderson, and R.J. Barnes, 

	(14) H. Martens, J.P. Nielsen, and S.B. Engelsen, 

	(15) J.-W. Jin, Z.-P. Chen, L.-M. Li, R. Steponavicius, S.N. Thennadil, J. Yang, and R.-Q. Yu, 

	(16) R.J. Barnes, M.S. Dhanoa, and S.J. Lister, 

	(17) A. Candolfi, R. De Maesschalck, D. Jouan-Rimbaud, P.A. Hailey, and D.L. Massart, 

	(18) T. Woodcock, G. Downey, D. Kelly, and C. O'Donnell, 

	(19) K. Tanaka, Y. Kuba, T. Sasaki, F. Hiwatashi, and K. Komatsu, J

	(20) K.H. Hazen, T.B. Blank, S. Monfre, snd T.L. Ruchti, "Method of Characterizing Spectrometer Instruments and Providing Calibration Models to Compensate for Instrument Variation," U.S. Patent 6864978 (2005).

	(22) Y. Sulub, R. LoBrutto, R. Vivilecchia, and B.W. Wabuyele, 

	(24) J.A.F. Pierna, P. Vermeulen, B. Lecler, V. Baeten, and P. Dardenne, 

	(26) B.R. Kowalski and Y. Wang, "Calibration transfer for second order analytical instruments," U.S. Patent 5559728 (1996).

	(30) C.M. Wehlburg, D.M. Haaland, D.K. Melgaard, and L.E. Martin, 

	(31) C.M. Wehlburg, D.M. Haaland, and D.K. Melgaard, 

	(32) I.S. Adhihetty, J.A. McGuire, B. Wangmaneerat, T.M. Niemczyk, and D.M. Haaland, 

Calibration transfer involves several strategies and mathematical techniques for applying a single calibration database consisting of samples, reference data, and calibration equations to two or more instruments. In this installment, we review the chemometric and tactical strategies used for the calibration transfer process.

Jerome Workman Jr.

Articles

More About CLS, Part 2: Spectral Results (Not Needing Constituent Values) and CLS

Calibration Transfer Chemometrics, Part II: A Review of the Subject