Precision and Accuracy in Chemometric Calculations

As we celebrate 40 years of Spectroscopy, we also mark 38 years of sharing mathematical musings through a column that began life as “Statistics in Spectroscopy (SiS).“ This all started back in January 1987 (Vol. 2, No. 1), with our debut column titled, “Introduction—Random Numbers or Real Data?”—a question some of us are still asking. Our very first article in Spectroscopy actually predates the column, appearing in 1986 (Vol. 1, No. 5) under the unforgettable title, “Critical Values of Several Statistics at High Probability Levels for Use with Near-Infrared Wavelength Searches” (Try computing those in your head). In 1993, the column evolved—both in name and in scope—becoming “Chemometrics in Spectroscopy” (CiS), launched with the aptly titled piece, “A New Beginning” (Vol. 8, No. 4). And three decades later, we’re still crunching chemometrics problems and challenging assumptions—one column at a time.

“Everyone knows” that the National Institute of Standards and Technology (NIST) is the official government agency that defines physical standards (for example, the second, the foot, the meter, the volt...) for the United States with the maximum precision and accuracy that modern science is capable of. Chemists know that NIST provides samples of specified composition, defined and prepared with the maximum precision and accuracy it is capable of, as a means of transferring NIST’s precision and accuracy capabilities to its users. Hardly anyone knows that NIST provides data sets for testing mathematical algorithms, along with the results of applying error-free computation of those algorithms to the specified data sets, for the purpose of enabling users to compare the correct results to the results obtained from applying their implementations of the algorithms. Currently, test data for, and correct results from, only a few algorithms are available on the NIST.gov website; these do not include our favorite algorithms.

Introduction

How many of us have never wondered about how accurate the results from their favorite algorithms are? Our government, embodied in NIST, a Congressionally-defined and created agency within the Department of Commerce, that has a mandate to create a capability to perform error-free statistical/chemometric computations and disseminate their certified results to users via the NIST.gov website. Users can then apply their own software implementations of the algorithm to the same data, and compare the results from their implementation of the algorithm to the error-free results and thereby find out the answer to the question heading this paragraph. As of this writing, that capability has not been fully realized inasmuch as only a few of the simplest statistical algorithms have been processed and the corresponding certified results posted on the NIST.gov website.

As chemists, we are all familiar with the concepts of precision (the ability to repeatedly obtain a result close to the same value of a measurement when an experiment is repeated) and accuracy (the ability to repeatedly obtain a result as close to the “truth” as possible (whatever “truth” may be in any given situation). Precision is relatively straightforward to assess and to test for (although difficulties sometimes arise in practice); a chemist need only to repeat a measurement multiple times (which may or may not require performing auxiliary operations, including but not limited to heating, grinding, dissolving, and running chemical reactions) and determine the resulting value of the property of interest for each measurement. Those operations are important for the chemist, but not of particular interest for chemometricians, whose concern is with the final resulting values.

The most important characteristic of the “final resulting value” is the agreement of that value with “truth.” Ignoring the philosophical implications of the “search for truth,” a chemist performing quantitative analysis, whether using wet chemistry, spectroscopy, or any other analytical technique, believes that there is indeed a number characterizing the amount of the analyte of interest in the sample being examined. The “amount” of analyte in a sample is typically expressed as “concentration” in order to ensure that the determination of the amount of analyte is independent of the size of sample being analyzed. This implicit value representing the amount of analyte in the sample is known to statistical and chemometric science as a parameter of the sample and is generally considered to be the best estimate of “truth” that is available. Over the years, chemists have created numerous ways to specify (and measure) “concentration”: molarity, molality, weight fractions or percents (common in commercial and industrial applications), milligrams per deciliter (mg/dL; common in medical applications), and others. For at least some chemometric usages, it turns out that it matters what the units the concentration is expressed in (1) (also see [2] and the references therein). The agreement with “truth” is what we call “accuracy.”

This brings us back to the question of knowing what “truth” is. Trying to figure that out occupied the attention and efforts of scientists (and philosophers!) for hundreds of years: chemists, physicists, biologists, geologists, mathematicians, astronomers, and others, all trying to decipher and make sense of a plethora of confusing and sometimes almost contradictory results. This is not the place to go into the details; consult a book about the history of science for that. What we can do here is to note that the key knowledge accumulated slowly, as various scientists found relationships between the different phenomena they studied. Hence today, we know of relationships that we now call Boyle’s Law (the property of a gas that the volume it occupies is inversely proportional to the pressure applied to it), Charles’ Law (the property of a gas that the volume it occupies is directly proportional to the (absolute) temperature of the gas), the (chemical) Law of Constant Proportions (a given chemical reaction always involves the same relative amounts of the reacting materials), and knowledge of numerous other laws of nature.

While many of the investigations leading to that knowledge were performed for the sake of the knowledge gained, much of it also contributed to the technological and commercial advancements of the times. For these reasons, it was soon recognized that, in order for the proposed benefits of those advancements to be achieved, it was important that scientists be able to compare results from separate, independent measurements, in order to verify that the same results could be achieved from the same experiment regardless of where the experiment was conducted or who conducted it. That, in turn, required that the conditions of the experiment be the same regardless of who or where the experiment was performed; that “a gram” is the same gram for all users. Similarly, “a centimeter” should be the same centimeter for all, “a volt” is the same volt for all, “a second” is the same second, and so on, for all users. In short, all the units of measurement had to be agreed on (standardized) in order to ensure that the results could be comparable between scientists and be usable for commerce, as well as for further development of measurement standardization efforts.

In order to achieve that goal for all users, governments of the countries joining this effort established agencies to implement the necessary functions domestically and to consult with other corresponding agencies internationally in order to implement this program worldwide. In the United States, the agency with that responsibility was initially called the National Bureau of Standards (NBS), currently called the NIST, and is organized as part of the Department of Commerce. Official information about NIST, its products and activities is available on its website, www.NIST.gov.

NIST

The NBS was established by Congress in 1901 as an agency within the Department of Commerce in order to ensure that the United States would become and remain competitive on the world stage as a leader in scientific and technological development.

The need for standards is recognized by the U.S. government at a very high level (3). The title of that report is “What Standards are and Why They Matter” and that is the reason why NIST, along with many other standard-setting organizations exist in the United States and around the world. There are many standards-setting organizations in the United States that create consensus standards for specific scientific, commercial, and industrial applications; a key example is the American Society for Testing and Materials (ASTM) International (4,5). NIST, on the other hand, having been chartered by Congress has unique legal status; some of its standards (such as weights and measures) comprise their legal definitions for the whole of the United States. When appropriate, other standard-setting organizations collaborate with NIST to define and issue joint standards. To achieve all this, NIST contains many subagencies to deal with the wide variety of sciences and activities within its purview. In this article, we discuss only a few of the activities that NIST pursues, and only those that are of interest to chemists and chemometricians. Upon entering the NIST website (www.nist.gov), the user is presented with a list of sixteen classes of resources that NIST provides, grouped into three categories:

• Services and resources
• Laboratories
• Extramural programs

Some of these subagencies are moderately well known to the general public. For example, anyone wanting to know the correct time can select “Official U.S. Time” from the list or, more directly, simply enter “www.time.gov” into the search box of their Internet browser to find the current time (corrected for network delays) in their time zone.

Information about some of the more obscure and specialized topics can be found in several different ways. One simple way is to enter the topic name into a search box provided at the top of the main page (and other pages) of the NIST.gov website. Other subagencies are still more obscure and known mainly to those scientists or other users dealing with that particular scientific topic.

For Chemistry

Of interest to chemists (and other scientists) is a section of the main page explaining NIST’s support for Standards & Measurements of chemical interest. This section contains the following sub-sections:

Calibrations: As of this writing, this section provides entrance to the new NIST storefront, which enables a user to order calibration services.

Documentary Standards: These can define terms, classify components, delineate procedures, and specify dimensions, materials and processes (www.standards.gov).

Standard Reference Data: NIST provides both free and fee-based standard data and standard samples so that people can base significant decisions on the data, which must be compliant with rigorous critical evaluation criteria.

Standard Reference Materials (SRM): Materials that are certified for properties of interest and may be used to calibrate instruments, processes, or any other activity for a numerically-defined property, These are arguably the most common source of interaction between chemists and NIST. A video, “What is a Standard Reference material or SRM,” is available at www.NIST.gov/video/what-standard-reference-material-or-SRM. A list of available SRMs may be inspected at:www.shop.NIST.gov/ccrz_CCpage?pageKey=SRMCategory&ccld=en_US%29.

Under subsection “Laboratories,” one finds “Material Measurement Laboratory,” which includes:

• Products and Services
• Standard Reference Materials (SRM)
• Tables of SRM categories.

To obtain an SRM: SRMs may be ordered at the NIST store (https://shop.nist.gov). A table of composition values for SRM may contain both certified and non-certified values. In use, the SRM may be analyzed by the instrument or method under test and the measured value compared to the certified value.

Standard Reference Instruments: This section contains a list of 12 instruments that transfer to the user the ability to make reference measurements or generate reference responses based on NIST reference instrument designs. These include voltage standards, vacuum standards, optical radiometers and others.

For Chemometrics

As can be seen from the above breakdown, the NIST website contains a good deal of information and can sometimes be complicated to navigate. The preceding gives the reader much general information about NIST. Where does chemometrics fit in?

As the title of this article implies, as with other functions of this government agency, the concern is with precision and accuracy in measurement. We all know, to some extent, NIST’s reputation and activities in creating, promoting, organizing, and executing activities to improve physical measurements. So what has all this got to do with chemometrics?

With the errors due to physics and chemistry under control, the scientists at NIST turned their attention to discovering and eliminating errors due to the mathematics. From what I’ve seen, very few people know what NIST has done to improve the mathematical and computational aspects of performing the calculations for determining the final answer for a measurement. But unbeknownst to most, scientists at NIST have indeed been busy in analyzing sources of error in computation, developing ways to eliminate or minimize the errors and reduce or eliminate their effect on the final answer. From the point of view of chemometrics, NIST’s main fault may be in not going far enough.

The contents of the NIST website relevant to our chemometric interests are currently very sparse, but the agency is amenable to expanding on and improving its offerings based on the needs of its users, so it’s up to us to let the agency know what we need and want. On the other hand, the current offerings provide a framework that we can build on, and use to take us in directions we have not previously considered.

Recall that NIST is, among its other purposes, a repository of ultimate precision and accuracy. Entering the following URL into your browser: https://itl.nist.gov/div898/strd/index.html will bring you to a fairly obscure part of the NIST.gov website, a part of the website containing the NIST Standard Reference Databases (SRDs). Here, we find several sets of data, each suitable as input for one or another statistical algorithm and designed to stress the capabilities of the corresponding algorithm to the maximum.

Before chemometrics was a defined discipline, the mathematics underlying its constructions existed, were available, and were classified as belonging to the science of statistics. In fact, some scientists, including the authors of this article, consider chemometrics to be a collection of the more advanced and sophisticated parts of statistical science. NIST, however, took the statistical sciences under its wing and into its website, long before chemometrics was recognized as an independent discipline. Thus our favorite chemometric algorithms and corresponding datasets we commonly use are generally not included among the NIST SRDs.

The treatment of statistics under the NIST umbrella was constructed so as to parallel how NIST’s efforts were directed to ensuring that any results they produced, or that users would produce by following their methods and instructions, would be the result of performing the statistical operations with maximum precision and accuracy.

NIST implemented this capability by creating (or finding, as appropriate) datasets designed to test the capabilities of the software to compute the results without error from applying a specified algorithm to the data.

How could this be otherwise? An overview of the operations of mathematics tells us that any mathematical operation on given dataset can produce one and only one exact result.

Error Sources

Nevertheless, closer inspection reveals problem areas and weaknesses. Every statistical algorithm has a set of error sources that affect the results. NIST specialists found that even the most basic statistical calculations—such as mean, standard deviation, and correlation coefficient—are affected by at least three types of errors (see Background Information for Univariate Summary Statistics):

• Truncation error
• Cancellation error
• Accumulation error

Truncation error occurs when the underlying basis of the data values are changed (for example, Binary to Decimal conversion) as we’ve discussed previously (6). Indeed, even without a change of the numerical basis, any number written into a computer memory, or even on a piece of paper, is subject to the truncation error, although we may never know what might follow the last digit of the number we are presented with, regardless of the number base.

Cancellation error occurs in data with small variations (compared to the magnitude of the data). Accumulation error increases in proportion to the total number of arithmetic operations performed, which is to say that the errors of the individual operations are additive.

When larger, more complicated operations and algorithms are executed, additional error sources can raise their nasty heads. Some algorithms are subject to as many as five error sources. Due to space limitations we do not copy all the information from the NIST website here, but see the “Background Information” for the various algorithms discussed on the NIST.gov website

The discussions (“Background Information”) accompanying each algorithm on the NIST.gov website list the error sources the algorithms is subject to and include recommendations for simple steps to address the problems each algorithm is subject to.

As mentioned above, the NIST.gov website presents and analyzes only a small set of algorithms. Those comprise a very limited set of the algorithms known to chemometric science. Part of the reason for writing this column is to encourage the chemometric community to band together and approach NIST to include the more sophisticated algorithms that we use “all the time,” but that NIST has not dealt with at all.

How Does NIST Distribute the Data Standards?

On the NIST.gov website, NIST provides, at no charge, several data sets for testing each of the various statistical algorithms included in the study. Each algorithm has its own set of problematic error-causing sources, which are discussed, and the data sets include the problems. Values of important results appropriate to each algorithm are provided, certified accurate to 13 significant figures, this enables users to compare results from their own software to the certified values. The certification process for each algorithm is unique to the algorithm and is described along with the results for that test dataset.

Currently, data sets and certified results are provided for testing algorithms to compute:

• Univariate summary statistics
• Analysis of variance
• Linear regression
• Nonlinear regression (See the NIST.gov discussion for this algorithm. NIST ascribes a different meaning to “non-linear regression” than the meaning we understand in chemometric applications.
• Markov Chain Monte Carlo

Each algorithm’s information is comprised of:

• A descriptive paragraph of background Information
• One or more sets of test data
• Certified results from applying the specified algorithm to the test data
• Any pertinent details, including a description of the certification process.

High–Precision Computations

In order to avoid the error-inducing problems attendant on limited computational precision, a special multi-precision software package (7) was used to perform all the computations used for this exercise. The software used was a special high-precision package written in FORTRAN. There was mention of a corresponding package written in C, but no details about that software were available, and in any case the NIST scientists used the FORTRAN programs for this exercise, with the operating parameters being set to perform computations to 500 digits; the final results were truncated to 13 digits for display. The software is available from NETLIB: https://www.netlib.org/mpfun/. A nice discussion of multiple-precision (also called “arbitrary-precision”) programming is available on Wikipedia (8).

Conclusion

As the title of this article implies, as with other functions of this government agency the concern is with precision and accuracy in measurement. The contents of the NIST website relevant to our chemometric interests are currently very sparse, but the agency is amenable to expanding on and improving its offerings based on the needs of its users, so it’s up to us to let the agency know what we need and want. On the other hand the current offerings provide a framework that we can build on, and take us in directions we have not previously considered.

The NIST.gov website includes the following statement, scattered in various places on the website:

“We plan to update the collection with datasets for additional statistical methods as well as for the existing methods. We welcome your feedback on which statistical methods to provide datasets for, specific datasets to include, and other ways to improve the web service.”

Currently the offerings are sparse, but with that statement NIST has indicated a willingness, perhaps even an eagerness, to extend and expand the offerings in that section of the NIST.gov website. To achieve this, it is up to the chemometric community to coordinate with NIST to find out the “whats” and “hows” of information we need to provide for them, to accomplish this. Your authors have been in contact with NIST, in order to obtain information used in writing this column; this can serve as a starting point for further discussion.

References

1. Mark, H., Rubinovitz, R., Heaps, D., Gemperline, P., Dahm, D., Dahm, K. Appl. Spect. 2010, 64 (9), 995–1006.
2. Mark, H.; Workman, J., Jr. Classical Least Squares, Part XI: Comparison of Results from the Two Laboratories Continued, and Then the Light Dawns Spectroscopy 2012, 27 (10), 12–17.
3. What Standards Are and Why They Matter. https://bidenwhitehouse.archives.gov/wp-content/uploads/2023/05/US-Gov-National-Standards-Strategy-2023.pdf (accessed 2025-04-30).
4. Mark, H. An Overview of the American Society for Testing and Materials (ASTM), Part 1. NIR News 2009, 20 (5), 14–15
5. Mark, H. An Overview of the American Society for Testing and Materials (ASTM), Part 2. NIR News 2009, 20 (7), 22–23.
6. Mark, H.; Workman, J., Jr. Decimal Versus Binary Representation of Numbers in Computers Spectroscopy 2022, 37 (10), 13–20. DOI: 10.56530/spectroscopy.mm1179p4
7. Bailey, D. H. A Fortran 90-based multiprecision system. ACM Transactions on Mathematical Software (TOMS), 1995, 21 (4), 379–387. DOI: 10.1145/212066.212075
8. Arbitrary-Precision Arithmetic. Wikipedia. https://url.us.m.mimecastprotect.com/s/ERnICADx2xfJMoKlcGfkfGLjWR?domain=en.wikipedia.org

Howard Mark serves on the Editorial Advisory Board of Spectroscopy, and runs a consulting service, Mark Electronics, in Suffern, New York. Direct correspondence to: SpectroscopyEdit@mmhgroup.com ●

Jerome Workman, Jr. serves on the Editorial Advisory Board of Spectroscopy and is the Executive Editor for LCGC and Spectroscopy. He is the co-host of the Analytically Speaking podcast and has published multiple reference text volumes, including the three-volume Academic Press Handbook of Organic Compounds, the five-volume The Concise Handbook of Analytical Spectroscopy, the 2nd edition of Practical Guide and Spectral Atlas for Interpretive Near-Infrared Spectroscopy, the 2nd edition of Chemometrics in Spectroscopy, and the 4th edition of The Handbook of Near-Infrared Analysis.●