Thomas Bocklitz

Raman spectroscopy is based on Raman scattering, an inelastic scattering process that is only observed in a tiny proportion (~1 in 10

) of the photons scattered by a molecule. As Raman spectroscopy typically measures the vibrational states, it can be used to characterize the molecular components of the sample, and hence, it provides a unique fingerprint of the sample. The technology is being applied in many scenarios in biology, forensics, diagnostics, pharmaceutics, and food science. Its power is further enhanced by spectral analysis methods, which can extract subtle changes from Raman spectra and translate the fingerprint information to high-level knowledge efficiently. The Raman spectral analysis is composed of three main parts: the experimental design; the preprocessing; and the data modeling (1). In this article, we would like to highlight the 11 key steps to analyze Raman data, which is depicted in Figure 1.

FIGURE 1: The 11 key steps of Raman spectral analysis, which are linked to the main steps: DoE, pre-processing, and modelling.

Sample size planning (SSP) is the most important step of experimental design (DoE) in the field of Raman spectral analysis. It aims to estimate the minimal number of samples to reach a statistically meaningful conclusion or to build a model with acceptable performance. The term sample refers to the highest level of the sampling hierarchy; for example, it could be a patient, a (biological) batch, or a (biological/technical) replicate. SSP can be achieved based on the learning curve that characterizes a predefined metric over the increasing sample size; look for the minimal sample size where the metric does not improve further (2).

Measured Raman spectra are not perfect, but are corrupted by different (side) effects originating from the instrument, the environment, and the sample itself. These corrupting effects are removed via a series of preprocessing steps, including quality control, spikes removal, (spectrometer) calibration, baseline correction, smoothing, and normalization. These steps are not independent, but they do impact each other. We recommend putting them into a closed loop and adjusting one step according to the feedback of the other steps.

Cosmic spikes occur when high-energy cosmic particles hit the charge coupled device (CCD) or complementary metal oxide semiconductor (CMOS) detector. They manifest themselves as narrow but extremely intense bands, and they can appear randomly at any wavenumber positions. In practice, spikes are normally detected as abnormal intensity changes when comparing two successively measured spectra or screening along the wavenumber axis within one spectrum. Methods are also reported to jointly inspect such intensity changes along the wavenumber axis and between the successive measurements simultaneously (3). The detected spikes are replaced with either an interpolation-based value using the boundary points of the spike position or the intensities of its successive measurement at the same wavenumber positions. In the latter way, the fluorescence and the intensity variations between the two measurements have to be handled well.

The spectrometer calibration is to remove spectral variations caused by the changes in devices and environment so that the spectra become comparable among different measurements. Typically, it involves a wavenumber and intensity calibration (4). Spectra of standard materials are measured in both cases. For wavenumber calibration, the calibration function is estimated by aligning the measured band positions of the wavenumber standard to their theoretical values—for instance, through a polynomial fitting. For the intensity calibration, the intensity response function can be calculated as the ratio between the measured and theoretical emission of an intensity standard. This response function is then used to divide the measured Raman intensities for the calibration.

Baseline correction normally refers to the correction of fluorescence of the sample. Fluorescence shows as a slowly changing baseline under Raman bands; this property can be used to correct it. Many mathematical approaches exist for this correction, including derivative spectra, the sensitive nonlinear iterative peak (SNIP) clipping, asymmetric least squares smoothing, polynomial fitting, standard normal variate (SNV), multiplicative scattering correction (MSC), and extended multiplicative signal correction (EMSC).

Smoothing is usually achieved via a moving-window low-pass filtering (for example, mean, median, or Gaussian) in the spectral or spatial domain. The spectral filtering works along the wavenumber axis; hence, it can be applied for a single spectrum. The spatial filtering, in contrast, is calculated over multiple spectra collected from a spatial neighborhood, and thus, it is only applicable in hyperspectral data cubes. In either manner, smoothing may degrade the subsequent analysis and is recommended only for highly noisy data.

Normalization is performed to suppress the fluctuation of the excitation intensity or changes in the focusing conditions of the setup. Here, the spectral intensities are divided by the area, maximum, or l

 norm of a selected spectral region (R), respectively; R, dependent on the task, can be the whole spectrum, a single wavenumber position, or a Raman band with supposedly constant intensities.

After the data is preprocessed, the spectral signals are translated into high-level information (such as response variables), for instance, the disease status or the concentration of a component. This requires a certain number of samples with known response variables. A statistical or machine learning model is constructed with part of the data set (training data) and evaluated with the rest (testing data). The model with optimal performance is afterward used for unknown samples to obtain their response variables directly. The major steps of statistical or data modeling are explained in the next section.

Dimension reduction helps to extract useful features from the data and reduce redundant information. Depending on whether the response information is needed or not, existent approaches can be categorized as supervised or unsupervised procedures. Typical examples are principal component analysis (PCA) and the partial least squares (PLS), respectively. Dimension reduction can also be grouped into feature selection or feature extraction. Methods for feature selection pick the variables that are the most significant according to a predefined benchmark, whereas feature extraction transforms the spectral data into a new space of lower dimension and represent the original spectra with new coordinates. Typical approaches in such case are PCA, vertex component analysis (VCA), N-Finder, independent component analysis (ICA), and multivariate curve resolution alternative least squares (MCR-ALS) (1).

The output of the dimension reduction is supplied to a machine learning model to translate the spectral signal into high-level information. Existent models can be categorized as unmixing, clustering, classification, or regression. The unmixing and clustering are applicable for image generation and rough data inspection—for example, to see the spatial pattern of a hyperspectral data cube. The classification relates the spectral signals to a categorical variable such as disease status. Regression models derive a quantitative value, such as the concentration of given substances, from the spectral signal. No clear separation exists between classification and regression models; as a matter of fact, classification is often achieved via a regression model.

After being constructed, the prediction performance of the model is evaluated with testing data. All supervised procedures contributing to the model prediction should be involved in such an evaluation, be it dimension reduction or the machine learning model itself. The prediction performance is characterized via the deviation of the prediction from the known true values. Examples of performance markers of the models are the root-mean squared-error of prediction (RMSEP) for regression models and accuracy, sensitivity, specificity, selectivity, and Cohen’s kappa for classification models (1). To make better use of the known samples, it is recommended to perform model evaluation in the framework of cross-validation, where the known data is split into training and testing data for multiple times. The predictions over all data splits are averaged to evaluate the performance of the model. In all splits, the testing data must be independent to the training data to avoid model overestimation. That means, the two data sets must be from different patients or batches (5).

Model interpretation is another important step in data modeling, which could be performed based on the variable significance indicated by the established model. Variables of the spectroscopic data that are shown to be important to the model should link to chemical or biological explanations. This can help further feature selection to build a more parsimonious model. In the long run, such a model interpretation can help to discover the biomarkers that play a role in the given investigation.

As sensitive as Raman spectroscopy, it is common to see significant shifts of Raman bands or changes in Raman intensities between spectra measured from different replicates or on different devices. Hence, an established model can perform badly when predicting new data—for instance, Raman spectra measured from a patient to be diagnosed. Model transfer is utilized in this case, which helps to improve the prediction of an existent model on new data. This can be achieved either by either removing the spectral variations between the different measurements or adjusting the model parameters by taking the features of the new data into account (6).

To summarize, we walked through the key steps of the Raman spectral analysis workflow and briefly discussed their aims and available approaches. Possible errors upon misconducting of the steps are summarized in (7). With such a condensed article, we hope to unveil the mystery of Raman spectral analysis so that the results of the analysis are more easily understood and interpreted. For more detailed and comprehensive conceptions of each step, the readers are kindly referred elsewhere (1,8).

This work is supported by the BMBF, funding program Photonics Research Germany (FKZ:13N15466 (BT1), FKZ:13N15706 (BT2) FKZ:13N15715 (BT4)) and is integrated into the Leibniz Center for Photonics in Infection Research (LPI). The LPI initiated by Leibniz-IPHT, Leibniz-HKI, UKJ and FSU Jena is part of the BMBF national road map for research infrastructures.

(1) Guo, S.; Popp, J.; Bocklitz, T. Chemometric Analysis in Raman Spectroscopy from Experimental Design to Machine Learning–based Modeling. 

 5426–5459. DOI: 10.1038/s41596-021-00620-3

(2) Ali, N.; Girnus, S.; Roesch, P.; Popp, J.; Bocklitz, T. W. Sample-Size Planning for Multivariate Data: A Raman-Spectroscopy-Based Example. 

 12485–12492. DOI: 10.1021/acs.analchem.8b02167

(3) Ryabchykov, O.; Bocklitz, T.; Ramoji, A.; Neugebauer, U.; Foerster, M.; Kroegel, C.; Bauer, M.; Kiehntopf, M.; Popp, Automatization of Spike Correction in Raman Spectra of Biological Samples. J. 

 1–6. DOI: 10.1016/j.chemolab.2016.03.024

(4) Bocklitz, T. W.; Dörfer, T.; Heinke, R.; Schmitt, M.; Popp, J. Spectrometer Calibration Protocol for Raman Spectra Recorded with Different Excitation Wavelengths. 

Spectrochim. Acta. A. Mol. Biomol. Spectrosc.

 544–549. DOI: 10.1016/j.saa.2015.04.079

(5) Guo, S.; Bocklitz, T.; Neugebauer, U.; Popp, J. Common Mistakes in Cross-validating Classification Models. 

 (30), 4410–4417. DOI: 10.1039/C7AY01363A

(6) Guo, S.; Kohler, A.; Zimmermann, B.; Heinke, R.; Stöckel, S.; Rösch, P.; Popp, J.: Bocklitz, T. Extended Multiplicative Signal Correction Based Model Transfer for Raman Spectroscopy in Biological Applications. 

 (16), 9787–9795. DOI: 10.1021/acs.analchem.8b01536

(7) Ryabchykov, O.; Schie, I.; Popp, J.; Bocklitz, T. Errors and Mistakes to Avoid when Analyzing Raman Spectra. 

(4), 48–50. DOI: 10.56530/spectroscopy.zz8373x6

(8) Guo, S.; Ryabchykov, O.; Ali, N.; Houhou, R.; T. Bocklitz, T, in 

Reference Module in Chemistry, Molecular Sciences, and Chemical Engineering; 

 are with the Leibniz Institute of Photonic Technology in Jena, Germany. 

 are also with the Friedrich Schiller University in Jena, Germany. 

 is also with the Institute of Computer Science, Faculty of Mathematics, Physics & Computer Science, at the University Bayreuth (UBT), in Bayreuth, Germany. Direct correspondence to: 

Articles

A more successful blueprint for analyzing Raman spectral data is outlined by following the 11 important steps, which are outlined here.

Key Steps in the Workflow to Analyze Raman Spectra

Seven common mistakes in the analysis of Raman spectra can lead to overestimating the performance of a model.

Raman spectroscopy is an inelastic scattering process on a quantified energy system where vibrational energy levels are typically measured. The technique provides an indirect measurement of vibrational energy levels within the sample and hence can characterize the biochemical composition of a sample. Normally, Raman spectroscopy is applied in a label-free manner, in contrast to using fluorescence labels, surface-enhanced Raman spectroscopy (SERS) labels, or Raman tags. The unlabeled Raman measurement provides a characteristic and phenotypical molecular fingerprint of the sample. In combination with artificial intelligence (AI) methods, such as chemometrics and machine learning, this fingerprint can be translated into high-level information. Because of the weakness of the Raman effect, AI-based data analysis pipelines are constructed, which aim at correcting, standardizing, and purifying the Raman data before the translating of high-level information can take place.

The data analysis pipeline for Raman spectra contains a sequence of algorithms to correct, standardize, and purify the measured Raman data. The data analysis pipeline starts with correcting cosmic spikes originating from high-energy cosmic particles or their secondary particles. Sometimes, other artifacts, such as fixed pattern noise, are removed as well. The next step deals with the wavelength, wavenumber, and intensity calibration. The first two calibration steps aim to generate a stable wavenumber axis with respect to the excitation wavelength. Generating a stable wavenumber is necessary because in Raman spectroscopy a relative measurement with respect to the excitation wavelength is performed. The intensity calibration aims at generat- ing setup-independent Raman spectra by correcting the spectral transfer function of optical components, such as lenses, filters, dispersive elements, and the quantum efficiency of the detector. After this step, multiple preprocessing methods are applied, which further standardize the data. This preprocessing block typically contains a baseline correction, because the Raman effect overlaps energetically with fluorescence. Thereafter, a denoising and a normalization step are applied to make spectra from different measurement days comparable. Denoising is not always applied, but the mixed Poisson–Gaussian noise needs to be considered in the selection of the algorithm. Finally, a feature extraction or dimension reduction (such as principal component analysis, or PCA) is applied. The selected features are further analyzed using classical machine learning or chemometric models. These last two steps can be combined with deep learning methods. Finally, these machine learning or deep learning models included in the whole data analysis pipeline are evaluated to judge the generalization performance of the models; that is, its performance on a new data set. Finally, the models are interpreted with visualization approaches.

The above described data analysis pipeline is described elsewhere (1) and sometimes there are changes in the ordering of the algorithms based on researchers’ idea of their data. Nevertheless, here we want to focus on specific problems or mistakes that should be avoided. These mistakes are shown in Figure 1 and these potential mistakes are explained in more detail below.

FIGURE 1: A visualization of the mistakes within the data analysis pipeline for Raman spectra. The mistakes are indicated in the arrow with respect to the location within the data analysis pipeline. The mistakes may arise from the sample size planning, which should determine enough independent observations. Thereafter, a wavenumber, wavelength, and intensity calibration are performed. The baseline correction is important, but the normalization should be done after the baseline correction. The complexity of the model is linked to the number of independent observations and the model should not be too complex. Thereafter, an unbiased estimation of the model generalizability is needed, which requires a cross-validation (or test validation) over independent measurements. Finally, the multiple testing problem needs to be corrected for and the correct statistical tests need to be applied.

Mistake #1: Absence of Independent Samples

The first critical issue is sample size planning, which is important because a sufficient quantity of independent data is needed to training and test data-driven AI models. It is advisable to measure at least three independent replicates in cell studies (five is preferable) and roughly 20–100 patients for diagnostic studies (2), which is especially important for the model evaluation, as discussed further in this tutorial (see mistake #6).

Generally, it is advisable to measure a wavenumber standard like 4-acetamidophenol with a high number of peaks in the wavenumber region of interest. These measurements can be used to construct a new wavenumber axis for each measurement day, which is subsequently interpolated to a common and fixed wavenumber axis (3). If this wavenumber (wavelength) calibration is not performed, systematic drifts in the measurement system overlap with possible sample related changes. An optimal spectrometer calibration is focus of ongoing research, but a white light measurement should be done every week or when the setup is modified. If these spectra (wavenumber standard or white light spectra) are not used for calibration, they can act as quality control measurements.

Mistake #3: Over-Optimized Preprocessing 

Because Raman spectra overlap with fluorescence and the fluorescence background can be 2–3 orders more intense than the Raman bands, a baseline correction or other approach, like performing the derivative, is important (4). Nevertheless, it is hard so select good parameters of the baseline correction and a grid search or optimization should be performed. To avoid overfitting, it is preferable to utilize spectral markers as the merit of such optimization rather than the performance of the model.

Mistake #4: Performing Spectral Normalization Before Background Correction 

Sometimes, it is seen that spectral normalization is performed before the background correction, resulting in a strong bias of the normalized Raman spectra. The fluorescence background and its intensity are coded within the normalization constant and hence every model might be biased. This issue should be avoided. Baseline correction needs to be done before normalization is performed.

Mistake #5: Selecting an Unsuitable Model

Depending on the number of independent measurements, the model and its complexity should be selected. For large independent data sets, modeling with a highly parameterized model such as a deep learning model can and should be applied. If the data set contains only a few independent observations, low-parameterized models should be used, such as linear models.

Another mistake that is often seen relates to the model evaluation (cross-validation or training validation). It is paramount that the different data sets within the validation be independent. To fulfill this requirement, biological replicates or patients, which act as independent measurements, need to be in either a training, validation, or test data subset. If this requirement is violated, the model and its performance is highly overestimated. In a previous study, we could show that a model with 60% classification accuracy, which was evaluated reliably (replicate-out cross-validation), was overestimated to nearly 100% classification accuracy with normal cross-validation (5). The main problem that can arise is an information leakage between the data sets that leads to a bias in the model performance estimation.

The last mistake that is often encountered is when a statistical analysis is performed on band intensities. If multiple Raman intensities are tested, then a Bonferroni correction that accounts for the alpha-error cumulation is needed. This issue occurs when 100 tests are performed, roughly five are positive by chance alone if an error probability of 95% is used. The second point is that a 

-test is normally not applicable because the assumptions are not meet. Therefore, nonparameterized alternative tests like the Mann-Whitney-Wilcoxon U test should be used.

These seven mistakes are often seen in publications about Raman spectroscopy and should be avoided, because they affect the final performance estimate of the model. These biased performance estimates need to be interpreted with caution, especially when Raman spectroscopy together with AI-based modeling is used in real world applications such as clinical diagnostics.

The financial support given from the European Union (EU), the Thüringer Ministerium für Wirtschaft, Wissenschaft und Digitale Gesellschaft, the Thüringer Aufbaubank, the Federal Ministry of Education and Research, Germany (BMBF), the German Science Foundation, the Fonds der Chemischen Industrie, and the Carl-Zeiss Foundation are greatly acknowledged.

(1) S. Guo, O. Ryabchykov, N. Ali, R. Houhou, and T. Bocklitz, in 

Reference Module in Chemistry, Molecular Sciences, and Chemical Engineering

 (Elsevier, Amsterdam, The Netherlands, 2020). https://doi.org/10.1016/B978-0-12-409547-2.14600-1.

(2) N. Ali, S. Girnus, P. Roesch, J. Popp, and T.W. Bocklitz, 

, 12485–12492 (2018). https://doi.org/10.1021/acs.analchem.8b02167.

(3) T.W. Bocklitz, T. Dörfer, R. Heinke, M. Schmitt, and J. Popp, 

, 544–549 (2015). https://doi.org/10.1016/j.saa.2015.04.079.

, 2396–2404 (2016). https://doi.org/10.1039/C6AN00041J.

(5) S. Guo, T. Bocklitz, U. Neugebauer, and J. Popp, 

 are with the Leibniz Institute of Photonic Technology in Jena, Germany, and with the Institute of Physical Chemistry and Abbe Center of Photonics at Friedrich-Schiller University in Jena, Germany. Direct correspondence to: 

Errors and Mistakes to Avoid when Analyzing Raman Spectra

	This article highlights the use of optical molecular spectroscopy in combination with artificial intelligence (AI) methods in the framework of process analytical technology (PAT). Specifically, we focus on the combination of Raman, infrared (IR), mid-IR, and near-infrared (NIR) spectroscopy and AI methods such as machine learning, deep learning, and chemometrics for PAT. The combination of these molecular spectroscopy methods in combination with tailor-made AI data analysis methods has the potential to revolutionize PAT in the upcoming years.	 

	Innovations in quality assurance and process control are of essential importance for the coming decades given the immensely growing world population, the development of demography, and the increasing demands of people on products and services. Quality assurance is subject to constantly increasing requirements and is essential for every individual as well as for companies for economic reasons. As a result of industrial processes one expects to see high safety, high and stable product quality, high yields, low consumption of resources, reduced influence of the variability of raw materials as well as long product durability. In recent years, new developments in the field of process analytics have been continuously observed.

	The quality by design (QbD) concept plays a decisive role in this development (1). This concept is based on the fact that a high-quality level of end products cannot be achieved by testing the products alone, but by an intelligent design of the entire manufacturing process into which the products are incorporated. This approach requires a comprehensive understanding of the entire manufacturing process. If one is informed about all relevant characteristics along a production process, one gets a better picture of the quality of a product or intermediate product than by certifying the final product- which is still generally accepted today as a measure of quality. The technology that makes QbD possible is process analytical technology (PAT). PAT is an approach for the improved control of production processes. Although originally promoted by the U.S. Food and Drug Administration (FDA) for use in the pharmaceutical sector, the PAT initiative is also gaining in importance for related industries, such as the food industry. If PAT is moved further forward along the entire production process, deviations from specified guideline values and possible risks in production and manufacturing processes can be detected and eliminated immediately. 

	PAT distinguishes between off-line, at-line, on-line and in-line measurement methods (1). In off-line measurements, samples are taken from the process and analyzed in a laboratory environment that is clearly separated from the industrial plants. Off-line analysis therefore always shows significant delay times between the detection of irregularities and the counteraction. In at-line measurements, the sample is taken from the process stream and analyzed with analytical equipment located in the immediate vicinity of the industrial plants. This process significantly reduces the reaction time for countermeasures. In on-line measurements, the samples are not completely removed from the process stream, but are temporarily separated, such as via a bypass system that transports the sample directly through the on-line measuring device, where the sample is analyzed in the immediate vicinity of the industrial processing and then recombined with the process stream. When using in-line instruments, the sensor is immersed directly into the process stream and remains in direct contact with the unaltered material stream. Increasingly, the need for in-line process analytics is growing and with it the availability of appropriate measurement technologies. From the large variety of analytical methods, optical molecular spectroscopy is one of the most promising approaches, which can best be integrated in-situ within an in-line process control.	 

	In order to be able to intervene quickly in the event of changes in the process that lead to a reduction in the quality of the end product, powerful in-situ analytical approaches are required. In this context, optical spectroscopic methods in particular are becoming increasingly important for in-line process monitoring. Optical spectroscopic methods are usually nondestructive and provide continuous information that can be used for process control. Therefore, optical spectroscopic methods are becoming more and more important in PAT. Depending on the measurement problem, a wide range of wavelength ranges and phenomena of light-matter interaction can be used, such as:	

ultraviolet-visible (UV-vis) spectroscopy (electronic excitation)

fluorescence spectroscopy (absorption and emission of electrons)

vibrational spectroscopy (NIR or mid-IR absorption and Raman spectroscopy)

laser induced breakdown spectroscopy (LIBS).

	All these spectroscopic techniques differ in molecular selectivity, sensitivity, readiness for use in PAT as a result of special sampling conditions, and so on. Vibrational spectroscopic methods are characterized by a particularly high molecular selectivity, because they provide direct information about characteristic molecular vibrations and thus about the molecular structure. A vibrational spectrum can therefore be seen as a kind of characteristic “molecular fingerprint” of a molecular species, which enables the identification of organic, inorganic, or biological components. Absorption in the mid-IR range excites fundamental vibrations that can be directly assigned to the basic building blocks in a molecule. NIR spectroscopy, on the other hand, measures vibrational overtones and combination bands and is therefore less selective (no easy direct interpretability) and also less sensitive as compared to mid-IR absorption spectroscopy. Despite these disadvantages, the main advantage of NIR spectroscopy compared to mid-IR spectroscopy is that no sample preparation (such as dilution) is required even at higher concentrations. It is important to emphasize that both NIR and mid-IR spectroscopy are very sensitive to water absorption, which limits a broad applicability of these methods in aqueous environments, such as for the investigation of fermentation processes. For the analysis of aqueous systems Raman spectroscopy can be advantageous compared to NIR and mid-IR spectroscopy. In recent years the potential of Raman spectroscopic methods for PAT has been increasingly recognized (1–3). The advantages of Raman spectroscopy are an unprecedented high specificity and a variety of applications. Raman spectroscopy is a noninvasive technique where no special sample preparation is required. Solid, liquid, and gaseous samples as well as transparent and nontransparent samples or samples with inhomogeneous surface properties can be measured. Limitations to the applicability of Raman spectroscopy result from the possible occurrence of an interfering fluorescence background and the low signal yields in comparison with, for example, mid-IR or NIR absorption or fluorescence spectroscopy. In recent years, however, decisive technical progress has been made to overcome these limitations, in particular through the realization of highly efficient spectrographs and the implementation of novel detector technologies. These advances have led to a significant reduction in acquisition times down to the ms range (4,5).	 

	In recent years, new developments in the field of PAT have been continuously observed using optical spectroscopic methods. Especially in the field of chemical and pharmaceutical analysis, optical and spectroscopic methods are used in on-line process monitoring (2,3,6). It can be stated that especially the interest in NIR and Raman spectroscopy-given their potential for fast and nondestructive analysis of samples in different physical states (liquids, solids, pastes, and so on) and their capability to be used on-line or in-line-as potential PAT analysis methods has increased significantly. Although NIR spectroscopy is still predominant in the pharmaceutical industry, Raman spectroscopy is becoming more widespread and has already proven its feasibility for the analysis of critical quality characteristics throughout the drug manufacturing process (2,3). 

	Worldwide, the demand for high-quality and safety in the food supply are also constantly increasing. Thus, a safe and high-quality food supply can only be guaranteed if new possibilities in analytics and holistic process control are generated. However, food analysis presents PAT with extreme challenges. In comparison to the chemical and pharmaceutical industries, in the field of the food industry, the sample compositions are much more complex. They are mixtures of different substance classes. 

	Furthermore, the materials are often soft, variable in size, brittle, or slippery. They are extremely susceptible to changes in pressure, temperature, and humidity, making noninvasive, fast methods for process analysis essential. In recent years, technological developments have already been made that make it possible to replace laboratory-based methods with on-line process control. These are mainly based on fluorescence or IR or NIR spectroscopy (7–9). Furthermore, the potential of Raman spectroscopic methods has been increasingly recognized. This potential has been successfully demonstrated using the example of determining the content of glucose, fructose, and sucrose in commercially available soft drinks (10). In the field of drinking water monitoring, Raman-based methods for the in-line detection of smallest amounts of chemicals and pharmaceutical impurities were applied (11).  

	Furthermore, surface-enhanced Raman spectroscopy (SERS) was applied for the detection of prohibited dyes as well as azo dyes in food (12,13). These representative examples show that optical spectroscopic methods and in particular Raman spectroscopy have enormous potential for on-line process analysis in food production.

	To extend the analytical potential with respect to selectivity, sensitivity, and so on, it can be very advantageous to combine several of the above mentioned optical spectroscopic contrast mechanisms in a multimodal approach (14). From a practical point of view, besides the need for high sensitivity and selectivity, the price-to-performance ratio and the simplicity of application of optical spectroscopic methods are of particular importance. The availability of compact, high-performance spectrometers including detectors as well as easy-to-operate compact light sources-and especially lasers with acceptable operating times-play a decisive role. 

	An essential feature of optical process spectrometers is the coupling of probes or the use of flow cells. Using fiber-optic coupled probes, it is possible to spatially separate spectrometer and application in the process and thus to look directly into individual processes. Fiber-optical sensors help to record data at critical points. As the “sensory organs of process control technology,” they enable quality-optimized automation of processes. In this context, particularly great progress has been made in recent years in the development of high-performance, tailor-made Raman probe concepts, mainly for biomedical applications (15–17), which are also of high relevance for in-line process analysis. In summary, technological developments have been made in the last few years in the areas of light sources, spectrometers, detectors and fiber probes, which will make it possible to replace laboratory-based methods with on-line process control in the future.

	Essential challenges for a continuous quality and safety monitoring of the processes as well as the resulting products and materials are seen in the still missing automation of process analytical techniques and here especially in the automated analysis of the recorded spectral measurement data. This automated analysis of the spectral data is accomplished by data analysis pipelines, which combine standardization and correction procedures with chemometric techniques and machine learning models (18,19). The overall aim of these data pipelines is the translation of the optical or spectral data into meaningful higher information (see Figure 1). In PAT this higher information might by the concentrations of given substances, the quantification of the sample composition or the prediction of a sample property like “good”or “bad.”	 

	 To perform this translation of the experimentally recorded spectral data, standardization and correction procedures are utilized first. For Raman spectroscopy a de-spiking (20), a wavenumber, and spectrometer calibration (21), a background correction (22) and a normalization are performed. These sequential steps correct for corrupting effects and standardize the Raman data, while similar preprocessing steps are done for other optical technologies like IR or NIR spectroscopy. After the data is standardized, machine learning models such as deep learning (23) or chemometric models (19) are utilized for the translation of the preprocessed data into meaningful information. As mentioned above, this higher information might be concentration estimates of substances or compounds of the sample or prediction of sample attributes like “good” and “bad.” Depending on the choice of higher lever information, either regression or classification models are utilized. Most often for spectral measurements, chemometric techniques are utilized whereas for image-based measurements, deep learning methods, like convolutional neuronal networks, are chosen for the analysis. All of these models are optimized based on training data and afterward the models are evaluated for their generalization performance (24). Thereafter, the models can be used for the prediction of unknown test data (with the estimated generalization error) and the prediction can be utilized for process analytics in chemical, pharmaceutical, and food industry.	 

	This contribution provides a short overview on the use of optical molecular spectroscopy such as Raman, IR, mid-IR, NIR spectroscopy as process analytical technology (PAT). An essential step to successfully apply these measurement methods as PAT is their combination with artificial intelligence methods like machine learning, deep learning, and chemometrics in order to translate the spectroscopic information into meaningful higher information such as quantification of the sample composition or the prediction of a sample property. This combination is done by tailor-made data pipelines and is briefly explained for the example of Raman spectroscopy. Overall, this contribution highlights the great potential of optical molecular spectroscopy in combination with artificial intelligence for PAT.	 

	Financial support of the EU, the ”Thüringer Ministerium für Wirtschaft, Wissenschaft und Digitale Gesellschaft,” the ”Thüringer Aufbaubank”, the Federal Ministry of Education and Research, Germany (BMBF), the German Science Foundation, the Fonds der Chemischen Industrie, and the Carl-Zeiss Foundation are greatly acknowledged.	 

Kandelbauer, M. Ruhe, and R. W. Kessler, in J. Popp, V.V. Tuchin, A. Chiou, and S.H. Heinemann, Eds., 

Handbook of Biophotonics Vol. 3: Photonics in Pharmaceutics, Bioanalysis and Environmental Research (

Wiley-VCH Verlag GmbH & Co. KGaA, Weinheim, Germany, 2011), pp. 3–70.

K.A. Esmonde-White, M. Cuellar, C. Uepmann, B. Lenain, and I.R. Lewis, 

Nagy, A. Farkas, E. Borbás, P. Vass, Z.K. Nagy, and G. Marosi, 

Krafft, I.W. Schie, T. Meyer, M. Schmitt, and J. Popp, 

Schie, J. Rüger, A.S. Mondol, A. Ramoji, U. Neugebauer, C. Krafft, and J. Popp, 

Process Analytical Technology: Spectroscopic Tools and Implementation Strategies for the Chemical and Pharmaceutical Industries 

(Wiley, Hoboken, New Jersey, 2nd Ed., 2010).

Niemöller and D. Behmer, in J. Irudayaraj and C. Reh, Eds., 

 (Wiley Blackwell, Ames, Iowa, 2008), Chapter 4.

Hitzmann, R. Hauselmann, A. Niemoeller, D. Sangi, J. Traenkle, and J. Glassey, 

K. Ilaslana, I. Hakki Boyaci, and A. Top, 

Z. Li, M.J. Deen, S. Kumar, and P.R. Selvaganapathy, 

S. Hea, W. Xiea, W. Zhanga, L. Zhang, Y. Wang, X. Liu, Y. Liu, and C. Du, 

Spectrochim Acta A Mol. Biomol Spectrosc. 

V. Peksa, M. Jahn, L. Štolcová, V. Schulz, J. Proška, M. Procházka, K. Weber, D. Cialla-May, and J. Popp, 

T. Meyer, M. Schmitt, B. Dietzek, and J. Popp, 

Krafft, S. Dochow, I. Latka, B. Dietzek, and J. Popp, 

Latka, S. Dochow, C. Krafft, B. Dietzek, and J. Popp, 

Cordero, I. Latka, C. Matthäus, I. W. Schie, and J. Popp, 

Ryabchykov, S. Guo, and T. Bocklitz, in J. Popp, Ed., 

Micro-Raman Spectroscopy Theory and Application 

S. Guo, O. Ryabchykov, N. Ali, R. Houhou, and T. Bocklitz, in 

Comprehensive Chemometrics Molecular Sciences and Chemical Engineering 

Ryabchykov, T. Bocklitz, A. Ramoji, U. Neugebauer, M. Foester, C. Kroegel, M. Bauer, M. Kiehntopf, and J. Popp, 

T. Bocklitz, T. Dörfer, R. Heinke, M. Schmitt, and J. Popp, 

Pradhan, S. Guo, O. Ryabchykov, J. Popp, and T. W. Bocklitz, 

Guo, T. Bocklitz, U. Neugebauer, and J. Popp, 

 is the head of the Research Department of Photonic Data Science, at the Leibniz Institute of Photonic Technology, in Jena, Germany. 

 is an assistant professor at the Institute of Physical Chemistry and Abbe Center of Photonics at Friedrich Schiller University Jena, Germany. 	 

 is the scientific director of the Leibniz Institute of Photonic Technology, Jena, Germany, and also holds the chair for Physical Chemsitry at the Friedrich Schiller University Jena, Germany. Direct correspondence to: 

In celebration of Spectroscopy’s 35th Anniversary, leading experts discuss important issues and challenges in analytical spectroscopy.

Thomas Bocklitz

Articles

Key Steps in the Workflow to Analyze Raman Spectra

Errors and Mistakes to Avoid when Analyzing Raman Spectra

Optical Molecular Spectroscopy in Combination with Artificial Intelligence for Process Analytical Technology