Towards Automated NIR Spectroscopy

Jacob Søgaard Larsen

Research output: Book/ReportPh.D. thesis

142 Downloads (Pure)


Near infrared spectroscopic instruments are used in various industries for quality assessment and process control. Near infrared spectroscopic data are known to be a mixture of sample related chemical and physical effects combined with instrument and environment specific effects [Lod07]. This means that expert knowledge is required when deploying, maintaining and monitoring these instruments. Due to an increasing demand for utilizing these instruments, automation of these tasks are in high demand. This thesis focusses on three topics related to the automation of these tasks: Domain
learning, domain adaptation and monitoring. Near infrared spectroscopic data are assumed to obey Lambert Beer’s law. However, due to various scattering effects, this assumption is rarely satisfied for the raw measurements. Therefore, prior to fitting any model to this type of data, a careful choice of pre-processing strategy has to be made. Due to the high cost of chemical analyses, chemometric experts often work with small and fat data sets, limiting the choice of calibration method to linear models. Convolutional neural networks have been demonstrated to automatically learn appropriate pre-processing of near infrared spectroscopic data [CF18], but are limited by their demand of large amounts of data. Due to a lack of  standards with respect to the number of wavelengths and which wavelengths to collect data at, merging of multiple data sets is not straightforward. We propose a novel method called Weight Share, which enables learning of deep convolutional neural networks by co-training on multiple data sets with different input sizes. The method is shown to perform significantly better than various transfer learning strategies. Changes in production environment, recipes and raw materials all affect the measured spectrum. Effects that are either larger than previously or never observed before are mathematically known as covariate shifts. These are challenging for both linear and non-linear models necessitating the need for maintenance of calibration models in the form of domain adaptation. We proposed a novel method named the Extended Joint Trained framework for performing domain adaptation of linear models in a semi-supervised manner. Contrary to labelled data, unlabelled data are often available at low costs, and are therefore preferable as they provide information about
the current production domain. The method is demonstrated on both simulated and real data sets, and is shown to perform significantly better than other frameworks for semi-supervised domain adaptation. In order to know whether maintenance of a calibration model is necessary, a suitable monitoring strategy has to be implemented. We propose an online adaptation
of an offline model based method for detection of outliers in univariate time series data. Based on an ARMA model, the method uses a model of the one-step prediction errors after the impact of an exogenous disturbance in the form of a mean shift. The proposed method is shown to perform similar to state-of-the-art methods for the detecting of mean shifts, while being easier to tune.
Original languageEnglish
PublisherTechnical University of Denmark
Number of pages190
Publication statusPublished - 2020

Fingerprint Dive into the research topics of 'Towards Automated NIR Spectroscopy'. Together they form a unique fingerprint.

Cite this