Deep Bayesian Modelling for Uncertainty Estimation in Transportation Systems

This thesis aims to develop and present methods to model the uncertainty in transportation systems and machine learning models. This is motivated by the complex and dynamic nature of the transportation system, making it ideal for machine learning applications. However, uncertainty can arise in the system from various sources, such as in data collection, the system’s inherent non-determinism or the models’ incomplete knowledge. These uncertainties are generally not accounted for when machine learning is used and applied in the context of transportation. To model and address these uncertainties, we adopt Bayesian concepts and deep learning methods to address the three following types of uncertainty: (i) censored data, (ii) aleatoric uncertainty, and (iii) epistemic uncertainty. While the research we presented in this thesis has a basis and motivation for transportation systems, the methodological contributions have broad applicability across various domains and research areas.

The first part of the thesis studies the emergence of censorship in mobility demand data. The available supply of a service is an upper limit for the demand that can be observed, which leads to censored data, implying that the true demand remains unobserved or latent. We introduce the Censored Quantile Regression Neural Network to model the true latent demand by providing a non-parametric estimation of the target distribution with the quantiles. The method is based on neural network architectures, which makes the model applicable to a wide range of data modalities. We show that it better fits the true demand for a service when compared to parametric and censorship-unaware models. Next, we study how censorship emerges in electric vehicle charging demand based on GPS traces, charging dynamics, and different queuing models. The charging data is likely censored and varies across space and time. To model the censored demand in space and time, we extend the Censored Quantile Regression Neural Network to temporal graph neural networks, which can model the demand data in space and time.

Subsequently, we delve into the always-on nature of the transportation systems where adaptive intelligent transportation systems continuously collect new data points to refine and retrain new models. Motivated by this challenge, twe study how much information new data points can provide to a machine learning model and how this information can guide the acquisition of new informative data points. However, estimating this information proves challenging with censored distributions, as it relies on the censorship status of new observations. We propose a novel modelling approach for estimating the information gained from new observations to address this challenge, and it combines it with active data collection to demonstrate its efficacy in improving uncertainty estimation by collecting informative points. Furthermore, we aim to extend the applicability of these findings beyond transportation by examining data from diverse fields such as healthcare and business churn data.

Then, we study how to model epistemic uncertainty in neural networks and deep learning models. This is challenging because modern deep learning models are overparameterised and highly non-linear. The state of the art is to sample different models to estimate the epistemic uncertainty. It is computationally expensive because it requires multiple forward passes or models trained in parallel. Research has focused on estimating these uncertainties with a single neural network. However, these methods are limited in assuming a Gaussian target distribution. we address this issue by proposing the Deep Evidential Quantile Regression model, which can model non-parametric target distributions while
estimating epistemic uncertainty. We study how the Deep Evidential Quantile Regression can estimate uncertainties for non-Gaussian distributions and scale to complex tasks such as monocular depth estimation.

Lastly, we discuss future research directions for uncertainty estimation in machine learning and transportation systems. The discussion contains concrete steps to continue the work presented in the thesis, including adaptive transportation systems and uncertainty estimation techniques. In conclusion, we present several findings and algorithms for modelling uncertainty in transportation systems and machine learning models. The contributions presented in this thesis extend beyond transportation applications and offer valuable insights into other fields dealing with censored data and uncertainty estimation.