Use of signatures for systematic diagnostic comparison of time series from urban drainage models and data

Digital Twins (DT) are multipurpose, physically-based models of the urban drainage system that can be used by utilities for planning, design and operation. In order to gain trust in these DTs, comparisons with observations from sensors are needed (Pedersen et al., in prep. a). The utility company, VCS Denmark, has for several years made automated daily runs with a DT of their urban drainage system and compared observations with simulated values. However, this has until recently been a visual comparison including a few metrics, which has been a large undertaking given the +200 sensors that are distributed throughout the system. In the coming years, a digital platform is therefore being developed that will automate the process of systematically comparing DT output with sensor data and provide diagnostic evaluations of where and how they disagree.

Model performance is often characterised by average “distance measures” between two time series such as NSE, statistical Likelihood, and R2 (e.g. Bennett et al., 2013). These measures provide information about a model’s degree of “wrongness” but say nothing about why and how the model is wrong. To improve the diagnostic power of model evaluation Gupta et al. (2008) introduced the concept of hydrological “signatures”, which has since been widely used in the hydrological community. In this work, we introduce the concept of signatures to the urban drainage community and show how it can be the foundation of a systematic evaluation of DTs that can inform the user about what the DT can and cannot be used for.