Verification of Occupants’ Behaviour Models in Residential Buildings

During the last decade, studies about stochastic models of occupants’ behaviour in relation to control of the indoor environment have been published. Often the overall aim of these models is to enable more reliable predictions of building performance using building performance simulations (BPS). However, the validity of these models has only been sparsely tested.
In this talk three methods for evaluating the models’ performance (listed below) will be described, discussed and exemplified using a dataset of window openings from Denmark.
1) Validation of state - TPR/FPR method
2) Validation of state transitions - Residuals method
3) Validation by simulation
The first two methods rely on a full dataset different to the one the models were derived from.
In the TPR/FPR method, the probability of an event is calculated using the model under evaluation. The probability is then compared to random numbers to determine if the event takes place or not. Finally, the simulated window position is compared to the measured ones and the True Positive Rate and False Positive Rate along with other metrics can be calculated and compared. The method evaluates the models abilities to predict the position of the window and the method works well if the model only relies on outdoor conditions. However, if the model under evaluation relies on variables that are affected by the window position (most indoor environmental variables), the method has inherent problems.
In the Residuals method the probabilities are compared directly to the measurements and no comparisons with random numbers are required. This has the benefit of avoiding feedback problems described above. The method evaluates the models abilities to predict the events rather than the position of the window. In the method, the model is used to calculate transition probabilities based on the dataset. In each time step, the probabilities are subtracted from the observed transitions, to find the residuals. Finally, the residuals can be averaged, and compared.
The validation by simulation relies on detailed Building Performance Simulations (BPS) using models under evaluation. In the method, different models of occupant behaviour are implemented in a BPS programme and detailed simulations are performed. The simulation results are compared to measurements in the simulated building to see which models best predicted the measurements. The method works under the assumption that any differences between measurements and simulation results are due to the occupant behaviour models’ inabilities to make correct predictions. This assumption is only true if all other sources of uncertainty have been ruled out by careful calibration of the BPS model.