Data Pre-processing Techniques in the Regional Emission's Load Profiles Case

Different data sources, data types and platforms are involved in modelling emissions load profiles. In our case, we model emission load profiles at the regional or city level. However, we found missing values, redundancy and inconsistency in the datasets, and in most cases data preprocessing is unavoidable. Data pre-processing converts the data into a clean and tidy dataset for the subsequent modelling steps or statistical analyses. Therefore, some common techniques for data pre-processing such as cleaning, transformation, integration, reduction and some terms in data mining such as filtering and selection have been applied in our case study. We usually do the data pre-processing of moderate problems and a small amount of data using a spreadsheet application, whereas we use the programming language to do the more complex and big data size tasks. As a result, it has been found that understanding the nature of our data collection, the data flow process and the desired output comprehensively is the key for efficiency in data pre-processing. The applied techniques have helped us to provide the proper input for modelling the regional emission load profile efficiently.