Data Pre-processing Techniques in the Regional Emission's Load Profiles Case

Angreine Kewo, Pinrolinvic Manembu, Per Sieverts Nielsen

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

Abstract

Different data sources, data types and platforms are involved in modelling emissions load profiles. In our case, we model emission load profiles at the regional or city level. However, we found missing values, redundancy and inconsistency in the datasets, and in most cases data preprocessing is unavoidable. Data pre-processing converts the data into a clean and tidy dataset for the subsequent modelling steps or statistical analyses. Therefore, some common techniques for data pre-processing such as cleaning, transformation, integration, reduction and some terms in data mining such as filtering and selection have been applied in our case study. We usually do the data pre-processing of moderate problems and a small amount of data using a spreadsheet application, whereas we use the programming language to do the more complex and big data size tasks. As a result, it has been found that understanding the nature of our data collection, the data flow process and the desired output comprehensively is the key for efficiency in data pre-processing. The applied techniques have helped us to provide the proper input for modelling the regional emission load profile efficiently.
Original languageEnglish
Title of host publication 2019 6th International Conference on Control, Decision and Information Technologies (CoDIT)
Publication date2019
Pages2016-2021
DOIs
Publication statusPublished - 2019
Event6th International Conference on Control, Decision and
Information Technologies
- Paris, France
Duration: 23 Apr 201926 Apr 2019

Conference

Conference6th International Conference on Control, Decision and
Information Technologies
CountryFrance
CityParis
Period23/04/201926/04/2019
Series2019 6th International Conference on Control, Decision and Information Technologies (codit)

Cite this

Kewo, A., Manembu, P., & Nielsen, P. S. (2019). Data Pre-processing Techniques in the Regional Emission's Load Profiles Case. In 2019 6th International Conference on Control, Decision and Information Technologies (CoDIT) (pp. 2016-2021). 2019 6th International Conference on Control, Decision and Information Technologies (codit) https://doi.org/10.1109/CoDIT.2019.8820303
Kewo, Angreine ; Manembu, Pinrolinvic ; Nielsen, Per Sieverts. / Data Pre-processing Techniques in the Regional Emission's Load Profiles Case. 2019 6th International Conference on Control, Decision and Information Technologies (CoDIT). 2019. pp. 2016-2021 (2019 6th International Conference on Control, Decision and Information Technologies (codit)).
@inproceedings{2a01776842f54b20af7a00fe24c163fa,
title = "Data Pre-processing Techniques in the Regional Emission's Load Profiles Case",
abstract = "Different data sources, data types and platforms are involved in modelling emissions load profiles. In our case, we model emission load profiles at the regional or city level. However, we found missing values, redundancy and inconsistency in the datasets, and in most cases data preprocessing is unavoidable. Data pre-processing converts the data into a clean and tidy dataset for the subsequent modelling steps or statistical analyses. Therefore, some common techniques for data pre-processing such as cleaning, transformation, integration, reduction and some terms in data mining such as filtering and selection have been applied in our case study. We usually do the data pre-processing of moderate problems and a small amount of data using a spreadsheet application, whereas we use the programming language to do the more complex and big data size tasks. As a result, it has been found that understanding the nature of our data collection, the data flow process and the desired output comprehensively is the key for efficiency in data pre-processing. The applied techniques have helped us to provide the proper input for modelling the regional emission load profile efficiently.",
author = "Angreine Kewo and Pinrolinvic Manembu and Nielsen, {Per Sieverts}",
year = "2019",
doi = "10.1109/CoDIT.2019.8820303",
language = "English",
pages = "2016--2021",
booktitle = "2019 6th International Conference on Control, Decision and Information Technologies (CoDIT)",

}

Kewo, A, Manembu, P & Nielsen, PS 2019, Data Pre-processing Techniques in the Regional Emission's Load Profiles Case. in 2019 6th International Conference on Control, Decision and Information Technologies (CoDIT). 2019 6th International Conference on Control, Decision and Information Technologies (codit), pp. 2016-2021, 6th International Conference on Control, Decision and
Information Technologies, Paris, France, 23/04/2019. https://doi.org/10.1109/CoDIT.2019.8820303

Data Pre-processing Techniques in the Regional Emission's Load Profiles Case. / Kewo, Angreine; Manembu, Pinrolinvic; Nielsen, Per Sieverts.

2019 6th International Conference on Control, Decision and Information Technologies (CoDIT). 2019. p. 2016-2021 (2019 6th International Conference on Control, Decision and Information Technologies (codit)).

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

TY - GEN

T1 - Data Pre-processing Techniques in the Regional Emission's Load Profiles Case

AU - Kewo, Angreine

AU - Manembu, Pinrolinvic

AU - Nielsen, Per Sieverts

PY - 2019

Y1 - 2019

N2 - Different data sources, data types and platforms are involved in modelling emissions load profiles. In our case, we model emission load profiles at the regional or city level. However, we found missing values, redundancy and inconsistency in the datasets, and in most cases data preprocessing is unavoidable. Data pre-processing converts the data into a clean and tidy dataset for the subsequent modelling steps or statistical analyses. Therefore, some common techniques for data pre-processing such as cleaning, transformation, integration, reduction and some terms in data mining such as filtering and selection have been applied in our case study. We usually do the data pre-processing of moderate problems and a small amount of data using a spreadsheet application, whereas we use the programming language to do the more complex and big data size tasks. As a result, it has been found that understanding the nature of our data collection, the data flow process and the desired output comprehensively is the key for efficiency in data pre-processing. The applied techniques have helped us to provide the proper input for modelling the regional emission load profile efficiently.

AB - Different data sources, data types and platforms are involved in modelling emissions load profiles. In our case, we model emission load profiles at the regional or city level. However, we found missing values, redundancy and inconsistency in the datasets, and in most cases data preprocessing is unavoidable. Data pre-processing converts the data into a clean and tidy dataset for the subsequent modelling steps or statistical analyses. Therefore, some common techniques for data pre-processing such as cleaning, transformation, integration, reduction and some terms in data mining such as filtering and selection have been applied in our case study. We usually do the data pre-processing of moderate problems and a small amount of data using a spreadsheet application, whereas we use the programming language to do the more complex and big data size tasks. As a result, it has been found that understanding the nature of our data collection, the data flow process and the desired output comprehensively is the key for efficiency in data pre-processing. The applied techniques have helped us to provide the proper input for modelling the regional emission load profile efficiently.

U2 - 10.1109/CoDIT.2019.8820303

DO - 10.1109/CoDIT.2019.8820303

M3 - Article in proceedings

SP - 2016

EP - 2021

BT - 2019 6th International Conference on Control, Decision and Information Technologies (CoDIT)

ER -

Kewo A, Manembu P, Nielsen PS. Data Pre-processing Techniques in the Regional Emission's Load Profiles Case. In 2019 6th International Conference on Control, Decision and Information Technologies (CoDIT). 2019. p. 2016-2021. (2019 6th International Conference on Control, Decision and Information Technologies (codit)). https://doi.org/10.1109/CoDIT.2019.8820303