A hybrid ICT-solution for smart meter data analytics

Research output: Contribution to journalJournal articleResearchpeer-review

175 Downloads (Pure)

Abstract

Smart meters are increasingly used worldwide. Smart meters are the advanced meters capable of measuring energy consumption at a fine-grained time interval, e.g., every 15 min. Smart meter data are typically bundled with social economic data in analytics, such as meter geographic locations, weather conditions and user information, which makes the data sets very sizable and the analytics complex. Data mining and emerging cloud computing technologies make collecting, processing, and analyzing the so-called big data possible. This paper proposes an innovative ICT-solution to streamline smart meter data analytics. The proposed solution offers an information integration pipeline for ingesting data from smart meters, a scalable platform for processing and mining big data sets, and a web portal for visualizing analytics results. The implemented system has a hybrid architecture of using Spark or Hive for big data processing, and using the machine learning toolkit, MADlib, for doing in-database data analytics in PostgreSQL database. This paper evaluates the key technologies of the proposed ICT-solution, and the results show the effectiveness and efficiency of using the system for both batch and online analytics.
Original languageEnglish
JournalEnergy
Volume115
Issue numberPart 3
Pages (from-to)1710–1722
Number of pages13
ISSN0360-5442
DOIs
Publication statusPublished - 2016

Keywords

  • ICT-solution
  • Smart meter data
  • Big data
  • Data analytics

Cite this

@article{0b1985f3f2ab42278bc950d0e727806c,
title = "A hybrid ICT-solution for smart meter data analytics",
abstract = "Smart meters are increasingly used worldwide. Smart meters are the advanced meters capable of measuring energy consumption at a fine-grained time interval, e.g., every 15 min. Smart meter data are typically bundled with social economic data in analytics, such as meter geographic locations, weather conditions and user information, which makes the data sets very sizable and the analytics complex. Data mining and emerging cloud computing technologies make collecting, processing, and analyzing the so-called big data possible. This paper proposes an innovative ICT-solution to streamline smart meter data analytics. The proposed solution offers an information integration pipeline for ingesting data from smart meters, a scalable platform for processing and mining big data sets, and a web portal for visualizing analytics results. The implemented system has a hybrid architecture of using Spark or Hive for big data processing, and using the machine learning toolkit, MADlib, for doing in-database data analytics in PostgreSQL database. This paper evaluates the key technologies of the proposed ICT-solution, and the results show the effectiveness and efficiency of using the system for both batch and online analytics.",
keywords = "ICT-solution, Smart meter data, Big data, Data analytics",
author = "Xiufeng Liu and Nielsen, {Per Sieverts}",
year = "2016",
doi = "10.1016/j.energy.2016.05.068",
language = "English",
volume = "115",
pages = "1710–1722",
journal = "Energy",
issn = "0360-5442",
publisher = "Elsevier",
number = "Part 3",

}

A hybrid ICT-solution for smart meter data analytics. / Liu, Xiufeng; Nielsen, Per Sieverts.

In: Energy, Vol. 115, No. Part 3, 2016, p. 1710–1722.

Research output: Contribution to journalJournal articleResearchpeer-review

TY - JOUR

T1 - A hybrid ICT-solution for smart meter data analytics

AU - Liu, Xiufeng

AU - Nielsen, Per Sieverts

PY - 2016

Y1 - 2016

N2 - Smart meters are increasingly used worldwide. Smart meters are the advanced meters capable of measuring energy consumption at a fine-grained time interval, e.g., every 15 min. Smart meter data are typically bundled with social economic data in analytics, such as meter geographic locations, weather conditions and user information, which makes the data sets very sizable and the analytics complex. Data mining and emerging cloud computing technologies make collecting, processing, and analyzing the so-called big data possible. This paper proposes an innovative ICT-solution to streamline smart meter data analytics. The proposed solution offers an information integration pipeline for ingesting data from smart meters, a scalable platform for processing and mining big data sets, and a web portal for visualizing analytics results. The implemented system has a hybrid architecture of using Spark or Hive for big data processing, and using the machine learning toolkit, MADlib, for doing in-database data analytics in PostgreSQL database. This paper evaluates the key technologies of the proposed ICT-solution, and the results show the effectiveness and efficiency of using the system for both batch and online analytics.

AB - Smart meters are increasingly used worldwide. Smart meters are the advanced meters capable of measuring energy consumption at a fine-grained time interval, e.g., every 15 min. Smart meter data are typically bundled with social economic data in analytics, such as meter geographic locations, weather conditions and user information, which makes the data sets very sizable and the analytics complex. Data mining and emerging cloud computing technologies make collecting, processing, and analyzing the so-called big data possible. This paper proposes an innovative ICT-solution to streamline smart meter data analytics. The proposed solution offers an information integration pipeline for ingesting data from smart meters, a scalable platform for processing and mining big data sets, and a web portal for visualizing analytics results. The implemented system has a hybrid architecture of using Spark or Hive for big data processing, and using the machine learning toolkit, MADlib, for doing in-database data analytics in PostgreSQL database. This paper evaluates the key technologies of the proposed ICT-solution, and the results show the effectiveness and efficiency of using the system for both batch and online analytics.

KW - ICT-solution

KW - Smart meter data

KW - Big data

KW - Data analytics

U2 - 10.1016/j.energy.2016.05.068

DO - 10.1016/j.energy.2016.05.068

M3 - Journal article

VL - 115

SP - 1710

EP - 1722

JO - Energy

JF - Energy

SN - 0360-5442

IS - Part 3

ER -