Benchmarking Smart Meter Data Analytics

Xiufeng Liu, Lukasz Golab, Wojciech Golab, Ihab F. Ilyas

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

1600 Downloads (Pure)


Smart electricity meters have been replacing conventional meters worldwide, enabling automated collection of fine-grained (every 15 minutes or hourly) consumption data. A variety of smart meter analytics algorithms and applications have been proposed, mainly in the smart grid literature, but the focus thus far has been on what can be done with the data rather than how to do it efficiently. In this paper, we examine smart meter analytics from a software performance perspective. First, we propose a performance benchmark that includes common data analysis tasks on smart meter data. Second, since obtaining large amounts of smart meter data is difficult due to privacy issues, we present an algorithm for generating large realistic data sets from a small seed of real data. Third, we implement the proposed benchmark using five representative platforms: a traditional numeric computing platform (Matlab), a relational DBMS with a built-in machine learning toolkit (PostgreSQL/ MADLib), a main-memory column store (“System C”), and two distributed data processing platforms (Hive and Spark). We compare the five platforms in terms of application development effort and performance on a multi-core machine as well as a cluster of 16 commodity servers. We have made the proposed benchmark and data generator freely available online.
Original languageEnglish
Title of host publicationProceedings of the 18th International Conference on Extending Database Technology (EDBT)
Number of pages12
Publication date2015
ISBN (Electronic)978-3-89318-067-7
Publication statusPublished - 2015
Externally publishedYes
Event18th International Conference on Extending Database Technology - Brussels, Belgium
Duration: 23 Mar 201527 Aug 2017
Conference number: 18


Conference18th International Conference on Extending Database Technology

Bibliographical note

Distribution of this paper is permitted under the terms of the Creative Commons license CC-by-nc-nd 4.0


Dive into the research topics of 'Benchmarking Smart Meter Data Analytics'. Together they form a unique fingerprint.

Cite this