Scalable Prediction-based Online Anomaly Detection for Smart Meter Data

Xiufeng Liu*, Per Sieverts Nielsen

*Corresponding author for this work

    Research output: Contribution to journalJournal articleResearchpeer-review

    Abstract

    Abstract Today smart meters are widely used in the energy sector to record energy consumption in real time. Large amounts of smart meter data have been accumulated and used for diverse analysis purposes. Anomaly detection raises the big data problem, namely the detection of abnormal events or unusual consumption behaviors. However, there is a lack of appropriate online systems that can handle anomaly detection for large-scale smart meter data effectively and efficiently. This paper proposes a lambda system for detecting anomalous consumption patterns, aiming at assisting decision makings for smart energy management. The proposed system uses a prediction-based detection method, combined with a novel lambda architecture for iterative model updates and real-time anomaly detection. This paper evaluates the system using a real-world data set and a large synthetic data set, and compares with three baselines. The results show that the proposed system has good scalability, and has a competitive advantage over others in anomaly detection.
    Original languageEnglish
    JournalInformation Systems
    Volume77
    Pages (from-to)34-47
    Number of pages14
    ISSN0306-4379
    DOIs
    Publication statusPublished - 2018

    Keywords

    • Anomaly detection
    • Lambda architecture
    • Real-time
    • Data mining
    • Scalability

    Cite this

    @article{fd5d3658b7de46d9944410adb19e438c,
    title = "Scalable Prediction-based Online Anomaly Detection for Smart Meter Data",
    abstract = "Abstract Today smart meters are widely used in the energy sector to record energy consumption in real time. Large amounts of smart meter data have been accumulated and used for diverse analysis purposes. Anomaly detection raises the big data problem, namely the detection of abnormal events or unusual consumption behaviors. However, there is a lack of appropriate online systems that can handle anomaly detection for large-scale smart meter data effectively and efficiently. This paper proposes a lambda system for detecting anomalous consumption patterns, aiming at assisting decision makings for smart energy management. The proposed system uses a prediction-based detection method, combined with a novel lambda architecture for iterative model updates and real-time anomaly detection. This paper evaluates the system using a real-world data set and a large synthetic data set, and compares with three baselines. The results show that the proposed system has good scalability, and has a competitive advantage over others in anomaly detection.",
    keywords = "Anomaly detection, Lambda architecture, Real-time, Data mining, Scalability",
    author = "Xiufeng Liu and Nielsen, {Per Sieverts}",
    year = "2018",
    doi = "10.1016/j.is.2018.05.007",
    language = "English",
    volume = "77",
    pages = "34--47",
    journal = "Information Systems",
    issn = "0306-4379",
    publisher = "Elsevier",

    }

    Scalable Prediction-based Online Anomaly Detection for Smart Meter Data. / Liu, Xiufeng; Nielsen, Per Sieverts.

    In: Information Systems, Vol. 77, 2018, p. 34-47.

    Research output: Contribution to journalJournal articleResearchpeer-review

    TY - JOUR

    T1 - Scalable Prediction-based Online Anomaly Detection for Smart Meter Data

    AU - Liu, Xiufeng

    AU - Nielsen, Per Sieverts

    PY - 2018

    Y1 - 2018

    N2 - Abstract Today smart meters are widely used in the energy sector to record energy consumption in real time. Large amounts of smart meter data have been accumulated and used for diverse analysis purposes. Anomaly detection raises the big data problem, namely the detection of abnormal events or unusual consumption behaviors. However, there is a lack of appropriate online systems that can handle anomaly detection for large-scale smart meter data effectively and efficiently. This paper proposes a lambda system for detecting anomalous consumption patterns, aiming at assisting decision makings for smart energy management. The proposed system uses a prediction-based detection method, combined with a novel lambda architecture for iterative model updates and real-time anomaly detection. This paper evaluates the system using a real-world data set and a large synthetic data set, and compares with three baselines. The results show that the proposed system has good scalability, and has a competitive advantage over others in anomaly detection.

    AB - Abstract Today smart meters are widely used in the energy sector to record energy consumption in real time. Large amounts of smart meter data have been accumulated and used for diverse analysis purposes. Anomaly detection raises the big data problem, namely the detection of abnormal events or unusual consumption behaviors. However, there is a lack of appropriate online systems that can handle anomaly detection for large-scale smart meter data effectively and efficiently. This paper proposes a lambda system for detecting anomalous consumption patterns, aiming at assisting decision makings for smart energy management. The proposed system uses a prediction-based detection method, combined with a novel lambda architecture for iterative model updates and real-time anomaly detection. This paper evaluates the system using a real-world data set and a large synthetic data set, and compares with three baselines. The results show that the proposed system has good scalability, and has a competitive advantage over others in anomaly detection.

    KW - Anomaly detection

    KW - Lambda architecture

    KW - Real-time

    KW - Data mining

    KW - Scalability

    U2 - 10.1016/j.is.2018.05.007

    DO - 10.1016/j.is.2018.05.007

    M3 - Journal article

    VL - 77

    SP - 34

    EP - 47

    JO - Information Systems

    JF - Information Systems

    SN - 0306-4379

    ER -