Evaluating the usability of open source frameworks in energy system modelling

S. Berendes; S. Hilpert; S. Günther; C. Muschner; S. Candas; K. Hainsch; J. van Ouwerkerk; S. Buchholz; M. Söthe

doi:10.1016/j.rser.2022.112174

Evaluating the usability of open source frameworks in energy system modelling

S. Berendes^*
, S. Hilpert
, S. Günther
, C. Muschner
, S. Candas
, K. Hainsch
, J. van Ouwerkerk
, S. Buchholz
, M. Söthe

^*Corresponding author for this work

Department of Technology, Management and Economics

University of Flensburg
Technical University of Munich
Technical University of Berlin
RWTH Aachen University
Reiner-Lemoine-Institut gGmbH

Research output: Contribution to journal › Journal article › Research › peer-review

158 Downloads (Orbit)

Abstract

Usability is considered one of the key factors in determining the success of Open Source Software, but is it sufficiently addressed within the development process as of yet? Thus far, there are no studies on record that explicitly examine the usability of Open Source Software in the field of Energy System Modelling. In this paper, we publish a novel method, the Energy System Modelling Usability Testing (ESMUT), including a step-by-step guide on how to apply the method and the corresponding usability questionnaire (ESMUQ). The questionnaire is based on the Post Study System Usability Questionnaire and adopted for quantitative usability testing of Open Source Energy System Modelling frameworks. To illustrate its usage and show its applicability, we apply the method in a case study with five frameworks (Balmorel, GENeSYS-MOD, GENESYS-2, oemof, and urbs) and within a group of eight framework developers. Based on the case study results, we find that the participants were largely satisfied in working with the frameworks analysed, and identify correct handling of input data and error messages as the most frequently mentioned problems when working with the frameworks. Consequently, we find that the usability of the frameworks analysed in the case study requires further improvement. Due to the fact that only the developer perspective was taken into account, and the number of participants involved in the study was limited, further research is required to assess the usability of Open Source Energy System Modelling frameworks.

Original language	English
Article number	112174
Journal	Renewable and Sustainable Energy Reviews
Volume	159
ISSN	1364-0321
DOIs	https://doi.org/10.1016/j.rser.2022.112174
Publication status	Published - 2022

Bibliographical note

Funding Information:
The research for this paper was performed within the open_MODEX project supported by the German Federal Ministry of Economic Affairs and Energy under grant number 03ET4076 . All figures and tables used in this manuscript were created by the authors themselves and are subject to the CC-BY 4.0 licence unless otherwise declared. You are free to share and adapt the figures and tables. The authors thank Ciara Dunks for proof reading, Stefanie Nguyen, Shaquille Henriques and Jonas Huber for small content and editorial contributions to the publication.

Publisher Copyright:
© 2022 The Authors

Keywords

Energy system modelling
Open Science
Open Source
Optimisation frameworks
PSSUQ
Software engineering
Software usability
Usability questionnaire
Usability testing method

Access to Document

10.1016/j.rser.2022.112174

1-s2.0-S1364032122001009-mainFinal published version, 1 MBLicence: CC BY

OpenUrl availability

Check availability in DTU Findit

Cite this

@article{1f771824441e491ab959b7f8352a3858,

title = "Evaluating the usability of open source frameworks in energy system modelling",

abstract = "Usability is considered one of the key factors in determining the success of Open Source Software, but is it sufficiently addressed within the development process as of yet? Thus far, there are no studies on record that explicitly examine the usability of Open Source Software in the field of Energy System Modelling. In this paper, we publish a novel method, the Energy System Modelling Usability Testing (ESMUT), including a step-by-step guide on how to apply the method and the corresponding usability questionnaire (ESMUQ). The questionnaire is based on the Post Study System Usability Questionnaire and adopted for quantitative usability testing of Open Source Energy System Modelling frameworks. To illustrate its usage and show its applicability, we apply the method in a case study with five frameworks (Balmorel, GENeSYS-MOD, GENESYS-2, oemof, and urbs) and within a group of eight framework developers. Based on the case study results, we find that the participants were largely satisfied in working with the frameworks analysed, and identify correct handling of input data and error messages as the most frequently mentioned problems when working with the frameworks. Consequently, we find that the usability of the frameworks analysed in the case study requires further improvement. Due to the fact that only the developer perspective was taken into account, and the number of participants involved in the study was limited, further research is required to assess the usability of Open Source Energy System Modelling frameworks.",

keywords = "Energy system modelling, Open Science, Open Source, Optimisation frameworks, PSSUQ, Software engineering, Software usability, Usability questionnaire, Usability testing method",

author = "S. Berendes and S. Hilpert and S. G{\"u}nther and C. Muschner and S. Candas and K. Hainsch and \{van Ouwerkerk\}, J. and S. Buchholz and M. S{\"o}the",

note = "Funding Information: The research for this paper was performed within the open\_MODEX project supported by the German Federal Ministry of Economic Affairs and Energy under grant number 03ET4076 . All figures and tables used in this manuscript were created by the authors themselves and are subject to the CC-BY 4.0 licence unless otherwise declared. You are free to share and adapt the figures and tables. The authors thank Ciara Dunks for proof reading, Stefanie Nguyen, Shaquille Henriques and Jonas Huber for small content and editorial contributions to the publication. Publisher Copyright: {\textcopyright} 2022 The Authors",

year = "2022",

doi = "10.1016/j.rser.2022.112174",

language = "English",

volume = "159",

journal = "Renewable and Sustainable Energy Reviews",

issn = "1364-0321",

publisher = "Elsevier",

}

TY - JOUR

T1 - Evaluating the usability of open source frameworks in energy system modelling

AU - Berendes, S.

AU - Hilpert, S.

AU - Günther, S.

AU - Muschner, C.

AU - Candas, S.

AU - Hainsch, K.

AU - van Ouwerkerk, J.

AU - Buchholz, S.

AU - Söthe, M.

N1 - Funding Information: The research for this paper was performed within the open_MODEX project supported by the German Federal Ministry of Economic Affairs and Energy under grant number 03ET4076 . All figures and tables used in this manuscript were created by the authors themselves and are subject to the CC-BY 4.0 licence unless otherwise declared. You are free to share and adapt the figures and tables. The authors thank Ciara Dunks for proof reading, Stefanie Nguyen, Shaquille Henriques and Jonas Huber for small content and editorial contributions to the publication. Publisher Copyright: © 2022 The Authors

PY - 2022

Y1 - 2022

N2 - Usability is considered one of the key factors in determining the success of Open Source Software, but is it sufficiently addressed within the development process as of yet? Thus far, there are no studies on record that explicitly examine the usability of Open Source Software in the field of Energy System Modelling. In this paper, we publish a novel method, the Energy System Modelling Usability Testing (ESMUT), including a step-by-step guide on how to apply the method and the corresponding usability questionnaire (ESMUQ). The questionnaire is based on the Post Study System Usability Questionnaire and adopted for quantitative usability testing of Open Source Energy System Modelling frameworks. To illustrate its usage and show its applicability, we apply the method in a case study with five frameworks (Balmorel, GENeSYS-MOD, GENESYS-2, oemof, and urbs) and within a group of eight framework developers. Based on the case study results, we find that the participants were largely satisfied in working with the frameworks analysed, and identify correct handling of input data and error messages as the most frequently mentioned problems when working with the frameworks. Consequently, we find that the usability of the frameworks analysed in the case study requires further improvement. Due to the fact that only the developer perspective was taken into account, and the number of participants involved in the study was limited, further research is required to assess the usability of Open Source Energy System Modelling frameworks.

AB - Usability is considered one of the key factors in determining the success of Open Source Software, but is it sufficiently addressed within the development process as of yet? Thus far, there are no studies on record that explicitly examine the usability of Open Source Software in the field of Energy System Modelling. In this paper, we publish a novel method, the Energy System Modelling Usability Testing (ESMUT), including a step-by-step guide on how to apply the method and the corresponding usability questionnaire (ESMUQ). The questionnaire is based on the Post Study System Usability Questionnaire and adopted for quantitative usability testing of Open Source Energy System Modelling frameworks. To illustrate its usage and show its applicability, we apply the method in a case study with five frameworks (Balmorel, GENeSYS-MOD, GENESYS-2, oemof, and urbs) and within a group of eight framework developers. Based on the case study results, we find that the participants were largely satisfied in working with the frameworks analysed, and identify correct handling of input data and error messages as the most frequently mentioned problems when working with the frameworks. Consequently, we find that the usability of the frameworks analysed in the case study requires further improvement. Due to the fact that only the developer perspective was taken into account, and the number of participants involved in the study was limited, further research is required to assess the usability of Open Source Energy System Modelling frameworks.

KW - Energy system modelling

KW - Open Science

KW - Open Source

KW - Optimisation frameworks

KW - PSSUQ

KW - Software engineering

KW - Software usability

KW - Usability questionnaire

KW - Usability testing method

U2 - 10.1016/j.rser.2022.112174

DO - 10.1016/j.rser.2022.112174

M3 - Journal article

AN - SCOPUS:85124616018

SN - 1364-0321

VL - 159

JO - Renewable and Sustainable Energy Reviews

JF - Renewable and Sustainable Energy Reviews

M1 - 112174

ER -

Evaluating the usability of open source frameworks in energy system modelling

Abstract

Bibliographical note

Keywords

Access to Document

OpenUrl availability

Fingerprint

Cite this