Active learning metamodeling for policy analysis: Application to an emergency medical service simulator

Francisco Antunes, M. Amorim, Francisco Camara Pereira, Bernardete Ribeiro

Research output: Contribution to journalJournal articleResearchpeer-review

Abstract

Simulation approaches constitute a well-established tool to model, understand, and predict the behavior of transportation systems, and ultimately to assess the performance of transportation policies. Due to their multidimensionality nature, such systems are not often approachable through conventional analytic methods, making simulation modeling the only reliable tool of study. Nevertheless, simulation models can turn out to be computationally expensive when embedded with enough detail. An immediate answer to this shortcoming is the use of simulation metamodels that are designed to approximate the simulators’ results. In this work, the authors propose a metamodeling approach based on active learning that seeks to improve the exploration process of the simulation input space and the associated output behavior. A Gaussian Process (GP) is used as a metamodel to approximate the function inherently defined by the simulation model itself. The GPs can nicely handle the uncertainty associated with their predictions, which eventually can be improved with active learning through simulation runs. This property provides a practical and efficient way to analyze the simulator's behavior and therefore, to assess the performance of policies regarding the underlying real-world systems and services. The authors illustrate the proposed methodology using an Emergency Medical Service (EMS) simulator. Two outputs are analyzed and compared, namely, the survival rate and response time averages. The medical emergency response time recommendation of 8 min is explored as well its relation with the survival rate. The results show that this methodology can identify regions in the simulation input space that might affect the performance and success of medical policies with regards to emergency vehicle dispatching services.
Original languageEnglish
JournalSimulation Notes Europe
Volume97
Pages (from-to)101947
ISSN2305-9974
DOIs
Publication statusPublished - 2019

Keywords

  • Active learning
  • Simulation metamodeling
  • Gaussian processes
  • Kriging
  • Emergency medical service

Cite this

@article{ba5f4bcdf2db423a8c402b3f85602b5a,
title = "Active learning metamodeling for policy analysis: Application to an emergency medical service simulator",
abstract = "Simulation approaches constitute a well-established tool to model, understand, and predict the behavior of transportation systems, and ultimately to assess the performance of transportation policies. Due to their multidimensionality nature, such systems are not often approachable through conventional analytic methods, making simulation modeling the only reliable tool of study. Nevertheless, simulation models can turn out to be computationally expensive when embedded with enough detail. An immediate answer to this shortcoming is the use of simulation metamodels that are designed to approximate the simulators’ results. In this work, the authors propose a metamodeling approach based on active learning that seeks to improve the exploration process of the simulation input space and the associated output behavior. A Gaussian Process (GP) is used as a metamodel to approximate the function inherently defined by the simulation model itself. The GPs can nicely handle the uncertainty associated with their predictions, which eventually can be improved with active learning through simulation runs. This property provides a practical and efficient way to analyze the simulator's behavior and therefore, to assess the performance of policies regarding the underlying real-world systems and services. The authors illustrate the proposed methodology using an Emergency Medical Service (EMS) simulator. Two outputs are analyzed and compared, namely, the survival rate and response time averages. The medical emergency response time recommendation of 8 min is explored as well its relation with the survival rate. The results show that this methodology can identify regions in the simulation input space that might affect the performance and success of medical policies with regards to emergency vehicle dispatching services.",
keywords = "Active learning, Simulation metamodeling, Gaussian processes, Kriging, Emergency medical service",
author = "Francisco Antunes and M. Amorim and Pereira, {Francisco Camara} and Bernardete Ribeiro",
year = "2019",
doi = "10.1016/j.simpat.2019.101947",
language = "English",
volume = "97",
pages = "101947",
journal = "Simulation Notes Europe",
issn = "2305-9974",
publisher = "Elsevier",

}

Active learning metamodeling for policy analysis: Application to an emergency medical service simulator. / Antunes, Francisco; Amorim, M.; Pereira, Francisco Camara; Ribeiro, Bernardete.

In: Simulation Notes Europe, Vol. 97, 2019, p. 101947.

Research output: Contribution to journalJournal articleResearchpeer-review

TY - JOUR

T1 - Active learning metamodeling for policy analysis: Application to an emergency medical service simulator

AU - Antunes, Francisco

AU - Amorim, M.

AU - Pereira, Francisco Camara

AU - Ribeiro, Bernardete

PY - 2019

Y1 - 2019

N2 - Simulation approaches constitute a well-established tool to model, understand, and predict the behavior of transportation systems, and ultimately to assess the performance of transportation policies. Due to their multidimensionality nature, such systems are not often approachable through conventional analytic methods, making simulation modeling the only reliable tool of study. Nevertheless, simulation models can turn out to be computationally expensive when embedded with enough detail. An immediate answer to this shortcoming is the use of simulation metamodels that are designed to approximate the simulators’ results. In this work, the authors propose a metamodeling approach based on active learning that seeks to improve the exploration process of the simulation input space and the associated output behavior. A Gaussian Process (GP) is used as a metamodel to approximate the function inherently defined by the simulation model itself. The GPs can nicely handle the uncertainty associated with their predictions, which eventually can be improved with active learning through simulation runs. This property provides a practical and efficient way to analyze the simulator's behavior and therefore, to assess the performance of policies regarding the underlying real-world systems and services. The authors illustrate the proposed methodology using an Emergency Medical Service (EMS) simulator. Two outputs are analyzed and compared, namely, the survival rate and response time averages. The medical emergency response time recommendation of 8 min is explored as well its relation with the survival rate. The results show that this methodology can identify regions in the simulation input space that might affect the performance and success of medical policies with regards to emergency vehicle dispatching services.

AB - Simulation approaches constitute a well-established tool to model, understand, and predict the behavior of transportation systems, and ultimately to assess the performance of transportation policies. Due to their multidimensionality nature, such systems are not often approachable through conventional analytic methods, making simulation modeling the only reliable tool of study. Nevertheless, simulation models can turn out to be computationally expensive when embedded with enough detail. An immediate answer to this shortcoming is the use of simulation metamodels that are designed to approximate the simulators’ results. In this work, the authors propose a metamodeling approach based on active learning that seeks to improve the exploration process of the simulation input space and the associated output behavior. A Gaussian Process (GP) is used as a metamodel to approximate the function inherently defined by the simulation model itself. The GPs can nicely handle the uncertainty associated with their predictions, which eventually can be improved with active learning through simulation runs. This property provides a practical and efficient way to analyze the simulator's behavior and therefore, to assess the performance of policies regarding the underlying real-world systems and services. The authors illustrate the proposed methodology using an Emergency Medical Service (EMS) simulator. Two outputs are analyzed and compared, namely, the survival rate and response time averages. The medical emergency response time recommendation of 8 min is explored as well its relation with the survival rate. The results show that this methodology can identify regions in the simulation input space that might affect the performance and success of medical policies with regards to emergency vehicle dispatching services.

KW - Active learning

KW - Simulation metamodeling

KW - Gaussian processes

KW - Kriging

KW - Emergency medical service

U2 - 10.1016/j.simpat.2019.101947

DO - 10.1016/j.simpat.2019.101947

M3 - Journal article

VL - 97

SP - 101947

JO - Simulation Notes Europe

JF - Simulation Notes Europe

SN - 2305-9974

ER -