Certified Dimension Reduction for Bayesian Updating with the Cross-Entropy Method

Max Ehre, Rafael Flock, Martin Fußeder, Iason Papaioannou, Daniel Straub

Research output: Contribution to journalJournal articleResearchpeer-review

5 Downloads (Pure)


In inverse problems, the parameters of a model are estimated based on observations of the model response. The Bayesian approach is powerful for solving such problems; one formulates a prior distribution for the parameter state that is updated with the observations to compute the posterior parameter distribution. Solving for the posterior distribution can be challenging when, e.g., prior and posterior significantly differ from one another and/or the parameter space is high-dimensional. We use a sequence of importance sampling measures that arise by tempering the likelihood to approach inverse problems exhibiting a significant distance between prior and posterior. Each importance sampling measure is identified by cross-entropy minimization as proposed in the context of Bayesian inverse problems in Engel et al. [J. Comput. Phys., 473 (2023), 111746]. To efficiently address problems with high-dimensional parameter spaces, we set up the minimization procedure in a low-dimensional subspace of the original parameter space. The principal idea is to analyze the spectrum of the second-moment matrix of the gradient of the log-likelihood function to identify a suitable subspace. Following Zahm et al. [Math. Comp., 91 (2022), pp. 1789-1835], an upper bound on the Kullback-Leibler divergence between full-dimensional and subspace posterior is provided, which can be utilized to determine the effective dimension of the inverse problem corresponding to a prescribed approximation error bound. We suggest heuristic criteria for optimally selecting the number of model and model gradient evaluations in each iteration of the importance sampling sequence. We investigate the performance of this approach using examples from engineering mechanics set in various parameter space dimensions.

Original languageEnglish
JournalSIAM-ASA Journal on Uncertainty Quantification
Issue number1
Pages (from-to)358-388
Publication statusPublished - 2023

Bibliographical note

Funding: The work of the authors was supported by German Research Foundation (DFG) grants STR 1140/11-1, PA 2901/1-1, and BL 306/33. \dagger Engineering Risk Analysis Group, School of Engineering and Design, Technical University of Munich, Munich 80333, Germany (max.ehre@tum.de, iason.papaioannou@tum.de, straub@tum.de). \ddagger DTU Compute, Department of Applied Mathematics and Computer Science, Technical University of Denmark, DK-2800 Lyngby, Denmark (raff@dtu.dk). \S School of Engineering and Design, Technical University of Munich, Munich 80333, Germany (martin. fusseder@tum.de).


  • Bayesian inverse problems
  • Certified dimension reduction
  • Cross-entropy method
  • High dimensions
  • Importance sampling


Dive into the research topics of 'Certified Dimension Reduction for Bayesian Updating with the Cross-Entropy Method'. Together they form a unique fingerprint.

Cite this