Fast and Provably Accurate Sequential Designs using Hilbert Space Gaussian Processes

arXiv stat.ML / 4/23/2026

💬 OpinionIdeas & Deep AnalysisModels & Research

Key Points

  • The paper addresses a key limitation in Gaussian-process-based sequential design: computing the integrated mean squared error (IMSE) acquisition function is difficult because the required integrals often have no closed-form solutions for most kernels.
  • It proposes a Hilbert space Gaussian process approximation that uses a truncated eigenbasis representation of the integral, enabling closed-form evaluation of the IMSE acquisition function.
  • The authors derive sharp, non-asymptotic global error bounds for both the approximation error (for isotropic kernels) and the resulting acquisition-function error.
  • Numerical experiments show that, for gamma-stabilizing settings, the method achieves substantially lower prediction error while also reducing computation time versus existing benchmarks.

Abstract

Gaussian processes are widely used for accurate emulation of unknown surfaces in sequential design of expensive simulation experiments. Integrated mean squared error (IMSE) is an effective acquisition function for sequential designs based on Gaussian processes. However, existing approaches struggle with its implementation because the required integrals often lack closed-form expressions for most kernel functions. We propose a novel and computationally efficient Hilbert space Gaussian process approximation for the IMSE acquisition function, where a truncated eigenbasis representation of the integral enables closed-form evaluation. We establish sharp global non-asymptotic bounds for both the approximation error of isotropic kernels and the resulting error in the acquisition function. In a series of numerical experiments with \gamma-stabilizing, the proposed method achieves substantially lower prediction error and reduced computation time compared to existing benchmarks. These results demonstrate that the proposed Hilbert space Gaussian process framework provides an accurate and computationally efficient approach for Gaussian process based sequential design.