Optimization with SpotOptim

arXiv cs.LG / 4/16/2026

📰 NewsIdeas & Deep AnalysisTools & Practical UsageModels & Research

共有:

Key Points

The spotoptim Python package introduces surrogate-model-based optimization for expensive black-box functions using a Kriging loop with Expected Improvement guidance.
It supports continuous, integer, and categorical decision variables, includes noise-aware evaluation via OCBA, and offers multi-objective extensions.
The framework uses steady-state parallelization to overlap surrogate model search with objective evaluations and employs a success-rate-based restart strategy to handle stagnation without losing the best found solution.
spotoptim is designed to interoperate with the scientific Python ecosystem by returning SciPy-compatible OptimizeResult objects and accepting any scikit-learn-compatible surrogate model, with TensorBoard logging for live monitoring.
The paper/article provides architecture details, worked examples (including neural-network hyperparameter tuning), and a comparison against tools such as BoTorch, Optuna, Ray Tune, BOHB, SMAC, and Hyperopt, while being open-source.

Abstract

The `spotoptim` package implements surrogate-model-based optimization of expensive black-box functions in Python. Building on two decades of Sequential Parameter Optimization (SPO) methodology, it provides a Kriging-based optimization loop with Expected Improvement, support for continuous, integer, and categorical variables, noise-aware evaluation via Optimal Computing Budget Allocation (OCBA), and multi-objective extensions. A steady-state parallelization strategy overlaps surrogate search with objective evaluation on multi-core hardware, and a success-rate-based restart mechanism detects stagnation while preserving the best solution found. The package returns scipy-compatible `OptimizeResult` objects and accepts any scikit-learn-compatible surrogate model. Built-in TensorBoard logging provides real-time monitoring of convergence and surrogate quality. This report describes the architecture and module structure of spotoptim, provides worked examples including neural network hyperparameter tuning, and compares the framework with BoTorch, Optuna, Ray Tune, BOHB, SMAC, and Hyperopt. The package is open-source.