ParamBoost: Gradient Boosted Piecewise Cubic Polynomials

arXiv cs.LG / 4/22/2026

📰 NewsIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper introduces ParamBoost, a new Generalized Additive Model (GAM) whose per-feature shape functions are learned via gradient boosting using piecewise cubic polynomial functions at leaf nodes.
ParamBoost adds multiple parametric-analysis constraints to improve interpretability and functional quality, including C2 continuity (up to second derivatives), monotonicity, convexity, feature interaction constraints, and specification constraints.
Experiments indicate that the unconstrained ParamBoost variant outperforms existing state-of-the-art GAM approaches on several real-world datasets.
The authors show that users can selectively apply only the constraints they need, achieving strong customization of interpretability with only a modest reduction in predictive performance.

Abstract

Generalized Additive Models (GAMs) can be used to create non-linear glass-box (i.e. explicitly interpretable) models, where the predictive function is fully observable over the complete input space. However, glass-box interpretability itself does not allow for the incorporation of expert knowledge from the modeller. In this paper, we present ParamBoost, a novel GAM whose shape functions (i.e. mappings from individual input features to the output) are learnt using a Gradient Boosting algorithm that fits cubic polynomial functions at leaf nodes. ParamBoost incorporates several constraints commonly used in parametric analysis to ensure well-refined shape functions. These constraints include: (i) continuity of the shape functions and their derivatives (up to C2); (ii) monotonicity; (iii) convexity; (iv) feature interaction constraints; and (v) model specification constraints. Empirical results show that the unconstrained ParamBoost model consistently outperforms state-of-the-art GAMs across several real-world datasets. We further demonstrate that modellers can selectively impose required constraints at a modest trade-off in predictive performance, allowing the model to be fully tailored to application-specific interpretability and parametric-analysis requirements.