Auxiliary Finite-Difference Residual-Gradient Regularization for PINNs

arXiv cs.LG / 4/17/2026

📰 NewsIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper proposes a hybrid regularization for physics-informed neural networks (PINNs) where the PDE residual stays AD-based, while finite differences (FD) are used only in a weak auxiliary term to penalize gradients of the sampled residual field.
Experiments are run in two stages: a 2D Poisson benchmark and a 3D annular heat-conduction benchmark, with the auxiliary FD grid implemented as a body-fitted shell near a wavy outer wall.
In the Poisson benchmark, the FD residual-gradient regularizer achieves much of the residual-gradient control effect while revealing a trade-off between field accuracy and residual “cleanliness.”
In the 3D heat-conduction task, the shell-based auxiliary term substantially improves application-facing metrics such as outer-wall flux and boundary-condition behavior, with the best configuration using a fixed shell weight of 5e-4 under the Kourkoutas-beta optimizer regime.
The study concludes that hybrid PINNs benefit most when the auxiliary FD regularization is aligned with the specific physical quantity of interest (e.g., outer-wall flux), rather than replacing the AD-based PDE residual.

Abstract

Physics-informed neural networks (PINNs) are often selected by a single scalar loss even when the quantity of interest is more specific. We study a hybrid design in which the governing PDE residual remains automatic-differentiation (AD) based, while finite differences (FD) appear only in a weak auxiliary term that penalizes gradients of the sampled residual field. The FD term regularizes the residual field without replacing the PDE residual itself. We examine this idea in two stages. Stage 1 is a controlled Poisson benchmark comparing a baseline PINN, the FD residual-gradient regularizer, and a matched AD residual-gradient baseline. Stage 2 transfers the same logic to a three-dimensional annular heat-conduction benchmark (PINN3D), where baseline errors concentrate near a wavy outer wall and the auxiliary grid is implemented as a body-fitted shell adjacent to the wall. In Stage 1, the FD regularizer reproduces the main effect of residual-gradient control while exposing a trade-off between field accuracy and residual cleanliness. In Stage 2, the shell regularizer improves the application-facing quantities, namely outer-wall flux and boundary-condition behavior. Across seeds 0-5 and 100k epochs, the most reliable tested configuration is a fixed shell weight of 5e-4 under the Kourkoutas-beta optimizer regime: relative to a matched run without the shell term, it reduces the mean outer-wall BC RMSE from 1.22e-2 to 9.29e-4 and the mean wall-flux RMSE from 9.21e-3 to 9.63e-4. Adam with beta2=0.999 becomes usable when the initial learning rate is reduced to 1e-3, although its shell benefit is less robust than under Kourkoutas-beta. Overall, the results support a targeted view of hybrid PINNs: an auxiliary-only FD regularizer is most valuable when it is aligned with the physical quantity of interest, here the outer-wall flux.