AI Navigate

Operator-Theoretic Foundations and Policy Gradient Methods for General MDPs with Unbounded Costs

arXiv cs.LG / 3/19/2026

📰 NewsIdeas & Deep AnalysisModels & Research
Read original →
共有:

Key Points

  • The paper proposes an operator-theoretic view of MDPs that optimizes over linear operators on general function spaces and uses perturbation theory to express derivatives of the objective as functions of these operators.
  • It extends reinforcement learning theory from finite-state finite-action MDPs to general state and action spaces, including settings with unbounded costs.
  • The framework yields low-complexity PPO-type reinforcement learning algorithms applicable to general state and action spaces.
  • By unifying existing RL results under an operator-theoretic perspective, the work highlights new theoretical and practical directions for general MDPs.

Abstract

Markov decision processes (MDPs) is viewed as an optimization of an objective function over certain linear operators over general function spaces. Using the well-established perturbation theory of linear operators, this viewpoint allows one to identify derivatives of the objective function as a function of the linear operators. This leads to generalization of many well-known results in reinforcement learning to cases with generate state and action spaces. Prior results of this type were only established in the finite-state finite-action MDP settings and in settings with certain linear function approximations. The framework also leads to new low-complexity PPO-type reinforcement learning algorithms for general state and action space MDPs.

関連おすすめサービス

※当サイトはアフィリエイト広告を利用しています
🎧

Notta搭載AI議事録イヤホン ZENCHORD1

AI時代の仕事術。Notta搭載で会議の議事録を自動生成するスマートイヤホン。

🎙️

AI搭載ボイスレコーダー Plaud

世界100万人が愛用。AIで文字起こし・要約を自動化するボイスレコーダー。

🖼️

画像高画質化AIツール Aiarty Image Enhancer

AIで画像を高画質化。写真・イラストを簡単にアップスケール。