AI Navigate

The Phasor Transformer: Resolving Attention Bottlenecks on the Unit Circle

arXiv cs.LG / 3/19/2026

📰 NewsIdeas & Deep AnalysisModels & Research

Key Points

  • The Phasor Transformer block represents sequence states on the unit-circle manifold and combines trainable phase shifts with a parameter-free Discrete Fourier Transform to enable global token coupling.
  • It achieves global mixing with O(N log N) complexity, avoiding explicit attention maps and reducing the computational bottleneck for long-context sequences.
  • Stacking these blocks defines the Large Phasor Model (LPM), which demonstrates stable dynamics and competitive forecasting on synthetic multi-frequency time-series benchmarks with a compact parameter budget.
  • The approach emphasizes geometry-constrained phase computation as a practical path toward scalable temporal modeling in oscillatory domains.
  • This work establishes an efficiency-performance frontier for time-series modeling and may influence future architectures beyond standard self-attention.

Abstract

Transformer models have redefined sequence learning, yet dot-product self-attention introduces a quadratic token-mixing bottleneck for long-context time-series. We introduce the \textbf{Phasor Transformer} block, a phase-native alternative representing sequence states on the unit-circle manifold S^1. Each block combines lightweight trainable phase-shifts with parameter-free Discrete Fourier Transform (DFT) token coupling, achieving global \mathcal{O}(N\log N) mixing without explicit attention maps. Stacking these blocks defines the \textbf{Large Phasor Model (LPM)}. We validate LPM on autoregressive time-series prediction over synthetic multi-frequency benchmarks. Operating with a highly compact parameter budget, LPM learns stable global dynamics and achieves competitive forecasting behavior compared to conventional self-attention baselines. Our results establish an explicit efficiency-performance frontier, demonstrating that large-model scaling for time-series can emerge from geometry-constrained phase computation with deterministic global coupling, offering a practical path toward scalable temporal modeling in oscillatory domains.