Ultra-low-light computer vision using trained photon correlations

arXiv cs.CV / 4/15/2026

📰 NewsSignals & Early TrendsIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper proposes using correlated-photon illumination in a hybrid optical-electronic computer-vision pipeline to improve object recognition under ultra-low-light, noisy conditions.
It introduces correlation-aware training (CAT), which jointly optimizes a trainable correlated-photon illumination source and a Transformer backend so the model learns to exploit photon correlation structure.
The method achieves notable gains with very few measurements (≤100 shots), demonstrating improved classification accuracy by up to 15 percentage points versus conventional uncorrelated illumination approaches.
Results also show advantages over using untrained correlated-photon illumination, indicating the benefit of task-specific training rather than relying on generic correlation patterns.
Overall, the work highlights that training the illumination correlations together with the digital vision model can push accuracy limits in photon-budget-constrained sensing beyond reconstruction-focused methods.

Abstract

Illumination using correlated photon sources has been established as an approach to allowing high-fidelity images to be reconstructed from noisy camera frames by taking advantage of the knowledge that signal photons are spatially correlated whereas detector clicks due to noise are uncorrelated. However, in computer-vision tasks, the goal is often not ultimately to reconstruct an image, but to make inferences about a scene -- such as what object is present. Here we show how correlated-photon illumination can be used to gain an advantage in a hybrid optical-electronic computer-vision pipeline for object recognition. We demonstrate correlation-aware training (CAT): end-to-end optimization of a trainable correlated-photon illumination source and a Transformer backend in a way that the Transformer can learn to benefit from the correlations, using a small number (<= 100) of shots. We show a classification accuracy enhancement of up to 15 percentage points over conventional, uncorrelated-illumination-based computer vision in ultra-low-light and noisy imaging conditions, as well as an improvement over using untrained correlated-photon illumination. Our work illustrates how specializing to a computer-vision task -- object recognition -- and training the pattern of photon correlations in conjunction with a digital backend allows us to push the limits of accuracy in highly photon-budget-constrained scenarios beyond existing methods focused on image reconstruction.