IMPACT-HOI: Supervisory Control for Onset-Anchored Partial HOI Event Construction

arXiv cs.CV / 5/5/2026

📰 NewsDeveloper Stack & InfrastructureSignals & Early TrendsModels & Research

共有:

Key Points

The IMPACT-HOI paper introduces a mixed-initiative annotation framework that builds structured event graphs for human-object interactions (HOI) from egocentric procedural videos.
It models event construction as incremental resolution of a partially specified, onset-anchored event state, improving the quality of structured supervision for robot manipulation learning from human demonstrations.
A trust-calibrated controller chooses between direct queries, human-confirmed suggestions, and conservative completions based on annotator behavior and evidence quality.
The framework includes a risk-bounded execution protocol with atomic rollback to preserve human-confirmed decisions when automated updates conflict.
In a small user study (9 participants), the approach reportedly reduced manual annotation actions by 13.5%, improved event match rate by 46.67%, and achieved zero confirmed-field violations under the tested protocol, with code planned for public release.

Abstract

We present IMPACT-HOI, a mixed-initiative framework for annotating egocentric procedural video by constructing structured event graphs for Human-Object Interactions (HOI), motivated by the need for high-quality structured supervision for learning robot manipulation from human demonstration. IMPACT-HOI frames this task as the incremental resolution of a partially specified, onset-anchored event state. A trust-calibrated controller selects among direct queries, human-confirmed suggestions, and conservative completions based on empirical annotator behavior and evidence quality. A risk-bounded execution protocol, utilizing atomic rollback, ensures that human-confirmed decisions are preserved against conflicting automated updates. A user study with 9 participants shows a 13.5% reduction in manual annotation actions, a 46.67% event match rate, and zero confirmed-field violations under the studied protocol. The code will be made publicly available at https://github.com/541741106/IMPACT_HOI.