Unlocking Positive Transfer in Incrementally Learning Surgical Instruments: A Self-reflection Hierarchical Prompt Framework

arXiv cs.CV / 4/6/2026

💬 OpinionIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper addresses incremental learning for surgical video scene parsing, where models must learn to segment an expanding set of instruments over time without catastrophic forgetting.
It proposes a self-reflection hierarchical prompt framework that enables both positive forward transfer (reusing past knowledge to learn new instrument classes) and positive backward transfer (improving earlier learned classes after learning new ones).
The method uses a frozen pre-trained model with dynamically appended instrument-aware prompts organized in a hierarchical prompt parsing tree, exposing shared knowledge for easier learning of new classes.
To strengthen backward transfer while preserving old capabilities, it applies self-reflection refinement using directed-weighted graph propagation informed by knowledge associations in the tree.
Experiments indicate the framework works for both CNN-based and transformer-based (foundation) models, improving over competing approaches by more than 5% and 11% on two public benchmarks.

Abstract

To continuously enhance model adaptability in surgical video scene parsing, recent studies incrementally update it to progressively learn to segment an increasing number of surgical instruments over time. However, prior works constantly overlooked the potential of positive forward knowledge transfer, i.e., how past knowledge could help learn new classes, and positive backward knowledge transfer, i.e., how learning new classes could help refine past knowledge. In this paper, we propose a self-reflection hierarchical prompt framework that unlocks the power of positive forward and backward knowledge transfer in class incremental segmentation, aiming to proficiently learn new instruments, improve existing skills of regular instruments, and avoid catastrophic forgetting of old instruments. Our framework is built on a frozen, pre-trained model that adaptively appends instrument-aware prompts for new classes throughout training episodes. To enable positive forward knowledge transfer, we organize instrument prompts into a hierarchical prompt parsing tree with the instrument-shared prompt partition as the root node, n-part-shared prompt partitions as intermediate nodes and instrument-distinct prompt partitions as leaf nodes, to expose the reusable historical knowledge for new classes to simplify their learning. Conversely, to encourage positive backward knowledge transfer, we conduct self-reflection refining on existing knowledge by directed-weighted graph propagation, examining the knowledge associations recorded in the tree to improve its representativeness without causing catastrophic forgetting. Our framework is applicable to both CNN-based models and advanced transformer-based foundation models, yielding more than 5% and 11% improvements over the competing methods on two public benchmarks respectively.