DDCL: Deep Dual Competitive Learning: A Differentiable End-to-End Framework for Unsupervised Prototype-Based Representation Learning

arXiv cs.LG / 4/3/2026

💬 OpinionSignals & Early TrendsIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper argues that a key limitation of deep clustering comes from the gap between feature learning and cluster assignment when external methods like k-means generate pseudo-labels for training.
It proposes Deep Dual Competitive Learning (DDCL), an end-to-end differentiable prototype-based framework that replaces external k-means with an internal Dual Competitive Layer (DCL) producing prototypes and soft cluster assignments directly from the network.
The approach uses a single unified loss optimized via backpropagation, eliminating pseudo-label discretization and iterative Lloyd steps typically required by k-means-based pipelines.
The authors derive a theoretical algebraic decomposition of the soft quantization loss into a simplex-constrained reconstruction error plus a non-negative prototype variance term that provides an implicit separation force against prototype collapse.
Experiments report strong empirical gains, including 65% higher clustering accuracy than a non-differentiable ablation and 122% improvement over an end-to-end DeepCluster variant, alongside long-run stability claims over extensive training.

Abstract

A persistent structural weakness in deep clustering is the disconnect between feature learning and cluster assignment. Most architectures invoke an external clustering step, typically k-means, to produce pseudo-labels that guide training, preventing the backbone from directly optimising for cluster quality. This paper introduces Deep Dual Competitive Learning (DDCL), the first fully differentiable end-to-end framework for unsupervised prototype-based representation learning. The core contribution is architectural: the external k-means is replaced by an internal Dual Competitive Layer (DCL) that generates prototypes as native differentiable outputs of the network. This single inversion makes the complete pipeline, from backbone feature extraction through prototype generation to soft cluster assignment, trainable by backpropagation through a single unified loss, with no Lloyd iterations, no pseudo-label discretisation, and no external clustering step. To ground the framework theoretically, the paper derives an exact algebraic decomposition of the soft quantisation loss into a simplex-constrained reconstruction error and a non-negative weighted prototype variance term. This identity reveals a self-regulating mechanism built into the loss geometry: the gradient of the variance term acts as an implicit separation force that resists prototype collapse without any auxiliary objective, and leads to a global Lyapunov stability theorem for the reduced frozen-encoder system. Six blocks of controlled experiments validate each structural prediction. The decomposition identity holds with zero violations across more than one hundred thousand training epochs; the negative feedback cycle is confirmed with Pearson -0.98; with a jointly trained backbone, DDCL outperforms its non-differentiable ablation by 65% in clustering accuracy and DeepCluster end-to-end by 122%.