Sparse clustering via the Deterministic Information Bottleneck algorithm

arXiv stat.ML / 4/14/2026

💬 OpinionIdeas & Deep AnalysisModels & Research

Key Points

  • The paper introduces an information-theoretic clustering framework called the Deterministic Information Bottleneck algorithm to handle cases where cluster structure exists only within a sparse subset of the feature space.
  • It jointly learns feature weighting and cluster assignments, addressing challenges that traditional clustering methods struggle with under sparse or high-dimensional data.
  • The authors report competitive performance versus existing sparse-data clustering approaches based on simulations on synthetic datasets.
  • They validate the approach with an application to a real-world genomics dataset, demonstrating practical utility beyond controlled experiments.

Abstract

Cluster analysis relates to the task of assigning objects into groups which ideally present some desirable characteristics. When a cluster structure is confined to a subset of the feature space, traditional clustering techniques face unprecedented challenges. We present an information theoretic framework that overcomes the problems associated with sparse data, allowing for joint feature weighting and clustering. Our proposal constitutes a competitive alternative to existing clustering algorithms for sparse data, as demonstrated through simulations on synthetic data. The effectiveness of our method is established by an application on a real-world genomics data set.