Instruction Data Selection via Answer Divergence

arXiv cs.CL / 4/14/2026

💬 OpinionSignals & Early TrendsIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper introduces Answer Divergence-Guided Selection (ADG) to pick instruction-tuning data using the geometric properties of multiple generated answers per instruction.
ADG samples several high-temperature outputs, embeds them, and computes a divergence score that captures both dispersion magnitude and directional/shape anisotropy to identify multi-modal answer behavior.
Experiments across two model backbones and three public instruction pools show that fine-tuning on just 10K ADG-selected examples outperforms other strong selection methods on six benchmarks covering reasoning, knowledge, and coding.
Ablation/analysis indicates that both dispersion magnitude and shape anisotropy are jointly necessary, supporting answer divergence as a practical signal for instruction data quality.
The work provides code and appendix in supplementary materials to enable further evaluation and replication.

Abstract

Instruction tuning relies on large instruction-response corpora whose quality and composition strongly affect downstream performance. We propose Answer Divergence-Guided Selection (ADG), which selects instruction data based on the geometric structure of multi-sample outputs. ADG draws several high-temperature generations per instruction, maps responses into an embedding space, and computes an output divergence score that jointly encodes dispersion magnitude and shape anisotropy. High scores correspond to instructions whose answers are both far apart and multi-modal, rather than clustered paraphrases along a single direction. Across two backbones and three public instruction pools, fine-tuning on only 10K ADG-selected examples consistently outperforms strong selectors on six benchmarks spanning reasoning, knowledge, and coding. Analyses further show that both dispersion magnitude and shape anisotropy are necessary, supporting answer divergence as a practical signal for instruction data selection. Code and appendix are included in the supplementary materials.