New Hybrid Fine-Tuning Paradigm for LLMs: Algorithm Design and Convergence Analysis Framework

arXiv cs.AI / 4/14/2026

💬 OpinionSignals & Early TrendsIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper proposes a hybrid fine-tuning paradigm that jointly updates full LLM parameters and PEFT modules to address the cost of full fine-tuning and the knowledge/quality limitations of PEFT alone.
It introduces an optimization scheme combining zeroth-order and first-order methods to better handle the differing optimization behaviors of the heterogeneous parameter sets.
The authors develop a theoretical framework using a “hybrid smoothness” condition to model the mixed optimization landscape and derive a convergence analysis for a reshuffling-type SGD variant with multiple learning rates.
Empirical results across multiple downstream tasks and model architectures reportedly show consistent performance improvements, suggesting the approach is practical for large-scale LLM fine-tuning.

Abstract

Fine-tuning Large Language Models (LLMs) typically involves either full fine-tuning, which updates all model parameters, or Parameter-Efficient Fine-Tuning (PEFT), which adjusts a small subset of parameters. However, both approaches have inherent limitations: full fine-tuning is computationally expensive, while PEFT often struggles to learn new knowledge and exhibits suboptimal performance. To overcome these issues, we propose a novel hybrid fine-tuning approach that jointly updates both LLMs and PEFT modules using a combination of zeroth-order and first-order optimization methods. To analyze our new algorithm, we develop a theoretical framework centered on the concept of hybrid smoothness condition, which accounts for the heterogeneous nature of the optimization landscape in joint LLM and PEFT training. We derive a rigorous convergence analysis for the convergence of reshuffling-type SGD algorithm under multiple learning rates and demonstrate its effectiveness through extensive empirical studies across various downstream tasks and model architectures. On the practical side, our results demonstrate consistent performance improvement, making the approach a viable solution for large-scale language model fine-tuning.