Exploring Language-Agnosticity in Function Vectors: A Case Study in Machine Translation

arXiv cs.CL / 4/22/2026

📰 NewsIdeas & Deep AnalysisModels & Research

共有:

Key Points

The study investigates whether function vectors (FVs)—task representations derived from model activations during in-context learning—are language-agnostic in multilingual LLMs.
Using machine translation as a test case, it finds that translation FVs learned from an English→Target direction transfer to other target languages and improve the ranking of correct translation tokens across multiple previously unseen languages.
Ablation experiments indicate that removing the function vectors significantly degrades cross-language translation quality, while having limited effect on unrelated tasks.
The research also shows that FVs transfer from base models to instruction-tuned variants and can partially generalize from word-level to sentence-level translation.
The results extend prior findings about multilingual representation language-agnosticity by demonstrating similar behavior for function vectors specifically in translation settings.

Abstract

Function vectors (FVs) are vector representations of tasks extracted from model activations during in-context learning. While prior work has shown that multilingual model representations can be language-agnostic, it remains unclear whether the same holds for function vectors. We study whether FVs exhibit language-agnosticity, using machine translation as a case study. Across three decoder-only multilingual LLMs, we find that translation FVs extracted from a single English

\rightarrow

Target direction transfer to other target languages, consistently improving the rank of correct translation tokens across multiple unseen languages. Ablation results show that removing the FV degrades translation across languages with limited impact on unrelated tasks. We further show that base-model FVs transfer to instruction-tuned variants and partially generalize from word-level to sentence-level translation.