AI Navigate
インサイト最新記事AI大全カオスマップ

Evaluating Singular Value Thresholds for DNN Weight Matrices based on Random Matrix Theory

arXiv stat.ML / 4/10/2026

💬 OpinionIdeas & Deep AnalysisModels & Research
Read original →
共有:

Key Points

  • The paper studies how to choose singular-value thresholds for low-rank approximations of DNN weight matrices using random matrix theory to separate signal from noise components.
  • It models each weight matrix as the sum of a signal matrix plus a noise matrix, then removes “noise-related” singular values to obtain the low-rank approximation.
  • To validate whether a threshold is appropriate, the authors introduce an evaluation metric based on cosine similarity between singular vectors of the inferred signal and the original weight matrix.
  • Numerical experiments compare two threshold estimation methods using the proposed cosine-similarity metric to judge approximation adequacy.

Abstract

This study evaluates thresholds for removing singular values from singular value decomposition-based low-rank approximations of deep neural network weight matrices. Each weight matrix is modeled as the sum of signal and noise matrices. The low-rank approximation is obtained by removing noise-related singular values using a threshold based on random matrix theory. To assess the adequacy of this threshold, we propose an evaluation metric based on the cosine similarity between the singular vectors of the signal and original weight matrices. The proposed metric is used in numerical experiments to compare two threshold estimation methods.

関連おすすめサービス

※当サイトはアフィリエイト広告を利用しています
🎧

Notta搭載AI議事録イヤホン ZENCHORD1

AI時代の仕事術。Notta搭載で会議の議事録を自動生成するスマートイヤホン。

🎙️

AI搭載ボイスレコーダー Plaud

世界100万人が愛用。AIで文字起こし・要約を自動化するボイスレコーダー。

🖼️

画像高画質化AIツール Aiarty Image Enhancer

AIで画像を高画質化。写真・イラストを簡単にアップスケール。

Evaluating Singular Value Thresholds for DNN Weight Matrices based on Random Matrix Theory | AI Navigate