Llama.cpp Mi50 ROCm 7 対 Vulkan ベンチマーク

Reddit r/LocalLLaMA / 2026/3/23

💬 オピニオンDeveloper Stack & InfrastructureTools & Practical UsageModels & Research

原文を読む →

共有:

要点

Mi50 32GB システム（EPYC 7532、Proxmox 仮想化、Ubuntu Server 24.04、カーネル 6.8）上で、llama.cpp ビルド 8467 および llama-bench を用いた ROCm 7.13 nightly 対 Vulkan 1.4.341.1 の比較ベンチマーク。
テスト対象モデルには Qwen 3.5 9B/27B/122B および Nemotron Cascade 2 を含み、122B は -ncmoe 28 設定で CPU にオフロードされる（-mmp 0）。
プロンプト処理では、密度の高いモデルの短いコンテキスト実行で Vulkan が高速で、長いコンテキストや MOE モデルでは ROCm が優位となり、特に GPU/CPU を分割した推論でその傾向が顕著である。
256トークンの生成でも同じパターンが見られ、MOE シナリオでは再度 ROCm が有利になる。ビルドでは -fa 1 を使用し、デフォルトの f16 キャッシュを利用した。

Llama.cpp Mi50 ROCm 7 vs Vulkan Benchmarks

Testing ROCm 7 using TheRock nightly tarballs against Vulkan on Mi50.

System Setup

System	Spec	Note
GPU	1x Mi50 32GB	113-D1631700-111 vbios
CPU	EPYC 7532	Proxmox virtualized 28c/56t allocated
RAM	8x16GB DDR4 2933Mhz
OS	Ubuntu Server 24.04	Kernel 6.8.0-106-generic
ROCm Version	7.13.0a20260321	TheRock Nightly Page
Vulkan	1.4.341.1
Llama.ccp Build	8467	Built using recommended commands from build wiki

Models Tested

All models run with -fa 1 and default f16 cache types using llama-bench

Model	Quant	Notes
Qwen 3.5 9B	Bartowski Q8_0
Qwen 3.5 27B	Bartowski Q8_0
Qwen 3.5 122B	Bartowski Q4_0	28 layers offloaded to CPU with -ncmoe 28, -mmp 0
Nemotron Cascade 2	mradermacher il-Q5_K_M

Prompt Processing

Vulkan at short context (sub-16k) is reliably faster than ROCm on dense-models only (Q3.5 9B and 27B). At long context on dense models or basically any context length on MOE models, ROCm is consistently faster.

Token Generation

All generations standardized at 256 tokens at varying depths. The pattern from Prompt Processing repeats here; Vulkan is faster with dense models. Speed doesn't decay with depth as much as prompt processing does. If you're using MOEs and especially split GPU/CPU inference, ROCm is faster.

Conclusions

Vulkan is the winner at short context dense models. If you're chatting and changing chats often with dense models, Vulkan wins.
ROCm is faster for anything beyond 16k context when you factor in prompt processing and generation speeds combined. Dense or MOE, doesn't matter when Vulkan prompt processing falls off a cliff. The Vulkan prompt processing numbers (not pictured but included in the full dataset below) at depth were bleak. However, read the limitations below as the nightly builds do sacrifice stability...

Limitations

TheRock's ROCm nightly builds are not a stable release. You probably will encounter weird behavior. Whether a ROCm bug or a Llama.cpp bug I am not sure, but I currently cannot run ROCm llama-server with Qwen 3.5B 27B Q8 because it keeps trying to allocate the 8192MB prompt cache to VRAM instead of system ram causing an OOM error (-cram 0 isn't disabling it, -cram 1024 doesn't lower the size, don't know why). Runs with Vulkan though.

I also noticed what seemed to be a memory leak with a different ROCm nightly from a few weeks ago and an earlier llama.cpp version, which was resolved by switching back to Vulkan. OpenCode with 100k+ context resulted in memory usage on the GPU slowly creeping up from 90% up to an OOM using Qwen Next Coder and a ROCm nightly build. I have not tried to replicate it since switching back to ROCm and the newer nightly version though.

I'm an ex-dev turned product manager just learning and doing this as a hobby though, so it's fine :)

Full data set: https://pastebin.com/4pPuGAcV

submitted by /u/JaredsBored
[link] [comments]