Hi!
I’ve been messing around with LLMs for a while, and I recently upgraded to a 5070ti (16 GB). It feels like a breath of fresh air compared to my old 4060 (8 GB), but now I’m finding myself wanting a bit more VRAM. I’ve searched the market, and 3060 (12 GB) seems like a pretty decent option.
I know it’s an old GPU, but it should still be better than CPU offloading, right? These GPUs are supposed to be going into my home server, so I’m trying to stay on a budget. I am going to use them to inference and train models.
Do you think I might run into any issues with CUDA drivers, inference engine compatibility, or inter-GPU communication? Mixing different architectures makes me a bit nervous.
Also, I’m worried about temperatures. On my motherboard, the hot air from the first GPU would go straight into the second one. My 5070ti usually doesn’t go above 75°C under load so could 3060 be able to handle that hot intake air?
[link] [comments]
