Two things: a repaired 3090 24gb can be sold for 1000+ euro in Germany at least. It's still crazy expensive, so if you have the skills and devices, it can be done with a profit when you buy 3 broken cards and can sell 2 repaired one.

I never reballed any IC, so I would not dare to try it on my card.

CPU and GPUs are kind of a different game. I would rather like to have a shop, which has a good reputation on RTX gpu repairs. But thanks :)

Good idea. Haven't thought about mikrocontroller.net, I even have a login there for quite a long time.

I am open for shops with a good reputation

Apple didn't design the PowerPC or make custom variances. Motorola and IBM did it. Especially Altivec was added by Motorola, and IBM didn't like to add it to their PowerPC CPUs when Apple asked for help, when Motorola had the 500 MHz glitch bug back in the day.

There is a nice coverage on this topic at https://www.youtube.com/watch?v=Tld91M_bcEI (Why the Original Apple Silicon Failed)

FTP, Email, Webspace and newsgroups access was the norm. good old times at uunet ...

I used a similar budget and build something like this:

7x RTX 3060 - 12 GB which results in 84GB Vram AMD Ryzen 5 - 5500GT with 32GB Ram

All in a 19-inch rack with a nice cooling solution and a beefy power supply.

My costs? 1300 Euro, but yeah, I sourced my parts on ebay / second hand.

(Added some 3d printed parts into the mix: https://www.printables.com/model/1142963-inter-tech-and-gene... https://www.printables.com/model/1142973-120mm-5mm-rised-noc... https://www.printables.com/model/1142962-cable-management-fu... if you think about building something similar)

My power consumption is below 500 Watt at the wall, when using LLLMs,since I did some optimizations:

* Worked on power optimizations and after many weeks of benchmarking, the sweet spot on the RTX3060 12GB cards is a 105 Watt limit

* Created Patches for Ollama ( https://github.com/ollama/ollama/pull/10678) to group models to exactly memory allocation instead of spreading over all available GPUs (This also reduces the VRAM overhead)

* ensured that ASPM is used on all relevant PCI components (Powertop is your friend)

It's not all shiny:

* I still use PCIe3 X1 for most of the cards, which limits their capability, but all I found so far (PCIe Gen4 x4 extender and bifurcation/special PCIE routers) are just too expensive to be used on such low powered cards

* Due to the slow PCIe bandwidth, the performance drops significantly

* Max VRAM per GPU is king. If you split up a model over several cards, the RAM allocation overhead is huge! (See Examples in my ollama patch about). I would rather use 3x 48GB instead of 7x 12G.

* Some RTX 3060 12GB Cards do idle at 11-15 Watt, which is unacceptable. Good BIOSes like the one from Gigabyte (Windforce xxx) do idle at 3 Watt, which is a huge difference when you use 7 or more cards. These BIOSes can be patched, but this can be risky

All in all, this server idles at 90-100Watt currently, which is perfect as a central service for my tinkerings and my family usage.

Great info in this post with some uncommon questions answered. I have a 3060 with unimpressive idle power consumption, interesting that it varies so much.

I know it would increase the idle power consumption, but have you considered a server platform instead of Ryzen to get more lanes?

Even so, you could probably get at least 4x for 4 cards without getting to crazy. 2 m.2 -> pcie adapters, the main GPU slot and the fairly common 4x wired secondary slot.

Splitting the main 16x GPU slot is possible but whenever I looked into this I kind of found the same thing you did. In addition to being a cabling/mounting nightmare the necessary hardware started to eat up enough total system cost that just ponying up for a 3090 started to make more sense.

Impressive! What kind of motherboard do you use to host 7 GPUs?

My 3060 idles sometimes at 19 watt, only sleep and wakeup of the machine helps.

If you want to see the included images, jump back to 2022: https://web.archive.org/web/20221218011802/https://akapugs.b...

I am a German native speaker as well. That is horrible idea!

A simple explanation can be seen here: https://www.youtube.com/watch?v=7j_NE6Pjv-E