Add a PoE switch and PoE splitters for IP-KVMs that don't already have PoE-in?

I went with Sipeed NanoKVM (PCIe) units across my homelab as well and I've also been happy with them. For a while it's been the best value option (not to mention the most consistently available option) cf. GL.iNet, PiKVM, and JetKVM. The PoE versions are great in a rack and the integrated ATX control is fully-featured (including the little power switch icon in the web UI turning green when the system is powered on). I set up an isolated OoBM VLAN with no Internet access and any switch ports assigned to it are isolated by default as well.

Have you played around with OpenBMC or any of the open-sourced BMC firmware alternatives?

ETA: specifically on older server/workstation boards instead of the stock firwmare

I've been interested but the idea of SPI flasher recovery has always meant I've not bothered actually trying. Maybe I will finally on one of these old boards I have lying around..

Same here; trying to get OpenBMC flashed onto older motherboards can be a rabbit-hole. I generally throw an IP KVM on top if I really need remote control and can't settle on just SSH and a remote power cycle if that fails.

The memory controller is integrated into the CPU, so the motherboard chipset is irrelevant. There are some OEM-only v3/v4 parts with dual memory controllers, but the E5-2620 v4 is not one of them.

Ooh weird!

There are some OEM-only v3/v4 parts with dual memory controllers (because of a RAM supply crunch at the time, funnily enough), but the E5-2620 v4 is not one of them. The classic example is the very popular 12-core E5-2678 v3.

> Nowadays I'm not actually sure if DACs are cheaper than fibre now

Yep still are, although fiber transceivers are surprisingly affordable, and of course you can usually re-use jumpers when you upgrade transceivers so TCO over a couple generations might be comparable.

> I'm getting a really bad taste in my mouth for 10GBASE-T RJ45 SFP+ modules mostly due to the god awful heat they pump out

It's truly bananas especially in a homelab environment. Nearly every time I've thought to myself "oh I can just do a copper gbic for this" it's been the wrong thing to do.

If you start doing bonded links with DACs or if you have a bunch of servers, the cable management situation gets ugly in a hurry, and the usual solutions like patch panels and keystones aren't applicable. Source: my basement

That's my home lab, it's ugly and I am starting to use MTP breakouts instead because those DACs get in the way so much.

> the usual solutions like patch panels and keystones aren't applicable

Why not ?

You can't punch down twinax and the connectors are too big for keystones.

I've started buying E810s even for 10G links. PCIe Gen4, lower power draw, RDMA support, generally backward-compatible with SFP+ DACs and transceivers, and relatively inexpensive. Not nearly as dirt-cheap as the X520s but not crazy expensive (last I looked, at least). As I gradually replace switches over the next few years I can start taking advantage of 25G.

This is the way. The 10G cards are ancient and hog all the PCIe lanes.

I’ve had the best luck with Mellanox ConnectX 4 or 5 cards. The 5 can happily run 25G on a modern lane constrained system.

I've seen lots of pretty terrible experiences with the i40e and newer Intel drivers.

For newer NICs than the X520s I'd probably grab a Connect-X card.

They've been stable for me but I'd be lying if I said I'd only heard good things.

Looks like prices on the E810-XXVDA2 have come up since the last time I looked while prices on the ConnectX-6 Lx have come down, so that'd be a good option!

I use UniFi for most of my home network so It Just Works™, but I've thought about mixing in Mikrotik for e.g. the compute rack so I can play around with 100G+ links and more esoteric stuff like VXLAN.