I have been running Kubernetes at home from October 2024 onward. That exercise was single-node though, using (relatively small) part of the Frankenrouter resources. This is about next Kubernetes iteration.. or its hardware choice.
Why I did not want to stick with the kind setup forever?
Frankenrouter hardware (Intel N305) at least officially supports only 32GB of RAM. In addition to OpenWrt LXC container, and some native Debian processes, it is packing about 49 containers at the time of writing (give or take few, this Grafana thing is only an approximation based on unique images on podman side and pods on Kubernetes side):
While the CPUs were and still are fine (only Frigate NVR uses nontrivial amount of CPU), the memory usage started to dip below half every now and then, and especially when running ollama with some tiny models it was starting to get dangerously low. Due to that, the hardware part of the exercise had actually started two months before I write this.
Frankenrouter was also doing bit too many things even for me in a single box. ‘The crux’ (last week) was when after upgrading kernel(?) it actually hard crashed once. It is still unclear why (I did not bother attaching display to the device, and due to volatile systemd journald, the logs if any are gone in case of hard crash if they were not persisted to my log infra), but that prompted the final decision to start migrating towards bit more distributed setup at home.
Enter high availability home Kubernetes
Frankenrouter is one (relatively) low powered box with 32GB RAM, 2TB of NVME SSD and some CPU cores. I wanted to get couple of even less powerful nodes for Kubernetes use. You need 3 for HA (due to etcd requiring that number of nodes to survive loss of one).
I first thought of getting 3 nodes, but then I realized that Frankenrouter can be silent partner in the new Kubernetes too, providing just etcd services but (due to taint) not allowing scheduling of any workloads within.
So I went looking for 2 reasonable nodes to add to my home infrastructure. I considered Raspberry Pis, but the cost-performance ratio felt too bad to me - you really need lots of extra stuff besides the base device (nice case, some way to attach proper SSD storage) and the pricing starts to approach quickly that of mini PCs. The more recent Raspberry Pi variants are not THAT low power either, so due to that I was not that interested in using them.
MeLE Quieter 4C - Fanless Mini PC
‘The one’ that was found by googling for fanless (Intel) N100s is this: MeLE Quieter 4C - Fanless Mini PC N100/N150. I opted for 512GB SSD, 32GB RAM versions (because you can never have too much RAM, and the SSD cost was not that bad either). They were bit over 300€ apiece when I ordered them in April from Amazon. I guess in hindsight modern Mac Minis would have been perhaps better but the price is ~3x for 32GB RAM configuration and that is what I mainly cared about at the time.
I got the devices sometime mid April. They were quite small - the envelope beneath them is A4 sized:
Here is one unboxed (Airpods Pro case for scale):
I took backup of the Windows installation on one of the devices (just in case), and after that I promptly installed Linux on top of it.
After lots of playing with the software (follow-up post), the cluster was finally installed behind our TV:
This is partially to keep my office room cool enough (I do not need the extra heat), and partially to keep the WAF (Wife Acceptance Factor) high - you can barely know there is functioning Kubernetes cluster in that box.
Next stop will be about software they’re running..
Spoiler: fingon/sssmemvault: Shamir’s Secret Sharing based in-memory secret vault is involved.
Am I happy with them?
Their power usage is relatively low, and they manage without fan fine (in my not so heavy use, the CPU temperature seems to hover around 50 degrees or so). So so far I have been quite happy with them. Only sad thing is that I got version with Windows 11 - I am not sure if they were available without it on Amazon (at original supplier they seem to be), but as I rather not import from outside Europe, it was easier just to order it from Amazon.