Oh my god, it is full of containers!

This is probably the last post on the home infra, at least one that I thought about from the start. I will perhaps write more if I come up with something, but probably the future ones will be less about the overall software stack and more about e.g. Lixie or some other hobby project that is related to the infra, and less about the composition of the infrastructure. Container types, recap - and their lifetime Types of containers: As noted in an earlier post about the architecture, the home router is running plain Debian stable, with 3 types of containers, all provisioned using pyinfra so I do not mess with them on the host itself: ...

18.6.2024 · 7 min · 1283 words · Markus Stenberg

Testing in the wonderful world of infrastructure as code

I was planning to write about my home infrastructure (I have at least one post in mind already), but as it is fresh in my mind I decided to write about the state of IaC code, or more specifically, testing IaC code (or lack of it). Testing is the foundation of most workflows I have recently spent quite a bit of time setting up Github Actions-based workflows both for my personal and professional projects. Ultimately, that stuff works quite well: ...

14.6.2024 · 7 min · 1452 words · Markus Stenberg

Messaging (, alerting) at home (and outside)

I have used a lot of different messaging tools over the years, and some have stuck, others have not. Back in the days: IRC (90s, early 2000s) There was some sporadic use of e.g. talk for point-to-point chatting among users of UNIX hosts. I used to use IRC a lot, starting in the 90s. There were various subject matter channels of interest, and I was quite keen about it at the time. However, its usage petered off gradually. I think my last serious use of it was with erc (Emacs client) in conjunction with bitlbee (which provided bridging to some other IM things that were being used at the time). ...

7.6.2024 · 5 min · 875 words · Markus Stenberg

IPv6 only life must be great, right?

Two weeks ago I experienced IPv6-less life for a week thanks to my ISP, but my ISP provided me this week an experience of what IPv6-only life looks like. The start of the IPv6-only experience (bit before 6am) I wake up usually crazily early in the morning (by nerd standards, at any rate), if it is light out there. And in Finland, at summer, it gets light quite early so I must have woken up a bit before 6 am or so. I usually start working (or tinkering with some hobby project) even before breakfast, so I hopped on my computer. ...

31.5.2024 · 5 min · 944 words · Markus Stenberg

You can never have too many backups

I have made a point of backing up about anything I do for a long time. Why? (Especially automated) backup systems will eventually be helpful no matter what you do. Whether it is changing or deleting some file that you did not have under version control (oops), hardware going bad, software going bad, hardware getting lost.. I have had all of them, but I have still yet to lose significant amount of data once I started taking backups consistently. ...

29.5.2024 · 8 min · 1500 words · Markus Stenberg

Observability at home

During the new home router infra exercise, I also chose to set up as reasonable as possible (lightweight) observability stack for my use. The holy trinity is defined to be metrics, logs, and traces, but as I do not really do much that requires tracing I focused initially on the first two. This exercise occurred mostly in February. Software choices Visualization I chose to go with Grafana. I am very familiar with it, and there seems to be an ecosystem of third-party tools which replicate some of what Aiven had (e.g. dashboard backup/restore tooling, dashboard rewriter I wrote fingon/gg-grafana: Grafana dashboard sanitizer as I could not find something similar in the open world, and so on). ...

24.5.2024 · 6 min · 1202 words · Markus Stenberg

It is 2024, and IPv6 is an optional feature

Based on my metrics and logs, around 1 am on the 15th of May, 2023, my (cable modem-based) internet provider just stopped providing IPv6 addressing to me. According to the logs, the first indication of trouble was 03:34 or so, when odhcpd started logging messages about no default route being present (IPv6 one) and my home network hosts lost IPv6 connectivity subsequently. Then I looked at stored netflows: mstenber@fw ~>nfdump -R /flows/2024/05/ -t 2024/05/15.00:00:00 ipv6 2>&1 | egrep -v '(fe80|fd42|ff02):' Date first seen Duration Proto Src IP Addr:Port Dst IP Addr:Port Packets Bytes Flows ... 2024-05-15 01:20:45.525 00:00:00.000 TCP 2001:14..79:115c.60513 -> 2a01:b7..01::2:4.443 1 84 1 The last failed attempt to even use IPv6 was apparently around 1:20. So it most likely broke two, three hours earlier than what was shown in the logs, and just the default route lifetime expired at the point when odhcpd started being unhappy. ...

22.5.2024 · 3 min · 579 words · Markus Stenberg

Securing Terraform, or, OpenTofu for the win?

Disclaimer: Sorry, still not related to my home network infrastructure, although I have one or two posts I want to write about later.. I have been thinking recently about how to deal with some personal resources I have in the cloud - mainly in Oracle Cloud thanks to their generous free tier. So far I’ve dealt with them manually doing click-ops in their GUI, but I have been trying to up my personal infra game so I started to think about how to make this not manual. ...

17.5.2024 · 4 min · 851 words · Markus Stenberg

Initial software setup of 2023 home router

As discussed in earlier post, the hardware choice was ultimately simple given the requirements. Even top-level software choice (Linux running Debian) did not seem particularly hard, but then started the hard part: coming up with reasonable design for what to run there. The requirements (2023 November) As I had 2,5GB ethernet ports (finally), my main Mac and NAS had to be directly connected to the box I used to have separate 2,5GB point-to-point between the two, but using simply one port in each instead of two made management slightly simpler and less error prone ’Internet-facing’ part would need to be relatively static and not touching Debian directly at all (instead, something bit more hardened would be the first ingress point) Both rootful and rootless containers would be nice to have The more containers the merrier - at this point I foresaw about half dozen useful containers something firewall/router-y facing internet DNS ad blocker (e.g. adguard, or pi-hole) reverse proxy to handle HTTPS (from both inside home, as well as from the internet) using proper certificates (e.g. caddy, traefik) home assistant (for home visibility and automation) jellyfin (for media) Infrastructure as Code (IaC from now on) What I did not choose to use Proxmox ( https://www.proxmox.com/en/ ) seems like the toy of the hour in the enthusiast circles. I am not quite sure why; perhaps it is the GUI. But the combination of GUI driven material (mostly), as well as not particularly good IaC (no first-party Terraform support - at the time there is third-party one in Proxmox Provider - Terraform Registry) did not make it particularly tempting for me. And besides, I was mostly planning to use pyinfra anyway. ...

3.5.2024 · 5 min · 854 words · Markus Stenberg

Playing with local LLMs

I have been somewhat interested about LLM performance for years, and it used to be that playing with them was quite painful (e.g. conda ecosystem in general sucks and it used to be that GPU was mandatory), but now with ollama ( https://ollama.com/ ) they’re quite trivial to benchmark across different devices without need for setting up complex stack. So this morning I indulged.. I have not yet gotten around to checking the numbers on a real GPU card, but here’s what I found out at my home (without starting gaming PC). ...

25.4.2024 · 3 min · 634 words · Markus Stenberg