I tinkered bit with Google’s new gemma2 model on my 32GB RAM M1 Pro. It seems so far quite useful, although I have dabbled with it only day or two so far. Here’s summary from some of the things I tested with it. Benchmarking Using the script from earlier iterations: for MODEL in gemma2:27b-instruct-q5_K_M gemma2:27b \ gemma2:9b-instruct-fp16 gemma2:9b-instruct-q8_0 gemma2 \ llama3:8b-instruct-q8_0 llama3 do echo ${MODEL}: ollama run $MODEL --verbose 'Why is sky blue?' 2>&1 \ | grep -E '^(load duration|eval rate)' echo done with the following models: ...

This is a really brief follow-up on the earlier local llm performance benchmarking post. Nvidia RTX 3080 Today I decided to check out also performance of RTX 3080. Now that Windows beta of ollama is available, testing it out was straightforward. As it turns out, it was almost exactly double the speed of Apple Silicon hardware, e.g. llama2:7B model produced around 80 tokens per second, but model load duration was a bit slower (3-4s -> 4,9s). ...

This is probably the last post on the home infra, at least one that I thought about from the start. I will perhaps write more if I come up with something, but probably the future ones will be less about the overall software stack and more about e.g. Lixie or some other hobby project that is related to the infra, and less about the composition of the infrastructure. Container types, recap - and their lifetime Types of containers: As noted in an earlier post about the architecture, the home router is running plain Debian stable, with 3 types of containers, all provisioned using pyinfra so I do not mess with them on the host itself: ...

I was planning to write about my home infrastructure (I have at least one post in mind already), but as it is fresh in my mind I decided to write about the state of IaC code, or more specifically, testing IaC code (or lack of it). Testing is the foundation of most workflows I have recently spent quite a bit of time setting up Github Actions-based workflows both for my personal and professional projects. Ultimately, that stuff works quite well: ...

I have used a lot of different messaging tools over the years, and some have stuck, others have not. Back in the days: IRC (90s, early 2000s) There was some sporadic use of e.g. talk for point-to-point chatting among users of UNIX hosts. I used to use IRC a lot, starting in the 90s. There were various subject matter channels of interest, and I was quite keen about it at the time. However, its usage petered off gradually. I think my last serious use of it was with erc (Emacs client) in conjunction with bitlbee (which provided bridging to some other IM things that were being used at the time). ...

Two weeks ago I experienced IPv6-less life for a week thanks to my ISP, but my ISP provided me this week an experience of what IPv6-only life looks like. The start of the IPv6-only experience (bit before 6am) I wake up usually crazily early in the morning (by nerd standards, at any rate), if it is light out there. And in Finland, at summer, it gets light quite early so I must have woken up a bit before 6 am or so. I usually start working (or tinkering with some hobby project) even before breakfast, so I hopped on my computer. ...

I have made a point of backing up about anything I do for a long time. Why? (Especially automated) backup systems will eventually be helpful no matter what you do. Whether it is changing or deleting some file that you did not have under version control (oops), hardware going bad, software going bad, hardware getting lost.. I have had all of them, but I have still yet to lose significant amount of data once I started taking backups consistently. ...

During the new home router infra exercise, I also chose to set up as reasonable as possible (lightweight) observability stack for my use. The holy trinity is defined to be metrics, logs, and traces, but as I do not really do much that requires tracing I focused initially on the first two. This exercise occurred mostly in February. Software choices Visualization I chose to go with Grafana. I am very familiar with it, and there seems to be an ecosystem of third-party tools which replicate some of what Aiven had (e.g. dashboard backup/restore tooling, dashboard rewriter I wrote fingon/gg-grafana: Grafana dashboard sanitizer as I could not find something similar in the open world, and so on). ...

Based on my metrics and logs, around 1 am on the 15th of May, 2023, my (cable modem-based) internet provider just stopped providing IPv6 addressing to me. According to the logs, the first indication of trouble was 03:34 or so, when odhcpd started logging messages about no default route being present (IPv6 one) and my home network hosts lost IPv6 connectivity subsequently. Then I looked at stored netflows: mstenber@fw ~>nfdump -R /flows/2024/05/ -t 2024/05/15.00:00:00 ipv6 2>&1 | egrep -v '(fe80|fd42|ff02):' Date first seen Duration Proto Src IP Addr:Port Dst IP Addr:Port Packets Bytes Flows ... 2024-05-15 01:20:45.525 00:00:00.000 TCP 2001:14..79:115c.60513 -> 2a01:b7..01::2:4.443 1 84 1 The last failed attempt to even use IPv6 was apparently around 1:20. So it most likely broke two, three hours earlier than what was shown in the logs, and just the default route lifetime expired at the point when odhcpd started being unhappy. ...

Disclaimer: Sorry, still not related to my home network infrastructure, although I have one or two posts I want to write about later.. I have been thinking recently about how to deal with some personal resources I have in the cloud - mainly in Oracle Cloud thanks to their generous free tier. So far I’ve dealt with them manually doing click-ops in their GUI, but I have been trying to up my personal infra game so I started to think about how to make this not manual. ...