Background

We’ve been building an app ( Automatic Life Tracker ) for year and half now. For the first half year, we used Github Actions macOS runners to run the CI for iOS part of the app. That was both slow and expensive, as unfortunately it seems that the cost of the mac runners are quite crazy. I think we were paying few hundred dollars per month for handful of developers’ rare Mac builds’ CI.

Hosted service is not usually the cheapest (or best)

Our office has symmetric 1 gigabit fiber connectivity. That means that network is not a problem, and electricity is included in the rent. So having a ‘server at office’ seemed to make this much simpler.

So we just bought 96GB Mac Studio with M3 Ultra CPU. It cost perhaps half years worth of Mac CI (at the time, and the cost was gradually creeping up every month).

How our own Github Actions runners work on our Mac?

Containerizing Linux GHA runner is quite easy. Mac on the other hand turned out to be bit more work. While Toolset to build, run and manage macOS and Linux VMs exists, and is free for our minuscule scale, just having a VM didn’t mean having GHA that performs well on it.

So I quickly vibe coded in my copious spare time a tool - fingon/tarty: Vibe coded tart VM manager - which solves exactly my problem. I am not going to write too many details about the solution, but basic idea is that you have

  • base tart image (which is static and manually created - I do it every time we really want to change the Xcode version running in CI)
  • daily build image (which tarty builds in a VM by cloning base image, running script, and if it succeeds, using that as builds for that day)
  • two runners that are started every time daily build image changes

Note that I chose not to go for ephemeral runners, or particularly clever caching, but instead ripped out all caching from the workflows, and run the basic step during the daily build (which runs at night time) and populates caches within the daily build image ( homebrew update, bump Go toolchain+packages and Swift packages to match what is in main branch at the time ).

If the daily image build succeeds, then the runners which actually run the workflows are restarted when they do not have jobs ongoing, and stuff just happens.

Performance

The CI works a lot faster than it did when we used hosted runner. The speed difference is in 2-10x range (there seems to be a lot of variability in the hosted runners’ speed). And it costs us almost nothing, and has been quite hassle free. Of course, if the machine blows up, we get to set it up, but the tooling is relatively straightforward (although tedious) to set up again.

Lessons learned

During this exercise I found out that macOS is limited to running at most 2 macOS VMs. So our 96GB machine was overkill in the end, although we planned also to use it for playing with local models, but that usage hasn’t been that significant.

So it is really better to buy handful of ‘big enough’ Mac Minis than one big Studio, if you want to maximize the number of VMs you can use. This is why tarty turns off one github actions runner to do the daily build job, and it feels like unnecessary hack.