This is actually bit unscheduled, but as I feel the content will not age gracefully, I chose to write what I experienced with my different LLM model experiments recently.
In general, I value local compute over cloud compute, and especially if things of private nature are involved. However, sometimes local compute simply does not work.
I have dabbled with different models as main model for the Hermes Agent:
- GPT 5.6 Sol (using ChatGPT Plus subscription)
- GPT 5.6 Luna (-//-)
- Qwen 3.6 35B A3B q4 mtp
and here is a brief write-up about it. This picture unfortunately spoils the outcome:
GPT 5.6 Sol experience (few days)
I used it for a couple of days, and sometimes the (non token paid) version seemed bit slow, but it is not clear if the expense is really justified, although subscription seems to cover it. The answers were on the mark.
It was good at using tools I gave it, and also creatively worked around some restrictions I had (because at the time my Hermes setup was bit misconfigured to be too secure - as it is running in a rootless container with only public data, I am not that paranoid about what happens there).
GPT 5.6 Luna experience (two weeks)
I switched to Luna after Sol, and for day to day use, it does not seem to be substantially different from Sol (given all the tooling Hermes provides it). At max thinking effort, it is even useful for coding, although only if given design from elsewhere.. and at fast/priority mode, it is substantially faster than Sol so the interactive experience feels nicer.
As a daily driver it has been briilliant. But I always want to try the options, so..
Qwen 3.6 35B A3B q4 mtp (day or two)
As I adore local models, and the 35B q4 is ’the best I can do’ with my local 32GB Mac, I wanted to check how it works out in practice. with oMLX prompt caching, it was actually surprisingly fast at starting to respond (sometimes even faster than OpenAI - thanks to the machine being dedicated to me), and even token generation, while slow, was sort of bearable. But it was simply stupid.
Pretty much all interactions I had with Luna were much, much worse with Qwen. The crowning one was following one yesterday:
It crunched at it for probably a minute or two, it tried to use previous year (now being 2026), and then just finally gave up. I think the training data must have a lot of stuff from 2025, as it switched to that and started querying only that going forward.
For comparison, I switched model in a new session, and then asked Luna the same question, and it came up with the answer quite fast (and slightly more correctly, as of yesterday):
Interesting thing is that while the event date is correct (it found it from the Apple event calendar), it guessed about the content - to be same as that of the 2025 event.
Conclusion
For now, I will not put any really personal data to be available to my agent, and use it through OpenAI’s GPT 5.6 Luna Max on priority/fast mode. I will re-evaluate this with future models at times, but at least this time around closed source simply worked better for me.
For OpenViking use of memory handling and embeddings, I am quite happy with Gemma 4 series (the 25B MoE, E4B/E2B), and small Qwen embedding model. Those are already used locally, but for the actual chat where (some) intelligence is needed, Qwen isn’t there for me at least yet.
There is some content still left from the TODO list in Me and my agent - part 2 (of N), so we shall see what comes out next.