Running big AI models
on my own hardware.
This is my public log of chasing open-weight language models from my desk — what works, what melts the fan, and what surprised me. Everything here is tested, not theorised, and every lesson is turned into a tip you can use the same day.
Why bother going local?
Because "local" turned out to be a superpower, not a limitation. These are the reasons I never went back to cloud-only.
Privacy by default
Contracts, code, and diary entries never leave the machine. No telemetry, no training on your prompts, no "we may use your data" fine print.
Free at the margin
After the hardware, every token is free. Heavy use — summarising a whole codebase, churning drafts — costs only electricity.
Works offline
Planes, trains, dead zones, outages. The model doesn't care. Mine has been surprisingly useful on flights.
Totally tinkerable
Quantisation, context windows, sampling, system prompts — every knob is yours. Cloud APIs hide most of them.
Rapid iteration
No rate limits, no metered anxiety. You can prompt a local model 50 times in a row to find the phrasing that works.
Supporting open research
Running open models means open science wins. Llama, Qwen, Mistral, Gemma — local adoption is how these ecosystems stay healthy.
Latest from the lab
Recent entries from the journey log.
📱 Let's see what you can do
A paragraph of clear prompting, a few hours of collaborating — and an iOS app ready to ship.
🍎 Latest silicon, latest models
The noisy GPU rig retires to the office; a whisper-quiet Mac mini M6 takes over at home.
🖥️ Sourced a GPU — now the fun really begins
Pointing local models at years of operational data — and why keeping it local is the point.
🏗️ Things I've built
From a fruit machine and a shop till to four arcade games and a quick IQ test — seven builds and counting, on the Projects page.