Running language models on your own hardware, where the tradeoffs are unavoidable and the marketing claims fall away. Coverage spans small specialist models, quantization and its surprising effects at tiny parameter counts, GPU provisioning and VRAM math, throughput optimization, and the honest question of when self-hosting beats a cloud API. Expect hands-on reports from people who profiled the kernels, built the routers, and measured what actually happens in production.
small-language-models, open-weight-models, ai-model-comparison, local-llms, finops-ai
Forget the flagship hype. The biggest AI disruption is happening in fast, affordable models that handle everyday work.
quantization, local-llms, small-models, ai-development
We shipped a 1.5B router that produced valid JSON 97% of the time. One quantization step took it to two. Here is the day we spent finding out why — and why the rules everyone repeats about quantization were written for models ten times the size. It started with a
training-data, prompt-engineering, local-llms, ai-development
We set out to teach a small model how to route developer prompts. The data taught us that our taxonomy — and our sources — encoded a picture of developers that doesn't exist. It started with a quota. I wanted a small, fast model that could sit in front of
local-llms, gpu, self-hosting, ai-development
What happens when you throw 96 GB of VRAM at open-source models, optimize every last flag, profile CUDA kernels, redesign system prompts, and still end up reaching for a cloud API. It started with a simple premise: why pay per token when you have the hardware? I have four