The power of LLama

Disclaimer: This is an introduction to a series of articles I plan to write to introduce people to LLMs. Hopefully, by the end of the series, the reader should be knowledgeable enough to run their own LLMs locally.

Update (06/30/26): The first article of the series has been published. Check it out.
Update (07/06/26): The second article of the series has been published. Check it out.
Update (07/13/26): The third article of the series has been published. Check it out.
Update (07/20/26): The forth article of the series has been published. Check it out.

But why?

Despite all the talk about sovereign AI and privacy, there is a more practical reason for you to run local LLMs (at least for part of your workload).

See the image above. This is how many tokens I've run through my setup in a single day of work. It might not seem like much, but take a look at the cached tokens count.

That's right. It is about 360 million tokens. In a single day.

If I were running Claude Sonnet (at current 06/2026 pricing), the cache alone would cost me about 100 bucks. But it costs me nothing, besides eletricity.

Still, local LLMs are quite niche. They largely exist outside a handful of subreddits, YouTube channels, and Discord servers.

But, when a space is niche, it has candor.

People post their actual benchmark numbers. They complain when a model underperforms. They share the exact config that broke and the fix that worked. You won't find that polish-free truth in a vendor blog.

So if you've been waiting for local AI to "go mainstream" before you dig in, I'd argue you have it backwards. The best time to learn is while it's still weird and underexplored.

Moreover, I bet it will teach a set of skills that will be extremely valuable in the times to come.

The pandora box is opened

Most folks experience AI through a chat box. You type, something answers, and the whole machine behind it stays invisible. That works fine until you want control. When a python tutorial on how to call an OpenAI completions API doesn't suffice.

This is the moment you ask "can I run it myself".

And then you are met with a wall of jargon and knobs to tweak.

• What a model file even is, or why it comes in sizes like 7B, 13B, or 70B.
• What quantization means and why a Q4 version runs on a laptop while the full one needs a server.
• What the hell is a KV cache and why it also needs quantization
• What is perplexity and its relation to how well a model behave

Fear not for I'll address those in the future.

Where I Actually Go for Information

For now, I'd like to share with you where I get my information. This is not an exhaustive list, it's just what I think is enough to give a beginner a foothold.

1. News From the Trenches

For the raw, unfiltered pulse of what's happening, two subreddits do the heavy lifting:

• r/LocalLLaMA
• r/LocalLLM

This is where new model releases get dissected within hours. Someone downloads the latest open weights, runs it on their setup, and posts whether it's actually good or just hype. You'll see the wins and the failures side by side. By the time something comes up in LinkedIn, it is already last week's news.

A word to the wise: these places move fast and assume some context. Don't expect pleasantries and hand holding. Read them for a week before you post (it is fine to just lurk in the threads if you don't feel comfortable posting). Treat it like learning a language by immersion.

2. Tutorials and Recipes for Hardware You Own

Reading about it is one thing. Getting them running is another. For the hands-on side, Codacus is exceptional. The channel focuses on tutorials and recipes built around hardware regular people actually have and models that are actually useful.

This video is particularly noteworthy. It teaches how to host Qwen 35b a3b on a 6GB GPU. Of course, it won't win any speed record, but keep in mind that this isn't a toy model (it ranks better than Anthropic Haiku 4.5).

That framing is the whole point. Plenty of guides assume you've got enterprise gear sitting around. The useful ones meet you where you are, on the gear you've already paid for. Step-by-step instructions turn the black box back into something with knobs and switches you understand.

3. Meaningful Model Comparisons

Knowing a model exists is useless if you don't know whether it is worth its salt. TokenChaser (and the YouTube channel) provides some fun comparisons between models -- both open and proprietary.

This matters more than people realize. See, benchmarks are not meaningless, quite the contrary. But they usually don't tell the whole story.

An open weight model is less powerful than a frontier model. But that is not the crux of the question. The real question you should be asking is:

Can the open weight model keep up?

How to Take the First Steps

If I were starting over today, here's the order I'd follow:

1. Spend a few days lurking in r/LocalLLaMA and r/LocalLLM to absorb the vocabulary and see what people are excited about.
2. Pick one model that the comparisons suggest runs well on hardware similar to mine.
3. Follow a single tutorial start to finish instead of jumping between ten different ones.
4. Run something, anything, even a small model that disappoints you. The first working setup teaches more than a month of reading.

The goal isn't to build the perfect rig on day one. It's to crack open the black box just enough to lose your fear of it. Don't be afraid to crash and burn (for you will).

The Real Reason This Is Worth Your Time

I'll be honest about my bias. I think relying entirely on cloud AI is a slow trap. Prices change. Terms change. Models you depend on get nerfed to make the next big thing appear greater than it is. When you run locally, the model on your drive is yours. It doesn't change unless you change it.

That stability is rare, and it's the thing the "sovereign AI" talk gestures at without explaining. It is not a matter of national policy. It is a matter of having a model file, some hardware, a weekend, and knowledge.

The information is still scattered. But the doors are wide open, and the people inside are happy to help if you show up willing to learn. Start with the links above, run something this weekend, and you'll already be ahead of most of the people loudly debating AI online.

See you next week with the first part of the series of the articles. It will be on how LLMs work.

FAQ

What is a local LLM?

A local LLM is a large language model that runs entirely on your own computer or hardware instead of on a remote cloud server. You download the model file and run it yourself, which means your prompts and data never leave your machine and you don't have to pay per token or deal with recurring subscriptions.

Why are local LLMs still considered niche?

Local LLMs stay niche mostly because people see LLMs as a black box. Most people interact with AI through a chat interface, an agent, or an API and have no mental model of how they truly work. The path from curiosity to a running setup feels complicated, so few people cross over.

What hardware do I need to run a local LLM?

You need less than most people assume. A decent consumer grade computer with a gaming GPU can run small to mid-sized models. The exact requirement depends on the model size, which is why checking comparisons tied to real hardware before downloading is the smart move.

Where can I find reliable news about local LLMs?

The subreddits r/LocalLLaMA and r/LocalLLM are the best places for unfiltered, fast-moving news. New models get tested and reviewed by real users within hours of release, so you see honest results instead of marketing claims. Lurk for a while first to learn the vocabulary.

Are local models toys compared to cloud-based ones?

While less powerful, local models have come a long way (Qwen-35b-a3b being better than Haiku across the board). But the real question is: are they good enough for the task at hand?