The power of LLama - Part 1: The Brain, the Engine, and Your First Llama on Ollama

This is the first weekly post in a series on running large language models locally. Today is the foundation: how the brain works, why the file and the software are different things, and how to get your first model talking. Do not forget to check the series introductory post.

TL;DR. Every LLM is created using the transformer architecture. The model is the brain built upon it. But this brain does not execute anything. By itself, it is just a bunch of numbers (aka the weights). It needs an engine to run. The quickest way to start is Ollama: install it, run one command, and you have a chatbot on your own hardware in minutes.

The (not so) Ancient History

For decades, computers have been surprisingly bad at language.

Before transformers, most language models were built using architectures called Recurrent Neural Networks (RNNs) and later Long Short-Term Memory (LSTM) networks. These models processed text one word at a time, carrying a hidden "memory" from one step to the next. It worked well for short text, but they struggle to remember information from the beginning of the text.

Think of those as someone who forgets things you told them 1 minute ago. They can greet and talk about mundane things, but struggle when the texts got longer.

Everything changed in 2017, when researchers at Google published a paper called Attention Is All You Need. That paper introduced a new neural network architecture -- the Transformer architecture -- that changed artificial intelligence almost overnight.

But what the Heck is a Transformer?

A transformer is a kind of neural network designed to process sequences of information.

Think about it: when you read a sentence, you don't interpret each word in isolation. The meaning of the word bank depends on whether you're talking about a river, money, or someone banking an airplane. Humans naturally look at the surrounding context before deciding what a word means.

Note: For simplicity I'm using "words", but internally models work with tokens, which are pieces of information. We'll cover tokens in more detail later in the series.

Transformers try to do something similar. Instead of processing words one after another, it looks at all the words in the context at once. For every word, it asks:

"Which other words should I pay attention to in order to understand this one?"

This mechanism is called attention.

Take a look at the example above, which shows a model reading the sentence: "The cat jumped onto the table because it was scared". In this case, the word it should be related to (or pay attention to) cat, not table. The attention mechanism learns these implicit relationships during training. During training the model also captures grammar, long-range dependencies, and subtle meaning without anyone explicitly programming language rules.

That ability to connect information across an entire document -- or, in other words, to remember, is the main reason transformers outperformed the architectures that came before them.

But there is (always) a Catch

While incredibly powerful, the attention mechanism comes with a major drawback.

Imagine a document containing N words. For every word, the model compares it with every other word to determine what deserves attention. That means the number of comparisons grows roughly as N².

Or, in layman terms:

Every time you double the context length, you do about four times as much work.

The table below shows exactly how quickly this computational workload explodes as your text grows:

This quadratic scaling became one of the biggest engineering challenges in modern AI. As people wanted models capable of reading books, entire codebases, simply making the context window larger became increasingly impractical.

Researchers have developed numerous techniques to reduce this cost. Innovations such as sparse attention, sliding windows, FlashAttention, among others, are the main reasons today's models can handle context windows of hundreds of thousands—or even millions—of tokens.

But even with these tricks, the underlying math doesn't change: the quadratic pressure is always there, waiting to test the limits of your hardware.

The Model: A Giant Brain Made of Numbers

It is important to keep in mind that the transformer is the architecture. It defines how information flows through the network and how the words interact with each other. By itself, the Transformer architecture doesn't know English, Python, or anything else.

The actual intelligence comes later, during a massive training process. Billions of knobs are carefully adjusted until the system implicitly learns language structure, facts, reasoning patterns, and coding logic.

We call these knobs parameters, and the entire completed package of them is the model.

But what parameters (really) are?

A parameter is neither a sentence, a rule, nor a fact.

It is a number. Just a number.

Modern language models contain billions of these numbers. A 7-billion parameter model literally stores seven billion learned values.

Individually, a single parameter is meaningless. One number doesn't represent "Paris is the capital of France" or "how to write Java." But together, billions of them form a mathematical system that has learned patterns from enormous amounts of text.

How are parameters created?

Initially, the parameters start as random numbers.

During training, the model is shown trillions of words from books, articles, source code, conversations, research papers, and many other kinds of text. The model repeatedly tries to predict the next word.

When the model gets something wrong (which at the beginning is almost always), an algorithm slightly adjusts those billions of parameters.

This process happens again ... and again ... and again.

Until these tiny adjustments slowly teach the model grammar, facts, reasoning patterns, programming languages, writing styles, and even surprisingly abstract concepts.

Nothing is explicitly programmed, knowledge arises from statistical patterns over an enormous amount of data.

The Inference Engine: Bringing the Brain to Life

At this point, we have a transformer architecture and a model containing billions of learned parameters.

You might expect that double-clicking the model file would launch a chatbot.

Spoiler alert: It doesn't.

The model is nothing more than data. It contains all the knowledge, but it has no idea how to execute itself. Just like a brain preserved in a jar wouldn't suddenly start having conversations, a model file needs a body, something that can bring it to life.

That something is called an inference engine, the software responsible for running a language model.

Video games provide a useful analogy:

While game files contain all the assets: maps, textures, sounds, characters, and rules. On their own, they don't do anything. You need a game engine like Unreal Engine or Unity to load those assets, simulate the world, and render each frame.

Language models work in much the same way.

The model file contains the learned parameters, while the inference engine reads those parameters, executes the transformer, and generates one word after another until a complete response emerges.

Examples of Inference Engines

There are many inference engines, each optimized for different hardware and workloads.

Some of the most popular are:

llama.cpp: The de-facto standard for running quantized models on CPUs and consumer GPUs.
vLLM: Designed for high-throughput inference servers and APIs.
TensorRT-LLM: NVIDIA's high-performance inference engine for datacenter GPUs.

But what about Ollama?

You may have also heard of Ollama.

Strictly speaking, it is not an inference engine. Instead, it's a user-friendly model management system built on top of llama.cpp. It handles downloading and managing models, exposing APIs, and provides a simple command-line interface, while relying on llama.cpp to perform the actual inference.

That's why many people recommend Ollama for beginners: it hides much of the complexity while still providing acceptable local inference speed.

Let's proceed with installing ollama and running our first chat bot.

Installing Ollama

Go to ollama.com and grab the installer for your platform. It runs on macOS, Linux, and Windows.

On Linux you can do it in one line:

curl -fsSL https://ollama.com/install.sh | sh

On macOS and Windows, download the app and open it. Ollama installs a background service and a command-line tool called ollama.

Downloading a model

In this tutorial, we will use the llama 3.2 1b model. To download it run the following command in the terminal.

ollama pull llama3.2:1b

Important: The model's size is about 1 GB, so it might take a while for it to download.

If you are familiar with Docker, you will notice that the ollama command is very similar. As a matter of fact, Ollama try to follow the same paradigm (i.e. a model is treated like an image).

Send your first message

In the terminal type the command below.

ollama run llama3.2:1b

After a while, you'll be greet with a prompt. Try something like:

>>> In the context of AI, what is a transformer?

Congratulations

You're now talking to a transformer running entirely on your own hardware, no API key, no cloud, no data leaving your machine.

A few commands worth knowing:

• Type /bye to exit the chat.
• Run ollama list to see the models you've downloaded.
• Run ollama ps to see what's currently loaded in memory.
• Pull a different model with ollama pull mistral.

If the model runs slowly, you are either running on CPU instead of GPU, or you've picked a model too big for your memory.

Start small. A 1B to 8B model is plenty for learning.

Where This Goes Next

Next week, we'll delve into creating a chat bot with ollama and trying to get an agentic workflow to run locally.

Spoiler alert: Ollama will not take us very far. But it will help us lay the foundation to more complex use cases.
Update (07/06/2026): The second part of the series has been published.

FAQ

What is a transformer in simple terms?

A transformer is the architecture behind every modern LLM. In simple terms, it reads a sequence of text and predicts what comes next, one word at a time. Its key mechanism is attention, which lets each word look back at earlier words and decide which ones are relevant.

Does my LLM learns from my conversation?

No. A standard LLM is a static, read-only system. When you use it you are simply running a calculation based on weights that were "frozen" when the model finished its training. There is a process called fine-tunning that allows the model to be further trained, but this process does not run during inference.

What is the difference between a model and an engine in local LLMs?

A model is a file containing the learned weights of a neural network. An engine is the software that loads that file and runs it to generate text. Examples of engines include llama.cpp and vllm. The model determines how smart the output is; the engine determines how to efficiently run it.

What is the difference between Ollama and llama.cpp?

Think of llama.cpp as the "engine". It contains the raw, high-performance logic required to run models on local hardware. Ollama, in the other hand, is the " chassis" wrapped around the engine. It simplifies the experience by handling model downloading, library management, and API hosting.