The Day the AI Moved Into Your Browser

We built two features for AIpster, a Playground and a Post Companion, that run real language models entirely in your browser using WebGPU and WebLLM. Nothing gets sent to a server, there is no API key, and each use costs us zero. The lesson is bigger than the build: small-context AI work belongs on the device in front of you, not in someone else's data center, and the technology to ship it is ready right now.

This is a story about a small idea that quietly turned into a conviction. It started with something almost mundane. We wanted the AIpster site to feel less like a place you read and then close, and more like a place you can actually do something with.

Blog posts are one-way. You arrive, you read, you leave. We kept asking ourselves how to turn that into a conversation. How do you let a reader poke at an idea, argue with it, or get a model to chew on the very article in front of them?

The boring answer we refused to ship

For a while we circled the obvious answer, which is also the dull one. Wire up a chatbot to a cloud AI provider, drop a little bubble in the corner, and pay per message.

We didn't love it, and the reasons matter.

• Every curious reader becomes a line item on a bill.
• Your questions, the half-formed thoughts you'd type into a box, travel off to someone else's servers to be processed and, who knows, logged.

For a project whose whole personality is local AI, human control, and your data is yours, that felt like quietly betraying the thesis just to ship a feature.

Then a friend gave me a spark. Just an offhand comment: "You know you can run these models directly in the browser now, right?" No grand plan. A nudge in a direction I hadn't taken seriously. And that nudge is where the real journey began, because the only way to know if an idea is real is to build it and watch whether it falls apart in your hands.

Building the Playground

The first thing we built was a Playground. The premise sounds like a magic trick when you say it out loud.

A real language model, downloaded once into your browser, running entirely on your own graphics card, with nothing sent to any server. No API key. No account required to try it. You click a button, a model downloads to your machine, and then you're talking to an AI that lives on your hardware. You could pull the network cable out of the wall and keep chatting.

The tech underneath

The engine room is WebGPU, a relatively new capability that lets web pages tap into your GPU the way a native app would. We paired it with WebLLM, which knows how to run compressed models inside that sandbox.

We started small. Suspiciously small. A 360-million-parameter model, about the size of a couple of photos, light enough to load on a phone.

Honesty as the product

Here we made a deliberate choice that defines the AIpster voice. We didn't pretend the tiny model was a genius.

We wrote it an honest verdict. This thing is brilliant at tightening a sentence or shifting a tone, and it will lie to your face with total confidence the moment you ask it for a fact or an exact number. That honesty is the product. Anyone can give you a chatbot. We wanted to give you a chatbot and tell you exactly where it's bluffing.

From there it grew:

• A 1.5-billion-parameter model for members.
• Two different 3.8-billion models, including an uncensored variant we converted and tuned ourselves and published openly for the wider community.
• All the model weights hosted on our own infrastructure, not borrowed from someone else, so the experience belongs to us end to end.
• Streaming responses, a meter showing how full the model's memory is, an editable system prompt so you can watch the personality change, and controls for the nerds who want to turn the knobs.

None of it costs us a cent per use, because every single token is generated on the visitor's machine, not ours.

The Post Companion

Once the Playground proved the technology was solid, the original idea came back around. Now we knew how to do it right.

We built the Post Companion, a small chat panel that appears on blog posts and quietly loads the article you're reading into a local model's memory. You can ask it to summarize the post in three bullets, to find the strongest claim in it, or, my favorite because it's so on-brand, to tell you what the author got wrong or oversimplified.

Here's the elegant part. The article is already on your screen. It's already on your device. Feeding it to a model that also runs on your device means nothing leaves at all.

The thing people usually call RAG, retrieving documents and stuffing them into an AI's context, became trivial. There was only ever one document, and it was already in your hands. No embeddings. No search servers. No egress. No privacy compromise.

The feature we almost built the cloud-dependent, bill-generating way turned out better built locally. Cheaper, more private, and more honest.

Members-only, but not a toll booth

We made it members-only, which sounds like a paywall but is really the opposite. The model and the experience are the reason to sign up. It's a gift behind a free door, not a toll booth.

We also taught the site to be smart about hardware. If your browser can run the local AI, you get the Companion. If it can't, you get a gentler nudge instead. Each visitor gets the best experience their machine can actually deliver, and we never promise something the hardware can't keep.

What we actually learned

The build was fun. The realization underneath it is the part I keep coming back to.

First: the technology is ready. This is not a lab demo held together with duct tape. Models run in the browser, on consumer GPUs, fast enough to be genuinely useful, today. The gap between neat experiment and ship it on a real site closed while most people weren't looking.

Second: I think we just saw the future of a whole category of AI. Not every task. Training frontier models and answering questions that need the entire internet will live in big data centers for a long time yet.

But a huge slice of what we actually use AI for is small-context work. Talk to me about this document. Summarize this page. Help me with this one specific thing in front of me. That category doesn't need a billion-dollar cloud. It needs a competent small model and the device already sitting on your desk.

Once you see that, the consequences cascade.

• Privacy stops being a promise and becomes a fact. Not "we don't store your data," but "your data physically never left the room."
• The cloud AI bill disappears. Not reduced. Gone. The user's own hardware does the work, so a feature that gets ten million uses costs exactly the same as one that gets ten.

That breaks the most painful economics in this entire industry, where every successful AI feature punishes you with a bigger invoice. Here, success is free.

The part that opened my mind

The third thing is the one I can't stop thinking about. If a small model can run locally and reliably use tools, call a function, manipulate the page, do a calculation, drive a little interface, then we're not just talking about chatbots anymore.

We're talking about agents that live on your machine, doing real work, with your data, under your control, costing the site nothing. The space of things you could build like this is enormous, and we've barely scratched it.

I genuinely believe we're going to see a wave of this kind of software. Local-first, private-by-construction, sovereign. A lot of it is going to arrive faster than people expect.

Where it lives now

All of this is live on the site. There's a Playground you can open and try right now, with an honest little verdict on each model telling you what it's good at and where it bluffs. If you're a member, the Post Companion is waiting on the articles, ready to discuss whatever you're reading, running entirely on your GPU, with nothing leaving your device.

That, really, is the whole AIpster bet made concrete. Everyone has access to AI now. That's not the differentiator. The differentiator is control: keeping the intelligence close, keeping your data yours, keeping a human in the loop and a clear head about what these models can and can't do.

A friend's offhand comment sent us down this road. What we found at the end of it wasn't just a feature. It was a glimpse of where a big chunk of AI is heading. We figured we'd build the future on the site first, so you could come touch it.

FAQ

Does the in-browser AI send my questions or the article to a server?

No. Both the Playground and the Post Companion run the model entirely on your own GPU through WebGPU and WebLLM. Your prompts, the article text, and the model's responses never leave your device. You could disconnect from the internet after the model downloads and keep chatting.

What does it cost AIpster to run these features?

Nothing per use. Every token is generated on the visitor's machine, so a feature used ten million times costs the same as one used ten times. The only cost is hosting the model weights for the one-time download, which we serve from our own infrastructure.

How small are the models, and are they any good?

The free Playground model is 360 million parameters, roughly the size of a couple of photos, and runs even on a phone. Members get a 1.5-billion model and two 3.8-billion models, including an uncensored variant we tuned and published openly. Small models excel at rewriting, summarizing, and tone shifts, but they will confidently invent facts and exact numbers, so we label that limitation directly.

Will local AI replace cloud AI?

No, and that's not the claim. Training frontier models and answering questions that require the whole internet will stay in data centers. The argument is narrower: small-context work like summarizing a page or discussing one document belongs on the device in front of you, and that category is larger than most people assume.

Do I need special hardware to use the Post Companion?

You need a browser and a machine that support WebGPU. The site checks your hardware automatically. If your device can run the local model, you get the Companion. If it can't, you get a gentler experience instead, so nobody is promised something their machine can't deliver.