How AI actually works

Artificial intelligence is software that learns patterns from examples instead of following rules a person wrote by hand. A spam filter that has seen millions of emails learns what spam looks like; nobody types in every rule.

The AI in coding tools is a large language model, or LLM. It was trained by reading a huge amount of text and code — books, websites, documentation, open-source repositories — and adjusting billions of internal numbers, called parameters, until it got very good at one task: guessing what text comes next.

That sounds small, but to guess the next line of a program well, a model has to pick up grammar, facts, the rules of programming languages and a working sense of how problems are solved. Everything it does — answering questions, writing code, fixing bugs — is built on that one skill.

Building a model happens in two stages. Training is the expensive part: thousands of chips run for weeks while the model reads its data and its parameters are tuned. After that, the model is frozen. Inference is what happens when you use it: your text goes in, and the model produces an answer one small piece at a time.

At every step the model looks at everything so far and gives each possible next piece a probability. One is picked, added to the text, and the whole thing repeats until the answer is finished. That is why answers stream onto the screen word by word.

A setting called temperature controls how adventurous that pick is. Low temperature almost always takes the top choice, which suits code. Higher temperature lets less likely words through, which suits brainstorming.

Models don't read letters or whole words. They read tokens: chunks of text, often a word, part of a word, or a symbol. Common words like “the” are one token; a rare word like “tokenization” might be split into “token” and “ization”. In English, one token is about four characters, or roughly three quarters of a word. Code usually uses more tokens per line, because of symbols and indentation.

Tokens matter because they are how models are measured and billed. Prices are quoted per million tokens, with separate prices for what you send in, called input, and what the model writes back, called output. Output is usually several times more expensive.

The context window is how much text a model can look at in one go: your instructions, the conversation so far, the files a coding agent has opened, and the answer it is writing. It is the model's working memory. Anything outside it simply doesn't exist for the model.

Context is measured in tokens. A model with a one-million-token window can hold roughly 750,000 words at once — about ten novels, or a mid-sized codebase. A model doesn't remember past chats by itself; tools give it memory by putting earlier text back into the window. When the window fills up, the oldest material is dropped or summarised.

Everything you type is a prompt. But before your message arrives, the app you're using has already sent the model its own hidden instructions: the system prompt. It sets who the model is, which tools it can use, what style to write in and what it must never do. Coding agents send long ones, describing how to edit files, when to run tests and when to ask permission.

That's why the same model can feel completely different in two tools: same brain, different briefing. Good prompts are specific — say what you want, give context the model can't guess, show an example of the output, and say what “done” looks like.

A coding agent sends the model the same big block of text over and over: the system instructions, your project files, and the conversation so far. Only the last few lines change each turn. Processing all of that from scratch every time would be slow and expensive.

Prompt caching fixes this. The provider saves its work on the start of your request. Next time a request begins with exactly the same text, it reuses that work instead of redoing it. That reuse is a cache hit, and cached tokens are billed at a steep discount. If even one character near the start changes, the rest is a cache miss and is billed at the full price.

Caches are short-lived — often minutes — so they help most in a busy session. Some providers also charge a little extra the first time text is written to the cache.

Same work, much cheaper — and faster, because cached tokens skip most of the processing. That's why tools try hard to keep the start of every request identical.

Some models can think before they answer. Instead of replying straight away, they first write out a private chain of reasoning: breaking the problem down, trying an approach, checking it, and correcting mistakes. Only then do they write the final answer.

This makes them much better at hard problems like tricky bugs, maths and planning a large change. The catch is that the thinking is made of tokens too. Reasoning tokens are billed as output, even when you never see them. Many tools let you choose a reasoning effort, from low to high.

On its own, a model can only write text. Tool calling changes that. The app tells the model which tools exist — read a file, run a command, search the web — and when the model needs one, it writes a short structured request instead of a normal reply. The app runs the tool for real and hands the result back. The model reads it and decides what to do next.

Put that in a loop and you have an agent: read the failing test, open the file, make a change, run the tests again, repeat until they pass. Every coding agent on this site is, at heart, this loop.

Every tool a model uses has to be wired in by someone. Before MCP, every app wired in every service its own way. The Model Context Protocol is a shared standard for that wiring — think of it as USB-C for AI tools.

An MCP server wraps a service — GitHub, a database, a browser, your design files — and describes the tools it offers. Any app that speaks MCP can plug it in, and the model can use those tools straight away. Write the server once and it works everywhere, which is why so many appeared so quickly.

Browse hundreds of real servers, ranked by how fast they're growing, on our MCP page.

Computers can't compare meanings directly, so models turn text into lists of numbers called embeddings. Each one is a point in a space with hundreds of dimensions, and texts with similar meanings land close together — “fix the login bug” ends up near “authentication error”, even though they share no words.

That makes search by meaning possible: embed every document once, then embed each question and grab the nearest documents. Retrieval-augmented generation, or RAG, puts those documents into the context window before the model answers. It's how a model can answer questions about your codebase without ever being trained on it.

A model's job is to produce likely text, not true text. Usually those are the same; when they aren't, it can state something false with total confidence. That's a hallucination.

In code they look like a function that doesn't exist, a config option that was never added, or a package name that sounds just right. They're most common with niche libraries, recent changes and exact details. The defences: give the model the real source (docs, files, search), let it run the code and tests, and check anything that matters.

A model is built in stages. Pre-training comes first: it reads trillions of tokens and learns to predict the next one. That's where its knowledge comes from, and it's by far the most expensive step — but the result isn't yet a helpful assistant.

Next, fine-tuning on examples of good conversations teaches it to follow instructions. Finally, reinforcement learning: it attempts tasks, its answers are scored — by people, other models, or automatic checks such as whether the code passes its tests — and good answers are rewarded. That last step is a big reason coding models have improved so fast.

A normal, or dense, model uses every one of its parameters for every token. A mixture-of-experts model, or MoE, is split into many smaller sub-networks called experts. For each token, a small router picks just a few experts to do the work, and the rest stay idle.

The result: the model can hold the knowledge of a huge network while each token only costs as much as a small one. That's why total parameters and active parameters are listed separately. Total size decides how much memory is needed to host the model; active size decides how fast and cheap each token is.

Every parameter is a number, and numbers can be stored with more or less precision. Quantization stores them with fewer bits — for example 8 bits, called fp8, or 4 bits, called fp4, instead of 16. The model gets smaller and faster to serve, and usually cheaper, at the cost of a small and sometimes noticeable drop in quality.

That's why the same open model can cost different amounts from different hosts: they may be running different precisions on different hardware.

A model's weights are its learned parameters — the file that is the model. Closed models keep them private: you use them only through the lab's own app or API. Open-weight models publish them, so anyone can download, run, fine-tune or resell access to them.

Closed models have often led on raw ability. Open ones give you choice: many hosts competing on price and speed, data that stays on your own machines, and a model that keeps working after its maker moves on.

Early language models only handled text. Multimodal models also take in images, PDFs, audio and even video. For coding that's surprisingly useful: paste a screenshot of a broken layout or a design mock-up and ask the model to build it, or let an agent screenshot the app it's working on to check its own work.

Under the hood, an image becomes tokens too. It's cut into small patches and each patch becomes a token or so — so a big image can use as much context as a page of text.

Two numbers decide how fast a model feels. Latency, or time to first token, is how long you wait before anything appears. Throughput, in tokens per second, is how fast text flows once it starts. For a quick question latency matters most; for an agent writing a big file, throughput does.

Speed depends on the model's size, how busy it is, and the hardware and software of whoever serves it. Reasoning models also spend time thinking before the visible answer starts.

A benchmark is a standard test — the same tasks given to every model so scores can be compared. Some test knowledge and reasoning with hard exam-style questions; coding benchmarks hand the model real bugs from real projects and check whether its fix makes the tests pass.

Read them with care. A model may have seen the questions during training, which inflates its score; labs choose which results to show; and a high score on one kind of task says little about another. The best benchmark is your own work — try a few models on what you actually do.

Examples are computed from OpenRouter's public model and provider catalog, refreshed Oct 20, 2026. Prices are USD and change often. Narration voiced with Magnific.