What a Large Language Model Actually Does

In a few years these tools went from research labs to everyone's phone. Between the people calling them imminent consciousness and the people calling them a parlour trick, a simpler fact gets lost: the core idea can be explained in one paragraph — and understanding it changes how you use them.

The core idea: predict the next piece

Start with an unfinished sentence: "Coffee is a drink made from roasted…". Most people would finish it with "beans". That is exactly what the model does, with two differences.

First, it does not pick one word — it computes a probability for every possible continuation, then selects from among them.

Second, it repeats. It appends the chosen piece to the text and asks again: and after that? And after that? Until the answer is complete.

That is the whole mechanism. There is no database of facts inside, and no separate reasoning engine. Long coherent passages are what you get from repeating that one small step thousands of times.

Where the "learning" comes from

During training the model is shown vast amounts of text with parts hidden, and asked to predict what is missing. At first it is completely wrong, so its internal settings are nudged slightly in the right direction. Then it happens again, billions of times.

Those settings — the weights — are all that remains at the end. The model does not keep a copy of the text it trained on. It keeps patterns extracted from it: how sentences are built, which words accompany which, what an explanation looks like, what a polite reply looks like.

And because those patterns come from text written by people, the accuracy in that writing and the bias in it both carry over together.

Why it is confidently wrong

This is the most practically important point in the article.

When you ask about something it does not know, there is no internal mechanism that tells it so. Its only job is to produce likely text, and the likeliest text in that position is an answer that has the right shape.

So it generates a book that was never written, a legal citation that does not exist, or a plausible-looking number. This is called hallucination.

The dangerous part is that the tone never changes. Correct and invented information are written with identical confidence, because to the model confidence is a stylistic feature, not a signal of certainty.

What it is good and bad at

The unifying rule: the more the task is transforming text you gave it, the better the output. The more it is retrieving a fact from memory, the more it needs checking.

Terms that keep coming up

Token. The unit the model reads, usually a fragment of a word. This is why it can fail at apparently trivial tasks like counting the letters in a word — it never sees letters.

Context window. The most text it can take into account at once. In a long conversation, whatever falls outside the window drops out of its reasoning.

Knowledge cut-off. Training stops at a point in time. Anything after it is unknown, unless the model has a search tool fetching text at the moment you ask.

Temperature. A setting controlling how varied the selection is. Low gives more consistent, predictable answers; high gives more variety and more risk.

How to ask better

Understanding the mechanism leads straight to better use:

1. Give it the text rather than asking it to recall the text. "Summarise this article" beats "what did that article say" by a wide margin.
2. Specify the shape you want. Length, audience, tone, format. It does not guess your intent.
3. Ask for the working, not just the result on anything with steps — it reduces errors.
4. Correct rather than restart. Saying exactly what is wrong with an answer is faster than rewriting the question from scratch.
5. Verify what matters. Any number, name or citation you will act on deserves a minute of checking.

The bottom line

A language model is not a mind and not a search engine. It is an extremely capable pattern machine for language, producing likely text rather than verified text.

Understand that distinction and you have an excellent tool. Miss it, and sooner or later you get an invented fact written with perfect confidence.

Frequently asked questions

Does a language model understand what it says?

Not in the sense a person does. It predicts the next piece of text from patterns learned across an enormous amount of writing. The output can be accurate and useful, but the process that produced it involved no understanding and no checking.

Why does it invent things that are not true?

Because it is built to produce likely text, not true text. When it does not have the fact, nothing inside it stops — so it generates whatever most resembles a correct answer. This is called hallucination.

Does it search the internet for every question?

No, not unless it has explicitly been given a search tool. On its own it relies on what it learned during training, which is fixed and has a cut-off date.

What is a token?

The unit of text the model works in, usually smaller than a word. It does not read letters or whole words but these fragments, which is why it sometimes fails at seemingly trivial tasks like counting letters.

Is my data used to train it?

It depends on the service and its settings, and it differs between personal and business accounts. The safe working assumption is to treat anything you type as potentially readable, and keep secrets and sensitive personal data out of it.