How ChatGPT and Language Models Actually Work

ChatGPT in one sentence

ChatGPT is a large language model (LLM) — a neural network trained on a gigantic pile of text that can do one thing: given the start of a passage, predict what should come next. Everything else — conversations, code, translations, poems — grows out of that single skill repeated many times in a row.

If you have no background on neural networks, take a quick detour through our piece on what a neural network is: it covers neurons, layers and weights. Here we go a step further and see how those become a conversation partner.

Tokens: how the model slices text

First surprise: the model does not see letters and words the way we do. Before text goes in, it is sliced into tokens — chunks that are more often parts of words than whole words. The word "hello" might be a single token, while a rare word like "quantum-mechanical" splits into several.

A rough rule: one token ≈ 0.75 of an English word. Why do it this way? Tokens are a compromise between letters (few of them, but the text becomes far too long) and words (millions of them, with new ones appearing constantly). Chunk-sized tokens let the model build any word, even an unfamiliar one, from a limited "vocabulary" of tens of thousands of pieces. Each token is turned into a long list of numbers (a vector) — and that is the form in which the model works with language.

The core trick: predicting the next token

Now the heart of it all. ChatGPT keeps asking itself one question: "Given all the text so far, which token is most likely to come next?" The model does not produce a single answer — it produces probabilities for every token at once. For example, after "The sky today is absolutely…" the options "clear," "cloudless," "blue" get high probability, while "herring" gets almost none.

Then the model picks one token, appends it to the text, and… starts over — now taking into account the word it just added. Then again. And again. That is how a whole paragraph is built, one chunk at a time. This is exactly why the answer "types out" left to right in the interface: you are literally watching the model add token after token in real time.

Sounds too simple to explain coherent text? That is the paradox: to predict the next word well across billions of different texts, the model is forced to pick up grammar, facts about the world, logical links and style. "Predict the continuation" is actually a monstrously hard task, and by solving it the network learns a great deal about language and the world along the way.

The transformer and "attention"

What device inside predicts tokens so deftly? It is a transformer — a type of neural network invented in 2017. The "T" in GPT is exactly that "Transformer" (in full, Generative Pre-trained Transformer).

The transformer's key idea is the attention mechanism. It lets the model, while processing each word, "look back" at all the other words in the text and decide which of them matter right now. In the sentence "The cat sat on the mat because it was warm," attention helps tie "it" to "the mat" rather than "the cat." Attention is what gives the model a sense of context over long distances — something the old approaches, which read text strictly one word at a time, sorely lacked.

How the model was trained: two big stages

A finished ChatGPT is the result of two very different training stages. It helps to understand them, because they explain both the model's strength and its quirks.

Stage 1. Pre-training. The model is fed a colossal amount of text — books, articles, websites, forums — and made to guess the next token billions of times. In this stage it absorbs language, facts and style, but behaves like "smart autocomplete": it continues text without realizing you want a conversation. Ask such a raw model "Capital of France?" and it might continue with a list of similar questions instead of answering.

Stage 2. Fine-tuning and RLHF. To turn "autocomplete" into a helpful assistant, you apply RLHF — reinforcement learning from human feedback. It works like this: human raters look at different model answers and mark which is better, which is worse, which is harmful or dangerous. Those judgments train a separate "reward model," which then acts like a strict editor and tunes ChatGPT's behavior — making answers more helpful, honest and safe. RLHF is exactly what taught the model to follow instructions, admit when it does not know, and refuse harmful requests.

Temperature: the randomness dial

Remember the step "pick one token from the likely ones"? How boldly the model picks is controlled by a parameter called temperature. It is essentially a "predictable ↔ creative" dial.

• Low temperature (near 0). The model almost always takes the single most likely token. Answers are precise and stable, but repetitive — good for facts, code, instructions.
• High temperature. The model is more willing to pick less obvious options. Text becomes livelier and more varied — handy for brainstorming and creative work, but the risk of drifting into nonsense or invention rises.

That is why the same prompt at high temperature gives slightly different answers each time, while at zero it gives almost identical ones. In the everyday chat interface, temperature is already set to a balanced value for you, but in the API settings you can control it by hand.

Hallucinations and the model's limits

Because the model is trained to produce a plausible continuation rather than to check the truth, it has a built-in weakness — hallucination. That is the term for cases where ChatGPT confidently reports invented facts: nonexistent books, wrong dates, fake links and quotes. The model is not lying on purpose — it simply generates smooth text, and smooth does not always mean true.

From this follow a few practical takeaways worth keeping in mind:

• Double-check facts, numbers and quotes — especially when the cost of an error is high.
• Remember the "knowledge cutoff": the base model knows the world only up to when its training ended and, without special tools, is unaware of recent events.
• Phrasing decides a lot. The clearer you spell out the task, context and desired format, the better the answer. There is a dedicated breakdown for that — how to write AI prompts.

The "magic" of ChatGPT is now broken down into understandable parts: tokens, predicting the next chunk, attention inside the transformer, two training stages and the temperature dial. And if you want to see how a similar idea produces images rather than text, read about generative AI.