People ask what is "underneath" these tools and expect the answer to be complicated. The core is not.
Your text is broken into tokens, roughly word fragments. Each token becomes a long list of numbers. The model pushes those numbers through many layers of arithmetic, weighing how every part of your input relates to every other part, and out the other end comes a score for every possible next token. One is chosen. It is appended to the sequence. The whole thing runs again. That loop repeats until the answer ends.
That is the core. Everything the product does on top, such as search, file reading, memory and tool calls, is software wrapped around that loop, feeding it more text to condition on.
Three consequences follow, and they explain most of the surprises people report.
It has no memory of truth, only of patterns. Training adjusted a very large set of numerical parameters, often billions or more in modern large models, so that the model's predictions matched human writing. Facts that appear often and consistently in that writing tend to come out right. Facts that are rare, recent, contested or specific to your organization come out as whatever is statistically plausible. A citation can be plausible without existing. A policy clause can be plausible without being yours.
Confidence is a writing style, not a signal. The model learned that expert prose sounds assured, so its output sounds assured. Fluent tone is not a calibrated confidence signal. Some systems expose token probabilities or other uncertainty signals, but the prose itself does not tell you whether a claim is true. If you want a signal you can act on, build one in: ask it to separate what the supplied evidence says from what it is inferring, and to state explicitly when the evidence does not cover the question. Then check that it did.
Context is everything, and context is finite. The model sees only what is in the request window: your message, recent history, any documents or search results the application inserted, and whatever instructions the product prepends. Anything not made available in the model's current context cannot directly influence that answer. Long conversations push earlier material out, or bury the important passage under noise. A model that answered correctly on page one of a chat can answer wrongly on page twenty about the same document, because the relevant paragraph is no longer in view.
A few product features change how this plays out, and they are worth knowing by name.
- Search or browsing can make current information available to the model. It does not guarantee that the sources are current, authoritative or correctly interpreted.
- File attachments and connectors insert your material. That is how the assistant "knows" your document, and also how your document leaves your machine.
- Memory stores things you said earlier and inserts them later. Useful, and also a way for a stale or wrong fact to follow you into every future conversation.
- Reasoning or thinking modes give some systems more inference effort for difficult tasks. That can improve performance on problems involving planning, mathematics, coding or multi-step analysis. Any reasoning shown to you should be treated as an explanation of the answer, not as a forensic record of the model's internal computation.
- Tools and agents let the model propose an action, such as running code or querying a system, which the application may execute and return the result. The model never reaches into a database on its own. The application decides.
The remainder of this chapter describes the transformer mechanism for readers who want it.
Text becomes tokens, which may be words, word fragments or other symbols. Token identifiers are mapped to numerical representations. In a typical decoder-only transformer, repeated layers combine information from earlier tokens using attention and transform those representations through learned networks. Position information helps preserve order. [8]
The model produces scores for possible next tokens. A decoding procedure selects a token, appends it to the sequence and continues until a stopping condition is reached. A token probability describes a continuation under the model. It is not a calibrated probability that the complete answer is true.
The context window limits the information a model can process in a request. Input, history, retrieved evidence and generated output consume capacity according to the model and service. More context can preserve useful detail, but can also increase delay and distract from the relevant passage. Deliberate selection matters even when a document fits.
An unsupported or false generated claim is often called a hallucination. It can arise from missing evidence, ambiguous instructions or patterns learned during training. Lowering randomness may make an answer more repeatable without making it correct. A requested explanation of reasoning is also generated text, not a dependable audit of internal computation.
The application may interrupt generation to execute a proposed tool call and return its result as new context. That is how an assistant can consult current information. The model does not gain unrestricted access to a database simply by mentioning one.