Tokens, not words

Before a model sees your prompt, it's chopped into tokens — chunks of text roughly three-quarters of a word each. "Unbelievable" might become "un," "believ," "able." You don't need to count tokens by hand, but two consequences matter: everything you send (and everything it replies) costs tokens, and there's a hard ceiling on how many fit at once.

The context window is the model's desk

That ceiling is the context window — the total amount of text the model can consider at one time. Picture it as the size of a desk: your instructions, the documents you paste, the conversation so far, and the answer all have to fit on it together. When the desk fills up, the oldest things slide off the edge — which is why a long chat can start to "forget" what you said early on.

This explains a lot of everyday behavior:

• Pasting a huge document can crowd out your actual instruction.
• In a long conversation, restating the key constraint near the end helps, because recent text carries more weight.
• "Summarize and start fresh" often beats fighting an overstuffed thread.

It attends to position

Models pay disproportionate attention to the beginning and end of a prompt and can gloss over the middle — the so-called "lost in the middle" effect. Practical takeaway: put your most important instruction first, your source material in the middle, and any final formatting reminder last.

It predicts, it doesn't look up

Remember what's happening under the hood: the model is predicting the next token from everything on the desk. It isn't retrieving your answer from a database. That's why the same prompt can give slightly different answers each time, and why what you put in the window — good examples, the real source text, a clear instruction — matters far more than hoping it already "knows."

Your prompt isn't a search query; it's the entire workspace the model thinks inside. Whatever you want it to use, put it on the desk — clearly, and where it'll be noticed.

Try this: Paste a long article and ask about one sentence buried in the middle. Then move that sentence to the very top and ask again. The jump in accuracy is the context window and the position effect, live.