Just a few short years ago, the only people who ever said the word “token” were arcade owners and subway riders. Today it is on your invoice. It is in your subscription plan. It is the reason your AI assistant suddenly “forgot” what you told it forty messages ago, and it is the reason one model costs $0.20 per million and another costs $30.
We have all seen it; you open a billing dashboard or a plan comparison page and there it is — 1M token context window, $5/MTok input, 128k max output — presented as though everyone already knows what that means. Most people don’t. And that’s a shame, because tokens are not complicated. Once the concept clicks, an enormous amount of AI jargon stops being jargon and starts being obvious.
So let’s demystify it. No math degree required.
Tokens: Not Words, Not Letters — Something In Between
Here is the first thing to understand: language models do not read the way you and I read. They do not see letters. They do not really see words, either. They see tokens — chunks of text that sit somewhere in between.
Anthropic’s own documentation defines tokens as “the smallest individual units of a language model,” which “can correspond to words, subwords, characters, or even bytes.” For Claude, a single token works out to roughly 3.5 English characters. Google’s teams use a rounder rule of thumb: about 750,000 words per million tokens.
Common words tend to be one token all by themselves — the, and, technology. Rarer words get chopped into pieces. Punctuation and spaces count. Even the invisible scaffolding of a conversation counts.
In its simplest form, tokens = the syllables of machine language. And just like syllables, they are the unit the system actually counts, bills, and budgets against.
The Tokenizer: Why AI Can’t Count the R’s in “Strawberry”
If you want a perfect demonstration of why this matters, ask a chatbot how many R’s are in the word “strawberry.” For a long stretch, the most advanced models on Earth got that wrong — confidently.
The reason is beautifully mundane. GPT-4 doesn’t see s-t-r-a-w-b-e-r-r-y. It sees three tokens: str, aw, and berry — internally just the ID numbers 496, 675, and 15717. Asking it to count letters is like handing someone who doesn’t read Chinese a character and asking them to count the brush strokes; the information is technically there, but not in a form the reader can act on.
Make no mistake — this is not a sign that the model is stupid. It is a sign that the model’s eyes work differently than yours. Understanding that single fact explains an entire category of weird AI behavior, from spelling puzzles to arithmetic quirks to why some languages cost more to process than English.
The Context Window: Your AI’s Working Memory
Now for the term that actually affects your day-to-day: the context window.
The context window is the total number of tokens a model can hold in its head at once — the system’s instructions, your entire conversation history, any files or images you attached, the tool results it fetched, and the answer it is currently writing. Anthropic describes it as the model’s “working memory,” which is exactly right, and it’s importantly different from what the model learned during training. Training is what it knows. Context is what it’s currently holding.
And here is the part that surprises people: the context window refills from scratch on every single turn. When you send message number twelve, the model is re-reading messages one through eleven — every word, plus every one of its own replies — and then answering. The conversation doesn’t stream past it; it stacks up. That is why long chats get slower and pricier as they go, and why an assistant eventually starts dropping details from early on.
The good news is that the ceilings have gotten enormous. Claude Opus 5, Sonnet 5, and the entire current lineup run a 1 million token context window with up to 128,000 tokens of output. Google’s Gemini 3.1 Pro and the Flash models sit at 1 million as well, with roughly 64,000 tokens of output. A million tokens is about 750,000 words — call it a 1,500-page technical manual, an entire mid-sized codebase, or somewhere north of seven full-length novels. In one prompt. That is a remarkable place to be, considering models were choking on 8,000 tokens not that long ago.
Let’s Talk Numbers; Token Sized Numbers
Tokens aren’t just a memory unit — they are the billing unit, and this is where the concept pays for itself. Providers charge separately for tokens going in (what you send) and tokens coming out (what the model writes), and output is always the expensive half.
As of August 2026, on the Claude API: Opus 5 runs $5 per million input tokens and $25 per million output. Sonnet 5 is $2 in / $10 out — that introductory rate is now permanent. Haiku 4.5 is $1 in / $5 out.
On Google’s side, Gemini 3.1 Pro is $2 in / $12 out, Gemini 3 Flash is $0.50 / $3.00, and Flash-Lite bottoms out at $0.25 / $1.50.
OpenAI’s current flagships price similarly by tier: gpt-5.6-sol at $5 in / $30 out, gpt-5.6-terra at $2 / $12, and gpt-5.6-luna at a whopping-in-the-other-direction $0.20 / $1.20 — with batch processing knocking roughly 50% off if you can tolerate waiting.
Put a real number on it. A 3,000-word document is somewhere around 4,000 tokens. Sending it to Opus 5 costs you two cents. Sending it two hundred and fifty times still costs less than lunch. The reason enterprise AI bills get scary isn’t any one request — it is thousands of requests, each one dragging a growing conversation history behind it. The tokens you forgot you were re-sending are the ones that get you.
One more wrinkle worth knowing: tokenizers change. Anthropic notes that Claude 4.7 and later use a newer tokenizer that produces roughly 30% more tokens than earlier versions for the same text. Same document, same price per token, different token count. If you ever upgrade models and your bill moves without your usage moving, that’s why.
Pictures Count Too: Images, PDFs, and Everything Else
Tokens are not a text-only tax. Everything you feed a modern model gets converted into tokens first.
Claude, for instance, chews images into patches of 28×28 pixels, each patch becoming a visual token. The math is refreshingly simple: divide the width by 28, divide the height by 28, round both up, multiply. A 1000×1000 pixel image? 1,296 tokens. A 4K screenshot on a high-resolution model? Up to 4,784 tokens — though the models automatically downscale oversized images, which mercifully caps the damage.
So that PDF you dropped in “real quick”? It has a token count. That folder of screenshots? Token count. The tool that went and fetched a webpage for you? Every character of that page came back as tokens and took a seat in your context window.
Context Rot: More Room Isn’t Always More Brains
Here’s the counterintuitive one, and it may be the most practically useful idea in this entire piece.
A bigger context window is not automatically a better answer. As the token count climbs, accuracy and recall measurably degrade — a phenomenon the industry has taken to calling context rot. Anthropic puts it plainly: curating what goes into the context matters as much as how much space you have.
Think of it like a desk. A bigger desk is genuinely useful. But a bigger desk buried under 400 pages of paper is worse than a small desk holding the three documents you actually need. Just because you can paste a million tokens in doesn’t mean you should.
Which is why the practical advice is almost embarrassingly old-fashioned: give it the relevant material, not all the material. Start a fresh conversation when the topic genuinely changes. And if a long thread starts feeling foggy, it probably is — summarize where you landed, open a new chat, and paste the summary in. You’ll get sharper answers AND a smaller bill.
What’s Next?
Will tokens still be the unit of measure in five years? Probably — BUT the way we experience them is already changing fast. Context windows went from 8,000 tokens to 1,000,000 in about three years, and the frontier labs are openly experimenting with the tier above that. Prompt caching now lets you re-send the same background material at a steep discount instead of paying full freight every turn. Server-side compaction quietly summarizes the early parts of a long conversation so it can keep going past the ceiling. And research into byte-level and patch-based models is chipping away at tokenization itself — the day a model can simply see the letters in “strawberry” is coming.
More often than not, the technologies that end up mattering most are the ones that eventually become invisible. Nobody thinks about the megabytes in an email attachment anymore, and eventually nobody will think about tokens either. Until that day, though, understanding them is undoubtedly the cheapest upgrade you can give yourself — it makes you better at prompting, better at budgeting, and much harder to confuse with a pricing page.
The path there will certainly be fascinating to watch.
Sources
- Anthropic. “Glossary — Tokens and Context Window.” Claude Developer Platform, https://platform.claude.com/docs/en/about-claude/glossary. Accessed 15 Aug. 2026.
- Anthropic. “Context Windows.” Claude Developer Platform, https://platform.claude.com/docs/en/build-with-claude/context-windows. Accessed 15 Aug. 2026.
- Anthropic. “Vision — Image Token Costs.” Claude Developer Platform, https://platform.claude.com/docs/en/build-with-claude/vision. Accessed 15 Aug. 2026.
- Anthropic. “Pricing.” Claude Developer Platform, https://platform.claude.com/docs/en/about-claude/pricing. Accessed 15 Aug. 2026.
- OpenAI. “API Pricing.” OpenAI Developers, https://developers.openai.com/api/docs/pricing. Accessed 15 Aug. 2026.
- AI Toolbox. “Gemini Context Window & Token Limits (2026).” ai-toolbox.co, https://www.ai-toolbox.co/gemini-models/gemini-context-window-token-limits-2026. Accessed 15 Aug. 2026.
- 16x Prompt. “Why ChatGPT Can’t Count R’s in Strawberry.” 16x Prompt Engineering Blog, https://prompt.16x.engineer/blog/why-chatgpt-cant-count-rs-in-strawberry. Accessed 15 Aug. 2026.
Thinking about putting AI to work in your business?
We build practical AI automation and custom software — and we'll tell you plainly
where it pays off and where it doesn't.
Book a free assessment →