Why models can't count the R's in "strawberry"

The short version: A model never sees letters — it sees "tokens," opaque chunks of one or more characters. Counting the letters in a word asks about information that was thrown away before the model ever saw it. A whole class of "dumb" failures are really this: quirks of how text gets fed in, not failures of reasoning.

There's a famous embarrassment: ask a state-of-the-art language model how many times the letter R appears in "strawberry," and for a long time it would confidently answer "two." A system that can write working code and pass professional exams, tripped up by a question a six-year-old gets right. It became a punchline about how AI is secretly dumb.

It isn't dumb here. It's blind in a very specific way, and understanding that way explains a surprising number of other failures that look unrelated.

The model never sees letters

When you type "strawberry," the model does not receive the string s-t-r-a-w-b-e-r-r-y. Before any neural network touches your text, a separate component called the tokenizer chops it into tokens — chunks of one or more characters — and replaces each chunk with an ID number. The model only ever sees those IDs.

"strawberry" doesn't become ten letters. It becomes a handful of subword pieces — something along the lines of str, aw, berry, depending on the tokenizer — each mapped to an integer. From the model's perspective, the word is three opaque symbols, not nine letters. Asking it to count the R's is like asking you to count the R's in three flashcards that read "STR," "AW," and "BERRY" — except you're not even allowed to look at the cards, only at the numbers 651, 707, and 19772. You'd have to have memorized, from prior experience, how many R's each chunk contains. The information you need was destroyed before the question reached you.

That's the whole trick. The letter-counting failure isn't a reasoning failure. It's that the input representation throws away the thing the question is about.

Why tokenize at all?

You might ask why models don't just read one character at a time, which would make spelling trivial. The answer is a tradeoff.

If you tokenize by character, your vocabulary is tiny (a few hundred symbols) but your sequences get very long — a paragraph becomes thousands of tokens, and since a transformer's cost grows with sequence length, that's slow and expensive.

If you tokenize by whole word, sequences are short but your vocabulary explodes — you need an entry for every word, every plural, every typo, every name, and anything you didn't see in training becomes an unknown.

So nearly every model uses a subword scheme that sits in between, most commonly an algorithm called Byte-Pair Encoding. It starts from characters and repeatedly merges the most frequently co-occurring pairs into bigger chunks. Common words end up as a single token; rare words get split into pieces; truly novel strings fall back to smaller fragments. The result is a vocabulary of typically tens of thousands to a couple hundred thousand tokens that balances sequence length against vocabulary size. It's an efficiency decision, made for reasons that have nothing to do with the model's reasoning, and it quietly shapes everything the model can and can't do with text.

The other things it breaks

Letter counting is the famous one, but the same blind spot shows up all over, and once you know to look for it the pattern is obvious — anything that depends on the characters inside a token is hard:

• Spelling and reversing words. "Spell 'restaurant' backwards" requires character-level access the model doesn't natively have. It often gets close by having memorized spellings, but it's reconstructing, not reading.
• Rhyming and wordplay. Rhyme lives in letters and sounds; the model operates on chunks, so it's working at a disadvantage.
• Arithmetic. Numbers tokenize inconsistently — "1234" might be one token, or split as "123" and "4," or "12" and "34," depending on the tokenizer and what appeared in training. A model doing math is partly fighting the fact that the digits aren't laid out the way arithmetic needs them. (Newer tokenizers handle digits more carefully for exactly this reason.)
• Non-English text costs more and works worse. Tokenizers are trained mostly on English, so English words are efficient (often one token each) while other languages — especially ones not written in the Latin alphabet — get shattered into many small tokens. The same sentence in English and in, say, Burmese can differ several-fold in token count. That means non-English users pay more (you're billed per token), fit less in the context window, and often get lower quality, because the text is chopped into less meaningful pieces. It's an invisible, structural inequity baked into the tokenizer.
• Glitch tokens. Tokenizers occasionally create tokens for strings that then barely appear in the training data — usernames, scraping artifacts. A famous one, "SolidGoldMagikarp," would make older models behave erratically when it was mentioned, because the token existed but the model had essentially never learned what it meant. These are rare, but they're a vivid reminder that the token vocabulary is its own artifact with its own bugs.

Why it doesn't just get fixed

Models have gotten better at "strawberry" — partly through training on more spelling-aware data, partly because reasoning models can spell the word out letter by letter in their working steps and then count, routing around the blind spot. But the underlying cause hasn't gone away, because tokenization is what makes affordable, long-context inference possible in the first place. There's active research into tokenizer-free and byte-level models that would see characters directly, but they pay the sequence-length cost, and for now the subword tokenizer is the standard.

So the fix is mostly a patch over the symptom, not a removal of the cause. The model still doesn't see letters. It's just gotten better at reconstructing them when it has to.

What this should change about how you read failures

The practical lesson is diagnostic. When a model fails at something, it's worth asking whether the failure is about reasoning or about representation — whether the model thought wrong, or never had access to the relevant information in the first place. A startling number of "the AI is dumb" examples are the second kind: counting letters, exact string manipulation, character-level edits, precise arithmetic on long numbers. Those aren't telling you the model can't reason. They're telling you the model is reasoning over tokens, and you asked about letters.

It's a useful habit because it generalizes. The model's view of the world is assembled from a representation that was chosen for engineering convenience, and that representation has seams. Knowing where the seams are tells you which tasks to hand to the model directly, which to restructure so the answer doesn't depend on what got chopped up, and which to just do with a line of ordinary code instead.