What is Token?
Last updated · 1 min read
Models do not read letters; they read tokens — common words are one token, rarer words split into pieces ("unbelievable" → several). English averages roughly four characters per token, so 1,000 tokens ≈ 750 words.
Why you care: API pricing is per token, context windows are sized in tokens, and per-token limits explain why long documents get truncated. It is also why asking a model "how many r's in strawberry" has been a notorious failure — the model sees tokens, not letters.