Guide
Tokens vs words vs characters
Three units, three different jobs. Words are for humans. Characters are for storage. Tokens are for models — and for billing.
The conversions
| Text | Words / chars | ≈ tokens |
|---|---|---|
| 1000 English words | 1000 words | 1,330 |
| 1000 Chinese characters | 1000 chars | 600 |
| An 80,000-word novel | 80,000 words | 106,400 |
Why tokens exist at all
Models split text into subword chunks (tokens) learned from training data, because a fixed vocabulary of whole words can't cover every language, name, or typo. Common words are often one token; rare words split into two or three; a long number can be several. That is why two passages with the same word count can differ by 30% in tokens — and why counting words to budget an AI bill always drifts.
Stop estimating from word counts: paste the actual text into the token counter.