Token
The unit a model reads and generates text in — roughly a word-piece; usage and cost are measured in tokens.
Models do not process text as characters or whole words but as tokens: chunks that are often a short word or a fragment of a longer one. As a rough guide, a token is about four characters of English, and a hundred tokens is around seventy-five words.
Tokens matter for two practical reasons: pricing is per token, and every model has a maximum number of tokens it can handle at once — the context window. Long prompts, large retrieved contexts, and verbose outputs all consume tokens, so concise prompting is also cheaper prompting.