DEFINITION
Token
LLM Token
Definition
The basic unit of text processed by LLMs. Roughly 1 token ≈ 0.75 words in English. LLM pricing and context limits are measured in tokens. For agents running complex tasks with large contexts, token optimization is critical for cost control.
Examples in the Wild
- Example 1:'Hello world' = 2 tokens
- Example 2:A typical agent session might use 50,000-200,000 tokens