Input
0 characters · ~0 tokens (rough estimate)
Results
Run a count to compare providers side by side
No counts yet
Add text, an image, or a PDF in the input panel, then select Count tokens.
Multi-Provider LLM Token Counter
Count tokens across GPT, Claude, Gemini, DeepSeek and more.
Frequently asked questions about LLM token counting
What tokens are, why counts differ by model, and how TokenCounts measures text, images, and PDFs across Gemini, OpenAI, Claude, and open-source models.
What is a token?
A token is the smallest chunk of text (or multimodal content) that a large language model reads and bills. Models do not see raw characters directly — they see a sequence of token IDs produced by a tokenizer. In English, one token is often about four characters or roughly three quarters of a word, but punctuation, code, emoji, and other languages can use more or fewer tokens per character. API pricing, context limits, and “maximum length” errors are all expressed in tokens, not words.
Why do different models have different token counts for the same sentence?
Each model family uses its own tokenizer vocabulary and splitting rules. The sentence “Hello, world!” might become two tokens on one model and four on another because of different handling of spaces, punctuation, and subwords. Open-weight models (Llama, Qwen) use SentencePiece-style tokenizers; GPT models use BPE; Claude uses Anthropic’s tokenizer. The sentence is the same; the integer sequence the model receives is not. TokenCounts counts with each provider’s method so you see those real differences side by side.
Is a lower token count always better?
No. A lower count means less input cost and more room left in the context window — which is usually good for budget and latency. But cutting tokens by removing instructions, examples, or retrieved context can hurt answer quality. The goal is the smallest prompt that still gives the model enough information to do the task well. Compare counts before and after edits in TokenCounts; do not strip content only to win a lower number.
How can I reduce token counts?
Use shorter system prompts, delete duplicate instructions, summarize long documents instead of pasting them whole, trim chat history, compress structured data (JSON fields you do not need), and pick a smaller model when full capability is not required. For RAG, retrieve fewer or smaller chunks and deduplicate overlapping text. For images, use lower resolution when the provider allows detail settings. Count again after each change — TokenCounts makes it easy to see whether an edit saved meaningful tokens.
How are tokens counted for images?
LLM providers do not count image tokens based on file size (like binary bytes or base64 length). Instead, they process images visually by dividing them into patches or tiles. OpenAI uses two detail modes: Low Detail (costs a flat 85 tokens) and High Detail (resizes the image and counts 170 tokens for each 512x512 tile, plus 85 base tokens). Anthropic Claude estimates tokens based on dimensions with the rule of thumb (width * height) / 750, scaling down images if they exceed native resolution limits. Google Gemini charges a flat 258 tokens for small images (<= 384x384px) and tiles larger images into 768x768 grids, costing 258 tokens per tile. Local open-source models do not natively support vision in standard text tokenizers.
How do attachments (PDFs) token counting work?
PDF token counting is split between visual page rendering and raw text extraction. Google Gemini and Anthropic Claude convert each PDF page into an image (multimodal processing) to capture tables, diagrams, and layout formatting along with text. Consequently, pages are billed as images: Claude typically consumes 1,500 to 3,000 tokens per page based on content density, while Gemini charges 258 tokens per page (or more for high-resolution tiling). OpenAI and open-source models traditionally perform text extraction, passing the extracted plain text characters directly through their subword tokenizer (e.g. tiktoken), which ignores layout, styling, and images unless rendered visually via Assistants API integrations.
How does TokenCounts count tokens for Google Gemini?
TokenCounts calls Google’s official countTokens API for your selected model (e.g. Gemini 3.5 Flash). Text, images, and PDFs use the same API shape as generation, so the count aligns with what Gemini will charge.
Is Anthropic’s token counting API paid?
count_tokens does not charge per token like a full message, but your account must have API access (credits available). Token counting is free of inference cost; an empty or blocked billing account still returns errors.
How accurate is OpenAI token counting in TokenCounts?
TokenCounts calls OpenAI’s official input_tokens endpoint (Responses API). Text, images, and PDFs use the same input shape as generation, so the count matches what the model will receive. Counting does not run inference and is not billed like a completion.
Why does tiktoken give a different count than TokenCounts for OpenAI?
tiktoken counts only your raw text using a local BPE tokenizer. TokenCounts calls OpenAI’s official input_tokens API, which returns the count the model actually receives — including formatting and buffer tokens added by the Responses API. OpenAI and Anthropic both attach overhead beyond your literal input; those extra tokens are included when you call their inference APIs and are billed as part of the request. The number of buffer tokens varies by model, so a short string like “hi” can be one token in tiktoken but several more through the live API.
How accurate are these values?
TokenCounts uses the official APIs provided by each model provider — Gemini countTokens, OpenAI input_tokens, Anthropic count_tokens, and each open-source model’s Hugging Face tokenizer — so the count matches what you would get when calling that provider’s inference endpoints with the same input.
How does open-source token counting work?
The respective model's tokenizers are used to count the tokens.
What is a context window?
The maximum tokens a model can handle in one request (input and often output combined, per provider rules). TokenCounts shows your count against each model’s published limit.
Should I use one token count for every LLM provider?
No. Budget each provider with that provider’s count. Migrating a prompt from GPT-5.5 to Claude Sonnet 4.6 requires re-counting on Claude before production.