How We Count Google Gemini Tokens
TokenCounts counts Google Gemini tokens by submitting payloads to the official generativelanguage.googleapis.com `/countTokens` endpoint. We send text, image binary details, and PDF attachments in the same format as inference requests.
Google Gemini Tokenization Methodology details
Google uses SentencePiece subword tokenization with a large vocabulary, resulting in different compression layouts than BPE-based systems.
| Factor / Area | How Gemini Counts It | TokenCounts Handling |
|---|---|---|
| Text Inputs | Splits characters using Google's large SentencePiece subword models. | Sends text content inside structured parts to the :countTokens API. |
| Image Inputs | Visual contents scale based on size, typically adding 258 to 3,000 tokens per image. | Submits base64 content with the correct MIME type (e.g. image/png) to return Gemini's official count. |
| PDF Attachments | Processes each PDF page and bills based on page count and content density. | Submits PDF base64 as application/pdf data parts to obtain the correct token cost. |