The Changing World of AI -- Tokens and Users

 

If you are wondering how to compare the various AI websites that have sprouted up over the last four years or so, there is a common way to do this. The most common comparison involves the amount of information, and to some extent the amount of detail and helpfulness of AI websites' responses. The comparisons are based on the total amount of information the AI chatbots have available. Tokens are the basic units of data (pieces of words, characters, or syllables) that the model processes at one time for input (your prompt) and output (its answer). Roughly, 1,000 tokens equal about 750 words.  See What are tokens and how to count them? 

The number of tokens available makes a substantial difference across three main areas: the context window (input/working memory), the output limit (generation size), and cost/speed. The following explanations came from Google Gemini. 

Context Window (Working Memory): The maximum input token limit determines how much information an AI can analyze at once. A small window (e.g., 4k–8k tokens) limits you to brief queries, while a massive window (e.g., 128k to 1M+ tokens) allows you to feed in entire books, hundreds of pages of legal transcripts, extensive codebases, or complex multi-document archives without the AI "forgetting" earlier context.

Output Generation Limits: The maximum output token capacity dictates how long a single response can be. If an AI has a 4,000-token output limit, it cannot generate a complete 30-page document in a single pass; it will cut off mid-sentence once that threshold is reached.

Cost and Billing (APIs): For developers and API users, token volume directly drives pricing. Every token sent (prompt) and received (completion) is metered. A larger token budget means higher processing costs.

Back to Me:

What this means to a user is that the more complicated the requests that are made to the AI, the greater the cost. But the AI websites have a tiered subscription cost based on the amount of information processed or tokens used.

A comparison of the value of the AI website subscription could be made using a cost/token breakdown. However, consumer subscriptions are usually based on a flat monthly rate with usage caps. But comparisons can be made through the cost versus rate caps. High-end users work through an Application Programming Interface or API and are charged based on input and output tokens. Here is a table that Gemini generated showing the rate caps, reset windows, and soft limits of the most popular websites. 

Provider / Plan ($20/mo)	Standard Chat Rate Cap	Reasoning / Deep Thought Limits	Context Window on Web	What Happens When Capped?
ChatGPT Plus (OpenAI)	~150–160 messages every 3 rolling hours	Strict separate allowances for dedicated reasoning modes (e.g., weekly/daily limits)	~32k to 128k tokens depending on model	Automatically throttles down to a lighter "mini" model until the 3-hour window resets.
Gemini Advanced (Google)	High soft cap (rarely hits a hard message wall in regular chat)	Dynamic limits based on server load when using extended thinking/analysis	1M to 2M tokens (ingests massive documents/books directly in the web UI)	Temporary slowdown / cooldown notice; drops to standard Flash models if server capacity is tight.
Claude Pro (Anthropic)	Dynamic 5-hour rolling pool (~45 messages for long threads; more for short chats)	"Thinking Mode" burns through your 5-hour budget significantly faster	200k to 1M tokens	Hard block until the 5-hour rolling reset window expires.
Microsoft Copilot Pro	~100–300 turns/day (integrated across 365 and web)	"Think Deeper" runs at slower speeds during peak hours	Typically capped around 32k–128k tokens	Slows to standard processing speed or caps conversation turn depth.

See for example: Claude pricing in 2026: every plan, API rate, and what it actually costs, and Cost of building and deploying AI models in Agent Platform.

Again, from Gemini:

Core Mechanics That Dictate Limits

Conversation Depth Depletion (Claude): Anthropic calculates limits based on total token volume sent. Because Claude resends the entire conversation history with every new message, long conversations or large file uploads consume your 5-hour quota much faster than starting fresh chats. 

Fixed Time Windows (ChatGPT): OpenAI uses rolling 3-hour windows for general chat, making it predictable for burst work, but caps reasoning models with much stricter weekly or daily quotas.

Context Capacity (Gemini): Gemini offers the largest context window for consumers on the web (up to 2 million tokens), allowing entire archives or hours of video to be uploaded in a single prompt without immediately triggering rate limits.

Back to Me:

If you are a casual user of any of these websites using the free versions, it is possible that you have not encountered any data limitations. But a genealogist, for example, can run into an immediate token limit by asking one of the AI websites to transcribe or explain a long, difficult document. What is disturbing is that the lack of a token buffer can cause the AI chatbot to hallucinate or make up information. See Why Language Models Hallucinate.

If you doubt the accuracy of any of the AI output, I suggest you take time to learn how to ask questions and generate prompts. See How to Improve Any Prompt.

Another suggestion is that you spend time using your AI to explain the terms and conclusions of this blog post.