When evaluating AI solutions, many organizations focus on per-token API pricing without calculating true costs. for any document type. A company processing millions of tokens monthly may find that the convenience of public APIs comes with a significant price tag.
Understanding Token Economics
LLM APIs charge per token, roughly equivalent to 0.75 words. Both input (your prompts and context) and output (AI responses) count toward costs. Vendor pricing pages confirm this structure: the Anthropic Claude pricing reference and the AWS Bedrock pricing page both list rates as input and output prices per million tokens. For applications like document processing, RAG systems, or customer service automation, token volumes add up quickly.
Consider a document processing workflow that analyzes contracts. Each contract might be 5,000 tokens. Add a 2,000 token system prompt and 1,000 token response. That is 8,000 tokens per document. Processing 1,000 contracts monthly means 8 million tokens, just for one use case.
Direct API Costs
At current pricing for frontier models, 8 million tokens costs roughly $80 to $240 monthly depending on the model and provider. That seems reasonable. But enterprises rarely have just one use case.
Add customer service automation handling 10,000 conversations monthly (50 million tokens). Add internal knowledge search for 500 employees making 20 queries daily (150 million tokens). Add code assistance for 50 developers (100 million tokens). Suddenly you are processing 300+ million tokens monthly at costs exceeding $3,000 to $10,000 depending on model choice.
Hidden Cost Categories
Compliance and Legal Exposure
For regulated industries, sending data to external AI services creates compliance burden. Legal review of data processing agreements, additional security assessments, and audit preparation all have costs. A single compliance incident involving improperly handled data can cost far more than any infrastructure investment.
Rate Limits and Reliability
Public APIs have rate limits. Enterprise tiers help, but you still depend on provider availability. Outages at AI providers have affected major companies. Building redundancy (multiple providers, fallback logic) adds development and maintenance costs.
Vendor Lock-in
Applications built for one provider's API require rework to switch. Prompt engineering that works for one model may not work for another. This creates switching costs and reduces negotiating leverage.
When Self-Hosting Saves Money
The breakeven point varies by use case, but general patterns emerge.
- High volume: Processing 100+ million tokens monthly often makes self-hosting cheaper
- Predictable workloads: Steady usage benefits from fixed infrastructure costs vs. variable API charges
- Long context applications: RAG systems with large context windows consume tokens rapidly
- Fine-tuning needs: Custom models require private deployment anyway
A dedicated GPU instance capable of running a 70B parameter model costs roughly $3 to $8 per hour on major cloud providers. Running 24/7, that is $2,200 to $5,800 monthly. For organizations processing hundreds of millions of tokens, this is often 50-70% cheaper than API pricing.
Calculating Your TCO
To calculate true cost of ownership for AI, include: direct API or infrastructure costs, development time for integration and maintenance, compliance and security overhead, reliability and redundancy requirements, and opportunity cost of vendor dependencies.
We help organizations model these costs for their specific use cases. Often, the answer is a hybrid approach: public APIs for experimentation and low-volume applications, private deployment for high-volume production workloads.
Sources
- Anthropic Claude API pricing reference. Current per-million-token pricing for Claude models, used as the basis for the token-economics figures.
- AWS Bedrock pricing. Managed foundation-model pricing across providers, including per-token rates referenced for cloud platform comparisons.