The Economics of Tokens
Every word you speak to an AI costs something. Learn the currency before the bill comes due.
Home
Tag
Infrastructure
36 entries tagged Infrastructure · 1 scroll · 6 wires · 29 terms.
Wisdom Wire
OpenAI Ships GPT-Realtime-2 Voice API as AI Moves Into Apps
OpenAI launched GPT-Realtime-2 plus voice translation and transcription. Same week, AI moved inside Excel, Word, and Acrobat. The tools are reshaping us.
ChatGPT Goes Ad-Supported: The Free Ride Has a New Price Tag
OpenAI took ChatGPT ads to five new countries and opened a self-serve manager with CPC bidding. The free AI tool is now an ad-supported consumer product.
OpenAI Makes GPT-5.5 Instant the Default for Every ChatGPT User
OpenAI made GPT-5.5 Instant the default for every ChatGPT user, claiming a 52.5% drop in hallucinations. The default still earns no automatic trust.
Anthropic's $900B Pre-IPO Round: The Lab Leapfrogs OpenAI
Anthropic is closing a $50B round at a ~$900B valuation in two weeks, leapfrogging OpenAI's $852B. The frontier labs are now pricing each other.
Google Cloud Next 2026: TPU 8t/8i, Gemini Enterprise Agent Platform, and a $750M Partner Fund
Google unveils eighth-gen TPU 8t/8i chips and the Gemini Enterprise Agent Platform at Cloud Next 2026, backed by a $750M partner fund. OpenAI, Tencent, Anthropic, and NEC round out a week of consolidation moves.
The Borrower and the Lender
Amazon put up to $25B more into Anthropic. Anthropic pledged $100B back to AWS over a decade. The frontier lab is now a tenant of its largest investor.
Dictionary
Agent Client Protocol
An open protocol that lets AI coding agents communicate with editors and IDEs in a standardized way, so the same agent can run inside Cursor, Devin Desktop, Zed, or any other compliant client.
Agent Control Surface
The user interface that lets a human watch, approve, and redirect an AI agent that is running somewhere else, while the underlying work stays on the machine where the agent lives.
Agent Registry
A centralized directory that catalogs every AI agent operating within a system, assigns each one a verifiable identity, and tracks what each agent is authorized to do.
Autonomous Software Engineer
An AI system sold as a unit of engineering labor rather than a coding assistant: you assign it a task and it plans, writes, tests, and submits the work for review on its own.
Background Agent
An AI agent that runs continuously on a remote server, executing tasks on a user's behalf even when the client device is closed.
Batch Inference
A processing mode where you submit a batch of requests and the provider returns results over minutes or hours instead of seconds. In exchange, you pay fifty percent less. Anthropic and OpenAI both offer it. For non-urgent work, it cuts your bill in half.
Compute Commitment
A multi-year contractual promise by an AI company to spend a specified amount on a single cloud provider's compute capacity, usually tied to an investment, priority access, or dedicated hardware.
Default Model
The model that a chat product silently serves to users who have not specified one, rotating beneath them without notice as providers update their infrastructure.
Embedded AI
The practice of AI capabilities moving into the response surface of existing productivity tools rather than living as a standalone application the user switches to.
Frontier Lab
A small set of AI organizations training and operating the largest, most capable models at the boundary of current research, with the compute and capital to push that boundary forward.
Frontier Model
A frontier model is one of the most capable AI systems in existence at a given moment, advanced enough that its risks are not yet fully understood.
In-Memory Compute
A processor architecture that performs calculations directly inside the memory cells holding the data, eliminating the round trip to a separate compute unit.
Indirect Prompt Injection
An attack that hides instructions inside content an AI agent reads, such as a web page or an email, so the agent executes them as if the user had given the command.
Input and Output Tokens
Input tokens are what you send to the model. Output tokens are what the model writes back. Output is almost always priced higher than input because it requires the model to actually generate rather than just read.
Managed Agent
An AI agent hosted by a platform (most commonly Anthropic's infrastructure) rather than running inside a personal terminal session. A Managed Agent persists beyond the operator's session, exposes a stable invocation surface (an API call or a button in an internal app), and reaches external systems through a governed tool gateway.
Memory Consolidation
The background process by which an AI assistant rewrites and merges short-term context, saved facts, and prior chats into a smaller, cleaner, longer-lived store the model can use later.
Non-Human Identity
A credential, token, API key, service account, or OAuth grant issued to a software system, automation, or AI agent rather than to a human user, used to authenticate and authorize machine-to-machine actions inside enterprise environments.
Pre-Deployment Evaluation
The practice of an outside party, often a government body, testing a frontier AI model for dangerous capabilities before it is released to the public.
Private Cloud Compute
Apple's server-side compute architecture for Apple Intelligence requests too large for on-device processing, designed so Apple itself cannot read the input.
Prompt Caching
A feature that stores a chunk of your prompt on the provider's side so repeated calls read from the cache instead of re-processing. Claude caches are one-tenth the price of fresh input tokens. Hitting the cache is the single biggest cost lever in production AI.
Safety Classifier
An in-model mechanism that detects when a query falls into a high-risk category and reroutes it to a safer model or refuses it outright.
Sovereign AI
The practice of governments running their own LLM training and inference infrastructure inside national borders to keep model weights, data, and compute under domestic jurisdiction.
Supercluster
A supercluster is a single, centrally managed collection of tens of thousands of AI-training GPUs operating as one coordinated computing system.
Token Pricing
The per-million-token rate a provider charges for input and output. Expressed as dollars per million tokens. Claude Opus output is $75 per million tokens. Haiku is $5. The gap is fifteen times, which is what pricing tiers exist to navigate.
Tokens
The basic unit an AI model reads and writes. Roughly 3-4 characters of English per token, so 750 words equals about 1,000 tokens. Every API call is priced by tokens in and tokens out.
Transformer
A neural network architecture, introduced in 2017, that uses attention mechanisms to process sequences of tokens in parallel and powers every modern large language model.
Usage-Based Pricing
A billing model that meters each AI prompt, token, or task against a per-model rate card instead of bundling unlimited usage into a flat subscription.
Wafer-Scale Engine
A processor built from an entire silicon wafer rather than smaller chips diced from one, integrating dramatically more on-chip memory and compute cores to eliminate the chip-to-chip communication bottleneck that limits conventional GPU clusters at inference time.
Zero-Day
A software vulnerability that is exploited or disclosed before the people responsible for fixing it know it exists, leaving zero days to patch.