The Economics of Tokens
Every word you speak to an AI costs something. Learn the currency before the bill comes due.
Home
Tag
Infrastructure
50 entries tagged Infrastructure · 1 scroll · 49 terms.
Dictionary
Adversarial Distillation
A technique in which a less capable AI model is trained on the outputs of a more capable model, extracted through fraudulent or unauthorized access at scale, to approximate the frontier model's skills without paying the research cost required to develop them.
Agent Client Protocol
An open protocol that lets AI coding agents communicate with editors and IDEs in a standardized way, so the same agent can run inside Cursor, Devin Desktop, Zed, or any other compliant client.
Agent Control Surface
The user interface that lets a human watch, approve, and redirect an AI agent that is running somewhere else, while the underlying work stays on the machine where the agent lives.
Agent Registry
A centralized directory that catalogs every AI agent operating within a system, assigns each one a verifiable identity, and tracks what each agent is authorized to do.
Agentic Billing
The metered, usage-based pricing model for AI agents, where cost is driven by tokens consumed per loop turn rather than a flat monthly seat fee.
Agentic Email
The ability of an AI agent to compose, send, organize, and manage email autonomously on behalf of a user, operating within the user's account permissions without requiring manual review of each outbound message.
Agentic Super-App
A unified software platform that combines conversational AI, autonomous task execution, computer use, code generation, and background task scheduling in a single product surface marketed to consumers and enterprise users alike.
AI Capex
The capital that hyperscalers pour into AI infrastructure (data centers, chips, and power) to build the capacity their models will run on, often years before the revenue arrives.
AI Supernode
An AI supernode is a single rack-scale system that fuses thousands of AI accelerators into one logical computer over an ultra-fast internal fabric.
API Key
A secret string that identifies you to a service's API and authorizes each request, so the provider knows who is calling, what they may access, and what to bill.
Autonomous Patching
A workflow in which an AI model both detects software vulnerabilities and generates the code fix, then opens or commits the patch with little or no human review in the loop.
Autonomous Software Engineer
An AI system sold as a unit of engineering labor rather than a coding assistant: you assign it a task and it plans, writes, tests, and submits the work for review on its own.
Background Agent
An AI agent that runs continuously on a remote server, executing tasks on a user's behalf even when the client device is closed.
Batch Inference
A processing mode where you submit a batch of requests and the provider returns results over minutes or hours instead of seconds. In exchange, you pay fifty percent less. Anthropic and OpenAI both offer it. For non-urgent work, it cuts your bill in half.
Blackwell
NVIDIA's GPU architecture for AI computing, named after mathematician David Blackwell, now reaching from data-center racks down to a desktop box through the GB10 Grace Blackwell Superchip.
Capability Compression
The accelerating trend in which AI performance once achievable only by flagship, top-tier models migrates into midtier or lower-cost alternatives within months, resetting the market's baseline expectations each cycle.
Compute Commitment
A multi-year contractual promise by an AI company to spend a specified amount on a single cloud provider's compute capacity, usually tied to an investment, priority access, or dedicated hardware.
Default Model
The model that a chat product silently serves to users who have not specified one, rotating beneath them without notice as providers update their infrastructure.
Deployment Simulation
A pre-release method of running a candidate AI model against stripped real-user conversation logs to predict the behaviors it will exhibit in production before any user sees it.
Embedded AI
The practice of AI capabilities moving into the response surface of existing productivity tools rather than living as a standalone application the user switches to.
Enterprise-Managed Authorization
A protocol extension that lets enterprise admins grant AI agents access to third-party connectors once through the company's identity provider, so individual employees inherit the access on first login instead of clicking through OAuth screens.
Frontier Lab
A small set of AI organizations training and operating the largest, most capable models at the boundary of current research, with the compute and capital to push that boundary forward.
Frontier Model
A frontier model is one of the most capable AI systems in existence at a given moment, advanced enough that its risks are not yet fully understood.
Hosted-Model Tier
A cloud distribution option where a third-party AI model runs entirely inside a hyperscaler's infrastructure with the hyperscaler's billing, identity, and data-residency controls, rather than being proxied to the model vendor's own endpoint.
In-Memory Compute
A processor architecture that performs calculations directly inside the memory cells holding the data, eliminating the round trip to a separate compute unit.
Indirect Prompt Injection
An attack that hides instructions inside content an AI agent reads, such as a web page or an email, so the agent executes them as if the user had given the command.
Inference ASIC
A chip designed from the ground up for one job, running already-trained AI models, rather than a general-purpose GPU adapted to that job.
Inference Tiering
The practice of routing AI API requests to different speed and cost tiers, paying less for work that can tolerate slower response times.
Input and Output Tokens
Input tokens are what you send to the model. Output tokens are what the model writes back. Output is almost always priced higher than input because it requires the model to actually generate rather than just read.
Managed Agent
An AI agent hosted by a platform (most commonly Anthropic's infrastructure) rather than running inside a personal terminal session. A Managed Agent persists beyond the operator's session, exposes a stable invocation surface (an API call or a button in an internal app), and reaches external systems through a governed tool gateway.
Memory Consolidation
The background process by which an AI assistant rewrites and merges short-term context, saved facts, and prior chats into a smaller, cleaner, longer-lived store the model can use later.
Non-Human Identity
A credential, token, API key, service account, or OAuth grant issued to a software system, automation, or AI agent rather than to a human user, used to authenticate and authorize machine-to-machine actions inside enterprise environments.
Pre-Deployment Evaluation
The practice of an outside party, often a government body, testing a frontier AI model for dangerous capabilities before it is released to the public.
Private Cloud Compute
Apple's server-side compute architecture for Apple Intelligence requests too large for on-device processing, designed so Apple itself cannot read the input.
Prompt Caching
A feature that stores a chunk of your prompt on the provider's side so repeated calls read from the cache instead of re-processing. Claude caches are one-tenth the price of fresh input tokens. Hitting the cache is the single biggest cost lever in production AI.
Regional AI Anchor
A frontier lab's in-country presence that links its models to a nation's regulator, top enterprises, and research universities so the stack becomes the default path for everyone downstream.
Safety Classifier
An in-model mechanism that detects when a query falls into a high-risk category and reroutes it to a safer model or refuses it outright.
Sandbox Escape
A sandbox escape is when an AI agent breaks out of, or reaches around, the isolated environment meant to contain it, so its code or actions run with access the sandbox was supposed to deny.
Sovereign AI
The practice of governments running their own LLM training and inference infrastructure inside national borders to keep model weights, data, and compute under domestic jurisdiction.
Sovereign Training Stack
A frontier model whose complete training dependency chain (accelerators, interconnect, orchestration, storage, deployment) is sourced from a single nation's domestic supply chain, so that no foreign export-control regime can gate it.
Supercluster
A supercluster is a single, centrally managed collection of tens of thousands of AI-training GPUs operating as one coordinated computing system.
Synthetic Media
Content, including images, video, audio, and text, generated by AI without being captured from the real world.
Token Pricing
The per-million-token rate a provider charges for input and output. Expressed as dollars per million tokens. Claude Opus output is $75 per million tokens. Haiku is $5. The gap is fifteen times, which is what pricing tiers exist to navigate.
Tokens
The basic unit an AI model reads and writes. Roughly 3-4 characters of English per token, so 750 words equals about 1,000 tokens. Every API call is priced by tokens in and tokens out.
Transformer
A neural network architecture, introduced in 2017, that uses attention mechanisms to process sequences of tokens in parallel and powers every modern large language model.
Usage-Based Pricing
A billing model that meters each AI prompt, token, or task against a per-model rate card instead of bundling unlimited usage into a flat subscription.
Vendor Financing
A supplier lending or guaranteeing the money its own customer needs to buy the supplier's product, so the sale happens now and much of the risk moves onto the seller's books.
Wafer-Scale Engine
A processor built from an entire silicon wafer rather than smaller chips diced from one, integrating dramatically more on-chip memory and compute cores to eliminate the chip-to-chip communication bottleneck that limits conventional GPU clusters at inference time.
Zero-Day
A software vulnerability that is exploited or disclosed before the people responsible for fixing it know it exists, leaving zero days to patch.