The New Stack: A Practical Guide to Every Layer of the Agentic Web
The agentic web isn’t a single product or platform. It’s a stack, a layered infrastructure, like the web itself, where each layer serves a distinct function and multiple companies are building competing implementations.
Most coverage of the agentic web is company-by-company: Coinbase did this, Stripe did that, Cloudflare launched X. This reference maps all of it onto the stack it represents, so you can see the full architecture, understand how the layers relate, and make informed decisions about where to build.
The Four Layers
The agentic web stack has four distinct functional layers:
- Payments Layer, How agents spend, earn, and transact
- Content Access Layer, How agents read and retrieve information
- Search Layer, How agents find relevant information programmatically
- Execution Layer, How agents take action in the real world
Each layer is being built simultaneously by multiple companies, often without coordination but with converging architectural assumptions.
Layer 1: Payments
An agent that can’t spend money is fundamentally limited. The payments layer is what makes agents economic actors rather than just advisors.
Key protocols and providers:
| Provider | Protocol | Key Feature | Further Reading |
|---|---|---|---|
| Coinbase | X402 | Non-custodial agentic wallets; 50M+ machine transactions; gasless on Base | Deep dive → |
| Stripe | Shared Payment Tokens | Scoped credentials for agent purchases; fraud model rebuilt for non-human buyers | ACS → / Fraud → |
| Universal Commerce Protocol (UCP) | Open standard for agent-to-commerce interaction; auto-compatible with Stripe ACS | Protocol Race → | |
| Visa | Trusted Agent Protocol (TAP) | Announced at NRF 2026; cross-network agent payment framework | Protocol Race → |
| PayPal | Instant Checkout in ChatGPT | Partnership with OpenAI for in-chat purchase completion | Protocol Race → |
X402 is the emerging protocol standard for crypto-side machine payments. Stripe’s ACS is the dominant implementation for traditional payment rails. The notable development: Stripe’s ACS auto-supports Google’s UCP, creating de facto interoperability without a formal standards process.
What to build on: For crypto-native agent applications, X402 via Coinbase SDK. For e-commerce and traditional payments, Stripe ACS. Both via Cloudflare if you want native protocol support at the CDN layer.
Layer 2: Content Access
The web was written in HTML for human browsers. Agents need something different. The content access layer converts the human web into agent-readable structured content.
Key providers and mechanisms:
| Provider | Mechanism | Scale | Further Reading |
|---|---|---|---|
| Cloudflare | Accept: text/markdown header → auto-conversion; X-Markdown-Tokens | ~20% of web | Deep dive → |
| Firecrawl | Scraping and extraction API optimized for agent consumption | On-demand | , |
| Exa.ai | Neural search with full content return | Index-based | Agent search → |
Cloudflare’s markdown conversion is the highest-scale passive implementation, it requires no change from site owners and no additional work from developers. Firecrawl and similar tools handle sites not behind Cloudflare.
What to implement as a site owner:
- Structure key content pages with semantic HTML that converts cleanly to markdown
- Consider Cloudflare AI Index enrollment for agent discoverability
- If you produce premium content accessed at scale by agents, evaluate X402 monetization
Layer 3: Search
Human search returns result pages. Agent search needs to return structured, accurate, low-latency content that can be programmatically processed in a workflow.
Key providers:
| Provider | Index | Latency | Accuracy (QA) | Further Reading |
|---|---|---|---|---|
| Exa.ai | Own neural index | ~1-2s | 95% simple QA | Exa benchmarks |
| Brave Search | Independent index | 669ms | High (composite) | AI Multiple benchmark |
| Firecrawl | Web crawl + extraction | Variable | Extraction quality | , |
| Parallel Pro | Aggregated | 13.6s | Composite top tier | , |
| Perplexity | Own + synthesis | ~2s | Lower than Exa | , |
See also: The Rise of Agent-Native Search for full latency benchmark analysis.
The key differentiator is index ownership. Providers with their own indexes (Exa, Brave) have structural latency and independence advantages over providers wrapping Google or Bing APIs. In agent workflows where search is repeated across many steps, latency compounds: a 13-second search provider makes a 10-step research workflow 2+ minutes slower than a 669ms provider.
What to build on: Exa for accuracy-critical research workflows. Brave for latency-critical high-volume workflows. Firecrawl for direct page extraction alongside search.
Layer 4: Execution
Execution is where agents stop researching and start doing. The execution layer provides the environment for agents to run code, call APIs, write files, and take real-world actions.
Key providers and tools:
| Provider | Tool | Capability |
|---|---|---|
| OpenAI | Shell Tool | Real Linux terminal: install dependencies, run scripts, write files |
| OpenAI | Skills | Versioned, mountable instruction packages (like Docker images for AI workflows) |
| OpenAI | Compaction | Server-side context window management for long-running workflows |
| IronClaw (NEAR AI) | WASM Sandbox | Tool-level isolation for security-critical deployments |
| OpenClaw | Skill Library | Community-built skills; 1,000s of tasks available |
Skills are the defining execution primitive. OpenAI’s skills are versioned, mountable instruction packages, not prompts, not templates. When Glean deployed a well-structured Salesforce skill, accuracy on Salesforce tasks jumped from 73% to 85% and time-to-first-token decreased by 18%. Skills bring software engineering discipline (version control, testing, rollback) to AI workflow deployment.
What to build with: The shell tool for agent workflows that need to install tools and produce file-based artifacts. Skills for standardizing and sharing repeatable procedures across agent deployments. Compaction for any workflow running longer than a few minutes.
Full Stack Diagram
┌─────────────────────────────────────────────┐
│ AGENT APPLICATIONS │
│ (shopping, research, content, support) │
├─────────────────────────────────────────────┤
│ EXECUTION LAYER │
│ Skills (OpenAI) | Shell | Compaction │
│ IronClaw WASM | OpenClaw Skill Library │
├─────────────────────────────────────────────┤
│ SEARCH LAYER │
│ Exa.ai | Brave | Firecrawl | Parallel │
├─────────────────────────────────────────────┤
│ CONTENT ACCESS LAYER │
│ Cloudflare Markdown | Firecrawl │
│ AI Index | X402 Paywalls │
├─────────────────────────────────────────────┤
│ PAYMENTS LAYER │
│ X402 (Coinbase) | Stripe ACS │
│ Google UCP | Visa TAP | PayPal │
└─────────────────────────────────────────────┘
Build Examples
E-commerce agent pipeline: Payments (Stripe ACS) + Search (Exa) + Content Access (Cloudflare/Firecrawl) + Execution (OpenAI Skills for product matching logic)
Research workflow: Search (Exa for accuracy) + Content Access (Firecrawl for full article extraction) + Execution (Shell tool for data processing and report generation)
Content production pipeline: Content Access (Crawl source URLs) + Execution (Skills for content transformation + Shell for file output) + optional Payments for licensed content access via X402
Autonomous trading agent: Payments (Coinbase X402) + Search (Brave for speed) + Execution (Shell + Skills for decision logic), high risk, requires robust security architecture