Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.xenarch.com/llms.txt

Use this file to discover all available pages before exploring further.

Set up pay-per-crawl on your site in five minutes — detect AI bots (GPTBot, ClaudeBot, PerplexityBot, etc.), return HTTP 402, collect USDC micropayments on Base. Works standalone or as a non-custodial alternative to Cloudflare pay-per-crawl.

​
Xenarch vs Cloudflare pay-per-crawl

XenarchCloudflare pay-per-crawl
CustodyNon-custodial — USDC settles on-chain via splitter contractCustodial — Cloudflare holds your earnings
Fee0% (capped at 0.99% forever)Cloudflare takes a cut
SettlementUSDC on Base L2, every payment on-chainFiat payout, off-chain ledger
StackAny stack (Python, Node, WordPress, Joomla, custom)Cloudflare Workers required
PricingPer-page or per-path, publisher-definedCloudflare-defined

​
Prerequisites

  • A USDC wallet on Base (this is where you receive payments)
  • Your content served over HTTPS

​
Step 1: Register

npx xenarch register
This creates your publisher account and returns an API key (xk_pub_...).

​
Step 2: Add your site

npx xenarch site-add example.com
Returns a site token (st_...) and site ID. Save both.

​
Step 3: Set pricing

npx xenarch pricing example.com --default 0.003
Sets a default price of $0.003 per page. Add path-specific rules: 
npx xenarch pricing example.com \
  --rule "/premium/*=0.01" \
  --rule "/api/*=0.05" \
  --rule "/public/*=0"

​
Step 4: Install middleware

Choose your integration: 
pip install xenarch[fastapi]
Site-wide middleware (gates all bot traffic):
from xenarch.middleware import XenarchMiddleware

app.add_middleware(
    XenarchMiddleware,
    site_token="st_...",
    site_id="your-site-uuid",
    access_token_secret="your_secret",
    excluded_paths={"/healthz", "/docs"},
)
Per-route decorator (gates individual endpoints):
from xenarch.decorator import require_payment

gate = require_payment(
    site_token="st_...",
    site_id="your-site-uuid",
    access_token_secret="your_secret",
)

@app.get("/premium")
@gate
async def premium(request: Request):
    return {"content": "premium data"}
See the FastAPI middleware guide.

​
Step 5: Add pay.json (optional)

Serve a pay.json file at /.well-known/pay.json so agents can discover your pricing without hitting a 402 first: 
npx pay-json-generate \
  --wallet 0xYOUR_WALLET \
  --receiver 0xC6D3a6B6fcCD6319432CDB72819cf317E88662ae \
  --price 0.003 \
  --provider xenarch \
  --output .well-known/pay.json
See the pay.json spec.

​
How it works for your users

  • Humans: Pass through completely unaffected. The middleware only gates bot traffic.
  • AI agents with payment: Pay USDC, get an access token, access your content.
  • AI agents without payment: See HTTP 402 with payment instructions.

​
Check your earnings

npx xenarch stats your-site-id