Handling 402 and 403 Responses: An Error Reference for Agents

This guide is for developers building agents or crawlers that fetch paid publisher content over the AISA pay-per-crawl network. You will be able to branch on every denial code, follow the guidance block that comes with each denial, and know when to retry, when to hand the user a payment link, and when to skip a URL and move on. The guide ends with a reference table of error codes and a template for what the agent should tell its user, so the conversation stays accurate and calm.

How the flow works

Before you start

• A registered AISA crawler account with its crawler token
• A paid publisher URL behind an AISA-integrated gateway
• curl or Python with the requests library

Step by step

1. Send the token header and read the status

   Request the URL with your crawler token and your genuine User-Agent. The header is X-Crawler-Token; it replaces the earlier name X-AISA-Crawler-Token that older AISA pages still show. Never put the token in Authorization: Bearer, which is reserved for the gateway layer. A 200 is content, free or paid. A 402 or 403 is a denial and costs $0. Paid 200 responses may carry X-AISA-Charged and X-AISA-Charge-Amount.

   curl -s -i 'https://PUBLISHER-SITE/research/report.data.json' \
     -H 'X-Crawler-Token: YOUR_CRAWLER_TOKEN' \
     -A 'MyBot/1.0 (+https://example.com/bot)'

   S1 · esa.aisa.oneS2 · esa.aisa.one

2. Branch on the error field

   Denial bodies are JSON, and the error field says why. Check it first, then choose the action. 402 means a payment-side fix: invalid_token, insufficient_balance, or a price negotiation state. 403 access_blocked means the publisher withholds the resource from crawlers, and no payment unlocks it. Keep the branching in one place so every URL in a crawl is handled the same way.

   import requests
   
   url = 'https://PUBLISHER-SITE/research/report.data.json'
   r = requests.get(url, headers={
       'X-Crawler-Token': 'YOUR_CRAWLER_TOKEN',
       'User-Agent': 'MyBot/1.0 (+https://example.com/bot)'})
   
   if r.status_code in (402, 403):
       body = r.json()
       err = body.get('error')
       if err == 'invalid_token':
           print('fix or register the token, retry same URL')
       elif err == 'insufficient_balance':
           print('shortfall:', body.get('crawler', {}).get('shortfall'))
       elif err == 'access_blocked':
           print('skip this URL, check pricing.json')
       else:
           print('quote:', body.get('price'), body.get('currency'))

   S2 · esa.aisa.one

3. Resolve 402 invalid_token and insufficient_balance

   For invalid_token, register or fix the header, then retry the same URL; the guidance.steps list walks through it. For insufficient_balance, crawler.shortfall says how much more is needed. The agent must not fund itself: create a Stripe checkout session (minimum 0.50), give the checkout_url to the user, and wait. Fetch the real balance before reporting any figure, using the same crawler token.

   # Real balance, same crawler token you crawl with
   curl -s 'https://cdn.aisa.one/api/agent/balance' \
     -H 'X-Crawler-Token: YOUR_CRAWLER_TOKEN'
   
   # Top-up link for the user (session token from login)
   curl -s -X POST 'https://cdn.aisa.one/api/agent/topup/create-session' \
     -H 'Content-Type: application/json' \
     -H 'Authorization: Bearer SESSION_TOKEN' \
     -d '{ 'amount': 5.00 }'

   S1 · esa.aisa.oneS2 · esa.aisa.oneS3 · esa.aisa.one

4. Follow the guidance block verbatim

   Every denial can carry a machine-readable guidance block with onboarding, docs, llmsTxt and skill links, a registerApi entry, numbered steps, a signupBonus notice when a trial credit is active, and agentAdvice. Follow guidance.steps as written, because the same system judges your retry. If guidance.registerApi names a different AISA host, prefer that host. Narrate the steps to your user as agentAdvice suggests.

   guidance.onboarding    onboarding guide link
   guidance.docs          documentation link
   guidance.llmsTxt       llms.txt link
   guidance.skill         skill.md link
   guidance.registerApi   register endpoint for this site
   guidance.steps         numbered steps, follow in order
   guidance.signupBonus   present only when a trial credit is active
   guidance.agentAdvice   what to narrate to the user

   S2 · esa.aisa.one

5. Pivot on 403 access_blocked

   A 403 access_blocked is a publisher policy choice, not a payment problem. Do not retry that URL and do not top up for it. Do not stop the task either: fetch the free /pricing.json, which lists the purchasable pay-mode directories, and continue with those. Skipping a blocked URL and buying pay-mode content is the intended use of the API, not a bypass.

   curl -s 'https://PUBLISHER-SITE/pricing.json' \
     -H 'X-Crawler-Token: YOUR_CRAWLER_TOKEN'

   S2 · esa.aisa.oneS3 · esa.aisa.one

6. Tell the user what happened

   Narrate in plain language. Say whether content was free or paid, the exact amount charged, and the remaining balance fetched live, not remembered. For insufficient_balance, state the shortfall and share the checkout link. For access_blocked, explain that the publisher chose to withhold that resource from crawlers, that payment cannot change it, and what you are fetching instead. Never print the full token; show a prefix like aisa_crawler_9a4d...

   Balance is out: you need a little more to continue. Open this link to add funds: CHECKOUT_URL. After you pay, I will retry.
   
   The publisher does not allow crawlers on this page, and payment cannot change that. I will use its purchasable sections instead.

   S2 · esa.aisa.oneS3 · esa.aisa.one

AISA denial codes and next actions

Common pitfalls

• Sending the crawler token in Authorization: Bearer makes the request unauthenticated, so you get 402 invalid_token. Use X-Crawler-Token only.
• Never call POST /api/agent/balance/add to top yourself up. It is for human operators seeding test accounts; real funding goes through the Stripe checkout the user pays.
• Quoting a balance from memory goes stale after the first spend. Call GET /api/agent/balance before reporting any figure, and reuse one account throughout.
• Do not give up on a whole task because one URL returned 403 access_blocked. Skip it and move on to the site's pay-mode directories.

Key takeaways

• Read the error field first: invalid_token and insufficient_balance are fixable with payment-side action, access_blocked is not.
• Denials never charge. Follow guidance.steps, and prefer the host in guidance.registerApi.
• Always give the user the free or paid status, the amount charged, and a freshly fetched balance.

FAQ

How this page is made. Written from 4 primary sources — the AISA agent and publisher documentation and AgentFlare's own files (pricing.json, robots.txt, llms.txt, live responses, .data.json field lists). Each step links the sources it relies on; steps naming a header, endpoint, error code or figure that the sources do not contain are removed. Refreshed weekly.