How the Edge Verify API Decides Access

This guide is for publishers and edge-gateway integrators who want AgentFlare (AISA) to decide per request whether a crawler gets free access, must pay, or is blocked. You will see exactly what the gateway sends to the verify endpoint, which decisions can come back, what the gateway does with each one, and how a refusal teaches a new crawler to register, top up and retry. A table of verify outcomes closes the guide.

How the flow works

Before you start

• A gateway token issued by AISA, delivered out of band
• A publisher account with URL rules (allow, pay, block) configured
• Python with the requests library for the examples

Step by step

1. Send the verify call before origin fetch

   When a crawler hits a protected URL, the gateway makes an out-of-band POST to /api/v1/access/verify. Put the gateway token in Authorization: Bearer. Forward the crawler's token in X-Crawler-Token (it replaces the earlier name X-AISA-Crawler-Token, which older AISA pages still show). Pass the original crawler user agent too, because gateways rewrite User-Agent. The test host is https://esa.aisa.one; production is https://cdn.aisa.one.

   import requests
   
   resp = requests.post(
       'https://esa.aisa.one/api/v1/access/verify',
       headers={
           'Authorization': 'Bearer YOUR_GATEWAY_TOKEN',
           'X-Crawler-Token': 'YOUR_CRAWLER_TOKEN',
           'X-ESA-Original-Crawler-Agent': 'GPTBot/1.0',
           'Content-Type': 'application/json',
       },
       json={
           'publisherDomain': 'PUBLISHER-SITE',
           'resourceUrl': '/premium/article.html',
           'crawlerAutoPrice': True,
           'crawlerAgent': 'gptbot',
       },
       timeout=10,
   )
   print(resp.status_code, resp.json())

   S1 · esa.aisa.oneS3 · esa.aisa.oneS4 · esa.aisa.one

2. Read accessGranted and act on it

   A minimal integration reads one field. When accessGranted is true, serve the content by fetching from origin or cache. When it is false, do not serve anything; return statusCode and the response body to the crawler. The verify endpoint is a pure decision API: it returns JSON only and never page content, so your content stays with the gateway and origin.

   def handle(verdict):
       if verdict['accessGranted']:
           return fetch_from_origin()  # your own origin or cache fetch
       # denied: pass status and JSON body through unchanged
       return respond(verdict['statusCode'], verdict)

   S2 · esa.aisa.oneS3 · esa.aisa.one

3. Know how the rules decide

   Publisher rules are first-match-wins, and URL query strings are stripped before matching. A block rule returns 403 access_blocked whatever the token or balance. An allow rule returns 200 free with no token needed. A pay rule needs a valid crawler token and, with crawlerAutoPrice set to true and enough balance, returns 200 plus a charge. If no rule matches, the result is 200 default allow with no charge. Billing is per request.

   S3 · esa.aisa.one

4. Pass 402 and 403 bodies through untouched

   On denial, deliver the verify JSON to the crawler unmodified. A 402 body carries onboarding and a guidance block that tell a new crawler how to register, fund a balance (possibly with a signup trial credit) and retry the same URL with X-Crawler-Token. A 402 means payment can unlock the page; a 403 means the publisher blocked it and paying will not help. Never rewrite one into the other.

   S2 · esa.aisa.oneS3 · esa.aisa.one

5. Forward the price intent correctly

   Price intent travels in the request body. The three fields crawlerAutoPrice, crawlerExactPrice (for example USD 0.10) and crawlerMaxPrice (for example USD 0.50) are mutually exclusive, so send at most one. Sending none triggers price discovery: no charge, and a 402 price_required response with price and currency. An exact price that differs returns 402 price_mismatch; a maximum below the resource price returns 402 price_too_high.

   S2 · esa.aisa.oneS4 · esa.aisa.one

6. Separate gateway failures from crawler failures

   The two auth layers are independent. A missing or invalid gateway token returns 401 gateway_unauthorized, which is for the gateway operator to fix, not the crawler. A missing or invalid crawler token returns 402 invalid_token instead, because registering solves it. A bad price string returns 400 invalid_price_format, and a provider fault returns 500 internal_error. Reuse the same optional X-Request-Id on retries so both sides can reconcile charges.

   S2 · esa.aisa.oneS3 · esa.aisa.oneS4 · esa.aisa.one

Verify outcomes and gateway action

Common pitfalls

• Never send the crawler token in Authorization: Bearer; that slot is the gateway layer, and the call fails with 401 gateway_unauthorized.
• Do not mix 402 and 403. A block rule returns 403 even if the crawler pays, so its guidance should say not to register for that URL.
• Combining crawlerAutoPrice, crawlerExactPrice or crawlerMaxPrice in one request is invalid; the three price fields are mutually exclusive.
• The original user agent header is spelled X-Original-Crawler-Agent in the Open Verify specification and X-ESA-Original-Crawler-Agent in the AISA endpoint docs; match the endpoint you call.

Key takeaways

• The gateway enforces; AISA decides. One boolean, accessGranted, controls whether content is fetched from origin.
• Send the crawler token in X-Crawler-Token and keep Authorization: Bearer for the gateway token only.
• Forward 402 and 403 JSON unmodified, since its onboarding and guidance fields convert new crawlers.

FAQ

How this page is made. Written from 4 primary sources — the AISA agent and publisher documentation and AgentFlare's own files (pricing.json, robots.txt, llms.txt, live responses, .data.json field lists). Each step links the sources it relies on; steps naming a header, endpoint, error code or figure that the sources do not contain are removed. Refreshed weekly.