Using AgentFlare Newswire and Markets Data

This guide is for developers and agent builders who want structured news and stock data from AgentFlare. You will learn which archive URLs exist for the Newswire and Markets datasets and what fields each file holds. You will also see the per-request price of each directory and fetch a file with the X-Crawler-Token header. By the end you can request a dated digest or a markets snapshot and read headlines, stories, prices, sectors and valuation fields in code.

How the flow works

Before you start

• A crawler token registered at esa.aisa.one
• curl installed, or Python with the requests package
• Basic familiarity with JSON

Step by step

1. Pick the dataset and archive URL

   Both datasets use dated files plus an index. Newswire has one file per day and cluster: ai, crypto, earnings, macro and markets, plus one file per item at /news/<slug>.data.json. Markets has one snapshot per day and one file per ticker, such as aapl.data.json or xom.data.json. Replace YYYY-MM-DD with the date you want. Each directory also has an index.data.json.

   https://agentflare.org/news/index.data.json
   https://agentflare.org/news/YYYY-MM-DD-ai.data.json
   https://agentflare.org/news/YYYY-MM-DD-crypto.data.json
   https://agentflare.org/news/YYYY-MM-DD-earnings.data.json
   https://agentflare.org/news/YYYY-MM-DD-macro.data.json
   https://agentflare.org/news/YYYY-MM-DD-markets.data.json
   https://agentflare.org/markets/index.data.json
   https://agentflare.org/markets/YYYY-MM-DD.data.json
   https://agentflare.org/markets/xom.data.json

   S2 · agentflare.orgS3 · agentflare.org

2. Check the per-request price

   Index pages, .html articles and .md teasers are free. The complete .data.json and .full.md files are the paid L2 tier, metered per request. The pricing map lists news at 0.005 USD and markets at 0.01 USD, both with billingType per-request. The Newswire rule matches paths under /news/ and the Markets rule matches paths under /markets/. Read the live map before you budget.

   curl -s https://agentflare.org/pricing.json

   S4 · agentflare.orgS5 · agentflare.org

3. Fetch a dated file with your token

   Send your crawler token in the X-Crawler-Token header. This name replaces the earlier X-AISA-Crawler-Token that older AISA pages still show. Never put the token in Authorization: Bearer, which is reserved for the gateway layer. The site notes that agents set crawlerAutoPrice=true with an X-Crawler-Token header, and payment settles over the AISA HTTP 402 protocol. Replace YYYY-MM-DD with a real date.

   curl -s 'https://agentflare.org/news/YYYY-MM-DD-ai.data.json' \
     -H 'X-Crawler-Token: YOUR_CRAWLER_TOKEN'

   S1 · esa.aisa.oneS5 · agentflare.org

4. Read the Newswire fields

   A dated Newswire file has envelope fields such as title, description, canonical, category, updated, generated_at and facts. The data object carries date, cluster, headline, dek, freshness_hours, stories, market, prediction_markets, watch, takeaways, faq, count and items. Each story has title, summary, why, sources and entities. Each item has title, url, source, date, tickers and iso.

   import requests
   
   url = 'https://agentflare.org/news/YYYY-MM-DD-ai.data.json'
   headers = {'X-Crawler-Token': 'YOUR_CRAWLER_TOKEN'}
   doc = requests.get(url, headers=headers).json()
   
   print(doc['data']['headline'])
   print(doc['data']['dek'])
   for story in doc['data']['stories']:
       print(story['title'], '|', story['why'])
   for m in doc['data']['market']:
       print(m['id'], m['last'], m['d1'])

   S1 · esa.aisa.oneS2 · agentflare.org

5. Read the Markets fields

   The dated Markets snapshot holds data.count, data.advancers and data.decliners, a data.sectors list with label, value and n, and a data.stocks list. Each stock has ticker, price_usd, day_change_pct, return_6m_pct, market_cap_usd, pe and sector. A per-ticker file such as xom.data.json adds returns_pct (m1, m3, m6), a long metrics object and a company object with cik, sector and industry.

   import requests
   
   url = 'https://agentflare.org/markets/YYYY-MM-DD.data.json'
   headers = {'X-Crawler-Token': 'YOUR_CRAWLER_TOKEN'}
   doc = requests.get(url, headers=headers).json()
   
   d = doc['data']
   print(d['count'], d['advancers'], d['decliners'])
   for s in d['stocks']:
       print(s['ticker'], s['price_usd'], s['day_change_pct'], s['sector'])

   S1 · esa.aisa.oneS3 · agentflare.org

Key fields and price by dataset

Common pitfalls

• Do not send the crawler token in Authorization: Bearer. That header is reserved for the gateway layer; use X-Crawler-Token.
• Older AISA pages still show X-AISA-Crawler-Token. The current header name is X-Crawler-Token.
• Newswire market entries differ by cluster: the ai file has d30, while the crypto file has d7, symbol and name.
• Only .data.json and .full.md files are metered. Index pages, .html and .md teasers stay free, so do not budget for them.

Key takeaways

• Newswire .data.json costs 0.005 USD per request and Markets .data.json costs 0.01 USD.
• Dated files follow /news/YYYY-MM-DD-cluster.data.json and /markets/YYYY-MM-DD.data.json; per-item files use a slug or ticker.
• Authenticate with the X-Crawler-Token header, never Authorization: Bearer.

FAQ

How this page is made. Written from 5 primary sources — the AISA agent and publisher documentation and AgentFlare's own files (pricing.json, robots.txt, llms.txt, live responses, .data.json field lists). Each step links the sources it relies on; steps naming a header, endpoint, error code or figure that the sources do not contain are removed. Refreshed weekly.