How this site knows what it says

A map of the AI training data market, from every page you read down to its source.

Everything on this site sits on one of five levels, and each level is built only from the one below it. Click any number anywhere on the site and it takes you down: to the figure, then to the records it is computed from, then to the sources they were read from, and to a permanent copy of each source.

  1. Level 5

    Content

    The pages, guides, emails and datasets that show figures and records, each linked down.

    The market board, the valuation, the guides, the weekly brief, open datasets and the MCP server: they show figures and records by reference, never a typed number.

  2. Level 4

    Figures

    Every number computed from records, with its derivation.

    338 figures: counts, medians, ranges, ratios and the valuation model's parameters, each with its derivation and inputs.

  3. Level 3

    Entities

    The organisations, kinds of data, holders and business systems the records are about.

    300 organisations, 9 kinds of data, 8 kinds of holder, 18 business systems and the companies closing.

  4. Level 2
  5. Level 1

    Sources

    Every page, filing, docket, article and report the site reads, each with a permanent copy.

    2,370 sources, 590 of them with a permanent copy in the Wayback Machine, read by the pipelines below.

The pipelines that keep each level current

Each pipeline reads on a schedule, writes one level and feeds the next. None of them types a number into a page.

  • Market research

    Every day

    Reads
    Buyers' and vendors' own pages, regulatory filings, court dockets, the press, research, and the AITraining.jobs pay medians feed.
    Writes
    Level 2. Price observations, negotiation levers and supply facts in the evidence store, each with its source's own words.
    Feeds
    Wayback archive, Figures, Weekly brief, Open datasets, MCP server, Feed and search announcements

  • Source watch

    Every day, before the research

    Reads
    Every source a current record cites, for the words each record was read from.
    Writes
    Level 1. Which sources changed or lost a figure, for the day's research to re-read first.
    Feeds
    Market research
    Shown on
    Sources

  • Wayback archive

    Every day, after the research

    Reads
    Every cited source without a permanent copy, and the Internet Archive's captures of it.
    Writes
    Level 1. A permanent copy of each source from near the day it was read, so a citation always shows what the page said then.
    Feeds
    Figures
    Shown on
    Sources

  • Legal watch

    Every week

    Reads
    Every statute, regulation, judgment, guidance page and counterparty's published terms the legal positions cite, for the words each was quoted in.
    Writes
    Level 1. Whether each authority's words are still on its page; the legal check fails on one that is not, until it is read again.
    Feeds
    Wayback archive

  • Ledgers

    Every morning, from the money watch's leads

    Reads
    Announcements, filings, dockets and the press about deals, lawsuits, funding, revenue, buyer programmes and contract terms.
    Writes
    Level 2. Licensing deals, court cases, funding events, financial figures, market records, buyers' terms and clauses, rights terms.
    Feeds
    Figures, Weekly brief, Open datasets, MCP server, Feed and search announcements

  • Holder watch

    Every morning

    Reads
    The Gazette (UK insolvency notices), Hacker News, Y Combinator directory, Site watch, SEC filings (8-K), California WARN notices, Companies House strike-off proposals, Companies House: in administration or liquidation, Sites that turn away AI crawlers, Hugging Face non-commercial datasets, News headlines, and each company's site in the Wayback Machine.
    Writes
    Level 3. Companies whose records may sell: found the day something happened to them (an insolvency, a strike-off, a shutdown, a sale, a site gone dark) or because they already guard data AI developers want, each with its history, its register facts and who decides.
    Feeds
    Closures register, Outreach desk
    Shown on
    Closures

  • Company intelligence

    Every morning, after the holder watch

    Reads
    For every company on the lead list, what is due of: UK register match (every 30 days), UK register snapshot (every 30 days), UK officers and owners (every 30 days), UK insolvency cases (every 14 days), UK filed accounts (every 14 days), The company's own site (every 30 days), Wayback Machine (every 30 days), Y Combinator directory (every 30 days), SEC company filings (every 30 days).
    Writes
    Level 3. Each company resolved to one record (its number or name in each register and directory, two leads for one company merged), every fact with its source and the day it is true of, who decides about its records, and a timeline of every change.
    Feeds
    Outreach desk

  • AI crawler census

    Every morning, with the holder watch

    Reads
    The robots.txt of one slice of the 300,000 most visited sites on the Tranco list (4,000 a day, the whole list in about 45 days).
    Writes
    Level 2. One record per sweep: how many sites answered and how many shut each AI crawler out. The sites that do are holders, written to the holder watch.
    Feeds
    Holder watch, Figures
    Shown on
    AI crawlers

  • AITraining.jobs pay feed

    Every morning

    Reads
    The median advertised hourly rate per kind of AI training work, recomputed daily by AITraining.jobs (same owner as this site, disclosed on every row).
    Writes
    Level 2. The day's pay medians as price observations, kept current in place, with their history in a daily series.
    Feeds
    Figures
    Shown on
    Rates

  • Integrity checks

    Every morning, after the pipelines

    Reads
    Every page, figure and record the site publishes.
    Writes
    Level 4. Nothing: it fails loudly on a token that names nothing, a figure with no inputs, a number typed into a page, a store past its cadence or a search-engine integrity problem.
    Shown on
    Figures

  • Money watch

    Every morning, before the research

    Reads
    The news of the last days for every company the site tracks (rounds, financing, acquisitions, run-rates, budgets to buy data) and for the market (licensing deals, data sales, lawsuits and rulings over training data).
    Writes
    Level 2. Leads only: each is read on the company's own page, the docket or named press before it becomes a funding event, a financial figure, a deal, a case or a market record.
    Feeds
    Ledgers

  • Lot index

    Every morning, with the research

    Reads
    Every public catalogue of data on offer: Grepped marketplace, Appen data catalog, Kled dataset library, each read whole.
    Writes
    Level 2. One record per catalogue per day (how many lots it listed, which were new, which left it), and every lot side by side in this site's terms.
    Feeds
    Figures, Pages and guides

  • Closures register

    Every morning, after the holder watch

    Reads
    The companies the holder watch found that our review confirmed, from public signals only.
    Writes
    Level 3. A public page for each technology company closing, sold or gone dark, with every signal linked.
    Feeds
    Figures
    Shown on
    Closures

  • Figures

    On every publish

    Reads
    The records.
    Writes
    Level 4. Every count, median, range, ratio and model parameter the site shows, with its derivation and inputs.
    Feeds
    Pages and guides
    Shown on
    Figures

  • Pages and guides

    On every publish

    Reads
    Figures and records, by reference (never typed).
    Writes
    Level 5. The site's pages and guides.
    Shown on
    Home

  • Outreach desk

    Every day

    Reads
    Leads our review marked worth an approach, and the people who decide about them (licensed people search).
    Writes
    Level 5. One email per company, naming it, every figure in it read from the records.
    Shown on
    For estates

  • Weekly brief

    Every Monday

    Reads
    The week's new records.
    Writes
    Level 5. One issue to every confirmed subscriber, each item linked to its record.
    Shown on
    The brief

  • Open datasets

    When the tables change

    Reads
    Every table the site serves.
    Writes
    Level 5. CSV and JSON downloads, and dated snapshots with a DOI at Zenodo, Figshare and the Internet Archive.
    Shown on
    Datasets

  • MCP server

    Live

    Reads
    The records and figures.
    Writes
    Level 5. Answers for AI assistants, each with a link to its record.
    Shown on
    MCP server

  • Feed and search announcements

    Every day

    Reads
    The day's new and changed pages.
    Writes
    Level 5. The Atom feed and IndexNow pings, so search engines read new records the day they appear.
    Shown on
    Feed

How current each store is

The day each store was last read against its sources, and how often it has to be. A store that misses its day is the first thing the next morning's run brings current.

In this topic

Research

  1. Start hereWhere does every number come from?The evidence underneath everything: every source, every record, every derived figure, the method, and the open datasets.
  2. GuideHow We Source and Check Every Number on This SiteEvery number here links to a record, every record to its source's own words, re-read daily. How sources are weighed, figures derived and guides built on them.
  3. GuideWhat Is Training Data Provenance, and Why Buyers Now Ask for ItTraining data provenance is the record of where data came from, the rights it was collected under and what was done to it. Buyers now ask for it in writing.
  4. RegisterRecordsEvery fact about the market, each with its source.
  5. RegisterSourcesEvery page, filing and report the records were read from.
  6. RegisterFiguresEvery number on the site and how it is computed.
  7. PageDatasetsThe registers as open datasets to download and cite.
  8. ToolMCP serverAsk an AI assistant for prices, buyers and deals from these records.
  9. PageAll guidesEverything published here, as one tree.

Start from a number, a record or a source: each links to the rest.

Every figureEvery recordEvery source
Map of the AI training data market: sources, records, figures | AIData.Broker