We Checked 152 B2B Software Sites for AI Readiness. 86% Failed.

Everyone selling AI search optimisation opens with the same warning: the AI crawlers cannot read your site. It is a good opening because it is frightening and because it sounds technical enough to be true. We wanted to know whether it actually was, so we built the checks and ran them against 152 real B2B software company sites. The alarming part is not what we found. It is what we did not find.

What we checked

Four things, all mechanical. No model was asked for an opinion at any point, which means every number here is one you could reproduce yourself this afternoon.

• Crawler access. Whether robots.txt permits the engines that answer live questions: OAI-SearchBot, ChatGPT-User, PerplexityBot, Claude-User, and the training crawlers behind them.
• llms.txt. Whether the site publishes a plain-text map of itself at the agreed location.
• Schema. Whether the structured data describes the business, or only announces that a web page exists.
• Server HTML. How much of the page arrives before any JavaScript runs, and whether there is an H1 in it.

The sample was 166 company sites. Fourteen were dropped because their homepage never responded during the run. Counting a site that was down as failing four checks would have been our instrument reporting its own outage, so those are excluded rather than scored. That leaves 152.

The finding nobody sells

Of 166 domains, one blocked an answer engine in robots.txt. One. Ninety-six percent scored a perfect hundred on crawler access.

We did not believe it either, so we validated the checker against a control group of news and media sites. There, 88% block at least one answer engine, and several block every one of them. The instrument works. Publishers block AI crawlers because their business model depends on it. B2B software companies, almost without exception, do not.

So the industry's favourite opening line is, for this audience, a dud. If someone runs a scan on your site and leads with "good news, you are not blocking GPTBot", they have told you something true about 96% of your peers.

What actually separates sites

The checks that did discriminate, across the 152 reachable sites:

• 78% publish no llms.txt.
• 32% have no schema that describes the business. Half of those have no JSON-LD at all. The other half emit only the plugin default, which says a page exists and says nothing about what the company sells.
• 16% have no H1 in the server HTML.
• 9% send under 250 words before JavaScript runs.
• 86% fail at least one of the four.

The median homepage sends 680 words of readable text before any JavaScript executes.

The schema number is the one to care about

Thirty-two percent is the figure we would act on first, and it is worth being precise about what it means.

A site running a standard content management system with a standard SEO plugin will emit structured data automatically. It will say: this is a web page, it belongs to a website, here is a breadcrumb trail, here is an image. All true, all useless to a model trying to work out what your company does and whether to name you in an answer.

What is missing is the type that carries meaning: Organization, Product, SoftwareApplication, FAQPage, Service. Sixteen percent of sites emit nothing at all, and another sixteen percent emit only the furniture. To a model grounding an answer those are the same condition, which is why we count them together.

This is also the cheapest thing on the list to fix. It is a block of JSON in the head of your pages.

The 9% with a rendering problem

Nine percent of these sites send fewer than 250 words in their server HTML. Open one in a browser and it looks complete, because your browser runs the JavaScript. Most crawlers do not, or do so on a delay and a budget.

If your homepage is a shell that assembles itself client-side, the version most machines read is close to empty. This is not an AI-era problem, it is an old problem that the AI era made expensive again.

What we would do about it

In order, for a typical B2B software site:

1. Check what your schema actually says. View source, search for application/ld+json, and read the @type values. If they are all WebPage and BreadcrumbList, you have furniture and no description.
2. Check your server HTML. Run curl -s https://yoursite.com | wc -w. If the number is small, your content is arriving by JavaScript.
3. Confirm there is an H1 in that raw HTML, not just in the rendered page.
4. Then, and only then, consider llms.txt. It has the highest failure rate on our list and the lowest stakes, for reasons worth their own article.
5. Do not bother panicking about robots.txt unless you are in the 1%. Check it once, in ten seconds, and move on.

The honest caveat

These checks read what a server sends. They do not prove a model will cite you, and nobody selling you a score can prove that either. What they measure is whether the material is there to be read at all. That is a floor, not a forecast, and any vendor who tells you otherwise is selling a number they cannot stand behind.

Run it on your own site

We turned these checks into a free tool. Enter your URL and four of them run on screen in about two seconds, with the evidence for each one so you can verify it yourself rather than take our word for it. Check your site against the same four checks, or see how we handle this as part of a monthly marketing plan if you would rather it was somebody else's job.