Nine reasons AI engines ignore your site
Most generative visibility problems are not subtle. They are access failures, rendering failures and contradictions, and each one is diagnosable in under an hour.
Key takeaways
- Most generative invisibility is an access or rendering failure, not a content one.
- CDN bot protection blocks crawlers at the edge while robots.txt still reads correctly; check it separately.
- Test by fetching your page without JavaScript; if the content is missing, that is the whole problem.
- A confidently wrong assistant answer indicates contradictory entity data and points at the stale source.
When a company is absent from AI answers the cause is almost never mysterious. In roughly the order we find them, here are nine reasons, most of which can be confirmed or eliminated in an afternoon without any specialist tooling.
Access failures
- AI crawler user agents blocked in robots.txt, usually collateral damage from a security tightening nobody reviewed for side effects.
- CDN or firewall bot protection blocking the same agents at the edge, so robots.txt reads correctly while the request never reaches your server. This one is invisible unless you check the CDN settings directly.
- Content behind an interstitial, a cookie wall or a consent gate that a crawler cannot pass, leaving it with a page containing nothing but the gate.
Rendering failures
- Main content rendered only after hydration. Most AI crawlers do not execute JavaScript, so the page they receive is close to empty. Test by requesting the page with scripting disabled and reading what comes back.
- Navigation and internal links that exist only in client-side routing, so nothing beyond the entry page is discoverable.
- Slow responses that time out. A crawler with a budget does not wait.
Content failures
- The answer buried below preamble. Retrieval takes the first clear statement; if that is marketing throat-clearing, that is what represents you, and it will lose to a competitor who answered immediately.
- Claims with no figures, sources or dates. Nothing quotable means nothing quoted, however well-written the prose.
- Contradictory entity data across your site and profiles, forcing a resolver to guess which description of your company is authoritative.
The diagnosis, in order
Request your most important page as a plain HTTP fetch with no JavaScript and read the output. If the content is not there, stop; you have found the problem. If it is, read your live robots.txt and check each AI user agent you care about. If that passes, check the CDN bot rules. Only when all three pass is it worth looking at the content itself.
Then ask three assistants what your company does. A confidently wrong answer tells you the entity data is contradictory and usually points at which stale source is winning. Silence tells you that you are not being retrieved at all, which sends you back to access and to whether you rank for anything in the first place.
The ranking prerequisite people skip
One cause sits underneath several of the others: the site does not rank for anything. Because several assistants ground their answers in live search results, search visibility is often the mechanism by which you enter the candidate set at all. A site with no organic visibility asking why it is not cited in AI answers has a search problem being diagnosed as a generative one, and no amount of formatting work will resolve it.
Fix in this order
Access, then rendering, then search visibility, then content, then entity consistency. Working in that order means each fix is meaningful when you make it. Working in the reverse order, which is the common instinct because content feels like the controllable part, means doing careful editorial work on pages that nothing can reach.
If the page arrives empty without JavaScript, stop looking. You have found it.
Sources