An accidental noindex instruction is one possible reason a public page is excluded from search. Inspect the production response before assuming AI search or your framework is responsible.
Check both places an instruction can appear
Inspect the robots meta tag in the page’s HTML and the X-Robots-Tag HTTP response header. A CDN or host can add a header even when the page template looks correct. Compare the live domain with staging and document where the instruction originates.
curl -I https://your-site.example/
# Inspect X-Robots-Tag in the headers.
curl -L https://your-site.example/
# Inspect <meta name="robots" ...> in the HTML.Remove it only from pages meant for search
Correct the production setting or deployment rule, then fetch the page again. Keep exclusions on account screens, private reports, and other pages that should stay out of results. A change to the source repository is not enough until the production response reflects it.
Rescan and request inspection
SiteReadyFor.SI checks for noindex in fetched page metadata and response headers. Use a repeat audit to check the signal after deployment. For actual indexing status and recrawl requests, use the search engine’s own inspection tools. Removing noindex does not guarantee immediate inclusion or a particular ranking.