crawl · crawl.robots.syntaxErrors
robots.txt has syntax errors
Malformed directives are handled inconsistently between crawlers — some ignore the line, others misread the whole file.
Why it matters
Malformed lines may be ignored or, worse, change crawler behavior unpredictably across user agents.
Authoritative reference
https://developers.google.com/search/docs/crawling-indexing/robots/robots_txt
Run a full audit of your site
Free, no signup, no crawl cap. Check this and 90+ other factors across every public page.
Audit my site →