Can I combine robots.txt rules on one line, separated by whitespace?
Will this work?
User-agent: * Disallow: /
No. See the Google Robots.txt specifications, specifically the File Formats section.
So line breaks between lines are necessary.
The file MUST consist of one or more records separated by one or more blank lines.
Robots.txt must use this format:
Which looks like:
Or without optional spaces and more line breaks: