2

Which one is correct?

User-agent: *
Disallow: index.php

or

User-agent: *
Disallow: /index.php

inorder to block all pages like index.php?day=7 index.php?day=15 index.php?day=30

2 Answers 2

3

The second one is better form as it clearly marks the index.php as being the in web root and not in some other subdirectory.

2
  • Isn't the robots.txt used to block directories and not files? Feb 9, 2012 at 0:02
  • It can block both
    – John Conde
    Feb 9, 2012 at 0:12
2

You can use the following robots.txt file if you wish to block all robots from accessing your pages.

# robots.txt for www.domain.com to block all bots

User-agent: * 
Disallow: /

If you wish to block all all URLs that include a question mark (?) you can use the following format

# robots.txt for www.domain.com to block URL with ? mark

User-agent: *
Disallow: /*?

For more detailed information on how to write robots.txt visit the following link How to Create Robots.txt Files

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.