I want to allow Google bots to crawl only my index.php and the content of some folders. Would that work?
User-agent: * Allow: /index*.php Allow: /folder1/ Allow: /folder2/ Allow: /folder3/ Allow: /folder4/ Disallow: /
The idea is for Google to crawl URLs like:
and exclude everything else.