Search type Search syntax
Tags [tag]
Exact "words here"
Author user:1234
user:me (yours)
Score score:3 (3+)
score:0 (none)
Answers answers:3 (3+)
answers:0 (none)
isaccepted:yes
hasaccepted:no
inquestion:1234
Views views:250
Sections title:apples
body:"apples oranges"
URL url:"*.example.com"
Favorites infavorites:mine
infavorites:1234
Status closed:yes
duplicate:no
migrated:no
wiki:no
Types is:question
is:answer
Exclude -[tag]
-apples
For more details on advanced search visit our help page
Results tagged with Search options user 65902

Robots.txt is text file used by Website owners to give instructions about their site to web robots. Basically it tells robots which parts of the site are open and which parts are closed. This is called The Robots Exclusion Protocol.

1
vote
1answer
frequency <url> <loc>https://www.myurl.com/reportlist.html</loc> <changefreq>Weekly</changefreq> </url> I forgot to update my robots.txt initially for an allow rule as I had a disallow on … / Allow: /includes/getreport.php My Search Console is now showing the following under Index Status so the links to the PDF's are being crawled, but apparently links are blocked by robots.txt from being …
asked Jun 17 '16 by LJT