Timeline for Is it necessary to block googlebot and googlebot-video to block video crawl?
Current License: CC BY-SA 4.0
13 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jun 21, 2018 at 9:08 | comment | added | Tony McCreath | The disallow is a request for them to not crawl. The video may be considered for indexing. e.g. Google knows their is a video on the page and their may be mark up to tell Google what the video is about, and a thumbnail. That may be enough for them to still show it in search. | |
| Jun 20, 2018 at 16:48 | comment | added | GidSut | @TonyMcCreath Good approach. Thanks. So having Googlebot blocked means it shouldn't be able to crawl my video OR it will still crawl except won't take my video content into consideration? | |
| Jun 20, 2018 at 14:17 | history | tweeted | twitter.com/StackWebmasters/status/1009440195433910274 | ||
| Jun 20, 2018 at 14:04 | comment | added | Sandy Lee | Tony McCreath, good approach! | |
| Jun 20, 2018 at 12:37 | comment | added | Tony McCreath | Why not check your server logs to see which bots access the videos. | |
| Jun 20, 2018 at 12:18 | comment | added | GidSut | 1. Yes. like this 2. No. as we don't want to give more video "signal" to Google | |
| Jun 20, 2018 at 10:39 | comment | added | unor | 1) Do you mean their video rich result? 2) Do you provide the Schema.org structured data? | |
| Jun 20, 2018 at 10:33 | history | edited | unor | CC BY-SA 4.0 |
deleted 3 characters in body; edited tags; edited title
|
| Jun 20, 2018 at 10:27 | history | edited | GidSut | CC BY-SA 4.0 |
edited body
|
| Jun 20, 2018 at 10:27 | comment | added | GidSut | might be a little bit overkill. Just want to know what exactly does Googlebot and Googlebot-video crawl for, so I can apply appropriate directive. thanks for the suggestion. | |
| Jun 20, 2018 at 9:49 | comment | added | Stephen Ostermiller♦ |
Are there any crawlers you want downloading videos? Maybe use User-Agent: *
|
|
| Jun 20, 2018 at 9:35 | review | First posts | |||
| Jun 20, 2018 at 10:07 | |||||
| Jun 20, 2018 at 9:33 | history | asked | GidSut | CC BY-SA 4.0 |
