As I'm new and do not have much insight about security what I have found while scraping websites using scrapy and such tools is that most of the websites like google while naming there image or other files have a long string of characters attached to them in their urls.Like a url of image from google will have a structure like - lh3.googlecontent.com/b/img/svwjhhjwhbwJNShvxwnkjbwjbxwhbxjHBhwbxjhwbx.png

Does it make it safe from getting crawled in bulk by using tools like wget etc because I have never found a problem crawling them or getting them downloaded.

closed as off-topic by Simon Hayter Jul 13 '18 at 20:19

  • This question does not appear to be about webmastering within the scope defined in the help center.
If this question can be reworded to fit the rules in the help center, please edit the question.

  • Sorry but this question is off-topic because it is not about a website you operate. Also, security-related questions are best asked on the Security Stack. – Simon Hayter Jul 13 '18 at 20:19
  • Thanks.. I believe security stack had a similar one..so that helped. – Vishal Sharma Jul 13 '18 at 23:11