Is that the way to go if I want to visually see what the googlebot sees? I'm trying to check a folder which I have just blocked in my robots.txt.
- If I fetch the folder as googlebot, it fetches ok, so that doesn't tell me nothing about whether the block is working
- I know there's a tool to check for blocking, but it is dependent on the input of the robots.txt
Therefore I've tried the Instant Preview, and I don't get a preview for what the bot sees ("pre-render), so I think that means that it's because the robots.txt blocks it; however - I don't see the bot tried beforehand to access my updated robots.txt, so I'm not sure how does it know that this folder is blocked? (it does preview another new folder, that is not blocked)