How do I use robots.txt to disallow folders which are being crawled due to a bad URL structure? They're currently causing a duplicate page error.
The URL which been crawled incorrectly is:
However, The correct page is:
Is robots.txt a correct way of excluding this? I'm thinking about using:
Will that not block legitimate content in the
/forum/ folder of my site?