1
vote
1answer
110 views

How can I set my robots.txt file to let bots not crawl certain folders in root directory?

I have a small website with about 20 pages with .html extensions, containing unique articles I wrote consisting of about 1000 words or more. These 20 pages are the main feature of my site. Root ...
6
votes
1answer
2k views

.htaccess rule to redirect from 404 to search results

When a user (or a bot) request on non existing page, I want to redirect him to the search results. I prefere to do it with the .htaccess file using something like: ErrorDocument 404 ...