Tag Info

Hot answers tagged

14

I've used Xenu's Link Sleuth. It works pretty well, just be sure not to DOS yourself!


12

For Windows users: Xenu For *nix/Mac: wget (usually installed by default) Update: Typical wget usage: wget -r --spider -b -o /var/tmp/wget.log http://target.site.com/ -r - Recursive download --spider - Check downloaded content for new links, then discard -b - Run in the background -o /var/tmp/wget.log - Target log file http://target.site.com/ - ...


6

If you are using windows 7 the best tool is IIS7's SEO Toolkit 1.0. It is free and you can download it for free. The tool will scan any site and tell you where all of the dead links are, what pages take to long to load, what pages have missing titles, duplicate titles, same for keywords and descriptions, and what pages have broken HTML.


3

Try W3C's open source tool Link Checker. You can use it online or install it locally


3

Web apps that offer scheduled link checking with emailed test results: Nodeworks No tags, no glory! LinkAlarm All offer free trial versions or limited accounts with paid upgrade options. The W3C offers a free online link checker tool, the source code for which is available here, but this isn't an automated service -- you'd have to run the link ...


3

A1WebsiteAnalyzer is easy to use. You can automate via Windows scheduled tasks. I haven't used InSpyder but it's the same price and does the same thing.


3

I use a piece of software called Xenu Link Sleuth. I first used it about 10 years ago and the interface hasn't changed much in that time, but it works great for small to medium size websites. It will still work on large websites, but it will take an age to crawl tens of thousands of dynamically generated content pages.


1

Link Examiner is a really good freeware too for your need.


1

There are several products from Microsys, especially their A1 Sitemap Generator and A1 Website Analyzer that will crawl your website and report everything you can possibly imagine about it. That includes broken links, but also a table view of all your pages so you can compare things like identical <title> and meta description tags, nofollow links, meta ...


1

If "fetch as Google" is able to retrieve the pages without errors, then it sounds like Googlebot just hasn't gotten around to re-crawling all the pages yet. The errors will go away within a couple weeks as Googlebot recrawls the error pages. You can use the "mark as fixed" feature in Google Webmaster Tools. It will remove the errors from your list for ...


1

Yes, there are crawlers that can crawl a site which requires login. This requires that you log into your site using your web browser and export your cookies. Then you start the crawler with the cookies that you had used to log in and the the crawler crawls the site as your logged in user. To export your cookies, use Firefox with the Export Cookies ...


1

However, the change would have broken existing links. So, nicely, they stuck in 302 redirects. Well, the important bit here is that the change would have broken existing links. /commerce/product_info.php?products_id=9266 isn't the location of the product page anymore, /products/9266 is. It's not at all unreasonable to expect the old-style links to stop ...


1

Note: You'll get a better answer to this by emailing team@stackoverflow.com, they handle questions and issues with the validator. IMHO sparkfun have done the wrong thing SEO wise by using a 302, it should be a 301 (see "SEO: ecommerce item deleted by user, 301 redirect to HOME PAGE or 404 not found?"). At a complete guess perhaps the stack bot regards a ...


1

The W3C Link Checker is one of the oldest online services that checks for broken links.


1

You could also try A1 Website Analyzer (Windows tool, but can run in VMWare/Fusion under Mac) It can tell you of all URL references site-wide (e.g. which pages link to a 404 URL + usually also line numbers), HTML/CSS errors etc.


1

Google Webmaster tools will report on broken links. It's a simple tool, free to install to a website.....and platform agnostic. If you're not already using it, it can help you boost your SEO and useability as well. https://www.google.com/webmasters/tools/home?hl=en


1

I'm a big fan of linklint for linkchecking large static sites, if you have a unix command line around (I've used on linux, MacOS, and FreeBSD). See their site for installation instructions. Once installed, I create a file called check.ll and do: linklint @check.ll Here's what my check.ll file looks like # linklint -doc . -delay 0 -http -htmlonly -limit ...



Only top voted, non community-wiki answers of a minimum length are eligible