When you are building a new website from scratch, all links should of course first work just fine. There is basically now reason – or let’s rather say no excuse for broken links in a fresh setup. All internal and external links should be checked and there should be not a single broken link on a fresh install.

When you are instead moving moving or even relaunching a website, things are quite different. Chances are that sections have changed, URLs are different – or some small pages may be combined to one larger page. Then you should try to ‘save’ potentially valuable links, for example by redirecting them properly trough 301-redirects inside the .htaccess file of your website. But these are not the subject of this article.

Why Website-Care / Website-Maintenance is important

In this post I want to look at websites, that have been up and running for a longer time period. Articles and pages have been added over the years, internal and external links have been set. The problem is: over the years URLs may have changed, pages may have been deleted. Some of your links may be outdated – the number of broken links is slowly but surely rising.

You may first not realize you have a problem. But while the number of broken links is rizing, your website’s reputation may drop. Both users and search engines don’t like dead links at all. Dead links are bad for UX and bad for SEO.

And even if it does not appear to be a problem at first sight, because ‘nobody’ is looking at your old articles anyways… Well – the reason why nobody is actually looking at those old articles might be the high number of broken links.

So both for UX and for SEO it is a good idea to have a closer look at you website from a spider’s point of view and check all those precious links you once set. I don’t like the term myself, but let’s call it Link-Checking. But this can be a lot of annoying and (even worse) time consuming clicking, right? Of course there are some friendly bots or actually spiders that can do the job for you.

Link-Checking: how to find broken links and avoid 404 errors

First things first:

• Link-Checking is done best by some bot / spider / app – not manually. Even if your website is that that large, it may take hours or even days to actually find and check every link you once set. And chances are that you will not even find every dead link.
• Link-Checking is best done from ‘outside’. Consider Link-Checking something like an automatic site critique – and the best critics are often not the once that are running the system. So even if you CMS is capable of checking links, you are probably better off with an external service.
• TLDR / my favorite free service at this moment: deadlinkchecker.com

WordPress Plugin: Broken Link Checker

As far as I know there is currently only one WordPress Plugin that is capable of finding and listing broken links on your “WordPress) website: the Broken Link Checker. One disadvantage jumps right into my face: the Broken Link Checker can currently not check sidebars / widgets for broken links. This non-feature make the plugin actually useless for my use case.

Additionally the plugins has received quite some negative ratings for being resource hungry. There seems to be a serious performance problem when checking larger websites for dead links, up to server unavailability – the server might just not accept any further requests, shuts down, gives an error 500. Something I would not want to see on my website.

If you want to test the plugin anyways you’ll find it here: wordpress.org/plugins/broken-link-checker

Find dead links with Screaming Frog SEO Spider (Cross-Platform Program)

First the good news about the program Screaming Frog SEO Spider: the paid version seems to be a really complex application that enables you to do much more good for your website than just link checking. And it is available for Mac OS, Windows and Linux. The bad news: the free version of Screaming Frog SEO Spider is almost useless.

I have downloaded and installed the free version of Screaming Frog SEO Spider and tried what you can actually do with it. It is a stand alone application and you could basically use one license to check any number of websites ‘from the outside’. The only draw back is that you can only check one URL at a time. So if you only want to check your links page, it may be worth installing the free version. But if you want to check more URLs, then the free version is just not an option.

The paid version appears to be a mighty chunk of software – a SEO suite for professionals that regularly need to perform various tests on a number of websites. To me this is a bit over the top – and with a price tag of $99 per year it is just a bit too expensive for my rather basic needs.

If you want to test the program and/or download the free version: screamingfrog.co.uk/seo-spider

Find broken links with integrity (Mac OS Stand Alone Program)

The program integrity from Peacockmedia is a beautiful little piece of software. The interface design is cleanand compact and analyzing links is basically possible within the free version of the tool. You just need to enter the URL of the website you want to analyze and then it will take a few minutes until the tool has analyzed the website for broken links.

You can then get a list of all links – or just all broken links, responsible for the  404 errors. The sorting and filtering options of integrity lets you filter the results by response / error message.

One draw back: the tool will only list linked websites and pages – but not other files like JavaScript or webfonts and the like. Also feeds (RSS/XML) are not listed. So while the tool is actually quite good in giving a quick and direct answer to the question, what links are causing 404 errors, the tool is not good for a deep analyzes of all links assets.

You can find the tool here: peacockmedia.software/mac/integrity

Online Service: Link-Checker by w3.org

Next I took a look at the official link checker tool by w3.org. First a few advantages: the tool is free to use, no registration needed, no fees, no fuss. And the tool is a serious link checker: it basically checks everything it can – and that may take time.

Disadvantages: it can take quite a while until the link checker tool has analyzed a website for broken links. No doubt, the spider does its job well – but for my taste it does its job just a bit too well. So for a first test it could be a good idea to check the “Summary only” option. In some cases people report that the tool made their webserver break down – and you don’t want to get an error “500” looking for “440” errors.

If you want to test the tool anyways: validator.w3.org/checklink

Online Service: Dead Link Check

The Dead Link Check is another free online tool that lets you search your website for broken links. You just need to enetr the URL – then a CAPTCHA check. The the tool statrs analyzing…

After about 2500 pages you may get another CAPTCHA check – and if you have a large website to analyze, you may get more CAPTCHA checks on the way. In the end you get a listing of all problematic pages.

The link report does also list broken links to assets like JavaScript files, CSS files, webfonts and the like. This can come very handy since the options of finding broken links to assets are otherwise limited. For example: I head to learn that one WordPress plugin was generating feeds for a custom post type – but the feeds were not accessible. This led to literally hundreds of broken links, probably only visible to search engines. The solution to fix those broken links is documented in this forum post. Without the link checking I would probably have never found out that I had a problem in the first place.

Here is the tool: deadlinkchecker.com

Favorite Tool: Online Broken Link Checker

Also the Broken Link Checker is a free online tool. Running the broken link analyzer can take some time – but in the end it does pay off.

The Broken Link Checker is probably a bit faster than the Dead Link Check. The report is however not as detailed, but still it is a good option to run a test every now and then. It does its job.

Here is the tool: brokenlinkcheck.com

…well then… Happy Link-Checking! :)