One of the most common scenarios where you might need to accomplish this task is in the SEO industry, when you are trying to perhaps check out the competition or even check out a list of sites that you are planning to post to.
In addition to the SEO industry you may be checking a list of sites that you manage to make sure they are up and responding as fast as they should be.
Regardless of your requirement, it is hard to find a tool that will perform these checks on bulk URLs and also return all of the information from the checks in a format whereby you can then export and manipulate the results as you need.
Bulk HTTP Status Checker
Kostas Voudouris like many others had a requirement to produce URL status reports in batch, but could not find the correct tool out there for doing so. Although there are some other tools available that can query URLs in bulk, and even some that return the desired results, there were none that could do those things and also present them in a format that was easy to manipulate in various ways.
As a result of these issues, Kostas designed and built urlitor.com. This tool is completely free online tool for checking URLs in bulk and returning various metrics including HTTP status, redirects and several other key elements of each web address.
HTTP Status Code
The primary function of the online tool is to check the status code of each URL in the list that is submitted. This includes the initial status code of the URL as well as the final status code – there are sometimes several status codes due to the fact that the initial URL may be a redirect to another URL. In this situation there would first be a 301 redirect status code returned, and then perhaps the final URL would return a 200 status code confirming the site is up and working.
Number of redirects
The next metric that the tool will check and report on is the number of redirects in place before the URL reaches its final destination.
For example, we tested one of our websites, but we used our old URL which we knew was forwarded to a new URL (due to a re-branding). As expected the tool reported 1 redirect in this case.
Another handy little feature concerning redirects is the ability to click on the redirect count in the returned results which then shows you the URLs that were redirected from and to – this can be very useful if you are trying to analyze a competitor and perhaps see what tricks they are playing in terms of redirecting and cloaking.
The next metric in line for this tool is a response time report. This consists of the time in seconds (or milliseconds) to retrieve all of the information from the URL and is basically an indicator of the URLs load time.
This is great if you manage a lot of sites and want to compare the performance of them all against one another. Its also great for SEO purposes if you want to analyze your competitors for load time and for various other reasons as well.
User Agent change
This might be just part of an attempt to remain anonymous when checking in bulk or it may be to look for the returned status in particular circumstances.
Either way, Urlitor has the facility built in to modify the user agent used during the process.
That pretty much covers the types of data that you can retrieve in bulk by using this tool. But its not the end of the tools functionality.
Probably the coolest thing about this online URL checker is the ways in which you can manipulate the resulting data.
By default all of the information is returned in a really handy tabular view making it very easy to take a look at the results there and then.
In addition to the standard tabular view which shows all of the information returned, you can also change the view to a redirection summary.
If you want to use the information elsewhere then the tool offers two options:
- An on screen display specifically formated for use with Excel – this shows all of the data in what doesn’t look like a particular useful layout at first, but once you copy all of the data you can then past it straight into Excel where it will automatically arrange itself into the correct columns and rows.
- You also have the option to export all of the information straight to CSV file and download it from the site to your computer. This option can be useful if you are importing the data elsewhere, as CSV is a very common format used for the manipulation and transport of data.
All of these features make this a really cool tool for checking URLs in bulk. By default the tool does have a limit of 150 URLs at one time. But this limit can be removed by tweaking the sites URL slightly, although beware that checking too many URLS at once may cause unexpected results.
If you are ok with that, here is the URL for checking an unlimited number of URLs at once: