Complete Screaming Frog SEO Tool Review

What is Screaming Frog?

It’s an SEO spider. It’s a bot that crawls all over your site and looks for problems.

In this Screaming Frog review, I’ll go over the tool. I’ll also list its pros and cons so you can decide if it’s right for your business.

How many broken links are on your website right now? How many broken images? Which of your pages are missing meta descriptions? Do any of your URLs have a missing H1 tag? To answer those questions, you could manually audit your website by visiting each and every page. Or you could let a tool like Screaming Frog handle that tedious process for you. Once it crawls your site, it generates reports so you can find what’s wrong and take corrective action.

Screaming Frog SEO Tool Review: It’s Not a Cloud-Based Solution

The first thing you should know about Screaming Frog is that it’s not a cloud-based solution. It’s software that you have to download and install on your PC. As of now, the tool runs on Windows, macOS, and Ubuntu. If you’re attached to any other operating system, you’re out of luck. Also, if you’re accustomed to using tools that run in the cloud (like SEMRush, for example), you might be disappointed that you have to install something just to check for errors on your website. There’s a reason for it, though. The company that created Screaming Frog (which, oddly enough, is also named Screaming Frog) doesn’t want to host a website that runs bots all over cyberspace. Fortunately, it’s a breeze to install the software locally.

You Can Get Started for Free

Like many other digital marketing tools, you can get started with Screaming Frog for free. You’ll only have limited use, though. Currently, Screaming Frog will cap the number of pages you can crawl at 500 if you’re using the free version. You also won’t be able to integrate with popular tools like Google Analytics, Google Search Console, and Link Metrics. If you decide to buy, the price is £149.00 annually. Screaming Frog is a British company so you’ll have to pay in pounds. Be sure to check the exchange rate before you purchase. Right now, that works out to about $200 per year. The reality is that you can’t get that much use out of the free version. It’s just enough to whet your appetite. If you’re serious about SEO, you should pay the annual fee. Most of the features discussed here are only available with the licensed version of the software.

Screaming Frog SEO Tool Review: It All Starts With Crawling

Screaming Frog is first and foremost a crawler. Once you’ve installed it, you’ll use it to visit pages on your website. Just enter the URL of your website and the tool will start there and crawl every page it finds. If you’ve configured your site properly, it should hit all the pages. How does it crawl? The same way that Google crawls your site: it follows links. It will find links on your menu, header, footer, and within your content. Then, it will follow those links. If you’ve got a website with thousands of pages, you might need to increase the amount of memory that the spider uses. Fortunately, Screaming Frog lets you do that very easily. Alternatively, you can switch to database mode. That way, the tool will store your crawl data on disk instead of in memory. You don’t have to crawl your entire site with Screaming Frog, though. You can specify a subdomain or folder to crawl. For example, if you want the tool to crawl only pages under blog.mydomain.com or mydomain.com/blog, you can configure it to do that. It’s flexible. You can also give Screaming Frog a list of URLs to crawl. That’s useful for site migrations. Keep in mind, Screaming Frog will obey your robots.txt file. If there are certain pages that are disallowed, don’t expect the tool to crawl them. Finally, You can save your crawls so you can refer back to them later. In fact, Screaming Frog allows you to save files mid-crawl and then resume the crawl later on when you open the file again.

What Does the Crawling Do?

At this point you might be asking yourself: what’s the point of a bot crawling all around my website? In a nutshell, it’s designed to find problems. For example, you might have broken links on your site. If so, Screaming Frog will find them. You could have a page where the title tag doesn’t match the H1 tag. Screaming Frog will identify that as a problem as well. There are numerous other issues that the tool will identify for you. Screaming Frog is designed to help with on-site SEO. It’s especially great at uncovering technical problems.

Screaming Frog SEO Tool Review: The Tabs

Once Screaming Frog is done crawling your site, you’ll have access to more information than you can possibly need. Fortunately, it’s all categorized and put into separate tabs. In the next several sections, I’ll go over each tab.

Internal Tab

The Internal tab gives you all the details about pages on your website. As is the case with the rest of the tabs, the data is presented in a table. Each line in the table represents a separate URL. For each URL, you’ll see important info such as the status code returned, page title, meta description, H1 contents, H2 contents, and word count. It’s the best starting point if you’re new to the tool. That’s why it’s also the default tab.

Screaming Frog SEO Tool Review: External Tab

Unsurprisingly, the External tab will show you info about external URLs. While you won’t see as much detail about external pages as you do about internal pages, you’ll still get plenty of useful info, such as the content type, status code returned, and the number of links that point the URL.

Protocol Tab

The Protocol tab shows you which pages use the secure (HTTPS) protocol and which use the non-secure (HTTP) protocol. Each line will also show you the status code and content type of the URL crawled.

Screaming Frog SEO Tool Review: Response Codes Tab

Use the Response Codes tab to quickly see which URLs on your site are returning anything other than a good (200) response. The typical “bad” response is the 404 code (not found). However, the report will also show you redirect (3XX) and server error (5XX) response codes.

URI Tab

The URI tab includes data related to the requested URLs. If you’re really interested in learning about the difference between URLs and URIs, you can check out Daniel Miessler’s explanation. It’s not a subject that you’ll find exciting, though. The tab includes info about each URI, such as its content, status code returned, and hash value.

Page Titles Tab

You probably know about the importance of page titles in SEO. That’s why the Page Titles tab is so valuable. It includes info about titles found on each URL, such as the number of titles per page and the character length of each title. The tab will also show you pages that have missing titles, duplicate titles, and titles that are either too short (less than 35 characters) or too long (more than 65 characters).

Screaming Frog SEO Tool Review: Meta Description Tab

The Meta Description tab is yet another tab that lives up to its name. For each URL crawled, you’ll see the number of meta descriptions found on the page, their contents, and their lengths. The tab will also show you pages that are missing meta descriptions, pages with duplicate meta descriptions, and pages with meta descriptions that are more than 320 characters in length.

Meta Keyword Tab

Although Google ignores the meta keywords tag these days, you might still find this tab useful. It will show you the number of occurrences and contents of meta keywords tags on each page.

H1 Tab

The H1 tag is another important part of on-site SEO. That’s why you should check this tab out once Screaming Frog is done crawling your site. Use it to find pages that have a missing H1 tag, a duplicate H1 tag, or an H1 tag with contents that are longer than 70 characters.

H2 Tab

If you’re writing long form content, it’s usually best to break it up into manageable chunks so that people can more easily read it. Each chunk should have its own subheading, usually created with an H2 tag. This tab will show you which pages are missing H2 tags, which have duplicate H2 tags, and which have H2 tags with contents longer than 70 characters.

Screaming Frog SEO Tool Review: Images Tab

Screaming Frog doesn’t just crawl your text content. It crawls image links as well. The Images tab will show you which images are larger than 100kb, which are missing alt text, and which have alt text that’s longer than 100 characters.

Directives Tab

The Directives tab will show you details about meta robots and canonicals. Use it to see which pages have canonicals, which are using rel=”next” or rel=”prev,” and which are using directives like “nofollow” or “noindex.”

Screaming Frog SEO Tool Review: hreflang Tab

If you’re into international SEO, you’ll find the hreflang tab valuable. As the name implies, it will show you details about the hreflang attribute on each page. Use the hreflang tab to find missing confirmation links, inconsistent language confirmation links, non-canonical confirmation links, noindex confirmation links, and incorrect language codes.

AJAX Tab

AJAX (Asynchronous Javascript and XML) is a popular technology that creates a rich user experience. Software-as-a-Service (SaaS) solutions use it all the time. To view valid data in the AJAX tab, you’ll have to configure Screaming Frog to crawl your site with Javascript rendering. If you’re interested in learning more about your AJAX URLs and the status codes they return, click on this tab.

Custom Tab

The Custom tab will show data retrieved with a Screaming Frog custom search. You can use custom search to go through your HTML source and extract data with XPath, regex, or CSS Path. Then, you can create a search-specific filter in the Custom tab to view the data you’re looking for.

Screaming Frog SEO Tool Review: Analytics Tab

The Analytics tab will show you metrics gathered from Google Analytics. That’s assuming you integrated the tool with GA. If you’re already familiar with Google Analytics, then the Analytics tab will show you familiar info, such as the number of sessions, percentage of new sessions, number of new users, and average session duration. You can use this tab to find pages that have a high bounce rate.

Search Console Tab

Just as the Analytics tab shows you data gathered from Google Analytics, the Search Console tab shows you data gathered from Google Search Console. Once again, you’ll need to integrate the tool with Search Console to see valid data in the tab. Use this tab to see the number of times each page appears in the search results as well as the number of clicks each page gets from the SERPs.

Screaming Frog SEO Tool Review: Reports

Screaming Frog offers several different reports. Each of them can be accessed under the Reports menu. In the next several sections, I’ll go over the various reports.

Crawl Overview Report

The Crawl Overview Report is the summary of the crawl. It includes important data such as:

• Number of URLs found
• Number of URLs blocked by robots.txt
• Number of URLs crawled
• Content types encountered
• Response codes encountered

Basically, the Crawl Overview Report is a bird’s-eye view of the info you’ll see in all the tabs.

Screaming Frog SEO Tool Review: Redirect Chains Report

Do you want to know which of your pages are redirecting to different URLs? If so, have a look at the Redirect Chains Report. The format of the report depends on whether you crawled the site in Spider mode or List mode. In Spider mode, you’ll see redirect chains that include two or more URLs. Use that info to optimize your redirects. In List mode, you’ll see every URL on a separate line.

Canonical Errors Report

If you’re using canonicals to deal with duplicate content issues, you’ll find the Canonical Errors Report especially valuable. The report will show you any problems with canonicals. That includes no response, 3XX, 4XX, or 5XX response codes. Additionally, the report will show you canonicals discovered within pages but not linked to on the website.

Screaming Frog SEO Tool Review: Hreflang Report

There are four Hreflang Reports:

• Errors – Hreflang attributes which return anything other than a 200 (good) response.
• Missing Confirmation Links – Pages that are missing a confirmation link or aren’t confirming.
• Inconsistent Language Confirmation Links – Pages that use different language codes to the same page.
• Non-Canonical Confirmation Links – Confirmation links that point to non-canonical URLs.

Insecure Content Report

If you’re using HTTPS (and you should be if you aren’t), you’ll want to know about non-secure elements on your pages. That’s why you should look at the Insecure Content Report. For example, you might have a page that people access with the HTTPS protocol. However, embedded within that page is a link to an image using the HTTP protocol. If Screaming Frog encounters that situation, it will flag the issue in the Insecure Content Report. It’s best to move your site entirely to HTTPS. That means links to images, JavaScript libraries, and CSS code should all use HTTPS.

Screaming Frog SEO Tool Review: Orphan Pages Report

The Orphan Pages Report shows you URLs found via the Google Analytics and Search Analytics APIs that weren’t found during the crawl. If you’re seeing URLs on this report, that means people are finding some of your content via the search engines but you don’t have any links to that content on your website. That often happens after a site migration.

Crawl Path Report

The Crawl Path Report shows you the shortest path Screaming Frog took to get to a specific URL. That’s useful if you’re finding that some of your content is buried deeply on your site. It’s usually a good idea to bring quality content as close to the top as possible.

Screaming Frog SEO Tool Review: Exporting

As with many other tools available on the market, you can export Screaming Frog data so you don’t have to view it in the tool. In fact, Screaming Frog allows you to select which data you want to export. So if you’re only interested in looking at image-specific data, for example, you can create an export file that only includes that information. There are numerous other export options as well, including All Inlinks, All Outlinks, All Anchor Text, and Response Codes.

Pros and Cons

As with any other tool, you’ll find that people have opinions about Screaming Frog. Some of those opinions are good and some are bad. Here’s what users are saying about the tool. First, the good:

• Easy-to-understand user interface
• Integrates with Google Analytics and Google Search Console
• Crawls websites quickly
• Easy to export data
• Empowers users to configure crawling options
• Inexpensive

Then, the bad:

• Constantly updating and you have to wait for it to finish updating before you use it
• Doesn’t run in the cloud
• User interface looks a bit like an Excel spreadsheet
• Lacks an API
• Free account is very limited

Screaming Frog SEO Tool Review: The Verdict

There’s a very good reason why many SEO professionals view Screaming Frog as an indispensable tool: it helps them quickly find website problems. Although there are other similar tools on the market, Screaming Frog is one of the “pioneers” in its space. That’s how it gained such widespread popularity. It’s also maintained by a group that handles SEO consulting as part of their core business. So you know they’ll make sure it keeps up with all the changes in the industry. The final answer is yes, you should add Screaming Frog to your toolset. It’s indispensable for on-site SEO.