Source: Neil Patel I don’t know about you, but I’m always looking for ways to enhance my SEO. I’ve been in digital marketing for more than 10 years
I don’t know about you, but I’m always looking for ways to enhance my SEO.
I’ve been in digital marketing for more than 10 years now. I’ve heard thousands of people say, “SEO is dead!”
And it’s still a super important part of any website’s health.
That’s why I like finding new, interesting, and/or little-known methods to improve SEO.
Today, I’m going to share an SEO technique that I don’t hear about often, but it’s done a lot for me.
It’s the Crawl Stats feature of Google Search Console.
You probably know that Google Search Console, or GSC for short, is one of my favorite tools for managing a website.
You’ve most likely used it before. You might even be a GSC expert.
But I’ve realized that with GSC, there’s more than meets the eye.
It has a lot of features that I use, but I don’t see other people using them that much.
In particular, the Crawl Stats feature doesn’t get enough praise.
It’s a tiny page––it only has three graphs and some numeric data. But don’t underestimate it.
It tells you a lot about your site is reacting to Google’s search engine crawler. If you’re looking to maximize your SEO, that’s important information to have.
So I’m going to show you how to use Crawl Stats to beef up your SEO and get insights you might not get anywhere else.
What is Crawl Stats?
First, I’ll give you a rundown on what Crawl Stats is.
To access it, head to Google Search Console and select the right property.
In the sidebar on the left, click on Crawl.
In the menu that pops up below, click Crawl Stats.
You’re now at your Crawl Stats page! It should look something like this:
I’ll admit that at first glance it doesn’t seem too helpful. It’s not obvious how this data will make your SEO better.
So let’s talk about what these results actually are.
Basically, this data is measuring your crawl rate.
Crawl rate is how often search engine robots crawl your site. In this case, Crawl Stats is showing Googlebot’s (Google’s search engine robot) activity.
A fast crawl rate is almost always desirable. That means the bots can index your site more easily and more quickly.
And if your site gets more attention from Googlebot, you’ll likely earn a higher SERP ranking.
In a nutshell, that’s why the Crawl Stats data is so important. If your crawl rate is low, your SEO is taking a hit.
On the other hand, if your crawl rate shoots up all of a sudden, something could be wrong with your site.
The point is, it’s important to monitor your crawl rate.
That said, these graphs might not make sense to you at first, and that’s okay!
So let’s look at how to interpret them.
How to read the graphs
There are three main sections of the Crawl Stats page:
- Pages crawled per day
- Kilobytes downloaded per day
- Time spent downloading a page (in milliseconds)
They’re all important to getting the most out of Crawl Stats, even though it might not seem like that. So you really should consider all three every time.
I’ll tackle each section separately.
Pages crawled per day
This section shows you how many pages Googlebot crawls every day. You’ll see results from the last 90 days:
If you want to find a specific point in time, you can hover your mouse over the graph and view results for a particular day:
On the right side, you can see your high, average, and low crawl amounts.
Here’s the tricky part.
Crawl rate is different from a metric like Domain Authority. That’s because you can’t change how often Google crawls your site.
What you can do is understand why there might be fluctuations in your graph, especially if it goes from really low to really high in a day or two.
Because crawl rate depends heavily on how fast and bot-friendly your site is, crawl rates are good indicators of whether or not your site is easily crawlable. (And you want it to be!)
That means you want consistent crawl rates. That should look roughly like this:
There are some ups and downs, but it stays right around the same spot on average.
If you see something that looks more along the lines of this:
Then there’s a problem.
Generally, your graphs will all look relatively similar. However, the pages crawled per day graph is the one you want to focus on for crawl rate.
Super sudden drops or spikes are both signs that something’s probably up with your site.
You might be looking at that right now. What should you do?
Let’s look at each scenario.
If you see a sudden drop, one of the following things could be happening.
1. You might have broken code (like HTML) or unsupported content on one or more of your pages.
If you’ve recently added new code, this could be the problem.
You can run your code through one of W3’s validators to see if it’s working correctly or not.
2. Your robots.txt file might be blocking too much.
It’s a good practice to carefully modify your robots.txt file, but you can actually block resources that Googlebot needs to crawl your site.
If your robots.txt file is huge like this one:
Then you might want to revise it.
3. Your site has stale content.
It’s no secret that Google loves fresh content.
Here’s how it works (in theory, at least): When you make a change to a page on your site, Google is alerted of that change and recrawls your page.
Every time Googlebot crawls one of your pages, it reindexes your page.
And if your page is super high quality, you’ll likely get a ranking boost from it.
…
COMMENTS