In today's Practical SEO Guide, I will tell you about website "Blocked Resources," how to view this report in Google Search Console, and how Google alerts you when they find a blocked resource.
First, let me explain that a Blocked Resource is a page, script, or some file that Google cannot read because you are telling them not to read it; yet, that same page, script, or file is used on the public site.
There's a difference between your public site and your backend area that you use to manage your site. With a WordPress site, you would log into the /wp-admin/ directory to manage a website, whereas a system like Magento has you log into the /admin/ directory. Normally, you do not want Google to read any file that exists in those two subdirectories. Therefore, you would block those resource directories from being readable by Google.
How Resources Get Blocked
There's a file on your website called the robots.txt that tells Google and the rest of the search engines what directories they are allowed to read and which ones they are not allowed to read. This is just a text file that you upload to the main directory of your website.
Here's a simple sample of a robots.txt file:
User-agent: *
Disallow: /app/
Disallow: /cgi-bin/
Disallow: /downloader/
Disallow: /includes/
Disallow: /js/
Disallow: /lib/
Disallow: /files/
Disallow: /media/
Disallow: /app/
Disallow: /cgi-bin/
Disallow: /downloader/
Disallow: /includes/
Disallow: /js/
Disallow: /lib/
Disallow: /files/
Disallow: /media/
A "User-agent" usually refers to the web browser or device that is used by a person when they visit your website, but in this case it refers to the different search engines that will visit your website. The "*" on that line simply tells all search engines to follow the next several commands.
We use all those "Disallow" commands to tell Google, Bing, and all the other search engines to ignore those directories, but sometimes the wrong directories are included in that list, and that's what Google refers to as a Blocked Resource.
Blocked Resources Report
Log into your Google Search Console to see the Blocked Resources report by clicking on Google Index --> Blocked Resources as shown in this animated screen capture:
This is what the report will look like if you do have blocked resources on your website:
Clicking on one of those line items will bring up this report:
Notice the "/js/" I've pointed to in the above report. That's the typical directory where were JavaScripts are stored. Don't know what a JavaScript is? It's a programming script that's used to create some type of effect or interaction on your website, like a rotating image on your home page or even the fading effect on your navigation menu.
Now compare the "/js/" in the report to the "Disallow: /js/" in my example robots.txt file. That's where the issue come in, because the robots.txt file is telling Google and Bing to ignore the JavaScript directory, thus, it's a blocked resource.
Effects of Blocked Resources
You're probably wondering why a blocked resource is a big deal... The reason it's a big deal is because Google and Bing will follow the directions you give them. If you tell them not to read the JavaScript directory, then they won't.
Think about the examples I gave you above, where the JavaScript is used to rotate images on the home page or control your drop down navigation menu. If they can't read the scripts then that means they can't see those image, or worse, they can't read your entire website because the drop down menus are blocked.
The Blocked Resources report only shows you things that Google has found to create issues. The only way they can fully read your website and include it in search results is if you remove these important blocked resources. I used JavaScripts and the /js/ directory in my example because it's pretty common to accidentally include them in a "Disallow" statement.
Google Notification of Blocked Resources
It used to be that you had to rely on your SEO agency to find these blocked resources for you, but recently Google started sending emails to alert you of the problem. As long as you have your website set up in Google Search Console, they will send you alert emails if the ever find one of these blocked resources.
Here's an example of such an email:
The email doesn't exactly tell you how to fix the problem, but it gives you the directions to track the exact problem down. You might need to ask your SEO agency for help figuring out exactly what the issue is.
Fixing Blocked Resources
Fixing a blocked resource is pretty easy; just delete that line from your robots.txt file. Google and Bing will start to read your full site again the next time they visit. Of course, if you're afraid to edit your robots.txt file, or don't know how to, you'll have to ask your SEO agency for help.