Blogger Custom Robot.txt file To Improve SEO

How to update Custom Robot.txt file in Blogger to boost SEO

The Ultimate Guide To Custom Robot.txt file in Blogger to boost SEO

Each and everyone of us want our websites to rank well on Google. Unlike WordPress, where there are so many ways to control search engines bot on our websites, blogger is limited to that. Google search engine is just a large library that is capable of taking over 2 + billion pages or posts each and everyday. With this, Google wouldn’t mind indexing your entire blogger website without skipping a single page. If you have a blogger website, then you should be the one to tell Search engine bots how to behave on your website. Updating your blogger site custom robot.txt file helps to direct the path of crawlers whenever they come to your site. With the help of this file, you can control how , when and what search engine bots should crawl and index. Today, I am going to walk you through the steps to update your custom robot.txt file in Blogger to boost SEO. First of all let’s understand:

What is custom Robot.txt file?

Custom Robot.txt file in Blogger is a code that is used to instruct search engine bots about how to crawl and index your site. You can use the robot.txt file to tell Google bot to ignore some pages, posts etc. when crawling your website. In some cases, you can direct search engines bot to ignore some pages but will still crawl and index it. When this happens, then you didn’t do what was right to stop the bot from crawling that particular page. You should know what you are doing when you are updating your blogger robot.txt file.

Important Of Robot.txt File In Blogger.

1. It helps prevent or control how search engine bots indexes your website. In other words, crawling and indexing your entire website is prevented.

2. You can use the custom robot.txt file to block some specific posts and pages from getting indexed. 

Example : Assuming you want to block a post from indexing, then you should update the file to do that. This can be done at the bottom of the code like this; Disallow: /2022/06/yourblogposturl.html. The same thing applies to the situation of blocking a page. In this case, it will be Disallow:/2022/06/ yourpageurl.html. ( Note : You should replace the year and the month with the month and the year you publish that post or page.)

3. It can be used to prevent tags from appearing on Google or other search engines.

Read Also : How to index blogger post or website on google search console step by step

About Robot.txt File In Blogger. ( Full Explanation)

User-agent: Mediapartners-Google

Disallow:

User-agent: *

Disallow: /search?q=

Allow: /

Sitemap: https://www.yourdomain.com/sitemap.xml

Sitemap: https://www.yourdomain.com/sitemap-pages.xml

When you take the custom robot.txt file above, for example, the;

User agent

In the first section, it means that or indicates the types of crawlers that you want it behavior to be directed on the path you want.

Disallow

The disallow in the second line means that the crawler you mention is not supposed to crawl on your site.

Disallow:/search?q=

This section (Disallow:/search?q=) means, search pages are not allowed to get indexed.

Allow.

The allow section means that all posts and pages are allowed to get crawled and indexed with the exception of the Disallow section.

The last part of the code is the sitemap of your website. It contains all the pages and posts in your website.

How To Write Sitemaps For Your Blogger Website.

Sitemaps are very helpful for your website. It tells Google bots the number of pages and posts that your website contains in order to facilitate faster indexing. Sitemaps are written as;

https:/www.yoursdomain.com/sitemap.xml for posts. For pages, it is written as https:/www.yourdomain.com/sitemap-pages.xml.  Please don’t forget to add your blogger website url in place of “your domain.com” .

(Note: If you are using a subdomain, you can use this format, it works for both.)

Read More : How to get AdSense approval on Blogger websites

Best custom Robot.txt File To Boost SEO For Blogger Websites.

This robot.txt file is going to boost your SEO. I believe you have seen many of them but trust this one and you will come back to testify. Copy and paste it into your blogger site.

User-agent: Mediapartners-Google

Disallow:

User-agent: *

Disallow: /search?q=

Allow: /

Sitemap: https://www.yourdomain.com/sitemap.xml

Sitemap: https://www.yourdomain.com/sitemap-pages.xml

Steps To Update custom custom Robot.txt File In Blogger To Boost Ranking.

To add the custom robot.txt file in blogger, kindly follow the steps below.

1. Go to your blogger dashboard and click on the 3 lines at the left-hand corner on top as shown in the picture below.

How to update Custom Robot.txt file in Blogger to boost SEO

2. Click on the settings in the options over there as shown below.

How to update Custom Robot.txt file in Blogger to boost SEO

3. Scroll down to find “crawlers and indexing” and enable it as shown in the picture below.

How to update Custom Robot.txt file in Blogger to boost SEO

4. Now click on “Custom robot.txt” 

How to update Custom Robot.txt file in Blogger to boost SEO

5. Paste the code you copied above and paste it here ( In the blank space)

How to update Custom Robot.txt file in Blogger to boost SEO

6. Now click on save to fully save the code to your website.

How to update Custom Robot.txt file in Blogger to boost SEO

When you complete these steps, you should wait and see the difference in your indexing and ranking on Google.

Read Also : How to connect AdSense account to blogger

Conclusion.

I believe that this tutorial is helpful. Put this in mind that knowing the best custom robot.txt file will help boost your blogger SEO. This file will direct search engine bots how to behave when crawling your website. Thanks for reading. Please, if you find this article helpful, kindly leave a comment below, but please don’t add links. Thanks again. See you in my next article.

Leave a Reply

mersin escort kızkalesi
%d bloggers like this: