What is Robot.txt file and how to attach this on your blog ?

robots.txtThere are many blogges who do not understand the robots.txt option and some blogger do not understand the benefits of this.They left this blank and this decrease the traffic of their site and we all know the importance of visitors.To solve all problems about this today I am gonna tell you, what is this ? How it works and how to attach this file to your website ?

Robots.txt file play a important roll for SEO of your blog.But if you do not know about this and you copy and paste this file from other website then it can harm your blog.For this before it is activated, you need to know about the details of robots.txt file.

What is Robot.txt file ?

Each search engines have their own Robots.You may be wondering that it is the robot which you seen on the hollywood movie.That's not my friend.It is a function of the search engines for crawling and indexing any website.You can allow or disallow to crawl and index any post to search engines using this robots.txt file.

How robots.txt file works ?

This file is similar to the traffic police. When you publish any new post and search engine bots come to your site then this file tell him to crawl and index that new post, so that new post comes quickly in the search result.

You may saw this kind of code on the robots.txt file -

User-agent: *
Disallow: /search
Allow: /
Sitemap: http://www.helperbus.com/feeds/posts/default?orderby=UPDATED



Most of the robots.txt files are like this. You also may seen this kind of file.But now I am explaining these codes for better understanding.

User-agent:* This notify that you are ordering all kind of robots.

Disallow: /search This is telling to disallow the keyword.In fact this is telling not to index and crawl the search links of your blog.All labels of your blog include this search links. You do not need to crawl and index these labels.

Allow: / this code allow to index and crawl your homepage.

Sitemap When you publish a new post then it will tell the search engine bots to crawl and index that new post.
How to insert this robots.txt file to your blogspot site ?

>At first, Log in to blogger
>Then, go to your blogger dashboard.
>Then Go to settings> search preferences.
custom-robots.txt
>After that, Click on yes option of "Enable Custom Robots.txt" option
>Then copy-paste the above codes in the text box
>Save it.

Thats all.

Note: Replace "helperbus.com" by your website's url.

Thanks for reading this article.If you have any question then comment us and if you like this then share it.

To get some more visit helperbus.com regularly.

By By. 

SHARE THIS

Author:

I'm an 18 years aged Passionate Blogger from West Bengal, India. I just love to write articles and share them with people. The day was 28th September 2015, when my dream of having a website comes true and I made this website to help you and peoples like you!

Previous Post
First