Robots.txt file play a important roll for SEO of your blog.But if you do not know about this and you copy and paste this file from other website then it can harm your blog.For this before it is activated, you need to know about the details of robots.txt file.
What is Robot.txt file ?
Each search engines have their own Robots.You may be wondering that it is the robot which you seen on the hollywood movie.That's not my friend.It is a function of the search engines for crawling and indexing any website.You can allow or disallow to crawl and index any post to search engines using this robots.txt file.
How robots.txt file works ?
This file is similar to the traffic police. When you publish any new post and search engine bots come to your site then this file tell him to crawl and index that new post, so that new post comes quickly in the search result.
You may saw this kind of code on the robots.txt file -
Most of the robots.txt files are like this. You also may seen this kind of file.But now I am explaining these codes for better understanding.
User-agent:* This notify that you are ordering all kind of robots.
Disallow: /search This is telling to disallow the keyword.In fact this is telling not to index and crawl the search links of your blog.All labels of your blog include this search links. You do not need to crawl and index these labels.
Allow: / this code allow to index and crawl your homepage.
Sitemap When you publish a new post then it will tell the search engine bots to crawl and index that new post.
How to insert this robots.txt file to your blogspot site ?
>At first, Log in to blogger
>Then, go to your blogger dashboard.
>Then Go to settings> search preferences.
>After that, Click on yes option of "Enable Custom Robots.txt" option
>Then copy-paste the above codes in the text box
Note: Replace "helperbus.com" by your website's url.
Thanks for reading this article.If you have any question then comment us and if you like this then share it.
To get some more visit helperbus.com regularly.