Setting Robots Robots.txt and Custom Header Tags Blogger

«Prev Next»

The purpose of setting or set the robots.txt and robots custom header tags to help the crawling and indexing of blogs on search engines. In addition, arrangements robots.txt and robots custom header tags is also a search engine optimization (SEO) pages including bloggers who post, arcives and homepage so that a more accurate post can be indexed in search engines. Well, for application in your blog, the following brief explanation:

1. Setting Header Tag Special Robot (Robots Custom Header Tags)

Well arrangement as above is the recommended setting, it aims to tell the search engines (search engines) to index the entire contents but does not allow the User-Agent Open Directory Project to crawl the entire contents of the blog and do not allow to index the blog archive.

2. Robots.txt settings

Still in Settings »Search Preferences, exactly at the top of the Tag Header Special Robot, select Edit to edit the Robots.txt and do not forget to select yes to continue the arrangement.

•  Copy and paste the code below and insert it into the form robots.txt

User-agent: Mediapartners-Google
Disallow:
User-agent: Googlebot
Allow: /
Disallow: /search
Disallow: /?m=0
Disallow: /view/*
User-agent:*
Allow: /
Disallow: /search
Disallow: /?m=0
Disallow: /view/*
Sitemap: http://url-blog.blogspot.com/feeds/posts/default?orderby=updated&max-results=999

• In addition to the above, there is also using code like the following:

User-agent: Mediapartners-Google
Disallow:
User-agent: *
Disallow: /search
Allow: /
Sitemap: http://url-blog.blogspot.com/feeds/posts/default?orderby=updated

• Click save.

Thus tutorial setting Robots Robots.txt and Header Tags Costum this. Thank you.

«Prev Next»