![]() ![]() If you set robots.txt like this, then the Google Adsense bot can crawl the entire blog for the best Adsense performance. If you want search engine, bots, only work on the most recent 25 posts, and then you should use robots.txt type 1 given below. By this method, the most recent 25 posts are submitted to search engines. Now Blogger is reading sitemap entries through the feed. Presently Blogger has completed his work on sitemap.xml. Mostly sitemap is found in XML format, i.e., sitemap.xml. A sitemap is a file located on a server that contains all posts’ s of your website or blog. In this robots.txt, you can also write the location of your sitemap file. In Blogger, by default, the search link is disallowed to crawl. ![]() If you are not using labels wisely, you should disallow the crawl of the search result pages. In Blogger search option is related to Labels. Now custom robots.txt is available for Blogspot. It means you can restrict search engine bots to crawl some directories and web pages or links of your website or blog. It tells search engine bots which directories, web pages, or links should be indexed or not be indexed in search results. Robots.txt is a text file on the server that you can customize for search engine bots.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |