How to add the Sitemap Tag to your Robots.txt file
by Bill Baker on Thursday, May 31st, 2007 | No Comments
In your robots.txt file on a line by itself, add the following command to tell the search engines your Sitemap location:
| Sitemap: <sitemap_location> |
The <sitemap_location> should be the complete URL to your Sitemap according to sitemaps.org.
Below is 4WebResults.com’s robots.txt file with the Sitemap Tag.
| User-agent: * Sitemap: http://4webresults.com/sitemap.xml Disallow: /createMember Disallow: /portal_memberdata/portal_factory/ Disallow: /*sendto_form$ Disallow: /*folder_contents$ Disallow: /*_edit$ Disallow: /*month:int* Disallow: /*SimpleBlogDaySearch* |
We are using our XML Sitemap file. You can also you use a text Sitemap or HTML Sitemap file.
| Sitemap: http://www.example.com/sitemap.txt |
| Sitemap: http://www.example.com/sitemap.htm |
The following sources were used to put the blog together:
sitemaps.org: http://www.sitemaps.org/protocol.php#informing
Market Position: http://www.marketposition.com/blog/archives/2007/05/teaching_new_tr_1.html.
