Submit XML Sitemaps with Robots.txt
Search engines and robots.txt
Since the birth of internet search engines, the robots.txt file has been how webmasters could let search engines like Google know what content should get crawled and indexed. However, as part of Google Sitemaps, later named XML Sitemaps Protocol, the usage was expanded with Sitemaps Autodiscovery. It is now possible for webmaster to direct search engines to the website XML sitemap. The moment a search engine has found your website and the robots.txt file, it will also know where to find your XML sitemap.
Submit your XML sitemaps
In the beginning XML sitemaps submission required you had created and verified a Google Webmaster Tools account. You also had to submit your sitemap files manually. Now, instead of submitting XML sitemaps to all search engines individually, you can be done with them all in seconds.
After crawling your website and creating the XML sitemap, the sitemap generator program can create the robots.txt file with correct and complete path to your XML sitemap.
As see from above, to add XML sitemaps autodiscovery to a robots.txt file, add the fully qualified XML sitemap file path like this: Sitemap: http://www.example.com/sitemap.xml.
Complete robots.txt example for XML sitemaps autodiscovery
User-agent: *
Disallow:
Sitemap: http://www.example.com/sitemap.xml
If you have created a sitemap index file, you can also reference that:
User-agent: *
Disallow:
Sitemap: Sitemap: http://www.example.com/sitemap-index.xml
Manual XML sitemap submission
There can be good reasons to submit your sitemaps manually the first time, e.g. to get acquainted with the different search engine and webmaster tools available:
Advanced manage and submit sitemaps
In the beginning no search engines supported cross submit multiple websites in one XML sitemap file. However, now most include support for new ways of managing sitemaps across multiple sites. Requirement is you need to verify ownership of all websites:
- Sitemaps protocol: Cross sitemaps submit and manage using robots.txt.
- Google: More website verification methods than sitemaps protocol defines.
Best results with XML sitemaps
When updating your website, create, upload and ping sitemaps regularly.
Comments
Comment from admin
Time: August 28, 2008, 3:41 pm
Yeah sure, just check the XML Sitemaps Ping tutorial
Comment from investorsbay.com
Time: September 3, 2008, 7:14 pm
thx u for your kind service i got many tips from here and ll support u
Comment from Null Logics Software
Time: September 30, 2008, 11:42 am
Thanks, this is a great tool!
Comment from phpbb3stylers
Time: January 3, 2009, 8:17 pm
Thanx mate..I would add my site phpbb3stylers as per your procedure
Comment from Bobster
Time: February 12, 2009, 6:17 pm
Thanks…
Can we change the XML file with the RSS one ?
Comment from admin
Time: February 13, 2009, 3:26 am
Hmm… I am not entirely sure. I will try find the time to research it 
Comment from kikykuang
Time: March 20, 2009, 5:11 pm
i have done a XML for my site so many days, but the google have not receive it, how can i?
Comment from SEO Company
Time: May 6, 2009, 6:17 am
thanks , for this free submission tool , i find free submission tool , it is best tool
Comment from image recovery software
Time: June 22, 2009, 10:32 pm
Thanks for such a nice tool. I used it and found very useful.
Comment from Calgary Painters
Time: July 9, 2009, 4:34 am
Sweet. I was doing this manually. This little program formats better and at least I don’t have to worry if I have errors.
Comment from kiky
Time: July 15, 2009, 7:26 pm
it easy to do XML file, but how to submit?
Comment from admin
Time: July 16, 2009, 1:43 am
Follow the links in this article to Google/Yahoo/MSN if you wish to submit directly to their webmaster tools.
Comment from Webmarketing Blog
Time: August 13, 2009, 4:52 am
Thanks for the tool. I’m starting to use it right now!
Comment from Flavia
Time: August 13, 2009, 4:53 am
What a nice tool!
Comment from BizBamboo
Time: August 26, 2009, 7:28 pm
Good… But check this:
http://sitemaps.org/protocol.php#submit_robots
You can add more XML Files in one robot.txt
So I think sitemap index file is not required for normal users.
Comment from Telefonia Empresarial
Time: September 11, 2009, 8:04 pm
Fantastic this resource. Hepls me a lot! Thanks for sharing.
Comment from Vivo Empresas
Time: September 11, 2009, 8:05 pm
Thanks for it, this is the best tool for search engines managament I know.
Comment from chinaphone
Time: December 4, 2009, 11:00 pm
Since the birth of internet search engines, the robots.txt file has been how webmasters could let search engines like Google know what content should get crawled and indexed.
Comment from Katrina Kaif
Time: December 23, 2009, 9:22 am
Very useful and descriptive information about xml sitemap and robots.txt. You can find more information about xml sitemap protocol at http://www.sitemaps.org/protocol.php.
Comment from Goa on wheels
Time: January 25, 2010, 9:04 am
how to do it can some one help me
Comment from admin
Time: January 26, 2010, 9:32 am
What problems are you encountering? Contact Microsys support if you need help doing something with A1 Sitemap Generator!
Comment from Shahi
Time: February 7, 2010, 3:17 pm
This Is Very Good Program.
Post new comments in forum
Discuss articles and get help at Web Help Forums.
Comment from Emboodo
Time: August 28, 2008, 2:33 pm
Thanks, this is a great tool for search engines managament. Do you have the code that ping your site map to Gooogle?