XML Sitemaps Generator

What are XML Sitemaps
It’s a special sitemap document protocol to help search engines index websites. This protocol uses XML for describing its content. The full name is XML Sitemaps Protocol, but is commonly referred to as XML Sitemaps. The idea and format was conceived by Google, and was in the beginning called Google Sitemaps. If you are unsure what the difference it between XML Sitemaps, HTML sitemaps, RSS feeds etc. you can check this sitemaps article.

Create XML Sitemap
A quick search on Google shows various services and products for creating XML sitemaps. Read our create XML sitemaps tutorial for more information.

Upload XML Sitemap
After creating your XML sitemaps, you will need to upload it either using your favorite FTP program or use the built-in FTP functionality (if any) in your chosen sitemap generator tool. For more information, check our XML sitemap FTP upload tutorial.

Submit XML Sitemap
You can submit XML sitemaps to all primary search engines, or/and add a reference to your XML sitemap in robots.txt to allow for search engine XML sitemap auto discovery. For more information about both options, check this submit XML sitemap article.

Ping Search Engines with XML Sitemap
While search engines will check your XML sitemap for changes themselves, you can speed this process by pinging search engines when your XML sitemap has changed. You can see how in the XML sitemaps ping post.

Search engines and robots.txt
Since the birth of internet search engines, the robots.txt file has been how webmasters could let search engines like Google know what content should get crawled and indexed. However, as part of Google Sitemaps, later named XML Sitemaps Protocol, the usage was expanded with Sitemaps Autodiscovery. It is now possible for webmaster to direct search engines to the website XML sitemap. The moment a search engine has found your website and the robots.txt file, it will also know where to find your XML sitemap.

Submit your XML sitemaps
In the beginning XML sitemaps submission required you had created and verified a Google Webmaster Tools account. You also had to submit your sitemap files manually. Now, instead of submitting XML sitemaps to all search engines individually, you can be done with them all in seconds.

After crawling your website and creating the XML sitemap, the sitemap generator program can create the robots.txt file with correct and complete path to your XML sitemap.

As see from above, to add XML sitemaps autodiscovery to a robots.txt file, add the fully qualified XML sitemap file path like this: Sitemap: http://www.example.com/sitemap.xml.

Complete robots.txt example for XML sitemaps autodiscovery
User-agent: *
Disallow:
Sitemap: http://www.example.com/sitemap.xml


If you have created a sitemap index file, you can also reference that:
User-agent: *
Disallow:
Sitemap: Sitemap: http://www.example.com/sitemap-index.xml


Manual XML sitemap submission
There can be good reasons to submit your sitemaps manually the first time, e.g. to get acquainted with the different search engine and webmaster tools available:

• Google Webmaster Tools
• Live Webmaster Tools (MSN)
• Yahoo SiteExplorer

Advanced manage and submit sitemaps
In the beginning no search engines supported cross submit multiple websites in one XML sitemap file. However, now most include support for new ways of managing sitemaps across multiple sites. Requirement is you need to verify ownership of all websites:

• Sitemaps protocol: Cross sitemaps submit and manage using robots.txt.
• Google: More website verification methods than sitemaps protocol defines.

Best results with XML sitemaps
When updating your website, create, upload and ping sitemaps regularly.

XML sitemaps protocol and pinging
First appearing in Google sitemaps and later in XML Sitemaps Protocol, sitemap ping is a way to inform search engines that you have updated your XML sitemap files. Pinging is not required for search engines to discover changes in XML sitemaps, but it will speed the process.

Ping XML sitemap changes to search engines
Some sitemap generator tools have builtin support for pinging. In this tutorial we use A1 Sitemap Generator as can be seen from the screenshot.

To add Google, Yahoo, Ask, MSN/Live and MoreOver, click the Add common pings button. If you have scanned a website and generated an XML sitemap, A1 Sitemap Generator will automatically generate correct ping URLs for you. It is possible to add more sitemap ping URLs if wanted. When you are done, click the Ping now button.

XML sitemaps FTP upload tool
Having created your XML sitemap, you will need to upload the sitemap files to your website. You can either use a standard FTP upload program, or the built-in sitemaps FTP upload tool in A1 Sitemap Generator. After configuration, upload of all generated sitemap files including Google XML sitemap protocol files becomes a 1-click operation.

Configure XML sitemaps upload
If you have never configured FTP options before, it is recommend you carefully check the general instructions at your webhost provider.

When you configure FTP upload directory path, you need to take into account that most webhosts have files accessible from the outside (i.e. those viewable through a webbrowser) placed in a subdirectory, usually called htdocs, public, html or similar. (If unsure, try find the directory where your index file for the given domain is located. Another solution is to ask your webhost administrator.)

Note: If you are FTP uploading an XML sitemap for an addon domain, your upload directory path will often be similar to e.g. /html/example.com/ or /public/example.com/.

Concerning host, remember that you should not include www in front of your domain name.

FTP upload and progress
Click Upload now to FTP upload the sitemap you last created.
Click Upload all to have all generated sitemap files uploaded.
You can view progress while upload happens.

Priority Values in XML Sitemaps Explained
XML sitemaps protocol was originally created by Google and called Google Sitemaps. Later other search engines such as Yahoo, Ask and MSN adopted the Google Sitemaps format, and Google Sitemaps got renamed into Sitemaps Protocol. Common for both is they define a <priority></priority> tag which can be used by the website owner to help search engines determine which pages are most important.

Quick Facts About Priority Values

• Priority values are considered to be relative within your website.
• Valid priority values are: 0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0.
• Usually your most important page URL will have priority value: 1.0.
• Priority of pages in the XML sitemap should decrease after importance.

Priority Values and Search Engine Optimization
Setting priority values can not be considered search engine optimization. You can not set all pages to maximum priority and assume search engines will agree That said, here are some thoughts about how priorities in XML sitemaps are used:

• Pages with high priority values are likely to get indexed faster and crawled more often.
• Help search engines to decide which url to show if multiple pages from same website rank for search query.
• While highly speculative, one can imagine that if a website and XML sitemap is highly trusted, you might be able boost rankings in search engine result pages, just like good internal linking within a website can.

Always ensure you have backlinks, i.e. incoming links. This will improve chances that search engines index and crawl all urls listed in your XML sitemap.

How to Set Priorities in XML Sitemaps
Never use the same priority value for all pages, better to leave priority values out of your XML sitemaps! You can set priority value for all pages manually, but this quickly gets tedious if you have a large website. Alternatively, if you use the XML Sitemaps Generator program A1 Sitemap Generator you can have all priority values calculated. It does so by counting and weighing importance of all internal links within a website and converting this into a priority value for each URL in the XML sitemap.

This tutorial will explain how to create XML sitemaps using A1 Sitemap Generator. We use default settings such as automated calculation of priority values. If you are interested in some of the advanced options or the command line interface for scheduling and automation, you can always check the help.

Crawl entire website first
Before building the XML sitemap, A1 Sitemap Generator needs to scan your website. If you want certain areas restricted, you can either create custom crawler filters or tick options for obeying no-index, no-follow, robots.txt etc. With most websites, you only need to set root path of your website followed by clicking start scan.

Website analysis and links check
After the site scan is done, you can view the website structure and detailed information for the individual pages. These tools can help you solve various website problems related to crawling. When you are done, click the Create sitemap tab.

Building the XML sitemap
In the dropdown, pick XML Sitemap and click build sitemap. The XML Sitemaps Protocol has some limits to max number of urls per XML sitemap file. However, if exceeded, A1 Sitemap Generator will automatically split all URLs out to multiple XML sitemaps and create an XML sitemap index file.

Final steps for your XML sitemap
At this point, you can choose to upload your XML Sitemap. You can also ping and update it to various search engines such as Google. If you haven’t already, it is also a good idea to create a robots.txt file with a reference to your generated XML sitemap.