Webmaster Forums - Website and SEO Help

XML Sitemaps Protocol Summary
The XML sitemaps protocol was created by Google to assist search engines in crawling websites. This file format uses XML for describing its content, a list of URLs in a website, and it usually referred to as XML Sitemaps. The original concept was created by Google, and was therefore in the beginning also simply called Google Sitemaps. If you do not know what the differences are between different sitemap file formats such as XML Sitemaps, HTML sitemaps and RSS feeds you should read the sitemaps explained (http://www.microsystools.com/products/sitemap-generator/help/website-sitemap-kinds-comparison/) guide.

Crawl Entire Website First
Before building the XML sitemap, A1 Sitemap Generator needs to scan your website. If you want certain areas restricted, you can either create custom crawler filters or tick options for obeying no-index, no-follow, robots.txt etc. With most websites, you only need to set root path of your website followed by clicking start scan.

Website Analysis and Links Check
After the website crawl has finished you view the website structure and detailed information for the individual pages. These tools can help you solve various website problems related to crawling. When you are done, click the Create sitemap tab.

Creating the XML Sitemap
If all you need to do is create an XML sitemap, simply pick XML Sitemap in the drop-down and click build sitemap.If your website is rich on multimedia, you can also create both image and video sitemaps for your website. The XML Sitemaps Protocol has some limits to max number of URLs per XML sitemap file. However, if exceeded, A1 Sitemap Generator will automatically split all URLs out to multiple XML sitemaps and create an XML sitemap index file.

Priority Values in XML Sitemaps Explained
XML sitemaps protocol was originally created by Google and called Google Sitemaps. Later other search engines such as Yahoo, Ask and MSN adopted the Google Sitemaps format, and Google Sitemaps got renamed into Sitemaps Protocol. Common for both is they define a <priority></priority> tag which can be used by the website owner to help search engines determine which pages are most important.

Quick Facts About Priority Values

• Priority values are considered to be relative within your website.
• Valid priority values are: 0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0.
• Usually your most important page URL will have priority value: 1.0.
• Priority of pages in the XML sitemap should decrease after importance.

Priority Values and Search Engine Optimization
Setting priority values can not be considered search engine optimization. You can not set all pages to maximum priority and assume search engines will agree :) That said, here are some thoughts about how priorities in XML sitemaps are used:

• Pages with high priority values are likely to get indexed faster and crawled more often.
• Help search engines to decide which url to show if multiple pages from same website rank for search query.
• While highly speculative, one can imagine that if a website and XML sitemap is highly trusted, you might be able boost rankings in search engine result pages, just like good internal linking within a website can.

Always ensure you have backlinks, i.e. incoming links. This will improve chances that search engines index and crawl all urls listed in your XML sitemap.

How to Set Priorities in XML Sitemaps
Never use the same priority value for all pages, better to leave priority values out of your XML sitemaps! :) You can set priority value for all pages manually, but this quickly gets tedious if you have a large website. Alternatively, if you use A1 Sitemap Generator (http://www.microsystools.com/products/sitemap-generator/), you can have all priority values calculated. It does so by counting and weighing importance of all internal links within a website and converting this into a priority value for each URL in the XML sitemap.

Final steps for your XML sitemap
At this point, you can choose to upload your XML Sitemap. You can also ping and update it to various search engines such as Google. If you haven't already, it is also a good idea to create a robots.txt file with a reference to your generated XML sitemap.

FTP Upload Generated Sitemaps
After you have created your XML sitemap, image sitemap, video sitemap etc. you will need to upload the generated sitemaps to your site. While you can use your normal favourite FTP client software for uploading files, you can also use the integrated FTP upload functionality in A1 Sitemap Generator (http://www.microsystools.com/products/sitemap-generator/). When done with configuration, simply upload all your site map files.

How to Setup FTP Upload
If you are not used to FTP upload files, you should check the instructions and information your webhost provides.

When configuring the FTP directory upload path, you should remember that many web servers have the files that can be accessed from the outside (i.e. those that can be seen through browser) placed in a sub directory, e.g. named htdocs, public, html or similar. You should refer to your webhost for this information, but a way to sometimes identify the correct directory it is to try find where you have an index file for your domain.Notice: If you are uploading sitemaps for addon domains, your upload directory path will usually look like this /html/example.com/ or /public/example.com/.Concerning host, just remember that it is not not necessary to write www in front of your FTP domain name.

• Press the Upload now button to start uploading the site map file you just built.
• Press the Upload all button to have all created site map files uploaded.

You can view progress while upload happens.

Search Engines, Robots Text File and XML Sitemaps
Since the birth of internet search engines, the robots.txt file has been how webmasters could let search engines like Google know what content should get crawled and indexed. However, as part of Google Sitemaps, later named XML Sitemaps Protocol, the usage was expanded with Sitemaps Autodiscovery. It is now possible for webmaster to direct search engines to the website XML sitemap. The moment a search engine has found your website and the robots.txt file, it will also know where to find your XML sitemap.

Submit Your XML Sitemaps
In the beginning XML sitemaps submission required you had created and verified a Google Webmaster Tools account. You also had to submit your sitemap files manually. Now, instead of submitting XML sitemaps to all search engines individually, you can be done with them all in seconds.

After crawling your website and creating the XML sitemap, the sitemap generator (http://www.microsystools.com/products/sitemap-generator/) program can create the robots.txt file with correct and complete path to your XML sitemap.

To add XML sitemaps autodiscovery to a robots.txt file, add the fully qualified XML sitemap file path like this: Sitemap: http://www.example.com/sitemap.xml.

Example robots.txt for XML Sitemaps Autodiscovery

User-agent: * Disallow:
Sitemap: http://www.example.com/sitemap.xml

If you have created a sitemap index file, you can also reference that:
User-agent: * Disallow:
Sitemap: Sitemap: http://www.example.com/sitemap-index.xml


Manual XML Sitemap Submission
There are some good reasons to submit your XML sitemaps manually the first time, e.g. to get acquainted with the different search engine and webmaster tools available:

• Google Webmaster Tools (http://www.google.com/webmasters/tools/)
• Bing Webmaster Tools (http://www.bing.com/toolbox/webmaster/)

Advanced Manage and Submit Sitemaps
In the beginning no search engines supported cross submit multiple websites in one XML sitemap file. However, now most include support for new ways of managing sitemaps across multiple sites. Requirement is you need to verify ownership of all websites:

• Sitemaps protocol (http://www.sitemaps.org/protocol.php#location): Cross sitemaps submit and manage using robots.txt.
• Google (http://www.google.com/support/webmasters/bin/answer.py?answer=75712): More website verification methods than sitemaps protocol defines.

Ping Search Engines Using XML Sitemaps Protocol
First appearing in Google sitemaps and later in XML Sitemaps Protocol, sitemap ping is a way to inform search engines that you have updated your XML sitemap files. Pinging is not required for search engines to discover changes in XML sitemaps, but it will speed the process.Some sitemap generator tools like ours (http://www.microsystools.com/products/sitemap-generator/) have builtin support for pinging.

To add more search engines like Google and Bing, click the Add common pings button. If you have scanned a website and generated an XML sitemap, A1 Sitemap Generator will automatically generate the correct ping URLs for you. It is possible to add more sitemap ping URLs if wanted. When you are done, click the Ping now button.

Video Demonstrating How to Create XML Sitemaps

We have uploaded a video to YouTube that shows how you can make xml sitemaps (http://www.youtube.com/watch?v=uW06oSoh7mo): Besides explaining the benefits of sitemaps, it also shows how to scan websites, create xml sitemaps, FTP upload sitemaps and sitemap ping search engines!Note: You can also check this shorter building sitemaps video (http://www.youtube.com/watch?v=xZ-LrxSA3Yo).

General Information About XML Sitemaps Protocol
Here is a collection of articles containing general information about XML sitemaps:

http://en.wikipedia.org/wiki/Sitemaps (http://en.wikipedia.org/wiki/Sitemaps)
General summary of what sitemaps are.

http://www.sitemaps.org/protocol.html (http://www.sitemaps.org/protocol.html)
The official specification/protocol for XML sitemaps.

http://www.microsystools.com/products/sitemap-generator/help/website-sitemap-kinds-comparison/ (http://www.microsystools.com/products/sitemap-generator/help/website-sitemap-kinds-comparison/)
All sitemaps formats explained including code examples. Explains the differences between RSS feeds, ROR files and XML sitemaps.

https://support.google.com/webmasters/answer/80472?hl=en (https://support.google.com/webmasters/answer/80472?hl=en)
Help page by Google about video sitemaps - an XML sitemaps extension.

https://support.google.com/webmasters/answer/178636?hl=en (https://support.google.com/webmasters/answer/178636?hl=en)
Help page by Google about image sitemaps - an XML sitemaps extension.

A1 Sitemap Generator Tutorials
Tutorials specific on how to use our sitemapping software to create various kinds of XML sitemaps:

http://www.microsystools.com/products/sitemap-generator/help/xml-sitemap-generator-tutorial/ (http://www.microsystools.com/products/sitemap-generator/help/xml-sitemap-generator-tutorial/)
Tutorial on how to create standard XML sitemaps.

http://www.microsystools.com/products/sitemap-generator/help/video-xml-sitemaps/ (http://www.microsystools.com/products/sitemap-generator/help/video-xml-sitemaps/)
Tutorial on how to create video sitemaps.

http://www.microsystools.com/products/sitemap-generator/help/image-xml-sitemaps/ (http://www.microsystools.com/products/sitemap-generator/help/image-xml-sitemaps/)
Tutorial on how to create image sitemaps.

Get Help With XML Sitemaps and Related Webmaster Problems

Webmaster forums where you can ask for help incase you run into problems:
A1 Sitemap Generator (http://www.webhelpforums.net/sitemap-generator/)

Forum at WebHelpForums for helping webmasters specificly on how to use A1 Sitemap Generator.

Website Crawling and Indexing (http://www.webhelpforums.net/website-crawling-and-indexing/)
Forum at WebHelpForums for general webmaster help on website crawling, indexing and sitemaps.