Webmaster Forums - Website and SEO Help

Microsys Products and Webmaster Tools => A1 Sitemap Generator => Topic started by: Black Ninja on February 20, 2012, 03:18:27 AM

Title: XML Sitemap for more than 50k pages.
Post by: Black Ninja on February 20, 2012, 03:18:27 AM
 Hi Everyone,

 Actually, my site has more than 50k pages and all pages are dynamic.  I want to create xml sitemap. Can somebody tell me any XML sitemap generator tools that can create sitemap more than 50k pages?  Please, pleaseā€¦ suggest me, if you know any tools.
Title: Re: XML Sitemap for more than 50k pages.
Post by: Webhelpforums on February 20, 2012, 03:46:14 AM
Note: This is the A1 Sitemap Generator help forum which is for discussion about this product only.

XML sitemaps are automatically split into multiple sitemap files at latest when passing 50k URLs, see help:
http://www.microsystools.com/products/sitemap-generator/help/xml-sitemaps-page-limit/ (http://www.microsystools.com/products/sitemap-generator/help/xml-sitemaps-page-limit/)

However, if your website is over e.g. half million URLs, you may want to wait for 64bit version which is coming soon I believe.
Title: Re: XML Sitemap for more than 50k pages.
Post by: Black Ninja on February 21, 2012, 06:03:23 AM
Thanks for your feedback, I have used A1 sitemap, But it,s works very slow. Its take 5-6 hours to create sitemap for 30k pages.  Is there any other tools?
Title: Re: XML Sitemap for more than 50k pages.
Post by: Webhelpforums on February 21, 2012, 06:21:36 AM
Hi,

If it takes 5-6 hours for 30k pages, it is most likely a server issue since A1SG can crawl much higher number of pages. Remember A1SG is not a database solution, i.e. something that just reads a database, it actually crawls your website. (And if you are using e.g. a video sitemap preset, it will also have to crawl some external pages as well.)

A rough speed example using default settings with 5 simultaneous threads;
http://www.computergameplayer.com is done 427 (analyzed and listed) pages in 58 seconds

A rough speed example increasing the count to 31 simultaneous threads:
http://www.computergameplayer.com is 427 (analyzed and listed) pages in 33 seconds

(And the website is hosted other side of the Atlantic)

...

I suggest you try switch of "Easy mode"
http://www.microsystools.com/products/sitemap-generator/help/easy-sitemap-generator-mode/ (http://www.microsystools.com/products/sitemap-generator/help/easy-sitemap-generator-mode/)

Then you can try some of things listed here:
http://www.microsystools.com/products/sitemap-generator/help/creating-sitemaps-large-websites/ (http://www.microsystools.com/products/sitemap-generator/help/creating-sitemaps-large-websites/)

Most important thing to try is increasing crawler count:
Scan website - Crawler engine - Max simultaneous connections
(But unless your server/db/etc. can handle it, the benefit will be limited.)
Title: Re: XML Sitemap for more than 50k pages.
Post by: Webhelpforums on March 03, 2012, 11:53:06 AM
I am closing this thread. If you would like further help, please email support:
http://www.microsystools.com/home/contact.php (http://www.microsystools.com/home/contact.php)