XML Sitemap for more than 50k pages.

Started by Black Ninja, February 20, 2012, 03:18:27 AM

Black Ninja

 Hi Everyone,

Actually, my site has more than 50k pages and all pages are dynamic.  I want to create xml sitemap. Can somebody tell me any XML sitemap generator tools that can create sitemap more than 50k pages?  Please, please... suggest me, if you know any tools.

Webhelpforums

#1
Note: This is the A1 Sitemap Generator help forum which is for discussion about this product only.

XML sitemaps are automatically split into multiple sitemap files at latest when passing 50k URLs, see help:
http://www.microsystools.com/products/sitemap-generator/help/xml-sitemaps-page-limit/

However, if your website is over e.g. half million URLs, you may want to wait for 64bit version which is coming soon I believe.
TechSEO360 | MicrosysTools.com  | A1 Sitemap Generator, A1 Website Analyzer etc.

Black Ninja

Thanks for your feedback, I have used A1 sitemap, But it,s works very slow. Its take 5-6 hours to create sitemap for 30k pages.  Is there any other tools?

Webhelpforums

#3
Hi,

If it takes 5-6 hours for 30k pages, it is most likely a server issue since A1SG can crawl much higher number of pages. Remember A1SG is not a database solution, i.e. something that just reads a database, it actually crawls your website. (And if you are using e.g. a video sitemap preset, it will also have to crawl some external pages as well.)

A rough speed example using default settings with 5 simultaneous threads;
http://www.computergameplayer.com is done 427 (analyzed and listed) pages in 58 seconds

A rough speed example increasing the count to 31 simultaneous threads:
http://www.computergameplayer.com is 427 (analyzed and listed) pages in 33 seconds

(And the website is hosted other side of the Atlantic)

...

I suggest you try switch of "Easy mode"
http://www.microsystools.com/products/sitemap-generator/help/easy-sitemap-generator-mode/

Then you can try some of things listed here:
http://www.microsystools.com/products/sitemap-generator/help/creating-sitemaps-large-websites/

Most important thing to try is increasing crawler count:
Scan website - Crawler engine - Max simultaneous connections
(But unless your server/db/etc. can handle it, the benefit will be limited.)
TechSEO360 | MicrosysTools.com  | A1 Sitemap Generator, A1 Website Analyzer etc.

Webhelpforums

I am closing this thread. If you would like further help, please email support:
http://www.microsystools.com/home/contact.php
TechSEO360 | MicrosysTools.com  | A1 Sitemap Generator, A1 Website Analyzer etc.

More About Our Webmaster Tools for Windows and Mac

HTML, image, video and hreflang XML sitemap generatorA1 Sitemap Generator
      
website analysis spider tool for technical SEOA1 Website Analyzer
      
SEO tools for managing keywords and keyword listsA1 Keyword Research
      
complete website copier toolA1 Website Download
      
create custom website search enginesA1 Website Search Engine
      
scrape data into CSV, SQL and databasesA1 Website Scraper