See Our Webmaster Tools for Windows and Mac

                                  

Sitemap Generator Runs for Hours on one site

Started by aplusnetsolutions, April 05, 2011, 10:03:05 AM

aplusnetsolutions

Hello,
I use the sitemap generator for 30+ sites.  For most it runs fine.  But there are a few for which it runs and runs with no end in sight.  For this site, paesanos1604.com, it ran for 3+ hours and was still running.  It finds database records and indexes those.  But when I looked at it there were over 21,000 Internal sitemap URLs.  I know there were not 21,000 database records for that site.

Any idea what to do on a situation like this?
Thanks,
Greg

Webhelpforums

Disable "Easy mode":
http://www.microsystools.com/products/sitemap-generator/help/easy-sitemap-generator-mode/

Configure for using resume:
http://www.microsystools.com/products/sitemap-generator/help/sitemap-generator-resume-scan/

I would disable:
Scan website | Crawler options | Apply "webmaster" and "list" filters after website scan.
(This means A1SG will show you URLs it would normally delete when website scan stops. It could be URLs filtered off by robots.txt or a multitude other reasons. However, the sitemap.xml won't contain these URLs.)

Run a website scan an hour or whatever. See the URLs found. Are there URLs you believe should not be analyzed for links or be included in sitemap output?

Add Them to analyze exclude filters:
http://www.microsystools.com/products/sitemap-generator/help/website-crawler-scanner-filters/

Add them to output exclude filters:
http://www.microsystools.com/products/sitemap-generator/help/website-crawler-output-filters/
TechSEO360 | MicrosysTools.com  | A1 Sitemap Generator, A1 Website Analyzer etc.





recommended related video to Sitemap Generator Runs for Hours on one site
Note: Check our video related to keywords in "Sitemap Generator Runs for Hours on one site" on YouTube.