Can't Get A1 Sitemap Generator to crawl a certain directory

Started by rudypohl, January 12, 2012, 03:36:00 PM

rudypohl

Hi There:

I can't Get A1 Sitemap Generator to crawl a certain directory. I've tried everything. Set the chMod to 777 makes no difference

Webhelpforums

#1
If you try to crawl the directory directly, please make sure the directory does not redirect to e.g. the domain root.

If above or similar is the case, and you have pages in that directory linked from elsewhere, you can use a combination of:

1)
Click off "Easy mode"
http://www.microsystools.com/products/sitemap-generator/help/easy-sitemap-generator-mode/

2)
Configure "start search paths"
http://www.microsystools.com/products/sitemap-generator/help/root-aliases-start-paths/

3)
Configure include/exclude paths to analyze:
http://www.microsystools.com/products/sitemap-generator/help/website-crawler-scanner-filters/

4)
Configure include/exclude paths to output and keep:
http://www.microsystools.com/products/sitemap-generator/help/website-crawler-output-filters/

...

You may also want to check if it is a robots.txt, noindex, nofollow or similar issue:
http://www.microsystools.com/products/sitemap-generator/help/sitemap-robots-noindex-nofollow/
TechSEO360 | MicrosysTools.com  | A1 Sitemap Generator, A1 Website Analyzer etc.

More About Our Webmaster Tools for Windows and Mac