A1 Sitemap Generator Looping ...

  • 2 Replies
  • 1445 Views
*

drgeorgep

  • Newbie
  • *
  • 5
  • +0/-0
    • View Profile
A1 Sitemap Generator Looping ...
« on: May 01, 2016, 07:42:13 PM »
Hi ... for some reason my 7.2.0 version keeps looping. Just now, it has run for more than six hours and found more than 54 thousand URLs, although my site has fewer than 5000 URLs. This has been going on for a couple of weeks and is getting worse. I thought a fix might have come out, but, alas, not. Can someone offer helpful ideas? Thanks. dgp

*

Webhelpforums

  • Administrator
  • Hero Member
  • *****
  • 1379
  • +6/-0
  • Shared between Microsys, WebHelpForums and helpers
    • View Profile
    • Webmaster and Website Help Forums
Re: A1 Sitemap Generator Looping ...
« Reply #1 on: May 02, 2016, 06:29:29 AM »
In 99% of such cases, it is because the website generates neverending duplicate/error urls.

From:
http://www.microsystools.com/products/sitemap-generator/help/creating-sitemaps-large-websites/

Quote
Check if your website is generating an infinite amount of unique URLs. If it does, it will cause the crawler to never stop as new unique page URLs are found all the time. A good method to discover and solve these kinds of problems is by:
  • Start a website scan.
  • Stop the website scan after e.g. half an hour.
  • Inspect if everything appears correct, i.e. if most of the URLs found seem correct.
       
Example #1
A website returns 200 instead of 404 for broken page URLs. Example of infinite pattern:
Original 1/broken.html links to 1/1/broken.html links to 1/1/1/broken.html etc.

Example #2
The website platform CMS generates a huge number of 100% duplicate URLs for each actual existing URL.



By the way, here's how to stop and resume website scans:
http://www.microsystools.com/products/sitemap-generator/help/sitemap-generator-resume-scan/
« Last Edit: May 02, 2016, 06:31:10 AM by Webhelpforums »
MicrosysTools.com | Website and SEO Software for webmasters | A1 Sitemap Generator, A1 Website Analyzer etc.

*

drgeorgep

  • Newbie
  • *
  • 5
  • +0/-0
    • View Profile
Re: A1 Sitemap Generator Looping ...
« Reply #2 on: May 02, 2016, 12:58:51 PM »
Hi ... thanks for the prompt and clear response. I'll give it a try, right now, and get back. Thanks, again. dgp

 




See Our Webmaster Tools for Windows and Mac

A1 Sitemap Generator
      
A1 Website Analyzer
      
A1 Keyword Research
      
A1 Website Download
      
A1 Website Search Engine
      
A1 Website Scraper