problem with making sitemap

  • 5 Replies
  • 1325 Views
*

soundssupreme

  • Newbie
  • *
  • 11
  • +0/-0
    • View Profile
    • Rega Spares by Soundssupreme
problem with making sitemap
« on: February 21, 2013, 01:38:45 AM »
Hi,
I recently had a problem viewing images using the image sitemap part of A1, which you fixed, but now I seem to have another problem making a basic sitemap. After choosing a standard google sitemap, and scanning the web site I get just 404 errors and the sitemap looks almost empty. Please can you help with this. The scan and sitemap details look like this :-






Please can you tell what is wrong here ?

Thank you.

*

soundssupreme

  • Newbie
  • *
  • 11
  • +0/-0
    • View Profile
    • Rega Spares by Soundssupreme
Re: problem with making sitemap
« Reply #1 on: February 21, 2013, 05:12:03 AM »
I seem to have fixed the problem by setting the "max simultaneous connections" to 25 and "max worker threads" to 50.

Don't know why that would make a difference, but it now shows all the pages.

Thanks.

P.S.

Other than the online tutorials, is there a full user manual to would explain all the setting in detail so I can try and learn the features.

*

Webhelpforums

  • Administrator
  • Hero Member
  • *****
  • 1385
  • +6/-0
  • Shared between Microsys, WebHelpForums and helpers
    • View Profile
    • Webmaster and Website Help Forums
Re: problem with making sitemap
« Reply #2 on: February 21, 2013, 03:05:10 PM »
Hi,

If you decreased the amount of threads/connetions and that then worked, the problem was probably caused by your server throttling/blocking the crawler due to overload. Otherwise I would call it a fluke :)

Some of the URLs in the first screenshot were probably URLs A1SG search-in-Javascript-for-URL/file-references found. While A1SG sometimes incorrectly detects strings in Javascript as possible URLs, it does not matter since such URLs will return "404 - not found" and hence not included in sitemap. (You can find the Javascript option in "Scan website | Crawler options".)

There is a ton of help pages at:
http://www.microsystools.com/products/sitemap-generator/help/

(Although they are organized around "how to" questions)
« Last Edit: February 21, 2013, 03:10:18 PM by Webhelpforums »
MicrosysTools.com | Website and SEO Software for webmasters | A1 Sitemap Generator, A1 Website Analyzer etc.

*

soundssupreme

  • Newbie
  • *
  • 11
  • +0/-0
    • View Profile
    • Rega Spares by Soundssupreme
Re: problem with making sitemap
« Reply #3 on: February 21, 2013, 04:20:40 PM »
Hi,

Thanks for writing back,

It was actually the other way around, i.e. when the sliders for the crawler were at default it would not work, but when I slide them fully to the right, this is when it worked.

Thanks.

*

Webhelpforums

  • Administrator
  • Hero Member
  • *****
  • 1385
  • +6/-0
  • Shared between Microsys, WebHelpForums and helpers
    • View Profile
    • Webmaster and Website Help Forums
Re: problem with making sitemap
« Reply #4 on: February 21, 2013, 07:34:33 PM »
Have you tried waiting a long time? Are you completely sure that it was not just the single/few threads waiting a very very long time on a slow website/URL? (They will continue to try with default settings 3 times with pauses in-between.) E.g. if they tested external URLs? And you have enabled that option, e.g. if using external image sitemap preset?

If you can repeat the problem using version 4.1.6 feel free to email your project :)
http://www.microsystools.com/home/contact.php

« Last Edit: February 21, 2013, 07:49:23 PM by Webhelpforums »
MicrosysTools.com | Website and SEO Software for webmasters | A1 Sitemap Generator, A1 Website Analyzer etc.

*

soundssupreme

  • Newbie
  • *
  • 11
  • +0/-0
    • View Profile
    • Rega Spares by Soundssupreme
Re: problem with making sitemap
« Reply #5 on: February 21, 2013, 08:30:18 PM »
I tried it again by setting the crawler options to default i.e. 4 and 9, and it worked !!

As you say, maybe the server was slow when I tried it.

Anyway, problem solved, thanks for the consistent help you always give.

Nice to know there is help here when it is needed.


 




See Our Webmaster Tools for Windows and Mac

A1 Sitemap Generator
      
A1 Website Analyzer
      
A1 Keyword Research
      
A1 Website Download
      
A1 Website Search Engine
      
A1 Website Scraper