Image Sitemap taking forever!

  • 1 Replies
  • 1555 Views
*

Cody J.

  • Newbie
  • *
  • 1
  • +0/-0
    • View Profile
Image Sitemap taking forever!
« on: February 07, 2013, 05:11:57 PM »
Hello,

I run an e-commerce site that has quite a few images and am trying to setup a image sitemap. To exclude lots of the temporary redirects etc our commerce puts into place I limited the output to url response codes of 0 and 200 which came out OK on my previous sitemaps - just to skip any 301's, 302's, 404's (hope they dont exist), etc.

Thing is... it's taking hours! I let it run overnight the other day and it took 7 hrs 30 minutes and then when I went to build the sitemap, it errored and said I need to pick 'google image sitemap' from the drop-down.. in which I did before I scanned and it still error'd. So today I am re-running it with the response codes in place.

Currently at 3 hrs 23 minute.
http://imgur.com/S3fyrD6 - EXAMPLE PIC

Is this normal? Thanks!


*

Webhelpforums

  • Administrator
  • Hero Member
  • *****
  • 1408
  • +6/-0
  • Shared between Microsys, WebHelpForums and helpers
    • View Profile
    • Webmaster and Website Help Forums
Re: Image Sitemap taking forever!
« Reply #1 on: February 08, 2013, 04:18:22 AM »
Doing the 301/404 exclusions in "scan website | output filters" won't always save you time because:

1)
before A1SG knows an URL errors (404 etc.) it has already downloaded it. (You can override this before and configure A1SG to use HEAD(ers) requests before GET. However, some servers refuse / are buggy when it comes to HEAD requests.)

2)
even if an URL is excluded in "output filters" it may still be an URL that passes "analysis filters", and if so, it is still downloaded and analyzed for links (just not included in output if excluded in "output filters")

...

Concerning the missing images. Are you sure you are no hosting images on external domains? And if you do, did you use the image preset where images are hosted on external domains? If you used the external preset, that will also explain why scan takes long time since lots of external URLs are then checked.

Anyhow, you can just bumb connections + threads in "scan website | crawler engine" the - default speed configuration in A1SG is about 5-10% of max. (Depends on many factors, so hard to give an exact number)

...

See also:
http://www.microsystools.com/products/sitemap-generator/help/image-xml-sitemaps/

If you still have problems, drop an email to support with link to the website:
http://www.microsystools.com/home/contact.php
« Last Edit: February 08, 2013, 04:19:55 AM by Webhelpforums »
TechSEO360 | MicrosysTools.com  | A1 Sitemap Generator, A1 Website Analyzer etc.

 




See Our Webmaster Tools for Windows and Mac

A1 Sitemap Generator
      
A1 Website Analyzer
      
A1 Keyword Research
      
A1 Website Download
      
A1 Website Search Engine
      
A1 Website Scraper