Hey, I am trying your sitemap software and i really like it so far, and I am sure I will be buying it, but I have the inevitable newbie question:
I set it up, and was working no problem, then I tried it on a site of mine that has a bad bot trap that excludes any crawler or bot except ones that I specifically allow....I could allow it by ip address but my IP address changes sometimes so would prefer to allow it by name, so could you tell what name to use, or is there a field that I could configure to a certain name, so I can allow it?
Sure, there is an entire help page devoted to help people get past such problems! :)
http://www.microsystools.com/products/sitemap-generator/help/sitemap-generator-website-platform/ (http://www.microsystools.com/products/sitemap-generator/help/sitemap-generator-website-platform/)
Essentially, just change the user agent crawler ID at "General options | Internet crawler"
thanks......I tried that, and also just making a directive for the miggibot, but the issue i still have is that your software is still getting trapped as a "bad" bot..........I will look at my robots.txt, but it looks like it either is following the link set to no follow, or I have the robots set wrong, i will look at it some more before i bother u about it
Hi,
Please drop an email to support with your website address
http://www.microsystools.com/home/contact.php (http://www.microsystools.com/home/contact.php)
It should not be a problem, but easier to help solve when seeing and testing on the website!
Problem was resolved over email quite some time ago. (A1SG had not been allowed in robots.txt, wrong user agent string was used in robots.txt)