Webmaster Forums - Website and SEO Help

Microsys Products and Webmaster Tools => A1 Sitemap Generator => Topic started by: ericleonardo on May 12, 2017, 07:22:38 PM

Title: Any option to avoid External link?
Post by: ericleonardo on May 12, 2017, 07:22:38 PM
Hello! Please, is there any way to avoid following External links?
So crawler would spend time only on what matters (my website).

Another minor question: Does crawler load all script on current page or just HTML content?

Thanks!
Eric
Title: Re: Any option to avoid External link?
Post by: Webhelpforums on May 13, 2017, 10:03:27 AM
Quote
Hello! Please, is there any way to avoid following External links?
So crawler would spend time only on what matters (my website).

Default is to ignore external links.  However, certain scan presets enable checking and analyzing external links. The option hat controls this is found here:
Scan website | Data collection | Verify external links exist (and analyze if applicable)
Scan website | Data collection | Store found external URLs


Quote
Another minor question: Does crawler load all script on current page or just HTML content?

Default HTML, but can also to some degree do AJAX, Javascript, CSS and more, but it depends on settings.

Settings that affect this are:
Javascript: "Scan website | Crawler options | Try search inside Javascript"
CSS: "Scan website | Crawler options | Try search inside CSS"
PDF: "Scan website | Crawler options | Try search inside PDF"
Flash: "Scan website | Crawler options | Try search inside Flash"
AJAX: "Scan website | Crawler engine | default path type and handler"  where you can choose "HTTP using WinInet/IE" (IE is used to load the page and execute any "onload" Javascript/AJAX that alters the content.)

Title: Re: Any option to avoid External link?
Post by: ericleonardo on May 13, 2017, 10:38:54 AM
Thank you very much!
Title: Re: Any option to avoid External link?
Post by: Webhelpforums on May 16, 2017, 07:03:26 AM
Just expanded the earlier answer. If you have more questions about A1 Sitemap Generator, let us know :)