Limit to selected Folder and Prevent Query Options

Started by cazpian, October 22, 2014, 11:47:18 AM

cazpian

Hi,

1) I'd like to be able to return scan results for just anchor links to lists pages found here ...

http://www.prodirectsoccer.com/de/lists/

I've tried a number of combination using the Analysis and the Output filters but can't seem to get it to return anything.

2) Each of these list pages contain filter options which set url parameters limiting the list, and unfortunately all possible combinations of these appear to be returned within the results window. Is there way of ensuring only the page is returned and not all of the possible options. i.e.

http://www.prodirectsoccer.com/de/lists/neue-sonderangebote.aspx
http://www.prodirectsoccer.com/de/lists/neue-sonderangebote.aspx?p=2
http://www.prodirectsoccer.com/de/lists/neue-sonderangebote.aspx?brand=adidas_Burrda
http://www.prodirectsoccer.com/de/lists/neue-sonderangebote.aspx?brand=adidas_Burrda&s=8

Any help very much appreciated.

Regards,

Tim

Webhelpforums

#1
Hi Tim,





If you only want pages within http://www.example.com/de/lists/ analyzed so you can see where their anchors go to:

Set root to
http://www.example.com/

Add this to analysis "limit to" filters

  • ::^de/lists/$

Add this to analysis "exclude" filters

  • ?

Add this to output "exclude" filters

  • ?

To start search paths add
http://www.example.com/de/lists/

(change example.com with your own domain)




If you want to perform a normal website scan, but not include "?" variations of pages found in http://www.example.com/de/lists/ simply use this;





If you still have problems, email your project to info email address found at
http://www.microsystools.com/home/contact.php

Set root to
http://www.example.com/

Add this to analysis "exclude" filters

  • ::^de/lists/.*?\?.*?

Add this to output "exclude" filters (not important, only nicer)

  • ::^de/lists/.*?\?.*?
TechSEO360 | MicrosysTools.com  | A1 Sitemap Generator, A1 Website Analyzer etc.

cazpian

Thank you so much, this has really helped me.

Do I still need to do a full web site scan and then filter the responses so that only /de/lists are displayed or is there a way to limit the scan (upfront) to only return anchors referring to pages in this path.?

Kind Regards,

Tim

Webhelpforums

Hi Tim,

I have just corrected and expanded on the earlier configuration and posted a new one as well.

So depending on what you what, one of them should work :)
TechSEO360 | MicrosysTools.com  | A1 Sitemap Generator, A1 Website Analyzer etc.

cazpian

Hi,

I'm a little confused how the output filters work.

I've assumed that if you limit the output url's to ::^de/lists/$ then only those url's which match will be returned in the results window (Analyze website). But if you run this against our websites for a couple of minutes and then stop the scan, items are returned for a whole host of pages which do not match the specific criteria.

Are these output filters only applied after the scan has fully completed ? Is this due to me suspending the scan ?

Regards,

Tim


Webhelpforums

Before you scan, enable/check this option in
Scan website | crawler options
called
Apply "webmaster" and "output" filters after website scan stops

It is not enabled by default which is why you will see unexpected URLs after scan stops.

The reason for this behavior is that some people might want to see URLs excluded by e.g. "noindex" tag - and thus excluded URLs are per default still shown after website scan stops - they are just "tagged" as excluded in the data A1WA has. using above option should give the results you want.
TechSEO360 | MicrosysTools.com  | A1 Sitemap Generator, A1 Website Analyzer etc.

More About Our Webmaster Tools for Windows and Mac

HTML, image, video and hreflang XML sitemap generatorA1 Sitemap Generator
      
website analysis spider tool for technical SEOA1 Website Analyzer
      
SEO tools for managing keywords and keyword listsA1 Keyword Research
      
complete website copier toolA1 Website Download
      
create custom website search enginesA1 Website Search Engine
      
scrape data into CSV, SQL and databasesA1 Website Scraper