Excluding a folder and file type

  • 1 Replies
  • 1003 Views
*

tphughes

  • Newbie
  • *
  • 1
  • +0/-0
    • View Profile
Excluding a folder and file type
« on: March 25, 2013, 02:16:17 PM »
I want to exclude certain pages from being indexed. However, it is an AND statement. I want to exclude all pages that are in the /products/ folder AND have the extension of .html. How would I do this?

*

Webhelpforums

  • Administrator
  • Hero Member
  • *****
  • 1386
  • +6/-0
  • Shared between Microsys, WebHelpForums and helpers
    • View Profile
    • Webmaster and Website Help Forums
Re: Excluding a folder and file type
« Reply #1 on: March 25, 2013, 02:47:39 PM »
You will want to use regular expressions in those case.

First see:

http://www.microsystools.com/products/sitemap-generator/help/website-crawler-scanner-filters/
(which URLs/pages get analyzed)

http://www.microsystools.com/products/sitemap-generator/help/website-crawler-output-filters/
(which URLs/pages get included in output)

Other help pages
http://www.microsystools.com/products/sitemap-generator/help/easy-sitemap-generator-mode/
http://www.microsystools.com/products/sitemap-generator/help/sitemap-generator-user-interface/

Then create a regular expression like this
Code: [Select]
/products/.*\.htmlwhich will match any URL with */products/* and *.html*

As seen from help above, A1SG needs regex filters prepended with "::" so it becomes
Code: [Select]
::/products/.*\.html
MicrosysTools.com | Website and SEO Software for webmasters | A1 Sitemap Generator, A1 Website Analyzer etc.

 




See Our Webmaster Tools for Windows and Mac

A1 Sitemap Generator
      
A1 Website Analyzer
      
A1 Keyword Research
      
A1 Website Download
      
A1 Website Search Engine
      
A1 Website Scraper