Excluding a folder and file type

Started by tphughes, March 25, 2013, 02:16:17 PM

tphughes

I want to exclude certain pages from being indexed. However, it is an AND statement. I want to exclude all pages that are in the /products/ folder AND have the extension of .html. How would I do this?

Webhelpforums

You will want to use regular expressions in those case.

First see:

http://www.microsystools.com/products/sitemap-generator/help/website-crawler-scanner-filters/
(which URLs/pages get analyzed)

http://www.microsystools.com/products/sitemap-generator/help/website-crawler-output-filters/
(which URLs/pages get included in output)

Other help pages
http://www.microsystools.com/products/sitemap-generator/help/easy-sitemap-generator-mode/
http://www.microsystools.com/products/sitemap-generator/help/sitemap-generator-user-interface/

Then create a regular expression like this
/products/.*\.html
which will match any URL with */products/* and *.html*

As seen from help above, A1SG needs regex filters prepended with "::" so it becomes
::/products/.*\.html
TechSEO360 | MicrosysTools.com  | A1 Sitemap Generator, A1 Website Analyzer etc.

More About Our Webmaster Tools for Windows and Mac

HTML, image, video and hreflang XML sitemap generatorA1 Sitemap Generator
      
website analysis spider tool for technical SEOA1 Website Analyzer
      
SEO tools for managing keywords and keyword listsA1 Keyword Research
      
complete website copier toolA1 Website Download
      
create custom website search enginesA1 Website Search Engine
      
scrape data into CSV, SQL and databasesA1 Website Scraper