Does A1 Website Scrapper support dynamic parameter passing?

Started by gjackson123, February 20, 2012, 07:40:40 AM


 :)Dear  A1 Website Scrapper forum members,

I would like to extract a comprehensive list of real estate data from certain website before generating the data in CSV / XML, or both. However, I am not sure whether A1 Website Scrapper could do the job or not. This web page provides the following selections prior to query the website for answer:

( i ) Address,Suburbs, Postcodes, or Regions,
( ii ) Property Type (Pull down menu),
( iii ) Bedrooms (Pull down menu),
( iv ) Min Price (Pull down menu),
( v) Max Price (Pull down menu),

As a result, can A1 Website Scrapper provide advanced filtering configuration / settings to manually supply these data to achieve the desire query? Moreover, does it support some form of automation using APIs to read the content of these fields possibly from an external file for instance, to dynamically populate these fields to come up with all combinations of fields to achieve a comprehensive set of data?

Thanks a lot,



Concerning which pages to crawl/analyze for links see:

Then combine it with which pages you want to scrape for data:

You can also have A1 Website Scraper combine elements in forms and try visit them as URLs. (Option found if you switch off Easy mode" and then check "Scan website - Crawler options".)

So concerning the URLs themselves and which of them to scrape, it is very flexible.


However, the data extraction itself is still done though regular expression which may or may not be suitable for your needs. See:

You can also have A1 Website Scraper combine elements in forms and try visit them as URLs. (Option found if you switch off Easy mode" and then check "Scan website - Crawler options".)
TechSEO360 |  | A1 Sitemap Generator, A1 Website Analyzer etc.



Thanks for your guidance. I am still taking some time to become familiarize with this product before getting back to you.




I am wondering whether you could provide some guidance on how to quickly grasp and make use of this product since I am still a little overwhelmed with all the configuration settings in the Advanced mode, which is the minimum functionalies needed for the job at hand:

( i ) Need to go through all states (New South Wales, Northern Territory, Queensland, South Australia, Tasmania, Victoria, Western Australia) in alphabetical from
( ii ) Get the following suburb details from each of the states using Sydney as an example, where the URL is
      ( a ) Suburb name
      ( b ) Direct URL (optional)
      ( c ) Latitude
      ( d ) Locality
      ( e ) Nearest urban centre
      ( f ) Dialing code
      ( g ) Time zone
      ( h ) Nearest airport
      ( i ) Nearest train station
      ( j ) Nearest hotels
      ( k ) Near by features
I am hoping to write a wrapper in Java that will pass these details as parameter to A1 Website Scraper and let it carry out the
data extraction.

Does regular expression in A1 Website Scraper zero-width / length word boundaries (\b) and negation to narrow down the search to
specific string, there by getting exactly what one is looking for?

Hope that this can be a feedback to development if the current version does not support the functionalities I am looking for.

Many thanks,



I suggest you:

- Download latest version. (Each version usually includes a new scrape demo/example.)
- Check the latest instance of scrape websites tutorial. (The online help is often updated.)

If I recall correctly, there should be no problem using the regex constructs you mention, but if you have problems getting something to work, do feel free to email support directly at:

(Be sure to include your existing test-project and make it as simple as possible to demonstrate the problem you have. You can always expand it later when the problem has been resolved.)
TechSEO360 |  | A1 Sitemap Generator, A1 Website Analyzer etc.

More About Our Webmaster Tools for Windows and Mac

HTML, image, video and hreflang XML sitemap generatorA1 Sitemap Generator
website analysis spider tool for technical SEOA1 Website Analyzer
SEO tools for managing keywords and keyword listsA1 Keyword Research
complete website copier toolA1 Website Download
create custom website search enginesA1 Website Search Engine
scrape data into CSV, SQL and databasesA1 Website Scraper