can this scraper plugin scrape a website that requires a password set via js?

Started by wowza, November 27, 2012, 04:07:23 AM


I have a membership to a site that needs a username/password.

Everything in the site, once I am logged in sets me as a master seller with resell rights, so I have 100% permission to download the zip files from the site. That is what I paid for.

the problem is there is about 1700 pages, and I do not want to sit here for 2 days clicking button after button to download all of the zip files, that is why i got this scraper program.

Here is the site:

So can this scraper plugin let me login and then simply download all the zip files?
* I do not want anything else but the zips, no need *



TechSEO360 |  | A1 Sitemap Generator, A1 Website Analyzer etc.


I am so confused.

Can you do me one favor and try to use the link for the website I gave in my first post and let me know how I am supposed to login to that site using the website download tool?

I followed the guides the best I could but I do not understand which type of authentication I am supposed to use for that website.

If you could just let me know what settings to add to your program, then I can simply add in the username and password I have and see if it works.

Or I could just send you the username/pass, tried that but can't pm via your profile, so not sure how.

Your plugin looks great, but it's hard to figure out and this seems like a basic use case scenario.

thanks so much


Here is the form data

<form method="post" id="loginForm" name="loginForm">

<input type="text" maxlength="60" size="30" name="nicknameLogin">
<input type="password" maxlength="60" size="30" name="password">
<a onmouseout="this.className='button-ltblue'; " onblur="this.className='button-ltblue';" onclick="javascript:document.getElementById('loginForm').submit();" onmouseover="javascript:this.className='button-black';" href="javascript:void(0);" class="button-ltblue">
<a onmouseout="this.className='button-ltblue'; " onblur="this.className='button-ltblue';" onclick="javascript:document.getElementById('loginForm').reset();" onmouseover="javascript:this.className='button-black';" href="javascript:void(0);" class="button-ltblue">

<input type="hidden" value="" name="redirectMember">
<input type="hidden" value="         Login         " name="login">
<input type="hidden" value="login" name="action">
<input type="hidden" value="1" name="retry">



Using the method outlined in section:
Configure Login by Using FireFox Live HTTP Headers

Described at help page

The configuration should be something like this:

Login Path (post form):

Login paramaters (post form):

(obviously you need to correct YOURUSERNAME and YOURPASSWORD in above)

However, without actual user/pass it is hard to test, but there is a good chance above works.

If you continue to have problems you can try email support:

Note #1:
Both A1 Website Download and A1 Website Scraper are invidividual software programs (not "plugins")

Note #2:
This thread is going to be moved to A1 Websie Download section later :)
(A1 Website Scraper is for extracting structured website data into CSV files / databases)
TechSEO360 |  | A1 Sitemap Generator, A1 Website Analyzer etc.

More About Our Webmaster Tools for Windows and Mac