Tutorial on Downloading Websites

Started by Webhelpforums, February 09, 2017, 07:09:34 PM


Do you ever need to:
  • Take sites you created and designed with you to clients?
  • Keep a backup if important information found on sites online?
  • Browse sites while offline, e.g. when traveling in areas with no internet?
  • Provide alternative mirrors for sites on the intranet and internet.?
If you can answer yes to any of above, you will want to read our tutorial on using our software solution on downloading websites. (Our tool can be used to all of above things, but it has even more uses. )

Finding the Right Tool for The Job
Finding the best web site download solution is not an easy task. Here is a short list of the capabilities in the A1 Website Download software (a tool we have developed) for Windows:
  • Download and convert large dynamic database sites to static ones.
  • Customize download speed through crawl delay and max simultaneous connections.
  • Includes support for a wide range of crawl filters to finely control the download.
  • Can follow and correct a variety of file uses including redirects, frames and images.
Downloading Websites
Have you ever felt you just had to to store a local copy of a website? Perhaps even save it to a CD or USB stick, so you could take it with you. (This can be very useful in many cases, e.g. if helping someone with a computer problem that has no internet access.) This guide will demonstrate how to download entire sites in a snap.We will be using the A1 Website Download program, our own developed software for the Windows and Mac platforms. (It can also run on Linux if using virtualization solutions.) As the default options setup works with the vast majority of all sites, we will get started right-away without any delays.

As a general rule of thumb, it matters little if anything what the website you are trying to download uses as back-end platform, e.g. if it uses PHP as server side language, uses databases or has query parameters in its URLs. The only setting you really need to setup before downloading is the "website root".

In case you are planning on downloading multiple websites, you may need to configure "Save files crawled to disk directory path" to avoid downloaded sites to overwrite each another on your local disk.When done, you can click the "Start scan" button to initiate the site crawl and download.

After The Website Download
The A1 Website Download tool will default to show you the site content retrieved when it has finished downloading it all. At this point, you can, e.g. by using "Windows Explorer", move and store the downloaded website somewhere else, e.g. on an USB stick or disk, and take the complete website with you. You can also browse the web site offline with your preferred internet browser.

It is worth noting that our websites download tool can also be configured to e.g. only download website images - this being useful in case you are mainly interested in downloading e.g. picture or photo galleries. The same concept can however be used for files of any type such as .PDF documents whatever kind of files.

Tip for Web Developers: Discover Broken Links
If you are helping customers finding broken links in their websites, it can sometimes be difficult to prove and track down error if their websites change content (e.g. stuff keywords), add links (which may be broken) etc. depending on the crawler user agent, cookies support, the last visited page and similar.However, by downloading the website, you can verify the broken links reported in the software with the actual downloaded files. This makes it much easier to point out exactly where a broken link occurs, even on websites that changes their content randomly.
TechSEO360 | MicrosysTools.com  | A1 Sitemap Generator, A1 Website Analyzer etc.

More About Our Webmaster Tools for Windows and Mac

HTML, image, video and hreflang XML sitemap generatorA1 Sitemap Generator
website analysis spider tool for technical SEOA1 Website Analyzer
SEO tools for managing keywords and keyword listsA1 Keyword Research
complete website copier toolA1 Website Download
create custom website search enginesA1 Website Search Engine
scrape data into CSV, SQL and databasesA1 Website Scraper