What are spiders?

Started by Nichole Green, November 17, 2011, 01:45:03 AM

Nichole Green

Search engine spiders are basically automated programs that scan websites to provide information to search engines for the purpose of indexing or ranking them.

AllenSantiago

Spider is an automated program which helps in feeding the webpages of the website. After feeding the webpages, it grabs the data or information and stores the important information from that webpage in the search engine index. This is called Indexing. When a person enters a query in the search engine relevant to this content, it takes a person to that webpage. The program crawls over the Web (www), therefore it is known as 'Spider'.

pamela88

Spiders are used to feed pages to search engines. It's called a spider because it crawls over the Web. Another term for these programs is webcrawler.
Because most webpages contain links to other pages, a spider can start almost anywhere. As soon as it sees a link to another page, it goes off and fetches it. Large search engines, like Alta Vista, have many spiders working in parallel.

myname

i think in simple words search engine spider is program that scan your site and then it send this report to the search engine .......

Jacobdixon

They seems to be same as crawlers

Cick

great
thanks for sharing the info with us

Nile Hadwards

Spider is call robot ,it is teling about which page you would like to spide or not ....

messi

Search engine spiders are by far one of the most useful things to come around in the last 10 years of the internet. They are useful not only to the web sites (Google and many others) that use them, but also to people who are searching for a particular site and those who run web sites. Spiders allow your site to be seen by the millions of people who use search engines every day. In this newsletter, we will discuss what search engine spiders do, how they work, and how to set up a robots.txt file and upload that to your site to keep spiders from visiting your site.

varun01

Search engine spiders are by far one of the most useful things to come around in the last 10 years of the internet. They are useful not only to the web sites (Google and many others) that use them, but also to people who are searching for a particular site and those who run web sites.

hitman

Spider is an computerized program which helps in eating the websites of the website. As soon as it recognizes a link to another website, it goes off and brings it. Large google, like Alta Windows vista, have many robots working in similar.This then gets a new Website, and add them to google optimization catalog. Google is a crawler based seo, it will depend on to instantly create the list of robots.

wilsondavid

Spider is the program of the Google who can crawl the content of your website and according to find the result it can assign the index number for your website so that according to that index number your website appear on the search engine.

Alex Thompson

Web spiders can also be used with malicious intent, although each server spidered may have a file called robots.txt which may contain rules for the bot to follow.


harryjames

Alternate name of spiders are web crawlers which feeds the page to search engines and crawled pages are than stored in index after indexing process is over.

Shinely

They are the ones who crawl our websites for ranking.

rizwanshafco

Site crawled by spiders of search engines, Spider is a software of search engine which crawl your site and then save that into the data base of search engine.

Divyarai

Spider is a tool which is used to crawl your website contents and links.

anshuera

They are  spiders because they crawl over the Web in search of content . Search engines gather data about a website by sending' the spider or bots to read' the site and copy its content.


More About Our Webmaster Tools for Windows and Mac

HTML, image, video and hreflang XML sitemap generatorA1 Sitemap Generator
      
website analysis spider tool for technical SEOA1 Website Analyzer
      
SEO tools for managing keywords and keyword listsA1 Keyword Research
      
complete website copier toolA1 Website Download
      
create custom website search enginesA1 Website Search Engine
      
scrape data into CSV, SQL and databasesA1 Website Scraper