what is the difference between crawling and Indexing?

  • 2 Replies
  • 22 Views
*

sonyrobin

  • Newbie
  • *
  • 21
  • +0/-0
    • View Profile
    • Mobile app development company Dubai
what is the difference between crawling and Indexing?
« on: May 01, 2018, 08:08:14 AM »
what is the difference between crawling and Indexing?

*

mvminfotech

  • Newbie
  • *
  • 33
  • +0/-0
    • View Profile
    • https://mvminfotech.com
Re: what is the difference between crawling and Indexing?
« Reply #1 on: May 02, 2018, 12:25:42 AM »
A web crawler (also known as a web spider or web robot) is a program or automated script which browses the World Wide Web in a methodical, automated manner. This process is called Web crawling or spidering. Many legitimate sites, in particular search engines, use spidering as a means of providing up-to-date data.

Indexing is the processing of the information gathered by the Googlebot from its crawling activities. Once documents are processed, they are added to Google's searchable index if they are determined to be quality content.

*

joelhuang

  • Jr. Member
  • **
  • 72
  • +0/-0
    • View Profile
    • Metabizverse
Re: what is the difference between crawling and Indexing?
« Reply #2 on: May 02, 2018, 01:06:36 AM »
Crawling or spidering is a term used when Google, or another search engine, sends its bot to a web page or web post and “reads” the page.  Don’t let this be confused with having that page being indexed.  Crawling is the first part of having a search engine recognize your page and show it in search results.  Having your page crawled does not necessary mean your page was indexed and will be found.  Pages get crawled for a variety of reasons…the most common is having an XML sitemap that Google reads, which points to your new page.

Indexing means making sense out of the retrieved contents, storing the processing results in a (more or less complex) document index. Link analysis is a way to measure URI importance, popularity, trustworthiness and so on. Link analysis is often just a helper within the indexing process, sometimes the end in itself, but traditionally a task of the indexer, not the crawler (high sophisticated crawling engines do use link data to steer their crawlers, but that has nothing to do with link analysis in document indexes).

 




See Our Webmaster Tools for Windows and Mac

A1 Sitemap Generator
      
A1 Website Analyzer
      
A1 Keyword Research
      
A1 Website Download
      
A1 Website Search Engine
      
A1 Website Scraper