Different ways for stopping Google bot for crawling

  • 6 Replies
  • 1278 Views
*

lillianabe

  • Full Member
  • ***
  • 182
  • +0/-0
    • View Profile
Different ways for stopping Google bot for crawling
« on: October 14, 2011, 02:21:02 AM »

use meta tags no follow no index
specifying in robots.txt
creating captcha or form to be filled by real users with page load

I know these three methods, if you know any other method please post here.

*

anandiphone

  • Full Member
  • ***
  • 201
  • +0/-0
    • View Profile
Re: Different ways for stopping Google bot for crawling
« Reply #1 on: October 14, 2011, 08:13:01 AM »
Robots.txt is used for not crawling particular pages in the website , first we need to stop the SEO activities for Google bot to stop crawling.

*

zaboka

  • Newbie
  • *
  • 12
  • +0/-0
    • View Profile
Re: Different ways for stopping Google bot for crawling
« Reply #2 on: December 27, 2011, 11:02:22 PM »
I also want to say that Robots.txt is best way for this purpose.

*

tesori

  • Jr. Member
  • **
  • 79
  • +0/-0
    • View Profile
Re: Different ways for stopping Google bot for crawling
« Reply #3 on: January 04, 2012, 01:27:21 AM »
Hi.
You can create the captcha for the real users or you can go for the robots.txt, which is the best purpose for this.

*

anandiphone

  • Full Member
  • ***
  • 201
  • +0/-0
    • View Profile
Re: Different ways for stopping Google bot for crawling
« Reply #4 on: January 05, 2012, 06:39:25 AM »
We need to stop the SEO activities and use robots.txt to stop crawling.

*

sancon77

  • Newbie
  • *
  • 15
  • +0/-0
    • View Profile
Re: Different ways for stopping Google bot for crawling
« Reply #5 on: February 25, 2012, 01:25:17 AM »
Hi.
You can create the captcha for the real users or you can go for the robots.txt, which is the best purpose for this.
It is best suggestion indeed.

*

seenathkumar

  • Newbie
  • *
  • 16
  • +0/-0
    • View Profile
    • Business to Business Directory
Re: Different ways for stopping Google bot for crawling
« Reply #6 on: February 25, 2012, 05:45:20 AM »
Website owners give instruction to robots about their website and some pages that robots has to ignore certain pages, robot.txt file is used to is used to stop crawler to crawl that page which contains confidential content or the website owner don't want to show it. If you have a sensitive data which you don't want to show locally on search engine then robot.txt file can play an important role to stop the crawler from crawling that page. Robot.txt is a text file that you put on your website to tell crawler which pages they don't have to visit.   

 




See Our Webmaster Tools for Windows and Mac

A1 Sitemap Generator
      
A1 Website Analyzer
      
A1 Keyword Research
      
A1 Website Download
      
A1 Website Search Engine
      
A1 Website Scraper