What is robots.txt?

  • 3 Replies
  • 124 Views
*

rahuljoshi23

  • Newbie
  • *
  • 32
  • +0/-0
    • View Profile
    • Tradingstacks
What is robots.txt?
« on: June 20, 2019, 02:19:58 AM »
What is robots.txt?

*

suhel

  • Newbie
  • *
  • 29
  • +0/-0
    • View Profile
Re: What is robots.txt?
« Reply #1 on: June 20, 2019, 02:24:17 AM »
Robot.txt are bot used by google to crawl your website

*

gaurav06

  • Newbie
  • *
  • 18
  • +0/-0
    • View Profile
Re: What is robots.txt?
« Reply #2 on: June 20, 2019, 03:57:53 AM »
Robots.txt is one way of telling the Search Engine Bots about the web pages on your website which you do not want them to visit.

*

sohailkhan1

  • Jr. Member
  • **
  • 52
  • +0/-0
    • View Profile
Re: What is robots.txt?
« Reply #3 on: June 20, 2019, 05:57:53 AM »
 Robot.txt is useful for preventing the indexation of the parts of any online content that website owners do not want to display.

 




See Our Webmaster Tools for Windows and Mac

A1 Sitemap Generator
      
A1 Website Analyzer
      
A1 Keyword Research
      
A1 Website Download
      
A1 Website Search Engine
      
A1 Website Scraper