What is the limit of a robot.txt file?

[pihu147741]

Hello friends,

What is the limit of a robot.txt file?

[sinelogixtech]

Yes, there is a limit to robots.txt file in terms of filesize 500 KB as Vipin points out.

[RH-Calvin]

Google actually crawls 500 KB of text in robots.txt file.

[Raj]

Hello friends,

What is the limit of a robot.txt file?

500 kb

[Neha Patil]

Hello friends,

What is the limit of a robot.txt file?

500 kb

[vinukum]

It must be less than 500kb.

[mvminfotech]

A maximum file size may be enforced per crawler. Content which is after the maximum file size may be ignored. Google currently enforces a size limit of 500kb.

[kylojoe]

Robots.txt is a text file which use webmaster creat to instruct web robots how to crawl pages on their website. It is the part of the robots exclusion protocol (REP) that regulate how robots crawl the web access and index page content and serve that content to end users.
Robots.txt is a text file you put on your site to tell search robots which pages you would like them not to visit.
structure
------------
User-agent:
Disallow:

Limit of Robots.txt File is 500 mb

[Akash-Varma]

500 kb

[efusionworld]

Your robots.txt file must be smaller than 500KB. John Mueller of Google reminded webmasters via Google+ that Google has a limit of only being able to process up to 500kb of your robots.txt file.