How to solve robots.txt error ..?

Started by indiaastro, June 05, 2019, 03:51:21 AM

indiaastro

Hello friends,

How to solve robots.txt error ..?

Rscomp

have you made sure your robots.txt is loading in your browser by adding robots.txt after your domain same as a normal page and can you see contents? has your site been down in this period? have you changed the contents of the file just before this issue? are you sure Googlebot hasn't come back since that date - whats your analytics say? do an index site: search for your domain to see if it is in google.

Jessicad

A robots.txt file is handy for telling search engines which parts of a website should be crawled/indexed and which parts shouldn't. This can be useful in certain situations where you want to keep a page or an asset hidden from search engines.

tanwirjain

Create a new robots.txt file and reupload it.