How to Block folder and file through Roots.txt File?

Started by ferriesthai, September 20, 2019, 02:01:41 AM


Raj

In robots.txt file, simply use the disavow tag. Write down the URL of the file which you want to block just in front of Disavow tag. Same thing with the directory path, just place the directory path in front of Disavow tag. In this way you can block the things.

saisaanvi

Just mention disallow:/URL to the folder which you want to block. Then Google and other search engine bots won't crawl that particular folder