Hello,
What is the importance of robot.txt in SEO?

Robots.txt files are useful:

If you want search engines to ignore any duplicate pages on your website
If you don't want search engines to index your internal search results pages
If you don't want search engines to index certain areas of your website or a whole website
If you don't want search engines to index certain files on your website (images, PDFs, etc.)
If you want to tell search engines where your sitemap is located

Robots.txt is a text file that lists webpages which contain instructions for search engines robots. The file lists webpages that are allowed and disallowed from search engine crawling.