Quote from: pankaj0008 on July 22, 2021, 07:40:29 AM
Robots. txt is a text file webmasters create to instruct web robots (typically search engine robots) how to crawl pages on their website. When a search engine lands on a site, it looks at the command for instructions. It can seem counterintuitive for a site to want to instruct a search engine not to crawl its pages, but it can also give webmasters powerful control over their crawl budget.

Yes, you are right robots.txt is a text file that a website master create to give instructions to the various search engines like Google, Bing to control crawl rate of a website. It tells to the search engines which parts should be crawl and which part should be avoided.

Panda assesses the quality of each post that you make on your website. It is concerned with if you are adding meaningful content rather than going to an unnecessary high word count. To get the most of Panda, stay away from keyword-driven posts that don't come across as natural and never duplicate content from another website.

Penguin also devalues low-quality sites, but it looks for different criteria. While Panda assesses your actual content, Penguin looks at how your website has been promoted. If your website has been promoted through spam like bots and spiders, or has been linked by a number of disreputable sites, your rankings will go down. The one thing to note about Penguin is that no site is automatically assumed to be high-quality by virtue of its address. This means that websites that are immensely popular or belong to .edu domains are automatically considered high-quality in Google's eyes.

There are many benefits of using Google webmaster tools. Check out in the following points some of them.

ADD A SITEMAP TO HELP GOOGLE INDEX RELEVANT PAGES
OPTIMIZE EXISTING POSTS TO INCREASE TRAFFIC
TIDY UP YOUR SITE LINKS
RESOLVE ANY HTML ERRORS
RESOLVE ANY SERVER ERRORS

Indexing is storing and organizing the information found on the pages. The bot renders the code on the page in the same way a browser does. It catalogs all the content, links, and metadata on the page.

Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol.

It works likes this: a robot wants to vists a Web site URL, say http://www.example.com/welcome.html. Before it does so, it firsts checks for http://www.example.com/robots.txt, and finds:
User-agent: *
Disallow: /

The "User-agent: *" means this section applies to all robots. The "Disallow: /" tells the robot that it should not visit any pages on the site.

Google Sandbox is a commonly believed filter that's used by Google to prevent new websites from ranking high in Google's Search Engine Results Pages (SERPs). It's believed that Google uses this filter to put some restrictions on rankings for new websites.

The links shown below some of Google's search results, called sitelinks, are meant to help users navigate your site. Our systems analyze the link structure of your site to find shortcuts that will save users time and allow them to quickly find the information they're looking for.

https://support.google.com/webmasters/answer/47334?hl=en

Schema markup is code (semantic vocabulary) that you place on your website to help the search engines return more informative results for users. The schema markup told the SERP to display a schedule of upcoming hotel events. That, for the user, is exceptionally helpful.

SEO off page will never dead as it is the major p[art of SEO as without generating back links and good references you can not rank in the search engine, So never think about SEO off page dead.

first of all find broken links in your website via any broken link finder like SEMrush then try to find their source and try to rectify these links with their original page OR you can redirect these links at home page of your website.