Text only | Text with Images

Webmaster Forums - Website and SEO Help

Search Engines and Marketing => Website Crawling and Indexing => Topic started by: Appdeveloperdallas on September 05, 2019, 02:25:45 AM

Title: What is X robots tag?
Post by: Appdeveloperdallas on September 05, 2019, 02:25:45 AM
What is X robots tag?
Title: Re: What is X robots tag?
Post by: MVMInfotech18 on September 05, 2019, 05:33:40 AM
The X-Robots-Tag can be used as an element of the HTTP header response for a given URL. Any directive that can be used in an robots meta tag can also be specified as an X-Robots-Tag
Title: Re: What is X robots tag?
Post by: sinelogixtech on September 05, 2019, 05:48:01 AM
Hi Friends,
The X-Robots-Tag differs from the robots.txt file and meta robots tag, though, in that the X-Robots-Tag is a part of the HTTP header that controls indexing of a page on the whole, in addition to specific elements on a page.

According to Google:

"Any directive that can be used in a robots meta tag can also be specified as an X-Robots-Tag."
Title: Re: What is X robots tag?
Post by: davied.kb on September 05, 2019, 07:33:52 AM
X-Robots-Tag can be used as an element of the HTTP header response for a given URL. Any instructions that can be used in the robot meta tag can also be specified as X-Robots-Tag
Title: Re: What is X robots tag?
Post by: RH-Calvin on September 10, 2019, 04:27:10 AM
The X-Robots-Tag can be used as an element of the HTTP header response for a given URL. Any directive that can be used in an robots meta tag can also be specified as an X-Robots-Tag.
Title: Re: What is X robots tag?
Post by: davied.kb on September 28, 2019, 01:48:35 AM
X-Robots-Tag can be used as an element of the HTTP header response for a given URL. Any instructions that can be used in the robots meta tag can also be specified as X-Robots-Tag. ... X-Robots-Tag:noindex(...)
Title: Re: What is X robots tag?
Post by: naazaniqua on October 18, 2019, 03:59:34 AM
The X-Robots-Tag can be used as an element of the HTTP header response for a given URL. Any directive that can be used within a robots meta tag can also be specified as an X-Robots-Tag. ... X-Robots-Tag: noindex (...)
Title: Re: What is X robots tag?
Post by: propertyseo on November 04, 2019, 02:16:07 AM
The x-robots tag is a field in the HTTP response header that allows sites to tell search engines and other crawlers whether or not they are not allowed to access the content found on the URL. In this way, it is very similar to the meta robots tag or the robots.txt file. The difference is that this information is found in the HTTP response header instead of the page source or the robots.txt file on the root of the domain.

A typical x-robots tag:

X-Robots-Tag: noindex
Title: Re: What is X robots tag?
Post by: naazaniqua on November 19, 2019, 04:32:29 AM
The X-Robots-Tag is different from the file robots.txt and robots meta tags, though, in that the X-Robots-Tag HTTP header is part of the control index a page in all, apart from certain elements on the page.
Title: Re: What is X robots tag?
Post by: cityweb on March 26, 2020, 02:11:41 AM
The x-robots tag is a field in the HTTP response header that allows sites to tell search engines and other crawlers whether or not they are not allowed to access.
Text only | Text with Images