meta robots
Directives that control how search engines index the page and follow its links.
The meta robots tag gives page-level crawling and indexing instructions to search engines. The most common values are "index" (include this page in search results — the default) and "noindex" (exclude this page). Follow/nofollow control whether Googlebot follows the links on the page.
Multiple directives are comma-separated: content="noindex, nofollow" means do not index and do not follow links. Additional values include "noarchive" (do not cache the page), "nosnippet" (do not show a text snippet or video preview), "max-snippet:N" (limit snippet to N characters), "max-image-preview:large|standard|none", and "noimageindex" (do not index images on this page).
Critical distinction from robots.txt: a page blocked by robots.txt cannot have its meta robots tag read (Googlebot never fetches it), so noindex in meta tags only works on crawlable pages. If you want to remove a page from the index, keep it crawlable and add meta robots noindex.
HTML Example
<meta name="robots" content="noindex, nofollow"> <!-- Or for specific crawlers: --> <meta name="googlebot" content="noindex"> <meta name="bingbot" content="noindex">