What does "crawlable" and "no-index/follow" mean?



When I view the list of Streams in my Hub, I see either "crawlable" or "no-index/follow" next to each Stream under the SEO column. What does this mean?



The SEO column in the list of Streams (under Content > Streams) shows the current setting of each Stream's Robots Meta Tag, which is a tag placed in the <head> section of every Stream's HTML code. This tag allows you to modify behavior of search engine robots, which are the automated systems that analyze ("crawl") websites for inclusion in a search engine's results ("indexing"), with regard to the Stream.

You'll see one of three settings under the SEO column for each Stream:

  • no-index/follow: A Robots Meta Tag is set on the Stream. The presence of the tag tells search engines not to include the Stream or any Items within it in their search results. The follow part of the tag tells search engines that they can still follow links in the Stream to discover other content (that might be in a crawlable Stream).
  • crawlable: No Robots Meta Tag is set on the Stream. Search engines are able to crawl and index the Stream, and its content will appear in search results for relevant search terms.
  • crawlable (canonical): Same as crawlable, plus the Enable Canonical Meta Tag option is also enabled. This indicates that a separate meta tag has been set on the Stream that tells search engines that the Items in this Stream are originally from somewhere else (like an external blog).

You can learn how to adjust the settings that determine what you see under the SEO column in this article:

Block Google (and Other Search Engines) From Searching your Hub Content

0 out of 0 found this helpful



Please sign in to leave a comment.