Question
When I view the list of Streams in my Hub, I see either "crawlable" or "no-index/follow" next to each Stream under the SEO column. What does this mean?
Answer
The SEO column in the list of Streams (under Content > Streams) shows the current status of each Stream's "robots meta directive". The robots meta directive allows you to influence the behavior of search engine robots, which are the automated systems that analyze ("crawl") websites for inclusion in a search engine's results ("indexing").
You'll see one of three statuses under the SEO column for each Stream:
- no-index/follow: There is a robots meta directive set on the Stream, and it is configured to tell search engines not to include (index) the Stream or any Items within it in their search results. The
followpart of the tag tells search engines that they can still follow links in the Stream to discover other pages (which might be in a crawlable Stream).- Corresponding Stream Option: No Robots Meta Tag is set to off.
- crawlable: There is no robots meta directive set on the Stream. Search engines are able to crawl and index the Stream, and its content will appear in search results for relevant search terms.
- Corresponding Stream Option: No Robots Meta Tag is set to on.
- crawlable (canonical): The same as crawlable, but "canonical tags" are also added to all Item pages that belong to the Stream. The canonical tag tells search engines that a page is a duplicate of an original page (and where to find the original) to avoid duplicate content issues. For more details about canonical tags, see this article.
- Corresponding Stream Options: No Robots Meta Tag is set to on, Enable Canonical Meta Tag is set to on.
You can adjust the setting of a Stream’s robots meta directive by using the No Robots Meta Tag Stream Option. To learn how, see this article:
Block Google (and Other Search Engines) From Searching your Hub Content