(Entry was posted by Loren Baker on this post in the Dumb SEO Questions community on Facebook, 04/08/2017).

robots.txt of Shopify sites

Looking at a Shopify site which has a mess of /collections/ pages that I`d like to prune down. Also thinking of using them for an upcoming project at SEJ. Problem is, their default robots.txt addresses /collections/ but only weird variable occurrences. Here`s an example of their default (not the page, but a cool site nonetheless) https://shop.saltstrong.com/robots.txt Essentially, I would see this like disallowing / deindexing tag, archive and category pages in WordPress (when it makes sense) and to assist with overall crawl budget / doop content. Does anyone have experience editing the robots.txt of Shopify sites and any positive results?

This question begins at 00:18:21 into the clip. Did this video clip play correctly? Watch this question on YouTube commencing at 00:18:21

Share on Twitter

Share on Facebook

YOUR ANSWERS

Selected answers from the Dumb SEO Questions Facebook & G+ community.

Loren Baker: On WordPress I do quite a bit (depends on site), helps with crawl budget and not serving Google tons of thin / duplicate content. Same with tags. Same with Archives. Just haven`t done it on Shopify yet.
Chase Reiner: Why would you noindex categories
Chase Reiner: And are you asking if you should edit robots.txt to noindex categories or archives? If so that would be much easier to understand.
Loren Baker: Also kind of funny they block Nutch and then try and crawl delay Majestic, by default.
Alan Bleiweiss: Recent audit client "we need an audit however we need to inform you up front that we use Shopify, and that will not change, even if you tell us it has to"... Â¯\_(ãƒ„)_/Â¯

View original question in the Dumb SEO Questions community on Facebook, 04/08/2017).

robots.txt of Shopify sites

Selected answers from the Dumb SEO Questions Facebook & G+ community.

CATEGORIES

All Questions in this Hangout

By Jim Munro

Next Hangout:

Find us on Facebook

Today's Temperature.