I have got a e-commerce website and and just ran a audit in semrush and it`s come back with loads of duplicate content issues best way to fix this? no-index the one`s that have no traffic or ranking? Cheers
Selected answers from the Dumb SEO Questions G+ community.
Alan Bleiweiss: The best way to resolve it will depend on what you mean by "loads of duplicate content" based on a 3rd party industry tool that sets their own pre-determined threshold for how much duplication triggers that specific warning. Those pre-determined thresholds are artificially determined, and not considerate of individual site or topical strength needs or circumstances.
While it`s good to get a warning, it`s vital to apply critical thinking to the situation to determine if it`s even a problem to address.
Maybe you have 500 product pages and 80% of the content is the same, but you have enough variations in the other 20% to actually justify those as unique product pages.
Maybe you need to consolidate multiple product pages into fewer pages where you have a radio button or dropdown menu form for size, color, style.
Maybe you have pages that use product sorting and filtering functionality where only the raw, unsorted or unfiltered version deserves to be indexed and all the other variations need to have a robots.txt setting to not even crawl those URL variations.
Maybe you have a case where you need to first set some pages to noindex in the meta robots tag, wait for the site to be recrawled, and only after that, add the entry in the robots.txt file.
Maybe you have many pages with very little unique content, yet they serve the needs of your unique site goals and visitor value.
Maybe your situation is a hybrid of all of these or something else entirely.
Jason Mun: Without seeing your SEMrush report and having dealt with lots of Ecommerce websites in the past, I am going to guess that your filtering, sorting and faceted navigation at a product category level is causing the issue (as mentioned by Alan Bleiweiss). These generally create URL parameters like:
Because the core content of these URL variations are almost identical - meta-data, heading, text content they would be flagged as duplicates by a crawler like SEMrush.
How you deal with it would depend on your situation. In most cases, I like to apply NOINDEX, FOLLOW on URLs that have those offending URL parameters if they are already indexed and causing bloat issues. Once you confirm that they are out of the index, you apply a robots.txt block.
Remember to make use of your URL parameters handling feature in Google Search Console that is at your disposal.
Tatsat Savsani: Dealing with same issue, even pages with about 1, 2 unique paragraphs are shown as duplicate because of top navigation and footer, decided to block those along with tags and categories page