An agency we hired to assist with SEO mentions that at the start of this year (2025) our web stores indexation has gone down the drain. We haven't changed anything significant. I have also looked into our robots.txt file, Sitemap Generator configuration, SEO Page generator has no errors. Any ideas on what could be causing this?

• What does it say when you click the (1) button? • What does it say when you click the tab at the top left which gives you reasons why pages aren't indexed?

@Steve Goldberg I do find that the Sitemap files that NetSuite generates don't have a standard XML format like so:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="<http://www.sitemaps.org/schemas/sitemap/0.9>">
  <url>
    <loc><https://www.example.com/foo.html></loc>
    <lastmod>2022-06-04</lastmod>
  </url>
</urlset>

There are no tags, is this something with the configuration in the Sitemap generator?

AFAIK NetSuite sitemap generators work fine. Does Google say that your sitemaps are invalid?

Santiago can you share the website url?

@Flo apologies for the late reply: It's happening in http://solarshop.baywa-re.be and http://solarshop.baywa-re.nl

Are you aware of any changes on the website on that specific date? When checking site:solarshop.baywa-re.be/ in google 99% of the results are media files, PDFs actually. Have you checked the index coverage report Error tab? Have you tried using the URL inspection tool for any of the pages that are not being indexed?

I would reiterate what I mentioned earlier: you should click the button in Google Search Console which will give you details about why your pages are not being indexed, because Google will literally tell you the reason and so you don't have to guess. Another thing I would say is that your cookie popup is possibly being flagged as an intrusive interstitial. I would recommend using JavaScript to hide it from bots using by wrapping its call in

!SC.isPageGenerator()

so that it is only called when a human user visits.

I checked on that, but the one on https://solarshop.baywa-re.lu/robots.txt is exactly the same and that domain is being indexed

I don't like the disallow: statement there, it makes me nervous. Also, there's no links to sitemaps

I couldn’t find sitemaps either. But the robots.txt is ok, less than ideal but it is ok. It is basically telling the robot to crawl your entire site.

Alright! But yes, please click that tab in GSC next to the green one, it will tell you why your pages are not indexed.

Hi, I do see the sitemaps submitted:

image.png

.lu works fine

I wonder if there's a redirect somewhere? I used his .nl link at the top of the thread and wound up on a .lu page.

Yes, we have a language selector that can send you to .lu or .nl. Could it be this?

@Steve Goldberg Do you know if NetSuite's built-in Cookie Consent banner implements this same isPageGenerator() method (or one similar) to prevent it from showing to bots and crawlers? I'm about to turn it on, but have anxiety over how it presents to Google and the potential for search & traffic disruption.

Well you could turn it on in your sandbox and then force the page generator to run to see 🙂

Thanks, Steve. Yes, I had enabled in my Sandbox and it didn't look like there was anything suppressing it for non-humans. Seems like it probably won't be good for SEO so I may need to go another route. Appreciate your help, though!

Why do you think it won't be good for SEO?

Google reportedly wants to see the banner load in the footer or using pop-up that doesn't block content. It also wants the banner content to be non-indexed. I'm not worried about Cookie Consent Banners in general. I'm worried about this specific built-in NetSuite one, though. It seems clunky and simple (in a bad way). It's not suppressing or deferring any cookies or tracking assets, and it doesn't give any options other than "Accept" (x to close). If you don't have a way to reject/decline third party tracking cookies, they'll bounce. A higher bounce rate definitely hurts SEO.

Google reportedly wants to see the banner load in the footer or using pop-up that doesn't block content. It also wants the banner content to be non-indexed.

I've not heard this. Can you point me to specific docs?

It seems clunky and simple (in a bad way). It's not suppressing or deferring any cookies or tracking assets, and it doesn't give any options other than "Accept" (x to close).

Yes, this is why we created the Cookie Consent extension when we implemented SuiteAnalytics for SuiteCommerce. We also have extensibility API methods so it's easier to add your own cookies to it.

I'm (stuck) on SiteBuilder, so unfortunately that extension isn't an option for me. Thank you, though. As with many things Google SEO / algorithm related, there's no specific docs. Just anecdotal reports flying around the internet under the heading of "best practice" or "dos and don'ts". Your call out for this gentleman's banner being potentially an intrusive interstitial was a good example a cookie banner potentially going really wrong (and confusing or angering page crawls).