2

I am working for an online casino, and the blog page is deemed Duplicate without user-selected canonical to the main casino page? The two pages are completely different and have nothing to do with each other. The casino page has slots on it and the blog page has articles on it.

https://betbeard.com/en/blog/all - Duplicate

https://betbeard.com/en/casino/all/all -Main

As you can see, both pages are completely different and have nothing to do with each other, why is then the main casino page with all the slots, considered canonical to a blog page that has articles in it?

In the image you can see how the blog page is duplicated to the main casino page

1 Answer 1

2

Googlebot Can't Tell the Difference Between the Pages

Googlebot is not rendering your js, and to a crawler, these two URLs have identical source code:

<!DOCTYPE html>
<!--[if IE 9]>
    <html class="ie-9">
    <![endif]-->
<!--[if (gt IE 9)|!(IE)]><!--><html><!--<![endif]-->
    <head>
        <meta charset="UTF-8" />
        <meta http-equiv="X-UA-Compatible" content="IE=edge" />
        <meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=1,user-scalable=0" />
        <base href="/" />
        <script>
            window.dataLayer = window.dataLayer || [];
            function gtag() {
                dataLayer.push(arguments);
            }
            gtag("js", new Date());

            gtag("config", "G-FDGQ0V5B3K");
        </script>
        <script src="//www.googleadservices.com/pagead/conversion.js" id="tracking-for-register" async></script>
        <meta name="theme-color" content="#11171f" />
        <meta name="msapplication-navbutton-color" content="#11171f" />
        <meta name="apple-mobile-web-app-status-bar-style" content="#11171f" />
        <link rel="icon" href="/dist/favicon.png?ed73165d90311d557ff2" />
        <script defer="defer" src="/dist/scripts/sharedComponents.1c24bda6bdb4126272c3.js?ed73165d90311d557ff2"></script>
        <script defer="defer" src="/dist/scripts/utilityVendor.7a5e9735d0f8337bb8ff.js?ed73165d90311d557ff2"></script>
        <script defer="defer" src="/dist/scripts/reactvendor.9719aeaf251394ac1b6b.js?ed73165d90311d557ff2"></script>
        <script defer="defer" src="/dist/scripts/1681.b20fe3cacd0206be8f83.js?ed73165d90311d557ff2"></script>
        <script defer="defer" src="/dist/scripts/6057.a06b9c565142f80cd869.js?ed73165d90311d557ff2"></script>
        <script defer="defer" src="/dist/scripts/7584.82fbcf467fe788c913d2.js?ed73165d90311d557ff2"></script>
        <script defer="defer" src="/dist/scripts/bundle.de577ed9a6c8b92d717b.js?ed73165d90311d557ff2"></script>
        <link href="/dist/styles/sharedComponents.5a8edf81186e714f3fc6.css?ed73165d90311d557ff2" rel="stylesheet" />
        <link href="/dist/styles/7584.3fc3fd39ddfb930e1b52.css?ed73165d90311d557ff2" rel="stylesheet" />
        <link href="/dist/styles/bundle.90fca7636f6f097f619d.css?ed73165d90311d557ff2" rel="stylesheet" />
    </head>

    <body>
        <div class="app" style="height: 100%;"></div>
        <script>
            navigator.serviceWorker.getRegistrations().then(function (registrations) {
                for (let registration of registrations) {
                    registration.unregister();
                }
            });
        </script>
    </body>
</html>

Pre-rendering/Dynamic rendering

Generally speaking, server side rendering is the safest bet for crawlability. However, I have a handful of friends in the SEO community that have been successful with hybrids.

Pre-rendering/Dynamic rendering are a popular solution.

I don't know the specifics of your app, but pre-rendering would likely be the easiest solution to implement. Generally (to my knowledge), you do not need to change your existing codebase. If you do, the changes will likely be minimal.

Dynamic rendering: Users render client-side version on their device, bots get sent pre-rendered version.

The idea is that first, you preload and cache all HTML pages in advance with Headless Chrome. So when the request comes in, you determine if it’s a user or a bot. If it is, you send that request to a dynamic renderer server that renders the page then gives the static HTML back to the bot.

dynamic rendering pre-rendering diagram

Since Googlebot can and does render js, your underlying problem is likely that your pages are just taking too long to render.

When crawling the web, Googlebot tries to balance getting as much content as possible without overloading everyone's servers.

So it may be the case that Googlebot can render your pages, however, once it gets started backs off because it believes it might crash your server.

How fast do you need to deliver content to bots?

"As fast as you can. One of the things that happens when you’re crawling the web is you’re running into a tradeoff you have to make. On the one hand, you want to make as many HTTP requests as possible to get as much content back from a website as possible. If you are an e-commerce site with a million products, optimally as a crawler, I would make a million HTTP requests in one go, get all the product information back, and then I can update my index based on that, and tomorrow I’ll do the same thing. But at the same time, web servers vary in capability.” - Martin Splitt, Google

"As fast as you can" seems to be the official advice from Google based on that quote from Martin. I don't think they're confident enough yet to say anything more definitive.

However, per the comments it looks like within 3s could be a good target - I'd trust Stephen.

Maybe double check how you handle routing?

In the source on both pages you can see <base href="/" />. Stephen validated in the comments this is probably not part of the problem, but it might be worth looking at to ensure you're handling everything optimally.

<BrowserRouter basename="/calendar">
    <Link to="/today"/> // renders <a href="/calendar/today">
    <Link to="/tomorrow"/> // renders <a href="/calendar/tomorrow">
    ...
</BrowserRouter>

My concern initially was that if this is something that happens during rendering, and Googlebot isn't rendering, how does it see the URL?

We have a lot of regulars that are more experienced in JavaScript SEO than I - so I appreciate any corrections.

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.