4

Here is the deal, I have an app that gets content via a webview of my website on the internet. The only thing am changing is that am setting a custom user-agent for this webview app, so I am able to target it.

Meaning:

Case 1: The regular website viewers with a browser will see the normal site.

Case 2: The users from the app will see the same website but with some changes in the page all on CSS and JS level, like changing styles, hiding elements, editing titles with js.

Will this affect my site SEO in any way? My thinking is no since the webpage is the same thing, and if search engine bots visit the page they will see the regular page, since they are not using the "custom user-agent" I created. Do search engines notice the changes made by JS? Are my thoughts valid? what do you guys think of this concept.

I know that's not how to do a proper, and friendly app, I am implementing this temporary, until I move the project to an actual native app, I just want to make sure it won't affect the site SEO in anyway.

1

Your app has not only the issue with a quasi-cloaking (displaying slightly different kind of view, as the website does), but with duplicated content too (taking the content from the website). Forbidding the bot access to the app will close all issues. If I were you, I would close the app crawling.

Up I don't know, HOW you wrap the website into the app, but imagine, that URLs remain the same, and you change just a kind of displaying.

In example, you website has urls like example.com/page, and your app has urls like example.com/page?view=app. So, and now you setup for urls like ?view=app your special JS and CSS, which let content displaying on the app way.

In this case in would be fully enough and secure to do the following:

  • close with robots.txt crawling of app's JS and CSS, like Disallow: /app/staff/
  • close your app urls from crawling with robots, like Disallow: *view=app*
  • if your app urls are already indexed, then firstly noindex them, and, after you realize, they are no longer indexed, then close it with robots.

Deindex based on ?view=app could be done on the simple way with something like

<meta name="robots" content="<?php
if ($_GET["app"]) {
    echo 'noindex, nofollow';
}
else {
    echo 'index, follow';
}
?>" />
  • What do you mean by app crawling, bots have access to the app? do you mean the user-agent? How can they be stopped? – Kash Sep 18 '18 at 9:07
  • it depends on what you already did to make app indexing possible. If your developers were following app indexing guidelines, like this developer.android.com/training/app-indexing, then just roll back these steps. – Evgeniy Sep 18 '18 at 9:12
  • 1
    @Evgeniy I believe (from the mention of "native" apps) that Kash is talking about a mobile app that is currently wrapping a web site - rather than a web-app that is duplicating content that can be discovered by Googlebot. – Zhaph - Ben Duguid Sep 18 '18 at 15:07
  • 1
    @Zhaph-BenDuguid in this case is it much easier do close it from crawling - just a line into the robots.txt, and, if something is already indexed, firstly deindex, and then close from crawling. Main thing i mean is to get rid of duplicated and variated content in index. – Evgeniy Sep 18 '18 at 15:23
  • That's true am only, am only wrapping the website not creating a web app. editing robots.txt seems a good caution step. but this does not answer my question fully. are bots able to dedict changes made by JS and CSS? if the HTML is the same? – Kash Sep 19 '18 at 17:06
2

We did something similar for a couple of mobile apps we built without any obvious detrimental effects to the site's ranking.

We had native apps (Android and iOS), but wanted to show terms and conditions, help and privacy pages in-app as well as on the marketing websites for the apps. Rather than duplicating the content across all three we just used the web pages and removed the header, footer and all external links when viewed via WebViews in the apps.

The main issue is when you show the Googlebot crawler different views to other web users (including the mobile Googlebot and mobile users).

  • "showing the Google crawler different views to other web users." what do you mean by this, you confused me a bit, do you mean there's a conflict in how bots crawl pages based on the user-agent? – Kash Sep 18 '18 at 9:10
  • 1
    Google penalizes for showing Googlebot different content compared to regular users of the main website. – Stephen Ostermiller Sep 18 '18 at 9:25
  • 1
    @Kash - yes, as Stephen mentions (and hopefully I've clarified) Google will penalise your site if it notices that you display different content and views to the Googlebot crawler than you display to a "regular" user. If you use a custom useragent to show different views, that should be fine - as the Googlebot won't know about it. – Zhaph - Ben Duguid Sep 18 '18 at 15:06
  • 1
    @Zhaph-BenDuguid That's answers my concern thank you! I am indeed using a seperate custom user-agent. – Kash Sep 19 '18 at 17:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.