12

I'm building a Wordpress site for a parent of an 11 year old that wanted something to memorialize her daughters athletic, academic and personal achievements. The site includes photos and videos of her and friends, biographical info and blog posts. The domain is privately registered in my company's name, I'm not adding them to Google console and I'm keeping other SEO minimized. There's no last names or physical addresses. I want to have as much web safety in mind as possible to avoid scrapers from grabbing her photos etc, prying eyes etc. I'm probably being paranoid and like all my sites thinking it could get more traffic than it actually does but I figure it's worth the investigation and worthwhile to dot my i's. The child's website is in excellent taste and the parent is very down-to-earth, she understands the internet etc...

Are there any reliable methods I can take to increase Web safety for this 11 year old and her site?

  • 3
    This is a start: webmasters.stackexchange.com/questions/77031/… I will think about other ideas for the site in general. BTW- Good for you in taking on this task! It is a bit of a tough one. But well worth the effort!! I used to do free charitable hosting along with the paid hosting and the noble causes were always my favorite by far. These are the ones I remember! – closetnoc Nov 8 '15 at 23:55
  • 3
    You do realise that nothing you do will make it 100% invisible... all it would take is someone to post a link to it on Facebook or Tumblr, and that website will be out there... The best thing to do is have the parent supervise/approve all content the kid is posting, and educate both on what they need to be careful about – HorusKol Nov 9 '15 at 8:28
  • 4
    A robots.txt file with the right contents can keep all the legitimate bots out. The hard part is the rest. A lot of them can be kept away if the address of the site is hard to find. – kasperd Nov 9 '15 at 8:35
  • 5
    In addition to everything that's already been said, be aware that Wordpress leaves image EXIF data intact... – user1103 Nov 9 '15 at 8:47
  • 3
    Does this need to be a website? If you don't want a lot of it spreading, and getting into the wrong hands, why put it on the internet? Could you not make something else in her memory and give it to the parents/family/people involved? – Tom.Bowen89 Nov 9 '15 at 9:58
20

I'm probably being paranoid

Maybe I'm being paranoid, but this sounds like it should be an entirely private blog/website. ie. password protected. Who exactly is the target audience?

Apart from the security aspect (preventing the unscrupulous from finding and using the content), this sort of content sounds like it would be ripe for bullying from other school "friends". Content which might be OK at first - to an 11 year old - might just become embarrassing in a few years.

I'm not adding them to Google console

This would seem to be backward? The way you hide content from Google (ie. a "good" bot) is to use the robots meta tag (or X-Robots-Tag header) and maybe robots.txt. Omitting it from Google Search Console will not help in this respect.

At least if you add it to Google Search Console you can monitor things like backlinks, check robots.txt, etc. If indeed you do go public.

  • 1
    This is really the only sane solution. +1 – MonkeyZeus Nov 9 '15 at 13:23
  • 4
    Brief clarification - not adding a site to Google Search Console means not telling Google directly about a site. This means you're not going out of your way to make Google notice you. However, that doesn't mean that Google CAN'T notice you - you would use the robots files for that, as w3d suggested. Furthermore, making the site password-protected would mean that the most Google could index would be the login page. – Jake Nov 9 '15 at 15:11
  • 11
    "The way you hide content from Google" is to not put it on the internet in the first place. – Lightness Races in Orbit Nov 9 '15 at 16:28
  • 2
    I think password protecting the directory would defeat the reasons the mom wants a site , in that case we could have just made a word doc news letter and emailed it out. The daughter wants to start a blog. The moms not stupid she's screening and editing the content. I don't believe they are doing anything irresponsible here. This is excellent feedback on this question. – rhill45 Nov 9 '15 at 16:40
  • Password protecting the site in WP has nothing to do with securing the media. – blankip Nov 9 '15 at 23:24
6

The only appropriate answer is to password protect the whole thing. HTTP BASIC_AUTH is probably the simplest to set up, as it will not interact with WordPress in any way. That on its own will be enough to deter all scrapers, but if you want proper security you should also use HTTPS.

(Sidenote: With many systems, a HTTP page will redirect to HTTPS. However, with HTTP BASIC_AUTH, that redirect can be after the prompt for your password. The HTTPS page will then prompt for password again. This means that your password has been entered twice, once in cleartext and once over a secure channel. It is in principle possible to have different passwords for the HTTP and HTTPS versions, or to have no password for the HTTP version: all it does is redirect to the HTTPS version, which then asks for your password. How easy this is to set up depends on which tools you are using to manage your website hosting preferences. Alternatively, simply ensure that you always navigate directly to the HTTPS page, bypassing the insecure version. If you use a password system other than HTTP BASIC_AUTH, then probably none of this sidenote will apply.)

  • 4
    If you want to go the password protected route, given that this is a WordPress site, simply using WordPress to handle all of it is the far easier method: codex.wordpress.org/Content_Visibility#Private_Content – Doyle Lewis Nov 9 '15 at 14:19
  • 1
    @DoyleLewis. Will that protect static assets, such as uploaded images? Admittedly, a crawler is unlikely to find them (as long as you have Options -Indexes). – TRiG Nov 9 '15 at 16:40
  • I considered https but only issue is cost. I wish I could find a way to run her site under my companies ssl but of course not possible – rhill45 Nov 9 '15 at 16:42
  • 2
    letsencrypt.org @rhill45. – TRiG Nov 9 '15 at 16:49
  • 1
    @TRiG If someone had the URL to a static media file, then no, it wouldn't protect that. But no crawler would ever get to it since they wouldn't be able to get to the content that would link to the file. – Doyle Lewis Nov 9 '15 at 17:12
3

First, I'll make a grand apology to all the professional webmasters out there, but for this OP, I have one golden suggestion:

Violate search engine guidelines

And I mean do it to the point where the important content is in complex javascript and the content robots can crawl is not in proper HTML. This includes a bad description tag, a bad title tag, etc. Heck, maybe turn the entire content into just a video made in flash, or show the entire content as just one image. That would really make the search engine crawler cringe.

I'll show by example in code:

Here's a way to get something indexed:

<!DOCTYPE html>
<html>
<head>
<title>Web page</title>
<meta name="description" content="This is a wonderful web page">
</head>
<body>
<h1>A wonderful web page</h1>
<h2>By John Smith</h2>
<p>This is a wonderful page. ya de ya de ya de ya de ya de ya de</p>
<p>This is wonderful. ya de ya de ya de ya de ya de ya de</p>
</body>
</html>

Ok, I admit, the text isn't perfect, but you understand what I mean.

Now if you want to hide it from crawlers and do it the simple way, you can try this:

<!DOCTYPE html>
<html>
<head>
<title>Private</title>
</head>
<body>
<img src="mywebsite.jpg" width=1024 height=768>
</body>
</html>

then make an image named mywebsite.jpg and include all the text in that, not in the html shown above. Then you need to protect mywebsite.jpg by making a watermarked version of it to the users who are unauthorized to see the real thing. Simply compare user agent strings or ip addresses against those you allow/disallow for the image. This type of thing can be done in .htaccess with some rewrite rules.

For example, to force googlebot to see the watermarked image instead of the real thing, use these rules:

RewriteCond %{HTTP_USER_AGENT} ^googlebot$ [NC]
RewriteRule ^mywebsite.jpg$ specialrobotimage.jpg [L]

I'm assuming here that mywebsite.jpg is your real website as an image and specialrobotimage.jpg is the watermark or the image as a message stating only real users are allowed to see the information. Also, the rules assume everything is in the same folder.

  • Indeed the JS thing might be the way to go for a lot of it. While some bots do run JS, scrapers and what not often do not. This means that various HTML DOM objects can be set to the real content when the JS runs. I do not suggest relying upon user agents since this is often forged by scrapers. Consider installing ModSecurity and let that do most of the work for you. – closetnoc Nov 9 '15 at 3:17
  • 11
    This is really poor advice. It's a lot of work with no real benefit. A ton of bots run JavaScript these days. Content in a video or image is not easily maintainable (plus both are still regularly indexed). Even content in Flash has been indexed for years. – Brad Nov 9 '15 at 22:16
  • Ok I did forget to mention that no-indexing should be applied to the images and videos. I understand they are not easily maintainable, but at least the text cannot be as easily modified. If on the other hand, just raw text is on a page, then a crawler could take the chunk of text, modify it, add a template to it and then build another website site out of it. I doubt the crawler has a capability of extracting text from images or videos. – Mike Nov 10 '15 at 4:15
3

First this is really a WP question. I have wrote 20+ sites that do what you need so this is pretty easy.

1 You make everyone login to view every page.

2 You lock the uploads folder via script and .htaccess. There are scripts that will check for user login before allowing them to view media.

If you want to do anything in between this and having your site wide open - it is a lot of work. The easiest way to do it is have two uploads folders - one with security and one for everything else if you want to have some pages wide open to the public and some not.

As for what others are saying about content - can't find it if your pages are locked... that is not really true. I have robot scripts that will search the crap out of a folder for file names.

All the google and robots talk is nonsense. That stuff only matters if you want to half-ass it. If you do then take advice from some of the upvoted questions above.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.