8

I have a random domain name which has it's A record going to my dedicated IP. Therefore duplicating content. Is there a way I can disable the domain at IP level maybe in iptables?

I have a .htaccess setup to redirect to the main domain as its always force HTTPS.

Improve this question
1
  • Do you know the domain name? Is it just one?
    – closetnoc
    Dec 4, 2014 at 17:39

2 Answers 2

Reset to default
8

I deal with this type of situation using my virtual host configuration. Under Apache, the first virtual host is the "default" virtual host. I configure it to serve a 404 error with the message

404 Not Found -- Hostname Not Recognized

This server is not configured to serve documents for foo.example.com

Then I create specific virtual hosts for each of my sites that serve the correct content when the host name is correct.

Here is my default virtual host configuration that uses 404.pl to handle all requests:

<VirtualHost *:80>
    Servername localhost.localdomain
    DocumentRoot /var/www/default
    <Directory /var/www/default/>
        Require all granted
        Options +ExecCGI
        AddHandler cgi-script .pl
        RewriteEngine on
        RewriteCond $1 !-f
        RewriteRule ^(.*)$ 404.pl
        AllowOverride None
    </Directory> 
</VirtualHost>

And here is the 404.pl script that prints out the "hostname not recognized" message as well as does redirects for domain names that are almost correct but not canonical:

#!/usr/bin/perl

use strict;

# Put the host names you actually use in here to enable redirects
# The left side should be the "main" domain name and the right should include the TLD
# This enables redirects for alternate TLDs.
my $hostnameredirects = {
  'example' => 'example.com',
  'foo' => 'foo.example.com',
};
my $hostname = `hostname --fqdn`;
chomp $hostname;

my $server = $ENV{'SERVER_NAME'};
$server = "" if (!$server);
$server =~ s/[^\-\_\.A-Za-z0-9]//g;
$server = lc($server);
my $uri = $ENV{'REQUEST_URI'};
$uri = "" if (!$uri);
$uri =~ s/[ \r\n]+//g;
$uri = "/$uri" if ($uri !~ /^\//);

&serverNameRedirect();
&noVirtualHostError();
&show404();

sub serverNameRedirect(){
    my $domain = &removeTld($server);
    while ($domain){
        if ($hostnameredirects->{$domain}){
            &redirect('http://'.$hostnameredirects->{$domain}.$uri);
        }
        $domain =~ s/^[^\.]*[\.]?//g;
    }
}

sub removeTld(){
    my ($domain) = @_;
    $domain =~ s/\.(([^\.]+)|((([A-Za-z]{2})|com|org|net)\.[A-Za-z]{2}))$//g;
    return $domain;
}

sub redirect(){
    my ($redirect) = @_;
    my $eRedirect = &escapeHTML($redirect);
    print "Status: 301 Moved Permanently\n";
    print "Location: $redirect\n";
    print "Content-type: text/html\n";
    print "\n";
    print "<html><body><p>Moved permanently: <a href=\"$eRedirect\">$eRedirect</a></p></body></html>\n";
    exit;
}

sub show404(){
    my $eServer = &escapeHTML($server);
    &errorPage(
        '404 Not Found',
        '404 Not Found -- Hostname Not Recognized',
        "This server is not configured to serve documents for '$eServer'"
    );
}

sub noVirtualHostError(){
    if ($server !~ /^\d+\.\d+\.\d+\.\d+$/){
        return;
    }
    &errorPage(
        '400 Bad request',
        '400 Bad Request -- No Hostname Sent',
        "This server only accepts requests with a domain name, not requests for an ip address such as $server"
    );
}

sub errorPage(){
    my ($status, $title, $message) = @_;
    print STDERR "$title\n";
    print STDERR "$message\n";
    print "Status: $status\n";
    print "Content-type: text/html\n";
    print "\n";
    print "<html>\n";
    print "<head>\n";
    print "<title>$title</title>\n";
    print "</head>\n";
    print "<body>\n";
    print "<h1>$title</h1>\n";
    print "ERROR: $message\n";
    print "</body>\n";
    print "</html>\n";
    exit;
}

# Convert <, >, & and " to their HTML equivalents.
sub escapeHTML {
     my $value = $_[0];
     $value =~ s/\&/\&amp;/g;
     $value =~ s/</\&lt;/g;
     $value =~ s/>/\&gt;/g;
     $value =~ s/"/\&quot;/g;
     return $value;
}
Improve this answer
5
  • 1
    Excellent answer as usual! Nice detail. I may be borrowing from this for my catch-all site which really does nothing.
    – closetnoc
    Dec 4, 2014 at 16:02
  • Better to make it a 410 Gone error. 404 is temporary; 410 is forever.
    – Michael Hampton
    Feb 19, 2019 at 18:04
  • @MichaelHampton to me "410 Gone" implies that it used to be there but is no longer. In my mind a "404 Not Found" for unknown domains is more appropriate.
    – Stephen Ostermiller ♦
    Feb 19, 2019 at 19:02
  • The purpose is SEO: The 410 is a much stronger signal that the URL is not valid and is not likely to ever be valid. But search engines almost ignore the 404. The point is to reduce the search engine ranking much more significantly and prevent it from coming back very easily.
    – Michael Hampton
    Feb 19, 2019 at 19:44
  • Google treats a 404 and a 410 identically except the give 404 errors a 24 grace period before removing them from the search index after crawling them. It often takes weeks to get all the pages to be removed crawled so the extra day doesn't make it much faster.
    – Stephen Ostermiller ♦
    Feb 19, 2019 at 21:10
1

Another proper instrument for dealing with black hat sites that steal content: (canonical URLs):

<link rel="canonical" href="https://blog.example.com/green-dresses-are-awesome" />

Criminals can make an unauthorized copy of your site and host it on thir own servers in order to get the benefits from your content (via Web scraping). Using the canonical link tag helps unless the criminal modifies the links to point only to his own domain. I don't know how to fight him in that case.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.