28

Regular Expressions are not valid in robots.txt, but Google, Bing and some other bots do recognise some pattern matching. Say if you wanted to block all URLs that have a example any where in the URL, you can use a wild card entry * User-agent: * Disallow: /*example You can also use the dollar sign $ to specify that the URLs must end that way. So if you ...


3

RewriteCond %{QUERY_STRING} (^|&)v1=123&v2=(\d+).*$ RewriteRule ^sub/$ sub/?v1=789&v2=%1 [NE] You are using the wrong backreference in the substitution string. You have 2 capturing subgroups in the preceding condition, so you would need to use %2, not %1 as you have done. %1 is a backreference to the first captured subgroup, ie. (^|&) - ...


3

Maybe something like: RewriteEngine On RewriteRule ^(example)/(\d+)/([\w-]+)/(static/1\.html)$ /$3/$1/$2/$4 [R=302,L] This externally "redirects" a URL of the form /example/{dynamic_id}/{dynamic_name}/static/1.html to /{dynamic_name}/example/{dynamic_id}/static/1.html. I'm assuming that {dynamic_id} consists of only digits and {dynamic_name} is ...


3

After attempting what lunametrics did to combine their URIs with & without ending slashes, and not getting it to work, I started to think about the problem in a different way. Instead of appending slashes when they are missing, why not remove them if they are there? i.e. example.com/path/ would then become example.com/path in my reports. Note: This is ...


3

RewriteRule ^lp/?$ /index.php [L] This fails to match because your actual URL is probably /lp/index.php (or whatever your DirectoryIndex is), not simply /lp/. (That is after Apache has internally mapped the URL to a filesystem path). So, try changing your RewriteRule pattern: RewriteRule ^lp/index\.php$ /index.php [L] Specifically, there is a conflict ...


3

If the URL contains a / followed by any characters or none followed by a / followed by sitename followed by any characters or none, remove sitename: rewrite ^(.+)/sitename(/.*)$ $1$2; Explanation: you need to capture everything before ($1) and after ($2) the match, in order to construct the result. If the rewritten URL contains /civicrm/ more than ...


3

Use HTTP_HOST not HTTP_REFERER <IfModule mod_rewrite.c> RewriteEngine on RewriteCond %{HTTP_HOST} ^(www\.)?olddomain\.com$ [NC] Rewriterule ^(.*) http://newdomain.com/ [L,R=301] </IfModule>


2

You could add data-attributes to your ad link if you are using HTML5: <a href="example.com" class="someclass" data-adposition="Banner Ad - Top" data-adnamesize="Ad Title - (588x88)" data-advalue="1.00" ><img src="example.com/ads/ad_588x88.jpg" alt=""></a> Then add a new ...


2

I wasn't convinced this would work, but I've tested it and it does seem to. Quite simply: ^.{0}$ Where: ^ is the start of the match . is any character (except newline) {0} is precisely zero times $ is the end of the match


2

Ok, so I was able to to solve my problem. First I setup a new advanced filter: Filter name: remove all query parameters from URI except query and page Filter type: advanced Filter field A: Request URI Filter Field A pattern: (.*?)\? Filter field B: Request URI Filter Field B: [\?|&](page=\d+|query=[^&]*) ...


2

it looks quite similar to PCRE syntax Yes, many of the parameters to Apache directives do take regular expressions (regex) and the "flavour" of regex used is indeed PCRE. This applies to mod_rewrite and in fact any other Apache module that uses regex. There are some minor differences (and additions) in the syntax used by Apache, but this is primarily ...


2

This can be done using the -v (for --invert-match) option of grep: grep -v "excluded_word" access_log | grep ' 404 ' grep -v "excluded_word" access_log will return all the lines that don't have the unwanted word and then it is piped to grep ' 404 ' to list only lines with that pattern. Since an access_log can be pretty big, a faster way is to use awk awk ...


2

RewriteCond %{REQUEST_URI} /(apples|bananas)/* RewriteRule . /page/faqs.php?cat=$1&question=%{REQUEST_URI} Almost. But you need to use %1 instead of $1 as a backreference to the first captured group in the last matched CondPattern (RewriteCond pattern). $1 is a backreference to the first captured group in the RewriteRule pattern (but you have none, so $...


2

I would use the regular expression on the host name: ^sub17(\.exh)?\.domain\.com$ The . need to be escaped because otherwise they match any character. The parenthesis around \.exh with the question mark make it optional to allow either version of your host name to be matched. The ^ at the beginning and $ at the end ensure that it is an exact match. The ...


2

Your regular expression is pretty close if you remove the $s. Those means "ends with," which isn't what you want. I'd take the approach of having a single "starts with slash" (^/) followed by your dates with parenthesis for grouping, and explicitly say it can end with anything (.* ): ^/((2019/03/01)|(2019/03/02)|(2019/03/03))/.* That should work with ...


1

I find that one of the most useful ways to analyze 404 errors from a log file is to look at the referrer. You want to know where the 404 error was clicked, and the referrer tells you that. In addition, bots almost never send the referrer string. When you look at only records that contain the referrer, it gives you more information and excludes bots. ...


1

%1 is the contents from the first set of parenthesis in your rewrite condition. You have used TWO sets of parenthesis: (^|&) and (\d+). %1 is matching the first of those which contains nothing but the "start of the expression" (^), so no characters. You have two possible solutions: Change your rewrite rule from using %1 to using %2. Change your ...


1

RewriteRule ^de/kontakt/anfahrt$ kontakt_anfahrt.html [NC] RewriteRule ^en/contact/approach$ kontakt_anfahrt.html [NC] im rewriting evertything to a single file and this this acts like a front-controller. The above two mod_rewrite rules rewrite the specific URLs /de/kontakt/anfahrt and /en/contact/approach to your "front-controller". To rewrite ...


1

If the thank you page is the same url for both goals, then anyone reaching the thank you page will show as a goal conversion in the goal reports. Goal funnels only affect the goal funnel report. I would also change the regex. ATM your not using regular expressions for the goal urls, you simply have urls entered. Assuming none of the pages have ...


1

Perhaps try simplifying your filter to something like this: So something like /test1/test2////// becomes /test1/test2/, or /test becomes /test/. Note that you don't need to escape / slashes as GA uses a different regex engine than what people normally use.


1

I found a solution, may will be useful for somebody. If searched substring located at the end of URL: RewriteCond %{REQUEST_URI} ^(.*)(machine-ab123|machine-123)$ RewriteCond %{REQUEST_URI} !^/cat11/cat22/cat33/bla11-bla22-bla33-machine-ab123 RewriteRule ^(.*)(machine-ab123|machine-123)$ /cat11/cat22/cat33/bla11-bla22-bla33-machine-ab123 [R=301,L] If ...


1

If first url string 1 contain "tool-model-" at the end AND does not contain "tool" catalog at the beginning make redirect to http://www.example.com/tool/tool-model- PLUS two digits (string 2) To simplify/combine your redirects into a single ruleset I would use mod_rewrite, rather than mod_alias (Redirect or RedirectMatch) since you want a condition that ...


1

Just put /contact-us/thanks.aspx and it should work.


1

Since you've not clarified the specifics of your question, I'll make some assumptions: You wish to redirect to a different domain. eg. example.net example.net points to a different server. /CDadmin is a physical directory on the filesystem. At the top of the .htaccess file in the document root of example.com, place the following: RewriteEngine On ...


1

This restructures the way an url is displayed in GA. It uses a regular expression, i.e. pattern matching, on the request uri (the url a visitor has called). The parenthesis are capture groups, i.e. the matching values are stored in variables. The variables originate in field A, so their name begins with "A", and they have to numeric index that matches the ...


1

That question made me mix quite some things. This one works as expected according to what I tested, with | operator, taking advantage of the possibility in Apache 2.4 to use the $ sign as line ending: <DirectoryMatch "/opt/lampp/htdocs/[^/]*/public/(uploads(/[^/]*)?|(wp/wp-content/uploads(/[^/]*)?))$"> Require all granted </DirectoryMatch> &...


1

For blocking urls by regular expressions not from crawling (downloading) but from indexing in search engines you can use https://developers.google.com/search/reference/robots_meta_tag dynamic adding of HTTP header X-Robots-Tag into mod_rewrite rules of Apache (or nginx) that widely supports regular expressions.


1

I was able to get it to work using the following: <IfModule mod_rewrite.c> RewriteEngine on RewriteCond %{HTTP_HOST} ^(www\.)?olddomain\.com$ [NC] Rewriterule ^(.*) http://newdomain.com/ [L,R=301] </IfModule> Answer provided here: 301 redirect doersnt work in SERP


1

Yes your regular expression is wrong. \/ is a literal slash (/). A slash never preceeds the domain name. You also specify that the referrer is exactly just the domain name. That is never the case. You are looking for a "contains" rule so you should leave off the starts with (^ ) and ends with ($) markers. You also might want to escape the literal ...


Only top voted, non community-wiki answers of a minimum length are eligible