snork.ca: Welcome to the brown age of computing! Mainly Sunny, 24.2°C [H: 30°C] - No precip

Update: Redirecting Script Kiddies With Nginx 2016-07-27

I was screwing around with a VPS from SSD Nodes when I discovered that the

if=$logstuff

directive did not exist before Nginx version 1.7.0 (thanks to this post here). I then discovered that the Nginx version in the Jessie repositories is actually 1.6.2-5+deb8u2. So... how does one get a more recent version of Nginx? According to the Nginx wiki you have to:

sudo apt-get remove nginx

Then add the following to your /etc/apt/sources.list:

deb http://nginx.org/packages/debian/ jessie nginx
deb-src http://nginx.org/packages/debian/ jessie nginx

Followed by these commands:

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys ABF5BD827BD9BF62
sudo apt-get update
sudo apt-get install nginx

This will give you the current version of Nginx (which is 1.10.1-1~jessie) and will give you the ability to upgrade with a simple apt-get update/upgrade. Of course, if you are running Ubuntu, or a different version of Debian, then you'll have to modify the sources.list entries accordingly.


Update: Redirecting Script Kiddies With Nginx 2016-07-09

As it turns out some of the rewrite rules I had made were not always firing, because they were case sensitive. It didn't take long to find the answer to how to fix that. I found this post here at ServerFault describing how to make Nginx rewrites case insensitive. So, as a result, I have modified the global and non_wordpress rewrite sections below.


Redirecting Script Kiddies With Nginx 2016-06-15

Back in April I made a post about redirecting script kiddies with Apache rewrite rules and conditions. These days www.snork.ca is just a redirector site that points browsers in the right direction, archive.snork.ca is where I have left almost all of my old [pre 2016] posts, and snork.ca is this not-so-fancy site that should hopefully keep me happy for a while. All of them are now running on Nginx, and have been since the "archive split". The transition went okay, but the rewrite rules were the most irritating to migrate. Don't get we wrong, I know there are simple utilities that can be used to convert your rules from Apache to Nginx, but a simple converter is not what I needed, especially with the rules being applied to multiple Apache virtual hosts. What I needed (and perhaps you do as well) was a complete rewrite of all my script kiddie redirection scheme.

image Where I walk my dog.

The main difference between redirecting with Apache and redirecting with Nginx is that for some rewrites, Nginx is just plain better off doing it with the map module, while Apache can easily just have a bunch of rewrite statements thrown at it. The map module lets you easily create a new variable, populate it based on some other pre-existing variable, and then make changes based on the content of the new variable. Reread that sentence again if it doesn't make sense the first time... the use for this will become clear soon.

For starters, before we even get to the maps, I think it is a good idea to split up your Nginx config in to multiple files which helps to keep things organized in to manageable chunks. In my /etc/nginx/nginx.conf file I have a statement that looks like this:

include /etc/nginx/conf.d/*.conf;

I am pretty sure that is already in the default install of Nginx that comes from the debian repos, and I assume also most other default installs of Nginx. This basically allows you to create a bunch of .conf files in that directory and Nginx will read and interpret all of them as config files. I then created a separate .conf file for each virtual host (Apache terminology I guess) such as snork.ca.conf, archive.snork.ca.conf, and gfy.com.conf. Having your config split across multiple files means you may have to duplicate some of your config options in multiple places but it also allows you to apply [or not apply] config options on a per-host basis.

Now, let's jump right in and begin with an example map:

map $http_user_agent $myagent {
   ~MyMagicAgent 1;
   ~PopularCrawler 1;
   default 0;
   }

What this map essentially does is to create a new variable called "myagent", and for each request sent to Nginx the variable will be a 0 (zero) by default or a 1 (one) if the requesting user-agent string is MyMagicAgent or PopularCrawler. So far all this does is populate $myagent with a one or zero, it has not affected how Nginx treats the request in any way. Now, you can also use any of Nginx's internal variables to set your request-specific new variable. For example, you could setup a map based on the actual file being requested like this:

map $request $nologurl {
   ~/favicon\.ico 1;
   ~/apple-touch-icon 1;
   default 0;
   }

There is also the geo module which is very similar to the map module. It allows you to set your own variables based on the IP address the request is coming from. For example, if you did not want to log requests coming from your own LAN or from well known crawlers you could do this:

geo $remote_addr $crawler {
# Localhost and LAN
   127.0.0.0/8 1;
   192.168.0.0/24 1;
#Googlebot
   66.249.64.0/19 1;
# Yahoo! Slurp
   68.180.228.0/23 1;
   68.180.230.0/24 1;
# bing
   40.77.0.0/16 1;
   157.55.39.0/24 1;
   207.46.0.0/16 1;
# and don't forget default
   default 0;
   }

Remember, these maps just populate the variables, they do not affect how Nginx treats the request in any way. So what I like to do is put these maps (and geos) in the nginx.conf file so that the contents of my mapped variables is available to the other sub-config files for each virtual host... at least I think it works that way. Then in each of my virtual host conf files I can decide which maps I wish to use and which I do not. So, let's say for example that I do not wish to log any of the items listed above for my "snork.ca" virtual host. Not the special user-agents, not the favicons and apple icons, not my own local traffic, and not even traffic from the popular crawlers. First I would setup my logging like this in my ./conf.d/snork.ca.conf:

access_log /var/log/nginx/access.log snorklog if=$logstuff;

This means that requests to snork.ca will be logged to the access.log file, they will be in the special "snorklog" custom format I created (you could just as easily use the "main" log format that comes by default with nginx), and requests will only be logged if the variable $logstuff is zero. Now, also in your sub-conf file you also need to setup your decision making "if" statements to decide which rules to use. On a side note, a lot of Nginx fanbois seem to enjoy pointing out the If Is Evil article on the official Nginx web site, which makes it sound like nobody should use "if" statements ever. However, the very first paragraph of that article says:

Directive if has problems when used in location context, in some cases it doesn't do what you expect but something completely different instead. In some cases it even segfaults. It's generally a good idea to avoid it if possible.

So the lesson is to not use if statements in a location context. As near as I can tell, that means using an if statement in your server context is not what they are warning against. Now, having said that... in your server context in your ./conf.d/virtualhost.com.conf file before your location statement(s) put something like this:

if ($myagent) { set $logstuff 0; }
if ($nologurl) { set $logstuff 0; }
if ($crawler) { set $logstuff 0; }

These statements will set the $logstuff variable for you and your access_log statement will not log those particular requests. But what if you wanted to block some requests instead of excluding them from logs? Well, in the case of my fancy new snork.ca site, I have no need for POST requests (as opposed to HEAD or GET requests) and as it turns out you can block POST requests really easily without even creating a map, by putting something like this in the snork.ca.conf right after the previously mentioned if statements:

if ( $request_method ~ ^POST$ ) { return 403; }

BAM! No more POST requests. This is probably a good time to mention that this is a good example of why I wanted to separate my conf files by virtual host... because I also have archive.snork.ca running on this server, which is a Wordpress based site and POST requests might actually be needed there. With my conf files separated by virtual host I can allow POST requests on the hosts I decide by just not using that if statement on them. Okay, so what about an example of blocking that does use a map? Well, how about blocking and not logging certain crappy user-agents? Make a map of crappy user-agents in your nginx.conf like this:

map $http_user_agent $badua {
   ~*masscan 1;
   ~*enzu 1;
   ~*ia_archiver 1;
   ~*SafeDNSBot 1;
   ~*Cliqzbot 1;
   ~*appengine 1;
   ~*/dev/null 1;
   ~*/bin/sh 1;
   ~*Mozilla/4.0 (compatible; Synapse) 1;
   ~*semanticbot 1;
   ~*SurdotlyBot 1;
   default 0;
   }

And then go to your server specific conf file and add this at the end of your block of if statements:

if ( $badua ) {
   set $logstuff 0;
   return 403;
   }

This will barf up a "403 Forbidden" message at the request, and also not bother to log it. Once you have these more complex rules out of the way, you can pretty much just use an online utility to convert your remaining Apache rewrites to Nginx rewrites. I made a conf file with global rewrites that I apply to all my virtual hosts and a non-wordpress set of rewrites that I apply to all of my sites that do not run Wordpress (this really cust down on the logs of WP attacks). Here are some examples from each, including a couple of if statements that take care of Wordpress related arguments that do not belong on non-WP sites:

global_rewrites.conf # Global ReWrite rules that apply to ALL virtual hosts
# Bad Wordpress plugins, themes and components
rewrite (?i)^/wp-content/plugins/1-flash-gallery/ http://$remote_addr/NOTICE-$remote_addr-IS-INFECTED-OR-HACKING.html last;
rewrite (?i)^/wp-content/plugins/advanced-custom-fields/ http://$remote_addr/NOTICE-$remote_addr-IS-INFECTED-OR-HACKING.html last;
rewrite (?i)^/wp-content/plugins/formcraft/ http://$remote_addr/NOTICE-$remote_addr-IS-INFECTED-OR-HACKING.html last;
rewrite (?i)^/wp-content/plugins/gravityforms/ http://$remote_addr/NOTICE-$remote_addr-IS-INFECTED-OR-HACKING.html last;
rewrite (?i)^/wp-content/plugins/revslider/ http://$remote_addr/NOTICE-$remote_addr-IS-INFECTED-OR-HACKING.html last;
# Common web server directories that get attacked
rewrite (?i)^/cgi-bin http://$remote_addr/NOTICE-$remote_addr-IS-INFECTED-OR-HACKING.html last;
rewrite (?i)^/cgi-sys http://$remote_addr/NOTICE-$remote_addr-IS-INFECTED-OR-HACKING.html last;
rewrite (?i)^/sys-cgi http://$remote_addr/NOTICE-$remote_addr-IS-INFECTED-OR-HACKING.html last;
# Directories I have seen being attacked in my logs
rewrite (?i)^/administrator/ http://$remote_addr/NOTICE-$remote_addr-IS-INFECTED-OR-HACKING.html last;
rewrite (?i)^/bitrix/ http://$remote_addr/NOTICE-$remote_addr-IS-INFECTED-OR-HACKING.html last;
rewrite (?i)^/blog/ http://$remote_addr/NOTICE-$remote_addr-IS-INFECTED-OR-HACKING.html last;
rewrite (?i)^/dana-na/ http://$remote_addr/NOTICE-$remote_addr-IS-INFECTED-OR-HACKING.html last;
rewrite (?i)^/install/ http://$remote_addr/NOTICE-$remote_addr-IS-INFECTED-OR-HACKING.html last;
rewrite (?i)^/joomla/ http://$remote_addr/NOTICE-$remote_addr-IS-INFECTED-OR-HACKING.html last;
rewrite (?i)^/myadmin/ http://$remote_addr/NOTICE-$remote_addr-IS-INFECTED-OR-HACKING.html last;
rewrite (?i)^/mySql/ http://$remote_addr/NOTICE-$remote_addr-IS-INFECTED-OR-HACKING.html last;
rewrite (?i)^/phpMyAdmin/ http://$remote_addr/NOTICE-$remote_addr-IS-INFECTED-OR-HACKING.html last;

non_wordpress_rewrites.conf # These rules should be applied to virtual hosts that do NOT run Wordpress
rewrite (?i)^/wp- http://$remote_addr/NOTICE-$remote_addr-IS-INFECTED-OR-HACKING.html last;
rewrite (?i)/wp-.+\.php$ http://$remote_addr/NOTICE-$remote_addr-IS-INFECTED-OR-HACKING.html last;
rewrite (?i)/wp-admin/ http://$remote_addr/NOTICE-$remote_addr-IS-INFECTED-OR-HACKING.html last;
rewrite (?i)/wp-includes/ http://$remote_addr/NOTICE-$remote_addr-IS-INFECTED-OR-HACKING.html last;
rewrite (?i)/wp-content/ http://$remote_addr/NOTICE-$remote_addr-IS-INFECTED-OR-HACKING.html last;
# There are no query strings for "posts" and "months" on non-WP sites
if ($args ~ "^p=" ){
   rewrite ^(/.*)$ http://$remote_addr/NOTICE-$remote_addr-IS-INFECTED-OR-HACKING.html last;
}
if ($args ~ "^m=" ){
   rewrite ^(/.*)$ http://$remote_addr/NOTICE-$remote_addr-IS-INFECTED-OR-HACKING.html last;
}

These are clearly not comprehensive, but they should offer an idea as to what you can do with these kinds of rewrites. Not everyone monitors their logs all day but if you at least browse through them now and then you'll notice some stuff that can be kicked out without too much effort.

Made with Notepad++ & FastStone, without javascript, cookies, or the help of Clippy or ai. Hosted on Devuan with nginx & powered by NK shrooms.