Anth's Computer Cave

AAIMI Server Dog 0.1 (archived)

This article covers an older version of AAIMI Server Dog. It is here for historical purposes.

Click here to see the current version

15th Feb 2017

Today I'll officially introduce to the AAIMI Project's latest module.

AAIMI Server Dog is a Python server-monitoring script for htaccess-protected Apache2 web servers running private web sites and applications.

It produces HTML files with lists of suspicious access attempts. You then embed the logs for viewing in a webpage or web application.

We've embedded it into the latest versions of AAIMI Home Automation and it works great in that, and now it's time to release a stand-alone version.

Have a look at the image below, which shows an actual readout from AAIMI Server Dog running on my AAIMI Home Automation system. These are the creatures that come knocking on your router at night if you run a web-facing server.

The readout from AAIMI Server Dog showing unauthorized access attempts on an Apache web server. Picture: Anthony Hartup.
The readout from AAIMI Server Dog showing unauthorized access attempts on an Apache web server. Picture: Anthony Hartup.

All the above attempts happened over a two-day period, and this is on a dynamic IP address with no domain. With a static IP and a domain you would see many more entries every day.

Before you start worrying, these bots aren't getting through into the server. It has htaccess enabled and only genuine users can access anything. All the bad attempts above simply bounce off. It's still good to know who's knocking, though, so I made AAIMI Server Dog


How it works

AAIMI Server Dog reads two pairs of Apache2 log files, the access.log files and the error.log files. These logs are anything-but human-friendly, but AAIMI can read and sort the logs in seconds.

First it determines the details for your server then it reads the access.log files and catalogs general access issues. It writes this to a HTML file listed by names attempted, then IP address.

Next AAIMI checks the error.log files, and gathers more data on the main offenders.

The readout from AAIMI Server Dog showing unauthorized access attempts on an Apache web server. Picture: Anthony Hartup.

You can see the time and duration of the attempts, whether they were flagged because of name or password and all the names used. The names generally seem to be default usernames for common routers, camera, web applications and content-management systems like Wordpress. There's a good lesson from this: Change the default username and passwords on your devices!

It also displays the actual target directories the bots tried to access. You can see above the attempts to get into an sqlite database, most likely using default configuration.


Limitations

The program lacks a bit of direction at the moment, because I built it on the fly without any real objectives except to see who was knocking. It is not ready to use in a mission-critical envirnment.

I need to create more criteria for detection, and tie the results together in a more-useful, searchable database.

AAIMI Server Dog in this version also only monitors access attempts. This can help you to spot a concerted attacks and allow you to take action.

The next version will be able to actively block these bots on the fly. We already have it running in the dev-version, we're just streamlining a few things and putting some safeguards in-place to avoid false-positives.


Download and configure the Dog

You can download the AAIMI Server Dog setup folder here. Extract the folder on your server and open the aaimi_server_dog.py file for editing.

First we'll add our IP details. Ignore the public_ip variable(AAIMI will detect that) and go to the my_ips list on line 18.

The python list for excluding IP addresses from AAIMI monitoring. Picture: Anthony Hartup.

The localhost partial IP address is already in the list to exclude internal connections from monitoring, as is the IPV6 locahost IP. You need to add the first two sections of your LAN IP address, or the network IP the server is on, to the end of the list. For instance, if your LAN IP address for your server is 192.168.0.12, enter "192,168". You'll also need to add the first two sections of the IP your phones and other devices will connect with remotely. This excludes these IP ranges from the main monitoring functions.

We're using partial IP addresses instead of full addresses because your WAN devices like phones, etc, will receive a new IP regularly. In most cases the first two sections will remain the same and only the last two will change, meaning you shouldn't need to update the IP list each time. This means others with that IP range are filtered from results as well, but IP addresses are just one of the factors in the calculations and the other factors should still trigger. You can enter full IPs if you are really locking things down, but using partial IPs is a compromise between ease of use and the level of detection. In a moment I'll also cover how you can completely turn off detection-by-IP.

Next we go to line 31 and add the server's authorized users to the allowed_names list in quotes, separated by commas

The python list for excluding usernames from AAIMI monitoring. Picture: Anthony Hartup.

Finally, go to line 82 and note the mode for the read_apache_log() function is "standard". This is the default and checks access attempts by all criteria. That means bad IPs, bad usernames, bad passwords, and CONNECT attempts. If you want to refine the mode to filter just by username or passwords, change mode to "user". This may be necessary if you have many devices connecting to your server and wish to disable filtering results by IP. If you wish to filter soley by IP, change mode to "ip_range". This could help if you have many users, but most of them connect over the same LAN.


Release the Hound

That's all you need to do to take the Dog for a quick run from the terminal. SSH into your server and navigate to the aaimi_server_dog folder. Type:

sudo aaimi_server_dog.py and press Enter

The access list from AAIMI monitoring. Picture: Anthony Hartup.

Now open the access_file.html file with your browser and you should see the same list.

The access list from AAIMI monitoring. Picture: Anthony Hartup.

Open the badlogins.html file with your browser and you should see the main bots.

The access list from AAIMI monitoring. Picture: Anthony Hartup.

It looks pretty dull like this, but when you load the list into your web html via PHP or Javascript it will use your websites CSS styles to smarten it up.


Refreshing the list

There are several ways to automatically or manually refresh the list. You could also run a cron job on the server that calls the Dog at intervals, or import it into a loop of a background Python program running as root. The AAIMI Home Automation GUI has a button in the Logs column to send a PHP call to the main Python automation program, which then calls AAIMI Server Dog.

I'll go into more details about embedding AAIMI Server Dog in you applications in the next article, coming soon.

_____________________________________________


Comments

Leave a comment on this article