Anth's Computer Cave

AAIMI Home Automation archive page version 0.85

This article covers an older version of AAIMI Home Automation. It is here purely for historical purposes.

Use the buttons below to go to the current article series or view other archived articles.

New features in AAIMI Home Automation 0.85

23rd May 2017

The main AAIMI Home Automation web GUI. Picture: Anthony Hartup
The main AAIMI Home Automation web GUI. Click for larger image.

The latest version of AAIMI Home has GPS-based occupancy-awareness, a new data-save structure and a gallery for viewing security images. There is also a new outdoor movement sensor and some enhanced network-access-monitoring capabilities.

Gallery

You can now view all security images in your browser using AAIMI's Gallery module.

This module will soon handle all image storage and viewing. It creates thumbnails for all images and displays them in the left-hand column of the gallery, allowing you to scroll though your images to look for events.

This is a brand new feature so there are a few options missing, mainly the lack of an ongoing deletion system for older images. I hope to add that as a single-file, drop-in update in coming weeks.

Data

We've completely re-written the long-term data saving functions.

Before AAIMI would load the entire save-file to memory and keep it there. With 565 days of comprehensive home data on my main system this meant starting and stopping the system was taking too long, and the CPU was maxed out during save operations.

We've now broken up the save-file into two, with the base file holding all older data and a live file holding just the current month. Only the live file is loaded now. This has made saving much quicker, and dropped the RAM usage and CPU activity dramatically.

Other programs that use the data can automatically load both save files and append the live file data to the main file data. These programs are usually running on real PCs, which can load and parse the larger data file much quicker than the Raspberry Pi.

The other major long-term data change is the complete move to JSON. We'd already moved to JSON for most of the data systems, now the move is complete.

Converting from a Pickle-based save file to JSON format reduced my save-file by thirty-three percent from 33MB to 22MB. JSON is also a lot more compatible with a lot more programming languages and platforms, and being an almost-identical format to Python's dictionaries it is a natural fit for Python.

There are terminal-based tools included to convert your existing data file to the new JSON format, and to archive all but the current month's data to the base file. I'll cover those in the advanced article at the end of this series.

GPS

The readout from AAIMI GPS Mapper. Picture: Anthony Hartup

The AAIMI Project has a new stand-alone module called AAIMI GPS Mapper that creates and monitors perimeters. We've embedded this module into AAIMI Home Automation to determine where you and your housemates are. This augments AAIMI's existing IP based occupancy-awareness capabilities and adds the option to use an occupant's current location and distance from home to estimate the earliest possible time an occupant can arrive home.

AAIMI GPS Mapper is easy to use, and there are full setup and usage intructions in the AAIMI Mapper article series.

There is also a Pythone script that runs on Android phones and checks into the AAIMI server at intervals to provide its coordinates.

I've just started playing with Python on android, and it looks promising so far. I tried Python on Android a couple of years back and library support was hit and miss, but things have improved since then. I'm using the QPython interpreter, which supports a lot of the main Python libraries and can run 24/7 in the background.

It's great being able to run programs on Android using my go-to language, without having to build apps in Java with Android Studio. I've got more ideas to try out. The next function will be vibration alerts from AAIMI to your phone. I also plan to explore using Python to operate the sensors on dedicated AAIMI control devices to turn them into AAIMI satellites. Camera control is another likely area this could work.

Access monitoring

We're using the soon-to-be released second version of AAIM Server Dog for this release, and the Dog has learnt some new tricks.

The readout from AAIMI Server Dog. Picture: Anthony Hartup

AAIMI not only detects automated access attempts from bots, it now also attempts to identify the culprits.

It provides known domain details for each uninvited IP address that tries to access the AAIMI server. This is my favourite feature, because it's fascinating seeing the names of the various search engines, universities and other mysterious domains visiting your home in the middle of the night, instead of just a bland IP address.

After a month's operation there seems to be a domain associate with around half of the visiting IP addresses. The reverse DNS lookup comfirms about half of those as genuine, the others may be either badly configured or maliciously faked domains.

AAIMI now monitors TCP ports for open connections, and once again attempts to identify the remote connection.

There is also a blacklist of known-bad IP address AAIMI can download and compare to the visiting IPs.

Outdoor movement sensor

The PIR sensors we use with AAIMI work great inside, but place them outdoors and you can expect more than a few false alarms caused by cats, birds and other critters. We've improved this false alarm rate by creating a hybrid, PIR and distance based unit.

A combined PIR and distance sensor motion detector. Picture: Anthony Hartup

The PIR has an extra wide field of detection, and it will pick up side-on movement from four or five metres. The distance sensor has a much narrower field. Instead of triggering a movement event when the PIR senses motion, AAIMI starts a new thread and fires the distance sensor at half-second intervals while the PIR is HIGH. AAIMI triggers a movement event only if the distance sensor detects an object. In the case of a flying bird, it is less likely to be in front of the distance sensor at the exact time it fires.

Once the PIR goes LOW (generally two seconds without motion) without a distance-confirmation the distance sensor thread will stop and AAIMI will forget the whole thing ever happened.

The distance sensor's narrower field also means you can direct it to a certain height to allow cats and walking birds to pass underneath without triggering a false alarm.

AAIMI logs the number of PIR detections along with the number of distance-confirmed detections and the ratio is generall ten-to-one, meaning at least ninety-percent of the false alarms are negated.

Birds taking flight or landing directly in front of the sensor may still trigger a comfirmed event. For this reason I'm not using these as a full alarm zones, but as an audible alert, or chime zone. When someone walks past the sensor AAIMI pumps the speakers and plays a load voice declaring the system is armed. A would-be intruder can hear the voice from outside. Next I'll set it to switch one of the inside lights on to simulate a person.

I haven't built this in the main AAIMI Room Control file yet, it's currently in the AAIMI Sub Base file, where I generally trial new hardware to avoid affecting the main AAIMI program. It will be fully operational in the main program next release. if you want to try it out you'll need to configure and run aaimi_sub_base.py. I'll provide more details in the advanced tutorial. I'll also have a full tutorial on building the device.

New default settings

There were a few features that most people don't really need, and we've turned them off by default, although they are easily enabled if you wish.

The slideshow background is disabled, replaced by a plain background. That's less data for your phones to retreive, and fewer calls to the server. You can re-enable the slideshow background by choosing a theme in the Themes tab.

We've also disabled the incoming email system. With all the control options available in the browser, the email system is really only there for people who wish to use IP filtering and need to contact AAIMI to whitelist a new IP range when they travel.

For that limited purpose there was a 24/7 background program with frequent network activity, and it also meant dealing with a third-party in Gmail. Thus, we've turned it off.

You can re-enable incoming email by changing one variable in the main code and starting the AAIMI Email program alongside AAIMI Room Control. I'll provide more details in the advanced tutorial.

The out-bound email capabilities, needed for fire and intruder alerts, are still enabled. Unlike the incoming mail, the out-going system runs only when called, and uses no resources when dormat.

Article series

As always with a new release we have a full series of articles covering the setup and use of AAIMI Home Automation.

In this first article we've covered the new features, and in the next article we'll cover our slightly mad prototype hardware. In the third article we'll download and configure AAIMI Home and set up our home GPS perimeter with AAIMI GPS Mapper.

In article four we'll start AAIMI Room Control, the program to rule them all, while article five will cover controlling and using AAIMI Home with the web-based GUI. Article six covers the Python GUI used to visualize your long-term data, then article seven is the advanced article for everything else.

Cheers

Anth


_____________________________________________


Comments

Leave a comment on this article