Mid-cycle upgrade for AAIMI Home Automation
15th March, 2016
This article covers an older version of AAIMI. Click here to see the current article series.
We've updated a lot of the AAIMI Home Automation code this week to version 0.55. I'm calling this a "minor upgrade" because there are no user-facing changes, but it's not actually minor, and it's not really an upgrade.
It is a full rewrite, with major changes to the program structure, and it requires a clean install.
Version 0.5 arrived less than a month ago, and if you are already using that you could skip this update and wait for version 0.6. That will contain new user-facing features.
If you are just starting out with the AAIMI system, you definitely want version 0.55.
AAIMI's various features have been created over several years, and each feature utilized whatever methods I was using at the time. These methods have change a lot in that time as I have learnt new skills, and by version 0.5 the variety of ways AAIMI controlled different features was making the code more complicated than necessary.
I've been putting this off for a while because bringing all these features together required a full rewrite. I decided after version 0.5 that I should do it now, before I start on the new features for version 0.6.
I am happy to announce that the work is done, and we now have AAIMI Home Automation 0.55.
The most drastic modification was switching to JSON for almost all communication between program modules.
Before that, every part of the browser-based GUI used a different communication method. Most were text-based, with different PHP files parsing data in different ways. The Python-based GUI used a separate method again, with an entire SQlite database to hold just one set of live stats. The user-interrupt system was another SQlite database requiring different parsing methods depending on which GUI you used. None of these methods were easily human-readable, which made debugging and trouble-shooting difficult.
Discovering the JSON library was one of those wow moments, where you wish you had found something years ago. It is the perfect way to share data between a Python back end and a web page.
Everything sent to either of the GUIs now is encoded in JSON format, and on the browser-based GUI it is displayed using JQuery rather than PHP. All incoming user-requests and interrupts now also use JSON, and while they still use PHP, I've streamlined the five incoming PHP files into two.
This has farewelled six PHP files and two databases, and it has tidied up the web folder in the process.
Here is the old program layout:
And below is the new layout:
As you can see, it's a lot tidier.
With no databases I was able to cut the Python module previously used for database management. Dumping the live database has also reduced the workload on AAIMI. Before it would make frequent (sometimes every few seconds) updates to the database for the Python GUI, then update that same data to another file for the web server. Now it only has to update one JSON file, and both GUIs use that for live display.
The main Python file, aaimi_room_control.py, is now around 1600 lines of code, and once again this has been created over a long period. By version 0.5 I was spending way too much time scrolling around searching for functions, etc, that were placed in no particular order. This has changed.
I started with an empty page and reinserted all of the code in clearly defined sections. For example, all switching functions are in the switching section, all outgoing functions are in the outgoing communication sections and all incoming functions are in the incoming section. I now know exactly where everything is, and making modifications is a breeze.
This was the first of two major code tweaks for AAIMI.
Next on the agenda is the long-term data system, with particular focus on the saving mechanism.
I have seven months' home data saved now, and while the file is only around 10MB, there are millions of entries and it takes about 45 seconds to do a full save. I plan to create a separate permanent save-file that AAIMI appends to every 24 hours rather than holding, and saving, all the data all the time.
I plan to add more types of data as well, and improve the way AAIMI handles some existing data.
I want to store things like the number of lamp-hours and appliance-hours for each room.
I also want to build a lamp-compensation system for the light-levels data. At present you can choose to either keep monitoring and storing as normal when the lamps are on, or not monitor and store light data at all when lamps are on. The former can skew the long-term light data, the latter results in fewer readings. To solve this I will have AAIMI study the differences between lamp-on and lamp-off readings at various times of the day and calculate the average variation. It can then adjust the light-readings accordingly before adding them to the data. This knowledge will also help AAIMI to decide when to turn lamps off as rooms brighten.
I'm not sure if I'll do this as another mid-cycle upgrade, or wait and build it into version 0.6 instead.