5 Jan 2009
tech
12 comments
Monitoring energy use with an Orb

I spoke at Homecamp recently about how an ambient orb could be used to monitor home energy usage. I’ve finally gotten around to putting some of it into practice so thought I would share some of the details of the setup as well as some more of my thoughts on the subject.

There are three key pieces of hardware in use. The Viglen MPC-L is the heart of the system. As I’ve mentioned previously, this is a low powered linux box running Ubuntu. The CurrentCost meter is connected to the MPC over USB-serial and my trusty arduino acts as an integration point for homebrew toys – including my ambient orb.

The MPC is running a Really Small Message Broker (RSMB). This is a small-footprint pub-sub message broker that talks MQTT. Each time the CurrentCost sends out a update, a piece of perl (“cc_pub.pl“) parses the data, sticks it into an RRDTool database for graphing and also publishes it to the home/cc/power and home/cc/temp topics.

Another piece of perl (“orbcontrol.pl“) is subscribed to the house/orb topic. When messages arrive on that topic, they are passed over serial to the arduino.

The sketch on the arduino currently listens on its serial port for commands that are then passed to the BlinkM in the orb. The format of the commands is identical to those in the BlinkMTester sketch that comes with the BlinkM. In the future this will do more as more things are attached to the arduino.

So far I have described how the orb is controlled and how the power data gets into the system. The next piece is how the two are plumbed together. Unsurprisingly, yet another perl script is running the on the MPC that provides the glue for this mixed metaphor.

orblogic.pl is subscribed to the home/cc/power topic so it receives all of the updates from the CurrentCost. It then makes a decision as to what colour the orb should be and then publishes the appropriate command to the house/orb topic.

Orb setup sketch

That is pretty much it – simple eh? Well, I did skip over the most interesting part – how to decide what colour the orb should be.

There are two key philosophies that come into play here:

  • Alert the unusual – have the ‘default’ state be the least obtrusive, such as ‘off’
  • Band the data – don’t react to single-point changes in the value. For example, when monitoring temperature, nothing is worse than constantly saying “its 19°C, its 18°C, its 19°C…”

These points should be applied whenever thinking about how to turn a raw stream of data into a useful stream of information.

I currently have a very simple piece of logic controlling the orb:

  • If power < 400, turn the orb off
  • If power between 400 and 1000, turn the orb orange
  • If power > 1000, turn the orb red

When we’re sat in the living room watching TV, we typically use under 400 watts, so the orb will only show anything if we’re over this value. It is very unusual to go over 1 kw unless we’re cooking, which is why that gets a stronger alert.

powergraph sketch

Despite its simplicity, this has already had the effect of making us more aware of when lights have been left on or when the kettle has finished boiling.

But as with all things, it could get much smarter. Having said we typically use under 400 watts watching TV, I also made the caveat ‘unless we’re cooking’. Given the system has a history of our power usage, it could feasibly determine suitable bands dynamically – so going over 1 kw between 6pm and 7pm isn’t necessarily a bad thing. Also, it chould only go red if the value goes over 1 kw for more than the time it takes to boil the kettle.

It all comes back to determining what is ‘unusual’.

12 comments
19 Nov 2008
code
2 comments
CurrentCostuino

With homecamp fast approaching I spent some time last weekend reviving some CurrentCost hacking I started in the summer.

One of the downsides of the data connection on the CurrentCost is the need for an always-on computer in order to capture the data. In practice, the existence of low powered machines like the MPC-L, or slug, make this a pretty inconsequential downside. That said, it was enough motivation to see what I could do with an Arduino and Ethernet Shield.

I have previously mentioned that part of the setup used to graph the power data online was to publish the data to a broker-in-the-sky using MQTT. To achieve this with an arduino it was clear that I needed one thing in particular – an arduino that is capable of connecting and publishing to an MQTT server.

With my ethernet shield in hand, I set about implementing enough of the MQTT spec to support basic publishing and subscribing. With that working, I looked into getting the arduino and CurrentCost talking. Thanks to Alexis, this was very straightforward, although I did rework the parsing of the data to take advantage of its somewhat-fixed length format.

Here’s the sketch I ended up with:

#include <Ethernet.h>
#include <PubSubClient.h>
#include <SoftwareSerial.h>

byte mac[] = {  0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };
byte ip[] = { 172, 16, 0, 50 };
byte server[] = { 172, 16, 0, 103 };

PubSubClient client(server, 1883,0);

#define rx 6
#define tx 7

SoftwareSerial softSerial =  SoftwareSerial(rx, tx);

void setup()  {
  pinMode(rx, INPUT);
  pinMode(tx, OUTPUT);
  softSerial.begin(9650);
  Ethernet.begin(mac, ip);
  delay(500);
}

void connect() {
  while(!client.connect("nolcc")) {
    delay(10000);
  }
  delay(200);
}

int offset = -1;
void loop() {
  if (!client.connected()) {
    connect();
    offset = -1;
  }
  client.loop();

  // scan the data for <msg>
  while(offset == -1) {
    char c = ' ';
    while (c!='<') {
      c = softSerial.read();
    }
    if (softSerial.read()=='m' &&
        softSerial.read()=='s' &&
        softSerial.read()=='g' &&
        softSerial.read()=='>') {
      offset = 5;
    }
  }

  // skip to the current value
  while(offset < 156) {
    offset++;
    softSerial.read();
  }
  char current[5];
  int value = 0;
  for (int i=0;i<5;i++) {
    current[i] = softSerial.read();
  }
  offset += 5;

  // skip to the temperature value
  while(offset < 243) {
    offset++;
    softSerial.read();
  }
  char temp[5];
  for (int i=0;i<4;i++) {
    temp[i] = softSerial.read();
  }
  offset += 4;

  client.publish("current",(uint8_t*)current,5);
  client.publish("temp",(uint8_t*)temp,4);

  // scan for </msg>
  while(offset != -1) {
    char c = ' ';
    while (c!='<') {
      c = softSerial.read();
    }
    if (softSerial.read()=='/' &&
        softSerial.read()=='m' &&
        softSerial.read()=='s' &&
        softSerial.read()=='g' &&
        softSerial.read()=='>') {
      offset = -1;
    }
  }
}

It publishes each power reading to a topic called ‘current’ and each temperature reading to a topic called ‘temp’. This isn’t quite the right configuration for the existing broker setup, but it wouldn’t take much to modify it appropriately.

You’ll note this sketch isn’t much use without the library that does the MQTT bit. Luckily, I have been given permission to release the library to the world. I should stress at this point, in case it wasn’t obvious from the fact I am releasing this code on my personal site, that my employer has nothing to do with it.

You’ll find the Client for MQTT library here.

If you want an MQTT server to play with, go buy Lotus Expeditor to get your hands on a microbroker <gratuitous-plug />, or check out the links on mqtt.org for the Really Small Message Broker.

2 comments
7 Jul 2008
events
4 comments
Open Tech 2008

Roo and I presented at Open Tech 2008 this weekend on the Current Cost hacking that has been going on in these parts. It was my first time presenting to quite such a large audience and all things considered, it was a lot of fun and went very well.

Although I took some notes through the day of the other talks I went to, Roo has done a much better write-up than I would. So I’ll point you his way.

4 comments
5 May 2008
tech
29 comments
Power Graphing

Since getting my meter online, it has been sending its readings to a server in the sky so charts like these can be produced. To really play with the data, I needed to start logging it locally and producing my own graphs. Here is a rambling rundown of how I got from XML being spat out by the device every 6 seconds to something like this:

Step One – Parsing the data

This was the easy part; AndySC had already put together a perl script for reading the serial port and doing the necessary parsing.

Step Two – Publishing the data

Again, not much for me to do here as Andy’s script already publishes the data to a set of topics over MQTT.

Step Three – Logging the data

Finally, something for me to do. A couple years ago, I would have joined Andy’s perl script with one of my own, but python is more my thing these days. I already had a piece of python that subscribes to the appropriate topics and posts the temperature values to twitter. It didn’t take much to get the same script to subscribe to the power data and dump it into a MySQL database on the local machine.

+----------------+---------+
| when           | power |
+----------------+---------+
| 20080502231747 |    0.34 |
| 20080502231753 |    0.33 |
| 20080502231804 |    0.34 |
+----------------+---------+

The table in the database is a very simple one at the moment; logging the power along with a timestamp. An entry is added to the table for every reading from the meter and having been running for 3 days there are just over 11,000 of them. I’ll have to keep an eye on this to make sure it doesn’t run away with my free disk space.

Step Four – Graphing the data

Roo showed me some stuff he’s been playing with using the Google Chart API. Whilst I generally prefer to roll my own (aka, reinvent the wheel), I couldn’t ignore just how easy it is to produce pretty graphs this way.

Before delving into the API, I needed to decide just what I wanted to produce. As there is such a range of chart types available, there are plenty of interesting things that could be done. Initially, however, I decided to stick with the traditional “power-usage-in-the-last-24-hours” chart.

The API has a limit on the amount of data that can be passed to it. So I needed to find a meaningful way to reduce the 4200 data points generated in 24 hours to around 100 at most. I soon settled on using the average value for each 10 minute period. This loses some resolution in the data, but it still shows the trends.

Generating the averages is a simple question of the right query on the database. With some trial and error, I eventually got to:

select concat(substring(substring(`when`,1,11),9,4),'0'), truncate(avg(`power`),3) from currentcost where `when` > SUBDATE(NOW(), INTERVAL 1 DAY) group by substring(`when`,1,11) order by `when`;

Simple huh?

This produces results like this:

+------+-------+
| 2240 | 0.714 |
| 2250 | 0.637 |
| 2300 | 0.406 |
+------+-------+

With that in place, all it took was to throw it together into an appropriate URL for google to generate the image:

http://chart.apis.google.com/chart?chg=100,20,1,7&chxt=y,x&
chxl=0:|0|1|2|3|4|5| 1:|2100|2300|0100|0300|0500|0700|0900|1100|1300|1500|1700|1900& chxp=1,2,11,19,27,36,44,53,61,69,78,86,95&chs=400×200 &cht=lc&chds=0,5&chm=B,f3f3f3,0,0,0&chco=aaaaff& chd=t:0.348,0.348,0.319,0.311,1.285,0.683,0.338,0.349, 0.398,0.39,0.356,0.477,0.274,0.264,0.572,0.637,0.406, 0.382,0.368,0.324,0.325,0.917,0.322,0.358,0.342,0.318, 0.191,0.124,0.209,0.204,0.192,0.213,0.205,0.196,0.374, 1.753,1.537,0.976,0.552,0.556,0.513,0.433,0.363,0.341, 0.575,0.46,0.188,0.123,0.248,0.204,0.195,0.116,0.215, 0.207,0.226,1.574,0.636,0.214,0.206,0.209,0.204,0.648, 0.694,0.628,0.599,0.689,0.743,1.222,0.278,0.394,0.326, 0.394,0.225,0.184,0.13,0.145,0.23,0.226,0.222,0.133, 0.133,0.237,0.231,0.223,0.13,0.234,0.229,0.222,0.203, 0.197,0.193,0.17,0.198,0.176,0.275,0.328,0.287,0.212, 0.159,0.192,0.19,0.212,0.292,0.298,0.418,0.332,0.789,0.894,0.719

Again, simple huh? Well, maybe not so much. Here’s a break down of what that does (and just to confuse matters, this is in a different order to where things appear in the url above…)

http://chart.apis.google.com/chart?

This is base url of the Google Charts api – all the magic comes from here.

chs=400×200

Set the size of the image.

&cht=lc

Set the type of chart – a line chart.

&chg=100,20,1,7

Gives the chart a grid in the background.

&chxt=y,x

&chxl=0:|0|1|2|3|4|5| 1:|2100|2300|0100|0300|0500|0700|0900|1100|1300|1500|1700|1900

&chxp=1,2,11,19,27,36,44,53,61,69,78,86,95

Describes the axis labels. 0 to 5 on the y-axis, and the relevant times along the x-axis. The script figures out approximate positions along the axis for the labels.

&chds=0,5

Specifies the minimum and maximum values for the data – although I occasionally go over 5Kw, I decided it wasn’t worth squeezing the data for 99% of the time when it is below that.

&chm=B,f3f3f3,0,0,0

Fills the area under the line with a light gray.

&chco=aaaaff

Draws the line with a light blue.

&chd=t:0.348,0.348,0.319,0.311,1.285,0.683,0.338,0.349, 0.398,0.39,0.356,0.477,0.274,0.264,0.572,0.637,0.406, 0.382,0.368,0.324,0.325,0.917,0.322,0.358,0.342,0.318, 0.191,0.124,0.209,0.204,0.192,0.213,0.205,0.196,0.374, 1.753,1.537,0.976,0.552,0.556,0.513,0.433,0.363,0.341, 0.575,0.46,0.188,0.123,0.248,0.204,0.195,0.116,0.215, 0.207,0.226,1.574,0.636,0.214,0.206,0.209,0.204,0.648, 0.694,0.628,0.599,0.689,0.743,1.222,0.278,0.394,0.326, 0.394,0.225,0.184,0.13,0.145,0.23,0.226,0.222,0.133, 0.133,0.237,0.231,0.223,0.13,0.234,0.229,0.222,0.203, 0.197,0.193,0.17,0.198,0.176,0.275,0.328,0.287,0.212, 0.159,0.192,0.19,0.212,0.292,0.298,0.418,0.332,0.789,0.894,0.719

Specifies the data points.

What next?

  • Chris has started doing some interesting data analysis to see if he can automatically spot ‘events’ on the graph. Will be interesting to see what can be achieved here.
  • Andy’s twittering house got some linkage last week from both Earth2Tech and Wired Science. They mention the power orb that was written about last year. I really like the idea of an ambient device for displaying this information – another project for the arduino list.
29 comments
10 Apr 2008
life
tech
12 comments
Going power crazy

As a part of the “CurrentCost craze” that is rife at Hursley these days, I’ve had mine up and running for a few days and am very excited by the possibilities.

The CurrentCost meter gives you a real-time display of how much electricity your house is using. On its own, this would be interesting, but hard to relate to anything. One of its cool features is its awareness of how much electricity actually costs – being told your usage costs £17 per month is much more effective than saying you’re using 1.23kw.

I’m not the only one who has got this meter setup and then obsessively gone around the house to reduce how much is being wasted.

What is great about this particular device, and what appeals to the Hursley crowd, is the ability to plug it into a PC and capture the information it produces. It doesn’t take much plumbing to get the data being graphed online – something which does raise an interesting issue.

I can now see when Jo has got home from work as I see the spike of the kettle being used, but equally, anyone can get a pretty good idea if we’re in the house. Is that a genuine concern? There are certainly easier ways of working out if we’re in or not.

That aside, the graph it produces definitely does give food for thought. Here’s a snapshot of the graph from today (click through for a fully annotated version on flickr):
Power Usage

This graph has left me intrigued to find out what the 100watt-ish blip is that happens every 2 hours; there is a PC running all the time, which has the meter connected – I wonder what cron tasks are configured.

The meter has definitely left me feeling slightly guilty for the 600watts of halogen spot-lights we put in the new kitchen.

12 comments