8 Jun 2012

Cosm, formerly known as Pachube, has had MQTT support for a while now and I’ve been meaning to do something with it.

One of the strengths of a publish/subscribe system, such as MQTT, is the decoupling of the producers and consumers of messages. I don’t have to modify the code that reads my energy usage from the CurrentCost meter each time I think of a new use of the data – I just add a new, self-contained, subscriber.

So to get my energy usage data into a datastream on Cosm, I could simply write a client that subscribes to the house/cc/power topic on my home broker and republishes it to the appropriate topic on the Cosm broker.

But even that’s far more work than is really needed – why write code when the broker can do it for me? RSMB lets you create bridge connections to other brokers and define how topics should be mapped between them. Now, I’ve done this plenty of times, but I still get confused with the topic-mapping syntax RSMB uses in its config file. So, for future reference, here’s how to do it.

connection cosm
try_private false
address 216.52.233.121 1883
username <COSM_API_KEY>
clientid knolleary
topic "" out house/cc/power /v2/feeds/62907/datastreams/0.csv

The topic line shows how you can achieve a one-to-one mapping between different topics – the non-obvious part being the need to have the empty “” as the topicString.

If you are using Mosquitto, an additional parameter is needed in the topic line to specify the QoS to use:

topic "" out 0 house/cc/power /v2/feeds/62907/datastreams/0.csv

Going the other way, subscribing to a feed on Cosm, is just as easy:

topic "" in inbound/cosm /v1/feeds/XXXXXX/datastreams/0.csv

Or, if you are using Mosquitto:

topic "" in 0 inbound/cosm /v1/feeds/XXXXXX/datastreams/0.csv

Note the use of /v1/ in the remote topic string this time around. As documented, this gets you just the raw data value, rather than the comma-separated time-stamp and data value returned by the /v2/ api. Of course, depending on what you’re doing, you may want the time-stamp. Equally, you may want the full JSON or <shuddder> XML formats – in which case you would change the .csv to .json or .xml respectively.

In a future post, I’ll look more at how the topic-mapping in RSMB (and Mosquitto) can be used to good effect in scaling an MQTT system.

  1. Roger • June 9, 2012

    Thanks for the remininder that this feature exists in rsmb, I’d forgotten about it. I solved the same problem by writing a client (mqtt2pachube/cosm) to do the topic mapping much like you say. Topic mapping doesn’t exist in mosquitto yet, but is now on the list for 0.16.

  2. Richard Appleby • June 9, 2012

    Hi Nick – what a coincidence – I was trying to get this working myself today. I’ve never played with COSM (Pachtube as was) before, and as a part of tidying up my RSMB implementation on my home server, I wanted to try publishing a temperature feed from my broker to COSM (before using it to integrate “twitter alerts” into my RSMB)

    I followed your instructions, and now have a bridged connection to my COSM feed working nicely. Thanks for the information – fathoming out that mapping would have been a pain otherwise.

    Interestingly there seemed to be a few minutes lag before the data started showing up in the feed (did you experience that too?) but it all seems to be working fine now.

    Cheers
    Richard

  3. nickJune 9, 2012

    Roger – as I wrote this post at midnight, it didn’t occur to me until after I published to double check if Mosquitto supported that syntax. Good to hear its on the todo list – there’s a lot of interesting tricks to be done with it.

    Richard – glad to hear it worked for you. It’s possible Cosm are showing cached data rather than live; I have seen a couple comments to that end, but I don’t know if it’s still the case.

  4. Richard Appleby • June 9, 2012

    Nick – no problem, I can live with cached data on Cosm, certainly at this point.

    Having got this working, one thing that occurred to me is that it doesn’t seem to be possible to arrange a LWT for a bridge connection. I know I’m a bit rusty on MQTT, but is that right, or have I forgotton something? It’s a bit of a shame if not, as it would be a very useful feature.

  5. nickJune 10, 2012

    Richard – you can use the notifications setting to achieve what you want. When enabled, it will publish (on both local and remote brokers) a ‘1’ when the bridge connects and a ‘0’ when the bridge loses its connection. You cannot modify the payload, but you can change the topic:

    notifications true
    notification_topic /v2/feeds/00000/datastreams/0.csv

  6. Richard Appleby • June 25, 2012

    Are you still able to get this to work?

    I had forwarding of a single topic (very similar to yours) from RSMB to Cosm working beautifully for a time, and then moved on to look into notifications, which I never managed to get to work at all. However, I now see that my basic forwarding seemed to stop working at 14:03:16 on the 10th, so all my problems with notifications may be grounded in a common root issue, possibly at the Cosm end.

    I’ve since had a pretty serious go at getting that topic forwarded again, with absolutely no success at all. Normal MQTT clients however, still seem to work OK.

    Thoughts?

  7. nickJune 26, 2012

    Hi Richard – yes, it’s still working fine for me. Odd that it wouldn’t stop working like that for you; sure you didn’t tinker with some other setting whilst playing with notifications?

  8. Richard Appleby • June 26, 2012

    Ok, I’ve found it. At some point I replaced “216.52.233.121” with “api.cosm.com”, and it seems that RSMB cannot resolve api.cosm.com to 216.52.233.121 in my environment – despite api.cosm.com being pingable from the command line. Maybe something to do with dnsmasq (which I’m using for name resolution internally in my network) being a DNS forwarder rather than a proper resolver?

    But this is the first thing I’ve found that doesn’t work, so I’ll chase it up with IanC anyway.

  9. Tony • June 28, 2012

    Hi Nick,
    How do you get the data from the CurrentCost device (i’m assuming you are using the bridge?) or are you hooked into it via USB/Serial?

    Tony

  10. pingback from Internet of things « Richard's BlogJune 28, 2012

  11. nickJune 28, 2012

    Hi Tony. I started out with using the serial connection via an Arduino. More recently, I’ve been using a CurrentCost Bridge running a custom MQTT firmware.

  12. leave a comment

    You must be logged in to post a comment.