3 Sep 2016

In a previous post, I wrote about how I ended up at IBM. I finished that post with a question: Will I still be here in another 10 years? Well, I’m half-way to finding out, but I’m no closer to an answer.

The last 5 years in the Emerging Technologies group have flown by. The last three years of which have been pretty much dominated by my work on Node-RED. It’s been a lot of hard work, but it has been incredibly satisfying work. I’m very fortunate to be in this position; not everyone gets to turn a side-project they created into a job.

That said, as I spoke about in my Monkigras talk this year, when a side-project becomes a job, it risks becoming a chore. I have to continually make sure I strike a balance between the fun and the not-so-fun parts. I don’t always get that balance right which can make it harder work than it should be.

So what of the next 5 years? No doubt Node-RED will continue to be my focus for now, but that doesn’t stop me wondering what else I could be doing.

4 Dec 2014

Earlier this year, Tom Coates wrote a blog post about his session at this year’s O’Reilly Foo Camp. Over tea with colleagues, we talked about some of the ideas from the post and how some of our research work might be interesting when applied to them.

One thing led to another and I found myself talking about it at ThingMonk this year. What follows is a slightly expanded version of my talk.


ciot-0

ciot-2

Humanising Things

We have a traditional of putting human faces on things. Whether it’s literally seeing faces on the Things in our everyday lives, such as the drunk octopus spoiling for a fight, or possibly the most scary drain pipe ever.

Equally, we have a tendency to put a human persona onto things. The advent of Twitter brought an onslaught of Things coming online. It seemingly isn’t possible for me to do a talk without at least a fleeting mention of Andy Standford-Clark’s twittering ferries; where regular updates are provided for where each ferry is.

ciot-5

One of the earliest Things on Twitter was Tower Bridge. Tom Armitage, who was working near to the bridge at the time, wrote some code that grabbed the schedule for the bridge opening and closing times, and created the account to relay that information.

ciot-6

One key difference between the ferries and the bridge is that the ferries are just relaying information, a timestamp and a position, whereas the bridge is speaking to us in the first-person. This small difference immediately begins to bring a more human side to the account.
But ultimately, they are simple accounts that relay their state with whomever is following them.

This sort of thing seems to have caught on particularly with the various space agencies. We no longer appear able to send a robot to Mars, or land a probe on a comet without an accompanying twitter account bringing character to the events.

ciot-7

There’s always a sense of excitement when these inanimate objects start to have a conversation with one another. The conversations between the philae lander and its orbiter were particularly touching as they waved goodbye to one another. Imagine, the lander, which was launched into space years before Twitter existed, chose to use its last few milliamps of power to send a final goodbye.

ciot-8

But of course as soon as you peek behind the curtain, you see someone running Tweetdeck, logged in and typing away. I was watching the live stream as the ESA team were nervously awaiting to hear from philae. And I noticed the guy in the foreground, not focused on the instrumentation as his colleagues were, but rather concentrating on his phone. Was he the main behind the curtain, preparing Philae’s first tweet from the surface? Probably not, but for the purposes of this talk, let’s pretend he was.

ciot-9

The idea of giving Things a human personality isn’t a new idea. There is a wealth of rigorous scientific research in this area.

One esteemed academic, Douglas Adams, tells us about the work done by the The Sirius Cybernetics Corporation, who invented a concept called Genuine People Personalities (“GPP”) which imbue their products with intelligence and emotion.

He writes:

Thus not only do doors open and close, but they thank their users for using them, or sigh with the satisfaction of a job well done. Other examples of Sirius Cybernetics Corporation’s record with sentient technology include an armada of neurotic elevators, hyperactive ships’ computers and perhaps most famously of all, Marvin the Paranoid Android. Marvin is a prototype for the GPP feature, and his depression and “terrible pain in all the diodes down his left side” are due to unresolved flaws in his programming.

In a related field, we have the Talkie Toaster created by Crapola, Inc and seen aboard Red Dwarf. The novelty kitchen appliance was, on top of being defective, only designed to provide light conversation at breakfast time, and as such it was totally single-minded and tried to steer every conversation to the subject of toast.

ciot-13

Seam[less|ful]ness

In this era of the Internet of Things, we talk about a future where our homes and workplaces are full of connected devices, sharing their data, making decisions, collaborating to make our lives ‘better’.

Whilst there are people who celebrate this invisible ubiquity and utility of computing, the reality is going to much more messy.

Mark Weiser, Chief Scientist at Xerox PARC, coined the term “ubiquitous computing” in 1988.

Ubiquitous computing names the third wave in computing, just now beginning. First were mainframes, each shared by lots of people. Now we are in the personal computing era, person and machine staring uneasily at each other across the desktop. Next comes ubiquitous computing, or the age of calm technology, when technology recedes into the background of our lives.

Discussion of Ubiquitous Computing often celebrated the idea of seamless experiences between the various devices occupying our lives. But in reality, Mark Weiser advocated for the opposite; that seamlessness was undesirable and self-defeating attribute of such a system.

He preferred a vision of “Seamfulness, with beautiful seams”

ciot-15

The desire to present a single view of the system, with no joins, is an unrealistic aspiration in the face of the cold realities of wifi connectivity, battery life, system reliability and whether the Cloud is currently turned on.

Presenting a user with a completely monolithic system gives them no opportunity to connect with and begin to understand the constituent parts. That is not it say this information is needed to all users all of the time. But there is clearly utility to some users some of the time.

When you come home from work and the house is cold, what went wrong? Did the thermostat in the living room break and decide it was the right temperature already? Did the message from the working thermostat fail to get to the boiler? Is the boiler broken? Did you forgot to cancel the entry in your calendar saying you’d be late home that day?

Without some appreciation of the moving parts in a system, how can a user feel any ownership or empowerment when something goes wrong with it. Or worse yet, how can they avoid feeling anything other than intimidated by this monolithic system that simply says “I’m Sorry Dave, I’m afraid I can’t do that”.

Tom Armitage wrote up his talk from Web Directions South and published it earlier this week, just as I was writing this talk. He covers a lot of what I’m talking about here so much more eloquently than I am – go read it. One piece his post pointed me at that I hadn’t seen was Techcrunch’s recent review of August’s Smart Lock.

ciot-16

Tom picked out some choice quotes from the review which I’ll share here:

“…much of the utility of the lock was negated by the fact that I have roommates and not all of them were willing or able to download the app to test it out with me […] My dream of using Auto-Unlock was stymied basically because my roommates are luddites.”

“Every now and then it didn’t recognize my phone as I approached the door.”

“There was also one late night when a stranger opened the door and walked into the house when August should have auto-locked the door.”

This is the reason for having beautiful seams; seams help you understand the edges of a devices sphere of interaction, but should not be so big to trip you up. Many similar issues exists with IP connected light bulbs. When I need to remember which app to launch on my phone depending on which room I’m walking into, and which bulbs happen to be in there, the seams have gotten too big.

In a recent blog post, Tom Coates wrote about the idea of a chatroom for the house – go read it.

Much like a conference might have a chatroom, so might a home. And it might be a space that you could duck into as you pleased to see what was going on. By turning the responses into human language you could make the actions of the objects less inscrutable and difficult to understand.

ciot-17

This echoes back to the world of Twitter accounts for Things. But rather than them being one-sided conversations presenting raw data in a more consumable form, or Wizard-of-Oz style man-behind-the-curtain accounts, a chatroom is a space where the conversation can flow both ways; both between the owner and their devices, but also between the devices themselves.

What might it take to turn such a chatroom into a reality?

ciot-18

Getting Things Talking

Getting Things connected is no easy task.

We’re still in the early days of the protocol wars.

Whilst I have to declare allegiance to the now international OASIS standard MQTT, I’m certainly not someone who thinks one protocol will rule them all. It pains me whenever I see people make those sorts of claims. But that’s a talk for a different day.

Whatever your protocol of choice, there are an emerging core set that seem to be the more commonly talked about. Each with its strengths and weaknesses. Each with its backers and detractors.

ciot-19

What (mostly) everyone agrees on is the need for more than just efficient protocols for the Things to communicate by. A protocol is like a telephone line. It’s great that you and I have agreed on the same standards so when I dial this number, you answer. But what do we say to each other once we’re connected? A common protocol does not mean I understand what you’re trying to say to me.

And thus began the IoT meta-model war.

There certainly a lot of interesting work being done in this area.

For example, HyperCat, a consortium of companies coming out of a Technology Strategy Board funded Demonstrator project in the last year or so.

ciot-21

HyperCat is an open, lightweight JSON-based hypermedia catalogue format for exposing collections of URIs. Each HyperCat catalogue may expose any number of URIs, each with any number of RDF-like triple statements about it. HyperCat is simple to work with and allows developers to publish linked-data descriptions of resources.

URIs are great. The web is made of them and they are well understood. At least, they are well understood by machines. What we’re lacking is the human view of this world. How can this well-formed, neatly indented JSON be meaningful or helpful to the user who is trying to understand what is happening.

This is by no means a criticism of HyperCat, or any of the other efforts to create models of the IoT. They are simply trying to solve a different set of problems to the ones I’m talking about today.

ciot-23

Talking to Computers

We live in an age where the talking to computers is becoming less the reserve of science fiction.

Siri, OK Google, Cortana all exist as ways to interact with the devices in your pocket. My four year old son walks up to me when I have my phone out and says: “OK Google, show me a picture of the Octonauts” and takes over my phone without even having to touch it. To him, as to me, voice control is still a novelty. But I wonder what his 6 month old sister will find to be the intuitive way of interacting with devices in a few years time.

The challenge of Natural Language Parsing, NLP, is one of the big challenges in Computer Science. Correctly identifying the words being spoken is relatively well solved. But understanding what those words mean, what intent they try to convey, is still a hard thing to do.

To answer the question “Which bat is your favourite?” without any context is hard to do. Are we talking to a sportsman with their proud collection of cricket bats? Is it the zoo keeper with their colony of winged animals. Or perhaps a comic book fan being asked to chose between George Clooney and Val Kilmer.

Context is also key when you want to hold a conversation. The English language is riddled with ambiguity. Our brains are constantly filling in gaps, making theories and assertions over what the other person is saying. The spoken word also presents its own challenges over the written word.

ciot-28

“Hu was the premiere of China until 2012″

When said aloud, you don’t know if I’ve asked you a question or stated a fact. When written down, it is much clearer.

ciot-29

In their emerging technology report for 2014, Gartner put the Internet of Things at the peak of inflated expectation. But if you look closely at the curve, up at the peak, right next to IoT, is NLP Question Answering. If this was a different talk, I’d tell you all about how IBM Watson is solving those challenges. But this isn’t that talk.

ciot-30

A Conversational Internet of Things

To side step a lot of the challenges of NLP, one area of research we’re involved with is that of Controlled Natural Language and in particular, Controlled English.

CE is designed to be readable by a native English speaker whilst representing information in a structured and unambiguous form. It is structured by following a simple but fully defined syntax, which may be parsed by a computer system.

It is unambiguous by using only words that are defined as part of a conceptual model.

CE serves as a language that is both understandable by human and computer system – which allows them to communicate.

For example,

there is a thermometer named t1 that is located in the room r1

A simple sentence that establishes the fact that a thermometer exists in a given room.

the thermometer t1 can measure the environment variable temperature

Each agent in the system builds its own model of the world that can be used to define concepts such thermometer, temperature, room and so on. As the model is itself defined in CE, the agents build their models through conversing in CE.

there is a radiator valve v1 that is located in the room r1
the radiator valve v1 can control the environment variable temperature

It is also able to using reasoning to determine new facts.

the room r1 has the environment variable temperature that can be measured and that can be controlled

As part of some research work with Cardiff University, we’ve been looking at how CE can be extended to a conversational style of interaction.

These range from exchanging facts between devices – the tell

the environment variable temperature in room r1 has value "21"

Being able to ask question – ask-tell

for which D1 is it true that
      ( the device D1 is located in room V1 ) and
      ( the device D1 can measure the environment variable temperature ) and
      ( the value V1 == "r1")

Expanding on and explaining why certain facts are believed to be true:

the room r1 has the environment variable temperature that can be measured and that can be controlled
    because
the thermometer named t1 is located in the room r1 and can measure the environment variable temperature
    and
the radiator valve v1 is located in the room r1 and can control the environment variable temperature

The fact that the devices communicate in CE means the user can passively observe the interactions. But whilst CE is human readable, it isn’t necessarily human writeable. So some of the research is also looking at how to bridge from NL to CE using a confirm interaction:

NL: The thermometer in the living room has moved to the dining room
CE: the thermometer t1 is located in the room r2

Whilst the current research work is focused on scenarios for civic agencies – for example managing information exchange in a policing context, I’m interested in applying this work to the IoT domain.

With these pieces, you can begin to see how you could have an interaction like this:

    User: I will be late home tonight.
    House: the house will have a state of occupied at 1900
    User: confirmed
    House: the room r1 has a temperature with minimum allowable value 20 after time 1900
           the roomba, vc1, has a clean cycle scheduled for time 1800

Of course this is still quite dry and formal. It would be much more human, more engaging, if the devices came with their own genuine people personality. Or at least, the appearance of one.

    User: I will be late home tonight.
    House: Sorry to hear that, shall I tell everyone to expect you back by 7?
    User: yes please    
    Thermometer: I'll make sure its warm when you get home
    Roomba: *grumble*

I always picture the Roomba as being a morose, reticent creature who really hates its own existence. We have one in the kitchen next to our lab at work, set to clean at 2am. If we leave the door to the lab open, it heads in and, without fail, maroons itself on a set of bar stools we have with a sloped base. Some might call that a fault in its programming, much like Marvin, but I like to think its just trying to find a way to end it all.

This is all some way from having a fully interactive chat room for your devices. But the building blocks are there and I’ll be exploring them some more.

12 Nov 2014

Last month, I was fortunate enough to fly off to Austin with a group of colleagues for a week long IBM Design Thinking camp. It was an opportunity to get away from the day job, with laptops all-but banned, and have a deep-dive into what IBM Design is about and how it can be applied.

As a relatively new effort within the company, IBM Design sets out to bring a focus back to where it should be; the human-experience of our products and services. This isn’t just about making pretty user interfaces; it is the entire experience of our products.

As an engineer, the temptation is always there to create shiny new features. But no matter how shiny it is, if it isn’t what a user needs, then it’s a waste of effort. The focus has to be on what the user wants to be able to do. This is something I’ve always tried to do with Node-RED; we often get suggestions for features that, once you start picking at them, are really solutions looking for a problem. Once you work back and identify the problem, we’re often able to identify alternative solutions that are even better.

P1070048

It’s often just a matter of asking the right question; At Designcamp, the very first exercise we were asked to do was to draw a new type of vase. Everyone drew something that looked vaguely vase-like. Then (spoilers…) we were asked to draw a better way to display flowers. At this point we got lots of decidedly un-vase-like ideas that were much more imaginative. It’s the difference between asking for a feature and asking for an idea. The former presupposes a lot about the nature of the answer, the latter is focused on not just the what, but also the why.

This relentless focus on the user isn’t a new idea. GDS, who are doing incredible things with government services, have it as their very first Design Principle. But it is refreshing to see this focus being brought to bear within a transformation of how the entire company operates.

Oh, and of course being in Austin, we got to screen print our own IBM Designcamp T-Shirts to commemorate the visit.

Go to Designcamp, screen print your own t-shirt. Obvs.

Lots more photos from the week over on flickr.

18 Apr 2014

Having recently added a Travis build for the Node-RED repository, we get a nice green, or sometimes nasty red, merge button on pull-requests, along with a comment from Travis thanks to GitHub’s Commit Status API.

From a process point of view, the other thing we require before merging a pull-request is to ensure the submitter has completed a Contributor License Agreement, or CLA. This has been a manual check against a list we maintain. But with GitHub’s recent addition of the Combined Status api I decided we ought to automate this check.

And of course, why not implement it in a Node-RED flow.

There are a few simple steps to take:

  1. know there has been a pull-request submitted, or that we want to manually trigger a check
  2. check the submitter against the list
  3. update the status accordingly

Rather than poll the api to see when a new pull-request has arrived, GitHub allows you to register a Webhook to get an http POST when certain events occur. For this to work, we need an HTTP-In node to accept the request. As with any request, we need to make sure it is responded to properly, as well as keeping a log – handy for debugging as you go.

nr-gh-flow-1

With that in place, we can register the webhook on the repositories we want to run the checks on.

  1. From the repository page, find your way to the webhook page via Settings ➝ WebHooks & Services ➝ Add webhook
  2. Enter the url of the http-in node
  3. Select ‘Let me select individual events.’ and then pick, as a minimum, ‘Pull Request’ and ‘Issue comment’
  4. Ensure the ‘Active’ option is ticked and add the webhook

When the POST arrives from GitHub, the X-GitHub-Event http header identifies the type of event that triggered the hook. As we’ve got a couple different types registered for the hook we can use a Switch node to route the flow depending on what happened. We also need a function node to access the http header as the Switch node can’t do by itself.

// GitHub Event Type
msg.eventType = msg.req.get('X-GitHub-Event');
return msg;

nr-gh-flow-2

Skipping ahead a bit, to update the status of the commit requires an http POST to the GitHub API. This is easily achieved with an HTTP Request node, but it does need to be configured to point to the right pull-request. Pulling that information out of the ‘pull-request’ webhook event is easy as it’s all in there.

// PR Opened
msg.login = msg.payload.pull_request.user.login;
msg.commitSha = msg.payload.pull_request.head.sha;
msg.repos = msg.payload.pull_request.base.repo.full_name;
return msg;

It’s a bit more involved with the ‘issue-comment’ event which fires for all comments made in the repository. We only want to process comments against pull-requests that contain the magic text to trigger a CLA check to run.

Even after identifying the comments we’re interested in, there’s more work to be done as they don’t contain the information needed to update the status. Before we can do that, we must first make another http request to the GitHub API to get the pull-request details.

// Filter PR Comments
msg.repos = msg.payload.repository.full_name;
if (msg.payload.issue.pull_request &&
       msg.payload.comment.body.match(/node-red-gitbot/i)) {
    msg.url = "https://api.github.com:443/repos/"+msg.repos+
              "/pulls/"+msg.payload.issue.number+
              "?access_token=XXX";
    msg.headers = {
        "user-agent": "node-red",
	    "accept": "application/vnd.github.v3"
    };
    return msg;
}
return null;

Taking the result of the request, we can extract the information needed.

// Extract PR details
var payload = JSON.parse(msg.payload);
msg.login = payload.user.login;
msg.commitSha = payload.head.sha;
msg.repos = payload.base.repo.full_name;
return msg;

nr-gh-flow-4

Now, regardless of which event has been triggered we have everything we need to check the CLA status; if msg.login is in the list of users who have completed a CLA, send a ‘success’ status, otherwise send a ‘failure’.

// Update CLA Status
var approved = ['list','of','users'];

var login = msg.login;
var commitSha = msg.commitSha;
var repos = msg.repos;

msg.headers = {
	"user-agent": "node-red",
	"accept": "application/vnd.github.she-hulk-preview+json"
};

msg.url = "https://api.github.com:443/repos/"+repos+
          "/statuses/"+commitSha+
          "?access_token=XXX";

msg.payload = {
    state:"failure",
    description:"No CLA on record",
    context:"node-red-gitbot/cla",
    target_url:"https://github.com/node-red/node-red/blob/master/CONTRIBUTING.md#contributor-license-aggreement"
}

if (approved.indexOf(login) > -1) {
    msg.payload.state = "success";
    msg.payload.description = "CLA check passed";
}
return msg;

nr-gh-flow-3

And that’s all it takes.

One of the things I like about Node-RED is the way it forces you to break a problem down into discrete logical steps. It isn’t suitable for everything, but when you want to produce event-drive logic that interacts with on-line services it makes life easy.

You want to be alerted that a pull-request has been raised by someone without CLA? Drag on a twitter node and you’re there.

There are some improvements to be made.

Currently the node has the list of users who have completed the CLA hard-coded in – this is okay for now, but will need to be pulled out into a separate lookup in the future.

It also assumes the PR only contains commits by the user who raised it – if a team has collaborated on the PR, this won’t spot that. Something I’ll have to keep an eye on and refine the flow if it becomes a real problem.

Some of the flow can already be simplified since we added template support for the HTTP Request node’s URL property.

The final hiccup with this whole idea is the way GitHub currently present commit statuses on the pull-request page. They only show the most recent status update, not the combined status. If the CLA check updates first with a failure, then the Travis build passes a couple minutes later, the PR will go green thanks to the Travis success. The only way to spot the Combined Status is currently ‘failure’ is to dive into the API. I’ve written a little script to give me a summary of all PR’s against the repositories, which will do for now.

Given the Combined Status API was only added as a preview last month, hopefully once it gets added to the stable API they will update how the status is presented to use it.

A final note, I’ve chosen not to share the full flow json here – hopefully there’s enough above to let you recreate it, but if you really want it, ping me.

26 Jan 2014

I’m a big fan of the Ghost blogging platform. Having heard Hannah Wolfe talk about it a couple of times, I was keen to try it out. In fact I purposefully held off setting up a blog for Node-RED until Ghost was available.

It was painless to set up and get running on my Web Faction hosted server – even though at that point Web Faction hadn’t created their one-click installer for it.

Ghost released their second stable version a couple weeks ago, which added, amongst many other new features, the ability for a post’s url to contain its date. This was a feature I hadn’t realised I was missing until I wrote a post on the blog called ‘Community News’. The intent was this would be a commonly used blog title as we post things we find people doing with Node-RED. However, without a date in the post, the urls would end up being .../community-news-1, .../community-new-2 .. etc. Perfectly functional, but not to my taste.

So I clicked to enable dated permalinks on the blog, and promptly found I had broken all of the existing post urls. I had half-hoped that on enabling the option, any already-published posts would not be affected – but that wasn’t the case; they all changed.

To fix this, I needed a way to redirect anyone going to one of the old urls to the corresponding new one. I don’t have access to the proxy running in front of my web host, so that wasn’t an option.

Instead, I added a bit of code to my installation of Ghost that deals with it.

In the top level directory of Ghost is a file called index.js. This is what gets invoked to run the platform and is itself a very simple file:

var ghost = require('./core');

ghost();

What this doesn’t reveal is that the call to ghost() accepts an argument of an instance of express, the web application framework that Ghost uses. This allows you to pass in an instance of Express that you’ve tweaked for your own needs – such as one that knows about the old urls and what they should redirect to:

var ghost = require('./core');
var express = require("express");

var app = express();

ghost(app);

var redirects = express();

var urlMap = [
   {from:'/version-0-2-0-released/', to:'/2013/10/16/version-0-2-0-released/'},
   {from:'/internet-of-things-messaging-hangout/', to:'/2013/10/21/internet-of-things-messaging-hangout/'},
   {from:'/version-0-3-0-released/', to:'/2013/10/31/version-0-3-0-released/'},
   {from:'/version-0-4-0-released/', to:'/2013/11/14/version-0-4-0-released/'},
   {from:'/version-0-5-0-released/', to:'/2013/12/21/version-0-5-0-released/'},
   {from:'/community-news-14012014/', to:'/2014/01/22/community-news/'}
];

for (var i=0; i<urlMap.length; i+=1) {
   var to = urlMap[i].to;
   redirects.all(urlMap[i].from,function(req,res) {
      res.redirect(301,to);
   });
}

app.use(redirects);

It’s all a bit hard-coded and I don’t know how compatible this will be with future releases of Ghost, but at least it keeps the old urls alive. After all, cool uris don’t change.