IoT Open Basic

Basic course for IoT Open developers and integrators.

Chapter 3

After chapter 2 you should have created a function and got some data published on its topic_read topic. The data should also be visible in the standard user interface as well as in the workbench.

More about how data flows

In IoT Open data flows all the times. In our systems, thousands of messages fly every second. Every hour of the year. In this section we look deeper into how this data is managed and how to make it work for you.

Let's start by looking at this figure:

What we try to explain in this figure is that a lot if different data flies in the system. Every metric to or from a sensor is sent on the MQTT bus. On that bus, the data have a topic and a value. We will soon see that there is a little more to it, but for now topic and value is fine. We start by looking at the thermometer up to the right. At given interval it sends the temperature on a topic x/obj/temp. As we know from before that is likely a function with x/obj/temp as the value of topic_read. Every one that subscribes to this topic will get the data in near real time as soon as it comes from the sensor.

But as seen in the figure, the data also ends up in a database. Here the data is indexed by time and topic for later use.

On the right side of the image we have a mobile app and above it a light. The app sends a command to the light on x/set/obj/light and the light lights up and then sends a report back on x/obj/light these are probably topic_write and topic_read in the function for the light. Please observe that these also ends up in the database and can be retrieved later.

Now let's look at the data i more detail.

Data / Metrics

Data or sometimes referred to as metrics is really simple in IoT Open. If you want to look at it more in detail you can read on it in IoT Open MQTT API Specification but let's keep it simple. Data is a topic and a payload.

Topic

The topic tells us a lot of things, but we should not trust the topic a source of all truth. That is of course the job of functions as we have already seen.

This is how a topic is constructed:

As you can see, the topic is a string with different data separated by forward slashes to form different levels. No space or other strange characters may occur in the topic.

The first three levels are standardized and mandatory.

Let's look at each level.

Client id

The topic, when looking at it on MQTT, is prefixed with a number at the first level. Let's have a closer look at that first. That number is the client_id. Each installation have a client_id used for just this purpose. Please observe that the client_id and installation_id might be the same, but they don't have to be. The client object is an old object no longer used in the platform but for backward compatibility reasons it is still around even though it no longer has a meaning. But it gives us something to think about.

• When using the MQTT API you need to prefix the client_id to the topic of every message.
• When using the REST API you use the installation_id and never the client_id.

Like said before. They are often the same, but may differ.

Now look at both the standard user interface and also in the MQTT debug window of the workbench and locate your installation_id and your client_id.

Type

The second level is the type. It is defined as any of the following.

Subsystem

The third level is the subsystem. It may be anything, but you should choose something that is unlikely to be mixed up with anything else. For testing use local, test or something not used by anything else.

Subsystem specific

The rest of the topic is depending on the subsystem. You as an integrator might be your own subsystem and set your own rules. Since the functions are always the catalog to relevant data, it is not a problem. Here are some examples of real world topics:

34/obj/zwave/usb0/node/15/switch: {"value": 255, "timestamp": 1612243870.29}
34/obj/gateway_monitor/9684/icmp/max_rtt: {"value":0.681068,"timestamp":1620749372}
34/obj/local/daylight: {"timestamp":1620749377.785,"value":1}
34/obj/ikea/switch2: {"timestamp":1620749443.576,"value":0}
34/obj/lora/647fda00000012de/temperature: {"timestamp":1620749528,"value":26.70}
34/obj/zwave/usb0/node/17/voltage: {"meter_type": "electric_meter", "value": 235.447, "scale": "V", "timestamp": 1620749938.201}

Note that in the last one there are more information in the payload than specified. This is not a violation and works but the extra data except timestamp, value and msg is not handled by the platform nor saved in the database.

Now it is time to look deeper into the payload.

Payload

The payload is also quite simple. Depending on what type of message it is, it might look a bit different. All details are of course in the documentation, but here we explain it some more.

{client_id}/obj/{subsystem}/#

Format:

{
    "timestamp": unix timestamp, float,
    "value": float,
    "msg": "string"
}

Example:

34/obj/lora/647fda00000012de/temperature: {"timestamp":1620749528.12,"value":26.70}

This is the way the sensors update their data. In the example above something reports a value of 26.7. From the topic we can see that it is a LoRa sensor with EUI 647fda00000012de, but that part is subsystem dependent. This topic is also used for sending values if you are the data provider.

There is also a msg field that may contain a string. This is saved in the platform and can be used in several ways. We will discuss it more later.

Please note that there might be other objects in the message, but these will not be saved in the platform.

{client_id}/set/obj/{subsystem}/#

Note: This will be renamed {client_id}/set/{subsystem}/# in an upcoming release in order to better follow the format.

This topic is used to ask a subsystem to set an object to a new value. E.g. turning the light on, or set a new target temperature on a thermostat. If you are the subsystem, then it's work destined for you of course. Most subsystems uses the obj topic to send the new status back when the object has changed its status. By doing so, the signaling for statuses of object and commands to set status of objects are separated witch makes it easier to separate the roles.

{client_id}/evt/{subsystem}/#

These topics are used for sending events from a subsystems. E.g. you will get an event if any of the functions in an installation is changed. But it can (and was initially intended for) also be used for sending events from sensors. For instance if motion is detected. Note that there is good practice to also send obj to keep track of the status of the object.

{client_id}/cmd/{subsystem}/#

The cmd is for sending commands to the subsystem itself. It is used among other things to tell a subsystem to set itself in inclusion mode to accept a new device. But as always, it is you that decides how it works in your case and each subsystem may have its own commands.

Time to try it out

Start two browsers one with the standard user interface and one with the workbench. Change the name of your function on the standard user interface and watch the MQTT debug window in the workbench. You should see an event coming.

When you use the data fiddler you should also see a message with a topic that now makes sense to you.

This is about as much we can do with the platform without introducing some more tools. But don't worry, there is much more fun things to do and learn. Get ready to try Node-RED and Grafana in the next chapters as a way to illustrate the possibilities.