R223 release notes

Firmware release 223 introduces some bug fixes and a series of new features. The most significant new feature is support for the B-version of the FLM02. We've now got a unified firmware for both the FLM02A as well as the FLM02B. Hence we do not have to worry about back porting code changes from one firmware branch to the other. Below is a list of new features in r223:

  • Improved robustness of the sensor board in case of a power loss, meaning less false positives will be triggered.
  • A supervisor daemon that continuously monitors the correctness of the local system time setting and the health of the wifi link. The supervisor takes corrective action where needed.
  • The local, real-time HTTP/REST sensor interface is now enabled by default. This should reduce the reporting of dashboard minute tab errors.
  • A Mosquitto MQTT broker on each FLM. Sensor readings are injected into the broker and can be read out by any MQTT client. Point the client to the FLM's IP address and select port 1883. Gauge (Watt) and counter (Wh) readings are published on /sensor/xyz/gauge and /sensor/xyz/counter topics respectively.
  • A new local user interface for configuring the FLM. The UI should be more user friendly, providing detailed FLM status feedback on the status page. When settings are being saved to the FLM, a progress bar and text area will provide details on the actions being carried out.

Specific FLM02B-related features:

  • Support for toggling the port type of ports 2 and 3 between analog (clamp) and pulse.
  • A driver for the RFM12B 868MHz radio that's present on the new v2.2 sensor board. The radio driver packet format is compatible with the Jeenode format, opening up a world of possibilities to extend the Fluksometer with a range of wireless sensors like temperature, humidity, movement detection. Because the full coupling between wireless sensors and Flukso back-end hasn't yet been completed, this feature is still considered experimental.
  • A Dutch P1-port telegram parser. It converts the P1 port data to standard Flukso sensor readings. Hence, there's no difference in the dash or the API when accessing a sensor whose time-series data was generated by a current clamp, a DIN-rail kWh meter or a local smart meter port. The sensor associated with the smart meter will count backwards when power is injected into the grid.

Known issues:

  • Real-time readings are not correct for pulse inputs with meter constants that are is not an integer fraction, see this forum post. So constants like 0.5, 0.25, 0.2, 0.1 will work correctly with the real-time interface, while 0.625 and 0.4 will cause bogus values.

Everyone with a Fluksometer serial in the FL02xxxxxx or FL03xxxxxx range can now upgrade his FLM via a simple click on the settings (cogwheels in the top right corner) > devices page. The actual upgrade will take place on the next hourly FLM heartbeat. Make sure to keep your FLM humming during a scheduled upgrade.

Server Maintenance

The Flukso server is scheduled to receive both hardware and software maintenance between 22h and 24h CET today. While some server downtime is unavoidable, we will try to make this service interruption as short as possible.

Valentine

It's always nice when your shipper delivers a much-awaited package on Valentine's day. Mine contained the first production run of the version 2.2 sensor boards. Initial testing is looking good with more probing planned for next week.

Flukso HQ is moving

Flukso HQ will be moving February 1st. You can find the new address in the contact info. Hence no orders will be shipped from now till the start of next week. I'm swallowing the red pill and hoping to be back in Wonderland on Monday!

A Mosquitto on the Fluksometer

While the current Fluksometer daemon implementation has proven to be very robust, it was written for a unidirectional data flow. Readings from the sensor board are sourced at the SPI interface, timestamped, processed and sunk in a buffer. This buffer is periodically sent out in an HTTPS call to the Flukso server.

To make the Fluksometer software architecture future-proof, it should provide asynchronous bi-directional communication between internal daemon components as well as external clients. The architecture should allow multiple consumers of incoming data and be modular, extensible and fault-tolerant. We can meet both the async two-way communication and the multiple consumers requirements by introducing a message broker as a central component. Since this broker needs to run on an embedded machine like the Fluksometer, it should be lightweight as well. An attractive candidate is Roger Light's Mosquitto, an open source message broker based on the MQ Telemetry Transport protocol.

A big plus of using Mosquitto is that it comes with libmosquitto, a C client library for talking to an MQTT broker. I wrote a set of Lua bindings to libmosquitto. The bindings allow us to include the MQTT TCP connection into an external event loop, minimizing message latency of the system, compared to an polling-based implementation.

I then wrote a broker performance test based on the lua-mosquitto bindings. The test sets up a process ring: N processes are created by the test and are chained by the broker. M messages will be injected into the ring with a TTL (time-to-live). Once the message has passed TTL times through the broker it will report back to the master process. When all messages are accounted for by the master process, the broker's message throughput will be calculated. Here's an example of a ring test running on my laptop against a Mosquitto broker on the FLM.

  1. icarus75@cirrus:~/dev/lua-mosquitto/test/ring (master)$ ./ring.lua 192.168.255.1 100 1000 1000
  2. Sending 1000 messages with ttl 1000 into a ring of 100 nodes
  3. Elapsed time: 2054.324 sec
  4. Throughput: 486 msg/sec

1000 messages flow in and out the MQTT broker 1000 times with 100 nodes on the laptop acting as MQTT clients.

Out of curiosity, I ran the test against a Mosquitto server on the laptop. So instead of a 180MHz MIPS 4KEc we've now got a 2GHz Sandy Bridge i7-2630QM hosting Mosquitto. The core count doesn't matter that much as the broker is single-threaded.

  1. icarus75@cirrus:~/dev/lua-mosquitto/test/ring (master)$ ./ring.lua localhost 100 1000 1000
  2. Sending 1000 messages with ttl 1000 into a ring of 100 nodes
  3. Elapsed time: 19.336 sec
  4. Throughput: 51717 msg/sec

So while the FLM took 2000+ seconds to process the 1 million messages, the Intel Core i7 gets the job done in less than 20 seconds, churning through 50k+ messages per second. That's a two orders of magnitude difference. While the 500 msg/sec is still more than adequate for the tasks at hand on the FLM, it's a good reminder that the FLM still is an embedded system.