Industruino MQTT over SSL

Ethernet with TLS using the SSLClient library

Tom

These days many applications require secure communication with a cloud server, using HTTPS or other protocols based on TLS. Traditionally, Arduino style microcontrollers have lacked the necessary computing power to deal with encryption, but the SSLClient library works with the D21G MCU used in Industruinos. This means we can add SSL functionality to  Industruino ETHERNET applications. We continue to use the Ethernet2 library as before, and use the SSLClient library as an extra layer dealing with TLS. (During our further testing it appeared that the Ethernet2 library does no use large enough buffers, causing the sketch to crash. This can be fixed by using the SSLClient author's version EthernetLarge with a minor modification to change the SPI speed to 4MHz as described here).

You can easily test the setup with the library example EthernetHTTPS, just remember to change 'Serial' to 'SerialUSB'. This will connect to arduino.cc over HTTPS port 443, and show you the Arduino logo in ASCII in the Serial Monitor. It also reports the time it takes to establish the SSL connection: 3.7 seconds in my case.

In the below example, we will be using the MQTT protocol with TLS, on the standard port 8883. We use the Mosquitto test server/broker at https://test.mosquitto.org/ MQTT allows different levels of encryption; we will go for the standard server side only, on port 8883. To implement the MQTT client, we use the popular pubsubclient library, which does not need to be configured for TLS, we can just pass the SSLClient as the client.

MQTT

We will connect to the server, and subscribe to the topic "to_industruino". This will allow us to send messages from our laptop/broker/platform to the Industruino.

We will publish messages to the MQTT server with topic "from_industruino", and they will contain a JSON with key "uptime" and value in seconds.

SSLCLIENT

In order to connect to the MQTT server using TLS, the Industruino needs to have the server's x509 certificate. It is available for download on https://test.mosquitto.org/ in PEM format (this is a self-signed certificate valid until 2030). We need to convert this to the format used by the SSLClient library: TRUST ANCHORS (TA). We can use a python script supplied by the library in the tools folder, to generate the TA file we have to add to the sketch. If you want to use this sketch with a different server, you will have to generate the file for your server. If you just want to test the sketch, you can just copy/paste the mosquitto_org.h file below, no need to generate it.

$ python pycert_bearssl.py convert ~/Downloads/mosquitto.org.crt --no-search --output mosquitto_org.h

This file needs to be placed in the sketch folder, so that it can be included as a tab in the Arduino IDE. Uploading will compile these files into 1 binary, including the server's certificate.

TEST

Below screenshot shows how to send data to the Industruino by publishing to the "to_industruino" topic.

$ mosquitto_pub -h test.mosquitto.org -p 8883 -d -t "to_industruino" -m "{ssl_test:2}" --cafile mosquitto.org.crt

And we can receive data from the Industruino by subscribing to the "from_industruino" topic.

$ mosquitto_sub -h test.mosquitto.org -p 8883 -d -t "from_industruino" --cafile mosquitto.org.crt

In both cases it is necessary to specify the server certificate file downloaded earlier from the server website.

The -d argument enables verbose debugging, useful for testing. Removing it makes the output cleaner.

Of course this server can only be used for testing purposes.

When using a production server, it is possible to use its regular CA certificates, but the certificate vailidity is currently limited to 1 year. Because we have to hard code the certificate in the Industruino sketch, it would have to be updated after (or better just before!) its expiry. Therefore, it makes more sense to use a self-signed certificate, which can have a much longer validity (10+ years).

This is the main sketch file (.ino)

It also contains 2 functions to extract the unique MAC address from the Industruino's EEPROM chip.

/*
   Industruino MQTT over SSL with ETHERNET

   to test MQTT server over SSL: test.mosquitto.org
   download the certifacte from download link on https://test.mosquitto.org/
   $ mosquitto_pub -h test.mosquitto.org -p 8883 -d -t "to_industruino" -m "{ssl_test:2}" --cafile mosquitto.org.crt

   SSL CERTIFICATE
   use additional library for SSL: https://github.com/OPEnSLab-OSU/SSLClient
   use utility to convert .pem to TA trusted anchor file
   $ python pycert_bearssl.py convert ~/Downloads/mosquitto.org.crt --no-search --output mosquitto_org.h
   copy this file into the folder of this sketch

   Tom Tobback, Feb 2021

 *******************************************************************************************************************************************************
    MIT LICENSE
    Copyright (c) 2021 Cassiopeia Ltd
    Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"),
    to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense,
    and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
    The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
    FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
    LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
    IN THE SOFTWARE.
 *******************************************************************************************************************************************************

*/

const char* mqtt_server = "test.mosquitto.org";          // test server
const int mqtt_port = 8883;                           // with SSL

// ETHERNET ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
#include <Wire.h>                               // for RTC
#include <SPI.h>
#include <Ethernet2.h>        // Industruino version of Ethernet2
EthernetClient client;
byte mac[6];                                    // read from RTC
#include <SSLClient.h>        // needed for SSL
#include "mosquitto_org.h"    // server certificate in Trust Anchor format
SSLClient eth_client(client, TAs, (size_t)TAs_NUM, A13);   // analog pin for random generator
// +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

// MQTT
#include <PubSubClient.h>
PubSubClient mqtt_client(eth_client);
String out_topic = "from_industruino";
String in_topic = "to_industruino";
unsigned long posting_ts = 0;

// LCD display
#include <UC1701.h>
static UC1701 lcd;

///////////////////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////////////////////////////////////////////////

void setup() {

  pinMode(26, OUTPUT);     // backlight on Industruino LCD
  digitalWrite(26, HIGH);
  lcd.begin();
  lcd.clear();
  lcd.setCursor(0, 0);
  lcd.print("Industruino ETH_SSL");

  SerialUSB.begin(115200);
  delay(2000);
  SerialUSB.println();
  SerialUSB.println();
  SerialUSB.println("=================================");
  SerialUSB.println("Industruino MQTT SSL WIFI test");
  SerialUSB.println("=================================");

  // ETHERNET ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  Wire.begin();
  SerialUSB.println("Get unique MAC from RTC EEPROM");
  readMACfromRTC();             // mac stored in rtc eeprom
  Ethernet.begin(mac);
  SerialUSB.print("Ethernet started with IP: ");
  SerialUSB.println(Ethernet.localIP());
  lcd.setCursor(0, 1);
  lcd.print(Ethernet.localIP());
  // +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

  // MQTT server
  mqtt_client.setServer(mqtt_server, mqtt_port);
  mqtt_client.setCallback(mqttCallback);
  lcd.setCursor(0, 2);
  lcd.print(mqtt_server);
  lcd.print(":");
  lcd.print(mqtt_port);
  connectMQTT();

}

///////////////////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////////////////////////////////////////////////

void loop() {
  mqtt_client.loop();

  // regular data posting
  if (millis() - posting_ts > 10000) {
    sendData();
    posting_ts = millis();
  }
}

///////////////////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////////////////////////////////////////////////

void connectMQTT() {
  SerialUSB.print("[MQTT] attempting connection to server: ");
  SerialUSB.print(mqtt_server);
  SerialUSB.print(":");
  SerialUSB.print(mqtt_port);
  SerialUSB.print(" >>> ");
  if (mqtt_client.connect("INDUSTRUINO")) {
    SerialUSB.println("MQTT connected");
    lcd.setCursor(0, 3);
    lcd.print("connected");
    // SUBSCRIBE to in_topic
    if (mqtt_client.subscribe(in_topic.c_str())) {
      SerialUSB.println("[MQTT] subscribed to topic: " + in_topic);
    }
  } else {
    SerialUSB.print("MQTT failed, rc=");
    int mqtt_status = mqtt_client.state();
    SerialUSB.print(mqtt_status);
    if (mqtt_status == -2) SerialUSB.println(" connect fail");
    if (mqtt_status == -4) SerialUSB.println(" connect timeout");
    lcd.setCursor(0, 3);
    lcd.print("connect failed");
  }
}

//////////////////////////////////////////////////////////////////////////////////

void mqttCallback(char* topic, byte* payload, unsigned int length) {
  String payload_str = "";
  SerialUSB.print("[MQTT] message arrived [");
  SerialUSB.print(topic);
  SerialUSB.print("] " + payload_str);
  for (int i = 0; i < length; i++) {
    SerialUSB.print((char)payload[i]);
    payload_str += (char)payload[i];
  }
  lcd.setCursor(0, 6);
  lcd.print("received:");
  lcd.setCursor(0, 7);
  lcd.print(payload_str);
  SerialUSB.println();
}

///////////////////////////////////////////////////////////////////////////////////////

boolean sendData() {

  if (!mqtt_client.connected()) {
    SerialUSB.println("[MQTT] not connected, cannot publish data");
    return false; // if no mqtt, failed
  } else {
    String payload = "{uptime:";
    payload += String(millis() / 1000);
    payload += "}";
    lcd.setCursor(0, 4);
    lcd.print("sending:");
    lcd.setCursor(0, 5);
    lcd.print(payload);

    if (mqtt_client.publish(out_topic.c_str(), payload.c_str(), 0)) {
      SerialUSB.println("[MQTT] publish topic:" + out_topic + " message:" + payload);
    } else {
      SerialUSB.println("[MQTT] publish topic:" + out_topic + " failed!");
      return false;
    }
  }
  return true;
}

/////////////////////////////////////////////////////////////////////////
// the RTC has a MAC address stored in EEPROM - 8 bytes 0xf0 to 0xf7
void readMACfromRTC() {
  SerialUSB.println("READING MAC from RTC EEPROM");
  int mac_index = 0;
  for (int i = 0; i < 8; i++) {   // read 8 bytes of 64-bit MAC address, 3 bytes valid OUI, 5 bytes unique EI
    byte m = readByte(0x57, 0xf0 + i);
    SerialUSB.print(m, HEX);
    if (i < 7) SerialUSB.print(":");
    if (i != 3 && i != 4) {       // for 6-bytes MAC, skip first 2 bytes of EI
      mac[mac_index] = m;
      mac_index++;
    }
  }
  SerialUSB.println();
  SerialUSB.print("Extracted 6-byte MAC address: ");
  for (int u = 0; u < 6; u++) {
    SerialUSB.print(mac[u], HEX);
    if (u < 5) SerialUSB.print(":");
  }
  SerialUSB.println();
}

/////////////////////////////////////////////////////////////////////////
// the RTC has a MAC address stored in EEPROM
uint8_t readByte(uint8_t i2cAddr, uint8_t dataAddr) {
  Wire.beginTransmission(i2cAddr);
  Wire.write(dataAddr);
  Wire.endTransmission(false); // don't send stop
  Wire.requestFrom(i2cAddr, 1);
  return Wire.read();
}

This is the mosquitto_org.h file to be put in the sketch folder.

#ifndef _MOSQUITTO_ORG_H_
#define _MOSQUITTO_ORG_H_

#ifdef __cplusplus
extern "C"
{
#endif

/* This file is auto-generated by the pycert_bearssl tool.  Do not change it manually.
 * Certificates are BearSSL br_x509_trust_anchor format.  Included certs:
 *
 * Index:    0
 * Label:    mosquitto.org
 * Subject:  1.2.840.113549.1.9.1=roger@atchoo.org,CN=mosquitto.org,OU=CA,O=Mosquitto,L=Derby,ST=United Kingdom,C=GB
 */

#define TAs_NUM 1

static const unsigned char TA_DN0[] = {
    0x30, 0x81, 0x90, 0x31, 0x0b, 0x30, 0x09, 0x06, 0x03, 0x55, 0x04, 0x06,
    0x13, 0x02, 0x47, 0x42, 0x31, 0x17, 0x30, 0x15, 0x06, 0x03, 0x55, 0x04,
    0x08, 0x0c, 0x0e, 0x55, 0x6e, 0x69, 0x74, 0x65, 0x64, 0x20, 0x4b, 0x69,
    0x6e, 0x67, 0x64, 0x6f, 0x6d, 0x31, 0x0e, 0x30, 0x0c, 0x06, 0x03, 0x55,
    0x04, 0x07, 0x0c, 0x05, 0x44, 0x65, 0x72, 0x62, 0x79, 0x31, 0x12, 0x30,
    0x10, 0x06, 0x03, 0x55, 0x04, 0x0a, 0x0c, 0x09, 0x4d, 0x6f, 0x73, 0x71,
    0x75, 0x69, 0x74, 0x74, 0x6f, 0x31, 0x0b, 0x30, 0x09, 0x06, 0x03, 0x55,
    0x04, 0x0b, 0x0c, 0x02, 0x43, 0x41, 0x31, 0x16, 0x30, 0x14, 0x06, 0x03,
    0x55, 0x04, 0x03, 0x0c, 0x0d, 0x6d, 0x6f, 0x73, 0x71, 0x75, 0x69, 0x74,
    0x74, 0x6f, 0x2e, 0x6f, 0x72, 0x67, 0x31, 0x1f, 0x30, 0x1d, 0x06, 0x09,
    0x2a, 0x86, 0x48, 0x86, 0xf7, 0x0d, 0x01, 0x09, 0x01, 0x16, 0x10, 0x72,
    0x6f, 0x67, 0x65, 0x72, 0x40, 0x61, 0x74, 0x63, 0x68, 0x6f, 0x6f, 0x2e,
    0x6f, 0x72, 0x67,
};

static const unsigned char TA_RSA_N0[] = {
    0xc1, 0x34, 0x1c, 0xa9, 0x88, 0xcd, 0xf4, 0xce, 0xc2, 0x42, 0x8b, 0x4f,
    0x74, 0xc7, 0x1d, 0xef, 0x8e, 0x6d, 0xd8, 0xb3, 0x6a, 0x63, 0xe0, 0x51,
    0x99, 0x83, 0xeb, 0x84, 0xdf, 0xdf, 0x32, 0x5d, 0x35, 0xe6, 0x06, 0x62,
    0x7e, 0x02, 0x11, 0x76, 0xf2, 0x3f, 0xa7, 0xf2, 0xde, 0xd5, 0x9c, 0xf1,
    0x2d, 0x9b, 0xa1, 0x6e, 0x9d, 0xce, 0xb1, 0xfc, 0x49, 0xd1, 0x5f, 0xf6,
    0xea, 0x37, 0xdb, 0x41, 0x89, 0x03, 0xd0, 0x7b, 0x53, 0x51, 0x56, 0x4d,
    0xed, 0xf1, 0x75, 0xaf, 0xcb, 0x9b, 0x72, 0x45, 0x7d, 0xa1, 0xe3, 0x91,
    0x6c, 0x3b, 0x8c, 0x1c, 0x1c, 0x6a, 0xe4, 0x19, 0x8e, 0x91, 0x88, 0x34,
    0x76, 0xa9, 0x1d, 0x19, 0x69, 0x88, 0x26, 0x6c, 0xaa, 0xe0, 0x2d, 0x84,
    0xe8, 0x31, 0x5b, 0xd4, 0xa0, 0x0e, 0x06, 0x25, 0x1b, 0x31, 0x00, 0xb3,
    0x4e, 0xa9, 0x90, 0x41, 0x62, 0x33, 0x0f, 0xaa, 0x0d, 0xf2, 0xe8, 0xfe,
    0xcc, 0x45, 0x28, 0x1e, 0xaf, 0x42, 0x51, 0x5e, 0x90, 0xc7, 0x82, 0xca,
    0x68, 0xcb, 0x09, 0xb3, 0x70, 0x3c, 0x9c, 0xaa, 0xca, 0x11, 0x66, 0x3d,
    0x6c, 0x22, 0xa3, 0xf3, 0xc3, 0x32, 0xbb, 0x81, 0x4f, 0x33, 0xc7, 0xdd,
    0xc8, 0xa8, 0x06, 0x7a, 0xc9, 0x58, 0xa5, 0xdc, 0xdc, 0xe8, 0xd7, 0x74,
    0xb1, 0x85, 0x24, 0xe7, 0xe3, 0xee, 0x93, 0xf4, 0x8f, 0xf7, 0x6b, 0xd8,
    0xb1, 0xfb, 0xd9, 0xe4, 0xaf, 0xbf, 0x73, 0xd0, 0x40, 0x59, 0x7d, 0xd0,
    0x26, 0x4f, 0x16, 0x1a, 0xc2, 0x51, 0xc4, 0x47, 0x49, 0x2c, 0x68, 0x13,
    0xac, 0xa3, 0x18, 0xe7, 0x67, 0xcf, 0xb7, 0xfa, 0x3e, 0xf7, 0x8b, 0x20,
    0x1e, 0x7b, 0xe2, 0x44, 0x0e, 0x47, 0x0b, 0x7c, 0x78, 0xf9, 0xf4, 0xca,
    0x27, 0x6b, 0x4c, 0x2d, 0x62, 0x72, 0xd8, 0xa4, 0x10, 0x3d, 0xe7, 0x1d,
    0x88, 0x4c, 0x50, 0xe5,
};

static const unsigned char TA_RSA_E0[] = {
    0x01, 0x00, 0x01,
};

static const br_x509_trust_anchor TAs[] = {
    {
        { (unsigned char *)TA_DN0, sizeof TA_DN0 },
        BR_X509_TA_CA,
        {
            BR_KEYTYPE_RSA,
            { .rsa = {
                (unsigned char *)TA_RSA_N0, sizeof TA_RSA_N0,
                (unsigned char *)TA_RSA_E0, sizeof TA_RSA_E0,
            } }
        }
    },
};

#ifdef __cplusplus
} /* extern "C" */
#endif

#endif /* ifndef _MOSQUITTO_ORG_H_ */