-
Notifications
You must be signed in to change notification settings - Fork 154
Home
MQTT IO is a server daemon which exposes general purpose IO (GPIO), hardware sensors and serial devices to an MQTT server, enabling remote monitoring and control. Home automation is a particular focus of the project, and is where it's been implemented and tested the most.
Originally built for the Raspberry Pi and its built in GPIO, the project quickly moved on to support other hardware platforms and many other GPIO modules, such as Banana Pi and PCF8574 GPIO chip.
- Usage Scenarios
- Design Overview
Since this is a piece of software that ties one thing (MQTT) to another (GPIO, sensors, streams), there are endless scenarios in which it can be used. Home Automation is where the software was born, but anywhere you require programmatic control of hardware devices, this software may be of use.
The following are a few simple examples which attempt to show most of the basic configuration options in place, and how they might be used.
A Raspberry Pi with an 8 channel PCF8574 IO chip, connected to relays which connect and disconnect the live wire to 4 mains sockets.
mqtt:
host: test.mosquitto.org
port: 1883
user: ""
password: ""
topic_prefix: home/livingroom/sockets
gpio_modules:
- name: sockets_gpio
module: pcf8574
i2c_bus_num: 1
chip_addr: 0x20
digital_outputs:
- name: socket1
module: sockets_gpio
pin: 1
on_payload: "ON"
off_payload: "OFF"
pullup: yes
- name: socket2
module: sockets_gpio
pin: 2
on_payload: "ON"
off_payload: "OFF"
pullup: yes
- name: socket3
module: sockets_gpio
pin: 3
on_payload: "ON"
off_payload: "OFF"
pullup: yes
- name: socket4
module: sockets_gpio
pin: 4
on_payload: "ON"
off_payload: "OFF"
pullup: yesThis configuration uses the PCF8574 GPIO module to set 4 GPIO pins of the PCF8574 as outputs, then subscribes to messages on an MQTT topic for each output. Sending the configured on/off payload to these topics will cause the software to turn the outputs on and off, therefore supplying power to, or removing power from the individual sockets. These sockets may then be used for any general purpose, such as powering lights, heaters or fans.
In order to turn each individual socket on and off, you'd send the following MQTT messages:
home/livingroom/sockets/output/socket1/set: ONhome/livingroom/sockets/output/socket1/set: OFFBy varying the socket1 part of the topic, you're able to choose which socket you'd like to control.
A Raspberry Pi with a DHT22 temperature and humidity sensor connected to pin 4 of its built in GPIO pins.
mqtt:
host: test.mosquitto.org
port: 1883
user: ""
password: ""
topic_prefix: home/livingroom/climate
sensor_modules:
- name: dht22_sensor
module: dht22
type: AM2302
pin: 4
sensor_inputs:
- name: temperature
module: dht22_sensor
interval: 10
digits: 4
type: temperature
- name: humidity
module: dht22_sensor
interval: 10
digits: 4
type: humidityThis configuration will poll the DHT22 sensor every 10 seconds and publish MQTT messages such as the following:
home/livingroom/climate/sensor/temperature: 23
home/livingroom/climate/sensor/humidity: 45A Beaglebone Black with a float switch connected to one of its built in GPIO pins which is pulled to ground when the water tank is full and the float switch engages.
mqtt:
host: test.mosquitto.org
port: 1883
user: ""
password: ""
topic_prefix: home/rainwater
gpio_modules:
- name: beaglebone_gpio
module: beaglebone
digital_inputs:
- name: tank
module: beaglebone_gpio
pin: GPIO0_26
on_payload: full
off_payload: ok
inverted: yes
pullup: yesThis configuration will poll the GPIO0_26 pin of the Beaglebone's built in GPIO and publish an MQTT message when it changes from low to high and vice versa:
home/rainwater/input/tank: ok
home/rainwater/input/tank: fullTODO: Add example for serial stream
The software reads the configuration file on startup, then initialises any configured IO, sensor and stream modules, connects to MQTT and subscribes to the relevant topics. GPIO and sensor inputs will either be polled as part of the main loop, or have their own polling co-routines on the asyncio version.
GPIO, sensor and stream modules are written specifically for each individual type of hardware, typically utilising a third party Python library to handle the actual communication with it. Each module implements a specific interface (GPIO, sensor or stream) so that there's a common layer of abstraction available to control them.
If a module requires any third party Python dependencies, they will be listed in the module code and installed as part of the module initialisation at runtime. This is so that the requirements for this project as a whole do not have to list and install every possible piece of hardware that's supported.
For more detailed information about configuration visit the Configuration page.
The software is configured using a single YAML file specified as the first argument upon execution.
In order to help avoid any misconfigurations, the provided configuration file is tested against a Cerberus schema and the program will display errors and exit during its initialisation phase if errors are found. Failing fast is preferable to only failing when, for example, an MQTT is received and the software attempts to control a module accordingly. This enables the user to fix the config while it's still fresh in mind, instead of some arbitrary amount of time down the line when the software may no longer be being supervised.
The main configuration schema is laid out in config.schema.yml and further schema may be optionally set as part of each module in the CONFIG_SCHEMA constant. This behaviour allows the modules to optionally require extra configuration specific to them.
Sensor and stream modules may also specify a config schema to be applied to each of the configured sensors and streams within the sensor_inputs, stream_reads and stream_writes sections.
There are various sections within the config file:
Contains everything to do with the software's connection to the MQTT server.
These are lists of modules in use by the software. Modules may be used multiple times, for example, there are multiple PCF8574 IO chips connected with different chip addresses. Each module instance is given a name, which is how it is referred to when setting up the individual inputs and outputs.
This list of modules is used as part of the software initialisation when calculating which Python packages are required in order to communicate with the configured modules.
Lists of each individual digital input or output to be used. Inputs will be polled and MQTT messages will be sent upon changes. MQTT topics will be subscribed to in order to change outputs when messages are received to them.
Entries in these lists will specify a name, which GPIO pin it relates to, which module to use and optionally, what the MQTT payloads should be to relate to the 'on' and 'off' values, among other configuration values.
A list of sensors that will be polled and their values published to MQTT at the given intervals.
These entries also specify a name, which module they use, the interval at which to poll them and perhaps which kind of value to pull from the sensor.
A list of streams to read from or write to.
These entries specify a name and a module. Some optional configuration such as encoding and interval is accepted for stream_reads, and further configuration may be specified by the module itself.
During initialisation of the software, the config file will be parsed and a list of inputs and outputs will be defined. For each of the outputs, a set of MQTT topics will be subscribed to, in order to give external applications access to control the outputs in various ways.
The structure of topics follow a set of rules. Each topic will begin with the string set for topic_prefix in the mqtt section of the config file. The name set within the config file for each of the inputs and outputs will be used to identify it within the topic.
Status
<topic_prefix>/<status_topic> - messages will be published to this topic containing information about the state of the software. Unless changed in the mqtt section of the config file, the payloads will either be running, stopped or dead.
Digital Inputs
<topic_prefix>/input/<input_name> - each change to the logic level of an input GPIO pin will be published here with the payload specified in on_payload and off_payload for this input.
Digital Outputs
<topic_prefix>/output/<output_name> - any time an output is set by this software, a message will be published on this topic to confirm that the change has been carried out.
<topic_prefix>/output/<output_name>/set - most of the time, this will be the topic that you will publish to in order to immediately set the output to a specific value, using the payloads set in on_payload and off_payload for this output.
<topic_prefix>/output/<output_name>/set_on_ms - publish a message to this topic using an integer payload in order to set this output 'on' for the given number of milliseconds. After this time, the output will be set to 'off'.
<topic_prefix>/output/<output_name>/set_off_ms - as above, but set the output to 'off', and then 'on' after the specified number of milliseconds.
Sensors
<topic_prefix>/sensor/<sensor_name> - each time a sensor is read, its value will be published in the payload of a message to this topic.
Stream Reads
<topic_prefix>/stream/<stream_read_name> - each time new data are received on this stream, they will be published in the payload of a message to this topic.
Stream Writes
<topic_prefix>/stream/<stream_write_name> - publish a message to this topic and the data in its payload will be sent to this stream.
The MQTT client ID identifies an instance of the software with the MQTT broker. It allows the broker to keep track of the state of the instance so that it can resume when it reconnects. This means that the ID must be unique for each instance that connects to the MQTT broker.
Since the MQTT client ID for each instance of the software is based on the topic_prefix supplied in config (#24), having multiple instances share the same topic_prefix will require you to set a different client_id for each:
mqtt:
host: test.mosquitto.org
client_id: mqtt-io-device1
topic_prefix: home/officemqtt:
host: test.mosquitto.org
client_id: mqtt-io-device2
topic_prefix: home/officeThis configuration isn't explicitly supported by the software, so it's important to understand that you'll get multiple responses when setting outputs and won't be able to identify their confirmations or and changed input messages.
One of the main goals for the project is to make it easy to implement support for different hardware. A GPIO abstraction layer containing all of the functionality that GPIO commonly exposes makes it simple to map specific GPIO library functions to the API of MQTT IO.
General Purpose Input and Output (GPIO) allows you to set individual device pins "high" or "low", or to detect whether they are being pulled high or low by an external device such as a switch.
When used as an output, one can send MQTT messages to a specific topic and this software will set the pin to the desired state. When set up as an input, an MQTT message will be published whenever the state of the pin changes.
Sensors take measurements at regular intervals and the values are published to specific MQTT topics.
Stream modules allow the sending and receiving of data from streams, such as a serial port. When data is received on the serial port, it is published to MQTT. When data is received on a specific MQTT topic, it's sent to the relevant serial port.
Most of the modules use an external Python library to control the hardware for which the module is written. This means that we don't know until runtime which Python packages should be installed. Just after the configuration file is parsed, the software works out which packages are required and installs any that are missing. Each of the modules specifies a REQUIREMENTS constant which lists the name(s) of any required Python packages installable with pip. For example, in the dht22 sensor module:
REQUIREMENTS = ("Adafruit_DHT",)- Loads and parses the configuration file
- Configures Python logging module
- Sets MQTT last will and testament (LWT) on MQTT client
- Validates config schema for GPIO modules
- Installs missing requirements for GPIO modules
- Validates config schema for sensor modules
- Installs missing requirements for sensor modules
- Validates config schema for stream modules
- Installs missing requirements for stream modules
- TODO: Validate GPIO input and output configurations
- Sets GPIO input pins to be inputs and configures their pullup/down values
- Configures interrupts for GPIO input pins if required/available
- Sets GPIO output pins to be outputs and sets their initial state if configured to do so
- Validates sensor input configs and performs any setup required by the sensor module for individual sensors
- Validates stream read configs and performs any setup required by the stream module for individual streams
- Validates stream write configs and performs any setup required by the stream module for individual streams
- Connects to MQTT
- Subscribes to MQTT topics for digital outputs and stream writes
- Publishes Home Assistant MQTT announcement messages for digital inputs, digital output and sensors
- Publishes initial states for digital outputs if configured to do so
- Starts sensor reading loop in a separate thread
- Starts stream reading loop in a separate thread
- Starts GPIO input reading loop in the main thread
Interrupts are an experimental feature that uses a GPIO module's underlying Python library to configure interrupts on digital inputs. The software provides a callback function to handle publishing to MQTT that an interrupt occurred. TODO: Include example of published message. This has mostly been tested on the Raspberry Pi's own GPIO.
Unfortunately there are some difficult issues to solve with interrupts. These include currently relying on the underlying library's own debouncing logic, which isn't perfect on Raspberry Pi, and the sub-optimal necessity to poll the inputs to receive their values after receiving an interrupt that says something changed.
This software was designed in a way that intends to match some of the design principles of Home Assistant. It also contains a feature that specifically enables Home Assistant auto-discovery of the digital inputs, digital outputs and sensors configured for use by this software.
After connecting to the MQTT server, the software will announce digital inputs, digital outputs and sensors to Home Assistant by publishing a JSON payload containing details of the input/output/sensor to the Home Assistant discovery topics. For example, the following JSON might be sent to the homeassistant/binary_sensor/pi-mqtt-gpio-429373a4/button/config topic for a digital input:
{
"name": "button",
"unique_id": "pi-mqtt-gpio-429373a4_stdio_input_button",
"state_topic": "pimqttgpio/mydevice/input/button",
"availability_topic": "pimqttgpio/mydevice/status",
"payload_available": "running",
"payload_not_available": "dead",
"payload_on": "ON",
"payload_off": "OFF",
"device": {
"manufacturer": "MQTT GPIO",
"identifiers": [
"mqtt-gpio",
"pi-mqtt-gpio-429373a4"
],
"name": "MQTT GPIO"
}
}The software is packaged like any other Python package, and uploaded to PyPI. This means that as long as you have pip installed, you can install this software with:
pip install pi-mqtt-gpioTo run the software, you must create a config file such as in the examples in this document, then use the following command (where config.yml is your configuration file):
python -m pi_mqtt_gpio.server config.ymlThe software isn't tied to any specific deployment method, so it's left up to the user to decide how to deploy it. There is a short tutorial on how to configure supervisor in the project's README.md file.
An experimental Docker image is provided at https://hub.docker.com/r/flyte/mqtt-gpio but it's currently unmaintained, with the intention of setting up an automated build at some point soon.
- Scheduler? (for set_on_ms etc.)
- GPIO abstraction API
- Sensor abstraction API
- Streams abstraction API