_0ukIFTYutB.png?auto=compress%2Cformat)
Hardware components | ||||||
 |
| × | 1 | |||
What?
This post is a small tutorial of how to build a local device discovery service for devices such as Particle Photon and Redbear Duo. It uses consul.io as a service registration tool. This tool is well known in the DevOps and Cloud/IT Infrastructure space, offering a clustered key/value store combined with a REST API and DNS.
At the end of the tutorial, we'll have a simple Particle App, that publishes its own Device ID and local IP Address via UDP Multicast to a python-based proxy. This proxy forwards the information to consul, which acts as a DNS server - so our device can be ping'ed by its Device ID.
Why?
Particle Photons or Redbear Duos usually connect to the Particle Cloud Service after being powered. This way, they are able to receive updates on Cloud Variables, can call functions, receive new application firmware etc. Devices that want to communicate with each other can use the Cloud services, but this can become difficult for some use cases where i.e. internet access is not available or devices with different access tokens need to communicate.
Using this solution, all Devices are known by their IDs and IP Addresses on the local network, without the need to publish data on a cloud service.
The post consists of three parts:
- Writing the Firmware Application that publishes its data using Multicast UDP
- Writing the proxy app
- Installing and using Consul
Writing the Particle App
The first step is to write an App that upon startup reads its own Device Data (ID, Local IP, ...), starts UDP in sending mode and sends the data as a single packet to a multicast group. I've chosen multicast, because 1) this way, one does not have to bake the target IP of the proxy service into the firmware and 2) there's is a great tutorial on using Multicast UDP on Photons, and it was working out really well.
Luckily, from Firmware version 0.4.7 on, Photons do support timers. So i decided to write a single function which gets called periodically. It implements a simple state machine for multicasting the data a number of times, and do retries in case of an error.
The .ino file can be found on the github link below. Its structure in short:
#define SERIAL_DEBUG
UDP udp;
// Structure of discovery_buffer:
// 0..3 local IP address, 4 octets
// 4..7 netmask, 4 octets
// 8..11 gateway IP, 4 octets
// 12..15 version info, 4 bytes
// 16..16+25 device ID, including \0 byte
const size_t discovery_bufferSize = 64;
unsigned char discovery_buffer[discovery_bufferSize];
IPAddress discovery_remoteIP(239,1,10,10);
int discovery_remotePort = 9000;
#define DISCOVERY_STATE_NEW 0
#define DISCOVERY_STATE_SENDING_START 1
#define DISCOVERY_STATE_SENDING_ERROR 2
#define DISCOVERY_STATE_SENDING_OK 3
#define DISCOVERY_STATE_DONE 4
// Maximum number of successful discovery messages sent.
// After this count has been reached, discovery messages
// are discontinued.
#define DISCOVERY_SEND_MAX_COUNT 5
int discovery_state = DISCOVERY_STATE_NEW;
int discovery_send_count = 0;
Timer *discovery_timer = NULL;
/**
* cp_ipa
* copy the four octets of an IPAddress to an array
*/
void discovery_cp_ipa(unsigned char *dest, IPAddress addr) {
(...)
}
/**
* set_buffer_wifidata
* copies the local IP, subnet mask and gateway IP to global buffer var
*/
void discovery_set_buffer_wifidata() {
(...)
}
void discovery_information_thread() {
(...)
}
// Create a timer, so that our discovery routine gets
// called every 2 secs.
Timer timer(2000,discovery_information_thread);
void setup() {
#ifdef SERIAL_DEBUG
Serial.begin(9600);
#endif
udp.begin(0);
// initialize our discovery code ,start the timer
discovery_state = DISCOVERY_STATE_NEW;
discovery_timer = &timer;
timer.start();
}
void loop() {
// your code goes here
}
It starts by defining SERIAL_DEBUG, so that detailed debug messages go out to the UART - very helpful in the beginning. Next, buffers for constructing and sending data are declared, and the Multicast IP is defined - 239.1.10.10, Port 9000. You're free to change that to whatever multicast address works for your network.
discovery_set_buffer_wifidata() fills the buffer with
- Local IP
- Netmask
- Gateway IP
- Firmware Version
- Device ID
Important part is discovery_information_thread(), the callback function of our timer. It starts in State DISCOVERY_STATE_NEW, collecting data and advancing itself to DISCOVERY_STATE_SENDING and so on, finally reaching DISCOVERY_STATE_DONE.
This function is wrapper in a Timer object, which is started in the setup() function. Note that the loop() is empty, so you can put whatever code you like into here. The state machine of the discovery_information_thread makes this approach non-blocking.
Powering the device, let's look at the serial output:
$ particle serial monitor
Opening serial monitor for com port: "/dev/cu.usbmodem1411"
send_discovery_information_thread(state=0)
send_discovery_information_thread: starting to send to 239.1.10.10:9000
send_discovery_information_thread(state=1)
send_discovery_information_thread: successfully sent, count=1
send_discovery_information_thread(state=3)
send_discovery_information_thread: successfully sent, count=2
send_discovery_information_thread(state=3)
send_discovery_information_thread: successfully sent, count=3
send_discovery_information_thread(state=3)
send_discovery_information_thread: successfully sent, count=4
send_discovery_information_thread(state=3)
send_discovery_information_thread: successfully sent, count=5
send_discovery_information_thread(state=3)
send_discovery_information_thread: exceeded maximum send count.
send_discovery_information_thread(state=4)
send_discovery_information_thread: done sending, disposing timer
It tries to send its buffer up to 5 times, with growing delays in between. In the end, it automatically disposes of the timer, so the multicasting steps terminate themselves.
Writing the Proxy App
So our device sends data to whoever is listening, now we need to implement a listener. I used the Python listener code from the Multicast UDP example as a starting point. Here it goes (also on github project):
#!/usr/bin/python
import socket
import struct
MCAST_GRP = '239.1.10.10'
MCAST_PORT = 9000
def notify_device(device_id,version_buf,local_ip,netmask,gateway_ip):
print "Device(id=%s,v=%s,ip=%s/%s,gateway=%s)" % (device_id,version_buf,local_ip,netmask,gateway_ip)
def main():
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind((MCAST_GRP, MCAST_PORT))
mreq = struct.pack("4sl", socket.inet_aton(MCAST_GRP), socket.INADDR_ANY)
sock.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq)
print "Listening on %s:%s, hit CTRL-C to stop..." % (MCAST_GRP, MCAST_PORT)
while True:
data = sock.recv(64)
local_ip = "%d.%d.%d.%d" % (ord(data[0]),ord(data[1]),ord(data[2]),ord(data[3]))
netmask = "%d.%d.%d.%d" % (ord(data[4]),ord(data[5]),ord(data[6]),ord(data[7]))
gateway_ip = "%d.%d.%d.%d" % (ord(data[8]),ord(data[9]),ord(data[10]),ord(data[11]))
version_buf = (ord(data[12]) << 24 | ord(data[13]) << 16 | ord(data[14]) << 8 | ord(data[15]))
device_id = data[16:16+24]
notify_device(device_id,version_buf,local_ip,netmask,gateway_ip)
if __name__ == "__main__":
main()
It starts UDP-listening on the multicast address, looping forever and dumping out whatever it receives (after parsing out the bytes from the buffer)
After starting this script, powering or resetting a device, we obtain the device data:
$ ./discovery-dump.py
Listening on 239.1.10.10:9000, hit CTRL-C to stop...
Device(id=2f0028001647343337363432,v=264448,ip=192.168.179.23/255.255.255.0,gateway=192.168.179.1)
Device(id=2f0028001647343337363432,v=264448,ip=192.168.179.23/255.255.255.0,gateway=192.168.179.1)
Device(id=2f0028001647343337363432,v=264448,ip=192.168.179.23/255.255.255.0,gateway=192.168.179.1)
Device(id=2f0028001647343337363432,v=264448,ip=192.168.179.23/255.255.255.0,gateway=192.168.179.1)
Device(id=2f0028001647343337363432,v=264448,ip=192.168.179.23/255.255.255.0,gateway=192.168.179.1)
Good. Now that needs to go into consul!
Installing and using Consul
Consul can be downloaded and installed from consul.io. It is available as a precompiled Go binary for multiple platforms, source code is at github for those who like to build the software themselves. Consul is under MPL license.
For Ubuntu, i installed
- consul_0.6.3_linux_amd64.zip and
- consult_0.6.3_web_ui.zip
These files can reside in a local directory, unzipping is sufficient. I created a configuration file to tie some settings to a local context, i.e. serving only on 127.0.0.1.
The configuration file is named localiotdev_config.json in the git hub repository.
Make sure, both zip files are unzipped, and the configuration file is placed at the same directory. Starting consul:
~$ consul agent -config-file=./localiotdev_config.json
==> WARNING: Bootstrap mode enabled! Do not enable unless necessary
==> Starting Consul agent...
==> Starting Consul agent RPC...
==> Consul agent running!
Node name: 'photon-discovery-node'
Datacenter: 'local'
Server: true (bootstrap: true)
Client Addr: 127.0.0.1 (HTTP: 8500, HTTPS: -1, DNS: 8600, RPC: 8400)
Cluster Addr: 127.0.0.1 (LAN: 8301, WAN: 8302)
Gossip encrypt: false, RPC-TLS: false, TLS-Incoming: false
Atlas: <disabled>
Consul prints out a lot of messages, including the API ports at the top. Needless to say that at present this is a complete insecure setup: No TLS, no Tokens, no ACLs so far.
Pointing a browser to 127.0.0.1:8500/ui gives us the nice UI view:
And there are already some device ids inside! For that to work we need to improve the python proxy script. Luckily, there is a python module for consul, so that we're easily able to add new services etc.
~$ sudo pip install consulate
Collecting consulate
Downloading consulate-0.6.0-py2-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): requests<3.0.0,>=2.0.0 in /usr/local/lib/python2.7/dist-packages (from consulate)
Installing collected packages: consulate
Successfully installed consulate-0.6.0
discovery-forwarder.py extends the simple dump by a call to consul:
import consulate
(...)
def notify_device(device_id,version_buf,local_ip,netmask,gateway_ip):
print "Device(id=%s,v=%s,ip=%s/%s,gateway=%s)" % (device_id,version_buf,local_ip,netmask,gateway_ip)
# register
consul = consulate.Consul()
consul.agent.service.register(device_id,address=local_ip);
The consulate module puts together a piece JSON, POSTing it to the consul API.
Running the scripts and powering devices, all devices are registered to consul. Now what do we do with that?
Querying Consul by DNS
Fortunately, consul offers a DNS interface, so we can use tools such as dig to query it. We need dig to supply the DNS interface of consul (@127.0.0.1 -p 8600). Devices are stored as services, so a domain (service.localiotdev) needs to be appended to the "host" (=device) name.
$ dig @127.0.0.1 -p 8600 400031000d47353033323637.service.localiotdev
; <<>> DiG 9.9.5-3ubuntu0.8-Ubuntu <<>> @127.0.0.1 -p 8600 400031000d47353033323637.service.localiotdev
(...)
;; QUESTION SECTION:
;400031000d47353033323637.service.localiotdev. IN A
;; ANSWER SECTION:
400031000d47353033323637.service.localiotdev. 0 IN A 192.168.179.39
;; Query time: 0 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
Voilà, given the device ID of a photon, consul tells us the ip (192.168.179.39) on our local network.
The last step is about including consuls DNS service into the regular DNS lookup mechanism, so that tools such as curl, wget or ping are able to use these kind of host names. Caution: please only follow these instructions if you're familiar with your network and os infrastructure, things might break here.
Unfortunately this depends on the type and set up of the operating system we're running on. In this example, it's Ubuntu 14.04, where NetworkManager orchestrates a local dnsmasq. Other OSes might do this different. If you're unsure, please look for good methods in other forums.
For Ubuntu, it is sufficient to include consul into the dnsmasq configuration by adding a new file to /etc/NetworkManager/dnsmasq.d/, i.e. 10-consul, with:
server=/localiotdev/127.0.0.1#8600
and restart network manager:
~$ sudo service network-manager restart
Then:
$ ping 400031000d47353033323637.service.localiotdev
PING 400031000d47353033323637.service.localiotdev (192.168.179.39) 56(84) bytes of data.
64 bytes from 192.168.179.39: icmp_seq=1 ttl=255 time=21.6 ms
64 bytes from 192.168.179.39: icmp_seq=2 ttl=255 time=9.51 ms
64 bytes from 192.168.179.39: icmp_seq=3 ttl=255 time=103 ms
64 bytes from 192.168.179.39: icmp_seq=4 ttl=255 time=129 ms
64 bytes from 192.168.179.39: icmp_seq=5 ttl=255 time=77.1 ms
64 bytes from 192.168.179.39: icmp_seq=6 ttl=255 time=6.75 ms
^C
--- 400031000d47353033323637.service.localiotdev ping statistics ---
6 packets transmitted, 6 received, 0% packet loss, time 5002ms
rtt min/avg/max/mdev = 6.755/58.075/129.845/48.145 ms
Comments