-----Original Message-----
From: Alex J Lennon [mailto:ajlennon@...]
Sent: Saturday, March 08, 2014 4:18 AM
To: Bryan Evenson
Cc: yocto@...
Subject: Re: [yocto] Remote management of embedded devices

+
On 07/03/2014 13:41, Bryan Evenson wrote:

Alex,

-----Original Message-----
From: yocto-bounces@... [mailto:yocto-
bounces@...] On Behalf Of Alex J Lennon
Sent: Friday, March 07, 2014 7:30 AM
To: yocto@...
Subject: [yocto] Remote management of embedded devices

Hi,

I'm looking into remote management solutions for an upcoming headless
mesh edge router running Poky. I think, at least in the initial
rollout we're going to need something more than, say, a cron-based
package update facility.

I'm currently thinking of going down the route of a cloud based
server providing SSH port forwarding to the embedded devices, and
then perhaps putting some scripting together on top of that to enable
monitoring, configuration, and control.

This is similar to what we have running. We're using opkg for the package

management system and allow firmware upgrade through the USB stick or
through remote access. We have an SSH server for remote access, and each
device has its own private key for access into the server. If the device is in
the middle of an SSH session with the remote access server, we can then SSH
into the device from our server if we want to do some deeper diagnostics on
an issue with a device. We have separate HTTP server which each device
queries to see if it needs to check in. So instead of having to do an ssh login
each time to check if there's a firmware upgrade available, it just needs to do
a HTTP GET to see if there's firmware available or if we want to check on its
status.

The biggest issues we've had have been due to our network setup and

handling upgrade both through the network and the USB stick. We are using
"opkg upgrade --download-only" as the first step of the upgrade process to
make sure that we don't do a partial upgrade. opkg-0.1.8 doesn't do --
download-only for file:// sources; why download a file that is already on the
filesystem? So I had to add a patch so it would download files from the USB
stick. We also had an issue with DNS names because the server has a
different name when the device finds it locally on our intranet then when it
connects remotely, so we had to setup separate mirrors. Other than that it's
been working pretty well.

Thanks Bryan, thanks really interesting. We do something similar with HTTP
requests on various projects and that might well be something to apply here.
I guess the limitation of this approach is that you are limited in when you can
obtain your back channel to the device to the frequency with which they
connect to the HTTP server. I do wonder just how much load there would be
on a server handling a lot of SSH connections if almost all of those
connections were just idling, sending keep alives now and again. I keep
meaning to try to put together some metrics on that as, if I'm right and the
load is low (as long as we ensure connections from devices are never
synchronised, e.g. power
failure) then I'd like to have an "always on" back-channel available

We considered the same thing. We weren't certain how scalable the idle SSH connection would be, as every device doesn't need to be connected at any given time. Depends on how many devices you plan on deploying, how many you think need to be connected at any given time, and what kind of response time you need. In our case, an HTTP request followed by the SSH connection if needed was a better solution.

Regards,
Bryan