Jan 12, 2015

Starting systemd service instance for device from udev

Needed to implement a thing that would react on USB Flash Drive inserted (into autonomous BBB device) - to get device name, mount fs there, rsync stuff to it, unmount.

To avoid whatever concurrency issues (i.e. multiple things screwing with device in parallel), proper error logging and other startup things, most obvious thing is to wrap the script in a systemd oneshot service.

Only non-immediately-obvious problem for me here was how to pass device to such service properly.

With a bit of digging through google results (and even finding one post here somehow among them), eventually found "Pairing udev's SYSTEMD_WANTS and systemd's templated units" resolved thread, where what seem to be current-best approach is specified.

Adapting it for my case and pairing with generic patterns for device-instantiated services, resulted in the following configuration.

99-sync-sensor-logs.rules:

SUBSYSTEM=="block", ACTION=="add", ENV{ID_TYPE}="disk", ENV{DEVTYPE}=="partition",\
  PROGRAM="/usr/bin/systemd-escape -p --template=sync-sensor-logs@.service $env{DEVNAME}",\
  ENV{SYSTEMD_WANTS}+="%c"

sync-sensor-logs@.service:

[Unit]
BindTo=%i.device
After=%i.device

[Service]
Type=oneshot
TimeoutStartSec=300
ExecStart=/usr/local/sbin/sync-sensor-logs /%I
This makes things stop if it works for too long or if device vanishes (due to BindTo=) and properly delays script start until device is ready.
"sync-sensor-logs" script at the end gets passed original unescaped device name as an argument.
Easy to apply all the systemd.exec(5) and systemd.service(5) parameters on top of this.

Does not need things like systemctl invocation or manual systemd escaping re-implementation either, though running "systemd-escape" still seem to be necessary evil there.

systemd-less alternative seem to be having a script that does per-device flock, timeout logic and a lot more checks for whether device is ready and/or still there, so this approach looks way saner and clearer, with a caveat that one should probably be familiar with all these systemd features.

Jun 16, 2012

Proper(-ish) way to start long-running systemd service on udev event (device hotplug)

Update 2015-01-12: There's a follow-up post with a different way to do that, enabled by "systemd-escape" tool available in more recent systemd versions.

I use a smartcard token which requires long-running (while device is plugged) handler process to communicate with the chip.
Basically, udev has to start a daemon process when the device get plugged.
Until recently, udev didn't mind doing that via just RUN+="/path/to/binary ...", but in recent merged systemd-udevd versions this behavior was deprecated:
RUN
...
Starting daemons or other long running processes is not appropriate for
udev; the forked processes, detached or not, will be unconditionally killed
after the event handling has finished.

I think it's for the best - less accumulating cruft and unmanageable pids forked from udevd, but unfortunately it also breaks existing udev rule-files, the ones which use RUN+="..." to do just that.

One of the most obvious breakage for me was the smartcard failing, so decided to fix that. Documentation on the whole migration process is somewhat lacking (hence this post), even though docs on all the individual pieces are there (which are actually awesome).

Main doc here is systemd.device(5) for the reference on the udev attributes which systemd recognizes, and of course udev(7) for a generic syntax reference.
Also, there's this entry on Lennart's blog.

In my case, when device (usb smartcard token) get plugged, ifdhandler process should be started via openct-control (OpenCT sc middleware), which then creates unix socket through which openct libraries (used in turn by OpenSC PKCS#11 or PCSClite) can access the hardware.

So, basically I've had something like this (there are more rules for different hw, of course, but for the sake of clarity...):

SUBSYSTEM!="usb", GOTO="openct_rules_end"
ACTION!="add", GOTO="openct_rules_end"
PROGRAM="/bin/sleep 0.1"
SUBSYSTEM=="usb", ENV{DEVTYPE}=="usb_device",\
  ENV{ID_VENDOR_ID}=="0529", ENV{ID_MODEL_ID}=="0600",\
  GROUP="usb",\
  RUN+="/usr/sbin/openct-control attach usb:$env{PRODUCT} usb $env{DEVNAME}"
LABEL="openct_rules_end"

Instead of RUN here, ENV{SYSTEMD_WANTS} can be used to start a properly-handled service, but note that some hardware parameters are passed from udev properties and in general systemd unit can't reference these.

I.e. if just ENV{SYSTEMD_WANTS}="openct-handler.service" (or more generic smartcard.target) is started, it won't know which device to pass to "openct-control attach" command.

One way might be storing these parameters in some dir, where they'll be picked by some path unit, a bit more hacky way would be scanning usb bus in the handler, and yet another one (which I decided to go along with) is to use systemd unit-file templating to pass these parameters.

openct-handler@.service:

[Unit]
Requires=openct.service

[Service]
Type=forking
GuessMainPID=false
ExecStart=/bin/sh -c "exec openct-control attach %I"

Note that it requires openct.service, which is basically does "openct-control init" once per boot to setup paths and whatnot:

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/sbin/openct-control init
ExecStop=/usr/sbin/openct-control shutdown

[Install]
WantedBy=multi-user.target
Another thing to note is that "sh" used in the handler.
It's intentional, because just %I will be passed by systemd as a single argument, while it should be three of them after "attach".

Finally, udev rules file for the device:

SUBSYSTEM!="usb", GOTO="openct_rules_end"
ACTION!="add", GOTO="openct_rules_end"
SUBSYSTEM=="usb", ENV{DEVTYPE}=="usb_device",\
  ENV{ID_VENDOR_ID}=="0529", ENV{ID_MODEL_ID}=="0600",\
  GROUP="usb", TAG+="systemd",\
  ENV{SYSTEMD_WANTS}="openct-handler@\
  usb:$env{ID_VENDOR_ID}-$env{ID_MODEL_ID}-$env{ID_REVISION}\
  \x20usb\x20-dev-bus-usb-$env{BUSNUM}-$env{DEVNUM}.service"
LABEL="openct_rules_end"

(I highly doubt newline escaping in ENV{SYSTEMD_WANTS} above will work - added them just for readability, so pls strip these in your mind to a single line without spaces)

Systemd escaping in the rule above is described in systemd.unit(5) and produces a name - and start a service - like this one:

openct-handler@usb:0529-0600-0100\x20usb\x20-dev-bus-usb-002-003.service

Which then invokes:

sh -c "exec openct-control attach\
  usb:0529/0600/0100 usb /dev/bus/usb/002/003"

And it forks ifdhandler process, which works with smartcard from then on.

ifdhandler seem to be able to detect unplugging events and exits gracefully, but otherwise BindTo= unit directive can be used to stop the service when udev detects that device is unplugged.

Note that it might be more obvious to just do RUN+="systemctl start whatever.service", but it's a worse way to do it, because you don't bind that service to a device in any way, don't produce the "whatever.device" unit and there are lot of complications due to systemctl being a tool for the user, not the API proper.

Member of The Internet Defense League