My blog_title_here

cgroups initialization, libcgroup and my ad-hoc replacement for it

Update 2019-10-02: This still works, but only for old cgroup-v1 interface, and is deprecated as such, plus largely unnecessary with modern systemd - see cgroup-v2 resource limits for apps with systemd scopes and slices post for more info.

I've lived with all of the above on the desktop in the (not so distant) past and cgroups just make all this shit go away - even forkbombs and super-multithreaded i/o can just be isolated in their own cgroup (from which there's no way to escape, not with any amount of forks) and scheduled/throttled (cpu hard-throttling - w/o cpusets - seem to be coming soon as well) as necessary.

Some problems with process classification and management of these limits seem to exist though.

Luckily, cgroups can be controlled via regular filesystem calls, and thoroughly documented (in Documentation/cgroups).

Anyways, my humble needs (for the purposes of this post) are:

• isolate compilation processes, usually performed by "cave" client of paludis package mangler (exherbo) and occasionally shell-invoked make in a kernel tree, from all the other processes;
• insulate specific "desktop" processes like firefox and occasional java-based crap from the rest of the system as well;
• create all these hierarchies in a freezer and have a convenient stop-switch for these groups.

So, how would I initially approach it with libcgroup? Ok, here's the cgconfig.conf:

### Mounts

mount {
  cpu = /sys/fs/cgroup/cpu;
  blkio = /sys/fs/cgroup/blkio;
  freezer = /sys/fs/cgroup/freezer;
}

### Hierarchical RCs

group tagged/cave {
  perm {
    task {
      uid = root;
      gid = paludisbuild;
    }
    admin {
      uid = root;
      gid = root;
    }
  }

  cpu {
    cpu.shares = 100;
  }
  freezer {
  }
}

group tagged/desktop/roam {
  perm {
    task {
      uid = root;
      gid = users;
    }
    admin {
      uid = root;
      gid = root;
    }
  }

  cpu {
    cpu.shares = 300;
  }
  freezer {
  }
}

group tagged/desktop/java {
  perm {
    task {
      uid = root;
      gid = users;
    }
    admin {
      uid = root;
      gid = root;
    }
  }

  cpu {
    cpu.shares = 100;
  }
  freezer {
  }
}

### Non-hierarchical RCs (blkio)

group tagged.cave {
  perm {
    task {
      uid = root;
      gid = users;
    }
    admin {
      uid = root;
      gid = root;
    }
  }

  blkio {
    blkio.weight = 100;
  }
}

group tagged.desktop.roam {
  perm {
    task {
      uid = root;
      gid = users;
    }
    admin {
      uid = root;
      gid = root;
    }
  }

  blkio {
    blkio.weight = 300;
  }
}

group tagged.desktop.java {
  perm {
    task {
      uid = root;
      gid = users;
    }
    admin {
      uid = root;
      gid = root;
    }
  }

  blkio {
    blkio.weight = 100;
  }
}

So, what do I want it to look like? This:

path: /sys/fs/cgroup

defaults:
  _tasks: root:wheel:664
  _admin: root:wheel:644
  freezer:

groups:

  base:
    _default: true
    cpu.shares: 1000
    blkio.weight: 1000

  tagged:
    cave:
      _tasks: root:paludisbuild
      _admin: root:paludisbuild
      cpu.shares: 100
      blkio.weight: 100

    desktop:
      roam:
        _tasks: root:users
        cpu.shares: 300
        blkio.weight: 300
      java:
        _tasks: root:users
        cpu.shares: 100
        blkio.weight: 100

It's parseable and readable YAML, not some parenthesis-semicolon nightmare of a C junkie (you may think that because of these spaces don't matter there btw... well, think again!).

Configuration file above (YAML) consists of three basic definition blocks:

roam:
  _tasks: root:users
  cpu:
    shares: 300
  blkio:
    weight: 300
    throttle.write_bps_device: 253:9 1000000

Or just with one line per rc knob, like this:

roam:
  _tasks: root:users
  cpu.shares: 300
  blkio.weight: 300
  blkio.throttle.write_bps_device: 253:9 1000000

"defaults" section mirrors the structure of any leaf cgroup. RCs/parameters here will be used for created cgroups, unless overidden in "groups" section.

That handles the limits and initial (default cgroup for all tasks) classification part, but then chosen tasks also need to be assigned to a dedicated cgroups.

Since I knew exactly which (few) apps should be confined to which groups, I just wrote a simple wrapper scripts for each, putting these in a separate dir, in the head of PATH. Example:

#!/usr/local/bin/cgrc -s desktop/roam/usr/bin/firefox

Guess I'll direct the next poor soul struggling with libcgroup here, instead of wasting time explaining how to work around that crap and facing the inevitable question "what else is there?" *sigh*.

All the mentioned scripts can be found here.

Sane playback for online streaming video via stream dumping

Problem is, "watching stuff online" is even worse than just "watching stuff" - either you pay attention or you just miss it, so I set up recording as a sort of "fs-based cache", at the very least to watch the recorded streams right as they get written, being able to pause or rewind, should I need to do so.

So, mplayer needs to be wrapped in a while loop, but then you also need to give dump files unique names to keep mplayer from overwriting them, and actually do several such loops for several streams you're recording (different halls, different talks, same time), and then probably because of strain on the streaming servers mplayer tend to reconnect several times in a row, producing lots of small dumps, which aren't really good for anything, and you'd also like to get some feedback on what's happening, and... so on.

Well, I just needed a better tool, and it looks like there aren't much simple non-gui dumpers for video+audio streams and not many libs to connect to http video streams from python, existing one being vlc bindings, which isn't probably any better than mplayer, provided all I need is just to dump a stream to a file, without any reconnection or rotation or multi-stream handling mechanism.

And now, back to the ongoing talks of day 3.

Commandline pulseaudio mixer tool

It was good enough for hotkeys, but I still wanted some nice alsamixer-like bars and thought it might be a good place to implement control per-stream sound levels as well, which is really another nice feature, but only as long as there's a way to actually adjust these levels, which there wasn't.

A few hours of python and learning curses and there we go:

ALSA plug-in [aplay] (fraggod@sacrilege:1424)        [ #############---------- ]
MPlayer (fraggod@sacrilege:1298)                     [ ####################### ]
Simple client (TCP/IP client from 192.168.0.5:49162) [ #########-------------- ]

Result was quite responsive and solid, which I kinda didn't expect from any sort of interactive interface.

os.listdir and os.walk in python without lists (by the grace of c-api generator) and recursion (custom stack)

As I got around to update some older crap in the my shared codebase (I mean mostly fgc by that), I've noticed that I use os.walk (although in most cases indirectly) in quite a few places, and it's implementation leaves a few things to be desired:

And another annoyance I've found there is it's weird interface - in nearly all cases I need to get nodes just one-by-one, so I'd be able to work with such pipeline with itertools or any other iterable-oriented stuff, but string and two lists is definitely not what I need in any case.

One good thing about the current os.walk I can see though, is that it shouldn't hold the dentry in cache any longer than necessary for a single scan, plus then it goes on to probe all the inodes there, which should be quite cache-friendly behavior, not taking into acount further usage of these entries.

Anyway, to put my mind at ease on the subject, and as a kind of exercise, I thought I'd fix all these issues.

At the lowest level, that's os.listdir, which I thought I'd replace with a simple generator. Alas, generators in py c-api aren't very simple, but certainly nothing to be afraid of either. Most (and probably the only) helpful info on the subject (with non-native C ppl in mind) was this answer on stack overflow, giving the great sample code.

Recursion, on the other hand, can be solved entirely in python, all that's needed is to maintain the custom processing stack, mirroring the natural recursion pattern. "Depth" ordering control can be easily implemented by making stack double-edged (as collections.deque) and the rest is just a simple logic excercise.

zcat, bzcat, lzcat, xzcat... Arrrgh! Autodetection rocks

Playing with dracut today, noticed that it can create lzma-compressed initrd's without problem, but it's "lsinitrd" script uses zcat to access initrd data, thus failing for lzma or bzip2 compression.

Of course the "problem" is nothing new, and I've bumped against it a zillion times in the past, although it looks like today I was a bit less (or more?) lazy than usual and tried to seek a solution - some *cat tool, which would be able to read any compressed format without the need to specify it explicitly.

Finding nothing of the /usr/bin persuasion, I noticed that there's a fine libarchive project, which can do all sort of magic just for this purpose, alas there seem to be no cli client for it to utilize this magic, so I got around to write my own one.

These few minutes of happy-hacking probably saved me a lot of time in the long run, guess the result may as well be useful to someone else:

#include <archive.h>
#include <archive_entry.h>
#include <stdio.h>
#include <stdlib.h>

const int BS = 16384;

int main(int argc, const char **argv) {
    if (argc > 2) {
        fprintf(stderr, "Usage: %s [file]\n", argv[0]);
        exit(1); }

    struct archive *a = archive_read_new();
    archive_read_support_compression_all(a);
    archive_read_support_format_raw(a);

    int err;
    if (argc == 2) err = archive_read_open_filename(a, argv[1], BS);
    else err = archive_read_open_fd(a, 0, BS);
    if (err != ARCHIVE_OK) {
        fprintf(stderr, "Broken archive (1)\n");
        exit(1); }

    struct archive_entry *ae;
    err = archive_read_next_header(a, &ae);
    if (err != ARCHIVE_OK) {
        fprintf(stderr, "Broken archive (2)\n");
        exit(1); }

    (void) archive_read_data_into_fd(a, 1);

    archive_read_finish(a);
    exit(0);
}

I've also created a project on sourceforge for it, in hopes that it'd save someone like me a bit of time with google-fu, but I doubt I'll add any new features here.

Further improvements for notification-daemon

It's been a while since I augmented libnotify / notification-daemon stack to better suit my (maybe not-so-) humble needs, and it certainly was an improvement, but there's no limit to perfection and since then I felt the urge to upgrade it every now and then.

Binding it to a key is just a matter of (a bit inelegant, but powerful) dbus-send tool invocation:

% dbus-send --type=method_call\
   --dest=org.freedesktop.Notifications\
   /org/freedesktop/Notifications\
   org.freedesktop.Notifications.CloseNotification uint32:0

% dbus-send --type=method_call\
   --dest=org.freedesktop.Notifications\
   /org/freedesktop/Notifications\
   org.freedesktop.Notifications.Set\
   dict:string:boolean:plug_toggle,true

(define-macro define-matcher (lambda
  (name comp last rev-args)
  `(define ,name (lambda args
    (if (= (length args) 1) ,last
      (let ((atom (car args)) (args (cdr args)))
      (,comp
        (~ ,@(if rev-args '((car args) atom) '(atom (car args))))
        (apply ,name (cons atom (cdr args))))))))))

(define-matcher ~all and #t #f)
(define-matcher all~ and #t #t)
(define-matcher ~any or #f #f)
(define-matcher any~ or #f #t)
(lambda (summary body)
  (not (and
    (~ "^erc: #\S+" summary)
    (~ "^\*\*\* #\S+ (was created on|modes:) " body))
    (all~ summary "^erc: #pulseaudio$" "^mail:")))

# Notification-proxy control
Print :Exec dbus-send --type=method_call\
    --dest=org.freedesktop.Notifications\
    /org/freedesktop/Notifications org.freedesktop.Notifications.Set\
    dict:string:boolean:plug_toggle,true
Shift Print :Exec dbus-send --type=method_call\
    --dest=org.freedesktop.Notifications\
    /org/freedesktop/Notifications org.freedesktop.Notifications.Set\
    dict:string:boolean:cleanup_toggle,true
Pause :Exec dbus-send --type=method_call\
    --dest=org.freedesktop.Notifications\
    /org/freedesktop/Notifications\
    org.freedesktop.Notifications.CloseNotification\
    uint32:0
Shift Pause :Exec dbus-send --type=method_call\
    --dest=org.freedesktop.Notifications\
    /org/freedesktop/Notifications\
    org.freedesktop.Notifications.Flush

Pretty sure there's more room for improvement in this aspect, so I'd have to extend the system once again, which is fun all by itself.

Resulting (and maybe further extended) script is here, now linked against a bit revised lis.py scheme implementation.

MooseFS usage experiences

Guess I can say that it's quite remarkable in a way that it's really unremarkable - I just kinda forgot it's there, which is probably the best thing one can expect from a filesystem.

Chunkservers' storage consists of idexed (AA/AABCD...) paths, according to chunk names, which can be easily retreived from master. They rely on fs scanning to determine which chunks they have, so I've been able to successfully merge two nodes into one w/o storing the chunks on different filesystems/paths (which is also perfectly possible).

Chunkservers talk to each other on p2p-basis (doesn't imply that they don't need connection to master, but bandwidth there doesn't seem to be an issue at all) to maintain requested replication goal and auto-balance disk space between themselves, so the free percentage tries to be equal on all nodes (w/o compromising the goal, of course), so with goal=2 and 4 nodes I have 30% space usage on backend-fs on both 500 GiB node and 3 TiB one.

Balancing seem to be managed by every chunkserver in background (not quite sure if I've seen it in any docs, but there's a "chunk testing" process, which seem to imply that, and can be tuned btw), according to info about chunk and other currently-available nodes' space utilization from master.

Hence, adding/removing nodes is a bliss - just turn it on/off, no configuration changes for other nodes are necessary - master sees the change (new/lost connection) and all the chunkservers start relocating/getting the chunks to restore the balance and maintain the requested goal. In a few hours everything will be balanced again.

Of course, it's just a commodity hardware and lighter loads, but it's still way above other stuff I've tried here in virtually every aspect, so thumbs up for moose.

Moar free time!

As of today, I'm unemployed once again.

Guess now I'll have time to debug and report a btrfs-systemd crash, read all the feeds, fix some long-standing issues on my home servers, update an antique web setup, write a few watch-notify scripts there, deploy/update a configuration management systems, update/finish/publish a few of my spare-time projects, start playing with a lot of new ideas, check out networking tools like connman, wicd, nm and a bunch of other cool-stuff oss projects, write a few hooks for plotting and graphing stuff in real-time, adapt emacs mail/rss tools, update other elisp stuff, emacs itself, a few symbian-to pc event hooks, check out gnustep environment, ltu lang articles, pybrain and a few other simple machine-learning implementations, some lua-ai for spring, play a lot of games I've missed in past few years, read a few dozen books I've already uploaded but never had a time to, study linear and geometric algebra... maybe find a new job, even, before I starve?

Nah, nobody in the world have that much time... ;)

From Baselayout to Systemd setup on Exherbo

#!/sbin/runscript
depend() {
    use logger
    need clock hostname
    provide cron
}
start() {
    ebegin "Starting ${SVCNAME}"
    start-stop-daemon --start --pidfile ${FCRON_PIDFILE}\
      --exec /usr/sbin/fcron -- -c ${FCRON_CONF}
    eend $?
}
stop() {
    ebegin "Stopping ${SVCNAME}"
    start-stop-daemon --stop --pidfile ${FCRON_PIDFILE}
    eend $?
}

...with $SVCNAME taken from the script name and other vars from complimentary "/etc/conf.d/someservice" file (with sensible defaults in initscript itself).

Such script already allows nice and logged output (with e* commands) and clearly-defined, relatively uncluttered sections for startup and shutdown. You don't have to parse commandline arguments (although it's perfectly possible), since baselayout scripts will do that, and every daemon is accounted for via "start-stop-daemon" wrapper - it has a few simple ways to check their status via passed --pidfile or --exec lines, plus it handles forking (if necessary), IO redirection, dropping privileges and stuff like that.

All these feats lead to much more consistent init and control over services' state:

root@damnation:~# rc-status -a
Runlevel: shutdown
  killprocs        [ stopped ]
  savecache        [ stopped ]
  mount-ro         [ stopped ]
Runlevel: single
Runlevel: nonetwork
  local            [ started ]
Runlevel: cryptinit
  rsyslog          [ started ]
  ip6tables        [ started ]
...
  twistd           [ started ]
  local            [ started ]
Runlevel: sysinit
  dmesg            [ started ]
  udev             [ started ]
  devfs            [ started ]
Runlevel: boot
  hwclock          [ started ]
  lvm              [ started ]
...
  wdd              [ started ]
  keymaps          [ started ]
Runlevel: default
  rsyslog          [ started ]
  ip6tables        [ started ]
...
  twistd           [ started ]
  local            [ started ]
Dynamic Runlevel: hotplugged
Dynamic Runlevel: needed
  sysfs            [ started ]
  rpc.pipefs       [ started ]
...
  rpcbind          [ started ]
  rpc.idmapd       [ started ]
Dynamic Runlevel: manual

Now, systemd provides all of these features, in a cleaner nicer form and much more, but that makes migration from one to the other actually harder.

Thus, it makes sense for systemd on exherbo to be totally independent of baselayout and it's scripts, and having a separate package option to install systemd and baselayout-specific init stuff:

root@sacrilege:~# cave show -f acpid
* sys-power/acpid
   ::arbor   2.0.6-r2* {:0}
   ::installed  2.0.6-r2 {:0}
   sys-power/acpid-2.0.6-r2:0::installed
   Description
acpid is designed to notify user-space programs of ACPI events. It will
will attempt to connect to the Linux kernel via the input layer and
netlink. When an ACPI event is received from one of these sources, acpid
will examine a list of rules, and execute the rules that match the event.
   Homepage  http://tedfelix.com/linux/acpid-netlink.html
   Summary  A configurable ACPI policy daemon for Linux
   From repositories arbor
   Installed time Thu Oct 21 23:11:55 YEKST 2010
   Installed using paludis-0.55.0-git-0.54.2-44-g203a470
   Licences  GPL-2
   Options  (-baselayout) (systemd) build_options: -trace

  sys-power/acpid-2.0.6-r2:0::arbor
   Homepage  http://tedfelix.com/linux/acpid-netlink.html
   Summary  A configurable ACPI policy daemon for Linux
   Description
acpid is designed to notify user-space programs of ACPI events. It will
will attempt to connect to the Linux kernel via the input layer and
netlink. When an ACPI event is received from one of these sources, acpid
will examine a list of rules, and execute the rules that match the event.
   Options  -baselayout systemd
     build_options: -recommended_tests split strip jobs -trace -preserve_work
   Overridden Masks
     Supported platforms ~amd64 ~x86

So, basically, the migration to systemd consists of enabling the option and flipping the "eclectic init" switch:

root@sacrilege:~# eclectic init list
Available providers for init:
 [1] systemd *
 [2] sysvinit

• A dedicated lv for whole setup.
• luksFormat it, so it'd represent an encrypted "raw" partition.
• pvcreate / vgcreate / lvcreate / mkfs on top of it, identical (although much smaller) to original system.
• A script to mount all these and rsync the "original" system to this replica, with a few post-sync hooks to make some vm-specific changes - different vg name, no extra devices for media content, simpler passwords.

And the first launch gave quite a mixed result: systemd fired a bunch of basic stuff at once, then hanged for about a minute before presenting a getty. After login, it turned out that none of the filesystems in /etc/fstab got mounted.

While this may look a bit like a problem, it's quite surprising how transparent and easy-to-debug whole process is, regardless of it's massively-parallel nature - all the information is available via "systemctl" and it's show/status commands, all the services are organized (and monitored) in systemd-cgls tree, and can be easily debugged with systemd monitoring and console/dmesg-logging features:

root@sacrilege:~# systemd-cgls
├ 2 [kthreadd]
├ 3 [ksoftirqd/0]
├ 6 [migration/0]
├ 7 [migration/1]
├ 9 [ksoftirqd/1]
├ 10 [kworker/0:1]
...
├ 2688 [kworker/0:2]
├ 2700 [kworker/u:0]
├ 2728 [kworker/u:2]
├ 2729 [kworker/u:4]
├ user
│ └ fraggod
│ └ no-session
│ ├ 1444 /bin/sh /usr/bin/startx
│ ├ 1462 xinit /home/fraggod/.xinitrc -- /etc/X11/xinit/xserverrc :0 -auth /home/fraggod/.serveraut...
...
│ ├ 2407 ssh root@anathema -Y
│ └ 2751 systemd-cgls
└ systemd-1
 ├ 1 /sbin/init
 ├ var-src.mount
 ├ var-tmp.mount
 ├ ipsec.service
 │ ├ 1059 /bin/sh /usr/lib/ipsec/_plutorun --debug --uniqueids yes --force_busy no --nocrsend no --str...
 │ ├ 1060 logger -s -p daemon.error -t ipsec__plutorun
 │ ├ 1061 /bin/sh /usr/lib/ipsec/_plutorun --debug --uniqueids yes --force_busy no --nocrsend no --str...
 │ ├ 1062 /bin/sh /usr/lib/ipsec/_plutoload --wait no --post
 │ ├ 1064 /usr/libexec/ipsec/pluto --nofork --secretsfile /etc/ipsec.secrets --ipsecdir /etc/ipsec.d -...
 │ ├ 1069 pluto helper # 0
 │ ├ 1070 pluto helper # 1
 │ ├ 1071 pluto helper # 2
 │ └ 1223 _pluto_adns
 ├ sys-kernel-debug.mount
 ├ var-cache-fscache.mount
 ├ net@.service
 ├ rpcidmapd.service
 │ └ 899 /usr/sbin/rpc.idmapd -f
 ├ rpcstatd.service
 │ └ 892 /sbin/rpc.statd -F
 ├ rpcbind.service
 │ └ 890 /sbin/rpcbind -d
 ├ wpa_supplicant.service
 │ └ 889 /usr/sbin/wpa_supplicant -c /etc/wpa_supplicant/wpa_supplicant.conf -u -Dwext -iwlan0
 ├ cachefilesd.service
 │ └ 883 /sbin/cachefilesd -n
 ├ dbus.service
 │ └ 784 /usr/bin/dbus-daemon --system --address=systemd: --nofork --systemd-activation
 ├ acpid.service
 │ └ 775 /usr/sbin/acpid -f
 ├ openct.service
 │ └ 786 /usr/sbin/ifdhandler -H -p etoken64 usb /dev/bus/usb/002/003
 ├ ntpd.service
 │ └ 772 /usr/sbin/ntpd -u ntp:ntp -n -g -p /var/run/ntpd.pid
 ├ bluetooth.service
 │ ├ 771 /usr/sbin/bluetoothd -n
 │ └ 1469 [khidpd_046db008]
 ├ syslog.service
 │ └ 768 /usr/sbin/rsyslogd -n -c5 -6
 ├ getty@.service
 │ ├ tty1
 │ │ └ 1451 /sbin/agetty 38400 tty1
 │ ├ tty3
 │ │ └ 766 /sbin/agetty 38400 tty3
 │ ├ tty6
 │ │ └ 765 /sbin/agetty 38400 tty6
 │ ├ tty5
 │ │ └ 763 /sbin/agetty 38400 tty5
 │ ├ tty4
 │ │ └ 762 /sbin/agetty 38400 tty4
 │ └ tty2
 │ └ 761 /sbin/agetty 38400 tty2
 ├ postfix.service
 │ ├ 872 /usr/lib/postfix/master
 │ ├ 877 qmgr -l -t fifo -u
 │ └ 2631 pickup -l -t fifo -u
 ├ fcron.service
 │ └ 755 /usr/sbin/fcron -f
 ├ var-cache.mount
 ├ var-run.mount
 ├ var-lock.mount
 ├ var-db-paludis.mount
 ├ home-fraggod-.spring.mount
 ├ etc-core.mount
 ├ var.mount
 ├ home.mount
 ├ boot.mount
 ├ fsck@.service
 ├ dev-mapper-prime\x2dswap.swap
 ├ dev-mqueue.mount
 ├ dev-hugepages.mount
 ├ udev.service
 │ ├ 240 /sbin/udevd
 │ ├ 639 /sbin/udevd
 │ └ 640 /sbin/udevd
 ├ systemd-logger.service
 │ └ 228 //lib/systemd/systemd-logger
 └ tmp.mount

root@sacrilege:~# systemctl status ipsec.service
ipsec.service - IPSec (openswan)
  Loaded: loaded (/etc/systemd/system/ipsec.service)
  Active: active (running) since Fri, 05 Nov 2010 15:16:54 +0500; 2h 16min ago
  Process: 981 (/usr/sbin/ipsec setup start, code=exited, status=0/SUCCESS)
  Process: 974 (/bin/sleep 10, code=exited, status=0/SUCCESS)
  CGroup: name=systemd:/systemd-1/ipsec.service
   ├ 1059 /bin/sh /usr/lib/ipsec/_plutorun --debug --uniqueids yes --force_busy no --noc...
   ├ 1060 logger -s -p daemon.error -t ipsec__plutorun
   ├ 1061 /bin/sh /usr/lib/ipsec/_plutorun --debug --uniqueids yes --force_busy no --noc...
   ├ 1062 /bin/sh /usr/lib/ipsec/_plutoload --wait no --post
   ├ 1064 /usr/libexec/ipsec/pluto --nofork --secretsfile /etc/ipsec.secrets --ipsecdir ...
   ├ 1069 pluto helper # 0
   ├ 1070 pluto helper # 1
   ├ 1071 pluto helper # 2
   └ 1223 _pluto_adns

It's not just hacking at some opaque *sh hacks (like debian init or even interactive-mode baselayout) and takes so little effort to the point that it's really enjoyable process.

Systemd boot time is way faster than sysvinit, as it is supposed to, although I don't really care, since reboot is seldom necessary here.

As a summary, I'd recommend everyone to give systemd a try, or at least get familiar with it's rationale and features (plus this three-part blog series: one, two, three). My units aren't perfect (and I'll probably update network-related stuff to use ConnMan), but if you're lazy, grab them here. Also, here is a repo with units for archlinux, which I loosely used as a reference point along with /lib/systemd contents.

Info feeds

Thanks to feedjack, I'm able to keep in sync with 120 feeds (many of them, like slashdot or reddit, being an aggregates as well), as of today. Quite a lot of stuff I couldn't even imagine handling a year ago, and a good aggregator definitely helps, keeping all the info just one click away.

And every workstation-based (desktop) aggregator I've seen is a fail:

• RSSOwl. Really nice interface and very powerful. That said, it eats more ram than a firefox!!! Hogs CPU till the whole system stutters, and eats more of it than every other app I use combined (yes, including firefox). Just keeping it in the background costs 20-30% of dualcore cpu. Changing "show new" to "show all" kills the system ;)
• liferea. Horribly slow, interface hangs on any action (like fetching feed "in the background"), hogs cpu just as RSSOwl and not quite as feature-packed.
• Claws-mail's RSSyl. Quite nice resource-wise and very responsive, unlike dedicated software (beats me why). Pity it's also very limited interface-wise and can't reliably keep track of many of feeds by itself, constantly loosing a few if closed non-properly (most likely it's a claws-mail fault, since it affects stuff like nntp as well).
• Emacs' gnus and newsticker. Good for a feed or two, epic fail in every way with more dozen of them.
• Various terminal-based readers. Simply intolerable.

Maybe someday I'll get around to use something like Google Reader, but it's still one hell of a mess, and it's no worse than similar web-based services out there. So much for the cloud services. *sigh*