My blog_title_here

Auto-updating desktop background with scaling via LQR and some other tricks

Got around to publish my (pretty-sophisticated at this point) background updater script.

Check it out, if you keep a local image collection, as a sample of how to batch-process images in most perversive ways using gimp and python or if you've never heard of "liquid rescale" algorithm:

• Sourceforge project for the whole thing
• GIMP plugin in the registry

Dashboard for enabled services in systemd

Systemd does a good job at monitoring and restarting services. It also keeps track of "failed" services, which you can easily see in systemctl output.

root@damnation:~# rc-status -a
Runlevel: shutdown
  killprocs  [ stopped ]
  savecache  [ stopped ]
  mount-ro   [ stopped ]
Runlevel: single
Runlevel: nonetwork
  local      [ started ]
Runlevel: cryptinit
  rsyslog    [ started ]
  ip6tables  [ started ]
...
  twistd     [ started ]
  local      [ started ]
Runlevel: sysinit
  dmesg      [ started ]
  udev       [ started ]
  devfs      [ started ]
Runlevel: boot
  hwclock    [ started ]
  lvm        [ started ]
...
  wdd        [ started ]
  keymaps    [ started ]
Runlevel: default
  rsyslog    [ started ]
  ip6tables  [ started ]
...
  twistd     [ started ]
  local      [ started ]
Dynamic Runlevel: hotplugged
Dynamic Runlevel: needed
  sysfs      [ started ]
  rpc.pipefs [ started ]
...
  rpcbind    [ started ]
  rpc.idmapd [ started ]
Dynamic Runlevel: manual

So, meet the systemd-dashboard tool:

root@damnation:~# systemd-dashboard -h
usage: systemd-dashboard [-h] [-s] [-u] [-n]

Tool to compare the set of enabled systemd services against currently running
ones. If started without parameters, it'll just show all the enabled services
that should be running (Type != oneshot) yet for some reason they aren't.

optional arguments:
 -h, --help  show this help message and exit
 -s, --status Show status report on found services.
 -u, --unknown Show enabled but unknown (not loaded) services.
 -n, --not-enabled Show list of services that are running but are not
   enabled directly.

Simple invocation will show what's not running while it should be:

root@damnation:~# systemd-dashboard
smartd.service
systemd-readahead-replay.service
apache.service

Adding "-s" flag will show what happened there in more detail (by the grace of "systemctl status" command):

root@damnation:~# systemd-dashboard -s

smartd.service - smartd
  Loaded: loaded (/lib64/systemd/system/smartd.service)
  Active: failed since Sun, 27 Feb 2011 11:44:05 +0500; 2s ago
  Process: 16322 ExecStart=/usr/sbin/smartd --no-fork --capabilities (code=killed, signal=KILL)
  CGroup: name=systemd:/system/smartd.service

systemd-readahead-replay.service - Replay Read-Ahead Data
  Loaded: loaded (/lib64/systemd/system/systemd-readahead-replay.service)
  Active: inactive (dead)
  CGroup: name=systemd:/system/systemd-readahead-replay.service

apache.service - apache2
  Loaded: loaded (/lib64/systemd/system/apache.service)
  Active: inactive (dead) since Sun, 27 Feb 2011 11:42:34 +0500; 51s ago
  Process: 16281 ExecStop=/usr/bin/apachectl -k stop (code=exited, status=0/SUCCESS)
  Main PID: 5664 (code=exited, status=0/SUCCESS)
  CGroup: name=systemd:/system/apache.service

Would you've noticed that readahead fails on a remote machine because the kernel is missing fanotify and the service apparently thinks "it's okay not to start" in this case? What about smartd you've killed a while ago and forgot to restart?

And you can check if you forgot to enable something with "-n" flag, which will show all the running stuff that was not explicitly enabled.

Code is under a hundred lines of python with the only dep of dbus-python package. You can grab the initial (probably not updated much, although it's probably finished as it is) version from here or a maintained version from fgtk repo (yes, there's an anonymous login form to pass).

If someone will also find the thing useful, I'd appreciate if you'll raise awareness to the issue within systemd project - I'd rather like to see such functionality in the main package, not hacked-up on ad-hoc basis around it.

Update (+20d): issue was noticed and will probably be addressed in systemd. Yay!

cgroups initialization, libcgroup and my ad-hoc replacement for it

Update 2019-10-02: This still works, but only for old cgroup-v1 interface, and is deprecated as such, plus largely unnecessary with modern systemd - see cgroup-v2 resource limits for apps with systemd scopes and slices post for more info.

I've lived with all of the above on the desktop in the (not so distant) past and cgroups just make all this shit go away - even forkbombs and super-multithreaded i/o can just be isolated in their own cgroup (from which there's no way to escape, not with any amount of forks) and scheduled/throttled (cpu hard-throttling - w/o cpusets - seem to be coming soon as well) as necessary.

Some problems with process classification and management of these limits seem to exist though.

Luckily, cgroups can be controlled via regular filesystem calls, and thoroughly documented (in Documentation/cgroups).

Anyways, my humble needs (for the purposes of this post) are:

• isolate compilation processes, usually performed by "cave" client of paludis package mangler (exherbo) and occasionally shell-invoked make in a kernel tree, from all the other processes;
• insulate specific "desktop" processes like firefox and occasional java-based crap from the rest of the system as well;
• create all these hierarchies in a freezer and have a convenient stop-switch for these groups.

So, how would I initially approach it with libcgroup? Ok, here's the cgconfig.conf:

### Mounts

mount {
  cpu = /sys/fs/cgroup/cpu;
  blkio = /sys/fs/cgroup/blkio;
  freezer = /sys/fs/cgroup/freezer;
}

### Hierarchical RCs

group tagged/cave {
  perm {
    task {
      uid = root;
      gid = paludisbuild;
    }
    admin {
      uid = root;
      gid = root;
    }
  }

  cpu {
    cpu.shares = 100;
  }
  freezer {
  }
}

group tagged/desktop/roam {
  perm {
    task {
      uid = root;
      gid = users;
    }
    admin {
      uid = root;
      gid = root;
    }
  }

  cpu {
    cpu.shares = 300;
  }
  freezer {
  }
}

group tagged/desktop/java {
  perm {
    task {
      uid = root;
      gid = users;
    }
    admin {
      uid = root;
      gid = root;
    }
  }

  cpu {
    cpu.shares = 100;
  }
  freezer {
  }
}

### Non-hierarchical RCs (blkio)

group tagged.cave {
  perm {
    task {
      uid = root;
      gid = users;
    }
    admin {
      uid = root;
      gid = root;
    }
  }

  blkio {
    blkio.weight = 100;
  }
}

group tagged.desktop.roam {
  perm {
    task {
      uid = root;
      gid = users;
    }
    admin {
      uid = root;
      gid = root;
    }
  }

  blkio {
    blkio.weight = 300;
  }
}

group tagged.desktop.java {
  perm {
    task {
      uid = root;
      gid = users;
    }
    admin {
      uid = root;
      gid = root;
    }
  }

  blkio {
    blkio.weight = 100;
  }
}

So, what do I want it to look like? This:

path: /sys/fs/cgroup

defaults:
  _tasks: root:wheel:664
  _admin: root:wheel:644
  freezer:

groups:

  base:
    _default: true
    cpu.shares: 1000
    blkio.weight: 1000

  tagged:
    cave:
      _tasks: root:paludisbuild
      _admin: root:paludisbuild
      cpu.shares: 100
      blkio.weight: 100

    desktop:
      roam:
        _tasks: root:users
        cpu.shares: 300
        blkio.weight: 300
      java:
        _tasks: root:users
        cpu.shares: 100
        blkio.weight: 100

It's parseable and readable YAML, not some parenthesis-semicolon nightmare of a C junkie (you may think that because of these spaces don't matter there btw... well, think again!).

Configuration file above (YAML) consists of three basic definition blocks:

roam:
  _tasks: root:users
  cpu:
    shares: 300
  blkio:
    weight: 300
    throttle.write_bps_device: 253:9 1000000

Or just with one line per rc knob, like this:

roam:
  _tasks: root:users
  cpu.shares: 300
  blkio.weight: 300
  blkio.throttle.write_bps_device: 253:9 1000000

"defaults" section mirrors the structure of any leaf cgroup. RCs/parameters here will be used for created cgroups, unless overidden in "groups" section.

That handles the limits and initial (default cgroup for all tasks) classification part, but then chosen tasks also need to be assigned to a dedicated cgroups.

Since I knew exactly which (few) apps should be confined to which groups, I just wrote a simple wrapper scripts for each, putting these in a separate dir, in the head of PATH. Example:

#!/usr/local/bin/cgrc -s desktop/roam/usr/bin/firefox

Guess I'll direct the next poor soul struggling with libcgroup here, instead of wasting time explaining how to work around that crap and facing the inevitable question "what else is there?" *sigh*.

All the mentioned scripts can be found here.

Sane playback for online streaming video via stream dumping

Problem is, "watching stuff online" is even worse than just "watching stuff" - either you pay attention or you just miss it, so I set up recording as a sort of "fs-based cache", at the very least to watch the recorded streams right as they get written, being able to pause or rewind, should I need to do so.

So, mplayer needs to be wrapped in a while loop, but then you also need to give dump files unique names to keep mplayer from overwriting them, and actually do several such loops for several streams you're recording (different halls, different talks, same time), and then probably because of strain on the streaming servers mplayer tend to reconnect several times in a row, producing lots of small dumps, which aren't really good for anything, and you'd also like to get some feedback on what's happening, and... so on.

Well, I just needed a better tool, and it looks like there aren't much simple non-gui dumpers for video+audio streams and not many libs to connect to http video streams from python, existing one being vlc bindings, which isn't probably any better than mplayer, provided all I need is just to dump a stream to a file, without any reconnection or rotation or multi-stream handling mechanism.

And now, back to the ongoing talks of day 3.

Commandline pulseaudio mixer tool

It was good enough for hotkeys, but I still wanted some nice alsamixer-like bars and thought it might be a good place to implement control per-stream sound levels as well, which is really another nice feature, but only as long as there's a way to actually adjust these levels, which there wasn't.

A few hours of python and learning curses and there we go:

ALSA plug-in [aplay] (fraggod@sacrilege:1424)        [ #############---------- ]
MPlayer (fraggod@sacrilege:1298)                     [ ####################### ]
Simple client (TCP/IP client from 192.168.0.5:49162) [ #########-------------- ]

Result was quite responsive and solid, which I kinda didn't expect from any sort of interactive interface.

os.listdir and os.walk in python without lists (by the grace of c-api generator) and recursion (custom stack)

As I got around to update some older crap in the my shared codebase (I mean mostly fgc by that), I've noticed that I use os.walk (although in most cases indirectly) in quite a few places, and it's implementation leaves a few things to be desired:

And another annoyance I've found there is it's weird interface - in nearly all cases I need to get nodes just one-by-one, so I'd be able to work with such pipeline with itertools or any other iterable-oriented stuff, but string and two lists is definitely not what I need in any case.

One good thing about the current os.walk I can see though, is that it shouldn't hold the dentry in cache any longer than necessary for a single scan, plus then it goes on to probe all the inodes there, which should be quite cache-friendly behavior, not taking into acount further usage of these entries.

Anyway, to put my mind at ease on the subject, and as a kind of exercise, I thought I'd fix all these issues.

At the lowest level, that's os.listdir, which I thought I'd replace with a simple generator. Alas, generators in py c-api aren't very simple, but certainly nothing to be afraid of either. Most (and probably the only) helpful info on the subject (with non-native C ppl in mind) was this answer on stack overflow, giving the great sample code.

Recursion, on the other hand, can be solved entirely in python, all that's needed is to maintain the custom processing stack, mirroring the natural recursion pattern. "Depth" ordering control can be easily implemented by making stack double-edged (as collections.deque) and the rest is just a simple logic excercise.

zcat, bzcat, lzcat, xzcat... Arrrgh! Autodetection rocks

Playing with dracut today, noticed that it can create lzma-compressed initrd's without problem, but it's "lsinitrd" script uses zcat to access initrd data, thus failing for lzma or bzip2 compression.

Of course the "problem" is nothing new, and I've bumped against it a zillion times in the past, although it looks like today I was a bit less (or more?) lazy than usual and tried to seek a solution - some *cat tool, which would be able to read any compressed format without the need to specify it explicitly.

Finding nothing of the /usr/bin persuasion, I noticed that there's a fine libarchive project, which can do all sort of magic just for this purpose, alas there seem to be no cli client for it to utilize this magic, so I got around to write my own one.

These few minutes of happy-hacking probably saved me a lot of time in the long run, guess the result may as well be useful to someone else:

#include <archive.h>
#include <archive_entry.h>
#include <stdio.h>
#include <stdlib.h>

const int BS = 16384;

int main(int argc, const char **argv) {
    if (argc > 2) {
        fprintf(stderr, "Usage: %s [file]\n", argv[0]);
        exit(1); }

    struct archive *a = archive_read_new();
    archive_read_support_compression_all(a);
    archive_read_support_format_raw(a);

    int err;
    if (argc == 2) err = archive_read_open_filename(a, argv[1], BS);
    else err = archive_read_open_fd(a, 0, BS);
    if (err != ARCHIVE_OK) {
        fprintf(stderr, "Broken archive (1)\n");
        exit(1); }

    struct archive_entry *ae;
    err = archive_read_next_header(a, &ae);
    if (err != ARCHIVE_OK) {
        fprintf(stderr, "Broken archive (2)\n");
        exit(1); }

    (void) archive_read_data_into_fd(a, 1);

    archive_read_finish(a);
    exit(0);
}

I've also created a project on sourceforge for it, in hopes that it'd save someone like me a bit of time with google-fu, but I doubt I'll add any new features here.

Further improvements for notification-daemon

It's been a while since I augmented libnotify / notification-daemon stack to better suit my (maybe not-so-) humble needs, and it certainly was an improvement, but there's no limit to perfection and since then I felt the urge to upgrade it every now and then.

Binding it to a key is just a matter of (a bit inelegant, but powerful) dbus-send tool invocation:

% dbus-send --type=method_call\
   --dest=org.freedesktop.Notifications\
   /org/freedesktop/Notifications\
   org.freedesktop.Notifications.CloseNotification uint32:0

% dbus-send --type=method_call\
   --dest=org.freedesktop.Notifications\
   /org/freedesktop/Notifications\
   org.freedesktop.Notifications.Set\
   dict:string:boolean:plug_toggle,true

(define-macro define-matcher (lambda
  (name comp last rev-args)
  `(define ,name (lambda args
    (if (= (length args) 1) ,last
      (let ((atom (car args)) (args (cdr args)))
      (,comp
        (~ ,@(if rev-args '((car args) atom) '(atom (car args))))
        (apply ,name (cons atom (cdr args))))))))))

(define-matcher ~all and #t #f)
(define-matcher all~ and #t #t)
(define-matcher ~any or #f #f)
(define-matcher any~ or #f #t)
(lambda (summary body)
  (not (and
    (~ "^erc: #\S+" summary)
    (~ "^\*\*\* #\S+ (was created on|modes:) " body))
    (all~ summary "^erc: #pulseaudio$" "^mail:")))

# Notification-proxy control
Print :Exec dbus-send --type=method_call\
    --dest=org.freedesktop.Notifications\
    /org/freedesktop/Notifications org.freedesktop.Notifications.Set\
    dict:string:boolean:plug_toggle,true
Shift Print :Exec dbus-send --type=method_call\
    --dest=org.freedesktop.Notifications\
    /org/freedesktop/Notifications org.freedesktop.Notifications.Set\
    dict:string:boolean:cleanup_toggle,true
Pause :Exec dbus-send --type=method_call\
    --dest=org.freedesktop.Notifications\
    /org/freedesktop/Notifications\
    org.freedesktop.Notifications.CloseNotification\
    uint32:0
Shift Pause :Exec dbus-send --type=method_call\
    --dest=org.freedesktop.Notifications\
    /org/freedesktop/Notifications\
    org.freedesktop.Notifications.Flush

Pretty sure there's more room for improvement in this aspect, so I'd have to extend the system once again, which is fun all by itself.

Resulting (and maybe further extended) script is here, now linked against a bit revised lis.py scheme implementation.

MooseFS usage experiences

Guess I can say that it's quite remarkable in a way that it's really unremarkable - I just kinda forgot it's there, which is probably the best thing one can expect from a filesystem.

Chunkservers' storage consists of idexed (AA/AABCD...) paths, according to chunk names, which can be easily retreived from master. They rely on fs scanning to determine which chunks they have, so I've been able to successfully merge two nodes into one w/o storing the chunks on different filesystems/paths (which is also perfectly possible).

Chunkservers talk to each other on p2p-basis (doesn't imply that they don't need connection to master, but bandwidth there doesn't seem to be an issue at all) to maintain requested replication goal and auto-balance disk space between themselves, so the free percentage tries to be equal on all nodes (w/o compromising the goal, of course), so with goal=2 and 4 nodes I have 30% space usage on backend-fs on both 500 GiB node and 3 TiB one.

Balancing seem to be managed by every chunkserver in background (not quite sure if I've seen it in any docs, but there's a "chunk testing" process, which seem to imply that, and can be tuned btw), according to info about chunk and other currently-available nodes' space utilization from master.

Hence, adding/removing nodes is a bliss - just turn it on/off, no configuration changes for other nodes are necessary - master sees the change (new/lost connection) and all the chunkservers start relocating/getting the chunks to restore the balance and maintain the requested goal. In a few hours everything will be balanced again.

Of course, it's just a commodity hardware and lighter loads, but it's still way above other stuff I've tried here in virtually every aspect, so thumbs up for moose.

Moar free time!

As of today, I'm unemployed once again.

Guess now I'll have time to debug and report a btrfs-systemd crash, read all the feeds, fix some long-standing issues on my home servers, update an antique web setup, write a few watch-notify scripts there, deploy/update a configuration management systems, update/finish/publish a few of my spare-time projects, start playing with a lot of new ideas, check out networking tools like connman, wicd, nm and a bunch of other cool-stuff oss projects, write a few hooks for plotting and graphing stuff in real-time, adapt emacs mail/rss tools, update other elisp stuff, emacs itself, a few symbian-to pc event hooks, check out gnustep environment, ltu lang articles, pybrain and a few other simple machine-learning implementations, some lua-ai for spring, play a lot of games I've missed in past few years, read a few dozen books I've already uploaded but never had a time to, study linear and geometric algebra... maybe find a new job, even, before I starve?

Nah, nobody in the world have that much time... ;)