My blog_title_here

cgroups initialization, libcgroup and my ad-hoc replacement for it

Update 2019-10-02: This still works, but only for old cgroup-v1 interface, and is deprecated as such, plus largely unnecessary with modern systemd - see cgroup-v2 resource limits for apps with systemd scopes and slices post for more info.

I've lived with all of the above on the desktop in the (not so distant) past and cgroups just make all this shit go away - even forkbombs and super-multithreaded i/o can just be isolated in their own cgroup (from which there's no way to escape, not with any amount of forks) and scheduled/throttled (cpu hard-throttling - w/o cpusets - seem to be coming soon as well) as necessary.

Some problems with process classification and management of these limits seem to exist though.

Luckily, cgroups can be controlled via regular filesystem calls, and thoroughly documented (in Documentation/cgroups).

Anyways, my humble needs (for the purposes of this post) are:

• isolate compilation processes, usually performed by "cave" client of paludis package mangler (exherbo) and occasionally shell-invoked make in a kernel tree, from all the other processes;
• insulate specific "desktop" processes like firefox and occasional java-based crap from the rest of the system as well;
• create all these hierarchies in a freezer and have a convenient stop-switch for these groups.

So, how would I initially approach it with libcgroup? Ok, here's the cgconfig.conf:

### Mounts

mount {
  cpu = /sys/fs/cgroup/cpu;
  blkio = /sys/fs/cgroup/blkio;
  freezer = /sys/fs/cgroup/freezer;
}

### Hierarchical RCs

group tagged/cave {
  perm {
    task {
      uid = root;
      gid = paludisbuild;
    }
    admin {
      uid = root;
      gid = root;
    }
  }

  cpu {
    cpu.shares = 100;
  }
  freezer {
  }
}

group tagged/desktop/roam {
  perm {
    task {
      uid = root;
      gid = users;
    }
    admin {
      uid = root;
      gid = root;
    }
  }

  cpu {
    cpu.shares = 300;
  }
  freezer {
  }
}

group tagged/desktop/java {
  perm {
    task {
      uid = root;
      gid = users;
    }
    admin {
      uid = root;
      gid = root;
    }
  }

  cpu {
    cpu.shares = 100;
  }
  freezer {
  }
}

### Non-hierarchical RCs (blkio)

group tagged.cave {
  perm {
    task {
      uid = root;
      gid = users;
    }
    admin {
      uid = root;
      gid = root;
    }
  }

  blkio {
    blkio.weight = 100;
  }
}

group tagged.desktop.roam {
  perm {
    task {
      uid = root;
      gid = users;
    }
    admin {
      uid = root;
      gid = root;
    }
  }

  blkio {
    blkio.weight = 300;
  }
}

group tagged.desktop.java {
  perm {
    task {
      uid = root;
      gid = users;
    }
    admin {
      uid = root;
      gid = root;
    }
  }

  blkio {
    blkio.weight = 100;
  }
}

So, what do I want it to look like? This:

path: /sys/fs/cgroup

defaults:
  _tasks: root:wheel:664
  _admin: root:wheel:644
  freezer:

groups:

  base:
    _default: true
    cpu.shares: 1000
    blkio.weight: 1000

  tagged:
    cave:
      _tasks: root:paludisbuild
      _admin: root:paludisbuild
      cpu.shares: 100
      blkio.weight: 100

    desktop:
      roam:
        _tasks: root:users
        cpu.shares: 300
        blkio.weight: 300
      java:
        _tasks: root:users
        cpu.shares: 100
        blkio.weight: 100

It's parseable and readable YAML, not some parenthesis-semicolon nightmare of a C junkie (you may think that because of these spaces don't matter there btw... well, think again!).

Configuration file above (YAML) consists of three basic definition blocks:

roam:
  _tasks: root:users
  cpu:
    shares: 300
  blkio:
    weight: 300
    throttle.write_bps_device: 253:9 1000000

Or just with one line per rc knob, like this:

roam:
  _tasks: root:users
  cpu.shares: 300
  blkio.weight: 300
  blkio.throttle.write_bps_device: 253:9 1000000

"defaults" section mirrors the structure of any leaf cgroup. RCs/parameters here will be used for created cgroups, unless overidden in "groups" section.

That handles the limits and initial (default cgroup for all tasks) classification part, but then chosen tasks also need to be assigned to a dedicated cgroups.

Since I knew exactly which (few) apps should be confined to which groups, I just wrote a simple wrapper scripts for each, putting these in a separate dir, in the head of PATH. Example:

#!/usr/local/bin/cgrc -s desktop/roam/usr/bin/firefox

Guess I'll direct the next poor soul struggling with libcgroup here, instead of wasting time explaining how to work around that crap and facing the inevitable question "what else is there?" *sigh*.

All the mentioned scripts can be found here.