Setting up cgroups on Ubuntu 14.04

May. 29, 2014

hljs.initHighlightingOnLoad();

This is a summary of what you can do in order to get Cgroups working on Ubuntu 14.04. Some init scripts have been modified in order to get the userland tools up and running. This guide will make you install the dependencies, place the necessary configuration files, making changes to the init scripts of the userland tools for cgroups.

Install dependencies:

apt-get install cgroup-bin cgroup-lite libcgroup1

cgconfigparser is the program that needs to run in order to set up the configuration put in cgconfig.conf. this program does not start at startup, but it should (create a init script or place in cgroup-lite config script, a sample is located in this post) cgroup-lite service needs to run for cgconfigparser to run correctly cgroup-lite is responsible for mounting the cgroups. It is also possible to manage cgroups with cgexec(1) and mkdir(1), this article wont cover that, but configuration of cgconfig.conf and cgrules.conf

service cgroup-lite start cgconfigparser -l /etc/cgconfig.conf

cgrulesengd needs to run in order to enforce cgroups policy, it is not started automatically. A init script for cgrulesengd should be created. Start the cgrulesengd with this command:

cgrulesengd -d -f /tmp/debug.log

Cgrules sample:

# cat /etc/cgrules.conf # /etc/cgrules.conf # #Each line describes a rule for a user in the forms: # # #: # #Where: # can be: # - an user name # - a group name, with @group syntax # - the wildcard *, for any user or group. # - The %, which is equivalent to "ditto". This is useful for # multiline rules where different cgroups need to be specified # for various hierarchies for a single user. # # is optional and it can be: # - a process name # - a full command path of a process # # can be: # - comma separated controller names (no spaces) # - * (for all mounted controllers) # # can be: # - path with-in the controller hierarchy (ex. pgrp1/gid1/uid1) # # Note: # - It currently has rules based on uids, gids and process name. # # - Don't put overlapping rules. First rule which matches the criteria # will be executed. # # - Multiline rules can be specified for specifying different cgroups # for multiple hierarchies. In the example below, user "peter" has # specified 2 line rule. First line says put peter's task in test1/ # dir for "cpu" controller and second line says put peter's tasks in # test2/ dir for memory controller. Make a note of "%" sign in second line. # This is an indication that it is continuation of previous rule. # # # # #john cpu usergroup/faculty/john/ #john:cp cpu usergroup/faculty/john/cp #@student cpu,memory usergroup/student/ #peter cpu test1/ #% memory test2/ #@root * admingroup/ #* * default/ # End of file frank:/home/frank/mem-limit memory limitgroup/ #CAN ALSO BE frank memory limitgroup/ #for all commands

The file /etc/cgconfig.conf

group limitgroup { perm { admin { uid = root; gid = root; } task { uid = 1002; gid = 1002; } } cpu { cpu.shares = "768"; } memory { memory.limit_in_bytes = "30000000"; } }

Now start cgrulesengd with debug and see that everything works as expected

root@vagrant:/etc# cgrulesengd -d -f /tmp/hei.log tail /tmp/hei.log CGroup Rules Engine Daemon log started Current time: Thu May 7 13:16:35 2015 Opened log file: /tmp/hei.log, log facility: 0, log level: 7 Proceeding with PID 5036 Rule: frank:* UID: 1002 GID: N/A DEST: limitgroup/ CONTROLLERS: *

Create the cgrulesengd.conf file in /etc/init/

root@vagrant:/etc/init# cat cgrulesengd.conf # cgrulesengd description "cgrulesengd" author "Serge Hallyn <serge.hallyn@canonical.com>" start on started cgroup-lite stop on stopped cgroup-lite pre-start script test -x /usr/sbin/cgrulesengd || { stop; exit 0; } end script script # get default options OPTIONS="" CGRED_CONF=/etc/cgrules.conf if [ -r "/etc/default/cgrulesengd" ]; then . /etc/default/cgrulesengd fi # Don't run if no configuration file if [ ! -s "$CGRED_CONF" ]; then echo "Cgred unconfigured" stop exit 0 fi # Make sure the kernel supports cgroups # This check is retained from the original sysvinit job, but should # be superfluous since we depend on cgconfig running, which will # have mounted this. grep -q "^cgroup" /proc/mounts || { stop; exit 0; } exec /usr/sbin/cgrulesengd --nodaemon $OPTIONS end script

Now create the init script

root@vagrant:~# cat /etc/init/cgroup-lite.conf description "mount available cgroup filesystems" author "Serge Hallyn <serge.hallyn@canonical.com>" start on mounted MOUNTPOINT=/sys/fs/cgroup pre-start script test -x /bin/cgroups-mount || { stop; exit 0; } test -d /sys/fs/cgroup || { stop; exit 0; } /bin/cgroups-mount /usr/sbin/cgconfigparser -l /etc/cgconfig.conf end script post-stop script if [ -x /bin/cgroups-umount ] then /bin/cgroups-umount fi end script

Resources and further reading:

http://linux.die.net/man/5/cgconfig.conf http://linux.die.net/man/5/cgrules.conf http://tuxion.com/2009/10/13/ubuntu-resource-managment-simple-example.html https://www.kernel.org/doc/Documentation/cgroups/ https://sysadmincasts.com/episodes/14-introduction-to-linux-control-groups-cgroups http://www.gen.cam.ac.uk/local/it/projects/ubuntu-cgroups-and-trying-to-stop-users-making-a-system-unusable http://docs.oracle.com/cd/E37670_01/E37355/html/ol_use_cases_cgroups.html http://devinhoward.ca/technology/2015/feb/implementing-cgroups-ubuntu-or-debian https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/sec-cpu.html http://blog.hintcafe.com/post/60223405371/resource-limiting-using-cgroups