Suppose we want to run Firefox but restrict it to using only 50MB of physical memory. (Hey, times are tough!) In operating system terms, we want to limit Firefox to a 50MB resident set size (RSS). If Firefox tries to use, say, 100MB of memory, half of that will be paged out to disk.

On Linux, ulimit is designed to let you do precisely this. Unfortunately ulimit -m just doesn't work in newer kernels.

I thought that was my last hope, but sfink suggested that no, there is another: cgroups. I managed to get it to work on my Ubuntu 11.04 machine. Here's what I did:

You can theoretically move a running process into a group using cgclassify, but it didn't seem to do anything when I tried. Strangely, cgroup/memory/limited/memory.usage_in_bytes would contain a value within the desired limit, but /proc/PID/status would show an RSS way above the limit.