Resident Set Size (rss) Limit Has No Effect

The following problem occurs on a machine running Ubuntu 10.04 with the 2.6.32-22-generic kernel: Setting a limit for the Resident Set Size (RSS) of a process does not seem to have

Solution 1:

You can accomplish this using cgroups. The long version is on my blog, but the short version (tested on Ubuntu 11.04) is:

• Install the cgroup-bin package.

• Edit /etc/cgconfig.config and create a group with limited memory. For instance, I added:

  group limited {
    memory {
      memory.limit_in_bytes = 50M;
    }
  }
  

• Run

  $ sudo restart cgconfig
  $ sudo chown -R jlebar /sys/fs/cgroup/memory/limited
  $ cgexec -g memory:limited your/program
  

I observed my process with an RSS of 93M when I asked it to use only 50M, but that wasn't a problem for me, since my goal was just to get the program to page.

cgclassify lets you attach restrictions to a running process too. Note for RSS this only applies to memory allocated after the restriction comes into effect.

Solution 2:

Form the getrlimit manpage:

RLIMIT_RSS
Specifies the limit (in pages) of  the  process's  resident  set
(the  number of virtual pages resident in RAM).  This limit only
has effect in Linux 2.4.x, x < 30, and there only affects  calls
to madvise(2) specifying MADV_WILLNEED.

It seems this is just not supported on Linux kernel 2.6.

Solution 3:

A related limit - virtual memory or address space (RLIMIT_AS) - does work. This allows limiting the python process and subprocesses memory without external tools.

>>>size = 50*1024*1024# In bytes>>>resource.setrlimit(resource.RLIMIT_AS, (size, resource.RLIM_INFINITY))>>>a = 'a' * size
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
MemoryError

From the man page:

RLIMIT_AS. The maximum size of the process's virtual memory (address space) in bytes.

Here is a good explanation of the difference between the Resident Set and the VM size - What is RSS and VSZ in Linux memory management.

Solution 4:

I created a script to limit memory usage using cgroups and cgroup manager, useable for ad-hoc commands and not needing root privileges. See https://unix.stackexchange.com/questions/134414/how-to-limit-the-total-resources-memory-of-a-process-and-its-children/174894#174894