According to this answer, which you may have already seen, there is a line that reads:
Note that a “share-able” page is counted as a private mapping until it
is actually shared. i.e. if there is only one process currently using
libfoo, that library’s text section will appear in the process’s
private mappings. It will be accounted in the shared mappings (and
removed from the private ones) only if/when another process starts
using that library.
What I would do to test whether you’re getting the benefits outlined in this article, is put a 10MB xml file as a literal string directly into your source code. Then, if you fire up 20 workers, you’ll be able to see if you’re using 200MB of memory, or only 10MB, as is expected with the new garbage collection feature.
UPDATE:
I was looking through the unicorn source and found a reference to this wonderful article.
To summarize, it states that in order to adapt your applications to take advantage of Ruby Enterprise Edition’s copy-on-write friendly garbage collector, you must set GC.copy_on_write_friendly to true before you fork.
if GC.respond_to?(:copy_on_write_friendly=)
GC.copy_on_write_friendly = true
end
Based on your provided unicorn configuration file, it appears to be missing the assignment.
Also, I enjoyed reading these related articles:
- http://merbist.com/2011/02/22/concurrency-in-ruby-explained/
- http://linux.die.net/man/2/fork
- http://linux.die.net/man/2/clone
According to the fork man page:
Under Linux, fork() is implemented using copy-on-write pages, so the
only penalty that it incurs is the time and memory required to
duplicate the parent’s page tables, and to create a unique task
structure for the child.Since version 2.3.3, rather than invoking the kernel’s fork() system
call, the glibc fork() wrapper that is provided as part of the NPTL
threading implementation invokes clone(2) with flags that provide the
same effect as the traditional system call. (A call to fork() is
equivalent to a call to clone(2) specifying flags as just SIGCHLD.)
The glibc wrapper invokes any fork handlers that have been established
using pthread_atfork(3).
And according to the clone man page:
Unlike fork(2), these calls allow the child process to share parts of
its execution context with the calling process, such as the memory
space, the table of file descriptors, and the table of signal
handlers.
So, I’m reading this to mean: linux’s fork copy-on-write, which is the feature that unicorn relies on to implement memory sharing, was not implemented until libc 2.2.3 (please, someone correct me if I’m wrong in this interpretation).
To check which version of libc you’re running, you can type:
ldd --version
Or, find glibc and run it directly. On my system it found the file at the following location:
locate libc.so
/lib/x86_64-linux-gnu/libc.so.6