[RndTbl] linux not caching files very long

[Trevor Cordes]

On 2012-11-21 Gilbert E. Detillieux wrote:
> You might also be interested in this article...
> 
> http://www.admin-magazine.com/Articles/Tuning-Your-Filesystem-s-Cache/
> 
> Specifically, the part about the vmtouch command and the "-dl" 
> (daemonize and lock) option.  It would allow you to lock your large 
> file(s) into memory while you work on them, I think.  That way, other 
> system activity won't evict your cached pages.

That's good to know for sure, but in this case my files are in
thousands of little files (maildir emails) so locking certain files
isn't going to help.

I finally had a chance to poke around more and I think I found the
solution!!

/proc/sys/vm/vfs_cache_pressure

Defaults to 100 (what that value means appears arbitrary).  I set mine
to 60 and all of a sudden my files are staying cached nearly forever
like they used to.  There *must* have been a change to either the
default setting or what the number means to the kernel around 3.3 or
3.4.  It's like night and day.

Probably the best way I found to illustrate and explain this problem is
that back in 2.6 and 3.0-3.1-ish days my 8GB RAM would show almost
nothing free, and massive (like 5GB) cached files (using top as the
investigative method).  I noticed with 3.4 that I'd always have around
1.5-2.0 GB free, and cached files would sit between 1.0-2.0.  My
working set (all normal apps) must be closer to 3 or 4GB of files.

With vfs_cache_pressure set to 100 in 3.4 it must be dropping cached
files to try to leave a lot of open mem for apps.  But it's nutso about
it, as why would I ever need 2GB free mem at all times?

vfs_cache_pressure to 60 and now my free mem sits around 400-500MB,
which is much better, with around 4GB file cache.

So anyone who works with tons of GB of files they want cached and
upgrades to 3.4+ and notices a huge slowdown, set your
vfs_cache_pressure to 60(ish) and your swappiness to 80(ish).  (I
haven't had to go to swappiness 100 yet.)