[RndTbl] load throttling

[Trevor Cordes]

(A real question this time!)

I have a web (+ other tasks) server that sometimes does a lot of tasks in 
a short time that spike the load up to large values (like 30).  The tasks 
are all long jobs of lots (tens of thousands) of little (10/s) jobs.  I've 
written code to, between each little job, check the /proc/loadavg load (1 
and 5 min) and insert usleeps (.5+ secs) based on the loadavg (exponential 
relationship).  My goal is to keep the load < 2.0 on a 2 core system.

Anyhow, it seems to be working fine.  The weird thing is, at 2.0 loadavg, 
looking at top, I see the CPUs mostly at idle.  Not even very much wait 
going on.  Why would the load be 2 when both CPUs are mostly at 5-10% 
including wait?

I basically want to make sure the system doesn't get overloaded and can 
respond to non-batch request (like a web hit) promptly.  What loadavg 
should I target if not (1.0 * numcores)?  I guess I can arbitrarily pick 
numbers and see what happens, but I like things like this to make sense, 
and to me 1.0 * numcores makes sense, unless I'm not understanding loadavg 
properly.

Thanks!