linux - High load average, when should I be worried?

I have a server which runs a few hundred processes simultaneously, most of them are idle, it is some sort of web crawler it sleeps between requests for various reasons.

So as a result, my load average is usually something like: 21.64, 27.05, 29.16

That's very very high right? But everything runs smoothly!

And my CPU consumption is something like (mpstat 60 1 output):

11:07:06 AM  CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle

11:08:06 AM  all   34.82    0.00    4.16   10.70    0.00    0.31    0.00    0.00    0.00   50.01
Average:     all   34.82    0.00    4.16   10.70    0.00    0.31    0.00    0.00    0.00   50.01

So, since I'm not even running at 100% CPU usage I feel like I do not have a reason to be worried, or am I missing something? There is a slight delay when nginx is serving requests, but that's expected given the large number of queued requests, But I read somewhere that a load average higher than 1 is a cause for alarm, and I honestly don't see why that is.

So please advise.

Thanks