What Happens To Application Performance When CPU Utilisation Increases?

There was an interesting question on Oracle-L the other day about system performance when CPU utilisation goes up towards 100%:

"I'm not a hardware guy or sys admin person so forgive me if this is a stupid question. Leaving out all other variables(such as IO), should I expect performance to be the same in a databse if the server it is riding at is at 90% cpu usages as opposed to 10%? since there would still be spare cycles? Or is there a declining returns as you get closing to the maximum available cpu usage?"

Cary Millsap actually answered in the end and came back with this answer:

"Good question actually. It depends on how many CPUs are in there. If you have a copy of "Optimizing Oracle Performance" handy, check out table 9-3 on page 260. What it says is that you can expect performance to vary unpredictably on a 1-CPU machine when CPU utilization exceeds 50%. On a 2-CPU machine, the magic number is about 57.7%. On a 4-CPU machine, it's 66.5%.

The table lists "knee in the curve" numbers for CPU counts = ranging from 1 to 128. The whole section called "Behavior of M/M/m Systems" (pp254-266) should = be of interest to you."

From looking through the relevant pages in Cary's book, it appears that, once you get to the level of CPU usage specified for each CPU count (50% for a single CPU, 57.7% for 2 CPUs and so on) average response time not only goes up dramatically, but the variation in response time differs greatly as well. The chapter also answers the question about whether it's better to get a one single fast CPU, or two (or more) slower CPUs - the answer being that it depends on the nature of your system; single CPU systems are good for dealing with applications with low request 'arrival rates', and in these circumstances will give superior performance. If your application typically has lots of concurrent users, with lots of 'arrivals' all jockeying for CPU, multi-CPU systems are usually faster as they reduce the amount of 'queueing' for CPU resources. All interesting stuff.