About 15 years ago I gave an application performance presentation to some storage guys. Since I was a database jock at the time and didn’t “speak” I/O very well, I had to try a different approach. Rather than speaking over their heads in my own language, or faking it using their language, I tried to pick something neither of us were experts in – microeconomics. What’s interesting is that that presentation became my modus operandi for tuning systems. As I went from customer to customer, each with a different application or architecture, I needed a normalized approach so I didn’t get lost in the technology.
Apologies in advance to people who really know economics – I use microeconomics as a communication tool; this is not a strict interpretation…
Most people are familiar with the concept of supply and demand, and the impact of these forces on relative price. High demand and low supply for a desirable good or service drive prices up, while low demand and high supply drive prices down. Over the long term, as influences on supply and demand normalize, price equilibrium or “stabilization” occurs.
Which leads me to my simple mantra for performance management: If you want something to run faster, you can either ask for less stuff (i.e., decrease demand), or deliver results faster (i.e., increase supply). Whichever factor in this equation you have influence over dictates your next step. In other words: work on what you can fix.
There’s one catch: Supply doesn’t automatically equate to capacity. A bigger shoe does not make you run faster, just like a full tank of fuel doesn’t make you drive faster.
Here’s an example of a true supply-side issue. Note that all the illustrated virtual machines, processes, and services depend on the processors of a single ESXi host, and these processors can’t handle the load.
One impact of this supply-side shortage is a service slowdown. Notice the big delay in CPU…
But have a look at the processor metrics for the same time period. There’s still plenty of capacity here:
So supply can be thought of as capacity or speed, depending on the situation. To solve the first supply-side issue you need more processors. To solve the second supply-side issue you need faster processors.
But there are ways to solve such problems from the demand side of the equation. Reducing demand may make sense from a performance perspective, but who wants less? Think of any important, business-facing application: stakeholders always want to deliver more, not less. More traffic. More page views. Bigger shopping carts. More orders. Mo’ money, ya’ll.
So throttling down user demand is a big no-no. This means that, for performance purposes at least, your focus should be on the supply side, satisfying demand requests via the efficient execution of commands. Why make two trips when one will do? Why fetch a row 10 times when you can get all 10 in in a single go?
Check out this classic problem pattern in the image below. The issue is that, in order to service the user request, the application makes way too many calls to the database. Rather than being efficient, it creates unnecessarily high demand and results in a poorly performing application.
The last consideration in my efficient execution of commands approach to supply is that the approach needs to be applied recursively. For every tier, layer, service, or infrastructure component in your environment, review is needed to confirm that supply efficiently meets demand:
- Do you have enough supply (capacity) on the front end?
- Is the server side code that supports the front end efficient (demand)?
- Do you need faster (supply/speed) or more (supply/capacity) CPU cycles?
- Is your network healthy (demand/re-transmissions) or do you need additional bandwidth (supply/capacity)?
This model has served me well for years across a range of systems. Plan well, monitor often, deliver faster, and demand less.