Multi Thread Me

In the movie “Super Size Me“, Morgan Spurlock goes from healthy to sick in one month of McDonald’s products. I’m reminded of this movie when I hear talk about adapting protocol stacks to use 64 core systems - doing that in the wrong manner, which usually means multithreading them out of their minds.

Why Multi-Core?

Multi-core processors are very reasonable. It’s a simple way of getting more processor power without raising the processor clock speed (generating more heat and consuming more power), and without using a server farm (requiring load balancing, and maybe even process migration, not to mention a whole new design). It makes even more sense when you consider servers that use virtualization to run different operating systems or providing many different services. In short, it works well when the different processors are mostly doing different things.

Why multi thread?

Multithreading is great when you have processes that are regularly inactive. For instance, some sort of garbage collector that wakes up every few seconds to clean up old transactions, or a thread that waits for system events. It also makes sense when you fear some processes will get stuck (known as “head of the line blocking“), and you want to be able to handle other events while that process is waiting. Usually, when you expect your program to smoothly execute a series of routines, there is no point in multithreading, as you will only pay an overhead for context switching, locking and inter-thread blocking. That is, until multi-processor-multi-core systems are in the picture. Then multithreading makes sense: getting better numbers.

The chart does not lie

Why am I being such a wet blanket? Because eventually, the demands of customers for increases in bottom line performance will cause SDK manufacturers to multithread beyond reason, damaging code stability and eventually reducing performance. The path to better utilization of multi-core processing is not by adding more threads; it is by adding threads in a sensible way: a thread per data flow (connection), for instance, or a model where different threads manipulate limited amounts of data, so that the system bus utilization is kept to a minimum. In addition, Anwar Ghuloum specifies the task granularity, the synchronization algorithm and task reuse as threading model performance influencers.

As Justin Rattner says: “multi-core performance standards needed“. Maybe other measurements should be added to performance tests, beyond memory consumption and transactions per second (see also memory efficiency), like a “performance per core”, or better yet, a “performance per core-MHz” column. Some researchers call this “performance scalability“.