The effects of multi-core on SIP servers

[I’ve had a chat the other day with a colleague of mine, Danny Loeb, who is the Product Manager for our SIP Server Platform. As part of his work, he needs to deal with the multi-core trend and how that changes the architecture of his product.]

Multi-core is all the rage these days. Or at least this is what Intel tell us - almost every month Intel arranges a seminar, workshop, training or some other session about multi-core here in Israel.

Simply put, multi-core is the process of placing several CPUs inside a single silicon die and make them act as a single entity that can be utilized by the operating system and its applications.

Scalability before multi-core

Prior to the new multi-core trend, there were two ways to scale, either vertically or horizontally.

• Vertical scalability occurs when you rely on the strength of your hardware, combined with local performance optimization to gain scalability. The problem with this approach is that it is limited by the current hardware technology and costs increase exponentially when scaling. It is also the easier to develop.
• Horizontal scalability refers to the distribution of the solution among machines. Scaling up equals adding machines, which makes scaling closer to linear in its cost. The problem is, developing a distributed solution can be a complex task - this is why Google’s AppEngine is such an important announcement. To top it off, you will usually suffer on performance due to communication and synchronization requirements in this solution.

The Achilles heel in multi-core architectures

Until a year or two ago, programmers had an easy life. All you needed to do was upgrade, and the programs you were running usually started running faster. The simple reason for that is that CPU developments focused on pumping more clock cycles and pipelining the commands to the processor - both making newer CPUs able to process more commands in less time.

It was great while it lasted, but once you hit the invisible ceiling of improvements (physical transistor sizes, power consumption and heating), you reach a point where the ROI in terms of CPU upgrades, declines.

This caused both Intel and AMD to go for a different approach - adding more CPUs onto the same die, and now the race is who can get more cores on the same die.

Multi-core is great when you execute multiple processes, so now if you surf the web, and you’ve got firewall and antivirus software running at the same time, chances are you feel it less running on a dual core machine.

The problem starts once you want a single program to take the full use of a single machine with a multi-core architecture. If that program is single threaded (i.e. runs on a single core only), you are not getting all the juice you can. It also changes the way programmers need to design and write their applications, and this is why Intel is zealously trying to teach programmers how to use multi-core architectures. This mainly focuses on splitting applications into multiple threads that can run in parallel on different cores.

SIP Servers and multi-core

I’ll focus on pure signaling servers that don’t need to deal with media, such as various VoIP servers, IMS application servers, SIP proxies, etc - these deal with high traffic.

Taking it to the extreme, large scale servers need to be distributed among machines and clustered together as a single logical unit to the outside world no matter which processor you choose to run them on.

As multi-core gets into server designs, it is essential for VoIP servers to be able to utilize the full capacity of these chipsets. This means the current architecture used for servers needs to be changed in several aspects - you no longer have to choose to scale either horizontally or vertically. In multi-core, you need to do something in-between - you need to think of vertical scaling when you optimize your application and design your threads; and you need to think of horizontal scaling when you synchronize between threads instead of building a distributed solution.

1. Multi-thread on the protocol level

At the heart of your VoIP server there’s a shy protocol stack and a server framework to go along with it. Your multi-threading solution should begin at this level - incoming messages should be served by multiple worker threads. Otherwise, a bottleneck will be created at this low level and in scenarios such as peak communication congestion.

2. Thread only when needed

Worker threads or thread pools need to be viewed as resources that scale with the cores you have on your machine. You don’t want to multi-thread out of your mind. Empiric testing always suggests the number of threads in the application to be twice the number of CPUs you have. While I am sure this is probably close to being true for threads that eat up CPU or I/O resources, it might not be true for all kinds of threads in the system. Different threads might have different characteristics in terms of the tasks they do and the amount of workload they handle. You will need to map your threads and only then decide on the number of threads to allocate for the thread pools in your application.

3. Multi-thread on the business logic

While it is important that the underlying framework is multi-core-friendly, it doesn’t stop there. The application has to take multi-core seriously as well to work properly. This is important, for example, when handling incoming requests. This business logic needs to be multi-threaded as well, simply to provide faster response times to incoming requests.

4. Multi-XXX?

In the near future, you might see a new trend whereby the multi-core concept is used for other resources in addition to the CPU. These other peripherals can be networking interfaces, bus communications, etc; and these changes will affect server architectures.