Danny Loeb provided his insights on the effect of multicore on SIP servers, but what about video communication clients? Up until today, there is a tendency to either push multi-core hardware as the next best thing (mainly by the relevant chipset vendors) or to dismiss it for most uses. While that has been true up until today, it’s popularity is changing - the latest evidence being Skype version 4.0 beta release, which is video-centric.
There is a multitude of other indications that video communications is becoming the “next big thing,” both in the consumer market and for enterprises. I get a lot more requests from customers for video applications and most concern high definition. This move to high definition has made multi-core platforms essential - there’s no way around it (unless you are willing to settle for crappy video).
Why is multi-core that important? The answer is that there’s no other way to encode high definition video; at least not if you are planning something that is cost effective.
Ariel already gave his views on the bandwidth requirements of high definition at Video over Enterprise. There are also computing requirements that need to be taken into account: assuming we want to encode 720p at 30 frames per second, this would require to handle 1280×720 pixels per frame which gives us 27,648,000 pixels each second to process. That’s over 27 MILLION pixels to encode and another 27 MILLION pixels to decode. High definition is usually done using the H.264 standard (in order to lower the bandwidth required while keeping high quality) and H.264 is quite complex to compute.
Modern Video encoding techniques take each frame,split it into smaller blocks called macroblocks - usually in sizes of 8×8 - and then analyze and encode each such macroblock on its own. This allows for splitting an image in order to perform parallel processing. In other words, multi-core processors can be utilized in a fairly simple way: you split the pictures into regions and encode the regions in parallel. This is not trivial because sometimes the actual processing of a macroblock is dependent on previous images or its neighbor macroblocks (especially for motion estimation algorithms), but other than that, it is quite straightforward.
A frame split into 4 areas, each being processed by a different core processor
In today’s x86 architecture there is no way one can encode a high definition video stream in real-time - at least not if you want to maintain video quality and allow for 30 frames per second. Chip architectures move towards adding additional processing cores instead of increasing clock speeds. But programmers are still not embracing these changes in their applications. Jeff Atwood even points out the diminishing returns of using multicore. A video conferencing client, supporting high definition, really needs these multicores - there’s no way around it today.
Developing a video telephony client that is going to do high definition? You will probably use multicore technology in the underlying hardware sooner or later.
This post is better suited for our Video Over Enterprise blog, but this should be a good lesson to all of us in the VoIP market.
For the past several years, our company has been investing in VoIP infrastructure. Today, I am capable of making video calls and joining audio and video conferences from practically everywhere - from meeting rooms to my work station, my laptop and even my mobile handset. Or as Sagee would say - “The shoe-maker does not go bare-footed.”
Still, there are some things I have learned about what one should NOT do when deploying a video conferencing solution in an enterprise, at least from my past experience:
No complex dialing plans, please. Ordinary users do not make video calls every day. If they did, they might remember those complex dialing plans (a prefix for dialing multipoint video calls, a number for dialing out through a gateway, etc.). Presence based dialing can be a good solution here.
Take care of the bandwidth. Video calls take up bandwidth. A lot of it. Make sure you have enough to go around - especially between your corporate branches. Don’t forget to make sure it is reliable. Between offices, a private network or an MPLS is necessary to ensure sustained quality. Inside the office, a dedicated VPN is usually required in order to ensure file transfers on the network won’t interfere with the video and audio flows.
Don’t force all calls to be video calls. Allow users to select audio calls if they feel like it.
Don’t assume everybody wants to participate in a video call. Allow users to use audio only for incoming calls and the option to add the video later if they wish to (or remove the video at a later point during the call).
Don’t have an inclusive “logic” that prefers all calls to be video (or audio). My suggestion would be to make all calls from the same subnet/location/office/region/country audio calls and all “long distance” calls video calls by default.
Don’t assume that if you have “logic”, users don’t need options. Let users decide and change the defaults. The best way is probably to learn what the user is doing and change the “logical” behavior accordingly.
Don’t limit video to meeting rooms. People tend to join meetings from different places and not only participate from meeting rooms. In order for video to work for you, make sure you have it accessible from the desktops of employees and allow them to join video sessions from their homes or on the go as well.
Don’t make an island out of your video system. Doing video? Make sure it can do voice calls with regular phones as well. Connect it to mobile handsets. Connect it to your enterprise email and presence systems. In a word - Unified Communications. Well, that’s two words.
Don’t forget scheduling. When doing multipoint video calls, you should make the scheduling part a breeze. This is true for scheduled sessions and for ad-hoc video calls. The best way is probably to integrate with your calendar’s scheduler (be it Microsoft Office or IBM Lotus Sametime or other).
Give proper feedback. This one is tricky. You do a transatlantic call between several branches of your company, and lo and behold - video doesn’t work that well. And yet answering the simple question - where the weak link is - is quite hard. If different systems along the way can provide proper feedback to the lay man, it might actually work.
Bonus points for those doing video calls using their PC:
Don’t eat up the PC resources for video calls. You should remember that the person using that PC might want to do other things besides watching the incoming video, so clogging up his system is not a good move.
Leave the webcam alone. Not in a video call? Leave the webcam so that other applications may use it.
This is a move that is probably going to change (yet again) the landscape of the mobile market. It began with Apple’s iPhone and continued with Google’s Android OS.
Although this move can be attributed to this chain of events, as Ted from Signal to Noise points out, I think it is only the tip of the iceberg. There are other reasons for this move which are as important.
If we split this news into two activities, we can find the reason for each: the purchase of Symbian, and making it open source.
Nokia held around 53% market share of the whole smart phone market, where Symbian holds 67% of that market.
What does that tell us? That over 75% of the phones that use Symbian are Nokia phones (that will be well over 13 million phones in Q1 2008) - a lot more than Nokia’s piece of the Symbian pie. For an average of $4.1 per phone, they are paying to Symbian over 53 million dollars a year, which is then “distributed” among the various stake holders. In a sense, Nokia has been funding the other Symbian handset vendors it is competing against.
It is no wonder that purchasing the share of Symbian from the other handset vendors is the way to go. That having been said, there is the second piece of the puzzle - open source.
2. Why make it open source?
While Ted attributes the move to open source to Google’s Android OS, it seems that Om Malik places more weight on LiMo, giving them 60% likelihood to succeed while giving Google only 35%. I think there is another important aspect: maintaining an operating system is expensive. VERY expensive. To be able to overcome that, Nokia decided to tap the resources of the open source community - without doing so, taking full ownership of the Symbian OS would have been suicide.
Going open source was also the right move looking at the changing landscape of the mobile handsets. Just last week I have written about the migration from Windows Mobile to Mobile Linux.
It seems that in the last several years, the only business case presented by companies is advertising.
Search? Advertising.
Blogs? Advertising.
Calls? Advertising.
Entertainment? Advertising.
Although advertising is a good monetization technique for some, it is not a ‘one size fits all’ solution.
Let me first split the markets into two specific types: hunters and gatherers. I am not talking about prehistoric human societies, but rather about the concept of what we do in different situations.
Gatherers
As gatherers, we’re collecting - usually information. Examples of this include searching or reading blogs.
When searching I am already on the lookout for something - searching for it, gathering information. When reading blogs, I am usually reading specific blogs and blog posts to gain insights about a given market.
In both these activities, we skim through information, usually not trying to concentrate and focus on anything in depth. Nicholas Carr has insights about this behavior overseveralposts in his blog.
This kind of an activity is quite suitable for advertising. As the user is already searching, why not provide him with relevant advertisements? He is not focused on anything specific thing after all.
Hunters
As hunters, we’re obsessed on a specific target.
Hunting is like calling - you cannot do it in parallel to other activities - you either talk to someone on the phone or do something else. Whenever I type on my computer and talk to my wife over the phone at the same time, she notices it immediately and I get reprimanded on the spot.
Entertainment is also a hunting activity. If you are watching a good movie at the cinema, you are bound to get annoyed at that nasty mobile call that someone receives in the most thrilling scene. It is no wonder that ABI research found out that people using IPTV are very likely to skip advertisements: “Nearly four of five TiVo customers and over 82% of service provider DVR customers indicated they skip all or most commercials”.
In entertainment today, the only possibility is degrading to silent advertising techniques. Now that we’ve gotten used to advertising in gathering scenarios (mainly searching), who wants to digress back to TV commercials?
For those thinking of monetizing through advertising, it is important to note that it fits a small set of large companies who provide the infrastructure. As Stacey Higginbotham from GigaOm explains:
“Application companies have some ad revenue, but right now they’re kind of like cable channels for the web, while an ad platform is the means to a business model that supports that cable channel.”
Companies need to search for other monetization techniques, especially in the entertainment and telephony markets. Advertising fits best to search, and even then, you’d rather be the ad platform and not the companies using it.
Crunch Gear had an interesting post - somehow, they are expecting mobile IM to go mainstream next year. Somehow, this seems a bit farfetched. Mobile IM will take at least 2-3 years or more to become popular.
IM vs SMS
IM is usually seen as an SMS competitor. Taking this viewpoint, there is only a single reason why IM will become popular, which is also the reason it won’t. The reason is cost. People are used to paying high prices for sending SMS messages. With over two trillion messages predicted for 2008, this is an important revenue generator for mobile operators. IM is on the other end of the same scale - people are used to having free IM solutions on their desktops. Moving to the mobile domain, the expectations will be the same, and operators will have a hard time trying to get money for the service, especially when mobile IM clients from Yahoo!, Microsoft and others are already available for free.
This means that with a modest data plan from the operators, users will be able to get a free IM service and decide to ditch SMS. Such an approach will reduce the revenue of the operators and will place them more as pipe sellers than service providers. This means that operators have no incentive of providing IM services or promoting them - they will make them as hard as they can to deploy. Handsets today don’t come with an IM client preinstalled because of this reason - as most handsets are sold today through operators, operators simply don’t want such an application to be part of your phone.
People can still decide to download, install and use an IM client on their smart phones today, but they need to decide to do so. Getting to a critical mass for this service will take more than a year. SMS on the other hand is operational on all handsets worldwide - no installation needed.
SMS benefits
SMS has additional advantages over IM as well.
It works to every mobile phone number. No need to befriend the “person” you are contacting
It is used for micro payments - especially for donation purposes
It is used for TV voting
These services are not natural to IM. SMS is definitely here to stay.
What might change this schedule and bring it sooner to handsets? The iPhone of course. It is changing the way we use phones and the way handset vendors are developing them.
In the past several years, there were two main handset platforms for smart phones: Symbian and Windows Mobile. While Symbian enjoyed a large market share, Windows Mobile was starting to gain ground - especially because it was not owned by a handset vendor (Nokia owns 47.9% of Symbian). Linux was always around, but was never interesting enough. During the Asia road show I attended last month, it became obvious to me that Linux is set to displace Windows Mobile.
Handset ODM vendors are still developing and investing heavily in Windows Mobile based handsets - this can be seen by Microsoft’s expectations of 50% growth annually. On the other hand, ODMs are starting to show roadmaps based around Linux - simply because their customers, who are the handset brand vendors, are starting to look for such handsets. The reason is probably related to royalty costs of the operating system, but also to the emergence of stable mobile Linux platforms such as Android OS and LiMo.
Even Nokia, who uses Symbian in most of its handsets, has started using Linux in their N800 platform and even purchased Trolltech, who provide the well-known Qtopia platform.
What are the challenges for handset vendors moving to Linux?
The move to Linux is not an easy one. It is also a different kind of a challenge than other platforms.
On Symbian, the challenge is usually one of learning a new way to work with an operating system which is drastically different than Windows and Linux - to the point of having (crippled) C++ API set.
On Windows Mobile, the challenge is integrating the Phone Canvas UI technology.
For Linux the challenge is quite different - selecting the phone distribution/middleware. To date, there are several different Linux based distributions which are positioned as mobile handset ones:
LiMo
Google Android OS
Trolltech Qtopia
MontaVista Linux
VxWorksWindRiver Linux
More?
Each one is different in the types of services it provides “out of the box” and in the types of interfaces these services have. Android, as an example, has a Java based UI, while MontaVista comes with nothing that is handset-specific. MobileCrunch posted a good comparison between LiMo and Android - I suggest reading it.
When a handset vendor selects a distribution, this will affect his software architecture - not only for the UI but also for the multimedia system integration. This means that a handset ODM will need to select one Linux platform and stick to it for his customers - switching from LiMo to Android, for example, will require additional investment which doesn’t exist when using Windows Mobile for several customers.
This variance between distributions is going to be tricky for vendors. Those who select the wrong distribution will lose the market.
iPhone 3G is not out. As I have predicted, it doesn’t include any video conferencing technology. Instead of starting to analyze and explain what this new version is going to do to the market, I’d like to step back and look at what the first iPhone already did. Simply put, the iPhone has placed the ODMs and the handset vendors in a UI-frenzy.The iPhone came out as an innovative handset. Apple boldly decided not to try and compete in the same feature-list game of the other handsets and instead of going directly to 3G, went to a 2G phone but came up with two unique features: multi-touch screen and great web browsing. The end result? They won the market.
In all of my last engagements with customers, there is one thing that is evident: everyone today is trying to focus on the user interface. Features are not important. Stability is not important. Interoperability is not important. UI is.
The only problem I see here is that most are trying to copy what Apple already did - develop a 3D UI, add a multi-touch interface to it and viola - you have a phone. I am sorry to say, but the iPhone requires two hands to operate. You can’t just dial without looking at the screen. There’s no way to quickly skim through the menus to start writing an SMS message. While this is a good solution for some, I wouldn’t be happy with it. I want a handset that has buttons and menus on it - buttons that I can press without looking at the phone and menus that I can operate without having to think about. With my current mid-range Nokia 6120 handset, I can do these things. It is not the best of handsets and I have a lot of criticism about it, but one good thing I can say is that it is a working horse for me. So please - don’t develop an iPhone for me - I want a REAL phone.
The iPhone ODM Frenzy effect in action
As a friend of mine recently told me:
“In the 20th century, when SMS 1st came out in South Africa, I’d go to tech parties and IT journalists were staggering around, cell phone in one hand, a drink in the other, competing with one another to see who could accurately type out SMS’ without looking at the screen - this after having downed LOTS of Jack Daniels shooters! Try doing that on your iPhone!”
Handset vendors need to think about the users first. They tend to forget about them. Most handsets have complex user interfaces. I hope that this iPhone frenzy won’t make them do complex visual menus where the flexibility of the fingers is a requirement from the user. A multi-touch 3D UI is NOT important. The user experience is.
3G-324M can be found in over 450 million handsets worldwide. A software based protocol stack that is needed in every 3G handset, 3G-324M, however, has absolutely zero open source implementation? Because it is not IP based.
Commercial value
The open source model succeeds; not because people want to contribute and invest their time in free software; it is because corporations are finding a way to support such investments by getting their revenues indirectly - by selling maintenance, services, training, advertising or other techniques.
Most of these revenue generators tend to rely on a large, sharable computing platform - the internet. Once you provide an open source technology over the internet “platform,” you can complement it by selling a service (Software as a Service anyone?).
Low entry barrier
Open source resembles organic growth. People opt to invest their time and effort in it and they will place that investment in their own desires and needs. No exact roadmap exists, no real target. To be able to develop a successful open source project, there should be a real market need for it, with a low entry barrier. The open source community needs to be able to come up with a solution that can be used, tested, developed and deployed at a low cost.
This usually boils down to technologies that are IP-based, as an IP-based stack exists today in every operating system that values itself. Since this is the case, developing open source products over IP is relatively simple - developers can focus on what they are developing and not on building the infrastructure from scratch.
Back to 3G-324M
3G-324M lacks these two aspects.
There is a lot of commercial value in 3G-324M, but this is a closed system solution. It is a point to point protocol, with no servers in the middle that can be used to monetize the use of a protocol stack. There is no internet connectivity, no web browser involved, and no way to get any commercial value in an indirect way.
3G-324M also possesses quite a high entry barrier. It is not IP-based, but rather circuit switched-based. To develop it, people need to deal with low-level standards and real-time multiplexing (H.223 protocol). It is different in its mind-set than IP protocols and it requires a large number of tasks to be accomplished before getting anything useful:
Implement the low-level H.223 multiplexing
Implement the 3G-324M signaling and call control mechanisms
Find and integrate video codecs
Find and integrate audio codecs (AMR - no G.711…)
Find and integrate with a baseband
All of these tasks are not easy ones and require a lot of hard work. As there is no way to monetize on it besides building a full product, there are either in-house 3G-324M stacks or commercial 3G-324M stacks.
Last week, Paul Jones explained on VoIP Survivor the basic concepts behind AMS. The name might ring a bell to those familiar with IMS, but there is no real connection (besides the unfortunate use of a similar acronym). While IMS is an ongoing work, AMS is just beginning.
There are many people out there engaged today in one way or another with IMS. They come from different companies and participate in various organizations (IETF, 3GPP, IMTC, IMS Forum, TISPAN, Cable Labs, WiMAX Forum to name a few). The result is a system built around SIP and several other protocols that enables the deployment of communication services. To date, IMS is the only standardized IP-based communication solution that a service provider can select and deploy - it is also one he can monetize on. Anything else will be proprietary.
My guess is that this is also the reason why there are those who are against IMS - or for that matter, against any standardized phone system. I think a standard is necessary for the industry to grow, even if at a crawling rate. Proprietary solutions serve as great opportunities to leap ahead, and once this is done standardization can kick in.
AMS brings something new and fresh. It takes a good look at today’s networks and communication habits of people while also taking into account current standards and proprietary solutions, weighing their advantages and disadvantages, and then moving away from the requirements and hopefully get to the specification and implementation stages to bring a solution that is suitable for future needs without trying to stitch and patch current protocols.
Frens Jan Rumph was the first to compare AMS and IMS. He definitely nailed that charging and billing stuff. There are those who believe that IMS is a network designed to make money, while AMS is a network designed to provide services to users.
As AMS is still in its infancy, there is no real way of comparing it to IMS in any fair way. AMS is a chance at fixing future problems that will most definitely arise when current standards meet their limits.
MobileCrunch recently compared Android OS with LiMo. Both are viewed as Linux-based platforms for handsets. However, I think they missed a crucial point - Android OS is simply not Linux.I’d like to first fix the comparison table on MobileCrunch’s post based on the interesting comments found in that specific post.
Here is a list to summarize the differences:
An SDK will be available in LiMo “soon”. Probably later on this year. With Android having one, I am sure LiMo members are working in this direction as well.
Apps in Java. This is probably a sad story on behalf of Android. As there is no C/C++ API’s to talk about and a “Java-like” infrastructure is what exists on Android with a new UI framework, it is going to be hard for developers to use.
Handset support was missing. While some of the Android’s OHA members are handset vendors, to date only HTC committed to the platform. With LiMo, there are a lot more vendors. The reason for that is simply because they view LiMo Foundation as more open than OHA.
Of all these, there is one that stands out the most - the Java part. Linux is a great platform. Applications in Linux are written in a myriad of languages, but at the heart of it there is C/C++ code. This includes the many C system calls in Linux as well as a very rich infrastructure of libraries and development tools - all of which are simply inaccessible to Android developers. Unless Android will provide access to the C/C++ level, it will not matter if it is written on top of Linux, Windows Mobile, Symbian or even Commodore. As the operating system and its ecosystem is buried deep inside Android - who will really care?
It doesn’t matter if Java is faster or slower, or even if it’s J2EE, J2ME or Google’s Java flavor - it is simply not Linux. Starting to develop on Android is going to be similar to migrating to Symbian. You start by relearning everything you already know just to write your applications. This is not a good starting point for a new platform. The only reason why Android may succeed is due to the strength of Google.
Modern Video encoding techniques take each frame,split it into smaller blocks called macroblocks - usually in sizes of 8×8 - and then analyze and encode each such macroblock on its own. This allows for splitting an image in order to perform parallel processing. In other words, multi-core processors can be utilized in a fairly simple way: you split the pictures into regions and encode the regions in parallel. This is not trivial because sometimes the actual processing of a macroblock is dependent on previous images or its neighbor macroblocks (especially for motion estimation algorithms), but other than that, it is quite straightforward.
For the past several years, our company has been investing in VoIP infrastructure. Today, I am capable of making video calls and joining audio and video conferences from practically everywhere - from meeting rooms to my work station, my laptop and even my mobile handset. Or as Sagee would say - “
Let’s look at some facts:
