---

I’ve been writing about innovation here for a long time. Last week I discussed (more so – complained about) the lack of innovation in the video conferencing experience in the past few years. Well, in this post I will be much more optimistic, and tell you a bit about the state-of-the-art in video conferencing innovation, a project that RADVISION is proud to take part in: 3DPresence.

Einat Yellin, who leads the 3DPresence development effort in RADVISION, was more than happy to share with me the ins and outs of the project.

Telepresence?  Try 3DPresence!

I’ve already discussed here the long-time problem of the “camera-on-top-of-a-TV-set” video conferencing systems, which failed to provide a viable alternative to physical travel. They were replaced by high-end solutions, such as Telepresence, which partially removed some of the shortcomings.

Still, even in the most sophisticated and expensive systems some problems remain unsolved: systems are expensive, systems are not easily scaled, but most of all – experience as they may be – they do not really provide a 3D life-sized representation of the remote party, and eye contact and gesture-based interactions are met in a very limited way. The current state-of-the-art is impressive, but still fails to provide a “natural” impression of the remote conference participants. This is exactly what the 3DPresence projects aims to achieve.

The 3DPresence project was initiated by a consortium of five high-quality partners, bringing together a well-balanced and diverse mixture of industries, research and higher education institutes. The project is part of the Seventh Research Framework Programme of the European Union (FP7).

3DPresence In a Nut-Shell

What 3DPresence proposes is a multi-party, high-end 3D videoconferencing system, that will be able to transmit the feeling of physical presence in real-time to multiple remote locations in a transparent, natural way. This involves accurate representation of physical presence cues, such as multiple user (auto) stereopsis, multi-party eye contact and multi-party gesture-based interaction.

In order to provide the experience of “presence”, one of the project goals was to find out what does it really mean to “be there”. One of the key elements found was that the physical (geometrical) properties of the rest of the participants should remain as true as possible. This requires 3D representation.

The evolution (some may say – revolution) from 2D to 3D communication is one of the key components of a true tele-presence system, but it is not enough. Keeping eye contact and reproducing the correct gesture perspectives are both crucial for maintaining the non-verbal portion of inter-personal communication.

Even though the 3DPresence project is a research project, it has down-to-earth objectives. The main objective was that the system should be set in a conference room with geometrical restraints. The result, however, is quite different than what you would expect (and learned to know):

3DPresence Multi-Party Video Conferencing Concept

As shown in the figure above, the system consists of 4 displays for each “end point” (2 per external site). The system will allow up to two participants per “end point” and up to 3 “end points” in a conference.

Left: the “virtual” setup of 3 parties (6 participants).
Right: the “physical” setup with 2 local participants and 4 displays

The unique set-up allows maintaining eye contact and gesture awareness, as the displays render two different views of the same remote user per display.

Multi-view and Multi-display

As you may have guessed, all of the above requires unique “features” on the display side. The display for the 3DPresence system, built by Philips, one of the consortium members, is key for the project. Without boring you with too many details, the display is “multi-perspective”, which allows it to display different “views” simultaneously to different viewers, depending on their viewing angle. In more technical terms, it is using a set of optical lenses that direct different pixels of the monitor to different viewing angles.

While this is very impressive (just think about yourself watching your favorite soap, while your wife is watching the big Baseball game at the same time using the same screen…), the big challenge when using this technology for video conferencing is to allow participants to move freely in front of the screen, while preserving good picture quality (not switching views too often, refraining from cross-talk of views).

The images above, taken from a Philips demo clip, show how 2 perspectives can be viewed on the same screen at the same time: on the left a “left” participant feels he is being “watched”, while the “right” participant on the right feels the girl is looking away:

Fiction or Reality?

The 3DPresence project may look like science fiction, but it is more of a reality than you may think. RADVISION already hosts a 3DPresence room fully capable of supporting this technology, and last week, in a 3D workshop hosted by HHI Fraunhofer in Berlin, a live demo was presented, connecting the 1^st ever 3DPresence conference.

Live demo of a 3DPresence conference, as captured last week in Berlin

The project will end in Mid 2010. Until then, it will focus on enhancing quality and efficiency of the different algorithms involved, while working on interoperability with existing “legacy” systems (2D).

Although this sounds just around the corner, Einat clearly mentions that all the technologies involved – 3D video acquisition, display, coding and transmission – are still relatively new, and there’s a long way to go from prototype, impressive as it may be, to a commercial product.

Still, this innovation may lead the way to an as-close-to-real-life-as-possible video conferencing experience, which may offer a real alternative to in-person meetings without any drawbacks. When thinking about where our industry can, and should, go, I would really bet my chips on Einat and the 3DPresence project.