---

There’s a trend these days to employ Forward Error Correction (FEC) as a means to improve media quality in voice and video calls. It is also something that RADVISION has been actively pushing at the ITU-T – and will probably be added in one of the next ITU-T meetings as H.323 Annex I.

When we started working on our SVC solution, and had to decide where to place the error correction data, we debated on a very basic question: At what level should FEC be employed?

I’d like to list the options we’ve debated over, but first, let me try to explain a bit about the different types of error handling mechanisms common today.

Error Handling of Real Time Media

Let’s start with a short explanation of the different error handling methods that exist for real time media:

1. Ignore and conceal – the most basic one is simply to ignore such errors. When you lose a media packet, you simply skip it. At times, you might try to conceal it:
   1. In audio by playing back the previous packet with reduced volume, for example.
   2. In video by asking to refresh the video using a fresh I-frame and freezing the video until then, for example.
2. Ask for a retransmission, which may be considered irrelevant, as by the time it takes the retransmission to arrive it will be too late. That being said, there are scenarios where it might make sense.
3. Add error correction in advance – by adding redundancy to the media being sent, you can let the receiver deal with a certain amount of packet loss.

Here’s a short table to give an outline of the differences between these methods:

	Concealment	Retransmission	FEC
Signaling	None	Additional signaling for the request and response of retransmissions	Negotiate FEC-support at beginning
Bitrate overhead	None	Only when retransmissions occur	Considerable overhead on data
Sender	No extra work	Wait for retransmission requests and handle them	Send redundant data
Receiver	Try to conceal when necessary	Know when to ask for retransmissions	Use redundant data when it makes sense
Location of mechanism	Sometimes the codec, but mostly the media system	Media system, with a signaling protocol above (RTP or SIP for example)	Either the codec or the media system (=RTP)

As you can see, the various techniques are very different from each other, and the mechanisms employed can be located at different places in the architecture.

For FEC, the options will usually be placing it inside the codec itself, or on the RTP level – the protocol used to send media packets.

Codec or RTP Level FEC?

We had to debate this question while working on our H.264/SVC codec. If you want the details on our error protection mechanism there, I recommend our whitepaper dedicated to this technology. The idea behind the mechanism we devised is quite simple: we rearrange the video frames in a way that makes it easier for us to “mark”  which are “important” , and all that is left is to add the redundancy data (=FEC) on top of these frames to get a better level of protection and a higher quality video.

The debate then was whether we wanted to add that redundancy at the codec level, as proprietary data that would sit within the encoded bitstream itself and be ignored by H.264/AVC decoders, or to move it up to the RTP level, sending it as RTP packets that would get tagged as FEC on a higher level.

Our decision was to move the mechanism to the RTP, and utilize a standard to do that – RFC 5109. We had several reasons for this decision:

• It will be easier to base it on existing FEC standards and enhance them as necessary.
• It will be easier to interop with our solution when others follow suit and start implementing SVC.
• We will be able to use the same mechanism to protect other codecs in the future – both audio and video.
• It will be easier to change the characteristics of error protection scheme used on the fly during an ongoing call, as all is done on the signaling protocol and not within the codec itself.
• It will be easier to implement and port faster between our product lines – codecs are notoriously hard to port.

We knew it would take a bit more work to integrate it at that level, but in the long run it was definitely worth the trouble.

I think that the reasons above are quite universal when trying to decide what goes into a codec and what can be done on the media system instead. My own personal feeling is that whatever can go into the media system without lowering the quality and functionality of the end result is worth doing on that level.