---

[This post is part of our Designing Hardware for HD series. Be sure to check it out!]

In the last two posts of this series, I’ve dealt with the layouts required for a videophone. In these posts, I’ve shown the range of different layouts that are used in today’s video conferencing terminals, and I’ve even hinted that scaling (up and down) the video streams is often required to support the various layouts. I’d like to elaborate a bit further in this post on scaling.

In a way, scaling is a known issue. You’ve got an image, at a given resolution, and you need to either upscale it to fit a larger window resolution or to downscale it to squeeze it into a smaller area. There are several algorithms suitable for the task, and here are a few popular ones:

• Nearest neighbor filter
• Bi-linear filter
• Bi-cubic filter
• Polyphase filter
• Lanczos filter

While I’d LOVE to start explaining each of these in detail, I really can’t, as I have no clue as to the exact nature of each algorithm. I did, however, placed them in order – from the one that provides the lowest quality down to the one with the highest quality.

When you are debating on how to deal with scaling in your videophone, make sure you take into consideration the following guidelines.

1. What Algorithms?

I guess that’s an obvious question. There are various algorithms. You’d want the best possible one, but the better the quality the scaler algorithm provides – the more resources it takes. So you’ll probably end up compromising here.

My suggestion would be to use a Bi-cubic filter or better – it should be good enough for the task at hand.

2. Can I Downscale as Well as Upscale?

This one is an important issue. When you integrate a scaler, make sure you can use it for both downscaling and upscaling. Sometimes, the scaler will only be able to provide upscaling, which won’t help you in all cases.

For a video conferencing terminal you will definitely need both – at times you will need to take an image you captured using the camera, encode it in full resolution and at the same time downscale it to show in a small self-view window on the display. At other times, you will need to decode incoming video at low resolutions and upscale them to fit the whole screen.

Upscaling a low resolution QCIF image to fit a 4CIF window

So make sure you have both, and also check out the limits of the scaler: they usually scale up to a given resolution, and down to a given resolution; they often scale in limited ratios (for instance,  multiplies of 8), etc.

3. Preserving Aspect Ratio

If you’re doing HD video, then most of the time you’ll want to use 16:9 screen ratios. Problem is, some devices use a 4:3 ratio, so in some use-cases you will need to convert a video feed from one ratio into another.

Different aspect ratios

My advice here would be not to change the aspect ratio during the process, as the result will look plain ugly. So take into consideration the different video ratios when you plan layout sizes and scaling strategy.

Changing the aspect ratio of an image

4. How Many Scalers?

Go back to my previous posts about layouts. Check out which layouts you want in your product, and count the number of scalers you’ll need. It is important, as each scaler takes up its own resources. You’ll need to make sure you have enough of them around.

5. Software-based Scalers, or Dedicated Hardware?

Scalers come in different shapes and sizes. Some system-on-chip solutions come with hardware scaling capabilities – the DM6467 is one of those. If you can use such a hardware scaler, then it will be the best solution – it will be faster than anything else and will not eat up any of your resources. BUT – check the limitations, as these scalers will be the most limited ones.

If you don’t have any hardware scalers available, then you will simply need to get a software implementation. I guess that anyway you’ll probably end up using a combination of both hardware and software scalers – depending on the layout and video feed you will have in each use case. When doing so, start by allocating the hardware scalers first, and then add software to complement them.

6. The Camera

This is not really scaler specific, but is important to mention here. You will probably need the local camera video feed to be used twice – once to encode and send out and once to show locally. This also means that you will most likely have two different resolutions here, and it will require one or two scalers. If you can configure the camera on the fly to provide you with a given resolution, then you will need only one additional scaler for the local view. If you can’t – you might find yourself in need of two scalers.

7. Scaling Too Much

If you are upscaling, then there is such a thing as too much upscaling. Let’s say your video input is QCIF (176×144), common in mobile devices, and you want to place it on a 1080p monitor in full screen. The end result is going to look poor, no matter what scaler you use. My suggestion would be to limit the upscaling by some factor, and add borders to compensate.

Upscaling an image x3 in size and centering it without changing its aspect ratio

8. Am I Double-Scaling?

This is the last scaling issue, and I’ll handle this one with another example.

Let’s say I have a video feed of 480p (704×480), and you use 720p (1280×720) as your video output. But then the LCD TV you are using is 1080p (1920×1080). What will happen?  You will first upscale from 480p to 720p inside your system. Then the LCD monitor itself will upscale from 720p to 1080p. This is double scaling, and it will result in poorer quality than just upscaling directly from 480p directly to 1080p.

In the use-case I’ve given, you can probably change the video output you use from 720p to 1080p, which will complicate your system; Alternatively you can use a 720p LCD monitor.

There will be other use-cases, where you will find yourself double scaling, and not always for good reasons. Analyze the system to make sure you scale each of the video feeds as little as possible.