CMOS Image Sensor Processing with FPGAs

An overview of the image sensor processing pipeline, including camera front-end IP implementation on a low-cost FPGA fabric, Bayer to RGB conversion and efficient use of memory.

By Suhel Dhanani, Senior Manager, DSP, Altera Corporation
and Dr. Andy Robertson, CEO, Bitec

Page 3 of 6
Video/Imaging DesignWire
(11/2/2009 2:00:19 AM)

Bayer to RGB (Bayer Demosaicing)
To convert an image from a Bayer representation to an RGB format, the two missing color values in each pixel need to be interpolated. Several standard interpolation methods (nearest neighbor, linear, cubic, cubic spline, etc.) have been proposed for this purpose (see T. Sakamoto, C. Nakanishi and T. Hase, “Software pixel interpolation for digital still cameras suitable for a 32-bit MCU,” IEEE Trans. Consumer Electronics, vol. 44, no. 4, November 1998). In this publication the authors have measured interpolation accuracy as well as the speed of the method and concluded that the best performance is achieved by a correlation-adjusted version of the linear interpolation. The two algorithms discussed are bi-linear interpolation and a more complex adaptive scheme.

Bi-linear Interpolation
R and B values are interpolated linearly from the nearest neighbors of the same color. There are four possible cases, as shown in Figure 5(a) to (d) for which the missing color components for the center pixel are required.

null
View full size

Figure 5. Bayer Patterns

When interpolating the missing values of R and B on a green pixel, as in Figure 5(a) and (b), the average values of the two nearest neighbors of the same color are used. For example, in Figure 5(a), the value for the blue component on a shaded G pixel will be the average of the blue pixels above and below the G pixel, while the value for the red component will be the average of the two red pixels to the left and right of the G pixel.

In Figure 6(a), the value of the green component is to be interpolated on an R pixel. The value used for the G component here is:

null

To determine the value of the red component on a B pixel in Figure 6(b), take the average of the four nearest red pixels cornering the B pixel.

null
View full size

Figure 6. Two Bayer Pattern Cases

NEXT: Advanced Bayer Conversion

Page 3: next page

Pages: 1 2 3 4 5 6