CMOS Image Sensor Processing with FPGAs

An overview of the image sensor processing pipeline, including camera front-end IP implementation on a low-cost FPGA fabric, Bayer to RGB conversion and efficient use of memory.

By Suhel Dhanani, Senior Manager, DSP, Altera Corporation
and Dr. Andy Robertson, CEO, Bitec

Page 4 of 6
Video/Imaging DesignWire
(11/2/2009 2:00:19 AM)

Advanced Bayer Conversion
Albeit simple in computation, the bi-linear interpolation described above is prone to image blur and spurious hue corruption where abrupt color changes occur in the image. The images in Figure 7 illustrate this effect.

null
View full size

Figure 7. Bi-linear Interpolation of Bayer Image

In order to improve the resulting image quality an adaptive interpolation algorithm must be employed. Here, a 5×5 grid is used to estimate the missing pixel colors. Consider the Bayer pattern in Figure 8.

null
View full size

Figure 8. 5×5 Bayer Pattern Grid

First, a horizontal laplacian is determined using the following equation.

null

Secondly, a corresponding vertical laplacian is determined using the following equation.

null

Using these horizontal and vertical metrics the interpolation algorithm is adapted according to the following rules.

null

A similar algorithm is applied to the remaining mosaic colors. These rules determine a pixel location in which a color change has taken place and adapts the interpolation accordingly. The result of this adaptation on the output image quality can be seen below in Figures 9(a), (b) and (c). The adaptive algorithm greatly reduces the blur and hue effect.

null
View full size

Figure 9. Comparison of Bi-linear and Adaptive Algorithms

In order to execute the algorithm efficiently in an FPGA, careful computation pipelining is necessary. This is achieved to balance device resources with computation time. The resulting algorithm produces excellent image quality results while using minimum on-chip memory and logic elements.

NEXT:A Camera Front-End IP

Page 4: next page

Pages: 1 2 3 4 5 6