CMOS Image Sensor Processing with FPGAs

An overview of the image sensor processing pipeline, including camera front-end IP implementation on a low-cost FPGA fabric, Bayer to RGB conversion and efficient use of memory.

By Suhel Dhanani, Senior Manager, DSP, Altera Corporation
and Dr. Andy Robertson, CEO, Bitec

Page 2 of 6
Video/Imaging DesignWire
(11/2/2009 2:00:19 AM)

Color Filter Arrays
There are many types of color filter arrays, with the most common being invented by Dr. Bayer of Eastman Kodak. This color filter array — or the Bayer mosaic as it is more commonly referred to — utilizes a filter pattern that is half green, only a quarter red and a quarter blue. This is based on the fact that the human eye is more sensitive to the green luminance, illustrated in Figure 3.

null

Figure 3. A Resultant Bayer Mosaic

Sensor Processing Signal Chain
A sensor processing signal chain is generally developed to:

  • Correct the pixel information from the CMOS sensor
  • Convert the Bayer image to RGB
  • Apply any color correction that may be required

Figure 4 shows the essential blocks of such a signal chain

null
View full size

Figure 4. A Sensor Processing Signal Chain

The Bayer to RGB conversion is the key standard function required in a sensor processing signal chain. This is also the most complex if implemented using a multi-tap filter.

Pixel correction and formatting is a function that is highly dependent on the image sesnor used, so this is generally a custom implementation. Color correction, while a simple function to implement in hardware, is complex as far as the determination of the correction factor.

Next let’s look at two implementations of a Bayer to RGB conversion algorithm. The operation of a design that brings together all of these functions to implement a complete image sensor processing chain will then be detailed, and this is verified and implemented in hardware with an Aptina CMOS sensor.

NEXT: Bayer to RGB (Bayer Demosaicing)

Page 2: next page

Pages: 1 2 3 4 5 6