FotoNation announces "Dust-Map" dust removal solution

FotoNation has a new solution to a problem that plagues digital SLR owners everywhere: dust on the sensor. FotoNation's idea is to have in-camera (or not) software keep an up-to-date "dust map," which will automatically clean up the images by removing the marks it knows are there.

An interesting idea, and they actually have a prototype already running. Click on the headline of this article to read more information about this technology.

FotoNation develop 'Dust-Map' in-camera dust removal software

The presence of dust on the plate in front of the CDD of DSLR cameras is a common problem for both professional and amateur photographers.

The dust on the imaging sensor, as can be seen in the illustration above, is persistent and usually remains on the sensor unless physically removed. The only other way to remove the defects in the pictures is to manually correct them. Needless to say, this is a very time consuming activity. FotoNation® provides an automatic software solution for this problem.

FotoNation's approach is based on the idea that the camera maintains an up-to-date dust map. This map can be used to clean up images immediately upon capture (as an in-camera process). Alternatively, the map can be stored in the image files and the dust removal step can be integrated into the application software bundled with the camera.

The dust removal algorithm takes into account pixel information around the dust spot as well as various parameters of the camera and the lens. The software has a built-in intelligence to detect the changes in the dust position, changes in lens, focal length and aperture, and can determine when the camera is too dirty and needs to be cleaned physically.

The software takes less than 0.5 second to remove dust on a 6 Mega pixel image on a Pentium 1.6 GHz computer. The library is implemented as a portable ANSI-C library and designed for both embedded and desktop applications.

The technology is covered by 6 pending patent applications, and one granted patent with 1998 priority

Dust Map
The starting point for dust removal is a current dust map that can be associated with the captured image. The dust map describes the location, shape and size of the dust particles, amongst other parameters, and is used as the basis for the image reconstruction process.

The dust map is generated in the camera. The camera's intelligence verifies that the dust map is compatible with the current dust on the CCD. This is done with pixel-wise as well as context-wise decisions. When no dust map exists or when the camera determines that there is a need for creating a new dust map, the user is asked to take a calibration shot of a uniform region using a high f-stop setting, for example a picture of the sky or a blank wall. This calibration image is analyzed in the camera and a dust map is created from it. The dust map is stored in the camera memory. The memory size for such map is insignificant.

Dust Detection
As part of the detection and marking stage of the dust, which may include some verification of the dust map current state, the algorithm takes into account numerous parameters such as the camera and shooting parameters, including camera model, lens type, focal length, aperture and location of the dust relative to the CCD coordinates.

Reconstruction Algorithms
Several different reconstruction algorithms are implemented. The chosen dust removal method depends on some of the shooting parameters as defined above. In addition, dust specific information such as the location of the dust as well as the nature of the region of the image surrounding the dust is taken into account as part of the reconstruction stage.

Desktop image reconstruction
When an image is taken, the dust map is inserted into the image file header.

When the image is transferred to the PC the dust map is extracted from the header and is used in the reconstruction process. In this case, the image may be in RAW format or RGB based, such as EXIF format.
In-camera image reconstruction
The reconstruction process can also be implemented in the camera. This method is preferred and addresses a few quality and usability matters. Correcting the image in the camera avoids the need to store the dust map in an image file. It also prevents the exaggeration of the dust artefacts throughout the image conversion process as is illustrated in the figures below where the dust after conversion to RGB is substantially more pronounced and harder to correct.

The dust correction algorithm can be implemented as part of the image conversion. It is thus efficient in time, and allow the user to save JPG images that are already dust-free.
Camera and lens suuport
The algorithm depends on the optical characteristics of individual lens and camera models. FotoNation built a database that will cover most popular combinations. Upon implementation in a product, a more comprehensive database will be available for cameras and lenses.

Features:

Desktop or in-camera dust detection and removal algorithm. The algorithm is able to reconstruct the obscured area accurately
The software takes less than 1 second to remove dust on a 6 Mega pixel image on a Pentium 1.6 GHz computer. All calculations are fixed point, no need for a floating point unit
Support for RGB and raw mosaic image formats
Implements its own memory management for ease of integration
Minimal incremental memory and execution time cost as image size increases