Smartphone camera performance: What does the sensor's megapixel count really tell you?

Digital camera and smartphone marketing often highlights the number of pixels on the camera sensor on the basis that big numbers are good, and huge numbers are even better. The resolving power of the lens is never mentioned -- indeed, for smartphones the camera lens is rarely mentioned at all. Handset manufacturers frequently source sensors and lenses separately and, since image sensors are relatively cheap compared to good-quality lenses, it's not uncommon to find a high-resolution sensor coupled with a lens of lower resolution. In which case, the resolution of the camera is limited by the lens and the true resolution of the camera is lower than the advertising would have you believe.

Resolution

Resolution is the ability of any imaging device to show sufficient contrast between the elements of fine detail in an image, so that these details can be seen and don't just appear as a blur. A pattern of alternating black and white lines is often used as a reference when measuring and specifying resolution, and this is given as resolvable line pairs per unit of measurement -- usually line pairs per millimetre (LP/mm) -- at a certain level of contrast between black and white.

Analogue to digital sampling

The requirement, according to Nyquist sampling theory, when converting between analogue and digital is that the digital sampling should take place at least twice the maximum analogue frequency. This means that, as far as the sensor is concerned, resolvable image detail across the area of a camera sensor is really one quarter of the number of available megapixels.

Typical lenses and sensors

Perhaps the most important factor for a lens is its diameter. Even if all other aspects of the lens are perfect, the diameter places a limit on resolution and on how much light the lens can gather, with larger lenses having higher resolution and gathering more light. Typical digital camera lenses range from around 10mm up to 50mm, or larger, while smartphone cameras are much smaller, usually around 2mm in diameter. Despite their generally small lens size, smartphones are often fitted with camera sensors with high pixel counts: 20-megapixel sensors such as the Sony IMX220 are not uncommon, giving the impression of the ability to produce highly detailed images.

f-stops

Traditional camera lenses usually have a variable, mechanical aperture, diaphragm or iris, which allows the photographer to control the amount of light passing through the lens. It also allows creative use of wide variations in depth of field -- one of the most distinctive features of film and something that many video cameras, with their relatively small sensors and limited optics, find hard to mimic. This explains the popularity of dSLR cameras with video capabilities, because their larger sensors and interchangeable lenses allow shooting over a range of f-stops with all the depth-of-field control of traditional film-based movie cameras.

The majority of smartphone cameras have a fixed aperture (the lens is 'wide open') to allow the maximum amount of light through the relatively tiny lens, but for a camera lens with a variable aperture, the diameter of the iris becomes the limiting factor. Photographers refer indirectly to aperture diameter by the 'f' number, where f is equal to the focal length divided by the aperture diameter. Typical smartphone cameras have an f of around 2.0 and a focal length of 3mm to 5mm. For example, an f of 2.2 and focal length of 4.8mm for the Meizu MX4's rear camera gives an aperture of 2.18mm diameter; BQ's Aquaris E4.5 has a shorter focal length of 3.5mm and an f of 2.4 for an aperture of 1.46mm.

Each decrease in the f-stop marks on a camera lens doubles the area of the aperture and therefore doubles the amount of light passing through the lens. The seemingly odd graduation of the f numbers is due to the area of the aperture being proportional to the square of the radius. Since f is a ratio or 'dimensionless' number, it imparts the same information regardless of the lens being used.

Photographers use the term 'stopping down' when adjusting to a smaller aperture with a larger f number. Small apertures result in images that are in focus over a large range from foreground to background (a wide depth of field) and because of this, may appear to have greater clarity. In fact, a lens will produce lower resolution when stopped down, compared to the in-focus portions of images from a wide-open lens, although the depth of field will be narrower. However optical aberration, which tends to increase towards the edge of a lens, can reduce the clarity and work against the resolution increase gained by opening up a lens.

At their highest f numbers, where the aperture is closed down as far as it will go, even market-leading dSLR cameras, with their companion default lenses, tend to have a lens resolution that is 9 times poorer than their imaging sensor. At the other extreme, with the aperture wide open, lens resolution might be 9 times better than the sensor, at least as far as diffraction is concerned.

Lens performance

Real (imperfect) lenses are usually evaluated using Modulation Transfer Function (MTF) plots that show the overall performance of a lens from its centre to its edge, and take account of both diffraction and optical aberration. However even the most perfect lens is limited in its resolving power by diffraction, which occurs when light passes through a small aperture to fall on an image plane such as a camera sensor. The light spreads out in a hemispherical wave beyond the aperture, and as it does so interference effects cause light and dark rings in the image, resulting in blurring. In this case, lens resolution is said to be diffraction limited.

Before reaching the diffraction limit, the resolution of a real camera lens may be aberration limited, although optical aberration of some of the best lenses available today is low enough that diffraction does set the limit on performance. The point here is that, while optical aberration can be minimised by good design and construction, the diffraction limit, which is directly related to lens diameter (/aperture), places an absolute physical limit on lens performance.

Calculating lens resolution

Fortunately, the resolution limit of a smartphone camera lens (or any lens) due to diffraction can be quite easily calculated from a piece of information that many manufacturers do supply: the f number for the lens.

The above formula for Dx assumes a white-to-black contrast of 15 percent (see the Rayleigh criterion or limit). As the formula shows, the resolution limit due to diffraction varies with wavelength (λ), so the resolution for violet (390nm wavelength) is better than it is for red (700nm wavelength). The following resolution graph is plotted for 500nm.

Lens-less cameras

Given that lenses are expensive, with the price rising rapidly for higher performance, and that physics imposes a size-related performance limit, very low-cost imaging sensors might perhaps be achieved by doing away with the conventional lens altogether.

The principle of a camera without a lens has been known for over 2,000 years in the form of the camera obscura (from the Latin for 'dark room') and later portable pinhole cameras, where a small round hole acts like a lens. Researchers at California-based Rambus Labs think that practical, small, lens-less digital cameras may be made possible by placing a thin layer of glass, etched with spiral diffraction patterns, directly over an imaging sensor.

On the Rambus blog Dr David Stork explains the technology, which it has dubbed LSS (Lens-less Smart Sensor). Dr Stork says that "Light from the scene diffracts through the grating and yields an apparently chaotic and unintelligible blob on the sensor array -- appearing nothing like a familiar image in a traditional camera". This blob contains information from the original scene, which, through computation, can be resolved into a recognisable image. Diffraction gratings are simpler and easier to make than precision lenses and computing power is relatively cheap, making it possible to produce small, low-cost, LSS devices that lend themselves to a wide array of applications.

A real-world group comparison

Although a comparison of the cameras in specific smartphones wasn't the original intention of this article, the diffraction equation provides a useful tool for this purpose (in contrast, MTF cannot easily be calculated but must be measured). Providing that a smartphone manufacturers specifications state the f number for the lens and the sensor pixel size, the optical limit on resolution due to diffraction can be calculated from the f number and compared directly to the sensor pixel size. In turn these parameters can be viewed alongside a comparison of the number of megapixels (MP) on the camera sensor to illustrate how significant the MP number might really be.

We created two bar graphs for the rear cameras in a group of nine popular smartphones. The first simply shows the megapixel (MP) counts, while the second plots both the published sensor pixel size and the calculated distance (Dx) between two Airy discs at the diffraction resolution limit, with both quantities in microns.

Judged on megapixels alone, it might seem that, with over 20MP, the Meizu MX4 has the superior camera, with a ratio of 1.79 to 1 across the group from largest to smallest. However, looking at the graph of pixel size and Dx it's clear that the LG G4 and the Samsung Galaxy S6 should provide the best images, at least in terms of clarity. The range of variation between the phones, at 1.33 to 1 for Dx, is lens-limited and smaller than the MP figures would suggest.

The comparison of pixel size and Dx also shows that the pixel size is generally about half the calculated size of Dx. Of course it isn't a bad thing to have a camera sensor resolution exceed that of its optics since accepted digital sampling techniques dictate that sampling should be at least twice that of the analogue optical image. While there are other factors, such as internal image processing and colour gamut, that will affect the final captured image quality, this group comparison does suggest that concentrating on the sensor's megapixel count as an indicator of camera resolution is misleading.