More poppycock: are mobile phones
as good as dedicated cameras?

Mobile phone makers have repeatedly stated that the cameras built into their top-of-the-line phones are as good as, or better than, consumer digital cameras. Is this true, or just marketing hyperbole and false representation of their own products?

The realm of the possible

It is useful to start this discussion by examining what it is possible to achieve, with current technology, in a digital camera that is restricted by the physical size of a modern mobile phone. This article on dxomark.com is a good starting point, sufficiently technical but still non-mathematical.

While the discussion at the above link is useful, the conclusions fail to address three important points:

1- How are the pictures used, and - especially - watched? The article at the above link contains only small images, and except for a few details of images selected to illustrate a point in the discussion, does not provide the original images for a comparison. Pictures shot on a mobile phone are most often just watched on the screen of the same phone. This is hardly a sufficient medium to compare image quality, principally because of its small physical size. At most, pictures shot on a mobile phone are uploaded to social media, which almost invariably place severe limits on the pixel-count of uploaded images (and the latter are often watched by others on a phone screen).
2- How much in-camera image processing is acceptable? This is discussed in detail below.
3- Specifically, what are the most visible failures of mobile phone cameras? Also discussed below.

Make it or fake it

The sensors of current digital system cameras have a pixel size between roughly 3.3 and 6.6 μm. Typical current mobile phones have pixel sizes around 0.8 μm. By applying Abbe's criterion (used in microscopy to calculate the maximum possible resolution of an optical microscope), we see that the smallest pixel size that allows a sensor to resolve the smallest detail at a given wavelength is

d = λ / (2 NA)

where d is the minimum resolvable distance on the sensor, λ is the wavelength, and NA the numerical aperture of the lens. Assuming an f/2 lens, NA = 0.24, and the wavelength of green light (approximately in the middle of the visible spectrum) is 530 nm. In these conditions, d = 1.1 μm, which means the lens resolves a minimum distance higher than one pixel. The Nyquist limit of the sensor is twice the pixel size (i.e. a line pair can be at a minimum two pixels wide), but the lens cannot resolve a line pair this narrow. To make full use of the sensor pixel size, one should use a faster lens (i.e. a lower f/ratio, like f/1.4 or f/1.2). This is difficult in a phone camera, gives the space and cost constraints.At a minimum, a faster lens must have a wider front element, which in turn requires a larger number of optical elements to correct aberrations.

One way mobile phone designers get around this problem is by grouping adjacent pixels into a square cluster of two by two pixels. This effectively reduces the pixel count by four times, and simultaneously reduces the image noise in low light. Decreasing the pixel count, for example, from 16 Mpixels to 4 Mpixels implies a massive reduction in image resolution. Removing some of the image noise in post-processing, on the other hand, does not reduce the pixel count, but usually reduces the effective resolution, and the image becomes even fuzzier.

Things get even worse in the red channel of an image, where the wavelength is around 690 nm. Blue light has a shorter wavelength than green, so the image can theoretically be sharper in the blue channel, but this channel in a Bayer sensor only has half the number of sensels as the green channel. Mobile phone designers apply a variety of algorithms to reduce noise and increase the sharpness of the image. Trouble is, with the computing resources available on the CPU of a mobile phone with its limited battery power, none of these algorithms can recover true image information once it has been lost in the recorded image. They can only do a "best effort" job at creating make-believe information, which is not the same thing. Each algorithm produces its characteristic artifacts, and lessening one problem (e.g. noise, or sharpness, or dynamic range) is done at the expense of increasing other problems. The stronger the algorithm, the heavier and more visible are the resulting artifacts. Algorithms that remove image noise lower the effective resolution of the image (which is not the same thing as the pixel count), and artificially increasing the image resolution creates even worse artifacts.

The very small sensors of mobile phone cameras also result in a very high DOF. Since the current agreement among mass-consumers of digital images is that a high DOF is a bad thing, mobile phones process their camera images to separate the background from the subject, and subsequently make the background fuzzier to simulate a low DOF. The algorithms used to identify the background are only a best effort, and end up regarding some parts of the subject as background, or vice versa (the former case is more disturbing). Thus you can often see, for example, hair strands around the face of a person, or even the edges of a model's ears, inexplicably turn into bundles of fluffy mush, a golf club partly disappearing, a sharp tree branch with well-detailed leaves grow out of a person's head while the rest of the tree in the background is an indistinct blob, a model waving a three-fingered hand, and other evident artifacts (at least, evident to anyone who is used to watch better images).

In this respect, it would be better to leave the high DOF unchanged, but mobile phone producers have decided that their phones must produce images that ape the results provided by expensive system cameras and lenses, and do so at the expense of fundamental things like image accuracy. Some mobile phones allow the user to switch off or dial down the image "enhancements", but this requires a deep-dive into the phone menus. Most phone users find it easier to simply ignore the problems.

Typical failures (1) A dog with green spots?

Stella at home, informal picture taken with relatively old iPhone.

The above example is from an iPhone a few years old. It was taken in fairly normal interior diffuse illumination, of the same type I routinely use with my cameras for naturally-looking candid pictures. This little dog has been living with us for the past seven years, and I can guarantee that its fur has never had piebald green patches. Nor does it show green patches on any of the many pictures I have taken with my Olympus, OM System and Sony system cameras, regardless of the illumination type. In fact, Stella's fur is not piebald at all, it only shows subtle shifts of white, light brown and light gray shades. The green patches are entirely an artifact of the phone camera. Largely gone is the warm deep-brown of the eyes, which is faithfully reproduced by "real" cameras. Some of the slightly yellowish tint of the carpet (light hazel in reality) also seems to have "leaked" onto the fur of Stella's legs.

The illumination cannot be blamed, since there were no fluorescent tubes (which sometimes give greenish tones), only LED. In any case, even fluorescent illumination cannot produce the observed, well-delimited green patches, only a general greenish tone of the whole picture. It looks like some image-"enhancement" algorithm was at work looking for green patches of vegetation and brown patches of soil, and not finding them anywhere else in the picture, decided it had "found" them in parts of Stella's fur.

Typical failures (2) Motion blur and shutter lag

Especially when light is not abundant, mobile phone images often fail to freeze motion. The above image shows a slow parade of cars, literally moving at a brisk walking pace. The front of the rightmost car, which is the moving object closest to the camera, is visibly motion-blurred (see 1:1 crop), while the rest of the image, including the rear of the same car, is reasonably sharp. The phone seems to have applied sharpening to the whole picture, and the algorithm it used worked reasonably well, except for the portion of the car near the right edge of the frame, where motion happened to exceed the algorithm's capabilities. This is not out-of-focus blurring or lens aberrations, because both the foreground and background in the same part of the frame are not blurred.

The front of the car is also closer to the right edge of the frame than I wanted. In spite of the car moving really slowly, there was a lag between the image being recorded and its display on the phone screen, and a further delay between my tapping the screen and the actual recording of the image. In my digital cameras, I experienced the very same problem (to an even higher extent, making it problematic to shoot people walking on a street) with my very first digital camera, a Nikon Coolpix 990. My second camera (Nikon Coolpix 5700) was already much better, and by the time of my third camera (Nikon D70s) the problem had disappeared. Mobile phones are not there yet, especially when light is less than optimal, in spite of being built a quarter of a century later.

Other discussions of phone images artifacts

The artifacts I personally observed on mobile phone images are by no means unique. Here is a short list of links. Hundreds more can be found by Googling "phone camera artifacts" and similar expressions.

Extrapolating to the future

Mobile phone cameras still have a long way to go, before they can catch up with present-day digital cameras in straight-out-of-camera results. While mobile phone cameras continue to improve, digital cameras also continue to do so, and this will remain a race toward a continuously receding finish line. Since both phone cameras and dedicated digital cameras use the same technology, there is always bound to be an objective performance gap between the two device types, one designed to work best as a camera, the other designed as a compromise between multiple functions in a pocketable device with limited power and processing resources.

In theory, at some undetermined point in the future, both device types might reach a level of performance where the differences between the two, while still existing, no longer matter for most uses of the images. However, regardless of what mobile phone CEOs and marketers try to make us believe, we are still far from that point. With present technology, there is a clearly visible limit to the image quality of phone cameras. Also with present technology, computational imaging can only go part of the way to bridge the gap in image quality between the two camera types. My concern is that the mobile phone industry has already moved to a level of camera miniaturization and pixel count where further gains in objective performance are denied by the laws of physics, and is already using too much computational imaging to try and hide the former fact. Because of this overreaching, modern mobile phones produce tell-tale image artifacts, clearly visible when one knows what to look for. No amount of salesmen hype can hide this, if you compare phone images with artifact-free pictures.

I do expect that practical factors will make both system cameras and mobile phones obsolete well before we approach the theoretical point where the differences between mobile phone cameras and system cameras no longer matter, and replace both device types with more capable devices.

at www.freevisitorcounters.com