Resolution targets

In modern lab tests of camera lens resolution and overall performance, as little subjective judgement as possible (ideally, none) should be involved in the evaluation of results. Largely computerized methods should be more objective. A foremost example is the automated measurement of MTF (Modulation Transfer Function, see e.g. Wikipedia and an example of a special application here) and its display in graphic or numeric format. MTF testing generally requires the use of resolution targets with one or more sharp, slightly oblique edges between a black/opaque and a white/transparent area. These targets are designed for computer processing and are not suitable for a visual appraisal of image resolution.

MTF tests published by lens manufacturers are often not real tests on a physical lens with a physical resolution target, but theoretical, computerized tests performed on the design parameters of the lens. As such, they typically display the theoretical performance of the lens or, slightly more honestly, the average performance expected of a production batch. Actual specimens of the lens may perform differently, depending on the manufacturing and testing tolerances of the maker, making it possible for both "lucky" lens specimens (performing better than average) as well as "unlucky" ones (poorer performers) to be sold together on the retail market.

Even when a real lens and resolution target are used in lab tests, there is often no camera behind the lens, but instead a microscope focused on the focal plane of the lens. This allows the image produced by the lens to be examined in high detail, to make sure that the lens, rather than the camera sensor, is the limiting factor in these tests.

There is nothing wrong in using a real lens and a real camera for a resolution test, as long as it is clear to all parts involved that this is not just a lens test, but a test of the whole imaging chain. This is the setup that most closely resembles the performance of a lens in real-life, which is what photographers should be most interested in. Unsurprisingly, repeating the test with the same lens on a different camera can give different results. Some caution should therefore be applied when using test results of legacy lenses obtained one or two decades ago. In more than a few cases, the higher sensor resolution and different hardware/firmware of modern cameras may make a legacy lens not as suitable for a given application as it once was. Images produced by modern lenses on modern cameras are subjected to in-camera processing, transparent to the user and specific to the lens being used. This effectively removes aberrations like vignetting, geometric distortion and lateral chromatic aberration. However, this is not possible with legacy lenses (including those that in the past had their aberrations corrected by digital cameras of the same age). In a few special cases, a lens formerly regarded as excellent on a film camera gives worse results on a digital camera, because of the presence of optical windows and filters between lens and sensor of the latter.

There is a long tradition of testing lens resolution by using targets designed for visual interpretation of the test images. These targets usually contain groups of 3 or 5 vertically or horizontally arranged lines of equal thickness, separated by equally thick spaces. Each set of one line and one line spacing is called line pair (lp), and image resolution is often specified in lp/mm (less frequently lp across the image frame). Somewhat confusingly, a group of identical lines on a target is sometimes called element, and a set of adjacent elements of a range of lp/mm, collectively, is called group.

The line thickness and spacing varies among elements according to some mathematical rule, specific to the type of target. Less frequently, there are no groupings of identical lines, and each line pair differs from its adjacent neighbors (e.g. CIPA target). In some target types, lines are radially arranged from the center to the perimeter of a circle, with thickness and spacing increasing toward the perimeter (e.g. Siemens star target).

As a whole, traditional resolution targets for visual evaluation are still quite useful, especially for lenses primarily meant to produce images that will be visually inspected by humans. This is the case, for example, of consumer camera lenses.

In many resolution targets, the thinnest line pairs cannot be resolved by the naked eye. Some resolution targets of historical interest, however, were designed to test the image resolution of spy aircraft, and their line thickness is on the order of meters. Several such targets are still visible in Google Maps images (also here).

A common type of target for testing lens resolution by visual evaluation is a set of geometrical figures, either printed on an opaque background or, for higher precision, made from vacuum-evaporated chrome (for VIS, or other metals for e.g. NIR) on an optically flat glass plate. Printed resolution targets can be quite large, typically A4 or larger, up to around A0, and meant to be placed at a distance of one to several meters from the lens being tested. Vacuum-evaporated metal-on-glass targets are used for smaller target sizes, nominally 4", 2" (5 cm) or 1" squares, and for imaging at higher magnification. Resolution targets vacuum-evaporated on a transparent glass substrate may be illuminated with incident or transmitted light. Very expensive targets manufactured by electron beam lithography are used for testing down to the size range of a single light wavelength, or below (for use with TEM or SEM devices).

As a rule, resolution targets on a transparent substrate must be used with the metal pattern facing toward the imaging system, since the thickness of the glass substrate may cause aberrations with lenses not designed to shoot through glass. This is especially likely at high magnification, high angle of view and/or high lens NA.

The most common targets contain two-tone patterns (black and white, or reflective and transparent). These targets can be positive (black/reflective pattern on white/transparent background) or negative (white/transparent pattern on black/reflective background). Some resolution targets are made in a number of different contrast ratios. Color resolution targets are used for purposes like measuring chromatic aberrations. Gray-scale targets are also used for specific purposes, like measuring the dynamic range of an imaging system and setting a custom white balance. By and large, however, positive and negative two-tone targets are used far more frequently, in part because of advantages they offer in the processing of test results, and in part because they are cheaper than gray-scale targets to manufacture with close tolerances and high repeatability.

Large resolution targets printed on paper for use by amateur photographers tend to be relatively cheap, and sometimes can even be printed on a reasonably good laser printer. For example, a typical office laser printer has a resolution of 600 or 1,200 DPI and, on an A3 page, produces an image with a pixel count of approximately 9,800 x 6,950 or 19,600 x 13,900, respectively. However, laser printouts display multiple problems that substantially reduce the actual resolution of printed images, especially the fact that toner particles scatter around the point where in theory they should remain, before being fused in place by heat. This reduces the actual resolution of laser-printed images by more than one order of magnitude. Gray-scale tones are rendered as a density pattern of discrete toner particles by laser printers, commonly with an easily detectable spatial unevenness that takes the form of longitudinal stripes and random patches. Contrast between darkest and lightest areas can also be limited and non-repeatable in the long term. Laser-printed sharp edges of the type used in slanted-edge targets for MTF measurements are only usable if imaged from a sufficient distance. A partly exhausted toner cartridge also tends to print with a lower contrast and higher unevenness than a new cartridge.

Laser-printed targets have an advantage when combining together several A4 or A3 printouts into a very large composite target. Each individual printout can contain an orthogonal pattern and be attached at a different oblique angle onto a large composite target to avoid the edge aliasing intrinsic in oblique lines printed on a laser printer.

Metal-evaporated targets on glass are more expensive than printed targets on paper by orders of magnitude. Regardless of the technology used to produce a commercial target, a target that contains a broad range (2-3 orders of magnitude) of element sizes is especially expensive to produce.

The main advantage of standard targets, like the USAF 1951 and NBS 1963A, is that test results are more easily reproducible, and therefore reliable and credible, among geographically distant locations and separate labs, without requiring physical access to the very same individual test targets and fixtures.

Accidental resolution targets

Silicon wafers

When a high repeatability of test results at multiple locations is not necessary, it is possible to use resolution targets not specifically designed for this purpose. One example is using silicon wafers discarded during the manufacture of ICs. Wafers that contain optical test targets are especially desirable for this purpose. Most wafers do not contain test patterns, or contain just a few, because including them decreases the wafer "real estate" available to produce saleable devices. The wafers most likely to be useful as resolution targets are those consisting in large part of a variety of test patterns, probably made in very small numbers for testing the manufacturing equipment, or for training personnel and university students. They are also the hardest to find.

Advantages of silicon wafers are that their flatness is extremely precise (they do have some three-dimensional relief, albeit it is significant only at very high magnifications), and their fine detail can easily exceed the precision of ordinary micrometer scales for microscopy. Among the disadvantages are the rainbow-colored reflections produced by interference within very thin layers (which must be reduced with very diffuse light but often remain problematic, despite one's best efforts), that they are only suitable for incident ilumination, and that they must be illuminated at angles that do not project strong reflections of the light source into the optics being tested.

A further disadvantage is that prices for discarded wafers have become unrealistically high, especially for large wafers, perhaps because they are regarded as tech novelty items. The best variety of wafers and the lowest prices are generally found among US-based eBay sellers (with a few EU exceptions, albeit offering limited batches or one-of-a-kind items). Some discarded wafers are available from China-based eBay sellers, but at significantly higher prices.

Wafers from the 1980s and 1990s are more suitable than current VLSI wafers. The latter often contain literally invisible details, smaller than the Abbe's limit for visible light. The best wafer sizes for using as resolution targets are probably 4" and 6", albeit these sizes are rarely used today, and therefore the supplies of these small wafers are limited and decreasing.

Silicon wafers are safely stored in polypropylene boxes designed for this purpose. The flat bottom of the box allows the wafer to be used without completely extracting it from its box. To close the box, first one places a clamp of resilient but soft plastic on top of the wafer. This clamp presses the wafer down near its edges. The cover of the box is then put into place, which presses down the clamp, and the four latches of the cover closed. Boxes are available for all popular wafer sizes.

A major obstacle to repeatability at different locations is that the supply of suitable silicon wafers is limited. Once these small batches are sold out, different wafers must be used, making the test results obtained with one of these wafers essentially non-repeatable.

Silicon dies

A partial solution to the repeatability problem of silicon wafers is the use of individual non‑packaged IC dies. The latter rarely carry test patterns similar to resolution targets, but some dies do carry a sufficiently broad range of usable features. Batches of non-packaged IC dies that did not make their way into further fabrication stages are available on the second‑hand market, usually packaged in small multiple‑well plastic trays containing roughly 50-200 individual dies. A full tray may sell for around 10 USD and provides enough identical dies to distribute among dozens of photographers or test labs.

These dies have been tested on the wafer (and possibly also after splitting the latter), and the defective dies have already been discarded. Therefore, these trays of dies have passed through more production stages than wafers. Nonetheless, individual dies are, paradoxically, far cheaper - by the die - on the second-hand market than the dies located on whole wafers. This shows that the prices charged for discarded wafers have nothing to do with their real cost.

Since individual IC dies have surfaces of only a few square mm, they are difficult to safely handle, and should be held by two opposite thin sides (not top and bottom) with plastic tweezers. Around three to five individual dies (for redundancy in case one die becomes soiled or scratched) can be glued to a glass microscope slide with a highly fluid instant glue or epoxy (not gel or thick glue), making sure that they lay truly flat against the glass slide by gently pressing them in place with plastic tweezers before the glue sets. The slide can be cleaned of dust by blowing air with a rubber bulb, or occasionally washed with running very diluted dish detergent and rinsed with running distilled water, then left to dry in a dust-free enclosure. The dies should never be touched, let alone brushed. Just handle the glass slide by its edges, instead.

For storing these slides and any other similarly sized optical components, I prefer small and cheap plastic boxes with space for about five individual slides inserted above each other into parallel grooves. The boxes are sealed against dust and humidity by an attached door, and are tough enough for mailing microscope slides to labs and hospitals. They can be bought on eBay or from mail‑order lab suppliers in sets of five or ten.

Micrometer scales

Micrometer scales laser-etched or photolithographed on glass, designed to test microscope resolution and/or compute the exact magnification, can also be useful, and are significantly cheaper than proper resolution targets. Micrometer scales etched and then filled with a black pigment are usually fine, but those lacking a pigment produce a much lower image contrast. These scales only display thin tick‑marks of different spacings (100 μm, 10 μm and 1/1,000 ", the latter being 25.4 μm). They are therefore not quite suitable for estimates of resolution in lp/mm, which require alternating black and white/transparent lines of the same thickness.

Photolithographic masks

For tests at somewhat lower magnification than micrometer scales, legacy metal-on-glass photolithographic masks used to expose light‑sensitized semiconductor wafers during IC manufacturing are suitable for transmitted illumination, may display usable patterns. The edges around a mask may contain resolution targets, cross-hairs, micrometer scales, reticle patterns, and/or Ronchi rulings.

These masks typically contain a magnified image of the circuitry of a single IC or a cluster of ICs, which is projected by specialized optics onto the photoresist-covered surface of a wafer in preparation for developing, etching, deposition of metal, and/or chemical processing. Precision steppers move the wafer to expose multiple copies of the mask onto the whole wafer surface.

I randomly purchased on eBay a batch of six masks made by Intel in 2005, of which two apparently empty ones that I could not leave out of the purchase. Each of the "empty" masks turned out to contain a dozen small resolution targets of different types, extremely useful for testing photomacrography lenses and invisible in the eBay pictures.

Optical targets

Although not strictly designed for resolution measurements, other types of optical targets are often salvaged from devices used in the semiconductor industry, and can be repurposed. The above picture shows a few examples.

My needs for resolution testing

In my use of photographic equipment, the capability of performing resolution tests on general-purpose camera lenses at focus distances of one or more meters is not particularly useful. Unless I have reason to suspect that one of my lenses performs unusually poorly, I tend to rely on tests of other specimens of the same lens by reliable Internet sources, like DPReview, DXOMark and Lensrentals. I have a much greater need, instead, of testing equipment for photomacrography and photomicrography. In the past, I have used all the above alternatives to proper resolution targets, and more. None of them, however, is as reliable and repeatable as a standard resolution target, especially if my test results are to be compared to tests by other people who cannot borrow my physical targets, or find identical ones. A common problem with surplus silicon wafers, for example, is that specimens of the same wafer are only available in small batches, and once these are sold, people wishing to perform similar tests are forced to use different types of wafers.

A strong reason for performing one's own lens tests in photomacrography and photomicrography is that the lens tests on generic photography web sites are almost invariably performed at a distance of meters from the test target, which says little or nothing about lens performance in the macro range. Lens tests performed in the macro range with standardized targets and procedures easily duplicated on an international scale are far less common on the Internet.

My previous experience

A few years ago, I purchased a chrome-on-glass 6 x 6 cm USAF 1951 made in China, which turned out to be largely useless because of its exceedingly poor quality. Only groups -2 to 4 are fully usable, with the lines in group 5 too thin and uneven and the accompanying numbers hardly readable, and groups 6 and higher completely effaced and unreadable. The target seems to be corroded by over-etching and/or energetically brushed to remove the photoresist after the etching process. This target is not fully standard because it combines black and transparent backgrounds in order to accommodate both positive and negative targets. This can cause flare from the transparent areas when using the negative targets.

At around 150 € with import fees, it was not cheap, and essentially a waste of money. At present, USAF 1951 chrome‑on‑glass targets made in China are advertised on eBay at fantasy prices of up to 800 €. You can buy chrome-on-glass USAF 1951 targets of far better quality from Thorlabs and Edmund for around 200-250 €. The price of current identical Chinese targets advertised by two different Chinese eBay sellers may vary by up to a factor of four.

I have no idea whether the quality of Chinese targets has improved in the last few years, but their current prices make them uninteresting even in the unlikely case they are as good as western-made ones. Some of these expensive targets are advertised in sizes up to 6", but seem to be mostly empty background, with the test pattern always of the same absolute size regardless of the size of the glass blank. Thus, the premium you pay for a larger target size in these cases only buys you a larger featureless background.

One thing I know is that some Chinese makers have resigned themselves to importing optical flats for resolution targets from western countries, because the quality of China‑made flats is too unreliable. Poor materials, sloppy manufacturing techniques, and abrupt switches to cheaper and worse materials/processes without telling the customer are pervasive industrial practices in China, and make it very difficult for Chinese manufacturers to hold world-level industrial standards even when they are trying really hard to. Even simple materials like aluminum blocks for use in CNC machining must be imported from outside China when a minimum quality is required.

The USAF 1951 is probably the best-known type of resolution target, in spite of being old and dated. The latest specifications of this target are found in section 5.1.1.7 of MIL-STD-150A, which was published in 1959 and cancelled without replacement in 2006.

The patterns on the USAF 1951 are clustered in groups and elements. Each element is labeled on one side with a number from 1 to 6, while there is a group number at the top of each group. Within the same group, element 1 is sometimes placed at the top of its group, other times far away from elements 2-6 of the same group. To find out the actual value in lp/mm, one needs to read both the group number and the element number, then look them up in a separate table. There is even a risk of inadvertently switching the group and element numbers and getting the wrong results.

There are variants of USAF 1951 directly labeled in lp/mm, which are still regarded as standard (the USAF 1951 specifications do not mandate a labeling type), which are more user-friendly. In my tests, I prefer using an NBS 1963A target, for reasons explained here.

Normally, one is interested in measuring image resolution on the sensor, rather than on the subject. Therefore, the resolution in lp/mm displayed on the target must be divided by the actual magnification. Some targets are labelled lp/mm values appropriate for a magnification different from 1x, and other targets, especially those produced without following the appropriate standards or reproduced from images downloaded from the web, are labelled with plainly wrong lp/mm values.

Summary

Resolution targets can be subdivided into two categories: targets used for computer processing of the test images (e.g. slanted-edge targets for MTF measurements) and targets used for visual evaluation of the test results (usually carrying sets of lines of different thickness and spacing).

The latter target type usually involve a visual assessment of which elements of the target are resolved in a test image. A standard resolution target allows test results to be compared among tests performed by different labs/people at geographically separate locations. However, standard resolution targets for use at high magnification are very expensive. Several types of cheaper "accidental" targets not purpose-designed for this use can be a substitute, but are often only available in small batches, which makes test results difficult to compare among tests performed by different labs or individuals.