We all want to know how good that next lens will be before we buy it.  Will it be sharp?  How much distortion will it exhibit?  Will it have the dreaded defect of bokeh killing spherical aberration? And how about color fringing?  These are problems to fret over endlessly while searching the web for clues that might help us decide whether or not to buy.

Inexpensive Lens Test Chart

But, where do we turn for reliable information?  Internet is the digital equivalent of a wild west free-for-all.  Like belly buttons, everyone has an opinion and with the anonymity Internet provides many are only too pleased to offer their anecdotal opinings as if they were gospel revealed from on high.  What most of us prefer is precise and accurate information from a reliable source.

Naturally, such rigor implies a structured and repeatable testing approach.  Surprisingly, this concept doesn't appeal to some, who scoff at test sites but instead prefer Internet forum posts and web-sized Jpegs to hard test data.  We do recognize the value of field reports from legitimate sources -- such as working pros and experienced amateurs with a long-established reputation for objective reporting.  But, we do think that carefully designed and executed tests by sites that are in the business of doing such on a regular basis can repeatably and reliably quantify certain aspects of lens performance substantially better than that which may be obtained from sources of uncertain pedigree and soundness.

Lens Review Sources

Having said the above, where on Internet might one find suitable lens test resources?  My favorites are listed below, followed by a summary of what I think each offers to visitors.  Any time I contemplate a lens purchase I visit one or more of these sites to gather as much information as possible.

• The Digital Picture Review

• Digital Photography Review Lens Reviews

• Photo Zone Lens Reviews

• DxO Mark Lens Reviews

• SLR Gear Lens Reviews

• LensTip Lens Reviews

• Lens Rentals Blog (occasional lens tests)

• Fred Miranda Reviews (user subjective comments)

A couple of cautions should be noted before proceeding.  It is a sad reality that variations exist from sample to sample in lenses of just about every brand. Yet most sites test only one sample. A few test a second and even a third if the first seems anomalous.  Only one site in this list tests many samples -- but that site is not a lens test site!  Consequently one is faced with the inevitable fact that, while test sites may try to obtain representative lenses for test purposes, one may occasionally come across a published test result that represents a substandard example of a particular lens.  Unfortunate, perhaps, but it can happen, just as one can have a brand new but imperfect lens delivered to one's doorstep.  (Which, in turn, means that every new purchase should be tested as soon as it arrives!)

Good Test Result

Which leads to the second caution.  One should check as many resources as possible while forming an opinion.  There is no reason to rely on only one site.  Nor is there anything wrong with reading the assessment of photographers with a reputation for objectivity, particularly working pros, those not indebted to a maker.  Look for an analysis of pros and cons rather than one-sided inflammatory or overly flattering language.  I tend to skip posts by self-appointed experts who hide behind Internet screen names.  But the livelihood of working pros depends on making accurate judgments, so most tend to take care in testing their gear.

You Don't Want This Lens!

A final thought before we begin.  Internet forums are rife with too-wise posters who have little or no scientific or technical training and don't conduct technically sound and repeatable tests themselves but who exercise zero self-censorship when it comes to blasting out anynomous opinions into the ether, possibly because their favorite lens tested poorly.  The fact that there is variation among samples of a particular lens can lead to some rather aggresive purple prose condemning very good lens test web sites.  Yet, the better sites appear to try to obtain representative samples to test, and some even retest a second or even a third lens and report all results.  Nevertheless, the fact remains that variation is real, and some even get tested, and no amount of armchair denial is going to change that.

Perhaps the most valuable contribution to the photographic community in this regard comes from Roger Cicala of LensRentals.com.  As the CEO of an equipment rental business Roger has the capability to precision test dozens, even hundreds of lenses, and routinely does so.  For those who doubt, the best advice I can give is to read carefully his highly enlightening blog articles, The Limits of Variation and There is No Perfect Lens.  This will not quiet those with a personal grudge (nothing ever does), but it does reveal their lack of perspective.

 The Digital Picture.com

As a Canon shooter, my primary go-to site is The-Digital-Picture.com.  TDP began as a Canon site, but its tests now also include Nikon, Sigma, Tamron, Tokina, Zeiss, and even Samyang offerings.   Each lens gets a rather thorough written treatment giving the facts and figures concerning the lens -- along with the author's opinion of the lens' usability and suitability for various tasks.  But the real strength of the site is its extensive database of lens test data, which includes vignetting, flare, distortion, and resolution crops from an ISO 12233 test chart.  The ISO 12233 test crop database can be very valuable to anyone considering a lens acquisition.

A key feature of the database is that the site allows comparison of the resolution results for any two lenses at any tested aperture and, for zooms, any tested focal length.  TDP even provides test data for many telephoto and teleconverter combinations.  And, in some cases where the first lens tested did not perform up to expectations the results of a second lens, or even a third, are available for comparison. 

Yet despite this, one occasionally finds negative comments directed at TDP (and other sites) on Internet forums.  Given that no test of a single lens can represent the entire lens population a few bad examples are inevitable.  However, a few posters will claim that poor results must be the product of sloppy testing rather than a poor lens sample.  We suggest they read TDP's test procedure, which is far more rigorous and suggestive of quality results than anything casual do-it-yourselfers are likely to produce.  One suspects the same may be said of every test site listed here.

 DPReview.com

DPReview is one of the most visited photography web sites on Internet. The site has long been known for its tests of cameras and sensors.  Not so many years ago it began testing lenses as well, using DXO's DXOMark as the test vehicle.  Test results include graphs for sharpness, vignetting and distortion.  Also included are sample images for assessing flare, chromatic aberration, vignetting and bokeh.  The DPReview lens test database is still comparatively sparce.

Like many other sites DPReview's results are from a single lens.  And, some of the results are graphical rather than photovisual.  E.g., for sharpness one gets an MTF curve rather than a test chart crop.  However, MTF curves are one of the best technical standards for mathematically comparing multiple lenses across apertures, focal lengths, types and manufacturers.  DPReview tests are also backed by actual field photographs illustrating various aspects of lens performance.  This is a bit like having one's cake and eating it too.  For instance, if one wants to know how much chromatic aberration a lens exhibits, the review provides a graph for that parameter.  If one wants to know what effect that has on a photograph, an image and a 100% crop illustrates the phenomenon.

In the final analysis, DPReview is a different and complementary resource, albeit not yet as well populated, for researching lens purchases.

 PhotoZone.com

PhotoZone has been around for many years and has built up an extensive database of test results.  Included are lenses from Canon, Nikon, Sony, Pentax, Olympus, Samyang, Fujifilm, Sigma, Zeiss, Tamron, Tokina and Voigtlander.  Like most sites, PhotoZone tests only a single lens, often a sample provided on loan to the site owner.  Test results include distortion, vignetting, MTF, and lateral chromatic aberration.  The widely used and well respected tool, Imatest is used for these measurements. 

The above values are provided graphically.  In addition, visual images are included at various apertures to illustrate lens bokeh as well as longitudinal chromatic aberration.  The review concludes with a series of field images and a set of one to five star ratings for optical quality, field quality, mechanical quality and price/performance.  Given that PhotoZone uses yet another test methodology it forms a useful cross check on the preceding two sites.

 DXOMark.com

DXOMark by DXO Labs is a high quality measurement suite for testing lens and camera combinations.  The DXOMark database usually contains tests results for multiple cameras for each lens tested, making the site especially valuable.  Unfortunately, DXOMark had not been around very long so the database is not yet as well populated as others.  Tested lens attributes include sharpness, transmission, distortion, vignetting and chromatic aberration.  Results are reported in graphical form and are color coded where appropriate.  Finally, an overall camera/lens combination numerical DXOMark Score is reported for each lens/camera combination, an explanation for which is given here and here. 

We do have to question DXOMark's choice of how to report lens sharpness.  Rather than reporting long accepted line pairs per millimeter or the more recent line pairs per picture height, or the scientifically based MTF standard, DXO invented their own method, collapsing sharpness into a summary number called P-Mpix, Perceptual Megapixels.  We think DXO has taken the modern trend of dumbing down everything to the lowest common denominator too far. There is no substitute for test chart crops and MTF curves when it comes to predicting how a lens will perform from center to edge of field of view.

SLRGEAR.com

SLRGear provides yet a different view.  In addition to text and specs, test data consists of a 3D chart for sharpness and line charts for chromatic aberration, vignetting, and distortion.  The text provides a rough timing of autofocus travel from infinity to closest focusing distance.  So far, these findings may seem rather mundane, and taken by themselves might not be worth a second look.  But, SLRGear also provides a standard still life setup image rich in detail, pattern, and color for evaluating lens sharpness and color rendering as well as a useful set of test chart crops.  The test chart includes a Macbeth color checker, a gray scale shading chart, and a set of USAF 1951 resolution test patterns.  The test patterns covers center frame and all four corners, and they can be used to judge sharpness in line pairs per millimeter.

LensTip.com

The LensTip.com database contains over 1100 lenses as of this writing.  Of those, some 250+ are in the lens review data base.  Like some other sites, LensTip uses the ISO 12233 test chart for its lens testing.  The test procedure may found at this link.  Evaluation criteria, listed at this link, include resolution (evaluated by Imatest in terms of MTF and lpmm & lp/ph, but also including test image crops as well), chromatic aberration, distortion, coma and astigmatism, vignetting, ghosting and flares and autofocus. Build quality is also assessed qualitatively.  LensTip has a compare feature, but it only allows a comparison of specs and features, not of test results.  For that, one should consider visiting The-Digital-Picture.com reviews.

LensRentals.com

LensRentals is located in the Memphis Tennessee area.  It rents camera gear of all types, and as a result it has a rather large inventory of gear on hand.  This would be unexceptional for our purposes except for the fact that President Roger Cicala is more than just a successful business man.  (2011 Memphis Business Journal's Small Business Award for 21-50 Employees!)  Roger also has a penchant for humor as well as bringing some very good test data to the photographic community via his blog.

While Roger does not routinely publish test lenses, he does have something that no one else has.  His rental business has dozens of lenses available, each of which is evaluated upon arrival and during its life as a rental asset.  Roger occasionally publishes articles providing scatter statistics for many samples of a particular lens -- and sometimes even for multiple brands of a certain type of lens.  This type of information is available nowhere else, and it can be invaluable in providing a picture of how even high end lenses vary in optical quality.

FredMiranda.com

The FredMiranda equipment review section is not a test page.  Rather, it consists of commentary by registered site participants, along with a numerical score between 1 and 10. While the reviews are subjective once many reviews have accumulated the effect is to provide reasonable guidance regarding gear quality.  One should use such commentary with caution, filtering out obvious grudge comments and overly enthusiastic ones.  The best reviews are balanced, objectively listing pros and cons rather than employing overly fawning or hostile prose.

Final Thoughts

No Lens Test Needed! (Click for larger image)

Internet forums can sometimes be a cacophony of disparate voices, with no clear consensus. However, one should not rely on forums for decision quality information.  The best place to look is sites that report systematic quantitative testing.  The good sites provide repeatable and accurate results that may be relied on in the great majority of cases.  Pros not financially indebted to a particular maker may also be worth a read.

Of course, there will still be quality variations in even the most expensive lens lines -- and among brands.  Thus, it is important to check with several sources before purchasing.  And, it is critical to test each new lens to verify performance while within the seller's allowed return window.  One should never accept a lens that is defective out of the box, intending to fix it under warranty.  For our test methodology, read Testing Camera Lenses. You'd be surprised at just how bad some samples of supposedly good lenses can be!