info@grayfoximages.com

 

Author's note:  The viewpoint expressed herein is personal, speculative and derivative of publicly available technical information as well as suppositions expressed by observers.  It does not contain or reflect any internal corporate knowledge.  All linked material is referenced under fair use for educational purposes.  Click on reference superscripts to access referenced sources.

In The Mirrorless Conundrum, we discussed our initial thoughts on mirrorless interchangeable lens cameras -- especially Sony’s role in driving the MILC market segment.  We reached the conclusion that, based on our personal use cases, the time for us to adopt mirrorless technology had not yet arrived, and we therefore compared current DSLR offerings from Canon and Nikon relative to our personal interests -- concluding that Canon still best met our needs, albeit by a much narrower margin than when we switched from Minolta to Canon in 1999.

But, now that Canon and Nikon have entered the mirrorless fray with the EOS R and the Z6 and Z7 respectively -- and granting that MILCs perform very well for landscape and other non-action applications -- it’s time to revisit the mirrorless question.  And, in the process we introduce a discussion of the direction that digital camera sensor technology is evolving -- and the capabilities that are thus enabled.  In the process, we’ll also discuss the fact that Canon, once a leader in digital camera sensors, has now fallen behind in some, although not all, areas -- and what that means to Canon DSLR and mirrorless users.

A Brief Sensor History

In September of 2002, Canon introduced the 11 megapixel EOS 1Ds, the first professional grade full frame CMOS DSLR on the market. In doing so Canon established itself in an almost unassailable position of leadership in full frame DSLRs; most prior DSLR introductions, by all manufacturers, were APS-C crop factor cameras -- and were largely based on CCD technology.  But, CMOS had technological advantages in power consumption, chip integration and fabrication costs,1 and with this introduction Canon leapt to the front of the full frame DSLR market, a position that it has not relinquished from a sales perspective -- although it is a more mixed story when it comes to leadership in sensor technological innovation.  Sony has moved ahead of Canon in dynamic range and sensor readout bandwidth with its Exmor line, and Nikon, through its partnership with Sony for sensor manufacturing, shares that lead.   A notable exception is Canon's dual pixel autofocus, a highly useful feature for video.

Sensor Pixel

Source credit:  http://micro.magnet.fsu.edu/primer/digitalimaging/cmosimagesensors.html

Image credit:  http://micro.magnet.fsu.edu/primer/digitalimaging/images/cmos/cmoschipsfigure3.jpg

The reasons why Canon has fallen behind are hotly debated on Internet forums.  A frequently offered explanation is that Canon deliberately restricts camera features to entice buyers to move upscale in the Canon line, at higher consumer costs and thus higher corporate profits.  While there might be an element of truth in this supposition, it does seem a bit overly simplistic.  One suspects most companies follow this practice to a greater or lesser degree.

Another rationalization might derive from Canon's having long been a sales leader based on past technological leadership and the continuing superb optical quality of Canon lenses.  Many users may simply have such a large investment in Canon gear, particularly in high quality lenses, that they are reluctant to make the financial sacrifice (and to settle for a lesser lens selection) that changing brands would entail.  Add to this the fact that Canon gear performs quite well despite not always being at the top of the heap from a specification perspective (there's an old engineering aphorism thought to be adapted from Voltaire, "better is the enemy of good enough"), and the enticement to switch simply isn't compelling for a great majority of Canon users.    Absent a decline in sales Canon has little incentive to invest in better sensors.  Or, so goes the theory.

Perhaps a more skeptical look might suggest that Canon has allowed its sensor technology to stagnate as it has exploited its position of initial leadership and market dominance (Canon’s current ILC market share is almost 50% worldwide) to rake in immoderate profits, to the detriment of investment in continued sensor technology evolution.  Supporting evidence offered is the fact that Canon was very late in its introduction of sub 500nm circuit design, as verified by ChipWorks,2 and on-chip analog to digital (ADC) conversion, a technology that Sony, and by extension, Nikon, had had for years prior.  On-chip ADCs increase dynamic range and reduce noise by removing the external “noise susceptible” analog path from sensor chip to external ADCs.  Undoubtedly the cost of a new fabrication line calls for a significant capital investment, possibly a daunting prospect for a board of directors watching profits and market position soar from past investment successes.  In fairness, it must be added that Canon has easily the largest, most varied and optically excellent lineup of lenses in the industry, undoubtedly a prime area for continued corporate R&D investment.

Exmor 

Image credit:  https://www.framos.com/media/image/8b/61/4e/csm_exmor-figure1.jpg

In addition to on-sensor ADCs, Sony's Exmor sensor technology also incorporated column parallel data readouts.  The combination of these design factors reduces noise and greatly increases the readout speed possible from the light collecting elements of the sensor.  A comparison of the two approaches appears in the excellent article, What is Sony/s Exmor Technology Anyway? by Darren Bessette of FRAMOS Technologies, Inc., from which much of this discussion is taken.

None of these explanations fully takes into consideration Canon's overall sensor R&D position in comparison to its competitors.  They all suggest that Canon's lag in sensor development are solely attributable to factors and choices that Canon fully controls.  The reality may be quite different, as a German analyst's take on sensor R&D by the major still camera players suggests.3  (Translation here.)  Fundamentally, since Canon makes most of its own sensors -- exceptions are mostly limited to lower end point-and-shoot models -- it is disadvantaged in available R&D funding by its own sales.  As the referenced article points out, Sony has had access to a massive sensor R&D advantage due to the millions of smartphone sensors it has sold to various smartphone makers (actually billions; Sony holds approx. 40% of the total image sensor market!).  Also, the still photography market has been in a long period of decline, partly due to the rise of smartphones and partly due to market saturation as film cameras have disappeared and the initial wave of customer buy-in to digital cameras recedes into the past.  This leaves Canon in a trailing position financially.  From the  reference above,

  • Nonetheless, it becomes abundantly clear that Canon, as a total company, ceased to make a significant contribution to [sensor] research and development around the period 2005 to 2007.
  • The consequences of the savings in R & D only become apparent years later and gradually. This is hardly noticeable to many observers in the short term. But after ten years, it should be clear to every analyst today. - The photographer is already clear on the basis of the now produced cameras.
  • Even if it sounds hard: De facto, Canon has already ceased to be a technologically leading group in the field of photography [specifically digital sensors] about 10 years ago.
  • The main problem is that Canon needs to do a lot of research and development on its large sensors (mostly APS-C and full-frame), while Sony can do this on very small sensors for the smartphones. The latter is easier, cheaper and faster to perform. Subsequently, Sony adapted this only for large sensors. Canon is missing not only the sector of small sensors for testing, but Canon are also missing the huge economies of scale and profits from that sector. Therefore, I think it is out of the question that Canon . . . can ever achieve the same sensor development efficiency as Sony. It follows, in turn, that Canon ultimately has to spend a lot more on R & D for the same image quality. That's a strategic disadvantage.

Sony Exmor R Backside Illumination

Initial Exmor sensors, like CMOS sensors from Canon and others, were front-side illuminated, i.e. the wiring traces were on the front, light collecting side of the sensor, reducing the sensor area available for photon capture (fill factor).  However, in 2008 sony announced the Exmor R, a backside illuminated sensor (BSI), wherein the wiring traces were behind the light sensitive photodiodes, thus making almost the entire front area of each pixel available for light collection.  The increased fill factor available for photon collection is illustrated in Sony’s 2012 announcement of the Exmore RS BSI stacked sensor (discussed next).

Exmor Fill Factor1   Exmor fill factor2         

 Image credit:  https://www.sony.net/SonyInfo/News/Press/201208/ik0fha000000gfvq-img/figure1.png

Image credit:  https://www.sony.net/SonyInfo/News/Press/201208/ik0fha000000gfvq-img/figure2.png

By moving to a BSI design, the Exmore R thereby increased the level of circuitry that could be packed onto the sensor and also the size of the light sensitive photon wells.  Sony’s evolution was, in part, fueled by its leading role in manufacturing mobile phone camera sensors, a market measured in the millions worldwide, and a market requiring ultra high efficiency in the tiny sensor chips used in mobile phones.  Perhaps Canon’s much more limited sensor manufacturing base could simply not generate the corporate revenues needed to stay ahead of Sony -- or perhaps Canon simply chose to invest elsewhere.

Exmor BSI

 Image credit:  https://www.framos.com/media/image/68/36/3d/exmor-figure5.jpg

Sony Exmor RS Stacked BSI

In 2012, Sony announced Exmor RS technology, a stacked sensor configuration for MILCs.  The design included an additional processing layer bonded to the back of the image data collection segment of the chip.  This allowed even more of the processing pipeline to be performed on chip, with resulting reduced latency and higher data movement bandwidth.  The design was further enhanced with a third layer incorporating buffer memory in 2017.

Stacked BSI

Source credit:  Quora/Dave Haynie https://www.quora.com/BSI-sensor-is-better-But-why-expensive-DSLR-Cameras-uses-CMOS-sensor-instead-of-BSI-sensor/a>

Image credit:  https://qph.fs.quoracdn.net/main-qimg-8c87af080e060aa94ec9ee063c2bb5ba

Stacked BSI sensors confer a number of advantages.  Because more processing is done on-chip, there is less opportunity for noise to build up as is the case during external data transfers, resulting in lower noise and higher dynamic range (DR).  Sony sensors have reached the point where they are essentially ISO invariant -- that is conveying the ability to under expose a scene at a low ISO value and raise exposure in post processing without impacting image quality.4  In addition, the use of stacked sensor technology is understood to be one of the enablers for global electronic shutters, a feature with many benefits, as discussed in the next section.

What Next?

Tremendous progress has been made in camera sensor technology.  However, there is yet more to be done.  One of the foremost problems to be addressed next is the video phenomenon known as rolling shutter, the skewing of video images as the shutter traverses the image frame and as data is read out sequentially.  The technological solution to rolling shutter is global electronic shutters -- the simultaneous exposure and readout of the entire sensor as a single step, thus not only solving the rolling shutter phenomenon but also obviating the need for mechanical shutters.  Architecturally, global shutters require a far greater degree of parallelism, as is shown in the diagram below illustrating in-pixel noise cancellation in a new Panasonic global shutter design.

Panasonic 8K global shutter

Content source:  https://news.panasonic.com/global/press/data/2018/02/en180214-2/en180214-2.html

Image source:  https://news.panasonic.com/global/press/data/2018/02/en180214-2/en180214-2-3.jpg

Global shutter capable sensors expose all lines of an image at the same time.  In addition to removing rolling shutter artifacts, global shutters enable higher frame rates, higher resolution, lower noise, wider dynamic range and higher quantum efficiency.5  Sony is well situated with its stacked BSI Exmore RS sensor architecture to develop global shutters.  The latest version of the Exmor RS incorporates a layer of DRAM buffer memory as a means of facilitating internal sensor data transfer.  From Sony’s 2017 3-layer stacked Exmore RS announcement:

Exmor stacked BSI  Italian Trulli 

Image credit:  https://www.sony.net/SonyInfo/News/Press/201702/17-013E/ik0fha000000f60h-img/img01.jpg

Image credit:  https://www.sony.net/SonyInfo/News/Press/201702/17-013E/ik0fha000000f60h-img/img02_02.jpg

Rolling shutter video artifacts continue to characterize Canon’s DSLR and MILC offerings, a problem likely attributable to insufficient parallelism in the image processing pipeline.  Clearly it is in Canon's best interest to upgrade its sensor technology if it is to maintain market leadership.  Canon's status and future direction are covered in the section that follows.

Where is Canon? 

Canon did not begin using sub-500nm circuit etching and putting ADCs on-chip until the introduction of the 1Dx Mark II in 2015, by which time Canon’s sensors lagged Sony and Nikon in base ISO dynamic range by 2-3 stops.  Canon still trails by about one stop, but the gap has been substantially narrowed and as a practical matter the difference is probably not significant in most applications.  In fact, high-end Canon cameras do exhibit excellent noise characteristics.  However, Canon has not fully moved to on-chip ADCs. The EOS 6D Mark II exhibits the same relatively poor dynamic range as previous Canon sensors.  The implication of this is that the old 500nm fabrication line is still churning out obsolescent products.  This, in turn, suggests a manufacturing capacity bottleneck, which would likely trace back to inadequate investment in the requisite sub 500nm fabrication lines.

Quite possibly Sony patents are constraining Canon’s design space for BSI and stacked sensor technology.  Perhaps as a result, Canon has concentrated on dual pixel autofocus (DPAF), a highly effective approach to on-sensor phase detection autofocus (PDAF), a feature that has given Canon a strong position in video autofocus as well as a technology suitable for incorporation into mirrorless cameras.  Other mirrorless cameras devote selected pixels within the sensor array to phase detection or possbily a hybrid combination of phase detection and contrast detection.6

Dual Pixel Autofocus

Source credit:  https://www.extremetech.com/electronics/160232-canon-70d-with-dual-pixel-cmos-af-the-first-dslr-that-can-autofocus-videos

Image credit:  https://www.extremetech.com/wp-content/uploads/2013/07/vcsPRAsset_513951_109512_6d82de1a-a274-4030-9c05-ea89cae2cb6c_0.png

Canon has also experimented with extremely high MP sensors, e.g. the 120 MP APS-H (1.3 crop factor) demonstration sensor announced a few years ago.  But Canon's sensor readout bottleneck remains despite on-chip ADCs, constraining a whole host of sensor capabilities.  As a result, Canon cameras typically have cropped 4K video output and exhibit limited MILC frame rates compared to the competition.  Canon's initial full frame offereing, the EOS R, also does not have sensor-based in-body image stabilization, a staple feature of Sony's product line for years, and now of Nikon's Z6 and Z7.  Canon cites heat dissipation concerns for the absence of IBIS, but one can only surmise that the underlying lack of this desirable feature, which turns every non-IS lens into a stabilized lens, traces back once again to Canon's trailing position in stacked BSI sensor technology.

However, Canon is not standing still.  Especially critical is the vital area of sensor readout speed, a factor which pervasively restricts Canon's CMOS sensors in vital high performance features relative to the competition.  A number of patents in BSI and stacked sensor technology and in global shutter technology have been filed.7  These  patents suggest that Canon will catch up.  It does, however, remain to be seen how long it will take for Canon to finally deploy these technologies in production products.  And, needless to say, others, Sony included, are not standing still.  How the sensor feature space shakes out over the next few years among the top players will be an interesting progression interesting to watch.
Canon Sensor Patent
BSI Sensor Manufacturing 
Canon Sensor Patent
Organic Sensor 
Canon Sensor Patent
Photon Counting Sensor 
Canon Sensor Patent
Stacked Sensor Global Shutter 
Canon Sensor Patent 
Stacked Sensor for High Frame Rates
Canon Sensor Patent
Stacked Sensor with Memory 
Canon Sensor Patent
Global Shutter Improved Response
Canon Sensor Patent
Stacked Sensor with Electronic Shutter 

Canon Sensor Patent
DPAF Sensor Refinement

Image credits: https://www.canonnews.com/category/canon-patents-4

  Return to top

§

If you would like to express thoughts on this subject use the link to send an email.    © 2018 Michael W. Masters
Notice: All images and written material on this web site are © 1999-2018 Michael W. Masters. All rights are reserved under US copyright laws. Images may not be downloaded or otherwise used without written permission of the artist. Written material may be quoted under fair use so long as attribution is given.