20

So I know it might not be the best place to ask this question, but maybe some of you are familiar with the mechanics of digital mirrorless cameras and the technology of CMOS sensors.

I don't quite understand why electronic image sensors, that create rolling shutter artifacts, don't produce images with this issue, when combined with a mechanical shutter. The thing I don't get is the following:

Rolling shutter happens due to the sensor readout from one side to another (usually top to bottom), so the actual image is stitched together from scan lines of different consecutive moments. In my understanding the scan line readout mimics the mechanical shutter window traveling above the sensor(?). Now when a mechanical shutter is used in front of the sensor, the shutter takes over this task, whereas the sensor is read out globally at one time(?). Therefore rolling shutter artifacts are not appearing in the final image. BUT if the sensor can be read out globally at one time, why is this not simply happening, when using a electronic shutter? Why can the sensor not just be turned on and off within e.g. 1/2000 of a second completely, avoiding rolling shutter? Why is the "scan line method" needed to take an image, if the sensor can be turned on and off globally?

When I have a camera, that can take still images at 10fps with a mechanical shutter, why doesn't it mean the sensor can take images at 10fps electronically without producing rolling shutter?

I've found this post which explains the general reason for rolling shutter, but not the specific question I have.

I don't even know if my assumptions are correct, but I'd be pleased, if someone could shed some light on this.

Thomas D.
  • 309
  • 2
  • 6

2 Answers2

30

When I have a camera, that can take still images at 10fps with a mechanical shutter, why doesn't it mean the sensor can take images at 10fps electronically without producing rolling shutter?

In order to understand why, we have to take a look at a typical 3T(ransistor) pixel:

3T pixel

This 3T pixel can be used with rolling shutters, but not with (electronic) global shutters. The RST signal will reset the voltage across the photodiode to a positive voltage. When light is detected, that voltage will decreases proportional to the detected photogenerated charges. The annoying part for global shuttering is that we cannot shut off the photodiode. We cannot read out all pixels simultaneously for practical reasons (too many wires, read-out circuits, power, etc.), so during read-out light will keep getting collected in other pixels causing their output to change. Adding a mechanical shutter will make it possible to keep light from reaching the photodiode, bypassing the issue.

In order to implement global shuttering in CMOS, you at least need a 4T pixel:

4T pixel

The TG-signal can be used to close or open the path for detected photogenerated charges to be collected on the gate of \$M_{sf}\$. This is an electronic way of controlling the shutter.

The main disadvantage of global shuttering over rolling shutter is that the time window of capturing becomes shorter. This is explained in the post you mentioned. It is also demonstrated in the following diagram (that I quickly drew up in paint).

Global vs Rolling shutter

The reason why global shuttering is a more viable option nowadays, is because of technological advancements (back-side illumination, reduced noise \$<1e^-\$, ...), usually at the expense of cost.

Sven B
  • 5,072
  • 8
  • 24
  • 1
    Damn this is a pretty precise answer to what I was wondering about! Thanks so much! – Thomas D. May 28 '18 at 14:48
  • 7
    This answer is slightly confusing to read because it uses "rolling shutter" to mean what the question calls "mechanical shutter". In the question, "rolling shutter" is the *visual artifact* caused by the object (or scene) changing between the image exposure/capture at one side of the sensor and the image exposure/capture at the other side, and most noticeable when using an electronic shutter; but in the answer it seems to be the *method* by which the sensor is exposed. – Peter Taylor May 28 '18 at 16:05
  • 2
    When I use "rolling shutter" I mean that the time window during which light is integrated is shifted in time per pixel. Global shutter is where that time window is synchronized for all pixels. I meant mechanical shutter as blocking the light mechanically rather than electronically, and controlling the time window that way. – Sven B May 28 '18 at 18:41
  • 1
    Pretty good for being done in paint! – pipe May 29 '18 at 09:40
19

There are two parts to this: First, rolling shutter can still occur with (some, see note) mechanical shutters. However, this is only at short exposure times. The shutter is built out of two curtains. Before the exposure, curtain 1 is in front of the sensor. When the exposure starts, curtain 1 moves down (or up or whatever) and starts exposing the sensor. At the end of the exposure, curtain 2 moves in and covers the sensor.

Say the curtains take 2 milliseconds to do the entire movement. If you have an exposure time of say 100 milliseconds, that would mean that for 98 milliseconds, the entire sensor is exposed at the same time. As a result, there is no rolling shutter.

At some point, however, the shutters can't move fast enough, and at no point the entire sensor is exposed (this is the point where your camera will not be able to use a simple flash sync). For example, let's say we have a exposure time of 1/1000. That means our sensor may only be exposed for 1 millisecond. However, if the second curtain waits for the first curtain to have fully exposed the sensor, parts of the sensor will already have been exposed for 2 milliseconds! Instead, the second curtain starts moving before the first curtain is finished, and the exposure occurs as a "line" of exposed sensor. See this video from the Slow Mo Guys on youtube, where you can clearly see this happening:

https://www.youtube.com/watch?v=CmjeCchGRQo

So why don't we see the same rolling shutter effect with these cameras? We do! We just don't notice. Here is The shutters on a DSLR move so fast that even at a 5 millisecond exposure time, they will often be exposing the entire sensor at some point (which results in just a blurry picture if movement is high, and that hides any "rolling shutter" effect. At higher shutter speeds, the amount of motion needed to give you the rolling shutter effect (say at 1/1000 of a second exposure) is very high, far higher than your typical exposures in video. But if we do reach those speeds, rolling shutter is a thing with mechanical shutters, just take a look at this image from wikipedia:

enter image description here

I am not sure how sensor readout of a modern DSLR sensor happens, but I think they do still do some form of scanline. This is just not corelated with illumination of the sensor, which is done by a mechanical shutter.

Note: One could perhaps say that the leaf shutters used in (very) expensive cameras are free of rolling shutter.

Joren Vaes
  • 12,376
  • 33
  • 64
  • Thanks so much for your answer! I was wondering if rolling shutter is a thing with mechanical shutters. I found cameras which seem to have a "rolling shutter free" electronical shutter by stacking multiple CMOS sensors. I guess the idea behind this is to offset the scanlines of both sensors and stich the data together afterwards. – Thomas D. May 28 '18 at 14:54
  • 2
    There are also ways to avoid rolling shutter using an electronic rolling shutter. Imagine every pixel has it's only little memory next to it (though analog memory) which it can dump it's value in on command. All pixels can now do this to at the same time, and then the memories can be slowly read, but since they don't change with exposure, this doesn't introduce rolling shutter. (to my knowledge this is pretty much only done on the most high-end of cinema cameras, but this is not my field so citation needed) – Joren Vaes May 28 '18 at 15:18
  • 1
    @JorenVaes there are _plenty_ of ways to avoid rolling shutter without global shuttering. The best, IMO, is properly thought-through digital post-processing. The whole concept of discrete images isn't really conceptually right anyway and can lead to artifacts even with a global shutter. A good implementation should _always_ interpolate between multiple exposures to avoid aliasing, and could then simply shift the coefficients along the image y-axis to avoid the rolling shutter effect. – leftaroundabout May 29 '18 at 08:53
  • @leftaroundabout But won't you be missing some information caused by the needed offset, when shifting the scanlines? – Thomas D. May 29 '18 at 11:48
  • @ThomasD. you're anyways missing information. All video is generated by chopping down the infinitely-dimensional space of time-dependent light-fields to some finite amount of information. Any method of showing the video “guesses” the information back through some kind of interpolation. The rolling shutter effect really is just an artifact of doing this in an inappropriate way. – leftaroundabout May 29 '18 at 12:11
  • @leftaroundabout for movie you are correct, but from some of the context given in the question (still camera that can take 10 images per second) I figured it was to not only cover video but also stills (Where you can have the rolling shutter effect just as well). – Joren Vaes May 29 '18 at 12:14
  • @JorenVaes there's no such thing as a still image. If you make a still image with a digital camera, you're simply extracting one snapshot from the reconstructed video stream. If that “still” contains the rolling-shutter effect, this again comes back to incorrect interpolation in the video stream. (I think most cameras will tweak the way the sensor is read out a bit when in still-image mode, but fundamentally it's still true – the main difference between video mode and still mode is the codec used for storing the files.) – leftaroundabout May 29 '18 at 12:23
  • 1
    I'm sorry, but are you trying to say my DSLR records 40 megapixel (16 bit per color!) videostreams and then supposed to "reconstruct" a still from that? Last time I checked, those sensors were just cleared, then exposed, and then after the exposure the charge in each sensor pixels is read out. At no point is there any video stream. – Joren Vaes May 29 '18 at 12:30
  • this should be the accepted answer with regards to the question :) – Lukáš Řádek Jun 12 '20 at 11:58
  • What about rolling shutter with global reset? – Pedro77 Aug 11 '20 at 14:25