Estimating shot noise - what's the origin?

Question

If I have some photon detector, say a CCD, how do I estimate the error introduced by shot noise correctly? Typically, sources found on the internet say that the shot noise is the square root of the number of electrons counted (multiple sources possible, but here's one example: http://hamamatsu.magnet.fsu.edu/articles/ccdsnr.html).

What is then the origin of the shot noise? Is it in the process when photons generate electrons within the semiconductor? Say 10 photons may cause 1 electron, or sometimes 0 or 2 - and hence the Poisson distribution of electrons?

Other sources describe shot noise resulting from varying numbers of photons arriving at the detector. So sometimes 10 photons arrive, and sometimes just 9 or even 11 - and hence the Poisson distribution of electrons, that results from this variation in number of photons?

Is it both? The Wikipedia Article for Shot Noise describes both, shot noise due to quantization of light (photons) as well as of electric current (electrons).

I wonder because if I want to estimate the error introduced by shot noise in terms of standard deviation, then the unit of measure is important.

If the origin is in photons, where 10 photons generate 1 electron on average (i.e. a quantum efficiency of 10 %), but the number of photons varies over time, the shot noise is given as:

$$ \begin{align} \sigma_\mathrm{shot, photons} &= \sqrt{N_\mathrm{photons}}\\ \sigma_\mathrm{shot, electrons} &= \sqrt{N_\mathrm{photons}} / 10 \end{align} $$

Example: A signal of 100 photons has a shot noise standard deviation of 10. This converts to an avergae signal of 10 electrons with a shot noise standard deviation of 1. If I would estimate now the shot noise based on the number of electrons, I would estimate a standard deviation of \$\sqrt{10}\$ - which would be wrong.

If the origin is in conversion to electrons, where 10 photons generate approximately 1 electron, but rather a Poisson distribution around 1, then the shot noise is given as:

$$ \sigma_\mathrm{shot, electrons} = \sqrt{N_\mathrm{electrons}} = \sqrt{N_\mathrm{photons}/10} $$

If I now estimate the shot noise for a signal averaging to 10 electrons, I'd get again \$\sqrt{10}\$ - which would be correct if photons to electron counts conversion was the origin.

An example with Python:

>>> import numpy as np
>>> photon_count = 100 + 10*np.random.randn(10000)
>>> photon_count.std()
10.022125370282811
>>> electron_count = photon_count/10
>>> electron_count.std()
1.002212537028281
>>> np.sqrt(electron_count.mean())  # estimating shot noise as sqrt of number of electrons yields wrong result
3.1664560020385544

Note: I've asked this question over at Signal Processing Stackexchange, and have been hinted to ask here. Please let me know whether the cross-post is fine or not. Thanks!

Answer 1

Is it both? The Wikipedia Article for Shot Noise describes both, shot noise due to quantization of light (photons) as well as of electric current (electrons).

In most array sensors, shot noise comes from photon arrivals, so technically it is photons. However, it does not matter, you'll get the same answer either way since there is a 1:1 relationship between photons and photoelectrons, at least in a conventional CCD detector without photoelectron gain.

Note that if you have a detector with photoelectron gain such as a photomultiplier tube then you actually get significant shot noise from both electrons and photons.

If the origin is in photons, where 10 photons generate 1 electron, but the number of photons varies over time, the shot noise is given as:

Your premise does not make sense. While each photon may or may not be detected (due to quantum efficiency being less than 100%), you cannot have 10 photons generate 1 electron. Each photon either generates an electron or is not detected. Photon arrivals are independent events. Are perhaps thinking of the sensor's A/D gain, which is the relationship between the number of photoelectrons and the A/D counts?

The crux is, in order to detect 1 photon, on average, with a QE of 10 %, I'll need 10 photons to arrive.

QE encompasses losses due to reflection, finite fill factor, and photons passing through the pixel without being absorbed. If you want to count photons that miss the detector elements or bounce back at the source without being absorbed as having "arrived", you also need to include the photons that bounce off the lens, the photons that miss the lens, the photons from the sun that miss the Earth, the photons from other stars that miss the solar system, extragalactic photons that miss the galaxy, ...

Clearly that is a very, very large number of photons. But since these are photon that were never detected, they also don't matter. If something goes through the detector without being absorbed or misses the camera entirely, you can ignore it for purposes of calculating shot noise.

And then you say, "a measurement can only encompass information from the thing you measured". As I measure 10 electrons, I estimate the shot noise uncertainty in terms of standard deviation correctly by taking the square root of my electron count. Or not?

Outside of very specific quantum optical experiments, one cannot measure a photon without absorbing it and collecting its photoelectron. So all of these photons missing your pixel are not measured.