Why is dynamic range limited?

Richard

I have been playing around with HDR in CS2. So far, the results have been awful--crummy tripod and/or lousy photographer. But it got me wondering...everyone knows that dSLRs do not have the dynamic range to capture detail in both shadows and highlights in very high contrast scenes. But why not? What, exactly, is the limitation of the sensor or signal processing technology? Is this something that might change in a future generation of cameras? Is anyone working on it? Speaking for myself, I wouldn't be in any rush to upgrade from my 20D to get more pixels, but more dynamic range would certainly grab my interest.

Regards,

Nikolai

Richard,

rsinmadrid wrote:

I have been playing around with HDR in CS2. So far, the results have been awful--crummy tripod and/or lousy photographer. But it got me wondering...everyone knows that dSLRs do not have the dynamic range to capture detail in both shadows and highlights in very high contrast scenes. But why not? What, exactly, is the limitation of the sensor or signal processing technology? Is this something that might change in a future generation of cameras? Is anyone working on it? Speaking for myself, I wouldn't be in any rush to upgrade from my 20D to get more pixels, but more dynamic range would certainly grab my interest.

Regards,

A very nice and rather short explanation can be found right here, in the famous Expose To The Right article on Luminous Landscape.

Executive version: brightness is an exponential function (you need twice as much light to get a pixel twice as bright, so it goes by the power of two), but the existing sensors are all linear devices.

I honestly don't know what are the technical obstacles preventing sensors from being exponential, too.
I hope somebody with CCD/CMOS industrial/manufacturing experience (Bill?) can shed more light on this...

HTH

mercphoto

rsinmadrid wrote:

I have been playing around with HDR in CS2. So far, the results have been awful--crummy tripod and/or lousy photographer. But it got me wondering...everyone knows that dSLRs do not have the dynamic range to capture detail in both shadows and highlights in very high contrast scenes. But why not? What, exactly, is the limitation of the sensor or signal processing technology? Is this something that might change in a future generation of cameras? Is anyone working on it?

Well, everything has its limits. And I'm sure someone is working on it. I don't know much about analog circuitry, I'm in the digital realm. And even then I'm a computer scientists working in digital design verification, not in design itself. But I'll answer as best I can.

The linear versus exponential explanation is pretty accurate. Our senses are not linear. This includes not only vision, but also hearing (the dB scale is not a linear scale). Our vision is exponential but the sensor is linear. Add another photon to the sensor well and it will jump the stored energy in that well by a certain amount. That "delta" is the same per photon no matter how many photons happen to have hit that well before. That's physics, and you cannot change that. You cannot make an exponential sensor.

From what I gather, our eyes are the same - the retina is a linear device. Some other part of the vision system converts an exponential curve out of it. That is pretty much what our cameras do as well.

If you want more dynamic range in a digital camera the only way to do that is to get more bits from the sensor. This is not going to be easy. You'll end up with a higher noise floor, which is not good. You also have the problem of a wider analog-to-digital converter. Its hard enough to make a 12-bit ADC that is linear, accurate, and converts fast enough to dump an 8-million pixel image out to memory in time to take the next picture. Expand that to 16 bits and you will start getting to the point where your converter is no longer linear and the extra bits buy you nothing but bragging rights.

There is a similar problem in the audio realm on the reverse side, the digital to analog converter side. DVD-Audio and SACD are new formats that have yet to gain traction, even though you can start to find players at surprisingly good prices. However the lower end players often have these 24-bit DAC's that aren't hardly linear at all past about 18 bits, making them not much better than your standard 16-bit CD. But extra bits, coupled also with a much faster sample rate, still produces better sound (but much better still with better units with better DAC's).

So two problems with more DR. One is the noise issue inherint in either CMOS or CCD semiconductors. Two is the ADC problem. Both will probably eventually get solved. Fuji's approach is more practical and is rather novel.

MrBook2

Another wrinkle to throw into this mix is the way that compters actually display color. (This is directly related to the file formats that we all use.)

We currently use 24 bit color. This means that there are 256 levels of each of the three primary colors (8 bits per color). This gives us over 16 million discrete colors. This is ok since we find it very hard to tell the differece between R=200, G=200, B=200 and R=201, G=200, B=200. But, this means that we have to map the entire dynamic range of a scene into these discrete levels. This is what you are doing when you have a raw image (which uses more bits) and you chose where the exposure is with a histogram. Most raw formats keep 12 or 14 bits and you get to choose how to remap them to 8 (per color). There is another handy link from LL:

http://www.luminous-landscape.com/tutorials/understanding-series/u-raw-files.shtml

--Aaron

ziggy53

To see the future of where Dynamic Range is headed:

100000-pixel, 120-dB imager in TFA technology

http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=841501&isnumber=18193

(The average dSLR is around 65-67 dB. That's about a 500:1 difference to the above)

Some of the technology is available now:

http://www.digitalcamerawebsites.com/node/309?PHPSESSID=0794bd66198d2e479061fc835c7f923f


ziggy53

Nikolai

Ziggy

ziggy53 wrote:

To see the future of where Dynamic Range is headed:

100000-pixel, 120-dB imager in TFA technology

http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=841501&isnumber=18193

(The average dSLR is around 65-67 dB. That's about a 500:1 difference to the above)

It's nice to know that they at least work in this direction :-)

ziggy53 wrote:

Some of the technology is available now:
http://www.digitalcamerawebsites.com/node/309?PHPSESSID=0794bd66198d2e479061fc835c7f923f
Ziggy53

SMaL? Sorry, man, "been there done that".
They were bragging about this whole thing since, I think, 1998. After a few years they managed to find a VC and actually produce a camera under "Creative" trademark (SoundBlaster guys:-). I was stupid enough to buy one ($70).
Well, there was absolutely nothing in their "autobrite" technology to talk about. No better than your average cellphone camera or any webcam, for what I care. Besides, it went dead after only of couple of weeks of a moderate usage...
What's funny, this last link brags about the same features they were trying to sell in 98, i.e. 8 years ago.
I guess it's just not there. And they probably should change their name from SMaL to SCaM... :

pathfinder

It is interesting that the retinal sensitivity is linear, but the optics of the eye allow the iris to change diameter in real time as the eye surveys a scene, allowing the eye to change light sensitivty independent of the retinal sensitivity. This is why we can see into shadows better than sensors and then look into highlights without being too dazzled - the pupil gets smaller or larger as we scan the scene. Kind of like shooting a frame for highlights and a seperate frame for shadows.

The retinal sensitivity also changes as a function of light intensity, dark adaptation occurring over 45 minute span.

wxwax

Except when you're over 40. :bluduh

pathfinder

All sensitivity dials down with age, waxy, deal with it!!:D

ziggy53

pathfinder wrote:

All sensitivity dials down with age, waxy, deal with it!!:D

Except for our sensitive feelings, sniff, sniff.:cry

... and the sensitivity of our trigger finger. gun2:grim

(Go ahead punk, ... make me gray.)


ziggy53

wxwax

ziggy53 wrote:

(Go ahead punk, ... make me gray.)

colourbox

pathfinder wrote:

optics of the eye allow the iris to change diameter in real time as the eye surveys a scene, allowing the eye to change light sensitivty independent of the retinal sensitivity. This is why we can see into shadows better than sensors and then look into highlights without being too dazzled - the pupil gets smaller or larger as we scan the scene. Kind of like shooting a frame for highlights and a seperate frame for shadows.

That is what I thought HDR photography was all about. Mimicking what the human eye does by taking the whole range of the scene and presenting each local area the way it might look after your eye adjusted to it, so that the sky doesn't look blown out and the area under the trees doesn't look black. With the limitation that the full picture has to make some sense as a whole, without looking flat or uneven.

That doesn't mean that is what HDR is all about, only what I have perceived it to be.

pathfinder

The real limit that HDR has to deal with, is that there is about a 100x more variation in light intensity possible from an LCD screen, and at least 10,000x times more variation in light intensity in real life, both of which have to somehow be compressed to the limited range range of ~256 steps of white to black possible on a printed page.

Any print on paper, is an abstraction, a creation, a weak copy of the reality visible to the naked eye.

People think of the eye as a camera that snaps an image in an instant, but it is really more like a high quality scanner that scans various segments of the scene in front of it, and then the brain assembles these seperate and various scans of light, dark, close, far, sunlit, tungsten lit, all into a single perception that the mind then calls "seeing" Cameras do not do this, they capture a single frame at a time.

Richard

Thanks to everyone for the replies. Seems like this is a difficult nut to crack, but that there are both hardware and software approaches that might provide some improvements in the future.

In the meantime, I have noticed that camera manufacturers do not include a standard measure of dynamic range along with the other camera specs. Does anyone else find this odd? I wouldn't buy an audio amplifier without knowing its frequency response and THD. Many camera reviews mention dynamic range (which is usually said to be "wide") but nothing seems to be quantified. From Ziggy's comment, it would appear that a method exists to express dynamic range in decibels, so why aren't reviewers checking and reporting results? Seems like it would be useful information for the customer. It might also give manufacturers a greater incentive to address the problem.

ziggy53

rsinmadrid wrote:

Thanks to everyone for the replies. Seems like this is a difficult nut to crack, but that there are both hardware and software approaches that might provide some improvements in the future.

In the meantime, I have noticed that camera manufacturers do not include a standard measure of dynamic range along with the other camera specs. Does anyone else find this odd? I wouldn't buy an audio amplifier without knowing its frequency response and THD. Many camera reviews mention dynamic range (which is usually said to be "wide") but nothing seems to be quantified. From Ziggy's comment, it would appear that a method exists to express dynamic range in decibels, so why aren't reviewers checking and reporting results? Seems like it would be useful information for the customer. It might also give manufacturers a greater incentive to address the problem.

Richard,

Do a Google for +"Dynamic Range" +imager +dB and you will see the paucity that exists for measurement.

Part of the reason is that there are no solid standards to test against. Signal to Noise is not defined, but could be, but is greatly complicated because manufacturers can put SN filtering on the imager itself (and they do) and that effect can be made variable dependant on the ISO sensitivity expected from the chip (more filtering at high ISO).

The only fair comparison is to measure the DR "out of the camera". If you look at the high ISO output from the Nikon D200, D80 and Sony Alpha 100 cameras, all reputed to use a very similar chip, you see a great diversity of results. To be completely accurate, one should measure each color channel separately, because different filtering is often used on each channel.

I don't claim to be expert or to understand even a small segment of what would be required for testing Dynamic Range on a color imaging camera, but I kinda don't hope for it. It could encourage camera manufacturers to "skew" their development to satisfy the tests, rather than concentrate on improving the cameras in more meaningful ways.

I much rather rely on comparative and empirical tests, run by independent reviewers, which I can judge for myself the actual value in my application.

Thank goodness dSLRs haven't had to deal with "digital zoom", for instance, but you can see it coming as live preview gets incorporated into dSLR designs.:uhoh

ziggy53

jfriend

Digital zoom in dSLRs kind of already here

ziggy53 wrote:

Thank goodness dSLRs haven't had to deal with "digital zoom", for instance, but you can see it coming as live preview gets incorporated into dSLR designs.

I was thinking about digital zoom the other day and it occurred to me that it is already here in a few dSLRs, it's just called a different name. On the Nikon D2Xs, they have "high speed crop mode". While they don't "zoom" the viewfinder, they do just crop out of the middle and record only the middle portion of the image which is almost the same end result as digital zoom.

As I understand it, a consumer camera with digital zoom, just takes the center pixels, crops away the rest and then resamples the image up to your expect number of pixels. The D2Xs is doing all those same steps except it doesn't resample, it just leaves you with the pixels actually recorded. When you look at the recorded image from high speed crop mode in any app that displays it at nearly anything other than 100%, it will appear to be zoomed in more just like digital zoom.

As sensor resolution continues to increase, I expect this to be more common because you can make great prints with a high quality 6MP image so if you have 12, 16 or even 24MP in your sensor, you have room to crop signficantly and still have a pretty good quality image. It's not the best way to zoom, but it can come in pretty handy in some circumstances.

ziggy53

jfriend wrote:

I was thinking about digital zoom the other day and it occurred to me that it is already here in a few dSLRs, it's just called a different name. On the Nikon D2Xs, they have "high speed crop mode". While they don't "zoom" the viewfinder, they do just crop out of the middle and record only the middle portion of the image which is almost the same end result as digital zoom.

As I understand it, a consumer camera with digital zoom, just takes the center pixels, crops away the rest and then resamples the image up to your expect number of pixels. The D2Xs is doing all those same steps except it doesn't resample, it just leaves you with the pixels actually recorded. When you look at the recorded image from high speed crop mode in any app that displays it at nearly anything other than 100%, it will appear to be zoomed in more just like digital zoom.

As sensor resolution continues to increase, I expect this to be more common because you can make great prints with a high quality 6MP image so if you have 12, 16 or even 24MP in your sensor, you have room to crop signficantly and still have a pretty good quality image. It's not the best way to zoom, but it can come in pretty handy in some circumstances.

Hi John,

While it's true that the two seem similar, they aren't. Digital zoom is interpolated and saved, making believe that it is something it isn't. It is saved at the interpolated size, basically wasting storage space.

Digital zoom was originally designed as a tool for people that didn't have, or couldn't use, image processing software. It quickly became a marketing "hype" that manufacturers use to sell to an unsuspecting public.

The High Speed Crop mode provides the tangible benefit of speed. The image is processed more quickly because of its smaller size, and it takes less time and space to save to the memory card. This is almost opposite what the digital zoom produces, from an acquisition perspective.

Comparisons made of digital zoom versus software crop and interpolate show what a poor job digital zoom does for the image. The reason is that the in-camera interpolation has to use processing shortcuts to retain as much speed as possible.

High speed crop does no interpolation in camera, but I understand what you mean that using image viewer/browser software does interpolate for the purpose of screen display.

If you only care about screen quality images, I agree, either will do.

BTW, there is one instance when digital zoom is preferable:

If you have a digicam with no RAW file capability, that only has high-comression JPG files, and you need much more zoom than the optical zoom provides, and you need to either digital zoom or crop in software later and interpolate, it is generally considered better to digital zoom.

The reason? The digital zoom is working against the RAW image stored in the buffer of the camera, before the effects of high-compression JPG. So you aren't interpolating an already poor image.

ziggy53