20D Noise Reduction Questions

Richard

Custom function 02 enables in-camera noise reduction when exposure time is greater than 1 sec. I set up my camera with this enabled when I first got it and never much thought about it since. Last night, I tried to shoot some fireworks and was reminded that noise reduction greatly increases the time between shots. In fact, the delay is equal to the exposure time of the last shot. So that got me wondering:

1) Is noise reduction applied to RAW files at all or only to JPG? I was shooting RAW only but the delay was certainly there.

2) How does the quality of in-camera noise reduction compare to PP software? I have generally used RSE or CS2 noise reduction when I needed it.

3) Is there any shooting situation where in-camera noise reduction has an advantage over PP?

Thanks for your wisdom. :thumb

ziggy53

rsinmadrid wrote:

Custom function 02 enables in-camera noise reduction when exposure time is greater than 1 sec. I set up my camera with this enabled when I first got it and never much thought about it since. Last night, I tried to shoot some fireworks and was reminded that noise reduction greatly increases the time between shots. In fact, the delay is equal to the exposure time of the last shot. So that got me wondering:

1) Is noise reduction applied to RAW files at all or only to JPG? I was shooting RAW only but the delay was certainly there.

2) How does the quality of in-camera noise reduction compare to PP software? I have generally used RSE or CS2 noise reduction when I needed it.

3) Is there any shooting situation where in-camera noise reduction has an advantage over PP?

Thanks for your wisdom.

Richard,

The way this particular noise reduction works is to allow the camera to take a "black frame" exposure, which basically builds an image mask, based on "your" imaging chip "at that point in time". The idea is that the noise varies from pixel site to pixel site, and it varies according to the duration of the exposure and according to ambient temperature (and some other stuff, but those are the biggest contributors.) This mask is used to offset the "predictable" noise produced by that sensor during a long exposure. There is still random noise, but that tends to be reduced automatically by the averaging effect of the long exposure itself.

Since this frame is not the same from camera to camera, or even day to day in the same camera, this is truly a customized mask, optimized for "that" exposure. There is no way to duplicate this action or its effect in software.

That said, noise reduction software can still be used to good effect against the problem, it just won't be as effective. It may be effective enough, but you need to run tests for yourself.

ziggy53

ziggy53

I should also mention that some astro-photographers make a seperate "black frame" exposure every half-hour, or so, and then use that frame as a layer to semi-automatically reduce the effects of long exposure noise. It is considered usually "close enough" to what the camera can do automatically, and can eliminate the wait you describe when the camera does it automatically.

So you have another choice available to you.

ziggy53

Richard

ziggy53 wrote:

I should also mention that some astro-photographers make a seperate "black frame" exposure every half-hour, or so, and then use that frame as a layer to semi-automatically reduce the effects of long exposure noise. It is considered usually "close enough" to what the camera can do automatically, and can eliminate the wait you describe when the camera does it automatically.

So you have another choice available to you.

ziggy53

Thanks for the explanation, Ziggy. I don't understand the black frame layer option, though. I gather that you shoot an entirely black frame, say, with the lens cap on. Then you import that as a layer into a subsequent shot. Right? Anything that is not 0,0,0 on that layer is presumably noise. But what do you do next? scratch

Cheers,

ziggy53

I'll defer your question to more useful sites of information (who call it "Dark Frame Subtraction):

http://www.takegreatpictures.com/HOME/Columns/Digital_Photography/Details/params/object/9503/default.aspx
http://www.anandtech.com/digitalcameras/showdoc.aspx?i=2351&p=2
http://www.mile23.com/node/64
http://www.astrosurf.com/re/process.html

erich6

Thinking about this as a "layer" will confuse the issue. It really isn't a layer and in reality it is not "noise" removal. Noise is random in nature by definition. What you do with a dark frame is capture the inherent pixel-to-pixel nonuniformity in the array (there's always some distribution of dark current flowing through). Think about it as removing a "bias" from your image.

Generally this isn't an issue because it is such a small amount compared to the scene signal. But, when you are doing long exposures you're usually dealing with low signal levels and now your long exposure integrates not only the scene but this dark current as well. The dark subtraction effectively gets rid of this.

Your question about the RAW image is a good one. I think the RAW image also gets the dark frame subtracted otherwise I would expect to see a RAW dark frame out of the camera and I've never seen that. Anyone out there know the answer to this question?

Erich

erich6

OK. I was intrigued by this and I've now confirmed that the RAW file also gets the dark frame subtracted when the noise reduction option is set. I performed two tests one at 15 sec exposure and another at 80 sec. In both cases I left the lens cap on and used the remote shutter to take two frames: one without noise reduction and one with.

What I found is that for both cases the noise reduction is effective at removing nonuniformity due to stray light but at the expense of slightly more random noise in the 15-sec exposure (the 80-sec exposure was improved in both uniformity and noise).

I'll post the images and stats from the histogram later. I checked both RAW and in-camera JPG's and both result in the same conclusion. Stats were all derived from the RAW image.

Erich

gus

rsinmadrid wrote:

and was reminded that noise reduction greatly increases the time between shots. In fact, the delay is equal to the exposure time of the last shot.

3) Is there any shooting situation where in-camera noise reduction has an advantage over PP?

Thanks for your wisdom.

I cant shed light on the technicalities RS but i was in all sorts of bother when i tried it. I thought something was wrong with the camera after my first & only star trail shot. It works great though. I should have tried to do the same shot without it turned on to see the difference.

This is 55 min exposure.

Nikolai

What Ziggy said
In-camera noise-reduction is hardware-only, RAW or not. It simply "takes ANOTHER shot" with all the same parameters (like ISO and shutter speed) without lifting the mirror and thus blocking the light from the lens hitting the sensor. Since it happens immediately following the actual shot, the idea is that the dark currents in a non-uniform array should produce a distribution similar to what happened in the actual shot. After that built-in software takes this distribution into account and adjusts the original image while it is still in the buffer (i.e. before writing to your card, even in case of RAW file).

It definitely can be helpful, but you have to be ready to pay the price of waiting (exactly!) twice as much. Thus, if you're doing super long shots (like in Gus case, you'd have to wait almost an extra hour), you may simply miss your next one, unless you count for this effect.

I normally keep this setting off, since I rarely do very dark and very long exposures. However, I agree that it helps with the noise if switched on.

I have my own idea how to clean with the "stars trails" shots, however. We were discussing Zion stars shots while being at the post-shootout and the idea dawned upon me.:):

See, in those shots most of the noise is coming from the skies. Trees and rocks (and other ground-based objects) are usually pure silhouettes, or can be easily made into such, so noise is not the problem there.

Start with making a new copy of your layer (Ctrl+J).
To get rid of the sky noise you need to find an exact "center of rotation", which is relatively easy task. Mark it with two rulers (H and V). Then Select All (Ctrl+A), Free Transform (Ctrl+T) and then - move the central point to the exact intersection of the rulers (it will normally snap to it). Then rotate the image a little. If the trails skew (you can verify it by changing the layer's opacity to 50% or less) - you made a mistake in selecting the center and need to repeat the procedure from the start (ESC to undo the Free Transform). If the trails remain in place - you're golden. Rotate image just so the most (or all) of the "noise particles" completely clear their original locations. Press ENTER (to lock the results of the Free Transform).
And now - drum roll - change the blending mode of your upper (rotated) layer to Darken (or Multiply for an even stronger effect). You will notice that *all* the noise is wiped out - just for the price of slightly affected length of the trails. You can always adjust opacity of the layer if the effect is too strong, of course.
If the center of rotation is "off the grid" (not on your image), you still can try to figure out where it is by zooming out (Ctrl+"minus") and using a "larger than life" selection rectangle before proceeding to Free Transform.
As a variation of this method you can apply it in several different steps, with different radii, each time masking out the areas you don't want to be affected.
Try it, you may like it:-)

HTH

Nikolai

Doppler effect in action

gus wrote:

It's really funny how such a simple thing as a digicam on a tripod at night can efectively tell you which stars fly away from the Earth (red ones) and which one come towards it (bluish ones)..
They should include this shot (or one like it) into Optics (or Physics) text books...

gus

Nikolai wrote:

It's really funny how such a simple thing as a digicam on a tripod at night can efectively tell you which stars fly away from the Earth (red ones) and which one come towards it (bluish ones)..
They should include this shot (or one like it) into Optics (or Physics) text books...

Sure its not the temp nik

http://zebu.uoregon.edu/~soper/Stars/color.html

Nikolai

Could be that, too:-)

gus wrote:

Sure its not the temp nik

http://zebu.uoregon.edu/~soper/Stars/color.html

.. and I tend to forget the simplest facts before thinking of some way more complicated ones...

But my point remains: this IS a physics textbook shot, be it doppler or simply solor=temperature effect :

luke_church

gus wrote:

Sure its not the temp nik

http://zebu.uoregon.edu/~soper/Stars/color.html

I suspect it will be almost exclusively temperature + mank in the atmosphere + some of them may be planets.

The majority of stars appear red-shifted which is supposed to be a justification for the expanding universe. There are also apparently funkier reasons to do with physics that I don't even pretend to understand.

Or at least that's what clever people than me have told me.

Nice photo anyhow, it clearly deserves to be in a text-book for its "cool" factor if nothing else. Pun is intentional.

All the best,

Luke

gus

Nikolai wrote:

.. and I tend to forget the simplest facts before thinking of some way more complicated ones...

But my point remains: this IS a physics textbook shot, be it doppler or simply solor=temperature effect :

Simplest of facts ...i think its a bit more than that mate.

luke_church

gus wrote:

Simplest of facts ...i think its a bit more than that mate.

Nah mate, relativistic stellar physics is what Nik does when he's too tired to do something that actually challenges him ):

Nikolai

I know you're pulling Gus' leg

luke_church wrote:

Nah mate, relativistic stellar physics is what Nik does when he's too tired to do something that actually challenges him ):

.. but it was one of my projects in my senior years...:-)

erich6

OK. 15-sec exposure, lens cap on, no noise reduction:



mean: 42.85 st.dev.: 33.87 median: 38

15-sec exposure, lens cap on, noise reduction ON:



mean: 50.51 st.dev: 39.59 median: 46 (note bottom right corner stray light goes away...also seems to do well getting rid of vertical structure but not so much the horizontal)

80-sec exposure, lens cap on, noise reduction OFF:



mean: 59.82 st.dev: 45.48 median: 55

80-sec exposure, lens cap on, noise reduction ON:



mean: 69.83 std.dev: 51.29 median: 65 (same effect as 15-sec exposure).

Erich

Nikolai

Erich,
Thanks for the samples!

Interesting - deviation gets higher (worse) with the noise reduction ON. yet grid pattern partially disappears..

Richard

I had no idea this was so complex, but I think I understand the big picture here. Let me try to replay this to see that I have it right:

1) In-camera noise reduction does affect the RAW file.
2) In-camera noise reduction addresses a problem inherent in long-exposure, low-light shots. This problem is distinct from the random noise that typical PP software addresses.
3) The longer the exposure, the more useful in-camera reduction is likely to be.

Based on all that, I think I would probably have been better off without the in-camera processing when I was shooting fireworks. I could have taken twice as many shots, and a slight increase in noise wouldn't really matter much. Or I could have taken a single black frame with fn-02 off, then done some work in post to approximate the in-camera processing.

Thanks so much, guys, for all the useful information. So little time and so much to know ...

Regards,

Red Bull

Wow, I've actually learned a lot about this noise reduction from reading this thread. Thanks for all the info!

Nikolai

Exactement, mon ami!

rsinmadrid wrote:

Let me try to replay this to see that I have it right:

1) In-camera noise reduction does affect the RAW file.
2) In-camera noise reduction addresses a problem inherent in long-exposure, low-light shots. This problem is distinct from the random noise that typical PP software addresses.
3) The longer the exposure, the more useful in-camera reduction is likely to be.

Oui:-)

rsinmadrid wrote:

Based on all that, I think I would probably have been better off without the in-camera processing when I was shooting fireworks. I could have taken twice as many shots, and a slight increase in noise wouldn't really matter much. Or I could have taken a single black frame with fn-02 off, then done some work in post to approximate the in-camera processing.

Thanks so much, guys, for all the useful information. So little time and so much to know ...

Regards,

Yes, probably. With the fireworks (and lightings, etc) you do want to get more shots, so I'd turn it off (which I usuaaly do, as I mentioned before:-). When working with stars (which do not go anywhere as rapidly as fireworks) it might be useful to turn ON - provided you're ready to pay with 200% of your time.

Good luck!

Antonio Correia

Red Bull wrote:

Wow, I've actually learned a lot about this noise reduction from reading this thread. Thanks for all the info!

So have I.
Thank you all.