To Raw or not to Raw

latourette

Hello Everyone, this is my first posting on dgrin.

I'm on Canon 40D / canon 50mm / canon 70-200 f:4 is / Sigma 105mm Macro / Canon 17-85 is.

I have started shooting RAW, being pursuaded by all those posts advocating raw as THE way to go for all the reasons I won't repeat here.

However, the more I go, the more I am starting to question this, maybe because of my lack of experience but... here are the reasons, why - FOR ME - raw is not always so great (so far).

First, I often shoot in the dark or at night, iso 1250 - 1600 - 3200, and consequently I have to deal with noise.
I have Noise Ninja as a plugin on PhotoShop-CS3.

The performance of Noise Ninja, on a highres jpeg, is definitely superior(to my eyes) to noise reduction in the RAW application from adobe for CR2 files.

I made several tests comparing the 2 tools and consistantly, even on blind-tests (with a friend not knowing which file was which) got far better results with Ninja. (*Do you guys agree with this first observation?)

Now one would think, "well simply do your adjustments in raw, and then open it in PS and reduce your noise with NNinja".

I am also a sound designer, and one of the most basic principles in sound post-prod is "Remove Noise First".. I also apply this principle to images cause so far this is what gets me the best results. (* I also need your feedback on this, do you agree that NReduction goes first?)

So, because I do noise reduction first, everytime I open a raw , I have all those nice parameters I can play with, but I keep myself from doing because i definitely noticed that if I do those things before Noise Reduction, I don't get the best results. So I open the file right away in JPEG to do the noise reduction, and of course I can't go back to raw after...

I have to mention though, when I am shooting in Studio or OUtdoors with good light, the Noise issue because less important and then RAW is fantastic.

So... questions are:

1) Do you agree Noise Ninja is superior to the Adobe Raw plugin for Noise Reduction?

2) Do you agree Noise Reduction should be applied prior to Levels, Exposure, Sharpness, temperature etc... ?

3) Is there a Workaround that would allow using Noise Ninja(or equivalent) directly on the RAW file, and then proceed with all the other RAW adjustments (That would be paradise to me) ?

arodney

You might want to check out Bibble. It offers noise reduction using Noise Ninja within its Raw processing engine. Otherwise, you use the Raw converter to produce the best possible master (high bit, full rez, wide gamut) pixel based file, use noise reduction in Photoshop. Move on from there in Photoshop.

ziggy53

arodney wrote:

You might want to check out Bibble. It offers noise reduction using Noise Ninja within its Raw processing engine. Otherwise, you use the Raw converter to produce the best possible master (high bit, full rez, wide gamut) pixel based file, use noise reduction in Photoshop. Move on from there in Photoshop.

If you process first in ACR and then Open in PS you stay in 16 bit mode until you save the file, and then you have the option of saving in a lossless 16 bit file format or a lossy 8 bit format like JPG.

The problem with shooting to JPG is that you are discarding a tremendous amount of information in translating to 8 bit as well as compressing the image data with lossy algorithms.

Since you deal with audio I will assume that you know the difference between a 16 bit WAV file and an 8 bit MP3 file? So it is with digital image files. Stay in the highest data sampling mode until ready for publishing/presentation.

latourette

Well that leads me to another question I wanted to ask... (Not sure if I should add additional questions within the thread though?)

When I open my CR2 file in ARC and then click on Open File, I get a jpeg but it is in 8bits, and then I have to change the image mode to 16 bit. Is this normal or shouldn't it be directly opening in 16 bit?

jfriend

latourette wrote:

Well that leads me to another question I wanted to ask... (Not sure if I should add additional questions within the thread though?)

When I open my CR2 file in ARC and then click on Open File, I get a jpeg but it is in 8bits, and then I have to change the image mode to 16 bit. Is this normal or shouldn't it be directly opening in 16 bit?

Before you click Open File, you need to set it to 16-bits. At the bottom middle of the ACR window is a link to click on that will let you change it to 16-bits. Then, after setting this, when you do Open File, it will automatically be in 16-bits in Photoshop and will have preserved the full 16-bits of data.

You want it done this way, not converting it to 8-bits from ACR into Photoshop and then back to 16-bits once in Photoshop. At that point, you've thrown away the extra data.

pathfinder

You can adjust the output parameters of ARC to give you 8 or 16 bit images as jpgs or tiffs, and you can adjust the size is pixels as well.

ARC 4.3, which is the RAW converter associated with PSCS3, does allow noise reduction in luminance and chroma. It is not as good as Noise Ninja, or NoiseWear( which is what I use ) but it can be rather helpful, before dropping into Photoshop as a 16 bit image in ProPhoto RGB color space. Then you can reduce noise on the 16 bit jpg or tiff for maximum image quality.

Remember, when you are popping out jpgs in your camera, the conversion from RAW to jpg is being done before noise reduction software, just not by you, but by the algorithms built in by your camera manufacturer.

ISO 800 or 1600 should look fine as a print, if not under exposed, from a Canon 40D.

These are both ISO 1600 from a 40D - via RAW and NoiseWear I am sure

[imgl]http://Pathfinder.smugmug.com/photos/240040567_hxhry-XL.jpg[/imgl][imgr]http://Pathfinder.smugmug.com/photos/240041860_pwMaT-L.jpg[/imgr]

Van Isle

In photography noise reduction should be done after all your other corrections, but before sharpening, cropping, and resizing. This is in accordance with a guideline set out by photographer/author Jason P. Odell in his PP book based on Capture NX.

I'd assume the same principles apply to ACR/Photoshop, or any other PP tool.

Reasoning is this: PP can create more noise in photography, so you don't want to have to remove it twice, reducing detail more than necessary.

VI

jfriend

Van Isle wrote:

In photography noise reduction should be done after all your other corrections, but before sharpening, cropping, and resizing. This is in accordance with a guideline set out by photographer/author Jason P. Odell in his PP book based on Capture NX.

I'd assume the same principles apply to ACR/Photoshop, or any other PP tool.

Reasoning is this: PP can create more noise in photography, so you don't want to have to remove it twice, reducing detail more than necessary.

VI

Hmmm. I think your recommendation is controversial and not accepted by all. This is not how I've heard it or done it. I think you want to do noise reduction as early as possible in the workflow before any signficant post processing. Post processing doesn't create noise. It might affect how visible noise is, but it doesn't create noise where none was before.

The logic I've heard and followed is that you want to get rid of the noise first so your post processing doesn't operate on the noise and so you can see what the image looks like without the noise while you are doing your post processing. For example, if you enhance contrast, you'd certainly rather get rid of the noise first rather than enhance the contrast of the noise and then have to do even more noise reduction to get rid of that extra noise. If you occasionally find too much shadow noise near the end of your post processing (perhaps where you had to raise the brightness of shadows in post processing), it's no big deal to do an extra step near the end before sharpening to knock the shadow noise down and way better to have gotten rid of the main noise before you started editing.

latourette

Very Interesting
Well, for a first thread i'm proud So much information in return for my questions! Summary of what i've learned today:

1) The existence of Bibble Lab (Which I tested but it gives me a blank when I try to open a CR2)

2) How RAW to switch to 16-bit before importing to PS

3) Some good feedback on the task order in postprod... I think in fact this topic diserves a thread for itself. I've heard two different opinions on "when to reduce noise". I have to admit that for now I am still opting for reducing first... which is the root of my problem in fact... I'd like to reduce first, in raw, and then proceed with other parameters, but noise reduction in ACR is defenitely not as good as Noise Ninja. I'd be curious in fact in knowing how all those filters are functionally stacked in ACR? I mean, no matter which slide you touch first, it doesn't determine what is virtually processed first. I believe there must be a predetermined optimal order which the program follows. Any white-paper on that ?

claudermilk

Andrew beat me to it. Yes, Bibble has NN integrated. It should support your 40D on the latest version (hit their support forum if you continue to have trouble). It sounds like this is really the answer to your issue with RAW. IIRC, the developers have mentioned they have NN very early in the pipeline. You want to eliminate noise first--why in the world would you want to sharpen noise, or color correct noise and now have a worse data set to deal with?

mercphoto

latourette wrote:

1) Do you agree Noise Ninja is superior to the Adobe Raw plugin for Noise Reduction?

Probably.

2) Do you agree Noise Reduction should be applied prior to Levels, Exposure, Sharpness, temperature etc... ?

First I'm going to ask you the question of why you think that when you shoot JPG in-camera that this has not been done for you automatically by the camera itself. In other words, repeat after me: "I always shoot RAW.... I always shoot RAW". Do that until you believe it. Because remember this: the camera ALWAYS records a RAW image. The only difference is which RAW converter are you using: the one built-into the camera, or the one on your computer?

In other words, when you are shooting a JPG, the camera is doing some amount of sharpening, levels, curves and the such BEFORE you go and use Noise Ninja. So, your question number 2 is moot.

ziggy53

I should add, just to muddy the waters, that I use the latest version of "Neat Image", the paid commercial version, within PS CS2 and after ACR and basic exposure compensation and maybe Levels and a light touch on Curves. I find that Neat Image works pretty well against noise especially after the image is more "normalized" (for lack of a better term). All the rest of the processing is done after noise reduction.

BTW, the "free" version of Neat image is based on a very old algorithm so please don't anybody try to compare those results with Noise Ninja. The commercial version is splendid and highly adjustable and you can get noise profiles for most cameras.

The more I use RAW Therapee, the more I like its internal noise reduction algorithm, even though it often won't correct all of a high ISO image. If I save the image as a 16 bit TIFF and bring that image into PS CS2, Neat Image does a great "clean up" of any remaining noise.

KED

pathfinder wrote:

[imgl]http://Pathfinder.smugmug.com/photos/240040567_hxhry-XL.jpg[/imgl][imgr]http://Pathfinder.smugmug.com/photos/240041860_pwMaT-L.jpg[/imgr]

So which one is before noise reduction and which one is after?

pathfinder

Smartypants!

How about a comment on the Noise in the images?

KED

pathfinder wrote:

Smartypants!

How about a comment on the Noise in the images?

Assuming you are replying to me, I do try to inject some humor from time to time. There's not really much to say about the noise -- there's more on the left one than on the right, at least to my eye. Noiseware, Neat Image, NN, all are out there, I presume they are all effective (I have NN, may move to NW for its automatic mode).

I have not been nyah-nyah'ed as "smartypants" in approximately 45 years. That is startling and depressing. :cry

pathfinder

KED wrote:

I have not been nyah-nyah'ed as "smartypants" in approximately 45 years. That is startling and depressing. :cry

I'll try to not let that happen again.....

BigAl

jfriend wrote:

Post processing doesn't create noise. It might affect how visible noise is, but it doesn't create noise where none was before.

I'm not sure I agree here. If you try to boost exposure you can get noise in shadow areas, even at ISO100. That noise is created from how the software interprets (and tries to create) the RGB values in those areas. Also, Canon's DPP adds really horrible noise artifacts if you use it for sharpening - the noise is not present if you use more sophisticated software for sharpening (a case for switching off in-camera sharpening if you use DPP for RAW conversion).

I often do light noise reduction at the end of the PP cycle.

jfriend

BigAl wrote:

I'm not sure I agree here. If you try to boost exposure you can get noise in shadow areas, even at ISO100. That noise is created from how the software interprets (and tries to create) the RGB values in those areas. Also, Canon's DPP adds really horrible noise artifacts if you use it for sharpening - the noise is not present if you use more sophisticated software for sharpening (a case for switching off in-camera sharpening if you use DPP for RAW conversion).

I often do light noise reduction at the end of the PP cycle.

Raising the brightness of shadows doesn't create noise. The noise was already there and raising the brightness of shadows makes it more visible. The computer process of raising the brightness of the shadows is a numerical process that doesn't have any way to introduce noise where none existed before. We're not talking about semiconductors here, we're just talking about math and this kind of math can't make new noise. But, if there was subtle noise there already in the shadows (which there is even at ISO 100), then the mathematical process will make that noise more visible.

If that happens to an image of mine, I have no problem knocking the newly visible noise down with noise reduction software later in the PP process when it becomes visible, but if I see noise at the beginning of the process, I'd much rather knock it down before I add contrast to it, enhance colors, etc... because I don't want to be exaggerating the noise with these operations (which is what tends to happen to it).

If you want to do it near the end of the process, that's fine with me - I'm just explaining why I do it as early as I see the noise.

BigAl

jfriend wrote:

Raising the brightness of shadows doesn't create noise. The noise was already there and raising the brightness of shadows makes it more visible. The computer process of raising the brightness of the shadows is a numerical process that doesn't have any way to introduce noise where none existed before. We're not talking about semiconductors here, we're just talking about math and this kind of math can't make new noise. But, if there was subtle noise there already in the shadows (which there is even at ISO 100), then the mathematical process will make that noise more visible.

I still disagree with you. It's like upscaling a picture, how does the software know what colours to create? No amount of mathematics is going to get that right. Luminance noise maybe yes, but colour noise no. Have a look at Ziggy's pics in this thread

mercphoto

BigAl wrote:

I still disagree with you.

But he's right that post-processing will not create noise, it will only make it more or less visible. To convince yourself of this consider a hypothetical example of shooting and capturing completely noise-free and try to figure out what PP steps could possibly introduce any noise into this image. For example, photographing a uniformly and perfectly gray card with no noise during capture. What steps in the raw converter could possible introduce noise that isn't there at capture?

claudermilk

I have to agree with John & Bill. The noise is there to begin with, many PP operations simply amplify it to the point it becomes visible and objectionable.

ziggy53

mercphoto wrote:

But he's right that post-processing will not create noise, it will only make it more or less visible. To convince yourself of this consider a hypothetical example of shooting and capturing completely noise-free and try to figure out what PP steps could possibly introduce any noise into this image. For example, photographing a uniformly and perfectly gray card with no noise during capture. What steps in the raw converter could possible introduce noise that isn't there at capture?

While not noise, rounding errors caused by mathematical shortcutting can lead to something similar to noise when the errors are amplified and visible.

Without knowing the exact transformations involved, and only a few open-source authors allow us that glimpse, all we can do is compare the results of multiple approaches.

As a for instance, integer mathematics is often employed to speed up mathematical processes, but floating point mathematics is potentially much more accurate. If you are a software designer, when do you choose one over the other?

The length of the integer or float also makes a tremendous difference in transformations. Who chooses what methodology for what circumstance?

Those of us who remember, and possibly still have, 8 bit only image processing might also remember the improvements that came with those extra bits. Do you also remember the pain involved in extra processing time and extra RAM required to make the operations worthwhile?

I have great hope for the newest 14 bit imagers, especially when the hardware, firmware and software technologies coincide to show us the best that that technology can offer. It will probably take a couple of years to happen, but I do believe that we will see ultimate improvements.

Ozone

One thing that I have found out about noise, both chromatic and luminance, is that it tends to vary a bit from converter to converter. Whether this is because of interpolation algorithms or from something else, I haven't the faintest.

What I do is stack 3 or 4 outputs (one from each converter) in PS and set the opacity of each layer to something like 30-60%, depending on taste. If one of the layers has superior color, I like to set this one near the bottom of the "stack" and let the colors bleed upward. This will provide a little smoothing of graduations and lessen the impact of luminescent noise. This also gives a little better color rendition because of the averaging. When I have prints made (usually at Wally World), they come out both smooth and detailed. There are almost no mosiac or banding artifacts and look like I have taken them at 100 ISO (or lower), even when using 400 ISO for the original. The downside is that it is time consuming. I sometimes have spent quite a few minutes on each print. However, this can be a really good technique for a few special prints.

claudermilk

Interesting idea. What about the slightly different pixel dimensions that some converters produce?

ziggy53

claudermilk wrote:

Interesting idea. What about the slightly different pixel dimensions that some converters produce?

The conversions that seem to generate a slightly larger pixel count are usually using a slightly different demosaicing algorithm near the periphery of the image.

Normally, the demosaicing software has a full set of neighboring pixels to sample for the color interpolation. It should be obvious that near the outside of the image, there are no neighboring pixels beyond, so the interpolation has to be generated a little differently and, arguably, less reliably.

As such, the images are not scaled up, they are actually showing "extra" pixels in the perimeter.

It's best to trim those images using PhotoShop, Image, Canvas Size, and then set the pixel count to the camera manufacturer's rated pixels. You will get a warning that "The new canvas size is smaller than the current canvas size, some clipping will occur.", and that is what you want so proceed.

Now the image should line up with other "standard" image conversions for an overlay.

Ozone

Ziggy

The differences are indeed at the edges. What I did, however, was to take the larger sized prints and import the smaller dimensioned prints into them. Then I would line up the smaller over the larger, set the marquis tool to the dimensions of the smaller print and then click on the upper left hand pixel of the smaller print (I turn the visibility of the background print off) and crop.

I think that your technique is a little easier.

I do wonder why some of the converters that have several conversion algorithms don't take advantage of all of them and create a multi-layered conversion that gets flattened before the other steps are performed. I noticed that RawTherapee uses three different conversions (EAHD, HPHD, VNG4). Even if they didn't put all three together in one image, it would be nice to allow the capability to export 3 different images based on all three coversions. It would be a simple matter to import all three into PS and write an action that would combine the images. It seems to give a noise reduction that does not sacrifice IQ.