What to get right at shoot time vs in post

rutt

I wanted to divert a hijack from this thread http://www.dgrin.com/showthread.php?p=187323 to its own thread on the right forum. I think Ian, Tomas, and I are all in violent agreement, but I wanted to say it very clearly.

1. Getting exposure right at shoot time is a very good thing.
2. Shooting RAW means you can put off white balance decisions until post time. WB is essentially a post issue, whether in camera firmware, in the RAW converter, or in PS curves.
3. Shooting RAW gives you some wiggle room in terms of exposure, but it's still good to get it right when you shoot.
4. Good post technique can save some shots you wouldn't think could be saved.
5. This is usually easier to do if a RAW version is available.

In addition:

• Sometimes out-of-the-camera jpegs are important, for example at a party where prints are made at the time. Or the shots are projected large nearly as shot. Then WB at shoot time is important. Also in-camera sharpening and saturation.
• Even with the best possible exposure, dynamic range can be an issue. All cameras have this issue, some more than others, but it exists in film as well as in digital. Shooting RAW opens the door a little farther for addressing this issue in post.

mercphoto

rutt wrote:

[*]Getting exposure right at shoot time is a very good thing.

Correct. Its perhaps the most important thing about digital photography. Doing so means one less mathematical operation on the file afterwards, and the less math we do on these images the better off they are. Even in RAW. RAW is still math.

[*]Shooting RAW means you can put off white balance decisions until post time. WB is essentially a post issue, whether in camera firmware, in the RAW converter, or in PS curves.

Correct. Dealing with white balance (unlike exposure) is always a mathematical operation in digital photography. As such, it is indeed a post issue. And as you say, sometimes post is done in-camera (i.e. shooting JPG).

[*]Sometimes out-of-the-camera jpegs are important, for example at a party where prints are made at the time.

Or peope who take a very large number of pictures. At the Austin Grand Prix I took 2,200 photos on Saturday. Given the sheer number of shots taken, the small percent of them that will sell, and the fact they don't sell for large sums of money, shooting RAW for me in that instance is a waste of time and effort.

Now, if doing a wedding, or doing some new photography that I am unsure of camera settings, I'll shoot RAW. But lets face it, wedding photos are indeed much more important and critical than the racing photography I usually do.

In the particular case of basketball without the aid of flash or strobes the use of RAW might actually be unavoidable due to the white balance difficulties.

mercphoto

rutt wrote:

Even with the best possible exposure, dynamic range can be an issue. All cameras have this issue, some more than others, but it exists in film as well as in digital. Shooting RAW opens the door a little farther for addressing this issue in post.

True. But what is often forgotten in this very case is that if you are doing a RAW conversion so that you can later twiddle with shadow/highlight, curves, etc. for bringing out more dynamic range or whatever, if you are converting from RAW into an 8-bit file then, in reality, you have done very little to improve your situation by shooting RAW. An 8-bit RAW conversion has the very same dynamic range as an 8-bit JPG does.

If you are shooting RAW so that you can do more in post, don't destroy your advantage by converting to 8-bit. Convert to 16-bit. Do your edits. Then convert to 8-bit as the last step before saving to JPG.

Shay Stephens

For the RAW shooter
In Camera: Exposure, lighting, composition, DOF, motion blur (shutter speed) if you want it or not, taking multiple frames for either noise reduction or increasing dynamic range,

In Post: White balance, contrast, saturation, gamma, optical flaw compensation (vignette, CA, etc), noise reduction,

note: I left commas at the ends in case I think of more ;-)

rutt

mercphoto wrote:

in reality, you have done very little to improve your situation by shooting RAW. An 8-bit RAW conversion has the very same dynamic range as an 8-bit JPG does.

If you are shooting RAW so that you can do more in post, don't destroy your advantage by converting to 8-bit. Convert to 16-bit. Do your edits. Then convert to 8-bit as the last step before saving to JPG.

This is what I do, but don't tell Dan Margulis. Chapter 6 of the LAB book goes into great detail about why 8 bits is enough. Read Dan's mailing list and you'll get tired of the topic pretty quickly. It's controversial, to say the least.

But let's get the theory very clear. 8 bits doesn't limit dynamic range. The darkest dark and the lightest light which can be expressed in 8 bit images are identical to what can be expressed in 16 bit images. The difference is in how many shades there are in between and where the "steps" occur. I've been caught by Dan's disciples confusing the two things when making a very similar point to the above.

But as I said, I do convert to 16-bit from RAW almost always.

rutt

Shay Stephens wrote:

In Camera: Exposure, lighting, composition, DOF, motion blur (shutter speed) if you want it or not, taking multiple frames for either noise reduction or increasing dynamic range,

In Post: White balance, contrast, saturation, gamma, optical flaw compensation (vignette, CA, etc), noise reduction,

note: I left commas at the ends in case I think of more ;-)

Right on, Shay! The photographer and darkroom rat agree.

Sam

RutT wrote: "I wanted to divert a hijack from this thread http://www.dgrin.com/showthread.php?p=187323 to its own thread on the right forum. I think Ian, Tomas, and I are all in violent agreement, but I wanted to say it very clearly."

I feel I must stand up and protest this statement. Violence is never the answer. I demand the moderator chastise Rutt imediatly. This is after all a family site!

Sam

DoctorIt

I'm perfect*, so I just shoot in JPG-only all the time...




(where perfect = lazy, and too broke to buy another hard drive to keep all the RAW files!)

Shay Stephens

Too funny
So you are opposed to violence, but you want me to beat Rutt?!?!?!

http://dictionary.reference.com/search?db=*&q=chastise

hahahaha

Violence is never the answer. I demand the moderator chastise Rutt imediatly. This is after all a family site!

Sam

Fear not Rutt, the long arm of the moderators wrath is reaching elsewhere right now ;-)

tmlphoto

OK, OK. I admit it to the world. I convert to 8bit (the shame, the shame). Is there some kind of support group I can contact? While I'm outing myself - I also shoot exclusively sRGB. The temptation was too great!

Harveylevine

Shay Stephens wrote:

In Camera: Exposure, lighting, composition, DOF, motion blur (shutter speed) if you want it or not, taking multiple frames for either noise reduction or increasing dynamic range,

In Post: White balance, contrast, saturation, gamma, optical flaw compensation (vignette, CA, etc), noise reduction,

note: I left commas at the ends in case I think of more ;-)

Shay:
True on all counts. After the comma on "In Post" , you probably would want to include "Sharpening".

Harvey

mercphoto

rutt wrote:

But let's get the theory very clear. 8 bits doesn't limit dynamic range. The darkest dark and the lightest light which can be expressed in 8 bit images are identical to what can be expressed in 16 bit images. The difference is in how many shades there are in between and where the "steps" occur.

Egads, I can't believe I made that mistake. Yes, its the number of steps that change, not the overall height of the ladder, so to speak.

I do, however, take issue with "8 bits being enough". The more you manipulate an image the more bits of information you are going to want to have. I do not know Dan's mathematical background, but let me tell you, integer math when you only have 8-bits to deal with can escalate errors rather quickly. Having 16-bits to deal with is a big plus. The accumulation of error as a result is much less, and the more manipulation you do, the more steps you take, the more 8-bits is gonna hurt you. There is a reason why, for example, the floating point unit of modern microprocessors store numbers internally as 80-bit numbers (in many implementations), even though the largest number they can store back out to memory is only 64-bits.

I am going to go out on a limb here and state that if you are working with 8-bit images after RAW conversion then you have no real benefit to shooting RAW once you are done with the conversion part. You get a benefit pre-conversion, in terms of exposure adjustment and white balance. But after that, once you've gone to 8-bit, you are not going to gain anything meaningful in your further editing.

Andy

mercphoto wrote:

I am going to go out on a limb here and state that if you are working with 8-bit images after RAW conversion then you have no real benefit to shooting RAW once you are done with the conversion part. You get a benefit pre-conversion, in terms of exposure adjustment and white balance. But after that, once you've gone to 8-bit, you are not going to gain anything meaningful in your further editing.

ahh yes but all jpgs are 8-bit, so when you create that jpg for web display, or print, you are at 8-bit. you still have the raw benefit, of being able to create 16-bit files anytime the spirit or client moves you to

rutt

Here's what's really bad about shooting RAW: having by 1dmii reduced to the speed of a rebel for shooting sports when I really just want to get all the shots I can for 2 seconds or so. Shot an XC meet in RAW and was sorry. It's kind of a trade off, though, because if I do get a shot with RAW and if the light is difficult, I'm a little more likely to make it work.

Next time: overcast: no raw Harsh light: raw

TOF guy

Shay Stephens wrote:

In Camera: Exposure

In Post: White balance

How do you decide that exposure is correct ?

Thierry

Shay Stephens

Primarily through the use of my light meter and the cameras histogram.

Other tools can come into play as well like rules of thumb (e.g. the sunny 16 rule, the ultimate exposure computer, the zone system, etc), the cameras meter, and the image preview (relative only).

TOF guy wrote:

How do you decide that exposure is correct ?

Thierry

tmlphoto

rutt wrote:

Here's what's really bad about shooting RAW: having by 1dmii reduced to the speed of a rebel for shooting sports when I really just want to get all the shots I can for 2 seconds or so. Shot an XC meet in RAW and was sorry. It's kind of a trade off, though, because if I do get a shot with RAW and if the light is difficult, I'm a little more likely to make it work.

Next time: overcast: no raw Harsh light: raw

Rutt, I've been shooting RAW sports action shots without ever running into the buffer wall. I usually shoot is short bursts of 2-4 frames instead of just standing on the shutter. I routinely use the 8 fps on my MII. I think you have a better chance of catching the peak action that way. Using the "spray & pray" method can still leave a gap where you were hoping to catch the peak action IMO.

TOF guy

Shay Stephens wrote:

Primarily through the use of my light meter and the cameras histogram.

Well then WB matters !

Thierrry

Shay Stephens

It would be nice if you could be a little more cryptic

TOF guy wrote:

Well then WB matters !

Thierrry

(edit)
I found my Inspector Gadget decoder ring. And if I am right, I think you are making the point that WB influences the histogram, and thus you have to have the right WB in order to get a good exposure.

If that is your argument, then let me just say that this is only true when using the histogram in Photoshop.

When I shoot a scene with the 20D and set the WB too cool, the histogram is the same (in camera) as when I shoot the same thing with the WB too warm.

Now it may be that other cameras handle the histogram differently, so each person should verify how their camera operates, but for me, I don't need to set WB to determine exposure using the *camera's* histogram.

TOF guy

Shay Stephens wrote:

You are making the point that WB influences the histogram

Indeed.

Shay Stephens wrote:

let me just say that this is only true when using the histogram in Photoshop.

Cameras - like Photoshop - always show the histogram of a JPEG image, that is after applying curves (for the cameras that support curves), WB coefficients, demosaicing, gamma corrections, and various additional settings (saturation and the like). One important difference: Photoshop ignores in-camera curves.

Shay Stephens wrote:

When I shoot a scene with the 20D and set the WB too cool, the histogram is the same (in camera) as when I shoot the same thing with the WB too warm.

Check again. Takes pictures of the 20D LCD histogram with your 717 to be able to compared them carefully. Also the differences are more obvious to see if you exagerate the WB offset (remember that differences in WB are in fact proportional to the difference of the inverse of the temperature).

Shay Stephens wrote:

it may be that other cameras handle the histogram differently

No, currently all cameras show histogram as stated above. Some cameras show a composite histogram, others have the additional capability to show histograms per channel, but they all show histograms based on the JPEG image.

But here is an interesting technical discussion: should camera manufacturers offer the possibility to see the histograms for each channel based on RAW data.

Shay Stevens wrote:

I don't need to set WB to determine exposure using the *camera's* histogram.

You need to be aware that WB influences the camera's histograms, since this is affecting the way you check exposure

Shay Stevens wrote:

and thus you have to have the right WB in order to get a good exposure.

That is the question, isn't it ? Maybe yes, maybe not. Here is an answer: all white balance coefficients equal (implies 5300K) for all pictures. WB corrected in post

Thierry

Shay Stephens

TOF guy wrote:

Check again. Takes pictures of the 20D LCD histogram with your 717 to be able to compared them carefully. Also the differences are more obvious to see if you exagerate the WB offset (remember that differences in WB are in fact proportional to the difference of the inverse of the temperature).

I did that before responding. I shot a daylight scene from a tripod using incandescent WB, and then immediately shot again using shade WB setting. The 20D's histogram of each photo was nearly identical. I saw no difference that would indicate a change in exposure.

Now I was shooting RAW. And like I said, other cameras may handle this differently. I took both shots into Photoshop, and there was a difference in all the histograms I viewed. None of them looked like the in-camera histogram review.

ginger_55

Wrapping my brain around this "non" violent conversation is not happening.
However, I do want to mention on the WB, and I always shoot RAW, it matters if I do/don't get it right at the time I shoot. Especially noticed at the church baptisms. If I have gotten it terribly wrong, it is nearly impossible to correct all elements involved: the wall at off white and the faces at whatever color they maybe should be. I have had some major disasters, RAW advantage or not.

Of course, the church and the people do not seem to mind, but it bugs the H out of me to see the "off" colors, even with all the color correction that I can think of. Maybe you experts do this better, but it must be at some cost?

ginger

Shay Stephens

Do you shoot RAW + JPG or RAW only?

ginger_55 wrote:

Wrapping my brain around this "non" violent conversation is not happening.
However, I do want to mention on the WB, and I always shoot RAW, it matters if I do/don't get it right at the time I shoot. Especially noticed at the church baptisms. If I have gotten it terribly wrong, it is nearly impossible to correct all elements involved: the wall at off white and the faces at whatever color they maybe should be. I have had some major disasters, RAW advantage or not.

Of course, the church and the people do not seem to mind, but it bugs the H out of me to see the "off" colors, even with all the color correction that I can think of. Maybe you experts do this better, but it must be at some cost?

ginger

Shay Stephens

20D sample in-camera histograms
Here is the histogram review from the 20D setup to shoot RAW only. Both were shot within seconds of each other from a tripod. The lighting was daylight. The first shot was taken using a WB of 3000K and the second one was shot using the cloudy WB preset (6500K), the two extremes I typically deal with.





I have not tried shooting RAW + JPG and seeing if that has any effect on the resulting histogram. I will leave that up to someone who regularly shoots RAW + JPG.

Khaos

I would think that the histograms from the camera preview and in PS would look different. Different algorhythms are being used to determine what's white and what isn't. I use the histogram to ensure I'm getting the best dynamic range to work with later in post.

I've been playing a lot lately with the camera's auto functions campared to me adjusting the same shot manually to get an idea how this camera works best. Canon states more info is used on the right end of the histogram than the left. For the 20D when allowing the camera to choose TV when I choose AV, I usually need to lower the EB a few clicks. But in manual when using the meter, It's best if I'm on the right a few clicks. So the bottom line is, for me anyways, the in camera histogram is a tool to give you an idea of the range, but every camera is different and you need to know what the best way is to use yours.

rutt

mercphoto wrote:

Egads, I can't believe I made that mistake. Yes, its the number of steps that change, not the overall height of the ladder, so to speak.

I do, however, take issue with "8 bits being enough". The more you manipulate an image the more bits of information you are going to want to have. I do not know Dan's mathematical background, but let me tell you, integer math when you only have 8-bits to deal with can escalate errors rather quickly. Having 16-bits to deal with is a big plus. The accumulation of error as a result is much less, and the more manipulation you do, the more steps you take, the more 8-bits is gonna hurt you. There is a reason why, for example, the floating point unit of modern microprocessors store numbers internally as 80-bit numbers (in many implementations), even though the largest number they can store back out to memory is only 64-bits.

I am going to go out on a limb here and state that if you are working with 8-bit images after RAW conversion then you have no real benefit to shooting RAW once you are done with the conversion part. You get a benefit pre-conversion, in terms of exposure adjustment and white balance. But after that, once you've gone to 8-bit, you are not going to gain anything meaningful in your further editing.

As I said, Dan Margulis has been struggling with this for quite some time. Just today I noticed that he has started to post his most current thoughts on the topic. If you are interested in the topic, it pays to read what he thinks. He is very stubborn, but he is also very pragmatic and empirical. His thoughts on this topic are way deeper than you might imagine possible. And he doesn't agree with the intuition that 16 bits is way better than 8 bits (to say he doesn't agree is understating the case.)

So here is a link to the first of 4 posts of his on this topic, if you are interested. I think you'll have to register for yahoogroups and perhaps even subscribe to the colortheory group before you can read it, but maybe not.

http://groups.yahoo.com/group/colortheory/message/12363

[NOTE: I botched this reply at first and posted as an edit of my original reply to Mercphoto, so I just deleted that and posted this instead.]

mercphoto

Yeah, can't get to that thread without joining it. Quick question, is the analysis that 8-bits enough based on a mathematical argument? Because that is basically my argument that it is not enough. Digital photography is math. Worse, its integer math. Worse still its only 8-bits in depth. Nasty things happen when you manipulate numbers that small.

Let's not forget, I think Dan used to say evil things about sRGB as well...

rutt

mercphoto wrote:

Yeah, can't get to that thread without joining it. Quick question, is the analysis that 8-bits enough based on a mathematical argument? Because that is basically my argument that it is not enough. Digital photography is math. Worse, its integer math. Worse still its only 8-bits in depth. Nasty things happen when you manipulate numbers that small.

Let's not forget, I think Dan used to say evil things about sRGB as well...

Dan has strong opinions and doesn't mind sharing them. But he is very thoughtful and his opinions do evolve over time. That's what makes them worth listening to. In the case of the 8 vs 16 bit opinion, there has been a huge amount of energy and thought put into the topic over the years (just google a little and you'll see.)

Yes, at least part of his argument is highly mathematical. I'm in danger of misrepresenting him or plagiarizing if I go into it too deeply, but roughly the very highest level outline of the argument is like this:

1. Empirically, there is far less (no?) difference between 8 and 16 bit images, even after a lot of processing has been done to them. In fact, Dan has an outstanding challenge to provide examples.
2. Gee that's surprising given one's first intuitions about the math.
3. Detailed examination of the math, involving statistics among other things.

Chapter 6 of the LAB book would give you a taste of this, but as nerdly as that is, it's only the tip of the iceberg. Join the yahoogroup if you find this kind of thing entertaining. I'm sure Dan would welcome another participant in this long standing debate. Maybe you'll find a hole in his mathematical reasoning.

mercphoto

I think I'll join that list then and get more details on the topic. I would be surprised to find that 16-bit is not noticeably better than 8-bit but I could be wrong. Then again, I'm one who also doesn't see a large benefit to RAW shooting (outside of white balance and exposure adjustments, which I go through pains to not have in the first place).

I could be convinced otherwise. I did, after all, shoot part of a kart race as RAW+JPG, then processed both files and created 20x30 poster prints to judge the difference. I couldn't find any. Just as I believe the RAW proponents are basing their arguments on shaky ground, my thoughts on 16-bit could be as well.

TOF guy

Shay Stephens wrote:

Here is the histogram review from the 20D setup to shoot RAW only. (snip) The first shot was taken using a WB of 3000K and the second one was shot using the cloudy WB preset (6500K), (snip).

First, thank you for taking the time to do the tests and posting the results :

Well one learns something everyday. Looks like Canon's choice of histogram is simply the G channel histogram. So you were right about differences between manufacturers (but it is still calculated from the JPEG image and therefore after applying WB coefficients) . A side-note: I believed that the 20D displays separate RGB channels (some of my comments were more with this type of histograms in mind, and how to use them best, to get the best info on setting exposure), and I stand corrected.

If you have the time, try "incandescent" (should not be the same as 3000K - even if the incandescent color temperature is set at 3000K - if Canon has done its homework) or even better, "fluorescent", rather than 3000K to confirm. You will also understand what I mean by WB will change choice of exposure.

On a side note, for better accuracy you should avoid using temperatures to set WB in your 20D. Use as stated above "incandescent", do a preset measurement of the WB, but avoid using the T scale, unless you play with the 2-D WB bracketing feature on the 20D.

Other questions raised:

- Adobe Photoshop vs. camera JPEG histogram. The calculation of the JPEG image from the RAW data involves a step called demosaicing. This is were the camera evaluates 3 colors per channel while most dSLRs only measure one of the channel per pixel. There are several public algorithms known, and no doubt that each camera make / commercial software has its own. In fact conversion to JPEG algorithm is one of the secrets the most jealously kept by a camera company. Different demosaicing algorithms change the histogram quite a bit. And that is only the beginning of the processing chain. Adobe for instance may have a very different idea than Canon about what is the best tonality curve for an image. Etc etc. Therefore it is no surprising that an histogram shown in ACR does not look like the one shown by Canon.

- RAW+JPEG vs RAW. No matter which mode you use, the camera has to calculate a JPEG image for review on the LCD display. The difference between these two modes is that the JPEG is not saved to the compact flash memory in the latter mode, saving some space.

- Issues with colors and histograms. I can think of many reasons for colors issues not necessarily involving a histogram (hard to say without seeing an example). But one issue to be aware of with histograms that do not show each separate channel is as follow. You may be overflowing one channel, but the histogram will not show it. If the camera shows the G channel, possibly the case for the 20D (at least G seems very dominant), then you may have issues with the other 2 channels without knowing it. The Nikon D70 shows luminescence, which is a linear combination of the RGB channel (and G is dominant in calculating luminescence). But that barely helps. When adjusting WB. make sure that you don't choose an area where one of the channels has been overflowed. Many RAW converters can "recover" the lost information by making educated guesses about what the value should be. So you may have the impression that the color information is there, but it was not entirely recorded. Obviously this is not going to give you good WB estimates.

Thierry

rutt

mercphoto wrote:

I think I'll join that list then and get more details on the topic. I would be surprised to find that 16-bit is not noticeably better than 8-bit but I could be wrong. Then again, I'm one who also doesn't see a large benefit to RAW shooting (outside of white balance and exposure adjustments, which I go through pains to not have in the first place).

We agree about RAW, but I have come to use it all the time anyway because I do like not worrying about WB when I shoot, and I often rely on it for dynamic range recovery. Getting WB really right at shoot time can involve custom WB and a grey card which is way more trouble than I ready for.

As for dynamic range recovery, I didn't tell you that I agreed with Dan. All I did was tell you that he is far deeper into this topic than I every plan to get and that my simpleminded way of thinking about it at first didn't hold water. Nevertheless, I've found that shooting raw and converting to 16 bit while being careful not to clip the endpoints of the histogram sets up my workflow very well. In particular, I use shadow/highlight in RGB to recover highlights and in LAB to recover shadows. Especially since I learned to use the two colorspaces this way, this turns out to be useful a lot more than I would have guessed, maybe 50% of the time.

Dan isn't arguing that 16 bit is worse than 8 bit, only that it isn't as useful as you might think. My computer is fast enough that I can barely tell the difference, so I stick to 16 bit for most of the processing and don't worry about it.