Tutorial: Basic Layer Mask Mechanics

RichardRichard Administrators, Vanilla Admin Posts: 19,967 moderator
edited October 19, 2010 in Finishing School
If you have successfully used layer masks in Photoshop, move along--this tutorial is not for you. Rather, it is directed at people who have never been able to get started at all because they have stumbled on the basic mechanics of creating and modifying a mask. Some highly accomplished photographers have had difficulty understanding these concepts, so I thought a Chapter 0 primer might be helpful. My only goal is to clarify the differences between the layer content, the layer mask and the layer mask display.

A mask makes part of a layer transparent. In a portrait, you might want to sharpen only certain portions of the subject. By using a sharpened layer with a mask on top of an unsharpened layer, you can sharpen only once then experiment with changing the mask to control which areas appear sharpened. While you can accomplish the same thing using a combination of the eraser and history brush, masks have many intermediate and advanced applications that the eraser cannot do at all.

To begin, I created an image consisting of two layers, one yellow and one blue:

90558718-M.jpg

The blue layer is active (as shown by the highlight in the layers palette). By clicking on the icon circled in red, I added a layer mask to the blue layer.

90558438-M.jpg

Notice that a mask icon now appears in the blue layer. It is all white, which means that nothing is currently being masked (that is, hidden) in the blue layer. It also has a black rectangular frame in it, which means that the mask is currently selected for editing, rather than the blue pixels themselves. Finally, notice that a Layer 2 Mask channel has been added automatically, but is currently not checked to be visible. By making it visible and making the RGB channels invisible, the mask itself can be displayed. Note that you can do this quickly by simply alt-clicking on the mask icon in the layers palette. Currently, it is simply all white:

90558538-M.jpg

After making the RGB channels visible again by checking the little RGB box in the channels palette, I took a black brush and painted an M on the mask. Notice that the mask icon is selected in the layers palette and the mask channel is visible. You can see the M on the mask icon, but the blue image icon remains solid blue.

90558624-M.jpg

The M looks orange, not yellow because the layer mask channel is set to be visible. It displays as red at 80% opacity in this example. Having the layer mask visible allows you to see the content of the layer (all blue) simultaneously with the mask. To just see the effect of the mask, click the little box in the Layer 2 Mask channel to suppress the mask channel display:

90558180-M.jpg

The black part of the mask blocks the pixels in that layer, thus allowing the pixels from the layer below to show. It is almost like erasing them, though they are not really gone. If you turn off the visibility of the yellow layer, this is what the blue layer looks like by itself:

90558908-M.jpg

Notice that the blue image icon in the layers palette is still solid blue. The pixels of the M are still there, but are being blocked by the mask. If you want to see the mask itself, click off the RGB channel visibility and click on the Layer 2 Mask channel (or alt-click on the mask icon in the layers palette).

90559008-M.jpg

The mask is always a gray scale image. Anything in black is completely blocked, that is, becomes transparent in that layer. Anything in white is completely visible. Anything in between is visible in proportion to the gray level. Note that since I used an airbrush to paint the M, the edges are not sharply defined, but have a gray halo. You can see the effects of this by looking closely at the yellow edges above. You can paint directly on the mask if you like, but you will not be able to see what the mask is covering. Sometimes this is useful for cleaning up areas of the mask that have imperfections of various sorts.

You can add more areas to be blocked by simply painting in black with the layer mask selected in the layers palette. Likewise, you can paint in white on the layer mask to reveal an area that has been blocked. Here, I used a white brush and painted over the middle part of the M.


90559170-M.jpg

The blue pixels reappear in the part I painted the mask white. Note that I did not have to use a blue brush. The blue pixels were always there; they were just made invisible by the mask. By turning off the yellow layer, you can see the current state of the masked blue layer:

90559348-M.jpg

I hope that by now the difference between the mask and the layer image is starting to become clear. If not, you should play around with the example above until it starts to make sense. There are two other areas that seem to cause a lot of confusion that I want to touch on.

The first is the color of the mask display when it is made visible. Take the image above and make the mask visible by checking its box in the channels palette. Now double click the Layer 2 Mask channel:

90559446-M.jpg

The layer mask display is not the layer mask itself, but rather a colorized, partially transparent version of the mask. Remember, the mask itself is always black, white and gray, and the level of grayness controls the visibility of the image in that layer. The mask display, on the other hand, lets you visualize the areas that are masked while looking at the image itself. In the above case, the actual black of the mask is displayed as red at 80% opacity. The actual white of the mask is transparent. Thus, the masked vertical shapes appear as orange-yellow from the underlying layer and 80% red from the mask. This color is for your convenience only and does not affect the final image in any way. You can control both the color and opacity of the mask display to suit your needs. 50% red is the PS default, but if you are working, say, on an image of a sunset, the red of the mask display is going to get lost in the colors of the sky. If you double-click on the color in the popup box, the PS color selector opens and you can select any color you want for the mask display. I chose green:

90559593-M.jpg

The mask itself has not changed at all. Only the display of the mask is affected. The same is true of the opacity of the mask display. It just makes it easier (or harder) to see the mask and the underlying image depending on the setting. The image itself is not affected. If you turn off the mask display by clearing the Layer 2 Mask box in the channels palette, you see that the image is still only blue and yellow.

90559658-M.jpg

OK, one final point. The layer mask alone does not prevent changes from being made to the image in that layer. If I click on the layer 2 image icon (thus making the image active, not the mask) and fill the layer with red, the entire layer becomes red, but the mask still prevents the masked area from showing:

90559767-M.jpg

If you right click on the mask icon in the layers palette, a popup menu lets you delete or disable the mask, among other things.

90559895-M.jpg

I deleted the mask, and you can see that even though I did the red fill with the mask present, the entire layer became red.

90559988-M.jpg

If you want to make changes to a layer that affect (or exclude) only the masked area, you can create selections from the mask using this same popup menu, then make your changes. That's beyond the scope of this primer, though.

Masks are powerful tools that help you make changes to very specific parts of your images while leaving the rest untouched. What I have covered here is like the part of the computer manual that tells you how to plug it in and turn it on. To see how you can use masks in real-world editing, check out the fine tutorials that Andy and Pathfinder posted some time ago.

Hope this helps somebody.

Comments

  • DavidTODavidTO Registered Users, Retired Mod Posts: 19,160 Major grins
    edited September 6, 2006
    Most Excellent!!!

    This tutorial has been added to our How To site, here.
    Moderator Emeritus
    Dgrin FAQ | Me | Workshops
  • RichardRichard Administrators, Vanilla Admin Posts: 19,967 moderator
    edited September 7, 2006
    DavidTO wrote:
    Most Excellent!!!

    This tutorial has been added to our How To site, here.

    Thanks, David. Hope somebody finds it useful. thumb.gif
  • cletuscletus Registered Users Posts: 1,930 Major grins
    edited September 16, 2006
    Nice work thumb.gif
  • abarabar Registered Users Posts: 6 Beginner grinner
    edited October 18, 2010
    Hi Richard, I am looking at your tutorial because I want to understand about layers and masks, but I can't seem to get past the first part. As I read the tutorial, I'm also trying to do the same things on Photoshop on my computer. However, I can't seem to open two layers in two different colors, like you have done. I was able to start by clicking "file", "new" and then opening up new color fill layers on top of that one with the colors. But I can't understand how to open just two layers in two different colors. Can you help me get past this point? thanks

    Ava
    http://herosportsphoto.com
  • angevin1angevin1 Registered Users Posts: 3,403 Major grins
    edited October 18, 2010
    Richard wrote: »
    Thanks, David. Hope somebody finds it useful. thumb.gif

    wave.gifuh...that would be Me!

    I am making the transition from GIMP to PS, and really need to start over, to ensure i have the concepts as well as the methods.

    this is a great TUTE!

    Thanks Richard!
    tom wise
  • RichardRichard Administrators, Vanilla Admin Posts: 19,967 moderator
    edited October 19, 2010
    abar wrote: »
    Hi Richard, I am looking at your tutorial because I want to understand about layers and masks, but I can't seem to get past the first part. As I read the tutorial, I'm also trying to do the same things on Photoshop on my computer. However, I can't seem to open two layers in two different colors, like you have done. I was able to start by clicking "file", "new" and then opening up new color fill layers on top of that one with the colors. But I can't understand how to open just two layers in two different colors. Can you help me get past this point? thanks

    Ava
    http://herosportsphoto.com

    Hi Ava,

    Like everything in Photoshop, there are 17 different ways to get the same result, so it's easy to get confused. Sorry I didn't make this clearer in the first place.

    What you need to use is a solid color fill layer. From the Layer menu select New Fill Layer->Solid Color, then choose whatever color you like. Or you could use the little button at the bottom of the Layer palette (round, half black half white) and select Solid Color, then choose a color.

    HTH.
  • divamumdivamum Registered Users Posts: 9,021 Major grins
    edited October 19, 2010
    Can I add here that the thing that made the penny drop about masks for me was learning that the brushes switch to BLACK and WHITE when you're in mask mode... and that WHITE adds what's in the layer and BLACK subtracts ("erases") it. Because the brush/colour palette is cut off the lh side here (and other tutes), I didn't understand that part. I then saw a youtube tutorial where I WATCHED somebody do it and... I got it!

    Just throwing that out there for other mask n00bs :Dthumb.gif
Sign In or Register to comment.