This is the original essay on color reproduction and a color-preserving workflow, which was published on 6 October 2005. It is currently severely overdue for an update, so it’s only reachable by direct links. I hope to replace it with an up to date version some time in the near future. Read and use at your own risk.
Based on very general principles, desirable properties regarding color accuracy and tonal control are defined for a digital workflow. It is shown that current practices for the application of tone curves violate a key property of color integrity. A new method for tonal control is proposed that maintains the tonal flexibility advantages of the current approach, yet does not sacrifice color integrity. Tools and tips are offered to implement this workflow in Adobe Camera Raw and Photoshop.
I will speak a number of times about true gamma spaces. These are RGB spaces with an exact gamma encoding of the form output=inputg, such as Adobe RGB (2.2), ProPhoto RGB (1.8) or any linear RGB space. Contrary to popular belief, sRGB is not exactly a gamma 2.2 space, although it is similar.
Photoshop and Adobe Camera Raw are trademarks of Adobe, Inc.
Introduction
Photography is all about capturing an image for presentation either in print or (digitally) on screen. In digital photography a significant amount of image processing is involved in transforming the raw sensor data to a final image. In this article I’ll take a closer look at this data flow, neglecting all things spatial (details, noise) and instead focusing only on the reproduction of color and tone (brightness).
First, I discuss in a general context what one might expect from a digital camera in terms of both color and tone reproduction, compacted into a few conditions. Then, I’ll give a generic overview of the image processing pipeline that is used to convert the raw sensor data into a final image.
I will focus on the use of tone curves in this process. We will see that the way these are implemented in many systems causes a violation of a basic color integrity rule. For cases in which this is undesirable, I propose a new method for applying a tone curve that improves on this shortcoming. Finally, tools will be presented to integrate this approach into an Adobe Raw Converter/Photoshop workflow.
If you’re familiar with color theory and know the difficulties of profiling input devices, you can skip right ahead to the subsection minimal demands on color reproduction.
Color Basics
The colours and shades of light in the world around us exist as an intensity distribution over a spectrum of wavelengths. However, in the computer, colors are represented as tristimulus values. This is usually an RGB triplet, but can also be Lab or HSB. The justification for this reduction to only three degrees of freedom lies in the human visual system itself. There are three types of color-sensitive receptors (cones) in the eye so the visual system only has three signals to work with.
The generally (most) accepted model of the visual system is the CIE-XYZ model that describes the response of the eye together with the first levels of visual processing. It characterizes the visual system by an effective set of three color filters, named x, y and z. The total normalized intensity that passes these three values defines the values of the three components X, Y and Z respectively. The Y channel response is directly proportional to the perception of luminosity. For detailed information regarding this subject, see for example https://en.wikipedia.org/wiki/CIE_1931_color_space
Knowing a certain color’s XYZ response, we can transform it to Lab or some RGB space in an unambiguous way. The colorimeters that are used often for monitor calibration are based on this principle. They have color filters that closely resemble the xyz filters and can therefore measure the human response to a certain stimulus and relate this to the RGB values sent to the screen.
Input Devices
Accurate color management for cameras and input devices in general, is quite complicated, for two reasons. Firstly, the human visual system judges colors with respect to the overall color of the light. This can be taken care of rather well by a white balance correction. Secondly, it is difficult to cover a Bayer sensor with high-quality xyz filters. Remember, a colorimeter, which is essentially a single pixel, costs over 100 euros/dollars already.
The latter problem leads to complications of a more fundamental nature. Using different filters than the xyz filters causes color errors at the input stage. There is nothing you can do to obtain true XYZ values afterwards without detailed knowledge of the input spectra. Therefore, the best one can do in general is to make a ‘best effort’ approximation that renders the most important colors correctly and leaves relations between colors more or less intact. Choosing such a ‘best’ match is a complicated business that is as much an art as it is a science.
Minimal demands on color reproduction
Given the difficulties stated above are all cameras doomed to be a black box with proprietary witchcraft inside™? No. We can still make two statements that should remain true regardless of how the above issues are handled. In fact, statement two is a stronger version of statement one.
| A1. | Two colors with identical spectral distributions up to a total intensity factor, i.e. same hue & saturation, different brightness, should be mapped onto RGB triplets that differ only by a constant factor, in any true gamma space. |
| A2. | The scaling factor between those RGB triplets, when viewed in a linear space representing identical spectral distributions should be equal to the ratio of their intensities. |
These are fairly simple and intuitive statements. It might therefore come as a surprise that these conditions are almost never met in practice. More about that in the section on current tone curve implementations.
Tone
Another major component in image reproduction is tonal mapping. This is a vital ingredient for a number of reasons.
First, there is the problem of dynamic range: the ratio between the brightest and darkest point. Many digital SLRs can capture in the order of 10 stops of dynamic range from the maximum pixel response to the noise floor. Prints, on the other hand, only have a contrast ratio of about 100:1, or approx. 7 stops. LCD monitors are somewhere in between with their 500:1 to 1000:1 contrast ratios. It is clear from this comparison that one either has to select a certain brightness interval or perform some sort of dynamic range compression.
Also, in all photography destined for print or screen, there is the problem of blown-out highlights. Assuming the standard 18% grey level for midtones, you only have 2.5 stops of highlight headroom, implying that some form of highlight compression will be necessary for many subjects.
Finally, there is of course also a need for creative tonal mappings, such as contrast adjustments to increase or decrease contrast compared to ‘reality’. Concluding we can make the following statement:
| B1. | Advanced tonal control, especially for highlights, is essential for (digital) photography. |
The image processing pipeline
Now that we have defined some reasonable criteria for digital imaging systems, let’s take a look at the image processing pipeline from the raw sensor data to the final image. We will later relate this to the conditions stated above. Below is a generic diagram that details the major steps in the processing of raw sensor data from digital cameras, applicable both to the generation of in-camera JPEGs and the development of RAW images.
Note that the following processing scheme is a generalization and may not be an accurate description of every device and raw conversion software package. Deviations from the following scheme do not invalidate the central claim that luminance curves can be used to maintain color integrity, as long as the processing stages before application of the tone curve satisfy conditions A1 and A2.
In the first processing steps a color’s XYZ value is constructed from the camera’s sensor data. This may involve an initial application of curves to linearize the output. The curve changes should be small though, as digital sensors are inherently linear. Then, the linearized RGB values are converted to XYZ values using a color matrix, defining the color response of the camera. For a camera with ideal xyz filters, this would be the identity matrix. For Adobe Camera Raw, this matrix seems to be the place where the sliders on the calibration tab have their effect. The RGB hue & saturation sliders appear to move the corners of the camera’s ‘gamut triangle’ in color space (although digital cameras do not have a ‘gamut’ in the sense that for example printers do).
Next, the XYZ channels are multiplied each by a different multiplier to perform exposure and white balance adjustments. The XYZ data is then transformed by a second matrix and subsequently gamma encoded into an RGB output space, such as Adobe RGB. It should be noted that when the output space is not a ‘true’ gamma space, a function other than a power law is used for the encoding.
It is my impression that in many cases the white balancing step is performed before conversion to XYZ on the RAW sensor data. Then, the RGBcam-XYZ and XYZ-RGBout matrices can be combined into a single color matrix. The results from both methods are not equivalent, but the visual impact of the difference seems to be small since the RGBcam-XYZ matrix is supposed to be close to the identity matrix (i.e. no effect).
Finally, a tone curve is applied to the image, usually compressing the highlight information and applying a contrast boost to the midtones. The tone curve does not have to be applied at the end of the chain. It can be inserted at every point after the exposure and white balance correction.
Current tone curve methods
In the processing flow described above, the color integrity conditions A1 and A2 are satisfied up to the tone curve. In order to serve its purpose, any tone curve must invariably violate condition A2. However, any violations of condition A1 (hue and saturation preservation) are a side-effect of the way that particular tone curve operates. Let’s delve a little deeper into this relation between tone curves and color integrity.
The simplest implementation of a tone curve is a single curve applied to each of the RGB channels separately (the well-known ‘curves’ command in Photoshop). In that sense they mimic their film counterparts, the exposure-density curves that also operate on a per-layer (color) basis. These per-channel curves violate condition A1 due to saturation side-effects that are related to the curvature of the tone selected tone curve. In regions of positive curvature saturation increases and in regions of negative curvature the saturation decreases. A common S-shaped curve therefore leads to oversaturated shadows and undersaturated highlights. The effect of simple curves and other per-channel operations on color integrity is described in great detail by David Dunthorn in the document CFS-243 that is available at http://www.c-f-systems.com/PhotoMathDocs.html.
So, clearly a per-channel curve adjustment is not the way to go and information from different channels needs to be combined. Photoshop examples of this type are the use of a Lab-lightness curve in Lab space or use of the standard curve tool with luminosity blending mode. Unfortunately, although they show a marked improvement over the simple curves, the cross-talk between tone and saturation is still there.
The above options are not exhaustive and some programs use proprietary tone curve algorithms. For example, Thomas Knoll (main ACR author) has commented that ACR uses a “special hue-preserving curve”. Nevertheless, based on my testing of raw converters, I have concluded that in every single program I’ve tried, the saturation side-effect of the tone curve is qualitatively the same as for the simple per-channel curves. The magnitude of the effect may be smaller though.
Where does this leave us? Given the current state of affairs one should consider if a tone curve algorithm that abides by condition A1 is really necessary. Thomas Knoll has remarked in a forum post that is no longer available that he had found that most users prefer the saturation side-effect of current tone curves. If this describes you, the rest of this article will likely not be of much interest to you. However, the situation might be different for those who are less into images that ‘pop’, those who strive for absolute accuracy or those who work with images that will undergo large edits.
The luminance curve
So is it possible to define a tone curve that satisfies condition A1? Yes, it can be done by what I call a luminance curve. A luminance curve maps pixels of a certain brightness onto pixels of another brightness, without changing the ratio R:G:B. It works as follows: every pixel has a luminance value Y. Through the luminance curve, Y is mapped onto another value Y’. Now, the RGB triplet as a whole is scaled such that the resulting triplet has a luminance Y’. Special care should be taken in the highlights to prevent hue shifts due to clipping of channels. Pseudo-code for a specific algorithm can be found on this page.
HDR imaging: future approaches
Such a luminance curve can be used exactly in those situations where a different tone curve was used before, without the added complication of color integrity errors. If a luminance curve tool were to be built into a raw converter, its natural position in the chain would be directly following the white balance correction. There, the image data resides in the XYZ color space, allowing for ‘free’ use of the luminance channel Y.
The luminance curve lends itself naturally to the emerging concept of HDR (high dynamic range) imaging. In HDR images every RGB value is represented by a 32-bit floating point number, thus eliminating the cut-off at the top (for all practical intents and purposes) and delivering nearly limitless dynamic range.
The output of one or multiple combined raw conversion can be stored in a linear HDR format. The application of a luminance curve to this data can serve two purposes. First, to creatively change the brightness relations within the image. The second application would be to select or compress a certain contrast range for use in a final output. For example, for prints and present-day monitors, the highlights need to be compressed into the approximately 2.5 stops of highlight headroom defined earlier. On the other hand, the next generation LED displays will be able to show a much larger contrast range directly on screen. Saving HDR master files with correct or pleasing color information and applying different luminance curves for specific output processes could therefore become part of a regular workflow.
Implementation for Adobe Camera Raw & Photoshop
I hope to have demonstrated that the luminance curve described above offers certain improvements over the existing options. Unfortunately, as of yet, it is not supported directly by any of the major software packages. In this section I will present some practical tips and tools to implement a linearized conversion + luminance curve workflow within Adobe Camera Raw (ACR) in combination with Photoshop CS2.
Modification of the standard workflow to the new workflow requires three steps that are discussed in order:
- Linearization: bypassing the internal tone curve.
- Adjusting the calibration for the new situation.
- Using the luminance curve tool.
Linearization
ACR can be set to produce linear output by setting the brightness, contrast and shadow sliders to zero and selecting a linear tone curve (in ACR 3+). After the linearization, the image at default exposure will look quite dark. It is not necessary to compensate, as this can be done later using the luminance curve tool. In fact, boosting the exposure at this point can lead to an irrecoverable loss of highlight detail.
Technically, in my testing of ACR with the above ‘linear’ settings, I have not been able to distinguish between linearity and the somewhat weaker homogeneity of order 1. The difference between the two is that for strictly linear processing, in addition to the conditions A1 and A2, there is the demand that a mixture between two colors at the input stage produces exactly the same mixture of the output colours. However, this difference is not important in the context of this text. For this reason I have decided to keep using the more familiar word ‘linear’ in the text, although it might not be techically correct.
Calibration
In the linearized state, the default calibration of ACR might be off. The calibration can be adjusted with the RGB hue and saturation sliders on the ‘calibration’ tab. For this purpose, I have modified Thomas Fors’ ACR calibrator script to support linearized calibration. If you have a GretagMacBeth ColorChecker, download the modified script here.
Luminance curve
The final step, the luminance curve adjustment, can unfortunately not be done inside ACR. Instead, I’ve written a pair of Photoshop actions (LuminanceCurve & LightnessCurve) for Photoshop versions from CS upwards that construct a luminance curve tool using layer sets. They can be downloaded from this page. After running the appropriate action, a curves adjustment layer is available to perform luminance (Y) or lightness (L*) adjustments.
ACR’s linearized output tends to look quite dark and it needs to be, to preserve highlight detail. To lift the brightness and compress the captured highlights into a narrower luminance range, simply use a curve that starts steeply and levels off at the highlights. Included in the CurveTools download package is the ACR default curve action, for use in combination with LightnessCurve, that loads a curve that roughly recovers the look generated by the ACR default tone curve. Before using it, import the image through ACR with the same exposure setting you’d also use at default settings.
Generally, for maximal highlight recovery, adjust the exposure in ACR such that the brightest highlight is not overexposed and then pull up the rest of the image from the shadows using one of the curve tools.
Results
Now, after all the reasoning above, let’s talk images. All images below originate from an Olympus E-1 and are converted to sRGB for web viewing.
First, a comparison using a GretagMacBeth ColorChecker chart. On the left you see images processed using the default curve settings in ACR (shadows 5, brightness 50, contrast 25, “medium contrast” tone curve). On the right are images processed with a linear workflow, and application of the LightnessCurve action with the ACR default look tonal preset. Both workflows were calibrated using the modified ACR Calibrator script with fixed tone curve presets (ACR default curve settings on the left, linear presets on the right). The images on the top row are properly exposed, the ones one the bottom row are underexposed by 2 stops using the ACR exposure slider.
Comparing the underexposed images one can see that the standard ACR tone curve leads to a slightly higher saturation in the shadows. The difference becomes more obvious when it happens to ‘memory colors’ such as skin tones. See the following image of Yours Truly taken on a wedding party. The processing was identical to that of the properly exposed ColorChecker images above.
Not only does the image on the right avoid the slightly oversaturated skin tones of the image on the left, it also retains some of the saturation in the sky. Now, I know that both images leave something to be desired, but remember this is using default setings in both worflows. Comparing the maximum quality that both workflows can provide is something you can only do for yourself, due the the subjective nature of quality. The example below is the same image processed with the linear workflow, but this time using a custom luminance curve to lighten up the shadows a little. Generally, I found that the linearized workflow images are better suited to this type of further enhancement, due to their balanced saturation.
Finally, here’s a final example comparing the ACR tone curve with a luminance curve on a high contrast scene. Except for the tone curves that were applied respectively in ACR and the LuminanceCurve layer set, the conversion settings were identical.
Architecture comparison between the ACR tone curve and the luminance curve. Pay specific attention to the highlights and shadows.
In these images, it can be seen that the luminance curve retains some of the blue saturation in the windows on the top right and the red building has a more or less uniform saturation from the shadows towards the highlights. On the other hand, the ACR-curve image loses the blue saturation in the highlights and has a clearly decreasing saturation profile from the shadows to the highlights of the red building.
In this article I have given a general overview of the color processing of raw sensors data in digital photography. I have focused on the ability to achieve correct hue and saturation, independent of the brightness. I hope to have shown you that the current implementations of tone curves in digital imaging are often detrimental in this respect.
A better method, a luminance curve, is possible – and now also available for Photoshop CS2. Unfortunately, the tools presented in this article are only an add-on to the standard ACR+Photoshop workflow. I hope that developers will consider offering the luminance curve as an integrated option in their software.
If you agree, disagree or have suggestions for further improvement of the tools and workflow described in this article, I’d be happy to discuss this with you. If you are a software developer and decide to use these concepts in your programs, I’d appreciate it if you inform me. And finally, if you’re a commercial software developer implementing these ideas, I’d be grateful if you’ll let me play with the resulting software.
Acknowledgements
I wish to thank Peter Lange for asking the tough questions that started this project, giving helpful comments and for his help in testing the CurveTools.
Essay by Simon Tindemans