Colour problems with my photographs #1

Friends, stalkers and the easily bored might have noticed that I haven’t uploaded many pictures to Flickr of late. This isn’t because my Year 25 project has stalled – I have the last couple of weeks worth of images ready to go – but because of a problem between Adobe Lightroom and my new camera. Two problems, actually, both related to colour. Here’s the first:

Problem #1 – RAW Colour Deconstruction

Every time a digital camera takes a picture it gets a stream of raw data from its sensor. The camera then converts this data into an image file. Higher-end cameras, though, are capable of saving the raw data so that it can be processed on a computer rather than in-camera. This has a few advantages:

Firstly, RAW files contain slightly more information of the extreme shadows and highlights in an image, so extra detail can be extracted.

Secondly, RAW files allow the white balance to be manipulated after-the-fact. If you hold a white piece of paper under the noon sun, then under a motorway lamp at midnight, you’ll see the same white piece of paper both times. But the lighting is actually very different – it’s obvious that motorway lights are very, very orange compared to daylight. Take a photograph in both circumstances and the digital camera has no way of knowing what colour things ‘really’ are, so it makes its best guess. A standard image file takes the guess, alters all the colours and saves the results. You can manipulate it manually afterwards by pointing out which particular area of the image should be white, but a RAW file skips the guessing part – it lets you say exactly ‘I was standing under a light emitting light of this particular colour, please adapt all colours appropriately’.

Thirdly, RAW files aren’t compressed. Even the highest quality standard image will exhibit signs of compression. Zoom in on a blue sky in a normal digital photo and you’ll eventually see unpleasant blocks.

There are disadvantages, too. For example, RAW files are larger and therefore slower – my camera can take 27 consecutive JPEGs but only 9 RAWs before its buffer fills up. Also, processing RAW files takes time, and needs special software. Here’s where Adobe Lightroom steps in.

Lightroom is a powerful and very capable RAW processor, as well as a library management tool. I think it’s fantastic. It can recover shadow / highlight detail while keeping the rest of the image stable, it can apply changes to batches of images simultaneously and it can edit a photo while rendering a PDF contact sheet and importing from a memory card. I used it for months and eventually paid £200 for a license, figuring I’d use it for years. It is the business.

At least, it was with my old camera.

The problem stems from differing RAW files. My old 300D used CRW files, while my new 400D uses CR2 files. Both RAW formats are proprietary, meaning that the exact structure of the file is known only to Canon. I don’t know whether Lightroom’s programmers reverse-engineer the formats or there’s some other scheme, but either way the result is the same: Lightroom interprets the RAW data in the best way it knows. With the CRW this was spot on, and Lightroom’s processing would produce results as-good-as-if-not-better-than the camera’s own processing (a Canon camera knows exactly how to deconstruct a Canon RAW file to display the optimal image). But the CR2 is broken – the colours just aren’t correct.

It’s most noticeable in the reds. Here’re three different versions of the same holiday scene:

Lightroom Colour Problems - Mr Christmas

On the left is a JPEG produced in-camera; on the right is Lightroom’s interpretation of the RAW file; in the middle is the RAW interpretation by Capture One Pro, a rival to Lightroom. All used the same aperture/shutter speed/white balance/flash power. As you can see, Lightroom is waaaaaay orange compared to the JPG and Capture One. Visually, I’d say the JPG has the most accurate colour rendition, if slightly over-saturated. This makes a big difference in skin tones, and I have a fair few pictures of sickly-looking babies.

Why don’t I use Capture One Pro instead? Because it sucks compared to Lightroom. Also I paid £200 for Lightroom, and I’m not giving it up, so there.

I investigated the issue, and it turned out to be a common complaint with CR2 files. But no easy fixes presented themselves. So, I figured, why not just use JPEGs? In practice the extra exposure data isn’t useful very often, and compression isn’t noticeable in high quality images. White balance can be convenient, but that’s the trade-off to get decent colour. For a while this was exactly what I did.

It was like taking a step backwards. A series of images from a cold New Year walk were just…annoying. The white balance would shift depending on whether the sun was out or hidden by clouds, and in the most extreme pictures people’s skin tones vary wildly. I can fix this in post-production, but only roughly – it’s not like I had people holding a grey card in every shot. Obviously I would have this problem with RAW images too, but I can at least say ‘the average light from a cloudy sky is this colour, please show me the appropriate colours’, rather than having to slide things around until it looks right. Even after fixing there’s still a fair difference between shots taken only a few minutes apart. Also, altering the exposure was far more tricksy. With RAW I can say ‘alter the exposure by one stop up’, whereas with JPEG it’s, again, an approximation.

I found JPEGs far more limiting than I expected. Often the results were great – the camera’s guesses are usually excellent – but whenever I wanted to alter anything I’d get frustrated by the lack of precision. I didn’t like it. So it was back to the RAWs.

Reading up on the topic revealed that plenty of people are having the same problems. But others say: so what, RAW is meant to be more work! I apparently shouldn’t expect perfect results – RAW just gives you a basis from which to start. I should just fix the colours manually in Lightroom and apply the same fix to every photo I import (Lightroom can do this automatically, even limiting it only to photos from a particular camera).

Unfortunately, I cannot for the life of me get Lightroom to match the JPEG colours. It’s more than upping the reds – there’s extra blue in there too, along with saturation differences and blah. I’ve tried pretty hard, and I just can’t match it with Lightroom’s calibration tools. Others have struggled similarly. I can get it not-too-bad, but that’s not good enough – I want it pretty-good.

Some people are very cross about Lightroom’s obvious problems with CR2 files. I admit that it’s frustrating. But, there is a solution. It’s just not cheap.

Solution: Get hold of a Gretag Macbeth ColourChecker chart. This is a 6 x 4 grid of reference colours. Take a photograph of one of these in RAW, and run the Thomas Fors ACR Calibrator Script in Photoshop CS. Because the colours are standardised the script knows exactly what they should look like, and it’s capable of telling Lightroom exactly how to adapt its colours to get the correct results. Brilliant!

But, a Gretag Macbeth ColourChecker chart is £60 (I don’t actually have the £130 Photoshop CS either, although I may have to bite the bullet on that soon as my course will probably require it). £60 on a piece of cardboard is simply unjustifiable at the moment, even if I can use it as a grey card afterwards.

So I’m not sure what to do at the moment. I can run RAW files through a demo of Capture One Pro and manipulate the resulting JPEGs, I guess, but that’s far from ideal. I might put out a call to see whether someone has a ColourChecker chart I can borrow 🙂

I ran into another colour-related problem recently, but I’ll save that for another post.

Update: I found a non-expensive solution! Ben in the comments suggested searching for 400D owners who’d already run the calibration, and something turned up! It’s not completely perfect, but easily pretty-good.