For my dissertation I’ve been researching how the physical structure of our visual system affects the way we perceive paintings and photographs. I’m been particularly interested in how the cells in the retina make a difference, given that they’re obviously fundamental in how we see. There’s been a fair bit of research done in this area, and a few investigations relate to photography, which is obviously of interest to me. One in particular is causing me some trouble, as I think it’s quite dramatic, but sometimes I wonder whether there’s actually anything that needs an explanation. It relates to our perception of colour.
It’s pretty well established that we have have two visual pathways, fed by different cells on the retina. One works in black and white and works very quickly – this system is older, evolutionarily-speaking, and is shared by mammals. The other works in colour, but is much slower – this system we only share with primates. When you look at the actual cells, you find that those for colour are larger than those for black/white. This means we can’t see as well in colour as we can in black/white – there are fewer cells per area, so we can’t see as much detail.
Obviously, this isn’t something we notice in day to day life, so it seems reasonable to suggest the brain is making up for this difference. We know the brain has a colossal role in how we see the world – for starters, the image on the retina is upside-down, and our brains flip it. When you look at the raw data arriving from the retina, it’s amazing we can see anything at all. So the brain is certainly up to the task. But is this difference in colour resolution something we can detect?
We have no trouble recognising this as a bowl of flowers. Yet when you look, the colours aren’t within the lines. At times they’re way out. But we don’t get confused by this – indeed, we don’t even register it at first. Prof. Livingstone’s hypothesis is that the resolution of colour is so rough that Dufy doesn’t need to colour within the lines – the brain is continually fitting rough colours to the shapes it finds in the much higher resolution black/white system. Here’s another example by the same guy:
The violin players in the lower-left look fine at first glance. It’s only when you look more closely that you realise the red of the violins has no real shape. It’s just a blotch. But our brains fit it roughly to the shape of the violin, and we understand it fine.
The theory relies upon the brain having access to high resolution black/white data, which it fits the colour into. Here’s a greyscale version of the orchestra:
While Raoul Dufy plays fast and loose with the colour, when it comes to black and white everything is much more normal. The shapes are clearly defined – there are obvious lines to everything.
Ok, so far, I reckon this is an interesting theory. My immediate thought is to wonder how much difference cell size can actually make – large colour cells are still, presumably, pretty small. But I’m no expert, and have to defer on this. Let’s assume the physiology of the theory makes sense.
I figured, if high-quality black/white data is important, this effect must be particularly strong with photography. After all, the detail on a photo is far in excess of anything in a painting. So I started playing around. Here’s a photo I found on Flickr that seemed apt. It’s by jonnyr1, and is creative commons licensed:
Let’s separate this into black/white and colour:
Both versions have very fine details. The shape of the violin, his hand, and the windowframes behind him are all clearly defined. I’m going to blur the colour information, then recombine it with the black/white. Here’s the blurred colour:
That’s obviously far less clear. You’d have trouble recognising the shape as a violin, I’d say. Add this to the black/white image and you get:
What do you think? I’d say on first glance you might not even notice anything was up. The violin is the colour of a violin, and nothing about it looks odd. The red/yellow/green stripe on the behatted gentleman’s jacket is still clear. Direct comparison with the first image shows differences, and once you start looking they become more obvious. But for such a huge loss of colour information, it’s still remarkably intact.
Blurring the colour info isn’t quite the same as smearing it outside the lines – the colour fades as it’s blurred. So I think it’s fair to boost the saturation a touch to make up for this:
I’d say this makes it even closer to the original. (in all of the remaining steps I’ve given the same saturation boost).
So how far can we push the blurring?
Now you’re starting to lose the shape of the background windowframes. Everything’s becoming a bit nothingy. The combined image looks like this:
Still not bad. The window frames are most definitely blue – it takes a careful look to see the blue is spreading into the windows themselves. The yellow window is still yellow. The violin is still entirely intact, although the guy’s hands are going a bit odd. Mr Hat’s jacket is still obviously striped. How much further can we push it? For the next step I’ve applied a lot more blurring:
Nothing is recognisable now. It’s all just vague blobs. Combined, this looks like:
Obviously we’re starting to lose a lot of detail now. But the violin is still clear, as are the windowframes and yellow window. The jacket stripes are almost gone, but retain a hint of colour. The guy on the right has barely changed since the initial photo. Yet in the colour data they’re hardly there are all.
What do you think? My worry is: is this impressive? Is this a phenomenon that’s calling out for an explanation?
I’m concerned I’m convincing myself it is, and I need some fresh eyes. It has the feeling of something a philosopher could come along and destroy. The theory of the brain making up for low colour resolution does make intuitive sense, but is it really possible that, if this weren’t the case, we’d be confused whenever colours drifted from their boundaries? I’m not sure how to judge that.