Question #31: What the F*ck is Colour?

There’s a lurking question sitting atop the pile of these posts, which hopefully has been emerging up and out of the swamp. The shambling shibboleth ultimately is one that we are going to need to tackle in order to revisit some of the previously answered questions in a much more valid manner.

I know that I promised that this would be focused on “Getting Sh*t Done”, and that this romp man initially appear as a sidebar. Sadly though, when the client screams at you about colour being “different”, or when you carefully match a swatch using the proper colour space and test a soft proof and the print comes back seemingly “weird”, or when you go out and shoot your moving picture project and lean into some grand “colour management” system only to discover none of it worked… it’s a cold awareness that something is not well in the discussion of “colour”.

Question #31: Just what exactly is colour?

To answer this in any useful way that allows us to walk away with that sort of deeper “intuitive” understanding, we would do well to do a brief jaunt down the neurophysiological chain. Hopefully this won’t hurt too much, and perhaps instead begin a bit of fascination with what we call “vision”.

This is a longer post. In order to tackle the broad ideas, breaking the following out into smaller bite sized pieces would probably end up less than optimal. I hope you will forgive the length…

---

If we start with the retina itself, we can begin to flesh out a high level understanding of colour in terms of where we immediately begin to get things wrong. For example, few folks realize that our retinal assemblies that are capable of sensing radiation are inverted! Here’s a great diagram from Nathan Lents¹.

Now immediately it might seem completely strange that we do not cognize the layers of veins, cells, and other assemblies that are in front of our radiation sensitive layers. This terrific little video might work to let one see the array of biology that is spread across the incoming energy surface of the retina.

The hint as to the why we don’t see our veins and other bits has a research trail that goes back at least as far as 1952 in terms of controlled experiments. Ditchburn and Ginsborg², Riggs³, and later Yarbus⁴ were all exploring this phenomena around the same time.

Through a series of incredibly clever setups, the researchers revealed that if the incoming stimulus was fully fixated to the observer’s eye, that the cognition of vision would completely cease to manifest! If we manage to remove the eye movement out of the equation, and keep the stimulus constant, our biological systems remove all cognition of vision!

To appreciate the mechanics of this strange “no difference – no vision!” engine, we briefly must romp down the primate vision chain. Remember, the following diagram, which is from Retinal Connectivity and Primate Vision⁵, should be visualized as “inverted”; the radiation enters the retina and passes from bottom to up in the diagram!

Now at this point, everyone around these parts is familiar with the whole broad understanding of trichromancy; three signals blah blah. Sadly, this is also exactly where our understanding of human colour goes off the rails.

Note those bipolar cells in the diagram above? These cells are responsible for differencing the basic electrostatic signals generated at the cones, against each other. This differential is achieved via what is commonly referred to as an inhibitory mechanism. Key takeaway however, is that the all too familiar “Long”, “Medium”, and “Short” (sometimes labelled “Protan”, “Deutan”, and “Tritan”) absorptive profiles never make it out of the retina as an “optic signal”, a term borrowed from D. B. Judd⁶. To be more forceful in an explanation, the idea that the Protan, Deutan, and Tritan absorptive profiles that generate electrostatic signals appear to never⁷ be propagated beyond the cone cells to the differential inhibitory mechanisms of the On-Off and Off-On assemblies!

---

So what then is something closer to what the “optic signal” may be? The “truth” is much slipperier and murkier than some would have us believe. There’s no clear demarcation line in the messy world of biology, and this is doubly so with respect to visual cognitive pathways. What is generally accepted is that the differenced signal follows three broad assemblies:

1. P+D: The Protan is combined with the Deutan absorptive profile signals.
2. P-D: The Protan is inhibited by the Deutan absorptive profile signals.
3. (P+D)-T: The Protan is combined with the Deutan absorptive profile signals, and inhibited by the Tritan absorptive profile signal.

I’ve chosen to use the Protan, Deutan, and Tritan terms here to avoid a typical over-ontologizing by way of the terms “Long”, “Medium”, and “Short”. The reason is that if we hear “Long”, we lose sight of the fact that the absorptive sensitivities of all of the cellular assemblies wrap the entire visible radiation range, to varying or lesser degrees. If one assumes that the “Long” only detects “Long”, it’s a precarious entry point toward “understanding” if there is any “understanding” to be had at all.

Worse, it’s also worth noting that the Protan and Deutan absorptive characteristics differ only by approximately 30 nanometers! That is, the absorptive profiles are more or less similar, “offset” from each other. Think of it as a “stereo” signal version of the attenuated radiation. I did up this somewhat clunky animation to showcase how the attenuation of energy profiles are “wrapped” around the spectral locus. This animation abuses the CIE xy diagram “map”, which in and of itself is an abuse of the actual “territory” of human visual cognition.

Phew… so where does this leave us? The reason that an understanding of how this differentiation mechanic may arise echoes back to the Yarbus work; the totality of what we call “vision” is utterly dependent upon the manifestations of differences of signals. That is, Judd’s “optic signal” term is not at all “quantities” of attenuated radiation as many folks erroneously infer!

---

At this point dear reader, I want to stress that if you hear someone say “The human eye is like a camera” they are talking directly out of the hole that exists just after their colon. It’s pure, unadulterated farm animal excrement. Alan Gilchrist has coined this construct The Photometer Metaphor⁸:

Implicit in most theory and research on the perception of surface lightness is a conception we would call the photometer metaphor. This term refers to the assumption that, fundamentally, the visual system measures the intensity of light reflected by each point in a visual scene.

Gilchrist, Delman, and Jacobsen – “The Classification and Integration of Edges as Critical to the Perception of Reflectance and Illumination”

If we follow the basic anatomy of the retinal tissue in our primate systems, we can get a high level understanding as to just how wrong this metaphor is, and why it inhibits, no pun intended, deeper understanding. How we ontologize and break down our metaphors influences our inferences and subsequent understanding. From the rather unfortunate “Long”, “Medium”, and “Short” terminology, to the even worse “Red”, “Green”, and “Blue” leap which projects centuries old false ideas of the locus of “colour” existing in the radiation, to the Photometer “The eye is just a camera” noodling nonsense, we see structures of knowledge that undermine a much more nuanced, and likely more tenable, model.

The differentiation that happens somewhere along this signal chain can be broadly broken down into what is generally regarded as ON and OFF signal chains. That is, not only is there a complex web of inhibitory mechanisms at work to differentiate between the absorptive profiles of the first stage radiation sensitive cells, there’s also a distinction in terms of an increment versus decrement signals⁹. The types of assemblies that formulate these distinct increment versus decrement signals are varied. Some are considered “simple” cells; which respond strictly to a full assembly increment versus a decrement signal, and others are considered “complex”, which have a central region that differentiates spatially from the outer region, as well as orient specific assemblies that perform similar differentiation. It should go without saying that the mechanics of the visual system are ridiculously complex!

---

At this point, a simplified understanding of the system as a “Differencing Machine” that smashes up and discretizes the visual arrangement of the retinal array into overlapping and differenced spatiotemporal arrangements is useful. The majority of the research leans toward a model that doesn’t “measure” light as a camera sensor does, providing signal “quantities” to our higher orders of brain activity. Instead, the basic research done by cat and monkey torture experiments, indicates that the totality of the signals are differential in nature. To appreciate this, the canonized work of Hubel and Wiesel¹⁰ is elegantly summarized in this terrific short video:

To wrap this massive romp up, let’s consider how the receptive fields of the differencing machine are arranged. There is evidence to suggest that the various cells are “pooled” and differenced against each other. This pooling happens at various levels of the system, from the ON-OFF / OFF-ON assemblies, to the amacrine and horizontal cell influence, to the ganglion cell assemblies, among other regions not well understood or even potentially known. To give us an idea as to just how complex the receptive field layout is, this diagram from Das and Gilbert¹¹ illustrates the overlapping of various receptive fields in panels d, e, and f:

---

Let’s try to give ourselves a summary of the impossibly complex biological insights we have reason to believe are broadly useful:

• The retinal assemblies that are able to respond to radiation are arranged backwards with respect to the incoming radiation, and underneath layers of veins and what not. The mechanic that renders these veins invisible is critical to note.
• The retinal absorptive signals are differenced against each other in complex ways, in varying sized receptive field constellations. Following the above, without a difference, visual cognition ceases. Fixed veins and other points of stasis cannot be easily identified if at all.
• The retina is not at all like a camera or photoreceptor. Again, this follows due to the biological nature of the ON-OFF and OFF-ON basic mechanics.
• The differenced signals traverse up the stream and somehow, break apart the signals into notions of “boundaries” and “fills”¹², involving some degree of resonance / diffusion-like mechanics¹³.

All of this is to say that even though we have a bit of a better than poorly informed grasp on the physiology, we still haven’t identified the locus of colour. It would be erroneous and utterly foolish to suggest:

• The absorptive profiles of the lowest order assemblies are “red”, “green”, and “blue”. The signals are just that, signals. Notions of colour are not yet formed.
• Following this, the idea of any fixed colour orientation in relation to the neurophysiological signals is equally foolish; colour exists as an active, cognitive process from higher order interactions.
• That colour is a fluid dynamic of cognitive analysis of relationships. Colour is much closer to a cognitive compass-like heuristic, than a fixed ground truth relationship.

We can get a sense pretty quickly that every time someone utters the damn word “colour”, they are probably using it wrong, despite it being an intimately understood thing to everyone reading this. Why?

As has been repeated for nearly two hundred years of more or less “rigorous” attempts to define “colour”, the locus of colour has all-too-often been placed in the extremely limited range of external energy we can “detect”. This is likely the ground zero of the contemporary comprehension disaster around “colour”.

Why? Because the differencing system appears to use a plethora of parallel mechanics to assemble “colour”. This passage from the foreword to Alan Gilchrist’s Seeing Black and White¹⁴ is a great summation:

The second reason for using multiple parallel heuristics is that it buys you tolerance for noisy images. It’s a bit like two drunks striving to reach a goal. Each will stumble if he tries on his own (“illusions”), but by leaning on each other they can stagger toward the goal. That’s perception in a nutshell.

V. S. Ramachandran in the foreword from Alan Gilchrist’s Seeing Black and White

That is, we know that the neurophysiological signals are divided into an increment and decrement direction, and therefore we can at least leave the door open that the relationship between differences seems to be of incredible importance heuristically, not the energy “quantities”. Part of this mechanic is likely helpful to deducing boundaries, but there are other mind boggling examples of how an increment versus decrement signal can provide us with further meaning. A “glint”, “glare”, “reflection”? Increment vs decrement. A “haze” where the differences are gradually collapsed together? Increment vs decrement. Analysis of what is a “figure” and what is a “ground”? Increment vs decrement. And on and on…

One of the more striking series of examples of how this increment and decrement polarity shifts our entire cognition comes from Anderson and Winawer¹⁵, and Anderson and Khang¹⁶. In the following picture, the “shapes” are identical tristimulus. But note that our cognition of what is what shifts wildly between the variations, where only the “ground” is adjusted to shift the polarity.

The effect is even more pronounced when we consider something that generates a chromatic response. This from Anderson and Khang:

Finally, note that these orientations of articulations shift the cognitive meaning that forms. Not only are the above “circles” cognized as different colours, the articulations of colour and increment versus decrement signals seems to arrange our understandings of “transparency”. Again, another from Anderson and Khang:

One could make a case that entire generation has become so seduced by the century old idea that stimulus carries the colour so as to be deemed entirely blind to the more deep mechanics of cognition. Here is one final example from Ekroll and Faul¹⁷. The “discs” are identical tristimulus, and have varied only in their orientation of rotation. Yet in one case, we cognize not only a subtly different hue, value, and chroma, but also a very different cognition of transparency versus occlusion. Pay attention to the increment versus decrement polarity of the tristimulus relationships for a hint that might elucidate intriguing patterns!

---

Hopefully we can, by way of negation, arrive at a semblance of what colour can be adequately described as, via a tear down of what we know colour can not be. From the latter demonstrations above, we can get a keen insight that colour is not some “sensing” of stimulus, as the articulations of the tristimulus seem to be arranged into higher order cognitions. Nor is it “capturing”, as we know from the biological differential signals via the high level understanding of the physiology. In fact, our visual system is so ridiculously focused on such a tremendously small range of electromagnetic radiation that it is almost absurd.

Under a wider lens, we can begin to carve out an idea that colour is an active biological, organism-centric mechanism. So…

Answer #31: Colour is not in the spectral radiation of the articulations, but rather an incredibly complex active cognition mechanic.

That is, here at Question #31, we’ve managed to totally torpedo a sizeable number of the prior answers in no small way. But in doing so, we are able to look down and kick away the ladder that allowed us to get to where we are now, and tackle further concepts ahead of us…

---

1. Lents, Nathan H. Human Errors: A Panorama of Our Glitches, from Pointless Bones to Broken Genes. Boston: Houghton Mifflin Harcourt, 2018. ↩︎
2. Ditchburn, R. W., and B. L. Ginsborg. “Vision with a Stabilized Retinal Image.” Nature 170, no. 4314 (July 1952): 36–37. https://doi.org/10.1038/170036a0. ↩︎
3. Riggs, Lorrin A., Floyd Ratliff, Janet C. Cornsweet, and Tom N. Cornsweet. “The Disappearance of Steadily Fixated Visual Test Objects*.” Journal of the Optical Society of America 43, no. 6 (June 1, 1953): 495. https://doi.org/10.1364/JOSA.43.000495. ↩︎
4. Yarbus, Alfred L. Eye Movements and Vision. Boston, MA: Springer US, 1967. https://doi.org/10.1007/978-1-4899-5379-7. ↩︎
5. Lee, Barry B., Paul R. Martin, and Ulrike Grünert. “Retinal Connectivity and Primate Vision.” Progress in Retinal and Eye Research 29, no. 6 (November 2010): 622–39. https://doi.org/10.1016/j.preteyeres.2010.08.004. ↩︎
6. Judd, Deane B. “Response Functions for Types of Vision According to the Muller Theory.” Journal of Research of the National Bureau of Standards 42, no. 1 (January 1949): 1. https://doi.org/10.6028/jres.042.001. ↩︎
7. Barlow, H. B., and W. R. Levick. “Changes in the Maintained Discharge with Adaptation Level in the Cat Retina.” The Journal of Physiology 202, no. 3 (June 1969): 699–718. https://doi.org/10.1113/jphysiol.1969.sp008836.
   ↩︎
8. Gilchrist, Alan, Stanley Delman, and Alan Jacobsen. “The Classification and Integration of Edges as Critical to the Perception of Reflectance and Illumination.” Perception & Psychophysics 33, no. 5 (September 1983): 425–36. https://doi.org/10.3758/BF03202893. ↩︎
9. Whittle, Paul. “Increments and Decrements: Luminance Discrimination.” Vision Research 26, no. 10 (January 1986): 1677–91. https://doi.org/10.1016/0042-6989(86)90055-6. ↩︎
10. Hubel, D. H., and T. N. Wiesel. “Receptive Fields of Single Neurones in the Cat’s Striate Cortex.” The Journal of Physiology 148, no. 3 (October 1, 1959): 574–91. https://doi.org/10.1113/jphysiol.1959.sp006308. ↩︎
11. Das, Aniruddha, and Charles D. Gilbert. “Distortions of Visuotopic Map Match Orientation Singularities in Primary Visual Cortex.” Nature 387, no. 6633 (June 1997): 594–98. https://doi.org/10.1038/42461. ↩︎
12. Grossberg, Stephen, and Dejan Todorovic. “Neural Dynamics of 1-D and 2-D Brightness Perception: A Unified Model of Classical and Recent Phenomena.” Perception & Psychophysics 43, no. 3 (May 1988): 241–77. https://doi.org/10.3758/BF03207869. ↩︎
13. Cohen, Michael A., and Stephen Grossberg. “Neural Dynamics of Brightness Perception: Features, Boundaries, Diffusion, and Resonance.” Perception & Psychophysics 36, no. 5 (September 1984): 428–56. https://doi.org/10.3758/BF03207497. ↩︎
14. Gilchrist, Alan. Seeing Black and White. Oxford University Press, 2006. https://doi.org/10.1093/acprof:oso/9780195187168.001.0001. ↩︎
15. Anderson, Barton L., and Jonathan Winawer. “Image Segmentation and Lightness Perception.” Nature 434, no. 7029 (March 2005): 79–83. https://doi.org/10.1038/nature03271. ↩︎
16. Anderson, B. L., and B.-G. Khang. “The Role of Scission in the Perception of Color and Opacity.” Journal of Vision 10, no. 5 (May 1, 2010): 26–26. https://doi.org/10.1167/10.5.26. ↩︎
17. Faul, F., and V. Ekroll. “On the Filter Approach to Perceptual Transparency.” Journal of Vision 11, no. 7 (June 9, 2011): 7–7. https://doi.org/10.1167/11.7.7. ↩︎