Royal Conservatoire Antwerp
In the last century, scientists have discovered that some people can in fact see colours and visuals when they hear music, through the neurological condition of synaesthesia. In this article we will compare the ways in which some composers have used their synaesthesia in their music with the concept of cross-modality and our collective tendency to associate certain colour types with certain musical ideas.
In de jongste eeuw hebben wetenschappers ontdekt en met feiten kunnen ondersteunen dat sommige mensen kleuren en beelden zien wanneer zij muziek horen, met name door het neurologische verschijnsel 'synesthesie'. In dit artikel vergelijken we de manier waarop enkele componisten synesthesie gebruikt hebben in hun muziek met de notie van ‘cross-modaliteit’ en onze collectieve neiging om welbepaalde kleurtypes te associëren met muzikale ideeën.
In everyday language, it does not strike us as strange that we sometimes describe senses and concepts by using a word from another sense. For example, a sound can be described as dry, a sight as sweet, or the truth as bitter. We do intuitively know that the meanings are not literal (How can a sight taste anything?), but we do not need to think on these metaphors to understand their meaning. This phenomenon extends to how we talk about musical concepts. A note can be high or low, or the sound of an instrument bright. Again, a note cannot be physically high, but it can be high pitched. At certain points the use of these inter-sensory metaphors goes deeper than our language and into our perception. We do indeed perceive higher pitched sounds as physically higher 1 and brighter 2.
The idea that other senses might be connected with our musical experience is, of course, not new. Indeed, this idea has appeared over and over again in history, from as early as Ancient Greece when Aristotle equated scales of colour to scales of notes, to the multimedia art of the modern era 3 While not always agreeing on how exactly this relationship functioned, the notion that it existed stayed constant. There have even been people who claimed that they could see sounds in certain shapes and colours. In respect to the colour-sound relationship, it is only in the last century that the definition got more specific than coloured hearing. Thanks to medical and scientific advancements, it is now possible to identify people who do indeed see colours when they hear music. This ability is categorised as synaesthesia, a neurological condition in which a person involuntarily experiences two or more sensory perceptions from a single sensory input.
This condition manifests itself in many variations. There are cases of people seeing black and white words coloured, ‘smelling’ words, ‘tasting’ shapes, and ‘seeing’ music. In people with synaesthesia, these associations with other senses are always fixed and constant. They always experience the same sensations and connections throughout their whole lives.4 The associations are personal and idiosyncratic; thus one person may see the note ‘D’ as red, and another as blue. There has been considerable discussion about the prevalence of synaesthesia, but recent estimations are around 1 in 23 for any type of synaesthesia, and 1 in 100 for sound-to-colour synaesthesia. 5
Examples in the field of music led me to investigate the phenomenon of synaesthesia in composers. Through a review of their writings on the subject, analyses of their synaesthetic compositions, and interviewing the synaesthetic composer Michael Torke (°1961) 6, I could establish common points between synaesthesia and cross-modal and non-synaesthetic associations people make with regard to sound and colour. These findings form the basis for further interdisciplinary research into the application of colour-to-sound associations in the field of musical creation and performance.
There have naturally been notable composers with sound-to-colour synaesthesia. However, it becomes progressively harder to ‘diagnose’ synaesthesia the further we go back in time and the less contact with the composer we have. It is important to note here that due to the quite recent distinction between coloured hearing (which is any kind of association between sound and colour) and synaesthesia (which is a neurological condition), we are only able to fully confirm the existence of synaesthesia in the twentieth and twenty first century. While it is interesting that Franz Liszt (1811-1886) has said ‘That is a deep violet, please, depend on it! Not so rose’7 at an orchestral rehearsal of his work, this is not proof of his synaesthesia. We would need evidence of his fixed associations and consistency over time in order to make such a claim. Which, if it existed, is unfortunately lost to history.
Luckily, we do have sufficient evidence for the synaesthesia of other composers, such as Olivier Messiaen (1908 - 1992), Alexander Scriabin (1871 - 1915), and Michael Torke. Messiaen had synaesthesia between musical scales and colour. He saw musical scales, and especially his modes of limited transposition as combinations of colours and shapes. He explains it as such:
Mode 2 is thrice transposable, so it has only three possibilities of coloration. For me, the first transposition of Mode 2 is defined like this: “blue-violet rocks speckled with little gray cubes, cobalt blue, deep Prussian blue, highlighted by a bit of violet-purple, gold, red, ruby, and stars of mauve, black and white. Blue-violet is dominant.” The same mode in its second transposition is totally different: “gold and silver spirals against a background of brown and ruby-red vertical stripes. Gold and brown are dominant.” And here’s the third transposition: “light green and prairie-green foliage, with specks of blue, silver, and reddish orange. Dominant is green.” 8
These colourful descriptions of his modes sometimes appear in his scores and related material, such as his Préludes 9 and Des canyons aux étoiles….10 His description of the Préludes as ‘studies in colour’11 suggests that he used his synaesthesia as a source of inspiration for at least several of his works in which his visual colour perception of sound dictated some of the compositional preferences and choices. Sometimes his synaesthesia was the direct source of the musical content, such as “Bryce Canyon et les rochers rouge-orange”12 in the eighth movement of Des canyons aux étoiles…, where the use of his modes and chords related to the colours red and orange, depicting the rocky landscape of Bryce Canyon.
Scriabin has had (or was influenced by) synaesthesia in a similar manner as Messiaen.13 He also had strong associations between musical scales and colours. He has systematically categorised his relationship between the circle of fifths in music and the colour circle in visual arts to create a synaesthetic system for depicting inter-sensory relations. Perhaps the clearest example to his synaesthetic relationship between colour and sound is his final work Prometheus: Poem of Fire composed for piano, orchestra, choir, and the invention of a clavier à lumières 14, a colour organ that emitted coloured lights instead of sound when keys were pressed. Regrettably, the specific design of the colour organ has been lost, so we do not know the specifics of how the colours indicated on the score would be used.
Michael Torke is a contemporary composer from Milwaukee, America. He has followed musical studies at the Eastman School of Music in composition and piano performance. His musical style can be described as post-minimalist, with influences from the romantic, pop, and jazz canon. As one the few synaesthetic artists whose condition is documented, I have had the chance to talk at length with him on how his synaesthesia functions, and how he has applied it to his own works.
Torke’s synaesthesia also has to do with musical scales. He sees each key tonality as a certain colour. While individual notes also trigger a colour response, he has said that these responses are ‘based on their relations to the key tonality’. His experience of hearing the colour is described as ‘a transparent sheet over the eyes’ on top of what he is seeing. For him, this key tonality to colour relationship sometimes extends to musical modes and other scales, but the experiences with those are not as common and reliable compared to key tonalities. Listening to music with no tonal content, such as atonal or serial music, makes him see a ‘grey-ish’ colour, which he describes as ‘not having any colour in it’. His experiences of synaesthesia are very closely linked with his perfect pitch, both of which he has possessed as long as he can remember. In fact, if he mistakenly registers a sound as the wrong pitch, the colour he sees is associated with that pitch, instead of what is actually sounding. He has based his collection of orchestral works Color Music on these synaesthetic associations; the name of each piece corresponds to the colour that matches with the key tonality. For example, “Bright Blue Music” is in the tonality of D Major, and “Purple” in F# Major. Each of these pieces stays in a certain tonality throughout the whole movement. Other than these pieces, he has not utilised his synaesthesia in an artistic or inspirational manner, choosing to turn to other sources for inspiration. He does not think of his synaesthesia as something that adds to or subtracts from his compositional presence and treats it as a personal characteristic.
Looking at the different associations by each of the composers shows the idiosyncratic nature of synaesthesia. This does not yet present us with meaningful information about what synaesthesia might mean for non-synaesthetes. Indeed, when collected together, the musical key-to-colour associations are different, even sometimes clashing. For Messiaen, the key of D Major was the colour green, while for Scriabin it was yellow, and for Torke blue. This naturally raises some questions. How can the colour perception of the same musical key be so wildly different in these synaesthetic composers? Are there differences in how they associate other musical concepts with colours? What does this mean for our perception of sound-colour associations in a more general sense? To answer these questions, we need to look at how non-synaesthetic people perceive the sound-colour relationship.
If we are thinking about cross-modal associations, the associations between two senses of regular people, we have to distinguish their association processes from the involuntary responses of synaesthetes. People with synaesthesia always see the colours (or other modalities) as a response, without any conscious effort. In regular people, however, making an association between a colour and a sound is a conscious process. A person would have to be asked to think about which colour would match a given sound. As a result of this, the associations made are very sporadic and irregular when compared to that of synaesthetes. For example, during an experiment by Jamie Ward, synaesthete LHM reported consistent red-to-yellow hues when presented with a piano recording of rising fifth intervals, while non-synaesthete CE picked the colours grey, blue, purple, and green when asked to match them to the same recording.15 Despite this irregularity in the associations, further experiments expose some consistent trends in sound-colour associations, both synaesthetic and non-synaesthetic. In the same experiment, the collection of synaesthetic and non-synaesthetic associations reveals the inclination to map ascending notes to brighter and warmer colours. The approaches employed by both groups remain the same in making these associations.
Stephen Palmer’s experiment about music-colour associations of non-synaesthetic people reveal a tendency to map warmer colours, such as red, orange, or yellow onto music that is fast or in a major tonality or both. In a similar fashion, music that is in a minor tonality or slow or both gets mapped to colder colours, such as blue, violet, or indigo.16 In another experiment by Erin Isbilen, when listening to Johann Sebastian Bach’s “Well Tempered Clavier”, the colour responses of synaesthetes were consistent with the colour associations of non-synaesthetes.17 In a similar manner, consistent colour choices of people in different music genres were shown in the experiments of Kelly Whiteford, covering a wide range of genres such as Classical, Funk, Reggae, and Dubstep.18 An inclination to synaesthetic metaphors in perception and language have also been shown in children through the findings of Marks Lawrence.19 These experiments suggest the same conclusion: that there might be an underlying mechanism tying together the associations of synaesthetes and non-synaesthetes.
The fact that non-synaesthetes need to use a conscious effort to make such associations does not mean that their perception has to be forced, however. Even though we have to make an effort in order to create such connections, the recognition of patterns happen automatically. When presented with a collection of paired colours and sounds, we gravitate towards certain selections that we label as making sense or aesthetic. The aesthetic pairing of the senses is also consistent with synaesthesia. Jamie Ward’s research into this subject has revealed that non-synaesthetic people also prefer synaesthetic perceptions as more pleasant and aesthetic.20 Once again, an underlying mechanism that connects synaesthesia and cross-modality shows itself. This, of course, is not only through the physical attributes of the colour such as the the photons that make up the physical light, but also via the symbolic and the cultural meanings. For example, let us think of the colour red. The photon frequency of the colour generally applies more heat compared to the photon frequencies of other colours, and we associate the colour itself intrinsically as warmer.21 We do in fact have a biologic response to the colour red. In a dominantly red environment, we perceive our body heat as higher. 22 And in a similar manner, we perceive objects that are coloured red as warmer in contrast to objects of the same heat but of a different colour.23 In this way we can think of the response to the colour red as more, or more intense. Going further, cultural associations with the colour red also relate to these concepts. The colour of the blood and flesh, red is often associated with violence, adrenaline, and sexuality. It is no coincidence that the presence of the colour red itself increases men’s attraction to women.24
A musical piece that keeps getting faster and faster, or a harmonic progression that becomes more and more intense and dissonant as it goes on can be interpreted as a motion towards warmer colours.
But what do these associations mean for our perception of music? At this level we have to broaden our definition from colour red to warm colours, as in music these associations are related to a group of colours. Going in line with our intrinsic, symbolic, and cultural associations, we can say that warmer colours have to do with musical concepts that are more or more intense. As mentioned earlier in Stephen Palmer’s experiment, musical concepts such as higher tempi, faster rhythms, and major tonalities are already shown to create such associations in people. To this list we can add the concepts of dissonance, irregularity, and relatively complex structures with regard to musical form. All of these factors do not have to be present all at once for a warmer association. A few of these musical concepts should evoke a relation. The specifics of this association, such as the precise hue of the associated colour, or the amount of warm musical concepts needed to trigger such a connection, is subjective and dependant on the musical background and the expectations of the listener. However, the cross-modal connections between colour and sound still stand.
We can easily turn this connection upside down. In the same experiment, cold colours have been associated with lower tempi, slower rhythms, and minor tonalities. We can thus further associate them with consonance, regularity, and relatively simpler structures. Given the temporal nature of musical concepts like harmonic progression, rhythm, and tempi, such associations occur not only through the existence of these musical concepts within a piece, but also through the movement from one concept to another. For example, a musical piece that keeps getting faster and faster, or a harmonic progression that becomes more and more intense and dissonant as it goes on can be interpreted as a motion towards warmer colours. (illustration 2, A simple diagram of the musical concepts we associate with the hues in the colour spectrum. Note that not all of these concepts have to be present all at once to trigger the association.)
Going back to our synaesthetic composers, it becomes apparent why the key tonalities and scales are not consistent between each of them. Tonality is not part of a graduated system, it is circular, as suggested by the circle of fifths. Therefore, it does not make sense to claim that the tonality of D Major is more or less than C Major. Their linear placing on a keyboard puts them into an order, but their execution in music, their musical placement by composers, is not linear. A D Major piece can be harmonically simple or complex, rhythmically fast or slow, containing high or low pitches, or a mixture of any of these. And naturally, the colour and visual associations of this D Major piece would be dependant on these factors. It is not the existence of D Major, but the usage of it that creates synaesthetic associations.25 The associations of our synaesthetic composers might have appeared and solidified in childhood, such as Torke partly attributing his colour associations to a coloured toy piano he had as a child.26 This might partly explain why the same key tonality can have different responses from synaesthetic people.
The synaesthetic associations of pitch to colour in some people also fall into the same category.27 While we can place pitches on a linear scale, the concept of pitch classes are circular, and thus the synaesthetic perception of it does not (usually) manifest itself in a gradient of colours. It is, however, interesting that when placed on a linear scale, the synaesthetic perception of pitches immediately relate to cross-modal associations. As explained by Messiaen:
However, when I move the same chord from midrange up one octave, the same colour is reproduced, shaded toward white – which is to say, lighter. When I move the same chord from midrange down one octave, the came colour is reproduced, toned down by black – which is to say, darker. 28
The colour-to-sound synaesthetic experience thus relates to cross-modal associations more and more as we present the sound information in a quantifiable manner. We can hypothesise therefore that synaesthetic composers incline towards musical concepts that are related to the colour information in a cross-modal way when taking their synaesthesia as the subject. We need to not just look at the colour-sound associations of these synaesthetic composers, but also at how they manifest themselves within their pieces, consciously and unconsciously.
As a part of my research, I have analysed Torke’s Color Music, and the movement “Bryce Canyon et les rochers rouge-orange” from Messiaen’s work Des canyons aux étoiles. The analysis is based on the three musical aspects of sound; pitch, loudness, and timbre. For pitch, the harmonic material in relation to the used scale or mode and the movement within and between pitch groups are analysed. For loudness, the scope of the dynamic movement (the quietest and loudest dynamic indications) and the movement between them are investigated. The instrumentation and the use of different instrument groups are inspected for the timbral content. The results are then compared to the colour that they synaesthetically relate to, and our collective cross-modal tendencies.
Looking at the works of these composers in this mind-set reveals several inclinations. In Torke’s collection of orchestral pieces Color Music, we can see “Green Music”, “Ecstatic Orange”, “Purple”, and “Bright Blue Music” standing out as pieces that are related to his colour associations the strongest.29 Taking our reactions to cold and warm colours in mind, we can order the colours of the pieces as such:
(Coldest) Purple-Blue-Green-Orange (Warmest)
The coldest piece, “Purple”, immediately shows characteristics that may have a cross-modal association with cold colours. Starting with a soft, quiet passage, its tempo is relatively slower when compared with the other pieces in the collection. An eighth note groove dominates the piece, the constant string of eighth notes creating a static effect, as there are many few stops in between. This regularity creates a flow where there is not much tension within the rhythmic movement. The harmony stays mostly in the F# Major chord, with any deviations going very quickly back to the main chord. The relatively tension-free and consonant nature of the piece is in line with the cross-modal associations related to this colour, in both a synaesthetic and non-synaesthetic manners. The bright in “Bright Blue Music” automatically sets our expectations for a bright characteristic, shown to be represented by high pitches, intervals, and movements to high pitches. We can clearly see these within the piece by the ostinati woodwinds and percussions, creating a minimalistic layer of these high pitches and intervals, and the high doubling of melodies by the xylophone and violin. Any downward melodic motion is counterbalanced by a grander movement upward, to a ‘brighter’ area. The minimalistic repetition of the same rhythms and the I-V-I progression in the key of D Major creates once again a harmonically static setting, fitting to its colour association.
Getting warmer, “Green” opens with a tension-filled V chord, resolving to a very fast repetition of I-IV-V-I progression in E Major that repeats itself many times throughout the piece. Yet again there is an eighth note groove, but it is notably faster and more tension-filled than its “Purple” counterpart. This groove also contains a variety of V-I progressions, disturbing the flow that would have been otherwise established. In the harmony, while it still remains relatively simple, we can see the extra tension created by the focus on the dominant V chord, contrasting the colder pieces in the collection. This play on the consonance and tension is explained by the composer:
The repeated and often frenzied use of these basic consonant building blocks creates a kind of mounting tension, which ironically finds relief in the more harmonically complex second group, where dissonant string chords are used in an expressive, but more relaxed way. This result is the opposite of the fundamental law of harmony: that dissonance creates tension and consonance relieves tension.30
The final piece, “Ecstatic Orange”, is distinctly different than its ‘colder’ counterparts; there is a noticeable focus on dissonance, the tonal focus is reduced, and the regular groove is exchanged for explosive and bright fragments on the same pitch material. The tempo is fast, in the range of “Bright Blue Music”, but any semblance of consonance and regularity is immediately disrupted. The pitch material is still static, as the main idea of G#-A-D-C#-B-E evokes an orange colour for Torke, but its application becomes more dissonant and tension-filled. The form of the piece is similarly fragmented, as there is little focus on development and instead the idea of one fragment after the other is realised. Once again, this dissonance, tension, and irregularity matches up with our perception of warm orange.
This progression of the musical concepts, while apparent, was not the composer’s direct intention. When I have asked him about the contrast between the pieces during our interview, Torke has said that he did not in fact consider these details when composing, and that he used the colour as a metaphor of his personal associations and the static tonality of the pieces. He agreed that such a musical colour-progression does exist in the pieces, and he was surprised and intrigued by it.26 The clear existence of these elements in his pieces that directly relate to his synaesthesia suggests that cross-modality plays a role in aesthetic decision-making for music when other senses are involved, and that the role of a composer’s synaesthesia may correlate to more than one musical dimension.
A similar relationship between colour-musical material can be seen in the eighth movement of Messiaen’s work Des canyons aux étoiles, titled “Bryce Canyon et les rochers rouge-orange”. In the piece, the composer uses his synaesthetic associations with the colours red and orange to paint a picture of the Bryce Canyon, a gigantic rocky canyon located in Utah, U.S.A. The utilisation of the composer’s synaesthesia is more direct and visible here, as we have information about how he saw each of his ‘modes of limited transposition’ through his writings on the matter. In the movement, he extensively uses his musical modes 3-1, 3-4, 6-2, and 4-6, all of which he synaesthetically relates to combinations of the colours orange, red, and milky white. The colours for these modes are not his inventions for this piece, but concrete associations that have been with him throughout all of his life. The movement is in four parts, modelled after the Ancient Greek choral ode; Strophe, Anti-strophe 1, Anti-strophe 2, and Epode (Coda).31 There is a focus on his third and sixth musical modes in the strophe, and on his fourth musical mode in the anti-strophes. The movement has many quasi-cadences in E Major, which Messiaen has been shown to associate with the colour red. In the score, many musical concepts and movements that relate to his associations exist. After the movement opens with a piano solo, we are presented with a pianissimo cluster chord played by the whole orchestra that slides up to a higher pitched fortissimo cluster chord. The ideas presented here, movement from pianissimo to fortissimo, cluster chords, and sliding movements to higher pitches all repeat themselves throughout the piece. There is a focus on tutti in the orchestra, with the instrument groups splitting only when a special attention to a musical mode is made. Once again, all of these musical ideas directly relate to our perception of what these colours might musically mean. The musical modes and scales used by the composer create his personal cross-modal connection with the subject, and his method of using them in his music allows for our cross-modal connections.
Our intrinsic relationship between sound and colour reveals new avenues for research and discussion. How deep does the synaesthetic composers’ experience run within their music? Are there similar trends in the visual arts, with synaesthetic artists depicting musical ideas in the visual medium? How can we use this information to enhance our understanding and performance of music relating to other senses?
Taking the findings discussed in this article as the basis, my current PhD research aims at answering these questions. This will be done through further analysis of other synaesthetic compositions and non-synaesthetic works that are inspired by colour elements (and thus having cross-modal associations as an inspiration), making an extended list of tendencies in pitch, dynamic, and timbre-related material, and comparing the results with the existing scientific literature of synaesthesia and cross-modality, creating a reliable system of musical colour-progressions that documents our cross-modal associations in a concrete manner. This system will be further tested and developed into strategies for the musical performance practice, through experiments on the effects of adding pseudo-synaesthetic colour elements in technically demanding scores when applied to sight-reading and practicing. The goal is to methodically apply the system of musical colour-progressions into assisting musical practice.
When implemented creatively, synaesthesia and cross-modal associations suggest a great value. In the creation and performance of my work Prelude (for Electronics, Narrator, and Coloured Light) I have made use of these principles when considering the relationship between the music, word, and light.32 Such an approach made the process of rehearsing with the narrator and performer more straightforward as it facilitated a common, inter-sensory language when discussing how the musical cues and how the music relates to their part of the performance. Throughout my PhD research, I intend to create and perform more audiovisual pieces founded on these principles, using them in the creative process and the performance practice. I aim to take this idea further in workshops where musicians are encouraged to express their musical ideas in visual means and translate visual concepts into musical ideas, discovering their own cross-modal tendencies and ways of applying them within their practice. It is my hope that the investigation into these subjects will show us new ways to talk about music and arts, will help understand how our aesthetic perception is interconnected within our senses, and most importantly, will deepen our connection to art.
Umut Eldem is a musician, composer, and researcher. Since 2017, he has been researching the subject of Synaesthesia in Music in the Conservatoire of Antwerp. Currently he is a PhD student and a researcher in the same institute.