Overture: Alien Music

Steven Spielberg’s 1977 movie Close Encounters of the Third Kind declares that we are not alone, and we should not be afraid. The film follows ordinary people after they experience a close encounter with an unidentified flying object. After this experience, the protagonist of the movie, Roy Neary, becomes obsessed with seeing the UFO again, as well as with a strange shape that has a deep meaning that he cannot quite fathom. Eventually he realizes that this shape represents Devils Tower, Wyoming, which is the location selected for first contact between humans and aliens.

A famous short musical signal composed by John Williams plays a key role in the film. Williams strove to write a signal that was long enough to be set apart from the simplest musical elements (chords or intervals) but not so long as to exist on its own as a melody. He decided that these goals required a theme that was only five notes in length, and composed about 350 different five-note permutations. Spielberg preferred one in particular, and it became one of the most famous musical themes in film history. At the climax of the film, it is performed on an ARP 2500 synthesizer located on a massive runway constructed atop Devils Tower. The music greets, and communicates with, the alien visitors.

A young, fresh-faced Philip Dodds was the ARP engineer who set up the system used in the movie. When the original actor who was to play it became sick, Spielberg saw Dodds working on the machine, liked his look, and cast him in the role of playing the synthesizer. As the scene unfolds, Dodds plays the musical signal faster and louder while the chief scientist (Lacombe, played by François Truffaut) strides out along the runway. Eventually the enormous mother ship arrives, hovers over the runway, and loudly echoes the notes that Dodds plays. This musical mimicking quickly erupts into an interstellar jam session of increasing tempo and complexity. Awestruck and wide eyed, Dodds exclaims, “What are we saying to each other?” This is a very deep question indeed.

The intended message in the film is that music is—literally—a universal language, one shared by all intelligent life forms. That the alien ship generates the same notes, and that it jams with Dodds’s character in the same musical system, supports this optimistic view. To answer Dodds’s question a scientist standing beside him says, “Seems they’re trying to teach us a basic tonal vocabulary.” Another immediately adds, “It’s the first day of school, fellas.”

However, other intriguing scenarios exist; there are alternative musical possibilities that Spielberg did not explore in his film. Does the fact that both the ARP 2500 and the mother ship generate the same basic tonal vocabulary imply that the humans and the aliens share the same underlying musical theory? Alternatively, is it possible that a completely different musical theory—an alien music theory that is dramatically different from our own—is still capable of generating the same patterns of musical notes? Perhaps all these fellas know they are attending the first day of school, but are not aware of the lecture topic!

Networks and New Musical Theory

The purpose of this book is to explore the possibility that different systems that are capable of generating the same musical inputs and outputs can do so by using quite different theories of music. Fortunately, instead of requiring a close encounter of the third kind for this exploration, this book adopts a more practical approach. A particular type of computer simulation, an artificial neural network, is taught to generate responses that are consistent with Western musical theory. For instance, the computer simulation is presented the tones that define a particular scale, and learns to respond with the tonic note of that scale, or to identify that scale as being major or minor.

However, when a network learns to generate the correct outputs to various musical inputs, it is not constrained by traditional Western music theory. Many researchers argue that the internal workings of artificial neural networks are quite distinct from the clear formal properties found in logic, mathematics, or music theory. As a result, it is possible that an artificial neural network can discover a completely different method —an alien or novel music theory—that generates the same input/output relationships as are defined by Western music theory.

In order to determine whether this is possible, it is necessary to examine the internal structure of a trained network to discover exactly how it generates musical responses. An artificial neural network is a messy collection of different processors (analogous to neurons) that send signals to one another through a larger and messier collection of weighted connections (analogous to synapses between neurons). The musical knowledge of a trained network lies in its internal patterns of connectivity. The messiness of this knowledge makes the existence of a new music theory possible. By making sense of a network’s internal structure, we can discover the musical regularities that the network has learned, on its own, to exploit. Is the theory that the network learns the same as our own?

We will soon see that artificial neural networks can discover novel musical theories that seem quite different from typical accounts of Western music. This has interesting implications for music, insofar as it reveals alternative musical formalisms. This also has important implications for the study of musical cognition, because it reveals a variety of different kinds of representations that the human brain might use to process music.

Structure of the Book

The use of artificial neural networks to study human cognition is flourishing, and they are growing in importance in the study of musical cognition as well (Griffith & Todd, 1999; Todd & Loy, 1991). However, in this connectionist literature one rarely finds detailed interpretations of the internal structure of networks. A main purpose of this book is to provide numerous case studies of this approach, and to demonstrate its value to theories of music and to theories of musical cognition. I have hoped to convey this general message with the organization of the book.

The first two chapters provide a historical context for the current research. Chapter 1 provides a brief history of the scientific study of music and musical cognition, from the scientific revolution of the 17th century, through 19th century psychophysics, to modern musical cognitivism. One general theme that emerges from this discussion is that the psychology of music is intrinsically interdisciplinary; it involves the science of sound, the formal understanding of music theory, and the experimental understanding of music perception. It also makes clear that the modern psychology of music, musical cognition, views music perception as involving the active processing of auditory stimuli by organizing these stimuli using mental representations of music.

Chapter 2 then relates connectionist cognitive science, which uses artificial neural networks, to classical cognitive science, which views cognition as the rule-governed manipulation of symbols. Connectionism reacts against this symbolic view, and many musical connectionists seek to deal with regularities that cannot be captured formally. Chapter 2 introduces some basic properties of artificial neural networks, and introduces the methodology adopted in the chapters that follow. The chapter culminates in the argument that these networks should not be viewed as systems that are sensitive to informal properties of music. Instead, one should use these networks to inform both music theory and musical cognition by making sense of their internal structure. This requires a detailed understanding of how networks solve problems; this understanding will be formal. Networks do not merely capture informal properties of music; they capture new formal musical regularities.

The next six chapters provide case studies of this new approach to musical connectionism. Chapter 3 begins by introducing a case study in which a very simple network, called a perceptron, accomplishes a basic musical task: it learns to identify the tonic pitch of a presented scale. When we examine the connection weights of this network to attempt to explain how it works, we find a perfectly plausible formal musical interpretation of the perceptron’s structure. However, this structure differs from the typical account of scale structure.

Chapter 4 continues with the study of scales by training a network to identify the mode of a presented scale, deciding whether a stimulus is a major scale or a harmonic minor scale. A more powerful artificial neural network, called a multilayer perceptron, is required to accomplish this task. The multilayer perceptron is more powerful than the perceptron discussed in Chapter 3 because it includes intermediate processors called hidden units. Interpreting this multilayer perceptron’s internal structure reveals a quite novel account of the formal difference between major and minor keys, one that focuses upon the relationship between pairs of pitch-classes that are separated from one another by a particular musical interval, the tritone.

Chapter 5 provides another set of example networks that are trained on a particular task, called key-finding, which combines the tasks discussed in Chapters 3 and 4. To key-find a musical stimulus is to assert both the mode (major vs. minor) and the tonic of the scale used as the source of the musical stimulus’s notes. The chapter begins with a multilayer perceptron that learns to generate the tonic and mode for a presented scale. It solves this problem with four hidden units that implement an interesting type of distributed representation called a coarse code. The interpretation of this network introduces the basic properties of coarse coding. Chapter 5 then considers a broader task: identifying the keys of various musical compositions. This is done using simpler networks, perceptrons, to create connectionist variants of three more traditional key-finding theories. The perceptrons identify the musical keys of a large number of test stimuli with a high degree of accuracy. The chapter ends by pointing out that an interpretation of the structure of these perceptrons suggests possible modifications to more traditional theories of key-finding.

Chapter 6 turns to the study of musical problems involving harmony: a combination of different pitches that occur simultaneously. It begins by introducing a very basic element of harmony, the triad, and defines four different triad types: major, minor, diminished, and augmented. It then explores a multilayer perceptron that learns to identify these triad types. An examination of the structure of this network demonstrates that its hidden units apply the same “name” to a number of different pitch-classes that are related to one another by particular musical intervals. We call such an equivalence class a strange circle. The chapter then proceeds to define the properties of these strange circles and ends with another case study of a network trained to identify different types of chords, tetrachords. The analysis of this network’s structure also reveals that it organizes inputs according to a variety of strange circles.

Chapter 7 provides a more complex example of a network trained to classify chords. It describes additional formulae for defining 12 different types of tetrachords for each of music’s 12 major keys. It then reports the training of a multilayer perceptron that learned to classify an input tetrachord into these different tetrachord types. This more complex network requires seven hidden units to solve this classification problem. However, we can interpret its internal structure. This is because it also organizes input pitch-classes using strange circles. The interpretation of this network introduces an additional interpretative technique (examining bands in jittered density plots). As well, the structure of this extended tetrachord network provides another elegant example of coarse coding.

Chapter 8 explores an important source of tonality in Western music: musically related sequences of chords called chord progressions. The chapter begins by discussing an important chord sequence in jazz called the ii-V-I progression. We teach a number of different networks this progression; when provided one chord, a network responds with the next chord in the progression. A key difference between these various networks is that different codes are used to represent input and output chords. The question of interest is whether the choice of encoding affects the ease of learning the progression. Next, we turn to an elaboration of the ii-V-I progression and train networks on different encodings of a more complex jazz progression, the Coltrane changes. Again encoding is critical: the choice of encoding for the Coltrane changes determines not only the amount of learning but also the complexity of the network required to learn this progression. I will interpret a perceptron that learns one encoding of the Coltrane changes; this interpretation reveals once again the utility of the strange circles that we have already encountered in earlier chapters.

The final chapter of the book, Chapter 9, steps back to examine the general results reported in the various case studies. It begins with a discussion of the nature of this research program and then summarizes its most important results. It then turns to considering the implications of these results, first to the theory of music and then to the cognitive science of music. It concludes that our straightforward approach to musical networks has succeeded; even though the case studies involved training simple networks on basic properties of Western tonality, they led to the discovery a number of interesting, new musical properties. This success points the way to future musical studies; network interpretation is at the heart of each of these potential areas of investigation.

One purpose of this book is to present case studies that show that musical connectionism of the form championed here is viable. A second purpose is to inspire researchers to pursue similar studies, by studying the network architectures detailed in the chapters that follow or by exploring musical connectionism by interpreting different types of networks. All of the simulations that are described in this book were performed using networks whose properties have been described in detail elsewhere (Dawson, 2004). The software used to conduct these simulations has also been described in detail (Dawson, 2005) and is available free of charge from the author’s website. Web resources for this book, which include links to software, the training files used to conduct the various simulations, resources for building new training files, and other relevant information are available here: http://www.bcp.psych.ualberta.ca/~mike/AlienMusic/