6

Classifying Chords with Strange Circles

6.1 Four Types of Triads

Up to this point, most of the musical networks considered have explored tonality in terms of components related to melody, such as the pitch-classes that make up a particular scale. I will now explore networks that are faced with making judgments involving harmony, where multiple stimulus elements combine together to define a musical combination like a chord. I begin by defining basic chords called triads, and interpret the structure of a network that learns to classify triads by their quality. Next, I discover a property called strange circles, in which hidden units in a network organize pitch-classes into musical systems called interval cycles, but then treat each member of an interval cycle as being the same pitch. Then, I turn to defining, geometrically, all of the possible interval cycles in Western music in order to have a catalog of strange circles available for network interpretation. Finally, I define another kind of chord, the added note tetrachord, and train a network to classify stimuli into different types of such chords. When this network is interpreted, I once again discover that the network uses strange circles.

6.1.1 Triads

The simulations described in Chapter 3, Chapter 4, and the first part of Chapter 5 involved training artificial neural networks to identify properties of a particular type of monophonic music, the scale. In this chapter, I turn to training networks to detect some basic properties of harmonic music, where combinations of inputs are of key import. To begin, I focus on a simple chord, the triad, which is composed of three distinct notes. The most basic form of triad is the major triad, created by combining the first, third, and fifth notes of a major scale. Consider the A major scale that was presented in Chapter 3 as Figure 3-1. Its first note is A, its third note is C♯, and its fifth note is E. Therefore the A major triad, symbolized as A, is the notes A-C♯-E. Figure 6-1 illustrates this triad in the very first bar of the score.

Figure 6-1 The top staff provides examples of four different types of triads built on the root note of A: A major (A), A minor (Am), A diminished (Adim) and A augmented (Aaug).

When the triad’s notes are placed on a musical staff such that the first note of the major scale is the lowest note, the third note is the next lowest, and the fifth note is the highest, the triad is said to be in root position. The A triad at the top left of Figure 6-1 is in root position. The major triad can be manipulated to create other types of triads that have different sonority. For instance, if one takes the middle note of a major triad and lowers it by a semitone, the result is a minor triad. The second bar of the Figure 6-1 score provides the A minor (Am) triad in root position; it is composed of the notes A-C-E. Note that an alternative way to construct a minor triad is to combine the first, third, and fifth notes of a minor scale. For instance, A, C, and E are the first, third, and fifth notes of the A harmonic minor scale presented earlier in Figure 3-1.

A different alteration of the major triad produces a third kind of triad, the diminished triad. A diminished triad is produced by lowering both the second and the third notes of a major triad by a semitone. The A diminished triad (Adim) is shown in the third bar of the Figure 6-1 score, and consists of the notes A-C-E♭. A third alteration of the major triad produces yet another triad, the augmented triad. An augmented triad is created by taking a major triad and raising its third note by a semitone. The augmented triad for A (Aaug) is illustrated in root position in the fourth bar of the Figure 6-1 score. The first four bars of Figure 6-1 provide four different triads built upon the root of A. Obviously major, minor, diminished, and augmented triads can be built using any of the 12 possible Western music pitch-classes as the root. Therefore, there are 12 different versions of each of these four different types of triad. Later in this chapter we will explore a multilayer perceptron that has the task of identifying triad type, ignoring the root or key of the triad.

6.1.2 Inversions and Representations

Triads like those presented in the first line of Figure 6-1 can take different forms through a process called inversion. To create the first inversion of a triad in root position, one raises its lowest note an octave. To create the second inversion of a triad in root position, one raises both its lowest and its middle notes an octave. The first and second inversions of each of the triads are shown in Figure 6-1.

Inverting a triad does not alter the triad’s quality. As a result, one interesting task is to train a network to identify triad quality (major, minor, diminished, augmented) regardless of the triad’s key and regardless of the triad’s inversion. However, to accomplish this task I must adopt a different input representation than the one used to represent scales in Chapters 3 through 5. All of these networks used 12 input units to represent stimuli in terms of their component pitch-classes. However, in order to present different inversions of the same triad I must replace one pitch with another pitch that is an octave higher. This distinction between pitches an octave apart is not possible with the pitch-class representation that has been used for the earlier networks, because it representation assumes octave equivalence.

In order to present different inversions of the same chord, I must replace the pitch-class representation with a local representation of musical pitch. In this local representation, each input unit can be considered analogous to a key on a piano that is associated with a particular pitch. For instance, one input unit might represent the pitch A3 (the A below middle C) while a different input unit represents the pitch A4 (the A above middle C). Both of these units represent the same pitch-class (A), but different pitches.

Figure 6-2 illustrates a multilayer network for triad classification that uses a local representation of pitch in order to present triads in different inversions. Its 28 different input units represent 28 different musical pitches. The input units of this network are analogous to a 28-key piano keyboard whose lowest key plays the note A3 (the A below middle C), and whose highest key plays the note C6 (the C two octaves higher than middle C).

6.2 Triad Classification Networks

6.2.1 Task

The task for the multilayer perceptron described in this section is to identify triad type (major, minor, diminished, or augmented), ignoring both the triad’s key and its inversion.

Figure 6-2 A multilayer perceptron with local pitch encoding that learns to identify four types of triads, ignoring a triad’s key and inversion.

6.2.2 Network Architecture

The multilayer perceptron illustrated in Figure 6-2 employs four output units. Each is a value unit. This network requires four hidden value units to converge to a solution to the triad classification problem. The network uses local encoding of specific pitches in order to present different inversions of triads as was discussed in Section 6.1.2. This input unit layout provides enough “room” to encode the three different forms of each triad. The lowest triad presented to the network is A major in root position. The highest triad presented to the network is G♯ augmented in second inversion.

6.2.3 Training Set

The training set consists of 144 stimuli: 36 different major triads, 36 different minor triads, 36 different diminished triads, and 36 different augmented triads. The four different triad types are constructed using the 12 different root pitch-classes that were available. In addition, each type of triad, for each root, is created in three different forms (see Figure 6-1): root position, first inversion, and second inversion. A particular triad (i.e., a triad related to a particular root, in a particular form) only appears once in the training set. For instance, there is only one A major triad in root position (whose lowest note is the first of the 28 input units), even though this same triad could be presented using other input units higher up along the network’s “keyboard.”

Each triad is encoded as an input pattern in which three input units are activated with a value of one, and the remaining 25 input units are all activated with a value of zero. Each input pattern is paired with an output pattern that requires one output unit to activate with a value of one, and the other three output units to activate with a value of zero. The output unit trained to activate is one that represents the input pattern’s correct triad type.

6.2.4 Training

The multilayer perceptron is trained using the generalized delta rule developed for networks of value units (Dawson & Schopflocher, 1992) that is part of the Rumelhart software program (Dawson, 2005). All connection weights in the network are set to random values between −0.1 and 0.1 before training begins. The µs of the output and hidden units are all set to zero throughout training. We employ a learning rate of 0.005. Training proceeds until the network generates a “hit” for every output unit for each of the 144 patterns in the training set. Once again, a “hit” is defined as activity of 0.9 or higher when the desired response is one or as activity of 0.1 or lower when the desired response is zero. The multilayer perceptron typically learns to solve this problem in fewer than 1000 epochs of training. The example network described in more detail below converged after only 576 sweeps of training.

6.2.5 Connection Weight Patterns

This triad classification task produces some very interesting properties when the connection weights of a trained network are examined. To begin, let us examine the connection weights from each of the 28 input units to Hidden Unit 1 of this multilayer perceptron, which appear in Figure 6-3. A quick glance at this figure reveals a striking regularity in connection weight patterns. Ignoring slight variance in connection weight magnitude, there is a pattern that repeats itself every three input units: a weak negative connection weight, followed by a strong negative connection weight, followed by a strong positive connection weight.

Figure 6-3 The connection weights from the 28 input units for pitch to Hidden Unit 1 in the network trained to classify triad types for different keys and inversions.

This repeating pattern illustrates a number of different equivalences between pitch-classes separated by a specific musical interval. The first is octave equivalence: every input unit that represents a pitch that belongs to the same pitch-class (e.g., the three different A pitches) is assigned the same weight. Second, pitch-classes separated by the interval of a minor third (three semitones) are all assigned roughly the same weight. For instance, B, D, F, and G♯ all belong to the interval cycle of minor thirds; that is, each of these pitches is a minor third away from its adjacent neighbours in the list. (The properties of such interval cycles [Roig-Francolí, 2008] are detailed later in this chapter.) Critically, they all have approximately the same connection weight in Figure 6-3. Similarly A, C, D♯, and F♯ belong to a different interval cycle of minor thirds; they too are assigned the same connection weight (weak negative), but one that is different from the weight assigned the first set of pitches. Finally A♯, C♯, E, and G all belong to yet a third interval cycle of minor thirds; they are also assigned the same connection weight (strong negative), but one that distinguishes them from the two other groups of pitches.

Paying more attention to the subtle differences in connection weights in Figure 6-3, one also finds evidence for tritone equivalence. For instance, A and D♯ have nearly identical connection weights, as do C and F♯. All pitches related by tritones that belong to the same “circle of tritones” are assigned the same weight. In other words, this set of connection weights indicates that pitches that are specific musical intervals apart from one another are assigned nearly identical connection weights. For Hidden Unit 1, this means that these different pitches are in reality all the same, because the connection weight from an input unit to the hidden unit in essence can be interpreted as Hidden Unit 1’s “name” for the input pitch.

Figure 6-4 presents the connection weights from the input units of this network to a different hidden unit, Hidden Unit 2. It too reveals some striking interval equivalences, some of which differ from those seen in Figure 6-3. In Figure 6-4, a general pattern of connection weights repeats itself every four input units instead of every three. This pattern is a weak negative weight, then a strong positive weight, then a weak positive weight, and finally a strong negative weight.

Figure 6-4 The connection weights from the 28 input units for pitch to Hidden Unit 2 in the network trained to classify triad types for different keys and inversions.

Once again, this pattern exhibits octave equivalence: each pitch that belongs to the same pitch-class has the same weight. Ignoring subtle variations in magnitude, the pattern of weights in Figure 6-4 separates the different pitches into four different groups, each containing three pitch-classes. For example A, C♯, and F are the only pitches that have weak negative weights. Analogous groupings via connection weights are found for the pitch-classes A♯, D, and F♯, for the pitch-classes B, D♯, and G, and for the pitch-classes C, E, and G♯.

The weights in Figure 6-4 do not exhibit tritone equivalence but do demonstrate tritone balance: note that pitch-classes separated by a tritone (like A and D♯) have weights that are equal in magnitude but opposite in sign.

Figure 6-5 presents the connection weights between the input units and Hidden Unit 3. Its organization is very similar to that of Hidden Unit 2 (Figure 6-5). It exhibits octave equivalence, organizes pitches into the four different groups related by major thirds, and demonstrates tritone balance. The key difference between Figure 6-5 and Figure 6-5 is that while both exhibit the same organizational pattern, there appears to be a phase shift of this pattern when we compare the two hidden units. That is, the weak negative weight that starts the pattern is associated with A in Figure 6-4 but has been shifted three semitones to the right to start with C in Figure 6-5.

Figure 6-5 The connection weights from the 28 input units for pitch to Hidden Unit 3 in the network trained to classify triad types for different keys and inversions.

The final pattern of connectivity to consider involves Hidden Unit 4 (Figure 6-6). This pattern demonstrates octave equivalence, with the exception of the final weight for C, which is a dramatic outlier. (This particular note is included in only one chord; occasionally I find that networks can solve problems by treating rarely used input units uniquely.) Paying attention only to the sign of connection weight (and ignoring the deviant final weight!), it is notable that connection weight sign alternates from positive to negative every semitone. This divides the pitches into groups that belong to different interval cycles of major seconds. One of these cycles contains A, B, C♯, D♯, F, and G; the other cycle contains A♯, C, D, E, F♯, and G♯. This pattern of connection weights also demonstrates tritone equivalence; pitches separated by a tritone have roughly the same weight.

Figure 6-6 The connection weights from the 28 input units for pitch to Hidden Unit 4 in the network trained to classify triad types for different keys and inversions.

The pattern of connection weights that feeds into each of this multilayer perceptron’s hidden units clearly indicates that pitches and pitch-classes are often organized into distinct equivalence classes that involve musical intervals. We will see below that this is a common discovery in networks of value units trained to make judgments about chords. Network interpretation can often be implified by seeking these sorts of relationships out, but they are even more interesting because they point to a very different kind of formal music theory.

To get a better understanding of these relationships and their implications, let us first detail the properties of various interval cycles. I will then consider a second multilayer perceptron trained on a different chord classification task, and discover interesting equivalence classes between pitches and pitch-classes when we examine its hidden unit weights.

6.3 Interval Cycles and Strange Circles

Section 6.2 described a multilayer perceptron for classifying triad types. The connection weights feeding into its hidden units grouped different sets of pitches and pitch-classes into various equivalence classes. I call such equivalence classes strange circles. They are circles, in the sense that they involve sets of pitch-classes that belong to musical sets called interval cycles, each of which can be represented as a circle of pitch-classes. They are strange, although, as far as networks are concerned, pitch-classes that belong to the same interval cycle are all treated as being the identical note. In other words, when I find networks that employ strange circles in their internal representations, these networks are using fewer than the 12 pitch-classes that are the core components of Western music theory.

I now explore such relationships in more detail, because they appear frequently when I train artificial neural networks on tasks involving musical harmony (Yaremchuk & Dawson, 2005, 2008). I begin by using the notion of interval cycles from music theory (Roig-Francolí, 2008) to generate the possible interval equivalences that might be discovered inside a network. I will generate a catalogue of the various interval cycles that can be defined for Western music, and then point out that each of these cycles could be used as a strange circle. Later in this chapter, and in chapters that follow, we will see several examples of networks whose internal structure reveals such strange circles.

6.3.1 Piano Geography

Our general approach to defining strange circles is to generate geometric representations of interval cycles. The foundation of our geometric representation of pitch-class relationships is a physical artifact, the piano keyboard. Each piano key, when struck, produces a unique pitch. For instance, the lowest (left-most) shaded note at the top of Figure 6-7 corresponds to the pitch “middle C,” which is sometimes designated with the name C4.

Figure 6-7 The geography of the piano.

The layout of piano keys is quite regular. This is evident in Figure 6-7 from the arrangement of black keys, which alternate in groups of twos and threes across the figure. The pattern of 12 differently named piano keys at the top of Figure 6-7 repeats itself along the keyboard.

While every piano key plays a differently pitched note, that note belongs to one of the 12 pitch-classes of Western music that we have already encountered. Therefore, several different piano keys play different pitches that all belong to the same pitch-class, and they occur at regular intervals along the piano keyboard. This is illustrated at the bottom of Figure 6-7, which highlights the locations of four different instances of the pitch-class C. Nearest neighbours on the keyboard that belong to the same pitch-class are separated by a span of 12 adjacent piano keys. This distance is equivalent to 12 semitones, or a musical interval of a perfect octave.

6.3.2 Distance and Intervals

With respect to our piano geography, what is the distance between two notes? For instance, what is the distance between the highlighted notes C and E at the top of Figure 6-8? We can measure this distance in terms of the number of piano keys that separate the two notes.

Examine the top illustration in Figure 6-8. If one starts at the highlighted C and moves up in pitch (i.e., to the right along the keyboard), then the first key encountered is C♯, the second is D, the third is D♯, and the fourth is E. Therefore, the distance between C and E is four piano keys. Alternatively, we can say that the distance between C and E in this figure is four semitones, which is a musical interval of a major third.

Figure 6-8 Using the number of piano keys as a measure of the distance between pitches.

We can identify sets of pitches that are spaced the same distance apart by continuing to count the same number of keys to the next note, as illustrated in the middle part of Figure 6-8. The distance up from C to E is four piano keys; if we move the same distance up from E, we reach G♯. If we move up four piano keys from G♯, we reach another C. In other words, if we start at C, and always move four piano keys up, we will only encounter three different pitch-classes: C, E, and G♯.

It is convenient to arrange a set of pitch-classes picked out by moving a fixed distance along the piano in a circle. The circle that results when a distance of four piano keys is used starts with C, moves next to E, moves next to G♯, and then returns to C. It is the fact that after a few moves we return to the pitch-class that we started from (C in this example) that motivates arranging this set of pitch-classes in a circle. We literally come full circle back to the pitch-class from which we started. Such circles of intervals are called interval cycles (Roig-Francolí, 2008).

If we use the same distance between notes but start at a different piano key, we can define a different instance of the same interval cycle. For instance, if we start at C♯ and move up four keys at a time, our circle will only include the pitch-classes C♯, F, and A. With a between-note distance of four piano keys, we can define four different circles of three pitch-classes.

Interestingly, we can encounter the same subset of pitch-classes by starting at C and counting a different distance. The bottom part of Figure 6-8 shows that G♯ is eight piano keys higher than C, and that E is eight piano keys higher than G♯. With this different distance, we encounter the same pitch-classes highlighted in the middle of Figure 6-8, but encounter them in a different order. That the same subset of pitch-classes are encountered when one moves different distances along the keyboard indicates that we can consider these pitch-classes as being separated by two different musical intervals (Roig-Francolí, 2008). For instance, C and E can be considered to be a major third apart (four piano keys) or a minor sixth apart (eight piano keys).

Clearly, there is a musical interpretation for each distance between pitches measured in terms of number of piano keys or in terms of semitones. Table 6-1 presents the names of the different intervals in Western music, as well as their associated distance between pitches. Note that this table indicates that there are only 13 different types of interval cycles to be concerned about, because there are only 13 different semitone distances listed in the table, ranging from 0 semitones to 12 semitones. The next sections consider the properties of these possible interval cycles.

Table 6-1 The 13 possible distances between pitches that can be used to create interval cycles.

6.3.3 Single Interval Cycles

The first set of interval cycles to discuss are those distances between pitches which, when used, combine all 12 different pitch-classes into a single set or a single interval cycle. To begin, let us consider starting on a piano keyboard at a note named C, and moving up from this note (i.e., to the right along the keyboard) a distance of one piano key. The first note that we will encounter is C♯. Moving the same distance up from C♯ we will next encounter D. Continuing to move along the keyboard in this fashion, we will encounter every possible pitch-class before we finally reach another note named C. The set of pitch-classes that we encounter, and the order in which we encounter them, can be represented in a single circle (Figure 6-9). (To make this figure easier to compare to other versions, instead of drawing the circle, we arrange the pitch-classes in a circle, and then draw a radius to the centre of the circle to place each pitch-class on a “spoke.”) Because the distance between adjacent notes in this circle is one piano key (one semitone), the interval between adjacent notes is a minor second. Therefore, we can name Figure 6-9 the circle of minor seconds. Note that moving in a clockwise direction around this circle is equivalent to moving up (i.e., to the right) along a piano keyboard, and moving in a counter-clockwise direction is equivalent to moving down along a piano keyboard.

Figure 6-9 The circle of minor seconds.

Earlier we noted that by choosing a different distance to move along the piano keyboard we will encounter the same set of pitch-classes, but will do so in a different order. This is the case if we start at a note named C and move up the keyboard a distance of 11 keys (or 11 semitones), which corresponds to a musical interval of a major seventh. The order of notes that are encountered is illustrated in Figure 6-10 as a circle of major sevenths.

Figure 6-10 The circle of major sevenths.

An inspection of the circle of major sevenths indicates that it picks out the same pitch-classes as does the circle of minor seconds, but it does so in a different order. Indeed, the two circles are complementary: the circle of major sevenths is a mirror reflection of the circle of minor seconds.

We have seen that distances of one or 11 piano keys pick out all 12 pitch-classes; they can be arranged around a single circle to represent the order in which pitch-classes are encountered. Another distance that picks out all 12 pitch-classes is one of five piano keys (or semitones). Moving up a piano keyboard at this distance produces the circle of perfect fourths, illustrated in Figure 6-11. Note that this circle arranges pitch-classes in a very different order than we saw in Figures 6-9 and 6-10.

As was the case with the circle of minor seconds, the circle of perfect fourths has a complementary circle that is its reflection. The circle of perfect fifths is produced when one starts at C and moves up a piano keyboard a distance of seven piano keys or seven semitones (Figure 6-12). Note that if one were to move counter-clockwise around the circle of perfect fourths, one would encounter pitch-classes in the same order as moving clockwise around the circle of perfect fifths. This is because when a musical interval of a perfect fourth is inverted, the result is a musical interval of a perfect fifth.

Figure 6-11 The circle of perfect fourths.

Of the four interval cycles presented in this section, the one most frequently seen in music is the circle of perfect fifths. Music students learn how to use this circle to determine the number of sharps or flats are in a particular musical key. Later we will see how this circle serves as a map to guide a musician who wishes to play the chords of a particular jazz progression, the ii-V-I. The circle of minor seconds is also commonly encountered; for instance, it is frequently found in geometric discussions of musical regularities (Tymoczko, 2011).

Each of the interval cycles presented in this section is defined musically: each is created by moving upward along a piano keyboard in steps of a set distance. However, these cycles—as well as those presented in following sections—can also be considered as mathematical objects called manifolds.

Figure 6-12 The circle of perfect fifths.

A manifold is a surface upon which objects are represented as points. Manifolds have specific shapes, and exist in a space that has a set number of dimensions. For instance, all of the manifolds described in this section are circular shapes. They are one-dimensional manifolds, in the sense that you can trace the entire shape with a finger without having to lift the finger up before the trace is completed. These manifolds are embedded in a higher-dimensional space. For instance, each of these manifolds is depicted in a two-dimensional space, the plane of the page in which each figure is drawn. In short, each of the circles that we have discussed is a one-dimensional manifold, circular in shape, embedded in a two-dimensional space.

The shape of a manifold is important, because it constrains how one moves from location to location along the manifold’s surface. That is, to move from one location to another, one must never leave the manifold’s surface. This property of manifolds has been used in theories of visual perception and visual imagery (Farrell & Shepard, 1981; Shepard, 1984) to model the appearance of three-dimensional objects as they move or as they are mentally rotated.

In describing the single interval cycles as manifolds, I am asserting that their shape and layout place certain constraints on how one can move from one note (i.e., from one point on the manifold’s surface) to another. For instance, on the circle of perfect fifths, to move from C to D one must necessarily pass through an intermediate location, G. This is because G occupies an intermediate location between C and D on the manifold’s surface.

Interpreting each of the circles as a manifold has further implications concerning the notion of distances between notes (Tymoczko, 2011). Each manifold is derived by moving along a piano keyboard at a set distance. However, after creating a manifold, I could measure the distance between notes along the manifold’s surface. For instance, in the circle of perfect fifths, G is one unit of distance away from C because it is next to C on the manifold; similarly, D is two units away from C on the same manifold.

From this perspective, the distance between notes depends upon a particular context: the specific manifold under consideration. In the circle of perfect fifths, G is one distance unit away from C. In the circle of minor seconds, the shortest distance between C and G is five distance units. The idea that the distance between notes can be measured along a manifold, and that the size of this distance depends on the particular manifold being considered, is strongly related to Dmitri Tymoczko’s idea of measuring distance between notes in the context of different musical scales (Tymoczko, 2011).

Importantly, if one uses a musical manifold or a musical scale as the context in which to measure the distance between pitch-classes, then one is tacitly assuming that different points on the manifold represent different pitch-classes. Of particular interest to us in this chapter is that in some instances artificial neural networks capture pitch-classes that can be represented in a manifold, but do not treat these pitch-classes as being different. For these networks, the manifold is an equivalence class, because all the pitch-classes that belong to it are the same. However, in order for equivalence classes of this sort to be useful there must be more than one, so that some pitch-classes belong to one equivalence class but not others. In the next section, we consider circles of musical intervals that pick out different and complementary subsets of pitch-classes.

6.3.4 Pairs of Interval Cycles

In the previous section, I detailed four different musical manifolds that are interval cycles. They are single in the sense that one manifold captures all 12 pitch-classes on its surface. Next, I will describe some new manifolds, each of which only captures half of the available pitch-classes. As a result, two different versions of the same process—moving along the piano keys—are required to build two complementary manifolds which, when combined, capture all 12 pitch-classes.

Consider starting at a C note on a piano, and moving upward a distance of two keys to the next note, which will be D. Following this procedure, we will next encounter E, F♯, G♯, and A♯ before encountering another C. One can build a manifold of these notes —a circle of major seconds—but it will only hold six of the 12 available pitch-classes. If one repeats this process, but start on C♯ instead of C, we will create a second circle of major seconds that complements the first because it captures the remaining six pitch-classes. Figure 6-13 illustrates these two circles of major seconds.

Figure 6-13 The two circles of major seconds.

If one takes a major second interval and inverts it, then the result is a minor seventh. This is because after inverting the major second interval, the distance between the low D and the higher C would be 10 semitones. From this, one should expect to be able to produce two circles of minor sevenths that complement the circles of major seconds illustrated in Figure 6-13. Indeed, if one gains at C and moves upward along the piano keyboard 10 keys at a time a circle of minor sevenths that is a reflection of the first circle of major seconds in Figure 6-13 is produced. Starting instead at C♯, one produces a manifold that complements the first circle of minor sevenths and reflects the second circle of major seconds in Figure 6-13. Figure 6-14 provides these two circles of minor sevenths.

Figure 6-14 The two circles of minor sevenths.

6.3.5 Trios of Interval Cycles

In this section, I define sets of three complementary manifolds, each of which captures four pitch-classes; all three combined contain all 12 pitch-classes. Imagine starting on the piano at some C note and moving upward along the keyboard a distance of three keys. The first note encountered is D♯. Moving the same distance upward, one encounters F♯, then A, and then another C. Thus, this defines a circle that captures four of the 12 pitch-classes. This manifold is a circle of minor thirds, because if two adjacent notes are three semitones apart, they are separated by an interval of a minor third.

To define other, complementary, manifolds I must move the same distance along the keyboard but from different starting points. If we start at C♯ instead of C, a second circle of minor thirds is defined; if we start at D instead of C, a third circle of minor thirds is defined. Figure 6-15 illustrates the three possible circles of minor thirds.

Figure 6-15 The three circles of minor thirds.

If one takes a minor third and inverts it, then one produces an interval of a major sixth whose notes are nine semitones apart. Not surprisingly, if I start at each of the three notes used above (C, C♯, D) and move along the piano nine keys at a time, I produce three different complementary circles of major sixths, shown in Figure 6-16. Each of these circles is a reflection of one of the circles of minor thirds illustrated in Figure 6-15.

Figure 6-16 The three circles of major sixths.

6.3.6 Quartets of Interval Cycles

Imagine starting on the piano at some C note and moving upward along the keyboard a distance of four keys. The first note encountered is E. Moving the same distance upward, one encounters G♯, and then encounters another C. Thus, this defines a circle that captures only three of the 12 pitch-classes. This manifold is a circle of major thirds.

Three other circles of major thirds are possible, and are required to capture the remaining pitch-classes. I create them by moving the same distance along the piano keyboard, but from different starting points: C♯, D, and D♯ respectively. Figure 6-17 illustrates the four circles of major thirds.

Figure 6-17 The four circles of major thirds.

If one takes a major third and inverts it, one produces an interval of a minor sixth whose notes are separated by eight semitones. If one starts at each of the four starting points used to create the manifolds of Figure 6-17 (C, C♯, D, D♯) and moves along the piano eight keys at a time, then one produces the four complementary circles of minor sixths, each of which is illustrated in Figure 6-18. Each of these circles is a reflection of one of the circles illustrated in Figure 6-17.

Figure 6-18 The four circles of minor sixths.

6.3.7 Sextets of Interval Cycles

If one starts at some C note on the piano keyboard and moves up six piano keys, one encounters F♯. Moving up another six piano keys one reaches another C. This defines a simple manifold that contains only two points. To capture the remaining pitch-classes requires starting from five additional notes on the keyboards. This produces the six different circles of tritones that Figure 6-19 provides.

Figure 6-19 The six circles of tritones.

The inversion of a tritone is itself a tritone, because this interval is defined by six semitones, a distance that is exactly half an octave. Thus, no other interval cycles are reflections of those illustrated in Figure 6-19.

6.3.8 Dodecal Interval Cycles

Choose some note C on a piano, and move 12 keys—a perfect octave—upward. You reach another C. This produces a special case manifold, a “circle” that represents a single pitch-class as a single point. Obviously 11 other such manifolds are required to capture the remaining pitch-classes (Figure 6-20).

Figure 6-20 The 12 circles of octaves, or circles of unison.

If one inverts an octave interval by raising the lower note an octave, the result is two identical notes—the distance between them is zero semitones. This musical interval, perfect unison, produces exactly the same set of 12 manifolds given in Figure 6-20.

6.3.9 Strange Circles

We introduced single interval cycles in Section 6.3.3 (e.g., the circle of minor seconds and the circle of perfect fifths). Then we discussed a number of interval cycles in which more than one cycle existed for the same interval. These were the two circles of major seconds, the two circles of minor sevenths, the three circles of minor thirds, the three circles of major sixths, the four circles of major thirds, the four circles of minor sixths, the six circles of tritones, the 12 circles of perfect octaves and the 12 circles of unison.

As was the case for the single circles of intervals, each of these multiple circles is a manifold. For instance, on one of the circles of major seconds the distance between C and D is one unit. However, we can interpret each of these manifolds in a different way: as an equivalence class. For instance, let us return for a moment to consider the hidden units of the multilayer perceptron that learned to classify triads (Section 6.2). Hidden Unit 1 (Figure 6-3) organizes inputs in terms of circles of minor thirds: all the pitches that belong to the same circle have the identical weight from the input unit to this hidden unit. However, this also means that all of the different (to us) members of this circle are identical for this hidden unit. Similarly, Hidden Unit 2 (Figure 6-4) assigns the same connection weights to inputs that belong to the same circle of major thirds; Hidden Unit 3 (Figure 6-5) also organizes inputs into equivalence classes based on circles of major thirds. Hidden Unit 4 (Figure 6-6) organizes inputs into equivalence classes based on circles of major seconds (if one only examines connection weight signs) and into equivalence classes based on circles of tritones (if one examines both the magnitude and sign of each connection weight).

I call equivalence classes based upon circles of intervals strange circles. These circles are strange in two ways.

First, while entities like the two circles of major seconds or the three circles of minor thirds are proper components of music theory, they are rarely encountered in theories of tonal music—music associated with a tonal centre. Instead, they are more likely to be encountered in theories about atonal or post-tonal music (Laitz, 2008; Roig-Francolí, 2008; Straus, 2005). This is because these interval cycles are all examples of symmetric sets of pitch-classes. That is, one can draw at least one axis through images such as those illustrated in Figures 6-13, 6-15, 6-17, and 6-19 such that the arrangement of the pitch-classes on one side of the axis mirrors the arrangement of those on the other side.

Symmetric sets of pitch-classes are important elements in post-tonal music. This is because “the basis of tonality’s gravitational field, which pulls scale degrees and harmonies toward tonic, is predicated on asymmetry” (Laitz, 2008, p. 812). For instance, key elements of tonal music, like the major and harmonic minor scales that we have encountered earlier, depend upon the asymmetric arrangement of pitch-classes that result when one creates a scale in which neighbouring notes are spaced apart by an irregular arrangement of tones and semitones (see Chapter 2). If this asymmetry is eliminated by choosing sets of pitch-classes that are equally spaced (as in the strange circles), then “a sense of goal-directed motion and tonal grounding disappears because every scale step is as stable (or as unstable) as every other step” (Laitz, 2008, p. 813).

In short, when one trains a network to make a musical judgment about tonal musical stimuli, and discovers that it does so by employing symmetric interval cycles, then this is indeed strange.

A second reason for calling these circles strange is that when one finds them in networks, they typically involve assigning different input units (i.e., pitch-classes) identical connection weights that feed into the same hidden unit. This means that as far as this hidden unit is concerned, these different (to us) pitch-classes are identical. In other words, a hidden unit that assigns one connection weight to the pitch-classes that belong to one circle of major seconds, and assigns another connection weight to all of the pitch-classes that belong to the other circle of major seconds, is operating as if music is constructed from only two pitch-classes instead of 12.

On the one hand, the use of equivalence classes to represent musical regularities is not odd. For instance, we have already encountered the notion of octave equivalence that motivated our discussion of pitch-class representations in Chapter 2. We saw in Chapter 3 that scales constructed on different tonics could be assigned to equivalence classes based on scale mode (e.g., major vs. harmonic minor). Clearly, the use of equivalence classes is central to music theory.

On the other hand, the foundation of almost all theory concerning Western music assumes the existence of 12 different pitch-classes. For instance, when the principle of octave equivalence is invoked in the theory of atonal or post-tonal music (Forte, 1973; Roig-Francolí, 2008; Straus, 2005), this implies the assumption of 12 different pitch-classes. Music theory has not explored the consequences of using interval cycles to define equivalence classes that imply fewer than 12 pitch-classes.

Let us now turn to another network whose interpretation reveals that it treats a number of different pitch-classes as being the same, because they have the same connection weight. Whenever this occurs, one finds that the equivalence class that they belong to is one of the strange circles described above.

6.4 Added Note Tetrachords

6.4.1 Tetrachords

The major and minor scales that serve as the foundation for much of Western music are rooted in musical formalisms invented by the ancient Greeks. The foundation of Greek music was not the scale but the tetrachord. The Greek tetrachord was a set of four different notes, the lowest always separated from the highest by an interval of a perfect fourth. Two additional notes were placed between these two, carving the tetrachord’s perfect fourth into three smaller intervals. There were three main types of tetrachords, depending upon the choice of the inner two notes (Barbera, 1977; Chalmers, 1992). Our modern major and harmonic minor scales are constructed from two adjacent Greek tetrachords.

Figure 6-21 Added note tetrachords in the key of C major.

The modern definition of tetrachord includes a much wider variety of chords than does the Greek definition. A modern tetrachord is any chord that includes four different pitches. Figure 6-21 illustrates the construction of a subset of modern tetrachords. The top line of this score provides the notes of the C major scale. In the middle line, each of these notes serves as the root of a triad. The added notes are always two scale notes higher than the lower note, so triads are constructed by skipping over notes. For instance, the C major triad is C-E-G, which skips over D and F. Similarly, the D minor triad is D-F-A (skipping over E and G), and so on. This process produces three different major triads (C, F, and G), three different minor triads (Dm, Em, and Am), and one diminished triad (Bdim).

The last line in Figure 6-21 converts each triad into a tetrachord by adding another note from the C major scale. Again, the added note is two scale notes higher than the highest note in the triad. For instance, the Cmaj7 tetrachord is C-E-G-B (skipping over the A to add the B). Similarly, the Dmin7 tetrachord is D-F-A-C, and so on. Each of these tetrachords is a seventh chord. There are different types of these tetrachords created from this process: two major seventh chords (Cmaj7 and Fmaj7), three minor seventh chords (Dm7, Em7, and Am7), one dominant seventh chord (G7), and one minor seventh flat fifth chord (Bm7♭5).

The same approach to chord construction can be applied to any major scale, producing a set of seven different chords for each key. However, if I create these seven different chords for each of the 12 major keys, then I will not create 84 unique chords. This is because when a pitch-class representation is used the same chord will appear in different musical keys. For example, in the set of 84 chords, each min7 chord will appear three different times, and each maj7 chord will appear twice. As a result, our total set of 84 chords will include 48 unique tetrachords and 36 duplicates of some of these chords.

6.4.2 Tetrachord Properties

In order to illustrate networks that solve musical problems by assigning notes to equivalence classes based upon circles of intervals, we will consider a multilayer perceptron that is presented modern tetrachords of the type illustrated in Figure 6-21, and which learns to assign one to each of four different tetrachord classes. Prior to describing this network, let us consider the musical properties of these chords.

Earlier in this chapter, I noted that each different type of triad possesses a particular pattern of musical intervals between adjacent notes. The same is true for the different tetrachords. For instance, consider the Cmaj7 tetrachord in root position, whose notes (in order) are C, E, G, and B. There is an interval of a major third from C to E, of a minor third from E to G, and of a major third from G to B. This pattern of intervals distinguishes this type of tetrachord from the other three types, as can be seen from the third column of Table 6-2 below.

Of course, if one considers the distances between nonadjacent notes in a tetrachord, then there are more intervals than those presented in the third column of Table 6-2. In order to obtain a deeper understanding of the structure of these tetrachords we can use musical set theory (Forte, 1973) to determine each tetrachord’s Forte number, prime form, and interval-class vector (ic vector). The final three columns of Table 6-2 provide the results of this analysis.

Table 6-2 Musical properties of each type of tetrachord in Figure 6-21.

Note. The first column provides the chord type, and the second column provides an example of the chord. The third column provides the structure of the chord in terms of the musical intervals between adjacent pitches. The final three columns provide descriptors of the chord type taken from Forte’s (1973) set theory, including Forte’s classification number for each chord type, the prime form of the chord, and the IC vector that provides the interval structure of the chord.

Two particularly interesting findings emerge from this set-theoretic analysis. First, both the dominant seventh and the minor seventh flat fifth tetrachords have the same prime form and the same ic vector. This means that a network may only be able to differentiate these two tetrachords by considering the specific order in which the component musical intervals occur. Second, the ic vectors for each tetrachord type provide some indication of the musical regularities that a network may be able to exploit to differentiate tetrachord types, as detailed below.

In an ic vector, the first number indicates how many minor second/major seventh intervals occur in a musical object. The second number indicates the frequency of major second/minor seventh intervals. The third number indicates the frequency of minor third/major sixth intervals. The fourth number indicates the frequency of major third/minor sixth intervals. The fifth number indicates the frequency of perfect fourth/perfect fifth intervals. The sixth number indicates the frequency of tritones.

With this understanding of ic vectors, we can now see what the ic vectors in the final column of Table 6-2 reveal. For instance, a major seventh tetrachord is the only one that has a minor second or major seventh interval, and the only one that does not have a major second or a minor seventh interval in its structure. Neither the major nor the minor seventh tetrachords contain a tritone, but the other two types of chords do. The minor seventh tetrachord shares individual ic vector values with each of the other types of tetrachords; this means that it can only be distinguished from them by considering several interval types at the same time. For instance, it can be distinguished from the major seventh by the presence of a major second or minor seventh interval, but the other two types of tetrachords share this property. A minor seventh can only be distinguished from them by detecting the absence of a tritone interval.

Now let us turn to describing the training of a multilayer perceptron to detect these four different types of tetrachords, regardless of the musical key in which they occur.

6.5 Classifying Tetrachords

Figure 6-22 A multilayer perceptron that classifies tetrachords into four different types.

6.5.1 Task

Our goal is to train an artificial neural network, when presented with four notes that define a tetrachord constructed from the notes of a major scale (Figure 6-21), to identify the type of tetrachord (major seventh, minor seventh, dominant seventh, or minor seventh flat fifth), ignoring the key of the tetrachord.

At the end of training, this multilayer perceptron turns one output unit “on” to identify tetrachord type, and turns the remaining three output units “off,” when presented a tetrachord. Thus, this network has four different output units, each one dedicated to representing a particular tetrachord type.

6.5.2 Network Architecture

Figure 6-22 presents the architecture that accomplishes this tetrachord classification task. It uses four output value units to represent tetrachord type, and requires three hidden value units to converge to a solution to this problem. It uses 12 input units to represent input tetrachords in the same pitch-class representation used for the training of the networks in several previous chapters. Figure 6-22 illustrates the presentation of the C major seventh tetrachord (grey input units), resulting in the “Major Seventh” output unit activating.

6.5.3 Training Set

The training set consists of 84 stimuli: the seven different tetrachords for a major scale (see Figure 6-21); these tetrachords are constructed for each of the 12 different major scales. Within this set of 84 stimuli there are 36 duplicate patterns (each maj7 tetrachord appears twice, and each min7 tetrachord appears three times). For the current network, this simply means that these two different types of tetrachords receive more training than the other two types. This difference is not relevant to the point that the network illustrates: the presence of strange circles in the connection weights of its hidden units. I encode each tetrachord as an input pattern in which four input units are activated with a value of one, and the remaining eight input units are all activated with a value of zero. Each input pattern is paired with an output pattern that requires one output unit to activate with a value of one, and the other three output units to activate with a value of zero. The output unit trained to activate is the one that represents the input pattern’s correct tetrachord type.

6.5.4 Training

The multilayer perceptron is trained with the generalized delta rule developed for networks of value units (Dawson & Schopflocher, 1992) using the Rumelhart software program (Dawson, 2005). During a single epoch of training each pattern is presented to the network once; the order of pattern presentation is randomized before each epoch.

All connection weights in the network are set to random values between −0.1 and 0.1 before training begins. In the network to be described in detail below, each µ is initialized to zero but is then modified by training. A learning rate of 0.01 is employed. Training proceeds until the network generates a “hit” for every output unit for each of the 84 patterns in the training set. Again, I define a “hit” as activity of 0.9 or higher when the desired response is one or as activity of 0.1 or lower when the desired response is zero.

I explored a number of different network architectures with this training set. When networks have four or five hidden value units, the problem is very easy and is often solved in a few hundred epochs. However, in order to get a three-hidden-unit network to converge, each µ was modified during training. On some occasions, a three-hidden-unit network would converge very quickly. For instance, the network described in more detail in the next section converged after 11,566 epochs of training. However, on many occasions a three-hidden-unit network would settle to a local minimum and fail to converge to a solution even after more than 20,000 epochs of training. In other words, the network analyzed in the next section required a fair amount of patience during training!

Importantly, all of the networks trained on this problem developed patterns of connectivity that reflect equivalence classes defined by interval cycles. I simply focus on the smallest of these networks because with only three hidden units it is easier to consider some of its properties, such as its hidden unit space.

6.6 Interpreting the Tetrachord Network

6.6.1 Hidden Unit Space

How does this multilayer perceptron identify the four different types of tetrachords? Let us first consider the hidden unit space for this network, illustrated in Figure 6-23. This space is very sparse because the different instances of tetrachord types are very near one another in the space. Indeed, in many cases different tetrachords occupy the same coordinates in this space, which is why it appears to have so few symbols illustrated, even when there are 84 different input patterns plotted in this figure.

We saw such overlapping of points in an earlier hidden unit space, the one illustrated in Figure 4-2. It is important to realize that this feature of the hidden unit space is one of the key properties made explicit by this visualization of the hidden unit space. While the overlapping of symbols in Figure 6-23 seems to make the graph harder to inspect, it delivers a fundamental characteristic: as far as this network is concerned, chords that are of the same type, but which belong to different keys, are identical. This is why the different chords occupy the same location in the hidden unit space.

Figure 6-23 The hidden unit space for a multilayer perceptron trained to identify the four types of tetrachords.

In this space, all of the major seventh tetrachords are at two general locations at the cube’s upper left. All of the minor seventh tetrachords fall along the front right edge of the Figure 6-23 cube. All of the dominant seventh tetrachords fall in two regions in the lower left-hand corner of the cube. All of the minor seventh flat fifth tetrachords fall either in a single tight area located in the upper back region of the cube or in a similar location in the lower left-hand corner at the front of the cube. Importantly, all of these locations of tetrachord types in the hidden unit space can easily be separated from the other tetrachords by two parallel planes that are carved through the space by each output value unit. Let us consider the tetrachord properties detected by each hidden unit.

The fact that all of the different types of tetrachords are near one another, typically in two different areas of the hidden unit space, suggests that each type of tetrachord produces a small number of different patterns of activity in the hidden units. We can confirm this by taking each type of tetrachord and examining the hidden unit activities produced by each. Three of the tetrachord types produced two distinct patterns of hidden unit activity, while the fourth (the minor sevenths) produced three distinct patterns of hidden unit activity. Table 6-3 presents the activity in each hidden unit, averaged over all of the hidden units that fall together in a particular type. Each row of hidden unit activities represents the general coordinates of tetrachord locations in Figure 6-23, and confirms our visual inspection of that figure.

Table 6-3 indicates that each subtype of a tetrachord shares some similarities in terms of some hidden unit activities, but they differ from each other in terms of the other activities that they produce. For instance, consider the three different subtypes of the minor seventh tetrachords. Each of these subtypes is similar in producing very high activity in Hidden Unit 1, and in producing very low activity in Hidden Unit 2. The three differ in terms of the activity that each produces in Hidden Unit 3: one produces very high activity in this unit, another produces near zero activity, and the third produces weak activity.

Table 6-3 The different patterns of hidden unit activity produced by different subsets of each type of tetrachord.

Note. Each subset is given a number and the number of tetrachords that belong to that subset is indicated in the column labelled #. The H1, H2, and H3 columns provide the average activity produced in each hidden unit by a tetrachord that belongs to the subset.

In order to understand these different patterns of activity, and why particular tetrachords produce specific activities in specific hidden units, let us describe the pattern of connectivity between the 12 pitch-class input units and the three hidden units. Once we have a sense of the regularities in these connection weights, we can use this knowledge to explain the regularities of Figure 6-23 and Table 6-3. In the sections that follow, we will consider hidden units from the most easily interpreted (musically) to the least; as a result, the order in which units are discussed will not agree with the names of the hidden units.

6.6.2 Hidden Unit 1

To begin, let us examine the pattern of connections between the 12 input units and Hidden Unit 1 (Figure 6-24). This figure provides a strong indication that this hidden unit classifies input pitch-classes in terms of the circles of tritones. That is, Figure 6-24 exhibits tritone equivalence: pitch-classes that are a tritone apart have essentially the same connection weight.

Figure 6-24 The connection weights from the 12 input units to Hidden Unit 1.

Tritone equivalence in this hidden unit is important because at the end of training its µ had a value of 0.00. We might expect that the presence of a pair of pitch-classes that are a tritone apart would produce an extreme net input, turning Hidden Unit 1 off. The ic vectors in Table 6-2 might lead us to predict that Hidden Unit 1 would therefore turn on to either major seventh or to minor seventh tetrachords (which do not include a tritone). However, the data in Table 6-3 does not support this prediction. Major seventh tetrachords never activate Hidden Unit 1. There must be something more sophisticated within the Hidden Unit 1 connection weights.

In networks described in earlier chapters, we observed a phenomenon called tritone balance. In tritone balance, pitch-classes a tritone apart had connection weights that were equal in magnitude but opposite in sign. As a result, if both pitch-classes were present they would cancel each other out. An examination of Figure 6-24 reveals that these connection weights are balanced, but not in terms of tritones. Instead, this hidden unit balances minor thirds. Pitch-classes that are a minor third apart have connection weights that are equal in magnitude but opposite in sign.

This relationship is explicit in Figure 6-25, which plots exactly the same connection weights from Figure 6-24 but stacks the weights from pitch-classes separated by a minor third on top of one another. The symmetry of this bar graph provides the evidence that these connection weights balance minor thirds.

Figure 6-25 The connection weights of Figure 6-24 re-plotted so that weights from pitch-classes a minor third apart are stacked on top of one another.

Our earlier discussion of circles of minor thirds indicated that, unlike two pitch-classes separated by a tritone, one pitch-class is a minor third away from two other pitch-classes. For example, an examination of the first circle of minor thirds in Figure 6-15 indicates that C is not only a minor third away from A but is also a minor third away from D♯. If Hidden Unit 1 is truly balancing minor thirds, then we expect to find that one pitch-class is balanced with two others, and not just one.

This appears to be the case for Hidden Unit 1. Figure 6-26 presents yet another depiction of its connection weights from Figure 6-24. However, in this second figure each connection weight is stacked against the other pitch-class that is a minor third away (i.e., the pitch-class that it was not stacked against in Figure 6-25). Once again, this figure is very symmetrical, although the balancing is not as perfect as that shown in Figure 6-25. Together, Figures 6-25 and 6-26 reveal that Hidden Unit 1 balances a pitch-class with either of the pitch-classes that are a minor third away from it.

Our investigation of Hidden Unit 1 connection weights has revealed that they assign pitch-classes a tritone apart to the same equivalence class, and they balance minor thirds. Major seventh tetrachords do not include a tritone, but do include pitches that are a minor third apart (Table 6-2). Why, then, do these tetrachords fail to activate Hidden Unit 1? The answer to this question comes from the pitch-class representation used for the multilayer perceptron. This representation rearranges the structure of the various tetrachords, and the hidden units must process this rearranged structure.

Figure 6-26 The connection weights of Figure 6-24 re-plotted so that weights from pitch-classes a minor third apart are stacked on top of one another. Note the difference in stacking between this figure and Figure 6-25.

Table 6-4 presents the pitch-class representation of two major seventh tetrachords in its first two rows; one is an example of Pattern 1 from Table 6-3 and the other is an example of Pattern 2. The key property to observe in both is that after being represented in this format, each contains two pitch-classes that are a minor second apart (A and A♯ in A♯maj7, D and D♯ in D♯maj7). This property is true of every major seventh tetrachord in the training set.

The presence of adjacent pitch-classes in any major seventh input pattern causes Hidden Unit 1 to turn off. This is because any four connection weights that include adjacent pitch-classes do not balance to produce a net input near zero to activate this unit. Instead, major seventh tetrachords that belong to Pattern 1 from Table 6-3 will include two balanced weights (e.g., D and F for A♯maj7), one near-zero weight (e.g., A♯ for A♯maj7), and one extreme weight that is out of balance with the other three (e.g., A for A♯maj7). Major seventh tetrachords that belong to Pattern 2 from Table 6-3 combine four weights that are even more unbalanced, causing more extreme net input (e.g., the D♯maj7 chord in Table 6-4).

Table 6-4 Example pitch-class representations of two major seventh tetrachords and three minor seventh tetrachords, along with the net input they provide to Hidden Unit 1 (Net) and its resulting activity.

Minor seventh tetrachords do not include adjacent pitch-classes in their pitch-class representation for the network, and as a result always include four connection weights that when combined produce near-zero net input to turn Hidden Unit 1 on. Table 6-4 also provides three example pitch-class representations of minor seventh tetrachords (one for each pattern in Table 6-3).

Each of the three minor seventh tetrachords presented in Table 6-4 contains two pairs of pitch-classes that are a minor third apart and have balanced weights; we can see all of these balanced pairs in Figures 6-25 and 6-26. For F♯min7 they are [A, F♯] and [C♯, E]. For Dmin7 they are [A, C] and [D, F]. For Gmin7 they are [A♯, G] and [D, F]. The balancing of each of these pairs of connection weights produces very small net inputs, and high Hidden Unit 1 activities, as presented in Table 6-4. Every other minor seventh tetrachord in the training set also exhibits this property. In other words, for minor seventh tetrachords Hidden Unit 1 behaves as expected from our interpretation of connection weights!

Let us now briefly turn to explaining the activity produced in Hidden Unit 1 by the two other types of tetrachords, the dominant seventh and the minor seventh flat fifth. Table 6-3 reveals that the majority of both types of these chords produce zero activity in Hidden Unit 1. This is consistent with our analysis of this unit’s weights. Earlier, I noted that these weights exhibit tritone equivalence. Therefore, pitch-classes a tritone apart cannot cancel each other’s signal out, because both pitch-classes are associated with the same connection weight. Indeed, in the pitch-class representation of each of the dominant seventh and the minor seventh flat fifth chords that turns this unit off, one finds two pitch-classes a tritone apart. Their combined weights produce an extreme net input that is very far from µ.

What is surprising about Table 6-3 is that a minority of both of these types of tetrachords produce weak activity in Hidden Unit 1. How is this possible if these stimuli include a tritone?

Table 6-5 presents the pitch-class representation of four example tetrachords that produce this surprising behaviour in Hidden Unit 1. All four of these chords include a pair of tones a tritone apart: [A, D♯] in B7, [C, F♯] in G♯7, [D, G♯] in Dmin7flat5, and [B, F] in Bmin7flat5. However, other intervals, when present, moderate the effect of the unbalanced tritone. For example, B7 includes the pitch-classes [A, F♯]; these provide a balanced minor third (Figure 6-26). The same is true for the pitch-classes [C, D♯] in C♯7, [F, G♯] for Dmin7flat5, and [D, B] in Bmin7flat5. The remaining two connection weights together produce less extreme net input (see Table 6-5) that produces moderate Hidden Unit 1 activity. In other words, for this subset of tetrachords, Hidden Unit 1 compromises its activity because it detects one interval that should turn it off (a tritone), but another that should turn it on (a minor third).

Table 6-5 Example pitch-class representations of two dominant seventh tetrachords and two minor seventh flat five tetrachords, along with the net input they provide to Hidden Unit 1 (Net) and its resulting activity.

6.6.3 Hidden Unit 3

Let us next consider the connection weights of Hidden Unit 3 (Figure 6-27). As was the case with Hidden Unit 1, Hidden Unit 3 assigns input pitch-classes to equivalence classes related to circles of intervals. For Hidden Unit 3 these equivalence classes involve the three circles of minor thirds.

All four pitch-classes that belong to the first of these circles in Figure 6-15 are assigned the same negative weight (−0.43) in Figure 6-27. All that belong to the second circle of Figure 6-15 have the same weak positive weight (0.07) in Figure 6-27. Finally, all of the pitch-classes that belong to the third circle in Figure 6-15 have the same stronger positive weight (0.33) in Figure 6-27.

Figure 6-27 The weights of the connections from the input units to Hidden Unit 3.

Unlike Hidden Unit 1, Hidden Unit 3 does not exhibit any obvious balancing between pairs of weights a particular musical interval apart. However, the weights that it assigns to the three different equivalence classes reveal some very interesting properties. If one considers combinations of four different weights, then one discovers specific patterns that cancel net input signals and cause high activity in Hidden Unit 3.

First, it is important to recognize that the value of µ for Hidden Unit 3 is −0.08. This means that for an input pattern to generate a maximum response in this hidden unit, the net input generated by this pattern will be slightly negative.

An examination of different combinations of four weights from Figure 6-27 reveals that there are three different patterns that accomplish this (Figure 6-28). Figure 6-28 arranges the four different bars representing connection weights in such a way that the balance between negative and positive weights is apparent.

The first combination occurs when a tetrachord contains only one member from the equivalence class assigned a negative weight, only one member from the equivalence class assigned a strong positive weight, and two members from the equivalence class assigned a weak positive weight. This pattern is represented as a stack of four bars on the left of Figure 6-28. When I sum these four weight values, the resulting net input is approximately −0.04, which generates activity of approximately 0.95 in Hidden Unit 3. This pattern appears in four different major seventh chords: Emaj7 [B, D♯, E, G♯], C♯maj7 [C, C♯, F, G♯], Gmaj7 [B, D, F♯, G], and A♯maj7 [A, A♯, D, F]. No other tetrachords, including the other major seventh chords, exhibit this pattern.

Figure 6-28 Three different combinations of four Hidden Unit 3 weights that produce net inputs close enough to µ to generate high activity.

Interestingly, the other major seventh chords produce moderate activity in this hidden unit (0.63 as shown in Table 6-3). They exhibit a slightly less optimal combination of four weights than the three shown in Figure 6-28. This involves one weak positive weight, one negative weight, and two stronger positive weights or one weak positive weight, one stronger positive weight, and two negative weights. Any of these combinations produces a net input that ranges between −0.47 and 0.30 depending upon which particular weights are included.

The second combination of four Hidden Unit 3 weights that produces high activity involves two pitch-classes that have strong negative weights and two pitch-classes that have strong positive weights. This pattern is illustrated with the group of four bars in the middle of Figure 6-28. This pattern only appears in four different minor seventh tetrachords: F♯min7 [A, C♯, E, F♯], Cmin7 [A♯, C, D♯, G], Amin7 [A, C, E, G], and D♯min7 [A♯, C♯, D♯, F♯].

The third combination of four Hidden Unit 3 weights that produces high activity involves one pitch-class that has a strong negative weight and three pitch-classes that have weak positive weights. This pattern is illustrated with the stack of four bars at the right of Figure 6-28. This pattern only appears in four different minor seventh flat fifth tetrachords: G♯min7♭5 [B, D, F♯, G♯], D min7♭5 [C, D, F, G♯], Fmin7♭5 [B, D♯, F, G♯], and Bmin7♭5 [A, B, D, F].

One type of tetrachord that does not produce high activity in Hidden Unit 3 is the dominant seventh. The highest activity is produced by an input pattern like A♯7: [A♯, D, F, G♯]. Note that this pattern includes three weak positive weights (D, F, G♯), but the fourth weight is a stronger positive weight (A♯). Because most of these weights are weak, this type of input pattern produces a relatively small net input (0.54). However, this net input is extreme enough to reduce Hidden Unit 3 activity to about 0.29. This pattern is true of eight of the 12 different dominant seventh chords. The other four dominant seventh chords include three of the extreme negative weights balanced by only a single weak positive, producing a net input of −1.23 and essentially turning Hidden Unit 3 off.

Finally, while some minor seventh and some minor seventh flat fifth tetrachords cause Hidden Unit 3 to activate, most do not. All of these tetrachords include a pair of pitch-classes that are a minor third apart. As Hidden Unit 3 exploits minor third equivalence, these pitch-classes do not cancel their signals out. Instead, they produce more extreme net input for Hidden Unit 3, reducing its activity. Importantly, the combined effect of a pair of such pitch-classes is not uniform: [A♯, C♯] has a greater effect than does [B, D] because the former pair has more extreme connection weights than the latter pair (see Figure 6-27). This explains why some tetrachords that include at least one minor third can still produce mild activity in Hidden Unit 3 (e.g., 0.27 produced by some minor seventh input patterns).

6.6.4 Hidden Unit 2

Let us finally consider Hidden Unit 2 (Figure 6-29). Although these connection weights have a very regular appearance, they are less musically general than the weights for both Hidden Units 1 and 3. This is because Hidden Unit 2 fulfills a very specialized task for the tetrachord classification network.

Why might one say that the pattern of weights in Figure 6-29 is less musically general than those we have seen earlier in this chapter? One reason is that the weights in Figure 6-29 do not exhibit any systematic assignment of pitch-classes to equivalence classes. For instance, Figure 6-29 begins by suggesting tritone equivalence because the weight for A is nearly identical to the weight for D♯. However, the weights for the next tritone (A♯, E) are not equivalent, nor do they balance. No other systematic equivalences based upon circles of intervals are apparent in this figure either.

Figure 6-29 The connection weights from the 12 input units to Hidden Unit 2.

We saw earlier that both Hidden Units 1 and 3 organized pitch-classes using interval-based equivalence classes, but also balanced other combinations of pitch-classes related by different intervals. Hidden Unit 2 balances several different pairs of pitch-classes as well. Figure 6-30 illustrates this by presenting the same weights that are in Figure 6-29, but stacking balanced weights on top of each other to highlight their symmetry.

Once again, though, the balancing in Figure 6-30 is not musically systematic. For instance, the balanced pair [A, C♯] is a major third apart, as is the balanced pair [C, E]. However, the balanced pair [A♯, F♯] is a minor sixth apart, while the balanced pair [B, F] is a tritone apart. In short, the connection weights for Hidden Unit 2 seem to balance specific pairs of pitch-classes, and do not balance specific types of musical intervals.

Why does Hidden Unit 2 exhibit properties that are less musically general than those exhibited by the other two hidden units? An answer to this question comes from considering the role of Hidden Unit 2 in arranging input patterns in the hidden unit space.

Figure 6-30 The connection weights from the 12 input units to Hidden Unit 2, with balanced weights stacked on top of each other.

To begin, let us consider the hidden unit space in the context of output unit functions. Figure 6-31 attempts to make this context explicit. On its left is a copy of the hidden unit space presented earlier in Figure 6-23. On its right is the same space, but with an additional four planes. These four planes illustrate that in this three-dimensional hidden unit space all of the input patterns that belong to a particular tetrachord type align along a two-dimensional plane that passes through the space.

The planes drawn on the left part of Figure 6-31 are important in terms of how output units function for this particular network. Recall that each output unit is a value unit. This type of unit carves a three-dimensional hidden unit space into decision regions by placing two parallel planes that cut through this space. Any input patterns that fall between these two planes are patterns that turn the output unit on. These planes are very close together, because a value unit is sensitive to a very narrow range of net inputs. The single planes illustrated in Figure 6-31 are important, because they will fall between the two parallel cuts that an output value unit carves through this hidden unit space. This permits the output unit to respond correctly to these patterns by turning on, and by correctly turning off to any other patterns that do not fall between the two cuts.

What is Hidden Unit 2’s role in arranging patterns in this space? To answer this question, one can redraw the three-dimensional hidden unit space in Figure 6-31 as a two-dimensional hidden unit space. This two-dimensional space arranges input patterns using Hidden unit 1 and 3 activities as coordinates. In other words, this hidden unit space would exist if Hidden Unit 2 was absent from the multilayer perceptron (Figure 6-32).

Figure 6-31 The input patterns in their position in the hidden unit space are illustrated on the left.

Figure 6-32 A two-dimensional hidden unit space for the input patterns created by removing the Hidden Unit 2 coordinate from Figure 6-31.

When an output value unit confronts a two-dimensional hidden unit space, it does not carve it with parallel planes. Instead, it carves two parallel lines through this space; patterns that fall between the two lines turn the output unit on. The lines are very close together, because an output value unit is sensitive to a very narrow range of net inputs. The hidden unit space on the left side of Figure 6-32 illustrates the parallel cuts that can be made through this space by three of the output units. Each of these three pairs of cuts separates one type of tetrachord from all three of the other types, permitting the output unit to classify the chords. The three cuts illustrated on the left of Figure 6-32 demonstrate that this two-dimensional space arranges input patterns that would permit the network to correctly classify all of the minor seventh, dominant seventh, and minor seventh flat fifth tetrachords.

When an output value unit confronts a two-dimensional hidden unit space, it does not carve it with parallel planes. Instead, it carves two parallel lines through this space; patterns that fall between the two lines turn the output unit on. The lines are very close together, because an output value unit is sensitive to a very narrow range of net inputs. The hidden unit space on the left side of Figure 6-32 illustrates the parallel cuts that can be made through this space by three of the output units. Each of these three pairs of cuts separates one type of tetrachord from all three of the other types, permitting the output unit to classify the chords. The three cuts illustrated on the left of Figure 6-32 demonstrate that this two-dimensional space arranges input patterns that would permit the network to correctly classify all of the minor seventh, dominant seventh, and minor seventh flat fifth tetrachords.

The problem with this two-dimensional hidden unit space, though, is that it does not permit the major seventh tetrachords to be identified. This is illustrated on the right side of Figure 6-32. This graph is the same hidden unit space as the one on the left. In this version of the space, two parallel lines are added to capture the major seventh tetrachords (the triangles). Note that this is the only orientation of these two parallel lines that results in all of the triangles falling between them. However, this positioning of the two lines does not separate the major seventh tetrachords from all of the other types: notice that dominant seventh and minor seventh flat fifth chords also fall between these two lines. This suggests that the functional role of Hidden Unit 2 in the multilayer perceptron is to arrange the major seventh tetrachords in a pattern to separate them from the other chords that they cannot be separated from when Hidden Unit 2 is absent. Looking back at Figure 6-31, this seems to be exactly what the Hidden Unit 2 dimension is adding to the hidden unit space. That dimension appears to capture a handful of major seventh tetrachords and pull them toward the back of the cube. This permits these chords to be arranged along a plane, and permits an output unit to define a decision region that only captures these patterns. The other effect of Hidden Unit 2 is that it also draws a handful of minor seventh flat fifth tetrachords to the back of the cube. This suggests that these input patterns possess some musical property that is being used by the hidden unit to pull the major seventh tetrachords to the back.

We can confirm this functional account of Hidden Unit 2’s role in the network by examining the subset of input patterns that produce higher Hidden Unit 2 activity in the context of the connection weights provided earlier in Figures 6-29 and 6-30. First, Hidden Unit 2 generates moderate to high activity to only eight different tetrachords. This confirms our observation that Hidden Unit 2 only moves a small number of input patterns to permit their correct detection. Second, four of the tetrachords that produce moderate activity in Hidden Unit 2 are major seventh chords whose properties are provided below in Table 6-6. This observation is important because we noted above that the key function of Hidden Unit 2 is to enable this type of chord to be classified; major seventh chords are the only chords that cannot be correctly separated in the two-dimensional pattern space.

Table 6-6 The four major seventh tetrachords and then the four minor seventh flat five tetrachords that produce moderate activity in Hidden Unit 2.

Note. The grey cells indicate the one difference between the notes of each of these major seventh tetrachords and the minor seventh flat five tetrachord to which it is similar. The “Net” column provides the net input produced for Hidden Unit 2, and the “H2” column provides Hidden Unit 2 activity.

Third, the four tetrachords that produce high activity in Hidden Unit 2 are all minor seventh flat fifth chords that are nearly identical to the four major seventh chords that produce moderate activity in this same unit. They are nearly identical in several respects. They share three pitch-classes with one of the major seventh tetrachords. Their remaining pitch-class is only a minor second away from the fourth pitch-class. Finally, the connection weight associated with the fourth pitch-class has a similar value to the connection weight associated with the fourth (i.e., the dissimilar) pitch-class in the major seventh chord. Table 6-6 provides the properties of the four minor seventh flat fifth tetrachords. The grey cells in Tables 6-6 indicate the single difference, in either a pitch-class or a connection weight, between a major seventh tetrachord and its similar minor seventh flat fifth tetrachord.

The high specificity of the connection weights presented in Figures 6-29 and 6-30 now make perfect sense in light of the properties detailed in the two tables above. First, these connection weights capture very specific relationships (and not general musical properties) because all Hidden Unit 2 really has to do is move four major seventh tetrachords away from the others in the three-dimensional hidden unit space. If this unit detects properties that are more general, then this would affect the position of a larger number of tetrachords. Second, the specific balancing observed in Figure 6-30 nicely accomplishes the main task of Hidden Unit 2. The A♯maj7 chord produces moderate activity in this unit because the weights from A and from D roughly balance one another, as do the weights from A♯ and F. A similar rough balancing of pairs of pitch-classes is true for Emaj7. The remaining two major seventh chords balance one extreme positive weight (C for C♯maj7, B for Gmaj7) with a combination of three negative weights. Third, the four minor seventh flat fifth tetrachords that produce high activity in Hidden Unit 2 do so because they are each nearly identical to one of the major seventh chords that this unit moves. They differ in only one pitch-class, and this difference is only a semitone. This in turn means that one structural property of the weights in Figure 6-29 is that pairs of pitch-classes a minor second apart ([A♯, B], [C♯, D], [E, F] and [G, G♯]) must have similar weight values. An inspection of this figure indicates that this property is indeed apparent.

6.7 Summary and Implications

This chapter began by describing a multilayer perceptron for classifying triads regardless of their key or inversion. This network revealed interesting patterns of connection weights between its input and hidden units. In particular, its connection weights organize pitches and pitch-classes into different subsets. For instance, Hidden Unit 1 assigns weights so that [A, C, D♯, F♯] define one subset, and similarly uses its weights to organize [A♯, C♯, E, G], [B, D, F, G♯], and [C, D♯, F♯, A] as three other subsets. That is, pitch-classes that belong to the same subset have a very similar weight, but pitch-classes that belong to different subsets have very different weights. We see similar sorts of organizations, involving different subsets of pitch-classes, in the other hidden units as well.

The chapter then proceeded to describe the various interval cycles created by moving set distances from pitch to pitch along a piano keyboard. These interval cycles are a standard element of post-tonal music theory (Roig-Francolí, 2008). I noted that they also provide a convenient formalism for the subsets of pitches and pitch-classes picked out by connection weights. This in turn led to the notion of strange circles. A strange circle is a set of pitch-classes that belong to the same interval cycle. The circle is strange because each member of the same strange circle has the same connection weight. This means that a strange circle is an equivalence class of pitch-classes. It captures a set of pitch-classes that are distinct in Western music, but identical from the perspective of a hidden unit in a network.

The chapter ended by providing a detailed interpretation of the internal structure of a multilayer perceptron that learned to classify tetrachords into four different types. This network provided another example of using strange circles. Both Hidden Units 1 and 3 of this network organize pitch-classes into equivalence classes based upon interval cycles. Hidden Unit 1 assigns the same weight values to two pitch-classes that belong to the same circle of tritones. Hidden Unit 3 assigns the same weight values to three pitch-classes that belong to the same circle of minor thirds. Furthermore, the particular weight values that both Hidden Units 1 and 3 assign to each equivalence class are very systematic. Their values permit pitch-classes separated by other musical intervals to balance, increasing hidden unit activation.

Taken together, these two above observations indicate that Hidden Units 1 and 3 detect general musical properties that permit varied and useful responses to pitch-class combinations that are either part of, or not part of, specific tetrachord structures. Indeed, Figure 6-32 indicates that these two hidden units alone are capable of supporting the correct identification of all of the members of three of the four types of tetrachords. The network’s remaining hidden unit, Hidden Unit 2, detects very specific properties (i.e., properties related to a small number of individual chords, and not to a larger set or chord types) that serve to arrange the major seventh chords in hidden unit space in such a way that they can be correctly identified. The specific properties detected by this hidden unit also capture four different minor seventh flat fifth chords. There is a strong musical relationship between these chords and the major seventh chords segregated by Hidden Unit 2.

To relate this interpretation to material covered in Chapter 4, these three hidden units provide another example of coarse coding. None of the hidden units detects a specific property that is consistent with only one type of tetrachord: two or more different types of tetrachords can produce high activity in any of the hidden units. However, when we consider the activities produced by an input pattern in the hidden units simultaneously, we can identify the input pattern’s type.

The connection weights for the two more general hidden units (Hidden Units 1 and 3) of the network interpreted in Section 6.6 have one interesting implication to keep in mind for later network interpretations: both of these hidden units use their weights to organize pitch-classes by more than one musical interval. The reason that this is possible is because some of the interval cycles introduced in Section 6.3 relate hierarchically to others. For example, each of the circles of major seconds contains the pitch-classes that belong to two of the circles of major thirds. Similarly, each of the three circles of major thirds contains the notes that belong to two of the circles of tritones. These hierarchical relationships permit one set of connection weights to organize input pitch-classes in complex ways. For example, one hidden unit could use the sign of the connection weight to separate pitch-classes into the two circles of major seconds. However, variations in magnitudes of these same weights can simultaneously be used to organize the pitch-classes into circles of tritones because of a hierarchical relationship between the two. In Chapter 7, I explore a more complex network trained on an elaboration of the tetrachord task, and discover that many of its hidden units exploit the hierarchical relationships among strange circles in their representation of input pitch-classes.