Musical performance and the machine: thoughts on the interplay between technology and the human musician
Paper given at Coventry University National Music Performance Symposium (10th June 2009)
‘Bravo for this software. Works like a Swiss watch’. (R. Sanchez, Cirque du Soleil)
In March 2004 the New York Times reported on a legal tiff between Local 802 (the New York branch of the American Federation of Musicians) and a music technology company called Realtime Music Solutions. At the heart of the dispute was Sinfonia, a new ‘virtual orchestra machine’ designed to substitute for an entire pit orchestra in performances of musical theatre. Realtime’s gripe concerned a recent deal made between Local 802 and the Opera Company of Brooklyn to ban the machine from future productions. Rumblings had been apparent since the previous year when Broadway musicians had gone on strike over attempts to reduce the number of pit orchestra players, at which point the machine had been used as a fill-in to cover the shortfall. Broadway was shut down for three days as a result of this strike and the machine was subsequently demonized.
Sinfonia is essentially a sequencer which is programmed with full orchestral accompaniments of entire Broadway musicals. Each score is articulated via a high quality sample unit containing banks of orchestral sounds similar to many of the top-end libraries presently on the market, such as East West Symphonic, Vienna Symphonic Library and Garritan. What makes the device unique is the fact that a single musician controls the entire set of orchestral parts from a single MIDI keyboard. The player, who is referred to as a Sinfonist, can if he or she wishes, control the musical lines of the performance individually, transpose sections, mute parts and so on. Of particular interest is the ability to control tempo on the fly so that the device can be synchronized with live musicians. It has a ‘TAP’ key which when struck according to an assigned value (crotchet, minim etc) moves the music along with the tempo of the performance which is set by a real conductor. There is even a ‘CRUISE TAP’ facility which can be likened to ‘cruise control’ on a motor vehicle, allowing the operator to sit back and relax as the machine churns along. It is this aspect which enables the machine’s creators to sidestep accusations that their device is a substitute for ‘liveness’.
Realtime state that they have a ‘strong philosophical commitment to the theatrical tradition and the process by which technology is carefully integrated into the performing arts’. ‘Integrated’ is a cautious choice of word and really just softens the blow. Although the machine might have hoped to take its place alongside its human counterparts it effectively came across as a musical ‘Terminator’ which threatened to stand in wholesale for the live orchestral medium. Writing on the virtual orchestra machine in Local 802’s journal, Allegro, union member Doug Caine commented that:
‘As professional musicians we are obligated to take into account not only the bottom-line, profit-and-loss side of the equation, but we must hold fast to a transcendent artistic truth: the music we are called to perform was written to be performed those who have spent their lives perfecting our craft.’
This statement is typical of Local 802’s rhetoric and encapsulates the historical concern of musicians’ interest groups to protect their livelihoods by upholding received ideas of the notion of musical performance. Such groups, which Stanley Godlovitch has referred to as ‘performance communities’, are collectives which operate in the manner of the professional trades ‘Guilds’. They ‘establish membership credentials, regulate standards of proficiency and ensure consistency in recognition of individual merit’. The performance ideologies of such communities are reinforced by educational institutions which set themselves up as beacons for standardisation. One only has to think of the graded examination system of the Associated Board of the Royal Schools of Music whose prescribed pedagogical materials construct solo musical performance in terms of a facility which is acquired through progressive stages of apprenticeship. Its teaching is is based in deep rooted tradition, founded on repertoire dating back centuries and streamlined technical exercises (major and minor scales and arpeggios etc) which must be overcome to earn the right to be considered a performer. Technological developments in the form of machines like Sinfonia have typically presented a challenge to the notion of the hard-won path to performance proficiency because they call into question the requirement for such apprenticeship. Programmable sequencers effectively undertake the activity of performing on behalf of their operators once they have been set in motion. The question of what constitutes human agency in musical performance is redefined as the machine already possesses the ‘technique’ to carry out the task- it doesn’t ever need to acquire and maintain its skills through regular practice as the human being does. Performance as it is recognized traditionally is challenged because it exposes the process and effectively short circuits it. However to be seen to be working through the process of performing is crucial to the perception of ‘liveness’ which ultimately bestows authenticity.
It would seem appropriate at this point to allow the neo-Luddities to wade in with their sledgehammers to deal with what appears to be a clear-cut case of a technology dabbling in an activity in which it has no place. Before doing so however let us dissect for a moment this position with a reminder that musical performance has always been subject to some degree to technological mediation. Jacques Barzun puts it like this:
‘the moment man ceased to make music with his voice alone the art became machine-ridden. Orpheus’s lyre was a machine, a symphony orchestra is a regular factory for making artificial sounds, and a piano is the most appalling contrivance of levers and wires this side of the steam engine.’ (Eisenberg, p. 176).
Barzun’s factory metaphor is convincing and is echoed by communications theorist Marshall McLuhan in his seminal work Understanding Media, when he describes the orchestra as a ‘natural model of the mechanical and industrial age’. A complex technology designed to bring a sound image into existence, the orchestra was a reflection of the fragmentation and specialization which characterized industrial process. As in a real factory, orchestral process over time became streamlined for efficiency with the result that what was at first a loosely organized raggle-taggle bag of individual timbres became a highly disciplined organism of collaborative departments or families. Instruction manuals (or treatises) by specialist orchestral programmers such as Berlioz, Rimsky-Korsakov and others evolved in late nineteenth century to systematize the orchestra’s operation, documenting every nuance of is functioning. Orchestral music itself accumulated a repertoire of machine-like gestures beginning in the eighteenth century when the Mannheim school developed its peculiar effects (the ‘rocket’, the ‘sigh’ and the ‘birds’ for example). By the nineteenth century the orchestra had a well established repertoire of devices and cliches specific to its participant departments. By the twentieth century composers were refashioning the orchestra into an actual machine - consider George Antheil’s Ballet Mecanique (1924), for example, and the factory music of the Soviet composer Alexander Mosolov (the ballet Steel, 1927) - while the Futurists were pointing out that the orchestra was really just a stand-in for a machine and suggested that actual machines (noise producing devices) take over its role entirely.
Uniformity and anonymity have characterized the orchestral musician’s existence for centuries - one only has to consider the robotic synchronisation of string bowing insisted upon by Corelli and Lully’s Quatre-Vingt Violons du Roi of the court of Louis XIV in the Baroque period. Like the manual labourer in the factory, each individual player in the orchestra is little more than a moving part in a contraption much bigger than him or herself and has no other function than to adhere strictly to a set of fixed instructions. While each orchestral musician might employ considerable skill in operating his/her own device, individualism is not permitted. The rest of the group might fall back on occasion to listen to a comrade emit a pleasant solo on the flute in bars 150-158, but this was a tidbit thrown to him by the composer, who is of course a co-owner of the factory. To take a solo in an orchestral performance is rather like being ‘employee of the month’ for thirty seconds. By way of pacification of the throng, the occasional instrumental virtuoso was permitted to rise above his station if he proved himself particularly adept with his machine. However, orchestral performances ultimately comprise the manifestation of the musical outlook of a single individual, the conductor, who in terms of his manner of controlling the proceedings is really not that different from Sinfonia’s keyboardist. The conductor is rather like the leader of a chain gang in fact - he wields a small piece of wood rather than a shotgun but his authority is equally unchallenged. What I am casually suggesting here is that the orchestral player’s lot, characterized as it is by a certain amount of servitude, is not necessarily a happy one.
McLuhan saw all technologies as extensions of humankind - for example, ‘the wheel as an extension of the foot, clothing as an extension of the skin, the phonetic alphabet as an extension of the eye’. At the point of its emergence, according to McLuhan, a new technology produces a physical disjuncture between the human being and the activity which it now takes responsibility for. A car, for example, numbs the foot from the trauma of long-distance travel and protects the driver from the impact of high speed movement. The development of musical instruments might equally be viewed in terms of an increasing performer disjuncture from the labour of sound production. Different instruments engender varying degrees of mediation dependent upon what they demand of the performer in their operation: a woodwind instrument, for example, requires the direct engagement of the lungs but mechanical keys are used to stop the notes. Drummers use their arms and feet as levers but are still a step removed from the source of the sound by thin pieces of wood. Conversely, classical guitar playing involves a good deal of adaptation of the human physical mechanism to the instrument to produce the desired musical end. Classical guitar technique in fact often appears to involve effort that is disproportionate to the musical result, necessitating considerable contortion of the performer’s hands and the development of calluses on the fingers - in other words the performer submits a good part of him or herself to the machine. This of course is viewed as a positive attribute of the process by the performance community. The ultimate mechanical device for rendering music in an uninvolved fashion was the 19th century piano which enabled detachment by means of its lever system from direct contact with the strings. Its efficiency of design meant that a pianist’s physical involvement beyond the obvious economical movements of the hands is hardly apparent to the viewer unless the performer makes an effort to exaggerate this. Interestingly developments in popular music after 1950s saw performers attempt to overcome this sedentary relationship. Jerry Lee Lewis, for example, incorporated leg movements into his performances, broad sweeping gestures with his arms and hammered the keyboard in the manner of a drummer. This attitude, which has been retained in the work of more recent popular music performers such as Jamie Cullum, is designed to remind the audience that there is a human being at work here, not a machine. To recapitulate: music technology at every stage of its evolution has typically been concerned with facilitating the means by which sound-producing devices operate and this has produced a number of responses ranging from indifference to explicit attempts to refashion the human relationship to the device in question. Automation, as promised by Sinfonia, might be seen as the logical last step in the process of releasing the human being completely from involvement.
In their defence of the product, David Smith and Frederick Bianchi, the developers of Sinfonia, rely heavily on the idea of the role of mechanized processes in music-making:
‘The fact is that so much of what an instrumentalist does is preordained […] [P]it musicians actually have very little choice about what they do. If they change a pitch, dynamic level, rhythm or other information, what they have done is just plain WRONG. So much of what a musician does is mechanical.’
Sinfonia is the result of an investigation into an analysis of the music it is intended to perform to ‘determine those aspects that are fixed and assign that responsibility to a computer instrument. Those aspects that require human expression can then be given to a human performer, who has been freed from mechanical production requirements’. Smith and Bianchi identify a particular distinction between monophonic and multiphonic instruments to highlight the range of what it is necessary to control. Multiphonic types, they say, require ‘the performer to generate many notes simultaneously, require less responsibility for the production of each note. […] This relationship between polyphony and responsibility is obvious, and created by the limitations of the human bandwidth.’
This opens a can of worms where musical performance is concerned because it suggests that specific musical gestures might be potentially ranked in importance relative to a perception of the extent to which individualized human agency is required in their rendition. The implication is that the importance of each gesture would presumably be decreased in proportion to the increase of the number of participant musicians. It thus becomes increasingly difficult to argue for individualized human agency in any collaborative musical effort once the number of players exceeds say four instruments. Anything that could be defined as ‘accompaniment’, for example, where several instruments are combined to form sustained chords behind a melody - might conceivably be assignable to the responsibility of a machine.
Beyond the categorisation of individual player roles in terms of their functionality, this approach might easily be made to extend to genre typecasting whereby, for example, such factors as the circumstances of the performance of a particular kind of music, or the limits of its interpretative scope, might qualify it to be reduced as a whole to a mechanized process. To take the example of the Broadway Musical for which Sinfonia was originally intended, could one succeed with an argument that such music lends itself well to machine-based rendition if it could be pointed out that its performance values are concerned with the repeated verbatim reproduction of a largely immovable text for weeks on end with little perceptible variation? Many musical texts performed by orchestras are essentially fixed in this manner and, to compound matters, already very familiar to audiences as a result of over-exposure in recordings. The fact that such performances are essentially a foregone conclusion as far as their content is concerned must make them an attractive propostion for the Sinfonia repertoire storehouse. What exactly is valuable about any kind of musical performance in that case? Surely this must lie elsewhere.
One of Marshall McLuhan’s most stimulating ideas concerns the manner in which any new medium has a tendency to be employed retrospectively during the period in which its impact is being assimilated. In effect the new medium acts as a container for the preceding media which it renders obsolete, an effect which McLuhan refers to as the ‘rear-view mirror’. Local 802 representative Harvey Mars observed precisely this phenomenon in regard to Sinfonia, when he commented that the machine has ‘a faux keyboard to make it look like a musical instrument. However these keys do not actually play notes.’ The keyboard in Mars’ eyes is designed to create the illusion of an instrument behind which is something considerably more alien. The extent of the computer’s rear-view mirrorism can be seen in its use in all walks of life to relieve the drudgery of execution of numerous tasks that had previously been undertaken in a mechanical fashion by human beings. With specific regard to the activities undertaken by musicians prior to the computer’s appearance, its impact has been considerable. For example, in the early 1990s the first incarnation of Sibelius Software’s score-writer removed in one fell swoop the labour-intensive task of writing music down on paper. It made explicit the fact that much of the activity of creating musical scores prior to the computer’s involvement was repetitive in nature, often involving the copying by hand of passages of music dozens of times over. In the spirit of neo-Luddism - that is, a distrust of the new medium - it took composers whose medium was the score some time to realize that they had been freed, such was the extent of their belief in the necessity of this drudgery as part of the compositional process. As per my earlier point on the importance of performance apprenticeship, this probably stems again from a need on the part of artists to relate the worthwhileness of their creations proportionally to the expenditure of effort in their production.
The thinking behind the use of the computer to reproduce older musical performance media, as demonstrated by Sinfonia, has been bound up with the same criteria - namely to purge the drudgery of manual execution by rationalising the process that is involved and letting the machine facilitate the activity. Realtime Music Solutions’ website states that their team has, ‘identified and solved a myriad of musical, acoustical, technical and procedural problems indigenous to the live performing arts’, which encapsulates this attitude succinctly. From one perspective Realtime has a point: when considering the complexity of coordination involved in rendering a piece of orchestral music in sound, the relative inefficiency of that medium is obvious when compared to the processing power of a modern computer. Nobody is really disputing this. The Sinfonia tiff draws attention to the fact that what Realtime Music Solutions perceives is a problem that needs to be resolved constitutes, as pointed out by Local 802, the very aspect of the performance process that is crucial in their view to the identity of a genuine live musical performance. While it is certainly possible to view the orchestra as a machine-like entity, it does not necessarily follow that an actual machine is required to take over its role with a view to liberating the musician from bondage. When Sinfonia acts as a container for the orchestral medium, it is entering into territory governed by all kinds of preconceptions of the notion of performance which relate not just to the role of human agency but also to the nature of the medium that has already constructed the idea of what musical performance in this context is. Computerisation under the supervision of a single individual obviously negates the human striving that is so essential in the creation of a meaningful performance, and which exists in the context of a subtlety of flux which is outside the realms of what can be encapsulated in rationalized process without a considerable amount of contrivance. Lewis Mumford comments to this effect that, ‘The most ineffective kind of machine is the realistic mechanical imitation of man or another animal’. The computer medium, like the orchestral medium does without a doubt have an important musical message of its own to convey, but it is one that looks forward rather than backwards.
Bibliography
Auslander, Philip, Liveness (Abingdon: Routledge, 2008).
Auslander, Philip, ‘Liveness: Performance and the anxiety of simulation’ in The Popular Music Studies Reader.
Banham, Reyner, Theory and Design in the First Machine Age (London: Architectural Press, 1960).
Barnett, David, The Performance of Music (London: Barrie and Jenkins, 1972).
Bianchi, Frederick and Smith, David B., ‘Arguments for the Developing of Enhancement Solutions for Orchestras in Live Performance’ (http://www.rms.biz/index.cfm?page=podium&podium=Aesthetics)
Boberg, Meg, ‘Controversy Surrounding Virtual Music’, Journalism Research, 16.12.2005.
Eisenberg, Evan, The Recording Angel (New York: Penguin, 1988).
Emmerson, Simon, ‘ “Losing touch?”: the human performer and electronics’ in Emmerson, S., ed., Music, electronic media and culture (Aldershot: Ashgate, 2000), pp. 194-215.
Fleischer, Rasmus, ‘”Mechanical Music” as a threat against public performance’ (unpublished article, May 2006).
Godlovitch, Stanley, Musical Performance: A philosophical study (London: Routledge, 1998).
Jones, Steven, Against Technology: From the Luddites to Neo-Luddism (Abingdon: Routledge, 2006).
Leff, Jason, ‘Rage against the machine: How the NLRB used Section 8(e) of the National Labor Relations Act to Kill the Virtual Orchestra’, North Carolina Journal of Law and Technology, 6, 1 (Fall 2004).
Levinson, Paul, Digital McLuhan (New York: Routledge, ).
Mellers, W., ‘Present and Past: Intermediaries and Interpreters’ in Paynter, Howell, Orton and Seymour eds, Companion to Contemporary Musical Thought, 2 (London: Routledge, 1992).
‘Moriarty Interview’ (http://www.rms.biz/Documents/UnionDebate/moriarty.htm)
McLuhan, Marshall, Understanding Media: the extensions of man (Abingdon: Routledge, 1964).
Schmidgall, Gary, ‘Making the Virtual Orchestra a Reality’ (July 2003).
‘Synthesizer Maker Fights Musicians’ Union’, New York Times, 12.3.2004.
Trescott, Jacqueline, ‘Opera House forgoes “Virtual Orchestra” in Kennedy Center Contract’, Washington Post, 18.10 2005.