The difference in depiction of bow movement between Mili’s photographs and the kinetic signatures of the FiddleLights Project is that Mili attached the flashlight to the end of the bow, which is perpendicular to the strings. I have attached the light to my bow hand, which is parallel to the strings, and creates more diversity of gesture. For Mili to accomplish the effect that he achieved in the kinetic portrait of Heifetz playing the Bach Suite No. 3, Prelude, he moved the camera to the right as Heifetz was playing.

I have recently discovered these photographs, well into the third year of working with the FiddleLights Project, and I wondered why Heifetz didn’t suggest showing the hand movement rather than the movement of the end of the bow. My guess is that

Heifetz had never really thought of the gestures drawn by his hand movements. Mili, as an observer of a violinist’s movements from an audience point of view, which is traditionally a side view of the violin, would easily connect with the movement of the bow stick itself, rather than the movement of the hand. In addition, he would not have seen the results of the photographs until after they were printed. The digital photos, which I am able to work with, provide a more immediate result, enabling a shorter time frame between experimentation and evaluation.

In effect, Heifetz was a violinist who really didn’t understand what Mili was attempting to accomplish by recording the gestures of his bow, and Mili was a photographer who didn’t really understand that the bow gestures originated from the hand. I am an artist and a violinist, and have thought about the gestures of my bow hand for many years, although I have only recently realized them in the form of photographs and video. Mili’s photographs of Heifetz are the only photographs that I have discovered in which another artist has attempted to depict the movements of the bow hand on a bowed string instrument.

4.2 Kinetic Portraits of the FiddleLights Project

From examining kinetic signatures of various categories of fiddle tune dances; reels, jigs, airs, waltzes all had their own individual gestures and visual character. This led to a question. Could gestures be related not only to the type of dance, but also the genre of tradition, such as Irish, Scottish, Old Time, etc. or even individual players?

Mili explains that his basic intent as a portrait photographer was to establish character. He described how subjects of portraits would try to thwart the photographer at every turn by posing, by making a point of belittling themselves in front of the camera, or by thinking they know exactly how they should look. It is then the job of the photographer to free the subject of their inhibitions. (Mili 1980. p.140)

Mili manages to achieve this in his portraits, the flash often catching the subject unaware. He explains that he would engage his subjects in conversation, encouraged actors to act, musicians to play, athletes to show their prowess. Because the subjects were engaged in something else which occupied their attention, he was able to catch the nuance of gesture in his portraits, not only of facial expression, but the expression of the entire body.

Capturing kinetic portraits in the FiddleLights Project uses strategies similar to those of Mili by involving the subject in playing the fiddle. The result is a candid portrait, honestly capturing the subject’s character through gesture.

In July 2012, I presented my paper The FiddleLights Project – an Artist’s Recording of Bow Hand Movements in Fiddle Tunes at the North Atlantic Fiddle Convention 2012 at the University of Ulster, N. Ireland. While I was there, I had the opportunity to record the gestures of two renowned fiddlers from different traditions. Dr. Matt Cranitch is a departmental tutor at the University of Cork, Republic of Ireland, and Alasdair Fraser is a Scottish fiddler currently residing in California, United States. I wanted to compare the kinetic signatures of a traditional well-known tune, as performed by three individual fiddlers from three diverse geographical fiddling traditions. The third fiddler is myself, from a Canadian tradition that contains many style influences.

I chose Mason’s Apron, a well-known and popular reel in fiddling circles. The results can be compared in Figures 6, 7, and 8. As we can see, the three versions of this melody are visually diverse. The variation of the brightness in the drawings is due to the brightness of the laser pointers used, which is difficult to control.

FiddleLights: Kinetic Signatures of the Bow Hand- Visualising Fiddle Tunes and Contemporary Violin Performance

Submitted to

Electronic Visualisation and the Arts 2013

British Computer Society

London, England

April 5, 2013

ABSTRACT

FiddleLights is a body of practice-based research exploring the kinetic signatures of fiddle tunes in relation to the sound produced by the bow on the fiddle. Experimenting with photography, video, and computer enhanced compositing, FiddleLights creates light-based visualisations of movement and technique captured from the bow-hand of individual fiddlers, resulting in kinetic signatures of individual tunes, and kinetic portraits of the fiddlers, themselves. This technique is then used to inform musical compositions to create improvised musical performances of light and sound.

1. INTRODUCTION

FiddleLights is a body of practice-based research exploring the kinetic signatures of fiddle tunes in relation to the sound produced on a violin. Experimenting with photography, video and computer enhanced compositing, FiddleLights creates light-based visualisations of movement and technique captured from the bow hand of individual fiddlers. This technique is then used to inform musical composition, creating a synthesis of light and sound, in a multi-level installation where the music and the drawing are inextricably entwined, in process as well as product.

The word fiddle is the Old English word for the instrument, as crwth in Welsh and Irish (Apel 1979) or Geige in German. Our word violin comes from the Italian, violino. In this paper, the term violin will be used when referring to the instrument. However, fiddle music references music played from folk traditions, and violin music will refer to classical and improvised music played on the violin.

The fiddle dance terms used in this paper are waltz, jig and reel. A waltz is a dance in 3/4 time, at either a slow speed with a feel of three beats per measure for a romantic effect, or a fast speed with a feel of one beat per measure, which is popular in country waltzes. A jig is a fast dance in 6/8, but with a feel of two beats per measure. A reel is a fast dance in 2/4 with a feel of two beats per measure.

The term kinetic signature refers to the time-lapse photos of a particular tune. Kinetic portrait alludes to the light recording of an individual fiddler. Fiddle Trails describes the processed video which leaves a trail (after image).

2. CONCEPTION OF THE FIDDLELIGHTS PROJECT

The shapes that my bow hand draws in the air when I play the violin have always fascinated me. Sometimes when teaching fiddlers and violinists, I have drawn my students’ attention to the shapes made by their bow hand as they play. When they smooth out the curves and pay attention to their movements, their playing becomes more graceful, with smoother transitions between strings and between bow strokes.

As an artist, I was very curious as to how I could represent the bow hand movement used in playing the fiddle in a two-dimensional visual form. My theory was that I would discover basic shapes such as circles, ovals and figure eights. I tried many different methods of actualizing the shapes that I create with my bow hand when I play. I tacked a piece of paper to the wall. Holding a felt pen between the fingers of my bow hand, I played my violin, transferring for the first time the shape my bow hand was making in the air onto the paper. Immediately, I was surprised by the diversity of gesture these photos began to show. Every tune had its own unique kinetic signature.

Although this method was showing me how varied the shapes were for each tune, the resistance of the drawing tool with the paper confined my gestures. The result was an inexact representation of the movement.

In the spring of 2011, I began experimenting with time-lapse photography and a light source to trace the bow hand gestures. I taped a laser pointer to my bow hand, so that the light was pointing off my knuckle, and shining on the wall at a 90° angle. I turned out the lights, so that the only light recorded in the photograph would be from the laser pointer. Because the laser pointer weighs so little as to be almost imperceptible, I was able to play with almost total freedom of movement. In Figures 1, 2 and 3, we can compare the kinetic signatures of three different fiddle tunes. They represent three different dance forms. Figure 1 is a waltz, Figure 2 is a jig, and Figure 3 is a reel.

3. 3.KINETIC SIGNATURES
   

Following the visual clues, the music itself can be analysed through these kinetic signatures. The position of the lines within the frame is related to pitch. The density and length of lines indicate the speed of the music. Circles and figure eights represent string crossings. Correlating the position of the lines in the photograph to the movement and position of the bow hand, the lower pitched notes on the fiddle, played on the G string are represented by the higher lines in the drawing. The lines in the lower regions of the drawing represent high-pitched notes played on the E string. For example, in Figure 1, the drawing Snow in April is less dense than the other two. This suggests a slower tune. The second tune (Figure 2) Fairhaired Boy is predominantly circles and stylized figure eights. This indicates many string crossings. In the third kinetic signature (Figure 3), Growlin’ Old Man and Old Woman, the high density of marks indicates that it a very fast tune with many bow strokes as well as layers of figure eights. I discovered that fiddle tunes provided material more conducive to visualisation than classical music. Classical music tends to have sections where the ideas are developed over a long period of time, with more similar motivic repeated gestures, while fiddle tunes encapsulate a diversity of motion within only 16 bars of music. For each of the kinetic signatures, I played the A section and the B section of the fiddle tune only once, without the traditional repeats. This allowed for more space in the drawing.

For a more detailed musical analysis of kinetic signatures of fiddle tunes, see The FiddleLights Project: an Artist’s Recording of Bow Hand Movements in Fiddle Tunes (Sereno-Janz 2013).

4. KINETIC PORTRAITS

As every violinist/fiddler has a unique sound and interpretation of a tune, every kinetic signature, in recording the exact movements made in the manipulation of the bow, becomes a unique kinetic portrait, generating a unique visual reference of the artistic expression of each individual fiddler.

4.1 Gion Mili’s Kinetic Portraits of Jascha Heifetz

In the June 1952 issue of Life Magazine, Gjon Mili (1904-1984), an American freelance photographer working for Life Magazine, published a series of portraits of Jascha Heifetz (1901-1987), an eminent concert violinist of the time. Mili, trained as an engineer at MIT, became very interested in the technology of flash photography and the possibilities of depicting movement in a single frame of film.

In these portraits of Heifetz (Figures 4, 5) Mili captured the gesture of Heifetz’s bow with a flashlight attached to the end of the bow. This was photographed in the dark, so that the only light recorded was that made from the flashlight. The portraits of Heifetz’s face were achieved through flash photography.

Cranitch’s version of Mason’s Apron, Figure 6, contains many more curly shapes and loops within the scope of the larger gestures. From previous analysis (Sereno-Janz 2013) we know that the loops and curly shapes are indicative of lilting sounds in the music itself. In my version (Figure 7), there aren’t nearly as many small curls and loops in the shape. There is an abundance of straight lines, indicating fast single bow strokes. There are circles and stylized figure eights, but the dominant line orientation is vertical. Figure 8, the performance by Fraser seems to be a combination of the other two. He seems to have quite a few of the vertical gestures which are common in Figure 6, but also shows quite a few of the smaller loops and curls within the larger shapes.

Although there is considerable diversity in these three performances of Mason’s Apron, we notice there are similarities as well. The centre of these kinetic portraits all contain the highest density of line, which corresponds to bow strokes on the A and D strings. All three also contain a sweeping gesture on the lower right side of the drawing and stylized figure eights in the upper right corner.

I think that it is fascinating to consider that these three figures are kinetic portraits not only of genre and style, but also a personal expression of the character of the players themselves. In capturing the movements of the bow of each of the fiddlers in the process of playing a tune well known to them, they are caught in a space of time, their body moving in a well-rehearsed dance of its own.

5. FIDDLE TRAILS AND VIDEO PROCESSING

The FiddleLights’ kinetic signatures are the entry point into a time based, animated experience through video. When viewing the kinetic signature, it is often difficult to find where the movement starts and where it ends. It is one level of experience to look at the kinetic signature of a tune and another to imagine how it sounds. The next step was to record the music and the gesture with video to simultaneously provide visual and aural experiences. Initially I experimented with video capture of the point moving in space, but this did not give the sense of line found in the kinetic signatures. The moving dot needed an afterimage trail to show its path.

To accomplish this, I experimented with several methods and programs that did not provide a high enough resolution in the finished result. Dr. Alan Dunning, one of my professors at the Alberta College of Art and Design (ACAD), suggested that I process the video with Quartz Composer. This computer program can add a delay to the video. This leaves a trail long enough to give the viewer the sense that they are watching a drawing being made of the bow hand movement. This now enabled the making of kinetic signatures to be observed concurrently with the unfolding of the music audio.

5.1 Fiddle Trails and Kinetic Portraits

While I was working with Cranitch and Fraser, I also recorded videos of their light-based performances of Mason’s Apron. It adds yet another dimension to the kinetic portrait, to see the gestures as they are being made, and concurrently listening to the music. We can compare the kinetic portraits with the videos. Video 1 corresponds with the kinetic portrait of Cranitch (Figure 6). Video 2 is the audio and visual representation of myself (Figure 7), and Video 3 is a kinetic portrait in real time of Fraser, correlating with Figure 8. The video reveals the synchronization of the gestures with the music, while the kinetic signature of the still photo shows the accumulation of the gestures made over the entire sixteen bars of music.

In the lilting sound of Cranitch’s fiddle, we can now also see the effect that the loops and curls in his bow movements have on the music. In my performance, note how the fast separate bow strokes, predominantly on the A and D strings are illustrated in the video. Fraser’s performance of the reel is the fastest, requiring him to keep his movements somewhat confined into shorter bow strokes.

The kinetic portraits, along with the processed video portraits of fiddlers are unique in the history of portraiture of musicians. Even Mili’s photographs included the face of Heifetz, his hands and his arms. In the FiddleLights kinetic portrait there is no face, no conventional way of seeing the corporeal self. It is a trace of a movement, but more than just any movement, it is the trace of a movement practiced, refined and polished over many hours, months, years, even decades, creating a dance within the body in response to the dance music performed on the fiddle.

The processed video portraits, a synthesis of light and sound, in time, are unconventional. One sees the face of Cranitch briefly (Video 1), as he introduces himself. After that, the viewer is captivated by the combination of gesture and music described by the line of moving light. It is a portrait of Cranitch, yet a distilled portrait. One is aware only of the gestures made by his hand in the act of playing Mason’s Apron and the sound of the music, the direct result of the action. In Video 2, I do not face the camera, yet the viewer is aware of my physical form, albeit fragmented. Because of this, many of the conventional ways of describing a person are accessible to the viewer before the lights are extinguished. Fraser begins playing completely in the dark, so there is no physical view of him at all. We only gain a sense of his character in how his body moves, purposefully and with energy and flair. We have no idea what he looks like, whether he is tall or short, dark or fair, ways in which we will often describe others. Rather, we are left with an ephemeral experience of expression of character through movement and sound that disappears as soon as it is realized.

6. MUSIC COMPOSITION AND LAYERED VIDEO

It was intriguing for me to view these kinetic signatures and layered videos (video compositing) that were the result of the performance of traditional fiddle tunes. However, as a composer, I wanted to explore more possibilities with the music on the violin. The idea of creating a composition where my musical choices were influenced by the graphic gestures of the bow hand movement was compelling.

My most recent video composition, Invocation (Video 4) is a new form of artistic expression for me, culminating in a complete synthesis of my music making , art making and  the proprioception of playing the violin.   

Invocation was conceived of in three parts (layers of music). For me, the process is a subtle combination of risk, control, and abandon. All the parts are freely improvised, yet within the improvisation I set certain parameters for myself. First I experiment with some gestures and musical ideas on the violin. There are many musical ideas that sound wonderful, yet do not describe very interesting gestures on the violin. Likewise, just because a particular movement of the bow hand is interesting, it does not mean that it will integrate well with the other parts in the music.

For this particular composition, I decided to tune my violin differently than I usually do. I used a common scordatura (wrong tuning) often used in Scandinavian fiddle music where the G string is tuned up a whole tone to an A, so that the open strings are A-D-A-E rather than the common G-D-A-E tuning. Although this was in part an element of control on my part, it also forced me to play differently, with diverse note combinations that I wouldn’t normally choose in my regular tuning.

While I film each layer of the video, I also record the sound with Logic Audio. This is to ensure that I can balance the audio tracks and add a reverb separately from the video. All the filming is done in the dark. I cannot see any notes, musical, or written. I feel no sense of time. The track is finished when I feel it has come to an end. Next, I listen to the first track while I record the second track and the second layer of video. This process is repeated for the third track. I cannot see the interaction of the lines of drawings between the different layers, nor can I hear, other than in real time, the culmination of the three parts of music. The audio is mixed separately from the video and then synchronized with the video. Next, I begin the process of video compositing in Final Cut which layers the three videos into a single image. The final result is only unveiled to me after I have run it through Quartz Composer which adds the after image through video processing.

The ability to extend the Fiddle Trails from a single performance resulting in a single line, to a composition in three parts involving three performances and three lines, allows for more complexity in the sound as well as the image.

7. AUDIENCE EXPERIENCE

The corporeal experience of the fiddler, in which the manipulation of the bow creates the sound on the fiddle, is something that is largely known only through a haptic sense experienced by fiddlers themselves. In my FiddleLights Project, I am sharing aspects of the haptic experience of fiddling for many to experience.

Musicians, dancers and athletes commonly use what we call ‘mental practice’ to prepare for performances. Recent studies indicate that imagination and action are truly integrated. (Doige 2007, p.207) Studies record that imagining an action and performing an action will not only make the same patterns in the brain on a transcranial magnetic stimulation (TMS) scan, but mental practice alone can produce the same physical changes in the motor system as actually doing the action physically (Doige 2007, p.201). If we can train ourselves to achieve physical responses through mental practice, I believe that by watching a visual representation of bow gestures, a non-musician’s imagination can be triggered to experience a physical response. This also incurs an element of synaesthesia, where one concurrently hears the music, sees movement and feels a haptic, physical response.

Janet Cardiff, a well-known Canadian artist created the installation Forty Part Motet by Thomas Tallis, (A reworking of “Spem in Alium” by Thomas Tallis 1573). This installation creates a similar synaesthetic response in the audience. The installation consists of forty speakers placed in a circle around a large space, often a cathedral. Each speaker is a recording of an individual chorister in the Salisbury Cathedral Choir. The choristers each had an individual microphone, and all the recordings were made simultaneously in one performance. Cardiff’s intention is that the audience experience a piece of music from the viewpoint of the singers (Cardiff 2001). She maintains that by “enabling the audience to move throughout the space it allows them to be intimately connected with the voices.” (Ibid.) In this installation, as with the FiddleLights Project, the artist is expanding the experience of non-musicians as well as musicians, by stimulating the imagination of the viewer in a new way. In the Forty Part Motet this is achieved through a privileged proximity of audience and performer, and the ability of the audience to walk around and listen to the choristers individually. In the FiddleLights Project, the simultaneity of the kinetic, visual, and aural experience provides the viewer with a heightened kinetic and sensory experience.

For choristers, experiencing the Forty Part Motet allows them to move around the choir, listening to each individual voice as it is heard both apart and simultaneously as a part of the whole, which for them is an unusual audial experience. In this way, Cardiff’s work enhances the experience of musicians as well as non-musicians. Likewise, in the FiddleLights Project, musicians who play bowed string instruments, for the first time, can see a detailed visual representation of what they do, enlivening their own haptic response to the music. Both projects, The Forty Part Motet and the FiddleLights Project are inclusive of a specialist and a non-specialist audience, weaving together the music and visual art communities, enhancing the experience of both.

8. FURTHER RESEARCH

8.1 Kinetic Portraits

I have only begun to research kinetic portraits of individual fiddlers. Attending the North Atlantic Fiddle Convention is an excellent way to find many fiddlers of varying traditions in one place, and I plan to attend the next one that will be held in Cape Breton, Canada in 2015. Because that is two years from now, I hope to travel and record some other well-known fiddlers. I would like to document some fiddlers from Canada, particularly from the Quebecois tradition, the Cape Breton tradition, and the Métis tradition. Every fall the Scandinavian Club in Calgary hosts a fiddle and dance workshop. I would like to take the opportunity this fall of recording the fiddle teachers who come from Norway or Sweden.

8.2 Ethnomusicology and Fiddle Music

I suggest that every genre of fiddle tunes will also have its own particular set of gestures. Hardanger fiddlers from Norway will have different elements in their lines than will be found in those of Irish fiddlers or Scottish fiddlers. Styles also vary widely within any national tradition, with regional diversity from north to south and east to west. Because of this, the possibilities for the exploration of gesture between individuals and within style sets are endless.

8.3 Three-Dimensional Representation

Although my documentation of bow movement through the FiddleLights Project gives a true representation of the bow hand movement of the fiddler, it neglects to incorporate the movement of the whole body of the fiddler. Rather, the fiddler must remain at the same angle to the wall, with the light pointing at 90° to the wall, which can be restrictive for the player. I am hoping to work with Dr. Sheelagh Carpendale at the University of Calgary, using the Vicon MoCap System. It is a machine that will record the movements of the body of the fiddler in three dimensions, allowing complete freedom of movement to be recorded. This will present more possibilities for recording the movements of individual fiddle players, leading to more research on the subject of regional and national style sets as well as kinetic portraits. Working with three-dimensional imaging will also help me work towards a sculptural representation of the bow hand movements.

8.4 Music Composition and Performance

I have started to work with the idea of projecting the FiddleLights compositions in a triple screen format, where the kinetic gestures are surrounding the viewer on three sides. I believe that this will further enhance the haptic response in the viewer, as well as the synaesthesia. I have tried this just once so far, where each track was projected onto a separate wall. I would like the drawing to interact between the frames, perhaps eventually even using more than three tracks at a time. In my conception of this new format for composition it will require me to be more aware of when I enter and exit the frame with the light during the video capture. It will require also more editing and choosing which frames and segments to layer, and which to leave as a single line.

As a musician and performer, the idea of FiddleLights as a live performance is intriguing. So far, I have been unable to find a way of performing in real time while recording an accurate registration of the gesture, as well as an aesthetic result.

9. CONCLUSION

The FiddleLights Project continues to unfold as an indexial, experimental and creative concept. As a research project, I wish to engage in collaboration with fiddlers from traditions of Norway, Sweden, Scotland, Ireland, England and Brittany, as well as the US and Canada to further expand my collection of kinetic portraits. The further development of my layered videos, synthesizing bow hand gestures with music composition, creates a platform for exploration and sharing of a visual, aural and haptic synaesthetic experience.

10. REFERENCES

Apel, W. ed., 1972. The Harvard Dictionary of Music. Third ed. Cambridge MA: The Belknap Press of Harvard University Press. 215.

Cardiff, Janet, 2001. Forty Part Motet by Thomas Tallis, (A reworking of “Spem in Alium” by Thomas Tallis 1573 [online] Available at: <http://www.cardiffmiller.com/artworks/inst/motet.html>. Wed. 4 Feb. 2013

Doige, N., 2007. The Brain that Changes Itself. New York: Penguin Books. 196-214

Mili, G., 1952. Jascha Heifetz Light Drawings. Life Magazine, June 2, pp.10, 11.

Mili, G., 1980. Photographs and Recollections of Gjon Mili. First Ed. Boston: New York Graphic Society.

Sereno-Janz, E., (under peer review) FiddleLights - An Artist’s Recording of Bow Movements in Fiddle Tunes. Derry: University of Ulster. http://www.elisaserenojanz.com/NAFCO_2012.html

Figures

Figure 1. Sereno-Janz, Elisa, 2011. Snow in April. Kinetic Signature from the series FiddleLights. [digital photograph]

Figure 2. Sereno-Janz, Elisa, 2012. Fairhaired Boy. Kinetic Signature from the series FiddleLights. [digital photograph]

Figure 3. Sereno-Janz, Elisa, 2012. Growlin’ Old Man and Old Woman. Kinetic Signature from the series FiddleLights. [digital photograph]

Figures 4, 5. Mili, G., 1952. Jascha Heifetz Light Drawings. Life Magazine, June 2, pp.10, 11

Figure 6. Sereno-Janz, Elisa, 2012. Matt Cranitch Performing Mason’s Apron. Kinetic Portrait from the series FiddleLights. [digital photograph]

Figure 7. Sereno-Janz, Elisa, 2012. Elisa Sereno-Janz Performing Mason’s Apron. Kinetic Portrait from the series FiddleLights. [digital photograph]

Figure 8. Sereno-Janz, Elisa, 2012. Alasdair Fraser Performing Mason’s Apron. Kinetic Portrait from the series FiddleLights. [digital photograph]

Videos

Video 1. Alasdair Fraser Performing Mason’s Apron. from the series FiddleLights. 2012. [video] Elisa Sereno-Janz. Canada: Blue Arch Music. http://www.elisaserenojanz.com/EVA_2013.html

Video 2. Elisa Sereno-Janz Performing Mason’s Apron. from the series FiddleLights. 2012. [video] Elisa Sereno-Janz. Canada: Blue Arch Music. http://www.elisaserenojanz.com/EVA_2013.html

Video 3. Matt Cranitch Performing Mason’s Apron. Kinetic Portrait from the series FiddleLights. 2012. [video] Elisa Sereno-Janz. Canada: Blue Arch Music. http://www.elisaserenojanz.com/EVA_2013.html

Video 4. Invocation. From the series FiddleLights. 2013. [video] Elisa Sereno-Janz. Canada: Blue Arch Music. http://www.elisaserenojanz.com/EVA_2013.html

Many thanks to Matt Cranitch and Alasdair Fraser for participating in the FiddleLights Project.