Generating Real-Time Visual Meaning for Live Indian Drumming

Generating Real-Time Visual Meaning for Live Indian Drumming

Length: 2126 words (6.1 double-spaced pages)

Rating: Excellent

Open Document

Essay Preview

More ↓
Generating Real-Time Visual Meaning for Live Indian Drumming

Abstract: In this essay, a system developed to generate visual meaning for live performed music is described. Specifically, this system is calibrated to respond to North Indian classical and folk drumming tradition, using custom designed digital musical interfaces, such as the Electronic Tabla and Electronic Dholak.

A drum, when struck, does not generate its sound as a record of the force applied, but as an artifact of a physical response to that strike within the artistically controlled conditions of its material state. With the design of the North Indian drum controllers, we developed a physical model for digital audio synthesis to recreate the aural qualities of the drums’ response under the control of the player. From there, we abstracted that concept to develop a dynamically responsive model for real-time rhythmic visual synthesis.

In creating a visual experience for North Indian classical music, we sought to create a dynamic visual accompaniment with an appropriate ambience for the patterns and complexities of North Indian drumming. The design process was shaped by the need to react to a series of signals that would be received from the musician through the digitized musical interfaces, while giving a visual performer the ability to modify and shape those reactions over the course of the performance. In contrast to using a prerendered or abstract visualization, we aim to create an audiovisual composition which is both aesthetically compelling and responsive to the conditions of live performance, in addition to providing a meaningful visual context for what the performer is playing.

In this essay, we will describe:

- Veldt: a custom built application for visual expression of musical performance.

- Digital Indian Drums: the Electronic Tabla and Electronic Dholak which digitize
gestural information of a live performer.

- Rhythmic visualization of our system used in live performances.


Veldt is an application which was designed from the ground up for the purpose of visual expression and performance. It receives MIDI (Music Instrument Digital Interface) and OSC (Open SoundControl)[1] messages from digital musical interfaces and maps them to a system of reactive events in order to generate live visuals, which are rendered real-time using the OpenGL[2] graphics language. Mappings are flexible: sets of mappings may be arranged and modified during the design and rehearsal process, and triggered by control events during different movements of a performance, and arbitrary text, images, video, and geometric models may be used as source material.

How to Cite this Page

MLA Citation:
"Generating Real-Time Visual Meaning for Live Indian Drumming." 15 Aug 2018

Need Writing Help?

Get feedback on grammar, clarity, concision and logic instantly.

Check your paper »

Essay on The Subcortical and Cortical Visual Pathways Analysis

- In order to be selective in attending to a complex environment infants need to be capable of engaging and disengaging in attention. When newborns are awake and alert their attention is not random it is organised and selective (Ruff & Rothbart, 2001). Attention engagement is an effortful engagement with stimuli; it is also referred to as sustained attention (Stechlar & Latz, 1966). The ability to detect changes in stimuli over sustained periods requires effort (Ruff & Rothbart, 2001). Attention disengagement refers to when attention to a stimulus is terminated (Colombo, 2001)....   [tags: attention, visual stimuli, visual system]

Research Papers
1862 words (5.3 pages)

The Real Meaning of Brain Cancer Essay

- Around New Year's Eve, HHH FFF’s husband noticed changes in his wife's personality. Soon after, she was diagnosed with primary central nervous system lymphoma, which arises in lymphocytes, a type of white blood cell. A few weeks before her 29th birthday, CC had a seizure and was later diagnosed with an oligodendroglioma, a slow-growing, low-grade brain tumor. One Valentine's Day, BBB collapsed with a seizure. Emergency room doctors diagnosed him with a high-grade glioblastoma, the most aggressive and deadly form of brain tumor....   [tags: brain tumor, radiation, environmental factors]

Research Papers
1006 words (2.9 pages)

Essay on Commercial Real Estate

- Commercial real estate is any property other than a single family home or a residential lot in a neighborhood. The commercial real estate market is handled very similarly industrial and residential real estate markets. The commercial real estate market goes through ups and downs just like any other investment market however investments into commercial real estate can be a very wise decision. Commercial real estate is similar to industrial and residential real estate in that when you buy it you are buying a building or land....   [tags: Real Estate]

Research Papers
974 words (2.8 pages)

Essay about The Visual Rhetoric of Traumatic Histories

- The Visual Rhetoric of Traumatic Histories Among the problematics that guide my understanding of the possibility of visual rhetorics are three. Each might be considered to exists within/bring together the nexus of history, images, and power. This nexus helps to form a framework for an economy of verbal and visual images that, in turn, might become the fabric of a visual rhetorics. The first is what I want to call the "enigma of unrepresentability." The second is that images become especially important for us when they can be read as "self-reflexive." Finally, the third, is the "ideological privileging" of the visual that renders its apparatus, quite literally, hard to "see." Let me briefly...   [tags: Visual Rhetoric Essays]

Research Papers
1111 words (3.2 pages)

Visual Ethnographic Research Study Essay

- Throughout the young lives of children, there are many methods that can be used to interpret and understand their respective childhoods. We can appreciate that each child has a unique way of interpreting their childhood and through different methods, they have a distinctive way of how each of them perceive their childhood to be. One excellent methodology that children can show what they recognize about their childhood to be is with the concept of photo-voice. Photo-voice grants each child the capability of capturing visual moments through the use of photographs....   [tags: Visual Ethnography]

Research Papers
1515 words (4.3 pages)

The Real Meaning of Marriage Essay

- ... Okeke sent back a letter telling Nnaemeka how disrespectful it was to send him a picture of himself and Nene together. He cut the picture in two sending back the part with Nene on it. The relationship between Okeke and his son was nonexistent for eight years. Okeke did not change his mind about his son’s marriage. He still found it disgraceful, though others were beginning to accept non-arranged marriages. One day Nene wrote Okeke saying that his son and her now had two sons’ of their own....   [tags: arranged, parents, children, love]

Research Papers
702 words (2 pages)

Essay on What is Visual Crowding?

- The Fly in the ointment was the visual crowding of the Visual System Actually, many people don’t understand their visual system and don’t know how it functions. Most people believe if they can see the world, object clearly that means their visual system is perfect. However, the Fly in the ointment was the visual crowding of the Visual System. What is visual crowding. Why it is important in our visual system. And how it functioned. Can it be avoided. We can get the answer in below. Visual crowding According to D....   [tags: visual discrimination, flankers]

Research Papers
960 words (2.7 pages)

Visual Culture vs Television Essay

- Visual culture has shifted our traditional, ritual, ceremony with television. In this generation we are heading towards a more visual culture. Technology such as television has held to countless antisocial behavior in young developing minds leading to a lot of apathy in our traditional American culture. With an increasing demand in technology, television viewing habits is starting to materialize as a problem in the American society. This habit has negative effect on our perceptions, behaviors and how we interact with each other....   [tags: Visual Culture Decline]

Research Papers
2621 words (7.5 pages)

Essay about Visual Media, Allegorical Consciousness, and Postmodern Culture

- Visual Media, Allegorical Consciousness, and Postmodern Culture I think many of us would agree that we are living in an era of transition: generally, from one phase of modernity to another; more specifically, it is harder to say. Let's ask ourselves for a moment how this sense of change might guide the rhetorical study of visual media. Of the many possible answers to this question, there are two I want to put on the table. The first consideration is that the study of visual media is likely to be occurring at all, or in a particular form, because our society now is moving beyond those media to other communication technologies....   [tags: Visual Media Technology Essays]

Research Papers
1653 words (4.7 pages)

Essay about Generating Response in film Bound

- Generating Response in film Bound The film I have chosen is a crime thriller called 'Bound'. Mafia moll, Violet, finds herself attracted to a female ex-con, Corky. Together they attempt to steal $2 million from Violet's boyfriend, Ceasar. It brings about the question of who can trust who throughout the film. Although it is a crime thriller it also has gangster undertones. I will discuss cause and effect, other films of the same genre, the characters, the settings, if there is equilibrium and any possible theorists....   [tags: Papers]

Research Papers
1013 words (2.9 pages)

Related Searches

We display a real-time composition of these media sources over geometric elements which are generated and modified according to the parameters of the current mapping. In addition to control events received from the performer, a physical simulation environment is incorporated to allow for a variety of secondary motion effects. This visually (and contextually) rich combination of source material over physically reactive structural elements allows for a response that is dynamically generated and artistically controlled. While the parameters that govern the overall response of the system to the drum controllers may be modified through cues such as MIDI program change messages, Veldt allows an additional visual performer to control the finer aspects of the performance.

Digital Indian Drums

One challenge of this project was retrieving musical data from a performing musician quick enough to make a visual response in real time. There are many methods of generating MIDI or OSC messages from audio data by extracting features from the frequency domain and time domain. These methods proved to be too slow, and inaccurate for real time applications as the number of parameters increased. Hence, we choose to build instruments which could perceive the gestures of the performers, using sensors to capture data in real time, and convert them directly to MIDI or OSC messages. The instruments we will discuss are the Electronic Tabla and Electronic Dholak.

The Electronic Tabla

Tabla are a pair of hand drums traditionally used to accompany North Indian vocal and instrumental music. The silver, larger drum is known as the Bayan. The smaller wooden drum is known as the Dahina. We model the Electronic Tabla (ETabla) on the hand positioning and movements of the many elaborate stroke techniques. We use a set of force sensing resistors to measure the pressure and location when fingers strike the drums. All data is converted to MIDI signals and sent out via a MIDI outport, triggering both sound and graphical feedback (using veldt).[3] For our visual response system, we focused on two strokes performed with the left hand, known as Ka and Ga,[4] to make it easy for the audience to grasp how graphics were being triggered. The Ka stroke is triggered by striking sensors at the upper edge of the EBayan. The Ga stroke is triggered by striking the center of drum, while the pressure of the left wrist on the drum determines its pitch.

The Electronic Dholak

The Dholak is a barrel shaped hand drum originating in Northern India, with two membranes on either side of the barrel.[5] Two musicians play the Dholak. The first musician strikes the two membranes with their left and right hands. The second musician sits on the other side of the drum, striking the barrel with a hard object, such as a spoon or stick, giving rhythmic hits similar to a woodblock sound.[6] We created the Electronic Dholak (EDholak) inspired by the collaborative nature of this traditional drum. Two musicians play the EDholak, the first striking both heads of the double-sided drum, and the second keeping time with a “Digital Spoon” and manipulating the sounds of the first player with custom built controls on the barrel of the drum and in software. We further explore multiplayer controllers by networking four drummers playing two EDholaks at the two geographically diverse sites. The EDholak captures finger-striking data with piezo electric sensors. The “Digital Spoon” has a piezo sensor attached to it, as well as a force-sensing resistor on the EDholak itself, to capture how hard (pressure) and where (location) the spoon strikes. All this music gestural data is converted to MIDI and used to trigger sound and graphical response (using veldt). [7]

Rhythmic Visualization

To further integrate the digital Indian drums into the performance environment, we have integrated the drum controllers with the veldt graphics system to generate live visual projections during set. We have developed and tested customized methods of rhythmic visualization in two performances, and will describe in further detail.

The ETabla Concert

For our first concert performance in May 2002, we focused on a system of geometric forms and fluid motion, driven by the percussive energy of the ETabla. The visualization we developed was based on a particle model where strikes made by the player appeared as patterns composed of small geometric elements which would appear, and then merge into a background of fluid motion. As the drum player made Ka and Ga strikes on the EBayan controller, new strike patterns were generated according to their type (Ka or Ga), and the intensity and pitch of those strikes as recorded by the digital sensors. Additional control messages were sent by the visual performer to modify the mapping of ETabla signals and other ambient qualities, such as lighting and color, as the performance progressed.

Once created, the motion of these particles was governed by a dynamic vector field, governed by a distribution of fluid cells which determined how forces are exerted in their vicinity, in response to the number, distribution, and motion of elements in and out of their domain. The energy introduced to the system by ETabla strikes excited secondary behaviors under the control of the visual controller. By properly modifying the characteristics of particles and cell response behaviors, we could evoke an impression of different actions or environments: e.g. fanning a flame, striking the surface of water, or blowing leaves. In each of the configurations the Ka and Ga strikes had different roles for the simulation. For example, in the fire style, Ka strokes produced small rings of sparks, while Ga strokes generated broad flames from the bottom. Variation in the pressure of the player’s hand on the surface of the controller, which altered the pitch of Ga strikes, would act as a bellows, stirring up the scene. Video links to this performance are located at the bottom of this article.

The Gigapop Ritual

For the Gigapop performance, we received EDholak drum messages from performers playing in two separate (Montreal, QC and Princeton, NJ) locations over Gigabit Ethernet and used them to generate visuals in the theater at the Montreal location. Our intent was to create an environment in which the actions of both drummers were visible and distinguishable.

Our solution for this concert was to allow two players to interact through a sculptural metaphor. Using a dynamic geometry representation to allow modifications to the structures in real-time, the two performers interacted through a series of operations to create a visual artifact of their drum patterns. Their strikes were dynamically mapped to a series of geometric operations that generated, deleted, deformed, or detached elements of the structure and generated unique artifacts from the rhythms they played. In Figures 5, 6, and 7 we see structures which have evolved under different mapping rules. In Figure 6, for example, we chose a mapping which created smaller, separate elements rather than building from a central structure as in Fig. 5. In Figure 7, we chose rules which resulted in a solid, sheet-like structure. To add a convincing physical response to the addition and alteration of new elements, we used a mass-spring model to apply and distribute forces as the structures developed. In these figures, the actions of the drummer bend and distort the figure, while secondary forces try to smooth and straighten the figure, like a plucked string which vibrates to rest.

To represent the shared performance space, we experimented with several different forms of visual “interaction” between the signals received from two performance spaces. To begin, we assigned the two drummers into separate visual spaces: one drum would excite patterns as in the ETabla performance, while the second was assigned to build structures. We then assigned both performers as builders, so that their rhythms would build upon one another. In the next, one performer’s strikes would build while the strikes of the second would rattle or delete those structures.

We are currently developing a system to visually represent the cyclic nature of North Indian Rhythm (known as Theka). We are also building a new musical controller based on the Sitar to collect musical data about Raga, adding a new dimension to our visual system to represent pitch and modal melodies.

While developing veldt as a visualization system for the ETabla and Gigapop concerts, we began with the goal of enriching a musical experience for the audience. Our system allowed for the further abstraction of a digital musical interface to a visual context, and required that we consider our visualization process in the role of an instrument as well as accompaniment. This incorporation of visual feedback in concert performance can provide the audience with another means of conceptualizing their experience of it, and an additional means of becoming actively engaged with the artist’s performance. Beyond the context of concert performance, our system can also be used in a pedagogical context, allowing a student to receive a variety of feedback and coaching while learning to play these instruments. In addition, we have seen that performers sometimes choose to change from using the device as an audio controller to treating it as a video controller. To that effect, we might also explore this work in the context of game environments, where our controller is an interface to a complex inner system which provides audiovisual feedback.


[1] Wright, M. and A. Freed. “Open SoundControl: A New Protocol for Communicating with Sound Synthesizers,” Proceeding of the International Computer Music Conference. 1997.

[2] Segal, M and Akeley, K, The OpenGL TM Graphics System: A Specification, Version 1.0, Silicon Graphics, April 30, 1993.

[3] Kapur, A., Essl, G., Davidson, P. & Cook, P. R. (2002) The Electronic Tabla Controller. Journal of New Music Research. 32(4) 2003, pp. 351-360.

[4] Courtney, D. R. (1995) Fundamentals of Tabla: Complete Reference for Tabla. vol. 1,. Houston TX: Sur Sangeet Services, 1-15.

[5] Kothari, K.S. Indian Folk Musical Instruments, pp. 43-44, Sangeet Natak Akademi, New Delhi, (1968).

[6] "Dholak", RagaNet, Batish Institute, Available at: http://www. /RagaNet/ Issues/7/dholak.html

[7] Kapur, A., G. Wang, P. Davidson, P. R. Cook, D. Trueman, T. H. Park, M. Bhargava. "The Gigapop Ritual: A Live Networked Piece for 2 Electronic Dholaks, Digital Spoon, DigitalDoo, 6-string Electric Violin, Rbow, Sitar, Tabla, and Bass Guitar." Performed Live at Princeton University and McGill University at the International Conference for New Instruments for Musical Expression (NIME 2003). May 23, 2003. Available at: http://www.cs. sound/publications/gigapop_nime_2003.pdf
Return to