MATTHEW GOUGH

Notation Reloaded: eXtensible Dance Scripting Notation

ABSTRACT

This paper presents a new form of dance notation that has a hand written and machine readable format: eXtensible Dance Scripting Notation (xdsn). The notation system is designed to be quick to write, simple to learn and able to handle complex, emergent and deconstructivist dance works. The machine readable format of xdsn is platform independent, searchable, and well formed to encourage the development of dance notation software. An overview of the system is given illustrating its context in current contemporary dance ontology and as a mechanism for notation driven avatar animation. eXtensible Dance Scripting Notation was developed by the author of this paper in direct response to a proposal from Michael Klein for alternative methods of dance scripting. The system (version 0.3), is currently in localised use and will be developed at the School of Computing Sciences, University of East Anglia, Norwich. This work will be undertaken as a Phd Research project under the leadership of Professor John Glauert.

I - BACKGROUND

Michael Klein [1] proposed that multiple choreographers’ timelines, dynamic lines, space-diagrams and scribbles could be examined to develop a common scripting language. This choreographic scripting language could then be used to develop and plan dance works. Although he advocated a paper based, left to right reading format his intention was avoid a linear notation system that would potentially prevent the scripting of complex and emergent works. Klein asserts that:

Visualisation is a broader problem these days as now [sic] longer everything is logically determinable. In science problems of complexity, quantum physics, chaos theory, etc. give us 'un-visualsable'[sic] ideas to deal with. How can even a concept of relative time be meaningfully represented. It seems like a certain 'class of visualisation' stopped with relativity theory. Nevertheless this hasn't stopped us to notate along [sic], scribble on drawing pads, making lines to imagine a performer's path, dots to make clear that there is a stress in movement and 'really, really, really black dots' to 'double stress' an event. Klein [1]

Hagendoorn supports this view claiming, '[t]he more complex a system becomes, the more information is needed to describe it, in terms of a choreography: the more instructions one has to give to the dancers' [2]. However, the perception of complexity is dependant on the observer, for whilst it is possible to illustrate complex systems (by grouping and linking) the act itself negates their complexity. Conversely, ostensibly simple looking systems can potentially be hugely complex. It can be argued therefore, when notating complex dance works the ability to define events is more important than the ability to illustrate their complex links.

William Forsythe's deconstructivist works clearly illustrate the challenges to traditional notation systems when documenting complex or emergent choreography. When using Laban Notation to document William Forsythe's Artifact II (1984) Sandra Aberkaln used colour to define structured improvisations (tasks), set choreography and choreographic directives open to dancer choice [3]. Although Aberkaln's solution demonstrates that adaptation of existing systems is possible, it only adds to the difficulty of mastering such systems and reduces the speed of notation. Any new system of notation (in order to be useful) must be able to handle such functions natively and quickly to aid the planning and development of new choreographies.

Royce Neagle's review of dance notation software [4] highlights the need for any new notation to be designed with software development and avatar animation in mind. Aside from the difficulties of converting abstract symbol (glyph) based notations into computer based languages, Neagle also reveals the inability of current machine readable notation scores to be searched for specific content. For the purposes of dance research and analysis, the ability to search a database of machine readable notation would be a huge advantage when examining an artist's body of work. Searchable and well formed data would also ease integration and communication with existing software, facilitate new software development and provide clear mapping points for avatar animation systems.

II - CONCEPT & GOALS

To facilitate development a clear set of concepts and goals were laid out. The notation should:

  • be simple to learn and quick to write.
  • be dual format (hand written and machine readable).
  • be able to handle complex, emergent deconstructivist works.
  • be capable of scripting and planning dance works in addition to documentation.
  • read from left to right (allowing alignment with music scores and timeline based software).
  • be capable of preserving the choreographer's unique approach.
  • be extendable to allow notation of new or work specific concepts or terms.
  • allow the inclusion of drawings, scribbles or any abstract symbols whilst retaining the notations machine readable capabilities.
  • allow referencing of supporting materials such as video, photographs, sound, text etc.
  • have a well formed machine readable syntax to facilitate data mining [5], software development and avatar animation.
  • allow the creation of machine readable scores and notation software with minimal resources.
  • be platform and software independent.
  • allow human comprehension of the machine readable format.

The notation should inform, not instruct.

III - THE SCORE

The design requirements for the notation score called for a similar approach to western music notation. Benesh Movement Notation [6] and Sutton Dance Writing [7] both use scores based on the five staff music stave. Although this arrangement is useful when presenting the notation alongside music scores, this advantage is gradually being negated by the use of electronically composed music and sound scapes which have no written score. The limitations of western music notation when notating emergent forms can be inferred by the various adaptations of the system made by the 20th century avant garde composers such as John Cage, Steve Reich and Karlheinz Stockhausen [8].
An analysis of choreographic scribbles alongside 20th century music scores revealed several parallels in the use of abstract symbols and textual instructions. The work of Stockhausen was particularly informative in this context, especially the score for Kontakte für elektronische Klänge, Klavier und Schlagzeug (1960). Dack claims that 'The "realisation" score [sic] of Kontakte is, in effect, a published version of his principal sketches and work-plans.' [9] Dack's assertion is due in part to the 'boxed in' scribbles used to represents sound events, unrefined and ambiguous they appear to be 'workings out' rather than finished notation. The final design for the new score draws on Stockhausen's work for its form and the visual design theory of Edward R. Tufte as a guide to arranging the content [10][11][12].

The basic score for the notation system consists of a single stave, single staff and bar lines:

The formal, yet open design of the score allows maximum flexibility of use whilst remaining simple to implement and interpret.
The score has several guidelines:

  • use the single staff or multiple staves to separate different elements (e.g. dynamics, comments, floor plan).
  • group events that occur in a close time frame.
  • nest events that are related and / or occur together.
  • define what element/s are most important and then specify secondary, tertiary elements and so on.
  • script as you work, let the score reflect your working process
  • be concise.

As long as the notation occurs within the basic score it is possible to make a machine readable version of the scripting.

Notes on the score:

The single staff can occur at any point on the stave, the default position is centre stave:

When multiple staves occur in the same timeline bar lines run across the staves and link them together:

The end of a stave is signified by a bar without a staff. This is the 'at rest' bar and may be used at throughout the score. When making detailed notations to describe a specific movement event, the 'at rest' bar should be used to define the start and end positions. This is particularly relevant when using the notation to animate avatars as many animation packages use a standing avatar as default. This notation does not have a default body position.

To indicate a stave has finished (whilst other staves remain) or the absence of events in a bar, the staff is continued but the stave is not drawn:

Comments on the movement, performance based tasks, sound effects, lighting effects, props, floor plans (etc) can be placed on the score. The length of the bars can be specified in terms of time (hh:mm:ss), music time signatures, or left open to interpretation. These markings should be placed immediately after the bar line at the top of the stave.

The single staff can also be used to guide the creation of one and three dimensional scores [13] that embody post structuralist principles. With the range of scripting approaches available, it is possible to document, plan or script dance works whilst retaining the 'essence' of a work and documenting the choreographer's working process.

IV - THE NOTATION

The xdsn system draws on the strengths of existing notation systems [14] utilising word, abstract symbol and numerical notation markings. Because xdsn aims to imply rather than instruct, movement can be separated from body location and limbs can be notated without specifying which side (e.g. leg). If very concise notation is required combinations of the symbols can be used to define what, why, where and how an event happens. For example, first arabesque 'could' be notated as:

In order to ease the transformation of the hand written notation into the machine readable format, the hand written notation is grouped into defined elements and values. Only the values are used in the hand written score allowing the elements to be derived from them and added during conversion into the machine readable format. An overview of the elements and values is given below.

IMPULSE

The impulse element is based on the work of Longstaff [15][16], Nakata [17], Bernstein [18][19], Kandinsky [20][21] and Jeannerod [22].

To notate the orientation of movement in three dimensional (Euclidean) space, Euler angles (Pitch, Roll, Yaw), with an axis rotation (_) are used. The Euler angles and rotation are given as a set of four numerical values:

  • Pitch (y axis)
  • Yaw (z axis)
  • Roll (x axis)
  • Axis-rotation (_, around the z-axis)

The values are written as four sets of three digit numbers from 000 to 360 (a full circle being 360 degrees) in negative and positive numbers. The movement direction is clockwise and upward for positive numbers, anti clockwise and downward for negative numbers. To understand how this works try the following exercise:

Stand up with you palm facing your thigh (this is the start position for each set of values), the right wrist is the body location leading the movement.

090 000 000 000 - the wrist impulse is 90° (degrees) pitch, the arm rises to elbow level in front of you. you palm is facing the left.

090 090 000 000 - 90° pitch 90° yaw, the arm comes to elbow level and out to the side in one continuous motion. you palm is facing the front.

090 090 090 000 - 90° pitch 90° yaw 90° rotation. same as above except now the palm is facing the ceiling.

090 045 090 045 - 90° pitch 45° yaw 90° rotation, 45° axis-rotation. same as above, the axis-rotation (_) means that this impulse starts at 45° angle from the start point (z).

Due to the nature of the impulse element it is not possible to derive the location of other body parts form the notation (unless given) nor understand 'where you are going' without following the full passage. Hutchinson Guest describes the situation as being on a boat at sea '... tacking this way and that, ... But to avoid getting lost one has to determine the boat's position, to relate to points on the globe, to know one's latitude and longitude. At some point movement also needs to be similarly defined' [23]. Although such definitions are important for the repeatable reconstruction of set works, many modern works are not as clearly defined. Focus has shifted away from where the dancer moves through the space, to how they move through their body in a defined space. The impulse element reflects this change. To speed up the writing process up to two zeroes can be stripped from the values as needed.

Because the four numerical values of the impulse element relate to Euler angles and rotation matrices they can be converted to Quaternions, a standard method for describing motion in computer animation and motion capture data. One advantage of this method is that little interpolation is needed when using the notation for computer animation or converting motion capture data into human readable notation scores.

PATH

The path element is used to describe the path a movement takes (impulse defines direction). Although the path element has a defined set of values, these can be added to if required by the choreographer. When adding custom values they should be prefixed with the word 'path' in the first instance for reading clarity. The defined values for the path element are:

line, angle, rectangle, curve, circle, spiral, twist, zigzag, release, arbitrary, translation.

The path element can be used alone or in conjunction with other elements (090 000 000 000 curve).

MOVE

The move element is used to describe global movements that would otherwise involve extraneous notation. The values for the move element are:

rise, hop, leap, jump, step, walk, run, slide

the attribute is applied to the leading rather than the working leg:

  • hop with right knee in air: k_r hop
  • leap left to right leg: ll_r leap
  • kneel left knee on floor: k_l kneel

Remember that: hop= no change, leap= one foot to other foot, jump= both feet together. The move element can be used alone or in conjunction with other elements (090 000 000 000 curve slide).

Effort

The effort element is drawn directly from the work of Laban [24] and is used to describe movement dynamics. The values for the effort element have been design to work with the EMOTE model for effort and shape [25] and the work of Liwei Zhao [26]. The values are paired and only one value from each pair can be given.

  • direct / indirect
  • strong / light
  • sudden / sustained
  • bound / free

The values can also be given a numerical value from minus ten to plus ten for more accurate description and EMOTE mapping.

The effort element can be used alone or in conjunction with other elements (090 000 000 000 curve slide sudden_10 free_-5).

BODY LOCATIONS

Because the elements relating to movement can be placed at any point on the score, a list of body parts must be specified. The body location elements are given below:

<w> wrist (undefined)
<w_l> wrist left
<w_r> wrist right
<e> elbow (undefined)
<e_l> elbow left
<e_r> elbow right
<s> shoulder (undefined)
<s_l> shoulder left
<s_r> shoulder right

The same notation model continues for other body parts: <hip> hip, <k> knee, <a> ankle, <ua> upper arm, <la> lower arm, <ul> upper leg, <ll> lower leg, <hand> hand, <foot> foot. Fingers and toes are notated from 1 to 5 (1 being the big toe or thumb) given the prefix 'm' (metatarsal / metacarpal) and used as a prefix of the hand or foot element, for example:

<m5_foot_l> little toe left foot

Each bone in the finger or toe can also be defined. The head and spine are identified and abbreviated as follows:

<head> head
<vc> vertebral column (whole spine)
<cerv> cervical spine
<thor> thorasic spine
<lumb> lumber spine
<sacr> sacral spine

The short form should be used (with or without the angle brackets >) and not the longer description. Although body locations are set the system allows any point to be defined by the choreographer according to need. This should be noted on the score and the same form used consistently, e.g. '<liver>=body location' then just the short form 'liver' can be used throughout the score.

PRESSURE POINT (WEIGHT)

In everyday use the term weight is often synonymous with the term mass. However, mass is different. What you weigh has to do with gravity; your mass is measured without gravity. To define mass more specifically, it is necessary to use Newton's second law of motion: F = Ma.

  • Weight: the force that gravity exerts on a body of mass
  • Mass: the amount of matter an object contains
  • Centre of gravity: the point through which the force of gravity seems
    to act (the balance point) in uniform gravity - it is equal to the centre of mass

What we perceive when we 'feel weight' in a movement is the change of location at which we can 'feel' the force of gravity act. We cannot transfer this force as it acts on all parts of our body. Because we are unable to perceive our centre of gravity (as it can lie outside the body) it is unsuitable for notation and this is the reason why we have the term 'weight'. However, rather than 'weight' the element used is pressure point <pp>. This re labeling means the element can be used to identify where the 'weight' is, and for describing any point of contact where pressure is given or received (such as lifting or supporting). Using a combination of impulse, path , body location and pressure point elements it should be possible to notate the complex weight bearing and dynamic shifting found in contact improvisation.

The pressure point element, as with the other elements, can occur at multiple body locations at the same time.

NOTES ON THE NOTATION:

The impulse element is not confined by the angles or degrees of freedom available in the human body (unlike human movement simulation software). During real human motion we adapt the position of related joints or whole body alignment to allow greater rotational freedom. This can be observed clearly in salsa style dancing and contact improvisation. Limited degrees of freedom should be used to describe how the body behaves not how it moves, it is the limiting of degrees of freedom that defines the style of movement not how it is generated. The unusual implementation of the impulse element is intended to be used with a novel, epikinetic framework for avatar animation that synthesises rather than simulates human movement. Instead of replicating 'how' the body moves (degrees of freedom or visual impressions such as motion texture), the framework models the mechanisms that 'drive' movement using algorithms to simulate and synthesize motion impulse. Using this combined approach (the framework and notation) it should be possible to create real time, autonomous avatar animation with a high level of movement fidelity using minimal resources. Epikinetic modelling is based on Hypokinetic and Hyperkinetic Kinesthesia with particular reference to Sydenham's Chorea.