Temporal Modelling: Summary of Research and Project Outline
Johanna Drucker and Bethany Nowviskie
October 2001, Intel Presentation
Abstract:The practical goal of this project is to create a visual scheme and interactive tool set for the representation of temporal relations in humanities-based or qualitative research, with particular emphasis on the subjective experience of temporality. Our rhetorical goal is to bring visualization and interface design into the early content modeling phase of humanities computing projects, with an eye toward both enriching the design aspects of established scholarly endeavors and making humanities computing attractive and comprehensible to students and faculty without specialized technical skills.
Providing a foundation for the project involved several aspects:
1)Drawing on research across a range of disciplines, we focused on:
a)classifying existing concepts of time and temporality,
b)elaborating a stable nomenclature culled from the existing literature, and
c)establishing the requirements for a graphical system for the visualization of temporal relations.
Our approach emphasizes the need to create a structural and notational scheme that accommodates subjective perceptions and experiences of time as well as multiple, even contradictory, temporalities within the document-dependent domains of humanities and social sciences research.
The goal of our first phase of research was to lay out these conceptual and representational issues in such a way that they could expand our understanding of time and temporal relations beyond the usual linear, time-line approach.
2)The application of this research to the development of a demonstration gave rise to a distinction between approaches to visualization, fundamentally different in aim and functionality, though capable of sharing the same visualization schemes. These two approaches we named the “composition” or “play” space and the “display” space. Our conviction was that the capability to use a composition space to conceptualize temporal relations in humanities research in advance of the content modeling used for XML mark-up would enrich the possibilities for interpretation. If we can create a space in which visualization is a primary epistemological tool (both in sequence of use and in importance), then our composition space will permit design-conscious thinking about temporal relations, which can then lead to generating a content model that better lends itself to display. The visualization scheme (types of visual metaphors and templates) will be the same in the composition and display modes. Our posited use of a data visualization tool for experimenting with and generating a content model in the absence of marked data – in fact as an aid to marking data – is a departure from established humanities computing practices, under which visualization and interface design are generally marginalized as last steps.
3)The next goal was to distill these concepts into schemata that meet requirements for translation into parameterized diagrams within a digital composition space, as per the preliminary demonstration. The result of the research seminar and technical workshop held in June at University of Virginia was the design of a more refined (more carefully specified and graphically effective) visual composition space and development of a set of fundamental "primitives" that meet technical requirements for the underlying architecture of that composition space. These structures and requirements will be formalized in an XML Schema document (or a similarly unambiguous form). This family of temporal objects and relations will be sufficiently rich and flexible to accomodate radically different conceptual models of time, events, and temporal relations.
4)Future work involves research in the area of smart diagrams, temporal linguistic analysis, and greater refinement of the visual schemata. A final phase will link the composition space/schemata with display capabilities that engage existing databases for query, manipulation, and analysis.
Basic argument and issues
The visual conventions used for schematically representing elements (items, objects, photographs, events, etc.) in temporal relation generally share certain common assumptions: time is unidirectional, neutral, and homogenous. The design proposed challenges all three of these assumptions and creates a visual scheme in which alternative approaches may be represented for purposes of basic research and visual display. We don’t disagree with these assumptions as ways of interpreting or representing interpretations of time, but we disagree with the idea that they are the only way to represent temporal relations or that they have any greater validity or objectivity than our system for representing subjective temporality. Humanities scholars deal with many variables in the temporal relations within accounts and among documents and have a need for a less rigidly empirical and more flexible system of representing these relations.
We will discuss each of these assumptions in turn.
The first assumption is often referred to as the time arrow, a familiar notion that time has a unidirectional, irreversible flow in which the past is unchangeable, the future unknowable, and the present elusive but all-present. The very concept of the time-arrow is itself premised on a second assumption – that time is unified, a pre-existing whole that simply is. According to this notion, events are contained in time in much the same manner as objects are contained in space. Finally, the conventional conception of time used for measuring phenomena in objective, empirically-based time measures, is that time has a single metric. Though degrees of granularity may differ in their usefulness for assessing various phenomena or different parts of complex events, the underlying assumption is that there is a single unified temporality within which such granular differences can be reconciled (as different scales of the same measure).
We suggest that all three of these are assumptions, and that the vocabulary of interpretive possibilities may be extended through addition of alternatives that modify each of these assumptions.
The idea that time is an arrow, and that events follow each other in fixed sequence such that past events are unchanged by those that succeed is the first assumption that is contradicted by the way temporal relations are experienced within individual human perception and within the broader context of documents that form the basis of humanities research. The frames of interpretation that are common practices within history or narrative, for instance, show that a map of past events may change dramatically according to new information, or occurrences, that do not merely recast our interpretation of what occurred but our conviction about what actually occurred. (The development of theories of the geological history of the earth in the 19th century offer a very clear example of such a transformation, since biblical measures of past events, taken quite seriously as metrics by historians as well as theologians, were subject to radical reconfiguration in order to conform to empirical evidence offered in the physical record.) Similarly, anticipation of future events and the degree to which this anticipation shapes the present, a major aspect of narrative practice in prose and drama, is difficult to chart on a standard time-line.
The idea that time is neutral and that it provides a bland, container-like setting for events outside of their individual existence is countermanded by the realization that temporality is constructed precisely by the relations among elements. Tensions and pressures exerted by such events inflect all temporality with subjective qualities. The idea of “the distant future” or “someday” or “after my lover comes back” – all quite logically compatible with subjective experience of temporality, are not able to be absorbed into a neutral concept of time with a stable metric that exists independent of events. The relations among events separated by time, rather than an experience of time itself, is the focus of such experience.
This leads directly to the final alternative to the idea of time as a single, linear, scale of temporality that is homogenous and consistent. In much humanities-based research and much lived human experience, subjective notions of time are distinctly different depending upon circumstances and emotional or other investments. Not only is the perception of granularity different among various areas of particular events or phenomena, but the relation of parts to each other, parts to a whole, or metric scales to each other are not always able to be unified within a single homogenous frame. (Clearly the appropriate granularity for a historian documenting the burning of Atlanta during the Civil War, for instance, is quite different from that used in the narrative of Gone with the Wind in either its film or book versions). Breaks, ruptures, inequities and discrepancies in pacing – these are all elements of lived experience of time and its record in humanistic documents -- and these ruptures or lacunae are often the periods of greatest interest to the humanities scholar and lay user of time-based digital media alike.
The challenge is to create a framework for graphically representing such elusive and subjective seeming phenomena. What notation scheme allows us to map anticipation or regret, both ways of conceiving of future and past in modes that inherently involve transformation of past record and future events? Consider the unfolding of multiple narratives simultaneously with contradictory accounts – another standard feature of historical record – or of individual memory against the backdrop of official history. How may we create a graphical communication scheme that allows these concepts clear enough representation to be useful? What metaphors and templates are capable of presenting a conceptualization framework within which interpretation of such events may take place according to these mutable and inflected timescales in a way that may nonetheless prove useful as a research tool for interpretation, analysis, and display of temporal data?
A. Classification of concepts
Our starting point was to delve into the literature on time and temporality across a number of disciplines in order to compile as extensive a list as possible of viable concepts. Our sources included works in humanities, social sciences, and informatics. (See Figures 1A-1D and Bibliography)
The most fundamental distinction in all of these areas, already mentioned above, is between time and temporal relations. Time is thought of as neutral, unbounded, a given. Temporal relations are specific to the relative sequence and duration of events within a frame of reference. Most conventional time-lines are conceived of according to the first notion, with assumptions about time built in to the linear structure and standardized measure. Such concepts are convenient because they appear objective and stable. They allow complex phenomena to be charted in graphic form as single or multiple variables in stable relation to each other. Again, our argument is not with the logical validity or communicative efficacy of such systems, merely with their limitation for use in humanities research or in the expression of more complex aspects of human experience of temporality.
B. Stable Nomenclature
We have made every effort to use existing nomenclature rather than invent an idiosyncratic terminology. To this end, we have culled vocabulary from across disciplines, with every attempt to keep the usage and definitions within established conventions. This nomenclature list outlines the concepts that are fundamental to our project. A number of specialized terms from various fields, though not immediately aligned with our visual project, have been kept on this list because they have suggestivity for later use. (See Figure 2: Vocabulary of Concepts)
C. Visualizations and schemata
Because almost all conventional graphical schemes for representing temporality assume the unidirectional, neutral, and homogenous character of time, they are describable in terms of tables with single or multiple variables. The exception to this is found among those conventions (such as calendars, clocks, or other time-keeping devices) whose form is not necessarily derived from a mathematically based structure or metric, but from a convention of visualization established through other cultural patterns of use. Almost without exception, these schemes assume that temporal sequence, once enacted, is not changed and that future events are similarly stable, even if inaccessible to our perception and understanding. (We may count on the fifty-two weeks of the year having passed, and coming ahead in the next, for instance, without any doubt about their being of the same duration in every case.) Our aim in developing usable visualization tools is to marry the metrical with the cultural and evocative in such a way that our tools remain intuitive but are able to describe complex temporal inflections. (See Figure 3:Typology of schemata and image archive under “Visualizations” at
D. Composition and Display Spaces
Our demonstration of a composition space allowed us to test certain concepts in graphic and conceptual terms and to refine them with the help of interested humanities scholars, graphic designers, and computer scientists. We decided we wanted to be able to show patterns within conventional schemes such as calendars or time lines through effects of visual layering or other graphic devices, for instance, or to be able to use a dynamic time-line with a “now” slider to introduce elements into a past sequence from a newly arrived at perspective (or moment). (See Demo 1 and 2.)
Our research and project development were guided by certain principles. We wanted to keep our schemata as simple and as few in number as possible. We would not needlessly complicate our system. Nor would we introduce new or idiosyncratic terminology, concepts, or visual metaphors and templates if existing terminology, concepts, or visual templates existed that were sufficient to our purpose.
Pursuant to our seminar in June 2001 and the technical workshop that followed it, we came up with a clearer, more reduced (though perhaps not less complex) set of concepts to organize the composition space. These concepts are divided into three types of elements: objects, relations among objects, and actions that can be performed. This list of elements is extended by a notation system for inflecting these entities as vocabulary (a modification of semantic value) or as grammar (a modification of syntactic value). Our intention is to create a basic set of metaphors and templates for the visualization and representation of temporal relations in a composition space, one that includes some flexibility for user-defined notation, and one that will also serve as the basis of visual conventions for display. A continuing criterion is that our basic set of elements permit the visualization of our counter-assumptions about time: that it is dynamically multi-directional for purposes of representation and interpretation, that it is not neutral, and that it is not homogenous. (See: Figure 4 and Figure 5)
E. Technical conclusions: Concepts and XML Schema Definition
We are currently in the process of using our outline of fundamental elements (objects, relations, and actions) as the basis on which to define the XML Schema underlying the visual components of our prototype. Our temporal data model is being captured in a formal schema, so it is unambiguous in the strictest sense, though the meaning of a given model will certainly be subject to interpretation by thinking people. A primary goal of our design process is to develop a legible set of visual conventions that can be employed by a non-technical user for pedagogical or critical purposes in the composition space, but which remain useful for the retrospective display of marked data. These conventions will be manifested as parameterized Flash smart clips, which follow strict rules and communicate with our XML data through Actionscripts and Javascript wrapper objects. Because the diagrams of temporal relations a user will be able to generate inside the composition space are to be restricted and recorded as XML, the composition space itself becomes a tool for formal experiments in content modeling.
Conclusions: The goals for the first phase of this project were to make a survey of ways of conceptualizing and representing time and temporal relations, come up with a set of metaphors and templates for visualization, and lay the technical foundation for creation of an interactive tool set for composition and display spaces. These goals have been accomplished. The next phases of work include conceptual and technical aspects as well. On the technical side, we are proceeding with the creation of an XML schema, development of an interactive visual tool set in Flash, as well as the graphic design of a legible notation system. These are still in preliminary stages, but are mapped out for prototype development. Research goals in the conceptual realm focus on three specialized areas: smart diagrams, linguistic analysis of temporal relations in discourse, and topological mathematics for event modeling.
Next steps: Phase II: Research goals: smart diagrams, natural language
Production goals: refining the visual system, continuing to build
and test the prototype
Phase III: Production goal: linking composition/display spaces.
Figure 1.A: DISTINCTIONS between TIME and TEMPORALITY