April 28, 2009

Time in Computation

This is the last post in the series Brain - Time - Music - Computing.
Previous: Time and the Brain

The notion of computation was explicitly created for use outside of the flow of MWT. Models of computation provide primitives for describing processes in a purely timeless context (computability), or in an artificial and abstract flow of time marked by computational operations (algorithmic complexity). The resulting abstract manifestation of time in computing is enforced as a strong invariant, universally and implicitly relied upon.

This state of affairs has serious implications for the design and use of computing artifacts in MW. Music exists at the confluence of creation, representation and performance, where inherent limitations of time representations in computation become evident. Music systems fall into two broad categories: online or real-time systems, and off-line system. The significance of these categories in the context of interaction is explored in [1]. Henkjan Honing more specifically analyses and classifies the issues related to the representation of time in music computing [2].

Process-oriented systems, which address real-time needs such as sound synthesis or performance (sequencing), adopt either tacit or implicit representations of time. Tacit representations restrict the notion of time to the “now,” the flow of MWT. Implicit representations tie the flow of time in the system to that of MWT in a direct, fixed and inescapable relation. For example, the Max paradigm in its various forms, e.g. Max/MSP and Pure Data, embraces this approach. Composition and editing systems, on the other hand, manipulate static representations of music, with explicit modeling of time relationships in their mathematical abstraction, outside of any flow of time. OpenMusic is a representative member of this category. The dataflow model of OpenMusic maps to the functional programming model, and does not provide for the creation and manipulation of a flow of time in relation to MWT.

These orthogonal representations of time, and the computational models they support, serve their particular needs well. However, the dichotomy they introduce with respect to time representations seems irreversible. Puckette for example denounces the divide between processing and representation paradigms as a major obstacle to creating a comprehensive system for music making [3]. Musical improvisation requires a seamless blend of composition (representation) and real-time performance (processing), the ability to move freely in and out of flows of PT and MWT. The design of a real-time interactive musical improvisation system, such as the multimodal interactive improvisation system MIMI, requires to bridge the representation/processing gap. Past efforts in this area have leveraged existing programming models to produce ad hoc solutions; MIMI benefits from the use of a new computational paradigm that offers a general solution.

The Hermes/dl design language

Hermes/dl adopts an asynchronous concurrent computation model. Its data model distinguishes volatile data, in a flow of time, from persistent data outside of any flow of time. As a result, the language primitives afford the modeling of concurrent flows of time and computation, and of processes that take information in and out of these flows.

Arrangements of language primitives consistently associated with specific functionalities or behavior constitute patterns of the language; they can be used as models or guides for system design. The patterns of primitives and interactions that characterize the transfer of information from a flow of time into persistent form constitute instances of the Aggregator pattern. The patterns of primitives and their interactions that characterize the transfer of information from persistent form into a specific flow of time constitute instances of the Sampler pattern. The next section introduces the Hermes/dl design language in the wider context of the motivations that lead to its creation.

[1] Alexandre R.J. Francois and Elaine Chew. An architectural framework for interactive music systems. In Proceedings of the International Conference on New Interfaces for Musical Expression, Paris, France, June 2006.
[2] Henkjan Honing. Issues in the representation of time and structure in music. Contemporary Music Review, 9:221–239, 1993.
[3] Miller S. Puckette. A divide between ‘compositional’ and ‘performative’ aspects of Pd. In Proceedings of the First International Pd Convention, Graz, Austria, 2004.

No comments: