Alexandre R.J. Francois

The objective for MuSA_RT 2.0 as a holiday project was to use cutting edge software development tools and frameworks/packages to put MuSA.RT in the hands of anyone with a phone, tablet or computer (limited to the Apple ecosystem because of resource and time constraints). The version currently in the Apple App Store , although quite crude in many respects, achieves this goal, and will serve, time permitting, as a starting point for exciting explorations. 3D graphics and Augmented Reality Rendering 3D geometry that approximates the original MuSA_RT graphics was almost too easy, and as a result hasn't yet received the attention it deserves. There is much to explore and improve in terms of geometry and appearance (materials and lighting), and then efficiency. Similarly, the Augmented Reality (AR) mode is but a bare proof of concept, which only sets the stage for exciting explorations. The first question is of course: what, if anything, can an AR experience of MuSA_RT bring to the pe...

The Music on the Spiral Array . Real Time (MuSA.RT) project started almost 20 years ago. My first collaboration with Elaine Chew , MuSA_RT applies music analysis algorithms rooted in her  Spiral Array model  of tonality, which also provides the 3D geometry for the interactive visualization space. The MuSA.RT project lasted many years, produced numerous publications, and various versions of the system featured in lectures and performances all around the world. The software produced for this project was a constantly evolving research prototype (not something to put in the hands of a general public user), and subject to contemporary technical limitations. A Mac App released in 2012, intended as companion software for the book  Mathematical and Computational Modeling of Tonality: Theory and Applications , Elaine Chew (2014) , made the system accessible to general users. MuSA_RT 2.0  is a universal iOS/iPadOS/macOS app that analyses the audio signal from a mic...

Check it out!

This is part 1 of 5 in the series Brain - Time - Music - Computing . Next: The Brain in Middle World Dawkins remarks that brains have evolved to help animals survive within the orders of magnitude of size and speed at which their bodies operate. He calls Middle World (MW) this relatively narrow range of phenomena directly and intuitively accessible to perceptual and cognitive processes [2]. Dawkins invokes the human brain’s evolutionary entanglement with MW to explain humans' difficulty in grasping, and coping with, the physical realities of the universe outside of its familiar confines, from the sub-atomic scales of quantum physics to the universe-size scales of relativity. But the fundamental properties of MW can also help characterize the nature of the tasks at which brains came to excel, in particular the fundamentally dynamic nature of these tasks. Everything in MW is subject to what the human brain perceives and understands as time, “the continuum of experience in which even...

"...but he (the mathematician) is always speaking of a given moment -- a static moment, that is -- and not of flowing time. In short, the world the mathematician deals with dies and is reborn at every instant [...] To know a living being or natural system is to get at the very interval of duration, while the knowledge of an artificial or mathematical system applies only to the extremity." Bergson, Creative Evolution , 1914, cited by William Gooddy, Time and the Nervous System , 1988.

MuSA.RT Opus 2: real-time tonal analysis and visualizations in concert. Video of live performance by Elaine Chew, at USC's Alfred Newman Recital Hall, during rehearsal for "The Mathematics in Music." The MuSA.RT system is by Elaine Chew and Alexandre Francois. (Video captured on a Canon S500 Digital Elph camera, afternoon of Jan 24, 2007.)

A couple of weeks ago, Elaine Chew gave a spectacular lecture/recital titled The Mathematics in Music . She performed Tamar Diesendruck's Sudoku Variations for the piece's world premiere, Ivan Tcherepnin's Fetes - Variations on Happy Birthday and Peter Child's Doubles III . The performance of Fetes was accompanied by a live real-time visualization of the tonal structures of the piece as computed by the MuSA .RT system (which I created with Chew). The project demo page links to video excerpts captured during rehearsal.

"Many scientists who study artificial intelligence use the mathematics of formal logics--the predicate calculus--as their major tool to simulate thought. But human thought--and its close relatives, problem solving and planning--seem more rooted in past experience than in logical deduction. Mental life is not neat and orderly. It does not proceed smoothly and gracefully in neat, logical form. Instead it hops, skips and jumps its way from idea to idea, tying together things that have no business being put together; forming new creative leaps, new insights and concepts. Human thought is not like logic; it is fundamentally different in kind and in spirit. The difference is neither worse nor better. But it is the difference that leads to creative discovery and to great robustness of behavior. " Donald A. Norman, The Design of Everyday Things , Basic Books, 1988. (p. 115 in 2002 paperback edition; highlights added for this post)

"In theory, there is no difference between theory and practice. But, in practice, there is." Jan L.A. van de Snepscheut

"If people do not believe that mathematics is simple, it is only because they do not realize how complicated life is." -- J. H. Von Neumann One can wonder why computer vision (CV) has not benefited from the increase in computational power as much as computer graphics (CG), although the two fields grew from the same seminal work (Roberts, 1965). CG is a synthesis activity, that starts from mathematical models of some physical objects and their environment, and applies some mathematical models of light and material physics to create an image of the objects in their environments under various conditions. Better models, and computational power directly translate into more realistic images. The models are designed by humans. Computer vision is an analysis activity, that seeks to form models of objects, environments, situations, activities, etc., from images (still pictures or video streams). Depending on their intended purpose, these models are not necessarily of the same natur...