SqueakMusic

SqueakMusic

Squeak and Music Computing

Squeak is an open source, cross-platform version of the programming language Smalltalk. Smalltalk is a canonical pure object language with intercessive reflection (full self-modification) and full closures. The Squeak virtual machine is fully open source and itself written in Smalltalk. User Interaction in Squeak can be handled using any combination of Morphic, MVC and newer experimental UI frameworks. These properties give Squeak great flexibility for Music Computing, especially if one uses OSC to glue together whatever else one is using. Below are packages I have written in Squeak that are useful for Music Computing.

1. Tuio For Squeak

Reactivision is an open source, cross-platform computer vision framework for the rapid tracking of graphical markers attached to physical objects, also capable of multi-touch finger tracking. It is used in a range of table-based tangible user interfaces, including the tangible digital musical instrument Reactable. Note that the marker tracking framework Reactivision is open source, but the music application Reactable is not. Tuio for Squeak provides a client for the former, not the latter.

Reactivision communicates with applications via the TUIO protocol (standing for tangible user interaction). Open source code for TUIO clients and demonstration applications is available in, Java, C++ and Max, and now Smalltalk.

A Tuio client for Squeak seems to occupy an interesting local maximum in design space, since it combines the power of Reactivision for tangible user interaction with the ease of programming complex animated user interfaces for multiple users in Squeak.

Getting and running Tuio for Squeak

Tuio for Squeak includes the following components,
TUIOClient, TUIODemo, TUIODump, TUIOListener and supporting classes. The package can be found at SqueakSource, or at SqueakMap.

To make practical use of Tuio you will need to download the computer vision application Reactivision for your operating system from somewhere such as here. The Reactivision application works with a wide range of webcams and video cameras and can track the movement and orientation of fiducial markers (paper markers) attached to physical objects. The fiducial markers can be downloaded and printed off from here.

Tuio for Squeak is most easily demonstrated by creating and running an instance of the demo class TuioDemo. This can be done by evaluating an expression such as the following.

demo := TuioDemo new start

This will create and start the sole instance of TuioClient automatically. To stop the demo and clean up, evaluate the following expression.

demo stop

Under the hood

When running, the TuioClient client requests the OSCserver to listen on Port 3333 for tuio messages from reactivision. The Tuioclient translates these messages into a higher level protocol representing the movement and orientation marked objects and cursors. TuioDemo uses this protocol to create moving morphs whose movement and orientation matches that of the fiducial markers. Alternatively TuioDump simply prints out the location and orientation of markers to the Transcript.

Creating new Tuio applications in Squeak

New Tuio applications can be created quickly and conveniently in Squeak by subclassing from TuioListener or by modifying TuioDemo.

Acknowledgements

• Tuio for Squeak is based on the Tuio client by Martin Kaltenbrunner.
• Tuio for Squeak makes use of the OSC Squeak implementation by Markus Gaelli.

Download

2. Open Sound Control for Squeak

Open Sound Control for Squeak (version 5.5.1)
OSC (Open Sound Control), here implemented for Squeak, is a vital component in many music computing and pervasive interaction systems. Open Sound Control for Squeak version 5.5. includes enhancements, fixes & new unit tests to support bundles, intensive input & the Tuio protocol for Reactivision, and new unit tests. This software now forms part of the tool kit underpinning research prototypes and demos for the OU Pervasive Interaction Research Group. Revisions by Simon Holland. Original versions by Markus Galli.