I’m going to make this brief as it’s the Easter weekend. I mainly wanted to post an image and movie from the process showing on Thursday which ended the Mons residency. Hopefully they speak for themselves, since the primary focus of this stage was to develop the visual and sonic aesthetics. Neither is completely there yet – we had some technical limitations in this residency as we didn’t have the high-end PC we’ll use for the finalised portal, and could only use two Kinects, while the final portal will have three (and finally not have a Kinect stuck in the middle of the screen!). Also of course, it’s only one portal, so the other three users aren’t there. I’m happy that the ‘look and feel’ of it is approaching the final form though, so mission accomplished! Thanks to all at Transcultures for making it happen, and the University of Mons for hosting us. Now, on to the final stretch, the final residency in Enghien-les-Bains, Paris in May, and the opening in London, Paris, Brussels and Istanbul on the 9th June..

Significant progress has been made with the first physical portal this week, which is very exciting. Yesterday I went down to Aurelian’s house in the French countryside, where the portal has been taking shape in his garage. Very impressive and imposing on first sighting. I also went to see the site that has been arranged for the opening in June, which is equally impressive. This Galeries Royales St Hubert, right in the centre of Brussels, just literally off the main square. It’s a great big space, very light and airy, and no-one will be able to miss the great big me and my shadow portal appearing from nowhere in the middle of it, like a 2001 (the film, not the year) monolith (thanks Nicolas for that one). This is being arranged with the Cinema Galeries, who have been very helpful and enthusiastic – thanks!

Today the portal arrived in the chapel. The design is great for putting up (and taking down) super quickly, just like lego (and I love lego). It’s still not complete, as it doesn’t have the inner (fabric) wall or a roof, but I think it’ll serve for now to give an impression of what the space will feel like. Today we’re moving all the equipment etc. into there, and hopefully we’ll have a prototype portal!

Just a little example of the attention to detail I’m trying to bring to the sound rebuild:

As before, the sounds for the ‘traces’ – the little trails of particles that are left behind by user movement in the space – which I think might be one of the key interaction sounds, is made up of little clicks which set off (pitched) resonant filters.

In Istanbul and London, these clicks were made synthetically (just a tiny burst of white noise with an envelope) – they had no ‘texture’ to them and were all exactly the same.

What I want to do now is for every click to be different, and for the clicks for each portal to be very slightly different to the others. To do this, I’ve gone back to my musique concrete roots, and have used recorded sounds instead of synthetic ones. I’ve found four sources that are excellent sources of clicks – polystyrene packing material, combs, magnetoids (anyone remember them?) and dead leaves (from the back yard of the Transcultures house). I’ve recorded a stretch of sound for each of them, and then edited out every seperate little click. So now, each portal has over 100 clicks to choose from, which makes a subtle but crucial difference, and makes the whole thing a little more ‘alive’ somehow.

It’s the little things..

I’ve evolved the sound a bit more over the last few days. It’s the same basic idea and material, but with a little more variety in terms of the layers I’m using and the way they’re controlled. I feel like this is a bit of a balancing act here – it needs to be quite approachable I think, given the sheer variety of people who are likely to engage with the final piece. But it also needs to be distinctive and characterful – I don’t want to play to the lowest common denominator. It also needs to be interactive, but I want it to have a convincing musical flow to it. It needs to be a result of the users’ movement, but also encourage people to move (in some way be ‘good to dance to’). Finally it needs to be sparse enough that cause and effect are clear (I’m hoping the sound might function as an integral part of the navigation and general usability), but complex and varied enough to be interesting. Tricky..

I don’t think I should develop this much more before we reconnect the sound and image (and interactivity). Hopefully we’ll do this over the next couple of days and start to form an overall audiovisual aesthetic and a new model for interaction. We should hopefully be building the first portal during this time, so it’s an exciting few days we have ahead!

I’ve completely re-thought the sound over the last couple of days. I didn’t develop it at all in London as we focused on the core logistics of getting four portals working, so as the piece started to take shape the sound didn’t really keep up.

One of the key things we developed in London was the way in which users can be creative in the space, and we ended up with two paradigms – ‘shadows’ (we called them ‘sculptures’ in London, but shadows makes more sense given the title of the piece) and ‘traces’ (‘trails’ in London, but I think traces sounds better). The former is long term, and full-body – regular imprints users leave in the space. The latter is short-term, and involves the particles left behind by the key points of the body (which we first developed in Istanbul).

Only the latter has been sonified so far, making a bit of a disconnect between sound and image. I’ve also realised that the shadows potentially have more audio, and audiovisual potential. If the shadows get left at regular intervals (but different intervals for each user), and these have corresponding audio events, then some nice polyrhythms can be produced. These would be more sonically interesting that the signification of the particle traces, which are so numerous that they tend to produce overly-dense sounds, which just blend into sonic soup. They could also provide cues to the users as to when the shadows will be created, and – being more clearly identifiable, could even help with the navigation.

This sound is a first attempt at what this might sound like at this point. Of course, it’s slightly meaningless without the visual element (so far!), but it’s an attempt to ‘compose the soundtrack’. The high ringing sounds represent the shadows, on regular cycles corresponding to a 4:5:6:7 ratio. The tinkly/rustly (those are technical terms) represent the traces. The low pulse and short high rhythmic sounds are global, and are a byproduct of the 4:5:6:7 polyrhythm. They’d be spatially attached to the light at the centre of the space I think, or possibly ubiquitous throughout the space.

Phill has been walking around with furrowed brow today. He’s been spending a lot of time scribbling diagrams; staring at boxes, and chairs, and boxes on top of chairs. I’m a bit worried about him.

What’s been vexing him is the particularly knarly problem of combining data from several Kinects. A few people have used multiple Kinects, but I haven’t seen much work where the Kinects are actually pointed at the same thing (or box, or chair, or person). The problem is that many points will actually be seen by both (eventually all three) cameras, so combining the two sets of data requires a kind of 3D jigsaw-puzzle thinking that hurts.  For the full technical detail on this see Phill’s own blog.

On a more satisfying note – I think we’ve cracked the problem we had with the navigation in London. This proved so tricky and sensitive that even highly trained dancers had trouble with it. I’m happy to say that we’ve made a lot of progress on this over the last couple of days – it’s now much smoother and ‘steadier’ somehow, and can be easily adjusted to calibrate the sensitivity. We won’t know for sure until next week when we try it out with a few more people, but I’m hopeful it’s sorted now.

So.. having said we’d focus on the aesthetics this time, we do seem to have spent a bit of time here in Mons on technical issues. They’re kind of big though, and do impact on the aesthetics to a certain extent. It seemed crazy not to deal with them while we had a bit of time. In the meantime I have been making great strides with the sound – more v. soon.

This marks the beginning of the third MADE me and my shadow residency in Mons, Belgium – hosted by Transcultures.  Phill and I arrived last night, and this morning Lucie from Transcultures took us to where we’ll be working for the next couple of weeks.  We’re in the architecture department at the University of Mons, working in the Salle du Bélian exhibition space, an old chapel.  We’ve had an amazing variety of places to work on the project – quite inspiring!

For this residency, we’ll be focusing primarily on the aesthetics of me and my shadow, both sonic and visual.  We’ll also be building the first actual portal installation, which I’m very excited about.  I’ll keep you posted…