Description

"Together, Apart" was an interactive art installation showcased during Toronto's Nuit Blanche in 2022. Projectors were mapped to all the walls in a room, with Xbox Kinects mapped to them. Over time, as people cycled in and out of the room, different videos of interviews would play randomly on different sections of the walls, one at a time. These were only represented visually as silhouettes through movement. The walls were covered with particles that reacted to any motion, whether from the dialogue videos or from people interacting with the exhibit. As more people entered the room, the visuals and music would change, with colors becoming more vibrant and the music more intense.

The project's core involved eight people, plus our producer and lead. I primarily handled technical implementation, doing a lot of testing and prototyping for ideas that I or other people had. I was kept on after the primary period of the project ended to help fully prepare it.

On the software side, we used TouchDesigner for visuals and FMOD for audio. The hardware used five projectors and six Kinects. The project ran off one PC, rendering across five projectors, so optimization was a must. For example, recreating some effects done by our visual artist with less overhead. The systems were kept modular where possible, as most work and testing were done independently of the final hardware. This allowed us to easily scale up the number of projectors or Kinects.

Simple Python scripts and GLSL shaders were used sparingly to simplify some node operations. For example, it was much easier to parse the Kinect data through Python rather than split it up between many nodes. For our audio, we used FMOD for its better dynamic capabilities. It was run through a small C# program communicating data from TouchDesigner to FMOD using Open Sound Control (OSC), as there was no native support.