This project explored the development of an interactive, collaborative music-making ecosystem designed for novices, using four primitive-shaped objects that influence the soundscape in unique ways.
Pyramid - alternates between 4 drum patterns
Cube - alternates between 5 synth melodies
Long cylinder - closes low pass filter
Short cylinder - adjusts tempo
My responsibilities were to develop the technical infrastructure, including hardware wiring, coding, and all necessary software setup. I took the lead in designing the interactive objects, with some contributions from my groupmates, and handled the laser cutting.
In a small study with five non-musicians, participants enjoyed shaping music from preset patterns, and most worked out each shape's general role within ten minutes. The rounded shapes were easier to map thanks to their continuous feedback, while the cube and pyramid needed instructions before people could use them with confidence.
Instruments for novices
Novices don't need precise control over musical elements, they want to enjoy playing and connect with others. Blaine & Fels argue such interfaces should be simple enough to approach in minutes, while offering layers of complexity to uncover over time.
Musical networks
Weinberg describes interconnected playing as a musical ecosystem. We chose a process-centred network, focused on the players' experience rather than the composition, where every node influences the others and playing together becomes a discovery of the system itself.
Precedents
Piano Cubes, glass jars tilted to shift a constantly playing arpeggio, heavily inspired our interaction. Polymetros showed how people play together at public exhibitions, and how confusing it gets when players can't tell their contribution apart from others'.
Research questions
We asked how a human-machine ecosystem of interdependent nodes could empower non-musicians to create music, and what methods people use to map such a system's inputs to its outputs.
Technology
Each shape holds a Puck.js Bluetooth sensor broadcasting its data to a central computer. Receiving the Bluetooth signals directly on the computer proved unreliable, so I added a development board that only listens for them and passes them on as MIDI, giving us fast, robust communication. Interference from other Bluetooth devices still caused some latency during testing.
Sound engine
We built the soundscape in VCV Rack, a modular synthesizer software. Each object produces its own sound or alters the sounds of the others, all from a predetermined set of musical patterns.
Shape interactions
Building on Piano Cubes' turning and flipping, we designed four shapes. The pyramid plays a different drum pattern on each side, the cube plays melodies, the tall cylinder filters the drums and background music when rotated or rolled, and the short cylinder sets the tempo of the whole ecosystem.
Object size
In the first tests the objects were small and light, so people picked them up instead of moving and rotating them. We made them significantly larger to invite the intended interaction.
Cube sound
The cube repeated a single note on each side, which some testers found annoying. We changed it to play a predefined melody once each time it is flipped.
Short cylinder
The short cylinder originally changed the density of the background music, which most testers didn't notice. We remapped it to control the tempo of the whole system.
The study
We ran seven tests with five non-musicians, two of them in pairs. Each participant got minimal, intermediate or full instructions, explored freely for 10 minutes while thinking aloud on video, and then took part in a semi-structured interview.
Observations
Everyone started carefully, one shape at a time, and only began composing once they understood each shape. Disorganised exploration like shaking and tapping far outnumbered systematic flipping and rolling, 46 to 11 occurrences, and people picked the objects up 36 times.
Interviews
Participants liked making music from preset patterns, though the simplicity made them lose interest after a while. The shapes invited intuitive motions but confused people when the sound didn't match the shape, and collaborating helped them discover interactions together.
Mapping the shapes
The cylinders were easy to map, as rolling and rotating give immediate, continuous feedback. The cube and pyramid only change when flipped to another side, and that discrete response made them noticeably harder to understand.