A generative audio-reactive work using Gaussian splat photogrammetry scans.

Made in TouchDesigner I’m wandering through 3D point cloud data, cycling through different scanned botanical scenes I recently captured, distorting the points and pulling them apart. The piece is driven by the music, a track I composed for my second album Blackfeather. An audio analysis patch listens for swells in the music which pushes the movement forward—infinitely evolving as several generative layers interact. The video here is a capture of a single cycle but this can run in real-time and evolve endlessly.

I’ve been fascinated with point cloud photogrammetry and the recent development of Gaussian splats. This new way of distributing data across 3D points creates extremely photo-realistic results. But here I’m going in the opposite direction. 

When I started making 3D captures I quickly noticed parallels between the way points clouds are captured and the way our experience and memories are also a similar collection of randomly encountered data points. What the camera sees from multiple angles becomes rendered with great detail and accuracy. But the parts of an object or scene that get less coverage can be highly warped, inaccurate representations.

If you go to a new city, you have memory of the streets you walked, you remember a nice restaurant etc. But the next street over is completely missing from our memory. Our realities are constructed by warped and incomplete data sets. 

Rather than using point cloud data to construct perfectly rendered 3D geometry, I’m interested in showing them in dream-like flashes. I see this piece as an intentional misrepresentation of collected data, focusing on the distortions and artifacts that color our memory. This is a journey through a dream—a “sweven”.