Bang…Hisssssss…Clunk…Whirrrrrrr！

Sonogram Mapping of New York City
Augmenting the Cityscape with Audiovisual Experience

Tech:
AR.js
ShaderPark
Figma
Sound Design
Touchdesigner
Ambisonics
Using data sculpting, this project captures and continuously displays the unrecognized repercussions in our environment. The piece employs specific algorithms to process this information and translates it into images superimposed on landscapes viewed through a phone or tablet. The work’s data filters, subtracts, and ultimately overlays 3D organic visuals. These forms represent the pulses of New York City's infrastructure system.

When participants plug in their headphones, lift their phones, and walk down the street, they experience an amplification of the city's subtle sounds at spots tagged by the interface. These locations provide access to sounds generated underground beneath the pavement, linked to the “A-tag” (see image above)—construction markings on street roads that indicate different types of work with various colors.



Concept of Vectors,
Invisible informations,
Immersive street walkers,
How do you see the networks of New York City...
Koolhaas, Rem, and Harvard Project on the City. Mutations. Barcelona: Actar, 2001.

 

The project uses these tags as portals to sound, mapping the interface to the existing symbolic system of construction signs, fire hydrants, and markers on pavement and sidewalks. These symbols are typically interpreted by technicians, such as pipe-layers and firefighters.




The interface is built in order for the audience in the pop-up exbhition better to understand the process of sound analysis and the visuals.

Some Other Visualizations





Design Process, Sound Analysis, and Visualizations


Defining Materials, Sound Quality, and Colors with A-tags
Based on the user testing results and sound analysis process conducted through TouchDesigner, here is a series of audio visualizations created through multiple processes. The sound materials are sourced from field recordings around Union Square Park (see sound libraries section



Prototyping the Application Interface


Sound Libraries(On-Going)