︎

Bang…Hisssssss…Clunk…Whirrrrrrr！



Sonograms of New York City

Augmenting the Cityscape with Audiovisual Experience

A Testing Application Running in Glitch.io, to test the visuals running in shader and audios under the Vue.js framework

05-13-2022

︎

︎ AR.js, ShaderPark
︎ Vue+Vite
︎ Figma
︎ Sound Design
︎ Touchdesigner
︎ Ambisonic Sound
Process, Pieces, Works, Experience


A Series of Live Sound-Walk near Union Square Park. Build with Vite+ Vue.js, Shader Park, and Three.js


A-tag

By using data sculpting, this project captures and constantly displays unrecognized repercussions in our environment.  The piece uses specific algorithms to process this information and transfers them to images imposed on landscapes being viewed through a phone or a tablet. The work’s info-data subtracts, filters, and ultimately overlays 3D organic visuals. These forms represent the pulses of the infrastructure system of New York City.
When the participant plugs in their headphones, lifts their phones and walks down the street, they will experience the amplification of subtle sounds of the city at spots tagged by interface to access sounds generated underground beneath the pavement by the “A-tag”(see image above), a construction marks over the street roads and indicates a different type of works with different colors. It is used as the portal to sound that mapping interface to the existing symbolic system construction signs, fire hydrants, and construction markers on pavement and sidewalks. Technicians can usually read these systems, such as pipe-layers and firefighters.


The interface is built in order for the audience in the pop-up exbhition better to understand the process of sound analysis and the visuals.



Some Other Visualizations 





Design Process, Sound Analysis, and Visualizations





Defining Materials, Sound Quality, and Colors with A-tags

Based on the User-Testing Result and Sound Analysis Process through TD, Here are a series of Audio Visualizations through Multiple Processes. The sound materials are coming from the field recordings around Union Square Park.(see sound libraries part) User Testing Process(left image): According to the user test, testers are invited to listen to a bunch of samples of sound, and write down their feelings to describe these sounds. Finally, they signed each color dot (which stands for the meaning of A-tag Categories to different shapes of soundwaves, see the sound analysis process shown in the interface built with TouchDesigner)



Prototyping the Application Interface



Sound Libraries(On-Going)