I forgot to write a post for Adventure #3 because I was busy getting a start on my project. Perhaps I’ll write an Adventure #3 post after this one, but for now I want to start by describing the project that Sheraz and I are working on.
As I mentioned in my Sound Maps post, I have chatted with Ryerson’s SMART Lab about their sound mapping project. Sheraz joined me in our second meeting because he know GIS stuff and has a better grasp on the math behind their work than I do. But firs, let me explain the basics of our idea.
The SMART lab has over 1,000 recordings from all round the City of Toronto and they have software that can analyze it for different kinds of data. They have done some studies with this data and have focused on three kinds of information that will predict whether a particular sound will create a stressful sonic environment. The three things they look at are spectral irregularity, pulse clarity, and RMS.
RMS is the easiest to understand. It is, essentially, loudness; the higher the RMS the louder the sound. Pulse clarity is pretty straight forward too. The higher the pulse clarity value, the more rhythmic the sound. Finally, spectral irregularity indicates what the sound looks like when mapped across the sound spectrum. The easiest way I found to understand this is to compare white noise, which has a flat line across the spectrum (check out simplynoise.com for a great example–a life-saver for tinnitus sufferers) to the erratic noise of a construction site, for example. A high spectral irregularity means it is difficult, or impossible, to plot the sound in a nice, straight line on the sound spectrum. Or, this is the difference between graphing a smooth line and a jagged line with lots of peaks. Again, I don’t understand math well enough to really explain this.
So the basic idea is to take these three bits of information and map them on to hue, saturation, and value (brightness) of the pixels in an image. We’re creating a bit of a tenuous relationship between sound and colour values, but the idea is to create images that present the meaning of the data and get people interested and engaged with the sound map.
So far the plan looks something like this: Spectral Irregularity will be mapped to hue using random Gaussian numbers, pulse clarity will be mapped to saturation, and RMS will be mapped to brightness. We still need to figure out some of the math and we also need to figure out some of the technicalities behind processing over 1,000 images.
So far Sheraz and I have been working on a Processing sketch that does some of the math we want. Ideally, we’d have an interactive website with a heat map of Toronto and the user would be able to “walk” through the manipulated images using Street View. I think there are some restrictions on how Street View is used, though, so I think I’ll be difficult to make this work. Right now I’m working on grabbing a static street view image from each location.