Design:
On the design side, our major focus for the week was the solidification of our EQ Prototype, as well as polishing our documentation for the Reverb prototype and further follow up discussions with industry professionals. For our EQ prototype, we decided to go with visualizing the frequency waves on the Z-axis within an “audio cube”:
Guests will see every track within the song floating in this cube, each one containing a tower of rectangles that represent the frequency spectrum of it. The lower frequencies are located at the bottom, and get higher the closer to the top of the cube the rectangles are positioned. Guests will be able to see in real time which frequencies each track is putting out at any given time.
Guests will be able to equalize these frequencies in two variations. The first variation, shown above, will allow the guest to stretch out the low, mid, and high frequencies by interacting with the cubes that surround each section. The second variation will have the guest place objects on the frequency tower to change its “brightness” or “muddiness”. The object for brightness will be a sun, and the muddiness a clouded nebula. We are still solidifying exactly what will change visually when adding brightness/muddiness on the frequency tower. This discussion will be our highest priority going into next week. Along with finalizing our second EQ variation, we will also be preparing playtesting surveys for EQ. We hope to begin playtesting at the end of next week so we have sufficient playtesting and documentation before going into Thanksgiving break.
Tech:
This week’s focus was to get up and running on the EQ and Audio Cube prototype. We decided that Shiva should concentrate on setting up the framework necessary for getting the FFT (Fast Fourier Transform) data from each of the tracks of the songs and being able to use this data to drive animations/visuals. On top of that, we wanted to make sure guests are also able to tweak EQ for every track and have them do that in real-time as well.
Major source for FFT related setup was this FMOD forum post : FMOD Unity FFT
And parallelly, Sam would focus on getting the visual setup from Brandon and the interaction setup done since we had to change quite a bit from the arc-like track field setup to a 3-dimensional cube setup.
We settled on using the 3 Band Parametric EQ in FMOD since that had enough controls for us to play around with and all would be a good starting point for a novice musician.
Source : https://fmod.com/resources/documentation-studio?version=1.10&page=mixing.html
The main math involved in processing the FFT data from each channel was to isolate for every frame the frequency bin around which the average energy is centered around at any given point. We chose to do the weighted average of the frequency domain graph to get this data. This data is then internally used to drive the y-position of each track orb as well. Next week’s focus is to refine the visuals substantially, and finish with all the interactions.