You can play Volo Airsport right now! Get it through our own website, the Humble Store, or Steam.
What’s Happening
This week we continued work on the Great Big Architecture Overhaul mentioned last time, and worked with Michael to choose audio technology. Last week I explained in some detail what the architecture work entails, so this post will mostly be about audio.
Other than that I spent an abnormal amount of time fighting with hardware faults, plugin problems and Windows annoyances. Yay, computers!
Right, welcome to What’s Happening, episode 2!
(Had a nasty headache over the weekend so this post is a bit late, hope you don’t mind!)
Audio Technology
We met with Michael to refine the ideas for the first batch of audio work, but most time was spent figuring out which audio technology to use, and what the workflow should be like.
It’d be great if he could work on complex game sounds without constant support from us. More independence leads to more productivity, as it saves everyone from having to wait on one another. In addition, Michael worked with Fmod and Wwise on other projects, so it’d be nice if he could at least work with something similar. Finally, it’d be great if Frank and I didn’t have to spend many hours programming audio tools. We already have full plates thank-you-very-much.
We compared the new audio tools available in the Unity 5 beta to the Fmod Unity plugin. And long story short: it’s likely that we’re going with Fmod.
Mixing & Effects
Both Unity 5 and Fmod have competent mixing and grouping capabilities. Both let us group multiple sound sources under a single mixer channel for easy mastering and effect application.
This was a hassle in Unity 4 and below, where you couldn’t simply lower the volume of similar sounds (e.g. all the footsteps) with a single fader, but had to set the volume of all those individual sounds. Furthermore, if you wanted to apply a specific effect (e.g. echo) to a group of sounds you’d have to apply that effect to each separate sound as well, which is not very efficient.
Events
Where Unity 5 stops and Fmod continuous is complex, parameterized audio events.
Fmod lets you compose audio events that can contain arbitrary layers, clips and effects; of which parameters can be controlled during playback to make each copy of the event sound different. This is a step up from Unity’s AudioSource components, which have no notion of layers. You have the option of playing a simple clip, a sequence of clips of which parts fade in and out based on your speed, mixing in some distortion based on the amount of damage done, anything. You could do this in Unity, but the workflow isn’t there by default, you’d have to program the tools to do it.
The neat thing here is that the audio events and the code that triggers them are completely separated. We can fire a simple event from our code, and we don’t care what happens after that. If Michael made sounds for that event, they play. If we don’t have sounds for it yet (or audio is turned off for testing purposes), that’s fine too.
FMod events can be further expanded to contain non-linear sequences which behave like state machines. That means you can more easily design sounds that react to changes in the wind, to how tired wingman feels, to tree collisions, whatever you want. Events are edited visually, using a (somewhat strange) adaption of the linear timeline metaphor found in modern audio creation applications such as Ableton Live.
Basically, Fmod events can do a whole lot of fancy things that Unity 5 can’t out of the box, and they are things we’d like to have. We thought for a moment of implementing this kind of event system on top of Unity audio, but I’ve recently been learning that just because we could, doesn’t mean we should. The basics would be done in a day, but the fine details would take a year.
Latency
There is one big unsolved problem though! When I got my first Fmod-powered sound going I found there was quite a bit of delay between my inputs and the resulting sound. I did an input-to-audio-output latency test to get a better picture of the situation.
I created an empty Unity project which lets me play a OneShot Fmod event that plays a simple sound (with a near-instant attack) at the press of a button. That’s all it does. I then measured (with a microphone) the time between the sound of me pressing that button on my keyboard to the sound coming out of my speakers.
On Windows: ~0.4 seconds.
On Mac: ~0.3 seconds.
With Unity’s default audio I seem to shave off a bit of time, but windows still chugs along at about 0.33 seconds.
It’s funny, I never really payed much attention to audio latency while on gameplay scripting duties, but those are big numbers. I would expect the sound to still output a little faster than this given that input-to-output latency for graphics is considerably lower.
I did the same tests on some other games to compare, to get a better feel for how they deal with audio latency:
Evolve (click-to-gunshot time): ~0.22s (surprisingly slow, but still faster than us)
Street Fighter 4 ( click-to-start-menu time): ~0.1s (they’re showing off, obviously)
Anyway, enough geeking out. I’m dropping the latency issue for now and will just implement our existing sounds in FMOD as planned, then we can evaluate whether the latency is a problem in the first place. Even if we decide the latency is a problem I doubt there’s too much we can do about it.
I’m slightly worried though, a third of a seconds is very noticeable. Make the delay between action and reaction, and the two will appear to be separate, unrelated events.
Ugh, at these times I wish we could just have our games use ASIO or something.
Architecture Overhaul
Yep, still working on this. Frank and I started out pair programming (where one is in the driver’s seat and the other is minding the map, so to speak) on some of the finer points of how the new event systems and GUI elements work. This week though, I’m mostly trying to get FMOD to work while Frank experiments on his own with ideas inspired by HTML5, Javascript and Facebook’s React library. We looked at Unity 4’s new GUI system too, but while it looks spiffy the way events work doesn’t hook into Volo all that well. I still hope we can leverage the visual aspects of Unity 4 GUI though! It’s fast, is easy to compose with, and has support for animation and everything; it’s just the way user input is routed through it that bugs us a little.
Actually, note to self: try to implement a simple version of the current UI with Unity 4 GUI to see how it stack up, perhaps we’re making this too difficult for ourselves.
I’d write more about this, but this post is already late. Next time!
By the way, it’s interesting to note that pair programming didn’t work at all for us a year ago! Our programming styles were too different, and we were just blocking each other’s flow all the time. These days it seems to go much better, and we’re actually coming up with better solutions to problems than we would have working alone.
Any New Features, then?
Not yet, sorry. Like I mentioned last week, this work is very important, but you won’t get to taste the fruits of our labor until we’ve finished the new framework and can start building new features on top. If you haven’t read it yet, What’s Happening 1 contains a thorough explanation of why we’re doing this.
I suspect the most difficult part will be finished after the coming week though. At least I hope so, I’m anxious to start writing multiplayer networking code. Fingers crossed.
As always, let me know what you think in the comments or on the forums. Blue skies!