Handle Mechanical Sounds

[freezy]

This is a proposition how to handle mechanical sounds, described in #226. There are a few additional constraints that aren't described in that issue that made me go back and forth on a few different approaches. But here is what I think could work.

Sound Definition

As described in #226, a sound can contain multiple clips, random pitch and so on. So the first step is to define such a sound asset, typically as a ScriptableObject.

In the inspector, it could look something like this:

These ScriptableObjects are files that reference audio clips, and they sit on the file system (as opposed to the scene), which makes them also usable as assets in the asset library.

Mapping

One of the main concerns is that we don't want authors to have to manually assign let's say a collision sound to every single element that collides with the ball. Given we have an asset library, the sound should already be part of a prefab. Of course, it should be possible to assign different sounds, but assets should be ready-to-go where possible.

Which means that the sound configuration should be GameObject-specific. Let's start with a simple case:

So this is the inspector of a Flipper object, where we see a new Mechanical Sounds component. In here, we have a list of sounds that to be emitted on various actions.

• The first sound is the flipper coil when enabled. The Trigger property looks up which trigger events this GameObject supports, and presents them in a drop-down.
• The Sound property is a reference to the sound asset we defined above.
• The Volume property defines how to set the volume. Fixed set to 1 means same volume as configured by the sound asset. This includes the asset's own volume correction and randomization, if set. The slider just acts a multiplicator, allowing fine-tuning on a per-object basis.
• Action defines, well, what to do (start, loop, stop).
• Finally, the Fade property sets a time for how long the sound should be faded in or out.

In this example, there are three sounds:

1. Play the "Flipper Coil On" sound once when the coil of the flipper turns on.
2. Start looping the "Flipper Coil Hum" sound when the coil of the flipper turns on.
3. Stop looping the "Flipper Coil Hum" sound when the coil of the flipper turns off.

So these are sounds whose volume might be randomized but is not dependent on any playfield action. So let's look at a use case where the sound volume actually depends on something gameplay related, like a collision.

This is an example of a ball colliding with a metal post. We're in the metal post's inspector, and instead of Fixed, we now have Ball Velocity under volume. This value is also retrieved from the GameObject. Speaking in code, our game item elements that emit sounds would implement an interface like this:

interface ISoundEmitter {
  SoundTrigger[] AvailableTriggers { get; }
  VolumeEmitter[] GetVolumeEmitters(SoundTrigger trigger);
  event EventHandler<SoundEventArgs> OnSound;
}

public struct SoundTrigger {
  string Id;
  string Name;
}

public struct VolumeEmitter {
  string Id;
  string Name;
}

public struct SoundEventArgs {
  SoundTrigger Trigger;
  float Volume;
}

This would allow for our new mechanical sound component to show the appropriate options to the user.

Runtime

During runtime, the Player would first retrieve all GameObjects with the mechanical sound component, subscribe to their OnSound event and route the audio output according to the configuration and the GameObject's position to an audio source.

Future

There are a few things that aren't covered in here, namely:

• What about different sounds from different balls for the same GameObject? Well, we could add a ball filter to each sound, but for that we have to define how we deal with balls first (right now there is only one type of ball)
• What about a global sound manager? Yes, it might be useful to at least list all defined sounds in a central editor, like Unity's Light Explorer does.
• What about sound sets? Let's say we have multiple recorded machines, and I'd like to swap all of those sounds with sounds recorded from a different machine? Maybe we could also tag sounds with a type, so we know which ones to replace.
• How about effect processing? I don't know much about that. Could be clip-specific, sound-specific, GameObject specific or global. TBD.

[markran]

Thanks for the link to this thread from #226! Continuing discussion here...

I randomly found that thread when Google searching hoping to find a way to avoid manually tweaking a zillion sound files across a hundred tables to get SSF balanced properly on the cab I'm putting together. As for your suggestion of "working on this...", I'd be happy to help out where I can as I think what you're doing here presents a unique opportunity to accelerate the evolution of virtual pinball while simplifying authoring and bringing some standardization to the growing library tables. Going through installing and configuring my first cabinet over the last couple months has been a big learning experience but I'm still a relative newbie to VP.

Fortunately, I do have a fair bit of experience managing the development of large software projects (large = millions of lines of code, hundreds of engineers, tens of millions of users). Unfortunately, it's been a lot of decades since I've done any real coding myself :-(

Reading what you've laid out above, I think we're very much on the same page. Spending recent weeks experiencing the challenges of getting disparate tables all working together on one hardware configuration demonstrates the need for some kind of Global Sound Manager like you mention. Between AltSound, PinSound, PupPacks, SSF, DOF and original ROM sounds, it's the kind of gloriously eclectic multi-headed beast you'd expect from a naturally evolving diaspora of content creators and developers working over many years. It's more than a little amazing and also a testament to the community that it all works as well as it does.

Having so recently experienced the joy of hand-tweaking the volume of dozens of individual WAV samples and changing each line in an AltSound spreadsheet, the need for some kind of global "meta-knobs" to control the relative volume of groups of sound types akin to a mix bus in an audio mixer seems obvious. Perhaps just as useful, would be some kind of recommended standard for how sound samples should be normalized which authors can reference. I've found sample sets where each sample seems to be locally maximized within its waveform and attenuated by a unique value at play back and I've seen sets where the waveforms seem set to some shared reference level so each sounds "correct" relative to the others in a given table context if played back at the same volume level. While I have lots of ideas I can suggest, being completely new to VP, I'm reluctant to propose potential solutions in the absence of experience. One way to get started might be to have some experienced table authors suggest a workable standard approach based on their practical experience actually implementing a sound set. Or, perhaps you are already moving in that direction?