The next version of the Raytraced Audio Demo is now available! If you've already signed up at https://vercidium.com/audio, you should receive an email shortly with the download link and an invite to the Raytraced Audio Discord server.
Below is a recap of the new features.
Reverb
The demo is no longer blending between small, large and outdoor reverb presets as covered in my YouTube video. Instead, it's calculating each reverb setting based on:
the size of the room
average length of returning rays
percentage of rays that return to the listener
percentage of rays that escape outdoors
which Materials the rays collide with
direction of nearby walls
This means that you'll hear the correct echo in a room made of sand/cloth, versus a room made of metal/rock. This is made possible by Materials.
Materials
When sound waves collide with surfaces in real life, some of the sound energy is absorbed. The amount of energy that's absorbed depends on the material, for example a carpeted room will absorb more sound than a tiled room.
At one extreme, an Anechoic Room absorbs nearly all sound. This makes clapping your hands sound strange.
At the other extreme, a Reverberation Room absorbs as little sound as possible, meaning the echo trails on for a long time.
The shape of the material also changes the direction the sound waves bounce. In the Anechoic Room, the walls are spiky, causing sound to be reflected up and down, rather than straight back to your ears. Sound gets 'trapped' in the spikes, preventing it from returning back to your ears. This is called sound scattering.
To simulate this in digital environments, we need to know these two values for each material:
Absorption percentage, e.g. a value of 0.02 means 2% of sound is absorbed
Scattering percentage, e.g. a value of 0.08 means 8% of sound is scattered away
Each time a ray bounces, we'll keep track of the materials the rays bounced off. When they all finish bouncing, we can calculate the average absorption and scattering values of all the materials we collided against.
Finally, we can use these values to control reverb parameters like:
Decay Time - how long it takes for the echo to disappear. Higher absorption = lower decay time
Diffusion - how 'clear' the reflected sound is. For example in a tunnel, you can say 'echo' and hear it say 'echo' back. Low scattering = clearer reflections
My understanding of reverb is still very primitive. If you know of better definitions for these terms, please let me know.
Also, there are different absorption/scattering percentages for low and high frequency sounds. This is why metallic rooms have a high pitched / ringing echo, whereas rocky / wooden rooms sound more mellow.
These percentages were not easy to find! It took a lot of googling to find the right percentages for each material. Interestingly, the architecture/construction industry has done a lot of research in this area (e.g. how to not hear your neighbours walking upstairs). So far the materials I support are:
Brick
Concrete
Polished concrete
Rock / limestone
Marble
Indoor wood (plywood)
Rough wood (construction)
Metal
Cloth
Dirt / mud
Sand / snow
Grass
Leaves
Trees
Ice
The complete formula for calculating each reverb parameter is available in the scripts/reverb.cs file, and can be customised in real time.
Air Absorption
On the topic of absorption, I played the Arc Raiders playtest and was blown away by how incredible the audio sounded. One thing that stood out to me was how distant gunshots sound deep / mellow, which is because low frequencies travel further than high frequency sounds.
To figure out how much each frequency should be reduced over distance, I'm using this code by nrtaylor, with 10% humidity, 26 degrees Celsius (78 F), and 101325 hPa.
In the next demo version you'll be able to customise these variables, and hear the difference between fog / clear weather.
Early Reflection Direction
I've learned that there are two types of echo:
Early reflection
Reverb
Early reflection is the first response, e.g. saying 'echo' in a tunnel and hearing that first 'echo' back. It's often more legible and easy to hear.
After that, many different sound waves reach your ears from different directions at different times, resulting in a more 'blurred' echo.
For example, if there's a wall to your left and you fire a gun, you'll hear the loud gunshot from the gun, then the early reflection from the left, then the reverb from all around you.
Your brain instinctively uses this early reflection to help with spatial awareness.
Currently this early direction is calculated as the average of all first bounce positions that are less than 10 voxels away. But this doesn't work for long tunnels, so I need to think of a smarter solution. Possibly just use the average direction of all rays, while giving more weight to shorter rays.
Bots
To simulate war-like environments and stress test the system, you'll be able to spawn up to 100 bots that will fight each other in a huge free-for-all battle.
These bots helped stress test the raytracing system, which helped fix a few pesky crashes and bugs. After leaving 40 bots to fight over 8 hours overnight, I can say the system is robust and working great!
Bots can be enabled by clicking Settings > Raytracing menu in the top right, then scroll down to the 'Bot count' setting.
Ambience
Directional ambience has been disabled, since the direction changes rapidly with low ray counts.
Now when moving indoors, wind and rain will simply reduce in volume. I plan to revisit this feature later, as the next main task on my list is occlusion:
better ray heuristics, rather than relying on the percentage of rays that reach a sound
flood-fill approach, for a consistent coverage of all rooms and crevices
Settings
This demo version was delayed by 3 weeks because of a nasty crash that could only be tracked down by divide-and-conquering the codebase. This means I had to add feature toggles to all the raytracing features, then disable them all and re-enable them one by one till I tracked down the crash.
In the end, the crash was related to bots and OpenGL stuff. But on the plus side, the raytracing system got a thorough cleanup, and you can now control many more raytracing settings.
The Settings UI was also cleaned up, and everything now lives in the top-right settings menu that appears when pressing escape. Old, unused settings have been removed, and you can also search through all the settings now.
New settings include:
Distance falloff model
Enable/disable the direct line-of-sight check
Toggles for occlusion, permeation, indoor detection, reverb, materials, early reflection direction, room centers, leeching
Scripts
Each script file now has clear input and output variables, and there's a new comprehensive reverb.cs file which contains the calculations for each reverb parameter (diffusion, density, echo delay, etc).
These files are located in the scripts folder, and can be edited in real time:
Miscellaneous
Fixed a crash when spam toggling Deaf Mode
Fixed a crash when there are no microphones
Fixed missing UI icons
Fixed crosshair and kill-feed UI
Fixed UI scrolling
Next Steps
Reverb is not quite right yet, I need to do more research and learn what affects each reverb parameter. I'd also like to remove the 'outdoor percent' variable from all reverb calculations, since there isn't a generic 'outdoor' sound. One game might be set in a wasteland with no echo, and another could be a city with lots of large reflective surfaces.
I also plan to expose more variables, settings and code, to allow greater customisation over:
air absorption settings (fog vs sunny day)
materials (absorption and scattering values)
maps - create custom maps in MagicaVoxel and load them into the demo
Then the next step is occlusion - as I'm working on this demo I'm discovering many scenarios where the occlusion system gets things wrong.
The C# SDK is also in the works, which is the next step after this demo.
If you haven't gained access to the demo yet, you can sign up at vercidium.com/audio. I send out invites manually, so don't stress if you don't receive an invite straight away.
That's all for today. Thanks for reading and I'm keen to hear your thoughts on the demo!