⭐ This article revolves around a Pro feature.
This article is part of the "Use case" series, designed to offer practical demonstrations of HeavyM. It aims to help you understand the software and show you its potential by providing detailed, step-by-step instructions of the creation of real-world projects.
Here, we'll present a use case using the Timeline and Audio Analysis.
Note: if you're looking for specs or details on the operation of each Sequencer module individually, feel free to check out the other Sequencer articles!
Use case presentation
We want to create content based on audio tracks. Unlike in this other Timeline-related use case, we're not going to place cues on the audio waveform to decide when to trigger events, instead we're going to use the Audio Analysis module to make visuals react directly to sound.
Let's imagine we're preparing a little show for kids, with objects that react to sound effects. At our disposal we have:
- audio tracks:
- Timeline track (ambient sound, train station bell): https://sound-effects.bbcrewind.co.uk/search?q=07071139
- Sequence track (train sound, crossing warning): https://sound-effects.bbcrewind.co.uk/search?q=NHU05075079
- a project (2.12.2) created using only HeavyM content, with only 1 sequence that shows a cute train arriving at a station (don't laugh, it's an example and it's for kids!)
And that's all we need to get started!
Note: as we won't be editing shapes, you can use the Canvas as a preview by disabling View → Show Geometry to keep only the rendering.
1. Setting the timings of the visuals and audio tracks
Setting up the sequence on the Timeline
The single Sequence of the project shows the little train crossing the length of the Canvas from right (start) to left (end).
Let set this Sequence on the Timeline so that it starts at the beginning. The easiest way to place the "Play Seq 1" cue is to drag & drop the sequence directly onto the Timeline, at the 00.000 mark.
Now, when you hit Play on the Timeline, you can see the train starting from the tunnel on the right. By default, the animation lasts 10min and loops, but we're going to set the correct timings right after with our audio track.
Setting up the audio track of the Timeline and its timings
Now we'll add the sound effects we have selected. When preparing our show, based on the speed of animation we wanted and the audio tracks, we've settled that a duration of 22.5s for the whole animation would suit this project. First off, let's import our ambient audio file that represents the alarm bell of the station on the Timeline.
-
Import the audio file using this button
. Its waveform appears on the track.
- By default, the audio file is adjusted at 00:00 but we want to delay it in order for the ringing to start when the train starts approaching the station platform. In the Timeline settings
set the "Start audio at" timer to 00:00:07.000. You'll see that the audio waveform on the Timeline is translated to start at 7s (zoom in with the mouse wheel or + button on the right.)
- Then, in the same settings window, set the Timeline length to 00:00:22.500 to fix the duration of the animation.
- Finally, add a "Stop" marker from the add cue button
and place it at the very end of the cue track in order to stop the animation automatically at the end (otherwise it will loop).
Click on Play on the Timeline to restart the animation and see (and hear) that the audio now only starts when the train gets closer to the station and that the animation stops once the train arrives.
Note: if you need more audio editing capabilities, there are many free apps out there to prepare your soundtrack before importing it into HeavyM. Audacity is a great one!
Setting up the audio track of the Sequence
There is another audio track that we'd like to add, the train sound effect that represents the sound of the train getting close to the station and its crossing alarm ring.
As Seq 1 is our only sequence, it's already Edited so you can access its parameters from the Sequence Settings panel of the Sequencer. Scroll down to the "Audio" section and click on "..." to select the audio file to import.
Click on Play on the Timeline to restart the animation and hear that the train sound effect starts at the beginning, at the same time as Seq 1 starts playing.
Note: as you can also hear after 7s when the station audio starts, both sounds simply overlap.
2. Setting up the audio analysis and reactive visuals
Adjusting the Audio Analysis settings
What we want to do now is make some visual effects react to the audio in the project. For that, we can setup the Audio Analysis module so that it listens to the audio files we've added, and configure the frequency bands so that they target specific sound events.
- In the Audio Analysis & Tempo panel
, click on the settings button
- This opens up the parameters of the Audio Analysis module. There, select "HeavyM Timeline & Sequences Audio" in the dropdown list of sources in the header and click on the switch next to it to toggle the module ON.
- Then, with the settings window still open, click on Play on the Timeline to start the animation and its audio and observe the spectrum that is created from the analysis. You can clearly see some peaks that correspond to the ringing of the station and some that correspond to the alarm of the train.
- Let's try to "isolate" them. While still playing the audio, play with the handles of the frequency bands so that the Mid band (blue) corresponds to the range of the station bell and the High band (green) corresponds to the the range of the train horn. You can see in real-time the resulting mean that is calculated for each band in the 3 bars in the module. (If needed, you can adjust it with the gain sliders.)
Making effect parameters audio-reactive
Now to make the scene audio-reactive. Remember that any parameter that has this selector can be made audio or tempo-reactive. Here’s what we’re going to do:
- Click on the group « Bell » to make it active and go to its Properties on the left. There, set the LFO Mode of the Opacity to « High ». That way the value of the opacity of the station warning will be high when the bell rings and it will fade otherwise.
- Then click on the group « Ring » to make it active and go to its Properties on the left. There, set the LFO Mode of the Opacity to « Mid ». That way the value of the opacity of the train horn will be high when the bell rings and it will fade otherwise.
An here's the full result, once we hit play, all is automated and in sync: (volume up!)
3. (optional) Exporting the rendering as a video [⭐ Pro+]
You might want to get a single video file of the result. In HeavyM, you can use the Video Export feature (only in Pro+) to get a .mov rendering of the Timeline, including the Timeline and Sequences audio files.
Here, we have prepared our file perfectly for this: the total length of the Timeline is exported, with "Play" cues taken into account and the changes due to the audio analysis on the Timeline & Sequences audio also precisely rendered. So the resulting video file will correspond exactly to the animation we see when we click on the Play button of the Timeline in this project!