A "Face swap" for audio - module

The backstory:

During my freelance years I worked several times at the Donaueschinger Musiktage.
For those who never heard of it, the festival was founded in 1921!! and it’s sort of the “Glastonbury” for contemporary composers.

In 1999 I had the job to set up and maintain a large PA system in the park outside the Donaueschingen palace, a multichannel sound installation
that played transformed fragments of music all day. Kind of a potpourri throughout the festivals long history.
The whole thing was called totodonaueschingen by Alvin Curran.

The audio library was processed with csound through randomly selected modules and played out over a maxMSP patch to the various channels.
It was quite spectacular.

One set of algorithms I found particularly interesting.
2 tracks (let’s say an orchestra piece and spoken word)
were split into envelope and waveform somehow and then swapped across,
like a face-swap app for sound.

The result was 2 very strange signals and each carried both information in a way.
The person spoke with an orchestra voice and the orchestra played a speech.

Any ideas on how to build an analog hardware module for that?
Or maybe it already exists?

3 Likes

Are you talking about an envelope follower? It’ll take running amplitude average of one track and use for something else, like the amplitude of another track. If you have two and play them against each other, that’d sound pretty weird and cool.

2 Likes

That reminds me of a vocoder, but yes,

sounds like an envelope follower was involved.

4 Likes

I was also thinking about 2 envelope followers (maybe using these chips I have) and 2 compressor/expanders to make the signal into a brickwall before applying the other envelope curve.

4 Likes

This really made me think of this “cool trick” (I like math, ok?): when you compute the FFT of two images, you can swap the magnitude and the phase of these two, and have two new images that share each other features. For example, lets take two “random” image on the internet:
image

Now, if we swap the mag/phase couple or the real/imag parts, we get:

(As we can see, that’s the phase that holds all the information)

I will try with sound, this may give interesting results!

7 Likes

So!
With these two sounds:

The results:

… Not so great. The voice over orchestra (phase_mag.wav) is pretty nice tho. Maybe it would be better with carefully chosen sound clips.

4 Likes

I learned some jargon today. The phase_mag.wav for sure sounds very similar to vocoders as @analogoutput mentioned, but the others are also rather interesting.

For instance, if for imag_real.wav, you used a more rhythm focused loop instead i bet you will end up with some dang neat results. It’s shifting will be very complimentary for drum loops i wager.

Try it with this loop and the “uplifting orchestra”. If it still sounds a bit off, i might have to match BPMs.

image

Now that you mentioned, it sounded a lot like a vocoder effect rather than the result of telec16’s experiment. I may even have footage somewhere on minidisc or DV tape. I’m happy to have sparked some interest.

hahahahaha thats funny the morphing pic±! love it haha

2 Likes

Have a listen to the demo I uploaded some hours ago. This is a demo of an analog vocoder I built in which the envelope of one signal (called the modulator) is used to modulate the amplitude of another signal (carrier).

Well, it is a bit more complicated in that the modulator is split into 14 frequency bands using a B914 filterbank, the envelopes of these filter bank outputs are then used to modulate the 14 frequency bands of the carrier signal (it is send through another B914) and the 14 modulated signal are added together again to give one output signal:

In principle the mechanism used resembles your description of the project you worked on.

2 Likes

that sounds very nice and pretty close to what I remember.

Done!
(still here: Dropbox - test sound - Simplify your life )
Once again the real/imag mix doesn’t really add something… (but it reminded me of Homestuck somehow)
drums_orchestra_phase_mag.wav sound nice!

And just FYI, this is what I have in mind when I’m talking about complex numbers:
image

1 Like