[Fwd: Re: [libavg-devel] Audio Support Patch]

[Ulrich von Zadow]

Hi,

I sent some mails to Nick in private by mistake - this should be an open
discussion :-).

Cheers,

  Uli

-------- Original Message --------
Subject: Re: [libavg-devel] Audio Support Patch
Date: Wed, 12 Dec 2007 15:24:24 +0100
From: Ulrich von Zadow <uzadow at libavg.de>
To: Nick Hebner <hebnern at gmail.com>
References:
<7122baa90712111513s30065c8bjc258f434a5b8c8a7 at mail.gmail.com>	
<475F3A0E.9010807 at libavg.de>
<7122baa90712111815g2e4f80d4sc9c4c481b79318b1 at mail.gmail.com>

Nick Hebner wrote:
>     > per-video volumes controllable from python.
> 
>     You're not fading the volume internally, right? Then we'll have audible
>     clicks whenever there's a large volume jump.
> 
> Currently, I am simply using an SDL function (SDL_MixAudio) to do mixing
> of multiple buffers. The documentation of this says that it does
> 'addition, volume adjustment, and overflow clipping' but it is fairly
> basic, and I realize that it will probably have to be replaced. I don't
> know much about this topic, do you know of any good resources for
> learning more about volume fading?

Actually, volume fading is not that difficult. Applying a volume just
means multiplying each sample with that volume. If you change the volume
from one sample to the next, you'll get a discontinuity, which our ear
hears as a click. To solve that, just do a linear interpolation between
the original and the destination volume over a number of samples (I
thing 100 or so are enough).

A good debugging setup for something like this is an output device that
writes audio files. You can easily look at this file with a sample
editor and see what's going on.

>     > I have not
>     > added any formal testing yet (any ideas on how to accomplish this?)
> 
>     Jup - lots :-).
> 
>     - Add a test to src/video/testvideo that decodes a video with a silent
>     audio track. Check if the data delivered is silence (or nearly so).
>     - Same, but have a sine wave (440 Hz or something) as audio track.
>     - Repeat with different sampling rates (once that's supported).
>     - Repeat with and without a video channel (once that's supported).
>     - Repeat with different video container formats and codecs.
>     - Once we add volume fading, we can change the volume during a decode
>      and have the test look for changes in the sample value that are too
>      large - those are the clicks.
> 
>     - Add tests to src/test that runs a video/two videos with audio.
> 
> Good ideas, this would have to be accomplished in C++ based test
> programs then right?

Jup. Have a look at src/video/testvideo.cpp.

>     ffmpeg can handle audio resampling with just a few lines of code :-).
> 
> Cool. It seems that SDL can too. Any idea which is better?

Hm. Instinctively, I'd assume the ffmpeg guys know more about sampling
and resampling, but I haven't looked at the code or seen it run.

>     Maybe add an <audio> node? Or leave audio-only nodes
>     (because avg files are layout-oriented) and instead add an audio object
>     that can be created, started etc. separate from the avg node hierarchy?
> 
>     As I said, I'm not sure. Any ideas?
> 
> Hmm. An <audio> node would be nice for things like background music
> (where the sound simply plays without anything interacting with it).
> Also, it might nice to be able to define audio resources in the avg file
> instead of having to write python to create them (the avg file in
> addition to being layout-oriented, is resource-oriented to some degree
> as well). Also, it might be nice to include sounds in the node heirarchy
> so that they are associated with other objects in the heirarchy ( i.e. a
> sound plays while its containing DivNode is active).
> 
> On the other hand, dynamic audio objects might be better suited to quick
> sound effects. These could still be accomplished by adding, removing,
> and interacting with <audio> nodes in the heirarchy as well, so maybe
> this could be filed under the 'better support for dynamic node creation'
> endeavor?

Ok, convinced. <audio> node it is, then.

Are you sending these messages by private mail on purpose? If not: I
prefer discussing general things like this on the mailing list so we
have access to all the ideas floating around there :-).

Cheers,

  Uli

-- 

Ulrich von Zadow | +49-172-7872715
Jabber: cocacoder at jabber.berlin.ccc.de
Skype: uzadow