[Fwd: Re: [libavg-devel] Audio Support Patch]

[Ulrich von Zadow]

-------- Original Message --------
Subject: Re: [libavg-devel] Audio Support Patch
Date: Wed, 12 Dec 2007 02:31:58 +0100
From: Ulrich von Zadow <uzadow at libavg.de>
To: Nick Hebner <hebnern at gmail.com>
References: <7122baa90712111513s30065c8bjc258f434a5b8c8a7 at mail.gmail.com>

Hi Nick,

incredible! It's high time someone tackled that. Lots of comments are
inline, and I'll have a longer look this weekend.

Nick Hebner wrote:
> Depending on your version, you may need to upgrade to get this function.

Then we need to at least run as before (i.e. without audio) if the
function isn't there before this can be put in a release - or else we'll
probably break libavg for lots of people.

> I have gotten the basics of it working: support for multiple videos
> playing audio, 

Very nice.

> per-video volumes controllable from python.

You're not fading the volume internally, right? Then we'll have audible
clicks whenever there's a large volume jump.

> I have not
> added any formal testing yet (any ideas on how to accomplish this?)

Jup - lots :-).

- Add a test to src/video/testvideo that decodes a video with a silent
audio track. Check if the data delivered is silence (or nearly so).
- Same, but have a sine wave (440 Hz or something) as audio track.
- Repeat with different sampling rates (once that's supported).
- Repeat with and without a video channel (once that's supported).
- Repeat with different video container formats and codecs.
- Once we add volume fading, we can change the volume during a decode
  and have the test look for changes in the sample value that are too
  large - those are the clicks.

- Add tests to src/test that runs a video/two videos with audio.

Also some hints:
- You probably know this, but: the test files should be pretty short,
otherwise the download times for releases and the time the tests take go
through the roof.
- The tests should be silent by default, or you'll have to work in a
soundproof room. (That's also a good reason for adding a separate global
volume value :-) ).

> There are a number of issues outstandi)ng:
> - Audio and video streams are not sync'd. They seem to look ok if you
> use the video stream fps, but will probably get out of sync eventually.
> - Need to create fancier resampling function to handle mixing of audio
> with different bitrates/channels.

ffmpeg can handle audio resampling with just a few lines of code :-).

> - Video/FFMpegDecoder should handle audio-only files in addition to a/v
> or v files
>     - Does this mean that the 'video' node should become a more general
> 'media' node or something like that?

Hm. Not sure. A media node becomes confusing (what's the width of an
audio-only node?). Maybe add an <audio> node? Or leave audio-only nodes
(because avg files are layout-oriented) and instead add an audio object
that can be created, started etc. separate from the avg node hierarchy?

As I said, I'm not sure. Any ideas?

> - Output bitrate/channels should be configurable from an avg file.
> - Formal testing

Cheers,

  Uli

-- 

Ulrich von Zadow | +49-172-7872715
Jabber: cocacoder at jabber.berlin.ccc.de
Skype: uzadow