[libavg-devel] Audio Support Patch

[Ulrich von Zadow]

Nick Hebner wrote:
> Hi,
> 
> You may have already seen in svn that I have made a few commits to the
> audio branch, but I just wanted to discuss them a bit and get any
> feedback 

Appreciated.

> (I forgot to mail patches before my commit). 

A non-issue, since I can always look at WebSVN, you're working on a
branch and the code is more-or-less what we've discussed anyway :-).

> - Player::setChannels() and Player::setSampleRate() were merged into
> Player::setAudioOptions() (mirroring Player::setOGLOptions()).
> - I changed Node::setDisplayEngine() to Node::setRenderingEngines()
> throughout the source. Video nodes now use this to add themselves as
> AudioSources to the AudioEngine.
> - Added Node::getAudioEngine() and changed Node::getEngine() to
> Node::getDisplayEngine()
> - Video nodes take care of removing themselves from the AudioEngine in
> disconnect()
> - Moved channels and samplerate to avgrc under a new 'aud' subsection
> (and removed them as properties of the avg node)

Nice up to here.

> - Stubbed out AsyncDecoder::fillAudioFrame() and added a TODO. I will
> return to this a bit later.
> 
> Audio/video syncing:
> After spending a few days on vacation, a few mulling over this problem,
> and a few more trying to implement something, I think that I have
> arrived at a good solution, although it has flaws.
> 
> At first, I thought I would try to make the audio stream sync to the
> system clock since the video stream is basically sync'd to this through
> display time requests. After a ton of playing with this, I could not get
> this approach to work well enough to reliably keep the streams in sync,
> and the audible artifacts were clearly perceptible, even when I tried to
> minimize any changes. I expected this coming in. Syncing to the current
> video position worked a bit better, but again, audible artifacts were
> perceptible.

Right. Inserting/removing samples from an audio stream can't be done
without some signal processing. One way to implement this is be to
adaptively resample the audio stream. Another is repeating/omitting
small (50ms?) chunks of audio, with a small cross-fade between any two
chunks...

Anyway, what you've done is perfectly ok :-).

> My next approach was to sync the video stream to the freely playing
> audio stream. To accomplish this, I keep track of the position of the
> audio stream as it is playing. When a video frame is requested, and the
> video has audio, it overrides the requested time with the audio clock.
> This way, the frame that is displayed is always the one that is closest
> to the audio position, and the audio stream is not modified.

Have you tested this with longer movies?

> This approach works well for keeping the streams in sync, however there
> are two issues:
> 1. There will be a delay between when the audio is requested by SDL, and
> when it is actually rendered. This should be fairly constant for a given
> hardware setup, but may vary from system to system (right?). I was
> thinking that we could add an A/V delay parameter in avgrc to adjust for
> this.

Sounds good, but it'd be nice to know if this is actually relevant
before adding the parameter.

> 2. By syncing the video stream to the audio, setFakeFPS() breaks for
> videos with audio (although videos without audio still work). To me,
> this seems an acceptable loss. What do you think?

How about just turning off all audio when setFakeFPS() is called?

Cheers,

  Uli

-- 

Ulrich von Zadow | +49-172-7872715
Jabber: cocacoder at jabber.berlin.ccc.de
Skype: uzadow