[libavg-devel] Audio Support Patch

[Nick Hebner]

Hi,

You may have already seen in svn that I have made a few commits to the audio
branch, but I just wanted to discuss them a bit and get any feedback (I
forgot to mail patches before my commit). My changes are as follows:

Initial patch application:
- Player::setChannels() and Player::setSampleRate() were merged into
Player::setAudioOptions() (mirroring Player::setOGLOptions()).
- I changed Node::setDisplayEngine() to Node::setRenderingEngines()
throughout the source. Video nodes now use this to add themselves as
AudioSources to the AudioEngine.
- Added Node::getAudioEngine() and changed Node::getEngine() to
Node::getDisplayEngine()
- Video nodes take care of removing themselves from the AudioEngine in
disconnect()
- Moved channels and samplerate to avgrc under a new 'aud' subsection (and
removed them as properties of the avg node)
- Stubbed out AsyncDecoder::fillAudioFrame() and added a TODO. I will return
to this a bit later.

Audio/video syncing:
After spending a few days on vacation, a few mulling over this problem, and
a few more trying to implement something, I think that I have arrived at a
good solution, although it has flaws.

At first, I thought I would try to make the audio stream sync to the system
clock since the video stream is basically sync'd to this through display
time requests. After a ton of playing with this, I could not get this
approach to work well enough to reliably keep the streams in sync, and the
audible artifacts were clearly perceptible, even when I tried to minimize
any changes. I expected this coming in. Syncing to the current video
position worked a bit better, but again, audible artifacts were perceptible.

My next approach was to sync the video stream to the freely playing audio
stream. To accomplish this, I keep track of the position of the audio stream
as it is playing. When a video frame is requested, and the video has audio,
it overrides the requested time with the audio clock. This way, the frame
that is displayed is always the one that is closest to the audio position,
and the audio stream is not modified.

This approach works well for keeping the streams in sync, however there are
two issues:
1. There will be a delay between when the audio is requested by SDL, and
when it is actually rendered. This should be fairly constant for a given
hardware setup, but may vary from system to system (right?). I was thinking
that we could add an A/V delay parameter in avgrc to adjust for this.
2. By syncing the video stream to the audio, setFakeFPS() breaks for videos
with audio (although videos without audio still work). To me, this seems an
acceptable loss. What do you think?

And those are my changes from the last week or two. Let me know what you
think.

Thanks,
Nick
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.datenhain.de/pipermail/libavg-devel/attachments/20071228/4db7170a/attachment.htm