[libavg-devel] Audio Support Patch

[Nick Hebner]

On Dec 29, 2007 10:58 AM, Ulrich von Zadow <uzadow at libavg.de> wrote:

> Nick Hebner wrote:
> > Hi,
> >
> > You may have already seen in svn that I have made a few commits to the
> > audio branch, but I just wanted to discuss them a bit and get any
> > feedback
>
> Appreciated.
>
> > (I forgot to mail patches before my commit).
>
> A non-issue, since I can always look at WebSVN, you're working on a
> branch and the code is more-or-less what we've discussed anyway :-).
>
> > - Player::setChannels() and Player::setSampleRate() were merged into
> > Player::setAudioOptions() (mirroring Player::setOGLOptions()).
> > - I changed Node::setDisplayEngine() to Node::setRenderingEngines()
> > throughout the source. Video nodes now use this to add themselves as
> > AudioSources to the AudioEngine.
> > - Added Node::getAudioEngine() and changed Node::getEngine() to
> > Node::getDisplayEngine()
> > - Video nodes take care of removing themselves from the AudioEngine in
> > disconnect()
> > - Moved channels and samplerate to avgrc under a new 'aud' subsection
> > (and removed them as properties of the avg node)
>
> Nice up to here.
>
> > - Stubbed out AsyncDecoder::fillAudioFrame() and added a TODO. I will
> > return to this a bit later.
> >
> > Audio/video syncing:
> > After spending a few days on vacation, a few mulling over this problem,
> > and a few more trying to implement something, I think that I have
> > arrived at a good solution, although it has flaws.
> >
> > At first, I thought I would try to make the audio stream sync to the
> > system clock since the video stream is basically sync'd to this through
> > display time requests. After a ton of playing with this, I could not get
> > this approach to work well enough to reliably keep the streams in sync,
> > and the audible artifacts were clearly perceptible, even when I tried to
> > minimize any changes. I expected this coming in. Syncing to the current
> > video position worked a bit better, but again, audible artifacts were
> > perceptible.
>
> Right. Inserting/removing samples from an audio stream can't be done
> without some signal processing. One way to implement this is be to
> adaptively resample the audio stream. Another is repeating/omitting
> small (50ms?) chunks of audio, with a small cross-fade between any two
> chunks...
>

I figured that more signal processing would help with the artifacts, but I
couldn't get that approach to work well even with artifacts, so I gave up on
it.

>
> Anyway, what you've done is perfectly ok :-).
>

Great.

>
> > My next approach was to sync the video stream to the freely playing
> > audio stream. To accomplish this, I keep track of the position of the
> > audio stream as it is playing. When a video frame is requested, and the
> > video has audio, it overrides the requested time with the audio clock.
> > This way, the frame that is displayed is always the one that is closest
> > to the audio position, and the audio stream is not modified.
>
> Have you tested this with longer movies?
>

I watched a ~25 min video, and it stayed sync'd throughout. Do you think I
should test with anything longer?

>
> > This approach works well for keeping the streams in sync, however there
> > are two issues:
> > 1. There will be a delay between when the audio is requested by SDL, and
> > when it is actually rendered. This should be fairly constant for a given
> > hardware setup, but may vary from system to system (right?). I was
> > thinking that we could add an A/V delay parameter in avgrc to adjust for
> > this.
>
> Sounds good, but it'd be nice to know if this is actually relevant
> before adding the parameter.
>

Yeah, I wasnt't planning on writing any code until I saw the problem, I just
assume that it is there.

>
> > 2. By syncing the video stream to the audio, setFakeFPS() breaks for
> > videos with audio (although videos without audio still work). To me,
> > this seems an acceptable loss. What do you think?
>
> How about just turning off all audio when setFakeFPS() is called?
>

That could work.

>
> Cheers,
>
>  Uli
>
> --
>
> Ulrich von Zadow | +49-172-7872715
> Jabber: cocacoder at jabber.berlin.ccc.de
> Skype: uzadow
>
> _______________________________________________
> libavg-devel mailing list
> libavg-devel at datenhain.de
> https://mail.datenhain.de/mailman/listinfo/libavg-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.datenhain.de/pipermail/libavg-devel/attachments/20071229/64d1ea7f/attachment.htm