Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 22 Dec 1997 06:20:26 +0100 (MET)
From:      Luigi Rizzo <luigi@labinfo.iet.unipi.it>
To:        rhh@ct.picker.com (Randall Hopper)
Cc:        hasty@rah.star-gate.com, multimedia@FreeBSD.ORG
Subject:   Re: MPEG audio/video sync & Re: These mtv video pauses are murder
Message-ID:  <199712220520.GAA01789@labinfo.iet.unipi.it>
In-Reply-To: <19971221225947.08124@ct.picker.com> from "Randall Hopper" at Dec 21, 97 10:59:28 pm

next in thread | previous in thread | raw e-mail | index | archive | help
> Say, something I've been meaning to ask you and the group.  Is there
> something in the MPEG system stream format that you know of which allows
> you to "attach" (or suggest a relation) between a particular point in the
> video and the audio streams.  Or is that implicit in the interleaving.
> 
> What I'm a little concerned about is that for longer recordings, without
> these attachments, the strea audio will likely slip out of sync with the
> video during playback (due to a dropped frame or two per second [or more]

I was just thinking about the same problem in the context of
videoconferencing tools, where you have separate tools for audio and
video, and they have separate timestamps derived from different clocks
(the one for video is presumably the sender's frame rate, which can be
relatively good for broadcast tv, but I am not sure how precise it is
from a camera or a tape; the one for audio is derived from the sample
clock, and that is quite often off by 1% or more).

...
> Really, its kind of a nutty process that goes on now because of the tools
> that are available.  First fxtv writes a RAW capture file that has the
> audio/video interleaved (so we know the audio/video relationship).  Then we
> separate those out into separate audio and video streams (video padded up
> to 30fps) so mpeg_encode can be used to build an MPEG video stream, and
> mpeg_audio can be used to encode an MPEG audio stream.  Finally those
> streams are merged together with mplex into a MPEG system stream, where we
> hope the video still "lines-up" with the audio.

I suppose the problem if any will be in the capture phase. We have a
way to timestamp (using the cpu clock) incoming video frames (do you
use that ?), but e.g. for audio you cannot rely on the sample rate
of the card.  Depending on how unlucky you are, speed might be off
by 1% or so -- even if smaller, the drift _is_ noticeable on long times.

I am not sure on how you determine the timing of the streams, but I
believe you should only use the audio sample rate for timing frames
both during capture and playback (the latter I believe is implicit
once you have interleaved frames ?) so that during playback you
don't have drifts.

	Cheers
	Luigi
-----------------------------+--------------------------------------
Luigi Rizzo                  |  Dip. di Ingegneria dell'Informazione
email: luigi@iet.unipi.it    |  Universita' di Pisa
tel: +39-50-568533           |  via Diotisalvi 2, 56126 PISA (Italy)
fax: +39-50-568522           |  http://www.iet.unipi.it/~luigi/
_____________________________|______________________________________



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199712220520.GAA01789>