From owner-freebsd-multimedia Sun Dec 21 22:48:36 1997 Return-Path: Received: (from root@localhost) by hub.freebsd.org (8.8.7/8.8.7) id WAA03087 for multimedia-outgoing; Sun, 21 Dec 1997 22:48:36 -0800 (PST) (envelope-from owner-freebsd-multimedia) Received: from labinfo.iet.unipi.it (labinfo.iet.unipi.it [131.114.9.5]) by hub.freebsd.org (8.8.7/8.8.7) with SMTP id WAA03078 for ; Sun, 21 Dec 1997 22:48:32 -0800 (PST) (envelope-from luigi@labinfo.iet.unipi.it) Received: from localhost (luigi@localhost) by labinfo.iet.unipi.it (8.6.5/8.6.5) id GAA01789; Mon, 22 Dec 1997 06:20:26 +0100 From: Luigi Rizzo Message-Id: <199712220520.GAA01789@labinfo.iet.unipi.it> Subject: Re: MPEG audio/video sync & Re: These mtv video pauses are murder To: rhh@ct.picker.com (Randall Hopper) Date: Mon, 22 Dec 1997 06:20:26 +0100 (MET) Cc: hasty@rah.star-gate.com, multimedia@FreeBSD.ORG In-Reply-To: <19971221225947.08124@ct.picker.com> from "Randall Hopper" at Dec 21, 97 10:59:28 pm X-Mailer: ELM [version 2.4 PL23] Content-Type: text Sender: owner-freebsd-multimedia@FreeBSD.ORG X-Loop: FreeBSD.org Precedence: bulk > Say, something I've been meaning to ask you and the group. Is there > something in the MPEG system stream format that you know of which allows > you to "attach" (or suggest a relation) between a particular point in the > video and the audio streams. Or is that implicit in the interleaving. > > What I'm a little concerned about is that for longer recordings, without > these attachments, the strea audio will likely slip out of sync with the > video during playback (due to a dropped frame or two per second [or more] I was just thinking about the same problem in the context of videoconferencing tools, where you have separate tools for audio and video, and they have separate timestamps derived from different clocks (the one for video is presumably the sender's frame rate, which can be relatively good for broadcast tv, but I am not sure how precise it is from a camera or a tape; the one for audio is derived from the sample clock, and that is quite often off by 1% or more). ... > Really, its kind of a nutty process that goes on now because of the tools > that are available. First fxtv writes a RAW capture file that has the > audio/video interleaved (so we know the audio/video relationship). Then we > separate those out into separate audio and video streams (video padded up > to 30fps) so mpeg_encode can be used to build an MPEG video stream, and > mpeg_audio can be used to encode an MPEG audio stream. Finally those > streams are merged together with mplex into a MPEG system stream, where we > hope the video still "lines-up" with the audio. I suppose the problem if any will be in the capture phase. We have a way to timestamp (using the cpu clock) incoming video frames (do you use that ?), but e.g. for audio you cannot rely on the sample rate of the card. Depending on how unlucky you are, speed might be off by 1% or so -- even if smaller, the drift _is_ noticeable on long times. I am not sure on how you determine the timing of the streams, but I believe you should only use the audio sample rate for timing frames both during capture and playback (the latter I believe is implicit once you have interleaved frames ?) so that during playback you don't have drifts. Cheers Luigi -----------------------------+-------------------------------------- Luigi Rizzo | Dip. di Ingegneria dell'Informazione email: luigi@iet.unipi.it | Universita' di Pisa tel: +39-50-568533 | via Diotisalvi 2, 56126 PISA (Italy) fax: +39-50-568522 | http://www.iet.unipi.it/~luigi/ _____________________________|______________________________________