Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 10 Dec 1997 18:20:48 -0500
From:      Randall Hopper <rhh@ct.picker.com>
To:        Robert Eckardt <roberte@MEP.Ruhr-Uni-Bochum.de>
Cc:        multimedia@FreeBSD.ORG
Subject:   Re: fxtv-0.46 to raw device
Message-ID:  <19971210182048.27146@ct.picker.com>
In-Reply-To: <199712022329.AAA26946@ghost.mep.ruhr-uni-bochum.de>; from Robert Eckardt on Wed, Dec 03, 1997 at 12:29:58AM %2B0100
References:  <199712022329.AAA26946@ghost.mep.ruhr-uni-bochum.de>

next in thread | previous in thread | raw e-mail | index | archive | help
Robert Eckardt:
 |Hi (Randall ;-)

Hi.  Sorry its been a week since your post.  I've been busy and haven't
gotten back to my filtered multimedia list folder until today.

 |using a Haupauge video card I tried capturing a minute or so to disk.
 |(FreeBSD-2.2.2, fxtv-0.46, bt848.971105,
 | P133, ASUS TVP4, 64MB, HDs: WD (IDE) and IBM (IDE+SCSI/ncr53c810),
 | Hauppauge WinCast/TV, Temic PAL tuner + SoundBlaster 16 Virge PnP)
 |I used MPEG output mode.
 |
 |However, the captured sequence has a lot of "hickups".
 |(several ppm-files are 5- to 48-fold hard links)
 |(This is with PAL/BDGHI, 25fps, 256x192)
 |(ideal would be 384x288 @ 25fps)
 |Even reducing the rate to 15 fps didn't get rid of them.
 |
 |I tried spreading the raw files over 3 disks yielding only a
 |small improvement (5- to 11-fold hard links).

Back when I was on my P100 with a 540Meg Connor IDE, I found I had to crank
the frame rate down to around 20 and the size down to around 240x180 to get
relatively fluid video on 1 disk (e.g. ftp://rah.star-gate.com/pub/Knight.mps).
This was on:

    P100,ASUS T2P4,32Meg,Conner 540M IDE, Hauppauge Wincast/TV,FreeBSD-2.2.1

Now on a 233MMX with a faster 3.5Gig disk (still on 2.2.1), I get much
better results.  320x240 or so in 16bpp w/ frame rate in the high 20s isn't
unreasonable.  Looking at a recent 76 second 320x240 capture (2300 frames),
I see link counts of 1, 2, some 3s, and 8 4s -- nothing higher. I'm
surprised that you're not getting better rates given your system config.

The process of shuffling the frames to disk is really pretty mindless.  It
really boils down to whether the disk can keep up with the app, OS disk
drivers, buffer cache, etc..  When the OS disk buffers fill, the app blocks
and frames start getting skipped.  During conversion, this results in all
those multiply linked files being created by fxtv.  It's buffering the
image stream up to 30fps, since mpeg_encode wants a video stream at a
constant FPS.

One thing to check.  Until I or someone has time to dig in there and add
YUV support, do your capturing in 16bpp mode.  If you're doing this in
24bpp this could explain some of the lag you're seeing.

Once we add YUV, this should cut down on the amount of data we need to
write by about 25% (according to mail from Amancio) over writing 16bpp RGB.

 |Therefore I tried to write to a raw partition (rwd1s2).
 |But, after "Converting RAW to PPM...", which produced indeed a lot
 |of ppm-files, the next step "Failed reading raw image file".
 |(trying to convert sound ?)
 |
 |Even after looking at the source, I still dont't understand why. ???

Well, I hacked in raw device support at one point for my testing.  I dumped
it though when I found out I didn't get anywhere near as good a results as
when writing to a UFS file system.  So I quit implementing it, disabled raw
write support, and put in a dialog to tell the user this wasn't supported.
The only problem is, during my testing, I commented out this dialog for
other reasons and never uncommented it.  :-) So now it lets you capture to
the device anyway, only to error out later trying to read.

In particular, you'll notice in vidsav_dlg.c:UnlinkRawFiles (called to
cleanup after we're done with the raw capture datafiles), it always tries
to unlink them.  This is one of the things that needs tweaked to support
raw device writes again.  It doesn't check unlink's return so it doesn't
error out here.  But you don't want to run fxtv as root, or you'll have to
re-mknod your device(s).  Also, in rawvideo.c:TVRAWVIDEOImageRead, you'll see
that its depending on the read() to return 0 to indicate its reached
end-of-stream.  To support raw devices, there needs to be an
"end-of-stream" record of sorts written instead, and this used to detect
end-of-stream.  There may be some other changes needed to support raw
devices, but this is what comes to mind right now.

I suspect the reason that performance writing to raw disk devices was less
than steller for me is because raw devices aren't buffered and the app
blocks a good bit longer writing records to these files.  Whereas when
writing to file systems, the OS writes occur in parallel with the app,
while the app is back getting the next frame ready to go.

So basically writing to a raw device isn't supported right now.  I don't
think it'd take much to get it there again though as fxtv does use 4K
writes exclusively for dumping raw video/audio now.

If you can think of anything we can do to speed up disk transfers and
encoding, I would be interested.  Mods to capture and save in YUV, now that
Amancio's added it to the driver, should help some.

 |Another thing I noticed:
 |dd bs=4k count=10000 if=/dev/zero of=/dev/rwd1s2  gives me
 |6307378 bytes/sec, running smoothly.

Pretty good raw transfer rate you have there.  That's about what I get on my
relatively-new WD 3.4Meg IDE (6299090 bytes/sec).  This is without the IDE
DMA patches (i.e. stock 2.2.1 drivers).

 |Using fxtv on that device lets the disk heads scatter (swap ?)
 |and Optimize only gives 256x192@24fps with no CPU- or disk-intensive
 |processes running.

Well, at 16bpp, I guess that's about 2.4Meg/s.  3.5Meg/s for 24bpp.  Don't
know about the VX chipset on the TVP4, but I recall that it doesn't do as
well on a heavily loaded bus as the HX.  I wonder if the PCI video transfer
from the Wincast to system memory could be starving out the transfers from
memory to the MB PCI IDE controller.

Of course, it could be any number of other hardware and software issues,
only some of which I might be vaguely aware of.  Best I can suggest is to
play around with system settings, driver revisions, hardware configuration,
and capture parameters and see what makes the most difference.  

BTW, I suspect the DMA IDE support in -current will speed this raw
video/audio stream writing somewhat, but I have yet to find time to upgrade
and try it.  That's another thing you might investigate.

 |Finally, is it planned or is s.o. already working on converting
 |a RAW-file directly (e.g., a stream of piped commands) to an
 |MPEG-stream ?
 |Wouldn't this be the canonical approach on a Unix system ?
 |
 |It seems that the intermediate steps use much more disk space
 |then the raw file (ca. 200MB for a min., but my partition with
 |780MB free runs out of space already for 1 1/2 min.).

I agree--the intermediate files do take a lot of space and (almost as
important) time to generate.

Up until a few minutes ago, I didn't think that mpeg_encode supported
streaming into it via pipe.  I didn't happen on any examples of this at the
time.

But checking the man page, I see there are a few references to input from
stdin though it doesn't say anything about the stream format (frame
delimeters, etc.).  I'll have to look into this to see how it works, but
I'm glad you suggested it.  This'd be a good thing to add in with YUV
support.  Consider it on my list.

Thanks,

Randall





Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?19971210182048.27146>