Date: Tue, 30 May 2006 13:50:58 +0200 (CEST) From: Petr Holub <hopet@ics.muni.cz> To: FreeBSD-gnats-submit@FreeBSD.org Subject: bin/98134: MPEG2-TS/HDV support for fwcontrol Message-ID: <200605301150.k4UBowUG037849@evenstar.ics.muni.cz> Resent-Message-ID: <200605301200.k4UC0TDa041007@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 98134 >Category: bin >Synopsis: MPEG2-TS/HDV support for fwcontrol >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: update >Submitter-Id: current-users >Arrival-Date: Tue May 30 12:00:28 GMT 2006 >Closed-Date: >Last-Modified: >Originator: Petr Holub >Release: FreeBSD 6.1-RELEASE i386 >Organization: Masaryk University >Environment: System: FreeBSD evenstar.ics.muni.cz 6.1-RELEASE FreeBSD 6.1-RELEASE #0: Sun May 7 04:42:56 UTC 2006 root@opus.cse.buffalo.edu:/usr/obj/usr/src/sys/SMP i386 >Description: This code has been designed to allow reading MPEG2-TS and namely HDV streams from the FireWire interface. The patch has been discussed in freebsd-firewire list: http://docs.freebsd.org/cgi/getmsg.cgi?fetch=19187+0+archive/2005/freebsd-firewire/20050206.freebsd-firewire but perhaps due to low activity of the list, is has never been thoroughly reviewed and committed. After some cleanup proposed here: http://docs.freebsd.org/cgi/getmsg.cgi?fetch=1096154+0+archive/2005/freebsd-current/20051113.freebsd-current I'm sending the patch for 6-STABLE and 7-CURRENT branches attached to this PR. The patch is also available from here: http://sitola.fi.muni.cz/~hopet/HDV/fwcontrol-6.0.patch If there are some suggestions how to improve the code, I'd be happy to include them. I belive the code is pretty self-documented otherwise. For those interested in a patch for 5.x, it is available from here http://sitola.fi.muni.cz/~hopet/HDV/fwcontrol.patch >How-To-Repeat: >Fix: cd /usr/src/usr.sbin patch -p0 <fwcontrol-6.0.patch --- fwcontrol-6.0.patch begins here --- Index: fwcontrol/Makefile =================================================================== RCS file: /home/hopet/fwcontrol/fwcontrol-6.0/Makefile,v retrieving revision 1.1 retrieving revision 1.2 diff -u -r1.1 -r1.2 --- fwcontrol/Makefile 6 Sep 2005 20:29:40 -0000 1.1 +++ fwcontrol/Makefile 6 Sep 2005 20:40:31 -0000 1.2 @@ -1,7 +1,7 @@ # $FreeBSD: src/usr.sbin/fwcontrol/Makefile,v 1.4 2003/04/30 03:39:25 simokawa Exp $ PROG= fwcontrol -SRCS= fwcontrol.c fwcrom.c fwdv.c +SRCS= fwcontrol.c fwcrom.c fwdv.c fwmpegts.c MAN= fwcontrol.8 .PATH: ${.CURDIR}/../../sys/dev/firewire Index: fwcontrol/fwcontrol.8 =================================================================== RCS file: /home/hopet/fwcontrol/fwcontrol-6.0/fwcontrol.8,v retrieving revision 1.1 retrieving revision 1.2 diff -u -r1.1 -r1.2 --- fwcontrol/fwcontrol.8 6 Sep 2005 20:29:40 -0000 1.1 +++ fwcontrol/fwcontrol.8 6 Sep 2005 20:40:31 -0000 1.2 @@ -85,7 +85,7 @@ .Dv PRIORITY_BUDGET register on all supported nodes. .It Fl R Ar filename -Receive DV stream and dump it to a file. +Receive DV or MPEG TS stream and dump it to a file. Use Ctrl-C to stop the receiving. Some DV cameras seem not to send the stream if a bus manager exits. If you cannot get the stream, try the following commands: @@ -100,7 +100,14 @@ .Nm libdv in the .Fx -Ports Collection. +Ports Collection. Resulting MPEG TS stream can be played and sent over a +network using the VideoLAN +.Nm vlc +tool in the +.Fx +Ports Collection. The stream can be piped directly to +.Nm vlc, +see EXAMPLES. .It Fl S Ar filename Send a DV file as isochronous stream. .It Fl m Ar EUI64 | hostname @@ -117,7 +124,7 @@ .Pp .Dl "fwcontrol -R original.dv" .Pp -Receive stream. +Receive a DV stream with DV camera attached. .Pp .Dl "dd if=original.dv of=first.dv bs=120000 count=30" .Pp @@ -135,6 +142,34 @@ .Dq Li bs=120000 with .Dq Li bs=144000 . +.Pp +.Dl "fwcontrol -R file.m2t +.Pp +Receive an MPEG TS stream from a camera producing MPEG transport stream. This +has been tested with SONY HDR-FX1E camera that produces HD MPEG-2 stream at +25 Mbps bandwidth. +.Pp +To send the stream from the camera over the network using TCP (which supprisingly works better with vlc), you can use +.Dl "fwcontrol -R - | nc 192.168.10.11 9000 +with +.Nm netcat +from ports and to receive the stream, use +.Dl nc -l -p 9000 | vlc - +.Pp +To netcast via UDP, you need to use +.Nm buffer +program from ports, since vlc is not fast enough to read UDP packets from +buffers and thus it experiences dropouts when run directly. The sending side +can use +.Dl "fwcontrol -R - | nc 192.168.10.11 9000 +and to receive the stream, use +.Dl nc -l -u -p 9000 | buffer -s 10k -b 1000 -m 20m -p 5 | vlc - +.Pp + +.Pp +For more information on how to work with +.Nm vlc +see its docs. .Sh SEE ALSO .Xr firewire 4 , .Xr fwe 4 , @@ -150,3 +185,5 @@ .An Hidetoshi Shimokawa Aq simokawa@FreeBSD.org .Sh BUGS This utility is still under development and provided for debugging purposes. +Especially MPEG TS reception support is very rudimental and supports only +high-bandwidth MPEG-2 streams (fn field in CIP header equals 3). Index: fwcontrol/fwcontrol.c =================================================================== RCS file: /home/hopet/fwcontrol/fwcontrol-6.0/fwcontrol.c,v retrieving revision 1.1 retrieving revision 1.4 diff -u -r1.1 -r1.4 --- fwcontrol/fwcontrol.c 6 Sep 2005 20:29:40 -0000 1.1 +++ fwcontrol/fwcontrol.c 8 Nov 2005 14:41:48 -0000 1.4 @@ -46,6 +46,7 @@ #include <dev/firewire/firewire.h> #include <dev/firewire/iec13213.h> #include <dev/firewire/fwphyreg.h> +#include <dev/firewire/iec68113.h> #include <netinet/in.h> #include <fcntl.h> @@ -53,10 +54,12 @@ #include <err.h> #include <stdlib.h> #include <string.h> +#include <sysexits.h> #include <unistd.h> extern int dvrecv(int, char *, char, int); extern int dvsend(int, char *, char, int); +extern int mpegtsrecv(int, char *, char, int); int sysctl_set_int(const char *, int); @@ -77,10 +80,10 @@ "\t-t: read topology map\n" "\t-d: hex dump of configuration ROM\n" "\t-l: load and parse hex dump file of configuration ROM\n" - "\t-R: Receive DV stream\n" + "\t-R: Receive DV or MPEG TS stream\n" "\t-S: Send DV stream\n" "\t-m: set fwmem target\n"); - exit(0); + exit(EX_USAGE); } static void @@ -610,6 +613,57 @@ err(1, "sysctl %s failed.", name); } +static int (* +detect_recv_fn(int fd, char ich)) (int, char *, char, int) +{ + char *buf; + struct fw_isochreq isoreq; + struct fw_isobufreq bufreq; + int len; + u_int32_t *ptr; + struct ciphdr *ciph; + int (*retfn)(int, char *, char, int); + + bufreq.rx.nchunk = 8; + bufreq.rx.npacket = 16; + bufreq.rx.psize = 1024; + bufreq.tx.nchunk = 0; + bufreq.tx.npacket = 0; + bufreq.tx.psize = 0; + + if (ioctl(fd, FW_SSTBUF, &bufreq) < 0) { + err(1, "ioctl"); + } + + isoreq.ch = ich & 0x3f; + isoreq.tag = (ich >> 6) & 3; + + if( ioctl(fd, FW_SRSTREAM, &isoreq) < 0) + err(1, "ioctl"); + + buf = (char *)malloc(1024*16); + len = read(fd, buf, 1024*16); + ptr = (u_int32_t *) buf; + ciph = (struct ciphdr *)(ptr + 1); + + switch(ciph->fmt) { + case CIP_FMT_DVCR: + fprintf(stderr, "Detected DV format on input.\n"); + retfn = dvrecv; + break; + case CIP_FMT_MPEG: + fprintf(stderr, "Detected MPEG TS format on input.\n"); + retfn = mpegtsrecv; + break; + default: + errx(1, "Unsupported format for receiving: fmt=0x%x", ciph->fmt); + } + + free(buf); + return retfn; + +} + int main(int argc, char **argv) { @@ -618,6 +672,7 @@ int fd, tmp, ch, len=1024; struct fw_eui64 eui; struct eui64 target; + int (*recvfn)(int, char *, char, int); fd = -1; @@ -702,7 +757,11 @@ #define CHANNEL 63 case 'R': open_dev(&fd, devbase); - dvrecv(fd, optarg, TAG | CHANNEL, -1); + recvfn = detect_recv_fn(fd, TAG | CHANNEL); + close(fd); + fd = -1; + open_dev(&fd, devbase); + (*recvfn)(fd, optarg, TAG | CHANNEL, -1); break; case 'S': open_dev(&fd, devbase); Index: fwcontrol/fwdv.c =================================================================== RCS file: /home/hopet/fwcontrol/fwcontrol-6.0/fwdv.c,v retrieving revision 1.1 retrieving revision 1.2 diff -u -r1.1 -r1.2 --- fwcontrol/fwdv.c 6 Sep 2005 20:29:40 -0000 1.1 +++ fwcontrol/fwdv.c 6 Sep 2005 20:40:31 -0000 1.2 @@ -102,7 +102,12 @@ int nblocks[] = {250 /* NTSC */, 300 /* PAL */}; struct iovec wbuf[NPACKET_R]; - fd = open(filename, O_CREAT | O_WRONLY | O_TRUNC, 0660); + if(strcmp(filename, "-") == 0) { + fd = STDOUT_FILENO; + } + else { + fd = open(filename, O_CREAT | O_WRONLY | O_TRUNC, 0660); + } buf = (char *)malloc(RBUFSIZE); pad = (char *)malloc(DSIZE*MAXBLOCKS); memset(pad, 0xff, DSIZE*MAXBLOCKS); @@ -186,7 +191,7 @@ if (dv->sct == DV_SCT_HEADER && dv->dseq == 0) { if (system < 0) { system = ciph->fdf.dv.fs; - printf("%s\n", system_name[system]); + fprintf(stderr, "%s\n", system_name[system]); } /* Fix DSF bit */ @@ -237,7 +242,9 @@ if (vec > 0) writev(fd, wbuf, vec); } - close(fd); + if(fd != STDOUT_FILENO) { + close(fd); + } fprintf(stderr, "\n"); return 0; } @@ -320,7 +327,7 @@ if (len < 0) warn("read"); else - printf("\nend of file\n"); + fprintf(stderr, "\nend of file\n"); goto send_end; } tlen += len; Index: fwcontrol/fwmpegts.c =================================================================== RCS file: fwcontrol/fwmpegts.c diff -N fwcontrol/fwmpegts.c --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ fwcontrol/fwmpegts.c 8 Nov 2005 14:33:32 -0000 1.4 @@ -0,0 +1,263 @@ +/* + * Copyright (C) 2005 + * Petr Holub, Hidetoshi Shimokawa. All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. All advertising materials mentioning features or use of this software + * must display the following acknowledgement: + * + * This product includes software developed by Hidetoshi Shimokawa. + * + * 4. Neither the name of the author nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD: src/usr.sbin/fwcontrol/fwdv.c,v 1.5 2003/04/17 03:38:03 simokawa Exp $ + */ +#include <sys/param.h> +#include <sys/ioctl.h> +#include <sys/time.h> +#include <sys/types.h> +#include <sys/uio.h> + +#if __FreeBSD_version >= 500000 +#include <arpa/inet.h> +#endif + +#include <err.h> +#include <errno.h> +#include <unistd.h> +#include <fcntl.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> + +#include <dev/firewire/firewire.h> +#include <dev/firewire/iec68113.h> + +#define DEBUG 0 + +/***************************************************************************** + +MPEG-2 Transport Stream (MPEG TS) packet format according to IEC 61883: + +31 15 0 ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ -------- +| len |tag| channel | tcode | sy | ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1394 +| header_CRC | ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ -------- +|0|0| sid | dbs |fn | qpc |S|RSV| dbc | ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ CIP +|1|0| fmt | fdf | fdf/syt | ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ -------- +| reserved | cycle_count | cycle_offset | ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +| | N x +. . MPEG +. MPEG TS payload 188 bytes . +. . +| | ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ -------- +| data_CRC | ++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + +N.b. that CRCs are removed by firewire layer! + +The following fiels are fixed for IEEE-1394: +tag = 01b +tcode = 1010b +The length is payload length, i.e. includes CIP header and data size. + +The following fields are constant for MPEG TS: +sph = 1 (denoted as S in CIP header above) +dbs = 6 +fmt = (1<<5) +fdf = reserved +In the supported streams we also require +qpc = 0 +fn = 3 +and thus the payload is divided in 8 blocks as follows: + + +-----+-----+-----+-----+-----+-----+-----+-----+ + | db0 | db1 | db2 | db3 | db4 | db5 | db6 | db7 | + +-----+-----+-----+-----+-----+-----+-----+-----+ + +We have several cases of payload distribution based on stream +bandwidth (R): +1) R < 1.5 Mbps: any of db0..db7 may be payload, +2) 1.5 < R < 3 Mbps: db0/db1 or db2/db3 or db4/db5 or db6/db7 is payload, +3) 3 < R < 6 Mbps: db0/db1/db2/db3 or db4/db5/db6/db7 is payload, +4) R > 6 Mbps: all db0..db7 contain the payload. +Curently, only case (4) is supported in fwmpegts.c + +Each packet may contain N MPEG TS data blocks with timestamp header, +which are (4+188)B long. Experimentally, the N ranges from 0 through 3. + +*****************************************************************************/ + + +typedef uint8_t mpeg_ts_pld[188]; + +struct mpeg_pldt { +#if BYTE_ORDER == BIG_ENDIAN + uint32_t :7, + c_count:13, + c_offset:12; +#else /* BYTE_ORDER != BIG_ENDIAN */ + uint32_t c_offset:12, + c_count:13, + :7; +#endif /* BYTE_ORDER == BIG_ENDIAN */ + mpeg_ts_pld payload; +}; + + +#define NCHUNK 8 +#define PSIZE 596 +#define NPACKET_R 4096 +#define RBUFSIZE (PSIZE * NPACKET_R) + +int +mpegtsrecv(int d, char *filename, char ich, int count) +{ + struct ciphdr *ciph; + struct fw_isochreq isoreq; + struct fw_isobufreq bufreq; + struct fw_pkt *pkt; + struct mpeg_pldt *pld; + uint32_t *ptr; + int fd, k, len, m, pkt_size, startwr, tlen; + char *buf; + + startwr = 0; + + if (strcmp(filename, "-") == 0) + fd = STDOUT_FILENO; + else + fd = open(filename, O_CREAT | O_WRONLY | O_TRUNC, 0660); + buf = (char *)malloc(RBUFSIZE); + + bufreq.rx.nchunk = NCHUNK; + bufreq.rx.npacket = NPACKET_R; + bufreq.rx.psize = PSIZE; + bufreq.tx.nchunk = 0; + bufreq.tx.npacket = 0; + bufreq.tx.psize = 0; + if (ioctl(d, FW_SSTBUF, &bufreq) < 0) + err(1, "ioctl"); + + isoreq.ch = ich & 0x3f; + isoreq.tag = (ich >> 6) & 3; + + if (ioctl(d, FW_SRSTREAM, &isoreq) < 0) + err(1, "ioctl"); + + k = m = 0; + while (count <= 0 || k <= count) { + len = tlen = read(d, buf, RBUFSIZE); +#if DEBUG + fprintf(stderr, "Read %d bytes.\n", len); +#endif /* DEBUG */ + if (len < 0) { + if (errno == EAGAIN) { + fprintf(stderr, "(EAGAIN)\n"); + fflush(stderr); + if (len <= 0) + continue; + } else + err(1, "read failed"); + } + ptr = (uint32_t *) buf; + + do { + pkt = (struct fw_pkt *) ptr; +#if DEBUG + fprintf(stderr, "\nReading new packet.\n"); + fprintf(stderr, "%08x %08x %08x %08x\n", + htonl(ptr[0]), htonl(ptr[1]), + htonl(ptr[2]), htonl(ptr[3])); +#endif /* DEBUG */ + /* there is no CRC in the 1394 header */ + ciph = (struct ciphdr *)(ptr + 1); /* skip iso header */ + if (ciph->fmt != CIP_FMT_MPEG) + errx(1, "unknown format 0x%x", ciph->fmt); + if (ciph->fn != 3) { + errx(1, + "unsupported MPEG TS stream, fn=%d (only fn=3 is supported)", + ciph->fn); + } + ptr = (uint32_t *) (ciph + 1); /* skip cip header */ + + if (pkt->mode.stream.len <= sizeof(struct ciphdr)) { + /* no payload */ + /* tlen needs to be decremented before end of the loop */ + goto next; + } +#if DEBUG + else { + fprintf(stderr, + "Packet net payload length (IEEE1394 header): %d\n", + pkt->mode.stream.len - sizeof(struct ciphdr)); + fprintf(stderr, "Data block size (CIP header): %d [q], %d [B]\n", + ciph->len, ciph->len * 4); + fprintf(stderr, + "Data fraction number (CIP header): %d => DBC increments with %d\n", + ciph->fn, (1<<ciph->fn) ); + fprintf(stderr, "QCP (CIP header): %d\n", ciph->qpc ); + fprintf(stderr, "DBC counter (CIP header): %d\n", ciph->dbc ); + fprintf(stderr, "MPEG payload type size: %d\n", + sizeof(struct mpeg_pldt)); + } +#endif /* DEBUG */ + + /* This is a condition that needs to be satisfied to start + writing the data */ + if (ciph->dbc % (1<<ciph->fn) == 0) + startwr = 1; + /* Read out all the MPEG TS data blocks from current packet */ + for (pld = (struct mpeg_pldt *)ptr; + (int)pld < (int)((char *)ptr + pkt->mode.stream.len - + sizeof(struct ciphdr) ); + pld++) { + if (startwr == 1) + write(fd, pld->payload, sizeof(pld->payload)); + } + +next: + /* CRCs are removed from both header and trailer + so that only 4 bytes of 1394 header remains */ + pkt_size = pkt->mode.stream.len + 4; + ptr = (uint32_t *) ((int)pkt + pkt_size); + tlen -= pkt_size; + } while (tlen > 0); +#if DEBUG + fprintf(stderr, "\nReading a data from firewire.\n"); +#endif /* DEBUG */ + + } + if (fd != STDOUT_FILENO) + close(fd); + fprintf(stderr, "\n"); + return 0; +} + --- fwcontrol-6.0.patch ends here --- >Release-Note: >Audit-Trail: >Unformatted:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200605301150.k4UBowUG037849>