Date: Wed, 26 Mar 2003 13:37:46 -0500 From: Gunther Schadow <gunther@aurora.regenstrief.org> To: freebsd-questions@freebsd.org Subject: streaming tape problem ... Message-ID: <3E81F37A.5020401@aurora.regenstrief.org>
next in thread | raw e-mail | index | archive | help
Hi, does anyone of you know of some smart buffering dd-like program that can use asynchronous I/O (or some other smart trick) to maintain a constant bit-rate writing to a SCSI tape? I have a TK88 DLT IV on my (one and only) SCSI bus now and I can't make dump(8) or even something like "dd if=/dev/rda0f" maintain enough output to the tape for the highest density option. The only way I can stream backups now is using dump with -b 128 after turning off the compression option on the drive. My question is whether dump(8), tar(1) and perhaps dd(1) could be modified to use asynchronous IO and then maintain a buffer just as the typical CDROM writers do it to prevent buffer underrun. If that is not feasible, what would a program such as dd have to do to maintain a constant output rate? I am thinking that if dump and tar can't be modified to use async IO, then may be one could modify dd and use it as a smart buffer like this: dump -au0 -b 128 -f - /usr \ |dd of=/dev/nsa0 bs=128k sb=1280k where the sb option would say that a 1280k buffer were to be used, filled to the highwater mark before output would begin. Then output would be done using aio_write such that the upstream programs such as dump would not be blocked unless the buffer were full. On my VAX6460 with the TU81 9-track tape drive I did a similar trick like this: tar cvb 20 -f - /usr5/gcc-2.7.2 \ |dd ibs=2048000 of=/dev/nrmt0h obs=10240 that seemed like it would help. The idea here was that the large ibs would get filled before dd starts outputting smaller pieces. On the multi CPU machine with the right OS and fingers crossed one could expect that the pipe would not be blocked so much (some OS buffering in the pipe.) But still it may be more wishful thinking. I'm afraid that the dd still blocks the upstream pipe as long as the buffer is written to tape and then ends up with an empty buffer. I just found another interesting article: http://www.dmst.aueb.gr/dds/pubs/trade/1999-login-dd/html/dd.html this auther suggests to put an additional dd process into the pipe for buffering: tar cvb 20 -f - /usr5/gcc-2.7.2 \ |dd obs=512 \ |dd obs=512 of=/dev/nrmt0h I will try that as soon as my backup finishes. It is streaming now, just not in compressed mode. regards -Gunther -- Gunther Schadow, M.D., Ph.D. gschadow@regenstrief.org Medical Information Scientist Regenstrief Institute for Health Care Adjunct Assistant Professor Indiana University School of Medicine tel:1(317)630-7960 http://aurora.regenstrief.org
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3E81F37A.5020401>