From owner-freebsd-questions Sat Jan 6 6:30:35 2001 From owner-freebsd-questions@FreeBSD.ORG Sat Jan 6 06:30:18 2001 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from home.smithconcepts.com (ubr-95.237.221.indiatlantic.cfl.rr.com [24.95.237.221]) by hub.freebsd.org (Postfix) with ESMTP id 915DF37B400 for ; Sat, 6 Jan 2001 06:30:17 -0800 (PST) Received: from ieee.org (IDENT:bjsmith@bitman.oviedo.smithconcepts.com [172.24.24.192]) by home.smithconcepts.com (8.9.3/8.9.3) with ESMTP id JAA14055; Sat, 6 Jan 2001 09:25:14 -0500 Sender: bjsmith@smithconcepts.com Message-ID: <3A572D15.8B3B7F0F@ieee.org> Date: Sat, 06 Jan 2001 09:35:01 -0500 From: "Bryan J. Smith" Reply-To: b.j.smith@ieee.org, thebs@theseus.com Organization: SmithConcepts/Personal X-Mailer: Mozilla 4.75 [en] (X11; U; Linux 2.2.17-0.10 i686) X-Accept-Language: en MIME-Version: 1.0 To: david2000la@yahoo.com Cc: david@nns.ne.jp, freebsd-questions@freebsd.org Subject: RE: How about a free TCSH shell to backup to CDR? -- WAS: CDR Backup Copies of Software for Windows References: Content-Type: multipart/mixed; boundary="------------96831CFD466F3AD9AB8F1BC0" Sender: owner-freebsd-questions@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG This is a multi-part message in MIME format. --------------96831CFD466F3AD9AB8F1BC0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit RE: How about a free TCSH shell to backup to CDR? -- WAS: CDR Backup Copies of Software for Windows CDR Software wrote: > Hello. I am selling CDR backup copies of software for Windows. > I charge only $10/CD and I have a huge collection including all > of the very latest programs. If you are interested, please > e-mail me and I'll send you a list. Hi David (assuming you have not masqueraded your E-mail as someone else)! Appreciate the SPAM! This is a FreeBSD (UNIX) list and we do not believe in piracy. Futhermore, our software is free and we have no use for pirated Windows software which won't run on our computers. Now if you are intersted in backup software to CDRs, I wrote this free TCSH shell script (attached) that let's you backup to CD (including per-file compression). I cover the features of the script below ... --- I recently (2000Dec15) created this script after a long user group discussion on backups, file integrity and recovery of corrupted backups, etc... For example, while Tar is quite recoverable when corrupted (where the corrupted section only affects files in that section), a compressed tarball is not (the whole archive becomes unrecoverable). Likewise, PKZip has this same issue. We even had one user who was so new to UNIX, he PKZiped his files and then burned that to a CD (talk about putting your data at risk, let alone it would be a pain to restore!). Since ISO9660 is an archive format in itself, you don't need to put it in a Tar, Cpio or even PKZip archive format. You just need to get the files in the ISO9660 master ISO file. Now we want to maximize storage, so we want to compress files individually. And we want to minimize temporary space, so we want to compress as we copy to the temporary area where we will master our CDs from. The end result is a burnable ISO9660 CD (with RockRidge & Joliet extensions + Apple/Mac if you use mkhybrid) that is filled with files that can be directly accessed as normal, with the exception that they are compressed for maximum capacity. I currently backup almost 2GB into a 650MB CD using this utility. I wanted a nice complement for CD backup that I have in afio for tape backup. It works on Linux, Solaris, FreeBSD and even Cygwin (for NT systems). Hope everyone enjoy's this Tcsh script. Review of features: - Per file compression! - Uses GZip by default - User selectable (see CONSTANTS) - Individually selectable for text and non-text (since GZip is faster/better for text, BZip2 better, although slower for most non-text) - Minimizes disk usage by compressing as it finds files -- but doesn't compress the existing files -- it compresses them in the temporary archive space - Identification of pre-compressed files (so it doesn't waste time recompressing) in 2 ways: - Via "file" string (catches anything compressed with Compress/Gzip/BZip2, including Dia and files not easily identifyable) - Via filename "extension" - Both are extensible (see CONSTANTS) - Takes one or more relative and/or absolute paths - You can mix'n match relative and absolute paths - Builds a hierarchy with the directory name in root (unlike stock mkisofs/mkhybrid, where the source directory is stripped -- I hate that!) - Can take the current directory (via ".") as a parameter - It checks _all_parameters_ to make sure they are _valid_ relative/absolute directories (yet another "niceity" over stock mkisofs/mkhybrid) - Does not cross filesystem boundaries (for obvious sanity reasons) - Symlinks are copied as symlinks and special files are copied over as special files (i.e. no issues/worries about dealing with them) - Supports both mkisofs (UNIX/Windows) and mkhybrid (+Mac) - If "iso_mac" set (see VARIABLES), myhybrid is used - Can specify alternative, mkisofs/mkhybrid compatible binary (see CONSTANTS) - Can also reset UNIX permissions so all files are world readable (default is to preserve UID, GID and bits -- see VARIABLES) - You can set both the output and temporary directories (see VARIABLES) - Great for setting an output path to a Samba shared directory where you can copy the .iso file over to your Windows system with a CD burner (and pull up CD Creator and tell it to burn a CD from an .iso image) - Does not assume binaries are in path, or are set. - You can override just about any binary location in CONSTANTS. - Only need tcsh and which, it either finds, or you can set, anything else - Minimal console clutter (nice simple ticks when archiving), but maximum logging to files Limitations (currently): - Parameters must be directories - A file cannot be a command-line parameter - But you can still send "." for the current directory - Parameters cannot have spaces in name - Only the actually parameters passed on the command-line - Spaces in their subdirectories and filenames are fine - Again, only what you pass on the command-line cannot have spaces (not even if you use "" or \ -- it's an internal variable issue I haven't accomodated yet), but any dirs/files under them are fine - Symlinks are blindly copied over, two issues: - It does not check that it points to anything useful in the CD image (especilly for absolute paths) [ I will probably never address this, no one does ] - Even if it points to a relative path that is on the CD, if that file is now compressed, it will not point to the new filename [ I'll probably fix this soon ] - Cannot have both a file and a compressed file in the same directory - Otherwise, only the first file will get copied and used (e.g., if file and file.gz both exist, only the first one seen by find will get backed up) -- TheBS -- Bryan "TheBS" Smith, Engineer CONTACT INFO *********************************************************** Chat: thebs413 @ AOL/MSN/Yahoo (see http://Everybuddy.com) Email: mailto:thebs@smithconcepts.com,thebs@theseus.com --------------96831CFD466F3AD9AB8F1BC0 Content-Type: text/plain; charset=us-ascii; name="back2cd.tcsh" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="back2cd.tcsh" #!/bin/tcsh -f # # back2cd.tcsh # $Id: back2cd.tcsh,v 1.4 2000/12/15 16:12:48 bjsmith Exp $ # # Make a CD-R ISO9660 Backup CD Image with UNIX/Windows/Mac extensions # compressing individual files in the image for maximum capacity. # # Written by Bryan "TheBS" Smith (mailto:b.j.smith@ieee.org) # Licensed under the GNU Public License (GPL) version 2.1 # All rights reserved to the Free Software Foundation ########################################################## # VARIABLES ########################################################## # Directories #========================================================= # Temporary Directory (need upto 700MB + biggest, uncompressed file) #--------------------------------------------------------- set iso_tmp=/tmp # ISO Output Directory (need upto 700MB) #--------------------------------------------------------- set iso_dir=/home # Extensions/Options #========================================================= # Set iso_mac to use Mac long filenames (requires Mkhybrid) #--------------------------------------------------------- #DISABLED#set iso_mac=set # Set iso_rrr to reset UNIX permissions (so all files are readable by everyone) #--------------------------------------------------------- #DISABLED#set iso_rrr=set # Information (for ISO9660 namespace) #========================================================= # Publisher #--------------------------------------------------------- set iso_pub="my name, company, address, phone" # Preparer #--------------------------------------------------------- set iso_pre="`whoami`@`uname -n`" ########################################################## # CONSTANTS ########################################################## # Binaries #========================================================= # Mkisofs/Mkhybrid (defaults to mkisofs in path, mkhybrid if "iso_mac" set) #--------------------------------------------------------- #OPTION#set bin_mkiso=/usr/bin/mkisofs #OPTION#set bin_mkiso=/usr/bin/mkhybrid # Compression for text files (defaults to GZip in path) #--------------------------------------------------------- #OPTION#set bin_cmptxt=/usr/gzip #OPTION#set bin_cmptxt=/usr/bin/bzip2 # Compression for non-text files (defaults to GZip in path) #--------------------------------------------------------- #OPTION#set bin_cmpbin=/usr/bin/gzip #OPTION#set bin_cmpbin=/usr/bin/bzip2 # Other Binaries (defaults to binary in path) #--------------------------------------------------------- #OPTION#set bin_cpio=/bin/cpio #OPTION#set bin_find=/usr/bin/find #OPTION#set bin_file=/usr/bin/file #OPTION#set bin_grep=/bin/grep #OPTION#set bin_mkdir=/bin/mkdir #OPTION#set bin_rm=/bin/rm # Files that should NOT be compressed #========================================================= # Strings reported by "file" (catches files compressed by Compress, GZip, BZip2, compressed XML apps) #--------------------------------------------------------- set noc_typ="compress" # By filename extension #--------------------------------------------------------- set noc_ext="arc lha lhz pak rpm zip zoo" ########################################################## # SCRIPT ########################################################## # Help #========================================================= if ( $# < 2 || $1 == "--help" || $1 == "-?" || $1 == "-h" || $1 == "-H" ) then echo "" echo "Syntax:" echo " back2cd