From owner-freebsd-questions@freebsd.org Thu Feb 9 23:55:32 2017 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C9B12CD8A5E for ; Thu, 9 Feb 2017 23:55:32 +0000 (UTC) (envelope-from freebsd@edvax.de) Received: from mailrelay12.qsc.de (mailrelay12.qsc.de [212.99.163.153]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.antispameurope.com", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 50F0D2B for ; Thu, 9 Feb 2017 23:55:32 +0000 (UTC) (envelope-from freebsd@edvax.de) Received: from mx01.qsc.de ([213.148.129.14]) by mailrelay12.qsc.de; Fri, 10 Feb 2017 00:58:47 +0100 Received: from r56.edvax.de (port-92-195-22-222.dynamic.qsc.de [92.195.22.222]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx01.qsc.de (Postfix) with ESMTPS id E69553CBF9; Fri, 10 Feb 2017 00:55:28 +0100 (CET) Received: from r56.edvax.de (localhost [127.0.0.1]) by r56.edvax.de (8.14.5/8.14.5) with SMTP id v19NtS77002085; Fri, 10 Feb 2017 00:55:28 +0100 (CET) (envelope-from freebsd@edvax.de) Date: Fri, 10 Feb 2017 00:55:28 +0100 From: Polytropon To: freebsd-questions@freebsd.org Cc: David Christensen Subject: Re: FreeBSD 11 and 4+ GB files on optical disc Message-Id: <20170210005528.17bd100f.freebsd@edvax.de> In-Reply-To: <2f1806ae-cfd2-704c-298c-5ad93d59e628@holgerdanske.com> References: <2f1806ae-cfd2-704c-298c-5ad93d59e628@holgerdanske.com> Reply-To: Polytropon Organization: EDVAX X-Mailer: Sylpheed 3.1.1 (GTK+ 2.24.5; i386-portbld-freebsd8.2) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-cloud-security-sender: freebsd@edvax.de X-cloud-security-recipient: freebsd-questions@freebsd.org X-cloud-security-Virusscan: CLEAN X-cloud-security-disclaimer: This E-Mail was scanned by E-Mailservice on mailrelay12.qsc.de with 9ADC46A427B X-cloud-security-connect: mx01.qsc.de[213.148.129.14], TLS=1, IP=213.148.129.14 X-cloud-security: scantime:.2351 X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Feb 2017 23:55:32 -0000 On Thu, 9 Feb 2017 15:04:13 -0800, David Christensen wrote: > I have a computer: > > dpchrist@freebsd:/usr/home/dpchrist $ freebsd-version > 11.0-RELEASE-p7 > > dpchrist@freebsd:/usr/home/dpchrist $ uname -a > FreeBSD freebsd 11.0-RELEASE-p1 FreeBSD 11.0-RELEASE-p1 #0 r306420: Thu > Sep 29 03:40:55 UTC 2016 > root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC i386 > > > I periodically create archive files on Debian 7, encrypt the archive, > create checksum files, burn the archive and checksum files to optical > disc, and then verify the contents of the disc. So I can assume you're using ISO-9660 as the file system for the optical disc - a very important aspect you never mentioned. :-) > The checksums verify okay on Debian 7. > > > But when I put the optical disc into the subject FreeBSD computer and > browse to it using Xfce and Thunar, the first thing I notice is that the > archive file is listed twice: > > dpchrist@freebsd:/media/HOLGERDANSKE_COM $ ll > total 8388607 > drwxr-xr-x 1 root wheel - 2048 Feb 9 11:05 ./ > drwxr-xr-x 3 root wheel uarch 5 Feb 9 14:34 ../ > -rw-r--r-- 1 root wheel - 4294965248 Feb 9 08:52 201701.tar.cpt > -rw-r--r-- 1 root wheel - 4294965248 Feb 9 08:52 201701.tar.cpt > -rw-r--r-- 1 root wheel - 49 Feb 9 08:52 201701.tar.cpt.md5 > -rw-r--r-- 1 root wheel - 81 Feb 9 08:52 > 201701.tar.cpt.sha256 > > > When I attempt to verify the checksums, they fail: > > dpchrist@freebsd:/media/HOLGERDANSKE_COM $ cat 201701.tar.cpt.md5 > dd2dfc838c03a1819efceab530a6c7f8 *201701.tar.cpt > > dpchrist@freebsd:/media/HOLGERDANSKE_COM $ md5 -c > dd2dfc838c03a1819efceab530a6c7f8 201701.tar.cpt > MD5 (201701.tar.cpt) = ac49e95889f44adbdaa2f68f4d7bb96e [ Failed ] > > dpchrist@freebsd:/media/HOLGERDANSKE_COM $ cat 201701.tar.cpt.sha256 > 87b0ba7ed7e706f55f571ec73daad0a2ab9820de9db8e0c889ba8541b5b8f40e > *201701.tar.cpt > > dpchrist@freebsd:/media/HOLGERDANSKE_COM $ sha256 -c > 87b0ba7ed7e706f55f571ec73daad0a2ab9820de9db8e0c889ba8541b5b8f40e > 201701.tar.cpt > SHA256 (201701.tar.cpt) = > 10ebce65e89144fe6161b96a43920cf3437a18b34f1e92c7941343fa9a950ad3 [ Failed ] > > > I suspect that FreeBSD is having problems because the archive file is > larger than 4 GB (?). Well, if you accurately consider the file size conversion to kB, MB and GB (for example 4294965248/1024/1024/1024 = 3), you mught or might not be bitten by a file size limitation. Check what the file system specification says: All numbers in ISO 9660 file systems except the single byte value used for the GMT offset are unsigned numbers. As the length of a file's extent on disk is stored in a 32 bit value, it allows for a maximum length of just over 4.2 GB (more precisely, one byte less than 4 GiB). [...] Based on this, it is often assumed that a file on an ISO 9660 formatted disc cannot be larger than 232-1 in size, as the file's size is stored in an unsigned 32 bit value, for which 232-1 is the maximum. It is, however, possible to circumvent this limitation by using the multi-extent (fragmentation) feature of ISO 9660 Level 3 to create ISO 9660 filesystems and single files up to 8 TB. With this, files larger than 4 GB can be split up into multiple extents (sequential series of sectors), each not exceeding the 4 GB limit. For example, the free software such as InfraRecorder, ImgBurn and mkisofs as well as Roxio Toast are able to create ISO 9660 filesystems that use multi-extent files to store files larger than 4 GB on appropriate media such as recordable DVDs. https://en.wikipedia.org/wiki/ISO_9660#The_2.2F4_GiB_file_size_limit So what you're seeing (file appearing twice) exactly matches the assumption that mkisofs is creating a "split entry" for a source file that the ISO-9660 file system cannot contain due to a size limitation. > Any comments or suggestions? Don't use ISO-9660. Use tar as a file system (or better, instead of a file system). You can write it to the DVD like this: % growisofs -dvd-compat -Z /dev/dvd=2017.01.tar And read (restore) it like this: % tar xvf /dev/dvd The advantage: No size limitation imposed by ISO-9660. Additional security: "There are no files on the DVD!" ;-) -- Polytropon Magdeburg, Germany Happy FreeBSD user since 4.0 Andra moi ennepe, Mousa, ...