From owner-freebsd-hackers@FreeBSD.ORG Fri Mar 27 09:55:24 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1B26D77C for ; Fri, 27 Mar 2015 09:55:24 +0000 (UTC) Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8A5553DC for ; Fri, 27 Mar 2015 09:55:23 +0000 (UTC) Received: from scdbackup.webframe.org ([79.192.89.235]) by mail.gmx.com (mrgmx103) with ESMTPSA (Nemesis) id 0MRGvT-1Z3EsZ1puR-00Ue03; Fri, 27 Mar 2015 10:55:20 +0100 Date: Fri, 27 Mar 2015 10:55:02 +0100 From: "Thomas Schmitt" To: freebsd-hackers@freebsd.org Subject: Re: Seagate Archive HDD Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit References: In-Reply-To: Message-Id: <24923565183434755291@scdbackup.webframe.org> X-Provags-ID: V03:K0:DxbGL0xjlo0EqP6Kpmyfa5EsuiOilGHNciEHUcyaogXwmRPkwJs VNfmBNuW+npYApoDHNDQDr4DBN1SKnwS7t8EVHkgDzKNHcnVnF33fpFoLF5ahrpHd6UAzuF CMMWTbIvQw5uFeB+8cMMgJ8noqr8hOOY/Nb0qviFkE3r+SdPrQv6U2arQo5PZZFawgav9he mf+v343KF9bz9ulF8PnQQ== X-UI-Out-Filterresults: notjunk:1; Cc: wojtek@puchar.net X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Mar 2015 09:55:24 -0000 Hi, Wojciech Puchar wrote: > 2) replacement for rsync or other backup tools [...] > Written data can have geom_uzip&cd9660 (or ffs) compatible > layout to be readable by just anything. The visible hardware constraints resemble those of BD-RE or DVD-RAM media (but on a much larger scale, of course). So multi-session ISO 9660 + Rock Ridge would be well suitable. A write run consists of one sequential tree-and-data write and one small update write of the PVD (superblock). One would either use the raw device resp. a partition or write the ISO into some data file of a file system. My own ISO 9660 backup program xorriso would bail out at 4 TiB, because of some signed 32 bit integers in the APIs which it uses. One would have to organize the backup into several smaller areas. Or one could simply make a new base backup when the file with the ISO filesystem grows near to 4 TiB. Incremental ISO 9660 works by adding a new file tree and the whole data content of changed files. So the storage efficiency is worse than rsync, especially with large data files which change frequently. On the other hand ISO 9660 multi-session by xorriso allows to retrieve the older backup states by mount_cd9660 option -s. xorriso can tell the necessary block number. But the "cd9660" reader in FreeBSD is very restricted in respect to location of directory trees (not above 4 GiB - 1) and to file size (not more than 4 GiB - 1 bytes). The tree restriction is because the byte location of directory records is used as 32 bit inode number which then is used again to address the directory record. The file size restriction is because multiple data extents per file are not supported. NetBSD offers an escape path from the tree problem by its generous 64-bit inode numbers. Linux obviously does not use the inode number for addressing purposes and thus can afford to work with byte address divided by 32. This results in a 128 GiB window for trees, where inode numbers are unique. Not a totally general solution, but sufficient for any imaginable real purpose. You can put a lot of file names and meta data into 128 GiB. xorriso can retrieve what it wrote. So if the inconvenience of a random-access archive backup is acceptable, then 4 TiB filesystem size is supposed to be the only restriction. Have a nice day :) Thomas