From owner-freebsd-hackers@FreeBSD.ORG  Tue Jul 29 09:00:26 2003
Return-Path: <owner-freebsd-hackers@FreeBSD.ORG>
Delivered-To: freebsd-hackers@freebsd.org
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 3411037B404
	for <hackers@freebsd.org>; Tue, 29 Jul 2003 09:00:26 -0700 (PDT)
Received: from webserver.get-linux.org
	(adsl-64-161-78-226.dsl.lsan03.pacbell.net [64.161.78.226])
	by mx1.FreeBSD.org (Postfix) with SMTP id 585F943FB1
	for <hackers@freebsd.org>; Tue, 29 Jul 2003 09:00:25 -0700 (PDT)
	(envelope-from root@webserver.get-linux.org)
Received: (qmail 4766 invoked by uid 0); 29 Jul 2003 16:00:48 -0000
Date: Tue, 29 Jul 2003 09:00:48 -0700
From: Joshua Oreman <oremanj@get-linux.org>
To: Bogdan TARU <bgd@icomag.de>
Message-ID: <20030729160048.GA4647@webserver>
References: <200307291524.h6TFO3J06998@revolt.poohsticks.org>
	<20030729172615.N13255-100000@fw.office.icom>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20030729172615.N13255-100000@fw.office.icom>
User-Agent: Mutt/1.4.1i
cc: hackers@freebsd.org
Subject: Re: file size different from ls to du
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.1
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
	<freebsd-hackers.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
	<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
	<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 29 Jul 2003 16:00:26 -0000

On Tue, Jul 29, 2003 at 05:27:14PM +0200 or thereabouts, Bogdan TARU wrote:
> 
> 	Hi Drew,
> 
>  I have tried to create some files of myself, with 'spaces' in them
> (holes?), but they don't act like this. So could you please explain what
> 'sparse' means, and the 'trick' to create them?

Basically, seek into the file and write a zero byte. The file will logically
be as long as the offset you seeked to, but physically only the zero byte
you wrote is stored on the disk. Everything else is "empty", so it's just
"assumed" to be zero bytes when read.

You have two main ways to create sparse files:

1) To "sparsify" a file: Install port sysutils/fileutils and use
   $ gcp --sparse=always oldfile sparsefile
2) To make a sparse file of X length (mainly just for "fun" purposes), use
   the C program below.

Usually programs will manage their own sparse files; it's something hard
to do at the shell.

Use these commands to create that C program I mentioned:

cat > sparse.c << "EOF"

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/types.h>
#include <fcntl.h>

#define KB *1024
#define MB *1024 *1024

int main (int argc, char **argv) {
	int offset = 10 MB, fd;
	char ch, file[1024];

	while ((ch = getopt (argc, argv, "s:")) != EOF) {
		if (ch == 's') {
			offset = atoi (optarg) KB;
		}
	}

	argc -= optind; argv += optind;

	if (argc) {
		strcpy (file, argv[0]);
	} else {
		strcpy (file, "sparse.out");
	}

	/* Open the file */
	if ((fd = open (file, O_WRONLY | O_CREAT | O_TRUNC, 0666)) == -1) {
		perror ("open");
		exit (1);
	}

	/* Seek `offset' bytes into it */
	lseek (fd, offset, SEEK_SET);

	/* Write a zero */
	write (fd, "\0", 1);

	/* Close the file */
	close (fd);

	return 0;
}

EOF
gcc -o sparse sparse.c
./sparse -s <how-big-in-KB> <file-to-make>

Replace <how-big-in-KB> with something like 10000 for an (about) 10 MB file, and <file-to-make>
with the name of the sparse file to make (it will be overwritten if it already exists).

Example:
$ ./sparse -s 10000 sparsefile
$ ls -alsh sparsefile
4.0k -rw-r--r--    1 oremanj  users        9.8M Jul 29 08:55 sparsefile
^^^^ physical size                         ^^^^ logical size
$

-- Josh

> 
>  Thanks,
>  bogdan