From owner-svn-src-user@FreeBSD.ORG Sun Sep 28 18:34:22 2014 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EB3603E4; Sun, 28 Sep 2014 18:34:21 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D7D24762; Sun, 28 Sep 2014 18:34:21 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.9/8.14.9) with ESMTP id s8SIYLCh091092; Sun, 28 Sep 2014 18:34:21 GMT (envelope-from marcel@FreeBSD.org) Received: (from marcel@localhost) by svn.freebsd.org (8.14.9/8.14.9/Submit) id s8SIYLcm091090; Sun, 28 Sep 2014 18:34:21 GMT (envelope-from marcel@FreeBSD.org) Message-Id: <201409281834.s8SIYLcm091090@svn.freebsd.org> X-Authentication-Warning: svn.freebsd.org: marcel set sender to marcel@FreeBSD.org using -f From: Marcel Moolenaar Date: Sun, 28 Sep 2014 18:34:21 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org Subject: svn commit: r272265 - user/marcel/mkimg X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Sep 2014 18:34:22 -0000 Author: marcel Date: Sun Sep 28 18:34:20 2014 New Revision: 272265 URL: http://svnweb.freebsd.org/changeset/base/272265 Log: Implement image_copyin_mapped(): 1. If we can't seek, then let image_copyin_stream() deal with it. 2. Use SEEK_HOLE and SEEK_DATA to iterate over the regions of a sparse file. 3. Round or truncate to the sector size based on whether it's a hold or data. We truncate for holes and round for data. This to make sure to handle sparse files correctly when the sector size is larger than the file system's block/fragment size. Since we detect runts of zeroes for mmap'd data (to handle streams), we correct ourselves if such is applicable. We never mishandle data, which is the most important. I need to check with a file system that doesn't support sparse files what SEEK_HOLE and SEEK_DATA return. I'm thinking -1, which is what I don't handle yet. It's easy to do, because everything should be treated as data and we'll find the holes the hard way. While here: set DEBUG_FLAGS unconditionally to make sure I can always debug a core. Modified: user/marcel/mkimg/Makefile user/marcel/mkimg/image.c Modified: user/marcel/mkimg/Makefile ============================================================================== --- user/marcel/mkimg/Makefile Sun Sep 28 17:53:10 2014 (r272264) +++ user/marcel/mkimg/Makefile Sun Sep 28 18:34:20 2014 (r272265) @@ -10,6 +10,8 @@ MKIMG_VERSION=20140927 CFLAGS+=-DMKIMG_VERSION=${MKIMG_VERSION} CFLAGS+=-DSPARSE_WRITE +DEBUG_FLAGS=-O0 -gdwarf-2 + # List of formats to support SRCS+= \ qcow.c \ Modified: user/marcel/mkimg/image.c ============================================================================== --- user/marcel/mkimg/image.c Sun Sep 28 17:53:10 2014 (r272264) +++ user/marcel/mkimg/image.c Sun Sep 28 18:34:20 2014 (r272265) @@ -307,8 +307,88 @@ image_copyin_stream(lba_t blk, int fd, u static int image_copyin_mapped(lba_t blk, int fd, uint64_t *sizep) { + off_t cur, data, end, hole, pos; + void *buf; + uint64_t bytesize; + size_t sz; + int error; + + /* + * We'd like to know the size of the file and we must + * be able to seek in order to mmap(2). If this isn't + * possible, then treat the file as a stream/pipe. + */ + end = lseek(fd, 0L, SEEK_END); + if (end == -1L) + return (image_copyin_stream(blk, fd, sizep)); + + /* + * We need the file opened for the duration and our + * caller is going to close the file. Make a dup(2) + * so that control the faith of the descriptor. + */ + fd = dup(fd); + if (fd == -1) + return (errno); - return (image_copyin_stream(blk, fd, sizep)); + bytesize = 0; + cur = pos = 0; + error = 0; + while (!error && cur < end) { + hole = lseek(fd, cur, SEEK_HOLE); + data = lseek(fd, cur, SEEK_DATA); + + fprintf(stderr, "XXX: %s: cur=%jd, pos=%jd, hole=%jd, " + "data=%jd\n", __func__, (intmax_t)cur, (intmax_t)pos, + (intmax_t)hole, (intmax_t)data); + + if (cur == hole && data > hole) { + hole = pos; + pos = data & ~(secsz - 1); + + fprintf(stderr, "GAP %jd-%jd\n", + (intmax_t)hole, (intmax_t)pos); + + blk += (pos - hole) / secsz; + error = image_chunk_skipto(blk); + + bytesize += pos - hole; + cur = data; + } else if (cur == data && hole > data) { + data = pos; + pos = (hole + secsz - 1) & ~(secsz - 1); + + fprintf(stderr, "DATA %jd-%jd data\n", + (intmax_t)data, (intmax_t)pos); + + /* Sloppy... */ + sz = pos - data; + assert((off_t)sz == pos - data); + + buf = image_file_map(fd, data, sz); + if (buf != NULL) { + error = image_chunk_copyin(blk, buf, sz, + data, fd); + image_file_unmap(buf, sz); + } else + error = errno; + blk += sz / secsz; + bytesize += sz; + cur = hole; + } else { + /* + * I don't know what this means or whether it + * can happen at all... + */ + error = EDOOFUS; + break; + } + } + if (error) + close(fd); + if (!error && sizep != NULL) + *sizep = bytesize; + return (error); } int