From owner-svn-src-user@FreeBSD.ORG Sun Oct 25 00:26:24 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BC9D31065670; Sun, 25 Oct 2009 00:26:24 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 9324B8FC0A; Sun, 25 Oct 2009 00:26:24 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id n9P0QOFr020147; Sun, 25 Oct 2009 00:26:24 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id n9P0QOYj020146; Sun, 25 Oct 2009 00:26:24 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200910250026.n9P0QOYj020146@svn.freebsd.org> From: Kip Macy Date: Sun, 25 Oct 2009 00:26:24 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r198455 - user/kmacy/releng_8_fcs_buf X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Oct 2009 00:26:24 -0000 Author: kmacy Date: Sun Oct 25 00:26:24 2009 New Revision: 198455 URL: http://svn.freebsd.org/changeset/base/198455 Log: create branch for backing ZFS buf by page cache Added: user/kmacy/releng_8_fcs_buf/ (props changed) - copied from r198454, user/kmacy/releng_8_fcs/ From owner-svn-src-user@FreeBSD.ORG Sun Oct 25 00:37:59 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CFC39106568B; Sun, 25 Oct 2009 00:37:59 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id BF44B8FC0A; Sun, 25 Oct 2009 00:37:59 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id n9P0bxbR020525; Sun, 25 Oct 2009 00:37:59 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id n9P0bxT2020522; Sun, 25 Oct 2009 00:37:59 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200910250037.n9P0bxT2020522@svn.freebsd.org> From: Kip Macy Date: Sun, 25 Oct 2009 00:37:59 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r198457 - user/kmacy/releng_8_fcs/sys/sys X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Oct 2009 00:37:59 -0000 Author: kmacy Date: Sun Oct 25 00:37:59 2009 New Revision: 198457 URL: http://svn.freebsd.org/changeset/base/198457 Log: - increase MAXBSIZE to MAXPHYS - define GB_NODUMP for excluding pages from dump Modified: user/kmacy/releng_8_fcs/sys/sys/buf.h user/kmacy/releng_8_fcs/sys/sys/param.h Modified: user/kmacy/releng_8_fcs/sys/sys/buf.h ============================================================================== --- user/kmacy/releng_8_fcs/sys/sys/buf.h Sun Oct 25 00:28:01 2009 (r198456) +++ user/kmacy/releng_8_fcs/sys/sys/buf.h Sun Oct 25 00:37:59 2009 (r198457) @@ -444,6 +444,7 @@ buf_countdeps(struct buf *bp, int i) #define GB_LOCK_NOWAIT 0x0001 /* Fail if we block on a buf lock. */ #define GB_NOCREAT 0x0002 /* Don't create a buf if not found. */ #define GB_NOWAIT_BD 0x0004 /* Do not wait for bufdaemon */ +#define GB_NODUMP 0x0008 /* exclude this buffer from core dump */ #ifdef _KERNEL extern int nbuf; /* The number of buffer headers */ Modified: user/kmacy/releng_8_fcs/sys/sys/param.h ============================================================================== --- user/kmacy/releng_8_fcs/sys/sys/param.h Sun Oct 25 00:28:01 2009 (r198456) +++ user/kmacy/releng_8_fcs/sys/sys/param.h Sun Oct 25 00:37:59 2009 (r198457) @@ -221,7 +221,7 @@ * The default is 16384, roughly 2x the block size used by a * normal UFS filesystem. */ -#define MAXBSIZE 65536 /* must be power of 2 */ +#define MAXBSIZE MAXPHYS /* must be power of 2 */ #define BKVASIZE 16384 /* must be power of 2 */ #define BKVAMASK (BKVASIZE-1) From owner-svn-src-user@FreeBSD.ORG Sun Oct 25 00:42:03 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 65E49106566C; Sun, 25 Oct 2009 00:42:03 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 537CB8FC0A; Sun, 25 Oct 2009 00:42:03 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id n9P0g3g5020919; Sun, 25 Oct 2009 00:42:03 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id n9P0g3sO020912; Sun, 25 Oct 2009 00:42:03 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200910250042.n9P0g3sO020912@svn.freebsd.org> From: Kip Macy Date: Sun, 25 Oct 2009 00:42:03 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r198458 - in user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs: . sys X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Oct 2009 00:42:03 -0000 Author: kmacy Date: Sun Oct 25 00:42:03 2009 New Revision: 198458 URL: http://svn.freebsd.org/changeset/base/198458 Log: initial support for backing the ARC cache by the page cache Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/arc.h user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Sun Oct 25 00:37:59 2009 (r198457) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Sun Oct 25 00:42:03 2009 (r198458) @@ -126,6 +126,7 @@ #include #ifdef _KERNEL #include +#include #endif #include #include @@ -258,6 +259,7 @@ static arc_state_t ARC_l2c_only; typedef struct arc_stats { kstat_named_t arcstat_hits; + kstat_named_t arcstat_page_cache_hits; kstat_named_t arcstat_misses; kstat_named_t arcstat_demand_data_hits; kstat_named_t arcstat_demand_data_misses; @@ -307,6 +309,7 @@ typedef struct arc_stats { static arc_stats_t arc_stats = { { "hits", KSTAT_DATA_UINT64 }, + { "page_cache_hits", KSTAT_DATA_UINT64 }, { "misses", KSTAT_DATA_UINT64 }, { "demand_data_hits", KSTAT_DATA_UINT64 }, { "demand_data_misses", KSTAT_DATA_UINT64 }, @@ -512,6 +515,7 @@ static void arc_evict_ghost(arc_state_t #define ARC_L2_EVICTED (1 << 17) /* evicted during I/O */ #define ARC_L2_WRITE_HEAD (1 << 18) /* head of write list */ #define ARC_STORED (1 << 19) /* has been store()d to */ +#define ARC_BUF_CLONING (1 << 21) /* is being cloned */ #define HDR_IN_HASH_TABLE(hdr) ((hdr)->b_flags & ARC_IN_HASH_TABLE) #define HDR_IO_IN_PROGRESS(hdr) ((hdr)->b_flags & ARC_IO_IN_PROGRESS) @@ -632,9 +636,10 @@ struct l2arc_buf_hdr { typedef struct l2arc_data_free { /* protected by l2arc_free_on_write_mtx */ + arc_buf_t *l2df_buf; void *l2df_data; size_t l2df_size; - void (*l2df_func)(void *, size_t); + void (*l2df_func)(arc_buf_t *, void *, size_t); list_node_t l2df_list_node; } l2arc_data_free_t; @@ -1190,8 +1195,8 @@ arc_data_buf_free(void *buf, uint64_t si atomic_add_64(&arc_size, -size); } -arc_buf_t * -arc_buf_alloc(spa_t *spa, int size, void *tag, arc_buf_contents_t type) +static arc_buf_t * +_arc_buf_alloc(spa_t *spa, int size, void *tag, arc_buf_contents_t type, dva_t dva) { arc_buf_hdr_t *hdr; arc_buf_t *buf; @@ -1201,6 +1206,7 @@ arc_buf_alloc(spa_t *spa, int size, void ASSERT(BUF_EMPTY(hdr)); hdr->b_size = size; hdr->b_type = type; + hdr->b_dva = dva; hdr->b_spa = spa; hdr->b_state = arc_anon; hdr->b_arc_access = 0; @@ -1220,6 +1226,14 @@ arc_buf_alloc(spa_t *spa, int size, void return (buf); } +arc_buf_t * +arc_buf_alloc(spa_t *spa, int size, void *tag, arc_buf_contents_t type) +{ + dva_t dva = {0ULL, 0ULL}; + + return (_arc_buf_alloc(spa, size, tag, type, dva)); +} + static arc_buf_t * arc_buf_clone(arc_buf_t *from) { @@ -1234,8 +1248,11 @@ arc_buf_clone(arc_buf_t *from) buf->b_private = NULL; buf->b_next = hdr->b_buf; hdr->b_buf = buf; + hdr->b_flags |= ARC_BUF_CLONING; arc_get_data_buf(buf); +#ifdef nomore bcopy(from->b_data, buf->b_data, size); +#endif hdr->b_datacnt += 1; return (buf); } @@ -1272,17 +1289,95 @@ arc_buf_add_ref(arc_buf_t *buf, void* ta data, metadata, hits); } +static void +arc_getblk(arc_buf_t *buf) +{ + uint64_t size = buf->b_hdr->b_size; + arc_buf_contents_t type = buf->b_hdr->b_type; + spa_t *spa = buf->b_hdr->b_spa; + off_t blkno = buf->b_hdr->b_dva.dva_word[1] & ~(1UL<<63); + struct buf *newbp, *bp; + arc_buf_t *tbuf; + struct vnode *vp; + int flags = 0; + + if (type == ARC_BUFC_METADATA) { + arc_space_consume(size); + } else { + ASSERT(type == ARC_BUFC_DATA); + flags = GB_NODUMP; + atomic_add_64(&arc_size, size); + } + + if (buf->b_hdr->b_flags & ARC_BUF_CLONING) { + newbp = geteblk(size, flags); + tbuf = buf; + + while (tbuf->b_next != NULL) + tbuf = tbuf->b_next; + bp = tbuf->b_bp; + vp = spa_get_vnode(spa); + + KASSERT((bp->b_blkno == bp->b_lblkno) && + (bp->b_blkno == blkno), + ("blkno mismatch b_blkno %ld b_lblkno %ld blkno %ld", + bp->b_blkno, bp->b_lblkno, blkno)); + newbp->b_bufobj = &vp->v_bufobj; + newbp->b_lblkno = blkno; + newbp->b_blkno = blkno; + newbp->b_offset = (blkno<<9); + + if (bp->b_vp != NULL) { + KASSERT(bp->b_xflags & BX_VNCLEAN, ("brelvp() on buffer that is not in splay")); + brelvp(bp); + } + + BO_LOCK(&vp->v_bufobj); + bgetvp(vp, newbp); + BO_UNLOCK(&vp->v_bufobj); + newbp->b_flags &= ~B_INVAL; + newbp->b_flags |= B_CACHE; + bp->b_flags |= B_INVAL; + bp->b_flags &= ~B_CACHE; + bcopy(bp->b_data, newbp->b_data, size); + buf->b_hdr->b_flags &= ~ARC_BUF_CLONING; + + } else if (BUF_EMPTY(buf->b_hdr)) { + newbp = geteblk(size, flags); + } else + newbp = getblk(spa_get_vnode(spa), blkno, + size, 0, 0, flags); + CTR2(KTR_BUF, "arc_getblk() bp=%p flags %X", + newbp, newbp->b_flags); + + BUF_KERNPROC(newbp); + buf->b_bp = newbp; + buf->b_data = newbp->b_data; +} + +static void +arc_brelse(arc_buf_t *buf, void *data, size_t size) +{ + +#ifdef INVARIANTS + if (buf->b_bp->b_vp) + KASSERT(buf->b_bp->b_xflags & BX_VNCLEAN, ("brelse() on buffer that is not in splay")); +#endif + brelse(buf->b_bp); +} + /* * Free the arc data buffer. If it is an l2arc write in progress, * the buffer is placed on l2arc_free_on_write to be freed later. */ static void -arc_buf_data_free(arc_buf_hdr_t *hdr, void (*free_func)(void *, size_t), - void *data, size_t size) +arc_buf_data_free(arc_buf_hdr_t *hdr, void (*free_func)(arc_buf_t *, void *, size_t), + arc_buf_t *buf, void *data, size_t size) { if (HDR_L2_WRITING(hdr)) { l2arc_data_free_t *df; df = kmem_alloc(sizeof (l2arc_data_free_t), KM_SLEEP); + df->l2df_buf = buf; df->l2df_data = data; df->l2df_size = size; df->l2df_func = free_func; @@ -1291,7 +1386,7 @@ arc_buf_data_free(arc_buf_hdr_t *hdr, vo mutex_exit(&l2arc_free_on_write_mtx); ARCSTAT_BUMP(arcstat_l2_free_on_write); } else { - free_func(data, size); + free_func(buf, data, size); } } @@ -1309,13 +1404,13 @@ arc_buf_destroy(arc_buf_t *buf, boolean_ arc_cksum_verify(buf); if (!recycle) { if (type == ARC_BUFC_METADATA) { - arc_buf_data_free(buf->b_hdr, zio_buf_free, - buf->b_data, size); + arc_buf_data_free(buf->b_hdr, arc_brelse, + buf, buf->b_data, size); arc_space_return(size); } else { ASSERT(type == ARC_BUFC_DATA); - arc_buf_data_free(buf->b_hdr, - zio_data_buf_free, buf->b_data, size); + arc_buf_data_free(buf->b_hdr, arc_brelse, + buf, buf->b_data, size); atomic_add_64(&arc_size, -size); } } @@ -1514,7 +1609,7 @@ arc_buf_size(arc_buf_t *buf) * it can't get a hash_lock on, and so may not catch all candidates. * It may also return without evicting as much space as requested. */ -static void * +static struct buf * arc_evict(arc_state_t *state, spa_t *spa, int64_t bytes, boolean_t recycle, arc_buf_contents_t type) { @@ -1526,11 +1621,12 @@ arc_evict(arc_state_t *state, spa_t *spa kmutex_t *lock, *evicted_lock; kmutex_t *hash_lock; boolean_t have_lock; - void *stolen = NULL; + struct buf *stolen = NULL; static int evict_metadata_offset, evict_data_offset; int idx, offset, list_count, count; ASSERT(state == arc_mru || state == arc_mfu); + ASSERT(recycle == FALSE); evicted_state = (state == arc_mru) ? arc_mru_ghost : arc_mfu_ghost; @@ -1598,7 +1694,7 @@ evict_start: if (buf->b_efunc) { mutex_enter(&arc_eviction_mtx); arc_buf_destroy(buf, - buf->b_data == stolen, FALSE); + buf->b_bp == stolen, FALSE); ab->b_buf = buf->b_next; buf->b_hdr = &arc_eviction_hdr; buf->b_next = arc_eviction_list; @@ -1608,7 +1704,7 @@ evict_start: } else { rw_exit(&buf->b_lock); arc_buf_destroy(buf, - buf->b_data == stolen, TRUE); + buf->b_bp == stolen, TRUE); } } if (ab->b_datacnt == 0) { @@ -2267,14 +2363,7 @@ arc_get_data_buf(arc_buf_t *buf) * just allocate a new buffer. */ if (!arc_evict_needed(type)) { - if (type == ARC_BUFC_METADATA) { - buf->b_data = zio_buf_alloc(size); - arc_space_consume(size); - } else { - ASSERT(type == ARC_BUFC_DATA); - buf->b_data = zio_data_buf_alloc(size); - atomic_add_64(&arc_size, size); - } + arc_getblk(buf); goto out; } @@ -2297,17 +2386,8 @@ arc_get_data_buf(arc_buf_t *buf) state = (arc_mru->arcs_lsize[type] > 0 && mfu_space > arc_mfu->arcs_size) ? arc_mru : arc_mfu; } - if ((buf->b_data = arc_evict(state, NULL, size, TRUE, type)) == NULL) { - if (type == ARC_BUFC_METADATA) { - buf->b_data = zio_buf_alloc(size); - arc_space_consume(size); - } else { - ASSERT(type == ARC_BUFC_DATA); - buf->b_data = zio_data_buf_alloc(size); - atomic_add_64(&arc_size, size); - } - ARCSTAT_BUMP(arcstat_recycle_miss); - } + (void) arc_evict(state, NULL, size, FALSE, type); + arc_getblk(buf); ASSERT(buf->b_data != NULL); out: /* @@ -2528,6 +2608,9 @@ arc_read_done(zio_t *zio) arc_cksum_compute(buf, B_FALSE); + buf->b_bp->b_flags &= ~B_INVAL; + buf->b_bp->b_flags |= B_CACHE; + /* create copies of the data buffer for the callers */ abuf = buf; for (acb = callback_list; acb; acb = acb->acb_next) { @@ -2734,9 +2817,9 @@ top: /* this block is not in the cache */ arc_buf_hdr_t *exists; arc_buf_contents_t type = BP_GET_BUFC_TYPE(bp); - buf = arc_buf_alloc(spa, size, private, type); + buf = _arc_buf_alloc(spa, size, private, type, + *BP_IDENTITY(bp)); hdr = buf->b_hdr; - hdr->b_dva = *BP_IDENTITY(bp); hdr->b_birth = bp->blk_birth; hdr->b_cksum0 = bp->blk_cksum.zc_word[0]; exists = buf_hash_insert(hdr, &hash_lock); @@ -2783,7 +2866,19 @@ top: arc_get_data_buf(buf); ASSERT(hdr->b_datacnt == 0); hdr->b_datacnt = 1; - + } + /* + * We hit in the page cache + * + */ + if ((buf->b_bp->b_flags & (B_CACHE|B_INVAL)) == B_CACHE) { + /* + * track the number of times + * the buffer was found in the cache + */ + ARCSTAT_BUMP(arcstat_page_cache_hits); + mutex_exit(hash_lock); + goto top; } acb = kmem_zalloc(sizeof (arc_callback_t), KM_SLEEP); @@ -3211,7 +3306,6 @@ arc_write_done(zio_t *zio) arc_buf_hdr_t *hdr = buf->b_hdr; hdr->b_acb = NULL; - hdr->b_dva = *BP_IDENTITY(zio->io_bp); hdr->b_birth = zio->io_bp->blk_birth; hdr->b_cksum0 = zio->io_bp->blk_cksum.zc_word[0]; @@ -3224,7 +3318,32 @@ arc_write_done(zio_t *zio) if (!BUF_EMPTY(hdr)) { arc_buf_hdr_t *exists; kmutex_t *hash_lock; + /* + * Associate buffer with offset in the page cache + */ + struct buf *bp = buf->b_bp; + struct vnode *vp = spa_get_vnode(hdr->b_spa); + off_t blkno = hdr->b_dva.dva_word[1] & ~(1UL<<63); + + CTR2(KTR_BUF, "arc_write_done(%p) flags %X", + bp, bp->b_flags); + + if ((hdr->b_buf == buf) && + (bp->b_bufobj == NULL)) { + + bp->b_bufobj = &vp->v_bufobj; + bp->b_lblkno = blkno; + bp->b_blkno = blkno; + bp->b_offset = (blkno << 9); + BO_LOCK(bp->b_bufobj); + bgetvp(vp, bp); + BO_UNLOCK(bp->b_bufobj); + bp->b_flags &= ~B_INVAL; + bp->b_flags |= B_CACHE; + } + /* + */ arc_cksum_verify(buf); exists = buf_hash_insert(hdr, &hash_lock); @@ -3987,7 +4106,7 @@ l2arc_do_free_on_write() df_prev = list_prev(buflist, df); ASSERT(df->l2df_data != NULL); ASSERT(df->l2df_func != NULL); - df->l2df_func(df->l2df_data, df->l2df_size); + df->l2df_func(df->l2df_buf, df->l2df_data, df->l2df_size); list_remove(buflist, df); kmem_free(df, sizeof (l2arc_data_free_t)); } Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c Sun Oct 25 00:37:59 2009 (r198457) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c Sun Oct 25 00:42:03 2009 (r198458) @@ -4299,3 +4299,10 @@ done: #endif #endif } + +struct vnode * +spa_get_vnode(spa_t *spa) +{ + + return (spa->spa_root_vdev->vdev_vnode); +} Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/arc.h ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/arc.h Sun Oct 25 00:37:59 2009 (r198457) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/arc.h Sun Oct 25 00:42:03 2009 (r198458) @@ -52,6 +52,7 @@ struct arc_buf { void *b_data; arc_evict_func_t *b_efunc; void *b_private; + struct buf *b_bp; }; typedef enum arc_buf_contents { Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h Sun Oct 25 00:37:59 2009 (r198457) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa.h Sun Oct 25 00:42:03 2009 (r198458) @@ -534,6 +534,8 @@ extern void spa_prop_clear_bootfs(spa_t /* asynchronous event notification */ extern void spa_event_notify(spa_t *spa, vdev_t *vdev, const char *name); +extern struct vnode *spa_get_vnode(spa_t *spa); + #ifdef ZFS_DEBUG #define dprintf_bp(bp, fmt, ...) do { \ if (zfs_flags & ZFS_DEBUG_DPRINTF) { \ Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h Sun Oct 25 00:37:59 2009 (r198457) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h Sun Oct 25 00:42:03 2009 (r198458) @@ -144,7 +144,8 @@ struct vdev { list_node_t vdev_state_dirty_node; /* state dirty list */ uint64_t vdev_deflate_ratio; /* deflation ratio (x512) */ uint64_t vdev_islog; /* is an intent log device */ - + struct vnode *vdev_vnode; /* container for page cache */ + /* * Leaf vdev state. */ Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c Sun Oct 25 00:37:59 2009 (r198457) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c Sun Oct 25 00:42:03 2009 (r198458) @@ -1060,10 +1060,24 @@ vdev_open(vdev_t *vd) * inconsistently account for existing bp's. */ if (vd->vdev_top == vd) { + struct vnode *vp; + vd->vdev_deflate_ratio = (1<<17) / (vdev_psize_to_asize(vd, 1<<17) >> SPA_MINBLOCKSHIFT); + } + if (vd->vdev_parent == NULL) { + struct vnode *vp; + error = getnewvnode("zpool" , NULL, &dead_vnodeops, &vp); + if (error != 0) + return (error); + + vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); + vnode_create_vobject(vp, 512, curthread); + vd->vdev_vnode = vp; + VOP_UNLOCK(vp, 0); + } /* * If a leaf vdev has a DTL, and seems healthy, then kick off a * resilver. But don't do this if we are doing a reopen for a @@ -1192,6 +1206,8 @@ vdev_close(vdev_t *vd) else vd->vdev_state = VDEV_STATE_CLOSED; vd->vdev_stat.vs_aux = VDEV_AUX_NONE; + + vn_free(vd->vdev_vnode); } void From owner-svn-src-user@FreeBSD.ORG Sun Oct 25 00:51:16 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 50D30106566C; Sun, 25 Oct 2009 00:51:16 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 406788FC0C; Sun, 25 Oct 2009 00:51:16 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id n9P0pGiW021387; Sun, 25 Oct 2009 00:51:16 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id n9P0pGYA021384; Sun, 25 Oct 2009 00:51:16 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200910250051.n9P0pGYA021384@svn.freebsd.org> From: Kip Macy Date: Sun, 25 Oct 2009 00:51:16 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r198459 - user/kmacy/releng_8_fcs_buf/sys/sys X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Oct 2009 00:51:16 -0000 Author: kmacy Date: Sun Oct 25 00:51:15 2009 New Revision: 198459 URL: http://svn.freebsd.org/changeset/base/198459 Log: - define GB_NODUMP excluding buffers from dump - increase MAXBSIZE to MAXPHYS Modified: user/kmacy/releng_8_fcs_buf/sys/sys/buf.h user/kmacy/releng_8_fcs_buf/sys/sys/param.h Modified: user/kmacy/releng_8_fcs_buf/sys/sys/buf.h ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/sys/buf.h Sun Oct 25 00:42:03 2009 (r198458) +++ user/kmacy/releng_8_fcs_buf/sys/sys/buf.h Sun Oct 25 00:51:15 2009 (r198459) @@ -444,6 +444,7 @@ buf_countdeps(struct buf *bp, int i) #define GB_LOCK_NOWAIT 0x0001 /* Fail if we block on a buf lock. */ #define GB_NOCREAT 0x0002 /* Don't create a buf if not found. */ #define GB_NOWAIT_BD 0x0004 /* Do not wait for bufdaemon */ +#define GB_NODUMP 0x0008 /* exclude this buffer from core dump */ #ifdef _KERNEL extern int nbuf; /* The number of buffer headers */ Modified: user/kmacy/releng_8_fcs_buf/sys/sys/param.h ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/sys/param.h Sun Oct 25 00:42:03 2009 (r198458) +++ user/kmacy/releng_8_fcs_buf/sys/sys/param.h Sun Oct 25 00:51:15 2009 (r198459) @@ -221,7 +221,7 @@ * The default is 16384, roughly 2x the block size used by a * normal UFS filesystem. */ -#define MAXBSIZE 65536 /* must be power of 2 */ +#define MAXBSIZE MAXPHYS /* must be power of 2 */ #define BKVASIZE 16384 /* must be power of 2 */ #define BKVAMASK (BKVASIZE-1) From owner-svn-src-user@FreeBSD.ORG Wed Oct 28 09:55:43 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DB9951065694; Wed, 28 Oct 2009 09:55:42 +0000 (UTC) (envelope-from des@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id CA40F8FC2B; Wed, 28 Oct 2009 09:55:42 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id n9S9tgOe057518; Wed, 28 Oct 2009 09:55:42 GMT (envelope-from des@svn.freebsd.org) Received: (from des@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id n9S9tgDG057509; Wed, 28 Oct 2009 09:55:42 GMT (envelope-from des@svn.freebsd.org) Message-Id: <200910280955.n9S9tgDG057509@svn.freebsd.org> From: Dag-Erling Smorgrav Date: Wed, 28 Oct 2009 09:55:42 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r198536 - in user/des/svnsup/src: distill libsvnsup X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Oct 2009 09:55:43 -0000 Author: des Date: Wed Oct 28 09:55:42 2009 New Revision: 198536 URL: http://svn.freebsd.org/changeset/base/198536 Log: Implement most of the delta-generating stuff. Added: user/des/svnsup/src/libsvnsup/svnsup_base64.c (contents, props changed) user/des/svnsup/src/libsvnsup/svnsup_string.c (contents, props changed) Modified: user/des/svnsup/src/distill/distill.c user/des/svnsup/src/distill/editor.c user/des/svnsup/src/distill/txdelta.c user/des/svnsup/src/libsvnsup/Makefile.am user/des/svnsup/src/libsvnsup/svnsup.h user/des/svnsup/src/libsvnsup/svnsup_delta.c Modified: user/des/svnsup/src/distill/distill.c ============================================================================== --- user/des/svnsup/src/distill/distill.c Wed Oct 28 07:05:32 2009 (r198535) +++ user/des/svnsup/src/distill/distill.c Wed Oct 28 09:55:42 2009 (r198536) @@ -50,6 +50,8 @@ distill(const char *url, unsigned long r apr_array_header_t *auth_providers; svn_ra_session_t *ra_session; svn_error_t *error; + svnsup_delta_t sd; + svnsup_err_t err; /* our root pool */ status = apr_pool_create(&pool, NULL); @@ -70,6 +72,10 @@ distill(const char *url, unsigned long r NULL, config, pool); SVNSUP_SVN_ERROR(error, "svn_ra_open3()"); + /* XXX create delta */ + err = svnsup_create_delta(&sd); + SVNSUP_SVNSUP_ERROR(err, "svnsup_delta_create()"); + /* get revision metadata */ error = svn_ra_get_log2(ra_session, NULL, revision, revision, 0, TRUE, TRUE, FALSE, NULL, log_entry_receiver, NULL, pool); @@ -77,9 +83,13 @@ distill(const char *url, unsigned long r /* replay the requested revision */ error = svn_ra_replay(ra_session, revision, revision - 1, TRUE, - &delta_editor, NULL, pool); + &delta_editor, sd, pool); SVNSUP_SVN_ERROR(error, "svn_ra_replay()"); + /* finish off the delta */ + err = svnsup_close_delta(sd); + SVNSUP_SVNSUP_ERROR(err, "svnsup_delta_close()"); + /* clean up */ apr_pool_destroy(pool); return (0); Modified: user/des/svnsup/src/distill/editor.c ============================================================================== --- user/des/svnsup/src/distill/editor.c Wed Oct 28 07:05:32 2009 (r198535) +++ user/des/svnsup/src/distill/editor.c Wed Oct 28 09:55:42 2009 (r198536) @@ -51,11 +51,11 @@ open_root(void *edit_baton, apr_pool_t *dir_pool, void **root_baton) { + svnsup_delta_t sd = (svnsup_delta_t)edit_baton; - (void)edit_baton; (void)dir_pool; SVNSUP_DEBUG("%s(%ld)\n", __func__, (long)base_revision); - *root_baton = (void *)__LINE__; + *root_baton = sd; return (SVN_NO_ERROR); } @@ -65,10 +65,11 @@ delete_entry(const char *path, void *parent_baton, apr_pool_t *pool) { + svnsup_delta_t sd = (svnsup_delta_t)parent_baton; - (void)parent_baton; (void)pool; SVNSUP_DEBUG("%s(%ld, %s)\n", __func__, (long)revision, path); + svnsup_delta_remove(sd, path); return (SVN_NO_ERROR); } @@ -80,12 +81,13 @@ add_directory(const char *path, apr_pool_t *dir_pool, void **child_baton) { + svnsup_delta_t sd = (svnsup_delta_t)parent_baton; - (void)parent_baton; (void)dir_pool; SVNSUP_DEBUG("%s(%s, %s, %ld)\n", __func__, path, copyfrom_path, (long)copyfrom_revision); - *child_baton = (void *)__LINE__; + svnsup_delta_create_directory(sd, path); + *child_baton = sd; /* XXX */ return (SVN_NO_ERROR); } @@ -96,12 +98,12 @@ open_directory(const char *path, apr_pool_t *dir_pool, void **child_baton) { + svnsup_delta_t sd = (svnsup_delta_t)parent_baton; - (void)parent_baton; (void)dir_pool; SVNSUP_DEBUG("%s(%s, %ld)\n", __func__, path, (long)base_revision); - *child_baton = (void *)__LINE__; + *child_baton = sd; /* XXX */ return (SVN_NO_ERROR); } @@ -134,8 +136,9 @@ absent_directory(const char *path, void *parent_baton, apr_pool_t *pool) { + svnsup_delta_t sd = (svnsup_delta_t)parent_baton; - (void)parent_baton; + (void)sd; (void)pool; SVNSUP_DEBUG("%s(%s)\n", __func__, path); return (SVN_NO_ERROR); @@ -149,12 +152,14 @@ add_file(const char *path, apr_pool_t *file_pool, void **file_baton) { + svnsup_delta_t sd = (svnsup_delta_t)parent_baton; + svnsup_delta_file_t sdf; - (void)parent_baton; (void)file_pool; SVNSUP_DEBUG("%s(%s, %s, %ld)\n", __func__, path, copyfrom_path, (long)copyfrom_revision); - *file_baton = (void *)__LINE__; + svnsup_delta_create_file(sd, &sdf, path); + *file_baton = sdf; return (SVN_NO_ERROR); } @@ -165,12 +170,15 @@ open_file(const char *path, apr_pool_t *file_pool, void **file_baton) { + svnsup_delta_t sd = (svnsup_delta_t)parent_baton; + svnsup_delta_file_t sdf; - (void)parent_baton; + (void)sd; (void)file_pool; SVNSUP_DEBUG("%s(%s, %ld)\n", __func__, path, (long)base_revision); - *file_baton = (void *)__LINE__; + svnsup_delta_open_file(sd, &sdf, path); + *file_baton = sdf; return (SVN_NO_ERROR); } @@ -181,12 +189,15 @@ apply_textdelta(void *file_baton, svn_txdelta_window_handler_t *handler, void **handler_baton) { + svnsup_delta_file_t sdf = (svnsup_delta_file_t)file_baton; - (void)file_baton; + (void)sdf; (void)pool; SVNSUP_DEBUG("%s(%s)\n", __func__, base_checksum); + if (base_checksum) + svnsup_delta_file_checksum(sdf, base_checksum); *handler = txdelta_window_handler; - *handler_baton = (void *)__LINE__; + *handler_baton = file_baton; return (SVN_NO_ERROR); } @@ -196,8 +207,9 @@ change_file_prop(void *file_baton, const svn_string_t *value, apr_pool_t *pool) { + svnsup_delta_file_t sdf = (svnsup_delta_file_t)file_baton; - (void)file_baton; + (void)sdf; (void)pool; SVNSUP_DEBUG("%s(%s, %s)\n", __func__, name, value->data); return (SVN_NO_ERROR); @@ -208,10 +220,11 @@ close_file(void *file_baton, const char *text_checksum, apr_pool_t *pool) { + svnsup_delta_file_t sdf = (svnsup_delta_file_t)file_baton; - (void)file_baton; (void)pool; SVNSUP_DEBUG("%s(%s)\n", __func__, text_checksum); + svnsup_delta_close_file(sdf, text_checksum); return (SVN_NO_ERROR); } @@ -220,8 +233,9 @@ absent_file(const char *path, void *parent_baton, apr_pool_t *pool) { + svnsup_delta_t sd = (svnsup_delta_t)parent_baton; - (void)parent_baton; + (void)sd; (void)pool; SVNSUP_DEBUG("%s(%s)\n", __func__, path); return (SVN_NO_ERROR); Modified: user/des/svnsup/src/distill/txdelta.c ============================================================================== --- user/des/svnsup/src/distill/txdelta.c Wed Oct 28 07:05:32 2009 (r198535) +++ user/des/svnsup/src/distill/txdelta.c Wed Oct 28 09:55:42 2009 (r198536) @@ -35,49 +35,42 @@ #include "distill.h" +// XXX documentation + error handling svn_error_t * txdelta_window_handler(svn_txdelta_window_t *window, void *baton) { + svnsup_delta_file_t sdf = (svnsup_delta_file_t)baton; const svn_txdelta_op_t *op; - int i; + unsigned int txtid = 0; + int i, ret; - (void)baton; SVNSUP_DEBUG("%s()\n", __func__); - if (!debug) + if (window == NULL) return (SVN_NO_ERROR); - if (window == NULL) { - fprintf(stderr, "end of delta\n"); - return (SVN_NO_ERROR); - } else { - fprintf(stderr, "delta\n"); + if (window->new_data != NULL && window->new_data->len > 0) { + SVNSUP_DEBUG("%lu bytes of data\n", + (unsigned long)window->new_data->len); + ret = svnsup_delta_file_text(sdf, window->new_data->data, + window->new_data->len, &txtid); + SVNSUP_SVNSUP_ERROR(ret, "svnsup_delta_file_text()"); } - fprintf(stderr, " src off: %ld\n", (long)window->sview_offset); - fprintf(stderr, " src len: %ld\n", (long)window->sview_len); - fprintf(stderr, " tgt len: %ld\n", (long)window->tview_len); - fprintf(stderr, " ops: %d (%d src)\n", window->num_ops, window->src_ops); for (i = 0, op = window->ops; i < window->num_ops; ++i, ++op) { - fprintf(stderr, " op #%d: ", i); switch (op->action_code) { case svn_txdelta_source: - fprintf(stderr, "src %ld:%ld\n", - (long)op->offset, (long)op->length); + svnsup_delta_file_copy(sdf, op->offset, op->length); break; case svn_txdelta_target: - fprintf(stderr, "tgt %ld:%ld\n", - (long)op->offset, (long)op->length); + svnsup_delta_file_repeat(sdf, op->offset, op->length); break; case svn_txdelta_new: - fprintf(stderr, "new %ld:%ld\n", - (long)op->offset, (long)op->length); - if (verbose) - fprintf(stderr, "%.*s\n", (int)op->length, - window->new_data->data + op->offset); + svnsup_delta_file_insert(sdf, txtid, op->offset, + op->length); break; default: - fprintf(stderr, "???\n"); + SVNSUP_ASSERT(0, "invalid window operation"); break; } } Modified: user/des/svnsup/src/libsvnsup/Makefile.am ============================================================================== --- user/des/svnsup/src/libsvnsup/Makefile.am Wed Oct 28 07:05:32 2009 (r198535) +++ user/des/svnsup/src/libsvnsup/Makefile.am Wed Oct 28 09:55:42 2009 (r198536) @@ -3,7 +3,9 @@ lib_LIBRARIES = libsvnsup.a libsvnsup_a_SOURCES = \ - svnsup_delta.c + svnsup_base64.c \ + svnsup_delta.c \ + svnsup_string.c noinst_HEADERS = \ svnsup.h Modified: user/des/svnsup/src/libsvnsup/svnsup.h ============================================================================== --- user/des/svnsup/src/libsvnsup/svnsup.h Wed Oct 28 07:05:32 2009 (r198535) +++ user/des/svnsup/src/libsvnsup/svnsup.h Wed Oct 28 09:55:42 2009 (r198536) @@ -37,11 +37,52 @@ typedef enum svnsup_err { SVNSUP_ERR_MAX, } svnsup_err_t; +/* + * svnsup_delta.c + */ typedef struct svnsup_delta *svnsup_delta_t; +typedef struct svnsup_delta_file *svnsup_delta_file_t; -int svnsup_delta_create(svnsup_delta_t *); -int svnsup_delta_close(svnsup_delta_t); +int svnsup_create_delta(svnsup_delta_t *); +int svnsup_close_delta(svnsup_delta_t); int svnsup_delta_comment(svnsup_delta_t, const char *, ...); +int svnsup_delta_meta(svnsup_delta_t, const char *, const char *, ...); +int svnsup_delta_create_directory(svnsup_delta_t, const char *); +int svnsup_delta_remove(svnsup_delta_t, const char *); +int svnsup_delta_text(svnsup_delta_t, const char *, size_t, + unsigned int *); + +int svnsup_delta_create_file(svnsup_delta_t, svnsup_delta_file_t *, + const char *); +int svnsup_delta_open_file(svnsup_delta_t, svnsup_delta_file_t *, + const char *); +int svnsup_delta_file_checksum(svnsup_delta_file_t, const char *); +int svnsup_delta_file_text(svnsup_delta_file_t, const char *, size_t, + unsigned int *); +int svnsup_delta_file_copy(svnsup_delta_file_t, off_t, size_t); +int svnsup_delta_file_repeat(svnsup_delta_file_t, off_t, size_t); +int svnsup_delta_file_insert(svnsup_delta_file_t, unsigned int, off_t, size_t); +int svnsup_delta_close_file(svnsup_delta_file_t, const char *); + +/* + * svnsup_string.c + */ +int svnsup_string_is_safe(const char *); +int svnsup_buf_is_safe(const char *, size_t); +char *svnsup_string_encode(const char *); +char *svnsup_buf_encode(const char *, size_t); + +#ifdef FOPEN_MAX /* defined by stdio.h, cf. IEEE 1003.1 */ +int svnsup_string_fencode(FILE *, const char *); +int svnsup_buf_fencode(FILE *, const char *, size_t); +#endif + +/* + * svnsup_base64.c + */ +#ifdef FOPEN_MAX /* defined by stdio.h, cf. IEEE 1003.1 */ +int svnsup_base64_fencode(FILE *, const unsigned char *, size_t); +#endif #endif Added: user/des/svnsup/src/libsvnsup/svnsup_base64.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ user/des/svnsup/src/libsvnsup/svnsup_base64.c Wed Oct 28 09:55:42 2009 (r198536) @@ -0,0 +1,84 @@ +/*- + * Copyright (c) 2009 Dag-Erling Coïdan Smørgrav + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer + * in this position and unchanged. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $Id$ + */ + +#ifdef HAVE_CONFIG_H +#include "config.h" +#endif + +#include + +#include + +#include "svnsup.h" + +static const char b64t[65] = + "ABCDEFGHIJKLMNOPQRSTUVWXYZ" + "abcdefghijklmnopqrstuvwxyz" + "0123456789" + "+/" + "="; + +int +svnsup_base64_fencode(FILE *f, const unsigned char *buf, size_t size) +{ + int count = 0; +#if 0 + int width = 0; +#endif + + while (size >= 3) { + putc(b64t[buf[0] >> 2], f); + putc(b64t[(buf[0] << 4 | buf[1] >> 4) & 0x3f], f); + putc(b64t[(buf[1] << 2 | buf[2] >> 6) & 0x3f], f); + putc(b64t[buf[2] & 0x3f], f); + count += 4; + buf += 3; + size -= 3; +#if 0 + if ((width += 3) == 64 && size > 0) { + putc('\n', f); + ++count; + width = 0; + } +#endif + } + if (size > 0) { + putc(b64t[buf[0] >> 2], f); + if (size > 1) { + putc(b64t[(buf[0] << 4 | buf[1] >> 4) & 0x3f], f); + putc(b64t[(buf[1] << 2) & 0x3f], f); + } else { + putc(b64t[(buf[0] << 4) & 0x3f], f); + putc('=', f); + } + putc('=', f); + count += 4; + } + return (count); +} Modified: user/des/svnsup/src/libsvnsup/svnsup_delta.c ============================================================================== --- user/des/svnsup/src/libsvnsup/svnsup_delta.c Wed Oct 28 07:05:32 2009 (r198535) +++ user/des/svnsup/src/libsvnsup/svnsup_delta.c Wed Oct 28 09:55:42 2009 (r198536) @@ -31,20 +31,61 @@ #include "config.h" #endif +#include #include #include +#include #include #include +#include #include "svnsup.h" +// XXX missing I/O error handling + struct svnsup_delta { FILE *f; - int started:1; + struct svnsup_delta_file *sdf; + unsigned int ntxt; +}; + +struct svnsup_delta_file { + struct svnsup_delta *sd; + char *fn; + int create:1; + int checksum:1; }; +static svnsup_delta_file_t +svnsup_delta_file_alloc(svnsup_delta_t sd, const char *fn) +{ + svnsup_delta_file_t sdf; + + if ((sdf = calloc(1, sizeof *sdf)) == NULL) { + return (NULL); + } else if ((sdf->fn = strdup(fn)) == NULL) { + free(sdf); + return (NULL); + } + sdf->sd = sd; + sd->sdf = sdf; + return (sdf); +} + +void +svnsup_delta_file_free(svnsup_delta_file_t sdf) +{ + + sdf->sd->sdf = NULL; + free(sdf->fn); + free(sdf); +} + +/* + * Create an svnsup delta. + */ int -svnsup_delta_create(svnsup_delta_t *sdp) +svnsup_create_delta(svnsup_delta_t *sdp) { svnsup_delta_t sd; @@ -55,14 +96,21 @@ svnsup_delta_create(svnsup_delta_t *sdp) return (SVNSUP_ERR_NONE); } +/* + * Close an svnsup delta. + */ int -svnsup_delta_close(svnsup_delta_t sd) +svnsup_close_delta(svnsup_delta_t sd) { + assert(sd->sdf == NULL); free(sd); return (SVNSUP_ERR_NONE); } +/* + * Comment + */ int svnsup_delta_comment(svnsup_delta_t sd, const char *fmt, ...) { @@ -77,15 +125,226 @@ svnsup_delta_comment(svnsup_delta_t sd, return (SVNSUP_ERR_MEMORY); p = commentbuf; while (*p != '\0') { - fputs("# ", sd->f); + fprintf(sd->f, "# "); while (*p != '\0' && *p != '\n') { - putc(isprint(*p) ? *p : ' ', sd->f); + fprintf(sd->f, "%c", isprint(*p) ? *p : ' '); ++p; } - putc('\n', sd->f); + fprintf(sd->f, "\n"); if (*p == '\n') ++p; } free(commentbuf); return (SVNSUP_ERR_NONE); } + +/* + * Metadata + */ +int +svnsup_delta_meta(svnsup_delta_t sd, const char *key, const char *fmt, ...) +{ + + (void)sd; + (void)key; + (void)fmt; + return (SVNSUP_ERR_NONE); +} + +/* + * Create a directory + */ +int +svnsup_delta_create_directory(svnsup_delta_t sd, const char *dn) +{ + + assert(sd != NULL); + assert(dn != NULL && *dn != '\0'); + assert(sd->sdf == NULL); + fprintf(sd->f, "@mkdir "); + svnsup_string_fencode(sd->f, dn); + fprintf(sd->f, "\n"); + return (SVNSUP_ERR_NONE); +} + +/* + * Remove a file or directory + */ +int +svnsup_delta_remove(svnsup_delta_t sd, const char *fn) +{ + + assert(sd != NULL); + assert(fn != NULL && *fn != '\0'); + assert(sd->sdf == NULL); + fprintf(sd->f, "@remove "); + svnsup_string_fencode(sd->f, fn); + fprintf(sd->f, "\n"); + return (SVNSUP_ERR_NONE); +} + +/* + * Text to be used in later edits + */ +int +svnsup_delta_text(svnsup_delta_t sd, const char *src, size_t len, + unsigned int *txtid) +{ + + assert(sd != NULL); + assert(src != NULL); + assert(len > 0); + assert(txtid != NULL); + *txtid = sd->ntxt++; + fprintf(sd->f, "@text %u ", *txtid); + svnsup_buf_fencode(sd->f, src, len); + fprintf(sd->f, "\n"); + return (SVNSUP_ERR_NONE); +} + +/* + * Create a file and start working on it + */ +int +svnsup_delta_create_file(svnsup_delta_t sd, svnsup_delta_file_t *sdfp, + const char *fn) +{ + svnsup_delta_file_t sdf; + + assert(sd != NULL); + assert(sd->sdf == NULL); + assert(sdfp != NULL); + assert(fn != NULL && *fn != '\0'); + if ((sdf = svnsup_delta_file_alloc(sd, fn)) == NULL) + return (SVNSUP_ERR_MEMORY); + sdf->create = 1; + *sdfp = sdf; + fprintf(sd->f, "@create "); + svnsup_string_fencode(sd->f, fn); + fprintf(sd->f, "\n"); + return (SVNSUP_ERR_NONE); +} + +/* + * Start working on the specified file + */ +int +svnsup_delta_open_file(svnsup_delta_t sd, svnsup_delta_file_t *sdfp, + const char *fn) +{ + svnsup_delta_file_t sdf; + + assert(sd != NULL); + assert(sd->sdf == NULL); + assert(sdfp != NULL); + assert(fn != NULL && *fn != '\0'); + if ((sdf = svnsup_delta_file_alloc(sd, fn)) == NULL) + return (SVNSUP_ERR_MEMORY); + *sdfp = sdf; + return (SVNSUP_ERR_NONE); +} + +/* + * Checksum of the original file + */ +int +svnsup_delta_file_checksum(svnsup_delta_file_t sdf, const char *md5) +{ + + assert(sdf != NULL); + assert(sdf->sd != NULL); + assert(sdf->sd->sdf == sdf); + assert(sdf->fn != NULL); + assert(!sdf->create); + assert(*sdf->fn != '\0'); + assert(md5 != NULL && *md5 != '\0'); + fprintf(sdf->sd->f, "@open "); + svnsup_string_fencode(sdf->sd->f, sdf->fn); + fprintf(sdf->sd->f, " md5 "); + svnsup_string_fencode(sdf->sd->f, md5); + fprintf(sdf->sd->f, "\n"); + sdf->checksum = 1; + return (SVNSUP_ERR_NONE); +} + +/* + * Shortcut to svnsup_delta_text() + */ +int +svnsup_delta_file_text(svnsup_delta_file_t sdf, const char *src, size_t len, + unsigned int *txtid) +{ + + return (svnsup_delta_text(sdf->sd, src, len, txtid)); +} + +/* + * Copy text from the original file to the new file + */ +int +svnsup_delta_file_copy(svnsup_delta_file_t sdf, off_t off, size_t size) +{ + + assert(sdf != NULL); + assert(sdf->sd != NULL); + assert(sdf->sd->sdf == sdf); + assert(sdf->create || sdf->checksum); + assert(size > 0); + fprintf(sdf->sd->f, "@copy %ju %zu\n", (uintmax_t)off, size); + return (SVNSUP_ERR_NONE); +} + +/* + * Repeat text in the new file + */ +int +svnsup_delta_file_repeat(svnsup_delta_file_t sdf, off_t off, size_t size) +{ + + assert(sdf != NULL); + assert(sdf->sd != NULL); + assert(sdf->sd->sdf == sdf); + assert(sdf->create || sdf->checksum); + assert(size > 0); + fprintf(sdf->sd->f, "@repeat %ju %zu\n", (uintmax_t)off, size); + return (SVNSUP_ERR_NONE); +} + +/* + * Insert text into the new file + */ +int +svnsup_delta_file_insert(svnsup_delta_file_t sdf, unsigned int txtid, + off_t off, size_t size) +{ + + assert(sdf != NULL); + assert(sdf->sd != NULL); + assert(sdf->sd->sdf == sdf); + assert(sdf->create || sdf->checksum); + assert(txtid < sdf->sd->ntxt); + assert(size > 0); + fprintf(sdf->sd->f, "@insert %u %ju %zu\n", txtid, (uintmax_t)off, size); + return (SVNSUP_ERR_NONE); +} + +/* + * Stop working on the specified file + */ +int +svnsup_delta_close_file(svnsup_delta_file_t sdf, const char *md5) +{ + + assert(sdf != NULL); + assert(sdf->sd != NULL); + assert(sdf->sd->sdf == sdf); + assert(sdf->create || sdf->checksum); + assert(md5 != NULL && *md5 != '\0'); + fprintf(sdf->sd->f, "@close "); + svnsup_string_fencode(sdf->sd->f, sdf->fn); + fprintf(sdf->sd->f, " md5 "); + svnsup_string_fencode(sdf->sd->f, md5); + fprintf(sdf->sd->f, "\n"); + svnsup_delta_file_free(sdf); + return (SVNSUP_ERR_NONE); +} Added: user/des/svnsup/src/libsvnsup/svnsup_string.c ============================================================================== --- /dev/null 00:00:00 1970 (empty, because file is newly added) +++ user/des/svnsup/src/libsvnsup/svnsup_string.c Wed Oct 28 09:55:42 2009 (r198536) @@ -0,0 +1,111 @@ +/*- + * Copyright (c) 2009 Dag-Erling Coïdan Smørgrav + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer + * in this position and unchanged. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $Id$ + */ + +#ifdef HAVE_CONFIG_H +#include "config.h" +#endif + +#include + +#include +#include +#include +#include + +#include "svnsup.h" + +/* + * Safe to send as is + */ +int +svnsup_string_is_safe(const char *str) +{ + + while (*str != '\0') { + if (!isprint(*str) || isspace(*str)) + return (0); + ++str; + } + return (1); +} + +/* + * Safe to send as is + */ +int +svnsup_buf_is_safe(const char *buf, size_t size) +{ + + while (size > 0) { + if (!isprint(*buf) || isspace(*buf)) + return (0); + ++buf; + --size; + } + return (1); +} + +char * +svnsup_string_encode(const char *str) +{ + + assert(0); + (void)str; + return (NULL); +} + +char * +svnsup_buf_encode(const char *buf, size_t size) +{ + + assert(0); + (void)buf; + (void)size; + return (NULL); +} + +int +svnsup_string_fencode(FILE *f, const char *str) +{ + + return (svnsup_buf_fencode(f, str, strlen(str))); +} + +int +svnsup_buf_fencode(FILE *f, const char *buf, size_t size) +{ + int len; + + if (svnsup_buf_is_safe(buf, size)) + return (fprintf(f, "%zu[%.*s]", size, (int)size, buf)); + len = fprintf(f, "%zu{", size); + len += svnsup_base64_fencode(f, (const unsigned char *)buf, size); + len += fprintf(f, "}"); + return (len); +} From owner-svn-src-user@FreeBSD.ORG Sat Oct 31 22:50:29 2009 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7454A106566C; Sat, 31 Oct 2009 22:50:29 +0000 (UTC) (envelope-from kmacy@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 652E88FC14; Sat, 31 Oct 2009 22:50:29 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id n9VMoTpl005616; Sat, 31 Oct 2009 22:50:29 GMT (envelope-from kmacy@svn.freebsd.org) Received: (from kmacy@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id n9VMoTFJ005614; Sat, 31 Oct 2009 22:50:29 GMT (envelope-from kmacy@svn.freebsd.org) Message-Id: <200910312250.n9VMoTFJ005614@svn.freebsd.org> From: Kip Macy Date: Sat, 31 Oct 2009 22:50:29 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r198734 - user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 Oct 2009 22:50:29 -0000 Author: kmacy Date: Sat Oct 31 22:50:29 2009 New Revision: 198734 URL: http://svn.freebsd.org/changeset/base/198734 Log: force unmount ZFS volumes earlier in shutdown to release bufs back to the system Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Sat Oct 31 22:27:31 2009 (r198733) +++ user/kmacy/releng_8_fcs_buf/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Sat Oct 31 22:50:29 2009 (r198734) @@ -3657,6 +3657,7 @@ arc_tempreserve_space(uint64_t reserve, static kmutex_t arc_lowmem_lock; #ifdef _KERNEL static eventhandler_tag arc_event_lowmem = NULL; +static eventhandler_tag arc_event_shutdown = NULL; static void arc_lowmem(void *arg __unused, int howto __unused) @@ -3670,6 +3671,30 @@ arc_lowmem(void *arg __unused, int howto tsleep(&needfree, 0, "zfs:lowmem", hz / 5); mutex_exit(&arc_lowmem_lock); } +void +arc_shutdown(void *arg __unused, int howto __unused) +{ + struct mount *mp, *tmpmp; + int error; + + arc_flush(NULL); + TAILQ_FOREACH_SAFE(mp, &mountlist, mnt_list, tmpmp) { + if (strcmp(mp->mnt_vfc->vfc_name, "zfs") == 0) { + error = dounmount(mp, MNT_FORCE, curthread); + if (error) { + TAILQ_REMOVE(&mountlist, mp, mnt_list); + printf("unmount of %s failed (", + mp->mnt_stat.f_mntonname); + if (error == EBUSY) + printf("BUSY)\n"); + else + printf("%d)\n", error); + } + } + + } +} + #endif void @@ -3793,8 +3818,10 @@ arc_init(void) TS_RUN, minclsyspri); #ifdef _KERNEL - arc_event_lowmem = EVENTHANDLER_REGISTER(vm_lowmem, arc_lowmem, NULL, - EVENTHANDLER_PRI_FIRST); + arc_event_lowmem = EVENTHANDLER_REGISTER(vm_lowmem, arc_lowmem, + NULL, EVENTHANDLER_PRI_FIRST); + arc_event_shutdown = EVENTHANDLER_REGISTER(shutdown_pre_sync, + arc_event_shutdown, NULL, EVENTHANDLER_PRI_FIRST); #endif arc_dead = FALSE; @@ -3885,6 +3912,8 @@ arc_fini(void) #ifdef _KERNEL if (arc_event_lowmem != NULL) EVENTHANDLER_DEREGISTER(vm_lowmem, arc_event_lowmem); + if (arc_event_shutdown != NULL) + EVENTHANDLER_DEREGISTER(shutdown_pre_sync, arc_event_shutdown); #endif }