From owner-freebsd-fs@FreeBSD.ORG Sun Jul 1 02:36:06 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7FFC91065670 for ; Sun, 1 Jul 2012 02:36:06 +0000 (UTC) (envelope-from mueller23@insightbb.com) Received: from mail.insightbb.com (smtp.insight.synacor.com [208.47.185.22]) by mx1.freebsd.org (Postfix) with ESMTP id 41D368FC18 for ; Sun, 1 Jul 2012 02:36:06 +0000 (UTC) X_CMAE_Category: 0,0 Undefined,Undefined X-CNFS-Analysis: v=1.1 cv=JliV1TWavljy2htMhoUIMxhlgBLBqsUyu4r3EPHUlag= c=1 sm=0 a=190A9ldbhagA:10 a=jLN7EqiLvroA:10 a=sg0gfQ5U3qcR-Zv8BogA:9 a=Q/oqmR4JO1zR3vNQamCQeQ==:117 X-CM-Score: 0 X-Scanned-by: Cloudmark Authority Engine Authentication-Results: smtp01.insight.synacor.com smtp.mail=mueller23@insightbb.com; spf=softfail; sender-id=softfail Authentication-Results: smtp01.insight.synacor.com header.from=mueller23@insightbb.com; sender-id=softfail Received-SPF: softfail (smtp01.insight.synacor.com: transitional domain insightbb.com does not designate 74.134.26.53 as permitted sender) Received: from [74.134.26.53] ([74.134.26.53:58750] helo=localhost) by mail.insightbb.com (envelope-from ) (ecelerity 2.2.2.40 r(29895/29896)) with ESMTP id 61/60-17943-E87BFEF4; Sat, 30 Jun 2012 22:35:59 -0400 Date: Sat, 30 Jun 2012 22:35:58 -0400 Message-ID: <61.60.17943.E87BFEF4@smtp01.insight.synacor.com> From: "Thomas Mueller" To: freebsd-current@freebsd.org Cc: FreeBSD FS , Attilio Rao Subject: Re: MPSAFE VFS -- List of upcoming actions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jul 2012 02:36:06 -0000 > You can not only run Linux on XFS (which I do) but it is still likely > the most reliable and consistently performant of the filesystems > available in Linux because of its origin and its maturity. XFS did > not originate in Linux (it originated in SGI's Irix) so it should not > surprise that Linux core developers are proponents of filesystems > originally developed under Linux. > Regardless, a key value of *BSD supporting non-native filesystems is > in order to be able to access filesystems created on other OSs. > XFS is a major filesystem so hopefully someone will volunteer to > support it. > Bob > -- > Bob Friesenhahn I remember starting threads, many months ago, both on FreeBSD and NetBSD lists about lingua franca file system for sharing data between OSes. I can see why NTFS support is important, due to its prevalence. But I believe very few people would have any use for HPFS access from BSD, and it would be practically impossible to find testers. Tom From owner-freebsd-fs@FreeBSD.ORG Sun Jul 1 12:52:15 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8F6911065673 for ; Sun, 1 Jul 2012 12:52:15 +0000 (UTC) (envelope-from cpghost@cordula.ws) Received: from mail-vc0-f182.google.com (mail-vc0-f182.google.com [209.85.220.182]) by mx1.freebsd.org (Postfix) with ESMTP id 45E088FC12 for ; Sun, 1 Jul 2012 12:52:15 +0000 (UTC) Received: by vcbfy7 with SMTP id fy7so3761466vcb.13 for ; Sun, 01 Jul 2012 05:52:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-originating-ip:in-reply-to:references:date :message-id:subject:from:to:cc:content-type:x-gm-message-state; bh=R7dmhUfaFLwDzEU+V8I12kPkQK7K5hoab9yFaA5/P/s=; b=GYQunKusEWP1w2f5NVoNtvZxhZd586yTfUTprQOLWTaxka8xpAPRyMC+GWNsdygn8/ +7GFDV1b7u4mJh66DnhKzYG58b/Pbx/CcvaCucPJSEam+98Ab0kRSk0AiMK/aCqKqPVT xy855bBiO7IdhA/s8g3yEyGfHdRtraQZhsmBS3Gjbtv7xWJt1OatfJHiChva30621v1b OI4pTMr2axgUYMEaDuHuT+HDeMBuCW0n1swkVSlwWkZBDPqiYlV6o0sJwCTQWok55GOb mrA/+tJS4rcbiEff95r/jh79m+ygMygPSQF/Wl5NUA1z5MopZVsWQH+dGO59rwOr6G0N hEjw== MIME-Version: 1.0 Received: by 10.220.218.141 with SMTP id hq13mr4487656vcb.8.1341147134790; Sun, 01 Jul 2012 05:52:14 -0700 (PDT) Received: by 10.220.240.197 with HTTP; Sun, 1 Jul 2012 05:52:14 -0700 (PDT) X-Originating-IP: [93.221.178.61] In-Reply-To: References: Date: Sun, 1 Jul 2012 14:52:14 +0200 Message-ID: From: "C. P. Ghost" To: Attilio Rao Content-Type: text/plain; charset=ISO-8859-1 X-Gm-Message-State: ALoCoQlq3cH7qc5YJ9g25okpkN6ko2NUKfuH+K3YV5kUkaRc22eqUulx/7l0HyhZxXDkSXxzMSmA Cc: FreeBSD FS , freebsd-current@freebsd.org Subject: Re: MPSAFE VFS -- List of upcoming actions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jul 2012 12:52:15 -0000 On Fri, Jun 29, 2012 at 10:32 PM, Attilio Rao wrote: > As already published several times, according to the following plan: > http://wiki.freebsd.org/NONMPSAFE_DEORBIT_VFS > > in 2 months the code dealing with non-MPSAFE filesystem will be > removed and filesystems not yet MPSAFE will be disconnected from the > tree. Their code will be however available in our official repository > yet for 6 months. This leaves a total time of 8 months to do actions. > > Current list of unmantained filesystems is: HPFS, NWFS, PortalFS and > XFS. Coda and SMBFS have current mantainership but the status of the > work has still to be determined. NTFS, is being worked for the Summer > of Code program. Finally, ReiserFS was successfully locked during this > campaign. Sorry if this has been discussed already, but... it's one thing to obsolete some code, it's a different thing to remove it entirely where there's no user-level replacement, especially since it would also remove the ability to access ancient media that some people may still have access to. Couldn't filesystems that are still not MP SAFE be kept in the tree in such a way that they at least provide read-only access in case of emergencies? > It is time for community members to step up and offer time and skills > to lock a filesystem or test the effort of other developers willing to > do so. I don't have the skills for that, but I'd love to see some brave soul step up to rescue PortalFS. ;-) > Thanks, > Attilio Regards, -cpghost. -- Cordula's Web. http://www.cordula.ws/ From owner-freebsd-fs@FreeBSD.ORG Sun Jul 1 13:52:07 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4A964106566B; Sun, 1 Jul 2012 13:52:07 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) by mx1.freebsd.org (Postfix) with ESMTP id 8FD368FC0A; Sun, 1 Jul 2012 13:52:06 +0000 (UTC) Received: by lbon10 with SMTP id n10so8085018lbo.13 for ; Sun, 01 Jul 2012 06:52:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=ZyqLcgTbvH8XtoJ1UYt4UB/CiaZeZ8hzjvy8/45mjnw=; b=cP+ZUWiiGL/SmuejbdFmuuJZvkJQBXwEfuoySqsOuVBsBpgirTNv+/MR8kJBuj3vTD e/9Gi4Ds2Dtve9/Ot6E5ZrLFLUcPJStaYDqjL5Bcu1L62JhRrF2bbqzj24+xNhztzUHO UuJJfB69OPUVN1cFjF/JUR+qlQhctKOG9A+j+aIbQgbOT6S7Blwnu1vv3qs8GRV4buoF RO2sogEDVvpomv/cPAfJquskyp4qi3N0QHRKPBC2DXj01BmRgNaa3pKOhAX54fuEtCeD F4AhJCBtS0fxnNzI2jqlN+QxOkMxPEH8us6GjfTia3zaFrJl+UUcR0oxhD2PM2+KhJIf rKHA== MIME-Version: 1.0 Received: by 10.152.46.6 with SMTP id r6mr9347323lam.7.1341150725329; Sun, 01 Jul 2012 06:52:05 -0700 (PDT) Sender: asmrookie@gmail.com Received: by 10.112.27.65 with HTTP; Sun, 1 Jul 2012 06:52:05 -0700 (PDT) In-Reply-To: References: Date: Sun, 1 Jul 2012 15:52:05 +0200 X-Google-Sender-Auth: 3KKjJXsWWzgtQuJq8SQUHw9VyQw Message-ID: From: Attilio Rao To: "C. P. Ghost" Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD FS , freebsd-current@freebsd.org Subject: Re: MPSAFE VFS -- List of upcoming actions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Jul 2012 13:52:07 -0000 2012/7/1 C. P. Ghost : > On Fri, Jun 29, 2012 at 10:32 PM, Attilio Rao wrote: >> As already published several times, according to the following plan: >> http://wiki.freebsd.org/NONMPSAFE_DEORBIT_VFS >> >> in 2 months the code dealing with non-MPSAFE filesystem will be >> removed and filesystems not yet MPSAFE will be disconnected from the >> tree. Their code will be however available in our official repository >> yet for 6 months. This leaves a total time of 8 months to do actions. >> >> Current list of unmantained filesystems is: HPFS, NWFS, PortalFS and >> XFS. Coda and SMBFS have current mantainership but the status of the >> work has still to be determined. NTFS, is being worked for the Summer >> of Code program. Finally, ReiserFS was successfully locked during this >> campaign. > > Sorry if this has been discussed already, but... > > it's one thing to obsolete some code, it's a different thing to > remove it entirely where there's no user-level replacement, > especially since it would also remove the ability to access > ancient media that some people may still have access to. > > Couldn't filesystems that are still not MP SAFE be kept in the > tree in such a way that they at least provide read-only access > in case of emergencies? Unfortunately not. First of all, they are mostly already READ-ONLY, in particular XFS and NTFS. Second, it would be meaning leave in place the Giant-VFS bloat that we necessarilly must get rid of. The most interesting ones in the list, IMHO, are still SMBFS and NTFS and possibly XFS (but this is really a personal preference). I've received an e-mail explaining that there are arrangements by a company to put their developers on work after 1st september on SMBFS locking which is certainly a good new, but I still haven't heard anything by SoC involved people about NTFS and certainly I don't see a plan to get XFS locked. However remind that at the worst (for filesystems like PortalFS, for example) the code will remain in Attic even after it gets depurated and then it can be revitalized and locked in whatever point in the future. Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-fs@FreeBSD.ORG Mon Jul 2 06:12:23 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id ACC721065673; Mon, 2 Jul 2012 06:12:23 +0000 (UTC) (envelope-from BATV+46b70ed064d2f657f9da+3235+infradead.org+hch@bombadil.srs.infradead.org) Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2001:4830:2446:ff00:4687:fcff:fea6:5117]) by mx1.freebsd.org (Postfix) with ESMTP id 54E3E8FC12; Mon, 2 Jul 2012 06:12:23 +0000 (UTC) Received: from hch by bombadil.infradead.org with local (Exim 4.76 #1 (Red Hat Linux)) id 1SlZrw-0004LL-19; Mon, 02 Jul 2012 06:12:20 +0000 Date: Mon, 2 Jul 2012 02:12:20 -0400 From: Christoph Hellwig To: Attilio Rao Message-ID: <20120702061219.GA16671@infradead.org> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html Cc: FreeBSD FS , Russell Cattelan , freebsd-current@freebsd.org, "C. P. Ghost" Subject: Re: MPSAFE VFS -- List of upcoming actions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2012 06:12:23 -0000 On Sun, Jul 01, 2012 at 03:52:05PM +0200, Attilio Rao wrote: > anything by SoC involved people about NTFS and certainly I don't see a > plan to get XFS locked. Stupid question, but what amount of locking does XFS in FreeBSD still need? I'm one of the maintainer of XFS on Linux, and while I know FreeBSD imported a really old version compared to the current one the codebases on IRIX and later Linux never relied on any global Giant-style locking. So if there is anything to fix it would be the in the small bits of FreeBSD-specific code. From owner-freebsd-fs@FreeBSD.ORG Mon Jul 2 11:07:10 2012 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A98B410656E6 for ; Mon, 2 Jul 2012 11:07:10 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 9270B8FC23 for ; Mon, 2 Jul 2012 11:07:10 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q62B7AUT012597 for ; Mon, 2 Jul 2012 11:07:10 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q62B79fx012595 for freebsd-fs@FreeBSD.org; Mon, 2 Jul 2012 11:07:09 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 2 Jul 2012 11:07:09 GMT Message-Id: <201207021107.q62B79fx012595@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-fs@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2012 11:07:10 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/169480 fs [zfs] ZFS stalls on heavy I/O o kern/169398 fs [zfs] Can't remove file with permanent error o kern/169339 fs panic while " : > /etc/123" o kern/169319 fs [zfs] zfs resilver can't complete o kern/168947 fs [nfs] [zfs] .zfs/snapshot directory is messed up when o kern/168942 fs [nfs] [hang] nfsd hangs after being restarted (not -HU o kern/168158 fs [zfs] incorrect parsing of sharenfs options in zfs (fs o kern/167979 fs [ufs] DIOCGDINFO ioctl does not work on 8.2 file syste o kern/167977 fs [smbfs] mount_smbfs results are differ when utf-8 or U o kern/167688 fs [fusefs] Incorrect signal handling with direct_io o kern/167685 fs [zfs] ZFS on USB drive prevents shutdown / reboot o kern/167612 fs [portalfs] The portal file system gets stuck inside po o kern/167272 fs [zfs] ZFS Disks reordering causes ZFS to pick the wron o kern/167260 fs [msdosfs] msdosfs disk was mounted the second time whe o kern/167109 fs [zfs] [panic] zfs diff kernel panic Fatal trap 9: gene o kern/167105 fs [nfs] mount_nfs can not handle source exports wiht mor o kern/167067 fs [zfs] [panic] ZFS panics the server o kern/167066 fs [zfs] ZVOLs not appearing in /dev/zvol o kern/167065 fs [zfs] boot fails when a spare is the boot disk o kern/167048 fs [nfs] [patch] RELEASE-9 crash when using ZFS+NULLFS+NF o kern/166912 fs [ufs] [panic] Panic after converting Softupdates to jo o kern/166851 fs [zfs] [hang] Copying directory from the mounted UFS di o kern/166477 fs [nfs] NFS data corruption. o kern/165950 fs [ffs] SU+J and fsck problem o kern/165923 fs [nfs] Writing to NFS-backed mmapped files fails if flu o kern/165521 fs [zfs] [hang] livelock on 1 Gig of RAM with zfs when 31 o kern/165392 fs Multiple mkdir/rmdir fails with errno 31 o kern/165087 fs [unionfs] lock violation in unionfs o kern/164472 fs [ufs] fsck -B panics on particular data inconsistency o kern/164370 fs [zfs] zfs destroy for snapshot fails on i386 and sparc o kern/164261 fs [nullfs] [patch] fix panic with NFS served from NULLFS o kern/164256 fs [zfs] device entry for volume is not created after zfs o kern/164184 fs [ufs] [panic] Kernel panic with ufs_makeinode o kern/163801 fs [md] [request] allow mfsBSD legacy installed in 'swap' o kern/163770 fs [zfs] [hang] LOR between zfs&syncer + vnlru leading to o kern/163501 fs [nfs] NFS exporting a dir and a subdir in that dir to o kern/162944 fs [coda] Coda file system module looks broken in 9.0 o kern/162860 fs [zfs] Cannot share ZFS filesystem to hosts with a hyph o kern/162751 fs [zfs] [panic] kernel panics during file operations o kern/162591 fs [nullfs] cross-filesystem nullfs does not work as expe o kern/162519 fs [zfs] "zpool import" relies on buggy realpath() behavi o kern/162362 fs [snapshots] [panic] ufs with snapshot(s) panics when g o kern/161968 fs [zfs] [hang] renaming snapshot with -r including a zvo o kern/161897 fs [zfs] [patch] zfs partition probing causing long delay o kern/161864 fs [ufs] removing journaling from UFS partition fails on o bin/161807 fs [patch] add option for explicitly specifying metadata o kern/161579 fs [smbfs] FreeBSD sometimes panics when an smb share is o kern/161533 fs [zfs] [panic] zfs receive panic: system ioctl returnin o kern/161438 fs [zfs] [panic] recursed on non-recursive spa_namespace_ o kern/161424 fs [nullfs] __getcwd() calls fail when used on nullfs mou o kern/161280 fs [zfs] Stack overflow in gptzfsboot o kern/161205 fs [nfs] [pfsync] [regression] [build] Bug report freebsd o kern/161169 fs [zfs] [panic] ZFS causes kernel panic in dbuf_dirty o kern/161112 fs [ufs] [lor] filesystem LOR in FreeBSD 9.0-BETA3 o kern/160893 fs [zfs] [panic] 9.0-BETA2 kernel panic o kern/160860 fs [ufs] Random UFS root filesystem corruption with SU+J o kern/160801 fs [zfs] zfsboot on 8.2-RELEASE fails to boot from root-o o kern/160790 fs [fusefs] [panic] VPUTX: negative ref count with FUSE o kern/160777 fs [zfs] [hang] RAID-Z3 causes fatal hang upon scrub/impo o kern/160706 fs [zfs] zfs bootloader fails when a non-root vdev exists o kern/160591 fs [zfs] Fail to boot on zfs root with degraded raidz2 [r o kern/160410 fs [smbfs] [hang] smbfs hangs when transferring large fil o kern/160283 fs [zfs] [patch] 'zfs list' does abort in make_dataset_ha o kern/159930 fs [ufs] [panic] kernel core o kern/159402 fs [zfs][loader] symlinks cause I/O errors o kern/159357 fs [zfs] ZFS MAXNAMELEN macro has confusing name (off-by- o kern/159356 fs [zfs] [patch] ZFS NAME_ERR_DISKLIKE check is Solaris-s o kern/159351 fs [nfs] [patch] - divide by zero in mountnfs() o kern/159251 fs [zfs] [request]: add FLETCHER4 as DEDUP hash option o kern/159077 fs [zfs] Can't cd .. with latest zfs version o kern/159048 fs [smbfs] smb mount corrupts large files o kern/159045 fs [zfs] [hang] ZFS scrub freezes system o kern/158839 fs [zfs] ZFS Bootloader Fails if there is a Dead Disk o kern/158802 fs amd(8) ICMP storm and unkillable process. o kern/158231 fs [nullfs] panic on unmounting nullfs mounted over ufs o f kern/157929 fs [nfs] NFS slow read o kern/157399 fs [zfs] trouble with: mdconfig force delete && zfs strip o kern/157179 fs [zfs] zfs/dbuf.c: panic: solaris assert: arc_buf_remov o kern/156797 fs [zfs] [panic] Double panic with FreeBSD 9-CURRENT and o kern/156781 fs [zfs] zfs is losing the snapshot directory, p kern/156545 fs [ufs] mv could break UFS on SMP systems o kern/156193 fs [ufs] [hang] UFS snapshot hangs && deadlocks processes o kern/156039 fs [nullfs] [unionfs] nullfs + unionfs do not compose, re o kern/155615 fs [zfs] zfs v28 broken on sparc64 -current o kern/155587 fs [zfs] [panic] kernel panic with zfs p kern/155411 fs [regression] [8.2-release] [tmpfs]: mount: tmpfs : No o kern/155199 fs [ext2fs] ext3fs mounted as ext2fs gives I/O errors o bin/155104 fs [zfs][patch] use /dev prefix by default when importing o kern/154930 fs [zfs] cannot delete/unlink file from full volume -> EN o kern/154828 fs [msdosfs] Unable to create directories on external USB o kern/154491 fs [smbfs] smb_co_lock: recursive lock for object 1 p kern/154228 fs [md] md getting stuck in wdrain state o kern/153996 fs [zfs] zfs root mount error while kernel is not located o kern/153753 fs [zfs] ZFS v15 - grammatical error when attempting to u o kern/153716 fs [zfs] zpool scrub time remaining is incorrect o kern/153695 fs [patch] [zfs] Booting from zpool created on 4k-sector o kern/153680 fs [xfs] 8.1 failing to mount XFS partitions o kern/153520 fs [zfs] Boot from GPT ZFS root on HP BL460c G1 unstable o kern/153418 fs [zfs] [panic] Kernel Panic occurred writing to zfs vol o kern/153351 fs [zfs] locking directories/files in ZFS o bin/153258 fs [patch][zfs] creating ZVOLs requires `refreservation' s kern/153173 fs [zfs] booting from a gzip-compressed dataset doesn't w o kern/153126 fs [zfs] vdev failure, zpool=peegel type=vdev.too_small o kern/152022 fs [nfs] nfs service hangs with linux client [regression] o kern/151942 fs [zfs] panic during ls(1) zfs snapshot directory o kern/151905 fs [zfs] page fault under load in /sbin/zfs o bin/151713 fs [patch] Bug in growfs(8) with respect to 32-bit overfl o kern/151648 fs [zfs] disk wait bug o kern/151629 fs [fs] [patch] Skip empty directory entries during name o kern/151330 fs [zfs] will unshare all zfs filesystem after execute a o kern/151326 fs [nfs] nfs exports fail if netgroups contain duplicate o kern/151251 fs [ufs] Can not create files on filesystem with heavy us o kern/151226 fs [zfs] can't delete zfs snapshot o kern/151111 fs [zfs] vnodes leakage during zfs unmount o kern/150503 fs [zfs] ZFS disks are UNAVAIL and corrupted after reboot o kern/150501 fs [zfs] ZFS vdev failure vdev.bad_label on amd64 o kern/150390 fs [zfs] zfs deadlock when arcmsr reports drive faulted o kern/150336 fs [nfs] mountd/nfsd became confused; refused to reload n o kern/149208 fs mksnap_ffs(8) hang/deadlock o kern/149173 fs [patch] [zfs] make OpenSolaris installa o kern/149015 fs [zfs] [patch] misc fixes for ZFS code to build on Glib o kern/149014 fs [zfs] [patch] declarations in ZFS libraries/utilities o kern/149013 fs [zfs] [patch] make ZFS makefiles use the libraries fro o kern/148504 fs [zfs] ZFS' zpool does not allow replacing drives to be o kern/148490 fs [zfs]: zpool attach - resilver bidirectionally, and re o kern/148368 fs [zfs] ZFS hanging forever on 8.1-PRERELEASE o kern/148138 fs [zfs] zfs raidz pool commands freeze o kern/147903 fs [zfs] [panic] Kernel panics on faulty zfs device o kern/147881 fs [zfs] [patch] ZFS "sharenfs" doesn't allow different " o kern/147560 fs [zfs] [boot] Booting 8.1-PRERELEASE raidz system take o kern/147420 fs [ufs] [panic] ufs_dirbad, nullfs, jail panic (corrupt o kern/146941 fs [zfs] [panic] Kernel Double Fault - Happens constantly o kern/146786 fs [zfs] zpool import hangs with checksum errors o kern/146708 fs [ufs] [panic] Kernel panic in softdep_disk_write_compl o kern/146528 fs [zfs] Severe memory leak in ZFS on i386 o kern/146502 fs [nfs] FreeBSD 8 NFS Client Connection to Server s kern/145712 fs [zfs] cannot offline two drives in a raidz2 configurat o kern/145411 fs [xfs] [panic] Kernel panics shortly after mounting an f bin/145309 fs bsdlabel: Editing disk label invalidates the whole dev o kern/145272 fs [zfs] [panic] Panic during boot when accessing zfs on o kern/145246 fs [ufs] dirhash in 7.3 gratuitously frees hashes when it o kern/145238 fs [zfs] [panic] kernel panic on zpool clear tank o kern/145229 fs [zfs] Vast differences in ZFS ARC behavior between 8.0 o kern/145189 fs [nfs] nfsd performs abysmally under load o kern/144929 fs [ufs] [lor] vfs_bio.c + ufs_dirhash.c p kern/144447 fs [zfs] sharenfs fsunshare() & fsshare_main() non functi o kern/144416 fs [panic] Kernel panic on online filesystem optimization s kern/144415 fs [zfs] [panic] kernel panics on boot after zfs crash o kern/144234 fs [zfs] Cannot boot machine with recent gptzfsboot code o kern/143825 fs [nfs] [panic] Kernel panic on NFS client o bin/143572 fs [zfs] zpool(1): [patch] The verbose output from iostat o kern/143212 fs [nfs] NFSv4 client strange work ... o kern/143184 fs [zfs] [lor] zfs/bufwait LOR o kern/142878 fs [zfs] [vfs] lock order reversal o kern/142597 fs [ext2fs] ext2fs does not work on filesystems with real o kern/142489 fs [zfs] [lor] allproc/zfs LOR o kern/142466 fs Update 7.2 -> 8.0 on Raid 1 ends with screwed raid [re o kern/142306 fs [zfs] [panic] ZFS drive (from OSX Leopard) causes two o kern/142068 fs [ufs] BSD labels are got deleted spontaneously o kern/141897 fs [msdosfs] [panic] Kernel panic. msdofs: file name leng o kern/141463 fs [nfs] [panic] Frequent kernel panics after upgrade fro o kern/141305 fs [zfs] FreeBSD ZFS+sendfile severe performance issues ( o kern/141091 fs [patch] [nullfs] fix panics with DIAGNOSTIC enabled o kern/141086 fs [nfs] [panic] panic("nfs: bioread, not dir") on FreeBS o kern/141010 fs [zfs] "zfs scrub" fails when backed by files in UFS2 o kern/140888 fs [zfs] boot fail from zfs root while the pool resilveri o kern/140661 fs [zfs] [patch] /boot/loader fails to work on a GPT/ZFS- o kern/140640 fs [zfs] snapshot crash o kern/140068 fs [smbfs] [patch] smbfs does not allow semicolon in file o kern/139725 fs [zfs] zdb(1) dumps core on i386 when examining zpool c o kern/139715 fs [zfs] vfs.numvnodes leak on busy zfs p bin/139651 fs [nfs] mount(8): read-only remount of NFS volume does n o kern/139564 fs [zfs] [panic] 8.0-RC1 - Fatal trap 12 at end of shutdo o kern/139407 fs [smbfs] [panic] smb mount causes system crash if remot o kern/138662 fs [panic] ffs_blkfree: freeing free block o kern/138421 fs [ufs] [patch] remove UFS label limitations o kern/138202 fs mount_msdosfs(1) see only 2Gb o kern/136968 fs [ufs] [lor] ufs/bufwait/ufs (open) o kern/136945 fs [ufs] [lor] filedesc structure/ufs (poll) o kern/136944 fs [ffs] [lor] bufwait/snaplk (fsync) o kern/136873 fs [ntfs] Missing directories/files on NTFS volume o kern/136865 fs [nfs] [patch] NFS exports atomic and on-the-fly atomic p kern/136470 fs [nfs] Cannot mount / in read-only, over NFS o kern/135546 fs [zfs] zfs.ko module doesn't ignore zpool.cache filenam o kern/135469 fs [ufs] [panic] kernel crash on md operation in ufs_dirb o kern/135050 fs [zfs] ZFS clears/hides disk errors on reboot o kern/134491 fs [zfs] Hot spares are rather cold... o kern/133676 fs [smbfs] [panic] umount -f'ing a vnode-based memory dis o kern/132960 fs [ufs] [panic] panic:ffs_blkfree: freeing free frag o kern/132397 fs reboot causes filesystem corruption (failure to sync b o kern/132331 fs [ufs] [lor] LOR ufs and syncer o kern/132237 fs [msdosfs] msdosfs has problems to read MSDOS Floppy o kern/132145 fs [panic] File System Hard Crashes o kern/131441 fs [unionfs] [nullfs] unionfs and/or nullfs not combineab o kern/131360 fs [nfs] poor scaling behavior of the NFS server under lo o kern/131342 fs [nfs] mounting/unmounting of disks causes NFS to fail o bin/131341 fs makefs: error "Bad file descriptor" on the mount poin o kern/130920 fs [msdosfs] cp(1) takes 100% CPU time while copying file o kern/130210 fs [nullfs] Error by check nullfs o kern/129760 fs [nfs] after 'umount -f' of a stale NFS share FreeBSD l o kern/129488 fs [smbfs] Kernel "bug" when using smbfs in smbfs_smb.c: o kern/129231 fs [ufs] [patch] New UFS mount (norandom) option - mostly o kern/129152 fs [panic] non-userfriendly panic when trying to mount(8) o kern/127787 fs [lor] [ufs] Three LORs: vfslock/devfs/vfslock, ufs/vfs o bin/127270 fs fsck_msdosfs(8) may crash if BytesPerSec is zero o kern/127029 fs [panic] mount(8): trying to mount a write protected zi o kern/126287 fs [ufs] [panic] Kernel panics while mounting an UFS file o kern/125895 fs [ffs] [panic] kernel: panic: ffs_blkfree: freeing free s kern/125738 fs [zfs] [request] SHA256 acceleration in ZFS o kern/123939 fs [msdosfs] corrupts new files o kern/122380 fs [ffs] ffs_valloc:dup alloc (Soekris 4801/7.0/USB Flash o bin/122172 fs [fs]: amd(8) automount daemon dies on 6.3-STABLE i386, o bin/121898 fs [nullfs] pwd(1)/getcwd(2) fails with Permission denied o bin/121072 fs [smbfs] mount_smbfs(8) cannot normally convert the cha o kern/120483 fs [ntfs] [patch] NTFS filesystem locking changes o kern/120482 fs [ntfs] [patch] Sync style changes between NetBSD and F o kern/118912 fs [2tb] disk sizing/geometry problem with large array o kern/118713 fs [minidump] [patch] Display media size required for a k o kern/118318 fs [nfs] NFS server hangs under special circumstances o bin/118249 fs [ufs] mv(1): moving a directory changes its mtime o kern/118126 fs [nfs] [patch] Poor NFS server write performance o kern/118107 fs [ntfs] [panic] Kernel panic when accessing a file at N o kern/117954 fs [ufs] dirhash on very large directories blocks the mac o bin/117315 fs [smbfs] mount_smbfs(8) and related options can't mount o kern/117158 fs [zfs] zpool scrub causes panic if geli vdevs detach on o bin/116980 fs [msdosfs] [patch] mount_msdosfs(8) resets some flags f o conf/116931 fs lack of fsck_cd9660 prevents mounting iso images with o kern/116583 fs [ffs] [hang] System freezes for short time when using o bin/115361 fs [zfs] mount(8) gets into a state where it won't set/un o kern/114955 fs [cd9660] [patch] [request] support for mask,dirmask,ui o kern/114847 fs [ntfs] [patch] [request] dirmask support for NTFS ala o kern/114676 fs [ufs] snapshot creation panics: snapacct_ufs2: bad blo o bin/114468 fs [patch] [request] add -d option to umount(8) to detach o kern/113852 fs [smbfs] smbfs does not properly implement DFS referral o bin/113838 fs [patch] [request] mount(8): add support for relative p o bin/113049 fs [patch] [request] make quot(8) use getopt(3) and show o kern/112658 fs [smbfs] [patch] smbfs and caching problems (resolves b o kern/111843 fs [msdosfs] Long Names of files are incorrectly created o kern/111782 fs [ufs] dump(8) fails horribly for large filesystems s bin/111146 fs [2tb] fsck(8) fails on 6T filesystem o bin/107829 fs [2TB] fdisk(8): invalid boundary checking in fdisk / w o kern/106107 fs [ufs] left-over fsck_snapshot after unfinished backgro o kern/104406 fs [ufs] Processes get stuck in "ufs" state under persist o kern/104133 fs [ext2fs] EXT2FS module corrupts EXT2/3 filesystems o kern/103035 fs [ntfs] Directories in NTFS mounted disc images appear o kern/101324 fs [smbfs] smbfs sometimes not case sensitive when it's s o kern/99290 fs [ntfs] mount_ntfs ignorant of cluster sizes s bin/97498 fs [request] newfs(8) has no option to clear the first 12 o kern/97377 fs [ntfs] [patch] syntax cleanup for ntfs_ihash.c o kern/95222 fs [cd9660] File sections on ISO9660 level 3 CDs ignored o kern/94849 fs [ufs] rename on UFS filesystem is not atomic o bin/94810 fs fsck(8) incorrectly reports 'file system marked clean' o kern/94769 fs [ufs] Multiple file deletions on multi-snapshotted fil o kern/94733 fs [smbfs] smbfs may cause double unlock o kern/93942 fs [vfs] [patch] panic: ufs_dirbad: bad dir (patch from D o kern/92272 fs [ffs] [hang] Filling a filesystem while creating a sna o kern/91134 fs [smbfs] [patch] Preserve access and modification time a kern/90815 fs [smbfs] [patch] SMBFS with character conversions somet o kern/88657 fs [smbfs] windows client hang when browsing a samba shar o kern/88555 fs [panic] ffs_blkfree: freeing free frag on AMD 64 o kern/88266 fs [smbfs] smbfs does not implement UIO_NOCOPY and sendfi o bin/87966 fs [patch] newfs(8): introduce -A flag for newfs to enabl o kern/87859 fs [smbfs] System reboot while umount smbfs. o kern/86587 fs [msdosfs] rm -r /PATH fails with lots of small files o bin/85494 fs fsck_ffs: unchecked use of cg_inosused macro etc. o kern/80088 fs [smbfs] Incorrect file time setting on NTFS mounted vi o bin/74779 fs Background-fsck checks one filesystem twice and omits o kern/73484 fs [ntfs] Kernel panic when doing `ls` from the client si o bin/73019 fs [ufs] fsck_ufs(8) cannot alloc 607016868 bytes for ino o kern/71774 fs [ntfs] NTFS cannot "see" files on a WinXP filesystem o bin/70600 fs fsck(8) throws files away when it can't grow lost+foun o kern/68978 fs [panic] [ufs] crashes with failing hard disk, loose po o kern/65920 fs [nwfs] Mounted Netware filesystem behaves strange o kern/65901 fs [smbfs] [patch] smbfs fails fsx write/truncate-down/tr o kern/61503 fs [smbfs] mount_smbfs does not work as non-root o kern/55617 fs [smbfs] Accessing an nsmb-mounted drive via a smb expo o kern/51685 fs [hang] Unbounded inode allocation causes kernel to loc o kern/36566 fs [smbfs] System reboot with dead smb mount and umount o bin/27687 fs fsck(8) wrapper is not properly passing options to fsc o kern/18874 fs [2TB] 32bit NFS servers export wrong negative values t 280 problems total. From owner-freebsd-fs@FreeBSD.ORG Mon Jul 2 12:12:30 2012 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 60A1D106566B; Mon, 2 Jul 2012 12:12:30 +0000 (UTC) (envelope-from markus.gebert@hostpoint.ch) Received: from mail.adm.hostpoint.ch (mail.adm.hostpoint.ch [IPv6:2a00:d70:0:a::e0]) by mx1.freebsd.org (Postfix) with ESMTP id E7F1C8FC12; Mon, 2 Jul 2012 12:12:29 +0000 (UTC) Received: from [2001:1620:2013:1:7163:2c2b:3e7f:1ce9] (port=49496) by mail.adm.hostpoint.ch with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.69 (FreeBSD)) (envelope-from ) id 1SlfUS-0000ZI-Lq; Mon, 02 Jul 2012 14:12:28 +0200 Mime-Version: 1.0 (Apple Message framework v1278) Content-Type: text/plain; charset=iso-8859-1 From: Markus Gebert In-Reply-To: <0EF22D00-1CDB-44DA-BF3C-3AC672072C05@hostpoint.ch> Date: Mon, 2 Jul 2012 14:12:26 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <201206251443.41768.jhb@freebsd.org> <201206270810.28589.jhb@freebsd.org> <0EF22D00-1CDB-44DA-BF3C-3AC672072C05@hostpoint.ch> To: John Baldwin X-Mailer: Apple Mail (2.1278) Cc: fs@freebsd.org Subject: Re: [PATCH] Simple ARC stats in top X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2012 12:12:30 -0000 Hi John On 27.06.2012, at 17:49, Markus Gebert wrote: > On 27.06.2012, at 14:10, John Baldwin wrote: >=20 >> Hmmm, my patch is relative to a top that has -P backported and I = can't=20 >> reproduce this, the ARC lines moves around properly if 'P' is toggled >> at runtime as well for me. Did you have any conflicts in your patch >> that you had to resolve by hand? >=20 > No, I did not merge anything by hand. But what I did, and that = probably was the cause, is only buildworld with NO_CLEAN=3D1 to save = time and just rebuild top. Now that I've done a full buildworld, the = probem seems to be gone. I'll test more and step up, if it happens = again, but consider this solved for now. Sorry for the noise. The problem is back. I don't know why I thought it was gone last week, = maybe I looked at the wrong terminal or something. Anyway, I tried again today, and I can reproduce this not only on 8.3 = but also on 9.0. To rule out a problem with iTerm I also tried OS X = Terminal and Gnome Terminal on Ubuntu. The terminal does not seem to = matter at all. This sample output is from a patched top (run with -S -P) on a 9.0 = server connected to via ssh: ---- last pid: 92857; load averages: 0.01, 0.01, 0.03 up 4+13:49:12 = 13:22:17 53 processes: 2 running, 50 sleeping, 1 waiting CPU 0: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle CPU 171 0.0% user, 50.0% nice, 0.0% system04 0.0% inter19pt, 100% idle CPU 2: 0.0% user, 020% nice10 0.0MFU, 1296K 0.0n, 42M Header 1006M = Other CPU 3: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle Mem: 66M Active, 154M Inact, 3174M Wired, 3152K Cache, 525M Free ARC: 2503M Total, 1191M MRU, 1003M MFU, 1680K Anon, 42M Header, 266M = Other Swap: 8192M Total, 11M Used, 8181M Free ---- After starting top, the first display is always correct. But as soon as = ARC values change, the new ones get written to a wrong line, in this = case CPU1 or CPU2. And the numbers on the ARC line don't seem to get = updated anymore. If I SHIFT-P twice, the display refreshes just fine, = but as soon as there are changes for the ARC values, things start to get = corrupted again. Markus From owner-freebsd-fs@FreeBSD.ORG Mon Jul 2 14:22:25 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AEFE4106566B; Mon, 2 Jul 2012 14:22:25 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) by mx1.freebsd.org (Postfix) with ESMTP id DE64B8FC18; Mon, 2 Jul 2012 14:22:24 +0000 (UTC) Received: by lbon10 with SMTP id n10so9508513lbo.13 for ; Mon, 02 Jul 2012 07:22:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=tAHkgVw5kJ3r56w+xZocmo5AiUeVl2u9KFOnFTLLl6A=; b=YLVQMbSm5FwuQ6KR7SzFqQmZaUNt9U2GWxb4BT5Jv9Df6QwpSc9ZdnCClfKx/wtScA gjJ7MOio1XF3Yg/HDioHX9Od8qp6VPL7fGPlzViBCMufE0db5HCP3qzzS1YqbO5gYhLU dg4Jb5IHBII1jyJBRSw2bSTbgOmdM4N4r66Ec3D7rh+83gNK7n5fxjGnA7rN5tYQnUtY iFo5ReKEy8z2GYLxWyL6Ebr/oG4tKaRLmPkbUPMuV5gQkSILDoP2Srkz1nCVmALDWt1A e2IFxgIENOqopreMI83eXDWPtWH3/WqvmUSR/pG0vmxIruf11kcpCapojB+wdupgLNkw Jiiw== MIME-Version: 1.0 Received: by 10.152.131.9 with SMTP id oi9mr13062883lab.39.1341238943768; Mon, 02 Jul 2012 07:22:23 -0700 (PDT) Sender: asmrookie@gmail.com Received: by 10.112.27.65 with HTTP; Mon, 2 Jul 2012 07:22:23 -0700 (PDT) In-Reply-To: <20120702061219.GA16671@infradead.org> References: <20120702061219.GA16671@infradead.org> Date: Mon, 2 Jul 2012 15:22:23 +0100 X-Google-Sender-Auth: LXn3NQyyCUDaSY6EFW8aBfNtzPI Message-ID: From: Attilio Rao To: Christoph Hellwig Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD FS , Russell Cattelan , freebsd-current@freebsd.org, "C. P. Ghost" Subject: Re: MPSAFE VFS -- List of upcoming actions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2012 14:22:25 -0000 2012/7/2, Christoph Hellwig : > On Sun, Jul 01, 2012 at 03:52:05PM +0200, Attilio Rao wrote: >> anything by SoC involved people about NTFS and certainly I don't see a >> plan to get XFS locked. > > Stupid question, but what amount of locking does XFS in FreeBSD still > need? I'm one of the maintainer of XFS on Linux, and while I know > FreeBSD imported a really old version compared to the current one the > codebases on IRIX and later Linux never relied on any global Giant-style > locking. So if there is anything to fix it would be the in the small > bits of FreeBSD-specific code. Basically when it cames to make a MPSAFE filesystem under FreeBSD there are 2 things to pay attention at: filesystem specific locking and dealing with FreeBSD's VFS locking. The former is usually the tricky part because it varies among the filesystems and it is where the developers might have more knowledge. The latter can be helped by testing with a debugging kernel for low hanging fruits, but special attention must be put in things like avoid to put half-constructed vnodes in the mount lists, lookup races (in particular with DOTDOT case) and others. For a reference, one can always look to simple filesystems that are already made MPSAFE (like MSDOS-FS likely). In the XFS case, I think it would be desirable to have a real maintainership. This means, basically, not only work on the locking but really be keen to have a working XFS. At the moment, we might still have write support as well, but it is badly broken. What I suggest for XFS is: - Remove the current write support entirely - Try to bring the sole read support to new(ish) XFS version (at least to a version that is known to not be totally broken) - Fix up the support to work with FreeBSD VFS Thanks, Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-fs@FreeBSD.ORG Mon Jul 2 14:33:51 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 11D37106566B; Mon, 2 Jul 2012 14:33:51 +0000 (UTC) (envelope-from cattelan@thebarn.com) Received: from x.digitalelves.com (x.digitalelves.com [209.98.77.55]) by mx1.freebsd.org (Postfix) with ESMTP id AA8608FC14; Mon, 2 Jul 2012 14:33:50 +0000 (UTC) Received: from Russells-Lion-Hackintosh.local (c-66-41-26-220.hsd1.mn.comcast.net [66.41.26.220]) (authenticated bits=0) by x.digitalelves.com (8.14.5/8.14.5) with ESMTP id q62EDGqZ071770 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Mon, 2 Jul 2012 09:13:19 -0500 (CDT) (envelope-from cattelan@thebarn.com) Message-ID: <4FF1AC74.7020205@thebarn.com> Date: Mon, 02 Jul 2012 09:13:08 -0500 From: Russell Cattelan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:13.0) Gecko/20120614 Thunderbird/13.0.1 MIME-Version: 1.0 To: Attilio Rao , "C. P. Ghost" , FreeBSD FS References: <20120702061219.GA16671@infradead.org> In-Reply-To: <20120702061219.GA16671@infradead.org> X-Enigmail-Version: 1.4.2 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig8E2EBB2A9B6D2F149D45CFED" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-current@freebsd.org Subject: Re: MPSAFE VFS -- List of upcoming actions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2012 14:33:51 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig8E2EBB2A9B6D2F149D45CFED Content-Type: multipart/mixed; boundary="------------090609090007020402060209" This is a multi-part message in MIME format. --------------090609090007020402060209 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 7/2/12 1:12 AM, Christoph Hellwig wrote: > On Sun, Jul 01, 2012 at 03:52:05PM +0200, Attilio Rao wrote: >> anything by SoC involved people about NTFS and certainly I don't see a= >> plan to get XFS locked. >=20 > Stupid question, but what amount of locking does XFS in FreeBSD still > need? I'm one of the maintainer of XFS on Linux, and while I know > FreeBSD imported a really old version compared to the current one the > codebases on IRIX and later Linux never relied on any global Giant-styl= e > locking. So if there is anything to fix it would be the in the small > bits of FreeBSD-specific code. >=20 I would be curious as well. Since I'm one of the people that has done the port of XFS to FreeBSD I'm wondering what this whole MP initiative with regards to filesystems is about. XFS certainly maintained any fine grain locking inherent to XFS it self. The XFS locks were mapped to FreeBSD sx locks in the code that is in the tree currently. The last time I made a pass at updating XFS some of locks were remapped to mtx locks. Is there somethings in the vfs layer in terms of locking that needs to be pushed down to the fs? -Russell --------------090609090007020402060209-- --------------enig8E2EBB2A9B6D2F149D45CFED Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk/xrHwACgkQNRmM+OaGhBidhwCfdDBRjUokO9eYrvB2goE3G7TS e1sAnjxqc7Co5rK2AbxD5r/R4L3GVM5E =hb9/ -----END PGP SIGNATURE----- --------------enig8E2EBB2A9B6D2F149D45CFED-- From owner-freebsd-fs@FreeBSD.ORG Mon Jul 2 14:35:59 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9D919106564A; Mon, 2 Jul 2012 14:35:59 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-bk0-f54.google.com (mail-bk0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id F0DE48FC0C; Mon, 2 Jul 2012 14:35:58 +0000 (UTC) Received: by bkcje9 with SMTP id je9so214027bkc.13 for ; Mon, 02 Jul 2012 07:35:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=EUgf1rOpNUuBy/2Mhn/BawTLANAf2bZT7bMS47BEVyw=; b=IsTZei13hEcHhBIBwRhwOL7ugfwihn5dilBQglpyX5CDGoqdY/GQgOQhNgE07rA9cP ZY1rMHGkCPdbHbk7B3Q4/LUWlFJ59p6rwfSBAgZqmNRXQzAnc18K1/YzU6+eFq49Uk8n +p9bQqfljasEWCn1aV9+cJrFiTGR9f9WJdYhq1cVOUS+HVLraUDEDfmb1sWxccAhuACF PhlhyiI/UuK8kgH5H2eYCWV7GfZGuZUUXTt1o+2t4qERPmcJxbCTNiIk/beOxeaXPUaG rCxIUhbTJE2ftLQztITI9M+oTS68CYtnTdKpw+1+cr1CpVhJt630f9bPnyD5B3eF5Pp4 if2w== MIME-Version: 1.0 Received: by 10.152.136.18 with SMTP id pw18mr13232556lab.17.1341239757762; Mon, 02 Jul 2012 07:35:57 -0700 (PDT) Sender: asmrookie@gmail.com Received: by 10.112.27.65 with HTTP; Mon, 2 Jul 2012 07:35:57 -0700 (PDT) In-Reply-To: <4FF1AC74.7020205@thebarn.com> References: <20120702061219.GA16671@infradead.org> <4FF1AC74.7020205@thebarn.com> Date: Mon, 2 Jul 2012 15:35:57 +0100 X-Google-Sender-Auth: nTOnsVR6fsqbde3UcZHs-9PSWwU Message-ID: From: Attilio Rao To: Russell Cattelan Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD FS , freebsd-current@freebsd.org, "C. P. Ghost" Subject: Re: MPSAFE VFS -- List of upcoming actions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2012 14:35:59 -0000 2012/7/2, Russell Cattelan : > On 7/2/12 1:12 AM, Christoph Hellwig wrote: >> On Sun, Jul 01, 2012 at 03:52:05PM +0200, Attilio Rao wrote: >>> anything by SoC involved people about NTFS and certainly I don't see a >>> plan to get XFS locked. >> >> Stupid question, but what amount of locking does XFS in FreeBSD still >> need? I'm one of the maintainer of XFS on Linux, and while I know >> FreeBSD imported a really old version compared to the current one the >> codebases on IRIX and later Linux never relied on any global Giant-style >> locking. So if there is anything to fix it would be the in the small >> bits of FreeBSD-specific code. >> > I would be curious as well. Since I'm one of the people that has done > the port of XFS to FreeBSD I'm wondering what this whole MP initiative > with regards to filesystems is about. So if you think that XFS doesn't need to acquire Giant why it is not yet marked MPSAFE? Also, did you really ever actually tested write support? (Not sure if it was added to you). Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-fs@FreeBSD.ORG Mon Jul 2 15:03:45 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D5FFF106566B; Mon, 2 Jul 2012 15:03:45 +0000 (UTC) (envelope-from kabaev@gmail.com) Received: from mail-qc0-f182.google.com (mail-qc0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 560408FC14; Mon, 2 Jul 2012 15:03:45 +0000 (UTC) Received: by qcsg15 with SMTP id g15so3402741qcs.13 for ; Mon, 02 Jul 2012 08:03:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=kDkWVGa3N+2gt5YKidDiC1CmwxYnsun92iAZTZJbBh4=; b=rhdJoMce//KjShpj72Cee4o2cJh7xTMaXZt+Y1IIcSSYdNDf4bvmzlGRViFLZ/CVbh i5S1GNiPXZWanJZbHOnvA/5gvPZBBwe7VhM2ZTHJ1ssi/6NDc/iD/g9c5txr4pkTkVdv OtoEhWr8DuLNxK8Pe7YpUnX6iTBtIr7tcQL2n0zgJrcb5m/S4+bjLVzDEWdvslcvZTkC miQ5M1eeGBq/MbRpitIHjVB3c7of6kFP68LCcOpaNI1TiH6A/hEDmA9ZLCsqLTdltNDR Mh7IKLRjwLDKcUYcre/AMbLGNp1MVcdWgs3sGgNf42VyLy4tAU7I8e3hHODA920EvQLU k6tA== Received: by 10.229.135.18 with SMTP id l18mr6862575qct.156.1341241424614; Mon, 02 Jul 2012 08:03:44 -0700 (PDT) Received: from kan.dyndns.org (c-24-63-226-98.hsd1.ma.comcast.net. [24.63.226.98]) by mx.google.com with ESMTPS id dz4sm31724960qab.4.2012.07.02.08.03.42 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 02 Jul 2012 08:03:43 -0700 (PDT) Date: Mon, 2 Jul 2012 11:03:40 -0400 From: Alexander Kabaev To: Christoph Hellwig Message-ID: <20120702150339.GA7226@kan.dyndns.org> References: <20120702061219.GA16671@infradead.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="EVF5PPMfhYS0aIcm" Content-Disposition: inline In-Reply-To: <20120702061219.GA16671@infradead.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Attilio Rao , FreeBSD FS , freebsd-current@freebsd.org, "C. P. Ghost" , Russell Cattelan Subject: Re: MPSAFE VFS -- List of upcoming actions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2012 15:03:46 -0000 --EVF5PPMfhYS0aIcm Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jul 02, 2012 at 02:12:20AM -0400, Christoph Hellwig wrote: > On Sun, Jul 01, 2012 at 03:52:05PM +0200, Attilio Rao wrote: > > anything by SoC involved people about NTFS and certainly I don't see a > > plan to get XFS locked. >=20 > Stupid question, but what amount of locking does XFS in FreeBSD still > need? I'm one of the maintainer of XFS on Linux, and while I know > FreeBSD imported a really old version compared to the current one the > codebases on IRIX and later Linux never relied on any global Giant-style > locking. So if there is anything to fix it would be the in the small > bits of FreeBSD-specific code. >=20 When I stopped being interested in XFS, I left is marked as non-MPSAFE entirely because of the lack of proper testing and because VFS locking was still evolving, there was no officially proper way of locking the FS and no other FS in the tree was MPSAFE. At that time the only problematic area was around inode instantiation, but sereval other lockingi changes have made it into the tree since then, namely ones that deal with insmntque and also VOP_LOOKUP changes. To mark XFS MPSAFE, one needs to simply audit the code and make sure it still makse sense for today= 's VFS, which is not a huge amount of work. One step further woule be to take most of the XFS from under the exclusive vnode locking to improve the performance. And there is a third option - just let current XFS port die and start with newer version. --=20 Alexander Kabaev --EVF5PPMfhYS0aIcm Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iD8DBQFP8bhLQ6z1jMm+XZYRAhpNAJ9/wX+/YBqya26vJdUdSl+NrlyOjwCg3K2J ursmLw9qFXWb8eIvglCWxJI= =q7N/ -----END PGP SIGNATURE----- --EVF5PPMfhYS0aIcm-- From owner-freebsd-fs@FreeBSD.ORG Mon Jul 2 15:04:37 2012 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4E1D8106566C for ; Mon, 2 Jul 2012 15:04:37 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 23D4B8FC19 for ; Mon, 2 Jul 2012 15:04:37 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 80BE8B96D; Mon, 2 Jul 2012 11:04:36 -0400 (EDT) From: John Baldwin To: Markus Gebert Date: Mon, 2 Jul 2012 10:50:13 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p17; KDE/4.5.5; amd64; ; ) References: <201206251443.41768.jhb@freebsd.org> <0EF22D00-1CDB-44DA-BF3C-3AC672072C05@hostpoint.ch> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201207021050.13974.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 02 Jul 2012 11:04:36 -0400 (EDT) Cc: fs@freebsd.org Subject: Re: [PATCH] Simple ARC stats in top X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2012 15:04:37 -0000 On Monday, July 02, 2012 8:12:26 am Markus Gebert wrote: > Hi John > > On 27.06.2012, at 17:49, Markus Gebert wrote: > > > On 27.06.2012, at 14:10, John Baldwin wrote: > > > >> Hmmm, my patch is relative to a top that has -P backported and I can't > >> reproduce this, the ARC lines moves around properly if 'P' is toggled > >> at runtime as well for me. Did you have any conflicts in your patch > >> that you had to resolve by hand? > > > > No, I did not merge anything by hand. But what I did, and that probably was the cause, is only buildworld with NO_CLEAN=1 to save time and just rebuild top. Now that I've done a full buildworld, the probem seems to be gone. I'll test more and step up, if it happens again, but consider this solved for now. Sorry for the noise. > > The problem is back. I don't know why I thought it was gone last week, maybe I looked at the wrong terminal or something. > > Anyway, I tried again today, and I can reproduce this not only on 8.3 but also on 9.0. To rule out a problem with iTerm I also tried OS X Terminal and Gnome Terminal on Ubuntu. The terminal does not seem to matter at all. > > This sample output is from a patched top (run with -S -P) on a 9.0 server connected to via ssh: I found it. I wasn't updating y_arc for PCPU stats. Try this: Index: usr.bin/top/machine.c =================================================================== --- machine.c (revision 225593) +++ machine.c (working copy) @@ -263,6 +263,7 @@ update_layout(void) { y_mem = 3; + y_arc = 4; y_swap = 4 + arc_enabled; y_idlecursor = 5 + arc_enabled; y_message = 5 + arc_enabled; @@ -271,7 +272,8 @@ update_layout(void) Header_lines = 7 + arc_enabled; if (pcpu_stats) { - y_mem = ncpus - 1; + y_mem += ncpus - 1; + y_arc += ncpus - 1; y_swap += ncpus - 1; y_idlecursor += ncpus - 1; y_message += ncpus - 1; -- John Baldwin From owner-freebsd-fs@FreeBSD.ORG Mon Jul 2 18:02:47 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 33F371065677 for ; Mon, 2 Jul 2012 18:02:47 +0000 (UTC) (envelope-from rar102@ra.msstate.edu) Received: from catalpa.its.msstate.edu (catalpa.its.msstate.edu [130.18.2.119]) by mx1.freebsd.org (Postfix) with ESMTP id AFDEF8FC12 for ; Mon, 2 Jul 2012 18:02:46 +0000 (UTC) Received: from riehlepc.localnet (ws62-55.hilbun.dynamic.msstate.edu [130.18.55.62]) (authenticated bits=0) by catalpa.its.msstate.edu (8.13.8/8.13.8) with ESMTP id q62HxrtY008942 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Mon, 2 Jul 2012 12:59:54 -0500 X-Sender: From: "R. Riehle" Organization: Miss. State Dept. of Physics & Astronomy To: freebsd-fs@freebsd.org Date: Mon, 2 Jul 2012 12:59:49 -0500 User-Agent: KMail/1.13.7 (FreeBSD/8.3-STABLE; KDE/4.8.4; amd64; ; ) MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-Id: <201207021259.49364.rar102@ra.msstate.edu> Subject: usb3 zpool disappears X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2012 18:02:47 -0000 I have an amd64 machine running freebsd 8-stable with three zpools, two on internal drives and one on an external Western Digital usb3 drive. When I rebuilt my world last week the pool on the external drive refused to mount. The other two pools are fine. The drive and the usb3 card seem to be intact. Some failed attempts to solve the problem: 1. zfs mount -a No response but doesn't hang. 2. zpool mount bigpool hangs 3. zpool status hangs forever. I waited for 24 hours. No instability. No logs. No appreciable cpu usage. It just stays there. 4. zfs status See #3 5. zpool mount -f bigpool hangs indefinitely just like #2, 3 Snippets from dmesg: : xhci0: mem 0xfeaf0000-0xfeafffff,0xfeaee000-0xfeaeffff irq 17 at device 0.0 on pci4 xhci0: [ITHREAD] xhci0: 64 byte context size. : ugen4.2: at usbus4 umass0: on usbus4 umass0: SCSI over Bulk-Only; quirks = 0x4100 umass0:2:0:-1: Attached to scbus2 da0 at umass-sim0 bus 0 scbus2 target 0 lun 0 da0: Fixed Direct Access SCSI-0 device da0: 400.000MB/s transfers da0: 953869MB (1953525167 512 byte sectors: 255H 63S/T 121601C) ~ $ uname -a FreeBSD xxxxxxx 8.3-STABLE FreeBSD 8.3-STABLE #0: Fri Jun 22 17:08:30 CDT 2012 xxxxxx@xxxxxxx:/usr/obj/usr/src/sys/CUSTOM amd64 google shows others have had the same problem but I haven't found any solutions. Any insights??? Robertsen A. Riehle From owner-freebsd-fs@FreeBSD.ORG Mon Jul 2 20:12:12 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6A1341065673; Mon, 2 Jul 2012 20:12:12 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 2E39C8FC1A; Mon, 2 Jul 2012 20:12:10 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q62KCJ8A033019; Mon, 2 Jul 2012 23:12:19 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q62KC77s099146; Mon, 2 Jul 2012 23:12:07 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q62KC6PH099145; Mon, 2 Jul 2012 23:12:06 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 2 Jul 2012 23:12:06 +0300 From: Konstantin Belousov To: Alexander Kabaev Message-ID: <20120702201206.GV2337@deviant.kiev.zoral.com.ua> References: <20120702061219.GA16671@infradead.org> <20120702150339.GA7226@kan.dyndns.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="0l0ctNieuEHM0Mkh" Content-Disposition: inline In-Reply-To: <20120702150339.GA7226@kan.dyndns.org> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: Attilio Rao , "C. P. Ghost" , Russell Cattelan , freebsd-current@freebsd.org, FreeBSD FS Subject: Re: MPSAFE VFS -- List of upcoming actions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2012 20:12:12 -0000 --0l0ctNieuEHM0Mkh Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Jul 02, 2012 at 11:03:40AM -0400, Alexander Kabaev wrote: > On Mon, Jul 02, 2012 at 02:12:20AM -0400, Christoph Hellwig wrote: > > On Sun, Jul 01, 2012 at 03:52:05PM +0200, Attilio Rao wrote: > > > anything by SoC involved people about NTFS and certainly I don't see a > > > plan to get XFS locked. > >=20 > > Stupid question, but what amount of locking does XFS in FreeBSD still > > need? I'm one of the maintainer of XFS on Linux, and while I know > > FreeBSD imported a really old version compared to the current one the > > codebases on IRIX and later Linux never relied on any global Giant-style > > locking. So if there is anything to fix it would be the in the small > > bits of FreeBSD-specific code. > >=20 >=20 > When I stopped being interested in XFS, I left is marked as non-MPSAFE > entirely because of the lack of proper testing and because VFS locking > was still evolving, there was no officially proper way of locking the > FS and no other FS in the tree was MPSAFE. At that time the only > problematic area was around inode instantiation, but sereval other > lockingi changes have made it into the tree since then, namely ones that > deal with insmntque and also VOP_LOOKUP changes. To mark XFS MPSAFE, one > needs to simply audit the code and make sure it still makse sense for tod= ay's > VFS, which is not a huge amount of work. One step further woule be to take > most of the XFS from under the exclusive vnode locking to improve the > performance. If filesystem uses some global internal locks, that locks usually are placed after the vnode locks in global lock order, because VOP methods call into fs with vnode locked. Then, VOP_LOOKUP() usual sequence of events, when method is called with lookup directory locked, causes LOR. It appears because you lock global lock upon entry into VOP_LOOKUP(), and then need to lock the returned vnode. Dropping global lock inside VOP_LOOKUP() usually exposes races which were the reason to introduce the global lock. Having filesystem non-MPSAFE makes the LOR go away without the need to drop global lock. Example of FreeBSD native filesystem which suffered from this issue and required quite non-trivial handling is devfs. Devfs uses per-mount global lock. See devfs_allocv() and devfs_allocv_drop_refs() for the gory details. --0l0ctNieuEHM0Mkh Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAk/yAJYACgkQC3+MBN1Mb4jGHQCfbUFjkOyR04wK/OubK1LWcRW/ hZsAoIk6tJQ2niBpuL3/3OmDFvfLL9hq =lP1N -----END PGP SIGNATURE----- --0l0ctNieuEHM0Mkh-- From owner-freebsd-fs@FreeBSD.ORG Mon Jul 2 21:58:29 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B29451065670; Mon, 2 Jul 2012 21:58:29 +0000 (UTC) (envelope-from cattelan@thebarn.com) Received: from x.digitalelves.com (x.digitalelves.com [209.98.77.55]) by mx1.freebsd.org (Postfix) with ESMTP id 79AED8FC17; Mon, 2 Jul 2012 21:58:29 +0000 (UTC) Received: from Russells-Lion-Hackintosh.local (c-66-41-26-220.hsd1.mn.comcast.net [66.41.26.220]) (authenticated bits=0) by x.digitalelves.com (8.14.5/8.14.5) with ESMTP id q62LwOof083241 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Mon, 2 Jul 2012 16:58:26 -0500 (CDT) (envelope-from cattelan@thebarn.com) Message-ID: <4FF21980.2090607@thebarn.com> Date: Mon, 02 Jul 2012 16:58:24 -0500 From: Russell Cattelan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:13.0) Gecko/20120614 Thunderbird/13.0.1 MIME-Version: 1.0 To: Alexander Kabaev References: <20120702061219.GA16671@infradead.org> <20120702150339.GA7226@kan.dyndns.org> In-Reply-To: <20120702150339.GA7226@kan.dyndns.org> X-Enigmail-Version: 1.4.2 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig63054884B80B5C3C56A7DE05" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Attilio Rao , freebsd-current@freebsd.org, "C. P. Ghost" , FreeBSD FS Subject: Re: MPSAFE VFS -- List of upcoming actions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2012 21:58:29 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig63054884B80B5C3C56A7DE05 Content-Type: multipart/mixed; boundary="------------030107020906030202000708" This is a multi-part message in MIME format. --------------030107020906030202000708 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 7/2/12 10:03 AM, Alexander Kabaev wrote: > On Mon, Jul 02, 2012 at 02:12:20AM -0400, Christoph Hellwig wrote: >> On Sun, Jul 01, 2012 at 03:52:05PM +0200, Attilio Rao wrote: >>> anything by SoC involved people about NTFS and certainly I don't see = a >>> plan to get XFS locked. >> >> Stupid question, but what amount of locking does XFS in FreeBSD still >> need? I'm one of the maintainer of XFS on Linux, and while I know >> FreeBSD imported a really old version compared to the current one the >> codebases on IRIX and later Linux never relied on any global Giant-sty= le >> locking. So if there is anything to fix it would be the in the small >> bits of FreeBSD-specific code. >> >=20 > When I stopped being interested in XFS, I left is marked as non-MPSAFE > entirely because of the lack of proper testing and because VFS locking > was still evolving, there was no officially proper way of locking the > FS and no other FS in the tree was MPSAFE. At that time the only > problematic area was around inode instantiation, but sereval other > lockingi changes have made it into the tree since then, namely ones tha= t > deal with insmntque and also VOP_LOOKUP changes. To mark XFS MPSAFE, on= e > needs to simply audit the code and make sure it still makse sense for t= oday's > VFS, which is not a huge amount of work. One step further woule be to t= ake > most of the XFS from under the exclusive vnode locking to improve the > performance. >=20 > And there is a third option - just let current XFS port die and start w= ith > newer version. >=20 I like option 3. The current code is way way out of date and doesn't even reflect the last round of sync up I did. Unless somebody says "hey I'm using XFS" we could just let the current code go and reintroduce a current port if it ever receives the needed attention. Unfortunately I think there were would have be be some sponsorship of the effort since getting xfs fully supported would require some significant developer hours. If anybody is interesting in the current state of things here is my git tree that I cleaned up and put online during BSDCan. http://git.digitalelves.com/?p=3DFreeBSD_xfs.git;a=3Dsummary xfs-FreeBSD_7 has the last somewhat functional code. This was based on a code drop from linux XFS at the time. Log recovery was just starting to come to life (very simple recovery would work) Write was working to the point there you could do a single thread to the fs and have the data cmp back with the original file. Read also still works but is unstable. xfs-FreeBSD_9 is quick code drop I did a month or so ago. I does not compile as many of the linux files moved around so not all the compat stuff lines up and some new compat code needs to written. -Russell --------------030107020906030202000708-- --------------enig63054884B80B5C3C56A7DE05 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk/yGYAACgkQNRmM+OaGhBhcYQCfYSzCjbJVmv+IMrdwoxARmHbs aVkAnRbx2YzxZrT6AY76vuB53sCTUZp9 =Ajfl -----END PGP SIGNATURE----- --------------enig63054884B80B5C3C56A7DE05-- From owner-freebsd-fs@FreeBSD.ORG Mon Jul 2 22:18:02 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 26449106564A; Mon, 2 Jul 2012 22:18:02 +0000 (UTC) (envelope-from kabaev@gmail.com) Received: from mail-qc0-f182.google.com (mail-qc0-f182.google.com [209.85.216.182]) by mx1.freebsd.org (Postfix) with ESMTP id 961348FC12; Mon, 2 Jul 2012 22:18:01 +0000 (UTC) Received: by qcsg15 with SMTP id g15so3709155qcs.13 for ; Mon, 02 Jul 2012 15:18:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:in-reply-to:references:x-mailer :mime-version:content-type; bh=YA229ZxuTLp7wYV3QgrlsfH1OApy1puoZbXNkWpw698=; b=AwjMYg/h0PYOhGM2qZGbOfy2ODnu/4UQt4uD8YWgcpy2mh78yQ4Gl+z174ZHZeajUM dkWx1y/YVyd+qtn5/aOhcgDohf+AXpGaTRK0x12dDNU9boqKmrxqRBJpyuOAqK5XAiGA Pn3F6p2LlQaLkDJPeKoK+J+YUg1pjgY+M0wKCkkSsUeXnsZTru11X0dtDLJSBHiD9paG so2i30F+S5x+bUnuEl050dQkFqJUAmqr3rYUqNT7YT4TOagp4X4WGom3bzRS3uzeMJ2h Uo+PANayujtO3NlFJwvVstNm3XUvzUgc8Vo8BuMe1henDuqBVyM/UQus4OgZh7WPs1WF mkmw== Received: by 10.224.181.6 with SMTP id bw6mr25970184qab.74.1341267480796; Mon, 02 Jul 2012 15:18:00 -0700 (PDT) Received: from kan.dyndns.org (c-24-63-226-98.hsd1.ma.comcast.net. [24.63.226.98]) by mx.google.com with ESMTPS id cz12sm33957314qab.5.2012.07.02.15.17.59 (version=SSLv3 cipher=OTHER); Mon, 02 Jul 2012 15:17:59 -0700 (PDT) Date: Mon, 2 Jul 2012 18:17:48 -0400 From: Alexander Kabaev To: Konstantin Belousov Message-ID: <20120702181748.6bb126f6@kan.dyndns.org> In-Reply-To: <20120702201206.GV2337@deviant.kiev.zoral.com.ua> References: <20120702061219.GA16671@infradead.org> <20120702150339.GA7226@kan.dyndns.org> <20120702201206.GV2337@deviant.kiev.zoral.com.ua> X-Mailer: Claws Mail 3.8.0 (GTK+ 2.24.6; amd64-portbld-freebsd10.0) Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/tC9gBXvdben2oNR.d08Mkfp"; protocol="application/pgp-signature" Cc: Attilio Rao , "C. P. Ghost" , Russell, Cattelan , freebsd-current@freebsd.org, FreeBSD FS Subject: Re: MPSAFE VFS -- List of upcoming actions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2012 22:18:02 -0000 --Sig_/tC9gBXvdben2oNR.d08Mkfp Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Mon, 2 Jul 2012 23:12:06 +0300 Konstantin Belousov wrote: > On Mon, Jul 02, 2012 at 11:03:40AM -0400, Alexander Kabaev wrote: > > On Mon, Jul 02, 2012 at 02:12:20AM -0400, Christoph Hellwig wrote: > > > On Sun, Jul 01, 2012 at 03:52:05PM +0200, Attilio Rao wrote: > > > > anything by SoC involved people about NTFS and certainly I > > > > don't see a plan to get XFS locked. > > >=20 > > > Stupid question, but what amount of locking does XFS in FreeBSD > > > still need? I'm one of the maintainer of XFS on Linux, and while > > > I know FreeBSD imported a really old version compared to the > > > current one the codebases on IRIX and later Linux never relied on > > > any global Giant-style locking. So if there is anything to fix > > > it would be the in the small bits of FreeBSD-specific code. > > >=20 > >=20 > > When I stopped being interested in XFS, I left is marked as > > non-MPSAFE entirely because of the lack of proper testing and > > because VFS locking was still evolving, there was no officially > > proper way of locking the FS and no other FS in the tree was > > MPSAFE. At that time the only problematic area was around inode > > instantiation, but sereval other lockingi changes have made it into > > the tree since then, namely ones that deal with insmntque and also > > VOP_LOOKUP changes. To mark XFS MPSAFE, one needs to simply audit > > the code and make sure it still makse sense for today's VFS, which > > is not a huge amount of work. One step further woule be to take > > most of the XFS from under the exclusive vnode locking to improve > > the performance. >=20 > If filesystem uses some global internal locks, that locks usually are > placed after the vnode locks in global lock order, because VOP methods > call into fs with vnode locked. Then, VOP_LOOKUP() usual sequence of > events, when method is called with lookup directory locked, causes > LOR. It appears because you lock global lock upon entry into > VOP_LOOKUP(), and then need to lock the returned vnode. > Dropping global lock inside VOP_LOOKUP() usually exposes races which > were the reason to introduce the global lock. Having filesystem > non-MPSAFE makes the LOR go away without the need to drop global lock. >=20 > Example of FreeBSD native filesystem which suffered from this issue > and required quite non-trivial handling is devfs. Devfs uses per-mount > global lock. See devfs_allocv() and devfs_allocv_drop_refs() for > the gory details. All very informative, though misplaced in case of XFS. XFS does not use global lock internally, it is quite well locked inside and exclusive _VNODE_ lock we take by default in all methods is usually unnecessary as all it does it prevents other locks in XFS proper from having any useful function. --=20 Alexander Kabaev --Sig_/tC9gBXvdben2oNR.d08Mkfp Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iD8DBQFP8h4VQ6z1jMm+XZYRAhbnAJwJ9FKxAHEhZQS3re4WJZZA5Jr4+wCcCXv8 AA5y7bwN5mHu/RbP0pxXh1M= =u2RK -----END PGP SIGNATURE----- --Sig_/tC9gBXvdben2oNR.d08Mkfp-- From owner-freebsd-fs@FreeBSD.ORG Tue Jul 3 00:10:55 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 75098106564A for ; Tue, 3 Jul 2012 00:10:55 +0000 (UTC) (envelope-from alex@trull.org) Received: from mail.internationalconspiracy.org (unknown [IPv6:2001:470:28:2ac::62]) by mx1.freebsd.org (Postfix) with ESMTP id 2D5108FC19 for ; Tue, 3 Jul 2012 00:10:55 +0000 (UTC) Received: from [192.168.124.68] (host109-153-121-178.range109-153.btcentralplus.com [109.153.121.178]) by mail.internationalconspiracy.org (Postfix) with ESMTPSA id C444C1A8C800 for ; Tue, 3 Jul 2012 00:10:53 +0000 (UTC) Message-ID: <1341274252.7031.6.camel@linprecis.turandot.home> From: Alex Trull To: freebsd-fs@freebsd.org Date: Tue, 03 Jul 2012 01:10:52 +0100 Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.2.3-0ubuntu6 Content-Transfer-Encoding: 7bit Mime-Version: 1.0 Subject: 8.3 + ZFS - steady leak in kmem, unrelated to arc size. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Jul 2012 00:10:55 -0000 Hi All, I'm seeing the following leak in 8.2/8.3 since I re-enabled zfs on my system. compare the arc graph http://web.internationalconspiracy.org/munin/internationalconspiracy.org/potjie.internationalconspiracy.org/zfsarc_l1.html with the kmem graph http://web.internationalconspiracy.org/munin/internationalconspiracy.org/potjie.internationalconspiracy.org/kmem.html a straight trajectory of base kmem growth. non-kmem vmem usage is getting squeezed into swap: CPU: % user, % nice, % system, % interrupt, % idle Mem: 976M Active, 543M Inact, 5963M Wired, 414M Cache, 778M Buf, 1372K Free Swap: 16G Total, 2083M Used, 14G Free, 12% Inuse I've dumped everything I can think of here if anyone wants to take a look: http://trull.org/~alex/src/Debuggery/20120703-leakykmem/ Regards, Alex From owner-freebsd-fs@FreeBSD.ORG Tue Jul 3 06:37:15 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2CA5B106566C for ; Tue, 3 Jul 2012 06:37:15 +0000 (UTC) (envelope-from peter.maloney@brockmann-consult.de) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.171]) by mx1.freebsd.org (Postfix) with ESMTP id B265D8FC17 for ; Tue, 3 Jul 2012 06:37:14 +0000 (UTC) Received: from [10.3.0.26] ([141.4.215.32]) by mrelayeu.kundenserver.de (node=mrbap0) with ESMTP (Nemesis) id 0MT5uQ-1SMCp942Qo-00S9Po; Tue, 03 Jul 2012 08:32:06 +0200 Message-ID: <4FF291E5.6060509@brockmann-consult.de> Date: Tue, 03 Jul 2012 08:32:05 +0200 From: Peter Maloney User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:12.0) Gecko/20120421 Thunderbird/12.0 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <1341274252.7031.6.camel@linprecis.turandot.home> In-Reply-To: <1341274252.7031.6.camel@linprecis.turandot.home> X-Enigmail-Version: 1.4.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Provags-ID: V02:K0:Cez7n8qKEWG928lnTfk88nHtN/+/iCXMVqM9dtQBS5R 0MNSn+9M8LId6jRP+58+aMKZ5doZ4BPYzbHM6zxMbpYqseqBuZ Sy/POMj0alSXIDYsEfYyptiteEKYqV4iU6m0Ge3e0NX++uVxoh YakkKHER+5vZMNLmNfqWlkKjeNTNjIHaSHZpDIChM2kLQ6VCgQ /VkrCgVmoH4cRZyUDmgKKIQdwuyjDpkvZjtu+LY9YwkgzTFHCK RsH9cNLd5v0L62FNypeX3jXV61Ke2zAGMvhINRfoae057f+peH lza2IBYrlFEnsQUb2d+liKoQqHBZ/CT9yj9jcg/WfpQcR9JGLP PLclHGuvQgwdnZc5hRJY2wE6v6yhfViPHDWOmZt6i Subject: Re: 8.3 + ZFS - steady leak in kmem, unrelated to arc size. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Jul 2012 06:37:15 -0000 Could it be this bug in the nfs v4 server? http://www.freebsd.org/cgi/query-pr.cgi?pr=167266 Check the output from this command (USED column I think): vmstat -z | egrep -i "NAMEI|ITEM" On 07/03/2012 02:10 AM, Alex Trull wrote: > Hi All, > > I'm seeing the following leak in 8.2/8.3 since I re-enabled zfs on my > system. > > compare the arc graph > http://web.internationalconspiracy.org/munin/internationalconspiracy.org/potjie.internationalconspiracy.org/zfsarc_l1.html > with the kmem graph > http://web.internationalconspiracy.org/munin/internationalconspiracy.org/potjie.internationalconspiracy.org/kmem.html > a straight trajectory of base kmem growth. > > non-kmem vmem usage is getting squeezed into swap: > > CPU: % user, % nice, % system, % interrupt, % idle > Mem: 976M Active, 543M Inact, 5963M Wired, 414M Cache, 778M Buf, 1372K Free > Swap: 16G Total, 2083M Used, 14G Free, 12% Inuse > > I've dumped everything I can think of here if anyone wants to take a > look: http://trull.org/~alex/src/Debuggery/20120703-leakykmem/ > > Regards, > > Alex > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" -- -------------------------------------------- Peter Maloney Brockmann Consult Max-Planck-Str. 2 21502 Geesthacht Germany Tel: +49 4152 889 300 Fax: +49 4152 889 333 E-mail: peter.maloney@brockmann-consult.de Internet: http://www.brockmann-consult.de -------------------------------------------- From owner-freebsd-fs@FreeBSD.ORG Tue Jul 3 11:37:05 2012 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 016DC106564A; Tue, 3 Jul 2012 11:37:05 +0000 (UTC) (envelope-from markus.gebert@hostpoint.ch) Received: from mail.adm.hostpoint.ch (mail.adm.hostpoint.ch [IPv6:2a00:d70:0:a::e0]) by mx1.freebsd.org (Postfix) with ESMTP id 758188FC0C; Tue, 3 Jul 2012 11:37:04 +0000 (UTC) Received: from [77.109.131.203] (port=62592 helo=ch4buk-en0.office.hostpoint.internal) by mail.adm.hostpoint.ch with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.69 (FreeBSD)) (envelope-from ) id 1Sm1Pj-000H8j-DX; Tue, 03 Jul 2012 13:37:03 +0200 Mime-Version: 1.0 (Apple Message framework v1278) From: Markus Gebert In-Reply-To: <201207021050.13974.jhb@freebsd.org> Date: Tue, 3 Jul 2012 13:37:02 +0200 Message-Id: References: <201206251443.41768.jhb@freebsd.org> <0EF22D00-1CDB-44DA-BF3C-3AC672072C05@hostpoint.ch> <201207021050.13974.jhb@freebsd.org> To: John Baldwin X-Mailer: Apple Mail (2.1278) Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: fs@freebsd.org Subject: Re: [PATCH] Simple ARC stats in top X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Jul 2012 11:37:05 -0000 On 02.07.2012, at 16:50, John Baldwin wrote: > On Monday, July 02, 2012 8:12:26 am Markus Gebert wrote: >> Hi John >>=20 >> On 27.06.2012, at 17:49, Markus Gebert wrote: >>=20 >>> On 27.06.2012, at 14:10, John Baldwin wrote: >>>=20 >>>> Hmmm, my patch is relative to a top that has -P backported and I = can't=20 >>>> reproduce this, the ARC lines moves around properly if 'P' is = toggled >>>> at runtime as well for me. Did you have any conflicts in your = patch >>>> that you had to resolve by hand? >>>=20 >>> No, I did not merge anything by hand. But what I did, and that = probably=20 > was the cause, is only buildworld with NO_CLEAN=3D1 to save time and = just=20 > rebuild top. Now that I've done a full buildworld, the probem seems to = be=20 > gone. I'll test more and step up, if it happens again, but consider = this=20 > solved for now. Sorry for the noise. >>=20 >> The problem is back. I don't know why I thought it was gone last = week, maybe=20 > I looked at the wrong terminal or something. >>=20 >> Anyway, I tried again today, and I can reproduce this not only on 8.3 = but=20 > also on 9.0. To rule out a problem with iTerm I also tried OS X = Terminal and=20 > Gnome Terminal on Ubuntu. The terminal does not seem to matter at all. >>=20 >> This sample output is from a patched top (run with -S -P) on a 9.0 = server=20 > connected to via ssh: >=20 > I found it. I wasn't updating y_arc for PCPU stats. Try this: Now it works perfectly. Thanks! Markus From owner-freebsd-fs@FreeBSD.ORG Wed Jul 4 08:44:13 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3DAFD1065670 for ; Wed, 4 Jul 2012 08:44:13 +0000 (UTC) (envelope-from devgs@ukr.net) Received: from ffe17.ukr.net (ffe17.ukr.net [195.214.192.83]) by mx1.freebsd.org (Postfix) with ESMTP id CA2A58FC15 for ; Wed, 4 Jul 2012 08:44:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=ffe; h=Date:Message-Id:From:To:References:In-Reply-To:Subject:Cc:Content-Type:Content-Transfer-Encoding:MIME-Version; bh=FX3xa5vxgRONV/Zle7USSmCoOFICE5fwkzmIcrOPqZY=; b=Vcyc2tzWmDElmHsZgOGmHO/0pI4nkq98rMOlM8G1iseTl2hrvZLwiXgS23jx3pQDXRAcmIj5khEi83VbF5rPMNzePUl4W1WKXMH15o6lieO7akl14GD9cWOwsWOoDjyGQrPW266Srf6U08PMcimIyWiYElFihJgU1iK3qkHfkug=; Received: from mail by ffe17.ukr.net with local ID 1SmKca-0006Tx-Pg ; Wed, 04 Jul 2012 11:07:36 +0300 MIME-Version: 1.0 In-Reply-To: <91943.1339669820.1305529125424791552@ffe15.ukr.net> X-New-Users: 0 References: <91943.1339669820.1305529125424791552@ffe15.ukr.net> To: "Pavlo" From: "Pavlo" X-Mailer: freemail.ukr.net 4.0 X-Originating-Ip: [212.42.94.154] Message-Id: <23856.1341389256.6316487571580649472@ffe17.ukr.net> X-Browser: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:13.0) Gecko/20100101 Firefox/13.0.1 Date: Wed, 04 Jul 2012 11:07:36 +0300 Content-Type: text/plain; charset="windows-1251" Content-Transfer-Encoding: binary Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org Subject: Re: mmap() incoherency on hi I/O load (FS is zfs) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jul 2012 08:44:13 -0000 --- Original message --- From: "Pavlo" To: freebsd-fs@freebsd.org Date: 14 June 2012, 13:30:20 Subject: mmap() incoherency on hi I/O load (FS is zfs) > There's a case when some parts of files that are mapped and then modified getting corrupted. By corrupted I mean some data is ok (one that was written using write()/pwrite()) but some looks like it never existed. Like it was some time in buffers, when several processes simultaneously (of course access was synchronised) used shared pages and reported it's existence. But after time pass they (processes) screamed that it is now lost. Only part of data written with pwrite() was there. Everything that was written via mmap() is zero. > > So as I said it occurs on hi I/O busyness. When in background 4+ processes do indexing of huge ammount of data. Also I want to note, it never occurred in the life of our project while we used mmap() under same I/O stress conditions when mapping was done for a whole file of just a part(header) starting from a beginning of a file. First time we used mapping of individual pages, just to save RAM, and this popped up. > > Solution for this problem is msync() before any munmap(). But man says: > > The msync() system call is usually not needed since BSD implements a coherent file system buffer cache. However, it may be used to associate dirty VM pages with file system buffers and thus cause them to be flushed to physical media sooner rather than later. > > Any thoughts? Thanks. > > So I tracked issue to the place where it occurs. When I commit data to file using mmap() and pwrite() side by side, sometimes 'newest data' is being overwritten by 'elder data'. From time to time 'elder data' can be something written with mmap() either with pwrite(). It never happens when I use exclusively mmap() either pwrite(). Also this issue reproduces on UFS as well. I think there is a problem keeping mmapep pages and FS cache synced. I will try to make test to reliably reproduces issue. From owner-freebsd-fs@FreeBSD.ORG Wed Jul 4 09:06:42 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E165E106566B for ; Wed, 4 Jul 2012 09:06:42 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 5FBF98FC0C for ; Wed, 4 Jul 2012 09:06:41 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q6496kYm086363; Wed, 4 Jul 2012 12:06:46 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q6496Xaf012855; Wed, 4 Jul 2012 12:06:33 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q6496XTu012854; Wed, 4 Jul 2012 12:06:33 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 4 Jul 2012 12:06:33 +0300 From: Konstantin Belousov To: Pavlo Message-ID: <20120704090633.GH2337@deviant.kiev.zoral.com.ua> References: <91943.1339669820.1305529125424791552@ffe15.ukr.net> <23856.1341389256.6316487571580649472@ffe17.ukr.net> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="vX2ve0hYIA9A871f" Content-Disposition: inline In-Reply-To: <23856.1341389256.6316487571580649472@ffe17.ukr.net> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-fs@freebsd.org Subject: Re: mmap() incoherency on hi I/O load (FS is zfs) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jul 2012 09:06:43 -0000 --vX2ve0hYIA9A871f Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Jul 04, 2012 at 11:07:36AM +0300, Pavlo wrote: >=20 >=20 >=20 > --- Original message --- > From: "Pavlo" > To: freebsd-fs@freebsd.org > Date: 14 June 2012, 13:30:20 > Subject: mmap() incoherency on hi I/O load (FS is zfs) >=20 >=20 > > There's a case when some parts of files that are mapped and then > modified getting corrupted. By corrupted I mean some data is ok (one that > was written using write()/pwrite()) but some looks like it never existed. > Like it was some time in buffers, when several processes simultaneously > (of course access was synchronised) used shared pages and reported it's > existence. But after time pass they (processes) screamed that it is now > lost. Only part of data written with pwrite() was there. Everything that > was written via mmap() is zero. > > > > So as I said it occurs on hi I/O busyness. When in background 4+ > processes do indexing of huge ammount of data. Also I want to note, it > never occurred in the life of our project while we used mmap() under > same I/O stress conditions when mapping was done for a whole file of just > a part(header) starting from a beginning of a file. First time we used > mapping of individual pages, just to save RAM, and this popped up. > > > > Solution for this problem is msync() before any munmap(). But man says: > > > > >=20 > The msync() system call is usually not needed since BSD implements a > coherent file system buffer cache. However, it may be used to associate > dirty VM pages with file system buffers and thus cause them to be flushed > to physical media sooner rather than later. > >=20 > > Any thoughts? Thanks. > >=20 > >=20 >=20 > So I tracked issue to the place where it occurs. When I commit data to > file using mmap() and pwrite() side by side, sometimes 'newest data' is > being overwritten by 'elder data'. From time to time 'elder data' can be > something written with mmap() either with pwrite(). It never happens when > I use exclusively mmap() either pwrite(). Also this issue reproduces on > UFS as well. I think there is a problem keeping mmapep pages and FS cache > synced. I am curious how do you label data with newer and elder labels. I do admit a possibility of a race in ZFS double-copy implementation of the mmap/cache coherency, but somewhat skeptical about the same possibility for UFS. What you saying might indicate that we loose modified/dirty bits for the page, but that would have much more firework then just eventual race with write. What version of the system ? Does the machine swap ? >=20 > I will try to make test to reliably reproduces issue. Yes, isolated test case is the best route forward. It would either show a bug or demonstrate a misunderstanding on your part. --vX2ve0hYIA9A871f Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAk/0B5kACgkQC3+MBN1Mb4gZ0QCg7SoPwIYcseI/gSsbOOyTboCN oxgAn0HWsYDFOdsxdsedeuEbucXyGPUc =DUC7 -----END PGP SIGNATURE----- --vX2ve0hYIA9A871f-- From owner-freebsd-fs@FreeBSD.ORG Wed Jul 4 09:29:35 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 588F7106566C for ; Wed, 4 Jul 2012 09:29:35 +0000 (UTC) (envelope-from devgs@ukr.net) Received: from ffe2.ukr.net (ffe2.ukr.net [195.214.192.44]) by mx1.freebsd.org (Postfix) with ESMTP id E260B8FC08 for ; Wed, 4 Jul 2012 09:29:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=ffe; h=Date:Message-Id:From:To:References:In-Reply-To:Subject:Cc:Content-Type:Content-Transfer-Encoding:MIME-Version; bh=wpAe0vo1yyYhHF4FfXYDjPzqoajgK+Akku53wctzKjA=; b=XC6DdYj2S+W2dGcMEqcPSkY9HK76W12K+0UmJA8n8L9POi05/W7cia8QA62XPy/v3z95z4nAJ7gHyyAtz4hiVt5Qq7dJ1w6q5eh/6WunDdKwZJ4g61+hAar5tlyCsiD+M5FC0yE/uqS4RNAoCAbAJYUVXfQ1xL11KfIENS8drvY=; Received: from mail by ffe2.ukr.net with local ID 1SmLtn-000DOR-OT ; Wed, 04 Jul 2012 12:29:27 +0300 MIME-Version: 1.0 Content-Disposition: inline Content-Transfer-Encoding: binary Content-Type: text/plain; charset="windows-1251" In-Reply-To: <1480.1341393955.14971952305407262720@ffe6.ukr.net> X-New-Users: 0 References: <20120704090633.GH2337@deviant.kiev.zoral.com.ua> <1480.1341393955.14971952305407262720@ffe6.ukr.net> <91943.1339669820.1305529125424791552@ffe15.ukr.net> <23856.1341389256.6316487571580649472@ffe17.ukr.net> To: "Pavlo" From: "Pavlo" X-Mailer: freemail.ukr.net 4.0 X-Originating-Ip: [212.42.94.154] Message-Id: <47555.1341394167.3760243713073545216@ffe2.ukr.net> X-Browser: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:13.0) Gecko/20100101 Firefox/13.0.1 Date: Wed, 04 Jul 2012 12:29:27 +0300 Cc: freebsd-fs@freebsd.org Subject: Re: mmap() incoherency on hi I/O load (FS is zfs) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jul 2012 09:29:35 -0000 --- Original message --- From: "Pavlo" To: "Konstantin Belousov" Date: 4 July 2012, 12:25:55 Subject: Re: mmap() incoherency on hi I/O load (FS is zfs) > --- Original message --- > From: "Konstantin Belousov" > To: "Pavlo" > Date: 4 July 2012, 12:06:44 > Subject: Re: mmap() incoherency on hi I/O load (FS is zfs) > > > > > > On Wed, Jul 04, 2012 at 11:07:36AM +0300, Pavlo wrote: > > > > > > > > > > > > --- Original message --- > > > From: "Pavlo" > > > To: freebsd-fs@freebsd.org > > > Date: 14 June 2012, 13:30:20 > > > Subject: mmap() incoherency on hi I/O load (FS is zfs) > > > > > > > > > > There's a case when some parts of files that are mapped and then > > > modified getting corrupted. By corrupted I mean some data is ok (one that > > > was written using write()/pwrite()) but some looks like it never existed. > > > Like it was some time in buffers, when several processes simultaneously > > > (of course access was synchronised) used shared pages and reported it's > > > existence. But after time pass they (processes) screamed that it is now > > > lost. Only part of data written with pwrite() was there. Everything that > > > was written via mmap() is zero. > > > > > > > > So as I said it occurs on hi I/O busyness. When in background 4+ > > > processes do indexing of huge ammount of data. Also I want to note, it > > > never occurred in the life of our project while we used mmap() under > > > same I/O stress conditions when mapping was done for a whole file of just > > > a part(header) starting from a beginning of a file. First time we used > > > mapping of individual pages, just to save RAM, and this popped up. > > > > > > > > Solution for this problem is msync() before any munmap(). But man says: > > > > > > > > > > > > > > The msync() system call is usually not needed since BSD implements a > > > coherent file system buffer cache. However, it may be used to associate > > > dirty VM pages with file system buffers and thus cause them to be flushed > > > to physical media sooner rather than later. > > > > > > > > Any thoughts? Thanks. > > > > > > > > > > > > > > So I tracked issue to the place where it occurs. When I commit data to > > > file using mmap() and pwrite() side by side, sometimes 'newest data' is > > > being overwritten by 'elder data'. From time to time 'elder data' can be > > > something written with mmap() either with pwrite(). It never happens when > > > I use exclusively mmap() either pwrite(). Also this issue reproduces on > > > UFS as well. I think there is a problem keeping mmapep pages and FS cache > > > synced. > > I am curious how do you label data with newer and elder labels. > > I have list header like: > > struct XXX > { > uint32_t alloc_size; > uint32_t list_size; > node_t node[1]; > } > > First I init it with pwrite() setting for example alloc_size to 10 and everything else to 0; > > Then add elements with mmap(); > > 1. Workers log elements existence... > 2. Workers log elements existence... > ... same thing for a few seconds. > X. One of the workers cry that list is empty. > > Then I inspect core file and see that list looks like if it was just initialised with pwrite() ie alloc_size equals 10, everything else is 0. > Hard to reproduce because it happen only on really high IO loads. And from tens of thousands of such files only a couple getting corrupted. > > > > > I do admit a possibility of a race in ZFS double-copy implementation of > > the mmap/cache coherency, but somewhat skeptical about the same possibility > > for UFS. What you saying might indicate that we loose modified/dirty bits > > for the page, but that would have much more firework then just eventual > > race with write. > > > > What version of the system ? Does the machine swap ? Forgot to tell system stat: uname -a FreeBSD zfs1.dev.ukr.net 8.2-STABLE FreeBSD 8.2-STABLE #7: Wed Aug 3 11:41:58 EEST 2011 root@dev.ukr.net:/usr/obj/usr/src/sys/DEV i386 Swap is turned off. For known reasons. > > Okay, after msync() helped but didn't fixed issue (just reduced occurrence) I did next thing: > tracked modification of mmaped pages using mprotect(). At the end of session before munpap() saved modified pages, then munmap() then I wrote those pages back to disk. > > Later worker accessed those pages again with mmap(), modified them and for some parts of those pages did read() instead of accessing via mmap(). What read() returned was data committed in previous session with write() but not the data, that was just modified by same process via mmap(). We reproduces this again and again on UFS on FreeBSD and only on high IO load. Though we could never reproduce this on Linux (ext4). > > > > > > > > > I will try to make test to reliably reproduces issue. > > Yes, isolated test case is the best route forward. It would either show > > a bug or demonstrate a misunderstanding on your part. > > I am trying, but it's really hard to make example to reproduce this issue. > > Thanks for reply. From owner-freebsd-fs@FreeBSD.ORG Wed Jul 4 09:45:18 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 731CC106566B for ; Wed, 4 Jul 2012 09:45:18 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id A85618FC08 for ; Wed, 4 Jul 2012 09:45:17 +0000 (UTC) Received: from skuns.kiev.zoral.com.ua (localhost [127.0.0.1]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id q649jEhI089800; Wed, 4 Jul 2012 12:45:14 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5) with ESMTP id q649j1CV013082; Wed, 4 Jul 2012 12:45:01 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.5/8.14.5/Submit) id q649j19i013081; Wed, 4 Jul 2012 12:45:01 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 4 Jul 2012 12:45:01 +0300 From: Konstantin Belousov To: Pavlo Message-ID: <20120704094501.GJ2337@deviant.kiev.zoral.com.ua> References: <20120704090633.GH2337@deviant.kiev.zoral.com.ua> <91943.1339669820.1305529125424791552@ffe15.ukr.net> <23856.1341389256.6316487571580649472@ffe17.ukr.net> <1480.1341393955.14971952305407262720@ffe6.ukr.net> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="NAIEQUuOioGa/ivv" Content-Disposition: inline In-Reply-To: <1480.1341393955.14971952305407262720@ffe6.ukr.net> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.0 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-fs@freebsd.org Subject: Re: mmap() incoherency on hi I/O load (FS is zfs) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jul 2012 09:45:18 -0000 --NAIEQUuOioGa/ivv Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Jul 04, 2012 at 12:25:55PM +0300, Pavlo wrote: >=20 > =9A=20 >=20 > --- Original message --- > From: "Konstantin Belousov" > To: "Pavlo" > Date: 4 July 2012, 12:06:44 > Subject: Re: mmap() incoherency on hi I/O load (FS is zfs) > =20 > =20 >=20 >=20 > > On Wed, Jul 04, 2012 at 11:07:36AM +0300, Pavlo wrote: > > >=20 > > >=20 > > >=20 > > > --- Original message --- > > > From: "Pavlo" > > > To: freebsd-fs@freebsd.org > > > Date: 14 June 2012, 13:30:20 > > > Subject: mmap() incoherency on hi I/O load (FS is zfs) > > >=20 > > >=20 > > > > There's a case when some parts of files that are mapped and then > > > modified getting corrupted. By corrupted I mean some data is ok (one = that > > > was written using write()/pwrite()) but some looks like it never exis= ted. > > > Like it was some time in buffers, when several processes simultaneous= ly > > > (of course access was synchronised) used shared pages and reported it= 's > > > existence. But after time pass they (processes) screamed that it is n= ow > > > lost. Only part of data written with pwrite() was there. Everything t= hat > > > was written via mmap() is zero. > > > > > > > > So as I said it occurs on hi I/O busyness. When in background 4+ > > > processes do indexing of huge ammount of data. Also I want to note, it > > > never occurred in the life of our project while we used mmap() under > > > same I/O stress conditions when mapping was done for a whole file of = just > > > a part(header) starting from a beginning of a file. First time we used > > > mapping of individual pages, just to save RAM, and this popped up. > > > > > > > > Solution for this problem is msync() before any munmap(). But man s= ays: > > > > > > > > > > >=20 > > > The msync() system call is usually not needed since BSD implements a > > > coherent file system buffer cache. However, it may be used to associ= ate > > > dirty VM pages with file system buffers and thus cause them to be flu= shed > > > to physical media sooner rather than later. > > > >=20 > > > > Any thoughts? Thanks. > > > >=20 > > > >=20 > > >=20 > > > So I tracked issue to the place where it occurs. When I commit data to > > > file using mmap() and pwrite() side by side, sometimes 'newest data' = is > > > being overwritten by 'elder data'. From time to time 'elder data' can= be > > > something written with mmap() either with pwrite(). It never happens = when > > > I use exclusively mmap() either pwrite(). Also this issue reproduces = on > > > UFS as well. I think there is a problem keeping mmapep pages and FS c= ache > > > synced. > > I am curious how do you label data with newer and elder labels. >=20 > I have list header like: >=20 > struct XXX > { > uint32_t alloc_size; > uint32_t list_size; > node_t node[1]; > } >=20 > First I init it with pwrite() setting for example alloc_size to 10 and ev= erything else to 0; >=20 > Then add elements with mmap(); >=20 > 1. Workers log elements existence... > 2. Workers log elements existence... > ... same thing for a few seconds. > X. One of the workers cry that list is empty. >=20 > Then I inspect core file and see that list looks like if it was just init= ialised with pwrite() ie alloc_size equals 10, everything else is 0. > Hard to reproduce because it happen only on really high IO loads. And fro= m tens of thousands of such files only a couple getting corrupted. >=20 > >=20 > > I do admit a possibility of a race in ZFS double-copy implementation of > > the mmap/cache coherency, but somewhat skeptical about the same possibi= lity > > for UFS. What you saying might indicate that we loose modified/dirty bi= ts > > for the page, but that would have much more firework then just eventual > > race with write. > >=20 > > What version of the system ? Does the machine swap ? You just ignored these ^^^^^^^^^^^^ questions. >=20 > Okay, after msync() helped but didn't fixed issue (just reduced occurrenc= e) I did next thing: > tracked modification of mmaped pages using mprotect(). At the end of sess= ion before munpap() saved modified pages, then munmap() then I wrote those = pages back to disk. >=20 > Later worker accessed those pages again with mmap(), modified them and fo= r some parts of those pages did read() instead of accessing via mmap(). Wha= t read() returned was data committed in previous session with write() but n= ot the data, that was just modified by same process via mmap(). We reproduc= es this again and again on UFS on FreeBSD and only on high IO load. Though = we could never reproduce this on Linux (ext4). >=20 So you are saying that the following sequence: 1. write at offset X 2. write into the shared mapping of the same file at offset X 3. read at offset X performed by single thread can return data at the point (1) instead of the data at the point (2) ? Knowing how write is implemented for UFS, I find this quite impossible. If the actions are executed in the different processes/threads, say process 1 executes (1, 2) and process 2 executes (3), or process 1 executes (1), and process 2 executes (2, 3), then my first guess would be a lack of proper synchronization between actions. This would indeed makes possible exactly the outcome I described. > >=20 > > >=20 > > > I will try to make test to reliably reproduces issue. > > Yes, isolated test case is the best route forward. It would either show > > a bug or demonstrate a misunderstanding on your part. >=20 > I am trying, but it's really hard to make example to reproduce this issue. This seems to be the only way forward, at least for you. And do answer about the version/swap question. >=20 > Thanks for reply. --NAIEQUuOioGa/ivv Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAk/0EJ0ACgkQC3+MBN1Mb4gMEgCdEoIKsIYTQo6I9fOmTEERgVHV 2AUAoOyYadvKrm9wKaUNT+H2L7OXPnom =kBhd -----END PGP SIGNATURE----- --NAIEQUuOioGa/ivv-- From owner-freebsd-fs@FreeBSD.ORG Wed Jul 4 10:00:29 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 6B3D11065676 for ; Wed, 4 Jul 2012 10:00:29 +0000 (UTC) (envelope-from devgs@ukr.net) Received: from ffe6.ukr.net (ffe6.ukr.net [195.214.192.56]) by mx1.freebsd.org (Postfix) with ESMTP id F3D938FC25 for ; Wed, 4 Jul 2012 10:00:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=ffe; h=Date:Message-Id:From:To:References:In-Reply-To:Subject:Cc:Content-Type:Content-Transfer-Encoding:MIME-Version; bh=eRevNwM47dnMNwKBoE2H4JRsMzXLLhzfITeRObpX0wg=; b=vO1AGCVF4Da8cKwOeT0WZ8VcuovOH8rBG+ghPT0vXCV0Q/g9K4NTdneP6VsT42tkZ1A2E9ge7qo0Gj1mCxmGk8CJz5b9Ffg2JysUMGFp426Vffe5tumD2/17MH7hO96br4Xg435QeOqaLvHSZwA377CCZjzMLgXXhJlonvyB690=; Received: from mail by ffe6.ukr.net with local ID 1SmMNn-000KTd-N5 ; Wed, 04 Jul 2012 13:00:27 +0300 MIME-Version: 1.0 Content-Disposition: inline Content-Transfer-Encoding: binary Content-Type: text/plain; charset="windows-1251" In-Reply-To: <20120704094501.GJ2337@deviant.kiev.zoral.com.ua> X-New-Users: 0 References: <20120704090633.GH2337@deviant.kiev.zoral.com.ua> <91943.1339669820.1305529125424791552@ffe15.ukr.net> <23856.1341389256.6316487571580649472@ffe17.ukr.net> <1480.1341393955.14971952305407262720@ffe6.ukr.net> <20120704094501.GJ2337@deviant.kiev.zoral.com.ua> To: "Konstantin Belousov" From: "Pavlo" X-Mailer: freemail.ukr.net 4.0 X-Originating-Ip: [212.42.94.154] Message-Id: <76170.1341396027.1389059181905903616@ffe6.ukr.net> X-Browser: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:13.0) Gecko/20100101 Firefox/13.0.1 Date: Wed, 04 Jul 2012 13:00:27 +0300 Cc: freebsd-fs@freebsd.org Subject: Re: mmap() incoherency on hi I/O load (FS is zfs) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jul 2012 10:00:29 -0000 > On Wed, Jul 04, 2012 at 12:25:55PM +0300, Pavlo wrote: > > > >   > > > On Wed, Jul 04, 2012 at 11:07:36AM +0300, Pavlo wrote: > > > > > > > > > > > > > There's a case when some parts of files that are mapped and then > > > > modified getting corrupted. By corrupted I mean some data is ok (one that > > > > was written using write()/pwrite()) but some looks like it never existed. > > > > Like it was some time in buffers, when several processes simultaneously > > > > (of course access was synchronised) used shared pages and reported it's > > > > existence. But after time pass they (processes) screamed that it is now > > > > lost. Only part of data written with pwrite() was there. Everything that > > > > was written via mmap() is zero. > > > > > > > > > > So as I said it occurs on hi I/O busyness. When in background 4+ > > > > processes do indexing of huge ammount of data. Also I want to note, it > > > > never occurred in the life of our project while we used mmap() under > > > > same I/O stress conditions when mapping was done for a whole file of just > > > > a part(header) starting from a beginning of a file. First time we used > > > > mapping of individual pages, just to save RAM, and this popped up. > > > > > > > > > > Solution for this problem is msync() before any munmap(). But man says: > > > > > > > > > > > > > > > > > > The msync() system call is usually not needed since BSD implements a > > > > coherent file system buffer cache. However, it may be used to associate > > > > dirty VM pages with file system buffers and thus cause them to be flushed > > > > to physical media sooner rather than later. > > > > > > > > > > Any thoughts? Thanks. > > > > > > > > > > > > > > > > > > So I tracked issue to the place where it occurs. When I commit data to > > > > file using mmap() and pwrite() side by side, sometimes 'newest data' is > > > > being overwritten by 'elder data'. From time to time 'elder data' can be > > > > something written with mmap() either with pwrite(). It never happens when > > > > I use exclusively mmap() either pwrite(). Also this issue reproduces on > > > > UFS as well. I think there is a problem keeping mmapep pages and FS cache > > > > synced. > > > I am curious how do you label data with newer and elder labels. > > > > I have list header like: > > > > struct XXX > > { > > uint32_t alloc_size; > > uint32_t list_size; > > node_t node[1]; > > } > > > > First I init it with pwrite() setting for example alloc_size to 10 and everything else to 0; > > > > Then add elements with mmap(); > > > > 1. Workers log elements existence... > > 2. Workers log elements existence... > > ... same thing for a few seconds. > > X. One of the workers cry that list is empty. > > > > Then I inspect core file and see that list looks like if it was just initialised with pwrite() ie alloc_size equals 10, everything else is 0. > > Hard to reproduce because it happen only on really high IO loads. And from tens of thousands of such files only a couple getting corrupted. > > > > > > > > I do admit a possibility of a race in ZFS double-copy implementation of > > > the mmap/cache coherency, but somewhat skeptical about the same possibility > > > for UFS. What you saying might indicate that we loose modified/dirty bits > > > for the page, but that would have much more firework then just eventual > > > race with write. > > > > > > What version of the system ? Does the machine swap ? > You just ignored these ^^^^^^^^^^^^ questions. Sorry, forgot to answer. Did in next reply but anyways I'll repeat: uname -a FreeBSD zfs1.dev.ukr.net 8.2-STABLE FreeBSD 8.2-STABLE #7: Wed Aug 3 11:41:58 EEST 2011 root@dev.ukr.net:/usr/obj/usr/src/sys/DEV i386 Swap is turned off. For known reasons. Also maybe I confused you with different cases. Thing about list header _does_not_reproduces_on_UFS_. Only on ZFS. > > > > > Okay, after msync() helped but didn't fixed issue (just reduced occurrence) I did next thing: > > tracked modification of mmaped pages using mprotect(). At the end of session before munpap() saved modified pages, then munmap() then I wrote those pages back to disk. > > > > Later worker accessed those pages again with mmap(), modified them and for some parts of those pages did read() instead of accessing via mmap(). What read() returned was data committed in previous session with write() but not the data, that was just modified by same process via mmap(). We reproduces this again and again on UFS on FreeBSD and only on high IO load. Though we could never reproduce this on Linux (ext4). > > > So you are saying that the following sequence: > 1. write at offset X > 2. write into the shared mapping of the same file at offset X > 3. read at offset X > performed by single thread can return data at the point (1) instead of > the data at the point (2) ? > > Knowing how write is implemented for UFS, I find this quite impossible. > > If the actions are executed in the different processes/threads, say > process 1 executes (1, 2) and process 2 executes (3), or process 1 > executes (1), and process 2 executes (2, 3), then my first guess would > be a lack of proper synchronization between actions. This would indeed > makes possible exactly the outcome I described. This was tested _ONLY_ on UFS. Process 1: 1. Write at offset X with mmap(); 2. Commit that page again after munmap() with write(). Later process 2. 1. Read at offset X with mmap(); 2. Write at offset X with mmap(); 3. Read at offset X with read() and see data written by process 1 in (2). All operations are guarded by lock. Never reproduces on Linux. When I remove step (2) for process 1. Never reproduces on UFS but does on ZFS (as I wrote before). Of course may be my mistakes. But same things done exclusively via mmap() or exclusively via read/write never break file. > > > > > > > > > > > I will try to make test to reliably reproduces issue. > > > Yes, isolated test case is the best route forward. It would either show > > > a bug or demonstrate a misunderstanding on your part. > > > > I am trying, but it's really hard to make example to reproduce this issue. > This seems to be the only way forward, at least for you. > And do answer about the version/swap question. > Roget that. Thanks for reply. From owner-freebsd-fs@FreeBSD.ORG Wed Jul 4 10:00:41 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0781010656D0 for ; Wed, 4 Jul 2012 10:00:41 +0000 (UTC) (envelope-from devgs@ukr.net) Received: from ffe6.ukr.net (ffe6.ukr.net [195.214.192.56]) by mx1.freebsd.org (Postfix) with ESMTP id A41098FC23 for ; Wed, 4 Jul 2012 10:00:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=ffe; h=Date:Message-Id:From:To:References:In-Reply-To:Subject:Cc:Content-Type:Content-Transfer-Encoding:MIME-Version; bh=O7p6H6YATxArEPBP/RQEymJSFoBWxZD+YAONAJaehys=; b=GIhvHGIokYksxBE87ecIhVtVEL1RFJAgMrKzxk32BNP+h0uQI9EGAuvgkg5ODb8zY7RzXdmjdxGRdmePhrL/hrZ/xy8k08qLF5zqU28TPKjNJ92m1fr1el99k2f5TOma9gpwzcmvUIICs3mTXOoUbMW/wykEYRYwq9rIpzh6axs=; Received: from mail by ffe6.ukr.net with local ID 1SmLqN-00024i-3h ; Wed, 04 Jul 2012 12:25:55 +0300 MIME-Version: 1.0 Content-Disposition: inline Content-Transfer-Encoding: binary Content-Type: text/plain; charset="windows-1251" In-Reply-To: <20120704090633.GH2337@deviant.kiev.zoral.com.ua> X-New-Users: 1 References: <20120704090633.GH2337@deviant.kiev.zoral.com.ua> <91943.1339669820.1305529125424791552@ffe15.ukr.net> <23856.1341389256.6316487571580649472@ffe17.ukr.net> To: "Konstantin Belousov" From: "Pavlo" X-Mailer: freemail.ukr.net 4.0 X-Originating-Ip: [212.42.94.154] Message-Id: <1480.1341393955.14971952305407262720@ffe6.ukr.net> X-Browser: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:13.0) Gecko/20100101 Firefox/13.0.1 Date: Wed, 04 Jul 2012 12:25:55 +0300 Cc: freebsd-fs@freebsd.org Subject: Re: mmap() incoherency on hi I/O load (FS is zfs) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jul 2012 10:00:41 -0000   --- Original message --- From: "Konstantin Belousov" To: "Pavlo" Date: 4 July 2012, 12:06:44 Subject: Re: mmap() incoherency on hi I/O load (FS is zfs) > On Wed, Jul 04, 2012 at 11:07:36AM +0300, Pavlo wrote: > > > > > > > > --- Original message --- > > From: "Pavlo" > > To: freebsd-fs@freebsd.org > > Date: 14 June 2012, 13:30:20 > > Subject: mmap() incoherency on hi I/O load (FS is zfs) > > > > > > > There's a case when some parts of files that are mapped and then > > modified getting corrupted. By corrupted I mean some data is ok (one that > > was written using write()/pwrite()) but some looks like it never existed. > > Like it was some time in buffers, when several processes simultaneously > > (of course access was synchronised) used shared pages and reported it's > > existence. But after time pass they (processes) screamed that it is now > > lost. Only part of data written with pwrite() was there. Everything that > > was written via mmap() is zero. > > > > > > So as I said it occurs on hi I/O busyness. When in background 4+ > > processes do indexing of huge ammount of data. Also I want to note, it > > never occurred in the life of our project while we used mmap() under > > same I/O stress conditions when mapping was done for a whole file of just > > a part(header) starting from a beginning of a file. First time we used > > mapping of individual pages, just to save RAM, and this popped up. > > > > > > Solution for this problem is msync() before any munmap(). But man says: > > > > > > > > > > The msync() system call is usually not needed since BSD implements a > > coherent file system buffer cache. However, it may be used to associate > > dirty VM pages with file system buffers and thus cause them to be flushed > > to physical media sooner rather than later. > > > > > > Any thoughts? Thanks. > > > > > > > > > > So I tracked issue to the place where it occurs. When I commit data to > > file using mmap() and pwrite() side by side, sometimes 'newest data' is > > being overwritten by 'elder data'. From time to time 'elder data' can be > > something written with mmap() either with pwrite(). It never happens when > > I use exclusively mmap() either pwrite(). Also this issue reproduces on > > UFS as well. I think there is a problem keeping mmapep pages and FS cache > > synced. > I am curious how do you label data with newer and elder labels. I have list header like: struct XXX { uint32_t alloc_size; uint32_t list_size; node_t node[1]; } First I init it with pwrite() setting for example alloc_size to 10 and everything else to 0; Then add elements with mmap(); 1. Workers log elements existence... 2. Workers log elements existence... ... same thing for a few seconds. X. One of the workers cry that list is empty. Then I inspect core file and see that list looks like if it was just initialised with pwrite() ie alloc_size equals 10, everything else is 0. Hard to reproduce because it happen only on really high IO loads. And from tens of thousands of such files only a couple getting corrupted. > > I do admit a possibility of a race in ZFS double-copy implementation of > the mmap/cache coherency, but somewhat skeptical about the same possibility > for UFS. What you saying might indicate that we loose modified/dirty bits > for the page, but that would have much more firework then just eventual > race with write. > > What version of the system ? Does the machine swap ? Okay, after msync() helped but didn't fixed issue (just reduced occurrence) I did next thing: tracked modification of mmaped pages using mprotect(). At the end of session before munpap() saved modified pages, then munmap() then I wrote those pages back to disk. Later worker accessed those pages again with mmap(), modified them and for some parts of those pages did read() instead of accessing via mmap(). What read() returned was data committed in previous session with write() but not the data, that was just modified by same process via mmap(). We reproduces this again and again on UFS on FreeBSD and only on high IO load. Though we could never reproduce this on Linux (ext4). > > > > > I will try to make test to reliably reproduces issue. > Yes, isolated test case is the best route forward. It would either show > a bug or demonstrate a misunderstanding on your part. I am trying, but it's really hard to make example to reproduce this issue. Thanks for reply. From owner-freebsd-fs@FreeBSD.ORG Wed Jul 4 19:15:37 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E5BF01065672; Wed, 4 Jul 2012 19:15:37 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) by mx1.freebsd.org (Postfix) with ESMTP id 7F0AC8FC14; Wed, 4 Jul 2012 19:15:36 +0000 (UTC) Received: by lbon10 with SMTP id n10so13561785lbo.13 for ; Wed, 04 Jul 2012 12:15:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type; bh=xc3N3/5bfTL3ukLflXlk1zMLE1kiGtwB09SZRagzS6s=; b=ktieQwlBKORf4uBtqaMCyXpLzr0jJFnJJ64Ev+77ztD1MB23JdeRbGgE91dQfNhdDa 7RmMHsFXocPaewrR5D8cbCqTYDb2Fbjau9ig2DtHBmbvoi2o1H/+54KSCogw33AqJVFT x4lMwoMiV4YrYqAwsRZgs8gqooL7wkZfCekSduNqQi2GFYd4M1HubHqqXTE92dW+esm3 A6unC3ktUPTVtcF6GIe490pG1Ea7LCLsH08wVJ9P6HQawhPhtnHvjT+bahumJLP32fM7 DjmAPdlFUQQveSqLHUKnoQQ3JuRNSncCMX3zWH+vidrPbLySEhsKrhpbogM8me1Jas2I ue6Q== MIME-Version: 1.0 Received: by 10.112.11.100 with SMTP id p4mr10649516lbb.35.1341429335301; Wed, 04 Jul 2012 12:15:35 -0700 (PDT) Sender: asmrookie@gmail.com Received: by 10.112.27.65 with HTTP; Wed, 4 Jul 2012 12:15:35 -0700 (PDT) In-Reply-To: References: Date: Wed, 4 Jul 2012 20:15:35 +0100 X-Google-Sender-Auth: 1mjJI24TgxwgjsoHfdJ21le5nI8 Message-ID: From: Attilio Rao To: FreeBSD FS , freebsd-current@freebsd.org, Peter Holm , =?UTF-8?Q?Edward_Tomasz_Napiera=C5=82a?= , Vivien Botton Content-Type: text/plain; charset=UTF-8 Cc: Subject: Re: MPSAFE VFS -- List of upcoming actions X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jul 2012 19:15:38 -0000 2012/6/29 Attilio Rao : > As already published several times, according to the following plan: > http://wiki.freebsd.org/NONMPSAFE_DEORBIT_VFS > I still haven't heard from Vivien or Edward, anyway as NTFS is basically only used RO these days (also the mount_ntfs code just permits RO mounting) I stripped all the uncomplete/bogus write support with the following patch: http://www.freebsd.org/~attilio/ntfs_remove_write.patch This is an attempt to make the code smaller and possibly just focus on the locking that really matter (as read-only filesystem). On some points of the patch I'm a bit less sure as we could easily take into account also write for things like vaccess() arguments, and make easier to re-add correct write support at some point in the future, but still force RO, even if the approach used in the patch is more correct IMHO. As an added bonus this patch cleans some dirty code in the mount operation and fixes a bug as vfs_mountedfrom() is called before real mounting is completed and can still fail. The patch has been kindly tested by pho@. If none has objections I will commit it friday evening. Thanks, Attilio -- Peace can only be achieved by understanding - A. Einstein From owner-freebsd-fs@FreeBSD.ORG Thu Jul 5 05:54:15 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EE7161065670 for ; Thu, 5 Jul 2012 05:54:15 +0000 (UTC) (envelope-from grarpamp@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id BADCA8FC15 for ; Thu, 5 Jul 2012 05:54:15 +0000 (UTC) Received: by obbun3 with SMTP id un3so16162923obb.13 for ; Wed, 04 Jul 2012 22:54:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=TR/gMNdKRiidj13hzRHFKjwu7pXmqdhRks1XXbkJkrw=; b=rhVTPo5crsqkyX8qhwXTztaw+UY/TifT6vFTUJxFWG7+tjbZMmUWHbOj8IwMmFOVRK Zm4PooiLMCUaUbHLEZeXJkqF6uQcMznXWRnNFhRn9DvVB+3c/kzdvkb6H/cXnWoAEcu6 lIp9aA/fKszRe/bwKG8UCmwPiHK5DCl6xNIVCWbIZgCMdqZBL62Rc4iihGf/GPAnY4uh eD55cNKiyTD66UZJ0rYA/gB1FN2FS0QOrqC1C5vvYpOhhXLmbULu97qjmV2ihFP92QRE 5Uxfmlr6+kOj298Eu6BaBrNjtBLB4FBTQQfew/cRUMVdKELUhkTN2Kj4m6/biaTlINGl 8oiQ== MIME-Version: 1.0 Received: by 10.182.2.165 with SMTP id 5mr6143750obv.41.1341467649048; Wed, 04 Jul 2012 22:54:09 -0700 (PDT) Received: by 10.76.81.10 with HTTP; Wed, 4 Jul 2012 22:54:08 -0700 (PDT) Date: Thu, 5 Jul 2012 01:54:08 -0400 Message-ID: From: grarpamp To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 Subject: zfs/speed, wow X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jul 2012 05:54:16 -0000 Somewhere between these two it seems like a notable speedup happened regarding [re]reads of large directories under i386+sha256+geli. Feels like dirhash did on ufs. Maybe it's just the post update newness faking me out, but I bow low anyways. FreeBSD 8.3-STABLE #0: Wed May 23 2012 FreeBSD 8.3-STABLE #0: Wed Jul 4 2012 From owner-freebsd-fs@FreeBSD.ORG Thu Jul 5 16:25:42 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A88B8106564A for ; Thu, 5 Jul 2012 16:25:42 +0000 (UTC) (envelope-from dudinskyj@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 51B0A8FC0A for ; Thu, 5 Jul 2012 16:25:42 +0000 (UTC) Received: by obbun3 with SMTP id un3so17105601obb.13 for ; Thu, 05 Jul 2012 09:25:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; bh=DuO/92nz3EF/I9/T0Fd4o84rnheCn31+ACXwytdMDIw=; b=HspVUr/O7y0DQ0Auf+7HhXamSxiWoCtGR1HK5ZpNERegyTgAN0NHbBGfoItxLWQi/H ZxIiC8ryhsCYZpz7cn3RmZr1RFScPK45yuzE0FLgmoaejRB/4FGk/LyULB+xroyZe9JV pOs4v75A9S2uA5DYzTVGjM4rywDe3JG06snfNLkA3M32KwtThEMWceW0ojlxtAeD7D9B Kh+yL3GQT3O19a2bVLV0IO18penEbJs5V1oLkLvWuDZ219FUcX/y15eqMh0TDvX8ayGh WTu2ur+DYN8Fy5L6TM0r97hu+NuqMWKXDF8SsQ0EPZr5IwN1VlL94wJswiITL7CebIa2 CRWg== MIME-Version: 1.0 Received: by 10.60.19.42 with SMTP id b10mr27286005oee.12.1341505541794; Thu, 05 Jul 2012 09:25:41 -0700 (PDT) Sender: dudinskyj@gmail.com Received: by 10.182.24.168 with HTTP; Thu, 5 Jul 2012 09:25:40 -0700 (PDT) Date: Thu, 5 Jul 2012 19:25:40 +0300 X-Google-Sender-Auth: akmwTsnIJksqRST9kyMukB_RgmI Message-ID: From: Oleksandr Dudinskyi To: freebsd-fs@freebsd.org Content-Type: multipart/mixed; boundary=e89a8fb206e8486bba04c41799b3 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Port NetBSD's UDF implementation X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jul 2012 16:25:42 -0000 --e89a8fb206e8486bba04c41799b3 Content-Type: text/plain; charset=ISO-8859-1 Hi. My name is Oleksandr Dudinskyi and I take part in GSoC this year with FreeBSD organization. My project is "Port NetBSD's UDF implementation" http://wiki.freebsd.org/%5BSummerOfCode2012/UDFImplementation I want to show my work in mid-term period of program. It has read-only support UDF file system. Many thanks to Will DeVries < william.devries@gmail.com> who worked on porting this driver before GSoC start, my work builds upon his work. All current UDF versions up to version 2.60 are supported. Implemented and tested media types are CD/DVD/BD disks. This is pretty crude and unfinished version, but it can already be tested. Feedback and suggestions are welcome. Regards, Oleksandr Dudinskyi. --e89a8fb206e8486bba04c41799b3-- From owner-freebsd-fs@FreeBSD.ORG Thu Jul 5 17:25:58 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0F6B9106566B; Thu, 5 Jul 2012 17:25:58 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: from smtprelay02.ispgateway.de (smtprelay02.ispgateway.de [80.67.18.14]) by mx1.freebsd.org (Postfix) with ESMTP id BD9648FC1F; Thu, 5 Jul 2012 17:25:57 +0000 (UTC) Received: from [84.44.153.111] (helo=fabiankeil.de) by smtprelay02.ispgateway.de with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.68) (envelope-from ) id 1Smpnd-0000Dm-7F; Thu, 05 Jul 2012 19:25:05 +0200 Date: Thu, 5 Jul 2012 19:21:58 +0200 From: Fabian Keil To: Oleksandr Dudinskyi Message-ID: <20120705192158.503fa2e5@fabiankeil.de> In-Reply-To: References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/iQSM7PCxMPGsjWX81MV7563"; protocol="application/pgp-signature" X-Df-Sender: Nzc1MDY3 Cc: freebsd-fs@freebsd.org Subject: Re: Port NetBSD's UDF implementation X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: freebsd-fs@freebsd.org List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jul 2012 17:25:58 -0000 --Sig_/iQSM7PCxMPGsjWX81MV7563 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Oleksandr Dudinskyi wrote: > My name is Oleksandr Dudinskyi and I take part in GSoC this year with > FreeBSD organization. My project is "Port NetBSD's UDF implementation" > http://wiki.freebsd.org/%5BSummerOfCode2012/UDFImplementation > I want to show my work in mid-term period of program. It has read-only > support UDF file system. Many thanks to Will DeVries < > william.devries@gmail.com> who worked on porting this driver before GSoC > start, my work builds upon his work. All current UDF versions up to > version 2.60 are supported. Implemented and tested media types are > CD/DVD/BD disks. Sounds great. > This is pretty crude and unfinished version, but it can already be tested. A link to the code would be helpful. Fabian --Sig_/iQSM7PCxMPGsjWX81MV7563 Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAk/1zTkACgkQBYqIVf93VJ1EMwCgmlbGB3x0P/NoS04VjjXEB63l ldQAnixnCBTzaFtKsOCLt72+Y30uOCP2 =Rk9c -----END PGP SIGNATURE----- --Sig_/iQSM7PCxMPGsjWX81MV7563-- From owner-freebsd-fs@FreeBSD.ORG Thu Jul 5 18:48:42 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6DB08106566B for ; Thu, 5 Jul 2012 18:48:42 +0000 (UTC) (envelope-from dudinskyj@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 319DB8FC08 for ; Thu, 5 Jul 2012 18:48:42 +0000 (UTC) Received: by obbun3 with SMTP id un3so17304674obb.13 for ; Thu, 05 Jul 2012 11:48:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=Dt6LGY4QXRTCPPAFSIhxsx47ozKJfCFU8AD2q3mwtZ8=; b=jsGs7OUfP9O1WyX3OPpq5V8llJr/lP4iWZMFuyOFMUxfE0ke3Xq2eVmL7d7H9TbfQa oh6Zt0YPvo3g9MKWroqCB+pMpfnXw18/AEBlKKQ1x/eg+D+PAM+kWGxqL+se1MbtRT/Q 8WrXM6uu/+Uw9ZQmUvSkFSzrX1sdyy3JAI9Cb0u2MG/YEhms/kjS7WXaaZLRvvoWrUWS 8dEfE3oTuJtBUtz3xA9XY7ghFAmMggcyoU2DvbQVvKWkEP09nQLS3rABIadvCqNLq+Br h44+g3n/hIrNIejDIjR0ZHC4LSW23arO9XMWA4xMWEGu+fNkmwmNoCUu6dYX6SFt9Tan p7oQ== MIME-Version: 1.0 Received: by 10.182.164.8 with SMTP id ym8mr22344516obb.51.1341514121709; Thu, 05 Jul 2012 11:48:41 -0700 (PDT) Received: by 10.182.24.168 with HTTP; Thu, 5 Jul 2012 11:48:41 -0700 (PDT) In-Reply-To: <20120705192158.503fa2e5@fabiankeil.de> References: <20120705192158.503fa2e5@fabiankeil.de> Date: Thu, 5 Jul 2012 21:48:41 +0300 Message-ID: From: Oleksandr Dudinskyi To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: Port NetBSD's UDF implementation X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jul 2012 18:48:42 -0000 I attached archive with the patch in letter. On 5 July 2012 20:21, Fabian Keil wrote: > Oleksandr Dudinskyi wrote: > > > My name is Oleksandr Dudinskyi and I take part in GSoC this year with > > FreeBSD organization. My project is "Port NetBSD's UDF implementation" > > http://wiki.freebsd.org/%5BSummerOfCode2012/UDFImplementation > > I want to show my work in mid-term period of program. It has read-only > > support UDF file system. Many thanks to Will DeVries < > > william.devries@gmail.com> who worked on porting this driver before > GSoC > > start, my work builds upon his work. All current UDF versions up to > > version 2.60 are supported. Implemented and tested media types are > > CD/DVD/BD disks. > > Sounds great. > > > This is pretty crude and unfinished version, but it can already be > tested. > > A link to the code would be helpful. > > Fabian > -- Regards From owner-freebsd-fs@FreeBSD.ORG Thu Jul 5 18:57:07 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EE031106566C for ; Thu, 5 Jul 2012 18:57:06 +0000 (UTC) (envelope-from dudinskyj@gmail.com) Received: from mail-gg0-f182.google.com (mail-gg0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id 779A78FC15 for ; Thu, 5 Jul 2012 18:57:06 +0000 (UTC) Received: by ggnm2 with SMTP id m2so9103744ggn.13 for ; Thu, 05 Jul 2012 11:57:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=aN/A5jWz1kyECkTZpeGJNhHSpw7wweYAva29ZpyrPSA=; b=YMp2MDNKIc3/sqUg4oQ6UYvXh9JSmfWwkwhlGStQjd5eoW77NFKJ3ySBihJL0hPXVV Ha4cFEgzXj0Sky8D1LkKYu4+6bKWa/9b6off6AvFDx5CfwGQgeIpfgMwJbDNTifEBXSO BD87xQ9RCdWxEC7kWT2mpXa8/C8GGQjtn8l29gt25f37ba+B1e2HtiLXslJRGvmQepDf Lsi0Ial46qhzEK3NqsCbv8Sq8BTNm9mKlLVfB5ozK4IvYJY62hJo+5Ozn8AsiG/rr2bH zNJ+ivO7QSVCQoOhxA6A5UrhQRNlWW7TYPx7PdknEnw4YUdYQmojmUVWBmDkLEagI+yl xE4Q== MIME-Version: 1.0 Received: by 10.60.29.72 with SMTP id i8mr27866926oeh.26.1341514625505; Thu, 05 Jul 2012 11:57:05 -0700 (PDT) Received: by 10.182.24.168 with HTTP; Thu, 5 Jul 2012 11:57:05 -0700 (PDT) In-Reply-To: References: <20120705192158.503fa2e5@fabiankeil.de> Date: Thu, 5 Jul 2012 21:57:05 +0300 Message-ID: From: Oleksandr Dudinskyi To: freebsd-fs@freebsd.org Content-Type: multipart/mixed; boundary=e89a8fb1f806b6d8c204c419b65d X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: Port NetBSD's UDF implementation X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jul 2012 18:57:07 -0000 --e89a8fb1f806b6d8c204c419b65d Content-Type: text/plain; charset=ISO-8859-1 Just in case attaches again. On 5 July 2012 21:48, Oleksandr Dudinskyi wrote: > I attached archive with the patch in letter. > > > On 5 July 2012 20:21, Fabian Keil wrote: > >> Oleksandr Dudinskyi wrote: >> >> > My name is Oleksandr Dudinskyi and I take part in GSoC this year with >> > FreeBSD organization. My project is "Port NetBSD's UDF implementation" >> > http://wiki.freebsd.org/%5BSummerOfCode2012/UDFImplementation >> > I want to show my work in mid-term period of program. It has read-only >> > support UDF file system. Many thanks to Will DeVries < >> > william.devries@gmail.com> who worked on porting this driver before >> GSoC >> > start, my work builds upon his work. All current UDF versions up to >> > version 2.60 are supported. Implemented and tested media types are >> > CD/DVD/BD disks. >> >> Sounds great. >> >> > This is pretty crude and unfinished version, but it can already be >> tested. >> >> A link to the code would be helpful. >> >> Fabian >> > > > > -- > Regards > -- Regards, Oleksandr Dudinskyi. --e89a8fb1f806b6d8c204c419b65d-- From owner-freebsd-fs@FreeBSD.ORG Thu Jul 5 19:03:58 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8B641106564A for ; Thu, 5 Jul 2012 19:03:58 +0000 (UTC) (envelope-from gperez@entel.upc.edu) Received: from dash.upc.es (dash.upc.es [147.83.2.50]) by mx1.freebsd.org (Postfix) with ESMTP id E06198FC0C for ; Thu, 5 Jul 2012 19:03:57 +0000 (UTC) Received: from ackerman2.upc.es (ackerman2.upc.es [147.83.2.244]) by dash.upc.es (8.14.1/8.13.1) with ESMTP id q65J3ox7008513 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL); Thu, 5 Jul 2012 21:03:50 +0200 Received: from portgus.lan (12.Red-79-145-236.dynamicIP.rima-tde.net [79.145.236.12]) (authenticated bits=0) by ackerman2.upc.es (8.14.4/8.14.4) with ESMTP id q65J3m9x005607 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Thu, 5 Jul 2012 21:03:49 +0200 Message-ID: <4FF5E516.8050509@entel.upc.edu> Date: Thu, 05 Jul 2012 21:03:50 +0200 From: =?ISO-8859-1?Q?Gustau_P=E9rez_i_Querol?= User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:13.0) Gecko/20120620 Thunderbird/13.0.1 MIME-Version: 1.0 To: Oleksandr Dudinskyi References: <20120705192158.503fa2e5@fabiankeil.de> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.70 on 147.83.2.244 X-Mail-Scanned: Criba 2.0 + Clamd X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (dash.upc.es [147.83.2.50]); Thu, 05 Jul 2012 21:03:50 +0200 (CEST) Cc: freebsd-fs@freebsd.org Subject: Re: Port NetBSD's UDF implementation X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jul 2012 19:03:58 -0000 On 05/07/2012 20:57, Oleksandr Dudinskyi wrote: > Just in case attaches again. > I'd like to test it but it would appear that the mailing list eats the attachment. Can you post it somewhere else and bring us the link? -- --------------------------------------------------------------------------- Prou top-posting : http://ca.wikipedia.org/wiki/Top-posting Stop top-posting : http://en.wikipedia.org/wiki/Posting_style O O O Gustau Pérez i Querol O O O Departament d'Enginyeria Telemàtica O O O Universitat Politècnica de Catalunya Edifici C3 - Despatx S101-B UPC Campus Nord UPC C/ Jordi Girona, 1-3 08034 - Barcelona From owner-freebsd-fs@FreeBSD.ORG Thu Jul 5 20:02:01 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7EA43106564A for ; Thu, 5 Jul 2012 20:02:01 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) by mx1.freebsd.org (Postfix) with ESMTP id F2C9C8FC14 for ; Thu, 5 Jul 2012 20:02:00 +0000 (UTC) Received: by lbon10 with SMTP id n10so15365292lbo.13 for ; Thu, 05 Jul 2012 13:02:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=ZJrFjSt8H6ilKNQv6qwurqvYR72dP+0uxBzzrvnMA9o=; b=qu1vJg5lOT0qIlw6CQ+2O6lQ/GeBncpI7nlTzNNdlu8VgqdR+ULovWwQmdEch0C+nM RnAfL0ot2wwOsbn1T0BhdLPsQ4ylDLVedaxAtMQvaiZXvSYS3u2VO3RWVcBzm9FojSCX 4qhDXgRnKb6sMvkYf3Rl4hjED0h24U1eOL3NGRsrvozDUxUu/dQhR9yThfQ3O2DohHNH 5saURtbBhmxNWysNJ1XAvdsl9qnxj5VIZ8XuOeMLcxGkIfRohiXartCwlUgu4bNzqcyE pIU5T2FJkcOSFXF2NHtmMm1WkjAVIddeKRZKOf0QIcJrA8IKweYzE3dDMjVYF0U93tjf WlyA== MIME-Version: 1.0 Received: by 10.152.109.166 with SMTP id ht6mr27512857lab.46.1341518519970; Thu, 05 Jul 2012 13:01:59 -0700 (PDT) Received: by 10.114.37.74 with HTTP; Thu, 5 Jul 2012 13:01:59 -0700 (PDT) In-Reply-To: References: <20120705192158.503fa2e5@fabiankeil.de> Date: Thu, 5 Jul 2012 13:01:59 -0700 Message-ID: From: Freddie Cash To: FreeBSD Filesystems Content-Type: text/plain; charset=UTF-8 Subject: Fwd: Port NetBSD's UDF implementation X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jul 2012 20:02:01 -0000 ---------- Forwarded message ---------- From: Oleksandr Dudinskyi Date: Thu, Jul 5, 2012 at 12:59 PM Subject: Re: Port NetBSD's UDF implementation To: Freddie Cash Sorry, I did not know this . Project in Google code. http://code.google.com/p/port-netbsd-udf-implementation/source/browse/udf2_patch.diff On 5 July 2012 22:04, Freddie Cash wrote: > > > On Jul 5, 2012 11:57 AM, "Oleksandr Dudinskyi" wrote: > > > > Just in case attaches again. > > Non-text attachments are stripped from mailing list posts. You need to put it on a website somewhere and post a URL. -- Regards -- Freddie Cash fjwcash@gmail.com From owner-freebsd-fs@FreeBSD.ORG Thu Jul 5 20:22:05 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 91364106566B for ; Thu, 5 Jul 2012 20:22:05 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) by mx1.freebsd.org (Postfix) with ESMTP id 0C2168FC15 for ; Thu, 5 Jul 2012 20:22:04 +0000 (UTC) Received: by lbon10 with SMTP id n10so15392685lbo.13 for ; Thu, 05 Jul 2012 13:22:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=0RqcjOfbJ5MAfJddsqe4OVtoCyxSQX0jlRgvk0zwr+A=; b=HJR20/2V/SrIRy1zcQ2ylYU1NFR+W3UiF8UcrI5h015LVYUMbxh+9gOhalO0nhOrns yPk0c8dIxxNokiZUu6nIs+pH1bkOPbz9SjOL8/Gn0OsdZLAyXtymiWhRhMlJoFyEBdnU rWXMUBrMAjX5obvfFo58st6ynxrvZXyLN3zixg8kt8BKLpL/yVpWgrvdJbV2gHDqkHx/ L1bMlxbXkFQW7iISXsgNlnMUN/XCUal+TCnAtbhhbbsDCujq7t6nk4N1TPe6lr7cpPEs kLrGMtd1Pt0hS8bKbwrzFHKPUZvruHHiddWZkJ0ZJ8X1ONCOGhqMcRFBSUX0uROnnuif HmJQ== MIME-Version: 1.0 Received: by 10.152.46.6 with SMTP id r6mr27634695lam.7.1341519723582; Thu, 05 Jul 2012 13:22:03 -0700 (PDT) Received: by 10.114.37.74 with HTTP; Thu, 5 Jul 2012 13:22:03 -0700 (PDT) In-Reply-To: References: <20120705192158.503fa2e5@fabiankeil.de> Date: Thu, 5 Jul 2012 13:22:03 -0700 Message-ID: From: Freddie Cash To: FreeBSD Filesystems Content-Type: text/plain; charset=UTF-8 Subject: Fwd: Port NetBSD's UDF implementation X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jul 2012 20:22:05 -0000 Redirect to list. ---------- Forwarded message ---------- From: Oleksandr Dudinskyi Date: Thu, Jul 5, 2012 at 1:20 PM Subject: Re: Port NetBSD's UDF implementation To: Freddie Cash Little description install driver. Install patch udf2_patch.diff then run "make && make install" in 'udf2', 'udf2_iconv' and then in 'mount_udf2' On 5 July 2012 22:59, Oleksandr Dudinskyi wrote: > > Sorry, I did not know this . Project in Google code. > http://code.google.com/p/port-netbsd-udf-implementation/source/browse/udf2_patch.diff > > > On 5 July 2012 22:04, Freddie Cash wrote: >> >> >> On Jul 5, 2012 11:57 AM, "Oleksandr Dudinskyi" wrote: >> > >> > Just in case attaches again. >> >> Non-text attachments are stripped from mailing list posts. You need to put it on a website somewhere and post a URL. > > > > > -- > Regards -- Regards, Oleksandr Dudinskyi. -- Freddie Cash fjwcash@gmail.com From owner-freebsd-fs@FreeBSD.ORG Fri Jul 6 00:18:49 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AEF40106566B for ; Fri, 6 Jul 2012 00:18:49 +0000 (UTC) (envelope-from tjg@soe.ucsc.edu) Received: from mail-we0-f182.google.com (mail-we0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 3650E8FC17 for ; Fri, 6 Jul 2012 00:18:49 +0000 (UTC) Received: by werp13 with SMTP id p13so5452170wer.13 for ; Thu, 05 Jul 2012 17:18:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ucsc.edu; s=ucsc-google; h=mime-version:date:message-id:subject:from:to:content-type; bh=WUKhyZJgWvzwD/79QPyO/mnHw2ms13AG/PEyGRyMpKQ=; b=Ax0Xs0BactTf2m22fKVpMcZA+BIe0BJRYeYg7W53tAQz6Ob2+ZuL4jnQlxx2wAfxM3 JEn3dXrzWFJLbSmfDs0axl1gu7RomklxukoIkdS5hNguIR07Ym1Jtrwjv12NOc19ezFk hg9qt35mUN6T4dLRoQz8JQ3szft9Yf10lO07w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type :x-gm-message-state; bh=WUKhyZJgWvzwD/79QPyO/mnHw2ms13AG/PEyGRyMpKQ=; b=eBWDg4/gQt8eazoxicic1gld0gVHrXT26BFymOBt5/A+mndfYhGeZRjR36zSYk9eU0 ubmNHJvwxp5y/+pFWQgvEwq4e0dxeuVOqGkPppii4QXZ7NMEtwwR0IB2iYtlNrf3Z6WL OoRwcd4mlRrTpGQUIB2KKMIQoow54KUU3QWyJa+DtkO7jUdXxHpSfuNbw0b2x/PxXkww p878f6uat5ihCNfeJfiZZRX5LUtRA5nXB5MdraucW1r1WEoXU8uxPDg2ULRAVovgxmWD WxxW9pxfdoI+SmkQcs7mYFauBN8Tm9D7BA0CTQi4GbXSBKUN7sk4ICAsOu7xIYtbTGBC rvSw== MIME-Version: 1.0 Received: by 10.216.241.200 with SMTP id g50mr9129293wer.79.1341533928018; Thu, 05 Jul 2012 17:18:48 -0700 (PDT) Received: by 10.223.157.204 with HTTP; Thu, 5 Jul 2012 17:18:47 -0700 (PDT) Date: Thu, 5 Jul 2012 17:18:47 -0700 Message-ID: From: Tim Gustafson To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Gm-Message-State: ALoCoQmKgaSydxoE9PrWLZA3t7YZzWVoqTzow9Nb5G8p+zH3BtDmCrgKM98k1a6dtlzns/6eqguh Subject: FreeBSD 9.0 + ZFS + NFSv4 + Kerberos X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2012 00:18:49 -0000 Hi, I'd like to set up a FreeBSD 9.0 box as a ZFS+NFSv4+Kerberos server. So far, I can mount a file system from a client machine, but whenever I try to do anything on that file system, I get errors that look like this: tjg@junta: cd /mnt nfsv4 err=10016 nfsv4 err=10016 /mnt: Input/output error. I can kinit on both boxes, and have done so on my client box; klist shows a valid ticket on the client box. I can "mount /mnt" on the client without any problems. Here are the relevant configuration files: server:/etc/rc.conf: nfs_server_enable="yes" nfsv4_server_enable="yes" mountd_enable="yes" mountd_flags="-r" rpcbind_enable="yes" rpc_lockd_enable="yes" rpc_statd_enable="yes" gssd_enable="yes" server:/etc/exports: V4: /tank/export -sec=krb5p client:/etc/rc.conf: nfs_client_enable="yes" rpc_lockd_enable="yes" rpc_statd_enable="yes" rpcbind_enable="yes" devfs_enable="yes" gssd_enable="yes" client:/etc/fstab: server:/ /mnt nfs rw,noauto,nfsv4,sec=krb5p 0 0 -- Tim Gustafson tjg@soe.ucsc.edu 831-459-5354 Baskin Engineering, Room 313A From owner-freebsd-fs@FreeBSD.ORG Fri Jul 6 01:16:48 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A31CE106566B for ; Fri, 6 Jul 2012 01:16:48 +0000 (UTC) (envelope-from jusher71@yahoo.com) Received: from nm18-vm0.bullet.mail.ne1.yahoo.com (nm18-vm0.bullet.mail.ne1.yahoo.com [98.138.91.37]) by mx1.freebsd.org (Postfix) with SMTP id 4D0818FC12 for ; Fri, 6 Jul 2012 01:16:48 +0000 (UTC) Received: from [98.138.90.54] by nm18.bullet.mail.ne1.yahoo.com with NNFMP; 06 Jul 2012 01:16:42 -0000 Received: from [98.138.88.235] by tm7.bullet.mail.ne1.yahoo.com with NNFMP; 06 Jul 2012 01:16:42 -0000 Received: from [127.0.0.1] by omp1035.mail.ne1.yahoo.com with NNFMP; 06 Jul 2012 01:16:42 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 727934.90293.bm@omp1035.mail.ne1.yahoo.com Received: (qmail 58422 invoked by uid 60001); 6 Jul 2012 01:16:42 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1341537402; bh=/jk72BvB3MnENaivEqEqMvHMFgE1ZvqzjK0Qom9+ujU=; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Subject:To:MIME-Version:Content-Type; b=R1q1EX9XorofVMzoAkn0Xap8hl3PRFOOIc/82Ptcu32CcuVuCxIMZxpnm96GcYjMCYK/p94+2xwOEkP810gfiYmCK+JW3U79XdgtHhZUht8k3vo5/Kk7UQiODGzQl3sVGj5c7d6FHjeSYLBJuK+D5Qe0pbGZMJ+CPI7peys3phg= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Subject:To:MIME-Version:Content-Type; b=Nw2x9Fg0bwb9CZNdcWktOJ/5sdZWWPx8RNjzKc/YMQeq2gK18RBWZUpf/uX/7v0FmLr4h+ujq3Yo4Tfz7BMOQFZVJwNQy8ii+UWn6n2XFmpPYwKCcEbVRiXa5LNJqhK2NcurL8QKzgVk588UbmRt+TxCGt+G2UQMV8J9b3hIDms=; X-YMail-OSG: Usm.hq4VM1ldUMzxci51p_.iFa4uX50KVILTV_PImGxVcZ7 WhGJSRqbg4qYtgV9KF0hi.mi6UEqznW6qoJFfIQmcxm0BNrumd9TpZ8SKUcC 4uDY8kd6O2baFDpNaVX4Mam49olXpWOKB8OJD87WxYNd4uMK2csZgrMbFXmS qPCo_bzWAzTKyTQ4bERu8QADO8Dwgu8Sis3HJD942REzFXptNGCYjl1MMIZU onXSprE4BQkzzKlhstKgLkWXYLybmrzGSD2ZzzmMckNIsHVp4kh0U0.GPhU8 H5ig6WiXQ1YdvgdsZ.T7OoiZi.fjdjZGeI3lNdAI5dsFSFYq5fTl9cEYfPn9 1kopuserwZpegF6P5fJzAQzUyUon.8_Itb965oHW8eY2HN8ozswFw.AnmCXX 1HA-- Received: from [166.217.14.237] by web122504.mail.ne1.yahoo.com via HTTP; Thu, 05 Jul 2012 18:16:42 PDT X-Mailer: YahooMailClassic/15.0.8 YahooMailWebService/0.8.118.349524 Message-ID: <1341537402.58301.YahooMailClassic@web122504.mail.ne1.yahoo.com> Date: Thu, 5 Jul 2012 18:16:42 -0700 (PDT) From: Jason Usher To: freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailman-Approved-At: Fri, 06 Jul 2012 01:53:56 +0000 Subject: vdev/pool math with combined raidzX vdevs... X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2012 01:16:48 -0000 It's easy to find the failure math for raidz2 and raidz3. But what if you create a pool with 3 different raidz3 vdevs inside of it ? For instance, 3 12-drive raidz3 vdevs in one big pool. For each individual vdev the failure probability is now higher, since not only will it fail when 4 drives in the vdev fail, but it will also fail if four drives in any of the other two vdevs fail. So each raidz3 vdev now has a failure rate higher than vanilla raidz3 ... but what is that new failure rate ? Is it still higher than vanilla raidz2 ? From owner-freebsd-fs@FreeBSD.ORG Fri Jul 6 07:08:52 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8E464106564A for ; Fri, 6 Jul 2012 07:08:52 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 546778FC12 for ; Fri, 6 Jul 2012 07:08:52 +0000 (UTC) Received: by obbun3 with SMTP id un3so18306148obb.13 for ; Fri, 06 Jul 2012 00:08:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=K3uMguqvk54dxcqQDrL/XGLFJkJqK/nB3qUQFX8zd/A=; b=KwTzVDR7whTh+X132KMDhRlbuv6ahTmYzzKJqcg9Prkc3e/06MBcXodBF1JTZ2B7WL LBGwvYGU8/fcKPBSZ717P3gY9xS6Q+nJKyYff+nw9E3wXFY6IeCkhuB0UMcrchJZ04sY wh+8O5QvkRLdV8M9QeVvKs+wSGTSMedMsAaN+kaZMuMarWRkaW7gnPX8dGZkNfbmpnH3 OJBqhAcxUgHAcF3M6V07eBhezlO4SuiJtpYkHWl05Z+DT1RtRkTegK6YZbNggQJ/qaAl ummR+xPpWf8pPlIwKj/2kUdTa/oV1s+NwpcFZEA/aAueyIKPm2vCaMMPJC6cIACgQPEI YJnQ== MIME-Version: 1.0 Received: by 10.60.25.6 with SMTP id y6mr30155815oef.42.1341558526432; Fri, 06 Jul 2012 00:08:46 -0700 (PDT) Received: by 10.76.135.67 with HTTP; Fri, 6 Jul 2012 00:08:46 -0700 (PDT) In-Reply-To: <1341537402.58301.YahooMailClassic@web122504.mail.ne1.yahoo.com> References: <1341537402.58301.YahooMailClassic@web122504.mail.ne1.yahoo.com> Date: Fri, 6 Jul 2012 03:08:46 -0400 Message-ID: From: Zaphod Beeblebrox To: Jason Usher Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-fs@freebsd.org Subject: Re: vdev/pool math with combined raidzX vdevs... X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2012 07:08:52 -0000 Is there some penalty for not googling some basic stats course? OK. This is from memory (hint: you probably should google). p(f) ... the probably of failure of one drive over some unit time (say one year). A two drive RAID-0 array has probability p(2dr0) = 2 * p(f) + p(f). That is (for the logic guys): the array fails if either drive fails. A two drive RAID-1 array has probability p(2dr1) = p(f) * p(f) ... that is: the array fails only if both drives fail. These are simple probabilities. It doesn't count that the RAID-1 case can be said to be more complex ... ie: given a certain failure distribution and a certain replacement distribution, what is the chance of total failure given a single drive failure (ie: 2nd drive failing before you replace the first drive to fail). ... but you get the jist. If no replacements are allowed and 10% of drives fail in a year, the R0 array has a 20% chance of failing in 1 year and the R1 array has a 2% chance ... this also says nothing about the fact that the drives are the same and have done mostly the same reads and writes and that their failure may not be independant. Geez... it's getting complex. Now... we start getting into the hard stuff. For a RAID-Z(1) array, you want to think about the possibility of 2 failures out of 12 drives (or ... if you're feeling up to it, the probability of the first failure and then the probability of the second failure given the first before you can replace it). p(12drz1) = 12 * p(f) * 11 * p(f) --- if no replacements are allowed and drive failures are independent. To kick it up a notch, the "11 * p(f)" can be replaced with eleven times the probability of failure before replacement --- which you can calculate with your MTBF tables and your service level for replacing drives in the array. Similarly, p(12drz2) = 12 * p(f) * 11 * p(f) * 10 * p(f) p(12drz3) = 12 * p(f) * 11 * p(f) * 10 * p(f) * 9 * p(f) ... again with those assumptions are more complex probabilities given your replacement strategy. ... so, again with simplistic assumptions, p(36drz3 --- 12 drives, 3 groups) = p(12drz3) * 3 A "vanilla" RAID-Z2 (if I make an assumption to what you're saying) is: p(36drz2) = 36 * p(f) * 35 * p(f) ... but I can't directly answer you question without knowing a) the structure of the RAID-Z2 array and p(f). If we use a 1% figure for p(f), then P(36drz3,12,3) = 0.035% and p(36drz2) = 4.3% ... that is the raid-Z2 case (one group of 36 drives, two redundant --- which is crazy) is 4.3% likely to fail where the 3-group RAID-Z3 is only 0.035% likely to fail. As a more sane comparison, p(36drz2,12,3) = 3.8% now it's worth saying that all these calculations assume that you never come to replace drives in the array and that drive failures are independent... neither of these is likely true. If you had (say) a 4 hour 7/24 contract with someone, the chances of more drives failing before a failed drive is replaced are much smaller. As for the independence of drive failures... that's a discussion over beer. Put simply, you add the probabilities of things where any can cause the failure (either drive of R0 failing, any one of the 3 plexes of a complex array failing) and you multiply things where all must fail to produce failure. From owner-freebsd-fs@FreeBSD.ORG Fri Jul 6 07:11:35 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CAF411065786 for ; Fri, 6 Jul 2012 07:11:35 +0000 (UTC) (envelope-from grarpamp@gmail.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 963988FC08 for ; Fri, 6 Jul 2012 07:11:35 +0000 (UTC) Received: by obbun3 with SMTP id un3so18309848obb.13 for ; Fri, 06 Jul 2012 00:11:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=bsiAHTow7JVGCQsPQiwNhmJ7oXKDK045eR7ed/WLSNs=; b=oP2JHgDimdS3VWiEkQEOqvJ/D43yEAqQVOfld4srxo6PROFsYw6LIxWpl9PfgPgmZ5 VROdvscN1+VsvwWm2ru7T2R2+ijtvLAAeIXJtLWyXyhcPdYiY9+XL/ILjyYuYnJMZV1H aEDqeI8Q3/hSrCzyr6fLuUk2a/NWY10STEEBZMTslkNXLqLlK1JI4L85dzu92HtsbsCm DMDG3YZD2+pbwIkQHGh+dyxca0ZS84xQi9ZC3eZ65Fmw1BxBA8LV/O02oPF5F/MFolWL H5AduGOhSk4y/wvDlKglliLo+jLzxWsJGzBtLCNpJiWafo9+Gca0KVBCjHKywWbjjkqv aMjg== MIME-Version: 1.0 Received: by 10.60.30.233 with SMTP id v9mr29628927oeh.6.1341558695229; Fri, 06 Jul 2012 00:11:35 -0700 (PDT) Received: by 10.76.81.10 with HTTP; Fri, 6 Jul 2012 00:11:35 -0700 (PDT) Date: Fri, 6 Jul 2012 03:11:35 -0400 Message-ID: From: grarpamp To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 Subject: Port NetBSD's UDF implementation X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2012 07:11:35 -0000 Very cool Oleksandr... soon hopefully we can finally read, and create for writing, UDF filesystems. I could not even mount them today. You may also like to say hi to a guy named Bryan who also claimed to be working on UDF a year ago, and is bcc'd on this as he said hi off list. Someone also told me about sysutils/udfclient. http://en.wikipedia.org/wiki/Universal_Disk_Format http://www.osta.org/specs/index.htm With luck we have UDF soon :) From owner-freebsd-fs@FreeBSD.ORG Fri Jul 6 11:27:37 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CBDF11065676 for ; Fri, 6 Jul 2012 11:27:37 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-scalar.mail.uoguelph.ca (esa-scalar.mail.uoguelph.ca [66.199.40.18]) by mx1.freebsd.org (Postfix) with ESMTP id 6A1768FC15 for ; Fri, 6 Jul 2012 11:27:37 +0000 (UTC) Received: from zcs3.mail.uoguelph.ca (new.mail.uoguelph.ca [131.104.93.37]) by esa-scalar.mail.uoguelph.ca (8.14.1/8.14.1) with ESMTP id q66A4MBR025780; Fri, 6 Jul 2012 06:04:22 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 241C1B3FE3; Fri, 6 Jul 2012 06:04:22 -0400 (EDT) Date: Fri, 6 Jul 2012 06:04:22 -0400 (EDT) From: Rick Macklem To: Tim Gustafson Message-ID: <1060732353.47610.1341569062083.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-fs@freebsd.org Subject: Re: FreeBSD 9.0 + ZFS + NFSv4 + Kerberos X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2012 11:27:37 -0000 Tim Gustafson wrote: > Hi, > > I'd like to set up a FreeBSD 9.0 box as a ZFS+NFSv4+Kerberos server. > So far, I can mount a file system from a client machine, but whenever > I try to do anything on that file system, I get errors that look like > this: > > tjg@junta: cd /mnt > nfsv4 err=10016 > nfsv4 err=10016 > /mnt: Input/output error. > Read this: http://code.google.com/p/macnfsv4/wiki/FreeBSD8KerberizedNFSSetup (Still basically applies to FreeBSD9.) > I can kinit on both boxes, and have done so on my client box; klist > shows a valid ticket on the client box. I can "mount /mnt" on the > client without any problems. > The client must have the appropriate TGT at time of mount. Unless you apply the patch mentioned in the above wiki and have the correct /etc/keytab entry in the client,the mount can only be done by a non-root user after they have done a kinit. (vfs.usermount=1) > Here are the relevant configuration files: > > server:/etc/rc.conf: > > nfs_server_enable="yes" > nfsv4_server_enable="yes" > mountd_enable="yes" > mountd_flags="-r" > rpcbind_enable="yes" > rpc_lockd_enable="yes" > rpc_statd_enable="yes" > gssd_enable="yes" > > server:/etc/exports: > > V4: /tank/export -sec=krb5p > > client:/etc/rc.conf: > > nfs_client_enable="yes" > rpc_lockd_enable="yes" > rpc_statd_enable="yes" > rpcbind_enable="yes" > devfs_enable="yes" > gssd_enable="yes" > > client:/etc/fstab: > > server:/ /mnt nfs rw,noauto,nfsv4,sec=krb5p 0 0 > Won't work unless the client has the above mentioned patch and the correct /etc/keytab entry. Good luck with it, rick > -- > > Tim Gustafson > tjg@soe.ucsc.edu > 831-459-5354 > Baskin Engineering, Room 313A > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@FreeBSD.ORG Fri Jul 6 11:29:41 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C13971065677 for ; Fri, 6 Jul 2012 11:29:41 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: from smtprelay03.ispgateway.de (smtprelay03.ispgateway.de [80.67.31.26]) by mx1.freebsd.org (Postfix) with ESMTP id 7364F8FC1B for ; Fri, 6 Jul 2012 11:29:41 +0000 (UTC) Received: from [87.79.197.110] (helo=fabiankeil.de) by smtprelay03.ispgateway.de with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.68) (envelope-from ) id 1Sn6YI-0003dg-HO; Fri, 06 Jul 2012 13:18:22 +0200 Date: Fri, 6 Jul 2012 13:18:10 +0200 From: Fabian Keil To: FreeBSD Filesystems Message-ID: <20120706131810.76ca679b@fabiankeil.de> In-Reply-To: References: <20120705192158.503fa2e5@fabiankeil.de> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA1; boundary="Sig_/l=Oyc=eQ+K.kDXSeKC00s7U"; protocol="application/pgp-signature" X-Df-Sender: Nzc1MDY3 Cc: Oleksandr Dudinskyi Subject: Re: Port NetBSD's UDF implementation X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: FreeBSD Filesystems List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2012 11:29:41 -0000 --Sig_/l=Oyc=eQ+K.kDXSeKC00s7U Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Freddie Cash wrote: > ---------- Forwarded message ---------- > From: Oleksandr Dudinskyi > Date: Thu, Jul 5, 2012 at 12:59 PM > Subject: Re: Port NetBSD's UDF implementation > To: Freddie Cash >=20 >=20 > Sorry, I did not know this . Project in Google code. > http://code.google.com/p/port-netbsd-udf-implementation/source/browse/udf= 2_patch.diff I successfully tested it with four BD images and two DVDs so far, but had to change one #include to get it to compile: diff --git a/sbin/mount_udf2/mount_udf.c b/sbin/mount_udf2/mount_udf.c index 8267977..d635e8c 100644 --- a/sbin/mount_udf2/mount_udf.c +++ b/sbin/mount_udf2/mount_udf.c @@ -52,8 +52,8 @@ #include #include #include -#include +#include #include "fs/udf2/udf_mount.h" #include Fabian --Sig_/l=Oyc=eQ+K.kDXSeKC00s7U Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAk/2yXUACgkQBYqIVf93VJ062wCgy2CyWedBhKvbyFitQYGKOfR+ fwkAniwyZFXuMrB5XHTTx5hCcV/htQGS =lLnd -----END PGP SIGNATURE----- --Sig_/l=Oyc=eQ+K.kDXSeKC00s7U-- From owner-freebsd-fs@FreeBSD.ORG Fri Jul 6 12:20:07 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id DAF681065676 for ; Fri, 6 Jul 2012 12:20:07 +0000 (UTC) (envelope-from attila.bogar@linguamatics.com) Received: from mail.linguamatics.com (mail.linguamatics.com [188.39.80.203]) by mx1.freebsd.org (Postfix) with ESMTP id 764B08FC0A for ; Fri, 6 Jul 2012 12:20:02 +0000 (UTC) Received: from [10.252.10.232] (random.linguamatics.com [10.252.10.232]) by mail.linguamatics.com (Postfix) with ESMTPSA id B02D1EFB44C for ; Fri, 6 Jul 2012 13:19:55 +0100 (BST) Message-ID: <4FF6D7EE.3000806@linguamatics.com> Date: Fri, 06 Jul 2012 13:19:58 +0100 From: =?ISO-8859-1?Q?Attila_Bog=E1r?= User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:13.0) Gecko/20120615 Thunderbird/13.0.1 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: ZFS: .zfs/snapshot directory vanished X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2012 12:20:07 -0000 Hi, I'm running FreeBSD-9 STABLE. I noticed, that my .zfs/snapshot directories vanished. Not all off them though, a few datasets have it available. # cd /export/home/user/.zfs/snapshot /export/home/user/.zfs/snapshot: Not a directory. # cd /export/home/user/.zfs # find . . find: ./snapshot: Bad file descriptor ./shares This looks similar to http://lists.freebsd.org/pipermail/freebsd-fs/2012-February/013684.html - though it's on 9 and I never turned on dedup. I appreciate any help solving this problem. Thanks, Attila From owner-freebsd-fs@FreeBSD.ORG Fri Jul 6 12:35:56 2012 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EF609106566C for ; Fri, 6 Jul 2012 12:35:56 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from smtp.infracaninophile.co.uk (smtp6.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3cd3:cd67:fafa:3d78]) by mx1.freebsd.org (Postfix) with ESMTP id 519298FC14 for ; Fri, 6 Jul 2012 12:35:56 +0000 (UTC) Received: from seedling.black-earth.co.uk (seedling.black-earth.co.uk [81.187.76.163]) (authenticated bits=0) by smtp.infracaninophile.co.uk (8.14.5/8.14.5) with ESMTP id q66CZlcn097127 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Fri, 6 Jul 2012 13:35:48 +0100 (BST) (envelope-from matthew@FreeBSD.org) X-DKIM: OpenDKIM Filter v2.5.2 smtp.infracaninophile.co.uk q66CZlcn097127 Authentication-Results: smtp.infracaninophile.co.uk/q66CZlcn097127; dkim=none (no signature); dkim-adsp=none Message-ID: <4FF6DB9A.7040905@FreeBSD.org> Date: Fri, 06 Jul 2012 13:35:38 +0100 From: Matthew Seaman User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:13.0) Gecko/20120614 Thunderbird/13.0.1 MIME-Version: 1.0 To: =?ISO-8859-1?Q?Attila_Bog=E1r?= References: <4FF6D7EE.3000806@linguamatics.com> In-Reply-To: <4FF6D7EE.3000806@linguamatics.com> X-Enigmail-Version: 1.4.2 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigA3AA6B493B23A929CF725A08" X-Virus-Scanned: clamav-milter 0.97.5 at lucid-nonsense.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lucid-nonsense.infracaninophile.co.uk Cc: freebsd-fs@FreeBSD.org Subject: Re: ZFS: .zfs/snapshot directory vanished X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2012 12:35:57 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigA3AA6B493B23A929CF725A08 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 06/07/2012 13:19, Attila Bog=E1r wrote: > I'm running FreeBSD-9 STABLE. >=20 > I noticed, that my .zfs/snapshot directories vanished. Not all off the= m > though, a few datasets have it available. >=20 > # cd /export/home/user/.zfs/snapshot > /export/home/user/.zfs/snapshot: Not a directory. >=20 > # cd /export/home/user/.zfs > # find . > . > find: ./snapshot: Bad file descriptor > ./shares >=20 > This looks similar to > http://lists.freebsd.org/pipermail/freebsd-fs/2012-February/013684.html= > - though it's on 9 and I never turned on dedup. >=20 > I appreciate any help solving this problem. >=20 Actually, it turned out that turning dedup on or off had no effect. That was a red herring. Neither did splitting the mirror and building a completely new zpool and using zfs send ... | zfs receive ... to transfer the data prevent the problem occurring. There's a PR about this: kern/156781 Only solution I've found to getting the snapshot dirs back is to reboot, but ZFS will crash on shutdown when this problem is occurring. Cheers, Matthew --=20 Dr Matthew J Seaman MA, D.Phil. PGP: http://www.infracaninophile.co.uk/pgpkey --------------enigA3AA6B493B23A929CF725A08 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.16 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk/226MACgkQ8Mjk52CukIya9QCfYEbFvKE9S6i2G8XCmIbfymdY C7AAnRASlcbZhqz8anMgNNRHMWjURnfZ =2ipG -----END PGP SIGNATURE----- --------------enigA3AA6B493B23A929CF725A08-- From owner-freebsd-fs@FreeBSD.ORG Fri Jul 6 13:26:39 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EECE0106566B for ; Fri, 6 Jul 2012 13:26:38 +0000 (UTC) (envelope-from dudinskyj@gmail.com) Received: from mail-gh0-f182.google.com (mail-gh0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id A21758FC1A for ; Fri, 6 Jul 2012 13:26:38 +0000 (UTC) Received: by ghbz22 with SMTP id z22so9980350ghb.13 for ; Fri, 06 Jul 2012 06:26:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=VMb/zTsR4A1OZlrVoui/17RigOkn/2kLRHbmSZtQ4Eg=; b=FF8froNV3UGjp3jhAnjuEqmU8RnEjXDXGGpk4EVnScG2cFcCiYgsDdD1w2em4dED3s INzRCk7owhBY+V4PYS0MKBhAEo8gFvw9BJy9Kwr4kzQ45whQtcPA0gynAtnmeytSR0El BJj9x/CnX4J4jC5+AEddEH643XAiisXG5kJXqG0yLw1c9kD3vBA6SP413FXV5cQsD6RL eGPg4ha6HFpk5Tfnus0lb9YR/35uJksVpeUQ3l6Isk1RJI9C535YThtdRxyjx4i85fe+ /L1SDAbHk+614oTYOdiPGnHdxyuokduuURYCJwYiYpM0/ZSI7XGXX6evSTdJxsCipY+X gvjg== MIME-Version: 1.0 Received: by 10.60.20.197 with SMTP id p5mr30923908oee.32.1341581197890; Fri, 06 Jul 2012 06:26:37 -0700 (PDT) Received: by 10.182.24.168 with HTTP; Fri, 6 Jul 2012 06:26:37 -0700 (PDT) In-Reply-To: <20120706131810.76ca679b@fabiankeil.de> References: <20120705192158.503fa2e5@fabiankeil.de> <20120706131810.76ca679b@fabiankeil.de> Date: Fri, 6 Jul 2012 16:26:37 +0300 Message-ID: From: Oleksandr Dudinskyi To: FreeBSD Filesystems Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: Port NetBSD's UDF implementation X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2012 13:26:39 -0000 Sorry for this trouble, error corrected. On 6 July 2012 14:18, Fabian Keil wrote: > Freddie Cash wrote: > > > ---------- Forwarded message ---------- > > From: Oleksandr Dudinskyi > > Date: Thu, Jul 5, 2012 at 12:59 PM > > Subject: Re: Port NetBSD's UDF implementation > > To: Freddie Cash > > > > > > Sorry, I did not know this . Project in Google code. > > > http://code.google.com/p/port-netbsd-udf-implementation/source/browse/udf2_patch.diff > > I successfully tested it with four BD images and two DVDs so > far, but had to change one #include to get it to compile: > > diff --git a/sbin/mount_udf2/mount_udf.c b/sbin/mount_udf2/mount_udf.c > index 8267977..d635e8c 100644 > --- a/sbin/mount_udf2/mount_udf.c > +++ b/sbin/mount_udf2/mount_udf.c > @@ -52,8 +52,8 @@ > #include > #include > #include > -#include > > +#include > #include "fs/udf2/udf_mount.h" > > #include > > Fabian > -- Regards, Oleksandr Dudinskyi. From owner-freebsd-fs@FreeBSD.ORG Fri Jul 6 13:31:27 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 435B51065673 for ; Fri, 6 Jul 2012 13:31:27 +0000 (UTC) (envelope-from nowakpl@platinum.linux.pl) Received: from platinum.linux.pl (platinum.edu.pl [81.161.192.4]) by mx1.freebsd.org (Postfix) with ESMTP id E7AF38FC17 for ; Fri, 6 Jul 2012 13:31:26 +0000 (UTC) Received: by platinum.linux.pl (Postfix, from userid 87) id 61A7947E14; Fri, 6 Jul 2012 15:22:58 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on platinum.linux.pl X-Spam-Level: X-Spam-Status: No, score=-1.5 required=3.0 tests=ALL_TRUSTED,AWL autolearn=disabled version=3.3.2 Received: from [172.19.191.4] (unknown [83.151.38.73]) by platinum.linux.pl (Postfix) with ESMTPA id C5B0B47E0F for ; Fri, 6 Jul 2012 15:22:53 +0200 (CEST) Message-ID: <4FF6E6A3.1010000@platinum.linux.pl> Date: Fri, 06 Jul 2012 15:22:43 +0200 From: Adam Nowacki User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.2.28) Gecko/20120306 Thunderbird/3.1.20 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <1341537402.58301.YahooMailClassic@web122504.mail.ne1.yahoo.com> In-Reply-To: <1341537402.58301.YahooMailClassic@web122504.mail.ne1.yahoo.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: vdev/pool math with combined raidzX vdevs... X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2012 13:31:27 -0000 On 2012-07-06 03:16, Jason Usher wrote: > It's easy to find the failure math for raidz2 and raidz3. > > But what if you create a pool with 3 different raidz3 vdevs inside of it ? > > For instance, 3 12-drive raidz3 vdevs in one big pool. > > For each individual vdev the failure probability is now higher, since not only will it fail when 4 drives in the vdev fail, but it will also fail if four drives in any of the other two vdevs fail. > > So each raidz3 vdev now has a failure rate higher than vanilla raidz3 ... but what is that new failure rate ? Is it still higher than vanilla raidz2 ? If you're asking if 3-stripe x 12-drive raidz3 has a higher failure probability than 36-drive raidz2 then the answer is no. It also doesn't have a higher failure probability than 36-drive raidz3. But if you compare 3-stripe x 12-drive raidz3 with no stripe 12-drive raidz2 then for a disk failure probability less than ~0.16 raidz3 wins, otherwise raidz2. From owner-freebsd-fs@FreeBSD.ORG Fri Jul 6 13:53:37 2012 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id A501B106564A; Fri, 6 Jul 2012 13:53:37 +0000 (UTC) (envelope-from attila.bogar@linguamatics.com) Received: from mail.linguamatics.com (mail.linguamatics.com [188.39.80.203]) by mx1.freebsd.org (Postfix) with ESMTP id 5D9578FC0A; Fri, 6 Jul 2012 13:53:37 +0000 (UTC) Received: from [10.252.10.232] (random.linguamatics.com [10.252.10.232]) by mail.linguamatics.com (Postfix) with ESMTPSA id 6D738EFB44C; Fri, 6 Jul 2012 14:53:36 +0100 (BST) Message-ID: <4FF6EDE2.7060801@linguamatics.com> Date: Fri, 06 Jul 2012 14:53:38 +0100 From: =?ISO-8859-1?Q?Attila_Bog=E1r?= User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:13.0) Gecko/20120615 Thunderbird/13.0.1 MIME-Version: 1.0 To: Matthew Seaman References: <4FF6D7EE.3000806@linguamatics.com> <4FF6DB9A.7040905@FreeBSD.org> In-Reply-To: <4FF6DB9A.7040905@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org Subject: Re: ZFS: .zfs/snapshot directory vanished X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2012 13:53:37 -0000 On 06/07/12 13:35, Matthew Seaman wrote: > Only solution I've found to getting the snapshot dirs back is to > reboot, but ZFS will crash on shutdown when this problem is occurring. > Cheers, Matthew This PR is more than a year old spanning all releases from 8.1 to 9.0. Is there an ugly workaround to wipe all pool disk, recreate the pool, restore data and crossing your fingers? Has anyone tried this? Is there a suspicion, what triggers this issue? Thanks, Attila From owner-freebsd-fs@FreeBSD.ORG Fri Jul 6 14:10:56 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9DAB7106564A for ; Fri, 6 Jul 2012 14:10:56 +0000 (UTC) (envelope-from daniel@digsys.bg) Received: from smtp-sofia.digsys.bg (smtp-sofia.digsys.bg [193.68.3.230]) by mx1.freebsd.org (Postfix) with ESMTP id 2165A8FC12 for ; Fri, 6 Jul 2012 14:10:55 +0000 (UTC) Received: from dcave.digsys.bg (dcave.digsys.bg [192.92.129.5]) (authenticated bits=0) by smtp-sofia.digsys.bg (8.14.5/8.14.5) with ESMTP id q66E7BQM080365 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Fri, 6 Jul 2012 17:07:11 +0300 (EEST) (envelope-from daniel@digsys.bg) Message-ID: <4FF6F10F.8040006@digsys.bg> Date: Fri, 06 Jul 2012 17:07:11 +0300 From: Daniel Kalchev User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:10.0.5) Gecko/20120607 Thunderbird/10.0.5 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <1341537402.58301.YahooMailClassic@web122504.mail.ne1.yahoo.com> In-Reply-To: <1341537402.58301.YahooMailClassic@web122504.mail.ne1.yahoo.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: vdev/pool math with combined raidzX vdevs... X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2012 14:10:56 -0000 On 06.07.12 04:16, Jason Usher wrote: > It's easy to find the failure math for raidz2 and raidz3. > > But what if you create a pool with 3 different raidz3 vdevs inside of it ? > > For instance, 3 12-drive raidz3 vdevs in one big pool. > > For each individual vdev the failure probability is now higher, since not only will it fail when 4 drives in the vdev fail, but it will also fail if four drives in any of the other two vdevs fail. > > So each raidz3 vdev now has a failure rate higher than vanilla raidz3 ... but what is that new failure rate ? Is it still higher than vanilla raidz2 ? Let's also not forget, that if a drive fails in any of the 3 raidz3 stripes, it can be replaced independently and the sucessful replacement depends only on the drives in the same stripe, not all drives in the zpool. It also means that such a resilvering will only stress small part of the pool, while at the same time other drives failing in other vdevs will not impact your risk analysis. Further, if you have 3 vdevs, you could be replacing three drives at the same time, while with one single vdev you will only be able to replace one failed drive at a time. Others have provided some math already. Daniel From owner-freebsd-fs@FreeBSD.ORG Fri Jul 6 14:52:07 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A3A3A106566B for ; Fri, 6 Jul 2012 14:52:07 +0000 (UTC) (envelope-from jusher71@yahoo.com) Received: from nm22-vm3.bullet.mail.ne1.yahoo.com (nm22-vm3.bullet.mail.ne1.yahoo.com [98.138.91.152]) by mx1.freebsd.org (Postfix) with SMTP id 64B218FC0A for ; Fri, 6 Jul 2012 14:52:07 +0000 (UTC) Received: from [98.138.90.52] by nm22.bullet.mail.ne1.yahoo.com with NNFMP; 06 Jul 2012 14:52:01 -0000 Received: from [98.138.89.197] by tm5.bullet.mail.ne1.yahoo.com with NNFMP; 06 Jul 2012 14:52:01 -0000 Received: from [127.0.0.1] by omp1055.mail.ne1.yahoo.com with NNFMP; 06 Jul 2012 14:52:01 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 114507.84514.bm@omp1055.mail.ne1.yahoo.com Received: (qmail 18995 invoked by uid 60001); 6 Jul 2012 14:52:00 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1341586320; bh=V5T7jxvSy46KOZ3pjyM27IAGVOJWfcd+CDxitWiDBxA=; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=aTSVuC2dMf1Cwab1kRfIFo5jG2mFhkNH+F2MTmX65EzBHK9+azKExN2DGnTwmoNiUpam2iagqzHUlm+Jl+O9jPMEuDK6+xuNWGzxlEBQBsx4qrpLhO5a65IfI4kPLlCkPdNkHD3/xuNByE9ZKslCACVZVSD7qjSHwbycC3pHNq8= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Mailer:Message-ID:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=hknasFCjvN+dDCY5s9DW3SCYHM+Xh0SYP2IPClXsoT47LqAOPVSndLBDCrzcFUCjX6jdeJ2BcwtUFj5+Ja0s1MYZjOCWjpWRxMw4iJRDT1LyTe79x+jNR11VKi5BnRWeW631SVabihVL+h+wjRw+4fnTIQQls2wHwGYPMjdR5o0=; X-YMail-OSG: TzeQAmkVM1ncNYsn3x2pPL4T4UBUpPN506ZMbV6eMukadQY o3RHedV_IMkse_6HhV60RIvK_wcxS6EmFpGxggVnzgWTK8igwIi0B_JMQiuN vn0f1GWIBiQU2qdfNyVLUhhTjc5yctxJjewTHjxkWzyi35P8r6fW0tMODv0x nBffVRmCWS6Nz7BubzkvYIym74TV6HjGaFhZz2TfDne3gHRMtpqeXv7Odszz FK4r.KPsQHP3oLzjqzWnCcn1VFFnsDDVIGYM4eO0fcTxCoWMmvLJV5sgLba. VN6Uwizrnkouc2IEce3pzfOPDCyqb.2Q7GGHcsXY9XTt._mObFg_l4BTcOc6 dTYmpITMvBVm024D3acknT3WKwJ95gm8oNVk_FS92NBa3uB_LPBOp5FyNzNH Q3OHw8Li3D.8ZV9nFvGau2Vl5e4pwKVg0n6IP.kuO9LJa0C_PnmH24X89lyE if_Po.4OdFQYpbYIT4Qe.selMe5Hij0uQ6qL0zRiVyUJRnbxAMBh1OgUs Received: from [32.178.191.35] by web122502.mail.ne1.yahoo.com via HTTP; Fri, 06 Jul 2012 07:52:00 PDT X-Mailer: YahooMailClassic/15.0.8 YahooMailWebService/0.8.120.356233 Message-ID: <1341586320.93244.YahooMailClassic@web122502.mail.ne1.yahoo.com> Date: Fri, 6 Jul 2012 07:52:00 -0700 (PDT) From: Jason Usher To: Zaphod Beeblebrox In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org Subject: Re: vdev/pool math with combined raidzX vdevs... X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2012 14:52:07 -0000 =0A=0A--- On Fri, 7/6/12, Zaphod Beeblebrox wrote:=0A= =0A> Is there some penalty for not=0A> googling some basic stats course?=A0= OK.=0A> This is from memory (hint: you probably should google).=0A=0A=0AAc= tually I spent a few hours googling zfs probabilities, and I cannot find an= y discussion of the changes in failure probability when multiple raidzX are= striped in a single pool.=0A=0AI did not try to google the math and do it = myself, as I wouldn't trust my own results.=0A=0A=0A> Similarly,=0A> =0A> p= (12drz2) =3D 12 * p(f) * 11 * p(f) * 10 * p(f)=0A> p(12drz3) =3D 12 * p(f) = * 11 * p(f) * 10 * p(f) * 9 * p(f)=0A> =0A> ... again with those assumption= s are more complex=0A> probabilities given=0A> your replacement strategy.= =0A> =0A> ... so, again with simplistic assumptions,=0A> =0A> p(36drz3 --- = 12 drives, 3 groups) =3D p(12drz3) * 3=0A> =0A> A "vanilla" RAID-Z2 (if I m= ake an assumption to what you're=0A> saying) is:=0A> =0A> p(36drz2) =3D 36 = * p(f) * 35 * p(f)=0A> =0A> ... but I can't directly answer you question wi= thout knowing=0A> a) the=0A> structure of the RAID-Z2 array and p(f).=A0 If= we use a=0A> 1% figure for=0A> p(f), then P(36drz3,12,3) =3D 0.035% and p(= 36drz2) =3D 4.3%=0A> =0A> ... that is the raid-Z2 case (one group of 36 dri= ves, two=0A> redundant=0A> --- which is crazy) is 4.3% likely to fail where= the 3-group=0A> RAID-Z3=0A> is only 0.035% likely to fail.=A0 As a more sa= ne=0A> comparison,=0A> p(36drz2,12,3) =3D 3.8%=0A=0A=0AOk, you're right - I= did not specify the structure of the raidz2 array.=0A=0AWhat I meant to co= mpare was the failure probabilities of:=0A=0A- a single raidz2 vdev made up= of 12 disks (10 data, 2 parity)=0A=0Avs.=0A=0A- a single raidz3 vdev made = up of 12 disks (9 data, 3 parity)=0A=0Avs.=0A=0A- a single raidz3 vdev made= up of 12 disks (9 data, 3 parity) which ALSO happens to be participating i= n a stripe with two other identical raidz3 vdevs, all in one zpool.=0A=0A= =0AI can see some probabilites for the first two examples here:=0A=0Ahttp:/= /hardforum.com/showthread.php?t=3D1621123=0A=0A(search for first post on pa= ge by "john4200")=0A=0Aas well as your own math, which appears to be the sa= me.=0A=0A=0AAlso, I think we don't need to specify the failure rate, F, sin= ce we are merely comparing three scenarios, and can compare results that st= ill contain an 'F' variable in them ... that is, our answers can contain a = yet undefined 'F', right ?=0A=0A=0AAs for myself, I have decided that raidz= 2 is "not enough" for me, and at the same time, would really, really like t= o combine three raidz3 into a single zpool ... but I don't want to do that = if that configuration brings me back to raidz2-ish failure probabilities... From owner-freebsd-fs@FreeBSD.ORG Fri Jul 6 16:40:46 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4FFE81065672 for ; Fri, 6 Jul 2012 16:40:46 +0000 (UTC) (envelope-from martin@lispworks.com) Received: from lwfs1-cam.cam.lispworks.com (mail.lispworks.com [193.34.186.230]) by mx1.freebsd.org (Postfix) with ESMTP id DA5508FC19 for ; Fri, 6 Jul 2012 16:40:45 +0000 (UTC) Received: from higson.cam.lispworks.com (higson.cam.lispworks.com [192.168.1.7]) by lwfs1-cam.cam.lispworks.com (8.14.5/8.14.5) with ESMTP id q66GUDx3049521; Fri, 6 Jul 2012 17:30:13 +0100 (BST) (envelope-from martin@lispworks.com) Received: from higson.cam.lispworks.com (localhost.localdomain [127.0.0.1]) by higson.cam.lispworks.com (8.14.4) id q66GUD2e019055; Fri, 6 Jul 2012 17:30:13 +0100 Received: (from martin@localhost) by higson.cam.lispworks.com (8.14.4/8.14.4/Submit) id q66GUDPD019051; Fri, 6 Jul 2012 17:30:13 +0100 Date: Fri, 6 Jul 2012 17:30:13 +0100 Message-Id: <201207061630.q66GUDPD019051@higson.cam.lispworks.com> From: Martin Simmons To: freebsd-fs@freebsd.org In-reply-to: (message from Zaphod Beeblebrox on Fri, 6 Jul 2012 03:08:46 -0400) References: <1341537402.58301.YahooMailClassic@web122504.mail.ne1.yahoo.com> Subject: Re: vdev/pool math with combined raidzX vdevs... X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2012 16:40:46 -0000 >>>>> On Fri, 6 Jul 2012 03:08:46 -0400, Zaphod Beeblebrox said: > > Is there some penalty for not googling some basic stats course? OK. > This is from memory (hint: you probably should google). > > p(f) ... the probably of failure of one drive over some unit time (say > one year). A two drive RAID-0 array has probability p(2dr0) = 2 * > p(f) + p(f). That is (for the logic guys): the array fails if either > drive fails. This looks wrong to me because it can be > 1 if p(f) > 1/3 :-) I think it should be p(2dr0) = (2 * p(f) * (1 - p(f))) + (p(f) * p(f)) I.e. the probability of either drive failing and the other one not failing plus the probability of both drives failing simultaneously. Or another way of thinking about it: p(2dr0) = 1 - ((1 - p(f)) * (1 - p(f))) I.e. the inverse of the probability of neither drive failing __Martin From owner-freebsd-fs@FreeBSD.ORG Sat Jul 7 16:34:51 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C1212106566B for ; Sat, 7 Jul 2012 16:34:51 +0000 (UTC) (envelope-from radiomlodychbandytow@o2.pl) Received: from moh1-ve3.go2.pl (moh1-ve3.go2.pl [193.17.41.134]) by mx1.freebsd.org (Postfix) with ESMTP id 796D98FC0C for ; Sat, 7 Jul 2012 16:34:51 +0000 (UTC) Received: from moh1-ve3.go2.pl (unknown [10.0.0.134]) by moh1-ve3.go2.pl (Postfix) with ESMTP id 4D5149D8002 for ; Sat, 7 Jul 2012 18:34:45 +0200 (CEST) Received: from unknown (unknown [10.0.0.108]) by moh1-ve3.go2.pl (Postfix) with SMTP for ; Sat, 7 Jul 2012 18:34:45 +0200 (CEST) Received: from unknown [93.175.69.74] by poczta.o2.pl with ESMTP id EdhEtz; Sat, 07 Jul 2012 18:34:45 +0200 Message-ID: <4FF8651F.8090102@o2.pl> Date: Sat, 07 Jul 2012 18:34:39 +0200 From: =?UTF-8?B?UmFkaW8gbcWCb2R5Y2ggYmFuZHl0w7N3?= User-Agent: Mozilla/5.0 (Windows NT 5.2; WOW64; rv:13.0) Gecko/20120614 Thunderbird/13.0.1 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <20120706120031.A47A41065733@hub.freebsd.org> In-Reply-To: <20120706120031.A47A41065733@hub.freebsd.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-O2-Trust: 1, 33 X-O2-SPF: neutral Subject: Re: freebsd-fs Digest, Vol 472, Issue 5 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Jul 2012 16:34:51 -0000 On 2012-07-06 14:00, freebsd-fs-request@freebsd.org wrote:> It's easy to find the failure math for raidz2 and raidz3. > > But what if you create a pool with 3 different raidz3 vdevs inside of it ? > > For instance, 3 12-drive raidz3 vdevs in one big pool. > > For each individual vdev the failure probability is now higher, since not only will it fail when 4 drives in the vdev fail, but it will also fail if four drives in any of the other two vdevs fail. > > So each raidz3 vdev now has a failure rate higher than vanilla raidz3 ... but what is that new failure rate ? Is it still higher than vanilla raidz2 ? Skip these calculations. They all assume that drive failures are independent and it's not the case in the real world. There's been a good study on the topic of drive failures several years ago, http://static.usenix.org/events/fast07/tech/schroeder/schroeder_html/index.html Among other findings they say: "We also present strong evidence for the existence of correlations between disk replacement interarrivals." -- Twoje radio