From owner-freebsd-fs@FreeBSD.ORG Sun Feb 21 01:58:25 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DE74B1065676 for ; Sun, 21 Feb 2010 01:58:24 +0000 (UTC) (envelope-from mailinglists@debank.tv) Received: from smtpout01.onlinespamfilter.nl (smtpout01.onlinespamfilter.nl [217.21.240.161]) by mx1.freebsd.org (Postfix) with ESMTP id 6DAD58FC15 for ; Sun, 21 Feb 2010 01:58:24 +0000 (UTC) Received: from smtp.onlinespamfilter.nl (localhost [127.0.0.1]) by smtp.onlinespamfilter.nl (Postfix) with ESMTP id D4F9825079 for ; Sun, 21 Feb 2010 02:58:17 +0100 (CET) Received: from smtp.debank.tv (59-80-ftth.onsneteindhoven.nl [88.159.80.59]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.onlinespamfilter.nl (Postfix) with ESMTP for ; Sun, 21 Feb 2010 02:58:17 +0100 (CET) Received: from smtp.debank.tv (smtp.debank.tv [172.16.143.25]) by smtp.debank.tv (Postfix) with ESMTP id 875F11DB2188 for ; Sun, 21 Feb 2010 02:58:17 +0100 (CET) Received: from jupi.local (121-73-247-214.broadband.telstraclear.net [121.73.247.214]) (using SSLv3 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: rob) by smtp.debank.tv (Postfix) with ESMTPSA id 5F9A01DB2161 for ; Sun, 21 Feb 2010 02:58:16 +0100 (CET) Message-ID: Date: Sun, 21 Feb 2010 14:58:12 +1300 From: mailinglists User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.7) Gecko/20100111 Thunderbird/3.0.1 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <38989605a696e2cc4c0bf03c2f53e21a, 4B7FD4D2.7040802@andric.com> In-Reply-To: <38989605a696e2cc4c0bf03c2f53e21a,4B7FD4D2.7040802@andric.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV using ClamSMTP @ debank.tv X-OSF-Virus: CLEAN X-OSF-Outgoing: Innocent X-OSF-SUM: b4381a2bb7edf557d7aafc9bc9537f51 X-OSF-Info: Checked for spam and viruses Subject: Re: zfs on 4k sector disks X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Feb 2010 01:58:25 -0000 On 21/02/10 1:25 AM, Dimitry Andric wrote: > On 2010-02-20 10:47, mailinglists wrote: >> This "Advanced Format" basically means the sectors size on disk is >> changed to 4k in stead of 512k. > > Please note those disks still seem to expose 512B sectors to any OS; the > 4kiB sectors are only used internally. And there seems to be no jumper > to "fix" this behaviour... Unaligned writes can cause multiple > read-write-modify operations, which are most likely bad for performance. > > >> I'm wondering if freeBSD 8.0 using ZFS raidz and drives like these will >> work properly, I've seen a lot of articles and postings about partitions >> not being properly aligned with the physical disk layout and suffering >> big performance hits. My question is; should I worry about these issues >> or does ZFS already align the data on a 4k border? > > You can at least try to make sure your partitions are aligned to 4kiB. > This should be easy enough to do with e.g. gpart(8). I see no specific > alignment options in the zpool(1M) or zfs(1M) manpages, but there may > be some trick that I am not aware of. :) > > With UFS, you might want to experiment with block and fragment sizes, to > see what is most optimal. For example, a block size of 32kiB and > fragment size of 4kiB, on a partition aligned to 4kiB. Dimitry, Thank you for your answer, I won't be bitten by this with the UFS partitions as they reside on a smaller disk with 512b sectors (although I did notice sysinstall still starts at sector 63 when creating ufs partitions which wouldn't work well with the 4k drives). I should probably have been more clear in explaining what I want to achieve setup: FreeBSD 8.0-Stable (RELENG_8) csupped 2 and updated 19 feb 2010. Core2 quad, 4GB mem OS Drive: 500 GB SATA using UFS (defaults from sysinstall) DATA: 3 * 1.5TB ZFS I'd like to setup the ZFS as a raidz using the whole disk, for this to work properly though I guess ZFS should align the start of the FS on a 4k border, I could as a fallback create the ZFS on partitions which start at the right spot but I'd rather stick with the "default" of using the whole drive. Does anybody know how ZFS aligns the FS on disk? Thanks, Rob Evers ***deze e-mail is gescand door Onlinespamfilter.nl*** From owner-freebsd-fs@FreeBSD.ORG Sun Feb 21 15:33:57 2010 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 63A93106566B; Sun, 21 Feb 2010 15:33:57 +0000 (UTC) (envelope-from gavin@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 3BEDF8FC13; Sun, 21 Feb 2010 15:33:57 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id o1LFXvNH095742; Sun, 21 Feb 2010 15:33:57 GMT (envelope-from gavin@freefall.freebsd.org) Received: (from gavin@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id o1LFXujR095738; Sun, 21 Feb 2010 15:33:56 GMT (envelope-from gavin) Date: Sun, 21 Feb 2010 15:33:56 GMT Message-Id: <201002211533.o1LFXujR095738@freefall.freebsd.org> To: giffunip@tutopia.com, gavin@FreeBSD.org, freebsd-fs@FreeBSD.org From: gavin@FreeBSD.org Cc: Subject: Re: kern/142558: [msdosfs] [patch] Minor updates to fs/msdosfs headers (from NetBSD) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Feb 2010 15:33:57 -0000 Old Synopsis: [msdosfs] patch] Minor updates to fs/msdosfs headers (from NetBSD) New Synopsis: [msdosfs] [patch] Minor updates to fs/msdosfs headers (from NetBSD) State-Changed-From-To: open->closed State-Changed-By: gavin State-Changed-When: Sun Feb 21 15:17:27 UTC 2010 State-Changed-Why: Patch applied in r203827 by kib, MFC in 2 weeks. Thanks for your submission! http://www.freebsd.org/cgi/query-pr.cgi?pr=142558 From owner-freebsd-fs@FreeBSD.ORG Mon Feb 22 11:06:57 2010 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 14D321065670 for ; Mon, 22 Feb 2010 11:06:57 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id D5AD68FC14 for ; Mon, 22 Feb 2010 11:06:56 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id o1MB6upo039683 for ; Mon, 22 Feb 2010 11:06:56 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id o1MB6uFa039681 for freebsd-fs@FreeBSD.org; Mon, 22 Feb 2010 11:06:56 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 22 Feb 2010 11:06:56 GMT Message-Id: <201002221106.o1MB6uFa039681@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-fs@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Feb 2010 11:06:57 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/143825 fs [nfs] [panic] Kernel panic on NFS client o kern/143345 fs [ext2fs] [patch] extfs minor header cleanups to better o kern/143343 fs [zfs] bug in sunlink flag on directories o kern/143212 fs [nfs] NFSv4 client strange work ... o kern/143184 fs [zfs] [lor] zfs/bufwait LOR o kern/142924 fs [ext2fs] [patch] Small cleanup for the inode struct in o kern/142914 fs [zfs] ZFS performance degradation over time o kern/142878 fs [zfs] [vfs] lock order reversal o kern/142872 fs [zfs] ZFS ZVOL Lockmgr Deadlock o kern/142597 fs [ext2fs] ext2fs does not work on filesystems with real o kern/142594 fs [zfs] Modification time reset to 1 Jan 1970 after fsyn o kern/142489 fs [zfs] [lor] allproc/zfs LOR o kern/142466 fs Update 7.2 -> 8.0 on Raid 1 ends with screwed raid [re o kern/142401 fs [ntfs] [patch] Minor updates to NTFS from NetBSD o kern/142306 fs [zfs] [panic] ZFS drive (from OSX Leopard) causes two o kern/142068 fs [ufs] BSD labels are got deleted spontaneously o kern/141950 fs [unionfs] [lor] ufs/unionfs/ufs Lock order reversal o kern/141897 fs [msdosfs] [panic] Kernel panic. msdofs: file name leng o kern/141718 fs [zfs] [panic] kernel panic when 'zfs rename' is used o o kern/141685 fs [zfs] zfs corruption on adaptec 5805 raid controller o kern/141463 fs [nfs] [panic] Frequent kernel panics after upgrade fro o kern/141305 fs [zfs] FreeBSD ZFS+sendfile severe performance issues ( o kern/141257 fs [gvinum] No puedo crear RAID5 por SW con gvinum o kern/141177 fs [zfs] fsync() on FIFO causes panic() on zfs o kern/141091 fs [patch] [nullfs] fix panics with DIAGNOSTIC enabled o kern/141086 fs [nfs] [panic] panic("nfs: bioread, not dir") on FreeBS o kern/141010 fs [zfs] "zfs scrub" fails when backed by files in UFS2 o kern/140888 fs [zfs] boot fail from zfs root while the pool resilveri o kern/140682 fs [netgraph] [panic] random panic in netgraph o kern/140661 fs [zfs] [patch] /boot/loader fails to work on a GPT/ZFS- o kern/140640 fs [zfs] snapshot crash o kern/140433 fs [zfs] [panic] panic while replaying ZIL after crash o kern/140134 fs [msdosfs] write and fsck destroy filesystem integrity o kern/140068 fs [smbfs] [patch] smbfs does not allow semicolon in file o kern/139725 fs [zfs] zdb(1) dumps core on i386 when examining zpool c o kern/139715 fs [zfs] vfs.numvnodes leak on busy zfs o bin/139651 fs [nfs] mount(8): read-only remount of NFS volume does n o kern/139597 fs [patch] [tmpfs] tmpfs initializes va_gen but doesn't u o kern/139564 fs [zfs] [panic] 8.0-RC1 - Fatal trap 12 at end of shutdo o kern/139407 fs [smbfs] [panic] smb mount causes system crash if remot o kern/139363 fs [nfs] diskless root nfs mount from non FreeBSD server o kern/138790 fs [zfs] ZFS ceases caching when mem demand is high o kern/138524 fs [msdosfs] disks and usb flashes/cards with Russian lab o kern/138421 fs [ufs] [patch] remove UFS label limitations o kern/138202 fs mount_msdosfs(1) see only 2Gb f kern/137037 fs [zfs] [hang] zfs rollback on root causes FreeBSD to fr o kern/136968 fs [ufs] [lor] ufs/bufwait/ufs (open) o kern/136945 fs [ufs] [lor] filedesc structure/ufs (poll) o kern/136944 fs [ffs] [lor] bufwait/snaplk (fsync) o kern/136873 fs [ntfs] Missing directories/files on NTFS volume o kern/136865 fs [nfs] [patch] NFS exports atomic and on-the-fly atomic o kern/136470 fs [nfs] Cannot mount / in read-only, over NFS o kern/135594 fs [zfs] Single dataset unresponsive with Samba o kern/135546 fs [zfs] zfs.ko module doesn't ignore zpool.cache filenam o kern/135469 fs [ufs] [panic] kernel crash on md operation in ufs_dirb o kern/135050 fs [zfs] ZFS clears/hides disk errors on reboot o kern/134491 fs [zfs] Hot spares are rather cold... o kern/133676 fs [smbfs] [panic] umount -f'ing a vnode-based memory dis o kern/133614 fs [panic] panic: ffs_truncate: read-only filesystem o kern/133174 fs [msdosfs] [patch] msdosfs must support utf-encoded int f kern/133150 fs [zfs] Page fault with ZFS on 7.1-RELEASE/amd64 while w o kern/132960 fs [ufs] [panic] panic:ffs_blkfree: freeing free frag o kern/132397 fs reboot causes filesystem corruption (failure to sync b o kern/132331 fs [ufs] [lor] LOR ufs and syncer o kern/132237 fs [msdosfs] msdosfs has problems to read MSDOS Floppy o kern/132145 fs [panic] File System Hard Crashes o kern/131995 fs [nfs] Failure to mount NFSv4 server o kern/131441 fs [unionfs] [nullfs] unionfs and/or nullfs not combineab o kern/131360 fs [nfs] poor scaling behavior of the NFS server under lo o kern/131342 fs [nfs] mounting/unmounting of disks causes NFS to fail o bin/131341 fs makefs: error "Bad file descriptor" on the mount poin o kern/130979 fs [smbfs] [panic] boot/kernel/smbfs.ko o kern/130920 fs [msdosfs] cp(1) takes 100% CPU time while copying file o kern/130229 fs [iconv] usermount fails on fs that need iconv o kern/130210 fs [nullfs] Error by check nullfs o kern/129760 fs [nfs] after 'umount -f' of a stale NFS share FreeBSD l o kern/129488 fs [smbfs] Kernel "bug" when using smbfs in smbfs_smb.c: o kern/129231 fs [ufs] [patch] New UFS mount (norandom) option - mostly o kern/129152 fs [panic] non-userfriendly panic when trying to mount(8) o kern/129059 fs [zfs] [patch] ZFS bootloader whitelistable via WITHOUT f kern/128829 fs smbd(8) causes periodic panic on 7-RELEASE o kern/127420 fs [gjournal] [panic] Journal overflow on gmirrored gjour o kern/127029 fs [panic] mount(8): trying to mount a write protected zi o kern/126287 fs [ufs] [panic] Kernel panics while mounting an UFS file s kern/125738 fs [zfs] [request] SHA256 acceleration in ZFS p kern/124621 fs [ext3] [patch] Cannot mount ext2fs partition f bin/124424 fs [zfs] zfs(8): zfs list -r shows strange snapshots' siz o kern/123939 fs [msdosfs] corrupts new files o kern/122380 fs [ffs] ffs_valloc:dup alloc (Soekris 4801/7.0/USB Flash o bin/122172 fs [fs]: amd(8) automount daemon dies on 6.3-STABLE i386, o bin/121898 fs [nullfs] pwd(1)/getcwd(2) fails with Permission denied o bin/121779 fs [ufs] snapinfo(8) (and related tools?) only work for t o bin/121366 fs [zfs] [patch] Automatic disk scrubbing from periodic(8 o bin/121072 fs [smbfs] mount_smbfs(8) cannot normally convert the cha f kern/120991 fs [panic] [fs] [snapshot] System crashes when manipulati o kern/120483 fs [ntfs] [patch] NTFS filesystem locking changes o kern/120482 fs [ntfs] [patch] Sync style changes between NetBSD and F f kern/119735 fs [zfs] geli + ZFS + samba starting on boot panics 7.0-B o kern/118912 fs [2tb] disk sizing/geometry problem with large array o kern/118713 fs [minidump] [patch] Display media size required for a k o bin/118249 fs mv(1): moving a directory changes its mtime o kern/118107 fs [ntfs] [panic] Kernel panic when accessing a file at N o bin/117315 fs [smbfs] mount_smbfs(8) and related options can't mount o kern/117314 fs [ntfs] Long-filename only NTFS fs'es cause kernel pani o kern/117158 fs [zfs] zpool scrub causes panic if geli vdevs detach on o bin/116980 fs [msdosfs] [patch] mount_msdosfs(8) resets some flags f o kern/116913 fs [ffs] [panic] ffs_blkfree: freeing free block p kern/116608 fs [msdosfs] [patch] msdosfs fails to check mount options o kern/116583 fs [ffs] [hang] System freezes for short time when using o kern/116170 fs [panic] Kernel panic when mounting /tmp o kern/115645 fs [snapshots] [panic] lockmgr: thread 0xc4c00d80, not ex o bin/115361 fs [zfs] mount(8) gets into a state where it won't set/un o kern/114955 fs [cd9660] [patch] [request] support for mask,dirmask,ui o kern/114847 fs [ntfs] [patch] [request] dirmask support for NTFS ala o kern/114676 fs [ufs] snapshot creation panics: snapacct_ufs2: bad blo o bin/114468 fs [patch] [request] add -d option to umount(8) to detach o kern/113852 fs [smbfs] smbfs does not properly implement DFS referral o bin/113838 fs [patch] [request] mount(8): add support for relative p o bin/113049 fs [patch] [request] make quot(8) use getopt(3) and show o kern/112658 fs [smbfs] [patch] smbfs and caching problems (resolves b o kern/111843 fs [msdosfs] Long Names of files are incorrectly created o kern/111782 fs [ufs] dump(8) fails horribly for large filesystems s bin/111146 fs [2tb] fsck(8) fails on 6T filesystem o kern/109024 fs [msdosfs] mount_msdosfs: msdosfs_iconv: Operation not o kern/109010 fs [msdosfs] can't mv directory within fat32 file system o bin/107829 fs [2TB] fdisk(8): invalid boundary checking in fdisk / w o kern/106030 fs [ufs] [panic] panic in ufs from geom when a dead disk o kern/104406 fs [ufs] Processes get stuck in "ufs" state under persist o kern/104133 fs [ext2fs] EXT2FS module corrupts EXT2/3 filesystems o kern/103035 fs [ntfs] Directories in NTFS mounted disc images appear o kern/101324 fs [smbfs] smbfs sometimes not case sensitive when it's s o kern/99290 fs [ntfs] mount_ntfs ignorant of cluster sizes o kern/97377 fs [ntfs] [patch] syntax cleanup for ntfs_ihash.c o kern/95222 fs [iso9660] File sections on ISO9660 level 3 CDs ignored o kern/94849 fs [ufs] rename on UFS filesystem is not atomic o kern/94769 fs [ufs] Multiple file deletions on multi-snapshotted fil o kern/94733 fs [smbfs] smbfs may cause double unlock o kern/93942 fs [vfs] [patch] panic: ufs_dirbad: bad dir (patch from D o kern/92272 fs [ffs] [hang] Filling a filesystem while creating a sna f kern/91568 fs [ufs] [panic] writing to UFS/softupdates DVD media in o kern/91134 fs [smbfs] [patch] Preserve access and modification time a kern/90815 fs [smbfs] [patch] SMBFS with character conversions somet o kern/88657 fs [smbfs] windows client hang when browsing a samba shar o kern/88266 fs [smbfs] smbfs does not implement UIO_NOCOPY and sendfi o kern/87859 fs [smbfs] System reboot while umount smbfs. o kern/86587 fs [msdosfs] rm -r /PATH fails with lots of small files o kern/85326 fs [smbfs] [panic] saving a file via samba to an overquot o kern/84589 fs [2TB] 5.4-STABLE unresponsive during background fsck 2 o kern/80088 fs [smbfs] Incorrect file time setting on NTFS mounted vi o kern/73484 fs [ntfs] Kernel panic when doing `ls` from the client si o bin/73019 fs [ufs] fsck_ufs(8) cannot alloc 607016868 bytes for ino o kern/71774 fs [ntfs] NTFS cannot "see" files on a WinXP filesystem o kern/68978 fs [panic] [ufs] crashes with failing hard disk, loose po o kern/65920 fs [nwfs] Mounted Netware filesystem behaves strange o kern/65901 fs [smbfs] [patch] smbfs fails fsx write/truncate-down/tr o kern/61503 fs [smbfs] mount_smbfs does not work as non-root o kern/55617 fs [smbfs] Accessing an nsmb-mounted drive via a smb expo o kern/51685 fs [hang] Unbounded inode allocation causes kernel to loc o kern/51583 fs [nullfs] [patch] allow to work with devices and socket o kern/36566 fs [smbfs] System reboot with dead smb mount and umount o kern/18874 fs [2TB] 32bit NFS servers export wrong negative values t 161 problems total. From owner-freebsd-fs@FreeBSD.ORG Mon Feb 22 19:36:16 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 36EAF10656A7; Mon, 22 Feb 2010 19:36:16 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 05A168FC1A; Mon, 22 Feb 2010 19:36:16 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 9280F46B2D; Mon, 22 Feb 2010 14:36:15 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 2F35E8A01F; Mon, 22 Feb 2010 14:36:03 -0500 (EST) From: John Baldwin To: Attilio Rao Date: Mon, 22 Feb 2010 12:02:29 -0500 User-Agent: KMail/1.12.1 (FreeBSD/7.2-CBSD-20100120; KDE/4.3.1; amd64; ; ) References: <179b97fb1001270941m2d8e9c8au20abc798c16b9c11@mail.gmail.com> <3bbf2fe11002201419v52b249ccg8d82c8ae747cf318@mail.gmail.com> In-Reply-To: <3bbf2fe11002201419v52b249ccg8d82c8ae747cf318@mail.gmail.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201002221202.29747.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Mon, 22 Feb 2010 14:36:03 -0500 (EST) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.3 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: freebsd-fs@freebsd.org, freebsd-emulation@freebsd.org, stable-list freebsd Subject: Re: ZFS and sh(1) panic: spin lock [lock addr] (smp rendezvous) held by [sh(1) proc tid] too long X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Feb 2010 19:36:16 -0000 On Saturday 20 February 2010 5:19:39 pm Attilio Rao wrote: > 2010/1/27 Brandon Gooch : > > The machine, a Dell Optiplex 755, has been locking up recently. The > > situation usually occurs while using VirtualBox (running a 64-bit > > Windows 7 instance) and doing anything else in another xterm (such as > > rebuilding a port). I've been unable to reliably reproduce it (I'm in > > an X session and the machine will not panic "properly"). > > > > However, while rebuilding Xorg today at ttyv0 and runnning > > VBoxHeadless on ttyv1, I managed to trigger what I believe is the > > lockup. > > > > I've attached a textdump in hopes that someone may be able to take a > > look and provide clues or instruction on debugging this. > > I think that jhb@ saw a similar problem while working on nVidia driver > or the like. > Not sure if he made any progress to debug this. That was due to a weird bug where I was accidentally zero'ing the local APIC due to a then-bug in sg_pager.c (it was forcing the VM system to clear the pages it mapped by accident). That should not be the explanation for this. -- John Baldwin From owner-freebsd-fs@FreeBSD.ORG Tue Feb 23 04:19:26 2010 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 48ED0106568B for ; Tue, 23 Feb 2010 04:19:26 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from mail2.fluidhosting.com (mx21.fluidhosting.com [204.14.89.4]) by mx1.freebsd.org (Postfix) with ESMTP id CE7658FC0A for ; Tue, 23 Feb 2010 04:19:25 +0000 (UTC) Received: (qmail 24694 invoked by uid 399); 23 Feb 2010 03:52:45 -0000 Received: from localhost (HELO foreign.dougb.net) (dougb@dougbarton.us@127.0.0.1) by localhost with ESMTPAM; 23 Feb 2010 03:52:45 -0000 X-Originating-IP: 127.0.0.1 X-Sender: dougb@dougbarton.us Message-ID: <4B83510B.1050805@FreeBSD.org> Date: Mon, 22 Feb 2010 19:52:43 -0800 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.1.7) Gecko/20100218 Thunderbird/3.0.1 MIME-Version: 1.0 To: freebsd-fs@FreeBSD.org References: <201002222138.o1MLcoGs086212@freefall.freebsd.org> In-Reply-To: <201002222138.o1MLcoGs086212@freefall.freebsd.org> X-Enigmail-Version: 1.0.1 OpenPGP: id=D5B2F0FB Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-rc@FreeBSD.org, Ruben van Staveren Subject: Re: conf/144213: [rc.d] [patch] Disappearing zvols on reboot X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Feb 2010 04:19:26 -0000 I don't see any problem with this change from an rc.d standpoint, but I don't know anything about zfs. Comments and/or suggestions welcome. Doug On 02/22/10 13:38, linimon@FreeBSD.org wrote: > Old Synopsis: Disappearing zvols on reboot > New Synopsis: [rc.d] [patch] Disappearing zvols on reboot > > Responsible-Changed-From-To: freebsd-bugs->freebsd-rc > Responsible-Changed-By: linimon > Responsible-Changed-When: Mon Feb 22 21:38:23 UTC 2010 > Responsible-Changed-Why: > Over to maintainer(s). > > http://www.freebsd.org/cgi/query-pr.cgi?pr=144213 > _______________________________________________ > freebsd-rc@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-rc > To unsubscribe, send any mail to "freebsd-rc-unsubscribe@freebsd.org" From owner-freebsd-fs@FreeBSD.ORG Tue Feb 23 13:01:26 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 87663106566B; Tue, 23 Feb 2010 13:01:26 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-gw0-f54.google.com (mail-gw0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id 13A838FC1B; Tue, 23 Feb 2010 13:01:25 +0000 (UTC) Received: by gwaa12 with SMTP id a12so1086184gwa.13 for ; Tue, 23 Feb 2010 05:01:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:date:from:to:cc :subject:in-reply-to:message-id:references:user-agent :x-openpgp-key-id:x-openpgp-key-fingerprint:mime-version :content-type; bh=8cY+NlyDxW6CMhcZd7wY8ec24nwFVz1Uw4CWwU7DCfk=; b=AEJVPCR4QJYregcVDTuuNro78B1eo/7f0/izOtpmjP+dSCd1bbdkRa477vpW/pEQQI rEAQwHwOchUmLDO8OAK1S/4xgIfVxHQUkRyQSP8Q8liOSCHrWZPvxDKJuG1lyXAPbae9 mxMUQrUiC+SWF34ylrQDmy5GFgwPXtr/pji9E= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:date:from:to:cc:subject:in-reply-to:message-id:references :user-agent:x-openpgp-key-id:x-openpgp-key-fingerprint:mime-version :content-type; b=wneJPB5iQ1Hl/7R6JLkYa2WtqFEWskHR+OlMEvnI04+DJO/7kneTCaXDrKM9qoNBIy pDxNLZxPA18COJ2VMd2sQZ8sLlTJuJagcL+8kESb2a8gxcC42QK8I/DGrOLnU4SM5LFO lsXT79E+OPOVV9B3+I2cKQ7JIC59lnf1zVhpM= Received: by 10.91.74.18 with SMTP id b18mr2179967agl.39.1266930081270; Tue, 23 Feb 2010 05:01:21 -0800 (PST) Received: from centel.dataix.local (ppp-21.197.dialinfree.com [209.172.21.197]) by mx.google.com with ESMTPS id 16sm4420841gxk.9.2010.02.23.05.01.16 (version=TLSv1/SSLv3 cipher=RC4-MD5); Tue, 23 Feb 2010 05:01:19 -0800 (PST) Sender: "J. Hellenthal" Date: Tue, 23 Feb 2010 08:00:47 -0500 From: jhell To: Doug Barton In-Reply-To: <4B83510B.1050805@FreeBSD.org> Message-ID: References: <201002222138.o1MLcoGs086212@freefall.freebsd.org> <4B83510B.1050805@FreeBSD.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) X-OpenPGP-Key-Id: 0x89D8547E X-OpenPGP-Key-Fingerprint: 85EF E26B 07BB 3777 76BE B12A 9057 8789 89D8 547E MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: FreeBSD Filesystems , freebsd-rc@freebsd.org, Ruben van Staveren Subject: Re: conf/144213: [rc.d] [patch] Disappearing zvols on reboot X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Feb 2010 13:01:26 -0000 On Mon, 22 Feb 2010 22:52, dougb@ wrote: > I don't see any problem with this change from an rc.d standpoint, but I > don't know anything about zfs. Comments and/or suggestions welcome. > > > Doug > > > On 02/22/10 13:38, linimon@FreeBSD.org wrote: >> Old Synopsis: Disappearing zvols on reboot >> New Synopsis: [rc.d] [patch] Disappearing zvols on reboot >> >> Responsible-Changed-From-To: freebsd-bugs->freebsd-rc >> Responsible-Changed-By: linimon >> Responsible-Changed-When: Mon Feb 22 21:38:23 UTC 2010 >> Responsible-Changed-Why: >> Over to maintainer(s). >> >> http://www.freebsd.org/cgi/query-pr.cgi?pr=144213 > This is a good catch. I believe it would probably be best suited to be 1). above mount -a or 2). at the very least above share -a. Other then that non-zfs-root systems should not see a negative difference. Regards, -- jhell From owner-freebsd-fs@FreeBSD.ORG Tue Feb 23 15:41:13 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2E48C1065672; Tue, 23 Feb 2010 15:41:13 +0000 (UTC) (envelope-from ruben@verweg.com) Received: from erg.verweg.com (erg.verweg.com [IPv6:2a02:898:96::5e8e:f508]) by mx1.freebsd.org (Postfix) with ESMTP id B5AA98FC08; Tue, 23 Feb 2010 15:41:12 +0000 (UTC) Received: from [192.168.1.27] (helium.xs4all.nl [83.163.52.241]) (authenticated bits=0) by erg.verweg.com (8.14.4/8.14.3) with ESMTP id o1NFe7OK030360 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Tue, 23 Feb 2010 15:40:14 GMT (envelope-from ruben@verweg.com) X-Authentication-Warning: erg.verweg.com: Host helium.xs4all.nl [83.163.52.241] claimed to be [192.168.1.27] Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Ruben van Staveren In-Reply-To: Date: Tue, 23 Feb 2010 16:40:07 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <4DA9D015-9AB7-4D10-89CA-DF087F0F5FC8@verweg.com> References: <201002222138.o1MLcoGs086212@freefall.freebsd.org> <4B83510B.1050805@FreeBSD.org> To: jhell X-Mailer: Apple Mail (2.1077) X-Spam-Status: No, score=3.3 required=5.0 tests=DATE_IN_FUTURE_12_24 autolearn=no version=3.2.5 X-Spam-Level: *** X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on erg.verweg.com X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.3 (erg.verweg.com [94.142.245.8]); Tue, 23 Feb 2010 15:40:16 +0000 (UTC) Cc: FreeBSD Filesystems , Doug Barton , freebsd-rc@freebsd.org Subject: Re: conf/144213: [rc.d] [patch] Disappearing zvols on reboot X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Feb 2010 15:41:13 -0000 On 23 Feb 2010, at 14:00, jhell wrote: >=20 > On Mon, 22 Feb 2010 22:52, dougb@ wrote: >>>=20 >>> http://www.freebsd.org/cgi/query-pr.cgi?pr=3D144213 >>=20 >=20 > This is a good catch. I believe it would probably be best suited to = be 1). above mount -a or 2). at the very least above share -a. Other = then that non-zfs-root systems should not see a negative difference. Best would be 2). mounting is only for filesystems, not volumes and when = iscsi target mode is supported in freebsd (and zfs) you definitely want = your zvols to be available prior to sharing them. >=20 > Regards, >=20 > --=20 >=20 > jhell Regards, Ruben From owner-freebsd-fs@FreeBSD.ORG Tue Feb 23 17:48:37 2010 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EC3981065670; Tue, 23 Feb 2010 17:48:37 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id C46648FC12; Tue, 23 Feb 2010 17:48:37 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id o1NHmbun065120; Tue, 23 Feb 2010 17:48:37 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id o1NHmb0H065116; Tue, 23 Feb 2010 17:48:37 GMT (envelope-from linimon) Date: Tue, 23 Feb 2010 17:48:37 GMT Message-Id: <201002231748.o1NHmb0H065116@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-fs@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/144234: [zfs] Cannot boot machine with recent gptzfsboot code [regression] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Feb 2010 17:48:38 -0000 Old Synopsis: Cannot boot machine with recent gptzfsboot code New Synopsis: [zfs] Cannot boot machine with recent gptzfsboot code [regression] Responsible-Changed-From-To: freebsd-bugs->freebsd-fs Responsible-Changed-By: linimon Responsible-Changed-When: Tue Feb 23 17:47:44 UTC 2010 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=144234 From owner-freebsd-fs@FreeBSD.ORG Tue Feb 23 19:16:00 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4B9C8106566B for ; Tue, 23 Feb 2010 19:16:00 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 093FA8FC13 for ; Tue, 23 Feb 2010 19:15:59 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1Nk0Eg-0007Pu-QC for freebsd-fs@freebsd.org; Tue, 23 Feb 2010 20:15:58 +0100 Received: from 93-138-8-241.adsl.net.t-com.hr ([93.138.8.241]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 23 Feb 2010 20:15:58 +0100 Received: from ivoras by 93-138-8-241.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 23 Feb 2010 20:15:58 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-fs@freebsd.org From: Ivan Voras Date: Tue, 23 Feb 2010 20:15:48 +0100 Lines: 4 Message-ID: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 93-138-8-241.adsl.net.t-com.hr User-Agent: Thunderbird 2.0.0.21 (X11/20090612) Subject: Some ZFS+NFS benchmarks (OpenSolaris) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Feb 2010 19:16:00 -0000 http://staff.science.uva.nl/~delaat/sne-2009-2010/p02/report.pdf It's curious how ZIL on SSD doesn't help them with NFS when they increase the load. From owner-freebsd-fs@FreeBSD.ORG Tue Feb 23 19:35:21 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0E5DD1065679; Tue, 23 Feb 2010 19:35:21 +0000 (UTC) (envelope-from ticso@cicely7.cicely.de) Received: from raven.bwct.de (raven.bwct.de [85.159.14.73]) by mx1.freebsd.org (Postfix) with ESMTP id 8C1DD8FC14; Tue, 23 Feb 2010 19:35:19 +0000 (UTC) Received: from mail.cicely.de ([10.1.1.37]) by raven.bwct.de (8.13.4/8.13.4) with ESMTP id o1NJZFMn066819 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 23 Feb 2010 20:35:16 +0100 (CET) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (cicely7.cicely.de [10.1.1.9]) by mail.cicely.de (8.14.3/8.14.3) with ESMTP id o1NJZDxh044979 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 23 Feb 2010 20:35:13 +0100 (CET) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (localhost [127.0.0.1]) by cicely7.cicely.de (8.14.2/8.14.2) with ESMTP id o1NJZDnq020762; Tue, 23 Feb 2010 20:35:13 +0100 (CET) (envelope-from ticso@cicely7.cicely.de) Received: (from ticso@localhost) by cicely7.cicely.de (8.14.2/8.14.2/Submit) id o1NJYx5J020755; Tue, 23 Feb 2010 20:34:59 +0100 (CET) (envelope-from ticso) Date: Tue, 23 Feb 2010 20:34:58 +0100 From: Bernd Walter To: Ivan Voras Message-ID: <20100223193458.GO13767@cicely7.cicely.de> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Operating-System: FreeBSD cicely7.cicely.de 7.0-STABLE i386 User-Agent: Mutt/1.5.11 X-Spam-Status: No, score=-4.4 required=5.0 tests=ALL_TRUSTED=-1.8, BAYES_00=-2.599 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on spamd.cicely.de Cc: freebsd-fs@freebsd.org Subject: Re: Some ZFS+NFS benchmarks (OpenSolaris) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: ticso@cicely.de List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Feb 2010 19:35:21 -0000 On Tue, Feb 23, 2010 at 08:15:48PM +0100, Ivan Voras wrote: > http://staff.science.uva.nl/~delaat/sne-2009-2010/p02/report.pdf > > It's curious how ZIL on SSD doesn't help them with NFS when they > increase the load. My assumption is because they already write linear on SSD and get a more or less fixed write rate, while parallel write rate with disks can increase because of reordering. I'm personally impressed by my own tests on how much our current USB stack can speed up random reads even with cheap USB flash sticks used as cache devices. While it is true that a SSD is much faster it is also true for me that the bigger capacity of same price USB sticks can be a better win. Especially working with svn and CVS trees get a lot of performance from high ARC2 hit rate. One bad thing about large capacity is that my system currently boots with empty cache devices. In some documents they talk that cache content is held over reboots. Is this just future work in ZFS, future work with FreeBSD, or is my system just a few months too old? -- B.Walter http://www.bwct.de Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm. From owner-freebsd-fs@FreeBSD.ORG Tue Feb 23 19:54:43 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 147101065679; Tue, 23 Feb 2010 19:54:43 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-iw0-f185.google.com (mail-iw0-f185.google.com [209.85.223.185]) by mx1.freebsd.org (Postfix) with ESMTP id B965A8FC1C; Tue, 23 Feb 2010 19:54:42 +0000 (UTC) Received: by iwn15 with SMTP id 15so3426372iwn.7 for ; Tue, 23 Feb 2010 11:54:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type; bh=l/UgOlL5BqW5EfdZnrmc22h/cSEVC0e8eWJt+zIg9lo=; b=CeDd4CcXRzJdaB49EazEO2GkBspnGIQ01jkCJWWGgpP2dFDebwXweBxMNNQNo9Cd6h 0ggQrgBe1LXW3HELh8GaVNY4QJpNdpUZDlh7fZvk3jlQu6Dbt4262Jypu7K0FOQlReE3 aGuvWlOl8vMuBGwNRGK9p/7e8UWEh2Ei1mTmk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=rz9XipVb6Udo8D4pHUKBTBr/KVQ2bdsArIqq5VBlIBV4rfZQ8tQTDIDJJQh/TC4hWZ N3hbDwwwOHtkKuM3DlW0T8Xi4eV4UZ4u7DbqP6JHrymEdiAiJNG3IppCRzGSPUkiIFSt 7c+ISf8rV6uajt61l4xCV/tEzFd8z1Av6QTzc= MIME-Version: 1.0 Sender: artemb@gmail.com Received: by 10.231.162.9 with SMTP id t9mr374099ibx.5.1266954872207; Tue, 23 Feb 2010 11:54:32 -0800 (PST) In-Reply-To: References: Date: Tue, 23 Feb 2010 11:54:32 -0800 X-Google-Sender-Auth: b5286967488d1d03 Message-ID: From: Artem Belevich To: Ivan Voras Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-fs@freebsd.org Subject: Re: Some ZFS+NFS benchmarks (OpenSolaris) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Feb 2010 19:54:43 -0000 OCZ Vertex Turbo SSD that they used may not be the best choice for Zil. It's not too bad, but according to benchmarks, it does lag behind Intel's X25-M in IOPS: http://www.tomshardware.com/reviews/ssd-x25-m-vertex,2399-13.html If we believe the benchmarks, OCZ's SSD could provide ~900 IOPS during fileserver benchmark. I assume that it would be in the ballpark of the iozone tests in the paper. The number seems to correlate fairly well with the paper's probes/sec graps on pages 12 and 13. My bet would be that if they used Intel's SSDs (X25-M or, even better, X25-E), they would probably see noticeable write performance bump. --Artem On Tue, Feb 23, 2010 at 11:15 AM, Ivan Voras wrote: > http://staff.science.uva.nl/~delaat/sne-2009-2010/p02/report.pdf > > It's curious how ZIL on SSD doesn't help them with NFS when they increase > the load. > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@FreeBSD.ORG Tue Feb 23 21:22:12 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 666B0106566B for ; Tue, 23 Feb 2010 21:22:12 +0000 (UTC) (envelope-from ivoras@gmail.com) Received: from mail-ew0-f227.google.com (mail-ew0-f227.google.com [209.85.219.227]) by mx1.freebsd.org (Postfix) with ESMTP id 471C48FC15 for ; Tue, 23 Feb 2010 21:22:11 +0000 (UTC) Received: by ewy27 with SMTP id 27so499914ewy.34 for ; Tue, 23 Feb 2010 13:22:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:from:date:x-google-sender-auth:message-id:subject:to:cc :content-type; bh=RyKFr7dvaUo1jSZL4vJjt40nWnggcnM1wC+i0WVqL3Y=; b=Cr47b7YE10H629vMzUKlFHRoO3ZbwcpP0zuIkxiQQcFkrUZsnlYAWBq5zzQx6HF+76 0ENLA84e3CwH3EONjYVr/9643CUqn8REkMd0ryLmd98RUw+7yWfu0pWXyO7bMCOtTleL CHdCFY33OueUSX9UVR+kohbfURFQvmPEMw5IM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type; b=bt+JqOBiYCsv2RmKaIM4C4VVDQc5qLfFrSsHSBX5G9mp3v5Qbxu301XtngrrDvcvy3 4KtZBFPeq0LIWNh2sveKzcL+EBqXY9zENrSV8kAqdytArL+DUkL5LsVoKbMmmMq9E3UZ Bf+KxFshcdEOCeSGUk2itceHLh+kHjmxhGiuU= MIME-Version: 1.0 Sender: ivoras@gmail.com Received: by 10.216.87.131 with SMTP id y3mr2439861wee.9.1266960122595; Tue, 23 Feb 2010 13:22:02 -0800 (PST) In-Reply-To: <20100223193458.GO13767@cicely7.cicely.de> References: <20100223193458.GO13767@cicely7.cicely.de> From: Ivan Voras Date: Tue, 23 Feb 2010 22:21:41 +0100 X-Google-Sender-Auth: 124dffe60205508d Message-ID: <9bbcef731002231321t352ce3e6y5fdafbf75b7fac54@mail.gmail.com> To: ticso@cicely.de Content-Type: text/plain; charset=UTF-8 Cc: freebsd-fs@freebsd.org Subject: Re: Some ZFS+NFS benchmarks (OpenSolaris) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Feb 2010 21:22:12 -0000 On 23 February 2010 20:34, Bernd Walter wrote: > On Tue, Feb 23, 2010 at 08:15:48PM +0100, Ivan Voras wrote: >> http://staff.science.uva.nl/~delaat/sne-2009-2010/p02/report.pdf >> >> It's curious how ZIL on SSD doesn't help them with NFS when they >> increase the load. > > My assumption is because they already write linear on SSD and get a more > or less fixed write rate, while parallel write rate with disks can > increase because of reordering. > > I'm personally impressed by my own tests on how much our current > USB stack can speed up random reads even with cheap USB flash sticks > used as cache devices. This is surprising to hear - I've just run some randomio (http://www.arctic.org/~dean/randomio/) tests on two little used USB flash sticks and got around 110 IOPS sequential writing (~~ 7 MB/s) and a bit less than 30 IOPS random writes of 4 KB buffers (amounting to ~~ 1 MB/s). (the test command was "randomio file 16 1 1 4096 10"). The ZIL should be written practically linearly - the sequential write rate is relevant here - and it is actually significantly slower than what mechanical HDDs can achieve. Is your result with ZIL perhaps simply because you moved it to another device and so freed the main device? From owner-freebsd-fs@FreeBSD.ORG Wed Feb 24 01:13:49 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A63AB106566C for ; Wed, 24 Feb 2010 01:13:49 +0000 (UTC) (envelope-from andrew@modulus.org) Received: from email.octopus.com.au (email.octopus.com.au [122.100.2.232]) by mx1.freebsd.org (Postfix) with ESMTP id 67B058FC13 for ; Wed, 24 Feb 2010 01:13:49 +0000 (UTC) Received: by email.octopus.com.au (Postfix, from userid 1002) id 158065CB90E; Wed, 24 Feb 2010 12:13:30 +1100 (EST) X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on email.octopus.com.au X-Spam-Level: * X-Spam-Status: No, score=1.9 required=10.0 tests=ALL_TRUSTED, FH_DATE_PAST_20XX autolearn=no version=3.2.3 Received: from [10.1.50.60] (142.19.96.58.static.exetel.com.au [58.96.19.142]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: admin@email.octopus.com.au) by email.octopus.com.au (Postfix) with ESMTP id 0DF155CB8E7; Wed, 24 Feb 2010 12:13:26 +1100 (EST) Message-ID: <4B847CB5.8030807@modulus.org> Date: Wed, 24 Feb 2010 12:11:17 +1100 From: Andrew Snow User-Agent: Thunderbird 2.0.0.14 (X11/20080523) MIME-Version: 1.0 To: Ivan Voras , freebsd-fs References: <20100223193458.GO13767@cicely7.cicely.de> <9bbcef731002231321t352ce3e6y5fdafbf75b7fac54@mail.gmail.com> In-Reply-To: <9bbcef731002231321t352ce3e6y5fdafbf75b7fac54@mail.gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: Re: Some ZFS+NFS benchmarks (OpenSolaris) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 01:13:49 -0000 The OCZ Vertex Turbo SSD used in this paper is a "consumer toy" grade SSD, completely unsuitable for use as a ZFS log device. Its small block synchronous write performance is about equivalent to a small RAID of traditional HDDs. - Andrew From owner-freebsd-fs@FreeBSD.ORG Wed Feb 24 01:39:28 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 94CDE106564A; Wed, 24 Feb 2010 01:39:28 +0000 (UTC) (envelope-from ticso@cicely7.cicely.de) Received: from raven.bwct.de (raven.bwct.de [85.159.14.73]) by mx1.freebsd.org (Postfix) with ESMTP id 005088FC1D; Wed, 24 Feb 2010 01:39:27 +0000 (UTC) Received: from mail.cicely.de ([10.1.1.37]) by raven.bwct.de (8.13.4/8.13.4) with ESMTP id o1NN9E9Q073876 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Wed, 24 Feb 2010 00:09:14 +0100 (CET) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (cicely7.cicely.de [10.1.1.9]) by mail.cicely.de (8.14.3/8.14.3) with ESMTP id o1NN97N2055422 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 24 Feb 2010 00:09:07 +0100 (CET) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (localhost [127.0.0.1]) by cicely7.cicely.de (8.14.2/8.14.2) with ESMTP id o1NN92pM021666; Wed, 24 Feb 2010 00:09:07 +0100 (CET) (envelope-from ticso@cicely7.cicely.de) Received: (from ticso@localhost) by cicely7.cicely.de (8.14.2/8.14.2/Submit) id o1NN8i14021664; Wed, 24 Feb 2010 00:08:44 +0100 (CET) (envelope-from ticso) Date: Wed, 24 Feb 2010 00:08:44 +0100 From: Bernd Walter To: Ivan Voras Message-ID: <20100223230844.GP13767@cicely7.cicely.de> References: <20100223193458.GO13767@cicely7.cicely.de> <9bbcef731002231321t352ce3e6y5fdafbf75b7fac54@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <9bbcef731002231321t352ce3e6y5fdafbf75b7fac54@mail.gmail.com> X-Operating-System: FreeBSD cicely7.cicely.de 7.0-STABLE i386 User-Agent: Mutt/1.5.11 X-Spam-Status: No, score=-4.4 required=5.0 tests=ALL_TRUSTED=-1.8, BAYES_00=-2.599 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on spamd.cicely.de Cc: freebsd-fs@freebsd.org, ticso@cicely.de Subject: Re: Some ZFS+NFS benchmarks (OpenSolaris) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: ticso@cicely.de List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 01:39:28 -0000 On Tue, Feb 23, 2010 at 10:21:41PM +0100, Ivan Voras wrote: > On 23 February 2010 20:34, Bernd Walter wrote: > > On Tue, Feb 23, 2010 at 08:15:48PM +0100, Ivan Voras wrote: > >> http://staff.science.uva.nl/~delaat/sne-2009-2010/p02/report.pdf > >> > >> It's curious how ZIL on SSD doesn't help them with NFS when they > >> increase the load. > > > > My assumption is because they already write linear on SSD and get a more > > or less fixed write rate, while parallel write rate with disks can > > increase because of reordering. > > > > I'm personally impressed by my own tests on how much our current > > USB stack can speed up random reads even with cheap USB flash sticks > > used as cache devices. > > This is surprising to hear - I've just run some randomio > (http://www.arctic.org/~dean/randomio/) tests on two little used USB > flash sticks and got around 110 IOPS sequential writing (~~ 7 MB/s) > and a bit less than 30 IOPS random writes of 4 KB buffers (amounting > to ~~ 1 MB/s). Yes - your values seem to fit with my assumed values, but I'm talking about L2ARC cache devices here. Those are written linear with small bandwith and read random. If they are too slow ZFS just drops data and cache fill slower. Random read access for USB sticks is great compared to HDD - although USB has a high latency overhead. > (the test command was "randomio file 16 1 1 4096 10"). > > The ZIL should be written practically linearly - the sequential write > rate is relevant here - and it is actually significantly slower than > what mechanical HDDs can achieve. A small test with ZIL on USB sticks was horrible because it couldn't take up with write speed. > Is your result with ZIL perhaps simply because you moved it to another > device and so freed the main device? I assume for ZIL you really need a device with fast write speed, but I have not much experience with ZIL devices. -- B.Walter http://www.bwct.de Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm. From owner-freebsd-fs@FreeBSD.ORG Wed Feb 24 02:09:03 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B4F761065670; Wed, 24 Feb 2010 02:09:03 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 347988FC08; Wed, 24 Feb 2010 02:09:02 +0000 (UTC) Received: by vws14 with SMTP id 14so2185136vws.13 for ; Tue, 23 Feb 2010 18:08:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:date:from:to:cc :subject:in-reply-to:message-id:references:user-agent :x-openpgp-key-id:x-openpgp-key-fingerprint:mime-version :content-type; bh=XDcmEJyXC2E54ojMo0roB2xG3Svb3EvCkoLdZNHvXhE=; b=t3Iwt879MqTU6qaPCVMqdu69jsrKJUNTYM9TQLmxA9WiBwvfIydGYcQvLJJBAN8CTI YOqZTJjXiKMKBWFu1NCcLjIB7BTWIz2VMBGO8ECdHmtf3o7tRtHitV7H52O3wE4FDGRG NnXIczZtJSzzg2ooFOVh2TUAoE5D2vATJRhnM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:date:from:to:cc:subject:in-reply-to:message-id:references :user-agent:x-openpgp-key-id:x-openpgp-key-fingerprint:mime-version :content-type; b=A4DgTl3QSx40TuGt96vnqAFJe+Y5a1z36gKub0PSv46faqJzs2NUn/mXU2U2cl3zAG z1MC1yn4Ao4SL/YWCabSoCnTwpNVuyf3qeeBiFgBP+LPKH+YEeu1q29V1e+OFp1m6Q56 mQYNSw3rkH42j8YF2BTvyhOFzpsIC/mken+yQ= Received: by 10.220.126.208 with SMTP id d16mr873453vcs.20.1266977336495; Tue, 23 Feb 2010 18:08:56 -0800 (PST) Received: from centel.dataix.local (ppp-23.147.dialinfree.com [209.172.23.147]) by mx.google.com with ESMTPS id 40sm47863836vws.2.2010.02.23.18.08.44 (version=TLSv1/SSLv3 cipher=RC4-MD5); Tue, 23 Feb 2010 18:08:55 -0800 (PST) Sender: "J. Hellenthal" Date: Tue, 23 Feb 2010 21:08:34 -0500 From: jhell To: Doug Barton In-Reply-To: Message-ID: References: <201002222138.o1MLcoGs086212@freefall.freebsd.org> <4B83510B.1050805@FreeBSD.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) X-OpenPGP-Key-Id: 0x89D8547E X-OpenPGP-Key-Fingerprint: 85EF E26B 07BB 3777 76BE B12A 9057 8789 89D8 547E MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: FreeBSD Filesystems , freebsd-rc@freebsd.org, Ruben van Staveren Subject: Re: conf/144213: [rc.d] [patch] Disappearing zvols on reboot X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 02:09:03 -0000 On Tue, 23 Feb 2010 08:00, jhell@ wrote: > > On Mon, 22 Feb 2010 22:52, dougb@ wrote: >> I don't see any problem with this change from an rc.d standpoint, but I >> don't know anything about zfs. Comments and/or suggestions welcome. >> >> >> Doug >> >> >> On 02/22/10 13:38, linimon@FreeBSD.org wrote: >>> Old Synopsis: Disappearing zvols on reboot >>> New Synopsis: [rc.d] [patch] Disappearing zvols on reboot >>> >>> Responsible-Changed-From-To: freebsd-bugs->freebsd-rc >>> Responsible-Changed-By: linimon >>> Responsible-Changed-When: Mon Feb 22 21:38:23 UTC 2010 >>> Responsible-Changed-Why: >>> Over to maintainer(s). >>> >>> http://www.freebsd.org/cgi/query-pr.cgi?pr=144213 >> > > This is a good catch. I believe it would probably be best suited to be 1). > above mount -a or 2). at the very least above share -a. Other then that > non-zfs-root systems should not see a negative difference. > > Regards, > > I take this back. It appears that stable/7 already has this. "above mount -a and share -a. -- jhell From owner-freebsd-fs@FreeBSD.ORG Wed Feb 24 06:48:42 2010 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 66D671065672; Wed, 24 Feb 2010 06:48:42 +0000 (UTC) (envelope-from remko@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 3D9C58FC17; Wed, 24 Feb 2010 06:48:42 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id o1O6mgmx034630; Wed, 24 Feb 2010 06:48:42 GMT (envelope-from remko@freefall.freebsd.org) Received: (from remko@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id o1O6mgQe034626; Wed, 24 Feb 2010 06:48:42 GMT (envelope-from remko) Date: Wed, 24 Feb 2010 06:48:42 GMT Message-Id: <201002240648.o1O6mgQe034626@freefall.freebsd.org> To: remko@FreeBSD.org, freebsd-i386@FreeBSD.org, freebsd-fs@FreeBSD.org From: remko@FreeBSD.org Cc: Subject: Re: bin/144214: zfsboot fails on gang block after upgrade to zfs v14 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 06:48:42 -0000 Synopsis: zfsboot fails on gang block after upgrade to zfs v14 Responsible-Changed-From-To: freebsd-i386->freebsd-fs Responsible-Changed-By: remko Responsible-Changed-When: Wed Feb 24 06:48:26 UTC 2010 Responsible-Changed-Why: Reassign to fs team http://www.freebsd.org/cgi/query-pr.cgi?pr=144214 From owner-freebsd-fs@FreeBSD.ORG Wed Feb 24 08:30:07 2010 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F2614106566C for ; Wed, 24 Feb 2010 08:30:07 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id C70FA8FC1B for ; Wed, 24 Feb 2010 08:30:07 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id o1O8U7IL046475 for ; Wed, 24 Feb 2010 08:30:07 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id o1O8U72s046472; Wed, 24 Feb 2010 08:30:07 GMT (envelope-from gnats) Date: Wed, 24 Feb 2010 08:30:07 GMT Message-Id: <201002240830.o1O8U72s046472@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org From: "Andrei V. Lavreniyuk" Cc: Subject: Re: bin/144214: zfsboot fails on gang block after upgrade to zfs v14 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: "Andrei V. Lavreniyuk" List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 08:30:08 -0000 The following reply was made to PR bin/144214; it has been noted by GNATS. From: "Andrei V. Lavreniyuk" To: bug-followup@FreeBSD.org, c.kworr@gmail.com Cc: Subject: Re: bin/144214: zfsboot fails on gang block after upgrade to zfs v14 Date: Wed, 24 Feb 2010 10:29:48 +0200 Hi! Fix: # gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ad0 -- Best regards, Andrei V. Lavreniyuk. From owner-freebsd-fs@FreeBSD.ORG Wed Feb 24 09:29:02 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E26AD106566B for ; Wed, 24 Feb 2010 09:29:02 +0000 (UTC) (envelope-from escholtz@argonsoft.de) Received: from coyote.quickmin.net (coyote.quickmin.net [217.14.112.24]) by mx1.freebsd.org (Postfix) with ESMTP id 4AE968FC14 for ; Wed, 24 Feb 2010 09:29:02 +0000 (UTC) Received: (qmail 34710 invoked from network); 24 Feb 2010 10:02:20 +0100 Received: by simscan 1.4.0 ppid: 34707, pid: 34708, t: 0.0165s scanners: clamav: 0.95.2/m:49/d:8609 Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=default; d=argonsoft.de; b=PwZC9yBOgElJvHFOkk0pdw6jhAyc61ajH4iwmSjBi3OBaMRjA/fyBiFLzmMacom6uOWHLoENxdUCqmZ1465/sG6nLpnFvNbTyDG03/o7WmSgqkWa/DIaT9x1P10jWwdnPwjzbBas1vV8pNVcr2+SlTKJyHXPB+AkHIKoWNQhTzo= ; Received: from dialbs-213-023-239-142.static.arcor-ip.net (HELO erik-scholtzs-macbook-pro-15.fritz.box) (00000150@213.23.239.142) by coyote.quickmin.net with SMTP; 24 Feb 2010 10:02:20 +0100 Message-ID: <4B84EB1C.1080700@argonsoft.de> Date: Wed, 24 Feb 2010 10:02:20 +0100 From: "Erik Scholtz, ArgonSoft GmbH" User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Problem with writing an UFS2 to iStore Storage System / can't find block in cyl 0 / bad magic number X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 09:29:03 -0000 Hi, I'm working on the problem I'm describing below since 12/2008. In Dec. 2008 I was trying to connect an iStor iSCSI-Storage-System to my FreeBSD (tested with all releases 6.4, 7.1, 7.2 and 8.0), which did not work. After having many tests and working together with Danny from the iSCSI Development Team, I'm now pretty sure the problem is in the UFS-tools and neither in the SCSI nor in the iSCSI-driver. I come to this conclusion, since I was able to get a filesystem working, after extending my Kernel with ext-3 and using ext-3 as filesystem - so SCSI and iSCSI seem to work properly. When trying to create the filesystem on an auto-build partitionmap ("use entire disk) I get: ********************************************************************************** |(create the filesystem) |newfs -O2 /dev/da0s1 |/dev/da0s1: 782023.5MB (1601584044 sectors) block size 16384, fragment |size 2048 | using 4256 cylinder groups of 183.77MB, 11761 blks, 23552 |inodes. |super-block backups (for fsck -b #) at: | 160, 376512, 752864, ... ... ... | 1601377920 |internal error: can't find block in cyl 0 ********************************************************************************** Trying to write the filesystem in dangerously dedicated mode results in: ********************************************************************************** |(create the filesystem) |newfs -O2 /dev/da0 | |..., ..., ... |419256288 |cg 0: bad magic number ********************************************************************************** After this, I tried to write the UFS to a partitionmap created by OS X. I was able to write a working UFS to this partitionmap (ignoring all warnings). The filesystem was stable and could be used; Running fsck to check the filesystem repairs the filesystem dead and destroys all data. The partitionmap I used was: ********************************************************************************** |geometry: 121403/255/63 [1950351360 sectors] | | 0 40 39 - 12 unused 0 | 40 409600 409639 da0p1 5 unknown 0 | 409640 1928708016 1929117655 da0p2 5 unknown 0 |1929117656 262184 1929379839 - 12 unused 0 ********************************************************************************** So does anybody out there have an idea how to get an UFS working on the storage? Greetings, Erik --- My blog: http://blog.elitecoderz.net From owner-freebsd-fs@FreeBSD.ORG Wed Feb 24 09:40:47 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 81273106568D for ; Wed, 24 Feb 2010 09:40:47 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id 27CB98FC0C for ; Wed, 24 Feb 2010 09:40:46 +0000 (UTC) Received: from outgoing.leidinger.net (pD9E2D1AD.dip.t-dialin.net [217.226.209.173]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id 25FEB84469B; Wed, 24 Feb 2010 10:24:37 +0100 (CET) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id E29051B19AB; Wed, 24 Feb 2010 10:24:33 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1267003474; bh=ccEb+qYxbZ8/QgDUnaYG9Mz9sWEMvAx5fqBRgrchWwI=; h=Message-ID:Date:From:To:Cc:Subject:References:In-Reply-To: MIME-Version:Content-Type:Content-Transfer-Encoding; b=DxlwHCryZg0yFsWCFT/s72ETwcqBJeU3/Y24SHQ8Ijn3DS70zEVFOHPPrtFMnbIjF 2KmIoBMIig0icGCxSLNcSdddpmOwekf/b7Cssnyb6bQhsX7aUYGZ2PLOlZ9fTjNYO/ HFY8+IkAOS7sUz/OPfJIpTc54aV34Uv+6chkMibFlDpsi2J/vQPqtajJ8y3iih/ER2 cPNawWtlYfcuer5OBk1NOdgQHpNBjmlg+cW8s6KzmvvOaqNYtyEJAqtIIQw5glKofo e/QHRxIScTR5c+F1gRlA0OBUSVTGu6lsgxT+n90sor3sU4nJ5woYZD7omlv4ailp2x kkTKj8bDioppw== Received: (from www@localhost) by webmail.leidinger.net (8.14.3/8.13.8/Submit) id o1O9OXx1014000; Wed, 24 Feb 2010 10:24:33 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Wed, 24 Feb 2010 10:24:33 +0100 Message-ID: <20100224102433.18641xjz8tdvmwdc@webmail.leidinger.net> Date: Wed, 24 Feb 2010 10:24:33 +0100 From: Alexander Leidinger To: ticso@cicely.de, Bernd Walter References: <20100223193458.GO13767@cicely7.cicely.de> <9bbcef731002231321t352ce3e6y5fdafbf75b7fac54@mail.gmail.com> <20100223230844.GP13767@cicely7.cicely.de> In-Reply-To: <20100223230844.GP13767@cicely7.cicely.de> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: 25FEB84469B.2CD4B X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-0.763, required 6, autolearn=disabled, ALL_TRUSTED -1.44, DKIM_SIGNED 0.00, DKIM_VERIFIED -0.00, J_CHICKENPOX_33 0.60, TW_ZF 0.08) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1267608278.62427@A6xXwX/8fCrwF6Ye+OgG6A X-EBL-Spam-Status: No Cc: freebsd-fs@freebsd.org, Ivan Voras , ticso@cicely.de Subject: Re: Some ZFS+NFS benchmarks (OpenSolaris) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 09:40:47 -0000 Quoting Bernd Walter (from Wed, 24 Feb 2010 00:08:44 +0100): > On Tue, Feb 23, 2010 at 10:21:41PM +0100, Ivan Voras wrote: >> On 23 February 2010 20:34, Bernd Walter wrote: >> > On Tue, Feb 23, 2010 at 08:15:48PM +0100, Ivan Voras wrote: >> >> http://staff.science.uva.nl/~delaat/sne-2009-2010/p02/report.pdf >> >> >> >> It's curious how ZIL on SSD doesn't help them with NFS when they >> >> increase the load. >> > >> > My assumption is because they already write linear on SSD and get a more >> > or less fixed write rate, while parallel write rate with disks can >> > increase because of reordering. See below for whta the SLOG is used for. I do not think the SLOG is used at all in this case (I haven't looked at the report, but NFS is not really used in situations for which the SLOG was designed for). >> > I'm personally impressed by my own tests on how much our current >> > USB stack can speed up random reads even with cheap USB flash sticks >> > used as cache devices. >> >> This is surprising to hear - I've just run some randomio >> (http://www.arctic.org/~dean/randomio/) tests on two little used USB >> flash sticks and got around 110 IOPS sequential writing (~~ 7 MB/s) >> and a bit less than 30 IOPS random writes of 4 KB buffers (amounting >> to ~~ 1 MB/s). > > Yes - your values seem to fit with my assumed values, but I'm talking > about L2ARC cache devices here. > Those are written linear with small bandwith and read random. > If they are too slow ZFS just drops data and cache fill slower. The cache is filled asynchron. And there is also no purge from ARC to L2ARC, so no increased latency if something needs to be thrown out of the ARC. The write speed does not really matter for L2ARC devices. > Random read access for USB sticks is great compared to HDD - although > USB has a high latency overhead. It depends, the corresponding numbers I present in http://www.leidinger.net/blog/2010/02/10/making-zfs-faster/ are mostly taken with gstat and depending on the workload I see only 0.4ms per read on the L2ARC. >> (the test command was "randomio file 16 1 1 4096 10"). >> >> The ZIL should be written practically linearly - the sequential write >> rate is relevant here - and it is actually significantly slower than >> what mechanical HDDs can achieve. > > A small test with ZIL on USB sticks was horrible because it couldn't > take up with write speed. The write latency for USB sticks is just too big (at least for the USB stick I use it is about 10 times slower than the write latency of my disks). Forget about it. FYI: what Sun is selling as their state of the art Storage appliances now comes with special SSD firmware. The SSDs for the L2ARC are optimized for read operations, while the SSDs for the ZIL are optimized for write performance. If you use the respective other one, the performance will not be as good as expected. >> Is your result with ZIL perhaps simply because you moved it to another >> device and so freed the main device? > > I assume for ZIL you really need a device with fast write speed, but > I have not much experience with ZIL devices. That is correct. And AFAIK the SLOG will only be used for direct I/O (O_FSYNC and the like), not for the normal write operations. So this should only speed up DBs with a good amount of changes (those DBs which take ACID sort of serious), and maybe busy mailservers... and similar workloads. Bye, Alexander. -- "Virtual" means never knowing where your next byte is coming from. http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-fs@FreeBSD.ORG Wed Feb 24 09:53:46 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 262E8106564A for ; Wed, 24 Feb 2010 09:53:46 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from mail-fx0-f223.google.com (mail-fx0-f223.google.com [209.85.220.223]) by mx1.freebsd.org (Postfix) with ESMTP id 9DEB48FC08 for ; Wed, 24 Feb 2010 09:53:45 +0000 (UTC) Received: by fxm23 with SMTP id 23so766373fxm.3 for ; Wed, 24 Feb 2010 01:53:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:from:content-type :content-transfer-encoding:subject:date:message-id:to:mime-version :x-mailer; bh=J04L7Qb6b2FOH8HCoVQ7LUN8EQQsWu/2zu0a5jwnt9s=; b=FzNW441N1x8N9GzCeqvZZ/p0/5brWZ83H+PXwdkLIzYlIrphaZIep8VJtXWat0V/Vu 56uU08c63k7Ex5orgb8Z5QxkQXCyYwdZUjf7Ulk1BCfyEU0i98qnJQANWW1wNWlK633O bO4iq8aXD0A6HJwS6KB66n5Pw8NSf86B+F4GI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:content-type:content-transfer-encoding:subject:date:message-id :to:mime-version:x-mailer; b=D1n6QZ990zMndEo82mawU8Xlbshpv+CdYyu/TGvOm4yGxIZeJmTZNx4ygLCKsGMogf RtFRF18A6OHHvTpXMdyGGRcstjWmbyEHd02qWemzA8Wqr+ceM0L+ZrR7+FTeMYbC0t6w OUfLk8KNRu04ITWkpS3DPKVs2JsYs8tEH6eX0= Received: by 10.103.87.39 with SMTP id p39mr4777388mul.95.1267005217469; Wed, 24 Feb 2010 01:53:37 -0800 (PST) Received: from ?10.32.23.105? ([195.34.111.178]) by mx.google.com with ESMTPS id 25sm7804223mul.16.2010.02.24.01.53.35 (version=TLSv1/SSLv3 cipher=RC4-MD5); Wed, 24 Feb 2010 01:53:36 -0800 (PST) From: Nikolay Denev Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Wed, 24 Feb 2010 11:53:32 +0200 Message-Id: <26CCAE99-B056-4754-9324-2945D1643C78@gmail.com> To: freebsd-fs@freebsd.org Mime-Version: 1.0 (Apple Message framework v1077) X-Mailer: Apple Mail (2.1077) Subject: should zfs care so much about device name changes X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 09:53:46 -0000 Hello, I wanted to test the new option ATA_CAM, but that would require a boot = from another media (USB Drive/CD) and a zpool export/import to update the vdev names. (actually it's shown as "path" in the = zpool.cache file) because otherwise the system would refuse to = open/mount the pool. But is that really necessary given the fact that all the devices are = here, and have matching GUIDS to those in the zpool.cache file? Shouldnt ZFS just import the pool?=20 In the current state what would one have to do to for example test = ATA_CAM on a remote machine, where export/import from a rescue media is not possible? Thanks, Niki= From owner-freebsd-fs@FreeBSD.ORG Wed Feb 24 10:02:26 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9C405106564A for ; Wed, 24 Feb 2010 10:02:26 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) by mx1.freebsd.org (Postfix) with ESMTP id 5939C8FC18 for ; Wed, 24 Feb 2010 10:02:26 +0000 (UTC) Received: from elsa.codelab.cz (localhost.codelab.cz [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 0634219E02A; Wed, 24 Feb 2010 11:02:25 +0100 (CET) Received: from [192.168.1.2] (r5bb235.net.upc.cz [86.49.61.235]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id B5BCB19E023; Wed, 24 Feb 2010 11:02:22 +0100 (CET) Message-ID: <4B84F92E.8040305@quip.cz> Date: Wed, 24 Feb 2010 11:02:22 +0100 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.1.8) Gecko/20100205 SeaMonkey/2.0.3 MIME-Version: 1.0 To: "Erik Scholtz, ArgonSoft GmbH" References: <4B84EB1C.1080700@argonsoft.de> In-Reply-To: <4B84EB1C.1080700@argonsoft.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: Problem with writing an UFS2 to iStore Storage System / can't find block in cyl 0 / bad magic number X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 10:02:26 -0000 Erik Scholtz, ArgonSoft GmbH wrote: > Hi, > > I'm working on the problem I'm describing below since 12/2008. In Dec. > 2008 I was trying to connect an iStor iSCSI-Storage-System to my FreeBSD > (tested with all releases 6.4, 7.1, 7.2 and 8.0), which did not work. > > After having many tests and working together with Danny from the iSCSI > Development Team, I'm now pretty sure the problem is in the UFS-tools > and neither in the SCSI nor in the iSCSI-driver. > > I come to this conclusion, since I was able to get a filesystem working, > after extending my Kernel with ext-3 and using ext-3 as filesystem - so > SCSI and iSCSI seem to work properly. Did you tried partitioning with gpart and then UFS2 + SoftUpdates on /dev/da0p1? I am using it with 7.2-STABLE from January and iscsi_initiator 2.2.4 on Dell MD3000i (2.7TB partition) gpart create -s GPT da0 gpart add -b 34 -s 205520830 -t freebsd-ufs -l MyLabel da0 newfs -U /dev/da0p1 Just substitute 205520830 with size of your storage. Miroslav Lachman From owner-freebsd-fs@FreeBSD.ORG Wed Feb 24 10:23:53 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4BACD1065674; Wed, 24 Feb 2010 10:23:53 +0000 (UTC) (envelope-from ticso@cicely7.cicely.de) Received: from raven.bwct.de (raven.bwct.de [85.159.14.73]) by mx1.freebsd.org (Postfix) with ESMTP id 7540B8FC19; Wed, 24 Feb 2010 10:23:52 +0000 (UTC) Received: from mail.cicely.de ([10.1.1.37]) by raven.bwct.de (8.13.4/8.13.4) with ESMTP id o1OANll9017628 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Wed, 24 Feb 2010 11:23:50 +0100 (CET) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (cicely7.cicely.de [10.1.1.9]) by mail.cicely.de (8.14.3/8.14.3) with ESMTP id o1OANeVK080424 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 24 Feb 2010 11:23:40 +0100 (CET) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (localhost [127.0.0.1]) by cicely7.cicely.de (8.14.2/8.14.2) with ESMTP id o1OANZMf025426; Wed, 24 Feb 2010 11:23:40 +0100 (CET) (envelope-from ticso@cicely7.cicely.de) Received: (from ticso@localhost) by cicely7.cicely.de (8.14.2/8.14.2/Submit) id o1OANHA7025425; Wed, 24 Feb 2010 11:23:17 +0100 (CET) (envelope-from ticso) Date: Wed, 24 Feb 2010 11:23:17 +0100 From: Bernd Walter To: Alexander Leidinger Message-ID: <20100224102317.GR13767@cicely7.cicely.de> References: <20100223193458.GO13767@cicely7.cicely.de> <9bbcef731002231321t352ce3e6y5fdafbf75b7fac54@mail.gmail.com> <20100223230844.GP13767@cicely7.cicely.de> <20100224102433.18641xjz8tdvmwdc@webmail.leidinger.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100224102433.18641xjz8tdvmwdc@webmail.leidinger.net> X-Operating-System: FreeBSD cicely7.cicely.de 7.0-STABLE i386 User-Agent: Mutt/1.5.11 X-Spam-Status: No, score=-4.4 required=5.0 tests=ALL_TRUSTED=-1.8, BAYES_00=-2.599 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on spamd.cicely.de Cc: freebsd-fs@freebsd.org, Bernd Walter , ticso@cicely.de, Ivan Voras Subject: Re: Some ZFS+NFS benchmarks (OpenSolaris) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: ticso@cicely.de List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 10:23:53 -0000 On Wed, Feb 24, 2010 at 10:24:33AM +0100, Alexander Leidinger wrote: > > Quoting Bernd Walter (from Wed, 24 Feb 2010 > 00:08:44 +0100): > > >Yes - your values seem to fit with my assumed values, but I'm talking > >about L2ARC cache devices here. > >Those are written linear with small bandwith and read random. > >If they are too slow ZFS just drops data and cache fill slower. > > The cache is filled asynchron. And there is also no purge from ARC to > L2ARC, so no increased latency if something needs to be thrown out of > the ARC. The write speed does not really matter for L2ARC devices. > > >Random read access for USB sticks is great compared to HDD - although > >USB has a high latency overhead. > > It depends, the corresponding numbers I present in > http://www.leidinger.net/blog/2010/02/10/making-zfs-faster/ > are mostly taken with gstat and depending on the workload I see only > 0.4ms per read on the L2ARC. This matches with my experience, which is already impressive fast for USB as such. But under load the latency only increses to a few ms, while HDD easily climb up to a few hundred ms. Even half rotational delay without seek time for a 7200 HDD is about 4ms - 10 times slower than USB stick. With USB sticks there is only a high latency expection from time to time when there was also write load, but this rarely happens for me, especially after adding multiple devices. What I want to say is that Performance wise SSD are the better choice, but depending on the workload the higher capacity you get with USB sticks can be more important. I'm currently running a mix of 1x 4G CF, 4x 4G USB, 1x 32G SATA SSD. Speed with SSD is best, read latency with CF isn't much different from USB, but price is higher. I've ordered 4x 16G cheap sticks, since I'm very pleased with the speed, but want a higher hit rate. The only drawback with USB sticks is that my BIOS sees them and is horrible slow to boot. -- B.Walter http://www.bwct.de Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm. From owner-freebsd-fs@FreeBSD.ORG Wed Feb 24 12:42:25 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BE6031065673 for ; Wed, 24 Feb 2010 12:42:25 +0000 (UTC) (envelope-from escholtz@argonsoft.de) Received: from coyote.quickmin.net (coyote.quickmin.net [217.14.112.24]) by mx1.freebsd.org (Postfix) with ESMTP id 2D5C78FC23 for ; Wed, 24 Feb 2010 12:42:24 +0000 (UTC) Received: (qmail 59466 invoked from network); 24 Feb 2010 13:42:23 +0100 Received: by simscan 1.4.0 ppid: 59462, pid: 59463, t: 0.0175s scanners: clamav: 0.95.2/m:49/d:8609 Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=default; d=argonsoft.de; b=W7gs/OpJP+/ztuaPiYDm035CaKUHxtbw2hIhmf50T/LaoZlquxAA59F7ocLNNP8P1smnLTCSYls1R7QIWNQmwAV6+CZVh59G/GBvlXEtz2KKuZvOrhOO3qdkQxE2iDFLiOSvvaQp1zGLHO8aP7QrSD9kPqyTKytFSvT8KG5eR/c= ; Received: from dialbs-213-023-239-142.static.arcor-ip.net (HELO erik-scholtzs-macbook-pro-15.fritz.box) (00000150@213.23.239.142) by coyote.quickmin.net with SMTP; 24 Feb 2010 13:42:23 +0100 Message-ID: <4B851EAF.40200@argonsoft.de> Date: Wed, 24 Feb 2010 13:42:23 +0100 From: "Erik Scholtz, ArgonSoft GmbH" User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: Miroslav Lachman <000.fbsd@quip.cz> References: <4B84EB1C.1080700@argonsoft.de> <4B84F92E.8040305@quip.cz> In-Reply-To: <4B84F92E.8040305@quip.cz> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: Problem with writing an UFS2 to iStore Storage System / can't find block in cyl 0 / bad magic number X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 12:42:25 -0000 Miroslav, thanks for your reply. No - I haven't tried this yet and so did this a minute ago, but got stucked on the second command: ********************************************************************************** |# gpart create -s GPT da0 |# fdisk /dev/da0 |******* Working on device /dev/da0 ******* |acpi0_check: ram0 attached |parameters extracted from in-core disklabel are: |acpi0_check: npx0 attached55 sectors/track=63 (16065 blks/cyl) | acpi0_check: acpi0 attached |Figures below won't work with BIOS for partitions not in cyl 1 |parameters to be used for BIOS calculations are: |cylinders=1305 heads=255 sectors/track=63 (16065 blks/cyl) | |Media sector size is 512 |Warning: BIOS sector numbering starts with sector 1 |Information from DOS bootblock is: |The data for partition 1 is: |sysid 238 (0xee),(EFI GPT) | start 1, size 20971519 (10239 Meg), flag 0 | beg: cyl 1023/ head 255/ sector 63; | end: cyl 1023/ head 255/ sector 63 |The data for partition 2 is: | |The data for partition 3 is: | |The data for partition 4 is: | | |# gpart add -b 1 -s 20971519 -t freebsd-ufs -l MyLabel da0 |gpart: start '1': Invalid argument | |# gpart add -b 34 -s 20971519 -t freebsd-ufs -l MyLabel da0 |gpart: size '20971519': Invalid argument | |# gpart add -b 34 -s 20971485 -t freebsd-ufs -l MyLabel da0 |gpart: size '20971485': Invalid argument ********************************************************************************** Since gpart set the begin (start) on 1 and not on 34, I changed the -b option too. After failing this, I set the begin to 34 as in your example, which did not work either. Then I reduced the size (since when the start is later, it reduces the size too), which also did not work. Any further ideas? Greetings, Erik --- My blog: http://blog.elitecoderz.net Miroslav Lachman wrote: > Erik Scholtz, ArgonSoft GmbH wrote: >> Hi, >> >> I'm working on the problem I'm describing below since 12/2008. In Dec. >> 2008 I was trying to connect an iStor iSCSI-Storage-System to my FreeBSD >> (tested with all releases 6.4, 7.1, 7.2 and 8.0), which did not work. >> >> After having many tests and working together with Danny from the iSCSI >> Development Team, I'm now pretty sure the problem is in the UFS-tools >> and neither in the SCSI nor in the iSCSI-driver. >> >> I come to this conclusion, since I was able to get a filesystem working, >> after extending my Kernel with ext-3 and using ext-3 as filesystem - so >> SCSI and iSCSI seem to work properly. > > Did you tried partitioning with gpart and then UFS2 + SoftUpdates on > /dev/da0p1? > I am using it with 7.2-STABLE from January and iscsi_initiator 2.2.4 on > Dell MD3000i (2.7TB partition) > > gpart create -s GPT da0 > gpart add -b 34 -s 205520830 -t freebsd-ufs -l MyLabel da0 > newfs -U /dev/da0p1 > > Just substitute 205520830 with size of your storage. > > Miroslav Lachman From owner-freebsd-fs@FreeBSD.ORG Wed Feb 24 13:33:04 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B448E1065672 for ; Wed, 24 Feb 2010 13:33:04 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) by mx1.freebsd.org (Postfix) with ESMTP id 71FF48FC1F for ; Wed, 24 Feb 2010 13:33:03 +0000 (UTC) Received: from elsa.codelab.cz (localhost.codelab.cz [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 4A11419E027; Wed, 24 Feb 2010 14:33:02 +0100 (CET) Received: from [192.168.1.2] (r5bb235.net.upc.cz [86.49.61.235]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id BB13519E02A; Wed, 24 Feb 2010 14:32:59 +0100 (CET) Message-ID: <4B852A8B.9070505@quip.cz> Date: Wed, 24 Feb 2010 14:32:59 +0100 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.1.8) Gecko/20100205 SeaMonkey/2.0.3 MIME-Version: 1.0 To: "Erik Scholtz, ArgonSoft GmbH" References: <4B84EB1C.1080700@argonsoft.de> <4B84F92E.8040305@quip.cz> <4B851EAF.40200@argonsoft.de> In-Reply-To: <4B851EAF.40200@argonsoft.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: Problem with writing an UFS2 to iStore Storage System / can't find block in cyl 0 / bad magic number X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 13:33:04 -0000 Erik Scholtz, ArgonSoft GmbH wrote: > Miroslav, > > thanks for your reply. No - I haven't tried this yet and so did this a > minute ago, but got stucked on the second command: > > ********************************************************************************** > > |# gpart create -s GPT da0 [...] > |# gpart add -b 1 -s 20971519 -t freebsd-ufs -l MyLabel da0 > |gpart: start '1': Invalid argument > | > |# gpart add -b 34 -s 20971519 -t freebsd-ufs -l MyLabel da0 > |gpart: size '20971519': Invalid argument > | > |# gpart add -b 34 -s 20971485 -t freebsd-ufs -l MyLabel da0 > |gpart: size '20971485': Invalid argument > ********************************************************************************** > > > Since gpart set the begin (start) on 1 and not on 34, I changed the -b > option too. After failing this, I set the begin to 34 as in your > example, which did not work either. Then I reduced the size (since when > the start is later, it reduces the size too), which also did not work. You can try 'gpart show da0' right after gpart create, it will show you available space. for example: # gpart show da2 => 34 209715133 da2 GPT (100G) 34 209715133 - free - (100G) The first line tells you about your hard drive. You can start partitioning from sector 34 and the last available sector is 209715133. The second line gives information that there is free space beginning at sector 34 and spreading to sector 209715133. It means that the drive is empty. I am not sure if -s must be given in 512 bytes sectors, I saw some HowTos with "-s 100G", but it didn't work in my setup with FreeBSD 7.2 (it may work on 8.0?) You can try both versions and you can try little smaller partition if it gives you an error. (I am not too experienced with gpart) Miroslav Lachman From owner-freebsd-fs@FreeBSD.ORG Wed Feb 24 13:59:08 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 56F81106566B for ; Wed, 24 Feb 2010 13:59:08 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from mail-bw0-f216.google.com (mail-bw0-f216.google.com [209.85.218.216]) by mx1.freebsd.org (Postfix) with ESMTP id D98268FC18 for ; Wed, 24 Feb 2010 13:59:07 +0000 (UTC) Received: by bwz8 with SMTP id 8so3695949bwz.3 for ; Wed, 24 Feb 2010 05:59:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:content-type:mime-version :subject:from:in-reply-to:date:content-transfer-encoding:message-id :references:to:x-mailer; bh=AoiRgbL4/SVWgPq5Nx8ZHHr6Nh4JZWkkGKbzQbnqQNc=; b=mqh7aXoSzBG8LGfVG6yH2/BalnRrOu5nXFbIwFWVkbomQJuptjBAphmEY42VZSn26f JAC3ElnZ9X+G5JjXZ+OwovUVH/Rla5cMRMYaFqF+g2aLtncWR/QNn9sciL7Zqi65b8XR GmhqERVOt1w/A+n3XJ+jVa+2l7g49JJC1bYhA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=content-type:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to:x-mailer; b=gKH8Ojy0YGtyzNjA4atIfjUx5Hh4p+/kMuq1TsHHVhkCG6B8z7CPNwhk+3ufrDrQok 4okv491ErlMd8zt4uylo1w8kFOLzzoGrRfj7RDC74PYaUCnw9w8FhdLVPb7MOFYVf1vK 2OUgml8pmWgz3FHd3sxlSd4v3YWsxE9Qh4k4w= Received: by 10.102.183.14 with SMTP id g14mr2875363muf.71.1267019941582; Wed, 24 Feb 2010 05:59:01 -0800 (PST) Received: from ?10.32.23.105? ([195.34.111.178]) by mx.google.com with ESMTPS id w5sm8852958mue.52.2010.02.24.05.58.59 (version=TLSv1/SSLv3 cipher=RC4-MD5); Wed, 24 Feb 2010 05:59:00 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Apple Message framework v1077) From: Nikolay Denev In-Reply-To: <26CCAE99-B056-4754-9324-2945D1643C78@gmail.com> Date: Wed, 24 Feb 2010 15:58:58 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <272C3933-72C9-41AF-AD3A-605A7C57FB61@gmail.com> References: <26CCAE99-B056-4754-9324-2945D1643C78@gmail.com> To: freebsd-fs@freebsd.org X-Mailer: Apple Mail (2.1077) Subject: Re: should zfs care so much about device name changes X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 13:59:08 -0000 On Feb 24, 2010, at 11:53 AM, Nikolay Denev wrote: > Hello, >=20 > I wanted to test the new option ATA_CAM, but that would require a boot = from another media (USB Drive/CD) and a zpool export/import > to update the vdev names. (actually it's shown as "path" in the = zpool.cache file) because otherwise the system would refuse to = open/mount > the pool. > But is that really necessary given the fact that all the devices are = here, and have matching GUIDS to those in the zpool.cache file? > Shouldnt ZFS just import the pool?=20 >=20 > In the current state what would one have to do to for example test = ATA_CAM on a remote machine, where export/import from a rescue > media is not possible? >=20 > Thanks, > Niki I have now looked at = sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c and it seems = that when a vdev isn't found by pathname it is being searched by guid. But it doesn't seem to be the case here. The = loader sees the pool and boots the kernel, but then the kernel does not = see the pool. Any ideas? Regards, Niki Denev From owner-freebsd-fs@FreeBSD.ORG Wed Feb 24 14:33:57 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 87B30106566C for ; Wed, 24 Feb 2010 14:33:57 +0000 (UTC) (envelope-from tevans.uk@googlemail.com) Received: from mail-ew0-f228.google.com (mail-ew0-f228.google.com [209.85.219.228]) by mx1.freebsd.org (Postfix) with ESMTP id 1E0548FC15 for ; Wed, 24 Feb 2010 14:33:56 +0000 (UTC) Received: by ewy28 with SMTP id 28so388070ewy.13 for ; Wed, 24 Feb 2010 06:33:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=ZUYKfw9HoAunSohIogqfg8sdvY6i5mYOnfwgDnNpyEQ=; b=PN6MLnab0fhKNBCj0CYVCC+JDmRunn87uU1GGviUKrX17K1Hqv6Y+062g1csnH1aCM +B8JqkHbBI+GEu13enMW2YqVqsB2FQcg8laO6Yq50oChritfEX3QTnHRiInKXJhQXYtW pPoi4WfU83Iyhvkt/LMDt+qGXSg2OgvW/a5dQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=OYVyeY++6hEtDUkah2DBlr0ocQDuKjF4I7lUE54cEn8wg+s/PM2EKJZIq+DjwQzMSp uizeKB3nPqhnu0NX4tZRZXxYmNcRPSTsGPH02fOpVQ8GlT/lGo0U1zDY5sNwgESeNF8F 63t4fJv0IIgpdC38MHmxqy3M9WiAKQcCszTug= MIME-Version: 1.0 Received: by 10.213.70.65 with SMTP id c1mr2987337ebj.13.1267022032297; Wed, 24 Feb 2010 06:33:52 -0800 (PST) In-Reply-To: <4B852A8B.9070505@quip.cz> References: <4B84EB1C.1080700@argonsoft.de> <4B84F92E.8040305@quip.cz> <4B851EAF.40200@argonsoft.de> <4B852A8B.9070505@quip.cz> Date: Wed, 24 Feb 2010 14:33:52 +0000 Message-ID: <2e027be01002240633p4ba7c9adm8dd115ed10dc0f58@mail.gmail.com> From: Tom Evans To: Miroslav Lachman <000.fbsd@quip.cz> Content-Type: text/plain; charset=UTF-8 Cc: freebsd-fs@freebsd.org Subject: Re: Problem with writing an UFS2 to iStore Storage System / can't find block in cyl 0 / bad magic number X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 14:33:57 -0000 On Wed, Feb 24, 2010 at 1:32 PM, Miroslav Lachman <000.fbsd@quip.cz> wrote: > ... > I am not sure if -s must be given in 512 bytes sectors, I saw some HowTos > with "-s 100G", but it didn't work in my setup with FreeBSD 7.2 (it may work > on 8.0?) You can try both versions and you can try little smaller partition > if it gives you an error. > (I am not too experienced with gpart) > > Miroslav Lachman I believe the size modifiers were added in 8.0 (they certainly work in 8.0). gpart in 8.0 can also calculate the start sector correctly, without needing to specify it, and if you omit a size, uses the maximum available. Makes things a lot easier! Eg, these are equivalent: gpart add -b 34 -s 20971519 -t freebsd-ufs -l MyLabel da0 gpart add -t freebsd-ufs -l MyLabel da0 (obviously, only if da0 has 20971519 sectors available and starts at sector 34). Cheers Tom From owner-freebsd-fs@FreeBSD.ORG Wed Feb 24 15:07:22 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 954ED106566C for ; Wed, 24 Feb 2010 15:07:22 +0000 (UTC) (envelope-from escholtz@argonsoft.de) Received: from coyote.quickmin.net (coyote.quickmin.net [217.14.112.24]) by mx1.freebsd.org (Postfix) with ESMTP id 170108FC18 for ; Wed, 24 Feb 2010 15:07:21 +0000 (UTC) Received: (qmail 47108 invoked from network); 24 Feb 2010 16:07:20 +0100 Received: by simscan 1.4.0 ppid: 47102, pid: 47106, t: 0.0167s scanners: clamav: 0.95.2/m:49/d:8609 Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=default; d=argonsoft.de; b=IDqRxhhb71SbDzONUx+TQHuvWIt/sEcKLyBsG+7yPYx56tt+JmAw8LdgXZZok68P4we1MTNKWTYxngDe9dRxnNi+2DNg/62tYNL6As68iPHWtOr8dirWqN9kjGVPk4bPTVZeJ/99K3pShVt1PfnhMRGGE3FLathAKZU71X+Frx8= ; Received: from dialbs-213-023-239-142.static.arcor-ip.net (HELO erik-scholtzs-macbook-pro-15.fritz.box) (00000150@213.23.239.142) by coyote.quickmin.net with SMTP; 24 Feb 2010 16:07:20 +0100 Message-ID: <4B8540A8.7010007@argonsoft.de> Date: Wed, 24 Feb 2010 16:07:20 +0100 From: "Erik Scholtz, ArgonSoft GmbH" User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: Miroslav Lachman <000.fbsd@quip.cz> References: <4B84EB1C.1080700@argonsoft.de> <4B84F92E.8040305@quip.cz> <4B851EAF.40200@argonsoft.de> <4B852A8B.9070505@quip.cz> In-Reply-To: <4B852A8B.9070505@quip.cz> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: Problem with writing an UFS2 to iStore Storage System / can't find block in cyl 0 / bad magic number X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 15:07:22 -0000 Miroslav, works perfectly! Many thanks. One last question. After creating the filesystem successfully, I tried to run a fsck on the new partition. unfortunatly, this does not seem to be possible: ********************************************************************************** |# fsck /dev/da0p1 |fsck: Could not determine filesystem type ********************************************************************************** Do you have a hint for me, how to repair the filesystem if it gets broken? fsck_ufs and fsck_ffs repair the filesystem to dead. Cheers, Erik --- My blog: http://blog.elitecoderz.net Miroslav Lachman wrote: > Erik Scholtz, ArgonSoft GmbH wrote: >> Miroslav, >> >> thanks for your reply. No - I haven't tried this yet and so did this a >> minute ago, but got stucked on the second command: >> >> ********************************************************************************** >> >> >> |# gpart create -s GPT da0 > > [...] > >> |# gpart add -b 1 -s 20971519 -t freebsd-ufs -l MyLabel da0 >> |gpart: start '1': Invalid argument >> | >> |# gpart add -b 34 -s 20971519 -t freebsd-ufs -l MyLabel da0 >> |gpart: size '20971519': Invalid argument >> | >> |# gpart add -b 34 -s 20971485 -t freebsd-ufs -l MyLabel da0 >> |gpart: size '20971485': Invalid argument >> ********************************************************************************** >> >> >> >> Since gpart set the begin (start) on 1 and not on 34, I changed the -b >> option too. After failing this, I set the begin to 34 as in your >> example, which did not work either. Then I reduced the size (since when >> the start is later, it reduces the size too), which also did not work. > > You can try 'gpart show da0' right after gpart create, it will show you > available space. > > for example: > # gpart show da2 > => 34 209715133 da2 GPT (100G) > 34 209715133 - free - (100G) > > The first line tells you about your hard drive. You can start > partitioning from sector 34 and the last available sector is 209715133. > The second line gives information that there is free space beginning at > sector 34 and spreading to sector 209715133. > It means that the drive is empty. > > I am not sure if -s must be given in 512 bytes sectors, I saw some > HowTos with "-s 100G", but it didn't work in my setup with FreeBSD 7.2 > (it may work on 8.0?) You can try both versions and you can try little > smaller partition if it gives you an error. > (I am not too experienced with gpart) > > Miroslav Lachman From owner-freebsd-fs@FreeBSD.ORG Wed Feb 24 15:56:55 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D018F106564A for ; Wed, 24 Feb 2010 15:56:55 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) by mx1.freebsd.org (Postfix) with ESMTP id 8DCD98FC0A for ; Wed, 24 Feb 2010 15:56:55 +0000 (UTC) Received: from elsa.codelab.cz (localhost.codelab.cz [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 2E7F619E02D; Wed, 24 Feb 2010 16:56:54 +0100 (CET) Received: from [192.168.1.2] (r5bb235.net.upc.cz [86.49.61.235]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id EDD8319E02A; Wed, 24 Feb 2010 16:56:51 +0100 (CET) Message-ID: <4B854C43.7000508@quip.cz> Date: Wed, 24 Feb 2010 16:56:51 +0100 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.1.8) Gecko/20100205 SeaMonkey/2.0.3 MIME-Version: 1.0 To: "Erik Scholtz, ArgonSoft GmbH" References: <4B84EB1C.1080700@argonsoft.de> <4B84F92E.8040305@quip.cz> <4B851EAF.40200@argonsoft.de> <4B852A8B.9070505@quip.cz> <4B8540A8.7010007@argonsoft.de> In-Reply-To: <4B8540A8.7010007@argonsoft.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: Problem with writing an UFS2 to iStore Storage System / can't find block in cyl 0 / bad magic number X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 15:56:55 -0000 Erik Scholtz, ArgonSoft GmbH wrote: > Miroslav, > > works perfectly! Many thanks. One last question. After creating the > filesystem successfully, I tried to run a fsck on the new partition. > unfortunatly, this does not seem to be possible: > > ********************************************************************************** > > |# fsck /dev/da0p1 > |fsck: Could not determine filesystem type > ********************************************************************************** > > > Do you have a hint for me, how to repair the filesystem if it gets > broken? fsck_ufs and fsck_ffs repair the filesystem to dead. fsck determines FS type from /etc/fstab so until you put info in to fstab, you must set the type manually: fsck -t ufs /dev/da0p1 Miroslav Lachman From owner-freebsd-fs@FreeBSD.ORG Thu Feb 25 02:20:08 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 13081106566C for ; Thu, 25 Feb 2010 02:20:08 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-gx0-f211.google.com (mail-gx0-f211.google.com [209.85.217.211]) by mx1.freebsd.org (Postfix) with ESMTP id B3BC68FC0C for ; Thu, 25 Feb 2010 02:20:07 +0000 (UTC) Received: by gxk3 with SMTP id 3so1563548gxk.13 for ; Wed, 24 Feb 2010 18:20:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:date:from:to:subject :message-id:user-agent:x-openpgp-key-id:x-openpgp-key-fingerprint :mime-version:content-type:content-transfer-encoding; bh=/V5OBHXIB99rpnVCn0bd2VRTiONF8Rq1TrPlr4SzCBs=; b=mqATwk/hVJUU0992/4bLoyG3Af8L7Eqdafgwz9NI+ekuqsW3prF6i6b6pmnnDpKO9Y UCYXaU+lApCfXBvxXXcM4DWH3gGZoWvqzSZQSUJnLdBJiGiLXoN5XEhB9bAsxrkxTlXk VtT7v7be7RD0cpUKcoeTb+3AVJ9U4FAjOW3Gk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:date:from:to:subject:message-id:user-agent:x-openpgp-key-id :x-openpgp-key-fingerprint:mime-version:content-type :content-transfer-encoding; b=r9pU1HPtn8lniwHreIyeglqrvbTOV0IUl6VPjjOY2J9VRH03M42zQuXuHAgTwf2dLr lNL0rJSmWrFacSKiidm95co2GDlu4ASNM46r2AvWN4r692Vn76IlEVdBGjph8svAqziZ eo6ASYM0gfJ2fF3Om9D981Z6M3lTJP+7g2kgA= Received: by 10.101.3.3 with SMTP id f3mr644525ani.150.1267064401345; Wed, 24 Feb 2010 18:20:01 -0800 (PST) Received: from centel.dataix.local (ppp-22.36.dialinfree.com [209.172.22.36]) by mx.google.com with ESMTPS id 15sm1452277gxk.10.2010.02.24.18.19.57 (version=TLSv1/SSLv3 cipher=RC4-MD5); Wed, 24 Feb 2010 18:20:00 -0800 (PST) Sender: "J. Hellenthal" Date: Wed, 24 Feb 2010 21:19:44 -0500 From: jhell To: FreeBSD Filesystems Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) X-OpenPGP-Key-Id: 0x89D8547E X-OpenPGP-Key-Fingerprint: 85EF E26B 07BB 3777 76BE B12A 9057 8789 89D8 547E MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=ISO-8859-1 Content-Transfer-Encoding: 8BIT Subject: stable/7 zdb(1M) Assertion failed. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 02:20:08 -0000 I keep getting the following when trying to use any options with zdb(1M) One for example, # zdb -dd pool/local Assertion failed: (´2), function rwlp->rw_count == 0, file /usr/src/cddl/lib/libzpool/../../../cddl/contrib/opensolaris/lib/libzpool/common/kernel.c, line 195. Anyone know what might be causing this ?. On the other hand if I do not use any options and just provide the pool it works happily, but with that alone it does not give me the information that I was looking for. If more information is needed let me know. Thanks, -- jhell From owner-freebsd-fs@FreeBSD.ORG Thu Feb 25 10:09:32 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1B76B106566B for ; Thu, 25 Feb 2010 10:09:32 +0000 (UTC) (envelope-from h.schmalzbauer@omnilan.de) Received: from host.omnilan.net (host.omnilan.net [62.245.232.135]) by mx1.freebsd.org (Postfix) with ESMTP id 164D98FC12 for ; Thu, 25 Feb 2010 10:09:30 +0000 (UTC) Received: from titan.flintsbach.schmalzbauer.de (titan.flintsbach.schmalzbauer.de [172.21.1.150]) (authenticated bits=0) by host.omnilan.net (8.13.8/8.13.8) with ESMTP id o1P9xHoM086706 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 25 Feb 2010 10:59:17 +0100 (CET) (envelope-from h.schmalzbauer@omnilan.de) Message-ID: <4B8649F4.4070303@omnilan.de> Date: Thu, 25 Feb 2010 10:59:16 +0100 From: Harald Schmalzbauer Organization: OmniLAN User-Agent: Thunderbird 2.0.0.23 (X11/20090906) MIME-Version: 1.0 To: freebsd-fs@freebsd.org X-Enigmail-Version: 0.95.6 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig607900AB5AA65E5D13F83BB1" Subject: Re: ZFS write stalls (starving reads) and tuning zfs_write_limit_override X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 10:09:32 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig607900AB5AA65E5D13F83BB1 Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: quoted-printable Hello, I found your very interesting patch:=20 http://lists.freebsd.org/pipermail/freebsd-fs/2010-February/007818.html Unfortunately it seems to be -current since on -stable my dsl_pool.c=20 looks entire different. My C skills don't allow me to adopt. Do you have any hints for me? Thanks, -Harry --------------enig607900AB5AA65E5D13F83BB1 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.13 (FreeBSD) iEYEARECAAYFAkuGSfUACgkQLDqVQ9VXb8jNWACglABL/NCWvdFtA8+0ax9TU3Eg 8voAn1EpW5zq7s72HKEXqFySTe7lZpLY =Uf4Q -----END PGP SIGNATURE----- --------------enig607900AB5AA65E5D13F83BB1-- From owner-freebsd-fs@FreeBSD.ORG Thu Feb 25 10:32:54 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3FFA8106564A for ; Thu, 25 Feb 2010 10:32:54 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from mail-fx0-f223.google.com (mail-fx0-f223.google.com [209.85.220.223]) by mx1.freebsd.org (Postfix) with ESMTP id C128C8FC1A for ; Thu, 25 Feb 2010 10:32:53 +0000 (UTC) Received: by fxm23 with SMTP id 23so610567fxm.3 for ; Thu, 25 Feb 2010 02:32:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:subject:mime-version :content-type:from:in-reply-to:date:cc:message-id:references:to :x-mailer; bh=PGJGpnadWLdPYAudmBKcF9gNpEPvxBKfFqNeSy2aelM=; b=UxiBS4grQDE1QVBu1+1A6ARl34IMUn4r/wpglwax80Csp2j1CTEV3I8evmFD6/Iz1c iZKmaNVjgKuX5zWk8kM0q5EiIOEAMWP1qHq2cBFrlMExbzS20JeaMsi2eNwXqKPrhruu fLV1t+KvSnq2siFf75pU4HT3a11kng6g6LG+k= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=subject:mime-version:content-type:from:in-reply-to:date:cc :message-id:references:to:x-mailer; b=DLp0bMqy3xE8lmqTsRJP0IWpCAEAaBwf91AQKLeDkhIvWMPtnfNtkIvaX21PMkmu8q NR+hm4d8Gya7jzE0yRWTRQ8DF2pXpw+SvfHbB5epj+bvhaAoJdxCeQcnD+v11fIUNr9W 1FnwMXOs2vUfWdEbGvSn4WoRgd2+/nYPedfS8= Received: by 10.103.86.17 with SMTP id o17mr731335mul.47.1267093967592; Thu, 25 Feb 2010 02:32:47 -0800 (PST) Received: from ?10.32.23.105? ([195.34.111.178]) by mx.google.com with ESMTPS id 7sm13598705mup.3.2010.02.25.02.32.45 (version=TLSv1/SSLv3 cipher=RC4-MD5); Thu, 25 Feb 2010 02:32:46 -0800 (PST) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: multipart/mixed; boundary=Apple-Mail-6-1023955540 From: Nikolay Denev In-Reply-To: <4B8649F4.4070303@omnilan.de> Date: Thu, 25 Feb 2010 12:32:44 +0200 Message-Id: <1C136B0A-E98F-4F93-8480-7FACD3FE4E4B@gmail.com> References: <4B8649F4.4070303@omnilan.de> To: Harald Schmalzbauer X-Mailer: Apple Mail (2.1077) Cc: freebsd-fs@freebsd.org Subject: Re: ZFS write stalls (starving reads) and tuning zfs_write_limit_override X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 10:32:54 -0000 --Apple-Mail-6-1023955540 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii On Feb 25, 2010, at 11:59 AM, Harald Schmalzbauer wrote: > Hello, >=20 > I found your very interesting patch: = http://lists.freebsd.org/pipermail/freebsd-fs/2010-February/007818.html > Unfortunately it seems to be -current since on -stable my dsl_pool.c = looks entire different. > My C skills don't allow me to adopt. Do you have any hints for me? >=20 > Thanks, >=20 > -Harry >=20 Hi, What I posted was not a patch, but just the lines need to be added to = dsl_pool.c and it should work for either 8 or CURRENT. Here is a patch for 8-STABLE that I just generated, can you try that one = (pasting in the email in case the email eats the attachment= --Apple-Mail-6-1023955540 Content-Disposition: attachment; filename=zfs_write_limit_override.patch Content-Type: application/octet-stream; x-unix-mode=0644; name="zfs_write_limit_override.patch" Content-Transfer-Encoding: 7bit --- .zfs/snapshot/orig/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_pool.c 2009-08-24 07:30:23.677549074 +0300 +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_pool.c 2010-02-19 09:17:50.058020997 +0200 @@ -47,6 +47,11 @@ uint64_t zfs_write_limit_override = 0; extern uint64_t zfs_write_limit_min; +SYSCTL_DECL(_vfs_zfs); +TUNABLE_ULONG("vfs.zfs.write_limit_override", &zfs_write_limit_override); +SYSCTL_ULONG(_vfs_zfs, OID_AUTO, zfs_write_limit_override, CTLFLAG_RW, &zfs_write_limit_override, 0, + "Override maximum TXG size"); + kmutex_t zfs_write_limit_lock; static pgcnt_t old_physmem = 0; --Apple-Mail-6-1023955540 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii ): --- = .zfs/snapshot/orig/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_pool= .c 2009-08-24 07:30:23.677549074 +0300 +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_pool.c = 2010-02-19 09:17:50.058020997 +0200 @@ -47,6 +47,11 @@ uint64_t zfs_write_limit_override =3D 0; extern uint64_t zfs_write_limit_min; =20 +SYSCTL_DECL(_vfs_zfs); +TUNABLE_ULONG("vfs.zfs.write_limit_override", = &zfs_write_limit_override); +SYSCTL_ULONG(_vfs_zfs, OID_AUTO, zfs_write_limit_override, CTLFLAG_RW, = &zfs_write_limit_override, 0, + "Override maximum TXG size"); + kmutex_t zfs_write_limit_lock; =20 static pgcnt_t old_physmem =3D 0; Regards, Niki Denev --Apple-Mail-6-1023955540-- From owner-freebsd-fs@FreeBSD.ORG Thu Feb 25 11:58:28 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B811E1065670 for ; Thu, 25 Feb 2010 11:58:28 +0000 (UTC) (envelope-from sfourman@gmail.com) Received: from mail-pz0-f199.google.com (mail-pz0-f199.google.com [209.85.222.199]) by mx1.freebsd.org (Postfix) with ESMTP id 8CF158FC26 for ; Thu, 25 Feb 2010 11:58:28 +0000 (UTC) Received: by pzk37 with SMTP id 37so1426351pzk.7 for ; Thu, 25 Feb 2010 03:58:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=RFgg+5waGdpX84NR4tgPls2lOAggU9YgVZ3r5IwZQm4=; b=n/4Lo+cN6Gtdrs/5fXWLYWs05tB/obPKjz+qpfDmQjRR0L1NRtaCEexxliRPo18io9 4Xn1XUIz1uAMY0bRE8UmXZVAwiC3TzN6mQLYwfapbYrlHaNnqlFDnsC/QNmcqFJqAvow KHWevO8ODk48HQUTSV6Rr6GfLRChM1FJBkwlU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=BYPIl9tduvh7KCnQ+n4/76uviacvrBvkM/mdPxZexAhukdPTg8N7nkFTjTqSs8om7Y 0gdm1cR6qk5JKZ290YtnF+2r4q3wtwWLsWNKuXlqZF+w++KS1jBhniAlsY6UbBNMYasT wkrLmVylJe0VkP5/6sB/jqAUa0yguc5iILnnc= MIME-Version: 1.0 Received: by 10.142.74.4 with SMTP id w4mr464127wfa.248.1267099101133; Thu, 25 Feb 2010 03:58:21 -0800 (PST) In-Reply-To: <1C136B0A-E98F-4F93-8480-7FACD3FE4E4B@gmail.com> References: <4B8649F4.4070303@omnilan.de> <1C136B0A-E98F-4F93-8480-7FACD3FE4E4B@gmail.com> Date: Thu, 25 Feb 2010 05:58:21 -0600 Message-ID: <11167f521002250358i46a09fb6j82aaddbd8e0b8334@mail.gmail.com> From: "Sam Fourman Jr." To: Nikolay Denev Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org, Harald Schmalzbauer Subject: Re: ZFS write stalls (starving reads) and tuning zfs_write_limit_override X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 11:58:28 -0000 > Hi, > > What I posted was not a patch, but just the lines need to be added to dsl= _pool.c and it should work for either 8 or CURRENT. > Here is a patch for 8-STABLE that I just generated, can you try that one = (pasting in the email in case the email eats the attachment > ): > > --- .zfs/snapshot/orig/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl= _pool.c =A0 =A0 =A0 =A02009-08-24 07:30:23.677549074 +0300 > +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_pool.c =A0 2010-02= -19 09:17:50.058020997 +0200 > @@ -47,6 +47,11 @@ > =A0uint64_t zfs_write_limit_override =3D 0; > =A0extern uint64_t zfs_write_limit_min; > > +SYSCTL_DECL(_vfs_zfs); > +TUNABLE_ULONG("vfs.zfs.write_limit_override", &zfs_write_limit_override)= ; > +SYSCTL_ULONG(_vfs_zfs, OID_AUTO, zfs_write_limit_override, CTLFLAG_RW, &= zfs_write_limit_override, 0, > + =A0 =A0"Override maximum TXG size"); > + > =A0kmutex_t zfs_write_limit_lock; > > =A0static pgcnt_t old_physmem =3D 0; > what setting have you found to be the best for zfs_write_limit_override ? Sam Fourman Jr. Fourman Networks From owner-freebsd-fs@FreeBSD.ORG Thu Feb 25 12:02:00 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4913510656A4 for ; Thu, 25 Feb 2010 12:02:00 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from mail-fx0-f223.google.com (mail-fx0-f223.google.com [209.85.220.223]) by mx1.freebsd.org (Postfix) with ESMTP id C88D38FC1F for ; Thu, 25 Feb 2010 12:01:59 +0000 (UTC) Received: by fxm23 with SMTP id 23so45402fxm.3 for ; Thu, 25 Feb 2010 04:01:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:subject:mime-version :content-type:from:in-reply-to:date:cc:content-transfer-encoding :message-id:references:to:x-mailer; bh=ODBUrvaTjdlo/87EtwBduDCeLL+AhxL0ytnYg+2r864=; b=sqUezwvNgpN8Rzv+MllDQVULr597+3K18niawJi9AHKLDGGnVfFMB4OW+LPzXCkq6A 7HnNNVwfaCk4lpk5dRjQEHLkLypwrW4vlhupZCyV+1DrFtYZAVcPFW+e13S2HXLOcEwO 31oyRFI0i0J6wmkhRQi3AU/jK9kb9y+173Se0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; b=QMHX53JVKu68jO0eIpbFphiHCizuEzYgYQBvt7XUZug3whvv8WKoooc6R7X5F0r7nG vWMhNJLbeGyd0FktMPkZER4d5ma/EtUJ6uWSLOTrciRA8gkeVeg1ze065cc8CHZ3Qwo0 X9ASP7XvVya4znhbLmoNMdpj7Nygl60ZTsDmc= Received: by 10.102.196.33 with SMTP id t33mr808861muf.2.1267099294664; Thu, 25 Feb 2010 04:01:34 -0800 (PST) Received: from ?10.32.23.105? ([195.34.111.178]) by mx.google.com with ESMTPS id w5sm13943072mue.22.2010.02.25.04.01.33 (version=TLSv1/SSLv3 cipher=RC4-MD5); Thu, 25 Feb 2010 04:01:33 -0800 (PST) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Nikolay Denev In-Reply-To: <11167f521002250358i46a09fb6j82aaddbd8e0b8334@mail.gmail.com> Date: Thu, 25 Feb 2010 14:01:32 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <72D31CA4-110E-4A85-AA19-BD74AFFC94B1@gmail.com> References: <4B8649F4.4070303@omnilan.de> <1C136B0A-E98F-4F93-8480-7FACD3FE4E4B@gmail.com> <11167f521002250358i46a09fb6j82aaddbd8e0b8334@mail.gmail.com> To: Sam Fourman Jr. X-Mailer: Apple Mail (2.1077) Cc: freebsd-fs@freebsd.org Subject: Re: ZFS write stalls (starving reads) and tuning zfs_write_limit_override X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 12:02:00 -0000 On Feb 25, 2010, at 1:58 PM, Sam Fourman Jr. wrote: >> Hi, >>=20 >> What I posted was not a patch, but just the lines need to be added to = dsl_pool.c and it should work for either 8 or CURRENT. >> Here is a patch for 8-STABLE that I just generated, can you try that = one (pasting in the email in case the email eats the attachment >> ): >>=20 >> --- = .zfs/snapshot/orig/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_pool= .c 2009-08-24 07:30:23.677549074 +0300 >> +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_pool.c = 2010-02-19 09:17:50.058020997 +0200 >> @@ -47,6 +47,11 @@ >> uint64_t zfs_write_limit_override =3D 0; >> extern uint64_t zfs_write_limit_min; >>=20 >> +SYSCTL_DECL(_vfs_zfs); >> +TUNABLE_ULONG("vfs.zfs.write_limit_override", = &zfs_write_limit_override); >> +SYSCTL_ULONG(_vfs_zfs, OID_AUTO, zfs_write_limit_override, = CTLFLAG_RW, &zfs_write_limit_override, 0, >> + "Override maximum TXG size"); >> + >> kmutex_t zfs_write_limit_lock; >>=20 >> static pgcnt_t old_physmem =3D 0; >>=20 >=20 >=20 > what setting have you found to be the best for = zfs_write_limit_override ? >=20 > Sam Fourman Jr. > Fourman Networks Hi, I think you should experiment. Here I have 2G of ram, and have found = 256M for the zfs_write_limit_override to provide smooth writes without the stalls.=20 I started one console with zpool iostat -v 1 and started writing to one = iSCSI exported zvol, and played with the sysctl until the writes were steady. Regards, Niki Denev From owner-freebsd-fs@FreeBSD.ORG Thu Feb 25 18:02:31 2010 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8794D106564A for ; Thu, 25 Feb 2010 18:02:31 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-iw0-f185.google.com (mail-iw0-f185.google.com [209.85.223.185]) by mx1.freebsd.org (Postfix) with ESMTP id 525508FC17 for ; Thu, 25 Feb 2010 18:02:30 +0000 (UTC) Received: by iwn15 with SMTP id 15so5348074iwn.7 for ; Thu, 25 Feb 2010 10:02:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=zrXznpqKeGjDZ7iNe4qoKkj8MDBD/HW9vmFLAJ1Z/00=; b=WMcx1+6TVmIXfIspFKyxTNXslm8B5eFqzCWCtly0xtdift7gk7RpE4lZw+FiUPrf0I gdfB0nGIbFSh3kF4pEWRLYgmBOzF23FoRHnEX/YAjzbCOhOtlsg4sk7jTQgF0JVi631V EMQ0SAo39ACYhXzOukBpTQtRRRebCY8z/MLDg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=q+CbxZUQwXi9NXIJsC+uMPSzilok1A/cTi+/CmTU3xeGCm5eIfnabweXl9ErF6A5A9 JvgDvnM1ft3vwQLPgODHWX2X4CnB0424Ool7xIZdAGGi2zKnyVh2k0wWRy7hBxz5GdIG UMW+Da7at8ZFmi/ZXPq5FRuYJbv9ftKvecZRw= MIME-Version: 1.0 Received: by 10.231.146.134 with SMTP id h6mr511607ibv.16.1267120943453; Thu, 25 Feb 2010 10:02:23 -0800 (PST) Date: Thu, 25 Feb 2010 10:02:23 -0800 Message-ID: From: Freddie Cash To: fs@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: HAST: file name consistency X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 18:02:31 -0000 Just curious: why is the config file called hast.conf, but all the RC vars and scripts use hastd? Shouldn't it all be "hastd" or all "hast"? For example, if the rc script is hastd, the rcvar is hastd_enable, shouldn't the config file be called hastd.conf, and the control program called hastdctl? Or, if the config file is hast.conf and the control program is hastctl, shouldn't the rcvar and script be just hast/hast_enable? Just seems strange to have some things called "hast" and other things called "hastd" when it's all the same program/setup. Perhaps I'm being too pedantic? -- Freddie Cash fjwcash@gmail.com From owner-freebsd-fs@FreeBSD.ORG Thu Feb 25 22:40:03 2010 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A732D106566B for ; Thu, 25 Feb 2010 22:40:03 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 7C8618FC08 for ; Thu, 25 Feb 2010 22:40:03 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id o1PMe3UT050918 for ; Thu, 25 Feb 2010 22:40:03 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id o1PMe3fu050917; Thu, 25 Feb 2010 22:40:03 GMT (envelope-from gnats) Date: Thu, 25 Feb 2010 22:40:03 GMT Message-Id: <201002252240.o1PMe3fu050917@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org From: Volodymyr Kostyrko Cc: Subject: Re: bin/144214: zfsboot fails on gang block after upgrade to zfs v14 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Volodymyr Kostyrko List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Feb 2010 22:40:03 -0000 The following reply was made to PR bin/144214; it has been noted by GNATS. From: Volodymyr Kostyrko To: bug-followup@FreeBSD.org, c.kworr@gmail.com Cc: Subject: Re: bin/144214: zfsboot fails on gang block after upgrade to zfs v14 Date: Fri, 26 Feb 2010 00:33:23 +0200 # gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0 gpart: No such geom: ada0 Could you explain what exactly it should do? Using gptzfsboot instead of simple zfsboot? -- Sphinx of black quartz judge my vow. From owner-freebsd-fs@FreeBSD.ORG Fri Feb 26 09:50:04 2010 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 10AD5106566C for ; Fri, 26 Feb 2010 09:50:04 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id D95748FC0C for ; Fri, 26 Feb 2010 09:50:03 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id o1Q9o3Gs047049 for ; Fri, 26 Feb 2010 09:50:03 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id o1Q9o3ix047048; Fri, 26 Feb 2010 09:50:03 GMT (envelope-from gnats) Date: Fri, 26 Feb 2010 09:50:03 GMT Message-Id: <201002260950.o1Q9o3ix047048@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org From: "Andrei V. Lavreniyuk" Cc: Subject: Re: bin/144214: zfsboot fails on gang block after upgrade to zfs v14 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: "Andrei V. Lavreniyuk" List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 09:50:04 -0000 The following reply was made to PR bin/144214; it has been noted by GNATS. From: "Andrei V. Lavreniyuk" To: bug-followup@FreeBSD.org, c.kworr@gmail.com Cc: Subject: Re: bin/144214: zfsboot fails on gang block after upgrade to zfs v14 Date: Fri, 26 Feb 2010 11:42:11 +0200 Hi! http://wiki.freebsd.org/ZFS http://blogs.freebsdish.org/lulf/2008/12/16/setting-up-a-zfs-only-system/ -- Best regards, Andrei V. Lavreniyuk. From owner-freebsd-fs@FreeBSD.ORG Fri Feb 26 10:00:08 2010 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2B1CF1065672 for ; Fri, 26 Feb 2010 10:00:08 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 0CB2B8FC14 for ; Fri, 26 Feb 2010 10:00:08 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id o1QA07WV047259 for ; Fri, 26 Feb 2010 10:00:07 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id o1QA0753047258; Fri, 26 Feb 2010 10:00:07 GMT (envelope-from gnats) Date: Fri, 26 Feb 2010 10:00:07 GMT Message-Id: <201002261000.o1QA0753047258@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org From: =?KOI8-R?Q?=F7=CF=CC=CF=C4=C9=CD=C9=D2_=EB=CF=D3=D4=C9=D2=CB=CF?= Cc: Subject: Re: bin/144214: zfsboot fails on gang block after upgrade to zfs v14 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: =?KOI8-R?Q?=F7=CF=CC=CF=C4=C9=CD=C9=D2_=EB=CF=D3=D4=C9=D2=CB=CF?= List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 10:00:08 -0000 The following reply was made to PR bin/144214; it has been noted by GNATS. From: =?KOI8-R?Q?=F7=CF=CC=CF=C4=C9=CD=C9=D2_=EB=CF=D3=D4=C9=D2=CB=CF?= To: andy.lavr@reactor-xg.kiev.ua Cc: bug-followup@FreeBSD.org Subject: Re: bin/144214: zfsboot fails on gang block after upgrade to zfs v14 Date: Fri, 26 Feb 2010 11:56:19 +0200 26.02.2010 11:42, Andrei V. Lavreniyuk wrote: > http://wiki.freebsd.org/ZFS > > http://blogs.freebsdish.org/lulf/2008/12/16/setting-up-a-zfs-only-system/ This is not my case. I have no partitions at all and I'm booting from a ZFS dedicated disk. And if I'm not missing something gpart requires separate partition with gptzfsboot to work. -- Sphinx of black quartz judge my vow. From owner-freebsd-fs@FreeBSD.ORG Fri Feb 26 22:25:45 2010 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 41C4E1065676; Fri, 26 Feb 2010 22:25:45 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 1A97A8FC1A; Fri, 26 Feb 2010 22:25:45 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id o1QMPits092371; Fri, 26 Feb 2010 22:25:44 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id o1QMPida092367; Fri, 26 Feb 2010 22:25:44 GMT (envelope-from linimon) Date: Fri, 26 Feb 2010 22:25:44 GMT Message-Id: <201002262225.o1QMPida092367@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-fs@FreeBSD.org From: linimon@FreeBSD.org Cc: Subject: Re: kern/144330: [nfs] mbuf leakage in nfsd with zfs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 22:25:45 -0000 Old Synopsis: mbuf leakage in nfsd with zfs New Synopsis: [nfs] mbuf leakage in nfsd with zfs Responsible-Changed-From-To: freebsd-bugs->freebsd-fs Responsible-Changed-By: linimon Responsible-Changed-When: Fri Feb 26 22:25:29 UTC 2010 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=144330 From owner-freebsd-fs@FreeBSD.ORG Sat Feb 27 00:00:09 2010 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E89FD1065670 for ; Sat, 27 Feb 2010 00:00:09 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id D7D3C8FC1E for ; Sat, 27 Feb 2010 00:00:09 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id o1R009wX068977 for ; Sat, 27 Feb 2010 00:00:09 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id o1R0091M068976; Sat, 27 Feb 2010 00:00:09 GMT (envelope-from gnats) Date: Sat, 27 Feb 2010 00:00:09 GMT Message-Id: <201002270000.o1R0091M068976@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org From: Dan Naumov Cc: Subject: RE: bin/144214: zfsboot fails on gang block after upgrade to zfs v14 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Dan Naumov List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Feb 2010 00:00:10 -0000 The following reply was made to PR bin/144214; it has been noted by GNATS. From: Dan Naumov To: bug-followup@FreeBSD.org, c.kworr@gmail.com Cc: Subject: RE: bin/144214: zfsboot fails on gang block after upgrade to zfs v14 Date: Sat, 27 Feb 2010 01:57:16 +0200 > I have no partitions at all and I'm booting from a ZFS dedicated disk. Sure you do have partitions. Booting off a purely ZFS disk without a valid partition table is not and has never been possible in neither FreeBSD nor Solaris. What you need to do is update your bootcode (because it has changed for zfs v14). How you do this depends on whether you use zfsboot or gptzfsboot. The links above describe the process. - Sincerely, Dan Naumov From owner-freebsd-fs@FreeBSD.ORG Sat Feb 27 08:23:19 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5527C1065673 for ; Sat, 27 Feb 2010 08:23:19 +0000 (UTC) (envelope-from h.schmalzbauer@omnilan.de) Received: from host.omnilan.net (host.omnilan.net [62.245.232.135]) by mx1.freebsd.org (Postfix) with ESMTP id B227B8FC18 for ; Sat, 27 Feb 2010 08:23:18 +0000 (UTC) Received: from titan.flintsbach.schmalzbauer.de (titan.flintsbach.schmalzbauer.de [172.21.1.150]) (authenticated bits=0) by host.omnilan.net (8.13.8/8.13.8) with ESMTP id o1R8NGsm031895 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 27 Feb 2010 09:23:16 +0100 (CET) (envelope-from h.schmalzbauer@omnilan.de) Message-ID: <4B88D66E.9040609@omnilan.de> Date: Sat, 27 Feb 2010 09:23:10 +0100 From: Harald Schmalzbauer Organization: OmniLAN User-Agent: Thunderbird 2.0.0.23 (X11/20090906) MIME-Version: 1.0 To: "Sam Fourman Jr." References: <4B8649F4.4070303@omnilan.de> <1C136B0A-E98F-4F93-8480-7FACD3FE4E4B@gmail.com> <11167f521002250358i46a09fb6j82aaddbd8e0b8334@mail.gmail.com> In-Reply-To: <11167f521002250358i46a09fb6j82aaddbd8e0b8334@mail.gmail.com> X-Enigmail-Version: 0.95.6 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig654D26BA0DFD027D19D9F936" Cc: freebsd-fs@freebsd.org Subject: Re: ZFS write stalls (starving reads) and tuning zfs_write_limit_override X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Feb 2010 08:23:19 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig654D26BA0DFD027D19D9F936 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable Sam Fourman Jr. schrieb am 25.02.2010 12:58 (localtime): >> Hi, >> >> What I posted was not a patch, but just the lines need to be added to = dsl_pool.c and it should work for either 8 or CURRENT. >> Here is a patch for 8-STABLE that I just generated, can you try that o= ne (pasting in the email in case the email eats the attachment >> ): >> >> --- .zfs/snapshot/orig/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/= dsl_pool.c 2009-08-24 07:30:23.677549074 +0300 >> +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_pool.c 2010-0= 2-19 09:17:50.058020997 +0200 >> @@ -47,6 +47,11 @@ >> uint64_t zfs_write_limit_override =3D 0; >> extern uint64_t zfs_write_limit_min; >> >> +SYSCTL_DECL(_vfs_zfs); >> +TUNABLE_ULONG("vfs.zfs.write_limit_override", &zfs_write_limit_overri= de); >> +SYSCTL_ULONG(_vfs_zfs, OID_AUTO, zfs_write_limit_override, CTLFLAG_RW= , &zfs_write_limit_override, 0, >> + "Override maximum TXG size"); >> + >> kmutex_t zfs_write_limit_lock; >> >> static pgcnt_t old_physmem =3D 0; >> >=20 >=20 > what setting have you found to be the best for zfs_write_limit_override= Unfortunately I couldn't find a sensible value. 250M was the best=20 compromise. But this reduced my avarage rsync rate to 30MB/s. Without zfs_write_limit_override and vfs.zfs.txg.timeout=3D1 I can get=20 65MB/s average, even with the mentioned pauses (peak is almost 90MB/s). Today I'm aware that 5400rpm spindles are not well suitable for ZFS, but = for big backup servers I'd take them again. If we could just find a way=20 to throttle ZFS _a bit_. Three 5400rpms should surely manage more than=20 30MB/s. Eventually I can transfer 65MB/s. Thanks, -Harry --------------enig654D26BA0DFD027D19D9F936 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.13 (FreeBSD) iEYEARECAAYFAkuI1nQACgkQLDqVQ9VXb8j/YACgmIa6Wu8eG7Y+k/StFfoSFGJ6 XVkAnRFzLQP6yghaibBznwK4+gNE5WPb =7vuk -----END PGP SIGNATURE----- --------------enig654D26BA0DFD027D19D9F936-- From owner-freebsd-fs@FreeBSD.ORG Sat Feb 27 19:21:16 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 066461065670; Sat, 27 Feb 2010 19:21:16 +0000 (UTC) (envelope-from gerrit@pmp.uni-hannover.de) Received: from mrelay1.uni-hannover.de (mrelay1.uni-hannover.de [130.75.2.106]) by mx1.freebsd.org (Postfix) with ESMTP id 7A01E8FC1B; Sat, 27 Feb 2010 19:21:09 +0000 (UTC) Received: from www.pmp.uni-hannover.de (www.pmp.uni-hannover.de [130.75.117.2]) by mrelay1.uni-hannover.de (8.14.2/8.14.2) with ESMTP id o1RJL5oM028046; Sat, 27 Feb 2010 20:21:07 +0100 Received: from pmp.uni-hannover.de (theq.pmp.uni-hannover.de [130.75.117.4]) by www.pmp.uni-hannover.de (Postfix) with SMTP id DF62624; Sat, 27 Feb 2010 20:21:05 +0100 (CET) Date: Sat, 27 Feb 2010 20:21:05 +0100 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: Willem Jan Withagen Message-Id: <20100227202105.f31cbef7.gerrit@pmp.uni-hannover.de> In-Reply-To: <4B892918.4080701@digiware.nl> References: <4B86F384.3010308@digiware.nl> <2a41acea1002251459v40e8c6ddxd0437decbada4594@mail.gmail.com> <4B8795B1.4020006@digiware.nl> <20100226120339.GB17798@icarus.home.lan> <20100226133138.d47dd080.gerrit@pmp.uni-hannover.de> <20100226134429.041ea6f2.gerrit@pmp.uni-hannover.de> <20100226141754.86ae5a3f.gerrit@pmp.uni-hannover.de> <20100226174021.8feadad9.gerrit@pmp.uni-hannover.de> <20100226224320.8c4259bf.gerrit@pmp.uni-hannover.de> <4B884757.9040001@digiware.nl> <20100227080220.ac6a2e4d.gerrit@pmp.uni-hannover.de> <4B892918.4080701@digiware.nl> Organization: Albert-Einstein-Institut (MPI =?ISO-8859-1?Q?f=FCr?= Gravitationsphysik & IGP =?ISO-8859-1?Q?Universit=E4t?= Hannover) X-Mailer: Sylpheed 2.4.2 (GTK+ 2.10.12; i386-portbld-freebsd6.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-PMX-Version: 5.5.9.388399, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2010.2.27.190629 Cc: freebsd-fs@freebsd.org, stable@freebsd.org, Jeremy Chadwick Subject: Re: mbuf leakage with nfs/zfs? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Feb 2010 19:21:16 -0000 On Sat, 27 Feb 2010 15:15:52 +0100 Willem Jan Withagen wrote about Re: mbuf leakage with nfs/zfs?: WJW> > 81492/2613/84105 mbufs in use (current/cache/total) WJW> > 80467/2235/82702/128000 mbuf clusters in use WJW> > (current/cache/total/max) 80458/822 mbuf+clusters out of packet WJW> > secondary zone in use (current/cache) WJW> Over the night I only had rsync and FreeBSD nfs traffic. WJW> WJW> 45337/2828/48165 mbufs in use (current/cache/total) WJW> 44708/1902/46610/262144 mbuf clusters in use (current/cache/total/max) WJW> 44040/888 mbuf+clusters out of packet secondary zone in use WJW> (current/cache) After about 24h I now have 128320/2630/130950 mbufs in use (current/cache/total) 127294/1200/128494/512000 mbuf clusters in use (current/cache/total/max) 127294/834 mbuf+clusters out of packet secondary zone in use (current/cache) WJW> I only have one Linux box runing Kubuntu 8.10, mounted UDP: WJW> (rw,udp,nolock,rsize=32768,wsize=32768,intr) Hm, are you able to narrow this down? Does a single Linux client with tcp mount cause the same trouble? Or a FreeBSD client with udp? If it was only Linux clients with udp mounts or something like this, I could understand why it took some time to pop up. WJW> But running something like 'find openembedded | xarg cat > /dev/null' WJW> Shows a steadily growing number of mbufs, and letting the system sit WJW> for 5 min. doesn't decrease the used mbufs I still have several udp and tcp mounts by Linux clients on my Server, though most of them are probably stale now after the upgrade; and my buffers keep draining... WJW> Doing this on another FreeBSD 7.2 client runs the mbufs up(max inc WJW> about 2000 mbuf), but within a few secs after the last file was WJW> fetched, the mbuf tab runs down to around to what is was before the WJW> command. FreeBSD client with udp mount? Then it is either Linux client with udp or all Linux clients triggering this leakage. I doubt that this is the case with all Linux clients, this would have caused more trouble earlier. WJW> Not shure where to go from here? I'm certainly not fluent enough in WJW> NFS to start interpreting a wireshark trace. Nor do I. I already wrote Rick Macklem an Email on Friday, but so far only got back an automated reply stating he is on "permanent vacation". I guess we need him or one of the other nfs guys to get this fixed. Could you try a single Linux client with tcp mount in the meantime? This would tell us if Linux clients as such are causing the issue, or if it is only Linux with udp mount. cu Gerrit P.S.: I cc'ed freebsd-fs because my PR went there. From owner-freebsd-fs@FreeBSD.ORG Sat Feb 27 19:27:19 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 724A81065676; Sat, 27 Feb 2010 19:27:19 +0000 (UTC) (envelope-from gerrit@pmp.uni-hannover.de) Received: from mrelay1.uni-hannover.de (mrelay1.uni-hannover.de [130.75.2.106]) by mx1.freebsd.org (Postfix) with ESMTP id E22548FC18; Sat, 27 Feb 2010 19:27:18 +0000 (UTC) Received: from www.pmp.uni-hannover.de (www.pmp.uni-hannover.de [130.75.117.2]) by mrelay1.uni-hannover.de (8.14.2/8.14.2) with ESMTP id o1RJQ7iS028144; Sat, 27 Feb 2010 20:26:08 +0100 Received: from pmp.uni-hannover.de (theq.pmp.uni-hannover.de [130.75.117.4]) by www.pmp.uni-hannover.de (Postfix) with SMTP id 6C06E24; Sat, 27 Feb 2010 20:26:07 +0100 (CET) Date: Sat, 27 Feb 2010 20:26:06 +0100 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: Daniel Braniss Message-Id: <20100227202606.67df5a59.gerrit@pmp.uni-hannover.de> In-Reply-To: References: <4B86F384.3010308@digiware.nl> <2a41acea1002251459v40e8c6ddxd0437decbada4594@mail.gmail.com> <4B8795B1.4020006@digiware.nl> <20100226120339.GB17798@icarus.home.lan> <20100226133138.d47dd080.gerrit@pmp.uni-hannover.de> <20100226134429.041ea6f2.gerrit@pmp.uni-hannover.de> <20100226141754.86ae5a3f.gerrit@pmp.uni-hannover.de> <20100226174021.8feadad9.gerrit@pmp.uni-hannover.de> <20100226224320.8c4259bf.gerrit@pmp.uni-hannover.de> <4B884757.9040001@digiware.nl> <20100227080928.e49cb82c.gerrit@pmp.uni-hannover.de> <20100227095702.ffa01da8.gerrit@pmp.uni-hannover.de> <20100227111618.df9d29fb.gerrit@pmp.uni-hannover.de> Organization: Albert-Einstein-Institut (MPI =?ISO-8859-1?Q?f=FCr?= Gravitationsphysik & IGP =?ISO-8859-1?Q?Universit=E4t?= Hannover) X-Mailer: Sylpheed 2.4.2 (GTK+ 2.10.12; i386-portbld-freebsd6.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-PMX-Version: 5.5.9.388399, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2010.2.27.190629 Cc: freebsd-fs@freebsd.org, stable@freebsd.org, Willem Jan Withagen , Jeremy Chadwick Subject: Re: mbuf leakage with nfs/zfs? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Feb 2010 19:27:19 -0000 On Sat, 27 Feb 2010 12:26:02 +0200 Daniel Braniss wrote about Re: mbuf leakage with nfs/zfs? : DB> > Hm, I've got a server with 8-PRE from somewhen in Nov09 that is DB> > serving nfs from zfs fine and shows no leakage... DB> the binary search has started! After considering the last email from Willem: My 8-PRE server does not have udp Linux clients, only Linux with tcp. If indeed Linux with udp is causing the problem, it may very well even be in 8-PRE, and I just did not see it so far. cu Gerrit From owner-freebsd-fs@FreeBSD.ORG Sat Feb 27 19:51:38 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 282D8106566B for ; Sat, 27 Feb 2010 19:51:38 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta10.westchester.pa.mail.comcast.net (qmta10.westchester.pa.mail.comcast.net [76.96.62.17]) by mx1.freebsd.org (Postfix) with ESMTP id C39508FC15 for ; Sat, 27 Feb 2010 19:51:37 +0000 (UTC) Received: from omta13.westchester.pa.mail.comcast.net ([76.96.62.52]) by qmta10.westchester.pa.mail.comcast.net with comcast id n5rf1d00217dt5G5A7eNyR; Sat, 27 Feb 2010 19:38:22 +0000 Received: from koitsu.dyndns.org ([98.248.46.159]) by omta13.westchester.pa.mail.comcast.net with comcast id n7eM1d0073S48mS3Z7eMNP; Sat, 27 Feb 2010 19:38:22 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id BB85B1E301A; Sat, 27 Feb 2010 11:38:19 -0800 (PST) Date: Sat, 27 Feb 2010 11:38:19 -0800 From: Jeremy Chadwick To: Gerrit =?iso-8859-1?Q?K=FChn?= Message-ID: <20100227193819.GA60576@icarus.home.lan> References: <20100226141754.86ae5a3f.gerrit@pmp.uni-hannover.de> <20100226174021.8feadad9.gerrit@pmp.uni-hannover.de> <20100226224320.8c4259bf.gerrit@pmp.uni-hannover.de> <4B884757.9040001@digiware.nl> <20100227080220.ac6a2e4d.gerrit@pmp.uni-hannover.de> <4B892918.4080701@digiware.nl> <20100227202105.f31cbef7.gerrit@pmp.uni-hannover.de> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20100227202105.f31cbef7.gerrit@pmp.uni-hannover.de> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-fs@freebsd.org, stable@freebsd.org, Willem Jan Withagen Subject: Re: mbuf leakage with nfs/zfs? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Feb 2010 19:51:38 -0000 On Sat, Feb 27, 2010 at 08:21:05PM +0100, Gerrit Kühn wrote: > On Sat, 27 Feb 2010 15:15:52 +0100 Willem Jan Withagen > wrote about Re: mbuf leakage with nfs/zfs?: > > WJW> > 81492/2613/84105 mbufs in use (current/cache/total) > WJW> > 80467/2235/82702/128000 mbuf clusters in use > WJW> > (current/cache/total/max) 80458/822 mbuf+clusters out of packet > WJW> > secondary zone in use (current/cache) > > WJW> Over the night I only had rsync and FreeBSD nfs traffic. > WJW> > WJW> 45337/2828/48165 mbufs in use (current/cache/total) > WJW> 44708/1902/46610/262144 mbuf clusters in use (current/cache/total/max) > WJW> 44040/888 mbuf+clusters out of packet secondary zone in use > WJW> (current/cache) > > After about 24h I now have > > 128320/2630/130950 mbufs in use (current/cache/total) > 127294/1200/128494/512000 mbuf clusters in use (current/cache/total/max) > 127294/834 mbuf+clusters out of packet secondary zone in use (current/cache) Follow-up regarding my server statistics shown here: http://lists.freebsd.org/pipermail/freebsd-stable/2010-February/055458.html I just pulled the statistics on the same servers for comparison (then vs. now). RELENG_7 amd64 2010/01/09 -- primary HTTP, pri DNS, SSH server + ZFS 515/1930/2445 mbufs in use (current/cache/total) 512/540/1052/25600 mbuf clusters in use (current/cache/total/max) 1152K/6394K/7547K bytes allocated to network (current/cache/total) RELENG_7 amd64 2010/01/11 -- secondary DNS, MySQL, dev box + ZFS 514/1151/1665 mbufs in use (current/cache/total) 512/504/1016/25600 mbuf clusters in use (current/cache/total/max) 1152K/2203K/3356K bytes allocated to network (current/cache/total) RELENG_7 i386 2008/04/19 -- secondary HTTP, SSH server, heavy memory I/O 515/820/1335 mbufs in use (current/cache/total) 513/631/1144/25600 mbuf clusters in use (current/cache/total/max) 1154K/2615K/3769K bytes allocated to network (current/cache/total) RELENG_8 amd64 2010/02/02 -- central backups + NFS+ZFS-based filer 1572/3423/4995 mbufs in use (current/cache/total) 1539/3089/4628/25600 mbuf clusters in use (current/cache/total/max) 3471K/7449K/10920K bytes allocated to network (current/cache/total) So, not much difference. I should point out that the NFS+ZFS-based filer doesn't actually do its backups using NFS; it uses rsnapshot (rsync) over SSH. There is intense network I/O during backup time though, depending on how much data there is to back up. The NFS mounts (on the clients) are only used to provide a way for people to get access to their nightly backups in a convenient way; it isn't used very heavily. I can do something NFS-intensive on any of the above clients if people want me to kind of testing. Possibly an rsync with a source of the NFS mount and a destination of the local disk would be a good test? Let me know if anyone's interested in me testing that. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Sat Feb 27 20:03:58 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A5262106566C for ; Sat, 27 Feb 2010 20:03:58 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.16.84]) by mx1.freebsd.org (Postfix) with ESMTP id 4CE698FC1A for ; Sat, 27 Feb 2010 20:03:58 +0000 (UTC) Received: from pampa.cs.huji.ac.il ([132.65.80.32]) by kabab.cs.huji.ac.il with esmtp id 1NlST1-000Jhg-KZ; Sat, 27 Feb 2010 21:36:47 +0200 X-Mailer: exmh version 2.7.2 01/07/2005 with nmh-1.2 To: Gerrit =?ISO-8859-1?Q?K=FChn?= In-reply-to: <20100227202606.67df5a59.gerrit@pmp.uni-hannover.de> References: <4B86F384.3010308@digiware.nl> <2a41acea1002251459v40e8c6ddxd0437decbada4594@mail.gmail.com> <4B8795B1.4020006@digiware.nl> <20100226120339.GB17798@icarus.home.lan> <20100226133138.d47dd080.gerrit@pmp.uni-hannover.de> <20100226134429.041ea6f2.gerrit@pmp.uni-hannover.de> <20100226141754.86ae5a3f.gerrit@pmp.uni-hannover.de> <20100226174021.8feadad9.gerrit@pmp.uni-hannover.de> <20100226224320.8c4259bf.gerrit@pmp.uni-hannover.de> <4B884757.9040001@digiware.nl> <20100227080928.e49cb82c.gerrit@pmp.uni-hannover.de> <20100227095702.ffa01da8.gerrit@pmp.uni-hannover.de> <20100227111618.df9d29fb.gerrit@pmp.uni-hannover.de> <20100227202606.67df5a59.gerrit@pmp.uni-hannover.de> Comments: In-reply-to Gerrit =?ISO-8859-1?Q?K=FChn?= message dated "Sat, 27 Feb 2010 20:26:06 +0100." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sat, 27 Feb 2010 21:36:47 +0200 From: Daniel Braniss Message-ID: Cc: freebsd-fs@freebsd.org, stable@freebsd.org, Willem Jan Withagen , Jeremy Chadwick Subject: Re: mbuf leakage with nfs/zfs? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Feb 2010 20:03:58 -0000 > On Sat, 27 Feb 2010 12:26:02 +0200 Daniel Braniss > wrote about Re: mbuf leakage with nfs/zfs? : > > > DB> > Hm, I've got a server with 8-PRE from somewhen in Nov09 that is > DB> > serving nfs from zfs fine and shows no leakage... > > DB> the binary search has started! > > After considering the last email from Willem: My 8-PRE server does not > have udp Linux clients, only Linux with tcp. If indeed Linux with udp is > causing the problem, it may very well even be in 8-PRE, and I just did not > see it so far. I have been running for the last few hours, 8-rel, and the only client is another 8-stable, furthermore, no ZFS, just plain UFS, and the leak is there! I am now trying 8-rc2 but will check in the morning, it is after all saturday night :-) danny From owner-freebsd-fs@FreeBSD.ORG Sat Feb 27 20:47:25 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2940C106566C for ; Sat, 27 Feb 2010 20:47:25 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id 69EC98FC0A for ; Sat, 27 Feb 2010 20:47:24 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 93A6445E90; Sat, 27 Feb 2010 21:47:22 +0100 (CET) Received: from localhost (shell.garage.wheel.pl [10.0.4.3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 0619845E87; Sat, 27 Feb 2010 21:47:16 +0100 (CET) Date: Sat, 27 Feb 2010 21:47:15 +0100 From: Pawel Jakub Dawidek To: Nikolay Denev Message-ID: <20100227204715.GC2586@garage.freebsd.pl> References: <26CCAE99-B056-4754-9324-2945D1643C78@gmail.com> <272C3933-72C9-41AF-AD3A-605A7C57FB61@gmail.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="h3LYUU6HlUDSAOzy" Content-Disposition: inline In-Reply-To: <272C3933-72C9-41AF-AD3A-605A7C57FB61@gmail.com> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@freebsd.org Subject: Re: should zfs care so much about device name changes X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Feb 2010 20:47:25 -0000 --h3LYUU6HlUDSAOzy Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Feb 24, 2010 at 03:58:58PM +0200, Nikolay Denev wrote: >=20 > On Feb 24, 2010, at 11:53 AM, Nikolay Denev wrote: >=20 > > Hello, > >=20 > > I wanted to test the new option ATA_CAM, but that would require a boot = from another media (USB Drive/CD) and a zpool export/import > > to update the vdev names. (actually it's shown as "path" in the zpool.c= ache file) because otherwise the system would refuse to open/mount > > the pool. > > But is that really necessary given the fact that all the devices are he= re, and have matching GUIDS to those in the zpool.cache file? > > Shouldnt ZFS just import the pool?=20 > >=20 > > In the current state what would one have to do to for example test ATA_= CAM on a remote machine, where export/import from a rescue > > media is not possible? > >=20 > > Thanks, > > Niki >=20 >=20 > I have now looked at sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_= geom.c and it seems that when a vdev isn't found by pathname it > is being searched by guid. But it doesn't seem to be the case here. The l= oader sees the pool and boots the kernel, but then the kernel does not see > the pool. Any ideas? Try adding vfs.zfs.debug=3D1 to /boot/loader.conf. This will hopefully show what's going on. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --h3LYUU6HlUDSAOzy Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkuJhNIACgkQForvXbEpPzR3PACghV36BwqjGku/MFnzq2MWAHVd wVUAoM06RdVsS7MO/9N4RpxoJJzZveif =HZmL -----END PGP SIGNATURE----- --h3LYUU6HlUDSAOzy-- From owner-freebsd-fs@FreeBSD.ORG Sat Feb 27 21:00:11 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0ACE6106564A; Sat, 27 Feb 2010 21:00:11 +0000 (UTC) (envelope-from ltning@anduin.net) Received: from mail.anduin.net (mail.anduin.net [213.225.74.249]) by mx1.freebsd.org (Postfix) with ESMTP id 715578FC17; Sat, 27 Feb 2010 21:00:10 +0000 (UTC) Received: from [212.62.248.146] (helo=[192.168.2.198]) by mail.anduin.net with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.69 (FreeBSD)) (envelope-from ) id 1NlTL6-000FX0-Mg; Sat, 27 Feb 2010 21:32:40 +0100 Mime-Version: 1.0 (Apple Message framework v1077) From: =?iso-8859-1?Q?Eirik_=D8verby?= In-Reply-To: <20100227193819.GA60576@icarus.home.lan> Date: Sat, 27 Feb 2010 21:32:39 +0100 Message-Id: References: <20100226141754.86ae5a3f.gerrit@pmp.uni-hannover.de> <20100226174021.8feadad9.gerrit@pmp.uni-hannover.de> <20100226224320.8c4259bf.gerrit@pmp.uni-hannover.de> <4B884757.9040001@digiware.nl> <20100227080220.ac6a2e4d.gerrit@pmp.uni-hannover.de> <4B892918.4080701@digiware.nl> <20100227202105.f31cbef7.gerrit@pmp.uni-hannover.de> <20100227193819.GA60576@icarus.home.lan> To: Jeremy Chadwick X-Mailer: Apple Mail (2.1077) Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org, stable@freebsd.org, Willem Jan Withagen Subject: Re: mbuf leakage with nfs/zfs? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Feb 2010 21:00:11 -0000 On 27. feb. 2010, at 20.38, Jeremy Chadwick wrote: > On Sat, Feb 27, 2010 at 08:21:05PM +0100, Gerrit K=FChn wrote: >> On Sat, 27 Feb 2010 15:15:52 +0100 Willem Jan Withagen = >> wrote about Re: mbuf leakage with nfs/zfs?: >>=20 >> WJW> > 81492/2613/84105 mbufs in use (current/cache/total) >> WJW> > 80467/2235/82702/128000 mbuf clusters in use >> WJW> > (current/cache/total/max) 80458/822 mbuf+clusters out of = packet >> WJW> > secondary zone in use (current/cache) >>=20 >> WJW> Over the night I only had rsync and FreeBSD nfs traffic. >> WJW>=20 >> WJW> 45337/2828/48165 mbufs in use (current/cache/total) >> WJW> 44708/1902/46610/262144 mbuf clusters in use = (current/cache/total/max) >> WJW> 44040/888 mbuf+clusters out of packet secondary zone in use >> WJW> (current/cache) >>=20 >> After about 24h I now have >>=20 >> 128320/2630/130950 mbufs in use (current/cache/total) >> 127294/1200/128494/512000 mbuf clusters in use = (current/cache/total/max) >> 127294/834 mbuf+clusters out of packet secondary zone in use = (current/cache) >=20 > Follow-up regarding my server statistics shown here: >=20 > = http://lists.freebsd.org/pipermail/freebsd-stable/2010-February/055458.htm= l >=20 > I just pulled the statistics on the same servers for comparison (then > vs. now). >=20 > RELENG_7 amd64 2010/01/09 -- primary HTTP, pri DNS, SSH server + ZFS >=20 > 515/1930/2445 mbufs in use (current/cache/total) > 512/540/1052/25600 mbuf clusters in use = (current/cache/total/max) > 1152K/6394K/7547K bytes allocated to network = (current/cache/total) >=20 > RELENG_7 amd64 2010/01/11 -- secondary DNS, MySQL, dev box + ZFS >=20 > 514/1151/1665 mbufs in use (current/cache/total) > 512/504/1016/25600 mbuf clusters in use = (current/cache/total/max) > 1152K/2203K/3356K bytes allocated to network = (current/cache/total) >=20 > RELENG_7 i386 2008/04/19 -- secondary HTTP, SSH server, heavy memory = I/O >=20 > 515/820/1335 mbufs in use (current/cache/total) > 513/631/1144/25600 mbuf clusters in use = (current/cache/total/max) > 1154K/2615K/3769K bytes allocated to network = (current/cache/total) >=20 > RELENG_8 amd64 2010/02/02 -- central backups + NFS+ZFS-based filer >=20 > 1572/3423/4995 mbufs in use (current/cache/total) > 1539/3089/4628/25600 mbuf clusters in use = (current/cache/total/max) > 3471K/7449K/10920K bytes allocated to network = (current/cache/total) >=20 > So, not much difference. >=20 > I should point out that the NFS+ZFS-based filer doesn't actually do = its > backups using NFS; it uses rsnapshot (rsync) over SSH. There is = intense > network I/O during backup time though, depending on how much data = there > is to back up. The NFS mounts (on the clients) are only used to = provide > a way for people to get access to their nightly backups in a = convenient > way; it isn't used very heavily. >=20 > I can do something NFS-intensive on any of the above clients if people > want me to kind of testing. Possibly an rsync with a source of the = NFS > mount and a destination of the local disk would be a good test? Let = me > know if anyone's interested in me testing that. I've had a discussion with some folks on this for a while. I can easily = reproduce this situation by mounting a FreeBSD ZFS filesystem via = NFS-UDP from an OpenBSD machine. Telling the OpenBSD machine to use TCP = instead of UDP makes the problem go away. Other FreeBSD systems mounting the same share, either using UDP or TCP, = does not cause the problem to show up. A patch was suggested by Rick Macklem, but that did not solve the issue: = http://lists.freebsd.org/pipermail/freebsd-current/2009-December/014181.ht= ml /Eirik > --=20 > | Jeremy Chadwick jdc@parodius.com | > | Parodius Networking http://www.parodius.com/ | > | UNIX Systems Administrator Mountain View, CA, USA | > | Making life hard for others since 1977. PGP: 4BD6C0CB | >=20 > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to = "freebsd-stable-unsubscribe@freebsd.org" >=20 From owner-freebsd-fs@FreeBSD.ORG Sat Feb 27 21:11:59 2010 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AF16A106566B for ; Sat, 27 Feb 2010 21:11:59 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id EDCBB8FC0A for ; Sat, 27 Feb 2010 21:11:58 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id F0C4D45CDC; Sat, 27 Feb 2010 21:44:18 +0100 (CET) Received: from localhost (shell.garage.wheel.pl [10.0.4.3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 1BD6F45C9C; Sat, 27 Feb 2010 21:44:13 +0100 (CET) Date: Sat, 27 Feb 2010 21:44:10 +0100 From: Pawel Jakub Dawidek To: Freddie Cash Message-ID: <20100227204410.GB2586@garage.freebsd.pl> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="f5QefDQHtn8hx44O" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: fs@freebsd.org Subject: Re: HAST: file name consistency X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Feb 2010 21:11:59 -0000 --f5QefDQHtn8hx44O Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Feb 25, 2010 at 10:02:23AM -0800, Freddie Cash wrote: > Just curious: why is the config file called hast.conf, but all the RC va= rs > and scripts use hastd? Shouldn't it all be "hastd" or all "hast"? >=20 > For example, if the rc script is hastd, the rcvar is hastd_enable, should= n't > the config file be called hastd.conf, and the control program called > hastdctl? >=20 > Or, if the config file is hast.conf and the control program is hastctl, > shouldn't the rcvar and script be just hast/hast_enable? >=20 > Just seems strange to have some things called "hast" and other things cal= led > "hastd" when it's all the same program/setup. >=20 > Perhaps I'm being too pedantic? Nope, I also did some thinking about this while I was working on it. Actually the configuration file name was hastd.conf for some time, but I changed it to hast.conf at the end. The reason for hast.conf is that it is not only used by hastd(8) daemon, but also by hastctl(8) control utility, so bascially by entire HAST software. That's why hastd.conf was in my opinion confusing, as it suggested that it was only used by the hastd(8) daemon. The rc.d/hastd script is there to start hastd(8) daemon. That's the only thing it does. It doesn't even configure resources role on start using hastctl(8) (it only stops resources on shutdown). That's why script and variables are named 'hastd*' and not 'hast*'. Now hastctl name. HAST is the software (project) name. hastctl means HAST-_c_on_t_ro_l_ just like hastd means HAST-_d_aemon. hastctl doesn't only control hastd, but can do other things too, eg. initialize metadata (create subcommand) and dump metadata (dump subcommand) without communicating with hastd. Does that make sense for you? --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --f5QefDQHtn8hx44O Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkuJhBoACgkQForvXbEpPzRpLwCguLgi//a2PBQLxwrzO6k8cr4H 5tgAniW3OvepALrXhww99DTWC00LEno7 =qHEe -----END PGP SIGNATURE----- --f5QefDQHtn8hx44O-- From owner-freebsd-fs@FreeBSD.ORG Sat Feb 27 21:31:58 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 97062106566B; Sat, 27 Feb 2010 21:31:58 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: from mail-yx0-f177.google.com (mail-yx0-f177.google.com [209.85.210.177]) by mx1.freebsd.org (Postfix) with ESMTP id ECB328FC1A; Sat, 27 Feb 2010 21:31:57 +0000 (UTC) Received: by yxe7 with SMTP id 7so367088yxe.3 for ; Sat, 27 Feb 2010 13:31:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=b2Yuaigc6fz2bdvIRNBqRLf4M3zNhVSQFqEnYz3ewro=; b=MjD+u3zw2bHtBQmOb05uopYs1bqXAvwpDUUX+hmS15kh32wZCnDkpz8CVMBVueejGI pSJgqxkdSbjNftUqVc4VQLsYc179Nf7owIiNJGj1IgCuPy8Vqjd+6XhST+YuWbyHSXeH C5CytuJ2OnsgamR3sJTgY7q3+bNIv2NCiJ4PE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=rM+tivnS1NZ26nptw0sUBEV6QopdkDMKzl9UPcj3fg4NyHxs1IT7NsRT9yPs1TAVyx mwcdcT+7l8c+bkEcxDwZuSIdgjTqWw4U+JdGCoVIY0Sw57FAlrVGOps0xdhW+hOe2IZv oh9qcC26iaQMGbJYR2YQQx8su9frZLg8n2gYY= MIME-Version: 1.0 Received: by 10.101.172.3 with SMTP id z3mr3225792ano.192.1267306310832; Sat, 27 Feb 2010 13:31:50 -0800 (PST) Date: Sat, 27 Feb 2010 23:31:50 +0200 Message-ID: From: Dan Naumov To: FreeBSD-STABLE Mailing List , freebsd-questions@freebsd.org, freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: Subject: Re: 8.0-RELEASE/amd64 - full ZFS install - low read and write disk performance (fixed) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Feb 2010 21:31:58 -0000 Hello folks A few weeks ago, there was a discussion started by me regarding abysmal read/write performance using ZFS mirror on 8.0-RELEASE. I was using an Atom 330 system with 2GB ram and it was pointed out to me that my problem was most likely having both disks attached to a PCI SIL3124 controller, switching to the new AHCI drivers didn't help one bit. To reitirate, here are the Bonnie and DD numbers I got on that system: =================================================== Atom 330 / 2gb ram / Intel board + PCI SIL3124 -------Sequential Output-------- ---Sequential Input-- --Random-- -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks--- Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU 8192 21041 53.5 22644 19.4 13724 12.8 25321 48.5 43110 14.0 143.2 3.3 dd if=/dev/zero of=/root/test1 bs=1M count=4096 4096+0 records in 4096+0 records out 4294967296 bytes transferred in 143.878615 secs (29851325 bytes/sec) (28,4 mb/s) =================================================== Since then, I switched the exact same disks to a different system: Atom D510 / 4gb ram / Supermicro X7SPA-H / ICH9R controller (native). Here are the updated results: -------Sequential Output-------- ---Sequential Input-- --Random-- -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks--- Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU 8192 30057 68.7 50965 36.4 27236 21.3 33317 58.0 53051 14.3 172.4 3.2 dd if=/dev/zero of=/root/test1 bs=1M count=4096 4096+0 records in 4096+0 records out 4294967296 bytes transferred in 54.977978 secs (78121594 bytes/sec) (74,5 mb/s) =================================================== Write performance now seems to have increased by a factor of 2 to 3 and is now definately in line with the expected performance of the disks in question (cheap 2TB WD20EADS with 32mb cache). Thanks to everyone who has offered help and tips! - Sincerely, Dan Naumov From owner-freebsd-fs@FreeBSD.ORG Sat Feb 27 21:38:11 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 42598106566C; Sat, 27 Feb 2010 21:38:11 +0000 (UTC) (envelope-from gerrit@pmp.uni-hannover.de) Received: from mrelay1.uni-hannover.de (mrelay1.uni-hannover.de [130.75.2.106]) by mx1.freebsd.org (Postfix) with ESMTP id B96028FC12; Sat, 27 Feb 2010 21:38:10 +0000 (UTC) Received: from www.pmp.uni-hannover.de (www.pmp.uni-hannover.de [130.75.117.2]) by mrelay1.uni-hannover.de (8.14.2/8.14.2) with ESMTP id o1RLc2nV031058; Sat, 27 Feb 2010 22:38:03 +0100 Received: from pmp.uni-hannover.de (theq.pmp.uni-hannover.de [130.75.117.4]) by www.pmp.uni-hannover.de (Postfix) with SMTP id 5E92924; Sat, 27 Feb 2010 22:38:02 +0100 (CET) Date: Sat, 27 Feb 2010 22:38:01 +0100 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: Eirik =?ISO-8859-1?Q?=D8verby?= Message-Id: <20100227223801.15aa657b.gerrit@pmp.uni-hannover.de> In-Reply-To: References: <20100226141754.86ae5a3f.gerrit@pmp.uni-hannover.de> <20100226174021.8feadad9.gerrit@pmp.uni-hannover.de> <20100226224320.8c4259bf.gerrit@pmp.uni-hannover.de> <4B884757.9040001@digiware.nl> <20100227080220.ac6a2e4d.gerrit@pmp.uni-hannover.de> <4B892918.4080701@digiware.nl> <20100227202105.f31cbef7.gerrit@pmp.uni-hannover.de> <20100227193819.GA60576@icarus.home.lan> Organization: Albert-Einstein-Institut (MPI =?ISO-8859-1?Q?f=FCr?= Gravitationsphysik & IGP =?ISO-8859-1?Q?Universit=E4t?= Hannover) X-Mailer: Sylpheed 2.4.2 (GTK+ 2.10.12; i386-portbld-freebsd6.1) Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-PMX-Version: 5.5.9.388399, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2010.2.27.212132 Cc: freebsd-fs@freebsd.org, stable@freebsd.org, Willem Jan Withagen , Jeremy Chadwick Subject: Re: mbuf leakage with nfs/zfs? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Feb 2010 21:38:11 -0000 On Sat, 27 Feb 2010 21:32:39 +0100 Eirik =D8verby wrote about Re: mbuf leakage with nfs/zfs?: E> I've had a discussion with some folks on this for a while. I can easily E> reproduce this situation by mounting a FreeBSD ZFS filesystem via E> NFS-UDP from an OpenBSD machine. Telling the OpenBSD machine to use TCP E> instead of UDP makes the problem go away. So we see this problem with udp clients from OpenBSD and Linux. E> Other FreeBSD systems mounting the same share, either using UDP or TCP, E> does not cause the problem to show up. As Daniel reported he saw the problem with FBSD 8-stable: Which version was the FBSD-client that worked for you with udp? E> A patch was suggested by Rick Macklem, but that did not solve the issue: E> http://lists.freebsd.org/pipermail/freebsd-current/2009-December/014181.= html Yeah, I also found and tried this on Friday - unfortunately without any success, the leakage is still there. cu Gerrit From owner-freebsd-fs@FreeBSD.ORG Sat Feb 27 21:39:50 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 84E111065670; Sat, 27 Feb 2010 21:39:50 +0000 (UTC) (envelope-from gerrit@pmp.uni-hannover.de) Received: from mrelay1.uni-hannover.de (mrelay1.uni-hannover.de [130.75.2.106]) by mx1.freebsd.org (Postfix) with ESMTP id F2AF28FC12; Sat, 27 Feb 2010 21:39:49 +0000 (UTC) Received: from www.pmp.uni-hannover.de (www.pmp.uni-hannover.de [130.75.117.2]) by mrelay1.uni-hannover.de (8.14.2/8.14.2) with ESMTP id o1RLdItd031072; Sat, 27 Feb 2010 22:39:19 +0100 Received: from pmp.uni-hannover.de (theq.pmp.uni-hannover.de [130.75.117.4]) by www.pmp.uni-hannover.de (Postfix) with SMTP id 4307F24; Sat, 27 Feb 2010 22:39:18 +0100 (CET) Date: Sat, 27 Feb 2010 22:39:17 +0100 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: Daniel Braniss Message-Id: <20100227223917.6d5d0238.gerrit@pmp.uni-hannover.de> In-Reply-To: References: <4B86F384.3010308@digiware.nl> <4B8795B1.4020006@digiware.nl> <20100226120339.GB17798@icarus.home.lan> <20100226133138.d47dd080.gerrit@pmp.uni-hannover.de> <20100226134429.041ea6f2.gerrit@pmp.uni-hannover.de> <20100226141754.86ae5a3f.gerrit@pmp.uni-hannover.de> <20100226174021.8feadad9.gerrit@pmp.uni-hannover.de> <20100226224320.8c4259bf.gerrit@pmp.uni-hannover.de> <4B884757.9040001@digiware.nl> <20100227080928.e49cb82c.gerrit@pmp.uni-hannover.de> <20100227095702.ffa01da8.gerrit@pmp.uni-hannover.de> <20100227111618.df9d29fb.gerrit@pmp.uni-hannover.de> <20100227202606.67df5a59.gerrit@pmp.uni-hannover.de> Organization: Albert-Einstein-Institut (MPI =?ISO-8859-1?Q?f=FCr?= Gravitationsphysik & IGP =?ISO-8859-1?Q?Universit=E4t?= Hannover) X-Mailer: Sylpheed 2.4.2 (GTK+ 2.10.12; i386-portbld-freebsd6.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-PMX-Version: 5.5.9.388399, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2010.2.27.213052 Cc: freebsd-fs@freebsd.org, stable@freebsd.org, Willem Jan Withagen , Jeremy Chadwick Subject: Re: mbuf leakage with nfs/zfs? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Feb 2010 21:39:50 -0000 On Sat, 27 Feb 2010 21:36:47 +0200 Daniel Braniss wrote about Re: mbuf leakage with nfs/zfs? : DB> I have been running for the last few hours, 8-rel, and the only client DB> is another DB> 8-stable, furthermore, no ZFS, just plain UFS, and the leak is there! Mounted via udp, not tcp, I guess...?! DB> I am now trying 8-rc2 but will check in the morning, it is after all DB> saturday night :-) Same here. :-) cu Gerrit From owner-freebsd-fs@FreeBSD.ORG Sat Feb 27 21:40:47 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3D2EB1065679; Sat, 27 Feb 2010 21:40:47 +0000 (UTC) (envelope-from ltning@anduin.net) Received: from mail.anduin.net (mail.anduin.net [213.225.74.249]) by mx1.freebsd.org (Postfix) with ESMTP id E8C688FC25; Sat, 27 Feb 2010 21:40:46 +0000 (UTC) Received: from [212.62.248.146] (helo=[192.168.2.198]) by mail.anduin.net with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.69 (FreeBSD)) (envelope-from ) id 1NlUOy-000L0L-9Q; Sat, 27 Feb 2010 22:40:44 +0100 Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=iso-8859-1 From: =?iso-8859-1?Q?Eirik_=D8verby?= In-Reply-To: <20100227223801.15aa657b.gerrit@pmp.uni-hannover.de> Date: Sat, 27 Feb 2010 22:40:43 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <0C3ADBC6-66EA-4316-9366-1814D27B3A87@anduin.net> References: <20100226141754.86ae5a3f.gerrit@pmp.uni-hannover.de> <20100226174021.8feadad9.gerrit@pmp.uni-hannover.de> <20100226224320.8c4259bf.gerrit@pmp.uni-hannover.de> <4B884757.9040001@digiware.nl> <20100227080220.ac6a2e4d.gerrit@pmp.uni-hannover.de> <4B892918.4080701@digiware.nl> <20100227202105.f31cbef7.gerrit@pmp.uni-hannover.de> <20100227193819.GA60576@icarus.home.lan> <20100227223801.15aa657b.gerrit@pmp.uni-hannover.de> To: =?iso-8859-1?Q?Gerrit_K=FChn?= X-Mailer: Apple Mail (2.1077) Cc: freebsd-fs@freebsd.org, stable@freebsd.org, Willem Jan Withagen , Jeremy Chadwick Subject: Re: mbuf leakage with nfs/zfs? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Feb 2010 21:40:47 -0000 On 27. feb. 2010, at 22.38, Gerrit K=FChn wrote: > On Sat, 27 Feb 2010 21:32:39 +0100 Eirik =D8verby = wrote > about Re: mbuf leakage with nfs/zfs?: >=20 > E> I've had a discussion with some folks on this for a while. I can = easily > E> reproduce this situation by mounting a FreeBSD ZFS filesystem via > E> NFS-UDP from an OpenBSD machine. Telling the OpenBSD machine to use = TCP > E> instead of UDP makes the problem go away. >=20 > So we see this problem with udp clients from OpenBSD and Linux. I have not had the opportunity to test with Linux or anything else. = Could try from Windows, but not sure I want to get my hands THAT dirty. > E> Other FreeBSD systems mounting the same share, either using UDP or = TCP, > E> does not cause the problem to show up. >=20 > As Daniel reported he saw the problem with FBSD 8-stable: Which = version > was the FBSD-client that worked for you with udp? 7.1, 7.2, 8.0-RCsomething and 8.0-RELEASE - no problems with either. > E> A patch was suggested by Rick Macklem, but that did not solve the = issue: > E> = http://lists.freebsd.org/pipermail/freebsd-current/2009-December/014181.ht= ml >=20 > Yeah, I also found and tried this on Friday - unfortunately without = any > success, the leakage is still there. >=20 >=20 > cu > Gerrit >=20 From owner-freebsd-fs@FreeBSD.ORG Sat Feb 27 21:43:37 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A47A0106566C; Sat, 27 Feb 2010 21:43:37 +0000 (UTC) (envelope-from gerrit@pmp.uni-hannover.de) Received: from mrelay1.uni-hannover.de (mrelay1.uni-hannover.de [130.75.2.106]) by mx1.freebsd.org (Postfix) with ESMTP id 29BD48FC12; Sat, 27 Feb 2010 21:43:36 +0000 (UTC) Received: from www.pmp.uni-hannover.de (www.pmp.uni-hannover.de [130.75.117.2]) by mrelay1.uni-hannover.de (8.14.2/8.14.2) with ESMTP id o1RLhXwC031167; Sat, 27 Feb 2010 22:43:34 +0100 Received: from pmp.uni-hannover.de (theq.pmp.uni-hannover.de [130.75.117.4]) by www.pmp.uni-hannover.de (Postfix) with SMTP id 9F56324; Sat, 27 Feb 2010 22:43:33 +0100 (CET) Date: Sat, 27 Feb 2010 22:43:33 +0100 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: Jeremy Chadwick Message-Id: <20100227224333.5bee38ac.gerrit@pmp.uni-hannover.de> In-Reply-To: <20100227193819.GA60576@icarus.home.lan> References: <20100226141754.86ae5a3f.gerrit@pmp.uni-hannover.de> <20100226174021.8feadad9.gerrit@pmp.uni-hannover.de> <20100226224320.8c4259bf.gerrit@pmp.uni-hannover.de> <4B884757.9040001@digiware.nl> <20100227080220.ac6a2e4d.gerrit@pmp.uni-hannover.de> <4B892918.4080701@digiware.nl> <20100227202105.f31cbef7.gerrit@pmp.uni-hannover.de> <20100227193819.GA60576@icarus.home.lan> Organization: Albert-Einstein-Institut (MPI =?ISO-8859-1?Q?f=FCr?= Gravitationsphysik & IGP =?ISO-8859-1?Q?Universit=E4t?= Hannover) X-Mailer: Sylpheed 2.4.2 (GTK+ 2.10.12; i386-portbld-freebsd6.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-PMX-Version: 5.5.9.388399, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2010.2.27.213052 Cc: freebsd-fs@freebsd.org, stable@freebsd.org, Willem Jan Withagen Subject: Re: mbuf leakage with nfs/zfs? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Feb 2010 21:43:37 -0000 On Sat, 27 Feb 2010 11:38:19 -0800 Jeremy Chadwick wrote about Re: mbuf leakage with nfs/zfs?: JC> I should point out that the NFS+ZFS-based filer doesn't actually do its JC> backups using NFS; it uses rsnapshot (rsync) over SSH. There is JC> intense network I/O during backup time though, depending on how much JC> data there is to back up. The NFS mounts (on the clients) are only JC> used to provide a way for people to get access to their nightly JC> backups in a convenient way; it isn't used very heavily. That's rather similar to my situation, I would say. Most traffic goes via rsync, nfs only gives access to home dirs, which are not intensively used. JC> I can do something NFS-intensive on any of the above clients if people JC> want me to kind of testing. Possibly an rsync with a source of the NFS JC> mount and a destination of the local disk would be a good test? Let me JC> know if anyone's interested in me testing that. >From the last emails I would say we get most out of it by comparing tcp and udp clients to make sure this happens only with udp (and it is still not quite clear to me if it also happens with a FBSD client using udp). OTOH it would be great if someone with the ability to actually fix something in the nfs code could get in this discussion to guide us to do the debugging needed to do so. cu Gerrit From owner-freebsd-fs@FreeBSD.ORG Sat Feb 27 21:47:00 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 62C18106566C; Sat, 27 Feb 2010 21:47:00 +0000 (UTC) (envelope-from gerrit@pmp.uni-hannover.de) Received: from mrelay1.uni-hannover.de (mrelay1.uni-hannover.de [130.75.2.106]) by mx1.freebsd.org (Postfix) with ESMTP id D997D8FC22; Sat, 27 Feb 2010 21:46:59 +0000 (UTC) Received: from www.pmp.uni-hannover.de (www.pmp.uni-hannover.de [130.75.117.2]) by mrelay1.uni-hannover.de (8.14.2/8.14.2) with ESMTP id o1RLkqai031235; Sat, 27 Feb 2010 22:46:53 +0100 Received: from pmp.uni-hannover.de (theq.pmp.uni-hannover.de [130.75.117.4]) by www.pmp.uni-hannover.de (Postfix) with SMTP id 6185524; Sat, 27 Feb 2010 22:46:52 +0100 (CET) Date: Sat, 27 Feb 2010 22:46:51 +0100 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: Eirik =?ISO-8859-1?Q?=D8verby?= Message-Id: <20100227224651.d2102fca.gerrit@pmp.uni-hannover.de> In-Reply-To: <0C3ADBC6-66EA-4316-9366-1814D27B3A87@anduin.net> References: <20100226141754.86ae5a3f.gerrit@pmp.uni-hannover.de> <20100226174021.8feadad9.gerrit@pmp.uni-hannover.de> <20100226224320.8c4259bf.gerrit@pmp.uni-hannover.de> <4B884757.9040001@digiware.nl> <20100227080220.ac6a2e4d.gerrit@pmp.uni-hannover.de> <4B892918.4080701@digiware.nl> <20100227202105.f31cbef7.gerrit@pmp.uni-hannover.de> <20100227193819.GA60576@icarus.home.lan> <20100227223801.15aa657b.gerrit@pmp.uni-hannover.de> <0C3ADBC6-66EA-4316-9366-1814D27B3A87@anduin.net> Organization: Albert-Einstein-Institut (MPI =?ISO-8859-1?Q?f=FCr?= Gravitationsphysik & IGP =?ISO-8859-1?Q?Universit=E4t?= Hannover) X-Mailer: Sylpheed 2.4.2 (GTK+ 2.10.12; i386-portbld-freebsd6.1) Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-PMX-Version: 5.5.9.388399, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2010.2.27.213325 Cc: freebsd-fs@freebsd.org, stable@freebsd.org, Willem Jan Withagen , Jeremy Chadwick Subject: Re: mbuf leakage with nfs/zfs? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Feb 2010 21:47:00 -0000 On Sat, 27 Feb 2010 22:40:43 +0100 Eirik =D8verby wrote about Re: mbuf leakage with nfs/zfs?: E> > So we see this problem with udp clients from OpenBSD and Linux. E> I have not had the opportunity to test with Linux or anything else. I guess all others who reported so far (including me) had Linux on the client side. E> Could try from Windows, but not sure I want to get my hands THAT dirty. :-))) E> > As Daniel reported he saw the problem with FBSD 8-stable: Which E> > version was the FBSD-client that worked for you with udp? E> 7.1, 7.2, 8.0-RCsomething and 8.0-RELEASE - no problems with either. Daniel, are you sure you had the leakage with 8-stable? Eirik, do you have the opportunity to try 8-stable with udp? cu Gerrit From owner-freebsd-fs@FreeBSD.ORG Sat Feb 27 21:53:03 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 327D81065670; Sat, 27 Feb 2010 21:53:03 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from mail.digiware.nl (mail.ip6.digiware.nl [IPv6:2001:4cb8:1:106::2]) by mx1.freebsd.org (Postfix) with ESMTP id B68EB8FC1A; Sat, 27 Feb 2010 21:53:02 +0000 (UTC) Received: from localhost (localhost.digiware.nl [127.0.0.1]) by mail.digiware.nl (Postfix) with ESMTP id E7817153433; Sat, 27 Feb 2010 22:53:00 +0100 (CET) X-Virus-Scanned: amavisd-new at digiware.nl Received: from mail.digiware.nl ([127.0.0.1]) by localhost (rack1.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id m54sQjbHQmCK; Sat, 27 Feb 2010 22:52:59 +0100 (CET) Received: from [127.0.0.1] (unknown [192.168.10.212]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.digiware.nl (Postfix) with ESMTPSA id 024BF15342F; Sat, 27 Feb 2010 22:52:59 +0100 (CET) Message-ID: <4B89943C.70704@digiware.nl> Date: Sat, 27 Feb 2010 22:53:00 +0100 From: Willem Jan Withagen User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.8) Gecko/20100216 Thunderbird/3.0.2 MIME-Version: 1.0 To: =?ISO-8859-1?Q?Eirik_=D8verby?= References: <20100226141754.86ae5a3f.gerrit@pmp.uni-hannover.de> <20100226174021.8feadad9.gerrit@pmp.uni-hannover.de> <20100226224320.8c4259bf.gerrit@pmp.uni-hannover.de> <4B884757.9040001@digiware.nl> <20100227080220.ac6a2e4d.gerrit@pmp.uni-hannover.de> <4B892918.4080701@digiware.nl> <20100227202105.f31cbef7.gerrit@pmp.uni-hannover.de> <20100227193819.GA60576@icarus.home.lan> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-fs@freebsd.org, stable@freebsd.org, Jeremy Chadwick , =?ISO-8859-1?Q?Gerrit_K?= Subject: Re: mbuf leakage with nfs/zfs? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Feb 2010 21:53:03 -0000 On 27-2-2010 21:32, Eirik Øverby wrote: > I've had a discussion with some folks on this for a while. I can easily > reproduce this situation by mounting a FreeBSD ZFS filesystem via > NFS-UDP from an OpenBSD machine. Telling the OpenBSD machine to use TCP > instead of UDP makes the problem go away. > > Other FreeBSD systems mounting the same share, either using UDP or TCP, > does not cause the problem to show up. > > A patch was suggested by Rick Macklem, but that did not solve the issue: > http://lists.freebsd.org/pipermail/freebsd-current/2009-December/014181.html I concur. Everything in my network is now on TCP, and there is no mbuf leakage. I just don't get over the 5500 mark, no matter what I throw at it. I do feel that TCP is not as well performing on a local net with Linux, hence the choice for UDP. But TCP is workable as next best. --WjW From owner-freebsd-fs@FreeBSD.ORG Sat Feb 27 22:16:23 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5923D1065678 for ; Sat, 27 Feb 2010 22:16:23 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta02.emeryville.ca.mail.comcast.net (qmta02.emeryville.ca.mail.comcast.net [76.96.30.24]) by mx1.freebsd.org (Postfix) with ESMTP id 376438FC2A for ; Sat, 27 Feb 2010 22:16:22 +0000 (UTC) Received: from omta08.emeryville.ca.mail.comcast.net ([76.96.30.12]) by qmta02.emeryville.ca.mail.comcast.net with comcast id n6Nh1d0030FhH24A2A3DJ4; Sat, 27 Feb 2010 22:03:13 +0000 Received: from koitsu.dyndns.org ([98.248.46.159]) by omta08.emeryville.ca.mail.comcast.net with comcast id nA3B1d00H3S48mS8UA3CbN; Sat, 27 Feb 2010 22:03:12 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 992AE1E301A; Sat, 27 Feb 2010 14:03:10 -0800 (PST) Date: Sat, 27 Feb 2010 14:03:10 -0800 From: Jeremy Chadwick To: rwatson@freebsd.org Message-ID: <20100227220310.GA65110@icarus.home.lan> References: <20100226174021.8feadad9.gerrit@pmp.uni-hannover.de> <20100226224320.8c4259bf.gerrit@pmp.uni-hannover.de> <4B884757.9040001@digiware.nl> <20100227080220.ac6a2e4d.gerrit@pmp.uni-hannover.de> <4B892918.4080701@digiware.nl> <20100227202105.f31cbef7.gerrit@pmp.uni-hannover.de> <20100227193819.GA60576@icarus.home.lan> <4B89943C.70704@digiware.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <4B89943C.70704@digiware.nl> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-fs@freebsd.org, stable@freebsd.org, Willem Jan Withagen , Eirik =?iso-8859-1?Q?=D8verby?= Subject: Re: mbuf leakage with nfs/zfs? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 27 Feb 2010 22:16:23 -0000 On Sat, Feb 27, 2010 at 10:53:00PM +0100, Willem Jan Withagen wrote: > On 27-2-2010 21:32, Eirik Øverby wrote: > >I've had a discussion with some folks on this for a while. I can easily > >reproduce this situation by mounting a FreeBSD ZFS filesystem via > >NFS-UDP from an OpenBSD machine. Telling the OpenBSD machine to use TCP > >instead of UDP makes the problem go away. > > > >Other FreeBSD systems mounting the same share, either using UDP or TCP, > >does not cause the problem to show up. > > > >A patch was suggested by Rick Macklem, but that did not solve the issue: > >http://lists.freebsd.org/pipermail/freebsd-current/2009-December/014181.html > > I concur. > Everything in my network is now on TCP, and there is no mbuf leakage. > I just don't get over the 5500 mark, no matter what I throw at it. > > I do feel that TCP is not as well performing on a local net with Linux, > hence the choice for UDP. But TCP is workable as next best. I'm pulling in Robert Watson, who has some familiarity with the UDP stack/code in FreeBSD. I'm not sure he'll be a sufficient source of knowledge for this specific issue since it appears (?) to be specific to NFS; Rick Macklem would be a better choice, but as reported, he's MIA. Robert, are you aware of any changes or implementation issues which might cause excessive (read: leaking) mbuf use under UDP-based NFS? Do you know of a way folks could determine the source of the leak, either via DDB or while the system is live? -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |