From owner-freebsd-stable@FreeBSD.ORG Sun Jan 2 04:06:11 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4C9211065672; Sun, 2 Jan 2011 04:06:11 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-gw0-f54.google.com (mail-gw0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id D05918FC12; Sun, 2 Jan 2011 04:06:10 +0000 (UTC) Received: by gwj21 with SMTP id 21so6325420gwj.13 for ; Sat, 01 Jan 2011 20:06:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :organization:user-agent:mime-version:to:cc:subject:references :in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; bh=cS/qlat1BQPQl//iEgsph7b2SgKxnSWlpSv2xQVxfyQ=; b=I8NUKNJsmJp12k76oNeFviG/gU6J4KRWlS+8Y6Hdi1uioE25tPz77JLM1kJ6VM6tSa bSBJVyGTIQ1ecHXNHPP0HF7sdP0ZdAjD0M4Imfl7P1edLcU4jVRSu3Yh3zbbqZY2k1xi EC+8yGzBFrxyYdQXLcQiC0UgnAr0yyNwm8Wo0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:organization:user-agent:mime-version:to :cc:subject:references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=VsFLyvuZXt0wJPvKTFIdX615Uu7NTPDH3DbyfuddNswOgG0asrZR6qxISTrOtSbexX S+tfUNnsUGan328UKcjGxXfQ/4ZVc4dhO+HFR4OVawbrpJ8fFOjG5iiPC/OBe8PsyMmA QMWKEO/9s4M3uSYTlLEKb5cu6Yhcov1XsCqII= Received: by 10.236.105.240 with SMTP id k76mr9825063yhg.96.1293941168757; Sat, 01 Jan 2011 20:06:08 -0800 (PST) Received: from centel.dataix.local ([99.181.155.97]) by mx.google.com with ESMTPS id n21sm11253631yha.45.2011.01.01.20.06.06 (version=SSLv3 cipher=RC4-MD5); Sat, 01 Jan 2011 20:06:06 -0800 (PST) Sender: "J. Hellenthal" Message-ID: <4D1FF9AC.60200@DataIX.net> Date: Sat, 01 Jan 2011 23:06:04 -0500 From: "J. Hellenthal" Organization: http://www.DataIX.net User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.13) Gecko/20101230 Lightning/1.0b1 Thunderbird MIME-Version: 1.0 To: Attila Nagy References: <4D0A09AF.3040005@FreeBSD.org> <4D1F7008.3050506@fsn.hu> In-Reply-To: <4D1F7008.3050506@fsn.hu> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, freebsd-stable@freebsd.org, Martin Matuska Subject: Re: New ZFSv28 patchset for 8-STABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Jan 2011 04:06:11 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 01/01/2011 13:18, Attila Nagy wrote: > On 12/16/2010 01:44 PM, Martin Matuska wrote: >> Link to the patch: >> >> http://people.freebsd.org/~mm/patches/zfs/v28/stable-8-zfsv28-20101215.patch.xz >> >> >> > I've used this: > http://people.freebsd.org/~mm/patches/zfs/v28/stable-8-zfsv28-20101223-nopython.patch.xz > > on a server with amd64, 8 G RAM, acting as a file server on > ftp/http/rsync, the content being read only mounted with nullfs in > jails, and the daemons use sendfile (ftp and http). > > The effects can be seen here: > http://people.fsn.hu/~bra/freebsd/20110101-zfsv28-fbsd/ > the exact moment of the switch can be seen on zfs_mem-week.png, where > the L2 ARC has been discarded. > > What I see: > - increased CPU load > - decreased L2 ARC hit rate, decreased SSD (ad[46]), therefore increased > hard disk load (IOPS graph) > > Maybe I could accept the higher system load as normal, because there > were a lot of things changed between v15 and v28 (but I was hoping if I > use the same feature set, it will require less CPU), but dropping the > L2ARC hit rate so radically seems to be a major issue somewhere. > As you can see from the memory stats, I have enough kernel memory to > hold the L2 headers, so the L2 devices got filled up to their maximum > capacity. > > Any ideas on what could cause these? I haven't upgraded the pool version > and nothing was changed in the pool or in the file system. > Running arc_summary.pl[1] -p4 should print a summary about your l2arc and you should also notice in that section that there is a high number of "SPA Mismatch" mine usually grew to around 172k before I would notice a crash and I could reliably trigger this while in scrub. What ever is causing this needs desperate attention! I emailed mm@ privately off-list when I noticed this going on but have not received any feedback as of yet. [1] http://bit.ly/fdRiYT - -- Regards, jhell,v JJH48-ARIN -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJNH/msAAoJEJBXh4mJ2FR+bFYH/0bBJbLYU5zzbqpUUXX1M/B9 +g8RwQ9Tek4/fxwpD8DNIfkpzO0MvUcx5Nhwld69jk7sSys9IUpYhuYVggcOgavx sl6AwNNUG0XD/spO2RvV3jD4tVbR6TjlSdLCyBG7iPFU2nNB6wZM+UfWxGYwEyUE loOr13Vk4eU2l2cepUwJH0oGu2hsDZ7qR/fTd+d33NfS6/PT43vbCjPNTsnDJeY9 MdeC5vBUPl3AW3iC/5hxBi9WABGMHeAXTolpAtBQVBNi22mINacYFO6FEdfANy9E Xo207Cd6vBmZb8aTs0BFHs5ZdTHUco/iNysaWvzx9TlIWlyyBRgOXgtBweB+6d4= =lcxW -----END PGP SIGNATURE----- From owner-freebsd-stable@FreeBSD.ORG Sun Jan 2 08:41:05 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F2E7A106566B for ; Sun, 2 Jan 2011 08:41:05 +0000 (UTC) (envelope-from miyamoto.31b@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 8C53A8FC17 for ; Sun, 2 Jan 2011 08:41:05 +0000 (UTC) Received: by wyf19 with SMTP id 19so12512223wyf.13 for ; Sun, 02 Jan 2011 00:41:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=89GJfG/7sEzQ5AwoAstgxILguDU9HCkI3ymr/X0lD7c=; b=VHpapk0g1YsXrhOayp9+k9sLTtkj7ZRR6I7+FIgDtla7PUkHDJ+ayv+Bm4Pp3rvoXp P6w+ZLndBytAJc2aN+Zv0LvEzVWJvip5hmXp7Fl+ZEeDjqipbKP5DuaoSGHBYoHGHs2J DiHPT8cUZpgzM05gXxJNt6iwiN79BpcRQhgy8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=Duhwlqu2enJBIobmcphekiKyQf0qmEewkUAdvsvDly2wZZkLoGugn8bh4EET1fzLa0 H1aScCUfJiUWO8IuEHBpjk+UMPKhEGdEVePxRrGl+zk1ALjpge7cSelmvQxOlJRYfhvn +UaILPktRkc56KuFrmbELy/v6gStGUaLZPY+w= MIME-Version: 1.0 Received: by 10.227.144.13 with SMTP id x13mr10853402wbu.69.1293957664357; Sun, 02 Jan 2011 00:41:04 -0800 (PST) Received: by 10.227.13.143 with HTTP; Sun, 2 Jan 2011 00:41:04 -0800 (PST) Date: Sun, 2 Jan 2011 08:41:04 +0000 Message-ID: From: miyamoto moesasji To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: Re: tmpfs runs out of space on 8.2pre-release, zfs related? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Jan 2011 08:41:06 -0000 miyamoto moesasji gmail.com> writes: > > In setting up tmpfs (so not tmpmfs) on a machine that is using > zfs(v15, zfs v4) on 8.2prerelease I run out of space on the tmpfs when > copying a file of ~4.6 GB file from the zfs-filesystem to the memory > disk. This machine has 8GB of memory backed by swap on the harddisk, > so I expected the file to copy to memory without problems. > ....this is in fact worse than I first thought. After leaving the machine running overnight the tmpfs is reduced to a size of 4K, which shows that tmpfs is in fact completely unusable for me. See the output of df: --- hge@PulsarX4:~/ > df -hi /tmp Filesystem Size Used Avail Capacity iused ifree %iused Mounted on tmpfs 4.0K 4.0K 0B 100% 18 0 100% /tmp --- Relevant zfs-stats info: --- System Memory Statistics: Physical Memory: 8161.74M Kernel Memory: 4117.40M DATA: 99.29% 4088.07M TEXT: 0.71% 29.33M ARC Size: Current Size (arcsize): 63.58% 4370.60M Target Size (Adaptive, c): 100.00% 6874.44M Min Size (Hard Limit, c_min): 12.50% 859.31M Max Size (High Water, c_max): ~8:1 6874.44M --- I'm not sure what triggered this further reduction in size; but the above 4K size is probably important to show how dramatic this goes wrong. From owner-freebsd-stable@FreeBSD.ORG Sun Jan 2 08:46:00 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6354F106566B; Sun, 2 Jan 2011 08:46:00 +0000 (UTC) (envelope-from bra@fsn.hu) Received: from people.fsn.hu (people.fsn.hu [195.228.252.137]) by mx1.freebsd.org (Postfix) with ESMTP id 599718FC14; Sun, 2 Jan 2011 08:45:58 +0000 (UTC) Received: by people.fsn.hu (Postfix, from userid 1001) id 220416B4901; Sun, 2 Jan 2011 09:45:57 +0100 (CET) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MF-ACE0E1EA [pR: 19.9304] X-CRM114-CacheID: sfid-20110102_09455_46FAFBEF X-CRM114-Status: Good ( pR: 19.9304 ) X-Spambayes-Classification: ham; 0.00 Message-ID: <4D203B43.7070607@fsn.hu> Date: Sun, 02 Jan 2011 09:45:55 +0100 From: Attila Nagy User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.23) Gecko/20090817 Thunderbird/2.0.0.23 Mnenhy/0.7.6.0 MIME-Version: 1.0 To: "J. Hellenthal" References: <4D0A09AF.3040005@FreeBSD.org> <4D1F7008.3050506@fsn.hu> <4D1FF9AC.60200@DataIX.net> In-Reply-To: <4D1FF9AC.60200@DataIX.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, freebsd-stable@freebsd.org, Martin Matuska Subject: Re: New ZFSv28 patchset for 8-STABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Jan 2011 08:46:00 -0000 On 01/02/2011 05:06 AM, J. Hellenthal wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 01/01/2011 13:18, Attila Nagy wrote: >> On 12/16/2010 01:44 PM, Martin Matuska wrote: >>> Link to the patch: >>> >>> http://people.freebsd.org/~mm/patches/zfs/v28/stable-8-zfsv28-20101215.patch.xz >>> >>> >>> >> I've used this: >> http://people.freebsd.org/~mm/patches/zfs/v28/stable-8-zfsv28-20101223-nopython.patch.xz >> >> on a server with amd64, 8 G RAM, acting as a file server on >> ftp/http/rsync, the content being read only mounted with nullfs in >> jails, and the daemons use sendfile (ftp and http). >> >> The effects can be seen here: >> http://people.fsn.hu/~bra/freebsd/20110101-zfsv28-fbsd/ >> the exact moment of the switch can be seen on zfs_mem-week.png, where >> the L2 ARC has been discarded. >> >> What I see: >> - increased CPU load >> - decreased L2 ARC hit rate, decreased SSD (ad[46]), therefore increased >> hard disk load (IOPS graph) >> >> Maybe I could accept the higher system load as normal, because there >> were a lot of things changed between v15 and v28 (but I was hoping if I >> use the same feature set, it will require less CPU), but dropping the >> L2ARC hit rate so radically seems to be a major issue somewhere. >> As you can see from the memory stats, I have enough kernel memory to >> hold the L2 headers, so the L2 devices got filled up to their maximum >> capacity. >> >> Any ideas on what could cause these? I haven't upgraded the pool version >> and nothing was changed in the pool or in the file system. >> > Running arc_summary.pl[1] -p4 should print a summary about your l2arc > and you should also notice in that section that there is a high number > of "SPA Mismatch" mine usually grew to around 172k before I would notice > a crash and I could reliably trigger this while in scrub. > > What ever is causing this needs desperate attention! > > I emailed mm@ privately off-list when I noticed this going on but have > not received any feedback as of yet. It's at zero currently (2 days of uptime): kstat.zfs.misc.arcstats.l2_write_spa_mismatch: 0 From owner-freebsd-stable@FreeBSD.ORG Sun Jan 2 09:18:07 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 664541065670 for ; Sun, 2 Jan 2011 09:18:07 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.16.84]) by mx1.freebsd.org (Postfix) with ESMTP id 1FCDA8FC15 for ; Sun, 2 Jan 2011 09:18:06 +0000 (UTC) Received: from pampa.cs.huji.ac.il ([132.65.80.32]) by kabab.cs.huji.ac.il with esmtp id 1PZK4i-000HSP-IX; Sun, 02 Jan 2011 11:18:04 +0200 X-Mailer: exmh version 2.7.2 01/07/2005 with nmh-1.2 To: Peter Jeremy In-reply-to: <20110101124149.GE48579@server.vk2pj.dyndns.org> References: <20110101124149.GE48579@server.vk2pj.dyndns.org> Comments: In-reply-to Peter Jeremy message dated "Sat, 01 Jan 2011 23:41:49 +1100." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sun, 02 Jan 2011 11:18:04 +0200 From: Daniel Braniss Message-ID: Cc: freebsd-stable@freebsd.org Subject: Re: Specifying root mount options on diskless boot. X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Jan 2011 09:18:07 -0000 > > --2iBwrppp/7QCDedR > Content-Type: text/plain; charset=us-ascii > Content-Disposition: inline > Content-Transfer-Encoding: quoted-printable > > [I'm not sure if -stable is the best list for this but anyway...] > > I'm trying to convert an old laptop running FreeBSD 8.0 into a diskless > client (since its internal HDD is growing bad spots faster than I can > repair them). I have it pxebooting nicely and running with an NFS root > but it then reports locking problems: devd, syslogd, moused (and maybe > others) lock their PID file to protect against multiple instances. > Unfortunately, these daemons all start before statd/lockd and so the > locking fails and reports "operation not supported". > > It's not practical to reorder the startup sequence to make lockd start > early enough (I've tried). > > Since the filesystem is reserved for this client, there's no real need > to forward lock requests across the wire and so specifying "nolockd" > would be another solution. Looking through sys/nfsclient/bootp_subr.c, > DHCP option 130 should allow NFS mount options to be specified (though > it's not clear that the relevant code path is actually followed because > I don't see the associated printf()s anywhere on the console. After > getting isc-dhcpd to forward this option (made more difficult because > its documentation is incorrect), it still doesn't work. > > Understanding all this isn't helped by kenv(8) reporting three different > sets of root filesystem options: > boot.nfsroot.path=3D"/tank/m3" > boot.nfsroot.server=3D"192.168.123.200" > dhcp.option-130=3D"nolockd" > dhcp.root-path=3D"192.168.123.200:/tank/m3" > vfs.root.mountfrom=3D"nfs:server:/tank/m3" > vfs.root.mountfrom.options=3D"rw,tcp,nolockd" > > And the console also reports conflicting root definitions: > Trying to mount root from nfs:server:/tank/m3 > NFS ROOT: 192.168.123.200:/tank/m3 > > Working through all these: > boot.nfsroot.* appears to be initialised by sys/boot/i386/libi386/pxe.c > but, whilst nfsclient/nfs_diskless.c can parse boot.nfsroot.options, > there's no code to initialise that kenv name in pxe.c > > dhcp.* appears to be initialised by lib/libstand/bootp.c - which does > include code to populate boot.nfsroot.options (using vendor specific > DHCP option 20) but this code is not compiled in. Further studying > of bootp.c shows that it's possible to initialise arbitrary kenv's > using DHCP options 246-254 - but the DHCPDISCOVER packets do not > request these options so they don't work without special DHCP server > configuration (to forward options that aren't requested). > > vfs.root.* is parsed out of /etc/fstab but, other than being > reported in the console message above, it doesn't appear to be > used in this environment (it looks like the root entry can be > commented out of /etc/fstab without problem). > > My final solution was to specify 'boot.nfsroot.options=3D"nolockd"' in > loader.conf - and this seems to actually work. > > It seems rather unfortunate that FreeBSD has code to allow NFS root > mount options to be specified via DHCP (admittedly in several > incompatible ways) but none actually work. A quick look at -current > suggests that the situation there remains equally broken. > > Has anyone else tried to use any of this? And would anyone be interested > in trying to make it actually work? Hi Peter, i have beed doing diskless booting for a long time, and am very pleased (though 8.2-prerelease is causing some problems :-). In my case /var is mfs, or ufs/zfs, and have no lockd problems. here is what you need to do: either change in libstand/bootp.c: #define DHCP_ENV DHCP_ENV_NO_VENDOR to #define DHCP_ENV DHCP_ENV_FREEBSD or pick my version from: ftp://ftp.cs.huji.ac.il/users/danny/freebsd/diskless-boot/ and compile a new pxeboot. this new pxeboot will allow you to pass via dhcp some key options. next, take a look at ftp://ftp.cs.huji.ac.il/users/danny/freebsd/diskless-boot/rc.initdiskless make sure that your exported root has /.etc If you'r /var is also nfs mounted, maybe unionfs might help too. just writing quickly so you won't feel discouraged, and that diskless actually works. danny From owner-freebsd-stable@FreeBSD.ORG Sun Jan 2 11:33:51 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E0696106564A for ; Sun, 2 Jan 2011 11:33:51 +0000 (UTC) (envelope-from dan@langille.org) Received: from nyi.unixathome.org (nyi.unixathome.org [64.147.113.42]) by mx1.freebsd.org (Postfix) with ESMTP id AB8EF8FC1E for ; Sun, 2 Jan 2011 11:33:51 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by nyi.unixathome.org (Postfix) with ESMTP id DE24B508AC; Sun, 2 Jan 2011 11:33:50 +0000 (GMT) X-Virus-Scanned: amavisd-new at unixathome.org Received: from nyi.unixathome.org ([127.0.0.1]) by localhost (nyi.unixathome.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pb+2-r9WgD0Y; Sun, 2 Jan 2011 11:33:50 +0000 (GMT) Received: from smtp-auth.unixathome.org (smtp-auth.unixathome.org [10.4.7.7]) (Authenticated sender: hidden) by nyi.unixathome.org (Postfix) with ESMTPSA id 94BBA50850 ; Sun, 2 Jan 2011 11:33:50 +0000 (GMT) Message-ID: <4D20629D.2030803@langille.org> Date: Sun, 02 Jan 2011 06:33:49 -0500 From: Dan Langille Organization: The FreeBSD Diary User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: Jeremy Chadwick References: <4D1AF388.3080107@infracaninophile.co.uk> <4D1B7431.7070808@infracaninophile.co.uk> <4D1BD8D0.5010402@langille.org> <4D1C4A2D.4020206@infracaninophile.co.uk> <4D1C7929.3040809@langille.org> <20101231233343.GB48579@server.vk2pj.dyndns.org> <20101231234747.GA8171@icarus.home.lan> In-Reply-To: <20101231234747.GA8171@icarus.home.lan> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org, Peter Jeremy Subject: Re: slow ZFS on FreeBSD 8.1 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Jan 2011 11:33:52 -0000 On 12/31/2010 6:47 PM, Jeremy Chadwick wrote: > On Sat, Jan 01, 2011 at 10:33:43AM +1100, Peter Jeremy wrote: >> Based on my experiences at home, I converted my desktop at work to >> pure ZFS. The only issues I've run into have been programs that >> extensively use mmap(2) - which is a known issue with ZFS. > > Is your ZFS root filesystem associated with a pool that's mirrored or > using raidzX? What about mismatched /boot content (ZFS vs. UFS)? What > about booting into single-user mode? > > http://wiki.freebsd.org/ZFSOnRoot indirectly hints at these problems but > doesn't outright admit them (yet should), so I'm curious to know how > people have solved them. Remembering manual "one-offs" for a system > configured this way is not acceptable (read: highly prone to > error/mistake). Is it worth the risk? Most administrators don't have > the tolerance for stuff like that in the middle of a system upgrade or > what not; they should be able to follow exactly what's in the handbook, > to a tee. > > There's a link to www.dan.me.uk at the bottom of the above Wiki page > that outlines "the madness" that's required to configure the setup, all > of which has to be done by hand. I don't know many administrators who > are going to tolerate this when deploying numerous machines, especially > when compounded by the complexities mentioned above. This basically outlines the reason why I do not use ZFS on root. -- Dan Langille - http://langille.org/ From owner-freebsd-stable@FreeBSD.ORG Sun Jan 2 12:24:30 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 808FE106566B for ; Sun, 2 Jan 2011 12:24:30 +0000 (UTC) (envelope-from bruce@cran.org.uk) Received: from muon.cran.org.uk (muon.cran.org.uk [IPv6:2a01:348:0:15:5d59:5c40:0:1]) by mx1.freebsd.org (Postfix) with ESMTP id 0BBD18FC0A for ; Sun, 2 Jan 2011 12:24:30 +0000 (UTC) Received: from muon.cran.org.uk (localhost [127.0.0.1]) by muon.cran.org.uk (Postfix) with ESMTP id 1EEBBE60EA; Sun, 2 Jan 2011 12:24:29 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=cran.org.uk; h=date:from :to:cc:subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; s=mail; bh=av1eOr3z1HEY J/yagkBxXF39LLw=; b=lF3T77sg/cyBNss4bdK9wDLZV5b408U9PZ7j07gZU/Hd duXf8pT1mnyJvVHw+549OMUS3YJUTYqbFu3mZTlbYpu8mT/3eJvXQKaO9QPPLxPM V4DHLp2b5uuBgGFUxAHhiTlQTlLxRW0iXFJQAa5BqInrKBfZNG7SMRpZPQJm4hI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=cran.org.uk; h=date:from:to :cc:subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; q=dns; s=mail; b=nOm+YZ bFRYM+EaUV4diO3kYL3ClqHRxUyuF5/znYO+0fQ8r2xiAOjjHwwWw1fgoCc9wPwz WDbv71HawCy84CGWdXpWuLELYUa6S31m7oDtCjYaIqxr4XE05dNzzDxjMvep5+tU FpIE/9KjkiL2xxw4PYvQfnRGLpVtZ3wFbAR8k= Received: from unknown (client-86-27-23-77.glfd.adsl.virginmedia.com [86.27.23.77]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by muon.cran.org.uk (Postfix) with ESMTPSA id CC688E60E8; Sun, 2 Jan 2011 12:24:28 +0000 (GMT) Date: Sun, 2 Jan 2011 12:24:26 +0000 From: Bruce Cran To: Jeremy Chadwick Message-ID: <20110102122426.0000486b@unknown> In-Reply-To: <20101231234747.GA8171@icarus.home.lan> References: <4D1AF388.3080107@infracaninophile.co.uk> <4D1B7431.7070808@infracaninophile.co.uk> <4D1BD8D0.5010402@langille.org> <4D1C4A2D.4020206@infracaninophile.co.uk> <4D1C7929.3040809@langille.org> <20101231233343.GB48579@server.vk2pj.dyndns.org> <20101231234747.GA8171@icarus.home.lan> X-Mailer: Claws Mail 3.7.6 (GTK+ 2.16.0; i586-pc-mingw32msvc) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: Dan, freebsd-stable@freebsd.org, Langille , Peter Jeremy Subject: Re: slow ZFS on FreeBSD 8.1 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Jan 2011 12:24:30 -0000 On Fri, 31 Dec 2010 15:47:47 -0800 Jeremy Chadwick wrote: > There's a link to www.dan.me.uk at the bottom of the above Wiki page > that outlines "the madness" that's required to configure the setup, > all of which has to be done by hand. I don't know many > administrators who are going to tolerate this when deploying numerous > machines, especially when compounded by the complexities mentioned > above. All of that page could be summarized as: mkdir /usb mount /dev/da4 /usb /usb/install_zfs.sh Where da4 is a USB drive and install_zfs.sh is essentially the commands with some changes needed to support different disks. I'd imagine that administrators wouldn't use sysinstall in interactive mode when deploying to many machines. -- Bruce Cran From owner-freebsd-stable@FreeBSD.ORG Sun Jan 2 14:31:53 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 904B4106566C for ; Sun, 2 Jan 2011 14:31:53 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 2CAD98FC0C for ; Sun, 2 Jan 2011 14:31:52 +0000 (UTC) Received: by wwf26 with SMTP id 26so12554759wwf.31 for ; Sun, 02 Jan 2011 06:31:52 -0800 (PST) Received: by 10.227.132.209 with SMTP id c17mr11122884wbt.135.1293978712062; Sun, 02 Jan 2011 06:31:52 -0800 (PST) Received: from dfleuriot.local (did75-17-88-165-130-96.fbx.proxad.net [88.165.130.96]) by mx.google.com with ESMTPS id q18sm13353288wbe.17.2011.01.02.06.31.50 (version=SSLv3 cipher=RC4-MD5); Sun, 02 Jan 2011 06:31:50 -0800 (PST) Message-ID: <4D208C55.9080409@my.gd> Date: Sun, 02 Jan 2011 15:31:49 +0100 From: Damien Fleuriot User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-stable@freebsd.org References: <4D1C6F90.3080206@my.gd> <4D1F442F.8060801@my.gd> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Jan 2011 14:31:53 -0000 On 1/1/11 6:28 PM, Jean-Yves Avenard wrote: > On 2 January 2011 02:11, Damien Fleuriot wrote: > >> I remember getting rather average performance on v14 but Jean-Yves >> reported good performance boosts from upgrading to v15. > > that was v28 :) > > saw no major difference between v14 and v15. > > JY Oopsie :) Seeing I for one will have no backups, I think I won't be using v28 on this box, and stick with v15 instead. Are there any views regarding the "best" implementation for a system ? I currently have a ZFS only system but I'm planning on moving it to UFS, with ZFS used only for mass storage. I understand ZFS root is much trickier, and my main fear is that if I somehow break ZFS (by upgrading to v28 for example) I won't be able to boot anymore, thus no repair process... From owner-freebsd-stable@FreeBSD.ORG Sun Jan 2 15:24:52 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7035B106564A for ; Sun, 2 Jan 2011 15:24:52 +0000 (UTC) (envelope-from ronald-freebsd8@klop.yi.org) Received: from fep11.mx.upcmail.net (fep11.mx.upcmail.net [62.179.121.31]) by mx1.freebsd.org (Postfix) with ESMTP id A77ED8FC08 for ; Sun, 2 Jan 2011 15:24:51 +0000 (UTC) Received: from edge01.upcmail.net ([192.168.13.236]) by viefep11-int.chello.at (InterMail vM.8.01.02.02 201-2260-120-106-20100312) with ESMTP id <20110102152449.LYTM28175.viefep11-int.chello.at@edge01.upcmail.net> for ; Sun, 2 Jan 2011 16:24:49 +0100 Received: from pinky ([213.46.23.80]) by edge01.upcmail.net with edge id qfQo1f00Q1jgp3H01fQpu2; Sun, 02 Jan 2011 16:24:49 +0100 X-SourceIP: 213.46.23.80 Content-Type: text/plain; charset=us-ascii; format=flowed; delsp=yes To: freebsd-stable@freebsd.org References: <4D1C6F90.3080206@my.gd> <4D1F442F.8060801@my.gd> <4D208C55.9080409@my.gd> Date: Sun, 02 Jan 2011 16:24:48 +0100 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: "Ronald Klop" Message-ID: In-Reply-To: <4D208C55.9080409@my.gd> User-Agent: Opera Mail/11.00 (Win32) X-Cloudmark-Analysis: v=1.1 cv=kRmApA04uPli/d8B8u1M0swu98hronvs6VvAqI8uQP0= c=1 sm=0 a=tZDC3L0OECsA:10 a=bgpUlknNv7MA:10 a=kj9zAlcOel0A:10 a=hLuhQxTjI5YOYCVGivYA:9 a=7cfVa2-A4pEzs-RSc2k64njx2TUA:4 a=CjuIK1q_8ugA:10 a=iGhaP2GBsIPPIOcz:21 a=hVrLo3FZL7S-a0Rb:21 a=HpAAvcLHHh0Zw7uRqdWCyQ==:117 Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Jan 2011 15:24:52 -0000 On Sun, 02 Jan 2011 15:31:49 +0100, Damien Fleuriot wrote: > > > On 1/1/11 6:28 PM, Jean-Yves Avenard wrote: >> On 2 January 2011 02:11, Damien Fleuriot wrote: >> >>> I remember getting rather average performance on v14 but Jean-Yves >>> reported good performance boosts from upgrading to v15. >> >> that was v28 :) >> >> saw no major difference between v14 and v15. >> >> JY > > > Oopsie :) > > > Seeing I for one will have no backups, I think I won't be using v28 on > this box, and stick with v15 instead. > > > Are there any views regarding the "best" implementation for a system ? > > I currently have a ZFS only system but I'm planning on moving it to UFS, > with ZFS used only for mass storage. > > > I understand ZFS root is much trickier, and my main fear is that if I > somehow break ZFS (by upgrading to v28 for example) I won't be able to > boot anymore, thus no repair process... You can repair by booting from USB of CD in a lot of cases. Ronald. From owner-freebsd-stable@FreeBSD.ORG Sun Jan 2 15:25:31 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 680141065672 for ; Sun, 2 Jan 2011 15:25:31 +0000 (UTC) (envelope-from peterjeremy@acm.org) Received: from mail34.syd.optusnet.com.au (mail34.syd.optusnet.com.au [211.29.133.218]) by mx1.freebsd.org (Postfix) with ESMTP id C14E28FC14 for ; Sun, 2 Jan 2011 15:25:30 +0000 (UTC) Received: from server.vk2pj.dyndns.org (c220-239-116-103.belrs4.nsw.optusnet.com.au [220.239.116.103]) by mail34.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id p02FPOCE015544 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 3 Jan 2011 02:25:27 +1100 X-Bogosity: Ham, spamicity=0.000000 Received: from server.vk2pj.dyndns.org (localhost.vk2pj.dyndns.org [127.0.0.1]) by server.vk2pj.dyndns.org (8.14.4/8.14.4) with ESMTP id p02FPMfG022423; Mon, 3 Jan 2011 02:25:22 +1100 (EST) (envelope-from peter@server.vk2pj.dyndns.org) Received: (from peter@localhost) by server.vk2pj.dyndns.org (8.14.4/8.14.4/Submit) id p02FPMpX022422; Mon, 3 Jan 2011 02:25:22 +1100 (EST) (envelope-from peter) Date: Mon, 3 Jan 2011 02:25:22 +1100 From: Peter Jeremy To: Damien Fleuriot Message-ID: <20110102152521.GF48579@server.vk2pj.dyndns.org> References: <4D1C6F90.3080206@my.gd> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="lteA1dqeVaWQ9QQl" Content-Disposition: inline In-Reply-To: <4D1C6F90.3080206@my.gd> X-PGP-Key: http://members.optusnet.com.au/peterjeremy/pubkey.asc User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-stable@freebsd.org Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Jan 2011 15:25:31 -0000 --lteA1dqeVaWQ9QQl Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2010-Dec-30 12:40:00 +0100, Damien Fleuriot wrote: >What are the steps for properly removing my drives from the zraid1 pool >and inserting them in the zraid2 pool ? I've documented my experiences in migrating from a 3-way RAIDZ1 to a 6-way RAIDZ2 at http://bugs.au.freebsd.org/dokuwiki/doku.php/zfsraid Note that, even for a home system, backups are worthwhile. In my case, I backup onto a 2TB disk in an eSATA enclosure. That's currently (just) adequate but I'll soon need to identify data that I can leave off that backup. --=20 Peter Jeremy --lteA1dqeVaWQ9QQl Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.15 (FreeBSD) iEYEARECAAYFAk0gmOEACgkQ/opHv/APuIeCCwCgol+ML2/rxtHv+tgtsV4H99X/ UOoAn3X2cwLSE/gPDtvLlI0hxoSYXf7X =DNa2 -----END PGP SIGNATURE----- --lteA1dqeVaWQ9QQl-- From owner-freebsd-stable@FreeBSD.ORG Sun Jan 2 15:26:45 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 94DDA1065670 for ; Sun, 2 Jan 2011 15:26:45 +0000 (UTC) (envelope-from ronald-freebsd8@klop.yi.org) Received: from fep16.mx.upcmail.net (fep16.mx.upcmail.net [62.179.121.36]) by mx1.freebsd.org (Postfix) with ESMTP id E584E8FC1A for ; Sun, 2 Jan 2011 15:26:44 +0000 (UTC) Received: from edge05.upcmail.net ([192.168.13.212]) by viefep16-int.chello.at (InterMail vM.8.01.02.02 201-2260-120-106-20100312) with ESMTP id <20110102152642.XKQC29594.viefep16-int.chello.at@edge05.upcmail.net> for ; Sun, 2 Jan 2011 16:26:42 +0100 Received: from pinky ([213.46.23.80]) by edge05.upcmail.net with edge id qfSg1f02H1jgp3H05fSioy; Sun, 02 Jan 2011 16:26:42 +0100 X-SourceIP: 213.46.23.80 Content-Type: text/plain; charset=us-ascii; format=flowed; delsp=yes To: freebsd-stable@freebsd.org References: Date: Sun, 02 Jan 2011 16:26:40 +0100 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: "Ronald Klop" Message-ID: In-Reply-To: User-Agent: Opera Mail/11.00 (Win32) X-Cloudmark-Analysis: v=1.1 cv=vUpxTctd+kpWCBtSXXIkt5ll4Z8E5Qu9nLREXC/hfIo= c=1 sm=0 a=_ho-Ba79acAA:10 a=bgpUlknNv7MA:10 a=kj9zAlcOel0A:10 a=pGLkceISAAAA:8 a=3H_skrNEcvHEVgA7-AQA:9 a=zkmQ3z6qwey37GMdCHIA:7 a=WYS-2Tq7iwWKPihgr1aLwuvs2D4A:4 a=CjuIK1q_8ugA:10 a=bDqfGn44ETEA:10 a=MSl-tDqOz04A:10 a=HpAAvcLHHh0Zw7uRqdWCyQ==:117 Subject: Re: tmpfs runs out of space on 8.2pre-release, zfs related? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Jan 2011 15:26:45 -0000 On Sun, 02 Jan 2011 09:41:04 +0100, miyamoto moesasji wrote: > miyamoto moesasji gmail.com> writes: > >> >> In setting up tmpfs (so not tmpmfs) on a machine that is using >> zfs(v15, zfs v4) on 8.2prerelease I run out of space on the tmpfs when >> copying a file of ~4.6 GB file from the zfs-filesystem to the memory >> disk. This machine has 8GB of memory backed by swap on the harddisk, >> so I expected the file to copy to memory without problems. >> > > ....this is in fact worse than I first thought. After leaving the > machine running overnight the tmpfs is reduced to a size of 4K, which > shows that tmpfs is in fact > completely unusable for me. See the output of df: > --- > hge@PulsarX4:~/ > df -hi /tmp > Filesystem Size Used Avail Capacity iused ifree %iused Mounted > on > tmpfs 4.0K 4.0K 0B 100% 18 0 100% /tmp > --- > > Relevant zfs-stats info: > --- > System Memory Statistics: > Physical Memory: 8161.74M > Kernel Memory: 4117.40M > DATA: 99.29% 4088.07M > TEXT: 0.71% 29.33M > > ARC Size: > Current Size (arcsize): 63.58% 4370.60M > Target Size (Adaptive, c): 100.00% 6874.44M > Min Size (Hard Limit, c_min): 12.50% 859.31M > Max Size (High Water, c_max): ~8:1 6874.44M > --- > > I'm not sure what triggered this further reduction in size; but the > above 4K size is probably important to show how dramatic this goes > wrong. Is it possible that some program is filling a file (on your tmpfs) which is deleted? You will not see it with df or ls, but it still takes space on the fs, until the application closes the file. Ronald. From owner-freebsd-stable@FreeBSD.ORG Sun Jan 2 15:50:21 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ED07E106566C for ; Sun, 2 Jan 2011 15:50:20 +0000 (UTC) (envelope-from miyamoto.31b@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 817E48FC0A for ; Sun, 2 Jan 2011 15:50:20 +0000 (UTC) Received: by wyf19 with SMTP id 19so12666653wyf.13 for ; Sun, 02 Jan 2011 07:50:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=vK6UwddiiWVSRRI5QqlmGfLuKZQdPZfS+XZC2TfDrJ0=; b=hv2lYo0xdzyYkNtexQbQ8Q9AHhxAAN1737yMyLoGLE/+UINbwiMB+UfDLxH90zJEdY 0xtqEgfDmjqhJ8mb4BAuhdyydVIulqpw4KHy78Iw0tszC8zzTZITBB/CoXFqEU3s4ZsU roPQbfuB3u89OFFX9W2bMKYodsaiA6WZ2sUAw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=UJsDbsH0VUJT+6EDEVkRJSXRg6bWQ/ZQ6y2/n9q8zm4PwswjeBFptDNK2RMW174qrf 3uQrAV+4lrSQlluHYP7NsJh+S0WbQAkUX+NhM/Nqh3WJ4TbpvfTGTeTZY4GVNVXlK2ZM uBIbqCJDEWZEcdKxu4B+H1AlF5IobhYGTBZbw= MIME-Version: 1.0 Received: by 10.227.143.18 with SMTP id s18mr11041325wbu.98.1293983418624; Sun, 02 Jan 2011 07:50:18 -0800 (PST) Received: by 10.227.13.143 with HTTP; Sun, 2 Jan 2011 07:50:18 -0800 (PST) In-Reply-To: References: Date: Sun, 2 Jan 2011 15:50:18 +0000 Message-ID: From: miyamoto moesasji To: Ronald Klop Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-stable@freebsd.org Subject: Re: tmpfs runs out of space on 8.2pre-release, zfs related? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Jan 2011 15:50:21 -0000 On Sun, Jan 2, 2011 at 3:26 PM, Ronald Klop w= rote: > On Sun, 02 Jan 2011 09:41:04 +0100, miyamoto moesasji > wrote: > >> miyamoto moesasji gmail.com> writes: >> >>> >>> In setting up tmpfs (so not tmpmfs) on a machine that is using >>> zfs(v15, zfs v4) on 8.2prerelease I run out of space on the tmpfs when >>> copying a file of ~4.6 GB file from the zfs-filesystem to the memory >>> disk. This machine has 8GB of memory backed by swap on the harddisk, >>> so I expected the file to copy to memory without problems. >>> >> >> ....this is in fact worse than I first thought. After leaving the >> machine running overnight the tmpfs is reduced to a size of 4K, which >> shows that tmpfs is in fact >> completely unusable for me. See the output of df: >> --- >> hge@PulsarX4:~/ > df -hi /tmp >> Filesystem =A0 =A0Size =A0 =A0Used =A0 Avail Capacity iused ifree %iused= =A0Mounted on >> tmpfs =A0 =A0 =A0 =A0 4.0K =A0 =A04.0K =A0 =A0 =A00B =A0 100% =A0 =A0 = =A018 =A0 =A0 0 =A0100% =A0 /tmp >> --- >> >> Relevant zfs-stats info: >> --- >> System Memory Statistics: >> =A0 =A0 =A0 =A0Physical Memory: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A08161.74M >> =A0 =A0 =A0 =A0Kernel Memory: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A04117.40M >> =A0 =A0 =A0 =A0DATA: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= 99.29% =A04088.07M >> =A0 =A0 =A0 =A0TEXT: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= 0.71% =A0 29.33M >> >> ARC Size: >> =A0 =A0 =A0 =A0Current Size (arcsize): =A0 =A0 =A0 =A0 63.58% =A04370.60= M >> =A0 =A0 =A0 =A0Target Size (Adaptive, c): =A0 =A0 =A0100.00% 6874.44M >> =A0 =A0 =A0 =A0Min Size (Hard Limit, c_min): =A0 12.50% =A0859.31M >> =A0 =A0 =A0 =A0Max Size (High Water, c_max): =A0 ~8:1 =A0 =A06874.44M >> --- >> >> I'm not sure what triggered this further reduction in size; but the >> above 4K size is probably important to show how dramatic this goes >> wrong. > > Is it possible that some program is filling a file (on your tmpfs) which = is > deleted? > You will not see it with df or ls, but it still takes space on the fs, un= til > the application closes the file. > > Ronald. > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > I'm pretty sure that this is not the case as the problem shows up immediately upon a clean reboot (and /tmp is normally pretty empty with just 22k in use at the moment with the older tmpmfs (determined with du -xh /tmp as root) ) Note that the key problem was given in the first post. There I posted that the tmpfs has 8.2GB available immediately after a reboot, yet it is impossible to copy a 4.6GB file to tmpfs from a zfs-drive even though the machine has 8GB of memory backed with swap. My feeling is that somehow the zfs file cache, which is also in memory is causing this, which is consistent with the solaris bugreport I linked to, see: http://bugs.opensolaris.org/bugdatabase/view_bug.do;jsessionid=3De4ae9c3298= 3000ef651e38edbba1?bug_id=3D6804661 From owner-freebsd-stable@FreeBSD.ORG Sun Jan 2 16:59:19 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 972C6106566B for ; Sun, 2 Jan 2011 16:59:19 +0000 (UTC) (envelope-from prvs=19835cea4b=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id 1FD5B8FC18 for ; Sun, 2 Jan 2011 16:59:18 +0000 (UTC) X-MDAV-Processed: mail1.multiplay.co.uk, Sun, 02 Jan 2011 16:48:15 +0000 X-Spam-Processed: mail1.multiplay.co.uk, Sun, 02 Jan 2011 16:48:15 +0000 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on mail1.multiplay.co.uk X-Spam-Level: X-Spam-Status: No, score=-5.0 required=6.0 tests=USER_IN_WHITELIST shortcircuit=ham autolearn=disabled version=3.2.5 Received: from r2d2 ([188.220.16.49]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50011911713.msg for ; Sun, 02 Jan 2011 16:48:14 +0000 X-MDRemoteIP: 188.220.16.49 X-Return-Path: prvs=19835cea4b=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk X-MDaemon-Deliver-To: freebsd-stable@freebsd.org Message-ID: <9C43FB11DF5143A5BA72AF7F79DB49AB@multiplay.co.uk> From: "Steven Hartland" To: "Jeremy Chadwick" , "Peter Jeremy" References: <4D1AF388.3080107@infracaninophile.co.uk><4D1B7431.7070808@infracaninophile.co.uk><4D1BD8D0.5010402@langille.org><4D1C4A2D.4020206@infracaninophile.co.uk><4D1C7929.3040809@langille.org><20101231233343.GB48579@server.vk2pj.dyndns.org> <20101231234747.GA8171@icarus.home.lan> Date: Sun, 2 Jan 2011 16:48:15 -0000 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.5994 Cc: freebsd-stable@freebsd.org, Dan Langille Subject: Re: slow ZFS on FreeBSD 8.1 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Jan 2011 16:59:19 -0000 ----- Original Message ----- From: "Jeremy Chadwick" >> Based on my experiences at home, I converted my desktop at work to >> pure ZFS. The only issues I've run into have been programs that >> extensively use mmap(2) - which is a known issue with ZFS. > > Is your ZFS root filesystem associated with a pool that's mirrored or > using raidzX? What about mismatched /boot content (ZFS vs. UFS)? What > about booting into single-user mode? > > http://wiki.freebsd.org/ZFSOnRoot indirectly hints at these problems but > doesn't outright admit them (yet should), so I'm curious to know how > people have solved them. Remembering manual "one-offs" for a system > configured this way is not acceptable (read: highly prone to > error/mistake). Is it worth the risk? Most administrators don't have > the tolerance for stuff like that in the middle of a system upgrade or > what not; they should be able to follow exactly what's in the handbook, > to a tee. > > There's a link to www.dan.me.uk at the bottom of the above Wiki page > that outlines "the madness" that's required to configure the setup, all > of which has to be done by hand. I don't know many administrators who > are going to tolerate this when deploying numerous machines, especially > when compounded by the complexities mentioned above. With regards installing machines with a root zfs we now use mfsBSD which makes the process as simple as pie, so for those that haven't used it give it a wirl:- http://mfsbsd.vx.sk/ > The mmap(2) and sendfile(2) complexities will bite an junior or > mid-level SA in the butt too -- they won't know why software starts > failing or behaving oddly (FreeBSD ftpd is a good example). It just so > happens that Apache, out-of-the-box, comes with mmap and sendfile use > disabled. This is the same with nginx which is rapidly taking over from apache due to its ability to scale much much better than apache does. Proper mmap and sendfile integration are the only major issue we have with moving all our machines to ZFS thanks to great work by everyone. I really hope sendfile support in particular is fixed in the near future but as I understanding it, that's not going to be simple at all :( Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-freebsd-stable@FreeBSD.ORG Sun Jan 2 19:39:05 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B1FB1106566B for ; Sun, 2 Jan 2011 19:39:05 +0000 (UTC) (envelope-from joerg.schilling9cd33xy531fokus.fraunhofer.de@bounce.antispameurope.com) Received: from relay03-haj2.antispameurope.com (relay03-haj2.antispameurope.com [83.246.65.53]) by mx1.freebsd.org (Postfix) with ESMTP id 6FE808FC15 for ; Sun, 2 Jan 2011 19:39:04 +0000 (UTC) Received: by relay03-haj2.antispameurope.com (ASE-Secure-MTA, from userid 1000) id 55F2963C16C; Sun, 2 Jan 2011 20:22:35 +0100 (CET) Received: from pluto.fokus.fraunhofer.de (pluto.fokus.fraunhofer.de [195.37.77.164]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by relay03-haj2.antispameurope.com (ASE-Secure-MTA) with ESMTP id 5244563C062; Sun, 2 Jan 2011 20:22:34 +0100 (CET) Received: from EXCHSRV.fokus.fraunhofer.de (bohr.fokus.fraunhofer.de [10.147.9.231]) by pluto.fokus.fraunhofer.de (8.14.4/8.14.2) with SMTP id p02JMY6f000513; Sun, 2 Jan 2011 20:22:34 +0100 (MET) Received: from rigel ([10.147.65.195]) by EXCHSRV.fokus.fraunhofer.de with Microsoft SMTPSVC(6.0.3790.4675); Sun, 2 Jan 2011 20:22:34 +0100 Date: Sun, 02 Jan 2011 20:22:30 +0100 From: Joerg.Schilling@fokus.fraunhofer.de (Joerg Schilling) To: jamesbrandongooch@gmail.com, freebsd-stable@freebsd.org, mav@freebsd.org Message-ID: <4d20d076.UvzysHkUtsfHxuNB%Joerg.Schilling@fokus.fraunhofer.de> User-Agent: nail 11.22 3/20/05 MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-OriginalArrivalTime: 02 Jan 2011 19:22:34.0255 (UTC) FILETIME=[686521F0:01CBAAB2] Cc: Subject: Re: Sense fetching [Was: cdrtools /devel ...] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Jan 2011 19:39:05 -0000 If FreeBSD writes any warning for SCSI commands send by cdrtools, then this is definitely a kernel bug that needs to be fixed. SCSI is a protocol that lives from aparently failing SCSI commands. These warnings are related to "high level interaction" between cdrecord and the drive. Only cdrecord is able to decide whether a SCSI command that apparently failed is worth for being reported or not. It therefore is a bug if the kernel prints related messages. Jörg -- EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin js@cs.tu-berlin.de (uni) joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily From owner-freebsd-stable@FreeBSD.ORG Sun Jan 2 20:51:52 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1B489106566B for ; Sun, 2 Jan 2011 20:51:52 +0000 (UTC) (envelope-from me@pollux.local.net) Received: from smtp4-g21.free.fr (smtp4-g21.free.fr [212.27.42.4]) by mx1.freebsd.org (Postfix) with ESMTP id A09388FC21 for ; Sun, 2 Jan 2011 20:51:49 +0000 (UTC) Received: from pollux.local.net (unknown [82.246.30.233]) by smtp4-g21.free.fr (Postfix) with ESMTP id 8C90B4C8169 for ; Sun, 2 Jan 2011 21:51:44 +0100 (CET) Received: by pollux.local.net (Postfix, from userid 2000) id 2AA6A287F8; Sun, 2 Jan 2011 21:51:43 +0100 (CET) Date: Sun, 2 Jan 2011 21:51:43 +0100 From: Harald Weis To: freebsd-stable@freebsd.org Message-ID: <20110102205142.GA2645@pollux.local.net> Mail-Followup-To: freebsd-stable@freebsd.org References: <20101211120033.DC0541065746@hub.freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Subject: Re: umass: AutoSense failed X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Jan 2011 20:51:52 -0000 On Tue, Dec 14, 2010 at 01:19:29PM -0700, Ross Alexander wrote: > On Sat, 11 Dec 2010, Harad Weiss wrote > > >Message: 7 > >Date: Fri, 10 Dec 2010 22:59:17 +0100 > >From: Harald Weis > >Subject: Re: umass: AutoSense failed > >To: freebsd-stable@freebsd.org > >Cc: Jeremy Chadwick > >Message-ID: <20101210215917.GA2447@pollux.local.net> > >Content-Type: text/plain; charset=us-ascii > > > >[big snip] > > > >The AutoSense failure occurs on two desktop computers, whether > >motherboard or not, and on three laptops. As I said, all PC's > >run 8.1-RELEASE. > >What I do not know is whether the Sony DSC (Digital Still Camera) has > >worked on 8.0-RELEASE. I didn't use it for some time, so I'm afraid > >I never tried it on 8.x. > >But I am sure it worked on all earlier releases. > >In my original post I have forgotten to mention that I never had any > >problems with all sorts of thumbdrives and USB disks. > >Fortunately, I can read the 256MB CF memory of the DSC on an Ubuntu > >laptop. > >Could it possibly be a DSC hardware problem which is ignored by Ubuntu, > >but not by FreeBSD? > > My Sony DSC camera did work under 8.0-RELEASE. It's a model DSC-W35. > It's seeing the umass problem now, and has since 8.1-PRERELEASE afair. > > This fault is the same on i386 or amd64, intel or amd chipsets, USB > 1.0 or 2.0, through hubs or straight off the m/b, etc.. I'm keeping a > backup box (intel atom / i386) running 7.3-STABLE around specifically > to work around the problem. I'd have said something eralier, but I've > gotten used to small nits poppping up and then disappearing; this one > is dragging along :( I haven't looked at the List for a while, so I only saw your message yesterday. In the mean time, I've re-learned to work with bootable memory sticks. I've had no luck yet with ReWritable CDs which I need for all boxes but one. Perhaps a blanking problem? Here is the summary of my trials in Fixit mode for my Sony DSC-P10 which is also (at last) the answer to Jeremy's message: 7.2-RELEASE-i386-livefs: OK 8.0-RELEASE-i386-memstick: OK FreeBSD-8.1-STABLE-201009-i386-memstick: AutoSense failed FreeBSD-8.2-PRERELEASE-201012-i386-memstick: booting OK, but "No USB devices found!" CAM status: SCSI Status Error SCSI sense: UNIT ATTENTION asc: 28,0 (Not ready to ready change, medium may have changed) da0: 1915MB (3922944 512 byte sectors: 255H 63S/T 244C) GEOM: da0: media size does not match label. FreeBSD-9.0-CURRENT-201009-i386-memstick: AutoSense failed Is it not a case for send-pr(1) ? Best regards, Harald From owner-freebsd-stable@FreeBSD.ORG Sun Jan 2 22:38:58 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 89104106564A; Sun, 2 Jan 2011 22:38:58 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 3B7B28FC13; Sun, 2 Jan 2011 22:38:58 +0000 (UTC) Received: by iyb26 with SMTP id 26so12163674iyb.13 for ; Sun, 02 Jan 2011 14:38:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :organization:user-agent:mime-version:to:cc:subject:references :in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; bh=DlS+VaP+DxZGRHWUSqBfykgdwbniHs8vw0f/JeOI5q0=; b=AxqWk9WWEDNdPW++/1jlRKBvF/N968XA7tvSYfSE+7SX1eWqWix0x5tq0H+owYF5lV jMVygvFGau9wgwZO0wKfNZJA1PBHNj1IKIW8YUCSHn68vdoOLmGjMoASUP20DVjxe4Bn Hn/AA0BqUFVOqeElyxbKeMuWm4vdwx022W0Fg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:organization:user-agent:mime-version:to :cc:subject:references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=YmURWDdFUedpRN0hrLamu/Deu2teI7pjjZk/Hrt5XDDdwmnWy4oTwB7s3CmNsoK8A+ qYtFwB3SqMc+9LW1IQmyuCHDWqVm8B76zI1mFcdMFAmG4qw8Bt/LhbMnl1AF7ywQ+VLT CR0t1mLSoO0OCbXKwhXSDdy7I/Tm7NsS7E1Nc= Received: by 10.231.171.197 with SMTP id i5mr3820668ibz.54.1294007937440; Sun, 02 Jan 2011 14:38:57 -0800 (PST) Received: from centel.dataix.local (adsl-99-181-155-97.dsl.klmzmi.sbcglobal.net [99.181.155.97]) by mx.google.com with ESMTPS id z4sm18133772ibg.13.2011.01.02.14.38.54 (version=SSLv3 cipher=RC4-MD5); Sun, 02 Jan 2011 14:38:55 -0800 (PST) Sender: "J. Hellenthal" Message-ID: <4D20FE7D.6030803@DataIX.net> Date: Sun, 02 Jan 2011 17:38:53 -0500 From: "J. Hellenthal" Organization: http://www.DataIX.net User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.13) Gecko/20101230 Lightning/1.0b1 Thunderbird MIME-Version: 1.0 To: Attila Nagy References: <4D0A09AF.3040005@FreeBSD.org> <4D1F7008.3050506@fsn.hu> <4D1FF9AC.60200@DataIX.net> <4D203B43.7070607@fsn.hu> In-Reply-To: <4D203B43.7070607@fsn.hu> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, freebsd-stable@freebsd.org Subject: Re: New ZFSv28 patchset for 8-STABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Jan 2011 22:38:58 -0000 On 01/02/2011 03:45, Attila Nagy wrote: > On 01/02/2011 05:06 AM, J. Hellenthal wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> On 01/01/2011 13:18, Attila Nagy wrote: >>> On 12/16/2010 01:44 PM, Martin Matuska wrote: >>>> Link to the patch: >>>> >>>> http://people.freebsd.org/~mm/patches/zfs/v28/stable-8-zfsv28-20101215.patch.xz >>>> >>>> >>>> >>>> >>> I've used this: >>> http://people.freebsd.org/~mm/patches/zfs/v28/stable-8-zfsv28-20101223-nopython.patch.xz >>> >>> >>> on a server with amd64, 8 G RAM, acting as a file server on >>> ftp/http/rsync, the content being read only mounted with nullfs in >>> jails, and the daemons use sendfile (ftp and http). >>> >>> The effects can be seen here: >>> http://people.fsn.hu/~bra/freebsd/20110101-zfsv28-fbsd/ >>> the exact moment of the switch can be seen on zfs_mem-week.png, where >>> the L2 ARC has been discarded. >>> >>> What I see: >>> - increased CPU load >>> - decreased L2 ARC hit rate, decreased SSD (ad[46]), therefore increased >>> hard disk load (IOPS graph) >>> >>> Maybe I could accept the higher system load as normal, because there >>> were a lot of things changed between v15 and v28 (but I was hoping if I >>> use the same feature set, it will require less CPU), but dropping the >>> L2ARC hit rate so radically seems to be a major issue somewhere. >>> As you can see from the memory stats, I have enough kernel memory to >>> hold the L2 headers, so the L2 devices got filled up to their maximum >>> capacity. >>> >>> Any ideas on what could cause these? I haven't upgraded the pool version >>> and nothing was changed in the pool or in the file system. >>> >> Running arc_summary.pl[1] -p4 should print a summary about your l2arc >> and you should also notice in that section that there is a high number >> of "SPA Mismatch" mine usually grew to around 172k before I would notice >> a crash and I could reliably trigger this while in scrub. >> >> What ever is causing this needs desperate attention! >> >> I emailed mm@ privately off-list when I noticed this going on but have >> not received any feedback as of yet. > It's at zero currently (2 days of uptime): > kstat.zfs.misc.arcstats.l2_write_spa_mismatch: 0 > Right but do you have a 'cache' 'l2arc' vdev attached to any pool in the system ? This suggests to me that you do not at this time. If not can you attach a cache vdev and run a scrub on it and monitor the value of that MIB ? -- Regards, jhell,v JJH48-ARIN From owner-freebsd-stable@FreeBSD.ORG Mon Jan 3 13:20:07 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0AF1B1065670 for ; Mon, 3 Jan 2011 13:20:07 +0000 (UTC) (envelope-from freebsd-stable@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id B1CCF8FC0C for ; Mon, 3 Jan 2011 13:20:06 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1PZkKT-00059L-5K for freebsd-stable@freebsd.org; Mon, 03 Jan 2011 14:20:05 +0100 Received: from cpe-188-129-77-254.dynamic.amis.hr ([188.129.77.254]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 03 Jan 2011 14:20:05 +0100 Received: from ivoras by cpe-188-129-77-254.dynamic.amis.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 03 Jan 2011 14:20:05 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-stable@freebsd.org From: Ivan Voras Date: Mon, 03 Jan 2011 14:17:57 +0100 Lines: 10 Message-ID: References: <4D1C6F90.3080206@my.gd> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: cpe-188-129-77-254.dynamic.amis.hr User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.12) Gecko/20101102 Thunderbird/3.1.6 In-Reply-To: <4D1C6F90.3080206@my.gd> Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Jan 2011 13:20:07 -0000 On 12/30/10 12:40, Damien Fleuriot wrote: > I am concerned that in the event a drive fails, I won't be able to > repair the disks in time before another actually fails. An old trick to avoid that is to buy drives from different series or manufacturers (the theory is that identical drives tend to fail at the same time), but this may not be applicable if you have 5 drives in a volume :) Still, you can try playing with RAIDZ levels and probabilities. From owner-freebsd-stable@FreeBSD.ORG Mon Jan 3 13:30:05 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9B7801065674 for ; Mon, 3 Jan 2011 13:30:05 +0000 (UTC) (envelope-from freebsd-stable@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 329398FC0C for ; Mon, 3 Jan 2011 13:30:04 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1PZkU8-0004Xb-5W for freebsd-stable@freebsd.org; Mon, 03 Jan 2011 14:30:04 +0100 Received: from cpe-188-129-77-254.dynamic.amis.hr ([188.129.77.254]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 03 Jan 2011 14:30:04 +0100 Received: from ivoras by cpe-188-129-77-254.dynamic.amis.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 03 Jan 2011 14:30:04 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-stable@freebsd.org From: Ivan Voras Date: Mon, 03 Jan 2011 14:24:37 +0100 Lines: 23 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: cpe-188-129-77-254.dynamic.amis.hr User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.12) Gecko/20101102 Thunderbird/3.1.6 In-Reply-To: Subject: Re: tmpfs runs out of space on 8.2pre-release, zfs related? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Jan 2011 13:30:05 -0000 On 01/02/11 09:41, miyamoto moesasji wrote: > miyamoto moesasji gmail.com> writes: > >> >> In setting up tmpfs (so not tmpmfs) on a machine that is using >> zfs(v15, zfs v4) on 8.2prerelease I run out of space on the tmpfs when >> copying a file of ~4.6 GB file from the zfs-filesystem to the memory >> disk. This machine has 8GB of memory backed by swap on the harddisk, >> so I expected the file to copy to memory without problems. >> > > ....this is in fact worse than I first thought. After leaving the > machine running overnight the tmpfs is reduced to a size of 4K, which > shows that tmpfs is in fact > completely unusable for me. See the output of df: > --- > hge@PulsarX4:~/> df -hi /tmp > Filesystem Size Used Avail Capacity iused ifree %iused Mounted on > tmpfs 4.0K 4.0K 0B 100% 18 0 100% /tmp This is a known problem. So far, no solution has been offered, which means that effectively, tmpfs cannot be used with ZFS on the same system. From owner-freebsd-stable@FreeBSD.ORG Mon Jan 3 15:08:46 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 34CF3106564A for ; Mon, 3 Jan 2011 15:08:46 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id C92968FC0C for ; Mon, 3 Jan 2011 15:08:43 +0000 (UTC) Received: by wyf19 with SMTP id 19so13349142wyf.13 for ; Mon, 03 Jan 2011 07:08:43 -0800 (PST) Received: by 10.227.24.73 with SMTP id u9mr11816623wbb.168.1294067323030; Mon, 03 Jan 2011 07:08:43 -0800 (PST) Received: from dfleuriot.local ([83.167.62.196]) by mx.google.com with ESMTPS id m10sm14208056wbc.16.2011.01.03.07.08.42 (version=SSLv3 cipher=RC4-MD5); Mon, 03 Jan 2011 07:08:42 -0800 (PST) Message-ID: <4D21E679.80002@my.gd> Date: Mon, 03 Jan 2011 16:08:41 +0100 From: Damien Fleuriot User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-stable@freebsd.org References: <4D1C6F90.3080206@my.gd> In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Jan 2011 15:08:46 -0000 On 1/3/11 2:17 PM, Ivan Voras wrote: > On 12/30/10 12:40, Damien Fleuriot wrote: > >> I am concerned that in the event a drive fails, I won't be able to >> repair the disks in time before another actually fails. > > An old trick to avoid that is to buy drives from different series or > manufacturers (the theory is that identical drives tend to fail at the > same time), but this may not be applicable if you have 5 drives in a > volume :) Still, you can try playing with RAIDZ levels and probabilities. > That's sound advice, although one also hears that they should get devices from the same vendor for maximum compatibility -.- Ah well, next time ;) A piece of advice I shall heed though is using 1% less capacity than what the disks really provide, in case one day I have to swap a drive and its replacement is a few kbytes smaller (thus preventing a rebuild). From owner-freebsd-stable@FreeBSD.ORG Mon Jan 3 15:26:54 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BA49F106564A for ; Mon, 3 Jan 2011 15:26:54 +0000 (UTC) (envelope-from nobody@host.kellhost.com) Received: from host.kellhost.com (host.kellhost.com [72.52.236.212]) by mx1.freebsd.org (Postfix) with ESMTP id 8A54E8FC0C for ; Mon, 3 Jan 2011 15:26:54 +0000 (UTC) Received: from nobody by host.kellhost.com with local (Exim 4.69) (envelope-from ) id 1PZlWt-0002tr-Oa for freebsd-stable@freebsd.org; Mon, 03 Jan 2011 06:36:59 -0800 To: freebsd-stable@freebsd.org MIME-Version: 1.0 From: alex@kap7inc.com; brad@kap7inc.com Content-type: text/plain; charset=iso-8859-1 Message-Id: Date: Mon, 03 Jan 2011 06:36:59 -0800 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - host.kellhost.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [99 99] / [47 12] X-AntiAbuse: Sender Address Domain - host.kellhost.com Subject: Rob Benwell recommends this site X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: instantprofits@yahoo.com List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Jan 2011 15:26:54 -0000 Your friend Rob Benwell (instantprofits@yahoo.com) has recommended this site to you, and sends you the following message: Hello Friend, GUARANTEED!: Get your $778.83 instantprofit Commissions Now!!! Underground Automated Method Allows 19 Years Old Kid Blogger To Raid The Market For $23.6 Million And Takes Complete Newbie To $37,730 In Just 30 Days It Doesn\'t Matter If You\'ve Never Even Earned A Single Cent Online Before... If A Total Beginner Can Make $37,730 In His First Month Then Anyone Can Do This... Imagine waking up at 10 in the morning.. doing your short workout routine.. opening your computer..and finding out you made $100 up to $200 - while you were a sleep! It\'s Easy To Make Money Everyday Even If You\'re Starting From Scratch With Zero Knowledge, Experience Or Budget!I\'ll Show You Exactly How. We\'ve Start putting New 39 Members in YOUR TEAM for the January 1th-20th 2011 commission cycle... and GROWING everyday earn by $100 up to $200 or more. IMPORTANT: January 20, 2011 is the Cut-Off day to lock in your position then faster you act the higher commission you will earn!!! Go Here To Secure not less than $778.83 commission Now and it still growing as many people joining under you. if you secure your position right away: The $778.83 commission will Arrive Through your Paypal or Credit Card on February 20/2011 next month. Hurry\'this limited time, only 8 days remaining Positions are available Now. You will access your money in any ATM when you join early & follow the instructions you receive. Click below!! https://www.plimus.com/jsp/redirect.jsp?contractId=2896494&referrer=lanielami TYPE DATE & TIME --- NEW MEMBERS --- COUNTRY P -- JAN.3 @ 2:38 AM-----Dianna---- Rosebilt----- United States P -- JAN.3 @ 2:53 AM-----JOan------ Jacckson ---- United Kingdom P -- JAN.3 @ 2:56 AM-----Mandene -- Jecob-------- Germany M -- JAN.3 @ 4:19 AM-----Cristy---- Chan--------- Hungary P -- JAN.3 @ 4:28 AM-----Carlo----- Wonder------- Italy M -- JAN.2 @ 6:01 AM-----lalaine--- ferguson----- Australia P -- JAN.2 @ 7:11 AM-----Rebecca--- Underwood---- Canada P -- JAN.2 @ 7:39 AM-----Jericho--- Morales------ Mexico P -- JAN.2 @ 9:42 AM-----Thomas---- Silva ------- California M -- JAN.2 @ 9:58 PM-----Grace----- Taylor------- Singapore P -- JAN.2 @ 10:21 PM-----Gina------ Henry-------- New Zealand P -- JAN.1 @ 11:24 PM-----Mohammed-- Ahmen ------- Oman M -- JAN.1 @ 11:33 PM-----Tracia---- Furlong------ Puerto Rico P -- JAN.1 @ 11:41 PM-----Jane------ Mckay-------- Russia P -- JAN.1 @ 9:42 AM-----Steve----- Scott ------- Netherlands M -- JAN.1 @ 9:58 PM-----Greg------ Stanley------ Denmark P -- JAN.1 @ 10:21 PM-----Jack------ Perkins------ Amsterdam P -- JAN.1 @ 11:24 PM-----Arlene---- Tan --------- China M -- JAN.1 @ 11:33 PM-----Andy------ Hopekins----- New York P -- JAN.1 @ 11:41 PM-----Jhon------ Robinson----- United States M -- JAN.1 @ 2:34 AM --- Kevin----- Hunt -------- Sweden P -- JAN.1 @ 12:34 AM --- Delia----- Lane -------- New York P -- JAN.1 @ 6:45 AM --- Mohamed--- Suhail------- Saudi Arabia M -- JAN.1 @ 5:34 AM --- Aleks----- Gjuroski ---- Japan P -- JAN.1 @ 12:34 PM --- Andrew---- Karim ------- England P -- JAN.1 @ 8:23 AM --- Carla----- Pereira ----- South Korea P -- JAN.1 @ 2:34 PM --- Carl------ Krause ------ India M -- JAN.1 @ 9:14 AM --- David----- Cook -------- Denmark P - JAN.1 @ 7:46 AM --- Paul------ Amid -------- Taiwan P -- JAN.1 @ 1:54 AM --- Dennis---- Wilkins ----- Findland P -- JAN.1 @ 12:34 AM --- Jonathan-- Wangyu ------ Thailand P -- JAN.1 @ 6:45 AM --- Mackie---- Anhui-------- Indonesia M -- JAN.1 @ 5:34 AM --- Alexis---- Mathew ------ Jerosalim P -- JAN.1 @ 8:23 AM --- Cheryl---- Moran ------- Philippines P -- JAN.1 @ 2:34 PM --- Vergie---- Petter ------ Ingland M -- JAN.1 @ 9:14 AM --- Tito------ Warren ------ Bangladish P - JAN.1 @ 7:46 AM --- Raul------ Strogher ---- United States Therefore, you have a GUARANTEED $778.83 CommissionS every month from now on!. Earn $19.97 Per Process!Each $19.97 x 39 = $778.83 Commission will be yours...! Be Sure to Copy the link below & Paste into your browser and press enter: To Secure your $778.83 commission! You will access your money in any ATM when you follow the instructions you receive. https://www.plimus.com/jsp/redirect.jsp?contractId=2896494&referrer=lanielami Big Chance to you ,Becuase $19.97 membership only automatic back to you,Guaranteed you can recieve lifetime commissions every 20th of the month. Today its $778.83 For the start of the month of January if goes up daily until the end of the month. You must UPGRADE right away or before others do.... Caring for Your Success, Rob Benwell https://kap7waterpolo.com/proddetail.php?prod=kap103Junior From owner-freebsd-stable@FreeBSD.ORG Mon Jan 3 20:03:12 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5D5761065695; Mon, 3 Jan 2011 20:03:12 +0000 (UTC) (envelope-from bra@fsn.hu) Received: from people.fsn.hu (people.fsn.hu [195.228.252.137]) by mx1.freebsd.org (Postfix) with ESMTP id 50EBE8FC24; Mon, 3 Jan 2011 20:03:10 +0000 (UTC) Received: by people.fsn.hu (Postfix, from userid 1001) id 0C5D56B624F; Mon, 3 Jan 2011 21:03:09 +0100 (CET) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000041, version=1.2.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MF-ACE0E1EA [pR: 14.1274] X-CRM114-CacheID: sfid-20110103_21030_C28A4C88 X-CRM114-Status: Good ( pR: 14.1274 ) X-Spambayes-Classification: ham; 0.00 Message-ID: <4D222B7B.1090902@fsn.hu> Date: Mon, 03 Jan 2011 21:03:07 +0100 From: Attila Nagy User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.23) Gecko/20090817 Thunderbird/2.0.0.23 Mnenhy/0.7.6.0 MIME-Version: 1.0 To: Artem Belevich References: <4D0A09AF.3040005@FreeBSD.org> <4D1F7008.3050506@fsn.hu> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, freebsd-stable@freebsd.org, Martin Matuska Subject: Re: New ZFSv28 patchset for 8-STABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Jan 2011 20:03:12 -0000 On 01/01/2011 08:09 PM, Artem Belevich wrote: > On Sat, Jan 1, 2011 at 10:18 AM, Attila Nagy wrote: >> What I see: >> - increased CPU load >> - decreased L2 ARC hit rate, decreased SSD (ad[46]), therefore increased >> hard disk load (IOPS graph) >> > ... >> Any ideas on what could cause these? I haven't upgraded the pool version and >> nothing was changed in the pool or in the file system. > The fact that L2 ARC is full does not mean that it contains the right > data. Initial L2ARC warm up happens at a much higher rate than the > rate L2ARC is updated after it's been filled initially. Even > accelerated warm-up took almost a day in your case. In order for L2ARC > to warm up properly you may have to wait quite a bit longer. My guess > is that it should slowly improve over the next few days as data goes > through L2ARC and those bits that are hit more often take residence > there. The larger your data set, the longer it will take for L2ARC to > catch the right data. > > Do you have similar graphs from pre-patch system just after reboot? I > suspect that it may show similarly abysmal L2ARC hit rates initially, > too. > > After four days, the L2 hit rate is still hovering around 10-20 percents (was between 60-90), so I think it's clearly a regression in the ZFSv28 patch... And the massive growth in CPU usage can also very nicely be seen... I've updated the graphs at (switch time can be checked on the zfs-mem graph): http://people.fsn.hu/~bra/freebsd/20110101-zfsv28-fbsd/ There is a new phenomenom: the large IOPS peaks. I use this munin script on a lot of machines and never seen anything like this... I'm not sure whether it's related or not. From owner-freebsd-stable@FreeBSD.ORG Mon Jan 3 21:51:33 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F032F106564A; Mon, 3 Jan 2011 21:51:33 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from blade.simplesystems.org (blade.simplesystems.org [65.66.246.74]) by mx1.freebsd.org (Postfix) with ESMTP id 888528FC0A; Mon, 3 Jan 2011 21:51:33 +0000 (UTC) Received: from freddy.simplesystems.org (freddy.simplesystems.org [65.66.246.65]) by blade.simplesystems.org (8.13.8+Sun/8.13.8) with ESMTP id p03LZLt0019766; Mon, 3 Jan 2011 15:35:22 -0600 (CST) Date: Mon, 3 Jan 2011 15:35:21 -0600 (CST) From: Bob Friesenhahn X-X-Sender: bfriesen@freddy.simplesystems.org To: Attila Nagy In-Reply-To: <4D222B7B.1090902@fsn.hu> Message-ID: References: <4D0A09AF.3040005@FreeBSD.org> <4D1F7008.3050506@fsn.hu> <4D222B7B.1090902@fsn.hu> User-Agent: Alpine 2.01 (GSO 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (blade.simplesystems.org [65.66.246.90]); Mon, 03 Jan 2011 15:35:22 -0600 (CST) Cc: freebsd-fs@freebsd.org, freebsd-stable@freebsd.org, Artem Belevich Subject: Re: New ZFSv28 patchset for 8-STABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Jan 2011 21:51:34 -0000 >> > After four days, the L2 hit rate is still hovering around 10-20 percents (was > between 60-90), so I think it's clearly a regression in the ZFSv28 patch... > And the massive growth in CPU usage can also very nicely be seen... > > I've updated the graphs at (switch time can be checked on the zfs-mem graph): > http://people.fsn.hu/~bra/freebsd/20110101-zfsv28-fbsd/ > > There is a new phenomenom: the large IOPS peaks. I use this munin script on a > lot of machines and never seen anything like this... I'm not sure whether > it's related or not. It is not so clear that there is a problem. I am not sure what you are using this server for but it is wise to consider that this is the funny time when a new year starts, SPAM delivery goes through the roof, and employees and customers behave differently. You chose the worst time of the year to implement the change and observe behavior. CPU use is indeed increased somewhat. A lower loading of the l2arc is not necessarily a problem. The l2arc is usually bandwidth limited compared with main store so if bulk data can not be cached in RAM, then it is best left in main store. A smarter l2arc algorithm could put only the data producing the expensive IOPS (the ones requiring a seek) in the l2arc, lessening the amount of data cached on the device. Bob -- Bob Friesenhahn bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From owner-freebsd-stable@FreeBSD.ORG Mon Jan 3 23:15:01 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B72B0106564A for ; Mon, 3 Jan 2011 23:15:01 +0000 (UTC) (envelope-from marek_sal@wp.pl) Received: from mx3.wp.pl (mx3.wp.pl [212.77.101.7]) by mx1.freebsd.org (Postfix) with ESMTP id 47EB18FC17 for ; Mon, 3 Jan 2011 23:15:00 +0000 (UTC) Received: (wp-smtpd smtp.wp.pl 13051 invoked from network); 3 Jan 2011 23:48:18 +0100 Received: from out.poczta.wp.pl (HELO localhost) ([212.77.101.240]) (envelope-sender ) by smtp.wp.pl (WP-SMTPD) with SMTP for ; 3 Jan 2011 23:48:18 +0100 Date: Mon, 03 Jan 2011 23:48:16 +0100 From: "Marek Salwerowicz" To: freebsd-stable@freebsd.org Message-ID: <4d2252305ca851.07700160@wp.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-2 Content-Transfer-Encoding: 8bit Content-Disposition: inline X-Mailer: Interfejs WWW nowej poczty Wirtualnej Polski X-User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; pl; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 Organization: Poznaj Poczte WP http://poczta.wp.pl/info-start.html X-WP-IP: 83.19.131.170 X-WP-AV: skaner antywirusowy poczty Wirtualnej Polski S. A. X-WP-SPAM: NO 0000000 [wdOE] Subject: NFSv4 - how to set up at FreeBSD 8.1 ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Jan 2011 23:15:01 -0000 Hi all, I have installed two VMs (server and client), regular installation with GENERIC kernel. I cannot manage to set up NFSv4 working on them. Configuration of server: /etc/rc.conf: keymap="pl_PL.ISO8859-2" hostname="nfs4-server" ifconfig_em0="dhcp" sshd_enable="YES" nfs_server_enable="YES" nfsv4_server_enable="YES" nfsuserd_enable="YES" /etc/exports: V4: / Configuration of client: /etc/rc.conf: keymap="pl_PL.ISO8859-2" hostname="nfs4-client" ifconfig_em0="dhcp" sshd_enable="YES" nfsuserd_enable="YES" nfscbd_enable="YES" Trying to mount '/' server share at client: nfs4-client# mount_nfs -o nfsv4 192.168.183.131:/ /marek_nfs4/ nfs4-client# ls /marek_nfs4/ ls: /marek_nfs4/: Input/output error nfs4-client# What am I doing wrong? My aim is to mount home directories from server to client but currently I am unable to mount any share. -- Marek Salwerowicz From owner-freebsd-stable@FreeBSD.ORG Mon Jan 3 23:37:19 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B929310656C3 for ; Mon, 3 Jan 2011 23:37:19 +0000 (UTC) (envelope-from milu@dat.pl) Received: from jab.dat.pl (dat.pl [80.51.155.34]) by mx1.freebsd.org (Postfix) with ESMTP id 730F48FC1E for ; Mon, 3 Jan 2011 23:37:19 +0000 (UTC) Received: from jab.dat.pl (jsrv.dat.pl [127.0.0.1]) by jab.dat.pl (Postfix) with ESMTP id D38E94D; Tue, 4 Jan 2011 00:37:17 +0100 (CET) X-Virus-Scanned: amavisd-new at dat.pl Received: from jab.dat.pl ([127.0.0.1]) by jab.dat.pl (jab.dat.pl [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 5wSOfsQrfcb2; Tue, 4 Jan 2011 00:37:15 +0100 (CET) Received: from snifi.laptop (213-238-69-122.adsl.inetia.pl [213.238.69.122]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by jab.dat.pl (Postfix) with ESMTPSA id A27F92E; Tue, 4 Jan 2011 00:37:14 +0100 (CET) From: Maciej Milewski To: freebsd-stable@freebsd.org Date: Tue, 4 Jan 2011 00:37:59 +0100 User-Agent: KMail/1.13.5 (FreeBSD/8.2-PRERELEASE; KDE/4.5.4; i386; ; ) References: <4d2252305ca851.07700160@wp.pl> In-Reply-To: <4d2252305ca851.07700160@wp.pl> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-2" Content-Transfer-Encoding: quoted-printable Message-Id: <201101040037.59785.milu@dat.pl> Cc: Marek Salwerowicz Subject: Re: NFSv4 - how to set up at FreeBSD 8.1 ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Jan 2011 23:37:19 -0000 Monday 03 of January 2011 23:48:16 Marek Salwerowicz napisa=B3(a): > /etc/exports: > V4: / > What am I doing wrong? My aim is to mount home directories from server > to client but currently I am unable to mount any share. Maybe it's only incorrect exports file?? In my case (NFSv3) it looks: /data -maproot=3Droot 192.168.0.10 =2D- Maciej From owner-freebsd-stable@FreeBSD.ORG Tue Jan 4 00:06:16 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 402CB106567A for ; Tue, 4 Jan 2011 00:06:16 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id EE1F08FC08 for ; Tue, 4 Jan 2011 00:06:15 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApwEAOzwIU2DaFvO/2dsb2JhbACDd6ExrQqOMYEggzZ0BIRlhh8 X-IronPort-AV: E=Sophos;i="4.60,268,1291611600"; d="scan'208";a="104177524" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 03 Jan 2011 18:56:34 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 4483EB3F06; Mon, 3 Jan 2011 18:56:34 -0500 (EST) Date: Mon, 3 Jan 2011 18:56:34 -0500 (EST) From: Rick Macklem To: Marek Salwerowicz Message-ID: <1549692872.56916.1294098994272.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <4d2252305ca851.07700160@wp.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - IE8 (Win)/6.0.10_GA_2692) Cc: freebsd-stable@freebsd.org Subject: Re: NFSv4 - how to set up at FreeBSD 8.1 ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2011 00:06:16 -0000 > Hi all, > > I have installed two VMs (server and client), regular installation > with > GENERIC kernel. > I cannot manage to set up NFSv4 working on them. > > Configuration of server: > > /etc/rc.conf: > keymap="pl_PL.ISO8859-2" > hostname="nfs4-server" > ifconfig_em0="dhcp" > sshd_enable="YES" > nfs_server_enable="YES" > nfsv4_server_enable="YES" > nfsuserd_enable="YES" > > /etc/exports: > V4: / This only states where the root of the nfsv4 tree is. You also need to export the volume(s) just like for nfsv3. If you are mounting "/" as below you'd also need something like the line: / -maproot=root -network 192.168.183.1 -mask 255.255.255.0 > > > Configuration of client: > /etc/rc.conf: > keymap="pl_PL.ISO8859-2" > hostname="nfs4-client" > ifconfig_em0="dhcp" > sshd_enable="YES" > nfsuserd_enable="YES" > nfscbd_enable="YES" > > Trying to mount '/' server share at client: > nfs4-client# mount_nfs -o nfsv4 192.168.183.131:/ /marek_nfs4/ > nfs4-client# ls /marek_nfs4/ > ls: /marek_nfs4/: Input/output error > nfs4-client# > > > What am I doing wrong? My aim is to mount home directories from server > to client but currently I am unable to mount any share. > From owner-freebsd-stable@FreeBSD.ORG Tue Jan 4 00:18:48 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 787601065670 for ; Tue, 4 Jan 2011 00:18:48 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 3926D8FC08 for ; Tue, 4 Jan 2011 00:18:47 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApwEALPvIU2DaFvO/2dsb2JhbACDd6ExrRCOMYEggzZ0BIRlhh8 X-IronPort-AV: E=Sophos;i="4.60,268,1291611600"; d="scan'208";a="104177167" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 03 Jan 2011 18:50:00 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 733E9B3F09; Mon, 3 Jan 2011 18:50:00 -0500 (EST) Date: Mon, 3 Jan 2011 18:50:00 -0500 (EST) From: Rick Macklem To: Maciej Milewski Message-ID: <1703110029.56794.1294098600382.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <201101040037.59785.milu@dat.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [172.17.91.200] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - IE8 (Win)/6.0.10_GA_2692) Cc: Marek Salwerowicz , freebsd-stable@freebsd.org Subject: Re: NFSv4 - how to set up at FreeBSD 8.1 ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2011 00:18:48 -0000 > Monday 03 of January 2011 23:48:16 Marek Salwerowicz napisa=C5=82(a): > > /etc/exports: > > V4: / >=20 > > What am I doing wrong? My aim is to mount home directories from > > server > > to client but currently I am unable to mount any share. > Maybe it's only incorrect exports file?? > In my case (NFSv3) it looks: > /data -maproot=3Droot 192.168.0.10 > -- If the above 2 lines are in your /etc/exports file and "/" is a ufs file system, then the above should work. For a zfs "/" you must either: - export / as well as /data or - use "v4: /data" so that the nfsv4 root is at /data Also, make sure you are running the experimental server: - either start both mountd and nfsd with the "-e" option or specify nfsv4_server_enable=3D"YES" nfs_server_enable=3D"YES" in your /etc/rc.conf. Also, you need to create an empty /var/db/nfs-stablerestart file before the experimental NFS server will start up the first time. (A fix for that is in the works, but isn't even in head yet.) Try looking at "man nfsv4" and checking that the daemons are running and that nothing got logged in /var/log/messages when they started up. Good luck with it, rick From owner-freebsd-stable@FreeBSD.ORG Tue Jan 4 03:08:52 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2881B106566B for ; Tue, 4 Jan 2011 03:08:51 +0000 (UTC) (envelope-from dan@langille.org) Received: from nyi.unixathome.org (nyi.unixathome.org [64.147.113.42]) by mx1.freebsd.org (Postfix) with ESMTP id B71498FC12 for ; Tue, 4 Jan 2011 03:08:51 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by nyi.unixathome.org (Postfix) with ESMTP id C6361509A5 for ; Tue, 4 Jan 2011 03:08:50 +0000 (GMT) X-Virus-Scanned: amavisd-new at unixathome.org Received: from nyi.unixathome.org ([127.0.0.1]) by localhost (nyi.unixathome.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uFUl+dsJsFgc for ; Tue, 4 Jan 2011 03:08:50 +0000 (GMT) Received: from smtp-auth.unixathome.org (smtp-auth.unixathome.org [10.4.7.7]) (Authenticated sender: hidden) by nyi.unixathome.org (Postfix) with ESMTPSA id 530E650852 for ; Tue, 4 Jan 2011 03:08:50 +0000 (GMT) Message-ID: <4D228F41.7040403@langille.org> Date: Mon, 03 Jan 2011 22:08:49 -0500 From: Dan Langille Organization: The FreeBSD Diary User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-stable Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: ZFS - hot spares : automatic or not? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2011 03:08:52 -0000 Hello folks, I'm trying to discover if ZFS under FreeBSD will automatically pull in a hot spare if one is required. This raised the issue back in March 2010, and refers to a PR opened in May 2009 * http://lists.freebsd.org/pipermail/freebsd-fs/2010-March/007943.html * http://www.freebsd.org/cgi/query-pr.cgi?pr=134491 In turn, the PR refers to this March 2010 post referring to using devd to accomplish this task. http://lists.freebsd.org/pipermail/freebsd-stable/2010-March/055686.html Does the above represent the the current state? I ask because I just ordered two more HDD to use as spares. Whether they sit on the shelf or in the box is open to discussion. -- Dan Langille - http://langille.org/ From owner-freebsd-stable@FreeBSD.ORG Tue Jan 4 11:42:29 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 700441065673 for ; Tue, 4 Jan 2011 11:42:29 +0000 (UTC) (envelope-from jyavenard@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 2C6C48FC18 for ; Tue, 4 Jan 2011 11:42:28 +0000 (UTC) Received: by iwn39 with SMTP id 39so14489658iwn.13 for ; Tue, 04 Jan 2011 03:42:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=AABalFHYUnSeId+faYnuFoIvO4bAbMPAxkLhVmB2q6Q=; b=vQPOLyy27ziBSHvVKeeKuQaFID52Ui67IjP6gz4QibQZdLCLK5qzSBww7eLvlp8xzs BNsvSrq1N8r2wjTLR2jGltfFBoE72r3J48GECVnhyxC6wnnBr7uSklc8CFIGWGKG77Rl GLD+twisqnBZAzsT9obWvQwhy3R9I1fwe48wY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=T62Hs6uXfIaGAZHR9CIrrNOQjhTLYCtmHHzxFcK7/g37oKfbIGOeS7p9DvSYGwXbc3 +MfUH5t1Z1dh4AMqgO569bC4SPP9w2dz/CJNz68tP4jhNSNFOyrAvukluknkGHlZB7bi 2PUlAMtYtnZzpFERwGlmWBUyV/vtNYI0AR9GY= MIME-Version: 1.0 Received: by 10.42.177.196 with SMTP id bj4mr22049911icb.129.1294141348363; Tue, 04 Jan 2011 03:42:28 -0800 (PST) Received: by 10.42.172.69 with HTTP; Tue, 4 Jan 2011 03:42:28 -0800 (PST) In-Reply-To: <1703110029.56794.1294098600382.JavaMail.root@erie.cs.uoguelph.ca> References: <201101040037.59785.milu@dat.pl> <1703110029.56794.1294098600382.JavaMail.root@erie.cs.uoguelph.ca> Date: Tue, 4 Jan 2011 22:42:28 +1100 Message-ID: From: Jean-Yves Avenard To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 Cc: Marek Salwerowicz , freebsd-stable@freebsd.org, Maciej Milewski Subject: Re: NFSv4 - how to set up at FreeBSD 8.1 ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2011 11:42:29 -0000 Hi On 4 January 2011 10:50, Rick Macklem wrote: > If the above 2 lines are in your /etc/exports file and "/" is a ufs > file system, then the above should work. For a zfs "/" you must either: > - export / as well as /data > or > - use "v4: /data" so that the nfsv4 root is at /data > > Also, make sure you are running the experimental server: > - either start both mountd and nfsd with the "-e" option or specify > nfsv4_server_enable="YES" > nfs_server_enable="YES" > > in your /etc/rc.conf. > > Also, you need to create an empty /var/db/nfs-stablerestart file before > the experimental NFS server will start up the first time. (A fix for that > is in the works, but isn't even in head yet.) > > Try looking at "man nfsv4" and checking that the daemons are running and > that nothing got logged in /var/log/messages when they started up. > > Good luck with it, rick After reading this thread, I tried NFSv4 today.. Whenever I tried to mount from a linux client, I get: mount -o vers=4 server4:/pool/backup/sites/m /mnt NFS compound failed for server server4: error 7 (RPC: Authentication error) NFS compound failed for server server4: error 7 (RPC: Authentication error) NFS compound failed for server server4: error 7 (RPC: Authentication error) NFS compound failed for server server4: error 7 (RPC: Authentication error) NFS compound failed for server server4: error 7 (RPC: Authentication error) NFS compound failed for server server4: error 7 (RPC: Authentication error) nfs mount: mount: /mnt: Permission denied with NFS v3 it mounts just fine any ideas? From owner-freebsd-stable@FreeBSD.ORG Tue Jan 4 12:49:38 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6F88C106564A for ; Tue, 4 Jan 2011 12:49:38 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 2B7298FC0A for ; Tue, 4 Jan 2011 12:49:37 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApwEAEamIk2DaFvO/2dsb2JhbACDdqE0rByOZ4EggzZ0BIRohh8 X-IronPort-AV: E=Sophos;i="4.60,272,1291611600"; d="scan'208";a="105846729" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 04 Jan 2011 07:49:36 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id E35E1B3FFE; Tue, 4 Jan 2011 07:49:36 -0500 (EST) Date: Tue, 4 Jan 2011 07:49:36 -0500 (EST) From: Rick Macklem To: Jean-Yves Avenard Message-ID: <1681062377.64173.1294145376835.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.198] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - IE8 (Win)/6.0.10_GA_2692) Cc: Marek Salwerowicz , freebsd-stable@freebsd.org, Maciej Milewski Subject: Re: NFSv4 - how to set up at FreeBSD 8.1 ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2011 12:49:38 -0000 > > > After reading this thread, I tried NFSv4 today.. > > Whenever I tried to mount from a linux client, I get: > mount -o vers=4 server4:/pool/backup/sites/m /mnt > NFS compound failed for server server4: error 7 (RPC: Authentication > error) > NFS compound failed for server server4: error 7 (RPC: Authentication > error) > NFS compound failed for server server4: error 7 (RPC: Authentication > error) > NFS compound failed for server server4: error 7 (RPC: Authentication > error) > NFS compound failed for server server4: error 7 (RPC: Authentication > error) > NFS compound failed for server server4: error 7 (RPC: Authentication > error) > nfs mount: mount: /mnt: Permission denied > > with NFS v3 it mounts just fine > > any ideas? > Hmm, try adding "sec=sys" and a network specification to the V4: line in /etc/exports. I had thought the default was "sec=sys" and "the world", but maybe I'm wrong w.r.t. the defaults. (I always specify them in my V4: lines.) For example: V4: / -sec=sys -network 192.168.138.0 -netmask 255.255.255.0 (You'll need to send a HUP signal to mountd after the change.) If that doesn't work, capture a packet trace of the mount attempt via: tcpdump -s 0 -w xxx host server4 and email me "xxx" (or look at it yourself with wireshark) to see what Linux is attempting that is failing. (If for some reason Linux is trying to use krb5, that would also explain the failure. I have no idea if Linux might decide krb5 should be the default for NFSv4.) Good luck with it and let us know how it goes, rick From owner-freebsd-stable@FreeBSD.ORG Tue Jan 4 13:17:29 2011 Return-Path: Delivered-To: stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E8217106564A for ; Tue, 4 Jan 2011 13:17:29 +0000 (UTC) (envelope-from hopet@ics.muni.cz) Received: from minas.ics.muni.cz (minas.ics.muni.cz [147.251.4.40]) by mx1.freebsd.org (Postfix) with ESMTP id 276C58FC0A for ; Tue, 4 Jan 2011 13:17:28 +0000 (UTC) Received: from kloboucek (dhcp3-81.ics.muni.cz [147.251.3.81]) (authenticated user=hopet@META bits=0) by minas.ics.muni.cz (8.13.8/8.13.8/SuSE Linux 0.8) with ESMTP id p04Cn2RT007926 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO) for ; Tue, 4 Jan 2011 13:49:02 +0100 From: "Petr Holub" To: Date: Tue, 4 Jan 2011 13:48:54 +0100 Message-ID: <016301cbac0d$c1b7ed80$4527c880$@muni.cz> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AcusDb0fpNKLrmJ0ST6fkh4k5ozxNg== Content-Language: cs X-Muni-Spam-TestIP: 147.251.3.81 X-Muni-Envelope-From: hopet@ics.muni.cz X-Muni-Virus-Test: Clean X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-3.0 (minas.ics.muni.cz [147.251.4.35]); Tue, 04 Jan 2011 13:49:02 +0100 (CET) Cc: Subject: spontaneous reboots on 8.2-PRERELEASE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2011 13:17:30 -0000 Hi list, I've installed PC-BSD 8.2-BETA1 aka FreeBSD 8.2-PRERELEASE on a machine that was running FreeBSD 7.3 rock solid and now I get random reboots of the machine, mostly when under the load. ACPI/no ACPI makes no difference and unfortunately, I'm unable to get reliable thermal readings (apci_thermal doesn't get loaded and healthd tends to give some bogus occasionaly, thus not very reliable source either). Sometimes, I also get some USB-related errors like this: usbus4: port reset timeout usbd_req_re_enumerate: addr=3, port reset failed, USB_ERR_TIMEOUT usbd_req_re_enumerate: addr=3, port reset failed, USB_ERR_TIMEOUT ugen4.3: at usbus4 (disconnected) uhub_reattach_port: could not allocate new device ugen4.3: at usbus4 (disconnected) uhub_reattach_port: could not allocate new device ugen4.3: at usbus4 (disconnected) uhub_reattach_port: could not allocate new device usbus4: port reset timeout usbd_req_re_enumerate: addr=3, port reset failed, USB_ERR_TIMEOUT usbd_req_re_enumerate: addr=3, port reset failed, USB_ERR_TIMEOUT and Root mount waiting for: usbus4 dumpdev="AUTO" and getting nothing in /var/crash, just plain reboots. Any clues how to debug this_ Dmesg is below. Thanks, Petr --- Copyright (c) 1992-2010 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 8.2-PRERELEASE #3: Mon Dec 13 08:50:49 PST 2010 root@build8x32.pcbsd.org:/usr/obj/usr/local_storage/pcbsd-build82/fbsd-source/8.2/sys/PCBSD i386 Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Pentium(R) 4 CPU 3.40GHz (3412.09-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0xf43 Family = f Model = 4 Stepping = 3 Features=0xbfebfbff Features2=0x649d AMD Features=0x20000000 TSC: P-state invariant real memory = 1073741824 (1024 MB) avail memory = 1018896384 (971 MB) MPTable: ioapic0: Assuming intbase of 0 ioapic1: Assuming intbase of 24 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard Cuse4BSD v0.1.13 @ /dev/cuse kbd1 at kbdmux0 cryptosoft0: on motherboard pcib0: pcibus 0 on motherboard pci0: on pcib0 pcib1: irq 16 at device 1.0 on pci0 pci6: on pcib1 vgapci0: port 0xcc00-0xcc7f mem 0xfd000000-0xfdffffff,0xd0000000-0xdfffffff,0xfc000000-0xfcffffff irq 16 at device 0.0 on pci6 nvidia0: on vgapci0 vgapci0: child nvidia0 requested pci_enable_busmaster vgapci0: child nvidia0 requested pci_enable_io vgapci0: child nvidia0 requested pci_enable_io nvidia0: [ITHREAD] pci0: at device 27.0 (no driver attached) pcib2: irq 16 at device 28.0 on pci0 pci4: on pcib2 pcib3: at device 0.0 on pci4 pci5: on pcib3 em0: port 0xbc00-0xbc3f mem 0xfa9e0000-0xfa9fffff,0xfa980000-0xfa9bffff irq 24 at device 1.0 on pci5 em0: [FILTER] em0: Ethernet address: 00:04:23:c3:90:83 pcib4: irq 16 at device 28.4 on pci0 pci3: on pcib4 mskc0: port 0xa800-0xa8ff mem 0xfa7fc000-0xfa7fffff irq 16 at device 0.0 on pci3 msk0: on mskc0 msk0: Ethernet address: 00:15:f2:eb:7d:f4 miibus0: on msk0 e1000phy0: PHY 0 on miibus0 e1000phy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow msk1: on mskc0 msk1: Ethernet address: 00:15:f2:eb:7d:f5 miibus1: on msk1 e1000phy1: PHY 0 on miibus1 e1000phy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto mskc0: [ITHREAD] pcib5: irq 17 at device 28.5 on pci0 pci2: on pcib5 atapci0: port 0x9c00-0x9c07,0x9880-0x9883,0x9800-0x983f,0x9480-0x949f mem 0xfa6ffc00-0xfa6fffff irq 17 at device 0.0 on pci2 atapci0: [ITHREAD] ahci0: on atapci0 ahci0: [ITHREAD] ahci0: AHCI v1.00 with 4 3Gbps ports, Port Multiplier supported ahcich0: at channel 0 on ahci0 ahcich0: [ITHREAD] ahcich1: at channel 1 on ahci0 ahcich1: [ITHREAD] ahcich2: at channel 2 on ahci0 ahcich2: [ITHREAD] ahcich3: at channel 3 on ahci0 ahcich3: [ITHREAD] ata2: on atapci0 ata2: [ITHREAD] uhci0: port 0xe480-0xe49f irq 20 at device 29.0 on pci0 uhci0: [ITHREAD] uhci0: LegSup = 0x2f00 usbus0: on uhci0 uhci1: port 0xe800-0xe81f irq 17 at device 29.1 on pci0 uhci1: [ITHREAD] uhci1: LegSup = 0x2f00 usbus1: on uhci1 uhci2: port 0xe880-0xe89f irq 18 at device 29.2 on pci0 uhci2: [ITHREAD] uhci2: LegSup = 0x2f00 usbus2: on uhci2 uhci3: port 0xec00-0xec1f irq 19 at device 29.3 on pci0 uhci3: [ITHREAD] uhci3: LegSup = 0x2f00 usbus3: on uhci3 ehci0: mem 0xfebfbc00-0xfebfbfff irq 20 at device 29.7 on pci0 ehci0: [ITHREAD] usbus4: EHCI version 1.0 usbus4: on ehci0 pcib6: at device 30.0 on pci0 pci1: on pcib6 pci1: at device 1.0 (no driver attached) pci1: at device 1.1 (no driver attached) fwohci0: <1394 Open Host Controller Interface> mem 0xfa5ff800-0xfa5fffff,0xfa5f8000-0xfa5fbfff irq 23 at device 1.2 on pci1 fwohci0: [ITHREAD] fwohci0: OHCI version 1.10 (ROM=0) fwohci0: No. of Isochronous channels is 4. fwohci0: EUI64 00:02:3c:01:53:00:aa:8d fwohci0: Phy 1394a available S400, 2 ports. fwohci0: Link S400, max_rec 2048 bytes. firewire0: on fwohci0 dcons_crom0: on firewire0 dcons_crom0: bus_addr 0x207c000 fwe0: on firewire0 if_fwe0: Fake Ethernet address: 02:02:3c:00:aa:8d fwe0: Ethernet address: 02:02:3c:00:aa:8d fwip0: on firewire0 fwip0: Firewire address: 00:02:3c:01:53:00:aa:8d @ 0xfffe00000000, S400, maxrec 2048 sbp0: on firewire0 fwohci0: Initiate bus reset fwohci0: fwohci_intr_core: BUS reset fwohci0: fwohci_intr_core: node_id=0x00000000, SelfID Count=1, CYCLEMASTER mode fwohci1: mem 0xfa5ff000-0xfa5ff7ff,0xfa5f4000-0xfa5f7fff irq 21 at device 3.0 on pci1 fwohci1: [ITHREAD] fwohci1: OHCI version 1.10 (ROM=1) fwohci1: No. of Isochronous channels is 4. fwohci1: EUI64 00:11:d8:00:00:9a:71:28 fwohci1: Phy 1394a available S400, 2 ports. fwohci1: Link S400, max_rec 2048 bytes. firewire1: on fwohci1 dcons_crom1: on firewire1 dcons_crom1: bus_addr 0x207c000 dcons_crom1: dcons_paddr is already set fwe1: on firewire1 if_fwe1: Fake Ethernet address: 02:11:d8:9a:71:28 fwe1: Ethernet address: 02:11:d8:9a:71:28 fwip1: on firewire1 fwip1: Firewire address: 00:11:d8:00:00:9a:71:28 @ 0xfffe00000000, S400, maxrec 2048 sbp1: on firewire1 fwohci1: Initiate bus reset fwohci1: fwohci_intr_core: BUS reset fwohci1: fwohci_intr_core: node_id=0x00000000, SelfID Count=1, CYCLEMASTER mode isab0: at device 31.0 on pci0 isa0: on isab0 atapci1: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xffa0-0xffaf at device 31.1 on pci0 ata0: on atapci1 ata0: [ITHREAD] ahci1: port 0xe400-0xe407,0xe080-0xe083,0xe000-0xe007,0xdc00-0xdc03,0xd880-0xd88f mem 0xfebfb800-0xfebfbbff irq 23 at device 31.2 on pci0 ahci1: [ITHREAD] ahci1: AHCI v1.10 with 4 3Gbps ports, Port Multiplier not supported ahcich4: at channel 0 on ahci1 ahcich4: [ITHREAD] ahcich5: at channel 1 on ahci1 ahcich5: [ITHREAD] ahcich6: at channel 2 on ahci1 ahcich6: [ITHREAD] ahcich7: at channel 3 on ahci1 ahcich7: [ITHREAD] pci0: at device 31.3 (no driver attached) cpu0 on motherboard est0: on cpu0 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 112d0000112d device_attach: est0 attach returned 6 p4tcc0: on cpu0 pmtimer0 on isa0 unknown: can't assign resources (memory) atrtc0: at port 0x70-0x71 irq 8 pnpid PNP0b00 on isa0 fdc0: at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 pnpid PNP0700 on isa0 fdc0: [FILTER] ppc0: at port 0x378-0x37f,0x778-0x77a irq 7 drq 3 pnpid PNP0401 on isa0 ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/9 bytes threshold ppc0: [ITHREAD] ppbus0: on ppc0 plip0: on ppbus0 plip0: [ITHREAD] lpt0: on ppbus0 lpt0: [ITHREAD] lpt0: Interrupt-driven port ppi0: on ppbus0 unknown: can't assign resources (port) sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] uart0: <16550 or compatible> at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0 uart0: [FILTER] unknown: can't assign resources (memory) unknown: can't assign resources (port) est0: on cpu0 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 112d0000112d device_attach: est0 attach returned 6 ZFS NOTICE: Prefetch is disabled by default on i386 -- to enable, add "vfs.zfs.prefetch_disable=0" to /boot/loader.conf. ZFS WARNING: Recommended minimum kmem_size is 512MB; expect unstable behavior. Consider tuning vm.kmem_size and vm.kmem_size_max in /boot/loader.conf. ZFS filesystem version 4 ZFS storage pool version 15 Timecounter "TSC" frequency 3412089754 Hz quality 800 Timecounters tick every 1.000 msec firewire0: 1 nodes, maxhop <= 0 cable IRM irm(0) (me) firewire0: bus manager 0 firewire1: 1 nodes, maxhop <= 0 cable IRM irm(0) (me) firewire1: bus manager 0 usbus0: 12Mbps Full Speed USB v1.0 usbus1: 12Mbps Full Speed USB v1.0 usbus2: 12Mbps Full Speed USB v1.0 usbus3: 12Mbps Full Speed USB v1.0 usbus4: 480Mbps High Speed USB v2.0 ugen0.1: at usbus0 uhub0: on usbus0 ugen1.1: at usbus1 uhub1: on usbus1 ugen2.1: at usbus2 uhub2: on usbus2 ugen3.1: at usbus3 uhub3: on usbus3 ugen4.1: at usbus4 uhub4: on usbus4 acd0: DVDR at ata0-master UDMA66 uhub0: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered uhub2: 2 ports with 2 removable, self powered uhub3: 2 ports with 2 removable, self powered uhub4: 8 ports with 8 removable, self powered acd0: FAILURE - INQUIRY ILLEGAL REQUEST asc=0x24 ascq=0x00 sks=0x48 0x00 0x01 (probe14:ata0:0:0:0): TEST UNIT READY. CDB: 0 0 0 0 0 0 (probe14:ata0:0:0:0): CAM status: SCSI Status Error (probe14:ata0:0:0:0): SCSI status: Check Condition (probe14:ata0:0:0:0): SCSI sense: NOT READY asc:3a,0 (Medium not present) ada0 at ahcich4 bus 0 scbus6 target 0 lun 0 ada0: ATA-7 SATA 2.x device ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 238475MB (488397168 512 byte sectors: 16H 63S/T 16383C) cd0 at ata0 bus 0 scbus10 target 0 lun 0 cd0: Removable CD-ROM SCSI-0 device cd0: 66.000MB/s transfers cd0: Attempt to query device size failed: NOT READY, Medium not present Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 Root mount waiting for: usbus4 ugen4.2: at usbus4 (disconnected) uhub_reattach_port: could not allocate new device Root mount waiting for: usbus4 usbd_set_config_index: could not read device status: USB_ERR_SHORT_XFER ugen4.2: at usbus4 umass0: on usbus4 umass0: SCSI over Bulk-Only; quirks = 0x4000 Root mount waiting for: usbus4 umass0:11:0:-1: Attached to scbus11 da0 at umass-sim0 bus 0 scbus11 target 0 lun 0 da0: Fixed Direct Access SCSI-2 device da0: 40.000MB/s transfers da0: 476940MB (976773168 512 byte sectors: 255H 63S/T 60801C) Root mount waiting for: usbus4 uhub_reattach_port: port 4 reset failed, error=USB_ERR_TIMEOUT uhub_reattach_port: device problem (USB_ERR_TIMEOUT), disabling port 4 Trying to mount root from ufs:/dev/label/rootfs0 ugen1.2: at usbus1 uhub5: on usbus1 uhub5: 4 ports with 2 removable, bus powered WARNING: /var was not properly dismounted WARNING: /usr was not properly dismounted ugen1.3: at usbus1 ukbd0: on usbus1 kbd2 at ukbd0 uhid0: on usbus1 ugen1.4: at usbus1 ums0: on usbus1 ums0: 3 buttons and [XY] coordinates ID=0 ums1: on usbus1 ums1: 3 buttons and [XY] coordinates ID=0 WARNING: /var was not properly dismounted WARNING: /usr was not properly dismounted hdac0: mem 0xfebfc000-0xfebfffff irq 19 at device 27.0 on pci0 hdac0: HDA Driver Revision: 20100226_0142 hdac0: [ITHREAD] hdac0: HDA Codec #0: Realtek ALC882 pcm0: at cad 0 nid 1 on hdac0 pcm1: at cad 0 nid 1 on hdac0 pcm2: at cad 0 nid 1 on hdac0 pcm3: port 0x8880-0x88bf irq 22 at device 1.0 on pci1 pcm3: pcm3: [ITHREAD] From owner-freebsd-stable@FreeBSD.ORG Tue Jan 4 13:57:24 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1F7CB106566C; Tue, 4 Jan 2011 13:57:24 +0000 (UTC) (envelope-from petefrench@ticketswitch.com) Received: from constantine.ticketswitch.com (constantine.ticketswitch.com [IPv6:2002:57e0:1d4e:1::3]) by mx1.freebsd.org (Postfix) with ESMTP id D8F6D8FC17; Tue, 4 Jan 2011 13:57:23 +0000 (UTC) Received: from dilbert.rattatosk ([10.64.50.6] helo=dilbert.ticketswitch.com) by constantine.ticketswitch.com with esmtps (TLSv1:AES256-SHA:256) (Exim 4.69 (FreeBSD)) (envelope-from ) id 1Pa7O5-000LNN-OI; Tue, 04 Jan 2011 13:57:21 +0000 Received: from petefrench by dilbert.ticketswitch.com with local (Exim 4.72 (FreeBSD)) (envelope-from ) id 1Pa7O5-0008rH-NQ; Tue, 04 Jan 2011 13:57:21 +0000 Date: Tue, 04 Jan 2011 13:57:21 +0000 Message-Id: To: freebsd-current@freebsd.org, freebsd-stable@freebsd.org, luke-lists@hybrid-logic.co.uk In-Reply-To: <1293726772.5853.1348.camel@pow> From: Pete French Cc: Subject: Re: Virtio drivers for FreeBSD on KVM X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2011 13:57:24 -0000 > With more cloud infrastructure providers using KVM than ever before, the > importance of having FreeBSD performant as a guest on these > infrastructures [1], [2], [3] is increasing. It seems that using Virtio > drivers give a pretty significant performance boost [4], [5]. > > There was a NetBSD driver, and there seems to (have been) some work > happening to port this to DragonFly BSD at [6] and [7] -- does anyone > know if this code is stable, or if it has stalled, or if anyone's > working on it? Are the virtio devices provided by Linux KVM the same as those provided by VirtualBox ? Certainly their website says that the networking one is. How about giving the drivers provided by /usr/ports/emulators/virtualbox-ose-additions a try and see if they will work with KVM ? -pete. From owner-freebsd-stable@FreeBSD.ORG Tue Jan 4 13:58:06 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 911031065672; Tue, 4 Jan 2011 13:58:06 +0000 (UTC) (envelope-from petefrench@ticketswitch.com) Received: from constantine.ticketswitch.com (constantine.ticketswitch.com [IPv6:2002:57e0:1d4e:1::3]) by mx1.freebsd.org (Postfix) with ESMTP id 58BEC8FC13; Tue, 4 Jan 2011 13:58:06 +0000 (UTC) Received: from dilbert.rattatosk ([10.64.50.6] helo=dilbert.ticketswitch.com) by constantine.ticketswitch.com with esmtps (TLSv1:AES256-SHA:256) (Exim 4.69 (FreeBSD)) (envelope-from ) id 1Pa7On-000LSj-Ny; Tue, 04 Jan 2011 13:58:05 +0000 Received: from petefrench by dilbert.ticketswitch.com with local (Exim 4.72 (FreeBSD)) (envelope-from ) id 1Pa7On-0008rK-N6; Tue, 04 Jan 2011 13:58:05 +0000 Date: Tue, 04 Jan 2011 13:58:05 +0000 Message-Id: To: freebsd-current@freebsd.org, freebsd-stable@freebsd.org, luke-lists@hybrid-logic.co.uk In-Reply-To: <1293726772.5853.1348.camel@pow> From: Pete French Cc: Subject: Re: Virtio drivers for FreeBSD on KVM X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2011 13:58:06 -0000 Actually, it does look like virtio is more than just for networking... http://vbox.innotek.de/pipermail/vbox-dev/2009-November/002053.html From owner-freebsd-stable@FreeBSD.ORG Tue Jan 4 14:21:33 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E73B5106564A for ; Tue, 4 Jan 2011 14:21:33 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.16.84]) by mx1.freebsd.org (Postfix) with ESMTP id 9ED2E8FC12 for ; Tue, 4 Jan 2011 14:21:33 +0000 (UTC) Received: from pampa.cs.huji.ac.il ([132.65.80.32]) by kabab.cs.huji.ac.il with esmtp id 1Pa7lU-0009EF-1F for freebsd-stable@freebsd.org; Tue, 04 Jan 2011 16:21:32 +0200 X-Mailer: exmh version 2.7.2 01/07/2005 with nmh-1.2 To: freebsd-stable@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 04 Jan 2011 16:21:31 +0200 From: Daniel Braniss Message-ID: Subject: gstripe/gpart problems. X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2011 14:21:34 -0000 Hi, I have 2 ada disks striped: # gstripe list Geom name: s1 State: UP Status: Total=2, Online=2 Type: AUTOMATIC Stripesize: 65536 ID: 2442772675 Providers: 1. Name: stripe/s1 Mediasize: 1000215674880 (932G) Sectorsize: 512 Stripesize: 65536 Stripeoffset: 0 Mode: r0w0e0 Consumers: 1. Name: ada0 Mediasize: 500107862016 (466G) Sectorsize: 512 Mode: r0w0e0 Number: 0 2. Name: ada1 Mediasize: 500107862016 (466G) Sectorsize: 512 Mode: r0w0e0 Number: 1 boot complains: GEOM_STRIPE: Device s1 created (id=2442772675). GEOM_STRIPE: Disk ada0 attached to s1. GEOM: ada0: corrupt or invalid GPT detected. GEOM: ada0: GPT rejected -- may not be recoverable. GEOM_STRIPE: Disk ada1 attached to s1. GEOM_STRIPE: Device s1 activated. # gpart show => 34 1953546173 stripe/s1 GPT (932G) 34 128 1 freebsd-boot (64K) 162 1953546045 - free - (932G) # gpart show => 34 1953546173 stripe/s1 GPT (932G) 34 128 1 freebsd-boot (64K) 162 1953546045 - free - (932G) # gpart add -t freebsd-ufs -s 20g stripe/s1 GEOM: ada0: corrupt or invalid GPT detected. GEOM: ada0: GPT rejected -- may not be recoverable. stripe/s1p2 added # gpart show => 34 1953546173 stripe/s1 GPT (932G) 34 128 1 freebsd-boot (64K) 162 41943040 2 freebsd-ufs (20G) 41943202 1911603005 - free - (912G) if I go the MBR road, all seems ok, but as soon as I try to write the boot block (boot0cfg -B /dev/stripe/s1) again the kernel starts to complain about corrupted GEOM too. any ideas? thanks, danny From owner-freebsd-stable@FreeBSD.ORG Tue Jan 4 15:45:49 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 756D61065672 for ; Tue, 4 Jan 2011 15:45:49 +0000 (UTC) (envelope-from ben.pyttipanna@spooty.net) Received: from localhost.surfimpress.com (server217-174-248-30.live-servers.net [217.174.248.30]) by mx1.freebsd.org (Postfix) with ESMTP id 1D6848FC0C for ; Tue, 4 Jan 2011 15:45:48 +0000 (UTC) Received: from pyttipanna.hogsedge8.net (cpc1-brig12-0-0-cust381.3-3.cable.virginmedia.com [82.24.9.126]) by localhost.surfimpress.com (Postfix) with ESMTP id 3A5BB5DCE5 for ; Tue, 4 Jan 2011 15:15:35 +0000 (GMT) Message-ID: <4D2339CA.1010309@spooty.net> Date: Tue, 04 Jan 2011 15:16:26 +0000 From: ben paley User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.13) Gecko/20110103 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: 8.2-PRERELEASE and Flash X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2011 15:45:49 -0000 Hello, I've been using FreeBSD since 3.4 but I've been away for a while. Now I've put 8.2 on an old laptop, and everything's lovely except power management (I'll get round to that eventually) and the Flash plug in. I've followed the steps at http://www.freebsd.org/doc/handbook/desktop-browsers.html without any errors, except that neither Firefox nor any other browser will play flash movies. about:plugins doesn't show a flash plugin. I've spent a while googling and all I can find is variations on the instructions from the handbook. I don't know where to start looking for the problem. I'd be really grateful to whoever could point me in the right direction. Thanks a lot, Ben From owner-freebsd-stable@FreeBSD.ORG Tue Jan 4 15:54:53 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8DAA91065675 for ; Tue, 4 Jan 2011 15:54:53 +0000 (UTC) (envelope-from xaero@xaerolimit.net) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 2FBE98FC1F for ; Tue, 4 Jan 2011 15:54:52 +0000 (UTC) Received: by wwf26 with SMTP id 26so14231326wwf.31 for ; Tue, 04 Jan 2011 07:54:52 -0800 (PST) Received: by 10.216.174.65 with SMTP id w43mr24501950wel.95.1294156491807; Tue, 04 Jan 2011 07:54:51 -0800 (PST) MIME-Version: 1.0 Received: by 10.216.133.130 with HTTP; Tue, 4 Jan 2011 07:54:31 -0800 (PST) In-Reply-To: <4D2339CA.1010309@spooty.net> References: <4D2339CA.1010309@spooty.net> From: Chris Brennan Date: Tue, 4 Jan 2011 10:54:31 -0500 Message-ID: To: ben paley Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-stable@freebsd.org Subject: Re: 8.2-PRERELEASE and Flash X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2011 15:54:53 -0000 On Tue, Jan 4, 2011 at 10:16 AM, ben paley wrote: > Hello, > > I've been using FreeBSD since 3.4 but I've been away for a while. Now I've > put 8.2 on an old laptop, and everything's lovely except power management > (I'll get round to that eventually) and the Flash plug in. > > I've followed the steps at > http://www.freebsd.org/doc/handbook/desktop-browsers.html without any > errors, except that neither Firefox nor any other browser will play flash > movies. about:plugins doesn't show a flash plugin. > > I've spent a while googling and all I can find is variations on the > instructions from the handbook. I don't know where to start looking for the > problem. > > I'd be really grateful to whoever could point me in the right direction. > > Did you try symlinking nsplugin.so to $HOME/.mozilla/plugins ? this is usually what I end up doing to make it work. I forget if 64bit flash was fixed or not, if it wasn't you may need nswrapper or the like to run the 32bit plugin binaries. hth/c- From owner-freebsd-stable@FreeBSD.ORG Tue Jan 4 16:10:27 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 694631065674 for ; Tue, 4 Jan 2011 16:10:27 +0000 (UTC) (envelope-from marco@tolstoy.tols.org) Received: from tolstoy.tols.org (tolstoy.tols.org [IPv6:2a02:898:0:20::57:1]) by mx1.freebsd.org (Postfix) with ESMTP id F27118FC1C for ; Tue, 4 Jan 2011 16:10:26 +0000 (UTC) Received: from tolstoy.tols.org (localhost [127.0.0.1]) by tolstoy.tols.org (8.14.4/8.14.4) with ESMTP id p04GANlO013805 for ; Tue, 4 Jan 2011 16:10:23 GMT (envelope-from marco@tolstoy.tols.org) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.96.5 at tolstoy.tols.org Received: (from marco@localhost) by tolstoy.tols.org (8.14.4/8.14.4/Submit) id p04GANUm013804 for freebsd-stable@freebsd.org; Tue, 4 Jan 2011 17:10:23 +0100 (CET) (envelope-from marco) Date: Tue, 4 Jan 2011 17:10:23 +0100 From: Marco van Tol To: freebsd-stable@freebsd.org Message-ID: <20110104161023.GB9444@tolstoy.tols.org> Mail-Followup-To: freebsd-stable@freebsd.org References: <4D2339CA.1010309@spooty.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4D2339CA.1010309@spooty.net> User-Agent: Mutt/1.4.2.3i X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on tolstoy.tols.org Subject: Re: 8.2-PRERELEASE and Flash X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2011 16:10:27 -0000 On Tue, Jan 04, 2011 at 03:16:26PM +0000, ben paley wrote: > Hello, > > I've been using FreeBSD since 3.4 but I've been away for a while. Now > I've put 8.2 on an old laptop, and everything's lovely except power > management (I'll get round to that eventually) and the Flash plug in. > > I've followed the steps at > http://www.freebsd.org/doc/handbook/desktop-browsers.html without any > errors, except that neither Firefox nor any other browser will play > flash movies. about:plugins doesn't show a flash plugin. > > I've spent a while googling and all I can find is variations on the > instructions from the handbook. I don't know where to start looking for > the problem. > > I'd be really grateful to whoever could point me in the right direction. What I usually do is install the freebsd native firefox through /usr/ports/www/firefox, and plug flash into it using: /usr/ports/www/nspluginwrapper /usr/ports/www/linux-f10-flashplugin10 Further details about this can be found at: http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/desktop-browsers.html On that page search for "6.2.3 Firefox and Macromedia" I am currently using this on 8.2-prerelease, and have been using it this way for the passed year or so. It works great. The only caveat is that sometimes you have to "pkill npviewer". You'll know when to do that by seeing your browser become very sluggish. Doesn't happen as often as it did anymore. Marco -- Micro$oft: Where do you want to go today? Linux: Where do you want to go tomorrow? FreeBSD: Are you guys coming or what? From owner-freebsd-stable@FreeBSD.ORG Tue Jan 4 16:27:16 2011 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AC9DD106564A for ; Tue, 4 Jan 2011 16:27:16 +0000 (UTC) (envelope-from barbara.xxx1975@libero.it) Received: from cp-out2.libero.it (cp-out2.libero.it [212.52.84.102]) by mx1.freebsd.org (Postfix) with ESMTP id 406F58FC15 for ; Tue, 4 Jan 2011 16:27:15 +0000 (UTC) X-CTCH-Spam: Unknown X-CTCH-RefID: str=0001.0A0B0205.4D2346CE.0025,ss=1,re=0.000,fgs=0 X-libjamoibt: 1419 Received: from wmail66 (172.31.0.61) by cp-out2.libero.it (8.5.133) (authenticated as barbara.xxx1975@libero.it) id 4D10BE6500CA972D for freebsd-stable@FreeBSD.org; Tue, 4 Jan 2011 17:11:57 +0100 Message-ID: <15348198.1842201294157517944.JavaMail.defaultUser@defaultHost> Date: Tue, 4 Jan 2011 17:11:57 +0100 (CET) From: Barbara To: freebsd-stable@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain;charset="UTF-8" Content-Transfer-Encoding: 7bit X-SenderIP: 79.3.217.8 Cc: Subject: ums0 diconnections X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Barbara List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2011 16:27:16 -0000 Sometimes my mouse seems frozen, then it become responsive again after few seconds. The following lines are added in /var/log/messages: Jan 4 16:52:59 satanasso kernel: ugen1.2: at usbus1 (disconnected) Jan 4 16:52:59 satanasso kernel: ums0: at uhub1, port 2, addr 2 (disconnected) Jan 4 16:52:59 satanasso hald[1508]: 16:52:59.420 [W] hf-devd.c:379: malformed devd event: Jan 4 16:52:59 satanasso hald[1508]: 16:52:59.435 [W] hf-devd.c:379: malformed devd event: Jan 4 16:53:01 satanasso kernel: ugen1.2: at usbus1 Jan 4 16:53:01 satanasso kernel: ums0: on usbus1 Jan 4 16:53:01 satanasso kernel: ums0: 3 buttons and [XYZ] coordinates ID=0 Jan 4 16:53:02 satanasso hald[1508]: 16:53:02.439 [W] hf-devd.c:379: malformed devd event: Jan 4 16:53:02 satanasso hald[1508]: 16:53:02.440 [W] hf-devd.c:379: malformed devd event: As far as I can remember, this is happening since about a month. I don't think that it's a hw problem, could it be a sw one? BTW, I see similar error from hald when, for example, I plug in USB drives. Thanks Barbara From owner-freebsd-stable@FreeBSD.ORG Tue Jan 4 16:27:44 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D86771065695 for ; Tue, 4 Jan 2011 16:27:44 +0000 (UTC) (envelope-from alex-goncharov@comcast.net) Received: from qmta05.westchester.pa.mail.comcast.net (qmta05.westchester.pa.mail.comcast.net [76.96.62.48]) by mx1.freebsd.org (Postfix) with ESMTP id 992A68FC17 for ; Tue, 4 Jan 2011 16:27:44 +0000 (UTC) Received: from omta11.westchester.pa.mail.comcast.net ([76.96.62.36]) by qmta05.westchester.pa.mail.comcast.net with comcast id rTPC1f00C0mv7h055UEVG6; Tue, 04 Jan 2011 16:14:29 +0000 Received: from hanssachs.home ([24.61.85.144]) by omta11.westchester.pa.mail.comcast.net with comcast id rUEF1f00E36qgMk3XUEJAe; Tue, 04 Jan 2011 16:14:24 +0000 Received: from algo by hanssachs.home with local (Exim 4.72 (FreeBSD)) (envelope-from ) id 1Pa9WW-000AtS-VC; Tue, 04 Jan 2011 11:14:12 -0500 Date: Tue, 04 Jan 2011 11:14:12 -0500 Message-Id: From: Alex Goncharov To: ben paley In-reply-to: <4D2339CA.1010309@spooty.net> (message from ben paley on Tue, 04 Jan 2011 15:16:26 +0000) References: <4D2339CA.1010309@spooty.net> Sender: Alex Goncharov Cc: freebsd-stable@freebsd.org, freebsd-ports@freebsd.org Subject: Re: 8.2-PRERELEASE and Flash X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Alex Goncharov List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2011 16:27:44 -0000 [ Suggest not to cc: freebsd-stable@ from this point on; freebsd-ports@ is added ] ,--- You/ben (Tue, 04 Jan 2011 15:16:26 +0000) ----* | I've followed the steps at | http://www.freebsd.org/doc/handbook/desktop-browsers.html without any | errors, except that neither Firefox nor any other browser will play | flash movies. about:plugins doesn't show a flash plugin. | I've spent a while googling and all I can find is variations on the | instructions from the handbook. I don't know where to start looking for | the problem. I play Flash in Firefox, (native) Opera and Chrome -- perfectly now. For Firefox, the instructions in the Handbook worked for me. | I'd be really grateful to whoever could point me in the right direction. Search freebsd-ports@ for "opera", "flash" and my name -- I was a part of a conversation about it, some three months ago. Running Opera with -debugplugin helps. -- Alex -- alex-goncharov@comcast.net -- From owner-freebsd-stable@FreeBSD.ORG Tue Jan 4 16:32:29 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B2977106564A for ; Tue, 4 Jan 2011 16:32:29 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 5DF008FC18 for ; Tue, 4 Jan 2011 16:32:29 +0000 (UTC) Received: by qwj9 with SMTP id 9so14135966qwj.13 for ; Tue, 04 Jan 2011 08:32:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=FQulUjBO/lU3tWa9nCF/7eP/+KgIW99O9eZockt6U4Q=; b=FPVzxbdCxX/6oRPEmaIXB/FEhj2JekJC3q8aaMX64tHq0ceWVFqy+KUIEaSaCRYo83 U9diku+q2GCQXXJeIiM6N8HSRGnIAaGJPOPU6m8yeF4wYYiyZzg2mSN/LeTRFeJ7DI6G FB1JckMzSrtMn7evoNsVZM3Vk5T/f7HS7mx+c= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=DHUWG8IrpewyLMqCItEMAhADendftk/egv6F39SRH+AM0Jbz9zzoh1sE4jD6ay0dFg Dsur9TabZ/HMlHqQmRRNyakre03nufi4jk7peWhrowOvbXqf4ULAKyg0/ro6cRQDtWxJ hFeX9nKYnoL4NPcKwKTt4wpd8n6tgNdo6/wyY= MIME-Version: 1.0 Received: by 10.229.227.129 with SMTP id ja1mr13647739qcb.107.1294158748462; Tue, 04 Jan 2011 08:32:28 -0800 (PST) Received: by 10.229.97.145 with HTTP; Tue, 4 Jan 2011 08:32:28 -0800 (PST) In-Reply-To: References: <201101040037.59785.milu@dat.pl> <1703110029.56794.1294098600382.JavaMail.root@erie.cs.uoguelph.ca> Date: Tue, 4 Jan 2011 08:32:28 -0800 Message-ID: From: Freddie Cash To: Jean-Yves Avenard Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: Rick Macklem , freebsd-stable@freebsd.org, Marek Salwerowicz , Maciej Milewski Subject: Re: NFSv4 - how to set up at FreeBSD 8.1 ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2011 16:32:29 -0000 On Tue, Jan 4, 2011 at 3:42 AM, Jean-Yves Avenard wro= te: > On 4 January 2011 10:50, Rick Macklem wrote: > After reading this thread, I tried NFSv4 today.. > > Whenever I tried to mount from a linux client, I get: > =C2=A0mount -o vers=3D4 server4:/pool/backup/sites/m /mnt > NFS compound failed for server server4: error 7 (RPC: Authentication erro= r) > NFS compound failed for server server4: error 7 (RPC: Authentication erro= r) > NFS compound failed for server server4: error 7 (RPC: Authentication erro= r) > NFS compound failed for server server4: error 7 (RPC: Authentication erro= r) > NFS compound failed for server server4: error 7 (RPC: Authentication erro= r) > NFS compound failed for server server4: error 7 (RPC: Authentication erro= r) > nfs mount: mount: /mnt: Permission denied > > with NFS v3 it mounts just fine > > any ideas? NFSv4 mounts are relative to the filesystem being exported. NFSv3 mounts are absolute paths on the server. IOW, if you export /pool/backup/sites/m/ on the server, then the mount line on the client is just: mount -o vers=3D4 server4:/ /mnt If you export / on the server, then the mount line on the client would be: mount -o vers=3D4 server4:/pool/backup/sites/m/ /mnt This tripped me up when I tried converting my NFSv3 setup at home to NFSv4. --=20 Freddie Cash fjwcash@gmail.com From owner-freebsd-stable@FreeBSD.ORG Tue Jan 4 16:35:35 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1651B1065672 for ; Tue, 4 Jan 2011 16:35:35 +0000 (UTC) (envelope-from geo.liaskos@gmail.com) Received: from mail-yx0-f182.google.com (mail-yx0-f182.google.com [209.85.213.182]) by mx1.freebsd.org (Postfix) with ESMTP id C81D68FC1A for ; Tue, 4 Jan 2011 16:35:34 +0000 (UTC) Received: by yxh35 with SMTP id 35so6162611yxh.13 for ; Tue, 04 Jan 2011 08:35:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=AG4/X4LzlCN2t+1jFvtwcctIpSct//KG3M2+w4ADX8Y=; b=ovbBPAmTlejaMLazDHLrq25rBnyselWPPwdWP1LMRtdUdDQ7nhFjaaGKwc/KgbtqaZ 3zJ0M7TvV6Jnzq207lfS53c6329ZsmwUiL6exSFCw4gyLddWkaAp8Nbgy+FiQrvrrmTK 5zcDXlEtpkWNDDWCExEQ0s4aEm+aSpeQ3RDXg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=TGw2tlVxwIYd2uygJ5bt21AotxH13poTQtjQaFO1EaClPOQr4eTgROymTkqDjdRIdA pMcglSKJALBZ4cFyCta5pmD6nnM6unD8U1As8O4HzSnEodlKcZ6wR/04qbiOAFXM7fKt WKpi6waZff71thxnAUVybQTR73IaPdwbC/H4E= MIME-Version: 1.0 Received: by 10.100.138.16 with SMTP id l16mr12794043and.0.1294157094168; Tue, 04 Jan 2011 08:04:54 -0800 (PST) Received: by 10.100.226.5 with HTTP; Tue, 4 Jan 2011 08:04:54 -0800 (PST) In-Reply-To: <4D2339CA.1010309@spooty.net> References: <4D2339CA.1010309@spooty.net> Date: Tue, 4 Jan 2011 18:04:54 +0200 Message-ID: From: George Liaskos To: ben paley Content-Type: text/plain; charset=UTF-8 Cc: freebsd-stable@freebsd.org Subject: Re: 8.2-PRERELEASE and Flash X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2011 16:35:35 -0000 On Tue, Jan 4, 2011 at 5:16 PM, ben paley wrote: > Hello, > > I've been using FreeBSD since 3.4 but I've been away for a while. Now I've > put 8.2 on an old laptop, and everything's lovely except power management > (I'll get round to that eventually) and the Flash plug in. > > I've followed the steps at > http://www.freebsd.org/doc/handbook/desktop-browsers.html without any > errors, except that neither Firefox nor any other browser will play flash > movies. about:plugins doesn't show a flash plugin. > > I've spent a while googling and all I can find is variations on the > instructions from the handbook. I don't know where to start looking for the > problem. > > I'd be really grateful to whoever could point me in the right direction. > > Thanks a lot, > Ben One common mistake is running "nspluginwrapper -v -a -i" as root. From owner-freebsd-stable@FreeBSD.ORG Tue Jan 4 16:50:23 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 050771065695 for ; Tue, 4 Jan 2011 16:50:23 +0000 (UTC) (envelope-from fvhemert@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id 723008FC16 for ; Tue, 4 Jan 2011 16:50:22 +0000 (UTC) Received: by gyf3 with SMTP id 3so6063873gyf.13 for ; Tue, 04 Jan 2011 08:50:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=6WFLVR7OJbvZOx+0JwkripuYQJ/BbH2dZUZbp7BE8Qk=; b=Aj47aM/E7+Pk9prNRoGwa6VVEPs9bjGok/RpefSgSd/rghDCUQfO7gbnNV+1Uaj6/c htaBHfFf8571F/2Um85dvC6w2orNpLLsd9R3wws+xdiTnP1gAkNFJOHvMNDCz3J4ZUZG U/Rv/en/uHO5N2D2G6nbeMOniWZZ/rQ8sD3KI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=EXxbDcucwnmxJIIe1PfkrOzucK6oLeQ3IuYJxIVnlOG6vm8vAvZHVtEuDwQjN1zGD4 vATC5zf3+PTyRLXAO2j15JYxe0g/yjBZiFH8A7A/1c8dOzNZhQOd59nQYNCuL3SDGaWa y1GMWqciYyo3voYFJ12XdaoaEvVxpo0/hFb3k= MIME-Version: 1.0 Received: by 10.100.254.20 with SMTP id b20mr5737413ani.167.1294159821462; Tue, 04 Jan 2011 08:50:21 -0800 (PST) Received: by 10.101.5.31 with HTTP; Tue, 4 Jan 2011 08:50:21 -0800 (PST) In-Reply-To: <4D20629D.2030803@langille.org> References: <4D1AF388.3080107@infracaninophile.co.uk> <4D1B7431.7070808@infracaninophile.co.uk> <4D1BD8D0.5010402@langille.org> <4D1C4A2D.4020206@infracaninophile.co.uk> <4D1C7929.3040809@langille.org> <20101231233343.GB48579@server.vk2pj.dyndns.org> <20101231234747.GA8171@icarus.home.lan> <4D20629D.2030803@langille.org> Date: Tue, 4 Jan 2011 17:50:21 +0100 Message-ID: From: Freek van Hemert To: Dan Langille Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-stable@freebsd.org, Peter Jeremy , Jeremy Chadwick Subject: Re: slow ZFS on FreeBSD 8.1 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2011 16:50:23 -0000 Thank you mailinglist, That was a lot of info. First of all, I don't think this is a cpu issue since the cpu is mostly idle while copying. Second, I tried the hacks in loader.conf and the other ones mentioned in the beginning of the thread. Allthough perfomance is somewhat increased, it is still horribly slow at something like 2.6MB/sec (my internet is faster.) For the moment I will put my FreeBSD plans back in the fridge, I thought ZFS would bring me instant performance and data safety without having to spend too much time. I'll move back to Arch Linux and just setup a rsync between the disks on ext4 or perhaps later I will move to btrfs. When 8,2 is stable who knows I might just switch again although as the disks fill it will be harder and harder... I will definitely revisit FreeBSD and zfs but this was not the time for me. Have a happy new year and thanks for all the comments. Freek. On 2 January 2011 12:33, Dan Langille wrote: > On 12/31/2010 6:47 PM, Jeremy Chadwick wrote: > >> On Sat, Jan 01, 2011 at 10:33:43AM +1100, Peter Jeremy wrote: >> > > Based on my experiences at home, I converted my desktop at work to >>> pure ZFS. The only issues I've run into have been programs that >>> extensively use mmap(2) - which is a known issue with ZFS. >>> >> >> Is your ZFS root filesystem associated with a pool that's mirrored or >> using raidzX? What about mismatched /boot content (ZFS vs. UFS)? What >> about booting into single-user mode? >> >> http://wiki.freebsd.org/ZFSOnRoot indirectly hints at these problems but >> doesn't outright admit them (yet should), so I'm curious to know how >> people have solved them. Remembering manual "one-offs" for a system >> configured this way is not acceptable (read: highly prone to >> error/mistake). Is it worth the risk? Most administrators don't have >> the tolerance for stuff like that in the middle of a system upgrade or >> what not; they should be able to follow exactly what's in the handbook, >> to a tee. >> >> There's a link to www.dan.me.uk at the bottom of the above Wiki page >> that outlines "the madness" that's required to configure the setup, all >> of which has to be done by hand. I don't know many administrators who >> are going to tolerate this when deploying numerous machines, especially >> when compounded by the complexities mentioned above. >> > > This basically outlines the reason why I do not use ZFS on root. > > > -- > Dan Langille - http://langille.org/ > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > From owner-freebsd-stable@FreeBSD.ORG Tue Jan 4 17:09:22 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 636D110656C2 for ; Tue, 4 Jan 2011 17:09:22 +0000 (UTC) (envelope-from hirez@libeljournal.com) Received: from outbound-queue-1.mail.thdo.gradwell.net (outbound-queue-1.mail.thdo.gradwell.net [212.11.70.34]) by mx1.freebsd.org (Postfix) with ESMTP id 1ADE28FC1B for ; Tue, 4 Jan 2011 17:09:21 +0000 (UTC) Received: from outbound-edge-2.mail.thdo.gradwell.net (bonnie.gradwell.net [212.11.70.2]) by outbound-queue-1.mail.thdo.gradwell.net (Postfix) with ESMTP id 4EC5922127; Tue, 4 Jan 2011 16:50:22 +0000 (GMT) Received: from cpc2-chap5-0-0-cust256.aztw.cable.virginmedia.com (HELO propellor.libeljournal.com) (77.103.165.1) (smtp-auth username hirez, mechanism cram-md5) by outbound-edge-2.mail.thdo.gradwell.net (qpsmtpd/0.83) with (AES256-SHA encrypted) ESMTPSA; Tue, 04 Jan 2011 16:50:22 +0000 Received: from propellor.libeljournal.com (localhost [127.0.0.1]) by propellor.libeljournal.com (Postfix) with ESMTP id 1F0A817084; Tue, 4 Jan 2011 16:48:31 +0000 (GMT) X-Virus-Scanned: amavisd-new at libeljournal.com Received: from propellor.libeljournal.com ([127.0.0.1]) by propellor.libeljournal.com (propellor.libeljournal.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id A56KaZkiXYa5; Tue, 4 Jan 2011 16:48:27 +0000 (GMT) Received: from ballard.uk.futurenet.com (propellor.libeljournal.com [172.16.0.2]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by propellor.libeljournal.com (Postfix) with ESMTPSA id 0D41217079; Tue, 4 Jan 2011 16:48:27 +0000 (GMT) Message-ID: <4D23504D.8060103@libeljournal.com> Date: Tue, 04 Jan 2011 16:52:29 +0000 From: John Hawkes-Reed User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: Dan Langille References: <4D228F41.7040403@langille.org> In-Reply-To: <4D228F41.7040403@langille.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Gradwell-MongoId: 4d234fce.3c9-24cb-2 X-Gradwell-Auth-Method: smtpauth X-Gradwell-Auth-Credentials: hirez Cc: freebsd-stable Subject: Re: ZFS - hot spares : automatic or not? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2011 17:09:22 -0000 On 04/01/2011 03:08, Dan Langille wrote: > Hello folks, > > I'm trying to discover if ZFS under FreeBSD will automatically pull in a > hot spare if one is required. > > This raised the issue back in March 2010, and refers to a PR opened in > May 2009 > > * http://lists.freebsd.org/pipermail/freebsd-fs/2010-March/007943.html > * http://www.freebsd.org/cgi/query-pr.cgi?pr=134491 > > In turn, the PR refers to this March 2010 post referring to using devd > to accomplish this task. > > http://lists.freebsd.org/pipermail/freebsd-stable/2010-March/055686.html > > Does the above represent the the current state? > > I ask because I just ordered two more HDD to use as spares. Whether they > sit on the shelf or in the box is open to discussion. As far as our testing could discover, it's not automatic. I wrote some Ugly Perl that's called by devd when it spots a drive-fail event, which seemed to DTRT when simulating a failure by pulling a drive. -- JH-R From owner-freebsd-stable@FreeBSD.ORG Tue Jan 4 18:12:45 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 97D311065670 for ; Tue, 4 Jan 2011 18:12:45 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 2F9168FC1E for ; Tue, 4 Jan 2011 18:12:44 +0000 (UTC) Received: by wyf19 with SMTP id 19so14648762wyf.13 for ; Tue, 04 Jan 2011 10:12:43 -0800 (PST) Received: by 10.227.129.68 with SMTP id n4mr12726571wbs.25.1294164763622; Tue, 04 Jan 2011 10:12:43 -0800 (PST) Received: from dfleuriot.local ([83.167.62.196]) by mx.google.com with ESMTPS id 11sm15251789wbi.6.2011.01.04.10.12.42 (version=SSLv3 cipher=RC4-MD5); Tue, 04 Jan 2011 10:12:42 -0800 (PST) Message-ID: <4D236319.7060702@my.gd> Date: Tue, 04 Jan 2011 19:12:41 +0100 From: Damien Fleuriot User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-stable@freebsd.org References: <15348198.1842201294157517944.JavaMail.defaultUser@defaultHost> In-Reply-To: <15348198.1842201294157517944.JavaMail.defaultUser@defaultHost> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Subject: Re: ums0 diconnections X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2011 18:12:45 -0000 I would suggest the following 2 very quick tests: 1/ swap for another mouse, see if you have the problem with this one 2/ check mouse batteries if wireless, one never knows On 1/4/11 5:11 PM, Barbara wrote: > > > Sometimes my mouse seems frozen, then it become responsive again after few > seconds. > The following lines are added in /var/log/messages: > > Jan 4 16:52:59 satanasso kernel: ugen1.2: at usbus1 (disconnected) > Jan 4 16:52:59 satanasso kernel: ums0: at uhub1, port 2, addr 2 > (disconnected) > Jan 4 16:52:59 satanasso hald[1508]: 16:52:59.420 [W] hf-devd.c:379: > malformed devd event: > Jan 4 16:52:59 satanasso hald[1508]: 16:52:59.435 [W] hf-devd.c:379: > malformed devd event: > Jan 4 16:53:01 satanasso kernel: ugen1.2: at usbus1 > Jan 4 16:53:01 satanasso kernel: ums0: 0/0, rev 2.00/3.40, addr 2> on usbus1 > Jan 4 16:53:01 satanasso kernel: ums0: 3 buttons and [XYZ] coordinates ID=0 > Jan 4 16:53:02 satanasso hald[1508]: 16:53:02.439 [W] hf-devd.c:379: > malformed devd event: > Jan 4 16:53:02 satanasso hald[1508]: 16:53:02.440 [W] hf-devd.c:379: > malformed devd event: > > As far as I can remember, this is happening since about a month. > I don't think that it's a hw problem, could it be a sw one? > BTW, I see similar error from hald when, for example, I plug in USB drives. > > Thanks > Barbara > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@FreeBSD.ORG Tue Jan 4 18:52:08 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3842A106566C; Tue, 4 Jan 2011 18:52:08 +0000 (UTC) (envelope-from bra@fsn.hu) Received: from people.fsn.hu (people.fsn.hu [195.228.252.137]) by mx1.freebsd.org (Postfix) with ESMTP id 88BBB8FC15; Tue, 4 Jan 2011 18:52:06 +0000 (UTC) Received: by people.fsn.hu (Postfix, from userid 1001) id 011256B70F0; Tue, 4 Jan 2011 19:52:05 +0100 (CET) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000002, version=1.2.2 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MF-ACE0E1EA [pR: 20.6888] X-CRM114-CacheID: sfid-20110104_19520_4D33DCF3 X-CRM114-Status: Good ( pR: 20.6888 ) X-Spambayes-Classification: ham; 0.00 Message-ID: <4D236C54.6050309@fsn.hu> Date: Tue, 04 Jan 2011 19:52:04 +0100 From: Attila Nagy User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.23) Gecko/20090817 Thunderbird/2.0.0.23 Mnenhy/0.7.6.0 MIME-Version: 1.0 To: Bob Friesenhahn References: <4D0A09AF.3040005@FreeBSD.org> <4D1F7008.3050506@fsn.hu> <4D222B7B.1090902@fsn.hu> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, freebsd-stable@freebsd.org, Artem Belevich Subject: Re: New ZFSv28 patchset for 8-STABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2011 18:52:08 -0000 On 01/03/2011 10:35 PM, Bob Friesenhahn wrote: >>> >> After four days, the L2 hit rate is still hovering around 10-20 >> percents (was between 60-90), so I think it's clearly a regression in >> the ZFSv28 patch... >> And the massive growth in CPU usage can also very nicely be seen... >> >> I've updated the graphs at (switch time can be checked on the zfs-mem >> graph): >> http://people.fsn.hu/~bra/freebsd/20110101-zfsv28-fbsd/ >> >> There is a new phenomenom: the large IOPS peaks. I use this munin >> script on a lot of machines and never seen anything like this... I'm >> not sure whether it's related or not. > > It is not so clear that there is a problem. I am not sure what you > are using this server for but it is wise The IO pattern has changed radically, so for me it's a problem. > to consider that this is the funny time when a new year starts, SPAM > delivery goes through the roof, and employees and customers behave > differently. You chose the worst time of the year to implement the > change and observe behavior. It's a free software mirror, ftp.fsn.hu, and I'm sure that it's (the very low hit rate and the increased CPU usage) not related to the time when I made the switch. > > CPU use is indeed increased somewhat. A lower loading of the l2arc is > not necessarily a problem. The l2arc is usually bandwidth limited > compared with main store so if bulk data can not be cached in RAM, > then it is best left in main store. A smarter l2arc algorithm could > put only the data producing the expensive IOPS (the ones requiring a > seek) in the l2arc, lessening the amount of data cached on the device. That would make sense, if I wouldn't have 100-120 IOPS (for 7k2 RPM disks, it's about their max, gstat tells me the same) on the disks, and as low as 10 percents of L2 hit rate. What's smarter? Having 60-90% hit rate from the SSDs and moving the slow disk heads less, or having 10-20 percent of hit rate and kill the disks with random IO? If you are right, ZFS tries to be too smart and falls on its face with this kind of workload. BTW, I've checked the v15-v28 patch for arc.c, and I can't see any L2ARC related change there. I'm not sure whether the hypothetical logic would be there, or a different file, I haven't read it end to end. From owner-freebsd-stable@FreeBSD.ORG Tue Jan 4 19:12:24 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 667D6106564A for ; Tue, 4 Jan 2011 19:12:24 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id 162F68FC13 for ; Tue, 4 Jan 2011 19:12:23 +0000 (UTC) Received: by gyf3 with SMTP id 3so6113661gyf.13 for ; Tue, 04 Jan 2011 11:12:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=y1dy+pVFWr7sn6qoasZgO8tBENvlVQkNfM8oFqs6ado=; b=b3tknkCII14ESiNbRZPShQgkN3fclnCt0q3/sd/66/ewCeN1bV8qTdTKd+ONH8JyUn jGK9ndswPUDnndk8Sb1GBjwpYrBWBipnpAE6FFFjkr6ZU93bY8wyhcOvL+kK8o6vX/e8 cpPraZjvoG+eM+Or6WOhTOiI+2tQtDgP+r1go= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=qf6XhYa/i4CLJ9OZyJfZnSOm3w5ZE0aOsJk1/cajodiRDwmvCkfMyiXIW7YpfaM/T6 sX9Hufyg5Zd+pvi1NZiLUJgaEw8V46k0oQCydUCvuoVT+3QNsF8SRYZFziLCXk+M7yln M5AHAh8Ekob+p2K3r7vZtAT3sf21eg0vu8HCE= Received: by 10.236.109.146 with SMTP id s18mr3627018yhg.28.1294168343316; Tue, 04 Jan 2011 11:12:23 -0800 (PST) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id f73sm13223915yhc.4.2011.01.04.11.12.20 (version=TLSv1/SSLv3 cipher=RC4-MD5); Tue, 04 Jan 2011 11:12:21 -0800 (PST) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Tue, 4 Jan 2011 11:11:41 -0800 From: Pyun YongHyeon Date: Tue, 4 Jan 2011 11:11:41 -0800 To: "Michael L. Squires" Message-ID: <20110104191141.GE8448@michelle.cdnetworks.com> References: <20101127184952.E90087@familysquires.net> <20101128160043.W11452@familysquires.net> <20101129023531.GB1380@michelle.cdnetworks.com> <20101220181005.P32987@familysquires.net> <20101221180543.GB5236@michelle.cdnetworks.com> <20101222135420.C47756@familysquires.net> <20101230181239.N2744@familysquires.net> <20101230235022.GA78602@icarus.home.lan> <20110101153603.X21482@familysquires.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110101153603.X21482@familysquires.net> User-Agent: Mutt/1.4.2.3i Cc: freebsd-stable@freebsd.org, Jeremy Chadwick Subject: Re: bge driver regression in 7.4-PRERELEASE, Tyan S4881 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2011 19:12:24 -0000 On Sat, Jan 01, 2011 at 03:43:09PM -0500, Michael L. Squires wrote: > > > On Thu, 30 Dec 2010, Jeremy Chadwick wrote: > > >Please provide output from the following command, as root: > > > >pciconf -lbvc > > > >And only include the bge1 and bge0 devices in your output. Thanks. > > > > This is the output, as root, using the kernel with the 10/7/2010 bge code > (which works for me). I can provide the code with the 7.4-PRERELEASE > kernel if you want that. > > OS is compiled as amd64. > > bge0@pci0:17:2:0: class=0x020000 card=0x164814e4 chip=0x164814e4 > rev=0x03 > hdr=0x00 > vendor = 'Broadcom Corporation' > device = 'NetXtreme Dual Gigabit Adapter (BCM5704)' > class = network > subclass = ethernet > bar [10] = type Memory, range 64, base 0xd0110000, size 65536, enabled > bar [18] = type Memory, range 64, base 0xd0100000, size 65536, enabled > cap 07[40] = PCI-X 64-bit supports 133MHz, 2048 burst read, 1 split > transact > ion > cap 01[48] = powerspec 2 supports D0 D3 current D0 > cap 03[50] = VPD > cap 05[58] = MSI supports 8 messages, 64 bit > bge1@pci0:17:2:1: class=0x020000 card=0x164814e4 chip=0x164814e4 > rev=0x03 > hdr=0x00 > vendor = 'Broadcom Corporation' > device = 'NetXtreme Dual Gigabit Adapter (BCM5704)' > class = network > subclass = ethernet > bar [10] = type Memory, range 64, base 0xd0130000, size 65536, enabled > bar [18] = type Memory, range 64, base 0xd0120000, size 65536, enabled > cap 07[40] = PCI-X 64-bit supports 133MHz, 2048 burst read, 1 split > transact > ion > cap 01[48] = powerspec 2 supports D0 D3 current D0 > cap 03[50] = VPD > cap 05[58] = MSI supports 8 messages, 64 bit > > This is a hobby system supporting a home server, > so it's not "mission-critical" and my current hack is working properly. > > Thanks to both of you for your assistance. > FYI: Patch committed to HEAD(r216970). From owner-freebsd-stable@FreeBSD.ORG Tue Jan 4 20:20:35 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 482EC1065673 for ; Tue, 4 Jan 2011 20:20:35 +0000 (UTC) (envelope-from cliftonr@oz.volcano.org) Received: from oz.volcano.org (oz.volcano.org [64.65.105.124]) by mx1.freebsd.org (Postfix) with ESMTP id D481B8FC12 for ; Tue, 4 Jan 2011 20:20:34 +0000 (UTC) Received: by oz.volcano.org (Postfix, from userid 1001) id 174F15081F; Tue, 4 Jan 2011 10:01:49 -1000 (HST) Date: Tue, 4 Jan 2011 10:01:49 -1000 From: Clifton Royston To: Daniel Braniss Message-ID: <20110104200149.GA33991@lava.net> Mail-Followup-To: Daniel Braniss , freebsd-stable@freebsd.org References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: freebsd-stable@freebsd.org Subject: Re: gstripe/gpart problems. X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2011 20:20:35 -0000 On Tue, Jan 04, 2011 at 04:21:31PM +0200, Daniel Braniss wrote: > Hi, > I have 2 ada disks striped: > > # gstripe list > Geom name: s1 > State: UP > Status: Total=2, Online=2 > Type: AUTOMATIC > Stripesize: 65536 > ID: 2442772675 > Providers: > 1. Name: stripe/s1 > Mediasize: 1000215674880 (932G) > Sectorsize: 512 > Stripesize: 65536 > Stripeoffset: 0 > Mode: r0w0e0 > Consumers: > 1. Name: ada0 > Mediasize: 500107862016 (466G) > Sectorsize: 512 > Mode: r0w0e0 > Number: 0 > 2. Name: ada1 > Mediasize: 500107862016 (466G) > Sectorsize: 512 > Mode: r0w0e0 > Number: 1 > > boot complains: > > GEOM_STRIPE: Device s1 created (id=2442772675). > GEOM_STRIPE: Disk ada0 attached to s1. > GEOM: ada0: corrupt or invalid GPT detected. > GEOM: ada0: GPT rejected -- may not be recoverable. > GEOM_STRIPE: Disk ada1 attached to s1. > GEOM_STRIPE: Device s1 activated. > > # gpart show > => 34 1953546173 stripe/s1 GPT (932G) > 34 128 1 freebsd-boot (64K) > 162 1953546045 - free - (932G) > # gpart show > => 34 1953546173 stripe/s1 GPT (932G) > 34 128 1 freebsd-boot (64K) > 162 1953546045 - free - (932G) > > # gpart add -t freebsd-ufs -s 20g stripe/s1 > GEOM: ada0: corrupt or invalid GPT detected. > GEOM: ada0: GPT rejected -- may not be recoverable. > stripe/s1p2 added > # gpart show > => 34 1953546173 stripe/s1 GPT (932G) > 34 128 1 freebsd-boot (64K) > 162 41943040 2 freebsd-ufs (20G) > 41943202 1911603005 - free - (912G) > > if I go the MBR road, all seems ok, but as soon as I try to write > the boot block (boot0cfg -B /dev/stripe/s1) again the kernel > starts to complain about corrupted GEOM too. So are you trying to partition the drives and then stripe the partitions within the drives, or are you trying to partition the stripe? It seems here as though you might be trying to first partition the drives (not clear on that) then stripe the whole drives - which will mean the partition info is wrong for the resulting striped drive set - and then repartition the striped drive set, and neither is ending up valid. If what you are intending is to partition after striping the raw drives, then you are doing the right steps, but when the geom layer tries to look at the info on the individual drives as at boot, it will find it invalid. If it the gpart layer is actually refusing to write partition info to the drives which is wrong for the drives taken individually, that would account for your problems. One valid order to do things in would be partition the drives with gpart, creating identical sets of partitions on both drives, then stripe the partitions created within them (syntax not exact): gpart add -t freebsd-ufs0 -s 10g ada0 gpart add -t freebsd-ufs1 -s 10g ada1 gstripe label freebsd-ufs freebsd-ufs0 freebsd-ufs1 That would give you a 20GB stripe, with valid partition info on each drive. If this will be your boot drive, depending on how much needs to be read from the drive before the geom_stripe kernel module gets loaded, I would think there could also be a problem booting from the drive. This is not like gmirroring two drives or partitions, where the info read from either disk early in boot will be identical, and identical (except for the last block of the partition) to what the OS sees later after the mirror is formed. I assume you're bearing in mind that if you lose either drive to a hardware fault you lose the whole thing, and consider the risk worth the potential speed/size gain. -- Clifton -- Clifton Royston -- cliftonr@iandicomputing.com / cliftonr@lava.net President - I and I Computing * http://www.iandicomputing.com/ Custom programming, network design, systems and network consulting services From owner-freebsd-stable@FreeBSD.ORG Tue Jan 4 20:40:48 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4B8091065670 for ; Tue, 4 Jan 2011 20:40:48 +0000 (UTC) (envelope-from danallen46@airwired.net) Received: from mail.utahbroadband.com (mail.utahbroadband.com [204.14.20.91]) by mx1.freebsd.org (Postfix) with ESMTP id 32DF68FC14 for ; Tue, 4 Jan 2011 20:40:47 +0000 (UTC) Received: (qmail 9295 invoked by uid 89); 4 Jan 2011 20:40:47 -0000 Received: from unknown (HELO ?192.168.0.18?) (danallen46@airwired.net@66.29.174.6) by mail.utahbroadband.com with ESMTPA; 4 Jan 2011 20:40:47 -0000 From: Dan Allen Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Date: Tue, 4 Jan 2011 13:40:45 -0700 Message-Id: To: FreeBSD-STABLE Mailing List Mime-Version: 1.0 (Apple Message framework v1082) X-Mailer: Apple Mail (2.1082) Subject: Build Broken: /usr/src/usr.bin/netstat/inet.c X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2011 20:40:48 -0000 I just did a csup of stable, and the build is broken. In function protopr various struct members are not defined. The build halts. First compile error is at /usr/src/usr.bin/netstat/inet.c line 462 Dan From owner-freebsd-stable@FreeBSD.ORG Tue Jan 4 21:51:12 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B4CB6106567A for ; Tue, 4 Jan 2011 21:51:12 +0000 (UTC) (envelope-from lists@jnielsen.net) Received: from ns1temp.jnielsen.net (ns1temp.jnielsen.net [69.55.230.42]) by mx1.freebsd.org (Postfix) with ESMTP id 77FC08FC2C for ; Tue, 4 Jan 2011 21:51:12 +0000 (UTC) Received: from jnielsen.socialserve.com ([12.249.176.26]) (authenticated bits=0) by ns1temp.jnielsen.net (8.14.3/8.14.3) with ESMTP id p04LJFZF094489 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Tue, 4 Jan 2011 16:19:16 -0500 (EST) (envelope-from lists@jnielsen.net) Mime-Version: 1.0 (Apple Message framework v1082) Content-Type: text/plain; charset=us-ascii From: John Nielsen In-Reply-To: Date: Tue, 4 Jan 2011 16:19:10 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Dan Allen X-Mailer: Apple Mail (2.1082) X-DCC-sonic.net-Metrics: ns1temp.jnielsen.net; whitelist X-Virus-Scanned: clamav-milter 0.96.3 at ns1temp.jnielsen.net X-Virus-Status: Clean Cc: FreeBSD-STABLE Mailing List Subject: Re: Build Broken: /usr/src/usr.bin/netstat/inet.c X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2011 21:51:12 -0000 On Jan 4, 2011, at 3:40 PM, Dan Allen wrote: > I just did a csup of stable, and the build is broken. >=20 > In function protopr various struct members are not defined. The build = halts. >=20 > First compile error is at /usr/src/usr.bin/netstat/inet.c line 462 Me too. It seems r216964 is the culprit. See my response on the SVN = lists. JN= From owner-freebsd-stable@FreeBSD.ORG Tue Jan 4 23:30:23 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5BEB11065670 for ; Tue, 4 Jan 2011 23:30:23 +0000 (UTC) (envelope-from marek_sal@wp.pl) Received: from mx3.wp.pl (mx3.wp.pl [212.77.101.7]) by mx1.freebsd.org (Postfix) with ESMTP id DEA638FC17 for ; Tue, 4 Jan 2011 23:30:22 +0000 (UTC) Received: (wp-smtpd smtp.wp.pl 23018 invoked from network); 5 Jan 2011 00:30:14 +0100 Received: from cwx170.internetdsl.tpnet.pl (HELO marekdesktop) (marek_sal@[83.19.131.170]) (envelope-sender ) by smtp.wp.pl (WP-SMTPD) with SMTP for ; 5 Jan 2011 00:30:14 +0100 Message-ID: From: "Marek Salwerowicz" To: "Freddie Cash" , "Jean-Yves Avenard" References: <201101040037.59785.milu@dat.pl><1703110029.56794.1294098600382.JavaMail.root@erie.cs.uoguelph.ca> Date: Wed, 5 Jan 2011 00:29:53 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="UTF-8"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.5994 X-WP-AV: skaner antywirusowy poczty Wirtualnej Polski S. A. X-WP-SPAM: NO 0000000 [cSME] X-Mailman-Approved-At: Wed, 05 Jan 2011 00:05:13 +0000 Cc: Rick Macklem , freebsd-stable@freebsd.org, Maciej Milewski Subject: Re: NFSv4 - how to set up at FreeBSD 8.1 ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2011 23:30:23 -0000 Hi, so it slowly starts working: nfs4-server# cat /etc/exports V4: / / -maproot=root -network 192.168.183.0 -mask 255.255.255.0 nfs4-server# nfs4-server# ps aux | grep mountd root 857 0.0 0.6 3348 1520 ?? Is 11:53PM 0:00.01 /usr/sbin/mountd -e -r nfs4-server# ps aux | grep nfsd root 1303 0.0 0.5 3288 1324 ?? Is 12:04AM 0:00.03 nfsd: master (nfsd) root 1304 0.0 0.5 3288 1260 ?? S 12:04AM 0:00.02 nfsd: server (nfsd) nfs4-server# I am able to mount the root '/' from nfs4-server: nfs4-client# mount_nfs -o nfsv4 192.168.183.131:/ /marek_nfs4/ nfs4-client# ls /marek_nfs4/ .cshrc cdrom home proc usr .profile compat lib rescue var .snap dev libexec root COPYRIGHT dist media sbin bin entropy mnt sys boot etc plik tmp nfs4-client# mount /dev/ad0s1a on / (ufs, local) devfs on /dev (devfs, local, multilabel) /dev/ad0s1e on /tmp (ufs, local, soft-updates) /dev/ad0s1f on /usr (ufs, local, soft-updates) /dev/ad0s1d on /var (ufs, local, soft-updates) 192.168.183.131:/ on /marek_nfs4 (newnfs) nfs4-client# it works also on different partition: nfs4-server# cat /etc/exports V4: /usr /usr/home -maproot=root -network 192.168.183.0 -mask 255.255.255.0 nfs4-server# nfs4-client# mount_nfs -o nfsv4 192.168.183.131:/home /marek_nfs4/ nfs4-client# ls /marek_nfs4/ marek nfs4-client# What I noticed is that in 'V4: ' line we have to specify the mount point of the WHOLE partition we want to export Regards -- Marek Salwerowicz From owner-freebsd-stable@FreeBSD.ORG Wed Jan 5 01:02:20 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B85B6106564A for ; Wed, 5 Jan 2011 01:02:20 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 75CCF8FC17 for ; Wed, 5 Jan 2011 01:02:20 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApwEAFxOI02DaFvO/2dsb2JhbACDd6EprluOGYEggzZ0BIRohh8 X-IronPort-AV: E=Sophos;i="4.60,275,1291611600"; d="scan'208";a="104290998" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 04 Jan 2011 20:02:19 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 5A6FFB3F55; Tue, 4 Jan 2011 20:02:19 -0500 (EST) Date: Tue, 4 Jan 2011 20:02:19 -0500 (EST) From: Rick Macklem To: Marek Salwerowicz Message-ID: <1036681015.111502.1294189339355.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - IE8 (Win)/6.0.10_GA_2692) Cc: freebsd-stable@freebsd.org, Maciej Milewski , Jean-Yves Avenard Subject: Re: NFSv4 - how to set up at FreeBSD 8.1 ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 01:02:20 -0000 > Hi, > > so it slowly starts working: > > nfs4-server# cat /etc/exports > V4: / > / -maproot=root -network 192.168.183.0 -mask 255.255.255.0 > nfs4-server# > > nfs4-server# ps aux | grep mountd > root 857 0.0 0.6 3348 1520 ?? Is 11:53PM 0:00.01 > /usr/sbin/mountd -e -r > nfs4-server# ps aux | grep nfsd > root 1303 0.0 0.5 3288 1324 ?? Is 12:04AM 0:00.03 nfsd: master > (nfsd) > root 1304 0.0 0.5 3288 1260 ?? S 12:04AM 0:00.02 nfsd: server > (nfsd) > nfs4-server# > > > I am able to mount the root '/' from nfs4-server: > > nfs4-client# mount_nfs -o nfsv4 192.168.183.131:/ /marek_nfs4/ > nfs4-client# ls /marek_nfs4/ > .cshrc cdrom home proc usr > .profile compat lib rescue var > .snap dev libexec root > COPYRIGHT dist media sbin > bin entropy mnt sys > boot etc plik tmp > nfs4-client# mount > /dev/ad0s1a on / (ufs, local) > devfs on /dev (devfs, local, multilabel) > /dev/ad0s1e on /tmp (ufs, local, soft-updates) > /dev/ad0s1f on /usr (ufs, local, soft-updates) > /dev/ad0s1d on /var (ufs, local, soft-updates) > 192.168.183.131:/ on /marek_nfs4 (newnfs) > nfs4-client# > > it works also on different partition: > > nfs4-server# cat /etc/exports > V4: /usr > /usr/home -maproot=root -network 192.168.183.0 -mask 255.255.255.0 > nfs4-server# > > > nfs4-client# mount_nfs -o nfsv4 192.168.183.131:/home /marek_nfs4/ > nfs4-client# ls /marek_nfs4/ > marek > nfs4-client# > > > What I noticed is that in 'V4: ' line we have to specify the mount > point of > the WHOLE partition we want to export > Yes, the NFSv4 protocol does not use the mount protocol (mountd) and only handles a single exported tree (with the root defined at the location specified by the "V4:" line). The protocol has an Op called Put Root File Handle, which sets the RPC to the location of the root and then Lookup Ops traverse down from there. Early in NFSv4 development, one of the authors said "NFSv4 is NFS in name only" and that is fairly accurate, imho. For example, one of the fundamental principals for NFSv2, 3 was a stateless server, whereas NFSv4 uses a statefull server and does lock state recovery after a server crash. rick From owner-freebsd-stable@FreeBSD.ORG Wed Jan 5 01:09:36 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D0C24106566B for ; Wed, 5 Jan 2011 01:09:36 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 905BF8FC08 for ; Wed, 5 Jan 2011 01:09:36 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApwEAFxOI02DaFvO/2dsb2JhbACDd6EprluOGYEggzZ0BIRohh8 X-IronPort-AV: E=Sophos;i="4.60,275,1291611600"; d="scan'208";a="104291481" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 04 Jan 2011 20:09:35 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id BFBCAB3F23; Tue, 4 Jan 2011 20:09:35 -0500 (EST) Date: Tue, 4 Jan 2011 20:09:35 -0500 (EST) From: Rick Macklem To: Marek Salwerowicz Message-ID: <241056373.111669.1294189775780.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - IE8 (Win)/6.0.10_GA_2692) Cc: freebsd-stable@freebsd.org, Maciej Milewski , Jean-Yves Avenard Subject: Re: NFSv4 - how to set up at FreeBSD 8.1 ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 01:09:36 -0000 > Hi, > > so it slowly starts working: > > nfs4-server# cat /etc/exports > V4: / > / -maproot=root -network 192.168.183.0 -mask 255.255.255.0 > nfs4-server# > > nfs4-server# ps aux | grep mountd > root 857 0.0 0.6 3348 1520 ?? Is 11:53PM 0:00.01 > /usr/sbin/mountd -e -r > nfs4-server# ps aux | grep nfsd > root 1303 0.0 0.5 3288 1324 ?? Is 12:04AM 0:00.03 nfsd: master > (nfsd) > root 1304 0.0 0.5 3288 1260 ?? S 12:04AM 0:00.02 nfsd: server > (nfsd) > nfs4-server# > > > I am able to mount the root '/' from nfs4-server: > > nfs4-client# mount_nfs -o nfsv4 192.168.183.131:/ /marek_nfs4/ > nfs4-client# ls /marek_nfs4/ > .cshrc cdrom home proc usr > .profile compat lib rescue var > .snap dev libexec root > COPYRIGHT dist media sbin > bin entropy mnt sys > boot etc plik tmp > nfs4-client# mount > /dev/ad0s1a on / (ufs, local) > devfs on /dev (devfs, local, multilabel) > /dev/ad0s1e on /tmp (ufs, local, soft-updates) > /dev/ad0s1f on /usr (ufs, local, soft-updates) > /dev/ad0s1d on /var (ufs, local, soft-updates) > 192.168.183.131:/ on /marek_nfs4 (newnfs) > nfs4-client# > > it works also on different partition: > > nfs4-server# cat /etc/exports > V4: /usr > /usr/home -maproot=root -network 192.168.183.0 -mask 255.255.255.0 > nfs4-server# > > > nfs4-client# mount_nfs -o nfsv4 192.168.183.131:/home /marek_nfs4/ > nfs4-client# ls /marek_nfs4/ > marek > nfs4-client# > You can also do the following: For /etc/exports V4: / /usr/home -maproot=root -network 192.168.183.0 -mask 255.255.255.0 Then mount: # mount_nfs -o nfsv4 192.168.183.131:/usr/home /marek_nfs4/ (But only if the file system for "/" is ufs and not zfs and, admittedly there was a debate that has to be continued someday that might make it necessary to export "/" as well for ufs like zfs requires.) rick ps: And some NFSv4 clients can cross server mount points, unlike NFSv2, 3. From owner-freebsd-stable@FreeBSD.ORG Wed Jan 5 01:41:57 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 677BB106564A for ; Wed, 5 Jan 2011 01:41:57 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 2763E8FC08 for ; Wed, 5 Jan 2011 01:41:56 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApwEAFxOI02DaFvO/2dsb2JhbACDd6EprluOGYEggzZ0BIRohh+GEg X-IronPort-AV: E=Sophos;i="4.60,275,1291611600"; d="scan'208";a="104293329" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 04 Jan 2011 20:41:56 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 3D00FB3F2A; Tue, 4 Jan 2011 20:41:56 -0500 (EST) Date: Tue, 4 Jan 2011 20:41:56 -0500 (EST) From: Rick Macklem To: Freddie Cash Message-ID: <1248031564.112426.1294191716178.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - IE8 (Win)/6.0.10_GA_2692) Cc: Marek Salwerowicz , freebsd-stable@freebsd.org, Maciej Milewski , Jean-Yves Avenard Subject: Re: NFSv4 - how to set up at FreeBSD 8.1 ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 01:41:57 -0000 > On Tue, Jan 4, 2011 at 3:42 AM, Jean-Yves Avenard > wrote: > > On 4 January 2011 10:50, Rick Macklem wrote: > > After reading this thread, I tried NFSv4 today.. > > > > Whenever I tried to mount from a linux client, I get: > > =C2=A0mount -o vers=3D4 server4:/pool/backup/sites/m /mnt > > NFS compound failed for server server4: error 7 (RPC: Authentication > > error) > > NFS compound failed for server server4: error 7 (RPC: Authentication > > error) > > NFS compound failed for server server4: error 7 (RPC: Authentication > > error) > > NFS compound failed for server server4: error 7 (RPC: Authentication > > error) > > NFS compound failed for server server4: error 7 (RPC: Authentication > > error) > > NFS compound failed for server server4: error 7 (RPC: Authentication > > error) > > nfs mount: mount: /mnt: Permission denied > > > > with NFS v3 it mounts just fine > > > > any ideas? >=20 > NFSv4 mounts are relative to the filesystem being exported. NFSv3 > mounts are absolute paths on the server. >=20 Well, they are actually relative to where the NFSv4 root is specified in the V4: line. For ZFS, the entire server file tree from that point down must be exported (each volume requiring at least one line in /etc/expo= rts). For UFS, it can traverse down to the exported volume, but only by specifying the path to the exported volume in the mount command. (This may someday change, since it is questionable that it should have different behaviour than ZFS and could be argued a security risk.) Since there can only be one root point specified by the "V4:" line, you can get to multiple volumes, but they must be within the subtree. For example, if the following three directories are roots of mounted volumes on the server: /usr /usr/home /usr/sub1 and the exports file looks like: /usr -maproot=3Droot -network 131.104.48.0 -mask 255.255.255.0 /usr/home -maproot=3Droot -network 131.104.48.0 -mask 255.255.255.0 /usr/sub1 -maproot=3Droot -network 131.104.48.0 -mask 255.255.255.0 V4: /usr Then anywhere within /usr, /usr/home and /usr/sub1 can be mounted, but the client specifies a path relative to /usr, such as: # mount -t nfs -o nfsv4 server:/home /home to mount /usr/home on the client. By using: V4: / you can make the client mount paths look like what would be done for NFSv3, but if the "/" file system isn't UFS, you must export it as well as all the others or it can't be traversed. rick > IOW, if you export /pool/backup/sites/m/ on the server, then the mount > line on the client is just: > mount -o vers=3D4 server4:/ /mnt >=20 > If you export / on the server, then the mount line on the client would > be: > mount -o vers=3D4 server4:/pool/backup/sites/m/ /mnt >=20 > This tripped me up when I tried converting my NFSv3 setup at home to > NFSv4. >=20 Just trying to clarify what was good useful information, rick From owner-freebsd-stable@FreeBSD.ORG Wed Jan 5 03:22:43 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3E5691065693 for ; Wed, 5 Jan 2011 03:22:43 +0000 (UTC) (envelope-from jyavenard@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 0418A8FC0C for ; Wed, 5 Jan 2011 03:22:42 +0000 (UTC) Received: by iyb26 with SMTP id 26so14151223iyb.13 for ; Tue, 04 Jan 2011 19:22:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=hHtmBxyKhXQBRx1wN6mCGWXRzHxaJos0v0JZEQQUgnE=; b=OI+rlb/s25FQ676VnYyg4DJAjJNQLeP2V4++4+1Z/rVCmxMnLjnWB5BZ1C4HWamVkA HbEyBIg+9wnraQtiXCRR5isgYpj+UI+9Si8BgGTdNdOam3qEFo3cEoMZYNXq0SzhY80H 4DnXl54iVjps2b4xW6KZKEuVk6/t8TanAS2x4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=tKEO0cyjCZBPJDLKgqrIfdbrluyBgmFWbdRdKffuQOskDAinafQGbpLnuzkd/arp0f c95HcFnwrtFrkWaMBNSzmVdZQq+RnOmBcCJ34JpFJ4I9gozwe8eZp5o0CXG3KH/cmHrC zbenRRCaJGAPj8dl01rfKT2A63aJ808AQEj1w= MIME-Version: 1.0 Received: by 10.42.170.199 with SMTP id g7mr22995420icz.62.1294197762388; Tue, 04 Jan 2011 19:22:42 -0800 (PST) Received: by 10.42.172.69 with HTTP; Tue, 4 Jan 2011 19:22:42 -0800 (PST) In-Reply-To: <241056373.111669.1294189775780.JavaMail.root@erie.cs.uoguelph.ca> References: <241056373.111669.1294189775780.JavaMail.root@erie.cs.uoguelph.ca> Date: Wed, 5 Jan 2011 14:22:42 +1100 Message-ID: From: Jean-Yves Avenard To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 Cc: Marek Salwerowicz , freebsd-stable@freebsd.org, Maciej Milewski Subject: Re: NFSv4 - how to set up at FreeBSD 8.1 ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 03:22:43 -0000 Hi On 5 January 2011 12:09, Rick Macklem wrote: > You can also do the following: > For /etc/exports > V4: / > /usr/home -maproot=root -network 192.168.183.0 -mask 255.255.255.0 > > Then mount: > # mount_nfs -o nfsv4 192.168.183.131:/usr/home /marek_nfs4/ > (But only if the file system for "/" is ufs and not zfs and, admittedly > there was a debate that has to be continued someday that might make it > necessary to export "/" as well for ufs like zfs requires.) > > rick > ps: And some NFSv4 clients can cross server mount points, unlike NFSv2, 3. > I've done that (exporting V4: /) but then when I mount a sub zfs filesystem (e.g. /pool/backup/sites/m) then it appears empty on the client. If I export /pool/backup/sites/m , then I see the content of the directory. Most of the sub-directory in /pool are actually zfs file system mounted. It is something I expected with NFSv3 .. but not with nfs v4. JY From owner-freebsd-stable@FreeBSD.ORG Wed Jan 5 06:02:25 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 21B32106564A for ; Wed, 5 Jan 2011 06:02:25 +0000 (UTC) (envelope-from labeachgeek@gmail.com) Received: from mail-ew0-f54.google.com (mail-ew0-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id A1AE58FC13 for ; Wed, 5 Jan 2011 06:02:24 +0000 (UTC) Received: by ewy24 with SMTP id 24so6991041ewy.13 for ; Tue, 04 Jan 2011 22:02:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:received :in-reply-to:references:date:message-id:subject:from:to:content-type; bh=vx7JQL2gSukLdta+gy0MTPzx5rBNoY5UeGvZe8rQnyM=; b=QAW9dDD87bQ+lCQa64VDnj/oEqEbqt1qXqf3psPXt7CIu0aOSa38JSg+LP6KKfj8kF aUew1OA7wKLqfneRLMcpbD2vxGDgd7MQHpVwE0YXupjB+zDe1HEcLXUS6D1ehOFWMnaC giIHqsC2fxb0720KIIsNml1R4xXyAnfarQHSI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=x1i+LIpJBYQAfx+A872rPmCk36ISO634y5xRhY7ARNcivi1B4RU3QHpg7jUE1LW7BZ HqH+Jp3yh3fs3Bu7LgEQbZOtmtj+4dpnOXXHlRA+SvHdHZdaIp1vJxx8jdkloZy7Nwu7 ymLBCROREcj1zVp9aJ4CWzS/3RmM5RRCPd4L8= MIME-Version: 1.0 Received: by 10.213.6.193 with SMTP id a1mr2690569eba.63.1294205868633; Tue, 04 Jan 2011 21:37:48 -0800 (PST) Received: by 10.213.32.205 with HTTP; Tue, 4 Jan 2011 21:37:48 -0800 (PST) Received: by 10.213.32.205 with HTTP; Tue, 4 Jan 2011 21:37:48 -0800 (PST) In-Reply-To: References: Date: Tue, 4 Jan 2011 23:37:48 -0600 Message-ID: From: Beach Geek To: "freebsd-stable@freebsd.org" Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: usb errors with 8 stable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 06:02:25 -0000 Compaq Presario 5xxx 2GHz FreeBSD 8 r216866 Tue Jan 4 2011 (rm -rf /usr/obj/* & make cleandir run before build) When plugging in devices that worked earlier, none ever show up as devices in /dev. (2 examples listed below with error messages) The devices work on an old Gateway box (450MHz) running FreeBSD 8 r216359 built Dec 11th (same build the laptop had). I loaded files on the thumb drives & Android phone using the Gateway before hitting the road. I'm not sure if it's the update or hardware failure. I only have the laptop, phone, and 4 usbdrives... no other hardware available. Messages from 2 of the devices when plugged in; no messages when unplugged. Device: Samsung m900 Android phone (Moment) usbus2: port reset timeout uhub_reattach_port: port 1 reset failed, error=USB_ERR_TIMEOUT uhub_reattach_port: device problem (USB_ERR_TIMEOUT), disabling port 1 Device: Verbatim 4GB thumb drive usb_alloc_device: set address 2 failed (USB_ERR_IOERROR, ignored) usbd_setup_device_desc: getting device descriptor at addr 2 failed, USB_ERR_IOERROR usbd_req_re_enumerate: addr=2, set address failed! (USB_ERR_IOERROR, ignored) usbd_setup_device_desc: getting device descriptor at addr 2 failed, USB_ERR_IOERROR ... above 2 messages repeated... ugen0.2: at usbus0 (disconnected) uhub_reattach_port: could not allocate new device Any pointers or links to what could have changed in stable or any other suggestions. There aren't any usb options in BIOS. I can provide more info, but please remember I'm using a 4inch screen and a keyboard that fits in one hand. ;-) That's one reason I didn't include pciconf or usbconfig. I'm hoping it was simply some commit I missed or a pointyhat mistake. Three weeks in the middle of nowhere with a hardware failure... won't get much done, but ... Android has games... doesn't it? ;-)) Thanks for any help, Beach Geek PS. Android does not detect the laptop being connected, except for getting power. From owner-freebsd-stable@FreeBSD.ORG Wed Jan 5 06:38:41 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BFADC106567A for ; Wed, 5 Jan 2011 06:38:41 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta02.westchester.pa.mail.comcast.net (qmta02.westchester.pa.mail.comcast.net [76.96.62.24]) by mx1.freebsd.org (Postfix) with ESMTP id 67B318FC17 for ; Wed, 5 Jan 2011 06:38:40 +0000 (UTC) Received: from omta14.westchester.pa.mail.comcast.net ([76.96.62.60]) by qmta02.westchester.pa.mail.comcast.net with comcast id rick1f0061HzFnQ52iegQl; Wed, 05 Jan 2011 06:38:40 +0000 Received: from koitsu.dyndns.org ([98.248.34.134]) by omta14.westchester.pa.mail.comcast.net with comcast id rieV1f00C2tehsa3aieYAJ; Wed, 05 Jan 2011 06:38:33 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id C160D9B427; Tue, 4 Jan 2011 22:38:27 -0800 (PST) Date: Tue, 4 Jan 2011 22:38:27 -0800 From: Jeremy Chadwick To: Beach Geek Message-ID: <20110105063827.GA28092@icarus.home.lan> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-stable@freebsd.org" Subject: Re: usb errors with 8 stable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 06:38:41 -0000 On Tue, Jan 04, 2011 at 11:37:48PM -0600, Beach Geek wrote: > Compaq Presario 5xxx 2GHz > FreeBSD 8 r216866 Tue Jan 4 2011 > (rm -rf /usr/obj/* & make cleandir run before build) > > When plugging in devices that worked earlier, none ever show up as devices > in /dev. (2 examples listed below with error messages) > The devices work on an old Gateway box (450MHz) running FreeBSD 8 r216359 > built Dec 11th (same build the laptop had). I loaded files on the thumb > drives & Android phone using the Gateway before hitting the road. > > I'm not sure if it's the update or hardware failure. I only have the laptop, > phone, and 4 usbdrives... no other hardware available. > > Messages from 2 of the devices when plugged in; no messages when unplugged. > Device: Samsung m900 Android phone (Moment) > usbus2: port reset timeout > uhub_reattach_port: port 1 reset failed, error=USB_ERR_TIMEOUT > uhub_reattach_port: device problem (USB_ERR_TIMEOUT), disabling port 1 > > Device: Verbatim 4GB thumb drive > usb_alloc_device: set address 2 failed (USB_ERR_IOERROR, ignored) > usbd_setup_device_desc: getting device descriptor at addr 2 failed, > USB_ERR_IOERROR > usbd_req_re_enumerate: addr=2, set address failed! (USB_ERR_IOERROR, > ignored) > usbd_setup_device_desc: getting device descriptor at addr 2 failed, > USB_ERR_IOERROR > ... above 2 messages repeated... > ugen0.2: at usbus0 (disconnected) > uhub_reattach_port: could not allocate new device > > Any pointers or links to what could have changed in stable or any other > suggestions. > > There aren't any usb options in BIOS. > > I can provide more info, but please remember I'm using a 4inch screen and a > keyboard that fits in one hand. ;-) > That's one reason I didn't include pciconf or usbconfig. I'm hoping it was > simply some commit I missed or a pointyhat mistake. Three weeks in the > middle of nowhere with a hardware failure... won't get much done, but ... > Android has games... doesn't it? ;-)) > > Thanks for any help, > Beach Geek > > PS. Android does not detect the laptop being connected, except for getting > power. I would start by reviewing the commits for RELENG_8 between the two timeframes and try to narrow down which commit may have caused your problem. http://www.freshbsd.org/?branch=RELENG_8&project=freebsd -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Wed Jan 5 09:02:04 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2166C1065674 for ; Wed, 5 Jan 2011 09:02:04 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id B5BFC8FC14 for ; Wed, 5 Jan 2011 09:02:02 +0000 (UTC) Received: by wyf19 with SMTP id 19so15257103wyf.13 for ; Wed, 05 Jan 2011 01:02:01 -0800 (PST) Received: by 10.216.182.212 with SMTP id o62mr21269992wem.52.1294218121398; Wed, 05 Jan 2011 01:02:01 -0800 (PST) Received: from [10.129.37.231] ([92.90.16.53]) by mx.google.com with ESMTPS id i80sm11029835wej.4.2011.01.05.01.01.59 (version=TLSv1/SSLv3 cipher=RC4-MD5); Wed, 05 Jan 2011 01:01:59 -0800 (PST) References: <4D1C6F90.3080206@my.gd> <4D21E679.80002@my.gd> In-Reply-To: <4D21E679.80002@my.gd> Mime-Version: 1.0 (iPhone Mail 8A293) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Message-Id: <84882169-0461-480F-8B4C-58E794BCC8E6@my.gd> X-Mailer: iPhone Mail (8A293) From: Damien Fleuriot Date: Wed, 5 Jan 2011 10:01:11 +0100 To: Damien Fleuriot Cc: "freebsd-stable@freebsd.org" Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 09:02:04 -0000 Hi again List, I'm not so sure about using raidz2 anymore, I'm concerned for the performanc= e. Basically I have 9x 1.5T sata drives. raidz2 and 2x raidz1 will provide the same capacity. Are there any cons against using 2x raidz1 instead of 1x raidz2 ? I plan on using a SSD drive for the OS, 40-64gb, with 15 for the system itse= lf and some spare. Is it worth using the free space for cache ? ZIL ? both ? @jean-yves : didn't you experience problems recently when using both ? --- Fleuriot Damien On 3 Jan 2011, at 16:08, Damien Fleuriot wrote: >=20 >=20 > On 1/3/11 2:17 PM, Ivan Voras wrote: >> On 12/30/10 12:40, Damien Fleuriot wrote: >>=20 >>> I am concerned that in the event a drive fails, I won't be able to >>> repair the disks in time before another actually fails. >>=20 >> An old trick to avoid that is to buy drives from different series or >> manufacturers (the theory is that identical drives tend to fail at the >> same time), but this may not be applicable if you have 5 drives in a >> volume :) Still, you can try playing with RAIDZ levels and probabilities.= >>=20 >=20 > That's sound advice, although one also hears that they should get > devices from the same vendor for maximum compatibility -.- >=20 >=20 > Ah well, next time ;) >=20 >=20 > A piece of advice I shall heed though is using 1% less capacity than > what the disks really provide, in case one day I have to swap a drive > and its replacement is a few kbytes smaller (thus preventing a rebuild). From owner-freebsd-stable@FreeBSD.ORG Wed Jan 5 09:37:01 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 55A74106566C for ; Wed, 5 Jan 2011 09:37:01 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.16.84]) by mx1.freebsd.org (Postfix) with ESMTP id 0AFDA8FC08 for ; Wed, 5 Jan 2011 09:37:00 +0000 (UTC) Received: from pampa.cs.huji.ac.il ([132.65.80.32]) by kabab.cs.huji.ac.il with esmtp id 1PaPnf-000N1w-2W; Wed, 05 Jan 2011 11:36:59 +0200 X-Mailer: exmh version 2.7.2 01/07/2005 with nmh-1.2 To: Clifton Royston In-reply-to: Your message of Tue, 4 Jan 2011 10:01:49 -1000 . Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Wed, 05 Jan 2011 11:36:59 +0200 From: Daniel Braniss Message-ID: Cc: freebsd-stable@freebsd.org Subject: Re: gstripe/gpart problems. X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 09:37:01 -0000 > On Tue, Jan 04, 2011 at 04:21:31PM +0200, Daniel Braniss wrote: > > Hi, > > I have 2 ada disks striped: > > > > # gstripe list > > Geom name: s1 > > State: UP > > Status: Total=2, Online=2 > > Type: AUTOMATIC > > Stripesize: 65536 > > ID: 2442772675 > > Providers: > > 1. Name: stripe/s1 > > Mediasize: 1000215674880 (932G) > > Sectorsize: 512 > > Stripesize: 65536 > > Stripeoffset: 0 > > Mode: r0w0e0 > > Consumers: > > 1. Name: ada0 > > Mediasize: 500107862016 (466G) > > Sectorsize: 512 > > Mode: r0w0e0 > > Number: 0 > > 2. Name: ada1 > > Mediasize: 500107862016 (466G) > > Sectorsize: 512 > > Mode: r0w0e0 > > Number: 1 > > > > boot complains: > > > > GEOM_STRIPE: Device s1 created (id=2442772675). > > GEOM_STRIPE: Disk ada0 attached to s1. > > GEOM: ada0: corrupt or invalid GPT detected. > > GEOM: ada0: GPT rejected -- may not be recoverable. > > GEOM_STRIPE: Disk ada1 attached to s1. > > GEOM_STRIPE: Device s1 activated. > > > > # gpart show > > => 34 1953546173 stripe/s1 GPT (932G) > > 34 128 1 freebsd-boot (64K) > > 162 1953546045 - free - (932G) > > # gpart show > > => 34 1953546173 stripe/s1 GPT (932G) > > 34 128 1 freebsd-boot (64K) > > 162 1953546045 - free - (932G) > > > > # gpart add -t freebsd-ufs -s 20g stripe/s1 > > GEOM: ada0: corrupt or invalid GPT detected. > > GEOM: ada0: GPT rejected -- may not be recoverable. > > stripe/s1p2 added > > # gpart show > > => 34 1953546173 stripe/s1 GPT (932G) > > 34 128 1 freebsd-boot (64K) > > 162 41943040 2 freebsd-ufs (20G) > > 41943202 1911603005 - free - (912G) > > > > if I go the MBR road, all seems ok, but as soon as I try to write > > the boot block (boot0cfg -B /dev/stripe/s1) again the kernel > > starts to complain about corrupted GEOM too. > > So are you trying to partition the drives and then stripe the > partitions within the drives, or are you trying to partition the > stripe? > > It seems here as though you might be trying to first partition the > drives (not clear on that) then stripe the whole drives - which will > mean the partition info is wrong for the resulting striped drive set - > and then repartition the striped drive set, and neither is ending up > valid. > > If what you are intending is to partition after striping the raw > drives, then you are doing the right steps, but when the geom layer > tries to look at the info on the individual drives as at boot, it will > find it invalid. If it the gpart layer is actually refusing to write > partition info to the drives which is wrong for the drives taken > individually, that would account for your problems. > > One valid order to do things in would be partition the drives with > gpart, creating identical sets of partitions on both drives, then > stripe the partitions created within them (syntax not exact): > > gpart add -t freebsd-ufs0 -s 10g ada0 > gpart add -t freebsd-ufs1 -s 10g ada1 > gstripe label freebsd-ufs freebsd-ufs0 freebsd-ufs1 > > That would give you a 20GB stripe, with valid partition info on each > drive. > > If this will be your boot drive, depending on how much needs to be read > from the drive before the geom_stripe kernel module gets loaded, I > would think there could also be a problem booting from the drive. This > is not like gmirroring two drives or partitions, where the info read > from either disk early in boot will be identical, and identical (except > for the last block of the partition) to what the OS sees later after > the mirror is formed. > > I assume you're bearing in mind that if you lose either drive to a > hardware fault you lose the whole thing, and consider the risk worth > the potential speed/size gain. > -- Clifton Hi Clifton, I was getting very frustrated yesterday, hence the cripted message, your response requieres some background :-) the box is a Sun Fire X2200, which has bays for 2 disks, (we have several of these) before the latest upgrade, the 2 disks were 'raided' via 'nVidia MediaShield' and appeared as ar0, when I upgraded to 8.2, it disappeared, since I had in the kernel config file ATA_CAM. So I starded fiddling with gstripe, which 'recoverd' the data. Next, since the kernel boot kept complaining abouf GEOM errors, (and not wanting to mislead the operators) I cleaned up the data, and started from scratch. the machine boots diskless, but I like to keep a root bootable partition just in case. the process was in every case the same, first the stripe, then gpart the stripe. btw, I know that if I loose a disk I lose everything. Also that I wont be able to boot from it (I can always boot via the net, and mount the root localy, or some other combination - USB, etc) But you gave me some ideas and will start experimenting soon. thanks, danny From owner-freebsd-stable@FreeBSD.ORG Wed Jan 5 11:01:24 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1D86C1065670 for ; Wed, 5 Jan 2011 11:01:24 +0000 (UTC) (envelope-from perryh@pluto.rain.com) Received: from agora.rdrop.com (agora.rdrop.com [IPv6:2607:f678:1010::34]) by mx1.freebsd.org (Postfix) with ESMTP id F08EA8FC15 for ; Wed, 5 Jan 2011 11:01:23 +0000 (UTC) Received: from agora.rdrop.com (66@localhost [127.0.0.1]) by agora.rdrop.com (8.13.1/8.12.7) with ESMTP id p05B1C3H069967 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 5 Jan 2011 03:01:12 -0800 (PST) (envelope-from perryh@pluto.rain.com) Received: (from uucp@localhost) by agora.rdrop.com (8.13.1/8.12.9/Submit) with UUCP id p05B1CMH069966; Wed, 5 Jan 2011 03:01:12 -0800 (PST) Received: from fbsd61 by pluto.rain.com (4.1/SMI-4.1-pluto-M2060407) id AA12608; Wed, 5 Jan 11 02:56:14 PST Date: Wed, 05 Jan 2011 02:55:53 -0800 From: perryh@pluto.rain.com To: marek_sal@wp.pl, rmacklem@uoguelph.ca Message-Id: <4d244e39.KoPcOoMaWed+H5De%perryh@pluto.rain.com> References: <1036681015.111502.1294189339355.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <1036681015.111502.1294189339355.JavaMail.root@erie.cs.uoguelph.ca> User-Agent: nail 11.25 7/29/05 Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org, milu@dat.pl, jyavenard@gmail.com Subject: Re: NFSv4 - how to set up at FreeBSD 8.1 ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 11:01:24 -0000 Rick Macklem wrote: > ... one of the fundamental principals for NFSv2, 3 was a stateless > server ... Only as long as UDP transport was used. Any NFS implementation that used TCP for transport had thereby abandoned the stateless server principle, since a TCP connection itself requires that state be maintained on both ends. From owner-freebsd-stable@FreeBSD.ORG Wed Jan 5 09:29:40 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CAF72106564A for ; Wed, 5 Jan 2011 09:29:40 +0000 (UTC) (envelope-from marek_sal@wp.pl) Received: from mx3.wp.pl (mx3.wp.pl [212.77.101.7]) by mx1.freebsd.org (Postfix) with ESMTP id 5B07F8FC17 for ; Wed, 5 Jan 2011 09:29:39 +0000 (UTC) Received: (wp-smtpd smtp.wp.pl 11687 invoked from network); 5 Jan 2011 10:29:36 +0100 Received: from cwx170.internetdsl.tpnet.pl (HELO marekdesktop) (marek_sal@[83.19.131.170]) (envelope-sender ) by smtp.wp.pl (WP-SMTPD) with SMTP for ; 5 Jan 2011 10:29:36 +0100 Message-ID: From: "Marek Salwerowicz" To: "Rick Macklem" References: <241056373.111669.1294189775780.JavaMail.root@erie.cs.uoguelph.ca> Date: Wed, 5 Jan 2011 10:29:15 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="utf-8"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.5994 X-WP-AV: skaner antywirusowy poczty Wirtualnej Polski S. A. X-WP-SPAM: NO 0000000 [oeMH] X-Mailman-Approved-At: Wed, 05 Jan 2011 12:17:03 +0000 Cc: freebsd-stable@freebsd.org, Maciej Milewski , Jean-Yves Avenard Subject: Re: NFSv4 - how to set up at FreeBSD 8.1 ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 09:29:40 -0000 > You can also do the following: > For /etc/exports > V4: / > /usr/home -maproot=root -network 192.168.183.0 -mask 255.255.255.0 Not in my configuration - '/' and '/usr' are different partitions (both UFS) -- Marek Salwerowicz From owner-freebsd-stable@FreeBSD.ORG Wed Jan 5 13:00:24 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1375A1065783 for ; Wed, 5 Jan 2011 13:00:24 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id D7A488FC13 for ; Wed, 5 Jan 2011 13:00:23 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 893DF46B3B; Wed, 5 Jan 2011 08:00:23 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id DC76B8A027; Wed, 5 Jan 2011 08:00:20 -0500 (EST) From: John Baldwin To: freebsd-stable@freebsd.org Date: Wed, 5 Jan 2011 07:57:07 -0500 User-Agent: KMail/1.13.5 (FreeBSD/7.3-CBSD-20101102; KDE/4.4.5; amd64; ; ) References: <1036681015.111502.1294189339355.JavaMail.root@erie.cs.uoguelph.ca> <4d244e39.KoPcOoMaWed+H5De%perryh@pluto.rain.com> In-Reply-To: <4d244e39.KoPcOoMaWed+H5De%perryh@pluto.rain.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201101050757.08116.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Wed, 05 Jan 2011 08:00:21 -0500 (EST) X-Virus-Scanned: clamav-milter 0.96.3 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.9 required=4.2 tests=BAYES_00 autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on bigwig.baldwin.cx Cc: jyavenard@gmail.com, marek_sal@wp.pl, perryh@pluto.rain.com, milu@dat.pl, rmacklem@uoguelph.ca Subject: Re: NFSv4 - how to set up at FreeBSD 8.1 ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 13:00:24 -0000 On Wednesday, January 05, 2011 5:55:53 am perryh@pluto.rain.com wrote: > Rick Macklem wrote: > > > ... one of the fundamental principals for NFSv2, 3 was a stateless > > server ... > > Only as long as UDP transport was used. Any NFS implementation that > used TCP for transport had thereby abandoned the stateless server > principle, since a TCP connection itself requires that state be > maintained on both ends. Not filesystem cache coherency state, only socket state. And even NFS UDP mounts maintain their own set of "socket" state to manage retries and retransmits for UDP RPCs. The filesystem is equally incoherent for both UDP and TCP NFSv[23] mounts. TCP did not change any of that. -- John Baldwin From owner-freebsd-stable@FreeBSD.ORG Wed Jan 5 13:13:21 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6967D1065693 for ; Wed, 5 Jan 2011 13:13:21 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 271138FC19 for ; Wed, 5 Jan 2011 13:13:20 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApwEALL8I02DaFvO/2dsb2JhbACDeKEisQGOE4Ehgzd0BIRohiI X-IronPort-AV: E=Sophos;i="4.60,278,1291611600"; d="scan'208";a="104327835" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 05 Jan 2011 08:13:20 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 2DF81B41AF; Wed, 5 Jan 2011 08:13:20 -0500 (EST) Date: Wed, 5 Jan 2011 08:13:20 -0500 (EST) From: Rick Macklem To: Marek Salwerowicz Message-ID: <1103592228.118408.1294233200171.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.200] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - IE8 (Win)/6.0.10_GA_2692) Cc: freebsd-stable@freebsd.org, Maciej Milewski , Jean-Yves Avenard Subject: Re: NFSv4 - how to set up at FreeBSD 8.1 ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 13:13:21 -0000 > > You can also do the following: > > For /etc/exports > > V4: / > > /usr/home -maproot=root -network 192.168.183.0 -mask 255.255.255.0 > > Not in my configuration - '/' and '/usr' are different partitions > (both UFS) > Hmm. Since entire volumes are exported for NFSv4, I can't remember if exporting a subtree of the volume works (I think it does, but??). However, I do know that if you change the /etc/exports for the above to: (note I also moved the V4: line to the end because at one time it was required to be at the end and I can't remember if that restriction is still enforced. Always check /var/log/messages after starting mountd with a modified /etc/exports and look for any messages related to problems with /etc/exports.) In other words, export the volume's mount point and put the V4: line at the end are changes that "might be required?". If you take a look at mountd.c, you'll understand why I have trouble remembering exactly what works and what doesn't.:-) /usr -maproot=root -network 192.168.183.0 -mask 255.255.255.0 V4: / then for the above situation: # mount -t nfs -o nfsv4 server:/ /mnt - will fail because "/" isn't exported however # mount -t nfs -o nfsv4 server:/usr /mnt - should work. If it doesn't work, it is not because /etc/exports are wrong. A small number of NFSv4 ops are allowed on non-exported UFS partitions so that "mount" can traverse the tree down to the mount point, but that mount point must be exported. When I did this I did not realize that ZFS did its own exporting and, as such, traveral of non-exported ZFS volumes doesn't work, because ZFS doesn't allow any operations on the non-exported volumes to work. At some point, there needs to be a debate w.r.t. inconsistent behaviour. The easiest fix is to disable the capability of traversal of non-exported UFS volumes. The downside of this is that it is harder to configure the single (sub)tree on the server that is needed for NFSv4. Have fun with it, rick From owner-freebsd-stable@FreeBSD.ORG Wed Jan 5 13:30:07 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 21E2D106566C; Wed, 5 Jan 2011 13:30:07 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id C108F8FC0A; Wed, 5 Jan 2011 13:30:06 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApwEAGIBJE2DaFvO/2dsb2JhbACDeKEisQCOD4Ehgzd0BIRohiI X-IronPort-AV: E=Sophos;i="4.60,278,1291611600"; d="scan'208";a="104328984" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 05 Jan 2011 08:30:05 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id D6C16B3F6D; Wed, 5 Jan 2011 08:30:05 -0500 (EST) Date: Wed, 5 Jan 2011 08:30:05 -0500 (EST) From: Rick Macklem To: John Baldwin Message-ID: <1870282066.118978.1294234205820.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <201101050757.08116.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - IE8 (Win)/6.0.10_GA_2692) Cc: freebsd-stable@freebsd.org, marek sal , perryh@pluto.rain.com, milu@dat.pl, jyavenard@gmail.com Subject: Re: NFSv4 - how to set up at FreeBSD 8.1 ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 13:30:07 -0000 > On Wednesday, January 05, 2011 5:55:53 am perryh@pluto.rain.com wrote: > > Rick Macklem wrote: > > > > > ... one of the fundamental principals for NFSv2, 3 was a stateless > > > server ... > > > > Only as long as UDP transport was used. Any NFS implementation that > > used TCP for transport had thereby abandoned the stateless server > > principle, since a TCP connection itself requires that state be > > maintained on both ends. > > Not filesystem cache coherency state, only socket state. And even NFS > UDP > mounts maintain their own set of "socket" state to manage retries and > retransmits for UDP RPCs. The filesystem is equally incoherent for > both UDP > and TCP NFSv[23] mounts. TCP did not change any of that. > Unfortunately even NFSv4 doesn't maintain cache coherency in general. The state it maintains/recovers after a server crash are opens/locks/delegations, but the opens are a Windows-like open share lock (can't remember the Windows/Samba term for them) and not a POSIX-like open. NFSv4 does tie cache coherency to file locking, so that clients will get a coherent view of file data for byte ranges they lock. The term stateless server refers to the fact that the server doesn't know anything about the file handling state in the client that needs to be recovered after a server crash (opens, locks, ...). When an NFSv2,3 server is rebooted, it normally knows nothing about what clients are mounted, what clients have files open, etc and just services RPCs as they come in. The design avoided the complexity of recovery after a crash but results in a non-POSIX compliant file system that can't do a good job of cache coherency, knows nothing about file locks, etc. (Sun did add a separate file locking protocol called the NLM or rpc.lockd if you prefer, but that protocol design was fundamentally flawed imho and, as such, using it is in the "your mileage may vary" category.) Further, since without any information about previous operations, retries of non-idempotent RPCs would cause weird failures, "soft state" in the form of a cache of recent RPCs (typically called the Duplicate Request Cache or DRC these days) was added, to avoid performing the non-idempotent operation twice. A server is not required to retain the contents of a DRC after a crash/reboot but some vendors with non-volatile RAM hardware may choose to do so in order to provide "closer to correct" behaviour after a server crash/reboot. rick From owner-freebsd-stable@FreeBSD.ORG Wed Jan 5 13:43:51 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 35EA51065670 for ; Wed, 5 Jan 2011 13:43:51 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id E7E638FC0A for ; Wed, 5 Jan 2011 13:43:50 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Al0GAKoEJE2DaFvO/2dsb2JhbACDeJIbjwexBI4QgSGDN3QEhGiGIg X-IronPort-AV: E=Sophos;i="4.60,278,1291611600"; d="scan'208";a="105973300" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 05 Jan 2011 08:43:50 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 25AB5B3F0F; Wed, 5 Jan 2011 08:43:50 -0500 (EST) Date: Wed, 5 Jan 2011 08:43:50 -0500 (EST) From: Rick Macklem To: perryh@pluto.rain.com Message-ID: <1229754447.119764.1294235030140.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <4d244e39.KoPcOoMaWed+H5De%perryh@pluto.rain.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - IE8 (Win)/6.0.10_GA_2692) Cc: marek sal , freebsd-stable@freebsd.org, milu@dat.pl, jyavenard@gmail.com Subject: Re: NFSv4 - how to set up at FreeBSD 8.1 ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 13:43:51 -0000 > Rick Macklem wrote: > > > ... one of the fundamental principals for NFSv2, 3 was a stateless > > server ... > > Only as long as UDP transport was used. Any NFS implementation that > used TCP for transport had thereby abandoned the stateless server > principle, since a TCP connection itself requires that state be > maintained on both ends. > You've seen the responses w.r.t. what stateless server referred to already. But you might find this "quirk" interesting... Early in the NFSv4 design, I suggested that handling of the server state (opens/locks/...) might be tied to the TCP connections (NFSv4 doesn't use UDP). Lets just say the idea "flew like a lead balloon". Lots of responses along the lines of "NFS should be separate for the RPC transport layer, etc. Then, several years later, they came up with Sessions for NFSv4.1, which does RPC transport management in a very NFSv4.1-specific way, including stuff like changing the RPC semantics to exactly-once... Although very different from what I had envisioned, in a sense Sessions does tie state handling (ordering of locking operations, for example) to RPC transport management as I currently understand it. rick From owner-freebsd-stable@FreeBSD.ORG Wed Jan 5 13:57:13 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 49162106566C for ; Wed, 5 Jan 2011 13:57:13 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 0923E8FC17 for ; Wed, 5 Jan 2011 13:57:12 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApwEAEkHJE2DaFvO/2dsb2JhbACDeKEjsQSOE4Ehgzd0BIRohiI X-IronPort-AV: E=Sophos;i="4.60,278,1291611600"; d="scan'208";a="104331360" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 05 Jan 2011 08:57:11 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 3A6F6B3F3B; Wed, 5 Jan 2011 08:57:12 -0500 (EST) Date: Wed, 5 Jan 2011 08:57:12 -0500 (EST) From: Rick Macklem To: Jean-Yves Avenard Message-ID: <641784536.120801.1294235832169.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.199] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - IE8 (Win)/6.0.10_GA_2692) Cc: Marek Salwerowicz , freebsd-stable@freebsd.org, Maciej Milewski Subject: Re: NFSv4 - how to set up at FreeBSD 8.1 ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 13:57:13 -0000 > Hi > > On 5 January 2011 12:09, Rick Macklem wrote: > > > You can also do the following: > > For /etc/exports > > V4: / > > /usr/home -maproot=root -network 192.168.183.0 -mask 255.255.255.0 > > > > Then mount: > > # mount_nfs -o nfsv4 192.168.183.131:/usr/home /marek_nfs4/ > > (But only if the file system for "/" is ufs and not zfs and, > > admittedly > > there was a debate that has to be continued someday that might make > > it > > necessary to export "/" as well for ufs like zfs requires.) > > > > rick > > ps: And some NFSv4 clients can cross server mount points, unlike > > NFSv2, 3. > > > > I've done that (exporting V4: /) > > but then when I mount a sub zfs filesystem (e.g. /pool/backup/sites/m) > then it appears empty on the client. > > If I export /pool/backup/sites/m , then I see the content of the > directory. > > Most of the sub-directory in /pool are actually zfs file system > mounted. > > It is something I expected with NFSv3 .. but not with nfs v4. > Yes, to access the file volumes via any version of NFS, they need to be exported. (I don't think it would make sense to allow access to all of the server's data without limitations for NFSv4?) What is different (and makes it confusing for folks familiar with NFSv2,3) is the fact that it is a single "mount tree" for NFSv4 that has to be rooted somewhere. Solaris10 - always roots it at "/" but somehow works around the ZFS case, so any exported share can be mounted with the same path used by NFSv2,3. Linux - Last I looked (which was a couple of years ago), it exported a single volume for NFSv4 and the rest of the server's volumes could only be accessed via NFSv2,3. (I don't know if they've changed this yet?) So, I chose to allow a little more flexibility than Solaris10 and allow /etc/exports to set the location of the "mount root". I didn't anticipate the "glitch" that ZFS introduced (where all ZFS volumes in the mount path must be exported for mount to work) because it does its own exporting. (Obviously, the glitch/inconsistency needs to be resolved at some point.) rick From owner-freebsd-stable@FreeBSD.ORG Wed Jan 5 14:56:03 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A11DA1065674 for ; Wed, 5 Jan 2011 14:56:03 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 626248FC25 for ; Wed, 5 Jan 2011 14:56:02 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApwEAB8VJE2DaFvO/2dsb2JhbACDd6EdsQuNV4Ehgzd0BIRohiI X-IronPort-AV: E=Sophos;i="4.60,278,1291611600"; d="scan'208";a="105981700" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 05 Jan 2011 09:56:01 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 9DD60B40B0; Wed, 5 Jan 2011 09:56:01 -0500 (EST) Date: Wed, 5 Jan 2011 09:56:01 -0500 (EST) From: Rick Macklem To: Jean-Yves Avenard Message-ID: <1097353631.125552.1294239361572.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <641784536.120801.1294235832169.JavaMail.root@erie.cs.uoguelph.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.199] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - IE8 (Win)/6.0.10_GA_2692) Cc: Marek Salwerowicz , freebsd-stable@freebsd.org, Maciej Milewski Subject: Re: NFSv4 - how to set up at FreeBSD 8.1 ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 14:56:03 -0000 > Yes, to access the file volumes via any version of NFS, they need to > be exported. (I don't think it would make sense to allow access to all > of the server's data without limitations for NFSv4?) > > What is different (and makes it confusing for folks familiar with > NFSv2,3) > is the fact that it is a single "mount tree" for NFSv4 that has to be > rooted > somewhere. > Solaris10 - always roots it at "/" but somehow works around the ZFS > case, > so any exported share can be mounted with the same path used > by NFSv2,3. > Linux - Last I looked (which was a couple of years ago), it exported a > single volume for NFSv4 and the rest of the server's volumes > could only be accessed via NFSv2,3. (I don't know if they've > changed this yet?) > > So, I chose to allow a little more flexibility than Solaris10 and > allow > /etc/exports to set the location of the "mount root". I didn't > anticipate > the "glitch" that ZFS introduced (where all ZFS volumes in the mount > path > must be exported for mount to work) because it does its own exporting. > (Obviously, the glitch/inconsistency needs to be resolved at some > point.) > Perhaps it would help to show what goes on the wire when a mount is done. # mount -t nfs -o nfsv4 server:/usr/home /mnt For NFSv2,3 there will be a Mount RPC with /usr/home as the argument. This goes directly to mountd and then mountd decides if it is ok and replies with the file handle (FH) for /usr/home if it is. For NFSv4, the client will do a compound RPC that looks something like this: (The exact structure is up to the client implementor.) PutRooFH <-- set the position to the "root mount location" as specified by the V4: line Lookup usr Lookup home GetFH <-- return the file handle at this location As such, there can only be one "root mount location" and at least Lookup operations must work for all elements of the path from there to the client's mount point. (For non-ZFS, it currently allows Lookup plus a couple of others that some clients use during mounting to work for non-exported file systems, so that setting "root mount location" == "/" works without exporting the entire file server's tree.) For all other operations, the file system must be exported just like for NFSv2,3. Hope this helps, rick From owner-freebsd-stable@FreeBSD.ORG Wed Jan 5 15:55:13 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1159B106564A for ; Wed, 5 Jan 2011 15:55:13 +0000 (UTC) (envelope-from cforgeron@acsi.ca) Received: from mta01.eastlink.ca (mta01.eastlink.ca [24.224.136.30]) by mx1.freebsd.org (Postfix) with ESMTP id CA4F78FC0C for ; Wed, 5 Jan 2011 15:55:12 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from ip04.eastlink.ca ([unknown] [24.222.39.52]) by mta01.eastlink.ca (Sun Java(tm) System Messaging Server 7u3-12.01 64bit (built Oct 15 2009)) with ESMTP id <0LEK00F0N47Z3S13@mta01.eastlink.ca> for freebsd-stable@freebsd.org; Wed, 05 Jan 2011 11:55:11 -0400 (AST) X-CMAE-Score: 0 X-CMAE-Analysis: v=1.1 cv=b0sI0M7bjhCmEOs51LbeKzGQ5ECIs9m+H5QCeOcUmtc= c=1 sm=1 a=tZDC3L0OECsA:10 a=kj9zAlcOel0A:10 a=6I5d2MoRAAAA:8 a=rIORHrRF1m2Q5aHVDYIA:9 a=O7Plqn6INxnUEzwNtqYA:7 a=7TZTqBoEkT9my9Q_mycHh42G1IkA:4 a=CjuIK1q_8ugA:10 a=SV7veod9ZcQA:10 a=Rb26mEfMWC2AlgGA:21 a=xT4QZSMC0MRh12XL:21 a=ZjIqTmGINkQKjhCx/60B3Q==:117 Received: from blk-222-10-85.eastlink.ca (HELO server7.acsi.ca) ([24.222.10.85]) by ip04.eastlink.ca with ESMTP; Wed, 05 Jan 2011 11:55:10 -0400 Received: from server7.acsi.ca ([192.168.9.7]) by server7.acsi.ca ([192.168.9.7]) with mapi; Wed, 05 Jan 2011 11:55:11 -0400 From: Chris Forgeron To: 'Damien Fleuriot' Date: Wed, 05 Jan 2011 11:55:10 -0400 Thread-topic: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks Thread-index: Acust0EEptjDHfiKS9Gx0g+Oy6u5TQAOGNeg Message-id: References: <4D1C6F90.3080206@my.gd> <4D21E679.80002@my.gd> <84882169-0461-480F-8B4C-58E794BCC8E6@my.gd> In-reply-to: <84882169-0461-480F-8B4C-58E794BCC8E6@my.gd> Accept-Language: en-US Content-language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Cc: "freebsd-stable@freebsd.org" Subject: RE: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 15:55:13 -0000 First off, raidz2 and raidz1 with copies=2 are not the same thing. raidz2 will give you two copies of parity instead of just one. It also guarantees that this parity is on different drives. You can sustain 2 drive failures without data loss. raidz1 with copies=2 will give you two copies of all your files, but there is no guarantee that they are on different drives, and you can still only sustain 1 drive failure. You'll have better space efficiency with raidz2 if you're using 9 drives. If I were you, I'd use your 9 disks as one big raidz, or better yet, get 10 disks, and make a stripe of 2 5 disk raidz's for the best performance. Save your SSD drive for the L2ARC (cache) or ZIL, you'll get better speed that way instead of throwing it away on a boot drive. -- -----Original Message----- From: owner-freebsd-stable@freebsd.org [mailto:owner-freebsd-stable@freebsd.org] On Behalf Of Damien Fleuriot Sent: January-05-11 5:01 AM To: Damien Fleuriot Cc: freebsd-stable@freebsd.org Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks Hi again List, I'm not so sure about using raidz2 anymore, I'm concerned for the performance. Basically I have 9x 1.5T sata drives. raidz2 and 2x raidz1 will provide the same capacity. Are there any cons against using 2x raidz1 instead of 1x raidz2 ? I plan on using a SSD drive for the OS, 40-64gb, with 15 for the system itself and some spare. Is it worth using the free space for cache ? ZIL ? both ? @jean-yves : didn't you experience problems recently when using both ? --- Fleuriot Damien On 3 Jan 2011, at 16:08, Damien Fleuriot wrote: > > > On 1/3/11 2:17 PM, Ivan Voras wrote: >> On 12/30/10 12:40, Damien Fleuriot wrote: >> >>> I am concerned that in the event a drive fails, I won't be able to >>> repair the disks in time before another actually fails. >> >> An old trick to avoid that is to buy drives from different series or >> manufacturers (the theory is that identical drives tend to fail at >> the same time), but this may not be applicable if you have 5 drives >> in a volume :) Still, you can try playing with RAIDZ levels and probabilities. >> > > That's sound advice, although one also hears that they should get > devices from the same vendor for maximum compatibility -.- > > > Ah well, next time ;) > > > A piece of advice I shall heed though is using 1% less capacity than > what the disks really provide, in case one day I have to swap a drive > and its replacement is a few kbytes smaller (thus preventing a rebuild). _______________________________________________ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@FreeBSD.ORG Wed Jan 5 17:15:00 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 265C61065696 for ; Wed, 5 Jan 2011 17:15:00 +0000 (UTC) (envelope-from cliftonr@oz.volcano.org) Received: from oz.volcano.org (oz.volcano.org [64.65.105.124]) by mx1.freebsd.org (Postfix) with ESMTP id E98F28FC19 for ; Wed, 5 Jan 2011 17:14:59 +0000 (UTC) Received: by oz.volcano.org (Postfix, from userid 1001) id A7A085081F; Wed, 5 Jan 2011 07:14:58 -1000 (HST) Date: Wed, 5 Jan 2011 07:14:58 -1000 From: Clifton Royston To: Daniel Braniss Message-ID: <20110105171458.GA43278@lava.net> Mail-Followup-To: Daniel Braniss , freebsd-stable@freebsd.org References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: freebsd-stable@freebsd.org Subject: Re: gstripe/gpart problems. X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 17:15:00 -0000 On Wed, Jan 05, 2011 at 11:36:59AM +0200, Daniel Braniss wrote: > Hi Clifton, > I was getting very frustrated yesterday, hence the cripted message, your > response requieres some background :-) > the box is a Sun Fire X2200, which has bays for 2 disks, (we have several of > these) > before the latest upgrade, the 2 disks were 'raided' via 'nVidia MediaShield' > and > appeared as ar0, when I upgraded to 8.2, it disappeared, since I had in the > kernel config file > ATA_CAM. So I starded fiddling with gstripe, which 'recoverd' the data. > Next, since the kernel boot kept complaining abouf GEOM errors, (and not > wanting to > mislead the operators) I cleaned up the data, and started from scratch. > the machine boots diskless, but I like to keep a root bootable partition just > in case. > the process was in every case the same, first the stripe, then gpart the > stripe. Thanks, that makes it very clear why things are as they are. Good to know that the booting issues are covered via diskless boot. I had never thought about being able to recover a RAID stripe using gstripe. That's a very interesting capability! Assuming that FreeBSD considers partitioning a stripe to be valid in principle - and you give reasons it should - then there may be a geom/driver interaction bug to investigate here if the geom layer is refusing to write a stripe-oriented partition to the raw drive. -- Clifton -- Clifton Royston -- cliftonr@iandicomputing.com / cliftonr@lava.net President - I and I Computing * http://www.iandicomputing.com/ Custom programming, network design, systems and network consulting services From owner-freebsd-stable@FreeBSD.ORG Wed Jan 5 21:34:26 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 570D51065679; Wed, 5 Jan 2011 21:34:26 +0000 (UTC) (envelope-from tinderbox@freebsd.org) Received: from freebsd-stable.sentex.ca (freebsd-stable.sentex.ca [IPv6:2607:f3e0:0:3::6502:9b]) by mx1.freebsd.org (Postfix) with ESMTP id 13DBD8FC12; Wed, 5 Jan 2011 21:34:25 +0000 (UTC) Received: from freebsd-stable.sentex.ca (localhost [127.0.0.1]) by freebsd-stable.sentex.ca (8.14.4/8.14.4) with ESMTP id p05LYPVx087568; Wed, 5 Jan 2011 21:34:25 GMT (envelope-from tinderbox@freebsd.org) Received: (from tinderbox@localhost) by freebsd-stable.sentex.ca (8.14.4/8.14.4/Submit) id p05LYPi5087554; Wed, 5 Jan 2011 21:34:25 GMT (envelope-from tinderbox@freebsd.org) Date: Wed, 5 Jan 2011 21:34:25 GMT Message-Id: <201101052134.p05LYPi5087554@freebsd-stable.sentex.ca> X-Authentication-Warning: freebsd-stable.sentex.ca: tinderbox set sender to FreeBSD Tinderbox using -f Sender: FreeBSD Tinderbox From: FreeBSD Tinderbox To: FreeBSD Tinderbox , , Precedence: bulk Cc: Subject: [releng_8 tinderbox] failure on i386/i386 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 21:34:26 -0000 TB --- 2011-01-05 20:34:44 - tinderbox 2.6 running on freebsd-stable.sentex.ca TB --- 2011-01-05 20:34:44 - starting RELENG_8 tinderbox run for i386/i386 TB --- 2011-01-05 20:34:44 - cleaning the object tree TB --- 2011-01-05 20:35:15 - cvsupping the source tree TB --- 2011-01-05 20:35:15 - /usr/bin/csup -z -r 3 -g -L 1 -h cvsup.sentex.ca /tinderbox/RELENG_8/i386/i386/supfile TB --- 2011-01-05 20:36:12 - building world TB --- 2011-01-05 20:36:12 - MAKEOBJDIRPREFIX=/obj TB --- 2011-01-05 20:36:12 - PATH=/usr/bin:/usr/sbin:/bin:/sbin TB --- 2011-01-05 20:36:12 - TARGET=i386 TB --- 2011-01-05 20:36:12 - TARGET_ARCH=i386 TB --- 2011-01-05 20:36:12 - TZ=UTC TB --- 2011-01-05 20:36:12 - __MAKE_CONF=/dev/null TB --- 2011-01-05 20:36:12 - cd /src TB --- 2011-01-05 20:36:12 - /usr/bin/make -B buildworld >>> World build started on Wed Jan 5 20:36:13 UTC 2011 >>> Rebuilding the temporary build tree >>> stage 1.1: legacy release compatibility shims >>> stage 1.2: bootstrap tools >>> stage 2.1: cleaning up the object tree >>> stage 2.2: rebuilding the object tree >>> stage 2.3: build tools >>> stage 3: cross tools >>> stage 4.1: building includes >>> stage 4.2: building libraries >>> stage 4.3: make dependencies >>> stage 4.4: building everything [...] gzip -cn /src/usr.bin/ncplogin/ncplogout.1 > ncplogout.1.gz ===> usr.bin/netstat (all) cc -O2 -pipe -fno-strict-aliasing -DIPSEC -DSCTP -DINET6 -DNETGRAPH -DIPX -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wno-uninitialized -Wno-pointer-sign -c /src/usr.bin/netstat/if.c cc -O2 -pipe -fno-strict-aliasing -DIPSEC -DSCTP -DINET6 -DNETGRAPH -DIPX -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wno-uninitialized -Wno-pointer-sign -c /src/usr.bin/netstat/inet.c cc1: warnings being treated as errors /src/usr.bin/netstat/inet.c: In function 'protopr': /src/usr.bin/netstat/inet.c:463: warning: format '%6u' expects type 'unsigned int', but argument 2 has type 'uint64_t' /src/usr.bin/netstat/inet.c:463: warning: format '%6u' expects type 'unsigned int', but argument 3 has type 'uint64_t' *** Error code 1 Stop in /src/usr.bin/netstat. *** Error code 1 Stop in /src/usr.bin. *** Error code 1 Stop in /src. *** Error code 1 Stop in /src. *** Error code 1 Stop in /src. TB --- 2011-01-05 21:34:25 - WARNING: /usr/bin/make returned exit code 1 TB --- 2011-01-05 21:34:25 - ERROR: failed to build world TB --- 2011-01-05 21:34:25 - 2527.38 user 365.25 system 3581.07 real http://tinderbox.freebsd.org/tinderbox-releng_8-RELENG_8-i386-i386.full From owner-freebsd-stable@FreeBSD.ORG Wed Jan 5 21:37:41 2011 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AE925106564A; Wed, 5 Jan 2011 21:37:41 +0000 (UTC) (envelope-from tinderbox@freebsd.org) Received: from freebsd-stable.sentex.ca (freebsd-stable.sentex.ca [IPv6:2607:f3e0:0:3::6502:9b]) by mx1.freebsd.org (Postfix) with ESMTP id 6B5EC8FC12; Wed, 5 Jan 2011 21:37:41 +0000 (UTC) Received: from freebsd-stable.sentex.ca (localhost [127.0.0.1]) by freebsd-stable.sentex.ca (8.14.4/8.14.4) with ESMTP id p05LbeOW018039; Wed, 5 Jan 2011 21:37:40 GMT (envelope-from tinderbox@freebsd.org) Received: (from tinderbox@localhost) by freebsd-stable.sentex.ca (8.14.4/8.14.4/Submit) id p05LbeUs018034; Wed, 5 Jan 2011 21:37:40 GMT (envelope-from tinderbox@freebsd.org) Date: Wed, 5 Jan 2011 21:37:40 GMT Message-Id: <201101052137.p05LbeUs018034@freebsd-stable.sentex.ca> X-Authentication-Warning: freebsd-stable.sentex.ca: tinderbox set sender to FreeBSD Tinderbox using -f Sender: FreeBSD Tinderbox From: FreeBSD Tinderbox To: FreeBSD Tinderbox , , Precedence: bulk Cc: Subject: [releng_8 tinderbox] failure on i386/pc98 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 21:37:41 -0000 TB --- 2011-01-05 20:39:14 - tinderbox 2.6 running on freebsd-stable.sentex.ca TB --- 2011-01-05 20:39:14 - starting RELENG_8 tinderbox run for i386/pc98 TB --- 2011-01-05 20:39:14 - cleaning the object tree TB --- 2011-01-05 20:39:38 - cvsupping the source tree TB --- 2011-01-05 20:39:38 - /usr/bin/csup -z -r 3 -g -L 1 -h cvsup.sentex.ca /tinderbox/RELENG_8/i386/pc98/supfile TB --- 2011-01-05 20:40:22 - building world TB --- 2011-01-05 20:40:22 - MAKEOBJDIRPREFIX=/obj TB --- 2011-01-05 20:40:22 - PATH=/usr/bin:/usr/sbin:/bin:/sbin TB --- 2011-01-05 20:40:22 - TARGET=pc98 TB --- 2011-01-05 20:40:22 - TARGET_ARCH=i386 TB --- 2011-01-05 20:40:22 - TZ=UTC TB --- 2011-01-05 20:40:22 - __MAKE_CONF=/dev/null TB --- 2011-01-05 20:40:22 - cd /src TB --- 2011-01-05 20:40:22 - /usr/bin/make -B buildworld >>> World build started on Wed Jan 5 20:40:23 UTC 2011 >>> Rebuilding the temporary build tree >>> stage 1.1: legacy release compatibility shims >>> stage 1.2: bootstrap tools >>> stage 2.1: cleaning up the object tree >>> stage 2.2: rebuilding the object tree >>> stage 2.3: build tools >>> stage 3: cross tools >>> stage 4.1: building includes >>> stage 4.2: building libraries >>> stage 4.3: make dependencies >>> stage 4.4: building everything [...] gzip -cn /src/usr.bin/ncplogin/ncplogout.1 > ncplogout.1.gz ===> usr.bin/netstat (all) cc -O2 -pipe -fno-strict-aliasing -DIPSEC -DSCTP -DINET6 -DNETGRAPH -DIPX -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wno-uninitialized -Wno-pointer-sign -c /src/usr.bin/netstat/if.c cc -O2 -pipe -fno-strict-aliasing -DIPSEC -DSCTP -DINET6 -DNETGRAPH -DIPX -std=gnu99 -fstack-protector -Wsystem-headers -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wno-uninitialized -Wno-pointer-sign -c /src/usr.bin/netstat/inet.c cc1: warnings being treated as errors /src/usr.bin/netstat/inet.c: In function 'protopr': /src/usr.bin/netstat/inet.c:463: warning: format '%6u' expects type 'unsigned int', but argument 2 has type 'uint64_t' /src/usr.bin/netstat/inet.c:463: warning: format '%6u' expects type 'unsigned int', but argument 3 has type 'uint64_t' *** Error code 1 Stop in /src/usr.bin/netstat. *** Error code 1 Stop in /src/usr.bin. *** Error code 1 Stop in /src. *** Error code 1 Stop in /src. *** Error code 1 Stop in /src. TB --- 2011-01-05 21:37:40 - WARNING: /usr/bin/make returned exit code 1 TB --- 2011-01-05 21:37:40 - ERROR: failed to build world TB --- 2011-01-05 21:37:40 - 2501.15 user 373.52 system 3506.43 real http://tinderbox.freebsd.org/tinderbox-releng_8-RELENG_8-i386-pc98.full From owner-freebsd-stable@FreeBSD.ORG Wed Jan 5 21:57:04 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DC37C106566B for ; Wed, 5 Jan 2011 21:57:04 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 74C2E8FC12 for ; Wed, 5 Jan 2011 21:57:04 +0000 (UTC) Received: by wyf19 with SMTP id 19so15946667wyf.13 for ; Wed, 05 Jan 2011 13:57:00 -0800 (PST) Received: by 10.227.132.83 with SMTP id a19mr14018559wbt.112.1294264618857; Wed, 05 Jan 2011 13:56:58 -0800 (PST) Received: from [10.134.136.45] ([92.90.16.16]) by mx.google.com with ESMTPS id i80sm11435853wej.4.2011.01.05.13.56.08 (version=TLSv1/SSLv3 cipher=RC4-MD5); Wed, 05 Jan 2011 13:56:57 -0800 (PST) References: <4D1C6F90.3080206@my.gd> <4D21E679.80002@my.gd> <84882169-0461-480F-8B4C-58E794BCC8E6@my.gd> In-Reply-To: Mime-Version: 1.0 (iPhone Mail 8A293) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Message-Id: <488AE93A-97B9-4F01-AD0A-0098E4B329C3@my.gd> X-Mailer: iPhone Mail (8A293) From: Damien Fleuriot Date: Wed, 5 Jan 2011 22:55:01 +0100 To: Chris Forgeron Cc: "freebsd-stable@freebsd.org" Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 21:57:04 -0000 Well actually... raidz2: - 7x 1.5 tb =3D 10.5tb - 2 parity drives raidz1: - 3x 1.5 tb =3D 4.5 tb - 4x 1.5 tb =3D 6 tb , total 10.5tb - 2 parity drives in split thus different raidz1 arrays So really, in both cases 2 different parity drives and same storage... --- Fleuriot Damien On 5 Jan 2011, at 16:55, Chris Forgeron wrote: > First off, raidz2 and raidz1 with copies=3D2 are not the same thing.=20 >=20 > raidz2 will give you two copies of parity instead of just one. It also gua= rantees that this parity is on different drives. You can sustain 2 drive fai= lures without data loss.=20 >=20 > raidz1 with copies=3D2 will give you two copies of all your files, but the= re is no guarantee that they are on different drives, and you can still only= sustain 1 drive failure. >=20 > You'll have better space efficiency with raidz2 if you're using 9 drives.=20= >=20 > If I were you, I'd use your 9 disks as one big raidz, or better yet, get 1= 0 disks, and make a stripe of 2 5 disk raidz's for the best performance.=20 >=20 > Save your SSD drive for the L2ARC (cache) or ZIL, you'll get better speed t= hat way instead of throwing it away on a boot drive.=20 >=20 > -- >=20 >=20 > -----Original Message----- > From: owner-freebsd-stable@freebsd.org [mailto:owner-freebsd-stable@freebs= d.org] On Behalf Of Damien Fleuriot > Sent: January-05-11 5:01 AM > To: Damien Fleuriot > Cc: freebsd-stable@freebsd.org > Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks >=20 > Hi again List, >=20 > I'm not so sure about using raidz2 anymore, I'm concerned for the performa= nce. >=20 > Basically I have 9x 1.5T sata drives. >=20 > raidz2 and 2x raidz1 will provide the same capacity. >=20 > Are there any cons against using 2x raidz1 instead of 1x raidz2 ? >=20 > I plan on using a SSD drive for the OS, 40-64gb, with 15 for the system it= self and some spare. >=20 > Is it worth using the free space for cache ? ZIL ? both ? >=20 > @jean-yves : didn't you experience problems recently when using both ? >=20 > --- > Fleuriot Damien >=20 > On 3 Jan 2011, at 16:08, Damien Fleuriot wrote: >=20 >>=20 >>=20 >> On 1/3/11 2:17 PM, Ivan Voras wrote: >>> On 12/30/10 12:40, Damien Fleuriot wrote: >>>=20 >>>> I am concerned that in the event a drive fails, I won't be able to=20 >>>> repair the disks in time before another actually fails. >>>=20 >>> An old trick to avoid that is to buy drives from different series or=20 >>> manufacturers (the theory is that identical drives tend to fail at=20 >>> the same time), but this may not be applicable if you have 5 drives=20 >>> in a volume :) Still, you can try playing with RAIDZ levels and probabil= ities. >>>=20 >>=20 >> That's sound advice, although one also hears that they should get=20 >> devices from the same vendor for maximum compatibility -.- >>=20 >>=20 >> Ah well, next time ;) >>=20 >>=20 >> A piece of advice I shall heed though is using 1% less capacity than=20 >> what the disks really provide, in case one day I have to swap a drive=20 >> and its replacement is a few kbytes smaller (thus preventing a rebuild). > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@FreeBSD.ORG Wed Jan 5 22:12:23 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6D0291065674 for ; Wed, 5 Jan 2011 22:12:23 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id B17E58FC18 for ; Wed, 5 Jan 2011 22:12:22 +0000 (UTC) Received: by bwz12 with SMTP id 12so8619953bwz.13 for ; Wed, 05 Jan 2011 14:12:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; bh=s2NZrsv9h8LEzMlJsHK3RS/HRDs3+n4yFgGqGNf3yhE=; b=pbQQi5qljCaB+oxwtfuVXQOo8YT/uKpkNAFJPL9dMFMZYTATBB2exoxo7x9VEmwIrI REegdOCBi52B0b0Sw9J/eMjNIjnImZ34Xzd3wQ1DjFv28fA4ki4lsryQk+cxczETz2mN ssoThXHfRG9Ugoj4yIxGhyLKWpel4VBcEBxIs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=hc0loAJZdZqZ9ssL0MffiW9yoF0fmS7d2xlKwJUFlCd0eJVqxCoyqvINbz5wqXGwt4 H5ByCMThSF5BQlGpV/gPvIpI4L6Che17dHBRaWGHcGbzeibjm3Mz9nCtpzfodGhYaLk7 pshVMUPTQt8bpTofUNbF54JdwhWnmLXKYARyE= MIME-Version: 1.0 Received: by 10.204.73.17 with SMTP id o17mr1551320bkj.26.1294265541749; Wed, 05 Jan 2011 14:12:21 -0800 (PST) Sender: artemb@gmail.com Received: by 10.204.22.18 with HTTP; Wed, 5 Jan 2011 14:12:21 -0800 (PST) In-Reply-To: <488AE93A-97B9-4F01-AD0A-0098E4B329C3@my.gd> References: <4D1C6F90.3080206@my.gd> <4D21E679.80002@my.gd> <84882169-0461-480F-8B4C-58E794BCC8E6@my.gd> <488AE93A-97B9-4F01-AD0A-0098E4B329C3@my.gd> Date: Wed, 5 Jan 2011 14:12:21 -0800 X-Google-Sender-Auth: Pjws985YJZLQewl9kI14N9v1Cyo Message-ID: From: Artem Belevich To: Damien Fleuriot Content-Type: text/plain; charset=ISO-8859-1 Cc: "freebsd-stable@freebsd.org" , Chris Forgeron Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 22:12:23 -0000 On Wed, Jan 5, 2011 at 1:55 PM, Damien Fleuriot wrote: > Well actually... > > raidz2: > - 7x 1.5 tb = 10.5tb > - 2 parity drives > > raidz1: > - 3x 1.5 tb = 4.5 tb > - 4x 1.5 tb = 6 tb , total 10.5tb > - 2 parity drives in split thus different raidz1 arrays > > So really, in both cases 2 different parity drives and same storage... In second case you get better performance, but lose some data protection. It's still raidz1 and you can't guarantee functionality in all cases of two drives failing. If two drives fail in the same vdev, your entire pool will be gone. Granted, it's better than single-vdev raidz1, but it's *not* as good as raidz2. --Artem From owner-freebsd-stable@FreeBSD.ORG Wed Jan 5 22:34:58 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D0D0E106564A for ; Wed, 5 Jan 2011 22:34:58 +0000 (UTC) (envelope-from cforgeron@acsi.ca) Received: from mta02.eastlink.ca (mta02.eastlink.ca [24.224.136.13]) by mx1.freebsd.org (Postfix) with ESMTP id 83ED08FC0A for ; Wed, 5 Jan 2011 22:34:58 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from ip06.eastlink.ca ([unknown] [24.222.39.84]) by mta02.eastlink.ca (Sun Java(tm) System Messaging Server 7.3-11.01 64bit (built Sep 1 2009)) with ESMTP id <0LEK00EY0MQ9WW53@mta02.eastlink.ca> for freebsd-stable@freebsd.org; Wed, 05 Jan 2011 18:34:57 -0400 (AST) X-CMAE-Score: 0 X-CMAE-Analysis: v=1.1 cv=zW4qkwwpA+AM09VaNOLkucDBqffYZTwBSaEYifkV/P4= c=1 sm=1 a=tZDC3L0OECsA:10 a=kj9zAlcOel0A:10 a=6I5d2MoRAAAA:8 a=vNhP-Wudz7bmYAJ9o5oA:9 a=MQioLfrucWnjU7pU0b8A:7 a=F0pSueVjc9bGzCcjncRArCPonsIA:4 a=CjuIK1q_8ugA:10 a=SV7veod9ZcQA:10 a=vKNzL2Iiy-SiEm6x:21 a=RlpcPMoTtnOkUV3n:21 a=12M4PSijgPY/TTHpO+5bpg==:117 Received: from blk-222-10-85.eastlink.ca (HELO server7.acsi.ca) ([24.222.10.85]) by ip06.eastlink.ca with ESMTP; Wed, 05 Jan 2011 18:34:56 -0400 Received: from server7.acsi.ca ([192.168.9.7]) by server7.acsi.ca ([192.168.9.7]) with mapi; Wed, 05 Jan 2011 18:34:56 -0400 From: Chris Forgeron To: Damien Fleuriot Date: Wed, 05 Jan 2011 18:34:54 -0400 Thread-topic: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks Thread-index: AcutI4m3A3c5xY6dTeKd5I2RI/z5JwAA4pZg Message-id: References: <4D1C6F90.3080206@my.gd> <4D21E679.80002@my.gd> <84882169-0461-480F-8B4C-58E794BCC8E6@my.gd> <488AE93A-97B9-4F01-AD0A-0098E4B329C3@my.gd> In-reply-to: <488AE93A-97B9-4F01-AD0A-0098E4B329C3@my.gd> Accept-Language: en-US Content-language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Cc: "freebsd-stable@freebsd.org" Subject: RE: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 22:34:58 -0000 Yup, but the second set (stripe of 2 raidz1's) can achieve slightly better performance, particularly on a system that has a lot of load. There's a number of blog articles that discuss that in more detail than I care to get into here. Of course, that's a bit of a moot point, as you're not going to heavily load a 9 drive system, like a 48 drive system, but.. In that example, the first (raidz2) would be a bit more safe as it could take 2 drives failing. The latter (2 raidz1's) would die if those two failing drives are within 1 raidz1 vdev. It all comes down to that final decision on how much risk do you want to take with your data, what your budget is, and what your performance requirements are. I'm starting to settle into a stripe of 6 vdevs that are each a 5 disk raidz1, with two hot-spares kicking about, and a collection of small SSD's adding up to either 500G or 1TB of SSD L2ARC. A bit more risk, but I'm also planning on having an entirely redundant (yet slower) SAN device that will get a daily ZFS send, so my worst nightmare is yesterday's data - Which I can stand. Oh - I am also a fan of buying drives at different time periods or from different suppliers.. I have seen entire 4 and 8 drive arrays fail within a month of the first drives going. Always really fun when you were too slack to handle the first drive failure, the second one put you in a tight spot the next week, and then the third one dies while you're madly trying to do data recovery.. :-) Really, in a big enough array, I like to swap out older drives for newer ones every now and then and repurpose the old - Just to keep the dreaded complete failure at bay. Things you learn to do with cheap SATA drives.. -----Original Message----- From: owner-freebsd-stable@freebsd.org [mailto:owner-freebsd-stable@freebsd.org] On Behalf Of Damien Fleuriot Sent: Wednesday, January 05, 2011 5:55 PM To: Chris Forgeron Cc: freebsd-stable@freebsd.org Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks Well actually... raidz2: - 7x 1.5 tb = 10.5tb - 2 parity drives raidz1: - 3x 1.5 tb = 4.5 tb - 4x 1.5 tb = 6 tb , total 10.5tb - 2 parity drives in split thus different raidz1 arrays So really, in both cases 2 different parity drives and same storage... --- Fleuriot Damien On 5 Jan 2011, at 16:55, Chris Forgeron wrote: > First off, raidz2 and raidz1 with copies=2 are not the same thing. > > raidz2 will give you two copies of parity instead of just one. It also guarantees that this parity is on different drives. You can sustain 2 drive failures without data loss. > > raidz1 with copies=2 will give you two copies of all your files, but there is no guarantee that they are on different drives, and you can still only sustain 1 drive failure. > > You'll have better space efficiency with raidz2 if you're using 9 drives. > > If I were you, I'd use your 9 disks as one big raidz, or better yet, get 10 disks, and make a stripe of 2 5 disk raidz's for the best performance. > > Save your SSD drive for the L2ARC (cache) or ZIL, you'll get better speed that way instead of throwing it away on a boot drive. > > -- > > > -----Original Message----- > From: owner-freebsd-stable@freebsd.org [mailto:owner-freebsd-stable@freebsd.org] On Behalf Of Damien Fleuriot > Sent: January-05-11 5:01 AM > To: Damien Fleuriot > Cc: freebsd-stable@freebsd.org > Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks > > Hi again List, > > I'm not so sure about using raidz2 anymore, I'm concerned for the performance. > > Basically I have 9x 1.5T sata drives. > > raidz2 and 2x raidz1 will provide the same capacity. > > Are there any cons against using 2x raidz1 instead of 1x raidz2 ? > > I plan on using a SSD drive for the OS, 40-64gb, with 15 for the system itself and some spare. > > Is it worth using the free space for cache ? ZIL ? both ? > > @jean-yves : didn't you experience problems recently when using both ? > > --- > Fleuriot Damien > > On 3 Jan 2011, at 16:08, Damien Fleuriot wrote: > >> >> >> On 1/3/11 2:17 PM, Ivan Voras wrote: >>> On 12/30/10 12:40, Damien Fleuriot wrote: >>> >>>> I am concerned that in the event a drive fails, I won't be able to >>>> repair the disks in time before another actually fails. >>> >>> An old trick to avoid that is to buy drives from different series or >>> manufacturers (the theory is that identical drives tend to fail at >>> the same time), but this may not be applicable if you have 5 drives >>> in a volume :) Still, you can try playing with RAIDZ levels and probabilities. >>> >> >> That's sound advice, although one also hears that they should get >> devices from the same vendor for maximum compatibility -.- >> >> >> Ah well, next time ;) >> >> >> A piece of advice I shall heed though is using 1% less capacity than >> what the disks really provide, in case one day I have to swap a drive >> and its replacement is a few kbytes smaller (thus preventing a rebuild). > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" _______________________________________________ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@FreeBSD.ORG Thu Jan 6 08:10:44 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BB380106564A; Thu, 6 Jan 2011 08:10:44 +0000 (UTC) (envelope-from perryh@pluto.rain.com) Received: from agora.rdrop.com (agora.rdrop.com [IPv6:2607:f678:1010::34]) by mx1.freebsd.org (Postfix) with ESMTP id 9A4E48FC15; Thu, 6 Jan 2011 08:10:44 +0000 (UTC) Received: from agora.rdrop.com (66@localhost [127.0.0.1]) by agora.rdrop.com (8.13.1/8.12.7) with ESMTP id p068AbqK035155 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Thu, 6 Jan 2011 00:10:38 -0800 (PST) (envelope-from perryh@pluto.rain.com) Received: (from uucp@localhost) by agora.rdrop.com (8.13.1/8.12.9/Submit) with UUCP id p068Ab0I035154; Thu, 6 Jan 2011 00:10:37 -0800 (PST) Received: from fbsd61 by pluto.rain.com (4.1/SMI-4.1-pluto-M2060407) id AA16440; Thu, 6 Jan 11 00:08:26 PST Date: Thu, 06 Jan 2011 00:08:04 -0800 From: perryh@pluto.rain.com To: jhb@freebsd.org Message-Id: <4d257864.YjgoS235V8eKvUX2%perryh@pluto.rain.com> References: <1036681015.111502.1294189339355.JavaMail.root@erie.cs.uoguelph.ca> <4d244e39.KoPcOoMaWed+H5De%perryh@pluto.rain.com> <201101050757.08116.jhb@freebsd.org> In-Reply-To: <201101050757.08116.jhb@freebsd.org> User-Agent: nail 11.25 7/29/05 Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: marek_sal@wp.pl, freebsd-stable@freebsd.org, jyavenard@gmail.com, rmacklem@uoguelph.ca, milu@dat.pl Subject: Re: NFSv4 - how to set up at FreeBSD 8.1 ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Jan 2011 08:10:44 -0000 John Baldwin wrote: > ... even NFS UDP mounts maintain their own set of "socket" state > to manage retries and retransmits for UDP RPCs. Not according to what I remember of the SunOS NFS documentation, which indicated that the driving force behind using UDP instead of TCP was to have the server be _completely_ stateless. (Of course locking is inherently stateful; they made it very clear that the locking protocol was considered to be an adjunct rather than part of the NFS protocol itself.) It's been quite a few years since I read that, and I didn't get into the details, but I suppose the handle returned to a client (in response to a mount or open request) must have contained both a representation of the inode number and a unique identification of the filesystem (so that, in the case where server crash recovery included a newfs and reload from backup, the FS ID would not match and the client would get a "stale handle" response). All of the retry and retransmit burden had to have been managed by the client, for both reading and writing. From owner-freebsd-stable@FreeBSD.ORG Thu Jan 6 08:50:41 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 34023106564A; Thu, 6 Jan 2011 08:50:41 +0000 (UTC) (envelope-from perryh@pluto.rain.com) Received: from agora.rdrop.com (agora.rdrop.com [IPv6:2607:f678:1010::34]) by mx1.freebsd.org (Postfix) with ESMTP id 13F8B8FC15; Thu, 6 Jan 2011 08:50:41 +0000 (UTC) Received: from agora.rdrop.com (66@localhost [127.0.0.1]) by agora.rdrop.com (8.13.1/8.12.7) with ESMTP id p068obeM039490 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Thu, 6 Jan 2011 00:50:37 -0800 (PST) (envelope-from perryh@pluto.rain.com) Received: (from uucp@localhost) by agora.rdrop.com (8.13.1/8.12.9/Submit) with UUCP id p068obwg039489; Thu, 6 Jan 2011 00:50:37 -0800 (PST) Received: from fbsd61 by pluto.rain.com (4.1/SMI-4.1-pluto-M2060407) id AA16538; Thu, 6 Jan 11 00:45:09 PST Date: Thu, 06 Jan 2011 00:44:48 -0800 From: perryh@pluto.rain.com To: rmacklem@uoguelph.ca Message-Id: <4d258100.Wu9WZrgdH2/KvDTa%perryh@pluto.rain.com> References: <1870282066.118978.1294234205820.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <1870282066.118978.1294234205820.JavaMail.root@erie.cs.uoguelph.ca> User-Agent: nail 11.25 7/29/05 Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: marek_sal@wp.pl, freebsd-stable@freebsd.org, milu@dat.pl, jyavenard@gmail.com, jhb@freebsd.org Subject: Re: NFSv4 - how to set up at FreeBSD 8.1 ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Jan 2011 08:50:41 -0000 Rick Macklem wrote: > Sun did add a separate file locking protocol called the NLM > or rpc.lockd if you prefer, but that protocol design was > fundamentally flawed imho and, as such, using it is in the > "your mileage may vary" category. I suppose it was not all that bad, considering that what it sought to accomplish is incomputable. There is simply no way for either the server or the client to distinguish between "the other end has crashed" and "there is a temporary communication failure" until the other end comes back up or communication is restored. On a good day, in a completely homogeneous environment (server and all clients running the same OS revision and patchlevel), I trust lockd about as far as I can throw 10GB of 1980's SMD disk drives :) Exporting /var/spool/mail read/write tends to ensure that good days will be rare. Been there, done that, seen the result. Never again. That's what IMAP is for. From owner-freebsd-stable@FreeBSD.ORG Thu Jan 6 09:17:08 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3AE3F106564A for ; Thu, 6 Jan 2011 09:17:08 +0000 (UTC) (envelope-from cochard@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id C65AB8FC12 for ; Thu, 6 Jan 2011 09:17:07 +0000 (UTC) Received: by fxm16 with SMTP id 16so15731749fxm.13 for ; Thu, 06 Jan 2011 01:17:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:sender:received:from:date :x-google-sender-auth:message-id:subject:to:content-type; bh=HdeyHizbPkuJ9ZKuv3gPUBreYVHtPP7t9a3T0mBJuUw=; b=DEVbAYLiOD6cCNEAKxqTxIPQj8QMQVCqOhul5TVWpelgHxaaQnNHxmgHNdSja8V0yK d9UHUBlDRVNLCIxqOy6GBDn7pQWzxKhZww4rxO+WGqO5VbOhtck8ODIcJGrkoFQz92I2 e3/zFH5i/SQ1N7q08QDdc+OU+c7qiCMlayN3A= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:from:date:x-google-sender-auth:message-id :subject:to:content-type; b=IErTwWN/LckcXwbq+5GWS8bXRio+DuqjJ7WPowiearvSYeHpXvR84aEvrmydLZC9uj 7arFZ9l0CSRKjLWMprvNCSMMwbkLiWCfhpuosLdSi6QJTOxVRgRF3wa9QNU3a5wqkaF5 c+t6fEZ1MH28E/W+g2JG+vFuF1cgBO/7mEwig= Received: by 10.223.78.132 with SMTP id l4mr82782fak.19.1294304595048; Thu, 06 Jan 2011 01:03:15 -0800 (PST) MIME-Version: 1.0 Sender: cochard@gmail.com Received: by 10.223.147.144 with HTTP; Thu, 6 Jan 2011 01:02:54 -0800 (PST) From: =?ISO-8859-1?Q?Olivier_Cochard=2DLabb=E9?= Date: Thu, 6 Jan 2011 10:02:54 +0100 X-Google-Sender-Auth: QaffjBGiyMTxNBU6ypvmFLw7J4c Message-ID: To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: Regression with iwn drivers (4965BGN) in 8.2-PRERELEASE ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Jan 2011 09:17:08 -0000 Hi all, Since I've upgraded from 8.1 to 8.2RC, my wireless negotiated speed is very very slow (more exactly it start at normal speed, but decrease each second still stopping a 1Mbps and became unusuable). I'm using iwn drivers: iwn0: mem 0xf6cfe000-0xf6cfffff irq 17 at device 0.0 on pci12 iwn0: MIMO 2T3R, MoW2, address 00:1d:e0:72:10:01 iwn0: [ITHREAD] iwn0: 11a rates: 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps iwn0: 11b rates: 1Mbps 2Mbps 5.5Mbps 11Mbps iwn0: 11g rates: 1Mbps 2Mbps 5.5Mbps 11Mbps 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps [root@d630]~#uname -a FreeBSD d630.bsdrp.net 8.2-PRERELEASE FreeBSD 8.2-PRERELEASE #1: Sun Jan 2 01:32:14 CET 2011 root@d630.bsdrp.net:/usr/obj/usr/src/sys/GENERIC amd64 Does anybody else meet the same problem ? Thanks, Olivier From owner-freebsd-stable@FreeBSD.ORG Thu Jan 6 09:20:51 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D7C141065694 for ; Thu, 6 Jan 2011 09:20:51 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 725A38FC14 for ; Thu, 6 Jan 2011 09:20:51 +0000 (UTC) Received: by wwf26 with SMTP id 26so15981669wwf.31 for ; Thu, 06 Jan 2011 01:20:50 -0800 (PST) Received: by 10.216.213.15 with SMTP id z15mr23189948weo.61.1294305650277; Thu, 06 Jan 2011 01:20:50 -0800 (PST) Received: from [192.168.0.20] (paris.c-mal.com [88.170.200.60]) by mx.google.com with ESMTPS id j49sm10452561wer.38.2011.01.06.01.20.47 (version=TLSv1/SSLv3 cipher=RC4-MD5); Thu, 06 Jan 2011 01:20:49 -0800 (PST) References: <4D1C6F90.3080206@my.gd> <4D21E679.80002@my.gd> <84882169-0461-480F-8B4C-58E794BCC8E6@my.gd> <488AE93A-97B9-4F01-AD0A-0098E4B329C3@my.gd> In-Reply-To: Mime-Version: 1.0 (iPhone Mail 8A293) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Message-Id: X-Mailer: iPhone Mail (8A293) From: Damien Fleuriot Date: Thu, 6 Jan 2011 10:20:00 +0100 To: Artem Belevich Cc: "freebsd-stable@freebsd.org" , Chris Forgeron Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Jan 2011 09:20:51 -0000 You both make good points, thanks for the feedback :) I am more concerned about data protection than performance, so I suppose rai= dz2 is the best choice I have with such a small scale setup. Now the question that remains is wether or not to use parts of the OS's ssd f= or zil, cache, or both ? --- Fleuriot Damien On 5 Jan 2011, at 23:12, Artem Belevich wrote: > On Wed, Jan 5, 2011 at 1:55 PM, Damien Fleuriot wrote: >> Well actually... >>=20 >> raidz2: >> - 7x 1.5 tb =3D 10.5tb >> - 2 parity drives >>=20 >> raidz1: >> - 3x 1.5 tb =3D 4.5 tb >> - 4x 1.5 tb =3D 6 tb , total 10.5tb >> - 2 parity drives in split thus different raidz1 arrays >>=20 >> So really, in both cases 2 different parity drives and same storage... >=20 > In second case you get better performance, but lose some data > protection. It's still raidz1 and you can't guarantee functionality in > all cases of two drives failing. If two drives fail in the same vdev, > your entire pool will be gone. Granted, it's better than single-vdev > raidz1, but it's *not* as good as raidz2. >=20 > --Artem From owner-freebsd-stable@FreeBSD.ORG Thu Jan 6 10:16:24 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A396B106566B for ; Thu, 6 Jan 2011 10:16:24 +0000 (UTC) (envelope-from bschmidt@techwires.net) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 3FD368FC13 for ; Thu, 6 Jan 2011 10:16:23 +0000 (UTC) Received: by fxm16 with SMTP id 16so15767618fxm.13 for ; Thu, 06 Jan 2011 02:16:23 -0800 (PST) Received: by 10.223.85.204 with SMTP id p12mr3621153fal.146.1294307155963; Thu, 06 Jan 2011 01:45:55 -0800 (PST) Received: from jessie.localnet (p5B2EC5B8.dip0.t-ipconnect.de [91.46.197.184]) by mx.google.com with ESMTPS id z1sm5770765fau.21.2011.01.06.01.45.54 (version=SSLv3 cipher=RC4-MD5); Thu, 06 Jan 2011 01:45:54 -0800 (PST) Sender: Bernhard Schmidt From: Bernhard Schmidt To: "Olivier =?iso-8859-1?q?Cochard-Labb=E9?=" Date: Thu, 6 Jan 2011 10:46:01 +0100 User-Agent: KMail/1.13.2 (Linux/2.6.32-25-generic; KDE/4.4.2; i686; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Message-Id: <201101061046.02128.bschmidt@freebsd.org> Cc: freebsd-stable@freebsd.org Subject: Re: Regression with iwn drivers (4965BGN) in 8.2-PRERELEASE ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: bschmidt@freebsd.org List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Jan 2011 10:16:24 -0000 On Thursday, January 06, 2011 10:02:54 Olivier Cochard-Labb=E9 wrote: > Hi all, > Since I've upgraded from 8.1 to 8.2RC, my wireless negotiated speed is > very very slow (more exactly it start at normal speed, but decrease > each second still stopping a 1Mbps and became unusuable). >=20 > I'm using iwn drivers: > iwn0: mem 0xf6cfe000-0xf6cfffff irq 17 > at device 0.0 on pci12 > iwn0: MIMO 2T3R, MoW2, address 00:1d:e0:72:10:01 > iwn0: [ITHREAD] > iwn0: 11a rates: 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps > iwn0: 11b rates: 1Mbps 2Mbps 5.5Mbps 11Mbps > iwn0: 11g rates: 1Mbps 2Mbps 5.5Mbps 11Mbps 6Mbps 9Mbps 12Mbps 18Mbps > 24Mbps 36Mbps 48Mbps 54Mbps >=20 > [root@d630]~#uname -a > FreeBSD d630.bsdrp.net 8.2-PRERELEASE FreeBSD 8.2-PRERELEASE #1: Sun > Jan 2 01:32:14 CET 2011 > root@d630.bsdrp.net:/usr/obj/usr/src/sys/GENERIC amd64 >=20 > Does anybody else meet the same problem ? Haven't seen this yet. What do you mean with 'unusable' exactly? Lots of packet loss, or just slow= =20 transfer rates? 'wlandebug +rate' might shed some light on this one. =2D-=20 Bernhard From owner-freebsd-stable@FreeBSD.ORG Thu Jan 6 10:24:07 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9F9D5106564A for ; Thu, 6 Jan 2011 10:24:06 +0000 (UTC) (envelope-from cochard@gmail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 315738FC19 for ; Thu, 6 Jan 2011 10:24:05 +0000 (UTC) Received: by fxm16 with SMTP id 16so15773417fxm.13 for ; Thu, 06 Jan 2011 02:24:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:sender:received :in-reply-to:references:from:date:x-google-sender-auth:message-id :subject:to:cc:content-type:content-transfer-encoding; bh=tYnKCBWXO4M/FXM92V1UaJy+A575EGqtdjTLHuBqR90=; b=I9TZT08YmQ3RGgn2k88pP5HtbLfnqWi2abWaFcHoKV5sdPdlryF0UoCMnjEabX7h9a CptaPY2E6q3rdhD3jdwPX0anQ57oXLrqHSQ90vLQqtIF0iM2NZw22cRePJn7aiwy+6HZ LwJmy1+QRPirThmIQK2hyx74CAO80wOF+/5qA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type :content-transfer-encoding; b=eTfCRXR2CafKAqUin/dPNwy3T93FLADUJAfOhWFZYgKUi4W8RtcUV+V6t69YHNUjVl qeUrPcvuNu+KnZh6syWvNpiYROzXX9kdFXVopLgkcLPUpLOlMv/sVmg3IXQlrUnw3OXD gMfiOWWS2vzHcQ6o7ZuKnDINAfMBoBmzQbTFA= Received: by 10.223.71.199 with SMTP id i7mr829061faj.57.1294309444938; Thu, 06 Jan 2011 02:24:04 -0800 (PST) MIME-Version: 1.0 Sender: cochard@gmail.com Received: by 10.223.147.144 with HTTP; Thu, 6 Jan 2011 02:23:44 -0800 (PST) In-Reply-To: <201101061046.02128.bschmidt@freebsd.org> References: <201101061046.02128.bschmidt@freebsd.org> From: =?ISO-8859-1?Q?Olivier_Cochard=2DLabb=E9?= Date: Thu, 6 Jan 2011 11:23:44 +0100 X-Google-Sender-Auth: fyo2UsdpjFcfxsTD5uwOTP-9ljU Message-ID: To: bschmidt@freebsd.org Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Cc: freebsd-stable@freebsd.org Subject: Re: Regression with iwn drivers (4965BGN) in 8.2-PRERELEASE ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Jan 2011 10:24:07 -0000 2011/1/6 Bernhard Schmidt : > > What do you mean with 'unusable' exactly? Lots of packet loss, or just sl= ow > transfer rates? 'wlandebug +rate' might shed some light on this one. Hi, it's just very slow transfer rates. I didn't know wlandebug, thanks for the tips. Here are the result just after a boot, during pinging my gateway (few traff= ic): Jan 6 11:02:25 d630 kernel: wlan0: [3a:41:c4:e3:1e:18] AMRR increasing rate 48 (txcnt=3D11 retrycnt=3D0) Jan 6 11:02:36 d630 kernel: wlan0: [3a:41:c4:e3:1e:18] AMRR increasing rate 72 (txcnt=3D11 retrycnt=3D0) Jan 6 11:03:02 d630 kernel: wlan0: [3a:41:c4:e3:1e:18] AMRR increasing rate 96 (txcnt=3D11 retrycnt=3D0) Now, I start xorg and a start a browser: Jan 6 11:04:04 d630 kernel: wlan0: [3a:41:c4:e3:1e:18] AMRR decreasing rate 72 (txcnt=3D11 retrycnt=3D10) Jan 6 11:04:09 d630 kernel: wlan0: [3a:41:c4:e3:1e:18] AMRR decreasing rate 48 (txcnt=3D13 retrycnt=3D7) Jan 6 11:04:10 d630 kernel: wlan0: [3a:41:c4:e3:1e:18] AMRR decreasing rate 36 (txcnt=3D43 retrycnt=3D24) Jan 6 11:04:11 d630 kernel: wlan0: [3a:41:c4:e3:1e:18] AMRR decreasing rate 24 (txcnt=3D84 retrycnt=3D38) Jan 6 11:04:12 d630 kernel: wlan0: [3a:41:c4:e3:1e:18] AMRR decreasing rate 22 (txcnt=3D25 retrycnt=3D9) Jan 6 11:04:12 d630 kernel: wlan0: [3a:41:c4:e3:1e:18] AMRR decreasing rate 18 (txcnt=3D31 retrycnt=3D11) Jan 6 11:04:13 d630 kernel: wlan0: [3a:41:c4:e3:1e:18] AMRR decreasing rate 12 (txcnt=3D29 retrycnt=3D11) Jan 6 11:04:13 d630 kernel: wlan0: [3a:41:c4:e3:1e:18] AMRR decreasing rate 11 (txcnt=3D53 retrycnt=3D28) Jan 6 11:04:18 d630 kernel: wlan0: [3a:41:c4:e3:1e:18] AMRR decreasing rate 4 (txcnt=3D17 retrycnt=3D7) Jan 6 11:04:20 d630 kernel: wlan0: [3a:41:c4:e3:1e:18] AMRR decreasing rate 2 (txcnt=3D11 retrycnt=3D10) Jan 6 11:06:07 d630 wpa_supplicant[413]: CTRL-EVENT-SCAN-RESULTS Jan 6 11:09:48 d630 kernel: wlan0: [3a:41:c4:e3:1e:18] AMRR increasing rate 4 (txcnt=3D11 retrycnt=3D0) Jan 6 11:09:57 d630 kernel: wlan0: [3a:41:c4:e3:1e:18] AMRR increasing rate 11 (txcnt=3D11 retrycnt=3D0) Jan 6 11:10:30 d630 kernel: wlan0: [3a:41:c4:e3:1e:18] AMRR increasing rate 12 (txcnt=3D11 retrycnt=3D0) Jan 6 11:10:38 d630 kernel: wlan0: [3a:41:c4:e3:1e:18] AMRR decreasing rate 11 (txcnt=3D35 retrycnt=3D16) Jan 6 11:10:43 d630 kernel: wlan0: [3a:41:c4:e3:1e:18] AMRR decreasing rate 4 (txcnt=3D11 retrycnt=3D4) Jan 6 11:11:04 d630 kernel: wlan0: [3a:41:c4:e3:1e:18] AMRR increasing rate 11 (txcnt=3D11 retrycnt=3D0) Jan 6 11:11:09 d630 wpa_supplicant[413]: CTRL-EVENT-SCAN-RESULTS Jan 6 11:11:09 d630 kernel: wlan0: [3a:41:c4:e3:1e:18] AMRR decreasing rate 4 (txcnt=3D11 retrycnt=3D4) The rate decrease too much for using a browser (but I can still ping my gateway)=85 Regards, Olivier From owner-freebsd-stable@FreeBSD.ORG Thu Jan 6 10:51:44 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C09B1106564A for ; Thu, 6 Jan 2011 10:51:44 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from ftp.translate.ru (ftp.translate.ru [80.249.188.42]) by mx1.freebsd.org (Postfix) with ESMTP id 21F8A8FC0A for ; Thu, 6 Jan 2011 10:51:42 +0000 (UTC) Received: from [192.168.134.2] (89.112.15.178.pppoe.eltel.net [89.112.15.178]) (Authenticated sender: lev@serebryakov.spb.ru) by ftp.translate.ru (Postfix) with ESMTPA id 1874A13DF5F for ; Thu, 6 Jan 2011 13:31:51 +0300 (MSK) Date: Thu, 6 Jan 2011 13:31:45 +0300 From: Lev Serebryakov Organization: FreeBSD X-Priority: 3 (Normal) Message-ID: <1608571029.20110106133145@serebryakov.spb.ru> To: "freebsd-stable@freebsd.org" MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----------4D51C134ECDCF3" Subject: FreeBSD 8.2-PRERELEASE hangs under load with "live" kernel X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Jan 2011 10:51:44 -0000 ------------4D51C134ECDCF3 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Hello, Freebsd-stable. I've added torrent client (transmission) to software on my home server and it starts to hang in very unusual way: kernel works but userland doesn't. I can ping it (and it answers). I can scroll console with "scrolllock" button and keys. I can break into debugger with Ctrl+SysReq and it shows, that one CPU is occupied by idle process and other by "Giant tasq", but no userland processes answer: I can not ssh to it, I cannot login on console, samba is dead, etc. "ps" in kernel debugger shows, that many of processes in "pfault" state, and noting more "special". memtest86+ doesn't show any errors after 8 passes of tests (about 10 hours), so RAM looks Ok. What should I do in kdb to understand what happens? Kernel config and /var/run/dmesg.boot is attached. --=20 // Black Lion AKA Lev Serebryakov ------------4D51C134ECDCF3 Content-Type: application/octet-stream; name=BLOB Content-transfer-encoding: base64 Content-Disposition: attachment; filename=BLOB Y3B1CQlIQU1NRVIKaWRlbnQJCUJMT0IKCm1ha2VvcHRpb25zCURFQlVHPS1nCQkjIEJ1aWxk IGtlcm5lbCB3aXRoIGdkYigxKSBkZWJ1ZyBzeW1ib2xzCgpvcHRpb25zCQlQUklOVEZfQlVG Ul9TSVpFPTEwMjQKCm9wdGlvbnMJCUREQgpvcHRpb25zCQlLREIKb3B0aW9ucwkJS0RCX1VO QVRURU5ERUQKb3B0aW9ucwkJQlJFQUtfVE9fREVCVUdHRVIKCm9wdGlvbnMgCVNDSEVEX1VM RQkJIyBVTEUgc2NoZWR1bGVyCm9wdGlvbnMgCVBSRUVNUFRJT04JCSMgRW5hYmxlIGtlcm5l bCB0aHJlYWQgcHJlZW1wdGlvbgpvcHRpb25zIAlJTkVUCQkJIyBJbnRlck5FVHdvcmtpbmcK b3B0aW9ucyAJRkZTCQkJIyBCZXJrZWxleSBGYXN0IEZpbGVzeXN0ZW0Kb3B0aW9ucyAJU09G VFVQREFURVMJCSMgRW5hYmxlIEZGUyBzb2Z0IHVwZGF0ZXMgc3VwcG9ydApvcHRpb25zIAlV RlNfQUNMCQkJIyBTdXBwb3J0IGZvciBhY2Nlc3MgY29udHJvbCBsaXN0cwpvcHRpb25zIAlV RlNfRElSSEFTSAkJIyBJbXByb3ZlIHBlcmZvcm1hbmNlIG9uIGJpZyBkaXJlY3Rvcmllcwpv cHRpb25zCQlVRlNfR0pPVVJOQUwJIyBFbmFibGUgZ2pvdXJuYWwtYmFzZWQgVUZTIGpvdXJu YWxpbmcKb3B0aW9ucyAJTURfUk9PVAkJCSMgTUQgaXMgYSBwb3RlbnRpYWwgcm9vdCBkZXZp Y2UKb3B0aW9ucyAJTkZTQ0xJRU5UCQkjIE5ldHdvcmsgRmlsZXN5c3RlbSBDbGllbnQKb3B0 aW9ucyAJTkZTU0VSVkVSCQkjIE5ldHdvcmsgRmlsZXN5c3RlbSBTZXJ2ZXIKb3B0aW9ucyAJ TkZTTE9DS0QJCSMgTmV0d29yayBMb2NrIE1hbmFnZXIKb3B0aW9ucyAJUFJPQ0ZTCQkJIyBQ cm9jZXNzIGZpbGVzeXN0ZW0gKHJlcXVpcmVzIFBTRVVET0ZTKQpvcHRpb25zIAlQU0VVRE9G UwkJIyBQc2V1ZG8tZmlsZXN5c3RlbSBmcmFtZXdvcmsKb3B0aW9ucyAJR0VPTV9MQUJFTAkJ IyBQcm92aWRlcyBsYWJlbGl6YXRpb24Kb3B0aW9ucyAJQ09NUEFUXzQzVFRZCQkjIEJTRCA0 LjMgVFRZIGNvbXBhdCBbS0VFUCBUSElTIV0Kb3B0aW9ucyAJQ09NUEFUX0lBMzIJCSMgQ29t cGF0aWJsZSB3aXRoIGkzODYgYmluYXJpZXMKb3B0aW9ucyAJQ09NUEFUX0ZSRUVCU0Q0CQkj IENvbXBhdGlibGUgd2l0aCBGcmVlQlNENApvcHRpb25zIAlDT01QQVRfRlJFRUJTRDUJCSMg Q29tcGF0aWJsZSB3aXRoIEZyZWVCU0Q1Cm9wdGlvbnMgCUNPTVBBVF9GUkVFQlNENgkJIyBD b21wYXRpYmxlIHdpdGggRnJlZUJTRDYKb3B0aW9ucyAJQ09NUEFUX0ZSRUVCU0Q3CQkjIENv bXBhdGlibGUgd2l0aCBGcmVlQlNENwpvcHRpb25zIAlTQ1NJX0RFTEFZPTUwMDAJCSMgRGVs YXkgKGluIG1zKSBiZWZvcmUgcHJvYmluZyBTQ1NJCm9wdGlvbnMgCUtUUkFDRQkJCSMga3Ry YWNlKDEpIHN1cHBvcnQKb3B0aW9ucyAJU1RBQ0sJCQkjIHN0YWNrKDkpIHN1cHBvcnQKb3B0 aW9ucyAJU1lTVlNITQkJCSMgU1lTVi1zdHlsZSBzaGFyZWQgbWVtb3J5Cm9wdGlvbnMgCVNZ U1ZNU0cJCQkjIFNZU1Ytc3R5bGUgbWVzc2FnZSBxdWV1ZXMKb3B0aW9ucyAJU1lTVlNFTQkJ CSMgU1lTVi1zdHlsZSBzZW1hcGhvcmVzCm9wdGlvbnMgCV9LUE9TSVhfUFJJT1JJVFlfU0NI RURVTElORyAjIFBPU0lYIFAxMDAzXzFCIHJlYWwtdGltZSBleHRlbnNpb25zCm9wdGlvbnMg CUtCRF9JTlNUQUxMX0NERVYJIyBpbnN0YWxsIGEgQ0RFViBlbnRyeSBpbiAvZGV2Cm9wdGlv bnMgCUFVRElUCQkJIyBTZWN1cml0eSBldmVudCBhdWRpdGluZwoKIyBNYWtlIGFuIFNNUC1j YXBhYmxlIGtlcm5lbCBieSBkZWZhdWx0Cm9wdGlvbnMgCVNNUAkJCSMgU3ltbWV0cmljIE11 bHRpUHJvY2Vzc29yIEtlcm5lbAoKIyBEZXZpY2UgcG9sbGluZwpvcHRpb25zCQlERVZJQ0Vf UE9MTElORwoKIyBDUFUgZnJlcXVlbmN5IGNvbnRyb2wKZGV2aWNlCQljcHVmcmVxCmRldmlj ZQkJY29yZXRlbXAKCmRldmljZQkJc21iCmRldmljZQkJc21idXMKZGV2aWNlCQlpY2hzbWIK CmRldmljZQkJaWljYnVzCmRldmljZQkJaWljc21iCmRldmljZQkJaWljCgojIEJ1cyBzdXBw b3J0LgpkZXZpY2UJCWFjcGkKZGV2aWNlCQlwY2kKCiMgU0NTSQpkZXZpY2UJCXNjYnVzCmRl dmljZQkJcGFzcwpkZXZpY2UJCWRhCgojIEFUQSBhbmQgQVRBUEkgZGV2aWNlcwojZGV2aWNl CQlhdGEKI2RldmljZQkJYXRhZGlzawkJIyBBVEEgZGlzayBkcml2ZXMKI29wdGlvbnMgCUFU QV9TVEFUSUNfSUQJIyBTdGF0aWMgZGV2aWNlIG51bWJlcmluZwpkZXZpY2UJCWFoY2kKCiMg YXRrYmRjMCBjb250cm9scyBib3RoIHRoZSBrZXlib2FyZCBhbmQgdGhlIFBTLzIgbW91c2UK ZGV2aWNlCQlhdGtiZGMJCSMgQVQga2V5Ym9hcmQgY29udHJvbGxlcgpkZXZpY2UJCWF0a2Jk CQkjIEFUIGtleWJvYXJkCmRldmljZQkJcHNtCQkjIFBTLzIgbW91c2UKZGV2aWNlCQlrYmRt dXgJCSMga2V5Ym9hcmQgbXVsdGlwbGV4ZXIKZGV2aWNlCQl2Z2EJCSMgVkdBIHZpZGVvIGNh cmQgZHJpdmVyCgojIHN5c2NvbnMgaXMgdGhlIGRlZmF1bHQgY29uc29sZSBkcml2ZXIsIHJl c2VtYmxpbmcgYW4gU0NPIGNvbnNvbGUKZGV2aWNlCQlzYwoKIyBTZXJpYWwgKENPTSkgcG9y dHMKZGV2aWNlCQl1YXJ0CQkjIEdlbmVyaWMgVUFSVCBkcml2ZXIKCiMgUGFyYWxsZWwgcG9y dApkZXZpY2UJCXBwYwpkZXZpY2UJCXBwYnVzCQkjIFBhcmFsbGVsIHBvcnQgYnVzIChyZXF1 aXJlZCkKCmRldmljZQkJZW0JCSMgSW50ZWwgUFJPLzEwMDAgYWRhcHRlciBHaWdhYml0IEV0 aGVybmV0IENhcmQKCiMgUHNldWRvIGRldmljZXMuCmRldmljZQkJbG9vcAkJIyBOZXR3b3Jr IGxvb3BiYWNrCmRldmljZQkJcmFuZG9tCQkjIEVudHJvcHkgZGV2aWNlCmRldmljZQkJZXRo ZXIJCSMgRXRoZXJuZXQgc3VwcG9ydApkZXZpY2UJCXR1bgkJIyBQYWNrZXQgdHVubmVsLgpk ZXZpY2UJCXB0eQkJIyBQc2V1ZG8tdHR5cyAodGVsbmV0IGV0YykKZGV2aWNlCQltZAkJIyBN ZW1vcnkgImRpc2tzIgpkZXZpY2UJCWZpcm13YXJlCSMgZmlybXdhcmUgYXNzaXN0IG1vZHVs ZQoKIyBUaGUgYGJwZicgZGV2aWNlIGVuYWJsZXMgdGhlIEJlcmtlbGV5IFBhY2tldCBGaWx0 ZXIuCiMgQmUgYXdhcmUgb2YgdGhlIGFkbWluaXN0cmF0aXZlIGNvbnNlcXVlbmNlcyBvZiBl bmFibGluZyB0aGlzIQojIE5vdGUgdGhhdCAnYnBmJyBpcyByZXF1aXJlZCBmb3IgREhDUC4K ZGV2aWNlCQlicGYJCSMgQmVya2VsZXkgcGFja2V0IGZpbHRlcgoKIyBVU0Igc3VwcG9ydApk ZXZpY2UJCXVoY2kJCSMgVUhDSSBQQ0ktPlVTQiBpbnRlcmZhY2UKZGV2aWNlCQllaGNpCQkj IEVIQ0kgUENJLT5VU0IgaW50ZXJmYWNlIChVU0IgMi4wKQpkZXZpY2UJCXVzYgkJIyBVU0Ig QnVzIChyZXF1aXJlZCkKZGV2aWNlCQl1Y29tCQkjIERpc2tzL01hc3Mgc3RvcmFnZSAtIFJl cXVpcmVzIHNjYnVzIGFuZCBkYQpkZXZpY2UJCXVtYXNzCQkjIERpc2tzL01hc3Mgc3RvcmFn ZSAtIFJlcXVpcmVzIHNjYnVzIGFuZCBkYQoKIyBGaXJlV2lyZSBzdXBwb3J0CmRldmljZQkJ ZmlyZXdpcmUJIyBGaXJlV2lyZSBidXMgY29kZQoK ------------4D51C134ECDCF3 Content-Type: application/octet-stream; name="dmesg.boot" Content-transfer-encoding: base64 Content-Disposition: attachment; filename="dmesg.boot" Q29weXJpZ2h0IChjKSAxOTkyLTIwMTEgVGhlIEZyZWVCU0QgUHJvamVjdC4KQ29weXJpZ2h0 IChjKSAxOTc5LCAxOTgwLCAxOTgzLCAxOTg2LCAxOTg4LCAxOTg5LCAxOTkxLCAxOTkyLCAx OTkzLCAxOTk0CglUaGUgUmVnZW50cyBvZiB0aGUgVW5pdmVyc2l0eSBvZiBDYWxpZm9ybmlh LiBBbGwgcmlnaHRzIHJlc2VydmVkLgpGcmVlQlNEIGlzIGEgcmVnaXN0ZXJlZCB0cmFkZW1h cmsgb2YgVGhlIEZyZWVCU0QgRm91bmRhdGlvbi4KRnJlZUJTRCA4LjItUFJFUkVMRUFTRSAj MzogTW9uIEphbiAgMyAxODo0NjozNiBNU0sgMjAxMQogICAgbGV2QGJsb2IuaG9tZS5zZXJl YnJ5YWtvdi5zcGIucnU6L3Vzci9vYmovdXNyL3NyYy9zeXMvQkxPQiBhbWQ2NApUaW1lY291 bnRlciAiaTgyNTQiIGZyZXF1ZW5jeSAxMTkzMTgyIEh6IHF1YWxpdHkgMApDUFU6IEludGVs KFIpIENvcmUoVE0pMiBEdW8gQ1BVICAgICBFNDUwMCAgQCAyLjIwR0h6ICgyMjAwLjA5LU1I eiBLOC1jbGFzcyBDUFUpCiAgT3JpZ2luID0gIkdlbnVpbmVJbnRlbCIgIElkID0gMHg2ZmQg IEZhbWlseSA9IDYgIE1vZGVsID0gZiAgU3RlcHBpbmcgPSAxMwogIEZlYXR1cmVzPTB4YmZl YmZiZmY8RlBVLFZNRSxERSxQU0UsVFNDLE1TUixQQUUsTUNFLENYOCxBUElDLFNFUCxNVFJS LFBHRSxNQ0EsQ01PVixQQVQsUFNFMzYsQ0xGTFVTSCxEVFMsQUNQSSxNTVgsRlhTUixTU0Us U1NFMixTUyxIVFQsVE0sUEJFPgogIEZlYXR1cmVzMj0weGUzOWQ8U1NFMyxEVEVTNjQsTU9O LERTX0NQTCxFU1QsVE0yLFNTU0UzLENYMTYseFRQUixQRENNPgogIEFNRCBGZWF0dXJlcz0w eDIwMTAwODAwPFNZU0NBTEwsTlgsTE0+CiAgQU1EIEZlYXR1cmVzMj0weDE8TEFIRj4KICBU U0M6IFAtc3RhdGUgaW52YXJpYW50CnJlYWwgbWVtb3J5ICA9IDIxNDc0ODM2NDggKDIwNDgg TUIpCmF2YWlsIG1lbW9yeSA9IDIwNTMzOTg1MjggKDE5NTggTUIpCkFDUEkgQVBJQyBUYWJs ZTogPEFfTV9JXyBPRU1BUElDID4KRnJlZUJTRC9TTVA6IE11bHRpcHJvY2Vzc29yIFN5c3Rl bSBEZXRlY3RlZDogMiBDUFVzCkZyZWVCU0QvU01QOiAxIHBhY2thZ2UocykgeCAyIGNvcmUo cykKIGNwdTAgKEJTUCk6IEFQSUMgSUQ6ICAwCiBjcHUxIChBUCk6IEFQSUMgSUQ6ICAxCmlv YXBpYzAgPFZlcnNpb24gMi4wPiBpcnFzIDAtMjMgb24gbW90aGVyYm9hcmQKa2JkMSBhdCBr YmRtdXgwCnVtY3M3ODQwOiBWZXJzaW9uIDEuMC4yMDEwMTIxMC41NyAocmV2IDJiZmNhNjVm NDYyMykKYWNwaTA6IDxBX01fSV8gT0VNWFNEVD4gb24gbW90aGVyYm9hcmQKYWNwaTA6IFtJ VEhSRUFEXQphY3BpMDogUG93ZXIgQnV0dG9uIChmaXhlZCkKYWNwaTA6IHJlc2VydmF0aW9u IG9mIDAsIGEwMDAwICgzKSBmYWlsZWQKYWNwaTA6IHJlc2VydmF0aW9uIG9mIDEwMDAwMCwg N2ZjMDAwMDAgKDMpIGZhaWxlZApUaW1lY291bnRlciAiQUNQSS1mYXN0IiBmcmVxdWVuY3kg MzU3OTU0NSBIeiBxdWFsaXR5IDEwMDAKYWNwaV90aW1lcjA6IDwyNC1iaXQgdGltZXIgYXQg My41Nzk1NDVNSHo+IHBvcnQgMHg4MDgtMHg4MGIgb24gYWNwaTAKY3B1MDogPEFDUEkgQ1BV PiBvbiBhY3BpMApBQ1BJIFdhcm5pbmc6IEluY29ycmVjdCBjaGVja3N1bSBpbiB0YWJsZSBb T0VNQl0gLSAweENCLCBzaG91bGQgYmUgMHhDNiAoMjAxMDEwMTMvdGJ1dGlscy0zNTQpCmNw dTE6IDxBQ1BJIENQVT4gb24gYWNwaTAKcGNpYjA6IDxBQ1BJIEhvc3QtUENJIGJyaWRnZT4g cG9ydCAweGNmOC0weGNmZiBvbiBhY3BpMApwY2kwOiA8QUNQSSBQQ0kgYnVzPiBvbiBwY2li MAp2Z2FwY2kwOiA8VkdBLWNvbXBhdGlibGUgZGlzcGxheT4gcG9ydCAweGVjMDAtMHhlYzA3 IG1lbSAweGZlYTgwMDAwLTB4ZmVhZmZmZmYsMHhkMDAwMDAwMC0weGRmZmZmZmZmLDB4ZmU5 MDAwMDAtMHhmZTlmZmZmZiBpcnEgMTYgYXQgZGV2aWNlIDIuMCBvbiBwY2kwCnBjaTA6IDxz aW1wbGUgY29tbXM+IGF0IGRldmljZSAzLjAgKG5vIGRyaXZlciBhdHRhY2hlZCkKcGNpMDog PG1hc3Mgc3RvcmFnZSwgQVRBPiBhdCBkZXZpY2UgMy4yIChubyBkcml2ZXIgYXR0YWNoZWQp CnBjaTA6IDxzaW1wbGUgY29tbXMsIFVBUlQ+IGF0IGRldmljZSAzLjMgKG5vIGRyaXZlciBh dHRhY2hlZCkKZW0wOiA8SW50ZWwoUikgUFJPLzEwMDAgTmV0d29yayBDb25uZWN0aW9uIDcu MS44PiBwb3J0IDB4ZGMwMC0weGRjMWYgbWVtIDB4ZmVhNDAwMDAtMHhmZWE1ZmZmZiwweGZl YTc5MDAwLTB4ZmVhNzlmZmYgaXJxIDIwIGF0IGRldmljZSAyNS4wIG9uIHBjaTAKZW0wOiBO byBNU0kvTVNJWCB1c2luZyBhIExlZ2FjeSBJUlEKZW0wOiBbRklMVEVSXQplbTA6IEV0aGVy bmV0IGFkZHJlc3M6IDAwOjFlOjhjOjc1OjAzOjBkCnVoY2kwOiA8SW50ZWwgODI4MDFJIChJ Q0g5KSBVU0IgY29udHJvbGxlcj4gcG9ydCAweGQ0ODAtMHhkNDlmIGlycSAxNiBhdCBkZXZp Y2UgMjYuMCBvbiBwY2kwCnVoY2kwOiBbSVRIUkVBRF0KdXNidXMwOiA8SW50ZWwgODI4MDFJ IChJQ0g5KSBVU0IgY29udHJvbGxlcj4gb24gdWhjaTAKdWhjaTE6IDxJbnRlbCA4MjgwMUkg KElDSDkpIFVTQiBjb250cm9sbGVyPiBwb3J0IDB4ZDgwMC0weGQ4MWYgaXJxIDIxIGF0IGRl dmljZSAyNi4xIG9uIHBjaTAKdWhjaTE6IFtJVEhSRUFEXQp1c2J1czE6IDxJbnRlbCA4Mjgw MUkgKElDSDkpIFVTQiBjb250cm9sbGVyPiBvbiB1aGNpMQp1aGNpMjogPEludGVsIDgyODAx SSAoSUNIOSkgVVNCIGNvbnRyb2xsZXI+IHBvcnQgMHhkODgwLTB4ZDg5ZiBpcnEgMTggYXQg ZGV2aWNlIDI2LjIgb24gcGNpMAp1aGNpMjogW0lUSFJFQURdCnVzYnVzMjogPEludGVsIDgy ODAxSSAoSUNIOSkgVVNCIGNvbnRyb2xsZXI+IG9uIHVoY2kyCmVoY2kwOiA8SW50ZWwgODI4 MDFJIChJQ0g5KSBVU0IgMi4wIGNvbnRyb2xsZXI+IG1lbSAweGZlYTdhODAwLTB4ZmVhN2Fi ZmYgaXJxIDE4IGF0IGRldmljZSAyNi43IG9uIHBjaTAKZWhjaTA6IFtJVEhSRUFEXQp1c2J1 czM6IEVIQ0kgdmVyc2lvbiAxLjAKdXNidXMzOiA8SW50ZWwgODI4MDFJIChJQ0g5KSBVU0Ig Mi4wIGNvbnRyb2xsZXI+IG9uIGVoY2kwCnVoY2kzOiA8SW50ZWwgODI4MDFJIChJQ0g5KSBV U0IgY29udHJvbGxlcj4gcG9ydCAweGQwMDAtMHhkMDFmIGlycSAyMyBhdCBkZXZpY2UgMjku MCBvbiBwY2kwCnVoY2kzOiBbSVRIUkVBRF0KdXNidXM0OiA8SW50ZWwgODI4MDFJIChJQ0g5 KSBVU0IgY29udHJvbGxlcj4gb24gdWhjaTMKdWhjaTQ6IDxJbnRlbCA4MjgwMUkgKElDSDkp IFVTQiBjb250cm9sbGVyPiBwb3J0IDB4ZDA4MC0weGQwOWYgaXJxIDE5IGF0IGRldmljZSAy OS4xIG9uIHBjaTAKdWhjaTQ6IFtJVEhSRUFEXQp1c2J1czU6IDxJbnRlbCA4MjgwMUkgKElD SDkpIFVTQiBjb250cm9sbGVyPiBvbiB1aGNpNAp1aGNpNTogPEludGVsIDgyODAxSSAoSUNI OSkgVVNCIGNvbnRyb2xsZXI+IHBvcnQgMHhkNDAwLTB4ZDQxZiBpcnEgMTggYXQgZGV2aWNl IDI5LjIgb24gcGNpMAp1aGNpNTogW0lUSFJFQURdCnVzYnVzNjogPEludGVsIDgyODAxSSAo SUNIOSkgVVNCIGNvbnRyb2xsZXI+IG9uIHVoY2k1CmVoY2kxOiA8SW50ZWwgODI4MDFJIChJ Q0g5KSBVU0IgMi4wIGNvbnRyb2xsZXI+IG1lbSAweGZlYTdhNDAwLTB4ZmVhN2E3ZmYgaXJx IDIzIGF0IGRldmljZSAyOS43IG9uIHBjaTAKZWhjaTE6IFtJVEhSRUFEXQp1c2J1czc6IEVI Q0kgdmVyc2lvbiAxLjAKdXNidXM3OiA8SW50ZWwgODI4MDFJIChJQ0g5KSBVU0IgMi4wIGNv bnRyb2xsZXI+IG9uIGVoY2kxCnBjaWIxOiA8QUNQSSBQQ0ktUENJIGJyaWRnZT4gYXQgZGV2 aWNlIDMwLjAgb24gcGNpMApwY2kxOiA8QUNQSSBQQ0kgYnVzPiBvbiBwY2liMQpmd29oY2kw OiA8THVjZW50IEZXMzIyLzMyMz4gbWVtIDB4ZmViZmYwMDAtMHhmZWJmZmZmZiBpcnEgMjAg YXQgZGV2aWNlIDIuMCBvbiBwY2kxCmZ3b2hjaTA6IFtJVEhSRUFEXQpmd29oY2kwOiBPSENJ IHZlcnNpb24gMS4wIChST009MSkKZndvaGNpMDogTm8uIG9mIElzb2Nocm9ub3VzIGNoYW5u ZWxzIGlzIDguCmZ3b2hjaTA6IEVVSTY0IDAwOjFlOjhjOjAwOjAwOjFhOjVmOmRhCmZ3b2hj aTA6IFBoeSAxMzk0YSBhdmFpbGFibGUgUzQwMCwgMiBwb3J0cy4KZndvaGNpMDogTGluayBT NDAwLCBtYXhfcmVjIDIwNDggYnl0ZXMuCmZpcmV3aXJlMDogPElFRUUxMzk0KEZpcmVXaXJl KSBidXM+IG9uIGZ3b2hjaTAKZndvaGNpMDogSW5pdGlhdGUgYnVzIHJlc2V0CmZ3b2hjaTA6 IGZ3b2hjaV9pbnRyX2NvcmU6IEJVUyByZXNldApmd29oY2kwOiBmd29oY2lfaW50cl9jb3Jl OiBub2RlX2lkPTB4MDAwMDAwMDAsIFNlbGZJRCBDb3VudD0xLCBDWUNMRU1BU1RFUiBtb2Rl CmlzYWIwOiA8UENJLUlTQSBicmlkZ2U+IGF0IGRldmljZSAzMS4wIG9uIHBjaTAKaXNhMDog PElTQSBidXM+IG9uIGlzYWIwCmFoY2kwOiA8SW50ZWwgSUNIOSBBSENJIFNBVEEgY29udHJv bGxlcj4gcG9ydCAweGM4ODAtMHhjODg3LDB4YzgwMC0weGM4MDMsMHhjNDgwLTB4YzQ4Nyww eGM0MDAtMHhjNDAzLDB4YzA4MC0weGMwOWYgbWVtIDB4ZmVhNzg4MDAtMHhmZWE3OGZmZiBp cnEgMjIgYXQgZGV2aWNlIDMxLjIgb24gcGNpMAphaGNpMDogW0lUSFJFQURdCmFoY2kwOiBB SENJIHYxLjIwIHdpdGggNiAzR2JwcyBwb3J0cywgUG9ydCBNdWx0aXBsaWVyIHN1cHBvcnRl ZAphaGNpY2gwOiA8QUhDSSBjaGFubmVsPiBhdCBjaGFubmVsIDAgb24gYWhjaTAKYWhjaWNo MDogW0lUSFJFQURdCmFoY2ljaDE6IDxBSENJIGNoYW5uZWw+IGF0IGNoYW5uZWwgMSBvbiBh aGNpMAphaGNpY2gxOiBbSVRIUkVBRF0KYWhjaWNoMjogPEFIQ0kgY2hhbm5lbD4gYXQgY2hh bm5lbCAyIG9uIGFoY2kwCmFoY2ljaDI6IFtJVEhSRUFEXQphaGNpY2gzOiA8QUhDSSBjaGFu bmVsPiBhdCBjaGFubmVsIDMgb24gYWhjaTAKYWhjaWNoMzogW0lUSFJFQURdCmFoY2ljaDQ6 IDxBSENJIGNoYW5uZWw+IGF0IGNoYW5uZWwgNCBvbiBhaGNpMAphaGNpY2g0OiBbSVRIUkVB RF0KYWhjaWNoNTogPEFIQ0kgY2hhbm5lbD4gYXQgY2hhbm5lbCA1IG9uIGFoY2kwCmFoY2lj aDU6IFtJVEhSRUFEXQppY2hzbWIwOiA8SW50ZWwgODI4MDFJIChJQ0g5KSBTTUJ1cyBjb250 cm9sbGVyPiBwb3J0IDB4NDAwLTB4NDFmIG1lbSAweGZlYTdhMDAwLTB4ZmVhN2EwZmYgaXJx IDE4IGF0IGRldmljZSAzMS4zIG9uIHBjaTAKaWNoc21iMDogW0lUSFJFQURdCnNtYnVzMDog PFN5c3RlbSBNYW5hZ2VtZW50IEJ1cz4gb24gaWNoc21iMApzbWIwOiA8U01CdXMgZ2VuZXJp YyBJL08+IG9uIHNtYnVzMApwY2kwOiA8ZGFzcD4gYXQgZGV2aWNlIDMxLjYgKG5vIGRyaXZl ciBhdHRhY2hlZCkKYWNwaV9idXR0b24wOiA8UG93ZXIgQnV0dG9uPiBvbiBhY3BpMAphdHJ0 YzA6IDxBVCByZWFsdGltZSBjbG9jaz4gcG9ydCAweDcwLTB4NzEgaXJxIDggb24gYWNwaTAK cHBjMDogPFBhcmFsbGVsIHBvcnQ+IHBvcnQgMHgzNzgtMHgzN2YsMHg3NzgtMHg3N2YgaXJx IDcgZHJxIDMgb24gYWNwaTAKcHBjMDogU01DLWxpa2UgY2hpcHNldCAoRUNQL0VQUC9QUzIv TklCQkxFKSBpbiBDT01QQVRJQkxFIG1vZGUKcHBjMDogRklGTyB3aXRoIDE2LzE2LzkgYnl0 ZXMgdGhyZXNob2xkCnBwYzA6IFtJVEhSRUFEXQpwcGJ1czA6IDxQYXJhbGxlbCBwb3J0IGJ1 cz4gb24gcHBjMAphY3BpX2hwZXQwOiA8SGlnaCBQcmVjaXNpb24gRXZlbnQgVGltZXI+IGlv bWVtIDB4ZmVkMDAwMDAtMHhmZWQwMDNmZiBvbiBhY3BpMApUaW1lY291bnRlciAiSFBFVCIg ZnJlcXVlbmN5IDE0MzE4MTgwIEh6IHF1YWxpdHkgOTAwCnVhcnQwOiA8MTY1NTAgb3IgY29t cGF0aWJsZT4gcG9ydCAweDNmOC0weDNmZiBpcnEgNCBmbGFncyAweDEwIG9uIGFjcGkwCnVh cnQwOiBbRklMVEVSXQphdGtiZGMwOiA8S2V5Ym9hcmQgY29udHJvbGxlciAoaTgwNDIpPiBw b3J0IDB4NjAsMHg2NCBpcnEgMSBvbiBhY3BpMAphdGtiZDA6IDxBVCBLZXlib2FyZD4gaXJx IDEgb24gYXRrYmRjMAprYmQwIGF0IGF0a2JkMAphdGtiZDA6IFtHSUFOVC1MT0NLRURdCmF0 a2JkMDogW0lUSFJFQURdCm9ybTA6IDxJU0EgT3B0aW9uIFJPTT4gYXQgaW9tZW0gMHhjYjgw MC0weGNjN2ZmIG9uIGlzYTAKc2MwOiA8U3lzdGVtIGNvbnNvbGU+IGF0IGZsYWdzIDB4MTAw IG9uIGlzYTAKc2MwOiBWR0EgPDE2IHZpcnR1YWwgY29uc29sZXMsIGZsYWdzPTB4MzAwPgp2 Z2EwOiA8R2VuZXJpYyBJU0EgVkdBPiBhdCBwb3J0IDB4M2MwLTB4M2RmIGlvbWVtIDB4YTAw MDAtMHhiZmZmZiBvbiBpc2EwCmNvcmV0ZW1wMDogPENQVSBPbi1EaWUgVGhlcm1hbCBTZW5z b3JzPiBvbiBjcHUwCmVzdDA6IDxFbmhhbmNlZCBTcGVlZFN0ZXAgRnJlcXVlbmN5IENvbnRy b2w+IG9uIGNwdTAKcDR0Y2MwOiA8Q1BVIEZyZXF1ZW5jeSBUaGVybWFsIENvbnRyb2w+IG9u IGNwdTAKY29yZXRlbXAxOiA8Q1BVIE9uLURpZSBUaGVybWFsIFNlbnNvcnM+IG9uIGNwdTEK ZXN0MTogPEVuaGFuY2VkIFNwZWVkU3RlcCBGcmVxdWVuY3kgQ29udHJvbD4gb24gY3B1MQpw NHRjYzE6IDxDUFUgRnJlcXVlbmN5IFRoZXJtYWwgQ29udHJvbD4gb24gY3B1MQpUaW1lY291 bnRlcnMgdGljayBldmVyeSAxLjAwMCBtc2VjCmZpcmV3aXJlMDogMSBub2RlcywgbWF4aG9w IDw9IDAgY2FibGUgSVJNIGlybSgwKSAgKG1lKSAKZmlyZXdpcmUwOiBidXMgbWFuYWdlciAw IApHRU9NX1JBSUQ1OiBNb2R1bGUgbG9hZGVkLCB2ZXJzaW9uIDEuMC4yMDEwMTEyNi40OSAo cmV2IDhmMjM2NjAxNmRkMSkKdXNidXMwOiAxMk1icHMgRnVsbCBTcGVlZCBVU0IgdjEuMAp1 c2J1czE6IDEyTWJwcyBGdWxsIFNwZWVkIFVTQiB2MS4wCnVzYnVzMjogMTJNYnBzIEZ1bGwg U3BlZWQgVVNCIHYxLjAKdXNidXMzOiA0ODBNYnBzIEhpZ2ggU3BlZWQgVVNCIHYyLjAKdXNi dXM0OiAxMk1icHMgRnVsbCBTcGVlZCBVU0IgdjEuMAp1c2J1czU6IDEyTWJwcyBGdWxsIFNw ZWVkIFVTQiB2MS4wCnVzYnVzNjogMTJNYnBzIEZ1bGwgU3BlZWQgVVNCIHYxLjAKdXNidXM3 OiA0ODBNYnBzIEhpZ2ggU3BlZWQgVVNCIHYyLjAKdWdlbjAuMTogPEludGVsPiBhdCB1c2J1 czAKdWh1YjA6IDxJbnRlbCBVSENJIHJvb3QgSFVCLCBjbGFzcyA5LzAsIHJldiAxLjAwLzEu MDAsIGFkZHIgMT4gb24gdXNidXMwCnVnZW4xLjE6IDxJbnRlbD4gYXQgdXNidXMxCnVodWIx OiA8SW50ZWwgVUhDSSByb290IEhVQiwgY2xhc3MgOS8wLCByZXYgMS4wMC8xLjAwLCBhZGRy IDE+IG9uIHVzYnVzMQp1Z2VuMi4xOiA8SW50ZWw+IGF0IHVzYnVzMgp1aHViMjogPEludGVs IFVIQ0kgcm9vdCBIVUIsIGNsYXNzIDkvMCwgcmV2IDEuMDAvMS4wMCwgYWRkciAxPiBvbiB1 c2J1czIKdWdlbjMuMTogPEludGVsPiBhdCB1c2J1czMKdWh1YjM6IDxJbnRlbCBFSENJIHJv b3QgSFVCLCBjbGFzcyA5LzAsIHJldiAyLjAwLzEuMDAsIGFkZHIgMT4gb24gdXNidXMzCnVn ZW40LjE6IDxJbnRlbD4gYXQgdXNidXM0CnVodWI0OiA8SW50ZWwgVUhDSSByb290IEhVQiwg Y2xhc3MgOS8wLCByZXYgMS4wMC8xLjAwLCBhZGRyIDE+IG9uIHVzYnVzNAp1Z2VuNS4xOiA8 SW50ZWw+IGF0IHVzYnVzNQp1aHViNTogPEludGVsIFVIQ0kgcm9vdCBIVUIsIGNsYXNzIDkv MCwgcmV2IDEuMDAvMS4wMCwgYWRkciAxPiBvbiB1c2J1czUKdWdlbjYuMTogPEludGVsPiBh dCB1c2J1czYKdWh1YjY6IDxJbnRlbCBVSENJIHJvb3QgSFVCLCBjbGFzcyA5LzAsIHJldiAx LjAwLzEuMDAsIGFkZHIgMT4gb24gdXNidXM2CnVnZW43LjE6IDxJbnRlbD4gYXQgdXNidXM3 CnVodWI3OiA8SW50ZWwgRUhDSSByb290IEhVQiwgY2xhc3MgOS8wLCByZXYgMi4wMC8xLjAw LCBhZGRyIDE+IG9uIHVzYnVzNwphZGEwIGF0IGFoY2ljaDAgYnVzIDAgc2NidXMwIHRhcmdl dCAwIGx1biAwCmFkYTA6IDxTQU1TVU5HIEhEMzIxS0ogQ1AxMDAtMTA+IEFUQS04IFNBVEEg Mi54IGRldmljZQphZGEwOiAzMDAuMDAwTUIvcyB0cmFuc2ZlcnMgKFNBVEEgMi54LCBVRE1B NiwgUElPIDgxOTJieXRlcykKYWRhMDogQ29tbWFuZCBRdWV1ZWluZyBlbmFibGVkCmFkYTA6 IDMwNTI0NU1CICg2MjUxNDI0NDggNTEyIGJ5dGUgc2VjdG9yczogMTZIIDYzUy9UIDE2Mzgz QykKYWRhMSBhdCBhaGNpY2gxIGJ1cyAwIHNjYnVzMSB0YXJnZXQgMCBsdW4gMAphZGExOiA8 V0RDIFdENTAwMEFBS1MtMDBZR0EwIDEyLjAxQzAyPiBBVEEtOCBTQVRBIDIueCBkZXZpY2UK YWRhMTogMzAwLjAwME1CL3MgdHJhbnNmZXJzIChTQVRBIDIueCwgVURNQTYsIFBJTyA4MTky Ynl0ZXMpCmFkYTE6IENvbW1hbmQgUXVldWVpbmcgZW5hYmxlZAphZGExOiA0NzY5NDBNQiAo OTc2NzczMTY4IDUxMiBieXRlIHNlY3RvcnM6IDE2SCA2M1MvVCAxNjM4M0MpCmFkYTIgYXQg YWhjaWNoMiBidXMgMCBzY2J1czIgdGFyZ2V0IDAgbHVuIDAKYWRhMjogPFdEQyBXRDUwMDBB QUtTLTAwQTdCMiAwMS4wM0IwMT4gQVRBLTggU0FUQSAyLnggZGV2aWNlCmFkYTI6IDMwMC4w MDBNQi9zIHRyYW5zZmVycyAoU0FUQSAyLngsIFVETUE2LCBQSU8gODE5MmJ5dGVzKQphZGEy OiBDb21tYW5kIFF1ZXVlaW5nIGVuYWJsZWQKYWRhMjogNDc2OTQwTUIgKDk3Njc3MzE2OCA1 MTIgYnl0ZSBzZWN0b3JzOiAxNkggNjNTL1QgMTYzODNDKQphZGEzIGF0IGFoY2ljaDMgYnVz IDAgc2NidXMzIHRhcmdldCAwIGx1biAwCmFkYTM6IDxXREMgV0Q1MDAwQUFLUy0wMFlHQTAg MTIuMDFDMDI+IEFUQS04IFNBVEEgMi54IGRldmljZQphZGEzOiAzMDAuMDAwTUIvcyB0cmFu c2ZlcnMgKFNBVEEgMi54LCBVRE1BNiwgUElPIDgxOTJieXRlcykKYWRhMzogQ29tbWFuZCBR dWV1ZWluZyBlbmFibGVkCmFkYTM6IDQ3Njk0ME1CICg5NzY3NzMxNjggNTEyIGJ5dGUgc2Vj dG9yczogMTZIIDYzUy9UIDE2MzgzQykKYWRhNCBhdCBhaGNpY2g0IGJ1cyAwIHNjYnVzNCB0 YXJnZXQgMCBsdW4gMAphZGE0OiA8V0RDIFdENTAwMEFBS1MtMDBZR0EwIDEyLjAxQzAyPiBB VEEtOCBTQVRBIDIueCBkZXZpY2UKYWRhNDogMzAwLjAwME1CL3MgdHJhbnNmZXJzIChTQVRB IDIueCwgVURNQTYsIFBJTyA4MTkyYnl0ZXMpCmFkYTQ6IENvbW1hbmQgUXVldWVpbmcgZW5h YmxlZAphZGE0OiA0NzY5NDBNQiAoOTc2NzczMTY4IDUxMiBieXRlIHNlY3RvcnM6IDE2SCA2 M1MvVCAxNjM4M0MpCmFkYTUgYXQgYWhjaWNoNSBidXMgMCBzY2J1czUgdGFyZ2V0IDAgbHVu IDAKYWRhNTogPFdEQyBXRDUwMDBBQUtTLTAwWUdBMCAxMi4wMUMwMj4gQVRBLTggU0FUQSAy LnggZGV2aWNlCmFkYTU6IDMwMC4wMDBNQi9zIHRyYW5zZmVycyAoU0FUQSAyLngsIFVETUE2 LCBQSU8gODE5MmJ5dGVzKQphZGE1OiBDb21tYW5kIFF1ZXVlaW5nIGVuYWJsZWQKYWRhNTog NDc2OTQwTUIgKDk3Njc3MzE2OCA1MTIgYnl0ZSBzZWN0b3JzOiAxNkggNjNTL1QgMTYzODND KQpTTVA6IEFQIENQVSAjMSBMYXVuY2hlZCEKdWh1YjA6IDIgcG9ydHMgd2l0aCAyIHJlbW92 YWJsZSwgc2VsZiBwb3dlcmVkCnVodWIxOiAyIHBvcnRzIHdpdGggMiByZW1vdmFibGUsIHNl bGYgcG93ZXJlZAp1aHViMjogMiBwb3J0cyB3aXRoIDIgcmVtb3ZhYmxlLCBzZWxmIHBvd2Vy ZWQKdWh1YjQ6IDIgcG9ydHMgd2l0aCAyIHJlbW92YWJsZSwgc2VsZiBwb3dlcmVkCnVodWI1 OiAyIHBvcnRzIHdpdGggMiByZW1vdmFibGUsIHNlbGYgcG93ZXJlZAp1aHViNjogMiBwb3J0 cyB3aXRoIDIgcmVtb3ZhYmxlLCBzZWxmIHBvd2VyZWQKR0VPTV9SQUlENTogc3RvcmFnZTog ZGV2aWNlIGNyZWF0ZWQgKHN0cmlwZXNpemU9MTMxMDcyKS4KR0VPTV9SQUlENTogc3RvcmFn ZTogYWRhMSg0KTogZGlzayBhdHRhY2hlZC4KR0VPTV9SQUlENTogc3RvcmFnZTogYWRhMigz KTogZGlzayBhdHRhY2hlZC4KR0VPTV9SQUlENTogc3RvcmFnZTogYWRhMygyKTogZGlzayBh dHRhY2hlZC4KR0VPTV9SQUlENTogc3RvcmFnZTogYWRhNCgwKTogZGlzayBhdHRhY2hlZC4K R0VPTV9SQUlENTogc3RvcmFnZTogYWRhNSgxKTogZGlzayBhdHRhY2hlZC4KR0VPTV9SQUlE NTogc3RvcmFnZTogYWN0aXZhdGVkIChuZWVkIGFib3V0IDk1TWlCIGttZW0gKG1heCkpLgpS b290IG1vdW50IHdhaXRpbmcgZm9yOiB1c2J1czcgdXNidXMzClJvb3QgbW91bnQgd2FpdGlu ZyBmb3I6IHVzYnVzNyB1c2J1czMKdWh1YjM6IDYgcG9ydHMgd2l0aCA2IHJlbW92YWJsZSwg c2VsZiBwb3dlcmVkCnVodWI3OiA2IHBvcnRzIHdpdGggNiByZW1vdmFibGUsIHNlbGYgcG93 ZXJlZAp1Z2VuMy4yOiA8dmVuZG9yIDB4OTcxMD4gYXQgdXNidXMzCnVtY3M3ODQwMDogPHZl bmRvciAweDk3MTAgcHJvZHVjdCAweDc4NDAsIHJldiAyLjAwLzAuMDEsIGFkZHIgMj4gb24g dXNidXMzCnVtY3M3ODQwMDogQ2hpcCBtY3M3ODQwLCBmb3VuZCAyIGFjdGl2ZSBwb3J0cwp1 bWNzNzg0MDA6IE9uLWRpZSBjb25mZ3VyYXRpb246IFJTVDogYWN0aXZlIGxvdywgSFJEOiB5 ZXMsIFBMTDogYXZhaWwsIFBPUjogYXZhaWwsIFBvcnRzOiA0LCBFRVBST00gd3JpdGUgZGlz YWJsZWQsIElyREEgaXMgbm90IGF2YWlsYWJsZQpSb290IG1vdW50IHdhaXRpbmcgZm9yOiB1 c2J1czcKdWdlbjcuMjogPEdlbmVyaWM+IGF0IHVzYnVzNwp1bWFzczA6IDxHZW5lcmljIE1h c3MgU3RvcmFnZSBEZXZpY2UsIGNsYXNzIDAvMCwgcmV2IDIuMDAvMS4wMCwgYWRkciAyPiBv biB1c2J1czcKUm9vdCBtb3VudCB3YWl0aW5nIGZvcjogdXNidXM3ClRyeWluZyB0byBtb3Vu dCByb290IGZyb20gdWZzOi9kZXYvYWRhMHMxYQpkYTAgYXQgdW1hc3Mtc2ltMCBidXMgMCBz Y2J1czYgdGFyZ2V0IDAgbHVuIDAKZGEwOiA8R2VuZXJpYyBVU0IgU0QgUmVhZGVyIDEuMDA+ IFJlbW92YWJsZSBEaXJlY3QgQWNjZXNzIFNDU0ktMCBkZXZpY2UgCmRhMDogNDAuMDAwTUIv cyB0cmFuc2ZlcnMKZGEwOiBBdHRlbXB0IHRvIHF1ZXJ5IGRldmljZSBzaXplIGZhaWxlZDog Tk9UIFJFQURZLCBNZWRpdW0gbm90IHByZXNlbnQKZGExIGF0IHVtYXNzLXNpbTAgYnVzIDAg c2NidXM2IHRhcmdldCAwIGx1biAxCmRhMTogPEdlbmVyaWMgVVNCIENGIFJlYWRlciAxLjAx PiBSZW1vdmFibGUgRGlyZWN0IEFjY2VzcyBTQ1NJLTAgZGV2aWNlIApkYTE6IDQwLjAwME1C L3MgdHJhbnNmZXJzCmRhMTogQXR0ZW1wdCB0byBxdWVyeSBkZXZpY2Ugc2l6ZSBmYWlsZWQ6 IE5PVCBSRUFEWSwgTWVkaXVtIG5vdCBwcmVzZW50CmRhMiBhdCB1bWFzcy1zaW0wIGJ1cyAw IHNjYnVzNiB0YXJnZXQgMCBsdW4gMgpkYTI6IDxHZW5lcmljIFVTQiBTTSBSZWFkZXIgMS4w Mj4gUmVtb3ZhYmxlIERpcmVjdCBBY2Nlc3MgU0NTSS0wIGRldmljZSAKZGEyOiA0MC4wMDBN Qi9zIHRyYW5zZmVycwpkYTI6IEF0dGVtcHQgdG8gcXVlcnkgZGV2aWNlIHNpemUgZmFpbGVk OiBOT1QgUkVBRFksIE1lZGl1bSBub3QgcHJlc2VudApkYTMgYXQgdW1hc3Mtc2ltMCBidXMg MCBzY2J1czYgdGFyZ2V0IDAgbHVuIDMKZGEzOiA8R2VuZXJpYyBVU0IgTVMgUmVhZGVyIDEu MDM+IFJlbW92YWJsZSBEaXJlY3QgQWNjZXNzIFNDU0ktMCBkZXZpY2UgCmRhMzogNDAuMDAw TUIvcyB0cmFuc2ZlcnMKZGEzOiBBdHRlbXB0IHRvIHF1ZXJ5IGRldmljZSBzaXplIGZhaWxl ZDogTk9UIFJFQURZLCBNZWRpdW0gbm90IHByZXNlbnQK ------------4D51C134ECDCF3-- From owner-freebsd-stable@FreeBSD.ORG Thu Jan 6 11:26:35 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EEE6F106566B for ; Thu, 6 Jan 2011 11:26:35 +0000 (UTC) (envelope-from cforgeron@acsi.ca) Received: from mta03.eastlink.ca (mta03.eastlink.ca [24.224.136.9]) by mx1.freebsd.org (Postfix) with ESMTP id AE3198FC26 for ; Thu, 6 Jan 2011 11:26:35 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from ip05.eastlink.ca ([unknown] [24.222.39.68]) by mta03.eastlink.ca (Sun Java(tm) System Messaging Server 7.3-11.01 64bit (built Sep 1 2009)) with ESMTP id <0LEL007X5MGACF02@mta03.eastlink.ca> for freebsd-stable@freebsd.org; Thu, 06 Jan 2011 07:26:34 -0400 (AST) X-CMAE-Score: 0 X-CMAE-Analysis: v=1.1 cv=8reSTVRqS4Rq5Xx4Jai9N41eZpHz3D5gSX5rA0od4mg= c=1 sm=1 a=tZDC3L0OECsA:10 a=kj9zAlcOel0A:10 a=6I5d2MoRAAAA:8 a=T1nuC21ZD-St4ExDUygA:9 a=IoEjELlaKcY4AH21wZ8A:7 a=4QatThj9dcmXS8ILLZVmoNZcXjUA:4 a=CjuIK1q_8ugA:10 a=SV7veod9ZcQA:10 a=/bLbuBD0lrv91xL1PDQKaA==:117 Received: from blk-222-10-85.eastlink.ca (HELO server7.acsi.ca) ([24.222.10.85]) by ip05.eastlink.ca with ESMTP; Thu, 06 Jan 2011 07:26:34 -0400 Received: from server7.acsi.ca ([192.168.9.7]) by server7.acsi.ca ([192.168.9.7]) with mapi; Thu, 06 Jan 2011 07:26:34 -0400 From: Chris Forgeron To: Damien Fleuriot , Artem Belevich Date: Thu, 06 Jan 2011 07:26:32 -0400 Thread-topic: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks Thread-index: AcutgwmStifwwHkTT2GLMQ23IwicogAETuug Message-id: References: <4D1C6F90.3080206@my.gd> <4D21E679.80002@my.gd> <84882169-0461-480F-8B4C-58E794BCC8E6@my.gd> <488AE93A-97B9-4F01-AD0A-0098E4B329C3@my.gd> In-reply-to: Accept-Language: en-US Content-language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Cc: "freebsd-stable@freebsd.org" Subject: RE: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Jan 2011 11:26:36 -0000 You know, these days I'm not as happy with SSD's for ZIL. I may blog about some of the speed results I've been getting over the last 6mo-1yr that I've been running them with ZFS. I think people should be using hardware RAM drives. You can get old Gigabyte i-RAM drives with 4 gig of memory for the cost of a 60 gig SSD, and it will trounce the SSD for speed. I'd put your SSD to L2ARC (cache). -----Original Message----- From: Damien Fleuriot [mailto:ml@my.gd] Sent: Thursday, January 06, 2011 5:20 AM To: Artem Belevich Cc: Chris Forgeron; freebsd-stable@freebsd.org Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks You both make good points, thanks for the feedback :) I am more concerned about data protection than performance, so I suppose raidz2 is the best choice I have with such a small scale setup. Now the question that remains is wether or not to use parts of the OS's ssd for zil, cache, or both ? --- Fleuriot Damien On 5 Jan 2011, at 23:12, Artem Belevich wrote: > On Wed, Jan 5, 2011 at 1:55 PM, Damien Fleuriot wrote: >> Well actually... >> >> raidz2: >> - 7x 1.5 tb = 10.5tb >> - 2 parity drives >> >> raidz1: >> - 3x 1.5 tb = 4.5 tb >> - 4x 1.5 tb = 6 tb , total 10.5tb >> - 2 parity drives in split thus different raidz1 arrays >> >> So really, in both cases 2 different parity drives and same storage... > > In second case you get better performance, but lose some data > protection. It's still raidz1 and you can't guarantee functionality in > all cases of two drives failing. If two drives fail in the same vdev, > your entire pool will be gone. Granted, it's better than single-vdev > raidz1, but it's *not* as good as raidz2. > > --Artem From owner-freebsd-stable@FreeBSD.ORG Thu Jan 6 11:34:35 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 88249106564A for ; Thu, 6 Jan 2011 11:34:35 +0000 (UTC) (envelope-from bschmidt@techwires.net) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 197E08FC0A for ; Thu, 6 Jan 2011 11:34:34 +0000 (UTC) Received: by bwz12 with SMTP id 12so8994150bwz.13 for ; Thu, 06 Jan 2011 03:34:33 -0800 (PST) Received: by 10.204.114.148 with SMTP id e20mr18298397bkq.168.1294313673718; Thu, 06 Jan 2011 03:34:33 -0800 (PST) Received: from jessie.localnet (p5B2EC5B8.dip0.t-ipconnect.de [91.46.197.184]) by mx.google.com with ESMTPS id p22sm13375115bkp.9.2011.01.06.03.34.32 (version=SSLv3 cipher=RC4-MD5); Thu, 06 Jan 2011 03:34:32 -0800 (PST) Sender: Bernhard Schmidt From: Bernhard Schmidt To: "Olivier =?windows-1252?q?Cochard-Labb=E9?=" Date: Thu, 6 Jan 2011 12:34:39 +0100 User-Agent: KMail/1.13.2 (Linux/2.6.32-25-generic; KDE/4.4.2; i686; ; ) References: <201101061046.02128.bschmidt@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="windows-1252" Content-Transfer-Encoding: quoted-printable Message-Id: <201101061234.39666.bschmidt@freebsd.org> Cc: freebsd-stable@freebsd.org Subject: Re: Regression with iwn drivers (4965BGN) in 8.2-PRERELEASE ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: bschmidt@freebsd.org List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Jan 2011 11:34:35 -0000 On Thursday, January 06, 2011 11:23:44 Olivier Cochard-Labb=E9 wrote: > 2011/1/6 Bernhard Schmidt : > > What do you mean with 'unusable' exactly? Lots of packet loss, or just > > slow transfer rates? 'wlandebug +rate' might shed some light on this > > one. >=20 > Hi, it's just very slow transfer rates. > I didn't know wlandebug, thanks for the tips. > Here are the result just after a boot, during pinging my gateway (few > traffic): >=20 > Jan 6 11:02:25 d630 kernel: wlan0: [3a:41:c4:e3:1e:18] AMRR > increasing rate 48 (txcnt=3D11 retrycnt=3D0) > Jan 6 11:02:36 d630 kernel: wlan0: [3a:41:c4:e3:1e:18] AMRR > increasing rate 72 (txcnt=3D11 retrycnt=3D0) > Jan 6 11:03:02 d630 kernel: wlan0: [3a:41:c4:e3:1e:18] AMRR > increasing rate 96 (txcnt=3D11 retrycnt=3D0) >=20 > Now, I start xorg and a start a browser: >=20 > Jan 6 11:04:04 d630 kernel: wlan0: [3a:41:c4:e3:1e:18] AMRR > decreasing rate 72 (txcnt=3D11 retrycnt=3D10) > [..] > Jan 6 11:11:09 d630 kernel: wlan0: [3a:41:c4:e3:1e:18] AMRR > decreasing rate 4 (txcnt=3D11 retrycnt=3D4) >=20 > The rate decrease too much for using a browser (but I can still ping > my gateway)=85 That looks indeed quite weird. I'll have a look into that. Can you post 'ifconfig wlan0 list scan' output, just to see how staffed the= =20 band is? =2D-=20 Bernhard From owner-freebsd-stable@FreeBSD.ORG Thu Jan 6 12:26:38 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 34D4F106564A for ; Thu, 6 Jan 2011 12:26:38 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 8681C8FC0C for ; Thu, 6 Jan 2011 12:26:36 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p06CQWOx063486 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 6 Jan 2011 14:26:32 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p06CQWsB024516; Thu, 6 Jan 2011 14:26:32 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p06CQWxk024515; Thu, 6 Jan 2011 14:26:32 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 6 Jan 2011 14:26:32 +0200 From: Kostik Belousov To: Lev Serebryakov Message-ID: <20110106122632.GO12599@deviant.kiev.zoral.com.ua> References: <1608571029.20110106133145@serebryakov.spb.ru> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="lLR1BQqf7txDtYcF" Content-Disposition: inline In-Reply-To: <1608571029.20110106133145@serebryakov.spb.ru> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: "freebsd-stable@freebsd.org" Subject: Re: FreeBSD 8.2-PRERELEASE hangs under load with "live" kernel X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Jan 2011 12:26:38 -0000 --lLR1BQqf7txDtYcF Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jan 06, 2011 at 01:31:45PM +0300, Lev Serebryakov wrote: > Hello, Freebsd-stable. >=20 > I've added torrent client (transmission) to software on my home > server and it starts to hang in very unusual way: kernel works but > userland doesn't. >=20 > I can ping it (and it answers). I can scroll console with > "scrolllock" button and keys. I can break into debugger with > Ctrl+SysReq and it shows, that one CPU is occupied by idle process and > other by "Giant tasq", but no userland processes answer: I can not > ssh to it, I cannot login on console, samba is dead, etc. >=20 > "ps" in kernel debugger shows, that many of processes in "pfault" > state, and noting more "special". >=20 > memtest86+ doesn't show any errors after 8 passes of tests (about > 10 hours), so RAM looks Ok. >=20 > What should I do in kdb to understand what happens? >=20 > Kernel config and /var/run/dmesg.boot is attached. http://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/kernel= debug-deadlocks.html --lLR1BQqf7txDtYcF Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk0ltPgACgkQC3+MBN1Mb4hAWwCfR7NEV7956dk02hAdeZu2LjLu bNMAoMix6mFxq7lL3mQ0btSuGbMN1xmx =dEsS -----END PGP SIGNATURE----- --lLR1BQqf7txDtYcF-- From owner-freebsd-stable@FreeBSD.ORG Thu Jan 6 12:59:09 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 932931065695; Thu, 6 Jan 2011 12:59:09 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 3F1698FC12; Thu, 6 Jan 2011 12:59:08 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqAGANtLJU2DaFvO/2dsb2JhbACDd5ISjwKwBY1ZgSGDN3QEhGeGIg X-IronPort-AV: E=Sophos;i="4.60,283,1291611600"; d="scan'208";a="104446800" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 06 Jan 2011 07:58:55 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 3FD1D793A7; Thu, 6 Jan 2011 07:58:55 -0500 (EST) Date: Thu, 6 Jan 2011 07:58:55 -0500 (EST) From: Rick Macklem To: perryh@pluto.rain.com Message-ID: <1301662185.172571.1294318735174.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <4d258100.Wu9WZrgdH2/KvDTa%perryh@pluto.rain.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - IE8 (Win)/6.0.10_GA_2692) Cc: marek sal , freebsd-stable@freebsd.org, milu@dat.pl, jyavenard@gmail.com, jhb@freebsd.org Subject: Re: NFSv4 - how to set up at FreeBSD 8.1 ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Jan 2011 12:59:09 -0000 > Rick Macklem wrote: > > > Sun did add a separate file locking protocol called the NLM > > or rpc.lockd if you prefer, but that protocol design was > > fundamentally flawed imho and, as such, using it is in the > > "your mileage may vary" category. > > I suppose it was not all that bad, considering that what it sought > to accomplish is incomputable. There is simply no way for either > the server or the client to distinguish between "the other end has > crashed" and "there is a temporary communication failure" until the > other end comes back up or communication is restored. > Yep. The blocking lock operation is also a trainwreck looking for a place to happen, imho. (In the NLM, the client can do an RPC that says "get a lock, waiting as long as necessary for it, and then let me know".) > On a good day, in a completely homogeneous environment (server and > all clients running the same OS revision and patchlevel), I trust > lockd about as far as I can throw 10GB of 1980's SMD disk drives :) > Heh, heh. For those too young to have had the priviledge, a 1980s SMD drive was big and HEAVY. I just about got a hernia every time one had to go in a 19inch rack. You definitely didn't throw them far:-) > Exporting /var/spool/mail read/write tends to ensure that good days > will be rare. Been there, done that, seen the result. Never again. > That's what IMAP is for. > Great post. I couldn't have said it as well, rick From owner-freebsd-stable@FreeBSD.ORG Thu Jan 6 13:11:50 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 905741065694 for ; Thu, 6 Jan 2011 13:11:50 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 285958FC13 for ; Thu, 6 Jan 2011 13:11:49 +0000 (UTC) Received: by wwf26 with SMTP id 26so16150915wwf.31 for ; Thu, 06 Jan 2011 05:11:48 -0800 (PST) Received: by 10.227.166.13 with SMTP id k13mr876364wby.178.1294319508641; Thu, 06 Jan 2011 05:11:48 -0800 (PST) Received: from dfleuriot.local ([195.234.88.154]) by mx.google.com with ESMTPS id m10sm16785304wbc.10.2011.01.06.05.11.46 (version=SSLv3 cipher=RC4-MD5); Thu, 06 Jan 2011 05:11:47 -0800 (PST) Message-ID: <4D25BF91.7070304@my.gd> Date: Thu, 06 Jan 2011 14:11:45 +0100 From: Damien Fleuriot User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: Chris Forgeron References: <4D1C6F90.3080206@my.gd> <4D21E679.80002@my.gd> <84882169-0461-480F-8B4C-58E794BCC8E6@my.gd> <488AE93A-97B9-4F01-AD0A-0098E4B329C3@my.gd> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: "freebsd-stable@freebsd.org" , Artem Belevich Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Jan 2011 13:11:50 -0000 I see, so no dedicated ZIL device in the end ? I could make a 15gb slice for the OS running UFS (I don't wanna risk losing the OS when manipulating ZFS, such as during upgrades), and a 25gb+ for L2ARC, depending on the disk. I can't afford a *dedicated* drive for the cache though, not enough room in the machine. On 1/6/11 12:26 PM, Chris Forgeron wrote: > You know, these days I'm not as happy with SSD's for ZIL. I may blog about some of the speed results I've been getting over the last 6mo-1yr that I've been running them with ZFS. I think people should be using hardware RAM drives. You can get old Gigabyte i-RAM drives with 4 gig of memory for the cost of a 60 gig SSD, and it will trounce the SSD for speed. > > I'd put your SSD to L2ARC (cache). > > > -----Original Message----- > From: Damien Fleuriot [mailto:ml@my.gd] > Sent: Thursday, January 06, 2011 5:20 AM > To: Artem Belevich > Cc: Chris Forgeron; freebsd-stable@freebsd.org > Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks > > You both make good points, thanks for the feedback :) > > I am more concerned about data protection than performance, so I suppose raidz2 is the best choice I have with such a small scale setup. > > Now the question that remains is wether or not to use parts of the OS's ssd for zil, cache, or both ? > > --- > Fleuriot Damien > > On 5 Jan 2011, at 23:12, Artem Belevich wrote: > >> On Wed, Jan 5, 2011 at 1:55 PM, Damien Fleuriot wrote: >>> Well actually... >>> >>> raidz2: >>> - 7x 1.5 tb = 10.5tb >>> - 2 parity drives >>> >>> raidz1: >>> - 3x 1.5 tb = 4.5 tb >>> - 4x 1.5 tb = 6 tb , total 10.5tb >>> - 2 parity drives in split thus different raidz1 arrays >>> >>> So really, in both cases 2 different parity drives and same storage... >> >> In second case you get better performance, but lose some data >> protection. It's still raidz1 and you can't guarantee functionality in >> all cases of two drives failing. If two drives fail in the same vdev, >> your entire pool will be gone. Granted, it's better than single-vdev >> raidz1, but it's *not* as good as raidz2. >> >> --Artem From owner-freebsd-stable@FreeBSD.ORG Thu Jan 6 13:18:12 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2B15A1065679; Thu, 6 Jan 2011 13:18:12 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id CB6848FC18; Thu, 6 Jan 2011 13:18:11 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqAGAF9PJU2DaFvO/2dsb2JhbACDd5ITjwKwCI1YgSGDN3QEhGc8hWY X-IronPort-AV: E=Sophos;i="4.60,283,1291611600"; d="scan'208";a="104448547" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 06 Jan 2011 08:18:11 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 1F167B3F6B; Thu, 6 Jan 2011 08:18:11 -0500 (EST) Date: Thu, 6 Jan 2011 08:18:11 -0500 (EST) From: Rick Macklem To: perryh@pluto.rain.com Message-ID: <1634633777.173061.1294319891074.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <4d257864.YjgoS235V8eKvUX2%perryh@pluto.rain.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - IE8 (Win)/6.0.10_GA_2692) Cc: marek sal , freebsd-stable@freebsd.org, milu@dat.pl, jyavenard@gmail.com, jhb@freebsd.org Subject: Re: NFSv4 - how to set up at FreeBSD 8.1 ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Jan 2011 13:18:12 -0000 > John Baldwin wrote: > > > ... even NFS UDP mounts maintain their own set of "socket" state > > to manage retries and retransmits for UDP RPCs. > > Not according to what I remember of the SunOS NFS documentation, > which indicated that the driving force behind using UDP instead of > TCP was to have the server be _completely_ stateless. (Of course > locking is inherently stateful; they made it very clear that the > locking protocol was considered to be an adjunct rather than part > of the NFS protocol itself.) > For UDP, in the server all requests show up at socket/port 2049. They pretty quickly discovered that retries of non-idempotent RPCs trashed things, so the Duplicate Request Cache was invented, which is really state that doesn't have to be recovered after a server crash. (By Chet Jacuzak at DEC, if I recall correctly, who is living on a little island on a lake up in Maine, last I heard.) My recollection of why Sun didn't use TCP was that "they knew that the overhead would be excessive", which wasn't completely untrue, given the speed of an MC68020. > It's been quite a few years since I read that, and I didn't get > into the details, but I suppose the handle returned to a client (in > response to a mount or open request) must have contained both a > representation of the inode number and a unique identification of > the filesystem (so that, in the case where server crash recovery > included a newfs and reload from backup, the FS ID would not match > and the client would get a "stale handle" response). All of the > retry and retransmit burden had to have been managed by the client, > for both reading and writing. Yea, it depended on how the backup was done. To avoid "stale handle" the backup/reload had to retain the same i-nodes, including the generation number in them. (But, then, those 1980s SMD disks never trashed the file systems, or did they?:-) You shouldn't get me reminising on the good ole days, rick From owner-freebsd-stable@FreeBSD.ORG Thu Jan 6 14:13:25 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 91AD110656A7 for ; Thu, 6 Jan 2011 14:13:25 +0000 (UTC) (envelope-from daniel@digsys.bg) Received: from relay2.digsys.bg (varna.digsys.bg [192.92.129.9]) by mx1.freebsd.org (Postfix) with ESMTP id 12EC18FC1B for ; Thu, 6 Jan 2011 14:13:24 +0000 (UTC) Received: from dcave.digsys.bg (daniel@dcave.digsys.bg [192.92.129.5]) by relay2.digsys.bg (8.14.4/8.14.4) with ESMTP id p06Dj4Er032946 for ; Thu, 6 Jan 2011 15:45:04 +0200 (EET) (envelope-from daniel@digsys.bg) Message-ID: <4D25C760.4070703@digsys.bg> Date: Thu, 06 Jan 2011 15:45:04 +0200 From: Daniel Kalchev User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.13) Gecko/20101217 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-stable@freebsd.org References: <4D1C6F90.3080206@my.gd> <4D21E679.80002@my.gd> <84882169-0461-480F-8B4C-58E794BCC8E6@my.gd> <488AE93A-97B9-4F01-AD0A-0098E4B329C3@my.gd> <4D25BF91.7070304@my.gd> In-Reply-To: <4D25BF91.7070304@my.gd> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Jan 2011 14:13:25 -0000 For pure storage, that is a place you send/store files, you don't really need the ZIL. You also need the L2ARC only if you read over and over again the same dataset, which is larger than the available ARC (ZFS cache memory). Both will not be significant for 'backup server' application, because it's very unlikely to do lots of SYNC I/O (where separate ZIL helps), or serve the same files back (where the L2ARC might help). You should also know that having large L2ARC requires that you also have larger ARC, because there are data pointers in the ARC that point to the L2ARC data. Someone will do good to the community to publish some reasonable estimates of the memory needs, so that people do not end up with large but unusable L2ARC setups. It seems that the upcoming v28 ZFS will help greatly with the ZIL in the main pool.. You need to experiment with the L2ARC (this is safe with current v14 and v15 pools) to see if your usage will see benefit from it's use. Experimenting with ZIL currently requires that you recreate the pool. With the experimental v28 code things are much easier. On 06.01.11 15:11, Damien Fleuriot wrote: > I see, so no dedicated ZIL device in the end ? > > I could make a 15gb slice for the OS running UFS (I don't wanna risk > losing the OS when manipulating ZFS, such as during upgrades), and a > 25gb+ for L2ARC, depending on the disk. > > I can't afford a *dedicated* drive for the cache though, not enough room > in the machine. > From owner-freebsd-stable@FreeBSD.ORG Thu Jan 6 15:09:09 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6899910656D6 for ; Thu, 6 Jan 2011 15:09:09 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 396988FC0C for ; Thu, 6 Jan 2011 15:09:09 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id E3D8546B58; Thu, 6 Jan 2011 10:09:08 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id DD9068A01D; Thu, 6 Jan 2011 10:09:07 -0500 (EST) From: John Baldwin To: perryh@pluto.rain.com Date: Thu, 6 Jan 2011 08:04:55 -0500 User-Agent: KMail/1.13.5 (FreeBSD/7.3-CBSD-20101102; KDE/4.4.5; amd64; ; ) References: <1036681015.111502.1294189339355.JavaMail.root@erie.cs.uoguelph.ca> <201101050757.08116.jhb@freebsd.org> <4d257864.YjgoS235V8eKvUX2%perryh@pluto.rain.com> In-Reply-To: <4d257864.YjgoS235V8eKvUX2%perryh@pluto.rain.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201101060804.56205.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Thu, 06 Jan 2011 10:09:08 -0500 (EST) X-Virus-Scanned: clamav-milter 0.96.3 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.9 required=4.2 tests=BAYES_00 autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on bigwig.baldwin.cx Cc: marek_sal@wp.pl, freebsd-stable@freebsd.org, jyavenard@gmail.com, rmacklem@uoguelph.ca, milu@dat.pl Subject: Re: NFSv4 - how to set up at FreeBSD 8.1 ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Jan 2011 15:09:09 -0000 On Thursday, January 06, 2011 3:08:04 am perryh@pluto.rain.com wrote: > John Baldwin wrote: > > > ... even NFS UDP mounts maintain their own set of "socket" state > > to manage retries and retransmits for UDP RPCs. > > Not according to what I remember of the SunOS NFS documentation, > which indicated that the driving force behind using UDP instead of > TCP was to have the server be _completely_ stateless. (Of course > locking is inherently stateful; they made it very clear that the > locking protocol was considered to be an adjunct rather than part > of the NFS protocol itself.) No extra NFS state is tied to a TCP mount aside from maintaining TCP state (i.e. congestion window for the socket, etc.). A TCP mount does not have a different amount of NFS state than a UDP mount. As Rick noted, many servers do maintain a DRPC, but that applies to both UDP and TCP mounts. > It's been quite a few years since I read that, and I didn't get > into the details, but I suppose the handle returned to a client (in > response to a mount or open request) must have contained both a > representation of the inode number and a unique identification of > the filesystem (so that, in the case where server crash recovery > included a newfs and reload from backup, the FS ID would not match > and the client would get a "stale handle" response). All of the > retry and retransmit burden had to have been managed by the client, > for both reading and writing. Yes, this is true for both UDP and TCP (if you exclude TCP's retransmit for missed packets in server replies on a TCP mount). Even with TCP a client can still retransmit requests for which it does not receive a reply in case the connection dies due to a network problem, server reboot, etc. -- John Baldwin From owner-freebsd-stable@FreeBSD.ORG Thu Jan 6 19:45:34 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CBA8A1065673 for ; Thu, 6 Jan 2011 19:45:34 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 688968FC0C for ; Thu, 6 Jan 2011 19:45:32 +0000 (UTC) Received: by fxm16 with SMTP id 16so16258076fxm.13 for ; Thu, 06 Jan 2011 11:45:32 -0800 (PST) MIME-Version: 1.0 Received: by 10.223.100.5 with SMTP id w5mr8769245fan.20.1294343132020; Thu, 06 Jan 2011 11:45:32 -0800 (PST) Received: by 10.223.13.20 with HTTP; Thu, 6 Jan 2011 11:45:31 -0800 (PST) In-Reply-To: <4D25C760.4070703@digsys.bg> References: <4D1C6F90.3080206@my.gd> <4D21E679.80002@my.gd> <84882169-0461-480F-8B4C-58E794BCC8E6@my.gd> <488AE93A-97B9-4F01-AD0A-0098E4B329C3@my.gd> <4D25BF91.7070304@my.gd> <4D25C760.4070703@digsys.bg> Date: Thu, 6 Jan 2011 20:45:31 +0100 Message-ID: From: Damien Fleuriot To: Daniel Kalchev Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-stable@freebsd.org Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Jan 2011 19:45:34 -0000 On 6 January 2011 14:45, Daniel Kalchev wrote: > For pure storage, that is a place you send/store files, you don't really > need the ZIL. You also need the L2ARC only if you read over and over again > the same dataset, which is larger than the available ARC (ZFS cache memory). > Both will not be significant for 'backup server' application, because it's > very unlikely to do lots of SYNC I/O (where separate ZIL helps), or serve > the same files back (where the L2ARC might help). > > You should also know that having large L2ARC requires that you also have > larger ARC, because there are data pointers in the ARC that point to the > L2ARC data. Someone will do good to the community to publish some reasonable > estimates of the memory needs, so that people do not end up with large but > unusable L2ARC setups. > > It seems that the upcoming v28 ZFS will help greatly with the ZIL in the > main pool.. > > You need to experiment with the L2ARC (this is safe with current v14 and v15 > pools) to see if your usage will see benefit from it's use. Experimenting > with ZIL currently requires that you recreate the pool. With the > experimental v28 code things are much easier. > I see, thanks for the pointers. The thing is, this will be a home storage (samba share, media server) box, but I'd also like to experiment a bit, and it seems like a waste to not try at least the cache, seeing I'll have a SSD at hand. If things go well, I may be able to recommend ZFS for production storage servers at work and I'd really like to know how the cache and ZIL work at that time ;) From owner-freebsd-stable@FreeBSD.ORG Thu Jan 6 19:50:38 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 182B31065674 for ; Thu, 6 Jan 2011 19:50:38 +0000 (UTC) (envelope-from grarpamp@gmail.com) Received: from mail-px0-f182.google.com (mail-px0-f182.google.com [209.85.212.182]) by mx1.freebsd.org (Postfix) with ESMTP id E6BA88FC17 for ; Thu, 6 Jan 2011 19:50:36 +0000 (UTC) Received: by pxi1 with SMTP id 1so3277088pxi.13 for ; Thu, 06 Jan 2011 11:50:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=/a5OVEhKz6xtaHctnhP4hP+nxhn6g5nZKlHgZo2d4g4=; b=AJtBfhyPipWT9qy94+t2J8AwnpCSe7KqXCWGhqREKuJzsmEn02ZDlAkZpztPkuGAB8 oDTYER5stb2fHCb9/6kqvTcMgV+/I6b4j+D5D2LzcsedsDeAepuzatD7Xvs0IgfUTNtC cX2dw7BtnMKLPApLkAvpUhjV7F3ghB0SlE0fk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=CKa9+fUaMPHmAUmTFtD3p7jG8pUnx52xm10LSGbNonXp1bSu55l/bytZ7Q0UADYm35 1Tc+LvMeQm7bSQe1FWzrJgB23gkmPkaScpUeS22FA0wja2xAFQthq8J9JAX8eRIDOetY wAjtuR6mgr7OBn46BUTaXUXno/v7fH3TdpGYs= MIME-Version: 1.0 Received: by 10.142.142.8 with SMTP id p8mr1068587wfd.2.1294341970517; Thu, 06 Jan 2011 11:26:10 -0800 (PST) Received: by 10.142.218.3 with HTTP; Thu, 6 Jan 2011 11:26:10 -0800 (PST) Date: Thu, 6 Jan 2011 14:26:10 -0500 Message-ID: From: grarpamp To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: NFS - DNS fail stops boot in mountlate X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Jan 2011 19:50:38 -0000 RELENG_8. ### setup mount -d -a -l -v -t nfs exec: mount_nfs -o ro -o tcp -o bg -o nolockd -o intr 192.168.0.10:/tmp /mnt exec: mount_nfs -o ro -o tcp -o bg -o nolockd -o intr foo:/tmp /mnt 192.168.0.10 has been unplugged, no arp entry. Host foo not found: 3(NXDOMAIN) ### result mount -v 192.168.0.10:/tmp ; echo $? [tcp] 192.168.0.10:/tmp: RPCPROG_NFS: RPC: Port mapper failure - RPC: Timed out mount_nfs: Cannot immediately mount 192.168.0.10:/tmp, backgrounding /dev/ad0s1a on / (ufs, local, read-only, fsid ) 0 [this is ok.] mount -v foo:/tmp ; echo $? mount_nfs: foo: hostname nor servname provided, or not known /dev/ad0s1a on / (ufs, local, read-only, fsid ) 1 [drops to shell, which is obviously bad behaviour.] [mount_nfs should background as in the former.] From owner-freebsd-stable@FreeBSD.ORG Thu Jan 6 21:16:48 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 18254106566C for ; Thu, 6 Jan 2011 21:16:48 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id C4C808FC08 for ; Thu, 6 Jan 2011 21:16:47 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AmsFAN6/JU2DaFvO/2dsb2JhbACDd5IZjwKwKI1WgSGDN3QEhGc8hWY X-IronPort-AV: E=Sophos;i="4.60,285,1291611600"; d="scan'208";a="104514403" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 06 Jan 2011 16:16:46 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id B93A4B3FF3; Thu, 6 Jan 2011 16:16:46 -0500 (EST) Date: Thu, 6 Jan 2011 16:16:46 -0500 (EST) From: Rick Macklem To: perryh@pluto.rain.com Message-ID: <2047622233.213674.1294348606324.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <201101060804.56205.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.200] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - IE8 (Win)/6.0.10_GA_2692) Cc: perryh@pluto.rain.com, marek sal , freebsd-stable@freebsd.org, milu@dat.pl, jyavenard@gmail.com Subject: Re: NFSv4 - how to set up at FreeBSD 8.1 ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Jan 2011 21:16:48 -0000 > > > > Not according to what I remember of the SunOS NFS documentation, > > which indicated that the driving force behind using UDP instead of > > TCP was to have the server be _completely_ stateless. (Of course > > locking is inherently stateful; they made it very clear that the > > locking protocol was considered to be an adjunct rather than part > > of the NFS protocol itself.) > When I said I recalled that they didn't do TCP because of excessive overhead, I forgot to mention that my recollection could be wrong. Also, I suspect you are correct w.r.t. the above statement. (ie. Sun's official position vs something I heard.) Anyhow, appologies if I gave the impression that I was correcting your statement. My intent was just to throw out another statement that I vaguely recalled someone an Sun stating. rick From owner-freebsd-stable@FreeBSD.ORG Fri Jan 7 00:52:00 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B12B7106566B for ; Fri, 7 Jan 2011 00:52:00 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from mail2.fluidhosting.com (mx22.fluidhosting.com [204.14.89.5]) by mx1.freebsd.org (Postfix) with ESMTP id 496038FC1D for ; Fri, 7 Jan 2011 00:52:00 +0000 (UTC) Received: (qmail 1087 invoked by uid 399); 7 Jan 2011 00:51:59 -0000 Received: from localhost (HELO doug-optiplex.ka9q.net) (dougb@dougbarton.us@127.0.0.1) by localhost with ESMTPAM; 7 Jan 2011 00:51:59 -0000 X-Originating-IP: 127.0.0.1 X-Sender: dougb@dougbarton.us Message-ID: <4D2663AE.2080805@FreeBSD.org> Date: Thu, 06 Jan 2011 16:51:58 -0800 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.13) Gecko/20101210 Thunderbird/3.1.7 MIME-Version: 1.0 To: grarpamp References: In-Reply-To: X-Enigmail-Version: 1.1.2 OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org Subject: Re: NFS - DNS fail stops boot in mountlate X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 00:52:00 -0000 It's generally better to post a description of your problem, rather than copy and pasting command line examples. What makes perfect sense to you may (or even probably does) not make sense to others. :) Doug -- Nothin' ever doesn't change, but nothin' changes much. -- OK Go Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ From owner-freebsd-stable@FreeBSD.ORG Fri Jan 7 01:25:09 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 018FC106564A for ; Fri, 7 Jan 2011 01:25:09 +0000 (UTC) (envelope-from jyavenard@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id BB6808FC18 for ; Fri, 7 Jan 2011 01:25:08 +0000 (UTC) Received: by iyb26 with SMTP id 26so16055129iyb.13 for ; Thu, 06 Jan 2011 17:25:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=5+rXy7TUapvg+9TESpyDaG840mXTZ+knvo+AW11bhWg=; b=wHSna6h5n2ntAIm48WwBVFcHoCBorpv09vCm1jaDtpH9avfjrytRqujQn4iFQu2PQI fkhyRv63PY4hcjvsaBk4x3OUo3yTn6Xv52eM4x4OBHQbmmtcR4pT+K/336Uel/AbqOtE QeGXtv7TgknQpNCCaA6TfUyxUQ52nUBDPHfJQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=CBW1kxiWUh+SDLtYRUO+0ISNYmKuk+nBCpF+YSvJSMZTzwqhYM4jPjLVqHkqYZwBGv EtrUvsYKWCpGlux8dWZFHDJlYIccNCQyP+asRwtwMcV90jF9fHPjQGRZHOO3bi17UOaM PVl7+EtjBAxlMDf1S7J0EZcnWDG3xoxINcp50= MIME-Version: 1.0 Received: by 10.42.213.201 with SMTP id gx9mr261196icb.330.1294363508065; Thu, 06 Jan 2011 17:25:08 -0800 (PST) Received: by 10.42.172.69 with HTTP; Thu, 6 Jan 2011 17:25:08 -0800 (PST) In-Reply-To: <2047622233.213674.1294348606324.JavaMail.root@erie.cs.uoguelph.ca> References: <201101060804.56205.jhb@freebsd.org> <2047622233.213674.1294348606324.JavaMail.root@erie.cs.uoguelph.ca> Date: Fri, 7 Jan 2011 12:25:08 +1100 Message-ID: From: Jean-Yves Avenard To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-stable@freebsd.org, marek sal , perryh@pluto.rain.com, milu@dat.pl Subject: Re: NFSv4 - how to set up at FreeBSD 8.1 ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 01:25:09 -0000 On 7 January 2011 08:16, Rick Macklem wrote: > When I said I recalled that they didn't do TCP because of excessive > overhead, I forgot to mention that my recollection could be wrong. > Also, I suspect you are correct w.r.t. the above statement. (ie. Sun's > official position vs something I heard.) > > Anyhow, appologies if I gave the impression that I was correcting your > statement. My intent was just to throw out another statement that I > vaguely recalled someone an Sun stating. After hitting yet another serious bug in 8.2 ; I reverted back to 8.1 Interestingly, it now complains about having V4: / in /etc/exports NFSv4 isn't available in 8.1 ? From owner-freebsd-stable@FreeBSD.ORG Fri Jan 7 01:28:12 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1AC791065672 for ; Fri, 7 Jan 2011 01:28:12 +0000 (UTC) (envelope-from jyavenard@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id D13438FC0A for ; Fri, 7 Jan 2011 01:28:11 +0000 (UTC) Received: by iyb26 with SMTP id 26so16056667iyb.13 for ; Thu, 06 Jan 2011 17:28:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=xfEZxhcQhA/8QI2RqEyumfcU42P8nNhrvbbfFN1lcoM=; b=nKC5wdH3Y4KebPziaTREzIBGAg4RnVdDvZuq0lshYEdQSy0HRddtjeyd8IzrI5zUCV SVTl1dMk9/FOERI8miN+tr8NV6PCdPa/bhTcTXXqxp3Sf3U/430NbZU1Gticbs6GURDO CJbygHjZUmnxtnhSdHM3KSwMRg0DhvuCGahQ0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=oVLqJlzWhZIPVgShErk8oXQSdgTH27Mn1AhIzmP8Y5PB8449Yg4B6DdVkeVudZAtcB Wsv/bjEUQbLvVNBtWrW10srFr4r9rx1yLDldSH4Sm1tbSbOXP3/32Nic78A/p+eyLLqa J32sDAyNJeX4vvtj65rt7YXe3Ic1KEuM+v93k= MIME-Version: 1.0 Received: by 10.42.169.9 with SMTP id z9mr289017icy.89.1294363691199; Thu, 06 Jan 2011 17:28:11 -0800 (PST) Received: by 10.42.172.69 with HTTP; Thu, 6 Jan 2011 17:28:11 -0800 (PST) In-Reply-To: <4D25C760.4070703@digsys.bg> References: <4D1C6F90.3080206@my.gd> <4D21E679.80002@my.gd> <84882169-0461-480F-8B4C-58E794BCC8E6@my.gd> <488AE93A-97B9-4F01-AD0A-0098E4B329C3@my.gd> <4D25BF91.7070304@my.gd> <4D25C760.4070703@digsys.bg> Date: Fri, 7 Jan 2011 12:28:11 +1100 Message-ID: From: Jean-Yves Avenard To: Daniel Kalchev Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-stable@freebsd.org Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 01:28:12 -0000 Hi On 7 January 2011 00:45, Daniel Kalchev wrote: > For pure storage, that is a place you send/store files, you don't really > need the ZIL. You also need the L2ARC only if you read over and over again > the same dataset, which is larger than the available ARC (ZFS cache memory). > Both will not be significant for 'backup server' application, because it's > very unlikely to do lots of SYNC I/O (where separate ZIL helps), or serve > the same files back (where the L2ARC might help). > > You should also know that having large L2ARC requires that you also have > larger ARC, because there are data pointers in the ARC that point to the > L2ARC data. Someone will do good to the community to publish some reasonable > estimates of the memory needs, so that people do not end up with large but > unusable L2ARC setups. > > It seems that the upcoming v28 ZFS will help greatly with the ZIL in the > main pool.. yes, it made a *huge* difference for me.. It went from "way too slow to comprehend what's going on" to "still slow but I can live with it" and I found no significant difference between ZIL on the main pool and on a separate SSD From owner-freebsd-stable@FreeBSD.ORG Fri Jan 7 01:29:18 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8CAC7106566C for ; Fri, 7 Jan 2011 01:29:18 +0000 (UTC) (envelope-from jyavenard@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 4CAB48FC16 for ; Fri, 7 Jan 2011 01:29:18 +0000 (UTC) Received: by iwn39 with SMTP id 39so17076511iwn.13 for ; Thu, 06 Jan 2011 17:29:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=6UOJ3tzT3Ou1WQ/HBKPFFEB0w7tPoUqrgS8Jp/7nQlU=; b=eMlMM5sRQ6sKCTVqlWtkRT6Oyi3ieAmHap1cgJHzBnjiTV3NsZFWrqZ0q2d3S5z9oQ kkim67/fXg7vt1VsfPquton8ILZ7hmi5E9hpTJo9xy5Dx6899A/70+dMY4otC+AP2Dks NFdJryKYmlXuPtvFyMMarMtozS/sguRWWJJnc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=dOZfknwJkURYT12TN+QHsec8XOzfpVSOotXVBSOAWBQBSB5jkfBbwT69Zb6tbovrMJ +3KRFS+AYMh00u952/HFoDwF0l+PfsEb1XoWHi8X8LEg4WlHdTFvbBiiMJ2KkjcClpO7 V4K5vhw0jZwRWdZHBOnLjiW6zqQj20WPTwS4k= MIME-Version: 1.0 Received: by 10.42.213.201 with SMTP id gx9mr264431icb.330.1294363757665; Thu, 06 Jan 2011 17:29:17 -0800 (PST) Received: by 10.42.172.69 with HTTP; Thu, 6 Jan 2011 17:29:17 -0800 (PST) In-Reply-To: References: <4D1C6F90.3080206@my.gd> <4D21E679.80002@my.gd> <84882169-0461-480F-8B4C-58E794BCC8E6@my.gd> <488AE93A-97B9-4F01-AD0A-0098E4B329C3@my.gd> Date: Fri, 7 Jan 2011 12:29:17 +1100 Message-ID: From: Jean-Yves Avenard To: Chris Forgeron Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-stable@freebsd.org" , Artem Belevich Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 01:29:18 -0000 On 6 January 2011 22:26, Chris Forgeron wrote: > You know, these days I'm not as happy with SSD's for ZIL. I may blog abou= t some of the speed results I've been getting over the last 6mo-1yr that I'= ve been running them with ZFS. I think people should be using hardware RAM = drives. You can get old Gigabyte i-RAM drives with 4 gig of memory for the = cost of a 60 gig SSD, and it will trounce the SSD for speed. > > I'd put your SSD to L2ARC (cache). Where do you find those though. I've looked and looked and all references I could find was that battery-powered RAM card that Sun used in their test setup, but it's not publicly available.. From owner-freebsd-stable@FreeBSD.ORG Fri Jan 7 01:43:19 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0BFD2106567A for ; Fri, 7 Jan 2011 01:43:19 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta15.westchester.pa.mail.comcast.net (qmta15.westchester.pa.mail.comcast.net [76.96.59.228]) by mx1.freebsd.org (Postfix) with ESMTP id A39AF8FC1B for ; Fri, 7 Jan 2011 01:43:18 +0000 (UTC) Received: from omta04.westchester.pa.mail.comcast.net ([76.96.62.35]) by qmta15.westchester.pa.mail.comcast.net with comcast id sRJq1f0040ldTLk5FRjJCD; Fri, 07 Jan 2011 01:43:18 +0000 Received: from koitsu.dyndns.org ([98.248.34.134]) by omta04.westchester.pa.mail.comcast.net with comcast id sRis1f00p2tehsa3QRj4X0; Fri, 07 Jan 2011 01:43:15 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 0869E9B422; Thu, 6 Jan 2011 17:42:49 -0800 (PST) Date: Thu, 6 Jan 2011 17:42:49 -0800 From: Jeremy Chadwick To: Jean-Yves Avenard Message-ID: <20110107014249.GA3719@icarus.home.lan> References: <4D1C6F90.3080206@my.gd> <4D21E679.80002@my.gd> <84882169-0461-480F-8B4C-58E794BCC8E6@my.gd> <488AE93A-97B9-4F01-AD0A-0098E4B329C3@my.gd> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-stable@freebsd.org" , Artem Belevich , Chris Forgeron Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 01:43:19 -0000 On Fri, Jan 07, 2011 at 12:29:17PM +1100, Jean-Yves Avenard wrote: > On 6 January 2011 22:26, Chris Forgeron wrote: > > You know, these days I'm not as happy with SSD's for ZIL. I may blog about some of the speed results I've been getting over the last 6mo-1yr that I've been running them with ZFS. I think people should be using hardware RAM drives. You can get old Gigabyte i-RAM drives with 4 gig of memory for the cost of a 60 gig SSD, and it will trounce the SSD for speed. > > > > I'd put your SSD to L2ARC (cache). > > Where do you find those though. > > I've looked and looked and all references I could find was that > battery-powered RAM card that Sun used in their test setup, but it's > not publicly available.. DDRdrive: http://www.ddrdrive.com/ http://www.engadget.com/2009/05/05/ddrdrives-ram-based-ssd-is-snappy-costly/ ACard ANS-9010: http://techreport.com/articles.x/16255 GC-RAMDISK (i-RAM) products: http://us.test.giga-byte.com/Products/Storage/Default.aspx Be aware these products are absurdly expensive for what they offer (the cost isn't justified), not to mention in some cases a bottleneck is imposed by use of a SATA-150 interface. I'm also not sure if all of them offer BBU capability. In some respects you might be better off just buying more RAM for your system and making md(4) memory disks that are used by L2ARC (cache). I've mentioned this in the past (specifically "back in the days" when the ARC piece of ZFS on FreeBSD was causing havok, and asked if one could work around the complexity by using L2ARC with md(4) drives instead). I tried this, but couldn't get rc.d/mdconfig2 to do what I wanted on startup WRT the aforementioned. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Fri Jan 7 02:19:07 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 792D4106564A for ; Fri, 7 Jan 2011 02:19:07 +0000 (UTC) (envelope-from grarpamp@gmail.com) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 5017F8FC08 for ; Fri, 7 Jan 2011 02:19:07 +0000 (UTC) Received: by pwi10 with SMTP id 10so2698191pwi.13 for ; Thu, 06 Jan 2011 18:19:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type; bh=29+PCrBLQOcijpQ6fwztlHml7Psz8zjbrNkeDwYY7lA=; b=YWtNcL/c7caPHA0vw925d9NW4Duj+04LvrZfQzOQikjqkS2J2An5uRQKaiABwX4wZ7 zTZ+NGrsUqvkolzlbhTnRLTsRc3ZwIkp1O0HmTjD4I8MdvS5TvGtdejqVn+E5Av2eKoQ yfRiz5XV7Ay4cjIzyZjVc3KLMWjpcy7xIRuwY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=WLFIT7bI0/VBbagB0/xCAGomU/FA2WlQ7NiF2PeKu2XBPc6KSgru1qVufkmQx3q007 MU6T7sUkn07GVrsxymJzayzJPYpM07l7K82N4nvnQqjezn2rMNHJ2Xtx2HGyGRI8bn8w 7GD8THAg49WbPMjfdheZcaaqhHncYNJmlOues= MIME-Version: 1.0 Received: by 10.142.142.8 with SMTP id p8mr1335046wfd.2.1294366746795; Thu, 06 Jan 2011 18:19:06 -0800 (PST) Received: by 10.142.218.3 with HTTP; Thu, 6 Jan 2011 18:19:06 -0800 (PST) In-Reply-To: <4D2663AE.2080805@FreeBSD.org> References: <4D2663AE.2080805@FreeBSD.org> Date: Thu, 6 Jan 2011 21:19:06 -0500 Message-ID: From: grarpamp To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: Re: NFS - DNS fail stops boot in mountlate X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 02:19:07 -0000 So what was unclear? mount_nfs emits a nonzero exit status upon failing to look up an FQDN causing mountlate to trigger a dump to shell on boot during rc processing. That's a *showstopper*. The right thing to do is to hack mount_nfs to punt to background mounting in this case with an appropriate exit status. Personally I'd distinguish mount_nfs exit codes between: 0 - mounted 1 - backgrounded, for any reason 2 - none of the above and adjust the rc's to deal with it accordingly. Words are subject to interpretation and take time. Though perhaps masked by brevity, I believe all the above elements were in the prior concise post. Thanks everybody :) From owner-freebsd-stable@FreeBSD.ORG Fri Jan 7 02:24:01 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3F8B31065673 for ; Fri, 7 Jan 2011 02:24:01 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from noop.in-addr.com (mail.in-addr.com [IPv6:2001:470:8:162::1]) by mx1.freebsd.org (Postfix) with ESMTP id 0837D8FC13 for ; Fri, 7 Jan 2011 02:24:01 +0000 (UTC) Received: from gjp by noop.in-addr.com with local (Exim 4.72 (FreeBSD)) (envelope-from ) id 1Pb1zh-000GNY-CW; Thu, 06 Jan 2011 21:23:57 -0500 Date: Thu, 6 Jan 2011 21:23:57 -0500 From: Gary Palmer To: Jeremy Chadwick Message-ID: <20110107022357.GA61893@in-addr.com> References: <4D21E679.80002@my.gd> <84882169-0461-480F-8B4C-58E794BCC8E6@my.gd> <488AE93A-97B9-4F01-AD0A-0098E4B329C3@my.gd> <20110107014249.GA3719@icarus.home.lan> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110107014249.GA3719@icarus.home.lan> X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: gpalmer@freebsd.org X-SA-Exim-Scanned: No (on noop.in-addr.com); SAEximRunCond expanded to false Cc: Chris Forgeron , "freebsd-stable@freebsd.org" , Artem Belevich , Jean-Yves Avenard Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 02:24:01 -0000 On Thu, Jan 06, 2011 at 05:42:49PM -0800, Jeremy Chadwick wrote: > On Fri, Jan 07, 2011 at 12:29:17PM +1100, Jean-Yves Avenard wrote: > > On 6 January 2011 22:26, Chris Forgeron wrote: > > > You know, these days I'm not as happy with SSD's for ZIL. I may blog about some of the speed results I've been getting over the last 6mo-1yr that I've been running them with ZFS. I think people should be using hardware RAM drives. You can get old Gigabyte i-RAM drives with 4 gig of memory for the cost of a 60 gig SSD, and it will trounce the SSD for speed. > > > > > > I'd put your SSD to L2ARC (cache). > > > > Where do you find those though. > > > > I've looked and looked and all references I could find was that > > battery-powered RAM card that Sun used in their test setup, but it's > > not publicly available.. > > DDRdrive: > http://www.ddrdrive.com/ > http://www.engadget.com/2009/05/05/ddrdrives-ram-based-ssd-is-snappy-costly/ > > ACard ANS-9010: > http://techreport.com/articles.x/16255 There is also https://www.hyperossystems.co.uk/07042003/hardware.htm which I believe is a rebadged ACard drive. They should be SATA-300, but the test results I saw were not that impressive to be honest. I think whatever FPGA they use for the SATA interface and DRAM controller is either underpowered or the gate layout needs work. Regards, Gary From owner-freebsd-stable@FreeBSD.ORG Fri Jan 7 02:28:56 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D9E0C106564A for ; Fri, 7 Jan 2011 02:28:56 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from mail2.fluidhosting.com (mx22.fluidhosting.com [204.14.89.5]) by mx1.freebsd.org (Postfix) with ESMTP id 67B458FC0A for ; Fri, 7 Jan 2011 02:28:55 +0000 (UTC) Received: (qmail 15284 invoked by uid 399); 7 Jan 2011 02:28:54 -0000 Received: from localhost (HELO doug-optiplex.ka9q.net) (dougb@dougbarton.us@127.0.0.1) by localhost with ESMTPAM; 7 Jan 2011 02:28:54 -0000 X-Originating-IP: 127.0.0.1 X-Sender: dougb@dougbarton.us Message-ID: <4D267A65.3080005@FreeBSD.org> Date: Thu, 06 Jan 2011 18:28:53 -0800 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.13) Gecko/20101210 Thunderbird/3.1.7 MIME-Version: 1.0 To: grarpamp References: <4D2663AE.2080805@FreeBSD.org> In-Reply-To: X-Enigmail-Version: 1.1.2 OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org Subject: Re: NFS - DNS fail stops boot in mountlate X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 02:28:56 -0000 On 01/06/2011 18:19, grarpamp wrote: > So what was unclear? I thought I probably understood your situation, but I wanted to be sure. Not to mention the value of the more general point. :) > mount_nfs emits a nonzero exit status upon failing to look > up an FQDN causing mountlate to trigger a dump to shell > on boot during rc processing. That's a *showstopper*. The canonical answer to this is to either mount them by IP, or to put the appropriate name in /etc/hosts. Depending on DNS for NFS mounts is not recommended. hth, Doug -- Nothin' ever doesn't change, but nothin' changes much. -- OK Go Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ From owner-freebsd-stable@FreeBSD.ORG Fri Jan 7 02:35:09 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 613B5106564A for ; Fri, 7 Jan 2011 02:35:09 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta02.emeryville.ca.mail.comcast.net (qmta02.emeryville.ca.mail.comcast.net [76.96.30.24]) by mx1.freebsd.org (Postfix) with ESMTP id 4597A8FC08 for ; Fri, 7 Jan 2011 02:35:08 +0000 (UTC) Received: from omta04.emeryville.ca.mail.comcast.net ([76.96.30.35]) by qmta02.emeryville.ca.mail.comcast.net with comcast id sSah1f0040lTkoCA2Sb8Fq; Fri, 07 Jan 2011 02:35:08 +0000 Received: from koitsu.dyndns.org ([98.248.34.134]) by omta04.emeryville.ca.mail.comcast.net with comcast id sSb01f00F2tehsa8QSb1m6; Fri, 07 Jan 2011 02:35:02 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 85C8E9B422; Thu, 6 Jan 2011 18:34:56 -0800 (PST) Date: Thu, 6 Jan 2011 18:34:56 -0800 From: Jeremy Chadwick To: grarpamp Message-ID: <20110107023456.GA4988@icarus.home.lan> References: <4D2663AE.2080805@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-stable@freebsd.org Subject: Re: NFS - DNS fail stops boot in mountlate X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 02:35:09 -0000 On Thu, Jan 06, 2011 at 09:19:06PM -0500, grarpamp wrote: > So what was unclear? > > mount_nfs emits a nonzero exit status upon failing to look > up an FQDN causing mountlate to trigger a dump to shell > on boot during rc processing. That's a *showstopper*. The > right thing to do is to hack mount_nfs to punt to background > mounting in this case with an appropriate exit status. > > Personally I'd distinguish mount_nfs exit codes between: > 0 - mounted > 1 - backgrounded, for any reason > 2 - none of the above > and adjust the rc's to deal with it accordingly. > > Words are subject to interpretation and take time. Though > perhaps masked by brevity, I believe all the above elements > were in the prior concise post. Thanks everybody :) So basically the problem is that the "bg" option in mount_nfs only applies to "network unreachable" conditions and not "DNS resolution failed" conditions. Initially I was going to refute the above request until I looked closely at the mount_nfs(8) man page which has the following clauses: For non-critical file systems, the bg and retrycnt options provide mechanisms to prevent the boot process from hanging if the server is unavailable. [...describing the "bg" option...] Useful for fstab(5), where the file system mount is not critical to multiuser operation. I read these statements to mean "if -o bg is used, the system should not hang/stall/fail during the boot process". Dumping to /bin/sh on boot as a result of a DNS lookup failure violates those statements, IMHO. I would agree that DNS resolution should be part of the bg/retry feature of "bg" in mount_nfs. How/whether this is feasible to implement is unknown to me. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Fri Jan 7 02:40:54 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1AA55106566C for ; Fri, 7 Jan 2011 02:40:54 +0000 (UTC) (envelope-from jyavenard@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id CDDE78FC12 for ; Fri, 7 Jan 2011 02:40:53 +0000 (UTC) Received: by iyb26 with SMTP id 26so16096372iyb.13 for ; Thu, 06 Jan 2011 18:40:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=td/t44ndbUwC95ntpe/FPS2EDCVZh38pOI8BUsNCwBU=; b=HMH4NdF5xZTm0L7dYW2+FX6Bb+33rD8T9tmnY/6GgA/3zulofv+F6/f34GEOSgY5f8 evgM6S8enfWwEP3Vdw6raNyBOp3IbSio8RMp+Nw3B4JHW3k5G9htLE8xgVKqgs+rJMoL Hukrg10ETzMO53rNE5YPD2Z2eeQY7IIoXeM8g= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=u5w1wpOkQPe4ZbIs9X1kXbFXIqdWNlI7aoCnajRYWCy/4GW2Az9/G0aD35/tjujG6v VUnmG4nLqckbIAardzgb0yBYAT7XrOPdpKgs2NZZL2CsqF823Q+VMSwCE5jYkYFj6pUp Q/zlY4jFyCK72rnohauIj1HMCgOnIPenMOdOA= MIME-Version: 1.0 Received: by 10.42.213.201 with SMTP id gx9mr318465icb.330.1294368052976; Thu, 06 Jan 2011 18:40:52 -0800 (PST) Received: by 10.42.172.69 with HTTP; Thu, 6 Jan 2011 18:40:52 -0800 (PST) In-Reply-To: <20110107014249.GA3719@icarus.home.lan> References: <4D1C6F90.3080206@my.gd> <4D21E679.80002@my.gd> <84882169-0461-480F-8B4C-58E794BCC8E6@my.gd> <488AE93A-97B9-4F01-AD0A-0098E4B329C3@my.gd> <20110107014249.GA3719@icarus.home.lan> Date: Fri, 7 Jan 2011 13:40:52 +1100 Message-ID: From: Jean-Yves Avenard To: Jeremy Chadwick Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-stable@freebsd.org" , Artem Belevich , Chris Forgeron Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 02:40:54 -0000 On 7 January 2011 12:42, Jeremy Chadwick wrote: > DDRdrive: > =A0http://www.ddrdrive.com/ > =A0http://www.engadget.com/2009/05/05/ddrdrives-ram-based-ssd-is-snappy-c= ostly/ > > ACard ANS-9010: > =A0http://techreport.com/articles.x/16255 > > GC-RAMDISK (i-RAM) products: > =A0http://us.test.giga-byte.com/Products/Storage/Default.aspx > > Be aware these products are absurdly expensive for what they offer (the > cost isn't justified), not to mention in some cases a bottleneck is > imposed by use of a SATA-150 interface. =A0I'm also not sure if all of > them offer BBU capability. Why not one of those SSD PCIe card that gives over 500MB/s read and write. And they aren't too expensive either... From owner-freebsd-stable@FreeBSD.ORG Fri Jan 7 03:53:55 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A7FF8106564A for ; Fri, 7 Jan 2011 03:53:55 +0000 (UTC) (envelope-from grarpamp@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id 7D2D08FC0A for ; Fri, 7 Jan 2011 03:53:55 +0000 (UTC) Received: by pzk32 with SMTP id 32so3861243pzk.13 for ; Thu, 06 Jan 2011 19:53:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type; bh=vowvNhieIMnxMU+2m/BSdhRPfMbRj42WECrrRIlZwMs=; b=ew3tcQRwgc0KtjdheabfvEKwWa/y3SDPCc1PDKX9NT4+sMzLocibPukd6hRLbu+zDP AprBLpjV0VSl3W23wB51wQRDSw7pwU7Ssk4/k7ATqujocYCL924ukjifkRMt6cctUlwE tsTBQCTLCK6xFbqkyXc82lml3/OCcweHLl9As= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=mZATniXq8rZIBIYB0/C4+CExiLfPFuOxJO88HEl34I7dY/jtfQki22PE46g/Rt5I0y /lY7EAPgw8drV1DDd6Y9tfK0NbPHTX4GchEVllDO/oPwHaURxsBHe+j9hK1gKK3SChJq DsJJIbG1/oEwPeb6Fg7mKuCHPgtpbdWKamEnk= MIME-Version: 1.0 Received: by 10.143.162.5 with SMTP id p5mr1359590wfo.230.1294372434789; Thu, 06 Jan 2011 19:53:54 -0800 (PST) Received: by 10.142.218.3 with HTTP; Thu, 6 Jan 2011 19:53:54 -0800 (PST) In-Reply-To: <20110107023456.GA4988@icarus.home.lan> References: <4D2663AE.2080805@FreeBSD.org> <20110107023456.GA4988@icarus.home.lan> Date: Thu, 6 Jan 2011 22:53:54 -0500 Message-ID: From: grarpamp To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: Re: NFS - DNS fail stops boot in mountlate X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 03:53:55 -0000 > understood Yep, I gathered that. Is cool :) > The canonical answer to this is to either mount them by IP, or > to put the appropriate name in /etc/hosts. Depending on DNS for > NFS mounts is not recommended. That is the only available answer at the moment, but perhaps not the best. And it's use case. Scalable systems require the flexibility of name resolution. Secure environments (which you may be alluding to) may already make use of secure/private DNS, hardcoding, keying, etc. The DNS server may also be the NFS server for which backgrounding could be appropriate. Forced hardcoding of IP's may not be ideal. Not to mention in split-horizon networks, etc. In this DNS down case, enhancing mount_nfs would allow the admin three choices: hosts+fqdn:, ip:, fqdn: However, not enhancing it only allows two: hosts+fqdn:, ip: FreeBSD is flexible (or should be) :) > I read these statements to mean "if -o bg is used, the system > should not hang/stall/fail during the boot process". Dumping to > /bin/sh on boot as a result of a DNS lookup failure violates those > statements, IMHO. Yep, that's what I was thinking. If the admin want's to play DNS and network games, sure, let them. But at least come up to let them do it :) I try not to argue use cases as someone will always have a need for the bizarre and it just wastes keystrokes :) Also, afaik, once mounted, the kernel uses only the resolved IP address thereafter. So that is also a 'safe', unchanged, semantic. Not sure about it's unmount, df and mount -v semantics, hopefully something sane. Haven't tried downing NFS or changing DNS lately to see. It's probably ok though. From owner-freebsd-stable@FreeBSD.ORG Fri Jan 7 04:20:04 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 01EAD106566C for ; Fri, 7 Jan 2011 04:20:04 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta09.westchester.pa.mail.comcast.net (qmta09.westchester.pa.mail.comcast.net [76.96.62.96]) by mx1.freebsd.org (Postfix) with ESMTP id 92A7B8FC14 for ; Fri, 7 Jan 2011 04:20:03 +0000 (UTC) Received: from omta16.westchester.pa.mail.comcast.net ([76.96.62.88]) by qmta09.westchester.pa.mail.comcast.net with comcast id sU7i1f0021uE5Es59UL3PB; Fri, 07 Jan 2011 04:20:03 +0000 Received: from koitsu.dyndns.org ([98.248.34.134]) by omta16.westchester.pa.mail.comcast.net with comcast id sUL21f0022tehsa3cUL2zj; Fri, 07 Jan 2011 04:20:03 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id B93059B422; Thu, 6 Jan 2011 20:20:00 -0800 (PST) Date: Thu, 6 Jan 2011 20:20:00 -0800 From: Jeremy Chadwick To: Jean-Yves Avenard Message-ID: <20110107042000.GA5300@icarus.home.lan> References: <4D21E679.80002@my.gd> <84882169-0461-480F-8B4C-58E794BCC8E6@my.gd> <488AE93A-97B9-4F01-AD0A-0098E4B329C3@my.gd> <20110107014249.GA3719@icarus.home.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-stable@freebsd.org" , Artem Belevich , Chris Forgeron Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 04:20:04 -0000 On Fri, Jan 07, 2011 at 01:40:52PM +1100, Jean-Yves Avenard wrote: > On 7 January 2011 12:42, Jeremy Chadwick wrote: > > > DDRdrive: > >  http://www.ddrdrive.com/ > >  http://www.engadget.com/2009/05/05/ddrdrives-ram-based-ssd-is-snappy-costly/ > > > > ACard ANS-9010: > >  http://techreport.com/articles.x/16255 > > > > GC-RAMDISK (i-RAM) products: > >  http://us.test.giga-byte.com/Products/Storage/Default.aspx > > > > Be aware these products are absurdly expensive for what they offer (the > > cost isn't justified), not to mention in some cases a bottleneck is > > imposed by use of a SATA-150 interface.  I'm also not sure if all of > > them offer BBU capability. > > Why not one of those SSD PCIe card that gives over 500MB/s read and write. > > And they aren't too expensive either... You need to be careful when you use the term "SSD" in this context. There are multiple types of SSDs with regards to what we're discussing; some are flash-based, some are RAM-based. Below are my opinions -- and this is getting WAY off-topic. I'm starting to think you just need to pull yourself up by the bootstraps and purchase something that suits *your* needs. You can literally spend weeks, months, years asking people "what should I buy?" or "what should I do?" or "how do I optimise this?" and never actually get anywhere. Sorry if it sounds harsh, but my advice would be to take the plunge and buy whatever suits *your* needs and meets your finances. HyperDrive 5M (DDR2-based; US$299) ==================================== 1) Product documentation claims that "the drive has built-in ECC so you can use non-ECC DDR2 DIMMs" -- this doesn't make sense to me from a technical perspective. How is this device doing ECC on a per-DIMM basis? And why can't I just buy ECC DIMMs and use those instead (they cost, from Crucial, $1 more than non-ECC)? 2) Monitoring capability -- how? Does it support SMART? If so, are the vendor-specific attributes documented in full? What if a single DIMM goes bad? How would you know which DIMM it is? Is there even an LED indicator of when there's a hard failure on a DIMM? What about checking its status remotely? 3) Use of DDR2; DDR2 right now is significantly more expensive then DDR3, and we already know DDR2 is on its way out. 4) Claims 175MB/s read, 145MB/s write; much slower than 500MB/s, so maybe you're talking about a different product? I don't know. 5) Uses 2x SATA ports; why? Probably because it uses SATA-150 ports, and thus 175MB/s would exceed that. Why not just go with SATA-300, or even SATA-600 these days? 6) Form factor requires a 5.25" bay; not effective for a 1U box. DDRdrive (DDR2-based; US$1995) ================================ 1) Absurdly expensive for a product of this nature, even more so because the price doesn't include the RAM. 2) Limited to 4GB maximum. 3) Absolutely no mention if the product supports ECC RAM or not. 4) PCIe x1 only (limited to 250MB/sec tops). 5) Not guaranteed to fit in all chassis (top DIMM exceeds height of the card itself). ACard ANS-9010 (DDR2-based) ============================= Looks like it's either identical to the HyperDrive 5, or maybe the HyperDrive is a copy of this. Either way... GC-RAMDISK ============ I'm not even going to bother with a review. I can't imagine anyone buying this thing. It's part of the "l33td00d" demographic. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Fri Jan 7 06:51:38 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D51791065672 for ; Fri, 7 Jan 2011 06:51:38 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from noop.in-addr.com (mail.in-addr.com [IPv6:2001:470:8:162::1]) by mx1.freebsd.org (Postfix) with ESMTP id 9DA5F8FC15 for ; Fri, 7 Jan 2011 06:51:38 +0000 (UTC) Received: from gjp by noop.in-addr.com with local (Exim 4.72 (FreeBSD)) (envelope-from ) id 1Pb6Af-000GtW-2y; Fri, 07 Jan 2011 01:51:33 -0500 Date: Fri, 7 Jan 2011 01:51:33 -0500 From: Gary Palmer To: Jeremy Chadwick Message-ID: <20110107065132.GB61893@in-addr.com> References: <84882169-0461-480F-8B4C-58E794BCC8E6@my.gd> <488AE93A-97B9-4F01-AD0A-0098E4B329C3@my.gd> <20110107014249.GA3719@icarus.home.lan> <20110107042000.GA5300@icarus.home.lan> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110107042000.GA5300@icarus.home.lan> X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: gpalmer@freebsd.org X-SA-Exim-Scanned: No (on noop.in-addr.com); SAEximRunCond expanded to false Cc: Chris Forgeron , "freebsd-stable@freebsd.org" , Artem Belevich , Jean-Yves Avenard Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 06:51:38 -0000 On Thu, Jan 06, 2011 at 08:20:00PM -0800, Jeremy Chadwick wrote: > HyperDrive 5M (DDR2-based; US$299) > ==================================== > 1) Product documentation claims that "the drive has built-in ECC so you > can use non-ECC DDR2 DIMMs" -- this doesn't make sense to me from a > technical perspective. How is this device doing ECC on a per-DIMM > basis? And why can't I just buy ECC DIMMs and use those instead (they > cost, from Crucial, $1 more than non-ECC)? > > 2) Monitoring capability -- how? Does it support SMART? If so, are the > vendor-specific attributes documented in full? What if a single DIMM > goes bad? How would you know which DIMM it is? Is there even an LED > indicator of when there's a hard failure on a DIMM? What about checking > its status remotely? > > 3) Use of DDR2; DDR2 right now is significantly more expensive then > DDR3, and we already know DDR2 is on its way out. > > 4) Claims 175MB/s read, 145MB/s write; much slower than 500MB/s, so > maybe you're talking about a different product? I don't know. > > 5) Uses 2x SATA ports; why? Probably because it uses SATA-150 ports, > and thus 175MB/s would exceed that. Why not just go with SATA-300, > or even SATA-600 these days? FAQ 2: Q Why does the HyperDrive5 have two SATA ports? A So that you can split one 8 DIMM slot device into two 4 DIMM slot deivces and run them both in RAID0 using a RAID controller for even faster performance. It claims SATA-300 (or SATA2 in the incorrect terminology from their website) Note, I have no relation to hyperos systems and don't use their gear. I did look at it for a while for journal/log type applications but to me the price/performance wasn't there. As it relates to the ACard, from memory the HyperDrive4 was ditched and then HyperOS came out with the HyperDrive 5 which looks remarkably similar to the ACard product. I was told by someone (or read somewhere) that HyperOS outsourced it to or OEMd it from some Asian country, which would fit if ACard was the manufacturer as they're in Taiwan. Regards, Gary From owner-freebsd-stable@FreeBSD.ORG Fri Jan 7 11:16:38 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2B73C106564A for ; Fri, 7 Jan 2011 11:16:38 +0000 (UTC) (envelope-from fullermd@over-yonder.net) Received: from thyme.infocus-llc.com (server.infocus-llc.com [206.156.254.44]) by mx1.freebsd.org (Postfix) with ESMTP id F0B778FC12 for ; Fri, 7 Jan 2011 11:16:37 +0000 (UTC) Received: from draco.over-yonder.net (c-75-64-226-141.hsd1.ms.comcast.net [75.64.226.141]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by thyme.infocus-llc.com (Postfix) with ESMTPSA id BFDCA37B507; Fri, 7 Jan 2011 05:16:36 -0600 (CST) Received: by draco.over-yonder.net (Postfix, from userid 100) id F261B61C5A; Fri, 7 Jan 2011 05:16:35 -0600 (CST) Date: Fri, 7 Jan 2011 05:16:35 -0600 From: "Matthew D. Fuller" To: Daniel Kalchev Message-ID: <20110107111635.GL28453@over-yonder.net> References: <4D21E679.80002@my.gd> <84882169-0461-480F-8B4C-58E794BCC8E6@my.gd> <488AE93A-97B9-4F01-AD0A-0098E4B329C3@my.gd> <4D25BF91.7070304@my.gd> <4D25C760.4070703@digsys.bg> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4D25C760.4070703@digsys.bg> X-Editor: vi X-OS: FreeBSD User-Agent: Mutt/1.5.21-fullermd.4 (2010-09-15) X-Virus-Scanned: clamav-milter 0.96.5 at thyme.infocus-llc.com X-Virus-Status: Clean Cc: freebsd-stable@freebsd.org Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 11:16:38 -0000 On Thu, Jan 06, 2011 at 03:45:04PM +0200 I heard the voice of Daniel Kalchev, and lo! it spake thus: > > You should also know that having large L2ARC requires that you also > have larger ARC, because there are data pointers in the ARC that > point to the L2ARC data. Someone will do good to the community to > publish some reasonable estimates of the memory needs, so that > people do not end up with large but unusable L2ARC setups. Estimates I've read in the past are that L2ARC consumes ARC space at around 1-2%. -- Matthew Fuller (MF4839) | fullermd@over-yonder.net Systems/Network Administrator | http://www.over-yonder.net/~fullermd/ On the Internet, nobody can hear you scream. From owner-freebsd-stable@FreeBSD.ORG Fri Jan 7 12:06:18 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6B95F106564A for ; Fri, 7 Jan 2011 12:06:18 +0000 (UTC) (envelope-from ronald-freebsd8@klop.yi.org) Received: from smtp-out2.tiscali.nl (smtp-out2.tiscali.nl [195.241.79.177]) by mx1.freebsd.org (Postfix) with ESMTP id 024DB8FC08 for ; Fri, 7 Jan 2011 12:06:17 +0000 (UTC) Received: from [212.123.145.58] (helo=sjakie.klop.ws) by smtp-out2.tiscali.nl with esmtp (Exim) (envelope-from ) id 1PbB5E-0007GS-S7 for freebsd-stable@freebsd.org; Fri, 07 Jan 2011 13:06:16 +0100 Received: from 212-123-145-58.ip.telfort.nl (localhost [127.0.0.1]) by sjakie.klop.ws (Postfix) with ESMTP id 999DC204F for ; Fri, 7 Jan 2011 13:06:11 +0100 (CET) Content-Type: text/plain; charset=us-ascii; format=flowed; delsp=yes References: <20110103163024.3C3FFCB76@ronald.office.base.nl> Date: Fri, 07 Jan 2011 13:06:11 +0100 To: "freebsd-stable@freebsd.org" MIME-Version: 1.0 From: "Ronald Klop" Message-ID: In-Reply-To: <20110103163024.3C3FFCB76@ronald.office.base.nl> User-Agent: Opera Mail/11.00 (FreeBSD) Content-Transfer-Encoding: quoted-printable Subject: Hang in VOP_LOCK1_APV on 8-STABLE with NFS. X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 12:06:18 -0000 Hi, OpenOffice hangs on NFS when I try to save a file or even when I try to =20 open the save dialog in this case. $ 17:25:35 ronald@ronald [~] procstat -kk 85575 PID TID COMM TDNAME KSTACK 85575 100322 soffice.bin initial thread mi_switch+0x176 =20 sleepq_wait+0x3b __lockmgr_args+0x655 vop_stdlock+0x39 VOP_LOCK1_APV+0x46= =20 _vn_lock+0x44 vget+0x67 vfs_hash_get+0xeb nfs_nget+0xa8 nfs_lookup+0x65e = =20 VOP_LOOKUP_APV+0x40 lookup+0x48a namei+0x518 kern_statat_vnhook+0x82 =20 kern_statat+0x15 lstat+0x22 syscallenter+0x186 syscall+0x40 85575 100502 soffice.bin - mi_switch+0x176 sleepq_catch_signals+0x309 sleepq_timedwait_sig+0x12 _sleep+0x1a0 do_cv_wait+0x639 __umtx_op_cv_wait+0x51 syscallenter+0x186 syscall+0x40 Xfast_syscall+0xe2 85575 100576 soffice.bin - mi_switch+0x176 sleepq_catch_signals+0x309 sleepq_timedwait_sig+0x12 _sleep+0x1a0 do_cv_wait+0x639 __umtx_op_cv_wait+0x51 syscallenter+0x186 syscall+0x40 Xfast_syscall+0xe2 85575 100577 soffice.bin - mi_switch+0x176 sleepq_catch_signals+0x309 sleepq_wait_sig+0xc _sleep+0x25d kern_accept+0x19c accept+0xfe syscallenter+0x186 syscall+0x40 Xfast_syscall+0xe2 85575 100578 soffice.bin - mi_switch+0x176 sleepq_catch_signals+0x309 sleepq_wait_sig+0xc _cv_wait_sig+0x10e seltdwait+0xed poll+0x457 syscallenter+0x186 syscall+0x40 Xfast_syscall+0xe2 85575 100579 soffice.bin - mi_switch+0x176 sleepq_catch_signals+0x309 sleepq_timedwait_sig+0x12 _cv_timedwait_sig+0x11d seltdwait+0x79 poll+0x457 syscallenter+0x186 syscall+0x40 Xfast_syscall+0xe2 $ 17:25:35 ronald@ronald [~] uname -a FreeBSD ronald.office.base.nl 8.2-PRERELEASE FreeBSD 8.2-PRERELEASE #6: Mon Dec 27 23:49:30 CET 2010 root@ronald.office.base.nl:/usr/obj/usr/src/sys/GENERIC amd64 It is not possible to exit or kill soffice.bin. I had a slighty different= =20 procstat stack before, but that was fixed a couple of days ago. Any thoughts? Enabling local locks in NFS doesn't fix it. The nfs server is an up-to-date Linux Debian 5 with kernel 2.6.26. If more info is needed. I can easily reproduce this. Ronald. From owner-freebsd-stable@FreeBSD.ORG Fri Jan 7 12:33:42 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 608BD106564A for ; Fri, 7 Jan 2011 12:33:42 +0000 (UTC) (envelope-from markiyan.kushnir@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id DA2A18FC17 for ; Fri, 7 Jan 2011 12:33:41 +0000 (UTC) Received: by iyb26 with SMTP id 26so16422756iyb.13 for ; Fri, 07 Jan 2011 04:33:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=T5HGj/dKkLyu5dWWh8qty1d0dvGQdUcvA2LUmQ0lK8g=; b=vAiY1y8zPmACe2LfLgive5oXHI97+UsdwbBUtv4nGvJej373ueNYQ5iwi2vYAKNJkL Re90QOIf35cANzB+rKuBcK648j8eVGSRi6Ty2hfFMdi01/dXIPfAhS093YcFEGPjc+4S o7MIC2UK18IC8XgMw4j1TDw87zE/+sS8a+tv8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=AlwkNH9UuUDw8kV7jMIHjovgRRF376inOqGKjWlcMG7LOQk3Vn4QZcvfUAN6fdrFok M0+ONIhH/w4V4yDhvaxFIQaUy62ZxXx66B3XccEsy9zFAA9lTT0RRcx5P6p5gHGj+sYr 6IwIE2UqPtkCX7iKzQe7XfHPaYEgxD7vX4ois= MIME-Version: 1.0 Received: by 10.231.207.73 with SMTP id fx9mr14702228ibb.137.1294402213832; Fri, 07 Jan 2011 04:10:13 -0800 (PST) Received: by 10.231.147.131 with HTTP; Fri, 7 Jan 2011 04:10:13 -0800 (PST) In-Reply-To: <20110107014249.GA3719@icarus.home.lan> References: <4D1C6F90.3080206@my.gd> <4D21E679.80002@my.gd> <84882169-0461-480F-8B4C-58E794BCC8E6@my.gd> <488AE93A-97B9-4F01-AD0A-0098E4B329C3@my.gd> <20110107014249.GA3719@icarus.home.lan> Date: Fri, 7 Jan 2011 14:10:13 +0200 Message-ID: From: Markiyan Kushnir To: Jeremy Chadwick Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Chris Forgeron , "freebsd-stable@freebsd.org" , Artem Belevich , Jean-Yves Avenard Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 12:33:42 -0000 2011/1/7 Jeremy Chadwick : > On Fri, Jan 07, 2011 at 12:29:17PM +1100, Jean-Yves Avenard wrote: >> On 6 January 2011 22:26, Chris Forgeron wrote: >> > You know, these days I'm not as happy with SSD's for ZIL. I may blog a= bout some of the speed results I've been getting over the last 6mo-1yr that= I've been running them with ZFS. I think people should be using hardware R= AM drives. You can get old Gigabyte i-RAM drives with 4 gig of memory for t= he cost of a 60 gig SSD, and it will trounce the SSD for speed. >> > >> > I'd put your SSD to L2ARC (cache). >> >> Where do you find those though. >> >> I've looked and looked and all references I could find was that >> battery-powered RAM card that Sun used in their test setup, but it's >> not publicly available.. > > DDRdrive: > =A0http://www.ddrdrive.com/ > =A0http://www.engadget.com/2009/05/05/ddrdrives-ram-based-ssd-is-snappy-c= ostly/ > > ACard ANS-9010: > =A0http://techreport.com/articles.x/16255 > > GC-RAMDISK (i-RAM) products: > =A0http://us.test.giga-byte.com/Products/Storage/Default.aspx > > Be aware these products are absurdly expensive for what they offer (the > cost isn't justified), not to mention in some cases a bottleneck is > imposed by use of a SATA-150 interface. =A0I'm also not sure if all of > them offer BBU capability. > > In some respects you might be better off just buying more RAM for your > system and making md(4) memory disks that are used by L2ARC (cache). > I've mentioned this in the past (specifically "back in the days" when > the ARC piece of ZFS on FreeBSD was causing havok, and asked if one > could work around the complexity by using L2ARC with md(4) drives > instead). > Once you have got extra RAM, why not just reserve it directly to ARC (via vm.kmem_size[_max] and vfs.zfs.arc_max)? Markiyan. > I tried this, but couldn't get rc.d/mdconfig2 to do what I wanted on > startup WRT the aforementioned. > > -- > | Jeremy Chadwick =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0 =A0 =A0 jdc@parodius.com | > | Parodius Networking =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 http://= www.parodius.com/ | > | UNIX Systems Administrator =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0Mountain = View, CA, USA | > | Making life hard for others since 1977. =A0 =A0 =A0 =A0 =A0 =A0 =A0 PGP= 4BD6C0CB | > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > From owner-freebsd-stable@FreeBSD.ORG Fri Jan 7 12:59:34 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2B8341065674 for ; Fri, 7 Jan 2011 12:59:34 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-ww0-f42.google.com (mail-ww0-f42.google.com [74.125.82.42]) by mx1.freebsd.org (Postfix) with ESMTP id B61DB8FC0C for ; Fri, 7 Jan 2011 12:59:33 +0000 (UTC) Received: by wwi17 with SMTP id 17so427138wwi.1 for ; Fri, 07 Jan 2011 04:59:32 -0800 (PST) Received: by 10.227.155.15 with SMTP id q15mr15488272wbw.64.1294405172618; Fri, 07 Jan 2011 04:59:32 -0800 (PST) Received: from dfleuriot.local ([83.167.62.196]) by mx.google.com with ESMTPS id 11sm17643850wbj.19.2011.01.07.04.59.31 (version=SSLv3 cipher=RC4-MD5); Fri, 07 Jan 2011 04:59:31 -0800 (PST) Message-ID: <4D270E32.30801@my.gd> Date: Fri, 07 Jan 2011 13:59:30 +0100 From: Damien Fleuriot User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-stable@freebsd.org References: <4D1C6F90.3080206@my.gd> <4D21E679.80002@my.gd> <84882169-0461-480F-8B4C-58E794BCC8E6@my.gd> <488AE93A-97B9-4F01-AD0A-0098E4B329C3@my.gd> <20110107014249.GA3719@icarus.home.lan> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 12:59:34 -0000 On 1/7/11 1:10 PM, Markiyan Kushnir wrote: > 2011/1/7 Jeremy Chadwick : >> On Fri, Jan 07, 2011 at 12:29:17PM +1100, Jean-Yves Avenard wrote: >>> On 6 January 2011 22:26, Chris Forgeron wrote: >>>> You know, these days I'm not as happy with SSD's for ZIL. I may blog about some of the speed results I've been getting over the last 6mo-1yr that I've been running them with ZFS. I think people should be using hardware RAM drives. You can get old Gigabyte i-RAM drives with 4 gig of memory for the cost of a 60 gig SSD, and it will trounce the SSD for speed. >>>> >>>> I'd put your SSD to L2ARC (cache). >>> >>> Where do you find those though. >>> >>> I've looked and looked and all references I could find was that >>> battery-powered RAM card that Sun used in their test setup, but it's >>> not publicly available.. >> >> DDRdrive: >> http://www.ddrdrive.com/ >> http://www.engadget.com/2009/05/05/ddrdrives-ram-based-ssd-is-snappy-costly/ >> >> ACard ANS-9010: >> http://techreport.com/articles.x/16255 >> >> GC-RAMDISK (i-RAM) products: >> http://us.test.giga-byte.com/Products/Storage/Default.aspx >> >> Be aware these products are absurdly expensive for what they offer (the >> cost isn't justified), not to mention in some cases a bottleneck is >> imposed by use of a SATA-150 interface. I'm also not sure if all of >> them offer BBU capability. >> >> In some respects you might be better off just buying more RAM for your >> system and making md(4) memory disks that are used by L2ARC (cache). >> I've mentioned this in the past (specifically "back in the days" when >> the ARC piece of ZFS on FreeBSD was causing havok, and asked if one >> could work around the complexity by using L2ARC with md(4) drives >> instead). >> > > Once you have got extra RAM, why not just reserve it directly to ARC > (via vm.kmem_size[_max] and vfs.zfs.arc_max)? > > Markiyan. > I haven't calculated yet but perhaps SSDs are cheaper by the GB than raw RAM. Not to mention DIMM slots are usually scarce, disk ones aren't. From owner-freebsd-stable@FreeBSD.ORG Fri Jan 7 15:03:35 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ACB54106566C for ; Fri, 7 Jan 2011 15:03:35 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 7FD948FC14 for ; Fri, 7 Jan 2011 15:03:35 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 3810646B09; Fri, 7 Jan 2011 10:03:35 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 4BF088A01D; Fri, 7 Jan 2011 10:03:34 -0500 (EST) From: John Baldwin To: freebsd-stable@freebsd.org Date: Fri, 7 Jan 2011 09:56:31 -0500 User-Agent: KMail/1.13.5 (FreeBSD/7.3-CBSD-20101102; KDE/4.4.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201101070956.31374.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Fri, 07 Jan 2011 10:03:34 -0500 (EST) X-Virus-Scanned: clamav-milter 0.96.3 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.9 required=4.2 tests=BAYES_00 autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on bigwig.baldwin.cx Cc: grarpamp Subject: Re: NFS - DNS fail stops boot in mountlate X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 15:03:35 -0000 On Thursday, January 06, 2011 2:26:10 pm grarpamp wrote: > RELENG_8. > > ### setup > mount -d -a -l -v -t nfs > exec: mount_nfs -o ro -o tcp -o bg -o nolockd -o intr 192.168.0.10:/tmp /mnt > exec: mount_nfs -o ro -o tcp -o bg -o nolockd -o intr foo:/tmp /mnt > > 192.168.0.10 has been unplugged, no arp entry. > Host foo not found: 3(NXDOMAIN) > > ### result > mount -v 192.168.0.10:/tmp ; echo $? > [tcp] 192.168.0.10:/tmp: RPCPROG_NFS: RPC: Port mapper failure - RPC: Timed out > mount_nfs: Cannot immediately mount 192.168.0.10:/tmp, backgrounding > /dev/ad0s1a on / (ufs, local, read-only, fsid ) > 0 > > [this is ok.] I've seen a regression in 8 at work where NFS mounts seem to fail on DNS on every boot (we have a small number of mounts, < 10) whereas 7 worked fine on every boot. I haven't tracked it down yet, but 8 is certainly more fragile than 7 for mounting NFS on boot. -- John Baldwin From owner-freebsd-stable@FreeBSD.ORG Fri Jan 7 15:06:01 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2FA901065674 for ; Fri, 7 Jan 2011 15:06:01 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id DBFFF8FC18 for ; Fri, 7 Jan 2011 15:06:00 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApwEAKe6Jk2DaFvO/2dsb2JhbACDd6EjriyNZYEhgzd0BIRnhiI X-IronPort-AV: E=Sophos;i="4.60,289,1291611600"; d="scan'208";a="104584623" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 07 Jan 2011 10:05:59 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id DD38DB4192; Fri, 7 Jan 2011 10:05:59 -0500 (EST) Date: Fri, 7 Jan 2011 10:05:59 -0500 (EST) From: Rick Macklem To: Jean-Yves Avenard Message-ID: <1467099208.237022.1294412759820.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - IE8 (Win)/6.0.10_GA_2692) Cc: perryh@pluto.rain.com, marek sal , freebsd-stable@freebsd.org, milu@dat.pl Subject: Re: NFSv4 - how to set up at FreeBSD 8.1 ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 15:06:01 -0000 > On 7 January 2011 08:16, Rick Macklem wrote: > > > When I said I recalled that they didn't do TCP because of excessive > > overhead, I forgot to mention that my recollection could be wrong. > > Also, I suspect you are correct w.r.t. the above statement. (ie. > > Sun's > > official position vs something I heard.) > > > > Anyhow, appologies if I gave the impression that I was correcting > > your > > statement. My intent was just to throw out another statement that I > > vaguely recalled someone an Sun stating. > > After hitting yet another serious bug in 8.2 ; I reverted back to 8.1 > > Interestingly, it now complains about having V4: / in /etc/exports > At one time the V4: line was required to be at the end of the /etc/exports file. (You could consider that a bug left over from the OpenBSD port, where it was a separate section of /etc/exports.) I removed that restriction from mountd.c at some point, but maybe after 8.1. So, try just moving the "V4:" line to the end of /etc/exports. > NFSv4 isn't available in 8.1 ? > It should be there, rick From owner-freebsd-stable@FreeBSD.ORG Fri Jan 7 15:24:21 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E2729106566C for ; Fri, 7 Jan 2011 15:24:21 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 9B44B8FC13 for ; Fri, 7 Jan 2011 15:24:21 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApwEAFa/Jk2DaFvO/2dsb2JhbACDd6EkriCNZoEhgzd0BIRnhiI X-IronPort-AV: E=Sophos;i="4.60,289,1291611600"; d="scan'208";a="106249595" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 07 Jan 2011 10:24:20 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 9FCA7B3F39; Fri, 7 Jan 2011 10:24:20 -0500 (EST) Date: Fri, 7 Jan 2011 10:24:20 -0500 (EST) From: Rick Macklem To: Jeremy Chadwick Message-ID: <1170032248.238606.1294413860593.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <20110107023456.GA4988@icarus.home.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - IE8 (Win)/6.0.10_GA_2692) Cc: freebsd-stable@freebsd.org, grarpamp Subject: Re: NFS - DNS fail stops boot in mountlate X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 15:24:22 -0000 > On Thu, Jan 06, 2011 at 09:19:06PM -0500, grarpamp wrote: > > So what was unclear? > > > > mount_nfs emits a nonzero exit status upon failing to look > > up an FQDN causing mountlate to trigger a dump to shell > > on boot during rc processing. That's a *showstopper*. The > > right thing to do is to hack mount_nfs to punt to background > > mounting in this case with an appropriate exit status. > > > > Personally I'd distinguish mount_nfs exit codes between: > > 0 - mounted > > 1 - backgrounded, for any reason > > 2 - none of the above > > and adjust the rc's to deal with it accordingly. > > > > Words are subject to interpretation and take time. Though > > perhaps masked by brevity, I believe all the above elements > > were in the prior concise post. Thanks everybody :) > > So basically the problem is that the "bg" option in mount_nfs only > applies to "network unreachable" conditions and not "DNS resolution > failed" conditions. > > Initially I was going to refute the above request until I looked > closely > at the mount_nfs(8) man page which has the following clauses: > > For non-critical file systems, the bg and retrycnt options > provide mechanisms to prevent the boot process from hanging > if the server is unavailable. > > [...describing the "bg" option...] > > Useful for fstab(5), where the file system mount is not > critical to multiuser operation. > > I read these statements to mean "if -o bg is used, the system should > not > hang/stall/fail during the boot process". Dumping to /bin/sh on boot > as > a result of a DNS lookup failure violates those statements, IMHO. > > I would agree that DNS resolution should be part of the bg/retry > feature > of "bg" in mount_nfs. How/whether this is feasible to implement is > unknown to me. > I don't think punting to "bg" when a DNS failure occurs is a particularily good idea, mostly because it doesn't help for critical mounts. (I haven't looked to see if the change is feasible, either.) It would be nice to get DNS working more reliably early in boot and the, of course, there is what Doug stated w.r.t. use IP numbers or put entries in /etc/hosts for NFS servers. rick ps: I do think that "server unavailable" doesn't imply "server is available, but DNS can't resolve it's address". From owner-freebsd-stable@FreeBSD.ORG Fri Jan 7 15:50:51 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 14C9E106564A; Fri, 7 Jan 2011 15:50:51 +0000 (UTC) (envelope-from jhs@berklix.com) Received: from tower.berklix.org (tower.berklix.org [83.236.223.114]) by mx1.freebsd.org (Postfix) with ESMTP id 8925C8FC22; Fri, 7 Jan 2011 15:50:50 +0000 (UTC) Received: from park.js.berklix.net (p5B22FEB6.dip.t-dialin.net [91.34.254.182]) (authenticated bits=0) by tower.berklix.org (8.14.2/8.14.2) with ESMTP id p07FTDwT043207; Fri, 7 Jan 2011 15:29:14 GMT (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (fire.js.berklix.net [192.168.91.41]) by park.js.berklix.net (8.13.8/8.13.8) with ESMTP id p07FTWnI038960; Fri, 7 Jan 2011 16:29:33 +0100 (CET) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (localhost [127.0.0.1]) by fire.js.berklix.net (8.14.3/8.14.3) with ESMTP id p07FTMpD009251; Fri, 7 Jan 2011 16:29:27 +0100 (CET) (envelope-from jhs@fire.js.berklix.net) Message-Id: <201101071529.p07FTMpD009251@fire.js.berklix.net> To: John Baldwin From: "Julian H. Stacey" Organization: http://www.berklix.com BSD Unix Linux Consultancy, Munich Germany User-agent: EXMH on FreeBSD http://www.berklix.com/free/ X-URL: http://www.berklix.com In-reply-to: Your message "Fri, 07 Jan 2011 09:56:31 EST." <201101070956.31374.jhb@freebsd.org> Date: Fri, 07 Jan 2011 16:29:22 +0100 Sender: jhs@berklix.com Cc: freebsd-stable@freebsd.org, grarpamp Subject: Re: NFS - DNS fail stops boot in mountlate X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 15:50:51 -0000 > I've seen a regression in 8 at work where NFS mounts seem to fail on DNS on > every boot (we have a small number of mounts, < 10) whereas 7 worked fine on > every boot. I haven't tracked it down yet, but 8 is certainly more fragile > than 7 for mounting NFS on boot. /etc/defaults/rc.conf has nfs_server_flags="-u -t -n 4" Once I had a server with example /etc/rc.conf -n 10 & had problems when I added partions beyond eg 10 ... so I suggest check rc.conf against fstab & /etc/exports Cheers, Julian -- Julian Stacey, BSD Unix Linux C Sys Eng Consultants Munich http://berklix.com Mail plain text; Not quoted-printable, or HTML or base 64. Avoid top posting, it cripples itemised cumulative responses. From owner-freebsd-stable@FreeBSD.ORG Fri Jan 7 16:08:00 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CB78410656AA for ; Fri, 7 Jan 2011 16:08:00 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 8B24C8FC13 for ; Fri, 7 Jan 2011 16:08:00 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 2F0C846B2C; Fri, 7 Jan 2011 11:08:00 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 419D48A009; Fri, 7 Jan 2011 11:07:59 -0500 (EST) From: John Baldwin To: "Julian H. Stacey" Date: Fri, 7 Jan 2011 11:03:35 -0500 User-Agent: KMail/1.13.5 (FreeBSD/7.3-CBSD-20101102; KDE/4.4.5; amd64; ; ) References: <201101071529.p07FTMpD009251@fire.js.berklix.net> In-Reply-To: <201101071529.p07FTMpD009251@fire.js.berklix.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201101071103.35500.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Fri, 07 Jan 2011 11:07:59 -0500 (EST) X-Virus-Scanned: clamav-milter 0.96.3 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.9 required=4.2 tests=BAYES_00 autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on bigwig.baldwin.cx Cc: freebsd-stable@freebsd.org, grarpamp Subject: Re: NFS - DNS fail stops boot in mountlate X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 16:08:00 -0000 On Friday, January 07, 2011 10:29:22 am Julian H. Stacey wrote: > > I've seen a regression in 8 at work where NFS mounts seem to fail on DNS on > > every boot (we have a small number of mounts, < 10) whereas 7 worked fine on > > every boot. I haven't tracked it down yet, but 8 is certainly more fragile > > than 7 for mounting NFS on boot. > > /etc/defaults/rc.conf has nfs_server_flags="-u -t -n 4" > > Once I had a server with example /etc/rc.conf -n 10 > & had problems when I added partions beyond eg 10 ... > so I suggest check rc.conf against fstab & /etc/exports That should not matter for establishing mounts. Also, keep in mind that 7 worked fine with the same settings. In fact, I'm just booting an 8 kernel on the same 7 userland currently and it's only the 8 kernel that has the problem (a pure 8 system also has the same symptoms, so it's not a problem due to mixing a 7 world with 8 kernel). -- John Baldwin From owner-freebsd-stable@FreeBSD.ORG Fri Jan 7 16:26:09 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 880B0106566B; Fri, 7 Jan 2011 16:26:09 +0000 (UTC) (envelope-from jhs@berklix.com) Received: from tower.berklix.org (tower.berklix.org [83.236.223.114]) by mx1.freebsd.org (Postfix) with ESMTP id EA22B8FC08; Fri, 7 Jan 2011 16:26:08 +0000 (UTC) Received: from park.js.berklix.net (p5B22FEB6.dip.t-dialin.net [91.34.254.182]) (authenticated bits=0) by tower.berklix.org (8.14.2/8.14.2) with ESMTP id p07GQ5VX043630; Fri, 7 Jan 2011 16:26:06 GMT (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (fire.js.berklix.net [192.168.91.41]) by park.js.berklix.net (8.13.8/8.13.8) with ESMTP id p07GQQ8s039156; Fri, 7 Jan 2011 17:26:26 +0100 (CET) (envelope-from jhs@berklix.com) Received: from fire.js.berklix.net (localhost [127.0.0.1]) by fire.js.berklix.net (8.14.3/8.14.3) with ESMTP id p07GQF5u028520; Fri, 7 Jan 2011 17:26:20 +0100 (CET) (envelope-from jhs@fire.js.berklix.net) Message-Id: <201101071626.p07GQF5u028520@fire.js.berklix.net> To: John Baldwin From: "Julian H. Stacey" Organization: http://www.berklix.com BSD Unix Linux Consultancy, Munich Germany User-agent: EXMH on FreeBSD http://www.berklix.com/free/ X-URL: http://www.berklix.com In-reply-to: Your message "Fri, 07 Jan 2011 11:03:35 EST." <201101071103.35500.jhb@freebsd.org> Date: Fri, 07 Jan 2011 17:26:15 +0100 Sender: jhs@berklix.com Cc: freebsd-stable@freebsd.org, grarpamp Subject: Re: NFS - DNS fail stops boot in mountlate X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 16:26:09 -0000 Hi, Reference: > From: John Baldwin > Date: Fri, 7 Jan 2011 11:03:35 -0500 > Message-id: <201101071103.35500.jhb@freebsd.org> John Baldwin wrote: > On Friday, January 07, 2011 10:29:22 am Julian H. Stacey wrote: > > > I've seen a regression in 8 at work where NFS mounts seem to fail on DNS on > > > every boot (we have a small number of mounts, < 10) whereas 7 worked fine on > > > every boot. I haven't tracked it down yet, but 8 is certainly more fragile > > > than 7 for mounting NFS on boot. > > > > /etc/defaults/rc.conf has nfs_server_flags="-u -t -n 4" > > > > Once I had a server with example /etc/rc.conf -n 10 > > & had problems when I added partions beyond eg 10 ... > > so I suggest check rc.conf against fstab & /etc/exports > > That should not matter for establishing mounts. Also, keep in mind that 7 > worked fine with the same settings. In fact, I'm just booting an 8 kernel > on the same 7 userland currently and it's only the 8 kernel that has the > problem (a pure 8 system also has the same symptoms, so it's not a problem > due to mixing a 7 world with 8 kernel). > > -- > John Baldwin OK, I have no /etc/fstab direct invoked NFS mounts, Just AMD & NFS invoked (on mixed net of Releases: 8,7,6,4 I guess my DNS has longer to start (or my AMD+NFS falls back to other hosts already running DNS) Good luck tracing it. Cheers, Julian -- Julian Stacey, BSD Unix Linux C Sys Eng Consultants Munich http://berklix.com Mail plain text; Not quoted-printable, or HTML or base 64. Avoid top posting, it cripples itemised cumulative responses. From owner-freebsd-stable@FreeBSD.ORG Fri Jan 7 16:41:24 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B37AA1065672 for ; Fri, 7 Jan 2011 16:41:24 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 695D48FC16 for ; Fri, 7 Jan 2011 16:41:24 +0000 (UTC) Received: by qwj9 with SMTP id 9so17320422qwj.13 for ; Fri, 07 Jan 2011 08:41:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=TKhVfAgUVlRO/IWRSZHE3mYotIcWulHJQG1r37yHQPY=; b=dyRPLXP9aEmU8j9TaHlRKDy0dWDqu94Qez/toVc9daLB3fphN2VLXnhZSDWoILQ7Vi 9bdPbe4WEFIZnysyDgy8fEhedx5LIu861iTQDaqQXA1XKi0obR7k4t3VjNbKL0mwedQj rcX7LreWCxGNZQqgIQwGOWavGQscK+QgdjF4Y= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=IIrB4SasieltlPwtwnV+O5cZIUsKiZNp9CF8nCjZ/VFKqpnQPqrP5WQ+P1MUM9EBxl Xup0y4PO1dBw0pP1tLPh35LaVr2+biqzcV2dSQwW86Wu16y6x5rE3tVn3szrjpeU2i2K 27HBkDiyYwYIBq4dTl8+q7VFnhRRyhVEtRu0E= MIME-Version: 1.0 Received: by 10.229.183.193 with SMTP id ch1mr540885qcb.107.1294418480038; Fri, 07 Jan 2011 08:41:20 -0800 (PST) Received: by 10.229.97.145 with HTTP; Fri, 7 Jan 2011 08:41:19 -0800 (PST) In-Reply-To: <1467099208.237022.1294412759820.JavaMail.root@erie.cs.uoguelph.ca> References: <1467099208.237022.1294412759820.JavaMail.root@erie.cs.uoguelph.ca> Date: Fri, 7 Jan 2011 08:41:19 -0800 Message-ID: From: Freddie Cash To: Rick Macklem Content-Type: text/plain; charset=UTF-8 Cc: freebsd-stable@freebsd.org, marek sal , perryh@pluto.rain.com, milu@dat.pl, Jean-Yves Avenard Subject: Re: NFSv4 - how to set up at FreeBSD 8.1 ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 16:41:24 -0000 On Fri, Jan 7, 2011 at 7:05 AM, Rick Macklem wrote: >> NFSv4 isn't available in 8.1 ? >> > It should be there, rick It is. I'm running FreeBSD 8.1 i386 at home, using NFSv4 to share folders out to my media PC/laptop beast thingy. [fcash@rogue /home/fcash]$ uname -a FreeBSD rogue.ashesofthe.net 8.1-RELEASE FreeBSD 8.1-RELEASE #0 r211388: Sun Aug 22 15:18:36 PDT 2010 root@rogue.ashesofthe.net:/usr/obj/usr/src-8/sys/ROGUE i386 [fcash@rogue /home/fcash]$ cat /etc/exports /home -mapall=fcash -network 172.20.0.0/24 V4: /home/samba/shared -sec=sys -network 172.20.0.0/24 Never tried it with the V4: line anywhere but at the end of the file. -- Freddie Cash fjwcash@gmail.com From owner-freebsd-stable@FreeBSD.ORG Fri Jan 7 17:31:01 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DD5811065670 for ; Fri, 7 Jan 2011 17:31:01 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id 62ACF8FC0C for ; Fri, 7 Jan 2011 17:31:01 +0000 (UTC) Received: by bwz12 with SMTP id 12so10185057bwz.13 for ; Fri, 07 Jan 2011 09:31:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; bh=1Yi5N3hvDSCXYSQkVwz9eYR7fmiak1i4ZTI850WACJs=; b=QQwoRPKe9AUyXHX7OYlsbTDXDdORCTzEMcenySZHd4eTqUZA1KPKQDhseHGTTs3OHP vQ2aUY0PeEQ0niPj3E4AVTHKTyqgZpp/EOHheHpkgUMmEgoUBwpSe75rcZfRqrUpS6Si kvNvfoPBmpGPdzumD3zRMYCK7ZkKPO//E0uO4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=mxSSLAdlqxWTFLpIAJ8f5KqjimsyDDP+sUXjFoW1UZ+ubg8JqFxlLxHfVsHH/kwaM8 7tp23Rt7EHlsYbBB4VHM3MuPVUoig/zYzK/C36FwCpztZKh6ZUnrkApblzibkuBAdGck Cxp5Cle2mDV/qmMWdgch3Fl0Pxh6v0bfTrmyw= MIME-Version: 1.0 Received: by 10.204.122.65 with SMTP id k1mr1865293bkr.80.1294421460046; Fri, 07 Jan 2011 09:31:00 -0800 (PST) Sender: artemb@gmail.com Received: by 10.204.22.18 with HTTP; Fri, 7 Jan 2011 09:30:59 -0800 (PST) In-Reply-To: <20110107111635.GL28453@over-yonder.net> References: <4D21E679.80002@my.gd> <84882169-0461-480F-8B4C-58E794BCC8E6@my.gd> <488AE93A-97B9-4F01-AD0A-0098E4B329C3@my.gd> <4D25BF91.7070304@my.gd> <4D25C760.4070703@digsys.bg> <20110107111635.GL28453@over-yonder.net> Date: Fri, 7 Jan 2011 09:30:59 -0800 X-Google-Sender-Auth: XQVsKOSToS0snEgteMPtVsWmQdE Message-ID: From: Artem Belevich To: "Matthew D. Fuller" Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-stable@freebsd.org, Daniel Kalchev Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 17:31:01 -0000 On Fri, Jan 7, 2011 at 3:16 AM, Matthew D. Fuller wrote: > On Thu, Jan 06, 2011 at 03:45:04PM +0200 I heard the voice of > Daniel Kalchev, and lo! it spake thus: >> >> You should also know that having large L2ARC requires that you also >> have larger ARC, because there are data pointers in the ARC that >> point to the L2ARC data. Someone will do good to the community to >> publish some reasonable estimates of the memory needs, so that >> people do not end up with large but unusable L2ARC setups. > > Estimates I've read in the past are that L2ARC consumes ARC space at > around 1-2%. Each record in L2ARC takes about 250 bytes in ARC. If I understand it correctly, not all records are 128K which is default record size on ZFS. If you end up with a lot of small records (for instance, if you have a lot of small files or due to a lot of synchronous writes or if record size is set to a lower value) then you could potentially end up with much higher ARC requirements. So, 1-2% seems to be a reasonable estimate assuming that ZFS deals with ~10K-20K records most of the time. If you mostly store large files your ratio would probably be much better. One way to get specific ratio for *your* pool would be to collect record size statistics from your pool using "zdb -L -b " and then calculate L2ARC:ARC ratio based on average record size. I'm not sure, though whether L2ARC stores records in compressed or uncompressed form. --Artem From owner-freebsd-stable@FreeBSD.ORG Fri Jan 7 18:52:40 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 604D51065673; Fri, 7 Jan 2011 18:52:40 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-gx0-f182.google.com (mail-gx0-f182.google.com [209.85.161.182]) by mx1.freebsd.org (Postfix) with ESMTP id 08DAF8FC12; Fri, 7 Jan 2011 18:52:38 +0000 (UTC) Received: by gxk8 with SMTP id 8so7370725gxk.13 for ; Fri, 07 Jan 2011 10:52:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:cc:content-type; bh=/UJiTwjPGLIhcFjTQ8Q1pulLxwJHUtBpPfwdkcuEj0A=; b=kxMPSTAt+IfGJllrL/g0xt3N7xpSXZpQNEUOzDNMOq7/g+JxYybAWzCo+pdfI2luek iGiSpngD9ReLk9AJ9NmDnQWKXh7qhb0a8p3IXQW4QxC2TAvAFOrlXp42dOmCNrpjf9h2 pdEUC9PzJxGW1JiUWI3tSFWidr8YF2esg3bSY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:cc:content-type; b=q2qQyDuqgjMAJiFCHJZZ3exDs/YEc5nM3NcWKQ7APpvuAE9c8GSNO2iwFaZ1ZM6CHo pJrWsigF7obduvpIc5DPisD2TBHEEY+Nn0IT9s3IR+U1IAG5amvAEk10kLEcu26uDk04 ru8L8p343Af7qJTGEnqJJEz+C++tQvUFzEk00= MIME-Version: 1.0 Received: by 10.150.133.17 with SMTP id g17mr5472481ybd.108.1294425049484; Fri, 07 Jan 2011 10:30:49 -0800 (PST) Received: by 10.147.137.15 with HTTP; Fri, 7 Jan 2011 10:30:49 -0800 (PST) Date: Fri, 7 Jan 2011 10:30:49 -0800 Message-ID: From: Jack Vogel To: FreeBSD Net , FreeBSD stable , FreeBSD Developers Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: "Vogel, Jack" Subject: Supermicro Bladeserver X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 18:52:40 -0000 I am trying to track down a problem being experienced at icir.org using SuperMicro bladeservers, the SERDES 82575 interfaces are having connectivity or perhaps autoneg problems, resulting in link transitions and watchdog resets. The closest hardware my org at Intel has is a Fujitsu server who's blades also have this device, but testing on that has failed to repro the problem. I was wondering if anyone else out there has this hardware, if so could you let me know your experience, have you had problems or not, etc etc? Thanks much for any information! Jack From owner-freebsd-stable@FreeBSD.ORG Fri Jan 7 19:37:26 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8E9A8106564A for ; Fri, 7 Jan 2011 19:37:26 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 4F6DA8FC1C for ; Fri, 7 Jan 2011 19:37:25 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApwEAO/5Jk2DaFvO/2dsb2JhbACDd6EnrXCNQoEhgzd0BIRnhiKFKA X-IronPort-AV: E=Sophos;i="4.60,290,1291611600"; d="scan'208";a="104624879" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 07 Jan 2011 14:37:25 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 78CA5B3F52; Fri, 7 Jan 2011 14:37:25 -0500 (EST) Date: Fri, 7 Jan 2011 14:37:25 -0500 (EST) From: Rick Macklem To: Ronald Klop Message-ID: <1542786719.258389.1294429045433.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - IE8 (Win)/6.0.10_GA_2692) Cc: freebsd-stable@freebsd.org Subject: Re: Hang in VOP_LOCK1_APV on 8-STABLE with NFS. X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 19:37:26 -0000 > Hi, > > OpenOffice hangs on NFS when I try to save a file or even when I try > to > open the save dialog in this case. > > > $ 17:25:35 ronald@ronald [~] > procstat -kk 85575 > PID TID COMM TDNAME KSTACK > 85575 100322 soffice.bin initial thread mi_switch+0x176 > sleepq_wait+0x3b __lockmgr_args+0x655 vop_stdlock+0x39 > VOP_LOCK1_APV+0x46 > _vn_lock+0x44 vget+0x67 vfs_hash_get+0xeb nfs_nget+0xa8 > nfs_lookup+0x65e > VOP_LOOKUP_APV+0x40 lookup+0x48a namei+0x518 kern_statat_vnhook+0x82 > kern_statat+0x15 lstat+0x22 syscallenter+0x186 syscall+0x40 > 85575 100502 soffice.bin - mi_switch+0x176 > sleepq_catch_signals+0x309 sleepq_timedwait_sig+0x12 _sleep+0x1a0 > do_cv_wait+0x639 __umtx_op_cv_wait+0x51 syscallenter+0x186 > syscall+0x40 > Xfast_syscall+0xe2 > 85575 100576 soffice.bin - mi_switch+0x176 > sleepq_catch_signals+0x309 sleepq_timedwait_sig+0x12 _sleep+0x1a0 > do_cv_wait+0x639 __umtx_op_cv_wait+0x51 syscallenter+0x186 > syscall+0x40 > Xfast_syscall+0xe2 > 85575 100577 soffice.bin - mi_switch+0x176 > sleepq_catch_signals+0x309 sleepq_wait_sig+0xc _sleep+0x25d > kern_accept+0x19c accept+0xfe syscallenter+0x186 syscall+0x40 > Xfast_syscall+0xe2 > 85575 100578 soffice.bin - mi_switch+0x176 > sleepq_catch_signals+0x309 sleepq_wait_sig+0xc _cv_wait_sig+0x10e > seltdwait+0xed poll+0x457 syscallenter+0x186 syscall+0x40 > Xfast_syscall+0xe2 > 85575 100579 soffice.bin - mi_switch+0x176 > sleepq_catch_signals+0x309 sleepq_timedwait_sig+0x12 > _cv_timedwait_sig+0x11d seltdwait+0x79 poll+0x457 syscallenter+0x186 > syscall+0x40 Xfast_syscall+0xe2 > > $ 17:25:35 ronald@ronald [~] > uname -a > FreeBSD ronald.office.base.nl 8.2-PRERELEASE FreeBSD 8.2-PRERELEASE > #6: > Mon Dec 27 23:49:30 CET 2010 > root@ronald.office.base.nl:/usr/obj/usr/src/sys/GENERIC amd64 > I think all the above tells us is that the thread is waiting for a vnode lock. The question then becomes "what is holding a lock on that vnode and why?". > It is not possible to exit or kill soffice.bin. I had a slighty > different > procstat stack before, but that was fixed a couple of days ago. Yea, it will be in an uniterruptible sleep when waiting for a vnode lock. > Any thoughts? Enabling local locks in NFS doesn't fix it. Here's some things you could try: 1 - apply the attached patch. It fixes a known problem w.r.t. the client side of the krpc. Not likely to fix this, but I can hope:-) 2 - If #1 doesn't fix the problem: - before making it hang, start capturing packets via: # tcpdump -s 0 -w xxx host server - then make it hang, kill the above and # procstat -ka # ps axHlww and capture the output of both of these. Hopefully these 2 commands will indicate what is holding the vnode lock and maybe, why. The "xxx" file can be looked at in wireshark to see what/if any NFS traffic is happening. If you aren't comfortable looking at the above, you can email them to me and I'll take a stab at them someday. 3 - Try the experimental client to see if it behaves differently. The mount command is: # mount -t newnfs -o nfsv3, server:/path /mntpath (This might ideantify if the regular client has an infrequently executed code path that forgets to unlock the vnode, since it uses a somewhat different RPC layer. The buffer cache handling etc are almost the same, but the RPC stuff is fairly different.) > The nfs server is an up-to-date Linux Debian 5 with kernel 2.6.26. > I'm afraid I can't blame Linux (at least not until we have more info;-). > If more info is needed. I can easily reproduce this. See above #2. Good luck with it and let us know how it goes, rick From owner-freebsd-stable@FreeBSD.ORG Fri Jan 7 19:53:02 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CBFB4106564A for ; Fri, 7 Jan 2011 19:53:02 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 629238FC12 for ; Fri, 7 Jan 2011 19:53:01 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p07JqwR9011851 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 7 Jan 2011 21:52:58 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p07JqvFM035520; Fri, 7 Jan 2011 21:52:57 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p07JqvFW035519; Fri, 7 Jan 2011 21:52:57 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 7 Jan 2011 21:52:57 +0200 From: Kostik Belousov To: Rick Macklem Message-ID: <20110107195257.GF12599@deviant.kiev.zoral.com.ua> References: <1542786719.258389.1294429045433.JavaMail.root@erie.cs.uoguelph.ca> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="0NUq3YI6/rBc6tSL" Content-Disposition: inline In-Reply-To: <1542786719.258389.1294429045433.JavaMail.root@erie.cs.uoguelph.ca> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-stable@freebsd.org, Ronald Klop Subject: Re: Hang in VOP_LOCK1_APV on 8-STABLE with NFS. X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 19:53:02 -0000 --0NUq3YI6/rBc6tSL Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jan 07, 2011 at 02:37:25PM -0500, Rick Macklem wrote: > > Hi, > >=20 > > OpenOffice hangs on NFS when I try to save a file or even when I try > > to > > open the save dialog in this case. > >=20 > >=20 > > $ 17:25:35 ronald@ronald [~] > > procstat -kk 85575 > > PID TID COMM TDNAME KSTACK > > 85575 100322 soffice.bin initial thread mi_switch+0x176 > > sleepq_wait+0x3b __lockmgr_args+0x655 vop_stdlock+0x39 > > VOP_LOCK1_APV+0x46 > > _vn_lock+0x44 vget+0x67 vfs_hash_get+0xeb nfs_nget+0xa8 > > nfs_lookup+0x65e > > VOP_LOOKUP_APV+0x40 lookup+0x48a namei+0x518 kern_statat_vnhook+0x82 > > kern_statat+0x15 lstat+0x22 syscallenter+0x186 syscall+0x40 > > 85575 100502 soffice.bin - mi_switch+0x176 > > sleepq_catch_signals+0x309 sleepq_timedwait_sig+0x12 _sleep+0x1a0 > > do_cv_wait+0x639 __umtx_op_cv_wait+0x51 syscallenter+0x186 > > syscall+0x40 > > Xfast_syscall+0xe2 > > 85575 100576 soffice.bin - mi_switch+0x176 > > sleepq_catch_signals+0x309 sleepq_timedwait_sig+0x12 _sleep+0x1a0 > > do_cv_wait+0x639 __umtx_op_cv_wait+0x51 syscallenter+0x186 > > syscall+0x40 > > Xfast_syscall+0xe2 > > 85575 100577 soffice.bin - mi_switch+0x176 > > sleepq_catch_signals+0x309 sleepq_wait_sig+0xc _sleep+0x25d > > kern_accept+0x19c accept+0xfe syscallenter+0x186 syscall+0x40 > > Xfast_syscall+0xe2 > > 85575 100578 soffice.bin - mi_switch+0x176 > > sleepq_catch_signals+0x309 sleepq_wait_sig+0xc _cv_wait_sig+0x10e > > seltdwait+0xed poll+0x457 syscallenter+0x186 syscall+0x40 > > Xfast_syscall+0xe2 > > 85575 100579 soffice.bin - mi_switch+0x176 > > sleepq_catch_signals+0x309 sleepq_timedwait_sig+0x12 > > _cv_timedwait_sig+0x11d seltdwait+0x79 poll+0x457 syscallenter+0x186 > > syscall+0x40 Xfast_syscall+0xe2 > >=20 > > $ 17:25:35 ronald@ronald [~] > > uname -a > > FreeBSD ronald.office.base.nl 8.2-PRERELEASE FreeBSD 8.2-PRERELEASE > > #6: > > Mon Dec 27 23:49:30 CET 2010 > > root@ronald.office.base.nl:/usr/obj/usr/src/sys/GENERIC amd64 > >=20 > I think all the above tells us is that the thread is waiting for > a vnode lock. The question then becomes "what is holding a lock > on that vnode and why?". >=20 > > It is not possible to exit or kill soffice.bin. I had a slighty > > different > > procstat stack before, but that was fixed a couple of days ago. >=20 > Yea, it will be in an uniterruptible sleep when waiting for a vnode lock. >=20 > > Any thoughts? Enabling local locks in NFS doesn't fix it. >=20 > Here's some things you could try: > 1 - apply the attached patch. It fixes a known problem w.r.t. the > client side of the krpc. Not likely to fix this, but I can hope:-) 1a - Look around of other processes in the uninterruptible sleep state, quite possible, one of them also owns the lock the openoffice is waiting for. Also see http://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/kernel= debug-deadlocks.html Of the particular interest are the witness output and backtraces for all threads that are reported by witness as owning the vnode locks. > 2 - If #1 doesn't fix the problem: > - before making it hang, start capturing packets via: > # tcpdump -s 0 -w xxx host server > - then make it hang, kill the above and > # procstat -ka > # ps axHlww > and capture the output of both of these. Hopefully these 2 commands > will indicate what is holding the vnode lock and maybe, why. The > "xxx" file can be looked at in wireshark to see what/if any NFS > traffic is happening. > If you aren't comfortable looking at the above, you can email them > to me and I'll take a stab at them someday. > 3 - Try the experimental client to see if it behaves differently. The > mount command is: > # mount -t newnfs -o nfsv3, server:/path= /mntpath > (This might ideantify if the regular client has an infrequently execu= ted code > path that forgets to unlock the vnode, since it uses a somewhat diff= erent RPC > layer. The buffer cache handling etc are almost the same, but the RP= C stuff is > fairly different.) >=20 > > The nfs server is an up-to-date Linux Debian 5 with kernel 2.6.26. > >=20 > I'm afraid I can't blame Linux (at least not until we have more info;-). >=20 > > If more info is needed. I can easily reproduce this. >=20 > See above #2. >=20 > Good luck with it and let us know how it goes, rick > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" --0NUq3YI6/rBc6tSL Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk0nbxkACgkQC3+MBN1Mb4jv/QCg6Czgkei9N4ZLz0yG7HR8YPw6 nksAoN4FiVoabV7SHS5aHpAs7Kam28DN =8Fzg -----END PGP SIGNATURE----- --0NUq3YI6/rBc6tSL-- From owner-freebsd-stable@FreeBSD.ORG Fri Jan 7 20:02:28 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8BDF610657C3; Fri, 7 Jan 2011 20:02:28 +0000 (UTC) (envelope-from jay@experts-exchange.com) Received: from mail.experts-exchange.com (mail.experts-exchange.com [72.29.183.251]) by mx1.freebsd.org (Postfix) with ESMTP id 694358FC1D; Fri, 7 Jan 2011 20:02:28 +0000 (UTC) Received: from mail.experts-exchange.com (localhost [127.0.0.1]) by mail.experts-exchange.com (Postfix) with ESMTP id A22E26F093C; Fri, 7 Jan 2011 11:45:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d= experts-exchange.com; h=content-transfer-encoding:content-type :content-type:mime-version:user-agent:from:from:subject:subject :date:date:message-id:received:received:received; s=ee; t= 1294429514; x=1296243914; bh=NmGQmsXS3R7cmTpulBJ7wf5eZcunzdcr69k vFGSGk84=; b=SHvM0yZ+yKX2NmIy1B0xg3wqL5fkgbYcrGbLNmi4j8M7024VyZr x584vUZgCmBy16RcyLoM3uHuvW7PyJkFChZuJUN9VfwWXuMX1ptPkTZLb9/gZBx6 GymKyAEY9s8WRTIKnYn1y5qkHlXv1kea6EFfGCIryjzRp4sCtqH2UtG8= X-Virus-Scanned: amavisd-new at experts-exchange.com Received: from mail.experts-exchange.com ([127.0.0.1]) by mail.experts-exchange.com (mail.experts-exchange.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xU8F+exIbgAk; Fri, 7 Jan 2011 11:45:14 -0800 (PST) Received: from mail.experts-exchange.com (localhost [127.0.0.1]) by mail.experts-exchange.com (Postfix) with ESMTP id 6545A6F0939; Fri, 7 Jan 2011 11:45:14 -0800 (PST) Received: from 192.168.103.176 (SquirrelMail authenticated user jay) by mail.experts-exchange.com with HTTP; Fri, 7 Jan 2011 11:45:14 -0800 Message-ID: <9b22ed5e31d4e547daf27e01d6d08528.squirrel@mail.experts-exchange.com> Date: Fri, 7 Jan 2011 11:45:14 -0800 From: jay@experts-exchange.com To: freebsd-hackers@freebsd.org User-Agent: SquirrelMail/1.4.20 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Cc: freebsd-stable@freebsd.org, freebsd-ports@freebsd.org Subject: stunnel transparent proxy X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Jan 2011 20:02:28 -0000 Folks, Would it be possible to devise an ipfw 'fwd' rule to pass along a socket connection with IP_BINDANY set via stunnel that forwards it to another process? The problem I'm having is the vnc service on the other side cannot reply back to the IP address because the routing does not redirect back through stunnel. I am testing configurations using apache (port 80 and 443) for convenience. Request : ext ip -> stunnel -> vnc svc Response : vnc svc X->ext ip instead of : vnc svc -> stunnel -> ext ip With stunnel's transparent set option traffic looks like : 19:31:34.162337 IP 192.168.103.69.52671 > 127.0.0.1.80: Flags [S], seq 2050938762, win 65535, options [mss 16344,nop,wscale 3,sackOK,TS val 7437993 ecr 0], length 0 19:31:37.153079 IP 192.168.103.69.52671 > 127.0.0.1.80: Flags [S], .. 19:31:40.351804 IP 192.168.103.69.52671 > 127.0.0.1.80: Flags [S], .. 19:31:43.550543 IP 192.168.103.69.52671 > 127.0.0.1.80: Flags [S], seq 2050938762, win 65535, options [mss 16344,sackOK,eol], length 0 Without transparent, traffic flows fine, and looks like : 19:32:55.883404 IP 127.0.0.1.30326 > 127.0.0.1.80: Flags [S], seq 2147354729, win 65535, options [mss 16344,nop,wscale 3,sackOK,TS val 7446169 ecr 0], length 0 19:32:55.883575 IP 127.0.0.1.80 > 127.0.0.1.30326: Flags [S.], seq 2770470513, ack 2147354730, win 65535, options [mss 16344,nop,wscale 3,sackOK,TS val 1229815108 ecr 7446169], length 0 19:32:55.883589 IP 127.0.0.1.30326 > 127.0.0.1.80: Flags [.], ack 1, win 8960, options [nop,nop,TS val 7446169 ecr 1229815108], length 0 ... I did try and devise pf rules to redirect or rdr and nat, but neither worked. I am only vaguely familiar with ipfw, and from some of my research led me to believe it may be possible. Thanks P.S. I did post the same question earlier on freebsd-pf list as well. http://lists.freebsd.org/pipermail/freebsd-pf/2011-January/005914.html From owner-freebsd-stable@FreeBSD.ORG Sat Jan 8 03:40:24 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7219610656A9; Sat, 8 Jan 2011 03:40:24 +0000 (UTC) (envelope-from nyan@FreeBSD.org) Received: from sakura.ccs.furiru.org (sakura.ccs.furiru.org [IPv6:2001:2f0:104:8060::1]) by mx1.freebsd.org (Postfix) with ESMTP id F21168FC14; Sat, 8 Jan 2011 03:40:23 +0000 (UTC) Received: from localhost (authenticated bits=0) by sakura.ccs.furiru.org (unknown) with ESMTP id p083eIJj049352; Sat, 8 Jan 2011 12:40:21 +0900 (JST) (envelope-from nyan@FreeBSD.org) Date: Sat, 08 Jan 2011 12:40:18 +0900 (JST) Message-Id: <20110108.124018.59640143160055980.nyan@FreeBSD.org> To: jfvogel@gmail.com From: TAKAHASHI Yoshihiro In-Reply-To: References: X-Mailer: Mew version 6.3 on Emacs 22.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, freebsd-stable@freebsd.org, jack.vogel@intel.com Subject: Re: Supermicro Bladeserver X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jan 2011 03:40:24 -0000 In article Jack Vogel writes: > I am trying to track down a problem being experienced at icir.org using > SuperMicro > bladeservers, the SERDES 82575 interfaces are having connectivity or perhaps > autoneg problems, resulting in link transitions and watchdog resets. > > The closest hardware my org at Intel has is a Fujitsu server who's blades > also have > this device, but testing on that has failed to repro the problem. > > I was wondering if anyone else out there has this hardware, if so could you > let me > know your experience, have you had problems or not, etc etc? My machine has the following em(4) device and it has a autoneg problem. When I was using 8-stable kernel at 2010/11/01, it has no problem. But I update to 8-stable at 2010/12/01, the kernel is only linked up as 10M. em0@pci0:0:25:0: class=0x020000 card=0x13d510cf chip=0x104a8086 rev=0x02 hdr=0x00 vendor = 'Intel Corporation' device = '82566DM Gigabit Network Connection' class = network subclass = ethernet --- TAKAHASHI Yoshihiro From owner-freebsd-stable@FreeBSD.ORG Sat Jan 8 09:29:10 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BCBEA10656C8 for ; Sat, 8 Jan 2011 09:29:10 +0000 (UTC) (envelope-from joe@tao.org.uk) Received: from alpha.tao.org.uk (alpha.tao.org.uk [212.42.1.232]) by mx1.freebsd.org (Postfix) with ESMTP id 7E7228FC12 for ; Sat, 8 Jan 2011 09:29:10 +0000 (UTC) Received: from [90.155.77.83] (p4.dhcp.tao.org.uk [90.155.77.83]) (Authenticated sender: joemail@alpha.tao.org.uk) by alpha.tao.org.uk (Postfix) with ESMTPA id 4C3A61076DE7; Sat, 8 Jan 2011 09:13:44 +0000 (GMT) References: <4D21E679.80002@my.gd> <84882169-0461-480F-8B4C-58E794BCC8E6@my.gd> <488AE93A-97B9-4F01-AD0A-0098E4B329C3@my.gd> <4D25BF91.7070304@my.gd> <4D25C760.4070703@digsys.bg> <20110107111635.GL28453@over-yonder.net> In-Reply-To: Mime-Version: 1.0 (iPad Mail 8C148) Content-Type: text/plain; charset=us-ascii Message-Id: <2E0FDC35-B6E3-49C7-A55B-B1B08FFBF1DB@tao.org.uk> Content-Transfer-Encoding: quoted-printable X-Mailer: iPad Mail (8C148) From: Josef Karthauser Date: Sat, 8 Jan 2011 09:14:19 +0000 To: Artem Belevich Cc: "freebsd-stable@freebsd.org" , "Matthew D. Fuller" , Daniel Kalchev Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jan 2011 09:29:10 -0000 On 7 Jan 2011, at 17:30, Artem Belevich wrote: > One way to get specific ratio for *your* pool would be to collect > record size statistics from your pool using "zdb -L -b " and > then calculate L2ARC:ARC ratio based on average record size. I'm not > sure, though whether L2ARC stores records in compressed or > uncompressed form. Can someone point me to a reference describing the various zfs caches availa= ble? What's the arc and zil? Ive been running some zfs for a few years now, a= nd must have missed thus entire subject :/. Joe= From owner-freebsd-stable@FreeBSD.ORG Sat Jan 8 09:55:07 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1AF7910656CE for ; Sat, 8 Jan 2011 09:55:07 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta11.emeryville.ca.mail.comcast.net (qmta11.emeryville.ca.mail.comcast.net [76.96.27.211]) by mx1.freebsd.org (Postfix) with ESMTP id D06948FC18 for ; Sat, 8 Jan 2011 09:55:06 +0000 (UTC) Received: from omta15.emeryville.ca.mail.comcast.net ([76.96.30.71]) by qmta11.emeryville.ca.mail.comcast.net with comcast id sxqL1f0061Y3wxoABxv6Gi; Sat, 08 Jan 2011 09:55:06 +0000 Received: from koitsu.dyndns.org ([98.248.34.134]) by omta15.emeryville.ca.mail.comcast.net with comcast id sxq51f0032tehsa8bxq5dD; Sat, 08 Jan 2011 09:50:06 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 38C859B427; Sat, 8 Jan 2011 01:50:05 -0800 (PST) Date: Sat, 8 Jan 2011 01:50:05 -0800 From: Jeremy Chadwick To: Josef Karthauser Message-ID: <20110108095005.GA4044@icarus.home.lan> References: <488AE93A-97B9-4F01-AD0A-0098E4B329C3@my.gd> <4D25BF91.7070304@my.gd> <4D25C760.4070703@digsys.bg> <20110107111635.GL28453@over-yonder.net> <2E0FDC35-B6E3-49C7-A55B-B1B08FFBF1DB@tao.org.uk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2E0FDC35-B6E3-49C7-A55B-B1B08FFBF1DB@tao.org.uk> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-stable@freebsd.org" , Artem Belevich , "Matthew D. Fuller" , Daniel Kalchev Subject: Re: ZFS - moving from a zraid1 to zraid2 pool with 1.5tb disks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jan 2011 09:55:07 -0000 On Sat, Jan 08, 2011 at 09:14:19AM +0000, Josef Karthauser wrote: > On 7 Jan 2011, at 17:30, Artem Belevich wrote: > > One way to get specific ratio for *your* pool would be to collect > > record size statistics from your pool using "zdb -L -b " and > > then calculate L2ARC:ARC ratio based on average record size. I'm not > > sure, though whether L2ARC stores records in compressed or > > uncompressed form. > > Can someone point me to a reference describing the various zfs caches available? What's the arc and zil? Ive been running some zfs for a few years now, and must have missed thus entire subject :/. ARC: http://en.wikipedia.org/wiki/ZFS#Cache_management L2ARC: http://en.wikipedia.org/wiki/ZFS#Storage_pools L2ARC: http://blogs.sun.com/brendan/entry/test Both: http://www.c0t0d0s0.org/archives/5329-Some-insight-into-the-read-cache-of-ZFS-or-The-ARC.html Both: http://nilesh-joshi.blogspot.com/2010/07/zfs-revisited.html ZIL: http://blogs.sun.com/perrin/entry/the_lumberjack ZIL: http://blogs.sun.com/realneel/entry/the_zfs_intent_log Enjoy. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Sat Jan 8 12:59:42 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3C9AA10656D7 for ; Sat, 8 Jan 2011 12:59:42 +0000 (UTC) (envelope-from ben.pyttipanna@spooty.net) Received: from localhost.surfimpress.com (server217-174-248-30.live-servers.net [217.174.248.30]) by mx1.freebsd.org (Postfix) with ESMTP id F1E668FC0C for ; Sat, 8 Jan 2011 12:59:41 +0000 (UTC) Received: from pyttipanna.hogsedge8.net (cpc1-brig12-0-0-cust381.3-3.cable.virginmedia.com [82.24.9.126]) by localhost.surfimpress.com (Postfix) with ESMTP id E17EC5DCFF; Sat, 8 Jan 2011 12:59:34 +0000 (GMT) Message-ID: <4D285FEE.7010809@spooty.net> Date: Sat, 08 Jan 2011 13:00:30 +0000 From: ben paley User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.13) Gecko/20110103 Thunderbird/3.1.7 MIME-Version: 1.0 To: Chris Brennan References: <4D2339CA.1010309@spooty.net> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org Subject: Re: 8.2-PRERELEASE and Flash X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jan 2011 12:59:42 -0000 On 01/04/11 15:54, Chris Brennan wrote: > On Tue, Jan 4, 2011 at 10:16 AM, ben paley > wrote: > I've followed the steps at > http://www.freebsd.org/doc/handbook/desktop-browsers.html without > any errors, except that neither Firefox nor any other browser will > play flash movies. about:plugins doesn't show a flash plugin. > > Did you try symlinking nsplugin.so to $HOME/.mozilla/plugins ? this is > usually what I end up doing to make it work. I forget if 64bit flash was > fixed or not, if it wasn't you may need nswrapper or the like to run the > 32bit plugin binaries. Thanks Chris, and also to George Liaskos and Alex Goncharov for your help. The symlink doesn't work, but oddly it's working perfectly in Konqueror now. All I had to do was scan for plugins. It's not ideal because I'd rather do without KDE entirely (not because I don't like it, just because this machine isn't whizzy enough for it) but takes the urgency out of it a bit. Thanks again. Ben From owner-freebsd-stable@FreeBSD.ORG Sat Jan 8 16:32:07 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DDE1610656F9 for ; Sat, 8 Jan 2011 16:32:06 +0000 (UTC) (envelope-from jyavenard@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id AA3708FC19 for ; Sat, 8 Jan 2011 16:32:06 +0000 (UTC) Received: by iyb26 with SMTP id 26so17326355iyb.13 for ; Sat, 08 Jan 2011 08:32:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=IZF5t6c2tBbRJxVTsy4fd+EgW9fGClTsLrqeffvps/A=; b=Ju/FaDqZDY4Sri3Ey4urQIrLqYl6VdmVPbh1AIX3c5/aVayhAc1R16IblG+T0wGe/R H6V3HuUppYghRHm6fisOSaRV9308Aq9QwtpM1CDzKj7ddfwQfIU4TLozmv0ol2nRSbjf upj2/5IA4DrLVFsDW97yh78aoU02lUwoVonB0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=wPouhNVvY/1zU8WRsnS0LQlYA/KMcb1lV0wFduheIxfJJ5aRNTt1y1Eyw5dEVIv+uB FIi6PQeqSjFzJoJUcatVXNlJMGJz5BkNprO2BQX46YnlJBrfuq2jUezXEanyp45G+eTk B9+1ZTyXUJp7QAl9iYUhUWX3i/Z7xX2Ijb8So= MIME-Version: 1.0 Received: by 10.42.169.9 with SMTP id z9mr2188076icy.89.1294504325945; Sat, 08 Jan 2011 08:32:05 -0800 (PST) Received: by 10.42.172.69 with HTTP; Sat, 8 Jan 2011 08:32:05 -0800 (PST) Date: Sun, 9 Jan 2011 03:32:05 +1100 Message-ID: From: Jean-Yves Avenard To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: faulted zpool , do not resilver X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jan 2011 16:32:07 -0000 Hi. I have a raidz2 pool on which I'm trying to replaced two of the drives. it is now showing: [root@server4 ~]# zpool import pool: pool id: 890764434375195435 state: DEGRADED action: The pool can be imported despite missing or damaged devices. The fault tolerance of the pool may be compromised if imported. config: pool DEGRADED raidz2 DEGRADED ada5 ONLINE ada6 ONLINE ada2 ONLINE ada1 ONLINE replacing DEGRADED 1880266799568700877 UNAVAIL cannot open ada4 ONLINE replacing DEGRADED 1406029845814225421 UNAVAIL cannot open ada3 ONLINE Previously, it was resilvering, for 36 hours later it was still not completed and it was showing: Problems with ZFS: pool: pool state: DEGRADED status: One or more devices has experienced an error resulting in data corruption. Applications may be affected. action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: http://www.sun.com/msg/ZFS-8000-8A scrub: resilver completed after 307445734561825853h19m with 7 errors on Sat Jan 8 02:19:18 2011 config: NAME STATE READ WRITE CKSUM pool DEGRADED 0 0 7 raidz2 DEGRADED 0 0 42 ada5 ONLINE 0 0 0 2.74G resilvered ada6 ONLINE 0 0 0 2.60G resilvered ada2 ONLINE 0 0 0 2.74G resilvered ada1 ONLINE 0 0 0 2.60G resilvered replacing DEGRADED 0 0 0 1880266799568700877 UNAVAIL 0 2.57M 0 was /dev/da0 ada4 ONLINE 0 0 0 373G resilvered replacing DEGRADED 0 0 0 1406029845814225421 UNAVAIL 0 2.54M 0 was /dev/da0 ada3 ONLINE 0 0 0 372G resilvered errors: 2 data errors, use '-v' for a list As I obviously didn't want to wait another 307445734561825853h19m (which I believe would be close to the end of the universe). I did a zpool export pool. and did a zpool import once again. Now I can see all the files, everything seems fine.. But no resilvering is occurring and I have waited over an hour now. Usually resilvering would happen after only a few seconds. What should I do? Thank you JY From owner-freebsd-stable@FreeBSD.ORG Sat Jan 8 17:51:09 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 71D791065721 for ; Sat, 8 Jan 2011 17:51:09 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe08.c2i.net [212.247.154.226]) by mx1.freebsd.org (Postfix) with ESMTP id 076A58FC12 for ; Sat, 8 Jan 2011 17:51:08 +0000 (UTC) X-Cloudmark-Score: 0.000000 [] X-Cloudmark-Analysis: v=1.1 cv=Samd3CrC35PkFKGRAwjtIWdtalA6bcxM9GrYwcNK+gA= c=1 sm=1 a=8SApFS9CFHoA:10 a=8nJEP1OIZ-IA:10 a=CL8lFSKtTFcA:10 a=i9M/sDlu2rpZ9XS819oYzg==:17 a=RU9IOlKYHoE_PFn-fakA:9 a=166cAAgmHP78R2t5z18A:7 a=ZbgZjwGghQg_5805nOodwNGAj6sA:4 a=wPNLvfGTeEIA:10 a=i9M/sDlu2rpZ9XS819oYzg==:117 Received: from [188.126.198.129] (account mc467741@c2i.net HELO laptop002.hselasky.homeunix.org) by mailfe08.swip.net (CommuniGate Pro SMTP 5.2.19) with ESMTPA id 71444815; Sat, 08 Jan 2011 18:41:04 +0100 To: freebsd-stable@freebsd.org, freebsd-multimedia@freebsd.org From: Hans Petter Selasky X-Face: +~\`s("[*|O,="7?X@L.elg*F"OA\I/3%^p8g?ab%RN'( =?iso-8859-1?q?=3B=5FIjlA=3A=0A=09hGE=2E=2EEw?=, =?iso-8859-1?q?XAQ*o=23=5C/M=7ESC=3DS1-f9=7BEzRfT=27=7CHhll5Q=5Dha5Bt-s=7Co?= =?iso-8859-1?q?TlKMusi=3A1e=5BwJl=7Dkd=7DGR=0A=09Z0adGx-x=5F0zGbZj=27e?=(Y[(UNle~)8CQWXW@:DX+9)_YlB[tIccCPN$7/L' Date: Sat, 8 Jan 2011 18:41:10 +0100 MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201101081841.10690.hselasky@c2i.net> Cc: Subject: Webcamd testers wanted on FreeBSD 8.2 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jan 2011 17:51:09 -0000 Hi, Can someone running FreeBSD 8.2-RC1 with more than one or external USB webcam or DVB-XXX devices verify the following: 1) Install /usr/ports/multimedia/webcamd 2) Reboot 3) Check that character devices are created for your device(s) under /dev /dev/videoX for webcams /dev/dvb/adapterX for DVB devices 3) HAL should show your device. lshal | grep -i video lshal | grep -i dvb 4) Hot replug the USB plug of your USB webcam/DVB device, perform checks 1,2 and 3. If same result everything is OK. --HPS From owner-freebsd-stable@FreeBSD.ORG Sat Jan 8 18:18:06 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6C65510656AA; Sat, 8 Jan 2011 18:18:06 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe02.c2i.net [212.247.154.34]) by mx1.freebsd.org (Postfix) with ESMTP id 87EE38FC1B; Sat, 8 Jan 2011 18:18:05 +0000 (UTC) X-Cloudmark-Score: 0.000000 [] X-Cloudmark-Analysis: v=1.1 cv=V/BmsFzUvwB31XiPFgiZP9ZGx4++v9AH3cfAZ6JEaj8= c=1 sm=1 a=d7IFvdU83yMA:10 a=CL8lFSKtTFcA:10 a=i9M/sDlu2rpZ9XS819oYzg==:17 a=Rm3tFodZxdn1bf3s9rAA:9 a=eWDDUQYRfZCoE4t-wDQA:7 a=hAnKZw0fKNzF20Is4fnkFTFWlbUA:4 a=wPNLvfGTeEIA:10 a=Gi6b3XPG2v-FT_WttCcA:9 a=E_ILL27bHwPWKX1PjvkA:7 a=ejEqwGQXeCW6pxHW1zkQhxGt_QIA:4 a=i9M/sDlu2rpZ9XS819oYzg==:117 Received: from [188.126.198.129] (account mc467741@c2i.net HELO laptop002.hselasky.homeunix.org) by mailfe02.swip.net (CommuniGate Pro SMTP 5.2.19) with ESMTPA id 71613595; Sat, 08 Jan 2011 19:18:03 +0100 From: Hans Petter Selasky To: freebsd-stable@freebsd.org, Nick Hibma Date: Sat, 8 Jan 2011 19:18:09 +0100 User-Agent: KMail/1.13.5 (FreeBSD/8.1-STABLE; KDE/4.4.5; amd64; ; ) References: <201101081841.10690.hselasky@c2i.net> In-Reply-To: <201101081841.10690.hselasky@c2i.net> X-Face: +~\`s("[*|O,="7?X@L.elg*F"OA\I/3%^p8g?ab%RN'(; _IjlA: hGE..Ew, XAQ*o#\/M~SC=S1-f9{EzRfT'|Hhll5Q]ha5Bt-s|oTlKMusi:1e[wJl}kd}GR Z0adGx-x_0zGbZj'e(Y[(UNle~)8CQWXW@:DX+9)_YlB[tIccCPN$7/L' MIME-Version: 1.0 Content-Type: Multipart/Mixed; boundary="Boundary-00=_hpKKNWNCVpMh/2S" Message-Id: <201101081918.09664.hselasky@c2i.net> Cc: freebsd-multimedia@freebsd.org, kwm@freebsd.org, Kris Moore Subject: Re: Webcamd testers wanted on FreeBSD 8.2 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jan 2011 18:18:06 -0000 --Boundary-00=_hpKKNWNCVpMh/2S Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit On Saturday 08 January 2011 18:41:10 Hans Petter Selasky wrote: > Hi, > > Can someone running FreeBSD 8.2-RC1 with more than one or external USB > webcam or DVB-XXX devices verify the following: > > 1) Install /usr/ports/multimedia/webcamd > > 2) Reboot > > 3) Check that character devices are created for your device(s) under /dev > > /dev/videoX for webcams > > /dev/dvb/adapterX for DVB devices > > 3) HAL should show your device. > > lshal | grep -i video > lshal | grep -i dvb > > 4) Hot replug the USB plug of your USB webcam/DVB device, perform checks > 1,2 and 3. If same result everything is OK. > Hi, After that the devd notify string was changed last year, hald stopped registering attached USB devices because the match string was too narrow. Can someone with HAL knowledge please review the attached patch for /usr/ports/sysutils/hal . My patch uses strstr() instead of strncmp(), but really a full parse with respect to " characters is required, because cdev=ugen, could appear inside some strings. When it works, lshal will show USB devices plugged after boot. --HPS --Boundary-00=_hpKKNWNCVpMh/2S Content-Type: text/x-patch; charset="ISO-8859-1"; name="hal.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="hal.patch" diff -ur hal.orig/files/patch-hald_freebsd_hf-usb2.c hal/files/patch-hald_freebsd_hf-usb2.c --- hal.orig/files/patch-hald_freebsd_hf-usb2.c 2011-01-08 19:01:54.000000000 +0100 +++ hal/files/patch-hald_freebsd_hf-usb2.c 2011-01-08 19:05:14.000000000 +0100 @@ -133,8 +133,8 @@ + (strcmp(type, "CREATE") && strcmp(type, "DESTROY"))) + return FALSE; + -+ if (! strncmp(data, "cdev=ugen", strlen("cdev=ugen")) || -+ ! strncmp(data, "cdev=usb", strlen("cdev=usb"))) ++ if ((strstr(data, "cdev=ugen") != NULL) || ++ (strstr(data, "cdev=usb") != NULL)) + return TRUE; + + return FALSE; --Boundary-00=_hpKKNWNCVpMh/2S-- From owner-freebsd-stable@FreeBSD.ORG Sat Jan 8 18:35:23 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D9B4B1065737; Sat, 8 Jan 2011 18:35:23 +0000 (UTC) (envelope-from subbsd@gmail.com) Received: from mail-qy0-f175.google.com (mail-qy0-f175.google.com [209.85.216.175]) by mx1.freebsd.org (Postfix) with ESMTP id 7133B8FC08; Sat, 8 Jan 2011 18:35:23 +0000 (UTC) Received: by qyk8 with SMTP id 8so339674qyk.13 for ; Sat, 08 Jan 2011 10:35:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=5drQk3e5hyvlrq0/L2cBWsABuICEFoLWLR/tQ0VFPSI=; b=IByjsJpW8T4uPf+QhLHVaSenberr0BMDgb/Q8yfqvbnKrMmeu4ranp2syPpK5w+WSL iK8e7jmcIQ0CQLgav8/W3t2fYkm1oorRD8OGWM82V0f8iHqLcrTEoXnoibp/k6/iZOO6 PqFSPl5q4h+30QRRY8MRH8DZOY8IxY0M56f2w= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=jALSE/d0NMX3V0CpaClPIPPhzG+4nuQgFm2JFJY0Ub7dVQQJusoTBg/kcSwYdwvuhy 0KyjQiLyGJVP+TWqqPaYpEDOxI5L8U9WU6Y2Z8iKHEqousVTxv4Lc7x+HRVZ++tV+xWK UYCkJpyrECSTp8XsZH7sYOFX9GjMIp5mKc7Og= MIME-Version: 1.0 Received: by 10.229.75.18 with SMTP id w18mr2208457qcj.95.1294510030181; Sat, 08 Jan 2011 10:07:10 -0800 (PST) Received: by 10.229.100.73 with HTTP; Sat, 8 Jan 2011 10:07:10 -0800 (PST) In-Reply-To: <201101081841.10690.hselasky@c2i.net> References: <201101081841.10690.hselasky@c2i.net> Date: Sat, 8 Jan 2011 21:07:10 +0300 Message-ID: From: Subbsd To: Hans Petter Selasky Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-multimedia@freebsd.org, freebsd-stable@freebsd.org Subject: Re: Webcamd testers wanted on FreeBSD 8.2 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jan 2011 18:35:23 -0000 On Sat, Jan 8, 2011 at 8:41 PM, Hans Petter Selasky wrote: > > Hi, > > Can someone running FreeBSD 8.2-RC1 with more than one or external USB webcam > or DVB-XXX devices verify the following: > > Hello. Can the boot sequence of .ko modules in loader.conf somehow influence the success of the detection camera? I've seen this problem on FreeBSD-current amd64 some time ago. Maybe it was a coincidence - but the /dev/video0 does not always appear when loading the module was at the end of the list. In the near future I will test in 9-0 CURRENT and 8.2-RC2 From owner-freebsd-stable@FreeBSD.ORG Sat Jan 8 18:45:13 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E8C3510656D3 for ; Sat, 8 Jan 2011 18:45:12 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from ftp.translate.ru (ftp.translate.ru [80.249.188.42]) by mx1.freebsd.org (Postfix) with ESMTP id B10F38FC18 for ; Sat, 8 Jan 2011 18:45:11 +0000 (UTC) Received: from [192.168.134.2] (89.112.15.178.pppoe.eltel.net [89.112.15.178]) (Authenticated sender: lev@serebryakov.spb.ru) by ftp.translate.ru (Postfix) with ESMTPA id 341D813DF5F for ; Sat, 8 Jan 2011 21:45:04 +0300 (MSK) Date: Sat, 8 Jan 2011 21:44:57 +0300 From: Lev Serebryakov Organization: FreeBSD X-Priority: 3 (Normal) Message-ID: <204344488.20110108214457@serebryakov.spb.ru> To: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----------B9A228150F548A" Subject: 8.2-PRERELEASE: live deadlock, almost all processes in "pfault" state X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jan 2011 18:45:13 -0000 ------------B9A228150F548A Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Hello, Freebsd-stable. I've added `transmission' BitTorrent client to my home server and now it deadlocks easily (after about 1 hour of intensive download and seeding). This server is upgraded from 7.x and last time I've run transmission on 7.x system without any problems. I have home partition on geom_raid5 device, so I can not exclude this third-party module from experiments. My home filsystem has 32KiB block and all other filesystems (/, /var, /tmp, /usr) has standard 16KiB block sizes. I know, that 7.x system had (has?) deadlock when 16KiB and 64KiB file systems are mixed up on one system, but I never experienced deadlocks with 16KiB and 32KiB mixture. All filesystems (Except root) is SU, but no gjournal so SU_J patch are in use. Same BitTorrent client on same filesystem, but accessed via NFS (from other host), doesn't cause deadlock and works rock-stable for days. I've built kernel with all debug options, waited for deadlock and collect all information, mentioned in Developer's Handbook / Debugging Deadlocks. Capture from debug session is attached, together with kernel config and dmesg from rebooting. As I can easily reproduce this deadlock, I could provide any additional information from kernel debugger, if needed. System: FreeBSD 8.2-PRERELEASE cvsup: 2011-01-08 00:41:24 MSK (GTM+3) time Platform: amd64 --=20 // Black Lion AKA Lev Serebryakov ------------B9A228150F548A Content-Type: application/octet-stream; name="debug-session.log" Content-transfer-encoding: base64 Content-Disposition: attachment; filename="debug-session.log" bG9naW46IEtEQjogZW50ZXI6IExpbmUgYnJlYWsgb24gY29uc29sZQ0NClt0aHJlYWQgcGlk IDExIHRpZCAxMDAwMDMgXQ0KU3RvcHBlZCBhdCAgICAgIGtkYl9lbnRlcisweDNkOiBtb3Zx ICAgICQwLDB4M2Q3YTkwKCVyaXApDQpkYj4gcHMNCiAgcGlkICBwcGlkICBwZ3JwICAgdWlk ICAgc3RhdGUgICB3bWVzZyAgICAgICAgIHdjaGFuICAgICAgICBjbWQNCiAxMjA5ICAxMjA3 ICAxMjA5ICAgICAwICBTKyAgICAgIHR0eWluICAgIDB4ZmZmZmZmMDAwNDk5MmNhOCBjc2gN CiAxMjA3ICAxMTU5ICAxMjA3ICAxMDAxICBTKyAgICAgIHdhaXQgICAgIDB4ZmZmZmZmMDAw NGMxYTAwMCBzdQ0KIDEyMDAgICA4MDQgICA4MDQgICAgIDAgIFNMICAgICAgcGZhdWx0ICAg MHhmZmZmZmZmZjgwODcyMjVjIHNtYmQNCiAxMTkxICAgICAxICAxMTkxICAxMDAyICBTTHMg ICAgICh0aHJlYWRlZCkgICAgICAgICAgICAgICAgICB0cmFuc21pc3Npb24tZGFlbW9uDQox MDAxNTEgICAgICAgICAgICAgICAgICAgRCAgICAgICBnZXRibGsgICAweGZmZmZmZjgwN2Jj NGQwNDggdHJhbnNtaXNzaW9uLWRhZW1vbg0KMTAwMTQ5ICAgICAgICAgICAgICAgICAgIFMg ICAgICAgc2VsZWN0ICAgMHhmZmZmZmYwMDExMGJjOWMwIHRyYW5zbWlzc2lvbi1kYWVtb24N CjEwMDE0OCAgICAgICAgICAgICAgICAgICBEICAgICAgIGJpb3JkICAgIDB4ZmZmZmZmODA3 YjkwNTk3MCB0cmFuc21pc3Npb24tZGFlbW9uDQoxMDAwOTYgICAgICAgICAgICAgICAgICAg RCAgICAgICBwZmF1bHQgICAweGZmZmZmZmZmODA4NzIyNWMgdHJhbnNtaXNzaW9uLWRhZW1v bg0KIDExNTkgIDExNTggIDExNTkgIDEwMDEgIFNzKyAgICAgcGF1c2UgICAgMHhmZmZmZmYw MDA0Yjc2MGEwIHRjc2gNCiAxMTU4ICAxMTU2ICAxMTU2ICAxMDAxICBTTCAgICAgIHBmYXVs dCAgIDB4ZmZmZmZmZmY4MDg3MjI1YyBzc2hkDQogMTE1NiAgMTAzMyAgMTE1NiAgICAgMCAg U3MgICAgICBzYndhaXQgICAweGZmZmZmZjAwMTExMmJiZTQgc3NoZA0KIDExNTMgICAgIDEg IDExNTMgICAgIDAgIFNMcysgICAgcGZhdWx0ICAgMHhmZmZmZmZmZjgwODcyMjVjIGdldHR5 DQogMTE1MiAgICAgMSAgMTE1MiAgICAgMCAgU0xzKyAgICBwZmF1bHQgICAweGZmZmZmZmZm ODA4NzIyNWMgZ2V0dHkNCiAxMDUyICAgICAxICAxMDUyICAgICAwICA/cyAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICBjcm9uDQogMTA0NSAgICAgMSAgMTA0NSAgICAyNSAg P3MgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc2VuZG1haWwNCiAxMDQxICAg ICAxICAxMDQxICAgICAwICBTTHMgICAgIHBmYXVsdCAgIDB4ZmZmZmZmZmY4MDg3MjI1YyBz ZW5kbWFpbA0KIDEwMzMgICAgIDEgIDEwMzMgICAgIDAgIFNzICAgICAgc2VsZWN0ICAgMHhm ZmZmZmYwMDA0ZjI3YTQwIHNzaGQNCiAgOTYyICAgICAxICAgOTYyICAgNjAwICBTcyAgICAg IHNlbGVjdCAgIDB4ZmZmZmZmMDAwNGYyNzE0MCBjdnN1cGQNCiAgODcyICAgICAxICAgODcy ICAgICAwICBTTHMgICAgIHBmYXVsdCAgIDB4ZmZmZmZmZmY4MDg3MjI1YyBudHBkDQogIDg1 NiAgIDgwNCAgIDgwNCAgICAgMCAgUyAgICAgICBzZWxlY3QgICAweGZmZmZmZjAwMDRmMjdh YzAgc21iZA0KICA4MDQgICAgIDEgICA4MDQgICAgIDAgIFNMcyAgICAgcGZhdWx0ICAgMHhm ZmZmZmZmZjgwODcyMjVjIHNtYmQNCiAgODAwICAgICAxICAgODAwICAgICAwICBTTHMgICAg IHBmYXVsdCAgIDB4ZmZmZmZmZmY4MDg3MjI1YyBubWJkDQogIDc0MyAgIDc0MiAgIDc0MiAg ICAgMCAgUyAgICAgICAodGhyZWFkZWQpICAgICAgICAgICAgICAgICAgbmZzZA0KMTAwMTIy ICAgICAgICAgICAgICAgICAgIFMgICAgICAgcnBjc3ZjICAgMHhmZmZmZmYwMDExMGJjNTIw IG5mc2Q6IHNlcnZpY2UNCjEwMDEyMSAgICAgICAgICAgICAgICAgICBTICAgICAgIHJwY3N2 YyAgIDB4ZmZmZmZmMDAxMTBiYzVhMCBuZnNkOiBzZXJ2aWNlDQoxMDAxMjAgICAgICAgICAg ICAgICAgICAgUyAgICAgICBycGNzdmMgICAweGZmZmZmZjAwMDRiZWZiYTAgbmZzZDogc2Vy dmljZQ0KMTAwMDg3ICAgICAgICAgICAgICAgICAgIFMgICAgICAgcnBjc3ZjICAgMHhmZmZm ZmYwMDA0YWQxMmEwIG5mc2Q6IG1hc3Rlcg0KICA3NDIgICAgIDEgICA3NDIgICAgIDAgIFNz ICAgICAgYWNjZXB0ICAgMHhmZmZmZmYwMDA0Y2EzMDY2IG5mc2QNCiAgNzMzICAgICAxICAg NzMzICAgICAwICBTcyAgICAgIHNlbGVjdCAgIDB4ZmZmZmZmMDAwNGE2ODY0MCBtb3VudGQN CiAgNjM4ICAgICAxICAgNjM4ICAgICAwICBTTHMgICAgIHBmYXVsdCAgIDB4ZmZmZmZmZmY4 MDg3MjI1YyBycGNiaW5kDQogIDYxMiAgICAgMSAgIDYxMiAgICAgMCAgU0xzICAgICBwZmF1 bHQgICAweGZmZmZmZmZmODA4NzIyNWMgc3lzbG9nZA0KICA0ODggICAgIDEgICA0ODggICAg IDAgIFNzICAgICAgc2VsZWN0ICAgMHhmZmZmZmYwMDA0YmVmNTQwIGRldmQNCiAgNDc0ICAg ICAxICAgNDc0ICAgIDY1ICBTcyAgICAgIHNlbGVjdCAgIDB4ZmZmZmZmMDAwNGFkMDhjMCBk aGNsaWVudA0KICA0NTggICAgIDEgICA0NTggICAgIDAgIFNzICAgICAgc2VsZWN0ICAgMHhm ZmZmZmYwMDA0YWQwMTQwIGRoY2xpZW50DQogIDEyMiAgICAgMSAgIDEyMiAgICAgMCAgU3Mg ICAgICBwYXVzZSAgICAweGZmZmZmZjAwMDRjMWI5NjAgYWRqa2VybnR6DQogICAyMCAgICAg MCAgICAgMCAgICAgMCAgU0wgICAgICAodGhyZWFkZWQpICAgICAgICAgICAgICAgICAgZ19y YWlkNQ0KMTAwMDc5ICAgICAgICAgICAgICAgICAgIEQgICAgICAgZ3I1ZG8gICAgMHhmZmZm ZmYwMDA0YTliYzY4IFtnX3JhaWQ1L2RvbmUgc3RvcmFnXQ0KMTAwMDc4ICAgICAgICAgICAg ICAgICAgIEQgICAgICAgdm13YWl0ICAgMHhmZmZmZmZmZjgwODcyMjVjIFtnX3JhaWQ1L21h aW4gc3RvcmFnXQ0KICAgMTkgICAgIDAgICAgIDAgICAgIDAgIFNMICAgICAgc2RmbHVzaCAg MHhmZmZmZmZmZjgwODcxN2Y4IFtzb2Z0ZGVwZmx1c2hdDQogICAxOCAgICAgMCAgICAgMCAg ICAgMCAgU0wgICAgICB2bHJ1d3QgICAweGZmZmZmZjAwMDRhOTg0NjAgW3ZubHJ1XQ0KICAg MTcgICAgIDAgICAgIDAgICAgIDAgIFNMICAgICAgd3N3YnVmMCAgMHhmZmZmZmZmZjgwNjY2 ZjYwIFtzeW5jZXJdDQogICAxNiAgICAgMCAgICAgMCAgICAgMCAgU0wgICAgICBwc2xlZXAg ICAweGZmZmZmZmZmODA4NjVkNDggW2J1ZmRhZW1vbl0NCiAgIDE1ICAgICAwICAgICAwICAg ICAwICBTTCAgICAgIHBnemVybyAgIDB4ZmZmZmZmZmY4MDg3MzJhYyBbcGFnZXplcm9dDQog ICAgOSAgICAgMCAgICAgMCAgICAgMCAgU0wgICAgICBwb2xsaWQgICAweGZmZmZmZmZmODA2 YTNkMjggW2lkbGVwb2xsXQ0KICAgIDggICAgIDAgICAgIDAgICAgIDAgIFNMICAgICAgcHNs ZWVwICAgMHhmZmZmZmZmZjgwODcyNjQ4IFt2bWRhZW1vbl0NCiAgICA3ICAgICAwICAgICAw ICAgICAwICBTTCAgICAgIHBzbGVlcCAgIDB4ZmZmZmZmZmY4MDg3MjYwYyBbcGFnZWRhZW1v bl0NCiAgICA2ICAgICAwICAgICAwICAgICAwICBTTCAgICAgIGNjYl9zY2FuIDB4ZmZmZmZm ZmY4MDY4OTRlMCBbeHB0X3RocmRdDQogICAgNSAgICAgMCAgICAgMCAgICAgMCAgU0wgICAg ICAtICAgICAgICAweGZmZmZmZjgwMDAzMzAwMDAgW2Z3MF9wcm9iZV0NCiAgIDE0ICAgICAw ICAgICAwICAgICAwICBTTCAgICAgICh0aHJlYWRlZCkgICAgICAgICAgICAgICAgICB1c2IN CjEwMDA2MSAgICAgICAgICAgICAgICAgICBEICAgICAgIC0gICAgICAgIDB4ZmZmZmZmODAw MDMyZGUxOCBbdXNidXM3XQ0KMTAwMDYwICAgICAgICAgICAgICAgICAgIEQgICAgICAgLSAg ICAgICAgMHhmZmZmZmY4MDAwMzJkZGMwIFt1c2J1czddDQoxMDAwNTkgICAgICAgICAgICAg ICAgICAgRCAgICAgICAtICAgICAgICAweGZmZmZmZjgwMDAzMmRkNjggW3VzYnVzN10NCjEw MDA1OCAgICAgICAgICAgICAgICAgICBEICAgICAgIC0gICAgICAgIDB4ZmZmZmZmODAwMDMy ZGQxMCBbdXNidXM3XQ0KMTAwMDU3ICAgICAgICAgICAgICAgICAgIEQgICAgICAgLSAgICAg ICAgMHhmZmZmZmY4MDAwMzI0ZWYwIFt1c2J1czZdDQoxMDAwNTYgICAgICAgICAgICAgICAg ICAgRCAgICAgICAtICAgICAgICAweGZmZmZmZjgwMDAzMjRlOTggW3VzYnVzNl0NCjEwMDA1 NSAgICAgICAgICAgICAgICAgICBEICAgICAgIC0gICAgICAgIDB4ZmZmZmZmODAwMDMyNGU0 MCBbdXNidXM2XQ0KMTAwMDU0ICAgICAgICAgICAgICAgICAgIEQgICAgICAgLSAgICAgICAg MHhmZmZmZmY4MDAwMzI0ZGU4IFt1c2J1czZdDQoxMDAwNTMgICAgICAgICAgICAgICAgICAg RCAgICAgICAtICAgICAgICAweGZmZmZmZjgwMDAzMWJlZjAgW3VzYnVzNV0NCjEwMDA1MiAg ICAgICAgICAgICAgICAgICBEICAgICAgIC0gICAgICAgIDB4ZmZmZmZmODAwMDMxYmU5OCBb dXNidXM1XQ0KMTAwMDUxICAgICAgICAgICAgICAgICAgIEQgICAgICAgLSAgICAgICAgMHhm ZmZmZmY4MDAwMzFiZTQwIFt1c2J1czVdDQoxMDAwNTAgICAgICAgICAgICAgICAgICAgRCAg ICAgICAtICAgICAgICAweGZmZmZmZjgwMDAzMWJkZTggW3VzYnVzNV0NCjEwMDA0OCAgICAg ICAgICAgICAgICAgICBEICAgICAgIC0gICAgICAgIDB4ZmZmZmZmODAwMDMxMmVmMCBbdXNi dXM0XQ0KMTAwMDQ3ICAgICAgICAgICAgICAgICAgIEQgICAgICAgLSAgICAgICAgMHhmZmZm ZmY4MDAwMzEyZTk4IFt1c2J1czRdDQoxMDAwNDYgICAgICAgICAgICAgICAgICAgRCAgICAg ICAtICAgICAgICAweGZmZmZmZjgwMDAzMTJlNDAgW3VzYnVzNF0NCjEwMDA0NSAgICAgICAg ICAgICAgICAgICBEICAgICAgIC0gICAgICAgIDB4ZmZmZmZmODAwMDMxMmRlOCBbdXNidXM0 XQ0KMTAwMDQzICAgICAgICAgICAgICAgICAgIEQgICAgICAgLSAgICAgICAgMHhmZmZmZmY4 MDAwMzA5ZTE4IFt1c2J1czNdDQoxMDAwNDIgICAgICAgICAgICAgICAgICAgRCAgICAgICAt ICAgICAgICAweGZmZmZmZjgwMDAzMDlkYzAgW3VzYnVzM10NCjEwMDA0MSAgICAgICAgICAg ICAgICAgICBEICAgICAgIC0gICAgICAgIDB4ZmZmZmZmODAwMDMwOWQ2OCBbdXNidXMzXQ0K MTAwMDQwICAgICAgICAgICAgICAgICAgIEQgICAgICAgLSAgICAgICAgMHhmZmZmZmY4MDAw MzA5ZDEwIFt1c2J1czNdDQoxMDAwMzkgICAgICAgICAgICAgICAgICAgRCAgICAgICAtICAg ICAgICAweGZmZmZmZjgwMDAzMDBlZjAgW3VzYnVzMl0NCjEwMDAzOCAgICAgICAgICAgICAg ICAgICBEICAgICAgIC0gICAgICAgIDB4ZmZmZmZmODAwMDMwMGU5OCBbdXNidXMyXQ0KMTAw MDM3ICAgICAgICAgICAgICAgICAgIEQgICAgICAgLSAgICAgICAgMHhmZmZmZmY4MDAwMzAw ZTQwIFt1c2J1czJdDQoxMDAwMzYgICAgICAgICAgICAgICAgICAgRCAgICAgICAtICAgICAg ICAweGZmZmZmZjgwMDAzMDBkZTggW3VzYnVzMl0NCjEwMDAzNCAgICAgICAgICAgICAgICAg ICBEICAgICAgIC0gICAgICAgIDB4ZmZmZmZmODAwMDJmN2VmMCBbdXNidXMxXQ0KMTAwMDMz ICAgICAgICAgICAgICAgICAgIEQgICAgICAgLSAgICAgICAgMHhmZmZmZmY4MDAwMmY3ZTk4 IFt1c2J1czFdDQoxMDAwMzIgICAgICAgICAgICAgICAgICAgRCAgICAgICAtICAgICAgICAw eGZmZmZmZjgwMDAyZjdlNDAgW3VzYnVzMV0NCjEwMDAzMSAgICAgICAgICAgICAgICAgICBE ICAgICAgIC0gICAgICAgIDB4ZmZmZmZmODAwMDJmN2RlOCBbdXNidXMxXQ0KMTAwMDI5ICAg ICAgICAgICAgICAgICAgIEQgICAgICAgLSAgICAgICAgMHhmZmZmZmY4MDAwMmVlZWYwIFt1 c2J1czBdDQoxMDAwMjggICAgICAgICAgICAgICAgICAgRCAgICAgICAtICAgICAgICAweGZm ZmZmZjgwMDAyZWVlOTggW3VzYnVzMF0NCjEwMDAyNyAgICAgICAgICAgICAgICAgICBEICAg ICAgIC0gICAgICAgIDB4ZmZmZmZmODAwMDJlZWU0MCBbdXNidXMwXQ0KMTAwMDI2ICAgICAg ICAgICAgICAgICAgIEQgICAgICAgLSAgICAgICAgMHhmZmZmZmY4MDAwMmVlZGU4IFt1c2J1 czBdDQogICAxMyAgICAgMCAgICAgMCAgICAgMCAgU0wgICAgICAtICAgICAgICAweGZmZmZm ZmZmODA2YTRjMjQgW3lhcnJvd10NCiAgICA0ICAgICAwICAgICAwICAgICAwICBTTCAgICAg IC0gICAgICAgIDB4ZmZmZmZmZmY4MDZhMGEwOCBbZ19kb3duXQ0KICAgIDMgICAgIDAgICAg IDAgICAgIDAgIFNMICAgICAgLSAgICAgICAgMHhmZmZmZmZmZjgwNmEwYTAwIFtnX3VwXQ0K ICAgIDIgICAgIDAgICAgIDAgICAgIDAgIFNMICAgICAgLSAgICAgICAgMHhmZmZmZmZmZjgw NmEwOWYwIFtnX2V2ZW50XQ0KICAgMTIgICAgIDAgICAgIDAgICAgIDAgIFdMICAgICAgKHRo cmVhZGVkKSAgICAgICAgICAgICAgICAgIGludHINCjEwMDA2OCAgICAgICAgICAgICAgICAg ICBJICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBbaXJxMTogYXRrYmQwXQ0K MTAwMDY3ICAgICAgICAgICAgICAgICAgIEkgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgIFtzd2kwOiB1YXJ0XQ0KMTAwMDY2ICAgICAgICAgICAgICAgICAgIEkgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgIFtpcnE3OiBwcGMwXQ0KMTAwMDY1ICAgICAg ICAgICAgICAgICAgIEkgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIFtpcnEy MjogYWhjaTBdDQoxMDAwNjIgICAgICAgICAgICAgICAgICAgSSAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgW2lycTIwOiBlbTAgZndvaGNpMF0NCjEwMDA0OSAgICAgICAg ICAgICAgICAgICBJICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBbaXJxMTk6 IHVoY2k0XQ0KMTAwMDQ0ICAgICAgICAgICAgICAgICAgIEkgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgIFtpcnEyMzogdWhjaTMgZWhjaTFdDQoxMDAwMzUgICAgICAgICAg ICAgICAgICAgSSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgW2lycTE4OiB1 aGNpMiBlaGNpMCpdDQoxMDAwMzAgICAgICAgICAgICAgICAgICAgSSAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgW2lycTIxOiB1aGNpMV0NCjEwMDAyNSAgICAgICAgICAg ICAgICAgICBJICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBbaXJxMTY6IHVo Y2kwXQ0KMTAwMDIzICAgICAgICAgICAgICAgICAgIEkgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgIFtpcnE5OiBhY3BpMF0NCjEwMDAyMiAgICAgICAgICAgICAgICAgICBJ ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBbc3dpNjogR2lhbnQgdGFza3Fd DQoxMDAwMjAgICAgICAgICAgICAgICAgICAgSSAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgW3N3aTU6ICtdDQoxMDAwMTkgICAgICAgICAgICAgICAgICAgSSAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgW3N3aTI6IGNhbWJpb10NCjEwMDAxNCAgICAg ICAgICAgICAgICAgICBJICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBbc3dp NjogdGFzayBxdWV1ZV0NCjEwMDAwOCAgICAgICAgICAgICAgICAgICBJICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICBbc3dpMTogbmV0aXNyIDBdDQoxMDAwMDcgICAgICAg ICAgICAgICAgICAgSSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgW3N3aTM6 IHZtXQ0KMTAwMDA2ICAgICAgICAgICAgICAgICAgIEkgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgIFtzd2k0OiBjbG9ja10NCjEwMDAwNSAgICAgICAgICAgICAgICAgICBJ ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBbc3dpNDogY2xvY2tdDQogICAx MSAgICAgMCAgICAgMCAgICAgMCAgUkwgICAgICAodGhyZWFkZWQpICAgICAgICAgICAgICAg ICAgaWRsZQ0KMTAwMDA0ICAgICAgICAgICAgICAgICAgIFJ1biAgICAgQ1BVIDAgICAgICAg ICAgICAgICAgICAgICAgIFtpZGxlOiBjcHUwXQ0KMTAwMDAzICAgICAgICAgICAgICAgICAg IFJ1biAgICAgQ1BVIDEgICAgICAgICAgICAgICAgICAgICAgIFtpZGxlOiBjcHUxXQ0KICAg IDEgICAgIDAgICAgIDEgICAgIDAgIFNMcyAgICAgd2FpdCAgICAgMHhmZmZmZmYwMDAxNThl OGMwIFtpbml0XQ0KICAgMTAgICAgIDAgICAgIDAgICAgIDAgIFNMICAgICAgYXVkaXRfd28g MHhmZmZmZmZmZjgwODcwYzUwIFthdWRpdF0NCiAgICAwICAgICAwICAgICAwICAgICAwICBT THMgICAgICh0aHJlYWRlZCkgICAgICAgICAgICAgICAgICBrZXJuZWwNCjEwMDA2MyAgICAg ICAgICAgICAgICAgICBEICAgICAgIC0gICAgICAgIDB4ZmZmZmZmMDAwMTc5MzEwMCBbZncw X3Rhc2txXQ0KMTAwMDI0ICAgICAgICAgICAgICAgICAgIEQgICAgICAgLSAgICAgICAgMHhm ZmZmZmYwMDAxNmMxMjAwIFtlbTAgdGFza3FdDQoxMDAwMjEgICAgICAgICAgICAgICAgICAg RCAgICAgICAtICAgICAgICAweGZmZmZmZjAwMDE1ZTY1MDAgW3RocmVhZCB0YXNrcV0NCjEw MDAxOCAgICAgICAgICAgICAgICAgICBEICAgICAgIC0gICAgICAgIDB4ZmZmZmZmMDAwMTVl NmQ4MCBbYWNwaV90YXNrXzJdDQoxMDAwMTcgICAgICAgICAgICAgICAgICAgRCAgICAgICAt ICAgICAgICAweGZmZmZmZjAwMDE1ZTZkODAgW2FjcGlfdGFza18xXQ0KMTAwMDE2ICAgICAg ICAgICAgICAgICAgIEQgICAgICAgLSAgICAgICAgMHhmZmZmZmYwMDAxNWU2ZDgwIFthY3Bp X3Rhc2tfMF0NCjEwMDAxNSAgICAgICAgICAgICAgICAgICBEICAgICAgIC0gICAgICAgIDB4 ZmZmZmZmMDAwMTVlNmUwMCBba3F1ZXVlIHRhc2txXQ0KMTAwMDEyICAgICAgICAgICAgICAg ICAgIEQgICAgICAgLSAgICAgICAgMHhmZmZmZmYwMDAxNWE0YTAwIFtmaXJtd2FyZSB0YXNr cV0NCjEwMDAwMCAgICAgICAgICAgICAgICAgICBEICAgICAgIHZtd2FpdCAgIDB4ZmZmZmZm ZmY4MDg3MjI1YyBbc3dhcHBlcl0NCmRiPiBzaG93IHBjcHUNCmNwdWlkICAgICAgICA9IDEN CmR5bmFtaWMgcGNwdSA9IDB4ZmZmZmZmODA3Zjk4NDk4MA0KY3VydGhyZWFkICAgID0gMHhm ZmZmZmYwMDAxNTkxNDYwOiBwaWQgMTEgImlkbGU6IGNwdTEiDQpjdXJwY2IgICAgICAgPSAw eGZmZmZmZjgwMDAwMmJkMDANCmZwY3VydGhyZWFkICA9IG5vbmUNCmlkbGV0aHJlYWQgICA9 IDB4ZmZmZmZmMDAwMTU5MTQ2MDogdGlkIDEwMDAwMyAiaWRsZTogY3B1MSINCmN1cnBtYXAg ICAgICA9IDB4ZmZmZmZmZmY4MDZhMTRmMA0KdHNzcCAgICAgICAgID0gMHhmZmZmZmZmZjgw OGFiNGU4DQpjb21tb250c3NwICAgPSAweGZmZmZmZmZmODA4YWI0ZTgNCnJzcDAgICAgICAg ICA9IDB4ZmZmZmZmODAwMDAyYmQwMA0KZ3MzMnAgICAgICAgID0gMHhmZmZmZmZmZjgwOGFh MzIwDQpsZHQgICAgICAgICAgPSAweGZmZmZmZmZmODA4YWEzNjANCnRzcyAgICAgICAgICA9 IDB4ZmZmZmZmZmY4MDhhYTM1MA0Kc3BpbiBsb2NrcyBoZWxkOg0KZGI+IHNob3cgYWxscGNw dQ0KQ3VycmVudCBDUFU6IDENCg0KY3B1aWQgICAgICAgID0gMA0KZHluYW1pYyBwY3B1ID0g MHgzNTc5ODANCmN1cnRocmVhZCAgICA9IDB4ZmZmZmZmMDAwMTU5MTAwMDogcGlkIDExICJp ZGxlOiBjcHUwIg0KY3VycGNiICAgICAgID0gMHhmZmZmZmY4MDAwMDMwZDAwDQpmcGN1cnRo cmVhZCAgPSBub25lDQppZGxldGhyZWFkICAgPSAweGZmZmZmZjAwMDE1OTEwMDA6IHRpZCAx MDAwMDQgImlkbGU6IGNwdTAiDQpjdXJwbWFwICAgICAgPSAweGZmZmZmZmZmODA2YTE0ZjAN CnRzc3AgICAgICAgICA9IDB4ZmZmZmZmZmY4MDhhYjQ4MA0KY29tbW9udHNzcCAgID0gMHhm ZmZmZmZmZjgwOGFiNDgwDQpyc3AwICAgICAgICAgPSAweGZmZmZmZjgwMDAwMzBkMDANCmdz MzJwICAgICAgICA9IDB4ZmZmZmZmZmY4MDhhYTJiOA0KbGR0ICAgICAgICAgID0gMHhmZmZm ZmZmZjgwOGFhMmY4DQp0c3MgICAgICAgICAgPSAweGZmZmZmZmZmODA4YWEyZTgNCnNwaW4g bG9ja3MgaGVsZDoNCg0KY3B1aWQgICAgICAgID0gMQ0KZHluYW1pYyBwY3B1ID0gMHhmZmZm ZmY4MDdmOTg0OTgwDQpjdXJ0aHJlYWQgICAgPSAweGZmZmZmZjAwMDE1OTE0NjA6IHBpZCAx MSAiaWRsZTogY3B1MSINCmN1cnBjYiAgICAgICA9IDB4ZmZmZmZmODAwMDAyYmQwMA0KZnBj dXJ0aHJlYWQgID0gbm9uZQ0KaWRsZXRocmVhZCAgID0gMHhmZmZmZmYwMDAxNTkxNDYwOiB0 aWQgMTAwMDAzICJpZGxlOiBjcHUxIg0KY3VycG1hcCAgICAgID0gMHhmZmZmZmZmZjgwNmEx NGYwDQp0c3NwICAgICAgICAgPSAweGZmZmZmZmZmODA4YWI0ZTgNCmNvbW1vbnRzc3AgICA9 IDB4ZmZmZmZmZmY4MDhhYjRlOA0KcnNwMCAgICAgICAgID0gMHhmZmZmZmY4MDAwMDJiZDAw DQpnczMycCAgICAgICAgPSAweGZmZmZmZmZmODA4YWEzMjANCmxkdCAgICAgICAgICA9IDB4 ZmZmZmZmZmY4MDhhYTM2MA0KdHNzICAgICAgICAgID0gMHhmZmZmZmZmZjgwOGFhMzUwDQpz cGluIGxvY2tzIGhlbGQ6DQoNCmRiPiBzaG93IGxvY2tzDQpkYj4gc2hvdyBhbGxsb2Nrcw0K UHJvY2VzcyAxMTkxICh0cmFuc21pc3Npb24tZGFlbW9uKSB0aHJlYWQgMHhmZmZmZmYwMDA0 YWZmOGMwICgxMDAxNTEpDQpzaGFyZWQgbG9ja21nciB1ZnMgKHVmcykgciA9IDAgKDB4ZmZm ZmZmMDA1NTNkNDdlOCkgbG9ja2VkIEAgL3Vzci9zcmMvc3lzL2tlcm4vdmZzX3Zub3BzLmM6 NTM5DQpQcm9jZXNzIDExOTEgKHRyYW5zbWlzc2lvbi1kYWVtb24pIHRocmVhZCAweGZmZmZm ZjAwMTEyMjU0NjAgKDEwMDE0OCkNCmV4Y2x1c2l2ZSBsb2NrbWdyIGJ1ZndhaXQgKGJ1Zndh aXQpIHIgPSAwICgweGZmZmZmZjgwN2I5MDVhMDgpIGxvY2tlZCBAIC91c3Ivc3JjL3N5cy9r ZXJuL3Zmc19iaW8uYzoxODkxDQpzaGFyZWQgbG9ja21nciB1ZnMgKHVmcykgciA9IDAgKDB4 ZmZmZmZmMDA2ZjYzMTU3OCkgbG9ja2VkIEAgL3Vzci9zcmMvc3lzL2tlcm4vdmZzX3Zub3Bz LmM6NTM5DQpQcm9jZXNzIDExNTYgKHNzaGQpIHRocmVhZCAweGZmZmZmZjAwMDRiZTk0NjAg KDEwMDEwOCkNCmV4Y2x1c2l2ZSBzeCBzb19yY3Zfc3ggKHNvX3Jjdl9zeCkgciA9IDAgKDB4 ZmZmZmZmMDAxMTEyYmI5OCkgbG9ja2VkIEAgL3Vzci9zcmMvc3lzL2tlcm4vdWlwY19zb2Nr YnVmLmM6MTQ4DQpQcm9jZXNzIDE3IChzeW5jZXIpIHRocmVhZCAweGZmZmZmZjAwMDRhOTU0 NjAgKDEwMDA3NSkNCmV4Y2x1c2l2ZSBsb2NrbWdyIGJ1ZndhaXQgKGJ1ZndhaXQpIHIgPSAw ICgweGZmZmZmZjgwN2I4YWE2MTgpIGxvY2tlZCBAIC91c3Ivc3JjL3N5cy9rZXJuL3Zmc19j bHVzdGVyLmM6NzczDQpleGNsdXNpdmUgbG9ja21nciB1ZnMgKHVmcykgciA9IDAgKDB4ZmZm ZmZmMDA2ZjNmN2NjOCkgbG9ja2VkIEAgL3Vzci9zcmMvc3lzL2tlcm4vdmZzX3N1YnIuYzoy MTIwDQpleGNsdXNpdmUgbG9ja21nciBzeW5jZXIgKHN5bmNlcikgciA9IDAgKDB4ZmZmZmZm MDAwNGJkNzMwOCkgbG9ja2VkIEAgL3Vzci9zcmMvc3lzL2tlcm4vdmZzX3N1YnIuYzoxNzMw DQpkYj4gc2hvdyBsb2NrZWR2bm9kcw0KTG9ja2VkIHZub2Rlcw0KDQoweGZmZmZmZjAwMDRi ZDcyNzA6IHRhZyBzeW5jZXIsIHR5cGUgVk5PTg0KICAgIHVzZWNvdW50IDEsIHdyaXRlY291 bnQgMCwgcmVmY291bnQgMiBtb3VudGVkaGVyZSAwDQogICAgZmxhZ3MgKCkNCiAgICBsb2Nr IHR5cGUgc3luY2VyOiBFWENMIGJ5IHRocmVhZCAweGZmZmZmZjAwMDRhOTU0NjAgKHBpZCAx NykNCiMwIDB4ZmZmZmZmZmY4MDJhNGFjNCBhdCBfX2xvY2ttZ3JfYXJncysweDc5NA0KIzEg MHhmZmZmZmZmZjgwMzNlNTE5IGF0IHZvcF9zdGRsb2NrKzB4MzkNCiMyIDB4ZmZmZmZmZmY4 MDQ5OTIzYiBhdCBWT1BfTE9DSzFfQVBWKzB4OWINCiMzIDB4ZmZmZmZmZmY4MDM1YWM3NyBh dCBfdm5fbG9jaysweDU3DQojNCAweGZmZmZmZmZmODAzNGZhZDAgYXQgc3luY192bm9kZSsw eDEzMA0KIzUgMHhmZmZmZmZmZjgwMzRmZDYyIGF0IHNjaGVkX3N5bmMrMHgxZDINCiM2IDB4 ZmZmZmZmZmY4MDI5M2M5YSBhdCBmb3JrX2V4aXQrMHgxMmENCiM3IDB4ZmZmZmZmZmY4MDQ0 OTUzZSBhdCBmb3JrX3RyYW1wb2xpbmUrMHhlDQoNCjB4ZmZmZmZmMDA2ZjYzMTRlMDogdGFn IHVmcywgdHlwZSBWUkVHDQogICAgdXNlY291bnQgMSwgd3JpdGVjb3VudCAwLCByZWZjb3Vu dCA0IG1vdW50ZWRoZXJlIDANCiAgICBmbGFncyAoKQ0KICAgIHZfb2JqZWN0IDB4ZmZmZmZm MDAwZTAzYTAwMCByZWYgMCBwYWdlcyAxNg0KICAgIGxvY2sgdHlwZSB1ZnM6IFNIQVJFRCAo Y291bnQgMSkNCiMwIDB4ZmZmZmZmZmY4MDJhNDg2MSBhdCBfX2xvY2ttZ3JfYXJncysweDUz MQ0KIzEgMHhmZmZmZmZmZjgwNDBiZGVjIGF0IGZmc19sb2NrKzB4OWMNCiMyIDB4ZmZmZmZm ZmY4MDQ5OTIzYiBhdCBWT1BfTE9DSzFfQVBWKzB4OWINCiMzIDB4ZmZmZmZmZmY4MDM1YWM3 NyBhdCBfdm5fbG9jaysweDU3DQojNCAweGZmZmZmZmZmODAzNWMyMjEgYXQgdm5fcmVhZCsw eDkxDQojNSAweGZmZmZmZmZmODAzMDNmMjEgYXQgZG9maWxlcmVhZCsweGExDQojNiAweGZm ZmZmZmZmODAzMDQwMDMgYXQga2Vybl9wcmVhZHYrMHg2Mw0KIzcgMHhmZmZmZmZmZjgwMzA0 MTI5IGF0IHByZWFkKzB4NTkNCiM4IDB4ZmZmZmZmZmY4MDJmYTVmMCBhdCBzeXNjYWxsZW50 ZXIrMHhmMA0KIzkgMHhmZmZmZmZmZjgwNDYxNDUwIGF0IHN5c2NhbGwrMHg2MA0KIzEwIDB4 ZmZmZmZmZmY4MDQ0OTNjMiBhdCBYZmFzdF9zeXNjYWxsKzB4ZTINCglpbm8gMTc4MzI4ODM1 LCBvbiBkZXYgdWZzL3N0b3JhZ2UNCg0KMHhmZmZmZmYwMDZmM2Y3YzMwOiB0YWcgdWZzLCB0 eXBlIFZSRUcNCiAgICB1c2Vjb3VudCAyLCB3cml0ZWNvdW50IDEsIHJlZmNvdW50IDIwMDEg bW91bnRlZGhlcmUgMA0KICAgIGZsYWdzICgpDQogICAgdl9vYmplY3QgMHhmZmZmZmYwMDZm OTkxNTEwIHJlZiAwIHBhZ2VzIDE1OTg0DQogICAgbG9jayB0eXBlIHVmczogRVhDTCBieSB0 aHJlYWQgMHhmZmZmZmYwMDA0YTk1NDYwIChwaWQgMTcpDQojMCAweGZmZmZmZmZmODAyYTRh YzQgYXQgX19sb2NrbWdyX2FyZ3MrMHg3OTQNCiMxIDB4ZmZmZmZmZmY4MDQwYmRlYyBhdCBm ZnNfbG9jaysweDljDQojMiAweGZmZmZmZmZmODA0OTkyM2IgYXQgVk9QX0xPQ0sxX0FQVisw eDliDQojMyAweGZmZmZmZmZmODAzNWFjNzcgYXQgX3ZuX2xvY2srMHg1Nw0KIzQgMHhmZmZm ZmZmZjgwMzRmNjJiIGF0IHZnZXQrMHg3Yg0KIzUgMHhmZmZmZmZmZjgwNDA3ZGZjIGF0IGZm c19zeW5jKzB4MTVjDQojNiAweGZmZmZmZmZmODAzNTE2ZmEgYXQgc3luY19mc3luYysweDEz YQ0KIzcgMHhmZmZmZmZmZjgwNDk3NTM1IGF0IFZPUF9GU1lOQ19BUFYrMHhiNQ0KIzggMHhm ZmZmZmZmZjgwMzRmYWY3IGF0IHN5bmNfdm5vZGUrMHgxNTcNCiM5IDB4ZmZmZmZmZmY4MDM0 ZmQ2MiBhdCBzY2hlZF9zeW5jKzB4MWQyDQojMTAgMHhmZmZmZmZmZjgwMjkzYzlhIGF0IGZv cmtfZXhpdCsweDEyYQ0KIzExIDB4ZmZmZmZmZmY4MDQ0OTUzZSBhdCBmb3JrX3RyYW1wb2xp bmUrMHhlDQoJaW5vIDE3MTU4NzMyOCwgb24gZGV2IHVmcy9zdG9yYWdlDQoNCjB4ZmZmZmZm MDA1NTNkNDc1MDogdGFnIHVmcywgdHlwZSBWUkVHDQogICAgdXNlY291bnQgMSwgd3JpdGVj b3VudCAwLCByZWZjb3VudCA0MDAxIG1vdW50ZWRoZXJlIDANCiAgICBmbGFncyAoKQ0KICAg IHZfb2JqZWN0IDB4ZmZmZmZmMDAxMWRhODM2MCByZWYgMCBwYWdlcyAzMTk5Mg0KICAgIGxv Y2sgdHlwZSB1ZnM6IFNIQVJFRCAoY291bnQgMSkNCiMwIDB4ZmZmZmZmZmY4MDJhNDg2MSBh dCBfX2xvY2ttZ3JfYXJncysweDUzMQ0KIzEgMHhmZmZmZmZmZjgwNDBiZGVjIGF0IGZmc19s b2NrKzB4OWMNCiMyIDB4ZmZmZmZmZmY4MDQ5OTIzYiBhdCBWT1BfTE9DSzFfQVBWKzB4OWIN CiMzIDB4ZmZmZmZmZmY4MDM1YWM3NyBhdCBfdm5fbG9jaysweDU3DQojNCAweGZmZmZmZmZm ODAzNWMyMjEgYXQgdm5fcmVhZCsweDkxDQojNSAweGZmZmZmZmZmODAzMDNmMjEgYXQgZG9m aWxlcmVhZCsweGExDQojNiAweGZmZmZmZmZmODAzMDQwMDMgYXQga2Vybl9wcmVhZHYrMHg2 Mw0KIzcgMHhmZmZmZmZmZjgwMzA0MTI5IGF0IHByZWFkKzB4NTkNCiM4IDB4ZmZmZmZmZmY4 MDJmYTVmMCBhdCBzeXNjYWxsZW50ZXIrMHhmMA0KIzkgMHhmZmZmZmZmZjgwNDYxNDUwIGF0 IHN5c2NhbGwrMHg2MA0KIzEwIDB4ZmZmZmZmZmY4MDQ0OTNjMiBhdCBYZmFzdF9zeXNjYWxs KzB4ZTINCglpbm8gMTczOTE0NzgxLCBvbiBkZXYgdWZzL3N0b3JhZ2UNCmRiPiBzaAggCAgg CGFsbHRyYWNlDQoNClRyYWNpbmcgY29tbWFuZCBjc2ggcGlkIDEyMDkgdGlkIDEwMDA4MSB0 ZCAweGZmZmZmZjAwMDRhOTQwMDANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRjaCsw eDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpzbGVlcHFfc3dpdGNoKCkg YXQgc2xlZXBxX3N3aXRjaCsweDEyMw0Kc2xlZXBxX2NhdGNoX3NpZ25hbHMoKSBhdCBzbGVl cHFfY2F0Y2hfc2lnbmFscysweDJhNg0Kc2xlZXBxX3dhaXRfc2lnKCkgYXQgc2xlZXBxX3dh aXRfc2lnKzB4MTYNCl9jdl93YWl0X3NpZygpIGF0IF9jdl93YWl0X3NpZysweDE3ZQ0KdHR5 X3dhaXQoKSBhdCB0dHlfd2FpdCsweDQ4DQp0dHlkaXNjX3JlYWQoKSBhdCB0dHlkaXNjX3Jl YWQrMHgyZjENCnR0eWRldl9yZWFkKCkgYXQgdHR5ZGV2X3JlYWQrMHhhYg0KZGV2ZnNfcmVh ZF9mKCkgYXQgZGV2ZnNfcmVhZF9mKzB4ODgNCmRvZmlsZXJlYWQoKSBhdCBkb2ZpbGVyZWFk KzB4YTENCmtlcm5fcmVhZHYoKSBhdCBrZXJuX3JlYWR2KzB4NjANCnJlYWQoKSBhdCByZWFk KzB4NTUNCnN5c2NhbGxlbnRlcigpIGF0IHN5c2NhbGxlbnRlcisweGYwDQpzeXNjYWxsKCkg YXQgc3lzY2FsbCsweDYwDQpYZmFzdF9zeXNjYWxsKCkgYXQgWGZhc3Rfc3lzY2FsbCsweGUy DQotLS0gc3lzY2FsbCAoMywgRnJlZUJTRCBFTEY2NCwgcmVhZCksIHJpcCA9IDB4ODAwOWVm ZmVjLCByc3AgPSAweDdmZmZmZmZmZTUzOCwgcmJwID0gMHgxIC0tLQ0KDQpUcmFjaW5nIGNv bW1hbmQgc3UgcGlkIDEyMDcgdGlkIDEwMDEwNSB0ZCAweGZmZmZmZjAwMDRiZWM0NjANCg0K VHJhY2luZyBjb21tYW5kIHNtYmQgcGlkIDEyMDAgdGlkIDEwMDExMyB0ZCAweGZmZmZmZjAw MDRmNzUwMDANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dp dGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpzbGVlcHFfc3dpdGNoKCkgYXQgc2xlZXBxX3N3 aXRjaCsweDEyMw0Kc2xlZXBxX3dhaXQoKSBhdCBzbGVlcHFfd2FpdCsweDRkDQpfc2xlZXAo KSBhdCBfc2xlZXArMHgzNjkNCnZtX2ZhdWx0KCkgYXQgdm1fZmF1bHQrMHgxNWJjDQp0cmFw X3BmYXVsdCgpIGF0IHRyYXBfcGZhdWx0KzB4MTI4DQp0cmFwKCkgYXQgdHJhcCsweDM2Yw0K Y2FsbHRyYXAoKSBhdCBjYWxsdHJhcCsweDgNCi0tLSB0cmFwIDB4YywgcmlwID0gMHhmZmZm ZmZmZjgwNDVmNWJiLCByc3AgPSAweGZmZmZmZjgwOTZjN2I4NTAsIHJicCA9IDB4ZmZmZmZm ODA5NmM3YmE5MCAtLS0NCmNvcHlvdXQoKSBhdCBjb3B5b3V0KzB4M2INCnNlbGVjdCgpIGF0 IHNlbGVjdCsweDVkDQpzeXNjYWxsZW50ZXIoKSBhdCBzeXNjYWxsZW50ZXIrMHhmMA0Kc3lz Y2FsbCgpIGF0IHN5c2NhbGwrMHg2MA0KWGZhc3Rfc3lzY2FsbCgpIGF0IFhmYXN0X3N5c2Nh bGwrMHhlMg0KLS0tIHN5c2NhbGwgKDkzLCBGcmVlQlNEIEVMRjY0LCBzZWxlY3QpLCByaXAg PSAweDgwMjY3NmY2YywgcnNwID0gMHg3ZmZmZmZmZmU0MTgsIHJicCA9IDB4OCAtLS0NCg0K VHJhY2luZyBjb21tYW5kIHRyYW5zbWlzc2lvbi1kYWVtb24gcGlkIDExOTEgdGlkIDEwMDE1 MSB0ZCAweGZmZmZmZjAwMDRhZmY4YzANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRj aCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpzbGVlcHFfc3dpdGNo KCkgYXQgc2xlZXBxX3N3aXRjaCsweDEyMw0Kc2xlZXBxX3dhaXQoKSBhdCBzbGVlcHFfd2Fp dCsweDRkDQpfX2xvY2ttZ3JfYXJncygpIGF0IF9fbG9ja21ncl9hcmdzKzB4N2FlDQpnZXRi bGsoKSBhdCBnZXRibGsrMHgxMDENCmNsdXN0ZXJfcmVhZCgpIGF0IGNsdXN0ZXJfcmVhZCsw eGM1DQpmZnNfcmVhZCgpIGF0IGZmc19yZWFkKzB4Mzg4DQpWT1BfUkVBRF9BUFYoKSBhdCBW T1BfUkVBRF9BUFYrMHhhZg0Kdm5fcmVhZCgpIGF0IHZuX3JlYWQrMHhkMA0KZG9maWxlcmVh ZCgpIGF0IGRvZmlsZXJlYWQrMHhhMQ0Ka2Vybl9wcmVhZHYoKSBhdCBrZXJuX3ByZWFkdisw eDYzDQpwcmVhZCgpIGF0IHByZWFkKzB4NTkNCnN5c2NhbGxlbnRlcigpIGF0IHN5c2NhbGxl bnRlcisweGYwDQpzeXNjYWxsKCkgYXQgc3lzY2FsbCsweDYwDQpYZmFzdF9zeXNjYWxsKCkg YXQgWGZhc3Rfc3lzY2FsbCsweGUyDQotLS0gc3lzY2FsbCAoNDc1LCBGcmVlQlNEIEVMRjY0 LCBwcmVhZCksIHJpcCA9IDB4ODAxMTNhZjZjLCByc3AgPSAweDdmZmZmZjVmYmU3OCwgcmJw ID0gMHg3M2RiNzNkIC0tLQ0KDQpUcmFjaW5nIGNvbW1hbmQgdHJhbnNtaXNzaW9uLWRhZW1v biBwaWQgMTE5MSB0aWQgMTAwMTQ5IHRkIDB4ZmZmZmZmMDAxMTIyNTAwMA0Kc2NoZWRfc3dp dGNoKCkgYXQgc2NoZWRfc3dpdGNoKzB4MTM4DQptaV9zd2l0Y2goKSBhdCBtaV9zd2l0Y2gr MHgyMWINCnNsZWVwcV9zd2l0Y2goKSBhdCBzbGVlcHFfc3dpdGNoKzB4MTIzDQpzbGVlcHFf Y2F0Y2hfc2lnbmFscygpIGF0IHNsZWVwcV9jYXRjaF9zaWduYWxzKzB4MmE2DQpzbGVlcHFf dGltZWR3YWl0X3NpZygpIGF0IHNsZWVwcV90aW1lZHdhaXRfc2lnKzB4MTkNCl9jdl90aW1l ZHdhaXRfc2lnKCkgYXQgX2N2X3RpbWVkd2FpdF9zaWcrMHgxOGMNCnNlbHRkd2FpdCgpIGF0 IHNlbHRkd2FpdCsweDU2DQprZXJuX3NlbGVjdCgpIGF0IGtlcm5fc2VsZWN0KzB4NjQwDQpz ZWxlY3QoKSBhdCBzZWxlY3QrMHg1ZA0Kc3lzY2FsbGVudGVyKCkgYXQgc3lzY2FsbGVudGVy KzB4ZjANCnN5c2NhbGwoKSBhdCBzeXNjYWxsKzB4NjANClhmYXN0X3N5c2NhbGwoKSBhdCBY ZmFzdF9zeXNjYWxsKzB4ZTINCi0tLSBzeXNjYWxsICg5MywgRnJlZUJTRCBFTEY2NCwgc2Vs ZWN0KSwgcmlwID0gMHg4MDExM2NmNmMsIHJzcCA9IDB4N2ZmZmZmOWZkZDU4LCByYnAgPSAw IC0tLQ0KDQpUcmFjaW5nIGNvbW1hbmQgdHJhbnNtaXNzaW9uLWRhZW1vbiBwaWQgMTE5MSB0 aWQgMTAwMTQ4IHRkIDB4ZmZmZmZmMDAxMTIyNTQ2MA0Kc2NoZWRfc3dpdGNoKCkgYXQgc2No ZWRfc3dpdGNoKzB4MTM4DQptaV9zd2l0Y2goKSBhdCBtaV9zd2l0Y2grMHgyMWINCnNsZWVw cV9zd2l0Y2goKSBhdCBzbGVlcHFfc3dpdGNoKzB4MTIzDQpzbGVlcHFfd2FpdCgpIGF0IHNs ZWVwcV93YWl0KzB4NGQNCl9zbGVlcCgpIGF0IF9zbGVlcCsweDM2OQ0KYndhaXQoKSBhdCBi d2FpdCsweDY0DQpidWZ3YWl0KCkgYXQgYnVmd2FpdCsweDU2DQpjbHVzdGVyX3JlYWQoKSBh dCBjbHVzdGVyX3JlYWQrMHg2MDkNCmZmc19yZWFkKCkgYXQgZmZzX3JlYWQrMHgzODgNClZP UF9SRUFEX0FQVigpIGF0IFZPUF9SRUFEX0FQVisweGFmDQp2bl9yZWFkKCkgYXQgdm5fcmVh ZCsweGQwDQpkb2ZpbGVyZWFkKCkgYXQgZG9maWxlcmVhZCsweGExDQprZXJuX3ByZWFkdigp IGF0IGtlcm5fcHJlYWR2KzB4NjMNCnByZWFkKCkgYXQgcHJlYWQrMHg1OQ0Kc3lzY2FsbGVu dGVyKCkgYXQgc3lzY2FsbGVudGVyKzB4ZjANCnN5c2NhbGwoKSBhdCBzeXNjYWxsKzB4NjAN ClhmYXN0X3N5c2NhbGwoKSBhdCBYZmFzdF9zeXNjYWxsKzB4ZTINCi0tLSBzeXNjYWxsICg0 NzUsIEZyZWVCU0QgRUxGNjQsIHByZWFkKSwgcmlwID0gMHg4MDExM2FmNmMsIHJzcCA9IDB4 N2ZmZmZmYmZlYzU4LCByYnAgPSAweDgwMjYzZjcwOCAtLS0NCg0KVHJhY2luZyBjb21tYW5k IHRyYW5zbWlzc2lvbi1kYWVtb24gcGlkIDExOTEgdGlkIDEwMDA5NiB0ZCAweGZmZmZmZjAw MDRiMDM4YzANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dp dGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpzbGVlcHFfc3dpdGNoKCkgYXQgc2xlZXBxX3N3 aXRjaCsweDEyMw0Kc2xlZXBxX3dhaXQoKSBhdCBzbGVlcHFfd2FpdCsweDRkDQpfc2xlZXAo KSBhdCBfc2xlZXArMHgzNjkNCnZtX2ZhdWx0KCkgYXQgdm1fZmF1bHQrMHgxNWJjDQp0cmFw X3BmYXVsdCgpIGF0IHRyYXBfcGZhdWx0KzB4MTI4DQp0cmFwKCkgYXQgdHJhcCsweDM2Yw0K Y2FsbHRyYXAoKSBhdCBjYWxsdHJhcCsweDgNCi0tLSB0cmFwIDB4YywgcmlwID0gMHhmZmZm ZmZmZjgwNDVmOTkwLCByc3AgPSAweGZmZmZmZjgwOTZjMjY4MDAsIHJicCA9IDB4ZmZmZmZm ODA5NmMyNjhjMCAtLS0NCmNvcHlpbnN0cigpIGF0IGNvcHlpbnN0cisweDQwDQprZXJuX3N0 YXRhdF92bmhvb2soKSBhdCBrZXJuX3N0YXRhdF92bmhvb2srMHg4Zg0Ka2Vybl9zdGF0YXQo KSBhdCBrZXJuX3N0YXRhdCsweDE1DQpzdGF0KCkgYXQgc3RhdCsweDJhDQpzeXNjYWxsZW50 ZXIoKSBhdCBzeXNjYWxsZW50ZXIrMHhmMA0Kc3lzY2FsbCgpIGF0IHN5c2NhbGwrMHg2MA0K WGZhc3Rfc3lzY2FsbCgpIGF0IFhmYXN0X3N5c2NhbGwrMHhlMg0KLS0tIHN5c2NhbGwgKDE4 OCwgRnJlZUJTRCBFTEY2NCwgc3RhdCksIHJpcCA9IDB4ODAxMTJkZDBjLCByc3AgPSAweDdm ZmZmZmZmZTcyOCwgcmJwID0gMHg4MDE0YWIzNzAgLS0tDQoNClRyYWNpbmcgY29tbWFuZCB0 Y3NoIHBpZCAxMTU5IHRpZCAxMDAwOTAgdGQgMHhmZmZmZmYwMDA0YmYzOGMwDQoNClRyYWNp bmcgY29tbWFuZCBzc2hkIHBpZCAxMTU4IHRpZCAxMDAxMTAgdGQgMHhmZmZmZmYwMDA0Zjc3 MDAwDQpzY2hlZF9zd2l0Y2goKSBhdCBzY2hlZF9zd2l0Y2grMHgxMzgNCm1pX3N3aXRjaCgp IGF0IG1pX3N3aXRjaCsweDIxYg0Kc2xlZXBxX3N3aXRjaCgpIGF0IHNsZWVwcV9zd2l0Y2gr MHgxMjMNCnNsZWVwcV93YWl0KCkgYXQgc2xlZXBxX3dhaXQrMHg0ZA0KX3NsZWVwKCkgYXQg X3NsZWVwKzB4MzY5DQp2bV9mYXVsdCgpIGF0IHZtX2ZhdWx0KzB4MTViYw0KdHJhcF9wZmF1 bHQoKSBhdCB0cmFwX3BmYXVsdCsweDEyOA0KdHJhcCgpIGF0IHRyYXArMHgzNmMNCmNhbGx0 cmFwKCkgYXQgY2FsbHRyYXArMHg4DQotLS0gdHJhcCAweGMsIHJpcCA9IDB4ZmZmZmZmZmY4 MDQ1ZjViYiwgcnNwID0gMHhmZmZmZmY4MDk2YzZjODUwLCByYnAgPSAweGZmZmZmZjgwOTZj NmNhOTAgLS0tDQpjb3B5b3V0KCkgYXQgY29weW91dCsweDNiDQpzZWxlY3QoKSBhdCBzZWxl Y3QrMHg1ZA0Kc3lzY2FsbGVudGVyKCkgYXQgc3lzY2FsbGVudGVyKzB4ZjANCnN5c2NhbGwo KSBhdCBzeXNjYWxsKzB4NjANClhmYXN0X3N5c2NhbGwoKSBhdCBYZmFzdF9zeXNjYWxsKzB4 ZTINCi0tLSBzeXNjYWxsICg5MywgRnJlZUJTRCBFTEY2NCwgc2VsZWN0KSwgcmlwID0gMHg4 MDE2ZGNmNmMsIHJzcCA9IDB4N2ZmZmZmZmZlMDI4LCByYnAgPSAweDdmZmZmZmZmZTBiMCAt LS0NCg0KVHJhY2luZyBjb21tYW5kIHNzaGQgcGlkIDExNTYgdGlkIDEwMDEwOCB0ZCAweGZm ZmZmZjAwMDRiZTk0NjANCg0KVHJhY2luZyBjb21tYW5kIGdldHR5IHBpZCAxMTUzIHRpZCAx MDAxMjQgdGQgMHhmZmZmZmYwMDA0ZjQwNDYwDQpzY2hlZF9zd2l0Y2goKSBhdCBzY2hlZF9z d2l0Y2grMHgxMzgNCm1pX3N3aXRjaCgpIGF0IG1pX3N3aXRjaCsweDIxYg0Kc2xlZXBxX3N3 aXRjaCgpIGF0IHNsZWVwcV9zd2l0Y2grMHgxMjMNCnNsZWVwcV93YWl0KCkgYXQgc2xlZXBx X3dhaXQrMHg0ZA0KX3NsZWVwKCkgYXQgX3NsZWVwKzB4MzY5DQp2bV9mYXVsdCgpIGF0IHZt X2ZhdWx0KzB4MTViYw0KdHJhcF9wZmF1bHQoKSBhdCB0cmFwX3BmYXVsdCsweDEyOA0KdHJh cCgpIGF0IHRyYXArMHgzNmMNCmNhbGx0cmFwKCkgYXQgY2FsbHRyYXArMHg4DQotLS0gdHJh cCAweGMsIHJpcCA9IDB4ZmZmZmZmZmY4MDQ1ZjVjMywgcnNwID0gMHhmZmZmZmY4MDk2Y2Iy NzUwLCByYnAgPSAweGZmZmZmZjgwOTZjYjI3ZDAgLS0tDQpjb3B5b3V0KCkgYXQgY29weW91 dCsweDQzDQp0dHlpbnFfcmVhZF91aW8oKSBhdCB0dHlpbnFfcmVhZF91aW8rMHgyNWQNCnR0 eWRpc2NfcmVhZCgpIGF0IHR0eWRpc2NfcmVhZCsweDJhYw0KdHR5ZGV2X3JlYWQoKSBhdCB0 dHlkZXZfcmVhZCsweGFiDQpkZXZmc19yZWFkX2YoKSBhdCBkZXZmc19yZWFkX2YrMHg4OA0K ZG9maWxlcmVhZCgpIGF0IGRvZmlsZXJlYWQrMHhhMQ0Ka2Vybl9yZWFkdigpIGF0IGtlcm5f cmVhZHYrMHg2MA0KcmVhZCgpIGF0IHJlYWQrMHg1NQ0Kc3lzY2FsbGVudGVyKCkgYXQgc3lz Y2FsbGVudGVyKzB4ZjANCnN5c2NhbGwoKSBhdCBzeXNjYWxsKzB4NjANClhmYXN0X3N5c2Nh bGwoKSBhdCBYZmFzdF9zeXNjYWxsKzB4ZTINCi0tLSBzeXNjYWxsICgzLCBGcmVlQlNEIEVM RjY0LCByZWFkKSwgcmlwID0gMHg4MDA4NGZmZWMsIHJzcCA9IDB4N2ZmZmZmZmZlY2M4LCBy YnAgPSAwIC0tLQ0KDQpUcmFjaW5nIGNvbW1hbmQgZ2V0dHkgcGlkIDExNTIgdGlkIDEwMDEz NCB0ZCAweGZmZmZmZjAwMTExMTY4YzANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRj aCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpzbGVlcHFfc3dpdGNo KCkgYXQgc2xlZXBxX3N3aXRjaCsweDEyMw0Kc2xlZXBxX3dhaXQoKSBhdCBzbGVlcHFfd2Fp dCsweDRkDQpfc2xlZXAoKSBhdCBfc2xlZXArMHgzNjkNCnZtX2ZhdWx0KCkgYXQgdm1fZmF1 bHQrMHgxNWJjDQp0cmFwX3BmYXVsdCgpIGF0IHRyYXBfcGZhdWx0KzB4MTI4DQp0cmFwKCkg YXQgdHJhcCsweDM2Yw0KY2FsbHRyYXAoKSBhdCBjYWxsdHJhcCsweDgNCi0tLSB0cmFwIDB4 YywgcmlwID0gMHhmZmZmZmZmZjgwNDVmNWMzLCByc3AgPSAweGZmZmZmZjgwOTZjZTQ3NTAs IHJicCA9IDB4ZmZmZmZmODA5NmNlNDdkMCAtLS0NCmNvcHlvdXQoKSBhdCBjb3B5b3V0KzB4 NDMNCnR0eWlucV9yZWFkX3VpbygpIGF0IHR0eWlucV9yZWFkX3VpbysweDI1ZA0KdHR5ZGlz Y19yZWFkKCkgYXQgdHR5ZGlzY19yZWFkKzB4MmFjDQp0dHlkZXZfcmVhZCgpIGF0IHR0eWRl dl9yZWFkKzB4YWINCmRldmZzX3JlYWRfZigpIGF0IGRldmZzX3JlYWRfZisweDg4DQpkb2Zp bGVyZWFkKCkgYXQgZG9maWxlcmVhZCsweGExDQprZXJuX3JlYWR2KCkgYXQga2Vybl9yZWFk disweDYwDQpyZWFkKCkgYXQgcmVhZCsweDU1DQpzeXNjYWxsZW50ZXIoKSBhdCBzeXNjYWxs ZW50ZXIrMHhmMA0Kc3lzY2FsbCgpIGF0IHN5c2NhbGwrMHg2MA0KWGZhc3Rfc3lzY2FsbCgp IGF0IFhmYXN0X3N5c2NhbGwrMHhlMg0KLS0tIHN5c2NhbGwgKDMsIEZyZWVCU0QgRUxGNjQs IHJlYWQpLCByaXAgPSAweDgwMDg0ZmZlYywgcnNwID0gMHg3ZmZmZmZmZmVjYzgsIHJicCA9 IDAgLS0tDQoNClRyYWNpbmcgY29tbWFuZCBjcm9uIHBpZCAxMDUyIHRpZCAxMDAxMDQgdGQg MHhmZmZmZmYwMDA0YmVjOGMwDQoNClRyYWNpbmcgY29tbWFuZCBzZW5kbWFpbCBwaWQgMTA0 NSB0aWQgMTAwMTQ1IHRkIDB4ZmZmZmZmMDAxMTIyNjQ2MA0KDQpUcmFjaW5nIGNvbW1hbmQg c2VuZG1haWwgcGlkIDEwNDEgdGlkIDEwMDEyMyB0ZCAweGZmZmZmZjAwMDRmNDA4YzANCnNj aGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlf c3dpdGNoKzB4MjFiDQpzbGVlcHFfc3dpdGNoKCkgYXQgc2xlZXBxX3N3aXRjaCsweDEyMw0K c2xlZXBxX3dhaXQoKSBhdCBzbGVlcHFfd2FpdCsweDRkDQpfc2xlZXAoKSBhdCBfc2xlZXAr MHgzNjkNCnZtX2ZhdWx0KCkgYXQgdm1fZmF1bHQrMHgxNWJjDQp0cmFwX3BmYXVsdCgpIGF0 IHRyYXBfcGZhdWx0KzB4MTI4DQp0cmFwKCkgYXQgdHJhcCsweDM2Yw0KY2FsbHRyYXAoKSBh dCBjYWxsdHJhcCsweDgNCi0tLSB0cmFwIDB4YywgcmlwID0gMHhmZmZmZmZmZjgwNDVmNWJi LCByc3AgPSAweGZmZmZmZjgwOTZjYWQ4NTAsIHJicCA9IDB4ZmZmZmZmODA5NmNhZGE5MCAt LS0NCmNvcHlvdXQoKSBhdCBjb3B5b3V0KzB4M2INCnNlbGVjdCgpIGF0IHNlbGVjdCsweDVk DQpzeXNjYWxsZW50ZXIoKSBhdCBzeXNjYWxsZW50ZXIrMHhmMA0Kc3lzY2FsbCgpIGF0IHN5 c2NhbGwrMHg2MA0KWGZhc3Rfc3lzY2FsbCgpIGF0IFhmYXN0X3N5c2NhbGwrMHhlMg0KLS0t IHN5c2NhbGwgKDkzLCBGcmVlQlNEIEVMRjY0LCBzZWxlY3QpLCByaXAgPSAweDgwMGRlNWY2 YywgcnNwID0gMHg3ZmZmZmZmZmMxZTgsIHJicCA9IDB4N2ZmZmZmZmZjMjgwIC0tLQ0KDQpU cmFjaW5nIGNvbW1hbmQgc3NoZCBwaWQgMTAzMyB0aWQgMTAwMTM5IHRkIDB4ZmZmZmZmMDAx MTE2ZjhjMA0Kc2NoZWRfc3dpdGNoKCkgYXQgc2NoZWRfc3dpdGNoKzB4MTM4DQptaV9zd2l0 Y2goKSBhdCBtaV9zd2l0Y2grMHgyMWINCnNsZWVwcV9zd2l0Y2goKSBhdCBzbGVlcHFfc3dp dGNoKzB4MTIzDQpzbGVlcHFfY2F0Y2hfc2lnbmFscygpIGF0IHNsZWVwcV9jYXRjaF9zaWdu YWxzKzB4MmE2DQpzbGVlcHFfd2FpdF9zaWcoKSBhdCBzbGVlcHFfd2FpdF9zaWcrMHgxNg0K X2N2X3dhaXRfc2lnKCkgYXQgX2N2X3dhaXRfc2lnKzB4MTdlDQpzZWx0ZHdhaXQoKSBhdCBz ZWx0ZHdhaXQrMHhhYw0Ka2Vybl9zZWxlY3QoKSBhdCBrZXJuX3NlbGVjdCsweDY0MA0Kc2Vs ZWN0KCkgYXQgc2VsZWN0KzB4NWQNCnN5c2NhbGxlbnRlcigpIGF0IHN5c2NhbGxlbnRlcisw eGYwDQpzeXNjYWxsKCkgYXQgc3lzY2FsbCsweDYwDQpYZmFzdF9zeXNjYWxsKCkgYXQgWGZh c3Rfc3lzY2FsbCsweGUyDQotLS0gc3lzY2FsbCAoOTMsIEZyZWVCU0QgRUxGNjQsIHNlbGVj dCksIHJpcCA9IDB4ODAxNmRjZjZjLCByc3AgPSAweDdmZmZmZmZmZTE0OCwgcmJwID0gMHhh IC0tLQ0KVHJhY2luZyBjb21tYW5kIGN2c3VwZCBwaWQgOTYyIHRpZCAxMDAwOTggdGQgMHhm ZmZmZmYwMDA0YjAzMDAwDQpzY2hlZF9zd2l0Y2goKSBhdCBzY2hlZF9zd2l0Y2grMHgxMzgN Cm1pX3N3aXRjaCgpIGF0IG1pX3N3aXRjaCsweDIxYg0Kc2xlZXBxX3N3aXRjaCgpIGF0IHNs ZWVwcV9zd2l0Y2grMHgxMjMNCnNsZWVwcV9jYXRjaF9zaWduYWxzKCkgYXQgc2xlZXBxX2Nh dGNoX3NpZ25hbHMrMHgyYTYNCnNsZWVwcV93YWl0X3NpZygpIGF0IHNsZWVwcV93YWl0X3Np ZysweDE2DQpfY3Zfd2FpdF9zaWcoKSBhdCBfY3Zfd2FpdF9zaWcrMHgxN2UNCnNlbHRkd2Fp dCgpIGF0IHNlbHRkd2FpdCsweGFjDQprZXJuX3NlbGVjdCgpIGF0IGtlcm5fc2VsZWN0KzB4 NjQwDQpzZWxlY3QoKSBhdCBzZWxlY3QrMHg1ZA0Kc3lzY2FsbGVudGVyKCkgYXQgc3lzY2Fs bGVudGVyKzB4ZjANCnN5c2NhbGwoKSBhdCBzeXNjYWxsKzB4NjANClhmYXN0X3N5c2NhbGwo KSBhdCBYZmFzdF9zeXNjYWxsKzB4ZTINCi0tLSBzeXNjYWxsICg5MywgRnJlZUJTRCBFTEY2 NCwgc2VsZWN0KSwgcmlwID0gMHg4MDBhMmZmNmMsIHJzcCA9IDB4NmIyOGIwLCByYnAgPSAw eDZiMjkzOCAtLS0NCg0KVHJhY2luZyBjb21tYW5kIG50cGQgcGlkIDg3MiB0aWQgMTAwMTI3 IHRkIDB4ZmZmZmZmMDAxMTEyMTAwMA0Kc2NoZWRfc3dpdGNoKCkgYXQgc2NoZWRfc3dpdGNo KzB4MTM4DQptaV9zd2l0Y2goKSBhdCBtaV9zd2l0Y2grMHgyMWINCnNsZWVwcV9zd2l0Y2go KSBhdCBzbGVlcHFfc3dpdGNoKzB4MTIzDQpzbGVlcHFfd2FpdCgpIGF0IHNsZWVwcV93YWl0 KzB4NGQNCl9zbGVlcCgpIGF0IF9zbGVlcCsweDM2OQ0Kdm1fZmF1bHQoKSBhdCB2bV9mYXVs dCsweDE1YmMNCnRyYXBfcGZhdWx0KCkgYXQgdHJhcF9wZmF1bHQrMHgxMjgNCnRyYXAoKSBh dCB0cmFwKzB4NTAxDQpjYWxsdHJhcCgpIGF0IGNhbGx0cmFwKzB4OA0KLS0tIHRyYXAgMHhj LCByaXAgPSAweDQzNzJlMCwgcnNwID0gMHg3ZmZmZmZmZmVhZTgsIHJicCA9IDAgLS0tDQoN ClRyYWNpbmcgY29tbWFuZCBzbWJkIHBpZCA4NTYgdGlkIDEwMDA5NSB0ZCAweGZmZmZmZjAw MDRiMDQwMDANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dp dGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpzbGVlcHFfc3dpdGNoKCkgYXQgc2xlZXBxX3N3 aXRjaCsweDEyMw0Kc2xlZXBxX2NhdGNoX3NpZ25hbHMoKSBhdCBzbGVlcHFfY2F0Y2hfc2ln bmFscysweDJhNg0Kc2xlZXBxX3RpbWVkd2FpdF9zaWcoKSBhdCBzbGVlcHFfdGltZWR3YWl0 X3NpZysweDE5DQpfY3ZfdGltZWR3YWl0X3NpZygpIGF0IF9jdl90aW1lZHdhaXRfc2lnKzB4 MThjDQpzZWx0ZHdhaXQoKSBhdCBzZWx0ZHdhaXQrMHg1Ng0Ka2Vybl9zZWxlY3QoKSBhdCBr ZXJuX3NlbGVjdCsweDY0MA0Kc2VsZWN0KCkgYXQgc2VsZWN0KzB4NWQNCnN5c2NhbGxlbnRl cigpIGF0IHN5c2NhbGxlbnRlcisweGYwDQpzeXNjYWxsKCkgYXQgc3lzY2FsbCsweDYwDQpY ZmFzdF9zeXNjYWxsKCkgYXQgWGZhc3Rfc3lzY2FsbCsweGUyDQotLS0gc3lzY2FsbCAoOTMs IEZyZWVCU0QgRUxGNjQsIHNlbGVjdCksIHJpcCA9IDB4ODAyNjc2ZjZjLCByc3AgPSAweDdm ZmZmZmZmZTZjOCwgcmJwID0gMHgxOCAtLS0NCg0KVHJhY2luZyBjb21tYW5kIHNtYmQgcGlk IDgwNCB0aWQgMTAwMDg2IHRkIDB4ZmZmZmZmMDAwNGIwNTAwMA0Kc2NoZWRfc3dpdGNoKCkg YXQgc2NoZWRfc3dpdGNoKzB4MTM4DQptaV9zd2l0Y2goKSBhdCBtaV9zd2l0Y2grMHgyMWIN CnNsZWVwcV9zd2l0Y2goKSBhdCBzbGVlcHFfc3dpdGNoKzB4MTIzDQpzbGVlcHFfd2FpdCgp IGF0IHNsZWVwcV93YWl0KzB4NGQNCl9zbGVlcCgpIGF0IF9zbGVlcCsweDM2OQ0Kdm1fZmF1 bHQoKSBhdCB2bV9mYXVsdCsweDE1YmMNCnRyYXBfcGZhdWx0KCkgYXQgdHJhcF9wZmF1bHQr MHgxMjgNCnRyYXAoKSBhdCB0cmFwKzB4MzZjDQpjYWxsdHJhcCgpIGF0IGNhbGx0cmFwKzB4 OA0KLS0tIHRyYXAgMHhjLCByaXAgPSAweGZmZmZmZmZmODA0NWY1YmIsIHJzcCA9IDB4ZmZm ZmZmODA5NmJmNDg1MCwgcmJwID0gMHhmZmZmZmY4MDk2YmY0YTkwIC0tLQ0KY29weW91dCgp IGF0IGNvcHlvdXQrMHgzYg0Kc2VsZWN0KCkgYXQgc2VsZWN0KzB4NWQNCnN5c2NhbGxlbnRl cigpIGF0IHN5c2NhbGxlbnRlcisweGYwDQpzeXNjYWxsKCkgYXQgc3lzY2FsbCsweDYwDQpY ZmFzdF9zeXNjYWxsKCkgYXQgWGZhc3Rfc3lzY2FsbCsweGUyDQotLS0gc3lzY2FsbCAoOTMs IEZyZWVCU0QgRUxGNjQsIHNlbGVjdCksIHJpcCA9IDB4ODAyNjc2ZjZjLCByc3AgPSAweDdm ZmZmZmZmZTcyOCwgcmJwID0gMHgxYiAtLS0NClRyYWNpbmcgY29tbWFuZCBubWJkIHBpZCA4 MDAgdGlkIDEwMDA5NCB0ZCAweGZmZmZmZjAwMDRiMDQ0NjANCnNjaGVkX3N3aXRjaCgpIGF0 IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpz bGVlcHFfc3dpdGNoKCkgYXQgc2xlZXBxX3N3aXRjaCsweDEyMw0Kc2xlZXBxX3dhaXQoKSBh dCBzbGVlcHFfd2FpdCsweDRkDQpfc2xlZXAoKSBhdCBfc2xlZXArMHgzNjkNCnZtX2ZhdWx0 KCkgYXQgdm1fZmF1bHQrMHgxNWJjDQp0cmFwX3BmYXVsdCgpIGF0IHRyYXBfcGZhdWx0KzB4 MTI4DQp0cmFwKCkgYXQgdHJhcCsweDM2Yw0KY2FsbHRyYXAoKSBhdCBjYWxsdHJhcCsweDgN Ci0tLSB0cmFwIDB4YywgcmlwID0gMHhmZmZmZmZmZjgwNDVmNWJiLCByc3AgPSAweGZmZmZm ZjgwOTZjMWM4NTAsIHJicCA9IDB4ZmZmZmZmODA5NmMxY2E5MCAtLS0NCmNvcHlvdXQoKSBh dCBjb3B5b3V0KzB4M2INCnNlbGVjdCgpIGF0IHNlbGVjdCsweDVkDQpzeXNjYWxsZW50ZXIo KSBhdCBzeXNjYWxsZW50ZXIrMHhmMA0Kc3lzY2FsbCgpIGF0IHN5c2NhbGwrMHg2MA0KWGZh c3Rfc3lzY2FsbCgpIGF0IFhmYXN0X3N5c2NhbGwrMHhlMg0KLS0tIHN5c2NhbGwgKDkzLCBG cmVlQlNEIEVMRjY0LCBzZWxlY3QpLCByaXAgPSAweDgwMWU3YWY2YywgcnNwID0gMHg3ZmZm ZmZmZmU3ZDgsIHJicCA9IDB4MTAgLS0tDQoNClRyYWNpbmcgY29tbWFuZCBuZnNkIHBpZCA3 NDMgdGlkIDEwMDEyMiB0ZCAweGZmZmZmZjAwMDRmNmEwMDANCnNjaGVkX3N3aXRjaCgpIGF0 IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpz bGVlcHFfc3dpdGNoKCkgYXQgc2xlZXBxX3N3aXRjaCsweDEyMw0Kc2xlZXBxX2NhdGNoX3Np Z25hbHMoKSBhdCBzbGVlcHFfY2F0Y2hfc2lnbmFscysweDJhNg0Kc2xlZXBxX3RpbWVkd2Fp dF9zaWcoKSBhdCBzbGVlcHFfdGltZWR3YWl0X3NpZysweDE5DQpfY3ZfdGltZWR3YWl0X3Np ZygpIGF0IF9jdl90aW1lZHdhaXRfc2lnKzB4MThjDQpzdmNfcnVuX2ludGVybmFsKCkgYXQg c3ZjX3J1bl9pbnRlcm5hbCsweDgxYQ0Kc3ZjX3RocmVhZF9zdGFydCgpIGF0IHN2Y190aHJl YWRfc3RhcnQrMHhiDQpmb3JrX2V4aXQoKSBhdCBmb3JrX2V4aXQrMHgxMmENCmZvcmtfdHJh bXBvbGluZSgpIGF0IGZvcmtfdHJhbXBvbGluZSsweGUNCi0tLSB0cmFwIDB4YywgcmlwID0g MHg4MDA2YTA0NWMsIHJzcCA9IDB4N2ZmZmZmZmZlNmQ4LCByYnAgPSAweDUgLS0tDQoNClRy YWNpbmcgY29tbWFuZCBuZnNkIHBpZCA3NDMgdGlkIDEwMDEyMSB0ZCAweGZmZmZmZjAwMDRm NmE0NjANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dpdGNo KCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpzbGVlcHFfc3dpdGNoKCkgYXQgc2xlZXBxX3N3aXRj aCsweDEyMw0Kc2xlZXBxX2NhdGNoX3NpZ25hbHMoKSBhdCBzbGVlcHFfY2F0Y2hfc2lnbmFs cysweDJhNg0Kc2xlZXBxX3RpbWVkd2FpdF9zaWcoKSBhdCBzbGVlcHFfdGltZWR3YWl0X3Np ZysweDE5DQpfY3ZfdGltZWR3YWl0X3NpZygpIGF0IF9jdl90aW1lZHdhaXRfc2lnKzB4MThj DQpzdmNfcnVuX2ludGVybmFsKCkgYXQgc3ZjX3J1bl9pbnRlcm5hbCsweDgxYQ0Kc3ZjX3Ro cmVhZF9zdGFydCgpIGF0IHN2Y190aHJlYWRfc3RhcnQrMHhiDQpmb3JrX2V4aXQoKSBhdCBm b3JrX2V4aXQrMHgxMmENCmZvcmtfdHJhbXBvbGluZSgpIGF0IGZvcmtfdHJhbXBvbGluZSsw eGUNCi0tLSB0cmFwIDB4YywgcmlwID0gMHg4MDA2YTA0NWMsIHJzcCA9IDB4N2ZmZmZmZmZl NmQ4LCByYnAgPSAweDUgLS0tDQoNClRyYWNpbmcgY29tbWFuZCBuZnNkIHBpZCA3NDMgdGlk IDEwMDEyMCB0ZCAweGZmZmZmZjAwMDRmNmE4YzANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVk X3N3aXRjaCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpzbGVlcHFf c3dpdGNoKCkgYXQgc2xlZXBxX3N3aXRjaCsweDEyMw0Kc2xlZXBxX2NhdGNoX3NpZ25hbHMo KSBhdCBzbGVlcHFfY2F0Y2hfc2lnbmFscysweDJhNg0Kc2xlZXBxX3RpbWVkd2FpdF9zaWco KSBhdCBzbGVlcHFfdGltZWR3YWl0X3NpZysweDE5DQpfY3ZfdGltZWR3YWl0X3NpZygpIGF0 IF9jdl90aW1lZHdhaXRfc2lnKzB4MThjDQpzdmNfcnVuX2ludGVybmFsKCkgYXQgc3ZjX3J1 bl9pbnRlcm5hbCsweDgxYQ0Kc3ZjX3RocmVhZF9zdGFydCgpIGF0IHN2Y190aHJlYWRfc3Rh cnQrMHhiDQpmb3JrX2V4aXQoKSBhdCBmb3JrX2V4aXQrMHgxMmENCmZvcmtfdHJhbXBvbGlu ZSgpIGF0IGZvcmtfdHJhbXBvbGluZSsweGUNCi0tLSB0cmFwIDB4YywgcmlwID0gMHg4MDA2 YTA0NWMsIHJzcCA9IDB4N2ZmZmZmZmZlNmQ4LCByYnAgPSAweDUgLS0tDQoNClRyYWNpbmcg Y29tbWFuZCBuZnNkIHBpZCA3NDMgdGlkIDEwMDA4NyB0ZCAweGZmZmZmZjAwMDRhOTA0NjAN CnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dpdGNoKCkgYXQg bWlfc3dpdGNoKzB4MjFiDQpzbGVlcHFfc3dpdGNoKCkgYXQgc2xlZXBxX3N3aXRjaCsweDEy Mw0Kc2xlZXBxX2NhdGNoX3NpZ25hbHMoKSBhdCBzbGVlcHFfY2F0Y2hfc2lnbmFscysweDJh Ng0Kc2xlZXBxX3RpbWVkd2FpdF9zaWcoKSBhdCBzbGVlcHFfdGltZWR3YWl0X3NpZysweDE5 DQpfY3ZfdGltZWR3YWl0X3NpZygpIGF0IF9jdl90aW1lZHdhaXRfc2lnKzB4MThjDQpzdmNf cnVuX2ludGVybmFsKCkgYXQgc3ZjX3J1bl9pbnRlcm5hbCsweDgxYQ0Kc3ZjX3J1bigpIGF0 IHN2Y19ydW4rMHg4Zg0KbmZzc3ZjX25mc2QoKSBhdCBuZnNzdmNfbmZzZCsweGEyDQpuZnNz dmNfbmZzc2VydmVyKCkgYXQgbmZzc3ZjX25mc3NlcnZlcisweDViDQpuZnNzdmMoKSBhdCBu ZnNzdmMrMHg3Mw0Kc3lzY2FsbGVudGVyKCkgYXQgc3lzY2FsbGVudGVyKzB4ZjANCnN5c2Nh bGwoKSBhdCBzeXNjYWxsKzB4NjANClhmYXN0X3N5c2NhbGwoKSBhdCBYZmFzdF9zeXNjYWxs KzB4ZTINCi0tLSBzeXNjYWxsICgxNTUsIEZyZWVCU0QgRUxGNjQsIG5mc3N2YyksIHJpcCA9 IDB4ODAwNmEwNDVjLCByc3AgPSAweDdmZmZmZmZmZTZkOCwgcmJwID0gMHg1IC0tLQ0KDQpU cmFjaW5nIGNvbW1hbmQgbmZzZCBwaWQgNzQyIHRpZCAxMDAxMDkgdGQgMHhmZmZmZmYwMDA0 YmU5MDAwDQoNClRyYWNpbmcgY29tbWFuZCBtb3VudGQgcGlkIDczMyB0aWQgMTAwMDkyIHRk IDB4ZmZmZmZmMDAwNGJmMzAwMA0Kc2NoZWRfc3dpdGNoKCkgYXQgc2NoZWRfc3dpdGNoKzB4 MTM4DQptaV9zd2l0Y2goKSBhdCBtaV9zd2l0Y2grMHgyMWINCnNsZWVwcV9zd2l0Y2goKSBh dCBzbGVlcHFfc3dpdGNoKzB4MTIzDQpzbGVlcHFfY2F0Y2hfc2lnbmFscygpIGF0IHNsZWVw cV9jYXRjaF9zaWduYWxzKzB4MmE2DQpzbGVlcHFfd2FpdF9zaWcoKSBhdCBzbGVlcHFfd2Fp dF9zaWcrMHgxNg0KX2N2X3dhaXRfc2lnKCkgYXQgX2N2X3dhaXRfc2lnKzB4MTdlDQpzZWx0 ZHdhaXQoKSBhdCBzZWx0ZHdhaXQrMHhhYw0Ka2Vybl9zZWxlY3QoKSBhdCBrZXJuX3NlbGVj dCsweDY0MA0Kc2VsZWN0KCkgYXQgc2VsZWN0KzB4NWQNCnN5c2NhbGxlbnRlcigpIGF0IHN5 c2NhbGxlbnRlcisweGYwDQpzeXNjYWxsKCkgYXQgc3lzY2FsbCsweDYwDQpYZmFzdF9zeXNj YWxsKCkgYXQgWGZhc3Rfc3lzY2FsbCsweGUyDQotLS0gc3lzY2FsbCAoOTMsIEZyZWVCU0Qg RUxGNjQsIHNlbGVjdCksIHJpcCA9IDB4ODAwODUzZjZjLCByc3AgPSAweDdmZmZmZmZmZWM3 OCwgcmJwID0gMHg3ZmZmZmZmZmVkNzAgLS0tDQoNClRyYWNpbmcgY29tbWFuZCBycGNiaW5k IHBpZCA2MzggdGlkIDEwMDA4OSB0ZCAweGZmZmZmZjAwMDRiMDQ4YzANCnNjaGVkX3N3aXRj aCgpIGF0IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4 MjFiDQpzbGVlcHFfc3dpdGNoKCkgYXQgc2xlZXBxX3N3aXRjaCsweDEyMw0Kc2xlZXBxX3dh aXQoKSBhdCBzbGVlcHFfd2FpdCsweDRkDQpfc2xlZXAoKSBhdCBfc2xlZXArMHgzNjkNCnZt X2ZhdWx0KCkgYXQgdm1fZmF1bHQrMHgxNWJjDQp0cmFwX3BmYXVsdCgpIGF0IHRyYXBfcGZh dWx0KzB4MTI4DQp0cmFwKCkgYXQgdHJhcCsweDM2Yw0KY2FsbHRyYXAoKSBhdCBjYWxsdHJh cCsweDgNCi0tLSB0cmFwIDB4YywgcmlwID0gMHhmZmZmZmZmZjgwNDVmNWMzLCByc3AgPSAw eGZmZmZmZjgwOTZjMDM5MTAsIHJicCA9IDB4ZmZmZmZmODA5NmMwM2FlMCAtLS0NCmNvcHlv dXQoKSBhdCBjb3B5b3V0KzB4NDMNCnN5c2NhbGxlbnRlcigpIGF0IHN5c2NhbGxlbnRlcisw eGYwDQpzeXNjYWxsKCkgYXQgc3lzY2FsbCsweDYwDQpYZmFzdF9zeXNjYWxsKCkgYXQgWGZh c3Rfc3lzY2FsbCsweGUyDQotLS0gc3lzY2FsbCAoMjA5LCBGcmVlQlNEIEVMRjY0LCBwb2xs KSwgcmlwID0gMHg4MDA4ZmYwNmMsIHJzcCA9IDB4N2ZmZmZmZmZjYTg4LCByYnAgPSAweDgw MGMxNTBjMCAtLS0NCg0KVHJhY2luZyBjb21tYW5kIHN5c2xvZ2QgcGlkIDYxMiB0aWQgMTAw MDgzIHRkIDB4ZmZmZmZmMDAwNGE5MzQ2MA0Kc2NoZWRfc3dpdGNoKCkgYXQgc2NoZWRfc3dp dGNoKzB4MTM4DQptaV9zd2l0Y2goKSBhdCBtaV9zd2l0Y2grMHgyMWINCnNsZWVwcV9zd2l0 Y2goKSBhdCBzbGVlcHFfc3dpdGNoKzB4MTIzDQpzbGVlcHFfd2FpdCgpIGF0IHNsZWVwcV93 YWl0KzB4NGQNCl9zbGVlcCgpIGF0IF9zbGVlcCsweDM2OQ0Kdm1fZmF1bHQoKSBhdCB2bV9m YXVsdCsweDE1YmMNCnRyYXBfcGZhdWx0KCkgYXQgdHJhcF9wZmF1bHQrMHgxMjgNCnRyYXAo KSBhdCB0cmFwKzB4MzZjDQpjYWxsdHJhcCgpIGF0IGNhbGx0cmFwKzB4OA0KLS0tIHRyYXAg MHhjLCByaXAgPSAweGZmZmZmZmZmODA0NWY1YmIsIHJzcCA9IDB4ZmZmZmZmODA5NmJlNTZi MCwgcmJwID0gMHhmZmZmZmY4MDk2YmU1YjAwIC0tLQ0KY29weW91dCgpIGF0IGNvcHlvdXQr MHgzYg0KcG9zdHNpZygpIGF0IHBvc3RzaWcrMHgyN2YNCmFzdCgpIGF0IGFzdCsweDE5OQ0K ZG9yZXRpX2FzdCgpIGF0IGRvcmV0aV9hc3QrMHgxZg0KDQpUcmFjaW5nIGNvbW1hbmQgZGV2 ZCBwaWQgNDg4IHRpZCAxMDAxMTEgdGQgMHhmZmZmZmYwMDA0Zjc1OGMwDQpzY2hlZF9zd2l0 Y2goKSBhdCBzY2hlZF9zd2l0Y2grMHgxMzgNCm1pX3N3aXRjaCgpIGF0IG1pX3N3aXRjaCsw eDIxYg0Kc2xlZXBxX3N3aXRjaCgpIGF0IHNsZWVwcV9zd2l0Y2grMHgxMjMNCnNsZWVwcV9j YXRjaF9zaWduYWxzKCkgYXQgc2xlZXBxX2NhdGNoX3NpZ25hbHMrMHgyYTYNCnNsZWVwcV93 YWl0X3NpZygpIGF0IHNsZWVwcV93YWl0X3NpZysweDE2DQpfY3Zfd2FpdF9zaWcoKSBhdCBf Y3Zfd2FpdF9zaWcrMHgxN2UNCnNlbHRkd2FpdCgpIGF0IHNlbHRkd2FpdCsweGFjDQprZXJu X3NlbGVjdCgpIGF0IGtlcm5fc2VsZWN0KzB4NjQwDQpzZWxlY3QoKSBhdCBzZWxlY3QrMHg1 ZA0Kc3lzY2FsbGVudGVyKCkgYXQgc3lzY2FsbGVudGVyKzB4ZjANCnN5c2NhbGwoKSBhdCBz eXNjYWxsKzB4NjANClhmYXN0X3N5c2NhbGwoKSBhdCBYZmFzdF9zeXNjYWxsKzB4ZTINCi0t LSBzeXNjYWxsICg5MywgRnJlZUJTRCBFTEY2NCwgc2VsZWN0KSwgcmlwID0gMHg0NDBmOWMs IHJzcCA9IDB4N2ZmZmZmZmZlODk4LCByYnAgPSAweDdmZmZmZmZmZThiMCAtLS0NCg0KVHJh Y2luZyBjb21tYW5kIGRoY2xpZW50IHBpZCA0NzQgdGlkIDEwMDExMiB0ZCAweGZmZmZmZjAw MDRmNzU0NjANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dp dGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpzbGVlcHFfc3dpdGNoKCkgYXQgc2xlZXBxX3N3 aXRjaCsweDEyMw0Kc2xlZXBxX2NhdGNoX3NpZ25hbHMoKSBhdCBzbGVlcHFfY2F0Y2hfc2ln bmFscysweDJhNg0Kc2xlZXBxX3RpbWVkd2FpdF9zaWcoKSBhdCBzbGVlcHFfdGltZWR3YWl0 X3NpZysweDE5DQpfY3ZfdGltZWR3YWl0X3NpZygpIGF0IF9jdl90aW1lZHdhaXRfc2lnKzB4 MThjDQpzZWx0ZHdhaXQoKSBhdCBzZWx0ZHdhaXQrMHg1Ng0KcG9sbCgpIGF0IHBvbGwrMHg0 MjENCnN5c2NhbGxlbnRlcigpIGF0IHN5c2NhbGxlbnRlcisweGYwDQpzeXNjYWxsKCkgYXQg c3lzY2FsbCsweDYwDQpYZmFzdF9zeXNjYWxsKCkgYXQgWGZhc3Rfc3lzY2FsbCsweGUyDQot LS0gc3lzY2FsbCAoMjA5LCBGcmVlQlNEIEVMRjY0LCBwb2xsKSwgcmlwID0gMHg4MDA2ZWUw NmMsIHJzcCA9IDB4N2ZmZmZmZmZlY2U4LCByYnAgPSAweDE0ODFjMTggLS0tDQoNClRyYWNp bmcgY29tbWFuZCBkaGNsaWVudCBwaWQgNDU4IHRpZCAxMDAxMTkgdGQgMHhmZmZmZmYwMDA0 ZjZiMDAwDQpzY2hlZF9zd2l0Y2goKSBhdCBzY2hlZF9zd2l0Y2grMHgxMzgNCm1pX3N3aXRj aCgpIGF0IG1pX3N3aXRjaCsweDIxYg0Kc2xlZXBxX3N3aXRjaCgpIGF0IHNsZWVwcV9zd2l0 Y2grMHgxMjMNCnNsZWVwcV9jYXRjaF9zaWduYWxzKCkgYXQgc2xlZXBxX2NhdGNoX3NpZ25h bHMrMHgyYTYNCnNsZWVwcV93YWl0X3NpZygpIGF0IHNsZWVwcV93YWl0X3NpZysweDE2DQpf Y3Zfd2FpdF9zaWcoKSBhdCBfY3Zfd2FpdF9zaWcrMHgxN2UNCnNlbHRkd2FpdCgpIGF0IHNl bHRkd2FpdCsweGFjDQpwb2xsKCkgYXQgcG9sbCsweDQyMQ0Kc3lzY2FsbGVudGVyKCkgYXQg c3lzY2FsbGVudGVyKzB4ZjANCnN5c2NhbGwoKSBhdCBzeXNjYWxsKzB4NjANClhmYXN0X3N5 c2NhbGwoKSBhdCBYZmFzdF9zeXNjYWxsKzB4ZTINCi0tLSBzeXNjYWxsICgyMDksIEZyZWVC U0QgRUxGNjQsIHBvbGwpLCByaXAgPSAweDgwMDZlZTA2YywgcnNwID0gMHg3ZmZmZmZmZmVj ZTgsIHJicCA9IDB4NiAtLS0NCg0KVHJhY2luZyBjb21tYW5kIGFkamtlcm50eiBwaWQgMTIy IHRpZCAxMDAxMDAgdGQgMHhmZmZmZmYwMDA0YmYyMDAwDQoNClRyYWNpbmcgY29tbWFuZCBn X3JhaWQ1IHBpZCAyMCB0aWQgMTAwMDc5IHRkIDB4ZmZmZmZmMDAwNGIwNTQ2MA0Kc2NoZWRf c3dpdGNoKCkgYXQgc2NoZWRfc3dpdGNoKzB4MTM4DQptaV9zd2l0Y2goKSBhdCBtaV9zd2l0 Y2grMHgyMWINCnNsZWVwcV9zd2l0Y2goKSBhdCBzbGVlcHFfc3dpdGNoKzB4MTIzDQpzbGVl cHFfdGltZWR3YWl0KCkgYXQgc2xlZXBxX3RpbWVkd2FpdCsweDRkDQpfc2xlZXAoKSBhdCBf c2xlZXArMHgzNTINCmdfcmFpZDVfd29ya2VyRCgpIGF0IGdfcmFpZDVfd29ya2VyRCsweDE3 NA0KZm9ya19leGl0KCkgYXQgZm9ya19leGl0KzB4MTJhDQpmb3JrX3RyYW1wb2xpbmUoKSBh dCBmb3JrX3RyYW1wb2xpbmUrMHhlDQotLS0gdHJhcCAwLCByaXAgPSAwLCByc3AgPSAweGZm ZmZmZjgwOTZiZDFjZjAsIHJicCA9IDAgLS0tDQpUcmFjaW5nIGNvbW1hbmQgZ19yYWlkNSBw aWQgMjAgdGlkIDEwMDA3OCB0ZCAweGZmZmZmZjAwMDRiMDU4YzANCnNjaGVkX3N3aXRjaCgp IGF0IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFi DQpzbGVlcHFfc3dpdGNoKCkgYXQgc2xlZXBxX3N3aXRjaCsweDEyMw0Kc2xlZXBxX3dhaXQo KSBhdCBzbGVlcHFfd2FpdCsweDRkDQpfc2xlZXAoKSBhdCBfc2xlZXArMHgzNjkNCmttZW1f YmFjaygpIGF0IGttZW1fYmFjaysweDExZQ0Ka21lbV9tYWxsb2MoKSBhdCBrbWVtX21hbGxv YysweDFmOA0KdW1hX2xhcmdlX21hbGxvYygpIGF0IHVtYV9sYXJnZV9tYWxsb2MrMHg0YQ0K bWFsbG9jKCkgYXQgbWFsbG9jKzB4MTU5DQpnX3JhaWQ1X21hbGxvYygpIGF0IGdfcmFpZDVf bWFsbG9jKzB4MTQzDQpnX3JhaWQ1X21hbGxvY3goKSBhdCBnX3JhaWQ1X21hbGxvY3grMHhj MA0KZ19yYWlkNV9pc3N1ZV9jaGVjaygpIGF0IGdfcmFpZDVfaXNzdWVfY2hlY2srMHgxMDIN CmdfcmFpZDVfd3JpdGUoKSBhdCBnX3JhaWQ1X3dyaXRlKzB4MzlmDQpnX3JhaWQ1X2ZsdXNo X3NxKCkgYXQgZ19yYWlkNV9mbHVzaF9zcSsweDFhZg0KZ19yYWlkNV93b3JrZXIoKSBhdCBn X3JhaWQ1X3dvcmtlcisweDViZQ0KZm9ya19leGl0KCkgYXQgZm9ya19leGl0KzB4MTJhDQpm b3JrX3RyYW1wb2xpbmUoKSBhdCBmb3JrX3RyYW1wb2xpbmUrMHhlDQotLS0gdHJhcCAwLCBy aXAgPSAwLCByc3AgPSAweGZmZmZmZjgwOTZiY2NjZjAsIHJicCA9IDAgLS0tDQpUcmFjaW5n IGNvbW1hbmQgc29mdGRlcGZsdXNoIHBpZCAxOSB0aWQgMTAwMDc3IHRkIDB4ZmZmZmZmMDAw NGE5NDhjMA0Kc2NoZWRfc3dpdGNoKCkgYXQgc2NoZWRfc3dpdGNoKzB4MTM4DQptaV9zd2l0 Y2goKSBhdCBtaV9zd2l0Y2grMHgyMWINCnNsZWVwcV9zd2l0Y2goKSBhdCBzbGVlcHFfc3dp dGNoKzB4MTIzDQpzbGVlcHFfdGltZWR3YWl0KCkgYXQgc2xlZXBxX3RpbWVkd2FpdCsweDRk DQpfc2xlZXAoKSBhdCBfc2xlZXArMHgzNTINCnNvZnRkZXBfZmx1c2goKSBhdCBzb2Z0ZGVw X2ZsdXNoKzB4MjU5DQpmb3JrX2V4aXQoKSBhdCBmb3JrX2V4aXQrMHgxMmENCmZvcmtfdHJh bXBvbGluZSgpIGF0IGZvcmtfdHJhbXBvbGluZSsweGUNCi0tLSB0cmFwIDAsIHJpcCA9IDAs IHJzcCA9IDB4ZmZmZmZmODA5NDhjNGNmMCwgcmJwID0gMCAtLS0NCg0KVHJhY2luZyBjb21t YW5kIHZubHJ1IHBpZCAxOCB0aWQgMTAwMDc2IHRkIDB4ZmZmZmZmMDAwNGE5NTAwMA0Kc2No ZWRfc3dpdGNoKCkgYXQgc2NoZWRfc3dpdGNoKzB4MTM4DQptaV9zd2l0Y2goKSBhdCBtaV9z d2l0Y2grMHgyMWINCnNsZWVwcV9zd2l0Y2goKSBhdCBzbGVlcHFfc3dpdGNoKzB4MTIzDQpz bGVlcHFfdGltZWR3YWl0KCkgYXQgc2xlZXBxX3RpbWVkd2FpdCsweDRkDQpfc2xlZXAoKSBh dCBfc2xlZXArMHgzNTINCnZubHJ1X3Byb2MoKSBhdCB2bmxydV9wcm9jKzB4NGRmDQpmb3Jr X2V4aXQoKSBhdCBmb3JrX2V4aXQrMHgxMmENCmZvcmtfdHJhbXBvbGluZSgpIGF0IGZvcmtf dHJhbXBvbGluZSsweGUNCi0tLSB0cmFwIDAsIHJpcCA9IDAsIHJzcCA9IDB4ZmZmZmZmODA5 NDhiZmNmMCwgcmJwID0gMCAtLS0NCg0KVHJhY2luZyBjb21tYW5kIHN5bmNlciBwaWQgMTcg dGlkIDEwMDA3NSB0ZCAweGZmZmZmZjAwMDRhOTU0NjANCnNjaGVkX3N3aXRjaCgpIGF0IHNj aGVkX3N3aXRjaCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpzbGVl cHFfc3dpdGNoKCkgYXQgc2xlZXBxX3N3aXRjaCsweDEyMw0Kc2xlZXBxX3dhaXQoKSBhdCBz bGVlcHFfd2FpdCsweDRkDQpfc2xlZXAoKSBhdCBfc2xlZXArMHgzNjkNCmdldHBidWYoKSBh dCBnZXRwYnVmKzB4NGUNCmNsdXN0ZXJfd2J1aWxkKCkgYXQgY2x1c3Rlcl93YnVpbGQrMHgx ZWUNCmZmc19zeW5jdm5vZGUoKSBhdCBmZnNfc3luY3Zub2RlKzB4MWE2DQpmZnNfc3luYygp IGF0IGZmc19zeW5jKzB4MjE2DQpzeW5jX2ZzeW5jKCkgYXQgc3luY19mc3luYysweDEzYQ0K Vk9QX0ZTWU5DX0FQVigpIGF0IFZPUF9GU1lOQ19BUFYrMHhiNQ0Kc3luY192bm9kZSgpIGF0 IHN5bmNfdm5vZGUrMHgxNTcNCnNjaGVkX3N5bmMoKSBhdCBzY2hlZF9zeW5jKzB4MWQyDQpm b3JrX2V4aXQoKSBhdCBmb3JrX2V4aXQrMHgxMmENCmZvcmtfdHJhbXBvbGluZSgpIGF0IGZv cmtfdHJhbXBvbGluZSsweGUNCi0tLSB0cmFwIDAsIHJpcCA9IDAsIHJzcCA9IDB4ZmZmZmZm ODA5NDhiYWNmMCwgcmJwID0gMCAtLS0NClRyYWNpbmcgY29tbWFuZCBidWZkYWVtb24gcGlk IDE2IHRpZCAxMDAwNzQgdGQgMHhmZmZmZmYwMDA0YTk1OGMwDQpzY2hlZF9zd2l0Y2goKSBh dCBzY2hlZF9zd2l0Y2grMHgxMzgNCm1pX3N3aXRjaCgpIGF0IG1pX3N3aXRjaCsweDIxYg0K c2xlZXBxX3N3aXRjaCgpIGF0IHNsZWVwcV9zd2l0Y2grMHgxMjMNCnNsZWVwcV90aW1lZHdh aXQoKSBhdCBzbGVlcHFfdGltZWR3YWl0KzB4NGQNCl9zbGVlcCgpIGF0IF9zbGVlcCsweDM1 Mg0KYnVmX2RhZW1vbigpIGF0IGJ1Zl9kYWVtb24rMHgxNGENCmZvcmtfZXhpdCgpIGF0IGZv cmtfZXhpdCsweDEyYQ0KZm9ya190cmFtcG9saW5lKCkgYXQgZm9ya190cmFtcG9saW5lKzB4 ZQ0KLS0tIHRyYXAgMCwgcmlwID0gMCwgcnNwID0gMHhmZmZmZmY4MDk0OGI1Y2YwLCByYnAg PSAwIC0tLQ0KDQpUcmFjaW5nIGNvbW1hbmQgcGFnZXplcm8gcGlkIDE1IHRpZCAxMDAwNzMg dGQgMHhmZmZmZmYwMDA0YTk2MDAwDQpzY2hlZF9zd2l0Y2goKSBhdCBzY2hlZF9zd2l0Y2gr MHgxMzgNCm1pX3N3aXRjaCgpIGF0IG1pX3N3aXRjaCsweDIxYg0Kc2xlZXBxX3N3aXRjaCgp IGF0IHNsZWVwcV9zd2l0Y2grMHgxMjMNCnNsZWVwcV90aW1lZHdhaXQoKSBhdCBzbGVlcHFf dGltZWR3YWl0KzB4NGQNCl9zbGVlcCgpIGF0IF9zbGVlcCsweDM1Mg0Kdm1fcGFnZXplcm8o KSBhdCB2bV9wYWdlemVybysweDczDQpmb3JrX2V4aXQoKSBhdCBmb3JrX2V4aXQrMHgxMmEN CmZvcmtfdHJhbXBvbGluZSgpIGF0IGZvcmtfdHJhbXBvbGluZSsweGUNCi0tLSB0cmFwIDAs IHJpcCA9IDAsIHJzcCA9IDB4ZmZmZmZmODA5NDhiMGNmMCwgcmJwID0gMCAtLS0NCg0KVHJh Y2luZyBjb21tYW5kIGlkbGVwb2xsIHBpZCA5IHRpZCAxMDAwNzIgdGQgMHhmZmZmZmYwMDAx Nzc4MDAwDQpzY2hlZF9zd2l0Y2goKSBhdCBzY2hlZF9zd2l0Y2grMHgxMzgNCm1pX3N3aXRj aCgpIGF0IG1pX3N3aXRjaCsweDIxYg0Kc2xlZXBxX3N3aXRjaCgpIGF0IHNsZWVwcV9zd2l0 Y2grMHgxMjMNCnNsZWVwcV90aW1lZHdhaXQoKSBhdCBzbGVlcHFfdGltZWR3YWl0KzB4NGQN Cl9zbGVlcCgpIGF0IF9zbGVlcCsweDM1Mg0KcG9sbF9pZGxlKCkgYXQgcG9sbF9pZGxlKzB4 OTQNCmZvcmtfZXhpdCgpIGF0IGZvcmtfZXhpdCsweDEyYQ0KZm9ya190cmFtcG9saW5lKCkg YXQgZm9ya190cmFtcG9saW5lKzB4ZQ0KLS0tIHRyYXAgMCwgcmlwID0gMCwgcnNwID0gMHhm ZmZmZmY4MDk0OGFiY2YwLCByYnAgPSAwIC0tLQ0KDQpUcmFjaW5nIGNvbW1hbmQgdm1kYWVt b24gcGlkIDggdGlkIDEwMDA3MSB0ZCAweGZmZmZmZjAwMDE3Nzg0NjANCnNjaGVkX3N3aXRj aCgpIGF0IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4 MjFiDQpzbGVlcHFfc3dpdGNoKCkgYXQgc2xlZXBxX3N3aXRjaCsweDEyMw0Kc2xlZXBxX3dh aXQoKSBhdCBzbGVlcHFfd2FpdCsweDRkDQpfc2xlZXAoKSBhdCBfc2xlZXArMHgzNjkNCnZt X2RhZW1vbigpIGF0IHZtX2RhZW1vbisweDRkDQpmb3JrX2V4aXQoKSBhdCBmb3JrX2V4aXQr MHgxMmENCmZvcmtfdHJhbXBvbGluZSgpIGF0IGZvcmtfdHJhbXBvbGluZSsweGUNCi0tLSB0 cmFwIDAsIHJpcCA9IDAsIHJzcCA9IDB4ZmZmZmZmODA5NDhhNmNmMCwgcmJwID0gMCAtLS0N Cg0KVHJhY2luZyBjb21tYW5kIHBhZ2VkYWVtb24gcGlkIDcgdGlkIDEwMDA3MCB0ZCAweGZm ZmZmZjAwMDE3Nzg4YzANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRjaCsweDEzOA0K bWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpzbGVlcHFfc3dpdGNoKCkgYXQgc2xl ZXBxX3N3aXRjaCsweDEyMw0Kc2xlZXBxX3RpbWVkd2FpdCgpIGF0IHNsZWVwcV90aW1lZHdh aXQrMHg0ZA0KX3NsZWVwKCkgYXQgX3NsZWVwKzB4MzUyDQp2bV9wYWdlb3V0KCkgYXQgdm1f cGFnZW91dCsweDI3Yw0KZm9ya19leGl0KCkgYXQgZm9ya19leGl0KzB4MTJhDQpmb3JrX3Ry YW1wb2xpbmUoKSBhdCBmb3JrX3RyYW1wb2xpbmUrMHhlDQotLS0gdHJhcCAwLCByaXAgPSAw LCByc3AgPSAweGZmZmZmZjgwOTQ4YTFjZjAsIHJicCA9IDAgLS0tDQoNClRyYWNpbmcgY29t bWFuZCB4cHRfdGhyZCBwaWQgNiB0aWQgMTAwMDY5IHRkIDB4ZmZmZmZmMDAwMTc3YTAwMA0K c2NoZWRfc3dpdGNoKCkgYXQgc2NoZWRfc3dpdGNoKzB4MTM4DQptaV9zd2l0Y2goKSBhdCBt aV9zd2l0Y2grMHgyMWINCnNsZWVwcV9zd2l0Y2goKSBhdCBzbGVlcHFfc3dpdGNoKzB4MTIz DQpzbGVlcHFfd2FpdCgpIGF0IHNsZWVwcV93YWl0KzB4NGQNCl9zbGVlcCgpIGF0IF9zbGVl cCsweDM2OQ0KeHB0X3NjYW5uZXJfdGhyZWFkKCkgYXQgeHB0X3NjYW5uZXJfdGhyZWFkKzB4 ZDUNCmZvcmtfZXhpdCgpIGF0IGZvcmtfZXhpdCsweDEyYQ0KZm9ya190cmFtcG9saW5lKCkg YXQgZm9ya190cmFtcG9saW5lKzB4ZQ0KLS0tIHRyYXAgMCwgcmlwID0gMCwgcnNwID0gMHhm ZmZmZmY4MDk0ODljY2YwLCByYnAgPSAwIC0tLQ0KDQpUcmFjaW5nIGNvbW1hbmQgZncwX3By b2JlIHBpZCA1IHRpZCAxMDAwNjQgdGQgMHhmZmZmZmYwMDAxNzdjOGMwDQpzY2hlZF9zd2l0 Y2goKSBhdCBzY2hlZF9zd2l0Y2grMHgxMzgNCm1pX3N3aXRjaCgpIGF0IG1pX3N3aXRjaCsw eDIxYg0Kc2xlZXBxX3N3aXRjaCgpIGF0IHNsZWVwcV9zd2l0Y2grMHgxMjMNCnNsZWVwcV9j YXRjaF9zaWduYWxzKCkgYXQgc2xlZXBxX2NhdGNoX3NpZ25hbHMrMHgyYTYNCnNsZWVwcV93 YWl0X3NpZygpIGF0IHNsZWVwcV93YWl0X3NpZysweDE2DQpfc2xlZXAoKSBhdCBfc2xlZXAr MHgzMWENCmZ3X2J1c19wcm9iZV90aHJlYWQoKSBhdCBmd19idXNfcHJvYmVfdGhyZWFkKzB4 YmMNCmZvcmtfZXhpdCgpIGF0IGZvcmtfZXhpdCsweDEyYQ0KZm9ya190cmFtcG9saW5lKCkg YXQgZm9ya190cmFtcG9saW5lKzB4ZQ0KLS0tIHRyYXAgMCwgcmlwID0gMCwgcnNwID0gMHhm ZmZmZmY4MDAwMTcyY2YwLCByYnAgPSAwIC0tLQ0KDQpUcmFjaW5nIGNvbW1hbmQgdXNiIHBp ZCAxNCB0aWQgMTAwMDYxIHRkIDB4ZmZmZmZmMDAwMTc3ZDhjMA0Kc2NoZWRfc3dpdGNoKCkg YXQgc2NoZWRfc3dpdGNoKzB4MTM4DQptaV9zd2l0Y2goKSBhdCBtaV9zd2l0Y2grMHgyMWIN CnNsZWVwcV9zd2l0Y2goKSBhdCBzbGVlcHFfc3dpdGNoKzB4MTIzDQpzbGVlcHFfd2FpdCgp IGF0IHNsZWVwcV93YWl0KzB4NGQNCl9jdl93YWl0KCkgYXQgX2N2X3dhaXQrMHgxN2ENCnVz Yl9wcm9jZXNzKCkgYXQgdXNiX3Byb2Nlc3MrMHgxNGUNCmZvcmtfZXhpdCgpIGF0IGZvcmtf ZXhpdCsweDEyYQ0KZm9ya190cmFtcG9saW5lKCkgYXQgZm9ya190cmFtcG9saW5lKzB4ZQ0K LS0tIHRyYXAgMCwgcmlwID0gMCwgcnNwID0gMHhmZmZmZmY4MDAwMTVmY2YwLCByYnAgPSAw IC0tLQ0KDQpUcmFjaW5nIGNvbW1hbmQgdXNiIHBpZCAxNCB0aWQgMTAwMDYwIHRkIDB4ZmZm ZmZmMDAwMTcyNTQ2MA0Kc2NoZWRfc3dpdGNoKCkgYXQgc2NoZWRfc3dpdGNoKzB4MTM4DQpt aV9zd2l0Y2goKSBhdCBtaV9zd2l0Y2grMHgyMWINCnNsZWVwcV9zd2l0Y2goKSBhdCBzbGVl cHFfc3dpdGNoKzB4MTIzDQpzbGVlcHFfd2FpdCgpIGF0IHNsZWVwcV93YWl0KzB4NGQNCl9j dl93YWl0KCkgYXQgX2N2X3dhaXQrMHgxN2ENCnVzYl9wcm9jZXNzKCkgYXQgdXNiX3Byb2Nl c3MrMHgxNGUNCmZvcmtfZXhpdCgpIGF0IGZvcmtfZXhpdCsweDEyYQ0KZm9ya190cmFtcG9s aW5lKCkgYXQgZm9ya190cmFtcG9saW5lKzB4ZQ0KLS0tIHRyYXAgMCwgcmlwID0gMCwgcnNw ID0gMHhmZmZmZmY4MDAwMTVhY2YwLCByYnAgPSAwIC0tLQ0KVHJhY2luZyBjb21tYW5kIHVz YiBwaWQgMTQgdGlkIDEwMDA1OSB0ZCAweGZmZmZmZjAwMDE3MjU4YzANCnNjaGVkX3N3aXRj aCgpIGF0IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4 MjFiDQpzbGVlcHFfc3dpdGNoKCkgYXQgc2xlZXBxX3N3aXRjaCsweDEyMw0Kc2xlZXBxX3dh aXQoKSBhdCBzbGVlcHFfd2FpdCsweDRkDQpfY3Zfd2FpdCgpIGF0IF9jdl93YWl0KzB4MTdh DQp1c2JfcHJvY2VzcygpIGF0IHVzYl9wcm9jZXNzKzB4MTRlDQpmb3JrX2V4aXQoKSBhdCBm b3JrX2V4aXQrMHgxMmENCmZvcmtfdHJhbXBvbGluZSgpIGF0IGZvcmtfdHJhbXBvbGluZSsw eGUNCi0tLSB0cmFwIDAsIHJpcCA9IDAsIHJzcCA9IDB4ZmZmZmZmODAwMDE1NWNmMCwgcmJw ID0gMCAtLS0NCg0KVHJhY2luZyBjb21tYW5kIHVzYiBwaWQgMTQgdGlkIDEwMDA1OCB0ZCAw eGZmZmZmZjAwMDE3NDkwMDANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRjaCsweDEz OA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpzbGVlcHFfc3dpdGNoKCkgYXQg c2xlZXBxX3N3aXRjaCsweDEyMw0Kc2xlZXBxX3dhaXQoKSBhdCBzbGVlcHFfd2FpdCsweDRk DQpfY3Zfd2FpdCgpIGF0IF9jdl93YWl0KzB4MTdhDQp1c2JfcHJvY2VzcygpIGF0IHVzYl9w cm9jZXNzKzB4MTRlDQpmb3JrX2V4aXQoKSBhdCBmb3JrX2V4aXQrMHgxMmENCmZvcmtfdHJh bXBvbGluZSgpIGF0IGZvcmtfdHJhbXBvbGluZSsweGUNCi0tLSB0cmFwIDAsIHJpcCA9IDAs IHJzcCA9IDB4ZmZmZmZmODAwMDE1MGNmMCwgcmJwID0gMCAtLS0NCg0KVHJhY2luZyBjb21t YW5kIHVzYiBwaWQgMTQgdGlkIDEwMDA1NyB0ZCAweGZmZmZmZjAwMDE3NDk0NjANCnNjaGVk X3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dp dGNoKzB4MjFiDQpzbGVlcHFfc3dpdGNoKCkgYXQgc2xlZXBxX3N3aXRjaCsweDEyMw0Kc2xl ZXBxX3dhaXQoKSBhdCBzbGVlcHFfd2FpdCsweDRkDQpfY3Zfd2FpdCgpIGF0IF9jdl93YWl0 KzB4MTdhDQp1c2JfcHJvY2VzcygpIGF0IHVzYl9wcm9jZXNzKzB4MTRlDQpmb3JrX2V4aXQo KSBhdCBmb3JrX2V4aXQrMHgxMmENCmZvcmtfdHJhbXBvbGluZSgpIGF0IGZvcmtfdHJhbXBv bGluZSsweGUNCi0tLSB0cmFwIDAsIHJpcCA9IDAsIHJzcCA9IDB4ZmZmZmZmODAwMDE0OWNm MCwgcmJwID0gMCAtLS0NCg0KVHJhY2luZyBjb21tYW5kIHVzYiBwaWQgMTQgdGlkIDEwMDA1 NiB0ZCAweGZmZmZmZjAwMDE3NDk4YzANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRj aCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpzbGVlcHFfc3dpdGNo KCkgYXQgc2xlZXBxX3N3aXRjaCsweDEyMw0Kc2xlZXBxX3dhaXQoKSBhdCBzbGVlcHFfd2Fp dCsweDRkDQpfY3Zfd2FpdCgpIGF0IF9jdl93YWl0KzB4MTdhDQp1c2JfcHJvY2VzcygpIGF0 IHVzYl9wcm9jZXNzKzB4MTRlDQpmb3JrX2V4aXQoKSBhdCBmb3JrX2V4aXQrMHgxMmENCmZv cmtfdHJhbXBvbGluZSgpIGF0IGZvcmtfdHJhbXBvbGluZSsweGUNCi0tLSB0cmFwIDAsIHJp cCA9IDAsIHJzcCA9IDB4ZmZmZmZmODAwMDE0NGNmMCwgcmJwID0gMCAtLS0NCg0KVHJhY2lu ZyBjb21tYW5kIHVzYiBwaWQgMTQgdGlkIDEwMDA1NSB0ZCAweGZmZmZmZjAwMDE3NGEwMDAN CnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dpdGNoKCkgYXQg bWlfc3dpdGNoKzB4MjFiDQpzbGVlcHFfc3dpdGNoKCkgYXQgc2xlZXBxX3N3aXRjaCsweDEy Mw0Kc2xlZXBxX3dhaXQoKSBhdCBzbGVlcHFfd2FpdCsweDRkDQpfY3Zfd2FpdCgpIGF0IF9j dl93YWl0KzB4MTdhDQp1c2JfcHJvY2VzcygpIGF0IHVzYl9wcm9jZXNzKzB4MTRlDQpmb3Jr X2V4aXQoKSBhdCBmb3JrX2V4aXQrMHgxMmENCmZvcmtfdHJhbXBvbGluZSgpIGF0IGZvcmtf dHJhbXBvbGluZSsweGUNCi0tLSB0cmFwIDAsIHJpcCA9IDAsIHJzcCA9IDB4ZmZmZmZmODAw MDEzZmNmMCwgcmJwID0gMCAtLS0NCg0KVHJhY2luZyBjb21tYW5kIHVzYiBwaWQgMTQgdGlk IDEwMDA1NCB0ZCAweGZmZmZmZjAwMDE3NGE0NjANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVk X3N3aXRjaCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpzbGVlcHFf c3dpdGNoKCkgYXQgc2xlZXBxX3N3aXRjaCsweDEyMw0Kc2xlZXBxX3dhaXQoKSBhdCBzbGVl cHFfd2FpdCsweDRkDQpfY3Zfd2FpdCgpIGF0IF9jdl93YWl0KzB4MTdhDQp1c2JfcHJvY2Vz cygpIGF0IHVzYl9wcm9jZXNzKzB4MTRlDQpmb3JrX2V4aXQoKSBhdCBmb3JrX2V4aXQrMHgx MmENCmZvcmtfdHJhbXBvbGluZSgpIGF0IGZvcmtfdHJhbXBvbGluZSsweGUNCi0tLSB0cmFw IDAsIHJpcCA9IDAsIHJzcCA9IDB4ZmZmZmZmODAwMDEzYWNmMCwgcmJwID0gMCAtLS0NCg0K VHJhY2luZyBjb21tYW5kIHVzYiBwaWQgMTQgdGlkIDEwMDA1MyB0ZCAweGZmZmZmZjAwMDE3 NGE4YzANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dpdGNo KCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpzbGVlcHFfc3dpdGNoKCkgYXQgc2xlZXBxX3N3aXRj aCsweDEyMw0Kc2xlZXBxX3dhaXQoKSBhdCBzbGVlcHFfd2FpdCsweDRkDQpfY3Zfd2FpdCgp IGF0IF9jdl93YWl0KzB4MTdhDQp1c2JfcHJvY2VzcygpIGF0IHVzYl9wcm9jZXNzKzB4MTRl DQpmb3JrX2V4aXQoKSBhdCBmb3JrX2V4aXQrMHgxMmENCmZvcmtfdHJhbXBvbGluZSgpIGF0 IGZvcmtfdHJhbXBvbGluZSsweGUNCi0tLSB0cmFwIDAsIHJpcCA9IDAsIHJzcCA9IDB4ZmZm ZmZmODAwMDEzNGNmMCwgcmJwID0gMCAtLS0NCg0KVHJhY2luZyBjb21tYW5kIHVzYiBwaWQg MTQgdGlkIDEwMDA1MiB0ZCAweGZmZmZmZjAwMDE3NGIwMDANCnNjaGVkX3N3aXRjaCgpIGF0 IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpz bGVlcHFfc3dpdGNoKCkgYXQgc2xlZXBxX3N3aXRjaCsweDEyMw0Kc2xlZXBxX3dhaXQoKSBh dCBzbGVlcHFfd2FpdCsweDRkDQpfY3Zfd2FpdCgpIGF0IF9jdl93YWl0KzB4MTdhDQp1c2Jf cHJvY2VzcygpIGF0IHVzYl9wcm9jZXNzKzB4MTRlDQpmb3JrX2V4aXQoKSBhdCBmb3JrX2V4 aXQrMHgxMmENCmZvcmtfdHJhbXBvbGluZSgpIGF0IGZvcmtfdHJhbXBvbGluZSsweGUNCi0t LSB0cmFwIDAsIHJpcCA9IDAsIHJzcCA9IDB4ZmZmZmZmODAwMDEyZmNmMCwgcmJwID0gMCAt LS0NCg0KVHJhY2luZyBjb21tYW5kIHVzYiBwaWQgMTQgdGlkIDEwMDA1MSB0ZCAweGZmZmZm ZjAwMDE3NGI0NjANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlf c3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpzbGVlcHFfc3dpdGNoKCkgYXQgc2xlZXBx X3N3aXRjaCsweDEyMw0Kc2xlZXBxX3dhaXQoKSBhdCBzbGVlcHFfd2FpdCsweDRkDQpfY3Zf d2FpdCgpIGF0IF9jdl93YWl0KzB4MTdhDQp1c2JfcHJvY2VzcygpIGF0IHVzYl9wcm9jZXNz KzB4MTRlDQpmb3JrX2V4aXQoKSBhdCBmb3JrX2V4aXQrMHgxMmENCmZvcmtfdHJhbXBvbGlu ZSgpIGF0IGZvcmtfdHJhbXBvbGluZSsweGUNCi0tLSB0cmFwIDAsIHJpcCA9IDAsIHJzcCA9 IDB4ZmZmZmZmODAwMDEyYWNmMCwgcmJwID0gMCAtLS0NCg0KVHJhY2luZyBjb21tYW5kIHVz YiBwaWQgMTQgdGlkIDEwMDA1MCB0ZCAweGZmZmZmZjAwMDE3NGI4YzANCnNjaGVkX3N3aXRj aCgpIGF0IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4 MjFiDQpzbGVlcHFfc3dpdGNoKCkgYXQgc2xlZXBxX3N3aXRjaCsweDEyMw0Kc2xlZXBxX3dh aXQoKSBhdCBzbGVlcHFfd2FpdCsweDRkDQpfY3Zfd2FpdCgpIGF0IF9jdl93YWl0KzB4MTdh DQp1c2JfcHJvY2VzcygpIGF0IHVzYl9wcm9jZXNzKzB4MTRlDQpmb3JrX2V4aXQoKSBhdCBm b3JrX2V4aXQrMHgxMmENCmZvcmtfdHJhbXBvbGluZSgpIGF0IGZvcmtfdHJhbXBvbGluZSsw eGUNCi0tLSB0cmFwIDAsIHJpcCA9IDAsIHJzcCA9IDB4ZmZmZmZmODAwMDEyNWNmMCwgcmJw ID0gMCAtLS0NCg0KVHJhY2luZyBjb21tYW5kIHVzYiBwaWQgMTQgdGlkIDEwMDA0OCB0ZCAw eGZmZmZmZjAwMDE3MjE0NjANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRjaCsweDEz OA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpzbGVlcHFfc3dpdGNoKCkgYXQg c2xlZXBxX3N3aXRjaCsweDEyMw0Kc2xlZXBxX3dhaXQoKSBhdCBzbGVlcHFfd2FpdCsweDRk DQpfY3Zfd2FpdCgpIGF0IF9jdl93YWl0KzB4MTdhDQp1c2JfcHJvY2VzcygpIGF0IHVzYl9w cm9jZXNzKzB4MTRlDQpmb3JrX2V4aXQoKSBhdCBmb3JrX2V4aXQrMHgxMmENCmZvcmtfdHJh bXBvbGluZSgpIGF0IGZvcmtfdHJhbXBvbGluZSsweGUNCi0tLSB0cmFwIDAsIHJpcCA9IDAs IHJzcCA9IDB4ZmZmZmZmODAwMDExYWNmMCwgcmJwID0gMCAtLS0NCg0KVHJhY2luZyBjb21t YW5kIHVzYiBwaWQgMTQgdGlkIDEwMDA0NyB0ZCAweGZmZmZmZjAwMDE3MjE4YzANCnNjaGVk X3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dp dGNoKzB4MjFiDQpzbGVlcHFfc3dpdGNoKCkgYXQgc2xlZXBxX3N3aXRjaCsweDEyMw0Kc2xl ZXBxX3dhaXQoKSBhdCBzbGVlcHFfd2FpdCsweDRkDQpfY3Zfd2FpdCgpIGF0IF9jdl93YWl0 KzB4MTdhDQp1c2JfcHJvY2VzcygpIGF0IHVzYl9wcm9jZXNzKzB4MTRlDQpmb3JrX2V4aXQo KSBhdCBmb3JrX2V4aXQrMHgxMmENCmZvcmtfdHJhbXBvbGluZSgpIGF0IGZvcmtfdHJhbXBv bGluZSsweGUNCi0tLSB0cmFwIDAsIHJpcCA9IDAsIHJzcCA9IDB4ZmZmZmZmODAwMDExNWNm MCwgcmJwID0gMCAtLS0NCg0KVHJhY2luZyBjb21tYW5kIHVzYiBwaWQgMTQgdGlkIDEwMDA0 NiB0ZCAweGZmZmZmZjAwMDE3MjIwMDANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRj aCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpzbGVlcHFfc3dpdGNo KCkgYXQgc2xlZXBxX3N3aXRjaCsweDEyMw0Kc2xlZXBxX3dhaXQoKSBhdCBzbGVlcHFfd2Fp dCsweDRkDQpfY3Zfd2FpdCgpIGF0IF9jdl93YWl0KzB4MTdhDQp1c2JfcHJvY2VzcygpIGF0 IHVzYl9wcm9jZXNzKzB4MTRlDQpmb3JrX2V4aXQoKSBhdCBmb3JrX2V4aXQrMHgxMmENCmZv cmtfdHJhbXBvbGluZSgpIGF0IGZvcmtfdHJhbXBvbGluZSsweGUNCi0tLSB0cmFwIDAsIHJp cCA9IDAsIHJzcCA9IDB4ZmZmZmZmODAwMDExMGNmMCwgcmJwID0gMCAtLS0NCg0KVHJhY2lu ZyBjb21tYW5kIHVzYiBwaWQgMTQgdGlkIDEwMDA0NSB0ZCAweGZmZmZmZjAwMDE3MjI0NjAN CnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dpdGNoKCkgYXQg bWlfc3dpdGNoKzB4MjFiDQpzbGVlcHFfc3dpdGNoKCkgYXQgc2xlZXBxX3N3aXRjaCsweDEy Mw0Kc2xlZXBxX3dhaXQoKSBhdCBzbGVlcHFfd2FpdCsweDRkDQpfY3Zfd2FpdCgpIGF0IF9j dl93YWl0KzB4MTdhDQp1c2JfcHJvY2VzcygpIGF0IHVzYl9wcm9jZXNzKzB4MTRlDQpmb3Jr X2V4aXQoKSBhdCBmb3JrX2V4aXQrMHgxMmENCmZvcmtfdHJhbXBvbGluZSgpIGF0IGZvcmtf dHJhbXBvbGluZSsweGUNCi0tLSB0cmFwIDAsIHJpcCA9IDAsIHJzcCA9IDB4ZmZmZmZmODAw MDEwYmNmMCwgcmJwID0gMCAtLS0NCg0KVHJhY2luZyBjb21tYW5kIHVzYiBwaWQgMTQgdGlk IDEwMDA0MyB0ZCAweGZmZmZmZjAwMDE3MjQwMDANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVk X3N3aXRjaCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpzbGVlcHFf c3dpdGNoKCkgYXQgc2xlZXBxX3N3aXRjaCsweDEyMw0Kc2xlZXBxX3dhaXQoKSBhdCBzbGVl cHFfd2FpdCsweDRkDQpfY3Zfd2FpdCgpIGF0IF9jdl93YWl0KzB4MTdhDQp1c2JfcHJvY2Vz cygpIGF0IHVzYl9wcm9jZXNzKzB4MTRlDQpmb3JrX2V4aXQoKSBhdCBmb3JrX2V4aXQrMHgx MmENCmZvcmtfdHJhbXBvbGluZSgpIGF0IGZvcmtfdHJhbXBvbGluZSsweGUNCi0tLSB0cmFw IDAsIHJpcCA9IDAsIHJzcCA9IDB4ZmZmZmZmODAwMDEwMGNmMCwgcmJwID0gMCAtLS0NCg0K VHJhY2luZyBjb21tYW5kIHVzYiBwaWQgMTQgdGlkIDEwMDA0MiB0ZCAweGZmZmZmZjAwMDE3 MjQ0NjANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dpdGNo KCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpzbGVlcHFfc3dpdGNoKCkgYXQgc2xlZXBxX3N3aXRj aCsweDEyMw0Kc2xlZXBxX3dhaXQoKSBhdCBzbGVlcHFfd2FpdCsweDRkDQpfY3Zfd2FpdCgp IGF0IF9jdl93YWl0KzB4MTdhDQp1c2JfcHJvY2VzcygpIGF0IHVzYl9wcm9jZXNzKzB4MTRl DQpmb3JrX2V4aXQoKSBhdCBmb3JrX2V4aXQrMHgxMmENCmZvcmtfdHJhbXBvbGluZSgpIGF0 IGZvcmtfdHJhbXBvbGluZSsweGUNCi0tLSB0cmFwIDAsIHJpcCA9IDAsIHJzcCA9IDB4ZmZm ZmZmODAwMDBmYmNmMCwgcmJwID0gMCAtLS0NCg0KVHJhY2luZyBjb21tYW5kIHVzYiBwaWQg MTQgdGlkIDEwMDA0MSB0ZCAweGZmZmZmZjAwMDE3MjQ4YzANCnNjaGVkX3N3aXRjaCgpIGF0 IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpz bGVlcHFfc3dpdGNoKCkgYXQgc2xlZXBxX3N3aXRjaCsweDEyMw0Kc2xlZXBxX3dhaXQoKSBh dCBzbGVlcHFfd2FpdCsweDRkDQpfY3Zfd2FpdCgpIGF0IF9jdl93YWl0KzB4MTdhDQp1c2Jf cHJvY2VzcygpIGF0IHVzYl9wcm9jZXNzKzB4MTRlDQpmb3JrX2V4aXQoKSBhdCBmb3JrX2V4 aXQrMHgxMmENCmZvcmtfdHJhbXBvbGluZSgpIGF0IGZvcmtfdHJhbXBvbGluZSsweGUNCi0t LSB0cmFwIDAsIHJpcCA9IDAsIHJzcCA9IDB4ZmZmZmZmODAwMDBmNmNmMCwgcmJwID0gMCAt LS0NCg0KVHJhY2luZyBjb21tYW5kIHVzYiBwaWQgMTQgdGlkIDEwMDA0MCB0ZCAweGZmZmZm ZjAwMDE3MjUwMDANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlf c3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpzbGVlcHFfc3dpdGNoKCkgYXQgc2xlZXBx X3N3aXRjaCsweDEyMw0Kc2xlZXBxX3dhaXQoKSBhdCBzbGVlcHFfd2FpdCsweDRkDQpfY3Zf d2FpdCgpIGF0IF9jdl93YWl0KzB4MTdhDQp1c2JfcHJvY2VzcygpIGF0IHVzYl9wcm9jZXNz KzB4MTRlDQpmb3JrX2V4aXQoKSBhdCBmb3JrX2V4aXQrMHgxMmENCmZvcmtfdHJhbXBvbGlu ZSgpIGF0IGZvcmtfdHJhbXBvbGluZSsweGUNCi0tLSB0cmFwIDAsIHJpcCA9IDAsIHJzcCA9 IDB4ZmZmZmZmODAwMDBmMWNmMCwgcmJwID0gMCAtLS0NCg0KVHJhY2luZyBjb21tYW5kIHVz YiBwaWQgMTQgdGlkIDEwMDAzOSB0ZCAweGZmZmZmZjAwMDE2ZjUwMDANCnNjaGVkX3N3aXRj aCgpIGF0IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4 MjFiDQpzbGVlcHFfc3dpdGNoKCkgYXQgc2xlZXBxX3N3aXRjaCsweDEyMw0Kc2xlZXBxX3dh aXQoKSBhdCBzbGVlcHFfd2FpdCsweDRkDQpfY3Zfd2FpdCgpIGF0IF9jdl93YWl0KzB4MTdh DQp1c2JfcHJvY2VzcygpIGF0IHVzYl9wcm9jZXNzKzB4MTRlDQpmb3JrX2V4aXQoKSBhdCBm b3JrX2V4aXQrMHgxMmENCmZvcmtfdHJhbXBvbGluZSgpIGF0IGZvcmtfdHJhbXBvbGluZSsw eGUNCi0tLSB0cmFwIDAsIHJpcCA9IDAsIHJzcCA9IDB4ZmZmZmZmODAwMDBlYWNmMCwgcmJw ID0gMCAtLS0NCg0KVHJhY2luZyBjb21tYW5kIHVzYiBwaWQgMTQgdGlkIDEwMDAzOCB0ZCAw eGZmZmZmZjAwMDE2ZjU0NjANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRjaCsweDEz OA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpzbGVlcHFfc3dpdGNoKCkgYXQg c2xlZXBxX3N3aXRjaCsweDEyMw0Kc2xlZXBxX3dhaXQoKSBhdCBzbGVlcHFfd2FpdCsweDRk DQpfY3Zfd2FpdCgpIGF0IF9jdl93YWl0KzB4MTdhDQp1c2JfcHJvY2VzcygpIGF0IHVzYl9w cm9jZXNzKzB4MTRlDQpmb3JrX2V4aXQoKSBhdCBmb3JrX2V4aXQrMHgxMmENCmZvcmtfdHJh bXBvbGluZSgpIGF0IGZvcmtfdHJhbXBvbGluZSsweGUNCi0tLSB0cmFwIDAsIHJpcCA9IDAs IHJzcCA9IDB4ZmZmZmZmODAwMDBlNWNmMCwgcmJwID0gMCAtLS0NClRyYWNpbmcgY29tbWFu ZCB1c2IgcGlkIDE0IHRpZCAxMDAwMzcgdGQgMHhmZmZmZmYwMDAxNmY1OGMwDQpzY2hlZF9z d2l0Y2goKSBhdCBzY2hlZF9zd2l0Y2grMHgxMzgNCm1pX3N3aXRjaCgpIGF0IG1pX3N3aXRj aCsweDIxYg0Kc2xlZXBxX3N3aXRjaCgpIGF0IHNsZWVwcV9zd2l0Y2grMHgxMjMNCnNsZWVw cV93YWl0KCkgYXQgc2xlZXBxX3dhaXQrMHg0ZA0KX2N2X3dhaXQoKSBhdCBfY3Zfd2FpdCsw eDE3YQ0KdXNiX3Byb2Nlc3MoKSBhdCB1c2JfcHJvY2VzcysweDE0ZQ0KZm9ya19leGl0KCkg YXQgZm9ya19leGl0KzB4MTJhDQpmb3JrX3RyYW1wb2xpbmUoKSBhdCBmb3JrX3RyYW1wb2xp bmUrMHhlDQotLS0gdHJhcCAwLCByaXAgPSAwLCByc3AgPSAweGZmZmZmZjgwMDAwZTBjZjAs IHJicCA9IDAgLS0tDQoNClRyYWNpbmcgY29tbWFuZCB1c2IgcGlkIDE0IHRpZCAxMDAwMzYg dGQgMHhmZmZmZmYwMDAxNmY2MDAwDQpzY2hlZF9zd2l0Y2goKSBhdCBzY2hlZF9zd2l0Y2gr MHgxMzgNCm1pX3N3aXRjaCgpIGF0IG1pX3N3aXRjaCsweDIxYg0Kc2xlZXBxX3N3aXRjaCgp IGF0IHNsZWVwcV9zd2l0Y2grMHgxMjMNCnNsZWVwcV93YWl0KCkgYXQgc2xlZXBxX3dhaXQr MHg0ZA0KX2N2X3dhaXQoKSBhdCBfY3Zfd2FpdCsweDE3YQ0KdXNiX3Byb2Nlc3MoKSBhdCB1 c2JfcHJvY2VzcysweDE0ZQ0KZm9ya19leGl0KCkgYXQgZm9ya19leGl0KzB4MTJhDQpmb3Jr X3RyYW1wb2xpbmUoKSBhdCBmb3JrX3RyYW1wb2xpbmUrMHhlDQotLS0gdHJhcCAwLCByaXAg PSAwLCByc3AgPSAweGZmZmZmZjgwMDAwZGJjZjAsIHJicCA9IDAgLS0tDQoNClRyYWNpbmcg Y29tbWFuZCB1c2IgcGlkIDE0IHRpZCAxMDAwMzQgdGQgMHhmZmZmZmYwMDAxNmY2OGMwDQpz Y2hlZF9zd2l0Y2goKSBhdCBzY2hlZF9zd2l0Y2grMHgxMzgNCm1pX3N3aXRjaCgpIGF0IG1p X3N3aXRjaCsweDIxYg0Kc2xlZXBxX3N3aXRjaCgpIGF0IHNsZWVwcV9zd2l0Y2grMHgxMjMN CnNsZWVwcV93YWl0KCkgYXQgc2xlZXBxX3dhaXQrMHg0ZA0KX2N2X3dhaXQoKSBhdCBfY3Zf d2FpdCsweDE3YQ0KdXNiX3Byb2Nlc3MoKSBhdCB1c2JfcHJvY2VzcysweDE0ZQ0KZm9ya19l eGl0KCkgYXQgZm9ya19leGl0KzB4MTJhDQpmb3JrX3RyYW1wb2xpbmUoKSBhdCBmb3JrX3Ry YW1wb2xpbmUrMHhlDQotLS0gdHJhcCAwLCByaXAgPSAwLCByc3AgPSAweGZmZmZmZjgwMDAw ZDBjZjAsIHJicCA9IDAgLS0tDQoNClRyYWNpbmcgY29tbWFuZCB1c2IgcGlkIDE0IHRpZCAx MDAwMzMgdGQgMHhmZmZmZmYwMDAxNmY3MDAwDQpzY2hlZF9zd2l0Y2goKSBhdCBzY2hlZF9z d2l0Y2grMHgxMzgNCm1pX3N3aXRjaCgpIGF0IG1pX3N3aXRjaCsweDIxYg0Kc2xlZXBxX3N3 aXRjaCgpIGF0IHNsZWVwcV9zd2l0Y2grMHgxMjMNCnNsZWVwcV93YWl0KCkgYXQgc2xlZXBx X3dhaXQrMHg0ZA0KX2N2X3dhaXQoKSBhdCBfY3Zfd2FpdCsweDE3YQ0KdXNiX3Byb2Nlc3Mo KSBhdCB1c2JfcHJvY2VzcysweDE0ZQ0KZm9ya19leGl0KCkgYXQgZm9ya19leGl0KzB4MTJh DQpmb3JrX3RyYW1wb2xpbmUoKSBhdCBmb3JrX3RyYW1wb2xpbmUrMHhlDQotLS0gdHJhcCAw LCByaXAgPSAwLCByc3AgPSAweGZmZmZmZjgwMDAwY2JjZjAsIHJicCA9IDAgLS0tDQoNClRy YWNpbmcgY29tbWFuZCB1c2IgcGlkIDE0IHRpZCAxMDAwMzIgdGQgMHhmZmZmZmYwMDAxNmY3 NDYwDQpzY2hlZF9zd2l0Y2goKSBhdCBzY2hlZF9zd2l0Y2grMHgxMzgNCm1pX3N3aXRjaCgp IGF0IG1pX3N3aXRjaCsweDIxYg0Kc2xlZXBxX3N3aXRjaCgpIGF0IHNsZWVwcV9zd2l0Y2gr MHgxMjMNCnNsZWVwcV93YWl0KCkgYXQgc2xlZXBxX3dhaXQrMHg0ZA0KX2N2X3dhaXQoKSBh dCBfY3Zfd2FpdCsweDE3YQ0KdXNiX3Byb2Nlc3MoKSBhdCB1c2JfcHJvY2VzcysweDE0ZQ0K Zm9ya19leGl0KCkgYXQgZm9ya19leGl0KzB4MTJhDQpmb3JrX3RyYW1wb2xpbmUoKSBhdCBm b3JrX3RyYW1wb2xpbmUrMHhlDQotLS0gdHJhcCAwLCByaXAgPSAwLCByc3AgPSAweGZmZmZm ZjgwMDAwYzZjZjAsIHJicCA9IDAgLS0tDQoNClRyYWNpbmcgY29tbWFuZCB1c2IgcGlkIDE0 IHRpZCAxMDAwMzEgdGQgMHhmZmZmZmYwMDAxNmY3OGMwDQpzY2hlZF9zd2l0Y2goKSBhdCBz Y2hlZF9zd2l0Y2grMHgxMzgNCm1pX3N3aXRjaCgpIGF0IG1pX3N3aXRjaCsweDIxYg0Kc2xl ZXBxX3N3aXRjaCgpIGF0IHNsZWVwcV9zd2l0Y2grMHgxMjMNCnNsZWVwcV93YWl0KCkgYXQg c2xlZXBxX3dhaXQrMHg0ZA0KX2N2X3dhaXQoKSBhdCBfY3Zfd2FpdCsweDE3YQ0KdXNiX3By b2Nlc3MoKSBhdCB1c2JfcHJvY2VzcysweDE0ZQ0KZm9ya19leGl0KCkgYXQgZm9ya19leGl0 KzB4MTJhDQpmb3JrX3RyYW1wb2xpbmUoKSBhdCBmb3JrX3RyYW1wb2xpbmUrMHhlDQotLS0g dHJhcCAwLCByaXAgPSAwLCByc3AgPSAweGZmZmZmZjgwMDAwYzFjZjAsIHJicCA9IDAgLS0t DQoNClRyYWNpbmcgY29tbWFuZCB1c2IgcGlkIDE0IHRpZCAxMDAwMjkgdGQgMHhmZmZmZmYw MDAxNjI5OGMwDQpzY2hlZF9zd2l0Y2goKSBhdCBzY2hlZF9zd2l0Y2grMHgxMzgNCm1pX3N3 aXRjaCgpIGF0IG1pX3N3aXRjaCsweDIxYg0Kc2xlZXBxX3N3aXRjaCgpIGF0IHNsZWVwcV9z d2l0Y2grMHgxMjMNCnNsZWVwcV93YWl0KCkgYXQgc2xlZXBxX3dhaXQrMHg0ZA0KX2N2X3dh aXQoKSBhdCBfY3Zfd2FpdCsweDE3YQ0KdXNiX3Byb2Nlc3MoKSBhdCB1c2JfcHJvY2Vzcysw eDE0ZQ0KZm9ya19leGl0KCkgYXQgZm9ya19leGl0KzB4MTJhDQpmb3JrX3RyYW1wb2xpbmUo KSBhdCBmb3JrX3RyYW1wb2xpbmUrMHhlDQotLS0gdHJhcCAwLCByaXAgPSAwLCByc3AgPSAw eGZmZmZmZjgwMDAwYjZjZjAsIHJicCA9IDAgLS0tDQoNClRyYWNpbmcgY29tbWFuZCB1c2Ig cGlkIDE0IHRpZCAxMDAwMjggdGQgMHhmZmZmZmYwMDAxNjcwMDAwDQpzY2hlZF9zd2l0Y2go KSBhdCBzY2hlZF9zd2l0Y2grMHgxMzgNCm1pX3N3aXRjaCgpIGF0IG1pX3N3aXRjaCsweDIx Yg0Kc2xlZXBxX3N3aXRjaCgpIGF0IHNsZWVwcV9zd2l0Y2grMHgxMjMNCnNsZWVwcV93YWl0 KCkgYXQgc2xlZXBxX3dhaXQrMHg0ZA0KX2N2X3dhaXQoKSBhdCBfY3Zfd2FpdCsweDE3YQ0K dXNiX3Byb2Nlc3MoKSBhdCB1c2JfcHJvY2VzcysweDE0ZQ0KZm9ya19leGl0KCkgYXQgZm9y a19leGl0KzB4MTJhDQpmb3JrX3RyYW1wb2xpbmUoKSBhdCBmb3JrX3RyYW1wb2xpbmUrMHhl DQotLS0gdHJhcCAwLCByaXAgPSAwLCByc3AgPSAweGZmZmZmZjgwMDAwYjFjZjAsIHJicCA9 IDAgLS0tDQoNClRyYWNpbmcgY29tbWFuZCB1c2IgcGlkIDE0IHRpZCAxMDAwMjcgdGQgMHhm ZmZmZmYwMDAxNjcwNDYwDQpzY2hlZF9zd2l0Y2goKSBhdCBzY2hlZF9zd2l0Y2grMHgxMzgN Cm1pX3N3aXRjaCgpIGF0IG1pX3N3aXRjaCsweDIxYg0Kc2xlZXBxX3N3aXRjaCgpIGF0IHNs ZWVwcV9zd2l0Y2grMHgxMjMNCnNsZWVwcV93YWl0KCkgYXQgc2xlZXBxX3dhaXQrMHg0ZA0K X2N2X3dhaXQoKSBhdCBfY3Zfd2FpdCsweDE3YQ0KdXNiX3Byb2Nlc3MoKSBhdCB1c2JfcHJv Y2VzcysweDE0ZQ0KZm9ya19leGl0KCkgYXQgZm9ya19leGl0KzB4MTJhDQpmb3JrX3RyYW1w b2xpbmUoKSBhdCBmb3JrX3RyYW1wb2xpbmUrMHhlDQotLS0gdHJhcCAwLCByaXAgPSAwLCBy c3AgPSAweGZmZmZmZjgwMDAwYWNjZjAsIHJicCA9IDAgLS0tDQoNClRyYWNpbmcgY29tbWFu ZCB1c2IgcGlkIDE0IHRpZCAxMDAwMjYgdGQgMHhmZmZmZmYwMDAxNjcwOGMwDQpzY2hlZF9z d2l0Y2goKSBhdCBzY2hlZF9zd2l0Y2grMHgxMzgNCm1pX3N3aXRjaCgpIGF0IG1pX3N3aXRj aCsweDIxYg0Kc2xlZXBxX3N3aXRjaCgpIGF0IHNsZWVwcV9zd2l0Y2grMHgxMjMNCnNsZWVw cV93YWl0KCkgYXQgc2xlZXBxX3dhaXQrMHg0ZA0KX2N2X3dhaXQoKSBhdCBfY3Zfd2FpdCsw eDE3YQ0KdXNiX3Byb2Nlc3MoKSBhdCB1c2JfcHJvY2VzcysweDE0ZQ0KZm9ya19leGl0KCkg YXQgZm9ya19leGl0KzB4MTJhDQpmb3JrX3RyYW1wb2xpbmUoKSBhdCBmb3JrX3RyYW1wb2xp bmUrMHhlDQotLS0gdHJhcCAwLCByaXAgPSAwLCByc3AgPSAweGZmZmZmZjgwMDAwYTdjZjAs IHJicCA9IDAgLS0tDQoNClRyYWNpbmcgY29tbWFuZCB5YXJyb3cgcGlkIDEzIHRpZCAxMDAw MTMgdGQgMHhmZmZmZmYwMDAxNWEzOGMwDQpzY2hlZF9zd2l0Y2goKSBhdCBzY2hlZF9zd2l0 Y2grMHgxMzgNCm1pX3N3aXRjaCgpIGF0IG1pX3N3aXRjaCsweDIxYg0Kc2xlZXBxX3N3aXRj aCgpIGF0IHNsZWVwcV9zd2l0Y2grMHgxMjMNCnNsZWVwcV90aW1lZHdhaXQoKSBhdCBzbGVl cHFfdGltZWR3YWl0KzB4NGQNCl9zbGVlcCgpIGF0IF9zbGVlcCsweDM1Mg0KcmFuZG9tX2t0 aHJlYWQoKSBhdCByYW5kb21fa3RocmVhZCsweDFhZA0KZm9ya19leGl0KCkgYXQgZm9ya19l eGl0KzB4MTJhDQpmb3JrX3RyYW1wb2xpbmUoKSBhdCBmb3JrX3RyYW1wb2xpbmUrMHhlDQot LS0gdHJhcCAwLCByaXAgPSAwLCByc3AgPSAweGZmZmZmZjgwMDAwNWRjZjAsIHJicCA9IDAg LS0tDQoNClRyYWNpbmcgY29tbWFuZCBnX2Rvd24gcGlkIDQgdGlkIDEwMDAxMSB0ZCAweGZm ZmZmZjAwMDE1YTU0NjANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRjaCsweDEzOA0K bWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpzbGVlcHFfc3dpdGNoKCkgYXQgc2xl ZXBxX3N3aXRjaCsweDEyMw0Kc2xlZXBxX3dhaXQoKSBhdCBzbGVlcHFfd2FpdCsweDRkDQpf c2xlZXAoKSBhdCBfc2xlZXArMHgzNjkNCmdfaW9fc2NoZWR1bGVfZG93bigpIGF0IGdfaW9f c2NoZWR1bGVfZG93bisweDIyOA0KZ19kb3duX3Byb2Nib2R5KCkgYXQgZ19kb3duX3Byb2Ni b2R5KzB4NmYNCmZvcmtfZXhpdCgpIGF0IGZvcmtfZXhpdCsweDEyYQ0KZm9ya190cmFtcG9s aW5lKCkgYXQgZm9ya190cmFtcG9saW5lKzB4ZQ0KLS0tIHRyYXAgMCwgcmlwID0gMCwgcnNw ID0gMHhmZmZmZmY4MDAwMDUzY2YwLCByYnAgPSAwIC0tLQ0KDQpUcmFjaW5nIGNvbW1hbmQg Z191cCBwaWQgMyB0aWQgMTAwMDEwIHRkIDB4ZmZmZmZmMDAwMTVhNThjMA0Kc2NoZWRfc3dp dGNoKCkgYXQgc2NoZWRfc3dpdGNoKzB4MTM4DQptaV9zd2l0Y2goKSBhdCBtaV9zd2l0Y2gr MHgyMWINCnNsZWVwcV9zd2l0Y2goKSBhdCBzbGVlcHFfc3dpdGNoKzB4MTIzDQpzbGVlcHFf d2FpdCgpIGF0IHNsZWVwcV93YWl0KzB4NGQNCl9zbGVlcCgpIGF0IF9zbGVlcCsweDM2OQ0K Z19pb19zY2hlZHVsZV91cCgpIGF0IGdfaW9fc2NoZWR1bGVfdXArMHgxMGENCmdfdXBfcHJv Y2JvZHkoKSBhdCBnX3VwX3Byb2Nib2R5KzB4NmYNCmZvcmtfZXhpdCgpIGF0IGZvcmtfZXhp dCsweDEyYQ0KZm9ya190cmFtcG9saW5lKCkgYXQgZm9ya190cmFtcG9saW5lKzB4ZQ0KLS0t IHRyYXAgMCwgcmlwID0gMCwgcnNwID0gMHhmZmZmZmY4MDAwMDRlY2YwLCByYnAgPSAwIC0t LQ0KDQpUcmFjaW5nIGNvbW1hbmQgZ19ldmVudCBwaWQgMiB0aWQgMTAwMDA5IHRkIDB4ZmZm ZmZmMDAwMTU5MjQ2MA0Kc2NoZWRfc3dpdGNoKCkgYXQgc2NoZWRfc3dpdGNoKzB4MTM4DQpt aV9zd2l0Y2goKSBhdCBtaV9zd2l0Y2grMHgyMWINCnNsZWVwcV9zd2l0Y2goKSBhdCBzbGVl cHFfc3dpdGNoKzB4MTIzDQpzbGVlcHFfdGltZWR3YWl0KCkgYXQgc2xlZXBxX3RpbWVkd2Fp dCsweDRkDQpfc2xlZXAoKSBhdCBfc2xlZXArMHgzNTINCmdfZXZlbnRfcHJvY2JvZHkoKSBh dCBnX2V2ZW50X3Byb2Nib2R5KzB4YTENCmZvcmtfZXhpdCgpIGF0IGZvcmtfZXhpdCsweDEy YQ0KZm9ya190cmFtcG9saW5lKCkgYXQgZm9ya190cmFtcG9saW5lKzB4ZQ0KLS0tIHRyYXAg MCwgcmlwID0gMCwgcnNwID0gMHhmZmZmZmY4MDAwMDQ5Y2YwLCByYnAgPSAwIC0tLQ0KDQpU cmFjaW5nIGNvbW1hbmQgaW50ciBwaWQgMTIgdGlkIDEwMDA2OCB0ZCAweGZmZmZmZjAwMDE3 N2E0NjANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dpdGNo KCkgYXQgbWlfc3dpdGNoKzB4MjFiDQppdGhyZWFkX2xvb3AoKSBhdCBpdGhyZWFkX2xvb3Ar MHgyM2MNCmZvcmtfZXhpdCgpIGF0IGZvcmtfZXhpdCsweDEyYQ0KZm9ya190cmFtcG9saW5l KCkgYXQgZm9ya190cmFtcG9saW5lKzB4ZQ0KLS0tIHRyYXAgMCwgcmlwID0gMCwgcnNwID0g MHhmZmZmZmY4MDk0ODkzY2YwLCByYnAgPSAwIC0tLQ0KDQpUcmFjaW5nIGNvbW1hbmQgaW50 ciBwaWQgMTIgdGlkIDEwMDA2NyB0ZCAweGZmZmZmZjAwMDE3N2E4YzANCnNjaGVkX3N3aXRj aCgpIGF0IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4 MjFiDQppdGhyZWFkX2xvb3AoKSBhdCBpdGhyZWFkX2xvb3ArMHgyM2MNCmZvcmtfZXhpdCgp IGF0IGZvcmtfZXhpdCsweDEyYQ0KZm9ya190cmFtcG9saW5lKCkgYXQgZm9ya190cmFtcG9s aW5lKzB4ZQ0KLS0tIHRyYXAgMCwgcmlwID0gMCwgcnNwID0gMHhmZmZmZmY4MDk0ODhlY2Yw LCByYnAgPSAwIC0tLQ0KDQpUcmFjaW5nIGNvbW1hbmQgaW50ciBwaWQgMTIgdGlkIDEwMDA2 NiB0ZCAweGZmZmZmZjAwMDE3N2MwMDANCmZvcmtfdHJhbXBvbGluZSgpIGF0IGZvcmtfdHJh bXBvbGluZQ0KDQpUcmFjaW5nIGNvbW1hbmQgaW50ciBwaWQgMTIgdGlkIDEwMDA2NSB0ZCAw eGZmZmZmZjAwMDE3N2M0NjANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRjaCsweDEz OA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQppdGhyZWFkX2xvb3AoKSBhdCBp dGhyZWFkX2xvb3ArMHgyM2MNCmZvcmtfZXhpdCgpIGF0IGZvcmtfZXhpdCsweDEyYQ0KZm9y a190cmFtcG9saW5lKCkgYXQgZm9ya190cmFtcG9saW5lKzB4ZQ0KLS0tIHRyYXAgMCwgcmlw ID0gMCwgcnNwID0gMHhmZmZmZmY4MDAwMTc3Y2YwLCByYnAgPSAwIC0tLQ0KDQpUcmFjaW5n IGNvbW1hbmQgaW50ciBwaWQgMTIgdGlkIDEwMDA2MiB0ZCAweGZmZmZmZjAwMDE3N2Q0NjAN CnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dpdGNoKCkgYXQg bWlfc3dpdGNoKzB4MjFiDQppdGhyZWFkX2xvb3AoKSBhdCBpdGhyZWFkX2xvb3ArMHgyM2MN CmZvcmtfZXhpdCgpIGF0IGZvcmtfZXhpdCsweDEyYQ0KZm9ya190cmFtcG9saW5lKCkgYXQg Zm9ya190cmFtcG9saW5lKzB4ZQ0KLS0tIHRyYXAgMCwgcmlwID0gMCwgcnNwID0gMHhmZmZm ZmY4MDAwMTY0Y2YwLCByYnAgPSAwIC0tLQ0KDQpUcmFjaW5nIGNvbW1hbmQgaW50ciBwaWQg MTIgdGlkIDEwMDA0OSB0ZCAweGZmZmZmZjAwMDE3MjEwMDANCmZvcmtfdHJhbXBvbGluZSgp IGF0IGZvcmtfdHJhbXBvbGluZQ0KDQpUcmFjaW5nIGNvbW1hbmQgaW50ciBwaWQgMTIgdGlk IDEwMDA0NCB0ZCAweGZmZmZmZjAwMDE3MjI4YzANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVk X3N3aXRjaCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQppdGhyZWFk X2xvb3AoKSBhdCBpdGhyZWFkX2xvb3ArMHgyM2MNCmZvcmtfZXhpdCgpIGF0IGZvcmtfZXhp dCsweDEyYQ0KZm9ya190cmFtcG9saW5lKCkgYXQgZm9ya190cmFtcG9saW5lKzB4ZQ0KLS0t IHRyYXAgMCwgcmlwID0gMCwgcnNwID0gMHhmZmZmZmY4MDAwMTA2Y2YwLCByYnAgPSAwIC0t LQ0KDQpUcmFjaW5nIGNvbW1hbmQgaW50ciBwaWQgMTIgdGlkIDEwMDAzNSB0ZCAweGZmZmZm ZjAwMDE2ZjY0NjANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlf c3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQppdGhyZWFkX2xvb3AoKSBhdCBpdGhyZWFk X2xvb3ArMHgyM2MNCmZvcmtfZXhpdCgpIGF0IGZvcmtfZXhpdCsweDEyYQ0KZm9ya190cmFt cG9saW5lKCkgYXQgZm9ya190cmFtcG9saW5lKzB4ZQ0KLS0tIHRyYXAgMCwgcmlwID0gMCwg cnNwID0gMHhmZmZmZmY4MDAwMGQ2Y2YwLCByYnAgPSAwIC0tLQ0KDQpUcmFjaW5nIGNvbW1h bmQgaW50ciBwaWQgMTIgdGlkIDEwMDAzMCB0ZCAweGZmZmZmZjAwMDE2Mjk0NjANCmZvcmtf dHJhbXBvbGluZSgpIGF0IGZvcmtfdHJhbXBvbGluZQ0KDQpUcmFjaW5nIGNvbW1hbmQgaW50 ciBwaWQgMTIgdGlkIDEwMDAyNSB0ZCAweGZmZmZmZjAwMDE2NzEwMDANCmZvcmtfdHJhbXBv bGluZSgpIGF0IGZvcmtfdHJhbXBvbGluZQ0KDQpUcmFjaW5nIGNvbW1hbmQgaW50ciBwaWQg MTIgdGlkIDEwMDAyMyB0ZCAweGZmZmZmZjAwMDE2NzE4YzANCmZvcmtfdHJhbXBvbGluZSgp IGF0IGZvcmtfdHJhbXBvbGluZQ0KDQpUcmFjaW5nIGNvbW1hbmQgaW50ciBwaWQgMTIgdGlk IDEwMDAyMiB0ZCAweGZmZmZmZjAwMDE2MjYwMDANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVk X3N3aXRjaCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQppdGhyZWFk X2xvb3AoKSBhdCBpdGhyZWFkX2xvb3ArMHgyM2MNCmZvcmtfZXhpdCgpIGF0IGZvcmtfZXhp dCsweDEyYQ0KZm9ya190cmFtcG9saW5lKCkgYXQgZm9ya190cmFtcG9saW5lKzB4ZQ0KLS0t IHRyYXAgMCwgcmlwID0gMCwgcnNwID0gMHhmZmZmZmY4MDAwMDhhY2YwLCByYnAgPSAwIC0t LQ0KDQpUcmFjaW5nIGNvbW1hbmQgaW50ciBwaWQgMTIgdGlkIDEwMDAyMCB0ZCAweGZmZmZm ZjAwMDE2MjY4YzANCmZvcmtfdHJhbXBvbGluZSgpIGF0IGZvcmtfdHJhbXBvbGluZQ0KDQpU cmFjaW5nIGNvbW1hbmQgaW50ciBwaWQgMTIgdGlkIDEwMDAxOSB0ZCAweGZmZmZmZjAwMDE2 MjcwMDANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dpdGNo KCkgYXQgbWlfc3dpdGNoKzB4MjFiDQppdGhyZWFkX2xvb3AoKSBhdCBpdGhyZWFkX2xvb3Ar MHgyM2MNCmZvcmtfZXhpdCgpIGF0IGZvcmtfZXhpdCsweDEyYQ0KZm9ya190cmFtcG9saW5l KCkgYXQgZm9ya190cmFtcG9saW5lKzB4ZQ0KLS0tIHRyYXAgMCwgcmlwID0gMCwgcnNwID0g MHhmZmZmZmY4MDAwMDdiY2YwLCByYnAgPSAwIC0tLQ0KDQpUcmFjaW5nIGNvbW1hbmQgaW50 ciBwaWQgMTIgdGlkIDEwMDAxNCB0ZCAweGZmZmZmZjAwMDE1YTM0NjANCnNjaGVkX3N3aXRj aCgpIGF0IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4 MjFiDQppdGhyZWFkX2xvb3AoKSBhdCBpdGhyZWFkX2xvb3ArMHgyM2MNCmZvcmtfZXhpdCgp IGF0IGZvcmtfZXhpdCsweDEyYQ0KZm9ya190cmFtcG9saW5lKCkgYXQgZm9ya190cmFtcG9s aW5lKzB4ZQ0KLS0tIHRyYXAgMCwgcmlwID0gMCwgcnNwID0gMHhmZmZmZmY4MDAwMDYyY2Yw LCByYnAgPSAwIC0tLQ0KDQpUcmFjaW5nIGNvbW1hbmQgaW50ciBwaWQgMTIgdGlkIDEwMDAw OCB0ZCAweGZmZmZmZjAwMDE1OTI4YzANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRj aCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQppdGhyZWFkX2xvb3Ao KSBhdCBpdGhyZWFkX2xvb3ArMHgyM2MNCmZvcmtfZXhpdCgpIGF0IGZvcmtfZXhpdCsweDEy YQ0KZm9ya190cmFtcG9saW5lKCkgYXQgZm9ya190cmFtcG9saW5lKzB4ZQ0KLS0tIHRyYXAg MCwgcmlwID0gMCwgcnNwID0gMHhmZmZmZmY4MDAwMDQ0Y2YwLCByYnAgPSAwIC0tLQ0KDQpU cmFjaW5nIGNvbW1hbmQgaW50ciBwaWQgMTIgdGlkIDEwMDAwNyB0ZCAweGZmZmZmZjAwMDE1 YTAwMDANCmZvcmtfdHJhbXBvbGluZSgpIGF0IGZvcmtfdHJhbXBvbGluZQ0KDQpUcmFjaW5n IGNvbW1hbmQgaW50ciBwaWQgMTIgdGlkIDEwMDAwNiB0ZCAweGZmZmZmZjAwMDE1YTA0NjAN CnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dpdGNoKCkgYXQg bWlfc3dpdGNoKzB4MjFiDQppdGhyZWFkX2xvb3AoKSBhdCBpdGhyZWFkX2xvb3ArMHgyM2MN CmZvcmtfZXhpdCgpIGF0IGZvcmtfZXhpdCsweDEyYQ0KZm9ya190cmFtcG9saW5lKCkgYXQg Zm9ya190cmFtcG9saW5lKzB4ZQ0KLS0tIHRyYXAgMCwgcmlwID0gMCwgcnNwID0gMHhmZmZm ZmY4MDAwMDNhY2YwLCByYnAgPSAwIC0tLQ0KDQpUcmFjaW5nIGNvbW1hbmQgaW50ciBwaWQg MTIgdGlkIDEwMDAwNSB0ZCAweGZmZmZmZjAwMDE1YTA4YzANCnNjaGVkX3N3aXRjaCgpIGF0 IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpp dGhyZWFkX2xvb3AoKSBhdCBpdGhyZWFkX2xvb3ArMHgyM2MNCmZvcmtfZXhpdCgpIGF0IGZv cmtfZXhpdCsweDEyYQ0KZm9ya190cmFtcG9saW5lKCkgYXQgZm9ya190cmFtcG9saW5lKzB4 ZQ0KLS0tIHRyYXAgMCwgcmlwID0gMCwgcnNwID0gMHhmZmZmZmY4MDAwMDM1Y2YwLCByYnAg PSAwIC0tLQ0KDQpUcmFjaW5nIGNvbW1hbmQgaWRsZSBwaWQgMTEgdGlkIDEwMDAwNCB0ZCAw eGZmZmZmZjAwMDE1OTEwMDANCmNwdXN0b3BfaGFuZGxlcigpIGF0IGNwdXN0b3BfaGFuZGxl cisweDNiDQppcGlfbm1pX2hhbmRsZXIoKSBhdCBpcGlfbm1pX2hhbmRsZXIrMHgzMA0KdHJh cCgpIGF0IHRyYXArMHgxNzUNCm5taV9jYWxsdHJhcCgpIGF0IG5taV9jYWxsdHJhcCsweDgN Ci0tLSB0cmFwIDB4MTMsIHJpcCA9IDB4ZmZmZmZmZmY4MDQ0MGVjNiwgcnNwID0gMHhmZmZm ZmZmZjgwOGIzMjQwLCByYnAgPSAweGZmZmZmZjgwMDAwMzBiMjAgLS0tDQphY3BpX2NwdV9j MSgpIGF0IGFjcGlfY3B1X2MxKzB4Ng0KYWNwaV9jcHVfaWRsZSgpIGF0IGFjcGlfY3B1X2lk bGUrMHgyMTQNCnNjaGVkX2lkbGV0ZCgpIGF0IHNjaGVkX2lkbGV0ZCsweDEyOA0KZm9ya19l eGl0KCkgYXQgZm9ya19leGl0KzB4MTJhDQpmb3JrX3RyYW1wb2xpbmUoKSBhdCBmb3JrX3Ry YW1wb2xpbmUrMHhlDQotLS0gdHJhcCAwLCByaXAgPSAwLCByc3AgPSAweGZmZmZmZjgwMDAw MzBjZjAsIHJicCA9IDAgLS0tDQoNClRyYWNpbmcgY29tbWFuZCBpZGxlIHBpZCAxMSB0aWQg MTAwMDAzIHRkIDB4ZmZmZmZmMDAwMTU5MTQ2MA0Ka2RiX2VudGVyKCkgYXQga2RiX2VudGVy KzB4M2QNCnVhcnRfaW50cigpIGF0IHVhcnRfaW50cisweDJiZQ0KaW50cl9ldmVudF9oYW5k bGUoKSBhdCBpbnRyX2V2ZW50X2hhbmRsZSsweDY4DQppbnRyX2V4ZWN1dGVfaGFuZGxlcnMo KSBhdCBpbnRyX2V4ZWN1dGVfaGFuZGxlcnMrMHg1Zg0KbGFwaWNfaGFuZGxlX2ludHIoKSBh dCBsYXBpY19oYW5kbGVfaW50cisweDM3DQpYYXBpY19pc3IxKCkgYXQgWGFwaWNfaXNyMSsw eGE1DQotLS0gaW50ZXJydXB0LCByaXAgPSAweGZmZmZmZmZmODA0NDBlYzYsIHJzcCA9IDB4 ZmZmZmZmODAwMDAyYmIxMCwgcmJwID0gMHhmZmZmZmY4MDAwMDJiYjIwIC0tLQ0KYWNwaV9j cHVfYzEoKSBhdCBhY3BpX2NwdV9jMSsweDYNCmFjcGlfY3B1X2lkbGUoKSBhdCBhY3BpX2Nw dV9pZGxlKzB4MjE0DQpzY2hlZF9pZGxldGQoKSBhdCBzY2hlZF9pZGxldGQrMHgxMjgNCmZv cmtfZXhpdCgpIGF0IGZvcmtfZXhpdCsweDEyYQ0KZm9ya190cmFtcG9saW5lKCkgYXQgZm9y a190cmFtcG9saW5lKzB4ZQ0KLS0tIHRyYXAgMCwgcmlwID0gMCwgcnNwID0gMHhmZmZmZmY4 MDAwMDJiY2YwLCByYnAgPSAwIC0tLQ0KDQpUcmFjaW5nIGNvbW1hbmQgaW5pdCBwaWQgMSB0 aWQgMTAwMDAyIHRkIDB4ZmZmZmZmMDAwMTU5MThjMA0Kc2NoZWRfc3dpdGNoKCkgYXQgc2No ZWRfc3dpdGNoKzB4MTM4DQptaV9zd2l0Y2goKSBhdCBtaV9zd2l0Y2grMHgyMWINCnNsZWVw cV9zd2l0Y2goKSBhdCBzbGVlcHFfc3dpdGNoKzB4MTIzDQpzbGVlcHFfY2F0Y2hfc2lnbmFs cygpIGF0IHNsZWVwcV9jYXRjaF9zaWduYWxzKzB4MmE2DQpzbGVlcHFfd2FpdF9zaWcoKSBh dCBzbGVlcHFfd2FpdF9zaWcrMHgxNg0KX3NsZWVwKCkgYXQgX3NsZWVwKzB4MzFhDQprZXJu X3dhaXQoKSBhdCBrZXJuX3dhaXQrMHgzZTkNCndhaXQ0KCkgYXQgd2FpdDQrMHgzNQ0Kc3lz Y2FsbGVudGVyKCkgYXQgc3lzY2FsbGVudGVyKzB4ZjANCnN5c2NhbGwoKSBhdCBzeXNjYWxs KzB4NjANClhmYXN0X3N5c2NhbGwoKSBhdCBYZmFzdF9zeXNjYWxsKzB4ZTINCi0tLSBzeXNj YWxsICg3LCBGcmVlQlNEIEVMRjY0LCB3YWl0NCksIHJpcCA9IDB4NDBjNzFjLCByc3AgPSAw eDdmZmZmZmZmZTgwOCwgcmJwID0gMHg0MDFkNDAgLS0tDQoNClRyYWNpbmcgY29tbWFuZCBh dWRpdCBwaWQgMTAgdGlkIDEwMDAwMSB0ZCAweGZmZmZmZjAwMDE1OTIwMDANCnNjaGVkX3N3 aXRjaCgpIGF0IHNjaGVkX3N3aXRjaCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNo KzB4MjFiDQpzbGVlcHFfc3dpdGNoKCkgYXQgc2xlZXBxX3N3aXRjaCsweDEyMw0Kc2xlZXBx X3dhaXQoKSBhdCBzbGVlcHFfd2FpdCsweDRkDQpfY3Zfd2FpdCgpIGF0IF9jdl93YWl0KzB4 MTdhDQphdWRpdF93b3JrZXIoKSBhdCBhdWRpdF93b3JrZXIrMHg3Nw0KZm9ya19leGl0KCkg YXQgZm9ya19leGl0KzB4MTJhDQpmb3JrX3RyYW1wb2xpbmUoKSBhdCBmb3JrX3RyYW1wb2xp bmUrMHhlDQotLS0gdHJhcCAwLCByaXAgPSAwLCByc3AgPSAweGZmZmZmZjgwMDAwMjFjZjAs IHJicCA9IDAgLS0tDQoNClRyYWNpbmcgY29tbWFuZCBrZXJuZWwgcGlkIDAgdGlkIDEwMDA2 MyB0ZCAweGZmZmZmZjAwMDE3N2QwMDANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRj aCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpzbGVlcHFfc3dpdGNo KCkgYXQgc2xlZXBxX3N3aXRjaCsweDEyMw0Kc2xlZXBxX3dhaXQoKSBhdCBzbGVlcHFfd2Fp dCsweDRkDQptc2xlZXBfc3BpbigpIGF0IG1zbGVlcF9zcGluKzB4MjA5DQp0YXNrcXVldWVf dGhyZWFkX2xvb3AoKSBhdCB0YXNrcXVldWVfdGhyZWFkX2xvb3ArMHg2Mg0KZm9ya19leGl0 KCkgYXQgZm9ya19leGl0KzB4MTJhDQpmb3JrX3RyYW1wb2xpbmUoKSBhdCBmb3JrX3RyYW1w b2xpbmUrMHhlDQotLS0gdHJhcCAwLCByaXAgPSAwLCByc3AgPSAweGZmZmZmZjgwMDAxNmRj ZjAsIHJicCA9IDAgLS0tDQoNClRyYWNpbmcgY29tbWFuZCBrZXJuZWwgcGlkIDAgdGlkIDEw MDAyNCB0ZCAweGZmZmZmZjAwMDE2NzE0NjANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3 aXRjaCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpzbGVlcHFfc3dp dGNoKCkgYXQgc2xlZXBxX3N3aXRjaCsweDEyMw0Kc2xlZXBxX3dhaXQoKSBhdCBzbGVlcHFf d2FpdCsweDRkDQptc2xlZXBfc3BpbigpIGF0IG1zbGVlcF9zcGluKzB4MjA5DQp0YXNrcXVl dWVfdGhyZWFkX2xvb3AoKSBhdCB0YXNrcXVldWVfdGhyZWFkX2xvb3ArMHg2Mg0KZm9ya19l eGl0KCkgYXQgZm9ya19leGl0KzB4MTJhDQpmb3JrX3RyYW1wb2xpbmUoKSBhdCBmb3JrX3Ry YW1wb2xpbmUrMHhlDQotLS0gdHJhcCAwLCByaXAgPSAwLCByc3AgPSAweGZmZmZmZjgwMDAw OWNjZjAsIHJicCA9IDAgLS0tDQoNClRyYWNpbmcgY29tbWFuZCBrZXJuZWwgcGlkIDAgdGlk IDEwMDAyMSB0ZCAweGZmZmZmZjAwMDE2MjY0NjANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVk X3N3aXRjaCsweDEzOA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpzbGVlcHFf c3dpdGNoKCkgYXQgc2xlZXBxX3N3aXRjaCsweDEyMw0Kc2xlZXBxX3dhaXQoKSBhdCBzbGVl cHFfd2FpdCsweDRkDQpfc2xlZXAoKSBhdCBfc2xlZXArMHgzNjkNCnRhc2txdWV1ZV90aHJl YWRfbG9vcCgpIGF0IHRhc2txdWV1ZV90aHJlYWRfbG9vcCsweGI3DQpmb3JrX2V4aXQoKSBh dCBmb3JrX2V4aXQrMHgxMmENCmZvcmtfdHJhbXBvbGluZSgpIGF0IGZvcmtfdHJhbXBvbGlu ZSsweGUNCi0tLSB0cmFwIDAsIHJpcCA9IDAsIHJzcCA9IDB4ZmZmZmZmODAwMDA4NWNmMCwg cmJwID0gMCAtLS0NCg0KVHJhY2luZyBjb21tYW5kIGtlcm5lbCBwaWQgMCB0aWQgMTAwMDE4 IHRkIDB4ZmZmZmZmMDAwMTYyNzQ2MA0Kc2NoZWRfc3dpdGNoKCkgYXQgc2NoZWRfc3dpdGNo KzB4MTM4DQptaV9zd2l0Y2goKSBhdCBtaV9zd2l0Y2grMHgyMWINCnNsZWVwcV9zd2l0Y2go KSBhdCBzbGVlcHFfc3dpdGNoKzB4MTIzDQpzbGVlcHFfd2FpdCgpIGF0IHNsZWVwcV93YWl0 KzB4NGQNCm1zbGVlcF9zcGluKCkgYXQgbXNsZWVwX3NwaW4rMHgyMDkNCnRhc2txdWV1ZV90 aHJlYWRfbG9vcCgpIGF0IHRhc2txdWV1ZV90aHJlYWRfbG9vcCsweDYyDQpmb3JrX2V4aXQo KSBhdCBmb3JrX2V4aXQrMHgxMmENCmZvcmtfdHJhbXBvbGluZSgpIGF0IGZvcmtfdHJhbXBv bGluZSsweGUNCi0tLSB0cmFwIDAsIHJpcCA9IDAsIHJzcCA9IDB4ZmZmZmZmODAwMDA3NmNm MCwgcmJwID0gMCAtLS0NCg0KVHJhY2luZyBjb21tYW5kIGtlcm5lbCBwaWQgMCB0aWQgMTAw MDE3IHRkIDB4ZmZmZmZmMDAwMTYyNzhjMA0Kc2NoZWRfc3dpdGNoKCkgYXQgc2NoZWRfc3dp dGNoKzB4MTM4DQptaV9zd2l0Y2goKSBhdCBtaV9zd2l0Y2grMHgyMWINCnNsZWVwcV9zd2l0 Y2goKSBhdCBzbGVlcHFfc3dpdGNoKzB4MTIzDQpzbGVlcHFfd2FpdCgpIGF0IHNsZWVwcV93 YWl0KzB4NGQNCm1zbGVlcF9zcGluKCkgYXQgbXNsZWVwX3NwaW4rMHgyMDkNCnRhc2txdWV1 ZV90aHJlYWRfbG9vcCgpIGF0IHRhc2txdWV1ZV90aHJlYWRfbG9vcCsweDYyDQpmb3JrX2V4 aXQoKSBhdCBmb3JrX2V4aXQrMHgxMmENCmZvcmtfdHJhbXBvbGluZSgpIGF0IGZvcmtfdHJh bXBvbGluZSsweGUNCi0tLSB0cmFwIDAsIHJpcCA9IDAsIHJzcCA9IDB4ZmZmZmZmODAwMDA3 MWNmMCwgcmJwID0gMCAtLS0NCg0KVHJhY2luZyBjb21tYW5kIGtlcm5lbCBwaWQgMCB0aWQg MTAwMDE2IHRkIDB4ZmZmZmZmMDAwMTYyOTAwMA0Kc2NoZWRfc3dpdGNoKCkgYXQgc2NoZWRf c3dpdGNoKzB4MTM4DQptaV9zd2l0Y2goKSBhdCBtaV9zd2l0Y2grMHgyMWINCnNsZWVwcV9z d2l0Y2goKSBhdCBzbGVlcHFfc3dpdGNoKzB4MTIzDQpzbGVlcHFfd2FpdCgpIGF0IHNsZWVw cV93YWl0KzB4NGQNCm1zbGVlcF9zcGluKCkgYXQgbXNsZWVwX3NwaW4rMHgyMDkNCnRhc2tx dWV1ZV90aHJlYWRfbG9vcCgpIGF0IHRhc2txdWV1ZV90aHJlYWRfbG9vcCsweDYyDQpmb3Jr X2V4aXQoKSBhdCBmb3JrX2V4aXQrMHgxMmENCmZvcmtfdHJhbXBvbGluZSgpIGF0IGZvcmtf dHJhbXBvbGluZSsweGUNCi0tLSB0cmFwIDAsIHJpcCA9IDAsIHJzcCA9IDB4ZmZmZmZmODAw MDA2Y2NmMCwgcmJwID0gMCAtLS0NCg0KVHJhY2luZyBjb21tYW5kIGtlcm5lbCBwaWQgMCB0 aWQgMTAwMDE1IHRkIDB4ZmZmZmZmMDAwMTVhMzAwMA0Kc2NoZWRfc3dpdGNoKCkgYXQgc2No ZWRfc3dpdGNoKzB4MTM4DQptaV9zd2l0Y2goKSBhdCBtaV9zd2l0Y2grMHgyMWINCnNsZWVw cV9zd2l0Y2goKSBhdCBzbGVlcHFfc3dpdGNoKzB4MTIzDQpzbGVlcHFfd2FpdCgpIGF0IHNs ZWVwcV93YWl0KzB4NGQNCl9zbGVlcCgpIGF0IF9zbGVlcCsweDM2OQ0KdGFza3F1ZXVlX3Ro cmVhZF9sb29wKCkgYXQgdGFza3F1ZXVlX3RocmVhZF9sb29wKzB4YjcNCmZvcmtfZXhpdCgp IGF0IGZvcmtfZXhpdCsweDEyYQ0KZm9ya190cmFtcG9saW5lKCkgYXQgZm9ya190cmFtcG9s aW5lKzB4ZQ0KLS0tIHRyYXAgMCwgcmlwID0gMCwgcnNwID0gMHhmZmZmZmY4MDAwMDY3Y2Yw LCByYnAgPSAwIC0tLQ0KDQpUcmFjaW5nIGNvbW1hbmQga2VybmVsIHBpZCAwIHRpZCAxMDAw MTIgdGQgMHhmZmZmZmYwMDAxNWE1MDAwDQpzY2hlZF9zd2l0Y2goKSBhdCBzY2hlZF9zd2l0 Y2grMHgxMzgNCm1pX3N3aXRjaCgpIGF0IG1pX3N3aXRjaCsweDIxYg0Kc2xlZXBxX3N3aXRj aCgpIGF0IHNsZWVwcV9zd2l0Y2grMHgxMjMNCnNsZWVwcV93YWl0KCkgYXQgc2xlZXBxX3dh aXQrMHg0ZA0KX3NsZWVwKCkgYXQgX3NsZWVwKzB4MzY5DQp0YXNrcXVldWVfdGhyZWFkX2xv b3AoKSBhdCB0YXNrcXVldWVfdGhyZWFkX2xvb3ArMHhiNw0KZm9ya19leGl0KCkgYXQgZm9y a19leGl0KzB4MTJhDQpmb3JrX3RyYW1wb2xpbmUoKSBhdCBmb3JrX3RyYW1wb2xpbmUrMHhl DQotLS0gdHJhcCAwLCByaXAgPSAwLCByc3AgPSAweGZmZmZmZjgwMDAwNThjZjAsIHJicCA9 IDAgLS0tDQoNClRyYWNpbmcgY29tbWFuZCBrZXJuZWwgcGlkIDAgdGlkIDEwMDAwMCB0ZCAw eGZmZmZmZmZmODA2YTBmODANCnNjaGVkX3N3aXRjaCgpIGF0IHNjaGVkX3N3aXRjaCsweDEz OA0KbWlfc3dpdGNoKCkgYXQgbWlfc3dpdGNoKzB4MjFiDQpzbGVlcHFfc3dpdGNoKCkgYXQg c2xlZXBxX3N3aXRjaCsweDEyMw0Kc2xlZXBxX3dhaXQoKSBhdCBzbGVlcHFfd2FpdCsweDRk DQpfc2xlZXAoKSBhdCBfc2xlZXArMHgzNjkNCnNjaGVkdWxlcigpIGF0IHNjaGVkdWxlcisw eDIyOQ0KbWlfc3RhcnR1cCgpIGF0IG1pX3N0YXJ0dXArMHg3Nw0KYnRleHQoKSBhdCBidGV4 dCsweDJjDQpkYj4gcmVib290DQo= ------------B9A228150F548A Content-Type: application/octet-stream; name="dmesg.boot" Content-transfer-encoding: base64 Content-Disposition: attachment; filename="dmesg.boot" S0RCOiBkZWJ1Z2dlciBiYWNrZW5kczogZGRiDQpLREI6IGN1cnJlbnQgYmFja2VuZDogZGRi DQpDb3B5cmlnaHQgKGMpIDE5OTItMjAxMSBUaGUgRnJlZUJTRCBQcm9qZWN0Lg0KQ29weXJp Z2h0IChjKSAxOTc5LCAxOTgwLCAxOTgzLCAxOTg2LCAxOTg4LCAxOTg5LCAxOTkxLCAxOTky LCAxOTkzLCAxOTk0DQoJVGhlIFJlZ2VudHMgb2YgdGhlIFVuaXZlcnNpdHkgb2YgQ2FsaWZv cm5pYS4gQWxsIHJpZ2h0cyByZXNlcnZlZC4NCkZyZWVCU0QgaXMgYSByZWdpc3RlcmVkIHRy YWRlbWFyayBvZiBUaGUgRnJlZUJTRCBGb3VuZGF0aW9uLg0KRnJlZUJTRCA4LjItUFJFUkVM RUFTRSAjMDogU2F0IEphbiAgOCAxNDo0OToyMyBNU0sgMjAxMQ0KICAgIGxldkBibG9iLmhv bWUuc2VyZWJyeWFrb3Yuc3BiLnJ1Oi91c3Ivb2JqL3Vzci9zcmMvc3lzL0JMT0JESUFHIGFt ZDY0DQpXQVJOSU5HOiBXSVRORVNTIG9wdGlvbiBlbmFibGVkLCBleHBlY3QgcmVkdWNlZCBw ZXJmb3JtYW5jZS4NCldBUk5JTkc6IERJQUdOT1NUSUMgb3B0aW9uIGVuYWJsZWQsIGV4cGVj dCByZWR1Y2VkIHBlcmZvcm1hbmNlLg0KVGltZWNvdW50ZXIgImk4MjU0IiBmcmVxdWVuY3kg MTE5MzE4MiBIeiBxdWFsaXR5IDANCkNQVTogSW50ZWwoUikgQ29yZShUTSkyIER1byBDUFUg ICAgIEU0NTAwICBAIDIuMjBHSHogKDIyMDAuMDktTUh6IEs4LWNsYXNzIENQVSkNCiAgT3Jp Z2luID0gIkdlbnVpbmVJbnRlbCIgIElkID0gMHg2ZmQgIEZhbWlseSA9IDYgIE1vZGVsID0g ZiAgU3RlcHBpbmcgPSAxMw0KICBGZWF0dXJlcz0weGJmZWJmYmZmPEZQVSxWTUUsREUsUFNF LFRTQyxNU1IsUEFFLE1DRSxDWDgsQVBJQyxTRVAsTVRSUixQR0UsTUNBLENNT1YsUEFULFBT RTM2LENMRkxVU0gsRFRTLEFDUEksTU1YLEZYU1IsU1NFLFNTRTIsU1MsSFRULFRNLFBCRT4N CiAgRmVhdHVyZXMyPTB4ZTM5ZDxTU0UzLERURVM2NCxNT04sRFNfQ1BMLEVTVCxUTTIsU1NT RTMsQ1gxNix4VFBSLFBEQ00+DQogIEFNRCBGZWF0dXJlcz0weDIwMTAwODAwPFNZU0NBTEws TlgsTE0+DQogIEFNRCBGZWF0dXJlczI9MHgxPExBSEY+DQogIFRTQzogUC1zdGF0ZSBpbnZh cmlhbnQNCnJlYWwgbWVtb3J5ICA9IDIxNDc0ODM2NDggKDIwNDggTUIpDQphdmFpbCBtZW1v cnkgPSAyMDQ5NDE3MjE2ICgxOTU0IE1CKQ0KQUNQSSBBUElDIFRhYmxlOiA8QV9NX0lfIE9F TUFQSUMgPg0KRnJlZUJTRC9TTVA6IE11bHRpcHJvY2Vzc29yIFN5c3RlbSBEZXRlY3RlZDog MiBDUFVzDQpGcmVlQlNEL1NNUDogMSBwYWNrYWdlKHMpIHggMiBjb3JlKHMpDQogY3B1MCAo QlNQKTogQVBJQyBJRDogIDANCiBjcHUxIChBUCk6IEFQSUMgSUQ6ICAxDQppb2FwaWMwIDxW ZXJzaW9uIDIuMD4gaXJxcyAwLTIzIG9uIG1vdGhlcmJvYXJkDQprYmQxIGF0IGtiZG11eDAN CmFjcGkwOiA8QV9NX0lfIE9FTVhTRFQ+IG9uIG1vdGhlcmJvYXJkDQphY3BpMDogW0lUSFJF QURdDQphY3BpMDogUG93ZXIgQnV0dG9uIChmaXhlZCkNCmFjcGkwOiByZXNlcnZhdGlvbiBv ZiAwLCBhMDAwMCAoMykgZmFpbGVkDQphY3BpMDogcmVzZXJ2YXRpb24gb2YgMTAwMDAwLCA3 ZmMwMDAwMCAoMykgZmFpbGVkDQpUaW1lY291bnRlciAiQUNQSS1mYXN0IiBmcmVxdWVuY3kg MzU3OTU0NSBIeiBxdWFsaXR5IDEwMDANCmFjcGlfdGltZXIwOiA8MjQtYml0IHRpbWVyIGF0 IDMuNTc5NTQ1TUh6PiBwb3J0IDB4ODA4LTB4ODBiIG9uIGFjcGkwDQpjcHUwOiA8QUNQSSBD UFU+IG9uIGFjcGkwDQpBQ1BJIFdhcm5pbmc6IEluY29ycmVjdCBjaGVja3N1bSBpbiB0YWJs ZSBbT0VNQl0gLSAweENCLCBzaG91bGQgYmUgMHhDNiAoMjAxMDEwMTMvdGJ1dGlscy0zNTQp DQpjcHUxOiA8QUNQSSBDUFU+IG9uIGFjcGkwDQpwY2liMDogPEFDUEkgSG9zdC1QQ0kgYnJp ZGdlPiBwb3J0IDB4Y2Y4LTB4Y2ZmIG9uIGFjcGkwDQpwY2kwOiA8QUNQSSBQQ0kgYnVzPiBv biBwY2liMA0KdmdhcGNpMDogPFZHQS1jb21wYXRpYmxlIGRpc3BsYXk+IHBvcnQgMHhlYzAw LTB4ZWMwNyBtZW0gMHhmZWE4MDAwMC0weGZlYWZmZmZmLDB4ZDAwMDAwMDAtMHhkZmZmZmZm ZiwweGZlOTAwMDAwLTB4ZmU5ZmZmZmYgaXJxIDE2IGF0IGRldmljZSAyLjAgb24gcGNpMA0K cGNpMDogPHNpbXBsZSBjb21tcz4gYXQgZGV2aWNlIDMuMCAobm8gZHJpdmVyIGF0dGFjaGVk KQ0KcGNpMDogPG1hc3Mgc3RvcmFnZSwgQVRBPiBhdCBkZXZpY2UgMy4yIChubyBkcml2ZXIg YXR0YWNoZWQpDQpwY2kwOiA8c2ltcGxlIGNvbW1zLCBVQVJUPiBhdCBkZXZpY2UgMy4zIChu byBkcml2ZXIgYXR0YWNoZWQpDQplbTA6IDxJbnRlbChSKSBQUk8vMTAwMCBOZXR3b3JrIENv bm5lY3Rpb24gNy4xLjg+IHBvcnQgMHhkYzAwLTB4ZGMxZiBtZW0gMHhmZWE0MDAwMC0weGZl YTVmZmZmLDB4ZmVhNzkwMDAtMHhmZWE3OWZmZiBpcnEgMjAgYXQgZGV2aWNlIDI1LjAgb24g cGNpMA0KZW0wOiBObyBNU0kvTVNJWCB1c2luZyBhIExlZ2FjeSBJUlENCmVtMDogW0ZJTFRF Ul0NCmVtMDogRXRoZXJuZXQgYWRkcmVzczogMDA6MWU6OGM6NzU6MDM6MGQNCnVoY2kwOiA8 SW50ZWwgODI4MDFJIChJQ0g5KSBVU0IgY29udHJvbGxlcj4gcG9ydCAweGQ0ODAtMHhkNDlm IGlycSAxNiBhdCBkZXZpY2UgMjYuMCBvbiBwY2kwDQp1aGNpMDogW0lUSFJFQURdDQp1c2J1 czA6IDxJbnRlbCA4MjgwMUkgKElDSDkpIFVTQiBjb250cm9sbGVyPiBvbiB1aGNpMA0KdWhj aTE6IDxJbnRlbCA4MjgwMUkgKElDSDkpIFVTQiBjb250cm9sbGVyPiBwb3J0IDB4ZDgwMC0w eGQ4MWYgaXJxIDIxIGF0IGRldmljZSAyNi4xIG9uIHBjaTANCnVoY2kxOiBbSVRIUkVBRF0N CnVzYnVzMTogPEludGVsIDgyODAxSSAoSUNIOSkgVVNCIGNvbnRyb2xsZXI+IG9uIHVoY2kx DQp1aGNpMjogPEludGVsIDgyODAxSSAoSUNIOSkgVVNCIGNvbnRyb2xsZXI+IHBvcnQgMHhk ODgwLTB4ZDg5ZiBpcnEgMTggYXQgZGV2aWNlIDI2LjIgb24gcGNpMA0KdWhjaTI6IFtJVEhS RUFEXQ0KdXNidXMyOiA8SW50ZWwgODI4MDFJIChJQ0g5KSBVU0IgY29udHJvbGxlcj4gb24g dWhjaTINCmVoY2kwOiA8SW50ZWwgODI4MDFJIChJQ0g5KSBVU0IgMi4wIGNvbnRyb2xsZXI+ IG1lbSAweGZlYTdhODAwLTB4ZmVhN2FiZmYgaXJxIDE4IGF0IGRldmljZSAyNi43IG9uIHBj aTANCmVoY2kwOiBbSVRIUkVBRF0NCnVzYnVzMzogRUhDSSB2ZXJzaW9uIDEuMA0KdXNidXMz OiA8SW50ZWwgODI4MDFJIChJQ0g5KSBVU0IgMi4wIGNvbnRyb2xsZXI+IG9uIGVoY2kwDQp1 aGNpMzogPEludGVsIDgyODAxSSAoSUNIOSkgVVNCIGNvbnRyb2xsZXI+IHBvcnQgMHhkMDAw LTB4ZDAxZiBpcnEgMjMgYXQgZGV2aWNlIDI5LjAgb24gcGNpMA0KdWhjaTM6IFtJVEhSRUFE XQ0KdXNidXM0OiA8SW50ZWwgODI4MDFJIChJQ0g5KSBVU0IgY29udHJvbGxlcj4gb24gdWhj aTMNCnVoY2k0OiA8SW50ZWwgODI4MDFJIChJQ0g5KSBVU0IgY29udHJvbGxlcj4gcG9ydCAw eGQwODAtMHhkMDlmIGlycSAxOSBhdCBkZXZpY2UgMjkuMSBvbiBwY2kwDQp1aGNpNDogW0lU SFJFQURdDQp1c2J1czU6IDxJbnRlbCA4MjgwMUkgKElDSDkpIFVTQiBjb250cm9sbGVyPiBv biB1aGNpNA0KdWhjaTU6IDxJbnRlbCA4MjgwMUkgKElDSDkpIFVTQiBjb250cm9sbGVyPiBw b3J0IDB4ZDQwMC0weGQ0MWYgaXJxIDE4IGF0IGRldmljZSAyOS4yIG9uIHBjaTANCnVoY2k1 OiBbSVRIUkVBRF0NCnVzYnVzNjogPEludGVsIDgyODAxSSAoSUNIOSkgVVNCIGNvbnRyb2xs ZXI+IG9uIHVoY2k1DQplaGNpMTogPEludGVsIDgyODAxSSAoSUNIOSkgVVNCIDIuMCBjb250 cm9sbGVyPiBtZW0gMHhmZWE3YTQwMC0weGZlYTdhN2ZmIGlycSAyMyBhdCBkZXZpY2UgMjku NyBvbiBwY2kwDQplaGNpMTogW0lUSFJFQURdDQp1c2J1czc6IEVIQ0kgdmVyc2lvbiAxLjAN CnVzYnVzNzogPEludGVsIDgyODAxSSAoSUNIOSkgVVNCIDIuMCBjb250cm9sbGVyPiBvbiBl aGNpMQ0KcGNpYjE6IDxBQ1BJIFBDSS1QQ0kgYnJpZGdlPiBhdCBkZXZpY2UgMzAuMCBvbiBw Y2kwDQpwY2kxOiA8QUNQSSBQQ0kgYnVzPiBvbiBwY2liMQ0KZndvaGNpMDogPEx1Y2VudCBG VzMyMi8zMjM+IG1lbSAweGZlYmZmMDAwLTB4ZmViZmZmZmYgaXJxIDIwIGF0IGRldmljZSAy LjAgb24gcGNpMQ0KZndvaGNpMDogW0lUSFJFQURdDQpmd29oY2kwOiBPSENJIHZlcnNpb24g MS4wIChST009MSkNCmZ3b2hjaTA6IE5vLiBvZiBJc29jaHJvbm91cyBjaGFubmVscyBpcyA4 Lg0KZndvaGNpMDogRVVJNjQgMDA6MWU6OGM6MDA6MDA6MWE6NWY6ZGENCmZ3b2hjaTA6IFBo eSAxMzk0YSBhdmFpbGFibGUgUzQwMCwgMiBwb3J0cy4NCmZ3b2hjaTA6IExpbmsgUzQwMCwg bWF4X3JlYyAyMDQ4IGJ5dGVzLg0KZmlyZXdpcmUwOiA8SUVFRTEzOTQoRmlyZVdpcmUpIGJ1 cz4gb24gZndvaGNpMA0KZndvaGNpMDogSW5pdGlhdGUgYnVzIHJlc2V0DQpmd29oY2kwOiBm d29oY2lfaW50cl9jb3JlOiBCVVMgcmVzZXQNCmZ3b2hjaTA6IGZ3b2hjaV9pbnRyX2NvcmU6 IG5vZGVfaWQ9MHgwMDAwMDAwMCwgU2VsZklEIENvdW50PTEsIENZQ0xFTUFTVEVSIG1vZGUN CmlzYWIwOiA8UENJLUlTQSBicmlkZ2U+IGF0IGRldmljZSAzMS4wIG9uIHBjaTANCmlzYTA6 IDxJU0EgYnVzPiBvbiBpc2FiMA0KYWhjaTA6IDxJbnRlbCBJQ0g5IEFIQ0kgU0FUQSBjb250 cm9sbGVyPiBwb3J0IDB4Yzg4MC0weGM4ODcsMHhjODAwLTB4YzgwMywweGM0ODAtMHhjNDg3 LDB4YzQwMC0weGM0MDMsMHhjMDgwLTB4YzA5ZiBtZW0gMHhmZWE3ODgwMC0weGZlYTc4ZmZm IGlycSAyMiBhdCBkZXZpY2UgMzEuMiBvbiBwY2kwDQphaGNpMDogW0lUSFJFQURdDQphaGNp MDogQUhDSSB2MS4yMCB3aXRoIDYgM0dicHMgcG9ydHMsIFBvcnQgTXVsdGlwbGllciBzdXBw b3J0ZWQNCmFoY2ljaDA6IDxBSENJIGNoYW5uZWw+IGF0IGNoYW5uZWwgMCBvbiBhaGNpMA0K YWhjaWNoMDogW0lUSFJFQURdDQphaGNpY2gxOiA8QUhDSSBjaGFubmVsPiBhdCBjaGFubmVs IDEgb24gYWhjaTANCmFoY2ljaDE6IFtJVEhSRUFEXQ0KYWhjaWNoMjogPEFIQ0kgY2hhbm5l bD4gYXQgY2hhbm5lbCAyIG9uIGFoY2kwDQphaGNpY2gyOiBbSVRIUkVBRF0NCmFoY2ljaDM6 IDxBSENJIGNoYW5uZWw+IGF0IGNoYW5uZWwgMyBvbiBhaGNpMA0KYWhjaWNoMzogW0lUSFJF QURdDQphaGNpY2g0OiA8QUhDSSBjaGFubmVsPiBhdCBjaGFubmVsIDQgb24gYWhjaTANCmFo Y2ljaDQ6IFtJVEhSRUFEXQ0KYWhjaWNoNTogPEFIQ0kgY2hhbm5lbD4gYXQgY2hhbm5lbCA1 IG9uIGFoY2kwDQphaGNpY2g1OiBbSVRIUkVBRF0NCmljaHNtYjA6IDxJbnRlbCA4MjgwMUkg KElDSDkpIFNNQnVzIGNvbnRyb2xsZXI+IHBvcnQgMHg0MDAtMHg0MWYgbWVtIDB4ZmVhN2Ew MDAtMHhmZWE3YTBmZiBpcnEgMTggYXQgZGV2aWNlIDMxLjMgb24gcGNpMA0KaWNoc21iMDog W0lUSFJFQURdDQpzbWJ1czA6IDxTeXN0ZW0gTWFuYWdlbWVudCBCdXM+IG9uIGljaHNtYjAN CnNtYjA6IDxTTUJ1cyBnZW5lcmljIEkvTz4gb24gc21idXMwDQpwY2kwOiA8ZGFzcD4gYXQg ZGV2aWNlIDMxLjYgKG5vIGRyaXZlciBhdHRhY2hlZCkNCmFjcGlfYnV0dG9uMDogPFBvd2Vy IEJ1dHRvbj4gb24gYWNwaTANCmF0cnRjMDogPEFUIHJlYWx0aW1lIGNsb2NrPiBwb3J0IDB4 NzAtMHg3MSBpcnEgOCBvbiBhY3BpMA0KcHBjMDogPFBhcmFsbGVsIHBvcnQ+IHBvcnQgMHgz NzgtMHgzN2YsMHg3NzgtMHg3N2YgaXJxIDcgZHJxIDMgb24gYWNwaTANCnBwYzA6IFNNQy1s aWtlIGNoaXBzZXQgKEVDUC9FUFAvUFMyL05JQkJMRSkgaW4gQ09NUEFUSUJMRSBtb2RlDQpw cGMwOiBGSUZPIHdpdGggMTYvMTYvOSBieXRlcyB0aHJlc2hvbGQNCnBwYzA6IFtJVEhSRUFE XQ0KcHBidXMwOiA8UGFyYWxsZWwgcG9ydCBidXM+IG9uIHBwYzANCmFjcGlfaHBldDA6IDxI aWdoIFByZWNpc2lvbiBFdmVudCBUaW1lcj4gaW9tZW0gMHhmZWQwMDAwMC0weGZlZDAwM2Zm IG9uIGFjcGkwDQpUaW1lY291bnRlciAiSFBFVCIgZnJlcXVlbmN5IDE0MzE4MTgwIEh6IHF1 YWxpdHkgOTAwDQp1YXJ0MDogPDE2NTUwIG9yIGNvbXBhdGlibGU+IHBvcnQgMHgzZjgtMHgz ZmYgaXJxIDQgZmxhZ3MgMHgxMCBvbiBhY3BpMA0KdWFydDA6IFtGSUxURVJdDQp1YXJ0MDog Y29uc29sZSAoMTE1MjAwLG4sOCwxKQ0KYXRrYmRjMDogPEtleWJvYXJkIGNvbnRyb2xsZXIg KGk4MDQyKT4gcG9ydCAweDYwLDB4NjQgaXJxIDEgb24gYWNwaTANCmF0a2JkMDogPEFUIEtl eWJvYXJkPiBpcnEgMSBvbiBhdGtiZGMwDQprYmQwIGF0IGF0a2JkMA0KYXRrYmQwOiBbR0lB TlQtTE9DS0VEXQ0KYXRrYmQwOiBbSVRIUkVBRF0NCm9ybTA6IDxJU0EgT3B0aW9uIFJPTT4g YXQgaW9tZW0gMHhjYjgwMC0weGNjN2ZmIG9uIGlzYTANCnNjMDogPFN5c3RlbSBjb25zb2xl PiBhdCBmbGFncyAweDEwMCBvbiBpc2EwDQpzYzA6IFZHQSA8MTYgdmlydHVhbCBjb25zb2xl cywgZmxhZ3M9MHgzMDA+DQp2Z2EwOiA8R2VuZXJpYyBJU0EgVkdBPiBhdCBwb3J0IDB4M2Mw LTB4M2RmIGlvbWVtIDB4YTAwMDAtMHhiZmZmZiBvbiBpc2EwDQpjb3JldGVtcDA6IDxDUFUg T24tRGllIFRoZXJtYWwgU2Vuc29ycz4gb24gY3B1MA0KZXN0MDogPEVuaGFuY2VkIFNwZWVk U3RlcCBGcmVxdWVuY3kgQ29udHJvbD4gb24gY3B1MA0KcDR0Y2MwOiA8Q1BVIEZyZXF1ZW5j eSBUaGVybWFsIENvbnRyb2w+IG9uIGNwdTANCmNvcmV0ZW1wMTogPENQVSBPbi1EaWUgVGhl cm1hbCBTZW5zb3JzPiBvbiBjcHUxDQplc3QxOiA8RW5oYW5jZWQgU3BlZWRTdGVwIEZyZXF1 ZW5jeSBDb250cm9sPiBvbiBjcHUxDQpwNHRjYzE6IDxDUFUgRnJlcXVlbmN5IFRoZXJtYWwg Q29udHJvbD4gb24gY3B1MQ0KVGltZWNvdW50ZXJzIHRpY2sgZXZlcnkgMS4wMDAgbXNlYw0K ZmlyZXdpcmUwOiAxIG5vZGVzLCBtYXhob3AgPD0gMCBjYWJsZSBJUk0gaXJtKDApICAobWUp IA0KZmlyZXdpcmUwOiBidXMgbWFuYWdlciAwIA0KdXNidXMwOiAxMk1icHMgRnVsbCBTcGVl ZCBVU0IgdjEuMA0KdXNidXMxOiAxMk1icHMgRnVsbCBTcGVlZCBVU0IgdjEuMA0KdXNidXMy OiAxMk1icHMgRnVsbCBTcGVlZCBVU0IgdjEuMA0KdXNidXMzOiA0ODBNYnBzIEhpZ2ggU3Bl ZWQgVVNCIHYyLjANCnVzYnVzNDogMTJNYnBzIEZ1bGwgU3BlZWQgVVNCIHYxLjANCnVzYnVz NTogMTJNYnBzIEZ1bGwgU3BlZWQgVVNCIHYxLjANCnVzYnVzNjogMTJNYnBzIEZ1bGwgU3Bl ZWQgVVNCIHYxLjANCnVzYnVzNzogNDgwTWJwcyBIaWdoIFNwZWVkIFVTQiB2Mi4wDQpHRU9N X1JBSUQ1OiBNb2R1bGUgbG9hZGVkLCB2ZXJzaW9uIDEuMC4yMDExMDEwNy4zNCAocmV2IDA2 MzFlYWNkYjU3NSkNCmFkYTAgYXQgYWhjaWNoMCBidXMgMCBzY2J1czAgdGFyZ2V0IDAgbHVu IDANCmFkYTA6IDxTQU1TVU5HIEhEMzIxS0ogQ1AxMDAtMTA+IEFUQS04dWdlbjAuMTogPElu dGVsPiBhdCB1c2J1czANCnVodWIwOiA8SW50ZWwgVUhDSSByb290IEhVQiwgY2xhc3MgOS8w LCByZXYgMS4wMC8xLjAwLCBhZGRyIDE+IG9uIHVzYnVzMA0KRXhwZW5zaXZlIHRpbWVvdXQo OSkgZnVuY3Rpb246IDB4ZmZmZmZmZmY4MDJmNzRmMCgweGZmZmZmZjAwMDE2NzAwMDApIDAu MDExNTYwNDA3IHMNCnVnZW4xLjE6IDxJbnRlbD4gYXQgdXNidXMxDQp1aHViMTogPEludGVs IFVIQ0kgcm9vdCBIVUIsIGNsYXNzIDkvMCwgcmV2IDEuMDAvMS4wMCwgYWRkciAxPiBvbiB1 c2J1czENCnVnZW4yLjE6IDxJbnRlbD4gYXQgdXNidXMyDQp1aHViMjogPEludGVsIFVIQ0kg cm9vdCBIVUIsIGNsYXNzIDkvMCwgcmV2IDEuMDAvMS4wMCwgYWRkciAxPiBvbiB1c2J1czIN CnVnZW4zLjE6IDxJbnRlbD4gYXQgdXNidXMzDQp1aHViMzogPEludGVsIEVIQ0kgcm9vdCBI VUIsIGNsYXNzIDkvMCwgcmV2IDIuMDAvMS4wMCwgYWRkciAxPiBvbiB1c2J1czMNCnVnZW40 LjE6IDxJbnRlbD4gYXQgdXNidXM0DQp1aHViNDogPEludGVsIFVIQ0kgcm9vdCBIVUIsIGNs YXNzIDkvMCwgcmV2IDEuMDAvMS4wMCwgYWRkciAxPiBvbiB1c2J1czQNCnVnZW41LjE6IDxJ bnRlbD4gYXQgdXNidXM1DQp1aHViNTogPEludGVsIFVIQ0kgcm9vdCBIVUIsIGNsYXNzIDkv MCwgcmV2IDEuMDAvMS4wMCwgYWRkciAxPiBvbiB1c2J1czUNCnVnZW42LjE6IDxJbnRlbD4g YXQgdXNidXM2DQp1aHViNjogPEludGVsIFVIQ0kgcm9vdCBIVUIsIGNsYXNzIDkvMCwgcmV2 IDEuMDAvMS4wMCwgYWRkciAxPiBvbiB1c2J1czYNCnVnZW43LjE6IDxJbnRlbD4gYXQgdXNi dXM3DQp1aHViNzogPEludGVsIEVIQ0kgcm9vdCBIVUIsIGNsYXNzIDkvMCwgcmV2IDIuMDAv MS4wMCwgYWRkciAxPiBvbiB1c2J1czcNCiBTQVRBIDIueCBkZXZpY2UNCmFkYTA6IDMwMC4w MDBNQi9zIHRyYW5zZmVycyAoU0FUQSAyLngsIFVETUE2LCBQSU8gODE5MmJ5dGVzKQ0KYWRh MDogQ29tbWFuZCBRdWV1ZWluZyBlbmFibGVkDQphZGEwOiAzMDUyNDVNQiAoNjI1MTQyNDQ4 IDUxMiBieXRlIHNlY3RvcnM6IDE2SCA2M1MvVCAxNjM4M0MpDQphZGExIGF0IGFoY2ljaDEg YnVzIDAgc2NidXMxIHRhcmdldCAwIGx1biAwDQphZGExOiA8V0RDIFdENTAwMEFBS1MtMDBZ R0EwIDEyLjAxQzAyPiBBVEEtOCBTQVRBIDIueCBkZXZpY2UNCmFkYTE6IDMwMC4wMDBNQi9z IHRyYW5zZmVycyAoU0FUQSAyLngsIFVETUE2LCBQSU8gODE5MmJ5dGVzKQ0KYWRhMTogQ29t bWFuZCBRdWV1ZWluZyBlbmFibGVkDQphZGExOiA0NzY5NDBNQiAoOTc2NzczMTY4IDUxMiBi eXRlIHNlY3RvcnM6IDE2SCA2M1MvVCAxNjM4M0MpDQphZGEyIGF0IGFoY2ljaDIgYnVzIDAg c2NidXMyIHRhcmdldCAwIGx1biAwDQphZGEyOiA8V0RDIFdENTAwMEFBS1MtMDBBN0IyIDAx LjAzQjAxPiBBVEEtOCBTQVRBIDIueCBkZXZpY2UNCmFkYTI6IDMwMC4wMDBNQi9zIHRyYW5z ZmVycyAoU0FUQSAyLngsIFVETUE2LCBQSU8gODE5MmJ5dGVzKQ0KYWRhMjogQ29tbWFuZCBR dWV1ZWluZyBlbmFibGVkDQphZGEyOiA0NzY5NDBNQiAoOTc2NzczMTY4IDUxMiBieXRlIHNl Y3RvcnM6IDE2SCA2M1MvVCAxNjM4M0MpDQphZGEzIGF0IGFoY2ljaDMgYnVzIDAgc2NidXMz IHRhcmdldCAwIGx1biAwDQphZGEzOiA8V0RDIFdENTAwMEFBS1MtMDBZR0EwIDEyLjAxQzAy PiBBVEEtOCBTQVRBIDIueCBkZXZpY2UNCmFkYTM6IDMwMC4wMDBNQi9zIHRyYW5zZmVycyAo U0FUQSAyLngsIFVETUE2LCBQSU8gODE5MmJ5dGVzKQ0KYWRhMzogQ29tbWFuZCBRdWV1ZWlu ZyBlbmFibGVkDQphZGEzOiA0NzY5NDBNQiAoOTc2NzczMTY4IDUxMiBieXRlIHNlY3RvcnM6 IDE2SCA2M1MvVCAxNjM4M0MpDQphZGE0IGF0IGFoY2ljaDQgYnVzIDAgc2NidXM0IHRhcmdl dCAwIGx1biAwDQphZGE0OiA8V0RDIFdENTAwMEFBS1MtMDBZR0EwIDEyLjAxQzAyPiBBVEEt OCBTQVRBIDIueCBkZXZpY2UNCmFkYTQ6IDMwMC4wMDBNQi9zIHRyYW5zZmVycyAoU0FUQSAy LngsIFVETUE2LCBQSU8gODE5MmJ5dGVzKQ0KYWRhNDogQ29tbWFuZCBRdWV1ZWluZyBlbmFi bGVkDQphZGE0OiA0NzY5NDBNQiAoOTc2NzczMTY4IDUxMiBieXRlIHNlY3RvcnM6IDE2SCA2 M1MvVCAxNjM4M0MpDQphZGE1IGF0IGFoY2ljaDUgYnVzIDAgc2NidXM1IHRhcmdldCAwIGx1 biAwDQphZGE1OiA8V0RDIFdENTAwMEFBS1MtMDBZR0EwIDEyLjAxQzAyPiBBVEEtOCBTQVRB IDIueCBkZXZpY2UNCmFkYTU6IDMwMC4wMDBNQi9zIHRyYW5zZmVycyAoU0FUQSAyLngsIFVE TUE2LCBQSU8gODE5MmJ5dGVzKQ0KYWRhNTogQ29tbWFuZCBRdWV1ZWluZyBlbmFibGVkDQph ZGE1OiA0NzY5NDBNQiAoOTc2NzczMTY4IDUxMiBieXRlIHNlY3RvcnM6IDE2SCA2M1MvVCAx NjM4M0MpDQpTTVA6IEFQIENQVSAjMSBMYXVuY2hlZCENCnN0cmF5IGlycTANCldBUk5JTkc6 IFdJVE5FU1Mgb3B0aW9uIGVuYWJsZWQsIGV4cGVjdCByZWR1Y2VkIHBlcmZvcm1hbmNlLg0K V0FSTklORzogRElBR05PU1RJQyBvcHRpb24gZW5hYmxlZCwgZXhwZWN0IHJlZHVjZWQgcGVy Zm9ybWFuY2UuDQpHRU9NX1JBSUQ1OiBzdG9yYWdlOiBkZXZpY2UgY3JlYXRlZCAoc3RyaXBl c2l6ZT0xMzEwNzIpLg0KR0VPTV9SQUlENTogc3RvcmFnZTogYWRhMSg0KTogbmV3ZXN0IGRp c2sgZGF0YSAoQ0FMTSk6IDEuDQpHRU9NX1JBSUQ1OiBzdG9yYWdlOiBhZGExKDQpOiBkaXNr IGF0dGFjaGVkLg0KR0VPTV9SQUlENTogc3RvcmFnZTogYWRhMigzKTogbmV3ZXN0IGRpc2sg ZGF0YSAoQ0FMTSk6IDEuDQpHRU9NX1JBSUQ1OiBzdG9yYWdlOiBhZGEyKDMpOiBkaXNrIGF0 dGFjaGVkLg0KR0VPTV9SQUlENTogc3RvcmFnZTogYWRhMygyKTogbmV3ZXN0IGRpc2sgZGF0 YSAoQ0FMTSk6IDEuDQpHRU9NX1JBSUQ1OiBzdG9yYWdlOiBhZGEzKDIpOiBkaXNrIGF0dGFj aGVkLg0KR0VPTV9SQUlENTogc3RvcmFnZTogYWRhNCgwKTogbmV3ZXN0IGRpc2sgZGF0YSAo Q0FMTSk6IDEuDQpHRU9NX1JBSUQ1OiBzdG9yYWdlOiBhZGE0KDApOiBkaXNrIGF0dGFjaGVk Lg0KR0VPTV9SQUlENTogc3RvcmFnZTogYWRhNSgxKTogbmV3ZXN0IGRpc2sgZGF0YSAoQ0FM TSk6IDEuDQpHRU9NX1JBSUQ1OiBzdG9yYWdlOiBhZGE1KDEpOiBkaXNrIGF0dGFjaGVkLg0K R0VPTV9SQUlENTogc3RvcmFnZTogYWN0aXZhdGVkIChuZWVkIGFib3V0IDk1TWlCIGttZW0g KG1heCkpLg0KR0VPTV9SQUlENTogc3RvcmFnZTogYWRhNSgxKTogcmUtc3luYyBpbiBwcm9n cmVzczogNjEuOTklIHA6MCBFVEE6MG1pbiAoY2F1c2U6IHN0b3JlIHZlcmlmeSBwcm9ncmVz cykuDQp1aHViMDogMiBwb3J0cyB3aXRoIDIgcmVtb3ZhYmxlLCBzZWxmIHBvd2VyZWQNCnVo dWIxOiAyIHBvcnRzIHdpdGggMiByZW1vdmFibGUsIHNlbGYgcG93ZXJlZA0KdWh1YjI6IDIg cG9ydHMgd2l0aCAyIHJlbW92YWJsZSwgc2VsZiBwb3dlcmVkDQp1aHViNDogMiBwb3J0cyB3 aXRoIDIgcmVtb3ZhYmxlLCBzZWxmIHBvd2VyZWQNCnVodWI1OiAyIHBvcnRzIHdpdGggMiBy ZW1vdmFibGUsIHNlbGYgcG93ZXJlZA0KdWh1YjY6IDIgcG9ydHMgd2l0aCAyIHJlbW92YWJs ZSwgc2VsZiBwb3dlcmVkDQpSb290IG1vdW50IHdhaXRpbmcgZm9yOiB1c2J1czcgdXNidXMz DQpSb290IG1vdW50IHdhaXRpbmcgZm9yOiB1c2J1czcgdXNidXMzDQpSb290IG1vdW50IHdh aXRpbmcgZm9yOiB1c2J1czcgdXNidXMzDQp1aHViMzogNiBwb3J0cyB3aXRoIDYgcmVtb3Zh YmxlLCBzZWxmIHBvd2VyZWQNCnVodWI3OiA2IHBvcnRzIHdpdGggNiByZW1vdmFibGUsIHNl bGYgcG93ZXJlZA0KUm9vdCBtb3VudCB3YWl0aW5nIGZvcjogdXNidXM3IHVzYnVzMw0KdWdl bjMuMjogPHZlbmRvciAweDk3MTA+IGF0IHVzYnVzMw0KdWdlbjcuMjogPEdlbmVyaWM+IGF0 IHVzYnVzNw0KdW1hc3MwOiA8R2VuZXJpYyBNYXNzIFN0b3JhZ2UgRGV2aWNlLCBjbGFzcyAw LzAsIHJldiAyLjAwLzEuMDAsIGFkZHIgMj4gb24gdXNidXM3DQpSb290IG1vdW50IHdhaXRp bmcgZm9yOiB1c2J1czcNClRyeWluZyB0byBtb3VudCByb290IGZyb20gdWZzOi9kZXYvYWRh MHMxYQ0KZGEwIGF0IHVtYXNzLXNpbTAgYnVzIDAgc2NidXM2IHRhcmdldCAwIGx1biAwDQpk YTA6IDxHZW5lcmljIFVTQiBTRCBSZWFkZXIgMS4wMD4gUmVtb3ZhYmxlIERpcmVjdCBBY2Nl c3MgU0NTSS0wIGRldmljZSANCmRhMDogNDAuMDAwTUIvcyB0cmFuc2ZlcnMNCmRhMDogQXR0 ZW1wdCB0byBxdWVyeSBkZXZpY2Ugc2l6ZSBmYWlsZWQ6IE5PVCBSRUFEWSwgTWVkaXVtIG5v dCBwcmVzZW50DQpkYTEgYXQgdW1hc3Mtc2ltMCBidXMgMCBzY2J1czYgdGFyZ2V0IDAgbHVu IDENCmRhMTogPEdlbmVyaWMgVVNCIENGIFJlYWRlciAxLjAxPiBSZW1vdmFibGUgRGlyZWN0 IEFjY2VzcyBTQ1NJLTAgZGV2aWNlIA0KZGExOiA0MC4wMDBNQi9zIHRyYW5zZmVycw0KZGEx OiBBdHRlbXB0IHRvIHF1ZXJ5IGRldmljZSBzaXplIGZhaWxlZDogTk9UIFJFQURZLCBNZWRp dW0gbm90IHByZXNlbnQNCmRhMiBhdCB1bWFzcy1zaW0wIGJ1cyAwIHNjYnVzNiB0YXJnZXQg MCBsdW4gMg0KZGEyOiA8R2VuZXJpYyBVU0IgU00gUmVhZGVyIDEuMDI+IFJlbW92YWJsZSBE aXJlY3QgQWNjZXNzIFNDU0ktMCBkZXZpY2UgDQpkYTI6IDQwLjAwME1CL3MgdHJhbnNmZXJz DQpkYTI6IEF0dGVtcHQgdG8gcXVlcnkgZGV2aWNlIHNpemUgZmFpbGVkOiBOT1QgUkVBRFks IE1lZGl1bSBub3QgcHJlc2VudA0KZGEzIGF0IHVtYXNzLXNpbTAgYnVzIDAgc2NidXM2IHRh cmdldCAwIGx1biAzDQpkYTM6IDxHZW5lcmljIFVTQiBNUyBSZWFkZXIgMS4wMz4gUmVtb3Zh YmxlIERpcmVjdCBBY2Nlc3MgU0NTSS0wIGRldmljZSANCmRhMzogNDAuMDAwTUIvcyB0cmFu c2ZlcnMNCmRhMzogQXR0ZW1wdCB0byBxdWVyeSBkZXZpY2Ugc2l6ZSBmYWlsZWQ6IE5PVCBS RUFEWSwgTWVkaXVtIG5vdCBwcmVzZW50DQpFbnRlciBmdWxsIHBhdGhuYW1lIG9mIHNoZWxs IG9yIFJFVFVSTiBmb3IgL2Jpbi9zaDogDQo= ------------B9A228150F548A Content-Type: application/octet-stream; name="kernel.config" Content-transfer-encoding: base64 Content-Disposition: attachment; filename="kernel.config" Y3B1CQlIQU1NRVINCmlkZW50CQlCTE9CRElBRw0KDQptYWtlb3B0aW9ucwlERUJVRz0tZwkJ IyBCdWlsZCBrZXJuZWwgd2l0aCBnZGIoMSkgZGVidWcgc3ltYm9scw0KDQpvcHRpb25zCQlQ UklOVEZfQlVGUl9TSVpFPTEwMjQNCg0Kb3B0aW9ucwkJRERCDQpvcHRpb25zCQlLREINCm9w dGlvbnMJCUJSRUFLX1RPX0RFQlVHR0VSDQoNCm9wdGlvbnMJCUlOVkFSSUFOVFMNCm9wdGlv bnMJCUlOVkFSSUFOVF9TVVBQT1JUDQpvcHRpb25zCQlXSVRORVNTDQpvcHRpb25zCQlERUJV R19MT0NLUw0Kb3B0aW9ucwkJREVCVUdfVkZTX0xPQ0tTDQpvcHRpb25zCQlESUFHTk9TVElD DQoNCm9wdGlvbnMgCVNDSEVEX1VMRQkJIyBVTEUgc2NoZWR1bGVyDQpvcHRpb25zIAlQUkVF TVBUSU9OCQkjIEVuYWJsZSBrZXJuZWwgdGhyZWFkIHByZWVtcHRpb24NCm9wdGlvbnMgCUlO RVQJCQkjIEludGVyTkVUd29ya2luZw0Kb3B0aW9ucyAJRkZTCQkJIyBCZXJrZWxleSBGYXN0 IEZpbGVzeXN0ZW0NCm9wdGlvbnMgCVNPRlRVUERBVEVTCQkjIEVuYWJsZSBGRlMgc29mdCB1 cGRhdGVzIHN1cHBvcnQNCm9wdGlvbnMgCVVGU19BQ0wJCQkjIFN1cHBvcnQgZm9yIGFjY2Vz cyBjb250cm9sIGxpc3RzDQpvcHRpb25zIAlVRlNfRElSSEFTSAkJIyBJbXByb3ZlIHBlcmZv cm1hbmNlIG9uIGJpZyBkaXJlY3Rvcmllcw0Kb3B0aW9ucwkJVUZTX0dKT1VSTkFMCSMgRW5h YmxlIGdqb3VybmFsLWJhc2VkIFVGUyBqb3VybmFsaW5nDQpvcHRpb25zIAlNRF9ST09UCQkJ IyBNRCBpcyBhIHBvdGVudGlhbCByb290IGRldmljZQ0Kb3B0aW9ucyAJTkZTQ0xJRU5UCQkj IE5ldHdvcmsgRmlsZXN5c3RlbSBDbGllbnQNCm9wdGlvbnMgCU5GU1NFUlZFUgkJIyBOZXR3 b3JrIEZpbGVzeXN0ZW0gU2VydmVyDQpvcHRpb25zIAlORlNMT0NLRAkJIyBOZXR3b3JrIExv Y2sgTWFuYWdlcg0Kb3B0aW9ucyAJUFJPQ0ZTCQkJIyBQcm9jZXNzIGZpbGVzeXN0ZW0gKHJl cXVpcmVzIFBTRVVET0ZTKQ0Kb3B0aW9ucyAJUFNFVURPRlMJCSMgUHNldWRvLWZpbGVzeXN0 ZW0gZnJhbWV3b3JrDQpvcHRpb25zIAlHRU9NX0xBQkVMCQkjIFByb3ZpZGVzIGxhYmVsaXph dGlvbg0Kb3B0aW9ucyAJQ09NUEFUXzQzVFRZCQkjIEJTRCA0LjMgVFRZIGNvbXBhdCBbS0VF UCBUSElTIV0NCm9wdGlvbnMgCUNPTVBBVF9JQTMyCQkjIENvbXBhdGlibGUgd2l0aCBpMzg2 IGJpbmFyaWVzDQpvcHRpb25zIAlDT01QQVRfRlJFRUJTRDQJCSMgQ29tcGF0aWJsZSB3aXRo IEZyZWVCU0Q0DQpvcHRpb25zIAlDT01QQVRfRlJFRUJTRDUJCSMgQ29tcGF0aWJsZSB3aXRo IEZyZWVCU0Q1DQpvcHRpb25zIAlDT01QQVRfRlJFRUJTRDYJCSMgQ29tcGF0aWJsZSB3aXRo IEZyZWVCU0Q2DQpvcHRpb25zIAlDT01QQVRfRlJFRUJTRDcJCSMgQ29tcGF0aWJsZSB3aXRo IEZyZWVCU0Q3DQpvcHRpb25zIAlTQ1NJX0RFTEFZPTUwMDAJCSMgRGVsYXkgKGluIG1zKSBi ZWZvcmUgcHJvYmluZyBTQ1NJDQpvcHRpb25zIAlLVFJBQ0UJCQkjIGt0cmFjZSgxKSBzdXBw b3J0DQpvcHRpb25zIAlTVEFDSwkJCSMgc3RhY2soOSkgc3VwcG9ydA0Kb3B0aW9ucyAJU1lT VlNITQkJCSMgU1lTVi1zdHlsZSBzaGFyZWQgbWVtb3J5DQpvcHRpb25zIAlTWVNWTVNHCQkJ IyBTWVNWLXN0eWxlIG1lc3NhZ2UgcXVldWVzDQpvcHRpb25zIAlTWVNWU0VNCQkJIyBTWVNW LXN0eWxlIHNlbWFwaG9yZXMNCm9wdGlvbnMgCV9LUE9TSVhfUFJJT1JJVFlfU0NIRURVTElO RyAjIFBPU0lYIFAxMDAzXzFCIHJlYWwtdGltZSBleHRlbnNpb25zDQpvcHRpb25zIAlLQkRf SU5TVEFMTF9DREVWCSMgaW5zdGFsbCBhIENERVYgZW50cnkgaW4gL2Rldg0Kb3B0aW9ucyAJ QVVESVQJCQkjIFNlY3VyaXR5IGV2ZW50IGF1ZGl0aW5nDQoNCiMgTWFrZSBhbiBTTVAtY2Fw YWJsZSBrZXJuZWwgYnkgZGVmYXVsdA0Kb3B0aW9ucyAJU01QCQkJIyBTeW1tZXRyaWMgTXVs dGlQcm9jZXNzb3IgS2VybmVsDQoNCiMgRGV2aWNlIHBvbGxpbmcNCm9wdGlvbnMJCURFVklD RV9QT0xMSU5HDQoNCiMgQ1BVIGZyZXF1ZW5jeSBjb250cm9sDQpkZXZpY2UJCWNwdWZyZXEN CmRldmljZQkJY29yZXRlbXANCg0KZGV2aWNlCQlzbWINCmRldmljZQkJc21idXMNCmRldmlj ZQkJaWNoc21iDQoNCmRldmljZQkJaWljYnVzDQpkZXZpY2UJCWlpY3NtYg0KZGV2aWNlCQlp aWMNCg0KIyBCdXMgc3VwcG9ydC4NCmRldmljZQkJYWNwaQ0KZGV2aWNlCQlwY2kNCg0KIyBT Q1NJDQpkZXZpY2UJCXNjYnVzDQpkZXZpY2UJCXBhc3MNCmRldmljZQkJZGENCg0KIyBBVEEg YW5kIEFUQVBJIGRldmljZXMNCmRldmljZQkJYWhjaQ0KDQojIGF0a2JkYzAgY29udHJvbHMg Ym90aCB0aGUga2V5Ym9hcmQgYW5kIHRoZSBQUy8yIG1vdXNlDQpkZXZpY2UJCWF0a2JkYwkJ IyBBVCBrZXlib2FyZCBjb250cm9sbGVyDQpkZXZpY2UJCWF0a2JkCQkjIEFUIGtleWJvYXJk DQpkZXZpY2UJCXBzbQkJIyBQUy8yIG1vdXNlDQpkZXZpY2UJCWtiZG11eAkJIyBrZXlib2Fy ZCBtdWx0aXBsZXhlcg0KZGV2aWNlCQl2Z2EJCSMgVkdBIHZpZGVvIGNhcmQgZHJpdmVyDQoN CiMgc3lzY29ucyBpcyB0aGUgZGVmYXVsdCBjb25zb2xlIGRyaXZlciwgcmVzZW1ibGluZyBh biBTQ08gY29uc29sZQ0KZGV2aWNlCQlzYw0KDQojIFNlcmlhbCAoQ09NKSBwb3J0cw0KZGV2 aWNlCQl1YXJ0CQkjIEdlbmVyaWMgVUFSVCBkcml2ZXINCg0KIyBQYXJhbGxlbCBwb3J0DQpk ZXZpY2UJCXBwYw0KZGV2aWNlCQlwcGJ1cwkJIyBQYXJhbGxlbCBwb3J0IGJ1cyAocmVxdWly ZWQpDQoNCmRldmljZQkJZW0JCSMgSW50ZWwgUFJPLzEwMDAgYWRhcHRlciBHaWdhYml0IEV0 aGVybmV0IENhcmQNCg0KIyBQc2V1ZG8gZGV2aWNlcy4NCmRldmljZQkJbG9vcAkJIyBOZXR3 b3JrIGxvb3BiYWNrDQpkZXZpY2UJCXJhbmRvbQkJIyBFbnRyb3B5IGRldmljZQ0KZGV2aWNl CQlldGhlcgkJIyBFdGhlcm5ldCBzdXBwb3J0DQpkZXZpY2UJCXR1bgkJIyBQYWNrZXQgdHVu bmVsLg0KZGV2aWNlCQlwdHkJCSMgUHNldWRvLXR0eXMgKHRlbG5ldCBldGMpDQpkZXZpY2UJ CW1kCQkjIE1lbW9yeSAiZGlza3MiDQpkZXZpY2UJCWZpcm13YXJlCSMgZmlybXdhcmUgYXNz aXN0IG1vZHVsZQ0KDQojIFRoZSBgYnBmJyBkZXZpY2UgZW5hYmxlcyB0aGUgQmVya2VsZXkg UGFja2V0IEZpbHRlci4NCiMgQmUgYXdhcmUgb2YgdGhlIGFkbWluaXN0cmF0aXZlIGNvbnNl cXVlbmNlcyBvZiBlbmFibGluZyB0aGlzIQ0KIyBOb3RlIHRoYXQgJ2JwZicgaXMgcmVxdWly ZWQgZm9yIERIQ1AuDQpkZXZpY2UJCWJwZgkJIyBCZXJrZWxleSBwYWNrZXQgZmlsdGVyDQoN CiMgVVNCIHN1cHBvcnQNCmRldmljZQkJdWhjaQkJIyBVSENJIFBDSS0+VVNCIGludGVyZmFj ZQ0KZGV2aWNlCQllaGNpCQkjIEVIQ0kgUENJLT5VU0IgaW50ZXJmYWNlIChVU0IgMi4wKQ0K ZGV2aWNlCQl1c2IJCSMgVVNCIEJ1cyAocmVxdWlyZWQpDQpkZXZpY2UJCXVjb20JCSMgRGlz a3MvTWFzcyBzdG9yYWdlIC0gUmVxdWlyZXMgc2NidXMgYW5kIGRhDQpkZXZpY2UJCXVtYXNz CQkjIERpc2tzL01hc3Mgc3RvcmFnZSAtIFJlcXVpcmVzIHNjYnVzIGFuZCBkYQ0KDQojIEZp cmVXaXJlIHN1cHBvcnQNCmRldmljZQkJZmlyZXdpcmUJIyBGaXJlV2lyZSBidXMgY29kZQ0K ------------B9A228150F548A-- From owner-freebsd-stable@FreeBSD.ORG Sat Jan 8 19:02:36 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6A330106572C; Sat, 8 Jan 2011 19:02:36 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 027468FC17; Sat, 8 Jan 2011 19:02:35 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p08J2WU0006053 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 8 Jan 2011 21:02:32 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p08J2W3U095326; Sat, 8 Jan 2011 21:02:32 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p08J2W4S095325; Sat, 8 Jan 2011 21:02:32 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 8 Jan 2011 21:02:32 +0200 From: Kostik Belousov To: Lev Serebryakov Message-ID: <20110108190232.GU12599@deviant.kiev.zoral.com.ua> References: <204344488.20110108214457@serebryakov.spb.ru> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="2JhMvOwMi9vhAgPY" Content-Disposition: inline In-Reply-To: <204344488.20110108214457@serebryakov.spb.ru> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-stable@freebsd.org Subject: Re: 8.2-PRERELEASE: live deadlock, almost all processes in "pfault" state X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jan 2011 19:02:36 -0000 --2JhMvOwMi9vhAgPY Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Jan 08, 2011 at 09:44:57PM +0300, Lev Serebryakov wrote: > Hello, Freebsd-stable. >=20 > I've added `transmission' BitTorrent client to my home server and now > it deadlocks easily (after about 1 hour of intensive download and > seeding). This server is upgraded from 7.x and last time I've run > transmission on 7.x system without any problems. >=20 > I have home partition on geom_raid5 device, so I can not exclude this > third-party module from experiments. >=20 > My home filsystem has 32KiB block and all other filesystems (/, /var, > /tmp, /usr) has standard 16KiB block sizes. I know, that 7.x system > had (has?) deadlock when 16KiB and 64KiB file systems are mixed up on > one system, but I never experienced deadlocks with 16KiB and 32KiB > mixture. >=20 > All filesystems (Except root) is SU, but no gjournal so SU_J patch > are in use. >=20 > Same BitTorrent client on same filesystem, but accessed via NFS (from > other host), doesn't cause deadlock and works rock-stable for days. >=20 > I've built kernel with all debug options, waited for deadlock and > collect all information, mentioned in Developer's Handbook / Debugging > Deadlocks. >=20 > Capture from debug session is attached, together with kernel config > and dmesg from rebooting. >=20 > As I can easily reproduce this deadlock, I could provide any > additional information from kernel debugger, if needed. >=20 > System: FreeBSD 8.2-PRERELEASE > cvsup: 2011-01-08 00:41:24 MSK (GTM+3) time > Platform: amd64 There is some weird backtrace at the pid 20, what is g_raid5 ? If I am guessing right, this creature has a classic deadlock when=20 bio processing requires memory allocation. It seems that tid 100079 is sleeping not even due to the free page shortage, but due to address space exhaustion. As result, read/write requests are stalled. Then, syncer is blocked waiting for some physical buffer (look at tid 100075), owning the vnode lock. Other processes also wait for the locked buffers, etc. So my belief is that this is plain driver (g_raid5, whatever is it) i/o loss. Try the same load without it. --2JhMvOwMi9vhAgPY Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk0otMcACgkQC3+MBN1Mb4je/QCgz7k3fEW5jQA3qC0CV1JTp8VL 2/4An0YNI0BSoc2zyzQBcYkwFQGXHOmn =TOTR -----END PGP SIGNATURE----- --2JhMvOwMi9vhAgPY-- From owner-freebsd-stable@FreeBSD.ORG Sat Jan 8 19:29:17 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0667A1065767 for ; Sat, 8 Jan 2011 19:29:17 +0000 (UTC) (envelope-from lev@freebsd.org) Received: from ftp.translate.ru (ftp.translate.ru [80.249.188.42]) by mx1.freebsd.org (Postfix) with ESMTP id B76B28FC12 for ; Sat, 8 Jan 2011 19:29:16 +0000 (UTC) Received: from [192.168.134.2] (89.112.15.178.pppoe.eltel.net [89.112.15.178]) (Authenticated sender: lev@serebryakov.spb.ru) by ftp.translate.ru (Postfix) with ESMTPA id 9F65613DF61; Sat, 8 Jan 2011 22:29:15 +0300 (MSK) Date: Sat, 8 Jan 2011 22:29:09 +0300 From: Lev Serebryakov X-Priority: 3 (Normal) Message-ID: <1792026896.20110108222909@serebryakov.spb.ru> To: Kostik Belousov In-Reply-To: <20110108190232.GU12599@deviant.kiev.zoral.com.ua> References: <204344488.20110108214457@serebryakov.spb.ru> <20110108190232.GU12599@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Cc: freebsd-stable@freebsd.org Subject: Re: 8.2-PRERELEASE: live deadlock, almost all processes in "pfault" state X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jan 2011 19:29:17 -0000 Hello, Kostik. You wrote 8 =FF=ED=E2=E0=F0=FF 2011 =E3., 22:02:32: > There is some weird backtrace at the pid 20, what is g_raid5 ? It is geom_raid5, with two threads -- working one and one for processing finished bios. > If I am guessing right, this creature has a classic deadlock when=20 > bio processing requires memory allocation. It seems that tid 100079 tid 100079 sleep in waiting for some data in queue. > is sleeping not even due to the free page shortage, but due to address > space exhaustion. As result, read/write requests are stalled. tid 100078 sleep in malloc(). But geom_raid5 never ever allocate more than 128MiB of memory and it is 64bit system with huge amount of kmem_size/kmem_size_max... How could I explore allocation (like vmstat -m) from kdb to be sure, it doesn't allocated more? And, if it is "classic deadlock" is here any "classical" solution to it? Really, I'm maintainer of geom_raid5 now, so I need fix this deadlock, but I don't really understand, why does it occur? I've hit panic with "kernel memory exhausted" symptoms when module allocate too much, but not deadlock :( > Then, syncer is blocked waiting for some physical buffer (look at tid > 100075), owning the vnode lock. Other processes also wait for the > locked buffers, etc. > So my belief is that this is plain driver (g_raid5, whatever is it) > i/o loss. Try the same load without it. I can not, because all data is on this GEOM :) --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-stable@FreeBSD.ORG Sat Jan 8 19:39:07 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 34F16106564A for ; Sat, 8 Jan 2011 19:39:07 +0000 (UTC) (envelope-from andy@fud.org.nz) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 59ABE8FC08 for ; Sat, 8 Jan 2011 19:39:05 +0000 (UTC) Received: by wwf26 with SMTP id 26so18192042wwf.31 for ; Sat, 08 Jan 2011 11:39:05 -0800 (PST) MIME-Version: 1.0 Received: by 10.227.159.68 with SMTP id i4mr9940831wbx.176.1294513684696; Sat, 08 Jan 2011 11:08:04 -0800 (PST) Sender: andy@fud.org.nz Received: by 10.227.143.200 with HTTP; Sat, 8 Jan 2011 11:08:04 -0800 (PST) In-Reply-To: References: <201101060804.56205.jhb@freebsd.org> <2047622233.213674.1294348606324.JavaMail.root@erie.cs.uoguelph.ca> Date: Sun, 9 Jan 2011 08:08:04 +1300 X-Google-Sender-Auth: EmeHYJpmdEMY2tmOFDvCZlL_yPo Message-ID: From: Andrew Thompson To: Jean-Yves Avenard Content-Type: text/plain; charset=ISO-8859-1 Cc: perryh@pluto.rain.com, Rick Macklem , freebsd-stable@freebsd.org, marek sal , milu@dat.pl Subject: Re: NFSv4 - how to set up at FreeBSD 8.1 ? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jan 2011 19:39:07 -0000 On 7 January 2011 14:25, Jean-Yves Avenard wrote: > On 7 January 2011 08:16, Rick Macklem wrote: > >> When I said I recalled that they didn't do TCP because of excessive >> overhead, I forgot to mention that my recollection could be wrong. >> Also, I suspect you are correct w.r.t. the above statement. (ie. Sun's >> official position vs something I heard.) >> >> Anyhow, appologies if I gave the impression that I was correcting your >> statement. My intent was just to throw out another statement that I >> vaguely recalled someone an Sun stating. > > After hitting yet another serious bug in 8.2 ; I reverted back to 8.1 Has the problem been reported? There is still time to fix serious bugs in 8.2 Andrew From owner-freebsd-stable@FreeBSD.ORG Sat Jan 8 19:42:27 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5E5CA1065693 for ; Sat, 8 Jan 2011 19:42:27 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from ftp.translate.ru (ftp.translate.ru [80.249.188.42]) by mx1.freebsd.org (Postfix) with ESMTP id 1B1388FC1D for ; Sat, 8 Jan 2011 19:42:26 +0000 (UTC) Received: from [192.168.134.2] (89.112.15.178.pppoe.eltel.net [89.112.15.178]) (Authenticated sender: lev@serebryakov.spb.ru) by ftp.translate.ru (Postfix) with ESMTPA id 18DD913DF61; Sat, 8 Jan 2011 22:42:26 +0300 (MSK) Date: Sat, 8 Jan 2011 22:42:19 +0300 From: Lev Serebryakov Organization: FreeBSD X-Priority: 3 (Normal) Message-ID: <427356441.20110108224219@serebryakov.spb.ru> To: Kostik Belousov In-Reply-To: <20110108190232.GU12599@deviant.kiev.zoral.com.ua> References: <204344488.20110108214457@serebryakov.spb.ru> <20110108190232.GU12599@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable Cc: freebsd-stable@freebsd.org Subject: Re: 8.2-PRERELEASE: live deadlock, almost all processes in "pfault" state X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jan 2011 19:42:27 -0000 Hello, Kostik. You wrote 8 =FF=ED=E2=E0=F0=FF 2011 =E3., 22:02:32: > If I am guessing right, this creature has a classic deadlock when > bio processing requires memory allocation. It seems that tid 100079 > is sleeping not even due to the free page shortage, but due to address > space exhaustion. As result, read/write requests are stalled. I want to say, that ZFS, for example, could allocate much more memory, and, yes, it had problems on i386 with this, but not on amd64, AFAIK... So, I'm (geom_radi5) doing something wrong... --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-stable@FreeBSD.ORG Sat Jan 8 19:56:18 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0244A106566C; Sat, 8 Jan 2011 19:56:18 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 761DB8FC08; Sat, 8 Jan 2011 19:56:17 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p08JuDQf010137 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 8 Jan 2011 21:56:13 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p08JuDod095571; Sat, 8 Jan 2011 21:56:13 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p08JuDhH095570; Sat, 8 Jan 2011 21:56:13 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 8 Jan 2011 21:56:13 +0200 From: Kostik Belousov To: Lev Serebryakov Message-ID: <20110108195613.GW12599@deviant.kiev.zoral.com.ua> References: <204344488.20110108214457@serebryakov.spb.ru> <20110108190232.GU12599@deviant.kiev.zoral.com.ua> <1792026896.20110108222909@serebryakov.spb.ru> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="iAq/E3df7QH9m+R5" Content-Disposition: inline In-Reply-To: <1792026896.20110108222909@serebryakov.spb.ru> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-stable@freebsd.org Subject: Re: 8.2-PRERELEASE: live deadlock, almost all processes in "pfault" state X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jan 2011 19:56:18 -0000 --iAq/E3df7QH9m+R5 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Jan 08, 2011 at 10:29:09PM +0300, Lev Serebryakov wrote: > Hello, Kostik. > You wrote 8 =D1=CE=D7=C1=D2=D1 2011 =C7., 22:02:32: >=20 >=20 > > There is some weird backtrace at the pid 20, what is g_raid5 ? > It is geom_raid5, with two threads -- working one and one for > processing finished bios. >=20 > > If I am guessing right, this creature has a classic deadlock when=20 > > bio processing requires memory allocation. It seems that tid 100079 > tid 100079 sleep in waiting for some data in queue. >=20 > > is sleeping not even due to the free page shortage, but due to address > > space exhaustion. As result, read/write requests are stalled. > tid 100078 sleep in malloc(). But geom_raid5 never ever allocate > more than 128MiB of memory and it is 64bit system with huge amount of > kmem_size/kmem_size_max... >=20 > How could I explore allocation (like vmstat -m) from kdb to be sure, > it doesn't allocated more? Use "show uma" and "show malloc" from ddb. >=20 > And, if it is "classic deadlock" is here any "classical" solution to > it? Do not allocate during bio processing. >=20 > Really, I'm maintainer of geom_raid5 now, so I need fix this > deadlock, but I don't really understand, why does it occur? I've > hit panic with "kernel memory exhausted" symptoms when module allocate > too much, but not deadlock :( Hm, I missed the kmem_back() in the stack. Yes, the thread is waiting for p= age allocation. >=20 > > Then, syncer is blocked waiting for some physical buffer (look at tid > > 100075), owning the vnode lock. Other processes also wait for the > > locked buffers, etc. >=20 > > So my belief is that this is plain driver (g_raid5, whatever is it) > > i/o loss. Try the same load without it. > I can not, because all data is on this GEOM :) >=20 > --=20 > // Black Lion AKA Lev Serebryakov >=20 --iAq/E3df7QH9m+R5 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk0owVwACgkQC3+MBN1Mb4gXngCgknenCiBYQm3Dnqcxk8QL4gcC L8cAoOD9mV9vUSfr71Nc2k2diqOUcauS =U4jp -----END PGP SIGNATURE----- --iAq/E3df7QH9m+R5-- From owner-freebsd-stable@FreeBSD.ORG Sat Jan 8 20:06:15 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4EDAF106564A; Sat, 8 Jan 2011 20:06:15 +0000 (UTC) (envelope-from subbsd@gmail.com) Received: from mail-qy0-f175.google.com (mail-qy0-f175.google.com [209.85.216.175]) by mx1.freebsd.org (Postfix) with ESMTP id E67F88FC0A; Sat, 8 Jan 2011 20:06:14 +0000 (UTC) Received: by qyk8 with SMTP id 8so371667qyk.13 for ; Sat, 08 Jan 2011 12:06:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=DOi8G+rtR6DmszuRlX/rvXWXdOAc2zHNHLdKhd0ZhL8=; b=HQBHwYu3ND+i6hwx5rcyp1qgZUxlNW6dXbMY2JzsQ0HDmhp0iZBxnwGAptNuTBcwfc QVc/njWWMIEYzVIKk7RhaiUx8yP4x/C3i1R9LAm1O/BdvystwZMdnVCM3VQztU6pgGHq k2Cds+XSz43cCkYMiuNZf2B8LRioASaj8kiwc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=q2sOu/XWzbN4UGQPLu2hgNUkkYyvyVTA+Q2+1SI2h9C6r1iC9FZw8+FtfxbuScv+8V 2kq9yc/EVnjeCE0V2H8R9pUZJsZyyXXpcJEcgSUsazBFGuwlwDX2RXG7MC/JBo83x62r kg7XJt61C1gIZcJBOR6kg/ekMq/f/l7YFG68M= MIME-Version: 1.0 Received: by 10.229.95.193 with SMTP id e1mr5688990qcn.171.1294517173764; Sat, 08 Jan 2011 12:06:13 -0800 (PST) Received: by 10.229.100.73 with HTTP; Sat, 8 Jan 2011 12:06:13 -0800 (PST) In-Reply-To: References: <201101081841.10690.hselasky@c2i.net> Date: Sat, 8 Jan 2011 23:06:13 +0300 Message-ID: From: Subbsd To: kerbzo Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-multimedia@freebsd.org, freebsd-stable@freebsd.org, Hans Petter Selasky Subject: Re: Webcamd testers wanted on FreeBSD 8.2 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jan 2011 20:06:15 -0000 On Sat, Jan 8, 2011 at 10:46 PM, kerbzo wrote: > I use one webcam with webcamd but sometime after reboot /dev/video0 is > not created at all. BTW, it may be associated with the initialization of the USB ports - this situation reminded me of the problem with external USB devices (cdrom or usb memstick) in sysinstall FreeBSD 8x - in some case the device was not found without the "Re-Scan Devices". Sometimes it is not required and it is not required on FreeBSD 7x. Re-Scan - it's just power reset as I understand From owner-freebsd-stable@FreeBSD.ORG Sat Jan 8 20:10:29 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 551221065672 for ; Sat, 8 Jan 2011 20:10:29 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from ftp.translate.ru (ftp.translate.ru [80.249.188.42]) by mx1.freebsd.org (Postfix) with ESMTP id 0FF538FC14 for ; Sat, 8 Jan 2011 20:10:28 +0000 (UTC) Received: from [192.168.134.2] (89.112.15.178.pppoe.eltel.net [89.112.15.178]) (Authenticated sender: lev@serebryakov.spb.ru) by ftp.translate.ru (Postfix) with ESMTPA id 7CE1613DF5F; Sat, 8 Jan 2011 23:10:27 +0300 (MSK) Date: Sat, 8 Jan 2011 23:10:21 +0300 From: Lev Serebryakov Organization: FreeBSD Project X-Priority: 3 (Normal) Message-ID: <1544327450.20110108231021@serebryakov.spb.ru> To: Kostik Belousov In-Reply-To: <20110108195613.GW12599@deviant.kiev.zoral.com.ua> References: <204344488.20110108214457@serebryakov.spb.ru> <20110108190232.GU12599@deviant.kiev.zoral.com.ua> <1792026896.20110108222909@serebryakov.spb.ru> <20110108195613.GW12599@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: quoted-printable Cc: freebsd-stable@freebsd.org Subject: Re: 8.2-PRERELEASE: live deadlock, almost all processes in "pfault" state X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jan 2011 20:10:29 -0000 Hello, Kostik. You wrote 8 =D1=CE=D7=C1=D2=D1 2011 =C7., 22:56:13: >> And, if it is "classic deadlock" is here any "classical" solution to >> it? > Do not allocate during bio processing. So, if GEOM need some cache, it needs pre-allocate it and implements custom allocator over allocated chunk? :( And what is "bio processing" in this context? geom_raid5 puts all bios into the (private, internal) queue and geom_start() exits immediately, and bio could spend rather long time in queue (if it is write request) before it will be sent to underlying provider. And, yes, it could be combined with other bios to form new one (why allocation of new bio is needed). So, is "bio processing" a whole time before bio is complete, or only geom_start() call or what? Also, RAID5 needs to read data (other stripes) and write data (new checksum) when "write" bio is processed. BTW, "system" geom_raid3 and geom_vinum (with raid5 volume) need to do the same to maintain checksums, so they could deadlock (in theory) too, if problem is "allocate memory during bio processing". And geom_mirror needs allocate bio for second (third, ...) component on every write... --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-stable@FreeBSD.ORG Sat Jan 8 20:13:57 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7A62C1065693; Sat, 8 Jan 2011 20:13:57 +0000 (UTC) (envelope-from kerbzo@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id DE2788FC0A; Sat, 8 Jan 2011 20:13:56 +0000 (UTC) Received: by wwf26 with SMTP id 26so18207441wwf.31 for ; Sat, 08 Jan 2011 12:13:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=qAoRwBcRUkFBkDXiSQBXp7hKblLD8A+cCLJkXLLtvLM=; b=cw7BnnD3Lv3YQwxeFE1H5UdF7Q+zGZNSLLV5ZF5TU9a/MeRtunEzpPoQYEd0o723Cs tRbtgx3zYr2MqAAHThow5zFxKt5BybTAt2y4z4YFgm/M50RtOG+vbXqEOdCtPeluUE1h I5qT/Kh3sEQxGHpt3meOlriBn5DS6Um3oJKDM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=gTLeIiZmGJb55DQXVvFUOfeJ1oicKg8eeBzpcQU/hMKB0QLwO/mIT5T9okwNhLq9qB 3EErezKD0iSHbp3agptWga696KukBz2SM6WjquxlM6VByO38Ao10dcO6nxd8QlYcrWtW Pg7HytoDGL5BfFApxqN+eKRnlxDNlMPJr2jkc= MIME-Version: 1.0 Received: by 10.227.159.131 with SMTP id j3mr1178733wbx.205.1294515967719; Sat, 08 Jan 2011 11:46:07 -0800 (PST) Received: by 10.227.148.7 with HTTP; Sat, 8 Jan 2011 11:46:07 -0800 (PST) In-Reply-To: References: <201101081841.10690.hselasky@c2i.net> Date: Sat, 8 Jan 2011 20:46:07 +0100 Message-ID: From: kerbzo To: Subbsd Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-multimedia@freebsd.org, freebsd-stable@freebsd.org, Hans Petter Selasky Subject: Re: Webcamd testers wanted on FreeBSD 8.2 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jan 2011 20:13:57 -0000 Hi, I can confirm this issue with 8-STABLE built about one week ago. I use one webcam with webcamd but sometime after reboot /dev/video0 is not created at all. However, cuse4bsd.ko was never the last loaded module. Bye, On Sat, Jan 8, 2011 at 7:07 PM, Subbsd wrote: > On Sat, Jan 8, 2011 at 8:41 PM, Hans Petter Selasky wrote: >> >> Hi, >> >> Can someone running FreeBSD 8.2-RC1 with more than one or external USB webcam >> or DVB-XXX devices verify the following: >> >> > > Hello. Can the boot sequence of .ko modules in loader.conf somehow > influence the success of the detection camera? > I've seen this problem on FreeBSD-current amd64 some time ago. Maybe > it was a coincidence - but the /dev/video0 does not always appear when > loading the module was at the end of the list. In the near future I > will test in 9-0 CURRENT and 8.2-RC2 From owner-freebsd-stable@FreeBSD.ORG Sat Jan 8 20:20:33 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 568EE106564A; Sat, 8 Jan 2011 20:20:33 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id C1C7B8FC08; Sat, 8 Jan 2011 20:20:32 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id p08KKTNC011590 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 8 Jan 2011 22:20:29 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id p08KKTTl095773; Sat, 8 Jan 2011 22:20:29 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id p08KKSDq095772; Sat, 8 Jan 2011 22:20:28 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 8 Jan 2011 22:20:28 +0200 From: Kostik Belousov To: Lev Serebryakov Message-ID: <20110108202028.GY12599@deviant.kiev.zoral.com.ua> References: <204344488.20110108214457@serebryakov.spb.ru> <20110108190232.GU12599@deviant.kiev.zoral.com.ua> <1792026896.20110108222909@serebryakov.spb.ru> <20110108195613.GW12599@deviant.kiev.zoral.com.ua> <1544327450.20110108231021@serebryakov.spb.ru> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="WzyiqVXNYYkrrY2o" Content-Disposition: inline In-Reply-To: <1544327450.20110108231021@serebryakov.spb.ru> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_05, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-stable@freebsd.org Subject: Re: 8.2-PRERELEASE: live deadlock, almost all processes in "pfault" state X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jan 2011 20:20:33 -0000 --WzyiqVXNYYkrrY2o Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Jan 08, 2011 at 11:10:21PM +0300, Lev Serebryakov wrote: > Hello, Kostik. > You wrote 8 =D1=CE=D7=C1=D2=D1 2011 =C7., 22:56:13: >=20 >=20 > >> And, if it is "classic deadlock" is here any "classical" solution to > >> it? > > Do not allocate during bio processing. > So, if GEOM need some cache, it needs pre-allocate it and implements > custom allocator over allocated chunk? :( >=20 > And what is "bio processing" in this context? geom_raid5 puts all bio processing =3D=3D whole time needed to finish pageout. Pageout is often performed to clean the page to lower the page shortage. If pageout requires more free pages to finish during the shortage, then we get the deadlock. Also, it seems that you allocate not only bios (small objects, not every request cause page allocation), but also the huge buffers, that require free pages each time. > bios into the (private, internal) queue and geom_start() exits > immediately, and bio could spend rather long time in queue (if it is > write request) before it will be sent to underlying provider. And, > yes, it could be combined with other bios to form new one (why > allocation of new bio is needed). >=20 > So, is "bio processing" a whole time before bio is complete, or only > geom_start() call or what? >=20 > Also, RAID5 needs to read data (other stripes) and write data (new > checksum) when "write" bio is processed. BTW, "system" geom_raid3 and > geom_vinum (with raid5 volume) need to do the same to maintain > checksums, so they could deadlock (in theory) too, if problem is > "allocate memory during bio processing". And geom_mirror needs > allocate bio for second (third, ...) component on every write... >=20 > --=20 > // Black Lion AKA Lev Serebryakov >=20 --WzyiqVXNYYkrrY2o Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk0oxwwACgkQC3+MBN1Mb4gPTQCgiL9vRWFvfd1a17Rssv9jmGt6 1xAAoI4StIuJ6/eCiriinVyGrzA3si9a =Fw1y -----END PGP SIGNATURE----- --WzyiqVXNYYkrrY2o-- From owner-freebsd-stable@FreeBSD.ORG Sat Jan 8 20:37:25 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4482B106566C for ; Sat, 8 Jan 2011 20:37:25 +0000 (UTC) (envelope-from dan@langille.org) Received: from nyi.unixathome.org (nyi.unixathome.org [64.147.113.42]) by mx1.freebsd.org (Postfix) with ESMTP id 165358FC16 for ; Sat, 8 Jan 2011 20:37:25 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by nyi.unixathome.org (Postfix) with ESMTP id 69E7C50901 for ; Sat, 8 Jan 2011 20:37:24 +0000 (GMT) X-Virus-Scanned: amavisd-new at unixathome.org Received: from nyi.unixathome.org ([127.0.0.1]) by localhost (nyi.unixathome.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xzhhuTjX7aMk for ; Sat, 8 Jan 2011 20:37:24 +0000 (GMT) Received: from smtp-auth.unixathome.org (smtp-auth.unixathome.org [10.4.7.7]) (Authenticated sender: hidden) by nyi.unixathome.org (Postfix) with ESMTPSA id 2B147508F4 for ; Sat, 8 Jan 2011 20:37:24 +0000 (GMT) Message-ID: <4D28CB06.8030301@langille.org> Date: Sat, 08 Jan 2011 15:37:26 -0500 From: Dan Langille Organization: The FreeBSD Diary User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-stable Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: ZFS - benchmark & tuning before and after doubling RAM X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jan 2011 20:37:25 -0000 I've been running a ZFS array for about 10 months on a system with 4GB of RAM. I'm about to add another 4GB of RAM. I think this might be an opportune time to run some simple benchmarks and do some tuning. Getting more out of the system is not a priority for me. It does what I need now. However, I do see some merit in writing something up for others to see/follow/learn. The system is running FreeBSD 8.2-PRERELEASE #1: Tue Nov 30 22:07:59 EST 2010 on a 64 bit box. The ZFS array consists of 7x2TB commodity drives on two SiI3124 SATA controllers. The OS runs off a gmirror RAID-1. More details here: http://www.freebsddiary.org/zfs-benchmark.php First, up, I've done a simple bonnie++ benchmark before I add more RAM. I ran this on two different datasets; one with compression enabled, one without. If anyone has suggestions for various tests, option settings, etc, I'm happy to run them and include the results. We have lots of time to play with this. -- Dan Langille - http://langille.org/ From owner-freebsd-stable@FreeBSD.ORG Sat Jan 8 22:02:01 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 60F60106564A for ; Sat, 8 Jan 2011 22:02:01 +0000 (UTC) (envelope-from m.e.sanliturk@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 2656C8FC08 for ; Sat, 8 Jan 2011 22:02:00 +0000 (UTC) Received: by iyb26 with SMTP id 26so17470150iyb.13 for ; Sat, 08 Jan 2011 14:02:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=3YRI1nS1Q2EJK/C215OQbwbQ6+zaFm1thgtkei8yDw0=; b=o/Txwi6TbDIdk+sKR4qkNjokm362/45vop02lbFSG+LATQbZO8+DtlXcN+GL1JC4/G jwOiPs7f836orYoluLnH18Pz+evPLN9f4rPGvAaMUYUvUd+zG3q1tmwlC6cKOjIvNK82 LMIX9IP11DeRnr6dmULxBlP6m+EkyqgFXNv7A= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=nVNVS+tegEB8kXupYaWOv2nuv9AnklWYYQmeun+bRbxFw4o59bYjEO/9KQsoeFfdll tzIwDxxOgDlCyrWW32TriL+G8OSuQHKYCawcMI5VqlUSCmh69h3mTVI8RolwMZUtmwhJ YjNxDQGsM4gv356o/Z4dDTBot5r4xVj3ht3aM= MIME-Version: 1.0 Received: by 10.231.35.1 with SMTP id n1mr10947082ibd.0.1294522389025; Sat, 08 Jan 2011 13:33:09 -0800 (PST) Received: by 10.231.79.197 with HTTP; Sat, 8 Jan 2011 13:33:08 -0800 (PST) In-Reply-To: <4D28CB06.8030301@langille.org> References: <4D28CB06.8030301@langille.org> Date: Sat, 8 Jan 2011 16:33:08 -0500 Message-ID: From: Mehmet Erol Sanliturk To: Dan Langille Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-stable Subject: Re: ZFS - benchmark & tuning before and after doubling RAM X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jan 2011 22:02:01 -0000 On Sat, Jan 8, 2011 at 3:37 PM, Dan Langille wrote: > I've been running a ZFS array for about 10 months on a system with 4GB of > RAM. I'm about to add another 4GB of RAM. > > I think this might be an opportune time to run some simple benchmarks and > do some tuning. Getting more out of the system is not a priority for me. > It does what I need now. However, I do see some merit in writing something > up for others to see/follow/learn. > > The system is running FreeBSD 8.2-PRERELEASE #1: Tue Nov 30 22:07:59 EST > 2010 on a 64 bit box. The ZFS array consists of 7x2TB commodity drives on > two SiI3124 SATA controllers. The OS runs off a gmirror RAID-1. > > More details here: http://www.freebsddiary.org/zfs-benchmark.php > > First, up, I've done a simple bonnie++ benchmark before I add more RAM. I > ran this on two different datasets; one with compression enabled, one > without. > > If anyone has suggestions for various tests, option settings, etc, I'm > happy to run them and include the results. We have lots of time to play > with this. > > -- > Dan Langille - http://langille.org/ > I think , you know the following pages : http://hub.opensolaris.org/bin/view/Community+Group+zfs/zfstestsuite http://dlc.sun.com/osol/test/downloads/current/ http://hub.opensolaris.org/bin/view/Community+Group+testing/testsuites http://hub.opensolaris.org/bin/view/Community+Group+testing/zones Some of the links may disappear spontaneously because of restructuring of their respective sites . Thank you very much . Mehmet Erol Sanliturk From owner-freebsd-stable@FreeBSD.ORG Sat Jan 8 22:06:09 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A2968106566B for ; Sat, 8 Jan 2011 22:06:09 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from ftp.translate.ru (ftp.translate.ru [80.249.188.42]) by mx1.freebsd.org (Postfix) with ESMTP id 5BE088FC12 for ; Sat, 8 Jan 2011 22:06:09 +0000 (UTC) Received: from [192.168.134.2] (89.112.15.178.pppoe.eltel.net [89.112.15.178]) (Authenticated sender: lev@serebryakov.spb.ru) by ftp.translate.ru (Postfix) with ESMTPA id 23CE013DF5F; Sun, 9 Jan 2011 01:06:08 +0300 (MSK) Date: Sun, 9 Jan 2011 01:06:01 +0300 From: Lev Serebryakov Organization: FreeBSD Project X-Priority: 3 (Normal) Message-ID: <959936032.20110109010601@serebryakov.spb.ru> To: Kostik Belousov In-Reply-To: <20110108202028.GY12599@deviant.kiev.zoral.com.ua> References: <204344488.20110108214457@serebryakov.spb.ru> <20110108190232.GU12599@deviant.kiev.zoral.com.ua> <1792026896.20110108222909@serebryakov.spb.ru> <20110108195613.GW12599@deviant.kiev.zoral.com.ua> <1544327450.20110108231021@serebryakov.spb.ru> <20110108202028.GY12599@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: quoted-printable Cc: freebsd-stable@freebsd.org Subject: Re: 8.2-PRERELEASE: live deadlock, almost all processes in "pfault" state X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jan 2011 22:06:09 -0000 Hello, Kostik. You wrote 8 =D1=CE=D7=C1=D2=D1 2011 =C7., 23:20:28: >> >> And, if it is "classic deadlock" is here any "classical" solution to >> >> it? >> > Do not allocate during bio processing. >> So, if GEOM need some cache, it needs pre-allocate it and implements >> custom allocator over allocated chunk? :( >>=20 >> And what is "bio processing" in this context? geom_raid5 puts all > bio processing =3D=3D whole time needed to finish pageout. Pageout is > often performed to clean the page to lower the page shortage. > If pageout requires more free pages to finish during the shortage, > then we get the deadlock. Ok, and transmission mmap() files on geom_raid5, so when these pages are paged out, and geom_raid5 asks for other pages, and there is no free ones... I see. It seems, that M_NOWAIT flag should help, if geom_raid5 could live with failed mallocs... > Also, it seems that you allocate not only bios (small objects, not > every request cause page allocation), but also the huge buffers, that > require free pages each time. Yes, in worst case RAID5 need a lot of additional memory to perform simple write. If it is lone write (geom_raid5 waits some time for writes in adjacent areas, but not forever), geom_raid5 need to read (Number of disks - 1) x (size of write) bytes of data to re-calculate checksum. And it need buffers for this data. Worst case for 5-disks RAID5 and 128KiB write will be 4x128KiB =3D 512KiB of buffers. For one 128KiB write. And I don;t understand how to avoid deadlock here :( Maybe, preallocating some memory at start (these 512KiB) and try to use them when malloc() failed... I need to look how raid3 and vinum/raid5 lives with that situation. --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-stable@FreeBSD.ORG Sat Jan 8 22:11:24 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4ACE71065679 for ; Sat, 8 Jan 2011 22:11:24 +0000 (UTC) (envelope-from spork@bway.net) Received: from xena.bway.net (xena.bway.net [216.220.96.26]) by mx1.freebsd.org (Postfix) with ESMTP id 136568FC13 for ; Sat, 8 Jan 2011 22:11:23 +0000 (UTC) Received: (qmail 6612 invoked by uid 0); 8 Jan 2011 22:11:23 -0000 Received: from smtp.bway.net (216.220.96.25) by xena.bway.net with (DHE-RSA-AES256-SHA encrypted) SMTP; 8 Jan 2011 22:11:23 -0000 Received: (qmail 6604 invoked by uid 90); 8 Jan 2011 22:11:22 -0000 Received: from unknown (HELO hotlap.nat.fasttrackmonkey.com) (spork@96.57.144.66) by smtp.bway.net with (DHE-RSA-AES256-SHA encrypted) SMTP; 8 Jan 2011 22:11:22 -0000 Date: Sat, 8 Jan 2011 17:11:22 -0500 (EST) From: Charles Sprickman X-X-Sender: spork@hotlap.local To: Dan Langille In-Reply-To: <4D28CB06.8030301@langille.org> Message-ID: References: <4D28CB06.8030301@langille.org> User-Agent: Alpine 2.00 (OSX 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-stable Subject: Re: ZFS - benchmark & tuning before and after doubling RAM X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jan 2011 22:11:24 -0000 On Sat, 8 Jan 2011, Dan Langille wrote: > I've been running a ZFS array for about 10 months on a system with 4GB of > RAM. I'm about to add another 4GB of RAM. > > I think this might be an opportune time to run some simple benchmarks and do > some tuning. Getting more out of the system is not a priority for me. It > does what I need now. However, I do see some merit in writing something up > for others to see/follow/learn. > > The system is running FreeBSD 8.2-PRERELEASE #1: Tue Nov 30 22:07:59 EST 2010 > on a 64 bit box. The ZFS array consists of 7x2TB commodity drives on two > SiI3124 SATA controllers. The OS runs off a gmirror RAID-1. > > More details here: http://www.freebsddiary.org/zfs-benchmark.php > > First, up, I've done a simple bonnie++ benchmark before I add more RAM. I > ran this on two different datasets; one with compression enabled, one > without. > > If anyone has suggestions for various tests, option settings, etc, I'm happy > to run them and include the results. We have lots of time to play with this. iozone is interesting. I bought the excel plugin from the developer to generate the nice 3D graphs. I'd love to put some of my graphs up somewhere for comparison... It's a very time-consuming test though. Charles > -- > Dan Langille - http://langille.org/ > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > From owner-freebsd-stable@FreeBSD.ORG Sat Jan 8 23:02:44 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0943E1065670 for ; Sat, 8 Jan 2011 23:02:44 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id C497D8FC14 for ; Sat, 8 Jan 2011 23:02:43 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 0F1BD73098; Sun, 9 Jan 2011 00:17:04 +0100 (CET) Date: Sun, 9 Jan 2011 00:17:04 +0100 From: Luigi Rizzo To: Hans Petter Selasky Message-ID: <20110108231704.GA80865@onelab2.iet.unipi.it> References: <201101081841.10690.hselasky@c2i.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201101081841.10690.hselasky@c2i.net> User-Agent: Mutt/1.4.2.3i Cc: freebsd-multimedia@freebsd.org, freebsd-stable@freebsd.org Subject: Re: Webcamd testers wanted on FreeBSD 8.2 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jan 2011 23:02:44 -0000 speaking of webcamd, any idea on how to debug operation of skype and webcamd ? The thread at http://lists.freebsd.org/pipermail/freebsd-multimedia/2010-October/011327.html indicates the problem and shows symptoms. The most recent version of webcamd (0.1.18) now supports more of my webcams but still with similar problems. If (as it seems) the problem is related to the output format generated by the camera, i wonder if it is possible to tweak webcamd to generate a specific video format on startup, irrespective of the native camera format ? I am under the impression (perhaps wrong) that webcamd (or libv4l1) implements some conversion of the native formats ? cheers luigi From owner-freebsd-stable@FreeBSD.ORG Sat Jan 8 23:10:54 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 956931065673; Sat, 8 Jan 2011 23:10:54 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe08.c2i.net [212.247.154.226]) by mx1.freebsd.org (Postfix) with ESMTP id E9AB08FC19; Sat, 8 Jan 2011 23:10:53 +0000 (UTC) X-Cloudmark-Score: 0.000000 [] X-Cloudmark-Analysis: v=1.1 cv=Samd3CrC35PkFKGRAwjtIWdtalA6bcxM9GrYwcNK+gA= c=1 sm=1 a=d7IFvdU83yMA:10 a=8nJEP1OIZ-IA:10 a=CL8lFSKtTFcA:10 a=i9M/sDlu2rpZ9XS819oYzg==:17 a=6I5d2MoRAAAA:8 a=JcObK8pvlSJ1zZra1LcA:9 a=YGMScB5BwmUucQcn7d2iKnrxC64A:4 a=wPNLvfGTeEIA:10 a=i9M/sDlu2rpZ9XS819oYzg==:117 Received: from [188.126.198.129] (account mc467741@c2i.net HELO laptop002.hselasky.homeunix.org) by mailfe08.swip.net (CommuniGate Pro SMTP 5.2.19) with ESMTPA id 71499800; Sun, 09 Jan 2011 00:10:52 +0100 From: Hans Petter Selasky To: Luigi Rizzo Date: Sun, 9 Jan 2011 00:10:57 +0100 User-Agent: KMail/1.13.5 (FreeBSD/8.2-PRERELEASE; KDE/4.4.5; amd64; ; ) References: <201101081841.10690.hselasky@c2i.net> <20110108231704.GA80865@onelab2.iet.unipi.it> In-Reply-To: <20110108231704.GA80865@onelab2.iet.unipi.it> X-Face: +~\`s("[*|O,="7?X@L.elg*F"OA\I/3%^p8g?ab%RN'(; _IjlA: hGE..Ew, XAQ*o#\/M~SC=S1-f9{EzRfT'|Hhll5Q]ha5Bt-s|oTlKMusi:1e[wJl}kd}GR Z0adGx-x_0zGbZj'e(Y[(UNle~)8CQWXW@:DX+9)_YlB[tIccCPN$7/L' MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201101090010.57118.hselasky@c2i.net> Cc: freebsd-multimedia@freebsd.org, freebsd-stable@freebsd.org Subject: Re: Webcamd testers wanted on FreeBSD 8.2 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jan 2011 23:10:54 -0000 On Sunday 09 January 2011 00:17:04 Luigi Rizzo wrote: > speaking of webcamd, any idea on how to debug operation > of skype and webcamd ? > > The thread at > > http://lists.freebsd.org/pipermail/freebsd-multimedia/2010-October/011327.h > tml > > indicates the problem and shows symptoms. > The most recent version of webcamd (0.1.18) now supports > more of my webcams but still with similar problems. > > If (as it seems) the problem is related to the output format > generated by the camera, i wonder if it is possible to > tweak webcamd to generate a specific video format > on startup, irrespective of the native camera format ? > Hi Luigi, You can try adding a printout in: kernel/kern_file.c, function linux_ioctl() to get all the ioctls that skype issues. Maybe you get some clues from that. > I am under the impression (perhaps wrong) that webcamd > (or libv4l1) implements some conversion of the native formats ? libv4l does format conversion yes. That's the idea behind libv4l, to not do that format conversion in the kernel. --HPS From owner-freebsd-stable@FreeBSD.ORG Sat Jan 8 23:14:19 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 067BC106566B for ; Sat, 8 Jan 2011 23:14:19 +0000 (UTC) (envelope-from dan@langille.org) Received: from nyi.unixathome.org (nyi.unixathome.org [64.147.113.42]) by mx1.freebsd.org (Postfix) with ESMTP id CA4E98FC13 for ; Sat, 8 Jan 2011 23:14:18 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by nyi.unixathome.org (Postfix) with ESMTP id 2DFF850852; Sat, 8 Jan 2011 23:14:18 +0000 (GMT) X-Virus-Scanned: amavisd-new at unixathome.org Received: from nyi.unixathome.org ([127.0.0.1]) by localhost (nyi.unixathome.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oI3c5mDJXM7g; Sat, 8 Jan 2011 23:14:18 +0000 (GMT) Received: from smtp-auth.unixathome.org (smtp-auth.unixathome.org [10.4.7.7]) (Authenticated sender: hidden) by nyi.unixathome.org (Postfix) with ESMTPSA id DD81F50850 ; Sat, 8 Jan 2011 23:14:17 +0000 (GMT) Message-ID: <4D28EFCB.3070308@langille.org> Date: Sat, 08 Jan 2011 18:14:19 -0500 From: Dan Langille Organization: The FreeBSD Diary User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: Mehmet Erol Sanliturk References: <4D28CB06.8030301@langille.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-stable Subject: Re: ZFS - benchmark & tuning before and after doubling RAM X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jan 2011 23:14:19 -0000 On 1/8/2011 4:33 PM, Mehmet Erol Sanliturk wrote: > > > On Sat, Jan 8, 2011 at 3:37 PM, Dan Langille > wrote: > > I've been running a ZFS array for about 10 months on a system with > 4GB of RAM. I'm about to add another 4GB of RAM. > > I think this might be an opportune time to run some simple > benchmarks and do some tuning. Getting more out of the system is > not a priority for me. It does what I need now. However, I do see > some merit in writing something up for others to see/follow/learn. > > The system is running FreeBSD 8.2-PRERELEASE #1: Tue Nov 30 22:07:59 > EST 2010 on a 64 bit box. The ZFS array consists of 7x2TB commodity > drives on two SiI3124 SATA controllers. The OS runs off a gmirror > RAID-1. > > More details here: http://www.freebsddiary.org/zfs-benchmark.php > > First, up, I've done a simple bonnie++ benchmark before I add more > RAM. I ran this on two different datasets; one with compression > enabled, one without. > > If anyone has suggestions for various tests, option settings, etc, > I'm happy to run them and include the results. We have lots of time > to play with this. > > -- > Dan Langille - http://langille.org/ > > > > > I think , you know the following pages : > > http://hub.opensolaris.org/bin/view/Community+Group+zfs/zfstestsuite > http://dlc.sun.com/osol/test/downloads/current/ > http://hub.opensolaris.org/bin/view/Community+Group+testing/testsuites > http://hub.opensolaris.org/bin/view/Community+Group+testing/zones > > Some of the links may disappear spontaneously because of restructuring > of their respective sites . Looking briefly, them seen to be more aimed at regression testing than a benchmark. They all seem to be the same thing (just different instances). Perhaps I am mistaken, but I will look closer. -- Dan Langille - http://langille.org/ From owner-freebsd-stable@FreeBSD.ORG Sat Jan 8 23:47:22 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 508B610656BA for ; Sat, 8 Jan 2011 23:47:22 +0000 (UTC) (envelope-from m.e.sanliturk@gmail.com) Received: from mail-iw0-f182.google.com (mail-iw0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 134418FC0C for ; Sat, 8 Jan 2011 23:47:21 +0000 (UTC) Received: by iwn39 with SMTP id 39so18521908iwn.13 for ; Sat, 08 Jan 2011 15:47:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=DUm5zV8ckl10dvuwGHUP+AeL/xXIU/7niC8g83qmL/A=; b=ncpt+FQAnRhi0e3ZO8aywFj80QocA3o6HeW+xIqgf7C+YcsWS2yuemBhWkwl5XEEGO iibNfXCZqahuH2yI017tR+MVQpItodGvLsLY+BOMpWWTJ68KI90bvzvUrLEKZ1Gnj/aQ d6R/jm6GdYwggz0f/kl8ZeyYLQ5kgvqP7v6C8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=qwScG+a2oYeuPfDf86KuUYAmr+NlXPlkFxOiLw4AN1jxnggQIOdVkn+Ytl0yLc+f4l K0Bu11Twr8K4IwWnNrD3eHpIyTf9gs5mOW2zCocIpHgV3W0OiTZkMXspYvKMud2xf3Tl j53tkjomPh7mTKwTfIBfPGXrpw06O0YKGPR8E= MIME-Version: 1.0 Received: by 10.231.59.197 with SMTP id m5mr27288545ibh.25.1294530440156; Sat, 08 Jan 2011 15:47:20 -0800 (PST) Received: by 10.231.79.197 with HTTP; Sat, 8 Jan 2011 15:47:20 -0800 (PST) In-Reply-To: <4D28EFCB.3070308@langille.org> References: <4D28CB06.8030301@langille.org> <4D28EFCB.3070308@langille.org> Date: Sat, 8 Jan 2011 18:47:20 -0500 Message-ID: From: Mehmet Erol Sanliturk To: Dan Langille Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-stable Subject: Re: ZFS - benchmark & tuning before and after doubling RAM X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Jan 2011 23:47:22 -0000 On Sat, Jan 8, 2011 at 6:14 PM, Dan Langille wrote: > On 1/8/2011 4:33 PM, Mehmet Erol Sanliturk wrote: > >> >> >> On Sat, Jan 8, 2011 at 3:37 PM, Dan Langille > > wrote: >> >> I've been running a ZFS array for about 10 months on a system with >> 4GB of RAM. I'm about to add another 4GB of RAM. >> >> I think this might be an opportune time to run some simple >> benchmarks and do some tuning. Getting more out of the system is >> not a priority for me. It does what I need now. However, I do see >> some merit in writing something up for others to see/follow/learn. >> >> The system is running FreeBSD 8.2-PRERELEASE #1: Tue Nov 30 22:07:59 >> EST 2010 on a 64 bit box. The ZFS array consists of 7x2TB commodity >> drives on two SiI3124 SATA controllers. The OS runs off a gmirror >> RAID-1. >> >> More details here: http://www.freebsddiary.org/zfs-benchmark.php >> >> First, up, I've done a simple bonnie++ benchmark before I add more >> RAM. I ran this on two different datasets; one with compression >> enabled, one without. >> >> If anyone has suggestions for various tests, option settings, etc, >> I'm happy to run them and include the results. We have lots of time >> to play with this. >> >> -- >> Dan Langille - http://langille.org/ >> >> >> >> >> I think , you know the following pages : >> >> http://hub.opensolaris.org/bin/view/Community+Group+zfs/zfstestsuite >> http://dlc.sun.com/osol/test/downloads/current/ >> http://hub.opensolaris.org/bin/view/Community+Group+testing/testsuites >> http://hub.opensolaris.org/bin/view/Community+Group+testing/zones >> >> Some of the links may disappear spontaneously because of restructuring >> of their respective sites . >> > > Looking briefly, them seen to be more aimed at regression testing than a > benchmark. They all seem to be the same thing (just different instances). > > Perhaps I am mistaken, but I will look closer. > > > -- > Dan Langille - http://langille.org/ > Please , you may assume that , you are testing whether a change in hardware is breaking anything or not , before starting benchmarks . After assuring that anything is not broken , your benchmarks results will be more reliable . The benchmarks may not test all or some required features . Thank you very much . Mehmet Erol Sanliturk