From owner-freebsd-fs@FreeBSD.ORG Sun Jul 20 14:04:05 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EBA20B36; Sun, 20 Jul 2014 14:04:04 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C2A5A2630; Sun, 20 Jul 2014 14:04:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Content-Type:MIME-Version:Message-ID:Subject:To:From:Date; bh=900LGaysOB2Y78iPGwks4OhbRxCvIzb8vQzUlwyExnY=; b=mSw4lN8GRSG6L+0WPBH6gu+bAHzhZAPQ2/KJvZpcYQlUgqnLQdY7IpW1m2Sus0wyG0sLtnsLFBDeKMZsr60v8R9RIu9UDBbxrOyTrfJxklS3qO3si9TbS9L2sxG5qjCw0C1aJFKWV0eYgyigjxEh0PZb5eKSJN1UEJe8E2WuTJE=; Received: from 107-128-180-255.lightspeed.austtx.sbcglobal.net ([107.128.180.255]:39578 helo=borg.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.82_1-5b7a7c0-XX (FreeBSD)) (envelope-from ) id 1X8ric-000ANz-35; Sun, 20 Jul 2014 09:04:04 -0500 Date: Sun, 20 Jul 2014 09:03:50 -0500 From: Larry Rosenman To: freebsd-fs@freebsd.org, freebsd-current@freebsd.org Subject: [ZFS][PANIC] Solaris Assert/zio.c:2548 Message-ID: <20140720140350.GA8498@borg.lerctr.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Score: -2.9 (--) X-LERCTR-Spam-Score: -2.9 (--) X-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, TVD_RCVD_IP=0.001 X-LERCTR-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, TVD_RCVD_IP=0.001 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Jul 2014 14:04:05 -0000 Got the following panic overnight (I think while a nightly rsync was running): Dump header from device /dev/gpt/swap0 Architecture: amd64 Architecture Version: 2 Dump Length: 8122101760B (7745 MB) Blocksize: 512 Dumptime: Sun Jul 20 03:22:18 2014 Hostname: borg.lerctr.org Magic: FreeBSD Kernel Dump Version String: FreeBSD 11.0-CURRENT #50 r268894M: Sat Jul 19 18:06:08 CDT 2014 root@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER Panic String: solaris assert: !(zio->io_flags & ZIO_FLAG_DELEGATED), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c, line: 2874 Dump Parity: 763150733 Bounds: 5 Dump Status: good borg.lerctr.org dumped core - see /var/crash/vmcore.5 Sun Jul 20 03:28:12 CDT 2014 FreeBSD borg.lerctr.org 11.0-CURRENT FreeBSD 11.0-CURRENT #50 r268894M: Sat Jul 19 18:06:08 CDT 2014 root@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER amd64 panic: solaris assert: !(zio->io_flags & ZIO_FLAG_DELEGATED), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c, line: 2874 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: panic: solaris assert: !(zio->io_flags & ZIO_FLAG_DELEGATED), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c, line: 2874 cpuid = 7 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe100c49f930 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe100c49f9e0 vpanic() at vpanic+0x126/frame 0xfffffe100c49fa20 panic() at panic+0x43/frame 0xfffffe100c49fa80 assfail() at assfail+0x1d/frame 0xfffffe100c49fa90 zio_vdev_io_assess() at zio_vdev_io_assess+0x2ed/frame 0xfffffe100c49fac0 zio_execute() at zio_execute+0x1e9/frame 0xfffffe100c49fb20 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe100c49fb80 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe100c49fbb0 fork_exit() at fork_exit+0x84/frame 0xfffffe100c49fbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe100c49fbf0 --- trap 0, rip = 0, rsp = 0xfffffe100c49fcb0, rbp = 0 --- Uptime: 8h57m17s (ada2:ahcich2:0:0:0): FLUSHCACHE48. ACB: ea 00 00 00 00 40 00 00 00 00 00 00 (ada2:ahcich2:0:0:0): CAM status: Command timeout (ada2:ahcich2:0:0:0): Error 5, Retries exhausted (ada2:ahcich2:0:0:0): Synchronize cache failed Dumping 7745 out of 64463 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Reading symbols from /boot/kernel/linux.ko.symbols...done. Loaded symbols for /boot/kernel/linux.ko.symbols Reading symbols from /boot/kernel/if_lagg.ko.symbols...done. Loaded symbols for /boot/kernel/if_lagg.ko.symbols Reading symbols from /boot/kernel/snd_envy24ht.ko.symbols...done. Loaded symbols for /boot/kernel/snd_envy24ht.ko.symbols Reading symbols from /boot/kernel/snd_spicds.ko.symbols...done. Loaded symbols for /boot/kernel/snd_spicds.ko.symbols Reading symbols from /boot/kernel/coretemp.ko.symbols...done. Loaded symbols for /boot/kernel/coretemp.ko.symbols Reading symbols from /boot/kernel/ichsmb.ko.symbols...done. Loaded symbols for /boot/kernel/ichsmb.ko.symbols Reading symbols from /boot/kernel/smbus.ko.symbols...done. Loaded symbols for /boot/kernel/smbus.ko.symbols Reading symbols from /boot/kernel/ichwd.ko.symbols...done. Loaded symbols for /boot/kernel/ichwd.ko.symbols Reading symbols from /boot/kernel/cpuctl.ko.symbols...done. Loaded symbols for /boot/kernel/cpuctl.ko.symbols Reading symbols from /boot/kernel/crypto.ko.symbols...done. Loaded symbols for /boot/kernel/crypto.ko.symbols Reading symbols from /boot/kernel/cryptodev.ko.symbols...done. Loaded symbols for /boot/kernel/cryptodev.ko.symbols Reading symbols from /boot/kernel/dtraceall.ko.symbols...done. Loaded symbols for /boot/kernel/dtraceall.ko.symbols Reading symbols from /boot/kernel/profile.ko.symbols...done. Loaded symbols for /boot/kernel/profile.ko.symbols Reading symbols from /boot/kernel/cyclic.ko.symbols...done. Loaded symbols for /boot/kernel/cyclic.ko.symbols Reading symbols from /boot/kernel/dtrace.ko.symbols...done. Loaded symbols for /boot/kernel/dtrace.ko.symbols Reading symbols from /boot/kernel/systrace_freebsd32.ko.symbols...done. Loaded symbols for /boot/kernel/systrace_freebsd32.ko.symbols Reading symbols from /boot/kernel/systrace.ko.symbols...done. Loaded symbols for /boot/kernel/systrace.ko.symbols Reading symbols from /boot/kernel/sdt.ko.symbols...done. Loaded symbols for /boot/kernel/sdt.ko.symbols Reading symbols from /boot/kernel/lockstat.ko.symbols...done. Loaded symbols for /boot/kernel/lockstat.ko.symbols Reading symbols from /boot/kernel/fasttrap.ko.symbols...done. Loaded symbols for /boot/kernel/fasttrap.ko.symbols Reading symbols from /boot/kernel/fbt.ko.symbols...done. Loaded symbols for /boot/kernel/fbt.ko.symbols Reading symbols from /boot/kernel/dtnfscl.ko.symbols...done. Loaded symbols for /boot/kernel/dtnfscl.ko.symbols Reading symbols from /boot/kernel/dtmalloc.ko.symbols...done. Loaded symbols for /boot/kernel/dtmalloc.ko.symbols Reading symbols from /boot/modules/vboxdrv.ko...done. Loaded symbols for /boot/modules/vboxdrv.ko Reading symbols from /boot/kernel/ipmi.ko.symbols...done. Loaded symbols for /boot/kernel/ipmi.ko.symbols Reading symbols from /boot/kernel/ipmi_linux.ko.symbols...done. Loaded symbols for /boot/kernel/ipmi_linux.ko.symbols Reading symbols from /boot/kernel/radeonkms.ko.symbols...done. Loaded symbols for /boot/kernel/radeonkms.ko.symbols Reading symbols from /boot/kernel/iicbb.ko.symbols...done. Loaded symbols for /boot/kernel/iicbb.ko.symbols Reading symbols from /boot/kernel/iicbus.ko.symbols...done. Loaded symbols for /boot/kernel/iicbus.ko.symbols Reading symbols from /boot/kernel/iic.ko.symbols...done. Loaded symbols for /boot/kernel/iic.ko.symbols Reading symbols from /boot/kernel/drm2.ko.symbols...done. Loaded symbols for /boot/kernel/drm2.ko.symbols Reading symbols from /boot/kernel/radeonkmsfw_R100_cp.ko.symbols...done. Loaded symbols for /boot/kernel/radeonkmsfw_R100_cp.ko.symbols Reading symbols from /boot/kernel/fdescfs.ko.symbols...done. Loaded symbols for /boot/kernel/fdescfs.ko.symbols Reading symbols from /boot/kernel/linprocfs.ko.symbols...done. Loaded symbols for /boot/kernel/linprocfs.ko.symbols Reading symbols from /boot/kernel/uhid.ko.symbols...done. Loaded symbols for /boot/kernel/uhid.ko.symbols Reading symbols from /boot/modules/vboxnetflt.ko...done. Loaded symbols for /boot/modules/vboxnetflt.ko Reading symbols from /boot/kernel/netgraph.ko.symbols...done. Loaded symbols for /boot/kernel/netgraph.ko.symbols Reading symbols from /boot/kernel/ng_ether.ko.symbols...done. Loaded symbols for /boot/kernel/ng_ether.ko.symbols Reading symbols from /boot/modules/vboxnetadp.ko...done. Loaded symbols for /boot/modules/vboxnetadp.ko #0 doadump (textdump=1) at pcpu.h:219 219 pcpu.h: No such file or directory. in pcpu.h (kgdb) #0 doadump (textdump=1) at pcpu.h:219 #1 0xffffffff80a055f7 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:445 #2 0xffffffff80a05b35 in vpanic (fmt=, ap=) at /usr/src/sys/kern/kern_shutdown.c:744 #3 0xffffffff80a05b83 in panic (fmt=0x0) at /usr/src/sys/kern/kern_shutdown.c:673 #4 0xffffffff8032d05d in assfail (a=, f=, l=) at /usr/src/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c:81 #5 0xffffffff8040ad6d in zio_vdev_io_assess (ziop=) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:2874 #6 0xffffffff80405dd9 in zio_execute (zio=0xfffff809e449b730) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1416 #7 0xffffffff80a4de60 in taskqueue_run_locked (queue=0xfffff8002255e800) at /usr/src/sys/kern/subr_taskqueue.c:356 #8 0xffffffff80a4e8db in taskqueue_thread_loop (arg=) at /usr/src/sys/kern/subr_taskqueue.c:623 #9 0xffffffff809d3cc4 in fork_exit ( callout=0xffffffff80a4e840 , arg=0xfffff80022611470, frame=0xfffffe100c49fc00) at /usr/src/sys/kern/kern_fork.c:977 #10 0xffffffff80df5afe in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:605 #11 0x0000000000000000 in ?? () Current language: auto; currently minimal (kgdb) vmcore is available. I re-ran the rsync, and no panic. Ideas? -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-fs@FreeBSD.ORG Sun Jul 20 16:08:30 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0F4B32CC; Sun, 20 Jul 2014 16:08:30 +0000 (UTC) Received: from mail-out.smeets.im (mail-out.smeets.im [5.9.17.157]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A222E2190; Sun, 20 Jul 2014 16:08:29 +0000 (UTC) Received: from mail.smeets.im (mail.smeets.im [IPv6:2a01:4f8:160:918a::25:3]) by mail-out.smeets.im (Postfix) with ESMTP id 9534A102; Sun, 20 Jul 2014 18:08:19 +0200 (CEST) Received: from amavis.smeets.im (amavis.smeets.im [IPv6:2a01:4f8:160:918a::aa:4]) by mail.smeets.im (Postfix) with ESMTP id B6EFB892BE; Sun, 20 Jul 2014 18:08:19 +0200 (CEST) X-Virus-Scanned: amavisd-new at smeets.im Received: from mail.smeets.im ([IPv6:2a01:4f8:160:918a::25:3]) by amavis.smeets.im (amavis.smeets.im [IPv6:2a01:4f8:160:918a::aa:4]) (amavisd-new, port 10025) with ESMTP id 8wP71vdvbJaS; Sun, 20 Jul 2014 18:08:18 +0200 (CEST) Received: from nibbler-wlan.home.lan (unknown [IPv6:2001:4dd0:fd65:d00d:b0ad:5d65:8fad:53ad]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mail.smeets.im (Postfix) with ESMTPSA id C1A75892A7; Sun, 20 Jul 2014 18:08:17 +0200 (CEST) Message-ID: <53CBE970.7040904@smeets.im> Date: Sun, 20 Jul 2014 18:08:16 +0200 From: Florian Smeets User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Larry Rosenman , freebsd-fs@freebsd.org, freebsd-current@freebsd.org Subject: Re: [ZFS][PANIC] Solaris Assert/zio.c:2548 References: <20140720140350.GA8498@borg.lerctr.org> In-Reply-To: <20140720140350.GA8498@borg.lerctr.org> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="AgcRWoGoSWFWI2mOpumx6aIf7NdNBOb4A" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Jul 2014 16:08:30 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --AgcRWoGoSWFWI2mOpumx6aIf7NdNBOb4A Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 20/07/14 16:03, Larry Rosenman wrote: > Panic String: solaris assert: !(zio->io_flags & ZIO_FLAG_DELEGATED), > file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c, > line: 2874 >=20 > Unread portion of the kernel message buffer: panic: solaris assert: > !(zio->io_flags & ZIO_FLAG_DELEGATED), file: > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c, line: > 2874 cpuid =3D 7 KDB: stack backtrace: db_trace_self_wrapper() at > db_trace_self_wrapper+0x2b/frame 0xfffffe100c49f930 kdb_backtrace() > at kdb_backtrace+0x39/frame 0xfffffe100c49f9e0 vpanic() at > vpanic+0x126/frame 0xfffffe100c49fa20 panic() at panic+0x43/frame > 0xfffffe100c49fa80 assfail() at assfail+0x1d/frame > 0xfffffe100c49fa90 zio_vdev_io_assess() at > zio_vdev_io_assess+0x2ed/frame 0xfffffe100c49fac0 zio_execute() at > zio_execute+0x1e9/frame 0xfffffe100c49fb20 taskqueue_run_locked() at > taskqueue_run_locked+0xf0/frame 0xfffffe100c49fb80=20 > taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame > 0xfffffe100c49fbb0 fork_exit() at fork_exit+0x84/frame > 0xfffffe100c49fbf0 fork_trampoline() at fork_trampoline+0xe/frame > 0xfffffe100c49fbf0 --- trap 0, rip =3D 0, rsp =3D 0xfffffe100c49fcb0, r= bp > =3D 0 --- Uptime: 8h57m17s (ada2:ahcich2:0:0:0): FLUSHCACHE48. ACB: ea > 00 00 00 00 40 00 00 00 00 00 00 (ada2:ahcich2:0:0:0): CAM status: Same here, running poudriere the box panics reproducibly within 2-5 secon= ds. panic: solaris assert: !(zio->io_flags & ZIO_FLAG_DELEGATED), file: /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs= /zio.c, line: 2874 cpuid =3D 1 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00002e97f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe00002e98a0 vpanic() at vpanic+0x126/frame 0xfffffe00002e98e0 panic() at panic+0x43/frame 0xfffffe00002e9940 assfail() at assfail+0x1d/frame 0xfffffe00002e9950 zio_vdev_io_assess() at zio_vdev_io_assess+0x2e8/frame 0xfffffe00002e9980= zio_execute() at zio_execute+0x1e9/frame 0xfffffe00002e99e0 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe00002e9= a40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe00002e9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00002e9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00002e9ab0 --- trap 0, rip =3D 0, rsp =3D 0xfffffe00002e9b70, rbp =3D 0 --- KDB: enter: panic [ thread pid 0 tid 100422 ] Stopped at kdb_enter+0x3e: movq $0,kdb_why Florian --AgcRWoGoSWFWI2mOpumx6aIf7NdNBOb4A Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTy+lwAAoJEOcFPfn/hvB2m1gP/jGaEsE8fYSEDzsR3mrkCcvj ufua/sWSJ0mh7NuLmBHilxEibeqqWIeodken6aZ/OghDWkIK4S2vVox8e9dA/arL h/LxdpmL1Px3jwUQIyQnySj4vw+4e+l32mipwQ2Dy24PMixkwnHhx7PafkHigAeW Dvtx2O1DSRw+ohWYYlsabFlhnWwORd5dPFB9T0h55px/j2pP5zEiDs9hNgA5eI87 Ez4pq5oW9i6i/HbqfBtNwd57bCSn/ezgcLZF3JMntICh42AOoxNIy+2xkTkfTeRL 8FFQKeaEYaDHDuAMLEGLPzzJRmjETs5O2lhKmYwo76JUtjgUmU7+VyrBm1tgGKqI q4zIqiCLX2fkXIeP86LsUCG614RH4mkNF2yIFwNwXuPmYhUJqaqxsFVrUGSGLEVk WxguymQ6PtSfSpfdH85pPCJdb+PyRqt/mkYGVz9ec/X4wsEKqCcX1jNCQ7TTm0yV Zag3SLE1Hq+9/WKddZyN+JcnHJEYD+xBhfM5J3gclpq02LPYo66jOV2gP4X3tkPb hhVolX+AjZ+ytMaTB0tg0uh8Z3qPRlG3K940YnlCwsOrtzi5n7bMz976h4MbshuV CkyzB3OefWiQMTLxTdR/xTCjOAFS2e8nO4xyPaUG9rTVqe/A0yA99JmI9+YDntNE 7kgMr9/XRLZn7zVsTzw0 =gSTb -----END PGP SIGNATURE----- --AgcRWoGoSWFWI2mOpumx6aIf7NdNBOb4A-- From owner-freebsd-fs@FreeBSD.ORG Sun Jul 20 18:02:43 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2777AA3D for ; Sun, 20 Jul 2014 18:02:43 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0FDB72BD8 for ; Sun, 20 Jul 2014 18:02:43 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s6KI2gjw040351 for ; Sun, 20 Jul 2014 18:02:42 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 191782] [ufs] [lor] ufs/devfs on reboot r264318 Date: Sun, 20 Jul 2014 18:02:42 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to short_desc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Jul 2014 18:02:43 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191782 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-fs@FreeBSD.org Summary|LOR ufs/devfs on reboot |[ufs] [lor] ufs/devfs on |r264318 |reboot r264318 --- Comment #1 from Mark Linimon --- over to maintainers. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Sun Jul 20 18:05:53 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9F29BBA2 for ; Sun, 20 Jul 2014 18:05:53 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8748B2C03 for ; Sun, 20 Jul 2014 18:05:53 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s6KI5rRn080959 for ; Sun, 20 Jul 2014 18:05:53 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 191858] [msdosfs] [panic] Disposed page 0xfffff807d41eeb60 is dirty Date: Sun, 20 Jul 2014 18:05:53 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: Needs Triage X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Jul 2014 18:05:53 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191858 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-fs@FreeBSD.org --- Comment #1 from Mark Linimon --- over to maintainers. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Sun Jul 20 18:06:58 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CE9A9C75 for ; Sun, 20 Jul 2014 18:06:58 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B65482C21 for ; Sun, 20 Jul 2014 18:06:58 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s6KI6w1E094179 for ; Sun, 20 Jul 2014 18:06:58 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 191895] [ext2fs] [panic] ext2_dirbad: /mnt: bad dir ino 32787 at offset 6136: mangled entry Date: Sun, 20 Jul 2014 18:06:58 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: Needs Triage X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Jul 2014 18:06:58 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191895 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-fs@FreeBSD.org --- Comment #1 from Mark Linimon --- Over to maintainers. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Sun Jul 20 19:19:03 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4AF26C7E; Sun, 20 Jul 2014 19:19:03 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id CD712227D; Sun, 20 Jul 2014 19:19:02 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id 1B26920E7088D; Sun, 20 Jul 2014 19:19:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: ** X-Spam-Status: No, score=2.3 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,HELO_NO_DOMAIN,RDNS_DYNAMIC,STOX_REPLY_TYPE autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id 1CFC620E7088A; Sun, 20 Jul 2014 19:18:57 +0000 (UTC) Message-ID: <8D84B82C674B495DBF4951E0EE0D7117@multiplay.co.uk> From: "Steven Hartland" To: "Larry Rosenman" , , References: <20140720140350.GA8498@borg.lerctr.org> Subject: Re: [ZFS][PANIC] Solaris Assert/zio.c:2548 Date: Sun, 20 Jul 2014 20:18:55 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Jul 2014 19:19:03 -0000 Can you provide the details of the zio which caused the panic? Also does any of your pools support trim? Regards Steve ----- Original Message ----- From: "Larry Rosenman" To: ; Sent: Sunday, July 20, 2014 3:03 PM Subject: [ZFS][PANIC] Solaris Assert/zio.c:2548 > Got the following panic overnight (I think while a nightly rsync was running): > > Dump header from device /dev/gpt/swap0 > Architecture: amd64 > Architecture Version: 2 > Dump Length: 8122101760B (7745 MB) > Blocksize: 512 > Dumptime: Sun Jul 20 03:22:18 2014 > Hostname: borg.lerctr.org > Magic: FreeBSD Kernel Dump > Version String: FreeBSD 11.0-CURRENT #50 r268894M: Sat Jul 19 18:06:08 CDT 2014 > root@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER > Panic String: solaris assert: !(zio->io_flags & ZIO_FLAG_DELEGATED), file: > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c, line: 2874 > Dump Parity: 763150733 > Bounds: 5 > Dump Status: good > > > borg.lerctr.org dumped core - see /var/crash/vmcore.5 > > Sun Jul 20 03:28:12 CDT 2014 > > FreeBSD borg.lerctr.org 11.0-CURRENT FreeBSD 11.0-CURRENT #50 r268894M: Sat Jul 19 18:06:08 CDT 2014 > root@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER amd64 > > panic: solaris assert: !(zio->io_flags & ZIO_FLAG_DELEGATED), file: > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c, line: 2874 > > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "amd64-marcel-freebsd"... > > Unread portion of the kernel message buffer: > panic: solaris assert: !(zio->io_flags & ZIO_FLAG_DELEGATED), file: > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c, line: 2874 > cpuid = 7 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe100c49f930 > kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe100c49f9e0 > vpanic() at vpanic+0x126/frame 0xfffffe100c49fa20 > panic() at panic+0x43/frame 0xfffffe100c49fa80 > assfail() at assfail+0x1d/frame 0xfffffe100c49fa90 > zio_vdev_io_assess() at zio_vdev_io_assess+0x2ed/frame 0xfffffe100c49fac0 > zio_execute() at zio_execute+0x1e9/frame 0xfffffe100c49fb20 > taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe100c49fb80 > taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe100c49fbb0 > fork_exit() at fork_exit+0x84/frame 0xfffffe100c49fbf0 > fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe100c49fbf0 > --- trap 0, rip = 0, rsp = 0xfffffe100c49fcb0, rbp = 0 --- > Uptime: 8h57m17s > (ada2:ahcich2:0:0:0): FLUSHCACHE48. ACB: ea 00 00 00 00 40 00 00 00 00 00 00 > (ada2:ahcich2:0:0:0): CAM status: Command timeout > (ada2:ahcich2:0:0:0): Error 5, Retries exhausted > (ada2:ahcich2:0:0:0): Synchronize cache failed > Dumping 7745 out of 64463 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% > > Reading symbols from /boot/kernel/linux.ko.symbols...done. > Loaded symbols for /boot/kernel/linux.ko.symbols > Reading symbols from /boot/kernel/if_lagg.ko.symbols...done. > Loaded symbols for /boot/kernel/if_lagg.ko.symbols > Reading symbols from /boot/kernel/snd_envy24ht.ko.symbols...done. > Loaded symbols for /boot/kernel/snd_envy24ht.ko.symbols > Reading symbols from /boot/kernel/snd_spicds.ko.symbols...done. > Loaded symbols for /boot/kernel/snd_spicds.ko.symbols > Reading symbols from /boot/kernel/coretemp.ko.symbols...done. > Loaded symbols for /boot/kernel/coretemp.ko.symbols > Reading symbols from /boot/kernel/ichsmb.ko.symbols...done. > Loaded symbols for /boot/kernel/ichsmb.ko.symbols > Reading symbols from /boot/kernel/smbus.ko.symbols...done. > Loaded symbols for /boot/kernel/smbus.ko.symbols > Reading symbols from /boot/kernel/ichwd.ko.symbols...done. > Loaded symbols for /boot/kernel/ichwd.ko.symbols > Reading symbols from /boot/kernel/cpuctl.ko.symbols...done. > Loaded symbols for /boot/kernel/cpuctl.ko.symbols > Reading symbols from /boot/kernel/crypto.ko.symbols...done. > Loaded symbols for /boot/kernel/crypto.ko.symbols > Reading symbols from /boot/kernel/cryptodev.ko.symbols...done. > Loaded symbols for /boot/kernel/cryptodev.ko.symbols > Reading symbols from /boot/kernel/dtraceall.ko.symbols...done. > Loaded symbols for /boot/kernel/dtraceall.ko.symbols > Reading symbols from /boot/kernel/profile.ko.symbols...done. > Loaded symbols for /boot/kernel/profile.ko.symbols > Reading symbols from /boot/kernel/cyclic.ko.symbols...done. > Loaded symbols for /boot/kernel/cyclic.ko.symbols > Reading symbols from /boot/kernel/dtrace.ko.symbols...done. > Loaded symbols for /boot/kernel/dtrace.ko.symbols > Reading symbols from /boot/kernel/systrace_freebsd32.ko.symbols...done. > Loaded symbols for /boot/kernel/systrace_freebsd32.ko.symbols > Reading symbols from /boot/kernel/systrace.ko.symbols...done. > Loaded symbols for /boot/kernel/systrace.ko.symbols > Reading symbols from /boot/kernel/sdt.ko.symbols...done. > Loaded symbols for /boot/kernel/sdt.ko.symbols > Reading symbols from /boot/kernel/lockstat.ko.symbols...done. > Loaded symbols for /boot/kernel/lockstat.ko.symbols > Reading symbols from /boot/kernel/fasttrap.ko.symbols...done. > Loaded symbols for /boot/kernel/fasttrap.ko.symbols > Reading symbols from /boot/kernel/fbt.ko.symbols...done. > Loaded symbols for /boot/kernel/fbt.ko.symbols > Reading symbols from /boot/kernel/dtnfscl.ko.symbols...done. > Loaded symbols for /boot/kernel/dtnfscl.ko.symbols > Reading symbols from /boot/kernel/dtmalloc.ko.symbols...done. > Loaded symbols for /boot/kernel/dtmalloc.ko.symbols > Reading symbols from /boot/modules/vboxdrv.ko...done. > Loaded symbols for /boot/modules/vboxdrv.ko > Reading symbols from /boot/kernel/ipmi.ko.symbols...done. > Loaded symbols for /boot/kernel/ipmi.ko.symbols > Reading symbols from /boot/kernel/ipmi_linux.ko.symbols...done. > Loaded symbols for /boot/kernel/ipmi_linux.ko.symbols > Reading symbols from /boot/kernel/radeonkms.ko.symbols...done. > Loaded symbols for /boot/kernel/radeonkms.ko.symbols > Reading symbols from /boot/kernel/iicbb.ko.symbols...done. > Loaded symbols for /boot/kernel/iicbb.ko.symbols > Reading symbols from /boot/kernel/iicbus.ko.symbols...done. > Loaded symbols for /boot/kernel/iicbus.ko.symbols > Reading symbols from /boot/kernel/iic.ko.symbols...done. > Loaded symbols for /boot/kernel/iic.ko.symbols > Reading symbols from /boot/kernel/drm2.ko.symbols...done. > Loaded symbols for /boot/kernel/drm2.ko.symbols > Reading symbols from /boot/kernel/radeonkmsfw_R100_cp.ko.symbols...done. > Loaded symbols for /boot/kernel/radeonkmsfw_R100_cp.ko.symbols > Reading symbols from /boot/kernel/fdescfs.ko.symbols...done. > Loaded symbols for /boot/kernel/fdescfs.ko.symbols > Reading symbols from /boot/kernel/linprocfs.ko.symbols...done. > Loaded symbols for /boot/kernel/linprocfs.ko.symbols > Reading symbols from /boot/kernel/uhid.ko.symbols...done. > Loaded symbols for /boot/kernel/uhid.ko.symbols > Reading symbols from /boot/modules/vboxnetflt.ko...done. > Loaded symbols for /boot/modules/vboxnetflt.ko > Reading symbols from /boot/kernel/netgraph.ko.symbols...done. > Loaded symbols for /boot/kernel/netgraph.ko.symbols > Reading symbols from /boot/kernel/ng_ether.ko.symbols...done. > Loaded symbols for /boot/kernel/ng_ether.ko.symbols > Reading symbols from /boot/modules/vboxnetadp.ko...done. > Loaded symbols for /boot/modules/vboxnetadp.ko > #0 doadump (textdump=1) at pcpu.h:219 > 219 pcpu.h: No such file or directory. > in pcpu.h > (kgdb) #0 doadump (textdump=1) at pcpu.h:219 > #1 0xffffffff80a055f7 in kern_reboot (howto=260) > at /usr/src/sys/kern/kern_shutdown.c:445 > #2 0xffffffff80a05b35 in vpanic (fmt=, > ap=) at /usr/src/sys/kern/kern_shutdown.c:744 > #3 0xffffffff80a05b83 in panic (fmt=0x0) > at /usr/src/sys/kern/kern_shutdown.c:673 > #4 0xffffffff8032d05d in assfail (a=, > f=, l=) > at /usr/src/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c:81 > #5 0xffffffff8040ad6d in zio_vdev_io_assess (ziop=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:2874 > #6 0xffffffff80405dd9 in zio_execute (zio=0xfffff809e449b730) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1416 > #7 0xffffffff80a4de60 in taskqueue_run_locked (queue=0xfffff8002255e800) > at /usr/src/sys/kern/subr_taskqueue.c:356 > #8 0xffffffff80a4e8db in taskqueue_thread_loop (arg=) > at /usr/src/sys/kern/subr_taskqueue.c:623 > #9 0xffffffff809d3cc4 in fork_exit ( > callout=0xffffffff80a4e840 , > arg=0xfffff80022611470, frame=0xfffffe100c49fc00) > at /usr/src/sys/kern/kern_fork.c:977 > #10 0xffffffff80df5afe in fork_trampoline () > at /usr/src/sys/amd64/amd64/exception.S:605 > #11 0x0000000000000000 in ?? () > Current language: auto; currently minimal > (kgdb) > > > vmcore is available. > > I re-ran the rsync, and no panic. > > Ideas? > > > -- > Larry Rosenman http://www.lerctr.org/~ler > Phone: +1 214-642-9640 E-Mail: ler@lerctr.org > US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > From owner-freebsd-fs@FreeBSD.ORG Sun Jul 20 19:20:48 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 57826E31; Sun, 20 Jul 2014 19:20:48 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 26093229E; Sun, 20 Jul 2014 19:20:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=j+F8ia0fTQnOBTnG4StOjA8CP63GfrH5fyR/MZ123Lc=; b=Kvp29JuEuw8tBqnk5IX7UXxQ+KVUxh7+yr904Rw0zLgkG6OgdM1uFtedMuO/6JNZXCa8L2iE2glM+nHZCkR3Efhr8xEN74ih8HReExcuWmEg/rfFRb18ALfxX6R8wR+EBmSUt+GKj+afQwtRcCQUoiTITx4+oBPYDnHKojOCoOo=; Received: from localhost.lerctr.org ([127.0.0.1]:27329 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.82_1-5b7a7c0-XX (FreeBSD)) (envelope-from ) id 1X8wf6-000Exd-AJ; Sun, 20 Jul 2014 14:20:46 -0500 Received: from 107-128-180-255.lightspeed.austtx.sbcglobal.net ([107.128.180.255]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Sun, 20 Jul 2014 14:20:44 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Sun, 20 Jul 2014 14:20:44 -0500 From: Larry Rosenman To: Steven Hartland Subject: Re: [ZFS][PANIC] Solaris Assert/zio.c:2548 In-Reply-To: <8D84B82C674B495DBF4951E0EE0D7117@multiplay.co.uk> References: <20140720140350.GA8498@borg.lerctr.org> <8D84B82C674B495DBF4951E0EE0D7117@multiplay.co.uk> Message-ID: <21dbecad27074fe34610bc587e6d0764@thebighonker.lerctr.org> X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.0.1 X-Spam-Score: -2.9 (--) X-LERCTR-Spam-Score: -2.9 (--) X-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 X-LERCTR-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Jul 2014 19:20:48 -0000 On 2014-07-20 14:18, Steven Hartland wrote: > Can you provide the details of the zio which caused the panic? > > Also does any of your pools support trim? No, on the trim. Can you walk me through getting the zio you need? > > Regards > Steve > > ----- Original Message ----- From: "Larry Rosenman" > To: ; > Sent: Sunday, July 20, 2014 3:03 PM > Subject: [ZFS][PANIC] Solaris Assert/zio.c:2548 > > >> Got the following panic overnight (I think while a nightly rsync was >> running): >> >> Dump header from device /dev/gpt/swap0 >> Architecture: amd64 >> Architecture Version: 2 >> Dump Length: 8122101760B (7745 MB) >> Blocksize: 512 >> Dumptime: Sun Jul 20 03:22:18 2014 >> Hostname: borg.lerctr.org >> Magic: FreeBSD Kernel Dump >> Version String: FreeBSD 11.0-CURRENT #50 r268894M: Sat Jul 19 >> 18:06:08 CDT 2014 >> root@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER >> Panic String: solaris assert: !(zio->io_flags & ZIO_FLAG_DELEGATED), >> file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c, >> line: 2874 >> Dump Parity: 763150733 >> Bounds: 5 >> Dump Status: good >> >> >> borg.lerctr.org dumped core - see /var/crash/vmcore.5 >> >> Sun Jul 20 03:28:12 CDT 2014 >> >> FreeBSD borg.lerctr.org 11.0-CURRENT FreeBSD 11.0-CURRENT #50 >> r268894M: Sat Jul 19 18:06:08 CDT 2014 >> root@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER amd64 >> >> panic: solaris assert: !(zio->io_flags & ZIO_FLAG_DELEGATED), file: >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c, line: >> 2874 >> >> GNU gdb 6.1.1 [FreeBSD] >> Copyright 2004 Free Software Foundation, Inc. >> GDB is free software, covered by the GNU General Public License, and >> you are >> welcome to change it and/or distribute copies of it under certain >> conditions. >> Type "show copying" to see the conditions. >> There is absolutely no warranty for GDB. Type "show warranty" for >> details. >> This GDB was configured as "amd64-marcel-freebsd"... >> >> Unread portion of the kernel message buffer: >> panic: solaris assert: !(zio->io_flags & ZIO_FLAG_DELEGATED), file: >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c, line: >> 2874 >> cpuid = 7 >> KDB: stack backtrace: >> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame >> 0xfffffe100c49f930 >> kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe100c49f9e0 >> vpanic() at vpanic+0x126/frame 0xfffffe100c49fa20 >> panic() at panic+0x43/frame 0xfffffe100c49fa80 >> assfail() at assfail+0x1d/frame 0xfffffe100c49fa90 >> zio_vdev_io_assess() at zio_vdev_io_assess+0x2ed/frame >> 0xfffffe100c49fac0 >> zio_execute() at zio_execute+0x1e9/frame 0xfffffe100c49fb20 >> taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame >> 0xfffffe100c49fb80 >> taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame >> 0xfffffe100c49fbb0 >> fork_exit() at fork_exit+0x84/frame 0xfffffe100c49fbf0 >> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe100c49fbf0 >> --- trap 0, rip = 0, rsp = 0xfffffe100c49fcb0, rbp = 0 --- >> Uptime: 8h57m17s >> (ada2:ahcich2:0:0:0): FLUSHCACHE48. ACB: ea 00 00 00 00 40 00 00 00 00 >> 00 00 >> (ada2:ahcich2:0:0:0): CAM status: Command timeout >> (ada2:ahcich2:0:0:0): Error 5, Retries exhausted >> (ada2:ahcich2:0:0:0): Synchronize cache failed >> Dumping 7745 out of 64463 >> MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% >> >> Reading symbols from /boot/kernel/linux.ko.symbols...done. >> Loaded symbols for /boot/kernel/linux.ko.symbols >> Reading symbols from /boot/kernel/if_lagg.ko.symbols...done. >> Loaded symbols for /boot/kernel/if_lagg.ko.symbols >> Reading symbols from /boot/kernel/snd_envy24ht.ko.symbols...done. >> Loaded symbols for /boot/kernel/snd_envy24ht.ko.symbols >> Reading symbols from /boot/kernel/snd_spicds.ko.symbols...done. >> Loaded symbols for /boot/kernel/snd_spicds.ko.symbols >> Reading symbols from /boot/kernel/coretemp.ko.symbols...done. >> Loaded symbols for /boot/kernel/coretemp.ko.symbols >> Reading symbols from /boot/kernel/ichsmb.ko.symbols...done. >> Loaded symbols for /boot/kernel/ichsmb.ko.symbols >> Reading symbols from /boot/kernel/smbus.ko.symbols...done. >> Loaded symbols for /boot/kernel/smbus.ko.symbols >> Reading symbols from /boot/kernel/ichwd.ko.symbols...done. >> Loaded symbols for /boot/kernel/ichwd.ko.symbols >> Reading symbols from /boot/kernel/cpuctl.ko.symbols...done. >> Loaded symbols for /boot/kernel/cpuctl.ko.symbols >> Reading symbols from /boot/kernel/crypto.ko.symbols...done. >> Loaded symbols for /boot/kernel/crypto.ko.symbols >> Reading symbols from /boot/kernel/cryptodev.ko.symbols...done. >> Loaded symbols for /boot/kernel/cryptodev.ko.symbols >> Reading symbols from /boot/kernel/dtraceall.ko.symbols...done. >> Loaded symbols for /boot/kernel/dtraceall.ko.symbols >> Reading symbols from /boot/kernel/profile.ko.symbols...done. >> Loaded symbols for /boot/kernel/profile.ko.symbols >> Reading symbols from /boot/kernel/cyclic.ko.symbols...done. >> Loaded symbols for /boot/kernel/cyclic.ko.symbols >> Reading symbols from /boot/kernel/dtrace.ko.symbols...done. >> Loaded symbols for /boot/kernel/dtrace.ko.symbols >> Reading symbols from >> /boot/kernel/systrace_freebsd32.ko.symbols...done. >> Loaded symbols for /boot/kernel/systrace_freebsd32.ko.symbols >> Reading symbols from /boot/kernel/systrace.ko.symbols...done. >> Loaded symbols for /boot/kernel/systrace.ko.symbols >> Reading symbols from /boot/kernel/sdt.ko.symbols...done. >> Loaded symbols for /boot/kernel/sdt.ko.symbols >> Reading symbols from /boot/kernel/lockstat.ko.symbols...done. >> Loaded symbols for /boot/kernel/lockstat.ko.symbols >> Reading symbols from /boot/kernel/fasttrap.ko.symbols...done. >> Loaded symbols for /boot/kernel/fasttrap.ko.symbols >> Reading symbols from /boot/kernel/fbt.ko.symbols...done. >> Loaded symbols for /boot/kernel/fbt.ko.symbols >> Reading symbols from /boot/kernel/dtnfscl.ko.symbols...done. >> Loaded symbols for /boot/kernel/dtnfscl.ko.symbols >> Reading symbols from /boot/kernel/dtmalloc.ko.symbols...done. >> Loaded symbols for /boot/kernel/dtmalloc.ko.symbols >> Reading symbols from /boot/modules/vboxdrv.ko...done. >> Loaded symbols for /boot/modules/vboxdrv.ko >> Reading symbols from /boot/kernel/ipmi.ko.symbols...done. >> Loaded symbols for /boot/kernel/ipmi.ko.symbols >> Reading symbols from /boot/kernel/ipmi_linux.ko.symbols...done. >> Loaded symbols for /boot/kernel/ipmi_linux.ko.symbols >> Reading symbols from /boot/kernel/radeonkms.ko.symbols...done. >> Loaded symbols for /boot/kernel/radeonkms.ko.symbols >> Reading symbols from /boot/kernel/iicbb.ko.symbols...done. >> Loaded symbols for /boot/kernel/iicbb.ko.symbols >> Reading symbols from /boot/kernel/iicbus.ko.symbols...done. >> Loaded symbols for /boot/kernel/iicbus.ko.symbols >> Reading symbols from /boot/kernel/iic.ko.symbols...done. >> Loaded symbols for /boot/kernel/iic.ko.symbols >> Reading symbols from /boot/kernel/drm2.ko.symbols...done. >> Loaded symbols for /boot/kernel/drm2.ko.symbols >> Reading symbols from >> /boot/kernel/radeonkmsfw_R100_cp.ko.symbols...done. >> Loaded symbols for /boot/kernel/radeonkmsfw_R100_cp.ko.symbols >> Reading symbols from /boot/kernel/fdescfs.ko.symbols...done. >> Loaded symbols for /boot/kernel/fdescfs.ko.symbols >> Reading symbols from /boot/kernel/linprocfs.ko.symbols...done. >> Loaded symbols for /boot/kernel/linprocfs.ko.symbols >> Reading symbols from /boot/kernel/uhid.ko.symbols...done. >> Loaded symbols for /boot/kernel/uhid.ko.symbols >> Reading symbols from /boot/modules/vboxnetflt.ko...done. >> Loaded symbols for /boot/modules/vboxnetflt.ko >> Reading symbols from /boot/kernel/netgraph.ko.symbols...done. >> Loaded symbols for /boot/kernel/netgraph.ko.symbols >> Reading symbols from /boot/kernel/ng_ether.ko.symbols...done. >> Loaded symbols for /boot/kernel/ng_ether.ko.symbols >> Reading symbols from /boot/modules/vboxnetadp.ko...done. >> Loaded symbols for /boot/modules/vboxnetadp.ko >> #0 doadump (textdump=1) at pcpu.h:219 >> 219 pcpu.h: No such file or directory. >> in pcpu.h >> (kgdb) #0 doadump (textdump=1) at pcpu.h:219 >> #1 0xffffffff80a055f7 in kern_reboot (howto=260) >> at /usr/src/sys/kern/kern_shutdown.c:445 >> #2 0xffffffff80a05b35 in vpanic (fmt=, >> ap=) at /usr/src/sys/kern/kern_shutdown.c:744 >> #3 0xffffffff80a05b83 in panic (fmt=0x0) >> at /usr/src/sys/kern/kern_shutdown.c:673 >> #4 0xffffffff8032d05d in assfail (a=, >> f=, l=) >> at >> /usr/src/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c:81 >> #5 0xffffffff8040ad6d in zio_vdev_io_assess (ziop=> out>) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:2874 >> #6 0xffffffff80405dd9 in zio_execute (zio=0xfffff809e449b730) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1416 >> #7 0xffffffff80a4de60 in taskqueue_run_locked >> (queue=0xfffff8002255e800) >> at /usr/src/sys/kern/subr_taskqueue.c:356 >> #8 0xffffffff80a4e8db in taskqueue_thread_loop (arg=> out>) >> at /usr/src/sys/kern/subr_taskqueue.c:623 >> #9 0xffffffff809d3cc4 in fork_exit ( >> callout=0xffffffff80a4e840 , >> arg=0xfffff80022611470, frame=0xfffffe100c49fc00) >> at /usr/src/sys/kern/kern_fork.c:977 >> #10 0xffffffff80df5afe in fork_trampoline () >> at /usr/src/sys/amd64/amd64/exception.S:605 >> #11 0x0000000000000000 in ?? () >> Current language: auto; currently minimal >> (kgdb) >> >> >> vmcore is available. >> >> I re-ran the rsync, and no panic. >> >> Ideas? >> >> >> -- Larry Rosenman http://www.lerctr.org/~ler >> Phone: +1 214-642-9640 E-Mail: ler@lerctr.org >> US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 >> _______________________________________________ >> freebsd-current@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-current >> To unsubscribe, send any mail to >> "freebsd-current-unsubscribe@freebsd.org" >> -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-fs@FreeBSD.ORG Sun Jul 20 19:25:38 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C9C56262; Sun, 20 Jul 2014 19:25:38 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id 8E9BA2336; Sun, 20 Jul 2014 19:25:38 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id BD9A420E7088C; Sun, 20 Jul 2014 19:25:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: ** X-Spam-Status: No, score=2.0 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,HELO_NO_DOMAIN,RDNS_DYNAMIC autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id D58C420E7088A; Sun, 20 Jul 2014 19:25:33 +0000 (UTC) Message-ID: From: "Steven Hartland" To: "Larry Rosenman" References: <20140720140350.GA8498@borg.lerctr.org> <8D84B82C674B495DBF4951E0EE0D7117@multiplay.co.uk> <21dbecad27074fe34610bc587e6d0764@thebighonker.lerctr.org> Subject: Re: [ZFS][PANIC] Solaris Assert/zio.c:2548 Date: Sun, 20 Jul 2014 20:25:31 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Jul 2014 19:25:38 -0000 Something like following should allow you to get the zio details assuming the compile has optimised it out: cd /var/crash kgdb /boot/kernel/kernel /var/crash/vmcore.5 kgdb> frame 5 kgdb> print zio Regards Steve ----- Original Message ----- From: "Larry Rosenman" To: "Steven Hartland" Cc: ; Sent: Sunday, July 20, 2014 8:20 PM Subject: Re: [ZFS][PANIC] Solaris Assert/zio.c:2548 > On 2014-07-20 14:18, Steven Hartland wrote: >> Can you provide the details of the zio which caused the panic? >> >> Also does any of your pools support trim? > > No, on the trim. Can you walk me through getting the zio you need? From owner-freebsd-fs@FreeBSD.ORG Sun Jul 20 23:21:44 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 92FC2CC; Sun, 20 Jul 2014 23:21:44 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id 55B4F285E; Sun, 20 Jul 2014 23:21:44 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id 8DBB420E7088D; Sun, 20 Jul 2014 23:21:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: ** X-Spam-Status: No, score=2.0 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,HELO_NO_DOMAIN,RDNS_DYNAMIC,TO_EQ_FM_DIRECT_MX autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id 063D220E7088A; Sun, 20 Jul 2014 23:21:37 +0000 (UTC) Message-ID: From: "Steven Hartland" To: "Steven Hartland" , "Larry Rosenman" References: <20140720140350.GA8498@borg.lerctr.org> <8D84B82C674B495DBF4951E0EE0D7117@multiplay.co.uk> <21dbecad27074fe34610bc587e6d0764@thebighonker.lerctr.org> Subject: Re: [ZFS][PANIC] Solaris Assert/zio.c:2548 Date: Mon, 21 Jul 2014 00:21:36 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Jul 2014 23:21:44 -0000 Can you try reverting r265321 and see if you still see the same crash? Regards Steve From owner-freebsd-fs@FreeBSD.ORG Sun Jul 20 23:22:58 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EA74A32C; Sun, 20 Jul 2014 23:22:58 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B98152876; Sun, 20 Jul 2014 23:22:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=ifuzjFA2nV/8eSTNn0SSPdd3dmGiTvffF34t/bziZb4=; b=IBkaZf1t3OsancoGorRVjvFyjFY7M3hyZQ7GxNnlfmJUF8B50LH1TSbY+hXSgKeUGeFhEW6HLFH5RCcPvlBVqigDXoVYJC12/F0+tXQpIrwmgrvwykHw9GcJFc8YDK786LlhTxj9MPvEehdgrg5c2shSJBiKKB4iZOEQMYXpNZk=; Received: from localhost.lerctr.org ([127.0.0.1]:41943 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.82_1-5b7a7c0-XX (FreeBSD)) (envelope-from ) id 1X90RU-000ILb-AS; Sun, 20 Jul 2014 18:22:57 -0500 Received: from 107-128-180-255.lightspeed.austtx.sbcglobal.net ([107.128.180.255]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Sun, 20 Jul 2014 18:22:56 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Sun, 20 Jul 2014 18:22:56 -0500 From: Larry Rosenman To: Steven Hartland Subject: Re: [ZFS][PANIC] Solaris Assert/zio.c:2548 In-Reply-To: References: <20140720140350.GA8498@borg.lerctr.org> <8D84B82C674B495DBF4951E0EE0D7117@multiplay.co.uk> <21dbecad27074fe34610bc587e6d0764@thebighonker.lerctr.org> Message-ID: <05c8bccf4faf5005881fbd2b22f35428@thebighonker.lerctr.org> X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.0.1 X-Spam-Score: -2.9 (--) X-LERCTR-Spam-Score: -2.9 (--) X-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 X-LERCTR-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Jul 2014 23:22:59 -0000 On 2014-07-20 18:21, Steven Hartland wrote: > Can you try reverting r265321 and see if you still see the > same crash? > > Regards > Steve I'll do the revert, but it's been a ONE TIME hit. There was a followup to mine with a reproducible poudriere crash like mine. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-fs@FreeBSD.ORG Sun Jul 20 23:46:17 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7E70FC95; Sun, 20 Jul 2014 23:46:17 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id 40BA52A4B; Sun, 20 Jul 2014 23:46:17 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id 9724D20E7088C; Sun, 20 Jul 2014 23:46:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: ** X-Spam-Status: No, score=2.0 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,HELO_NO_DOMAIN,RDNS_DYNAMIC autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id AA41E20E7088A; Sun, 20 Jul 2014 23:46:12 +0000 (UTC) Message-ID: <4A0B8A1798484B3DB4DCD924A2DFEF22@multiplay.co.uk> From: "Steven Hartland" To: "Larry Rosenman" , "Florian Smeets" References: <20140720140350.GA8498@borg.lerctr.org> <8D84B82C674B495DBF4951E0EE0D7117@multiplay.co.uk> <21dbecad27074fe34610bc587e6d0764@thebighonker.lerctr.org> <05c8bccf4faf5005881fbd2b22f35428@thebighonker.lerctr.org> Subject: Re: [ZFS][PANIC] Solaris Assert/zio.c:2548 Date: Mon, 21 Jul 2014 00:46:10 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Jul 2014 23:46:17 -0000 ----- Original Message ----- From: "Larry Rosenman" To: "Steven Hartland" Cc: ; Sent: Monday, July 21, 2014 12:22 AM Subject: Re: [ZFS][PANIC] Solaris Assert/zio.c:2548 > On 2014-07-20 18:21, Steven Hartland wrote: >> Can you try reverting r265321 and see if you still see the >> same crash? >> >> Regards >> Steve > I'll do the revert, but it's been a ONE TIME hit. > > There was a followup to mine with a reproducible poudriere crash like > mine. If you don't have a reproducable senario I'd hold off. Florian, is yours reproducable and can you send me a pretty print of the crashing zio? Regards Steve From owner-freebsd-fs@FreeBSD.ORG Sun Jul 20 23:57:03 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DDDF5EC6; Sun, 20 Jul 2014 23:57:03 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AB4E92B0B; Sun, 20 Jul 2014 23:57:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=mkvfKueoyVuyV3ppJLNRUPazFMEvYSDzpNXr6zgICds=; b=j7KyO3CBDfasQsSxW27gNDWw7gDngfDqD5pwc0DP9miD2jiPzSYy+Tx1Ns8KV+OoK4zx8f3LnlrDQpkKIBxU8scWD8AgZEO0Ap32Od2MIceXSvkOwxvdOASDsdsPWF5B/eGgaMTha06fGQOgZad70FL0CP8+m+EI6E3nHbWY6AY=; Received: from localhost.lerctr.org ([127.0.0.1]:47982 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.82_1-5b7a7c0-XX (FreeBSD)) (envelope-from ) id 1X90yP-000IoD-Iv; Sun, 20 Jul 2014 18:56:58 -0500 Received: from 107-128-180-255.lightspeed.austtx.sbcglobal.net ([107.128.180.255]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Sun, 20 Jul 2014 18:56:57 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Sun, 20 Jul 2014 18:56:57 -0500 From: Larry Rosenman To: Steven Hartland Subject: Re: [ZFS][PANIC] Solaris Assert/zio.c:2548 In-Reply-To: <4A0B8A1798484B3DB4DCD924A2DFEF22@multiplay.co.uk> References: <20140720140350.GA8498@borg.lerctr.org> <8D84B82C674B495DBF4951E0EE0D7117@multiplay.co.uk> <21dbecad27074fe34610bc587e6d0764@thebighonker.lerctr.org> <05c8bccf4faf5005881fbd2b22f35428@thebighonker.lerctr.org> <4A0B8A1798484B3DB4DCD924A2DFEF22@multiplay.co.uk> Message-ID: <4992c5f5097b32dfd1dfbe7cd573cfa0@thebighonker.lerctr.org> X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.0.1 X-Spam-Score: -2.9 (--) X-LERCTR-Spam-Score: -2.9 (--) X-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 X-LERCTR-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Jul 2014 23:57:04 -0000 On 2014-07-20 18:46, Steven Hartland wrote: > ----- Original Message ----- From: "Larry Rosenman" > To: "Steven Hartland" > Cc: ; > Sent: Monday, July 21, 2014 12:22 AM > Subject: Re: [ZFS][PANIC] Solaris Assert/zio.c:2548 > > >> On 2014-07-20 18:21, Steven Hartland wrote: >>> Can you try reverting r265321 and see if you still see the >>> same crash? >>> >>> Regards >>> Steve >> I'll do the revert, but it's been a ONE TIME hit. >> >> There was a followup to mine with a reproducible poudriere crash like >> mine. > > If you don't have a reproducable senario I'd hold off. > > Florian, is yours reproducable and can you send me > a pretty print of the crashing zio? > > Regards > Steve running on the reverted kernel. We'll see if it stays up, crashes or what. Haven't seen the crash again regardless. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-fs@FreeBSD.ORG Sun Jul 20 23:57:46 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5A34AC2; Sun, 20 Jul 2014 23:57:46 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id 164FC2B1A; Sun, 20 Jul 2014 23:57:46 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id 75B5420E7088D; Sun, 20 Jul 2014 23:57:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: ** X-Spam-Status: No, score=2.0 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,HELO_NO_DOMAIN,RDNS_DYNAMIC autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id 511C020E7088A; Sun, 20 Jul 2014 23:57:41 +0000 (UTC) Message-ID: From: "Steven Hartland" To: "Dan Mack" References: <20140720140350.GA8498@borg.lerctr.org> <8D84B82C674B495DBF4951E0EE0D7117@multiplay.co.uk> <21dbecad27074fe34610bc587e6d0764@thebighonker.lerctr.org> <05c8bccf4faf5005881fbd2b22f35428@thebighonker.lerctr.org> <4A0B8A1798484B3DB4DCD924A2DFEF22@multiplay.co.uk> Subject: Re: [ZFS][PANIC] Solaris Assert/zio.c:2548 Date: Mon, 21 Jul 2014 00:57:39 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Jul 2014 23:57:46 -0000 ----- Original Message ----- From: "Dan Mack" > I think I may have hit the same problem; I'm going to stay connected > to the console and see if it happens again; this is what I see > currently with the back-trace: > > db> bt > Tracing pid 0 tid 100070 td 0xfffff8000e088920 > kdb_enter() at kdb_enter+0x3e/frame 0xfffffe085ef1d980 > vpanic() at vpanic+0x146/frame 0xfffffe085ef1d9c0 > panic() at panic+0x43/frame 0xfffffe085ef1da20 > deadlkres() at deadlkres+0x35c/frame 0xfffffe085ef1da70 > fork_exit() at fork_exit+0x84/frame 0xfffffe085ef1dab0 > fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085ef1dab0 > --- trap 0, rip = 0, rsp = 0xfffffe085ef1db70, rbp = 0 --- > > I just updated to I think 268921 earlier today and this is the first > time I've had a panic (HEAD-268921 that is) > > I'll try to get some more data if I can get it back up and running. That doesn't look like a related trace tbh. Regards Steve From owner-freebsd-fs@FreeBSD.ORG Mon Jul 21 00:01:47 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 328D23A4; Mon, 21 Jul 2014 00:01:47 +0000 (UTC) Received: from borg.macktronics.com (borg.macktronics.com [209.181.253.68]) by mx1.freebsd.org (Postfix) with ESMTP id 0A22C2BCF; Mon, 21 Jul 2014 00:01:46 +0000 (UTC) Received: from olive.macktronics.com (olive.macktronics.com [209.181.253.67]) by borg.macktronics.com (Postfix) with ESMTP id 1D0D613C; Sun, 20 Jul 2014 18:53:18 -0500 (CDT) Date: Sun, 20 Jul 2014 18:53:18 -0500 (CDT) From: Dan Mack To: Steven Hartland Subject: Re: [ZFS][PANIC] Solaris Assert/zio.c:2548 In-Reply-To: <4A0B8A1798484B3DB4DCD924A2DFEF22@multiplay.co.uk> Message-ID: References: <20140720140350.GA8498@borg.lerctr.org> <8D84B82C674B495DBF4951E0EE0D7117@multiplay.co.uk> <21dbecad27074fe34610bc587e6d0764@thebighonker.lerctr.org> <05c8bccf4faf5005881fbd2b22f35428@thebighonker.lerctr.org> <4A0B8A1798484B3DB4DCD924A2DFEF22@multiplay.co.uk> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2014 00:01:47 -0000 On Mon, 21 Jul 2014, Steven Hartland wrote: > ----- Original Message ----- From: "Larry Rosenman" > To: "Steven Hartland" > Cc: ; > Sent: Monday, July 21, 2014 12:22 AM > Subject: Re: [ZFS][PANIC] Solaris Assert/zio.c:2548 > > >> On 2014-07-20 18:21, Steven Hartland wrote: >>> Can you try reverting r265321 and see if you still see the >>> same crash? >>> >>> Regards >>> Steve >> I'll do the revert, but it's been a ONE TIME hit. >> >> There was a followup to mine with a reproducible poudriere crash like mine. > > If you don't have a reproducable senario I'd hold off. > > Florian, is yours reproducable and can you send me > a pretty print of the crashing zio? > > Regards > Steve I think I may have hit the same problem; I'm going to stay connected to the console and see if it happens again; this is what I see currently with the back-trace: db> bt Tracing pid 0 tid 100070 td 0xfffff8000e088920 kdb_enter() at kdb_enter+0x3e/frame 0xfffffe085ef1d980 vpanic() at vpanic+0x146/frame 0xfffffe085ef1d9c0 panic() at panic+0x43/frame 0xfffffe085ef1da20 deadlkres() at deadlkres+0x35c/frame 0xfffffe085ef1da70 fork_exit() at fork_exit+0x84/frame 0xfffffe085ef1dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085ef1dab0 --- trap 0, rip = 0, rsp = 0xfffffe085ef1db70, rbp = 0 --- I just updated to I think 268921 earlier today and this is the first time I've had a panic (HEAD-268921 that is) I'll try to get some more data if I can get it back up and running. dan -- Dan Mack From owner-freebsd-fs@FreeBSD.ORG Mon Jul 21 00:12:36 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 08B8660F; Mon, 21 Jul 2014 00:12:36 +0000 (UTC) Received: from borg.macktronics.com (borg.macktronics.com [209.181.253.68]) by mx1.freebsd.org (Postfix) with ESMTP id D2FB42C8F; Mon, 21 Jul 2014 00:12:35 +0000 (UTC) Received: from olive.macktronics.com (olive.macktronics.com [209.181.253.67]) by borg.macktronics.com (Postfix) with ESMTP id 343DB145; Sun, 20 Jul 2014 19:12:35 -0500 (CDT) Date: Sun, 20 Jul 2014 19:12:35 -0500 (CDT) From: Dan Mack To: Steven Hartland Subject: Re: [ZFS][PANIC] Solaris Assert/zio.c:2548 In-Reply-To: Message-ID: References: <20140720140350.GA8498@borg.lerctr.org> <8D84B82C674B495DBF4951E0EE0D7117@multiplay.co.uk> <21dbecad27074fe34610bc587e6d0764@thebighonker.lerctr.org> <05c8bccf4faf5005881fbd2b22f35428@thebighonker.lerctr.org> <4A0B8A1798484B3DB4DCD924A2DFEF22@multiplay.co.uk> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2014 00:12:36 -0000 On Mon, 21 Jul 2014, Steven Hartland wrote: > > ----- Original Message ----- From: "Dan Mack" > >> I think I may have hit the same problem; I'm going to stay connected >> to the console and see if it happens again; this is what I see >> currently with the back-trace: >> >> db> bt >> Tracing pid 0 tid 100070 td 0xfffff8000e088920 >> kdb_enter() at kdb_enter+0x3e/frame 0xfffffe085ef1d980 >> vpanic() at vpanic+0x146/frame 0xfffffe085ef1d9c0 >> panic() at panic+0x43/frame 0xfffffe085ef1da20 >> deadlkres() at deadlkres+0x35c/frame 0xfffffe085ef1da70 >> fork_exit() at fork_exit+0x84/frame 0xfffffe085ef1dab0 >> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085ef1dab0 >> --- trap 0, rip = 0, rsp = 0xfffffe085ef1db70, rbp = 0 --- >> >> I just updated to I think 268921 earlier today and this is the first >> time I've had a panic (HEAD-268921 that is) >> >> I'll try to get some more data if I can get it back up and running. > > That doesn't look like a related trace tbh. > > Regards > Steve Awesome, something else perhaps :-) Thanks, dan -- Dan Mack From owner-freebsd-fs@FreeBSD.ORG Mon Jul 21 01:29:33 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4DC832D8; Mon, 21 Jul 2014 01:29:33 +0000 (UTC) Received: from borg.macktronics.com (borg.macktronics.com [209.181.253.68]) by mx1.freebsd.org (Postfix) with ESMTP id 2370921D4; Mon, 21 Jul 2014 01:29:32 +0000 (UTC) Received: from olive.macktronics.com (olive.macktronics.com [209.181.253.67]) by borg.macktronics.com (Postfix) with ESMTP id 65747149; Sun, 20 Jul 2014 20:29:31 -0500 (CDT) Date: Sun, 20 Jul 2014 20:29:31 -0500 (CDT) From: Dan Mack To: Steven Hartland Subject: Re: [ZFS][PANIC] Solaris Assert/zio.c:2548 In-Reply-To: Message-ID: References: <20140720140350.GA8498@borg.lerctr.org> <8D84B82C674B495DBF4951E0EE0D7117@multiplay.co.uk> <21dbecad27074fe34610bc587e6d0764@thebighonker.lerctr.org> <05c8bccf4faf5005881fbd2b22f35428@thebighonker.lerctr.org> <4A0B8A1798484B3DB4DCD924A2DFEF22@multiplay.co.uk> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2014 01:29:33 -0000 On Mon, 21 Jul 2014, Steven Hartland wrote: >> I just updated to I think 268921 earlier today and this is the first >> time I've had a panic (HEAD-268921 that is) >> >> I'll try to get some more data if I can get it back up and running. > > That doesn't look like a related trace tbh. > > Regards > Steve After rebooting with a dumpdev; I got this : kbd2 at ukbd0 Trying to mount root from zfs:tank []... panic: deadlkres: possible deadlock detected for 0xfffff8000e089000, blocked for 1801216 ticks cpuid = 6 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe085ef1d8d0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe085ef1d980 vpanic() at vpanic+0x126/frame 0xfffffe085ef1d9c0 panic() at panic+0x43/frame 0xfffffe085ef1da20 deadlkres() at deadlkres+0x35c/frame 0xfffffe085ef1da70 fork_exit() at fork_exit+0x84/frame 0xfffffe085ef1dab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085ef1dab0 --- trap 0, rip = 0, rsp = 0xfffffe085ef1db70, rbp = 0 --- KDB: enter: panic [ thread pid 0 tid 100070 ] Stopped at kdb_enter+0x3e: movq $0,kdb_why I cannot seem to get past this yet so I'm open to suggestions. I'm still at the db> prompt if you'd like me to attempt to collect more info. dan -- Dan Mack From owner-freebsd-fs@FreeBSD.ORG Mon Jul 21 02:49:59 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B89412B9; Mon, 21 Jul 2014 02:49:59 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id 78B1C286C; Mon, 21 Jul 2014 02:49:59 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id D99AE20E7088D; Mon, 21 Jul 2014 02:49:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: ** X-Spam-Status: No, score=2.0 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,HELO_NO_DOMAIN,RDNS_DYNAMIC autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id AB18B20E7088A; Mon, 21 Jul 2014 02:49:46 +0000 (UTC) Message-ID: <7A0E2BBBB44F4977B847749706605223@multiplay.co.uk> From: "Steven Hartland" To: "Dan Mack" References: <20140720140350.GA8498@borg.lerctr.org> <8D84B82C674B495DBF4951E0EE0D7117@multiplay.co.uk> <21dbecad27074fe34610bc587e6d0764@thebighonker.lerctr.org> <05c8bccf4faf5005881fbd2b22f35428@thebighonker.lerctr.org> <4A0B8A1798484B3DB4DCD924A2DFEF22@multiplay.co.uk> Subject: Re: [ZFS][PANIC] Solaris Assert/zio.c:2548 Date: Mon, 21 Jul 2014 03:49:45 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2014 02:49:59 -0000 ----- Original Message ----- From: "Dan Mack" To: "Steven Hartland" Cc: ; ; "Larry Rosenman" Sent: Monday, July 21, 2014 2:29 AM Subject: Re: [ZFS][PANIC] Solaris Assert/zio.c:2548 > On Mon, 21 Jul 2014, Steven Hartland wrote: > >>> I just updated to I think 268921 earlier today and this is the first >>> time I've had a panic (HEAD-268921 that is) >>> >>> I'll try to get some more data if I can get it back up and running. >> >> That doesn't look like a related trace tbh. >> >> Regards >> Steve > > After rebooting with a dumpdev; I got this : > > kbd2 at ukbd0 > Trying to mount root from zfs:tank []... > panic: deadlkres: possible deadlock detected for 0xfffff8000e089000, blocked for 1801216 ticks > > cpuid = 6 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe085ef1d8d0 > kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe085ef1d980 > vpanic() at vpanic+0x126/frame 0xfffffe085ef1d9c0 > panic() at panic+0x43/frame 0xfffffe085ef1da20 > deadlkres() at deadlkres+0x35c/frame 0xfffffe085ef1da70 > fork_exit() at fork_exit+0x84/frame 0xfffffe085ef1dab0 > fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085ef1dab0 > --- trap 0, rip = 0, rsp = 0xfffffe085ef1db70, rbp = 0 --- > KDB: enter: panic > [ thread pid 0 tid 100070 ] > Stopped at kdb_enter+0x3e: movq $0,kdb_why > > I cannot seem to get past this yet so I'm open to suggestions. I'm > still at the db> prompt if you'd like me to attempt to collect more > info. For some reason the deadlock detector is triggering, not sure why. I'd recommend starting a new thread to discuss this as it doesn't appear to be related to this thread. The only thing I could suggest is disabling it to see if it truely is a deadlock or if something is being really slow. vfs.zfs.deadman_enabled=0 If this is new then it would be good for you to try and identify which of the changes introduced it, so do a binary chop on versions back to your last known good. Regards Steve From owner-freebsd-fs@FreeBSD.ORG Mon Jul 21 02:55:03 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1F66F492; Mon, 21 Jul 2014 02:55:03 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id AE05D2916; Mon, 21 Jul 2014 02:55:02 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id F3F8A20E7088C; Mon, 21 Jul 2014 02:55:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: ** X-Spam-Status: No, score=2.0 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,HELO_NO_DOMAIN,RDNS_DYNAMIC autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id 4849C20E7088A; Mon, 21 Jul 2014 02:54:57 +0000 (UTC) Message-ID: <24B81F5E7779408AAA0F2D6CA3B16CED@multiplay.co.uk> From: "Steven Hartland" To: "Dan Mack" References: <20140720140350.GA8498@borg.lerctr.org> <8D84B82C674B495DBF4951E0EE0D7117@multiplay.co.uk> <21dbecad27074fe34610bc587e6d0764@thebighonker.lerctr.org> <05c8bccf4faf5005881fbd2b22f35428@thebighonker.lerctr.org> <4A0B8A1798484B3DB4DCD924A2DFEF22@multiplay.co.uk> Subject: Re: [ZFS][PANIC] Solaris Assert/zio.c:2548 Date: Mon, 21 Jul 2014 03:54:56 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2014 02:55:03 -0000 ----- Original Message ----- From: "Dan Mack" To: "Steven Hartland" Cc: ; ; "Larry Rosenman" Sent: Monday, July 21, 2014 2:29 AM Subject: Re: [ZFS][PANIC] Solaris Assert/zio.c:2548 > On Mon, 21 Jul 2014, Steven Hartland wrote: > >>> I just updated to I think 268921 earlier today and this is the first >>> time I've had a panic (HEAD-268921 that is) >>> >>> I'll try to get some more data if I can get it back up and running. >> >> That doesn't look like a related trace tbh. >> >> Regards >> Steve > > After rebooting with a dumpdev; I got this : > > kbd2 at ukbd0 > Trying to mount root from zfs:tank []... > panic: deadlkres: possible deadlock detected for 0xfffff8000e089000, blocked for 1801216 ticks > > cpuid = 6 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe085ef1d8d0 > kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe085ef1d980 > vpanic() at vpanic+0x126/frame 0xfffffe085ef1d9c0 > panic() at panic+0x43/frame 0xfffffe085ef1da20 > deadlkres() at deadlkres+0x35c/frame 0xfffffe085ef1da70 > fork_exit() at fork_exit+0x84/frame 0xfffffe085ef1dab0 > fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe085ef1dab0 > --- trap 0, rip = 0, rsp = 0xfffffe085ef1db70, rbp = 0 --- > KDB: enter: panic > [ thread pid 0 tid 100070 ] > Stopped at kdb_enter+0x3e: movq $0,kdb_why > > I cannot seem to get past this yet so I'm open to suggestions. I'm > still at the db> prompt if you'd like me to attempt to collect more > info. Just spotted an interesting message on a recent commit which may be relavent: > URL: http://svnweb.freebsd.org/changeset/base/268855 > This specific commit makes boot hang just before mounting the root > dataset for me when vfs.zfs.vdev.cache.size tunable is set. Unsetting > this tunable or reverting this commit (currently running r268933 minus > r268855) fixes the boot for me. > > Please let me know if I can provide any more information. > > - Nikolai Lifanov The current code disables vdev caching by default so this will only occur if manually enabled. The code details the reason for this as:- * TODO: Note that with the current ZFS code, it turns out that the * vdev cache is not helpful, and in some cases actually harmful. It * is better if we disable this. Once some time has passed, we should * actually remove this to simplify the code. For now we just disable * it by setting the zfs_vdev_cache_size to zero. Note that Solaris 11 * has made these same changes. Regards Steve From owner-freebsd-fs@FreeBSD.ORG Mon Jul 21 05:24:34 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D5C141A5; Mon, 21 Jul 2014 05:24:34 +0000 (UTC) Received: from mail-out.smeets.im (mail-out.smeets.im [IPv6:2a01:4f8:160:918a::25:11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7E1732546; Mon, 21 Jul 2014 05:24:34 +0000 (UTC) Received: from mail.smeets.im (mail.smeets.im [IPv6:2a01:4f8:160:918a::25:3]) by mail-out.smeets.im (Postfix) with ESMTP id 3C74918B; Mon, 21 Jul 2014 07:24:32 +0200 (CEST) Received: from amavis.smeets.im (amavis.smeets.im [IPv6:2a01:4f8:160:918a::aa:4]) by mail.smeets.im (Postfix) with ESMTP id B9C0D892BD; Mon, 21 Jul 2014 07:24:32 +0200 (CEST) X-Virus-Scanned: amavisd-new at smeets.im Received: from mail.smeets.im ([IPv6:2a01:4f8:160:918a::25:3]) by amavis.smeets.im (amavis.smeets.im [IPv6:2a01:4f8:160:918a::aa:4]) (amavisd-new, port 10025) with ESMTP id KmrFO1qGupFZ; Mon, 21 Jul 2014 07:24:31 +0200 (CEST) Received: from nibbler-osx.local (unknown [IPv6:2001:4dd0:fd65:d00d:fdfe:c788:e9be:e820]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mail.smeets.im (Postfix) with ESMTPSA id C5FD7892A7; Mon, 21 Jul 2014 07:24:30 +0200 (CEST) Message-ID: <53CCA40B.10503@smeets.im> Date: Mon, 21 Jul 2014 07:24:27 +0200 From: Florian Smeets User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Steven Hartland , Larry Rosenman Subject: Re: [ZFS][PANIC] Solaris Assert/zio.c:2548 References: <20140720140350.GA8498@borg.lerctr.org> <8D84B82C674B495DBF4951E0EE0D7117@multiplay.co.uk> <21dbecad27074fe34610bc587e6d0764@thebighonker.lerctr.org> <05c8bccf4faf5005881fbd2b22f35428@thebighonker.lerctr.org> <4A0B8A1798484B3DB4DCD924A2DFEF22@multiplay.co.uk> In-Reply-To: <4A0B8A1798484B3DB4DCD924A2DFEF22@multiplay.co.uk> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="rp9sFLx0TfVQoplMJscgoqQj9Iqaw3u1w" Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2014 05:24:34 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --rp9sFLx0TfVQoplMJscgoqQj9Iqaw3u1w Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 21/07/14 01:46, Steven Hartland wrote: > ----- Original Message ----- From: "Larry Rosenman" > To: "Steven Hartland" > Cc: ; > Sent: Monday, July 21, 2014 12:22 AM > Subject: Re: [ZFS][PANIC] Solaris Assert/zio.c:2548 >=20 >=20 >> On 2014-07-20 18:21, Steven Hartland wrote: >>> Can you try reverting r265321 and see if you still see the >>> same crash? >>> >>> Regards >>> Steve >> I'll do the revert, but it's been a ONE TIME hit. >> >> There was a followup to mine with a reproducible poudriere crash like >> mine. >=20 > If you don't have a reproducable senario I'd hold off. >=20 > Florian, is yours reproducable and can you send me > a pretty print of the crashing zio? >=20 My backtrace looks a little different. panic: solaris assert: !(zio->io_flags & ZIO_FLAG_DELEGATED), file: /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs= /zio.c, line: 2874 cpuid =3D 3 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00002e97f0 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe00002e98a0 vpanic() at vpanic+0x126/frame 0xfffffe00002e98e0 panic() at panic+0x43/frame 0xfffffe00002e9940 assfail() at assfail+0x1d/frame 0xfffffe00002e9950 zio_vdev_io_assess() at zio_vdev_io_assess+0x2e8/frame 0xfffffe00002e9980= zio_execute() at zio_execute+0x1e9/frame 0xfffffe00002e99e0 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe00002e9= a40 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe00002e9a70 fork_exit() at fork_exit+0x84/frame 0xfffffe00002e9ab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00002e9ab0 --- trap 0, rip =3D 0, rsp =3D 0xfffffe00002e9b70, rbp =3D 0 --- KDB: enter: panic (kgdb) where #0 doadump (textdump=3D-2125462752) at pcpu.h:219 #1 0xffffffff80347655 in db_fncall (dummy1=3D, dummy2=3D, dummy3=3D, dummy4=3D) at /usr/src/sys/ddb/db_command.c:578 #2 0xffffffff8034733d in db_command (cmd_table=3D0x0) at /usr/src/sys/ddb/db_command.c:449 #3 0xffffffff803470b4 in db_command_loop () at /usr/src/sys/ddb/db_command.c:502 #4 0xffffffff80349a90 in db_trap (type=3D, code=3D0= ) at /usr/src/sys/ddb/db_main.c:231 #5 0xffffffff80944159 in kdb_trap (type=3D3, code=3D0, tf=3D) at /usr/src/sys/kern/subr_kdb.c:654 #6 0xffffffff80d1e532 in trap (frame=3D0xfffffe00002e97d0) at /usr/src/sys/amd64/amd64/trap.c:542 #7 0xffffffff80d01202 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:231 #8 0xffffffff809438be in kdb_enter (why=3D0xffffffff80f9ce38 "panic", msg=3D) at cpufunc.h:63 #9 0xffffffff8090bb66 in vpanic (fmt=3D, ap=3D) at /usr/src/sys/kern/kern_shutdown.c:737 #10 0xffffffff8090bbd3 in panic (fmt=3D0xffffffff815a59a0 "\004") at /usr/src/sys/kern/kern_shutdown.c:673 #11 0xffffffff81fb821d in assfail (a=3D, ---Type to continue, or q to quit--- f=3D, l=3D) at /usr/src/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c:81= #12 0xffffffff81eca848 in zio_vdev_io_assess (ziop=3D) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs= /zio.c:2874 #13 0xffffffff81ec58b9 in zio_execute (zio=3D0xfffff801a8abc398) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs= /zio.c:1416 #14 0xffffffff80954150 in taskqueue_run_locked (queue=3D0xfffff80009249b0= 0) at /usr/src/sys/kern/subr_taskqueue.c:356 #15 0xffffffff80954c1b in taskqueue_thread_loop (arg=3D) at /usr/src/sys/kern/subr_taskqueue.c:623 #16 0xffffffff808d9834 in fork_exit ( callout=3D0xffffffff80954b80 , arg=3D0xfffff80003dfeed0, frame=3D0xfffffe00002e9ac0) at /usr/src/sys/kern/kern_fork.c:977 #17 0xffffffff80d0173e in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:605 #18 0x0000000000000000 in ?? () (kgdb) frame 12 #12 0xffffffff81eca848 in zio_vdev_io_assess (ziop=3D) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs= /zio.c:2874 2874 ASSERT(!(zio->io_flags & ZIO_FLAG_DELEGATED)); (kgdb) print zio $3 =3D (zio_t *) 0xfffff801a8abc398 (kgdb) print *zio $4 =3D {io_bookmark =3D {zb_objset =3D 4339, zb_object =3D 327827, zb_lev= el =3D 0, zb_blkid =3D 0}, io_prop =3D {zp_checksum =3D ZIO_CHECKSUM_INHERIT, zp_compress =3D ZIO_COMPRESS_INHERIT, zp_type =3D DMU_OT_NONE, zp_level =3D 0 '\0', zp_copies =3D 0 '\0', zp_dedup =3D 0, zp_dedup_v= erify =3D 0, zp_nopwrite =3D 0}, io_type =3D ZIO_TYPE_WRITE, io_child_type =3D ZIO_CHILD_VDEV, io_cmd =3D 0, io_priority =3D ZIO_PRIORITY_ASYNC_WRITE, io_reexecute =3D 0 '\0', io_state =3D "\001", io_txg =3D 1312558, io_spa =3D 0xfffffe00022e6000,= io_bp =3D 0xfffffe000a94a640, io_bp_override =3D 0x0, io_bp_copy =3D {blk_dva =3D {{ dva_word =3D {1, 58754170}}, {dva_word =3D {1, 69614673}}, {dva_w= ord =3D {0, 0}}}, blk_prop =3D 9229009297394892802, blk_pad =3D {0, 0}, blk_phys_birth =3D 0, blk_birth =3D 1312558, blk_fill =3D 1, blk_cksu= m =3D { zc_word =3D {72684358009, 6982033287555, 350329209490535, 12175142665158025}}}, io_parent_list =3D {list_size =3D 48, list_offset =3D 16, list_head =3D {list_next =3D 0xfffff800092e7520, list_prev =3D 0xfffff800092e7520}}, io_child_list =3D {list_size =3D= 48, list_offset =3D 32, list_head =3D {list_next =3D 0xfffff801a8abc4b8, list_prev =3D 0xfffff801a8abc4b8}}, io_walk_link =3D 0x0, io_logical =3D 0xfffff801a8cb7730, io_transform_stack =3D 0x0, io_ready= =3D 0, io_physdone =3D 0xffffffff81e34ee0 , io_done =3D 0xffffffff81ea7ea0 , io_private =3D 0xfffff80027266d18, io_prev_space_delta =3D 0, io_bp_ori= g =3D { blk_dva =3D {{dva_word =3D {1, 58754170}}, {dva_word =3D {1, 69614673= }}, { ---Type to continue, or q to quit--- dva_word =3D {0, 0}}}, blk_prop =3D 9229009297394892802, blk_pad = =3D {0, 0}, blk_phys_birth =3D 0, blk_birth =3D 1312558, blk_fill =3D 1, blk_cksu= m =3D { zc_word =3D {72684358009, 6982033287555, 350329209490535, 12175142665158025}}}, io_data =3D 0xfffff8017b0b3000, io_orig_data =3D 0xfffff8017b0b3000, io_size =3D 512, io_orig_size =3D = 512, io_vd =3D 0xfffff8000935f800, io_vsd =3D 0x0, io_vsd_ops =3D 0x0, io_offset =3D 30086329344, io_timestamp =3D 76492632312, io_queue_node = =3D { avl_child =3D {0x0, 0x0}, avl_pcb =3D 18446735284737973505}, io_flags =3D 269224064, io_stage =3D ZIO_STAGE_VDEV_IO_ASSESS, io_pipeline =3D 3014656, io_orig_flags =3D 524416, io_orig_stage =3D ZIO_STAGE_READY, io_orig_pipeline =3D 3014656, io_err= or =3D 0, io_child_error =3D {0, 0, 0, 0}, io_children =3D {{0, 0}, {0, 0}, {0, 0= }, {0, 0}}, io_child_count =3D 0, io_phys_children =3D 0, io_parent_count = =3D 1, io_stall =3D 0x0, io_gang_leader =3D 0x0, io_gang_tree =3D 0x0, io_executor =3D 0xfffff80009265490, io_waiter =3D 0x0, io_lock =3D {lock_object =3D { lo_name =3D 0xffffffff81f763a5 "zio->io_lock", lo_flags =3D 4096000= 0, lo_data =3D 0, lo_witness =3D 0x0}, sx_lock =3D 1}, io_cv =3D { cv_description =3D 0xffffffff81f763b3 "zio->io_cv", cv_waiters =3D 0}= , io_cksum_report =3D 0x0, io_ena =3D 0, io_tqent =3D {tqent_task =3D {ta= _link =3D { stqe_next =3D 0x0}, ta_pending =3D 0, ta_priority =3D 0, ta_func =3D 0xffffffff81e1dd60 , ta_context =3D 0xfffff801a8abc6d8}, tqent_func =3D 0xffffffff81ec56d0 , ---Type to continue, or q to quit--- tqent_arg =3D 0xfffff801a8abc398}, io_trim_node =3D {avl_child =3D {0= x0, 0x0}, avl_pcb =3D 0}, io_trim_link =3D {list_next =3D 0x0, list_prev =3D 0x= 0}} I'm not an expert with kgdb. If you need anything else let me know what you need. Florian --rp9sFLx0TfVQoplMJscgoqQj9Iqaw3u1w Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTzKQMAAoJEOcFPfn/hvB2+QIQAN0ut0ahLdyhD527Ecz3fskG YGZ/hW8R6pKPBTv9ZceXfAwgH5uJ4cF2yqXL9Mrj+hYQqcltDPhSwyyHXwIrkLig 4RrajKRBGF4ybYEZYzTwnYTUVCvv8EtmAeZm1Q1N3vQ5FAEw7cwNZzCEumZawMue ZmbVI+id1LvqALORBuX3HH3HcEB8QeJ2rWJboP+n9+ttxuPJMv69K6HPwxVEgRQ+ MIY0P+BVLRLX9Q5/Hl8q4YDJwlL3NB74jlNT46wyHcfLVWxc5FlGILXwsW0IlGbl ybEPQHlhgutSibQsuUY8bbxkhR4Xn+NISIv8tG6eH5aLE8OHDIyWiBvOoHm7HEfl hXdkA1UAoX5QzrhMmlvMnWoH72SPwlOh/1cpHb4FnfY5APcWkaURDCflx1CuXATQ fYaGHkrHHoT5K+h+ADwu4+W6nrx8rAC3f2vNjJrgvz3kwSsLVUBBIpoHy5pkwfkA bm8+iNhwBXyV6cs4Fyv8OsEAyN7EUd/7kxIeBoj5ogxcPUN1oiA+FXruMMx+wdTA vMSKIOV51NhSaFsppUwy+uQaLI3FZWvMshC1DVHDR12DX0i86F3X0MHyu3QUZqcz TFIoywoF2LtVYuzYCoewfyyCBOe3/1LkXCoRt9uxhzMOwOv9HVs8WQ0pbUN5RcY7 UY7ZUh0QMItMtlILk/c4 =d1HU -----END PGP SIGNATURE----- --rp9sFLx0TfVQoplMJscgoqQj9Iqaw3u1w-- From owner-freebsd-fs@FreeBSD.ORG Mon Jul 21 07:59:00 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1B41F33A; Mon, 21 Jul 2014 07:59:00 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DCFE1225B; Mon, 21 Jul 2014 07:58:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=lB1XuBgFw4VawDjxBGgpFG8vYzm8EAilOi5ZVNMHhYI=; b=cSUmACnK7aZXlOv4XZpFeiSh1N8g0QBoZmHZyxUqZpQybsRtRZHKhtCLWK1p3LiZcvamx4wM91HbbCGH3RHS4jl2z2xSg0zzkUqsE2pLbULTW8zSNgBgZY0Lx5Bh8m1PafgVB79yMhkmX2B9x8XkD/xYusKbHle1gzB92Kw2YCU=; Received: from localhost.lerctr.org ([127.0.0.1]:14264 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.82_1-5b7a7c0-XX (FreeBSD)) (envelope-from ) id 1X98Un-000ONY-Ck; Mon, 21 Jul 2014 02:58:55 -0500 Received: from 107-128-180-255.lightspeed.austtx.sbcglobal.net ([107.128.180.255]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Mon, 21 Jul 2014 02:58:53 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Mon, 21 Jul 2014 02:58:53 -0500 From: Larry Rosenman To: Florian Smeets Subject: Re: [ZFS][PANIC] Solaris Assert/zio.c:2548 In-Reply-To: <53CCA40B.10503@smeets.im> References: <20140720140350.GA8498@borg.lerctr.org> <8D84B82C674B495DBF4951E0EE0D7117@multiplay.co.uk> <21dbecad27074fe34610bc587e6d0764@thebighonker.lerctr.org> <05c8bccf4faf5005881fbd2b22f35428@thebighonker.lerctr.org> <4A0B8A1798484B3DB4DCD924A2DFEF22@multiplay.co.uk> <53CCA40B.10503@smeets.im> Message-ID: <79dfedb5b7110aebecb35b08d2a5454d@thebighonker.lerctr.org> X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.0.1 X-Spam-Score: -2.9 (--) X-LERCTR-Spam-Score: -2.9 (--) X-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 X-LERCTR-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2014 07:59:00 -0000 On 2014-07-21 00:24, Florian Smeets wrote: > On 21/07/14 01:46, Steven Hartland wrote: >> ----- Original Message ----- From: "Larry Rosenman" >> To: "Steven Hartland" >> Cc: ; >> Sent: Monday, July 21, 2014 12:22 AM >> Subject: Re: [ZFS][PANIC] Solaris Assert/zio.c:2548 >> >> >>> On 2014-07-20 18:21, Steven Hartland wrote: >>>> Can you try reverting r265321 and see if you still see the >>>> same crash? >>>> >>>> Regards >>>> Steve >>> I'll do the revert, but it's been a ONE TIME hit. >>> >>> There was a followup to mine with a reproducible poudriere crash like >>> mine. >> >> If you don't have a reproducable senario I'd hold off. >> >> Florian, is yours reproducable and can you send me >> a pretty print of the crashing zio? >> > can you set print pretty on and then reprint the zio? That makes it "pretty" for Steve. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-fs@FreeBSD.ORG Mon Jul 21 08:00:11 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7CE5B550 for ; Mon, 21 Jul 2014 08:00:11 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5E17A22C5 for ; Mon, 21 Jul 2014 08:00:11 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s6L80Bx4006267 for ; Mon, 21 Jul 2014 08:00:11 GMT (envelope-from bugzilla-noreply@freebsd.org) Message-Id: <201407210800.s6L80Bx4006267@kenobi.freebsd.org> From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bugzilla] Commit Needs MFC MIME-Version: 1.0 X-Bugzilla-Type: whine X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated Date: Mon, 21 Jul 2014 08:00:11 +0000 Content-Type: text/plain X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2014 08:00:11 -0000 Hi, You have a bug in the "Needs MFC" state which has not been touched in 7 or more days. This email serves as a reminder that you may want to MFC this bug or marked it as completed. In the event you have a longer MFC timeout you may update this bug with a comment and I won't remind you again for 7 days. This reminder is only sent on Mondays. Please file a bug about concerns you may have. This search was scheduled by eadler@FreeBSD.org. (5 bugs) Bug 133174: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=133174 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-fs@FreeBSD.org Status: Needs MFC Resolution: Summary: [msdosfs] [patch] msdosfs must support multibyte international characters in file names Bug 136470: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=136470 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-fs@FreeBSD.org Status: Needs MFC Resolution: Summary: [nfs] Cannot mount / in read-only, over NFS Bug 139651: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=139651 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-fs@FreeBSD.org Status: Needs MFC Resolution: Summary: [nfs] mount(8): read-only remount of NFS volume does not work Bug 144447: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=144447 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-fs@FreeBSD.org Status: Needs MFC Resolution: Summary: [zfs] sharenfs fsunshare() & fsshare_main() non functional Bug 155411: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=155411 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-fs@FreeBSD.org Status: Needs MFC Resolution: Summary: [regression] [8.2-release] [tmpfs]: mount: tmpfs : No space left on device From owner-freebsd-fs@FreeBSD.ORG Tue Jul 22 09:01:32 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 82679B00; Tue, 22 Jul 2014 09:01:32 +0000 (UTC) Received: from anubis.delphij.net (anubis.delphij.net [64.62.153.212]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "anubis.delphij.net", Issuer "StartCom Class 1 Primary Intermediate Server CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 646FB2DB5; Tue, 22 Jul 2014 09:01:31 +0000 (UTC) Received: from delphij-macbook.local (c-24-5-244-32.hsd1.ca.comcast.net [24.5.244.32]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by anubis.delphij.net (Postfix) with ESMTPSA id EFC4F20C28; Tue, 22 Jul 2014 02:01:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphij.net; s=anubis; t=1406019685; bh=E6El0jzCY+2R5XIxs+O3ctese7mggqNYm/SwU3b6MdE=; h=Date:From:Reply-To:To:CC:Subject:References:In-Reply-To; b=aD9DOk/AejvqxjM8HXEqs2LJXmK1nBA90mE1vZAFPiCgrxc3lQRmliMf9u40sxsAk wk6vCyV+k1PXw0oi3PPuxG2lXXItfhM5A9nbQlnKHNiobzNS+3C9R9s7npTH0tbeIu gPC4Esz5gv1fjXj/8bj4ZT15+frsAu//b+LBZcHY= Message-ID: <53CE2863.6070804@delphij.net> Date: Tue, 22 Jul 2014 02:01:23 -0700 From: Xin Li Reply-To: d@delphij.net Organization: The FreeBSD Project MIME-Version: 1.0 To: Florian Smeets , Steven Hartland , Larry Rosenman Subject: Re: [ZFS][PANIC] Solaris Assert/zio.c:2548 References: <20140720140350.GA8498@borg.lerctr.org> <8D84B82C674B495DBF4951E0EE0D7117@multiplay.co.uk> <21dbecad27074fe34610bc587e6d0764@thebighonker.lerctr.org> <05c8bccf4faf5005881fbd2b22f35428@thebighonker.lerctr.org> <4A0B8A1798484B3DB4DCD924A2DFEF22@multiplay.co.uk> <53CCA40B.10503@smeets.im> In-Reply-To: <53CCA40B.10503@smeets.im> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jul 2014 09:01:32 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 I think this is my stupid. Feeling ashamed. Please try r268980+ and report back if it's fixed or not, thanks! Cheers, -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJTzihjAAoJEJW2GBstM+ns2esP/3zfORqtE11QeveWI8wBzHav Pl4A3V8kgi8FHP8m33gim1yAERpqf2+WgYuAjUNloFrFD9JCslnnoGtk9yiaOH2N Y7SJMYfvYkZ50upGE+fJKAcpH3QxLuEpSIlaFMkvA9oXtAiZEJ+BYBJh8VFWFXIs +bnqY0Mba7T574oqw8KcEysdYYDqSVd4/M/HBUdDTWT0/ZgeStJZw9MKPiSYFJcK PvV9dglPZ08L+Ra7cqu/EtC93p9IxVtfqxMlNkhMeVkQt+0jMKzHgDZ0dg9j7AVn SA9e+YvE9Sg2riE6XT2iF9B8Z4vDFf5/8CGtpBpKrB4861bcctdgJ3LIJBdDsz9N lOCjWXc4Amv8GVpTKPKaAGxeGPeN52Drqd2pD1ljLLENJsBZ3WOdMjt/R4VhpLVl 9LuXNFZ8oQy/PqVxHhYWYOOuqcFQnexB/5Qdmic3grIC/fwELbQwEK+34md5q4iJ pvy34RzzZrA1pVRfBM2oQ0TnABrdA7+bsb44DLvkPHmVGxPPZdN7QUfVzIrbCNe9 apBkLk/Qx2kOtpuAe6xBxhaOZEAJ9Q26Souew1HOYDSAusyCETB5+tfp1hl7R0ic xdL4YjSq9FPKOyYATCBh8/F0f2hVPC3mB2SmDz9PvI1+POWHae1tpeSCGonoc3A+ DCOLXRPNLbyZyrxTUtXG =NzT+ -----END PGP SIGNATURE----- From owner-freebsd-fs@FreeBSD.ORG Tue Jul 22 11:31:36 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 46223598 for ; Tue, 22 Jul 2014 11:31:36 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2E08B2A76 for ; Tue, 22 Jul 2014 11:31:36 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s6MBVaQ0010675 for ; Tue, 22 Jul 2014 11:31:36 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 191573] [zfs] kernel panic when running zpool/add/files.t Date: Tue, 22 Jul 2014 11:31:36 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: yaneurabeya@gmail.com X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jul 2014 11:31:36 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191573 --- Comment #14 from yaneurabeya@gmail.com --- The panic didn't occur on 9.3-RELEASE. Going to try an 10-STABLE/11-CURRENT snapshot.. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Tue Jul 22 16:25:37 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C867C2F1 for ; Tue, 22 Jul 2014 16:25:37 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B019628F8 for ; Tue, 22 Jul 2014 16:25:37 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s6MGPbFM016648 for ; Tue, 22 Jul 2014 16:25:37 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 191573] [zfs] kernel panic when running zpool/add/files.t Date: Tue, 22 Jul 2014 16:25:37 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: yaneurabeya@gmail.com X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jul 2014 16:25:37 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191573 --- Comment #15 from yaneurabeya@gmail.com --- Panic didn't occur with 10.0-STABLE @ r268571. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Tue Jul 22 17:17:14 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1F08682E; Tue, 22 Jul 2014 17:17:14 +0000 (UTC) Received: from mail-out.smeets.im (mail-out.smeets.im [IPv6:2a01:4f8:160:918a::25:11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C63872DF2; Tue, 22 Jul 2014 17:17:13 +0000 (UTC) Received: from mail.smeets.im (mail.smeets.im [IPv6:2a01:4f8:160:918a::25:3]) by mail-out.smeets.im (Postfix) with ESMTP id 794F52E3; Tue, 22 Jul 2014 19:17:11 +0200 (CEST) Received: from amavis.smeets.im (amavis.smeets.im [IPv6:2a01:4f8:160:918a::aa:4]) by mail.smeets.im (Postfix) with ESMTP id 94062892A6; Tue, 22 Jul 2014 19:17:11 +0200 (CEST) X-Virus-Scanned: amavisd-new at smeets.im Received: from mail.smeets.im ([IPv6:2a01:4f8:160:918a::25:3]) by amavis.smeets.im (amavis.smeets.im [IPv6:2a01:4f8:160:918a::aa:4]) (amavisd-new, port 10025) with ESMTP id XwP8E9IN_h3g; Tue, 22 Jul 2014 19:17:10 +0200 (CEST) Received: from nibbler-wlan.home.lan (unknown [IPv6:2001:4dd0:fd65:d00d:80b2:e702:7dad:6da2]) (using TLSv1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mail.smeets.im (Postfix) with ESMTPSA id 0718A8928B; Tue, 22 Jul 2014 19:17:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=smeets.im; s=default; t=1406049430; bh=qgI0D2vjYgwQmsJadUr4U1iyHCOTVABaj/k07hjjF1w=; h=Date:From:To:CC:Subject:References:In-Reply-To; b=4QEVdmU9w/zduoYw+4RPD231Q3YnBJ4DTUWEEXarjQMwnDmraDo80VGhBIUzKmlAR CjX7TOHzVwVqHBnngg2XfiGws3pyAGGzHX4L8gEvsoymShsbkb1KeGHyTu3FA/ejlZ OGK3Dm5L61yqhabaVBuKYQxxVgcz6QnpYRHEx7EI= Message-ID: <53CE9C8F.6040609@smeets.im> Date: Tue, 22 Jul 2014 19:17:03 +0200 From: Florian Smeets User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Larry Rosenman , freebsd-fs@freebsd.org, Steven Hartland Subject: Re: [ZFS][PANIC] Solaris Assert/zio.c:2548 References: <20140720140350.GA8498@borg.lerctr.org> In-Reply-To: <20140720140350.GA8498@borg.lerctr.org> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="3PTfihOS3p7SB7AkC9VroxOWxd5JlU3r4" Cc: freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jul 2014 17:17:14 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --3PTfihOS3p7SB7AkC9VroxOWxd5JlU3r4 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 20/07/14 16:03, Larry Rosenman wrote: >=20 > panic: solaris assert: !(zio->io_flags & ZIO_FLAG_DELEGATED), file: > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c, line: > 2874 >=20 This was fixed by r268980. Florian --3PTfihOS3p7SB7AkC9VroxOWxd5JlU3r4 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTzpyUAAoJEOcFPfn/hvB2A8IQAOOI3f9sATLFbtKQdsTnURNh 8INu8rnoOY+tTw7MaaGfLBBDxdDqV1hEbZapRo5KqGozbmCUNJSCCA+rlIXw2vNr bUl4ApIiKZTQaiHlMu0mILYjD9cajw7dTQcO457T2ItOUjU8xltROlHKX/QNHGkd 7b3aj0ZX8+aAfCfGkRIrvx9ffkppucAHnK5AqdhQaM+Nk/oZ+9fVIuUDuIF6WxoD wewS41TAIbU9NihXCDNT79jmL7tSh6fkN4Nn3lG+XWn8ijJvrq7isteVcopxctci sL0DSj+oZ5maWyoM8nxGyVJ7ioL8uQiw4D3pUfQnwTsJaUJCYFT4Uk0sHm9iwXOt 1gzxByr0dYwsXEIaGgNfi17jRAmp13mGS3YteA2jUM+pUwe0Rvj3YTNKPacNaOSd Lq9CBpQKBJDEhI3w1Yt/7jnxo/goXByfHqfvek3LX12AZdBpGpkOdtNjigjhJPrN 5miT55pNq9An8tb/NTJGvM9HKDFIF4h0oqGrzIfXBtFcaK/TLNu2RMro0k0IcK53 ApM0sBUjtvjd39NgHTugwBxk90IkuPt5TuaYM6jOkmSZ+ZYBq/a3PF5wF6W1rTeZ 1qa/2tcAvNbCmbp9DOCqiv1mY226MwFsklfjzdmd3mmnzT7ytxKlckI//+qbk6Bi CPtqT52WXX7sBz4EOs9L =cNYC -----END PGP SIGNATURE----- --3PTfihOS3p7SB7AkC9VroxOWxd5JlU3r4-- From owner-freebsd-fs@FreeBSD.ORG Tue Jul 22 18:27:03 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 85246CFD for ; Tue, 22 Jul 2014 18:27:03 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6BD262506 for ; Tue, 22 Jul 2014 18:27:03 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s6MIR3pC066583 for ; Tue, 22 Jul 2014 18:27:03 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 191573] [zfs] kernel panic when running zpool/add/files.t Date: Tue, 22 Jul 2014 18:27:03 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: yaneurabeya@gmail.com X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jul 2014 18:27:03 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=191573 --- Comment #16 from yaneurabeya@gmail.com --- Confirmed that it occurs on 11-CURRENT @ r268111 when run against a ZFS root only install on an amd64 VM with 2GB of RAM, without forcing 4kB sectors on the virtual drives, and with 512MB of swap. Procedure (from scratch): 1. Install ZFS on root with http://ftp.freebsd.org/pub/FreeBSD/snapshots/ISO-IMAGES/11.0/FreeBSD-11.0-CURRENT-amd64-20140714-r268622-disc1.iso.xz . 2. Reboot the install. 3. Login as root. 4. Either fix bug 191574, or check out my code forked FreeBSD tree from GitHub: i. pkg install -y git ii. cd /usr/src iii. git clone http://github.com/yaneurabeya/freebsd . iv. git checkout isilon-atf 5. Run the `prove -rv tools/regression/zfs` as root. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Tue Jul 22 21:15:04 2014 Return-Path: Delivered-To: fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AE0D29F3 for ; Tue, 22 Jul 2014 21:15:04 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 94DEE2755 for ; Tue, 22 Jul 2014 21:15:04 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.9/8.14.9) with ESMTP id s6MLF4qA024045 for ; Tue, 22 Jul 2014 21:15:04 GMT (envelope-from bdrewery@freefall.freebsd.org) Received: (from bdrewery@localhost) by freefall.freebsd.org (8.14.9/8.14.9/Submit) id s6MLF4Rl024044 for fs@FreeBSD.org; Tue, 22 Jul 2014 21:15:04 GMT (envelope-from bdrewery) Received: (qmail 37911 invoked from network); 22 Jul 2014 16:15:02 -0500 Received: from unknown (HELO blah) (freebsd@shatow.net@67.182.131.225) by sweb.xzibition.com with ESMTPA; 22 Jul 2014 16:15:02 -0500 Message-ID: <53CED455.1030701@FreeBSD.org> Date: Tue, 22 Jul 2014 14:15:01 -0700 From: Bryan Drewery Organization: FreeBSD User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: fs@FreeBSD.org Subject: NFS DEADLKRES panic with getfsstat() on unresponsive mount Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jul 2014 21:15:04 -0000 I had an NFS server go offline for about an hour and had a client panic in deadlkres after a 'zfs list -o mounted' ran which called getfsstat(2). It seems reasonable that this could be a false-positive. Is deadlkres known to have acceptable false-positives? If so, is there a way to mark this one to not cause a panic? This was r268621. Some debugging is below from me figuring out it was not ZFS, but my ruby zfs snapshot script causing the issue. > panic: deadlkres: possible deadlock detected for 0xfffff8029ff37490, blocked for 1800185 ticks > (kgdb) p *(struct thread*)0xfffff8029ff37490 > $1 = {td_lock = 0xffffffff815fc7d0, td_proc = 0xfffff80f93f29000, td_plist = {tqe_next = 0x0, tqe_prev = 0xfffff80f93f29010}, td_runq = {tqe_next = 0x0, tqe_prev = 0xffffffff815d23a0}, > td_slpq = {tqe_next = 0xfffff8039d61a000, tqe_prev = 0xfffff8015e99dc80}, td_lockq = {tqe_next = 0x0, tqe_prev = 0xfffffe1247e7c6e8}, td_hash = {le_next = 0x0, > le_prev = 0xfffffe00008d8ae0}, td_cpuset = 0xfffff8000e3813a8, td_sel = 0xfffff8000e6c2580, td_sleepqueue = 0x0, td_turnstile = 0xfffff800431fccc0, td_rlqe = 0xfffff80038b68398, > td_umtxq = 0xfffff8000e718880, td_tid = 101724, td_sigqueue = {sq_signals = {__bits = {0, 0, 0, 0}}, sq_kill = {__bits = {0, 0, 0, 0}}, sq_list = {tqh_first = 0x0, > tqh_last = 0xfffff8029ff37548}, sq_proc = 0xfffff80f93f29000, sq_flags = 1}, td_lend_user_pri = 255 '�', td_flags = 8196, td_inhibitors = 2, td_pflags = 0, td_dupfd = 0, > td_sqqueue = 0, td_wchan = 0xfffff800aa101d50, td_wmesg = 0xffffffff80f8e15e "newnfs", td_lastcpu = 14 '\016', td_oncpu = 255 '�', td_owepreempt = 0 '\0', td_tsqueue = 0 '\0', > td_locks = 0, td_rw_rlocks = 0, td_lk_slocks = 0, td_stopsched = 0, td_blocked = 0x0, td_lockname = 0x0, td_contested = {lh_first = 0x0}, td_sleeplocks = 0xffffffff8187b9e8, > td_intr_nesting_level = 0, td_pinned = 0, td_ucred = 0xfffff8004306e200, td_estcpu = 0, td_slptick = 171026084, td_blktick = 0, td_swvoltick = 171026084, td_cow = 69, td_ru = { > ru_utime = {tv_sec = 0, tv_usec = 0}, ru_stime = {tv_sec = 0, tv_usec = 0}, ru_maxrss = 3936, ru_ixrss = 21896, ru_idrss = 1904, ru_isrss = 30464, ru_minflt = 339, ru_majflt = 0, > ru_nswap = 0, ru_inblock = 25393, ru_oublock = 0, ru_msgsnd = 0, ru_msgrcv = 0, ru_nsignals = 0, ru_nvcsw = 3, ru_nivcsw = 23}, td_rux = {rux_runtime = 4239501908, rux_uticks = 7, > rux_sticks = 231, rux_iticks = 0, rux_uu = 0, rux_su = 0, rux_tu = 0}, td_incruntime = 0, td_runtime = 4239501908, td_pticks = 0, td_sticks = 0, td_iticks = 0, td_uticks = 0, > td_intrval = 0, td_oldsigmask = {__bits = {0, 0, 0, 0}}, td_generation = 26, td_sigstk = {ss_sp = 0x0, ss_size = 0, ss_flags = 4}, td_xsig = 0, td_profil_addr = 0, td_profil_ticks = 0, > td_name = "zfs", '\0' , td_fpop = 0x0, td_dbgflags = 0, td_dbgksi = {ksi_link = {tqe_next = 0x0, tqe_prev = 0x0}, ksi_info = {si_signo = 0, si_errno = 0, si_code = 0, > si_pid = 0, si_uid = 0, si_status = 0, si_addr = 0x0, si_value = {sival_int = 0, sival_ptr = 0x0, sigval_int = 0, sigval_ptr = 0x0}, _reason = {_fault = {_trapno = 0}, _timer = { > _timerid = 0, _overrun = 0}, _mesgq = {_mqd = 0}, _poll = {_band = 0}, __spare__ = {__spare1__ = 0, __spare2__ = {0, 0, 0, 0, 0, 0, 0}}}}, ksi_flags = 0, ksi_sigq = 0x0}, > td_ng_outbound = 0, td_osd = {osd_nslots = 2, osd_slots = 0xfffff80b2cd28070, osd_next = {le_next = 0xfffff80038623c40, le_prev = 0xfffff8039d61a330}}, td_map_def_user = 0x0, > td_dbg_forked = 0, td_vp_reserv = 0, td_no_sleeping = 0, td_dom_rr_idx = 0, td_sigmask = {__bits = {0, 0, 0, 0}}, td_rqindex = 7 '\a', td_base_pri = 152 '\230', > td_priority = 152 '\230', td_pri_class = 3 '\003', td_user_pri = 177 '�', td_base_user_pri = 177 '�', td_pcb = 0xfffffe12481fdcc0, td_state = TDS_INHIBITED, td_uretoff = {tdu_retval = { > 0, 1}, tdu_off = 0}, td_slpcallout = {c_links = {le = {le_next = 0x0, le_prev = 0xfffffe0000a156a0}, sle = {sle_next = 0x0}, tqe = {tqe_next = 0x0, tqe_prev = 0xfffffe0000a156a0}}, > c_time = 733781545914723, c_precision = 134217727, c_arg = 0xfffff8029ff37490, c_func = 0xffffffff809589b0 , c_lock = 0x0, c_flags = 272, c_cpu = 4}, > td_frame = 0xfffffe12481fdc00, td_kstack_obj = 0xfffff80f7ec07600, td_kstack = 18446741953205739520, td_kstack_pages = 4, td_critnest = 1, td_md = {md_spinlock_count = 1, > md_saved_flags = 582, md_spurflt_addr = 0}, td_sched = 0xfffff8029ff378f8, td_ar = 0x0, td_lprof = {{lh_first = 0x0}, {lh_first = 0x0}}, td_dtrace = 0xfffff8001bb5e000, td_errno = 0, > td_vnet = 0x0, td_vnet_lpush = 0x0, td_intr_frame = 0x0, td_rfppwait_p = 0xfffff800aab78000, td_ma = 0x0, td_ma_cnt = 0} > Current language: auto; currently minimal > (kgdb) p $1.td_tid > $2 = 101724 > (kgdb) p $1.td_name > $3 = "zfs", '\0' > (kgdb) thread 611 > [Switching to thread 611 (Thread 101724)]#0 sched_switch (td=0xfffff8029ff37490, newtd=, flags=) at /usr/src/sys/kern/sched_ule.c:1931 > 1931 cpuid = PCPU_GET(cpuid); > (kgdb) bt > #0 sched_switch (td=0xfffff8029ff37490, newtd=, flags=) at /usr/src/sys/kern/sched_ule.c:1931 > #1 0xffffffff8091ba89 in mi_switch (flags=260, newtd=0x0) at /usr/src/sys/kern/kern_synch.c:493 > #2 0xffffffff80958ea2 in sleepq_switch (wchan=, pri=) at /usr/src/sys/kern/subr_sleepqueue.c:552 > #3 0xffffffff80958d03 in sleepq_wait (wchan=0xfffff800aa101d50, pri=96) at /usr/src/sys/kern/subr_sleepqueue.c:631 > #4 0xffffffff808f092a in sleeplk (lk=, flags=, ilk=, wmesg=, pri=, > timo=) at /usr/src/sys/kern/kern_lock.c:225 > #5 0xffffffff808efff1 in __lockmgr_args (lk=0xfffff800aa101d50, flags=, ilk=0xfffff800aa101d80, wmesg=, pri=, > timo=, file=0xffffffff80fb8028 "/usr/src/sys/kern/vfs_subr.c", line=2135) at /usr/src/sys/kern/kern_lock.c:931 > #6 0xffffffff80832b02 in nfs_lock1 (ap=) at lockmgr.h:97 > #7 0xffffffff80e4887c in VOP_LOCK1_APV (vop=, a=) at vnode_if.c:2082 > #8 0xffffffff809cf57a in _vn_lock (vp=0xfffff800aa101ce8, flags=, file=0xffffffff80fb8028 "/usr/src/sys/kern/vfs_subr.c", line=2135) at vnode_if.h:859 > #9 0xffffffff809bf977 in vget (vp=0xfffff800aa101ce8, flags=524544, td=0xfffff8029ff37490) at /usr/src/sys/kern/vfs_subr.c:2135 > #10 0xffffffff809b3661 in vfs_hash_get (mp=0xfffff80038d3b000, hash=245677930, flags=, td=0xfffff8029ff37490, vpp=0xfffffe12481fd670, > fn=0xffffffff8083a290 ) at /usr/src/sys/kern/vfs_hash.c:88 > #11 0xffffffff80835f2d in ncl_nget (mntp=0xfffff80038d3b000, fhp=0xfffff800aa1cf0a4 "\233�\tM006\236n", fhsize=28, npp=0xfffffe12481fd6e0, lkflags=) > at /usr/src/sys/fs/nfsclient/nfs_clnode.c:114 > #12 0xffffffff80838b6d in nfs_statfs (mp=0xfffff80038d3b000, sbp=0xfffff80038d3b0b8) at /usr/src/sys/fs/nfsclient/nfs_clvfsops.c:288 > #13 0xffffffff809b9e5e in __vfs_statfs (mp=0x0, sbp=0xfffff80038d3b0b8) at /usr/src/sys/kern/vfs_mount.c:1724 > #14 0xffffffff809c72be in kern_getfsstat (td=0xfffff8029ff37490, buf=, bufsize=, bufseg=UIO_USERSPACE, flags=) > at /usr/src/sys/kern/vfs_syscalls.c:511 > #15 0xffffffff80d280fa in amd64_syscall (td=0xfffff8029ff37490, traced=0) at subr_syscall.c:133 > #16 0xffffffff80d0a64b in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:407 > #17 0x000000080254955a in ?? () > (kgdb) down > #13 0xffffffff809b9e5e in __vfs_statfs (mp=0x0, sbp=0xfffff80038d3b0b8) at /usr/src/sys/kern/vfs_mount.c:1724 > 1724 error = mp->mnt_op->vfs_statfs(mp, &mp->mnt_stat); > (kgdb) p *sbp > $5 = {f_version = 537068824, f_type = 58, f_flags = 268435472, f_bsize = 512, f_iosize = 65536, f_blocks = 3365960256, f_bfree = 3281287109, f_bavail = 3281287109, f_files = 3281308671, > f_ffree = 3281287109, f_syncwrites = 0, f_asyncwrites = 0, f_syncreads = 0, f_asyncreads = 0, f_spare = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, f_namemax = 255, f_owner = 0, f_fsid = {val = { > 973143809, 58}}, f_charspare = '\0' , f_fstypename = "nfs", '\0' , > f_mntfromname = "tank:/tank/distfiles/freebsd\000\000\000\000/tank/distfiles/freebsd\000\000\000\000\000\000\000\000\000zroot/cache/ccache\000�", > f_mntonname = "/mnt/distfiles", '\0' } > (kgdb) p $1.td_proc.p_pptr.p_comm > $17 = "ruby", '\0' The NFS mountpoint /mnt/distfiles is on top of a ZFS /mnt. I also am sharing / from the client, but nothing is connecting to it. > # cat /etc/exports > V4: / -network 10.10.0.0/16 This zfs(1) call was initiated by my ruby snapshot script which runs 'zfs list -o mounted' which called the getfsstat(2). -- Regards, Bryan Drewery From owner-freebsd-fs@FreeBSD.ORG Wed Jul 23 14:16:23 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 176119A7 for ; Wed, 23 Jul 2014 14:16:23 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E229D2031 for ; Wed, 23 Jul 2014 14:16:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Content-Type:MIME-Version:Message-ID:Subject:To:From:Date; bh=PYjUqG8D623UT3cy2cjda0IffncUbhDIRUVMJnIlByA=; b=YLrbOAtUGXzawcMEvIUaUTZIDuzUCIoP5J1elu+R/I1x7vEy8hZyKbQUFwF4kXS3GFCatP0X/2vkHypoXjS/ImxDCZ0NPu1rxi0Cdex2DbYy0Fg01v4ySf95Pr8yvg2BQ9qIFXkAU+sQTcn36ttn4n86NuAYUGcEpK+W9Iy4+Kw=; Received: from 107-128-180-255.lightspeed.austtx.sbcglobal.net ([107.128.180.255]:28219 helo=borg.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.83 (FreeBSD)) (envelope-from ) id 1X9xLA-000BWo-Gc for freebsd-fs@freebsd.org; Wed, 23 Jul 2014 09:16:21 -0500 Date: Wed, 23 Jul 2014 09:16:09 -0500 From: Larry Rosenman To: freebsd-fs@freebsd.org Subject: 2 asserts that I've had issues with.... Message-ID: <20140723141609.GA71261@borg.lerctr.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Score: -2.9 (--) X-LERCTR-Spam-Score: -2.9 (--) X-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, TVD_RCVD_IP=0.001 X-LERCTR-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, TVD_RCVD_IP=0.001 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jul 2014 14:16:23 -0000 I've had the following 2 asserts #if'd out for a while. I was seeing lots of panics from them.... 2 questions: 1) is it a real issue? 2) how can I find the offending block(s) on disk? Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c (revision 269010) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c (working copy) @@ -476,7 +476,9 @@ * chain. There should be no chained leafs (as we have removed * support for them). */ +#if 0 /*LER: to see what else blows up */ ASSERT0(l->l_phys->l_hdr.lh_pad1); +#endif /* * There should be more hash entries than there can be @@ -531,7 +533,9 @@ ASSERT3U(l->l_blkid, ==, blkid); ASSERT3P(l->l_dbuf, ==, db); ASSERT3P(l->l_phys, ==, l->l_dbuf->db_data); +#if 0 /* LER */ ASSERT3U(l->l_phys->l_hdr.lh_block_type, ==, ZBT_LEAF); +#endif ASSERT3U(l->l_phys->l_hdr.lh_magic, ==, ZAP_LEAF_MAGIC); *lp = l; borg.lerctr.org /usr/src # Thanks. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-fs@FreeBSD.ORG Thu Jul 24 07:46:56 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E9D5DAA3 for ; Thu, 24 Jul 2014 07:46:56 +0000 (UTC) Received: from venus.codepro.be (venus.codepro.be [IPv6:2a01:4f8:162:1127::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.codepro.be", Issuer "Gandi Standard SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AE1572FC3 for ; Thu, 24 Jul 2014 07:46:56 +0000 (UTC) Received: from vega.codepro.be (unknown [172.16.1.3]) by venus.codepro.be (Postfix) with ESMTP id EEE878AFE for ; Thu, 24 Jul 2014 09:46:53 +0200 (CEST) Received: by vega.codepro.be (Postfix, from userid 1001) id E985C7891; Thu, 24 Jul 2014 09:46:53 +0200 (CEST) Date: Thu, 24 Jul 2014 09:46:53 +0200 From: Kristof Provost To: freebsd-fs@freebsd.org Subject: Re: ZFS panic on zvol resize Message-ID: <20140724074653.GQ2406@vega.codepro.be> References: <20140704194750.GU75721@vega.codepro.be> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20140704194750.GU75721@vega.codepro.be> X-PGP-Fingerprint: E114 D9EA 909E D469 8F57 17A5 7D15 91C6 9EFA F286 X-Checked-By-NSA: Probably User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jul 2014 07:46:57 -0000 Filed as #192085. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192085 Regards, Kristof On 2014-07-04 21:47:50 (+0200), Kristof Provost wrote: > Hi, > > On current r268263 (and also on stable-10 r268263) I can reliably panic > the machine by simply attempting to resize a zvol: > > # zfs create tank/zvol > # zfs set mountpoint=none tank/zvol > # zfs create -V100G tank/zvol/disk0 > # zfs set volsize=200G tank/zvol/disk0 > > It produces the following panic: > > panic: solaris assert: !rrw_held(&dp->dp_config_rwlock, RW_READER), file: /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_pool.c, line: 1120 > cpuid = 1 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe01217d54b0 > kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe01217d5560 > vpanic() at vpanic+0x126/frame 0xfffffe01217d55a0 > panic() at panic+0x43/frame 0xfffffe01217d5600 > assfail() at assfail+0x1d/frame 0xfffffe01217d5610 > dsl_pool_hold() at dsl_pool_hold+0x67/frame 0xfffffe01217d5650 > dmu_objset_hold() at dmu_objset_hold+0x21/frame 0xfffffe01217d5690 > dsl_prop_get_integer() at dsl_prop_get_integer+0x28/frame 0xfffffe01217d56d0 > zvol_set_volsize() at zvol_set_volsize+0x126/frame 0xfffffe01217d5760 > zfs_prop_set_special() at zfs_prop_set_special+0x2e2/frame 0xfffffe01217d57f0 > zfs_set_prop_nvlist() at zfs_set_prop_nvlist+0x23f/frame 0xfffffe01217d5880 > zfs_ioc_set_prop() at zfs_ioc_set_prop+0x106/frame 0xfffffe01217d58e0 > zfsdev_ioctl() at zfsdev_ioctl+0x6ee/frame 0xfffffe01217d5990 > devfs_ioctl_f() at devfs_ioctl_f+0xfb/frame 0xfffffe01217d59f0 > kern_ioctl() at kern_ioctl+0x22b/frame 0xfffffe01217d5a50 > sys_ioctl() at sys_ioctl+0x13c/frame 0xfffffe01217d5aa0 > amd64_syscall() at amd64_syscall+0x25a/frame 0xfffffe01217d5bb0 > Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe01217d5bb0 > --- syscall (54, FreeBSD ELF64, sys_ioctl), rip = 0x8019e89ba, rsp = 0x7fffffffb8c8, rbp = 0x7fffffffb940 --- > Uptime: 2m18s > Automatic reboot in 15 seconds - press a key on the console to abort > > Please let me know if there's any other information which could be > helpful, or any patch I could test. > > Regards, > Kristof > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@FreeBSD.ORG Thu Jul 24 13:36:54 2014 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 16A8485B; Thu, 24 Jul 2014 13:36:54 +0000 (UTC) Received: from mail-oa0-x230.google.com (mail-oa0-x230.google.com [IPv6:2607:f8b0:4003:c02::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CAA8220D5; Thu, 24 Jul 2014 13:36:53 +0000 (UTC) Received: by mail-oa0-f48.google.com with SMTP id m1so3632116oag.35 for ; Thu, 24 Jul 2014 06:36:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=QTzRWoJnXUlecfr3oVOKQeLBkad2K287uggRdvysHr4=; b=E2Zr98cQG+ZXfLamkM0Tiv80CASkiYMGXjNtD9UxvoM/hd9IvuCYqWAhCyR9zqk6Db NB4+FVQwry7KfSjK6CeLxusz0AGjTzkVcSjATcYCLM5zT3biWpFN9+B+D00T2h3tTmd9 B5X1tZDATw+xUUA5P8KGTMzUWFUZKzNrzKywr85Y0taxK1mXS5TOt/MEd5xm6Tnp6FR4 1/KAGueCajJACQPEU+4eEj80bTXL0ink0yQIKHyfcyDBe/kXFOqSrsalZ1436Z1YWDiB ZbZMhfbw8GRFsQQLnv1Tn5weoIq78HdKB+NMvCKWwJQGgxYK33gAs9SQrD6Zwu86uLEC tHdg== MIME-Version: 1.0 X-Received: by 10.60.132.176 with SMTP id ov16mr13096051oeb.13.1406209012781; Thu, 24 Jul 2014 06:36:52 -0700 (PDT) Received: by 10.76.213.7 with HTTP; Thu, 24 Jul 2014 06:36:52 -0700 (PDT) In-Reply-To: <53CED455.1030701@FreeBSD.org> References: <53CED455.1030701@FreeBSD.org> Date: Thu, 24 Jul 2014 09:36:52 -0400 Message-ID: Subject: Re: NFS DEADLKRES panic with getfsstat() on unresponsive mount From: Ryan Stone To: Bryan Drewery Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-fs@freebsd.org" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jul 2014 13:36:54 -0000 On Tue, Jul 22, 2014 at 5:15 PM, Bryan Drewery wrote: > I had an NFS server go offline for about an hour and had a client panic in > deadlkres after a 'zfs list -o mounted' ran which called getfsstat(2). It > seems reasonable that this could be a false-positive. Is deadlkres known to > have acceptable false-positives? If so, is there a way to mark this one to > not cause a panic? > > This was r268621. There is a list of "blessed" sx lock names in sys/kern/kern_clock.c. deadlkres will ignore any threads sleeping with one of those wmesgs. From owner-freebsd-fs@FreeBSD.ORG Thu Jul 24 17:33:53 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AF864E37; Thu, 24 Jul 2014 17:33:53 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 821882863; Thu, 24 Jul 2014 17:33:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:Subject:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=eJHyhypCjIR+hcRFcjlZEwwDMnZvYqDxx3KsjBXNAVI=; b=P7++1gyOyJQCEYCQ0sNqHLCl8pWsHp8XMse94h2cnrHYRTByIkFPO4hAMCm3zMDHjKMTyQ3absP5C3wIv4wlCf/n8tuJMMu9weeVAtc7H7WZvWGGmo3N86MYyZo+gLabAa3qzwC2FNE07XbSsmWxwEnYi39OuV4Dvgx537DEaZk=; Received: from localhost.lerctr.org ([127.0.0.1]:63418 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.83 (FreeBSD)) (envelope-from ) id 1XAMtr-000GSA-2f; Thu, 24 Jul 2014 12:33:52 -0500 Received: from host.alcatel.com ([198.205.55.139]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Thu, 24 Jul 2014 12:33:50 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 24 Jul 2014 12:33:50 -0500 From: Larry Rosenman To: freebsd-fs@freebsd.org, Freebsd current Subject: zfs send/recv: STILL invalid Backup Stream X-Priority: 1 (Highest) Message-ID: <62315eb454a95db636b7764aad3c0f9b@thebighonker.lerctr.org> X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.0.1 X-Spam-Score: -2.9 (--) X-LERCTR-Spam-Score: -2.9 (--) X-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 X-LERCTR-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jul 2014 17:33:53 -0000 TRYING to use zfs send/recv between a 10-STABLE and an 11-CURRENT system, and receive the non-descript "invalid backup stream". borg.lerctr.org /home/ler $ sudo bin/backup-TBH-ZFS-initial.sh Password: receiving full stream of zroot@2014-07-24 into zroot/backups/TBH@2014-07-24 received 41.7KB stream in 300 seconds (142B/sec) receiving full stream of zroot/usr@2014-07-24 into zroot/backups/TBH/usr@2014-07-24 received 41.7KB stream in 1 seconds (41.7KB/sec) receiving full stream of zroot/usr/local@2014-07-24 into zroot/backups/TBH/usr/local@2014-07-24 received 2.81GB stream in 1116 seconds (2.58MB/sec) receiving full stream of zroot/usr/src@2014-07-24 into zroot/backups/TBH/usr/src@2014-07-24 cannot receive new filesystem stream: invalid backup stream borg.lerctr.org /home/ler $ cat bin/backup-TBH-ZFS-initial.sh #!/bin/sh DATE=`date "+%Y-%m-%d"` #DATE2=2013-03-24 #DATE2=`date -v "-1d" "+%Y-%m-%d"` # snap the source ssh root@tbh.lerctr.org zfs snapshot -r zroot@${DATE} # zfs copy the source to here. ssh root@tbh.lerctr.org "zfs send -R -D zroot@${DATE} | \ ssh home.lerctr.org \"zfs recv -F -u -v -d zroot/backups/TBH\"" # make sure we NEVER allow the backup stuff to automount. /sbin/zfs list -H -t filesystem -r zroot/backups/TBH| \ awk '{printf "/sbin/zfs set canmount=noauto %s\n",$1}' | sh borg.lerctr.org /home/ler $ This has been happening for YEARS and I can't seem to interest anyone in fixing it. How can we get to the bottom of this? borg.lerctr.org /home/ler $ uname -a FreeBSD borg.lerctr.org 11.0-CURRENT FreeBSD 11.0-CURRENT #56 r268982M: Tue Jul 22 10:14:59 CDT 2014 root@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER amd64 borg.lerctr.org /home/ler $ ssh tbh uname -a FreeBSD thebighonker.lerctr.org 10.0-STABLE FreeBSD 10.0-STABLE #39 r269019M: Wed Jul 23 11:44:35 CDT 2014 root@thebighonker.lerctr.org:/usr/obj/usr/src/sys/GENERIC amd64 borg.lerctr.org /home/ler $ -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-fs@FreeBSD.ORG Thu Jul 24 17:43:04 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 815B73E8; Thu, 24 Jul 2014 17:43:04 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 536412955; Thu, 24 Jul 2014 17:43:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=QTuVNbiiO/NZzU8A8SZ4Jw6ZFexL6HmGNnGxdVRqyIc=; b=VOFdOOLCvg2xrRnAqTT22lQcHpogb/aV5tohOFbDFDfCMC5VYy/k9Re6wJ7QFcbpa8Tci44/oS+P6kC8Nawap0b7hjeTtV17LmDmaNxvhVmA4BMKHHq2tzx935boAYcgZSGZBz/T7J009XlNvcFIGaiXF30VtN0rktConbx4YnA=; Received: from localhost.lerctr.org ([127.0.0.1]:26447 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.83 (FreeBSD)) (envelope-from ) id 1XAN2j-000GcH-AY; Thu, 24 Jul 2014 12:43:03 -0500 Received: from host.alcatel.com ([198.205.55.139]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Thu, 24 Jul 2014 12:43:01 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 24 Jul 2014 12:43:01 -0500 From: Larry Rosenman To: Allan Jude Subject: Re: zfs send/recv: STILL invalid Backup Stream In-Reply-To: <53D1448C.40908@freebsd.org> References: <62315eb454a95db636b7764aad3c0f9b@thebighonker.lerctr.org> <53D1448C.40908@freebsd.org> Message-ID: X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.0.1 X-Spam-Score: -2.9 (--) X-LERCTR-Spam-Score: -2.9 (--) X-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 X-LERCTR-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 Cc: Freebsd fs , freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jul 2014 17:43:04 -0000 On 2014-07-24 12:38, Allan Jude wrote: > On 2014-07-24 13:33, Larry Rosenman wrote: >> TRYING to use zfs send/recv between a 10-STABLE and an 11-CURRENT >> system, and receive the non-descript >> "invalid backup stream". >> >> borg.lerctr.org /home/ler $ sudo bin/backup-TBH-ZFS-initial.sh >> Password: >> receiving full stream of zroot@2014-07-24 into >> zroot/backups/TBH@2014-07-24 >> received 41.7KB stream in 300 seconds (142B/sec) >> receiving full stream of zroot/usr@2014-07-24 into >> zroot/backups/TBH/usr@2014-07-24 >> received 41.7KB stream in 1 seconds (41.7KB/sec) >> receiving full stream of zroot/usr/local@2014-07-24 into >> zroot/backups/TBH/usr/local@2014-07-24 >> received 2.81GB stream in 1116 seconds (2.58MB/sec) >> receiving full stream of zroot/usr/src@2014-07-24 into >> zroot/backups/TBH/usr/src@2014-07-24 >> cannot receive new filesystem stream: invalid backup stream >> borg.lerctr.org /home/ler $ cat bin/backup-TBH-ZFS-initial.sh >> #!/bin/sh >> DATE=`date "+%Y-%m-%d"` >> #DATE2=2013-03-24 >> #DATE2=`date -v "-1d" "+%Y-%m-%d"` >> # snap the source >> ssh root@tbh.lerctr.org zfs snapshot -r zroot@${DATE} >> # zfs copy the source to here. >> ssh root@tbh.lerctr.org "zfs send -R -D zroot@${DATE} | \ >> ssh home.lerctr.org \"zfs recv -F -u -v -d zroot/backups/TBH\"" >> # make sure we NEVER allow the backup stuff to automount. >> /sbin/zfs list -H -t filesystem -r zroot/backups/TBH| \ >> awk '{printf "/sbin/zfs set canmount=noauto %s\n",$1}' | sh >> borg.lerctr.org /home/ler $ >> >> This has been happening for YEARS and I can't seem to interest anyone >> in >> fixing it. >> >> How can we get to the bottom of this? >> >> borg.lerctr.org /home/ler $ uname -a >> FreeBSD borg.lerctr.org 11.0-CURRENT FreeBSD 11.0-CURRENT #56 >> r268982M: >> Tue Jul 22 10:14:59 CDT 2014 >> root@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER amd64 >> borg.lerctr.org /home/ler $ ssh tbh uname -a >> FreeBSD thebighonker.lerctr.org 10.0-STABLE FreeBSD 10.0-STABLE #39 >> r269019M: Wed Jul 23 11:44:35 CDT 2014 >> root@thebighonker.lerctr.org:/usr/obj/usr/src/sys/GENERIC amd64 >> borg.lerctr.org /home/ler $ >> > > Try adding -v to the 'zfs send' and see if it gives you more detail. > > Can you also try this script for the replication: > > http://github.com/allanjude/zxfer I've done that in the past and nothing, but I will try again. I will also look at zxfer.... :) -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-fs@FreeBSD.ORG Thu Jul 24 18:25:21 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D58C916C; Thu, 24 Jul 2014 18:25:21 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A4D6E2D23; Thu, 24 Jul 2014 18:25:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=iyZmSjZ19dk3Zw9M1BwVpgSoQMLU1DbqhGW666oWCc0=; b=CcrSxzmn7nCFZtWGuF7dHvluzt3RRlyjbibg6gdBL6J9j1q5E1NRq3qKKPu7SeTyT4RAynAgONX+9A6rgTGZ9Ec9oqLt5dJ+ZazpJgXf7MxqcvmnICmCecONmR5KHiGOKziAHAddT/xxrxhDl7j21EQjoDWSlF4crKifMNJieiE=; Received: from localhost.lerctr.org ([127.0.0.1]:47248 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.83 (FreeBSD)) (envelope-from ) id 1XANhe-000HN6-PM; Thu, 24 Jul 2014 13:25:20 -0500 Received: from host.alcatel.com ([198.205.55.139]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Thu, 24 Jul 2014 13:25:18 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 24 Jul 2014 13:25:18 -0500 From: Larry Rosenman To: Allan Jude Subject: Re: zfs send/recv: STILL invalid Backup Stream In-Reply-To: References: <62315eb454a95db636b7764aad3c0f9b@thebighonker.lerctr.org> <53D1448C.40908@freebsd.org> Message-ID: X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.0.1 X-Spam-Score: -2.9 (--) X-LERCTR-Spam-Score: -2.9 (--) X-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 X-LERCTR-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 Cc: Freebsd fs , freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jul 2014 18:25:21 -0000 On 2014-07-24 12:43, Larry Rosenman wrote: > On 2014-07-24 12:38, Allan Jude wrote: >> On 2014-07-24 13:33, Larry Rosenman wrote: >>> TRYING to use zfs send/recv between a 10-STABLE and an 11-CURRENT >>> system, and receive the non-descript >>> "invalid backup stream". >>> >>> borg.lerctr.org /home/ler $ sudo bin/backup-TBH-ZFS-initial.sh >>> Password: >>> receiving full stream of zroot@2014-07-24 into >>> zroot/backups/TBH@2014-07-24 >>> received 41.7KB stream in 300 seconds (142B/sec) >>> receiving full stream of zroot/usr@2014-07-24 into >>> zroot/backups/TBH/usr@2014-07-24 >>> received 41.7KB stream in 1 seconds (41.7KB/sec) >>> receiving full stream of zroot/usr/local@2014-07-24 into >>> zroot/backups/TBH/usr/local@2014-07-24 >>> received 2.81GB stream in 1116 seconds (2.58MB/sec) >>> receiving full stream of zroot/usr/src@2014-07-24 into >>> zroot/backups/TBH/usr/src@2014-07-24 >>> cannot receive new filesystem stream: invalid backup stream >>> borg.lerctr.org /home/ler $ cat bin/backup-TBH-ZFS-initial.sh >>> #!/bin/sh >>> DATE=`date "+%Y-%m-%d"` >>> #DATE2=2013-03-24 >>> #DATE2=`date -v "-1d" "+%Y-%m-%d"` >>> # snap the source >>> ssh root@tbh.lerctr.org zfs snapshot -r zroot@${DATE} >>> # zfs copy the source to here. >>> ssh root@tbh.lerctr.org "zfs send -R -D zroot@${DATE} | \ >>> ssh home.lerctr.org \"zfs recv -F -u -v -d zroot/backups/TBH\"" >>> # make sure we NEVER allow the backup stuff to automount. >>> /sbin/zfs list -H -t filesystem -r zroot/backups/TBH| \ >>> awk '{printf "/sbin/zfs set canmount=noauto %s\n",$1}' | sh >>> borg.lerctr.org /home/ler $ >>> >>> This has been happening for YEARS and I can't seem to interest anyone >>> in >>> fixing it. >>> >>> How can we get to the bottom of this? >>> >>> borg.lerctr.org /home/ler $ uname -a >>> FreeBSD borg.lerctr.org 11.0-CURRENT FreeBSD 11.0-CURRENT #56 >>> r268982M: >>> Tue Jul 22 10:14:59 CDT 2014 >>> root@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER amd64 >>> borg.lerctr.org /home/ler $ ssh tbh uname -a >>> FreeBSD thebighonker.lerctr.org 10.0-STABLE FreeBSD 10.0-STABLE #39 >>> r269019M: Wed Jul 23 11:44:35 CDT 2014 >>> root@thebighonker.lerctr.org:/usr/obj/usr/src/sys/GENERIC amd64 >>> borg.lerctr.org /home/ler $ >>> >> >> Try adding -v to the 'zfs send' and see if it gives you more detail. >> >> Can you also try this script for the replication: >> >> http://github.com/allanjude/zxfer > I've done that in the past and nothing, but I will try again. > > I will also look at zxfer.... :) with the -v, no more info, just what looks like normal messages.... 13:23:55 3.68G zroot/usr/src@2014-07-24 13:23:56 3.68G zroot/usr/src@2014-07-24 13:23:57 3.68G zroot/usr/src@2014-07-24 13:23:58 3.68G zroot/usr/src@2014-07-24 13:23:59 3.68G zroot/usr/src@2014-07-24 13:24:00 3.69G zroot/usr/src@2014-07-24 13:24:01 3.69G zroot/usr/src@2014-07-24 13:24:02 3.69G zroot/usr/src@2014-07-24 13:24:03 3.69G zroot/usr/src@2014-07-24 13:24:04 3.69G zroot/usr/src@2014-07-24 13:24:05 3.70G zroot/usr/src@2014-07-24 13:24:06 3.70G zroot/usr/src@2014-07-24 13:24:07 3.70G zroot/usr/src@2014-07-24 13:24:08 3.70G zroot/usr/src@2014-07-24 13:24:09 3.70G zroot/usr/src@2014-07-24 13:24:10 3.71G zroot/usr/src@2014-07-24 13:24:11 3.71G zroot/usr/src@2014-07-24 cannot receive new filesystem stream: invalid backup stream borg.lerctr.org /home/ler/bin $ -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-fs@FreeBSD.ORG Thu Jul 24 18:46:26 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 41D1848A; Thu, 24 Jul 2014 18:46:26 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 107C92A37; Thu, 24 Jul 2014 18:46:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=CKplyrXSnxOQ8eYWV5hIc4WxW1JzBSOm3KqymhXNLow=; b=t2yZCQ3kXYCdNw19kCMuqsQ4mCV4P+DjSl3QcCmb4DAhJE9oKDz0g8UXGU7uZXHEZoKXopXXJpjY56OH6xkvxYl3w9NqhRaaSDYN2AS/PuUftpenluNmrjLdnj/qIEGTR+owYfYb+VhgTltOmQouOouA6Dw2lKH1lfU9DZguFbI=; Received: from localhost.lerctr.org ([127.0.0.1]:45581 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.83 (FreeBSD)) (envelope-from ) id 1XAO23-000HlW-Eh; Thu, 24 Jul 2014 13:46:25 -0500 Received: from host.alcatel.com ([198.205.55.139]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Thu, 24 Jul 2014 13:46:23 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 24 Jul 2014 13:46:23 -0500 From: Larry Rosenman To: Allan Jude Subject: Re: zfs send/recv: STILL invalid Backup Stream In-Reply-To: <53D15438.6040105@freebsd.org> References: <62315eb454a95db636b7764aad3c0f9b@thebighonker.lerctr.org> <53D1448C.40908@freebsd.org> <53D15438.6040105@freebsd.org> Message-ID: <0a9cd451c3b4304d2b9d899fcb3decc3@thebighonker.lerctr.org> X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.0.1 X-Spam-Score: -2.9 (--) X-LERCTR-Spam-Score: -2.9 (--) X-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 X-LERCTR-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 Cc: Freebsd fs , freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jul 2014 18:46:26 -0000 On 2014-07-24 13:45, Allan Jude wrote: > On 2014-07-24 14:25, Larry Rosenman wrote: >> On 2014-07-24 12:43, Larry Rosenman wrote: >>> On 2014-07-24 12:38, Allan Jude wrote: >>>> On 2014-07-24 13:33, Larry Rosenman wrote: >>>>> TRYING to use zfs send/recv between a 10-STABLE and an 11-CURRENT >>>>> system, and receive the non-descript >>>>> "invalid backup stream". >>>>> >>>>> borg.lerctr.org /home/ler $ sudo bin/backup-TBH-ZFS-initial.sh >>>>> Password: >>>>> receiving full stream of zroot@2014-07-24 into >>>>> zroot/backups/TBH@2014-07-24 >>>>> received 41.7KB stream in 300 seconds (142B/sec) >>>>> receiving full stream of zroot/usr@2014-07-24 into >>>>> zroot/backups/TBH/usr@2014-07-24 >>>>> received 41.7KB stream in 1 seconds (41.7KB/sec) >>>>> receiving full stream of zroot/usr/local@2014-07-24 into >>>>> zroot/backups/TBH/usr/local@2014-07-24 >>>>> received 2.81GB stream in 1116 seconds (2.58MB/sec) >>>>> receiving full stream of zroot/usr/src@2014-07-24 into >>>>> zroot/backups/TBH/usr/src@2014-07-24 >>>>> cannot receive new filesystem stream: invalid backup stream >>>>> borg.lerctr.org /home/ler $ cat bin/backup-TBH-ZFS-initial.sh >>>>> #!/bin/sh >>>>> DATE=`date "+%Y-%m-%d"` >>>>> #DATE2=2013-03-24 >>>>> #DATE2=`date -v "-1d" "+%Y-%m-%d"` >>>>> # snap the source >>>>> ssh root@tbh.lerctr.org zfs snapshot -r zroot@${DATE} >>>>> # zfs copy the source to here. >>>>> ssh root@tbh.lerctr.org "zfs send -R -D zroot@${DATE} | \ >>>>> ssh home.lerctr.org \"zfs recv -F -u -v -d >>>>> zroot/backups/TBH\"" >>>>> # make sure we NEVER allow the backup stuff to automount. >>>>> /sbin/zfs list -H -t filesystem -r zroot/backups/TBH| \ >>>>> awk '{printf "/sbin/zfs set canmount=noauto %s\n",$1}' | sh >>>>> borg.lerctr.org /home/ler $ >>>>> >>>>> This has been happening for YEARS and I can't seem to interest >>>>> anyone in >>>>> fixing it. >>>>> >>>>> How can we get to the bottom of this? >>>>> >>>>> borg.lerctr.org /home/ler $ uname -a >>>>> FreeBSD borg.lerctr.org 11.0-CURRENT FreeBSD 11.0-CURRENT #56 >>>>> r268982M: >>>>> Tue Jul 22 10:14:59 CDT 2014 >>>>> root@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER amd64 >>>>> borg.lerctr.org /home/ler $ ssh tbh uname -a >>>>> FreeBSD thebighonker.lerctr.org 10.0-STABLE FreeBSD 10.0-STABLE #39 >>>>> r269019M: Wed Jul 23 11:44:35 CDT 2014 >>>>> root@thebighonker.lerctr.org:/usr/obj/usr/src/sys/GENERIC amd64 >>>>> borg.lerctr.org /home/ler $ >>>>> >>>> >>>> Try adding -v to the 'zfs send' and see if it gives you more detail. >>>> >>>> Can you also try this script for the replication: >>>> >>>> http://github.com/allanjude/zxfer >>> I've done that in the past and nothing, but I will try again. >>> >>> I will also look at zxfer.... :) >> >> >> with the -v, no more info, just what looks like normal messages.... >> >> 13:23:55 3.68G zroot/usr/src@2014-07-24 >> 13:23:56 3.68G zroot/usr/src@2014-07-24 >> 13:23:57 3.68G zroot/usr/src@2014-07-24 >> 13:23:58 3.68G zroot/usr/src@2014-07-24 >> 13:23:59 3.68G zroot/usr/src@2014-07-24 >> 13:24:00 3.69G zroot/usr/src@2014-07-24 >> 13:24:01 3.69G zroot/usr/src@2014-07-24 >> 13:24:02 3.69G zroot/usr/src@2014-07-24 >> 13:24:03 3.69G zroot/usr/src@2014-07-24 >> 13:24:04 3.69G zroot/usr/src@2014-07-24 >> 13:24:05 3.70G zroot/usr/src@2014-07-24 >> 13:24:06 3.70G zroot/usr/src@2014-07-24 >> 13:24:07 3.70G zroot/usr/src@2014-07-24 >> 13:24:08 3.70G zroot/usr/src@2014-07-24 >> 13:24:09 3.70G zroot/usr/src@2014-07-24 >> 13:24:10 3.71G zroot/usr/src@2014-07-24 >> 13:24:11 3.71G zroot/usr/src@2014-07-24 >> cannot receive new filesystem stream: invalid backup stream >> borg.lerctr.org /home/ler/bin $ > > I notice you are doing a deduplicated stream. Does it work without > deduplication (zfs send -D)? I will try that after this zxfer test I'm running finishes, but IIRC it doesn't matter.. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-fs@FreeBSD.ORG Thu Jul 24 19:31:44 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BFB393AB; Thu, 24 Jul 2014 19:31:44 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 862B02FEE; Thu, 24 Jul 2014 19:31:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=fA8lbrwLz19Cc2q0fi4rw+VGDXRwWbN59v12GZwbaS4=; b=mRdHx59Krj5Jkb4MTiXdmFG9Du4MBX6A8c56fwuh6z8Y/FAbeFhcdUdIeSuWYtUbG+7qIOL9C9eyL7KJh+aufGMrzMxTA4kO8AueHPBRY3Ue2xoKYvCefzDdRAYvBqxmg/+zkxqAVq15Q3fwpmoKXQbataA2Sls/Z+il4bh4O0w=; Received: from localhost.lerctr.org ([127.0.0.1]:41287 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.83 (FreeBSD)) (envelope-from ) id 1XAOjt-000IaR-O9; Thu, 24 Jul 2014 14:31:43 -0500 Received: from host.alcatel.com ([198.205.55.139]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Thu, 24 Jul 2014 14:31:41 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 24 Jul 2014 14:31:41 -0500 From: Larry Rosenman To: Allan Jude Subject: Re: zfs send/recv: STILL invalid Backup Stream In-Reply-To: <0a9cd451c3b4304d2b9d899fcb3decc3@thebighonker.lerctr.org> References: <62315eb454a95db636b7764aad3c0f9b@thebighonker.lerctr.org> <53D1448C.40908@freebsd.org> <53D15438.6040105@freebsd.org> <0a9cd451c3b4304d2b9d899fcb3decc3@thebighonker.lerctr.org> Message-ID: X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.0.1 X-Spam-Score: -2.9 (--) X-LERCTR-Spam-Score: -2.9 (--) X-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 X-LERCTR-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 Cc: Freebsd fs , freebsd-current@freebsd.org, owner-freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jul 2014 19:31:44 -0000 On 2014-07-24 13:46, Larry Rosenman wrote: >> I notice you are doing a deduplicated stream. Does it work without >> deduplication (zfs send -D)? > I will try that after this zxfer test I'm running finishes, but IIRC > it doesn't matter.. borg.lerctr.org /home/ler # zxfer -dFkPvs -g 376 -O root@tbh.lerctr.org -R zroot zroot/backups/TBH Creating recursive snapshot zroot@zxfer_26699_20140724135840. Checking grandfather status of all snapshots marked for deletion... Grandfather check passed. Sending zroot@zxfer_26699_20140724135840 to zroot/backups/TBH/zroot. Sending zroot/ROOT@zxfer_26699_20140724135840 to zroot/backups/TBH/zroot/ROOT. Sending zroot/ROOT/default@zxfer_23699_20140724134435 to zroot/backups/TBH/zroot/ROOT/default. Sending zroot/ROOT/default@zxfer_26699_20140724135840 to zroot/backups/TBH/zroot/ROOT/default. (incremental to zroot/ROOT/default@zxfer_23699_20140724134435.) Sending zroot/home@zxfer_26699_20140724135840 to zroot/backups/TBH/zroot/home. Write failed: Cannot allocate memory cannot receive new filesystem stream: invalid backup stream Error when zfs send/receiving. borg.lerctr.org /home/ler # well that's different....... -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-fs@FreeBSD.ORG Thu Jul 24 19:54:03 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D11D4D41; Thu, 24 Jul 2014 19:54:03 +0000 (UTC) Received: from mail.ijs.si (mail.ijs.si [IPv6:2001:1470:ff80::25]) by mx1.freebsd.org (Postfix) with ESMTP id 815AC21B1; Thu, 24 Jul 2014 19:54:03 +0000 (UTC) Received: from amavis-proxy-ori.ijs.si (localhost [IPv6:::1]) by mail.ijs.si (Postfix) with ESMTP id 3hK4491d6rzLm; Thu, 24 Jul 2014 21:54:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ijs.si; h= user-agent:message-id:references:in-reply-to:organization :subject:subject:from:from:date:date:content-transfer-encoding :content-type:content-type:mime-version:received:received :received:received; s=jakla2; t=1406231636; x=1408823637; bh=n6J CFmYBR1SbStk/Ok+l23JXcR1ITa2xt0mIGz9Avvs=; b=TAcIAQ/x8lZfyKqZqGX /gIaMMLe/fE55fGtjWMM6AqpFGnJVv8h4LHuP99+Ph8XrLtQh+U13PGK7QVSHilh j+8+1muD4Z/qXhcm6Yp0IlOAZL4eXSYpFqAuxU7u1ClVOmGNw6PHlIGwbEkEAJSp F1dUER2Wn8ICoDvbGSgg+0oo= X-Virus-Scanned: amavisd-new at ijs.si Received: from mail.ijs.si ([IPv6:::1]) by amavis-proxy-ori.ijs.si (mail.ijs.si [IPv6:::1]) (amavisd-new, port 10012) with ESMTP id ZXWpu2G07fmD; Thu, 24 Jul 2014 21:53:56 +0200 (CEST) Received: from mildred.ijs.si (mailbox.ijs.si [IPv6:2001:1470:ff80::143:1]) by mail.ijs.si (Postfix) with ESMTP; Thu, 24 Jul 2014 21:53:56 +0200 (CEST) Received: from neli.ijs.si (neli.ijs.si [IPv6:2001:1470:ff80:88:21c:c0ff:feb1:8c91]) by mildred.ijs.si (Postfix) with ESMTP id 3hK4441CPjzlH; Thu, 24 Jul 2014 21:53:56 +0200 (CEST) Received: from sleepy.ijs.si ([2001:1470:ff80:e001::1:1]) by neli.ijs.si with HTTP (HTTP/1.1 POST); Thu, 24 Jul 2014 21:53:56 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 24 Jul 2014 21:53:56 +0200 From: Mark Martinec To: Larry Rosenman Subject: Re: zfs send/recv: STILL invalid Backup Stream Organization: J. Stefan Institute In-Reply-To: References: <62315eb454a95db636b7764aad3c0f9b@thebighonker.lerctr.org> <53D1448C.40908@freebsd.org> <53D15438.6040105@freebsd.org> <0a9cd451c3b4304d2b9d899fcb3decc3@thebighonker.lerctr.org> Message-ID: <3d2aac84d962a703fbf56a864ba5f19c@mailbox.ijs.si> X-Sender: Mark.Martinec+freebsd@ijs.si User-Agent: Roundcube Webmail/1.0.1 Cc: Freebsd fs , freebsd-current@freebsd.org, owner-freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jul 2014 19:54:03 -0000 2014-07-24 21:31, Larry Rosenman wrote: > borg.lerctr.org /home/ler # zxfer -dFkPvs -g 376 -O > root@tbh.lerctr.org -R zroot zroot/backups/TBH > Creating recursive snapshot zroot@zxfer_26699_20140724135840. > Checking grandfather status of all snapshots marked for deletion... > Grandfather check passed. > Sending zroot@zxfer_26699_20140724135840 to zroot/backups/TBH/zroot. > Sending zroot/ROOT@zxfer_26699_20140724135840 to > zroot/backups/TBH/zroot/ROOT. > Sending zroot/ROOT/default@zxfer_23699_20140724134435 to > zroot/backups/TBH/zroot/ROOT/default. > Sending zroot/ROOT/default@zxfer_26699_20140724135840 to > zroot/backups/TBH/zroot/ROOT/default. > (incremental to zroot/ROOT/default@zxfer_23699_20140724134435.) > Sending zroot/home@zxfer_26699_20140724135840 to > zroot/backups/TBH/zroot/home. > Write failed: Cannot allocate memory ==================================== > cannot receive new filesystem stream: invalid backup stream > Error when zfs send/receiving. > borg.lerctr.org /home/ler # > > well that's different....... Sounds familiar, check my posting of today and links therein: http://lists.freebsd.org/pipermail/freebsd-net/2014-July/039347.html Mark From owner-freebsd-fs@FreeBSD.ORG Thu Jul 24 19:58:00 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C6F37A7; Thu, 24 Jul 2014 19:58:00 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9414921E0; Thu, 24 Jul 2014 19:58:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=p8LjRnc0lqNt6ko4/sznoKQzD98gBIpyID5xA5trqJM=; b=iWx/ukxIO4S0bsU2Plc4NswOY7bur9AnMQbpdv2oGmALhLk96svufBh20UDgq5plTH6JV4hpzGbhk7vgBfG/Y/dXi5eDICJO7DiRyXhEOBvR502968rGAmk2Gf37yzORiUqxOqtxTfFiR7XsS9u7dfNIq6+scvLdN4mT/mhb4K8=; Received: from localhost.lerctr.org ([127.0.0.1]:23406 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.83 (FreeBSD)) (envelope-from ) id 1XAP9K-000J6m-2I; Thu, 24 Jul 2014 14:57:59 -0500 Received: from host.alcatel.com ([198.205.55.139]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Thu, 24 Jul 2014 14:57:57 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 24 Jul 2014 14:57:57 -0500 From: Larry Rosenman To: Mark Martinec Subject: Re: zfs send/recv: STILL invalid Backup Stream In-Reply-To: <3d2aac84d962a703fbf56a864ba5f19c@mailbox.ijs.si> References: <62315eb454a95db636b7764aad3c0f9b@thebighonker.lerctr.org> <53D1448C.40908@freebsd.org> <53D15438.6040105@freebsd.org> <0a9cd451c3b4304d2b9d899fcb3decc3@thebighonker.lerctr.org> <3d2aac84d962a703fbf56a864ba5f19c@mailbox.ijs.si> Message-ID: X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.0.1 X-Spam-Score: -2.9 (--) X-LERCTR-Spam-Score: -2.9 (--) X-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 X-LERCTR-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 Cc: Freebsd fs , freebsd-current@freebsd.org, owner-freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jul 2014 19:58:01 -0000 On 2014-07-24 14:53, Mark Martinec wrote: > 2014-07-24 21:31, Larry Rosenman wrote: >> borg.lerctr.org /home/ler # zxfer -dFkPvs -g 376 -O >> root@tbh.lerctr.org -R zroot zroot/backups/TBH >> Creating recursive snapshot zroot@zxfer_26699_20140724135840. >> Checking grandfather status of all snapshots marked for deletion... >> Grandfather check passed. >> Sending zroot@zxfer_26699_20140724135840 to zroot/backups/TBH/zroot. >> Sending zroot/ROOT@zxfer_26699_20140724135840 to >> zroot/backups/TBH/zroot/ROOT. >> Sending zroot/ROOT/default@zxfer_23699_20140724134435 to >> zroot/backups/TBH/zroot/ROOT/default. >> Sending zroot/ROOT/default@zxfer_26699_20140724135840 to >> zroot/backups/TBH/zroot/ROOT/default. >> (incremental to zroot/ROOT/default@zxfer_23699_20140724134435.) >> Sending zroot/home@zxfer_26699_20140724135840 to >> zroot/backups/TBH/zroot/home. > >> Write failed: Cannot allocate memory > ==================================== > >> cannot receive new filesystem stream: invalid backup stream >> Error when zfs send/receiving. >> borg.lerctr.org /home/ler # >> >> well that's different....... > > Sounds familiar, check my posting of today and links therein: > > http://lists.freebsd.org/pipermail/freebsd-net/2014-July/039347.html > > Mark I'm not using netgraph to the best of my knowledge.... and the only fails on the SENDING host are: 8 Bucket: 64, 0, 41, 3555, 257774, 11, 0 12 Bucket: 96, 0, 96, 2569, 123653, 0, 0 16 Bucket: 128, 0, 17195, 506, 215573, 0, 0 32 Bucket: 256, 0, 340, 4670, 900638, 50, 0 64 Bucket: 512, 0, 10691, 365, 546888,185232, 0 128 Bucket: 1024, 0, 3563, 905, 348419, 0, 0 256 Bucket: 2048, 0, 2872, 162, 249995,59834, 0 vmem btag: 56, 0, 192811, 51500, 502264,1723, 0 -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-fs@FreeBSD.ORG Thu Jul 24 20:11:44 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DD0EB650; Thu, 24 Jul 2014 20:11:44 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A82282376; Thu, 24 Jul 2014 20:11:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=3UiDi1+aocOnaMg1FHQjMIAViSgjux9q+dCNC03jEkg=; b=Qnh4MutrKnlLnOjNmdIc5Yi2bRiUvBTAjG3VOwFXHZrC/7f5lKdCz0tj5E32IEsDucuvCrkusC0rIohhWIR8tlWcacBEncyA7Sv1RyMRpGDqEfFdjP5ujyHTqT9S94Ti5m46+J9zn+yVCRwAWRllxZvMe3ckRBvMZRmSHgPQ5LE=; Received: from localhost.lerctr.org ([127.0.0.1]:30108 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.83 (FreeBSD)) (envelope-from ) id 1XAPMR-000JLE-MY; Thu, 24 Jul 2014 15:11:43 -0500 Received: from host.alcatel.com ([198.205.55.139]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Thu, 24 Jul 2014 15:11:31 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 24 Jul 2014 15:11:31 -0500 From: Larry Rosenman To: Allan Jude Subject: Re: zfs send/recv: STILL invalid Backup Stream In-Reply-To: <53D1678C.4000007@freebsd.org> References: <62315eb454a95db636b7764aad3c0f9b@thebighonker.lerctr.org> <53D1448C.40908@freebsd.org> <53D15438.6040105@freebsd.org> <0a9cd451c3b4304d2b9d899fcb3decc3@thebighonker.lerctr.org> <3d2aac84d962a703fbf56a864ba5f19c@mailbox.ijs.si> <53D1678C.4000007@freebsd.org> Message-ID: <6e382c7d7efc8b4ddc3b770985eac1e0@thebighonker.lerctr.org> X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.0.1 X-Spam-Score: -2.9 (--) X-LERCTR-Spam-Score: -2.9 (--) X-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 X-LERCTR-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 Cc: Freebsd fs , freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jul 2014 20:11:45 -0000 On 2014-07-24 15:07, Allan Jude wrote: > On 2014-07-24 15:57, Larry Rosenman wrote: >> On 2014-07-24 14:53, Mark Martinec wrote: >>> 2014-07-24 21:31, Larry Rosenman wrote: >>>> borg.lerctr.org /home/ler # zxfer -dFkPvs -g 376 -O >>>> root@tbh.lerctr.org -R zroot zroot/backups/TBH >>>> Creating recursive snapshot zroot@zxfer_26699_20140724135840. >>>> Checking grandfather status of all snapshots marked for deletion... >>>> Grandfather check passed. >>>> Sending zroot@zxfer_26699_20140724135840 to zroot/backups/TBH/zroot. >>>> Sending zroot/ROOT@zxfer_26699_20140724135840 to >>>> zroot/backups/TBH/zroot/ROOT. >>>> Sending zroot/ROOT/default@zxfer_23699_20140724134435 to >>>> zroot/backups/TBH/zroot/ROOT/default. >>>> Sending zroot/ROOT/default@zxfer_26699_20140724135840 to >>>> zroot/backups/TBH/zroot/ROOT/default. >>>> (incremental to zroot/ROOT/default@zxfer_23699_20140724134435.) >>>> Sending zroot/home@zxfer_26699_20140724135840 to >>>> zroot/backups/TBH/zroot/home. >>> >>>> Write failed: Cannot allocate memory >>> ==================================== >>> >>>> cannot receive new filesystem stream: invalid backup stream >>>> Error when zfs send/receiving. >>>> borg.lerctr.org /home/ler # >>>> >>>> well that's different....... >>> >>> Sounds familiar, check my posting of today and links therein: >>> >>> >>> http://lists.freebsd.org/pipermail/freebsd-net/2014-July/039347.html >>> >>> Mark >> I'm not using netgraph to the best of my knowledge.... >> and the only fails on the SENDING host are: >> 8 Bucket: 64, 0, 41, 3555, 257774, 11, >> 0 >> 12 Bucket: 96, 0, 96, 2569, 123653, 0, >> 0 >> 16 Bucket: 128, 0, 17195, 506, 215573, 0, >> 0 >> 32 Bucket: 256, 0, 340, 4670, 900638, 50, >> 0 >> 64 Bucket: 512, 0, 10691, 365, 546888,185232, >> 0 >> 128 Bucket: 1024, 0, 3563, 905, 348419, 0, >> 0 >> 256 Bucket: 2048, 0, 2872, 162, 249995,59834, >> 0 >> vmem btag: 56, 0, 192811, 51500, 502264,1723, >> 0 >> >> > > I regularly use zxfer to transfer 500+ GiB datasets over the internet. > This week I actually replicated a 2.1 TiB dataset with zxfer without > issue. > > I wonder which thing is running out of memory. Is there a delay while > it > is 'running out of memory', or does it fail immediately? Does running > top while it is working on running out of memory reveal anything? > > I would expect to use up a lot of memory while doing deduplication, but > not otherwise. > > Note: I most often use openssh-portable rather than base ssh for > replication, as I enable the nonecipher to reduce CPU usage, and adjust > the TcpRcvBuf upwards to actually saturate a gigabit over the internet. I wasn't watching exactly what it was doing, but the sending box has 16G and 18G Swap and swap has NOT been touched. last pid: 74288; load averages: 4.70, 5.61, 5.91 up 1+03:14:18 15:10:44 115 processes: 3 running, 112 sleeping CPU: 0.6% user, 33.3% nice, 0.6% system, 0.1% interrupt, 65.4% idle Mem: 847M Active, 761M Inact, 14G Wired, 4616K Cache, 357M Free ARC: 12G Total, 6028M MFU, 5281M MRU, 3152K Anon, 120M Header, 688M Other Swap: 18G Total, 18G Free so I have zero idea where to go here. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-fs@FreeBSD.ORG Thu Jul 24 20:26:21 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 17CF1D53; Thu, 24 Jul 2014 20:26:21 +0000 (UTC) Received: from mail.ijs.si (mail.ijs.si [IPv6:2001:1470:ff80::25]) by mx1.freebsd.org (Postfix) with ESMTP id BB60E24B4; Thu, 24 Jul 2014 20:26:20 +0000 (UTC) Received: from amavis-proxy-ori.ijs.si (localhost [IPv6:::1]) by mail.ijs.si (Postfix) with ESMTP id 3hK4nS0zKrzTv; Thu, 24 Jul 2014 22:26:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ijs.si; h= user-agent:message-id:references:in-reply-to:organization :subject:subject:from:from:date:date:content-transfer-encoding :content-type:content-type:mime-version:received:received :received:received; s=jakla2; t=1406233575; x=1408825576; bh=tpv 5Wj26lIjjF/uPaYuPQ98QNsx5mWc+3hLxzwesSlk=; b=SYd+WlLExOX8Q+n5TrZ ZsiHUeircXkqEHRMg/m7vfyev+Fnv4EKQ3OqJAYqfhD93pKeAHVXEEndz4h7w2cG yhYgBX4mvuya0hOVHIegLMzN4OKfj3XAVzo/+KlXAM7DuPQOByYuNCZG+kOEcWGJ wzJlWhTK2IIM3fjbL4pdPtZ0= X-Virus-Scanned: amavisd-new at ijs.si Received: from mail.ijs.si ([IPv6:::1]) by amavis-proxy-ori.ijs.si (mail.ijs.si [IPv6:::1]) (amavisd-new, port 10012) with ESMTP id NdrahVGRa37c; Thu, 24 Jul 2014 22:26:15 +0200 (CEST) Received: from mildred.ijs.si (mailbox.ijs.si [IPv6:2001:1470:ff80::143:1]) by mail.ijs.si (Postfix) with ESMTP; Thu, 24 Jul 2014 22:26:15 +0200 (CEST) Received: from neli.ijs.si (neli.ijs.si [IPv6:2001:1470:ff80:88:21c:c0ff:feb1:8c91]) by mildred.ijs.si (Postfix) with ESMTP id 3hK4nM6j1Nzrc; Thu, 24 Jul 2014 22:26:15 +0200 (CEST) Received: from sleepy.ijs.si ([2001:1470:ff80:e001::1:1]) by neli.ijs.si with HTTP (HTTP/1.1 POST); Thu, 24 Jul 2014 22:26:15 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 24 Jul 2014 22:26:15 +0200 From: Mark Martinec To: Larry Rosenman Subject: Re: zfs send/recv: STILL invalid Backup Stream Organization: J. Stefan Institute In-Reply-To: References: <62315eb454a95db636b7764aad3c0f9b@thebighonker.lerctr.org> <53D1448C.40908@freebsd.org> <53D15438.6040105@freebsd.org> <0a9cd451c3b4304d2b9d899fcb3decc3@thebighonker.lerctr.org> <3d2aac84d962a703fbf56a864ba5f19c@mailbox.ijs.si> Message-ID: <9eef3e7e964cb33fd163cc2f9300f326@mailbox.ijs.si> X-Sender: Mark.Martinec+freebsd@ijs.si User-Agent: Roundcube Webmail/1.0.1 Cc: Freebsd fs , freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jul 2014 20:26:21 -0000 2014-07-24 21:57, Larry Rosenman wrote: >>>> Sending zroot/home@zxfer_26699_20140724135840 to >>>> zroot/backups/TBH/zroot/home. >>> Write failed: Cannot allocate memory >> ==================================== >>> cannot receive new filesystem stream: invalid backup stream >>> Error when zfs send/receiving. >>> borg.lerctr.org /home/ler # >>> >>> well that's different....... >> >> Sounds familiar, check my posting of today and links therein: >> http://lists.freebsd.org/pipermail/freebsd-net/2014-July/039347.html > I'm not using netgraph to the best of my knowledge.... Check to make sure: vmstat -z | fgrep NetGraph > and the only fails on the SENDING host are: > 8 Bucket: 64, 0, 41, 3555, 257774, 11, > 0 > 12 Bucket: 96, 0, 96, 2569, 123653, 0, > 0 > 16 Bucket: 128, 0, 17195, 506, 215573, 0, > 0 > 32 Bucket: 256, 0, 340, 4670, 900638, 50, > 0 > 64 Bucket: 512, 0, 10691, 365, 546888,185232, > 0 > 128 Bucket: 1024, 0, 3563, 905, 348419, 0, > 0 > 256 Bucket: 2048, 0, 2872, 162, 249995,59834, > 0 > vmem btag: 56, 0, 192811, 51500, 502264,1723, > 0 Adam Vande More gave other suggestions on that thread from 2011: http://lists.freebsd.org/pipermail/freebsd-emulation/2011-July/008971.html Mark From owner-freebsd-fs@FreeBSD.ORG Thu Jul 24 20:44:34 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0442D3DC; Thu, 24 Jul 2014 20:44:34 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C07E9263A; Thu, 24 Jul 2014 20:44:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=X7iovG7EY1z554Rg8rDnBUceBOBfe6ToLZ15UYx5urg=; b=lUbMmsdTAOhyyia723zHWzi7NABSO9JOT+6KQ05Ws3YX1YEgA1oP4u4NK3AWTebA+dv8N5YglnMlAw4XgcR/5NBWoxH6Vz/G2jhiAfWMpCkb1o4RR28wX5d5HIaj53A7La8iiSchLPzkIggvu/8cqqJaS0BOXbsXOLxaL2NBMgU=; Received: from localhost.lerctr.org ([127.0.0.1]:56175 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.83 (FreeBSD)) (envelope-from ) id 1XAPsL-000JqL-Nz; Thu, 24 Jul 2014 15:44:32 -0500 Received: from host.alcatel.com ([198.205.55.139]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Thu, 24 Jul 2014 15:44:29 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 24 Jul 2014 15:44:29 -0500 From: Larry Rosenman To: Mark Martinec Subject: Re: zfs send/recv: STILL invalid Backup Stream In-Reply-To: <9eef3e7e964cb33fd163cc2f9300f326@mailbox.ijs.si> References: <62315eb454a95db636b7764aad3c0f9b@thebighonker.lerctr.org> <53D1448C.40908@freebsd.org> <53D15438.6040105@freebsd.org> <0a9cd451c3b4304d2b9d899fcb3decc3@thebighonker.lerctr.org> <3d2aac84d962a703fbf56a864ba5f19c@mailbox.ijs.si> <9eef3e7e964cb33fd163cc2f9300f326@mailbox.ijs.si> Message-ID: X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.0.1 X-Spam-Score: -2.9 (--) X-LERCTR-Spam-Score: -2.9 (--) X-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 X-LERCTR-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 Cc: Freebsd fs , freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jul 2014 20:44:34 -0000 On 2014-07-24 15:26, Mark Martinec wrote: > 2014-07-24 21:57, Larry Rosenman wrote: > >>>>> Sending zroot/home@zxfer_26699_20140724135840 to >>>>> zroot/backups/TBH/zroot/home. >>>> Write failed: Cannot allocate memory >>> ==================================== >>>> cannot receive new filesystem stream: invalid backup stream >>>> Error when zfs send/receiving. >>>> borg.lerctr.org /home/ler # >>>> >>>> well that's different....... >>> >>> Sounds familiar, check my posting of today and links therein: >>> >>> http://lists.freebsd.org/pipermail/freebsd-net/2014-July/039347.html > >> I'm not using netgraph to the best of my knowledge.... > > Check to make sure: > vmstat -z | fgrep NetGraph > >> and the only fails on the SENDING host are: >> 8 Bucket: 64, 0, 41, 3555, 257774, 11, >> 0 >> 12 Bucket: 96, 0, 96, 2569, 123653, 0, >> 0 >> 16 Bucket: 128, 0, 17195, 506, 215573, 0, >> 0 >> 32 Bucket: 256, 0, 340, 4670, 900638, 50, >> 0 >> 64 Bucket: 512, 0, 10691, 365, 546888,185232, >> 0 >> 128 Bucket: 1024, 0, 3563, 905, 348419, 0, >> 0 >> 256 Bucket: 2048, 0, 2872, 162, 249995,59834, >> 0 >> vmem btag: 56, 0, 192811, 51500, 502264,1723, >> 0 > > > Adam Vande More gave other suggestions on that thread from 2011: > > > http://lists.freebsd.org/pipermail/freebsd-emulation/2011-July/008971.html > > > Mark thebighonker.lerctr.org /home/ler $ vmstat -z|fgrep NetGraph thebighonker.lerctr.org /home/ler $ This on physical hardware, FWIW. thebighonker.lerctr.org /home/ler $ vmstat -z ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP UMA Kegs: 384, 0, 202, 8, 202, 0, 0 UMA Zones: 1664, 0, 202, 0, 202, 0, 0 UMA Slabs: 80, 0, 361152, 38448, 1561588, 0, 0 UMA RCntSlabs: 88, 0, 4360, 5, 4360, 0, 0 UMA Hash: 256, 0, 8, 82, 82, 0, 0 4 Bucket: 32, 0, 4816, 33684, 2974043, 0, 0 6 Bucket: 48, 0, 6118, 2514, 624434, 0, 0 8 Bucket: 64, 0, 161, 3435, 265443, 11, 0 12 Bucket: 96, 0, 177, 2488, 130226, 0, 0 16 Bucket: 128, 0, 20301, 531, 224788, 0, 0 32 Bucket: 256, 0, 433, 4577, 945047, 50, 0 64 Bucket: 512, 0, 11137, 575, 579209,211292, 0 128 Bucket: 1024, 0, 3509, 959, 369839, 0, 0 256 Bucket: 2048, 0, 2847, 187, 269044,59834, 0 vmem btag: 56, 0, 209393, 34918, 557576,1723, 0 VM OBJECT: 256, 0, 91384, 791, 3480613, 0, 0 RADIX NODE: 144, 0, 227133, 37683,16029821, 0, 0 MAP: 240, 0, 3, 61, 3, 0, 0 KMAP ENTRY: 128, 0, 10, 269, 10, 0, 0 MAP ENTRY: 128, 0, 12666, 23604, 9357932, 0, 0 VMSPACE: 448, 0, 108, 306, 75357, 0, 0 fakepg: 104, 0, 0, 0, 0, 0, 0 mt_zone: 4112, 0, 368, 0, 368, 0, 0 16: 16, 0, 267839, 2488,45160196, 0, 0 32: 32, 0, 148139, 1486,39174301, 0, 0 64: 64, 0, 291630, 504078,40379996, 0, 0 128: 128, 0, 182409, 35490,63455920, 0, 0 256: 256, 0, 89368, 230912,35841036, 0, 0 512: 512, 0, 741, 3987,24215854, 0, 0 1024: 1024, 0, 1371, 1385, 1726440, 0, 0 2048: 2048, 0, 277, 417,10240207, 0, 0 4096: 4096, 0, 38306, 17186, 913638, 0, 0 SLEEPQUEUE: 80, 0, 613, 658, 613, 0, 0 64 pcpu: 8, 0, 1884, 1188, 1884, 0, 0 Files: 80, 0, 829, 1571, 4279893, 0, 0 rl_entry: 40, 0, 209, 1891, 209, 0, 0 TURNSTILE: 136, 0, 613, 307, 613, 0, 0 umtx pi: 96, 0, 0, 0, 0, 0, 0 MAC labels: 40, 0, 0, 0, 0, 0, 0 PROC: 1208, 0, 131, 121, 76219, 0, 0 THREAD: 1168, 0, 519, 93, 2039, 0, 0 cpuset: 72, 0, 270, 280, 426, 0, 0 cyclic_id_cache: 64, 0, 0, 0, 0, 0, 0 audit_record: 1248, 0, 0, 0, 0, 0, 0 mbuf_packet: 256, 6519810, 4081, 3256, 4910227, 0, 0 mbuf: 256, 6519810, 1052, 3251,93444045, 0, 0 mbuf_cluster: 2048, 1018718, 7337, 13, 7337, 0, 0 mbuf_jumbo_page: 4096, 509359, 12, 673,15015378, 0, 0 mbuf_jumbo_9k: 9216, 150921, 0, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 84893, 0, 0, 0, 0, 0 mbuf_ext_refcnt: 4, 0, 0, 2510, 85094, 0, 0 dtrace_state_cache: 4096, 0, 0, 0, 0, 0, 0 g_bio: 248, 0, 0, 1600,29986631, 0, 0 ttyinq: 160, 0, 180, 570, 2100, 0, 0 DMAR_MAP_ENTRY: 120, 0, 0, 0, 0, 0, 0 ttyoutq: 256, 0, 95, 640, 1093, 0, 0 ata_request: 336, 0, 0, 242, 33344, 0, 0 vtnet_tx_hdr: 24, 0, 0, 0, 0, 0, 0 cryptop: 88, 0, 0, 0, 0, 0, 0 cryptodesc: 72, 0, 0, 0, 0, 0, 0 FPU_save_area: 512, 0, 0, 0, 0, 0, 0 taskq_zone: 48, 0, 1, 3070, 194184, 0, 0 taskq_zone: 48, 0, 0, 0, 0, 0, 0 VNODE: 472, 0, 109527, 66161, 2663226, 0, 0 VNODEPOLL: 112, 0, 8, 762, 10, 0, 0 BUF TRIE: 144, 0, 0, 105354, 0, 0, 0 S VFS Cache: 108, 0, 114973, 60552, 2293717, 0, 0 STS VFS Cache: 148, 0, 0, 0, 0, 0, 0 L VFS Cache: 328, 0, 468, 145536, 800279, 0, 0 LTS VFS Cache: 368, 0, 0, 0, 0, 0, 0 NAMEI: 1024, 0, 0, 156,23968745, 0, 0 DIRHASH: 1024, 0, 0, 0, 0, 0, 0 NCLNODE: 528, 0, 0, 0, 0, 0, 0 Mountpoints: 816, 0, 11, 19, 11, 0, 0 range_seg_cache: 64, 0, 58377, 7157, 3500156, 0, 0 zio_cache: 920, 0, 1, 42591,104595517, 0, 0 zio_link_cache: 48, 0, 0, 44073,72482492, 0, 0 zio_buf_512: 512, 0, 552060, 193836, 1557459, 0, 0 zio_data_buf_512: 512, 0, 114904, 2056, 526943, 0, 0 zio_buf_1024: 1024, 0, 4246, 58, 56140, 0, 0 zio_data_buf_1024: 1024, 0, 51117, 1255, 211875, 0, 0 zio_buf_1536: 1536, 0, 1957, 59, 72179, 0, 0 zio_data_buf_1536: 1536, 0, 22199, 719, 127817, 0, 0 zio_buf_2048: 2048, 0, 691, 43, 217643, 0, 0 zio_data_buf_2048: 2048, 0, 13419, 1147, 87149, 0, 0 zio_buf_2560: 2560, 0, 455, 20, 16265, 0, 0 zio_data_buf_2560: 2560, 0, 9335, 493, 73967, 0, 0 zio_buf_3072: 3072, 0, 245, 16, 4658, 0, 0 zio_data_buf_3072: 3072, 0, 6940, 512, 53932, 0, 0 zio_buf_3584: 3584, 0, 248, 10, 38558, 0, 0 zio_data_buf_3584: 3584, 0, 5076, 352, 43162, 0, 0 zio_buf_4096: 4096, 0, 464, 729,14217630, 0, 0 zio_data_buf_4096: 4096, 0, 3857, 288, 41192, 0, 0 zio_buf_5120: 5120, 0, 167, 9, 6442, 0, 0 zio_data_buf_5120: 5120, 0, 6025, 461, 51297, 0, 0 zio_buf_6144: 6144, 0, 115, 6, 3196, 0, 0 zio_data_buf_6144: 6144, 0, 4453, 309, 38753, 0, 0 zio_buf_7168: 7168, 0, 85, 12, 36945, 0, 0 zio_data_buf_7168: 7168, 0, 3373, 273, 29007, 0, 0 zio_buf_8192: 8192, 0, 44, 39, 1373975, 0, 0 zio_data_buf_8192: 8192, 0, 2543, 205, 28736, 0, 0 zio_buf_10240: 10240, 0, 66, 7, 2534, 0, 0 zio_data_buf_10240: 10240, 0, 3986, 275, 39568, 0, 0 zio_buf_12288: 12288, 0, 71, 15, 654156, 0, 0 zio_data_buf_12288: 12288, 0, 2937, 209, 37077, 0, 0 zio_buf_14336: 14336, 0, 32, 9, 3164, 0, 0 zio_data_buf_14336: 14336, 0, 2236, 194, 23959, 0, 0 zio_buf_16384: 16384, 0, 72627, 226, 2784131, 0, 0 zio_data_buf_16384: 16384, 0, 1778, 155, 22372, 0, 0 zio_buf_20480: 20480, 0, 210, 14, 598092, 0, 0 zio_data_buf_20480: 20480, 0, 2724, 216, 29228, 0, 0 zio_buf_24576: 24576, 0, 18, 14, 270172, 0, 0 zio_data_buf_24576: 24576, 0, 2101, 173, 21241, 0, 0 zio_buf_28672: 28672, 0, 10, 54, 503029, 0, 0 zio_data_buf_28672: 28672, 0, 1406, 113, 18667, 0, 0 zio_buf_32768: 32768, 0, 10, 11, 77998, 0, 0 zio_data_buf_32768: 32768, 0, 1088, 96, 14116, 0, 0 zio_buf_36864: 36864, 0, 6, 11, 79009, 0, 0 zio_data_buf_36864: 36864, 0, 871, 87, 10066, 0, 0 zio_buf_40960: 40960, 0, 8, 10, 80622, 0, 0 zio_data_buf_40960: 40960, 0, 785, 57, 8149, 0, 0 zio_buf_45056: 45056, 0, 50, 10, 70143, 0, 0 zio_data_buf_45056: 45056, 0, 659, 61, 6889, 0, 0 zio_buf_49152: 49152, 0, 190, 9, 73508, 0, 0 zio_data_buf_49152: 49152, 0, 512, 42, 8420, 0, 0 zio_buf_53248: 53248, 0, 25, 10, 99339, 0, 0 zio_data_buf_53248: 53248, 0, 465, 30, 4838, 0, 0 zio_buf_57344: 57344, 0, 2, 15, 95818, 0, 0 zio_data_buf_57344: 57344, 0, 337, 33, 4753, 0, 0 zio_buf_61440: 61440, 0, 1, 23, 98972, 0, 0 zio_data_buf_61440: 61440, 0, 294, 33, 3792, 0, 0 zio_buf_65536: 65536, 0, 1, 9, 114664, 0, 0 zio_data_buf_65536: 65536, 0, 291, 29, 5152, 0, 0 zio_buf_69632: 69632, 0, 2, 10, 90765, 0, 0 zio_data_buf_69632: 69632, 0, 291, 22, 3555, 0, 0 zio_buf_73728: 73728, 0, 1, 12, 87316, 0, 0 zio_data_buf_73728: 73728, 0, 219, 23, 3361, 0, 0 zio_buf_77824: 77824, 0, 2, 9, 101758, 0, 0 zio_data_buf_77824: 77824, 0, 196, 25, 2907, 0, 0 zio_buf_81920: 81920, 0, 0, 15, 129645, 0, 0 zio_data_buf_81920: 81920, 0, 187, 18, 3578, 0, 0 zio_buf_86016: 86016, 0, 1, 7, 99241, 0, 0 zio_data_buf_86016: 86016, 0, 180, 22, 26051, 0, 0 zio_buf_90112: 90112, 0, 1, 12, 51491, 0, 0 zio_data_buf_90112: 90112, 0, 163, 18, 2438, 0, 0 zio_buf_94208: 94208, 0, 0, 9, 51043, 0, 0 zio_data_buf_94208: 94208, 0, 149, 12, 2288, 0, 0 zio_buf_98304: 98304, 0, 2, 8, 50322, 0, 0 zio_data_buf_98304: 98304, 0, 170, 25, 3002, 0, 0 zio_buf_102400: 102400, 0, 2, 10, 44642, 0, 0 zio_data_buf_102400: 102400, 0, 145, 19, 2017, 0, 0 zio_buf_106496: 106496, 0, 0, 15, 161697, 0, 0 zio_data_buf_106496: 106496, 0, 161, 16, 1944, 0, 0 zio_buf_110592: 110592, 0, 0, 19, 146777, 0, 0 zio_data_buf_110592: 110592, 0, 97, 15, 1636, 0, 0 zio_buf_114688: 114688, 0, 2, 8, 51698, 0, 0 zio_data_buf_114688: 114688, 0, 115, 20, 2691, 0, 0 zio_buf_118784: 118784, 0, 0, 8, 62915, 0, 0 zio_data_buf_118784: 118784, 0, 96, 11, 1440, 0, 0 zio_buf_122880: 122880, 0, 0, 12, 85924, 0, 0 zio_data_buf_122880: 122880, 0, 73, 17, 1510, 0, 0 zio_buf_126976: 126976, 0, 1, 11, 86133, 0, 0 zio_data_buf_126976: 126976, 0, 80, 9, 1351, 0, 0 zio_buf_131072: 131072, 0, 43, 43, 548063, 0, 0 zio_data_buf_131072: 131072, 0, 67545, 2, 646567, 0, 0 lz4_ctx: 16384, 0, 0, 16, 2249605, 0, 0 sa_cache: 80, 0, 109455, 66445, 2663034, 0, 0 dnode_t: 744, 0, 501793, 259152, 873895, 0, 0 dmu_buf_impl_t: 224, 0, 635273, 343604, 3042828, 0, 0 arc_buf_hdr_t: 216, 0, 643049, 141229, 3158205, 0, 0 arc_buf_t: 72, 0, 481443, 119872, 3705908, 0, 0 zil_lwb_cache: 192, 0, 8, 5072, 66838, 0, 0 zfs_znode_cache: 368, 0, 109455, 66105, 2663034, 0, 0 pipe: 744, 0, 113, 177, 66135, 0, 0 procdesc: 128, 0, 0, 0, 0, 0, 0 ksiginfo: 112, 0, 237, 2528, 8311580, 0, 0 itimer: 352, 0, 1, 32, 1, 0, 0 KNOTE: 128, 0, 376, 1391,17307485, 0, 0 socket: 696, 523340, 258, 302, 668458, 0, 0 unpcb: 240, 523344, 151, 649, 130961, 0, 0 ipq: 56, 31879, 0, 2059, 317, 0, 0 udp_inpcb: 392, 523340, 19, 421, 469503, 0, 0 udpcb: 16, 523586, 19, 2993, 469503, 0, 0 tcp_inpcb: 392, 523340, 100, 370, 67983, 0, 0 tcpcb: 1024, 523340, 78, 234, 67983, 0, 0 tcptw: 88, 27810, 22, 1013, 22066, 0, 0 syncache: 160, 15375, 0, 675, 45270, 0, 0 hostcache: 136, 15370, 255, 499, 3892, 0, 0 tcpreass: 40, 63700, 0, 2600, 7616, 0, 0 sackhole: 32, 0, 0, 1875, 7184, 0, 0 sctp_ep: 1408, 523340, 0, 0, 0, 0, 0 sctp_asoc: 2352, 40000, 0, 0, 0, 0, 0 sctp_laddr: 48, 80012, 0, 1328, 8, 0, 0 sctp_raddr: 728, 80000, 0, 0, 0, 0, 0 sctp_chunk: 136, 400026, 0, 0, 0, 0, 0 sctp_readq: 104, 400026, 0, 0, 0, 0, 0 sctp_stream_msg_out: 104, 400026, 0, 0, 0, 0, 0 sctp_asconf: 40, 400000, 0, 0, 0, 0, 0 sctp_asconf_ack: 48, 400060, 0, 0, 0, 0, 0 udplite_inpcb: 392, 523340, 0, 0, 0, 0, 0 ripcb: 392, 523340, 2, 58, 3, 0, 0 rtentry: 200, 0, 32, 408, 34, 0, 0 selfd: 56, 0, 333, 2223,29658712, 0, 0 SWAPMETA: 288, 2037438, 0, 0, 0, 0, 0 thebighonker.lerctr.org /home/ler -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-fs@FreeBSD.ORG Thu Jul 24 23:36:40 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 047964C3; Thu, 24 Jul 2014 23:36:40 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C31FF25E0; Thu, 24 Jul 2014 23:36:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=hJ9dgIsT4SfrtS9XLlYHJuoes1PStJQO7tvSbPNv6Wk=; b=jqkJL2SbanAAlHxV2V903jTeZWhYy24TZptFuRZljycLZTau5Id7hLrcr+z5BTxFXbpqKlWOsNle3/ZIepx/URXlPR61HyKzmub0C8q6extvqff1l4B18KkuuR7BUUgqvJEpbC1AxRsFY0mqP+/YaT0b+/suWens1C/5ygpMjn0=; Received: from localhost.lerctr.org ([127.0.0.1]:57714 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.83 (FreeBSD)) (envelope-from ) id 1XASYs-000M9l-KU; Thu, 24 Jul 2014 18:36:37 -0500 Received: from host.alcatel.com ([198.205.55.139]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Thu, 24 Jul 2014 18:36:34 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 24 Jul 2014 18:36:34 -0500 From: Larry Rosenman To: Mark Martinec , Allan Jude Subject: Re: zfs send/recv: STILL invalid Backup Stream In-Reply-To: References: <62315eb454a95db636b7764aad3c0f9b@thebighonker.lerctr.org> <53D1448C.40908@freebsd.org> <53D15438.6040105@freebsd.org> <0a9cd451c3b4304d2b9d899fcb3decc3@thebighonker.lerctr.org> <3d2aac84d962a703fbf56a864ba5f19c@mailbox.ijs.si> <9eef3e7e964cb33fd163cc2f9300f326@mailbox.ijs.si> Message-ID: X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.0.1 X-Spam-Score: -2.9 (--) X-LERCTR-Spam-Score: -2.9 (--) X-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 X-LERCTR-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 Cc: Freebsd fs , freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jul 2014 23:36:40 -0000 On 2014-07-24 15:44, Larry Rosenman wrote: > On 2014-07-24 15:26, Mark Martinec wrote: >> 2014-07-24 21:57, Larry Rosenman wrote: >> >>>>>> Sending zroot/home@zxfer_26699_20140724135840 to >>>>>> zroot/backups/TBH/zroot/home. >>>>> Write failed: Cannot allocate memory >>>> ==================================== >>>>> cannot receive new filesystem stream: invalid backup stream >>>>> Error when zfs send/receiving. >>>>> borg.lerctr.org /home/ler # >>>>> >>>>> well that's different....... >>>> >>>> Sounds familiar, check my posting of today and links therein: >>>> >>>> http://lists.freebsd.org/pipermail/freebsd-net/2014-July/039347.html >> >>> I'm not using netgraph to the best of my knowledge.... >> >> Check to make sure: >> vmstat -z | fgrep NetGraph >> >>> and the only fails on the SENDING host are: >>> 8 Bucket: 64, 0, 41, 3555, 257774, 11, >>> 0 >>> 12 Bucket: 96, 0, 96, 2569, 123653, 0, >>> 0 >>> 16 Bucket: 128, 0, 17195, 506, 215573, 0, >>> 0 >>> 32 Bucket: 256, 0, 340, 4670, 900638, 50, >>> 0 >>> 64 Bucket: 512, 0, 10691, 365, >>> 546888,185232, >>> 0 >>> 128 Bucket: 1024, 0, 3563, 905, 348419, 0, >>> 0 >>> 256 Bucket: 2048, 0, 2872, 162, 249995,59834, >>> 0 >>> vmem btag: 56, 0, 192811, 51500, 502264,1723, >>> 0 >> >> >> Adam Vande More gave other suggestions on that thread from 2011: >> >> >> http://lists.freebsd.org/pipermail/freebsd-emulation/2011-July/008971.html >> >> >> Mark > > > thebighonker.lerctr.org /home/ler $ vmstat -z|fgrep NetGraph > thebighonker.lerctr.org /home/ler $ > > This on physical hardware, FWIW. > > thebighonker.lerctr.org /home/ler $ vmstat -z > ITEM SIZE LIMIT USED FREE REQ FAIL > SLEEP > > UMA Kegs: 384, 0, 202, 8, 202, 0, > 0 > UMA Zones: 1664, 0, 202, 0, 202, 0, > 0 > UMA Slabs: 80, 0, 361152, 38448, 1561588, 0, > 0 > UMA RCntSlabs: 88, 0, 4360, 5, 4360, 0, > 0 > UMA Hash: 256, 0, 8, 82, 82, 0, > 0 > 4 Bucket: 32, 0, 4816, 33684, 2974043, 0, > 0 > 6 Bucket: 48, 0, 6118, 2514, 624434, 0, > 0 > 8 Bucket: 64, 0, 161, 3435, 265443, 11, > 0 > 12 Bucket: 96, 0, 177, 2488, 130226, 0, > 0 > 16 Bucket: 128, 0, 20301, 531, 224788, 0, > 0 > 32 Bucket: 256, 0, 433, 4577, 945047, 50, > 0 > 64 Bucket: 512, 0, 11137, 575, 579209,211292, > 0 > 128 Bucket: 1024, 0, 3509, 959, 369839, 0, > 0 > 256 Bucket: 2048, 0, 2847, 187, 269044,59834, > 0 > vmem btag: 56, 0, 209393, 34918, 557576,1723, > 0 > VM OBJECT: 256, 0, 91384, 791, 3480613, 0, > 0 > RADIX NODE: 144, 0, 227133, 37683,16029821, 0, > 0 > MAP: 240, 0, 3, 61, 3, 0, > 0 > KMAP ENTRY: 128, 0, 10, 269, 10, 0, > 0 > MAP ENTRY: 128, 0, 12666, 23604, 9357932, 0, > 0 > VMSPACE: 448, 0, 108, 306, 75357, 0, > 0 > fakepg: 104, 0, 0, 0, 0, 0, > 0 > mt_zone: 4112, 0, 368, 0, 368, 0, > 0 > 16: 16, 0, 267839, 2488,45160196, 0, > 0 > 32: 32, 0, 148139, 1486,39174301, 0, > 0 > 64: 64, 0, 291630, 504078,40379996, 0, > 0 > 128: 128, 0, 182409, 35490,63455920, 0, > 0 > 256: 256, 0, 89368, 230912,35841036, 0, > 0 > 512: 512, 0, 741, 3987,24215854, 0, > 0 > 1024: 1024, 0, 1371, 1385, 1726440, 0, > 0 > 2048: 2048, 0, 277, 417,10240207, 0, > 0 > 4096: 4096, 0, 38306, 17186, 913638, 0, > 0 > SLEEPQUEUE: 80, 0, 613, 658, 613, 0, > 0 > 64 pcpu: 8, 0, 1884, 1188, 1884, 0, > 0 > Files: 80, 0, 829, 1571, 4279893, 0, > 0 > rl_entry: 40, 0, 209, 1891, 209, 0, > 0 > TURNSTILE: 136, 0, 613, 307, 613, 0, > 0 > umtx pi: 96, 0, 0, 0, 0, 0, > 0 > MAC labels: 40, 0, 0, 0, 0, 0, > 0 > PROC: 1208, 0, 131, 121, 76219, 0, > 0 > THREAD: 1168, 0, 519, 93, 2039, 0, > 0 > cpuset: 72, 0, 270, 280, 426, 0, > 0 > cyclic_id_cache: 64, 0, 0, 0, 0, 0, > 0 > audit_record: 1248, 0, 0, 0, 0, 0, > 0 > mbuf_packet: 256, 6519810, 4081, 3256, 4910227, 0, > 0 > mbuf: 256, 6519810, 1052, 3251,93444045, 0, > 0 > mbuf_cluster: 2048, 1018718, 7337, 13, 7337, 0, > 0 > mbuf_jumbo_page: 4096, 509359, 12, 673,15015378, 0, > 0 > mbuf_jumbo_9k: 9216, 150921, 0, 0, 0, 0, > 0 > mbuf_jumbo_16k: 16384, 84893, 0, 0, 0, 0, > 0 > mbuf_ext_refcnt: 4, 0, 0, 2510, 85094, 0, > 0 > dtrace_state_cache: 4096, 0, 0, 0, 0, 0, > 0 > g_bio: 248, 0, 0, 1600,29986631, 0, > 0 > ttyinq: 160, 0, 180, 570, 2100, 0, > 0 > DMAR_MAP_ENTRY: 120, 0, 0, 0, 0, 0, > 0 > ttyoutq: 256, 0, 95, 640, 1093, 0, > 0 > ata_request: 336, 0, 0, 242, 33344, 0, > 0 > vtnet_tx_hdr: 24, 0, 0, 0, 0, 0, > 0 > cryptop: 88, 0, 0, 0, 0, 0, > 0 > cryptodesc: 72, 0, 0, 0, 0, 0, > 0 > FPU_save_area: 512, 0, 0, 0, 0, 0, > 0 > taskq_zone: 48, 0, 1, 3070, 194184, 0, > 0 > taskq_zone: 48, 0, 0, 0, 0, 0, > 0 > VNODE: 472, 0, 109527, 66161, 2663226, 0, > 0 > VNODEPOLL: 112, 0, 8, 762, 10, 0, > 0 > BUF TRIE: 144, 0, 0, 105354, 0, 0, > 0 > S VFS Cache: 108, 0, 114973, 60552, 2293717, 0, > 0 > STS VFS Cache: 148, 0, 0, 0, 0, 0, > 0 > L VFS Cache: 328, 0, 468, 145536, 800279, 0, > 0 > LTS VFS Cache: 368, 0, 0, 0, 0, 0, > 0 > NAMEI: 1024, 0, 0, 156,23968745, 0, > 0 > DIRHASH: 1024, 0, 0, 0, 0, 0, > 0 > NCLNODE: 528, 0, 0, 0, 0, 0, > 0 > Mountpoints: 816, 0, 11, 19, 11, 0, > 0 > range_seg_cache: 64, 0, 58377, 7157, 3500156, 0, > 0 > zio_cache: 920, 0, 1, 42591,104595517, 0, > 0 > zio_link_cache: 48, 0, 0, 44073,72482492, 0, > 0 > zio_buf_512: 512, 0, 552060, 193836, 1557459, 0, > 0 > zio_data_buf_512: 512, 0, 114904, 2056, 526943, 0, > 0 > zio_buf_1024: 1024, 0, 4246, 58, 56140, 0, > 0 > zio_data_buf_1024: 1024, 0, 51117, 1255, 211875, 0, > 0 > zio_buf_1536: 1536, 0, 1957, 59, 72179, 0, > 0 > zio_data_buf_1536: 1536, 0, 22199, 719, 127817, 0, > 0 > zio_buf_2048: 2048, 0, 691, 43, 217643, 0, > 0 > zio_data_buf_2048: 2048, 0, 13419, 1147, 87149, 0, > 0 > zio_buf_2560: 2560, 0, 455, 20, 16265, 0, > 0 > zio_data_buf_2560: 2560, 0, 9335, 493, 73967, 0, > 0 > zio_buf_3072: 3072, 0, 245, 16, 4658, 0, > 0 > zio_data_buf_3072: 3072, 0, 6940, 512, 53932, 0, > 0 > zio_buf_3584: 3584, 0, 248, 10, 38558, 0, > 0 > zio_data_buf_3584: 3584, 0, 5076, 352, 43162, 0, > 0 > zio_buf_4096: 4096, 0, 464, 729,14217630, 0, > 0 > zio_data_buf_4096: 4096, 0, 3857, 288, 41192, 0, > 0 > zio_buf_5120: 5120, 0, 167, 9, 6442, 0, > 0 > zio_data_buf_5120: 5120, 0, 6025, 461, 51297, 0, > 0 > zio_buf_6144: 6144, 0, 115, 6, 3196, 0, > 0 > zio_data_buf_6144: 6144, 0, 4453, 309, 38753, 0, > 0 > zio_buf_7168: 7168, 0, 85, 12, 36945, 0, > 0 > zio_data_buf_7168: 7168, 0, 3373, 273, 29007, 0, > 0 > zio_buf_8192: 8192, 0, 44, 39, 1373975, 0, > 0 > zio_data_buf_8192: 8192, 0, 2543, 205, 28736, 0, > 0 > zio_buf_10240: 10240, 0, 66, 7, 2534, 0, > 0 > zio_data_buf_10240: 10240, 0, 3986, 275, 39568, 0, > 0 > zio_buf_12288: 12288, 0, 71, 15, 654156, 0, > 0 > zio_data_buf_12288: 12288, 0, 2937, 209, 37077, 0, > 0 > zio_buf_14336: 14336, 0, 32, 9, 3164, 0, > 0 > zio_data_buf_14336: 14336, 0, 2236, 194, 23959, 0, > 0 > zio_buf_16384: 16384, 0, 72627, 226, 2784131, 0, > 0 > zio_data_buf_16384: 16384, 0, 1778, 155, 22372, 0, > 0 > zio_buf_20480: 20480, 0, 210, 14, 598092, 0, > 0 > zio_data_buf_20480: 20480, 0, 2724, 216, 29228, 0, > 0 > zio_buf_24576: 24576, 0, 18, 14, 270172, 0, > 0 > zio_data_buf_24576: 24576, 0, 2101, 173, 21241, 0, > 0 > zio_buf_28672: 28672, 0, 10, 54, 503029, 0, > 0 > zio_data_buf_28672: 28672, 0, 1406, 113, 18667, 0, > 0 > zio_buf_32768: 32768, 0, 10, 11, 77998, 0, > 0 > zio_data_buf_32768: 32768, 0, 1088, 96, 14116, 0, > 0 > zio_buf_36864: 36864, 0, 6, 11, 79009, 0, > 0 > zio_data_buf_36864: 36864, 0, 871, 87, 10066, 0, > 0 > zio_buf_40960: 40960, 0, 8, 10, 80622, 0, > 0 > zio_data_buf_40960: 40960, 0, 785, 57, 8149, 0, > 0 > zio_buf_45056: 45056, 0, 50, 10, 70143, 0, > 0 > zio_data_buf_45056: 45056, 0, 659, 61, 6889, 0, > 0 > zio_buf_49152: 49152, 0, 190, 9, 73508, 0, > 0 > zio_data_buf_49152: 49152, 0, 512, 42, 8420, 0, > 0 > zio_buf_53248: 53248, 0, 25, 10, 99339, 0, > 0 > zio_data_buf_53248: 53248, 0, 465, 30, 4838, 0, > 0 > zio_buf_57344: 57344, 0, 2, 15, 95818, 0, > 0 > zio_data_buf_57344: 57344, 0, 337, 33, 4753, 0, > 0 > zio_buf_61440: 61440, 0, 1, 23, 98972, 0, > 0 > zio_data_buf_61440: 61440, 0, 294, 33, 3792, 0, > 0 > zio_buf_65536: 65536, 0, 1, 9, 114664, 0, > 0 > zio_data_buf_65536: 65536, 0, 291, 29, 5152, 0, > 0 > zio_buf_69632: 69632, 0, 2, 10, 90765, 0, > 0 > zio_data_buf_69632: 69632, 0, 291, 22, 3555, 0, > 0 > zio_buf_73728: 73728, 0, 1, 12, 87316, 0, > 0 > zio_data_buf_73728: 73728, 0, 219, 23, 3361, 0, > 0 > zio_buf_77824: 77824, 0, 2, 9, 101758, 0, > 0 > zio_data_buf_77824: 77824, 0, 196, 25, 2907, 0, > 0 > zio_buf_81920: 81920, 0, 0, 15, 129645, 0, > 0 > zio_data_buf_81920: 81920, 0, 187, 18, 3578, 0, > 0 > zio_buf_86016: 86016, 0, 1, 7, 99241, 0, > 0 > zio_data_buf_86016: 86016, 0, 180, 22, 26051, 0, > 0 > zio_buf_90112: 90112, 0, 1, 12, 51491, 0, > 0 > zio_data_buf_90112: 90112, 0, 163, 18, 2438, 0, > 0 > zio_buf_94208: 94208, 0, 0, 9, 51043, 0, > 0 > zio_data_buf_94208: 94208, 0, 149, 12, 2288, 0, > 0 > zio_buf_98304: 98304, 0, 2, 8, 50322, 0, > 0 > zio_data_buf_98304: 98304, 0, 170, 25, 3002, 0, > 0 > zio_buf_102400: 102400, 0, 2, 10, 44642, 0, > 0 > zio_data_buf_102400: 102400, 0, 145, 19, 2017, 0, > 0 > zio_buf_106496: 106496, 0, 0, 15, 161697, 0, > 0 > zio_data_buf_106496: 106496, 0, 161, 16, 1944, 0, > 0 > zio_buf_110592: 110592, 0, 0, 19, 146777, 0, > 0 > zio_data_buf_110592: 110592, 0, 97, 15, 1636, 0, > 0 > zio_buf_114688: 114688, 0, 2, 8, 51698, 0, > 0 > zio_data_buf_114688: 114688, 0, 115, 20, 2691, 0, > 0 > zio_buf_118784: 118784, 0, 0, 8, 62915, 0, > 0 > zio_data_buf_118784: 118784, 0, 96, 11, 1440, 0, > 0 > zio_buf_122880: 122880, 0, 0, 12, 85924, 0, > 0 > zio_data_buf_122880: 122880, 0, 73, 17, 1510, 0, > 0 > zio_buf_126976: 126976, 0, 1, 11, 86133, 0, > 0 > zio_data_buf_126976: 126976, 0, 80, 9, 1351, 0, > 0 > zio_buf_131072: 131072, 0, 43, 43, 548063, 0, > 0 > zio_data_buf_131072: 131072, 0, 67545, 2, 646567, 0, > 0 > lz4_ctx: 16384, 0, 0, 16, 2249605, 0, > 0 > sa_cache: 80, 0, 109455, 66445, 2663034, 0, > 0 > dnode_t: 744, 0, 501793, 259152, 873895, 0, > 0 > dmu_buf_impl_t: 224, 0, 635273, 343604, 3042828, 0, > 0 > arc_buf_hdr_t: 216, 0, 643049, 141229, 3158205, 0, > 0 > arc_buf_t: 72, 0, 481443, 119872, 3705908, 0, > 0 > zil_lwb_cache: 192, 0, 8, 5072, 66838, 0, > 0 > zfs_znode_cache: 368, 0, 109455, 66105, 2663034, 0, > 0 > pipe: 744, 0, 113, 177, 66135, 0, > 0 > procdesc: 128, 0, 0, 0, 0, 0, > 0 > ksiginfo: 112, 0, 237, 2528, 8311580, 0, > 0 > itimer: 352, 0, 1, 32, 1, 0, > 0 > KNOTE: 128, 0, 376, 1391,17307485, 0, > 0 > socket: 696, 523340, 258, 302, 668458, 0, > 0 > unpcb: 240, 523344, 151, 649, 130961, 0, > 0 > ipq: 56, 31879, 0, 2059, 317, 0, > 0 > udp_inpcb: 392, 523340, 19, 421, 469503, 0, > 0 > udpcb: 16, 523586, 19, 2993, 469503, 0, > 0 > tcp_inpcb: 392, 523340, 100, 370, 67983, 0, > 0 > tcpcb: 1024, 523340, 78, 234, 67983, 0, > 0 > tcptw: 88, 27810, 22, 1013, 22066, 0, > 0 > syncache: 160, 15375, 0, 675, 45270, 0, > 0 > hostcache: 136, 15370, 255, 499, 3892, 0, > 0 > tcpreass: 40, 63700, 0, 2600, 7616, 0, > 0 > sackhole: 32, 0, 0, 1875, 7184, 0, > 0 > sctp_ep: 1408, 523340, 0, 0, 0, 0, > 0 > sctp_asoc: 2352, 40000, 0, 0, 0, 0, > 0 > sctp_laddr: 48, 80012, 0, 1328, 8, 0, > 0 > sctp_raddr: 728, 80000, 0, 0, 0, 0, > 0 > sctp_chunk: 136, 400026, 0, 0, 0, 0, > 0 > sctp_readq: 104, 400026, 0, 0, 0, 0, > 0 > sctp_stream_msg_out: 104, 400026, 0, 0, 0, 0, > 0 > sctp_asconf: 40, 400000, 0, 0, 0, 0, > 0 > sctp_asconf_ack: 48, 400060, 0, 0, 0, 0, > 0 > udplite_inpcb: 392, 523340, 0, 0, 0, 0, > 0 > ripcb: 392, 523340, 2, 58, 3, 0, > 0 > rtentry: 200, 0, 32, 408, 34, 0, > 0 > selfd: 56, 0, 333, 2223,29658712, 0, > 0 > SWAPMETA: 288, 2037438, 0, 0, 0, 0, > 0 > > thebighonker.lerctr.org /home/ler and I ran the script again.... It got further: 18:20:52 25.1G zroot/home@zxfer_26699_20140724135840 18:20:53 25.1G zroot/home@zxfer_26699_20140724135840 18:20:54 25.1G zroot/home@zxfer_26699_20140724135840 18:20:55 25.1G zroot/home@zxfer_26699_20140724135840 18:20:56 25.1G zroot/home@zxfer_26699_20140724135840 18:20:57 25.1G zroot/home@zxfer_26699_20140724135840 18:20:58 25.1G zroot/home@zxfer_26699_20140724135840 Connection to home.lerctr.org closed by remote host. warning: cannot send 'zroot/home@zxfer_26699_20140724135840': Broken pipe TIME SENT SNAPSHOT warning: cannot send 'zroot/home@2014-07-24': Broken pipe cannot open 'zroot/backups/TBH2/home': dataset does not exist borg.lerctr.org /home/ler/bin # BUT we still fail. #!/bin/sh DATE=`date "+%Y-%m-%d"` #DATE2=2013-03-24 #DATE2=`date -v "-1d" "+%Y-%m-%d"` # snap the source ssh root@tbh.lerctr.org zfs snapshot -r zroot@${DATE} # zfs copy the source to here. ssh root@tbh.lerctr.org "zfs send -v -R zroot@${DATE} | \ ssh home.lerctr.org \"zfs recv -F -u -v -d zroot/backups/TBH2\"" # make sure we NEVER allow the backup stuff to automount. /sbin/zfs list -H -t filesystem -r zroot/backups/TBH2| \ awk '{printf "/sbin/zfs set canmount=noauto %s\n",$1}' | sh borg.lerctr.org /home/ler/bin # this was to a FRESH zroot/backups/TBH2 -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-fs@FreeBSD.ORG Fri Jul 25 00:46:50 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D601252A; Fri, 25 Jul 2014 00:46:50 +0000 (UTC) Received: from mail.ijs.si (mail.ijs.si [IPv6:2001:1470:ff80::25]) by mx1.freebsd.org (Postfix) with ESMTP id 80A662C16; Fri, 25 Jul 2014 00:46:50 +0000 (UTC) Received: from amavis-proxy-ori.ijs.si (localhost [IPv6:::1]) by mail.ijs.si (Postfix) with ESMTP id 3hKBZ04mW6z1BN; Fri, 25 Jul 2014 02:46:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ijs.si; h= user-agent:message-id:references:in-reply-to:organization :subject:subject:from:from:date:date:content-transfer-encoding :content-type:content-type:mime-version:received:received :received:received; s=jakla2; t=1406249205; x=1408841206; bh=saE KP6W/ydxAlua2Y9xQH66Q4LNfRKZjXaDbEI/MHuA=; b=On38cpScwQG6cgFcImQ qu/g3y5PDRa4p1rDmWx8M2s+4RtB97sSY7XIiFxIGQUXAQKtvu+aKgZPQ02pFOqG oKc+B04m35hd9+dIAH6LG3FUN4UvzbxhddICPnvSnSKiYOEQ94PwXdSn4HHEgD7h Mh3nEbHjEolLSDPMSmNuGvWk= X-Virus-Scanned: amavisd-new at ijs.si Received: from mail.ijs.si ([IPv6:::1]) by amavis-proxy-ori.ijs.si (mail.ijs.si [IPv6:::1]) (amavisd-new, port 10012) with ESMTP id l2rf58skVqRZ; Fri, 25 Jul 2014 02:46:45 +0200 (CEST) Received: from mildred.ijs.si (mailbox.ijs.si [IPv6:2001:1470:ff80::143:1]) by mail.ijs.si (Postfix) with ESMTP; Fri, 25 Jul 2014 02:46:44 +0200 (CEST) Received: from neli.ijs.si (neli.ijs.si [IPv6:2001:1470:ff80:88:21c:c0ff:feb1:8c91]) by mildred.ijs.si (Postfix) with ESMTP id 3hKBYw1chnz1Kr; Fri, 25 Jul 2014 02:46:44 +0200 (CEST) Received: from sleepy.ijs.si ([2001:1470:ff80:e001::1:1]) by neli.ijs.si with HTTP (HTTP/1.1 POST); Fri, 25 Jul 2014 02:46:44 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Fri, 25 Jul 2014 02:46:44 +0200 From: Mark Martinec To: Larry Rosenman Subject: Re: zfs send/recv: STILL invalid Backup Stream Organization: J. Stefan Institute In-Reply-To: References: <62315eb454a95db636b7764aad3c0f9b@thebighonker.lerctr.org> <53D1448C.40908@freebsd.org> <53D15438.6040105@freebsd.org> <0a9cd451c3b4304d2b9d899fcb3decc3@thebighonker.lerctr.org> <3d2aac84d962a703fbf56a864ba5f19c@mailbox.ijs.si> <9eef3e7e964cb33fd163cc2f9300f326@mailbox.ijs.si> Message-ID: X-Sender: Mark.Martinec+freebsd@ijs.si User-Agent: Roundcube Webmail/1.0.1 Cc: Freebsd fs , freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2014 00:46:50 -0000 2014-07-25 01:36 Larry Rosenman wrote: > #!/bin/sh > DATE=`date "+%Y-%m-%d"` > #DATE2=2013-03-24 > #DATE2=`date -v "-1d" "+%Y-%m-%d"` > # snap the source > ssh root@tbh.lerctr.org zfs snapshot -r zroot@${DATE} > # zfs copy the source to here. > ssh root@tbh.lerctr.org "zfs send -v -R zroot@${DATE} | \ > ssh home.lerctr.org \"zfs recv -F -u -v -d zroot/backups/TBH2\"" Btw, this double-ssh looks awkward, why not just: ssh root@tbh.lerctr.org "zfs send ..." | zfs recv ... or better yet: ssh root@tbh.lerctr.org "zfs send ..." | mbuffer -m 16M | zfs recv ... (The misc/mbuffer compensates for bursty zfs reads and writes. A note to myself: I should suggest to Allan to add mbuffer in a pipe as used in sysutils/zxfer, instead of patching zxfer for our local use :) Mark From owner-freebsd-fs@FreeBSD.ORG Fri Jul 25 04:18:50 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8F4023E3 for ; Fri, 25 Jul 2014 04:18:50 +0000 (UTC) Received: from mail-pa0-x231.google.com (mail-pa0-x231.google.com [IPv6:2607:f8b0:400e:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6B69E2D79 for ; Fri, 25 Jul 2014 04:18:50 +0000 (UTC) Received: by mail-pa0-f49.google.com with SMTP id hz1so5276620pad.36 for ; Thu, 24 Jul 2014 21:18:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=7XSlpBxws1xlhaF++VjewgkToQe6QrFFer88qIwCVc8=; b=ySJjmRlhxgrCmxSZNfVt0adVvd0lQFBp/aS6kYgE4TBn6djgiDYlZPp5nm3uoeVkAr IhcmTYwN4KHJaJwdI6rd6eATXJueGYxV94g9BxcDggIuiRMZGKYAGoMwnCBjoADFj5Q2 R2595SD7Ckfu+tQNzbRoRSf44eyqdmnd1rPARV06xkH3Gtfr7mGf93fppqDzWJxLR71L z5QkqfMQP/5Oig8XLKQlKqrUdBoWv7s6uhJsicmCDUQ96CwHrL0DXJ3xhWIPKN3iwPUe 30AR4WBBRuvvo8yprA4ay902E+UIQZoYRe5Pgyf4WsBMI+N29jD5mFad5qYaERxAOB0w I70w== MIME-Version: 1.0 X-Received: by 10.70.38.134 with SMTP id g6mr15686876pdk.135.1406261929897; Thu, 24 Jul 2014 21:18:49 -0700 (PDT) Sender: jdavidlists@gmail.com Received: by 10.70.22.34 with HTTP; Thu, 24 Jul 2014 21:18:49 -0700 (PDT) Date: Fri, 25 Jul 2014 00:18:49 -0400 X-Google-Sender-Auth: ao8S-hi8PFgzKKZy4VbWpPcQG3c Message-ID: Subject: FreeBSD NFS client vs. Linux server From: J David To: "freebsd-fs@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2014 04:18:50 -0000 Hello, We observed the following odd behavior on a FreeBSD 9.2 client connected to a Linux NFS server. Basically, the FreeBSD user's additional group memberships are not being honored: $ ls -l -rw-rw---- 1 test2 test2 19188391 Jul 25 03:53 error_log $ id -a uid=123(test1) gid=123 groups=123,456(test2) $ tail -1 error_log tail: error_log: Permission denied $ touch x $ ls -l x -rw-rw-r-- 1 test1 test1 0 Jul 25 03:54 x $ chgrp test2 x chgrp: x: Operation not permitted This is just NFSv3. The FreeBSD client mounts with only the nosuid option. The Linux server's export options are rw,async,no_subtree_check. Does anyone what's going on here? The same test works fine with FreeBSD<->FreeBSD and Linux<->Linux. Unfortunately I can't figure out if this is on the FreeBSD side, the Linux side, or (most likely) just a config error on my part. Thanks for any pointers! From owner-freebsd-fs@FreeBSD.ORG Fri Jul 25 04:43:17 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D0E9FA3A; Fri, 25 Jul 2014 04:43:17 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 978732FD7; Fri, 25 Jul 2014 04:43:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=Ie+AdQPg5R1l/u9a7a+Pxk+x7qwupq/UOFfWXLW5iZI=; b=Xx7CAFPA3VJCpY45vOlypNGfQRkQp8CkgofxoWDHQvjVEsHjIu79PQw2l/564cSq4j6V9+q3a7UfA5XX4a49bm9d2uXv/a4PeYc7BnRIyHi52nbpP/cdQ/iAW+fKRi+Zb3No4ISe6YRp71SwNkhnwXO10w7MYqiPwndKZv2yrtM=; Received: from localhost.lerctr.org ([127.0.0.1]:10110 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.83 (FreeBSD)) (envelope-from ) id 1XAXLa-000PSa-Qu; Thu, 24 Jul 2014 23:43:14 -0500 Received: from 107-128-180-255.lightspeed.austtx.sbcglobal.net ([107.128.180.255]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Thu, 24 Jul 2014 23:43:10 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Thu, 24 Jul 2014 23:43:10 -0500 From: Larry Rosenman To: Allan Jude , Mark Martinec Subject: Re: zfs send/recv: STILL invalid Backup Stream In-Reply-To: <53D1AB3D.1060205@freebsd.org> References: <62315eb454a95db636b7764aad3c0f9b@thebighonker.lerctr.org> <53D1448C.40908@freebsd.org> <53D15438.6040105@freebsd.org> <0a9cd451c3b4304d2b9d899fcb3decc3@thebighonker.lerctr.org> <3d2aac84d962a703fbf56a864ba5f19c@mailbox.ijs.si> <9eef3e7e964cb33fd163cc2f9300f326@mailbox.ijs.si> <53D1AB3D.1060205@freebsd.org> Message-ID: <33c478c3120c21d6cb9c325cc1119cf8@thebighonker.lerctr.org> X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.0.1 X-Spam-Score: -2.9 (--) X-LERCTR-Spam-Score: -2.9 (--) X-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 X-LERCTR-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 Cc: Freebsd fs , freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2014 04:43:18 -0000 On 2014-07-24 19:56, Allan Jude wrote: > On 2014-07-24 20:46, Mark Martinec wrote: >> 2014-07-25 01:36 Larry Rosenman wrote: >> >>> #!/bin/sh >>> DATE=`date "+%Y-%m-%d"` >>> #DATE2=2013-03-24 >>> #DATE2=`date -v "-1d" "+%Y-%m-%d"` >>> # snap the source >>> ssh root@tbh.lerctr.org zfs snapshot -r zroot@${DATE} >>> # zfs copy the source to here. >>> ssh root@tbh.lerctr.org "zfs send -v -R zroot@${DATE} | \ >>> ssh home.lerctr.org \"zfs recv -F -u -v -d zroot/backups/TBH2\"" >> >> Btw, this double-ssh looks awkward, why not just: >> >> ssh root@tbh.lerctr.org "zfs send ..." | zfs recv ... >> >> or better yet: >> >> ssh root@tbh.lerctr.org "zfs send ..." | mbuffer -m 16M | zfs recv >> ... >> >> (The misc/mbuffer compensates for bursty zfs reads and writes. >> A note to myself: I should suggest to Allan to add mbuffer >> in a pipe as used in sysutils/zxfer, instead of patching zxfer >> for our local use :) >> >> Mark >> _______________________________________________ >> freebsd-current@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-current >> To unsubscribe, send any mail to >> "freebsd-current-unsubscribe@freebsd.org" > > zxfer can already do this, with the -D option > I actually use misc/clpbar and get a progress bar as well > > -D 'bar -s %%size%% -bl 1m -bs 128m' > > or in your case: -D 'mbuffer -m 16M' Ok, I did the mbuffer trick, and the SEND side is where the memory issue is: borg.lerctr.org /home/ler/bin $ tail zfs-send.log 23:28:12 15.7G zroot/home@2014-07-24_22:56 23:28:13 15.7G zroot/home@2014-07-24_22:56 23:28:14 15.7G zroot/home@2014-07-24_22:56 23:28:15 15.7G zroot/home@2014-07-24_22:56 23:28:16 15.7G zroot/home@2014-07-24_22:56 23:28:17 15.7G zroot/home@2014-07-24_22:56 23:28:18 15.7G zroot/home@2014-07-24_22:56 23:28:19 15.7G zroot/home@2014-07-24_22:56 23:28:20 15.8G zroot/home@2014-07-24_22:56 Write failed: Cannot allocate memory borg.lerctr.org /home/ler/bin $ borg.lerctr.org /home/ler/bin $ tail zfs-recv.log cannot receive new filesystem stream: invalid backup stream borg.lerctr.org /home/ler/bin $ borg.lerctr.org /home/ler/bin $ cat backup-TBH-ZFS-initial.sh #!/bin/sh DATE=`date "+%Y-%m-%d_%H:%M"` #DATE2=2013-03-24 #DATE2=`date -v "-1d" "+%Y-%m-%d"` # snap the source ssh root@tbh.lerctr.org zfs snapshot -r zroot@${DATE} # zfs copy the source to here. ssh root@tbh.lerctr.org 2>zfs-send.log "zfs send -v -R zroot@${DATE} " | \ mbuffer -m 16M 2>mbuffer.log | \ zfs recv -F -u -v -d zroot/backups/TBH3 2>zfs-recv.log # make sure we NEVER allow the backup stuff to automount. /sbin/zfs list -H -t filesystem -r zroot/backups/TBH3| \ awk '{printf "/sbin/zfs set canmount=noauto %s\n",$1}' | sh borg.lerctr.org /home/ler/bin $ borg.lerctr.org /home/ler/bin $ ssh tbh vmstat -z ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP UMA Kegs: 384, 0, 202, 8, 202, 0, 0 UMA Zones: 1664, 0, 202, 0, 202, 0, 0 UMA Slabs: 80, 0, 363320, 44080, 2348572, 0, 0 UMA RCntSlabs: 88, 0, 4484, 16, 4484, 0, 0 UMA Hash: 256, 0, 7, 83, 82, 0, 0 4 Bucket: 32, 0, 1911, 36589, 5255345, 0, 0 6 Bucket: 48, 0, 9406, 3542, 878903, 0, 0 8 Bucket: 64, 0, 42, 3554, 298443, 11, 0 12 Bucket: 96, 0, 93, 2572, 166067, 0, 0 16 Bucket: 128, 0, 30447, 987, 301403, 0, 0 32 Bucket: 256, 0, 352, 4658, 1157489, 50, 0 64 Bucket: 512, 0, 13669, 995, 1113780,268080, 0 128 Bucket: 1024, 0, 3646, 822, 524977, 0, 0 256 Bucket: 2048, 0, 3648, 114, 482627,59834, 0 vmem btag: 56, 0, 208448, 49779, 758362,1821, 0 VM OBJECT: 256, 0, 98960, 1570, 4440323, 0, 0 RADIX NODE: 144, 0, 235166, 29650,22669417, 0, 0 MAP: 240, 0, 3, 61, 3, 0, 0 KMAP ENTRY: 128, 0, 10, 269, 10, 0, 0 MAP ENTRY: 128, 0, 11828, 24442,12463199, 0, 0 VMSPACE: 448, 0, 103, 311, 96786, 0, 0 fakepg: 104, 0, 0, 0, 0, 0, 0 mt_zone: 4112, 0, 368, 0, 368, 0, 0 16: 16, 0, 264961, 9382,56032463, 0, 0 32: 32, 0, 155626, 2874,55177821, 0, 0 64: 64, 0, 123597, 672111,53838666, 0, 0 128: 128, 0, 159107, 58792,82084329, 0, 0 256: 256, 0, 97004, 223276,48661927, 0, 0 512: 512, 0, 737, 3991,33323191, 0, 0 1024: 1024, 0, 1367, 1389, 2330023, 0, 0 2048: 2048, 0, 271, 449,16353342, 0, 0 4096: 4096, 0, 28044, 27448, 997566, 0, 0 SLEEPQUEUE: 80, 0, 613, 658, 613, 0, 0 64 pcpu: 8, 0, 1884, 1188, 1884, 0, 0 Files: 80, 0, 861, 1639, 5410116, 0, 0 rl_entry: 40, 0, 211, 1889, 211, 0, 0 TURNSTILE: 136, 0, 613, 307, 613, 0, 0 umtx pi: 96, 0, 0, 0, 0, 0, 0 MAC labels: 40, 0, 0, 0, 0, 0, 0 PROC: 1208, 0, 138, 114, 97778, 0, 0 THREAD: 1168, 0, 517, 95, 2429, 0, 0 cpuset: 72, 0, 270, 280, 426, 0, 0 cyclic_id_cache: 64, 0, 0, 0, 0, 0, 0 audit_record: 1248, 0, 0, 0, 0, 0, 0 mbuf_packet: 256, 6519810, 4080, 3257, 9746764, 0, 0 mbuf: 256, 6519810, 1025, 3278,259201769, 0, 0 mbuf_cluster: 2048, 1018718, 7337, 13, 7337, 0, 0 mbuf_jumbo_page: 4096, 509359, 0, 809,45525783, 0, 0 mbuf_jumbo_9k: 9216, 150921, 0, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 84893, 0, 0, 0, 0, 0 mbuf_ext_refcnt: 4, 0, 0, 2510, 116647, 0, 0 dtrace_state_cache: 4096, 0, 0, 0, 0, 0, 0 g_bio: 248, 0, 0, 1680,48127406, 0, 0 ttyinq: 160, 0, 120, 630, 2460, 0, 0 DMAR_MAP_ENTRY: 120, 0, 0, 0, 0, 0, 0 ttyoutq: 256, 0, 64, 671, 1279, 0, 0 ata_request: 336, 0, 0, 242, 42892, 0, 0 vtnet_tx_hdr: 24, 0, 0, 0, 0, 0, 0 cryptop: 88, 0, 0, 0, 0, 0, 0 cryptodesc: 72, 0, 0, 0, 0, 0, 0 FPU_save_area: 512, 0, 0, 0, 0, 0, 0 taskq_zone: 48, 0, 0, 3071, 250215, 0, 0 taskq_zone: 48, 0, 0, 0, 0, 0, 0 VNODE: 472, 0, 113296, 62392, 2709794, 0, 0 VNODEPOLL: 112, 0, 355, 660, 357, 0, 0 BUF TRIE: 144, 0, 0, 105354, 0, 0, 0 S VFS Cache: 108, 0, 118644, 56881, 2428046, 0, 0 STS VFS Cache: 148, 0, 0, 0, 0, 0, 0 L VFS Cache: 328, 0, 1110, 144894, 804385, 0, 0 LTS VFS Cache: 368, 0, 0, 0, 0, 0, 0 NAMEI: 1024, 0, 0, 172,30294048, 0, 0 DIRHASH: 1024, 0, 0, 0, 0, 0, 0 NCLNODE: 528, 0, 0, 0, 0, 0, 0 Mountpoints: 816, 0, 11, 19, 11, 0, 0 range_seg_cache: 64, 0, 59688, 6094, 4438296, 0, 0 zio_cache: 920, 0, 1, 42591,138793685, 0, 0 zio_link_cache: 48, 0, 0, 44073,97697152, 0, 0 zio_buf_512: 512, 0, 411934, 265850, 1785999, 0, 0 zio_data_buf_512: 512, 0, 154726, 20594, 797483, 0, 0 zio_buf_1024: 1024, 0, 3061, 175, 74166, 0, 0 zio_data_buf_1024: 1024, 0, 69831, 9029, 333451, 0, 0 zio_buf_1536: 1536, 0, 1372, 96, 93985, 0, 0 zio_data_buf_1536: 1536, 0, 29821, 3103, 190819, 0, 0 zio_buf_2048: 2048, 0, 523, 47, 281620, 0, 0 zio_data_buf_2048: 2048, 0, 17707, 2389, 128398, 0, 0 zio_buf_2560: 2560, 0, 390, 28, 22269, 0, 0 zio_data_buf_2560: 2560, 0, 12713, 28, 108546, 0, 0 zio_buf_3072: 3072, 0, 182, 22, 7255, 0, 0 zio_data_buf_3072: 3072, 0, 8798, 34, 79372, 0, 0 zio_buf_3584: 3584, 0, 206, 11, 50516, 0, 0 zio_data_buf_3584: 3584, 0, 6275, 11, 64145, 0, 0 zio_buf_4096: 4096, 0, 1552, 416,18817937, 0, 0 zio_data_buf_4096: 4096, 0, 4625, 6, 59910, 0, 0 zio_buf_5120: 5120, 0, 133, 9, 8048, 0, 0 zio_data_buf_5120: 5120, 0, 6326, 11, 77387, 0, 0 zio_buf_6144: 6144, 0, 86, 11, 4503, 0, 0 zio_data_buf_6144: 6144, 0, 4632, 13, 58064, 0, 0 zio_buf_7168: 7168, 0, 61, 13, 48012, 0, 0 zio_data_buf_7168: 7168, 0, 3090, 13, 44010, 0, 0 zio_buf_8192: 8192, 0, 24, 29, 1857848, 0, 0 zio_data_buf_8192: 8192, 0, 2418, 11, 42210, 0, 0 zio_buf_10240: 10240, 0, 49, 7, 3311, 0, 0 zio_data_buf_10240: 10240, 0, 3611, 15, 59310, 0, 0 zio_buf_12288: 12288, 0, 44, 14, 999494, 0, 0 zio_data_buf_12288: 12288, 0, 2340, 13, 53847, 0, 0 zio_buf_14336: 14336, 0, 23, 10, 4184, 0, 0 zio_data_buf_14336: 14336, 0, 1852, 15, 36058, 0, 0 zio_buf_16384: 16384, 0, 72595, 164, 3891311, 0, 0 zio_data_buf_16384: 16384, 0, 1707, 11, 33236, 0, 0 zio_buf_20480: 20480, 0, 201, 261, 832993, 0, 0 zio_data_buf_20480: 20480, 0, 1932, 12, 43873, 0, 0 zio_buf_24576: 24576, 0, 16, 11, 365481, 0, 0 zio_data_buf_24576: 24576, 0, 1852, 13, 31773, 0, 0 zio_buf_28672: 28672, 0, 9, 25, 731759, 0, 0 zio_data_buf_28672: 28672, 0, 1169, 10, 27692, 0, 0 zio_buf_32768: 32768, 0, 9, 14, 112680, 0, 0 zio_data_buf_32768: 32768, 0, 962, 13, 20775, 0, 0 zio_buf_36864: 36864, 0, 5, 14, 112387, 0, 0 zio_data_buf_36864: 36864, 0, 701, 13, 15139, 0, 0 zio_buf_40960: 40960, 0, 4, 11, 115602, 0, 0 zio_data_buf_40960: 40960, 0, 720, 10, 12360, 0, 0 zio_buf_45056: 45056, 0, 48, 12, 102919, 0, 0 zio_data_buf_45056: 45056, 0, 434, 14, 10533, 0, 0 zio_buf_49152: 49152, 0, 188, 12, 113099, 0, 0 zio_data_buf_49152: 49152, 0, 373, 10, 11797, 0, 0 zio_buf_53248: 53248, 0, 24, 13, 157215, 0, 0 zio_data_buf_53248: 53248, 0, 347, 15, 7265, 0, 0 zio_buf_57344: 57344, 0, 1, 11, 154125, 0, 0 zio_data_buf_57344: 57344, 0, 274, 14, 7110, 0, 0 zio_buf_61440: 61440, 0, 2, 10, 163599, 0, 0 zio_data_buf_61440: 61440, 0, 180, 11, 5565, 0, 0 zio_buf_65536: 65536, 0, 1, 11, 186705, 0, 0 zio_data_buf_65536: 65536, 0, 213, 11, 7273, 0, 0 zio_buf_69632: 69632, 0, 2, 14, 145850, 0, 0 zio_data_buf_69632: 69632, 0, 176, 8, 5232, 0, 0 zio_buf_73728: 73728, 0, 1, 13, 136031, 0, 0 zio_data_buf_73728: 73728, 0, 127, 12, 4816, 0, 0 zio_buf_77824: 77824, 0, 1, 22, 162862, 0, 0 zio_data_buf_77824: 77824, 0, 136, 14, 4207, 0, 0 zio_buf_81920: 81920, 0, 0, 149, 195810, 0, 0 zio_data_buf_81920: 81920, 0, 99, 15, 4936, 0, 0 zio_buf_86016: 86016, 0, 1, 216, 150971, 0, 0 zio_data_buf_86016: 86016, 0, 103, 14, 33989, 0, 0 zio_buf_90112: 90112, 0, 1, 120, 75823, 0, 0 zio_data_buf_90112: 90112, 0, 145, 11, 3517, 0, 0 zio_buf_94208: 94208, 0, 0, 20, 72296, 0, 0 zio_data_buf_94208: 94208, 0, 149, 11, 3315, 0, 0 zio_buf_98304: 98304, 0, 0, 19, 68257, 0, 0 zio_data_buf_98304: 98304, 0, 79, 11, 4092, 0, 0 zio_buf_102400: 102400, 0, 1, 19, 62463, 0, 0 zio_data_buf_102400: 102400, 0, 92, 12, 2940, 0, 0 zio_buf_106496: 106496, 0, 0, 15, 203444, 0, 0 zio_data_buf_106496: 106496, 0, 62, 14, 2869, 0, 0 zio_buf_110592: 110592, 0, 0, 10, 184072, 0, 0 zio_data_buf_110592: 110592, 0, 152, 12, 2463, 0, 0 zio_buf_114688: 114688, 0, 1, 12, 65538, 0, 0 zio_data_buf_114688: 114688, 0, 115, 15, 3609, 0, 0 zio_buf_118784: 118784, 0, 0, 13, 73524, 0, 0 zio_data_buf_118784: 118784, 0, 57, 12, 2092, 0, 0 zio_buf_122880: 122880, 0, 0, 23, 101816, 0, 0 zio_data_buf_122880: 122880, 0, 46, 15, 2131, 0, 0 zio_buf_126976: 126976, 0, 0, 13, 99774, 0, 0 zio_data_buf_126976: 126976, 0, 46, 15, 1927, 0, 0 zio_buf_131072: 131072, 0, 21, 97, 715685, 0, 0 zio_data_buf_131072: 131072, 0, 64623, 8, 1133068, 0, 0 lz4_ctx: 16384, 0, 0, 38, 2874956, 0, 0 sa_cache: 80, 0, 113220, 62680, 2719738, 0, 0 dnode_t: 744, 0, 369347, 391598, 903612, 0, 0 dmu_buf_impl_t: 224, 0, 459185, 519692, 3494852, 0, 0 arc_buf_hdr_t: 216, 0, 697777, 122573, 4738762, 0, 0 arc_buf_t: 72, 0, 541440, 121695, 5738782, 0, 0 zil_lwb_cache: 192, 0, 7, 5073, 90526, 0, 0 zfs_znode_cache: 368, 0, 113220, 62340, 2709584, 0, 0 pipe: 744, 0, 113, 177, 84427, 0, 0 procdesc: 128, 0, 0, 0, 0, 0, 0 ksiginfo: 112, 0, 1133, 2507,10649838, 0, 0 itimer: 352, 0, 1, 32, 1, 0, 0 KNOTE: 128, 0, 379, 1388,26313314, 0, 0 socket: 696, 523340, 243, 317, 827216, 0, 0 unpcb: 240, 523344, 152, 648, 171731, 0, 0 ipq: 56, 31879, 0, 2059, 385, 0, 0 udp_inpcb: 392, 523340, 19, 421, 567701, 0, 0 udpcb: 16, 523586, 19, 2993, 567701, 0, 0 tcp_inpcb: 392, 523340, 73, 457, 87773, 0, 0 tcpcb: 1024, 523340, 67, 245, 87773, 0, 0 tcptw: 88, 27810, 6, 1254, 29354, 0, 0 syncache: 160, 15375, 0, 800, 57377, 0, 0 hostcache: 136, 15370, 134, 620, 4951, 0, 0 tcpreass: 40, 63700, 0, 3100, 13421, 0, 0 sackhole: 32, 0, 0, 2125, 18903, 0, 0 sctp_ep: 1408, 523340, 0, 0, 0, 0, 0 sctp_asoc: 2352, 40000, 0, 0, 0, 0, 0 sctp_laddr: 48, 80012, 0, 1328, 8, 0, 0 sctp_raddr: 728, 80000, 0, 0, 0, 0, 0 sctp_chunk: 136, 400026, 0, 0, 0, 0, 0 sctp_readq: 104, 400026, 0, 0, 0, 0, 0 sctp_stream_msg_out: 104, 400026, 0, 0, 0, 0, 0 sctp_asconf: 40, 400000, 0, 0, 0, 0, 0 sctp_asconf_ack: 48, 400060, 0, 0, 0, 0, 0 udplite_inpcb: 392, 523340, 0, 0, 0, 0, 0 ripcb: 392, 523340, 2, 58, 3, 0, 0 rtentry: 200, 0, 32, 408, 34, 0, 0 selfd: 56, 0, 356, 2200,137062686, 0, 0 SWAPMETA: 288, 2037438, 0, 0, 0, 0, 0 borg.lerctr.org /home/ler/bin $ last pid: 97813; load averages: 7.79, 8.41, 8.24 up 1+11:45:45 23:42:11 118 processes: 7 running, 109 sleeping, 2 stopped Mem: 982M Active, 1172M Inact, 13G Wired, 148M Cache, 637M Free ARC: 11G Total, 4394M MFU, 5813M MRU, 1029K Anon, 181M Header, 469M Other Swap: 18G Total, 18G Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 97782 boinc 1 155 i31 91992K 26008K CPU1 1 0:40 100.00% wcgrid_f 96977 boinc 1 155 i31 375M 165M STOP 4 11:30 61.87% wcgrid_c 89981 boinc 1 155 i31 99308K 63248K CPU5 5 91:25 51.76% wcgrid_f 92270 boinc 1 155 i31 98M 64556K CPU3 3 62:55 50.88% wcgrid_f 92269 boinc 1 155 i31 99372K 63324K CPU4 4 63:49 50.20% wcgrid_f 92271 boinc 1 155 i31 99340K 63288K CPU0 0 64:00 48.19% wcgrid_f 89772 boinc 1 155 i31 99888K 63832K CPU7 7 92:52 41.36% wcgrid_f 89781 boinc 1 155 i31 99732K 63588K nanslp 5 94:54 38.18% wcgrid_f 85619 root 1 52 0 181M 97232K select 3 2:56 21.19% perl5 97806 root 1 28 0 44600K 8028K kqread 0 0:00 0.78% indexer- 837 clamav 3 20 0 486M 399M uwait 5 12:42 0.20% clamd 70563 boinc 2 155 i31 101M 51436K nanslp 7 202:02 0.00% setiatho 68437 boinc 2 155 i31 105M 52516K nanslp 3 185:45 0.00% setiatho 83114 boinc 2 155 i31 105M 51880K nanslp 4 103:15 0.00% setiatho 85300 boinc 2 155 i31 101M 51460K nanslp 4 78:55 0.00% setiatho 86148 boinc 2 155 i31 105M 51492K nanslp 3 71:54 0.00% setiatho 87150 boinc 2 155 i31 95112K 46940K nanslp 4 30:37 0.00% setiatho 87451 boinc 2 155 i31 95112K 46224K nanslp 3 28:07 0.00% setiatho borg.lerctr.org /home/ler/bin $ Where to now? -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-fs@FreeBSD.ORG Fri Jul 25 05:13:54 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B0327153 for ; Fri, 25 Jul 2014 05:13:54 +0000 (UTC) Received: from mail-ig0-x22f.google.com (mail-ig0-x22f.google.com [IPv6:2607:f8b0:4001:c05::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 81CA8220F for ; Fri, 25 Jul 2014 05:13:54 +0000 (UTC) Received: by mail-ig0-f175.google.com with SMTP id uq10so336023igb.2 for ; Thu, 24 Jul 2014 22:13:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:content-type; bh=1XXNFRyJgyEzoqKWSe/fVnB7ycOztF5LHpo9OQVOfBI=; b=dBo/ytEd843DF+oayooGTg10dKjXjThcYafmsObu/3K8trDDmvLBhv0lw9tLgxthEy E6ydfwgtEepi+jLNdm7Tk/smUA5lYQaTEDYVJfW2zVFJIGP/SXNqC0mkFOENj+E2tYB0 oq7D4pEWnx7QOEJbKhcnqQlKQHwltYh2qEpqyA/gBwE4D8XggmwgftPoYaC65fiIr0fa Qy6wcsbyT2GhaCijA11803oa9YaQos6rjmq4ZV3FEhfOQWHTwxmTIKpsNt12GSJPzbJa F92/v6uXGEEVdGq2oNR05GMsK8lxhhYcisNxLvgxhASa2YD/6AHL7buzOiqMG1IEZVs+ y+Aw== MIME-Version: 1.0 X-Received: by 10.43.55.74 with SMTP id vx10mr519378icb.87.1406265233727; Thu, 24 Jul 2014 22:13:53 -0700 (PDT) Sender: jdavidlists@gmail.com Received: by 10.43.65.203 with HTTP; Thu, 24 Jul 2014 22:13:53 -0700 (PDT) In-Reply-To: References: Date: Fri, 25 Jul 2014 01:13:53 -0400 X-Google-Sender-Auth: 6Bh9Psxq-FdL9gPzS9HIXo2Z_5Y Message-ID: Subject: Re: FreeBSD NFS client vs. Linux server From: J David To: "freebsd-fs@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2014 05:13:54 -0000 On Fri, Jul 25, 2014 at 12:18 AM, J David wrote: > We observed the following odd behavior on a FreeBSD 9.2 client > connected to a Linux NFS server. Basically, the FreeBSD user's > additional group memberships are not being honored: This turned out to be --manage-gids being set behind the scenes by default on the Linux side, causing it to throw away the group list provided by the FreeBSD client. Once that was disabled, it worked as expected. Thanks! From owner-freebsd-fs@FreeBSD.ORG Fri Jul 25 09:58:56 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1DD3AB6E; Fri, 25 Jul 2014 09:58:56 +0000 (UTC) Received: from mail.ijs.si (mail.ijs.si [IPv6:2001:1470:ff80::25]) by mx1.freebsd.org (Postfix) with ESMTP id B5A882C1D; Fri, 25 Jul 2014 09:58:55 +0000 (UTC) Received: from amavis-proxy-ori.ijs.si (localhost [IPv6:::1]) by mail.ijs.si (Postfix) with ESMTP id 3hKQq157tZzRG; Fri, 25 Jul 2014 11:58:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ijs.si; h= user-agent:message-id:references:in-reply-to:organization :subject:subject:from:from:date:date:content-transfer-encoding :content-type:content-type:mime-version:received:received :received:received; s=jakla2; t=1406282329; x=1408874330; bh=iQG sxwTE3CG9DTK8vqPZl69YvnECUv5Skg+aJftyidM=; b=DlBaoJr/B8gAx6igCH9 S1swqgdELa8n2Xyiiam14FPGDGeyz48b0MgRsmtYpaVrvP7B5GjLcBzxKV1Vpl4f fn4cwsopo9d1B7c4hKgIA0QYAw2vAVHps6BJs9IJpT1/MGsfDFjNkXSnaWtuHof4 yO1kbChdSLB7OwpQqBiQyT+s= X-Virus-Scanned: amavisd-new at ijs.si Received: from mail.ijs.si ([IPv6:::1]) by amavis-proxy-ori.ijs.si (mail.ijs.si [IPv6:::1]) (amavisd-new, port 10012) with ESMTP id gmjTzVhItnlS; Fri, 25 Jul 2014 11:58:49 +0200 (CEST) Received: from mildred.ijs.si (mailbox.ijs.si [IPv6:2001:1470:ff80::143:1]) by mail.ijs.si (Postfix) with ESMTP; Fri, 25 Jul 2014 11:58:48 +0200 (CEST) Received: from neli.ijs.si (neli.ijs.si [IPv6:2001:1470:ff80:88:21c:c0ff:feb1:8c91]) by mildred.ijs.si (Postfix) with ESMTP id 3hKQpw2cQ8z5W; Fri, 25 Jul 2014 11:58:48 +0200 (CEST) Received: from sleepy.ijs.si ([2001:1470:ff80:e001::1:1]) by neli.ijs.si with HTTP (HTTP/1.1 POST); Fri, 25 Jul 2014 11:58:48 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Fri, 25 Jul 2014 11:58:48 +0200 From: Mark Martinec To: Larry Rosenman Subject: Re: zfs send/recv: STILL invalid Backup Stream Organization: J. Stefan Institute In-Reply-To: <33c478c3120c21d6cb9c325cc1119cf8@thebighonker.lerctr.org> References: <62315eb454a95db636b7764aad3c0f9b@thebighonker.lerctr.org> <53D1448C.40908@freebsd.org> <53D15438.6040105@freebsd.org> <0a9cd451c3b4304d2b9d899fcb3decc3@thebighonker.lerctr.org> <3d2aac84d962a703fbf56a864ba5f19c@mailbox.ijs.si> <9eef3e7e964cb33fd163cc2f9300f326@mailbox.ijs.si> <53D1AB3D.1060205@freebsd.org> <33c478c3120c21d6cb9c325cc1119cf8@thebighonker.lerctr.org> Message-ID: <60845088153247fa31c50c9f52ef72cb@mailbox.ijs.si> X-Sender: Mark.Martinec+freebsd@ijs.si User-Agent: Roundcube Webmail/1.0.1 Cc: Freebsd fs , freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2014 09:58:56 -0000 > On 2014-07-24 19:56, Allan Jude wrote: >>> or better yet: >>> ssh root@tbh.lerctr.org "zfs send ..." | mbuffer -m 16M | zfs recv >>> ... >>> (The misc/mbuffer compensates for bursty zfs reads and writes. >>> A note to myself: I should suggest to Allan to add mbuffer >>> in a pipe as used in sysutils/zxfer, instead of patching zxfer >>> for our local use :) >> >> zxfer can already do this, with the -D option >> I actually use misc/clpbar and get a progress bar as well >> >> -D 'bar -s %%size%% -bl 1m -bs 128m' >> >> or in your case: -D 'mbuffer -m 16M' Thank you Allan! It shows it's been a while since the last time I inspected the guts of zxfer. 2014-07-25 06:43 Larry Rosenman wrote: > Ok, I did the mbuffer trick, and the SEND side is where the memory > issue is: > borg.lerctr.org /home/ler/bin $ tail zfs-send.log > 23:28:12 15.7G zroot/home@2014-07-24_22:56 > 23:28:13 15.7G zroot/home@2014-07-24_22:56 > 23:28:14 15.7G zroot/home@2014-07-24_22:56 > 23:28:15 15.7G zroot/home@2014-07-24_22:56 > 23:28:16 15.7G zroot/home@2014-07-24_22:56 > 23:28:17 15.7G zroot/home@2014-07-24_22:56 > 23:28:18 15.7G zroot/home@2014-07-24_22:56 > 23:28:19 15.7G zroot/home@2014-07-24_22:56 > 23:28:20 15.8G zroot/home@2014-07-24_22:56 > Write failed: Cannot allocate memory > borg.lerctr.org /home/ler/bin $ Good information. So the "Write failed: Cannot allocate memory" on the send side is what is causing the "invalid backup stream" on the receiving side. > borg.lerctr.org /home/ler/bin $ tail zfs-recv.log > cannot receive new filesystem stream: invalid backup stream > borg.lerctr.org /home/ler/bin $ > > borg.lerctr.org /home/ler/bin $ cat backup-TBH-ZFS-initial.sh > #!/bin/sh > DATE=`date "+%Y-%m-%d_%H:%M"` > #DATE2=2013-03-24 > #DATE2=`date -v "-1d" "+%Y-%m-%d"` > # snap the source > ssh root@tbh.lerctr.org zfs snapshot -r zroot@${DATE} > # zfs copy the source to here. > ssh root@tbh.lerctr.org 2>zfs-send.log "zfs send -v -R zroot@${DATE} " > | \ > mbuffer -m 16M 2>mbuffer.log | \ > zfs recv -F -u -v -d zroot/backups/TBH3 2>zfs-recv.log > # make sure we NEVER allow the backup stuff to automount. > /sbin/zfs list -H -t filesystem -r zroot/backups/TBH3| \ > awk '{printf "/sbin/zfs set canmount=noauto %s\n",$1}' | sh > borg.lerctr.org /home/ler/bin $ > > borg.lerctr.org /home/ler/bin $ ssh tbh vmstat -z > ITEM SIZE LIMIT USED FREE REQ FAIL > SLEEP [...] > Where to now? Don't know, I'd guess some network-related memory limit is being hit on the sending site. Why not try to decouple the 'zfs send' from a network copy and ssh: Login to a remote side, do a 'zfs send' to some local temporary file there, then feed that file to ssh and send it over to a home host, where it can be piped into some simple program like md5 or 'wc -c' instead of a zfs recv. Mark From owner-freebsd-fs@FreeBSD.ORG Fri Jul 25 13:49:41 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 96FC0B2C; Fri, 25 Jul 2014 13:49:41 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 638A32334; Fri, 25 Jul 2014 13:49:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=6ENH9mUVv1yygVqRx6MS8w1GgLeLPrgpqCCZzpfIDEc=; b=ccCvOuHzn8KxreNmQ4OY7FIaOZR0xft5XJa+ahIZ2kiSvdU0PZUnH968/tOdB0NDTYCk7I1K+Cpx16Lz2djwigkPVJsVAstQo1pADv87QD/HxG5z1isjZlg4MW3SLpb34RT1+78NRMim4ySFgXCWIcrm8Uzy5WQPDyt3ZMrYvfQ=; Received: from localhost.lerctr.org ([127.0.0.1]:34152 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.83 (FreeBSD)) (envelope-from ) id 1XAfsR-0005EQ-9h; Fri, 25 Jul 2014 08:49:40 -0500 Received: from host.alcatel.com ([198.205.55.139]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Fri, 25 Jul 2014 08:49:39 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Fri, 25 Jul 2014 08:49:39 -0500 From: Larry Rosenman To: freebsd-current@freebsd.org, Freebsd fs Subject: Re: zfs send/recv: STILL invalid Backup Stream In-Reply-To: <20140725134125.GA19293@thebighonker.lerctr.org> References: <3d2aac84d962a703fbf56a864ba5f19c@mailbox.ijs.si> <9eef3e7e964cb33fd163cc2f9300f326@mailbox.ijs.si> <53D1AB3D.1060205@freebsd.org> <33c478c3120c21d6cb9c325cc1119cf8@thebighonker.lerctr.org> <60845088153247fa31c50c9f52ef72cb@mailbox.ijs.si> <20140725134125.GA19293@thebighonker.lerctr.org> Message-ID: <258d3f814c0d0a1159608d9cf09d65ad@thebighonker.lerctr.org> X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.0.1 X-Spam-Score: -2.9 (--) X-LERCTR-Spam-Score: -2.9 (--) X-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 X-LERCTR-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2014 13:49:41 -0000 On 2014-07-25 08:41, Larry Rosenman wrote: > On Fri, Jul 25, 2014 at 11:58:48AM +0200, Mark Martinec wrote: >> Don't know, I'd guess some network-related memory limit is being hit >> on the sending site. >> >> Why not try to decouple the 'zfs send' from a network copy and ssh: >> Login to a remote side, do a 'zfs send' to some local temporary file >> there, then feed that file to ssh and send it over to a home host, >> where it can be piped into some simple program like md5 or 'wc -c' >> instead of a zfs recv. >> > I think I've done that, but it sort of defeats the purpose, as the > stream > becomes a big file on disk. > > I'd like to get some help chasing what parameter(s) need to be fixed > here. > sysctl output and other things at: http://www.lerctr.org/~ler/FreeBSD/ > Can I get some ideas on: > 1) what MIGHT need tweaking > 2) would (k|d)trace help? > 3) what else could we find out what's being told no memory? I *CAN* give SSH/Root/etc access to BOTH boxes on request. What else do you need? -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-fs@FreeBSD.ORG Fri Jul 25 14:25:02 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D07C7D68; Fri, 25 Jul 2014 14:25:02 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9B02D276F; Fri, 25 Jul 2014 14:25:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=mRYBlUEmymsa5DztnNC3VdSB98M8ptMmMf7yXLHpIvU=; b=GIXle22RcDfJoOAO7nTQ/lA6s1F9+MvCvCIvwqmDqLd6WWCdzLxGkggh1Kl/spu7tcFDSTA5fn3zPTfxuhFj7zYXYXPyAoZq2JLnCPcM7NbMRUKUnqJ6D+xsjj3LR5cCHm0jHT0qrcPv63juo+6o1oda6v0+sCfsqUP6f3WhFJ4=; Received: from localhost.lerctr.org ([127.0.0.1]:51560 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.83 (FreeBSD)) (envelope-from ) id 1XAgQb-0006aM-Ez; Fri, 25 Jul 2014 09:25:01 -0500 Received: from host.alcatel.com ([198.205.55.139]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Fri, 25 Jul 2014 09:24:27 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Fri, 25 Jul 2014 09:24:27 -0500 From: Larry Rosenman To: Allan Jude Subject: Re: zfs send/recv: STILL invalid Backup Stream In-Reply-To: <53D263A6.5010907@freebsd.org> References: <3d2aac84d962a703fbf56a864ba5f19c@mailbox.ijs.si> <9eef3e7e964cb33fd163cc2f9300f326@mailbox.ijs.si> <53D1AB3D.1060205@freebsd.org> <33c478c3120c21d6cb9c325cc1119cf8@thebighonker.lerctr.org> <60845088153247fa31c50c9f52ef72cb@mailbox.ijs.si> <20140725134125.GA19293@thebighonker.lerctr.org> <258d3f814c0d0a1159608d9cf09d65ad@thebighonker.lerctr.org> <53D263A6.5010907@freebsd.org> Message-ID: <7d0d742431172d4d2e36a68d209a4597@thebighonker.lerctr.org> X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.0.1 X-Spam-Score: -2.9 (--) X-LERCTR-Spam-Score: -2.9 (--) X-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 X-LERCTR-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 Cc: Freebsd fs , freebsd-current@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2014 14:25:03 -0000 On 2014-07-25 09:03, Allan Jude wrote: > On 2014-07-25 09:49, Larry Rosenman wrote: >> On 2014-07-25 08:41, Larry Rosenman wrote: >>> On Fri, Jul 25, 2014 at 11:58:48AM +0200, Mark Martinec wrote: >>>> Don't know, I'd guess some network-related memory limit is being hit >>>> on the sending site. >>>> >>>> Why not try to decouple the 'zfs send' from a network copy and ssh: >>>> Login to a remote side, do a 'zfs send' to some local temporary file >>>> there, then feed that file to ssh and send it over to a home host, >>>> where it can be piped into some simple program like md5 or 'wc -c' >>>> instead of a zfs recv. >>>> >>> I think I've done that, but it sort of defeats the purpose, as the >>> stream >>> becomes a big file on disk. >>> >>> I'd like to get some help chasing what parameter(s) need to be fixed >>> here. >>> >> sysctl output and other things at: >> >> http://www.lerctr.org/~ler/FreeBSD/ >> >> >>> Can I get some ideas on: >>> 1) what MIGHT need tweaking >>> 2) would (k|d)trace help? >>> 3) what else could we find out what's being told no memory? >> >> I *CAN* give SSH/Root/etc access to BOTH boxes on request. >> >> What else do you need? >> >> > > Try just ssh into the 'source' host, and do: > zfs send -v -R zroot@${DATE} > /dev/null > > and see if it actually finishes, or if it runs out of memory again. that, of course, worked fine :( -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-fs@FreeBSD.ORG Fri Jul 25 13:42:01 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 08DDA21F; Fri, 25 Jul 2014 13:42:01 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 85AB922A9; Fri, 25 Jul 2014 13:42:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=In-Reply-To:Content-Type:MIME-Version:References:Message-ID:Subject:Cc:To:From:Date; bh=E5weJL052z47kqJELFthnwgG7wQCq1NgLd3e6WhqVHk=; b=cTgmAEMfST4/SiFngbv4WToLp/tZ7A+0obMTTAkEBgtG3w1GrE/tAe5WBRz6Tqm11doqgk3Kffa+Ev/jYXudJb7THnZWyyi7IQf89/ayoPSCXEREinHCeL6PNizALcAdzHKZOk9nMmqj5veHLyWXUlr6JSdhjvz6zQrAZWOU+GE=; Received: from thebighonker.lerctr.org ([192.147.25.65]:32493) by thebighonker.lerctr.org with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.83 (FreeBSD)) (envelope-from ) id 1XAfkV-00053T-2k; Fri, 25 Jul 2014 08:41:56 -0500 Date: Fri, 25 Jul 2014 08:41:25 -0500 From: Larry Rosenman To: freebsd-current@freebsd.org, Freebsd fs Subject: Re: zfs send/recv: STILL invalid Backup Stream Message-ID: <20140725134125.GA19293@thebighonker.lerctr.org> Mail-Followup-To: freebsd-current@freebsd.org, Freebsd fs , Mark Martinec References: <3d2aac84d962a703fbf56a864ba5f19c@mailbox.ijs.si> <9eef3e7e964cb33fd163cc2f9300f326@mailbox.ijs.si> <53D1AB3D.1060205@freebsd.org> <33c478c3120c21d6cb9c325cc1119cf8@thebighonker.lerctr.org> <60845088153247fa31c50c9f52ef72cb@mailbox.ijs.si> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <60845088153247fa31c50c9f52ef72cb@mailbox.ijs.si> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Score: -2.9 (--) X-LERCTR-Spam-Score: -2.9 (--) X-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 X-LERCTR-Spam-Report: SpamScore (-2.9/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, RP_MATCHES_RCVD=-0.001 X-Mailman-Approved-At: Fri, 25 Jul 2014 15:36:20 +0000 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2014 13:42:01 -0000 On Fri, Jul 25, 2014 at 11:58:48AM +0200, Mark Martinec wrote: > Don't know, I'd guess some network-related memory limit is being hit > on the sending site. > > Why not try to decouple the 'zfs send' from a network copy and ssh: > Login to a remote side, do a 'zfs send' to some local temporary file > there, then feed that file to ssh and send it over to a home host, > where it can be piped into some simple program like md5 or 'wc -c' > instead of a zfs recv. > I think I've done that, but it sort of defeats the purpose, as the stream becomes a big file on disk. I'd like to get some help chasing what parameter(s) need to be fixed here. Full sysctl -a (from LONG after the failure). kern.ostype: FreeBSD kern.osrelease: 10.0-STABLE kern.osrevision: 199506 kern.version: FreeBSD 10.0-STABLE #39 r269019M: Wed Jul 23 11:44:35 CDT 2014 root@thebighonker.lerctr.org:/usr/obj/usr/src/sys/GENERIC kern.maxvnodes: 350155 kern.maxproc: 21748 kern.maxfiles: 523339 kern.argmax: 262144 kern.securelevel: -1 kern.hostname: thebighonker.lerctr.org kern.hostid: 3151778009 kern.clockrate: { hz = 1000, tick = 1000, profhz = 8128, stathz = 127 } kern.posix1version: 200112 kern.ngroups: 1023 kern.job_control: 1 kern.saved_ids: 0 kern.boottime: { sec = 1406134586, usec = 684885 } Wed Jul 23 11:56:26 2014 kern.domainname: kern.osreldate: 1000711 kern.bootfile: /boot/kernel/kernel kern.maxfilesperproc: 470997 kern.maxprocperuid: 19573 kern.ipc.maxsockbuf: 2097152 kern.ipc.sockbuf_waste_factor: 8 kern.ipc.max_linkhdr: 16 kern.ipc.max_protohdr: 60 kern.ipc.max_hdr: 76 kern.ipc.max_datalen: 92 kern.ipc.maxmbufmem: 8345339904 kern.ipc.nmbclusters: 1018718 kern.ipc.nmbjumbop: 509359 kern.ipc.nmbjumbo9: 452763 kern.ipc.nmbjumbo16: 339572 kern.ipc.nmbufs: 6519810 kern.ipc.maxpipekva: 267948032 kern.ipc.pipekva: 1835008 kern.ipc.pipefragretry: 0 kern.ipc.pipeallocfail: 0 kern.ipc.piperesizefail: 0 kern.ipc.piperesizeallowed: 1 kern.ipc.msgmax: 16384 kern.ipc.msgmni: 40 kern.ipc.msgmnb: 2048 kern.ipc.msgtql: 40 kern.ipc.msgssz: 8 kern.ipc.msgseg: 2048 kern.ipc.semmni: 50 kern.ipc.semmns: 340 kern.ipc.semmnu: 150 kern.ipc.semmsl: 340 kern.ipc.semopm: 100 kern.ipc.semume: 50 kern.ipc.semusz: 632 kern.ipc.semvmx: 32767 kern.ipc.semaem: 16384 kern.ipc.shmmax: 536870912 kern.ipc.shmmin: 1 kern.ipc.shmmni: 192 kern.ipc.shmseg: 128 kern.ipc.shmall: 131072 kern.ipc.shm_use_phys: 0 kern.ipc.shm_allow_removed: 1 kern.ipc.soacceptqueue: 128 kern.ipc.numopensockets: 247 kern.ipc.maxsockets: 523340 kern.ipc.sendfile.readahead: 1 kern.dummy: 0 kern.ps_strings: 140737488351200 kern.usrstack: 140737488351232 kern.logsigexit: 1 kern.iov_max: 1024 kern.hostuuid: b74ca469-74a4-11e3-8eb3-00237d9e6e8a kern.cam.sort_io_queues: 1 kern.cam.boot_delay: 0 kern.cam.num_doneqs: 2 kern.cam.dflags: 0 kern.cam.debug_delay: 0 kern.cam.pmp.retry_count: 1 kern.cam.pmp.default_timeout: 30 kern.cam.pmp.hide_special: 1 kern.cam.cam_srch_hi: 0 kern.cam.scsi_delay: 5000 kern.cam.cd.poll_period: 3 kern.cam.cd.retry_count: 4 kern.cam.cd.timeout: 30000 kern.cam.cd.0.minimum_cmd_size: 6 kern.cam.ada.legacy_aliases: 1 kern.cam.ada.retry_count: 4 kern.cam.ada.default_timeout: 30 kern.cam.ada.send_ordered: 1 kern.cam.ada.spindown_shutdown: 1 kern.cam.ada.spindown_suspend: 1 kern.cam.ada.read_ahead: 1 kern.cam.ada.write_cache: 1 kern.cam.da.poll_period: 3 kern.cam.da.retry_count: 4 kern.cam.da.default_timeout: 60 kern.cam.da.send_ordered: 1 kern.cam.da.0.delete_method: NONE kern.cam.da.0.delete_max: 131072 kern.cam.da.0.minimum_cmd_size: 6 kern.cam.da.0.sort_io_queue: -1 kern.cam.da.0.error_inject: 0 kern.cam.da.2.delete_method: NONE kern.cam.da.2.delete_max: 131072 kern.cam.da.2.minimum_cmd_size: 6 kern.cam.da.2.sort_io_queue: -1 kern.cam.da.2.error_inject: 0 kern.cam.da.4.delete_method: NONE kern.cam.da.4.delete_max: 131072 kern.cam.da.4.minimum_cmd_size: 6 kern.cam.da.4.sort_io_queue: -1 kern.cam.da.4.error_inject: 0 kern.cam.da.1.delete_method: NONE kern.cam.da.1.delete_max: 131072 kern.cam.da.1.minimum_cmd_size: 6 kern.cam.da.1.sort_io_queue: -1 kern.cam.da.1.error_inject: 0 kern.cam.da.3.delete_method: NONE kern.cam.da.3.delete_max: 131072 kern.cam.da.3.minimum_cmd_size: 6 kern.cam.da.3.sort_io_queue: -1 kern.cam.da.3.error_inject: 0 kern.cam.da.5.delete_method: NONE kern.cam.da.5.delete_max: 131072 kern.cam.da.5.minimum_cmd_size: 6 kern.cam.da.5.sort_io_queue: -1 kern.cam.da.5.error_inject: 0 kern.cam.enc.emulate_array_devices: 1 kern.random.adaptors: dummy,yarrow kern.random.active_adaptor: yarrow kern.random.live_entropy_sources: kern.random.yarrow.gengateinterval: 10 kern.random.yarrow.bins: 10 kern.random.yarrow.fastthresh: 96 kern.random.yarrow.slowthresh: 128 kern.random.yarrow.slowoverthresh: 2 kern.random.sys.seeded: 1 kern.random.sys.harvest.ethernet: 1 kern.random.sys.harvest.point_to_point: 1 kern.random.sys.harvest.interrupt: 1 kern.random.sys.harvest.swi: 1 kern.vt.enable_altgr: 1 kern.vt.debug: 0 kern.vt.deadtimer: 15 kern.vt.suspendswitch: 1 kern.disks: da5 da4 da3 da2 da1 da0 cd0 kern.geom.dev.delete_max_sectors: 262144 kern.geom.disk.cd0.led: kern.geom.disk.da0.led: kern.geom.disk.da1.led: kern.geom.disk.da2.led: kern.geom.disk.da3.led: kern.geom.disk.da4.led: kern.geom.disk.da5.led: kern.geom.transient_maps: 1 kern.geom.transient_map_retries: 10 kern.geom.transient_map_hard_failures: 0 kern.geom.transient_map_soft_failures: 0 kern.geom.inflight_transient_maps: 0 kern.geom.confxml: FD RAID MD LABEL da5p3 3 r1w1e2 r1w1e1 gptid/f7a4d13a-74a3-11e3-8040-00237d9e6e8a 143558337536 512 0 3221767168 0 143558337536 280387378 0 0 da5p2 3 r1w1e1 r1w1e0 gpt/swap5 3221225472 512 0 541696 0 3221225472 6291456 0 0 da5p1 3 r0w0e0 r0w0e0 gptid/f76af2cc-74a3-11e3-8040-00237d9e6e8a 524288 512 0 17408 0 524288 1024 0 0 da5p1 3 r0w0e0 r0w0e0 gpt/gptboot5 524288 512 0 17408 0 524288 1024 0 0 da4p3 3 r1w1e2 r1w1e1 gptid/f7043103-74a3-11e3-8040-00237d9e6e8a 143558337536 512 0 3221767168 0 143558337536 280387378 0 0 da4p2 3 r1w1e1 r1w1e0 gpt/swap4 3221225472 512 0 541696 0 3221225472 6291456 0 0 da4p1 3 r0w0e0 r0w0e0 gptid/f6c16edb-74a3-11e3-8040-00237d9e6e8a 524288 512 0 17408 0 524288 1024 0 0 da4p1 3 r0w0e0 r0w0e0 gpt/gptboot4 524288 512 0 17408 0 524288 1024 0 0 da3p3 3 r1w1e2 r1w1e1 gptid/f6600fef-74a3-11e3-8040-00237d9e6e8a 143558337536 512 0 3221767168 0 143558337536 280387378 0 0 da3p2 3 r1w1e1 r1w1e0 gpt/swap3 3221225472 512 0 541696 0 3221225472 6291456 0 0 da3p1 3 r0w0e0 r0w0e0 gptid/f623a95d-74a3-11e3-8040-00237d9e6e8a 524288 512 0 17408 0 524288 1024 0 0 da3p1 3 r0w0e0 r0w0e0 gpt/gptboot3 524288 512 0 17408 0 524288 1024 0 0 da2p3 3 r1w1e2 r1w1e1 gptid/f5bd173e-74a3-11e3-8040-00237d9e6e8a 143558337536 512 0 3221767168 0 143558337536 280387378 0 0 da2p2 3 r1w1e1 r1w1e0 gpt/swap2 3221225472 512 0 541696 0 3221225472 6291456 0 0 da2p1 3 r0w0e0 r0w0e0 gptid/f57bb25e-74a3-11e3-8040-00237d9e6e8a 524288 512 0 17408 0 524288 1024 0 0 da2p1 3 r0w0e0 r0w0e0 gpt/gptboot2 524288 512 0 17408 0 524288 1024 0 0 da1p3 3 r1w1e2 r1w1e1 gptid/f52330df-74a3-11e3-8040-00237d9e6e8a 143558337536 512 0 3221767168 0 143558337536 280387378 0 0 da1p2 3 r1w1e1 r1w1e0 gpt/swap1 3221225472 512 0 541696 0 3221225472 6291456 0 0 da1p1 3 r0w0e0 r0w0e0 gptid/f4dbf3c8-74a3-11e3-8040-00237d9e6e8a 524288 512 0 17408 0 524288 1024 0 0 da1p1 3 r0w0e0 r0w0e0 gpt/gptboot1 524288 512 0 17408 0 524288 1024 0 0 da0p3 3 r1w1e2 r1w1e1 gptid/f471cf71-74a3-11e3-8040-00237d9e6e8a 143558337536 512 0 3221767168 0 143558337536 280387378 0 0 da0p2 3 r1w1e1 r1w1e0 gpt/swap0 3221225472 512 0 541696 0 3221225472 6291456 0 0 da0p1 3 r0w0e0 r0w0e0 gptid/f4308ed8-74a3-11e3-8040-00237d9e6e8a 524288 512 0 17408 0 524288 1024 0 0 da0p1 3 r0w0e0 r0w0e0 gpt/gptboot0 524288 512 0 17408 0 524288 1024 0 0 SWAP swap 4 r1w1e0 r1w1e0 r1w1e0 r1w1e0 r1w1e0 r1w1e0 PART da5 2 GPT 128 34 286679891 63 255 OK false r2w2e5 r1w1e2 da5p3 143558337536 512 0 3221767168 6292514 286679891 3 freebsd-zfs 3221767168 143558337536 516e7cba-6ecf-11d6-8ff8-00022d09712b f7a4d13a-74a3-11e3-8040-00237d9e6e8a r1w1e1 da5p2 3221225472 512 0 541696 1058 6292513 2 freebsd-swap 541696 3221225472 516e7cb5-6ecf-11d6-8ff8-00022d09712b f7859ccb-74a3-11e3-8040-00237d9e6e8a r0w0e0 da5p1 524288 512 0 17408 34 1057 1 freebsd-boot 17408 524288 83bd6b9d-7f41-11dc-be0b-001560b84f0f f76af2cc-74a3-11e3-8040-00237d9e6e8a da4 2 GPT 128 34 286679891 63 255 OK false r2w2e5 r1w1e2 da4p3 143558337536 512 0 3221767168 6292514 286679891 3 freebsd-zfs 3221767168 143558337536 516e7cba-6ecf-11d6-8ff8-00022d09712b f7043103-74a3-11e3-8040-00237d9e6e8a r1w1e1 da4p2 3221225472 512 0 541696 1058 6292513 2 freebsd-swap 541696 3221225472 516e7cb5-6ecf-11d6-8ff8-00022d09712b f6dfc4d9-74a3-11e3-8040-00237d9e6e8a r0w0e0 da4p1 524288 512 0 17408 34 1057 1 freebsd-boot 17408 524288 83bd6b9d-7f41-11dc-be0b-001560b84f0f f6c16edb-74a3-11e3-8040-00237d9e6e8a da3 2 GPT 128 34 286679891 63 255 OK false r2w2e5 r1w1e2 da3p3 143558337536 512 0 3221767168 6292514 286679891 3 freebsd-zfs 3221767168 143558337536 516e7cba-6ecf-11d6-8ff8-00022d09712b f6600fef-74a3-11e3-8040-00237d9e6e8a r1w1e1 da3p2 3221225472 512 0 541696 1058 6292513 2 freebsd-swap 541696 3221225472 516e7cb5-6ecf-11d6-8ff8-00022d09712b f640232f-74a3-11e3-8040-00237d9e6e8a r0w0e0 da3p1 524288 512 0 17408 34 1057 1 freebsd-boot 17408 524288 83bd6b9d-7f41-11dc-be0b-001560b84f0f f623a95d-74a3-11e3-8040-00237d9e6e8a da2 2 GPT 128 34 286679891 63 255 OK false r2w2e5 r1w1e2 da2p3 143558337536 512 0 3221767168 6292514 286679891 3 freebsd-zfs 3221767168 143558337536 516e7cba-6ecf-11d6-8ff8-00022d09712b f5bd173e-74a3-11e3-8040-00237d9e6e8a r1w1e1 da2p2 3221225472 512 0 541696 1058 6292513 2 freebsd-swap 541696 3221225472 516e7cb5-6ecf-11d6-8ff8-00022d09712b f59a0a61-74a3-11e3-8040-00237d9e6e8a r0w0e0 da2p1 524288 512 0 17408 34 1057 1 freebsd-boot 17408 524288 83bd6b9d-7f41-11dc-be0b-001560b84f0f f57bb25e-74a3-11e3-8040-00237d9e6e8a da1 2 GPT 128 34 286679891 63 255 OK false r2w2e5 r1w1e2 da1p3 143558337536 512 0 3221767168 6292514 286679891 3 freebsd-zfs 3221767168 143558337536 516e7cba-6ecf-11d6-8ff8-00022d09712b f52330df-74a3-11e3-8040-00237d9e6e8a r1w1e1 da1p2 3221225472 512 0 541696 1058 6292513 2 freebsd-swap 541696 3221225472 516e7cb5-6ecf-11d6-8ff8-00022d09712b f4fd9ed9-74a3-11e3-8040-00237d9e6e8a r0w0e0 da1p1 524288 512 0 17408 34 1057 1 freebsd-boot 17408 524288 83bd6b9d-7f41-11dc-be0b-001560b84f0f f4dbf3c8-74a3-11e3-8040-00237d9e6e8a da0 2 GPT 128 34 286679891 63 255 OK false r2w2e5 r1w1e2 da0p3 143558337536 512 0 3221767168 6292514 286679891 3 freebsd-zfs 3221767168 143558337536 516e7cba-6ecf-11d6-8ff8-00022d09712b f471cf71-74a3-11e3-8040-00237d9e6e8a r1w1e1 da0p2 3221225472 512 0 541696 1058 6292513 2 freebsd-swap 541696 3221225472 516e7cb5-6ecf-11d6-8ff8-00022d09712b f44d1866-74a3-11e3-8040-00237d9e6e8a r0w0e0 da0p1 524288 512 0 17408 34 1057 1 freebsd-boot 17408 524288 83bd6b9d-7f41-11dc-be0b-001560b84f0f f4308ed8-74a3-11e3-8040-00237d9e6e8a VFS DISK da5 1 r2w2e5 da5 146780121600 512 0 0 255 63 PH8BMQ3157 600508b100103135372020202020000d HP RAID 0 da4 1 r2w2e5 da4 146780121600 512 0 0 255 63 PH8BMQ3157 600508b100103135372020202020000c HP RAID 0 da3 1 r2w2e5 da3 146780121600 512 0 0 255 63 PH8BMQ3157 600508b100103135372020202020000b HP RAID 0 da2 1 r2w2e5 da2 146780121600 512 0 0 255 63 PH8BMQ3157 600508b100103135372020202020000a HP RAID 0 da1 1 r2w2e5 da1 146780121600 512 0 0 255 63 PH8BMQ3157 600508b1001031353720202020200009 HP RAID 0 da0 1 r2w2e5 da0 146780121600 512 0 0 255 63 PH8BMQ3157 600508b1001031353720202020200008 HP RAID 0 cd0 1 r0w0e0 cd0 0 2048 0 0 0 0 TEAC DV-28E-V DEV gptid/f7a4d13a-74a3-11e3-8040-00237d9e6e8a 4 r0w0e0 gpt/swap5 4 r0w0e0 gptid/f76af2cc-74a3-11e3-8040-00237d9e6e8a 4 r0w0e0 gpt/gptboot5 4 r0w0e0 gptid/f7043103-74a3-11e3-8040-00237d9e6e8a 4 r0w0e0 gpt/swap4 4 r0w0e0 gptid/f6c16edb-74a3-11e3-8040-00237d9e6e8a 4 r0w0e0 gpt/gptboot4 4 r0w0e0 gptid/f6600fef-74a3-11e3-8040-00237d9e6e8a 4 r0w0e0 gpt/swap3 4 r0w0e0 gptid/f623a95d-74a3-11e3-8040-00237d9e6e8a 4 r0w0e0 gpt/gptboot3 4 r0w0e0 gptid/f5bd173e-74a3-11e3-8040-00237d9e6e8a 4 r0w0e0 gpt/swap2 4 r0w0e0 gptid/f57bb25e-74a3-11e3-8040-00237d9e6e8a 4 r0w0e0 gpt/gptboot2 4 r0w0e0 gptid/f52330df-74a3-11e3-8040-00237d9e6e8a 4 r0w0e0 gpt/swap1 4 r0w0e0 gptid/f4dbf3c8-74a3-11e3-8040-00237d9e6e8a 4 r0w0e0 gpt/gptboot1 4 r0w0e0 gptid/f471cf71-74a3-11e3-8040-00237d9e6e8a 4 r0w0e0 gpt/swap0 4 r0w0e0 gptid/f4308ed8-74a3-11e3-8040-00237d9e6e8a 4 r0w0e0 gpt/gptboot0 4 r0w0e0 da5p3 3 r0w0e0 da5p2 3 r0w0e0 da5p1 3 r0w0e0 da4p3 3 r0w0e0 da4p2 3 r0w0e0 da4p1 3 r0w0e0 da3p3 3 r0w0e0 da3p2 3 r0w0e0 da3p1 3 r0w0e0 da2p3 3 r0w0e0 da2p2 3 r0w0e0 da2p1 3 r0w0e0 da1p3 3 r0w0e0 da1p2 3 r0w0e0 da1p1 3 r0w0e0 da0p3 3 r0w0e0 da0p2 3 r0w0e0 da0p1 3 r0w0e0 da5 2 r0w0e0 da4 2 r0w0e0 da3 2 r0w0e0 da2 2 r0w0e0 da1 2 r0w0e0 da0 2 r0w0e0 cd0 2 r0w0e0 ZFS::VDEV zfs::vdev 4 r1w1e1 r1w1e1 r1w1e1 r1w1e1 r1w1e1 r1w1e1 ZFS::ZVOL kern.geom.confdot: digraph geom { z0xfffff8000e08f100 [shape=box,label="LABEL\nda5p3\nr#3"]; z0xfffff8000e09c400 [label="r1w1e2"]; z0xfffff8000e09c400 -> z0xfffff8000e0b4c00; z0xfffff8000e08f100 -> z0xfffff8000e09c400; z0xfffff8000e08f000 [shape=hexagon,label="gptid/f7a4d13a-74a3-11e3-8040-00237d9e6e8a\nr1w1e1\nerr#0"]; z0xfffff8000e08f000 -> z0xfffff8000e08f100; z0xfffff8000e0b4800 [shape=box,label="LABEL\nda5p2\nr#3"]; z0xfffff8000e09c600 [label="r1w1e1"]; z0xfffff8000e09c600 -> z0xfffff8000e0b4e00; z0xfffff8000e0b4800 -> z0xfffff8000e09c600; z0xfffff8000af3d600 [shape=hexagon,label="gpt/swap5\nr1w1e0\nerr#0"]; z0xfffff8000af3d600 -> z0xfffff8000e0b4800; z0xfffff8000e062c00 [shape=box,label="LABEL\nda5p1\nr#3"]; z0xfffff8000e070600 [label="r0w0e0"]; z0xfffff8000e070600 -> z0xfffff8000e0b5100; z0xfffff8000e062c00 -> z0xfffff8000e070600; z0xfffff8000e062b00 [shape=hexagon,label="gptid/f76af2cc-74a3-11e3-8040-00237d9e6e8a\nr0w0e0\nerr#0"]; z0xfffff8000e062b00 -> z0xfffff8000e062c00; z0xfffff8000e063000 [shape=box,label="LABEL\nda5p1\nr#3"]; z0xfffff8000e070680 [label="r0w0e0"]; z0xfffff8000e070680 -> z0xfffff8000e0b5100; z0xfffff8000e063000 -> z0xfffff8000e070680; z0xfffff8000e062e00 [shape=hexagon,label="gpt/gptboot5\nr0w0e0\nerr#0"]; z0xfffff8000e062e00 -> z0xfffff8000e063000; z0xfffff8000e08f800 [shape=box,label="LABEL\nda4p3\nr#3"]; z0xfffff8000e09c700 [label="r1w1e2"]; z0xfffff8000e09c700 -> z0xfffff8000e090500; z0xfffff8000e08f800 -> z0xfffff8000e09c700; z0xfffff8000e08f700 [shape=hexagon,label="gptid/f7043103-74a3-11e3-8040-00237d9e6e8a\nr1w1e1\nerr#0"]; z0xfffff8000e08f700 -> z0xfffff8000e08f800; z0xfffff8000e090200 [shape=box,label="LABEL\nda4p2\nr#3"]; z0xfffff8000e09c900 [label="r1w1e1"]; z0xfffff8000e09c900 -> z0xfffff8000e090700; z0xfffff8000e090200 -> z0xfffff8000e09c900; z0xfffff8000e090100 [shape=hexagon,label="gpt/swap4\nr1w1e0\nerr#0"]; z0xfffff8000e090100 -> z0xfffff8000e090200; z0xfffff8000ae21e00 [shape=box,label="LABEL\nda4p1\nr#3"]; z0xfffff8000e088580 [label="r0w0e0"]; z0xfffff8000e088580 -> z0xfffff8000e090900; z0xfffff8000ae21e00 -> z0xfffff8000e088580; z0xfffff8000e07d300 [shape=hexagon,label="gptid/f6c16edb-74a3-11e3-8040-00237d9e6e8a\nr0w0e0\nerr#0"]; z0xfffff8000e07d300 -> z0xfffff8000ae21e00; z0xfffff8000af3b700 [shape=box,label="LABEL\nda4p1\nr#3"]; z0xfffff8000e088600 [label="r0w0e0"]; z0xfffff8000e088600 -> z0xfffff8000e090900; z0xfffff8000af3b700 -> z0xfffff8000e088600; z0xfffff8000af3b500 [shape=hexagon,label="gpt/gptboot4\nr0w0e0\nerr#0"]; z0xfffff8000af3b500 -> z0xfffff8000af3b700; z0xfffff8000e0a2400 [shape=box,label="LABEL\nda3p3\nr#3"]; z0xfffff8000e0ad380 [label="r1w1e2"]; z0xfffff8000e0ad380 -> z0xfffff8000e0b5700; z0xfffff8000e0a2400 -> z0xfffff8000e0ad380; z0xfffff8000e0a2300 [shape=hexagon,label="gptid/f6600fef-74a3-11e3-8040-00237d9e6e8a\nr1w1e1\nerr#0"]; z0xfffff8000e0a2300 -> z0xfffff8000e0a2400; z0xfffff8000e0a2d00 [shape=box,label="LABEL\nda3p2\nr#3"]; z0xfffff8000e0ad580 [label="r1w1e1"]; z0xfffff8000e0ad580 -> z0xfffff8000e0b5900; z0xfffff8000e0a2d00 -> z0xfffff8000e0ad580; z0xfffff8000e0a2c00 [shape=hexagon,label="gpt/swap3\nr1w1e0\nerr#0"]; z0xfffff8000e0a2c00 -> z0xfffff8000e0a2d00; z0xfffff8000e073400 [shape=box,label="LABEL\nda3p1\nr#3"]; z0xfffff8000e052180 [label="r0w0e0"]; z0xfffff8000e052180 -> z0xfffff8000e0b5b00; z0xfffff8000e073400 -> z0xfffff8000e052180; z0xfffff8000e073300 [shape=hexagon,label="gptid/f623a95d-74a3-11e3-8040-00237d9e6e8a\nr0w0e0\nerr#0"]; z0xfffff8000e073300 -> z0xfffff8000e073400; z0xfffff8000e073700 [shape=box,label="LABEL\nda3p1\nr#3"]; z0xfffff8000e052200 [label="r0w0e0"]; z0xfffff8000e052200 -> z0xfffff8000e0b5b00; z0xfffff8000e073700 -> z0xfffff8000e052200; z0xfffff8000e073600 [shape=hexagon,label="gpt/gptboot3\nr0w0e0\nerr#0"]; z0xfffff8000e073600 -> z0xfffff8000e073700; z0xfffff8000e0c6c00 [shape=box,label="LABEL\nda2p3\nr#3"]; z0xfffff8000e0d2080 [label="r1w1e2"]; z0xfffff8000e0d2080 -> z0xfffff8000e0c7800; z0xfffff8000e0c6c00 -> z0xfffff8000e0d2080; z0xfffff8000e0c6b00 [shape=hexagon,label="gptid/f5bd173e-74a3-11e3-8040-00237d9e6e8a\nr1w1e1\nerr#0"]; z0xfffff8000e0c6b00 -> z0xfffff8000e0c6c00; z0xfffff8000e0b4900 [shape=box,label="LABEL\nda2p2\nr#3"]; z0xfffff8000e0d2280 [label="r1w1e1"]; z0xfffff8000e0d2280 -> z0xfffff8000e0c7a00; z0xfffff8000e0b4900 -> z0xfffff8000e0d2280; z0xfffff8000e0c7500 [shape=hexagon,label="gpt/swap2\nr1w1e0\nerr#0"]; z0xfffff8000e0c7500 -> z0xfffff8000e0b4900; z0xfffff800073fa100 [shape=box,label="LABEL\nda2p1\nr#3"]; z0xfffff8000ae83900 [label="r0w0e0"]; z0xfffff8000ae83900 -> z0xfffff8000e0c7c00; z0xfffff800073fa100 -> z0xfffff8000ae83900; z0xfffff80003f61e00 [shape=hexagon,label="gptid/f57bb25e-74a3-11e3-8040-00237d9e6e8a\nr0w0e0\nerr#0"]; z0xfffff80003f61e00 -> z0xfffff800073fa100; z0xfffff800073fa900 [shape=box,label="LABEL\nda2p1\nr#3"]; z0xfffff8000ae83980 [label="r0w0e0"]; z0xfffff8000ae83980 -> z0xfffff8000e0c7c00; z0xfffff800073fa900 -> z0xfffff8000ae83980; z0xfffff800073fa800 [shape=hexagon,label="gpt/gptboot2\nr0w0e0\nerr#0"]; z0xfffff800073fa800 -> z0xfffff800073fa900; z0xfffff8000e07d600 [shape=box,label="LABEL\nda1p3\nr#3"]; z0xfffff8000e088700 [label="r1w1e2"]; z0xfffff8000e088700 -> z0xfffff8000ae21c00; z0xfffff8000e07d600 -> z0xfffff8000e088700; z0xfffff8000e07d500 [shape=hexagon,label="gptid/f52330df-74a3-11e3-8040-00237d9e6e8a\nr1w1e1\nerr#0"]; z0xfffff8000e07d500 -> z0xfffff8000e07d600; z0xfffff800098a5700 [shape=box,label="LABEL\nda1p2\nr#3"]; z0xfffff8000ae81580 [label="r1w1e1"]; z0xfffff8000ae81580 -> z0xfffff8000ae21800; z0xfffff800098a5700 -> z0xfffff8000ae81580; z0xfffff800098a5900 [shape=hexagon,label="gpt/swap1\nr1w1e0\nerr#0"]; z0xfffff800098a5900 -> z0xfffff800098a5700; z0xfffff8000afcc300 [shape=box,label="LABEL\nda1p1\nr#3"]; z0xfffff8000e052300 [label="r0w0e0"]; z0xfffff8000e052300 -> z0xfffff8000afcd600; z0xfffff8000afcc300 -> z0xfffff8000e052300; z0xfffff8000e073900 [shape=hexagon,label="gptid/f4dbf3c8-74a3-11e3-8040-00237d9e6e8a\nr0w0e0\nerr#0"]; z0xfffff8000e073900 -> z0xfffff8000afcc300; z0xfffff8000afccb00 [shape=box,label="LABEL\nda1p1\nr#3"]; z0xfffff8000e052380 [label="r0w0e0"]; z0xfffff8000e052380 -> z0xfffff8000afcd600; z0xfffff8000afccb00 -> z0xfffff8000e052380; z0xfffff8000afcc900 [shape=hexagon,label="gpt/gptboot1\nr0w0e0\nerr#0"]; z0xfffff8000afcc900 -> z0xfffff8000afccb00; z0xfffff8000e073c00 [shape=box,label="LABEL\nda0p3\nr#3"]; z0xfffff8000e052480 [label="r1w1e2"]; z0xfffff8000e052480 -> z0xfffff8000afcc100; z0xfffff8000e073c00 -> z0xfffff8000e052480; z0xfffff8000e073b00 [shape=hexagon,label="gptid/f471cf71-74a3-11e3-8040-00237d9e6e8a\nr1w1e1\nerr#0"]; z0xfffff8000e073b00 -> z0xfffff8000e073c00; z0xfffff8000e074600 [shape=box,label="LABEL\nda0p2\nr#3"]; z0xfffff8000e052680 [label="r1w1e1"]; z0xfffff8000e052680 -> z0xfffff8000af3e900; z0xfffff8000e074600 -> z0xfffff8000e052680; z0xfffff8000e074500 [shape=hexagon,label="gpt/swap0\nr1w1e0\nerr#0"]; z0xfffff8000e074500 -> z0xfffff8000e074600; z0xfffff8000e0a3100 [shape=box,label="LABEL\nda0p1\nr#3"]; z0xfffff8000e0ad680 [label="r0w0e0"]; z0xfffff8000e0ad680 -> z0xfffff8000af3eb00; z0xfffff8000e0a3100 -> z0xfffff8000e0ad680; z0xfffff8000e0a3000 [shape=hexagon,label="gptid/f4308ed8-74a3-11e3-8040-00237d9e6e8a\nr0w0e0\nerr#0"]; z0xfffff8000e0a3000 -> z0xfffff8000e0a3100; z0xfffff8000afca100 [shape=box,label="LABEL\nda0p1\nr#3"]; z0xfffff8000e0ad700 [label="r0w0e0"]; z0xfffff8000e0ad700 -> z0xfffff8000af3eb00; z0xfffff8000afca100 -> z0xfffff8000e0ad700; z0xfffff8000e0a3300 [shape=hexagon,label="gpt/gptboot0\nr0w0e0\nerr#0"]; z0xfffff8000e0a3300 -> z0xfffff8000afca100; z0xfffff8000af3cc00 [shape=box,label="SWAP\nswap\nr#4"]; z0xfffff8000e09ab80 [label="r1w1e0"]; z0xfffff8000e09ab80 -> z0xfffff8000af3d600; z0xfffff8000af3cc00 -> z0xfffff8000e09ab80; z0xfffff8000e09b080 [label="r1w1e0"]; z0xfffff8000e09b080 -> z0xfffff8000e090100; z0xfffff8000af3cc00 -> z0xfffff8000e09b080; z0xfffff8000e09b500 [label="r1w1e0"]; z0xfffff8000e09b500 -> z0xfffff8000e0a2c00; z0xfffff8000af3cc00 -> z0xfffff8000e09b500; z0xfffff8000e09b980 [label="r1w1e0"]; z0xfffff8000e09b980 -> z0xfffff8000e0c7500; z0xfffff8000af3cc00 -> z0xfffff8000e09b980; z0xfffff8000e09be00 [label="r1w1e0"]; z0xfffff8000e09be00 -> z0xfffff800098a5900; z0xfffff8000af3cc00 -> z0xfffff8000e09be00; z0xfffff8000ebd2580 [label="r1w1e0"]; z0xfffff8000ebd2580 -> z0xfffff8000e074500; z0xfffff8000af3cc00 -> z0xfffff8000ebd2580; z0xfffff8000ae1e300 [shape=box,label="PART\nda5\nr#2"]; z0xfffff8000e0c1980 [label="r2w2e5"]; z0xfffff8000e0c1980 -> z0xfffff8000ae1e700; z0xfffff8000ae1e300 -> z0xfffff8000e0c1980; z0xfffff8000e0b4c00 [shape=hexagon,label="da5p3\nr1w1e2\nerr#0"]; z0xfffff8000e0b4c00 -> z0xfffff8000ae1e300; z0xfffff8000e0b4e00 [shape=hexagon,label="da5p2\nr1w1e1\nerr#0"]; z0xfffff8000e0b4e00 -> z0xfffff8000ae1e300; z0xfffff8000e0b5100 [shape=hexagon,label="da5p1\nr0w0e0\nerr#0"]; z0xfffff8000e0b5100 -> z0xfffff8000ae1e300; z0xfffff800098a5c00 [shape=box,label="PART\nda4\nr#2"]; z0xfffff8000e0c1a00 [label="r2w2e5"]; z0xfffff8000e0c1a00 -> z0xfffff8000ae1e100; z0xfffff800098a5c00 -> z0xfffff8000e0c1a00; z0xfffff8000e090500 [shape=hexagon,label="da4p3\nr1w1e2\nerr#0"]; z0xfffff8000e090500 -> z0xfffff800098a5c00; z0xfffff8000e090700 [shape=hexagon,label="da4p2\nr1w1e1\nerr#0"]; z0xfffff8000e090700 -> z0xfffff800098a5c00; z0xfffff8000e090900 [shape=hexagon,label="da4p1\nr0w0e0\nerr#0"]; z0xfffff8000e090900 -> z0xfffff800098a5c00; z0xfffff8000af3e800 [shape=box,label="PART\nda3\nr#2"]; z0xfffff8000e0d2300 [label="r2w2e5"]; z0xfffff8000e0d2300 -> z0xfffff800098a5a00; z0xfffff8000af3e800 -> z0xfffff8000e0d2300; z0xfffff8000e0b5700 [shape=hexagon,label="da3p3\nr1w1e2\nerr#0"]; z0xfffff8000e0b5700 -> z0xfffff8000af3e800; z0xfffff8000e0b5900 [shape=hexagon,label="da3p2\nr1w1e1\nerr#0"]; z0xfffff8000e0b5900 -> z0xfffff8000af3e800; z0xfffff8000e0b5b00 [shape=hexagon,label="da3p1\nr0w0e0\nerr#0"]; z0xfffff8000e0b5b00 -> z0xfffff8000af3e800; z0xfffff8000af3e200 [shape=box,label="PART\nda2\nr#2"]; z0xfffff800098cde80 [label="r2w2e5"]; z0xfffff800098cde80 -> z0xfffff8000af3e600; z0xfffff8000af3e200 -> z0xfffff800098cde80; z0xfffff8000e0c7800 [shape=hexagon,label="da2p3\nr1w1e2\nerr#0"]; z0xfffff8000e0c7800 -> z0xfffff8000af3e200; z0xfffff8000e0c7a00 [shape=hexagon,label="da2p2\nr1w1e1\nerr#0"]; z0xfffff8000e0c7a00 -> z0xfffff8000af3e200; z0xfffff8000e0c7c00 [shape=hexagon,label="da2p1\nr0w0e0\nerr#0"]; z0xfffff8000e0c7c00 -> z0xfffff8000af3e200; z0xfffff8000af3db00 [shape=box,label="PART\nda1\nr#2"]; z0xfffff800098cdb80 [label="r2w2e5"]; z0xfffff800098cdb80 -> z0xfffff8000af3e000; z0xfffff8000af3db00 -> z0xfffff800098cdb80; z0xfffff8000ae21c00 [shape=hexagon,label="da1p3\nr1w1e2\nerr#0"]; z0xfffff8000ae21c00 -> z0xfffff8000af3db00; z0xfffff8000ae21800 [shape=hexagon,label="da1p2\nr1w1e1\nerr#0"]; z0xfffff8000ae21800 -> z0xfffff8000af3db00; z0xfffff8000afcd600 [shape=hexagon,label="da1p1\nr0w0e0\nerr#0"]; z0xfffff8000afcd600 -> z0xfffff8000af3db00; z0xfffff8000af3d500 [shape=box,label="PART\nda0\nr#2"]; z0xfffff800098cd700 [label="r2w2e5"]; z0xfffff800098cd700 -> z0xfffff8000af3d900; z0xfffff8000af3d500 -> z0xfffff800098cd700; z0xfffff8000afcc100 [shape=hexagon,label="da0p3\nr1w1e2\nerr#0"]; z0xfffff8000afcc100 -> z0xfffff8000af3d500; z0xfffff8000af3e900 [shape=hexagon,label="da0p2\nr1w1e1\nerr#0"]; z0xfffff8000af3e900 -> z0xfffff8000af3d500; z0xfffff8000af3eb00 [shape=hexagon,label="da0p1\nr0w0e0\nerr#0"]; z0xfffff8000af3eb00 -> z0xfffff8000af3d500; z0xfffff8000ae1e500 [shape=box,label="DISK\nda5\nr#1"]; z0xfffff8000ae1e700 [shape=hexagon,label="da5\nr2w2e5\nerr#0"]; z0xfffff8000ae1e700 -> z0xfffff8000ae1e500; z0xfffff800098a5e00 [shape=box,label="DISK\nda4\nr#1"]; z0xfffff8000ae1e100 [shape=hexagon,label="da4\nr2w2e5\nerr#0"]; z0xfffff8000ae1e100 -> z0xfffff800098a5e00; z0xfffff800098a5800 [shape=box,label="DISK\nda3\nr#1"]; z0xfffff800098a5a00 [shape=hexagon,label="da3\nr2w2e5\nerr#0"]; z0xfffff800098a5a00 -> z0xfffff800098a5800; z0xfffff8000af3e400 [shape=box,label="DISK\nda2\nr#1"]; z0xfffff8000af3e600 [shape=hexagon,label="da2\nr2w2e5\nerr#0"]; z0xfffff8000af3e600 -> z0xfffff8000af3e400; z0xfffff8000af3dd00 [shape=box,label="DISK\nda1\nr#1"]; z0xfffff8000af3e000 [shape=hexagon,label="da1\nr2w2e5\nerr#0"]; z0xfffff8000af3e000 -> z0xfffff8000af3dd00; z0xfffff8000af3d700 [shape=box,label="DISK\nda0\nr#1"]; z0xfffff8000af3d900 [shape=hexagon,label="da0\nr2w2e5\nerr#0"]; z0xfffff8000af3d900 -> z0xfffff8000af3d700; z0xfffff8000af3d100 [shape=box,label="DISK\ncd0\nr#1"]; z0xfffff8000af3d300 [shape=hexagon,label="cd0\nr0w0e0\nerr#0"]; z0xfffff8000af3d300 -> z0xfffff8000af3d100; z0xfffff8000e08f200 [shape=box,label="DEV\ngptid/f7a4d13a-74a3-11e3-8040-00237d9e6e8a\nr#4"]; z0xfffff8000e09a580 [label="r0w0e0"]; z0xfffff8000e09a580 -> z0xfffff8000e08f000; z0xfffff8000e08f200 -> z0xfffff8000e09a580; z0xfffff8000e062a00 [shape=box,label="DEV\ngpt/swap5\nr#4"]; z0xfffff8000e09a780 [label="r0w0e0"]; z0xfffff8000e09a780 -> z0xfffff8000af3d600; z0xfffff8000e062a00 -> z0xfffff8000e09a780; z0xfffff8000e062d00 [shape=box,label="DEV\ngptid/f76af2cc-74a3-11e3-8040-00237d9e6e8a\nr#4"]; z0xfffff8000e09a880 [label="r0w0e0"]; z0xfffff8000e09a880 -> z0xfffff8000e062b00; z0xfffff8000e062d00 -> z0xfffff8000e09a880; z0xfffff8000e08f600 [shape=box,label="DEV\ngpt/gptboot5\nr#4"]; z0xfffff8000e09a900 [label="r0w0e0"]; z0xfffff8000e09a900 -> z0xfffff8000e062e00; z0xfffff8000e08f600 -> z0xfffff8000e09a900; z0xfffff8000e08f900 [shape=box,label="DEV\ngptid/f7043103-74a3-11e3-8040-00237d9e6e8a\nr#4"]; z0xfffff8000e09aa00 [label="r0w0e0"]; z0xfffff8000e09aa00 -> z0xfffff8000e08f700; z0xfffff8000e08f900 -> z0xfffff8000e09aa00; z0xfffff8000e07d200 [shape=box,label="DEV\ngpt/swap4\nr#4"]; z0xfffff8000e09ac00 [label="r0w0e0"]; z0xfffff8000e09ac00 -> z0xfffff8000e090100; z0xfffff8000e07d200 -> z0xfffff8000e09ac00; z0xfffff8000af3b300 [shape=box,label="DEV\ngptid/f6c16edb-74a3-11e3-8040-00237d9e6e8a\nr#4"]; z0xfffff8000e09ad00 [label="r0w0e0"]; z0xfffff8000e09ad00 -> z0xfffff8000e07d300; z0xfffff8000af3b300 -> z0xfffff8000e09ad00; z0xfffff8000e0a2200 [shape=box,label="DEV\ngpt/gptboot4\nr#4"]; z0xfffff8000e09ad80 [label="r0w0e0"]; z0xfffff8000e09ad80 -> z0xfffff8000af3b500; z0xfffff8000e0a2200 -> z0xfffff8000e09ad80; z0xfffff8000e0a2500 [shape=box,label="DEV\ngptid/f6600fef-74a3-11e3-8040-00237d9e6e8a\nr#4"]; z0xfffff8000e09ae80 [label="r0w0e0"]; z0xfffff8000e09ae80 -> z0xfffff8000e0a2300; z0xfffff8000e0a2500 -> z0xfffff8000e09ae80; z0xfffff8000e073200 [shape=box,label="DEV\ngpt/swap3\nr#4"]; z0xfffff8000e09b100 [label="r0w0e0"]; z0xfffff8000e09b100 -> z0xfffff8000e0a2c00; z0xfffff8000e073200 -> z0xfffff8000e09b100; z0xfffff8000e073500 [shape=box,label="DEV\ngptid/f623a95d-74a3-11e3-8040-00237d9e6e8a\nr#4"]; z0xfffff8000e09b200 [label="r0w0e0"]; z0xfffff8000e09b200 -> z0xfffff8000e073300; z0xfffff8000e073500 -> z0xfffff8000e09b200; z0xfffff8000e0c6a00 [shape=box,label="DEV\ngpt/gptboot3\nr#4"]; z0xfffff8000e09b280 [label="r0w0e0"]; z0xfffff8000e09b280 -> z0xfffff8000e073600; z0xfffff8000e0c6a00 -> z0xfffff8000e09b280; z0xfffff8000e0c6d00 [shape=box,label="DEV\ngptid/f5bd173e-74a3-11e3-8040-00237d9e6e8a\nr#4"]; z0xfffff8000e09b380 [label="r0w0e0"]; z0xfffff8000e09b380 -> z0xfffff8000e0c6b00; z0xfffff8000e0c6d00 -> z0xfffff8000e09b380; z0xfffff8000afcbd00 [shape=box,label="DEV\ngpt/swap2\nr#4"]; z0xfffff8000e09b580 [label="r0w0e0"]; z0xfffff8000e09b580 -> z0xfffff8000e0c7500; z0xfffff8000afcbd00 -> z0xfffff8000e09b580; z0xfffff800073fa500 [shape=box,label="DEV\ngptid/f57bb25e-74a3-11e3-8040-00237d9e6e8a\nr#4"]; z0xfffff8000e09b680 [label="r0w0e0"]; z0xfffff8000e09b680 -> z0xfffff80003f61e00; z0xfffff800073fa500 -> z0xfffff8000e09b680; z0xfffff8000e064900 [shape=box,label="DEV\ngpt/gptboot2\nr#4"]; z0xfffff8000e09b700 [label="r0w0e0"]; z0xfffff8000e09b700 -> z0xfffff800073fa800; z0xfffff8000e064900 -> z0xfffff8000e09b700; z0xfffff8000e07d700 [shape=box,label="DEV\ngptid/f52330df-74a3-11e3-8040-00237d9e6e8a\nr#4"]; z0xfffff8000e09b800 [label="r0w0e0"]; z0xfffff8000e09b800 -> z0xfffff8000e07d500; z0xfffff8000e07d700 -> z0xfffff8000e09b800; z0xfffff8000af3e700 [shape=box,label="DEV\ngpt/swap1\nr#4"]; z0xfffff8000e09ba00 [label="r0w0e0"]; z0xfffff8000e09ba00 -> z0xfffff800098a5900; z0xfffff8000af3e700 -> z0xfffff8000e09ba00; z0xfffff8000afcc700 [shape=box,label="DEV\ngptid/f4dbf3c8-74a3-11e3-8040-00237d9e6e8a\nr#4"]; z0xfffff8000e09bb00 [label="r0w0e0"]; z0xfffff8000e09bb00 -> z0xfffff8000e073900; z0xfffff8000afcc700 -> z0xfffff8000e09bb00; z0xfffff8000af3de00 [shape=box,label="DEV\ngpt/gptboot1\nr#4"]; z0xfffff8000e09bb80 [label="r0w0e0"]; z0xfffff8000e09bb80 -> z0xfffff8000afcc900; z0xfffff8000af3de00 -> z0xfffff8000e09bb80; z0xfffff8000e073d00 [shape=box,label="DEV\ngptid/f471cf71-74a3-11e3-8040-00237d9e6e8a\nr#4"]; z0xfffff8000e09bc80 [label="r0w0e0"]; z0xfffff8000e09bc80 -> z0xfffff8000e073b00; z0xfffff8000e073d00 -> z0xfffff8000e09bc80; z0xfffff8000e065100 [shape=box,label="DEV\ngpt/swap0\nr#4"]; z0xfffff8000e09be80 [label="r0w0e0"]; z0xfffff8000e09be80 -> z0xfffff8000e074500; z0xfffff8000e065100 -> z0xfffff8000e09be80; z0xfffff8000e0a3200 [shape=box,label="DEV\ngptid/f4308ed8-74a3-11e3-8040-00237d9e6e8a\nr#4"]; z0xfffff8000e09c000 [label="r0w0e0"]; z0xfffff8000e09c000 -> z0xfffff8000e0a3000; z0xfffff8000e0a3200 -> z0xfffff8000e09c000; z0xfffff8000e065500 [shape=box,label="DEV\ngpt/gptboot0\nr#4"]; z0xfffff8000e09c080 [label="r0w0e0"]; z0xfffff8000e09c080 -> z0xfffff8000e0a3300; z0xfffff8000e065500 -> z0xfffff8000e09c080; z0xfffff8000e0b4d00 [shape=box,label="DEV\nda5p3\nr#3"]; z0xfffff8000e09c500 [label="r0w0e0"]; z0xfffff8000e09c500 -> z0xfffff8000e0b4c00; z0xfffff8000e0b4d00 -> z0xfffff8000e09c500; z0xfffff8000e0b5000 [shape=box,label="DEV\nda5p2\nr#3"]; z0xfffff8000e070580 [label="r0w0e0"]; z0xfffff8000e070580 -> z0xfffff8000e0b4e00; z0xfffff8000e0b5000 -> z0xfffff8000e070580; z0xfffff8000af3d800 [shape=box,label="DEV\nda5p1\nr#3"]; z0xfffff8000e09c680 [label="r0w0e0"]; z0xfffff8000e09c680 -> z0xfffff8000e0b5100; z0xfffff8000af3d800 -> z0xfffff8000e09c680; z0xfffff8000e090600 [shape=box,label="DEV\nda4p3\nr#3"]; z0xfffff8000e09c800 [label="r0w0e0"]; z0xfffff8000e09c800 -> z0xfffff8000e090500; z0xfffff8000e090600 -> z0xfffff8000e09c800; z0xfffff8000e090800 [shape=box,label="DEV\nda4p2\nr#3"]; z0xfffff8000e088500 [label="r0w0e0"]; z0xfffff8000e088500 -> z0xfffff8000e090700; z0xfffff8000e090800 -> z0xfffff8000e088500; z0xfffff8000e074800 [shape=box,label="DEV\nda4p1\nr#3"]; z0xfffff8000e0ad300 [label="r0w0e0"]; z0xfffff8000e0ad300 -> z0xfffff8000e090900; z0xfffff8000e074800 -> z0xfffff8000e0ad300; z0xfffff8000e0b5800 [shape=box,label="DEV\nda3p3\nr#3"]; z0xfffff8000e0ad480 [label="r0w0e0"]; z0xfffff8000e0ad480 -> z0xfffff8000e0b5700; z0xfffff8000e0b5800 -> z0xfffff8000e0ad480; z0xfffff8000e0b5a00 [shape=box,label="DEV\nda3p2\nr#3"]; z0xfffff8000e052100 [label="r0w0e0"]; z0xfffff8000e052100 -> z0xfffff8000e0b5900; z0xfffff8000e0b5a00 -> z0xfffff8000e052100; z0xfffff8000af3d000 [shape=box,label="DEV\nda3p1\nr#3"]; z0xfffff8000e0d2000 [label="r0w0e0"]; z0xfffff8000e0d2000 -> z0xfffff8000e0b5b00; z0xfffff8000af3d000 -> z0xfffff8000e0d2000; z0xfffff8000e0c7900 [shape=box,label="DEV\nda2p3\nr#3"]; z0xfffff8000e0d2180 [label="r0w0e0"]; z0xfffff8000e0d2180 -> z0xfffff8000e0c7800; z0xfffff8000e0c7900 -> z0xfffff8000e0d2180; z0xfffff8000e0c7b00 [shape=box,label="DEV\nda2p2\nr#3"]; z0xfffff8000ae83880 [label="r0w0e0"]; z0xfffff8000ae83880 -> z0xfffff8000e0c7a00; z0xfffff8000e0c7b00 -> z0xfffff8000ae83880; z0xfffff8000e07db00 [shape=box,label="DEV\nda2p1\nr#3"]; z0xfffff8000e088680 [label="r0w0e0"]; z0xfffff8000e088680 -> z0xfffff8000e0c7c00; z0xfffff8000e07db00 -> z0xfffff8000e088680; z0xfffff8000ae21a00 [shape=box,label="DEV\nda1p3\nr#3"]; z0xfffff8000e070700 [label="r0w0e0"]; z0xfffff8000e070700 -> z0xfffff8000ae21c00; z0xfffff8000ae21a00 -> z0xfffff8000e070700; z0xfffff8000afcd800 [shape=box,label="DEV\nda1p2\nr#3"]; z0xfffff8000e052280 [label="r0w0e0"]; z0xfffff8000e052280 -> z0xfffff8000ae21800; z0xfffff8000afcd800 -> z0xfffff8000e052280; z0xfffff8000afccd00 [shape=box,label="DEV\nda1p1\nr#3"]; z0xfffff8000e052400 [label="r0w0e0"]; z0xfffff8000e052400 -> z0xfffff8000afcd600; z0xfffff8000afccd00 -> z0xfffff8000e052400; z0xfffff8000afcbe00 [shape=box,label="DEV\nda0p3\nr#3"]; z0xfffff8000e052580 [label="r0w0e0"]; z0xfffff8000e052580 -> z0xfffff8000afcc100; z0xfffff8000afcbe00 -> z0xfffff8000e052580; z0xfffff8000af3ea00 [shape=box,label="DEV\nda0p2\nr#3"]; z0xfffff8000e0ad600 [label="r0w0e0"]; z0xfffff8000e0ad600 -> z0xfffff8000af3e900; z0xfffff8000af3ea00 -> z0xfffff8000e0ad600; z0xfffff8000afca000 [shape=box,label="DEV\nda0p1\nr#3"]; z0xfffff8000e0ad780 [label="r0w0e0"]; z0xfffff8000e0ad780 -> z0xfffff8000af3eb00; z0xfffff8000afca000 -> z0xfffff8000e0ad780; z0xfffff8000e0b4a00 [shape=box,label="DEV\nda5\nr#2"]; z0xfffff8000e070780 [label="r0w0e0"]; z0xfffff8000e070780 -> z0xfffff8000ae1e700; z0xfffff8000e0b4a00 -> z0xfffff8000e070780; z0xfffff8000e090300 [shape=box,label="DEV\nda4\nr#2"]; z0xfffff8000ae81500 [label="r0w0e0"]; z0xfffff8000ae81500 -> z0xfffff8000ae1e100; z0xfffff8000e090300 -> z0xfffff8000ae81500; z0xfffff8000e0b5500 [shape=box,label="DEV\nda3\nr#2"]; z0xfffff8000e088800 [label="r0w0e0"]; z0xfffff8000e088800 -> z0xfffff800098a5a00; z0xfffff8000e0b5500 -> z0xfffff8000e088800; z0xfffff8000e0c7600 [shape=box,label="DEV\nda2\nr#2"]; z0xfffff8000e052700 [label="r0w0e0"]; z0xfffff8000e052700 -> z0xfffff8000af3e600; z0xfffff8000e0c7600 -> z0xfffff8000e052700; z0xfffff8000af3b100 [shape=box,label="DEV\nda1\nr#2"]; z0xfffff800098cdd80 [label="r0w0e0"]; z0xfffff800098cdd80 -> z0xfffff8000af3e000; z0xfffff8000af3b100 -> z0xfffff800098cdd80; z0xfffff8000afcc500 [shape=box,label="DEV\nda0\nr#2"]; z0xfffff800098cda80 [label="r0w0e0"]; z0xfffff800098cda80 -> z0xfffff8000af3d900; z0xfffff8000afcc500 -> z0xfffff800098cda80; z0xfffff8000af3cb00 [shape=box,label="DEV\ncd0\nr#2"]; z0xfffff8000ae80480 [label="r0w0e0"]; z0xfffff8000ae80480 -> z0xfffff8000af3d300; z0xfffff8000af3cb00 -> z0xfffff8000ae80480; z0xfffff8000ebd1600 [shape=box,label="ZFS::VDEV\nzfs::vdev\nr#4"]; z0xfffff8000e0bf400 [label="r1w1e1"]; z0xfffff8000e0bf400 -> z0xfffff8000e08f000; z0xfffff8000ebd1600 -> z0xfffff8000e0bf400; z0xfffff8000e0bf380 [label="r1w1e1"]; z0xfffff8000e0bf380 -> z0xfffff8000e08f700; z0xfffff8000ebd1600 -> z0xfffff8000e0bf380; z0xfffff8000e0bf300 [label="r1w1e1"]; z0xfffff8000e0bf300 -> z0xfffff8000e0a2300; z0xfffff8000ebd1600 -> z0xfffff8000e0bf300; z0xfffff8000e0bea00 [label="r1w1e1"]; z0xfffff8000e0bea00 -> z0xfffff8000e0c6b00; z0xfffff8000ebd1600 -> z0xfffff8000e0bea00; z0xfffff8000e0be980 [label="r1w1e1"]; z0xfffff8000e0be980 -> z0xfffff8000e07d500; z0xfffff8000ebd1600 -> z0xfffff8000e0be980; z0xfffff8000e050f00 [label="r1w1e1"]; z0xfffff8000e050f00 -> z0xfffff8000e073b00; z0xfffff8000ebd1600 -> z0xfffff8000e050f00; } kern.geom.conftxt: 0 DISK da5 146780121600 512 hd 255 sc 63 1 PART da5p3 143558337536 512 i 3 o 3221767168 ty freebsd-zfs xs GPT xt 516e7cba-6ecf-11d6-8ff8-00022d09712b 2 LABEL gptid/f7a4d13a-74a3-11e3-8040-00237d9e6e8a 143558337536 512 i 0 o 0 1 PART da5p2 3221225472 512 i 2 o 541696 ty freebsd-swap xs GPT xt 516e7cb5-6ecf-11d6-8ff8-00022d09712b 2 LABEL gpt/swap5 3221225472 512 i 0 o 0 1 PART da5p1 524288 512 i 1 o 17408 ty freebsd-boot xs GPT xt 83bd6b9d-7f41-11dc-be0b-001560b84f0f 2 LABEL gptid/f76af2cc-74a3-11e3-8040-00237d9e6e8a 524288 512 i 0 o 0 2 LABEL gpt/gptboot5 524288 512 i 0 o 0 0 DISK da4 146780121600 512 hd 255 sc 63 1 PART da4p3 143558337536 512 i 3 o 3221767168 ty freebsd-zfs xs GPT xt 516e7cba-6ecf-11d6-8ff8-00022d09712b 2 LABEL gptid/f7043103-74a3-11e3-8040-00237d9e6e8a 143558337536 512 i 0 o 0 1 PART da4p2 3221225472 512 i 2 o 541696 ty freebsd-swap xs GPT xt 516e7cb5-6ecf-11d6-8ff8-00022d09712b 2 LABEL gpt/swap4 3221225472 512 i 0 o 0 1 PART da4p1 524288 512 i 1 o 17408 ty freebsd-boot xs GPT xt 83bd6b9d-7f41-11dc-be0b-001560b84f0f 2 LABEL gptid/f6c16edb-74a3-11e3-8040-00237d9e6e8a 524288 512 i 0 o 0 2 LABEL gpt/gptboot4 524288 512 i 0 o 0 0 DISK da3 146780121600 512 hd 255 sc 63 1 PART da3p3 143558337536 512 i 3 o 3221767168 ty freebsd-zfs xs GPT xt 516e7cba-6ecf-11d6-8ff8-00022d09712b 2 LABEL gptid/f6600fef-74a3-11e3-8040-00237d9e6e8a 143558337536 512 i 0 o 0 1 PART da3p2 3221225472 512 i 2 o 541696 ty freebsd-swap xs GPT xt 516e7cb5-6ecf-11d6-8ff8-00022d09712b 2 LABEL gpt/swap3 3221225472 512 i 0 o 0 1 PART da3p1 524288 512 i 1 o 17408 ty freebsd-boot xs GPT xt 83bd6b9d-7f41-11dc-be0b-001560b84f0f 2 LABEL gptid/f623a95d-74a3-11e3-8040-00237d9e6e8a 524288 512 i 0 o 0 2 LABEL gpt/gptboot3 524288 512 i 0 o 0 0 DISK da2 146780121600 512 hd 255 sc 63 1 PART da2p3 143558337536 512 i 3 o 3221767168 ty freebsd-zfs xs GPT xt 516e7cba-6ecf-11d6-8ff8-00022d09712b 2 LABEL gptid/f5bd173e-74a3-11e3-8040-00237d9e6e8a 143558337536 512 i 0 o 0 1 PART da2p2 3221225472 512 i 2 o 541696 ty freebsd-swap xs GPT xt 516e7cb5-6ecf-11d6-8ff8-00022d09712b 2 LABEL gpt/swap2 3221225472 512 i 0 o 0 1 PART da2p1 524288 512 i 1 o 17408 ty freebsd-boot xs GPT xt 83bd6b9d-7f41-11dc-be0b-001560b84f0f 2 LABEL gptid/f57bb25e-74a3-11e3-8040-00237d9e6e8a 524288 512 i 0 o 0 2 LABEL gpt/gptboot2 524288 512 i 0 o 0 0 DISK da1 146780121600 512 hd 255 sc 63 1 PART da1p3 143558337536 512 i 3 o 3221767168 ty freebsd-zfs xs GPT xt 516e7cba-6ecf-11d6-8ff8-00022d09712b 2 LABEL gptid/f52330df-74a3-11e3-8040-00237d9e6e8a 143558337536 512 i 0 o 0 1 PART da1p2 3221225472 512 i 2 o 541696 ty freebsd-swap xs GPT xt 516e7cb5-6ecf-11d6-8ff8-00022d09712b 2 LABEL gpt/swap1 3221225472 512 i 0 o 0 1 PART da1p1 524288 512 i 1 o 17408 ty freebsd-boot xs GPT xt 83bd6b9d-7f41-11dc-be0b-001560b84f0f 2 LABEL gptid/f4dbf3c8-74a3-11e3-8040-00237d9e6e8a 524288 512 i 0 o 0 2 LABEL gpt/gptboot1 524288 512 i 0 o 0 0 DISK da0 146780121600 512 hd 255 sc 63 1 PART da0p3 143558337536 512 i 3 o 3221767168 ty freebsd-zfs xs GPT xt 516e7cba-6ecf-11d6-8ff8-00022d09712b 2 LABEL gptid/f471cf71-74a3-11e3-8040-00237d9e6e8a 143558337536 512 i 0 o 0 1 PART da0p2 3221225472 512 i 2 o 541696 ty freebsd-swap xs GPT xt 516e7cb5-6ecf-11d6-8ff8-00022d09712b 2 LABEL gpt/swap0 3221225472 512 i 0 o 0 1 PART da0p1 524288 512 i 1 o 17408 ty freebsd-boot xs GPT xt 83bd6b9d-7f41-11dc-be0b-001560b84f0f 2 LABEL gptid/f4308ed8-74a3-11e3-8040-00237d9e6e8a 524288 512 i 0 o 0 2 LABEL gpt/gptboot0 524288 512 i 0 o 0 0 DISK cd0 0 2048 hd 0 sc 0 kern.geom.debugflags: 0 kern.geom.notaste: 0 kern.geom.collectstats: 1 kern.geom.label.debug: 0 kern.geom.label.ext2fs.enable: 1 kern.geom.label.iso9660.enable: 1 kern.geom.label.msdosfs.enable: 1 kern.geom.label.ntfs.enable: 1 kern.geom.label.reiserfs.enable: 1 kern.geom.label.ufsid.enable: 1 kern.geom.label.ufs.enable: 1 kern.geom.label.gpt.enable: 1 kern.geom.label.gptid.enable: 1 kern.geom.label.disk_ident.enable: 1 kern.geom.part.check_integrity: 1 kern.geom.raid.enable: 1 kern.geom.raid.aggressive_spare: 0 kern.geom.raid.debug: 0 kern.geom.raid.read_err_thresh: 10 kern.geom.raid.start_timeout: 30 kern.geom.raid.clean_time: 5 kern.geom.raid.disconnect_on_failure: 1 kern.geom.raid.name_format: 0 kern.geom.raid.idle_threshold: 1000000 kern.geom.raid.legacy_aliases: 1 kern.geom.raid.ddf.enable: 1 kern.geom.raid.intel.enable: 1 kern.geom.raid.jmicron.enable: 1 kern.geom.raid.nvidia.enable: 1 kern.geom.raid.promise.enable: 1 kern.geom.raid.sii.enable: 1 kern.geom.raid.concat.enable: 1 kern.geom.raid.raid0.enable: 1 kern.geom.raid.raid1.rebuild_slab_size: 1048576 kern.geom.raid.raid1.rebuild_fair_io: 20 kern.geom.raid.raid1.rebuild_cluster_idle: 100 kern.geom.raid.raid1.rebuild_meta_update: 1024 kern.geom.raid.raid1.enable: 1 kern.geom.raid.raid1e.rebuild_slab_size: 1048576 kern.geom.raid.raid1e.rebuild_fair_io: 20 kern.geom.raid.raid1e.rebuild_cluster_idle: 100 kern.geom.raid.raid1e.rebuild_meta_update: 1024 kern.geom.raid.raid1e.enable: 1 kern.geom.raid.raid5.enable: 1 kern.elf64.fallback_brand: -1 kern.elf64.nxstack: 1 kern.elf32.fallback_brand: 3 kern.elf32.nxstack: 1 kern.elf32.read_exec: 0 kern.init_path: /sbin/init:/sbin/oinit:/sbin/init.bak:/rescue/init kern.init_shutdown_timeout: 120 kern.acct_suspend: 2 kern.acct_resume: 4 kern.acct_chkfreq: 15 kern.acct_configured: 0 kern.acct_suspended: 0 kern.cp_time: 1675366 79749892 1625346 183298 80230472 kern.cp_times: 261018 10493266 193063 19759 9465974 250977 9766695 183941 4359 10227070 259278 9748209 186754 4824 10233977 247299 9754175 186131 4971 10240466 157494 9919544 229828 17361 10108815 161377 10215089 218165 49537 9788874 155482 10040127 217898 76396 9943139 182441 9812787 209566 6091 10222157 kern.console: ttyv0,/ttyv0,uart,ttyv0, kern.consmute: 0 kern.consmsgbuf_size: 8192 kern.constty_wakeups_per_second: 5 kern.vty: sc kern.openfiles: 849 kern.eventtimer.et.LAPIC.flags: 15 kern.eventtimer.et.LAPIC.frequency: 166676115 kern.eventtimer.et.LAPIC.quality: 400 kern.eventtimer.et.i8254.flags: 1 kern.eventtimer.et.i8254.frequency: 1193182 kern.eventtimer.et.i8254.quality: 100 kern.eventtimer.et.HPET.flags: 3 kern.eventtimer.et.HPET.frequency: 14318180 kern.eventtimer.et.HPET.quality: 350 kern.eventtimer.et.HPET1.flags: 3 kern.eventtimer.et.HPET1.frequency: 14318180 kern.eventtimer.et.HPET1.quality: 340 kern.eventtimer.et.HPET2.flags: 3 kern.eventtimer.et.HPET2.frequency: 14318180 kern.eventtimer.et.HPET2.quality: 340 kern.eventtimer.et.RTC.flags: 17 kern.eventtimer.et.RTC.frequency: 32768 kern.eventtimer.et.RTC.quality: 0 kern.eventtimer.choice: LAPIC(400) HPET(350) HPET1(340) HPET2(340) i8254(100) RTC(0) kern.eventtimer.singlemul: 2 kern.eventtimer.idletick: 0 kern.eventtimer.timer: LAPIC kern.eventtimer.periodic: 0 kern.kq_calloutmax: 4096 kern.stackprot: 7 kern.ps_arg_cache_limit: 256 kern.disallow_high_osrel: 0 kern.lastpid: 19289 kern.randompid: 0 kern.ktrace.genio_size: 4096 kern.ktrace.request_pool: 100 kern.module_path: /boot/kernel;/boot/modules kern.malloc_count: 368 kern.ident: GENERIC kern.compiler_version: FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 kern.maxusers: 1358 kern.supported_archs: amd64 i386 kern.conftxt: options CONFIG_AUTOGENERATED ident GENERIC machine amd64 cpu HAMMER makeoptions WITH_CTF=1 makeoptions DEBUG=-g options XENHVM options USB_DEBUG options ATH_ENABLE_11N options AH_AR5416_INTERRUPT_MITIGATION options AH_SUPPORT_AR5416 options IEEE80211_SUPPORT_MESH options IEEE80211_AMPDU_AGE options IEEE80211_DEBUG options SC_PIXEL_MODE options VESA options AHD_REG_PRETTY_PRINT options AHC_REG_PRETTY_PRINT options ATA_STATIC_ID options ACPI_DMAR options SMP options KDB_TRACE options KDB options INCLUDE_CONFIG_FILE options DDB_CTF options KDTRACE_HOOKS options KDTRACE_FRAME options MAC options PROCDESC options CAPABILITIES options CAPABILITY_MODE options AUDIT options HWPMC_HOOKS options KBD_INSTALL_CDEV options PRINTF_BUFR_SIZE=128 options _KPOSIX_PRIORITY_SCHEDULING options SYSVSEM options SYSVMSG options SYSVSHM options STACK options KTRACE options SCSI_DELAY=5000 options COMPAT_FREEBSD7 options COMPAT_FREEBSD6 options COMPAT_FREEBSD5 options COMPAT_FREEBSD4 options COMPAT_FREEBSD32 options GEOM_LABEL options GEOM_RAID options GEOM_PART_GPT options PSEUDOFS options PROCFS options CD9660 options MSDOSFS options NFS_ROOT options NFSLOCKD options NFSD options NFSCL options MD_ROOT options QUOTA options UFS_GJOURNAL options UFS_DIRHASH options UFS_ACL options SOFTUPDATES options FFS options SCTP options TCP_OFFLOAD options INET6 options INET options PREEMPTION options SCHED_ULE options NEW_PCIB options GEOM_PART_MBR options GEOM_PART_EBR_COMPAT options GEOM_PART_EBR options GEOM_PART_BSD device isa device mem device io device uart_ns8250 device cpufreq device acpi device pci device fdc device ahci device ata device mvs device siis device ahc device ahd device esp device hptiop device isp device mpt device mps device mpr device sym device trm device adv device adw device aic device bt device isci device scbus device ch device da device sa device cd device pass device ses device amr device arcmsr device ciss device dpt device hptmv device hptnr device hptrr device hpt27xx device iir device ips device mly device twa device tws device aac device aacp device aacraid device ida device mfi device mlx device twe device atkbdc device atkbd device psm device kbdmux device vga device splash device sc device vt device vt_vga device agp device cbb device pccard device cardbus device uart device ppc device ppbus device lpt device ppi device puc device bxe device de device em device igb device ixgbe device le device ti device txp device vx device miibus device ae device age device alc device ale device bce device bfe device bge device cas device dc device et device fxp device gem device hme device jme device lge device msk device nfe device nge device pcn device re device rl device sf device sge device sis device sk device ste device stge device tl device tx device vge device vr device wb device xl device cs device ed device ex device ep device fe device sn device xe device wlan device wlan_wep device wlan_ccmp device wlan_tkip device wlan_amrr device an device ath device ath_pci device ath_hal device ath_rate_sample device ipw device iwi device iwn device malo device mwl device ral device wi device wpi device loop device random device padlock_rng device rdrand_rng device ether device vlan device tun device md device gif device faith device firmware device bpf device uhci device ohci device ehci device xhci device usb device ukbd device umass device sound device snd_cmi device snd_csa device snd_emu10kx device snd_es137x device snd_hda device snd_ich device snd_via8233 device mmc device mmcsd device sdhci device virtio device virtio_pci device vtnet device virtio_blk device virtio_scsi device virtio_balloon device hyperv device xenpci device vmx kern.features.scbus: 1 kern.features.compat_freebsd_32bit: 1 kern.features.ata_cam: 1 kern.features.nfscl: 1 kern.features.nfsd: 1 kern.features.geom_label: 1 kern.features.geom_part_bsd: 1 kern.features.geom_part_ebr: 1 kern.features.geom_part_ebr_compat: 1 kern.features.geom_part_gpt: 1 kern.features.geom_part_mbr: 1 kern.features.kposix_priority_scheduling: 1 kern.features.kdtrace_hooks: 1 kern.features.ktrace: 1 kern.features.compat_freebsd4: 1 kern.features.compat_freebsd5: 1 kern.features.compat_freebsd6: 1 kern.features.compat_freebsd7: 1 kern.features.hwpmc_hooks: 1 kern.features.stack: 1 kern.features.security_capability_mode: 1 kern.features.security_capabilities: 1 kern.features.process_descriptors: 1 kern.features.sysv_msg: 1 kern.features.sysv_sem: 1 kern.features.sysv_shm: 1 kern.features.posix_shm: 1 kern.features.inet: 1 kern.features.inet6: 1 kern.features.audit: 1 kern.features.security_mac: 1 kern.features.ffs_snapshot: 1 kern.features.softupdates: 1 kern.features.ufs_acl: 1 kern.features.ufs_gjournal: 1 kern.features.ufs_quota: 1 kern.features.ufs_quota64: 1 kern.features.linuxulator_v4l: 1 kern.features.linuxulator_v4l2: 1 kern.pid_max: 99999 kern.fallback_elf_brand: -1 kern.hwpmc.softevents: 16 kern.hwpmc.callchaindepth: 16 kern.hwpmc.hashsize: 1024 kern.hwpmc.nsamples: 1024 kern.hwpmc.mtxpoolsize: 2048 kern.hwpmc.logbuffersize: 4 kern.hwpmc.nbuffers: 1024 kern.kstack_pages: 4 kern.panic_reboot_wait_time: 15 kern.sync_on_panic: 0 kern.shutdown.show_busybufs: 0 kern.shutdown.poweroff_delay: 5000 kern.shutdown.kproc_shutdown_wait: 60 kern.shutdown.dumpdevname: gpt/swap0 kern.forcesigexit: 1 kern.sigqueue.max_pending_per_proc: 128 kern.sigqueue.preallocate: 1024 kern.sigqueue.overflow: 2183567 kern.sigqueue.alloc_fail: 0 kern.sugid_coredump: 0 kern.capmode_coredump: 0 kern.coredump: 1 kern.nodump_coredump: 0 kern.corefile: %N.core kern.fscale: 2048 kern.timecounter.tc.i8254.mask: 65535 kern.timecounter.tc.i8254.counter: 24876 kern.timecounter.tc.i8254.frequency: 1193182 kern.timecounter.tc.i8254.quality: 0 kern.timecounter.tc.HPET.mask: 4294967295 kern.timecounter.tc.HPET.counter: 1590263785 kern.timecounter.tc.HPET.frequency: 14318180 kern.timecounter.tc.HPET.quality: 950 kern.timecounter.tc.ACPI-fast.mask: 16777215 kern.timecounter.tc.ACPI-fast.counter: 8349275 kern.timecounter.tc.ACPI-fast.frequency: 3579545 kern.timecounter.tc.ACPI-fast.quality: 900 kern.timecounter.tc.TSC.mask: 4294967295 kern.timecounter.tc.TSC.counter: 1680884271 kern.timecounter.tc.TSC.frequency: 2500140705 kern.timecounter.tc.TSC.quality: -1000 kern.timecounter.stepwarnings: 0 kern.timecounter.alloweddeviation: 5 kern.timecounter.hardware: HPET kern.timecounter.choice: TSC(-1000) ACPI-fast(900) HPET(950) i8254(0) dummy(-1000000) kern.timecounter.tick: 1 kern.timecounter.fast_gettime: 1 kern.timecounter.invariant_tsc: 1 kern.timecounter.smp_tsc: 0 kern.timecounter.smp_tsc_adjust: 0 kern.timecounter.tsc_shift: 1 kern.threads.max_threads_per_proc: 1500 kern.threads.max_threads_hits: 0 kern.ncallout: 18508 kern.callout_stat: 0 kern.sched.cpusetsize: 8 kern.sched.preemption: 1 kern.sched.name: ULE kern.sched.quantum: 94488 kern.sched.slice: 12 kern.sched.interact: 30 kern.sched.preempt_thresh: 80 kern.sched.static_boost: 152 kern.sched.idlespins: 10000 kern.sched.idlespinthresh: 157 kern.sched.affinity: 1 kern.sched.balance: 1 kern.sched.balance_interval: 127 kern.sched.steal_idle: 1 kern.sched.steal_thresh: 2 kern.sched.topology_spec: 0, 1, 2, 3, 4, 5, 6, 7 0, 1, 2, 3 4, 5, 6, 7 kern.ccpu: 0 kern.devstat.numdevs: 14 kern.devstat.generation: 570 kern.devstat.version: 6 kern.hintmode: 0 kern.kobj_methodcount: 232 kern.log_wakeups_per_second: 5 kern.msgbuf_show_timestamp: 0 kern.hz: 1000 kern.nbuf: 105332 kern.nswbuf: 256 kern.msgbufsize: 98304 kern.maxswzone: 0 kern.maxbcache: 0 kern.bio_transient_maxcnt: 1024 kern.maxtsiz: 134217728 kern.dfldsiz: 34359738368 kern.maxdsiz: 34359738368 kern.dflssiz: 8388608 kern.maxssiz: 536870912 kern.sgrowsiz: 131072 kern.vm_guest: none kern.log_console_output: 1 kern.log_console_add_linefeed: 0 kern.always_console_output: 0 kern.msgbuf: d[92200]: login_getclass: unknown class 'root' <118>Jul 24 21:29:44 thebighonker sshd[92200]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:44 thebighonker sshd[92200]: login_getclass: unknown class 'root' <118>Jul 24 21:29:44 thebighonker sshd[92206]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:29:45 thebighonker sshd[92207]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:45 thebighonker sshd[92207]: login_getclass: unknown class 'root' <118>Jul 24 21:29:45 thebighonker sshd[92207]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:45 thebighonker sshd[92207]: login_getclass: unknown class 'root' <118>Jul 24 21:29:45 thebighonker sshd[92207]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:45 thebighonker sshd[92207]: login_getclass: unknown class 'root' <118>Jul 24 21:29:46 thebighonker sshd[92207]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:46 thebighonker sshd[92207]: login_getclass: unknown class 'root' <118>Jul 24 21:29:46 thebighonker sshd[92207]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:46 thebighonker sshd[92207]: login_getclass: unknown class 'root' <118>Jul 24 21:29:46 thebighonker sshd[92207]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:46 thebighonker sshd[92207]: login_getclass: unknown class 'root' <118>Jul 24 21:29:46 thebighonker sshd[92207]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:46 thebighonker sshd[92207]: login_getclass: unknown class 'root' <118>Jul 24 21:29:46 thebighonker sshd[92208]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:29:47 thebighonker sshd[92209]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:47 thebighonker sshd[92209]: login_getclass: unknown class 'root' <118>Jul 24 21:29:47 thebighonker sshd[92209]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:47 thebighonker sshd[92209]: login_getclass: unknown class 'root' <118>Jul 24 21:29:47 thebighonker sshd[92209]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:47 thebighonker sshd[92209]: login_getclass: unknown class 'root' <118>Jul 24 21:29:47 thebighonker sshd[92209]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:47 thebighonker sshd[92209]: login_getclass: unknown class 'root' <118>Jul 24 21:29:48 thebighonker sshd[92209]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:48 thebighonker sshd[92209]: login_getclass: unknown class 'root' <118>Jul 24 21:29:48 thebighonker sshd[92209]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:48 thebighonker sshd[92209]: login_getclass: unknown class 'root' <118>Jul 24 21:29:48 thebighonker sshd[92209]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:48 thebighonker sshd[92209]: login_getclass: unknown class 'root' <118>Jul 24 21:29:48 thebighonker sshd[92210]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:29:49 thebighonker sshd[92211]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:49 thebighonker sshd[92211]: login_getclass: unknown class 'root' <118>Jul 24 21:29:49 thebighonker sshd[92211]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:49 thebighonker sshd[92211]: login_getclass: unknown class 'root' <118>Jul 24 21:29:49 thebighonker sshd[92211]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:49 thebighonker sshd[92211]: login_getclass: unknown class 'root' <118>Jul 24 21:29:49 thebighonker sshd[92211]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:49 thebighonker sshd[92211]: login_getclass: unknown class 'root' <118>Jul 24 21:29:49 thebighonker sshd[92211]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:49 thebighonker sshd[92211]: login_getclass: unknown class 'root' <118>Jul 24 21:29:49 thebighonker sshd[92211]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:49 thebighonker sshd[92211]: login_getclass: unknown class 'root' <118>Jul 24 21:29:50 thebighonker sshd[92211]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:50 thebighonker sshd[92211]: login_getclass: unknown class 'root' <118>Jul 24 21:29:50 thebighonker sshd[92213]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:29:51 thebighonker sshd[92214]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:51 thebighonker sshd[92214]: login_getclass: unknown class 'root' <118>Jul 24 21:29:51 thebighonker sshd[92214]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:51 thebighonker sshd[92214]: login_getclass: unknown class 'root' <118>Jul 24 21:29:51 thebighonker sshd[92214]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:51 thebighonker sshd[92214]: login_getclass: unknown class 'root' <118>Jul 24 21:29:51 thebighonker sshd[92214]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:51 thebighonker sshd[92214]: login_getclass: unknown class 'root' <118>Jul 24 21:29:51 thebighonker sshd[92214]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:51 thebighonker sshd[92214]: login_getclass: unknown class 'root' <118>Jul 24 21:29:51 thebighonker sshd[92214]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:51 thebighonker sshd[92214]: login_getclass: unknown class 'root' <118>Jul 24 21:29:52 thebighonker sshd[92214]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:52 thebighonker sshd[92214]: login_getclass: unknown class 'root' <118>Jul 24 21:29:52 thebighonker sshd[92216]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:29:53 thebighonker sshd[92217]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:53 thebighonker sshd[92217]: login_getclass: unknown class 'root' <118>Jul 24 21:29:53 thebighonker sshd[92217]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:53 thebighonker sshd[92217]: login_getclass: unknown class 'root' <118>Jul 24 21:29:53 thebighonker sshd[92217]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:53 thebighonker sshd[92217]: login_getclass: unknown class 'root' <118>Jul 24 21:29:54 thebighonker sshd[92217]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:54 thebighonker sshd[92217]: login_getclass: unknown class 'root' <118>Jul 24 21:29:54 thebighonker sshd[92217]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:54 thebighonker sshd[92217]: login_getclass: unknown class 'root' <118>Jul 24 21:29:54 thebighonker sshd[92217]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:54 thebighonker sshd[92217]: login_getclass: unknown class 'root' <118>Jul 24 21:29:54 thebighonker sshd[92217]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:54 thebighonker sshd[92217]: login_getclass: unknown class 'root' <118>Jul 24 21:29:54 thebighonker sshd[92225]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:29:56 thebighonker sshd[92226]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:56 thebighonker sshd[92226]: login_getclass: unknown class 'root' <118>Jul 24 21:29:56 thebighonker sshd[92226]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:56 thebighonker sshd[92226]: login_getclass: unknown class 'root' <118>Jul 24 21:29:56 thebighonker sshd[92226]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:56 thebighonker sshd[92226]: login_getclass: unknown class 'root' <118>Jul 24 21:29:56 thebighonker sshd[92226]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:56 thebighonker sshd[92226]: login_getclass: unknown class 'root' <118>Jul 24 21:29:56 thebighonker sshd[92226]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:56 thebighonker sshd[92226]: login_getclass: unknown class 'root' <118>Jul 24 21:29:56 thebighonker sshd[92226]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:56 thebighonker sshd[92226]: login_getclass: unknown class 'root' <118>Jul 24 21:29:56 thebighonker sshd[92226]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:56 thebighonker sshd[92226]: login_getclass: unknown class 'root' <118>Jul 24 21:29:57 thebighonker sshd[92227]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:29:57 thebighonker sshd[92228]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:57 thebighonker sshd[92228]: login_getclass: unknown class 'root' <118>Jul 24 21:29:58 thebighonker sshd[92228]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:58 thebighonker sshd[92228]: login_getclass: unknown class 'root' <118>Jul 24 21:29:58 thebighonker sshd[92228]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:58 thebighonker sshd[92228]: login_getclass: unknown class 'root' <118>Jul 24 21:29:58 thebighonker sshd[92228]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:58 thebighonker sshd[92228]: login_getclass: unknown class 'root' <118>Jul 24 21:29:58 thebighonker sshd[92228]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:58 thebighonker sshd[92228]: login_getclass: unknown class 'root' <118>Jul 24 21:29:58 thebighonker sshd[92228]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:58 thebighonker sshd[92228]: login_getclass: unknown class 'root' <118>Jul 24 21:29:58 thebighonker sshd[92228]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:29:58 thebighonker sshd[92228]: login_getclass: unknown class 'root' <118>Jul 24 21:29:59 thebighonker sshd[92234]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:30:00 thebighonker sshd[92235]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:00 thebighonker sshd[92235]: login_getclass: unknown class 'root' <118>Jul 24 21:30:00 thebighonker sshd[92235]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:00 thebighonker sshd[92235]: login_getclass: unknown class 'root' <118>Jul 24 21:30:00 thebighonker sshd[92235]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:00 thebighonker sshd[92235]: login_getclass: unknown class 'root' <118>Jul 24 21:30:00 thebighonker sshd[92235]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:00 thebighonker sshd[92235]: login_getclass: unknown class 'root' <118>Jul 24 21:30:00 thebighonker sshd[92235]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:00 thebighonker sshd[92235]: login_getclass: unknown class 'root' <118>Jul 24 21:30:00 thebighonker sshd[92235]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:00 thebighonker sshd[92235]: login_getclass: unknown class 'root' <118>Jul 24 21:30:00 thebighonker sshd[92235]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:00 thebighonker sshd[92235]: login_getclass: unknown class 'root' <118>Jul 24 21:30:01 thebighonker sshd[92257]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:30:03 thebighonker sshd[92260]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:03 thebighonker sshd[92260]: login_getclass: unknown class 'root' <118>Jul 24 21:30:03 thebighonker sshd[92260]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:03 thebighonker sshd[92260]: login_getclass: unknown class 'root' <118>Jul 24 21:30:03 thebighonker sshd[92260]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:03 thebighonker sshd[92260]: login_getclass: unknown class 'root' <118>Jul 24 21:30:04 thebighonker sshd[92260]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:04 thebighonker sshd[92260]: login_getclass: unknown class 'root' <118>Jul 24 21:30:04 thebighonker sshd[92260]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:04 thebighonker sshd[92260]: login_getclass: unknown class 'root' <118>Jul 24 21:30:04 thebighonker sshd[92260]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:04 thebighonker sshd[92260]: login_getclass: unknown class 'root' <118>Jul 24 21:30:05 thebighonker sshd[92260]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:05 thebighonker sshd[92260]: login_getclass: unknown class 'root' <118>Jul 24 21:30:05 thebighonker sshd[92262]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:30:07 thebighonker sshd[92263]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:07 thebighonker sshd[92263]: login_getclass: unknown class 'root' <118>Jul 24 21:30:08 thebighonker sshd[92263]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:08 thebighonker sshd[92263]: login_getclass: unknown class 'root' <118>Jul 24 21:30:08 thebighonker sshd[92263]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:08 thebighonker sshd[92263]: login_getclass: unknown class 'root' <118>Jul 24 21:30:08 thebighonker sshd[92263]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:08 thebighonker sshd[92263]: login_getclass: unknown class 'root' <118>Jul 24 21:30:08 thebighonker sshd[92263]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:08 thebighonker sshd[92263]: login_getclass: unknown class 'root' <118>Jul 24 21:30:08 thebighonker sshd[92263]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:08 thebighonker sshd[92263]: login_getclass: unknown class 'root' <118>Jul 24 21:30:08 thebighonker sshd[92263]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:08 thebighonker sshd[92263]: login_getclass: unknown class 'root' <118>Jul 24 21:30:08 thebighonker sshd[92265]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:30:09 thebighonker sshd[92266]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:09 thebighonker sshd[92266]: login_getclass: unknown class 'root' <118>Jul 24 21:30:09 thebighonker sshd[92266]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:09 thebighonker sshd[92266]: login_getclass: unknown class 'root' <118>Jul 24 21:30:09 thebighonker sshd[92266]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:09 thebighonker sshd[92266]: login_getclass: unknown class 'root' <118>Jul 24 21:30:09 thebighonker sshd[92266]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:09 thebighonker sshd[92266]: login_getclass: unknown class 'root' <118>Jul 24 21:30:09 thebighonker sshd[92266]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:09 thebighonker sshd[92266]: login_getclass: unknown class 'root' <118>Jul 24 21:30:10 thebighonker sshd[92266]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:10 thebighonker sshd[92266]: login_getclass: unknown class 'root' <118>Jul 24 21:30:10 thebighonker sshd[92266]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:10 thebighonker sshd[92266]: login_getclass: unknown class 'root' <118>Jul 24 21:30:10 thebighonker sshd[92267]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:30:11 thebighonker sshd[92268]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:11 thebighonker sshd[92268]: login_getclass: unknown class 'root' <118>Jul 24 21:30:11 thebighonker sshd[92268]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:11 thebighonker sshd[92268]: login_getclass: unknown class 'root' <118>Jul 24 21:30:11 thebighonker sshd[92268]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:11 thebighonker sshd[92268]: login_getclass: unknown class 'root' <118>Jul 24 21:30:11 thebighonker sshd[92268]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:11 thebighonker sshd[92268]: login_getclass: unknown class 'root' <118>Jul 24 21:30:11 thebighonker sshd[92268]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:11 thebighonker sshd[92268]: login_getclass: unknown class 'root' <118>Jul 24 21:30:11 thebighonker sshd[92268]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:11 thebighonker sshd[92268]: login_getclass: unknown class 'root' <118>Jul 24 21:30:11 thebighonker sshd[92268]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:11 thebighonker sshd[92268]: login_getclass: unknown class 'root' <118>Jul 24 21:30:12 thebighonker sshd[92278]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:30:13 thebighonker sshd[92279]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:13 thebighonker sshd[92279]: login_getclass: unknown class 'root' <118>Jul 24 21:30:13 thebighonker sshd[92279]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:13 thebighonker sshd[92279]: login_getclass: unknown class 'root' <118>Jul 24 21:30:13 thebighonker sshd[92279]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:13 thebighonker sshd[92279]: login_getclass: unknown class 'root' <118>Jul 24 21:30:13 thebighonker sshd[92279]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:13 thebighonker sshd[92279]: login_getclass: unknown class 'root' <118>Jul 24 21:30:13 thebighonker sshd[92279]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:13 thebighonker sshd[92279]: login_getclass: unknown class 'root' <118>Jul 24 21:30:13 thebighonker sshd[92279]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:13 thebighonker sshd[92279]: login_getclass: unknown class 'root' <118>Jul 24 21:30:13 thebighonker sshd[92279]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:13 thebighonker sshd[92279]: login_getclass: unknown class 'root' <118>Jul 24 21:30:13 thebighonker sshd[92280]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:30:14 thebighonker sshd[92281]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:14 thebighonker sshd[92281]: login_getclass: unknown class 'root' <118>Jul 24 21:30:14 thebighonker sshd[92281]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:14 thebighonker sshd[92281]: login_getclass: unknown class 'root' <118>Jul 24 21:30:14 thebighonker sshd[92281]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:14 thebighonker sshd[92281]: login_getclass: unknown class 'root' <118>Jul 24 21:30:14 thebighonker sshd[92281]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:14 thebighonker sshd[92281]: login_getclass: unknown class 'root' <118>Jul 24 21:30:14 thebighonker sshd[92281]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:14 thebighonker sshd[92281]: login_getclass: unknown class 'root' <118>Jul 24 21:30:15 thebighonker sshd[92281]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:15 thebighonker sshd[92281]: login_getclass: unknown class 'root' <118>Jul 24 21:30:15 thebighonker sshd[92281]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:15 thebighonker sshd[92281]: login_getclass: unknown class 'root' <118>Jul 24 21:30:15 thebighonker sshd[92284]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:30:16 thebighonker sshd[92285]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:16 thebighonker sshd[92285]: login_getclass: unknown class 'root' <118>Jul 24 21:30:16 thebighonker sshd[92285]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:16 thebighonker sshd[92285]: login_getclass: unknown class 'root' <118>Jul 24 21:30:16 thebighonker sshd[92285]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:16 thebighonker sshd[92285]: login_getclass: unknown class 'root' <118>Jul 24 21:30:16 thebighonker sshd[92285]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:16 thebighonker sshd[92285]: login_getclass: unknown class 'root' <118>Jul 24 21:30:16 thebighonker sshd[92285]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:16 thebighonker sshd[92285]: login_getclass: unknown class 'root' <118>Jul 24 21:30:16 thebighonker sshd[92285]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:16 thebighonker sshd[92285]: login_getclass: unknown class 'root' <118>Jul 24 21:30:17 thebighonker sshd[92285]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:17 thebighonker sshd[92285]: login_getclass: unknown class 'root' <118>Jul 24 21:30:17 thebighonker sshd[92286]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:30:18 thebighonker sshd[92287]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:18 thebighonker sshd[92287]: login_getclass: unknown class 'root' <118>Jul 24 21:30:18 thebighonker sshd[92287]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:18 thebighonker sshd[92287]: login_getclass: unknown class 'root' <118>Jul 24 21:30:18 thebighonker sshd[92287]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:18 thebighonker sshd[92287]: login_getclass: unknown class 'root' <118>Jul 24 21:30:18 thebighonker sshd[92287]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:18 thebighonker sshd[92287]: login_getclass: unknown class 'root' <118>Jul 24 21:30:18 thebighonker sshd[92287]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:18 thebighonker sshd[92287]: login_getclass: unknown class 'root' <118>Jul 24 21:30:18 thebighonker sshd[92287]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:18 thebighonker sshd[92287]: login_getclass: unknown class 'root' <118>Jul 24 21:30:19 thebighonker sshd[92287]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:19 thebighonker sshd[92287]: login_getclass: unknown class 'root' <118>Jul 24 21:30:19 thebighonker sshd[92288]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:30:20 thebighonker sshd[92289]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:20 thebighonker sshd[92289]: login_getclass: unknown class 'root' <118>Jul 24 21:30:20 thebighonker sshd[92289]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:20 thebighonker sshd[92289]: login_getclass: unknown class 'root' <118>Jul 24 21:30:20 thebighonker sshd[92289]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:20 thebighonker sshd[92289]: login_getclass: unknown class 'root' <118>Jul 24 21:30:20 thebighonker sshd[92289]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:20 thebighonker sshd[92289]: login_getclass: unknown class 'root' <118>Jul 24 21:30:20 thebighonker sshd[92289]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:20 thebighonker sshd[92289]: login_getclass: unknown class 'root' <118>Jul 24 21:30:20 thebighonker sshd[92289]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:20 thebighonker sshd[92289]: login_getclass: unknown class 'root' <118>Jul 24 21:30:21 thebighonker sshd[92289]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:21 thebighonker sshd[92289]: login_getclass: unknown class 'root' <118>Jul 24 21:30:22 thebighonker sshd[92290]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:30:22 thebighonker sshd[92291]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:22 thebighonker sshd[92291]: login_getclass: unknown class 'root' <118>Jul 24 21:30:22 thebighonker sshd[92291]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:22 thebighonker sshd[92291]: login_getclass: unknown class 'root' <118>Jul 24 21:30:23 thebighonker sshd[92291]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:23 thebighonker sshd[92291]: login_getclass: unknown class 'root' <118>Jul 24 21:30:23 thebighonker sshd[92291]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:23 thebighonker sshd[92291]: login_getclass: unknown class 'root' <118>Jul 24 21:30:23 thebighonker sshd[92291]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:23 thebighonker sshd[92291]: login_getclass: unknown class 'root' <118>Jul 24 21:30:23 thebighonker sshd[92291]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:23 thebighonker sshd[92291]: login_getclass: unknown class 'root' <118>Jul 24 21:30:23 thebighonker sshd[92291]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:23 thebighonker sshd[92291]: login_getclass: unknown class 'root' <118>Jul 24 21:30:23 thebighonker sshd[92293]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:30:24 thebighonker sshd[92294]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:24 thebighonker sshd[92294]: login_getclass: unknown class 'root' <118>Jul 24 21:30:25 thebighonker sshd[92294]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:25 thebighonker sshd[92294]: login_getclass: unknown class 'root' <118>Jul 24 21:30:25 thebighonker sshd[92294]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:25 thebighonker sshd[92294]: login_getclass: unknown class 'root' <118>Jul 24 21:30:25 thebighonker sshd[92294]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:25 thebighonker sshd[92294]: login_getclass: unknown class 'root' <118>Jul 24 21:30:25 thebighonker sshd[92294]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:25 thebighonker sshd[92294]: login_getclass: unknown class 'root' <118>Jul 24 21:30:25 thebighonker sshd[92294]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:25 thebighonker sshd[92294]: login_getclass: unknown class 'root' <118>Jul 24 21:30:25 thebighonker sshd[92294]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:25 thebighonker sshd[92294]: login_getclass: unknown class 'root' <118>Jul 24 21:30:26 thebighonker sshd[92295]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:30:26 thebighonker sshd[92296]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:26 thebighonker sshd[92296]: login_getclass: unknown class 'root' <118>Jul 24 21:30:27 thebighonker sshd[92296]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:27 thebighonker sshd[92296]: login_getclass: unknown class 'root' <118>Jul 24 21:30:27 thebighonker sshd[92296]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:27 thebighonker sshd[92296]: login_getclass: unknown class 'root' <118>Jul 24 21:30:27 thebighonker sshd[92296]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:27 thebighonker sshd[92296]: login_getclass: unknown class 'root' <118>Jul 24 21:30:27 thebighonker sshd[92296]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:27 thebighonker sshd[92296]: login_getclass: unknown class 'root' <118>Jul 24 21:30:27 thebighonker sshd[92296]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:27 thebighonker sshd[92296]: login_getclass: unknown class 'root' <118>Jul 24 21:30:27 thebighonker sshd[92296]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:27 thebighonker sshd[92296]: login_getclass: unknown class 'root' <118>Jul 24 21:30:27 thebighonker sshd[92297]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:30:28 thebighonker sshd[92298]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:28 thebighonker sshd[92298]: login_getclass: unknown class 'root' <118>Jul 24 21:30:28 thebighonker sshd[92298]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:28 thebighonker sshd[92298]: login_getclass: unknown class 'root' <118>Jul 24 21:30:28 thebighonker sshd[92298]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:28 thebighonker sshd[92298]: login_getclass: unknown class 'root' <118>Jul 24 21:30:28 thebighonker sshd[92298]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:28 thebighonker sshd[92298]: login_getclass: unknown class 'root' <118>Jul 24 21:30:28 thebighonker sshd[92298]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:28 thebighonker sshd[92298]: login_getclass: unknown class 'root' <118>Jul 24 21:30:28 thebighonker sshd[92298]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:28 thebighonker sshd[92298]: login_getclass: unknown class 'root' <118>Jul 24 21:30:29 thebighonker sshd[92298]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:29 thebighonker sshd[92298]: login_getclass: unknown class 'root' <118>Jul 24 21:30:29 thebighonker sshd[92299]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:30:30 thebighonker sshd[92300]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:30 thebighonker sshd[92300]: login_getclass: unknown class 'root' <118>Jul 24 21:30:30 thebighonker sshd[92300]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:30 thebighonker sshd[92300]: login_getclass: unknown class 'root' <118>Jul 24 21:30:30 thebighonker sshd[92300]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:30 thebighonker sshd[92300]: login_getclass: unknown class 'root' <118>Jul 24 21:30:30 thebighonker sshd[92300]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:30 thebighonker sshd[92300]: login_getclass: unknown class 'root' <118>Jul 24 21:30:30 thebighonker sshd[92300]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:30 thebighonker sshd[92300]: login_getclass: unknown class 'root' <118>Jul 24 21:30:30 thebighonker sshd[92300]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:30 thebighonker sshd[92300]: login_getclass: unknown class 'root' <118>Jul 24 21:30:30 thebighonker sshd[92300]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:30 thebighonker sshd[92300]: login_getclass: unknown class 'root' <118>Jul 24 21:30:30 thebighonker sshd[92301]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:30:31 thebighonker sshd[92302]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:31 thebighonker sshd[92302]: login_getclass: unknown class 'root' <118>Jul 24 21:30:31 thebighonker sshd[92302]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:31 thebighonker sshd[92302]: login_getclass: unknown class 'root' <118>Jul 24 21:30:31 thebighonker sshd[92302]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:31 thebighonker sshd[92302]: login_getclass: unknown class 'root' <118>Jul 24 21:30:32 thebighonker sshd[92302]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:32 thebighonker sshd[92302]: login_getclass: unknown class 'root' <118>Jul 24 21:30:32 thebighonker sshd[92302]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:32 thebighonker sshd[92302]: login_getclass: unknown class 'root' <118>Jul 24 21:30:32 thebighonker sshd[92302]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:32 thebighonker sshd[92302]: login_getclass: unknown class 'root' <118>Jul 24 21:30:32 thebighonker sshd[92302]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:32 thebighonker sshd[92302]: login_getclass: unknown class 'root' <118>Jul 24 21:30:32 thebighonker sshd[92304]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:30:34 thebighonker sshd[92305]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:34 thebighonker sshd[92305]: login_getclass: unknown class 'root' <118>Jul 24 21:30:34 thebighonker sshd[92305]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:34 thebighonker sshd[92305]: login_getclass: unknown class 'root' <118>Jul 24 21:30:34 thebighonker sshd[92305]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:34 thebighonker sshd[92305]: login_getclass: unknown class 'root' <118>Jul 24 21:30:34 thebighonker sshd[92305]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:34 thebighonker sshd[92305]: login_getclass: unknown class 'root' <118>Jul 24 21:30:34 thebighonker sshd[92305]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:34 thebighonker sshd[92305]: login_getclass: unknown class 'root' <118>Jul 24 21:30:34 thebighonker sshd[92305]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:34 thebighonker sshd[92305]: login_getclass: unknown class 'root' <118>Jul 24 21:30:34 thebighonker sshd[92305]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:34 thebighonker sshd[92305]: login_getclass: unknown class 'root' <118>Jul 24 21:30:35 thebighonker sshd[92306]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:30:35 thebighonker sshd[92307]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:35 thebighonker sshd[92307]: login_getclass: unknown class 'root' <118>Jul 24 21:30:36 thebighonker sshd[92307]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:36 thebighonker sshd[92307]: login_getclass: unknown class 'root' <118>Jul 24 21:30:36 thebighonker sshd[92307]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:36 thebighonker sshd[92307]: login_getclass: unknown class 'root' <118>Jul 24 21:30:36 thebighonker sshd[92307]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:36 thebighonker sshd[92307]: login_getclass: unknown class 'root' <118>Jul 24 21:30:36 thebighonker sshd[92307]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:36 thebighonker sshd[92307]: login_getclass: unknown class 'root' <118>Jul 24 21:30:36 thebighonker sshd[92307]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:36 thebighonker sshd[92307]: login_getclass: unknown class 'root' <118>Jul 24 21:30:36 thebighonker sshd[92307]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:36 thebighonker sshd[92307]: login_getclass: unknown class 'root' <118>Jul 24 21:30:37 thebighonker sshd[92308]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:30:38 thebighonker sshd[92309]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:38 thebighonker sshd[92309]: login_getclass: unknown class 'root' <118>Jul 24 21:30:38 thebighonker sshd[92309]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:38 thebighonker sshd[92309]: login_getclass: unknown class 'root' <118>Jul 24 21:30:38 thebighonker sshd[92309]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:38 thebighonker sshd[92309]: login_getclass: unknown class 'root' <118>Jul 24 21:30:38 thebighonker sshd[92309]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:38 thebighonker sshd[92309]: login_getclass: unknown class 'root' <118>Jul 24 21:30:38 thebighonker sshd[92309]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:38 thebighonker sshd[92309]: login_getclass: unknown class 'root' <118>Jul 24 21:30:38 thebighonker sshd[92309]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:38 thebighonker sshd[92309]: login_getclass: unknown class 'root' <118>Jul 24 21:30:39 thebighonker sshd[92309]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:39 thebighonker sshd[92309]: login_getclass: unknown class 'root' <118>Jul 24 21:30:39 thebighonker sshd[92310]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:30:40 thebighonker sshd[92311]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:40 thebighonker sshd[92311]: login_getclass: unknown class 'root' <118>Jul 24 21:30:40 thebighonker sshd[92311]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:40 thebighonker sshd[92311]: login_getclass: unknown class 'root' <118>Jul 24 21:30:40 thebighonker sshd[92311]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:40 thebighonker sshd[92311]: login_getclass: unknown class 'root' <118>Jul 24 21:30:41 thebighonker sshd[92311]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:41 thebighonker sshd[92311]: login_getclass: unknown class 'root' <118>Jul 24 21:30:41 thebighonker sshd[92311]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:41 thebighonker sshd[92311]: login_getclass: unknown class 'root' <118>Jul 24 21:30:41 thebighonker sshd[92311]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:41 thebighonker sshd[92311]: login_getclass: unknown class 'root' <118>Jul 24 21:30:41 thebighonker sshd[92311]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:41 thebighonker sshd[92311]: login_getclass: unknown class 'root' <118>Jul 24 21:30:41 thebighonker sshd[92313]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:30:42 thebighonker sshd[92314]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:42 thebighonker sshd[92314]: login_getclass: unknown class 'root' <118>Jul 24 21:30:42 thebighonker sshd[92314]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:42 thebighonker sshd[92314]: login_getclass: unknown class 'root' <118>Jul 24 21:30:42 thebighonker sshd[92314]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:42 thebighonker sshd[92314]: login_getclass: unknown class 'root' <118>Jul 24 21:30:42 thebighonker sshd[92314]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:42 thebighonker sshd[92314]: login_getclass: unknown class 'root' <118>Jul 24 21:30:43 thebighonker sshd[92314]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:43 thebighonker sshd[92314]: login_getclass: unknown class 'root' <118>Jul 24 21:30:43 thebighonker sshd[92314]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:43 thebighonker sshd[92314]: login_getclass: unknown class 'root' <118>Jul 24 21:30:43 thebighonker sshd[92314]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:43 thebighonker sshd[92314]: login_getclass: unknown class 'root' <118>Jul 24 21:30:43 thebighonker sshd[92317]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:30:44 thebighonker sshd[92318]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:44 thebighonker sshd[92318]: login_getclass: unknown class 'root' <118>Jul 24 21:30:44 thebighonker sshd[92318]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:44 thebighonker sshd[92318]: login_getclass: unknown class 'root' <118>Jul 24 21:30:44 thebighonker sshd[92318]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:44 thebighonker sshd[92318]: login_getclass: unknown class 'root' <118>Jul 24 21:30:44 thebighonker sshd[92318]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:44 thebighonker sshd[92318]: login_getclass: unknown class 'root' <118>Jul 24 21:30:44 thebighonker sshd[92318]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:44 thebighonker sshd[92318]: login_getclass: unknown class 'root' <118>Jul 24 21:30:44 thebighonker sshd[92318]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:44 thebighonker sshd[92318]: login_getclass: unknown class 'root' <118>Jul 24 21:30:45 thebighonker sshd[92318]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:45 thebighonker sshd[92318]: login_getclass: unknown class 'root' <118>Jul 24 21:30:45 thebighonker sshd[92322]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:30:46 thebighonker sshd[92323]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:46 thebighonker sshd[92323]: login_getclass: unknown class 'root' <118>Jul 24 21:30:47 thebighonker sshd[92323]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:47 thebighonker sshd[92323]: login_getclass: unknown class 'root' <118>Jul 24 21:30:47 thebighonker sshd[92323]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:47 thebighonker sshd[92323]: login_getclass: unknown class 'root' <118>Jul 24 21:30:47 thebighonker sshd[92323]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:47 thebighonker sshd[92323]: login_getclass: unknown class 'root' <118>Jul 24 21:30:47 thebighonker sshd[92323]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:47 thebighonker sshd[92323]: login_getclass: unknown class 'root' <118>Jul 24 21:30:48 thebighonker sshd[92323]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:48 thebighonker sshd[92323]: login_getclass: unknown class 'root' <118>Jul 24 21:30:48 thebighonker sshd[92323]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:48 thebighonker sshd[92323]: login_getclass: unknown class 'root' <118>Jul 24 21:30:48 thebighonker sshd[92324]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:30:50 thebighonker sshd[92326]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:50 thebighonker sshd[92326]: login_getclass: unknown class 'root' <118>Jul 24 21:30:50 thebighonker sshd[92326]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:50 thebighonker sshd[92326]: login_getclass: unknown class 'root' <118>Jul 24 21:30:50 thebighonker sshd[92326]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:50 thebighonker sshd[92326]: login_getclass: unknown class 'root' <118>Jul 24 21:30:50 thebighonker sshd[92326]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:50 thebighonker sshd[92326]: login_getclass: unknown class 'root' <118>Jul 24 21:30:51 thebighonker sshd[92326]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:51 thebighonker sshd[92326]: login_getclass: unknown class 'root' <118>Jul 24 21:30:51 thebighonker sshd[92326]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:51 thebighonker sshd[92326]: login_getclass: unknown class 'root' <118>Jul 24 21:30:51 thebighonker sshd[92326]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:51 thebighonker sshd[92326]: login_getclass: unknown class 'root' <118>Jul 24 21:30:52 thebighonker sshd[92327]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:30:53 thebighonker sshd[92328]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:53 thebighonker sshd[92328]: login_getclass: unknown class 'root' <118>Jul 24 21:30:53 thebighonker sshd[92328]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:53 thebighonker sshd[92328]: login_getclass: unknown class 'root' <118>Jul 24 21:30:53 thebighonker sshd[92328]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:53 thebighonker sshd[92328]: login_getclass: unknown class 'root' <118>Jul 24 21:30:53 thebighonker sshd[92328]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:53 thebighonker sshd[92328]: login_getclass: unknown class 'root' <118>Jul 24 21:30:54 thebighonker sshd[92328]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:54 thebighonker sshd[92328]: login_getclass: unknown class 'root' <118>Jul 24 21:30:54 thebighonker sshd[92328]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:54 thebighonker sshd[92328]: login_getclass: unknown class 'root' <118>Jul 24 21:30:54 thebighonker sshd[92328]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:54 thebighonker sshd[92328]: login_getclass: unknown class 'root' <118>Jul 24 21:30:55 thebighonker sshd[92329]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:30:55 thebighonker sshd[92330]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:55 thebighonker sshd[92330]: login_getclass: unknown class 'root' <118>Jul 24 21:30:56 thebighonker sshd[92330]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:56 thebighonker sshd[92330]: login_getclass: unknown class 'root' <118>Jul 24 21:30:56 thebighonker sshd[92330]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:56 thebighonker sshd[92330]: login_getclass: unknown class 'root' <118>Jul 24 21:30:56 thebighonker sshd[92330]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:56 thebighonker sshd[92330]: login_getclass: unknown class 'root' <118>Jul 24 21:30:56 thebighonker sshd[92330]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:56 thebighonker sshd[92330]: login_getclass: unknown class 'root' <118>Jul 24 21:30:56 thebighonker sshd[92330]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:56 thebighonker sshd[92330]: login_getclass: unknown class 'root' <118>Jul 24 21:30:57 thebighonker sshd[92330]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:57 thebighonker sshd[92330]: login_getclass: unknown class 'root' <118>Jul 24 21:30:57 thebighonker sshd[92331]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:30:58 thebighonker sshd[92332]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:58 thebighonker sshd[92332]: login_getclass: unknown class 'root' <118>Jul 24 21:30:58 thebighonker sshd[92332]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:58 thebighonker sshd[92332]: login_getclass: unknown class 'root' <118>Jul 24 21:30:58 thebighonker sshd[92332]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:58 thebighonker sshd[92332]: login_getclass: unknown class 'root' <118>Jul 24 21:30:58 thebighonker sshd[92332]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:58 thebighonker sshd[92332]: login_getclass: unknown class 'root' <118>Jul 24 21:30:58 thebighonker sshd[92332]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:58 thebighonker sshd[92332]: login_getclass: unknown class 'root' <118>Jul 24 21:30:58 thebighonker sshd[92332]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:58 thebighonker sshd[92332]: login_getclass: unknown class 'root' <118>Jul 24 21:30:59 thebighonker sshd[92332]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:30:59 thebighonker sshd[92332]: login_getclass: unknown class 'root' <118>Jul 24 21:30:59 thebighonker sshd[92335]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:31:00 thebighonker sshd[92340]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:31:00 thebighonker sshd[92340]: login_getclass: unknown class 'root' <118>Jul 24 21:31:00 thebighonker sshd[92340]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:31:00 thebighonker sshd[92340]: login_getclass: unknown class 'root' <118>Jul 24 21:31:00 thebighonker sshd[92340]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:31:00 thebighonker sshd[92340]: login_getclass: unknown class 'root' <118>Jul 24 21:31:00 thebighonker sshd[92340]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:31:00 thebighonker sshd[92340]: login_getclass: unknown class 'root' <118>Jul 24 21:31:00 thebighonker sshd[92340]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:31:00 thebighonker sshd[92340]: login_getclass: unknown class 'root' <118>Jul 24 21:31:01 thebighonker sshd[92340]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:31:01 thebighonker sshd[92340]: login_getclass: unknown class 'root' <118>Jul 24 21:31:01 thebighonker sshd[92340]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:31:01 thebighonker sshd[92340]: login_getclass: unknown class 'root' <118>Jul 24 21:31:01 thebighonker sshd[92341]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:31:02 thebighonker sshd[92342]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:31:02 thebighonker sshd[92342]: login_getclass: unknown class 'root' <118>Jul 24 21:31:02 thebighonker sshd[92342]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:31:02 thebighonker sshd[92342]: login_getclass: unknown class 'root' <118>Jul 24 21:31:02 thebighonker sshd[92342]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:31:02 thebighonker sshd[92342]: login_getclass: unknown class 'root' <118>Jul 24 21:31:02 thebighonker sshd[92342]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:31:02 thebighonker sshd[92342]: login_getclass: unknown class 'root' <118>Jul 24 21:31:02 thebighonker sshd[92342]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:31:02 thebighonker sshd[92342]: login_getclass: unknown class 'root' <118>Jul 24 21:31:02 thebighonker sshd[92342]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:31:02 thebighonker sshd[92342]: login_getclass: unknown class 'root' <118>Jul 24 21:31:03 thebighonker sshd[92342]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:31:03 thebighonker sshd[92342]: login_getclass: unknown class 'root' <118>Jul 24 21:31:03 thebighonker sshd[92348]: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(unassigned.psychz.net, AF_INET) failed <118>Jul 24 21:31:04 thebighonker sshd[92349]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:31:04 thebighonker sshd[92349]: login_getclass: unknown class 'root' <118>Jul 24 21:31:04 thebighonker sshd[92349]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:31:04 thebighonker sshd[92349]: login_getclass: unknown class 'root' <118>Jul 24 21:31:04 thebighonker sshd[92349]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:31:04 thebighonker sshd[92349]: login_getclass: unknown class 'root' <118>Jul 24 21:31:04 thebighonker sshd[92349]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:31:04 thebighonker sshd[92349]: login_getclass: unknown class 'root' <118>Jul 24 21:31:04 thebighonker sshd[92349]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:31:04 thebighonker sshd[92349]: login_getclass: unknown class 'root' <118>Jul 24 21:31:04 thebighonker sshd[92349]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:31:04 thebighonker sshd[92349]: login_getclass: unknown class 'root' <118>Jul 24 21:31:04 thebighonker sshd[92349]: _secure_path: cannot stat /etc/login.conf: Not permitted in capability mode <118>Jul 24 21:31:04 thebighonker sshd[92349]: login_getclass: unknown class 'root' <118>Jul 24 22:44:59 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-101-232.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:44:59 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-101-232.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:44:59 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-101-232.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:44:59 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-101-232.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:44:59 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-101-232.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:45:09 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-101-232.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:45:13 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-101-232.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:45:16 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-101-232.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:46:09 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-98-88.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:46:09 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-98-88.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:46:09 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-98-88.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:46:09 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-98-88.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:46:09 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-98-88.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:47:10 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-98-88.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:47:10 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-98-88.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:47:10 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-98-88.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:47:10 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-98-88.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:47:10 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-98-88.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:47:21 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-98-88.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:47:24 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-98-88.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:48:05 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-98-88.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:48:08 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-98-88.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:48:08 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-98-88.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:48:12 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-98-88.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:48:12 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-98-88.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:59:28 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-96-2.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:59:29 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-96-2.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:59:29 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-96-2.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:59:30 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-96-2.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:59:30 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-96-2.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:59:30 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-96-2.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:59:30 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-96-2.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:59:30 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-96-2.pools.spcsdns.net, AF_INET) failed <118>Jul 24 22:59:31 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-96-2.pools.spcsdns.net, AF_INET) failed <118>Jul 24 23:06:43 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-96-2.pools.spcsdns.net, AF_INET) failed <118>Jul 24 23:06:43 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-96-2.pools.spcsdns.net, AF_INET) failed <118>Jul 24 23:06:43 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-96-2.pools.spcsdns.net, AF_INET) failed <118>Jul 24 23:06:43 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-96-2.pools.spcsdns.net, AF_INET) failed <118>Jul 24 23:06:43 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-96-2.pools.spcsdns.net, AF_INET) failed <118>Jul 24 23:11:03 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-96-2.pools.spcsdns.net, AF_INET) failed <118>Jul 24 23:11:03 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-96-2.pools.spcsdns.net, AF_INET) failed <118>Jul 24 23:11:03 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-96-2.pools.spcsdns.net, AF_INET) failed <118>Jul 24 23:11:03 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-96-2.pools.spcsdns.net, AF_INET) failed <118>Jul 24 23:11:03 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-96-2.pools.spcsdns.net, AF_INET) failed <118>Jul 25 02:08:34 thebighonker sshd[3018]: fatal: Write failed: Broken pipe [preauth] <118>Jul 25 04:02:04 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-3-58.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:02:04 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-3-58.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:02:04 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-3-58.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:02:04 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-3-58.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:02:04 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-3-58.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:02:21 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-3-58.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:02:25 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-3-58.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:02:28 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-3-58.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:02:32 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-3-58.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:02:32 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-3-58.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:03:15 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-3-58.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:03:15 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-3-58.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:03:15 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-3-58.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:03:15 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-3-58.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:03:15 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-3-58.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:03:27 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-3-58.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:03:30 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-3-58.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:09:30 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-23.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:09:30 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-23.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:09:30 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-23.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:09:30 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-23.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:09:30 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-23.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:10:45 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-23.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:10:48 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-23.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:10:56 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-23.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:10:56 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-23.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:10:56 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-23.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:10:56 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-23.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:11:37 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-23.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:12:41 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-23.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:12:42 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-23.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:12:55 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-23.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:13:08 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-23.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:13:14 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-23.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:13:18 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-23.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:14:05 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-23.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:14:19 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-23.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:14:50 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-23.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:14:56 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-23.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:18:19 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-35.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:18:19 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-35.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:18:19 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-35.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:18:19 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-35.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:18:19 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-35.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:18:19 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-35.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:18:19 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-35.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:18:19 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-35.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:18:19 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-35.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:18:20 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-35.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:18:20 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-35.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:20:58 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-35.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:21:41 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-35.pools.spcsdns.net, AF_INET) failed <118>Jul 25 04:24:36 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-35.pools.spcsdns.net, AF_INET) failed <6>pid 9998 (wcgrid_mcm1_7.32_i6), uid 1055: exited on signal 11 (core dumped) <6>pid 9999 (wcgrid_mcm1_7.32_i6), uid 1055: exited on signal 11 (core dumped) <118>Jul 25 05:17:37 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-35.pools.spcsdns.net, AF_INET) failed <118>Jul 25 05:46:46 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-35.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:35:40 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-176.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:35:40 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-176.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:35:40 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-176.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:35:40 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-176.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:35:40 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-176.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:35:40 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-176.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:35:40 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-176.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:35:40 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-176.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:35:40 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-176.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:35:40 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-176.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:35:55 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-176.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:35:57 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-176.pools.spcsdns.net, AF_INET) failed <6>pid 13916 (wcgrid_mcm1_7.32_i6), uid 1055: exited on signal 11 (core dumped) <6>pid 13917 (wcgrid_mcm1_7.32_i6), uid 1055: exited on signal 11 (core dumped) <6>pid 13920 (wcgrid_mcm1_7.32_i6), uid 1055: exited on signal 11 (core dumped) <118>Jul 25 06:39:14 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-71.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:39:14 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-71.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:39:14 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-71.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:39:14 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-71.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:39:14 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-71.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:39:45 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-71.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:39:46 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-71.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:40:00 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-71.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:40:02 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-71.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:40:03 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-71.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:40:03 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-71.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:40:03 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-71.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:41:03 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-71.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:41:04 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-71.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:41:04 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-71.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:41:04 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-71.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:41:05 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-71.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:45:11 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-99-8.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:45:11 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-99-8.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:45:11 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-99-8.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:45:11 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-99-8.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:45:11 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-99-8.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:45:23 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-99-8.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:45:24 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-99-8.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:51:47 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-252.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:51:51 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-252.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:52:17 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-252.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:52:17 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-252.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:52:17 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-252.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:52:17 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-252.pools.spcsdns.net, AF_INET) failed <118>Jul 25 06:52:17 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-252.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:04:24 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-96-187.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:04:25 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-96-187.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:06:43 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-252.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:06:43 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-252.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:06:43 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-252.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:06:43 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-252.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:06:43 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-252.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:06:56 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-252.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:06:57 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-100-252.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:09:10 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:09:10 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:09:10 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:09:10 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:09:10 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:09:22 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:09:22 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:09:22 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:09:22 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:09:22 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:10:06 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:10:06 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:10:06 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:10:06 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:10:06 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:10:39 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:10:41 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:10:48 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:10:48 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:10:48 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:10:48 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:10:48 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:11:32 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:11:32 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:11:33 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:11:33 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:11:33 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:19:52 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:38:15 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:52:09 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:52:09 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:52:09 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:52:09 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:52:09 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:52:27 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:52:30 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:55:01 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-121-152.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:55:01 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-121-152.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:55:01 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-121-152.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:55:01 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-121-152.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:55:01 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-121-152.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:55:08 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-121-152.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:55:08 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-121-152.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:55:08 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-121-152.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:55:08 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-121-152.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:55:08 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-121-152.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:55:22 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-121-152.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:55:23 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-121-152.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:55:25 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-121-152.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:56:50 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:56:50 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:56:50 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:56:50 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:56:50 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:57:10 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:57:12 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:57:14 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:57:15 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:57:16 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:58:02 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:58:02 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:58:02 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:58:02 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:58:02 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:58:17 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:58:18 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:59:10 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:59:10 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:59:10 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:59:10 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:59:10 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:59:26 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:59:27 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:59:27 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:59:27 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 07:59:27 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 08:00:18 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 08:00:19 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 08:00:19 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 08:00:19 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 08:00:19 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 08:00:37 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 08:00:38 thebighonker tcpwrap: warning: /etc/hosts.allow, line 74: can't verify hostname: getaddrinfo(66-87-120-160.pools.spcsdns.net, AF_INET) failed <118>Jul 25 08:04:57 thebighonker sshd[17807]: warning: /etc/hosts.allow, line 74: host name/address mismatch: 202.170.136.247 != user.nova.net.cn kern.msgbuf_clear: 0 kern.smp.maxid: 7 kern.smp.maxcpus: 64 kern.smp.active: 1 kern.smp.disabled: 0 kern.smp.cpus: 8 kern.smp.topology: 0 kern.smp.forward_signal_enabled: 1 kern.tty_inq_flush_secure: 1 kern.tty_nin: 6403 kern.tty_nout: 2423588 kern.filedelay: 30 kern.dirdelay: 29 kern.metadelay: 28 kern.minvnodes: 87538 kern.chroot_allow_open_directories: 1 kern.userasymcrypto: 1 kern.cryptodevallowsoft: 0 kern.dtrace.dof_maxsize: 8388608 kern.dtrace.helper_actions_max: 128 vm.vmtotal: System wide totals computed every five seconds: (values in kilobytes) =============================================== Processes: (RUNQ: 3 Disk Wait: 0 Page Wait: 0 Sleep: 120) Virtual Memory: (Total: 7098928K Active: 6770504K) Real Memory: (Total: 12554680K Active: 1579692K) Shared Virtual Memory: (Total: 374668K Active: 148632K) Shared Real Memory: (Total: 206812K Active: 131080K) Free Memory: 652136K vm.loadavg: { 5.41 5.80 6.07 } vm.v_free_min: 25728 vm.v_free_target: 86847 vm.v_free_reserved: 5355 vm.v_inactive_target: 130270 vm.v_cache_min: 0 vm.v_cache_max: 0 vm.v_pageout_free_min: 34 vm.swap_enabled: 1 vm.md_malloc_wait: 0 vm.kmem_size: 16690679808 vm.kmem_size_min: 0 vm.kmem_size_max: 1319413950874 vm.kmem_size_scale: 1 vm.kmem_map_size: 11065069568 vm.kmem_map_free: 5625610240 vm.swap_total: 19327352832 vm.swap_reserved: 44248530944 vm.overcommit: 0 vm.swzone: 586781568 vm.swap_maxpages: 65197952 vm.swap_async_max: 4 vm.dmmax: 32 vm.nswapdev: 6 vm.zone_count: 204 vm.zone_warnings: 1 vm.kstack_cache_size: 128 vm.kstacks: 515 vm.swap_idle_threshold1: 2 vm.swap_idle_threshold2: 10 vm.exec_map_entries: 16 vm.min_kernel_address: 18446741874686296064 vm.max_kernel_address: 18446744073709547520 vm.v_free_severe: 15541 vm.stats.sys.v_swtch: 510325476 vm.stats.sys.v_trap: 513335839 vm.stats.sys.v_syscall: 2174649482 vm.stats.sys.v_intr: 106111565 vm.stats.sys.v_soft: 19228202 vm.stats.vm.v_vm_faults: 307498916 vm.stats.vm.v_io_faults: 33081 vm.stats.vm.v_cow_faults: 13684600 vm.stats.vm.v_cow_optim: 35258 vm.stats.vm.v_zfod: 288086834 vm.stats.vm.v_ozfod: 41534 vm.stats.vm.v_swapin: 0 vm.stats.vm.v_swapout: 0 vm.stats.vm.v_swappgsin: 0 vm.stats.vm.v_swappgsout: 0 vm.stats.vm.v_vnodein: 33423 vm.stats.vm.v_vnodeout: 18739 vm.stats.vm.v_vnodepgsin: 425596 vm.stats.vm.v_vnodepgsout: 36534 vm.stats.vm.v_intrans: 1033 vm.stats.vm.v_reactivated: 78626 vm.stats.vm.v_pdwakeups: 0 vm.stats.vm.v_pdpages: 65636758 vm.stats.vm.v_tcached: 309063 vm.stats.vm.v_dfree: 0 vm.stats.vm.v_pfree: 87016027 vm.stats.vm.v_tfree: 346735024 vm.stats.vm.v_page_size: 4096 vm.stats.vm.v_page_count: 4074873 vm.stats.vm.v_free_reserved: 5355 vm.stats.vm.v_free_target: 86847 vm.stats.vm.v_free_min: 25728 vm.stats.vm.v_free_count: 161016 vm.stats.vm.v_wire_count: 3359450 vm.stats.vm.v_active_count: 188550 vm.stats.vm.v_inactive_target: 130270 vm.stats.vm.v_inactive_count: 363521 vm.stats.vm.v_cache_count: 2018 vm.stats.vm.v_cache_min: 0 vm.stats.vm.v_cache_max: 0 vm.stats.vm.v_pageout_free_min: 34 vm.stats.vm.v_interrupt_free_min: 2 vm.stats.vm.v_forks: 107834 vm.stats.vm.v_vforks: 9268 vm.stats.vm.v_rforks: 2017 vm.stats.vm.v_kthreads: 24 vm.stats.vm.v_forkpages: 7386644 vm.stats.vm.v_vforkpages: 338230 vm.stats.vm.v_rforkpages: 5992659 vm.stats.vm.v_kthreadpages: 0 vm.stats.misc.zero_page_count: 9371 vm.stats.misc.cnt_prezero: 0 vm.stats.object.collapses: 635183 vm.stats.object.bypasses: 112683 vm.old_mlock: 0 vm.old_msync: 0 vm.boot_pages: 64 vm.tryrelock_restart: 18 vm.pageout_wakeup_thresh: 28292 vm.max_launder: 32 vm.pageout_update_period: 600 vm.lowmem_period: 0 vm.swap_idle_enabled: 0 vm.defer_swapspace_pageouts: 0 vm.disable_swapspace_pageouts: 0 vm.pageout_lock_miss: 0 vm.max_wired: 1340804 vm.phys_free: DOMAIN 0: FREE LIST 0: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 | POOL 2 -- -- -- -- -- -- -- -- 12 ( 16384K) | 0 | 0 | 0 11 ( 8192K) | 0 | 0 | 0 10 ( 4096K) | 0 | 0 | 0 9 ( 2048K) | 0 | 0 | 0 8 ( 1024K) | 1 | 0 | 0 7 ( 512K) | 5 | 0 | 0 6 ( 256K) | 31 | 0 | 0 5 ( 128K) | 210 | 0 | 0 4 ( 64K) | 950 | 0 | 104 3 ( 32K) | 1741 | 4 | 6 2 ( 16K) | 9539 | 171 | 442 1 ( 8K) | 1261 | 2647 | 57 0 ( 4K) | 0 | 10078 | 126 FREE LIST 1: ORDER (SIZE) | NUMBER | POOL 0 | POOL 1 | POOL 2 -- -- -- -- -- -- -- -- 12 ( 16384K) | 0 | 0 | 0 11 ( 8192K) | 0 | 0 | 0 10 ( 4096K) | 0 | 0 | 0 9 ( 2048K) | 0 | 0 | 0 8 ( 1024K) | 0 | 0 | 0 7 ( 512K) | 0 | 0 | 0 6 ( 256K) | 1 | 0 | 0 5 ( 128K) | 1 | 0 | 0 4 ( 64K) | 3 | 1 | 0 3 ( 32K) | 5 | 0 | 0 2 ( 16K) | 4 | 0 | 0 1 ( 8K) | 5 | 0 | 0 0 ( 4K) | 15 | 0 | 0 vm.phys_segs: SEGMENT 0: start: 0x10000 end: 0x9c000 domain: 0 free list: 0xffffffff815925b8 SEGMENT 1: start: 0x100000 end: 0x200000 domain: 0 free list: 0xffffffff815925b8 SEGMENT 2: start: 0x1d74000 end: 0xcfe44000 domain: 0 free list: 0xffffffff81592210 SEGMENT 3: start: 0xcfe4c000 end: 0xcfe4d000 domain: 0 free list: 0xffffffff81592210 SEGMENT 4: start: 0x100000000 end: 0x414b1c000 domain: 0 free list: 0xffffffff81592210 vm.ndomains: 1 vm.reserv.broken: 83 vm.reserv.freed: 182633 vm.reserv.partpopq: LEVEL SIZE NUMBER -1: 254316K, 1022 vm.reserv.reclaimed: 5943 vm.idlezero_enable: 0 vm.pmap.pat_works: 1 vm.pmap.pg_ps_enabled: 1 vm.pmap.pcid_enabled: 0 vm.pmap.invpcid_works: 0 vm.pmap.pcid_save_cnt: 0 vm.pmap.pde.demotions: 113019 vm.pmap.pde.mappings: 0 vm.pmap.pde.p_failures: 74794 vm.pmap.pde.promotions: 121108 vm.pmap.pdpe.demotions: 0 vm.kvm_size: 2199023251456 vm.kvm_free: 2180067094528 vfs.ufs.dirhash_minsize: 2560 vfs.ufs.dirhash_maxmem: 26955776 vfs.ufs.dirhash_mem: 0 vfs.ufs.dirhash_docheck: 0 vfs.ufs.dirhash_lowmemcount: 0 vfs.ufs.dirhash_reclaimage: 60 vfs.ufs.rename_restarts: 0 vfs.nfs.downdelayinitial: 12 vfs.nfs.downdelayinterval: 30 vfs.nfs.bufpackets: 4 vfs.nfs.reconnects: 0 vfs.nfs.nfs3_jukebox_delay: 10 vfs.nfs.skip_wcc_data_onerr: 1 vfs.nfs.realign_test: 0 vfs.nfs.realign_count: 0 vfs.nfs.callback_addr: vfs.nfs.debuglevel: 0 vfs.nfs.access_cache_timeout: 60 vfs.nfs.prime_access_cache: 0 vfs.nfs.commit_on_close: 0 vfs.nfs.clean_pages_on_close: 1 vfs.nfs.nfs_directio_enable: 0 vfs.nfs.nfs_keep_dirty_on_error: 0 vfs.nfs.nfs_directio_allow_mmap: 1 vfs.nfs.nfs_ip_paranoia: 1 vfs.nfs.diskless_valid: 0 vfs.nfs.diskless_rootpath: vfs.nfs.iodmaxidle: 120 vfs.nfs.iodmin: 0 vfs.nfs.iodmax: 20 vfs.nfs.defect: 0 vfs.devfs.generation: 282 vfs.devfs.rule_depth: 1 vfs.zfs.arc_max: 15616937984 vfs.zfs.arc_min: 1952117248 vfs.zfs.arc_freepages: 86847 vfs.zfs.arc_freepage_percent: 0 vfs.zfs.arc_shrink_needed: 0 vfs.zfs.arc_meta_used: 2431042656 vfs.zfs.arc_meta_limit: 3904234496 vfs.zfs.l2arc_write_max: 8388608 vfs.zfs.l2arc_write_boost: 8388608 vfs.zfs.l2arc_headroom: 2 vfs.zfs.l2arc_feed_secs: 1 vfs.zfs.l2arc_feed_min_ms: 200 vfs.zfs.l2arc_noprefetch: 1 vfs.zfs.l2arc_feed_again: 1 vfs.zfs.l2arc_norw: 1 vfs.zfs.anon_size: 1376256 vfs.zfs.anon_metadata_lsize: 0 vfs.zfs.anon_data_lsize: 0 vfs.zfs.mru_size: 4896226304 vfs.zfs.mru_metadata_lsize: 61626368 vfs.zfs.mru_data_lsize: 4422528000 vfs.zfs.mru_ghost_size: 7108034048 vfs.zfs.mru_ghost_metadata_lsize: 94201856 vfs.zfs.mru_ghost_data_lsize: 7013832192 vfs.zfs.mfu_size: 5751376896 vfs.zfs.mfu_metadata_lsize: 671098880 vfs.zfs.mfu_data_lsize: 4925076480 vfs.zfs.mfu_ghost_size: 4891616256 vfs.zfs.mfu_ghost_metadata_lsize: 111141376 vfs.zfs.mfu_ghost_data_lsize: 4780474880 vfs.zfs.l2c_only_size: 0 vfs.zfs.dedup.prefetch: 1 vfs.zfs.nopwrite_enabled: 1 vfs.zfs.mdcomp_disable: 0 vfs.zfs.prefetch_disable: 0 vfs.zfs.zfetch.max_streams: 8 vfs.zfs.zfetch.min_sec_reap: 2 vfs.zfs.zfetch.block_cap: 256 vfs.zfs.zfetch.array_rd_sz: 1048576 vfs.zfs.top_maxinflight: 32 vfs.zfs.resilver_delay: 2 vfs.zfs.scrub_delay: 4 vfs.zfs.scan_idle: 50 vfs.zfs.scan_min_time_ms: 1000 vfs.zfs.free_min_time_ms: 1000 vfs.zfs.resilver_min_time_ms: 3000 vfs.zfs.no_scrub_io: 0 vfs.zfs.no_scrub_prefetch: 0 vfs.zfs.metaslab.gang_bang: 131073 vfs.zfs.metaslab.debug_load: 0 vfs.zfs.metaslab.debug_unload: 0 vfs.zfs.metaslab.df_alloc_threshold: 131072 vfs.zfs.metaslab.df_free_pct: 4 vfs.zfs.metaslab.min_alloc_size: 10485760 vfs.zfs.metaslab.load_pct: 50 vfs.zfs.metaslab.unload_delay: 8 vfs.zfs.metaslab.preload_limit: 3 vfs.zfs.metaslab.preload_enabled: 1 vfs.zfs.metaslab.weight_factor_enable: 0 vfs.zfs.condense_pct: 200 vfs.zfs.mg_noalloc_threshold: 0 vfs.zfs.write_to_degraded: 0 vfs.zfs.check_hostid: 1 vfs.zfs.recover: 0 vfs.zfs.deadman_synctime_ms: 1000000 vfs.zfs.deadman_checktime_ms: 5000 vfs.zfs.deadman_enabled: 1 vfs.zfs.spa_asize_inflation: 24 vfs.zfs.txg.timeout: 5 vfs.zfs.vdev.cache.max: 16384 vfs.zfs.vdev.cache.size: 0 vfs.zfs.vdev.cache.bshift: 16 vfs.zfs.vdev.trim_on_init: 1 vfs.zfs.vdev.max_active: 1000 vfs.zfs.vdev.sync_read_min_active: 10 vfs.zfs.vdev.sync_read_max_active: 10 vfs.zfs.vdev.sync_write_min_active: 10 vfs.zfs.vdev.sync_write_max_active: 10 vfs.zfs.vdev.async_read_min_active: 1 vfs.zfs.vdev.async_read_max_active: 3 vfs.zfs.vdev.async_write_min_active: 1 vfs.zfs.vdev.async_write_max_active: 10 vfs.zfs.vdev.scrub_min_active: 1 vfs.zfs.vdev.scrub_max_active: 2 vfs.zfs.vdev.aggregation_limit: 131072 vfs.zfs.vdev.read_gap_limit: 32768 vfs.zfs.vdev.write_gap_limit: 4096 vfs.zfs.vdev.bio_flush_disable: 0 vfs.zfs.vdev.bio_delete_disable: 0 vfs.zfs.vdev.trim_max_bytes: 2147483648 vfs.zfs.vdev.trim_max_pending: 64 vfs.zfs.max_auto_ashift: 13 vfs.zfs.min_auto_ashift: 9 vfs.zfs.zil_replay_disable: 0 vfs.zfs.cache_flush_disable: 0 vfs.zfs.zio.use_uma: 1 vfs.zfs.zio.exclude_metadata: 0 vfs.zfs.sync_pass_deferred_free: 2 vfs.zfs.sync_pass_dont_compress: 5 vfs.zfs.sync_pass_rewrite: 2 vfs.zfs.snapshot_list_prefetch: 0 vfs.zfs.super_owner: 1 vfs.zfs.debug: 0 vfs.zfs.version.ioctl: 4 vfs.zfs.version.acl: 1 vfs.zfs.version.spa: 5000 vfs.zfs.version.zpl: 5 vfs.zfs.vol.mode: 1 vfs.zfs.trim.enabled: 1 vfs.zfs.trim.txg_delay: 32 vfs.zfs.trim.timeout: 30 vfs.zfs.trim.max_interval: 1 vfs.nfsd.disable_checkutf8: 0 vfs.nfsd.nfs_privport: 0 vfs.nfsd.server_min_nfsvers: 2 vfs.nfsd.server_max_nfsvers: 4 vfs.nfsd.async: 0 vfs.nfsd.mirrormnt: 1 vfs.nfsd.commit_blks: 0 vfs.nfsd.commit_miss: 0 vfs.nfsd.issue_delegations: 0 vfs.nfsd.enable_locallocks: 0 vfs.nfsd.enable_stringtouid: 0 vfs.nfsd.tcphighwater: 0 vfs.nfsd.udphighwater: 500 vfs.nfsd.tcpcachetimeo: 43200 vfs.nfsd.cachetcp: 1 vfs.nfsd.minthreads: 1 vfs.nfsd.maxthreads: 1 vfs.nfsd.threads: 0 vfs.nfsd.groups: 1 vfs.nfsd.request_space_used: 0 vfs.nfsd.request_space_used_highest: 0 vfs.nfsd.request_space_high: 47185920 vfs.nfsd.request_space_low: 31457280 vfs.nfsd.request_space_throttled: 0 vfs.nfsd.request_space_throttle_count: 0 vfs.nfsd.fha.enable: 1 vfs.nfsd.fha.bin_shift: 22 vfs.nfsd.fha.max_nfsds_per_fh: 8 vfs.nfsd.fha.max_reqs_per_nfsd: 0 vfs.nfsd.fha.fhe_stats: No file handle entries. vfs.pfs.trace: 0 vfs.pfs.vncache.entries: 0 vfs.pfs.vncache.maxentries: 0 vfs.pfs.vncache.hits: 0 vfs.pfs.vncache.misses: 0 vfs.acl_nfs4_old_semantics: 0 vfs.vmiodirenable: 1 vfs.runningbufspace: 0 vfs.bufspace: 0 vfs.unmapped_bufspace: 0 vfs.maxbufspace: 1725759488 vfs.bufmallocspace: 0 vfs.maxmallocbufspace: 86255206 vfs.lobufspace: 1725038592 vfs.hibufspace: 1725104128 vfs.bufreusecnt: 0 vfs.buffreekvacnt: 0 vfs.bufdefragcnt: 0 vfs.lorunningspace: 11206656 vfs.hirunningspace: 16777216 vfs.dirtybufferflushes: 0 vfs.bdwriteskip: 0 vfs.altbufferflushes: 0 vfs.recursiveflushes: 0 vfs.numdirtybuffers: 0 vfs.lodirtybuffers: 13176 vfs.hidirtybuffers: 26353 vfs.dirtybufthresh: 23717 vfs.numfreebuffers: 105332 vfs.lofreebuffers: 5856 vfs.hifreebuffers: 11712 vfs.getnewbufcalls: 0 vfs.getnewbufrestarts: 0 vfs.mappingrestarts: 0 vfs.flushbufqtarget: 100 vfs.notbufdflushes: 0 vfs.barrierwrites: 0 vfs.unmapped_buf_allowed: 1 vfs.flushwithdeps: 0 vfs.ncnegfactor: 16 vfs.ncsizefactor: 2 vfs.cache.numneg: 7305 vfs.cache.numcache: 117915 vfs.cache.numcalls: 161605491 vfs.cache.dothits: 541341 vfs.cache.dotdothits: 577737 vfs.cache.numchecks: 167575412 vfs.cache.nummiss: 6036744 vfs.cache.nummisszap: 62418 vfs.cache.numposzaps: 349173 vfs.cache.numposhits: 150643401 vfs.cache.numnegzaps: 74135 vfs.cache.numneghits: 3316283 vfs.cache.numupgrades: 5651 vfs.cache.numfullpathcalls: 718238 vfs.cache.numfullpathfail1: 0 vfs.cache.numfullpathfail2: 80 vfs.cache.numfullpathfail4: 0 vfs.cache.numfullpathfound: 718166 vfs.write_behind: 1 vfs.read_max: 64 vfs.read_min: 1 vfs.typenumhash: 1 vfs.lookup_shared: 1 vfs.usermount: 1 vfs.numvnodes: 111777 vfs.wantfreevnodes: 87538 vfs.freevnodes: 83901 vfs.vlru_allow_cache_src: 0 vfs.reassignbufcalls: 0 vfs.timestamp_precision: 0 vfs.worklist_len: 0 vfs.ffs.doasyncfree: 1 vfs.ffs.doreallocblks: 1 vfs.ffs.compute_summary_at_mount: 0 net.local.stream.sendspace: 8192 net.local.stream.recvspace: 8192 net.local.dgram.maxdgram: 2048 net.local.dgram.recvspace: 4096 net.local.seqpacket.maxseqpacket: 8192 net.local.seqpacket.recvspace: 8192 net.local.inflight: 0 net.local.deferred: 0 net.local.recycled: 0 net.local.taskcount: 138 net.inet.ip.portrange.lowfirst: 1023 net.inet.ip.portrange.lowlast: 600 net.inet.ip.portrange.first: 10000 net.inet.ip.portrange.last: 65535 net.inet.ip.portrange.hifirst: 49152 net.inet.ip.portrange.hilast: 65535 net.inet.ip.portrange.reservedhigh: 1023 net.inet.ip.portrange.reservedlow: 0 net.inet.ip.portrange.randomized: 1 net.inet.ip.portrange.randomcps: 10 net.inet.ip.portrange.randomtime: 45 net.inet.ip.forwarding: 0 net.inet.ip.redirect: 1 net.inet.ip.ttl: 64 net.inet.ip.rtexpire: 3600 net.inet.ip.rtminexpire: 10 net.inet.ip.rtmaxcache: 128 net.inet.ip.sourceroute: 0 net.inet.ip.intr_queue_maxlen: 256 net.inet.ip.intr_queue_drops: 0 net.inet.ip.accept_sourceroute: 0 net.inet.ip.keepfaith: 0 net.inet.ip.gifttl: 30 net.inet.ip.no_same_prefix: 0 net.inet.ip.random_id_period: 8192 net.inet.ip.random_id_collisions: 0 net.inet.ip.random_id_total: 0 net.inet.ip.mcast.maxgrpsrc: 512 net.inet.ip.mcast.maxsocksrc: 128 net.inet.ip.mcast.loop: 1 net.inet.ip.fastforwarding: 0 net.inet.ip.sendsourcequench: 0 net.inet.ip.random_id: 0 net.inet.ip.check_interface: 0 net.inet.ip.fragpackets: 0 net.inet.ip.maxfragsperpacket: 16 net.inet.ip.maxfragpackets: 31834 net.inet.ip.process_options: 1 net.inet.icmp.maskrepl: 0 net.inet.icmp.icmplim: 200 net.inet.icmp.icmplim_output: 1 net.inet.icmp.maskfake: 0 net.inet.icmp.log_redirect: 0 net.inet.icmp.reply_src: net.inet.icmp.reply_from_interface: 0 net.inet.icmp.quotelen: 8 net.inet.icmp.bmcastecho: 0 net.inet.icmp.drop_redirect: 0 net.inet.igmp.recvifkludge: 1 net.inet.igmp.sendra: 1 net.inet.igmp.sendlocal: 1 net.inet.igmp.v1enable: 1 net.inet.igmp.v2enable: 1 net.inet.igmp.legacysupp: 0 net.inet.igmp.default_version: 3 net.inet.igmp.gsrdelay: 10 net.inet.tcp.rfc1323: 1 net.inet.tcp.mssdflt: 536 net.inet.tcp.keepidle: 7200000 net.inet.tcp.keepintvl: 75000 net.inet.tcp.sendspace: 32768 net.inet.tcp.recvspace: 65536 net.inet.tcp.keepinit: 75000 net.inet.tcp.delacktime: 100 net.inet.tcp.v6mssdflt: 1220 net.inet.tcp.cc.algorithm: newreno net.inet.tcp.cc.available: newreno net.inet.tcp.hostcache.cachelimit: 15360 net.inet.tcp.hostcache.hashsize: 512 net.inet.tcp.hostcache.bucketlimit: 30 net.inet.tcp.hostcache.count: 168 net.inet.tcp.hostcache.expire: 3600 net.inet.tcp.hostcache.prune: 300 net.inet.tcp.hostcache.purge: 0 net.inet.tcp.log_in_vain: 0 net.inet.tcp.blackhole: 0 net.inet.tcp.delayed_ack: 1 net.inet.tcp.drop_synfin: 0 net.inet.tcp.rfc3042: 1 net.inet.tcp.rfc3390: 1 net.inet.tcp.experimental.initcwnd10: 1 net.inet.tcp.rfc3465: 1 net.inet.tcp.abc_l_var: 2 net.inet.tcp.ecn.enable: 0 net.inet.tcp.ecn.maxretries: 1 net.inet.tcp.insecure_rst: 0 net.inet.tcp.recvbuf_auto: 1 net.inet.tcp.recvbuf_inc: 16384 net.inet.tcp.recvbuf_max: 2097152 net.inet.tcp.path_mtu_discovery: 1 net.inet.tcp.tso: 1 net.inet.tcp.sendbuf_auto: 1 net.inet.tcp.sendbuf_inc: 8192 net.inet.tcp.sendbuf_max: 2097152 net.inet.tcp.reass.maxsegments: 63700 net.inet.tcp.reass.cursegments: 0 net.inet.tcp.reass.overflows: 25 net.inet.tcp.sack.enable: 1 net.inet.tcp.sack.maxholes: 128 net.inet.tcp.sack.globalmaxholes: 65536 net.inet.tcp.sack.globalholes: 0 net.inet.tcp.minmss: 216 net.inet.tcp.log_debug: 0 net.inet.tcp.tcbhashsize: 131072 net.inet.tcp.do_tcpdrain: 1 net.inet.tcp.pcbcount: 74 net.inet.tcp.icmp_may_rst: 1 net.inet.tcp.isn_reseed_interval: 0 net.inet.tcp.soreceive_stream: 0 net.inet.tcp.syncookies: 1 net.inet.tcp.syncookies_only: 0 net.inet.tcp.syncache.bucketlimit: 30 net.inet.tcp.syncache.cachelimit: 15375 net.inet.tcp.syncache.count: 0 net.inet.tcp.syncache.hashsize: 512 net.inet.tcp.syncache.rexmtlimit: 3 net.inet.tcp.syncache.rst_on_sock_fail: 1 net.inet.tcp.msl: 30000 net.inet.tcp.rexmit_min: 30 net.inet.tcp.rexmit_slop: 200 net.inet.tcp.always_keepalive: 1 net.inet.tcp.fast_finwait2_recycle: 0 net.inet.tcp.finwait2_timeout: 60000 net.inet.tcp.keepcnt: 8 net.inet.tcp.rexmit_drop_options: 0 net.inet.tcp.per_cpu_timers: 0 net.inet.tcp.timer_race: 0 net.inet.tcp.maxtcptw: 27767 net.inet.tcp.nolocaltimewait: 0 net.inet.udp.checksum: 1 net.inet.udp.maxdgram: 9216 net.inet.udp.recvspace: 42080 net.inet.udp.log_in_vain: 0 net.inet.udp.blackhole: 0 net.inet.sctp.sendspace: 1864135 net.inet.sctp.recvspace: 1864135 net.inet.sctp.auto_asconf: 1 net.inet.sctp.ecn_enable: 1 net.inet.sctp.strict_sacks: 1 net.inet.sctp.peer_chkoh: 256 net.inet.sctp.maxburst: 4 net.inet.sctp.fr_maxburst: 4 net.inet.sctp.maxchunks: 127339 net.inet.sctp.tcbhashsize: 1024 net.inet.sctp.pcbhashsize: 256 net.inet.sctp.min_split_point: 2904 net.inet.sctp.chunkscale: 10 net.inet.sctp.delayed_sack_time: 200 net.inet.sctp.sack_freq: 2 net.inet.sctp.sys_resource: 1000 net.inet.sctp.asoc_resource: 10 net.inet.sctp.heartbeat_interval: 30000 net.inet.sctp.pmtu_raise_time: 600 net.inet.sctp.shutdown_guard_time: 180 net.inet.sctp.secret_lifetime: 3600 net.inet.sctp.rto_max: 60000 net.inet.sctp.rto_min: 1000 net.inet.sctp.rto_initial: 3000 net.inet.sctp.init_rto_max: 60000 net.inet.sctp.valid_cookie_life: 60000 net.inet.sctp.init_rtx_max: 8 net.inet.sctp.assoc_rtx_max: 10 net.inet.sctp.path_rtx_max: 5 net.inet.sctp.path_pf_threshold: 65535 net.inet.sctp.add_more_on_output: 1452 net.inet.sctp.incoming_streams: 2048 net.inet.sctp.outgoing_streams: 10 net.inet.sctp.cmt_on_off: 0 net.inet.sctp.nr_sack_on_off: 0 net.inet.sctp.cmt_use_dac: 0 net.inet.sctp.cwnd_maxburst: 1 net.inet.sctp.asconf_auth_nochk: 0 net.inet.sctp.auth_disable: 0 net.inet.sctp.nat_friendly: 1 net.inet.sctp.abc_l_var: 2 net.inet.sctp.max_chained_mbufs: 5 net.inet.sctp.do_sctp_drain: 1 net.inet.sctp.hb_max_burst: 4 net.inet.sctp.abort_at_limit: 0 net.inet.sctp.strict_data_order: 0 net.inet.sctp.min_residual: 1452 net.inet.sctp.max_retran_chunk: 30 net.inet.sctp.log_level: 0 net.inet.sctp.default_cc_module: 0 net.inet.sctp.default_ss_module: 0 net.inet.sctp.default_frag_interleave: 1 net.inet.sctp.mobility_base: 0 net.inet.sctp.mobility_fasthandoff: 0 net.inet.sctp.udp_tunneling_port: 0 net.inet.sctp.enable_sack_immediately: 0 net.inet.sctp.nat_friendly_init: 0 net.inet.sctp.vtag_time_wait: 60 net.inet.sctp.buffer_splitting: 0 net.inet.sctp.initial_cwnd: 3 net.inet.sctp.rttvar_bw: 4 net.inet.sctp.rttvar_rtt: 5 net.inet.sctp.rttvar_eqret: 0 net.inet.sctp.rttvar_steady_step: 20 net.inet.sctp.use_dcccecn: 1 net.inet.sctp.blackhole: 0 net.inet.sctp.diag_info_code: 0 net.inet.raw.maxdgram: 9216 net.inet.raw.recvspace: 9216 net.inet.accf.unloadable: 0 net.link.generic.system.ifcount: 4 net.link.ether.inet.max_age: 1200 net.link.ether.inet.maxtries: 5 net.link.ether.inet.useloopback: 1 net.link.ether.inet.proxyall: 0 net.link.ether.inet.wait: 20 net.link.ether.inet.maxhold: 1 net.link.ether.inet.log_arp_wrong_iface: 1 net.link.ether.inet.log_arp_movements: 1 net.link.ether.inet.log_arp_permanent_modify: 1 net.link.ether.inet.allow_multicast: 0 net.link.ether.inet.max_log_per_second: 1 net.link.vlan.soft_pad: 0 net.link.gif.max_nesting: 1 net.link.gif.parallel_tunnels: 0 net.link.ifqmaxlen: 50 net.link.log_link_state_change: 1 net.link.tun.devfs_cloning: 1 net.link.lagg.failover_rx_all: 0 net.link.lagg.default_use_flowid: 1 net.link.lagg.default_flowid_shift: 16 net.link.lagg.lacp.debug: 0 net.link.lagg.0.use_flowid: 1 net.link.lagg.0.flowid_shift: 16 net.link.lagg.0.count: 2 net.link.lagg.0.active: 2 net.link.lagg.0.flapping: 0 net.inet6.ip6.forwarding: 0 net.inet6.ip6.redirect: 1 net.inet6.ip6.hlim: 64 net.inet6.ip6.maxfragpackets: 254679 net.inet6.ip6.accept_rtadv: 0 net.inet6.ip6.keepfaith: 0 net.inet6.ip6.log_interval: 5 net.inet6.ip6.hdrnestlimit: 15 net.inet6.ip6.dad_count: 0 net.inet6.ip6.auto_flowlabel: 1 net.inet6.ip6.defmcasthlim: 1 net.inet6.ip6.gifhlim: 30 net.inet6.ip6.kame_version: FreeBSD net.inet6.ip6.use_deprecated: 1 net.inet6.ip6.rr_prune: 5 net.inet6.ip6.v6only: 1 net.inet6.ip6.rtmaxcache: 128 net.inet6.ip6.use_tempaddr: 0 net.inet6.ip6.temppltime: 86400 net.inet6.ip6.tempvltime: 604800 net.inet6.ip6.auto_linklocal: 1 net.inet6.ip6.prefer_tempaddr: 0 net.inet6.ip6.use_defaultzone: 0 net.inet6.ip6.maxfrags: 254679 net.inet6.ip6.mcast_pmtu: 0 net.inet6.ip6.no_radr: 0 net.inet6.ip6.norbit_raif: 0 net.inet6.ip6.rfc6204w3: 0 net.inet6.ip6.mcast.maxgrpsrc: 512 net.inet6.ip6.mcast.maxsocksrc: 128 net.inet6.ip6.mcast.loop: 1 net.inet6.ip6.deembed_scopeid: 1 net.inet6.icmp6.rediraccept: 1 net.inet6.icmp6.redirtimeout: 600 net.inet6.icmp6.nd6_prune: 1 net.inet6.icmp6.nd6_delay: 5 net.inet6.icmp6.nd6_umaxtries: 3 net.inet6.icmp6.nd6_mmaxtries: 3 net.inet6.icmp6.nd6_useloopback: 1 net.inet6.icmp6.nodeinfo: 3 net.inet6.icmp6.errppslimit: 100 net.inet6.icmp6.nd6_maxnudhint: 0 net.inet6.icmp6.nd6_debug: 0 net.inet6.icmp6.nd6_maxqueuelen: 1 net.inet6.icmp6.nodeinfo_oldmcprefix: 1 net.inet6.icmp6.nd6_onlink_ns_rfc4861: 0 net.inet6.icmp6.nd6_gctimer: 86400 net.inet6.mld.gsrdelay: 10 net.inet6.mld.v1enable: 1 net.inet6.mld.use_allow: 1 net.bpf.maxinsns: 512 net.bpf.zerocopy_enable: 0 net.bpf.optimize_writers: 0 net.bpf.bufsize: 4096 net.bpf.maxbufsize: 524288 net.ifdescr_maxlen: 1024 net.isr.dispatch: direct net.isr.maxthreads: 1 net.isr.bindthreads: 0 net.isr.maxqlimit: 10240 net.isr.defaultqlimit: 256 net.isr.maxprot: 16 net.isr.numthreads: 1 net.raw.sendspace: 8192 net.raw.recvspace: 8192 net.fibs: 1 net.add_addr_allfibs: 1 net.my_fibnum: 0 net.route.netisr_maxqlen: 256 net.wlan.nol_timeout: 1800 net.wlan.cac_timeout: 60 net.wlan.debug: 0 net.wlan.ampdu_age: 500 net.wlan.recv_bar: 1 net.wlan.addba_timeout: 250 net.wlan.addba_backoff: 10000 net.wlan.addba_maxtries: 3 net.wlan.hwmp.targetonly: 0 net.wlan.hwmp.pathlifetime: 5000 net.wlan.hwmp.maxpreq_retries: 3 net.wlan.hwmp.net_diameter_traversal_time: 512 net.wlan.hwmp.roottimeout: 5000 net.wlan.hwmp.rootint: 2000 net.wlan.hwmp.rannint: 1000 net.wlan.hwmp.rootconfint: 2000 net.wlan.hwmp.inact: 5000 net.wlan.mesh.gateint: 10000 net.wlan.mesh.retrytimeout: 40 net.wlan.mesh.holdingtimeout: 40 net.wlan.mesh.confirmtimeout: 40 net.wlan.mesh.backofftimeout: 5000 net.wlan.mesh.maxretries: 2 net.wlan.mesh.maxholding: 2 debug.acpi.max_tasks: 256 debug.acpi.max_threads: 3 debug.acpi.acpi_ca_version: 20130823 debug.acpi.enable_debug_objects: 0 debug.acpi.interpreter_slack: 1 debug.acpi.reset_clock: 1 debug.acpi.suspend_bounce: 0 debug.acpi.cpu_unordered: 0 debug.acpi.ec.burst: 0 debug.acpi.ec.polled: 0 debug.acpi.ec.timeout: 750 debug.acpi.batt.batt_sleep_ms: 0 debug.acpi.resume_beep: 0 debug.ipw: 0 debug.iwi: 0 debug.mddebug: 0 debug.wpi: 0 debug.elf64_legacy_coredump: 0 debug.elf32_legacy_coredump: 0 debug.boothowto: 0 debug.bootverbose: 0 debug.cpufreq.lowest: 0 debug.cpufreq.verbose: 0 debug.fail_point.sysctl_running: off debug.fail_point.buf_pressure: off debug.fail_point.nlm_deny_grant: off debug.sizeof.cdev: 288 debug.sizeof.cdev_priv: 376 debug.sizeof.g_class: 176 debug.sizeof.g_geom: 176 debug.sizeof.g_provider: 136 debug.sizeof.g_consumer: 96 debug.sizeof.g_bioq: 56 debug.sizeof.vnode: 472 debug.sizeof.proc: 1208 debug.sizeof.bio: 248 debug.sizeof.buf: 600 debug.sizeof.kinfo_proc: 1088 debug.sizeof.devstat: 288 debug.sizeof.namecache: 72 debug.sizeof.znode: 368 debug.adaptive_machine_arch: 1 debug.osd: 0 debug.rwlock.retry: 10 debug.rwlock.loops: 10000 debug.debugger_on_panic: 1 debug.trace_on_panic: 1 debug.ncores: 5 debug.sx.retries: 10 debug.sx.loops: 10000 debug.umtx.umtx_pi_allocated: 0 debug.clocktime: 0 debug.kdb.available: debug.kdb.current: debug.kdb.enter: 0 debug.kdb.panic: 0 debug.kdb.trap: 0 debug.kdb.trap_code: 0 debug.kdb.break_to_debugger: 0 debug.kdb.alt_break_to_debugger: 0 debug.rman_debug: 0 debug.iosize_max_clamp: 1 debug.devfs_iosize_max_clamp: 1 debug.nchash: 524287 debug.numneg: 7305 debug.numcache: 117915 debug.numcachehv: 22824 debug.vfscache: 1 debug.disablecwd: 0 debug.disablefullpath: 0 debug.rush_requests: 0 debug.vnlru_nowhere: 0 debug.vn_io_fault_enable: 1 debug.vn_io_faults: 0 debug.if_tun_debug: 0 debug.nlm_debug: 0 debug.fsckcmds: 0 debug.dopersistence: 0 debug.snapdebug: 0 debug.collectsnapstats: 0 debug.softdep.total.pagedep: 0 debug.softdep.total.inodedep: 0 debug.softdep.total.bmsafemap: 0 debug.softdep.total.newblk: 0 debug.softdep.total.allocdirect: 0 debug.softdep.total.indirdep: 0 debug.softdep.total.allocindir: 0 debug.softdep.total.freefrag: 0 debug.softdep.total.freeblks: 0 debug.softdep.total.freefile: 0 debug.softdep.total.diradd: 0 debug.softdep.total.mkdir: 0 debug.softdep.total.dirrem: 0 debug.softdep.total.newdirblk: 0 debug.softdep.total.freework: 0 debug.softdep.total.freedep: 0 debug.softdep.total.jaddref: 0 debug.softdep.total.jremref: 0 debug.softdep.total.jmvref: 0 debug.softdep.total.jnewblk: 0 debug.softdep.total.jfreeblk: 0 debug.softdep.total.jfreefrag: 0 debug.softdep.total.jseg: 0 debug.softdep.total.jsegdep: 0 debug.softdep.total.sbdep: 0 debug.softdep.total.jtrunc: 0 debug.softdep.total.jfsync: 0 debug.softdep.highuse.pagedep: 0 debug.softdep.highuse.inodedep: 0 debug.softdep.highuse.bmsafemap: 0 debug.softdep.highuse.newblk: 0 debug.softdep.highuse.allocdirect: 0 debug.softdep.highuse.indirdep: 0 debug.softdep.highuse.allocindir: 0 debug.softdep.highuse.freefrag: 0 debug.softdep.highuse.freeblks: 0 debug.softdep.highuse.freefile: 0 debug.softdep.highuse.diradd: 0 debug.softdep.highuse.mkdir: 0 debug.softdep.highuse.dirrem: 0 debug.softdep.highuse.newdirblk: 0 debug.softdep.highuse.freework: 0 debug.softdep.highuse.freedep: 0 debug.softdep.highuse.jaddref: 0 debug.softdep.highuse.jremref: 0 debug.softdep.highuse.jmvref: 0 debug.softdep.highuse.jnewblk: 0 debug.softdep.highuse.jfreeblk: 0 debug.softdep.highuse.jfreefrag: 0 debug.softdep.highuse.jseg: 0 debug.softdep.highuse.jsegdep: 0 debug.softdep.highuse.sbdep: 0 debug.softdep.highuse.jtrunc: 0 debug.softdep.highuse.jfsync: 0 debug.softdep.current.pagedep: 0 debug.softdep.current.inodedep: 0 debug.softdep.current.bmsafemap: 0 debug.softdep.current.newblk: 0 debug.softdep.current.allocdirect: 0 debug.softdep.current.indirdep: 0 debug.softdep.current.allocindir: 0 debug.softdep.current.freefrag: 0 debug.softdep.current.freeblks: 0 debug.softdep.current.freefile: 0 debug.softdep.current.diradd: 0 debug.softdep.current.mkdir: 0 debug.softdep.current.dirrem: 0 debug.softdep.current.newdirblk: 0 debug.softdep.current.freework: 0 debug.softdep.current.freedep: 0 debug.softdep.current.jaddref: 0 debug.softdep.current.jremref: 0 debug.softdep.current.jmvref: 0 debug.softdep.current.jnewblk: 0 debug.softdep.current.jfreeblk: 0 debug.softdep.current.jfreefrag: 0 debug.softdep.current.jseg: 0 debug.softdep.current.jsegdep: 0 debug.softdep.current.sbdep: 0 debug.softdep.current.jtrunc: 0 debug.softdep.current.jfsync: 0 debug.softdep.write.pagedep: 0 debug.softdep.write.inodedep: 0 debug.softdep.write.bmsafemap: 0 debug.softdep.write.newblk: 0 debug.softdep.write.allocdirect: 0 debug.softdep.write.indirdep: 0 debug.softdep.write.allocindir: 0 debug.softdep.write.freefrag: 0 debug.softdep.write.freeblks: 0 debug.softdep.write.freefile: 0 debug.softdep.write.diradd: 0 debug.softdep.write.mkdir: 0 debug.softdep.write.dirrem: 0 debug.softdep.write.newdirblk: 0 debug.softdep.write.freework: 0 debug.softdep.write.freedep: 0 debug.softdep.write.jaddref: 0 debug.softdep.write.jremref: 0 debug.softdep.write.jmvref: 0 debug.softdep.write.jnewblk: 0 debug.softdep.write.jfreeblk: 0 debug.softdep.write.jfreefrag: 0 debug.softdep.write.jseg: 0 debug.softdep.write.jsegdep: 0 debug.softdep.write.sbdep: 0 debug.softdep.write.jtrunc: 0 debug.softdep.write.jfsync: 0 debug.softdep.max_softdeps: 1400620 debug.softdep.tickdelay: 2 debug.softdep.maxindirdeps: 50 debug.softdep.softdep_mounts: 0 debug.softdep.worklist_push: 0 debug.softdep.blk_limit_push: 0 debug.softdep.ino_limit_push: 0 debug.softdep.blk_limit_hit: 0 debug.softdep.ino_limit_hit: 0 debug.softdep.sync_limit_hit: 0 debug.softdep.indir_blk_ptrs: 0 debug.softdep.inode_bitmap: 0 debug.softdep.direct_blk_ptrs: 0 debug.softdep.dir_entry: 0 debug.softdep.jaddref_rollback: 0 debug.softdep.jnewblk_rollback: 0 debug.softdep.journal_low: 0 debug.softdep.journal_min: 0 debug.softdep.journal_wait: 0 debug.softdep.jwait_filepage: 0 debug.softdep.jwait_freeblks: 0 debug.softdep.jwait_inode: 0 debug.softdep.jwait_newblk: 0 debug.softdep.cleanup_blkrequests: 0 debug.softdep.cleanup_inorequests: 0 debug.softdep.cleanup_high_delay: 0 debug.softdep.cleanup_retries: 0 debug.softdep.cleanup_failures: 0 debug.softdep.flushcache: 0 debug.softdep.emptyjblocks: 0 debug.bigcgs: 0 debug.dobkgrdwrite: 1 debug.dircheck: 0 debug.psm.loglevel: 0 debug.psm.hz: 20 debug.psm.errsecs: 2 debug.psm.errusecs: 0 debug.psm.secs: 0 debug.psm.usecs: 500000 debug.psm.pkterrthresh: 2 debug.vesa.shadow_rom: 0 debug.fdc.fifo: 8 debug.fdc.debugflags: 0 debug.fdc.retries: 10 debug.fdc.spec1: 175 debug.fdc.spec2: 16 debug.fdc.settle: 0 debug.x86bios.call: 0 debug.x86bios.int: 0 debug.hwpstate_verbose: 0 debug.minidump: 1 debug.zfs_flags: 0 debug.crypto_timing: 0 debug.dtrace.verbose_ioctl: 0 debug.dtrace.debug: 0 debug.dtrace.providers: dtrace linuxulator32 opencrypto mac_framework mac sctp udp tcp ip vfs io sched callout_execute sdt proc priv xbb dtmalloc nfscl fbt syscall lockstat profile syscall hw.machine: amd64 hw.model: Intel(R) Xeon(R) CPU L5420 @ 2.50GHz hw.ncpu: 8 hw.byteorder: 1234 hw.physmem: 17148788736 hw.usermem: 3388481536 hw.pagesize: 4096 hw.floatingpoint: 1 hw.machine_arch: amd64 hw.realmem: 17179869184 hw.aac.enable_msi: 1 hw.amr.force_sg32: 0 hw.an.an_dump: off hw.an.an_cache_mode: dbm hw.an.an_cache_mcastonly: 0 hw.an.an_cache_iponly: 1 hw.ata.ata_dma_check_80pin: 1 hw.ath.longcal: 30 hw.ath.shortcal: 100 hw.ath.resetcal: 1200 hw.ath.anical: 100 hw.ath.rxbuf: 512 hw.ath.txbuf: 512 hw.ath.txbuf_mgmt: 32 hw.ath.bstuck: 4 hw.bce.verbose: 1 hw.bce.tso_enable: 1 hw.bce.msi_enable: 1 hw.bce.rx_pages: 2 hw.bce.tx_pages: 2 hw.bce.hdr_split: 1 hw.bce.strict_rx_mtu: 0 hw.bce.tx_quick_cons_trip_int: 20 hw.bce.tx_quick_cons_trip: 20 hw.bce.tx_ticks_int: 80 hw.bce.tx_ticks: 80 hw.bce.rx_quick_cons_trip_int: 6 hw.bce.rx_quick_cons_trip: 6 hw.bce.rx_ticks_int: 18 hw.bce.rx_ticks: 18 hw.bge.allow_asf: 1 hw.cardbus.debug: 0 hw.cardbus.cis_debug: 0 hw.cs.ignore_checksum_failure: 0 hw.cs.recv_delay: 570 hw.em.tx_int_delay: 66 hw.em.rx_int_delay: 0 hw.em.tx_abs_int_delay: 66 hw.em.rx_abs_int_delay: 66 hw.em.rxd: 1024 hw.em.txd: 1024 hw.em.smart_pwr_down: 0 hw.em.sbp: 0 hw.em.enable_msix: 1 hw.em.rx_process_limit: 100 hw.em.eee_setting: 1 hw.igb.rxd: 1024 hw.igb.txd: 1024 hw.igb.enable_aim: 1 hw.igb.enable_msix: 1 hw.igb.max_interrupt_rate: 8000 hw.igb.buf_ring_size: 4096 hw.igb.header_split: 0 hw.igb.num_queues: 0 hw.igb.rx_process_limit: 100 hw.ix.enable_aim: 1 hw.ix.max_interrupt_rate: 31250 hw.ix.rx_process_limit: 256 hw.ix.tx_process_limit: 256 hw.ix.enable_msix: 1 hw.ix.num_queues: 0 hw.ix.txd: 2048 hw.ix.rxd: 2048 hw.malo.txcoalesce: 8 hw.malo.rxbuf: 256 hw.malo.rxquota: 256 hw.malo.txbuf: 256 hw.malo.pci.msi_disable: 0 hw.mfi.event_locale: 65535 hw.mfi.event_class: 0 hw.mfi.max_cmds: 128 hw.mfi.detect_jbod_change: 1 hw.mfi.polled_cmd_timeout: 60 hw.mfi.cmd_timeout: 30 hw.mfi.msi: 1 hw.mmc.debug: 0 hw.mwl.rxdesc: 256 hw.mwl.rxbuf: 640 hw.mwl.txbuf: 256 hw.mwl.txcoalesce: 8 hw.mwl.rxquota: 640 hw.mwl.rxdmalow: 3 hw.pccard.debug: 0 hw.pccard.cis_debug: 0 hw.cbb.start_memory: 2281701376 hw.cbb.start_16_io: 256 hw.cbb.start_32_io: 4096 hw.cbb.debug: 0 hw.pcic.intr_mask: 57016 hw.pcic.pd6722_vsense: 1 hw.pci.enable_io_modes: 1 hw.pci.realloc_bars: 0 hw.pci.do_power_nodriver: 0 hw.pci.do_power_resume: 1 hw.pci.do_power_suspend: 1 hw.pci.enable_msi: 1 hw.pci.enable_msix: 1 hw.pci.honor_msi_blacklist: 1 hw.pci.usb_early_takeover: 1 hw.pci.clear_bars: 0 hw.pci.clear_pcib: 0 hw.pci.default_vgapci_unit: 0 hw.pci.mcfg: 1 hw.pci.host_mem_start: 2147483648 hw.sdhci.debug: 0 hw.sdhci_pci.debug: 0 hw.snd.report_soft_formats: 1 hw.snd.report_soft_matrix: 1 hw.snd.latency: 5 hw.snd.latency_profile: 1 hw.snd.vpc_autoreset: 1 hw.snd.vpc_0db: 45 hw.snd.vpc_reset: 0 hw.snd.compat_linux_mmap: 0 hw.snd.feeder_eq_presets: PEQ:16000,0.2500,62,0.2500:-9,9,1.0:44100,48000,88200,96000,176400,192000 hw.snd.feeder_eq_exact_rate: 0 hw.snd.feeder_rate_presets: 100:8:0.85 100:36:0.92 100:164:0.97 hw.snd.feeder_rate_polyphase_max: 183040 hw.snd.feeder_rate_min: 1 hw.snd.feeder_rate_max: 2016000 hw.snd.feeder_rate_round: 25 hw.snd.feeder_rate_quality: 1 hw.snd.vpc_mixer_bypass: 1 hw.snd.verbose: 0 hw.snd.default_auto: -1 hw.snd.version: 2009061500/amd64 hw.snd.default_unit: -1 hw.snd.maxautovchans: 16 hw.midi.stat.verbose: 0 hw.midi.debug: 0 hw.midi.dumpraw: 0 hw.midi.instroff: 0 hw.midi.seq.debug: 0 hw.syscons.saver.keybonly: 1 hw.syscons.bell: 1 hw.syscons.kbd_reboot: 1 hw.syscons.kbd_debug: 1 hw.syscons.sc_no_suspend_vtswitch: 0 hw.broken_txfifo: 0 hw.usb.ehci.debug: 0 hw.usb.ehci.no_hs: 0 hw.usb.ehci.iaadbug: 0 hw.usb.ehci.lostintrbug: 0 hw.usb.ohci.debug: 0 hw.usb.uhci.debug: 0 hw.usb.uhci.loop: 0 hw.usb.xhci.streams: 0 hw.usb.xhci.debug: 0 hw.usb.xhci.xhci_port_route: 0 hw.usb.xhci.use_polling: 0 hw.usb.ctrl.debug: 0 hw.usb.no_boot_wait: 0 hw.usb.no_suspend_wait: 0 hw.usb.no_shutdown_wait: 0 hw.usb.umass.debug: 0 hw.usb.umass.throttle: 0 hw.usb.debug: 0 hw.usb.timings.port_reset_delay: 50 hw.usb.timings.port_root_reset_delay: 250 hw.usb.timings.port_reset_recovery: 250 hw.usb.timings.port_powerup_delay: 300 hw.usb.timings.port_resume_delay: 40 hw.usb.timings.set_address_settle: 10 hw.usb.timings.resume_delay: 250 hw.usb.timings.resume_wait: 50 hw.usb.timings.resume_recovery: 50 hw.usb.timings.extra_power_up_time: 20 hw.usb.dev.debug: 0 hw.usb.template: 0 hw.usb.usb_lang_id: 9 hw.usb.usb_lang_mask: 255 hw.usb.ugen.debug: 0 hw.usb.uhub.debug: 0 hw.usb.power_timeout: 30 hw.usb.proc.debug: 0 hw.usb.no_cs_fail: 0 hw.usb.full_ddesc: 0 hw.usb.ukbd.debug: 0 hw.usb.ukbd.no_leds: 0 hw.usb.ukbd.pollrate: 0 hw.usb.ums.debug: 0 hw.usb.ucom.debug: 0 hw.usb.ucom.cons_unit: -1 hw.usb.ucom.cons_subunit: 0 hw.usb.ucom.cons_baud: 9600 hw.usb.uplcom.debug: 0 hw.watchdog.wd_last_u: 0 hw.watchdog.wd_last_u_secs: 0 hw.wi.txerate: 0 hw.wi.debug: 0 hw.xe.debug: 0 hw.intr_storm_threshold: 1000 hw.availpages: 4186716 hw.pagesizes: 4096 2097152 0 hw.bus.devctl_disable: 0 hw.bus.devctl_queue: 1000 hw.clockrate: 2500 hw.instruction_sse: 1 hw.via_feature_rng: 0 hw.via_feature_xcrypt: 0 hw.psm.tap_enabled: -1 hw.psm.tap_threshold: 25 hw.psm.tap_timeout: 125000 hw.bxe.debug: 0 hw.bxe.interrupt_mode: 2 hw.bxe.queue_count: 4 hw.bxe.max_rx_bufs: 0 hw.bxe.hc_rx_ticks: 25 hw.bxe.hc_tx_ticks: 50 hw.bxe.rx_budget: -1 hw.bxe.max_aggregation_size: 0 hw.bxe.mrrs: -1 hw.bxe.autogreeen: 0 hw.bxe.udp_rss: 0 hw.kbd.keymap_restrict_change: 0 hw.dmar.tbl_pagecnt: 0 hw.dmar.match_verbose: 0 hw.busdma.total_bpages: 2439 hw.busdma.zone0.total_bpages: 1024 hw.busdma.zone0.free_bpages: 1024 hw.busdma.zone0.reserved_bpages: 0 hw.busdma.zone0.active_bpages: 0 hw.busdma.zone0.total_bounced: 0 hw.busdma.zone0.total_deferred: 0 hw.busdma.zone0.lowaddr: 0xffffffffff hw.busdma.zone0.alignment: 4096 hw.busdma.zone1.total_bpages: 1415 hw.busdma.zone1.free_bpages: 1415 hw.busdma.zone1.reserved_bpages: 0 hw.busdma.zone1.active_bpages: 0 hw.busdma.zone1.total_bounced: 10397 hw.busdma.zone1.total_deferred: 0 hw.busdma.zone1.lowaddr: 0xffffffff hw.busdma.zone1.alignment: 4096 hw.apic.enable_extint: 0 hw.mca.enabled: 1 hw.mca.amd10h_L1TP: 1 hw.mca.erratum383: 0 hw.mca.count: 0 hw.mca.interval: 3600 hw.mca.force_scan: 0 hw.ipmi.on: 1 hw.acpi.supported_sleep_state: S4 S5 hw.acpi.power_button_state: S5 hw.acpi.sleep_button_state: S4 hw.acpi.lid_switch_state: NONE hw.acpi.standby_state: NONE hw.acpi.suspend_state: NONE hw.acpi.sleep_delay: 1 hw.acpi.s4bios: 0 hw.acpi.verbose: 0 hw.acpi.disable_on_reboot: 0 hw.acpi.handle_reboot: 0 hw.acpi.reset_video: 0 hw.acpi.cpu.cx_lowest: C1 hw.acpi.thermal.min_runtime: 0 hw.acpi.thermal.polling_rate: 10 hw.acpi.thermal.user_override: 0 hw.acpi.thermal.tz0.temperature: 8.3C hw.acpi.thermal.tz0.active: -1 hw.acpi.thermal.tz0.passive_cooling: 1 hw.acpi.thermal.tz0.thermal_flags: 0 hw.acpi.thermal.tz0._PSV: 9.8C hw.acpi.thermal.tz0._HOT: -1 hw.acpi.thermal.tz0._CRT: 31.3C hw.acpi.thermal.tz0._ACx: -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 hw.acpi.thermal.tz0._TC1: 4 hw.acpi.thermal.tz0._TC2: 4 hw.acpi.thermal.tz0._TSP: 60 machdep.acpi_timer_freq: 3579545 machdep.enable_panic_key: 0 machdep.rtc_save_period: 1800 machdep.adjkerntz: 18000 machdep.wall_cmos_clock: 1 machdep.disable_rtc_set: 0 machdep.disable_mtrrs: 0 machdep.idle_mwait: 1 machdep.idle_available: spin, mwait, hlt, acpi machdep.idle: acpi machdep.nkpt: 24 machdep.max_ldt_segment: 1024 machdep.kdb_on_nmi: 1 machdep.panic_on_nmi: 1 machdep.prot_fault_translation: 0 machdep.uprintf_signal: 0 machdep.acpi_root: 1003264 machdep.i8254_freq: 1193182 machdep.disable_tsc: 0 machdep.disable_tsc_calibration: 0 machdep.tsc_freq: 2500140705 user.cs_path: user.bc_base_max: 0 user.bc_dim_max: 0 user.bc_scale_max: 0 user.bc_string_max: 0 user.coll_weights_max: 0 user.expr_nest_max: 0 user.line_max: 0 user.re_dup_max: 0 user.posix2_version: 0 user.posix2_c_bind: 0 user.posix2_c_dev: 0 user.posix2_char_term: 0 user.posix2_fort_dev: 0 user.posix2_fort_run: 0 user.posix2_localedef: 0 user.posix2_sw_dev: 0 user.posix2_upe: 0 user.stream_max: 0 user.tzname_max: 0 p1003_1b.asynchronous_io: 0 p1003_1b.mapped_files: 200112 p1003_1b.memlock: 0 p1003_1b.memlock_range: 0 p1003_1b.memory_protection: 0 p1003_1b.message_passing: 0 p1003_1b.prioritized_io: 0 p1003_1b.priority_scheduling: 200112 p1003_1b.realtime_signals: 200112 p1003_1b.semaphores: 0 p1003_1b.fsync: 200112 p1003_1b.shared_memory_objects: 200112 p1003_1b.synchronized_io: 0 p1003_1b.timers: 200112 p1003_1b.aio_listio_max: -1 p1003_1b.aio_max: -1 p1003_1b.aio_prio_delta_max: -1 p1003_1b.delaytimer_max: 2147483647 p1003_1b.mq_open_max: 0 p1003_1b.pagesize: 4096 p1003_1b.rtsig_max: 62 p1003_1b.sem_nsems_max: 0 p1003_1b.sem_value_max: 0 p1003_1b.sigqueue_max: 128 p1003_1b.timer_max: 32 compat.ia32.maxdsiz: 536870912 compat.ia32.maxssiz: 67108864 compat.ia32.maxvmem: 0 compat.linux.osname: Linux compat.linux.osrelease: 2.6.16 compat.linux.oss_version: 198144 compat.linux32.maxdsiz: 536870912 compat.linux32.maxssiz: 67108864 compat.linux32.maxvmem: 0 security.jail.jailed: 0 security.jail.vnet: 0 security.jail.jail_max_af_ips: 255 security.jail.set_hostname_allowed: 1 security.jail.socket_unixiproute_only: 1 security.jail.sysvipc_allowed: 0 security.jail.allow_raw_sockets: 0 security.jail.chflags_allowed: 0 security.jail.mount_allowed: 0 security.jail.mount_devfs_allowed: 0 security.jail.mount_nullfs_allowed: 0 security.jail.mount_procfs_allowed: 0 security.jail.mount_tmpfs_allowed: 0 security.jail.mount_zfs_allowed: 0 security.jail.enforce_statfs: 2 security.jail.devfs_ruleset: 0 security.jail.param.jid: 0 security.jail.param.parent: 0 security.jail.param.name: 256 security.jail.param.path: 1024 security.jail.param.securelevel: 0 security.jail.param.enforce_statfs: 0 security.jail.param.devfs_ruleset: 0 security.jail.param.persist: 0 security.jail.param.dying: 0 security.jail.param.children.cur: 0 security.jail.param.children.max: 0 security.jail.param.host.: 0 security.jail.param.host.hostname: 256 security.jail.param.host.domainname: 256 security.jail.param.host.hostuuid: 64 security.jail.param.host.hostid: 0 security.jail.param.cpuset.id: 0 security.jail.param.ip4.: 0 security.jail.param.ip4.saddrsel: 0 security.jail.param.ip6.: 0 security.jail.param.ip6.saddrsel: 0 security.jail.param.allow.set_hostname: 0 security.jail.param.allow.sysvipc: 0 security.jail.param.allow.raw_sockets: 0 security.jail.param.allow.chflags: 0 security.jail.param.allow.quotas: 0 security.jail.param.allow.socket_af: 0 security.jail.param.allow.mount.: 0 security.jail.param.allow.mount.devfs: 0 security.jail.param.allow.mount.nullfs: 0 security.jail.param.allow.mount.procfs: 0 security.jail.param.allow.mount.tmpfs: 0 security.jail.param.allow.mount.zfs: 0 security.jail.param.linux.: 0 security.jail.param.linux.osname: 65 security.jail.param.linux.osrelease: 65 security.jail.param.linux.oss_version: 0 security.bsd.map_at_zero: 0 security.bsd.suser_enabled: 1 security.bsd.unprivileged_mlock: 1 security.bsd.see_other_uids: 1 security.bsd.see_other_gids: 1 security.bsd.conservative_signals: 1 security.bsd.unprivileged_proc_debug: 1 security.bsd.unprivileged_idprio: 0 security.bsd.unprivileged_read_msgbuf: 1 security.bsd.hardlink_check_uid: 0 security.bsd.hardlink_check_gid: 0 security.bsd.unprivileged_get_quota: 0 security.bsd.stack_guard_page: 0 security.bsd.unprivileged_syspmcs: 0 security.mac.version: 4 security.mac.max_slots: 4 security.mac.labeled: 0 security.mac.mmap_revocation: 1 security.mac.mmap_revocation_via_cow: 0 dev.xen.balloon.current: 0 dev.xen.balloon.target: 0 dev.xen.balloon.driver_pages: 0 dev.xen.balloon.hard_limit: 0 dev.xen.balloon.low_mem: 0 dev.xen.balloon.high_mem: 0 dev.xen.xsd_port: 0 dev.xen.xsd_kva: 0 dev.nexus.%parent: dev.nexus.0.%desc: dev.nexus.0.%driver: nexus dev.nexus.0.%location: dev.nexus.0.%pnpinfo: dev.nexus.0.%parent: root0 dev.ram.%parent: dev.ram.0.%desc: System RAM dev.ram.0.%driver: ram dev.ram.0.%location: dev.ram.0.%pnpinfo: dev.ram.0.%parent: nexus0 dev.cryptosoft.%parent: dev.cryptosoft.0.%desc: software crypto dev.cryptosoft.0.%driver: cryptosoft dev.cryptosoft.0.%location: dev.cryptosoft.0.%pnpinfo: dev.cryptosoft.0.%parent: nexus0 dev.acpi.%parent: dev.acpi.0.%desc: HP ProLiant dev.acpi.0.%driver: acpi dev.acpi.0.%location: dev.acpi.0.%pnpinfo: dev.acpi.0.%parent: nexus0 dev.acpi_sysresource.%parent: dev.acpi_sysresource.0.%desc: System Resource dev.acpi_sysresource.0.%driver: acpi_sysresource dev.acpi_sysresource.0.%location: handle=\_SB_.PCI0.IBRG.MOMB dev.acpi_sysresource.0.%pnpinfo: _HID=PNP0C02 _UID=0 dev.acpi_sysresource.0.%parent: acpi0 dev.cpu.%parent: dev.cpu.0.%desc: ACPI CPU dev.cpu.0.%driver: cpu dev.cpu.0.%location: handle=\_PR_.CPU0 dev.cpu.0.%pnpinfo: _HID=none _UID=0 dev.cpu.0.%parent: acpi0 dev.cpu.0.freq: 2500 dev.cpu.0.freq_levels: 2500/-1 2187/-1 1875/-1 1562/-1 1250/-1 937/-1 625/-1 312/-1 dev.cpu.0.cx_supported: C1/1/1 C2/3/17 dev.cpu.0.cx_lowest: C1 dev.cpu.0.cx_usage: 100.00% 0.00% last 9138us dev.cpu.1.%desc: ACPI CPU dev.cpu.1.%driver: cpu dev.cpu.1.%location: handle=\_PR_.CPU1 dev.cpu.1.%pnpinfo: _HID=none _UID=0 dev.cpu.1.%parent: acpi0 dev.cpu.1.cx_supported: C1/1/1 C2/3/17 dev.cpu.1.cx_lowest: C1 dev.cpu.1.cx_usage: 100.00% 0.00% last 9170us dev.cpu.2.%desc: ACPI CPU dev.cpu.2.%driver: cpu dev.cpu.2.%location: handle=\_PR_.CPU2 dev.cpu.2.%pnpinfo: _HID=none _UID=0 dev.cpu.2.%parent: acpi0 dev.cpu.2.cx_supported: C1/1/1 C2/3/17 dev.cpu.2.cx_lowest: C1 dev.cpu.2.cx_usage: 100.00% 0.00% last 13113us dev.cpu.3.%desc: ACPI CPU dev.cpu.3.%driver: cpu dev.cpu.3.%location: handle=\_PR_.CPU3 dev.cpu.3.%pnpinfo: _HID=none _UID=0 dev.cpu.3.%parent: acpi0 dev.cpu.3.cx_supported: C1/1/1 C2/3/17 dev.cpu.3.cx_lowest: C1 dev.cpu.3.cx_usage: 100.00% 0.00% last 24us dev.cpu.4.%desc: ACPI CPU dev.cpu.4.%driver: cpu dev.cpu.4.%location: handle=\_PR_.CPU4 dev.cpu.4.%pnpinfo: _HID=none _UID=0 dev.cpu.4.%parent: acpi0 dev.cpu.4.cx_supported: C1/1/1 C2/3/17 dev.cpu.4.cx_lowest: C1 dev.cpu.4.cx_usage: 100.00% 0.00% last 12898us dev.cpu.5.%desc: ACPI CPU dev.cpu.5.%driver: cpu dev.cpu.5.%location: handle=\_PR_.CPU5 dev.cpu.5.%pnpinfo: _HID=none _UID=0 dev.cpu.5.%parent: acpi0 dev.cpu.5.cx_supported: C1/1/1 C2/3/17 dev.cpu.5.cx_lowest: C1 dev.cpu.5.cx_usage: 100.00% 0.00% last 10093us dev.cpu.6.%desc: ACPI CPU dev.cpu.6.%driver: cpu dev.cpu.6.%location: handle=\_PR_.CPU6 dev.cpu.6.%pnpinfo: _HID=none _UID=0 dev.cpu.6.%parent: acpi0 dev.cpu.6.cx_supported: C1/1/1 C2/3/17 dev.cpu.6.cx_lowest: C1 dev.cpu.6.cx_usage: 100.00% 0.00% last 5339us dev.cpu.7.%desc: ACPI CPU dev.cpu.7.%driver: cpu dev.cpu.7.%location: handle=\_PR_.CPU7 dev.cpu.7.%pnpinfo: _HID=none _UID=0 dev.cpu.7.%parent: acpi0 dev.cpu.7.cx_supported: C1/1/1 C2/3/17 dev.cpu.7.cx_lowest: C1 dev.cpu.7.cx_usage: 100.00% 0.00% last 18731us dev.attimer.%parent: dev.attimer.0.%desc: AT timer dev.attimer.0.%driver: attimer dev.attimer.0.%location: handle=\_SB_.PCI0.IBRG.TIME dev.attimer.0.%pnpinfo: _HID=PNP0100 _UID=0 dev.attimer.0.%parent: acpi0 dev.hpet.%parent: dev.hpet.0.%desc: High Precision Event Timer dev.hpet.0.%driver: hpet dev.hpet.0.%location: handle=\_SB_.PCI0.IBRG.HPET dev.hpet.0.%pnpinfo: _HID=PNP0103 _UID=0 dev.hpet.0.%parent: acpi0 dev.acpi_timer.%parent: dev.acpi_timer.0.%desc: 24-bit timer at 3.579545MHz dev.acpi_timer.0.%driver: acpi_timer dev.acpi_timer.0.%location: unknown dev.acpi_timer.0.%pnpinfo: unknown dev.acpi_timer.0.%parent: acpi0 dev.pci_link.%parent: dev.pci_link.0.%desc: ACPI PCI Link LNKA dev.pci_link.0.%driver: pci_link dev.pci_link.0.%location: handle=\_SB_.LNKA dev.pci_link.0.%pnpinfo: _HID=PNP0C0F _UID=1 dev.pci_link.0.%parent: acpi0 dev.pci_link.1.%desc: ACPI PCI Link LNKB dev.pci_link.1.%driver: pci_link dev.pci_link.1.%location: handle=\_SB_.LNKB dev.pci_link.1.%pnpinfo: _HID=PNP0C0F _UID=2 dev.pci_link.1.%parent: acpi0 dev.pci_link.2.%desc: ACPI PCI Link LNKC dev.pci_link.2.%driver: pci_link dev.pci_link.2.%location: handle=\_SB_.LNKC dev.pci_link.2.%pnpinfo: _HID=PNP0C0F _UID=3 dev.pci_link.2.%parent: acpi0 dev.pci_link.3.%desc: ACPI PCI Link LNKD dev.pci_link.3.%driver: pci_link dev.pci_link.3.%location: handle=\_SB_.LNKD dev.pci_link.3.%pnpinfo: _HID=PNP0C0F _UID=4 dev.pci_link.3.%parent: acpi0 dev.pci_link.4.%desc: ACPI PCI Link LNKE dev.pci_link.4.%driver: pci_link dev.pci_link.4.%location: handle=\_SB_.LNKE dev.pci_link.4.%pnpinfo: _HID=PNP0C0F _UID=5 dev.pci_link.4.%parent: acpi0 dev.pci_link.5.%desc: ACPI PCI Link LNKF dev.pci_link.5.%driver: pci_link dev.pci_link.5.%location: handle=\_SB_.LNKF dev.pci_link.5.%pnpinfo: _HID=PNP0C0F _UID=6 dev.pci_link.5.%parent: acpi0 dev.pci_link.6.%desc: ACPI PCI Link LNKG dev.pci_link.6.%driver: pci_link dev.pci_link.6.%location: handle=\_SB_.LNKG dev.pci_link.6.%pnpinfo: _HID=PNP0C0F _UID=7 dev.pci_link.6.%parent: acpi0 dev.pci_link.7.%desc: ACPI PCI Link LNKH dev.pci_link.7.%driver: pci_link dev.pci_link.7.%location: handle=\_SB_.LNKH dev.pci_link.7.%pnpinfo: _HID=PNP0C0F _UID=8 dev.pci_link.7.%parent: acpi0 dev.pcib.%parent: dev.pcib.0.%desc: ACPI Host-PCI bridge dev.pcib.0.%driver: pcib dev.pcib.0.%location: handle=\_SB_.PCI0 dev.pcib.0.%pnpinfo: _HID=PNP0A03 _UID=0 dev.pcib.0.%parent: acpi0 dev.pcib.0.wake: 0 dev.pcib.1.%desc: ACPI PCI-PCI bridge dev.pcib.1.%driver: pcib dev.pcib.1.%location: slot=2 function=0 handle=\_SB_.PCI0.PT02 dev.pcib.1.%pnpinfo: vendor=0x8086 device=0x25e2 subvendor=0x0000 subdevice=0x0000 class=0x060400 dev.pcib.1.%parent: pci0 dev.pcib.1.domain: 0 dev.pcib.1.pribus: 0 dev.pcib.1.secbus: 9 dev.pcib.1.subbus: 18 dev.pcib.2.%desc: ACPI PCI-PCI bridge dev.pcib.2.%driver: pcib dev.pcib.2.%location: slot=0 function=0 handle=\_SB_.PCI0.PT02.IPE4 dev.pcib.2.%pnpinfo: vendor=0x8086 device=0x3500 subvendor=0x0000 subdevice=0x0000 class=0x060400 dev.pcib.2.%parent: pci9 dev.pcib.2.domain: 0 dev.pcib.2.pribus: 9 dev.pcib.2.secbus: 10 dev.pcib.2.subbus: 15 dev.pcib.3.%desc: ACPI PCI-PCI bridge dev.pcib.3.%driver: pcib dev.pcib.3.%location: slot=0 function=0 handle=\_SB_.PCI0.PT02.IPE4.IPE1 dev.pcib.3.%pnpinfo: vendor=0x8086 device=0x3510 subvendor=0x0000 subdevice=0x0000 class=0x060400 dev.pcib.3.%parent: pci10 dev.pcib.3.domain: 0 dev.pcib.3.pribus: 10 dev.pcib.3.secbus: 11 dev.pcib.3.subbus: 13 dev.pcib.4.%desc: PCI-PCI bridge dev.pcib.4.%driver: pcib dev.pcib.4.%location: slot=1 function=0 dev.pcib.4.%pnpinfo: vendor=0x8086 device=0x3514 subvendor=0x0000 subdevice=0x0000 class=0x060400 dev.pcib.4.%parent: pci10 dev.pcib.4.domain: 0 dev.pcib.4.pribus: 10 dev.pcib.4.secbus: 14 dev.pcib.4.subbus: 14 dev.pcib.5.%desc: PCI-PCI bridge dev.pcib.5.%driver: pcib dev.pcib.5.%location: slot=2 function=0 dev.pcib.5.%pnpinfo: vendor=0x8086 device=0x3518 subvendor=0x0000 subdevice=0x0000 class=0x060400 dev.pcib.5.%parent: pci10 dev.pcib.5.domain: 0 dev.pcib.5.pribus: 10 dev.pcib.5.secbus: 15 dev.pcib.5.subbus: 15 dev.pcib.6.%desc: ACPI PCI-PCI bridge dev.pcib.6.%driver: pcib dev.pcib.6.%location: slot=0 function=3 handle=\_SB_.PCI0.PT02.P2P2 dev.pcib.6.%pnpinfo: vendor=0x8086 device=0x350c subvendor=0x0000 subdevice=0x0000 class=0x060400 dev.pcib.6.%parent: pci9 dev.pcib.6.domain: 0 dev.pcib.6.pribus: 9 dev.pcib.6.secbus: 16 dev.pcib.6.subbus: 18 dev.pcib.7.%desc: ACPI PCI-PCI bridge dev.pcib.7.%driver: pcib dev.pcib.7.%location: slot=3 function=0 handle=\_SB_.PCI0.PT03 dev.pcib.7.%pnpinfo: vendor=0x8086 device=0x25e3 subvendor=0x0000 subdevice=0x0000 class=0x060400 dev.pcib.7.%parent: pci0 dev.pcib.7.domain: 0 dev.pcib.7.pribus: 0 dev.pcib.7.secbus: 6 dev.pcib.7.subbus: 8 dev.pcib.8.%desc: ACPI PCI-PCI bridge dev.pcib.8.%driver: pcib dev.pcib.8.%location: slot=4 function=0 handle=\_SB_.PCI0.PT04 dev.pcib.8.%pnpinfo: vendor=0x8086 device=0x25f8 subvendor=0x0000 subdevice=0x0000 class=0x060400 dev.pcib.8.%parent: pci0 dev.pcib.8.domain: 0 dev.pcib.8.pribus: 0 dev.pcib.8.secbus: 19 dev.pcib.8.subbus: 21 dev.pcib.9.%desc: PCI-PCI bridge dev.pcib.9.%driver: pcib dev.pcib.9.%location: slot=5 function=0 dev.pcib.9.%pnpinfo: vendor=0x8086 device=0x25e5 subvendor=0x0000 subdevice=0x0000 class=0x060400 dev.pcib.9.%parent: pci0 dev.pcib.9.domain: 0 dev.pcib.9.pribus: 0 dev.pcib.9.secbus: 22 dev.pcib.9.subbus: 22 dev.pcib.10.%desc: ACPI PCI-PCI bridge dev.pcib.10.%driver: pcib dev.pcib.10.%location: slot=6 function=0 handle=\_SB_.PCI0.PT06 dev.pcib.10.%pnpinfo: vendor=0x8086 device=0x25e6 subvendor=0x0000 subdevice=0x0000 class=0x060400 dev.pcib.10.%parent: pci0 dev.pcib.10.domain: 0 dev.pcib.10.pribus: 0 dev.pcib.10.secbus: 2 dev.pcib.10.subbus: 3 dev.pcib.11.%desc: ACPI PCI-PCI bridge dev.pcib.11.%driver: pcib dev.pcib.11.%location: slot=0 function=0 handle=\_SB_.PCI0.PT06.NB01 dev.pcib.11.%pnpinfo: vendor=0x1166 device=0x0103 subvendor=0x0000 subdevice=0x0000 class=0x060400 dev.pcib.11.%parent: pci2 dev.pcib.11.domain: 0 dev.pcib.11.pribus: 2 dev.pcib.11.secbus: 3 dev.pcib.11.subbus: 3 dev.pcib.12.%desc: ACPI PCI-PCI bridge dev.pcib.12.%driver: pcib dev.pcib.12.%location: slot=7 function=0 handle=\_SB_.PCI0.PT07 dev.pcib.12.%pnpinfo: vendor=0x8086 device=0x25e7 subvendor=0x0000 subdevice=0x0000 class=0x060400 dev.pcib.12.%parent: pci0 dev.pcib.12.domain: 0 dev.pcib.12.pribus: 0 dev.pcib.12.secbus: 4 dev.pcib.12.subbus: 5 dev.pcib.13.%desc: ACPI PCI-PCI bridge dev.pcib.13.%driver: pcib dev.pcib.13.%location: slot=0 function=0 handle=\_SB_.PCI0.PT07.NB02 dev.pcib.13.%pnpinfo: vendor=0x1166 device=0x0103 subvendor=0x0000 subdevice=0x0000 class=0x060400 dev.pcib.13.%parent: pci4 dev.pcib.13.domain: 0 dev.pcib.13.pribus: 4 dev.pcib.13.secbus: 5 dev.pcib.13.subbus: 5 dev.pcib.14.%desc: ACPI PCI-PCI bridge dev.pcib.14.%driver: pcib dev.pcib.14.%location: slot=30 function=0 handle=\_SB_.PCI0.IP2P dev.pcib.14.%pnpinfo: vendor=0x8086 device=0x244e subvendor=0x103c subdevice=0x31fe class=0x060401 dev.pcib.14.%parent: pci0 dev.pcib.14.domain: 0 dev.pcib.14.pribus: 0 dev.pcib.14.secbus: 1 dev.pcib.14.subbus: 1 dev.pci.%parent: dev.pci.0.%desc: ACPI PCI bus dev.pci.0.%driver: pci dev.pci.0.%location: dev.pci.0.%pnpinfo: dev.pci.0.%parent: pcib0 dev.pci.0.wake: 0 dev.pci.9.%desc: ACPI PCI bus dev.pci.9.%driver: pci dev.pci.9.%location: dev.pci.9.%pnpinfo: dev.pci.9.%parent: pcib1 dev.pci.10.%desc: ACPI PCI bus dev.pci.10.%driver: pci dev.pci.10.%location: dev.pci.10.%pnpinfo: dev.pci.10.%parent: pcib2 dev.pci.11.%desc: ACPI PCI bus dev.pci.11.%driver: pci dev.pci.11.%location: dev.pci.11.%pnpinfo: dev.pci.11.%parent: pcib3 dev.pci.14.%desc: PCI bus dev.pci.14.%driver: pci dev.pci.14.%location: dev.pci.14.%pnpinfo: dev.pci.14.%parent: pcib4 dev.pci.15.%desc: PCI bus dev.pci.15.%driver: pci dev.pci.15.%location: dev.pci.15.%pnpinfo: dev.pci.15.%parent: pcib5 dev.pci.16.%desc: ACPI PCI bus dev.pci.16.%driver: pci dev.pci.16.%location: dev.pci.16.%pnpinfo: dev.pci.16.%parent: pcib6 dev.pci.6.%desc: ACPI PCI bus dev.pci.6.%driver: pci dev.pci.6.%location: dev.pci.6.%pnpinfo: dev.pci.6.%parent: pcib7 dev.pci.19.%desc: ACPI PCI bus dev.pci.19.%driver: pci dev.pci.19.%location: dev.pci.19.%pnpinfo: dev.pci.19.%parent: pcib8 dev.pci.22.%desc: PCI bus dev.pci.22.%driver: pci dev.pci.22.%location: dev.pci.22.%pnpinfo: dev.pci.22.%parent: pcib9 dev.pci.2.%desc: ACPI PCI bus dev.pci.2.%driver: pci dev.pci.2.%location: dev.pci.2.%pnpinfo: dev.pci.2.%parent: pcib10 dev.pci.3.%desc: ACPI PCI bus dev.pci.3.%driver: pci dev.pci.3.%location: dev.pci.3.%pnpinfo: dev.pci.3.%parent: pcib11 dev.pci.4.%desc: ACPI PCI bus dev.pci.4.%driver: pci dev.pci.4.%location: dev.pci.4.%pnpinfo: dev.pci.4.%parent: pcib12 dev.pci.5.%desc: ACPI PCI bus dev.pci.5.%driver: pci dev.pci.5.%location: dev.pci.5.%pnpinfo: dev.pci.5.%parent: pcib13 dev.pci.1.%desc: ACPI PCI bus dev.pci.1.%driver: pci dev.pci.1.%location: dev.pci.1.%pnpinfo: dev.pci.1.%parent: pcib14 dev.hostb.%parent: dev.hostb.0.%desc: Host to PCI bridge dev.hostb.0.%driver: hostb dev.hostb.0.%location: slot=0 function=0 dev.hostb.0.%pnpinfo: vendor=0x8086 device=0x25d8 subvendor=0x103c subdevice=0x31fd class=0x060000 dev.hostb.0.%parent: pci0 dev.hostb.1.%desc: Host to PCI bridge dev.hostb.1.%driver: hostb dev.hostb.1.%location: slot=16 function=0 dev.hostb.1.%pnpinfo: vendor=0x8086 device=0x25f0 subvendor=0x103c subdevice=0x31fd class=0x060000 dev.hostb.1.%parent: pci0 dev.hostb.2.%desc: Host to PCI bridge dev.hostb.2.%driver: hostb dev.hostb.2.%location: slot=16 function=1 handle=\_SB_.PCI0.CFG0 dev.hostb.2.%pnpinfo: vendor=0x8086 device=0x25f0 subvendor=0x103c subdevice=0x31fd class=0x060000 dev.hostb.2.%parent: pci0 dev.hostb.3.%desc: Host to PCI bridge dev.hostb.3.%driver: hostb dev.hostb.3.%location: slot=16 function=2 dev.hostb.3.%pnpinfo: vendor=0x8086 device=0x25f0 subvendor=0x103c subdevice=0x31fd class=0x060000 dev.hostb.3.%parent: pci0 dev.hostb.4.%desc: Host to PCI bridge dev.hostb.4.%driver: hostb dev.hostb.4.%location: slot=17 function=0 dev.hostb.4.%pnpinfo: vendor=0x8086 device=0x25f1 subvendor=0x103c subdevice=0x31fd class=0x060000 dev.hostb.4.%parent: pci0 dev.hostb.5.%desc: Host to PCI bridge dev.hostb.5.%driver: hostb dev.hostb.5.%location: slot=19 function=0 dev.hostb.5.%pnpinfo: vendor=0x8086 device=0x25f3 subvendor=0x103c subdevice=0x31fd class=0x060000 dev.hostb.5.%parent: pci0 dev.hostb.6.%desc: Host to PCI bridge dev.hostb.6.%driver: hostb dev.hostb.6.%location: slot=21 function=0 dev.hostb.6.%pnpinfo: vendor=0x8086 device=0x25f5 subvendor=0x103c subdevice=0x31fd class=0x060000 dev.hostb.6.%parent: pci0 dev.hostb.7.%desc: Host to PCI bridge dev.hostb.7.%driver: hostb dev.hostb.7.%location: slot=22 function=0 dev.hostb.7.%pnpinfo: vendor=0x8086 device=0x25f6 subvendor=0x103c subdevice=0x31fd class=0x060000 dev.hostb.7.%parent: pci0 dev.ciss.%parent: dev.ciss.0.%desc: HP Smart Array P400i dev.ciss.0.%driver: ciss dev.ciss.0.%location: slot=0 function=0 dev.ciss.0.%pnpinfo: vendor=0x103c device=0x3230 subvendor=0x103c subdevice=0x3235 class=0x010400 dev.ciss.0.%parent: pci6 dev.ciss.0.soft_reset: 0 dev.bce.%parent: dev.bce.0.%desc: HP NC373i Multifunction Gigabit Server Adapter (B2) dev.bce.0.%driver: bce dev.bce.0.%location: slot=0 function=0 dev.bce.0.%pnpinfo: vendor=0x14e4 device=0x164c subvendor=0x103c subdevice=0x7038 class=0x020000 dev.bce.0.%parent: pci3 dev.bce.0.l2fhdr_error_count: 0 dev.bce.0.mbuf_alloc_failed_count: 201 dev.bce.0.mbuf_frag_count: 207 dev.bce.0.dma_map_addr_rx_failed_count: 0 dev.bce.0.dma_map_addr_tx_failed_count: 0 dev.bce.0.unexpected_attention_count: 0 dev.bce.0.stat_IfHcInOctets: 2277358024 dev.bce.0.stat_IfHCInBadOctets: 1278432 dev.bce.0.stat_IfHCOutOctets: 63981414158 dev.bce.0.stat_IfHCOutBadOctets: 0 dev.bce.0.stat_IfHCInUcastPkts: 27336477 dev.bce.0.stat_IfHCInMulticastPkts: 922 dev.bce.0.stat_IfHCInBroadcastPkts: 190372 dev.bce.0.stat_IfHCOutUcastPkts: 47820956 dev.bce.0.stat_IfHCOutMulticastPkts: 5 dev.bce.0.stat_IfHCOutBroadcastPkts: 1 dev.bce.0.stat_emac_tx_stat_dot3statsinternalmactransmiterrors: 0 dev.bce.0.stat_Dot3StatsCarrierSenseErrors: 0 dev.bce.0.stat_Dot3StatsFCSErrors: 0 dev.bce.0.stat_Dot3StatsAlignmentErrors: 0 dev.bce.0.stat_Dot3StatsSingleCollisionFrames: 0 dev.bce.0.stat_Dot3StatsMultipleCollisionFrames: 0 dev.bce.0.stat_Dot3StatsDeferredTransmissions: 0 dev.bce.0.stat_Dot3StatsExcessiveCollisions: 0 dev.bce.0.stat_Dot3StatsLateCollisions: 0 dev.bce.0.stat_EtherStatsCollisions: 0 dev.bce.0.stat_EtherStatsFragments: 0 dev.bce.0.stat_EtherStatsJabbers: 0 dev.bce.0.stat_EtherStatsUndersizePkts: 0 dev.bce.0.stat_EtherStatsOversizePkts: 0 dev.bce.0.stat_EtherStatsPktsRx64Octets: 1022043 dev.bce.0.stat_EtherStatsPktsRx65Octetsto127Octets: 24978056 dev.bce.0.stat_EtherStatsPktsRx128Octetsto255Octets: 1333800 dev.bce.0.stat_EtherStatsPktsRx256Octetsto511Octets: 39850 dev.bce.0.stat_EtherStatsPktsRx512Octetsto1023Octets: 52295 dev.bce.0.stat_EtherStatsPktsRx1024Octetsto1522Octets: 101727 dev.bce.0.stat_EtherStatsPktsRx1523Octetsto9022Octets: 0 dev.bce.0.stat_EtherStatsPktsTx64Octets: 46522 dev.bce.0.stat_EtherStatsPktsTx65Octetsto127Octets: 1456745 dev.bce.0.stat_EtherStatsPktsTx128Octetsto255Octets: 2273188 dev.bce.0.stat_EtherStatsPktsTx256Octetsto511Octets: 929003 dev.bce.0.stat_EtherStatsPktsTx512Octetsto1023Octets: 2292647 dev.bce.0.stat_EtherStatsPktsTx1024Octetsto1522Octets: 40822857 dev.bce.0.stat_EtherStatsPktsTx1523Octetsto9022Octets: 0 dev.bce.0.stat_XonPauseFramesReceived: 0 dev.bce.0.stat_XoffPauseFramesReceived: 0 dev.bce.0.stat_OutXonSent: 0 dev.bce.0.stat_OutXoffSent: 0 dev.bce.0.stat_FlowControlDone: 0 dev.bce.0.stat_MacControlFramesReceived: 0 dev.bce.0.stat_XoffStateEntered: 0 dev.bce.0.stat_IfInFramesL2FilterDiscards: 3167 dev.bce.0.stat_IfInRuleCheckerDiscards: 0 dev.bce.0.stat_IfInFTQDiscards: 0 dev.bce.0.stat_IfInMBUFDiscards: 0 dev.bce.0.stat_IfInRuleCheckerP4Hit: 191293 dev.bce.0.stat_CatchupInRuleCheckerDiscards: 0 dev.bce.0.stat_CatchupInFTQDiscards: 0 dev.bce.0.stat_CatchupInMBUFDiscards: 0 dev.bce.0.stat_CatchupInRuleCheckerP4Hit: 0 dev.bce.0.com_no_buffers: 0 dev.bce.1.%desc: HP NC373i Multifunction Gigabit Server Adapter (B2) dev.bce.1.%driver: bce dev.bce.1.%location: slot=0 function=0 dev.bce.1.%pnpinfo: vendor=0x14e4 device=0x164c subvendor=0x103c subdevice=0x7038 class=0x020000 dev.bce.1.%parent: pci5 dev.bce.1.l2fhdr_error_count: 0 dev.bce.1.mbuf_alloc_failed_count: 62 dev.bce.1.mbuf_frag_count: 63 dev.bce.1.dma_map_addr_rx_failed_count: 0 dev.bce.1.dma_map_addr_tx_failed_count: 0 dev.bce.1.unexpected_attention_count: 0 dev.bce.1.stat_IfHcInOctets: 3098977792 dev.bce.1.stat_IfHCInBadOctets: 1278866 dev.bce.1.stat_IfHCOutOctets: 111245491475 dev.bce.1.stat_IfHCOutBadOctets: 0 dev.bce.1.stat_IfHCInUcastPkts: 39648145 dev.bce.1.stat_IfHCInMulticastPkts: 922 dev.bce.1.stat_IfHCInBroadcastPkts: 190372 dev.bce.1.stat_IfHCOutUcastPkts: 78809756 dev.bce.1.stat_IfHCOutMulticastPkts: 0 dev.bce.1.stat_IfHCOutBroadcastPkts: 0 dev.bce.1.stat_emac_tx_stat_dot3statsinternalmactransmiterrors: 0 dev.bce.1.stat_Dot3StatsCarrierSenseErrors: 0 dev.bce.1.stat_Dot3StatsFCSErrors: 0 dev.bce.1.stat_Dot3StatsAlignmentErrors: 0 dev.bce.1.stat_Dot3StatsSingleCollisionFrames: 0 dev.bce.1.stat_Dot3StatsMultipleCollisionFrames: 0 dev.bce.1.stat_Dot3StatsDeferredTransmissions: 0 dev.bce.1.stat_Dot3StatsExcessiveCollisions: 0 dev.bce.1.stat_Dot3StatsLateCollisions: 0 dev.bce.1.stat_EtherStatsCollisions: 0 dev.bce.1.stat_EtherStatsFragments: 0 dev.bce.1.stat_EtherStatsJabbers: 0 dev.bce.1.stat_EtherStatsUndersizePkts: 0 dev.bce.1.stat_EtherStatsOversizePkts: 0 dev.bce.1.stat_EtherStatsPktsRx64Octets: 1763213 dev.bce.1.stat_EtherStatsPktsRx65Octetsto127Octets: 36785937 dev.bce.1.stat_EtherStatsPktsRx128Octetsto255Octets: 1121050 dev.bce.1.stat_EtherStatsPktsRx256Octetsto511Octets: 36478 dev.bce.1.stat_EtherStatsPktsRx512Octetsto1023Octets: 48486 dev.bce.1.stat_EtherStatsPktsRx1024Octetsto1522Octets: 84275 dev.bce.1.stat_EtherStatsPktsRx1523Octetsto9022Octets: 0 dev.bce.1.stat_EtherStatsPktsTx64Octets: 37872 dev.bce.1.stat_EtherStatsPktsTx65Octetsto127Octets: 1359859 dev.bce.1.stat_EtherStatsPktsTx128Octetsto255Octets: 1511410 dev.bce.1.stat_EtherStatsPktsTx256Octetsto511Octets: 1523182 dev.bce.1.stat_EtherStatsPktsTx512Octetsto1023Octets: 2635770 dev.bce.1.stat_EtherStatsPktsTx1024Octetsto1522Octets: 71741663 dev.bce.1.stat_EtherStatsPktsTx1523Octetsto9022Octets: 0 dev.bce.1.stat_XonPauseFramesReceived: 0 dev.bce.1.stat_XoffPauseFramesReceived: 0 dev.bce.1.stat_OutXonSent: 0 dev.bce.1.stat_OutXoffSent: 0 dev.bce.1.stat_FlowControlDone: 0 dev.bce.1.stat_MacControlFramesReceived: 0 dev.bce.1.stat_XoffStateEntered: 0 dev.bce.1.stat_IfInFramesL2FilterDiscards: 3172 dev.bce.1.stat_IfInRuleCheckerDiscards: 0 dev.bce.1.stat_IfInFTQDiscards: 0 dev.bce.1.stat_IfInMBUFDiscards: 0 dev.bce.1.stat_IfInRuleCheckerP4Hit: 191294 dev.bce.1.stat_CatchupInRuleCheckerDiscards: 0 dev.bce.1.stat_CatchupInFTQDiscards: 0 dev.bce.1.stat_CatchupInMBUFDiscards: 0 dev.bce.1.stat_CatchupInRuleCheckerP4Hit: 0 dev.bce.1.com_no_buffers: 0 dev.miibus.%parent: dev.miibus.0.%desc: MII bus dev.miibus.0.%driver: miibus dev.miibus.0.%location: dev.miibus.0.%pnpinfo: dev.miibus.0.%parent: bce0 dev.miibus.1.%desc: MII bus dev.miibus.1.%driver: miibus dev.miibus.1.%location: dev.miibus.1.%pnpinfo: dev.miibus.1.%parent: bce1 dev.brgphy.%parent: dev.brgphy.0.%desc: BCM5708C 1000BASE-T media interface dev.brgphy.0.%driver: brgphy dev.brgphy.0.%location: phyno=1 dev.brgphy.0.%pnpinfo: oui=0x1018 model=0x36 rev=0x6 dev.brgphy.0.%parent: miibus0 dev.brgphy.1.%desc: BCM5708C 1000BASE-T media interface dev.brgphy.1.%driver: brgphy dev.brgphy.1.%location: phyno=1 dev.brgphy.1.%pnpinfo: oui=0x1018 model=0x36 rev=0x6 dev.brgphy.1.%parent: miibus1 dev.uhci.%parent: dev.uhci.0.%desc: Intel 631XESB/632XESB/3100 USB controller USB-1 dev.uhci.0.%driver: uhci dev.uhci.0.%location: slot=29 function=0 dev.uhci.0.%pnpinfo: vendor=0x8086 device=0x2688 subvendor=0x103c subdevice=0x31fe class=0x0c0300 dev.uhci.0.%parent: pci0 dev.uhci.1.%desc: Intel 631XESB/632XESB/3100 USB controller USB-2 dev.uhci.1.%driver: uhci dev.uhci.1.%location: slot=29 function=1 dev.uhci.1.%pnpinfo: vendor=0x8086 device=0x2689 subvendor=0x103c subdevice=0x31fe class=0x0c0300 dev.uhci.1.%parent: pci0 dev.uhci.2.%desc: Intel 631XESB/632XESB/3100 USB controller USB-3 dev.uhci.2.%driver: uhci dev.uhci.2.%location: slot=29 function=2 dev.uhci.2.%pnpinfo: vendor=0x8086 device=0x268a subvendor=0x103c subdevice=0x31fe class=0x0c0300 dev.uhci.2.%parent: pci0 dev.uhci.3.%desc: Intel 631XESB/632XESB/3100 USB controller USB-4 dev.uhci.3.%driver: uhci dev.uhci.3.%location: slot=29 function=3 dev.uhci.3.%pnpinfo: vendor=0x8086 device=0x268b subvendor=0x103c subdevice=0x31fe class=0x0c0300 dev.uhci.3.%parent: pci0 dev.uhci.4.%desc: UHCI (generic) USB controller dev.uhci.4.%driver: uhci dev.uhci.4.%location: slot=4 function=4 dev.uhci.4.%pnpinfo: vendor=0x103c device=0x3300 subvendor=0x103c subdevice=0x3305 class=0x0c0300 dev.uhci.4.%parent: pci1 dev.usbus.%parent: dev.usbus.0.%desc: dev.usbus.0.%driver: usbus dev.usbus.0.%location: dev.usbus.0.%pnpinfo: dev.usbus.0.%parent: uhci0 dev.usbus.1.%desc: dev.usbus.1.%driver: usbus dev.usbus.1.%location: dev.usbus.1.%pnpinfo: dev.usbus.1.%parent: uhci1 dev.usbus.2.%desc: dev.usbus.2.%driver: usbus dev.usbus.2.%location: dev.usbus.2.%pnpinfo: dev.usbus.2.%parent: uhci2 dev.usbus.3.%desc: dev.usbus.3.%driver: usbus dev.usbus.3.%location: dev.usbus.3.%pnpinfo: dev.usbus.3.%parent: uhci3 dev.usbus.4.%desc: dev.usbus.4.%driver: usbus dev.usbus.4.%location: dev.usbus.4.%pnpinfo: dev.usbus.4.%parent: ehci0 dev.usbus.5.%desc: dev.usbus.5.%driver: usbus dev.usbus.5.%location: dev.usbus.5.%pnpinfo: dev.usbus.5.%parent: uhci4 dev.ehci.%parent: dev.ehci.0.%desc: Intel 63XXESB USB 2.0 controller dev.ehci.0.%driver: ehci dev.ehci.0.%location: slot=29 function=7 dev.ehci.0.%pnpinfo: vendor=0x8086 device=0x268c subvendor=0x103c subdevice=0x31fe class=0x0c0320 dev.ehci.0.%parent: pci0 dev.vgapci.%parent: dev.vgapci.0.%desc: VGA-compatible display dev.vgapci.0.%driver: vgapci dev.vgapci.0.%location: slot=3 function=0 dev.vgapci.0.%pnpinfo: vendor=0x1002 device=0x515e subvendor=0x103c subdevice=0x31fb class=0x030000 dev.vgapci.0.%parent: pci1 dev.ipmi.%parent: dev.ipmi.0.%desc: IPMI System Interface dev.ipmi.0.%driver: ipmi dev.ipmi.0.%location: slot=4 function=6 dev.ipmi.0.%pnpinfo: vendor=0x103c device=0x3302 subvendor=0x103c subdevice=0x3305 class=0x0c0701 dev.ipmi.0.%parent: pci1 dev.isab.%parent: dev.isab.0.%desc: PCI-ISA bridge dev.isab.0.%driver: isab dev.isab.0.%location: slot=31 function=0 handle=\_SB_.PCI0.IBRG dev.isab.0.%pnpinfo: vendor=0x8086 device=0x2670 subvendor=0x0000 subdevice=0x0000 class=0x060100 dev.isab.0.%parent: pci0 dev.isa.%parent: dev.isa.0.%desc: ISA bus dev.isa.0.%driver: isa dev.isa.0.%location: dev.isa.0.%pnpinfo: dev.isa.0.%parent: isab0 dev.atapci.%parent: dev.atapci.0.%desc: Intel 63XXESB2 UDMA100 controller dev.atapci.0.%driver: atapci dev.atapci.0.%location: slot=31 function=1 dev.atapci.0.%pnpinfo: vendor=0x8086 device=0x269e subvendor=0x103c subdevice=0x31fe class=0x01018a dev.atapci.0.%parent: pci0 dev.ata.%parent: dev.ata.0.%desc: ATA channel dev.ata.0.%driver: ata dev.ata.0.%location: channel=0 dev.ata.0.%pnpinfo: dev.ata.0.%parent: atapci0 dev.acpi_tz.%parent: dev.acpi_tz.0.%desc: Thermal Zone dev.acpi_tz.0.%driver: acpi_tz dev.acpi_tz.0.%location: handle=\_TZ_.THM0 dev.acpi_tz.0.%pnpinfo: _HID=none _UID=0 dev.acpi_tz.0.%parent: acpi0 dev.atdma.%parent: dev.atdma.0.%desc: AT DMA controller dev.atdma.0.%driver: atdma dev.atdma.0.%location: handle=\_SB_.PCI0.IBRG.DMA0 dev.atdma.0.%pnpinfo: _HID=PNP0200 _UID=0 dev.atdma.0.%parent: acpi0 dev.atkbdc.%parent: dev.atkbdc.0.%desc: Keyboard controller (i8042) dev.atkbdc.0.%driver: atkbdc dev.atkbdc.0.%location: handle=\_SB_.PCI0.IBRG.KBD_ dev.atkbdc.0.%pnpinfo: _HID=PNP0303 _UID=0 dev.atkbdc.0.%parent: acpi0 dev.atkbd.%parent: dev.atkbd.0.%desc: AT Keyboard dev.atkbd.0.%driver: atkbd dev.atkbd.0.%location: dev.atkbd.0.%pnpinfo: dev.atkbd.0.%parent: atkbdc0 dev.psmcpnp.%parent: dev.psmcpnp.0.%desc: PS/2 mouse port dev.psmcpnp.0.%driver: psmcpnp dev.psmcpnp.0.%location: handle=\_SB_.PCI0.IBRG.PS2M dev.psmcpnp.0.%pnpinfo: _HID=PNP0F13 _UID=0 dev.psmcpnp.0.%parent: acpi0 dev.psm.%parent: dev.psm.0.%desc: PS/2 Mouse dev.psm.0.%driver: psm dev.psm.0.%location: dev.psm.0.%pnpinfo: dev.psm.0.%parent: atkbdc0 dev.uart.%parent: dev.uart.0.%desc: 16550 or compatible dev.uart.0.%driver: uart dev.uart.0.%location: handle=\_SB_.PCI0.IBRG.S417.COMA dev.uart.0.%pnpinfo: _HID=PNP0501 _UID=0 dev.uart.0.%parent: acpi0 dev.uart.1.%desc: Non-standard ns8250 class UART with FIFOs dev.uart.1.%driver: uart dev.uart.1.%location: dev.uart.1.%pnpinfo: dev.uart.1.%parent: isa0 dev.apic.%parent: dev.apic.0.%desc: APIC resources dev.apic.0.%driver: apic dev.apic.0.%location: dev.apic.0.%pnpinfo: dev.apic.0.%parent: nexus0 dev.orm.%parent: dev.orm.0.%desc: ISA Option ROMs dev.orm.0.%driver: orm dev.orm.0.%location: dev.orm.0.%pnpinfo: dev.orm.0.%parent: isa0 dev.sc.%parent: dev.sc.0.%desc: System console dev.sc.0.%driver: sc dev.sc.0.%location: dev.sc.0.%pnpinfo: dev.sc.0.%parent: isa0 dev.vga.%parent: dev.vga.0.%desc: Generic ISA VGA dev.vga.0.%driver: vga dev.vga.0.%location: dev.vga.0.%pnpinfo: dev.vga.0.%parent: isa0 dev.atrtc.%parent: dev.atrtc.0.%desc: AT realtime clock dev.atrtc.0.%driver: atrtc dev.atrtc.0.%location: dev.atrtc.0.%pnpinfo: dev.atrtc.0.%parent: isa0 dev.est.%parent: dev.p4tcc.%parent: dev.p4tcc.0.%desc: CPU Frequency Thermal Control dev.p4tcc.0.%driver: p4tcc dev.p4tcc.0.%location: dev.p4tcc.0.%pnpinfo: dev.p4tcc.0.%parent: cpu0 dev.p4tcc.0.freq_settings: 10000/-1 8750/-1 7500/-1 6250/-1 5000/-1 3750/-1 2500/-1 1250/-1 dev.p4tcc.1.%desc: CPU Frequency Thermal Control dev.p4tcc.1.%driver: p4tcc dev.p4tcc.1.%location: dev.p4tcc.1.%pnpinfo: dev.p4tcc.1.%parent: cpu1 dev.p4tcc.1.freq_settings: 10000/-1 8750/-1 7500/-1 6250/-1 5000/-1 3750/-1 2500/-1 1250/-1 dev.p4tcc.2.%desc: CPU Frequency Thermal Control dev.p4tcc.2.%driver: p4tcc dev.p4tcc.2.%location: dev.p4tcc.2.%pnpinfo: dev.p4tcc.2.%parent: cpu2 dev.p4tcc.2.freq_settings: 10000/-1 8750/-1 7500/-1 6250/-1 5000/-1 3750/-1 2500/-1 1250/-1 dev.p4tcc.3.%desc: CPU Frequency Thermal Control dev.p4tcc.3.%driver: p4tcc dev.p4tcc.3.%location: dev.p4tcc.3.%pnpinfo: dev.p4tcc.3.%parent: cpu3 dev.p4tcc.3.freq_settings: 10000/-1 8750/-1 7500/-1 6250/-1 5000/-1 3750/-1 2500/-1 1250/-1 dev.p4tcc.4.%desc: CPU Frequency Thermal Control dev.p4tcc.4.%driver: p4tcc dev.p4tcc.4.%location: dev.p4tcc.4.%pnpinfo: dev.p4tcc.4.%parent: cpu4 dev.p4tcc.4.freq_settings: 10000/-1 8750/-1 7500/-1 6250/-1 5000/-1 3750/-1 2500/-1 1250/-1 dev.p4tcc.5.%desc: CPU Frequency Thermal Control dev.p4tcc.5.%driver: p4tcc dev.p4tcc.5.%location: dev.p4tcc.5.%pnpinfo: dev.p4tcc.5.%parent: cpu5 dev.p4tcc.5.freq_settings: 10000/-1 8750/-1 7500/-1 6250/-1 5000/-1 3750/-1 2500/-1 1250/-1 dev.p4tcc.6.%desc: CPU Frequency Thermal Control dev.p4tcc.6.%driver: p4tcc dev.p4tcc.6.%location: dev.p4tcc.6.%pnpinfo: dev.p4tcc.6.%parent: cpu6 dev.p4tcc.6.freq_settings: 10000/-1 8750/-1 7500/-1 6250/-1 5000/-1 3750/-1 2500/-1 1250/-1 dev.p4tcc.7.%desc: CPU Frequency Thermal Control dev.p4tcc.7.%driver: p4tcc dev.p4tcc.7.%location: dev.p4tcc.7.%pnpinfo: dev.p4tcc.7.%parent: cpu7 dev.p4tcc.7.freq_settings: 10000/-1 8750/-1 7500/-1 6250/-1 5000/-1 3750/-1 2500/-1 1250/-1 dev.cpufreq.%parent: dev.cpufreq.0.%desc: dev.cpufreq.0.%driver: cpufreq dev.cpufreq.0.%location: dev.cpufreq.0.%pnpinfo: dev.cpufreq.0.%parent: cpu0 dev.cpufreq.1.%desc: dev.cpufreq.1.%driver: cpufreq dev.cpufreq.1.%location: dev.cpufreq.1.%pnpinfo: dev.cpufreq.1.%parent: cpu1 dev.cpufreq.2.%desc: dev.cpufreq.2.%driver: cpufreq dev.cpufreq.2.%location: dev.cpufreq.2.%pnpinfo: dev.cpufreq.2.%parent: cpu2 dev.cpufreq.3.%desc: dev.cpufreq.3.%driver: cpufreq dev.cpufreq.3.%location: dev.cpufreq.3.%pnpinfo: dev.cpufreq.3.%parent: cpu3 dev.cpufreq.4.%desc: dev.cpufreq.4.%driver: cpufreq dev.cpufreq.4.%location: dev.cpufreq.4.%pnpinfo: dev.cpufreq.4.%parent: cpu4 dev.cpufreq.5.%desc: dev.cpufreq.5.%driver: cpufreq dev.cpufreq.5.%location: dev.cpufreq.5.%pnpinfo: dev.cpufreq.5.%parent: cpu5 dev.cpufreq.6.%desc: dev.cpufreq.6.%driver: cpufreq dev.cpufreq.6.%location: dev.cpufreq.6.%pnpinfo: dev.cpufreq.6.%parent: cpu6 dev.cpufreq.7.%desc: dev.cpufreq.7.%driver: cpufreq dev.cpufreq.7.%location: dev.cpufreq.7.%pnpinfo: dev.cpufreq.7.%parent: cpu7 dev.uhub.%parent: dev.uhub.0.%desc: Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1 dev.uhub.0.%driver: uhub dev.uhub.0.%location: dev.uhub.0.%pnpinfo: dev.uhub.0.%parent: usbus1 dev.uhub.1.%desc: Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1 dev.uhub.1.%driver: uhub dev.uhub.1.%location: dev.uhub.1.%pnpinfo: dev.uhub.1.%parent: usbus0 dev.uhub.2.%desc: Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1 dev.uhub.2.%driver: uhub dev.uhub.2.%location: dev.uhub.2.%pnpinfo: dev.uhub.2.%parent: usbus4 dev.uhub.3.%desc: Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1 dev.uhub.3.%driver: uhub dev.uhub.3.%location: dev.uhub.3.%pnpinfo: dev.uhub.3.%parent: usbus3 dev.uhub.4.%desc: Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1 dev.uhub.4.%driver: uhub dev.uhub.4.%location: dev.uhub.4.%pnpinfo: dev.uhub.4.%parent: usbus2 dev.uhub.5.%desc: 0x103c UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1 dev.uhub.5.%driver: uhub dev.uhub.5.%location: dev.uhub.5.%pnpinfo: dev.uhub.5.%parent: usbus5 dev.ukbd.%parent: dev.ukbd.0.%desc: Virtual Keyboard dev.ukbd.0.%driver: ukbd dev.ukbd.0.%location: bus=0 hubaddr=1 port=1 devaddr=2 interface=0 dev.ukbd.0.%pnpinfo: vendor=0x03f0 product=0x1027 devclass=0x00 devsubclass=0x00 sernum="" release=0x0002 mode=host intclass=0x03 intsubclass=0x01 intprotocol=0x01 dev.ukbd.0.%parent: uhub1 dev.ums.%parent: dev.ums.0.%desc: Virtual Mouse dev.ums.0.%driver: ums dev.ums.0.%location: bus=0 hubaddr=1 port=1 devaddr=2 interface=1 dev.ums.0.%pnpinfo: vendor=0x03f0 product=0x1027 devclass=0x00 devsubclass=0x00 sernum="" release=0x0002 mode=host intclass=0x03 intsubclass=0x01 intprotocol=0x02 dev.ums.0.%parent: uhub1 dev.ums.0.parseinfo: i1: X:r0, p8, s8; Y:r0, p16, s8; B1:r0, p0, s1; B2:r0, p1, s1; B3:r0, p2, s1; dev.uplcom.%parent: dev.uplcom.0.%desc: Prolific Technology Inc. USB-Serial Controller D, class 0/0, rev 1.10/4.00, addr 2 dev.uplcom.0.%driver: uplcom dev.uplcom.0.%location: bus=1 hubaddr=1 port=1 devaddr=2 interface=0 dev.uplcom.0.%pnpinfo: vendor=0x067b product=0x2303 devclass=0x00 devsubclass=0x00 sernum="" release=0x0400 mode=host intclass=0xff intsubclass=0x00 intprotocol=0x00 ttyname=U0 ttyports=1 dev.uplcom.0.%parent: uhub0 dev.uplcom.0.ttyname: U0 dev.uplcom.0.ttyports: 1 hptmv.status: RocketRAID 18xx SATA Controller driver Version v1.16 kstat.zfs.misc.zio_trim.bytes: 0 kstat.zfs.misc.zio_trim.success: 0 kstat.zfs.misc.zio_trim.unsupported: 109 kstat.zfs.misc.zio_trim.failed: 0 kstat.zfs.misc.xuio_stats.onloan_read_buf: 0 kstat.zfs.misc.xuio_stats.onloan_write_buf: 0 kstat.zfs.misc.xuio_stats.read_buf_copied: 0 kstat.zfs.misc.xuio_stats.read_buf_nocopy: 0 kstat.zfs.misc.xuio_stats.write_buf_copied: 0 kstat.zfs.misc.xuio_stats.write_buf_nocopy: 269861 kstat.zfs.misc.zfetchstats.hits: 138270999 kstat.zfs.misc.zfetchstats.misses: 33608242 kstat.zfs.misc.zfetchstats.colinear_hits: 12496 kstat.zfs.misc.zfetchstats.colinear_misses: 33595758 kstat.zfs.misc.zfetchstats.stride_hits: 135331775 kstat.zfs.misc.zfetchstats.stride_misses: 51809 kstat.zfs.misc.zfetchstats.reclaim_successes: 485007 kstat.zfs.misc.zfetchstats.reclaim_failures: 33110775 kstat.zfs.misc.zfetchstats.streams_resets: 2057 kstat.zfs.misc.zfetchstats.streams_noresets: 2939172 kstat.zfs.misc.zfetchstats.bogus_streams: 0 kstat.zfs.misc.zcompstats.attempts: 4525377 kstat.zfs.misc.zcompstats.empty: 70427 kstat.zfs.misc.zcompstats.skipped_insufficient_gain: 212315 kstat.zfs.misc.arcstats.hits: 71628686 kstat.zfs.misc.arcstats.misses: 5374019 kstat.zfs.misc.arcstats.demand_data_hits: 45721762 kstat.zfs.misc.arcstats.demand_data_misses: 204017 kstat.zfs.misc.arcstats.demand_metadata_hits: 23807762 kstat.zfs.misc.arcstats.demand_metadata_misses: 459411 kstat.zfs.misc.arcstats.prefetch_data_hits: 1068568 kstat.zfs.misc.arcstats.prefetch_data_misses: 4439809 kstat.zfs.misc.arcstats.prefetch_metadata_hits: 1030594 kstat.zfs.misc.arcstats.prefetch_metadata_misses: 270782 kstat.zfs.misc.arcstats.mru_hits: 7898276 kstat.zfs.misc.arcstats.mru_ghost_hits: 70775 kstat.zfs.misc.arcstats.mfu_hits: 62189559 kstat.zfs.misc.arcstats.mfu_ghost_hits: 539415 kstat.zfs.misc.arcstats.allocated: 6917433 kstat.zfs.misc.arcstats.deleted: 4576446 kstat.zfs.misc.arcstats.stolen: 1802887 kstat.zfs.misc.arcstats.recycle_miss: 144327 kstat.zfs.misc.arcstats.mutex_miss: 350 kstat.zfs.misc.arcstats.evict_skip: 55338 kstat.zfs.misc.arcstats.evict_l2_cached: 0 kstat.zfs.misc.arcstats.evict_l2_eligible: 197708696064 kstat.zfs.misc.arcstats.evict_l2_ineligible: 150685407744 kstat.zfs.misc.arcstats.hash_elements: 413749 kstat.zfs.misc.arcstats.hash_elements_max: 820251 kstat.zfs.misc.arcstats.hash_collisions: 6971217 kstat.zfs.misc.arcstats.hash_chains: 95225 kstat.zfs.misc.arcstats.hash_chain_max: 20 kstat.zfs.misc.arcstats.p: 8060231540 kstat.zfs.misc.arcstats.c: 12006640591 kstat.zfs.misc.arcstats.c_min: 1952117248 kstat.zfs.misc.arcstats.c_max: 15616937984 kstat.zfs.misc.arcstats.size: 11779040864 kstat.zfs.misc.arcstats.hdr_size: 106454016 kstat.zfs.misc.arcstats.data_size: 10649110528 kstat.zfs.misc.arcstats.other_size: 1023476320 kstat.zfs.misc.arcstats.l2_hits: 0 kstat.zfs.misc.arcstats.l2_misses: 0 kstat.zfs.misc.arcstats.l2_feeds: 0 kstat.zfs.misc.arcstats.l2_rw_clash: 0 kstat.zfs.misc.arcstats.l2_read_bytes: 0 kstat.zfs.misc.arcstats.l2_write_bytes: 0 kstat.zfs.misc.arcstats.l2_writes_sent: 0 kstat.zfs.misc.arcstats.l2_writes_done: 0 kstat.zfs.misc.arcstats.l2_writes_error: 0 kstat.zfs.misc.arcstats.l2_writes_hdr_miss: 0 kstat.zfs.misc.arcstats.l2_evict_lock_retry: 0 kstat.zfs.misc.arcstats.l2_evict_reading: 0 kstat.zfs.misc.arcstats.l2_free_on_write: 0 kstat.zfs.misc.arcstats.l2_abort_lowmem: 0 kstat.zfs.misc.arcstats.l2_cksum_bad: 0 kstat.zfs.misc.arcstats.l2_io_error: 0 kstat.zfs.misc.arcstats.l2_size: 0 kstat.zfs.misc.arcstats.l2_asize: 0 kstat.zfs.misc.arcstats.l2_hdr_size: 0 kstat.zfs.misc.arcstats.l2_compress_successes: 0 kstat.zfs.misc.arcstats.l2_compress_zeros: 0 kstat.zfs.misc.arcstats.l2_compress_failures: 0 kstat.zfs.misc.arcstats.l2_write_trylock_fail: 0 kstat.zfs.misc.arcstats.l2_write_passed_headroom: 0 kstat.zfs.misc.arcstats.l2_write_spa_mismatch: 0 kstat.zfs.misc.arcstats.l2_write_in_l2: 0 kstat.zfs.misc.arcstats.l2_write_io_in_progress: 0 kstat.zfs.misc.arcstats.l2_write_not_cacheable: 3033782 kstat.zfs.misc.arcstats.l2_write_full: 0 kstat.zfs.misc.arcstats.l2_write_buffer_iter: 0 kstat.zfs.misc.arcstats.l2_write_pios: 0 kstat.zfs.misc.arcstats.l2_write_buffer_bytes_scanned: 0 kstat.zfs.misc.arcstats.l2_write_buffer_list_iter: 0 kstat.zfs.misc.arcstats.l2_write_buffer_list_null_iter: 0 kstat.zfs.misc.arcstats.memory_throttle_count: 0 kstat.zfs.misc.arcstats.duplicate_buffers: 0 kstat.zfs.misc.arcstats.duplicate_buffers_size: 0 kstat.zfs.misc.arcstats.duplicate_reads: 2 kstat.zfs.misc.vdev_cache_stats.delegations: 0 kstat.zfs.misc.vdev_cache_stats.hits: 0 kstat.zfs.misc.vdev_cache_stats.misses: 0 Can I get some ideas on: 1) what MIGHT need tweaking 2) would (k|d)trace help? 3) what else could we find out what's being told no memory? -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-fs@FreeBSD.ORG Fri Jul 25 15:57:23 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7754E9AA for ; Fri, 25 Jul 2014 15:57:23 +0000 (UTC) Received: from florenca.unicamp.br (florenca.unicamp.br [143.106.10.159]) by mx1.freebsd.org (Postfix) with ESMTP id 2C7A92099 for ; Fri, 25 Jul 2014 15:57:22 +0000 (UTC) Received: from [143.106.64.207] (bioq15.ib.unicamp.br [143.106.64.207]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by florenca.unicamp.br (Postfix) with ESMTPSA id 607A510085D for ; Fri, 25 Jul 2014 12:57:19 -0300 (BRT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=unicamp.br; s=default; t=1406303840; bh=Yw172bEdKqaJmJLkCgUbRJEe8Ob8vGySIcMG/mdF4Vw=; h=From; b=gjseKtuhaECNHHhRHDxd/g0+mU7TwFZqlywp7dWxn93dZbQ9YmtJzPd7jtLchGFXn TkPFCe3EkOcO21xM4Ini8MG4X+f1mPcgK0w84JY6XNPr9FIIXSGL/I30zWIPzVyffb 5Aj2N01VLQXRhTkdd8nja4eGq0cY0ROddHyt+lBc= Message-ID: <53D27E5F.6060101@unicamp.br> Date: Fri, 25 Jul 2014 12:57:19 -0300 From: =?windows-1252?Q?=22F=E1bio_R=2E_Medeiros=22?= User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.0 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Subject: Backup in iSCSI - FS corruption Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2014 15:57:23 -0000 Hello, I'm creating a backup solution with Bacula running on FreeBSD10. The storage device is an iSCSI NAS (it's one made by Enhance Inc.) running RAID-5. The array of disks are using UFS2+J. Now I know this was a bad choice... I've got issues with FS corruption probably due concurrency and different OS managing the iSCSI target. I've searched about clustered file systems but I haven't any experience on them. I don't know if these FS could solve the issues addressed above. And a concern is that FreeBSD ported just some of these clustered FS (probably I'll use MooseFS). Well, are they stable? Work fine with iSCSI devices? Best regards!! Fábio From owner-freebsd-fs@FreeBSD.ORG Fri Jul 25 16:04:04 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6212AB20; Fri, 25 Jul 2014 16:04:04 +0000 (UTC) Received: from mail.ijs.si (mail.ijs.si [IPv6:2001:1470:ff80::25]) by mx1.freebsd.org (Postfix) with ESMTP id E16032159; Fri, 25 Jul 2014 16:04:03 +0000 (UTC) Received: from amavis-proxy-ori.ijs.si (localhost [IPv6:::1]) by mail.ijs.si (Postfix) with ESMTP id 3hKZwK37dSz1Fl; Fri, 25 Jul 2014 18:04:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ijs.si; h= user-agent:message-id:references:in-reply-to:organization :subject:subject:from:from:date:date:content-transfer-encoding :content-type:content-type:mime-version:received:received :received:received; s=jakla2; t=1406304237; x=1408896238; bh=5tU xIkHU41RaDcB2wIFMAbcYiWVzkkWlF3LWCpOJ/hY=; b=o92OJgw8PAAXl0G6p5X rHDOoT2rKFMDC+uF9/FL1NaVqKfGCbKn/3HMhDC6pyScpxgRdwXnt5vk5lw9gg7i MC4yUCrc9oUkXNwwndWr97+r4gtsgL37kIzA3BOlp1ngOJgffYC3Hvg6l8eYEUY/ SSe4+A9/V85B91FMOvCLu33Y= X-Virus-Scanned: amavisd-new at ijs.si Received: from mail.ijs.si ([IPv6:::1]) by amavis-proxy-ori.ijs.si (mail.ijs.si [IPv6:::1]) (amavisd-new, port 10012) with ESMTP id Ao8VxMVqEYjR; Fri, 25 Jul 2014 18:03:57 +0200 (CEST) Received: from mildred.ijs.si (mailbox.ijs.si [IPv6:2001:1470:ff80::143:1]) by mail.ijs.si (Postfix) with ESMTP; Fri, 25 Jul 2014 18:03:57 +0200 (CEST) Received: from neli.ijs.si (neli.ijs.si [IPv6:2001:1470:ff80:88:21c:c0ff:feb1:8c91]) by mildred.ijs.si (Postfix) with ESMTP id 3hKZwF2rQCzVN; Fri, 25 Jul 2014 18:03:57 +0200 (CEST) Received: from neli.ijs.si ([2001:1470:ff80:88:21c:c0ff:feb1:8c91]) by neli.ijs.si with HTTP (HTTP/1.1 POST); Fri, 25 Jul 2014 18:03:57 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Fri, 25 Jul 2014 18:03:57 +0200 From: Mark Martinec To: freebsd-current@freebsd.org, Freebsd fs Subject: Re: zfs send/recv: STILL invalid Backup Stream Organization: J. Stefan Institute In-Reply-To: <20140725134125.GA19293@thebighonker.lerctr.org> References: <3d2aac84d962a703fbf56a864ba5f19c@mailbox.ijs.si> <9eef3e7e964cb33fd163cc2f9300f326@mailbox.ijs.si> <53D1AB3D.1060205@freebsd.org> <33c478c3120c21d6cb9c325cc1119cf8@thebighonker.lerctr.org> <60845088153247fa31c50c9f52ef72cb@mailbox.ijs.si> <20140725134125.GA19293@thebighonker.lerctr.org> Message-ID: <2f191a3e72b9d94a15128de806f8d1c8@mailbox.ijs.si> X-Sender: Mark.Martinec+freebsd@ijs.si User-Agent: Roundcube Webmail/1.0.1 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2014 16:04:04 -0000 2014-07-25 15:41, Larry Rosenman wrote: > On Fri, Jul 25, 2014 at 11:58:48AM +0200, Mark Martinec wrote: >> Don't know, I'd guess some network-related memory limit is being hit >> on the sending site. >> >> Why not try to decouple the 'zfs send' from a network copy and ssh: >> Login to a remote side, do a 'zfs send' to some local temporary file >> there, then feed that file to ssh and send it over to a home host, >> where it can be piped into some simple program like md5 or 'wc -c' >> instead of a zfs recv. >> > I think I've done that, but it sort of defeats the purpose, Does it? If you are lucky it would fail again, but this time you'd know if it was a zfs or a network problem. > as the stream becomes a big file on disk. If it's an incremental stream and you have sufficient free space available, than it's all fine and you should try that. If storage is a problem you may try piping /dev/zero or /dev/random through ssh to feed a remote consumer, simulating a huge stream, and hope that it will eventually fail. > I'd like to get some help chasing what parameter(s) need to be fixed > here. [...] > Can I get some ideas on: > 1) what MIGHT need tweaking > 2) would (k|d)trace help? > 3) what else could we find out what's being told no memory? Sorry, not deep enough into fs or network to be able to tell. My guess it that it's a networking issue (not zfs), and not directly related to virtual memory or swap or ARC management. > To: freebsd-current@freebsd.org, Freebsd fs If you'd be able to demonstrate that this is unrelated to zfs, then the freebsd-net@freebsd.org mailing list would perhaps be a suitable place to continue this thread. Mark From owner-freebsd-fs@FreeBSD.ORG Sat Jul 26 01:49:31 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 53D5BA5 for ; Sat, 26 Jul 2014 01:49:31 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3C1D324B6 for ; Sat, 26 Jul 2014 01:49:31 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.8/8.14.8) with ESMTP id s6Q1nVWq030887 for ; Sat, 26 Jul 2014 01:49:31 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 187594] [zfs] [patch] ZFS ARC behavior problem and fix Date: Sat, 26 Jul 2014 01:49:30 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: kvedulv@kvedulv.de X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Jul 2014 01:49:31 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=187594 kvedulv@kvedulv.de changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |kvedulv@kvedulv.de --- Comment #22 from kvedulv@kvedulv.de --- any news on this? especially regarding 10.1? -- You are receiving this mail because: You are the assignee for the bug.