From owner-freebsd-fs@FreeBSD.ORG Sun Jan 19 10:50:01 2014 Return-Path: Delivered-To: freebsd-fs@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8BA6D68A for ; Sun, 19 Jan 2014 10:50:01 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5E4F11298 for ; Sun, 19 Jan 2014 10:50:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id s0JAo11o044935 for ; Sun, 19 Jan 2014 10:50:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id s0JAo1qJ044934; Sun, 19 Jan 2014 10:50:01 GMT (envelope-from gnats) Date: Sun, 19 Jan 2014 10:50:01 GMT Message-Id: <201401191050.s0JAo1qJ044934@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org Cc: From: Anton Sayetsky Subject: Re: bin/155104: [zfs][patch] use /dev prefix by default when importing X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: Anton Sayetsky List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Jan 2014 10:50:01 -0000 The following reply was made to PR bin/155104; it has been noted by GNATS. From: Anton Sayetsky To: bug-followup@freebsd.org, swell.k@gmail.com Cc: Subject: Re: bin/155104: [zfs][patch] use /dev prefix by default when importing Date: Sun, 19 Jan 2014 12:45:17 +0200 Why this PR is still open even when the commit was made a long time ago? From owner-freebsd-fs@FreeBSD.ORG Sun Jan 19 11:00:01 2014 Return-Path: Delivered-To: freebsd-fs@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 71808824 for ; Sun, 19 Jan 2014 11:00:01 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 44647132A for ; Sun, 19 Jan 2014 11:00:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id s0JB015f046951 for ; Sun, 19 Jan 2014 11:00:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id s0JB01l3046950; Sun, 19 Jan 2014 11:00:01 GMT (envelope-from gnats) Date: Sun, 19 Jan 2014 11:00:01 GMT Message-Id: <201401191100.s0JB01l3046950@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org Cc: From: Anton Sayetsky Subject: Re: kern/134491: [zfs] Hot spares are rather cold... X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: Anton Sayetsky List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Jan 2014 11:00:01 -0000 The following reply was made to PR kern/134491; it has been noted by GNATS. From: Anton Sayetsky To: bug-followup@freebsd.org, michel.bouissou@bioclinica.com Cc: Subject: Re: kern/134491: [zfs] Hot spares are rather cold... Date: Sun, 19 Jan 2014 12:52:03 +0200 What is the purpose of ZFS spares on FreeBSD if they doesn't work at all? I cannot see any sense to use spare - I can just run "replace" subcommand, right? From owner-freebsd-fs@FreeBSD.ORG Sun Jan 19 17:28:06 2014 Return-Path: Delivered-To: freebsd-fs@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DCDF53E9; Sun, 19 Jan 2014 17:28:06 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B0A1F1D48; Sun, 19 Jan 2014 17:28:06 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id s0JHS6uB043270; Sun, 19 Jan 2014 17:28:06 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id s0JHS6Yd043269; Sun, 19 Jan 2014 17:28:06 GMT (envelope-from linimon) Date: Sun, 19 Jan 2014 17:28:06 GMT Message-Id: <201401191728.s0JHS6Yd043269@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-fs@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/185858: [zfs] zvol clone can't see new device X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Jan 2014 17:28:06 -0000 Old Synopsis: zvol clone can't see new device New Synopsis: [zfs] zvol clone can't see new device Responsible-Changed-From-To: freebsd-bugs->freebsd-fs Responsible-Changed-By: linimon Responsible-Changed-When: Sun Jan 19 17:27:49 UTC 2014 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=185858 From owner-freebsd-fs@FreeBSD.ORG Mon Jan 20 10:23:36 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 90C86D4A; Mon, 20 Jan 2014 10:23:36 +0000 (UTC) Received: from mail-pa0-x22f.google.com (mail-pa0-x22f.google.com [IPv6:2607:f8b0:400e:c03::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 59EEB18BE; Mon, 20 Jan 2014 10:23:36 +0000 (UTC) Received: by mail-pa0-f47.google.com with SMTP id kp14so6850808pab.20 for ; Mon, 20 Jan 2014 02:23:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=ofluTke3s4GLaL28lC3lW5vNswbxnk7MjstR08/9STQ=; b=yb82nU9thX8j2qsCFXVGkQpJvrhbyFsxNTKVYjpG64O3bVfigUnZiHHWkMXZ74RBIp KjS1MJpDES5ndM3GnqO4YZ+Wui3DoqzVNh9tP5eyxFWbV7tPx186dMfHvSjyPYu9dRSa yrm2j5z/o5rGQexGiko33Ua8YzbAYOFbKQ7uexd5Xj8YQtWWu+Oekw7ws9xFn1HLhXLD CoGsykHAGFSIBvidW7XLsRpsCpnK6czpXiY5K1Rqqx2FozhT8ZXazHczdSuxEJMiVojF tfqoI+Ywu1gV6GUY8VbkEF2ymV0Sw/1f3YpV2/SW/X+BAXOpHN+Zkn7ZoqdshBHGdjkP 4yhA== MIME-Version: 1.0 X-Received: by 10.68.164.4 with SMTP id ym4mr17954057pbb.53.1390213415982; Mon, 20 Jan 2014 02:23:35 -0800 (PST) Received: by 10.68.0.73 with HTTP; Mon, 20 Jan 2014 02:23:35 -0800 (PST) In-Reply-To: References: <52D71421.3030103@delphij.net> <52D82178.6030903@freebsd.org> <879A72E5-3F29-4AC4-A4B4-E65430800A7C@gmail.com> Date: Mon, 20 Jan 2014 11:23:35 +0100 Message-ID: Subject: Re: zpool import taking weeks ... From: Ulysse 31 To: Freddie Cash Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Filesystems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jan 2014 10:23:36 -0000 Hi all, Finally solved this problem ^^, hope that my answer would help others have similar problems in the future: After read some more threads, have confirmed that DDT is stored only on ARC, and since ARC is only in RAM (not swappable), the import process on the live cd was starting slowly filling ram and pushing userland to swap ... The result was that the system still running without out of swap message space but zfs simply "stalls" without having enought ram to continue import ... I then remember that i had 2 pre-prod servers with 64Gb RAM on each, scavenged one and added 64Gb on the zfs server (which ended so with 128Gb) Booted the server normally (native Freebsd) and the import run then correctly within approximately 12h, taking 90Gb for ARC (not checked but suppose that it was mainly DDT). Once booted, destroyed the remotely sync'ed dataset that caused crash, that took another 30Gb on RAM, loading used mem to 120Gb and something like 3.5Gb free during about a hour ^^' (finger crossed). Once dataset was destroy, lot of RAM was free'ed . Rebooted the server and now it boots normally without wait on import. Thanks all for the help. Cheers, 2014/1/16 Freddie Cash : > On Thu, Jan 16, 2014 at 11:18 AM, Gomes do Vale Victor > wrote: >> >> >> Le 16 janv. 2014 =E0 19:14, Stefan Esser a =E9crit : >> >> > Am 16.01.2014 08:00, schrieb Gomes do Vale Victor: >> >>> Thanks a lot for this info ! I have read about the "-T" option >> >>> on import with tgx but it was with ZFS on Linux ... Don't knew it >> >>> is available on newer implementation under FreeBSD. Will so wait >> >>> that it ends correctly and will think on upgrading on the system >> >>> (the rootfs is ZFS also ...) >> > >> > Just for the record, since your reply seems to imply that the -T >> > option is only available on Linux: >> > >> > The -T option of zpool import has been present in FreeBSD for years, >> > but is undocumented (same as in Solaris/OpenIndiana). It has been >> > mentioned in some ZFS recovery guides (most written for Solaris) and >> > it actually works quite well (I used it to recover an inconsistent poo= l >> > and helped somebody else who has lost access to his ZFS file-systems). >> > >> > Regards, STefan >> >> Sorry if you understood that. It was not what I meant. I wanted to say >> that I only found one thread talking about that option (the import with = the >> -T option) and that thread was about ZFS on Linux, and since I ain't fou= nd >> no mention of it on the actual man of my FreeBSD version I have here, I >> tough it was only available on newer ZFS versions (like v5000) under >> FreeBSD. >> Thanks for the info, so do you think I could use this method to recover = my >> pool that is on ZFS v28 ? >> Also, the import is now running without crash for 8 days now, and still >> does not have finished, and I do not see any revelant activity on disks,= nor >> any load under CTRL+T on the zpool process (sometimes goes to 0.16/0.08 = but >> quickly returns to 0.00 and stays there a while) do you think I should >> interrupt it, reboot and restart again ? Or directly test with the -T op= tion >> ? > > > I'd try a reboot to restart the process. Let it run for a day. If you > still aren't seeing any disk activity, then try the -T and/or the -F impo= rt > options. > > -- > Freddie Cash > fjwcash@gmail.com --=20 Ulysse31 From owner-freebsd-fs@FreeBSD.ORG Mon Jan 20 11:06:45 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 406CAA03 for ; Mon, 20 Jan 2014 11:06:45 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 209661D60 for ; Mon, 20 Jan 2014 11:06:45 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id s0KB6jl4088320 for ; Mon, 20 Jan 2014 11:06:45 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id s0KB6i4Q088318 for freebsd-fs@FreeBSD.org; Mon, 20 Jan 2014 11:06:44 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 20 Jan 2014 11:06:44 GMT Message-Id: <201401201106.s0KB6i4Q088318@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-fs@FreeBSD.org Subject: Current problem reports assigned to freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jan 2014 11:06:45 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/185858 fs [zfs] zvol clone can't see new device o kern/184478 fs [smbfs] mount_smbfs cannot read/write files o kern/182536 fs [zfs] zfs deadlock o kern/181966 fs [zfs] Kernel panic in ZFS I/O: solaris assert: BP_EQUA o kern/181834 fs [nfs] amd mounting NFS directories can drive a dead-lo o kern/181565 fs [swap] Problem with vnode-backed swap space. o kern/181377 fs [zfs] zfs recv causes an inconsistant pool o kern/181281 fs [msdosfs] stack trace after successfull 'umount /mnt' o kern/181082 fs [fuse] [ntfs] Write to mounted NTFS filesystem using F o kern/180979 fs [netsmb][patch]: Fix large files handling o kern/180876 fs [zfs] [hast] ZFS with trim,bio_flush or bio_delete loc o kern/180678 fs [NFS] succesfully exported filesystems being reported o kern/180438 fs [smbfs] [patch] mount_smbfs fails on arm because of wr p kern/180236 fs [zfs] [nullfs] Leakage free space using ZFS with nullf o kern/178854 fs [ufs] FreeBSD kernel crash in UFS s kern/178467 fs [zfs] [request] Optimized Checksum Code for ZFS o kern/178412 fs [smbfs] Coredump when smbfs mounted o kern/178388 fs [zfs] [patch] allow up to 8MB recordsize o kern/178387 fs [zfs] [patch] sparse files performance improvements o kern/178349 fs [zfs] zfs scrub on deduped data could be much less see o kern/178329 fs [zfs] extended attributes leak o kern/178238 fs [nullfs] nullfs don't release i-nodes on unlink. f kern/178231 fs [nfs] 8.3 nfsv4 client reports "nfsv4 client/server pr o kern/177985 fs [zfs] disk usage problem when copying from one zfs dat o kern/177971 fs [nfs] FreeBSD 9.1 nfs client dirlist problem w/ nfsv3, o kern/177966 fs [zfs] resilver completes but subsequent scrub reports o kern/177658 fs [ufs] FreeBSD panics after get full filesystem with uf o kern/177536 fs [zfs] zfs livelock (deadlock) with high write-to-disk o kern/177445 fs [hast] HAST panic o kern/177240 fs [zfs] zpool import failed with state UNAVAIL but all d o kern/176978 fs [zfs] [panic] zfs send -D causes "panic: System call i o kern/176857 fs [softupdates] [panic] 9.1-RELEASE/amd64/GENERIC panic o bin/176253 fs zpool(8): zfs pool indentation is misleading/wrong o kern/176141 fs [zfs] sharesmb=on makes errors for sharenfs, and still o kern/175950 fs [zfs] Possible deadlock in zfs after long uptime o kern/175897 fs [zfs] operations on readonly zpool hang o kern/175449 fs [unionfs] unionfs and devfs misbehaviour o kern/175179 fs [zfs] ZFS may attach wrong device on move o kern/175071 fs [ufs] [panic] softdep_deallocate_dependencies: unrecov o kern/174372 fs [zfs] Pagefault appears to be related to ZFS o kern/174315 fs [zfs] chflags uchg not supported o kern/174310 fs [zfs] root point mounting broken on CURRENT with multi o kern/174279 fs [ufs] UFS2-SU+J journal and filesystem corruption o kern/173830 fs [zfs] Brain-dead simple change to ZFS error descriptio o kern/173718 fs [zfs] phantom directory in zraid2 pool f kern/173657 fs [nfs] strange UID map with nfsuserd o kern/173363 fs [zfs] [panic] Panic on 'zpool replace' on readonly poo o kern/173136 fs [unionfs] mounting above the NFS read-only share panic o kern/172942 fs [smbfs] Unmounting a smb mount when the server became o kern/172348 fs [unionfs] umount -f of filesystem in use with readonly o kern/172334 fs [unionfs] unionfs permits recursive union mounts; caus o kern/171626 fs [tmpfs] tmpfs should be noisier when the requested siz o kern/171415 fs [zfs] zfs recv fails with "cannot receive incremental o kern/170945 fs [gpt] disk layout not portable between direct connect o bin/170778 fs [zfs] [panic] FreeBSD panics randomly o kern/170680 fs [nfs] Multiple NFS Client bug in the FreeBSD 7.4-RELEA o kern/170497 fs [xfs][panic] kernel will panic whenever I ls a mounted o kern/169945 fs [zfs] [panic] Kernel panic while importing zpool (afte o kern/169480 fs [zfs] ZFS stalls on heavy I/O o kern/169398 fs [zfs] Can't remove file with permanent error o kern/169339 fs panic while " : > /etc/123" o kern/169319 fs [zfs] zfs resilver can't complete o kern/168947 fs [nfs] [zfs] .zfs/snapshot directory is messed up when o kern/168942 fs [nfs] [hang] nfsd hangs after being restarted (not -HU o kern/168158 fs [zfs] incorrect parsing of sharenfs options in zfs (fs o kern/167979 fs [ufs] DIOCGDINFO ioctl does not work on 8.2 file syste o kern/167977 fs [smbfs] mount_smbfs results are differ when utf-8 or U o kern/167688 fs [fusefs] Incorrect signal handling with direct_io o kern/167685 fs [zfs] ZFS on USB drive prevents shutdown / reboot o kern/167612 fs [portalfs] The portal file system gets stuck inside po o kern/167272 fs [zfs] ZFS Disks reordering causes ZFS to pick the wron o kern/167260 fs [msdosfs] msdosfs disk was mounted the second time whe o kern/167109 fs [zfs] [panic] zfs diff kernel panic Fatal trap 9: gene o kern/167105 fs [nfs] mount_nfs can not handle source exports wiht mor o kern/167067 fs [zfs] [panic] ZFS panics the server o kern/167065 fs [zfs] boot fails when a spare is the boot disk o kern/167048 fs [nfs] [patch] RELEASE-9 crash when using ZFS+NULLFS+NF o kern/166912 fs [ufs] [panic] Panic after converting Softupdates to jo o kern/166851 fs [zfs] [hang] Copying directory from the mounted UFS di o kern/166477 fs [nfs] NFS data corruption. o kern/165950 fs [ffs] SU+J and fsck problem o kern/165521 fs [zfs] [hang] livelock on 1 Gig of RAM with zfs when 31 o kern/165392 fs Multiple mkdir/rmdir fails with errno 31 o kern/165087 fs [unionfs] lock violation in unionfs o kern/164472 fs [ufs] fsck -B panics on particular data inconsistency o kern/164370 fs [zfs] zfs destroy for snapshot fails on i386 and sparc o kern/164261 fs [nullfs] [patch] fix panic with NFS served from NULLFS o kern/164256 fs [zfs] device entry for volume is not created after zfs o kern/164184 fs [ufs] [panic] Kernel panic with ufs_makeinode o kern/163801 fs [md] [request] allow mfsBSD legacy installed in 'swap' o kern/163770 fs [zfs] [hang] LOR between zfs&syncer + vnlru leading to o kern/163501 fs [nfs] NFS exporting a dir and a subdir in that dir to o kern/162944 fs [coda] Coda file system module looks broken in 9.0 o kern/162860 fs [zfs] Cannot share ZFS filesystem to hosts with a hyph o kern/162751 fs [zfs] [panic] kernel panics during file operations o kern/162591 fs [nullfs] cross-filesystem nullfs does not work as expe o kern/162519 fs [zfs] "zpool import" relies on buggy realpath() behavi o kern/161968 fs [zfs] [hang] renaming snapshot with -r including a zvo o kern/161864 fs [ufs] removing journaling from UFS partition fails on o kern/161579 fs [smbfs] FreeBSD sometimes panics when an smb share is o kern/161533 fs [zfs] [panic] zfs receive panic: system ioctl returnin o kern/161438 fs [zfs] [panic] recursed on non-recursive spa_namespace_ o kern/161424 fs [nullfs] __getcwd() calls fail when used on nullfs mou o kern/161280 fs [zfs] Stack overflow in gptzfsboot o kern/161205 fs [nfs] [pfsync] [regression] [build] Bug report freebsd o kern/161169 fs [zfs] [panic] ZFS causes kernel panic in dbuf_dirty o kern/161112 fs [ufs] [lor] filesystem LOR in FreeBSD 9.0-BETA3 o kern/160893 fs [zfs] [panic] 9.0-BETA2 kernel panic f kern/160860 fs [ufs] Random UFS root filesystem corruption with SU+J o kern/160801 fs [zfs] zfsboot on 8.2-RELEASE fails to boot from root-o o kern/160790 fs [fusefs] [panic] VPUTX: negative ref count with FUSE o kern/160777 fs [zfs] [hang] RAID-Z3 causes fatal hang upon scrub/impo o kern/160706 fs [zfs] zfs bootloader fails when a non-root vdev exists o kern/160591 fs [zfs] Fail to boot on zfs root with degraded raidz2 [r o kern/160410 fs [smbfs] [hang] smbfs hangs when transferring large fil o kern/160283 fs [zfs] [patch] 'zfs list' does abort in make_dataset_ha o kern/159930 fs [ufs] [panic] kernel core o kern/159402 fs [zfs][loader] symlinks cause I/O errors o kern/159357 fs [zfs] ZFS MAXNAMELEN macro has confusing name (off-by- o kern/159356 fs [zfs] [patch] ZFS NAME_ERR_DISKLIKE check is Solaris-s o kern/159351 fs [nfs] [patch] - divide by zero in mountnfs() o kern/159251 fs [zfs] [request]: add FLETCHER4 as DEDUP hash option o kern/159077 fs [zfs] Can't cd .. with latest zfs version o kern/159048 fs [smbfs] smb mount corrupts large files o kern/159045 fs [zfs] [hang] ZFS scrub freezes system o kern/158839 fs [zfs] ZFS Bootloader Fails if there is a Dead Disk o kern/158802 fs amd(8) ICMP storm and unkillable process. o kern/158231 fs [nullfs] panic on unmounting nullfs mounted over ufs o f kern/157929 fs [nfs] NFS slow read o kern/157399 fs [zfs] trouble with: mdconfig force delete && zfs strip o kern/157179 fs [zfs] zfs/dbuf.c: panic: solaris assert: arc_buf_remov o kern/156797 fs [zfs] [panic] Double panic with FreeBSD 9-CURRENT and o kern/156781 fs [zfs] zfs is losing the snapshot directory, p kern/156545 fs [ufs] mv could break UFS on SMP systems o kern/156193 fs [ufs] [hang] UFS snapshot hangs && deadlocks processes o kern/156039 fs [nullfs] [unionfs] nullfs + unionfs do not compose, re o kern/155615 fs [zfs] zfs v28 broken on sparc64 -current o kern/155587 fs [zfs] [panic] kernel panic with zfs p kern/155411 fs [regression] [8.2-release] [tmpfs]: mount: tmpfs : No o kern/155199 fs [ext2fs] ext3fs mounted as ext2fs gives I/O errors o bin/155104 fs [zfs][patch] use /dev prefix by default when importing o kern/154930 fs [zfs] cannot delete/unlink file from full volume -> EN o kern/154828 fs [msdosfs] Unable to create directories on external USB o kern/154491 fs [smbfs] smb_co_lock: recursive lock for object 1 p kern/154228 fs [md] md getting stuck in wdrain state o kern/153996 fs [zfs] zfs root mount error while kernel is not located o kern/153753 fs [zfs] ZFS v15 - grammatical error when attempting to u o kern/153716 fs [zfs] zpool scrub time remaining is incorrect o kern/153695 fs [patch] [zfs] Booting from zpool created on 4k-sector o kern/153680 fs [xfs] 8.1 failing to mount XFS partitions o kern/153418 fs [zfs] [panic] Kernel Panic occurred writing to zfs vol o kern/153351 fs [zfs] locking directories/files in ZFS o bin/153258 fs [patch][zfs] creating ZVOLs requires `refreservation' s kern/153173 fs [zfs] booting from a gzip-compressed dataset doesn't w o bin/153142 fs [zfs] ls -l outputs `ls: ./.zfs: Operation not support o kern/153126 fs [zfs] vdev failure, zpool=peegel type=vdev.too_small o kern/152022 fs [nfs] nfs service hangs with linux client [regression] o kern/151942 fs [zfs] panic during ls(1) zfs snapshot directory o kern/151905 fs [zfs] page fault under load in /sbin/zfs o bin/151713 fs [patch] Bug in growfs(8) with respect to 32-bit overfl o kern/151648 fs [zfs] disk wait bug o kern/151629 fs [fs] [patch] Skip empty directory entries during name o kern/151330 fs [zfs] will unshare all zfs filesystem after execute a o kern/151326 fs [nfs] nfs exports fail if netgroups contain duplicate o kern/151251 fs [ufs] Can not create files on filesystem with heavy us o kern/151226 fs [zfs] can't delete zfs snapshot o kern/150503 fs [zfs] ZFS disks are UNAVAIL and corrupted after reboot o kern/150501 fs [zfs] ZFS vdev failure vdev.bad_label on amd64 o kern/150390 fs [zfs] zfs deadlock when arcmsr reports drive faulted o kern/150336 fs [nfs] mountd/nfsd became confused; refused to reload n o kern/149208 fs mksnap_ffs(8) hang/deadlock o kern/149173 fs [patch] [zfs] make OpenSolaris installa o kern/149015 fs [zfs] [patch] misc fixes for ZFS code to build on Glib o kern/149014 fs [zfs] [patch] declarations in ZFS libraries/utilities o kern/149013 fs [zfs] [patch] make ZFS makefiles use the libraries fro o kern/148504 fs [zfs] ZFS' zpool does not allow replacing drives to be o kern/148490 fs [zfs]: zpool attach - resilver bidirectionally, and re o kern/148368 fs [zfs] ZFS hanging forever on 8.1-PRERELEASE o kern/148138 fs [zfs] zfs raidz pool commands freeze o kern/147903 fs [zfs] [panic] Kernel panics on faulty zfs device o kern/147881 fs [zfs] [patch] ZFS "sharenfs" doesn't allow different " o kern/147420 fs [ufs] [panic] ufs_dirbad, nullfs, jail panic (corrupt o kern/146941 fs [zfs] [panic] Kernel Double Fault - Happens constantly o kern/146786 fs [zfs] zpool import hangs with checksum errors o kern/146708 fs [ufs] [panic] Kernel panic in softdep_disk_write_compl o kern/146528 fs [zfs] Severe memory leak in ZFS on i386 o kern/146502 fs [nfs] FreeBSD 8 NFS Client Connection to Server o kern/145750 fs [unionfs] [hang] unionfs locks the machine s kern/145712 fs [zfs] cannot offline two drives in a raidz2 configurat o kern/145411 fs [xfs] [panic] Kernel panics shortly after mounting an f bin/145309 fs bsdlabel: Editing disk label invalidates the whole dev o kern/145272 fs [zfs] [panic] Panic during boot when accessing zfs on o kern/145246 fs [ufs] dirhash in 7.3 gratuitously frees hashes when it o kern/145238 fs [zfs] [panic] kernel panic on zpool clear tank o kern/145229 fs [zfs] Vast differences in ZFS ARC behavior between 8.0 o kern/145189 fs [nfs] nfsd performs abysmally under load o kern/144929 fs [ufs] [lor] vfs_bio.c + ufs_dirhash.c p kern/144447 fs [zfs] sharenfs fsunshare() & fsshare_main() non functi o kern/144416 fs [panic] Kernel panic on online filesystem optimization s kern/144415 fs [zfs] [panic] kernel panics on boot after zfs crash o kern/144234 fs [zfs] Cannot boot machine with recent gptzfsboot code o kern/143825 fs [nfs] [panic] Kernel panic on NFS client o bin/143572 fs [zfs] zpool(1): [patch] The verbose output from iostat o kern/143212 fs [nfs] NFSv4 client strange work ... o kern/143184 fs [zfs] [lor] zfs/bufwait LOR o kern/142878 fs [zfs] [vfs] lock order reversal o kern/142489 fs [zfs] [lor] allproc/zfs LOR o kern/142466 fs Update 7.2 -> 8.0 on Raid 1 ends with screwed raid [re o kern/142306 fs [zfs] [panic] ZFS drive (from OSX Leopard) causes two o kern/142068 fs [ufs] BSD labels are got deleted spontaneously o kern/141950 fs [unionfs] [lor] ufs/unionfs/ufs Lock order reversal o kern/141897 fs [msdosfs] [panic] Kernel panic. msdofs: file name leng o kern/141463 fs [nfs] [panic] Frequent kernel panics after upgrade fro o kern/141091 fs [patch] [nullfs] fix panics with DIAGNOSTIC enabled o kern/141086 fs [nfs] [panic] panic("nfs: bioread, not dir") on FreeBS o kern/141010 fs [zfs] "zfs scrub" fails when backed by files in UFS2 o kern/140888 fs [zfs] boot fail from zfs root while the pool resilveri o kern/140661 fs [zfs] [patch] /boot/loader fails to work on a GPT/ZFS- o kern/140640 fs [zfs] snapshot crash o kern/140068 fs [smbfs] [patch] smbfs does not allow semicolon in file o kern/139725 fs [zfs] zdb(1) dumps core on i386 when examining zpool c o kern/139715 fs [zfs] vfs.numvnodes leak on busy zfs p bin/139651 fs [nfs] mount(8): read-only remount of NFS volume does n o kern/139407 fs [smbfs] [panic] smb mount causes system crash if remot o kern/138662 fs [panic] ffs_blkfree: freeing free block o kern/138421 fs [ufs] [patch] remove UFS label limitations o kern/138202 fs mount_msdosfs(1) see only 2Gb o kern/137588 fs [unionfs] [lor] LOR nfs/ufs/nfs o kern/136968 fs [ufs] [lor] ufs/bufwait/ufs (open) o kern/136945 fs [ufs] [lor] filedesc structure/ufs (poll) o kern/136944 fs [ffs] [lor] bufwait/snaplk (fsync) o kern/136873 fs [ntfs] Missing directories/files on NTFS volume p kern/136470 fs [nfs] Cannot mount / in read-only, over NFS o kern/135546 fs [zfs] zfs.ko module doesn't ignore zpool.cache filenam o kern/135469 fs [ufs] [panic] kernel crash on md operation in ufs_dirb o kern/135050 fs [zfs] ZFS clears/hides disk errors on reboot o kern/134491 fs [zfs] Hot spares are rather cold... o kern/133676 fs [smbfs] [panic] umount -f'ing a vnode-based memory dis p kern/133174 fs [msdosfs] [patch] msdosfs must support multibyte inter o kern/132960 fs [ufs] [panic] panic:ffs_blkfree: freeing free frag o kern/132397 fs reboot causes filesystem corruption (failure to sync b o kern/132331 fs [ufs] [lor] LOR ufs and syncer o kern/132237 fs [msdosfs] msdosfs has problems to read MSDOS Floppy o kern/132145 fs [panic] File System Hard Crashes o kern/131441 fs [unionfs] [nullfs] unionfs and/or nullfs not combineab o kern/131360 fs [nfs] poor scaling behavior of the NFS server under lo o kern/131342 fs [nfs] mounting/unmounting of disks causes NFS to fail o bin/131341 fs makefs: error "Bad file descriptor" on the mount poin o kern/130920 fs [msdosfs] cp(1) takes 100% CPU time while copying file o kern/130210 fs [nullfs] Error by check nullfs o kern/129760 fs [nfs] after 'umount -f' of a stale NFS share FreeBSD l o kern/129488 fs [smbfs] Kernel "bug" when using smbfs in smbfs_smb.c: o kern/129231 fs [ufs] [patch] New UFS mount (norandom) option - mostly o kern/129152 fs [panic] non-userfriendly panic when trying to mount(8) o kern/127787 fs [lor] [ufs] Three LORs: vfslock/devfs/vfslock, ufs/vfs o bin/127270 fs fsck_msdosfs(8) may crash if BytesPerSec is zero o kern/127029 fs [panic] mount(8): trying to mount a write protected zi o kern/126973 fs [unionfs] [hang] System hang with unionfs and init chr o kern/126553 fs [unionfs] unionfs move directory problem 2 (files appe o kern/126287 fs [ufs] [panic] Kernel panics while mounting an UFS file o kern/125895 fs [ffs] [panic] kernel: panic: ffs_blkfree: freeing free s kern/125738 fs [zfs] [request] SHA256 acceleration in ZFS o kern/123939 fs [msdosfs] corrupts new files o bin/123574 fs [unionfs] df(1) -t option destroys info for unionfs (a o kern/122380 fs [ffs] ffs_valloc:dup alloc (Soekris 4801/7.0/USB Flash o bin/122172 fs [fs]: amd(8) automount daemon dies on 6.3-STABLE i386, o bin/121898 fs [nullfs] pwd(1)/getcwd(2) fails with Permission denied o kern/121385 fs [unionfs] unionfs cross mount -> kernel panic o bin/121072 fs [smbfs] mount_smbfs(8) cannot normally convert the cha o kern/120483 fs [ntfs] [patch] NTFS filesystem locking changes o kern/120482 fs [ntfs] [patch] Sync style changes between NetBSD and F o kern/118912 fs [2tb] disk sizing/geometry problem with large array o kern/118713 fs [minidump] [patch] Display media size required for a k o kern/118318 fs [nfs] NFS server hangs under special circumstances o bin/118249 fs [ufs] mv(1): moving a directory changes its mtime o kern/118126 fs [nfs] [patch] Poor NFS server write performance o kern/118107 fs [ntfs] [panic] Kernel panic when accessing a file at N o kern/117954 fs [ufs] dirhash on very large directories blocks the mac o bin/117315 fs [smbfs] mount_smbfs(8) and related options can't mount o kern/117158 fs [zfs] zpool scrub causes panic if geli vdevs detach on o bin/116980 fs [msdosfs] [patch] mount_msdosfs(8) resets some flags f o conf/116931 fs lack of fsck_cd9660 prevents mounting iso images with o kern/116583 fs [ffs] [hang] System freezes for short time when using o bin/115361 fs [zfs] mount(8) gets into a state where it won't set/un o kern/114955 fs [cd9660] [patch] [request] support for mask,dirmask,ui o kern/114847 fs [ntfs] [patch] [request] dirmask support for NTFS ala o kern/114676 fs [ufs] snapshot creation panics: snapacct_ufs2: bad blo o bin/114468 fs [patch] [request] add -d option to umount(8) to detach o kern/113852 fs [smbfs] smbfs does not properly implement DFS referral o bin/113838 fs [patch] [request] mount(8): add support for relative p o bin/113049 fs [patch] [request] make quot(8) use getopt(3) and show o kern/112658 fs [smbfs] [patch] smbfs and caching problems (resolves b o kern/111843 fs [msdosfs] Long Names of files are incorrectly created o kern/111782 fs [ufs] dump(8) fails horribly for large filesystems s bin/111146 fs [2tb] fsck(8) fails on 6T filesystem o bin/107829 fs [2TB] fdisk(8): invalid boundary checking in fdisk / w o kern/106107 fs [ufs] left-over fsck_snapshot after unfinished backgro o kern/104406 fs [ufs] Processes get stuck in "ufs" state under persist o kern/103035 fs [ntfs] Directories in NTFS mounted disc images appear o kern/101324 fs [smbfs] smbfs sometimes not case sensitive when it's s o kern/99290 fs [ntfs] mount_ntfs ignorant of cluster sizes s bin/97498 fs [request] newfs(8) has no option to clear the first 12 o kern/97377 fs [ntfs] [patch] syntax cleanup for ntfs_ihash.c o kern/95222 fs [cd9660] File sections on ISO9660 level 3 CDs ignored o kern/94849 fs [ufs] rename on UFS filesystem is not atomic o bin/94810 fs fsck(8) incorrectly reports 'file system marked clean' o kern/94769 fs [ufs] Multiple file deletions on multi-snapshotted fil o kern/94733 fs [smbfs] smbfs may cause double unlock o kern/93942 fs [vfs] [patch] panic: ufs_dirbad: bad dir (patch from D o kern/92272 fs [ffs] [hang] Filling a filesystem while creating a sna o kern/91134 fs [smbfs] [patch] Preserve access and modification time a kern/90815 fs [smbfs] [patch] SMBFS with character conversions somet o kern/88657 fs [smbfs] windows client hang when browsing a samba shar o kern/88555 fs [panic] ffs_blkfree: freeing free frag on AMD 64 o bin/87966 fs [patch] newfs(8): introduce -A flag for newfs to enabl o kern/87859 fs [smbfs] System reboot while umount smbfs. o kern/86587 fs [msdosfs] rm -r /PATH fails with lots of small files o bin/85494 fs fsck_ffs: unchecked use of cg_inosused macro etc. o kern/80088 fs [smbfs] Incorrect file time setting on NTFS mounted vi o bin/74779 fs Background-fsck checks one filesystem twice and omits o kern/73484 fs [ntfs] Kernel panic when doing `ls` from the client si o bin/73019 fs [ufs] fsck_ufs(8) cannot alloc 607016868 bytes for ino o kern/71774 fs [ntfs] NTFS cannot "see" files on a WinXP filesystem o bin/70600 fs fsck(8) throws files away when it can't grow lost+foun o kern/68978 fs [panic] [ufs] crashes with failing hard disk, loose po o kern/67326 fs [msdosfs] crash after attempt to mount write protected o kern/65920 fs [nwfs] Mounted Netware filesystem behaves strange o kern/65901 fs [smbfs] [patch] smbfs fails fsx write/truncate-down/tr o kern/61503 fs [smbfs] mount_smbfs does not work as non-root o kern/55617 fs [smbfs] Accessing an nsmb-mounted drive via a smb expo o kern/51685 fs [hang] Unbounded inode allocation causes kernel to loc o kern/36566 fs [smbfs] System reboot with dead smb mount and umount o bin/27687 fs fsck(8) wrapper is not properly passing options to fsc o kern/18874 fs [2TB] 32bit NFS servers export wrong negative values t o kern/9619 fs [nfs] Restarting mountd kills existing mounts 335 problems total. From owner-freebsd-fs@FreeBSD.ORG Mon Jan 20 15:30:38 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A4DC07E; Mon, 20 Jan 2014 15:30:38 +0000 (UTC) Received: from pi.nmdps.net (pi.nmdps.net [109.61.102.5]) by mx1.freebsd.org (Postfix) with ESMTP id 0B4851A84; Mon, 20 Jan 2014 15:30:37 +0000 (UTC) Received: from pi.nmdps.net (localhost [127.0.0.1]) (Authenticated sender: krichy@cflinux.hu) by pi.nmdps.net (Postfix) with ESMTPSA id C467611EF; Mon, 20 Jan 2014 16:30:36 +0100 (CET) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=_e29d732e9be4cf639842cb9d8de74c22" Date: Mon, 20 Jan 2014 16:30:34 +0100 From: krichy@cflinux.hu To: Richard Kojedzinszky Subject: Re: ZFS .zfs DoS In-Reply-To: References: Message-ID: X-Sender: krichy@cflinux.hu User-Agent: Roundcube Webmail/0.9.5 Cc: freebsd-fs@freebsd.org, freebsd-security@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jan 2014 15:30:38 -0000 --=_e29d732e9be4cf639842cb9d8de74c22 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8; format=flowed Dear users, I've worked out a patch for my known issues, please somebody test them, and give recommendations, fixes. Regards, 2014-01-17 03:11 idÅ‘pontban Richard Kojedzinszky ezt írta: > Dear users, > > For a long time now I've been investigating problems relating FreeBSD > ZFS .zfs handling, and found that I am not enough to fix issues. Until > fixes arrive, unfortunately a regular user can DoS a FreeBSD system > which has ZFS filesystems with the attached script. While the script > expects a snapshot argument to be given, actually the first test case > does not need that, only a mounted zfs filesystem is enough. For more > of the tests a snapshot may be needed, and later ones need root > account also. > > I would recommend that until this gets rewritten or fixed at all, one > should disable access to .zfs at all with someting like I've attached. > > Regards, > Kojedzinszky Richard --=_e29d732e9be4cf639842cb9d8de74c22 Content-Transfer-Encoding: base64 Content-Type: text/x-diff; name=gfs-4.patch Content-Disposition: attachment; filename=gfs-4.patch; size=11842 Y29tbWl0IGY1NmQ2NTk2Yjc5YzliYTc2ODUxZWU2YmVhMjI1ZjIyY2M5ZjBhMjYKQXV0aG9yOiBS aWNoYXJkIEtvamVkemluc3preSA8a3JpY2h5QGNmbGludXguaHU+CkRhdGU6ICAgRnJpIEphbiAx NyAyMjo1NzozMyAyMDE0ICswMTAwCgogICAgWkZTL0dGUyBoYW5kbGluZyBmaXhlcwoKZGlmZiAt LWdpdCBhL3N5cy9jZGRsL2NvbXBhdC9vcGVuc29sYXJpcy9rZXJuL29wZW5zb2xhcmlzX2xvb2t1 cC5jIGIvc3lzL2NkZGwvY29tcGF0L29wZW5zb2xhcmlzL2tlcm4vb3BlbnNvbGFyaXNfbG9va3Vw LmMKaW5kZXggOTQzODNkNi4uNGNhYzA1MyAxMDA2NDQKLS0tIGEvc3lzL2NkZGwvY29tcGF0L29w ZW5zb2xhcmlzL2tlcm4vb3BlbnNvbGFyaXNfbG9va3VwLmMKKysrIGIvc3lzL2NkZGwvY29tcGF0 L29wZW5zb2xhcmlzL2tlcm4vb3BlbnNvbGFyaXNfbG9va3VwLmMKQEAgLTgxLDYgKzgxLDggQEAg dHJhdmVyc2Uodm5vZGVfdCAqKmN2cHAsIGludCBsa3R5cGUpCiAJICogcHJvZ3Jlc3Mgb24gdGhp cyB2bm9kZS4KIAkgKi8KIAorCXZuX2xvY2soY3ZwLCBsa3R5cGUpOworCiAJZm9yICg7Oykgewog CQkvKgogCQkgKiBSZWFjaGVkIHRoZSBlbmQgb2YgdGhlIG1vdW50IGNoYWluPwpAQCAtODksMTMg KzkxLDcgQEAgdHJhdmVyc2Uodm5vZGVfdCAqKmN2cHAsIGludCBsa3R5cGUpCiAJCWlmICh2ZnNw ID09IE5VTEwpCiAJCQlicmVhazsKIAkJZXJyb3IgPSB2ZnNfYnVzeSh2ZnNwLCAwKTsKLQkJLyoK LQkJICogdHZwIGlzIE5VTEwgZm9yICpjdnBwIHZub2RlLCB3aGljaCB3ZSBjYW4ndCB1bmxvY2su Ci0JCSAqLwotCQlpZiAodHZwICE9IE5VTEwpCi0JCQl2cHV0KGN2cCk7Ci0JCWVsc2UKLQkJCXZy ZWxlKGN2cCk7CisJCVZPUF9VTkxPQ0soY3ZwLCAwKTsKIAkJaWYgKGVycm9yKQogCQkJcmV0dXJu IChlcnJvcik7CiAKQEAgLTEwNyw2ICsxMDMsOSBAQCB0cmF2ZXJzZSh2bm9kZV90ICoqY3ZwcCwg aW50IGxrdHlwZSkKIAkJdmZzX3VuYnVzeSh2ZnNwKTsKIAkJaWYgKGVycm9yICE9IDApCiAJCQly ZXR1cm4gKGVycm9yKTsKKworCQlWTl9SRUxFKGN2cCk7CisKIAkJY3ZwID0gdHZwOwogCX0KIApk aWZmIC0tZ2l0IGEvc3lzL2NkZGwvY29tcGF0L29wZW5zb2xhcmlzL2tlcm4vb3BlbnNvbGFyaXNf dmZzLmMgYi9zeXMvY2RkbC9jb21wYXQvb3BlbnNvbGFyaXMva2Vybi9vcGVuc29sYXJpc192ZnMu YwppbmRleCBhMjUzMmY4Li5jMzAyYTU0IDEwMDY0NAotLS0gYS9zeXMvY2RkbC9jb21wYXQvb3Bl bnNvbGFyaXMva2Vybi9vcGVuc29sYXJpc192ZnMuYworKysgYi9zeXMvY2RkbC9jb21wYXQvb3Bl bnNvbGFyaXMva2Vybi9vcGVuc29sYXJpc192ZnMuYwpAQCAtMTk0LDEwICsxOTQsOCBAQCBtb3Vu dF9zbmFwc2hvdChrdGhyZWFkX3QgKnRkLCB2bm9kZV90ICoqdnBwLCBjb25zdCBjaGFyICpmc3R5 cGUsIGNoYXIgKmZzcGF0aCwKIAkJVklfTE9DSyh2cCk7CiAJCXZwLT52X2lmbGFnICY9IH5WSV9N T1VOVDsKIAkJVklfVU5MT0NLKHZwKTsKLQkJdnJlbGUodnApOwogCQl2ZnNfdW5idXN5KG1wKTsK IAkJdmZzX21vdW50X2Rlc3Ryb3kobXApOwotCQkqdnBwID0gTlVMTDsKIAkJcmV0dXJuIChlcnJv cik7CiAJfQogCkBAIC0yMjgsNyArMjI2LDcgQEAgbW91bnRfc25hcHNob3Qoa3RocmVhZF90ICp0 ZCwgdm5vZGVfdCAqKnZwcCwgY29uc3QgY2hhciAqZnN0eXBlLCBjaGFyICpmc3BhdGgsCiAJdmZz X2V2ZW50X3NpZ25hbChOVUxMLCBWUV9NT1VOVCwgMCk7CiAJaWYgKFZGU19ST09UKG1wLCBMS19F WENMVVNJVkUsICZtdnApKQogCQlwYW5pYygibW91bnQ6IGxvc3QgbW91bnQiKTsKLQl2cHV0KHZw KTsKKwlWT1BfVU5MT0NLKHZwLCAwKTsKIAl2ZnNfdW5idXN5KG1wKTsKIAkqdnBwID0gbXZwOwog CXJldHVybiAoMCk7CmRpZmYgLS1naXQgYS9zeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzL3V0 cy9jb21tb24vZnMvZ2ZzLmMgYi9zeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzL3V0cy9jb21t b24vZnMvZ2ZzLmMKaW5kZXggNTk5NDRhMS4uMjllYzQ1NCAxMDA2NDQKLS0tIGEvc3lzL2NkZGwv Y29udHJpYi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL2dmcy5jCisrKyBiL3N5cy9jZGRsL2Nv bnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy9nZnMuYwpAQCAtNDQ4LDcgKzQ0OCw2IEBA IGdmc19sb29rdXBfZG90KHZub2RlX3QgKip2cHAsIHZub2RlX3QgKmR2cCwgdm5vZGVfdCAqcHZw LCBjb25zdCBjaGFyICpubSkKIAkJCVZOX0hPTEQocHZwKTsKIAkJCSp2cHAgPSBwdnA7CiAJCX0K LQkJdm5fbG9jaygqdnBwLCBMS19FWENMVVNJVkUgfCBMS19SRVRSWSk7CiAJCXJldHVybiAoMCk7 CiAJfQogCkBAIC00ODUsNiArNDg0LDcgQEAgZ2ZzX2ZpbGVfY3JlYXRlKHNpemVfdCBzaXplLCB2 bm9kZV90ICpwdnAsIHZmc190ICp2ZnNwLCB2bm9kZW9wc190ICpvcHMpCiAJZnAgPSBrbWVtX3ph bGxvYyhzaXplLCBLTV9TTEVFUCk7CiAJZXJyb3IgPSBnZXRuZXd2bm9kZSgiemZzIiwgdmZzcCwg b3BzLCAmdnApOwogCUFTU0VSVChlcnJvciA9PSAwKTsKKwlWTl9MT0NLX0FTSEFSRSh2cCk7CiAJ dm5fbG9jayh2cCwgTEtfRVhDTFVTSVZFIHwgTEtfUkVUUlkpOwogCXZwLT52X2RhdGEgPSAoY2Fk ZHJfdClmcDsKIApAQCAtNDk2LDkgKzQ5Niw5IEBAIGdmc19maWxlX2NyZWF0ZShzaXplX3Qgc2l6 ZSwgdm5vZGVfdCAqcHZwLCB2ZnNfdCAqdmZzcCwgdm5vZGVvcHNfdCAqb3BzKQogCWZwLT5nZnNf c2l6ZSA9IHNpemU7CiAJZnAtPmdmc190eXBlID0gR0ZTX0ZJTEU7CiAKLQl2cC0+dl92ZmxhZyB8 PSBWVl9GT1JDRUlOU01ROworCXZwLT52X3ZmbGFnIHw9IFZWX0ZPUkNFSU5TTVEgfCBWVl9JTlNN UUhFQUQ7CiAJZXJyb3IgPSBpbnNtbnRxdWUodnAsIHZmc3ApOwotCXZwLT52X3ZmbGFnICY9IH5W Vl9GT1JDRUlOU01ROworCXZwLT52X3ZmbGFnICY9IH4oVlZfRk9SQ0VJTlNNUSB8IFZWX0lOU01R SEVBRCk7CiAJS0FTU0VSVChlcnJvciA9PSAwLCAoImluc21udHF1ZSgpIGZhaWxlZDogZXJyb3Ig JWQiLCBlcnJvcikpOwogCiAJLyoKQEAgLTYzNywxMiArNjM3LDcgQEAgZ2ZzX2ZpbGVfaW5hY3Rp dmUodm5vZGVfdCAqdnApCiAJaWYgKGZwLT5nZnNfcGFyZW50ID09IE5VTEwgfHwgKHZwLT52X2Zs YWcgJiBWX1hBVFRSRElSKSkKIAkJZ290byBmb3VuZDsKIAotCS8qCi0JICogWFhYIGNvcGUgd2l0 aCBhIEZyZWVCU0Qtc3BlY2lmaWMgcmFjZSB3aGVyZWluIHRoZSBwYXJlbnQncwotCSAqIHNuYXBz aG90IGRhdGEgY2FuIGJlIGZyZWVkIGJlZm9yZSB0aGUgcGFyZW50IGlzCi0JICovCi0JaWYgKChk cCA9IGZwLT5nZnNfcGFyZW50LT52X2RhdGEpID09IE5VTEwpCi0JCXJldHVybiAoTlVMTCk7CisJ ZHAgPSBmcC0+Z2ZzX3BhcmVudC0+dl9kYXRhOwogCiAJLyoKIAkgKiBGaXJzdCwgc2VlIGlmIHRo aXMgdm5vZGUgaXMgY2FjaGVkIGluIHRoZSBwYXJlbnQuCkBAIC02NjksMzcgKzY2NCw0NCBAQCBm b3VuZDoKIAlpZiAodnAtPnZfZmxhZyAmIFZfWEFUVFJESVIpCiAJCVZJX0xPQ0soZnAtPmdmc19w YXJlbnQpOwogI2VuZGlmCi0JVklfTE9DSyh2cCk7Ci0JLyoKLQkgKiBSZWFsbHkgcmVtb3ZlIHRo aXMgdm5vZGUKLQkgKi8KLQlkYXRhID0gdnAtPnZfZGF0YTsKLQlpZiAoZ2UgIT0gTlVMTCkgewor CWlmICh2cC0+dl9jb3VudCA9PSAwIHx8IHZwLT52X2lmbGFnICYgVklfRE9PTUVEKSB7CiAJCS8q Ci0JCSAqIElmIHRoaXMgd2FzIGEgc3RhdGljYWxseSBjYWNoZWQgZW50cnksIHNpbXBseSBzZXQg dGhlCi0JCSAqIGNhY2hlZCB2bm9kZSB0byBOVUxMLgorCQkgKiBSZWFsbHkgcmVtb3ZlIHRoaXMg dm5vZGUKIAkJICovCi0JCWdlLT5nZnNlX3Zub2RlID0gTlVMTDsKLQl9Ci0JVklfVU5MT0NLKHZw KTsKKwkJZGF0YSA9IHZwLT52X2RhdGE7CisJCWlmIChnZSAhPSBOVUxMKSB7CisJCQkvKgorCQkJ ICogSWYgdGhpcyB3YXMgYSBzdGF0aWNhbGx5IGNhY2hlZCBlbnRyeSwgc2ltcGx5IHNldCB0aGUK KwkJCSAqIGNhY2hlZCB2bm9kZSB0byBOVUxMLgorCQkJICovCisJCQlnZS0+Z2ZzZV92bm9kZSA9 IE5VTEw7CisJCX0KKyNpZmRlZiBUT0RPCisJCWlmICh2cC0+dl9mbGFnICYgVl9YQVRUUkRJUikK KwkJCVZJX1VOTE9DSyhmcC0+Z2ZzX3BhcmVudCk7CisjZW5kaWYKIAotCS8qCi0JICogRnJlZSB2 bm9kZSBhbmQgcmVsZWFzZSBwYXJlbnQKLQkgKi8KLQlpZiAoZnAtPmdmc19wYXJlbnQpIHsKLQkJ aWYgKGRwKQotCQkJZ2ZzX2Rpcl91bmxvY2soZHApOwotCQlWT1BfVU5MT0NLKHZwLCAwKTsKLQkJ Vk5fUkVMRShmcC0+Z2ZzX3BhcmVudCk7Ci0JCXZuX2xvY2sodnAsIExLX0VYQ0xVU0lWRSB8IExL X1JFVFJZKTsKKwkJLyoKKwkJICogRnJlZSB2bm9kZSBhbmQgcmVsZWFzZSBwYXJlbnQKKwkJICov CisJCWlmIChmcC0+Z2ZzX3BhcmVudCkgeworCQkJaWYgKGRwKQorCQkJCWdmc19kaXJfdW5sb2Nr KGRwKTsKKwkJCVZOX1JFTEUoZnAtPmdmc19wYXJlbnQpOworCQl9IGVsc2UgeworCQkJQVNTRVJU KHZwLT52X3Zmc3AgIT0gTlVMTCk7CisJCQlWRlNfUkVMRSh2cC0+dl92ZnNwKTsKKwkJfQogCX0g ZWxzZSB7Ci0JCUFTU0VSVCh2cC0+dl92ZnNwICE9IE5VTEwpOwotCQlWRlNfUkVMRSh2cC0+dl92 ZnNwKTsKLQl9CisJCWRhdGEgPSBOVUxMOwogI2lmZGVmIFRPRE8KLQlpZiAodnAtPnZfZmxhZyAm IFZfWEFUVFJESVIpCi0JCVZJX1VOTE9DSyhmcC0+Z2ZzX3BhcmVudCk7CisJCWlmICh2cC0+dl9m bGFnICYgVl9YQVRUUkRJUikKKwkJCVZJX1VOTE9DSyhmcC0+Z2ZzX3BhcmVudCk7CiAjZW5kaWYK KwkJaWYgKGRwKQorCQkJZ2ZzX2Rpcl91bmxvY2soZHApOworCX0KKwogCXJldHVybiAoZGF0YSk7 CiB9CiAKQEAgLTEyMzAsMTYgKzEyMzIsMTUgQEAgZ2ZzX3ZvcF9pbmFjdGl2ZShhcCkKIHsKIAl2 bm9kZV90ICp2cCA9IGFwLT5hX3ZwOwogCWdmc19maWxlX3QgKmZwID0gdnAtPnZfZGF0YTsKKwl2 b2lkICpkYXRhOwogCiAJaWYgKGZwLT5nZnNfdHlwZSA9PSBHRlNfRElSKQotCQlnZnNfZGlyX2lu YWN0aXZlKHZwKTsKKwkJZGF0YSA9IGdmc19kaXJfaW5hY3RpdmUodnApOwogCWVsc2UKLQkJZ2Zz X2ZpbGVfaW5hY3RpdmUodnApOworCQlkYXRhID0gZ2ZzX2ZpbGVfaW5hY3RpdmUodnApOwogCi0J VklfTE9DSyh2cCk7Ci0JdnAtPnZfZGF0YSA9IE5VTEw7Ci0JVklfVU5MT0NLKHZwKTsKLQlrbWVt X2ZyZWUoZnAsIGZwLT5nZnNfc2l6ZSk7CisJaWYgKGRhdGEgIT0gTlVMTCkKKwkJa21lbV9mcmVl KGRhdGEsIGZwLT5nZnNfc2l6ZSk7CiAKIAlyZXR1cm4gKDApOwogfQpkaWZmIC0tZ2l0IGEvc3lz L2NkZGwvY29udHJpYi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL3pmcy96ZnNfY3RsZGlyLmMg Yi9zeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMvemZzL3pmc19jdGxk aXIuYwppbmRleCAyOGFiMWZhLi4xNWE1NWQyIDEwMDY0NAotLS0gYS9zeXMvY2RkbC9jb250cmli L29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMvemZzL3pmc19jdGxkaXIuYworKysgYi9zeXMvY2Rk bC9jb250cmliL29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMvemZzL3pmc19jdGxkaXIuYwpAQCAt NjEyLDcgKzYxMiw3IEBAIHpmc2N0bF9mcmVlYnNkX3Jvb3RfbG9va3VwKGFwKQogCiAJZXJyID0g emZzY3RsX3Jvb3RfbG9va3VwKGR2cCwgbm0sIHZwcCwgTlVMTCwgMCwgTlVMTCwgY3IsIE5VTEws IE5VTEwsIE5VTEwpOwogCWlmIChlcnIgPT0gMCAmJiAobm1bMF0gIT0gJy4nIHx8IG5tWzFdICE9 ICdcMCcpKQotCQl2bl9sb2NrKCp2cHAsIExLX0VYQ0xVU0lWRSB8IExLX1JFVFJZKTsKKwkJZXJy ID0gdm5fbG9jaygqdnBwLCBhcC0+YV9jbnAtPmNuX2xrZmxhZ3MpOwogCXJldHVybiAoZXJyKTsK IH0KIApAQCAtOTc1LDggKzk3NSwxMSBAQCB6ZnNjdGxfc25hcGRpcl9sb29rdXAoYXApCiAJWkZT X0VOVEVSKHpmc3Zmcyk7CiAKIAlpZiAoZ2ZzX2xvb2t1cF9kb3QodnBwLCBkdnAsIHpmc3Zmcy0+ el9jdGxkaXIsIG5tKSA9PSAwKSB7CisJCWVyciA9IDA7CisJCWlmIChubVswXSAhPSAnLicgfHwg bm1bMV0gIT0gJ1wwJykKKwkJCWVyciA9IHZuX2xvY2soKnZwcCwgYXAtPmFfY25wLT5jbl9sa2Zs YWdzKTsKIAkJWkZTX0VYSVQoemZzdmZzKTsKLQkJcmV0dXJuICgwKTsKKwkJcmV0dXJuIChlcnIp OwogCX0KIAogCWlmIChmbGFncyAmIEZJR05PUkVDQVNFKSB7CkBAIC0xMDA0LDcgKzEwMDcsNyBA QCB6ZnNjdGxfc25hcGRpcl9sb29rdXAoYXApCiAJaWYgKChzZXAgPSBhdmxfZmluZCgmc2RwLT5z ZF9zbmFwcywgJnNlYXJjaCwgJndoZXJlKSkgIT0gTlVMTCkgewogCQkqdnBwID0gc2VwLT5zZV9y b290OwogCQlWTl9IT0xEKCp2cHApOwotCQllcnIgPSB0cmF2ZXJzZSh2cHAsIExLX0VYQ0xVU0lW RSB8IExLX1JFVFJZKTsKKwkJZXJyID0gdHJhdmVyc2UodnBwLCBhcC0+YV9jbnAtPmNuX2xrZmxh Z3MpOwogCQlpZiAoZXJyICE9IDApIHsKIAkJCVZOX1JFTEUoKnZwcCk7CiAJCQkqdnBwID0gTlVM TDsKQEAgLTEwMTMsNiArMTAxNiw4IEBAIHpmc2N0bF9zbmFwZGlyX2xvb2t1cChhcCkKIAkJCSAq IFRoZSBzbmFwc2hvdCB3YXMgdW5tb3VudGVkIGJlaGluZCBvdXIgYmFja3MsCiAJCQkgKiB0cnkg dG8gcmVtb3VudCBpdC4KIAkJCSAqLworCQkJVk5fSE9MRCgqdnBwKTsKKwkJCVZPUF9VTkxPQ0so KnZwcCwgMCk7CiAJCQlWRVJJRlkoemZzY3RsX3NuYXBzaG90X3puYW1lKGR2cCwgbm0sIE1BWE5B TUVMRU4sIHNuYXBuYW1lKSA9PSAwKTsKIAkJCWdvdG8gZG9tb3VudDsKIAkJfSBlbHNlIHsKQEAg LTEwNjQsNyArMTA2OSw2IEBAIHpmc2N0bF9zbmFwZGlyX2xvb2t1cChhcCkKIAlzZXAtPnNlX25h bWUgPSBrbWVtX2FsbG9jKHN0cmxlbihubSkgKyAxLCBLTV9TTEVFUCk7CiAJKHZvaWQpIHN0cmNw eShzZXAtPnNlX25hbWUsIG5tKTsKIAkqdnBwID0gc2VwLT5zZV9yb290ID0gemZzY3RsX3NuYXBz aG90X21rbm9kZShkdnAsIGRtdV9vYmpzZXRfaWQoc25hcCkpOwotCVZOX0hPTEQoKnZwcCk7CiAJ YXZsX2luc2VydCgmc2RwLT5zZF9zbmFwcywgc2VwLCB3aGVyZSk7CiAKIAlkbXVfb2Jqc2V0X3Jl bGUoc25hcCwgRlRBRyk7CkBAIC0xMDkxLDcgKzEwOTUsNiBAQCBkb21vdW50OgogCW11dGV4X2V4 aXQoJnNkcC0+c2RfbG9jayk7CiAJWkZTX0VYSVQoemZzdmZzKTsKIAotI2lmZGVmIGlsbHVtb3MK IAkvKgogCSAqIElmIHdlIGhhZCBhbiBlcnJvciwgZHJvcCBvdXIgaG9sZCBvbiB0aGUgdm5vZGUg YW5kCiAJICogemZzY3RsX3NuYXBzaG90X2luYWN0aXZlKCkgd2lsbCBjbGVhbiB1cC4KQEAgLTEx MDAsMTAgKzExMDMsNiBAQCBkb21vdW50OgogCQlWTl9SRUxFKCp2cHApOwogCQkqdnBwID0gTlVM TDsKIAl9Ci0jZWxzZQotCWlmIChlcnIgIT0gMCkKLQkJKnZwcCA9IE5VTEw7Ci0jZW5kaWYKIAly ZXR1cm4gKGVycik7CiB9CiAKQEAgLTExMzAsOCArMTEyOSwxMSBAQCB6ZnNjdGxfc2hhcmVzX2xv b2t1cChhcCkKIAlzdHJsY3B5KG5tLCBjbnAtPmNuX25hbWVwdHIsIGNucC0+Y25fbmFtZWxlbiAr IDEpOwogCiAJaWYgKGdmc19sb29rdXBfZG90KHZwcCwgZHZwLCB6ZnN2ZnMtPnpfY3RsZGlyLCBu bSkgPT0gMCkgeworCQllcnJvciA9IDA7CisJCWlmIChubVswXSAhPSAnLicgfHwgbm1bMV0gIT0g J1wwJykKKwkJCWVycm9yID0gdm5fbG9jaygqdnBwLCBhcC0+YV9jbnAtPmNuX2xrZmxhZ3MpOwog CQlaRlNfRVhJVCh6ZnN2ZnMpOwotCQlyZXR1cm4gKDApOworCQlyZXR1cm4gKGVycm9yKTsKIAl9 CiAKIAlpZiAoemZzdmZzLT56X3NoYXJlc19kaXIgPT0gMCkgewpAQCAtMTM0NCwyMiArMTM0Niwx NSBAQCB6ZnNjdGxfc25hcGRpcl9pbmFjdGl2ZShhcCkKIAl2bm9kZV90ICp2cCA9IGFwLT5hX3Zw OwogCXpmc2N0bF9zbmFwZGlyX3QgKnNkcCA9IHZwLT52X2RhdGE7CiAJemZzX3NuYXBlbnRyeV90 ICpzZXA7Ci0KLQkvKgotCSAqIE9uIGZvcmNlZCB1bm1vdW50IHdlIGhhdmUgdG8gZnJlZSBzbmFw c2hvdHMgZnJvbSBoZXJlLgotCSAqLwotCW11dGV4X2VudGVyKCZzZHAtPnNkX2xvY2spOwotCXdo aWxlICgoc2VwID0gYXZsX2ZpcnN0KCZzZHAtPnNkX3NuYXBzKSkgIT0gTlVMTCkgewotCQlhdmxf cmVtb3ZlKCZzZHAtPnNkX3NuYXBzLCBzZXApOwotCQlrbWVtX2ZyZWUoc2VwLT5zZV9uYW1lLCBz dHJsZW4oc2VwLT5zZV9uYW1lKSArIDEpOwotCQlrbWVtX2ZyZWUoc2VwLCBzaXplb2YgKHpmc19z bmFwZW50cnlfdCkpOworCXZvaWQgKnByaXZhdGU7CisKKwlwcml2YXRlID0gZ2ZzX2Rpcl9pbmFj dGl2ZSh2cCk7CisJaWYgKHByaXZhdGUgIT0gTlVMTCkgeworCQlBU1NFUlQoYXZsX251bW5vZGVz KCZzZHAtPnNkX3NuYXBzKSA9PSAwKTsKKwkJbXV0ZXhfZGVzdHJveSgmc2RwLT5zZF9sb2NrKTsK KwkJYXZsX2Rlc3Ryb3koJnNkcC0+c2Rfc25hcHMpOworCQlrbWVtX2ZyZWUocHJpdmF0ZSwgc2l6 ZW9mICh6ZnNjdGxfc25hcGRpcl90KSk7CiAJfQotCW11dGV4X2V4aXQoJnNkcC0+c2RfbG9jayk7 Ci0JZ2ZzX2Rpcl9pbmFjdGl2ZSh2cCk7Ci0JQVNTRVJUKGF2bF9udW1ub2Rlcygmc2RwLT5zZF9z bmFwcykgPT0gMCk7Ci0JbXV0ZXhfZGVzdHJveSgmc2RwLT5zZF9sb2NrKTsKLQlhdmxfZGVzdHJv eSgmc2RwLT5zZF9zbmFwcyk7Ci0Ja21lbV9mcmVlKHNkcCwgc2l6ZW9mICh6ZnNjdGxfc25hcGRp cl90KSk7CiAKIAlyZXR1cm4gKDApOwogfQpAQCAtMTQ0MSw3ICsxNDM2LDYgQEAgemZzY3RsX3Nu YXBzaG90X21rbm9kZSh2bm9kZV90ICpwdnAsIHVpbnQ2NF90IG9ianNldCkKIAogCXZwID0gZ2Zz X2Rpcl9jcmVhdGUoc2l6ZW9mICh6ZnNjdGxfbm9kZV90KSwgcHZwLCBwdnAtPnZfdmZzcCwKIAkg ICAgJnpmc2N0bF9vcHNfc25hcHNob3QsIE5VTEwsIE5VTEwsIE1BWE5BTUVMRU4sIE5VTEwsIE5V TEwpOwotCVZOX0hPTEQodnApOwogCXpjcCA9IHZwLT52X2RhdGE7CiAJemNwLT56Y19pZCA9IG9i anNldDsKIAlWT1BfVU5MT0NLKHZwLCAwKTsKQEAgLTE0NjIsMTggKzE0NTYsMjUgQEAgemZzY3Rs X3NuYXBzaG90X2luYWN0aXZlKGFwKQogCXpmc2N0bF9zbmFwZGlyX3QgKnNkcDsKIAl6ZnNfc25h cGVudHJ5X3QgKnNlcCwgKm5leHQ7CiAJaW50IGxvY2tlZDsKLQl2bm9kZV90ICpkdnA7CisJZ2Zz X2Rpcl90ICpkcCA9IHZwLT52X2RhdGE7CisJdm5vZGVfdCAqZHZwID0gZHAtPmdmc2RfZmlsZS5n ZnNfcGFyZW50OwogCi0JaWYgKHZwLT52X2NvdW50ID4gMCkKLQkJZ290byBlbmQ7Ci0KLQlWRVJJ RlkoZ2ZzX2Rpcl9sb29rdXAodnAsICIuLiIsICZkdnAsIGNyLCAwLCBOVUxMLCBOVUxMKSA9PSAw KTsKKwlWTl9IT0xEKGR2cCk7CisJVk9QX1VOTE9DSyh2cCwgMCk7CiAJc2RwID0gZHZwLT52X2Rh dGE7Ci0JVk9QX1VOTE9DSyhkdnAsIDApOwogCiAJaWYgKCEobG9ja2VkID0gTVVURVhfSEVMRCgm c2RwLT5zZF9sb2NrKSkpCiAJCW11dGV4X2VudGVyKCZzZHAtPnNkX2xvY2spOwogCisJdm5fbG9j ayh2cCwgTEtfRVhDTFVTSVZFIHwgTEtfUkVUUlkpOworCisJaWYgKHZwLT52X2NvdW50ID4gMCkg eworCQlpZiAoIWxvY2tlZCkKKwkJCW11dGV4X2V4aXQoJnNkcC0+c2RfbG9jayk7CisJCVZOX1JF TEUoZHZwKTsKKwkJcmV0dXJuKDApOworCX0KKwogCUFTU0VSVCghdm5faXNtbnRwdCh2cCkpOwog CiAJc2VwID0gYXZsX2ZpcnN0KCZzZHAtPnNkX3NuYXBzKTsKQEAgLTE0OTQsNyArMTQ5NSw2IEBA IHpmc2N0bF9zbmFwc2hvdF9pbmFjdGl2ZShhcCkKIAkJbXV0ZXhfZXhpdCgmc2RwLT5zZF9sb2Nr KTsKIAlWTl9SRUxFKGR2cCk7CiAKLWVuZDoKIAkvKgogCSAqIERpc3Bvc2Ugb2YgdGhlIHZub2Rl IGZvciB0aGUgc25hcHNob3QgbW91bnQgcG9pbnQuCiAJICogVGhpcyBpcyBzYWZlIHRvIGRvIGJl Y2F1c2Ugb25jZSB0aGlzIGVudHJ5IGhhcyBiZWVuIHJlbW92ZWQKQEAgLTE1ODgsNyArMTU4OCw3 IEBAIHpmc2N0bF9zbmFwc2hvdF9sb29rdXAoYXApCiAJZXJyb3IgPSB6ZnNjdGxfcm9vdF9sb29r dXAoemZzdmZzLT56X2N0bGRpciwgInNuYXBzaG90IiwgdnBwLAogCSAgICBOVUxMLCAwLCBOVUxM LCBjciwgTlVMTCwgTlVMTCwgTlVMTCk7CiAJaWYgKGVycm9yID09IDApCi0JCXZuX2xvY2soKnZw cCwgTEtfRVhDTFVTSVZFIHwgTEtfUkVUUlkpOworCQllcnJvciA9IHZuX2xvY2soKnZwcCwgYXAt PmFfY25wLT5jbl9sa2ZsYWdzKTsKIAlyZXR1cm4gKGVycm9yKTsKIH0KIApkaWZmIC0tZ2l0IGEv c3lzL2NkZGwvY29udHJpYi9vcGVuc29sYXJpcy91dHMvY29tbW9uL2ZzL3pmcy96ZnNfdmZzb3Bz LmMgYi9zeXMvY2RkbC9jb250cmliL29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMvemZzL3pmc192 ZnNvcHMuYwppbmRleCA4ZWI4OTUzLi44ZWE3NjYxIDEwMDY0NAotLS0gYS9zeXMvY2RkbC9jb250 cmliL29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMvemZzL3pmc192ZnNvcHMuYworKysgYi9zeXMv Y2RkbC9jb250cmliL29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMvemZzL3pmc192ZnNvcHMuYwpA QCAtMjAzNiwxMiArMjAzNiw3IEBAIHpmc191bW91bnQodmZzX3QgKnZmc3AsIGludCBmZmxhZykK IAkgKi8KIAlpZiAoemZzdmZzLT56X2N0bGRpciAhPSBOVUxMKQogCQl6ZnNjdGxfZGVzdHJveSh6 ZnN2ZnMpOwotCWlmICh6ZnN2ZnMtPnpfaXNzbmFwKSB7Ci0JCXZub2RlX3QgKnN2cCA9IHZmc3At Pm1udF92bm9kZWNvdmVyZWQ7CiAKLQkJaWYgKHN2cC0+dl9jb3VudCA+PSAyKQotCQkJVk5fUkVM RShzdnApOwotCX0KIAl6ZnNfZnJlZXZmcyh2ZnNwKTsKIAogCXJldHVybiAoMCk7CmRpZmYgLS1n aXQgYS9zeXMva2Vybi92ZnNfc3Vici5jIGIvc3lzL2tlcm4vdmZzX3N1YnIuYwppbmRleCA5MWM2 NGEzLi4yOWI2Mzk1IDEwMDY0NAotLS0gYS9zeXMva2Vybi92ZnNfc3Vici5jCisrKyBiL3N5cy9r ZXJuL3Zmc19zdWJyLmMKQEAgLTExOTgsNyArMTE5OCwxMCBAQCBpbnNtbnRxdWUxKHN0cnVjdCB2 bm9kZSAqdnAsIHN0cnVjdCBtb3VudCAqbXAsCiAJfQogCXZwLT52X21vdW50ID0gbXA7CiAJTU5U X1JFRihtcCk7Ci0JVEFJTFFfSU5TRVJUX1RBSUwoJm1wLT5tbnRfbnZub2RlbGlzdCwgdnAsIHZf bm1udHZub2Rlcyk7CisJaWYgKHZwLT52X3ZmbGFnICYgVlZfSU5TTVFIRUFEKQorCQlUQUlMUV9J TlNFUlRfSEVBRCgmbXAtPm1udF9udm5vZGVsaXN0LCB2cCwgdl9ubW50dm5vZGVzKTsKKwllbHNl CisJCVRBSUxRX0lOU0VSVF9UQUlMKCZtcC0+bW50X252bm9kZWxpc3QsIHZwLCB2X25tbnR2bm9k ZXMpOwogCVZOQVNTRVJUKG1wLT5tbnRfbnZub2RlbGlzdHNpemUgPj0gMCwgdnAsCiAJCSgibmVn IG1vdW50IHBvaW50IHZub2RlIGxpc3Qgc2l6ZSIpKTsKIAltcC0+bW50X252bm9kZWxpc3RzaXpl Kys7CmRpZmYgLS1naXQgYS9zeXMvc3lzL3Zub2RlLmggYi9zeXMvc3lzL3Zub2RlLmgKaW5kZXgg YjBjYmNjMC4uOGRmYWUzYSAxMDA2NDQKLS0tIGEvc3lzL3N5cy92bm9kZS5oCisrKyBiL3N5cy9z eXMvdm5vZGUuaApAQCAtMjUzLDYgKzI1Myw3IEBAIHN0cnVjdCB4dm5vZGUgewogI2RlZmluZQlW Vl9ERUxFVEVECTB4MDQwMAkvKiBzaG91bGQgYmUgcmVtb3ZlZCAqLwogI2RlZmluZQlWVl9NRAkJ MHgwODAwCS8qIHZub2RlIGJhY2tzIHRoZSBtZCBkZXZpY2UgKi8KICNkZWZpbmUJVlZfRk9SQ0VJ TlNNUQkweDEwMDAJLyogZm9yY2UgdGhlIGluc21udHF1ZSB0byBzdWNjZWVkICovCisjZGVmaW5l CVZWX0lOU01RSEVBRAkweDIwMDAJLyogaW5zZXJ0IGluc3RlYWQgb2YgYXBwZW5kaW5nIHRvIG1u dF9udm5vZGVsaXN0ICovCiAKIC8qCiAgKiBWbm9kZSBhdHRyaWJ1dGVzLiAgQSBmaWVsZCB2YWx1 ZSBvZiBWTk9WQUwgcmVwcmVzZW50cyBhIGZpZWxkIHdob3NlIHZhbHVlCg== --=_e29d732e9be4cf639842cb9d8de74c22-- From owner-freebsd-fs@FreeBSD.ORG Mon Jan 20 15:51:13 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B273894E for ; Mon, 20 Jan 2014 15:51:13 +0000 (UTC) Received: from mail05.raumopol.de (mail05.raumopol.de [217.11.57.242]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 775AF1DA1 for ; Mon, 20 Jan 2014 15:51:13 +0000 (UTC) Received: from mail03.raumopol.de (mail03.localnet.raumopol.de [192.168.1.10]) by mail05.raumopol.de (Postfix) with ESMTP id B51D61A0220D1 for ; Mon, 20 Jan 2014 16:39:53 +0100 (CET) Received: from authenticated_user by mail03.raumopol.de (Postfix) with ESMTPSA id 5B9343846362 for ; Mon, 20 Jan 2014 16:39:53 +0100 (CET) Date: Mon, 20 Jan 2014 16:39:51 +0100 From: Philipp To: freebsd-fs@freebsd.org Subject: Filesystem errors with VirtIO Message-ID: <20140120163951.7fe4c1f8@calavera> X-Mailer: Claws Mail 3.9.3 (GTK+ 2.24.22; amd64-portbld-freebsd10.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jan 2014 15:51:13 -0000 Hi everyone, I'm running FreeBSD 10.0-RELEASE on a virtualized KVM server. I'm using VirtIO from base to access NIC and HDD directly. Filesystem is UFS2 with Journaling and SU. Last week I had to reinstall FreeBSD because the salvation of the several filesystem errors wasn't possible anymore. I don't have any evidence but I guess it has something to do with the VirtIO driver. After running the machine for 48 hours with a freshly installed FreeBSD 10 several soft update inconsistencies occur: # fsck ** /dev/vtbd0p2 (NO WRITE) USE JOURNAL? no ** Skipping journal, falling through to full fsck SETTING DIRTY FLAG IN READ_ONLY MODE UNEXPECTED SOFT UPDATE INCONSISTENCY ** Last Mounted on / ** Root file system ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity LINK COUNT INCREASING UNEXPECTED SOFT UPDATE INCONSISTENCY ADJUST? no LINK COUNT DIR I=11075332 OWNER=root MODE=40755 SIZE=512 MTIME=Jan 20 15:06 2014 COUNT 0 SHOULD BE 1 LINK COUNT INCREASING UNEXPECTED SOFT UPDATE INCONSISTENCY ADJUST? no LINK COUNT DIR I=11316145 OWNER=root MODE=40755 SIZE=512 MTIME=Jan 20 15:26 2014 COUNT 0 SHOULD BE 1 LINK COUNT INCREASING UNEXPECTED SOFT UPDATE INCONSISTENCY ADJUST? no UNREF FILE I=11316146 OWNER=root MODE=100644 SIZE=227096 MTIME=Jan 20 15:26 2014 CLEAR? no LINK COUNT DIR I=11316147 OWNER=root MODE=40755 SIZE=512 MTIME=Jan 20 15:34 2014 COUNT 0 SHOULD BE 1 LINK COUNT INCREASING UNEXPECTED SOFT UPDATE INCONSISTENCY ADJUST? no LINK COUNT DIR I=11316148 OWNER=root MODE=40755 SIZE=512 MTIME=Jan 20 15:49 2014 COUNT 0 SHOULD BE 1 LINK COUNT INCREASING UNEXPECTED SOFT UPDATE INCONSISTENCY ADJUST? no UNREF FILE I=11316150 OWNER=root MODE=100644 SIZE=928 MTIME=Jan 20 15:49 2014 CLEAR? no LINK COUNT DIR I=11333352 OWNER=root MODE=40755 SIZE=512 MTIME=Jan 20 15:18 2014 COUNT 0 SHOULD BE 1 LINK COUNT INCREASING UNEXPECTED SOFT UPDATE INCONSISTENCY ADJUST? no ** Phase 5 - Check Cyl groups 250807 files, 860388 used, 36209947 free (8179 frags, 4525221 blocks, 0.0% fragmentation) After a reboot different inconsistencies appear. I had the very same problems with 9.2-RELEASE, which caused the need of a reinstallation of the OS (errors like "vtbd0: hard error cmd=write fsbn 24162" occured). Can someone confirm these problems with VirtIO? What else can I do? Regards, Philipp From owner-freebsd-fs@FreeBSD.ORG Mon Jan 20 16:55:49 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3AA33AD7 for ; Mon, 20 Jan 2014 16:55:49 +0000 (UTC) Received: from krichy.tvnetwork.hu (unknown [IPv6:2a01:be00:0:2::10]) by mx1.freebsd.org (Postfix) with ESMTP id F0FDB1346 for ; Mon, 20 Jan 2014 16:55:48 +0000 (UTC) Received: by krichy.tvnetwork.hu (Postfix, from userid 1000) id 0039A606F; Mon, 20 Jan 2014 17:55:21 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by krichy.tvnetwork.hu (Postfix) with ESMTP id E9526606E; Mon, 20 Jan 2014 17:55:21 +0100 (CET) Date: Mon, 20 Jan 2014 17:55:21 +0100 (CET) From: krichy@tvnetwork.hu To: Philipp Subject: Re: Filesystem errors with VirtIO In-Reply-To: <20140120163951.7fe4c1f8@calavera> Message-ID: References: <20140120163951.7fe4c1f8@calavera> User-Agent: Alpine 2.10 (DEB 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jan 2014 16:55:49 -0000 Do you have OCZ SSD's? http://lkcl.net/reports/ssd_analysis.html Kojedzinszky Richard Euronet Magyarorszag Informatikai Zrt. On Mon, 20 Jan 2014, Philipp wrote: > Date: Mon, 20 Jan 2014 16:39:51 +0100 > From: Philipp > To: freebsd-fs@freebsd.org > Subject: Filesystem errors with VirtIO > > Hi everyone, > > I'm running FreeBSD 10.0-RELEASE on a virtualized KVM server. I'm > using VirtIO from base to access NIC and HDD directly. Filesystem is > UFS2 with Journaling and SU. Last week I had to reinstall FreeBSD > because the salvation of the several filesystem errors wasn't possible > anymore. I don't have any evidence but I guess it has something to do > with the VirtIO driver. > > After running the machine for 48 hours with a freshly installed > FreeBSD 10 several soft update inconsistencies occur: > > # fsck > ** /dev/vtbd0p2 (NO WRITE) > > USE JOURNAL? no > > ** Skipping journal, falling through to full fsck > > SETTING DIRTY FLAG IN READ_ONLY MODE > > UNEXPECTED SOFT UPDATE INCONSISTENCY > ** Last Mounted on / > ** Root file system > ** Phase 1 - Check Blocks and Sizes > ** Phase 2 - Check Pathnames > ** Phase 3 - Check Connectivity > LINK COUNT INCREASING > UNEXPECTED SOFT UPDATE INCONSISTENCY > > ADJUST? no > > LINK COUNT DIR I=11075332 OWNER=root MODE=40755 > SIZE=512 MTIME=Jan 20 15:06 2014 COUNT 0 SHOULD BE 1 > LINK COUNT INCREASING > UNEXPECTED SOFT UPDATE INCONSISTENCY > > ADJUST? no > > LINK COUNT DIR I=11316145 OWNER=root MODE=40755 > SIZE=512 MTIME=Jan 20 15:26 2014 COUNT 0 SHOULD BE 1 > LINK COUNT INCREASING > UNEXPECTED SOFT UPDATE INCONSISTENCY > > ADJUST? no > > UNREF FILE I=11316146 OWNER=root MODE=100644 > SIZE=227096 MTIME=Jan 20 15:26 2014 > CLEAR? no > > LINK COUNT DIR I=11316147 OWNER=root MODE=40755 > SIZE=512 MTIME=Jan 20 15:34 2014 COUNT 0 SHOULD BE 1 > LINK COUNT INCREASING > UNEXPECTED SOFT UPDATE INCONSISTENCY > > ADJUST? no > > LINK COUNT DIR I=11316148 OWNER=root MODE=40755 > SIZE=512 MTIME=Jan 20 15:49 2014 COUNT 0 SHOULD BE 1 > LINK COUNT INCREASING > UNEXPECTED SOFT UPDATE INCONSISTENCY > > ADJUST? no > > UNREF FILE I=11316150 OWNER=root MODE=100644 > SIZE=928 MTIME=Jan 20 15:49 2014 > CLEAR? no > > LINK COUNT DIR I=11333352 OWNER=root MODE=40755 > SIZE=512 MTIME=Jan 20 15:18 2014 COUNT 0 SHOULD BE 1 > LINK COUNT INCREASING > UNEXPECTED SOFT UPDATE INCONSISTENCY > > ADJUST? no > > ** Phase 5 - Check Cyl groups > 250807 files, 860388 used, 36209947 free (8179 frags, 4525221 blocks, > 0.0% fragmentation) > > After a reboot different inconsistencies appear. I had the very same > problems with 9.2-RELEASE, which caused the need of a reinstallation > of the OS (errors like "vtbd0: hard error cmd=write fsbn 24162" > occured). > > Can someone confirm these problems with VirtIO? What else can I do? > > Regards, Philipp > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@FreeBSD.ORG Mon Jan 20 17:21:24 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 88FEDE1D for ; Mon, 20 Jan 2014 17:21:24 +0000 (UTC) Received: from mail02.raumopol.de (mail02.raumopol.de [89.238.79.3]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 4C07E1631 for ; Mon, 20 Jan 2014 17:21:24 +0000 (UTC) Received: from mail03.raumopol.de (mail03.localnet.raumopol.de [192.168.1.10]) by mail02.raumopol.de (Postfix) with ESMTP id 8FB5C214C11C; Mon, 20 Jan 2014 18:08:43 +0100 (CET) Received: from authenticated_user by mail03.raumopol.de (Postfix) with ESMTPSA id 6589338081A7; Mon, 20 Jan 2014 18:08:43 +0100 (CET) Date: Mon, 20 Jan 2014 18:08:42 +0100 From: Philipp To: krichy@tvnetwork.hu, freebsd-fs@freebsd.org Subject: Re: Filesystem errors with VirtIO Message-ID: <20140120180842.7d4bd75b@calavera> In-Reply-To: References: <20140120163951.7fe4c1f8@calavera> X-Mailer: Claws Mail 3.9.3 (GTK+ 2.24.22; amd64-portbld-freebsd10.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jan 2014 17:21:24 -0000 No, but thanks for the hint. Actually, my provider has serious issues with the RAID my virtual server is using. Thank you. On Mon, 20 Jan 2014 17:55:21 +0100 (CET) krichy@tvnetwork.hu wrote: > > Do you have OCZ SSD's? > > http://lkcl.net/reports/ssd_analysis.html > > Kojedzinszky Richard > Euronet Magyarorszag Informatikai Zrt. From owner-freebsd-fs@FreeBSD.ORG Tue Jan 21 11:29:11 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1D9883DF; Tue, 21 Jan 2014 11:29:11 +0000 (UTC) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 0DD111249; Tue, 21 Jan 2014 11:29:09 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id NAA29314; Tue, 21 Jan 2014 13:29:08 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1W5ZW0-000HfO-9G; Tue, 21 Jan 2014 13:29:08 +0200 Message-ID: <52DE59CC.5070408@FreeBSD.org> Date: Tue, 21 Jan 2014 13:28:12 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 MIME-Version: 1.0 To: freebsd-fs@FreeBSD.org, jlh@FreeBSD.org Subject: Re: ARC_SPACE_OTHER exceeds arc_max References: <20131011184206.GA60057@caravan.chchile.org> In-Reply-To: <20131011184206.GA60057@caravan.chchile.org> X-Enigmail-Version: 1.6 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jan 2014 11:29:11 -0000 on 11/10/2013 21:42 Jeremie Le Hen said the following: > Hi, > > (Please Cc: me on reply, as I'm not subscribed.) Then you probably should not have set Mail-Followup-To header to freebsd-fs@FreeBSD.org. > On my FreeBSD 9.1 machine, roughly 2/3 of the times the daily scripts > are run, my ARC size is outgrowing like crazy the vfs.zfs.arc_max being > set to 536870912 (512 MB). > > The consquence of this is that userland processes are killed (!). OK > this box has no swap space and should have but still, this sounds really > crazy that a filesystem cache is able to "reclaim" memory for running > processes :). ARC_SPACE_OTHER is used to account for dnode objects and dbuf objects as opposed to actual data buffers in ARC cache. The fact that ARC_SPACE_OTHER grows beyond limits means that the objects can not be evicted and thus they are in use. My guess that this is because you have rather high vnode limits and it is the ZFS vnodes that keep those objects in memory. The fact that arc_meta_used significantly exceeds arc_meta_limit supports my theory as arc_meta_used accounts for data buffers that include those that back dnode objects and dbuf objects. My other guess is that you have lots of very small files and/or lots of directories that are traversed by the daily scripts. FreeBSD does not have any feedback mechanism from ARC to vnode management code and can not ask for vnodes on a free list to be reclaimed when metdata limits are exceeded. Perhaps we should consider adding such a mechanism, if possible. But meanwhile please consider tuning your vnode limits so that they are in agreement with ARC limits. You may also want to set properties like nosuid / noexec on some of your filesystems to limit a number of visited files by daily security checks. Not sure if this advice is applicable to your configuration. > I've used top -b every 30 seconds to get an idea of the system's memory, > logs below. Here is also a snippet of my sysctls related to zfs: > > vfs.zfs.l2arc_norw: 1 > vfs.zfs.l2arc_feed_again: 1 > vfs.zfs.l2arc_noprefetch: 1 > vfs.zfs.l2arc_feed_min_ms: 200 > vfs.zfs.l2arc_feed_secs: 1 > vfs.zfs.l2arc_headroom: 2 > vfs.zfs.l2arc_write_boost: 8388608 > vfs.zfs.l2arc_write_max: 8388608 > vfs.zfs.arc_meta_limit: 134217728 > vfs.zfs.arc_meta_used: 236528568 > vfs.zfs.arc_min: 67108864 > vfs.zfs.arc_max: 536870912 > debug.adaptive_machine_arch: 1 > hw.machine_arch: amd64 > kstat.zfs.misc.arcstats.hits: 77149964 > kstat.zfs.misc.arcstats.misses: 13690743 > kstat.zfs.misc.arcstats.demand_data_hits: 8073317 > kstat.zfs.misc.arcstats.demand_data_misses: 274103 > kstat.zfs.misc.arcstats.demand_metadata_hits: 69076639 > kstat.zfs.misc.arcstats.demand_metadata_misses: 13416617 > kstat.zfs.misc.arcstats.prefetch_data_hits: 0 > kstat.zfs.misc.arcstats.prefetch_data_misses: 0 > kstat.zfs.misc.arcstats.prefetch_metadata_hits: 8 > kstat.zfs.misc.arcstats.prefetch_metadata_misses: 23 > kstat.zfs.misc.arcstats.mru_hits: 33260238 > kstat.zfs.misc.arcstats.mru_ghost_hits: 2830869 > kstat.zfs.misc.arcstats.mfu_hits: 43889719 > kstat.zfs.misc.arcstats.mfu_ghost_hits: 3452884 > kstat.zfs.misc.arcstats.allocated: 14361097 > kstat.zfs.misc.arcstats.deleted: 7860017 > kstat.zfs.misc.arcstats.stolen: 6994054 > kstat.zfs.misc.arcstats.recycle_miss: 7051205 > kstat.zfs.misc.arcstats.mutex_miss: 4479 > kstat.zfs.misc.arcstats.evict_skip: 4052583 > kstat.zfs.misc.arcstats.evict_l2_cached: 0 > kstat.zfs.misc.arcstats.evict_l2_eligible: 129211340800 > kstat.zfs.misc.arcstats.evict_l2_ineligible: 14336 > kstat.zfs.misc.arcstats.hash_elements: 88387 > kstat.zfs.misc.arcstats.hash_elements_max: 133658 > kstat.zfs.misc.arcstats.hash_collisions: 14197499 > kstat.zfs.misc.arcstats.hash_chains: 16044 > kstat.zfs.misc.arcstats.hash_chain_max: 27 > kstat.zfs.misc.arcstats.p: 188161024 > kstat.zfs.misc.arcstats.c: 536870912 > kstat.zfs.misc.arcstats.c_min: 67108864 > kstat.zfs.misc.arcstats.c_max: 536870912 > kstat.zfs.misc.arcstats.size: 534264760 > kstat.zfs.misc.arcstats.hdr_size: 20228688 > kstat.zfs.misc.arcstats.data_size: 406874112 > kstat.zfs.misc.arcstats.other_size: 107161960 > kstat.zfs.misc.arcstats.l2_hits: 0 > kstat.zfs.misc.arcstats.l2_misses: 0 > kstat.zfs.misc.arcstats.l2_feeds: 0 > kstat.zfs.misc.arcstats.l2_rw_clash: 0 > kstat.zfs.misc.arcstats.l2_read_bytes: 0 > kstat.zfs.misc.arcstats.l2_write_bytes: 0 > kstat.zfs.misc.arcstats.l2_writes_sent: 0 > kstat.zfs.misc.arcstats.l2_writes_done: 0 > kstat.zfs.misc.arcstats.l2_writes_error: 0 > kstat.zfs.misc.arcstats.l2_writes_hdr_miss: 0 > kstat.zfs.misc.arcstats.l2_evict_lock_retry: 0 > kstat.zfs.misc.arcstats.l2_evict_reading: 0 > kstat.zfs.misc.arcstats.l2_free_on_write: 0 > kstat.zfs.misc.arcstats.l2_abort_lowmem: 0 > kstat.zfs.misc.arcstats.l2_cksum_bad: 0 > kstat.zfs.misc.arcstats.l2_io_error: 0 > kstat.zfs.misc.arcstats.l2_size: 0 > kstat.zfs.misc.arcstats.l2_hdr_size: 0 > kstat.zfs.misc.arcstats.l2_write_trylock_fail: 0 > kstat.zfs.misc.arcstats.l2_write_passed_headroom: 0 > kstat.zfs.misc.arcstats.l2_write_spa_mismatch: 0 > kstat.zfs.misc.arcstats.l2_write_in_l2: 0 > kstat.zfs.misc.arcstats.l2_write_io_in_progress: 0 > kstat.zfs.misc.arcstats.l2_write_not_cacheable: 5 > kstat.zfs.misc.arcstats.l2_write_full: 0 > kstat.zfs.misc.arcstats.l2_write_buffer_iter: 0 > kstat.zfs.misc.arcstats.l2_write_pios: 0 > kstat.zfs.misc.arcstats.l2_write_buffer_bytes_scanned: 0 > kstat.zfs.misc.arcstats.l2_write_buffer_list_iter: 0 > kstat.zfs.misc.arcstats.l2_write_buffer_list_null_iter: 0 > kstat.zfs.misc.arcstats.memory_throttle_count: 0 > kstat.zfs.misc.arcstats.duplicate_buffers: 0 > kstat.zfs.misc.arcstats.duplicate_buffers_size: 0 > kstat.zfs.misc.arcstats.duplicate_reads: 0 > > > > > > > > > ct 8 03:00:11 CEST 2013 > Mem: 97M Active, 112M Inact, 1258M Wired, 552K Cache, 482M Free > ARC: 310M Total, 42M MFU, 147M MRU, 912K Anon, 20M Header, 101M Other > > Tue Oct 8 03:00:41 CEST 2013 > Mem: 97M Active, 112M Inact, 1258M Wired, 552K Cache, 482M Free > ARC: 310M Total, 42M MFU, 147M MRU, 912K Anon, 20M Header, 101M Other > > Tue Oct 8 03:01:11 CEST 2013 > Mem: 105M Active, 114M Inact, 1274M Wired, 552K Cache, 457M Free > ARC: 330M Total, 42M MFU, 159M MRU, 2241K Anon, 20M Header, 107M Other > > Tue Oct 8 03:01:41 CEST 2013 > Mem: 105M Active, 115M Inact, 1241M Wired, 552K Cache, 490M Free > ARC: 332M Total, 32M MFU, 137M MRU, 912K Anon, 21M Header, 141M Other > > Tue Oct 8 03:02:11 CEST 2013 > Mem: 105M Active, 121M Inact, 1251M Wired, 552K Cache, 473M Free > ARC: 326M Total, 34M MFU, 145M MRU, 912K Anon, 22M Header, 124M Other > > Tue Oct 8 03:02:42 CEST 2013 > Mem: 105M Active, 121M Inact, 1247M Wired, 552K Cache, 478M Free > ARC: 350M Total, 32M MFU, 143M MRU, 1265K Anon, 23M Header, 150M Other > > Tue Oct 8 03:03:12 CEST 2013 > Mem: 104M Active, 121M Inact, 1232M Wired, 552K Cache, 493M Free > ARC: 293M Total, 35M MFU, 126M MRU, 928K Anon, 24M Header, 107M Other > > Tue Oct 8 03:03:42 CEST 2013 > Mem: 104M Active, 119M Inact, 1253M Wired, 552K Cache, 475M Free > ARC: 367M Total, 37M MFU, 145M MRU, 928K Anon, 24M Header, 160M Other > > Tue Oct 8 03:04:12 CEST 2013 > Mem: 104M Active, 119M Inact, 1284M Wired, 552K Cache, 443M Free > ARC: 316M Total, 55M MFU, 157M MRU, 912K Anon, 24M Header, 78M Other > > Tue Oct 8 03:04:42 CEST 2013 > Mem: 104M Active, 119M Inact, 1379M Wired, 552K Cache, 348M Free > ARC: 413M Total, 84M MFU, 223M MRU, 1743K Anon, 24M Header, 81M Other > > Tue Oct 8 03:05:12 CEST 2013 > Mem: 104M Active, 119M Inact, 1451M Wired, 552K Cache, 276M Free > ARC: 484M Total, 106M MFU, 274M MRU, 928K Anon, 23M Header, 80M Other > > Tue Oct 8 03:05:42 CEST 2013 > Mem: 104M Active, 119M Inact, 1451M Wired, 552K Cache, 276M Free > ARC: 494M Total, 119M MFU, 261M MRU, 930K Anon, 23M Header, 90M Other > > Tue Oct 8 03:06:12 CEST 2013 > Mem: 104M Active, 119M Inact, 1451M Wired, 552K Cache, 276M Free > ARC: 501M Total, 151M MFU, 228M MRU, 1226K Anon, 24M Header, 96M Other > > Tue Oct 8 03:06:42 CEST 2013 > Mem: 104M Active, 119M Inact, 1450M Wired, 552K Cache, 277M Free > ARC: 502M Total, 191M MFU, 188M MRU, 944K Anon, 24M Header, 99M Other > > Tue Oct 8 03:07:12 CEST 2013 > Mem: 104M Active, 119M Inact, 1451M Wired, 552K Cache, 276M Free > ARC: 507M Total, 226M MFU, 153M MRU, 1094K Anon, 24M Header, 103M Other > > Tue Oct 8 03:07:42 CEST 2013 > Mem: 104M Active, 118M Inact, 1407M Wired, 552K Cache, 321M Free > ARC: 479M Total, 175M MFU, 161M MRU, 915K Anon, 25M Header, 117M Other > > Tue Oct 8 03:08:13 CEST 2013 > Mem: 94M Active, 116M Inact, 1387M Wired, 552K Cache, 353M Free > ARC: 506M Total, 90M MFU, 231M MRU, 912K Anon, 26M Header, 158M Other > > Tue Oct 8 03:08:43 CEST 2013 > Mem: 96M Active, 117M Inact, 1351M Wired, 552K Cache, 386M Free > ARC: 436M Total, 52M MFU, 231M MRU, 1322K Anon, 23M Header, 129M Other > > Tue Oct 8 03:09:13 CEST 2013 > Mem: 95M Active, 117M Inact, 1314M Wired, 552K Cache, 423M Free > ARC: 440M Total, 73M MFU, 173M MRU, 941K Anon, 23M Header, 170M Other > > Tue Oct 8 03:09:43 CEST 2013 > Mem: 94M Active, 117M Inact, 1327M Wired, 552K Cache, 412M Free > ARC: 510M Total, 80M MFU, 182M MRU, 912K Anon, 23M Header, 223M Other > > Tue Oct 8 03:10:13 CEST 2013 > Mem: 94M Active, 117M Inact, 1276M Wired, 552K Cache, 463M Free > ARC: 578M Total, 65M MFU, 150M MRU, 912K Anon, 27M Header, 336M Other > > Tue Oct 8 03:10:43 CEST 2013 > Mem: 95M Active, 117M Inact, 1262M Wired, 552K Cache, 476M Free > ARC: 585M Total, 56M MFU, 144M MRU, 913K Anon, 26M Header, 358M Other > > Tue Oct 8 03:11:13 CEST 2013 > Mem: 103M Active, 114M Inact, 1269M Wired, 552K Cache, 464M Free > ARC: 590M Total, 66M MFU, 140M MRU, 912K Anon, 26M Header, 357M Other > > Tue Oct 8 03:11:44 CEST 2013 > Mem: 103M Active, 114M Inact, 1294M Wired, 552K Cache, 439M Free > ARC: 667M Total, 70M MFU, 155M MRU, 1056K Anon, 27M Header, 415M Other > > Tue Oct 8 03:12:14 CEST 2013 > Mem: 103M Active, 114M Inact, 1389M Wired, 552K Cache, 343M Free > ARC: 792M Total, 73M MFU, 185M MRU, 1040K Anon, 27M Header, 507M Other > > Tue Oct 8 03:12:44 CEST 2013 > Mem: 94M Active, 119M Inact, 1472M Wired, 552K Cache, 265M Free > ARC: 891M Total, 82M MFU, 212M MRU, 1056K Anon, 27M Header, 568M Other > > Tue Oct 8 03:13:15 CEST 2013 > Mem: 94M Active, 119M Inact, 1538M Wired, 544K Cache, 199M Free > ARC: 951M Total, 85M MFU, 220M MRU, 928K Anon, 28M Header, 618M Other > > Tue Oct 8 03:13:45 CEST 2013 > Mem: 136M Active, 22M Inact, 1682M Wired, 46M Cache, 65M Free > ARC: 1113M Total, 90M MFU, 259M MRU, 912K Anon, 26M Header, 738M Other > > Tue Oct 8 03:14:15 CEST 2013 > Mem: 153M Active, 3936K Inact, 1736M Wired, 36M Cache, 21M Free > ARC: 1159M Total, 95M MFU, 271M MRU, 1040K Anon, 25M Header, 767M Other > > Tue Oct 8 03:14:46 CEST 2013 > Mem: 62M Active, 15M Inact, 1808M Wired, 35M Cache, 30M Free > ARC: 1213M Total, 80M MFU, 294M MRU, 819K Anon, 19M Header, 819M Other > > Tue Oct 8 03:15:17 CEST 2013 > Mem: 61M Active, 6488K Inact, 1816M Wired, 33M Cache, 34M Free > ARC: 1194M Total, 73M MFU, 293M MRU, 1040K Anon, 19M Header, 808M Other > > Tue Oct 8 03:15:47 CEST 2013 > Mem: 75M Active, 2548K Inact, 1808M Wired, 25M Cache, 41M Free > ARC: 1189M Total, 72M MFU, 292M MRU, 1475K Anon, 19M Header, 804M Other > > Tue Oct 8 03:16:17 CEST 2013 > Mem: 75M Active, 2928K Inact, 1806M Wired, 24M Cache, 43M Free > ARC: 1183M Total, 72M MFU, 291M MRU, 912K Anon, 18M Header, 801M Other > > Tue Oct 8 03:16:47 CEST 2013 > Mem: 64M Active, 14M Inact, 1805M Wired, 24M Cache, 44M Free > ARC: 1179M Total, 71M MFU, 290M MRU, 912K Anon, 18M Header, 798M Other > > Tue Oct 8 03:17:17 CEST 2013 > Mem: 21M Active, 57M Inact, 1802M Wired, 24M Cache, 47M Free > ARC: 1174M Total, 71M MFU, 289M MRU, 912K Anon, 18M Header, 795M Other > > Tue Oct 8 03:17:48 CEST 2013 > Mem: 16M Active, 61M Inact, 1801M Wired, 24M Cache, 49M Free > ARC: 1170M Total, 70M MFU, 288M MRU, 912K Anon, 18M Header, 792M Other > > Tue Oct 8 03:18:18 CEST 2013 > Mem: 16M Active, 61M Inact, 1800M Wired, 24M Cache, 50M Free > ARC: 1165M Total, 70M MFU, 287M MRU, 912K Anon, 18M Header, 789M Other > > Tue Oct 8 03:18:48 CEST 2013 > Mem: 17M Active, 62M Inact, 1799M Wired, 23M Cache, 50M Free > ARC: 1162M Total, 70M MFU, 286M MRU, 912K Anon, 18M Header, 787M Other > > Tue Oct 8 03:19:18 CEST 2013 > Mem: 14M Active, 57M Inact, 1797M Wired, 23M Cache, 60M Free > ARC: 1157M Total, 69M MFU, 285M MRU, 912K Anon, 18M Header, 784M Other > > Tue Oct 8 03:19:48 CEST 2013 > Mem: 14M Active, 57M Inact, 1796M Wired, 23M Cache, 61M Free > ARC: 1153M Total, 69M MFU, 284M MRU, 912K Anon, 18M Header, 781M Other > > Tue Oct 8 03:20:18 CEST 2013 > Mem: 15M Active, 56M Inact, 1794M Wired, 22M Cache, 63M Free > ARC: 1148M Total, 69M MFU, 283M MRU, 912K Anon, 18M Header, 777M Other > > Tue Oct 8 03:20:48 CEST 2013 > Mem: 14M Active, 56M Inact, 1793M Wired, 22M Cache, 65M Free > ARC: 1145M Total, 69M MFU, 282M MRU, 912K Anon, 18M Header, 775M Other > > -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Tue Jan 21 14:46:11 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AE620155 for ; Tue, 21 Jan 2014 14:46:11 +0000 (UTC) Received: from mail-we0-x22b.google.com (mail-we0-x22b.google.com [IPv6:2a00:1450:400c:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3DDFC14B1 for ; Tue, 21 Jan 2014 14:46:11 +0000 (UTC) Received: by mail-we0-f171.google.com with SMTP id w61so8006941wes.2 for ; Tue, 21 Jan 2014 06:46:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=w9i2en67LzIZgEVjRNP6bS+ikEypix3KruuvEoHH/+M=; b=MQ6L8zsIbzxoNdY88IJ5+NOEXGHFILanFCrJXwldxAJ9FjM7OUbApD1GgX9bJNHDVy ia+k3RqnDmNPPa7C0PxSyj0sGd8LWYBqk++KwmxNsokx74wmNVWUN4pqbs/1fuC4UkuB w00UE0zbYR5popVug9D4XXkBjzPW3q/Lv2OmHkMoXutTtpOCRNP18pEpczf7L1MsDKsU jyJcg5P/RqHC71tDrXcgCDNV5Hkzn9tXzanzpF1Djm8xApY/5L8On4hwmd63wHLianal 0Ywkqw0duO4tSswJ2yPfyo94sy9DGNkBqbarCEMD03igWUEzN8veJs3kA8cesixi4CSH 3Kfw== MIME-Version: 1.0 X-Received: by 10.180.97.71 with SMTP id dy7mr3311557wib.35.1390315569611; Tue, 21 Jan 2014 06:46:09 -0800 (PST) Received: by 10.216.18.129 with HTTP; Tue, 21 Jan 2014 06:46:09 -0800 (PST) Date: Tue, 21 Jan 2014 15:46:09 +0100 Message-ID: Subject: hast and zfs From: Joar Jegleim To: "freebsd-fs@freebsd.org" Content-Type: multipart/mixed; boundary=f46d04428fd0a7cdf204f07c1106 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jan 2014 14:46:11 -0000 --f46d04428fd0a7cdf204f07c1106 Content-Type: text/plain; charset=UTF-8 Hi list ! I've setup hast with zfs on 9.1-RELEASE-p10 I've got an lsi hba connected to an external sas hp msa that has 20 disks in it, same setup on both nodes. I've created a hast disk dev such as /dev/hast/disk1a, /dev/hast/disk1b etc... ( see attached hast.conf ) And built a zpool mirror accross the 20 disks. While setting up this, and testing, all looked pretty good, I've tried numerous failovers and so on. When I just switched over to this setup in production, it worked for about half an hour until the primary node got a zfs hang where zpool list hung and hastctl status hung completely . There was no output in dmesg nor /var/log/messages that said anything related to this (?) . The server serves some 3 000 000 jpegs for a site, I see anything from 50 to just above 1000 iops, though the average iops is bellow 100. It's completely random what pictures are being fetched at any time. Could there be some sysctls to tune related to this setup mabye ? Anybody using hast and zfs in production got any tip ? -- ---------------------- Joar Jegleim Homepage: http://cosmicb.no Linkedin: http://no.linkedin.com/in/joarjegleim fb: http://www.facebook.com/joar.jegleim AKA: CosmicB @Freenode ---------------------- --f46d04428fd0a7cdf204f07c1106 Content-Type: application/octet-stream; name="hast.conf" Content-Disposition: attachment; filename="hast.conf" Content-Transfer-Encoding: base64 X-Attachment-Id: f_hqp9zo6g0 Y2hlY2tzdW0gc2hhMjU2CnJlcGxpY2F0aW9uIGZ1bGxzeW5jCmxpc3RlbiAxOTIuMTY4Ljc3LjEx CiN0aW1lb3V0IDUKCnJlc291cmNlIGRpc2sxYSB7CiAgICAgICAgb24gc3RvcmUwMSB7CiAgICAg ICAgICAgICAgICBsb2NhbCAvZGV2L2RhMgogICAgICAgICAgICAgICAgcmVtb3RlIHN0b3JlMDIK ICAgICAgICB9CiAgICAgICAgb24gIHN0b3JlMDIgewogICAgICAgICAgICAgICAgbG9jYWwgL2Rl di9kYTIKICAgICAgICAgICAgICAgIHJlbW90ZSBzdG9yZTAxCiAgICAgICAgfQp9CgpyZXNvdXJj ZSBkaXNrMWIgewogICAgICAgIG9uIHN0b3JlMDEgewogICAgICAgICAgICAgICAgbG9jYWwgL2Rl di9kYTMKICAgICAgICAgICAgICAgIHJlbW90ZSBzdG9yZTAyCiAgICAgICAgfQogICAgICAgIG9u ICBzdG9yZTAyIHsKICAgICAgICAgICAgICAgIGxvY2FsIC9kZXYvZGEzCiAgICAgICAgICAgICAg ICByZW1vdGUgc3RvcmUwMQogICAgICAgIH0KfQoKcmVzb3VyY2UgZGlzazJhIHsKICAgICAgICBv biBzdG9yZTAxIHsKICAgICAgICAgICAgICAgIGxvY2FsIC9kZXYvZGE0CiAgICAgICAgICAgICAg ICByZW1vdGUgc3RvcmUwMgogICAgICAgIH0KICAgICAgICBvbiAgc3RvcmUwMiB7CiAgICAgICAg ICAgICAgICBsb2NhbCAvZGV2L2RhNAogICAgICAgICAgICAgICAgcmVtb3RlIHN0b3JlMDEKICAg ICAgICB9Cn0KCnJlc291cmNlIGRpc2syYiB7CiAgICAgICAgb24gc3RvcmUwMSB7CiAgICAgICAg ICAgICAgICBsb2NhbCAvZGV2L2RhNQogICAgICAgICAgICAgICAgcmVtb3RlIHN0b3JlMDIKICAg ICAgICB9CiAgICAgICAgb24gIHN0b3JlMDIgewogICAgICAgICAgICAgICAgbG9jYWwgL2Rldi9k YTUKICAgICAgICAgICAgICAgIHJlbW90ZSBzdG9yZTAxCiAgICAgICAgfQp9CgpyZXNvdXJjZSBk aXNrM2EgewogICAgICAgIG9uIHN0b3JlMDEgewogICAgICAgICAgICAgICAgbG9jYWwgL2Rldi9k YTYKICAgICAgICAgICAgICAgIHJlbW90ZSBzdG9yZTAyCiAgICAgICAgfQogICAgICAgIG9uICBz dG9yZTAyIHsKICAgICAgICAgICAgICAgIGxvY2FsIC9kZXYvZGE2CiAgICAgICAgICAgICAgICBy ZW1vdGUgc3RvcmUwMQogICAgICAgIH0KfQoKcmVzb3VyY2UgZGlzazNiIHsKICAgICAgICBvbiBz dG9yZTAxIHsKICAgICAgICAgICAgICAgIGxvY2FsIC9kZXYvZGE3CiAgICAgICAgICAgICAgICBy ZW1vdGUgc3RvcmUwMgogICAgICAgIH0KICAgICAgICBvbiAgc3RvcmUwMiB7CiAgICAgICAgICAg ICAgICBsb2NhbCAvZGV2L2RhNwogICAgICAgICAgICAgICAgcmVtb3RlIHN0b3JlMDEKICAgICAg ICB9Cn0KCnJlc291cmNlIGRpc2s0YSB7CiAgICAgICAgb24gc3RvcmUwMSB7CiAgICAgICAgICAg ICAgICBsb2NhbCAvZGV2L2RhOAogICAgICAgICAgICAgICAgcmVtb3RlIHN0b3JlMDIKICAgICAg ICB9CiAgICAgICAgb24gIHN0b3JlMDIgewogICAgICAgICAgICAgICAgbG9jYWwgL2Rldi9kYTgK ICAgICAgICAgICAgICAgIHJlbW90ZSBzdG9yZTAxCiAgICAgICAgfQp9CgpyZXNvdXJjZSBkaXNr NGIgewogICAgICAgIG9uIHN0b3JlMDEgewogICAgICAgICAgICAgICAgbG9jYWwgL2Rldi9kYTkK ICAgICAgICAgICAgICAgIHJlbW90ZSBzdG9yZTAyCiAgICAgICAgfQogICAgICAgIG9uICBzdG9y ZTAyIHsKICAgICAgICAgICAgICAgIGxvY2FsIC9kZXYvZGE5CiAgICAgICAgICAgICAgICByZW1v dGUgc3RvcmUwMQogICAgICAgIH0KfQoKcmVzb3VyY2UgZGlzazVhIHsKICAgICAgICBvbiBzdG9y ZTAxIHsKICAgICAgICAgICAgICAgIGxvY2FsIC9kZXYvZGExMAogICAgICAgICAgICAgICAgcmVt b3RlIHN0b3JlMDIKICAgICAgICB9CiAgICAgICAgb24gIHN0b3JlMDIgewogICAgICAgICAgICAg ICAgbG9jYWwgL2Rldi9kYTEwCiAgICAgICAgICAgICAgICByZW1vdGUgc3RvcmUwMQogICAgICAg IH0KfQoKcmVzb3VyY2UgZGlzazViIHsKICAgICAgICBvbiBzdG9yZTAxIHsKICAgICAgICAgICAg ICAgIGxvY2FsIC9kZXYvZGExMQogICAgICAgICAgICAgICAgcmVtb3RlIHN0b3JlMDIKICAgICAg ICB9CiAgICAgICAgb24gIHN0b3JlMDIgewogICAgICAgICAgICAgICAgbG9jYWwgL2Rldi9kYTEx CiAgICAgICAgICAgICAgICByZW1vdGUgc3RvcmUwMQogICAgICAgIH0KfQoKcmVzb3VyY2UgZGlz azZhIHsKICAgICAgICBvbiBzdG9yZTAxIHsKICAgICAgICAgICAgICAgIGxvY2FsIC9kZXYvZGEx MgogICAgICAgICAgICAgICAgcmVtb3RlIHN0b3JlMDIKICAgICAgICB9CiAgICAgICAgb24gIHN0 b3JlMDIgewogICAgICAgICAgICAgICAgbG9jYWwgL2Rldi9kYTEyCiAgICAgICAgICAgICAgICBy ZW1vdGUgc3RvcmUwMQogICAgICAgIH0KfQoKcmVzb3VyY2UgZGlzazZiIHsKICAgICAgICBvbiBz dG9yZTAxIHsKICAgICAgICAgICAgICAgIGxvY2FsIC9kZXYvZGExMwogICAgICAgICAgICAgICAg cmVtb3RlIHN0b3JlMDIKICAgICAgICB9CiAgICAgICAgb24gIHN0b3JlMDIgewogICAgICAgICAg ICAgICAgbG9jYWwgL2Rldi9kYTEzCiAgICAgICAgICAgICAgICByZW1vdGUgc3RvcmUwMQogICAg ICAgIH0KfQoKcmVzb3VyY2UgZGlzazdhIHsKICAgICAgICBvbiBzdG9yZTAxIHsKICAgICAgICAg ICAgICAgIGxvY2FsIC9kZXYvZGExNAogICAgICAgICAgICAgICAgcmVtb3RlIHN0b3JlMDIKICAg ICAgICB9CiAgICAgICAgb24gIHN0b3JlMDIgewogICAgICAgICAgICAgICAgbG9jYWwgL2Rldi9k YTE0CiAgICAgICAgICAgICAgICByZW1vdGUgc3RvcmUwMQogICAgICAgIH0KfQoKcmVzb3VyY2Ug ZGlzazdiIHsKICAgICAgICBvbiBzdG9yZTAxIHsKICAgICAgICAgICAgICAgIGxvY2FsIC9kZXYv ZGExNQogICAgICAgICAgICAgICAgcmVtb3RlIHN0b3JlMDIKICAgICAgICB9CiAgICAgICAgb24g IHN0b3JlMDIgewogICAgICAgICAgICAgICAgbG9jYWwgL2Rldi9kYTE1CiAgICAgICAgICAgICAg ICByZW1vdGUgc3RvcmUwMQogICAgICAgIH0KfQoKcmVzb3VyY2UgZGlzazhhIHsKICAgICAgICBv biBzdG9yZTAxIHsKICAgICAgICAgICAgICAgIGxvY2FsIC9kZXYvZGExNgogICAgICAgICAgICAg ICAgcmVtb3RlIHN0b3JlMDIKICAgICAgICB9CiAgICAgICAgb24gIHN0b3JlMDIgewogICAgICAg ICAgICAgICAgbG9jYWwgL2Rldi9kYTE2CiAgICAgICAgICAgICAgICByZW1vdGUgc3RvcmUwMQog ICAgICAgIH0KfQoKcmVzb3VyY2UgZGlzazhiIHsKICAgICAgICBvbiBzdG9yZTAxIHsKICAgICAg ICAgICAgICAgIGxvY2FsIC9kZXYvZGExNwogICAgICAgICAgICAgICAgcmVtb3RlIHN0b3JlMDIK ICAgICAgICB9CiAgICAgICAgb24gIHN0b3JlMDIgewogICAgICAgICAgICAgICAgbG9jYWwgL2Rl di9kYTE3CiAgICAgICAgICAgICAgICByZW1vdGUgc3RvcmUwMQogICAgICAgIH0KfQoKcmVzb3Vy Y2UgZGlzazlhIHsKICAgICAgICBvbiBzdG9yZTAxIHsKICAgICAgICAgICAgICAgIGxvY2FsIC9k ZXYvZGExOAogICAgICAgICAgICAgICAgcmVtb3RlIHN0b3JlMDIKICAgICAgICB9CiAgICAgICAg b24gIHN0b3JlMDIgewogICAgICAgICAgICAgICAgbG9jYWwgL2Rldi9kYTE4CiAgICAgICAgICAg ICAgICByZW1vdGUgc3RvcmUwMQogICAgICAgIH0KfQoKcmVzb3VyY2UgZGlzazliIHsKICAgICAg ICBvbiBzdG9yZTAxIHsKICAgICAgICAgICAgICAgIGxvY2FsIC9kZXYvZGExOQogICAgICAgICAg ICAgICAgcmVtb3RlIHN0b3JlMDIKICAgICAgICB9CiAgICAgICAgb24gIHN0b3JlMDIgewogICAg ICAgICAgICAgICAgbG9jYWwgL2Rldi9kYTE5CiAgICAgICAgICAgICAgICByZW1vdGUgc3RvcmUw MQogICAgICAgIH0KfQoKcmVzb3VyY2UgZGlzazEwYSB7CiAgICAgICAgb24gc3RvcmUwMSB7CiAg ICAgICAgICAgICAgICBsb2NhbCAvZGV2L2RhMjAKICAgICAgICAgICAgICAgIHJlbW90ZSBzdG9y ZTAyCiAgICAgICAgfQogICAgICAgIG9uICBzdG9yZTAyIHsKICAgICAgICAgICAgICAgIGxvY2Fs IC9kZXYvZGEyMAogICAgICAgICAgICAgICAgcmVtb3RlIHN0b3JlMDEKICAgICAgICB9Cn0KCnJl c291cmNlIGRpc2sxMGIgewogICAgICAgIG9uIHN0b3JlMDEgewogICAgICAgICAgICAgICAgbG9j YWwgL2Rldi9kYTIxCiAgICAgICAgICAgICAgICByZW1vdGUgc3RvcmUwMgogICAgICAgIH0KICAg ICAgICBvbiAgc3RvcmUwMiB7CiAgICAgICAgICAgICAgICBsb2NhbCAvZGV2L2RhMjEKICAgICAg ICAgICAgICAgIHJlbW90ZSBzdG9yZTAxCiAgICAgICAgfQp9CgoK --f46d04428fd0a7cdf204f07c1106-- From owner-freebsd-fs@FreeBSD.ORG Tue Jan 21 17:17:27 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3EFEC352 for ; Tue, 21 Jan 2014 17:17:27 +0000 (UTC) Received: from ns1.jnielsen.net (secure.freebsdsolutions.net [69.55.234.48]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 20E971316 for ; Tue, 21 Jan 2014 17:17:26 +0000 (UTC) Received: from [10.10.1.198] (office.betterlinux.com [199.58.199.60]) (authenticated bits=0) by ns1.jnielsen.net (8.14.4/8.14.4) with ESMTP id s0LH4EuW099242 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT); Tue, 21 Jan 2014 12:04:15 -0500 (EST) (envelope-from lists@jnielsen.net) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: Filesystem errors with VirtIO From: John Nielsen In-Reply-To: <20140120163951.7fe4c1f8@calavera> Date: Tue, 21 Jan 2014 10:04:10 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <8B7FDAA2-D987-4824-9826-B975552194F8@jnielsen.net> References: <20140120163951.7fe4c1f8@calavera> To: Philipp X-Mailer: Apple Mail (2.1827) X-DCC--Metrics: ns1.jnielsen.net 1282; Body=2 Fuz1=2 Fuz2=2 X-Virus-Scanned: clamav-milter 0.97.8 at ns1.jnielsen.net X-Virus-Status: Clean Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jan 2014 17:17:27 -0000 On Jan 20, 2014, at 8:39 AM, Philipp = wrote: > I'm running FreeBSD 10.0-RELEASE on a virtualized KVM server. I'm > using VirtIO from base to access NIC and HDD directly. Filesystem is > UFS2 with Journaling and SU. Last week I had to reinstall FreeBSD > because the salvation of the several filesystem errors wasn't possible > anymore. I don't have any evidence but I guess it has something to do > with the VirtIO driver. >=20 > After running the machine for 48 hours with a freshly installed > FreeBSD 10 several soft update inconsistencies occur: > [fsck output trimmed] >=20 > After a reboot different inconsistencies appear. I had the very same > problems with 9.2-RELEASE, which caused the need of a reinstallation > of the OS (errors like "vtbd0: hard error cmd=3Dwrite fsbn 24162" > occured). >=20 > Can someone confirm these problems with VirtIO? What else can I do? FWIW I've been running multiple FreeBSD VMs atop KVM+virtio for some = time and haven't seen any issues like this. My VMs include FreeBSD 9.2 = and 10.0-RC on both UFS and ZFS. What can you tell us about the underlying storage? LVM volume? iSCSI? = Qcow2 or other disk image file? If so, what type of filesystem is it on? = etc. JN From owner-freebsd-fs@FreeBSD.ORG Wed Jan 22 01:42:49 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 607FCF3 for ; Wed, 22 Jan 2014 01:42:49 +0000 (UTC) Received: from unit0.ironport.snap.net.nz (unit0.ironport.snap.net.nz [202.37.100.104]) by mx1.freebsd.org (Postfix) with ESMTP id C04DA1EF1 for ; Wed, 22 Jan 2014 01:42:48 +0000 (UTC) X-Cloudmark-SP-Filtered: true X-Cloudmark-SP-Result: v=1.1 cv=URPCDH+C/Vd6YCmNGulkvdqfk8JR2pQf21upMj5D9os= c=1 sm=2 a=BQmAyTJ5I0kA:10 a=NjuLKI5jG4EA:10 a=6I5d2MoRAAAA:8 a=-2jejKkGW-9fZRYWB1IA:9 a=wPNLvfGTeEIA:10 a=SV7veod9ZcQA:10 a=CqWZkgPtQDeKha5iNpUA:9 a=ZVk8-NSrHBgA:10 a=TEJWfmVIHgEA:10 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgAFAE4g31J7/y9j/2dsb2JhbABahBm7boEUFoMZAQEBAwFAAQE3AQQLCxEDAQIdEgJNCBmHfQelSYRSAQWYPxEGjhRQChEHgySBFJA/h2eVW4IU X-IronPort-AV: E=Sophos;i="4.95,697,1384254000"; d="p7s'?scan'208";a="229453237" Received: from rupert.snap.net.nz ([202.37.100.140]) by smtp0.ironport.snap.net.nz with ESMTP; 22 Jan 2014 14:37:39 +1300 X-Sender-IP: 123.255.47.99 X-Sender-IP: 123.255.47.99 X-Sender-IP: 123.255.47.99 Received: from x24.msqr.us (99.47.255.123.static.snap.net.nz [123.255.47.99]) by rupert.snap.net.nz (Postfix) with ESMTPS id 4CF902562D; Wed, 22 Jan 2014 14:37:39 +1300 (NZDT) Received: from renton.fritz.box (99.47.255.123.static.snap.net.nz [123.255.47.99]) (authenticated bits=0) by x24.msqr.us (8.14.7/8.14.5) with ESMTP id s0M1bWKQ034508 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Wed, 22 Jan 2014 14:37:35 +1300 (NZDT) (envelope-from matt@msqr.us) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=msqr.us; s=20121026; t=1390354657; bh=NHHUfW0xZpD0NAFDQozK1La8Axpd64CeAhP26Ss1NbU=; h=Subject:From:In-Reply-To:Date:Cc:References:To; b=s7Jic+pH745rhT7RB6YFzSMV4bQBqMeee3YeOCHAII7DufkXBtW/lcGwg82zYf5z6 hPSEiRffKxZlxBjot/M5ZBsGD6eoS/1Ptj5zfIxr5urFKUGopjuwLyZ48FMd25V3ib BQiZ97FMxQVMDaCfuLk2LumuQ9Ztd8+ecwUxXM/M= X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.98.1 at msqr.us X-Authentication-Warning: x24.msqr.us: Host 99.47.255.123.static.snap.net.nz [123.255.47.99] claimed to be renton.fritz.box Content-Type: multipart/signed; boundary="Apple-Mail=_B579718B-9CB4-4CA9-A5EE-28F8C6E90EAB"; protocol="application/pkcs7-signature"; micalg=sha1 Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Subject: Re: ZFS snapshot renames failing after upgrade to 9.2 From: Matt Magoffin In-Reply-To: Date: Wed, 22 Jan 2014 14:37:32 +1300 Message-Id: References: <0C9FD4E1-0549-4849-BFC5-D8C5D4A34D64@msqr.us> <54D3B3C002184A52BEC9B1543854B87F@multiplay.co.uk> <333D57C6A4544067880D9CFC04F02312@multiplay.co.uk> <26053_1387447492_52B2C4C4_26053_331_1_20131219105503.3a8d1df3.gerrit.kuehn@aei.mpg.de> <20131219165549.9f2ca709.gerrit.kuehn@aei.mpg.de> <20131219174054.91ac617a.gerrit.kuehn@aei.mpg.de> <20131220100522.382a39ac.gerrit.kuehn@aei.mpg.de> To: krichy@tvnetwork.hu X-Mailer: Apple Mail (2.1827) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jan 2014 01:42:49 -0000 --Apple-Mail=_B579718B-9CB4-4CA9-A5EE-28F8C6E90EAB Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 Hello, I missed the continuation of this thread, but wanted to add in that this = still happens for me (snapshot renames lead to unusable snapshots). I recently updated to 9.2-p3, and after restarting the system my scripts = were able to rename the existing snapshots as expected, but only one = time. When my backup script attempted to rename the new snapshot and the = older ones during its second run since the restart, it failed and ended = up in the unusable state. As for 9.1, I never had this problem in 9.1. It only started happening = for me after updating to 9.2. Kind regards, Matt=20 On Dec 21, 2013, at 5:18 AM, krichy@tvnetwork.hu wrote: > Dear Gerrit, >=20 > The problem is that when renaming snapshots, the mounted snapshot is = tried to be mounted again under a new name. Maybe, the rename code does = not remount them to their proper position, that may be the problem. >=20 > I will look if I can check or fix it. >=20 > Did this work in 9.1? >=20 > Regards, >=20 >=20 > Kojedzinszky Richard > Euronet Magyarorszag Informatikai Zrt. >=20 >>> Date: Fri, 20 Dec 2013 10:05:22 +0100 >>> From: Gerrit K=FChn >>> To: krichy@tvnetwork.hu >>> Cc: freebsd-fs@freebsd.org >>> Subject: Re: ZFS snapshot renames failing after upgrade to 9.2 >>> On Thu, 19 Dec 2013 20:08:22 +0100 (CET) krichy@tvnetwork.hu wrote = about >>> Re: ZFS snapshot renames failing after upgrade to 9.2: >>> KH> So a simple renaming can cause your system to hang? >>> No, it does not hang completely. >>> Just the snapshots become unusable. This night, it happened again: >>> --- >>> root@shapeshifter:~ # ll /tank/git/.zfs/snapshot/ >>> ls: daily.6: Device busy >>> total 33 >>> drwxr-xr-x 12 211 211 25 Dec 19 09:18 daily.0/ >>> drwxr-xr-x 12 211 211 25 Dec 19 00:00 daily.1/ >>> drwxr-xr-x 12 211 211 24 Dec 18 00:00 daily.2/ >>> drwxr-xr-x 12 211 211 24 Dec 17 00:00 daily.3/ >>> drwxr-xr-x 12 211 211 24 Dec 16 00:00 daily.4/ >>> drwxr-xr-x 12 211 211 24 Dec 14 00:00 daily.5/ >>> drwxr-xr-x 12 211 211 24 Dec 15 00:00 weekly.0/ >>> drwxr-xr-x 12 211 211 24 Dec 8 00:00 weekly.1/ >>> drwxr-xr-x 12 211 211 24 Dec 1 00:00 weekly.2/ >>> drwxr-xr-x 12 211 211 24 Nov 17 00:00 weekly.3/ >>> drwxr-xr-x 12 211 211 24 Nov 10 00:00 weekly.4/ >>> drwxr-xr-x 2 root wheel 3 Oct 20 00:00 weekly.5/ >>> drwxr-xr-x 2 root wheel 3 Oct 6 00:00 weekly.6/ >>> --- >>> root@shapeshifter:~ # zfs list -r -t snapshot -o >>> name,creation,used,referenced tank/git NAME >>> CREATION USED REFER tank/git@weekly.6 Sun Oct 6 = 0:00 >>> 2013 42.6K 62.8K tank/git@weekly.5 Sun Oct 20 0:00 2013 42.6K = 62.8K >>> tank/git@weekly.4 Sun Nov 10 0:00 2013 29.5M 146G >>> tank/git@weekly.3 Sun Nov 17 0:00 2013 27.1M 146G >>> tank/git@weekly.2 Sun Dec 1 0:00 2013 26.3M 146G >>> tank/git@weekly.1 Sun Dec 8 0:00 2013 27.3M 146G >>> tank/git@daily.6 Sat Dec 14 0:00 2013 26.5M 147G >>> tank/git@weekly.0 Sun Dec 15 0:00 2013 25.2M 147G >>> tank/git@daily.5 Mon Dec 16 0:00 2013 24.7M 147G >>> tank/git@daily.4 Tue Dec 17 0:00 2013 24.9M 147G >>> tank/git@daily.3 Wed Dec 18 0:00 2013 25.7M 147G >>> tank/git@daily.2 Thu Dec 19 0:00 2013 25.8M 147G >>> tank/git@daily.1 Thu Dec 19 9:19 2013 25.0M 147G >>> tank/git@daily.0 Fri Dec 20 0:00 2013 26.8M 147G >>> --- >>> As you can see, the snapshot rotating got stuck somewhere. What is >>> displayed under .zfs/snapshot does not reflect what zfs is really = seeing: >>> daily.6 is inaccessible, and the rotation that happened so far is = not >>> reflected under .zfs/snapshot, either. >>> cu >>> Gerrit --Apple-Mail=_B579718B-9CB4-4CA9-A5EE-28F8C6E90EAB Content-Disposition: attachment; filename=smime.p7s Content-Type: application/pkcs7-signature; name=smime.p7s Content-Transfer-Encoding: base64 MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIDgTCCA30w ggJloAMCAQICAQIwCwYJKoZIhvcNAQELMGsxEDAOBgNVBAMMB21zcXIgQ0ExDTALBgNVBAoMBG1z cXIxIDAeBgNVBAsMF0NlcnRpZmljYXRpb24gQXV0aG9yaXR5MQswCQYDVQQGEwJVUzEZMBcGCSqG SIb3DQEJARYKY2FAbXNxci51czAeFw0xMzA3MDcyMTI1MTJaFw0yMzA3MDUyMTI1MTJaMHMxFjAU BgNVBAMMDU1hdHQgTWFnb2ZmaW4xDTALBgNVBAoMBG1zcXIxIDAeBgNVBAsMF0NlcnRpZmljYXRp b24gQXV0aG9yaXR5MQswCQYDVQQGEwJVUzEbMBkGCSqGSIb3DQEJARYMbWF0dEBtc3FyLnVzMIIB IjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA80JsC5zGa1iuzGVegrbX8kpfIpqTsG4yaTGP 2XwgLpsyGzn0tule6d/6qa3OJRUR4pY8qCmFs2yxQoVfJvQNRfQDkBO/T6gyx5ml2CTppyCQiiRe Ki0BpD2xQB7SJOoKLTEoveWzAdC8sMT+WxYeS8R1i3HFtoth0Ll0u6hP6Ufp0XLOVXdW2jr0l4aY gn1KSM7KmFmLXDpOFlempaxUTXWLaXjGGDjRi5Ev1a1OTHyfPiA4YmC4/y3Npq2bl88/gxGw7u2Y zYshKZ0y24o5HxxmoxpE0EGl7Ub7+2osp54SKurKdFo/9wlNNIEPATDVoAT2NPDIOWBtZ9LUvYvH SQIDAQABoyYwJDAOBgNVHQ8BAf8EBAMCAf4wEgYDVR0lAQH/BAgwBgYEVR0lADANBgkqhkiG9w0B AQsFAAOCAQEAfuWLLZL5kryMYHEY0EE4cocnBTzXXkhDHUpqoWZvd1ivZpVTBizHIuVyCSUO7PL0 nqK5vlS0ldmyLgE8oaDr1yf/swQ6/yRoOH/4FrH/+bHsBtrtyCzWCPKsRfLY3M/pb02UaRt2lL2v H26I1VB7CTd5z9XFcQi4ZKxANPJE2oXqlWwCABmLpJjPzd6JpNIolMMEQL3cpohfHsSg7uKw6Jqk u7O11j9CBBb/YgGddGC95L7sRiSXlxSMRkjoXREngp2U7pWwh8EcFGo5FnVxuAlWIyVU8DdcCYNL r/euF+clJ3J1ztgPKMzY44CDLVLeCiwKU9GOSDCNbCehQ0vLRDGCAv0wggL5AgEBMHAwazEQMA4G A1UEAwwHbXNxciBDQTENMAsGA1UECgwEbXNxcjEgMB4GA1UECwwXQ2VydGlmaWNhdGlvbiBBdXRo b3JpdHkxCzAJBgNVBAYTAlVTMRkwFwYJKoZIhvcNAQkBFgpjYUBtc3FyLnVzAgECMAkGBSsOAwIa BQCgggFiMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTE0MDEyMjAx MzczM1owIwYJKoZIhvcNAQkEMRYEFH/yzMAm+KYt32dIENrag13eCY/oMH8GCSsGAQQBgjcQBDFy MHAwazEQMA4GA1UEAwwHbXNxciBDQTENMAsGA1UECgwEbXNxcjEgMB4GA1UECwwXQ2VydGlmaWNh dGlvbiBBdXRob3JpdHkxCzAJBgNVBAYTAlVTMRkwFwYJKoZIhvcNAQkBFgpjYUBtc3FyLnVzAgEC MIGBBgsqhkiG9w0BCRACCzFyoHAwazEQMA4GA1UEAwwHbXNxciBDQTENMAsGA1UECgwEbXNxcjEg MB4GA1UECwwXQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkxCzAJBgNVBAYTAlVTMRkwFwYJKoZIhvcN AQkBFgpjYUBtc3FyLnVzAgECMA0GCSqGSIb3DQEBAQUABIIBAGgjVn2zc6g+Y8I24X1v/qDnPhal HMVPfXYvgnwjMDA1R7oBHgwzy2nU0pEQ+B4WYN553GGA0gkrwv4IHdr9tOiJF4lsMNLDCkfyLbh8 32vywsJ+veLb8BUwKPmNBTNjROxbZVhpBfRreO09RAidnQqbw9vFx4aRD3bWQuPo4rz174YD0qMX OlhGNMtV3qvxfxLY6T94lam2TBpgs44jVYqWlwMeKGd0DeLmrQVazat3/8SF13gu4mY0UolybgAn SVVZjnqsSyR/24PRNPTbF/azQA77HdV8xPnj2kubrd95HEyAxOsSOJYG4aXfpAbw1+EkPnT6I3wk pfELdEPgs1sAAAAAAAA= --Apple-Mail=_B579718B-9CB4-4CA9-A5EE-28F8C6E90EAB-- From owner-freebsd-fs@FreeBSD.ORG Wed Jan 22 08:47:48 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C030369F for ; Wed, 22 Jan 2014 08:47:48 +0000 (UTC) Received: from mail02.raumopol.de (mail02.raumopol.de [89.238.79.3]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 835C11F72 for ; Wed, 22 Jan 2014 08:47:47 +0000 (UTC) Received: from mail03.raumopol.de (mail03.localnet.raumopol.de [192.168.1.10]) by mail02.raumopol.de (Postfix) with ESMTP id D49AE214C26D; Wed, 22 Jan 2014 09:47:39 +0100 (CET) Received: from authenticated_user by mail03.raumopol.de (Postfix) with ESMTPSA id B54CB3832001; Wed, 22 Jan 2014 09:47:39 +0100 (CET) Date: Wed, 22 Jan 2014 09:47:38 +0100 From: Philipp To: John Nielsen , freebsd-fs@freebsd.org Subject: Re: Filesystem errors with VirtIO Message-ID: <20140122094738.7b1d782e@calavera> In-Reply-To: <8B7FDAA2-D987-4824-9826-B975552194F8@jnielsen.net> References: <20140120163951.7fe4c1f8@calavera> <8B7FDAA2-D987-4824-9826-B975552194F8@jnielsen.net> X-Mailer: Claws Mail 3.9.3 (GTK+ 2.24.22; amd64-portbld-freebsd10.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jan 2014 08:47:48 -0000 I have no information concerning the underlying hardware. Proxmox just outputs: Hard Disk (virtio0): gfs3:239/vm-239-disk-1.raw,format=raw,size=150G I have contacted the support of my VPS provider and they told me that the problem is known already, and that they are working on it. They didn't give out any further information. But now I can confirm that the problem is not caused by the VirtIO driver of FreeBSD. Regards, Philipp On Tue, 21 Jan 2014 10:04:10 -0700 John Nielsen wrote: > FWIW I've been running multiple FreeBSD VMs atop KVM+virtio for some > time and haven't seen any issues like this. My VMs include FreeBSD > 9.2 and 10.0-RC on both UFS and ZFS. > > What can you tell us about the underlying storage? LVM volume? iSCSI? > Qcow2 or other disk image file? If so, what type of filesystem is it > on? etc. > > JN > From owner-freebsd-fs@FreeBSD.ORG Wed Jan 22 08:53:01 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id AFDF978A for ; Wed, 22 Jan 2014 08:53:01 +0000 (UTC) Received: from umail.aei.mpg.de (umail.aei.mpg.de [194.94.224.6]) by mx1.freebsd.org (Postfix) with ESMTP id 420CE1FF5 for ; Wed, 22 Jan 2014 08:53:00 +0000 (UTC) Received: from mailgate.aei.mpg.de (mailgate.aei.mpg.de [194.94.224.5]) by umail.aei.mpg.de (Postfix) with ESMTP id 0C5DF201004; Wed, 22 Jan 2014 09:52:59 +0100 (CET) Received: from mailgate.aei.mpg.de (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id DA022405889; Wed, 22 Jan 2014 09:52:58 +0100 (CET) Received: from intranet.aei.uni-hannover.de (ahin1.aei.uni-hannover.de [130.75.117.40]) by mailgate.aei.mpg.de (Postfix) with ESMTP id 5DF32406AF1; Wed, 22 Jan 2014 09:52:58 +0100 (CET) Received: from cascade.aei.uni-hannover.de ([10.117.15.111]) by intranet.aei.uni-hannover.de (Lotus Domino Release 8.5.3FP6) with ESMTP id 2014012209524798-1390 ; Wed, 22 Jan 2014 09:52:47 +0100 Date: Wed, 22 Jan 2014 09:52:46 +0100 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: Matt Magoffin Subject: Re: ZFS snapshot renames failing after upgrade to 9.2 Message-Id: <20140122095246.65ea2196.gerrit.kuehn@aei.mpg.de> In-Reply-To: References: <0C9FD4E1-0549-4849-BFC5-D8C5D4A34D64@msqr.us> <54D3B3C002184A52BEC9B1543854B87F@multiplay.co.uk> <333D57C6A4544067880D9CFC04F02312@multiplay.co.uk> <26053_1387447492_52B2C4C4_26053_331_1_20131219105503.3a8d1df3.gerrit.kuehn@aei.mpg.de> <20131219165549.9f2ca709.gerrit.kuehn@aei.mpg.de> <20131219174054.91ac617a.gerrit.kuehn@aei.mpg.de> <20131220100522.382a39ac.gerrit.kuehn@aei.mpg.de> Organization: Max Planck Gesellschaft X-Mailer: Sylpheed 3.1.3 (GTK+ 2.24.19; amd64-portbld-freebsd8.2) Mime-Version: 1.0 X-MIMETrack: Itemize by SMTP Server on intranet/aei-hannover(Release 8.5.3FP6|November 21, 2013) at 01/22/2014 09:52:48, Serialize by Router on intranet/aei-hannover(Release 8.5.3FP6|November 21, 2013) at 01/22/2014 09:52:58, Serialize complete at 01/22/2014 09:52:58 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-PMX-Version: 6.0.2.2308539, Antispam-Engine: 2.7.2.2107409, Antispam-Data: 2014.1.22.83915 X-PerlMx-Spam: Gauge=IIIIIIIII, Probability=9%, Report=' MULTIPLE_RCPTS 0.1, HTML_00_01 0.05, HTML_00_10 0.05, MIME_LOWER_CASE 0.05, BODYTEXTP_SIZE_3000_LESS 0, BODY_SIZE_1300_1399 0, BODY_SIZE_2000_LESS 0, BODY_SIZE_5000_LESS 0, BODY_SIZE_7000_LESS 0, __ANY_URI 0, __BOUNCE_CHALLENGE_SUBJ 0, __BOUNCE_NDR_SUBJ_EXEMPT 0, __CT 0, __CTE 0, __CT_TEXT_PLAIN 0, __HAS_FROM 0, __HAS_MSGID 0, __HAS_X_MAILER 0, __IN_REP_TO 0, __MIME_TEXT_ONLY 0, __MIME_VERSION 0, __MULTIPLE_RCPTS_CC_X2 0, __SANE_MSGID 0, __SUBJ_ALPHA_NEGATE 0, __TO_MALFORMED_2 0, __URI_NO_PATH 0, __URI_NO_WWW 0, __URI_NS ' Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jan 2014 08:53:01 -0000 On Wed, 22 Jan 2014 14:37:32 +1300 Matt Magoffin wrote about Re: ZFS snapshot renames failing after upgrade to 9.2: MM> I missed the continuation of this thread, but wanted to add in that MM> this still happens for me (snapshot renames lead to unusable MM> snapshots). Well, I guess there was no continuation so far. On the 9.2-system under question, I changed the backup/snapshot scheme to use the dates in the names instead of doing logrotate stuff (using sysutils/zfs-periodic instead of sysutils/freebsd-snapshot) to circumvent the issue. MM> I recently updated to 9.2-p3, and after restarting the system my MM> scripts were able to rename the existing snapshots as expected, but MM> only one time. When my backup script attempted to rename the new MM> snapshot and the older ones during its second run since the restart, MM> it failed and ended up in the unusable state. I have the feeling that the issue depends on wether the snapshot was used before, is still mounted or something like that. I think when I had just the snapshots running (without the backup jobs), I did not see this (but I may be wrong with this). MM> As for 9.1, I never had this problem in 9.1. It only started happening MM> for me after updating to 9.2. Indeed, in 9.1 this was fine. Maybe we should file a proper bug report for this? cu Gerrit From owner-freebsd-fs@FreeBSD.ORG Wed Jan 22 08:55:07 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7DEB5863 for ; Wed, 22 Jan 2014 08:55:07 +0000 (UTC) Received: from umail.aei.mpg.de (umail.aei.mpg.de [194.94.224.6]) by mx1.freebsd.org (Postfix) with ESMTP id 27ED0102E for ; Wed, 22 Jan 2014 08:55:07 +0000 (UTC) Received: from mailgate.aei.mpg.de (mailgate.aei.mpg.de [194.94.224.5]) by umail.aei.mpg.de (Postfix) with ESMTP id 7B6BC200FFD; Wed, 22 Jan 2014 09:47:10 +0100 (CET) Received: from mailgate.aei.mpg.de (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id 675E940588A; Wed, 22 Jan 2014 09:47:10 +0100 (CET) Received: from intranet.aei.uni-hannover.de (ahin1.aei.uni-hannover.de [130.75.117.40]) by mailgate.aei.mpg.de (Postfix) with ESMTP id 7B02C405889; Wed, 22 Jan 2014 09:47:09 +0100 (CET) Received: from cascade.aei.uni-hannover.de ([10.117.15.111]) by intranet.aei.uni-hannover.de (Lotus Domino Release 8.5.3FP6) with ESMTP id 2014012209465864-1375 ; Wed, 22 Jan 2014 09:46:58 +0100 Date: Wed, 22 Jan 2014 09:46:58 +0100 From: Gerrit =?ISO-8859-1?Q?K=FChn?= To: krichy@tvnetwork.hu Subject: Re: ZFS snapshot renames failing after upgrade to 9.2 Message-Id: <20140122094658.ea83cda2.gerrit.kuehn@aei.mpg.de> In-Reply-To: References: <0C9FD4E1-0549-4849-BFC5-D8C5D4A34D64@msqr.us> <54D3B3C002184A52BEC9B1543854B87F@multiplay.co.uk> <333D57C6A4544067880D9CFC04F02312@multiplay.co.uk> <26053_1387447492_52B2C4C4_26053_331_1_20131219105503.3a8d1df3.gerrit.kuehn@aei.mpg.de> <20131219165549.9f2ca709.gerrit.kuehn@aei.mpg.de> <20131219174054.91ac617a.gerrit.kuehn@aei.mpg.de> <20131220100522.382a39ac.gerrit.kuehn@aei.mpg.de> Organization: Max Planck Gesellschaft X-Mailer: Sylpheed 3.1.3 (GTK+ 2.24.19; amd64-portbld-freebsd8.2) Mime-Version: 1.0 X-MIMETrack: Itemize by SMTP Server on intranet/aei-hannover(Release 8.5.3FP6|November 21, 2013) at 01/22/2014 09:46:58, Serialize by Router on intranet/aei-hannover(Release 8.5.3FP6|November 21, 2013) at 01/22/2014 09:47:08, Serialize complete at 01/22/2014 09:47:08 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-PMX-Version: 6.0.2.2308539, Antispam-Engine: 2.7.2.2107409, Antispam-Data: 2014.1.22.82715 X-PerlMx-Spam: Gauge=IIIIIIII, Probability=8%, Report=' HTML_00_01 0.05, HTML_00_10 0.05, MIME_LOWER_CASE 0.05, BODYTEXTP_SIZE_3000_LESS 0, BODY_SIZE_1000_LESS 0, BODY_SIZE_2000_LESS 0, BODY_SIZE_400_499 0, BODY_SIZE_5000_LESS 0, BODY_SIZE_7000_LESS 0, __ANY_URI 0, __BOUNCE_CHALLENGE_SUBJ 0, __BOUNCE_NDR_SUBJ_EXEMPT 0, __CT 0, __CTE 0, __CT_TEXT_PLAIN 0, __HAS_FROM 0, __HAS_MSGID 0, __HAS_X_MAILER 0, __IN_REP_TO 0, __MIME_TEXT_ONLY 0, __MIME_VERSION 0, __SANE_MSGID 0, __SUBJ_ALPHA_NEGATE 0, __TO_MALFORMED_2 0, __TO_NO_NAME 0, __URI_NO_PATH 0, __URI_NO_WWW 0, __URI_NS ' Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jan 2014 08:55:07 -0000 On Fri, 20 Dec 2013 17:18:18 +0100 (CET) krichy@tvnetwork.hu wrote about Re: ZFS snapshot renames failing after upgrade to 9.2: KH> I will look if I can check or fix it. That would be great. KH> Did this work in 9.1? Looks like I missed to answer this, sorry. No, this does not happen with 9.1; specifically, I have a 9.1-RELEASE-p3 that does not show this behaviour. cu Gerrit From owner-freebsd-fs@FreeBSD.ORG Wed Jan 22 14:11:15 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9D049FD3 for ; Wed, 22 Jan 2014 14:11:15 +0000 (UTC) Received: from smarthost1.greenhost.nl (smarthost1.greenhost.nl [195.190.28.81]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5CA521CBB for ; Wed, 22 Jan 2014 14:11:14 +0000 (UTC) Received: from smtp.greenhost.nl ([213.108.104.138]) by smarthost1.greenhost.nl with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.72) (envelope-from ) id 1W5yWI-0005cu-TT for freebsd-fs@freebsd.org; Wed, 22 Jan 2014 15:11:07 +0100 Content-Type: text/plain; charset=us-ascii; format=flowed; delsp=yes To: freebsd-fs@freebsd.org Subject: Re: hast and zfs References: Date: Wed, 22 Jan 2014 15:11:04 +0100 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: "Ronald Klop" Message-ID: In-Reply-To: User-Agent: Opera Mail/12.16 (FreeBSD) X-Authenticated-As-Hash: 398f5522cb258ce43cb679602f8cfe8b62a256d1 X-Virus-Scanned: by clamav at smarthost1.samage.net X-Spam-Level: / X-Spam-Score: 0.8 X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50, T_FRT_BELOW2 autolearn=disabled version=3.3.2 X-Scan-Signature: dfea3049d3b923820beb462d65569822 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jan 2014 14:11:15 -0000 On Tue, 21 Jan 2014 15:46:09 +0100, Joar Jegleim wrote: > Hi list ! > > I've setup hast with zfs on 9.1-RELEASE-p10 > I've got an lsi hba connected to an external sas hp msa that has 20 > disks in it, same setup on both nodes. > I've created a hast disk dev such as /dev/hast/disk1a, > /dev/hast/disk1b etc... ( see attached hast.conf ) > And built a zpool mirror accross the 20 disks. > > While setting up this, and testing, all looked pretty good, I've tried > numerous failovers and so on. > When I just switched over to this setup in production, it worked for > about half an hour until the primary node got a zfs hang where zpool > list hung and hastctl status hung completely . > > There was no output in dmesg nor /var/log/messages that said anything > related to this (?) . > > The server serves some 3 000 000 jpegs for a site, I see anything from > 50 to just above 1000 iops, though the average iops is bellow 100. > It's completely random what pictures are being fetched at any time. > > Could there be some sysctls to tune related to this setup mabye ? > Anybody using hast and zfs in production got any tip ? > > > You did not get any reply yet. I'm not of any help either, but is there a reason you don't use a newer FreeBSD version? (9.1 is from December 2012.) It can be useful to read through the commits about hast which happened in the meantime. Ronald. From owner-freebsd-fs@FreeBSD.ORG Wed Jan 22 14:25:38 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9961B587; Wed, 22 Jan 2014 14:25:38 +0000 (UTC) Received: from thebighonker.lerctr.org (lrosenman-1-pt.tunnel.tserv8.dal1.ipv6.he.net [IPv6:2001:470:1f0e:3ad::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 5A6CD1DBE; Wed, 22 Jan 2014 14:25:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:Subject:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=MVRcS7qDXnmS2x9xxquFtNnKftV0GuvYd7ajw6g4uBE=; b=GyH4sur1ybdR7EzDAcOvPzXfaD2Z4TpPnQQisXrB4jLVsYAxApmr7632OLCqd8RuQsGImoYbJ4s2cMb6gkA8YbJXeZnHyGUd5VTlmwFyNwi1VqfzVchlSSE9lxi2z21RlL6Yclq/yKUtF+pSpewH04GvWYoZbaYDVgtfq5AyFE4=; Received: from localhost.lerctr.org ([127.0.0.1]:40046 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpa (Exim 4.82 (FreeBSD)) (envelope-from ) id 1W5ykE-000EjT-HD; Wed, 22 Jan 2014 08:25:36 -0600 Received: from proxy.lucent.com ([135.245.48.14]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Wed, 22 Jan 2014 08:25:30 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Wed, 22 Jan 2014 08:25:30 -0600 From: Larry Rosenman To: Freebsd fs , Freebsd current Subject: Fwd: Panic/Solaris Assert/ZAP.c:534 Message-ID: X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/0.9.5 X-Spam-Score: 2.0 (++) X-LERCTR-Spam-Score: 2.0 (++) X-Spam-Report: SpamScore (2.0/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, FH_RANDOM_SURE=0.001, KAM_STOCKTIP=5.5, RP_MATCHES_RCVD=-0.59 X-LERCTR-Spam-Report: SpamScore (2.0/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, FH_RANDOM_SURE=0.001, KAM_STOCKTIP=5.5, RP_MATCHES_RCVD=-0.59 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jan 2014 14:25:38 -0000 Repost, including the FS guys. Ideas? -------- Original Message -------- Subject: Panic/Solaris Assert/ZAP.c:534 Date: 2014-01-20 21:23 From: Larry Rosenman To: freebsd-current@freebsd.org Ideas? I'm getting this on a regular basis, and scrub/etc doesn't seem to get around it, but booting off: FreeBSD borg.lerctr.org 11.0-CURRENT FreeBSD 11.0-CURRENT #113 r260864: Fri Jan 17 19:10:06 CST 2014 root@:/usr/obj/usr/src/sys/BORG-DTRACE amd64 Seems to avoid it. HELP. borg.lerctr.org dumped core - see /var/crash/vmcore.1 Mon Jan 20 21:18:41 CST 2014 FreeBSD borg.lerctr.org 11.0-CURRENT FreeBSD 11.0-CURRENT #1 r260896: Mon Jan 20 12:31:46 CST 2014 root@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER amd64 panic: solaris assert: l->l_phys->l_hdr.lh_block_type == ((1ULL << 63) + 0) (0x0 == 0x8000000000000000), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c, line: 534 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: Copyright (c) 1992-2014 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.0-CURRENT #1 r260896: Mon Jan 20 12:31:46 CST 2014 root@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER amd64 FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 info: [drm] Initialized drm 1.1.0 20060810 CPU: Intel(R) Xeon(R) CPU E5410 @ 2.33GHz (2327.55-MHz K8-class CPU) Origin="GenuineIntel" Id=0x10676 Family=0x6 Model=0x17 Stepping=6 Features=0xbfebfbff Features2=0xce3bd AMD Features=0x20100800 AMD Features2=0x1 TSC: P-state invariant, performance statistics real memory = 68719476736 (65536 MB) avail memory = 65641644032 (62600 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs FreeBSD/SMP: 2 package(s) x 4 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP): APIC ID: 3 cpu4 (AP): APIC ID: 4 cpu5 (AP): APIC ID: 5 cpu6 (AP): APIC ID: 6 cpu7 (AP): APIC ID: 7 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard random: initialized kbd1 at kbdmux0 cryptosoft0: on motherboard acpi0: on motherboard acpi0: Power Button (fixed) unknown: I/O range not supported cpu0: on acpi0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 cpu4: on acpi0 cpu5: on acpi0 cpu6: on acpi0 cpu7: on acpi0 hpet0: iomem 0xfed00000-0xfed003ff irq 0,8 on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 atrtc0: port 0x70-0x71 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: at device 2.0 on pci0 pci1: on pcib1 pcib2: irq 16 at device 0.0 on pci1 pci2: on pcib2 pcib3: irq 16 at device 0.0 on pci2 pci3: on pcib3 pcib4: at device 0.0 on pci3 pci4: on pcib4 pcib5: at device 0.2 on pci3 pci5: on pcib5 pcib6: irq 18 at device 2.0 on pci2 pci6: on pcib6 em0: port 0x2000-0x201f mem 0xd9220000-0xd923ffff,0xd9200000-0xd921ffff irq 18 at device 0.0 on pci6 em0: Using an MSI interrupt em0: Ethernet address: 00:30:48:f2:29:9c em1: port 0x2020-0x203f mem 0xd9260000-0xd927ffff,0xd9240000-0xd925ffff irq 19 at device 0.1 on pci6 em1: Using an MSI interrupt em1: Ethernet address: 00:30:48:f2:29:9d pcib7: at device 0.3 on pci1 pci7: on pcib7 pcib8: at device 4.0 on pci0 pci8: on pcib8 vgapci0: port 0x3000-0x307f mem 0xd8000000-0xd8ffffff,0xc0000000-0xc7ffffff,0xc8000000-0xc9ffffff irq 16 at device 0.0 on pci8 nvidia0: on vgapci0 vgapci0: child nvidia0 requested pci_enable_io vgapci0: child nvidia0 requested pci_enable_io hdac0: mem 0xd9000000-0xd9003fff irq 17 at device 0.1 on pci8 pcib9: at device 6.0 on pci0 pci9: on pcib9 pci0: at device 8.0 (no driver attached) pcib10: irq 17 at device 28.0 on pci0 pci10: on pcib10 pcib11: irq 16 at device 0.0 on pci10 pci11: on pcib11 pcm0: port 0x4080-0x409f,0x4000-0x407f irq 16 at device 0.0 on pci11 pcm0: system configuration SubVendorID: 0x1412, SubDeviceID: 0x2403 XIN2 Clock Source: 24.576MHz(96kHz*256) MPU-401 UART(s) #: not implemented ADC #: 1 and SPDIF receiver connected DAC #: 4 Multi-track converter type: AC'97(SDATA_OUT:packed) S/PDIF(IN/OUT): 1/1 ID# 0x00 GPIO(mask/dir/state): 0xff/0xff/0xff uhci0: port 0x1800-0x181f irq 17 at device 29.0 on pci0 usbus0 on uhci0 uhci1: port 0x1820-0x183f irq 19 at device 29.1 on pci0 usbus1 on uhci1 uhci2: port 0x1840-0x185f irq 18 at device 29.2 on pci0 usbus2 on uhci2 ehci0: mem 0xd9600400-0xd96007ff irq 17 at device 29.7 on pci0 usbus3: EHCI version 1.0 usbus3 on ehci0 pcib12: at device 30.0 on pci0 pci12: on pcib12 vgapci1: port 0x5000-0x50ff mem 0xd0000000-0xd7ffffff,0xd9300000-0xd930ffff irq 18 at device 1.0 on pci12 drmn1: on vgapci1 info: [drm] RADEON_IS_PCI info: [drm] initializing kernel modesetting (RV100 0x1002:0x515E 0x15D9:0x8080). info: [drm] register mmio base: 0xD9300000 info: [drm] register mmio size: 65536 info: [drm] radeon_atrm_get_bios: ===> Try ATRM... info: [drm] radeon_atrm_get_bios: pci_find_class() found: 0:8:0:0, vendor=10de, device=104a info: [drm] radeon_atrm_get_bios: Get ACPI device handle info: [drm] radeon_acpi_vfct_bios: ===> Try VFCT... info: [drm] radeon_acpi_vfct_bios: Get "VFCT" ACPI table info: [drm] radeon_acpi_vfct_bios: Failed to get "VFCT" table: AE_NOT_FOUND info: [drm] igp_read_bios_from_vram: ===> Try IGP's VRAM... info: [drm] igp_read_bios_from_vram: VRAM base address: 0xd0000000 info: [drm] igp_read_bios_from_vram: Map address: 0xfffff800d0000000 (262144 bytes) info: [drm] igp_read_bios_from_vram: Incorrect BIOS signature: 0x0000 info: [drm] radeon_read_bios: ===> Try PCI Expansion ROM... info: [drm] radeon_read_bios: Map address: 0xfffff800000c0000 (131072 bytes) drmn1: info: VRAM: 128M 0x00000000D0000000 - 0x00000000D7FFFFFF (16M used) drmn1: info: GTT: 512M 0x00000000B0000000 - 0x00000000CFFFFFFF info: [drm] Supports vblank timestamp caching Rev 1 (10.10.2010). info: [drm] Driver supports precise vblank timestamp query. info: [drm] radeon: irq initialized. info: [drm] Detected VRAM RAM=128M, BAR=128M info: [drm] RAM width 64bits SDR [TTM] Zone kernel: Available graphics memory: 33006090 kiB [TTM] Zone dma32: Available graphics memory: 2097152 kiB [TTM] Initializing pool allocator info: [drm] radeon: 16M of VRAM memory ready info: [drm] radeon: 512M of GTT memory ready. info: [drm] GART: num cpu pages 131072, num gpu pages 131072 info: [drm] PCI GART of 512M enabled (table at 0x0000000011200000). drmn1: info: WB disabled drmn1: info: fence driver on ring 0 use gpu addr 0x00000000b0000000 and cpu addr 0x0xfffff8001083c000 info: [drm] Loading R100 Microcode info: [drm] radeon: ring at 0x00000000B0001000 info: [drm] ring test succeeded in 1 usecs info: [drm] ib test succeeded in 0 usecs info: [drm] radeon_device_init: Taking over the fictitious range 0xd0000000-0xd4000000 iicbus0: on iicbb0 addr 0xff iic0: on iicbus0 iicbus1: on iicbb1 addr 0xff iic1: on iicbus1 iicbus2: on iicbb2 addr 0xff iic2: on iicbus2 iicbus3: on iicbb3 addr 0xff iic3: on iicbus3 info: [drm] Radeon Display Connectors info: [drm] Connector 0: info: [drm] VGA-1 info: [drm] DDC: 0x60 0x60 0x60 0x60 0x60 0x60 0x60 0x60 info: [drm] Encoders: info: [drm] CRT1: INTERNAL_DAC1 info: [drm] Connector 1: info: [drm] DVI-I-1 info: [drm] HPD2 info: [drm] DDC: 0x6c 0x6c 0x6c 0x6c 0x6c 0x6c 0x6c 0x6c info: [drm] Encoders: info: [drm] CRT2: INTERNAL_DAC2 info: [drm] DFP2: INTERNAL_DVO1 composite sync not supported composite sync not supported info: [drm] fb mappable at 0xD0040000 info: [drm] vram apper at 0xD0000000 info: [drm] size 2076672 info: [drm] fb depth is 8 info: [drm] pitch is 1920 fbd1 on drmn1 vt_allocate: Replace existing VT driver. info: [drm] Initialized radeon 2.29.0 20080528 vgapci1: Boot video device isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1860-0x186f at device 31.1 on pci0 ata0: at channel 0 on atapci0 ahci0: port 0x18a0-0x18a7,0x1874-0x1877,0x1878-0x187f,0x1870-0x1873,0x1880-0x189f mem 0xd9600800-0xd9600bff irq 19 at device 31.2 on pci0 ahci0: AHCI v1.10 with 6 3Gbps ports, Port Multiplier supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahcich4: at channel 4 on ahci0 ahcich5: at channel 5 on ahci0 ichsmb0: port 0x1100-0x111f irq 19 at device 31.3 on pci0 smbus0: on ichsmb0 acpi_button0: on acpi0 ipmi0: port 0xca2-0xca3 on acpi0 ipmi0: KCS mode found at io 0xca2 on acpi atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fd0: <1440-KB 3.5" drive> on fdc0 drive 0 ppc0: port 0x378-0x37f,0x778-0x77f irq 7 drq 3 on acpi0 ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/9 bytes threshold ppbus0: on ppc0 lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 ichwd0 on isa0 orm0: at iomem 0xc0000-0xcafff on isa0 coretemp0: on cpu0 est0: on cpu0 p4tcc0: on cpu0 coretemp1: on cpu1 est1: on cpu1 p4tcc1: on cpu1 coretemp2: on cpu2 est2: on cpu2 p4tcc2: on cpu2 coretemp3: on cpu3 est3: on cpu3 p4tcc3: on cpu3 coretemp4: on cpu4 est4: on cpu4 p4tcc4: on cpu4 coretemp5: on cpu5 est5: on cpu5 p4tcc5: on cpu5 coretemp6: on cpu6 est6: on cpu6 p4tcc6: on cpu6 coretemp7: on cpu7 est7: on cpu7 p4tcc7: on cpu7 ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Timecounters tick every 1.000 msec vboxdrv: fAsync=0 offMin=0x387 offMax=0x491 hdacc0: at cad 0 on hdac0 hdaa0: at nid 1 on hdacc0 pcm1: at nid 4 on hdaa0 pcm2: at nid 5 on hdaa0 random: unblocking device. usbus0: 12Mbps Full Speed USB v1.0 usbus1: 12Mbps Full Speed USB v1.0 usbus2: 12Mbps Full Speed USB v1.0 usbus3: 480Mbps High Speed USB v2.0 ugen3.1: at usbus3 uhub0: on usbus3 ugen2.1: at usbus2 uhub1: on usbus2 ugen1.1: at usbus1 uhub2: on usbus1 ugen0.1: at usbus0 uhub3: on usbus0 ipmi0: IPMI device rev. 1, firmware rev. 1.64, version 2.0 ata0: DMA limited to UDMA33, controller found non-ATA66 cable ipmi0: Number of channels 8 ipmi0: Attached watchdog uhub2: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered uhub3: 2 ports with 2 removable, self powered ada0 at ahcich0 bus 0 scbus1 target 0 lun 0 ada0: ATA-8 SATA 3.x device ada0: Serial Number 5YDA1ZL4 ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada0: quirks=0x1<4K> ada0: Previously was known as ad4 ada1 at ahcich1 bus 0 scbus2 target 0 lun 0 ada1: ATA-8 SATA 3.x device ada1: Serial Number 5YD6FPLG ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada1: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada1: quirks=0x1<4K> ada1: Previously was known as ad6 ada2 at ahcich2 bus 0 scbus3 target 0 lun 0 ada2: ATA-8 SATA 3.x device ada2: Serial Number 5YDA3PC5 ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada2: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada2: quirks=0x1<4K> ada2: Previously was known as ad8 ada3 at ahcich3 bus 0 scbus4 target 0 lun 0 ada3: ATA-8 SATA 3.x device ada3: Serial Number 5YD9Y0P4 ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada3: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada3: quirks=0x1<4K> ada3: Previously was known as ad10 ada4 at ahcich4 bus 0 scbus5 target 0 lun 0 ada4: ATA-8 SATA 3.x device ada4: Serial Number 5YDA1R5W ada4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada4: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada4: quirks=0x1<4K> ada4: Previously was known as ad12 ada5 at ahcich5 bus 0 scbus6 target 0 lun 0 ada5: ATA-8 SATA 3.x device ada5: Serial Number 5YD5RBS8 ada5: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada5: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada5: quirks=0x1<4K> ada5: Previously was known as ad14 cd0 at ata0 bus 0 scbus0 target 0 lun 0 cd0: Removable CD-ROM SCSI-0 device cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present Netvsc initializing... done! SMP: AP CPU #6 Launched! SMP: AP CPU #1 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #7 Launched! SMP: AP CPU #5 Launched! SMP: AP CPU #4 Launched! SMP: AP CPU #3 Launched! Root mount waiting for: usbus3 uhub0: 6 ports with 6 removable, self powered Root mount waiting for: usbus3 Root mount waiting for: usbus3 ugen3.2: at usbus3 ukbd0: on usbus3 kbd2 at ukbd0 Trying to mount root from zfs:zroot/ROOT/default []... ugen0.2: at usbus0 ugen1.2: at usbus1 <118>Setting hostuuid: 53d19f64-d663-a017-8922-0030488e9ff3. <118>Setting hostid: 0xf53a926e. <118>Entropy harvesting: interrupts ethernet point_to_point swi. <118>Starting file system checks: <118>Mounting local file systems:. panic: solaris assert: l->l_phys->l_hdr.lh_block_type == ((1ULL << 63) + 0) (0x0 == 0x8000000000000000), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c, line: 534 cpuid = 7 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe100c499090 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe100c499140 vpanic() at vpanic+0x126/frame 0xfffffe100c499180 panic() at panic+0x43/frame 0xfffffe100c4991e0 assfail3() at assfail3+0x2f/frame 0xfffffe100c499200 zap_get_leaf_byblk() at zap_get_leaf_byblk+0x4e1/frame 0xfffffe100c499250 zap_deref_leaf() at zap_deref_leaf+0xd1/frame 0xfffffe100c4992a0 fzap_cursor_retrieve() at fzap_cursor_retrieve+0xce/frame 0xfffffe100c499310 zap_cursor_retrieve() at zap_cursor_retrieve+0x20a/frame 0xfffffe100c499390 ddt_zap_walk() at ddt_zap_walk+0x5e/frame 0xfffffe100c499650 ddt_walk() at ddt_walk+0x78/frame 0xfffffe100c499680 dsl_scan_sync() at dsl_scan_sync+0x6ec/frame 0xfffffe100c4999e0 spa_sync() at spa_sync+0x5c4/frame 0xfffffe100c499ad0 txg_sync_thread() at txg_sync_thread+0x256/frame 0xfffffe100c499bb0 fork_exit() at fork_exit+0x84/frame 0xfffffe100c499bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe100c499bf0 --- trap 0, rip = 0, rsp = 0xfffffe100c499cb0, rbp = 0 --- KDB: enter: panic Uptime: 14s Dumping 2263 out of 64465 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Reading symbols from /boot/kernel.old/linux.ko.symbols...done. Loaded symbols for /boot/kernel.old/linux.ko.symbols Reading symbols from /boot/kernel.old/if_lagg.ko.symbols...done. Loaded symbols for /boot/kernel.old/if_lagg.ko.symbols Reading symbols from /boot/kernel.old/snd_envy24ht.ko.symbols...done. Loaded symbols for /boot/kernel.old/snd_envy24ht.ko.symbols Reading symbols from /boot/kernel.old/snd_spicds.ko.symbols...done. Loaded symbols for /boot/kernel.old/snd_spicds.ko.symbols Reading symbols from /boot/kernel.old/coretemp.ko.symbols...done. Loaded symbols for /boot/kernel.old/coretemp.ko.symbols Reading symbols from /boot/kernel.old/ichsmb.ko.symbols...done. Loaded symbols for /boot/kernel.old/ichsmb.ko.symbols Reading symbols from /boot/kernel.old/smbus.ko.symbols...done. Loaded symbols for /boot/kernel.old/smbus.ko.symbols Reading symbols from /boot/kernel.old/ichwd.ko.symbols...done. Loaded symbols for /boot/kernel.old/ichwd.ko.symbols Reading symbols from /boot/kernel.old/cpuctl.ko.symbols...done. Loaded symbols for /boot/kernel.old/cpuctl.ko.symbols Reading symbols from /boot/kernel.old/crypto.ko.symbols...done. Loaded symbols for /boot/kernel.old/crypto.ko.symbols Reading symbols from /boot/kernel.old/cryptodev.ko.symbols...done. Loaded symbols for /boot/kernel.old/cryptodev.ko.symbols Reading symbols from /boot/kernel.old/dtraceall.ko.symbols...done. Loaded symbols for /boot/kernel.old/dtraceall.ko.symbols Reading symbols from /boot/kernel.old/profile.ko.symbols...done. Loaded symbols for /boot/kernel.old/profile.ko.symbols Reading symbols from /boot/kernel.old/cyclic.ko.symbols...done. Loaded symbols for /boot/kernel.old/cyclic.ko.symbols Reading symbols from /boot/kernel.old/dtrace.ko.symbols...done. Loaded symbols for /boot/kernel.old/dtrace.ko.symbols Reading symbols from /boot/kernel.old/systrace_freebsd32.ko.symbols...done. Loaded symbols for /boot/kernel.old/systrace_freebsd32.ko.symbols Reading symbols from /boot/kernel.old/systrace.ko.symbols...done. Loaded symbols for /boot/kernel.old/systrace.ko.symbols Reading symbols from /boot/kernel.old/sdt.ko.symbols...done. Loaded symbols for /boot/kernel.old/sdt.ko.symbols Reading symbols from /boot/kernel.old/lockstat.ko.symbols...done. Loaded symbols for /boot/kernel.old/lockstat.ko.symbols Reading symbols from /boot/kernel.old/fasttrap.ko.symbols...done. Loaded symbols for /boot/kernel.old/fasttrap.ko.symbols Reading symbols from /boot/kernel.old/fbt.ko.symbols...done. Loaded symbols for /boot/kernel.old/fbt.ko.symbols Reading symbols from /boot/kernel.old/dtnfscl.ko.symbols...done. Loaded symbols for /boot/kernel.old/dtnfscl.ko.symbols Reading symbols from /boot/kernel.old/dtmalloc.ko.symbols...done. Loaded symbols for /boot/kernel.old/dtmalloc.ko.symbols Reading symbols from /boot/modules/vboxdrv.ko...done. Loaded symbols for /boot/modules/vboxdrv.ko Reading symbols from /boot/modules/nvidia.ko...done. Loaded symbols for /boot/modules/nvidia.ko Reading symbols from /boot/kernel.old/ipmi.ko.symbols...done. Loaded symbols for /boot/kernel.old/ipmi.ko.symbols Reading symbols from /boot/kernel.old/ipmi_linux.ko.symbols...done. Loaded symbols for /boot/kernel.old/ipmi_linux.ko.symbols Reading symbols from /boot/kernel.old/radeonkms.ko.symbols...done. Loaded symbols for /boot/kernel.old/radeonkms.ko.symbols Reading symbols from /boot/kernel.old/iicbb.ko.symbols...done. Loaded symbols for /boot/kernel.old/iicbb.ko.symbols Reading symbols from /boot/kernel.old/iicbus.ko.symbols...done. Loaded symbols for /boot/kernel.old/iicbus.ko.symbols Reading symbols from /boot/kernel.old/iic.ko.symbols...done. Loaded symbols for /boot/kernel.old/iic.ko.symbols Reading symbols from /boot/kernel.old/drm2.ko.symbols...done. Loaded symbols for /boot/kernel.old/drm2.ko.symbols Reading symbols from /boot/kernel.old/radeonkmsfw_R100_cp.ko.symbols...done. Loaded symbols for /boot/kernel.old/radeonkmsfw_R100_cp.ko.symbols Reading symbols from /boot/kernel.old/fdescfs.ko.symbols...done. Loaded symbols for /boot/kernel.old/fdescfs.ko.symbols Reading symbols from /boot/kernel.old/linprocfs.ko.symbols...done. Loaded symbols for /boot/kernel.old/linprocfs.ko.symbols #0 doadump (textdump=1) at pcpu.h:219 219 pcpu.h: No such file or directory. in pcpu.h (kgdb) #0 doadump (textdump=1) at pcpu.h:219 #1 0xffffffff809b98d7 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:452 #2 0xffffffff809b9de5 in vpanic (fmt=, ap=) at /usr/src/sys/kern/kern_shutdown.c:759 #3 0xffffffff809b9e33 in panic (fmt=) at /usr/src/sys/kern/kern_shutdown.c:688 #4 0xffffffff80327a2f in assfail3 (a=, lv=, op=, rv=, f=, l=) at /usr/src/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c:91 #5 0xffffffff803bad81 in zap_get_leaf_byblk (zap=, blkid=, tx=0x0, lt=, lp=0xfffffe100c4994e8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c:534 #6 0xffffffff803b8461 in zap_deref_leaf (zap=0xfffff80020a4ec00, h=9377620324092215296, tx=0x0, lt=RW_READER, lp=0xfffffe100c4994e8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c:585 #7 0xffffffff803ba24e in fzap_cursor_retrieve (zap=0xfffff80020a4ec00, zc=0xfffffe100c4994d8, za=0xfffffe100c4993c0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c:1181 #8 0xffffffff803c076a in zap_cursor_retrieve (zc=0xfffffe100c4994d8, za=0xfffffe100c4993c0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap_micro.c:1290 #9 0xffffffff80352fde in ddt_zap_walk (os=0xfffff8002031bc00, object=156, dde=0xfffffe100c499828, walk=0xfffff800201ce3d0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/ddt_zap.c:117 #10 0xffffffff80352c88 in ddt_walk (spa=0xfffffe000d377000, ddb=0xfffff800201ce3b8, dde=0xfffffe100c499828) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/ddt.c:219 #11 0xffffffff803800dc in dsl_scan_sync (dp=0xfffff8002031f800, tx=0xfffff80022180000) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c:1204 #12 0xffffffff803999b4 in spa_sync (spa=0xfffffe000d377000, txg=14079627) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:6549 #13 0xffffffff803a4e96 in txg_sync_thread (arg=0xfffff8002031f800) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c:518 #14 0xffffffff80988f94 in fork_exit ( callout=0xffffffff803a4c40 , arg=0xfffff8002031f800, frame=0xfffffe100c499c00) at /usr/src/sys/kern/kern_fork.c:977 #15 0xffffffff80d9576e in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:605 #16 0x0000000000000000 in ?? () Current language: auto; currently minimal (kgdb) ------------------------------------------------------------------------ ps -axlww UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND 0 0 0 0 -8 0 0 0 - DLs - 0:00.10 [kernel] 0 1 0 0 21 0 9432 0 wait DLs - 0:00.00 [init] 0 2 0 0 -16 0 0 0 ftcl DL - 0:00.00 [ftcleanup] 0 3 0 0 -16 0 0 0 crypto_w DL - 0:00.00 [crypto] 0 4 0 0 -16 0 0 0 crypto_r DL - 0:00.00 [crypto returns] 0 5 0 0 -16 0 0 0 ccb_scan DL - 0:00.00 [cam] 0 6 0 0 -16 0 0 0 - DL - 0:00.00 [fdc0] 0 7 0 0 -8 0 0 0 - RL - 0:00.00 [zfskern] 0 8 0 0 -16 0 0 0 waiting_ DL - 0:00.00 [sctp_iterator] 0 9 0 0 -16 0 0 0 ipmireq DL - 0:00.00 [ipmi0: kcs] 0 10 0 0 -16 0 0 0 audit_wo DL - 0:00.00 [audit] 0 11 0 0 155 0 0 0 - RL - 0:00.00 [idle] 0 12 0 0 -84 0 0 0 - WL - 0:00.00 [intr] 0 13 0 0 -8 0 0 0 - DL - 0:00.00 [geom] 0 14 0 0 -16 0 0 0 - DL - 0:00.00 [rand_harvestq] 0 15 0 0 -68 0 0 0 - DL - 0:00.00 [usb] 0 16 0 0 -20 0 0 0 VBoxIS DL - 0:00.00 [TIMER] 0 17 0 0 -16 0 0 0 psleep DL - 0:00.00 [pagedaemon] 0 18 0 0 -16 0 0 0 psleep DL - 0:00.00 [vmdaemon] 0 19 0 0 155 0 0 0 pgzero DL - 0:00.00 [pagezero] 0 20 0 0 -16 0 0 0 psleep DL - 0:00.00 [bufdaemon] 0 21 0 0 16 0 0 0 syncer DL - 0:00.00 [syncer] 0 22 0 0 -16 0 0 0 vlruwt DL - 0:00.00 [vnlru] 0 23 0 0 -16 0 0 0 sdflush DL - 0:00.00 [softdepflush] 0 24 1 0 52 0 16992 0 wait Ds+ - 0:00.02 [sh] 0 72 24 0 52 0 16992 0 wait D+ - 0:00.00 [sh] 0 75 72 0 39 0 40064 0 zio->io_ D+ - 0:00.00 [zfs] ------------------------------------------------------------------------ vmstat -s 42891 cpu context switches 5520 device interrupts 462 software interrupts 5061 traps 5879 system calls 23 kernel threads created 37 fork() calls 15 vfork() calls 0 rfork() calls 0 swap pager pageins 0 swap pager pages paged in 0 swap pager pageouts 0 swap pager pages paged out 135 vnode pager pageins 923 vnode pager pages paged in 0 vnode pager pageouts 0 vnode pager pages paged out 0 page daemon wakeups 0 pages examined by the page daemon 0 pages reactivated 1770 copy-on-write faults 15 copy-on-write optimized faults 2266 zero fill pages zeroed 0 zero fill pages prezeroed 0 intransit blocking page faults 4682 total VM faults taken 112 page faults requiring I/O 0 pages affected by kernel thread creation 1350 pages affected by fork() 525 pages affected by vfork() 0 pages affected by rfork() 0 pages cached 55124 pages freed 0 pages freed by daemon 0 pages freed by exiting processes 473 pages active 869 pages inactive 0 pages in VM cache 140410 pages wired down 15933660 pages free 4096 bytes per page 1934 total name lookups cache hits (80% pos + 4% neg) system 0% per-directory deletions 0%, falsehits 0%, toolong 0% ------------------------------------------------------------------------ vmstat -m Type InUse MemUse HighUse Requests Size(s) cdev 8 2K - 8 256 ppbusdev 2 1K - 2 256 filedesc 27 54K - 76 2048 kdtrace 330 78K - 522 64,256 kenv 87 12K - 115 16,32,64,128 proc-args 3 1K - 31 32,64,128 hhook 2 1K - 2 256 ithread 125 22K - 125 32,128,256 KTRACE 100 13K - 100 128 entropy 1026 65K - 1026 32,64 acpiintr 1 1K - 1 64 linker 403 101K - 533 16,32,64,128,256,512,1024,2048,4096 lockf 2 1K - 2 64,128 loginclass 2 1K - 2 64 cache 1 1K - 1 32 devbuf 17314 34391K - 18687 16,32,64,128,256,512,1024,2048,4096 temp 57 18K - 548 16,32,64,128,256,512,1024,2048 ip6ndp 3 1K - 3 64 ata_pci 1 1K - 1 64 acpica 1824 187K - 63680 16,32,64,128,256,512,1024,2048 kbdmux 7 18K - 7 16,512,1024,2048 module 521 66K - 521 128 mtx_pool 2 16K - 2 LED 4 1K - 4 16,128 CAM XPT 58 4K - 309 16,32,64,128,512,1024,2048 osd 6 1K - 12 16,32,64,128 pmchooks 1 1K - 1 128 CAM DEV 15 30K - 24 2048 acpitask 1 8K - 1 hdaa 4 5K - 4 512,1024,2048 pgrp 2 1K - 2 128 session 2 1K - 2 128 proc 2 256K - 2 subproc 100 145K - 149 512,4096 cred 54 9K - 254 64,256 plimit 2 1K - 2 256 uidinfo 2 33K - 2 128 hdac 1 1K - 1 512 hdacc 1 1K - 1 32 vtbuf 24 5712K - 24 4096 acpisem 22 3K - 22 128 vt 11 3K - 11 256 sysctl 0 0K - 50 16,32,64 sysctloid 7607 378K - 7779 16,32,64,128 sysctltmp 0 0K - 6 32,64 tidhash 1 256K - 1 callout 9 3208K - 9 umtx 684 86K - 684 128 p1003.1b 1 1K - 1 16 SWAP 12 19681K - 12 64 bus 2257 362K - 8440 16,32,64,128,256,512,1024 bus-sc 157 310K - 5479 16,32,64,128,256,512,1024,2048,4096 devstat 32 65K - 32 32,4096 eventhandler 104 9K - 104 64,128 DEVFS3 230 58K - 270 256 kobj 349 1396K - 1111 4096 DEVFS1 202 101K - 238 512 Per-cpu 1 1K - 1 32 DEVFS 41 1K - 42 16,128 rman 360 41K - 754 16,32,128 sbuf 0 0K - 2295 16,32,64,128,256 sglist 3 1K - 3 32 taskqueue 127 19K - 171 16,32,64,128,256 terminal 11 3K - 11 256 Unitno 28 2K - 120 32,64 vmem 2 136K - 2 ioctlops 0 0K - 158 16,32,64 iov 1 1K - 34 64,256,512 msg 4 30K - 4 2048,4096 sem 4 106K - 4 2048,4096 shm 1 20K - 1 tty 14 14K - 14 1024 shmfd 1 8K - 1 pcb 12 8341K - 12 16,128,1024,2048 vfscache 1 16384K - 1 vfs_hash 1 8192K - 1 vnodes 1 1K - 1 256 mount 251 9K - 550 16,32,64,128,256 vnodemarker 0 0K - 5 512 BPF 3 1K - 3 128 ifnet 4 7K - 4 128,2048 ifaddr 34 10K - 34 32,64,256,512,2048,4096 clone 8 1K - 8 128 arpcom 2 1K - 2 16 lltable 6 3K - 6 512 routetbl 5 2K - 5 256,512 igmp 3 1K - 3 256 sctp_vrf 1 1K - 1 64 hostcache 1 28K - 1 syncache 1 64K - 1 mld 3 1K - 3 128 rpc 2 1K - 2 512 audit_evclass 187 6K - 228 32 ufs_quota 1 8192K - 1 vm_pgdata 7 8193K - 7 128 pfs_nodes 73 19K - 73 256 pfs_vncache 2 1K - 2 64 scsi_cd 0 0K - 11 16 GEOM 404 67K - 3278 16,32,64,128,256,512,1024,2048 memdesc 1 4K - 1 4096 feeder 20 2K - 24 32,128 acpidev 36 3K - 36 64 atkbddev 2 1K - 2 64 CAM CCB 0 0K - 5331 2048 mixer 3 12K - 3 4096 raid_data 0 0K - 480 32,128,256 CAM path 22 1K - 115 32 solaris 289279 163254K - 391115 16,32,64,128,256,512,1024,2048,4096 UART 6 5K - 6 16,1024 kstat_data 6 1K - 6 64 CAM periph 16 4K - 40 16,32,64,128,256 md_nvidia_data 0 0K - 78 512 pci_link 16 2K - 16 32,64,128 md_sii_data 0 0K - 78 512 acpi_perf 8 1K - 8 64 CAM queue 23 8K - 57 16,32,512 apmdev 1 1K - 1 128 madt_table 0 0K - 1 4096 CAM dev queue 8 1K - 8 64 CAM SIM 8 2K - 8 256 ddb_capture 1 48K - 1 io_apic 2 4K - 2 2048 MCA 8 1K - 8 128 msi 4 1K - 4 128 nexusdev 4 1K - 4 16 isadev 5 1K - 5 128 USB 46 59K - 52 16,32,64,128,256,512,1024,2048,4096 USBdev 35 4K - 35 32,64,128,256 linux 17 2K - 17 64 spicds 7 1K - 7 128 envy24ht 15 195K - 15 64,2048 cpuctl 1 1K - 1 64 crypto 1 1K - 1 512 cyclic 32 3K - 32 16,64,128 SDT 32 2K - 32 16,64 fbt 60661 7839K - 60661 128 iprtheap 17 54K - 17 32,64,128,256 nvidia 194 864K - 195 16,32,64,128,256,512,1024,2048,4096 ipmi 0 0K - 11 128 gem_name 59 10K - 62 32,4096 drm_global 2 1K - 2 128,256 drm_vblank 7 1K - 7 16,64 drm_dma 2 1K - 2 32 drm_sarea 1 1K - 1 16 drm_driver 32 2203K - 36 16,32,64,128,256,512,1024,4096 drm_ctxbitmap 1 4K - 1 4096 drm_sman 13 2K - 13 128 drm_hashtab 1 4096K - 1 drm_kms 90 18K - 107 16,32,64,128,256,2048 ttm_pd 6 9K - 6 16,2048 ttm_rman 2 1K - 2 256 ttm_zone 2 1K - 2 64 ttm_poolmgr 1 1K - 1 512 fdesc_mount 1 1K - 1 16 ------------------------------------------------------------------------ vmstat -z ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP UMA Kegs: 384, 0, 267, 3, 267, 0, 0 UMA Zones: 1664, 0, 267, 1, 267, 0, 0 UMA Slabs: 112, 0, 4719, 6, 7221, 0, 0 UMA RCntSlabs: 120, 0, 0, 0, 0, 0, 0 UMA Hash: 256, 0, 77, 13, 77, 0, 0 4 Bucket: 32, 0, 262, 854, 1813, 0, 0 6 Bucket: 48, 0, 276, 1052, 1590, 0, 0 8 Bucket: 64, 0, 62, 992, 883, 0, 0 12 Bucket: 96, 0, 36, 989, 167, 0, 0 16 Bucket: 128, 0, 131, 1016, 1263, 17, 0 32 Bucket: 256, 0, 130, 380, 285, 49, 0 64 Bucket: 512, 0, 119, 238, 465, 49, 0 128 Bucket: 1024, 0, 406, 46, 3736, 2, 0 vmem btag: 56, 0, 11416, 441, 11514, 167, 0 VM OBJECT: 256, 0, 590, 400, 1335, 0, 0 RADIX NODE: 144, 0, 6442, 416, 10926, 0, 0 MAP: 240, 0, 3, 61, 3, 0, 0 KMAP ENTRY: 128, 0, 8, 271, 8, 0, 0 MAP ENTRY: 128, 0, 103, 920, 1717, 0, 0 VMSPACE: 448, 0, 4, 204, 54, 0, 0 fakepg: 104, 0, 0, 0, 0, 0, 0 mt_zone: 4112, 0, 405, 0, 405, 0, 0 16: 16, 0, 6, 243, 6, 0, 0 16: 16, 0, 2762, 973, 3015, 0, 0 16: 16, 0, 147166, 740, 158653, 0, 0 16: 16, 0, 55, 941, 28215, 0, 0 16: 16, 0, 62, 685, 63, 0, 0 16: 16, 0, 732, 1011, 5473, 0, 0 16: 16, 0, 30, 219, 30, 0, 0 16: 16, 0, 1, 746, 18, 0, 0 32: 32, 0, 30, 1086, 117, 0, 0 32: 32, 0, 3091, 877, 3295, 0, 0 32: 32, 0, 65004, 716, 73960, 0, 0 32: 32, 0, 66, 554, 4216, 0, 0 32: 32, 0, 18, 1098, 98, 0, 0 32: 32, 0, 839, 897, 2874, 0, 0 32: 32, 0, 94, 650, 186, 0, 0 32: 32, 0, 8, 736, 164, 0, 0 64: 64, 0, 5, 181, 6, 0, 0 64: 64, 0, 517, 909, 589, 0, 0 64: 64, 0, 7078, 982, 33984, 0, 0 64: 64, 0, 767, 783, 1637, 0, 0 64: 64, 0, 80, 106, 80, 0, 0 64: 64, 0, 9187, 857, 9746, 0, 0 64: 64, 0, 131, 551, 131, 0, 0 64: 64, 0, 1038, 760, 1071, 0, 0 128: 128, 0, 34, 741, 95, 0, 0 128: 128, 0, 1899, 612, 1908, 0, 0 128: 128, 0, 121830, 806, 128094, 0, 0 128: 128, 0, 1077, 318, 27007, 0, 0 128: 128, 0, 112, 663, 192, 0, 0 128: 128, 0, 1895, 740, 4111, 0, 0 128: 128, 0, 29, 126, 30, 0, 0 128: 128, 0, 524, 499, 526, 0, 0 256: 256, 0, 2, 73, 2, 0, 0 256: 256, 0, 308, 307, 451, 0, 0 256: 256, 0, 372, 543, 2665, 0, 0 256: 256, 0, 64, 131, 740, 0, 0 256: 256, 0, 28, 347, 348, 0, 0 256: 256, 0, 909, 441, 1963, 0, 0 256: 256, 0, 31, 44, 31, 0, 0 256: 256, 0, 4, 191, 25, 0, 0 512: 512, 0, 2, 33, 9, 0, 0 512: 512, 0, 69, 197, 225, 0, 0 512: 512, 0, 130, 276, 32908, 0, 0 512: 512, 0, 16, 47, 35, 0, 0 512: 512, 0, 3, 32, 3, 0, 0 512: 512, 0, 327, 170, 2804, 0, 0 512: 512, 0, 47, 16, 47, 0, 0 512: 512, 0, 1, 62, 2, 0, 0 1024: 1024, 0, 0, 104, 99, 0, 0 1024: 1024, 0, 9, 19, 9, 0, 0 1024: 1024, 0, 8372, 68, 23684, 0, 0 1024: 1024, 0, 5, 23, 2074, 0, 0 1024: 1024, 0, 16, 12, 16, 0, 0 1024: 1024, 0, 230, 98, 1280, 0, 0 1024: 1024, 0, 2, 14, 2, 0, 0 1024: 1024, 0, 0, 0, 0, 0, 0 2048: 2048, 0, 2, 4, 9, 0, 0 2048: 2048, 0, 6, 4, 6, 0, 0 2048: 2048, 0, 11, 19, 39, 0, 0 2048: 2048, 0, 7, 53, 5343, 0, 0 2048: 2048, 0, 21, 9, 30, 0, 0 2048: 2048, 0, 48, 42, 466, 0, 0 2048: 2048, 0, 5, 1, 5, 0, 0 2048: 2048, 0, 0, 0, 0, 0, 0 4096: 4096, 0, 0, 0, 0, 0, 0 4096: 4096, 0, 5, 0, 5, 0, 0 4096: 4096, 0, 167, 4, 295, 0, 0 4096: 4096, 0, 10, 0, 12, 0, 0 4096: 4096, 0, 16, 0, 17, 0, 0 4096: 4096, 0, 25, 0, 415, 0, 0 4096: 4096, 0, 15, 0, 15, 0, 0 4096: 4096, 0, 349, 10, 1111, 0, 0 uint64 pcpu: 8, 0, 1398, 138, 1398, 0, 0 SLEEPQUEUE: 88, 0, 343, 680, 343, 0, 0 Files: 80, 0, 8, 874, 395, 0, 0 rl_entry: 40, 0, 6, 687, 6, 0, 0 TURNSTILE: 136, 0, 343, 277, 343, 0, 0 umtx pi: 96, 0, 0, 0, 0, 0, 0 MAC labels: 40, 0, 0, 0, 0, 0, 0 PROC: 1208, 0, 26, 46, 75, 0, 0 THREAD: 1168, 0, 302, 40, 445, 0, 0 cpuset: 72, 0, 287, 593, 430, 0, 0 cyclic_id_cache: 64, 0, 0, 0, 0, 0, 0 audit_record: 1248, 0, 0, 0, 0, 0, 0 mbuf_packet: 256, 25720710, 0, 10, 0, 0, 0 mbuf: 256, 25720710, 1, 134, 1, 0, 0 mbuf_cluster: 2048, 4018860, 0, 0, 0, 0, 0 mbuf_jumbo_page: 4096, 2009429, 0, 0, 0, 0, 0 mbuf_jumbo_9k: 9216, 1786158, 0, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 1339616, 0, 0, 0, 0, 0 mbuf_ext_refcnt: 4, 0, 0, 0, 0, 0, 0 sendfile_sync: 128, 0, 0, 0, 0, 0, 0 dtrace_state_cache: 4096, 0, 0, 0, 0, 0, 0 g_bio: 248, 0, 0, 672, 15403, 0, 0 DMAR_MAP_ENTRY: 120, 0, 0, 0, 0, 0, 0 ttyinq: 160, 0, 15, 57, 15, 0, 0 ttyoutq: 256, 0, 8, 67, 8, 0, 0 ata_request: 336, 0, 0, 88, 39, 0, 0 vtnet_tx_hdr: 24, 0, 0, 0, 0, 0, 0 cryptop: 88, 0, 0, 0, 0, 0, 0 cryptodesc: 72, 0, 0, 0, 0, 0, 0 nv_stack_t: 12288, 0, 2, 1, 6, 0, 0 FPU_save_area: 512, 0, 0, 0, 0, 0, 0 taskq_zone: 48, 0, 0, 415, 20, 0, 0 taskq_zone: 48, 0, 0, 0, 0, 0, 0 VNODE: 472, 0, 332, 164, 333, 0, 0 VNODEPOLL: 112, 0, 0, 0, 0, 0, 0 BUF TRIE: 144, 0, 0, 105948, 0, 0, 0 NAMEI: 1024, 0, 0, 104, 908, 0, 0 S VFS Cache: 108, 0, 248, 767, 294, 0, 0 STS VFS Cache: 148, 0, 0, 0, 0, 0, 0 L VFS Cache: 328, 0, 0, 0, 0, 0, 0 LTS VFS Cache: 368, 0, 0, 0, 0, 0, 0 NCLNODE: 528, 0, 0, 0, 0, 0, 0 DIRHASH: 1024, 0, 0, 0, 0, 0, 0 reference_cache: 40, 0, 0, 0, 0, 0, 0 reference_history_cache: 8, 0, 0, 0, 0, 0, 0 range_seg_cache: 64, 0, 9978, 934, 10433, 0, 0 zio_cache: 920, 0, 6, 814, 12019, 0, 0 zio_link_cache: 48, 0, 3, 1408, 7868, 0, 0 zio_buf_512: 512, 0, 590, 166, 1077, 0, 0 zio_data_buf_512: 512, 0, 54, 65, 56, 0, 0 zio_buf_1024: 1024, 0, 22, 58, 61, 0, 0 zio_data_buf_1024: 1024, 0, 40, 40, 40, 0, 0 zio_buf_1536: 1536, 0, 22, 24, 24, 0, 0 zio_data_buf_1536: 1536, 0, 23, 13, 23, 0, 0 zio_buf_2048: 2048, 0, 35, 25, 64, 0, 0 zio_data_buf_2048: 2048, 0, 14, 12, 15, 0, 0 zio_buf_2560: 2560, 0, 0, 15, 27, 0, 0 zio_data_buf_2560: 2560, 0, 17, 4, 17, 0, 0 zio_buf_3072: 3072, 0, 2, 4, 2, 0, 0 zio_data_buf_3072: 3072, 0, 8, 4, 8, 0, 0 zio_buf_3584: 3584, 0, 1, 0, 1, 0, 0 zio_data_buf_3584: 3584, 0, 2, 0, 2, 0, 0 zio_buf_4096: 4096, 0, 49, 706, 3790, 0, 0 zio_data_buf_4096: 4096, 0, 0, 24, 25, 0, 0 zio_buf_5120: 5120, 0, 1, 0, 1, 0, 0 zio_data_buf_5120: 5120, 0, 4, 0, 4, 0, 0 zio_buf_6144: 6144, 0, 1, 0, 1, 0, 0 zio_data_buf_6144: 6144, 0, 3, 0, 3, 0, 0 zio_buf_7168: 7168, 0, 0, 0, 0, 0, 0 zio_data_buf_7168: 7168, 0, 1, 0, 1, 0, 0 zio_buf_8192: 8192, 0, 1, 13, 187, 0, 0 zio_data_buf_8192: 8192, 0, 2, 1, 4, 0, 0 zio_buf_10240: 10240, 0, 1, 0, 1, 0, 0 zio_data_buf_10240: 10240, 0, 1, 0, 1, 0, 0 zio_buf_12288: 12288, 0, 1, 8, 45, 0, 0 zio_data_buf_12288: 12288, 0, 4, 0, 4, 0, 0 zio_buf_14336: 14336, 0, 0, 0, 0, 0, 0 zio_data_buf_14336: 14336, 0, 1, 0, 1, 0, 0 zio_buf_16384: 16384, 0, 398, 23, 543, 0, 0 zio_data_buf_16384: 16384, 0, 2, 0, 2, 0, 0 zio_buf_20480: 20480, 0, 0, 5, 19, 0, 0 zio_data_buf_20480: 20480, 0, 3, 0, 3, 0, 0 zio_buf_24576: 24576, 0, 0, 9, 16, 0, 0 zio_data_buf_24576: 24576, 0, 5, 0, 5, 0, 0 zio_buf_28672: 28672, 0, 0, 14, 83, 0, 0 zio_data_buf_28672: 28672, 0, 4, 0, 4, 0, 0 zio_buf_32768: 32768, 0, 0, 2, 3, 0, 0 zio_data_buf_32768: 32768, 0, 2, 0, 2, 0, 0 zio_buf_36864: 36864, 0, 1, 5, 8, 0, 0 zio_data_buf_36864: 36864, 0, 1, 2, 3, 0, 0 zio_buf_40960: 40960, 0, 0, 3, 3, 0, 0 zio_data_buf_40960: 40960, 0, 2, 0, 2, 0, 0 zio_buf_45056: 45056, 0, 0, 1, 1, 0, 0 zio_data_buf_45056: 45056, 0, 1, 0, 1, 0, 0 zio_buf_49152: 49152, 0, 0, 0, 0, 0, 0 zio_data_buf_49152: 49152, 0, 0, 0, 0, 0, 0 zio_buf_53248: 53248, 0, 0, 1, 1, 0, 0 zio_data_buf_53248: 53248, 0, 1, 0, 1, 0, 0 zio_buf_57344: 57344, 0, 0, 0, 0, 0, 0 zio_data_buf_57344: 57344, 0, 0, 0, 0, 0, 0 zio_buf_61440: 61440, 0, 0, 0, 0, 0, 0 zio_data_buf_61440: 61440, 0, 0, 0, 0, 0, 0 zio_buf_65536: 65536, 0, 0, 0, 0, 0, 0 zio_data_buf_65536: 65536, 0, 0, 0, 0, 0, 0 zio_buf_69632: 69632, 0, 0, 2, 2, 0, 0 zio_data_buf_69632: 69632, 0, 2, 0, 2, 0, 0 zio_buf_73728: 73728, 0, 0, 1, 1, 0, 0 zio_data_buf_73728: 73728, 0, 1, 0, 1, 0, 0 zio_buf_77824: 77824, 0, 0, 0, 0, 0, 0 zio_data_buf_77824: 77824, 0, 0, 0, 0, 0, 0 zio_buf_81920: 81920, 0, 0, 0, 0, 0, 0 zio_data_buf_81920: 81920, 0, 1, 0, 1, 0, 0 zio_buf_86016: 86016, 0, 0, 0, 0, 0, 0 zio_data_buf_86016: 86016, 0, 0, 0, 0, 0, 0 zio_buf_90112: 90112, 0, 0, 2, 2, 0, 0 zio_data_buf_90112: 90112, 0, 0, 1, 1, 0, 0 zio_buf_94208: 94208, 0, 0, 0, 0, 0, 0 zio_data_buf_94208: 94208, 0, 0, 0, 0, 0, 0 zio_buf_98304: 98304, 0, 1, 1, 2, 0, 0 zio_data_buf_98304: 98304, 0, 1, 0, 1, 0, 0 zio_buf_102400: 102400, 0, 0, 0, 0, 0, 0 zio_data_buf_102400: 102400, 0, 0, 0, 0, 0, 0 zio_buf_106496: 106496, 0, 0, 0, 0, 0, 0 zio_data_buf_106496: 106496, 0, 0, 0, 0, 0, 0 zio_buf_110592: 110592, 0, 0, 1, 1, 0, 0 zio_data_buf_110592: 110592, 0, 1, 0, 1, 0, 0 zio_buf_114688: 114688, 0, 0, 3, 14, 0, 0 zio_data_buf_114688: 114688, 0, 0, 0, 0, 0, 0 zio_buf_118784: 118784, 0, 0, 0, 0, 0, 0 zio_data_buf_118784: 118784, 0, 0, 0, 0, 0, 0 zio_buf_122880: 122880, 0, 0, 0, 0, 0, 0 zio_data_buf_122880: 122880, 0, 0, 0, 0, 0, 0 zio_buf_126976: 126976, 0, 0, 0, 0, 0, 0 zio_data_buf_126976: 126976, 0, 0, 0, 0, 0, 0 zio_buf_131072: 131072, 0, 1, 11, 61, 0, 0 zio_data_buf_131072: 131072, 0, 39, 5, 48, 0, 0 lz4_ctx: 16384, 0, 0, 0, 0, 0, 0 sa_cache: 80, 0, 272, 757, 272, 0, 0 dnode_t: 1096, 0, 746, 37, 1863, 0, 0 dmu_buf_impl_t: 336, 0, 1270, 127, 1729, 0, 0 arc_buf_hdr_t: 328, 0, 881, 163, 1332, 0, 0 arc_buf_t: 72, 0, 880, 770, 1359, 0, 0 zil_lwb_cache: 192, 0, 0, 0, 0, 0, 0 zfs_znode_cache: 368, 0, 272, 138, 272, 0, 0 pipe: 744, 0, 0, 40, 13, 0, 0 Mountpoints: 816, 0, 24, 41, 24, 0, 0 procdesc: 128, 0, 0, 0, 0, 0, 0 ksiginfo: 112, 0, 26, 1024, 26, 0, 0 itimer: 352, 0, 0, 0, 0, 0, 0 KNOTE: 128, 0, 0, 0, 0, 0, 0 socket: 696, 2062880, 0, 0, 0, 0, 0 ipq: 56, 125599, 0, 0, 0, 0, 0 udp_inpcb: 392, 2062880, 0, 0, 0, 0, 0 udpcb: 16, 2062965, 0, 0, 0, 0, 0 tcp_inpcb: 392, 2062880, 0, 0, 0, 0, 0 tcpcb: 1024, 2062880, 0, 0, 0, 0, 0 tcptw: 88, 27810, 0, 0, 0, 0, 0 syncache: 160, 15360, 0, 0, 0, 0, 0 hostcache: 136, 15370, 0, 0, 0, 0, 0 tcpreass: 40, 251262, 0, 0, 0, 0, 0 sackhole: 32, 0, 0, 0, 0, 0, 0 sctp_ep: 1408, 2062880, 0, 0, 0, 0, 0 sctp_asoc: 2352, 40000, 0, 0, 0, 0, 0 sctp_laddr: 48, 80012, 0, 0, 0, 0, 0 sctp_raddr: 728, 80000, 0, 0, 0, 0, 0 sctp_chunk: 136, 400026, 0, 0, 0, 0, 0 sctp_readq: 104, 400026, 0, 0, 0, 0, 0 sctp_stream_msg_out: 104, 400026, 0, 0, 0, 0, 0 sctp_asconf: 40, 400059, 0, 0, 0, 0, 0 sctp_asconf_ack: 48, 400060, 0, 0, 0, 0, 0 ripcb: 392, 2062880, 0, 0, 0, 0, 0 unpcb: 240, 2062880, 0, 0, 0, 0, 0 rtentry: 200, 0, 0, 0, 0, 0, 0 selfd: 56, 0, 0, 0, 0, 0, 0 SWAPMETA: 288, 8037731, 0, 0, 0, 0, 0 ------------------------------------------------------------------------ vmstat -i interrupt total rate irq6: fdc0 22 0 irq14: ata0 55 0 irq17: uhci0 ehci0 55 0 irq18: uhci2+ 304 0 irq19: uhci1 ahci0+ 5075 13 cpu0:timer 2410 6 irq259: hdac0 9 0 cpu6:timer 442 1 cpu1:timer 404 1 cpu2:timer 482 1 cpu7:timer 548 1 cpu5:timer 477 1 cpu4:timer 484 1 cpu3:timer 765 2 Total 11532 31 ------------------------------------------------------------------------ pstat -T 8/2062880 files 0M/147455M swap space ------------------------------------------------------------------------ pstat -s Device 512-blocks Used Avail Capacity /dev/#C:0xc0 50331392 0 50331392 0% /dev/ad14p2 50331392 0 50331392 0% /dev/usb/3.2.1 50331392 0 50331392 0% /dev/#C:0xd0 50331392 0 50331392 0% /dev/#C:0xd8 50331392 0 50331392 0% /dev/#C:0xe0 50331392 0 50331392 0% Total 301988352 0 301988352 0% ------------------------------------------------------------------------ iostat tty ada0 ada1 ada2 cpu tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 0 15 15.18 60 0.89 15.40 65 0.98 14.86 63 0.91 0 0 2 1 97 ------------------------------------------------------------------------ ipcs -a Message Queues: T ID KEY MODE OWNER GROUP CREATOR CGROUP CBYTES QNUM QBYTES LSPID LRPID STIME RTIME CTIME Shared Memory: T ID KEY MODE OWNER GROUP CREATOR CGROUP NATTCH SEGSZ CPID LPID ATIME DTIME CTIME Semaphores: T ID KEY MODE OWNER GROUP CREATOR CGROUP NSEMS OTIME CTIME ------------------------------------------------------------------------ ipcs -T msginfo: msgmax: 16384 (max characters in a message) msgmni: 40 (# of message queues) msgmnb: 2048 (max characters in a message queue) msgtql: 40 (max # of messages in system) msgssz: 8 (size of a message segment) msgseg: 2048 (# of message segments in system) shminfo: shmmax: 536870912 (max shared memory segment size) shmmin: 1 (min shared memory segment size) shmmni: 192 (max number of shared memory identifiers) shmseg: 128 (max shared memory segments per process) shmall: 131072 (max amount of shared memory in pages) seminfo: semmni: 50 (# of semaphore identifiers) semmns: 340 (# of semaphores in system) semmnu: 150 (# of undo structures in system) semmsl: 340 (max # of semaphores per id) semopm: 100 (max # of operations per semop call) semume: 50 (max # of undo entries per process) semusz: 632 (size in bytes of undo structure) semvmx: 32767 (semaphore maximum value) semaem: 16384 (adjust on exit max value) ------------------------------------------------------------------------ nfsstat Client Info: Rpc Counts: Getattr Setattr Lookup Readlink Read Write Create Remove 2 0 0 0 0 0 0 0 Rename Link Symlink Mkdir Rmdir Readdir RdirPlus Access 0 0 0 0 0 0 0 0 Mknod Fsstat Fsinfo PathConf Commit 0 1 1 0 0 Rpc Info: TimedOut Invalid X Replies Retries Requests 0 0 0 0 4 Cache Info: Attr Hits Misses Lkup Hits Misses BioR Hits Misses BioW Hits Misses 0 0 0 0 0 0 0 0 BioRLHits Misses BioD Hits Misses DirE Hits Misses Accs Hits Misses 0 0 0 0 0 0 0 0 Server Info: Getattr Setattr Lookup Readlink Read Write Create Remove 0 0 0 0 0 0 0 0 Rename Link Symlink Mkdir Rmdir Readdir RdirPlus Access 0 0 0 0 0 0 0 0 Mknod Fsstat Fsinfo PathConf Commit 0 0 0 0 0 Server Ret-Failed 0 Server Faults 0 Server Cache Stats: Inprog Idem Non-idem Misses 0 0 0 0 Server Write Gathering: WriteOps WriteRPC Opsaved 0 0 0 ------------------------------------------------------------------------ netstat -s tcp: 0 packets sent 0 data packets (0 bytes) 0 data packets (0 bytes) retransmitted 0 data packets unnecessarily retransmitted 0 resends initiated by MTU discovery 0 ack-only packets (0 delayed) 0 URG only packets 0 window probe packets 0 window update packets 0 control packets 0 packets received 0 acks (for 0 bytes) 0 duplicate acks 0 acks for unsent data 0 packets (0 bytes) received in-sequence 0 completely duplicate packets (0 bytes) 0 old duplicate packets 0 packets with some dup. data (0 bytes duped) 0 out-of-order packets (0 bytes) 0 packets (0 bytes) of data after window 0 window probes 0 window update packets 0 packets received after close 0 discarded for bad checksums 0 discarded for bad header offset fields 0 discarded because packet too short 0 discarded due to memory problems 0 connection requests 0 connection accepts 0 bad connection attempts 0 listen queue overflows 0 ignored RSTs in the windows 0 connections established (including accepts) 0 connections closed (including 0 drops) 0 connections updated cached RTT on close 0 connections updated cached RTT variance on close 0 connections updated cached ssthresh on close 0 embryonic connections dropped 0 segments updated rtt (of 0 attempts) 0 retransmit timeouts 0 connections dropped by rexmit timeout 0 persist timeouts 0 connections dropped by persist timeout 0 Connections (fin_wait_2) dropped because of timeout 0 keepalive timeouts 0 keepalive probes sent 0 connections dropped by keepalive 0 correct ACK header predictions 0 correct data packet header predictions 0 syncache entries added 0 retransmitted 0 dupsyn 0 dropped 0 completed 0 bucket overflow 0 cache overflow 0 reset 0 stale 0 aborted 0 badack 0 unreach 0 zone failures 0 cookies sent 0 cookies received 0 hostcache entries added 0 bucket overflow 0 SACK recovery episodes 0 segment rexmits in SACK recovery episodes 0 byte rexmits in SACK recovery episodes 0 SACK options (SACK blocks) received 0 SACK options (SACK blocks) sent 0 SACK scoreboard overflow 0 packets with ECN CE bit set 0 packets with ECN ECT(0) bit set 0 packets with ECN ECT(1) bit set 0 successful ECN handshakes 0 times ECN reduced the congestion window udp: 0 datagrams received 0 with incomplete header 0 with bad data length field 0 with bad checksum 0 with no checksum 0 dropped due to no socket 0 broadcast/multicast datagrams undelivered 0 dropped due to full socket buffers 0 not for hashed pcb 0 delivered 0 datagrams output 0 times multicast source filter matched ip: 0 total packets received 0 bad header checksums 0 with size smaller than minimum 0 with data size < data length 0 with ip length > max ip packet size 0 with header length < data size 0 with data length < header length 0 with bad options 0 with incorrect version number 0 fragments received 0 fragments dropped (dup or out of space) 0 fragments dropped after timeout 0 packets reassembled ok 0 packets for this host 0 packets for unknown/unsupported protocol 0 packets forwarded (0 packets fast forwarded) 0 packets not forwardable 0 packets received for unknown multicast group 0 redirects sent 0 packets sent from this host 0 packets sent with fabricated ip header 0 output packets dropped due to no bufs, etc. 0 output packets discarded due to no route 0 output datagrams fragmented 0 fragments created 0 datagrams that can't be fragmented 0 tunneling packets that can't find gif 0 datagrams with bad address in header icmp: 0 calls to icmp_error 0 errors not generated in response to an icmp message 0 messages with bad code fields 0 messages less than the minimum length 0 messages with bad checksum 0 messages with bad length 0 multicast echo requests ignored 0 multicast timestamp requests ignored 0 message responses generated 0 invalid return addresses 0 no return routes igmp: 0 messages received 0 messages received with too few bytes 0 messages received with wrong TTL 0 messages received with bad checksum 0 V1/V2 membership queries received 0 V3 membership queries received 0 membership queries received with invalid field(s) 0 general queries received 0 group queries received 0 group-source queries received 0 group-source queries dropped 0 membership reports received 0 membership reports received with invalid field(s) 0 membership reports received for groups to which we belong 0 V3 reports received without Router Alert 0 membership reports sent arp: 0 ARP requests sent 0 ARP replies sent 0 ARP requests received 0 ARP replies received 0 ARP packets received 0 total packets dropped due to no ARP entry 0 ARP entrys timed out 0 Duplicate IPs seen ip6: 0 total packets received 0 with size smaller than minimum 0 with data size < data length 0 with bad options 0 with incorrect version number 0 fragments received 0 fragments dropped (dup or out of space) 0 fragments dropped after timeout 0 fragments that exceeded limit 0 packets reassembled ok 0 packets for this host 0 packets forwarded 0 packets not forwardable 0 redirects sent 0 packets sent from this host 0 packets sent with fabricated ip header 0 output packets dropped due to no bufs, etc. 0 output packets discarded due to no route 0 output datagrams fragmented 0 fragments created 0 datagrams that can't be fragmented 0 packets that violated scope rules 0 multicast packets which we don't join Mbuf statistics: 0 one mbuf 0 one ext mbuf 0 two or more ext mbuf 0 packets whose headers are not contiguous 0 tunneling packets that can't find gif 0 packets discarded because of too many headers 0 failures of source address selection Source addresses selection rule applied: icmp6: 0 calls to icmp6_error 0 errors not generated in response to an icmp6 message 0 errors not generated because of rate limitation 0 messages with bad code fields 0 messages < minimum length 0 bad checksums 0 messages with bad length Histogram of error messages to be generated: 0 no route 0 administratively prohibited 0 beyond scope 0 address unreachable 0 port unreachable 0 packet too big 0 time exceed transit 0 time exceed reassembly 0 erroneous header field 0 unrecognized next header 0 unrecognized option 0 redirect 0 unknown 0 message responses generated 0 messages with too many ND options 0 messages with bad ND options 0 bad neighbor solicitation messages 0 bad neighbor advertisement messages 0 bad router solicitation messages 0 bad router advertisement messages 0 bad redirect messages 0 path MTU changes rip6: 0 messages received 0 checksum calculations on inbound 0 messages with bad checksum 0 messages dropped due to no socket 0 multicast messages dropped due to no socket 0 messages dropped due to full socket buffers 0 delivered 0 datagrams output ------------------------------------------------------------------------ netstat -m netstat: invalid address (0x0) 1/144/145 mbufs in use (current/cache/total) 18446744073709551606/10/0/4018860 mbuf clusters in use (current/cache/total/max) 0/10 mbuf+clusters out of packet secondary zone in use (current/cache) 0/0/0/2009429 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/1786158 9k jumbo clusters in use (current/cache/total/max) 0/0/0/1339616 16k jumbo clusters in use (current/cache/total/max) 18014398509481964K/56K/36K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) ------------------------------------------------------------------------ netstat -anr Routing tables Internet: Destination Gateway Flags Netif Expire Internet6: Destination Gateway Flags Netif Expire ------------------------------------------------------------------------ netstat -anA ------------------------------------------------------------------------ netstat -aL ------------------------------------------------------------------------ fstat fstat: can't read file 1 at 0x200007fffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 4 at 0x780000ffff fstat: can't read file 7 at 0x200007fffffffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0x200007fffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 4 at 0x780000ffff fstat: can't read file 7 at 0x200007fffffffff fstat: can't read file 8 at 0x4000000001fffff fstat: can't read file 10 at 0x780000ffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0x200007fffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 4 at 0x780000ffff fstat: can't read file 7 at 0x200007fffffffff fstat: can't read file 8 at 0x4000000001fffff fstat: can't read file 10 at 0x780000ffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 USER CMD PID FD MOUNT INUM MODE SZ|DV R/W root zfs 75 root - - error - root zfs 75 wd - - error - root zfs 75 text - - error - root zfs 75 ctty /dev 13 crw------- console rw root zfs 75 0 /dev 13 crw------- console rw root zfs 75 6 /dev 13 crw------- console rw root sh 72 root - - error - root sh 72 wd - - error - root sh 72 text - - error - root sh 72 ctty /dev 13 crw------- console rw root sh 72 0 /dev 13 crw------- console rw root sh 72 6 /dev 13 crw------- console rw root sh 24 root - - error - root sh 24 wd - - error - root sh 24 text - - error - root sh 24 ctty /dev 13 crw------- console rw root sh 24 0 /dev 13 crw------- console rw root sh 24 6 /dev 13 crw------- console rw root init 1 root - - error - root init 1 wd - - error - root init 1 text - - error - root kernel 0 root - - error - root kernel 0 wd - - error - ------------------------------------------------------------------------ dmesg Copyright (c) 1992-2014 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.0-CURRENT #1 r260896: Mon Jan 20 12:31:46 CST 2014 root@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER amd64 FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 info: [drm] Initialized drm 1.1.0 20060810 CPU: Intel(R) Xeon(R) CPU E5410 @ 2.33GHz (2327.55-MHz K8-class CPU) Origin="GenuineIntel" Id=0x10676 Family=0x6 Model=0x17 Stepping=6 Features=0xbfebfbff Features2=0xce3bd AMD Features=0x20100800 AMD Features2=0x1 TSC: P-state invariant, performance statistics real memory = 68719476736 (65536 MB) avail memory = 65641644032 (62600 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs FreeBSD/SMP: 2 package(s) x 4 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP): APIC ID: 3 cpu4 (AP): APIC ID: 4 cpu5 (AP): APIC ID: 5 cpu6 (AP): APIC ID: 6 cpu7 (AP): APIC ID: 7 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard random: initialized kbd1 at kbdmux0 cryptosoft0: on motherboard acpi0: on motherboard acpi0: Power Button (fixed) unknown: I/O range not supported cpu0: on acpi0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 cpu4: on acpi0 cpu5: on acpi0 cpu6: on acpi0 cpu7: on acpi0 hpet0: iomem 0xfed00000-0xfed003ff irq 0,8 on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 atrtc0: port 0x70-0x71 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: at device 2.0 on pci0 pci1: on pcib1 pcib2: irq 16 at device 0.0 on pci1 pci2: on pcib2 pcib3: irq 16 at device 0.0 on pci2 pci3: on pcib3 pcib4: at device 0.0 on pci3 pci4: on pcib4 pcib5: at device 0.2 on pci3 pci5: on pcib5 pcib6: irq 18 at device 2.0 on pci2 pci6: on pcib6 em0: port 0x2000-0x201f mem 0xd9220000-0xd923ffff,0xd9200000-0xd921ffff irq 18 at device 0.0 on pci6 em0: Using an MSI interrupt em0: Ethernet address: 00:30:48:f2:29:9c em1: port 0x2020-0x203f mem 0xd9260000-0xd927ffff,0xd9240000-0xd925ffff irq 19 at device 0.1 on pci6 em1: Using an MSI interrupt em1: Ethernet address: 00:30:48:f2:29:9d pcib7: at device 0.3 on pci1 pci7: on pcib7 pcib8: at device 4.0 on pci0 pci8: on pcib8 vgapci0: port 0x3000-0x307f mem 0xd8000000-0xd8ffffff,0xc0000000-0xc7ffffff,0xc8000000-0xc9ffffff irq 16 at device 0.0 on pci8 nvidia0: on vgapci0 vgapci0: child nvidia0 requested pci_enable_io vgapci0: child nvidia0 requested pci_enable_io hdac0: mem 0xd9000000-0xd9003fff irq 17 at device 0.1 on pci8 pcib9: at device 6.0 on pci0 pci9: on pcib9 pci0: at device 8.0 (no driver attached) pcib10: irq 17 at device 28.0 on pci0 pci10: on pcib10 pcib11: irq 16 at device 0.0 on pci10 pci11: on pcib11 pcm0: port 0x4080-0x409f,0x4000-0x407f irq 16 at device 0.0 on pci11 pcm0: system configuration SubVendorID: 0x1412, SubDeviceID: 0x2403 XIN2 Clock Source: 24.576MHz(96kHz*256) MPU-401 UART(s) #: not implemented ADC #: 1 and SPDIF receiver connected DAC #: 4 Multi-track converter type: AC'97(SDATA_OUT:packed) S/PDIF(IN/OUT): 1/1 ID# 0x00 GPIO(mask/dir/state): 0xff/0xff/0xff uhci0: port 0x1800-0x181f irq 17 at device 29.0 on pci0 usbus0 on uhci0 uhci1: port 0x1820-0x183f irq 19 at device 29.1 on pci0 usbus1 on uhci1 uhci2: port 0x1840-0x185f irq 18 at device 29.2 on pci0 usbus2 on uhci2 ehci0: mem 0xd9600400-0xd96007ff irq 17 at device 29.7 on pci0 usbus3: EHCI version 1.0 usbus3 on ehci0 pcib12: at device 30.0 on pci0 pci12: on pcib12 vgapci1: port 0x5000-0x50ff mem 0xd0000000-0xd7ffffff,0xd9300000-0xd930ffff irq 18 at device 1.0 on pci12 drmn1: on vgapci1 info: [drm] RADEON_IS_PCI info: [drm] initializing kernel modesetting (RV100 0x1002:0x515E 0x15D9:0x8080). info: [drm] register mmio base: 0xD9300000 info: [drm] register mmio size: 65536 info: [drm] radeon_atrm_get_bios: ===> Try ATRM... info: [drm] radeon_atrm_get_bios: pci_find_class() found: 0:8:0:0, vendor=10de, device=104a info: [drm] radeon_atrm_get_bios: Get ACPI device handle info: [drm] radeon_acpi_vfct_bios: ===> Try VFCT... info: [drm] radeon_acpi_vfct_bios: Get "VFCT" ACPI table info: [drm] radeon_acpi_vfct_bios: Failed to get "VFCT" table: AE_NOT_FOUND info: [drm] igp_read_bios_from_vram: ===> Try IGP's VRAM... info: [drm] igp_read_bios_from_vram: VRAM base address: 0xd0000000 info: [drm] igp_read_bios_from_vram: Map address: 0xfffff800d0000000 (262144 bytes) info: [drm] igp_read_bios_from_vram: Incorrect BIOS signature: 0x0000 info: [drm] radeon_read_bios: ===> Try PCI Expansion ROM... info: [drm] radeon_read_bios: Map address: 0xfffff800000c0000 (131072 bytes) drmn1: info: VRAM: 128M 0x00000000D0000000 - 0x00000000D7FFFFFF (16M used) drmn1: info: GTT: 512M 0x00000000B0000000 - 0x00000000CFFFFFFF info: [drm] Supports vblank timestamp caching Rev 1 (10.10.2010). info: [drm] Driver supports precise vblank timestamp query. info: [drm] radeon: irq initialized. info: [drm] Detected VRAM RAM=128M, BAR=128M info: [drm] RAM width 64bits SDR [TTM] Zone kernel: Available graphics memory: 33006090 kiB [TTM] Zone dma32: Available graphics memory: 2097152 kiB [TTM] Initializing pool allocator info: [drm] radeon: 16M of VRAM memory ready info: [drm] radeon: 512M of GTT memory ready. info: [drm] GART: num cpu pages 131072, num gpu pages 131072 info: [drm] PCI GART of 512M enabled (table at 0x0000000011200000). drmn1: info: WB disabled drmn1: info: fence driver on ring 0 use gpu addr 0x00000000b0000000 and cpu addr 0x0xfffff8001083c000 info: [drm] Loading R100 Microcode info: [drm] radeon: ring at 0x00000000B0001000 info: [drm] ring test succeeded in 1 usecs info: [drm] ib test succeeded in 0 usecs info: [drm] radeon_device_init: Taking over the fictitious range 0xd0000000-0xd4000000 iicbus0: on iicbb0 addr 0xff iic0: on iicbus0 iicbus1: on iicbb1 addr 0xff iic1: on iicbus1 iicbus2: on iicbb2 addr 0xff iic2: on iicbus2 iicbus3: on iicbb3 addr 0xff iic3: on iicbus3 info: [drm] Radeon Display Connectors info: [drm] Connector 0: info: [drm] VGA-1 info: [drm] DDC: 0x60 0x60 0x60 0x60 0x60 0x60 0x60 0x60 info: [drm] Encoders: info: [drm] CRT1: INTERNAL_DAC1 info: [drm] Connector 1: info: [drm] DVI-I-1 info: [drm] HPD2 info: [drm] DDC: 0x6c 0x6c 0x6c 0x6c 0x6c 0x6c 0x6c 0x6c info: [drm] Encoders: info: [drm] CRT2: INTERNAL_DAC2 info: [drm] DFP2: INTERNAL_DVO1 composite sync not supported composite sync not supported info: [drm] fb mappable at 0xD0040000 info: [drm] vram apper at 0xD0000000 info: [drm] size 2076672 info: [drm] fb depth is 8 info: [drm] pitch is 1920 fbd1 on drmn1 vt_allocate: Replace existing VT driver. info: [drm] Initialized radeon 2.29.0 20080528 vgapci1: Boot video device isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1860-0x186f at device 31.1 on pci0 ata0: at channel 0 on atapci0 ahci0: port 0x18a0-0x18a7,0x1874-0x1877,0x1878-0x187f,0x1870-0x1873,0x1880-0x189f mem 0xd9600800-0xd9600bff irq 19 at device 31.2 on pci0 ahci0: AHCI v1.10 with 6 3Gbps ports, Port Multiplier supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahcich4: at channel 4 on ahci0 ahcich5: at channel 5 on ahci0 ichsmb0: port 0x1100-0x111f irq 19 at device 31.3 on pci0 smbus0: on ichsmb0 acpi_button0: on acpi0 ipmi0: port 0xca2-0xca3 on acpi0 ipmi0: KCS mode found at io 0xca2 on acpi atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fd0: <1440-KB 3.5" drive> on fdc0 drive 0 ppc0: port 0x378-0x37f,0x778-0x77f irq 7 drq 3 on acpi0 ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/9 bytes threshold ppbus0: on ppc0 lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 ichwd0 on isa0 orm0: at iomem 0xc0000-0xcafff on isa0 coretemp0: on cpu0 est0: on cpu0 p4tcc0: on cpu0 coretemp1: on cpu1 est1: on cpu1 p4tcc1: on cpu1 coretemp2: on cpu2 est2: on cpu2 p4tcc2: on cpu2 coretemp3: on cpu3 est3: on cpu3 p4tcc3: on cpu3 coretemp4: on cpu4 est4: on cpu4 p4tcc4: on cpu4 coretemp5: on cpu5 est5: on cpu5 p4tcc5: on cpu5 coretemp6: on cpu6 est6: on cpu6 p4tcc6: on cpu6 coretemp7: on cpu7 est7: on cpu7 p4tcc7: on cpu7 ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Timecounters tick every 1.000 msec vboxdrv: fAsync=0 offMin=0x387 offMax=0x491 hdacc0: at cad 0 on hdac0 hdaa0: at nid 1 on hdacc0 pcm1: at nid 4 on hdaa0 pcm2: at nid 5 on hdaa0 random: unblocking device. usbus0: 12Mbps Full Speed USB v1.0 usbus1: 12Mbps Full Speed USB v1.0 usbus2: 12Mbps Full Speed USB v1.0 usbus3: 480Mbps High Speed USB v2.0 ugen3.1: at usbus3 uhub0: on usbus3 ugen2.1: at usbus2 uhub1: on usbus2 ugen1.1: at usbus1 uhub2: on usbus1 ugen0.1: at usbus0 uhub3: on usbus0 ipmi0: IPMI device rev. 1, firmware rev. 1.64, version 2.0 ata0: DMA limited to UDMA33, controller found non-ATA66 cable ipmi0: Number of channels 8 ipmi0: Attached watchdog uhub2: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered uhub3: 2 ports with 2 removable, self powered ada0 at ahcich0 bus 0 scbus1 target 0 lun 0 ada0: ATA-8 SATA 3.x device ada0: Serial Number 5YDA1ZL4 ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada0: quirks=0x1<4K> ada0: Previously was known as ad4 ada1 at ahcich1 bus 0 scbus2 target 0 lun 0 ada1: ATA-8 SATA 3.x device ada1: Serial Number 5YD6FPLG ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada1: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada1: quirks=0x1<4K> ada1: Previously was known as ad6 ada2 at ahcich2 bus 0 scbus3 target 0 lun 0 ada2: ATA-8 SATA 3.x device ada2: Serial Number 5YDA3PC5 ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada2: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada2: quirks=0x1<4K> ada2: Previously was known as ad8 ada3 at ahcich3 bus 0 scbus4 target 0 lun 0 ada3: ATA-8 SATA 3.x device ada3: Serial Number 5YD9Y0P4 ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada3: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada3: quirks=0x1<4K> ada3: Previously was known as ad10 ada4 at ahcich4 bus 0 scbus5 target 0 lun 0 ada4: ATA-8 SATA 3.x device ada4: Serial Number 5YDA1R5W ada4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada4: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada4: quirks=0x1<4K> ada4: Previously was known as ad12 ada5 at ahcich5 bus 0 scbus6 target 0 lun 0 ada5: ATA-8 SATA 3.x device ada5: Serial Number 5YD5RBS8 ada5: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada5: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada5: quirks=0x1<4K> ada5: Previously was known as ad14 cd0 at ata0 bus 0 scbus0 target 0 lun 0 cd0: Removable CD-ROM SCSI-0 device cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present Netvsc initializing... done! SMP: AP CPU #6 Launched! SMP: AP CPU #1 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #7 Launched! SMP: AP CPU #5 Launched! SMP: AP CPU #4 Launched! SMP: AP CPU #3 Launched! Root mount waiting for: usbus3 uhub0: 6 ports with 6 removable, self powered Root mount waiting for: usbus3 Root mount waiting for: usbus3 ugen3.2: at usbus3 ukbd0: on usbus3 kbd2 at ukbd0 Trying to mount root from zfs:zroot/ROOT/default []... ugen0.2: at usbus0 ugen1.2: at usbus1 Setting hostuuid: 53d19f64-d663-a017-8922-0030488e9ff3. Setting hostid: 0xf53a926e. Entropy harvesting: interrupts ethernet point_to_point swi. Starting file system checks: Mounting local file systems:. panic: solaris assert: l->l_phys->l_hdr.lh_block_type == ((1ULL << 63) + 0) (0x0 == 0x8000000000000000), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c, line: 534 cpuid = 7 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe100c499090 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe100c499140 vpanic() at vpanic+0x126/frame 0xfffffe100c499180 panic() at panic+0x43/frame 0xfffffe100c4991e0 assfail3() at assfail3+0x2f/frame 0xfffffe100c499200 zap_get_leaf_byblk() at zap_get_leaf_byblk+0x4e1/frame 0xfffffe100c499250 zap_deref_leaf() at zap_deref_leaf+0xd1/frame 0xfffffe100c4992a0 fzap_cursor_retrieve() at fzap_cursor_retrieve+0xce/frame 0xfffffe100c499310 zap_cursor_retrieve() at zap_cursor_retrieve+0x20a/frame 0xfffffe100c499390 ddt_zap_walk() at ddt_zap_walk+0x5e/frame 0xfffffe100c499650 ddt_walk() at ddt_walk+0x78/frame 0xfffffe100c499680 dsl_scan_sync() at dsl_scan_sync+0x6ec/frame 0xfffffe100c4999e0 spa_sync() at spa_sync+0x5c4/frame 0xfffffe100c499ad0 txg_sync_thread() at txg_sync_thread+0x256/frame 0xfffffe100c499bb0 fork_exit() at fork_exit+0x84/frame 0xfffffe100c499bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe100c499bf0 --- trap 0, rip = 0, rsp = 0xfffffe100c499cb0, rbp = 0 --- KDB: enter: panic Uptime: 14s Dumping 2263 out of 64465 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% ------------------------------------------------------------------------ kernel config options CONFIG_AUTOGENERATED ident VT-LER machine amd64 cpu HAMMER makeoptions WITH_CTF=1 makeoptions DEBUG=-g options ZFS options XENHVM options USB_DEBUG options ATH_ENABLE_11N options AH_AR5416_INTERRUPT_MITIGATION options AH_SUPPORT_AR5416 options IEEE80211_SUPPORT_MESH options IEEE80211_AMPDU_AGE options IEEE80211_DEBUG options SC_PIXEL_MODE options VESA options AHD_REG_PRETTY_PRINT options AHC_REG_PRETTY_PRINT options ATA_STATIC_ID options ACPI_DMAR options SMP options MALLOC_DEBUG_MAXZONES=8 options INVARIANT_SUPPORT options INVARIANTS options DEADLKRES options GDB options DDB options KDB_TRACE options KDB options INCLUDE_CONFIG_FILE options DDB_CTF options KDTRACE_HOOKS options KDTRACE_FRAME options MAC options CAPABILITIES options CAPABILITY_MODE options AUDIT options HWPMC_HOOKS options KBD_INSTALL_CDEV options PRINTF_BUFR_SIZE=128 options _KPOSIX_PRIORITY_SCHEDULING options SYSVSEM options SYSVMSG options SYSVSHM options STACK options KTRACE options SCSI_DELAY=5000 options COMPAT_FREEBSD7 options COMPAT_FREEBSD6 options COMPAT_FREEBSD5 options COMPAT_FREEBSD4 options COMPAT_FREEBSD32 options GEOM_LABEL options GEOM_RAID options GEOM_PART_GPT options PSEUDOFS options PROCFS options CD9660 options MSDOSFS options NFS_ROOT options NFSLOCKD options NFSD options NFSCL options MD_ROOT options QUOTA options UFS_GJOURNAL options UFS_DIRHASH options UFS_ACL options SOFTUPDATES options FFS options SCTP options TCP_OFFLOAD options INET6 options INET options PREEMPTION options SCHED_ULE options NEW_PCIB options GEOM_PART_MBR options GEOM_PART_EBR_COMPAT options GEOM_PART_EBR options GEOM_PART_BSD device isa device mem device io device uart_ns8250 device cpufreq device acpi device pci device fdc device ahci device ata device mvs device siis device ahc device ahd device esp device hptiop device isp device mpt device mps device sym device trm device adv device adw device aic device bt device isci device scbus device ch device da device sa device cd device pass device ses device amr device arcmsr device ciss device dpt device hptmv device hptnr device hptrr device hpt27xx device iir device ips device mly device twa device tws device aac device aacp device aacraid device ida device mfi device mlx device twe device atkbdc device atkbd device psm device kbdmux device splash device agp device cbb device pccard device cardbus device uart device ppc device ppbus device lpt device ppi device puc device bxe device de device em device igb device ixgbe device le device ti device txp device vx device miibus device ae device age device alc device ale device bce device bfe device bge device cas device dc device et device fxp device gem device hme device jme device lge device msk device nfe device nge device pcn device re device rl device sf device sge device sis device sk device ste device stge device tl device tx device vge device vr device wb device xl device cs device ed device ex device ep device fe device sn device xe device wlan device wlan_wep device wlan_ccmp device wlan_tkip device wlan_amrr device an device ath device ath_pci device ath_hal device ath_rate_sample device ipw device iwi device iwn device malo device mwl device ral device wi device wpi device loop device random device padlock_rng device rdrand_rng device ether device vlan device tun device md device gif device faith device firmware device bpf device uhci device ohci device ehci device xhci device usb device ukbd device umass device sound device snd_cmi device snd_csa device snd_emu10kx device snd_es137x device snd_hda device snd_ich device snd_via8233 device mmc device mmcsd device sdhci device virtio device virtio_pci device vtnet device virtio_blk device virtio_scsi device virtio_balloon device hyperv device xenpci device vmx device vt device vt_vga ------------------------------------------------------------------------ ddb capture buffer -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-fs@FreeBSD.ORG Wed Jan 22 16:56:39 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D88DBAEA for ; Wed, 22 Jan 2014 16:56:39 +0000 (UTC) Received: from smtp-int-m.obspm.fr (smtp-int-m.obspm.fr [145.238.187.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 72AEA1CA5 for ; Wed, 22 Jan 2014 16:56:39 +0000 (UTC) Received: from pcjas.obspm.fr (pcjas.obspm.fr [145.238.184.233]) by smtp-int-m.obspm.fr (8.14.4/8.14.4/SIO Observatoire de Paris - 07/2009) with ESMTP id s0MGtE90008546 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Wed, 22 Jan 2014 17:55:16 +0100 Date: Wed, 22 Jan 2014 17:55:14 +0100 From: Albert Shih To: freebsd-fs@freebsd.org Subject: zfs send/recv with lz4 Message-ID: <20140122165514.GA8110@pcjas.obspm.fr> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit User-Agent: Mutt/1.5.22 (2013-10-16) X-Miltered: at smtp-int-m.obspm.fr with ID 52DFF7F2.000 by Joe's j-chkmail (http : // j-chkmail dot ensmp dot fr)! X-j-chkmail-Enveloppe: 52DFF7F2.000/145.238.184.233/pcjas.obspm.fr/pcjas.obspm.fr/ X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jan 2014 16:56:39 -0000 Hi all, I've two servers with ZFS, the first (who got the data actually) don't have lz4 enable (FreeBSD 9.0 now FreeBSD 9.1) and the new one is under FreeBSD-10. So I would like to enable lz4 for the new server. The problem is I would like to have (for the futur) the possibility to make zfs send/recv from the old one (still in production) on the new one. So I should at the beginning do the zfs send/recv. But how can I say the recv got lz4 ? If I create a new zfs partition with lz4 activated I need to se rsync and for that I can't in the futur use zfs send/recv (but maybe I'm wrong about that). Any idea ? Regards. JAS -- Albert SHIH DIO bâtiment 15 Observatoire de Paris 5 Place Jules Janssen 92195 Meudon Cedex France Téléphone : +33 1 45 07 76 26/+33 6 86 69 95 71 xmpp: jas@obspm.fr Heure local/Local time: mer 22 jan 2014 17:52:08 CET From owner-freebsd-fs@FreeBSD.ORG Wed Jan 22 18:10:42 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B8595A13 for ; Wed, 22 Jan 2014 18:10:42 +0000 (UTC) Received: from mail-ob0-x22f.google.com (mail-ob0-x22f.google.com [IPv6:2607:f8b0:4003:c01::22f]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 7FF58132A for ; Wed, 22 Jan 2014 18:10:42 +0000 (UTC) Received: by mail-ob0-f175.google.com with SMTP id wn1so830616obc.6 for ; Wed, 22 Jan 2014 10:10:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=i/g8zd5ENSc0KbAM32FKjfFg9LOuOZHVCzR3X6HIZdQ=; b=mPsBADU4hYFyOoZX/qTtVLsWWcYWowYjpy0etSOrO9fVSQli48GwFB7L8/CpfV0DHr gVTtqNM01PagGSR+D8NvHh8UpKq8Y+ZMEWkN8byKzGuwVvI+Ti4NPn0qLpkxpSpUj1CK Rg/S9rsFYVBEWYFm8v0Y2WKG2g9Hr7VSrP/YJBNBESXE0PUpPhjY3GpcR4DxCzf/dPw7 e9DkR5iBuwnDQg34TCAgT0ADmErOjRd1hnDcrWOlaEpL43EwsX90tAi9tXFjivwTOcU0 Nw5vpMVAqCW7NVebq5AJnbKzxvI4v8Xi6mT34jP+o/QUYUAlVCDB7+eyg4hrCI0alhAc Mx/w== MIME-Version: 1.0 X-Received: by 10.182.223.114 with SMTP id qt18mr2486975obc.61.1390414241838; Wed, 22 Jan 2014 10:10:41 -0800 (PST) Received: by 10.76.132.9 with HTTP; Wed, 22 Jan 2014 10:10:41 -0800 (PST) In-Reply-To: <20140122165514.GA8110@pcjas.obspm.fr> References: <20140122165514.GA8110@pcjas.obspm.fr> Date: Wed, 22 Jan 2014 10:10:41 -0800 Message-ID: Subject: Re: zfs send/recv with lz4 From: Freddie Cash To: Albert Shih Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Filesystems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jan 2014 18:10:42 -0000 On Wed, Jan 22, 2014 at 8:55 AM, Albert Shih wrote: > Hi all, > > I've two servers with ZFS, the first (who got the data actually) don't ha= ve > lz4 enable (FreeBSD 9.0 now FreeBSD 9.1) and the new one is under > FreeBSD-10. > > So I would like to enable lz4 for the new server. The problem is I would > like to have (for the futur) the possibility to make zfs send/recv from t= he > old one (still in production) on the new one. So I should at the beginnin= g > do the zfs send/recv. But how can I say the recv got lz4 ? > > If I create a new zfs partition with lz4 activated I need to se rsync and > for that I can't in the futur use zfs send/recv (but maybe I'm wrong abou= t > that). =E2=80=8B > Unless you are using -R for a replication stream, then the properties of the sending dataset are not part of the send stream, and the properties of the receiving dataset take effect. IOW, create the dataset on the receiving system, set the properties you want (like lz4 compression). Then send the stream from the other box into that filesystem. Also, if you set the properties in the parent dataset on the receiving side, then they'll be inherited automatically in the newly created dataset via the recv process. Works without issues here. Have lzjb set on the sending side, and lz4 on the receiving side. No dedupe on the sender, dedupe on the receiver. --=20 Freddie Cash fjwcash@gmail.com From owner-freebsd-fs@FreeBSD.ORG Wed Jan 22 22:02:09 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 735EAB6 for ; Wed, 22 Jan 2014 22:02:09 +0000 (UTC) Received: from mail-qc0-x231.google.com (mail-qc0-x231.google.com [IPv6:2607:f8b0:400d:c01::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 30166188D for ; Wed, 22 Jan 2014 22:02:09 +0000 (UTC) Received: by mail-qc0-f177.google.com with SMTP id i8so1387923qcq.22 for ; Wed, 22 Jan 2014 14:02:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=RsB/1ob34JfEDVmrfprXtQKAl15466f36gQtEoe1rRM=; b=Tq4cC6oD0P6AUp6QeKouB6OW2tESln1V9D0IxADu/5OnwIciW+1JvIgw5AyOqXnRI8 y4eYDMnu/NZdzOpfb5vAeX74lvWV1NDldDVbSxf1yhfJPiwM0pDc7ayIzUXlDGI7fe0P rKiSl6mEvxMhnpITqmZcNUTYNA5LyHXK2yV8kSPWQVxL0GW9ZTPAz0wxsGelmxJarZ3S CuHsR+JlbMEIsC6SlNorL121Tjd8jRrza92C8170SuSPiTRmcziLI3c3gbTddvVoPYYq CWHV0ScVBYg4D3UDTDFXv47BglnUyAy+OEc+8VAJSUaVlvPb/q9GG2/s7Eu1Chk3dGgb UPHA== MIME-Version: 1.0 X-Received: by 10.140.81.240 with SMTP id f103mr6124465qgd.104.1390428128293; Wed, 22 Jan 2014 14:02:08 -0800 (PST) Received: by 10.96.120.193 with HTTP; Wed, 22 Jan 2014 14:02:08 -0800 (PST) Received: by 10.96.120.193 with HTTP; Wed, 22 Jan 2014 14:02:08 -0800 (PST) In-Reply-To: References: Date: Wed, 22 Jan 2014 23:02:08 +0100 Message-ID: Subject: Re: hast and zfs From: Joar Jegleim To: Ronald Klop Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jan 2014 22:02:09 -0000 Thnx for your reply ronald. Checking the commit logs seem like a good idea. Main reason for 9.1 is that I've taken over more than a hundred installations with everythimg from 7.0 up to 9.1 and pretty much every minor release inbetween. My initial goal was upgrading everything to 9.1, which may turn out to be too ambitious ( old inhouse apps, maintainers long gone) I did not mention in my first post that I used ssd cache on the primary node that wasn't part of hast. I switched over to the hast setup last night and it ran for about 3 hours without problems(without l2arc) . I'm not sure if it could be the ssd l2arc that is the issue, or if it's was because of less iops ( stopped all publishing of files while testing) . It also ran for about 2 hours during office hours today too, and I'm hopefully able to let it run for longer than that tomorrow. Regards Joar On 22 Jan 2014 15:11, "Ronald Klop" wrote: > On Tue, 21 Jan 2014 15:46:09 +0100, Joar Jegleim > wrote: > > Hi list ! >> >> I've setup hast with zfs on 9.1-RELEASE-p10 >> I've got an lsi hba connected to an external sas hp msa that has 20 >> disks in it, same setup on both nodes. >> I've created a hast disk dev such as /dev/hast/disk1a, >> /dev/hast/disk1b etc... ( see attached hast.conf ) >> And built a zpool mirror accross the 20 disks. >> >> While setting up this, and testing, all looked pretty good, I've tried >> numerous failovers and so on. >> When I just switched over to this setup in production, it worked for >> about half an hour until the primary node got a zfs hang where zpool >> list hung and hastctl status hung completely . >> >> There was no output in dmesg nor /var/log/messages that said anything >> related to this (?) . >> >> The server serves some 3 000 000 jpegs for a site, I see anything from >> 50 to just above 1000 iops, though the average iops is bellow 100. >> It's completely random what pictures are being fetched at any time. >> >> Could there be some sysctls to tune related to this setup mabye ? >> Anybody using hast and zfs in production got any tip ? >> >> >> >> > You did not get any reply yet. I'm not of any help either, but is there a > reason you don't use a newer FreeBSD version? (9.1 is from December 2012.) > It can be useful to read through the commits about hast which happened in > the meantime. > > Ronald. > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@FreeBSD.ORG Fri Jan 24 18:41:48 2014 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F098354B for ; Fri, 24 Jan 2014 18:41:48 +0000 (UTC) Received: from vlakno.cz (mail.vlakno.cz [95.129.96.251]) by mx1.freebsd.org (Postfix) with ESMTP id B868B1D95 for ; Fri, 24 Jan 2014 18:41:48 +0000 (UTC) Received: by vlakno.cz (Postfix, from userid 1002) id 1E3501CC558B; Fri, 24 Jan 2014 19:41:41 +0100 (CET) Date: Fri, 24 Jan 2014 19:41:41 +0100 From: Roman Divacky To: fs@freebsd.org Subject: BUG: possible NULL pointer dereference in nfs server Message-ID: <20140124184141.GA19458@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.22 (2013-10-16) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Jan 2014 18:41:49 -0000 Hi, In nfs_nfsdstate.c:nfsrv_lockctrl() we call getlckret = nfsrv_getlockfh(vp, new_stp->ls_flags, NULL, &nfh, p); then in nfsrv_getlockfh() we, based on the value of flags, might dereference the NULL pointer: nfsrv_getlockfh(vnode_t vp, u_short flags, struct nfslockfile **new_lfpp, fhandle_t *nfhp, NFSPROC_T *p) if (flags & NFSLCK_OPEN) { new_lfp = *new_lfpp; fhp = &new_lfp->lf_fh; I am not sure what the right fix is. Or if it's even possible to hit (but I think it is). Anyway the compiler currently generates a trap instruction (ud2 on x86) in this code. It's the only trap in GENERIC btw. Would be lovely to fix this. Roman P.S. CC me on your replies as I am not subscribed to the list. From owner-freebsd-fs@FreeBSD.ORG Sat Jan 25 00:38:37 2014 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 20D0BE6D; Sat, 25 Jan 2014 00:38:37 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id C7670194B; Sat, 25 Jan 2014 00:38:36 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: X-IronPort-AV: E=Sophos;i="4.95,716,1384318800"; d="scan'208";a="90887200" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 24 Jan 2014 19:38:35 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id A287FB3F46; Fri, 24 Jan 2014 19:38:35 -0500 (EST) Date: Fri, 24 Jan 2014 19:38:35 -0500 (EST) From: Rick Macklem To: Roman Divacky Message-ID: <1577222508.16050114.1390610315654.JavaMail.root@uoguelph.ca> In-Reply-To: <20140124184141.GA19458@freebsd.org> Subject: Re: BUG: possible NULL pointer dereference in nfs server MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Jan 2014 00:38:37 -0000 Roman Divacky wrote: > Hi, > > In nfs_nfsdstate.c:nfsrv_lockctrl() we call > > getlckret = nfsrv_getlockfh(vp, new_stp->ls_flags, NULL, &nfh, p); > > then in nfsrv_getlockfh() we, based on the value of flags, might > dereference the NULL pointer: > > > nfsrv_getlockfh(vnode_t vp, u_short flags, > struct nfslockfile **new_lfpp, fhandle_t *nfhp, NFSPROC_T *p) > > > if (flags & NFSLCK_OPEN) { > new_lfp = *new_lfpp; > fhp = &new_lfp->lf_fh; > > I took a look and don't see a problem. NFSLCK_OPEN is only set for Opens { nfsrvd_open() } and it never calls nfsrv_lockctrl(). nfsrv_lockctrl() is always called with other flags in new_stp->ls_flags set, but never NFSLCK_OPEN, because nfsrv_lockctrl() is handling byte range lock cases after a file has been opened { which means the "else" case for this "if" always applies. You could add a KASSERT() like: KASSERT((new_stp->ls_flags & NFSLCK_OPEN) == 0, ("nfsrv_lockctrl: calling nfsrv_getlockfh with NFSLCK_OPEN")); just before the nfsrv_getlockfh() call if you'd like, but I'd be very surprised if this ever happens. It wouldn't make sense to call nfsrv_lockctrl() with NFSLCK_OPEN and I can't see anywhere it the code that it would do this. (And I've never seen a report of a crash that this would have caused.) > I am not sure what the right fix is. Or if it's even possible to hit > (but I think it is). Anyway the compiler currently generates > a trap instruction (ud2 on x86) in this code. It's the only trap > in GENERIC btw. > Sorry, I'm not a compiler guy, so I don't know why a compiler would generate a trap instruction, but since new_lfpp is never NULL when this is executed, I don't see a problem. If others feel that this needs to be re-coded, please let me know what you think the code should look like? (A test for non-NULL with a panic() before it is used?) Is a trap instruction that never gets executed a problem? rick > Would be lovely to fix this. > > Roman > > P.S. CC me on your replies as I am not subscribed to the list. > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@FreeBSD.ORG Sat Jan 25 03:36:54 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9A074FCA; Sat, 25 Jan 2014 03:36:54 +0000 (UTC) Received: from thebighonker.lerctr.org (lrosenman-1-pt.tunnel.tserv8.dal1.ipv6.he.net [IPv6:2001:470:1f0e:3ad::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 59B1F1767; Sat, 25 Jan 2014 03:36:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:Subject:To:From:Date:Content-Transfer-Encoding:Content-Type:MIME-Version; bh=apOy3yejHFAeRXIqAPB8jdV9/w3geiG3hQ7TfxT55ys=; b=uYNNratvTIsfl/42JJmSLjhq0jU4b3Q5/uRR6oK8BHNQWLszA2WWno8cL5N5Sb9LErETiJ8dBMz2zJD3nEvIAovAQLd7EO6VZyqPPwhkx/urwCTAalScxqFpQLApJBX73jPSmnPZE5ZXzlm1TCYtzmvdlHL+PVuZ76JF3Xvwxuk=; Received: from localhost.lerctr.org ([127.0.0.1]:47940 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpa (Exim 4.82 (FreeBSD)) (envelope-from ) id 1W6u34-000Pub-GZ; Fri, 24 Jan 2014 21:36:52 -0600 Received: from 107-128-180-255.lightspeed.austtx.sbcglobal.net ([107.128.180.255]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Fri, 24 Jan 2014 21:36:46 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Fri, 24 Jan 2014 21:36:46 -0600 From: Larry Rosenman To: Freebsd current , Freebsd fs Subject: Fwd: Panic/Solaris Assert/ZAP.c:534 Message-ID: <3b2a67b62965f564578b5b084480c84b@webmail.lerctr.org> X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/0.9.5 X-Spam-Score: 2.0 (++) X-LERCTR-Spam-Score: 2.0 (++) X-Spam-Report: SpamScore (2.0/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, KAM_STOCKTIP=5.5, RP_MATCHES_RCVD=-0.634 X-LERCTR-Spam-Report: SpamScore (2.0/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, KAM_STOCKTIP=5.5, RP_MATCHES_RCVD=-0.634 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Jan 2014 03:36:54 -0000 Still looking for help here so I can move forward on 11-CURRENT. -------- Original Message -------- Subject: Fwd: Panic/Solaris Assert/ZAP.c:534 Date: 2014-01-22 08:25 From: Larry Rosenman To: Freebsd fs , Freebsd current Repost, including the FS guys. Ideas? -------- Original Message -------- Subject: Panic/Solaris Assert/ZAP.c:534 Date: 2014-01-20 21:23 From: Larry Rosenman To: freebsd-current@freebsd.org Ideas? I'm getting this on a regular basis, and scrub/etc doesn't seem to get around it, but booting off: FreeBSD borg.lerctr.org 11.0-CURRENT FreeBSD 11.0-CURRENT #113 r260864: Fri Jan 17 19:10:06 CST 2014 root@:/usr/obj/usr/src/sys/BORG-DTRACE amd64 Seems to avoid it. HELP. borg.lerctr.org dumped core - see /var/crash/vmcore.1 Mon Jan 20 21:18:41 CST 2014 FreeBSD borg.lerctr.org 11.0-CURRENT FreeBSD 11.0-CURRENT #1 r260896: Mon Jan 20 12:31:46 CST 2014 root@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER amd64 panic: solaris assert: l->l_phys->l_hdr.lh_block_type == ((1ULL << 63) + 0) (0x0 == 0x8000000000000000), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c, line: 534 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: Copyright (c) 1992-2014 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.0-CURRENT #1 r260896: Mon Jan 20 12:31:46 CST 2014 root@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER amd64 FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 info: [drm] Initialized drm 1.1.0 20060810 CPU: Intel(R) Xeon(R) CPU E5410 @ 2.33GHz (2327.55-MHz K8-class CPU) Origin="GenuineIntel" Id=0x10676 Family=0x6 Model=0x17 Stepping=6 Features=0xbfebfbff Features2=0xce3bd AMD Features=0x20100800 AMD Features2=0x1 TSC: P-state invariant, performance statistics real memory = 68719476736 (65536 MB) avail memory = 65641644032 (62600 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs FreeBSD/SMP: 2 package(s) x 4 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP): APIC ID: 3 cpu4 (AP): APIC ID: 4 cpu5 (AP): APIC ID: 5 cpu6 (AP): APIC ID: 6 cpu7 (AP): APIC ID: 7 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard random: initialized kbd1 at kbdmux0 cryptosoft0: on motherboard acpi0: on motherboard acpi0: Power Button (fixed) unknown: I/O range not supported cpu0: on acpi0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 cpu4: on acpi0 cpu5: on acpi0 cpu6: on acpi0 cpu7: on acpi0 hpet0: iomem 0xfed00000-0xfed003ff irq 0,8 on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 atrtc0: port 0x70-0x71 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: at device 2.0 on pci0 pci1: on pcib1 pcib2: irq 16 at device 0.0 on pci1 pci2: on pcib2 pcib3: irq 16 at device 0.0 on pci2 pci3: on pcib3 pcib4: at device 0.0 on pci3 pci4: on pcib4 pcib5: at device 0.2 on pci3 pci5: on pcib5 pcib6: irq 18 at device 2.0 on pci2 pci6: on pcib6 em0: port 0x2000-0x201f mem 0xd9220000-0xd923ffff,0xd9200000-0xd921ffff irq 18 at device 0.0 on pci6 em0: Using an MSI interrupt em0: Ethernet address: 00:30:48:f2:29:9c em1: port 0x2020-0x203f mem 0xd9260000-0xd927ffff,0xd9240000-0xd925ffff irq 19 at device 0.1 on pci6 em1: Using an MSI interrupt em1: Ethernet address: 00:30:48:f2:29:9d pcib7: at device 0.3 on pci1 pci7: on pcib7 pcib8: at device 4.0 on pci0 pci8: on pcib8 vgapci0: port 0x3000-0x307f mem 0xd8000000-0xd8ffffff,0xc0000000-0xc7ffffff,0xc8000000-0xc9ffffff irq 16 at device 0.0 on pci8 nvidia0: on vgapci0 vgapci0: child nvidia0 requested pci_enable_io vgapci0: child nvidia0 requested pci_enable_io hdac0: mem 0xd9000000-0xd9003fff irq 17 at device 0.1 on pci8 pcib9: at device 6.0 on pci0 pci9: on pcib9 pci0: at device 8.0 (no driver attached) pcib10: irq 17 at device 28.0 on pci0 pci10: on pcib10 pcib11: irq 16 at device 0.0 on pci10 pci11: on pcib11 pcm0: port 0x4080-0x409f,0x4000-0x407f irq 16 at device 0.0 on pci11 pcm0: system configuration SubVendorID: 0x1412, SubDeviceID: 0x2403 XIN2 Clock Source: 24.576MHz(96kHz*256) MPU-401 UART(s) #: not implemented ADC #: 1 and SPDIF receiver connected DAC #: 4 Multi-track converter type: AC'97(SDATA_OUT:packed) S/PDIF(IN/OUT): 1/1 ID# 0x00 GPIO(mask/dir/state): 0xff/0xff/0xff uhci0: port 0x1800-0x181f irq 17 at device 29.0 on pci0 usbus0 on uhci0 uhci1: port 0x1820-0x183f irq 19 at device 29.1 on pci0 usbus1 on uhci1 uhci2: port 0x1840-0x185f irq 18 at device 29.2 on pci0 usbus2 on uhci2 ehci0: mem 0xd9600400-0xd96007ff irq 17 at device 29.7 on pci0 usbus3: EHCI version 1.0 usbus3 on ehci0 pcib12: at device 30.0 on pci0 pci12: on pcib12 vgapci1: port 0x5000-0x50ff mem 0xd0000000-0xd7ffffff,0xd9300000-0xd930ffff irq 18 at device 1.0 on pci12 drmn1: on vgapci1 info: [drm] RADEON_IS_PCI info: [drm] initializing kernel modesetting (RV100 0x1002:0x515E 0x15D9:0x8080). info: [drm] register mmio base: 0xD9300000 info: [drm] register mmio size: 65536 info: [drm] radeon_atrm_get_bios: ===> Try ATRM... info: [drm] radeon_atrm_get_bios: pci_find_class() found: 0:8:0:0, vendor=10de, device=104a info: [drm] radeon_atrm_get_bios: Get ACPI device handle info: [drm] radeon_acpi_vfct_bios: ===> Try VFCT... info: [drm] radeon_acpi_vfct_bios: Get "VFCT" ACPI table info: [drm] radeon_acpi_vfct_bios: Failed to get "VFCT" table: AE_NOT_FOUND info: [drm] igp_read_bios_from_vram: ===> Try IGP's VRAM... info: [drm] igp_read_bios_from_vram: VRAM base address: 0xd0000000 info: [drm] igp_read_bios_from_vram: Map address: 0xfffff800d0000000 (262144 bytes) info: [drm] igp_read_bios_from_vram: Incorrect BIOS signature: 0x0000 info: [drm] radeon_read_bios: ===> Try PCI Expansion ROM... info: [drm] radeon_read_bios: Map address: 0xfffff800000c0000 (131072 bytes) drmn1: info: VRAM: 128M 0x00000000D0000000 - 0x00000000D7FFFFFF (16M used) drmn1: info: GTT: 512M 0x00000000B0000000 - 0x00000000CFFFFFFF info: [drm] Supports vblank timestamp caching Rev 1 (10.10.2010). info: [drm] Driver supports precise vblank timestamp query. info: [drm] radeon: irq initialized. info: [drm] Detected VRAM RAM=128M, BAR=128M info: [drm] RAM width 64bits SDR [TTM] Zone kernel: Available graphics memory: 33006090 kiB [TTM] Zone dma32: Available graphics memory: 2097152 kiB [TTM] Initializing pool allocator info: [drm] radeon: 16M of VRAM memory ready info: [drm] radeon: 512M of GTT memory ready. info: [drm] GART: num cpu pages 131072, num gpu pages 131072 info: [drm] PCI GART of 512M enabled (table at 0x0000000011200000). drmn1: info: WB disabled drmn1: info: fence driver on ring 0 use gpu addr 0x00000000b0000000 and cpu addr 0x0xfffff8001083c000 info: [drm] Loading R100 Microcode info: [drm] radeon: ring at 0x00000000B0001000 info: [drm] ring test succeeded in 1 usecs info: [drm] ib test succeeded in 0 usecs info: [drm] radeon_device_init: Taking over the fictitious range 0xd0000000-0xd4000000 iicbus0: on iicbb0 addr 0xff iic0: on iicbus0 iicbus1: on iicbb1 addr 0xff iic1: on iicbus1 iicbus2: on iicbb2 addr 0xff iic2: on iicbus2 iicbus3: on iicbb3 addr 0xff iic3: on iicbus3 info: [drm] Radeon Display Connectors info: [drm] Connector 0: info: [drm] VGA-1 info: [drm] DDC: 0x60 0x60 0x60 0x60 0x60 0x60 0x60 0x60 info: [drm] Encoders: info: [drm] CRT1: INTERNAL_DAC1 info: [drm] Connector 1: info: [drm] DVI-I-1 info: [drm] HPD2 info: [drm] DDC: 0x6c 0x6c 0x6c 0x6c 0x6c 0x6c 0x6c 0x6c info: [drm] Encoders: info: [drm] CRT2: INTERNAL_DAC2 info: [drm] DFP2: INTERNAL_DVO1 composite sync not supported composite sync not supported info: [drm] fb mappable at 0xD0040000 info: [drm] vram apper at 0xD0000000 info: [drm] size 2076672 info: [drm] fb depth is 8 info: [drm] pitch is 1920 fbd1 on drmn1 vt_allocate: Replace existing VT driver. info: [drm] Initialized radeon 2.29.0 20080528 vgapci1: Boot video device isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1860-0x186f at device 31.1 on pci0 ata0: at channel 0 on atapci0 ahci0: port 0x18a0-0x18a7,0x1874-0x1877,0x1878-0x187f,0x1870-0x1873,0x1880-0x189f mem 0xd9600800-0xd9600bff irq 19 at device 31.2 on pci0 ahci0: AHCI v1.10 with 6 3Gbps ports, Port Multiplier supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahcich4: at channel 4 on ahci0 ahcich5: at channel 5 on ahci0 ichsmb0: port 0x1100-0x111f irq 19 at device 31.3 on pci0 smbus0: on ichsmb0 acpi_button0: on acpi0 ipmi0: port 0xca2-0xca3 on acpi0 ipmi0: KCS mode found at io 0xca2 on acpi atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fd0: <1440-KB 3.5" drive> on fdc0 drive 0 ppc0: port 0x378-0x37f,0x778-0x77f irq 7 drq 3 on acpi0 ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/9 bytes threshold ppbus0: on ppc0 lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 ichwd0 on isa0 orm0: at iomem 0xc0000-0xcafff on isa0 coretemp0: on cpu0 est0: on cpu0 p4tcc0: on cpu0 coretemp1: on cpu1 est1: on cpu1 p4tcc1: on cpu1 coretemp2: on cpu2 est2: on cpu2 p4tcc2: on cpu2 coretemp3: on cpu3 est3: on cpu3 p4tcc3: on cpu3 coretemp4: on cpu4 est4: on cpu4 p4tcc4: on cpu4 coretemp5: on cpu5 est5: on cpu5 p4tcc5: on cpu5 coretemp6: on cpu6 est6: on cpu6 p4tcc6: on cpu6 coretemp7: on cpu7 est7: on cpu7 p4tcc7: on cpu7 ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Timecounters tick every 1.000 msec vboxdrv: fAsync=0 offMin=0x387 offMax=0x491 hdacc0: at cad 0 on hdac0 hdaa0: at nid 1 on hdacc0 pcm1: at nid 4 on hdaa0 pcm2: at nid 5 on hdaa0 random: unblocking device. usbus0: 12Mbps Full Speed USB v1.0 usbus1: 12Mbps Full Speed USB v1.0 usbus2: 12Mbps Full Speed USB v1.0 usbus3: 480Mbps High Speed USB v2.0 ugen3.1: at usbus3 uhub0: on usbus3 ugen2.1: at usbus2 uhub1: on usbus2 ugen1.1: at usbus1 uhub2: on usbus1 ugen0.1: at usbus0 uhub3: on usbus0 ipmi0: IPMI device rev. 1, firmware rev. 1.64, version 2.0 ata0: DMA limited to UDMA33, controller found non-ATA66 cable ipmi0: Number of channels 8 ipmi0: Attached watchdog uhub2: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered uhub3: 2 ports with 2 removable, self powered ada0 at ahcich0 bus 0 scbus1 target 0 lun 0 ada0: ATA-8 SATA 3.x device ada0: Serial Number 5YDA1ZL4 ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada0: quirks=0x1<4K> ada0: Previously was known as ad4 ada1 at ahcich1 bus 0 scbus2 target 0 lun 0 ada1: ATA-8 SATA 3.x device ada1: Serial Number 5YD6FPLG ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada1: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada1: quirks=0x1<4K> ada1: Previously was known as ad6 ada2 at ahcich2 bus 0 scbus3 target 0 lun 0 ada2: ATA-8 SATA 3.x device ada2: Serial Number 5YDA3PC5 ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada2: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada2: quirks=0x1<4K> ada2: Previously was known as ad8 ada3 at ahcich3 bus 0 scbus4 target 0 lun 0 ada3: ATA-8 SATA 3.x device ada3: Serial Number 5YD9Y0P4 ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada3: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada3: quirks=0x1<4K> ada3: Previously was known as ad10 ada4 at ahcich4 bus 0 scbus5 target 0 lun 0 ada4: ATA-8 SATA 3.x device ada4: Serial Number 5YDA1R5W ada4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada4: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada4: quirks=0x1<4K> ada4: Previously was known as ad12 ada5 at ahcich5 bus 0 scbus6 target 0 lun 0 ada5: ATA-8 SATA 3.x device ada5: Serial Number 5YD5RBS8 ada5: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada5: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada5: quirks=0x1<4K> ada5: Previously was known as ad14 cd0 at ata0 bus 0 scbus0 target 0 lun 0 cd0: Removable CD-ROM SCSI-0 device cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present Netvsc initializing... done! SMP: AP CPU #6 Launched! SMP: AP CPU #1 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #7 Launched! SMP: AP CPU #5 Launched! SMP: AP CPU #4 Launched! SMP: AP CPU #3 Launched! Root mount waiting for: usbus3 uhub0: 6 ports with 6 removable, self powered Root mount waiting for: usbus3 Root mount waiting for: usbus3 ugen3.2: at usbus3 ukbd0: on usbus3 kbd2 at ukbd0 Trying to mount root from zfs:zroot/ROOT/default []... ugen0.2: at usbus0 ugen1.2: at usbus1 <118>Setting hostuuid: 53d19f64-d663-a017-8922-0030488e9ff3. <118>Setting hostid: 0xf53a926e. <118>Entropy harvesting: interrupts ethernet point_to_point swi. <118>Starting file system checks: <118>Mounting local file systems:. panic: solaris assert: l->l_phys->l_hdr.lh_block_type == ((1ULL << 63) + 0) (0x0 == 0x8000000000000000), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c, line: 534 cpuid = 7 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe100c499090 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe100c499140 vpanic() at vpanic+0x126/frame 0xfffffe100c499180 panic() at panic+0x43/frame 0xfffffe100c4991e0 assfail3() at assfail3+0x2f/frame 0xfffffe100c499200 zap_get_leaf_byblk() at zap_get_leaf_byblk+0x4e1/frame 0xfffffe100c499250 zap_deref_leaf() at zap_deref_leaf+0xd1/frame 0xfffffe100c4992a0 fzap_cursor_retrieve() at fzap_cursor_retrieve+0xce/frame 0xfffffe100c499310 zap_cursor_retrieve() at zap_cursor_retrieve+0x20a/frame 0xfffffe100c499390 ddt_zap_walk() at ddt_zap_walk+0x5e/frame 0xfffffe100c499650 ddt_walk() at ddt_walk+0x78/frame 0xfffffe100c499680 dsl_scan_sync() at dsl_scan_sync+0x6ec/frame 0xfffffe100c4999e0 spa_sync() at spa_sync+0x5c4/frame 0xfffffe100c499ad0 txg_sync_thread() at txg_sync_thread+0x256/frame 0xfffffe100c499bb0 fork_exit() at fork_exit+0x84/frame 0xfffffe100c499bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe100c499bf0 --- trap 0, rip = 0, rsp = 0xfffffe100c499cb0, rbp = 0 --- KDB: enter: panic Uptime: 14s Dumping 2263 out of 64465 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Reading symbols from /boot/kernel.old/linux.ko.symbols...done. Loaded symbols for /boot/kernel.old/linux.ko.symbols Reading symbols from /boot/kernel.old/if_lagg.ko.symbols...done. Loaded symbols for /boot/kernel.old/if_lagg.ko.symbols Reading symbols from /boot/kernel.old/snd_envy24ht.ko.symbols...done. Loaded symbols for /boot/kernel.old/snd_envy24ht.ko.symbols Reading symbols from /boot/kernel.old/snd_spicds.ko.symbols...done. Loaded symbols for /boot/kernel.old/snd_spicds.ko.symbols Reading symbols from /boot/kernel.old/coretemp.ko.symbols...done. Loaded symbols for /boot/kernel.old/coretemp.ko.symbols Reading symbols from /boot/kernel.old/ichsmb.ko.symbols...done. Loaded symbols for /boot/kernel.old/ichsmb.ko.symbols Reading symbols from /boot/kernel.old/smbus.ko.symbols...done. Loaded symbols for /boot/kernel.old/smbus.ko.symbols Reading symbols from /boot/kernel.old/ichwd.ko.symbols...done. Loaded symbols for /boot/kernel.old/ichwd.ko.symbols Reading symbols from /boot/kernel.old/cpuctl.ko.symbols...done. Loaded symbols for /boot/kernel.old/cpuctl.ko.symbols Reading symbols from /boot/kernel.old/crypto.ko.symbols...done. Loaded symbols for /boot/kernel.old/crypto.ko.symbols Reading symbols from /boot/kernel.old/cryptodev.ko.symbols...done. Loaded symbols for /boot/kernel.old/cryptodev.ko.symbols Reading symbols from /boot/kernel.old/dtraceall.ko.symbols...done. Loaded symbols for /boot/kernel.old/dtraceall.ko.symbols Reading symbols from /boot/kernel.old/profile.ko.symbols...done. Loaded symbols for /boot/kernel.old/profile.ko.symbols Reading symbols from /boot/kernel.old/cyclic.ko.symbols...done. Loaded symbols for /boot/kernel.old/cyclic.ko.symbols Reading symbols from /boot/kernel.old/dtrace.ko.symbols...done. Loaded symbols for /boot/kernel.old/dtrace.ko.symbols Reading symbols from /boot/kernel.old/systrace_freebsd32.ko.symbols...done. Loaded symbols for /boot/kernel.old/systrace_freebsd32.ko.symbols Reading symbols from /boot/kernel.old/systrace.ko.symbols...done. Loaded symbols for /boot/kernel.old/systrace.ko.symbols Reading symbols from /boot/kernel.old/sdt.ko.symbols...done. Loaded symbols for /boot/kernel.old/sdt.ko.symbols Reading symbols from /boot/kernel.old/lockstat.ko.symbols...done. Loaded symbols for /boot/kernel.old/lockstat.ko.symbols Reading symbols from /boot/kernel.old/fasttrap.ko.symbols...done. Loaded symbols for /boot/kernel.old/fasttrap.ko.symbols Reading symbols from /boot/kernel.old/fbt.ko.symbols...done. Loaded symbols for /boot/kernel.old/fbt.ko.symbols Reading symbols from /boot/kernel.old/dtnfscl.ko.symbols...done. Loaded symbols for /boot/kernel.old/dtnfscl.ko.symbols Reading symbols from /boot/kernel.old/dtmalloc.ko.symbols...done. Loaded symbols for /boot/kernel.old/dtmalloc.ko.symbols Reading symbols from /boot/modules/vboxdrv.ko...done. Loaded symbols for /boot/modules/vboxdrv.ko Reading symbols from /boot/modules/nvidia.ko...done. Loaded symbols for /boot/modules/nvidia.ko Reading symbols from /boot/kernel.old/ipmi.ko.symbols...done. Loaded symbols for /boot/kernel.old/ipmi.ko.symbols Reading symbols from /boot/kernel.old/ipmi_linux.ko.symbols...done. Loaded symbols for /boot/kernel.old/ipmi_linux.ko.symbols Reading symbols from /boot/kernel.old/radeonkms.ko.symbols...done. Loaded symbols for /boot/kernel.old/radeonkms.ko.symbols Reading symbols from /boot/kernel.old/iicbb.ko.symbols...done. Loaded symbols for /boot/kernel.old/iicbb.ko.symbols Reading symbols from /boot/kernel.old/iicbus.ko.symbols...done. Loaded symbols for /boot/kernel.old/iicbus.ko.symbols Reading symbols from /boot/kernel.old/iic.ko.symbols...done. Loaded symbols for /boot/kernel.old/iic.ko.symbols Reading symbols from /boot/kernel.old/drm2.ko.symbols...done. Loaded symbols for /boot/kernel.old/drm2.ko.symbols Reading symbols from /boot/kernel.old/radeonkmsfw_R100_cp.ko.symbols...done. Loaded symbols for /boot/kernel.old/radeonkmsfw_R100_cp.ko.symbols Reading symbols from /boot/kernel.old/fdescfs.ko.symbols...done. Loaded symbols for /boot/kernel.old/fdescfs.ko.symbols Reading symbols from /boot/kernel.old/linprocfs.ko.symbols...done. Loaded symbols for /boot/kernel.old/linprocfs.ko.symbols #0 doadump (textdump=1) at pcpu.h:219 219 pcpu.h: No such file or directory. in pcpu.h (kgdb) #0 doadump (textdump=1) at pcpu.h:219 #1 0xffffffff809b98d7 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:452 #2 0xffffffff809b9de5 in vpanic (fmt=, ap=) at /usr/src/sys/kern/kern_shutdown.c:759 #3 0xffffffff809b9e33 in panic (fmt=) at /usr/src/sys/kern/kern_shutdown.c:688 #4 0xffffffff80327a2f in assfail3 (a=, lv=, op=, rv=, f=, l=) at /usr/src/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c:91 #5 0xffffffff803bad81 in zap_get_leaf_byblk (zap=, blkid=, tx=0x0, lt=, lp=0xfffffe100c4994e8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c:534 #6 0xffffffff803b8461 in zap_deref_leaf (zap=0xfffff80020a4ec00, h=9377620324092215296, tx=0x0, lt=RW_READER, lp=0xfffffe100c4994e8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c:585 #7 0xffffffff803ba24e in fzap_cursor_retrieve (zap=0xfffff80020a4ec00, zc=0xfffffe100c4994d8, za=0xfffffe100c4993c0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c:1181 #8 0xffffffff803c076a in zap_cursor_retrieve (zc=0xfffffe100c4994d8, za=0xfffffe100c4993c0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap_micro.c:1290 #9 0xffffffff80352fde in ddt_zap_walk (os=0xfffff8002031bc00, object=156, dde=0xfffffe100c499828, walk=0xfffff800201ce3d0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/ddt_zap.c:117 #10 0xffffffff80352c88 in ddt_walk (spa=0xfffffe000d377000, ddb=0xfffff800201ce3b8, dde=0xfffffe100c499828) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/ddt.c:219 #11 0xffffffff803800dc in dsl_scan_sync (dp=0xfffff8002031f800, tx=0xfffff80022180000) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c:1204 #12 0xffffffff803999b4 in spa_sync (spa=0xfffffe000d377000, txg=14079627) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c:6549 #13 0xffffffff803a4e96 in txg_sync_thread (arg=0xfffff8002031f800) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c:518 #14 0xffffffff80988f94 in fork_exit ( callout=0xffffffff803a4c40 , arg=0xfffff8002031f800, frame=0xfffffe100c499c00) at /usr/src/sys/kern/kern_fork.c:977 #15 0xffffffff80d9576e in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:605 #16 0x0000000000000000 in ?? () Current language: auto; currently minimal (kgdb) ------------------------------------------------------------------------ ps -axlww UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND 0 0 0 0 -8 0 0 0 - DLs - 0:00.10 [kernel] 0 1 0 0 21 0 9432 0 wait DLs - 0:00.00 [init] 0 2 0 0 -16 0 0 0 ftcl DL - 0:00.00 [ftcleanup] 0 3 0 0 -16 0 0 0 crypto_w DL - 0:00.00 [crypto] 0 4 0 0 -16 0 0 0 crypto_r DL - 0:00.00 [crypto returns] 0 5 0 0 -16 0 0 0 ccb_scan DL - 0:00.00 [cam] 0 6 0 0 -16 0 0 0 - DL - 0:00.00 [fdc0] 0 7 0 0 -8 0 0 0 - RL - 0:00.00 [zfskern] 0 8 0 0 -16 0 0 0 waiting_ DL - 0:00.00 [sctp_iterator] 0 9 0 0 -16 0 0 0 ipmireq DL - 0:00.00 [ipmi0: kcs] 0 10 0 0 -16 0 0 0 audit_wo DL - 0:00.00 [audit] 0 11 0 0 155 0 0 0 - RL - 0:00.00 [idle] 0 12 0 0 -84 0 0 0 - WL - 0:00.00 [intr] 0 13 0 0 -8 0 0 0 - DL - 0:00.00 [geom] 0 14 0 0 -16 0 0 0 - DL - 0:00.00 [rand_harvestq] 0 15 0 0 -68 0 0 0 - DL - 0:00.00 [usb] 0 16 0 0 -20 0 0 0 VBoxIS DL - 0:00.00 [TIMER] 0 17 0 0 -16 0 0 0 psleep DL - 0:00.00 [pagedaemon] 0 18 0 0 -16 0 0 0 psleep DL - 0:00.00 [vmdaemon] 0 19 0 0 155 0 0 0 pgzero DL - 0:00.00 [pagezero] 0 20 0 0 -16 0 0 0 psleep DL - 0:00.00 [bufdaemon] 0 21 0 0 16 0 0 0 syncer DL - 0:00.00 [syncer] 0 22 0 0 -16 0 0 0 vlruwt DL - 0:00.00 [vnlru] 0 23 0 0 -16 0 0 0 sdflush DL - 0:00.00 [softdepflush] 0 24 1 0 52 0 16992 0 wait Ds+ - 0:00.02 [sh] 0 72 24 0 52 0 16992 0 wait D+ - 0:00.00 [sh] 0 75 72 0 39 0 40064 0 zio->io_ D+ - 0:00.00 [zfs] ------------------------------------------------------------------------ vmstat -s 42891 cpu context switches 5520 device interrupts 462 software interrupts 5061 traps 5879 system calls 23 kernel threads created 37 fork() calls 15 vfork() calls 0 rfork() calls 0 swap pager pageins 0 swap pager pages paged in 0 swap pager pageouts 0 swap pager pages paged out 135 vnode pager pageins 923 vnode pager pages paged in 0 vnode pager pageouts 0 vnode pager pages paged out 0 page daemon wakeups 0 pages examined by the page daemon 0 pages reactivated 1770 copy-on-write faults 15 copy-on-write optimized faults 2266 zero fill pages zeroed 0 zero fill pages prezeroed 0 intransit blocking page faults 4682 total VM faults taken 112 page faults requiring I/O 0 pages affected by kernel thread creation 1350 pages affected by fork() 525 pages affected by vfork() 0 pages affected by rfork() 0 pages cached 55124 pages freed 0 pages freed by daemon 0 pages freed by exiting processes 473 pages active 869 pages inactive 0 pages in VM cache 140410 pages wired down 15933660 pages free 4096 bytes per page 1934 total name lookups cache hits (80% pos + 4% neg) system 0% per-directory deletions 0%, falsehits 0%, toolong 0% ------------------------------------------------------------------------ vmstat -m Type InUse MemUse HighUse Requests Size(s) cdev 8 2K - 8 256 ppbusdev 2 1K - 2 256 filedesc 27 54K - 76 2048 kdtrace 330 78K - 522 64,256 kenv 87 12K - 115 16,32,64,128 proc-args 3 1K - 31 32,64,128 hhook 2 1K - 2 256 ithread 125 22K - 125 32,128,256 KTRACE 100 13K - 100 128 entropy 1026 65K - 1026 32,64 acpiintr 1 1K - 1 64 linker 403 101K - 533 16,32,64,128,256,512,1024,2048,4096 lockf 2 1K - 2 64,128 loginclass 2 1K - 2 64 cache 1 1K - 1 32 devbuf 17314 34391K - 18687 16,32,64,128,256,512,1024,2048,4096 temp 57 18K - 548 16,32,64,128,256,512,1024,2048 ip6ndp 3 1K - 3 64 ata_pci 1 1K - 1 64 acpica 1824 187K - 63680 16,32,64,128,256,512,1024,2048 kbdmux 7 18K - 7 16,512,1024,2048 module 521 66K - 521 128 mtx_pool 2 16K - 2 LED 4 1K - 4 16,128 CAM XPT 58 4K - 309 16,32,64,128,512,1024,2048 osd 6 1K - 12 16,32,64,128 pmchooks 1 1K - 1 128 CAM DEV 15 30K - 24 2048 acpitask 1 8K - 1 hdaa 4 5K - 4 512,1024,2048 pgrp 2 1K - 2 128 session 2 1K - 2 128 proc 2 256K - 2 subproc 100 145K - 149 512,4096 cred 54 9K - 254 64,256 plimit 2 1K - 2 256 uidinfo 2 33K - 2 128 hdac 1 1K - 1 512 hdacc 1 1K - 1 32 vtbuf 24 5712K - 24 4096 acpisem 22 3K - 22 128 vt 11 3K - 11 256 sysctl 0 0K - 50 16,32,64 sysctloid 7607 378K - 7779 16,32,64,128 sysctltmp 0 0K - 6 32,64 tidhash 1 256K - 1 callout 9 3208K - 9 umtx 684 86K - 684 128 p1003.1b 1 1K - 1 16 SWAP 12 19681K - 12 64 bus 2257 362K - 8440 16,32,64,128,256,512,1024 bus-sc 157 310K - 5479 16,32,64,128,256,512,1024,2048,4096 devstat 32 65K - 32 32,4096 eventhandler 104 9K - 104 64,128 DEVFS3 230 58K - 270 256 kobj 349 1396K - 1111 4096 DEVFS1 202 101K - 238 512 Per-cpu 1 1K - 1 32 DEVFS 41 1K - 42 16,128 rman 360 41K - 754 16,32,128 sbuf 0 0K - 2295 16,32,64,128,256 sglist 3 1K - 3 32 taskqueue 127 19K - 171 16,32,64,128,256 terminal 11 3K - 11 256 Unitno 28 2K - 120 32,64 vmem 2 136K - 2 ioctlops 0 0K - 158 16,32,64 iov 1 1K - 34 64,256,512 msg 4 30K - 4 2048,4096 sem 4 106K - 4 2048,4096 shm 1 20K - 1 tty 14 14K - 14 1024 shmfd 1 8K - 1 pcb 12 8341K - 12 16,128,1024,2048 vfscache 1 16384K - 1 vfs_hash 1 8192K - 1 vnodes 1 1K - 1 256 mount 251 9K - 550 16,32,64,128,256 vnodemarker 0 0K - 5 512 BPF 3 1K - 3 128 ifnet 4 7K - 4 128,2048 ifaddr 34 10K - 34 32,64,256,512,2048,4096 clone 8 1K - 8 128 arpcom 2 1K - 2 16 lltable 6 3K - 6 512 routetbl 5 2K - 5 256,512 igmp 3 1K - 3 256 sctp_vrf 1 1K - 1 64 hostcache 1 28K - 1 syncache 1 64K - 1 mld 3 1K - 3 128 rpc 2 1K - 2 512 audit_evclass 187 6K - 228 32 ufs_quota 1 8192K - 1 vm_pgdata 7 8193K - 7 128 pfs_nodes 73 19K - 73 256 pfs_vncache 2 1K - 2 64 scsi_cd 0 0K - 11 16 GEOM 404 67K - 3278 16,32,64,128,256,512,1024,2048 memdesc 1 4K - 1 4096 feeder 20 2K - 24 32,128 acpidev 36 3K - 36 64 atkbddev 2 1K - 2 64 CAM CCB 0 0K - 5331 2048 mixer 3 12K - 3 4096 raid_data 0 0K - 480 32,128,256 CAM path 22 1K - 115 32 solaris 289279 163254K - 391115 16,32,64,128,256,512,1024,2048,4096 UART 6 5K - 6 16,1024 kstat_data 6 1K - 6 64 CAM periph 16 4K - 40 16,32,64,128,256 md_nvidia_data 0 0K - 78 512 pci_link 16 2K - 16 32,64,128 md_sii_data 0 0K - 78 512 acpi_perf 8 1K - 8 64 CAM queue 23 8K - 57 16,32,512 apmdev 1 1K - 1 128 madt_table 0 0K - 1 4096 CAM dev queue 8 1K - 8 64 CAM SIM 8 2K - 8 256 ddb_capture 1 48K - 1 io_apic 2 4K - 2 2048 MCA 8 1K - 8 128 msi 4 1K - 4 128 nexusdev 4 1K - 4 16 isadev 5 1K - 5 128 USB 46 59K - 52 16,32,64,128,256,512,1024,2048,4096 USBdev 35 4K - 35 32,64,128,256 linux 17 2K - 17 64 spicds 7 1K - 7 128 envy24ht 15 195K - 15 64,2048 cpuctl 1 1K - 1 64 crypto 1 1K - 1 512 cyclic 32 3K - 32 16,64,128 SDT 32 2K - 32 16,64 fbt 60661 7839K - 60661 128 iprtheap 17 54K - 17 32,64,128,256 nvidia 194 864K - 195 16,32,64,128,256,512,1024,2048,4096 ipmi 0 0K - 11 128 gem_name 59 10K - 62 32,4096 drm_global 2 1K - 2 128,256 drm_vblank 7 1K - 7 16,64 drm_dma 2 1K - 2 32 drm_sarea 1 1K - 1 16 drm_driver 32 2203K - 36 16,32,64,128,256,512,1024,4096 drm_ctxbitmap 1 4K - 1 4096 drm_sman 13 2K - 13 128 drm_hashtab 1 4096K - 1 drm_kms 90 18K - 107 16,32,64,128,256,2048 ttm_pd 6 9K - 6 16,2048 ttm_rman 2 1K - 2 256 ttm_zone 2 1K - 2 64 ttm_poolmgr 1 1K - 1 512 fdesc_mount 1 1K - 1 16 ------------------------------------------------------------------------ vmstat -z ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP UMA Kegs: 384, 0, 267, 3, 267, 0, 0 UMA Zones: 1664, 0, 267, 1, 267, 0, 0 UMA Slabs: 112, 0, 4719, 6, 7221, 0, 0 UMA RCntSlabs: 120, 0, 0, 0, 0, 0, 0 UMA Hash: 256, 0, 77, 13, 77, 0, 0 4 Bucket: 32, 0, 262, 854, 1813, 0, 0 6 Bucket: 48, 0, 276, 1052, 1590, 0, 0 8 Bucket: 64, 0, 62, 992, 883, 0, 0 12 Bucket: 96, 0, 36, 989, 167, 0, 0 16 Bucket: 128, 0, 131, 1016, 1263, 17, 0 32 Bucket: 256, 0, 130, 380, 285, 49, 0 64 Bucket: 512, 0, 119, 238, 465, 49, 0 128 Bucket: 1024, 0, 406, 46, 3736, 2, 0 vmem btag: 56, 0, 11416, 441, 11514, 167, 0 VM OBJECT: 256, 0, 590, 400, 1335, 0, 0 RADIX NODE: 144, 0, 6442, 416, 10926, 0, 0 MAP: 240, 0, 3, 61, 3, 0, 0 KMAP ENTRY: 128, 0, 8, 271, 8, 0, 0 MAP ENTRY: 128, 0, 103, 920, 1717, 0, 0 VMSPACE: 448, 0, 4, 204, 54, 0, 0 fakepg: 104, 0, 0, 0, 0, 0, 0 mt_zone: 4112, 0, 405, 0, 405, 0, 0 16: 16, 0, 6, 243, 6, 0, 0 16: 16, 0, 2762, 973, 3015, 0, 0 16: 16, 0, 147166, 740, 158653, 0, 0 16: 16, 0, 55, 941, 28215, 0, 0 16: 16, 0, 62, 685, 63, 0, 0 16: 16, 0, 732, 1011, 5473, 0, 0 16: 16, 0, 30, 219, 30, 0, 0 16: 16, 0, 1, 746, 18, 0, 0 32: 32, 0, 30, 1086, 117, 0, 0 32: 32, 0, 3091, 877, 3295, 0, 0 32: 32, 0, 65004, 716, 73960, 0, 0 32: 32, 0, 66, 554, 4216, 0, 0 32: 32, 0, 18, 1098, 98, 0, 0 32: 32, 0, 839, 897, 2874, 0, 0 32: 32, 0, 94, 650, 186, 0, 0 32: 32, 0, 8, 736, 164, 0, 0 64: 64, 0, 5, 181, 6, 0, 0 64: 64, 0, 517, 909, 589, 0, 0 64: 64, 0, 7078, 982, 33984, 0, 0 64: 64, 0, 767, 783, 1637, 0, 0 64: 64, 0, 80, 106, 80, 0, 0 64: 64, 0, 9187, 857, 9746, 0, 0 64: 64, 0, 131, 551, 131, 0, 0 64: 64, 0, 1038, 760, 1071, 0, 0 128: 128, 0, 34, 741, 95, 0, 0 128: 128, 0, 1899, 612, 1908, 0, 0 128: 128, 0, 121830, 806, 128094, 0, 0 128: 128, 0, 1077, 318, 27007, 0, 0 128: 128, 0, 112, 663, 192, 0, 0 128: 128, 0, 1895, 740, 4111, 0, 0 128: 128, 0, 29, 126, 30, 0, 0 128: 128, 0, 524, 499, 526, 0, 0 256: 256, 0, 2, 73, 2, 0, 0 256: 256, 0, 308, 307, 451, 0, 0 256: 256, 0, 372, 543, 2665, 0, 0 256: 256, 0, 64, 131, 740, 0, 0 256: 256, 0, 28, 347, 348, 0, 0 256: 256, 0, 909, 441, 1963, 0, 0 256: 256, 0, 31, 44, 31, 0, 0 256: 256, 0, 4, 191, 25, 0, 0 512: 512, 0, 2, 33, 9, 0, 0 512: 512, 0, 69, 197, 225, 0, 0 512: 512, 0, 130, 276, 32908, 0, 0 512: 512, 0, 16, 47, 35, 0, 0 512: 512, 0, 3, 32, 3, 0, 0 512: 512, 0, 327, 170, 2804, 0, 0 512: 512, 0, 47, 16, 47, 0, 0 512: 512, 0, 1, 62, 2, 0, 0 1024: 1024, 0, 0, 104, 99, 0, 0 1024: 1024, 0, 9, 19, 9, 0, 0 1024: 1024, 0, 8372, 68, 23684, 0, 0 1024: 1024, 0, 5, 23, 2074, 0, 0 1024: 1024, 0, 16, 12, 16, 0, 0 1024: 1024, 0, 230, 98, 1280, 0, 0 1024: 1024, 0, 2, 14, 2, 0, 0 1024: 1024, 0, 0, 0, 0, 0, 0 2048: 2048, 0, 2, 4, 9, 0, 0 2048: 2048, 0, 6, 4, 6, 0, 0 2048: 2048, 0, 11, 19, 39, 0, 0 2048: 2048, 0, 7, 53, 5343, 0, 0 2048: 2048, 0, 21, 9, 30, 0, 0 2048: 2048, 0, 48, 42, 466, 0, 0 2048: 2048, 0, 5, 1, 5, 0, 0 2048: 2048, 0, 0, 0, 0, 0, 0 4096: 4096, 0, 0, 0, 0, 0, 0 4096: 4096, 0, 5, 0, 5, 0, 0 4096: 4096, 0, 167, 4, 295, 0, 0 4096: 4096, 0, 10, 0, 12, 0, 0 4096: 4096, 0, 16, 0, 17, 0, 0 4096: 4096, 0, 25, 0, 415, 0, 0 4096: 4096, 0, 15, 0, 15, 0, 0 4096: 4096, 0, 349, 10, 1111, 0, 0 uint64 pcpu: 8, 0, 1398, 138, 1398, 0, 0 SLEEPQUEUE: 88, 0, 343, 680, 343, 0, 0 Files: 80, 0, 8, 874, 395, 0, 0 rl_entry: 40, 0, 6, 687, 6, 0, 0 TURNSTILE: 136, 0, 343, 277, 343, 0, 0 umtx pi: 96, 0, 0, 0, 0, 0, 0 MAC labels: 40, 0, 0, 0, 0, 0, 0 PROC: 1208, 0, 26, 46, 75, 0, 0 THREAD: 1168, 0, 302, 40, 445, 0, 0 cpuset: 72, 0, 287, 593, 430, 0, 0 cyclic_id_cache: 64, 0, 0, 0, 0, 0, 0 audit_record: 1248, 0, 0, 0, 0, 0, 0 mbuf_packet: 256, 25720710, 0, 10, 0, 0, 0 mbuf: 256, 25720710, 1, 134, 1, 0, 0 mbuf_cluster: 2048, 4018860, 0, 0, 0, 0, 0 mbuf_jumbo_page: 4096, 2009429, 0, 0, 0, 0, 0 mbuf_jumbo_9k: 9216, 1786158, 0, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 1339616, 0, 0, 0, 0, 0 mbuf_ext_refcnt: 4, 0, 0, 0, 0, 0, 0 sendfile_sync: 128, 0, 0, 0, 0, 0, 0 dtrace_state_cache: 4096, 0, 0, 0, 0, 0, 0 g_bio: 248, 0, 0, 672, 15403, 0, 0 DMAR_MAP_ENTRY: 120, 0, 0, 0, 0, 0, 0 ttyinq: 160, 0, 15, 57, 15, 0, 0 ttyoutq: 256, 0, 8, 67, 8, 0, 0 ata_request: 336, 0, 0, 88, 39, 0, 0 vtnet_tx_hdr: 24, 0, 0, 0, 0, 0, 0 cryptop: 88, 0, 0, 0, 0, 0, 0 cryptodesc: 72, 0, 0, 0, 0, 0, 0 nv_stack_t: 12288, 0, 2, 1, 6, 0, 0 FPU_save_area: 512, 0, 0, 0, 0, 0, 0 taskq_zone: 48, 0, 0, 415, 20, 0, 0 taskq_zone: 48, 0, 0, 0, 0, 0, 0 VNODE: 472, 0, 332, 164, 333, 0, 0 VNODEPOLL: 112, 0, 0, 0, 0, 0, 0 BUF TRIE: 144, 0, 0, 105948, 0, 0, 0 NAMEI: 1024, 0, 0, 104, 908, 0, 0 S VFS Cache: 108, 0, 248, 767, 294, 0, 0 STS VFS Cache: 148, 0, 0, 0, 0, 0, 0 L VFS Cache: 328, 0, 0, 0, 0, 0, 0 LTS VFS Cache: 368, 0, 0, 0, 0, 0, 0 NCLNODE: 528, 0, 0, 0, 0, 0, 0 DIRHASH: 1024, 0, 0, 0, 0, 0, 0 reference_cache: 40, 0, 0, 0, 0, 0, 0 reference_history_cache: 8, 0, 0, 0, 0, 0, 0 range_seg_cache: 64, 0, 9978, 934, 10433, 0, 0 zio_cache: 920, 0, 6, 814, 12019, 0, 0 zio_link_cache: 48, 0, 3, 1408, 7868, 0, 0 zio_buf_512: 512, 0, 590, 166, 1077, 0, 0 zio_data_buf_512: 512, 0, 54, 65, 56, 0, 0 zio_buf_1024: 1024, 0, 22, 58, 61, 0, 0 zio_data_buf_1024: 1024, 0, 40, 40, 40, 0, 0 zio_buf_1536: 1536, 0, 22, 24, 24, 0, 0 zio_data_buf_1536: 1536, 0, 23, 13, 23, 0, 0 zio_buf_2048: 2048, 0, 35, 25, 64, 0, 0 zio_data_buf_2048: 2048, 0, 14, 12, 15, 0, 0 zio_buf_2560: 2560, 0, 0, 15, 27, 0, 0 zio_data_buf_2560: 2560, 0, 17, 4, 17, 0, 0 zio_buf_3072: 3072, 0, 2, 4, 2, 0, 0 zio_data_buf_3072: 3072, 0, 8, 4, 8, 0, 0 zio_buf_3584: 3584, 0, 1, 0, 1, 0, 0 zio_data_buf_3584: 3584, 0, 2, 0, 2, 0, 0 zio_buf_4096: 4096, 0, 49, 706, 3790, 0, 0 zio_data_buf_4096: 4096, 0, 0, 24, 25, 0, 0 zio_buf_5120: 5120, 0, 1, 0, 1, 0, 0 zio_data_buf_5120: 5120, 0, 4, 0, 4, 0, 0 zio_buf_6144: 6144, 0, 1, 0, 1, 0, 0 zio_data_buf_6144: 6144, 0, 3, 0, 3, 0, 0 zio_buf_7168: 7168, 0, 0, 0, 0, 0, 0 zio_data_buf_7168: 7168, 0, 1, 0, 1, 0, 0 zio_buf_8192: 8192, 0, 1, 13, 187, 0, 0 zio_data_buf_8192: 8192, 0, 2, 1, 4, 0, 0 zio_buf_10240: 10240, 0, 1, 0, 1, 0, 0 zio_data_buf_10240: 10240, 0, 1, 0, 1, 0, 0 zio_buf_12288: 12288, 0, 1, 8, 45, 0, 0 zio_data_buf_12288: 12288, 0, 4, 0, 4, 0, 0 zio_buf_14336: 14336, 0, 0, 0, 0, 0, 0 zio_data_buf_14336: 14336, 0, 1, 0, 1, 0, 0 zio_buf_16384: 16384, 0, 398, 23, 543, 0, 0 zio_data_buf_16384: 16384, 0, 2, 0, 2, 0, 0 zio_buf_20480: 20480, 0, 0, 5, 19, 0, 0 zio_data_buf_20480: 20480, 0, 3, 0, 3, 0, 0 zio_buf_24576: 24576, 0, 0, 9, 16, 0, 0 zio_data_buf_24576: 24576, 0, 5, 0, 5, 0, 0 zio_buf_28672: 28672, 0, 0, 14, 83, 0, 0 zio_data_buf_28672: 28672, 0, 4, 0, 4, 0, 0 zio_buf_32768: 32768, 0, 0, 2, 3, 0, 0 zio_data_buf_32768: 32768, 0, 2, 0, 2, 0, 0 zio_buf_36864: 36864, 0, 1, 5, 8, 0, 0 zio_data_buf_36864: 36864, 0, 1, 2, 3, 0, 0 zio_buf_40960: 40960, 0, 0, 3, 3, 0, 0 zio_data_buf_40960: 40960, 0, 2, 0, 2, 0, 0 zio_buf_45056: 45056, 0, 0, 1, 1, 0, 0 zio_data_buf_45056: 45056, 0, 1, 0, 1, 0, 0 zio_buf_49152: 49152, 0, 0, 0, 0, 0, 0 zio_data_buf_49152: 49152, 0, 0, 0, 0, 0, 0 zio_buf_53248: 53248, 0, 0, 1, 1, 0, 0 zio_data_buf_53248: 53248, 0, 1, 0, 1, 0, 0 zio_buf_57344: 57344, 0, 0, 0, 0, 0, 0 zio_data_buf_57344: 57344, 0, 0, 0, 0, 0, 0 zio_buf_61440: 61440, 0, 0, 0, 0, 0, 0 zio_data_buf_61440: 61440, 0, 0, 0, 0, 0, 0 zio_buf_65536: 65536, 0, 0, 0, 0, 0, 0 zio_data_buf_65536: 65536, 0, 0, 0, 0, 0, 0 zio_buf_69632: 69632, 0, 0, 2, 2, 0, 0 zio_data_buf_69632: 69632, 0, 2, 0, 2, 0, 0 zio_buf_73728: 73728, 0, 0, 1, 1, 0, 0 zio_data_buf_73728: 73728, 0, 1, 0, 1, 0, 0 zio_buf_77824: 77824, 0, 0, 0, 0, 0, 0 zio_data_buf_77824: 77824, 0, 0, 0, 0, 0, 0 zio_buf_81920: 81920, 0, 0, 0, 0, 0, 0 zio_data_buf_81920: 81920, 0, 1, 0, 1, 0, 0 zio_buf_86016: 86016, 0, 0, 0, 0, 0, 0 zio_data_buf_86016: 86016, 0, 0, 0, 0, 0, 0 zio_buf_90112: 90112, 0, 0, 2, 2, 0, 0 zio_data_buf_90112: 90112, 0, 0, 1, 1, 0, 0 zio_buf_94208: 94208, 0, 0, 0, 0, 0, 0 zio_data_buf_94208: 94208, 0, 0, 0, 0, 0, 0 zio_buf_98304: 98304, 0, 1, 1, 2, 0, 0 zio_data_buf_98304: 98304, 0, 1, 0, 1, 0, 0 zio_buf_102400: 102400, 0, 0, 0, 0, 0, 0 zio_data_buf_102400: 102400, 0, 0, 0, 0, 0, 0 zio_buf_106496: 106496, 0, 0, 0, 0, 0, 0 zio_data_buf_106496: 106496, 0, 0, 0, 0, 0, 0 zio_buf_110592: 110592, 0, 0, 1, 1, 0, 0 zio_data_buf_110592: 110592, 0, 1, 0, 1, 0, 0 zio_buf_114688: 114688, 0, 0, 3, 14, 0, 0 zio_data_buf_114688: 114688, 0, 0, 0, 0, 0, 0 zio_buf_118784: 118784, 0, 0, 0, 0, 0, 0 zio_data_buf_118784: 118784, 0, 0, 0, 0, 0, 0 zio_buf_122880: 122880, 0, 0, 0, 0, 0, 0 zio_data_buf_122880: 122880, 0, 0, 0, 0, 0, 0 zio_buf_126976: 126976, 0, 0, 0, 0, 0, 0 zio_data_buf_126976: 126976, 0, 0, 0, 0, 0, 0 zio_buf_131072: 131072, 0, 1, 11, 61, 0, 0 zio_data_buf_131072: 131072, 0, 39, 5, 48, 0, 0 lz4_ctx: 16384, 0, 0, 0, 0, 0, 0 sa_cache: 80, 0, 272, 757, 272, 0, 0 dnode_t: 1096, 0, 746, 37, 1863, 0, 0 dmu_buf_impl_t: 336, 0, 1270, 127, 1729, 0, 0 arc_buf_hdr_t: 328, 0, 881, 163, 1332, 0, 0 arc_buf_t: 72, 0, 880, 770, 1359, 0, 0 zil_lwb_cache: 192, 0, 0, 0, 0, 0, 0 zfs_znode_cache: 368, 0, 272, 138, 272, 0, 0 pipe: 744, 0, 0, 40, 13, 0, 0 Mountpoints: 816, 0, 24, 41, 24, 0, 0 procdesc: 128, 0, 0, 0, 0, 0, 0 ksiginfo: 112, 0, 26, 1024, 26, 0, 0 itimer: 352, 0, 0, 0, 0, 0, 0 KNOTE: 128, 0, 0, 0, 0, 0, 0 socket: 696, 2062880, 0, 0, 0, 0, 0 ipq: 56, 125599, 0, 0, 0, 0, 0 udp_inpcb: 392, 2062880, 0, 0, 0, 0, 0 udpcb: 16, 2062965, 0, 0, 0, 0, 0 tcp_inpcb: 392, 2062880, 0, 0, 0, 0, 0 tcpcb: 1024, 2062880, 0, 0, 0, 0, 0 tcptw: 88, 27810, 0, 0, 0, 0, 0 syncache: 160, 15360, 0, 0, 0, 0, 0 hostcache: 136, 15370, 0, 0, 0, 0, 0 tcpreass: 40, 251262, 0, 0, 0, 0, 0 sackhole: 32, 0, 0, 0, 0, 0, 0 sctp_ep: 1408, 2062880, 0, 0, 0, 0, 0 sctp_asoc: 2352, 40000, 0, 0, 0, 0, 0 sctp_laddr: 48, 80012, 0, 0, 0, 0, 0 sctp_raddr: 728, 80000, 0, 0, 0, 0, 0 sctp_chunk: 136, 400026, 0, 0, 0, 0, 0 sctp_readq: 104, 400026, 0, 0, 0, 0, 0 sctp_stream_msg_out: 104, 400026, 0, 0, 0, 0, 0 sctp_asconf: 40, 400059, 0, 0, 0, 0, 0 sctp_asconf_ack: 48, 400060, 0, 0, 0, 0, 0 ripcb: 392, 2062880, 0, 0, 0, 0, 0 unpcb: 240, 2062880, 0, 0, 0, 0, 0 rtentry: 200, 0, 0, 0, 0, 0, 0 selfd: 56, 0, 0, 0, 0, 0, 0 SWAPMETA: 288, 8037731, 0, 0, 0, 0, 0 ------------------------------------------------------------------------ vmstat -i interrupt total rate irq6: fdc0 22 0 irq14: ata0 55 0 irq17: uhci0 ehci0 55 0 irq18: uhci2+ 304 0 irq19: uhci1 ahci0+ 5075 13 cpu0:timer 2410 6 irq259: hdac0 9 0 cpu6:timer 442 1 cpu1:timer 404 1 cpu2:timer 482 1 cpu7:timer 548 1 cpu5:timer 477 1 cpu4:timer 484 1 cpu3:timer 765 2 Total 11532 31 ------------------------------------------------------------------------ pstat -T 8/2062880 files 0M/147455M swap space ------------------------------------------------------------------------ pstat -s Device 512-blocks Used Avail Capacity /dev/#C:0xc0 50331392 0 50331392 0% /dev/ad14p2 50331392 0 50331392 0% /dev/usb/3.2.1 50331392 0 50331392 0% /dev/#C:0xd0 50331392 0 50331392 0% /dev/#C:0xd8 50331392 0 50331392 0% /dev/#C:0xe0 50331392 0 50331392 0% Total 301988352 0 301988352 0% ------------------------------------------------------------------------ iostat tty ada0 ada1 ada2 cpu tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 0 15 15.18 60 0.89 15.40 65 0.98 14.86 63 0.91 0 0 2 1 97 ------------------------------------------------------------------------ ipcs -a Message Queues: T ID KEY MODE OWNER GROUP CREATOR CGROUP CBYTES QNUM QBYTES LSPID LRPID STIME RTIME CTIME Shared Memory: T ID KEY MODE OWNER GROUP CREATOR CGROUP NATTCH SEGSZ CPID LPID ATIME DTIME CTIME Semaphores: T ID KEY MODE OWNER GROUP CREATOR CGROUP NSEMS OTIME CTIME ------------------------------------------------------------------------ ipcs -T msginfo: msgmax: 16384 (max characters in a message) msgmni: 40 (# of message queues) msgmnb: 2048 (max characters in a message queue) msgtql: 40 (max # of messages in system) msgssz: 8 (size of a message segment) msgseg: 2048 (# of message segments in system) shminfo: shmmax: 536870912 (max shared memory segment size) shmmin: 1 (min shared memory segment size) shmmni: 192 (max number of shared memory identifiers) shmseg: 128 (max shared memory segments per process) shmall: 131072 (max amount of shared memory in pages) seminfo: semmni: 50 (# of semaphore identifiers) semmns: 340 (# of semaphores in system) semmnu: 150 (# of undo structures in system) semmsl: 340 (max # of semaphores per id) semopm: 100 (max # of operations per semop call) semume: 50 (max # of undo entries per process) semusz: 632 (size in bytes of undo structure) semvmx: 32767 (semaphore maximum value) semaem: 16384 (adjust on exit max value) ------------------------------------------------------------------------ nfsstat Client Info: Rpc Counts: Getattr Setattr Lookup Readlink Read Write Create Remove 2 0 0 0 0 0 0 0 Rename Link Symlink Mkdir Rmdir Readdir RdirPlus Access 0 0 0 0 0 0 0 0 Mknod Fsstat Fsinfo PathConf Commit 0 1 1 0 0 Rpc Info: TimedOut Invalid X Replies Retries Requests 0 0 0 0 4 Cache Info: Attr Hits Misses Lkup Hits Misses BioR Hits Misses BioW Hits Misses 0 0 0 0 0 0 0 0 BioRLHits Misses BioD Hits Misses DirE Hits Misses Accs Hits Misses 0 0 0 0 0 0 0 0 Server Info: Getattr Setattr Lookup Readlink Read Write Create Remove 0 0 0 0 0 0 0 0 Rename Link Symlink Mkdir Rmdir Readdir RdirPlus Access 0 0 0 0 0 0 0 0 Mknod Fsstat Fsinfo PathConf Commit 0 0 0 0 0 Server Ret-Failed 0 Server Faults 0 Server Cache Stats: Inprog Idem Non-idem Misses 0 0 0 0 Server Write Gathering: WriteOps WriteRPC Opsaved 0 0 0 ------------------------------------------------------------------------ netstat -s tcp: 0 packets sent 0 data packets (0 bytes) 0 data packets (0 bytes) retransmitted 0 data packets unnecessarily retransmitted 0 resends initiated by MTU discovery 0 ack-only packets (0 delayed) 0 URG only packets 0 window probe packets 0 window update packets 0 control packets 0 packets received 0 acks (for 0 bytes) 0 duplicate acks 0 acks for unsent data 0 packets (0 bytes) received in-sequence 0 completely duplicate packets (0 bytes) 0 old duplicate packets 0 packets with some dup. data (0 bytes duped) 0 out-of-order packets (0 bytes) 0 packets (0 bytes) of data after window 0 window probes 0 window update packets 0 packets received after close 0 discarded for bad checksums 0 discarded for bad header offset fields 0 discarded because packet too short 0 discarded due to memory problems 0 connection requests 0 connection accepts 0 bad connection attempts 0 listen queue overflows 0 ignored RSTs in the windows 0 connections established (including accepts) 0 connections closed (including 0 drops) 0 connections updated cached RTT on close 0 connections updated cached RTT variance on close 0 connections updated cached ssthresh on close 0 embryonic connections dropped 0 segments updated rtt (of 0 attempts) 0 retransmit timeouts 0 connections dropped by rexmit timeout 0 persist timeouts 0 connections dropped by persist timeout 0 Connections (fin_wait_2) dropped because of timeout 0 keepalive timeouts 0 keepalive probes sent 0 connections dropped by keepalive 0 correct ACK header predictions 0 correct data packet header predictions 0 syncache entries added 0 retransmitted 0 dupsyn 0 dropped 0 completed 0 bucket overflow 0 cache overflow 0 reset 0 stale 0 aborted 0 badack 0 unreach 0 zone failures 0 cookies sent 0 cookies received 0 hostcache entries added 0 bucket overflow 0 SACK recovery episodes 0 segment rexmits in SACK recovery episodes 0 byte rexmits in SACK recovery episodes 0 SACK options (SACK blocks) received 0 SACK options (SACK blocks) sent 0 SACK scoreboard overflow 0 packets with ECN CE bit set 0 packets with ECN ECT(0) bit set 0 packets with ECN ECT(1) bit set 0 successful ECN handshakes 0 times ECN reduced the congestion window udp: 0 datagrams received 0 with incomplete header 0 with bad data length field 0 with bad checksum 0 with no checksum 0 dropped due to no socket 0 broadcast/multicast datagrams undelivered 0 dropped due to full socket buffers 0 not for hashed pcb 0 delivered 0 datagrams output 0 times multicast source filter matched ip: 0 total packets received 0 bad header checksums 0 with size smaller than minimum 0 with data size < data length 0 with ip length > max ip packet size 0 with header length < data size 0 with data length < header length 0 with bad options 0 with incorrect version number 0 fragments received 0 fragments dropped (dup or out of space) 0 fragments dropped after timeout 0 packets reassembled ok 0 packets for this host 0 packets for unknown/unsupported protocol 0 packets forwarded (0 packets fast forwarded) 0 packets not forwardable 0 packets received for unknown multicast group 0 redirects sent 0 packets sent from this host 0 packets sent with fabricated ip header 0 output packets dropped due to no bufs, etc. 0 output packets discarded due to no route 0 output datagrams fragmented 0 fragments created 0 datagrams that can't be fragmented 0 tunneling packets that can't find gif 0 datagrams with bad address in header icmp: 0 calls to icmp_error 0 errors not generated in response to an icmp message 0 messages with bad code fields 0 messages less than the minimum length 0 messages with bad checksum 0 messages with bad length 0 multicast echo requests ignored 0 multicast timestamp requests ignored 0 message responses generated 0 invalid return addresses 0 no return routes igmp: 0 messages received 0 messages received with too few bytes 0 messages received with wrong TTL 0 messages received with bad checksum 0 V1/V2 membership queries received 0 V3 membership queries received 0 membership queries received with invalid field(s) 0 general queries received 0 group queries received 0 group-source queries received 0 group-source queries dropped 0 membership reports received 0 membership reports received with invalid field(s) 0 membership reports received for groups to which we belong 0 V3 reports received without Router Alert 0 membership reports sent arp: 0 ARP requests sent 0 ARP replies sent 0 ARP requests received 0 ARP replies received 0 ARP packets received 0 total packets dropped due to no ARP entry 0 ARP entrys timed out 0 Duplicate IPs seen ip6: 0 total packets received 0 with size smaller than minimum 0 with data size < data length 0 with bad options 0 with incorrect version number 0 fragments received 0 fragments dropped (dup or out of space) 0 fragments dropped after timeout 0 fragments that exceeded limit 0 packets reassembled ok 0 packets for this host 0 packets forwarded 0 packets not forwardable 0 redirects sent 0 packets sent from this host 0 packets sent with fabricated ip header 0 output packets dropped due to no bufs, etc. 0 output packets discarded due to no route 0 output datagrams fragmented 0 fragments created 0 datagrams that can't be fragmented 0 packets that violated scope rules 0 multicast packets which we don't join Mbuf statistics: 0 one mbuf 0 one ext mbuf 0 two or more ext mbuf 0 packets whose headers are not contiguous 0 tunneling packets that can't find gif 0 packets discarded because of too many headers 0 failures of source address selection Source addresses selection rule applied: icmp6: 0 calls to icmp6_error 0 errors not generated in response to an icmp6 message 0 errors not generated because of rate limitation 0 messages with bad code fields 0 messages < minimum length 0 bad checksums 0 messages with bad length Histogram of error messages to be generated: 0 no route 0 administratively prohibited 0 beyond scope 0 address unreachable 0 port unreachable 0 packet too big 0 time exceed transit 0 time exceed reassembly 0 erroneous header field 0 unrecognized next header 0 unrecognized option 0 redirect 0 unknown 0 message responses generated 0 messages with too many ND options 0 messages with bad ND options 0 bad neighbor solicitation messages 0 bad neighbor advertisement messages 0 bad router solicitation messages 0 bad router advertisement messages 0 bad redirect messages 0 path MTU changes rip6: 0 messages received 0 checksum calculations on inbound 0 messages with bad checksum 0 messages dropped due to no socket 0 multicast messages dropped due to no socket 0 messages dropped due to full socket buffers 0 delivered 0 datagrams output ------------------------------------------------------------------------ netstat -m netstat: invalid address (0x0) 1/144/145 mbufs in use (current/cache/total) 18446744073709551606/10/0/4018860 mbuf clusters in use (current/cache/total/max) 0/10 mbuf+clusters out of packet secondary zone in use (current/cache) 0/0/0/2009429 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/1786158 9k jumbo clusters in use (current/cache/total/max) 0/0/0/1339616 16k jumbo clusters in use (current/cache/total/max) 18014398509481964K/56K/36K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) ------------------------------------------------------------------------ netstat -anr Routing tables Internet: Destination Gateway Flags Netif Expire Internet6: Destination Gateway Flags Netif Expire ------------------------------------------------------------------------ netstat -anA ------------------------------------------------------------------------ netstat -aL ------------------------------------------------------------------------ fstat fstat: can't read file 1 at 0x200007fffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 4 at 0x780000ffff fstat: can't read file 7 at 0x200007fffffffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0x200007fffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 4 at 0x780000ffff fstat: can't read file 7 at 0x200007fffffffff fstat: can't read file 8 at 0x4000000001fffff fstat: can't read file 10 at 0x780000ffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0x200007fffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 4 at 0x780000ffff fstat: can't read file 7 at 0x200007fffffffff fstat: can't read file 8 at 0x4000000001fffff fstat: can't read file 10 at 0x780000ffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 USER CMD PID FD MOUNT INUM MODE SZ|DV R/W root zfs 75 root - - error - root zfs 75 wd - - error - root zfs 75 text - - error - root zfs 75 ctty /dev 13 crw------- console rw root zfs 75 0 /dev 13 crw------- console rw root zfs 75 6 /dev 13 crw------- console rw root sh 72 root - - error - root sh 72 wd - - error - root sh 72 text - - error - root sh 72 ctty /dev 13 crw------- console rw root sh 72 0 /dev 13 crw------- console rw root sh 72 6 /dev 13 crw------- console rw root sh 24 root - - error - root sh 24 wd - - error - root sh 24 text - - error - root sh 24 ctty /dev 13 crw------- console rw root sh 24 0 /dev 13 crw------- console rw root sh 24 6 /dev 13 crw------- console rw root init 1 root - - error - root init 1 wd - - error - root init 1 text - - error - root kernel 0 root - - error - root kernel 0 wd - - error - ------------------------------------------------------------------------ dmesg Copyright (c) 1992-2014 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.0-CURRENT #1 r260896: Mon Jan 20 12:31:46 CST 2014 root@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER amd64 FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 info: [drm] Initialized drm 1.1.0 20060810 CPU: Intel(R) Xeon(R) CPU E5410 @ 2.33GHz (2327.55-MHz K8-class CPU) Origin="GenuineIntel" Id=0x10676 Family=0x6 Model=0x17 Stepping=6 Features=0xbfebfbff Features2=0xce3bd AMD Features=0x20100800 AMD Features2=0x1 TSC: P-state invariant, performance statistics real memory = 68719476736 (65536 MB) avail memory = 65641644032 (62600 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs FreeBSD/SMP: 2 package(s) x 4 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP): APIC ID: 3 cpu4 (AP): APIC ID: 4 cpu5 (AP): APIC ID: 5 cpu6 (AP): APIC ID: 6 cpu7 (AP): APIC ID: 7 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard random: initialized kbd1 at kbdmux0 cryptosoft0: on motherboard acpi0: on motherboard acpi0: Power Button (fixed) unknown: I/O range not supported cpu0: on acpi0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 cpu4: on acpi0 cpu5: on acpi0 cpu6: on acpi0 cpu7: on acpi0 hpet0: iomem 0xfed00000-0xfed003ff irq 0,8 on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 atrtc0: port 0x70-0x71 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: at device 2.0 on pci0 pci1: on pcib1 pcib2: irq 16 at device 0.0 on pci1 pci2: on pcib2 pcib3: irq 16 at device 0.0 on pci2 pci3: on pcib3 pcib4: at device 0.0 on pci3 pci4: on pcib4 pcib5: at device 0.2 on pci3 pci5: on pcib5 pcib6: irq 18 at device 2.0 on pci2 pci6: on pcib6 em0: port 0x2000-0x201f mem 0xd9220000-0xd923ffff,0xd9200000-0xd921ffff irq 18 at device 0.0 on pci6 em0: Using an MSI interrupt em0: Ethernet address: 00:30:48:f2:29:9c em1: port 0x2020-0x203f mem 0xd9260000-0xd927ffff,0xd9240000-0xd925ffff irq 19 at device 0.1 on pci6 em1: Using an MSI interrupt em1: Ethernet address: 00:30:48:f2:29:9d pcib7: at device 0.3 on pci1 pci7: on pcib7 pcib8: at device 4.0 on pci0 pci8: on pcib8 vgapci0: port 0x3000-0x307f mem 0xd8000000-0xd8ffffff,0xc0000000-0xc7ffffff,0xc8000000-0xc9ffffff irq 16 at device 0.0 on pci8 nvidia0: on vgapci0 vgapci0: child nvidia0 requested pci_enable_io vgapci0: child nvidia0 requested pci_enable_io hdac0: mem 0xd9000000-0xd9003fff irq 17 at device 0.1 on pci8 pcib9: at device 6.0 on pci0 pci9: on pcib9 pci0: at device 8.0 (no driver attached) pcib10: irq 17 at device 28.0 on pci0 pci10: on pcib10 pcib11: irq 16 at device 0.0 on pci10 pci11: on pcib11 pcm0: port 0x4080-0x409f,0x4000-0x407f irq 16 at device 0.0 on pci11 pcm0: system configuration SubVendorID: 0x1412, SubDeviceID: 0x2403 XIN2 Clock Source: 24.576MHz(96kHz*256) MPU-401 UART(s) #: not implemented ADC #: 1 and SPDIF receiver connected DAC #: 4 Multi-track converter type: AC'97(SDATA_OUT:packed) S/PDIF(IN/OUT): 1/1 ID# 0x00 GPIO(mask/dir/state): 0xff/0xff/0xff uhci0: port 0x1800-0x181f irq 17 at device 29.0 on pci0 usbus0 on uhci0 uhci1: port 0x1820-0x183f irq 19 at device 29.1 on pci0 usbus1 on uhci1 uhci2: port 0x1840-0x185f irq 18 at device 29.2 on pci0 usbus2 on uhci2 ehci0: mem 0xd9600400-0xd96007ff irq 17 at device 29.7 on pci0 usbus3: EHCI version 1.0 usbus3 on ehci0 pcib12: at device 30.0 on pci0 pci12: on pcib12 vgapci1: port 0x5000-0x50ff mem 0xd0000000-0xd7ffffff,0xd9300000-0xd930ffff irq 18 at device 1.0 on pci12 drmn1: on vgapci1 info: [drm] RADEON_IS_PCI info: [drm] initializing kernel modesetting (RV100 0x1002:0x515E 0x15D9:0x8080). info: [drm] register mmio base: 0xD9300000 info: [drm] register mmio size: 65536 info: [drm] radeon_atrm_get_bios: ===> Try ATRM... info: [drm] radeon_atrm_get_bios: pci_find_class() found: 0:8:0:0, vendor=10de, device=104a info: [drm] radeon_atrm_get_bios: Get ACPI device handle info: [drm] radeon_acpi_vfct_bios: ===> Try VFCT... info: [drm] radeon_acpi_vfct_bios: Get "VFCT" ACPI table info: [drm] radeon_acpi_vfct_bios: Failed to get "VFCT" table: AE_NOT_FOUND info: [drm] igp_read_bios_from_vram: ===> Try IGP's VRAM... info: [drm] igp_read_bios_from_vram: VRAM base address: 0xd0000000 info: [drm] igp_read_bios_from_vram: Map address: 0xfffff800d0000000 (262144 bytes) info: [drm] igp_read_bios_from_vram: Incorrect BIOS signature: 0x0000 info: [drm] radeon_read_bios: ===> Try PCI Expansion ROM... info: [drm] radeon_read_bios: Map address: 0xfffff800000c0000 (131072 bytes) drmn1: info: VRAM: 128M 0x00000000D0000000 - 0x00000000D7FFFFFF (16M used) drmn1: info: GTT: 512M 0x00000000B0000000 - 0x00000000CFFFFFFF info: [drm] Supports vblank timestamp caching Rev 1 (10.10.2010). info: [drm] Driver supports precise vblank timestamp query. info: [drm] radeon: irq initialized. info: [drm] Detected VRAM RAM=128M, BAR=128M info: [drm] RAM width 64bits SDR [TTM] Zone kernel: Available graphics memory: 33006090 kiB [TTM] Zone dma32: Available graphics memory: 2097152 kiB [TTM] Initializing pool allocator info: [drm] radeon: 16M of VRAM memory ready info: [drm] radeon: 512M of GTT memory ready. info: [drm] GART: num cpu pages 131072, num gpu pages 131072 info: [drm] PCI GART of 512M enabled (table at 0x0000000011200000). drmn1: info: WB disabled drmn1: info: fence driver on ring 0 use gpu addr 0x00000000b0000000 and cpu addr 0x0xfffff8001083c000 info: [drm] Loading R100 Microcode info: [drm] radeon: ring at 0x00000000B0001000 info: [drm] ring test succeeded in 1 usecs info: [drm] ib test succeeded in 0 usecs info: [drm] radeon_device_init: Taking over the fictitious range 0xd0000000-0xd4000000 iicbus0: on iicbb0 addr 0xff iic0: on iicbus0 iicbus1: on iicbb1 addr 0xff iic1: on iicbus1 iicbus2: on iicbb2 addr 0xff iic2: on iicbus2 iicbus3: on iicbb3 addr 0xff iic3: on iicbus3 info: [drm] Radeon Display Connectors info: [drm] Connector 0: info: [drm] VGA-1 info: [drm] DDC: 0x60 0x60 0x60 0x60 0x60 0x60 0x60 0x60 info: [drm] Encoders: info: [drm] CRT1: INTERNAL_DAC1 info: [drm] Connector 1: info: [drm] DVI-I-1 info: [drm] HPD2 info: [drm] DDC: 0x6c 0x6c 0x6c 0x6c 0x6c 0x6c 0x6c 0x6c info: [drm] Encoders: info: [drm] CRT2: INTERNAL_DAC2 info: [drm] DFP2: INTERNAL_DVO1 composite sync not supported composite sync not supported info: [drm] fb mappable at 0xD0040000 info: [drm] vram apper at 0xD0000000 info: [drm] size 2076672 info: [drm] fb depth is 8 info: [drm] pitch is 1920 fbd1 on drmn1 vt_allocate: Replace existing VT driver. info: [drm] Initialized radeon 2.29.0 20080528 vgapci1: Boot video device isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1860-0x186f at device 31.1 on pci0 ata0: at channel 0 on atapci0 ahci0: port 0x18a0-0x18a7,0x1874-0x1877,0x1878-0x187f,0x1870-0x1873,0x1880-0x189f mem 0xd9600800-0xd9600bff irq 19 at device 31.2 on pci0 ahci0: AHCI v1.10 with 6 3Gbps ports, Port Multiplier supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahcich4: at channel 4 on ahci0 ahcich5: at channel 5 on ahci0 ichsmb0: port 0x1100-0x111f irq 19 at device 31.3 on pci0 smbus0: on ichsmb0 acpi_button0: on acpi0 ipmi0: port 0xca2-0xca3 on acpi0 ipmi0: KCS mode found at io 0xca2 on acpi atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fd0: <1440-KB 3.5" drive> on fdc0 drive 0 ppc0: port 0x378-0x37f,0x778-0x77f irq 7 drq 3 on acpi0 ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/9 bytes threshold ppbus0: on ppc0 lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 ichwd0 on isa0 orm0: at iomem 0xc0000-0xcafff on isa0 coretemp0: on cpu0 est0: on cpu0 p4tcc0: on cpu0 coretemp1: on cpu1 est1: on cpu1 p4tcc1: on cpu1 coretemp2: on cpu2 est2: on cpu2 p4tcc2: on cpu2 coretemp3: on cpu3 est3: on cpu3 p4tcc3: on cpu3 coretemp4: on cpu4 est4: on cpu4 p4tcc4: on cpu4 coretemp5: on cpu5 est5: on cpu5 p4tcc5: on cpu5 coretemp6: on cpu6 est6: on cpu6 p4tcc6: on cpu6 coretemp7: on cpu7 est7: on cpu7 p4tcc7: on cpu7 ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Timecounters tick every 1.000 msec vboxdrv: fAsync=0 offMin=0x387 offMax=0x491 hdacc0: at cad 0 on hdac0 hdaa0: at nid 1 on hdacc0 pcm1: at nid 4 on hdaa0 pcm2: at nid 5 on hdaa0 random: unblocking device. usbus0: 12Mbps Full Speed USB v1.0 usbus1: 12Mbps Full Speed USB v1.0 usbus2: 12Mbps Full Speed USB v1.0 usbus3: 480Mbps High Speed USB v2.0 ugen3.1: at usbus3 uhub0: on usbus3 ugen2.1: at usbus2 uhub1: on usbus2 ugen1.1: at usbus1 uhub2: on usbus1 ugen0.1: at usbus0 uhub3: on usbus0 ipmi0: IPMI device rev. 1, firmware rev. 1.64, version 2.0 ata0: DMA limited to UDMA33, controller found non-ATA66 cable ipmi0: Number of channels 8 ipmi0: Attached watchdog uhub2: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered uhub3: 2 ports with 2 removable, self powered ada0 at ahcich0 bus 0 scbus1 target 0 lun 0 ada0: ATA-8 SATA 3.x device ada0: Serial Number 5YDA1ZL4 ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada0: quirks=0x1<4K> ada0: Previously was known as ad4 ada1 at ahcich1 bus 0 scbus2 target 0 lun 0 ada1: ATA-8 SATA 3.x device ada1: Serial Number 5YD6FPLG ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada1: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada1: quirks=0x1<4K> ada1: Previously was known as ad6 ada2 at ahcich2 bus 0 scbus3 target 0 lun 0 ada2: ATA-8 SATA 3.x device ada2: Serial Number 5YDA3PC5 ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada2: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada2: quirks=0x1<4K> ada2: Previously was known as ad8 ada3 at ahcich3 bus 0 scbus4 target 0 lun 0 ada3: ATA-8 SATA 3.x device ada3: Serial Number 5YD9Y0P4 ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada3: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada3: quirks=0x1<4K> ada3: Previously was known as ad10 ada4 at ahcich4 bus 0 scbus5 target 0 lun 0 ada4: ATA-8 SATA 3.x device ada4: Serial Number 5YDA1R5W ada4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada4: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada4: quirks=0x1<4K> ada4: Previously was known as ad12 ada5 at ahcich5 bus 0 scbus6 target 0 lun 0 ada5: ATA-8 SATA 3.x device ada5: Serial Number 5YD5RBS8 ada5: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada5: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada5: quirks=0x1<4K> ada5: Previously was known as ad14 cd0 at ata0 bus 0 scbus0 target 0 lun 0 cd0: Removable CD-ROM SCSI-0 device cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present Netvsc initializing... done! SMP: AP CPU #6 Launched! SMP: AP CPU #1 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #7 Launched! SMP: AP CPU #5 Launched! SMP: AP CPU #4 Launched! SMP: AP CPU #3 Launched! Root mount waiting for: usbus3 uhub0: 6 ports with 6 removable, self powered Root mount waiting for: usbus3 Root mount waiting for: usbus3 ugen3.2: at usbus3 ukbd0: on usbus3 kbd2 at ukbd0 Trying to mount root from zfs:zroot/ROOT/default []... ugen0.2: at usbus0 ugen1.2: at usbus1 Setting hostuuid: 53d19f64-d663-a017-8922-0030488e9ff3. Setting hostid: 0xf53a926e. Entropy harvesting: interrupts ethernet point_to_point swi. Starting file system checks: Mounting local file systems:. panic: solaris assert: l->l_phys->l_hdr.lh_block_type == ((1ULL << 63) + 0) (0x0 == 0x8000000000000000), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c, line: 534 cpuid = 7 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe100c499090 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe100c499140 vpanic() at vpanic+0x126/frame 0xfffffe100c499180 panic() at panic+0x43/frame 0xfffffe100c4991e0 assfail3() at assfail3+0x2f/frame 0xfffffe100c499200 zap_get_leaf_byblk() at zap_get_leaf_byblk+0x4e1/frame 0xfffffe100c499250 zap_deref_leaf() at zap_deref_leaf+0xd1/frame 0xfffffe100c4992a0 fzap_cursor_retrieve() at fzap_cursor_retrieve+0xce/frame 0xfffffe100c499310 zap_cursor_retrieve() at zap_cursor_retrieve+0x20a/frame 0xfffffe100c499390 ddt_zap_walk() at ddt_zap_walk+0x5e/frame 0xfffffe100c499650 ddt_walk() at ddt_walk+0x78/frame 0xfffffe100c499680 dsl_scan_sync() at dsl_scan_sync+0x6ec/frame 0xfffffe100c4999e0 spa_sync() at spa_sync+0x5c4/frame 0xfffffe100c499ad0 txg_sync_thread() at txg_sync_thread+0x256/frame 0xfffffe100c499bb0 fork_exit() at fork_exit+0x84/frame 0xfffffe100c499bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe100c499bf0 --- trap 0, rip = 0, rsp = 0xfffffe100c499cb0, rbp = 0 --- KDB: enter: panic Uptime: 14s Dumping 2263 out of 64465 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% ------------------------------------------------------------------------ kernel config options CONFIG_AUTOGENERATED ident VT-LER machine amd64 cpu HAMMER makeoptions WITH_CTF=1 makeoptions DEBUG=-g options ZFS options XENHVM options USB_DEBUG options ATH_ENABLE_11N options AH_AR5416_INTERRUPT_MITIGATION options AH_SUPPORT_AR5416 options IEEE80211_SUPPORT_MESH options IEEE80211_AMPDU_AGE options IEEE80211_DEBUG options SC_PIXEL_MODE options VESA options AHD_REG_PRETTY_PRINT options AHC_REG_PRETTY_PRINT options ATA_STATIC_ID options ACPI_DMAR options SMP options MALLOC_DEBUG_MAXZONES=8 options INVARIANT_SUPPORT options INVARIANTS options DEADLKRES options GDB options DDB options KDB_TRACE options KDB options INCLUDE_CONFIG_FILE options DDB_CTF options KDTRACE_HOOKS options KDTRACE_FRAME options MAC options CAPABILITIES options CAPABILITY_MODE options AUDIT options HWPMC_HOOKS options KBD_INSTALL_CDEV options PRINTF_BUFR_SIZE=128 options _KPOSIX_PRIORITY_SCHEDULING options SYSVSEM options SYSVMSG options SYSVSHM options STACK options KTRACE options SCSI_DELAY=5000 options COMPAT_FREEBSD7 options COMPAT_FREEBSD6 options COMPAT_FREEBSD5 options COMPAT_FREEBSD4 options COMPAT_FREEBSD32 options GEOM_LABEL options GEOM_RAID options GEOM_PART_GPT options PSEUDOFS options PROCFS options CD9660 options MSDOSFS options NFS_ROOT options NFSLOCKD options NFSD options NFSCL options MD_ROOT options QUOTA options UFS_GJOURNAL options UFS_DIRHASH options UFS_ACL options SOFTUPDATES options FFS options SCTP options TCP_OFFLOAD options INET6 options INET options PREEMPTION options SCHED_ULE options NEW_PCIB options GEOM_PART_MBR options GEOM_PART_EBR_COMPAT options GEOM_PART_EBR options GEOM_PART_BSD device isa device mem device io device uart_ns8250 device cpufreq device acpi device pci device fdc device ahci device ata device mvs device siis device ahc device ahd device esp device hptiop device isp device mpt device mps device sym device trm device adv device adw device aic device bt device isci device scbus device ch device da device sa device cd device pass device ses device amr device arcmsr device ciss device dpt device hptmv device hptnr device hptrr device hpt27xx device iir device ips device mly device twa device tws device aac device aacp device aacraid device ida device mfi device mlx device twe device atkbdc device atkbd device psm device kbdmux device splash device agp device cbb device pccard device cardbus device uart device ppc device ppbus device lpt device ppi device puc device bxe device de device em device igb device ixgbe device le device ti device txp device vx device miibus device ae device age device alc device ale device bce device bfe device bge device cas device dc device et device fxp device gem device hme device jme device lge device msk device nfe device nge device pcn device re device rl device sf device sge device sis device sk device ste device stge device tl device tx device vge device vr device wb device xl device cs device ed device ex device ep device fe device sn device xe device wlan device wlan_wep device wlan_ccmp device wlan_tkip device wlan_amrr device an device ath device ath_pci device ath_hal device ath_rate_sample device ipw device iwi device iwn device malo device mwl device ral device wi device wpi device loop device random device padlock_rng device rdrand_rng device ether device vlan device tun device md device gif device faith device firmware device bpf device uhci device ohci device ehci device xhci device usb device ukbd device umass device sound device snd_cmi device snd_csa device snd_emu10kx device snd_es137x device snd_hda device snd_ich device snd_via8233 device mmc device mmcsd device sdhci device virtio device virtio_pci device vtnet device virtio_blk device virtio_scsi device virtio_balloon device hyperv device xenpci device vmx device vt device vt_vga ------------------------------------------------------------------------ ddb capture buffer -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From owner-freebsd-fs@FreeBSD.ORG Sat Jan 25 06:07:59 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 78F16197 for ; Sat, 25 Jan 2014 06:07:59 +0000 (UTC) Received: from hergotha.csail.mit.edu (wollman-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1CDE412C2 for ; Sat, 25 Jan 2014 06:07:58 +0000 (UTC) Received: from hergotha.csail.mit.edu (localhost [127.0.0.1]) by hergotha.csail.mit.edu (8.14.7/8.14.7) with ESMTP id s0P67tWV024324 for ; Sat, 25 Jan 2014 01:07:55 -0500 (EST) (envelope-from wollman@hergotha.csail.mit.edu) Received: (from wollman@localhost) by hergotha.csail.mit.edu (8.14.7/8.14.4/Submit) id s0P67sjl024320; Sat, 25 Jan 2014 01:07:54 -0500 (EST) (envelope-from wollman) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <21219.21690.165105.561362@hergotha.csail.mit.edu> Date: Sat, 25 Jan 2014 01:07:54 -0500 From: Garrett Wollman To: freebsd-fs@freebsd.org Subject: ZFS read performance anomaly / performance regression X-Mailer: VM 7.17 under 21.4 (patch 22) "Instant Classic" XEmacs Lucid X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (hergotha.csail.mit.edu [127.0.0.1]); Sat, 25 Jan 2014 01:07:55 -0500 (EST) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=disabled version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hergotha.csail.mit.edu X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Jan 2014 06:07:59 -0000 Consider three 1-TiB files, equal in dignity, all on the same ZFS dataset: -rw-r--r-- 1 root wheel 1099511627776 Jan 7 19:03 test -rw-r--r-- 1 root wheel 1099511627776 Jan 24 22:38 test-truncate -rw-r--r-- 1 root wheel 1099511628800 Jan 24 21:32 test2 File "test" was created by writing zeros; "test2" was created by seeking to 1 TiB and writing a block of zeros, and "test-truncate" was created with "truncate -s 1T". Compression is disabled on this dataset, and "uname -a" reports: FreeBSD nfs-prod-3.csail.mit.edu 9.2-RELEASE-p2 FreeBSD 9.2-RELEASE-p2 #2 r260414M: Tue Jan 7 17:32:43 EST 2014 wollman@xyz.csail.mit.edu:/usr/obj/usr/src/sys/CSAIL amd64 A naive observer would expect that reading "test" should be significantly slower than reading either of the other files, since it requires reading data from the disk, and that there should be no significant difference between the other two, which consist entirely of cacheable metadata and have no, or very little, data stored on disk. But no: x noholes + oseek * truncate +------------------------------------------------------------------------------+ |x x x x * *+ * * * + + + + | | |________AM______| |________A__M___|__|_______A__M______|| +------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 5 71.300931 82.89801 77.044237 76.452097 4.3961747 + 5 97.931676 111.36284 108.52787 106.5687 5.5595081 Difference at 95.0% confidence 30.1166 +/- 7.30929 39.3928% +/- 9.56062% (Student's t, pooled s = 5.01171) * 5 89.187745 101.74893 98.825988 97.497923 4.9188155 Difference at 95.0% confidence 21.0458 +/- 6.80337 27.5281% +/- 8.89887% (Student's t, pooled s = 4.66482) (Values are the time to read 128 GiB of each file, which is more than the available memory for caching on this server. Measurements were interleaved abcabcabcabcabc, but server was not rebooted nor completely idle. Lower numbers are better.) Now consider the same test on an older, slower (but also completely idle) server, running an older FreeBSD release, but with an identical zpool configuration (88 spindles plus log and cache SSDs): FreeBSD nfs-backup-1.csail.mit.edu 9.1-RELEASE-p7 FreeBSD 9.1-RELEASE-p7 #34 r255669M: Fri Oct 11 13:04:40 EDT 2013 wollman@xyz.csail.mit.edu:/usr/obj/usr/src/sys/CSAIL amd64 Files were created in the same manner in a fresh, unsnapshotted dataset on this server, and the same shell script was used to time reads (even though the older machine has significantly less memory): x noholes2 (9.1) + oseek2 (9.1) * truncate2 (9.1) +------------------------------------------------------------------------------+ |** + x | |** * + + ++ xxx| ||A_| |_MA__| |A|| +------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 5 82.813107 83.970546 83.527358 83.427247 0.43568711 + 5 47.283031 51.450488 49.063725 49.493986 1.6446236 Difference at 95.0% confidence -33.9333 +/- 1.75456 -40.6741% +/- 2.10311% (Student's t, pooled s = 1.20304) * 5 44.460431 46.148431 44.636276 44.937538 0.70509763 Difference at 95.0% confidence -38.4897 +/- 0.854768 -46.1357% +/- 1.02457% (Student's t, pooled s = 0.586083) This is more like what I was expecting! But there's still this anomaly: x oseek2 (9.1) + truncate2 (9.1) +------------------------------------------------------------------------------+ | +++ + + x x x x x| ||___M___A______| |____________M___A_________________| | +------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 5 47.283031 51.450488 49.063725 49.493986 1.6446236 + 5 44.460431 46.148431 44.636276 44.937538 0.70509763 Difference at 95.0% confidence -4.55645 +/- 1.84536 -9.20606% +/- 3.72846% (Student's t, pooled s = 1.2653) Why is it 9% faster to read the beginning of a 1 TiB file that was crated with truncate(1) than to read the beginning of a 1 TiB file that was created with dd oseek=? Even more disturbing: the newer, faster server is in fact faster when reading from disk: x noholes (9.2) + noholes2 (9.1) +------------------------------------------------------------------------------+ |x x x x * + ++ +| | |_________________________A___M______________________| |__A_| | +------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 5 71.300931 82.89801 77.044237 76.452097 4.3961747 + 5 82.813107 83.970546 83.527358 83.427247 0.43568711 Difference at 95.0% confidence 6.97515 +/- 4.55587 9.12356% +/- 5.95912% (Student's t, pooled s = 3.12379) But the faster (9.2) server is much, much slower when reading from holes: x truncate (9.2) + truncate2 (9.1) +------------------------------------------------------------------------------+ |++ | |++ + x x x x x | ||A| |_____A_M____|| +------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 5 89.187745 101.74893 98.825988 97.497923 4.9188155 + 5 44.460431 46.148431 44.636276 44.937538 0.70509763 Difference at 95.0% confidence -52.5604 +/- 5.1245 -53.9092% +/- 5.25601% (Student's t, pooled s = 3.51368) Repeating the same experiments in a fresh dataset with compression turned on gives the expected results: ZFS automatically punches a big hole in the "no holes" test file, and the read performance is indistinguishable among all three creation methods. Unfortunately, this means that the newer, faster 9.2 server slows all accesses down to the speed of reading holes, whereas the older, slower 9.1 server speeds all accesses up: x noholes+lzjb (9.2) + noholes2+lzjb (9.1) +------------------------------------------------------------------------------+ | + x | |++++ x x xx| ||A_| |_AM|| +------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 5 100.23982 103.88802 102.84441 102.38784 1.4890753 + 5 43.747693 45.784714 44.914809 44.902244 0.82465096 Difference at 95.0% confidence -57.4856 +/- 1.75541 -56.1449% +/- 1.71447% (Student's t, pooled s = 1.20362) The 9.2 machine is unquestionably faster, at least when shoving data around without touching the filesystem ("dd if=/dev/zero of=/dev/null" with other parameters as for the filesystem-based tests): x overhead (9.2) + overhead2 (9.1) +------------------------------------------------------------------------------+ | x + + | | x x x + + +| ||M__A___| |____________A_______M____|| +------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 5 5.971316 6.357959 5.980432 6.0935198 0.17238797 + 5 8.015839 9.168025 8.948119 8.6249242 0.55062308 Difference at 95.0% confidence 2.5314 +/- 0.595023 41.5426% +/- 9.76484% (Student's t, pooled s = 0.407985) What's going on here? -GAWollman From owner-freebsd-fs@FreeBSD.ORG Sat Jan 25 15:54:20 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4A9C71F1 for ; Sat, 25 Jan 2014 15:54:20 +0000 (UTC) Received: from mta05.bitpro.no (mta05.bitpro.no [92.42.64.202]) by mx1.freebsd.org (Postfix) with ESMTP id 911D51CC6 for ; Sat, 25 Jan 2014 15:54:19 +0000 (UTC) Received: from mail.lockless.no (mail.lockless.no [46.29.221.38]) by mta05.bitpro.no (Postfix) with ESMTPS id B6C5017FC76 for ; Sat, 25 Jan 2014 16:54:10 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by mail.lockless.no (Postfix) with ESMTP id 998098FAB6F for ; Sat, 25 Jan 2014 16:55:01 +0100 (CET) X-Virus-Scanned: by amavisd-new-2.6.4 (20090625) (Debian) at lockless.no Received: from mail.lockless.no ([127.0.0.1]) by localhost (mail.lockless.no [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YDWfrOilaFPY for ; Sat, 25 Jan 2014 16:54:59 +0100 (CET) Received: from mail.lockless.no (localhost [127.0.0.1]) by mail.lockless.no (Postfix) with ESMTP id 987648FAB57 for ; Sat, 25 Jan 2014 16:54:59 +0100 (CET) Subject: FW: lock order reversals w/ backtrace From: =?utf-8?Q?Hans_Petter_Selasky?= To: =?utf-8?Q?freebsd-fs=40freebsd=2Eorg?= Date: Sat, 25 Jan 2014 16:54:59 +0100 Mime-Version: 1.0 X-Priority: 3 (Normal) X-Mailer: Zarafa 7.1.4-41394 Message-Id: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Jan 2014 15:54:20 -0000 FYI - can anyone comment=3F=0D=0A=0D=0A--HPS=0D=0A=20=0D=0A=20=0D=0A-----= Original message-----=0D=0A> From:Thomas Hoffmann >=0D=0A> Sent: Saturday 25th January 2014 15:55=0D=0A= > To: freebsd-current >=0D=0A> Cc: Hans Petter Selasky >=0D=0A> Subject: Re: lo= ck order reversals w/ backtrace=0D=0A>=20=0D=0A> On Fri, Jan 24, 2014 at = 11:47 PM, Thomas Hoffmann > w= rote:=0D=0A>=20=0D=0A> > On Fri, Jan 24, 2014 at 9:52 AM, Thomas Hoffmann= > wrote:=0D=0A> >=0D=0A> >> = On Thu, Jan 23, 2014 at 11:49 AM, Hans Petter Selasky <=0D=0A> >> hans.pe= tter.selasky@bitfrost.no > wrote= :=0D=0A> >>=0D=0A> >>> Hi,=0D=0A> >>>=0D=0A> >>> Can you see if you can = snap some keywords of the backtraces, like usb_xxx usbdx_xxx cam scsi or = something like that.=0D=0A> >>>=0D=0A> >>> Else I believe there are some = sysctl options to prevent the final reboot somehow so that you can write = down the messages.=0D=0A> >>>=0D=0A> >>>=0D=0A> >>> --HPS=0D=0A> >>>=0D=0A= > >>>=0D=0A> >>> -----Original message-----=0D=0A> >>> > From:Thomas Hoff= mann >=0D=0A> >>> > Sent: Thu= rsday 23rd January 2014 17:15=0D=0A> >>>=0D=0A> >>> > To: freebsd-current= >=0D=0A= > >>> > Subject: lock order reversals w/ backtrace=0D=0A> >>> >=0D=0A> >>= > > A few days ago I started running 11.0-CURRENT at r260971 for the firs= t time.=0D=0A> >>>=0D=0A> >>>=0D=0A> >>> >=0D=0A> >>> > The last couple o= f times I shutdown my system I noticed 2 or 3 short "lock=0D=0A> >>> > or= der reversal" messages with accompanying backtraces scroll by. Do these=0D= =0A> >>> > messages represent a problem that I should report or can I ign= ore them as=0D=0A> >>>=0D=0A> >>>=0D=0A> >>> > debug in nature=3F If I sh= ould report them, how or where do these messages=0D=0A> >>> > get logged=3F= I can find no reference to them in syslog or anywhere else upon=0D=0A> >= >> > my subsequent reboot.=0D=0A> >>> >=0D=0A> >>> > I also had a couple = of these messages pop up the other day while=0D=0A> >>>=0D=0A> >>>=0D=0A>= >>> > mounting/umounting USB thumb drives. I did not think anything of t= hem at=0D=0A> >>> > the time as the mounts/umounts completed successfully= =2E=0D=0A> >>> >=0D=0A> >>> > Please advise. Thanks.=0D=0A> >>> >=0D=0A> = >>> > -Tom=0D=0A> >>>=0D=0A> >>> I managed to snap a photo of my screen = during shutdown.=0D=0A> >> Here is the full text of the first of two lock= order reversals I got last=0D=0A> >> night during system shutdown, both = of which are zfs-related to (it=0D=0A> >> appears=3F) unmounts. A few lin= es got chopped as they were out-of-frame when=0D=0A> >> I took the photo:= =0D=0A> >>=0D=0A> >> shutting down local daemons:=0D=0A> >> lock order re= versal:=0D=0A> >> 1st 0xfffff801194f67c8 zfs (zfs) @ /usr/src/sys/kern/= vfs_mount.c:1237=0D=0A> >> 2nd 0xfffff801194f6420 syncer (syncer) @=0D=0A= > >> /usr/src/sys/kern/vfs_subr.c:22[..chopped...]=0D=0A> >> KDB: stack b= acktrace:=0D=0A> >> db_trace_self_wrapper() at db_trace_delf_wrapper+0x2b= /frame=0D=0A> >> 0xfffffe01ac78[...chopped...]=0D=0A> >> kdb_backtrace() = at kdb_backtrace+0x39/frame 0xfffffe01ac784650=0D=0A> >> witness_checkord= er() at witness_checkorder+0xd23/frame 0xfffffe01ac7846e0=0D=0A> >> __loc= kmgr_args() __lockmgr_args+0x878/frame 0xfffffe01ac784810=0D=0A> >> vop_s= tdlock() at vop_stdlock+0x3c/frame 0xfffffe01ac784830=0D=0A> >> VOP_LOCK1= _APV() at VOP_LOCK1_APV+0xf5/frame 0xfffffe01ac784860=0D=0A> >> _vn_lock(= ) at _vn_lock+0xab/frame 0xfffffe01ac7848d0=0D=0A> >> vputx() at vputx+0x= 240/frame 0xfffffe01ac784930=0D=0A> >> dounmount at dounmount+0x327/frame= 0xfffffe01ac7849b0=0D=0A> >> sys_unmount() at sys_unmount+0x356/frame 0x= fffffe01ac784ac0=0D=0A> >> amd64_syscall() at amd64_syscall+0x265/frame 0= xfffffe01ac784bf0=0D=0A> >> Xfast_syscall() at Xfast_syscall+0xfb/frame 0= xfffffe01ac784bf0=0D=0A> >> --- syscall (22, FreeBSD ELF64, sys_unmount, = rip =3D 0x80191c72a,=0D=0A> >> rsp[...chopped...]=0D=0A> >>=0D=0A> >> I h= ave a zpool on an external USB HDD that I export at shutdown via=0D=0A> >= > rc.shutdown.local. Don't know if that is contributing to this or not. W= hen=0D=0A> >> I get a chance to bring down the system I will manually exp= ort it before=0D=0A> >> shutdown to see if that makes any difference.=0D=0A= > >>=0D=0A> >=0D=0A> > Was able to capture the full text of the lock orde= r reversal I've been=0D=0A> > experiencing at shutdown. Turns out to be a= ssociated with the export of one=0D=0A> > of my zpools that is hosted on = an external USB HDD. Normally I export the=0D=0A> > zpool from rc.shutdow= n.local, but tonight I exported it manually before I=0D=0A> > shutdown an= d got the following.=0D=0A> >=0D=0A> > lock order reversal:=0D=0A> > : 1s= t 0xfffff8011952b5f0 zfs (zfs) @ /usr/src/sys/kern/vfs_mount.c:1237=0D=0A= > > : 2nd 0xfffff8011952b068 syncer (syncer) @=0D=0A> > /usr/src/sys/kern= /vfs_subr.c:2212=0D=0A> > KDB: stack backtrace:=0D=0A> > db_trace_self_wr= apper() at db_trace_self_wrapper+0x2b/frame=0D=0A> > 0xfffffe01add6e5a0=0D= =0A> > kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe01add6e650=0D=0A= > > witness_checkorder() at witness_checkorder+0xd23/frame 0xfffffe01add6= e6e0=0D=0A> > __lockmgr_args() at __lockmgr_args+0x878/frame 0xfffffe01ad= d6e810=0D=0A> > vop_stdlock() at vop_stdlock+0x3c/frame 0xfffffe01add6e83= 0=0D=0A> > VOP_LOCK1_APV() at VOP_LOCK1_APV+0xf5/frame 0xfffffe01add6e860= =0D=0A> > _vn_lock() at _vn_lock+0xab/frame 0xfffffe01add6e8d0=0D=0A> > v= putx() at vputx+0x240/frame 0xfffffe01add6e930=0D=0A> > dounmount() at do= unmount+0x327/frame 0xfffffe01add6e9b0=0D=0A> > sys_unmount() at sys_unmo= unt+0x356/frame 0xfffffe01add6eae0=0D=0A> > amd64_syscall() at amd64_sysc= all+0x265/frame 0xfffffe01add6ebf0=0D=0A> > Xfast_syscall() at Xfast_sysc= all+0xfb/frame 0xfffffe01add6ebf0=0D=0A> > --- syscall (22, FreeBSD ELF64= , sys_unmount), rip =3D 0x80191c72a, rsp =3D=0D=0A> > 0x7fffffffc4c8, rbp= =3D 0x7fffffffc960 ---=0D=0A> >=0D=0A> > When imported and mounted, the = zpool reports no errors and I have not=0D=0A> > experienced any anomalies= reading or writing to the zpool. Should I be=0D=0A> > concerned=3F=0D=0A= > >=0D=0A> > Hmm, seems like I'm the only one interested in this.=0D=0A>=20= =0D=0A> I think I have resolved the problem with the zpool on the externa= l USB HDD.=0D=0A> I copied off the data, destroyed the datasets and zpool= , recreated the=0D=0A> zpool and datasets and restored the data. I have e= xecuted several exports=0D=0A> on the zpool and have received no "lock or= der reversal" messages.=0D=0A>=20=0D=0A> I am still getting lock order re= versal messages for my bootpool and zroot=0D=0A> zpools at shutdown. Recr= eating bootpool would be relatively easy, but if=0D=0A> I'm going to recr= eate zroot, I might as well do a reinistall, am I am not=0D=0A> prepared = to do at this time. So I'm still in search of an alternative way=0D=0A> t= o resolve this issue. My zpools all show "no known data errors", scrub=0D= =0A> cleanly, zpool upgrade reports that all zpools support all features,= and=0D=0A> I've not experienced any other issues related to my zpools.=0D= =0A>=20=0D=0A> Based on the above lock order reversal data, the error is = in this block of=0D=0A> code (, specifically /usr/src/sys/kern/vfs_mount.= c:1237)=0D=0A>=20=0D=0A> if ((coveredvp =3D mp->mnt_vnodecovered) !=3D N= ULL) {=0D=0A> mnt_gen_r =3D mp->mnt_gen;=0D=0A> = VI_LOCK(coveredvp);=0D=0A> vholdl(coveredvp);=0D=0A= > vn_lock(coveredvp, LK_EXCLUSIVE | LK_INTERLOCK | LK_RET= RY);=0D=0A> /* LINE 1237 */=0D=0A> vdrop(coveredvp);=0D= =0A> /*=0D=0A> * Check for mp being unmo= unted while waiting for the=0D=0A> * covered vnode lock.= =0D=0A> */=0D=0A> if (coveredvp->v_mount= edhere !=3D mp ||=0D=0A> coveredvp->v_mountedhere->mn= t_gen !=3D mnt_gen_r) {=0D=0A> VOP_UNLOCK(covered= vp, 0);=0D=0A> return (EBUSY);=0D=0A> = }=0D=0A> }=0D=0A>=20=0D=0A>=20=0D=0A> -Tom=0D=0A> __________________= _____________________________=0D=0A> freebsd-current@freebsd.org mailing list=0D=0A> http://lists.freebsd.or= g/mailman/listinfo/freebsd-current =20=0D=0A> To unsubscribe, send any mail to "freebsd= -current-unsubscribe@freebsd.org "=0D=0A>=20=0D=0A=0D=0A From owner-freebsd-fs@FreeBSD.ORG Sat Jan 25 16:28:19 2014 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 68EDA7F4; Sat, 25 Jan 2014 16:28:19 +0000 (UTC) Received: from tensor.andric.com (unknown [IPv6:2001:7b8:3a7:1:2d0:b7ff:fea0:8c26]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 131141E7F; Sat, 25 Jan 2014 16:28:19 +0000 (UTC) Received: from [IPv6:2001:7b8:3a7::719e:bbe8:ba8a:34e0] (unknown [IPv6:2001:7b8:3a7:0:719e:bbe8:ba8a:34e0]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id E54C85C44; Sat, 25 Jan 2014 17:28:12 +0100 (CET) Subject: Re: BUG: possible NULL pointer dereference in nfs server Mime-Version: 1.0 (Mac OS X Mail 7.1 \(1827\)) Content-Type: multipart/signed; boundary="Apple-Mail=_70801F41-810B-4C7F-9BA3-A9451B541D04"; protocol="application/pgp-signature"; micalg=pgp-sha1 X-Pgp-Agent: GPGMail 2.1 (6062eb4) From: Dimitry Andric In-Reply-To: <1577222508.16050114.1390610315654.JavaMail.root@uoguelph.ca> Date: Sat, 25 Jan 2014 17:27:58 +0100 Message-Id: <97D8A7EA-12F3-4EFC-A933-BD8528B6306A@FreeBSD.org> References: <1577222508.16050114.1390610315654.JavaMail.root@uoguelph.ca> To: Rick Macklem X-Mailer: Apple Mail (2.1827) Cc: Roman Divacky , fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Jan 2014 16:28:19 -0000 --Apple-Mail=_70801F41-810B-4C7F-9BA3-A9451B541D04 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii On 25 Jan 2014, at 01:38, Rick Macklem wrote: > Roman Divacky wrote: >> Hi, >>=20 >> In nfs_nfsdstate.c:nfsrv_lockctrl() we call >>=20 >> getlckret =3D nfsrv_getlockfh(vp, new_stp->ls_flags, NULL, &nfh, p); >>=20 >> then in nfsrv_getlockfh() we, based on the value of flags, might >> dereference the NULL pointer: >>=20 >>=20 >> nfsrv_getlockfh(vnode_t vp, u_short flags, >> struct nfslockfile **new_lfpp, fhandle_t *nfhp, NFSPROC_T *p) >>=20 >>=20 >> if (flags & NFSLCK_OPEN) { >> new_lfp =3D *new_lfpp; >> fhp =3D &new_lfp->lf_fh; >>=20 >>=20 > I took a look and don't see a problem. NFSLCK_OPEN is only set for = Opens > { nfsrvd_open() } and it never calls nfsrv_lockctrl(). = nfsrv_lockctrl() is > always called with other flags in new_stp->ls_flags set, but never = NFSLCK_OPEN, > because nfsrv_lockctrl() is handling byte range lock cases after a = file has been > opened { which means the "else" case for this "if" always applies. The point is that the compiler cannot always know this, when it inlines nfsrv_getlockfh(). In the first invocation of nfsrv_getlockfh(), there is no problem: 712 APPLESTATIC void 713 nfsrv_dumplocks(vnode_t vp, struct nfsd_dumplocks *ldumpp, int = maxcnt, 714 NFSPROC_T *p) 715 { ... 726 ret =3D nfsrv_getlockfh(vp, 0, NULL, &nfh, p); Here both the 'flags' and 'new_lfpp' arguments are zero, so the part in nfsrv_getlockfh() where it writes to a NULL pointer can never be executed, therefore it can be optimized away safely. In the next invocation, the state of the 'flags' argument is not known to the compiler, however: 1369 tryagain: 1370 if (new_stp->ls_flags & NFSLCK_LOCK) 1371 MALLOC(other_lop, struct nfslock *, sizeof = (struct nfslock), 1372 M_NFSDLOCK, M_WAITOK); ... 1387 getlckret =3D nfsrv_getlockfh(vp, new_stp->ls_flags, = NULL, &nfh, p); If it inlines this, the result looks approximately like: 1 { 2 fhandle_t *fhp =3D NULL; 3 struct nfslockfile *new_lfp; 4 int error; 5 6 if (new_stp->ls_flags & NFSLCK_OPEN) { 7 new_lfp =3D *NULL; 8 fhp =3D &new_lfp->lf_fh; 9 } else if (&nfh) { 10 fhp =3D &nfh; 11 } else { 12 panic("nfsrv_getlockfh"); 13 } 14 error =3D nfsvno_getfh(vp, fhp, p); 15 NFSEXITCODE(error); 16 getlckret =3D error; 17 } The code in line 7 is the problematic part. Since this is undefined, the compiler inserts a trap instruction here. I think the problem Roman encountered is that on sparc64, there is no equivalent to x86's ud2 instruction, so it inserts a call to abort() instead, and that function is not available in kernel-land. ... > Sorry, I'm not a compiler guy, so I don't know why a compiler would > generate a trap instruction, but since new_lfpp is never NULL when > this is executed, I don't see a problem. >=20 > If others feel that this needs to be re-coded, please let me know what > you think the code should look like? (A test for non-NULL with a = panic() > before it is used?) >=20 > Is a trap instruction that never gets executed a problem? It's better to avoid undefined behavior in any case. Just add a NULL check, that should be sufficient. -Dimitry --Apple-Mail=_70801F41-810B-4C7F-9BA3-A9451B541D04 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.22 (Darwin) iEYEARECAAYFAlLj5hkACgkQsF6jCi4glqO2hACeKUFQagxvdOrYIgc3YqdmERwq VbIAnR5Td7N+Dqjj4BAiAN8nVTmqkW2A =9r0K -----END PGP SIGNATURE----- --Apple-Mail=_70801F41-810B-4C7F-9BA3-A9451B541D04-- From owner-freebsd-fs@FreeBSD.ORG Sat Jan 25 18:29:45 2014 Return-Path: Delivered-To: fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5158FB3D; Sat, 25 Jan 2014 18:29:45 +0000 (UTC) Received: from mail110.syd.optusnet.com.au (mail110.syd.optusnet.com.au [211.29.132.97]) by mx1.freebsd.org (Postfix) with ESMTP id 13719171D; Sat, 25 Jan 2014 18:29:44 +0000 (UTC) Received: from c122-106-144-87.carlnfd1.nsw.optusnet.com.au (c122-106-144-87.carlnfd1.nsw.optusnet.com.au [122.106.144.87]) by mail110.syd.optusnet.com.au (Postfix) with ESMTPS id 8EA977826D9; Sun, 26 Jan 2014 05:29:41 +1100 (EST) Date: Sun, 26 Jan 2014 05:29:40 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Dimitry Andric Subject: Re: BUG: possible NULL pointer dereference in nfs server In-Reply-To: <97D8A7EA-12F3-4EFC-A933-BD8528B6306A@FreeBSD.org> Message-ID: <20140126051258.M1750@besplex.bde.org> References: <1577222508.16050114.1390610315654.JavaMail.root@uoguelph.ca> <97D8A7EA-12F3-4EFC-A933-BD8528B6306A@FreeBSD.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=HZAtEE08 c=1 sm=1 tr=0 a=p/w0leo876FR0WNmYI1KeA==:117 a=PO7r1zJSAAAA:8 a=vWCuAoM2T_8A:10 a=kj9zAlcOel0A:10 a=JzwRw_2MAAAA:8 a=vwpRgUSxFbAA:10 a=8RJonftlxXZiz3JrQMUA:9 a=CjuIK1q_8ugA:10 Cc: Roman Divacky , fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Jan 2014 18:29:45 -0000 On Sat, 25 Jan 2014, Dimitry Andric wrote: > On 25 Jan 2014, at 01:38, Rick Macklem wrote: > ... > If it inlines this, the result looks approximately like: > > 1 { > 2 fhandle_t *fhp = NULL; > 3 struct nfslockfile *new_lfp; > 4 int error; > 5 > 6 if (new_stp->ls_flags & NFSLCK_OPEN) { > 7 new_lfp = *NULL; > 8 fhp = &new_lfp->lf_fh; > 9 } else if (&nfh) { > 10 fhp = &nfh; > 11 } else { > 12 panic("nfsrv_getlockfh"); > 13 } > 14 error = nfsvno_getfh(vp, fhp, p); > 15 NFSEXITCODE(error); > 16 getlckret = error; > 17 } > > The code in line 7 is the problematic part. Since this is undefined, > the compiler inserts a trap instruction here. I think the problem Roman > encountered is that on sparc64, there is no equivalent to x86's ud2 > instruction, so it inserts a call to abort() instead, and that function > is not available in kernel-land. Compiler bug. abort() is not available in freestanding implementations. The behaviour is only undefined if the null pointer is dereferenced at runtime, so it doesn't include failing to link to abort() at compile time. > ... >> Sorry, I'm not a compiler guy, so I don't know why a compiler would >> generate a trap instruction, but since new_lfpp is never NULL when >> this is executed, I don't see a problem. >> >> If others feel that this needs to be re-coded, please let me know what >> you think the code should look like? (A test for non-NULL with a panic() >> before it is used?) >> >> Is a trap instruction that never gets executed a problem? > > It's better to avoid undefined behavior in any case. Just add a NULL > check, that should be sufficient. That might only add bloat and unimprove debugging. Since the null pointer case cannot happen, it cannot be handed properly. It can be mishandled in the following ways: - return an error, so that all callers have to handle the null pointer case that can't happen. If the compiler is too smart, it will notice more undefined behaviour (that can't happen) in callers and "force" you to handle it there too - KASSERT() that the pointer cannot be null. Then: - on production systems where KASSERT() is null, this won't work around the compiler bug. Use a panic() instead. To maximize source code bloat, ifdef all of this. - when KASSERT() is not null, it will work around the compiler bug. If the case that can't happen actually happens, then this unimproves the debugging by messing up stack traces and turning restartable null pointer or SIGILL traps to non-restartable panics. Optimizations that replace a large block of code ending with a null pointer trap by a single unimplemented instruction would probably break restarting anyway. Bruce From owner-freebsd-fs@FreeBSD.ORG Sat Jan 25 19:43:29 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4CB70AFD for ; Sat, 25 Jan 2014 19:43:29 +0000 (UTC) Received: from mail-pd0-x233.google.com (mail-pd0-x233.google.com [IPv6:2607:f8b0:400e:c02::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 197E31DE2 for ; Sat, 25 Jan 2014 19:43:29 +0000 (UTC) Received: by mail-pd0-f179.google.com with SMTP id q10so4299099pdj.24 for ; Sat, 25 Jan 2014 11:43:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=MU98LGyRnnOHUg5SGOxaVgAJ7ksDXUs8JIDpFQv5ukU=; b=Y1+5aL1L2QwAlUQe224xfMVLKeU16/YQxuQQ6gQJTqIQiCnFEUaGDL7KABd9negAyW eenUeIiiqJb/fnB0MA0xHPolMsDQlBp4XIN482jVOtCoi4BfA5x+fWWSOc+UkCevXKyq 8nbD3dZev3yDcBkc/PZX2kLZ/yItYN6a2efos= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=MU98LGyRnnOHUg5SGOxaVgAJ7ksDXUs8JIDpFQv5ukU=; b=dDHOcFYpCQuUjvwoFczOhxmPxtLkVPGYS9Xf0mscfDyvxjTVCYgdiRrKRVC/NIn9eK d3PXB6TpDXQVHTpJ68UrdkmzAQlX7l2USWc0PGW/9aTJWF4Q9nRrB2d9+Vc4qAX+VDIa D85NrzIerb06Aana8MJZgcD9JvW7RMTxAl9rdc1dLEqX01nMeCX0+OtNW/YipFcFoW8a DqtnfyLeVvIRXgbJ0RdWa2Arf4U6gyyOGx5vIFBA/7SlZRGIvMFc8RiNw4fpxyAWo5x/ LU2phGZ4DeEgFjwdd+NSc+lEkktiLfabq0OIHlp+8i5sFWcXXSmvVHwWuw2/MYaVSEps 9h3g== X-Gm-Message-State: ALoCoQlCbeikIs8mYbqVBQNzVJK8QGRe1VL3U15mvrgm+6UgAyIUxen547n75WuR4qbGtuFy8WNO MIME-Version: 1.0 X-Received: by 10.68.198.97 with SMTP id jb1mr21372280pbc.104.1390679008689; Sat, 25 Jan 2014 11:43:28 -0800 (PST) Received: by 10.70.44.9 with HTTP; Sat, 25 Jan 2014 11:43:28 -0800 (PST) In-Reply-To: <21219.21690.165105.561362@hergotha.csail.mit.edu> References: <21219.21690.165105.561362@hergotha.csail.mit.edu> Date: Sat, 25 Jan 2014 11:43:28 -0800 Message-ID: Subject: Re: ZFS read performance anomaly / performance regression From: Matthew Ahrens To: Garrett Wollman Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: freebsd-fs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Jan 2014 19:43:29 -0000 How are you reading the files? Could you be reading in less than block-size chunks (default 128K)? This sounds similar to issue discussed in the thread with subject "ZFS slow reads for unallocated blocks", which was comparing sparse vs cached reads. Since your storage is so fast, prefetch will probably read and cache the blocks by the time you issue the read() syscall, so you may be hitting a similar issue. To quote my conclusion from the previous thread: Yes, this is also a problem on illumos, though much less so on my system, only about 2x. It looks like the difference is due to the fact that the zeroed dbufs are not cached, so we have to zero the entire dbuf (e.g. 128k) for every read syscall (e.g. 8k). Increasing the size of the reads to match the recordsize results in performance parity between reading cached data and sparse zeros. --matt On Fri, Jan 24, 2014 at 10:07 PM, Garrett Wollman wrote: > Consider three 1-TiB files, equal in dignity, all on the same ZFS > dataset: > > -rw-r--r-- 1 root wheel 1099511627776 Jan 7 19:03 test > -rw-r--r-- 1 root wheel 1099511627776 Jan 24 22:38 test-truncate > -rw-r--r-- 1 root wheel 1099511628800 Jan 24 21:32 test2 > > File "test" was created by writing zeros; "test2" was created by > seeking to 1 TiB and writing a block of zeros, and "test-truncate" was > created with "truncate -s 1T". Compression is disabled on this > dataset, and "uname -a" reports: > > FreeBSD nfs-prod-3.csail.mit.edu 9.2-RELEASE-p2 FreeBSD 9.2-RELEASE-p2 #2 > r260414M: Tue Jan 7 17:32:43 EST 2014 wollman@xyz.csail.mit.edu:/usr/obj/usr/src/sys/CSAIL > amd64 > > A naive observer would expect that reading "test" should be > significantly slower than reading either of the other files, since it > requires reading data from the disk, and that there should be no > significant difference between the other two, which consist entirely > of cacheable metadata and have no, or very little, data stored on > disk. > > But no: > > x noholes > + oseek > * truncate > > +------------------------------------------------------------------------------+ > |x x x x * *+ * * * + + > + + | > | |________AM______| > |________A__M___|__|_______A__M______|| > > +------------------------------------------------------------------------------+ > N Min Max Median Avg Stddev > x 5 71.300931 82.89801 77.044237 76.452097 4.3961747 > + 5 97.931676 111.36284 108.52787 106.5687 5.5595081 > Difference at 95.0% confidence > 30.1166 +/- 7.30929 > 39.3928% +/- 9.56062% > (Student's t, pooled s = 5.01171) > * 5 89.187745 101.74893 98.825988 97.497923 4.9188155 > Difference at 95.0% confidence > 21.0458 +/- 6.80337 > 27.5281% +/- 8.89887% > (Student's t, pooled s = 4.66482) > > (Values are the time to read 128 GiB of each file, which is more than > the available memory for caching on this server. Measurements were > interleaved abcabcabcabcabc, but server was not rebooted nor > completely idle. Lower numbers are better.) > > Now consider the same test on an older, slower (but also completely > idle) server, running an older FreeBSD release, but with an identical > zpool configuration (88 spindles plus log and cache SSDs): > > FreeBSD nfs-backup-1.csail.mit.edu 9.1-RELEASE-p7 FreeBSD 9.1-RELEASE-p7 > #34 r255669M: Fri Oct 11 13:04:40 EDT 2013 wollman@xyz.csail.mit.edu:/usr/obj/usr/src/sys/CSAIL > amd64 > > Files were created in the same manner in a fresh, unsnapshotted > dataset on this server, and the same shell script was used to time > reads (even though the older machine has significantly less memory): > > x noholes2 (9.1) > + oseek2 (9.1) > * truncate2 (9.1) > > +------------------------------------------------------------------------------+ > |** + > x | > |** * + + ++ > xxx| > ||A_| |_MA__| > |A|| > > +------------------------------------------------------------------------------+ > N Min Max Median Avg Stddev > x 5 82.813107 83.970546 83.527358 83.427247 0.43568711 > + 5 47.283031 51.450488 49.063725 49.493986 1.6446236 > Difference at 95.0% confidence > -33.9333 +/- 1.75456 > -40.6741% +/- 2.10311% > (Student's t, pooled s = 1.20304) > * 5 44.460431 46.148431 44.636276 44.937538 0.70509763 > Difference at 95.0% confidence > -38.4897 +/- 0.854768 > -46.1357% +/- 1.02457% > (Student's t, pooled s = 0.586083) > > This is more like what I was expecting! But there's still this > anomaly: > > x oseek2 (9.1) > + truncate2 (9.1) > > +------------------------------------------------------------------------------+ > | +++ + + x x x x > x| > ||___M___A______| > |____________M___A_________________| | > > +------------------------------------------------------------------------------+ > N Min Max Median Avg Stddev > x 5 47.283031 51.450488 49.063725 49.493986 1.6446236 > + 5 44.460431 46.148431 44.636276 44.937538 0.70509763 > Difference at 95.0% confidence > -4.55645 +/- 1.84536 > -9.20606% +/- 3.72846% > (Student's t, pooled s = 1.2653) > > Why is it 9% faster to read the beginning of a 1 TiB file that was > crated with truncate(1) than to read the beginning of a 1 TiB file > that was created with dd oseek=? > > Even more disturbing: the newer, faster server is in fact faster when > reading from disk: > > x noholes (9.2) > + noholes2 (9.1) > > +------------------------------------------------------------------------------+ > |x x x x * + > ++ +| > | |_________________________A___M______________________| > |__A_| | > > +------------------------------------------------------------------------------+ > N Min Max Median Avg Stddev > x 5 71.300931 82.89801 77.044237 76.452097 4.3961747 > + 5 82.813107 83.970546 83.527358 83.427247 0.43568711 > Difference at 95.0% confidence > 6.97515 +/- 4.55587 > 9.12356% +/- 5.95912% > (Student's t, pooled s = 3.12379) > > But the faster (9.2) server is much, much slower when reading from > holes: > > x truncate (9.2) > + truncate2 (9.1) > > +------------------------------------------------------------------------------+ > |++ > | > |++ + x x x > x x | > ||A| > |_____A_M____|| > > +------------------------------------------------------------------------------+ > N Min Max Median Avg Stddev > x 5 89.187745 101.74893 98.825988 97.497923 4.9188155 > + 5 44.460431 46.148431 44.636276 44.937538 0.70509763 > Difference at 95.0% confidence > -52.5604 +/- 5.1245 > -53.9092% +/- 5.25601% > (Student's t, pooled s = 3.51368) > > Repeating the same experiments in a fresh dataset with compression > turned on gives the expected results: ZFS automatically punches a big > hole in the "no holes" test file, and the read performance is > indistinguishable among all three creation methods. Unfortunately, > this means that the newer, faster 9.2 server slows all accesses down > to the speed of reading holes, whereas the older, slower 9.1 server > speeds all accesses up: > > x noholes+lzjb (9.2) > + noholes2+lzjb (9.1) > > +------------------------------------------------------------------------------+ > | + > x | > |++++ x > x xx| > ||A_| > |_AM|| > > +------------------------------------------------------------------------------+ > N Min Max Median Avg Stddev > x 5 100.23982 103.88802 102.84441 102.38784 1.4890753 > + 5 43.747693 45.784714 44.914809 44.902244 0.82465096 > Difference at 95.0% confidence > -57.4856 +/- 1.75541 > -56.1449% +/- 1.71447% > (Student's t, pooled s = 1.20362) > > The 9.2 machine is unquestionably faster, at least when shoving data > around without touching the filesystem ("dd if=/dev/zero of=/dev/null" > with other parameters as for the filesystem-based tests): > > x overhead (9.2) > + overhead2 (9.1) > > +------------------------------------------------------------------------------+ > | x + + > | > | x x x + + > +| > ||M__A___| > |____________A_______M____|| > > +------------------------------------------------------------------------------+ > N Min Max Median Avg Stddev > x 5 5.971316 6.357959 5.980432 6.0935198 0.17238797 > + 5 8.015839 9.168025 8.948119 8.6249242 0.55062308 > Difference at 95.0% confidence > 2.5314 +/- 0.595023 > 41.5426% +/- 9.76484% > (Student's t, pooled s = 0.407985) > > What's going on here? > > -GAWollman > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@FreeBSD.ORG Sat Jan 25 21:30:16 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D47CD830; Sat, 25 Jan 2014 21:30:16 +0000 (UTC) Received: from thebighonker.lerctr.org (lrosenman-1-pt.tunnel.tserv8.dal1.ipv6.he.net [IPv6:2001:470:1f0e:3ad::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 90F731860; Sat, 25 Jan 2014 21:30:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Content-Type:MIME-Version:Message-ID:Subject:To:From:Date; bh=V42EvLgsMe4aOdUy0tlK9Z2M2bcPVis+0zzxtk8HXYw=; b=Y6jt7+hMhbufRbTRd+Tfl8XN5hk4cK/T9mdDJTv7pHaIIbxNC8zdqynsPDcWKa6nuaayunQ/cTW4N4kKF/2cx9RNlm8pbtMjrZb6Sx4mj42Rp0FLscnjkYGfNLwED9LVyGxN4NehhAFQFWsL368D3kYmDIXLJbaa9k0rfbLyhmc=; Received: from 107-128-180-255.lightspeed.austtx.sbcglobal.net ([107.128.180.255]:21935 helo=borg.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.82 (FreeBSD)) (envelope-from ) id 1W7Anm-0009fI-FE; Sat, 25 Jan 2014 15:30:12 -0600 Date: Sat, 25 Jan 2014 15:29:55 -0600 From: Larry Rosenman To: freebsd-current@freebsd.org, freebsd-fs@freebsd.org, freebsd-zfs@freebsd.org Subject: Panic / Solaris Assert / ZAP.c:479 Message-ID: <20140125212954.GA1645@borg.lerctr.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.22 (2013-10-16) X-Spam-Score: 2.6 (++) X-LERCTR-Spam-Score: 2.6 (++) X-Spam-Report: SpamScore (2.6/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, KAM_STOCKTIP=5.5, TVD_RCVD_IP=0.001 X-LERCTR-Spam-Report: SpamScore (2.6/5.0) ALL_TRUSTED=-1, BAYES_00=-1.9, KAM_STOCKTIP=5.5, TVD_RCVD_IP=0.001 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Jan 2014 21:30:16 -0000 Ok, I just did a full rebuild at -CURRENT -HEAD and still get this (basic) panic. How can I get some help to fix it? This version stays up: FreeBSD borg.lerctr.org 11.0-CURRENT FreeBSD 11.0-CURRENT #113 r260864: Fri Jan 17 19:10:06 CST 2014 root@:/usr/obj/usr/src/sys/BORG-DTRACE amd64 Below is the latest panic. borg.lerctr.org dumped core - see /var/crash/vmcore.2 Sat Jan 25 15:25:30 CST 2014 FreeBSD borg.lerctr.org 11.0-CURRENT FreeBSD 11.0-CURRENT #3 r261166: Sat Jan 25 14:55:24 CST 2014 root@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER amd64 panic: solaris assert: l->l_phys->l_hdr.lh_pad1 == 0 (0xffff000000000000 == 0x0), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c, line: 479 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: Copyright (c) 1992-2014 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.0-CURRENT #3 r261166: Sat Jan 25 14:55:24 CST 2014 root@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER amd64 FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 info: [drm] Initialized drm 1.1.0 20060810 CPU: Intel(R) Xeon(R) CPU E5410 @ 2.33GHz (2327.55-MHz K8-class CPU) Origin="GenuineIntel" Id=0x10676 Family=0x6 Model=0x17 Stepping=6 Features=0xbfebfbff Features2=0xce3bd AMD Features=0x20100800 AMD Features2=0x1 TSC: P-state invariant, performance statistics real memory = 68719476736 (65536 MB) avail memory = 65641644032 (62600 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs FreeBSD/SMP: 2 package(s) x 4 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP): APIC ID: 3 cpu4 (AP): APIC ID: 4 cpu5 (AP): APIC ID: 5 cpu6 (AP): APIC ID: 6 cpu7 (AP): APIC ID: 7 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard random: initialized kbd1 at kbdmux0 cryptosoft0: on motherboard acpi0: on motherboard acpi0: Power Button (fixed) unknown: I/O range not supported cpu0: on acpi0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 cpu4: on acpi0 cpu5: on acpi0 cpu6: on acpi0 cpu7: on acpi0 hpet0: iomem 0xfed00000-0xfed003ff irq 0,8 on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 atrtc0: port 0x70-0x71 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: at device 2.0 on pci0 pci1: on pcib1 pcib2: irq 16 at device 0.0 on pci1 pci2: on pcib2 pcib3: irq 16 at device 0.0 on pci2 pci3: on pcib3 pcib4: at device 0.0 on pci3 pci4: on pcib4 pcib5: at device 0.2 on pci3 pci5: on pcib5 pcib6: irq 18 at device 2.0 on pci2 pci6: on pcib6 em0: port 0x2000-0x201f mem 0xd9220000-0xd923ffff,0xd9200000-0xd921ffff irq 18 at device 0.0 on pci6 em0: Using an MSI interrupt em0: Ethernet address: 00:30:48:f2:29:9c em1: port 0x2020-0x203f mem 0xd9260000-0xd927ffff,0xd9240000-0xd925ffff irq 19 at device 0.1 on pci6 em1: Using an MSI interrupt em1: Ethernet address: 00:30:48:f2:29:9d pcib7: at device 0.3 on pci1 pci7: on pcib7 pcib8: at device 4.0 on pci0 pci8: on pcib8 vgapci0: port 0x3000-0x307f mem 0xd8000000-0xd8ffffff,0xc0000000-0xc7ffffff,0xc8000000-0xc9ffffff irq 16 at device 0.0 on pci8 nvidia0: on vgapci0 vgapci0: child nvidia0 requested pci_enable_io vgapci0: child nvidia0 requested pci_enable_io hdac0: mem 0xd9000000-0xd9003fff irq 17 at device 0.1 on pci8 pcib9: at device 6.0 on pci0 pci9: on pcib9 pci0: at device 8.0 (no driver attached) pcib10: irq 17 at device 28.0 on pci0 pci10: on pcib10 pcib11: irq 16 at device 0.0 on pci10 pci11: on pcib11 pcm0: port 0x4080-0x409f,0x4000-0x407f irq 16 at device 0.0 on pci11 pcm0: system configuration SubVendorID: 0x1412, SubDeviceID: 0x2403 XIN2 Clock Source: 24.576MHz(96kHz*256) MPU-401 UART(s) #: not implemented ADC #: 1 and SPDIF receiver connected DAC #: 4 Multi-track converter type: AC'97(SDATA_OUT:packed) S/PDIF(IN/OUT): 1/1 ID# 0x00 GPIO(mask/dir/state): 0xff/0xff/0xff uhci0: port 0x1800-0x181f irq 17 at device 29.0 on pci0 usbus0 on uhci0 uhci1: port 0x1820-0x183f irq 19 at device 29.1 on pci0 usbus1 on uhci1 uhci2: port 0x1840-0x185f irq 18 at device 29.2 on pci0 usbus2 on uhci2 ehci0: mem 0xd9600400-0xd96007ff irq 17 at device 29.7 on pci0 usbus3: EHCI version 1.0 usbus3 on ehci0 pcib12: at device 30.0 on pci0 pci12: on pcib12 vgapci1: port 0x5000-0x50ff mem 0xd0000000-0xd7ffffff,0xd9300000-0xd930ffff irq 18 at device 1.0 on pci12 drmn1: on vgapci1 info: [drm] RADEON_IS_PCI info: [drm] initializing kernel modesetting (RV100 0x1002:0x515E 0x15D9:0x8080). info: [drm] register mmio base: 0xD9300000 info: [drm] register mmio size: 65536 info: [drm] radeon_atrm_get_bios: ===> Try ATRM... info: [drm] radeon_atrm_get_bios: pci_find_class() found: 0:8:0:0, vendor=10de, device=104a info: [drm] radeon_atrm_get_bios: Get ACPI device handle info: [drm] radeon_acpi_vfct_bios: ===> Try VFCT... info: [drm] radeon_acpi_vfct_bios: Get "VFCT" ACPI table info: [drm] radeon_acpi_vfct_bios: Failed to get "VFCT" table: AE_NOT_FOUND info: [drm] igp_read_bios_from_vram: ===> Try IGP's VRAM... info: [drm] igp_read_bios_from_vram: VRAM base address: 0xd0000000 info: [drm] igp_read_bios_from_vram: Map address: 0xfffff800d0000000 (262144 bytes) info: [drm] igp_read_bios_from_vram: Incorrect BIOS signature: 0x0000 info: [drm] radeon_read_bios: ===> Try PCI Expansion ROM... info: [drm] radeon_read_bios: Map address: 0xfffff800000c0000 (131072 bytes) drmn1: info: VRAM: 128M 0x00000000D0000000 - 0x00000000D7FFFFFF (16M used) drmn1: info: GTT: 512M 0x00000000B0000000 - 0x00000000CFFFFFFF info: [drm] Supports vblank timestamp caching Rev 1 (10.10.2010). info: [drm] Driver supports precise vblank timestamp query. info: [drm] radeon: irq initialized. info: [drm] Detected VRAM RAM=128M, BAR=128M info: [drm] RAM width 64bits SDR [TTM] Zone kernel: Available graphics memory: 33006090 kiB [TTM] Zone dma32: Available graphics memory: 2097152 kiB [TTM] Initializing pool allocator info: [drm] radeon: 16M of VRAM memory ready info: [drm] radeon: 512M of GTT memory ready. info: [drm] GART: num cpu pages 131072, num gpu pages 131072 info: [drm] PCI GART of 512M enabled (table at 0x0000000011200000). drmn1: info: WB disabled drmn1: info: fence driver on ring 0 use gpu addr 0x00000000b0000000 and cpu addr 0x0xfffff8001083c000 info: [drm] Loading R100 Microcode info: [drm] radeon: ring at 0x00000000B0001000 info: [drm] ring test succeeded in 1 usecs info: [drm] ib test succeeded in 0 usecs info: [drm] radeon_device_init: Taking over the fictitious range 0xd0000000-0xd4000000 iicbus0: on iicbb0 addr 0xff iic0: on iicbus0 iicbus1: on iicbb1 addr 0xff iic1: on iicbus1 iicbus2: on iicbb2 addr 0xff iic2: on iicbus2 iicbus3: on iicbb3 addr 0xff iic3: on iicbus3 info: [drm] Radeon Display Connectors info: [drm] Connector 0: info: [drm] VGA-1 info: [drm] DDC: 0x60 0x60 0x60 0x60 0x60 0x60 0x60 0x60 info: [drm] Encoders: info: [drm] CRT1: INTERNAL_DAC1 info: [drm] Connector 1: info: [drm] DVI-I-1 info: [drm] HPD2 info: [drm] DDC: 0x6c 0x6c 0x6c 0x6c 0x6c 0x6c 0x6c 0x6c info: [drm] Encoders: info: [drm] CRT2: INTERNAL_DAC2 info: [drm] DFP2: INTERNAL_DVO1 composite sync not supported composite sync not supported info: [drm] fb mappable at 0xD0040000 info: [drm] vram apper at 0xD0000000 info: [drm] size 2076672 info: [drm] fb depth is 8 info: [drm] pitch is 1920 fbd1 on drmn1 vt_allocate: Replace existing VT driver. info: [drm] Initialized radeon 2.29.0 20080528 vgapci1: Boot video device isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1860-0x186f at device 31.1 on pci0 ata0: at channel 0 on atapci0 ahci0: port 0x18a0-0x18a7,0x1874-0x1877,0x1878-0x187f,0x1870-0x1873,0x1880-0x189f mem 0xd9600800-0xd9600bff irq 19 at device 31.2 on pci0 ahci0: AHCI v1.10 with 6 3Gbps ports, Port Multiplier supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahcich4: at channel 4 on ahci0 ahcich5: at channel 5 on ahci0 ichsmb0: port 0x1100-0x111f irq 19 at device 31.3 on pci0 smbus0: on ichsmb0 acpi_button0: on acpi0 ipmi0: port 0xca2-0xca3 on acpi0 ipmi0: KCS mode found at io 0xca2 on acpi atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fd0: <1440-KB 3.5" drive> on fdc0 drive 0 ppc0: port 0x378-0x37f,0x778-0x77f irq 7 drq 3 on acpi0 ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/9 bytes threshold ppbus0: on ppc0 lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 ichwd0 on isa0 orm0: at iomem 0xc0000-0xcafff on isa0 coretemp0: on cpu0 est0: on cpu0 p4tcc0: on cpu0 coretemp1: on cpu1 est1: on cpu1 p4tcc1: on cpu1 coretemp2: on cpu2 est2: on cpu2 p4tcc2: on cpu2 coretemp3: on cpu3 est3: on cpu3 p4tcc3: on cpu3 coretemp4: on cpu4 est4: on cpu4 p4tcc4: on cpu4 coretemp5: on cpu5 est5: on cpu5 p4tcc5: on cpu5 coretemp6: on cpu6 est6: on cpu6 p4tcc6: on cpu6 coretemp7: on cpu7 est7: on cpu7 p4tcc7: on cpu7 ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Timecounters tick every 1.000 msec vboxdrv: fAsync=0 offMin=0x387 offMax=0x50f hdacc0: at cad 0 on hdac0 hdaa0: at nid 1 on hdacc0 pcm1: at nid 4 on hdaa0 pcm2: at nid 5 on hdaa0 random: unblocking device. usbus0: 12Mbps Full Speed USB v1.0 usbus1: 12Mbps Full Speed USB v1.0 usbus2: 12Mbps Full Speed USB v1.0 usbus3: 480Mbps High Speed USB v2.0 ugen3.1: at usbus3 uhub0: on usbus3 ugen2.1: at usbus2 uhub1: on usbus2 ugen1.1: at usbus1 uhub2: on usbus1 ugen0.1: at usbus0 uhub3: on usbus0 ipmi0: IPMI device rev. 1, firmware rev. 1.64, version 2.0 ata0: DMA limited to UDMA33, controller found non-ATA66 cable ipmi0: Number of channels 8 ipmi0: Attached watchdog uhub2: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered uhub3: 2 ports with 2 removable, self powered ada0 at ahcich0 bus 0 scbus1 target 0 lun 0 ada0: ATA-8 SATA 3.x device ada0: Serial Number 5YDA1ZL4 ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada0: quirks=0x1<4K> ada0: Previously was known as ad4 ada1 at ahcich1 bus 0 scbus2 target 0 lun 0 ada1: ATA-8 SATA 3.x device ada1: Serial Number 5YD6FPLG ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada1: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada1: quirks=0x1<4K> ada1: Previously was known as ad6 ada2 at ahcich2 bus 0 scbus3 target 0 lun 0 ada2: ATA-8 SATA 3.x device ada2: Serial Number 5YDA3PC5 ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada2: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada2: quirks=0x1<4K> ada2: Previously was known as ad8 ada3 at ahcich3 bus 0 scbus4 target 0 lun 0 ada3: ATA-8 SATA 3.x device ada3: Serial Number 5YD9Y0P4 ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada3: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada3: quirks=0x1<4K> ada3: Previously was known as ad10 ada4 at ahcich4 bus 0 scbus5 target 0 lun 0 ada4: ATA-8 SATA 3.x device ada4: Serial Number 5YDA1R5W ada4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada4: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada4: quirks=0x1<4K> ada4: Previously was known as ad12 ada5 at ahcich5 bus 0 scbus6 target 0 lun 0 ada5: ATA-8 SATA 3.x device ada5: Serial Number 5YD5RBS8 ada5: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada5: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada5: quirks=0x1<4K> ada5: Previously was known as ad14 cd0 at ata0 bus 0 scbus0 target 0 lun 0 cd0: Removable CD-ROM SCSI-0 device cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present Netvsc initializing... done! SMP: AP CPU #7 Launched! SMP: AP CPU #1 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #6 Launched! SMP: AP CPU #4 Launched! SMP: AP CPU #5 Launched! Root mount waiting for: usbus3 uhub0: 6 ports with 6 removable, self powered Root mount waiting for: usbus3 Root mount waiting for: usbus3 ugen3.2: at usbus3 ukbd0: on usbus3 kbd2 at ukbd0 Trying to mount root from zfs:zroot/ROOT/default []... ugen0.2: at usbus0 ugen1.2: at usbus1 <118>Setting hostuuid: 53d19f64-d663-a017-8922-0030488e9ff3. <118>Setting hostid: 0xf53a926e. <118>Entropy harvesting: interrupts ethernet point_to_point swi. <118>Starting file system checks: <118>Mounting local file systems:. panic: solaris assert: l->l_phys->l_hdr.lh_pad1 == 0 (0xffff000000000000 == 0x0), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c, line: 479 cpuid = 7 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe100c55d350 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe100c55d400 vpanic() at vpanic+0x126/frame 0xfffffe100c55d440 panic() at panic+0x43/frame 0xfffffe100c55d4a0 assfail3() at assfail3+0x2f/frame 0xfffffe100c55d4c0 zap_get_leaf_byblk() at zap_get_leaf_byblk+0x2ea/frame 0xfffffe100c55d510 zap_deref_leaf() at zap_deref_leaf+0xd1/frame 0xfffffe100c55d560 fzap_length() at fzap_length+0x31/frame 0xfffffe100c55d5d0 zap_length_uint64() at zap_length_uint64+0xbf/frame 0xfffffe100c55d630 ddt_zap_lookup() at ddt_zap_lookup+0x3d/frame 0xfffffe100c55d790 ddt_lookup() at ddt_lookup+0x17e/frame 0xfffffe100c55d960 zio_ddt_write() at zio_ddt_write+0xf2/frame 0xfffffe100c55dad0 zio_execute() at zio_execute+0x1eb/frame 0xfffffe100c55db30 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe100c55db80 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe100c55dbb0 fork_exit() at fork_exit+0x84/frame 0xfffffe100c55dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe100c55dbf0 --- trap 0, rip = 0, rsp = 0xfffffe100c55dcb0, rbp = 0 --- KDB: enter: panic Uptime: 23s Dumping 2330 out of 64465 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Reading symbols from /boot/kernel/linux.ko.symbols...done. Loaded symbols for /boot/kernel/linux.ko.symbols Reading symbols from /boot/kernel/if_lagg.ko.symbols...done. Loaded symbols for /boot/kernel/if_lagg.ko.symbols Reading symbols from /boot/kernel/snd_envy24ht.ko.symbols...done. Loaded symbols for /boot/kernel/snd_envy24ht.ko.symbols Reading symbols from /boot/kernel/snd_spicds.ko.symbols...done. Loaded symbols for /boot/kernel/snd_spicds.ko.symbols Reading symbols from /boot/kernel/coretemp.ko.symbols...done. Loaded symbols for /boot/kernel/coretemp.ko.symbols Reading symbols from /boot/kernel/ichsmb.ko.symbols...done. Loaded symbols for /boot/kernel/ichsmb.ko.symbols Reading symbols from /boot/kernel/smbus.ko.symbols...done. Loaded symbols for /boot/kernel/smbus.ko.symbols Reading symbols from /boot/kernel/ichwd.ko.symbols...done. Loaded symbols for /boot/kernel/ichwd.ko.symbols Reading symbols from /boot/kernel/cpuctl.ko.symbols...done. Loaded symbols for /boot/kernel/cpuctl.ko.symbols Reading symbols from /boot/kernel/crypto.ko.symbols...done. Loaded symbols for /boot/kernel/crypto.ko.symbols Reading symbols from /boot/kernel/cryptodev.ko.symbols...done. Loaded symbols for /boot/kernel/cryptodev.ko.symbols Reading symbols from /boot/kernel/dtraceall.ko.symbols...done. Loaded symbols for /boot/kernel/dtraceall.ko.symbols Reading symbols from /boot/kernel/profile.ko.symbols...done. Loaded symbols for /boot/kernel/profile.ko.symbols Reading symbols from /boot/kernel/cyclic.ko.symbols...done. Loaded symbols for /boot/kernel/cyclic.ko.symbols Reading symbols from /boot/kernel/dtrace.ko.symbols...done. Loaded symbols for /boot/kernel/dtrace.ko.symbols Reading symbols from /boot/kernel/systrace_freebsd32.ko.symbols...done. Loaded symbols for /boot/kernel/systrace_freebsd32.ko.symbols Reading symbols from /boot/kernel/systrace.ko.symbols...done. Loaded symbols for /boot/kernel/systrace.ko.symbols Reading symbols from /boot/kernel/sdt.ko.symbols...done. Loaded symbols for /boot/kernel/sdt.ko.symbols Reading symbols from /boot/kernel/lockstat.ko.symbols...done. Loaded symbols for /boot/kernel/lockstat.ko.symbols Reading symbols from /boot/kernel/fasttrap.ko.symbols...done. Loaded symbols for /boot/kernel/fasttrap.ko.symbols Reading symbols from /boot/kernel/fbt.ko.symbols...done. Loaded symbols for /boot/kernel/fbt.ko.symbols Reading symbols from /boot/kernel/dtnfscl.ko.symbols...done. Loaded symbols for /boot/kernel/dtnfscl.ko.symbols Reading symbols from /boot/kernel/dtmalloc.ko.symbols...done. Loaded symbols for /boot/kernel/dtmalloc.ko.symbols Reading symbols from /boot/modules/vboxdrv.ko...done. Loaded symbols for /boot/modules/vboxdrv.ko Reading symbols from /boot/modules/nvidia.ko...done. Loaded symbols for /boot/modules/nvidia.ko Reading symbols from /boot/kernel/ipmi.ko.symbols...done. Loaded symbols for /boot/kernel/ipmi.ko.symbols Reading symbols from /boot/kernel/ipmi_linux.ko.symbols...done. Loaded symbols for /boot/kernel/ipmi_linux.ko.symbols Reading symbols from /boot/kernel/radeonkms.ko.symbols...done. Loaded symbols for /boot/kernel/radeonkms.ko.symbols Reading symbols from /boot/kernel/iicbb.ko.symbols...done. Loaded symbols for /boot/kernel/iicbb.ko.symbols Reading symbols from /boot/kernel/iicbus.ko.symbols...done. Loaded symbols for /boot/kernel/iicbus.ko.symbols Reading symbols from /boot/kernel/iic.ko.symbols...done. Loaded symbols for /boot/kernel/iic.ko.symbols Reading symbols from /boot/kernel/drm2.ko.symbols...done. Loaded symbols for /boot/kernel/drm2.ko.symbols Reading symbols from /boot/kernel/radeonkmsfw_R100_cp.ko.symbols...done. Loaded symbols for /boot/kernel/radeonkmsfw_R100_cp.ko.symbols Reading symbols from /boot/kernel/fdescfs.ko.symbols...done. Loaded symbols for /boot/kernel/fdescfs.ko.symbols Reading symbols from /boot/kernel/linprocfs.ko.symbols...done. Loaded symbols for /boot/kernel/linprocfs.ko.symbols #0 doadump (textdump=1) at pcpu.h:219 219 pcpu.h: No such file or directory. in pcpu.h (kgdb) #0 doadump (textdump=1) at pcpu.h:219 #1 0xffffffff809b9c67 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:452 #2 0xffffffff809ba175 in vpanic (fmt=, ap=) at /usr/src/sys/kern/kern_shutdown.c:759 #3 0xffffffff809ba1c3 in panic (fmt=) at /usr/src/sys/kern/kern_shutdown.c:688 #4 0xffffffff80327abf in assfail3 (a=, lv=, op=, rv=, f=, l=) at /usr/src/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c:91 #5 0xffffffff803bac1a in zap_get_leaf_byblk (zap=, blkid=4858, tx=0x0, lt=RW_READER, lp=0xfffffe100c55d5a0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c:479 #6 0xffffffff803b84f1 in zap_deref_leaf (zap=0xfffff800220bf680, h=1610088556361285632, tx=0x0, lt=RW_READER, lp=0xfffffe100c55d5a0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c:585 #7 0xffffffff803b9481 in fzap_length (zn=0xfffff80028e95600, integer_size=0xfffffe100c55d658, num_integers=0xfffffe100c55d650) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c:897 #8 0xffffffff803bf5bf in zap_length_uint64 (os=, zapobj=, key=, key_numints=, integer_size=0xfffffe100c55d658, num_integers=0xfffffe100c55d650) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap_micro.c:934 #9 0xffffffff80352e7d in ddt_zap_lookup (os=0xfffff80020793c00, object=155, dde=0xfffff80028e96600) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/ddt_zap.c:64 #10 0xffffffff8035146e in ddt_lookup (ddt=0xfffffe000d515000, bp=, add=) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/ddt.c:178 #11 0xffffffff803fb242 in zio_ddt_write (zio=0xfffff800288a0730) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:2242 #12 0xffffffff803f893b in zio_execute (zio=0xfffff800288a0730) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1355 #13 0xffffffff80a01970 in taskqueue_run_locked (queue=0xfffff80020127000) at /usr/src/sys/kern/subr_taskqueue.c:355 #14 0xffffffff80a022cb in taskqueue_thread_loop (arg=) at /usr/src/sys/kern/subr_taskqueue.c:576 #15 0xffffffff80989324 in fork_exit ( callout=0xffffffff80a02230 , arg=0xfffff800200fd4e0, frame=0xfffffe100c55dc00) at /usr/src/sys/kern/kern_fork.c:977 #16 0xffffffff80d95b0e in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:605 #17 0x0000000000000000 in ?? () Current language: auto; currently minimal (kgdb) ------------------------------------------------------------------------ ps -axlww UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND 0 0 0 0 -8 0 0 0 - DLs - 0:00.10 [kernel] 0 1 0 0 24 0 9368 0 wait DLs - 0:00.00 [init] 0 2 0 0 -16 0 0 0 ftcl DL - 0:00.00 [ftcleanup] 0 3 0 0 -16 0 0 0 crypto_w DL - 0:00.00 [crypto] 0 4 0 0 -16 0 0 0 crypto_r DL - 0:00.00 [crypto returns] 0 5 0 0 -16 0 0 0 ccb_scan DL - 0:00.00 [cam] 0 6 0 0 -16 0 0 0 - DL - 0:00.00 [fdc0] 0 7 0 0 -8 0 0 0 zio->io_ DL - 0:00.00 [zfskern] 0 8 0 0 -16 0 0 0 waiting_ DL - 0:00.00 [sctp_iterator] 0 9 0 0 -16 0 0 0 ipmireq DL - 0:00.00 [ipmi0: kcs] 0 10 0 0 -16 0 0 0 audit_wo DL - 0:00.00 [audit] 0 11 0 0 155 0 0 0 - RL - 0:00.00 [idle] 0 12 0 0 -84 0 0 0 - WL - 0:00.00 [intr] 0 13 0 0 -8 0 0 0 - DL - 0:00.00 [geom] 0 14 0 0 -16 0 0 0 - DL - 0:00.00 [rand_harvestq] 0 15 0 0 -68 0 0 0 - DL - 0:00.00 [usb] 0 16 0 0 -20 0 0 0 VBoxIS DL - 0:00.00 [TIMER] 0 17 0 0 -16 0 0 0 psleep DL - 0:00.00 [pagedaemon] 0 18 0 0 -16 0 0 0 psleep DL - 0:00.00 [vmdaemon] 0 19 0 0 155 0 0 0 pgzero DL - 0:00.00 [pagezero] 0 20 0 0 -16 0 0 0 psleep DL - 0:00.00 [bufdaemon] 0 21 0 0 16 0 0 0 syncer DL - 0:00.00 [syncer] 0 22 0 0 -16 0 0 0 vlruwt DL - 0:00.00 [vnlru] 0 23 0 0 -16 0 0 0 sdflush DL - 0:00.00 [softdepflush] 0 24 1 0 52 0 16928 0 wait Ds+ - 0:00.01 [sh] 0 72 24 0 52 0 16928 0 wait D+ - 0:00.00 [sh] 0 75 72 0 52 0 40000 0 tx->tx_s D+ - 0:00.00 [zfs] ------------------------------------------------------------------------ vmstat -s 158237 cpu context switches 22164 device interrupts 800 software interrupts 5019 traps 5843 system calls 23 kernel threads created 37 fork() calls 15 vfork() calls 0 rfork() calls 0 swap pager pageins 0 swap pager pages paged in 0 swap pager pageouts 0 swap pager pages paged out 136 vnode pager pageins 911 vnode pager pages paged in 0 vnode pager pageouts 0 vnode pager pages paged out 0 page daemon wakeups 0 pages examined by the page daemon 0 pages reactivated 1771 copy-on-write faults 15 copy-on-write optimized faults 2232 zero fill pages zeroed 0 zero fill pages prezeroed 0 intransit blocking page faults 4649 total VM faults taken 113 page faults requiring I/O 0 pages affected by kernel thread creation 1334 pages affected by fork() 525 pages affected by vfork() 0 pages affected by rfork() 0 pages cached 55169 pages freed 0 pages freed by daemon 0 pages freed by exiting processes 465 pages active 860 pages inactive 0 pages in VM cache 190921 pages wired down 15883166 pages free 4096 bytes per page 1886 total name lookups cache hits (80% pos + 4% neg) system 0% per-directory deletions 0%, falsehits 0%, toolong 0% ------------------------------------------------------------------------ vmstat -m Type InUse MemUse HighUse Requests Size(s) cdev 8 2K - 8 256 ppbusdev 2 1K - 2 256 filedesc 27 54K - 76 2048 kdtrace 323 76K - 515 64,256 kenv 86 11K - 114 16,32,64,128 proc-args 3 1K - 31 32,64,128 hhook 2 1K - 2 256 ithread 125 22K - 125 32,128,256 KTRACE 100 13K - 100 128 entropy 1026 65K - 1026 32,64 acpiintr 1 1K - 1 64 linker 403 101K - 533 16,32,64,128,256,512,1024,2048,4096 lockf 2 1K - 2 64,128 loginclass 2 1K - 2 64 cache 1 1K - 1 32 devbuf 17314 34391K - 18687 16,32,64,128,256,512,1024,2048,4096 temp 57 18K - 538 16,32,64,128,256,512,1024,2048 ip6ndp 3 1K - 3 64 ata_pci 1 1K - 1 64 acpica 1824 187K - 63680 16,32,64,128,256,512,1024,2048 kbdmux 7 18K - 7 16,512,1024,2048 module 521 66K - 521 128 mtx_pool 2 16K - 2 LED 4 1K - 4 16,128 CAM XPT 58 4K - 308 16,32,64,128,512,1024,2048 osd 5 1K - 11 16,32,64,128 pmchooks 1 1K - 1 128 CAM DEV 15 30K - 24 2048 acpitask 1 8K - 1 hdaa 4 5K - 4 512,1024,2048 pgrp 2 1K - 2 128 session 2 1K - 2 128 proc 2 256K - 2 subproc 100 145K - 149 512,4096 cred 40 7K - 240 64,256 plimit 2 1K - 2 256 uidinfo 2 33K - 2 128 hdac 1 1K - 1 512 hdacc 1 1K - 1 32 vtbuf 24 5712K - 24 4096 acpisem 22 3K - 22 128 vt 11 3K - 11 256 sysctl 0 0K - 43 16,32,64 sysctloid 7607 378K - 7779 16,32,64,128 sysctltmp 0 0K - 6 32,64 tidhash 1 256K - 1 callout 9 3208K - 9 umtx 666 84K - 666 128 p1003.1b 1 1K - 1 16 SWAP 12 19681K - 12 64 bus 2257 362K - 8440 16,32,64,128,256,512,1024 bus-sc 157 310K - 5479 16,32,64,128,256,512,1024,2048,4096 devstat 32 65K - 32 32,4096 eventhandler 104 9K - 104 64,128 DEVFS3 230 58K - 270 256 kobj 349 1396K - 1111 4096 DEVFS1 202 101K - 238 512 Per-cpu 1 1K - 1 32 DEVFS 41 1K - 42 16,128 rman 360 41K - 754 16,32,128 sbuf 0 0K - 2295 16,32,64,128,256 sglist 3 1K - 3 32 taskqueue 115 17K - 159 16,32,64,128,256 terminal 11 3K - 11 256 Unitno 28 2K - 120 32,64 vmem 2 160K - 3 ioctlops 0 0K - 158 16,32,64 iov 1 1K - 27 64,256,512 msg 4 30K - 4 2048,4096 sem 4 106K - 4 2048,4096 shm 1 20K - 1 tty 14 14K - 14 1024 shmfd 1 8K - 1 pcb 12 8341K - 12 16,128,1024,2048 vfscache 1 16384K - 1 vfs_hash 1 8192K - 1 vnodes 1 1K - 1 256 mount 194 7K - 402 16,32,64,128,256 vnodemarker 0 0K - 8 512 BPF 3 1K - 3 128 ifnet 4 7K - 4 128,2048 ifaddr 34 10K - 34 32,64,256,512,2048,4096 clone 8 1K - 8 128 arpcom 2 1K - 2 16 lltable 6 3K - 6 512 routetbl 5 2K - 5 256,512 igmp 3 1K - 3 256 sctp_vrf 1 1K - 1 64 hostcache 1 28K - 1 syncache 1 64K - 1 mld 3 1K - 3 128 rpc 2 1K - 2 512 audit_evclass 187 6K - 228 32 ufs_quota 1 8192K - 1 vm_pgdata 7 8193K - 7 128 UMAHash 19 10K - 19 512 pfs_nodes 73 19K - 73 256 pfs_vncache 2 1K - 2 64 scsi_cd 0 0K - 11 16 GEOM 404 67K - 3278 16,32,64,128,256,512,1024,2048 memdesc 1 4K - 1 4096 feeder 20 2K - 24 32,128 acpidev 36 3K - 36 64 atkbddev 2 1K - 2 64 CAM CCB 1 2K - 22943 2048 mixer 3 12K - 3 4096 raid_data 0 0K - 480 32,128,256 CAM path 22 1K - 110 32 solaris 326492 166906K - 672607 16,32,64,128,256,512,1024,2048,4096 UART 6 5K - 6 16,1024 kstat_data 6 1K - 6 64 CAM periph 16 4K - 40 16,32,64,128,256 md_nvidia_data 0 0K - 78 512 pci_link 16 2K - 16 32,64,128 md_sii_data 0 0K - 78 512 acpi_perf 8 1K - 8 64 CAM queue 23 8K - 57 16,32,512 apmdev 1 1K - 1 128 madt_table 0 0K - 1 4096 CAM dev queue 8 1K - 8 64 CAM SIM 8 2K - 8 256 ddb_capture 1 48K - 1 io_apic 2 4K - 2 2048 MCA 8 1K - 8 128 msi 4 1K - 4 128 nexusdev 4 1K - 4 16 isadev 5 1K - 5 128 USB 46 59K - 52 16,32,64,128,256,512,1024,2048,4096 USBdev 35 4K - 35 32,64,128,256 linux 17 2K - 17 64 spicds 7 1K - 7 128 envy24ht 15 195K - 15 64,2048 cpuctl 1 1K - 1 64 crypto 1 1K - 1 512 cyclic 32 3K - 32 16,64,128 SDT 32 2K - 32 16,64 fbt 60661 7839K - 60661 128 iprtheap 17 54K - 17 32,64,128,256 nvidia 194 864K - 195 16,32,64,128,256,512,1024,2048,4096 ipmi 0 0K - 11 128 gem_name 59 10K - 62 32,4096 drm_global 2 1K - 2 128,256 drm_vblank 7 1K - 7 16,64 drm_dma 2 1K - 2 32 drm_sarea 1 1K - 1 16 drm_driver 32 2203K - 36 16,32,64,128,256,512,1024,4096 drm_ctxbitmap 1 4K - 1 4096 drm_sman 13 2K - 13 128 drm_hashtab 1 4096K - 1 drm_kms 90 18K - 108 16,32,64,128,256,2048 ttm_pd 6 9K - 6 16,2048 ttm_rman 2 1K - 2 256 ttm_zone 2 1K - 2 64 ttm_poolmgr 1 1K - 1 512 fdesc_mount 1 1K - 1 16 ------------------------------------------------------------------------ vmstat -z ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP UMA Kegs: 384, 0, 267, 3, 267, 0, 0 UMA Zones: 1664, 0, 267, 1, 267, 0, 0 UMA Slabs: 112, 0, 12325, 30, 14833, 0, 0 UMA RCntSlabs: 120, 0, 0, 0, 0, 0, 0 UMA Hash: 256, 0, 58, 32, 77, 0, 0 4 Bucket: 32, 0, 1296, 1184, 9186, 0, 0 6 Bucket: 48, 0, 310, 1018, 1646, 0, 0 8 Bucket: 64, 0, 103, 1199, 2674, 0, 0 12 Bucket: 96, 0, 64, 1084, 1327, 0, 0 16 Bucket: 128, 0, 100, 1171, 1676, 17, 0 32 Bucket: 256, 0, 240, 630, 4097, 49, 0 64 Bucket: 512, 0, 238, 287, 1922, 49, 0 128 Bucket: 1024, 0, 1190, 102, 6750, 0, 0 vmem btag: 56, 0, 14856, 409, 14954, 215, 0 VM OBJECT: 256, 0, 583, 407, 1328, 0, 0 RADIX NODE: 144, 0, 8773, 272, 13246, 0, 0 MAP: 240, 0, 3, 61, 3, 0, 0 KMAP ENTRY: 128, 0, 8, 271, 8, 0, 0 MAP ENTRY: 128, 0, 103, 1044, 1717, 0, 0 VMSPACE: 448, 0, 4, 172, 54, 0, 0 fakepg: 104, 0, 0, 0, 0, 0, 0 mt_zone: 4112, 0, 405, 0, 405, 0, 0 16: 16, 0, 6, 243, 6, 0, 0 16: 16, 0, 2762, 724, 3015, 0, 0 16: 16, 0, 154129, 749, 173890, 0, 0 16: 16, 0, 54, 693, 28214, 0, 0 16: 16, 0, 62, 436, 63, 0, 0 16: 16, 0, 690, 804, 5386, 0, 0 16: 16, 0, 30, 219, 30, 0, 0 16: 16, 0, 1, 746, 18, 0, 0 32: 32, 0, 30, 962, 116, 0, 0 32: 32, 0, 3092, 876, 3286, 0, 0 32: 32, 0, 74071, 577, 140388, 0, 0 32: 32, 0, 66, 430, 4216, 0, 0 32: 32, 0, 18, 1098, 98, 0, 0 32: 32, 0, 833, 903, 2868, 0, 0 32: 32, 0, 94, 526, 186, 0, 0 32: 32, 0, 8, 612, 164, 0, 0 64: 64, 0, 5, 181, 6, 0, 0 64: 64, 0, 516, 786, 586, 0, 0 64: 64, 0, 20714, 1172, 73124, 0, 0 64: 64, 0, 767, 659, 1637, 0, 0 64: 64, 0, 80, 106, 80, 0, 0 64: 64, 0, 9171, 749, 9688, 0, 0 64: 64, 0, 131, 303, 131, 0, 0 64: 64, 0, 1038, 884, 1071, 0, 0 128: 128, 0, 34, 865, 95, 0, 0 128: 128, 0, 1899, 364, 1908, 0, 0 128: 128, 0, 121894, 13359, 190707, 0, 0 128: 128, 0, 1077, 318, 27008, 0, 0 128: 128, 0, 112, 911, 192, 0, 0 128: 128, 0, 1877, 882, 4086, 0, 0 128: 128, 0, 29, 126, 30, 0, 0 128: 128, 0, 524, 499, 526, 0, 0 256: 256, 0, 2, 73, 2, 0, 0 256: 256, 0, 301, 254, 444, 0, 0 256: 256, 0, 7514, 4696, 54916, 0, 0 256: 256, 0, 64, 131, 740, 0, 0 256: 256, 0, 28, 287, 348, 0, 0 256: 256, 0, 903, 447, 1957, 0, 0 256: 256, 0, 31, 44, 31, 0, 0 256: 256, 0, 4, 251, 18, 0, 0 512: 512, 0, 2, 33, 9, 0, 0 512: 512, 0, 88, 206, 244, 0, 0 512: 512, 0, 347, 500, 78882, 0, 0 512: 512, 0, 16, 47, 35, 0, 0 512: 512, 0, 3, 32, 3, 0, 0 512: 512, 0, 327, 198, 2807, 0, 0 512: 512, 0, 47, 16, 47, 0, 0 512: 512, 0, 1, 62, 2, 0, 0 1024: 1024, 0, 0, 92, 99, 0, 0 1024: 1024, 0, 9, 19, 9, 0, 0 1024: 1024, 0, 8351, 125, 23312, 0, 0 1024: 1024, 0, 5, 23, 2074, 0, 0 1024: 1024, 0, 16, 12, 16, 0, 0 1024: 1024, 0, 230, 82, 1273, 0, 0 1024: 1024, 0, 2, 14, 2, 0, 0 1024: 1024, 0, 0, 0, 0, 0, 0 2048: 2048, 0, 2, 4, 9, 0, 0 2048: 2048, 0, 6, 4, 6, 0, 0 2048: 2048, 0, 11, 25, 39, 0, 0 2048: 2048, 0, 8, 58, 22955, 0, 0 2048: 2048, 0, 21, 9, 30, 0, 0 2048: 2048, 0, 48, 38, 466, 0, 0 2048: 2048, 0, 5, 1, 5, 0, 0 2048: 2048, 0, 0, 0, 0, 0, 0 4096: 4096, 0, 0, 0, 0, 0, 0 4096: 4096, 0, 5, 0, 5, 0, 0 4096: 4096, 0, 298, 2, 497, 0, 0 4096: 4096, 0, 10, 0, 12, 0, 0 4096: 4096, 0, 16, 0, 17, 0, 0 4096: 4096, 0, 25, 0, 415, 0, 0 4096: 4096, 0, 15, 0, 15, 0, 0 4096: 4096, 0, 349, 9, 1111, 0, 0 uint64 pcpu: 8, 0, 1398, 138, 1398, 0, 0 SLEEPQUEUE: 88, 0, 334, 565, 334, 0, 0 Files: 80, 0, 8, 1021, 395, 0, 0 rl_entry: 40, 0, 6, 588, 6, 0, 0 TURNSTILE: 136, 0, 334, 226, 334, 0, 0 umtx pi: 96, 0, 0, 0, 0, 0, 0 MAC labels: 40, 0, 0, 0, 0, 0, 0 PROC: 1208, 0, 26, 46, 75, 0, 0 THREAD: 1168, 0, 295, 38, 438, 0, 0 cpuset: 72, 0, 282, 488, 425, 0, 0 cyclic_id_cache: 64, 0, 0, 0, 0, 0, 0 audit_record: 1248, 0, 0, 0, 0, 0, 0 mbuf_packet: 256, 25720710, 0, 10, 0, 0, 0 mbuf: 256, 25720710, 1, 134, 1, 0, 0 mbuf_cluster: 2048, 4018860, 0, 0, 0, 0, 0 mbuf_jumbo_page: 4096, 2009429, 0, 0, 0, 0, 0 mbuf_jumbo_9k: 9216, 1786158, 0, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 1339616, 0, 0, 0, 0, 0 mbuf_ext_refcnt: 4, 0, 0, 0, 0, 0, 0 sendfile_sync: 128, 0, 0, 0, 0, 0, 0 dtrace_state_cache: 4096, 0, 0, 0, 0, 0, 0 g_bio: 248, 0, 3, 669, 68266, 0, 0 DMAR_MAP_ENTRY: 120, 0, 0, 0, 0, 0, 0 ttyinq: 160, 0, 15, 57, 15, 0, 0 ttyoutq: 256, 0, 8, 67, 8, 0, 0 ata_request: 336, 0, 0, 88, 42, 0, 0 vtnet_tx_hdr: 24, 0, 0, 0, 0, 0, 0 cryptop: 88, 0, 0, 0, 0, 0, 0 cryptodesc: 72, 0, 0, 0, 0, 0, 0 nv_stack_t: 12288, 0, 2, 1, 6, 0, 0 FPU_save_area: 512, 0, 0, 0, 0, 0, 0 taskq_zone: 48, 0, 0, 664, 47, 0, 0 taskq_zone: 48, 0, 0, 0, 0, 0, 0 VNODE: 472, 0, 4245, 139, 4246, 0, 0 VNODEPOLL: 112, 0, 0, 0, 0, 0, 0 BUF TRIE: 144, 0, 0, 105948, 0, 0, 0 NAMEI: 1024, 0, 0, 104, 887, 0, 0 S VFS Cache: 108, 0, 247, 908, 285, 0, 0 STS VFS Cache: 148, 0, 0, 0, 0, 0, 0 L VFS Cache: 328, 0, 0, 0, 0, 0, 0 LTS VFS Cache: 368, 0, 0, 0, 0, 0, 0 NCLNODE: 528, 0, 0, 0, 0, 0, 0 DIRHASH: 1024, 0, 0, 0, 0, 0, 0 reference_cache: 40, 0, 0, 0, 0, 0, 0 reference_history_cache: 8, 0, 0, 0, 0, 0, 0 range_seg_cache: 64, 0, 15246, 81288, 176756, 0, 0 zio_cache: 920, 0, 2874, 3294, 146114, 0, 0 zio_link_cache: 48, 0, 2867, 4686, 145725, 0, 0 zio_buf_512: 512, 0, 4917, 221, 5752, 0, 0 zio_data_buf_512: 512, 0, 362, 247, 801, 0, 0 zio_buf_1024: 1024, 0, 95, 113, 269, 0, 0 zio_data_buf_1024: 1024, 0, 421, 99, 521, 0, 0 zio_buf_1536: 1536, 0, 35, 127, 335, 0, 0 zio_data_buf_1536: 1536, 0, 717, 85, 845, 0, 0 zio_buf_2048: 2048, 0, 45, 91, 262, 0, 0 zio_data_buf_2048: 2048, 0, 295, 51, 408, 0, 0 zio_buf_2560: 2560, 0, 11, 70, 131, 0, 0 zio_data_buf_2560: 2560, 0, 202, 32, 291, 0, 0 zio_buf_3072: 3072, 0, 4, 53, 79, 0, 0 zio_data_buf_3072: 3072, 0, 151, 29, 225, 0, 0 zio_buf_3584: 3584, 0, 5, 28, 49, 0, 0 zio_data_buf_3584: 3584, 0, 106, 11, 153, 0, 0 zio_buf_4096: 4096, 0, 3767, 1199, 45373, 0, 0 zio_data_buf_4096: 4096, 0, 97, 116, 2911, 0, 0 zio_buf_5120: 5120, 0, 2, 55, 89, 0, 0 zio_data_buf_5120: 5120, 0, 185, 21, 272, 0, 0 zio_buf_6144: 6144, 0, 5, 35, 62, 0, 0 zio_data_buf_6144: 6144, 0, 150, 13, 206, 0, 0 zio_buf_7168: 7168, 0, 2, 37, 51, 0, 0 zio_data_buf_7168: 7168, 0, 110, 10, 164, 0, 0 zio_buf_8192: 8192, 0, 2, 286, 1766, 0, 0 zio_data_buf_8192: 8192, 0, 102, 9, 192, 0, 0 zio_buf_10240: 10240, 0, 1, 49, 59, 0, 0 zio_data_buf_10240: 10240, 0, 145, 10, 215, 0, 0 zio_buf_12288: 12288, 0, 1, 176, 1011, 0, 0 zio_data_buf_12288: 12288, 0, 102, 10, 149, 0, 0 zio_buf_14336: 14336, 0, 0, 24, 27, 0, 0 zio_data_buf_14336: 14336, 0, 67, 12, 100, 0, 0 zio_buf_16384: 16384, 0, 718, 40, 2238, 0, 0 zio_data_buf_16384: 16384, 0, 57, 8, 86, 0, 0 zio_buf_20480: 20480, 0, 0, 57, 571, 0, 0 zio_data_buf_20480: 20480, 0, 59, 7, 102, 0, 0 zio_buf_24576: 24576, 0, 0, 40, 565, 0, 0 zio_data_buf_24576: 24576, 0, 44, 6, 67, 0, 0 zio_buf_28672: 28672, 0, 0, 131, 1560, 0, 0 zio_data_buf_28672: 28672, 0, 40, 13, 60, 0, 0 zio_buf_32768: 32768, 0, 0, 27, 290, 0, 0 zio_data_buf_32768: 32768, 0, 31, 5, 43, 0, 0 zio_buf_36864: 36864, 0, 1, 16, 244, 0, 0 zio_data_buf_36864: 36864, 0, 32, 3, 40, 0, 0 zio_buf_40960: 40960, 0, 0, 14, 139, 0, 0 zio_data_buf_40960: 40960, 0, 18, 1, 19, 0, 0 zio_buf_45056: 45056, 0, 0, 13, 99, 0, 0 zio_data_buf_45056: 45056, 0, 21, 3, 25, 0, 0 zio_buf_49152: 49152, 0, 0, 18, 89, 0, 0 zio_data_buf_49152: 49152, 0, 16, 3, 22, 0, 0 zio_buf_53248: 53248, 0, 1, 17, 148, 0, 0 zio_data_buf_53248: 53248, 0, 22, 3, 26, 0, 0 zio_buf_57344: 57344, 0, 0, 11, 73, 0, 0 zio_data_buf_57344: 57344, 0, 14, 1, 15, 0, 0 zio_buf_61440: 61440, 0, 0, 12, 56, 0, 0 zio_data_buf_61440: 61440, 0, 10, 3, 13, 0, 0 zio_buf_65536: 65536, 0, 0, 11, 46, 0, 0 zio_data_buf_65536: 65536, 0, 4, 1, 5, 0, 0 zio_buf_69632: 69632, 0, 0, 8, 27, 0, 0 zio_data_buf_69632: 69632, 0, 8, 1, 9, 0, 0 zio_buf_73728: 73728, 0, 0, 9, 27, 0, 0 zio_data_buf_73728: 73728, 0, 5, 0, 5, 0, 0 zio_buf_77824: 77824, 0, 0, 9, 41, 0, 0 zio_data_buf_77824: 77824, 0, 6, 1, 7, 0, 0 zio_buf_81920: 81920, 0, 0, 13, 84, 0, 0 zio_data_buf_81920: 81920, 0, 9, 1, 10, 0, 0 zio_buf_86016: 86016, 0, 0, 8, 25, 0, 0 zio_data_buf_86016: 86016, 0, 5, 3, 8, 0, 0 zio_buf_90112: 90112, 0, 0, 10, 33, 0, 0 zio_data_buf_90112: 90112, 0, 3, 1, 4, 0, 0 zio_buf_94208: 94208, 0, 0, 6, 15, 0, 0 zio_data_buf_94208: 94208, 0, 4, 1, 5, 0, 0 zio_buf_98304: 98304, 0, 1, 5, 20, 0, 0 zio_data_buf_98304: 98304, 0, 9, 0, 9, 0, 0 zio_buf_102400: 102400, 0, 0, 7, 13, 0, 0 zio_data_buf_102400: 102400, 0, 2, 1, 3, 0, 0 zio_buf_106496: 106496, 0, 0, 16, 99, 0, 0 zio_data_buf_106496: 106496, 0, 12, 2, 14, 0, 0 zio_buf_110592: 110592, 0, 0, 12, 72, 0, 0 zio_data_buf_110592: 110592, 0, 6, 1, 7, 0, 0 zio_buf_114688: 114688, 0, 0, 22, 79, 0, 0 zio_data_buf_114688: 114688, 0, 6, 0, 6, 0, 0 zio_buf_118784: 118784, 0, 0, 8, 22, 0, 0 zio_data_buf_118784: 118784, 0, 0, 0, 0, 0, 0 zio_buf_122880: 122880, 0, 0, 9, 28, 0, 0 zio_data_buf_122880: 122880, 0, 6, 0, 6, 0, 0 zio_buf_126976: 126976, 0, 0, 10, 22, 0, 0 zio_data_buf_126976: 126976, 0, 0, 0, 0, 0, 0 zio_buf_131072: 131072, 0, 160, 79, 862, 0, 0 zio_data_buf_131072: 131072, 0, 383, 8, 556, 0, 0 lz4_ctx: 16384, 0, 0, 0, 0, 0, 0 sa_cache: 80, 0, 4199, 848, 4199, 0, 0 dnode_t: 1096, 0, 4649, 31, 5833, 0, 0 dmu_buf_impl_t: 336, 0, 9678, 68, 14441, 0, 0 arc_buf_hdr_t: 328, 0, 10162, 134, 11502, 0, 0 arc_buf_t: 72, 0, 9396, 614, 12289, 0, 0 zil_lwb_cache: 192, 0, 0, 0, 0, 0, 0 zfs_znode_cache: 368, 0, 4199, 151, 4199, 0, 0 pipe: 744, 0, 0, 55, 13, 0, 0 Mountpoints: 816, 0, 17, 23, 17, 0, 0 procdesc: 128, 0, 0, 0, 0, 0, 0 ksiginfo: 112, 0, 24, 1026, 24, 0, 0 itimer: 352, 0, 0, 0, 0, 0, 0 KNOTE: 128, 0, 0, 0, 0, 0, 0 socket: 696, 2062880, 0, 0, 0, 0, 0 ipq: 56, 125599, 0, 0, 0, 0, 0 udp_inpcb: 392, 2062880, 0, 0, 0, 0, 0 udpcb: 16, 2062965, 0, 0, 0, 0, 0 tcp_inpcb: 392, 2062880, 0, 0, 0, 0, 0 tcpcb: 1024, 2062880, 0, 0, 0, 0, 0 tcptw: 88, 27810, 0, 0, 0, 0, 0 syncache: 160, 15360, 0, 0, 0, 0, 0 hostcache: 136, 15370, 0, 0, 0, 0, 0 tcpreass: 40, 251262, 0, 0, 0, 0, 0 sackhole: 32, 0, 0, 0, 0, 0, 0 sctp_ep: 1408, 2062880, 0, 0, 0, 0, 0 sctp_asoc: 2352, 40000, 0, 0, 0, 0, 0 sctp_laddr: 48, 80012, 0, 0, 0, 0, 0 sctp_raddr: 728, 80000, 0, 0, 0, 0, 0 sctp_chunk: 136, 400026, 0, 0, 0, 0, 0 sctp_readq: 104, 400026, 0, 0, 0, 0, 0 sctp_stream_msg_out: 104, 400026, 0, 0, 0, 0, 0 sctp_asconf: 40, 400059, 0, 0, 0, 0, 0 sctp_asconf_ack: 48, 400060, 0, 0, 0, 0, 0 ripcb: 392, 2062880, 0, 0, 0, 0, 0 unpcb: 240, 2062880, 0, 0, 0, 0, 0 rtentry: 200, 0, 0, 0, 0, 0, 0 selfd: 56, 0, 0, 0, 0, 0, 0 SWAPMETA: 288, 8037731, 0, 0, 0, 0, 0 ------------------------------------------------------------------------ vmstat -i interrupt total rate irq6: fdc0 22 0 irq14: ata0 61 0 irq17: uhci0 ehci0 54 0 irq18: uhci2+ 306 2 irq19: uhci1 ahci0+ 21712 146 cpu0:timer 3524 23 irq259: hdac0 9 0 cpu7:timer 2042 13 cpu1:timer 1450 9 cpu3:timer 3609 24 cpu2:timer 1414 9 cpu6:timer 1949 13 cpu4:timer 1408 9 cpu5:timer 1575 10 Total 39135 264 ------------------------------------------------------------------------ pstat -T 8/2062880 files 0M/147455M swap space ------------------------------------------------------------------------ pstat -s Device 512-blocks Used Avail Capacity /dev/#C:0xc0 50331392 0 50331392 0% /dev/ad14p2 50331392 0 50331392 0% /dev/usb/3.2.1 50331392 0 50331392 0% /dev/#C:0xd0 50331392 0 50331392 0% /dev/#C:0xd8 50331392 0 50331392 0% /dev/#C:0xe0 50331392 0 50331392 0% Total 301988352 0 301988352 0% ------------------------------------------------------------------------ iostat tty ada0 ada1 ada2 cpu tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 0 9 15.20 161 2.39 14.99 169 2.47 15.24 167 2.49 0 0 4 1 96 ------------------------------------------------------------------------ ipcs -a Message Queues: T ID KEY MODE OWNER GROUP CREATOR CGROUP CBYTES QNUM QBYTES LSPID LRPID STIME RTIME CTIME Shared Memory: T ID KEY MODE OWNER GROUP CREATOR CGROUP NATTCH SEGSZ CPID LPID ATIME DTIME CTIME Semaphores: T ID KEY MODE OWNER GROUP CREATOR CGROUP NSEMS OTIME CTIME ------------------------------------------------------------------------ ipcs -T msginfo: msgmax: 16384 (max characters in a message) msgmni: 40 (# of message queues) msgmnb: 2048 (max characters in a message queue) msgtql: 40 (max # of messages in system) msgssz: 8 (size of a message segment) msgseg: 2048 (# of message segments in system) shminfo: shmmax: 536870912 (max shared memory segment size) shmmin: 1 (min shared memory segment size) shmmni: 192 (max number of shared memory identifiers) shmseg: 128 (max shared memory segments per process) shmall: 131072 (max amount of shared memory in pages) seminfo: semmni: 50 (# of semaphore identifiers) semmns: 340 (# of semaphores in system) semmnu: 150 (# of undo structures in system) semmsl: 340 (max # of semaphores per id) semopm: 100 (max # of operations per semop call) semume: 50 (max # of undo entries per process) semusz: 632 (size in bytes of undo structure) semvmx: 32767 (semaphore maximum value) semaem: 16384 (adjust on exit max value) ------------------------------------------------------------------------ nfsstat Client Info: Rpc Counts: Getattr Setattr Lookup Readlink Read Write Create Remove 2 0 0 0 0 0 0 0 Rename Link Symlink Mkdir Rmdir Readdir RdirPlus Access 0 0 0 0 0 0 0 0 Mknod Fsstat Fsinfo PathConf Commit 0 1 1 0 0 Rpc Info: TimedOut Invalid X Replies Retries Requests 0 0 0 0 4 Cache Info: Attr Hits Misses Lkup Hits Misses BioR Hits Misses BioW Hits Misses 0 0 0 0 0 0 0 0 BioRLHits Misses BioD Hits Misses DirE Hits Misses Accs Hits Misses 0 0 0 0 0 0 0 0 Server Info: Getattr Setattr Lookup Readlink Read Write Create Remove 0 0 0 0 0 0 0 0 Rename Link Symlink Mkdir Rmdir Readdir RdirPlus Access 0 0 0 0 0 0 0 0 Mknod Fsstat Fsinfo PathConf Commit 0 0 0 0 0 Server Ret-Failed 0 Server Faults 0 Server Cache Stats: Inprog Idem Non-idem Misses 0 0 0 0 Server Write Gathering: WriteOps WriteRPC Opsaved 0 0 0 ------------------------------------------------------------------------ netstat -s tcp: 0 packets sent 0 data packets (0 bytes) 0 data packets (0 bytes) retransmitted 0 data packets unnecessarily retransmitted 0 resends initiated by MTU discovery 0 ack-only packets (0 delayed) 0 URG only packets 0 window probe packets 0 window update packets 0 control packets 0 packets received 0 acks (for 0 bytes) 0 duplicate acks 0 acks for unsent data 0 packets (0 bytes) received in-sequence 0 completely duplicate packets (0 bytes) 0 old duplicate packets 0 packets with some dup. data (0 bytes duped) 0 out-of-order packets (0 bytes) 0 packets (0 bytes) of data after window 0 window probes 0 window update packets 0 packets received after close 0 discarded for bad checksums 0 discarded for bad header offset fields 0 discarded because packet too short 0 discarded due to memory problems 0 connection requests 0 connection accepts 0 bad connection attempts 0 listen queue overflows 0 ignored RSTs in the windows 0 connections established (including accepts) 0 connections closed (including 0 drops) 0 connections updated cached RTT on close 0 connections updated cached RTT variance on close 0 connections updated cached ssthresh on close 0 embryonic connections dropped 0 segments updated rtt (of 0 attempts) 0 retransmit timeouts 0 connections dropped by rexmit timeout 0 persist timeouts 0 connections dropped by persist timeout 0 Connections (fin_wait_2) dropped because of timeout 0 keepalive timeouts 0 keepalive probes sent 0 connections dropped by keepalive 0 correct ACK header predictions 0 correct data packet header predictions 0 syncache entries added 0 retransmitted 0 dupsyn 0 dropped 0 completed 0 bucket overflow 0 cache overflow 0 reset 0 stale 0 aborted 0 badack 0 unreach 0 zone failures 0 cookies sent 0 cookies received 0 hostcache entries added 0 bucket overflow 0 SACK recovery episodes 0 segment rexmits in SACK recovery episodes 0 byte rexmits in SACK recovery episodes 0 SACK options (SACK blocks) received 0 SACK options (SACK blocks) sent 0 SACK scoreboard overflow 0 packets with ECN CE bit set 0 packets with ECN ECT(0) bit set 0 packets with ECN ECT(1) bit set 0 successful ECN handshakes 0 times ECN reduced the congestion window udp: 0 datagrams received 0 with incomplete header 0 with bad data length field 0 with bad checksum 0 with no checksum 0 dropped due to no socket 0 broadcast/multicast datagrams undelivered 0 dropped due to full socket buffers 0 not for hashed pcb 0 delivered 0 datagrams output 0 times multicast source filter matched ip: 0 total packets received 0 bad header checksums 0 with size smaller than minimum 0 with data size < data length 0 with ip length > max ip packet size 0 with header length < data size 0 with data length < header length 0 with bad options 0 with incorrect version number 0 fragments received 0 fragments dropped (dup or out of space) 0 fragments dropped after timeout 0 packets reassembled ok 0 packets for this host 0 packets for unknown/unsupported protocol 0 packets forwarded (0 packets fast forwarded) 0 packets not forwardable 0 packets received for unknown multicast group 0 redirects sent 0 packets sent from this host 0 packets sent with fabricated ip header 0 output packets dropped due to no bufs, etc. 0 output packets discarded due to no route 0 output datagrams fragmented 0 fragments created 0 datagrams that can't be fragmented 0 tunneling packets that can't find gif 0 datagrams with bad address in header icmp: 0 calls to icmp_error 0 errors not generated in response to an icmp message 0 messages with bad code fields 0 messages less than the minimum length 0 messages with bad checksum 0 messages with bad length 0 multicast echo requests ignored 0 multicast timestamp requests ignored 0 message responses generated 0 invalid return addresses 0 no return routes igmp: 0 messages received 0 messages received with too few bytes 0 messages received with wrong TTL 0 messages received with bad checksum 0 V1/V2 membership queries received 0 V3 membership queries received 0 membership queries received with invalid field(s) 0 general queries received 0 group queries received 0 group-source queries received 0 group-source queries dropped 0 membership reports received 0 membership reports received with invalid field(s) 0 membership reports received for groups to which we belong 0 V3 reports received without Router Alert 0 membership reports sent arp: 0 ARP requests sent 0 ARP replies sent 0 ARP requests received 0 ARP replies received 0 ARP packets received 0 total packets dropped due to no ARP entry 0 ARP entrys timed out 0 Duplicate IPs seen ip6: 0 total packets received 0 with size smaller than minimum 0 with data size < data length 0 with bad options 0 with incorrect version number 0 fragments received 0 fragments dropped (dup or out of space) 0 fragments dropped after timeout 0 fragments that exceeded limit 0 packets reassembled ok 0 packets for this host 0 packets forwarded 0 packets not forwardable 0 redirects sent 0 packets sent from this host 0 packets sent with fabricated ip header 0 output packets dropped due to no bufs, etc. 0 output packets discarded due to no route 0 output datagrams fragmented 0 fragments created 0 datagrams that can't be fragmented 0 packets that violated scope rules 0 multicast packets which we don't join Mbuf statistics: 0 one mbuf 0 one ext mbuf 0 two or more ext mbuf 0 packets whose headers are not contiguous 0 tunneling packets that can't find gif 0 packets discarded because of too many headers 0 failures of source address selection Source addresses selection rule applied: icmp6: 0 calls to icmp6_error 0 errors not generated in response to an icmp6 message 0 errors not generated because of rate limitation 0 messages with bad code fields 0 messages < minimum length 0 bad checksums 0 messages with bad length Histogram of error messages to be generated: 0 no route 0 administratively prohibited 0 beyond scope 0 address unreachable 0 port unreachable 0 packet too big 0 time exceed transit 0 time exceed reassembly 0 erroneous header field 0 unrecognized next header 0 unrecognized option 0 redirect 0 unknown 0 message responses generated 0 messages with too many ND options 0 messages with bad ND options 0 bad neighbor solicitation messages 0 bad neighbor advertisement messages 0 bad router solicitation messages 0 bad router advertisement messages 0 bad redirect messages 0 path MTU changes rip6: 0 messages received 0 checksum calculations on inbound 0 messages with bad checksum 0 messages dropped due to no socket 0 multicast messages dropped due to no socket 0 messages dropped due to full socket buffers 0 delivered 0 datagrams output ------------------------------------------------------------------------ netstat -m netstat: invalid address (0x0) 1/144/145 mbufs in use (current/cache/total) 18446744073709551606/10/0/4018860 mbuf clusters in use (current/cache/total/max) 0/10 mbuf+clusters out of packet secondary zone in use (current/cache) 0/0/0/2009429 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/1786158 9k jumbo clusters in use (current/cache/total/max) 0/0/0/1339616 16k jumbo clusters in use (current/cache/total/max) 18014398509481964K/56K/36K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters delayed (4k/9k/16k) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) ------------------------------------------------------------------------ netstat -anr Routing tables Internet: Destination Gateway Flags Netif Expire Internet6: Destination Gateway Flags Netif Expire ------------------------------------------------------------------------ netstat -anA ------------------------------------------------------------------------ netstat -aL ------------------------------------------------------------------------ fstat fstat: can't read file 1 at 0x200007fffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 4 at 0x780000ffff fstat: can't read file 7 at 0x200007fffffffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0x200007fffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 4 at 0x780000ffff fstat: can't read file 7 at 0x200007fffffffff fstat: can't read file 8 at 0x4000000001fffff fstat: can't read file 10 at 0x780000ffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read file 1 at 0x200007fffffffff fstat: can't read file 2 at 0x4000000001fffff fstat: can't read file 4 at 0x780000ffff fstat: can't read file 7 at 0x200007fffffffff fstat: can't read file 8 at 0x4000000001fffff fstat: can't read file 10 at 0x780000ffff fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 fstat: can't read znode_phys at 0x1 USER CMD PID FD MOUNT INUM MODE SZ|DV R/W root zfs 75 root - - error - root zfs 75 wd - - error - root zfs 75 text - - error - root zfs 75 ctty /dev 13 crw------- console rw root zfs 75 0 /dev 13 crw------- console rw root zfs 75 6 /dev 13 crw------- console rw root sh 72 root - - error - root sh 72 wd - - error - root sh 72 text - - error - root sh 72 ctty /dev 13 crw------- console rw root sh 72 0 /dev 13 crw------- console rw root sh 72 6 /dev 13 crw------- console rw root sh 24 root - - error - root sh 24 wd - - error - root sh 24 text - - error - root sh 24 ctty /dev 13 crw------- console rw root sh 24 0 /dev 13 crw------- console rw root sh 24 6 /dev 13 crw------- console rw root init 1 root - - error - root init 1 wd - - error - root init 1 text - - error - root kernel 0 root - - error - root kernel 0 wd - - error - ------------------------------------------------------------------------ dmesg Copyright (c) 1992-2014 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.0-CURRENT #3 r261166: Sat Jan 25 14:55:24 CST 2014 root@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER amd64 FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 info: [drm] Initialized drm 1.1.0 20060810 CPU: Intel(R) Xeon(R) CPU E5410 @ 2.33GHz (2327.55-MHz K8-class CPU) Origin="GenuineIntel" Id=0x10676 Family=0x6 Model=0x17 Stepping=6 Features=0xbfebfbff Features2=0xce3bd AMD Features=0x20100800 AMD Features2=0x1 TSC: P-state invariant, performance statistics real memory = 68719476736 (65536 MB) avail memory = 65641644032 (62600 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs FreeBSD/SMP: 2 package(s) x 4 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP): APIC ID: 3 cpu4 (AP): APIC ID: 4 cpu5 (AP): APIC ID: 5 cpu6 (AP): APIC ID: 6 cpu7 (AP): APIC ID: 7 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard random: initialized kbd1 at kbdmux0 cryptosoft0: on motherboard acpi0: on motherboard acpi0: Power Button (fixed) unknown: I/O range not supported cpu0: on acpi0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 cpu4: on acpi0 cpu5: on acpi0 cpu6: on acpi0 cpu7: on acpi0 hpet0: iomem 0xfed00000-0xfed003ff irq 0,8 on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 atrtc0: port 0x70-0x71 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: at device 2.0 on pci0 pci1: on pcib1 pcib2: irq 16 at device 0.0 on pci1 pci2: on pcib2 pcib3: irq 16 at device 0.0 on pci2 pci3: on pcib3 pcib4: at device 0.0 on pci3 pci4: on pcib4 pcib5: at device 0.2 on pci3 pci5: on pcib5 pcib6: irq 18 at device 2.0 on pci2 pci6: on pcib6 em0: port 0x2000-0x201f mem 0xd9220000-0xd923ffff,0xd9200000-0xd921ffff irq 18 at device 0.0 on pci6 em0: Using an MSI interrupt em0: Ethernet address: 00:30:48:f2:29:9c em1: port 0x2020-0x203f mem 0xd9260000-0xd927ffff,0xd9240000-0xd925ffff irq 19 at device 0.1 on pci6 em1: Using an MSI interrupt em1: Ethernet address: 00:30:48:f2:29:9d pcib7: at device 0.3 on pci1 pci7: on pcib7 pcib8: at device 4.0 on pci0 pci8: on pcib8 vgapci0: port 0x3000-0x307f mem 0xd8000000-0xd8ffffff,0xc0000000-0xc7ffffff,0xc8000000-0xc9ffffff irq 16 at device 0.0 on pci8 nvidia0: on vgapci0 vgapci0: child nvidia0 requested pci_enable_io vgapci0: child nvidia0 requested pci_enable_io hdac0: mem 0xd9000000-0xd9003fff irq 17 at device 0.1 on pci8 pcib9: at device 6.0 on pci0 pci9: on pcib9 pci0: at device 8.0 (no driver attached) pcib10: irq 17 at device 28.0 on pci0 pci10: on pcib10 pcib11: irq 16 at device 0.0 on pci10 pci11: on pcib11 pcm0: port 0x4080-0x409f,0x4000-0x407f irq 16 at device 0.0 on pci11 pcm0: system configuration SubVendorID: 0x1412, SubDeviceID: 0x2403 XIN2 Clock Source: 24.576MHz(96kHz*256) MPU-401 UART(s) #: not implemented ADC #: 1 and SPDIF receiver connected DAC #: 4 Multi-track converter type: AC'97(SDATA_OUT:packed) S/PDIF(IN/OUT): 1/1 ID# 0x00 GPIO(mask/dir/state): 0xff/0xff/0xff uhci0: port 0x1800-0x181f irq 17 at device 29.0 on pci0 usbus0 on uhci0 uhci1: port 0x1820-0x183f irq 19 at device 29.1 on pci0 usbus1 on uhci1 uhci2: port 0x1840-0x185f irq 18 at device 29.2 on pci0 usbus2 on uhci2 ehci0: mem 0xd9600400-0xd96007ff irq 17 at device 29.7 on pci0 usbus3: EHCI version 1.0 usbus3 on ehci0 pcib12: at device 30.0 on pci0 pci12: on pcib12 vgapci1: port 0x5000-0x50ff mem 0xd0000000-0xd7ffffff,0xd9300000-0xd930ffff irq 18 at device 1.0 on pci12 drmn1: on vgapci1 info: [drm] RADEON_IS_PCI info: [drm] initializing kernel modesetting (RV100 0x1002:0x515E 0x15D9:0x8080). info: [drm] register mmio base: 0xD9300000 info: [drm] register mmio size: 65536 info: [drm] radeon_atrm_get_bios: ===> Try ATRM... info: [drm] radeon_atrm_get_bios: pci_find_class() found: 0:8:0:0, vendor=10de, device=104a info: [drm] radeon_atrm_get_bios: Get ACPI device handle info: [drm] radeon_acpi_vfct_bios: ===> Try VFCT... info: [drm] radeon_acpi_vfct_bios: Get "VFCT" ACPI table info: [drm] radeon_acpi_vfct_bios: Failed to get "VFCT" table: AE_NOT_FOUND info: [drm] igp_read_bios_from_vram: ===> Try IGP's VRAM... info: [drm] igp_read_bios_from_vram: VRAM base address: 0xd0000000 info: [drm] igp_read_bios_from_vram: Map address: 0xfffff800d0000000 (262144 bytes) info: [drm] igp_read_bios_from_vram: Incorrect BIOS signature: 0x0000 info: [drm] radeon_read_bios: ===> Try PCI Expansion ROM... info: [drm] radeon_read_bios: Map address: 0xfffff800000c0000 (131072 bytes) drmn1: info: VRAM: 128M 0x00000000D0000000 - 0x00000000D7FFFFFF (16M used) drmn1: info: GTT: 512M 0x00000000B0000000 - 0x00000000CFFFFFFF info: [drm] Supports vblank timestamp caching Rev 1 (10.10.2010). info: [drm] Driver supports precise vblank timestamp query. info: [drm] radeon: irq initialized. info: [drm] Detected VRAM RAM=128M, BAR=128M info: [drm] RAM width 64bits SDR [TTM] Zone kernel: Available graphics memory: 33006090 kiB [TTM] Zone dma32: Available graphics memory: 2097152 kiB [TTM] Initializing pool allocator info: [drm] radeon: 16M of VRAM memory ready info: [drm] radeon: 512M of GTT memory ready. info: [drm] GART: num cpu pages 131072, num gpu pages 131072 info: [drm] PCI GART of 512M enabled (table at 0x0000000011200000). drmn1: info: WB disabled drmn1: info: fence driver on ring 0 use gpu addr 0x00000000b0000000 and cpu addr 0x0xfffff8001083c000 info: [drm] Loading R100 Microcode info: [drm] radeon: ring at 0x00000000B0001000 info: [drm] ring test succeeded in 1 usecs info: [drm] ib test succeeded in 0 usecs info: [drm] radeon_device_init: Taking over the fictitious range 0xd0000000-0xd4000000 iicbus0: on iicbb0 addr 0xff iic0: on iicbus0 iicbus1: on iicbb1 addr 0xff iic1: on iicbus1 iicbus2: on iicbb2 addr 0xff iic2: on iicbus2 iicbus3: on iicbb3 addr 0xff iic3: on iicbus3 info: [drm] Radeon Display Connectors info: [drm] Connector 0: info: [drm] VGA-1 info: [drm] DDC: 0x60 0x60 0x60 0x60 0x60 0x60 0x60 0x60 info: [drm] Encoders: info: [drm] CRT1: INTERNAL_DAC1 info: [drm] Connector 1: info: [drm] DVI-I-1 info: [drm] HPD2 info: [drm] DDC: 0x6c 0x6c 0x6c 0x6c 0x6c 0x6c 0x6c 0x6c info: [drm] Encoders: info: [drm] CRT2: INTERNAL_DAC2 info: [drm] DFP2: INTERNAL_DVO1 composite sync not supported composite sync not supported info: [drm] fb mappable at 0xD0040000 info: [drm] vram apper at 0xD0000000 info: [drm] size 2076672 info: [drm] fb depth is 8 info: [drm] pitch is 1920 fbd1 on drmn1 vt_allocate: Replace existing VT driver. info: [drm] Initialized radeon 2.29.0 20080528 vgapci1: Boot video device isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1860-0x186f at device 31.1 on pci0 ata0: at channel 0 on atapci0 ahci0: port 0x18a0-0x18a7,0x1874-0x1877,0x1878-0x187f,0x1870-0x1873,0x1880-0x189f mem 0xd9600800-0xd9600bff irq 19 at device 31.2 on pci0 ahci0: AHCI v1.10 with 6 3Gbps ports, Port Multiplier supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 ahcich2: at channel 2 on ahci0 ahcich3: at channel 3 on ahci0 ahcich4: at channel 4 on ahci0 ahcich5: at channel 5 on ahci0 ichsmb0: port 0x1100-0x111f irq 19 at device 31.3 on pci0 smbus0: on ichsmb0 acpi_button0: on acpi0 ipmi0: port 0xca2-0xca3 on acpi0 ipmi0: KCS mode found at io 0xca2 on acpi atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fd0: <1440-KB 3.5" drive> on fdc0 drive 0 ppc0: port 0x378-0x37f,0x778-0x77f irq 7 drq 3 on acpi0 ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/9 bytes threshold ppbus0: on ppc0 lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 ichwd0 on isa0 orm0: at iomem 0xc0000-0xcafff on isa0 coretemp0: on cpu0 est0: on cpu0 p4tcc0: on cpu0 coretemp1: on cpu1 est1: on cpu1 p4tcc1: on cpu1 coretemp2: on cpu2 est2: on cpu2 p4tcc2: on cpu2 coretemp3: on cpu3 est3: on cpu3 p4tcc3: on cpu3 coretemp4: on cpu4 est4: on cpu4 p4tcc4: on cpu4 coretemp5: on cpu5 est5: on cpu5 p4tcc5: on cpu5 coretemp6: on cpu6 est6: on cpu6 p4tcc6: on cpu6 coretemp7: on cpu7 est7: on cpu7 p4tcc7: on cpu7 ZFS filesystem version: 5 ZFS storage pool version: features support (5000) Timecounters tick every 1.000 msec vboxdrv: fAsync=0 offMin=0x387 offMax=0x50f hdacc0: at cad 0 on hdac0 hdaa0: at nid 1 on hdacc0 pcm1: at nid 4 on hdaa0 pcm2: at nid 5 on hdaa0 random: unblocking device. usbus0: 12Mbps Full Speed USB v1.0 usbus1: 12Mbps Full Speed USB v1.0 usbus2: 12Mbps Full Speed USB v1.0 usbus3: 480Mbps High Speed USB v2.0 ugen3.1: at usbus3 uhub0: on usbus3 ugen2.1: at usbus2 uhub1: on usbus2 ugen1.1: at usbus1 uhub2: on usbus1 ugen0.1: at usbus0 uhub3: on usbus0 ipmi0: IPMI device rev. 1, firmware rev. 1.64, version 2.0 ata0: DMA limited to UDMA33, controller found non-ATA66 cable ipmi0: Number of channels 8 ipmi0: Attached watchdog uhub2: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered uhub3: 2 ports with 2 removable, self powered ada0 at ahcich0 bus 0 scbus1 target 0 lun 0 ada0: ATA-8 SATA 3.x device ada0: Serial Number 5YDA1ZL4 ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada0: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada0: quirks=0x1<4K> ada0: Previously was known as ad4 ada1 at ahcich1 bus 0 scbus2 target 0 lun 0 ada1: ATA-8 SATA 3.x device ada1: Serial Number 5YD6FPLG ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada1: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada1: quirks=0x1<4K> ada1: Previously was known as ad6 ada2 at ahcich2 bus 0 scbus3 target 0 lun 0 ada2: ATA-8 SATA 3.x device ada2: Serial Number 5YDA3PC5 ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada2: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada2: quirks=0x1<4K> ada2: Previously was known as ad8 ada3 at ahcich3 bus 0 scbus4 target 0 lun 0 ada3: ATA-8 SATA 3.x device ada3: Serial Number 5YD9Y0P4 ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada3: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada3: quirks=0x1<4K> ada3: Previously was known as ad10 ada4 at ahcich4 bus 0 scbus5 target 0 lun 0 ada4: ATA-8 SATA 3.x device ada4: Serial Number 5YDA1R5W ada4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada4: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada4: quirks=0x1<4K> ada4: Previously was known as ad12 ada5 at ahcich5 bus 0 scbus6 target 0 lun 0 ada5: ATA-8 SATA 3.x device ada5: Serial Number 5YD5RBS8 ada5: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada5: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada5: quirks=0x1<4K> ada5: Previously was known as ad14 cd0 at ata0 bus 0 scbus0 target 0 lun 0 cd0: Removable CD-ROM SCSI-0 device cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present Netvsc initializing... done! SMP: AP CPU #7 Launched! SMP: AP CPU #1 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #6 Launched! SMP: AP CPU #4 Launched! SMP: AP CPU #5 Launched! Root mount waiting for: usbus3 uhub0: 6 ports with 6 removable, self powered Root mount waiting for: usbus3 Root mount waiting for: usbus3 ugen3.2: at usbus3 ukbd0: on usbus3 kbd2 at ukbd0 Trying to mount root from zfs:zroot/ROOT/default []... ugen0.2: at usbus0 ugen1.2: at usbus1 Setting hostuuid: 53d19f64-d663-a017-8922-0030488e9ff3. Setting hostid: 0xf53a926e. Entropy harvesting: interrupts ethernet point_to_point swi. Starting file system checks: Mounting local file systems:. panic: solaris assert: l->l_phys->l_hdr.lh_pad1 == 0 (0xffff000000000000 == 0x0), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zap.c, line: 479 cpuid = 7 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe100c55d350 kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe100c55d400 vpanic() at vpanic+0x126/frame 0xfffffe100c55d440 panic() at panic+0x43/frame 0xfffffe100c55d4a0 assfail3() at assfail3+0x2f/frame 0xfffffe100c55d4c0 zap_get_leaf_byblk() at zap_get_leaf_byblk+0x2ea/frame 0xfffffe100c55d510 zap_deref_leaf() at zap_deref_leaf+0xd1/frame 0xfffffe100c55d560 fzap_length() at fzap_length+0x31/frame 0xfffffe100c55d5d0 zap_length_uint64() at zap_length_uint64+0xbf/frame 0xfffffe100c55d630 ddt_zap_lookup() at ddt_zap_lookup+0x3d/frame 0xfffffe100c55d790 ddt_lookup() at ddt_lookup+0x17e/frame 0xfffffe100c55d960 zio_ddt_write() at zio_ddt_write+0xf2/frame 0xfffffe100c55dad0 zio_execute() at zio_execute+0x1eb/frame 0xfffffe100c55db30 taskqueue_run_locked() at taskqueue_run_locked+0xf0/frame 0xfffffe100c55db80 taskqueue_thread_loop() at taskqueue_thread_loop+0x9b/frame 0xfffffe100c55dbb0 fork_exit() at fork_exit+0x84/frame 0xfffffe100c55dbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe100c55dbf0 --- trap 0, rip = 0, rsp = 0xfffffe100c55dcb0, rbp = 0 --- KDB: enter: panic Uptime: 23s Dumping 2330 out of 64465 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% ------------------------------------------------------------------------ kernel config options CONFIG_AUTOGENERATED ident VT-LER machine amd64 cpu HAMMER makeoptions WITH_CTF=1 makeoptions DEBUG=-g options ZFS options XENHVM options USB_DEBUG options ATH_ENABLE_11N options AH_AR5416_INTERRUPT_MITIGATION options AH_SUPPORT_AR5416 options IEEE80211_SUPPORT_MESH options IEEE80211_AMPDU_AGE options IEEE80211_DEBUG options SC_PIXEL_MODE options VESA options AHD_REG_PRETTY_PRINT options AHC_REG_PRETTY_PRINT options ATA_STATIC_ID options ACPI_DMAR options SMP options MALLOC_DEBUG_MAXZONES=8 options INVARIANT_SUPPORT options INVARIANTS options DEADLKRES options GDB options DDB options KDB_TRACE options KDB options INCLUDE_CONFIG_FILE options DDB_CTF options KDTRACE_HOOKS options KDTRACE_FRAME options MAC options CAPABILITIES options CAPABILITY_MODE options AUDIT options HWPMC_HOOKS options KBD_INSTALL_CDEV options PRINTF_BUFR_SIZE=128 options _KPOSIX_PRIORITY_SCHEDULING options SYSVSEM options SYSVMSG options SYSVSHM options STACK options KTRACE options SCSI_DELAY=5000 options COMPAT_FREEBSD7 options COMPAT_FREEBSD6 options COMPAT_FREEBSD5 options COMPAT_FREEBSD4 options COMPAT_FREEBSD32 options GEOM_LABEL options GEOM_RAID options GEOM_PART_GPT options PSEUDOFS options PROCFS options CD9660 options MSDOSFS options NFS_ROOT options NFSLOCKD options NFSD options NFSCL options MD_ROOT options QUOTA options UFS_GJOURNAL options UFS_DIRHASH options UFS_ACL options SOFTUPDATES options FFS options SCTP options TCP_OFFLOAD options INET6 options INET options PREEMPTION options SCHED_ULE options NEW_PCIB options GEOM_PART_MBR options GEOM_PART_EBR_COMPAT options GEOM_PART_EBR options GEOM_PART_BSD device isa device mem device io device uart_ns8250 device cpufreq device acpi device pci device fdc device ahci device ata device mvs device siis device ahc device ahd device esp device hptiop device isp device mpt device mps device sym device trm device adv device adw device aic device bt device isci device scbus device ch device da device sa device cd device pass device ses device amr device arcmsr device ciss device dpt device hptmv device hptnr device hptrr device hpt27xx device iir device ips device mly device twa device tws device aac device aacp device aacraid device ida device mfi device mlx device twe device atkbdc device atkbd device psm device kbdmux device splash device agp device cbb device pccard device cardbus device uart device ppc device ppbus device lpt device ppi device puc device bxe device de device em device igb device ixgbe device le device ti device txp device vx device miibus device ae device age device alc device ale device bce device bfe device bge device cas device dc device et device fxp device gem device hme device jme device lge device msk device nfe device nge device pcn device re device rl device sf device sge device sis device sk device ste device stge device tl device tx device vge device vr device wb device xl device cs device ed device ex device ep device fe device sn device xe device wlan device wlan_wep device wlan_ccmp device wlan_tkip device wlan_amrr device an device ath device ath_pci device ath_hal device ath_rate_sample device ipw device iwi device iwn device malo device mwl device ral device wi device wpi device loop device random device padlock_rng device rdrand_rng device ether device vlan device tun device md device gif device faith device firmware device bpf device uhci device ohci device ehci device xhci device usb device ukbd device umass device sound device snd_cmi device snd_csa device snd_emu10kx device snd_es137x device snd_hda device snd_ich device snd_via8233 device mmc device mmcsd device sdhci device virtio device virtio_pci device vtnet device virtio_blk device virtio_scsi device virtio_balloon device hyperv device xenpci device vmx device vt device vt_vga ------------------------------------------------------------------------ ddb capture buffer -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688