From owner-freebsd-fs@FreeBSD.ORG Mon Feb 9 05:16:20 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6873B106564A for ; Mon, 9 Feb 2009 05:16:20 +0000 (UTC) (envelope-from toasty@dragondata.com) Received: from tokyo01.jp.mail.your.org (tokyo01.jp.mail.your.org [204.9.54.5]) by mx1.freebsd.org (Postfix) with ESMTP id 29D658FC1A for ; Mon, 9 Feb 2009 05:16:19 +0000 (UTC) (envelope-from toasty@dragondata.com) Received: from tokyo01.jp.mail.your.org (localhost.your.org [127.0.0.1]) by tokyo01.jp.mail.your.org (Postfix) with ESMTP id B244C2AD6072 for ; Mon, 9 Feb 2009 04:59:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=dragondata.com; h= message-id:from:to:content-type:content-transfer-encoding :mime-version:subject:date; s=selector1; bh=FNZRv+VtLOXWhqPmNxyW iDR67pw=; b=oqHOpSDJHuS8xO+iOs2ZHax/J9HnrxWAHPPjdWJPo/l7Upukfdvs oPx5gGqIlinbzUD5trePm5i/x6VmrB53dS2rKgqAlV0QGUqGwiD0cR29jFgWvCNq E8DgrDQAUtVVccwvBgkKfYvQmbE9n96csSgA1RzD4QAMUuT3xKvroBY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=dragondata.com; h=message-id:from :to:content-type:content-transfer-encoding:mime-version:subject: date; q=dns; s=selector1; b=wfr+BWc54nZy/2M//P2cIKFvVGA1ap+71QHD qxNX3PfkBUJ6ez96/vowBNcCwhcx06uSvRFqfnjGfbPQcQWrxW+LtDFY6ktMsXXh 0IFPWrXRbtcs99wvMYCW71TCz9m4Vh/BNgTCe9cfyCya3UyyRXT/CS/FXn3NMxUz rnYUtxs= Received: from mail.your.org (server3-a.your.org [64.202.112.67]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by tokyo01.jp.mail.your.org (Postfix) with ESMTPS id 8844A2AD6069 for ; Mon, 9 Feb 2009 04:59:33 +0000 (UTC) Received: from pool011.dhcp.your.org (pool011.dhcp.your.org [69.31.99.11]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.your.org (Postfix) with ESMTPSA id 62AFBA0A414 for ; Mon, 9 Feb 2009 04:59:00 +0000 (UTC) Message-Id: <8E12CEFC-25DE-4B82-97BD-7ED717650089@dragondata.com> From: Kevin Day To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v930.3) Date: Sun, 8 Feb 2009 22:59:31 -0600 X-Mailer: Apple Mail (2.930.3) Subject: zio->io_cv deadlock X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Feb 2009 05:16:20 -0000 I'm playing with a -CURRENT install from a couple of weeks ago. Everything seems okay for a few days, then eventually every process ends up stuck in zio->io_cv. If I go to the console, it's responsive until I try logging in, then login is stuck in zio->io_cv as well. Ctrl-Alt-Esc drops me into ddb, but then ddb hangs instantly. Nothing on the console or syslog before it hangs. Anyone seen anything similar? -- Kevin Possibly relevant info: 8 core Opteron 64GB RAM da1 at twa0 bus 0 target 0 lun 1 da1: Fixed Direct Access SCSI-5 device da1: 100.000MB/s transfers da1: 4678158MB (9580867585 512 byte sectors: 255H 63S/T 596381C) server5# zpool list NAME SIZE USED AVAIL CAP HEALTH ALTROOT z 4.44T 1.19T 3.25T 26% ONLINE - server5# zpool status -v pool: z state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM z ONLINE 0 0 0 da1 ONLINE 0 0 0 errors: No known data errors server5# cat /boot/loader.conf vm.kmem_size_max="2048M" vm.kmem_size="2048M" vfs.zfs.arc_max="100M" zfs_load="YES" vfs.root.mountfrom="zfs:z" (tried lowering arc_max, didn't seem to help)