From owner-freebsd-fs@FreeBSD.ORG Thu Apr 4 16:12:53 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 4C9B0491 for ; Thu, 4 Apr 2013 16:12:53 +0000 (UTC) (envelope-from toasty@dragondata.com) Received: from mail-ob0-x234.google.com (mail-ob0-x234.google.com [IPv6:2607:f8b0:4003:c01::234]) by mx1.freebsd.org (Postfix) with ESMTP id 153D1DF8 for ; Thu, 4 Apr 2013 16:12:53 +0000 (UTC) Received: by mail-ob0-f180.google.com with SMTP id wo10so2762683obc.11 for ; Thu, 04 Apr 2013 09:12:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dragondata.com; s=google; h=x-received:content-type:mime-version:subject:from:in-reply-to:date :cc:content-transfer-encoding:message-id:references:to:x-mailer; bh=w8NskagA7ayeqhFwDoWnlxlRk/W5SadekaBvMW0ZZGw=; b=IjakrIiBgTeeU4riG4FrgR5vJrQAO6ebxZvc+4VfVgH81YHIiGxSuuYCdYGi6hC2LN FoAXYfqIU16xxXxtsf+zKTLP32j0Azr21cDax4Vd+gFyis1TZKAavdWCfbW0KvYAqAwX 5/XLD2JNK4mmYd1Zdy+94mzXZ5Evt/OA1bCTc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:content-type:mime-version:subject:from:in-reply-to:date :cc:content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=w8NskagA7ayeqhFwDoWnlxlRk/W5SadekaBvMW0ZZGw=; b=H5ldnDTYMlF8QzQF8bbyrvTrsEfDbquRC+qC6EAFbhuaH8uyV+tNm+a10lNr/v0SYR dz+vBO7t75jKWneGqrlm5GodfNtDv7a2s1R5geVI0bQr1Abf+hUyGton++JIWnBSdXtq XPK+UbVfsmpO74p3io46Ov9WARn2l5c0lRY5U+PVFE54GZYw8mgBblvdbw6SH4So+VsD xY8MxIv4Pxmye4IDTyhRrK13oor6tz/EqQLixNge1vY04s2P6m/zzd/SMp8Uq3zEOazu 0QuY30nS2dOlclLJGoaR7mOs8W9QU/Qt+DNo+z+cqvOZU3u8NkC4EF8noxBOCRFUvDId 71iw== X-Received: by 10.60.20.225 with SMTP id q1mr4714996oee.31.1365091972568; Thu, 04 Apr 2013 09:12:52 -0700 (PDT) Received: from vpn132.rw1.your.org (vpn132.rw1.your.org. [204.9.51.132]) by mx.google.com with ESMTPS id t9sm6998109obk.13.2013.04.04.09.12.50 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 04 Apr 2013 09:12:51 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.3 \(1503\)) Subject: Re: kern/177536: zfs livelock (deadlock) with high write-to-disk load From: Kevin Day In-Reply-To: <515DA525.3020006@FreeBSD.org> Date: Thu, 4 Apr 2013 11:12:48 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: References: <201304041540.r34Fe1Ka057203@freefall.freebsd.org> <515DA525.3020006@FreeBSD.org> To: Andriy Gapon X-Mailer: Apple Mail (2.1503) X-Gm-Message-State: ALoCoQmxKKCWBFihvlE/3lG69B6FsYqkOfOsyAVynEDv5Qw08OLyFFRGO7Z8j0pdlajx8zOsOjiK Cc: freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Apr 2013 16:12:53 -0000 On Apr 4, 2013, at 11:07 AM, Andriy Gapon wrote: > on 04/04/2013 19:05 Kevin Day said the following: > [a lot] >=20 > One link: https://wiki.freebsd.org/AvgZfsDeadlockDebug Sorry, should have mentioned i've seen this. I've tried procstat, but i've failed at keeping it cached long enough to = use it when this happens. If I try running it, the shell gets stuck in = 'ufs'. I'm going to add a cron job to just run procstat periodically so = hopefully i can run it without it needing to touch the disks. I built a more debug friendly kernel and tried to drop to ddb when this = happened, but it didn't provide anything useful. When I hit 'enter' on = 'alltrace' it hard locked without printing anything. I haven't been able to trigger a core dump, and this system has no = serial or firewire ports for live debugging.