From owner-freebsd-threads@FreeBSD.ORG Thu Jul 16 12:48:13 2009 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 20EDB106566C for ; Thu, 16 Jul 2009 12:48:13 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-bw0-f208.google.com (mail-bw0-f208.google.com [209.85.218.208]) by mx1.freebsd.org (Postfix) with ESMTP id 9568A8FC23 for ; Thu, 16 Jul 2009 12:48:12 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: by bwz4 with SMTP id 4so80292bwz.43 for ; Thu, 16 Jul 2009 05:48:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type:content-transfer-encoding; bh=wY56AJBfyAiCqc0CQTx7Qp6BCJaTK8yyEniaDZRi1SM=; b=SggtN1H2LDDx/0pRerTX2MTgVZTHDrYy6U0CyWaNgmigmlwshGkIRAOJAYLQiGIUjA l8eLD7UiM8sMllwsV5Vx+VCOKfOLrNDG5JSMkW3JdTwet7o+ZwjoOR3E29IZ9C2zEqhC Pyg37WzvZlBDfhBzbN6roKVUGtUd2tktVTVXc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=GlfVm+UPZcNbQyf9JGvmKKLmGQ8xsdbLKwoodHURsmt8BRBLyIKZWyFALhlQVf0J3u hnrS4sNt7P7YVX0OYCZfqtK6eQssB/cVq9uR10JfCbW8kWcZnximirZLe7WHSwTPKr91 U9FWedYjynN7xXP0G0okyyWUqLS96Rqg9QT9M= MIME-Version: 1.0 Sender: asmrookie@gmail.com Received: by 10.223.107.9 with SMTP id z9mr4432003fao.1.1247748491333; Thu, 16 Jul 2009 05:48:11 -0700 (PDT) In-Reply-To: <3bbf2fe10907160526l2f066698qce8a5e77aee6366b@mail.gmail.com> References: <200907152140.n6FLe42l045879@freefall.freebsd.org> <3bbf2fe10907160526l2f066698qce8a5e77aee6366b@mail.gmail.com> Date: Thu, 16 Jul 2009 14:48:11 +0200 X-Google-Sender-Auth: fb69cdd58ecf0064 Message-ID: <3bbf2fe10907160548l74de896bka609de7a9a994899@mail.gmail.com> From: Attilio Rao To: Nick Esborn Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-threads@freebsd.org Subject: Re: threads/136345: Recursive read rwlocks in thread A cause deadlock with write lock in thread B X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Jul 2009 12:48:13 -0000 2009/7/16 Attilio Rao : > 2009/7/15 Nick Esborn : >> The following reply was made to PR threads/136345; it has been noted by GNATS. >> >> From: Nick Esborn >> To: bug-followup@FreeBSD.org, >> rink@FreeBSD.org >> Cc: >> Subject: Re: threads/136345: Recursive read rwlocks in thread A cause deadlock with write lock in thread B >> Date: Wed, 15 Jul 2009 14:32:38 -0700 >> >> This is an OpenPGP/MIME signed message (RFC 2440 and 3156) >> --Apple-Mail-19-950902279 >> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes >> Content-Transfer-Encoding: 7bit >> >> Even after the above patch, I still run into occasional MySQL thread >> deadlocks, which I originally described in what is now threads/135673. >> >> I also posted on freebsd-current a few days ago: >> >> http://lists.freebsd.org/pipermail/freebsd-current/2009-July/009328.html >> >> I'd be happy to collect whatever data would be helpful in tracking >> down this deadlock. This only seems to happen under our production >> workload, so that might make it harder to capture meaningful debug >> data, but I'm certainly willing to try. I can also arrange for >> developer access to the system in question, if that would help >> significantly. > > So did you backport this to 7 and still experience deadlocks? > I just committed the fix to HEAD not to STABLE branch. Ok, I got, you just upgraded. Can you try the following things?: - Upgrade to the -CURRENT of today - Recompile the kernel with the following options: KDB, DDB, SCHED_ULE, PREEMPTION, FULL_PREEMPTION, INVARIANT_SUPPORT, INVARIANTS, WITNESS - When the deadlock takes place break into DDB and please retrieve the following info: db> show allpcpu db> ps db> alltrace db> show alllock - Save them with a serial console output or using the textdump(4) format. (if necessary read the ddb(4) and textdump(4) before to set up the whole system). This would shade a light if the problem lives within the kernel or not. Thanks, Attilio -- Peace can only be achieved by understanding - A. Einstein