From owner-freebsd-threads@FreeBSD.ORG  Fri Sep 12 22:16:38 2008
Return-Path: <owner-freebsd-threads@FreeBSD.ORG>
Delivered-To: freebsd-threads@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id E82D9106567C
	for <freebsd-threads@freebsd.org>; Fri, 12 Sep 2008 22:16:38 +0000 (UTC)
	(envelope-from ivoras@gmail.com)
Received: from rv-out-0506.google.com (rv-out-0506.google.com [209.85.198.229])
	by mx1.freebsd.org (Postfix) with ESMTP id B924A8FC0A
	for <freebsd-threads@freebsd.org>; Fri, 12 Sep 2008 22:16:38 +0000 (UTC)
	(envelope-from ivoras@gmail.com)
Received: by rv-out-0506.google.com with SMTP id b25so1255910rvf.43
	for <freebsd-threads@freebsd.org>; Fri, 12 Sep 2008 15:16:38 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=domainkey-signature:received:received:message-id:date:from:sender
	:to:subject:cc:in-reply-to:mime-version:content-type
	:content-transfer-encoding:content-disposition:references
	:x-google-sender-auth;
	bh=uz5Vbw+yj3c6OmKB1oHx34Oz7GN1kw1nTCskgseqOSU=;
	b=A3M0fW6ZeNZFfEbYH01A0Yu5Q/JBLxYWE30Y0+8nlAMfB5zYtY7Fr76SRKnp5an9fC
	5+3VtrxXoMnq+MOHGrbz1CEgThrpXSAFANLREOzSNQeLY4hkku6gqGrTqo8FtPMdK12T
	ebrG68fI39Q0ryI8/rJlJcju+sWsztPxuTZdc=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma;
	h=message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version
	:content-type:content-transfer-encoding:content-disposition
	:references:x-google-sender-auth;
	b=GccV4QItdAw/AhkmhzPkJ2lsXm2Sd48paePPSZOLoJP2OK4wbCTDUsVxgpS9gAJACh
	2CbNc0i8vAK2ZTTFO2G8iM++ulSpVMrLZm4kjGjOLaZYIKcbBhuWyqdMrGUyaBgEPQiI
	DaOE0HwHAkiJkPM+8ucbQtMtA4jxnJwUWo544=
Received: by 10.141.137.6 with SMTP id p6mr2905146rvn.279.1221255891221;
	Fri, 12 Sep 2008 14:44:51 -0700 (PDT)
Received: by 10.141.153.13 with HTTP; Fri, 12 Sep 2008 14:44:51 -0700 (PDT)
Message-ID: <9bbcef730809121444u34991c52m2cbc01a8ada47eb5@mail.gmail.com>
Date: Fri, 12 Sep 2008 23:44:51 +0200
From: "Ivan Voras" <ivoras@freebsd.org>
Sender: ivoras@gmail.com
To: "Alfred Perlstein" <alfred@freebsd.org>
In-Reply-To: <20080912165808.GE16977@elvis.mu.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <gadgrb$4uc$1@ger.gmane.org> <20080912165808.GE16977@elvis.mu.org>
X-Google-Sender-Auth: 9219905d203f2681
Cc: freebsd-threads@freebsd.org
Subject: Re: Apache-worker stuck at 100% CPU
X-BeenThere: freebsd-threads@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Threading on FreeBSD <freebsd-threads.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-threads>, 
	<mailto:freebsd-threads-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-threads>
List-Post: <mailto:freebsd-threads@freebsd.org>
List-Help: <mailto:freebsd-threads-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-threads>,
	<mailto:freebsd-threads-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 12 Sep 2008 22:16:39 -0000

2008/9/12 Alfred Perlstein <alfred@freebsd.org>:

> Try using "pstack" a few times.  It's in ports.
>
> Also, gcore(1) might help.

Will try.

> You can also try to attach using gdb.

I did, but either I'm missing something or I'm not using it well,
because I can't get a backtrace. How do I select threads to backtrace?
How do I pick what thread to backtrace?

> Basically, one of these tools should give you a stack trace which
> can help.
>
> It's interesting that the process is in "umtxn" though, is it
> multithreaded apache?  Can you dump the threads?  I think top(1)
> has an option to view each thread, how about trying that?

Yes, it's multithreaded apache. This did help somewhat - when I do it
I see that it's not actually stuck in umtxn - there's one thread that
consumes the CPU and it's apparently always running (in state CPUx).

  PID USERNAME PRI NICE   SIZE    RES STATE  C   TIME   WCPU COMMAND
 7212 www      103    0 30340K  7932K CPU2   2 444:23 99.02% httpd

I'm currently upgrading the system to 7-STABLE, to see if it helps.