From owner-freebsd-stable@FreeBSD.ORG Fri Oct 31 07:54:59 2008 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 87C30106568B for ; Fri, 31 Oct 2008 07:54:59 +0000 (UTC) (envelope-from dominique.goncalves@gmail.com) Received: from wf-out-1314.google.com (wf-out-1314.google.com [209.85.200.173]) by mx1.freebsd.org (Postfix) with ESMTP id 5C2BC8FC17 for ; Fri, 31 Oct 2008 07:54:59 +0000 (UTC) (envelope-from dominique.goncalves@gmail.com) Received: by wf-out-1314.google.com with SMTP id 24so1058675wfg.7 for ; Fri, 31 Oct 2008 00:54:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:cc:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references; bh=95TsQmVM9dCjULKl2UGRicgNstuuyxFyp605XLdrH5I=; b=PL81LTldAeg+PWJRRDNs78k2l1SUl/PpDpWUB1x4dxPnOEgAT9htCpJVu2dkALIPyC NmLyWP6qFzO/+AitBQH4DQ9m4pOapGEEoS7XMir4w1PCWusaeDJdbeQs63eeS2J+3fQk DCitOZhPdISGIrHKjrkwFeGj5xf+gTMGgxeWA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references; b=ULfIJK6JuHyTK8mEueYV055gfXBWFM0cHgsAE62l2zJmaz7Gwm8v7ubyRh++Jk6CSC GDDzZthKdfexp/431bznV+X6Yeq01HKNXhJLe3SVuAnxt5MW27aqsDREmZMMe+rOT0iB V4/c2qZCcNMEBZO7s1LCSsw/d4dOSV3m7wzSA= Received: by 10.143.2.19 with SMTP id e19mr5235206wfi.90.1225439698933; Fri, 31 Oct 2008 00:54:58 -0700 (PDT) Received: by 10.142.246.14 with HTTP; Fri, 31 Oct 2008 00:54:58 -0700 (PDT) Message-ID: <7daacbbe0810310054p73bec34eu9497f4d036b5d648@mail.gmail.com> Date: Fri, 31 Oct 2008 08:54:58 +0100 From: "Dominique Goncalves" To: "=?ISO-8859-1?Q?V=E1clav_Haisman?=" In-Reply-To: <49097F1F.7060109@sh.cvut.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline References: <7daacbbe0810290217l60236d8ci1b2c4011720a0cc8@mail.gmail.com> <49097F1F.7060109@sh.cvut.cz> Cc: freebsd-stable@freebsd.org Subject: Re: Perl application hang in umtxn state X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 31 Oct 2008 07:54:59 -0000 Hi On Thu, Oct 30, 2008 at 10:32 AM, V=E1clav Haisman w= rote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > Dominique Goncalves wrote: >> Hi, >> >> I run FreeBSD 7.1-PRERELEASE and with a Perl GTK2 application, Perl >> hang in umtxn state: >> >> PID USERNAME THR PRI NICE SIZE RES STATE TIME WCPU COMMAN= D >> 76288 dom 1 53 0 67860K 46080K umtxn 0:02 0.00% perl5.= 8.8 >> >> And the only way to stop the application is kill -9 >> >> I tried to use ktrace/kdump and ktrace.out show "_umtx_op" >> >> $ kdump -f ktrace.out >> [...] >> 13199 perl5.8.8 RET lstat 0 >> 13199 perl5.8.8 CALL gettimeofday(0xbfbfb144,0) >> 13199 perl5.8.8 RET gettimeofday 0 >> 13199 perl5.8.8 CALL _umtx_op(0xbfbfd18c,0x3,0x1,0,0) >> 13199 perl5.8.8 RET _umtx_op 0 >> 13199 perl5.8.8 CALL sigprocmask(SIG_BLOCK,0xbfbfd120,0x8121490) >> 13199 perl5.8.8 RET sigprocmask 0 >> 13199 perl5.8.8 CALL _umtx_op(0x287fb8a0,0x5,0,0,0) >> 13199 perl5.8.8 RET _umtx_op RESTART >> 13199 perl5.8.8 PSIG SIGKILL SIG_DFL >> >> >> How to debug this hang? (btw I'm not a developer) > You can try rebuilding perl with debugging information. I am not sure > this is the best way but I have used this in my /etc/make.conf: > > # Disable binaries stripping. > STRIP=3D > # Add debugging info to binaries. > CFLAGS?=3D > CFLAGS+=3D -g3 -ggdb > CXXFLAGS?=3D > CXXFLAGS+=3D -g3 -ggdb Ok, I've recompiled Perl and others applications with these options. > When you have that, try running your test case again. When it blocks > again, attach GDB to the hung process: gdb -p PID `which perl`. And then > obtain the call stack: (gdb) bt. # gdb -p 2528 `which perl` GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you ar= e welcome to change it and/or distribute copies of it under certain condition= s. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd"...(no debugging symbols found)... Attaching to program: /usr/bin/perl, process 2528 Reading symbols from /usr/local/lib/perl5/5.8.8/mach/CORE/libperl.so...done= . Loaded symbols for /usr/local/lib/perl5/5.8.8/mach/CORE/libperl.so Reading symbols from /lib/libm.so.5...done. Loaded symbols for /lib/libm.so.5 Reading symbols from /lib/libcrypt.so.4...done. Loaded symbols for /lib/libcrypt.so.4 Reading symbols from /lib/libutil.so.7...done. Loaded symbols for /lib/libutil.so.7 Reading symbols from /lib/libc.so.7...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /usr/local/lib/perl5/site_perl/5.8.8/mach/auto/Cwd/Cwd.so...done. Loaded symbols for /usr/local/lib/perl5/site_perl/5.8.8/mach/auto/Cwd/Cwd.s= o Reading symbols from /usr/local/lib/perl5/5.8.8/mach/auto/POSIX/POSIX.so...= done. Loaded symbols for /usr/local/lib/perl5/5.8.8/mach/auto/POSIX/POSIX.so Reading symbols from /usr/local/lib/perl5/site_perl/5.8.8/mach/auto/Glib/Glib.so...done. Loaded symbols for /usr/local/lib/perl5/site_perl/5.8.8/mach/auto/Glib/Glib= .so Reading symbols from /usr/local/lib/libgobject-2.0.so.0...done. Loaded symbols for /usr/local/lib/libgobject-2.0.so.0 Reading symbols from /usr/local/lib/libglib-2.0.so.0...done. Loaded symbols for /usr/local/lib/libglib-2.0.so.0 Reading symbols from /usr/local/lib/libintl.so.8...done. Loaded symbols for /usr/local/lib/libintl.so.8 Reading symbols from /usr/local/lib/libiconv.so.3...done. Loaded symbols for /usr/local/lib/libiconv.so.3 Reading symbols from /usr/local/lib/libpcre.so.0...done. Loaded symbols for /usr/local/lib/libpcre.so.0 Reading symbols from /usr/local/lib/perl5/site_perl/5.8.8/mach/auto/Cairo/Cairo.so...done. Loaded symbols for /usr/local/lib/perl5/site_perl/5.8.8/mach/auto/Cairo/Cai= ro.so Reading symbols from /usr/local/lib/libcairo.so.2...done. Loaded symbols for /usr/local/lib/libcairo.so.2 Reading symbols from /usr/local/lib/libfreetype.so.9...done. Loaded symbols for /usr/local/lib/libfreetype.so.9 Reading symbols from /lib/libz.so.4...done. Loaded symbols for /lib/libz.so.4 Reading symbols from /usr/local/lib/libfontconfig.so.1...done. Loaded symbols for /usr/local/lib/libfontconfig.so.1 Reading symbols from /usr/local/lib/libpixman-1.so.9...done. Loaded symbols for /usr/local/lib/libpixman-1.so.9 Reading symbols from /usr/local/lib/libpng.so.5...done. Loaded symbols for /usr/local/lib/libpng.so.5 Reading symbols from /usr/local/lib/libxcb-render-util.so.0...done. Loaded symbols for /usr/local/lib/libxcb-render-util.so.0 Reading symbols from /usr/local/lib/libxcb-render.so.0...done. Loaded symbols for /usr/local/lib/libxcb-render.so.0 Reading symbols from /usr/local/lib/libxcb.so.1...done. Loaded symbols for /usr/local/lib/libxcb.so.1 Reading symbols from /usr/local/lib/libXrender.so.1...done. Loaded symbols for /usr/local/lib/libXrender.so.1 Reading symbols from /usr/local/lib/libX11.so.6...done. Loaded symbols for /usr/local/lib/libX11.so.6 Reading symbols from /lib/libthr.so.3...done. [New Thread 0x8125400 (LWP 100093)] Loaded symbols for /lib/libthr.so.3 Reading symbols from /usr/local/lib/libexpat.so.6...done. Loaded symbols for /usr/local/lib/libexpat.so.6 Reading symbols from /usr/local/lib/libXau.so.6...done. Loaded symbols for /usr/local/lib/libXau.so.6 Reading symbols from /usr/local/lib/libXdmcp.so.6...done. Loaded symbols for /usr/local/lib/libXdmcp.so.6 Reading symbols from /usr/lib/librpcsvc.so.4...done. Loaded symbols for /usr/lib/librpcsvc.so.4 Reading symbols from /usr/local/lib/perl5/site_perl/5.8.8/mach/auto/Gtk2/Gtk2.so...done. Loaded symbols for /usr/local/lib/perl5/site_perl/5.8.8/mach/auto/Gtk2/Gtk2= .so Reading symbols from /usr/local/lib/libgtk-x11-2.0.so.0...done. Loaded symbols for /usr/local/lib/libgtk-x11-2.0.so.0 Reading symbols from /usr/local/lib/libgdk-x11-2.0.so.0...done. Loaded symbols for /usr/local/lib/libgdk-x11-2.0.so.0 Reading symbols from /usr/local/lib/libatk-1.0.so.0...done. Loaded symbols for /usr/local/lib/libatk-1.0.so.0 Reading symbols from /usr/local/lib/libgdk_pixbuf-2.0.so.0...done. Loaded symbols for /usr/local/lib/libgdk_pixbuf-2.0.so.0 Reading symbols from /usr/local/lib/libpangocairo-1.0.so.0...done. Loaded symbols for /usr/local/lib/libpangocairo-1.0.so.0 Reading symbols from /usr/local/lib/libgio-2.0.so.0...done. Loaded symbols for /usr/local/lib/libgio-2.0.so.0 Reading symbols from /usr/local/lib/libXext.so.6...done. Loaded symbols for /usr/local/lib/libXext.so.6 Reading symbols from /usr/local/lib/libXinerama.so.1...done. Loaded symbols for /usr/local/lib/libXinerama.so.1 Reading symbols from /usr/local/lib/libXi.so.6...done. Loaded symbols for /usr/local/lib/libXi.so.6 Reading symbols from /usr/local/lib/libXrandr.so.2...done. Loaded symbols for /usr/local/lib/libXrandr.so.2 Reading symbols from /usr/local/lib/libXcursor.so.1...done. Loaded symbols for /usr/local/lib/libXcursor.so.1 Reading symbols from /usr/local/lib/libXcomposite.so.1...done. Loaded symbols for /usr/local/lib/libXcomposite.so.1 Reading symbols from /usr/local/lib/libXdamage.so.1...done. Loaded symbols for /usr/local/lib/libXdamage.so.1 Reading symbols from /usr/local/lib/libpangoft2-1.0.so.0...done. Loaded symbols for /usr/local/lib/libpangoft2-1.0.so.0 Reading symbols from /usr/local/lib/libXfixes.so.3...done. Loaded symbols for /usr/local/lib/libXfixes.so.3 Reading symbols from /usr/local/lib/libpango-1.0.so.0...done. Loaded symbols for /usr/local/lib/libpango-1.0.so.0 Reading symbols from /usr/local/lib/libgmodule-2.0.so.0...done. Loaded symbols for /usr/local/lib/libgmodule-2.0.so.0 Reading symbols from /usr/local/lib/perl5/5.8.8/mach/auto/IO/IO.so...done. Loaded symbols for /usr/local/lib/perl5/5.8.8/mach/auto/IO/IO.so Reading symbols from /usr/local/lib/perl5/5.8.8/mach/auto/Fcntl/Fcntl.so...= done. Loaded symbols for /usr/local/lib/perl5/5.8.8/mach/auto/Fcntl/Fcntl.so Reading symbols from /usr/local/lib/perl5/5.8.8/mach/auto/File/Glob/Glob.so...done. Loaded symbols for /usr/local/lib/perl5/5.8.8/mach/auto/File/Glob/Glob.so Reading symbols from /usr/local/lib/perl5/5.8.8/mach/auto/Storable/Storable.so...done. Loaded symbols for /usr/local/lib/perl5/5.8.8/mach/auto/Storable/Storable.s= o Reading symbols from /usr/local/lib/perl5/5.8.8/mach/auto/List/Util/Util.so...done. Loaded symbols for /usr/local/lib/perl5/5.8.8/mach/auto/List/Util/Util.so Reading symbols from /usr/local/lib/perl5/site_perl/5.8.8/mach/auto/DateTime/DateTime.so...done. Loaded symbols for /usr/local/lib/perl5/site_perl/5.8.8/mach/auto/DateTime/DateTime.so Reading symbols from /usr/local/lib/perl5/site_perl/5.8.8/mach/auto/Params/Validate/Validate.so.= ..done. Loaded symbols for /usr/local/lib/perl5/site_perl/5.8.8/mach/auto/Params/Validate/Validate.so Reading symbols from /usr/local/lib/perl5/site_perl/5.8.8/mach/auto/List/MoreUtils/MoreUtils.so.= ..done. Loaded symbols for /usr/local/lib/perl5/site_perl/5.8.8/mach/auto/List/MoreUtils/MoreUtils.so Reading symbols from /usr/local/lib/perl5/site_perl/5.8.8/mach/auto/Gtk2/Spell/Spell.so...done. Loaded symbols for /usr/local/lib/perl5/site_perl/5.8.8/mach/auto/Gtk2/Spell/Spell.so Reading symbols from /usr/local/lib/libgtkspell.so.0...done. Loaded symbols for /usr/local/lib/libgtkspell.so.0 Reading symbols from /usr/local/lib/libenchant.so.1...done. Loaded symbols for /usr/local/lib/libenchant.so.1 Reading symbols from /usr/local/lib/perl5/site_perl/5.8.8/mach/auto/Gnome2/VFS/VFS.so...done. Loaded symbols for /usr/local/lib/perl5/site_perl/5.8.8/mach/auto/Gnome2/VFS/VFS.so Reading symbols from /usr/local/lib/libgnomevfs-2.so.0...done. Loaded symbols for /usr/local/lib/libgnomevfs-2.so.0 Reading symbols from /usr/local/lib/libgconf-2.so.4...done. Loaded symbols for /usr/local/lib/libgconf-2.so.4 Reading symbols from /usr/local/lib/libgthread-2.0.so.0...done. Loaded symbols for /usr/local/lib/libgthread-2.0.so.0 Reading symbols from /usr/local/lib/libxml2.so.5...done. Loaded symbols for /usr/local/lib/libxml2.so.5 Reading symbols from /usr/local/lib/libdbus-glib-1.so.2...done. Loaded symbols for /usr/local/lib/libdbus-glib-1.so.2 Reading symbols from /usr/local/lib/libdbus-1.so.3...done. Loaded symbols for /usr/local/lib/libdbus-1.so.3 Reading symbols from /usr/lib/libssl.so.5...done. Loaded symbols for /usr/lib/libssl.so.5 Reading symbols from /lib/libcrypto.so.5...done. Loaded symbols for /lib/libcrypto.so.5 Reading symbols from /usr/local/lib/libavahi-glib.so.1...done. Loaded symbols for /usr/local/lib/libavahi-glib.so.1 Reading symbols from /usr/local/lib/libavahi-common.so.3...done. Loaded symbols for /usr/local/lib/libavahi-common.so.3 Reading symbols from /usr/local/lib/libavahi-client.so.3...done. Loaded symbols for /usr/local/lib/libavahi-client.so.3 Reading symbols from /usr/local/lib/libORBit-2.so.0...done. Loaded symbols for /usr/local/lib/libORBit-2.so.0 Reading symbols from /lib/libssp.so.0...done. Loaded symbols for /lib/libssp.so.0 Reading symbols from /usr/local/lib/perl5/site_perl/5.8.8/mach/auto/HTML/Parser/Parser.so...done= . Loaded symbols for /usr/local/lib/perl5/site_perl/5.8.8/mach/auto/HTML/Parser/Parser.so Reading symbols from /usr/local/lib/perl5/5.8.8/mach/auto/Encode/Encode.so...done. Loaded symbols for /usr/local/lib/perl5/5.8.8/mach/auto/Encode/Encode.so Reading symbols from /usr/local/lib/perl5/5.8.8/mach/auto/Socket/Socket.so...done. Loaded symbols for /usr/local/lib/perl5/5.8.8/mach/auto/Socket/Socket.so Reading symbols from /usr/local/lib/perl5/site_perl/5.8.8/mach/auto/XML/Parser/Expat/Expat.so...= done. Loaded symbols for /usr/local/lib/perl5/site_perl/5.8.8/mach/auto/XML/Parser/Expat/Expat.so Reading symbols from /usr/local/lib/perl5/5.8.8/mach/auto/Unicode/Normalize/Normalize.so...done. Loaded symbols for /usr/local/lib/perl5/5.8.8/mach/auto/Unicode/Normalize/Normalize.so Reading symbols from /usr/local/lib/perl5/site_perl/5.8.8/mach/auto/Compress/Raw/Zlib/Zlib.so...= done. Loaded symbols for /usr/local/lib/perl5/site_perl/5.8.8/mach/auto/Compress/Raw/Zlib/Zlib.so Reading symbols from /usr/local/lib/perl5/site_perl/5.8.8/mach/auto/Compress/Raw/Bzip2/Bzip2.so.= ..done. Loaded symbols for /usr/local/lib/perl5/site_perl/5.8.8/mach/auto/Compress/Raw/Bzip2/Bzip2.so Reading symbols from /usr/local/lib/perl5/5.8.8/mach/auto/MIME/Base64/Base64.so...done. Loaded symbols for /usr/local/lib/perl5/5.8.8/mach/auto/MIME/Base64/Base64.= so Reading symbols from /usr/local/lib/perl5/5.8.8/mach/auto/Digest/MD5/MD5.so...done. Loaded symbols for /usr/local/lib/perl5/5.8.8/mach/auto/Digest/MD5/MD5.so Reading symbols from /usr/local/lib/perl5/5.8.8/mach/auto/Filter/Util/Call/Call.so...done. Loaded symbols for /usr/local/lib/perl5/5.8.8/mach/auto/Filter/Util/Call/Ca= ll.so Reading symbols from /usr/local/lib/gtk-2.0/2.10.0/loaders/libpixbufloader-png.so...done. Loaded symbols for /usr/local/lib/gtk-2.0/2.10.0/loaders/libpixbufloader-pn= g.so Reading symbols from /usr/local/lib/pango/1.6.0/modules/pango-basic-fc.so...done. Loaded symbols for /usr/local/lib/pango/1.6.0/modules/pango-basic-fc.so Reading symbols from /usr/local/lib/enchant/libenchant_aspell.so...done. Loaded symbols for /usr/local/lib/enchant/libenchant_aspell.so Reading symbols from /usr/local/lib/libaspell.so.16...done. Loaded symbols for /usr/local/lib/libaspell.so.16 Reading symbols from /usr/lib/libstdc++.so.6...done. Loaded symbols for /usr/lib/libstdc++.so.6 Reading symbols from /lib/libgcc_s.so.1...done. Loaded symbols for /lib/libgcc_s.so.1 Reading symbols from /usr/local/lib/enchant/libenchant_ispell.so...done. Loaded symbols for /usr/local/lib/enchant/libenchant_ispell.so Reading symbols from /usr/local/lib/enchant/libenchant_myspell.so...done. Loaded symbols for /usr/local/lib/enchant/libenchant_myspell.so Reading symbols from /usr/local/lib/enchant/libenchant_zemberek.so...done. Loaded symbols for /usr/local/lib/enchant/libenchant_zemberek.so Reading symbols from /usr/local/lib/gio/modules/libgiofam.so...done. Loaded symbols for /usr/local/lib/gio/modules/libgiofam.so Reading symbols from /usr/local/lib/libfam.so.0...done. Loaded symbols for /usr/local/lib/libfam.so.0 Reading symbols from /libexec/ld-elf.so.1...done. Loaded symbols for /libexec/ld-elf.so.1 [Switching to Thread 0x8125400 (LWP 100093)] 0x282269a9 in _umtx_op () from /lib/libc.so.7 (gdb) bt #0 0x282269a9 in _umtx_op () from /lib/libc.so.7 #1 0x28818e5d in __error () from /lib/libthr.so.3 #2 0x2881b8a0 in _thread_state_running () from /lib/libthr.so.3 #3 0x00000005 in ?? () #4 0x00000000 in ?? () #5 0x00000000 in ?? () #6 0x00000000 in ?? () #7 0x2881a96c in ?? () from /lib/libthr.so.3 #8 0xbfbfcde8 in ?? () #9 0x288150cb in pthread_mutex_destroy () from /lib/libthr.so.3 Previous frame identical to this frame (corrupt stack?) > I suspect there will be some pthreads call on top of the stack. I am not > FreeBSD developer but this seems really similar to situations reported > in thread "umtxn and Apache 2.2" and mine "Process in "uwait" state". > >> >> $ uname -a >> FreeBSD freebsd7 7.1-PRERELEASE FreeBSD 7.1-PRERELEASE #15 r184123: >> Tue Oct 21 16:24:59 CEST 2008 >> root@freebsd7:/usr/obj/usr/src/sys/GENERIC i386 >> >> Thanks for your help, >> Regards. >> > > - -- > VH > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.9 (MingW32) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iFYEAREIAAYFAkkJfx8ACgkQhQBMvHf/WHnzuwDffY0A82o1mpzdVBixQ6bjeqon > pLikFIEvlXke3ADeIGmhTjnC5TPnJE01XiiZyPuOcrrgYpXhFzTdPw=3D=3D > =3DKOCn > -----END PGP SIGNATURE----- > Regards. --=20 There's this old saying: "Give a man a fish, feed him for a day. Teach a man to fish, feed him for life."