From owner-freebsd-current@freebsd.org Sun Jun 19 09:10:02 2016 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E6B76A79EB8 for ; Sun, 19 Jun 2016 09:10:02 +0000 (UTC) (envelope-from daniel.engberg.lists@pyret.net) Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net [IPv6:2001:4b98:c:538::195]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id ADEBC2040 for ; Sun, 19 Jun 2016 09:10:02 +0000 (UTC) (envelope-from daniel.engberg.lists@pyret.net) Received: from mfilter15-d.gandi.net (mfilter15-d.gandi.net [217.70.178.143]) by relay3-d.mail.gandi.net (Postfix) with ESMTP id 922ADA80CE; Sun, 19 Jun 2016 11:10:00 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at mfilter15-d.gandi.net Received: from relay3-d.mail.gandi.net ([IPv6:::ffff:217.70.183.195]) by mfilter15-d.gandi.net (mfilter15-d.gandi.net [::ffff:10.0.15.180]) (amavisd-new, port 10024) with ESMTP id MkcNN17GVwx6; Sun, 19 Jun 2016 11:09:58 +0200 (CEST) X-Originating-IP: 90.231.139.91 Received: from [192.168.1.249] (h91n1-gl-a-a31.ias.bredband.telia.com [90.231.139.91]) (Authenticated sender: daniel.engberg@pyret.net) by relay3-d.mail.gandi.net (Postfix) with ESMTPSA id 4F18AA80CD; Sun, 19 Jun 2016 11:09:58 +0200 (CEST) Subject: Re: Samba 4.3 and 4.4 crashes on FreeBSD 11-ALPHA4 To: Konstantin Belousov References: <0092113e-6ad6-575c-2127-f211ba9aef50@pyret.net> <20160619080644.GL38613@kib.kiev.ua> Cc: freebsd-current@freebsd.org From: Daniel Engberg Message-ID: Date: Sun, 19 Jun 2016 11:09:02 +0200 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 In-Reply-To: <20160619080644.GL38613@kib.kiev.ua> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Jun 2016 09:10:03 -0000 On 2016-06-19 10:06, Konstantin Belousov wrote: > On Sun, Jun 19, 2016 at 09:10:08AM +0200, Daniel Engberg wrote: >> Hi, >> >> Decided to try ALPHA4 and Samba 4.3 and 4.4 doesn't seem very happy. >> >> Fatal error 'mutex 0x8013ec000 own 0x18cdc is on list 0x8112161a0 0x0' at line 153 in file /usr/src/lib/libthr/thread/thr_mutex.c (errno = 4) >> [2016/06/19 01:14:47.689256, 0] ../lib/util/fault.c:78(fault_report) >> =============================================================== >> [2016/06/19 01:14:47.689450, 0] ../lib/util/fault.c:79(fault_report) >> INTERNAL ERROR: Signal 6 in pid 45222 (4.4.3) >> Please read the Trouble-Shooting section of the Samba HOWTO >> [2016/06/19 01:14:47.689479, 0] ../lib/util/fault.c:81(fault_report) >> =============================================================== >> [2016/06/19 01:14:47.689498, 0] ../source3/lib/util.c:791(smb_panic_s3) >> PANIC (pid 45222): internal error >> [2016/06/19 01:14:47.690577, 0] ../source3/lib/util.c:902(log_stack_trace) >> BACKTRACE: 6 stack frames: >> #0 0x803c16fe8 at /usr/local/lib/samba4/libsmbconf.so.0 >> #1 0x803c16ed2 at /usr/local/lib/samba4/libsmbconf.so.0 >> #2 0x801696dad at /usr/local/lib/samba4/libsamba-util.so.0 >> #3 0x801696c57 at /usr/local/lib/samba4/libsamba-util.so.0 >> #4 0x8014616df at /lib/libthr.so.3 >> #5 0x801460cbf at /lib/libthr.so.3 >> [2016/06/19 01:14:47.690686, 0] ../source3/lib/dumpcore.c:313(dump_core) >> unable to change to %N.core >> refusing to dump core >> >> Fatal error 'mutex 0x8013eb000 own 0x18d3d is on list 0x812e161a0 0x0' at line 153 in file /usr/src/lib/libthr/thread/thr_mutex.c (errno = 4) >> [2016/06/19 01:26:41.063681, 0] ../lib/util/fault.c:78(fault_report) >> =============================================================== >> [2016/06/19 01:26:41.063827, 0] ../lib/util/fault.c:79(fault_report) >> INTERNAL ERROR: Signal 6 in pid 61297 (4.3.9) >> Please read the Trouble-Shooting section of the Samba HOWTO >> [2016/06/19 01:26:41.063844, 0] ../lib/util/fault.c:81(fault_report) >> =============================================================== >> [2016/06/19 01:26:41.063855, 0] ../source3/lib/util.c:789(smb_panic_s3) >> PANIC (pid 61297): internal error >> [2016/06/19 01:26:41.064679, 0] ../source3/lib/util.c:900(log_stack_trace) >> BACKTRACE: 6 stack frames: >> #0 0x803846eb8 at /usr/local/lib/libsmbconf.so.0 >> #1 0x803846da2 at /usr/local/lib/libsmbconf.so.0 >> #2 0x80168c12d at /usr/local/lib/libsamba-util.so.0 >> #3 0x80168bfd7 at /usr/local/lib/libsamba-util.so.0 >> #4 0x8014606df at /lib/libthr.so.3 >> #5 0x80145fcbf at /lib/libthr.so.3 >> [2016/06/19 01:26:41.064747, 0] ../source3/lib/dumpcore.c:313(dump_core) >> unable to change to %N.core >> refusing to dump core >> >> This occurs immediately when you start Samba so it's easy to replicate. >> I know that 4.3 worked on r298887 for sure, I'm going to try this in VM too but I'd apperciate if anyone else could give this a go. >> >> I noticed that there were some changes in libthr which might be a clue.... >> > > Please convince samba to allow to dump core on assert (I have no idea how), > then get a backtrace from the core, using ports/devel/gdb. After that I > may know where to look next. > Hi, Thanks for the quick reply, I'm not familiar to gdb but from what I found this seems to be correct way. sysctl kern.corefile=/var/log/samba4/%N.core (for whatever reason %N.core only doesn't seem to make Samba happy) /usr/local/bin/gdb /usr/local/sbin/smbd /var/log/samba4/smbd.core GNU gdb (GDB) 7.11.1 [GDB v7.11.1 for FreeBSD] Copyright (C) 2016 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-portbld-freebsd11.0". Type "show configuration" for configuration details. For bug reporting instructions, please see: . Find the GDB manual and other documentation resources online at: . For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /usr/local/sbin/smbd...done. [New LWP 101688] Core was generated by `smbd'. Program terminated with signal SIGABRT, Aborted. #0 0x00000008055ed10a in ?? () (gdb) bt #0 0x00000008055ed10a in ?? () #1 0x00000008055ed0db in ?? () #2 0x0000000000018d38 in ?? () #3 0x8979efeec3bd9290 in ?? () #4 0x00007fffffffd2a4 in ?? () #5 0x00000008020f6020 in ?? () #6 0x00007fffffffd2c0 in ?? () #7 0x00000008055ed049 in ?? () #8 0x0000000000000000 in ?? () I guess that doesn't help much.... (debug option enabled in Samba port) Disabled following lines in /etc/make.conf and recompiled Samba # WITHOUT_DEBUG=YES # MK_PROFILE=no /usr/ports/net/samba44 # gdb /usr/local/sbin/smbd /var/log/samba4/smbd.core GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Core was generated by `smbd'. Program terminated with signal 6, Aborted. #0 0x0000000805ce110a in ?? () (gdb) bt #0 0x0000000805ce110a in ?? () #1 0x0000000805ce10db in ?? () #2 0x0000000000018d31 in ?? () #3 0x3a0c606c1a35133b in ?? () #4 0x00007fffffffce04 in ?? () #5 0x0000000000000006 in ?? () #6 0x00007fffffffce20 in ?? () #7 0x0000000805ce1049 in ?? () #8 0x0000000000000000 in ?? () I have the following in /etc/src.conf MALLOC_PRODUCTION=yes WITHOUT_DEBUG_FILES=1 WITHOUT_LIB32=1 WITHOUT_KERNEL_SYMBOLS=yes and disabled the following in kernel config #makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols #makeoptions WITH_CTF=1 # Run ctfconvert(1) for DTrace support # Debugging support. Always need this: #options KDB # Enable kernel debugger support. #options KDB_TRACE # Print a stack trace for a panic. # For full debugger support use (turn off in stable branch): #options DDB # Support DDB. #options GDB # Support remote GDB. #options DEADLKRES # Enable the deadlock resolver #options INVARIANTS # Enable calls of extra sanity checking #options INVARIANT_SUPPORT # Extra sanity checks of internal structures, required by INVARIANTS #options WITNESS # Enable checks to detect deadlocks and cycles #options WITNESS_SKIPSPIN # Don't run witness on spinlocks for speed #options MALLOC_DEBUG_MAXZONES=8 # Separate malloc(9) zones So I guess I need to turn all this on again and recompile everything *sigh*, I'll do that and in the meantime try to replicate this in a VM. Best regards, Daniel