From owner-freebsd-stable@freebsd.org Tue Oct 13 02:47:42 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 762FDA10600 for ; Tue, 13 Oct 2015 02:47:42 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id F42091CDC; Tue, 13 Oct 2015 02:47:41 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:VaUYCh/6QOdEPv9uRHKM819IXTAuvvDOBiVQ1KB91O8cTK2v8tzYMVDF4r011RmSDdmdu6gP0bCN+4nbGkU+or+5+EgYd5JNUxJXwe43pCcHRPC/NEvgMfTxZDY7FskRHHVs/nW8LFQHUJ2mPw6anHS+4HYoFwnlMkItf6KuStKU3578jbrps7ToICx2xxOFKYtoKxu3qQiD/uI3uqBFbpgL9x3Sv3FTcP5Xz247bXianhL7+9vitMU7q3cYk7sb+sVBSaT3ebgjBfwdVWx+cjN92Mq+nhnZTBCT53IaGkYMmwZaDhKNuBr5dpzyqSz0qqxx1X/JE9fxSOUOWD+hp4JiQxzshSJPYyQ8+WrUjsF1pL9crw+sowR/hYXdNtLGfMFid7/QKItJDVFKWdxcAmkYWtux X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2BTAwBFcBxW/61jaINVCIN6bga+Bw6BWhcKgnKCCjVKAoFwFAEBAQEBAQEBgQmCH4IHAQEBAwEBAQEgBCcgCwUJAgIBCA4KERkCAgIZDAEJJgIECAcEARwEiAUIDaxbk1sBAQEBAQEBAQEBAQEBAQEBAQEBGQSGb4R+hCoMBAIBGwEZGweCaYFFBYc6hwWHVYJOgkuFGYRASJV0g20CHwFDghEdgXAiMweFZIEGAQEB X-IronPort-AV: E=Sophos;i="5.17,676,1437451200"; d="scan'208";a="244150647" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-annu.net.uoguelph.ca with ESMTP; 12 Oct 2015 22:47:39 -0400 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 0B73E15F55D; Mon, 12 Oct 2015 22:47:40 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id VmxkLXYEU6ON; Mon, 12 Oct 2015 22:47:39 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 16F7715F565; Mon, 12 Oct 2015 22:47:39 -0400 (EDT) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id lgUcKKPbZeOv; Mon, 12 Oct 2015 22:47:38 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id EA40915F55D; Mon, 12 Oct 2015 22:47:38 -0400 (EDT) Date: Mon, 12 Oct 2015 22:47:38 -0400 (EDT) From: Rick Macklem To: Christian Kratzer Cc: freebsd-stable@freebsd.org, John Baldwin Message-ID: <173739656.33429352.1444704458926.JavaMail.zimbra@uoguelph.ca> In-Reply-To: References: <3563189.eDHDcCgW5L@ralph.baldwin.cx> <358885214.31305796.1444518367048.JavaMail.zimbra@uoguelph.ca> <2135054744.32546564.1444653156980.JavaMail.zimbra@uoguelph.ca> Subject: Re: smbfs crashes since approx. 10.1-RELEASE MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_33429350_1097493246.1444704458924" X-Originating-IP: [172.17.95.11] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF34 (Win)/8.0.9_GA_6191) Thread-Topic: smbfs crashes since approx. 10.1-RELEASE Thread-Index: labyDeG/iqIuDF9w9rT8eAtMqRd0/w== X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Oct 2015 02:47:42 -0000 ------=_Part_33429350_1097493246.1444704458924 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Christian Kratzer wrote: > Hi Rick, > > On Mon, 12 Oct 2015, Rick Macklem wrote: > > > Christian Kratzer wrote: > >> Hi Rick, > >> > >> there was also a second more recent crash in /var/crash > >> > >> Mon Oct 12 03:01:16 CEST 2015 > >> > >> FreeBSD noc3.cksoft.de 10.2-STABLE FreeBSD 10.2-STABLE #2 r288980M: > >> Sun > >> Oct 11 08:37:40 CEST 2015 > >> ck@noc3.cksoft.de:/usr/obj/usr/src/sys/NOC amd64 > >> > >> panic: Assertion mtx_unowned(m) failed at > >> /usr/src/sys/kern/kern_mutex.c:955 > >> > > Oops, I screwed up. I should have looked at this panic assertion when you > > reported > > it before. Ok, so if I understand the assertion correctly, it means that > > another > > thread has the mutex locked. If this is correct, I'll have to take another > > look at > > the code and figure out how to wait for these other threads to finish with > > the mutexes. > > > > I do think the patch fixes the race I saw, but there must be other races in > > the code. > > > > I'll take another look, but if anyone else is conversant with netsmb, feel > > free to > > jump in, because it is all new to me. > > > > Unfortunately, I won't have any way to do testing for the next month or so, > > so any > > patches I do come up with will be "try this untested..". > > thats no problem. > > Just keep the patches coming when you have time and tell me when to reset > back to stable, > current or whatever so we don't lose sync of the status. > Well, you can try the attached one instead of the previous ones (ie. against stable). It just delays destroying the mutexes until the iod thread is exiting. I can't quite see why the previous patches wouldn't fix it, but this one leaves smb_iod_main() unchanged, so it is a simpler patch and doesn't affect semantics except for a slight delay in destroying the mutexes. > As it looks like that the race happens on unmount I could try putting a sleep > 60 into the > script that does the "mount && rsycn && umount" magic just before the umount. > That would > allow anything that it slow to go away to perhaps release the mutexes before > the umount. > If it still crashes with this patch, it might be worth a try. Or, if this patch still crashes, you could just delete the 3 lines that the patch moves, so the mutexes are never destroyed. This would result in a leak, but it would tell us if destroying these mutexes is the problem. Thanks for your willingness to test these, rick > Not a real fix of course but might help to verify what's going on. > > Greetings > Christian > > > -- > Christian Kratzer CK Software GmbH > Email: ck@cksoft.de Wildberger Weg 24/2 > Phone: +49 7032 893 997 - 0 D-71126 Gaeufelden > Fax: +49 7032 893 997 - 9 HRB 245288, Amtsgericht Stuttgart > Mobile: +49 171 1947 843 Geschaeftsfuehrer: Christian Kratzer > Web: http://www.cksoft.de/ > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > ------=_Part_33429350_1097493246.1444704458924 Content-Type: text/x-patch; name=smbiod2.patch Content-Disposition: attachment; filename=smbiod2.patch Content-Transfer-Encoding: base64 LS0tIHNtYl9pb2QuYy5vcmlnCTIwMTUtMTAtMTAgMTg6NTM6MzQuMDAwMDAwMDAwIC0wNDAwCisr KyBzbWJfaW9kLmMJMjAxNS0xMC0xMiAyMDozMDowMC4wMDAwMDAwMDAgLTA0MDAKQEAgLTY1OSw2 ICs2NTksMTEgQEAgc21iX2lvZF90aHJlYWQodm9pZCAqYXJnKQogCQkJYnJlYWs7CiAJCXRzbGVl cCgmaW9kLT5pb2RfZmxhZ3MsIFBXQUlULCAiOTBpZGxlIiwgaW9kLT5pb2Rfc2xlZXB0aW1vKTsK IAl9CisKKwkvKiBXZSBjYW4gbm93IHNhZmVseSBkZXN0cm95IHRoZSBtdXRleGVzIGFuZCBmcmVl IHRoZSBpb2Qgc3RydWN0dXJlLiAqLworCXNtYl9zbF9kZXN0cm95KCZpb2QtPmlvZF9ycWxvY2sp OworCXNtYl9zbF9kZXN0cm95KCZpb2QtPmlvZF9ldmxvY2spOworCWZyZWUoaW9kLCBNX1NNQklP RCk7CiAJbXR4X3VubG9jaygmR2lhbnQpOwogCWtwcm9jX2V4aXQoMCk7CiB9CkBAIC02OTUsOSAr NzAwLDYgQEAgaW50CiBzbWJfaW9kX2Rlc3Ryb3koc3RydWN0IHNtYmlvZCAqaW9kKQogewogCXNt Yl9pb2RfcmVxdWVzdChpb2QsIFNNQklPRF9FVl9TSFVURE9XTiB8IFNNQklPRF9FVl9TWU5DLCBO VUxMKTsKLQlzbWJfc2xfZGVzdHJveSgmaW9kLT5pb2RfcnFsb2NrKTsKLQlzbWJfc2xfZGVzdHJv eSgmaW9kLT5pb2RfZXZsb2NrKTsKLQlmcmVlKGlvZCwgTV9TTUJJT0QpOwogCXJldHVybiAwOwog fQogCg== ------=_Part_33429350_1097493246.1444704458924--