From owner-freebsd-stable@freebsd.org Mon Oct 5 22:17:26 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6CFD39C25B1 for ; Mon, 5 Oct 2015 22:17:26 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 096DE8B2; Mon, 5 Oct 2015 22:17:25 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:MdEqXx9e5I58Yv9uRHKM819IXTAuvvDOBiVQ1KB91OkcTK2v8tzYMVDF4r011RmSDdmdtaMP07uempujcFJDyK7JiGoFfp1IWk1NouQttCtkPvS4D1bmJuXhdS0wEZcKflZk+3amLRodQ56mNBXsq3G/pQQfBg/4fVIsYL+lRsiM3o/vhqibwN76XUZhvHKFe7R8LRG7/036l/I9ps9cEJs30QbDuXBSeu5blitCLFOXmAvgtI/rpMYwuwwZgf8q9tZBXKPmZOx4COUAVHV1e1wysebrrxjYUQqG4DM/SGwKgBdSS1zO7znxXYz2vDe8uucriweAOsijd7E/WnyH5qxoTBLtwHMdMjcy82Xaj+Rti61GrRa5p1p0ytiHM8muKPNic/aFLpshTm1bU5MUDnQZDw== X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2DLAQCs9hJW/61jaINeg3tuBr4OAQ2BWhcKhS9KAoFyFAEBAQEBAQEBgQmCHYIHAQEBAwEBAQEgKyALBQkCAgEIDgoCAg0ZAgIbDAEJGAENAgQIBwQBHASIBQgNqXyUHgEBAQEBAQEDAQEBAQEBAQEBARgEgR6FUYR+hCoLBQIBHDQHgi47EoExBZV+hReFGIQ+SIcUkjECHwEBQoQeIjMBBoZwQoEGAQEB X-IronPort-AV: E=Sophos;i="5.17,640,1437451200"; d="scan'208";a="242874778" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-annu.net.uoguelph.ca with ESMTP; 05 Oct 2015 18:16:55 -0400 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id A974715F55D; Mon, 5 Oct 2015 18:16:55 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 85kX8puYo9FS; Mon, 5 Oct 2015 18:16:54 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 84FA315F563; Mon, 5 Oct 2015 18:16:54 -0400 (EDT) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id Iy7MQOWFCyAP; Mon, 5 Oct 2015 18:16:54 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 6B0AC15F55D; Mon, 5 Oct 2015 18:16:54 -0400 (EDT) Date: Mon, 5 Oct 2015 18:16:54 -0400 (EDT) From: Rick Macklem To: Christian Kratzer Cc: freebsd-stable@freebsd.org, John Baldwin Message-ID: <1721669289.24365403.1444083414400.JavaMail.zimbra@uoguelph.ca> In-Reply-To: References: Subject: Re: smbfs crashes since approx. 10.1-RELEASE MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.12] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF34 (Win)/8.0.9_GA_6191) Thread-Topic: smbfs crashes since approx. 10.1-RELEASE Thread-Index: 9GB0f/hsrH9dvWLWGo36F6O7ZxK6Aw== X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Oct 2015 22:17:26 -0000 Christian Kratzer wrote: > Hi, > > I run a regular rsync job that runs from cron and copies stuff that gets > created on a Windows smbfs share. > > Starting about 10.1-RELEASE the VM has become unstable and started panicing. > > I have narrowed the issue down to the aforementioned rsync job. > > When I move the job to a different VM the the other VM starts crashing and > the VM without the job becomes stable agin. > > I have panics and crashinfos stored in /var/crash if anybody is interested: > > root@noc2:/var/crash # uname -a > FreeBSD noc2.cksoft.de 10.2-RELEASE FreeBSD 10.2-RELEASE #0 r286666: Wed > Aug 12 15:26:37 UTC 2015 > root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 > root@noc2:/var/crash # freebsd-version -u > 10.2-RELEASE-p5 > root@noc2:/var/crash # freebsd-version -k > 10.2-RELEASE > root@noc2:/var/crash # > > This is what I have in /var/crash/core.txt.0 > > Fatal trap 12: page fault while in kernel mode > cpuid = 0; apic id = 00 > fault virtual address = 0x20 > fault code = supervisor read data, page not present > instruction pointer = 0x20:0xffffffff80996c7c > stack pointer = 0x28:0xfffffe003d6c0ac0 > frame pointer = 0x28:0xfffffe003d6c0af0 > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags = resume, IOPL = 0 > current process = 1349 (smbiod10) > trap number = 12 > panic: page fault > cpuid = 0 > KDB: stack backtrace: > #0 0xffffffff80984e30 at kdb_backtrace+0x60 > #1 0xffffffff809489e6 at vpanic+0x126 > #2 0xffffffff809488b3 at panic+0x43 > #3 0xffffffff80d4aadb at trap_fatal+0x36b > #4 0xffffffff80d4addd at trap_pfault+0x2ed > #5 0xffffffff80d4a47a at trap+0x47a > #6 0xffffffff80d307f2 at calltrap+0x8 > #7 0xffffffff8092ebe0 at __mtx_unlock_sleep+0x60 > #8 0xffffffff8092eb69 at __mtx_unlock_flags+0x69 > #9 0xffffffff81a1b724 at smb_iod_thread+0xb4 > #10 0xffffffff8091244a at fork_exit+0x9a > #11 0xffffffff80d30d2e at fork_trampoline+0xe > Uptime: 2h43m55s > Dumping 103 out of 999 MB: (CTRL-C to abort) > ..16%..31%..47%..62%..78%..93% > This crash is occurring when doing an mtx_unlock(&Giant). Unfortunately, I'm not conversant w.r.t. this code. I've cc'd jhb@ in case he has some insight. If you don't get any responses, I'd suggest reposting to freebsd-current@ with "crashes in mtx_unlock(&Giant)" in the subject line. Btw John, the code does tsleep() in a loop before the mtx_unlock(&Giant). I do remember that was once allowed, but am not sure if it still is (ie a tsleep() call while holding Giant)? Hopefully someone who knows what is special about Giant that might cause this will respond. Good luck with it, rick > Reading symbols from /boot/kernel/smbfs.ko.symbols...done. > Loaded symbols for /boot/kernel/smbfs.ko.symbols > Reading symbols from /boot/kernel/libiconv.ko.symbols...done. > Loaded symbols for /boot/kernel/libiconv.ko.symbols > Reading symbols from /boot/kernel/libmchain.ko.symbols...done. > Loaded symbols for /boot/kernel/libmchain.ko.symbols > #0 doadump (textdump=) at pcpu.h:219 > 219 pcpu.h: No such file or directory. > in pcpu.h > (kgdb) #0 doadump (textdump=) at pcpu.h:219 > #1 0xffffffff80948642 in kern_reboot (howto=260) > at /usr/src/sys/kern/kern_shutdown.c:451 > #2 0xffffffff80948a25 in vpanic (fmt=, > ap=) at /usr/src/sys/kern/kern_shutdown.c:758 > #3 0xffffffff809488b3 in panic (fmt=0x0) > at /usr/src/sys/kern/kern_shutdown.c:687 > #4 0xffffffff80d4aadb in trap_fatal (frame=, > eva=) at /usr/src/sys/amd64/amd64/trap.c:851 > #5 0xffffffff80d4addd in trap_pfault (frame=0xfffffe003d6c0a10, > usermode=) at /usr/src/sys/amd64/amd64/trap.c:674 > #6 0xffffffff80d4a47a in trap (frame=0xfffffe003d6c0a10) > at /usr/src/sys/amd64/amd64/trap.c:440 > #7 0xffffffff80d307f2 in calltrap () > at /usr/src/sys/amd64/amd64/exception.S:236 > #8 0xffffffff80996c7c in turnstile_broadcast (ts=0x0, queue=0) > at /usr/src/sys/kern/subr_turnstile.c:838 > #9 0xffffffff8092ebe0 in __mtx_unlock_sleep (c=0xfffff80002df0390, > opts=, > file=0xffffffff81a25801 "%s: Can't handle disordered parameters %d:%d\n", > line=1) at /usr/src/sys/kern/kern_mutex.c:791 > #10 0xffffffff8092eb69 in __mtx_unlock_flags (c=, > opts=, > file=0xffffffff81a25801 "%s: Can't handle disordered parameters %d:%d\n", > line=1) at /usr/src/sys/kern/kern_mutex.c:254 > #11 0xffffffff81a1b724 in smb_iod_thread (arg=0xfffff80002df0300) > at /usr/src/sys/modules/smbfs/../../netsmb/smb_iod.c:624 > #12 0xffffffff8091244a in fork_exit ( > callout=0xffffffff81a1b670 , arg=0xfffff80002df0300, > frame=0xfffffe003d6c0c00) at /usr/src/sys/kern/kern_fork.c:1018 > #13 0xffffffff80d30d2e in fork_trampoline () > at /usr/src/sys/amd64/amd64/exception.S:611 > #14 0x0000000000000000 in ?? () > Current language: auto; currently minimal > (kgdb) > > > Any tips on howto proceeed with this ? > > I can setup a dedicated VM with a fresh install of whatever in case it helps. > > Greetings > Christian > > -- > Christian Kratzer CK Software GmbH > Email: ck@cksoft.de Wildberger Weg 24/2 > Phone: +49 7032 893 997 - 0 D-71126 Gaeufelden > Fax: +49 7032 893 997 - 9 HRB 245288, Amtsgericht Stuttgart > Mobile: +49 171 1947 843 Geschaeftsfuehrer: Christian Kratzer > Web: http://www.cksoft.de/ > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" >