From owner-freebsd-current@freebsd.org Thu Jul 5 19:44:06 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0B6BA1029289 for ; Thu, 5 Jul 2018 19:44:06 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from mail.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A91B473090 for ; Thu, 5 Jul 2018 19:44:05 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from John-Baldwins-MacBook-Pro-2.local (ralph.baldwin.cx [66.234.199.215]) by mail.baldwin.cx (Postfix) with ESMTPSA id 5358D10AFD2; Thu, 5 Jul 2018 15:44:04 -0400 (EDT) Subject: Re: atomic changes break drm-next-kmod? To: Konstantin Belousov , Hans Petter Selasky References: <845aca10-8c01-fa3b-087f-f957df4e7531@nomadlogic.org> <063ae5c3-0584-1284-dd9d-ab8b5790baf1@FreeBSD.org> <0bf8e57b-fdb4-4c1a-3d0d-a734f8187ca8@nomadlogic.org> <4c5411dd-9f6b-7245-6ade-e11040f74687@FreeBSD.org> <24f5d737-a205-6fcc-0a33-a84601d2ff7a@nomadlogic.org> <29ce4eab-6667-d2ca-b5d8-3deeef28f142@selasky.org> <20180705193646.GM5562@kib.kiev.ua> Cc: Pete Wright , Niclas Zeising , "O. Hartmann" , FreeBSD Current From: John Baldwin Message-ID: <5dc2a315-4b71-9ff0-0a37-576649e9144b@FreeBSD.org> Date: Thu, 5 Jul 2018 12:44:02 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: <20180705193646.GM5562@kib.kiev.ua> Content-Type: text/plain; charset=koi8-r Content-Language: en-US Content-Transfer-Encoding: 8bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mail.baldwin.cx); Thu, 05 Jul 2018 15:44:05 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.99.2 at mail.baldwin.cx X-Virus-Status: Clean X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jul 2018 19:44:06 -0000 On 7/5/18 12:36 PM, Konstantin Belousov wrote: > On Thu, Jul 05, 2018 at 09:12:24PM +0200, Hans Petter Selasky wrote: >> On 07/05/18 20:59, Hans Petter Selasky wrote: >>> On 07/05/18 19:48, Pete Wright wrote: >>>> >>>> >>>> On 07/05/2018 10:10, John Baldwin wrote: >>>>> On 7/3/18 5:10 PM, Pete Wright wrote: >>>>>> >>>>>> On 07/03/2018 15:56, John Baldwin wrote: >>>>>>> On 7/3/18 3:34 PM, Pete Wright wrote: >>>>>>>> On 07/03/2018 15:29, John Baldwin wrote: >>>>>>>>> That seems like kgdb is looking at the wrong CPU.š Can you use >>>>>>>>> 'info threads' and look for threads not stopped in 'sched_switch' >>>>>>>>> and get their backtraces?š You could also just do 'thread apply >>>>>>>>> all bt' and put that file at a URL if that is easiest. >>>>>>>>> >>>>>>>> sure thing John - here's a gist of "thread apply all bt" >>>>>>>> >>>>>>>> https://gist.github.com/gem-pete/d8d7ab220dc8781f0827f965f09d43ed >>>>>>> That doesn't look right at all.š Are you sure the kernel matches the >>>>>>> vmcore?š Also, which kgdb version are you using? >>>>>>> >>>>>> yea i agree that doesn't look right at all.š here is my setup: >>>>>> >>>>>> $ which kgdb >>>>>> /usr/bin/kgdb >>>>>> $ kgdb >>>>>> GNU gdb 6.1.1 [FreeBSD] >>>>>> $ ls -lh /var/crash/vmcore.1 >>>>>> -rw-------š 1 rootš wheelšš 1.6G Julš 3 15:03 /var/crash/vmcore.1 >>>>>> $ ls -l /usr/lib/debug/boot/kernel/kernel.debug >>>>>> -r-xr-xr-xš 1 rootš wheelš 87840496 Julš 3 13:54 >>>>>> /usr/lib/debug/boot/kernel/kernel.debug >>>>>> >>>>>> and i invoke kgdb like so: >>>>>> $ sudo kgdb /usr/lib/debug/boot/kernel/kernel.debug /var/crash/vmcore.1 >>>>>> >>>>>> here's a gist of my full gdb session: >>>>>> http://termbin.com/krsn >>>>>> >>>>>> dunno - maybe i have a bad core dump?š regardless, more than happy to >>>>>> help so let me know if i should try anything else or patches etc.. >>>>> Can you try installing gdb from ports and using /usr/local/bin/kgdb? >>>>> >>>> >>>> that seems to have done the trick, at least the output looks more >>>> encouraging. >>>> >>>> šš--- trap 0, rip = 0, rsp = 0, rbp = 0 --- >>>> KDB: enter: panic >>>> >>>> __curthread () at ./machine/pcpu.h:231 >>>> 231ššš ššš __asm("movq %%gs:%1,%0" : "=r" (td) >>>> >>>> >>>> here's my full kgdb session: >>>> http://termbin.com/qa4f >>>> >>>> i don't see any threads not in "sched_switch" though :( >>> >>> Hi, >>> >>> The problem may be that the patch to enable atomic inlining of all >>> macros forgot to set the SMP keyword which means SMP is not defined at >>> all for KLD's so all non-kernel atomic usage is with MPLOCKED empty! > Problem is that out-of-tree modules build does not have opt*.h files > from the kernel. UP config is a valid one, flipping some option's > default value does not solve the problem. Yes, but using the lock prefix in a generic module is ok (it will still work, just not quite as fast) whereas the lack of lock is fatal on SMP. I would amend Hans' patch slightly to honor the opt_* setting for KLD_TIED (but that is only true if KLD_TIED means "built as part of a kernel build, so has valid opt_foo.h headers" and not 'a standalone module where someone put MODULES_TIED=1 on the command line to make'). -- John Baldwin