From owner-freebsd-dtrace@freebsd.org Tue Nov 7 11:17:02 2017 Return-Path: Delivered-To: freebsd-dtrace@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A2CCAE5592E for ; Tue, 7 Nov 2017 11:17:02 +0000 (UTC) (envelope-from longwitz@incore.de) Received: from dss.incore.de (dss.incore.de [195.145.1.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3710C7AF23; Tue, 7 Nov 2017 11:17:01 +0000 (UTC) (envelope-from longwitz@incore.de) Received: from inetmail.dmz (inetmail.dmz [10.3.0.3]) by dss.incore.de (Postfix) with ESMTP id B0E6413AE; Tue, 7 Nov 2017 12:16:52 +0100 (CET) X-Virus-Scanned: amavisd-new at incore.de Received: from dss.incore.de ([10.3.0.3]) by inetmail.dmz (inetmail.dmz [10.3.0.3]) (amavisd-new, port 10024) with LMTP id Vcp7LoU3Ad_A; Tue, 7 Nov 2017 12:16:48 +0100 (CET) Received: from mail.local.incore (fwintern.dmz [10.0.0.253]) by dss.incore.de (Postfix) with ESMTP id 90E1C1409; Tue, 7 Nov 2017 12:15:22 +0100 (CET) Received: from bsdlo.incore (bsdlo.incore [192.168.0.84]) by mail.local.incore (Postfix) with ESMTP id 7FC01508DE; Tue, 7 Nov 2017 12:15:22 +0100 (CET) Message-ID: <5A0195CA.2010907@incore.de> Date: Tue, 07 Nov 2017 12:15:22 +0100 From: Andreas Longwitz User-Agent: Thunderbird 2.0.0.19 (X11/20090113) MIME-Version: 1.0 To: Mark Johnston CC: "freebsd-dtrace@freebsd.org" Subject: Re: dtrace pid provider on entry == Trace/BPT trap process References: <59AE834E.1090700@incore.de> <20170905144154.GC65864@wkstn-mjohnston.west.isilon.com> In-Reply-To: <20170905144154.GC65864@wkstn-mjohnston.west.isilon.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-dtrace@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "A discussion list for developers working on DTrace in FreeBSD." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Nov 2017 11:17:02 -0000 Maybe the following is of interest for DTrace experts: >> I use FreeBSD 10.3-STABLE r317936 with updated /usr/lib/dtrace/ip.c and >> see an issue that in my opinion should have been solved in r269342: > > 10.3 would be missing a number of other fixes for issues that might > cause the symptom you're seeing. Unfortunately, userland DTrace is not > very usable on FreeBSD before 11.0. I have repeated my looptest on a server running FreeBSD 12.0-CURRENT #0 r324306 with GENERIC kernel. I saw the same problem as described before for FreeBSD 10.3-Stable r317936: Nov 7 10:54:09 dssbkp2 kernel: pid 38041 (looptest), uid 1003: exited on signal 5 Further I saw following error message from dtrace: dtrace: failed to grab pid 38622: Device busy. After some minutes playing around the server crashed with panic: PHOLD of exiting process 0xfffff80004c80540 cpuid = 0 time = 1510051083 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0231e22700 vpanic() at vpanic+0x19c/frame 0xfffffe0231e22780 kassert_panic() at kassert_panic+0x126/frame 0xfffffe0231e227f0 uread() at uread+0x173/frame 0xfffffe0231e22830 fasttrap_tracepoint_remove() at fasttrap_tracepoint_remove+0x23/frame 0xfffffe0231e22860 fasttrap_tracepoint_disable() at fasttrap_tracepoint_disable+0x31a/frame 0xfffffe0231e228d0 fasttrap_pid_disable() at fasttrap_pid_disable+0x8e/frame 0xfffffe0231e22910 dtrace_ecb_disable() at dtrace_ecb_disable+0x11c/frame 0xfffffe0231e22940 dtrace_state_destroy() at dtrace_state_destroy+0x169/frame 0xfffffe0231e22980 dtrace_dtr() at dtrace_dtr+0x63/frame 0xfffffe0231e229a0 devfs_destroy_cdevpriv() at devfs_destroy_cdevpriv+0x8b/frame 0xfffffe0231e229c0 devfs_close_f() at devfs_close_f+0x65/frame 0xfffffe0231e229f0 closef() at closef+0x1f5/frame 0xfffffe0231e22a80 closefp() at closefp+0x9f/frame 0xfffffe0231e22ac0 amd64_syscall() at amd64_syscall+0x79b/frame 0xfffffe0231e22bf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe0231e22bf0 --- syscall (6, FreeBSD ELF64, sys_close), rip = 0x80128f47a, rsp = 0x7fffffffe878, rbp = 0x7fffffffe890 --- >> I run a simple program looptest, that loops around a sprintf() call and >> want to check this program with >> >> dtrace -n 'pid$target:libc.so.*::entry {@[probefunc] = count();}' -p >> $(pgrep looptest) >> dtrace: description 'pid$target:libc.so.*::entry ' matched 3863 probes >> >> After some seconds I hit CTRL-C and see >> >> __vfprintf 741685 >> localeconv_l 741685 >> memset 741685 >> sprintf 741685 >> vsprintf 741685 >> __sfvwrite 1483369 >> memcpy 2225052 >> >> and at the same time looptest stops with >> Trace/BPT trap (Speicherabzug geschrieben) >> and on the console I see the kernel message >> pid 67550 (looptest), uid 1003: exited on signal 5 (core dumped) >> >> The trap does not occur all the time, but it is no problem to repeat the >> trap. -- Andreas Longwitz