Date: Sat, 11 Nov 2017 21:14:39 +0100 From: Andreas Longwitz <longwitz@incore.de> To: Mark Johnston <markj@FreeBSD.org> Cc: "freebsd-dtrace@freebsd.org" <freebsd-dtrace@freebsd.org> Subject: Re: dtrace pid provider on entry == Trace/BPT trap process Message-ID: <5A075A2F.1040604@incore.de> In-Reply-To: <20171111035212.GA1176@bish> References: <59AE834E.1090700@incore.de> <20170905144154.GC65864@wkstn-mjohnston.west.isilon.com> <5A0195CA.2010907@incore.de> <20171111035212.GA1176@bish>
next in thread | previous in thread | raw e-mail | index | archive | help
> Maybe the following is of interest for DTrace experts:
>>
>>>> I use FreeBSD 10.3-STABLE r317936 with updated /usr/lib/dtrace/ip.c and
>>>> see an issue that in my opinion should have been solved in r269342:
>>> 10.3 would be missing a number of other fixes for issues that might
>>> cause the symptom you're seeing. Unfortunately, userland DTrace is not
>>> very usable on FreeBSD before 11.0.
>> I have repeated my looptest on a server running FreeBSD 12.0-CURRENT #0
>> r324306 with GENERIC kernel. I saw the same problem as described before
>> for FreeBSD 10.3-Stable r317936:
>>
>> Nov 7 10:54:09 <kern.info> dssbkp2 kernel: pid 38041 (looptest), uid
>> 1003: exited on signal 5
>>
>> Further I saw following error message from dtrace:
>>
>> dtrace: failed to grab pid 38622: Device busy.
>>
>> After some minutes playing around the server crashed with
>>
>> panic: PHOLD of exiting process 0xfffff80004c80540
>
> Sorry for the delayed reply. I think this assertion is bogus: the
> process in question is already held at this point, so it can't continue
> exiting anyway. I think the patch below will address this and the
> SIGTRAP crashes. Could you give it a try?
>
> diff --git a/sys/cddl/compat/opensolaris/kern/opensolaris_proc.c b/sys/cddl/compat/opensolaris/kern/opensolaris_proc.c
> index 652fa8928589..63db17199fb4 100644
> --- a/sys/cddl/compat/opensolaris/kern/opensolaris_proc.c
> +++ b/sys/cddl/compat/opensolaris/kern/opensolaris_proc.c
> @@ -35,9 +35,7 @@ uread(proc_t *p, void *kaddr, size_t len, uintptr_t uaddr)
> {
> ssize_t n;
>
> - PHOLD(p);
> n = proc_readmem(curthread, p, uaddr, kaddr, len);
> - PRELE(p);
> if (n != len)
> return (ENOMEM);
> return (0);
> @@ -48,9 +46,7 @@ uwrite(proc_t *p, void *kaddr, size_t len, uintptr_t uaddr)
> {
> ssize_t n;
>
> - PHOLD(p);
> n = proc_writemem(curthread, p, uaddr, kaddr, len);
> - PRELE(p);
> if (n != len)
> return (ENOMEM);
> return (0);
> diff --git a/sys/cddl/contrib/opensolaris/uts/common/dtrace/fasttrap.c b/sys/cddl/contrib/opensolaris/uts/common/dtrace/fasttrap.c
> index 82353cbb235d..d721270fd425 100644
> --- a/sys/cddl/contrib/opensolaris/uts/common/dtrace/fasttrap.c
> +++ b/sys/cddl/contrib/opensolaris/uts/common/dtrace/fasttrap.c
> @@ -64,6 +64,7 @@
> #include <sys/dtrace_bsd.h>
> #include <sys/eventhandler.h>
> #include <sys/rmlock.h>
> +#include <sys/smp.h>
> #include <sys/sysent.h>
> #include <sys/sysctl.h>
> #include <sys/u8_textprep.h>
> @@ -1097,6 +1098,8 @@ fasttrap_tracepoint_disable(proc_t *p, fasttrap_probe_t *probe, uint_t index)
> p->p_dtrace_count--;
> }
>
> + (void)quiesce_all_cpus("fasttrap", 0);
> +
> /*
> * Remove the probe from the hash table of active tracepoints.
> */
> diff --git a/sys/cddl/contrib/opensolaris/uts/intel/dtrace/fasttrap_isa.c b/sys/cddl/contrib/opensolaris/uts/intel/dtrace/fasttrap_isa.c
> index d6655a0b3c55..0364aaf55094 100644
> --- a/sys/cddl/contrib/opensolaris/uts/intel/dtrace/fasttrap_isa.c
> +++ b/sys/cddl/contrib/opensolaris/uts/intel/dtrace/fasttrap_isa.c
> @@ -1715,11 +1715,7 @@ fasttrap_pid_probe(struct reg *rp)
>
> ASSERT(i <= sizeof (scratch));
>
> -#ifdef illumos
> if (fasttrap_copyout(scratch, (char *)addr, i)) {
> -#else
> - if (uwrite(p, scratch, i, addr)) {
> -#endif
> fasttrap_sigtrap(p, curthread, pc);
> new_pc = pc;
> break;
I have applied your patch, no crash, no core or error message so far.
But dtrace does not exit after CTRL C. After starting looptest I enter
the dtrace command and type CTRL C after some seconds:
dtrace -n 'pid$target:libc.so.*::entry {@[probefunc] = count();}' -p
$(pgrep looptest)
dtrace: description 'pid$target:libc.so.*::entry ' matched 4068 probes
^C
__vfprintf 226824
localeconv_l 226824
memset 226824
sprintf 226824
vsprintf 226824
__sfvwrite 453647
memcpy 680469
^C^C^C^C^C^C
I could not stop dtrace, it does not exit. procstat gives
PID PPID PGID SID TSID THR LOGIN WCHAN EMUL COMM
28183 28031 28183 28025 28025 1 longwitz fasttrap FreeBSD ELF64 dtrace
If I stop looptest, then dtrace process stops immediately too.
--
Andreas Longwitz
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5A075A2F.1040604>