From owner-svn-src-all@FreeBSD.ORG Mon Feb 28 23:28:36 2011 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3A376106564A; Mon, 28 Feb 2011 23:28:36 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 242698FC08; Mon, 28 Feb 2011 23:28:36 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id p1SNSa27059970; Mon, 28 Feb 2011 23:28:36 GMT (envelope-from rwatson@svn.freebsd.org) Received: (from rwatson@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id p1SNSZZK059958; Mon, 28 Feb 2011 23:28:35 GMT (envelope-from rwatson@svn.freebsd.org) Message-Id: <201102282328.p1SNSZZK059958@svn.freebsd.org> From: Robert Watson Date: Mon, 28 Feb 2011 23:28:35 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-8@freebsd.org X-SVN-Group: stable-8 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r219107 - in stable/8/sys: amd64/amd64 amd64/include boot/common cddl/compat/opensolaris/kern cddl/compat/opensolaris/sys cddl/contrib/opensolaris/uts/common/dtrace cddl/contrib/opensol... X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Feb 2011 23:28:36 -0000 Author: rwatson Date: Mon Feb 28 23:28:35 2011 New Revision: 219107 URL: http://svn.freebsd.org/changeset/base/219107 Log: Merge userspace DTrace support from head to stable/8: r209721: Merge from vendor-sys/opensolaris: * add fasttrap files r209731: Introduce USD_{SET,GET}{BASE,LIMIT}. These help setting up the user segment descriptor hi and lo values. Idea from Solaris. Reviewed by: kib r209763: Fix style issues with the previous commit, namely use-tab-instead-of-space and don't use underscores in macro variables. Pointed out by: bde r210292: Fix typo in comment. r210357: MFamd64: Add USD_GETBASE(), USD_SETBASE(), USD_GETLIMIT() and USD_SETLIMIT(). r210611: Bump the witness pendlist to 768 to accomodate the increased number of spinlocks. r211553: Add sysname to struct opensolaris_utsname. This is needed by one DTrace test. r211566: Add a sysname char * to struct opensolaris_utsname. r211606: Add the FreeBSD definition for the fasttrap ioctls. r211607: Add a function compatibility function dtrace_instr_size_isa() that on FreeBSD does the same as dtrace_dis_isize(). r211608: Kernel DTrace support for: o uregs (sson@) o ustack (sson@) o /dev/dtrace/helper device (needed for USDT probes) r211610: Add more compatibility structure members needed by the upcoming fasttrap DTrace device. r211611: Destroy the helper device when unloading. r211613: Fix style issues. r211614: Bump KDTRACE_THREAD_ZERO and use M_ZERO as a malloc flag instead of calling bzero. r211615: Remove an elif and add an or-clause. r211616: Add an extra comment to the SDT probes definition. This allows us to get use '-' in probe names, matching the probe names in Solaris. Add userland SDT probes definitions to sys/sdt.h. r211617: Call the systrace_probe_func() when the error value. r211618: Port this to FreeBSD. We miss some suword functions, so we use copyout. r211738: Port the fasttrap provider to FreeBSD. This provider is responsible for injecting debugging probes in the userland programs and is the basis for the pid provider and the usdt provider. r211744: MD fasttrap implementation. r211745: Replace a pksignal() call with tdksignal(). Pointed out by: kib r211746: Update for the recent location of the fasttrap code. r211747: Replace structure assignments with explicity memcpy calls. This allows Clang to compile this file: it was using the builtin memcpy and we want to use the memcpy defined in gptboot.c. (Clang can't compile boot2 yet). Submitted by: Dimitry Andric Reviewed by: jhb r211751: Add a trap code for DTrace induced traps. r211752: Add two DTrace trap type values. Used by fasttrap. r211753: Enable fasttrap and make dtraceall depend on fasttrap when building i386 or amd64. r211804: Call the necessary DTrace function pointers when we have different kinds of traps. r211813: Add the necessary DTrace function pointers. r211839: Sync DTrace bits with amd64 and fix the build. r211924: Register an interrupt vector for DTrace return probes. There is some code missing in lapic to make sure that we don't overwrite this entry, but this will be done on a sequent commit. r211925: Replace a memory barrier with a mutex barrier. r211926: Add the path necessary to find fasttrap_isa.h to CFLAGS. r211929: Remove debugging. r212004: When DTrace is enabled, make sure we don't overwrite the IDT_DTRACE_RET entry with an IRQ for some hardware component. Reviewed by: jhb r212093: Make the /dev/dtrace/helper node have the mode 0660. This allows programs that refuse to run as root (pgsql) to install probes when their user is part of the wheel group. r212357: Fix two bugs in DTrace: * when the process exits, remove the associated USDT probes * when the process forks, duplicate the USDT probes. r212465: Avoid a LOR (sleepable after non-sleepable) in fasttrap_tracepoint_enable(). r212494: Revamp locking a bit. This fixes three problems: * processes now can't go away while we are inserting probes (fixes a panic) * if a trap happens, we won't be holding the process lock (fixes a hang) * fix a LOR between the process lock and the fasttrap bucket list lock Thanks to kib for pointing some problems. r212568: Bump __FreeBSD_version to reflect the userland DTrace changes Sponsored by: The FreeBSD Foundation Userspace DTrace work by: rpaulo Added: stable/8/sys/cddl/contrib/opensolaris/uts/common/sys/fasttrap_impl.h - copied, changed from r209721, head/sys/cddl/contrib/opensolaris/uts/common/sys/fasttrap_impl.h stable/8/sys/cddl/contrib/opensolaris/uts/intel/dtrace/ - copied from r209721, head/sys/cddl/contrib/opensolaris/uts/intel/dtrace/ stable/8/sys/cddl/contrib/opensolaris/uts/sparc/dtrace/ - copied from r209721, head/sys/cddl/contrib/opensolaris/uts/sparc/dtrace/ stable/8/sys/cddl/dev/dtrace/amd64/regset.h - copied unchanged from r211608, head/sys/cddl/dev/dtrace/amd64/regset.h stable/8/sys/cddl/dev/dtrace/i386/regset.h - copied unchanged from r211608, head/sys/cddl/dev/dtrace/i386/regset.h Modified: stable/8/sys/amd64/amd64/exception.S stable/8/sys/amd64/amd64/machdep.c stable/8/sys/amd64/amd64/trap.c stable/8/sys/amd64/include/segments.h stable/8/sys/amd64/include/trap.h stable/8/sys/boot/common/ufsread.c stable/8/sys/cddl/compat/opensolaris/kern/opensolaris_misc.c stable/8/sys/cddl/compat/opensolaris/sys/misc.h stable/8/sys/cddl/contrib/opensolaris/uts/common/dtrace/dtrace.c stable/8/sys/cddl/contrib/opensolaris/uts/common/dtrace/fasttrap.c stable/8/sys/cddl/contrib/opensolaris/uts/common/sys/dtrace.h stable/8/sys/cddl/contrib/opensolaris/uts/common/sys/dtrace_impl.h stable/8/sys/cddl/contrib/opensolaris/uts/common/sys/fasttrap.h stable/8/sys/cddl/contrib/opensolaris/uts/intel/dtrace/fasttrap_isa.c stable/8/sys/cddl/dev/dtrace/amd64/dtrace_isa.c stable/8/sys/cddl/dev/dtrace/amd64/instr_size.c stable/8/sys/cddl/dev/dtrace/dtrace_cddl.h stable/8/sys/cddl/dev/dtrace/dtrace_ioctl.c stable/8/sys/cddl/dev/dtrace/dtrace_load.c stable/8/sys/cddl/dev/dtrace/dtrace_unload.c stable/8/sys/cddl/dev/dtrace/i386/dtrace_isa.c stable/8/sys/cddl/dev/dtrace/i386/instr_size.c stable/8/sys/cddl/dev/systrace/systrace.c stable/8/sys/i386/i386/exception.s stable/8/sys/i386/i386/machdep.c stable/8/sys/i386/i386/trap.c stable/8/sys/i386/include/segments.h stable/8/sys/i386/include/trap.h stable/8/sys/kern/kern_dtrace.c stable/8/sys/kern/kern_exec.c stable/8/sys/kern/kern_exit.c stable/8/sys/kern/kern_fork.c stable/8/sys/kern/kern_priv.c stable/8/sys/kern/kern_proc.c stable/8/sys/kern/kern_sig.c stable/8/sys/kern/kern_timeout.c stable/8/sys/kern/subr_trap.c stable/8/sys/kern/subr_witness.c stable/8/sys/kern/vfs_cache.c stable/8/sys/kern/vfs_lookup.c stable/8/sys/kern/vfs_syscalls.c stable/8/sys/modules/dtrace/Makefile stable/8/sys/modules/dtrace/dtrace/Makefile stable/8/sys/modules/dtrace/dtraceall/dtraceall.c stable/8/sys/modules/dtrace/fasttrap/Makefile stable/8/sys/net/vnet.c stable/8/sys/opencrypto/deflate.c stable/8/sys/security/mac/mac_framework.c stable/8/sys/security/mac/mac_internal.h stable/8/sys/sys/dtrace_bsd.h stable/8/sys/sys/param.h stable/8/sys/sys/priv.h stable/8/sys/sys/sdt.h stable/8/sys/sys/signal.h stable/8/sys/sys/sysent.h stable/8/sys/tools/vnode_if.awk stable/8/sys/x86/x86/local_apic.c Directory Properties: stable/8/sys/ (props changed) stable/8/sys/amd64/include/xen/ (props changed) stable/8/sys/cddl/contrib/opensolaris/ (props changed) stable/8/sys/contrib/dev/acpica/ (props changed) stable/8/sys/contrib/pf/ (props changed) Modified: stable/8/sys/amd64/amd64/exception.S ============================================================================== --- stable/8/sys/amd64/amd64/exception.S Mon Feb 28 21:33:26 2011 (r219106) +++ stable/8/sys/amd64/amd64/exception.S Mon Feb 28 23:28:35 2011 (r219107) @@ -108,6 +108,10 @@ IDTVEC(dbg) TRAP_NOEN(T_TRCTRAP) IDTVEC(bpt) TRAP_NOEN(T_BPTFLT) +#ifdef KDTRACE_HOOKS +IDTVEC(dtrace_ret) + TRAP_NOEN(T_DTRACE_RET) +#endif /* Regular traps; The cpu does not supply tf_err for these. */ #define TRAP(a) \ Modified: stable/8/sys/amd64/amd64/machdep.c ============================================================================== --- stable/8/sys/amd64/amd64/machdep.c Mon Feb 28 21:33:26 2011 (r219106) +++ stable/8/sys/amd64/amd64/machdep.c Mon Feb 28 23:28:35 2011 (r219107) @@ -54,6 +54,7 @@ __FBSDID("$FreeBSD$"); #include "opt_msgbuf.h" #include "opt_perfmon.h" #include "opt_sched.h" +#include "opt_kdtrace.h" #include #include @@ -1094,6 +1095,9 @@ extern inthand_t IDTVEC(tss), IDTVEC(missing), IDTVEC(stk), IDTVEC(prot), IDTVEC(page), IDTVEC(mchk), IDTVEC(rsvd), IDTVEC(fpu), IDTVEC(align), IDTVEC(xmm), IDTVEC(dblfault), +#ifdef KDTRACE_HOOKS + IDTVEC(dtrace_ret), +#endif IDTVEC(fast_syscall), IDTVEC(fast_syscall32); #ifdef DDB @@ -1624,6 +1628,9 @@ hammer_time(u_int64_t modulep, u_int64_t setidt(IDT_AC, &IDTVEC(align), SDT_SYSIGT, SEL_KPL, 0); setidt(IDT_MC, &IDTVEC(mchk), SDT_SYSIGT, SEL_KPL, 0); setidt(IDT_XF, &IDTVEC(xmm), SDT_SYSIGT, SEL_KPL, 0); +#ifdef KDTRACE_HOOKS + setidt(IDT_DTRACE_RET, &IDTVEC(dtrace_ret), SDT_SYSIGT, SEL_UPL, 0); +#endif r_idt.rd_limit = sizeof(idt0) - 1; r_idt.rd_base = (long) idt; Modified: stable/8/sys/amd64/amd64/trap.c ============================================================================== --- stable/8/sys/amd64/amd64/trap.c Mon Feb 28 21:33:26 2011 (r219106) +++ stable/8/sys/amd64/amd64/trap.c Mon Feb 28 23:28:35 2011 (r219107) @@ -113,6 +113,13 @@ dtrace_doubletrap_func_t dtrace_doubletr * implementation opaque. */ systrace_probe_func_t systrace_probe_func; + +/* + * These hooks are necessary for the pid, usdt and fasttrap providers. + */ +dtrace_fasttrap_probe_ptr_t dtrace_fasttrap_probe_ptr; +dtrace_pid_probe_ptr_t dtrace_pid_probe_ptr; +dtrace_return_probe_ptr_t dtrace_return_probe_ptr; #endif extern void trap(struct trapframe *frame); @@ -243,6 +250,55 @@ trap(struct trapframe *frame) if (dtrace_trap_func != NULL) if ((*dtrace_trap_func)(frame, type)) goto out; + if (type == T_DTRACE_PROBE || type == T_DTRACE_RET || + type == T_BPTFLT) { + struct reg regs; + + regs.r_r15 = frame->tf_r15; + regs.r_r14 = frame->tf_r14; + regs.r_r13 = frame->tf_r13; + regs.r_r12 = frame->tf_r12; + regs.r_r11 = frame->tf_r11; + regs.r_r10 = frame->tf_r10; + regs.r_r9 = frame->tf_r9; + regs.r_r8 = frame->tf_r8; + regs.r_rdi = frame->tf_rdi; + regs.r_rsi = frame->tf_rsi; + regs.r_rbp = frame->tf_rbp; + regs.r_rbx = frame->tf_rbx; + regs.r_rdx = frame->tf_rdx; + regs.r_rcx = frame->tf_rcx; + regs.r_rax = frame->tf_rax; + regs.r_rip = frame->tf_rip; + regs.r_cs = frame->tf_cs; + regs.r_rflags = frame->tf_rflags; + regs.r_rsp = frame->tf_rsp; + regs.r_ss = frame->tf_ss; + if (frame->tf_flags & TF_HASSEGS) { + regs.r_ds = frame->tf_ds; + regs.r_es = frame->tf_es; + regs.r_fs = frame->tf_fs; + regs.r_gs = frame->tf_gs; + } else { + regs.r_ds = 0; + regs.r_es = 0; + regs.r_fs = 0; + regs.r_gs = 0; + } + if (type == T_DTRACE_PROBE && + dtrace_fasttrap_probe_ptr != NULL && + dtrace_fasttrap_probe_ptr(®s) == 0) + goto out; + if (type == T_BPTFLT && + dtrace_pid_probe_ptr != NULL && + dtrace_pid_probe_ptr(®s) == 0) + goto out; + if (type == T_DTRACE_RET && + dtrace_return_probe_ptr != NULL && + dtrace_return_probe_ptr(®s) == 0) + goto out; + + } #endif if ((frame->tf_rflags & PSL_I) == 0) { Modified: stable/8/sys/amd64/include/segments.h ============================================================================== --- stable/8/sys/amd64/include/segments.h Mon Feb 28 21:33:26 2011 (r219106) +++ stable/8/sys/amd64/include/segments.h Mon Feb 28 23:28:35 2011 (r219107) @@ -74,6 +74,13 @@ struct user_segment_descriptor { u_int64_t sd_hibase:8; /* segment base address (msb) */ } __packed; +#define USD_GETBASE(sd) (((sd)->sd_lobase) | (sd)->sd_hibase << 24) +#define USD_SETBASE(sd, b) (sd)->sd_lobase = (b); \ + (sd)->sd_hibase = ((b) >> 24); +#define USD_GETLIMIT(sd) (((sd)->sd_lolimit) | (sd)->sd_hilimit << 16) +#define USD_SETLIMIT(sd, l) (sd)->sd_lolimit = (l); \ + (sd)->sd_hilimit = ((l) >> 16); + /* * System segment descriptors (128 bit wide) */ @@ -207,6 +214,7 @@ struct region_descriptor { #define IDT_XF 19 /* #XF: SIMD Floating-Point Exception */ #define IDT_IO_INTS NRSVIDT /* Base of IDT entries for I/O interrupts. */ #define IDT_SYSCALL 0x80 /* System Call Interrupt Vector */ +#define IDT_DTRACE_RET 0x92 /* DTrace pid provider Interrupt Vector */ /* * Entries in the Global Descriptor Table (GDT) Modified: stable/8/sys/amd64/include/trap.h ============================================================================== --- stable/8/sys/amd64/include/trap.h Mon Feb 28 21:33:26 2011 (r219106) +++ stable/8/sys/amd64/include/trap.h Mon Feb 28 23:28:35 2011 (r219107) @@ -62,6 +62,8 @@ #define T_MCHK 28 /* machine check trap */ #define T_XMMFLT 29 /* SIMD floating-point exception */ #define T_RESERVED 30 /* reserved (unknown) */ +#define T_DTRACE_RET 31 /* DTrace pid return */ +#define T_DTRACE_PROBE 32 /* DTrace fasttrap probe */ /* XXX most of the following codes aren't used, but could be. */ Modified: stable/8/sys/boot/common/ufsread.c ============================================================================== --- stable/8/sys/boot/common/ufsread.c Mon Feb 28 21:33:26 2011 (r219106) +++ stable/8/sys/boot/common/ufsread.c Mon Feb 28 23:28:35 2011 (r219107) @@ -223,14 +223,19 @@ fsread(ino_t inode, void *buf, size_t nb return -1; n = INO_TO_VBO(n, inode); #if defined(UFS1_ONLY) - dp1 = ((struct ufs1_dinode *)blkbuf)[n]; + memcpy(&dp1, (struct ufs1_dinode *)blkbuf + n, + sizeof(struct ufs1_dinode)); #elif defined(UFS2_ONLY) - dp2 = ((struct ufs2_dinode *)blkbuf)[n]; + memcpy(&dp2, (struct ufs2_dinode *)blkbuf + n, + sizeof(struct ufs2_dinode)); #else if (fs->fs_magic == FS_UFS1_MAGIC) - dp1 = ((struct ufs1_dinode *)blkbuf)[n]; + memcpy(&dp1, (struct ufs1_dinode *)blkbuf + n, + sizeof(struct ufs1_dinode)); else - dp2 = ((struct ufs2_dinode *)blkbuf)[n]; + memcpy(&dp2, (struct ufs2_dinode *)blkbuf + n, + sizeof(struct ufs2_dinode)); + #endif inomap = inode; fs_off = 0; Modified: stable/8/sys/cddl/compat/opensolaris/kern/opensolaris_misc.c ============================================================================== --- stable/8/sys/cddl/compat/opensolaris/kern/opensolaris_misc.c Mon Feb 28 21:33:26 2011 (r219106) +++ stable/8/sys/cddl/compat/opensolaris/kern/opensolaris_misc.c Mon Feb 28 23:28:35 2011 (r219107) @@ -38,7 +38,8 @@ __FBSDID("$FreeBSD$"); char hw_serial[11] = "0"; struct opensolaris_utsname utsname = { - .nodename = "unset" + .nodename = "unset", + .sysname = "SunOS" }; int Modified: stable/8/sys/cddl/compat/opensolaris/sys/misc.h ============================================================================== --- stable/8/sys/cddl/compat/opensolaris/sys/misc.h Mon Feb 28 21:33:26 2011 (r219106) +++ stable/8/sys/cddl/compat/opensolaris/sys/misc.h Mon Feb 28 23:28:35 2011 (r219107) @@ -46,6 +46,7 @@ #ifdef _KERNEL struct opensolaris_utsname { char *nodename; + char *sysname; }; extern char hw_serial[11]; Modified: stable/8/sys/cddl/contrib/opensolaris/uts/common/dtrace/dtrace.c ============================================================================== --- stable/8/sys/cddl/contrib/opensolaris/uts/common/dtrace/dtrace.c Mon Feb 28 21:33:26 2011 (r219106) +++ stable/8/sys/cddl/contrib/opensolaris/uts/common/dtrace/dtrace.c Mon Feb 28 23:28:35 2011 (r219107) @@ -551,20 +551,16 @@ static void dtrace_enabling_provide(dtra static int dtrace_enabling_match(dtrace_enabling_t *, int *); static void dtrace_enabling_matchall(void); static dtrace_state_t *dtrace_anon_grab(void); -#if defined(sun) static uint64_t dtrace_helper(int, dtrace_mstate_t *, dtrace_state_t *, uint64_t, uint64_t); static dtrace_helpers_t *dtrace_helpers_create(proc_t *); -#endif static void dtrace_buffer_drop(dtrace_buffer_t *); static intptr_t dtrace_buffer_reserve(dtrace_buffer_t *, size_t, size_t, dtrace_state_t *, dtrace_mstate_t *); static int dtrace_state_option(dtrace_state_t *, dtrace_optid_t, dtrace_optval_t); static int dtrace_ecb_create_enable(dtrace_probe_t *, void *); -#if defined(sun) static void dtrace_helper_provider_destroy(dtrace_helper_provider_t *); -#endif uint16_t dtrace_load16(uintptr_t); uint32_t dtrace_load32(uintptr_t); uint64_t dtrace_load64(uintptr_t); @@ -2784,6 +2780,21 @@ dtrace_dif_variable(dtrace_mstate_t *mst return (dtrace_getreg(lwp->lwp_regs, ndx)); return (0); } +#else + case DIF_VAR_UREGS: { + struct trapframe *tframe; + + if (!dtrace_priv_proc(state)) + return (0); + + if ((tframe = curthread->td_frame) == NULL) { + DTRACE_CPUFLAG_SET(CPU_DTRACE_BADADDR); + cpu_core[curcpu].cpuc_dtrace_illval = 0; + return (0); + } + + return (dtrace_getreg(tframe, ndx)); + } #endif case DIF_VAR_CURTHREAD: @@ -2839,7 +2850,6 @@ dtrace_dif_variable(dtrace_mstate_t *mst } return (mstate->dtms_stackdepth); -#if defined(sun) case DIF_VAR_USTACKDEPTH: if (!dtrace_priv_proc(state)) return (0); @@ -2859,7 +2869,6 @@ dtrace_dif_variable(dtrace_mstate_t *mst mstate->dtms_present |= DTRACE_MSTATE_USTACKDEPTH; } return (mstate->dtms_ustackdepth); -#endif case DIF_VAR_CALLER: if (!dtrace_priv_kernel(state)) @@ -2896,7 +2905,6 @@ dtrace_dif_variable(dtrace_mstate_t *mst } return (mstate->dtms_caller); -#if defined(sun) case DIF_VAR_UCALLER: if (!dtrace_priv_proc(state)) return (0); @@ -2920,7 +2928,6 @@ dtrace_dif_variable(dtrace_mstate_t *mst } return (mstate->dtms_ucaller); -#endif case DIF_VAR_PROBEPROV: ASSERT(mstate->dtms_present & DTRACE_MSTATE_PROBE); @@ -5736,7 +5743,6 @@ dtrace_action_chill(dtrace_mstate_t *mst cpu->cpu_dtrace_chilled += val; } -#if defined(sun) static void dtrace_action_ustack(dtrace_mstate_t *mstate, dtrace_state_t *state, uint64_t *buf, uint64_t arg) @@ -5849,7 +5855,6 @@ dtrace_action_ustack(dtrace_mstate_t *ms out: mstate->dtms_scratch_ptr = old; } -#endif /* * If you're looking for the epicenter of DTrace, you just found it. This @@ -6172,7 +6177,6 @@ dtrace_probe(dtrace_id_t id, uintptr_t a (uint32_t *)arg0); continue; -#if defined(sun) case DTRACEACT_JSTACK: case DTRACEACT_USTACK: if (!dtrace_priv_proc(state)) @@ -6214,7 +6218,6 @@ dtrace_probe(dtrace_id_t id, uintptr_t a DTRACE_USTACK_NFRAMES(rec->dtrd_arg) + 1); DTRACE_CPUFLAG_CLEAR(CPU_DTRACE_NOFAULT); continue; -#endif default: break; @@ -8141,7 +8144,6 @@ dtrace_helper_provide(dof_helper_t *dhp, dtrace_enabling_matchall(); } -#if defined(sun) static void dtrace_helper_provider_remove_one(dof_helper_t *dhp, dof_sec_t *sec, pid_t pid) { @@ -8189,7 +8191,6 @@ dtrace_helper_provider_remove(dof_helper dtrace_helper_provider_remove_one(dhp, sec, pid); } } -#endif /* * DTrace Meta Provider-to-Framework API Functions @@ -8729,7 +8730,6 @@ dtrace_difo_validate(dtrace_difo_t *dp, return (err); } -#if defined(sun) /* * Validate a DTrace DIF object that it is to be used as a helper. Helpers * are much more constrained than normal DIFOs. Specifically, they may @@ -8887,7 +8887,6 @@ dtrace_difo_validate_helper(dtrace_difo_ return (err); } -#endif /* * Returns 1 if the expression in the DIF object can be cached on a per-thread @@ -9219,7 +9218,6 @@ dtrace_difo_init(dtrace_difo_t *dp, dtra dtrace_difo_hold(dp); } -#if defined(sun) static dtrace_difo_t * dtrace_difo_duplicate(dtrace_difo_t *dp, dtrace_vstate_t *vstate) { @@ -9263,7 +9261,6 @@ dtrace_difo_duplicate(dtrace_difo_t *dp, dtrace_difo_init(new, vstate); return (new); } -#endif static void dtrace_difo_destroy(dtrace_difo_t *dp, dtrace_vstate_t *vstate) @@ -13791,7 +13788,6 @@ dtrace_anon_property(void) } } -#if defined(sun) /* * DTrace Helper Functions */ @@ -13855,9 +13851,7 @@ dtrace_helper_trace(dtrace_helper_action ((uint64_t *)(uintptr_t)svar->dtsv_data)[curcpu]; } } -#endif -#if defined(sun) static uint64_t dtrace_helper(int which, dtrace_mstate_t *mstate, dtrace_state_t *state, uint64_t arg0, uint64_t arg1) @@ -13865,7 +13859,7 @@ dtrace_helper(int which, dtrace_mstate_t uint16_t *flags = &cpu_core[curcpu].cpuc_dtrace_flags; uint64_t sarg0 = mstate->dtms_arg[0]; uint64_t sarg1 = mstate->dtms_arg[1]; - uint64_t rval; + uint64_t rval = 0; dtrace_helpers_t *helpers = curproc->p_dtrace_helpers; dtrace_helper_action_t *helper; dtrace_vstate_t *vstate; @@ -14056,9 +14050,7 @@ dtrace_helper_destroygen(int gen) return (0); } -#endif -#if defined(sun) static int dtrace_helper_validate(dtrace_helper_action_t *helper) { @@ -14073,9 +14065,7 @@ dtrace_helper_validate(dtrace_helper_act return (err == 0); } -#endif -#if defined(sun) static int dtrace_helper_action_add(int which, dtrace_ecbdesc_t *ep) { @@ -14622,12 +14612,17 @@ dtrace_helpers_create(proc_t *p) return (help); } -static void -dtrace_helpers_destroy(void) +#if defined(sun) +static +#endif +void +dtrace_helpers_destroy(proc_t *p) { dtrace_helpers_t *help; dtrace_vstate_t *vstate; +#if defined(sun) proc_t *p = curproc; +#endif int i; mutex_enter(&dtrace_lock); @@ -14714,7 +14709,10 @@ dtrace_helpers_destroy(void) mutex_exit(&dtrace_lock); } -static void +#if defined(sun) +static +#endif +void dtrace_helpers_duplicate(proc_t *from, proc_t *to) { dtrace_helpers_t *help, *newhelp; @@ -14795,7 +14793,6 @@ dtrace_helpers_duplicate(proc_t *from, p if (hasprovs) dtrace_helper_provider_register(to, newhelp, NULL); } -#endif #if defined(sun) /* @@ -16466,6 +16463,7 @@ _fini(void) #else static d_ioctl_t dtrace_ioctl; +static d_ioctl_t dtrace_ioctl_helper; static void dtrace_load(void *); static int dtrace_unload(void); #if __FreeBSD_version < 800039 @@ -16474,6 +16472,7 @@ static struct clonedevs *dtrace_clones; static eventhandler_tag eh_tag; /* Event handler tag. */ #else static struct cdev *dtrace_dev; +static struct cdev *helper_dev; #endif void dtrace_invop_init(void); @@ -16488,6 +16487,13 @@ static struct cdevsw dtrace_cdevsw = { .d_name = "dtrace", }; +static struct cdevsw helper_cdevsw = { + .d_version = D_VERSION, + .d_flags = D_TRACKCLOSE | D_NEEDMINOR, + .d_ioctl = dtrace_ioctl_helper, + .d_name = "helper", +}; + #include #if __FreeBSD_version < 800039 #include Modified: stable/8/sys/cddl/contrib/opensolaris/uts/common/dtrace/fasttrap.c ============================================================================== --- stable/8/sys/cddl/contrib/opensolaris/uts/common/dtrace/fasttrap.c Mon Feb 28 21:33:26 2011 (r219106) +++ stable/8/sys/cddl/contrib/opensolaris/uts/common/dtrace/fasttrap.c Mon Feb 28 23:28:35 2011 (r219107) @@ -17,6 +17,10 @@ * information: Portions Copyright [yyyy] [name of copyright owner] * * CDDL HEADER END + * + * Portions Copyright 2010 The FreeBSD Foundation + * + * $FreeBSD$ */ /* @@ -24,7 +28,9 @@ * Use is subject to license terms. */ +#if defined(sun) #pragma ident "%Z%%M% %I% %E% SMI" +#endif #include #include @@ -32,11 +38,15 @@ #include #include #include +#if defined(sun) #include +#endif #include #include #include +#if defined(sun) #include +#endif #include #include #include @@ -44,9 +54,17 @@ #include #include #include -#include #include +#if defined(sun) #include +#endif +#include +#include +#if !defined(sun) +#include +#include +#include +#endif /* * User-Land Trap-Based Tracing @@ -125,11 +143,20 @@ * never hold the provider lock and creation lock simultaneously */ -static dev_info_t *fasttrap_devi; +static d_open_t fasttrap_open; +static d_ioctl_t fasttrap_ioctl; + +static struct cdevsw fasttrap_cdevsw = { + .d_version = D_VERSION, + .d_open = fasttrap_open, + .d_ioctl = fasttrap_ioctl, + .d_name = "fasttrap", +}; +static struct cdev *fasttrap_cdev; static dtrace_meta_provider_id_t fasttrap_meta_id; -static timeout_id_t fasttrap_timeout; -static kmutex_t fasttrap_cleanup_mtx; +static struct callout fasttrap_timeout; +static struct mtx fasttrap_cleanup_mtx; static uint_t fasttrap_cleanup_work; /* @@ -181,6 +208,10 @@ static void fasttrap_proc_release(fasttr #define FASTTRAP_PROCS_INDEX(pid) ((pid) & fasttrap_procs.fth_mask) +#if !defined(sun) +static kmutex_t fasttrap_cpuc_pid_lock[MAXCPU]; +#endif + static int fasttrap_highbit(ulong_t i) { @@ -229,6 +260,7 @@ fasttrap_hash_str(const char *p) void fasttrap_sigtrap(proc_t *p, kthread_t *t, uintptr_t pc) { +#if defined(sun) sigqueue_t *sqp = kmem_zalloc(sizeof (sigqueue_t), KM_SLEEP); sqp->sq_info.si_signo = SIGTRAP; @@ -241,6 +273,17 @@ fasttrap_sigtrap(proc_t *p, kthread_t *t if (t != NULL) aston(t); +#else + ksiginfo_t *ksi = kmem_zalloc(sizeof (ksiginfo_t), KM_SLEEP); + + ksiginfo_init(ksi); + ksi->ksi_signo = SIGTRAP; + ksi->ksi_code = TRAP_DTRACE; + ksi->ksi_addr = (caddr_t)pc; + PROC_LOCK(p); + (void) tdksignal(t, SIGTRAP, ksi); + PROC_UNLOCK(p); +#endif } /* @@ -257,9 +300,9 @@ fasttrap_mod_barrier(uint64_t gen) fasttrap_mod_gen++; - for (i = 0; i < NCPU; i++) { - mutex_enter(&cpu_core[i].cpuc_pid_lock); - mutex_exit(&cpu_core[i].cpuc_pid_lock); + CPU_FOREACH(i) { + mutex_enter(&fasttrap_cpuc_pid_lock[i]); + mutex_exit(&fasttrap_cpuc_pid_lock[i]); } } @@ -274,16 +317,15 @@ fasttrap_pid_cleanup_cb(void *data) fasttrap_provider_t **fpp, *fp; fasttrap_bucket_t *bucket; dtrace_provider_id_t provid; - int i, later; + int i, later = 0; static volatile int in = 0; ASSERT(in == 0); in = 1; - mutex_enter(&fasttrap_cleanup_mtx); while (fasttrap_cleanup_work) { fasttrap_cleanup_work = 0; - mutex_exit(&fasttrap_cleanup_mtx); + mtx_unlock(&fasttrap_cleanup_mtx); later = 0; @@ -349,10 +391,12 @@ fasttrap_pid_cleanup_cb(void *data) mutex_exit(&bucket->ftb_mtx); } - mutex_enter(&fasttrap_cleanup_mtx); + mtx_lock(&fasttrap_cleanup_mtx); } +#if 0 ASSERT(fasttrap_timeout != 0); +#endif /* * If we were unable to remove a retired provider, try again after @@ -364,14 +408,17 @@ fasttrap_pid_cleanup_cb(void *data) * get a chance to do that work if and when the timeout is reenabled * (if detach fails). */ - if (later > 0 && fasttrap_timeout != (timeout_id_t)1) - fasttrap_timeout = timeout(&fasttrap_pid_cleanup_cb, NULL, hz); + if (later > 0 && callout_active(&fasttrap_timeout)) + callout_reset(&fasttrap_timeout, hz, &fasttrap_pid_cleanup_cb, + NULL); else if (later > 0) fasttrap_cleanup_work = 1; - else - fasttrap_timeout = 0; + else { +#if !defined(sun) + /* Nothing to be done for FreeBSD */ +#endif + } - mutex_exit(&fasttrap_cleanup_mtx); in = 0; } @@ -381,11 +428,11 @@ fasttrap_pid_cleanup_cb(void *data) static void fasttrap_pid_cleanup(void) { - mutex_enter(&fasttrap_cleanup_mtx); + + mtx_lock(&fasttrap_cleanup_mtx); fasttrap_cleanup_work = 1; - if (fasttrap_timeout == 0) - fasttrap_timeout = timeout(&fasttrap_pid_cleanup_cb, NULL, 1); - mutex_exit(&fasttrap_cleanup_mtx); + callout_reset(&fasttrap_timeout, 1, &fasttrap_pid_cleanup_cb, NULL); + mtx_unlock(&fasttrap_cleanup_mtx); } /* @@ -400,9 +447,35 @@ fasttrap_fork(proc_t *p, proc_t *cp) pid_t ppid = p->p_pid; int i; +#if defined(sun) ASSERT(curproc == p); ASSERT(p->p_proc_flag & P_PR_LOCK); +#else + PROC_LOCK_ASSERT(p, MA_OWNED); +#endif +#if defined(sun) ASSERT(p->p_dtrace_count > 0); +#else + if (p->p_dtrace_helpers) { + /* + * dtrace_helpers_duplicate() allocates memory. + */ + _PHOLD(cp); + PROC_UNLOCK(p); + PROC_UNLOCK(cp); + dtrace_helpers_duplicate(p, cp); + PROC_LOCK(cp); + PROC_LOCK(p); + _PRELE(cp); + } + /* + * This check is purposely here instead of in kern_fork.c because, + * for legal resons, we cannot include the dtrace_cddl.h header + * inside kern_fork.c and insert if-clause there. + */ + if (p->p_dtrace_count == 0) + return; +#endif ASSERT(cp->p_dtrace_count == 0); /* @@ -419,9 +492,13 @@ fasttrap_fork(proc_t *p, proc_t *cp) * We don't have to worry about the child process disappearing * because we're in fork(). */ - mutex_enter(&cp->p_lock); +#if defined(sun) + mtx_lock_spin(&cp->p_slock); sprlock_proc(cp); - mutex_exit(&cp->p_lock); + mtx_unlock_spin(&cp->p_slock); +#else + _PHOLD(cp); +#endif /* * Iterate over every tracepoint looking for ones that belong to the @@ -451,8 +528,12 @@ fasttrap_fork(proc_t *p, proc_t *cp) mutex_exit(&bucket->ftb_mtx); } +#if defined(sun) mutex_enter(&cp->p_lock); sprunlock(cp); +#else + _PRELE(cp); +#endif } /* @@ -463,24 +544,30 @@ fasttrap_fork(proc_t *p, proc_t *cp) static void fasttrap_exec_exit(proc_t *p) { +#if defined(sun) ASSERT(p == curproc); - ASSERT(MUTEX_HELD(&p->p_lock)); - - mutex_exit(&p->p_lock); +#endif + PROC_LOCK_ASSERT(p, MA_OWNED); + _PHOLD(p); + PROC_UNLOCK(p); /* * We clean up the pid provider for this process here; user-land * static probes are handled by the meta-provider remove entry point. */ fasttrap_provider_retire(p->p_pid, FASTTRAP_PID_NAME, 0); - - mutex_enter(&p->p_lock); +#if !defined(sun) + if (p->p_dtrace_helpers) + dtrace_helpers_destroy(p); +#endif + PROC_LOCK(p); + _PRELE(p); } /*ARGSUSED*/ static void -fasttrap_pid_provide(void *arg, const dtrace_probedesc_t *desc) +fasttrap_pid_provide(void *arg, dtrace_probedesc_t *desc) { /* * There are no "default" pid probes. @@ -504,7 +591,9 @@ fasttrap_tracepoint_enable(proc_t *p, fa ASSERT(probe->ftp_tps[index].fit_tp->ftt_pid == pid); +#if defined(sun) ASSERT(!(p->p_flag & SVFORK)); +#endif /* * Before we make any modifications, make sure we've imposed a barrier @@ -610,7 +699,9 @@ again: * Increment the count of the number of tracepoints active in * the victim process. */ +#if defined(sun) ASSERT(p->p_proc_flag & P_PR_LOCK); +#endif p->p_dtrace_count++; return (rc); @@ -666,7 +757,7 @@ fasttrap_tracepoint_disable(proc_t *p, f fasttrap_bucket_t *bucket; fasttrap_provider_t *provider = probe->ftp_prov; fasttrap_tracepoint_t **pp, *tp; - fasttrap_id_t *id, **idp; + fasttrap_id_t *id, **idp = NULL; pid_t pid; uintptr_t pc; @@ -800,7 +891,9 @@ fasttrap_tracepoint_disable(proc_t *p, f * Decrement the count of the number of tracepoints active * in the victim process. */ +#if defined(sun) ASSERT(p->p_proc_flag & P_PR_LOCK); +#endif p->p_dtrace_count--; } @@ -851,26 +944,31 @@ fasttrap_enable_callbacks(void) static void fasttrap_disable_callbacks(void) { +#if defined(sun) ASSERT(MUTEX_HELD(&cpu_lock)); +#endif + mutex_enter(&fasttrap_count_mtx); ASSERT(fasttrap_pid_count > 0); fasttrap_pid_count--; if (fasttrap_pid_count == 0) { +#if defined(sun) cpu_t *cur, *cpu = CPU; for (cur = cpu->cpu_next_onln; cur != cpu; cur = cur->cpu_next_onln) { rw_enter(&cur->cpu_ft_lock, RW_WRITER); } - +#endif dtrace_pid_probe_ptr = NULL; dtrace_return_probe_ptr = NULL; - +#if defined(sun) for (cur = cpu->cpu_next_onln; cur != cpu; cur = cur->cpu_next_onln) { rw_exit(&cur->cpu_ft_lock); } +#endif } mutex_exit(&fasttrap_count_mtx); } @@ -880,13 +978,16 @@ static void fasttrap_pid_enable(void *arg, dtrace_id_t id, void *parg) { fasttrap_probe_t *probe = parg; - proc_t *p; + proc_t *p = NULL; int i, rc; + ASSERT(probe != NULL); ASSERT(!probe->ftp_enabled); ASSERT(id == probe->ftp_id); +#if defined(sun) ASSERT(MUTEX_HELD(&cpu_lock)); +#endif /* * Increment the count of enabled probes on this probe's provider; @@ -911,6 +1012,7 @@ fasttrap_pid_enable(void *arg, dtrace_id * a fork in which the traced process is being born and we're copying * USDT probes. Otherwise, the process is gone so bail. */ +#if defined(sun) if ((p = sprlock(probe->ftp_pid)) == NULL) { if ((curproc->p_flag & SFORKING) == 0) return; @@ -934,12 +1036,23 @@ fasttrap_pid_enable(void *arg, dtrace_id ASSERT(!(p->p_flag & SVFORK)); mutex_exit(&p->p_lock); +#else + if ((p = pfind(probe->ftp_pid)) == NULL) + return; +#endif /* * We have to enable the trap entry point before any user threads have * the chance to execute the trap instruction we're about to place * in their process's text. */ +#ifdef __FreeBSD__ + /* + * pfind() returns a locked process. + */ + _PHOLD(p); + PROC_UNLOCK(p); +#endif fasttrap_enable_callbacks(); /* @@ -967,8 +1080,12 @@ fasttrap_pid_enable(void *arg, dtrace_id i--; } +#if defined(sun) mutex_enter(&p->p_lock); sprunlock(p); +#else + PRELE(p); +#endif /* * Since we're not actually enabling this probe, @@ -978,9 +1095,12 @@ fasttrap_pid_enable(void *arg, dtrace_id return; } } - +#if defined(sun) mutex_enter(&p->p_lock); sprunlock(p); +#else + PRELE(p); +#endif probe->ftp_enabled = 1; } @@ -996,18 +1116,22 @@ fasttrap_pid_disable(void *arg, dtrace_i ASSERT(id == probe->ftp_id); + mutex_enter(&provider->ftp_mtx); + /* * We won't be able to acquire a /proc-esque lock on the process * iff the process is dead and gone. In this case, we rely on the * provider lock as a point of mutual exclusion to prevent other * DTrace consumers from disabling this probe. */ - if ((p = sprlock(probe->ftp_pid)) != NULL) { - ASSERT(!(p->p_flag & SVFORK)); - mutex_exit(&p->p_lock); + if ((p = pfind(probe->ftp_pid)) == NULL) { + mutex_exit(&provider->ftp_mtx); + return; } - - mutex_enter(&provider->ftp_mtx); +#ifdef __FreeBSD__ + _PHOLD(p); + PROC_UNLOCK(p); +#endif /* * Disable all the associated tracepoints (for fully enabled probes). @@ -1030,9 +1154,6 @@ fasttrap_pid_disable(void *arg, dtrace_i if (provider->ftp_retired && !provider->ftp_marked) whack = provider->ftp_marked = 1; mutex_exit(&provider->ftp_mtx); - - mutex_enter(&p->p_lock); - sprunlock(p); } else { /* * If the process is dead, we're just waiting for the @@ -1046,12 +1167,17 @@ fasttrap_pid_disable(void *arg, dtrace_i if (whack) fasttrap_pid_cleanup(); +#ifdef __FreeBSD__ + PRELE(p); +#endif if (!probe->ftp_enabled) return; probe->ftp_enabled = 0; +#if defined(sun) ASSERT(MUTEX_HELD(&cpu_lock)); +#endif fasttrap_disable_callbacks(); } @@ -1163,6 +1289,7 @@ fasttrap_proc_lookup(pid_t pid) fasttrap_bucket_t *bucket; fasttrap_proc_t *fprc, *new_fprc; + bucket = &fasttrap_procs.fth_table[FASTTRAP_PROCS_INDEX(pid)]; mutex_enter(&bucket->ftb_mtx); @@ -1189,6 +1316,10 @@ fasttrap_proc_lookup(pid_t pid) new_fprc->ftpc_pid = pid; new_fprc->ftpc_rcount = 1; new_fprc->ftpc_acount = 1; +#if !defined(sun) + mutex_init(&new_fprc->ftpc_mtx, "fasttrap proc mtx", MUTEX_DEFAULT, + NULL); +#endif *** DIFF OUTPUT TRUNCATED AT 1000 LINES ***