Date: Sat, 2 Jun 2018 05:48:44 +0000 (UTC) From: Bruce Evans <bde@FreeBSD.org> To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r334522 - in head/sys: amd64/amd64 amd64/include i386/i386 i386/include Message-ID: <201806020548.w525miR7019160@repo.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: bde Date: Sat Jun 2 05:48:44 2018 New Revision: 334522 URL: https://svnweb.freebsd.org/changeset/base/334522 Log: Fix high resolution kernel profiling just enough to not crash at boot time, especially for SMP. If configured, it turns itself on at boot time for calibration, so is fragile even if never otherwise used. Both types of kernel profiling were supposed to use a global spinlock in the SMP case. If hi-res profiling is configured (but not necessarily used), this was supposed to be optimized by only using it when necessary, and slightly more efficiently, in asm. But it was not done at all for mcount entry where it is necessary. This caused crashes in the SMP case when either type of profiling was enabled. For mcount exit, it only caused wrong times. The times were wrongest with an i8254 timer since using that requires exclusive access to the hardware. The i8254 timer was too slow to use here 20 years ago and is much less usable now, but it is the default for the SMP case since TSCs weren't invariant when SMP was new. Do the locking in all hi-res SMP cases for simplicity. Calibration uses special asms, and the clobber lists in these were sort of inverted. They contained the arg and return registers which are not clobbered, but on amd64 they didn't contain the residue of the call-used registers which may be clobbered (%r10 and %r11). This usually caused hangs at boot time. This usually affected even the UP case. Modified: head/sys/amd64/amd64/prof_machdep.c head/sys/amd64/include/profile.h head/sys/i386/i386/prof_machdep.c head/sys/i386/include/profile.h Modified: head/sys/amd64/amd64/prof_machdep.c ============================================================================== --- head/sys/amd64/amd64/prof_machdep.c Sat Jun 2 04:37:37 2018 (r334521) +++ head/sys/amd64/amd64/prof_machdep.c Sat Jun 2 05:48:44 2018 (r334522) @@ -56,6 +56,20 @@ static int cputime_prof_active; #endif /* GUPROF */ #ifdef __GNUCLIKE_ASM +#if defined(SMP) && defined(GUPROF) +#define MPLOCK " \n\ + movl $1,%edx \n\ +9: \n\ + xorl %eax,%eax \n\ + lock \n\ + cmpxchgl %edx,mcount_lock \n\ + jne 9b \n" +#define MPUNLOCK "movl $0,mcount_lock \n" +#else /* !(SMP && GUPROF) */ +#define MPLOCK +#define MPUNLOCK +#endif /* SMP && GUPROF */ + __asm(" \n\ GM_STATE = 0 \n\ GMON_PROF_OFF = 3 \n\ @@ -111,8 +125,10 @@ __mcount: \n\ movq 7*8(%rsp),%rsi \n\ \n\ pushfq \n\ - cli \n\ - call mcount \n\ + cli \n" + MPLOCK " \n\ + call mcount \n" + MPUNLOCK " \n\ popfq \n\ popq %r9 \n\ popq %r8 \n\ @@ -163,8 +179,10 @@ GMON_PROF_HIRES = 4 \n\ pushq %r9 \n\ movq 7*8(%rsp),%rdi \n\ pushfq \n\ - cli \n\ - call mexitcount \n\ + cli \n" + MPLOCK " \n\ + call mexitcount \n" + MPUNLOCK " \n\ popfq \n\ popq %r9 \n\ popq %r8 \n\ Modified: head/sys/amd64/include/profile.h ============================================================================== --- head/sys/amd64/include/profile.h Sat Jun 2 04:37:37 2018 (r334521) +++ head/sys/amd64/include/profile.h Sat Jun 2 05:48:44 2018 (r334522) @@ -64,11 +64,11 @@ __asm __volatile("pushq %0; call __mcount; popq %%rcx" \ : \ : "i" (label) \ - : "ax", "dx", "cx", "di", "si", "r8", "r9", "memory") + : "cx", "r10", "r11", "memory") #define MEXITCOUNT_OVERHEAD() \ __asm __volatile("call .mexitcount; 1:" \ : : \ - : "ax", "dx", "cx", "di", "si", "r8", "r9", "memory") + : "r10", "r11", "memory") #define MEXITCOUNT_OVERHEAD_GETLABEL(labelp) \ __asm __volatile("movq $1b,%0" : "=rm" (labelp)) #else Modified: head/sys/i386/i386/prof_machdep.c ============================================================================== --- head/sys/i386/i386/prof_machdep.c Sat Jun 2 04:37:37 2018 (r334521) +++ head/sys/i386/i386/prof_machdep.c Sat Jun 2 05:48:44 2018 (r334522) @@ -67,6 +67,20 @@ static int cputime_prof_active; #endif /* GUPROF */ #ifdef __GNUCLIKE_ASM +#if defined(SMP) && defined(GUPROF) +#define MPLOCK " \n\ + movl $1,%edx \n\ +9: \n\ + xorl %eax,%eax \n\ + lock \n\ + cmpxchgl %edx,mcount_lock \n\ + jne 9b \n" +#define MPUNLOCK "movl $0,mcount_lock \n" +#else /* !(SMP && GUPROF) */ +#define MPLOCK +#define MPUNLOCK +#endif /* SMP && GUPROF */ + __asm(" \n\ GM_STATE = 0 \n\ GMON_PROF_OFF = 3 \n\ @@ -110,8 +124,10 @@ __mcount: \n\ pushfl \n\ pushl %eax \n\ pushl %edx \n\ - cli \n\ - call mcount \n\ + cli \n" + MPLOCK " \n\ + call mcount \n" + MPUNLOCK " \n\ addl $8,%esp \n\ popfl \n\ .mcount_exit: \n\ @@ -155,8 +171,10 @@ GMON_PROF_HIRES = 4 \n\ movl 8(%esp),%eax \n\ pushfl \n\ pushl %eax \n\ - cli \n\ - call mexitcount \n\ + cli \n" + MPLOCK " \n\ + call mexitcount \n" + MPUNLOCK " \n\ addl $4,%esp \n\ popfl \n\ popl %eax \n\ Modified: head/sys/i386/include/profile.h ============================================================================== --- head/sys/i386/include/profile.h Sat Jun 2 04:37:37 2018 (r334521) +++ head/sys/i386/include/profile.h Sat Jun 2 05:48:44 2018 (r334522) @@ -64,11 +64,11 @@ __asm __volatile("pushl %0; call __mcount; popl %%ecx" \ : \ : "i" (label) \ - : "ax", "dx", "cx", "memory") + : "cx", "memory") #define MEXITCOUNT_OVERHEAD() \ __asm __volatile("call .mexitcount; 1:" \ : : \ - : "ax", "dx", "cx", "memory") + : "cx", "memory") #define MEXITCOUNT_OVERHEAD_GETLABEL(labelp) \ __asm __volatile("movl $1b,%0" : "=rm" (labelp)) #else
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201806020548.w525miR7019160>