From owner-svn-src-head@freebsd.org Wed Jun 13 18:26:03 2018 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 79DA41003CBF for ; Wed, 13 Jun 2018 18:26:03 +0000 (UTC) (envelope-from oliver.pinter@hardenedbsd.org) Received: from mail-yw0-x242.google.com (mail-yw0-x242.google.com [IPv6:2607:f8b0:4002:c05::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0ED607240F for ; Wed, 13 Jun 2018 18:26:03 +0000 (UTC) (envelope-from oliver.pinter@hardenedbsd.org) Received: by mail-yw0-x242.google.com with SMTP id s201-v6so1213495ywg.8 for ; Wed, 13 Jun 2018 11:26:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hardenedbsd-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=qDKuEZBAyR4eUEcG1ltskoSFEnyhYsbBbB2BAIQJ5PY=; b=18ff+0n1gVj2wZN9EszQq6hBMNNi/Xbr4IhKwl70mm9PekPlDYZ7KfFPV1g+WMywCo eKHcqqpBO1fxsRnhiJFZBl/JdflHWGpFFCzH6Ro+aZtrAwPkMvbLIgnVaDLQxcJjOezQ HZf9fgAXlfljU0gEXAsUB0weVBEzHLbu04Cxxw0Egi/v5YjtO0wKF0uC7KhZ8meYbk9w 96jbe9/Wyq0WXEOl658pJ9V9eUPb5vgIcNefVaKO4QrZ//nqAjFfbGFzGpQ6alKXvc0N Ybtr8zwGN3wLu5f+YC+4jHz18ycZ8xvCMKeDVW4pYwBCD5g9e9+ISdcm7oQqAzMwYABL thiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=qDKuEZBAyR4eUEcG1ltskoSFEnyhYsbBbB2BAIQJ5PY=; b=i/5tJyqW88mzAFhLaszNSNrfFoOgAyyK9RWq/5i6eiwchWPNaRHhhFhUE42jMYGEw1 vdyIzxx9nbMA9pQaOFubHjRtE8yHTSYrxO1tsuPP9K1ZVRu89dm4dcADRk3+f4MvO0KZ Bfjoke6IjaRSE/ijk8NNN2NkcanUyxrGVRM4FNkv3I3/y2KYAGAWtblPl1EA9OJZMFVL KQKyVfxfsvSLuNxwZoxJ30TQAwaaQDIzOD0+MiLI442bOaCJv6mVzOQBfth88WS4AKhp cAP4zQa+ryc49O9M5USqvt1CAtH+c5R1AgTosPPKn3npQ4E9DVHZ27Bez7QmQPC/s5I9 pLCw== X-Gm-Message-State: APt69E1Sd5kGKCJqjUtaG3Jy5RhgPpzvczp1gm5KUixTxkEwS138/Uoh /zAhz0NFGm49cWFf8fyAeSzVdVW/eUukoCS5LNxVlQ== X-Google-Smtp-Source: ADUXVKJjo/gvtyr57Bmv/aQm+wp9VjT6j42/5ZM6Y9UxOiElGkMFv/Hnx4flx1v/E8XnwjaaDHPb91N5cIUPXgsLu80= X-Received: by 2002:a81:5f55:: with SMTP id t82-v6mr2938389ywb.382.1528914362514; Wed, 13 Jun 2018 11:26:02 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a25:db86:0:0:0:0:0 with HTTP; Wed, 13 Jun 2018 11:26:02 -0700 (PDT) In-Reply-To: <201806131755.w5DHtAwo073314@repo.freebsd.org> References: <201806131755.w5DHtAwo073314@repo.freebsd.org> From: Oliver Pinter Date: Wed, 13 Jun 2018 20:26:02 +0200 Message-ID: Subject: Re: svn commit: r335072 - head/sys/amd64/amd64 To: Konstantin Belousov Cc: "src-committers@freebsd.org" , "svn-src-all@freebsd.org" , "svn-src-head@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.26 X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jun 2018 18:26:03 -0000 On Wednesday, June 13, 2018, Konstantin Belousov wrote: > Author: kib > Date: Wed Jun 13 17:55:09 2018 > New Revision: 335072 > URL: https://svnweb.freebsd.org/changeset/base/335072 > > Log: > Enable eager FPU context switch by default on amd64. > > With compilers making increasing use of vector instructions the > performance benefit of lazily switching FPU state is no longer a > desirable tradeoff. Linux switched to eager FPU context switch some > time ago, and the idea was floated on the FreeBSD-current mailing list > some years ago[1]. > > Enable eager FPU context switch by default on amd64, with a > tunable/sysctl > available to turn it back off. > > [1] https://lists.freebsd.org/pipermail/freebsd-current/ > 2015-March/055198.html > > http://www.openwall.com/lists/oss-security/2018/06/13/7 > Reviewed by: jhb > Tested by: pho > Sponsored by: The FreeBSD Foundation > > Modified: > head/sys/amd64/amd64/cpu_switch.S > head/sys/amd64/amd64/fpu.c > > Modified: head/sys/amd64/amd64/cpu_switch.S > ============================================================ > ================== > --- head/sys/amd64/amd64/cpu_switch.S Wed Jun 13 17:42:55 2018 > (r335071) > +++ head/sys/amd64/amd64/cpu_switch.S Wed Jun 13 17:55:09 2018 > (r335072) > @@ -128,10 +128,10 @@ done_store_dr: > > /* have we used fp, and need a save? */ > cmpq %rdi,PCPU(FPCURTHREAD) > - jne 3f > + jne 2f > movq PCB_SAVEFPU(%r8),%r8 > clts > - cmpl $0,use_xsave > + cmpl $0,use_xsave(%rip) > jne 1f > fxsave (%r8) > jmp 2f > @@ -143,12 +143,7 @@ ctx_switch_xsave: > /* This is patched to xsaveopt if supported, see fpuinit_bsp1() */ > xsave (%r8) > movq %rcx,%rdx > -2: smsw %ax > - orb $CR0_TS,%al > - lmsw %ax > - xorl %eax,%eax > - movq %rax,PCPU(FPCURTHREAD) > -3: > +2: > /* Save is done. Now fire up new thread. Leave old vmspace. */ > movq %rsi,%r12 > movq %rdi,%r13 > @@ -238,6 +233,8 @@ done_load_dr: > movq PCB_RBX(%r8),%rbx > movq PCB_RIP(%r8),%rax > movq %rax,(%rsp) > + movq PCPU(CURTHREAD),%rdi > + call fpu_activate_sw > ret > > /* > > Modified: head/sys/amd64/amd64/fpu.c > ============================================================ > ================== > --- head/sys/amd64/amd64/fpu.c Wed Jun 13 17:42:55 2018 (r335071) > +++ head/sys/amd64/amd64/fpu.c Wed Jun 13 17:55:09 2018 (r335072) > @@ -142,6 +142,11 @@ static void fpu_clean_state(void); > SYSCTL_INT(_hw, HW_FLOATINGPT, floatingpoint, CTLFLAG_RD, > SYSCTL_NULL_INT_PTR, 1, "Floating point instructions executed in > hardware"); > > +int lazy_fpu_switch = 0; > +SYSCTL_INT(_hw, OID_AUTO, lazy_fpu_switch, CTLFLAG_RWTUN | > CTLFLAG_NOFETCH, > + &lazy_fpu_switch, 0, > + "Lazily load FPU context after context switch"); > + > int use_xsave; /* non-static for cpu_switch.S */ > uint64_t xsave_mask; /* the same */ > static uma_zone_t fpu_save_area_zone; > @@ -242,6 +247,7 @@ fpuinit_bsp1(void) > uint64_t xsave_mask_user; > bool old_wp; > > + TUNABLE_INT_FETCH("hw.lazy_fpu_switch", &lazy_fpu_switch); > if (!use_xsave) > return; > cpuid_count(0xd, 0x0, cp); > @@ -651,6 +657,45 @@ fputrap_sse(void) > return (fpetable[(mxcsr & (~mxcsr >> 7)) & 0x3f]); > } > > +static void > +restore_fpu_curthread(struct thread *td) > +{ > + struct pcb *pcb; > + > + /* > + * Record new context early in case frstor causes a trap. > + */ > + PCPU_SET(fpcurthread, td); > + > + stop_emulating(); > + fpu_clean_state(); > + pcb = td->td_pcb; > + > + if ((pcb->pcb_flags & PCB_FPUINITDONE) == 0) { > + /* > + * This is the first time this thread has used the FPU or > + * the PCB doesn't contain a clean FPU state. Explicitly > + * load an initial state. > + * > + * We prefer to restore the state from the actual save > + * area in PCB instead of directly loading from > + * fpu_initialstate, to ignite the XSAVEOPT > + * tracking engine. > + */ > + bcopy(fpu_initialstate, pcb->pcb_save, > + cpu_max_ext_state_size); > + fpurestore(pcb->pcb_save); > + if (pcb->pcb_initial_fpucw != __INITIAL_FPUCW__) > + fldcw(pcb->pcb_initial_fpucw); > + if (PCB_USER_FPU(pcb)) > + set_pcb_flags(pcb, PCB_FPUINITDONE | > + PCB_USERFPUINITDONE); > + else > + set_pcb_flags(pcb, PCB_FPUINITDONE); > + } else > + fpurestore(pcb->pcb_save); > +} > + > /* > * Device Not Available (DNA, #NM) exception handler. > * > @@ -661,7 +706,9 @@ fputrap_sse(void) > void > fpudna(void) > { > + struct thread *td; > > + td = curthread; > /* > * This handler is entered with interrupts enabled, so context > * switches may occur before critical_enter() is executed. If > @@ -675,7 +722,7 @@ fpudna(void) > > KASSERT((curpcb->pcb_flags & PCB_FPUNOSAVE) == 0, > ("fpudna while in fpu_kern_enter(FPU_KERN_NOCTX)")); > - if (PCPU_GET(fpcurthread) == curthread) { > + if (PCPU_GET(fpcurthread) == td) { > printf("fpudna: fpcurthread == curthread\n"); > stop_emulating(); > critical_exit(); > @@ -684,40 +731,24 @@ fpudna(void) > if (PCPU_GET(fpcurthread) != NULL) { > panic("fpudna: fpcurthread = %p (%d), curthread = %p > (%d)\n", > PCPU_GET(fpcurthread), PCPU_GET(fpcurthread)->td_tid, > - curthread, curthread->td_tid); > + td, td->td_tid); > } > - stop_emulating(); > - /* > - * Record new context early in case frstor causes a trap. > - */ > - PCPU_SET(fpcurthread, curthread); > + restore_fpu_curthread(td); > + critical_exit(); > +} > > - fpu_clean_state(); > +void fpu_activate_sw(struct thread *td); /* Called from the context > switch */ > +void > +fpu_activate_sw(struct thread *td) > +{ > > - if ((curpcb->pcb_flags & PCB_FPUINITDONE) == 0) { > - /* > - * This is the first time this thread has used the FPU or > - * the PCB doesn't contain a clean FPU state. Explicitly > - * load an initial state. > - * > - * We prefer to restore the state from the actual save > - * area in PCB instead of directly loading from > - * fpu_initialstate, to ignite the XSAVEOPT > - * tracking engine. > - */ > - bcopy(fpu_initialstate, curpcb->pcb_save, > - cpu_max_ext_state_size); > - fpurestore(curpcb->pcb_save); > - if (curpcb->pcb_initial_fpucw != __INITIAL_FPUCW__) > - fldcw(curpcb->pcb_initial_fpucw); > - if (PCB_USER_FPU(curpcb)) > - set_pcb_flags(curpcb, > - PCB_FPUINITDONE | PCB_USERFPUINITDONE); > - else > - set_pcb_flags(curpcb, PCB_FPUINITDONE); > - } else > - fpurestore(curpcb->pcb_save); > - critical_exit(); > + if (lazy_fpu_switch || (td->td_pflags & TDP_KTHREAD) != 0 || > + !PCB_USER_FPU(td->td_pcb)) { > + PCPU_SET(fpcurthread, NULL); > + start_emulating(); > + } else if (PCPU_GET(fpcurthread) != td) { > + restore_fpu_curthread(td); > + } > } > > void > _______________________________________________ > svn-src-head@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/svn-src-head > To unsubscribe, send any mail to "svn-src-head-unsubscribe@freebsd.org" >