From owner-svn-src-all@freebsd.org Thu Sep 27 15:24:17 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 78B6F10B10F3; Thu, 27 Sep 2018 15:24:17 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2EAD289DF5; Thu, 27 Sep 2018 15:24:17 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 29AF627E1C; Thu, 27 Sep 2018 15:24:17 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w8RFOHxc075838; Thu, 27 Sep 2018 15:24:17 GMT (envelope-from mjg@FreeBSD.org) Received: (from mjg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w8RFOHrg075837; Thu, 27 Sep 2018 15:24:17 GMT (envelope-from mjg@FreeBSD.org) Message-Id: <201809271524.w8RFOHrg075837@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mjg set sender to mjg@FreeBSD.org using -f From: Mateusz Guzik Date: Thu, 27 Sep 2018 15:24:17 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r338967 - head/sys/amd64/amd64 X-SVN-Group: head X-SVN-Commit-Author: mjg X-SVN-Commit-Paths: head/sys/amd64/amd64 X-SVN-Commit-Revision: 338967 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Sep 2018 15:24:17 -0000 Author: mjg Date: Thu Sep 27 15:24:16 2018 New Revision: 338967 URL: https://svnweb.freebsd.org/changeset/base/338967 Log: amd64: clean up copyin/copyout - move the PSL.AC comment to the fault handler - stop testing for zero-sized ops. after several minutes of package building there were no copyin calls with zero bytes and very few copyout. the semantic of returning 0 in this case is preserved - shorten exit paths by clearing %eax earlier - replace xchg with 3 movs. this is what compilers do. a naive benchmark on EPYC suggests about 1% increase in thoughput thanks to this change. - remove the useless movb %cl,%al from copyout. it looks like a leftover from many years ago Reviewed by: kib Approved by: re (gjb) Differential Revision: https://reviews.freebsd.org/D17286 Modified: head/sys/amd64/amd64/support.S Modified: head/sys/amd64/amd64/support.S ============================================================================== --- head/sys/amd64/amd64/support.S Thu Sep 27 14:57:20 2018 (r338966) +++ head/sys/amd64/amd64/support.S Thu Sep 27 15:24:16 2018 (r338967) @@ -404,10 +404,7 @@ END(fillw) .macro COPYOUT smap erms PUSH_FRAME_POINTER movq PCPU(CURPCB),%r9 - /* Trap entry clears PSL.AC */ movq $copy_fault,PCB_ONFAULT(%r9) - testq %rdx,%rdx /* anything to do? */ - jz 2f /* * Check explicitly for non-user addresses. If 486 write protection @@ -432,10 +429,20 @@ END(fillw) cmpq %rcx,%rax ja copy_fault - xchgq %rdi,%rsi - /* bcopy(%rsi, %rdi, %rdx) */ + /* + * Set up arguments for rep movs*. + */ + movq %rdi,%r8 + movq %rsi,%rdi + movq %r8,%rsi movq %rdx,%rcx + /* + * Set return value to zero. Remaining failure mode goes through + * copy_fault. + */ + xorl %eax,%eax + SMAP_DISABLE \smap .if \erms == 0 cmpq $15,%rcx @@ -447,17 +454,16 @@ END(fillw) andb $7,%cl jne 1f SMAP_ENABLE \smap - xorl %eax,%eax movq %rax,PCB_ONFAULT(%r9) POP_FRAME_POINTER ret .endif + ALIGN_TEXT 1: rep movsb + SMAP_ENABLE \smap -2: - xorl %eax,%eax movq %rax,PCB_ONFAULT(%r9) POP_FRAME_POINTER ret @@ -487,8 +493,6 @@ END(copyout_smap_erms) PUSH_FRAME_POINTER movq PCPU(CURPCB),%r9 movq $copy_fault,PCB_ONFAULT(%r9) - testq %rdx,%rdx /* anything to do? */ - jz 2f /* * make sure address is valid @@ -500,10 +504,13 @@ END(copyout_smap_erms) cmpq %rcx,%rax ja copy_fault - xchgq %rdi,%rsi + movq %rdi,%r8 + movq %rsi,%rdi + movq %r8,%rsi movq %rdx,%rcx - movb %cl,%al + xorl %eax,%eax + SMAP_DISABLE \smap .if \erms == 0 cmpq $15,%rcx @@ -511,22 +518,20 @@ END(copyout_smap_erms) shrq $3,%rcx /* copy longword-wise */ rep movsq - movb %al,%cl + movb %dl,%cl andb $7,%cl /* copy remaining bytes */ jne 1f SMAP_ENABLE \smap - xorl %eax,%eax movq %rax,PCB_ONFAULT(%r9) POP_FRAME_POINTER ret .endif + ALIGN_TEXT 1: rep movsb SMAP_ENABLE \smap -2: - xorl %eax,%eax movq %rax,PCB_ONFAULT(%r9) POP_FRAME_POINTER ret @@ -549,9 +554,9 @@ ENTRY(copyin_smap_erms) END(copyin_smap_erms) ALIGN_TEXT + /* Trap entry clears PSL.AC */ copy_fault: - movq PCPU(CURPCB),%rdx - movq $0,PCB_ONFAULT(%rdx) + movq $0,PCB_ONFAULT(%r9) movl $EFAULT,%eax POP_FRAME_POINTER ret