From owner-svn-src-all@freebsd.org Mon Oct 23 20:50:09 2017 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EA329E54E70; Mon, 23 Oct 2017 20:50:09 +0000 (UTC) (envelope-from shurd@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B9C0B8420D; Mon, 23 Oct 2017 20:50:09 +0000 (UTC) (envelope-from shurd@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v9NKo8M3051201; Mon, 23 Oct 2017 20:50:08 GMT (envelope-from shurd@FreeBSD.org) Received: (from shurd@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v9NKo8fP051200; Mon, 23 Oct 2017 20:50:08 GMT (envelope-from shurd@FreeBSD.org) Message-Id: <201710232050.v9NKo8fP051200@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: shurd set sender to shurd@FreeBSD.org using -f From: Stephen Hurd Date: Mon, 23 Oct 2017 20:50:08 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r324937 - head/sys/net X-SVN-Group: head X-SVN-Commit-Author: shurd X-SVN-Commit-Paths: head/sys/net X-SVN-Commit-Revision: 324937 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Oct 2017 20:50:10 -0000 Author: shurd Date: Mon Oct 23 20:50:08 2017 New Revision: 324937 URL: https://svnweb.freebsd.org/changeset/base/324937 Log: Some cache related optimizations 1. prefetch 128 bytes of mbufs. 2. Re-order filling the pkt_info so cache stalls happen at the end 3. Define empty prefetch2cachelines() macro when the function isn't present. Provides small performance improvments on some hardware Reviewed by: sbruno Approved by: sbruno (mentor) Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D12447 Modified: head/sys/net/iflib.c Modified: head/sys/net/iflib.c ============================================================================== --- head/sys/net/iflib.c Mon Oct 23 20:32:18 2017 (r324936) +++ head/sys/net/iflib.c Mon Oct 23 20:50:08 2017 (r324937) @@ -1227,8 +1227,17 @@ prefetch(void *x) { __asm volatile("prefetcht0 %0" :: "m" (*(unsigned long *)x)); } +static __inline void +prefetch2cachelines(void *x) +{ + __asm volatile("prefetcht0 %0" :: "m" (*(unsigned long *)x)); +#if (CACHE_LINE_SIZE < 128) + __asm volatile("prefetcht0 %0" :: "m" (*(((unsigned long *)x)+CACHE_LINE_SIZE/(sizeof(unsigned long))))); +#endif +} #else #define prefetch(x) +#define prefetch2cachelines(x) #endif static void @@ -3086,12 +3095,12 @@ iflib_encap(iflib_txq_t txq, struct mbuf **m_headp) m_head = *m_headp; pkt_info_zero(&pi); - pi.ipi_len = m_head->m_pkthdr.len; pi.ipi_mflags = (m_head->m_flags & (M_VLANTAG|M_BCAST|M_MCAST)); - pi.ipi_csum_flags = m_head->m_pkthdr.csum_flags; - pi.ipi_vtag = (m_head->m_flags & M_VLANTAG) ? m_head->m_pkthdr.ether_vtag : 0; pi.ipi_pidx = pidx; pi.ipi_qsidx = txq->ift_id; + pi.ipi_len = m_head->m_pkthdr.len; + pi.ipi_csum_flags = m_head->m_pkthdr.csum_flags; + pi.ipi_vtag = (m_head->m_flags & M_VLANTAG) ? m_head->m_pkthdr.ether_vtag : 0; /* deliberate bitwise OR to make one condition */ if (__predict_true((pi.ipi_csum_flags | pi.ipi_vtag))) { @@ -3323,10 +3332,10 @@ _ring_peek_one(struct ifmp_ring *r, int cidx, int offs prefetch(items[(cidx + offset) & (size-1)]); if (remaining > 1) { - prefetch(&items[next]); - prefetch(items[(cidx + offset + 1) & (size-1)]); - prefetch(items[(cidx + offset + 2) & (size-1)]); - prefetch(items[(cidx + offset + 3) & (size-1)]); + prefetch2cachelines(&items[next]); + prefetch2cachelines(items[(cidx + offset + 1) & (size-1)]); + prefetch2cachelines(items[(cidx + offset + 2) & (size-1)]); + prefetch2cachelines(items[(cidx + offset + 3) & (size-1)]); } return (__DEVOLATILE(struct mbuf **, &r->items[(cidx + offset) & (size-1)])); }