From owner-svn-src-head@freebsd.org Sun Sep 24 18:12:11 2017 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 69F35E2CC4D; Sun, 24 Sep 2017 18:12:11 +0000 (UTC) (envelope-from shurd@sasktel.net) Received: from mail101c7.megamailservers.com (mail501c7.megamailservers.com [209.235.141.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BF17E8094E; Sun, 24 Sep 2017 18:12:10 +0000 (UTC) (envelope-from shurd@sasktel.net) X-Authenticated-User: hurds.sasktel.net X-VIP: 69.49.109.87 Received: from [192.168.0.33] (ip72-194-73-141.oc.oc.cox.net [72.194.73.141]) (authenticated bits=0) by mail101c7.megamailservers.com (8.14.9/8.13.1) with ESMTP id v8OHvOwm032555; Sun, 24 Sep 2017 13:57:26 -0400 Subject: Re: svn commit: r323942 - head/sys/net To: Hans Petter Selasky Cc: Stephen Hurd , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org References: <201709230135.v8N1ZE6S063264@repo.freebsd.org> <283397c7-a01e-3776-7ed3-b64d68003d0b@sasktel.net> <6F5DC92C-2CF6-4A33-9663-BFECB7DB65F2@lists.zabbadoz.net> <89d68ff8-84ed-83a6-4e77-9a321babe2fe@sasktel.net> <3601ee57-2bf5-036a-a3d1-a4795847d0ec@selasky.org> From: Stephen Hurd Message-ID: <311ab67b-7d37-69aa-8a7b-cbd5350e51c0@sasktel.net> Date: Sun, 24 Sep 2017 10:57:23 -0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:51.0) Gecko/20100101 Firefox/51.0 SeaMonkey/2.48 MIME-Version: 1.0 In-Reply-To: <3601ee57-2bf5-036a-a3d1-a4795847d0ec@selasky.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-CTCH-RefID: str=0001.0A020205.59C7F209.0018, ss=1, re=0.000, recu=0.000, reip=0.000, cl=1, cld=1, fgs=0 X-CTCH-VOD: Unknown X-CTCH-Spam: Unknown X-CTCH-Score: 0.000 X-CTCH-Rules: X-CTCH-Flags: 0 X-CTCH-ScoreCust: 0.000 X-CSC: 0 X-CHA: v=2.2 cv=V9cN6avi c=1 sm=1 tr=0 a=l4Y+EJuLrT/8f1z5FvEQ1g==:117 a=l4Y+EJuLrT/8f1z5FvEQ1g==:17 a=IkcTkHD0fZMA:10 a=2pFDieIRxZvowp__EMcA:9 a=QEXdDO2ut3YA:10 X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Sep 2017 18:12:11 -0000 Hans Petter Selasky wrote: > On 09/24/17 01:46, Stephen Hurd wrote: >> Basically, it changed from this: >> >> foreach (mbuf in rx) { >> if (lro && tcp_lro_rx(mbuf) == 0) >> continue; >> if_input(mbuf) >> } >> >> To this: >> >> prev_mbuf = first_mbuf = NULL; >> foreach (mbuf in rx) { >> if (lro && tcp_lro_rx(mbuf) == 0) >> continue; >> if (prev_mbuf) { >> prev_mbuf->m_nextpkt = mbuf; >> prev_mbuf = mbuf; >> } >> else { >> first_mbuf = prev_mbuf = mbuf; >> } >> } >> >> if (first_mbuf) >> if_input(first_mbuf); >> >> So while before it called if_input() for each separate mbuf that was >> not LROed, it now builds a chain of mbufs that were not LROed, and >> makes a single call to if_input() with the whole chain. For cases >> like packet forwarding where no packets are LROed, performance is >> better. >> > > Can such a similar logic be applied inside TCP LRO aswell? It looks like it would be more complex to do a similar thing in tcp_lro.c, and I'm not certain it would help much except in cases with a large number of streams that mostly end up not being coalesced. Taking a quick look, tcp_lro_flush() would need to be modified to return an mbuf head and tail, then the caller would need to be responsible for combining them into a single mbuf chain and calling if_input(). Either that, or an mbuf tail could be passed into tcp_lro_flush(), the tail modified in there, and an mbuf head returned... that way it would work something like this: The caller would be something like this: m_head = m_tail = NULL; LIST_FOREACH(le, bucket, hash_next) { head = tcp_lro_flush(lc, le, &m_tail); if (m_head == NULL) m_head = head; } if (m_head) if_input(m_head); And tcp_lro_flush() would be something like this: struct mbuf *tcp_lro_flush(struct lro_ctrl *lc, struct lro_entry *le, struct mbuf **tail) { ... if (*tail) *tail->m_next = le->m_head; *tail = le->m_tail; ... return le->m_head; } Hrm, maybe it wouldn't be all that difficult after all. :-) I'll be driving across the country later this week, so I don't want to start poking into LRO then disappear, so if nobody else tries it out before then, I should take a look in a couple weeks.