From owner-svn-src-head@FreeBSD.ORG Tue Jun 23 15:05:39 2009 Return-Path: Delivered-To: svn-src-head@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7C13E1065705; Tue, 23 Jun 2009 15:05:39 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 3A7278FC1A; Tue, 23 Jun 2009 15:05:39 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id D0E9246B35; Tue, 23 Jun 2009 11:05:38 -0400 (EDT) Date: Tue, 23 Jun 2009 16:05:38 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Andrew Gallatin In-Reply-To: <4A40DBD4.3070904@cs.duke.edu> Message-ID: References: <200906222308.n5MN856I055711@svn.freebsd.org> <4A40DBD4.3070904@cs.duke.edu> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: svn-src-head@FreeBSD.org, svn-src-all@FreeBSD.org, src-committers@FreeBSD.org, Andre Oppermann Subject: Re: svn commit: r194672 - in head/sys: kern netinet sys X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Jun 2009 15:05:40 -0000 On Tue, 23 Jun 2009, Andrew Gallatin wrote: > This is consistent across runs. Lockstat output for 10 seconds in the > middle of a run is very interesting and shows a huge reduction in lock > contention. I continue to wonder if we should be deferring TCP reassembly to the user thread when there's an effective pipeline going on -- currently we still contend on the inpcb lock heavily, generally, and pipelining the work via a separate queue (which is what Linux does, I believe) would mean the ithread could spend more time doing other things instead of spinning. Robert N M Watson Computer Laboratory University of Cambridge > > Before: > > Adaptive mutex spin: 369333 events in 10.017 seconds (36869 events/sec) > > Count indv cuml rcnt nsec Lock Caller > ------------------------------------------------------------------------------- > 303685 82% 82% 0.00 1080 0xffffff000f2f98d0 recvit+0x21 > 63847 17% 100% 0.00 25 0xffffff000f2f98d0 ip_input+0xad > 1788 0% 100% 0.00 172 0xffffff0001c57c08 > intr_event_execute_handlers+0x100 > 8 0% 100% 0.00 389 vm_page_queue_mtx trap+0x4ce > 1 0% 100% 0.00 30 0xffffff8000251598 ithread_loop+0x8e > 1 0% 100% 0.00 720 0xffffff8000251598 uhub_read_port_status+0x2d > 1 0% 100% 0.00 1639 0xffffff000f477190 vm_fault+0x112 > 1 0% 100% 0.00 1 0xffffff001fecce10 mxge_intr+0x425 > 1 0% 100% 0.00 1332 0xffffff0001845600 clnt_reconnect_call+0x105 > ------------------------------------------------------------------------------- > > Adaptive mutex block: 89 events in 10.017 seconds (9 events/sec) > > Count indv cuml rcnt nsec Lock Caller > ------------------------------------------------------------------------------- > 83 93% 93% 0.00 20908 0xffffff000f2f98d0 tcp_input+0xd96 > 3 3% 97% 0.00 45234 0xffffff8000259f08 fork_exit+0x118 > 3 3% 100% 0.00 44862 0xffffff8000251598 fork_exit+0x118 > ------------------------------------------------------------------------------- > > > After: > > Adaptive mutex spin: 105102 events in 10.020 seconds (10490 events/sec) > > Count indv cuml rcnt nsec Lock Caller > ------------------------------------------------------------------------------- > 75886 72% 72% 0.00 2860 0xffffff0001fdde20 ip_input+0xad > 28418 27% 99% 0.00 1355 0xffffff0001fdde20 recvit+0x21 > 779 1% 100% 0.00 171 0xffffff0001642808 > intr_event_execute_handlers+0x100 > 7 0% 100% 0.00 670 vm_page_queue_mtx trap+0x4ce > 5 0% 100% 0.00 46 0xffffff001fecce10 mxge_intr+0x425 > 1 0% 100% 0.00 105 vm_page_queue_mtx trap_pfault+0x142 > 1 0% 100% 0.00 568 0xffffff8000251598 usb_process+0xd8 > 1 0% 100% 0.00 880 0xffffff8000251598 ithread_loop+0x8e > 1 0% 100% 0.00 233 0xffffff001a224578 vm_fault+0x112 > 1 0% 100% 0.00 60 0xffffff001a1759b8 syscall+0x28f > 1 0% 100% 0.00 809 0xffffff0001846000 clnt_reconnect_call+0x105 > 1 0% 100% 0.00 1139 0xffffff0001fdde20 kern_recvit+0x1d4 > ------------------------------------------------------------------------------- > > Adaptive mutex block: 88 events in 10.020 seconds (9 events/sec) > > Count indv cuml rcnt nsec Lock Caller > ------------------------------------------------------------------------------- > 80 91% 91% 0.00 25891 0xffffff0001fdde20 tcp_input+0xd96 > 3 3% 94% 0.00 45979 0xffffff8000259f08 fork_exit+0x118 > 3 3% 98% 0.00 45886 0xffffff8000251598 fork_exit+0x118 > 1 1% 99% 0.00 38254 0xffffff8000259f08 > intr_event_execute_handlers+0x100 > 1 1% 100% 0.00 79858 0xffffff001a1760f8 kern_wait+0x7ee > ------------------------------------------------------------------------------- > > > Drew >