From owner-freebsd-current@FreeBSD.ORG Thu Aug 5 03:29:25 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3174416A4CE; Thu, 5 Aug 2004 03:29:25 +0000 (GMT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id DAFCE43D2F; Thu, 5 Aug 2004 03:29:24 +0000 (GMT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.11/8.12.11) with ESMTP id i753S71A022640; Wed, 4 Aug 2004 23:28:07 -0400 (EDT) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)i753S7bn022637; Wed, 4 Aug 2004 23:28:07 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Wed, 4 Aug 2004 23:28:07 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: cperciva@FreeBSD.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: current@FreeBSD.org Subject: Optimization opportunity: don't recurse callout mutex in timeout_reset() X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Aug 2004 03:29:25 -0000 I'm currently walking our somewhat overburdened interrupt path handling for network packets looking for optimization opportunities. The good (and bad) news is that there are a lot. I'll be sending out some e-mail with some of them. Here's the first. I've sent it to Colin but CC'd current@ because Colin has expressed interest in the callout code previously :-). timeout_reset() is called from the TCP code pretty frequently. I observed that the callout code path is recursing the callout mutex. Here's the KTR trace: 12053 0 596 LOCK (spin mutex) callout r = 0 at ../../../kern/kern_timeout.c:488 callout_reset(). 12054 0 1056 LOCK (spin mutex) callout r = 1 at ../../../kern/kern_timeout.c:525 12055 0 580 UNLOCK (spin mutex) callout r = 1 at ../../../kern/kern_timeout.c:562 12056 0 17320 UNLOCK (spin mutex) callout r = 0 at ../../../kern/kern_timeout.c:515 The line numbers are off because they're from the rwatson_netperf branch, which includes tracing and profiling for callouts. The translation to CVS locations is that :488 is the spin lock acquire in callout_reset(), :525 is the spin lock acquire in _callout_stop_safe(), :562 is the spin lock drop in _callout_stop_safe(), and :515 is the spin lock drop in callout_reset(). Eliminating the recursion would be beneficial, if we could. Thanks! Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Principal Research Scientist, McAfee Research