From owner-freebsd-performance@FreeBSD.ORG Sat Jun 17 12:50:49 2006 Return-Path: X-Original-To: performance@FreeBSD.org Delivered-To: freebsd-performance@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 473E716A482 for ; Sat, 17 Jun 2006 12:50:49 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.FreeBSD.org (Postfix) with ESMTP id F0B6943D45 for ; Sat, 17 Jun 2006 12:50:48 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 85CDC46BC3 for ; Sat, 17 Jun 2006 08:50:48 -0400 (EDT) Date: Sat, 17 Jun 2006 13:50:48 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: performance@FreeBSD.org Message-ID: <20060617134402.O8526@fledge.watson.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Subject: HZ=100: not necessarily better? X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jun 2006 12:50:49 -0000 Scott asked me if I could take a look at the impact of changing HZ for some simple TCP performance tests. I ran the first couple, and got some results that were surprising, so I thought I'd post about them and ask people who are interested if they could do some investigation also. The short of it is that we had speculated that the increased CPU overhead of a higher HZ would be significant when it came to performance measurement, but in fact, I measure improved performance under high HTTP load with a higher HZ. This was, of course, the reason we first looked at increasing HZ: improving timer granularity helps improve the performance of network protocols, such as TCP. Recent popular opinion has swung in the opposite direction, that higher HZ overhead outweighs this benefit, and I think we should be cautious and do a lot more investigating before assuming that is true. Simple performance results below. Two boxes on a gig-e network with if_em ethernet cards, one running a simple web server hosting 100 byte pages, and the other downloading them in parallel (netrate/http and netrate/httpd). The performance difference is marginal, but at least in the SMP case, likely more than a measurement error or cache alignment fluke. Results are transactions/second sustained over a 30 second test -- bigger is better; box is a dual xeon p4 with HTT; 'vendor.*' are the default 7-CURRENT HZ setting (1000) and 'hz.*' are the HZ=100 versions of the same kernels. Regardless, there wasn't an obvious performance improvement by reducing HZ from 1000 to 100. Results may vary, use only as directed. What we might want to explore is using a programmable timer to set up high precision timeouts, such as TCP timers, while keeping base statistics profiling and context switching at 100hz. I think phk has previously proposed doing this with the HPET timer. I'll run some more diverse tests today, such as raw bandwidth tests, pps on UDP, and so on, and see where things sit. The reduced overhead should be measurable in cases where the test is CPU-bound and there's no clear benefit to more accurate timing, such as with TCP, but it would be good to confirm that. Robert N M Watson Computer Laboratory University of Cambridge peppercorn:~/tmp/netperf/hz> ministat *SMP x hz.SMP + vendor.SMP +--------------------------------------------------------------------------+ |xx x xx x xx x + + + + + ++ + ++| | |_______A________| |_____________A___M________| | +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 10 13715 13793 13750 13751.1 29.319883 + 10 13813 13970 13921 13906.5 47.551726 Difference at 95.0% confidence 155.4 +/- 37.1159 1.13009% +/- 0.269913% (Student's t, pooled s = 39.502) peppercorn:~/tmp/netperf/hz> ministat *UP x hz.UP + vendor.UP +--------------------------------------------------------------------------+ |x x xx x xx+ ++x+ ++ * + + +| | |_________M_A_______|___|______M_A____________| | +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 10 14067 14178 14116 14121.2 31.279386 + 10 14141 14257 14170 14175.9 33.248058 Difference at 95.0% confidence 54.7 +/- 30.329 0.387361% +/- 0.214776% (Student's t, pooled s = 32.2787)