From owner-freebsd-performance@FreeBSD.ORG Tue Nov 25 20:09:20 2008 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3DB6B1065701 for ; Tue, 25 Nov 2008 20:09:20 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from qw-out-2122.google.com (qw-out-2122.google.com [74.125.92.25]) by mx1.freebsd.org (Postfix) with ESMTP id E42D38FC1A for ; Tue, 25 Nov 2008 20:09:19 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by qw-out-2122.google.com with SMTP id 9so53448qwb.7 for ; Tue, 25 Nov 2008 12:09:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:sender :to:subject:cc:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:references :x-google-sender-auth; bh=iKRb8wla2kjbbbA9S/rRunxwPpeYvptahT0Gta1tMpY=; b=Cwy+4ky8dbIbC9MTw5hQB/69PUGEfoi42DmhA6Zl0FuM/QflS3FnZDoLgxT6eoJmnz POEkaxBBrDVEg6e/mw9YiKgBQZmjessBrevmhruWRotYu8I7ms+//ZNjzbCoL3TAGyvG 8uOmuXFUS6DyOo/hqwd/S1D/OwG/cxSFc2z8s= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version :content-type:content-transfer-encoding:content-disposition :references:x-google-sender-auth; b=gP5pDpCu62jq4g7FNdkUbpqiMaGm7EAcVS1CWHF1YBefk/vIpIh7NMEhvuaY6fwDrq NvP9Jz623v+COQYvgfwGGka824iRHlCuIcd+fjsS+21svU8kmq7amUyEf9jB1x9Eco22 OnYDD75uKiUcxgf95tnIQuyXlo6Pg9M2c1aeI= Received: by 10.215.38.2 with SMTP id q2mr4294309qaj.7.1227643759280; Tue, 25 Nov 2008 12:09:19 -0800 (PST) Received: by 10.215.41.1 with HTTP; Tue, 25 Nov 2008 12:09:19 -0800 (PST) Message-ID: Date: Tue, 25 Nov 2008 15:09:19 -0500 From: "Adrian Chadd" Sender: adrian.chadd@gmail.com To: vadim_nuclight@mail.ru In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: X-Google-Sender-Auth: 02e52403a4d4b05c Cc: freebsd-performance@freebsd.org Subject: Re: hwpmc granularity and 6.4 network performance X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Nov 2008 20:09:20 -0000 A few things! * Since you've changed two things - hwpmc _AND_ the kernel version - you can't easily conclude which one (if any!) has any influence on Giant showing up in your top output. I suggest recompiling without hwpmc and seeing if the behaviour changes. * The gprof utility expects something resembling "time" for the sampling data, but pmcstat doesn't record time, it records "events". The counts you see in gprof are "events", so change "seconds" to "events" in your reading of the gprof output. * I don't know if the backported pmc to 6.4 handles stack call graphs or not. Easy way to check - pmcstat -R sample.out | more ; see if you just see "sample" lines or "sample" and "callgraph" lines. * I bet that ipfw_chk is a big enough hint. How big is your ipfw ruleset? :) Adrian 2008/11/24 Vadim Goncharov : > Hi! > > I've recently perfromed upgrade of busy production router from 6.2 to 6.4-PRE. > I have added two lines to my kernel config and did usual make buildkernel: > > device hwpmc # Driver (also a loadable module) > options HWPMC_HOOKS # Other necessary kernel hooks > > After rebooting with new world and kernel, I've noticed that CPU load has > slightly increased (not measured, it is different every second anyway, as > users do not genereate steady traffic), and in top -S 'swi1: net' became > often in state *Giant, but it not used to do so on 6.2, while kernel config > did not changed much, and device polling is still used. What could happen > to this? > > Another question, I've read "Sixty second HWPMC howto" and tried to find out > what exactly eats my CPU. BTW, that instruction did not apply exactly on my > machine, this is what I did: > > # cd /tmp > # pmcstat -S instructions -O /tmp/sample.out > # pmcstat -R /tmp/sample.out -k /boot/kernel/kernel -g > # gprof /boot/kernel/kernel p4-instr-retired/kernel.gmon > kernel.gmon.result > > Now in file kernel.gmon.result I see the following: > > granularity: each sample hit covers 4 byte(s) for 0.00% of 692213.00 seconds > > called/total parents > index %time self descendents called+self name index > called/total children > > > [1] 31.7 219129.00 0.00 ipfw_chk [1] > > ----------------------------------------------- > > > [...] > > Why does it show 0.00 in this column ? > > On next listing, flat profile, I see more readable listing, but columns are > empty again: > > granularity: each sample hit covers 4 byte(s) for 0.00% of 692213.00 seconds > > % cumulative self self total > time seconds seconds calls ms/call ms/call name > 31.7 219129.00 219129.00 ipfw_chk [1] > 10.4 291179.00 72050.00 bcmp [2] > 6.1 333726.00 42547.00 rn_match [3] > 2.7 352177.00 18451.00 generic_bzero [4] > 2.4 368960.00 16783.00 strncmp [5] > > OK, I can conclude from this that I should optimize my ipfw ruleset, but > that's all. I know from sources that ipfw_chk() is a big function with a > bunch of 'case's in a large 'switch'. I want to know which parts of that > switch are executed more often. It says in listing that granularity is > 4 bytes, I assume that it has a sample for each of 4-byte chunks of binary > code, so that it must have such information. My kernel is compiled with: > > makeoptions DEBUG=-g > > so kgdb does know where are instructions for each line of source code. > How can I obtain this info from profiling? It also would be useful to know > which places do calls to that bcmp() and rn_match(). > > -- > WBR, Vadim Goncharov. ICQ#166852181 mailto:vadim_nuclight@mail.ru > [Moderator of RU.ANTI-ECOLOGY][FreeBSD][http://antigreen.org][LJ:/nuclight] > > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org" >