Date: Sun, 26 Dec 2004 07:10:59 +0000 (GMT) From: Robert Watson <rwatson@freebsd.org> To: =?ISO-8859-1?Q?Jo=E3o_Carlos_Mendes_Lu=EDs?= <jonny@jonny.eng.br> Cc: freebsd-net@freebsd.org Subject: Re: %cpu in system - squid performance in FreeBSD 5.3 Message-ID: <Pine.NEB.3.96L.1041226070126.45272F-100000@fledge.watson.org> In-Reply-To: <41CE1FB5.4080401@jonny.eng.br>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, 26 Dec 2004, Jo=E3o Carlos Mendes Lu=EDs wrote: > It must not be this. Squid is mostly a single process system, with= =20 > scheduling based on descriptors and select/poll. Recent versions added= =20 > some parallelism in other processes, but just for file reading/writing=20 > (diskd) and regular expression processing for ACLs. Even DNS, which=20 > previously ran on blocking I/O in secondary processes now run internally= =20 > in the select/poll scheduler. Thanks for this information. > > I might start by turning on kernel profiling and doing a profile dump > > under load. Be aware that turning on profiling uses up a lot of CPU > > itself, so will reduce the capacity of the system. There's probably > > documentation elsewhere, but the process I use to set up profiling is > > here: >=20 > I did not make any tests on this, but I would expect profiling to > fail, since every step of the scheduler is very small, and deals with > the smallest I/O available at that time.=20 This is kernel profiling, not application profiling, and would hopefully give us information on where the kernel was spending most of its time, since in the environment in question system time appears to be dominant.=20 If SMP in theory makes little difference to Squid performance, then switching to a UP kernel may well make kernel profiling more reliable and hence more useful in tracking systemn time. > Indeed, based on the original report I would search for some > optimization on descriptor searching in poll or select, whichever squid > has chosen to use on FreeBSD (probably select, looking at the top > output). This is one of the crucial points on squid performance. The > other one is disk access, for sure, but the experimente describe would > not change disk access patterns, would it?=20 The reporter described a very high percentage of system time -- time spent blocked on disk I/O isn't billed to system time; if spending lots of time waiting on disk I/O for a single process, you'd see idle time rather than system time predominating, I believe. > > As a final question: other than CPU consumption, do you have a reliable > > way to measure how efficiently the system is operating -- in particular= , > > how fast it is able to serve data? Having some sort of metric for > > performance can be quite useful in optimizing, as it can tell us whethe= r >=20 > One thing I fail to measure in FreeBSD is the reason for delays in > disk access times. How can I prove that the delay is on disk, and > determine how to optimize it? systat -v is very useful, but does not > give me all answers.=20 I'm not sure there are useful summary tools at a system-wide level for this, but it is possible to use KTR(9) to trace the associated scheduler and disk events. In particular, I recently added high level tracing of g_down and g_up GEOM events to KTR. Jeff Roberson is about to commit a scheduler visualization tool that interprets KTR events relating to the scheduler that may also be useful. It would certainly be extremely useful to have a tool for normal system operation that could be pointed at a process to say "show me the percent of time spent on various wait channels for pid 50". ktrace(1) has the ability to track context switches but appears not to provide enough information to figure out why the context switch took place currently. I'll investigate this in the next couple of days -- the trick is to gather this sort of statistic without too much additional overhead. If that's not easily possible, then simply post-processing KTR may be the right approach. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Principal Research Scientist, McAfee Research
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.NEB.3.96L.1041226070126.45272F-100000>