From owner-freebsd-performance@FreeBSD.ORG Mon Jan 21 18:58:53 2008 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 48C2016A46C for ; Mon, 21 Jan 2008 18:58:53 +0000 (UTC) (envelope-from stefan.lambrev@moneybookers.com) Received: from blah.sun-fish.com (blah.sun-fish.com [217.18.249.150]) by mx1.freebsd.org (Postfix) with ESMTP id AFD4313C45D for ; Mon, 21 Jan 2008 18:58:52 +0000 (UTC) (envelope-from stefan.lambrev@moneybookers.com) Received: by blah.sun-fish.com (Postfix, from userid 1002) id D170C1B10ED2; Mon, 21 Jan 2008 19:39:11 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on blah.cmotd.com X-Spam-Level: X-Spam-Status: No, score=-10.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, J_CHICKENPOX_48 autolearn=no version=3.2.3 Received: from hater.haters.org (hater.cmotd.com [192.168.3.125]) by blah.sun-fish.com (Postfix) with ESMTP id C9AE71B10EA4 for ; Mon, 21 Jan 2008 19:39:08 +0100 (CET) Message-ID: <4794E6CC.1050107@moneybookers.com> Date: Mon, 21 Jan 2008 20:39:08 +0200 From: Stefan Lambrev User-Agent: Thunderbird 2.0.0.9 (X11/20071120) MIME-Version: 1.0 To: freebsd-performance@freebsd.org Content-Type: text/plain; charset=windows-1251; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.91.2/5509/Mon Jan 21 17:23:11 2008 on blah.cmotd.com X-Virus-Status: Clean Subject: network performance X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jan 2008 18:58:53 -0000 Greetings, I'm trying test a bridge firewall under FreeBSD 7. What I have as configuration is: Freebsd7 (web server) - bridge (FreeBSD7) - gigabit switch - flooders. Both FreeBSD servers are using FreeBSD 7.0-RC1 amd64 With netperf -l 60 -p 10303 -H 10.3.3.1 I have no problems to reach 116MB/s with and without pf enabled. But what I want to test is how well will perform the firewall during syn floods. For this I'm using hping3 (hping-devel in ports) to generate traffic from flooders to the web server. First think, that I notice is, that hping running on linux generate twice more traffic compared to freebsd. So I plan to separate a server with dual bootable linux and fbsd and to see what's the real difference. Second problem that I encountered is, that when running hping from freebsd. It exits after few seconds/minutes with this error message: [send_ip] sendto: No buffer space available And this happens on FreeBSD_7 and FreeBSD 6.2-p8 too amd64) Can I increase those buffers ? I'm able to generate 24MB/s SYN flood and during my test I can see this on the bridge firewall: netstat -w 1 -I em0 -d - external network input (em0) output packets errs bytes packets errs bytes colls drops 427613 1757 25656852 233604 0 14016924 0 0 428089 1274 25685358 233794 0 14025174 0 0 427433 1167 25645998 234775 0 14088834 0 0 438270 2300 26296218 233384 0 14004474 0 0 438425 2009 26305518 233858 0 14034114 0 0 and from the internal network: input (em1) output packets errs bytes packets errs bytes colls drops 232912 0 13974838 425796 0 25549446 0 1334 234487 0 14069338 423986 0 25432026 0 1631 233951 0 14037178 431330 0 25880286 0 3888 233509 0 14010658 436496 0 26191986 0 1437 234181 0 14050978 430291 0 25816806 0 4001 234144 0 14048870 430208 0 25810206 0 1621 234176 0 14050678 430292 0 25828926 0 3001 And here is top -S last pid: 21830; load averages: 1.01, 0.50, 0.72 up 3+04:59:43 20:27:49 84 processes: 7 running, 60 sleeping, 17 waiting CPU states: 0.0% user, 0.0% nice, 38.2% system, 0.0% interrupt, 61.8% idle Mem: 17M Active, 159M Inact, 252M Wired, 120K Cache, 213M Buf, 1548M Free Swap: 4056M Total, 4056M Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 14 root 1 171 ki31 0K 16K CPU0 0 76.8H 100.00% idle: cpu0 11 root 1 171 ki31 0K 16K RUN 3 76.0H 100.00% idle: cpu3 25 root 1 -68 - 0K 16K CPU1 1 54:26 86.28% em0 taskq 26 root 1 -68 - 0K 16K CPU2 2 39:13 66.70% em1 taskq 12 root 1 171 ki31 0K 16K RUN 2 76.0H 37.50% idle: cpu2 13 root 1 171 ki31 0K 16K RUN 1 75.9H 16.89% idle: cpu1 16 root 1 -32 - 0K 16K WAIT 0 7:00 0.00% swi4: clock sio 51 root 1 20 - 0K 16K syncer 3 4:30 0.00% syncer vmstat -i interrupt total rate irq1: atkbd0 544 0 irq4: sio0 10641 0 irq14: ata0 1 0 irq19: uhci1+ 123697 0 cpu0: timer 553887702 1997 irq256: em0 48227501 173 irq257: em1 46331164 167 cpu1: timer 553887682 1997 cpu3: timer 553887701 1997 cpu2: timer 553887701 1997 Total 2310244334 8333 netstat -m 594/2361/2955 mbufs in use (current/cache/total) 592/1854/2446/204800 mbuf clusters in use (current/cache/total/max) 592/1328 mbuf+clusters out of packet secondary zone in use (current/cache) 0/183/183/12800 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max) 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) 1332K/5030K/6362K bytes allocated to network (current/cache/total) systat -ifstat Interface Traffic Peak Total bridge0 in 38.704 MB/s 38.704 MB/s 185.924 GB out 38.058 MB/s 38.058 MB/s 189.855 GB em1 in 13.336 MB/s 13.402 MB/s 51.475 GB out 24.722 MB/s 24.722 MB/s 137.396 GB em0 in 24.882 MB/s 24.882 MB/s 138.918 GB out 13.336 MB/s 13.403 MB/s 45.886 GB Both FreeBSD servers have quad port intel network card, 2GB memory em0@pci0:3:0:0: class=0x020000 card=0x10bc8086 chip=0x10bc8086 rev=0x06 hdr=0x00 vendor = 'Intel Corporation' device = '82571EB Gigabit Ethernet Controller (Copper)' class = network subclass = ethernet Firewall server is running on CPU: Intel(R) Xeon(R) X3220 @ 2.40GHz (quad core) Web server is running on Intel(R) Xeon(R) CPU 3070 @ 2.66GHz (dual core) So in brief how can I get rid of "No buffer space available", increase the sent rate of hping in FreeBSD and get rid of dropped packets on rates like 24MB/s :) What other tests can I run (switching on of cpu cores and etc)? Anyone interested? P.S. I'm using custom kernel, with SCHED_ULE, both freebsds build from source with CPUTYPE?=core2 and net.inet.icmp.icmplim_output=0 -- Best Wishes, Stefan Lambrev ICQ# 24134177 From owner-freebsd-performance@FreeBSD.ORG Wed Jan 23 11:44:23 2008 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BD6B416A419 for ; Wed, 23 Jan 2008 11:44:23 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 0AD8413C457; Wed, 23 Jan 2008 11:44:22 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <47972895.4050005@FreeBSD.org> Date: Wed, 23 Jan 2008 12:44:21 +0100 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Erik Cederstrand References: <4796C717.9000507@cederstrand.dk> In-Reply-To: <4796C717.9000507@cederstrand.dk> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-performance@freebsd.org" Subject: Re: Performance Tracker project update X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jan 2008 11:44:23 -0000 Erik Cederstrand wrote: > Hi > > I'd like to send a small update on my progress on the Performance > Tracker project. > > I now have a small setup of a server and a slave chugging along, > currently collecting data. I'm following CURRENT and collecting results > from super-smack and unixbench. > > The project still needs some work, but there's a temporary web interface > to the data here: http://littlebit.dk:5000/plot/. Apart from the > plotting it's possible to compare two dates and see the files that have > changed. Error bars are 3*standard deviation, for the points with > multiple measurements. > > Of interest is e.g. super-smack (select-key, 1 client) right when the > GENERIC kernel was moved from the 4BSD to ULE scheduler on Oct. 19. > Unixbench (arithmetic test, float) also has a significant jump on Oct. 3. > > There setup of the slave is documented roughly on the page but I'll be > writing a full report and documentation over the next month. > > Comments are very welcome but please followup on performance@. This is coming along very nicely indeed! One suggestion I have is that as more metrics are added it becomes important for an "at a glance" overview of changes so we can monitor for performance improvements and regressions among many workloads. One way to do this would be a matrix of each metric with its change compared to recent samples. e.g. you could do a student's T comparison of today's numbers with those from yesterday, or from a week ago, and colour-code those that show a significant deviation from "no change". This might be a bit noisy on short timescales, so you could aggregrate data into larger bins and compare e.g. moving 1-week aggregates. Fluctuations on short timescales won't stand out, but if there is a real change then it will show up less than a week later. These significant events could also be graphed themselves and/or a history log maintained (or automatically annotated on the individual graphs) so historical changes can also be pinpointed. At some point the ability to annotate the data will become important (e.g. "We understand the cause of this, it was r1.123 of foo.c, which was corrected in r1.124. The developer responsible has been shot.") Kris P.S. If I understand correctly, the float test shows a regression? The metric is calculations/second, so higher = better? From owner-freebsd-performance@FreeBSD.ORG Wed Jan 23 13:49:34 2008 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2233B16A46B; Wed, 23 Jan 2008 13:49:34 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from mx2.itu.dk (unknown [130.226.142.29]) by mx1.freebsd.org (Postfix) with ESMTP id D421D13C457; Wed, 23 Jan 2008 13:49:33 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from wimac.littlebit.dk (unknown [85.233.238.191]) by mx2.itu.dk (Postfix) with ESMTP id 815A0F4806A; Wed, 23 Jan 2008 14:49:32 +0100 (CET) Message-ID: <479745DA.8010003@cederstrand.dk> Date: Wed, 23 Jan 2008 14:49:14 +0100 From: Erik Cederstrand User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Kris Kennaway References: <4796C717.9000507@cederstrand.dk> <47972895.4050005@FreeBSD.org> In-Reply-To: <47972895.4050005@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-performance@freebsd.org" Subject: Re: Performance Tracker project update X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jan 2008 13:49:34 -0000 Kris Kennaway wrote: > > This is coming along very nicely indeed! > > One suggestion I have is that as more metrics are added it becomes > important for an "at a glance" overview of changes so we can monitor for > performance improvements and regressions among many workloads. > > One way to do this would be a matrix of each metric with its change > compared to recent samples. e.g. you could do a student's T comparison > of today's numbers with those from yesterday, or from a week ago, and > colour-code those that show a significant deviation from "no change". > This might be a bit noisy on short timescales, so you could aggregrate > data into larger bins and compare e.g. moving 1-week aggregates. > Fluctuations on short timescales won't stand out, but if there is a real > change then it will show up less than a week later. I agree that there's a need for an overview and some sort of notification. I've been collecting historical data to get a baseline for the statistics and I'll try to see what I can do over the next weeks. > These significant events could also be graphed themselves and/or a > history log maintained (or automatically annotated on the individual > graphs) so historical changes can also be pinpointed. > > At some point the ability to annotate the data will become important > (e.g. "We understand the cause of this, it was r1.123 of foo.c, which > was corrected in r1.124. The developer responsible has been shot.") There's a field in the database for this sort of thing. I just think it needs some sort of authentication. That'll have to wait a bit. > P.S. If I understand correctly, the float test shows a regression? The > metric is calculations/second, so higher = better? The documentation on Unixbench is scarce, but I would think so. BTW if anyone's interested my SVN repo is online at: svn://littlebit.dk/website/trunk (Pylons project) svn://littlebit.dk/tracker/trunk (sh/Python scripts for runnning the server and slaves) Be careful with your eyes - this is my first attempt at both shell scripting and Python :-) Erik From owner-freebsd-performance@FreeBSD.ORG Wed Jan 23 19:11:17 2008 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 60F5F16A4C6 for ; Wed, 23 Jan 2008 19:11:17 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id AD21013C468; Wed, 23 Jan 2008 19:11:16 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <47979153.8090100@FreeBSD.org> Date: Wed, 23 Jan 2008 20:11:15 +0100 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Erik Cederstrand References: <4796C717.9000507@cederstrand.dk> <47972895.4050005@FreeBSD.org> <479745DA.8010003@cederstrand.dk> In-Reply-To: <479745DA.8010003@cederstrand.dk> Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-performance@freebsd.org" Subject: Re: Performance Tracker project update X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jan 2008 19:11:17 -0000 Erik Cederstrand wrote: > Kris Kennaway wrote: >> >> This is coming along very nicely indeed! >> >> One suggestion I have is that as more metrics are added it becomes >> important for an "at a glance" overview of changes so we can monitor >> for performance improvements and regressions among many workloads. > > >> One way to do this would be a matrix of each metric with its change >> compared to recent samples. e.g. you could do a student's T >> comparison of today's numbers with those from yesterday, or from a >> week ago, and colour-code those that show a significant deviation from >> "no change". This might be a bit noisy on short timescales, so you >> could aggregrate data into larger bins and compare e.g. moving 1-week >> aggregates. Fluctuations on short timescales won't stand out, but if >> there is a real change then it will show up less than a week later. > > I agree that there's a need for an overview and some sort of > notification. I've been collecting historical data to get a baseline for > the statistics and I'll try to see what I can do over the next weeks. > >> These significant events could also be graphed themselves and/or a >> history log maintained (or automatically annotated on the individual >> graphs) so historical changes can also be pinpointed. >> >> At some point the ability to annotate the data will become important >> (e.g. "We understand the cause of this, it was r1.123 of foo.c, which >> was corrected in r1.124. The developer responsible has been shot.") > > There's a field in the database for this sort of thing. I just think it > needs some sort of authentication. That'll have to wait a bit. Sounds good. >> P.S. If I understand correctly, the float test shows a regression? >> The metric is calculations/second, so higher = better? > > The documentation on Unixbench is scarce, but I would think so. Interesting. Some candidate changes from 2007-10-02: Modified files: contrib/gcc opts.c Log: Do not imply -ftree-vrp with -O2 and above. One must implicitly specify '-ftree-vrp' if one wants it. Some bad code generation has been tracked to -ftree-vrp. jdk1{5,6} are notable examples. sys/kern sched_ule.c Log: - Move the rebalancer back into hardclock to prevent potential softclock starvation caused by unbalanced interrupt loads. - Change the rebalancer to work on stathz ticks but retain randomization. - Simplify locking in tdq_idled() to use the tdq_lock_pair() rather than complex sequences of locks to avoid deadlock. sys/kern sched_ule.c Log: - Reassign the thread queue lock to newtd prior to switching. Assigning after the switch leads to a race where the outgoing thread still owns the local queue lock while another cpu may switch it in. This race is only possible on machines where cpu_switch can take significantly longer on different cpus which in practice means HTT machines with unfair thread scheduling algorithms. Is anyone else able to look into this? > BTW if anyone's interested my SVN repo is online at: > > svn://littlebit.dk/website/trunk (Pylons project) > svn://littlebit.dk/tracker/trunk (sh/Python scripts for runnning the > server and slaves) > > Be careful with your eyes - this is my first attempt at both shell > scripting and Python :-) :) Kris From owner-freebsd-performance@FreeBSD.ORG Wed Jan 23 19:54:17 2008 Return-Path: Delivered-To: performance@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4E72916A418 for ; Wed, 23 Jan 2008 19:54:17 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 2991C13C459 for ; Wed, 23 Jan 2008 19:54:16 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 52A6B48003; Wed, 23 Jan 2008 14:36:26 -0500 (EST) Date: Wed, 23 Jan 2008 19:36:26 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Erik Cederstrand In-Reply-To: <4796C717.9000507@cederstrand.dk> Message-ID: <20080123193400.N63024@fledge.watson.org> References: <4796C717.9000507@cederstrand.dk> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: performance@FreeBSD.org Subject: Re: Performance Tracker project update X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jan 2008 19:54:17 -0000 On Wed, 23 Jan 2008, Erik Cederstrand wrote: > I'd like to send a small update on my progress on the Performance Tracker > project. > > I now have a small setup of a server and a slave chugging along, currently > collecting data. I'm following CURRENT and collecting results from > super-smack and unixbench. > > The project still needs some work, but there's a temporary web interface to > the data here: http://littlebit.dk:5000/plot/. Apart from the plotting it's > possible to compare two dates and see the files that have changed. Error > bars are 3*standard deviation, for the points with multiple measurements. > > Of interest is e.g. super-smack (select-key, 1 client) right when the > GENERIC kernel was moved from the 4BSD to ULE scheduler on Oct. 19. > Unixbench (arithmetic test, float) also has a significant jump on Oct. 3. > > There setup of the slave is documented roughly on the page but I'll be > writing a full report and documentation over the next month. > > Comments are very welcome but please followup on performance@. This looks really exciting! Do you plan to add a way so that people can submit performance data? I.e., if I set up my own test box and want to submit a result once a week for that, will there be a way for me to get set up with a username/password, submit configuration information, and then automatically submit test result datapoints? Especially if I can specify both the X and Y coordinates so that I can backdate results should I go back and generate test results for old kernels? Very neat, and I look forward to seein gmore! Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-performance@FreeBSD.ORG Wed Jan 23 19:59:16 2008 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A0F4416A417 for ; Wed, 23 Jan 2008 19:59:16 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 8334213C458 for ; Wed, 23 Jan 2008 19:59:16 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 5395B47468; Wed, 23 Jan 2008 14:40:55 -0500 (EST) Date: Wed, 23 Jan 2008 19:40:55 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Erik Cederstrand In-Reply-To: <479745DA.8010003@cederstrand.dk> Message-ID: <20080123193731.K63024@fledge.watson.org> References: <4796C717.9000507@cederstrand.dk> <47972895.4050005@FreeBSD.org> <479745DA.8010003@cederstrand.dk> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Kris Kennaway , "freebsd-performance@freebsd.org" Subject: Re: Performance Tracker project update X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jan 2008 19:59:16 -0000 On Wed, 23 Jan 2008, Erik Cederstrand wrote: >> One way to do this would be a matrix of each metric with its change >> compared to recent samples. e.g. you could do a student's T comparison of >> today's numbers with those from yesterday, or from a week ago, and >> colour-code those that show a significant deviation from "no change". This >> might be a bit noisy on short timescales, so you could aggregrate data into >> larger bins and compare e.g. moving 1-week aggregates. Fluctuations on >> short timescales won't stand out, but if there is a real change then it >> will show up less than a week later. > > I agree that there's a need for an overview and some sort of notification. > I've been collecting historical data to get a baseline for the statistics > and I'll try to see what I can do over the next weeks. A thumbnail page of graphs would be quite neat also. :-) >> These significant events could also be graphed themselves and/or a history >> log maintained (or automatically annotated on the individual graphs) so >> historical changes can also be pinpointed. >> >> At some point the ability to annotate the data will become important (e.g. >> "We understand the cause of this, it was r1.123 of foo.c, which was >> corrected in r1.124. The developer responsible has been shot.") > > There's a field in the database for this sort of thing. I just think it > needs some sort of authentication. That'll have to wait a bit. Sounds great -- it would be nice to be able to have a few annotations such as "RELENG_7 branchpoint", "7.0 release", that could then appear as vertical lines in the graphs, and likewise things like "netisr made default", "libthr becomes default". Finally, in the interests of making your life more complicated, it would be neat to graph performance across a set of FreeBSD branches overlaid or vertically offset so you could monitor, say, MySQL performance on 8-CURRENT, 7-STABLE, and 6-STABLE over time. Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-performance@FreeBSD.ORG Wed Jan 23 20:01:09 2008 Return-Path: Delivered-To: performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A28F316A46C for ; Wed, 23 Jan 2008 20:01:09 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (cl-162.ewr-01.us.sixxs.net [IPv6:2001:4830:1200:a1::2]) by mx1.freebsd.org (Postfix) with ESMTP id C0C1213C4D9 for ; Wed, 23 Jan 2008 20:01:08 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (localhost [127.0.0.1]) by lor.one-eyed-alien.net (8.14.1/8.13.8) with ESMTP id m0NK14Vn013126; Wed, 23 Jan 2008 14:01:04 -0600 (CST) (envelope-from brooks@lor.one-eyed-alien.net) Received: (from brooks@localhost) by lor.one-eyed-alien.net (8.14.1/8.13.8/Submit) id m0NK14hj013125; Wed, 23 Jan 2008 14:01:04 -0600 (CST) (envelope-from brooks) Date: Wed, 23 Jan 2008 14:01:04 -0600 From: Brooks Davis To: Erik Cederstrand Message-ID: <20080123200104.GB10711@lor.one-eyed-alien.net> References: <4796C717.9000507@cederstrand.dk> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="EuxKj2iCbKjpUGkD" Content-Disposition: inline In-Reply-To: <4796C717.9000507@cederstrand.dk> User-Agent: Mutt/1.5.16 (2007-06-09) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (lor.one-eyed-alien.net [127.0.0.1]); Wed, 23 Jan 2008 14:01:05 -0600 (CST) Cc: performance@freebsd.org Subject: Re: Performance Tracker project update X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jan 2008 20:01:09 -0000 --EuxKj2iCbKjpUGkD Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Jan 23, 2008 at 05:48:23AM +0100, Erik Cederstrand wrote: > Hi >=20 > I'd like to send a small update on my progress on the Performance Tracker= =20 > project. >=20 > I now have a small setup of a server and a slave chugging along, currentl= y=20 > collecting data. I'm following CURRENT and collecting results from=20 > super-smack and unixbench. >=20 > The project still needs some work, but there's a temporary web interface = to=20 > the data here: http://littlebit.dk:5000/plot/. Apart from the plotting it= 's=20 > possible to compare two dates and see the files that have changed. Error= =20 > bars are 3*standard deviation, for the points with multiple measurements. >=20 > Of interest is e.g. super-smack (select-key, 1 client) right when the=20 > GENERIC kernel was moved from the 4BSD to ULE scheduler on Oct. 19.=20 > Unixbench (arithmetic test, float) also has a significant jump on Oct. 3. >=20 > There setup of the slave is documented roughly on the page but I'll be=20 > writing a full report and documentation over the next month. Nice work so far! It's neat that you've already been able to spot changes. A couple suggestions for the graphs. It would be nice if the graphs had an arrow indicating the "good" direction (i.e. which direction of movement is = an improvement). Also, a graph of the derivative of the curve might be interesting for at a glance trending. -- Brooks --EuxKj2iCbKjpUGkD Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (FreeBSD) iD8DBQFHl50AXY6L6fI4GtQRAktvAKCDpZyshqkBGYG6Bz5jRKPyIhVl1ACeJPbn rcsV0rgZMPQhmLxNLl34FIc= =iGt2 -----END PGP SIGNATURE----- --EuxKj2iCbKjpUGkD-- From owner-freebsd-performance@FreeBSD.ORG Wed Jan 23 20:23:54 2008 Return-Path: Delivered-To: performance@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0170F16A417; Wed, 23 Jan 2008 20:23:54 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from mx2.itu.dk (unknown [130.226.142.29]) by mx1.freebsd.org (Postfix) with ESMTP id C50DB13C43E; Wed, 23 Jan 2008 20:23:53 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from wimac.littlebit.dk (unknown [85.233.238.191]) by mx2.itu.dk (Postfix) with ESMTP id 60637F4806A; Wed, 23 Jan 2008 21:23:52 +0100 (CET) Message-ID: <4797A245.7080202@cederstrand.dk> Date: Wed, 23 Jan 2008 21:23:33 +0100 From: Erik Cederstrand User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Robert Watson References: <4796C717.9000507@cederstrand.dk> <20080123193400.N63024@fledge.watson.org> In-Reply-To: <20080123193400.N63024@fledge.watson.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: performance@FreeBSD.org Subject: Re: Performance Tracker project update X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jan 2008 20:23:54 -0000 Robert Watson wrote: > > This looks really exciting! > > Do you plan to add a way so that people can submit performance data? > I.e., if I set up my own test box and want to submit a result once a > week for that, will there be a way for me to get set up with a > username/password, submit configuration information, and then > automatically submit test result datapoints? Especially if I can > specify both the X and Y coordinates so that I can backdate results > should I go back and generate test results for old kernels? The website and benchmarking slave talk to a PostgreSQL database. It's definitely possible and part of the design to have multiple computers participating in a distributed fashion, although it's also possible to run the setup locally and privately for more ad-hoc testing purposes. I think it's best if participating machines supply data regularly for an extended period of time. Single or infrequent data points for a specific configuration don't make much sense. We need to compare apples to apples. Erik From owner-freebsd-performance@FreeBSD.ORG Wed Jan 23 20:25:29 2008 Return-Path: Delivered-To: performance@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0CAD916A41A for ; Wed, 23 Jan 2008 20:25:29 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id B80BC13C474 for ; Wed, 23 Jan 2008 20:25:28 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id A549046BCA; Wed, 23 Jan 2008 15:25:27 -0500 (EST) Date: Wed, 23 Jan 2008 20:25:27 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Erik Cederstrand In-Reply-To: <4797A245.7080202@cederstrand.dk> Message-ID: <20080123202433.E63024@fledge.watson.org> References: <4796C717.9000507@cederstrand.dk> <20080123193400.N63024@fledge.watson.org> <4797A245.7080202@cederstrand.dk> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: performance@FreeBSD.org Subject: Re: Performance Tracker project update X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jan 2008 20:25:29 -0000 On Wed, 23 Jan 2008, Erik Cederstrand wrote: > Robert Watson wrote: >> >> This looks really exciting! >> >> Do you plan to add a way so that people can submit performance data? >> I.e., if I set up my own test box and want to submit a result once a week >> for that, will there be a way for me to get set up with a >> username/password, submit configuration information, and then automatically >> submit test result datapoints? Especially if I can specify both the X and >> Y coordinates so that I can backdate results should I go back and generate >> test results for old kernels? > > The website and benchmarking slave talk to a PostgreSQL database. It's > definitely possible and part of the design to have multiple computers > participating in a distributed fashion, although it's also possible to run > the setup locally and privately for more ad-hoc testing purposes. Sounds good. > I think it's best if participating machines supply data regularly for an > extended period of time. Single or infrequent data points for a specific > configuration don't make much sense. We need to compare apples to apples. Yes -- I was mostly thinking about backdating in order to play "catchup" when a new benchmark is introduced. Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-performance@FreeBSD.ORG Wed Jan 23 20:48:05 2008 Return-Path: Delivered-To: performance@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5AB5116A420; Wed, 23 Jan 2008 20:48:05 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 0A12A13C44B; Wed, 23 Jan 2008 20:48:03 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <4797A802.8060509@FreeBSD.org> Date: Wed, 23 Jan 2008 21:48:02 +0100 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Robert Watson References: <4796C717.9000507@cederstrand.dk> <20080123193400.N63024@fledge.watson.org> <4797A245.7080202@cederstrand.dk> <20080123202433.E63024@fledge.watson.org> In-Reply-To: <20080123202433.E63024@fledge.watson.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: performance@FreeBSD.org, Erik Cederstrand Subject: Re: Performance Tracker project update X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jan 2008 20:48:05 -0000 Robert Watson wrote: >> I think it's best if participating machines supply data regularly for >> an extended period of time. Single or infrequent data points for a >> specific configuration don't make much sense. We need to compare >> apples to apples. > > Yes -- I was mostly thinking about backdating in order to play "catchup" > when a new benchmark is introduced. One thing I am looking at is how to best create a library of world tarballs that can be used to populate a nfsroot (or hybrid of periodic tarballs + binary diffs to save space). Then you could provide your benchmark in a standardized format (start/end/cleanup scripts, etc) and tell a machine "go and run this benchmark on every daily snapshot for the last year and give me the numbers". Kris From owner-freebsd-performance@FreeBSD.ORG Wed Jan 23 21:00:03 2008 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EA17A16A418; Wed, 23 Jan 2008 21:00:02 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from mx2.itu.dk (unknown [130.226.142.29]) by mx1.freebsd.org (Postfix) with ESMTP id 8389613C4CC; Wed, 23 Jan 2008 21:00:02 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from wimac.littlebit.dk (unknown [85.233.238.191]) by mx2.itu.dk (Postfix) with ESMTP id 5A763F48073; Wed, 23 Jan 2008 22:00:01 +0100 (CET) Message-ID: <4797AABE.80509@cederstrand.dk> Date: Wed, 23 Jan 2008 21:59:42 +0100 From: Erik Cederstrand User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Robert Watson References: <4796C717.9000507@cederstrand.dk> <47972895.4050005@FreeBSD.org> <479745DA.8010003@cederstrand.dk> <20080123193731.K63024@fledge.watson.org> In-Reply-To: <20080123193731.K63024@fledge.watson.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Kris Kennaway , "freebsd-performance@freebsd.org" Subject: Re: Performance Tracker project update X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jan 2008 21:00:03 -0000 Robert Watson wrote: > On Wed, 23 Jan 2008, Erik Cederstrand wrote: > >> I agree that there's a need for an overview and some sort of >> notification. I've been collecting historical data to get a baseline >> for the statistics and I'll try to see what I can do over the next weeks. > > A thumbnail page of graphs would be quite neat also. :-) I'd like to do that, but I'm a bit afraid of hitting the webserver too hard. Data is changing constantly, so graphs are created on the fly. I could probably cache some small versions though and put them on an overview page. >>> At some point the ability to annotate the data will become important >>> (e.g. "We understand the cause of this, it was r1.123 of foo.c, which >>> was corrected in r1.124. The developer responsible has been shot.") >> >> There's a field in the database for this sort of thing. I just think >> it needs some sort of authentication. That'll have to wait a bit. > > Sounds great -- it would be nice to be able to have a few annotations > such as "RELENG_7 branchpoint", "7.0 release", that could then appear as > vertical lines in the graphs, and likewise things like "netisr made > default", "libthr becomes default". Good idea. Thanks! > Finally, in the interests of making your life more complicated, it would > be neat to graph performance across a set of FreeBSD branches overlaid > or vertically offset so you could monitor, say, MySQL performance on > 8-CURRENT, 7-STABLE, and 6-STABLE over time. It's supported by the tools, database and website but currently I just have 2 (slow) machines to work with at the university. The server is churning out images for the slave to consume every 2 hours which is an acceptable sample rate if the intent is to monitor CURRENT development and provide performance alerts much like the tinderboxes do now. Adding releases would seriously hurt that rate. My focus has been strictly on comparing CVS versions because trying to cover anything else quickly makes the configuration space explode and the sample rate go down the drain. Lots of interesting comparisons could be made, however, provided sufficient hardware supplies. Comparing branches would on the top of the list. Erik From owner-freebsd-performance@FreeBSD.ORG Wed Jan 23 21:16:25 2008 Return-Path: Delivered-To: performance@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4D30116A421; Wed, 23 Jan 2008 21:16:25 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from mx2.itu.dk (unknown [130.226.142.29]) by mx1.freebsd.org (Postfix) with ESMTP id 1BF6A13C45A; Wed, 23 Jan 2008 21:16:25 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from wimac.littlebit.dk (unknown [85.233.238.191]) by mx2.itu.dk (Postfix) with ESMTP id E8DBEF48072; Wed, 23 Jan 2008 22:16:23 +0100 (CET) Message-ID: <4797AE94.6090502@cederstrand.dk> Date: Wed, 23 Jan 2008 22:16:04 +0100 From: Erik Cederstrand User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Kris Kennaway References: <4796C717.9000507@cederstrand.dk> <20080123193400.N63024@fledge.watson.org> <4797A245.7080202@cederstrand.dk> <20080123202433.E63024@fledge.watson.org> <4797A802.8060509@FreeBSD.org> In-Reply-To: <4797A802.8060509@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: performance@FreeBSD.org, Robert Watson Subject: Re: Performance Tracker project update X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jan 2008 21:16:25 -0000 Kris Kennaway wrote: > Robert Watson wrote: >> >> Yes -- I was mostly thinking about backdating in order to play >> "catchup" when a new benchmark is introduced. > > One thing I am looking at is how to best create a library of world > tarballs that can be used to populate a nfsroot (or hybrid of periodic > tarballs + binary diffs to save space). Then you could provide your > benchmark in a standardized format (start/end/cleanup scripts, etc) and > tell a machine "go and run this benchmark on every daily snapshot for > the last year and give me the numbers". That's basically what my server does. It creates world/kernel tarballs (around 90MB), dumps them in an NFS exported folder and adds them to a queue for the slave to consume. The slave installs the tarball on the local disk (via PXE) in less than 2 mins and runs whatever benchmarks it was told. I can pretty easily add more benchmarks later and let the slave collect the data using the tarballs I already created. Currently the benchmark script is contained within the tarball, so each tarball would need to be opened and closed to replace the script. I'd like to change that. It gets a bit complicated when the benchmark depends on another application (e.g. super-smack and mysql). Currently I compile mysql from scratch on every new world using whichever version was the latest in the ports tree at the time. This has some disadvantages. Erik From owner-freebsd-performance@FreeBSD.ORG Thu Jan 24 00:17:14 2008 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 78FD816A421 for ; Thu, 24 Jan 2008 00:17:14 +0000 (UTC) (envelope-from gofp-freebsd-performance@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 416E113C45D for ; Thu, 24 Jan 2008 00:17:13 +0000 (UTC) (envelope-from gofp-freebsd-performance@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1JHpmG-0002us-2w for freebsd-performance@freebsd.org; Thu, 24 Jan 2008 00:17:08 +0000 Received: from 78-1-98-105.adsl.net.t-com.hr ([78.1.98.105]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 24 Jan 2008 00:17:08 +0000 Received: from ivoras by 78-1-98-105.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 24 Jan 2008 00:17:08 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-performance@freebsd.org From: Ivan Voras Date: Thu, 24 Jan 2008 01:16:59 +0100 Lines: 41 Message-ID: References: <4796C717.9000507@cederstrand.dk> <47972895.4050005@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigFFA17735E9F0A78F5E4D99DD" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 78-1-98-105.adsl.net.t-com.hr User-Agent: Thunderbird 2.0.0.9 (Windows/20071031) In-Reply-To: <47972895.4050005@FreeBSD.org> X-Enigmail-Version: 0.95.6 Sender: news Subject: Re: Performance Tracker project update X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jan 2008 00:17:14 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigFFA17735E9F0A78F5E4D99DD Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable Kris Kennaway wrote: >> The project still needs some work, but there's a temporary web=20 >> interface to the data here: http://littlebit.dk:5000/plot/. Apart from= =20 >> the plotting it's possible to compare two dates and see the files that= =20 >> have changed. Error bars are 3*standard deviation, for the points with= =20 >> multiple measurements. I have a suggestion to make the graphs more readable: if a long period=20 was chosen by the user (e.g. > 100 days / plot points), don't plot=20 points and error bars, plot a simple line through the points. Also, set=20 all date strings on the X-axis to empty strings except for the dates on=20 1/10ths of the interval. Did you remove WITNESS,INVARIANTS and malloc debugging for the benchmarks= ? --------------enigFFA17735E9F0A78F5E4D99DD Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHl9j7ldnAQVacBcgRAswqAKDfyxPGrIemiNBbzC/CmU5r+fqlbACfZ+xn k/O4i91lMPnGBg3oGWIgUt0= =gYj0 -----END PGP SIGNATURE----- --------------enigFFA17735E9F0A78F5E4D99DD-- From owner-freebsd-performance@FreeBSD.ORG Thu Jan 24 01:59:00 2008 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A85DD16A41B; Thu, 24 Jan 2008 01:59:00 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id D113013C44B; Thu, 24 Jan 2008 01:58:58 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <4797F0E0.8050201@FreeBSD.org> Date: Thu, 24 Jan 2008 02:58:56 +0100 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Kris Kennaway References: <4796C717.9000507@cederstrand.dk> <47972895.4050005@FreeBSD.org> <479745DA.8010003@cederstrand.dk> <47979153.8090100@FreeBSD.org> In-Reply-To: <47979153.8090100@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-performance@freebsd.org" , Erik Cederstrand Subject: Re: Performance Tracker project update X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jan 2008 01:59:00 -0000 Kris Kennaway wrote: >>> P.S. If I understand correctly, the float test shows a regression? >>> The metric is calculations/second, so higher = better? >> >> The documentation on Unixbench is scarce, but I would think so. > > Interesting. Some candidate changes from 2007-10-02: > > Modified files: > contrib/gcc opts.c > Log: > Do not imply -ftree-vrp with -O2 and above. One must implicitly specify > '-ftree-vrp' if one wants it. > Some bad code generation has been tracked to -ftree-vrp. jdk1{5,6} are > notable examples. OK, so it was this one. The other interesting events seem to be: 2007-10-20: drop in super-smack performance and context switch benchmarks. This is due to the switch from SCHED_4BSD to SCHED_ULE (super-smack is largely a context switch benchmark due to retarded design). There are uncommitted patches that reduce ULE context switch overhead though, so it will be interesting to see how they affect this. 2007-12-30: file read/pipe read/pipe ping-pong/syscall overhead performance increases. This is due to Jeff's lockless struct file changes (syscall overhead is only affected because unixbench uses the dup2() syscall which is not in fact just a measure of syscall overhead but now has reduced non-syscall cost). Kris From owner-freebsd-performance@FreeBSD.ORG Thu Jan 24 08:01:54 2008 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AFD5916A41B; Thu, 24 Jan 2008 08:01:54 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from redbull.bpaserver.net (redbullneu.bpaserver.net [213.198.78.217]) by mx1.freebsd.org (Postfix) with ESMTP id 68F2113C46A; Thu, 24 Jan 2008 08:01:54 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from outgoing.leidinger.net (p54A56A2E.dip.t-dialin.net [84.165.106.46]) by redbull.bpaserver.net (Postfix) with ESMTP id 4FD3B2E15B; Thu, 24 Jan 2008 09:01:44 +0100 (CET) Received: from webmail.leidinger.net (webmail.Leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id 4547789706; Thu, 24 Jan 2008 08:58:01 +0100 (CET) Received: (from www@localhost) by webmail.leidinger.net (8.14.2/8.13.8/Submit) id m0O7w0sK057474; Thu, 24 Jan 2008 08:58:00 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde MIME library) with HTTP; Thu, 24 Jan 2008 08:58:00 +0100 Message-ID: <20080124085800.zq6aa2cf4k4ogwo8@webmail.leidinger.net> X-Priority: 3 (Normal) Date: Thu, 24 Jan 2008 08:58:00 +0100 From: Alexander Leidinger To: Erik Cederstrand References: <4796C717.9000507@cederstrand.dk> <47972895.4050005@FreeBSD.org> <479745DA.8010003@cederstrand.dk> <20080123193731.K63024@fledge.watson.org> <4797AABE.80509@cederstrand.dk> In-Reply-To: <4797AABE.80509@cederstrand.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: quoted-printable User-Agent: Internet Messaging Program (IMP) H3 (4.1.5) / FreeBSD-8.0 X-BPAnet-MailScanner-Information: Please contact the ISP for more information X-BPAnet-MailScanner: Found to be clean X-BPAnet-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=-14.9, required 6, BAYES_00 -15.00, RDNS_DYNAMIC 0.10) X-BPAnet-MailScanner-From: alexander@leidinger.net X-Spam-Status: No Cc: Kris Kennaway , Robert Watson , "freebsd-performance@freebsd.org" Subject: Re: Performance Tracker project update X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jan 2008 08:01:54 -0000 Quoting Erik Cederstrand (from Wed, 23 Jan 2008 =20 21:59:42 +0100): >> Finally, in the interests of making your life more complicated, it =20 >> would be neat to graph performance across a set of FreeBSD branches =20 >> overlaid or vertically offset so you could monitor, say, MySQL =20 >> performance on 8-CURRENT, 7-STABLE, and 6-STABLE over time. > > It's supported by the tools, database and website but currently I just > have 2 (slow) machines to work with at the university. The server is > churning out images for the slave to consume every 2 hours which is an > acceptable sample rate if the intent is to monitor CURRENT development > and provide performance alerts much like the tinderboxes do now. Adding > releases would seriously hurt that rate. > > My focus has been strictly on comparing CVS versions because trying to > cover anything else quickly makes the configuration space explode and > the sample rate go down the drain. Lots of interesting comparisons > could be made, however, provided sufficient hardware supplies. > Comparing branches would on the top of the list. In case you have some space left for more machines, maybe someone is =20 willing to help out by sponsoring some. Just tell us how many machines =20 you can handle (space/power/...) and if you are interested that we put =20 it up on our wantlist. Bye, Alexander. --=20 http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID =3D B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID =3D 72077137 From owner-freebsd-performance@FreeBSD.ORG Thu Jan 24 09:09:32 2008 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7814E16A419; Thu, 24 Jan 2008 09:09:32 +0000 (UTC) (envelope-from BORJAMAR@SARENET.ES) Received: from proxypop1.sarenet.es (proxypop1.sarenet.es [194.30.0.99]) by mx1.freebsd.org (Postfix) with ESMTP id 4137613C474; Thu, 24 Jan 2008 09:09:31 +0000 (UTC) (envelope-from BORJAMAR@SARENET.ES) Received: from localhost (matahari.sarenet.es [192.148.167.18]) by proxypop1.sarenet.es (Postfix) with ESMTP id E6F135D57; Thu, 24 Jan 2008 09:53:10 +0100 (CET) Message-Id: <3B3710F8-9308-48E1-AD0A-CF707A6898B6@SARENET.ES> From: Borja Marcos To: Kris Kennaway In-Reply-To: <47972895.4050005@FreeBSD.org> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v915) Date: Thu, 24 Jan 2008 09:53:11 +0100 References: <4796C717.9000507@cederstrand.dk> <47972895.4050005@FreeBSD.org> X-Mailer: Apple Mail (2.915) Cc: freebsd-performance@freebsd.org Subject: Re: Performance Tracker project update X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jan 2008 09:09:32 -0000 On Jan 23, 2008, at 12:44 PM, Kris Kennaway wrote: > One suggestion I have is that as more metrics are added it becomes > important for an "at a glance" overview of changes so we can monitor > for performance improvements and regressions among many workloads. > > One way to do this would be a matrix of each metric with its change > compared to recent samples. e.g. you could do a student's T > comparison of today's numbers with those from yesterday, or from a > week ago, and colour-code those that show a significant deviation > from "no change". This might be a bit noisy on short timescales, so > you could aggregrate data into larger bins and compare e.g. moving 1- > week aggregates. Fluctuations on short timescales won't stand out, > but if there is a real change then it will show up less than a week > later. And now for some publicity :) I am a big fan of Orca (www.orcaware.com) which I have used regularly with Solaris *and* FreeBSD. I wrote a performance data collector for FreeBSD, actually. The advantage of Orca is that it gets simple text files with the performance data and then it generates graphs and web pages periodically, keeping the data in RRD databases. I think Orca would be really useful for this project. And of course, if anyone wants to give "Devilator" a try, it currently graphs CPU usage in system, interrupt, user time, disk I/O bandwidth, memory usage, system and SWI CPU usage, process sleeping reasons... So far it's been really helpful for me in order to identify bottlenecks. Regards, Borja. From owner-freebsd-performance@FreeBSD.ORG Thu Jan 24 10:02:54 2008 Return-Path: Delivered-To: performance@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 71D3616A417 for ; Thu, 24 Jan 2008 10:02:54 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from mx2.itu.dk (unknown [130.226.142.29]) by mx1.freebsd.org (Postfix) with ESMTP id 3B9F213C4D9 for ; Thu, 24 Jan 2008 10:02:54 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from wimac.littlebit.dk (unknown [85.233.238.191]) by mx2.itu.dk (Postfix) with ESMTP id 01BABF4806A; Thu, 24 Jan 2008 11:02:52 +0100 (CET) Message-ID: <47986238.4080508@cederstrand.dk> Date: Thu, 24 Jan 2008 11:02:32 +0100 From: Erik Cederstrand User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Alexander Leidinger References: <4796C717.9000507@cederstrand.dk> <47972895.4050005@FreeBSD.org> <479745DA.8010003@cederstrand.dk> <20080123193731.K63024@fledge.watson.org> <4797AABE.80509@cederstrand.dk> <20080124085800.zq6aa2cf4k4ogwo8@webmail.leidinger.net> In-Reply-To: <20080124085800.zq6aa2cf4k4ogwo8@webmail.leidinger.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: performance@FreeBSD.org Subject: Re: Performance Tracker project update X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jan 2008 10:02:54 -0000 Alexander Leidinger wrote: > > In case you have some space left for more machines, maybe someone is > willing to help out by sponsoring some. Just tell us how many machines > you can handle (space/power/...) and if you are interested that we put > it up on our wantlist. I'm sharing an office with 4 others and I'm leaving for Real Work in a month, so I'm not really in a position to expand at the moment. However, I'd like to do so later on, and also get a more permanent and reliable location for the machines. After weeding out the worst bugs the system runs pretty much by itself, but since this is CURRENT there's always the possibility of the slaves needing attention. Erik From owner-freebsd-performance@FreeBSD.ORG Thu Jan 24 10:28:45 2008 Return-Path: Delivered-To: performance@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7C01E16A41A; Thu, 24 Jan 2008 10:28:45 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from mx2.itu.dk (unknown [130.226.142.29]) by mx1.freebsd.org (Postfix) with ESMTP id 473D513C468; Thu, 24 Jan 2008 10:28:45 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from wimac.littlebit.dk (unknown [85.233.238.191]) by mx2.itu.dk (Postfix) with ESMTP id 436C5F4806A; Thu, 24 Jan 2008 11:28:44 +0100 (CET) Message-ID: <47986848.7010602@cederstrand.dk> Date: Thu, 24 Jan 2008 11:28:24 +0100 From: Erik Cederstrand User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Ivan Voras References: <4796C717.9000507@cederstrand.dk> <47972895.4050005@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: performance@FreeBSD.org Subject: Re: Performance Tracker project update X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jan 2008 10:28:45 -0000 Ivan Voras wrote: > > I have a suggestion to make the graphs more readable: if a long period > was chosen by the user (e.g. > 100 days / plot points), don't plot > points and error bars, plot a simple line through the points. Also, set > all date strings on the X-axis to empty strings except for the dates on > 1/10ths of the interval. Noted. Thanks. > Did you remove WITNESS,INVARIANTS and malloc debugging for the benchmarks? The kernel configuration file has: include GENERIC PERFMON nomakeoptions DEBUG PERFMON nooptions INVARIANTS PERFMON nooptions GDB PERFMON nooptions DDB PERFMON nooptions KDB PERFMON nooptions WITNESS PERFMON nooptions WITNESS_SKIPSPIN PERFMON nooptions INVARIANT_SUPPORT I also ship the images with a GENERIC kernel in case debugging is needed. I haven't touched malloc.conf but realize that I should. What's the official recommendation on malloc settings? Erik From owner-freebsd-performance@FreeBSD.ORG Thu Jan 24 11:06:44 2008 Return-Path: Delivered-To: performance@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4520A16A418; Thu, 24 Jan 2008 11:06:44 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id BDAC613C4CC; Thu, 24 Jan 2008 11:06:42 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <47987141.3070606@FreeBSD.org> Date: Thu, 24 Jan 2008 12:06:41 +0100 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Erik Cederstrand References: <4796C717.9000507@cederstrand.dk> <47972895.4050005@FreeBSD.org> <47986848.7010602@cederstrand.dk> In-Reply-To: <47986848.7010602@cederstrand.dk> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: performance@FreeBSD.org, Ivan Voras Subject: Re: Performance Tracker project update X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jan 2008 11:06:44 -0000 Erik Cederstrand wrote: > Ivan Voras wrote: >> >> I have a suggestion to make the graphs more readable: if a long period >> was chosen by the user (e.g. > 100 days / plot points), don't plot >> points and error bars, plot a simple line through the points. Also, >> set all date strings on the X-axis to empty strings except for the >> dates on 1/10ths of the interval. > > Noted. Thanks. Actually the error bars are quite important to see what is going on. Some of the metrics are very (too) noisy and if you only look at the data points they sometimes appear to have a signal when they don't. Ultimately that just means more data points should be taken per run for those metrics, but the error bars are the signal for this. >> Did you remove WITNESS,INVARIANTS and malloc debugging for the >> benchmarks? > > The kernel configuration file has: > include GENERIC > PERFMON nomakeoptions DEBUG > PERFMON nooptions INVARIANTS > PERFMON nooptions GDB > PERFMON nooptions DDB > PERFMON nooptions KDB > PERFMON nooptions WITNESS > PERFMON nooptions WITNESS_SKIPSPIN > PERFMON nooptions INVARIANT_SUPPORT > > I also ship the images with a GENERIC kernel in case debugging is needed. > > I haven't touched malloc.conf but realize that I should. What's the > official recommendation on malloc settings? For benchmarking you should enable MALLOC_PRODUCTION in src/lib/libc/stdlib/malloc.c. Anyway, for now I am not worried about the particular benchmarks you are running, because they are just demonstrators for the framework and we can easily go back and add more in later on. Kris From owner-freebsd-performance@FreeBSD.ORG Thu Jan 24 11:08:09 2008 Return-Path: Delivered-To: performance@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EF7E016A418 for ; Thu, 24 Jan 2008 11:08:09 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 3F88013C467; Thu, 24 Jan 2008 11:08:08 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <47987197.2060809@FreeBSD.org> Date: Thu, 24 Jan 2008 12:08:07 +0100 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Erik Cederstrand References: <4796C717.9000507@cederstrand.dk> <47972895.4050005@FreeBSD.org> <479745DA.8010003@cederstrand.dk> <20080123193731.K63024@fledge.watson.org> <4797AABE.80509@cederstrand.dk> <20080124085800.zq6aa2cf4k4ogwo8@webmail.leidinger.net> <47986238.4080508@cederstrand.dk> In-Reply-To: <47986238.4080508@cederstrand.dk> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Alexander Leidinger , performance@FreeBSD.org Subject: Re: Performance Tracker project update X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jan 2008 11:08:10 -0000 Erik Cederstrand wrote: > Alexander Leidinger wrote: >> >> In case you have some space left for more machines, maybe someone is >> willing to help out by sponsoring some. Just tell us how many machines >> you can handle (space/power/...) and if you are interested that we put >> it up on our wantlist. > > I'm sharing an office with 4 others and I'm leaving for Real Work in a > month, so I'm not really in a position to expand at the moment. However, > I'd like to do so later on, and also get a more permanent and reliable > location for the machines. > > After weeding out the worst bugs the system runs pretty much by itself, > but since this is CURRENT there's always the possibility of the slaves > needing attention. I don't see machine availability being a problem once we are ready to "take this live". Kris From owner-freebsd-performance@FreeBSD.ORG Thu Jan 24 11:10:09 2008 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4FB6616A41A for ; Thu, 24 Jan 2008 11:10:09 +0000 (UTC) (envelope-from gofp-freebsd-performance@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 0926113C4D3 for ; Thu, 24 Jan 2008 11:10:08 +0000 (UTC) (envelope-from gofp-freebsd-performance@m.gmane.org) Received: from root by ciao.gmane.org with local (Exim 4.43) id 1JHzy6-0005zL-39 for freebsd-performance@freebsd.org; Thu, 24 Jan 2008 11:10:02 +0000 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 24 Jan 2008 11:10:02 +0000 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 24 Jan 2008 11:10:02 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-performance@freebsd.org From: Ivan Voras Date: Thu, 24 Jan 2008 12:05:02 +0100 Lines: 29 Message-ID: References: <4796C717.9000507@cederstrand.dk> <47972895.4050005@FreeBSD.org> <47986848.7010602@cederstrand.dk> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigB247A0AFA0205DB75CD8594E" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Thunderbird 2.0.0.6 (X11/20071022) In-Reply-To: <47986848.7010602@cederstrand.dk> X-Enigmail-Version: 0.95.0 Sender: news Subject: Re: Performance Tracker project update X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jan 2008 11:10:09 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigB247A0AFA0205DB75CD8594E Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Erik Cederstrand wrote: > I haven't touched malloc.conf but realize that I should. What's the > official recommendation on malloc settings? You'd have to patch /usr/src/lib/libc/stdlib/malloc.c and define MALLOC_PRODUCTION. Yes, it's not elegant. --------------enigB247A0AFA0205DB75CD8594E Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFHmHDeldnAQVacBcgRAhMKAKDvoRLXky1cW8f4QyRC/JHkXbIUwwCgzqBt 6XCPGztEfTINrsZD69Dr2IA= =jfv2 -----END PGP SIGNATURE----- --------------enigB247A0AFA0205DB75CD8594E-- From owner-freebsd-performance@FreeBSD.ORG Thu Jan 24 11:20:04 2008 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A7D3316A421 for ; Thu, 24 Jan 2008 11:20:04 +0000 (UTC) (envelope-from gofp-freebsd-performance@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 32F1E13C45D for ; Thu, 24 Jan 2008 11:20:04 +0000 (UTC) (envelope-from gofp-freebsd-performance@m.gmane.org) Received: from root by ciao.gmane.org with local (Exim 4.43) id 1JI07m-0006S7-VY for freebsd-performance@freebsd.org; Thu, 24 Jan 2008 11:20:02 +0000 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 24 Jan 2008 11:20:02 +0000 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 24 Jan 2008 11:20:02 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-performance@freebsd.org From: Ivan Voras Date: Thu, 24 Jan 2008 12:15:01 +0100 Lines: 46 Message-ID: References: <4796C717.9000507@cederstrand.dk> <47972895.4050005@FreeBSD.org> <47986848.7010602@cederstrand.dk> <47987141.3070606@FreeBSD.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig25CEE7FDD58BF1678523E18C" X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Thunderbird 2.0.0.6 (X11/20071022) In-Reply-To: <47987141.3070606@FreeBSD.org> X-Enigmail-Version: 0.95.0 Sender: news Subject: Re: Performance Tracker project update X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jan 2008 11:20:04 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig25CEE7FDD58BF1678523E18C Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Kris Kennaway wrote: > Erik Cederstrand wrote: >> Ivan Voras wrote: >>> >>> I have a suggestion to make the graphs more readable: if a long >>> period was chosen by the user (e.g. > 100 days / plot points), don't >>> plot points and error bars, plot a simple line through the points. >>> Also, set all date strings on the X-axis to empty strings except for >>> the dates on 1/10ths of the interval. >> >> Noted. Thanks. >=20 > Actually the error bars are quite important to see what is going on. > Some of the metrics are very (too) noisy and if you only look at the > data points they sometimes appear to have a signal when they don't. > Ultimately that just means more data points should be taken per run for= > those metrics, but the error bars are the signal for this. Of course they are useful, but do we really need to see them when there are hundreds of samples plotted 1 pixel apart? (As I've said: keep them when when the user "zooms in" on an appropriate small number of samples). But this is an aesthetics-oriented idea, functionality won't be impaired either way so feel free to ignore it. --------------enig25CEE7FDD58BF1678523E18C Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFHmHM1ldnAQVacBcgRAs1mAJ9MMMsBdS1lCroWH7h38ehxRuvu8wCeO/8s 9z35kgXHX0/FW4XGiVHFjhc= =nw4H -----END PGP SIGNATURE----- --------------enig25CEE7FDD58BF1678523E18C-- From owner-freebsd-performance@FreeBSD.ORG Thu Jan 24 11:26:14 2008 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 43D5216A41A; Thu, 24 Jan 2008 11:26:14 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id E8B4913C46E; Thu, 24 Jan 2008 11:26:12 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <479875D3.3020801@FreeBSD.org> Date: Thu, 24 Jan 2008 12:26:11 +0100 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Ivan Voras References: <4796C717.9000507@cederstrand.dk> <47972895.4050005@FreeBSD.org> <47986848.7010602@cederstrand.dk> <47987141.3070606@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-performance@freebsd.org Subject: Re: Performance Tracker project update X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jan 2008 11:26:14 -0000 Ivan Voras wrote: > Kris Kennaway wrote: >> Erik Cederstrand wrote: >>> Ivan Voras wrote: >>>> I have a suggestion to make the graphs more readable: if a long >>>> period was chosen by the user (e.g. > 100 days / plot points), don't >>>> plot points and error bars, plot a simple line through the points. >>>> Also, set all date strings on the X-axis to empty strings except for >>>> the dates on 1/10ths of the interval. >>> Noted. Thanks. >> Actually the error bars are quite important to see what is going on. >> Some of the metrics are very (too) noisy and if you only look at the >> data points they sometimes appear to have a signal when they don't. >> Ultimately that just means more data points should be taken per run for >> those metrics, but the error bars are the signal for this. > > Of course they are useful, but do we really need to see them when there > are hundreds of samples plotted 1 pixel apart? (As I've said: keep them > when when the user "zooms in" on an appropriate small number of > samples). But this is an aesthetics-oriented idea, functionality won't > be impaired either way so feel free to ignore it. I think so, yeah. Otherwise I'd waste time zooming in only to find the trends are measurement noise. Kris From owner-freebsd-performance@FreeBSD.ORG Thu Jan 24 12:05:05 2008 Return-Path: Delivered-To: performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4653A16A418 for ; Thu, 24 Jan 2008 12:05:05 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from redbull.bpaserver.net (redbullneu.bpaserver.net [213.198.78.217]) by mx1.freebsd.org (Postfix) with ESMTP id 0138B13C45A for ; Thu, 24 Jan 2008 12:05:04 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from outgoing.leidinger.net (p54A56A2E.dip.t-dialin.net [84.165.106.46]) by redbull.bpaserver.net (Postfix) with ESMTP id 8AE7D2E0FE; Thu, 24 Jan 2008 12:47:05 +0100 (CET) Received: from webmail.leidinger.net (webmail.Leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id 1D8B78B568; Thu, 24 Jan 2008 12:43:20 +0100 (CET) Received: (from www@localhost) by webmail.leidinger.net (8.14.2/8.13.8/Submit) id m0OBhJU5094960; Thu, 24 Jan 2008 12:43:19 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde MIME library) with HTTP; Thu, 24 Jan 2008 12:43:19 +0100 Message-ID: <20080124124319.1rae3sfggwg4kss0@webmail.leidinger.net> X-Priority: 3 (Normal) Date: Thu, 24 Jan 2008 12:43:19 +0100 From: Alexander Leidinger To: Kris Kennaway References: <4796C717.9000507@cederstrand.dk> <47972895.4050005@FreeBSD.org> <479745DA.8010003@cederstrand.dk> <20080123193731.K63024@fledge.watson.org> <4797AABE.80509@cederstrand.dk> <20080124085800.zq6aa2cf4k4ogwo8@webmail.leidinger.net> <47986238.4080508@cederstrand.dk> <47987197.2060809@FreeBSD.org> In-Reply-To: <47987197.2060809@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: quoted-printable User-Agent: Internet Messaging Program (IMP) H3 (4.1.5) / FreeBSD-8.0 X-BPAnet-MailScanner-Information: Please contact the ISP for more information X-BPAnet-MailScanner: Found to be clean X-BPAnet-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=-14.9, required 6, BAYES_00 -15.00, RDNS_DYNAMIC 0.10) X-BPAnet-MailScanner-From: alexander@leidinger.net X-Spam-Status: No Cc: performance@FreeBSD.org, Erik Cederstrand Subject: Re: Performance Tracker project update X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jan 2008 12:05:05 -0000 Quoting Kris Kennaway (from Thu, 24 Jan 2008 =20 12:08:07 +0100): > Erik Cederstrand wrote: >> Alexander Leidinger wrote: >>> >>> In case you have some space left for more machines, maybe someone =20 >>> is willing to help out by sponsoring some. Just tell us how many =20 >>> machines you can handle (space/power/...) and if you are =20 >>> interested that we put it up on our wantlist. >> >> I'm sharing an office with 4 others and I'm leaving for Real Work =20 >> in a month, so I'm not really in a position to expand at the =20 >> moment. However, I'd like to do so later on, and also get a more =20 >> permanent and reliable location for the machines. >> >> After weeding out the worst bugs the system runs pretty much by =20 >> itself, but since this is CURRENT there's always the possibility of =20 >> the slaves needing attention. > > I don't see machine availability being a problem once we are ready to > "take this live". Me too, but it's in the development stage and Robert asks for some new =20 features, and I've read the answer in a way that I thought that it =20 would be good to have 2-3 more machines available for development. Erik, would some more machines be helpful for development? Bye, Alexander. --=20 Lie, n.: =09A very poor substitute for the truth, but the only one =09discovered to date. http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID =3D B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID =3D 72077137 From owner-freebsd-performance@FreeBSD.ORG Thu Jan 24 12:14:21 2008 Return-Path: Delivered-To: performance@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9513B16A418; Thu, 24 Jan 2008 12:14:21 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from mx2.itu.dk (unknown [130.226.142.29]) by mx1.freebsd.org (Postfix) with ESMTP id 5CA2C13C4D3; Thu, 24 Jan 2008 12:14:21 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from wimac.littlebit.dk (unknown [85.233.238.191]) by mx2.itu.dk (Postfix) with ESMTP id 4B182F4806A; Thu, 24 Jan 2008 13:14:20 +0100 (CET) Message-ID: <47988108.60407@cederstrand.dk> Date: Thu, 24 Jan 2008 13:14:00 +0100 From: Erik Cederstrand User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: Alexander Leidinger References: <4796C717.9000507@cederstrand.dk> <47972895.4050005@FreeBSD.org> <479745DA.8010003@cederstrand.dk> <20080123193731.K63024@fledge.watson.org> <4797AABE.80509@cederstrand.dk> <20080124085800.zq6aa2cf4k4ogwo8@webmail.leidinger.net> <47986238.4080508@cederstrand.dk> <47987197.2060809@FreeBSD.org> <20080124124319.1rae3sfggwg4kss0@webmail.leidinger.net> In-Reply-To: <20080124124319.1rae3sfggwg4kss0@webmail.leidinger.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: Kris Kennaway , performance@FreeBSD.org Subject: Re: Performance Tracker project update X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jan 2008 12:14:21 -0000 Alexander Leidinger wrote: > Quoting Kris Kennaway (from Thu, 24 Jan 2008 12:08:07 >> >> I don't see machine availability being a problem once we are ready to >> "take this live". > > Me too, but it's in the development stage and Robert asks for some new > features, and I've read the answer in a way that I thought that it would > be good to have 2-3 more machines available for development. > > Erik, would some more machines be helpful for development? As Kris noted elsewhere I'm still just demonstrating the framework, and the hardware I have available for this is sufficient. When the framework goes live, more computing power is probably necessary. Erik From owner-freebsd-performance@FreeBSD.ORG Sat Jan 26 11:12:53 2008 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2FA4616A420 for ; Sat, 26 Jan 2008 11:12:53 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from mail.sweeplist.com (mail.sweeplist.com [77.93.199.51]) by mx1.freebsd.org (Postfix) with ESMTP id EA40113C448 for ; Sat, 26 Jan 2008 11:12:52 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from [192.168.1.2] (r3a200.net.upc.cz [213.220.192.200]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sweeplist.com (Postfix) with ESMTPSA id C2BB167808 for ; Sat, 26 Jan 2008 11:55:24 +0100 (CET) Message-ID: <479B1185.8020604@quip.cz> Date: Sat, 26 Jan 2008 11:55:01 +0100 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 X-Accept-Language: cz, cs, en, en-us MIME-Version: 1.0 To: freebsd-performance@freebsd.org Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Subject: PHP with open_basedir performance problem X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Jan 2008 11:12:53 -0000 Hi all, I found a painful performance problem with Apache + PHP 5 when open_basedir directive is enabled. Application performance drops by more than 50% with open_basedir enabled. There is also significant disproportion between usr / sys CPU load. (sys is much higher than usr) I do not know if this is problem on FreeBSD side or in PHP code itself. This problem exists on FreeBSD 6 and 7 (maybe older versions too - I did not check). I tested both shedulers on FreeBSD 7, no big difference (application performance is little better with 4BSD). Is there anyone with enough knowledge and time to look at it? This problem affects Mac OS X as well, but does not seem to appear on Linux. http://lists.apple.com/archives/macos-x-server/2006/Nov/msg00623.html Application workload - tested by http_load (on dual Xeon 3GHz!) with open_basedir: 11.5403 fetches/sec without open_basedir: 4.53845 fetches/sec Synthetic test: ## with open_basedir CPU states: 17.5% user, 0.0% nice, 82.3% system, 0.2% interrupt, 0.0% idle # http_load -parallel 10 -fetches 1000 load.urls 1000 fetches, 10 max parallel, 2000 bytes, in 29.3449 seconds 2 mean bytes/connection 34.0775 fetches/sec, 68.155 bytes/sec msecs/connect: 0.302057 mean, 0.441 max, 0.156 min msecs/first-response: 292.693 mean, 525.471 max, 159.389 min HTTP response codes: code 200 -- 1000 ## without open_basedir CPU states: 54.0% user, 0.0% nice, 45.2% system, 0.8% interrupt, 0.0% idle http_load -parallel 10 -fetches 1000 load.urls 1000 fetches, 10 max parallel, 2000 bytes, in 4.47231 seconds 2 mean bytes/connection 223.598 fetches/sec, 447.196 bytes/sec msecs/connect: 0.297157 mean, 0.442 max, 0.153 min msecs/first-response: 44.3433 mean, 157.334 max, 8.044 min HTTP response codes: code 200 -- 1000 Simple testcase: ---- content of index.php ---- ---- content of whatever.php ---- http_load is instructed to do 1000 fetches of index.php in 10 parallel connections. Miroslav Lachman From owner-freebsd-performance@FreeBSD.ORG Sat Jan 26 20:00:32 2008 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6BBAF16A47E for ; Sat, 26 Jan 2008 20:00:32 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from mx2.itu.dk (unknown [130.226.142.29]) by mx1.freebsd.org (Postfix) with ESMTP id 37D0E13C47E for ; Sat, 26 Jan 2008 20:00:32 +0000 (UTC) (envelope-from erik@cederstrand.dk) Received: from wimac.littlebit.dk (unknown [85.233.238.191]) by mx2.itu.dk (Postfix) with ESMTP id 0337AF48072 for ; Sat, 26 Jan 2008 21:00:30 +0100 (CET) Message-ID: <479B9147.1080801@cederstrand.dk> Date: Sat, 26 Jan 2008 21:00:07 +0100 From: Erik Cederstrand User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031) MIME-Version: 1.0 To: "freebsd-performance@freebsd.org" References: <4796C717.9000507@cederstrand.dk> In-Reply-To: <4796C717.9000507@cederstrand.dk> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: Performance Tracker project update X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Jan 2008 20:00:32 -0000 Erik Cederstrand wrote: > > I'd like to send a small update on my progress on the Performance > Tracker project. > > I now have a small setup of a server and a slave chugging along, > currently collecting data. I'm following CURRENT and collecting results > from super-smack and unixbench. > > The project still needs some work, but there's a temporary web interface > to the data here: http://littlebit.dk:5000/plot/. Apart from the > plotting it's possible to compare two dates and see the files that have > changed. Error bars are 3*standard deviation, for the points with > multiple measurements. Per the suggestions in this thread, the web interface has now grown an overview page and an alerts page. The alerts are calculated as anything outside mean +/- 2*standard deviation in a sliding window of 10 dates. This was found by trial-and-error to catch most significant changes without picking up too much noise. Anything below a 1% change is discarded. Erik