From owner-freebsd-performance@FreeBSD.ORG Tue Mar 14 06:03:05 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9FD6B16A400 for ; Tue, 14 Mar 2006 06:03:05 +0000 (UTC) (envelope-from hakmi@rogers.com) Received: from smtp103.rog.mail.re2.yahoo.com (smtp103.rog.mail.re2.yahoo.com [206.190.36.81]) by mx1.FreeBSD.org (Postfix) with SMTP id D5B8D43D45 for ; Tue, 14 Mar 2006 06:03:04 +0000 (GMT) (envelope-from hakmi@rogers.com) Received: (qmail 7997 invoked from network); 14 Mar 2006 06:03:03 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=rogers.com; h=Received:From:To:Subject:Date:MIME-Version:Content-Type:Content-Transfer-Encoding:X-Mailer:X-MimeOLE:Thread-Index; b=VYCbka6W8v6/aJaeOeDFvSVtgMysa+Nsmhb2MHKuvCxUhnYif5aDazB6C/PYi6yCZjs5keY2N679L6aepQfqJ/7gw8dkc4ZxouYTrD+0eKn9hCnCCRoIBBlljQHfasS/tUPN910A+h47kl5S0OecA9of1xBsAYK/KLxsDfRZf2k= ; Received: from unknown (HELO tamouh) (hakmi@rogers.com@70.27.160.99 with login) by smtp103.rog.mail.re2.yahoo.com with SMTP; 14 Mar 2006 06:03:03 -0000 From: "Tamouh H." To: Date: Tue, 14 Mar 2006 01:03:09 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Office Outlook, Build 11.0.6353 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2180 Thread-Index: AcZHLPg1Th+M1cqVQb+q84W12Kxojg== Message-Id: <20060314060304.D5B8D43D45@mx1.FreeBSD.org> Subject: Adaptec 2230SLP RAID-5 slow on FreeBSD 5.4 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Mar 2006 06:03:05 -0000 Hi, We're having trouble getting reasonable performance with an Adaptec = 2230SLP (FreeBSD shows it as 2130S SCSI U320) and latest FreeBSD 5.4 = snapshot using AAC drivers : __FBSDID("$FreeBSD: src/sys/dev/aac/aac.c,v 1.101.2.5=20 We're comparing diskinfo -t data between SCSI U320 and another machine = with SATA 150, and the results are surprising. We're getting around 45MB/s on the 2130S , while on SATA we receive = above 60MB/s. I've checked everything (cables, drives, played with read/write cache = enabled or disabled) but not with any significant results. I've seen other posts as well complaining about slow performance with = Adaptec 2100S series cards and FreeBSD 5.x . Any experiences with these adapters ? Is this Adaptec issue or something = with the AAC driver limiting its performance ? It is to the point where = I can actually feel the server is running awfully slow when doing = backups...etc Below are some info about the card, the drives and tests results: AAC0> controller details Executing: controller details Controller Information ---------------------- Remote Computer: S Device Name: S Controller Type: No Info Access Mode: READ-WRITE Controller Serial Number: Last Six Digits =3D 265763 Number of Buses: 1 Devices per Bus: 15 Controller CPU: Unknown Controller CPU Speed: 200 Mhz Controller Memory: 256 Mbytes Battery State: Not Present Component Revisions ------------------- CLI: 1.0-0 (Build #5263) API: 1.0-0 (Build #5263) Miniport Driver: 4.2-0 (Build #7348) Controller Software: 4.2-0 (Build #7348) Controller BIOS: 4.2-0 (Build #7348) Controller Firmware: (Build #7348) Controller Hardware: 2.64 The drives are in RAID-5 setup freebsd3# diskinfo -t /dev/aacd0s1f /dev/aacd0s1f 512 # sectorsize 221978296320 # mediasize in bytes (207G) 433551360 # mediasize in sectors 26987 # Cylinders according to firmware. 255 # Heads according to firmware. 63 # Sectors according to firmware. Seek times: Full stroke: 250 iter in 4.196076 sec =3D 16.784 msec Half stroke: 250 iter in 3.621842 sec =3D 14.487 msec Quarter stroke: 500 iter in 5.971962 sec =3D 11.944 msec Short forward: 400 iter in 6.133371 sec =3D 15.333 msec Short backward: 400 iter in 6.284642 sec =3D 15.712 msec Seq outer: 2048 iter in 3.366757 sec =3D 1.644 msec Seq inner: 2048 iter in 3.225525 sec =3D 1.575 msec Transfer rates: outside: 102400 kbytes in 3.137279 sec =3D 32640 = kbytes/sec middle: 102400 kbytes in 2.997041 sec =3D 34167 = kbytes/sec inside: 102400 kbytes in 4.061875 sec =3D 25210 = kbytes/sec -- COMPARE THIS WITH SATA DRIVE in RAID-1: freebsd1# diskinfo -t /dev/ar0s1g /dev/ar0s1g 512 # sectorsize 135622916608 # mediasize in bytes (126G) 264888509 # mediasize in sectors 16488 # Cylinders according to firmware. 255 # Heads according to firmware. 63 # Sectors according to firmware. Seek times: Full stroke: 250 iter in 1.494205 sec =3D 5.977 msec Half stroke: 250 iter in 1.417056 sec =3D 5.668 msec Quarter stroke: 500 iter in 2.862548 sec =3D 5.725 msec Short forward: 400 iter in 2.354758 sec =3D 5.887 msec Short backward: 400 iter in 2.300288 sec =3D 5.751 msec Seq outer: 2048 iter in 0.163812 sec =3D 0.080 msec Seq inner: 2048 iter in 0.177120 sec =3D 0.086 msec Transfer rates: outside: 102400 kbytes in 1.401566 sec =3D 73061 = kbytes/sec middle: 102400 kbytes in 1.664351 sec =3D 61525 = kbytes/sec inside: 102400 kbytes in 2.861386 sec =3D 35787 = kbytes/sec Any input is much appreciated... Thx, Tamouh Hakmi From owner-freebsd-performance@FreeBSD.ORG Sat Mar 18 11:44:13 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1725C16A422; Sat, 18 Mar 2006 11:44:13 +0000 (UTC) (envelope-from oxy@field.hu) Received: from green.field.hu (green.field.hu [217.20.130.28]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7B6F343D48; Sat, 18 Mar 2006 11:44:12 +0000 (GMT) (envelope-from oxy@field.hu) Received: from localhost (green.field.hu [217.20.130.28]) by green.field.hu (Postfix) with ESMTP id 77A53119CF5; Sat, 18 Mar 2006 12:43:39 +0100 (CET) Received: from green.field.hu ([217.20.130.28]) by localhost (green.field.hu [217.20.130.28]) (amavisd-new, port 10024) with ESMTP id 47312-08; Sat, 18 Mar 2006 12:43:39 +0100 (CET) Received: from oxy (dsl217-197-187-71.pool.tvnet.hu [217.197.187.71]) by green.field.hu (Postfix) with ESMTP id B75B2119CC4; Sat, 18 Mar 2006 12:43:38 +0100 (CET) Message-ID: <000a01c64a81$45eb6850$0201a8c0@oxy> From: "OxY" To: Date: Sat, 18 Mar 2006 12:44:10 +0100 MIME-Version: 1.0 X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2670 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2670 X-Virus-Scanned: by Amavisd-new (Spamassassin+Razor2+Pyzor+DCC+Bayes db, Clamd Antivirus) at field.hu Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-performance@freebsd.org Subject: packet drop with intel gigabit / marwell gigabit X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Mar 2006 11:44:13 -0000 hi! i had the packet drop problem with the marwell yukon gigabitcard: (system is an amd 2000+xp, 512mb ram, fbsd 6.0-p5) when the apache ran, with no http, just used to share files and the = traffic was 2-2,5MB/S i had 14-17% packet drop on the gigabit interface.. with the sysctl i succesfully pulled it down to 12-14%, but it was = terrible,=20 so i bought an intel pro/1000 gt. with this i have 3-6% drop with same traffic load on the other = interface.. when i stop the apache packet drop falls down to 0-0.1%, which is great. but with apache it's terrible.. first i changed some sysctl settings, benchmarked all change with iperf (udp test).i changed these: kern.maxfiles=3D65536 kern.ipc.maxsockbuf=3D50000000 kern.ipc.nmbclusters=3D5000000 kern.ipc.somaxconn=3D4096 net.inet.tcp.inflight.enable=3D1 net.inet.tcp.inflight.max=3D10737254400 net.inet.tcp.recvspace=3D65536 kern.ipc.shmmax=3D409600000 net.inet.tcp.delayed_ack=3D0 kern.polling.burst_max=3D1000 kern.polling.each_burst=3D120 kern.polling.idle_poll=3D1 kern.polling.user_frac=3D5 kern.polling.reg_frac=3D50 kern.polling.enable=3D1 net.inet.udp.maxdgram=3D57344 i have three variables in load.conf: kern.ipc.maxsockets=3D"81920" kern.maxusers=3D"512" kern.ipc.nsfbufs=3D"65536" then i tried 4BSD and ULE scheduler, ULE was better, but i accept any = suggestion and tip :) and after all i realized that it's not a bit better, than before... please give me some hints, advices, tips, sysctl settings, kernel config = parameters, anything to get rid of the packet loss.. here's my kernel config: http://field.hu/kernconf.txt thanks! From owner-freebsd-performance@FreeBSD.ORG Sat Mar 18 12:08:30 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B55A116A400 for ; Sat, 18 Mar 2006 12:08:30 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from pi.codefab.com (pi.codefab.com [199.103.21.227]) by mx1.FreeBSD.org (Postfix) with ESMTP id AF87643D6E for ; Sat, 18 Mar 2006 12:08:26 +0000 (GMT) (envelope-from cswiger@mac.com) Received: from localhost (localhost [127.0.0.1]) by pi.codefab.com (Postfix) with ESMTP id EBF7F5EFD; Sat, 18 Mar 2006 07:08:25 -0500 (EST) Received: from pi.codefab.com ([127.0.0.1]) by localhost (pi.codefab.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 27094-02; Sat, 18 Mar 2006 07:08:25 -0500 (EST) Received: from [192.168.1.3] (pool-68-160-194-11.ny325.east.verizon.net [68.160.194.11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by pi.codefab.com (Postfix) with ESMTP id D69EE5C3C; Sat, 18 Mar 2006 07:08:24 -0500 (EST) Message-ID: <441BF838.1080600@mac.com> Date: Sat, 18 Mar 2006 07:08:24 -0500 From: Chuck Swiger Organization: The Courts of Chaos User-Agent: Thunderbird 1.5 (Windows/20051201) MIME-Version: 1.0 To: OxY References: <000a01c64a81$45eb6850$0201a8c0@oxy> In-Reply-To: <000a01c64a81$45eb6850$0201a8c0@oxy> X-Enigmail-Version: 0.94.0.0 Content-Type: text/plain; charset=ISO-8859-2 Content-Transfer-Encoding: 7bit X-Virus-Scanned: amavisd-new at codefab.com Cc: freebsd-performance@freebsd.org Subject: Re: packet drop with intel gigabit / marwell gigabit X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Mar 2006 12:08:30 -0000 OxY wrote: > hi! > > i had the packet drop problem with the marwell yukon gigabitcard: > (system is an amd 2000+xp, 512mb ram, fbsd 6.0-p5) Hi-- The changes you've made in tuning the sysctls are unreasonable on a machine with only 512 MB of RAM; in particular: > net.inet.tcp.inflight.max=10737254400 ...you don't have 10GB of space for TCP!?! Remove all of the tuning you've done and start over with the default values, and then adjust those gradually if doing so improves your situation. > when the apache ran, with no http, just used to share files and the traffic was > 2-2,5MB/S i had 14-17% packet drop on the gigabit interface.. > with the sysctl i succesfully pulled it down to 12-14%, but it was terrible, > so i bought an intel pro/1000 gt. > with this i have 3-6% drop with same traffic load on the other interface.. > when i stop the apache packet drop falls down to 0-0.1%, which is great. > but with apache it's terrible.. Um. What are you measuring? If you start or stop Apache, that should make no difference to ICMP ping testing, for example.... -- -Chuck From owner-freebsd-performance@FreeBSD.ORG Sat Mar 18 12:27:28 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 92A1216A400 for ; Sat, 18 Mar 2006 12:27:28 +0000 (UTC) (envelope-from oxy@field.hu) Received: from green.field.hu (green.field.hu [217.20.130.28]) by mx1.FreeBSD.org (Postfix) with ESMTP id C1B6F43D45 for ; Sat, 18 Mar 2006 12:27:27 +0000 (GMT) (envelope-from oxy@field.hu) Received: from localhost (green.field.hu [217.20.130.28]) by green.field.hu (Postfix) with ESMTP id D868D119D2E; Sat, 18 Mar 2006 13:26:55 +0100 (CET) Received: from green.field.hu ([217.20.130.28]) by localhost (green.field.hu [217.20.130.28]) (amavisd-new, port 10024) with ESMTP id 52672-06; Sat, 18 Mar 2006 13:26:55 +0100 (CET) Received: from oxy (dsl217-197-187-71.pool.tvnet.hu [217.197.187.71]) by green.field.hu (Postfix) with ESMTP id 7AC80119CC4; Sat, 18 Mar 2006 13:26:55 +0100 (CET) Message-ID: <000601c64a87$51d7dee0$0201a8c0@oxy> From: "OxY" To: "Chuck Swiger" References: <000a01c64a81$45eb6850$0201a8c0@oxy> <441BF838.1080600@mac.com> Date: Sat, 18 Mar 2006 13:27:27 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-2"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2670 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2670 X-Virus-Scanned: by Amavisd-new (Spamassassin+Razor2+Pyzor+DCC+Bayes db, Clamd Antivirus) at field.hu Cc: freebsd-performance@freebsd.org Subject: Re: packet drop with intel gigabit / marwell gigabit X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Mar 2006 12:27:28 -0000 yeah, i googled these settings, but i put them back to default then! i measured iperf performance, and it showed that the packet drop is depending on the system load.. ----- Original Message ----- From: "Chuck Swiger" To: "OxY" Cc: Sent: Saturday, March 18, 2006 1:08 PM Subject: Re: packet drop with intel gigabit / marwell gigabit > OxY wrote: >> hi! >> >> i had the packet drop problem with the marwell yukon gigabitcard: >> (system is an amd 2000+xp, 512mb ram, fbsd 6.0-p5) > > Hi-- > > The changes you've made in tuning the sysctls are unreasonable on a > machine with > only 512 MB of RAM; in particular: > >> net.inet.tcp.inflight.max=10737254400 > > ...you don't have 10GB of space for TCP!?! Remove all of the tuning > you've done > and start over with the default values, and then adjust those gradually if > doing > so improves your situation. > >> when the apache ran, with no http, just used to share files and the >> traffic was >> 2-2,5MB/S i had 14-17% packet drop on the gigabit interface.. >> with the sysctl i succesfully pulled it down to 12-14%, but it was >> terrible, >> so i bought an intel pro/1000 gt. >> with this i have 3-6% drop with same traffic load on the other >> interface.. >> when i stop the apache packet drop falls down to 0-0.1%, which is great. >> but with apache it's terrible.. > > Um. What are you measuring? If you start or stop Apache, that should > make no > difference to ICMP ping testing, for example.... > > -- > -Chuck From owner-freebsd-performance@FreeBSD.ORG Sat Mar 18 12:38:00 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D10B616A400 for ; Sat, 18 Mar 2006 12:38:00 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from pi.codefab.com (pi.codefab.com [199.103.21.227]) by mx1.FreeBSD.org (Postfix) with ESMTP id 71AB143D45 for ; Sat, 18 Mar 2006 12:38:00 +0000 (GMT) (envelope-from cswiger@mac.com) Received: from localhost (localhost [127.0.0.1]) by pi.codefab.com (Postfix) with ESMTP id C7DC55DF4; Sat, 18 Mar 2006 07:37:59 -0500 (EST) Received: from pi.codefab.com ([127.0.0.1]) by localhost (pi.codefab.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 27094-04; Sat, 18 Mar 2006 07:37:59 -0500 (EST) Received: from [192.168.1.3] (pool-68-160-194-11.ny325.east.verizon.net [68.160.194.11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by pi.codefab.com (Postfix) with ESMTP id B254C5C6F; Sat, 18 Mar 2006 07:37:58 -0500 (EST) Message-ID: <441BFF26.90807@mac.com> Date: Sat, 18 Mar 2006 07:37:58 -0500 From: Chuck Swiger Organization: The Courts of Chaos User-Agent: Thunderbird 1.5 (Windows/20051201) MIME-Version: 1.0 To: OxY References: <000a01c64a81$45eb6850$0201a8c0@oxy> <441BF838.1080600@mac.com> <000601c64a87$51d7dee0$0201a8c0@oxy> In-Reply-To: <000601c64a87$51d7dee0$0201a8c0@oxy> X-Enigmail-Version: 0.94.0.0 Content-Type: text/plain; charset=ISO-8859-2 Content-Transfer-Encoding: 7bit X-Virus-Scanned: amavisd-new at codefab.com Cc: freebsd-performance@freebsd.org Subject: Re: packet drop with intel gigabit / marwell gigabit X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Mar 2006 12:38:00 -0000 OxY wrote: > yeah, i googled these settings, but i put them back to default then! > i measured iperf performance, and it showed that the packet drop is > depending on the system load.. If you are using the normal interrupt-driven configuration, you should look at netstat -i, -s, and vmstat -i. If you're turning on device polling, you ought to retry your testing at higher HZ (try 2000 or 5000): echo 'kern.hz="2000"' >> /boot/loader.conf -- -Chuck From owner-freebsd-performance@FreeBSD.ORG Sat Mar 18 12:41:28 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 89A9116A422 for ; Sat, 18 Mar 2006 12:41:28 +0000 (UTC) (envelope-from arne_woerner@yahoo.com) Received: from web30310.mail.mud.yahoo.com (web30310.mail.mud.yahoo.com [68.142.200.103]) by mx1.FreeBSD.org (Postfix) with SMTP id 122ED43D45 for ; Sat, 18 Mar 2006 12:41:28 +0000 (GMT) (envelope-from arne_woerner@yahoo.com) Received: (qmail 71621 invoked by uid 60001); 18 Mar 2006 12:41:27 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=ssAcZx7vd7YL4NMzKnln/OUQPXbH9Q0qQz5E8Ci2Of/KAGT0KeflyJh2azHSpCdsruTFWrihjnNVc7ufzqgI6hxyG7i6jgBmpxwjSvO4dMJmHGnu876XAdtwJxRyhn8g8wzxyTDnsoP1pQFowiOy9mFTXh5YYg55oplZ+l5nSH8= ; Message-ID: <20060318124127.71619.qmail@web30310.mail.mud.yahoo.com> Received: from [213.54.77.3] by web30310.mail.mud.yahoo.com via HTTP; Sat, 18 Mar 2006 04:41:27 PST Date: Sat, 18 Mar 2006 04:41:27 -0800 (PST) From: Arne Woerner To: OxY In-Reply-To: <000601c64a87$51d7dee0$0201a8c0@oxy> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: freebsd-performance@freebsd.org Subject: Re: packet drop with intel gigabit / marwell gigabit X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Mar 2006 12:41:28 -0000 --- OxY wrote: > i measured iperf performance, and it showed that the packet drop > is depending on the system load.. > I have something similar here, I think: When my CPU throttles to 1/16 (due to powerd) I get "vr0: rx packet lost" messages (while I transfer with about 10Mbit/sec) in my /var/log/messages, which does not seem to happen (so often), when CPU is not throttled... Do u have something like that in ur log files, too? -Arne __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-performance@FreeBSD.ORG Sat Mar 18 13:23:13 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 42CE516A400 for ; Sat, 18 Mar 2006 13:23:13 +0000 (UTC) (envelope-from oxy@field.hu) Received: from green.field.hu (green.field.hu [217.20.130.28]) by mx1.FreeBSD.org (Postfix) with ESMTP id BA77C43D45 for ; Sat, 18 Mar 2006 13:23:12 +0000 (GMT) (envelope-from oxy@field.hu) Received: from localhost (green.field.hu [217.20.130.28]) by green.field.hu (Postfix) with ESMTP id B3577119D2E; Sat, 18 Mar 2006 14:22:39 +0100 (CET) Received: from green.field.hu ([217.20.130.28]) by localhost (green.field.hu [217.20.130.28]) (amavisd-new, port 10024) with ESMTP id 56541-06; Sat, 18 Mar 2006 14:22:39 +0100 (CET) Received: from oxy (dsl217-197-187-71.pool.tvnet.hu [217.197.187.71]) by green.field.hu (Postfix) with ESMTP id 667FC119CC4; Sat, 18 Mar 2006 14:22:39 +0100 (CET) Message-ID: <000e01c64a8f$1b2bec80$0201a8c0@oxy> From: "OxY" To: "Chuck Swiger" References: <000a01c64a81$45eb6850$0201a8c0@oxy> <441BF838.1080600@mac.com> <000601c64a87$51d7dee0$0201a8c0@oxy> <441BFF26.90807@mac.com> Date: Sat, 18 Mar 2006 14:23:12 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-2"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2670 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2670 X-Virus-Scanned: by Amavisd-new (Spamassassin+Razor2+Pyzor+DCC+Bayes db, Clamd Antivirus) at field.hu Cc: freebsd-performance@freebsd.org Subject: Re: packet drop with intel gigabit / marwell gigabit X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Mar 2006 13:23:13 -0000 currently i use HZ=2000 here's the output of netstat -i, -s, and vmstat -i : (currently i am uploading on the gigabit with ftp, 3 threads) Field root# vmstat -i interrupt total rate irq0: clk 27503959 1993 irq1: atkbd0 1 0 irq3: fxp0 2 0 irq7: 146 0 stray irq7 146 0 irq8: rtc 1765569 127 irq10: atapci1 2807786 203 irq11: atapci0 475039 34 irq13: npx0 1 0 irq14: ata0 99 0 Total 32552748 2359 Field root# netstat -i Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll fxp0 1500 00:a0:c9:8d:79:68 13163545 0 21899372 1 0 fxp0 1500 195.38.96.64/ field 141 - 22056 - - em0 1500 00:0e:0c:a2:ac:42 68644181 4 66793904 0 0 em0 1500 195.38.96.64/ field 211255811 - 3903 - - lo0 16384 129622061 0 129622061 0 0 netstat -s is here: http://field.hu/netstat.txt ----- Original Message ----- From: "Chuck Swiger" To: "OxY" Cc: Sent: Saturday, March 18, 2006 1:37 PM Subject: Re: packet drop with intel gigabit / marwell gigabit > OxY wrote: >> yeah, i googled these settings, but i put them back to default then! >> i measured iperf performance, and it showed that the packet drop is >> depending on the system load.. > > If you are using the normal interrupt-driven configuration, you should > look at > netstat -i, -s, and vmstat -i. If you're turning on device polling, you > ought > to retry your testing at higher HZ (try 2000 or 5000): > > echo 'kern.hz="2000"' >> /boot/loader.conf > > -- > -Chuck From owner-freebsd-performance@FreeBSD.ORG Sat Mar 18 13:42:15 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 895ED16A400 for ; Sat, 18 Mar 2006 13:42:15 +0000 (UTC) (envelope-from oxy@field.hu) Received: from green.field.hu (green.field.hu [217.20.130.28]) by mx1.FreeBSD.org (Postfix) with ESMTP id CDCEA43D45 for ; Sat, 18 Mar 2006 13:42:14 +0000 (GMT) (envelope-from oxy@field.hu) Received: from localhost (green.field.hu [217.20.130.28]) by green.field.hu (Postfix) with ESMTP id 76F1D119D2E; Sat, 18 Mar 2006 14:41:42 +0100 (CET) Received: from green.field.hu ([217.20.130.28]) by localhost (green.field.hu [217.20.130.28]) (amavisd-new, port 10024) with ESMTP id 59800-02; Sat, 18 Mar 2006 14:41:41 +0100 (CET) Received: from oxy (dsl217-197-187-71.pool.tvnet.hu [217.197.187.71]) by green.field.hu (Postfix) with ESMTP id C9455119CC4; Sat, 18 Mar 2006 14:41:40 +0100 (CET) Message-ID: <000401c64a91$c3962c30$0201a8c0@oxy> From: "OxY" To: "OxY" , "Chuck Swiger" References: <000a01c64a81$45eb6850$0201a8c0@oxy> <441BF838.1080600@mac.com><000601c64a87$51d7dee0$0201a8c0@oxy> <441BFF26.90807@mac.com> <000e01c64a8f$1b2bec80$0201a8c0@oxy> Date: Sat, 18 Mar 2006 14:42:13 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-2"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2670 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2670 X-Virus-Scanned: by Amavisd-new (Spamassassin+Razor2+Pyzor+DCC+Bayes db, Clamd Antivirus) at field.hu Cc: freebsd-performance@freebsd.org Subject: Re: packet drop with intel gigabit / marwell gigabit X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Mar 2006 13:42:15 -0000 i increased hz from 2000 to 5000, now the packet loss is decreased from 5-6% to 0.6-0,8% !!! huge improve! should i increase hz more? ----- Original Message ----- From: "OxY" To: "Chuck Swiger" Cc: Sent: Saturday, March 18, 2006 2:23 PM Subject: Re: packet drop with intel gigabit / marwell gigabit > currently i use HZ=2000 > here's the output of netstat -i, -s, and vmstat -i : > (currently i am uploading on the gigabit with ftp, 3 threads) > > Field root# vmstat -i > interrupt total rate > irq0: clk 27503959 1993 > irq1: atkbd0 1 0 > irq3: fxp0 2 0 > irq7: 146 0 > stray irq7 146 0 > irq8: rtc 1765569 127 > irq10: atapci1 2807786 203 > irq11: atapci0 475039 34 > irq13: npx0 1 0 > irq14: ata0 99 0 > Total 32552748 2359 > > Field root# netstat -i > Name Mtu Network Address Ipkts Ierrs Opkts Oerrs > Coll > fxp0 1500 00:a0:c9:8d:79:68 13163545 0 21899372 1 > 0 > fxp0 1500 195.38.96.64/ field 141 - > 6 - - > em0 1500 00:0e:0c:a2:ac:42 68644181 4 66793904 0 > 0 > em0 1500 195.38.96.64/ field 211255811 - > - - > lo0 16384 129622061 0 129622061 > 0 0 > > netstat -s is here: > http://field.hu/netstat.txt > > ----- Original Message ----- > From: "Chuck Swiger" > To: "OxY" > Cc: > Sent: Saturday, March 18, 2006 1:37 PM > Subject: Re: packet drop with intel gigabit / marwell gigabit > > >> OxY wrote: >>> yeah, i googled these settings, but i put them back to default then! >>> i measured iperf performance, and it showed that the packet drop is >>> depending on the system load.. >> >> If you are using the normal interrupt-driven configuration, you should >> look at >> netstat -i, -s, and vmstat -i. If you're turning on device polling, you >> ought >> to retry your testing at higher HZ (try 2000 or 5000): >> >> echo 'kern.hz="2000"' >> /boot/loader.conf >> >> -- >> -Chuck > > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to > "freebsd-performance-unsubscribe@freebsd.org" From owner-freebsd-performance@FreeBSD.ORG Sat Mar 18 14:00:12 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B3A4916A41F for ; Sat, 18 Mar 2006 14:00:12 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from pi.codefab.com (pi.codefab.com [199.103.21.227]) by mx1.FreeBSD.org (Postfix) with ESMTP id 31B1643D46 for ; Sat, 18 Mar 2006 14:00:12 +0000 (GMT) (envelope-from cswiger@mac.com) Received: from localhost (localhost [127.0.0.1]) by pi.codefab.com (Postfix) with ESMTP id 8C61C5EFA; Sat, 18 Mar 2006 09:00:11 -0500 (EST) Received: from pi.codefab.com ([127.0.0.1]) by localhost (pi.codefab.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 67385-08; Sat, 18 Mar 2006 09:00:11 -0500 (EST) Received: from [192.168.1.3] (pool-68-160-194-11.ny325.east.verizon.net [68.160.194.11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by pi.codefab.com (Postfix) with ESMTP id A8E855EE9; Sat, 18 Mar 2006 09:00:10 -0500 (EST) Message-ID: <441C126A.7060109@mac.com> Date: Sat, 18 Mar 2006 09:00:10 -0500 From: Chuck Swiger Organization: The Courts of Chaos User-Agent: Thunderbird 1.5 (Windows/20051201) MIME-Version: 1.0 To: OxY References: <000a01c64a81$45eb6850$0201a8c0@oxy> <441BF838.1080600@mac.com> <000601c64a87$51d7dee0$0201a8c0@oxy> <441BFF26.90807@mac.com> <000e01c64a8f$1b2bec80$0201a8c0@oxy> <000401c64a91$c3962c30$0201a8c0@oxy> In-Reply-To: <000401c64a91$c3962c30$0201a8c0@oxy> X-Enigmail-Version: 0.94.0.0 Content-Type: text/plain; charset=ISO-8859-2 Content-Transfer-Encoding: 7bit X-Virus-Scanned: amavisd-new at codefab.com Cc: freebsd-performance@freebsd.org Subject: Re: packet drop with intel gigabit / marwell gigabit X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Mar 2006 14:00:12 -0000 OxY wrote: > i increased hz from 2000 to 5000, now the packet loss is decreased > from 5-6% to 0.6-0,8% !!! > huge improve! Good deal. > should i increase hz more? Experiment. :-) Keep track of the numbers you get, and post a summary once you've had a day or two to shake things down. -- -Chuck From owner-freebsd-performance@FreeBSD.ORG Sat Mar 18 14:12:19 2006 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 16F0216A400 for ; Sat, 18 Mar 2006 14:12:19 +0000 (UTC) (envelope-from oxy@field.hu) Received: from green.field.hu (green.field.hu [217.20.130.28]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8D16043D45 for ; Sat, 18 Mar 2006 14:12:18 +0000 (GMT) (envelope-from oxy@field.hu) Received: from localhost (green.field.hu [217.20.130.28]) by green.field.hu (Postfix) with ESMTP id E596F119D52; Sat, 18 Mar 2006 15:11:45 +0100 (CET) Received: from green.field.hu ([217.20.130.28]) by localhost (green.field.hu [217.20.130.28]) (amavisd-new, port 10024) with ESMTP id 59776-10; Sat, 18 Mar 2006 15:11:45 +0100 (CET) Received: from oxy (dsl217-197-187-71.pool.tvnet.hu [217.197.187.71]) by green.field.hu (Postfix) with ESMTP id A8D5F119D4E; Sat, 18 Mar 2006 15:11:45 +0100 (CET) Message-ID: <000c01c64a95$f7759aa0$0201a8c0@oxy> From: "OxY" To: "Chuck Swiger" References: <000a01c64a81$45eb6850$0201a8c0@oxy> <441BF838.1080600@mac.com> <000601c64a87$51d7dee0$0201a8c0@oxy> <441BFF26.90807@mac.com> <000e01c64a8f$1b2bec80$0201a8c0@oxy> <000401c64a91$c3962c30$0201a8c0@oxy> <441C126A.7060109@mac.com> Date: Sat, 18 Mar 2006 15:12:18 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-2"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.2670 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2670 X-Virus-Scanned: by Amavisd-new (Spamassassin+Razor2+Pyzor+DCC+Bayes db, Clamd Antivirus) at field.hu Cc: freebsd-performance@freebsd.org Subject: Re: packet drop with intel gigabit / marwell gigabit X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Mar 2006 14:12:19 -0000 damn! after the apache downloaders came back fully (40-50 download at time /2-4M/s traffic on fxp0) the loss is 5-8% again on em0.. polling is enabled for both interfaces. should i change something related to polling? sysctl vars for example? (i put back everything to default) ----- Original Message ----- From: "Chuck Swiger" To: "OxY" Cc: Sent: Saturday, March 18, 2006 3:00 PM Subject: Re: packet drop with intel gigabit / marwell gigabit > OxY wrote: >> i increased hz from 2000 to 5000, now the packet loss is decreased >> from 5-6% to 0.6-0,8% !!! >> huge improve! > > Good deal. > >> should i increase hz more? > > Experiment. :-) Keep track of the numbers you get, and post a summary > once > you've had a day or two to shake things down. > > -- > -Chuck