From owner-freebsd-performance@FreeBSD.ORG Fri Nov 11 13:55:04 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3C85916A41F for ; Fri, 11 Nov 2005 13:55:04 +0000 (GMT) (envelope-from joao.barros@gmail.com) Received: from xproxy.gmail.com (xproxy.gmail.com [66.249.82.203]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4581143D73 for ; Fri, 11 Nov 2005 13:54:50 +0000 (GMT) (envelope-from joao.barros@gmail.com) Received: by xproxy.gmail.com with SMTP id t12so547580wxc for ; Fri, 11 Nov 2005 05:54:49 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=odw1SU2ZTi12CFuK8EVpPGQuhVL5+tlVorNsynX60FartXY2tWd99KAEBTWPSUrBZoXiAmDVkW0chtQlOh2oyR8M8+OOq4ftAN7Gks+5oonanqzCEj0kMd4Psq47znIx9aEngcqH4QEs91PLGLpzNn+5KdUUA2m1VWQ/tjtWGoc= Received: by 10.70.35.9 with SMTP id i9mr2155001wxi; Fri, 11 Nov 2005 05:54:48 -0800 (PST) Received: by 10.70.9.10 with HTTP; Fri, 11 Nov 2005 05:54:48 -0800 (PST) Message-ID: <70e8236f0511110554v77ea474bvdf6a132eef23f954@mail.gmail.com> Date: Fri, 11 Nov 2005 13:54:48 +0000 From: Joao Barros To: Robert Watson In-Reply-To: <20051111121144.P68007@fledge.watson.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline References: <70e8236f0511050457s5ce6d8batf805fbc9edd91360@mail.gmail.com> <20051109060931.GD5188@obiwan.tataz.chchile.org> <70e8236f0511090517g29b3f887x1b97ef5dec04548@mail.gmail.com> <20051111121144.P68007@fledge.watson.org> Cc: freebsd-performance@freebsd.org, Jeremie Le Hen Subject: Re: Poor Samba throughput on 6.0 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Nov 2005 13:55:04 -0000 On 11/11/05, Robert Watson wrote: > > On Wed, 9 Nov 2005, Joao Barros wrote: > > > I tried using a single drive, an IDE and a SCSI-2 and on 2 machines at > > work both with a RAID1. Even better, there is a part in my initial emai= l > > where I mention that having a 700MB file cached (iostat reported no > > reads) the results were the same. With this in mind I don't think the > > problem is at the storage level. > > My feeling is that it is likely related to the network stack -- perhaps a > combination of Samba<->sockets and TCP<->Windows, hence pointing at thing= s > like TCP settings. It would be quite interesting to know what top -S > looks like during (perhaps two minutes into) a long-haul transfer, and th= e > output of systat -vmstat 1. I know Apple has done some work on the > efficiency of the FreeBSD SMB client when talking to windows servers > relating to how TCP behaves, but haven't heard anything about them lookin= g > at similar issues in Samba. > > Robert N M Watson > Output of top -S and systat -vmstat 1 after copying a file twice (read) uncached and then cached: top -S: last pid: 3744; load averages: 0.12, 0.06, 0.02 =20 up 0+23:41:09 13:48:20 100 processes: 3 running, 57 sleeping, 40 waiting CPU states: 0.0% user, 0.0% nice, 2.1% system, 4.9% interrupt, 93.0% id= le Mem: 16M Active, 812M Inact, 179M Wired, 212K Cache, 112M Buf, 116M Free Swap: 2048M Total, 2048M Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 11 root 1 171 52 0K 8K CPU1 0 0:00 99.71% idle: cpu= 1 12 root 1 171 52 0K 8K RUN 0 23.6H 83.06% idle: cpu= 0 540 j.barros 1 96 0 5736K 2704K select 0 0:22 6.35% smbd 31 root 1 -68 -187 0K 8K WAIT 0 0:22 4.64% irq19: bg= e0 44 root 1 -44 -163 0K 8K WAIT 0 0:13 1.22% swi1: net 45 root 1 -32 -151 0K 8K WAIT 0 0:54 0.00% swi4: clock sio 63 root 1 20 0 0K 8K syncer 0 0:45 0.00% syncer 4 root 1 -8 0 0K 8K - 0 0:05 0.00% g_down 3 root 1 -8 0 0K 8K - 0 0:05 0.00% g_up 47 root 1 -16 0 0K 8K - 0 0:05 0.00% yarrow 2 root 1 -8 0 0K 8K - 0 0:04 0.00% g_event 462 root 1 96 0 3552K 1404K select 0 0:02 0.00% nmbd 68 root 1 12 0 0K 8K - 0 0:02 0.00% schedcpu 425 root 1 96 0 3396K 2236K select 0 0:01 0.00% sendmail 60 root 1 171 52 0K 8K pgzero 0 0:01 0.00% pagezero 49 root 1 -40 -159 0K 8K WAIT 0 0:01 0.00% swi2: cam= bio 55 root 1 0 0 0K 8K tzpoll 0 0:01 0.00% acpi_ther= mal 33 root 1 -64 -183 0K 8K WAIT 0 0:01 0.00% irq21: ci= ss0 529 j.barros 1 96 0 6076K 2272K select 0 0:01 0.00% sshd 61 root 1 -16 0 0K 8K psleep 0 0:00 0.00% bufdaemon 57 root 1 -8 0 0K 8K - 0 0:00 0.00% fdc0 62 root 1 -4 0 0K 8K vlruwt 0 0:00 0.00% vnlru 441 root 1 8 0 1312K 884K nanslp 0 0:00 0.00% cron 3741 root 1 96 0 2300K 1404K CPU0 0 0:00 0.00% top 526 root 1 4 0 6096K 2216K sbwait 0 0:00 0.00% sshd 295 root 1 96 0 1292K 768K select 0 0:00 0.00% syslogd 58 root 1 -16 0 0K 8K psleep 0 0:00 0.00% pagedaemo= n 376 root 1 96 0 1208K 672K select 0 0:00 0.00% usbd 533 root 1 20 0 3752K 2352K pause 0 0:00 0.00% csh 3719 j.barros 1 96 0 6076K 2412K select 0 0:00 0.00% sshd 3716 root 1 4 0 6096K 2400K sbwait 0 0:00 0.00% sshd 429 smmsp 1 20 0 3296K 1980K pause 0 0:00 0.00% sendmail 466 root 1 96 0 5384K 2308K select 0 0:00 0.00% smbd 3723 root 1 5 0 3732K 2296K ttyin 0 0:00 0.00% csh 1 root 1 8 0 724K 244K wait 0 0:00 0.00% init systat -vmstat 1: 2 users Load 0.13 0.06 0.02 Nov 11 13:48 Mem:KB REAL VIRTUAL VN PAGER SWAP PAGER Tot Share Tot Share Free in out in out Act 15660 2460 58664 3832 119084 count All 1029840 4828 3304436 7592 pages Interrupts Proc:r p d s w Csw Trp Sys Int Sof Flt cow 8970 total 1 30 15030 8 2722 8500 2 22 182924 wire 1: at= kb 16352 act 6: fd= c0 3.7%Sys 3.7%Intr 0.4%User 0.0%Nice 92.2%Idl 831256 inact 13: n= px | | | | | | | | | | 212 cache 14: a= ta =3D=3D++ 118872 free 4955 1= 9: bge daefr 1 21: c= is Namei Name-cache Dir-cache prcfr 2007 cpu0:= time Calls hits % hits % react 2007 cpu1:= time pdwake zfod pdpgs Disks da0 pass0 ofod intrn KB/t 2.00 0.00 %slo-z 114464 buf tps 1 0 tfree 5 dirtybuf MB/s 0.00 0.00 78034 desiredvnodes % busy 2 0 54388 numvnodes 19504 freevnodes dmesg: Copyright (c) 1992-2005 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 6.0-RELEASE #5: Thu Nov 10 13:57:54 WET 2005 root@bio:/usr/obj/usr/src/sys/SMP Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Pentium(R) 4 CPU 3.06GHz (3065.82-MHz 686-class CPU) Origin =3D "GenuineIntel" Id =3D 0xf29 Stepping =3D 9 Features=3D0xbfebfbff Features2=3D0x4400> Hyperthreading: 2 logical CPUs real memory =3D 1207934976 (1151 MB) avail memory =3D 1173495808 (1119 MB) ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 ioapic0: Changing APIC ID to 2 ioapic1: Changing APIC ID to 3 MADT: Forcing active-low polarity and level trigger for SCI ioapic0 irqs 0-15 on motherboard ioapic1 irqs 16-31 on motherboard npx0: [FAST] npx0: on motherboard npx0: INT 16 interface acpi0: on motherboard acpi0: Power Button (fixed) pci_link0: on acpi0 pci_link1: on acpi0 pci_link2: on acpi0 pci_link3: on acpi0 pci_link4: on acpi0 pci_link5: on acpi0 pci_link6: on acpi0 pci_link7: on acpi0 Timecounter "ACPI-safe" frequency 3579545 Hz quality 1000 acpi_timer0: <32-bit timer at 3.579545MHz> port 0x920-0x923 on acpi0 cpu0: on acpi0 cpu1: on acpi0 pcib0: on acpi0 pci0: on pcib0 atapci0: port 0x2010-0x2017,0x2018-0x201b,0x2020-0x2027,0x2028-0x202b,0x2030-0x203f irq 17 at device 2.0 on pci0 ata2: on atapci0 ata3: on atapci0 pci0: at device 3.0 (no driver attached) pci0: at device 4.0 (no driver attached) bge0: mem 0xf6fd0000-0xf6fdffff irq 19 at device 5.0 on pci0 miibus0: on bge0 brgphy0: on miibus0 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, 1000baseTX-FDX, auto bge0: Ethernet address: 00:12:79:92:0e:e9 bge1: mem 0xf6fc0000-0xf6fcffff irq 20 at device 6.0 on pci0 miibus1: on bge1 brgphy1: on miibus1 brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, 1000baseTX-FDX, auto bge1: Ethernet address: 00:12:79:92:0e:e0 ciss0: port 0x2800-0x28ff mem 0xf6fb0000-0xf6fb1fff,0xf6f40000-0xf6f7ffff irq 21 at device 7.0 on pci0 ciss0: [GIANT-LOCKED] isab0: at device 15.0 on pci0 isa0: on isab0 atapci1: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x2000-0x200f at device 15.1 on pci0 ata0: on atapci1 ata1: on atapci1 ohci0: mem 0xf6f30000-0xf6f30fff irq 15 at device 15.2 on pci0 ohci0: [GIANT-LOCKED] usb0: OHCI version 1.0, legacy support usb0: SMM does not respond, resetting usb0: on ohci0 usb0: USB revision 1.0 uhub0: (0x1166) OHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 4 ports with 4 removable, self powered acpi_button0: on acpi0 acpi_tz0: on acpi0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] sio0: port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 sio0: type 16550A fdc0: port 0x3f2-0x3f5 irq 6 drq 2 on acpi0 fdc0: [FAST] fd0: <1440-KB 3.5" drive> on fdc0 drive 0 pmtimer0 on isa0 orm0: at iomem 0xc0000-0xc7fff,0xc8000-0xcbfff,0xcc000-0xcd7ff,0xee000-0xeffff on isa0 ppc0: parallel port not found. sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=3D0x300> sio1: configured irq 3 not in bitmap of probed irqs 0 sio1: port may not be enabled vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Timecounters tick every 1.000 msec acd0: CDROM at ata0-master PIO4 da0 at ciss0 bus 0 target 0 lun 0 da0: Fixed Direct Access SCSI-0 device da0: 135.168MB/s transfers da0: 34998MB (71677440 512 byte sectors: 255H 32S/T 8784C) SMP: AP CPU #1 Launched! Trying to mount root from ufs:/dev/da0s1a bge0: link state changed to UP Offtopic: Robert notice the 2000+ interrupts/s per cpu on systat. Is that normal? -- Joao Barros