Date: Tue, 21 Sep 2010 20:02:09 -0700 (PDT) From: Bryce <bryce@bryce.net> To: Peter Jeremy <peterjeremy@acm.org> Cc: freebsd-stable@freebsd.org Subject: Re: SuperMicro i7 (UP) - very slow performance Message-ID: <10b0bdef-2bb1-44c8-9ffb-7d3167147a4f@q2g2000vbk.googlegroups.com> In-Reply-To: <mailpost.1284981513.7745206.53335.mailing.freebsd.stable@FreeBSD.cs.nctu.edu.tw> References: <AANLkTin2XH8Gs0ytgHsuF50iUJ3JFQU_OO3xSRfq3-C8@mail.gmail.com> <mailpost.1284981513.7745206.53335.mailing.freebsd.stable@FreeBSD.cs.nctu.edu.tw>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sep 20, 6:17=A0am, peterjer...@acm.org (Peter Jeremy) wrote:
> On 2010-Sep-18 08:32:32 -0500, Bryce Edwards <br...@bryce.net> wrote:
>
> >I have a Supermicro with the C7X58 motherboard and an i7 930 cpu, and
> >it is nowhere near the performance it should be. =A0A buildworld just
> >took 22.5 hours!
>
> That does sound a bit poor. =A0I presume the system was basically unloade=
d
> during the buildworld.
>
> Can we see the output of:
> - vmstat -i
interrupt total rate
irq1: atkbd0 853 0
irq16: uhci0 mpt0 117706 0
irq17: fwohci0 1 0
irq23: uhci3 ehci1 22 0
cpu0: timer 1046064200 1979
irq256: em0 1861363 3
irq258: ahci0 3922875 7
cpu1: timer 1046058992 1979
cpu4: timer 1046056958 1979
cpu3: timer 1046058951 1979
cpu2: timer 1046055416 1979
cpu6: timer 1046057518 1979
cpu5: timer 1046058850 1979
cpu7: timer 1046058744 1979
Total 8374372449 15850
> - zpool status -v
pool: data
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
data ONLINE 0 0 0
mirror ONLINE 0 0 0
gpt/hitachi0 ONLINE 0 0 0
gpt/hitachi1 ONLINE 0 0 0
errors: No known data errors
pool: ext1
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
ext1 ONLINE 0 0 0
da2 ONLINE 0 0 0
errors: No known data errors
pool: storage
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
storage ONLINE 0 0 0
raidz1 ONLINE 0 0 0
gpt/samsung0 ONLINE 0 0 0
gpt/samsung1 ONLINE 0 0 0
gpt/samsung2 ONLINE 0 0 0
gpt/samsung3 ONLINE 0 0 0
errors: No known data errors
pool: system
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
system ONLINE 0 0 0
mirror ONLINE 0 0 0
gpt/disk0 ONLINE 0 0 0
gpt/disk1 ONLINE 0 0 0
errors: No known data errors
> - df -k
Filesystem 1024-blocks Used Avail Capacity
Mounted on
system 142091977 611793 141480184 0% /
devfs 1 1 0 100% /dev
linprocfs 4 4 0 100% /
compat/linux/proc
storage 2729825067 38 2729825028 0% /
storage
storage/archive 2959080344 229255316 2729825028 8% /
storage/archive
storage/backups 2906677465 176852436 2729825028 6% /
storage/backups
data 1817823312 2487426 1815335886 0% /
storage/data
data/Pictures 1912868417 97532531 1815335886 5% /
storage/data/Pictures
storage/files 2794105921 64280893 2729825028 2% /
storage/files
storage/incoming 3712303044 982478016 2729825028 26% /
storage/incoming
storage/media 2835215837 105390809 2729825028 4% /
storage/media
storage/media/Games 2752775684 22950655 2729825028 1% /
storage/media/Games
storage/media/Movies 3745007442 1015182414 2729825028 27% /
storage/media/Movies
storage/media/Music 2811591585 81766557 2729825028 3% /
storage/media/Music
system/usr 142342976 862792 141480184 1% /usr
system/usr/home 142105804 625620 141480184 0% /usr/
home
system/usr/obj 143296312 1816128 141480184 1% /usr/
obj
system/usr/ports 142059536 579352 141480184 0% /usr/
ports
system/usr/src 141792965 312781 141480184 0% /usr/
src
system/var 141715785 235601 141480184 0% /var
/dev/md0 19566 12 19554 0% /tmp
ext1 1092323986 25 1092323960 0% /mnt
ext1/data 1094811390 2487429 1092323960 0% /mnt/
data
ext1/data/Pictures 1189727469 97403509 1092323960 8% /mnt/
data/Pictures
ext1/storage 1092323987 27 1092323960 0% /mnt/
storage
ext1/storage/backups 1269282867 176958906 1092323960 14% /mnt/
storage/backups
ext1/storage/files 1156645316 64321355 1092323960 6% /mnt/
storage/files
ext1/system 1092935755 611794 1092323960 0% /mnt/
system
ext1/system/usr 1093186866 862905 1092323960 0% /mnt/
system/usr
ext1/system/usr/home 1092949651 625691 1092323960 0% /mnt/
system/usr/home
ext1/system/var 1092559374 235413 1092323960 0% /mnt/
system/var
> - mount -v
system on / (zfs, local, noatime)
devfs on /dev (devfs, local, multilabel)
linprocfs on /compat/linux/proc (linprocfs, local)
storage on /storage (zfs, local, noatime)
storage/archive on /storage/archive (zfs, local, noatime)
storage/backups on /storage/backups (zfs, local, noatime)
data on /storage/data (zfs, local)
data/Pictures on /storage/data/Pictures (zfs, local)
storage/files on /storage/files (zfs, local, noatime)
storage/incoming on /storage/incoming (zfs, local, noatime)
storage/media on /storage/media (zfs, local, noatime)
storage/media/Games on /storage/media/Games (zfs, local, noatime)
storage/media/Movies on /storage/media/Movies (zfs, local, noatime)
storage/media/Music on /storage/media/Music (zfs, local, noatime)
system/usr on /usr (zfs, local, noatime)
system/usr/home on /usr/home (zfs, local, noatime, nosuid)
system/usr/obj on /usr/obj (zfs, local, noatime)
system/usr/ports on /usr/ports (zfs, local, noatime, nosuid)
system/usr/src on /usr/src (zfs, local, noatime, noexec, nosuid)
system/var on /var (zfs, local, noatime)
/dev/md0 on /tmp (ufs, asynchronous, local, noatime)
ext1 on /mnt (zfs, local)
ext1/data on /mnt/data (zfs, local)
ext1/data/Pictures on /mnt/data/Pictures (zfs, local)
ext1/storage on /mnt/storage (zfs, local)
ext1/storage/backups on /mnt/storage/backups (zfs, local)
ext1/storage/files on /mnt/storage/files (zfs, local)
ext1/system on /mnt/system (zfs, local)
ext1/system/usr on /mnt/system/usr (zfs, local)
ext1/system/usr/home on /mnt/system/usr/home (zfs, local)
ext1/system/var on /mnt/system/var (zfs, local)
> - md5 -t =A0 [this will help determine if the problem is lack of CPU]
MD5 time trial. Digesting 100000 10000-byte blocks ... done
Digest =3D 766a2bb5d24bddae466c572bcabca3ee
Time =3D 5.421381 seconds
Speed =3D 184454848.000000 bytes/second
> - sysctl vm
vm.vmtotal:
System wide totals computed every five seconds: (values in kilobytes)
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
Processes: (RUNQ: 1 Disk Wait: 0 Page Wait: 0 Sleep: 32)
Virtual Memory: (Total: 1074279508K, Active 422412K)
Real Memory: (Total: 1252504K Active 28660K)
Shared Virtual Memory: (Total: 28792K Active: 13456K)
Shared Real Memory: (Total: 8716K Active: 6520K)
Free Memory Pages: 3035952K
vm.loadavg: { 0.00 0.00 0.00 }
vm.v_free_min: 9590
vm.v_free_target: 40381
vm.v_free_reserved: 2021
vm.v_inactive_target: 60571
vm.v_cache_min: 40381
vm.v_cache_max: 80762
vm.v_pageout_free_min: 34
vm.pageout_algorithm: 0
vm.swap_enabled: 1
vm.kmem_size_scale: 3
vm.kmem_size_max: 329853485875
vm.kmem_size_min: 0
vm.kmem_size: 2067320832
vm.nswapdev: 2
vm.dmmax: 32
vm.swap_async_max: 4
vm.overcommit: 0
vm.swap_reserved: 395976704
vm.swap_total: 8589934592
vm.zone_count: 191
vm.swap_idle_threshold2: 10
vm.swap_idle_threshold1: 2
vm.kstacks: 724
vm.kstack_cache_size: 128
vm.exec_map_entries: 16
vm.stats.misc.zero_page_count: 189
vm.stats.misc.cnt_prezero: 0
vm.stats.vm.v_kthreadpages: 0
vm.stats.vm.v_rforkpages: 0
vm.stats.vm.v_vforkpages: 36355379
vm.stats.vm.v_forkpages: 113127423
vm.stats.vm.v_kthreads: 28
vm.stats.vm.v_rforks: 0
vm.stats.vm.v_vforks: 173728
vm.stats.vm.v_forks: 441695
vm.stats.vm.v_interrupt_free_min: 2
vm.stats.vm.v_pageout_free_min: 34
vm.stats.vm.v_cache_max: 80762
vm.stats.vm.v_cache_min: 40381
vm.stats.vm.v_cache_count: 8123
vm.stats.vm.v_inactive_count: 1310
vm.stats.vm.v_inactive_target: 60571
vm.stats.vm.v_active_count: 41328
vm.stats.vm.v_wire_count: 712096
vm.stats.vm.v_free_count: 750836
vm.stats.vm.v_free_min: 9590
vm.stats.vm.v_free_target: 40381
vm.stats.vm.v_free_reserved: 2021
vm.stats.vm.v_page_count: 1514151
vm.stats.vm.v_page_size: 4096
vm.stats.vm.v_tfree: 167732869
vm.stats.vm.v_pfree: 98570493
vm.stats.vm.v_dfree: 0
vm.stats.vm.v_tcached: 103875
vm.stats.vm.v_pdpages: 0
vm.stats.vm.v_pdwakeups: 0
vm.stats.vm.v_reactivated: 72388
vm.stats.vm.v_intrans: 4408
vm.stats.vm.v_vnodepgsout: 12785
vm.stats.vm.v_vnodepgsin: 109922
vm.stats.vm.v_vnodeout: 10042
vm.stats.vm.v_vnodein: 109922
vm.stats.vm.v_swappgsout: 0
vm.stats.vm.v_swappgsin: 0
vm.stats.vm.v_swapout: 0
vm.stats.vm.v_swapin: 0
vm.stats.vm.v_ozfod: 0
vm.stats.vm.v_zfod: 113400918
vm.stats.vm.v_cow_optim: 33371
vm.stats.vm.v_cow_faults: 18233444
vm.stats.vm.v_vm_faults: 147005945
vm.stats.sys.v_soft: 81806059
vm.stats.sys.v_intr: 5903805
vm.stats.sys.v_syscall: 126779283
vm.stats.sys.v_trap: 157285021
vm.stats.sys.v_swtch: 252443163
vm.stats.object.bypasses: 143582
vm.stats.object.collapses: 1817130
vm.v_free_severe: 5805
vm.max_proc_mmap: 172276
vm.old_msync: 0
vm.msync_flush_flags: 3
vm.boot_pages: 48
vm.max_wired: 495315
vm.pageout_lock_miss: 0
vm.disable_swapspace_pageouts: 0
vm.defer_swapspace_pageouts: 0
vm.swap_idle_enabled: 0
vm.pageout_stats_interval: 5
vm.pageout_full_stats_interval: 20
vm.pageout_stats_max: 40381
vm.max_launder: 32
vm.phys_segs:
SEGMENT 0:
start: 0x1000
end: 0x99000
free list: 0xffffffff80c77e08
SEGMENT 1:
start: 0x1073000
end: 0xbea92000
free list: 0xffffffff80c77a60
SEGMENT 2:
start: 0x100000000
end: 0x1b3ff0000
free list: 0xffffffff80c77a60
vm.phys_free:
FREE LIST 0:
ORDER (SIZE) | NUMBER
| POOL 0 | POOL 1 | POOL 2
-- -- -- -- -- -- -- --
12 (016384K) | 000119 | 000000 | 000000
11 (008192K) | 000026 | 000001 | 000000
10 (004096K) | 000058 | 000001 | 000000
09 (002048K) | 000053 | 000001 | 000000
08 (001024K) | 000001 | 000000 | 000000
07 (000512K) | 000002 | 000000 | 000000
06 (000256K) | 000004 | 000000 | 000000
05 (000128K) | 000019 | 000001 | 000004
04 (000064K) | 000078 | 000001 | 000015
03 (000032K) | 000206 | 000004 | 000202
02 (000016K) | 000650 | 000012 | 000434
01 (000008K) | 001112 | 000016 | 001479
00 (000004K) | 000652 | 000005 | 003424
FREE LIST 1:
ORDER (SIZE) | NUMBER
| POOL 0 | POOL 1 | POOL 2
-- -- -- -- -- -- -- --
12 (016384K) | 000000 | 000000 | 000000
11 (008192K) | 000000 | 000000 | 000000
10 (004096K) | 000000 | 000000 | 000000
09 (002048K) | 000000 | 000000 | 000000
08 (001024K) | 000000 | 000000 | 000000
07 (000512K) | 000000 | 000000 | 000000
06 (000256K) | 000001 | 000000 | 000000
05 (000128K) | 000001 | 000000 | 000000
04 (000064K) | 000001 | 000000 | 000000
03 (000032K) | 000002 | 000000 | 000000
02 (000016K) | 000000 | 000000 | 000000
01 (000008K) | 000001 | 000000 | 000000
00 (000004K) | 000002 | 000000 | 000000
vm.reserv.reclaimed: 0
vm.reserv.partpopq:
LEVEL SIZE NUMBER
-00001: 432112K, 000734
vm.reserv.freed: 1850255
vm.reserv.broken: 436
vm.idlezero_enable: 0
vm.kvm_free: 546738008064
vm.kvm_size: 549755809792
vm.pmap.pmap_collect_active: 0
vm.pmap.pmap_collect_inactive: 0
vm.pmap.pv_entry_spare: 3523
vm.pmap.pv_entry_allocs: 490764507
vm.pmap.pv_entry_frees: 490746022
vm.pmap.pc_chunk_tryfail: 0
vm.pmap.pc_chunk_frees: 3217597
vm.pmap.pc_chunk_allocs: 3217728
vm.pmap.pc_chunk_count: 131
vm.pmap.pv_entry_count: 18485
vm.pmap.pdpe.demotions: 0
vm.pmap.pde.promotions: 183512
vm.pmap.pde.p_failures: 3208458
vm.pmap.pde.mappings: 0
vm.pmap.pde.demotions: 135426
vm.pmap.shpgperproc: 200
vm.pmap.pv_entry_max: 2746951
vm.pmap.pg_ps_enabled: 1
>
> >I have tested the two system drives independently (currently a zfs
> >mirror), so it is not likely to be an hdd issue.
>
> How did you test them and what were the results?
>
> Do you know what revision your
> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c is?
> (Or when/how did you last upgrade your source tree).
I did a make update just before the buildworld, the same day as the
original message post.
>
> --
> Peter Jeremy
>
> =A0application_pgp-signature_part
> < 1KViewDownload
In case it helps, here kldstat output:
Id Refs Address Size Name
1 32 0xffffffff80100000 d7b280 kernel
2 1 0xffffffff80e7c000 19e1f8 zfs.ko
3 2 0xffffffff8101b000 3860 opensolaris.ko
4 1 0xffffffff8101f000 f090 aio.ko
5 1 0xffffffff81030000 fe68 ahci.ko
6 1 0xffffffff81222000 39f7 linprocfs.ko
7 1 0xffffffff81226000 1cc4d linux.ko
8 1 0xffffffff81243000 216 green_saver.ko
9 1 0xffffffff81244000 86c coretemp.ko
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?10b0bdef-2bb1-44c8-9ffb-7d3167147a4f>