Date: Wed, 02 Nov 2005 14:58:36 +0000 From: Gavin Atkinson <gavin.atkinson@ury.york.ac.uk> To: freebsd-current@freebsd.org Cc: rwatson@freebsd.org Subject: Poor NFS server performance in 6.0 with SMP and mpsafenet=1 Message-ID: <1130943516.51544.34.camel@buffy.york.ac.uk>
next in thread | raw e-mail | index | archive | help
Hi, I'm seeing incredibly poor performance when serving files from an SMP FreeBSD 6.0RC1 server to a Solaris 10 client. I've done some experimenting and have discovered that either removing SMP from the kernel, or setting debug.mpsafenet=0 in loader.conf massively improves the speed. Switching preemption off seems to also help. A quick summary of my findings: switching SMP off and setting mpsafenet=0 gives more than 700% boost in performance. PREEMPTION is a pessimisation in the SMP case, mpsafenet=1 is a pessimisation in all cases. Given these are three of the major new features in 6.0, this doesn't seem ideal :) The following are average times, in seconds, to serve a 512meg file over NFS to the Solaris host. Each experiment was performed immediately after a reboot of the FreeBSD host, and after one untimed transfer (in order to warm the FreeBSD vfs cache). No SMP, mpsafenet=1 59.4 No SMP, mpsafenet=0 49.4 No SMP, mpsafenet=1, no PREEMPTION 53.1 No SMP, mpsafenet=0, no PREEMPTION 51.9 SMP, mpsafenet=1 351.7 SMP, mpsafenet=0 74.5 SMP, mpsafenet=1, no PREEMPTION 264.9 SMP, mpsafenet=0, no PREEMPTION 53.7 Obviously this is not ideal, but at least I have a usable workaround. I'm just worried that the default SMP configuration performs so poorly (although I appreciate that GENERIC does not include SMP, anyone with a multiprocessor machine presumably would want to use it...) The FreeBSD machine is a twin 866MHz P3, running 6.-RC1 with ACPI enabled. The network cards are 64bit/66MHz em(4) cards using checksum offloading etc. Apart from the changes mentioned above, the kernel is GENERIC. I'll get a dmesg online somewhere. I'll also recreate the experiment, serving to a FreeBSD machine as opposed to Solaris, just to compare. I'm happy to do any more investigating to get to the bottom of this, although the machines in question will be going live at the end of the month. Gavin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1130943516.51544.34.camel>