Date: Thu, 13 Nov 2014 10:46:46 +0100 From: Mark Schouten <mark@tuxis.nl> To: freebsd-hackers@freebsd.org Subject: Slow nfsd write performance, tweaks needed Message-ID: <2826701214-10966@kerio.tuxis.nl>
next in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] Hi, I am in the process of switching from a ZFS On Linux-based NFS-server to a FreeBSD-based NFS-server. The FreeBSD implementation of ZFS is way superiour over ZoL, and the box serves as storage for a virtualizationplatform, so stability is welcome. :) The box is stable, but performs terribly. Surely, I'm doing something wrong, but I would like some tips and tricks to speed things up. Here's my setup: CPU: Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz (HyperThreading is enabled) RAM: 64GB NIC: 2x igb in lagg0 (loadbalancing) Disks: export1 1.81T 914G 942G 49% 1.00x ONLINE - mirror 928G 457G 471G - da0 - - - - da1 - - - - mirror 928G 457G 471G - da2 - - - - da3 - - - - mirror 9.94G 173M 9.77G - da4p1 - - - - da5p1 - - - - cache - - - - - - da4p2 223G 223G 8M - da5p2 223G 223G 8M - da0-3 are 1TB WDs da4-5 are 240GB Samsung SSD 840s Here's (related) info from rc.conf. nfs_server_enable="YES" nfs_server_flags="-u -t -n 128" rpcbind_enable="YES" mountd_enable="YES" rpc_lockd_enable="YES" rpc_statd_enable="YES" I have compression enabled on all the ZFS-filesystems, and jumboframes are enabled on the nics. As soon as one of the (Linux) clients start to do some IO, NFS responsetimes go up bigtime (yesterday up to 13 seconds), while the hardware is pretty much idle, I must be doing something very wrong. I'm mostly a Linux-guy, so any hit with a FreeBSD cluebat is appreciated. Regards, -- Kerio Operator in de Cloud? https://www.kerioindecloud.nl/ Mark Schouten | Tuxis Internet Engineering KvK: 61527076 | http://www.tuxis.nl/ T: 0318 200208 | info@tuxis.nl [-- Attachment #2 --] 0 *H 01 0 + 0 *H 00,]~!dn=0 *H 01 0 UGB10UGreater Manchester10USalford10U COMODO CA Limited1907U0COMODO Client Authentication and Secure Email CA0 140701000000Z 150701235959Z0 1 0 *H mark@tuxis.nl0"0 *H 0 ,e1`4jEtuBeE")>ԁY 7ՀI&]KͨqkRnZÚC$jw4[ߞ? f)Sm6,M%#R 2U\?blOZǣۍq }(/nKNoovtڪmrw4Ԣ -H2R 0!0PnB_n?uW+RoqCڇvx|1B/ Lan / )` 00 U #0zN t[xcd'/[y{0 U ~tDeHO3 1M]0U 0 U 0 0 U %0+ +10 `HB 0FU ?0=0; +10+0)+ https://secure.comodo.net/CPS0WU P0N0LJHFhttp://crl.comodoca.com/COMODOClientAuthenticationandSecureEmailCA.crl0+|0z0R+0Fhttp://crt.comodoca.com/COMODOClientAuthenticationandSecureEmailCA.crt0$+0http://ocsp.comodoca.com0U 0 mark@tuxis.nl0 *H |0D7 -Ѐ1Ν {(pSiW;@Fλ2I)l`ϥ#<v.|Y;oa95z.dJ5 _[/>1I/ QSby@o@=T<cW4$?^KXkO=Uw NCdFC ӡLWk?@\˯K OdAWL{yC0zb%|{&1ӷ >g ʋlg !&600mOj3""2zq0 *H 01 0 UUS1 0 UUT10USalt Lake City1 0 U The USERTRUST Network1!0 U http://www.usertrust.com1604U-UTN-USERFirst-Client Authentication and Email0 110428000000Z 200530104838Z01 0 UGB10UGreater Manchester10USalford10U COMODO CA Limited1907U0COMODO Client Authentication and Secure Email CA0"0 *H 0 [KW^/@ȣSX_fe2N2}UxLUB'qi2@'Vbqi c^`ʢAj HmeC*.+c8w߱ڂ2jgo \5Tq 7 PSl Y1 LR@[HhJ$ :q_㬿;%qh=XF<hmz!W42~JRrd&N`ohQcB}"cөΞD\[5 K0G0 U #0g}ĝ&p KPH|=n}0 U zN t[xcd'/[y{0U 0U 0 0U 00U 0XU Q0O0MKIGhttp://crl.usertrust.com/UTN-USERFirst-ClientAuthenticationandEmail.crl0t+h0f0=+01http://crt.usertrust.com/UTNAddTrustClient_CA.crt0%+0http://ocsp.usertrust.com0 *H ־xWUm3DRB JAIZҭsn>&|L0(B<%> u=9fѡMo(l tZڱuz/y VtCr`9 G:eH<=%`I?C 3_н`j;:< I3B)93i.EMiڀ =]|Gm]W0KID~y83:]&XaU!ՙC@B0Ұun004=+'44pT0 *H 0o1 0 USE10U AddTrust AB1&0$U AddTrust External TTP Network1"0 UAddTrust External CA Root0 050607080910Z 200530104838Z01 0 UUS1 0 UUT10USalt Lake City1 0 U The USERTRUST Network1!0 U http://www.usertrust.com1604U-UTN-USERFirst-Client Authentication and Email0"0 *H 0 9}A;bF7`u9eJGHjM5BI/ |1Nd.)բdąQ5yNh{ zɤ2 O0 nFxoY^/m/묡j.g5y iF͠ v:z'[=s "HaLi.1 ,CZ qYں gT: we tbh ~GeMW(t40b0, 00 U #0z4&&T$T0 U g}ĝ&p KPH|=n}0U 0U 00U 00U 0DU =0;09753http://crl.usertrust.com/AddTrustExternalCARoot.crl05+)0'0%+0http://ocsp.usertrust.com0 *H c(1{b#1sSQ L֟/g~x3t&dp bP#4Vp4nx7 _j̉_|>Q5|`k:+߳}[| P-sxt1^˚ƹ7urDg% R%G<N 6wH\-?`q`q 6 lKuI;ٟMx&-n_ 10001 0 UGB10UGreater Manchester10USalford10U COMODO CA Limited1907U0COMODO Client Authentication and Secure Email CA,]~!dn=0 + 0 *H 1 *H 0 *H 1 141113094646Z0# *H 1>W K\g 0y *H 1l0j0 `He*0 `He0 `He0 *H 0*H 0 *H @0+0 *H (0 *H G)%Q &QVOwn%/zkZZ5tHP60Q4r?9Izf i8^Z }dKԉ B*J>%{Uw#x8!//}?ۄҗwUe-EW_ (缩y'܁..&xx"Iʎec 'Pc y!M72"SݾCW
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?2826701214-10966>