From owner-freebsd-stable@FreeBSD.ORG Mon Dec 15 20:48:09 2008 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 261F61065676 for ; Mon, 15 Dec 2008 20:48:09 +0000 (UTC) (envelope-from bsdlist@cogeco.ca) Received: from fep3.cogeco.net (smtp2.cogeco.ca [216.221.81.29]) by mx1.freebsd.org (Postfix) with ESMTP id EE29A8FC24 for ; Mon, 15 Dec 2008 20:48:08 +0000 (UTC) (envelope-from bsdlist@cogeco.ca) Received: from [192.168.1.126] (d150-251-98.home.cgocable.net [24.150.251.98]) by fep3.cogeco.net (Postfix) with ESMTP id 324231433; Mon, 15 Dec 2008 15:48:08 -0500 (EST) Message-ID: <4946C2FF.3080900@cogeco.ca> Date: Mon, 15 Dec 2008 15:50:07 -0500 From: Paul MacKenzie User-Agent: Thunderbird 3.0a1pre (Windows/2008022014) MIME-Version: 1.0 To: freebsd-stable@freebsd.org References: <2515BCEE3A2F4CBA8FFB9F1C052924AA@jmlaptop> <4934CB77.30906@transactionware.com> <494698E4.2070406@cogeco.ca> <200812151900.mBFJ0Jom084267@lava.sentex.ca> <4946B6EF.5080806@cogeco.ca> <200812152010.mBFKAlZf084580@lava.sentex.ca> <4946BDB1.8060208@cogeco.ca> <200812152035.mBFKZkj4084689@lava.sentex.ca> In-Reply-To: <200812152035.mBFKZkj4084689@lava.sentex.ca> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Subject: Re: 7.1-PRERELEASE: arcmsr write performance problem X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Dec 2008 20:48:09 -0000 > I would try the change to /etc/nsswitch.conf so that group and passwd > read > > group: files > passwd: files > > At that file size, it sounds like you only have about 200 entries ? I > doubt its the issue, but its worth a try. I know at around 9,000 > files anything to do with UID lookups (e.g. ls -l) takes forever. > > >> PID USERNAME VCSW IVCSW READ WRITE FAULT TOTAL PERCENT >> COMMAND >> 54 root 2 0 0 7 0 7 100.00% >> syncer > > Do you have any other special tuning other than polling ? Any in > /boot/loader.conf or /etc/sysctl.conf ? > > Does gstat show the disks busy ? > > ---Mike > Hi Mike, You were close. There are 109 entries. I too found the same problem on another server and I already changed this to files when I was trying to figure it all out. in boot/loader.conf I have: accf_http_load="YES". I tried turning this off but the same problems were seen so I moved on. In /etc/sysctl.conf I have: kern.maxvnodes=400000 kern.ipc.somaxconn=1024 vfs.ufs.dirhash_maxmem=10485760 I added these based on other posts of people's troubles or recommendations in other threads. For the gstat it does not really which was leading me elsewhere about a week: I stopped these at almost the same time and polling is now disabled with a -polling on both interfaces as part of the lagg0: dT: 1.110s w: 1.000s L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 0 0 0 0 0.0 0 0 0.0 0.0| acd0 0 2 2 173 4.4 0 0 0.0 0.8| da0 0 2 2 173 15.5 0 0 0.0 2.8| da0s1 0 0 0 0 0.0 0 0 0.0 0.0| da1 0 0 0 0 0.0 0 0 0.0 0.0| da0s1a 0 0 0 0 0.0 0 0 0.0 0.0| da0s1b 0 0 0 0 0.0 0 0 0.0 0.0| da0s1c 0 0 0 0 0.0 0 0 0.0 0.0| da0s1d 0 0 0 0 0.0 0 0 0.0 0.0| da0s1e 0 2 2 173 15.9 0 0 0.0 2.9| da0s1f 0 0 0 0 0.0 0 0 0.0 0.0| da1s1 0 0 0 0 0.0 0 0 0.0 0.0| da1s1c 0 0 0 0 0.0 0 0 0.0 0.0| da1s1d top -SI last pid: 56352; load averages: 14.64, 8.57, 6.38 up 0+10:59:33 15:45:32 287 processes: 22 running, 251 sleeping, 14 waiting CPU: 19.6% user, 0.0% nice, 60.2% system, 0.5% interrupt, 19.7% idle Mem: 705M Active, 3290M Inact, 492M Wired, 6888K Cache, 214M Buf, 11G Free Swap: 8192M Total, 8192M Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 56070 www 1 104 0 193M 61040K CPU3 3 1:32 37.89% httpd 1490 mysql 97 4 -5 406M 189M sbwait 7 0:31 37.35% mysqld 14 root 1 171 ki31 0K 16K RUN 4 507:42 30.86% idle: cpu4 17 root 1 171 ki31 0K 16K RUN 1 459:32 30.76% idle: cpu1 11 root 1 171 ki31 0K 16K RUN 7 545:35 30.47% idle: cpu7 16 root 1 171 ki31 0K 16K RUN 2 459:31 28.56% idle: cpu2 13 root 1 171 ki31 0K 16K RUN 5 526:39 24.27% idle: cpu5 15 root 1 171 ki31 0K 16K RUN 3 481:30 23.39% idle: cpu3 56331 samfilter 1 -4 0 14448K 4836K ufs 2 0:05 20.90% perl5.8.8 56201 www 1 -4 0 146M 25472K ufs 3 1:11 14.45% httpd 18 root 1 171 ki31 0K 16K RUN 0 454:31 13.77% idle: cpu0 56321 www 1 -4 0 147M 25728K ufs 2 0:03 11.38% httpd 56343 www 1 -4 0 144M 22772K CPU5 1 0:02 10.79% httpd 56327 www 1 20 0 145M 24408K lockf 7 0:03 10.60% httpd 56339 www 1 54 0 144M 22732K CPU7 1 0:02 10.35% httpd 56266 www 1 -4 0 153M 30560K ufs 2 0:14 10.16% httpd 56186 www 1 -4 0 156M 32720K RUN 4 0:59 9.86% httpd 56095 www 1 -4 0 146M 25700K RUN 1 0:58 9.77% httpd 56189 www 1 -4 0 147M 25620K RUN 5 0:47 9.67% httpd 56260 www 1 -4 0 146M 24748K ufs 2 0:13 9.57% httpd 56185 www 1 -4 0 154M 31068K ufs 2 0:50 9.47% httpd 56345 www 1 -4 0 144M 22732K ufs 3 0:02 9.47% httpd 56205 www 1 -4 0 153M 30840K ufs 2 0:24 9.28% httpd 56277 www 1 -4 0 153M 30228K ufs 2 0:10 9.28% httpd 56297 www 1 4 0 145M 24372K kqread 5 0:04 9.28% httpd 56292 www 1 -4 0 145M 24400K ufs 2 0:06 9.18% httpd 56325 www 1 -4 0 144M 23140K ufs 6 0:03 9.18% httpd 56301 www 1 -4 0 146M 25208K ufs 2 0:05 9.08% httpd 56326 www 1 4 0 144M 23320K sbwait 5 0:03 8.98% httpd 56099 www 1 -4 0 147M 26460K ufs 2 1:43 8.69% httpd 56287 www 1 -4 0 146M 24404K ufs 2 0:11 8.69% httpd 56294 www 1 -4 0 147M 25952K RUN 4 0:09 8.69% httpd 56298 www 1 -4 0 144M 23340K ufs 6 0:07 8.69% httpd 56322 www 1 -4 0 147M 25600K ufs 3 0:03 8.59% httpd 56175 www 1 -4 0 151M 29140K ufs 2 1:26 8.25% httpd 56169 www 1 -4 0 155M 32016K ufs 2 0:57 8.25% httpd 56261 www 1 -4 0 149M 27048K ufs 2 0:13 8.25% httpd 56323 www 1 -4 0 144M 23296K ufs 2 0:03 8.25% httpd 56115 www 1 -4 0 155M 32244K ufs 2 0:55 7.86% httpd 56168 www 1 -4 0 147M 26368K ufs 2 0:42 7.76% httpd 56202 www 1 -4 0 154M 31096K ufs 6 0:24 7.76% httpd 56338 root 1 48 0 8112K 2592K CPU4 4 0:02 7.76% top 56263 www 1 -4 0 150M 28044K ufs 3 0:10 7.67% httpd 56254 www 1 -4 0 145M 24476K ufs 2 0:08 7.67% httpd 56094 www 1 -4 0 155M 31984K ufs 2 0:48 7.57% httpd 56328 www 1 -4 0 144M 23120K CPU0 0 0:02 7.57% httpd Thanks, Paul