From owner-freebsd-bugs Fri Jun 23 17:20: 8 2000 Delivered-To: freebsd-bugs@freebsd.org Received: from freefall.freebsd.org (freefall.FreeBSD.ORG [204.216.27.21]) by hub.freebsd.org (Postfix) with ESMTP id 960AE37BAD6 for ; Fri, 23 Jun 2000 17:20:01 -0700 (PDT) (envelope-from gnats@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.9.3/8.9.2) id RAA92360; Fri, 23 Jun 2000 17:20:01 -0700 (PDT) (envelope-from gnats@FreeBSD.org) Received: by hub.freebsd.org (Postfix, from userid 32767) id 570B937BAB1; Fri, 23 Jun 2000 17:19:43 -0700 (PDT) Message-Id: <20000624001943.570B937BAB1@hub.freebsd.org> Date: Fri, 23 Jun 2000 17:19:43 -0700 (PDT) From: wessels@ircache.net To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-1.0 Subject: kern/19479: processes stuck in 'ffsvgt' and 'FFS no' states Sender: owner-freebsd-bugs@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org >Number: 19479 >Category: kern >Synopsis: processes stuck in 'ffsvgt' and 'FFS no' states >Confidential: no >Severity: serious >Priority: low >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Fri Jun 23 17:20:01 PDT 2000 >Closed-Date: >Last-Modified: >Originator: Duane Wessels >Release: RELENG_3 from 20000620 >Organization: IRCache >Environment: FreeBSD mr-garrison.measurement-factory.com 3.5-STABLE FreeBSD 3.5-STABLE #4: Tue Jun 20 14:15:04 MDT 2000 root@mr-garrison.measurement-factory.com:/usr/src/sys/compile/SQUID i386 >Description: I'm benchmarking the Squid proxy cache, which generates a lot of disk I/O. After an hour or two, all of the Squid processes get stuck in the 'ffsvgt' state, and one of them is stuck in 'FFS no': PID USERNAME PRI NICE SIZE RES STATE TIME WCPU CPU COMMAND 368 wessels 98 0 102M 99036K RUN 106:01 98.14% 98.14% squid 375 wessels -18 0 1272K 812K ffsvgt 2:23 0.00% 0.00% diskd 373 wessels -18 0 1272K 812K ffsvgt 2:22 0.00% 0.00% diskd 374 wessels -20 0 1272K 812K FFS no 2:22 0.00% 0.00% diskd 372 wessels -18 0 1272K 812K ffsvgt 2:20 0.00% 0.00% diskd 370 wessels -18 0 1272K 812K ffsvgt 2:19 0.00% 0.00% diskd 371 wessels -18 0 1272K 812K ffsvgt 2:19 0.00% 0.00% diskd Although the system continues to run, it seems like any process that tries to access a filesystem also becomes stuck in the ffsvgt state. I can't start new commands, log in, etc. Filesystems are mounted normally. No special options: /dev/da0c /cache1 ufs rw 0 3 /dev/da1c /cache2 ufs rw 0 3 /dev/da2c /cache3 ufs rw 0 3 /dev/da3c /cache4 ufs rw 0 3 /dev/da4c /cache5 ufs rw 0 3 /dev/da5c /cache6 ufs rw 0 3 I gather that 'FFS no' is related to filesystem vnodes. I wonder if I'm running out of vnodes, or if they're not getting freed? Here's systat after the benchmark has been running for 20 minutes: 2 users Load 1.23 1.16 0.90 Fri Jun 23 18:10 Mem:KB REAL VIRTUAL VN PAGER SWAP PAGER Tot Share Tot Share Free in out in out Act 68748 1064 74260 3860 21700 count All 513584 1440 3026816 4408 pages 53 cow Interrupts Proc:r p d s w Csw Trp Sys Int Sof Flt 75 zfod 2377 total 1 7 4 527 297 6247 2377 268 152 50408 wire 100 clk0 irq0 71716 act 128 rtc0 irq8 45.3%Sys 6.3%Intr 47.1%User 0.0%Nice 1.2%Idl 370040 inact 1864 pci irq10 | | | | | | | | | | 20564 cache 266 pci irq11 =======================+++>>>>>>>>>>>>>>>>>>>>>> 1136 free fdc0 irq6 daefr 19 wdc0 irq14 Namei Name-cache Dir-cache 106 prcfr Calls hits % hits % react 1920 1529 80 6 0 pdwake 525 pdpgs Discs da0 da1 da2 da3 da4 da5 wd0 intrn KB/t 7.18 7.38 8.13 8.26 10.58 8.03 8.00 8342 buf tps 55 57 50 40 18 47 1 202 dirtybuf MB/s 0.39 0.41 0.39 0.32 0.18 0.37 0.01 32702 desiredvnodes % busy 25 24 25 16 12 21 0 55047 numvnodes 24 freevnodes >How-To-Repeat: Install the latest squid-2.4 code and pummel it with a Web Polygraph workload. >Fix: >Release-Note: >Audit-Trail: >Unformatted: To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-bugs" in the body of the message