From owner-freebsd-geom@FreeBSD.ORG Sun Apr 3 13:07:39 2005 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 64DAF16A4CE for ; Sun, 3 Apr 2005 13:07:39 +0000 (GMT) Received: from shellma.zin.lublin.pl (shellma.zin.lublin.pl [212.182.126.68]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4B96643D31 for ; Sun, 3 Apr 2005 13:07:38 +0000 (GMT) (envelope-from pawmal-posting@freebsd.lublin.pl) Received: by shellma.zin.lublin.pl (Postfix, from userid 1018) id 647F8347116; Sun, 3 Apr 2005 15:06:02 +0200 (CEST) Date: Sun, 3 Apr 2005 15:06:02 +0200 From: Pawel Malachowski To: freebsd-geom@freebsd.org Message-ID: <20050403130602.GA86317@shellma.zin.lublin.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: 8bit User-Agent: Mutt/1.4.2i Subject: Processess running on gconcat lock in kserel or ufs state X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Apr 2005 13:07:39 -0000 Hello, Processess runing on gconcat drive frequently lock on my machine in kserele/ufs state. Could you try giving some hints/patches? Details below. Machine: FreeBSD x.pl 5.4-PRERELEASE FreeBSD 5.4-PRERELEASE #0: Mon Mar 28 21:35:57 CEST 2005 root@x.pl:/usr/obj/usr/src/sys/x i386 (previously 5.3-RELEASE, same problem). gconcat drive build on top of 6 GBDE drives. GBDE drives are on physical ATA drives (usually adXs1e slice), reported by S.M.A.R.T. as OK. This gconcat volume receives many concurrent read/write sessions (about 30-50 all the time, constant network traffic, about 10-15Mbit/s od reading and 2-4Mbit/s of writing activity). /dev/concat/c1 624G 143G 474G 23% 1445 84503129 0% /x /dev/concat/c1 on /x (ufs, local, soft-updates) tunefs -m was used on /x filesystem to reduce minfree to 1%. After day or two processes trying to read /x are locking. First (most active process) locks in kserel state, every next process (for example shell trying `cd /x') locks in ufs state. There are ongoing writing processes and it seems these are not locked (upload is proceeding). They will lock after upload is finished and read operation is performed. gstat -I 5s output (look at concat/c1 % busy): dT: 5.019 flag_I 5000000us sizeof 240 i -1 L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 0 0 0 0 0.0 0 0 0.0 0.0| fd0 0 0 0 0 0.0 0 0 0.0 0.0| ad0s1e.bde 0 0 0 0 0.0 0 0 0.0 0.0| ad0 0 11 11 79 16.3 0 0 0.0 1.7| ad1 0 7 7 53 12.9 0 0 0.0 1.0| ad4 0 14 14 106 27.5 0 0 0.0 3.2| ad5 0 30 20 108 20.3 11 79 17.1 6.4| ad6 0 4 4 26 12.4 0 0 0.0 0.5| ad7 0 0 0 0 0.0 0 0 0.0 0.0| ad0s1 0 1 1 77 29.5 0 0 0.0 1.8| ad1s1f.bde 0 0 0 51 26.3 0 0 0.0 1.0| ad4s1d.bde 0 11 11 79 16.6 0 0 0.0 1.7| ad1s1 0 1 1 102 41.0 0 0 0.0 3.3| ad5s1d.bde 0 1 1 102 38.2 1 77 58.2 6.5| ad6s1d.bde 0 7 7 53 13.3 0 0 0.0 1.0| ad4s1 0 0 0 26 25.7 0 0 0.0 0.5| ad7s1d.bde 0 14 14 106 27.8 0 0 0.0 3.2| ad5s1 0 30 20 108 20.7 11 79 17.2 6.4| ad6s1 6 3 3 357 34.9 1 77 58.5 101.5| concat/c1 0 4 4 26 12.8 0 0 0.0 0.5| ad7s1 0 0 0 0 0.0 0 0 0.0 0.0| ad0s1a 0 0 0 0 0.0 0 0 0.0 0.0| ad0s1b 0 0 0 0 0.0 0 0 0.0 0.0| ad0s1c 0 0 0 0 0.0 0 0 0.0 0.0| ad0s1d 0 0 0 0 0.0 0 0 0.0 0.0| ad0s1e 0 0 0 0 0.0 0 0 0.0 0.0| ad1s1c 0 0 0 0 0.0 0 0 0.0 0.0| ad1s1d 0 0 0 0 0.0 0 0 0.0 0.0| ad1s1e 0 11 11 79 16.8 0 0 0.0 1.8| ad1s1f 0 0 0 0 0.0 0 0 0.0 0.0| ad4s1c 0 7 7 53 13.4 0 0 0.0 1.0| ad4s1d 0 0 0 0 0.0 0 0 0.0 0.0| ad5s1c 0 14 14 106 28.0 0 0 0.0 3.2| ad5s1d 0 0 0 0 0.0 0 0 0.0 0.0| ad6s1c 0 30 20 108 20.8 11 79 17.4 6.5| ad6s1d 0 0 0 0 0.0 0 0 0.0 0.0| ad7s1c 0 4 4 26 12.9 0 0 0.0 0.5| ad7s1d Physical access to this machine is very problematic. Sorry, I'm going to reboot this host, so no more details for now. ;) Machine will reboot, but /x can't be unmounted (probably tries and gives up), so both / and /x are dirty and are fscked during next boot. -- Paweł Małachowski