From owner-freebsd-fs@FreeBSD.ORG Sun Jul 6 12:07:34 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7538737B401 for ; Sun, 6 Jul 2003 12:07:34 -0700 (PDT) Received: from web80010.mail.yahoo.com (web80010.mail.yahoo.com [66.163.168.140]) by mx1.FreeBSD.org (Postfix) with SMTP id DEA4843FFB for ; Sun, 6 Jul 2003 12:07:33 -0700 (PDT) (envelope-from soundsampler@sbcglobal.net) Message-ID: <20030706190733.67131.qmail@web80010.mail.yahoo.com> Received: from [66.74.132.113] by web80010.mail.yahoo.com via HTTP; Sun, 06 Jul 2003 12:07:33 PDT Date: Sun, 6 Jul 2003 12:07:33 -0700 (PDT) From: Trent George To: freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Content-Filtered-By: Mailman/MimeDel 2.1.1 Subject: Software Disk Cache Limit Issues X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jul 2003 19:07:34 -0000 Hi, I have a couple of questions. I have been experimenting with freebsd to provide similar functionality to netware's large write-back disk caching. Why ? I have an application that does frequent, large file saves 40mb+ every second or more. I have two new systems with the great i865 chipsets with the fast new gigabit on board chips (fast northbridge connections). My ttcp performace on these machines is 120mb/s sustained. But any disk releated writes are slowed down to disk speed. (vinum raid0 with 2 drives ~ 100mb/s - 50mb/s). these machines have 2gb of ram. using 5.1-beta, ide in udma Goal: Allow huge amounts of file data to be cached and allow burst writes (instant return) from tcp or local, not be blocked by storage device speed (if ram allows) The experiment: I wrote a little program that writes to a large file and displays wall time progress and vfs.runningbufspace after each 1mb block to see where and how long write bottlenecks occur in applications. test:sysctl vfs.hirunningspace=200000000 (200mb) results:system would limit to 16mb and app would block on wswbuf test:recompile with options NSWBUF_MIN=3200, (and tweak % of ram for nbuf) results:application would return "instantly" after 180mb of writing :-) downside:a second run before vfs.runningbufspace emptyed would seem to block at random places, read delays are large, usually until all runningbufspace emptied. it seemed there was blocks at "getbuf", and it would not be at hirunningspace and somtimes wait for runningbufspace was zero. observations: it seems "writeback caching of disk data" goes to nbuf then nswbuf then disk. comments: it seems nswbuf is using 64k blocks v's nbuf uses 16k blocks the hardcoded limit high limit of 256 nswbufs limits hirunningspace it seems space allocated for nswbuf is additional to nbuf, but no statistics show it it seems a write needs both nbuf space and nswbuf space to "cache it"(can't it use the memory once instead of twice?) questions: could not the async write back wait until no reads are in queue ? should not the amount of allocated space for nswbuf to "viewable" statisticlly like vfs.maxbufspace shows it seems the vfs.numdirtybuffer has no meaning in this quest. is this all too hard (after 10 experimental kernel compiles and tweaks and 5 hours later) would this not be of interest for the new machines with growing ram and tcp speed. ideal would be 1/ to use a tunable % of ram to use for nbuf 2/ allow nbuf to fill up with write-back blocks and trickly out when device has no reads queues (or idle) with vfs.hidirtybuffers 3/ use the smaller nswbufs for vfs.hidirtybuffers and vfs.lodirtybuffers unfortunately my skill set is limited and I don't understand vm well enough, it seems /usr/src/sys/kern/vfs_bio.c has all the relevent code. any tips would be appreciated, i don't know if this message should be more directed to scsi or hackers, there seems no vm discussion. Sorry for the long post and thanks for your time. Trent George ps: the test program read somthing like the following pseudo code long vt[200],vs[200]; long len,vallen,val; struct tm tm; int i,f; char *buf; vallen=4; len=1<<20; buf=malloc(len); f=open(test.dat,O_RDWR|O_CREAT) gettimeofday(&tm,0) for(i=0;i<180;i++) { oldtm=tm.tm_usec; write(f,buf,len); gettimeofday(&tm,0); sysctlbyname("vfs.runningbufspace",&val,&vallen,0,0); vt[i]=tm.tm_usec-oldtm; vs[i]=val; } close(f); for(i=0;i<180;i++) printf("%2d:%4d:%3d ", i, vt[i]/1000, vs[i]/1024/1024); From owner-freebsd-fs@FreeBSD.ORG Sun Jul 6 13:05:59 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 092B437B401 for ; Sun, 6 Jul 2003 13:05:59 -0700 (PDT) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8C0164402D for ; Sun, 6 Jul 2003 13:05:58 -0700 (PDT) (envelope-from bright@elvis.mu.org) Received: by elvis.mu.org (Postfix, from userid 1192) id 678F72ED436; Sun, 6 Jul 2003 13:05:58 -0700 (PDT) Date: Sun, 6 Jul 2003 13:05:58 -0700 From: Alfred Perlstein To: Trent George Message-ID: <20030706200558.GG72093@elvis.mu.org> References: <20030706190733.67131.qmail@web80010.mail.yahoo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030706190733.67131.qmail@web80010.mail.yahoo.com> User-Agent: Mutt/1.4.1i cc: freebsd-fs@freebsd.org Subject: Re: Software Disk Cache Limit Issues X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jul 2003 20:05:59 -0000 * Trent George [030706 12:07] wrote: > Hi, > > I have a couple of questions. I have been experimenting with freebsd to provide similar functionality to netware's large write-back disk caching. This is cool, thanks for the feedback. Have you tried raising the sysctl tunable for 'hirunningspace'? That may alleviate some of the issue. From owner-freebsd-fs@FreeBSD.ORG Mon Jul 7 23:34:16 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E69C637B401 for ; Mon, 7 Jul 2003 23:34:16 -0700 (PDT) Received: from web80006.mail.yahoo.com (web80006.mail.yahoo.com [66.163.168.136]) by mx1.FreeBSD.org (Postfix) with SMTP id 57DB243FBF for ; Mon, 7 Jul 2003 23:34:16 -0700 (PDT) (envelope-from soundsampler@sbcglobal.net) Message-ID: <20030708063416.53743.qmail@web80006.mail.yahoo.com> Received: from [66.74.132.113] by web80006.mail.yahoo.com via HTTP; Mon, 07 Jul 2003 23:34:16 PDT Date: Mon, 7 Jul 2003 23:34:16 -0700 (PDT) From: Trent George To: freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Content-Filtered-By: Mailman/MimeDel 2.1.1 Subject: Re: Software Disk Cache Limit Issues X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2003 06:34:17 -0000 Hi, Since my last post I found a couple more interesting observations 1/ nswbuf_min has no effect on msdosfs (fat32) mounted filesystems like it did on ffs this led me to beleive that it may be a metadata issue so: 2/ I experimented with newfs to see if blocksize, fragment size or ufs1, ufs2 effected the ratio of nswbuf's to usable hirunningspace. the answer is no newfs settings effected the seeming fixed ratio of 65k of usable hirunningspace per nswbuf allocated on ffs filesystems In summary: a/ the default kernel has a max allocation of 256 nswbuf (in vfs_bio.c) b/ this can be overridden with options NSWBUF_MIN=xxxx in custom kernel c/ it seems the effect of a large sysctl vfs.hirunningspace gets capped on ffs filesystems by the size of nswbuf. and this always seems fixed at 64k per nswbuf d/ fat32 filesystems seemed immuned from this "capping" effect it seems the more i try to understand the relationship, the more mysterious it becomes. it also seems that fat32 performance seems erratic and less re-producable with ffs Trent From owner-freebsd-fs@FreeBSD.ORG Tue Jul 8 00:57:56 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 30B6D37B401 for ; Tue, 8 Jul 2003 00:57:56 -0700 (PDT) Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2BB2E43F75 for ; Tue, 8 Jul 2003 00:57:55 -0700 (PDT) (envelope-from bde@zeta.org.au) Received: from katana.zip.com.au (katana.zip.com.au [61.8.7.246]) by mailman.zeta.org.au (8.9.3p2/8.8.7) with ESMTP id RAA10584; Tue, 8 Jul 2003 17:57:44 +1000 Date: Tue, 8 Jul 2003 17:57:43 +1000 (EST) From: Bruce Evans X-X-Sender: bde@gamplex.bde.org To: Trent George In-Reply-To: <20030708063416.53743.qmail@web80006.mail.yahoo.com> Message-ID: <20030708173454.N10674@gamplex.bde.org> References: <20030708063416.53743.qmail@web80006.mail.yahoo.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-fs@freebsd.org Subject: Re: Software Disk Cache Limit Issues X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2003 07:57:56 -0000 On Mon, 7 Jul 2003, Trent George wrote: > Since my last post I found a couple more interesting observations > > 1/ nswbuf_min has no effect on msdosfs (fat32) mounted filesystems like it did on ffs > > this led me to beleive that it may be a metadata issue so: This is because the implementation of msdosfs is too primitive and/or old to support write (or read) clustering. Try turning off write clustering for ffs (mount -u -onoclusterw... ...). The default ffs block size is now a relatively large fraction of the cluster size (16K / 64K or 16K / 128K), so perhaps clustering is not doing anything useful for ffs anyway. It would be much more useful for msdosfs with a block size of 512. I think read clustering is less than useful with normal block sizes for most current drives, since caching in the drives is effective, but write clustering is still useful if the drives' write cache is turned off. > 2/ I experimented with newfs to see if blocksize, fragment size or ufs1, ufs2 effected the > ratio of nswbuf's to usable hirunningspace. > > the answer is no newfs settings effected the seeming fixed ratio of 65k of usable > hirunningspace per nswbuf allocated on ffs filesystems I think you mean 64K. 64K is the cluster size. It is the drive[r]'s si_iosize_max, which defaults to DLTPHYS = 64K but may be larger. It should be 128K for ATA drives. Other values are unusual. > In summary: > a/ the default kernel has a max allocation of 256 nswbuf (in vfs_bio.c) > b/ this can be overridden with options NSWBUF_MIN=xxxx in custom kernel The quick fix seems to be to use this, and fix the setting of vfs.hirunningspace and any associated variables to be consistent with it (either using sysctl or in the sources). Changing nswbuf and nbuf to track changes to vfs.hirunningspace, etc. is not so easy. Bruce From owner-freebsd-fs@FreeBSD.ORG Wed Jul 9 01:21:31 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 02A9737B401 for ; Wed, 9 Jul 2003 01:21:31 -0700 (PDT) Received: from lurza.secnetix.de (lurza.secnetix.de [212.66.1.130]) by mx1.FreeBSD.org (Postfix) with ESMTP id EADB043F75 for ; Wed, 9 Jul 2003 01:21:29 -0700 (PDT) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (jylqnu@localhost [127.0.0.1]) by lurza.secnetix.de (8.12.8p1/8.12.8) with ESMTP id h698LRB5051632 for ; Wed, 9 Jul 2003 10:21:28 +0200 (CEST) (envelope-from oliver.fromme@secnetix.de) Received: (from olli@localhost) by lurza.secnetix.de (8.12.8p1/8.12.8/Submit) id h698LR5C051631; Wed, 9 Jul 2003 10:21:27 +0200 (CEST) Date: Wed, 9 Jul 2003 10:21:27 +0200 (CEST) Message-Id: <200307090821.h698LR5C051631@lurza.secnetix.de> From: Oliver Fromme To: freebsd-fs@FreeBSD.ORG X-Newsgroups: list.freebsd-fs User-Agent: tin/1.5.4-20000523 ("1959") (UNIX) (FreeBSD/4.8-RELEASE (i386)) MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Subject: NFS problem FreeBSD vs NetApp Filer using tomcat / java X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2003 08:21:31 -0000 Hi, Is this the right list for NFS problems? If it's not, then where would this be most appropriate? Thanks! We're running java + tomcat on a Compaq DL320 server. The software runs chrooted on an NFS volume that comes from a NetApp Filer, i.e. _everything_ comes via NFS (binaries, libs, data). Every few hours, the java process dies, and we get the following in the logs: Jul 9 08:23:38 tbox /kernel: nfs_getpages: error 4 Jul 9 08:23:38 tbox /kernel: vm_fault: pager read error, pid 60443 (java) Jul 9 08:23:38 tbox /kernel: pid 60443 (java), uid 0: exited on signal 6 (errno 4 is "interrupted system call", sig 6 is SIGABRT.) It doesn't seem to be a load problem, as the machine does not have a particularly high load (neither CPU nor network). We're running 4.8-Release, the mounts are NFSv3/UDP. As far as I can tell, there are no network problems. The server has intel fxp FastEthernet cards, the Filer even has Gigabit interfaces. They're connected with Cisco switches, ping RTTs are < 0.2ms. There's one special thing that we use a VLAN trunk to the server, so the NFS mounts are on virtual vlan interfaces, but I think that shouldn't be causing the above problem. I suspect it's a bug in FreeBSD's NFS implementation which occurs under certain conditions against a NetApp filer. But I'm not an NFS guru. Does anybody have an idea what to do? Or how to debug the problem further? If this cannot be resolved, the machine will be converted to Debian Linux. I'd prefer to avoid that. Thanks in advance! Regards Oliver PS: Yes, I did search the archives, but no luck. -- Oliver Fromme, secnetix GmbH & Co KG, Oettingenstr. 2, 80538 München Any opinions expressed in this message may be personal to the author and may not necessarily reflect the opinions of secnetix in any way. "All that we see or seem is just a dream within a dream" (E. A. Poe) From owner-freebsd-fs@FreeBSD.ORG Wed Jul 9 02:13:51 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DD75137B401 for ; Wed, 9 Jul 2003 02:13:51 -0700 (PDT) Received: from lurza.secnetix.de (lurza.secnetix.de [212.66.1.130]) by mx1.FreeBSD.org (Postfix) with ESMTP id CF34443FA3 for ; Wed, 9 Jul 2003 02:13:50 -0700 (PDT) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (lclejq@localhost [127.0.0.1]) by lurza.secnetix.de (8.12.8p1/8.12.8) with ESMTP id h699DnB5053576 for ; Wed, 9 Jul 2003 11:13:49 +0200 (CEST) (envelope-from oliver.fromme@secnetix.de) Received: (from olli@localhost) by lurza.secnetix.de (8.12.8p1/8.12.8/Submit) id h699DnK3053575; Wed, 9 Jul 2003 11:13:49 +0200 (CEST) Date: Wed, 9 Jul 2003 11:13:49 +0200 (CEST) Message-Id: <200307090913.h699DnK3053575@lurza.secnetix.de> From: Oliver Fromme To: freebsd-fs@FreeBSD.ORG In-Reply-To: <200307090821.h698LR5C051631@lurza.secnetix.de> X-Newsgroups: list.freebsd-fs User-Agent: tin/1.5.4-20000523 ("1959") (UNIX) (FreeBSD/4.8-RELEASE (i386)) MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Subject: Re: NFS problem FreeBSD vs NetApp Filer using tomcat / java X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2003 09:13:52 -0000 Small addendum: Someone asked me off-list whether I use the "interruptible" flag for the NFS mounts, and suggested switching it off. Well, I always set the interruptible flag, so that processes don't hang in case of NFS problems (so I can kill or Ctrl-C them). Otherwise they will hang forever until I reboot. It is my understanding that the interruptible flag has only an effect when a signal is delivered to a process which is blocked on NFS I/O. Or am I wrong? Regards Oliver -- Oliver Fromme, secnetix GmbH & Co KG, Oettingenstr. 2, 80538 München Any opinions expressed in this message may be personal to the author and may not necessarily reflect the opinions of secnetix in any way. From owner-freebsd-fs@FreeBSD.ORG Wed Jul 9 02:38:44 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F24F937B401 for ; Wed, 9 Jul 2003 02:38:43 -0700 (PDT) Received: from frontend2.aha.ru (bird.zenon.net [213.189.198.215]) by mx1.FreeBSD.org (Postfix) with ESMTP id 54E3043FB1 for ; Wed, 9 Jul 2003 02:38:42 -0700 (PDT) (envelope-from uitm@blackflag.ru) Received: from [195.2.90.70] (HELO slt.oz) by frontend2.aha.ru (CommuniGate Pro SMTP 4.0.6) with ESMTP id 197864442; Wed, 09 Jul 2003 13:38:39 +0400 Received: (from uitm@localhost) by slt.oz (8.8.8/8.8.8) id NAA01103; Wed, 9 Jul 2003 13:40:46 +0400 (MSD) From: Andrey Alekseyev Message-Id: <200307090940.NAA01103@slt.oz> In-Reply-To: <200307090913.h699DnK3053575@lurza.secnetix.de> from Oliver Fromme at "Jul 9, 3 11:13:49 am" To: olli@secnetix.de (Oliver Fromme) Date: Wed, 9 Jul 2003 13:40:40 +0400 (MSD) X-Mailer: ELM [version 2.4ME+ PL31 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit cc: freebsd-fs@FreeBSD.ORG Subject: Re: NFS problem FreeBSD vs NetApp Filer using tomcat / java X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2003 09:38:44 -0000 Oliver, > It is my understanding that the interruptible flag has only > an effect when a signal is delivered to a process which is > blocked on NFS I/O. Or am I wrong? Do you use "i" or "s" option or both? With "c" you may try to turn off the dynamic retransmit timeout estimator and set timeout explicitly with "t". We faced a problem similar to yours when using "ci" for UDP mounts from FreeBSD client and Intel Solaris 8 server. From owner-freebsd-fs@FreeBSD.ORG Wed Jul 9 02:42:47 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2947737B401 for ; Wed, 9 Jul 2003 02:42:47 -0700 (PDT) Received: from frontend3.aha.ru (elk.zenon.net [213.189.198.216]) by mx1.FreeBSD.org (Postfix) with ESMTP id ECFAB43FBF for ; Wed, 9 Jul 2003 02:42:45 -0700 (PDT) (envelope-from uitm@blackflag.ru) Received: from [195.2.90.70] (HELO slt.oz) by frontend3.aha.ru (CommuniGate Pro SMTP 4.0.6) with ESMTP id 13484699; Wed, 09 Jul 2003 13:42:43 +0400 Received: (from uitm@localhost) by slt.oz (8.8.8/8.8.8) id NAA01131; Wed, 9 Jul 2003 13:44:50 +0400 (MSD) From: Andrey Alekseyev Message-Id: <200307090944.NAA01131@slt.oz> In-Reply-To: <200307090940.NAA01103@slt.oz> from Andrey Alekseyev at "Jul 9, 3 01:40:40 pm" To: uitm@blackflag.ru Date: Wed, 9 Jul 2003 13:44:49 +0400 (MSD) X-Mailer: ELM [version 2.4ME+ PL31 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit cc: freebsd-fs@FreeBSD.ORG cc: olli@secnetix.de Subject: Re: NFS problem FreeBSD vs NetApp Filer using tomcat / java X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2003 09:42:47 -0000 Sorry for the typos (I'm really sick today). I'm not talking of "c", it's always "s". That is, I meant "s" and "i" options and never intended to talk about "c" :) Just to clean up the mess: s/"c"/"s"/g > Oliver, > > > It is my understanding that the interruptible flag has only > > an effect when a signal is delivered to a process which is > > blocked on NFS I/O. Or am I wrong? > > Do you use "i" or "s" option or both? With "c" you may try to turn off the > dynamic retransmit timeout estimator and set timeout explicitly with "t". > We faced a problem similar to yours when using "ci" for UDP mounts from > FreeBSD client and Intel Solaris 8 server. From owner-freebsd-fs@FreeBSD.ORG Wed Jul 9 03:17:39 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B7D9037B401 for ; Wed, 9 Jul 2003 03:17:39 -0700 (PDT) Received: from salmon.maths.tcd.ie (salmon.maths.tcd.ie [134.226.81.11]) by mx1.FreeBSD.org (Postfix) with SMTP id 6B21343F93 for ; Wed, 9 Jul 2003 03:17:38 -0700 (PDT) (envelope-from iedowse@maths.tcd.ie) Received: from walton.maths.tcd.ie by salmon.maths.tcd.ie with SMTP id ; 9 Jul 2003 11:17:28 +0100 (BST) To: Oliver Fromme In-Reply-To: Your message of "Wed, 09 Jul 2003 11:13:49 +0200." <200307090913.h699DnK3053575@lurza.secnetix.de> Date: Wed, 09 Jul 2003 11:17:27 +0100 From: Ian Dowse Message-ID: <200307091117.aa12894@salmon.maths.tcd.ie> cc: freebsd-fs@FreeBSD.ORG Subject: Re: NFS problem FreeBSD vs NetApp Filer using tomcat / java X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2003 10:17:40 -0000 In message <200307090913.h699DnK3053575@lurza.secnetix.de>, Oliver Fromme write s: >Someone asked me off-list whether I use the "interruptible" >flag for the NFS mounts, and suggested switching it off. ... >It is my understanding that the interruptible flag has only >an effect when a signal is delivered to a process which is >blocked on NFS I/O. Or am I wrong? Yes, the interruptible flag should only have an effect when the process receives a signal while waiting on an NFS response. Maybe it is possible that the progess is sending itself a signal? NFS checks for SIGINT, SIGTERM, SIGHUP, SIGKILL and SIGQUIT. If you are using the -s/soft option then that is quite different - there are well-known effects where an unusually long delay from the server can trigger occasional EINTR errors. Ian From owner-freebsd-fs@FreeBSD.ORG Wed Jul 9 03:44:20 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7753337B401 for ; Wed, 9 Jul 2003 03:44:20 -0700 (PDT) Received: from lurza.secnetix.de (lurza.secnetix.de [212.66.1.130]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6287E43FB1 for ; Wed, 9 Jul 2003 03:44:19 -0700 (PDT) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (vidqfm@localhost [127.0.0.1]) by lurza.secnetix.de (8.12.8p1/8.12.8) with ESMTP id h69AiHB5056292 for ; Wed, 9 Jul 2003 12:44:18 +0200 (CEST) (envelope-from oliver.fromme@secnetix.de) Received: (from olli@localhost) by lurza.secnetix.de (8.12.8p1/8.12.8/Submit) id h69AiHte056291; Wed, 9 Jul 2003 12:44:17 +0200 (CEST) Date: Wed, 9 Jul 2003 12:44:17 +0200 (CEST) Message-Id: <200307091044.h69AiHte056291@lurza.secnetix.de> From: Oliver Fromme To: freebsd-fs@FreeBSD.ORG In-Reply-To: <200307091117.aa12894@salmon.maths.tcd.ie> X-Newsgroups: list.freebsd-fs User-Agent: tin/1.5.4-20000523 ("1959") (UNIX) (FreeBSD/4.8-RELEASE (i386)) MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Subject: Re: NFS problem FreeBSD vs NetApp Filer using tomcat / java X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2003 10:44:20 -0000 Ian Dowse wrote: > In message <200307090913.h699DnK3053575@lurza.secnetix.de>, Oliver Fromme writes: > > It is my understanding that the interruptible flag has only > > an effect when a signal is delivered to a process which is > > blocked on NFS I/O. Or am I wrong? > > Yes, the interruptible flag should only have an effect when the > process receives a signal while waiting on an NFS response. Maybe > it is possible that the progess is sending itself a signal? NFS > checks for SIGINT, SIGTERM, SIGHUP, SIGKILL and SIGQUIT. Uhm. Well. It's a collection of Linux java processes (about 30 of them), and the load on them is very erratic, so it's a bit diffcult to find out what they do. I guess it's very possible that they send such signals to each other or to themselves. In the truss output of some of the processes there are calls to linux_rt_sigprocmask() and linux_kill(). I've also seen "SIGNAL 32" in the truss output, which seems strange to me, because there is no such signal number in . Well, maybe it's a Linux signal. I also see linux_kill(0xf6b3,0x20), and 0x20 is 32. But I guess the NFS code does not check for signal 32. Or maybe the NFS code has a bug with non- standard signal numbers >= 32? > If you are using the -s/soft option then that is quite different - > there are well-known effects where an unusually long delay from > the server can trigger occasional EINTR errors. Nope, I never use the "soft" option. I'm well aware that soft mounts are evil. I have now removed the intr flag from the mount. Now the only thing I can do is wait ... If the problem doesn't occur for a few days, I regard it to be solved. Thanks a lot to all of you! Regards Oliver -- Oliver Fromme, secnetix GmbH & Co KG, Oettingenstr. 2, 80538 München Any opinions expressed in this message may be personal to the author and may not necessarily reflect the opinions of secnetix in any way. From owner-freebsd-fs@FreeBSD.ORG Wed Jul 9 12:05:06 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 790D537B401 for ; Wed, 9 Jul 2003 12:05:06 -0700 (PDT) Received: from web80010.mail.yahoo.com (web80010.mail.yahoo.com [66.163.168.140]) by mx1.FreeBSD.org (Postfix) with SMTP id D062143FA3 for ; Wed, 9 Jul 2003 12:05:05 -0700 (PDT) (envelope-from soundsampler@sbcglobal.net) Message-ID: <20030709190505.37532.qmail@web80010.mail.yahoo.com> Received: from [64.171.77.116] by web80010.mail.yahoo.com via HTTP; Wed, 09 Jul 2003 12:05:05 PDT Date: Wed, 9 Jul 2003 12:05:05 -0700 (PDT) From: Trent George To: Bruce Evans In-Reply-To: <20030708173454.N10674@gamplex.bde.org> MIME-Version: 1.0 X-Mailman-Approved-At: Wed, 09 Jul 2003 12:40:44 -0700 Content-Type: text/plain; charset=us-ascii X-Content-Filtered-By: Mailman/MimeDel 2.1.1 cc: freebsd-fs@freebsd.org Subject: Re: Software Disk Cache Limit Issues X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2003 19:05:06 -0000 > Try turning off write clustering for ffs (mount -u -onoclusterw... ...).This solved the problem, thanks. now setting vfs.hirunningspace is not capped by the size of nswbuf. physical write speed does not seemed slowed down ether.Thanks a lot. From owner-freebsd-fs@FreeBSD.ORG Sat Jul 12 18:08:12 2003 Return-Path: Delivered-To: freebsd-fs@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F2ABE37B401; Sat, 12 Jul 2003 18:08:11 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7EA7043FAF; Sat, 12 Jul 2003 18:08:11 -0700 (PDT) (envelope-from kris@FreeBSD.org) Received: from freefall.freebsd.org (kris@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with ESMTP id h6D18BUp050586; Sat, 12 Jul 2003 18:08:11 -0700 (PDT) (envelope-from kris@freefall.freebsd.org) Received: (from kris@localhost) by freefall.freebsd.org (8.12.9/8.12.9/Submit) id h6D18BrS050582; Sat, 12 Jul 2003 18:08:11 -0700 (PDT) Date: Sat, 12 Jul 2003 18:08:11 -0700 (PDT) From: Kris Kennaway Message-Id: <200307130108.h6D18BrS050582@freefall.freebsd.org> To: kris@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-fs@FreeBSD.org Subject: Re: bin/43635: filesystem damage not detected by fsck X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Jul 2003 01:08:12 -0000 Synopsis: filesystem damage not detected by fsck Responsible-Changed-From-To: freebsd-bugs->freebsd-fs Responsible-Changed-By: kris Responsible-Changed-When: Sat Jul 12 18:07:55 PDT 2003 Responsible-Changed-Why: Assign to fs mailing list for evaluation of the included patch http://www.freebsd.org/cgi/query-pr.cgi?pr=43635 From owner-freebsd-fs@FreeBSD.ORG Sat Jul 12 18:59:57 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CB2E837B401; Sat, 12 Jul 2003 18:59:57 -0700 (PDT) Received: from mailbox.univie.ac.at (mail.univie.ac.at [131.130.1.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id CE33A43F93; Sat, 12 Jul 2003 18:59:56 -0700 (PDT) (envelope-from l.ertl@univie.ac.at) Received: from dialin202.cc.univie.ac.at (dialin202.cc.univie.ac.at [131.130.202.202]) by mailbox.univie.ac.at (8.12.2/8.12.2) with ESMTP id h6D1xYcq029802; Sun, 13 Jul 2003 03:59:43 +0200 Date: Sun, 13 Jul 2003 03:59:22 +0200 (CEST) From: Lukas Ertl To: Kris Kennaway In-Reply-To: <200307130108.h6D18BrS050582@freefall.freebsd.org> Message-ID: <20030713035643.D5665@korben.in.tern> References: <200307130108.h6D18BrS050582@freefall.freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=ISO-8859-1 Content-Transfer-Encoding: QUOTED-PRINTABLE X-DCC-ZID-Univie-Metrics: mailbox 4251; Body=3 Fuz1=3 Fuz2=3 cc: freebsd-fs@freebsd.org cc: freebsd-bugs@freebsd.org Subject: Re: bin/43635: filesystem damage not detected by fsck X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Jul 2003 01:59:58 -0000 On Sat, 12 Jul 2003, Kris Kennaway wrote: > Synopsis: filesystem damage not detected by fsck > > Responsible-Changed-From-To: freebsd-bugs->freebsd-fs > Responsible-Changed-By: kris > Responsible-Changed-When: Sat Jul 12 18:07:55 PDT 2003 > Responsible-Changed-Why: > Assign to fs mailing list for evaluation of the included patch This PR can be closed, it's a duplicate of bin/40697, and the fix was incorporated in rev. 1.34 of src/sbin/fsck_ffs/pass5.c. regards, le --=20 Lukas Ertl eMail: l.ertl@univie.ac.at UNIX-Systemadministrator Tel.: (+43 1) 4277-14073 Zentraler Informatikdienst (ZID) Fax.: (+43 1) 4277-9140 der Universit=E4t Wien http://mailbox.univie.ac.at/~le/