From owner-freebsd-questions@FreeBSD.ORG Mon Jun 20 10:26:01 2011 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 665BF106564A for ; Mon, 20 Jun 2011 10:26:01 +0000 (UTC) (envelope-from Traiano.Welcome@mtnbusiness.co.za) Received: from smtprelay01.ops.mtnbusiness.co.za (smtprelay01.ops.mtnbusiness.co.za [41.181.93.235]) by mx1.freebsd.org (Postfix) with ESMTP id AB0548FC13 for ; Mon, 20 Jun 2011 10:26:00 +0000 (UTC) Received: from [196.30.97.135] (helo=CPT-EXCH01.int.mtnbusiness.net) by smtprelay01.ops.mtnbusiness.co.za with esmtp (ULTRA Special SMTP Internal Alpha) (envelope-from ) id 1QYbg3-000H5l-33; Mon, 20 Jun 2011 12:25:55 +0200 Received: from CPT-EXCH01.int.mtnbusiness.net ([196.30.97.135]) by CPT-EXCH01.int.mtnbusiness.net ([196.30.97.135]) with mapi id 14.01.0218.012; Mon, 20 Jun 2011 12:25:53 +0200 From: Traiano Welcome To: Damien Fleuriot , "freebsd-questions@freebsd.org" Thread-Topic: (no subject) Thread-Index: AcwvIQlAgI1zA7i3RMifi7YpxK5AhP///mgAgAAmEwo= Date: Mon, 20 Jun 2011 10:25:52 +0000 Message-ID: References: , <4DFF1A70.3030300@my.gd> In-Reply-To: <4DFF1A70.3030300@my.gd> Accept-Language: en-US, en-ZA Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [196.30.72.139] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Cc: Subject: RE: (no subject) X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2011 10:26:01 -0000 Hi Damien=0A= =0A= (apologies for top-posting, handicapped mail client).=0A= =0A= Actually, "/" (by /tmp) is filling up, and clearing very rapidly due to tem= p files being created and removed at high speed. We ca only see this=0A= by doing:=0A= =0A= ---=0A= #!/usr/bin/perl=0A= while(1){=0A= $timestamp =3D localtime();=0A= system("echo $timestamp `df -h /tmp` >> /home/traianow/dfstats.txt");=0A= system("echo $timestamp `du -sh /tmp` >> /home/traianow/dfstats.txt");=0A= sleep 1;=0A= }=0A= ---=0A= =0A= =0A= We're seeing this fast-changing disk space usage patterns like this, repeat= ing every few tens of seconds:=0A= =0A= ----=0A= Mon Jun 20 11:41:54 2011 844M /tmp=0A= Mon Jun 20 11:41:55 2011 Filesystem Size Used Avail Capacity Mounted on /de= v/amrd0s1a 989M 987M -76M 108% /=0A= =0A= Mon Jun 20 11:41:55 2011 849M /tmp=0A= Mon Jun 20 11:41:56 2011 Filesystem Size Used Avail Capacity Mounted on /de= v/amrd0s1a 989M 987M -76M 108% /=0A= =0A= Mon Jun 20 11:41:56 2011 849M /tmp=0A= Mon Jun 20 11:41:57 2011 Filesystem Size Used Avail Capacity Mounted on /de= v/amrd0s1a 989M 987M -76M 108% /=0A= =0A= Mon Jun 20 11:41:57 2011 849M /tmp=0A= Mon Jun 20 11:41:58 2011 Filesystem Size Used Avail Capacity Mounted on /de= v/amrd0s1a 989M 987M -76M 108% /=0A= =0A= Mon Jun 20 11:41:58 2011 849M /tmp=0A= Mon Jun 20 11:42:01 2011 Filesystem Size Used Avail Capacity Mounted on /de= v/amrd0s1a 989M 987M -76M 108% /=0A= =0A= Mon Jun 20 11:42:01 2011 849M /tmp=0A= Mon Jun 20 11:42:02 2011 Filesystem Size Used Avail Capacity Mounted on /de= v/amrd0s1a 989M 141M 769M 15% /=0A= =0A= Mon Jun 20 11:42:02 2011 3.2M /tmp=0A= Mon Jun 20 11:42:03 2011 Filesystem Size Used Avail Capacity Mounted on /de= v/amrd0s1a 989M 142M 768M 16% /=0A= =0A= Mon Jun 20 11:42:03 2011 4.8M /tmp=0A= Mon Jun 20 11:42:04 2011 Filesystem Size Used Avail Capacity Mounted on /de= v/amrd0s1a 989M 145M 765M 16% /=0A= =0A= Mon Jun 20 11:42:04 2011 7.7M /tmp=0A= Mon Jun 20 11:42:06 2011 Filesystem Size Used Avail Capacity Mounted on /de= v/amrd0s1a 989M 148M 762M 16% /=0A= =0A= Mon Jun 20 11:42:06 2011 10M /tmp=0A= Mon Jun 20 11:42:07 2011 Filesystem Size Used Avail Capacity Mounted on /de= v/amrd0s1a 989M 150M 760M 16% /=0A= ----=0A= =0A= =0A= What I'm trying to determine is what caused the change in temp file writing= behaviour on the server, and if this is the kind behaviour likely on a hea= vily loaded box with cpu running at 100% (which this system is). i.e, do pr= ocesses like cvs that write tmp files suddenly start writing more temp file= s when starved for cpu, leading to this kind of behaviour? =0A= =0A= =0A= Thanks,=0A= Traiano=0A= =0A= ________________________________________=0A= From: owner-freebsd-questions@freebsd.org [owner-freebsd-questions@freebsd.= org] on behalf of Damien Fleuriot [ml@my.gd]=0A= Sent: Monday, June 20, 2011 12:01 PM=0A= To: freebsd-questions@freebsd.org=0A= Subject: Re: (no subject)=0A= =0A= On 6/20/11 10:13 AM, Traiano Welcome wrote:=0A= > Hi List=0A= >=0A= > We have a FreeBSD 6.2-STABLE #0 server running as a general unix shell se= rver. Recently the system has been running at high load (average 8, and cpu= 100%), and even more recently we've started seeing the following types o= f error when we do cvs commits on the system. The system has between 150 to= 200 users on it during the day.=0A= >=0A= > ---=0A= > "/: write failed, filesystem is full"=0A= > Error: /tmp/file.commit.72971.tmp: No space left on device; /tmp/file.com= mit.72971.tmp: WARNING: FILE TRUNCATED=0A= > ---=0A= >=0A= > The disks are definitely not full (this shows up in df -hi), both in term= s of storage space and inode utilisation. However the cpu utilisation is pe= rmanently at 100%, and we're aware of which processes are causing the utili= sation. My question is: Is it possible, under some circumstances that cpu = starvation could result in the type of "filesystem is full" errors we're se= eing above?=0A= >=0A= > Thanks in Advance,=0A= > Traiano Welcome=0A= >=0A= > _______________________________________________=0A= > freebsd-questions@freebsd.org mailing list=0A= > http://lists.freebsd.org/mailman/listinfo/freebsd-questions=0A= > To unsubscribe, send any mail to "freebsd-questions-unsubscribe@freebsd.o= rg"=0A= =0A= =0A= =0A= Are you really sure your file system is not full ?=0A= =0A= 1/ sync=0A= 2/ df -h=0A= 3/ df -i=0A= _______________________________________________=0A= freebsd-questions@freebsd.org mailing list=0A= http://lists.freebsd.org/mailman/listinfo/freebsd-questions=0A= To unsubscribe, send any mail to "freebsd-questions-unsubscribe@freebsd.org= "=0A=