Date: Mon, 20 Jun 2011 10:25:52 +0000 From: Traiano Welcome <Traiano.Welcome@mtnbusiness.co.za> To: Damien Fleuriot <ml@my.gd>, "freebsd-questions@freebsd.org" <freebsd-questions@freebsd.org> Subject: RE: (no subject) Message-ID: <E012414FCF65894B89F69DE76AE15E9905905FEF@CPT-EXCH01.int.mtnbusiness.net> In-Reply-To: <4DFF1A70.3030300@my.gd> References: <E012414FCF65894B89F69DE76AE15E9905905E67@CPT-EXCH01.int.mtnbusiness.net>, <4DFF1A70.3030300@my.gd>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Damien=0A=
=0A=
(apologies for top-posting, handicapped mail client).=0A=
=0A=
Actually, "/" (by /tmp) is filling up, and clearing very rapidly due to tem=
p files being created and removed at high speed. We ca only see this=0A=
by doing:=0A=
=0A=
---=0A=
#!/usr/bin/perl=0A=
while(1){=0A=
$timestamp =3D localtime();=0A=
system("echo $timestamp `df -h /tmp` >> /home/traianow/dfstats.txt");=0A=
system("echo $timestamp `du -sh /tmp` >> /home/traianow/dfstats.txt");=0A=
sleep 1;=0A=
}=0A=
---=0A=
=0A=
=0A=
We're seeing this fast-changing disk space usage patterns like this, repeat=
ing every few tens of seconds:=0A=
=0A=
----=0A=
Mon Jun 20 11:41:54 2011 844M /tmp=0A=
Mon Jun 20 11:41:55 2011 Filesystem Size Used Avail Capacity Mounted on /de=
v/amrd0s1a 989M 987M -76M 108% /=0A=
=0A=
Mon Jun 20 11:41:55 2011 849M /tmp=0A=
Mon Jun 20 11:41:56 2011 Filesystem Size Used Avail Capacity Mounted on /de=
v/amrd0s1a 989M 987M -76M 108% /=0A=
=0A=
Mon Jun 20 11:41:56 2011 849M /tmp=0A=
Mon Jun 20 11:41:57 2011 Filesystem Size Used Avail Capacity Mounted on /de=
v/amrd0s1a 989M 987M -76M 108% /=0A=
=0A=
Mon Jun 20 11:41:57 2011 849M /tmp=0A=
Mon Jun 20 11:41:58 2011 Filesystem Size Used Avail Capacity Mounted on /de=
v/amrd0s1a 989M 987M -76M 108% /=0A=
=0A=
Mon Jun 20 11:41:58 2011 849M /tmp=0A=
Mon Jun 20 11:42:01 2011 Filesystem Size Used Avail Capacity Mounted on /de=
v/amrd0s1a 989M 987M -76M 108% /=0A=
=0A=
Mon Jun 20 11:42:01 2011 849M /tmp=0A=
Mon Jun 20 11:42:02 2011 Filesystem Size Used Avail Capacity Mounted on /de=
v/amrd0s1a 989M 141M 769M 15% /=0A=
=0A=
Mon Jun 20 11:42:02 2011 3.2M /tmp=0A=
Mon Jun 20 11:42:03 2011 Filesystem Size Used Avail Capacity Mounted on /de=
v/amrd0s1a 989M 142M 768M 16% /=0A=
=0A=
Mon Jun 20 11:42:03 2011 4.8M /tmp=0A=
Mon Jun 20 11:42:04 2011 Filesystem Size Used Avail Capacity Mounted on /de=
v/amrd0s1a 989M 145M 765M 16% /=0A=
=0A=
Mon Jun 20 11:42:04 2011 7.7M /tmp=0A=
Mon Jun 20 11:42:06 2011 Filesystem Size Used Avail Capacity Mounted on /de=
v/amrd0s1a 989M 148M 762M 16% /=0A=
=0A=
Mon Jun 20 11:42:06 2011 10M /tmp=0A=
Mon Jun 20 11:42:07 2011 Filesystem Size Used Avail Capacity Mounted on /de=
v/amrd0s1a 989M 150M 760M 16% /=0A=
----=0A=
=0A=
=0A=
What I'm trying to determine is what caused the change in temp file writing=
behaviour on the server, and if this is the kind behaviour likely on a hea=
vily loaded box with cpu running at 100% (which this system is). i.e, do pr=
ocesses like cvs that write tmp files suddenly start writing more temp file=
s when starved for cpu, leading to this kind of behaviour? =0A=
=0A=
=0A=
Thanks,=0A=
Traiano=0A=
=0A=
________________________________________=0A=
From: owner-freebsd-questions@freebsd.org [owner-freebsd-questions@freebsd.=
org] on behalf of Damien Fleuriot [ml@my.gd]=0A=
Sent: Monday, June 20, 2011 12:01 PM=0A=
To: freebsd-questions@freebsd.org=0A=
Subject: Re: (no subject)=0A=
=0A=
On 6/20/11 10:13 AM, Traiano Welcome wrote:=0A=
> Hi List=0A=
>=0A=
> We have a FreeBSD 6.2-STABLE #0 server running as a general unix shell se=
rver. Recently the system has been running at high load (average 8, and cpu=
100%), and even more recently we've started seeing the following types o=
f error when we do cvs commits on the system. The system has between 150 to=
200 users on it during the day.=0A=
>=0A=
> ---=0A=
> "/: write failed, filesystem is full"=0A=
> Error: /tmp/file.commit.72971.tmp: No space left on device; /tmp/file.com=
mit.72971.tmp: WARNING: FILE TRUNCATED=0A=
> ---=0A=
>=0A=
> The disks are definitely not full (this shows up in df -hi), both in term=
s of storage space and inode utilisation. However the cpu utilisation is pe=
rmanently at 100%, and we're aware of which processes are causing the utili=
sation. My question is: Is it possible, under some circumstances that cpu =
starvation could result in the type of "filesystem is full" errors we're se=
eing above?=0A=
>=0A=
> Thanks in Advance,=0A=
> Traiano Welcome=0A=
>=0A=
> _______________________________________________=0A=
> freebsd-questions@freebsd.org mailing list=0A=
> http://lists.freebsd.org/mailman/listinfo/freebsd-questions=0A=
> To unsubscribe, send any mail to "freebsd-questions-unsubscribe@freebsd.o=
rg"=0A=
=0A=
=0A=
=0A=
Are you really sure your file system is not full ?=0A=
=0A=
1/ sync=0A=
2/ df -h=0A=
3/ df -i=0A=
_______________________________________________=0A=
freebsd-questions@freebsd.org mailing list=0A=
http://lists.freebsd.org/mailman/listinfo/freebsd-questions=0A=
To unsubscribe, send any mail to "freebsd-questions-unsubscribe@freebsd.org=
"=0A=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?E012414FCF65894B89F69DE76AE15E9905905FEF>