Date: Sat, 16 Oct 2010 23:38:55 -0400 From: jhell <jhell@DataIX.net> To: David Samms <dsamms@nw-ds.com> Cc: freebsd-fs@freebsd.org Subject: Re: ZFS trouble: unbelievably large files created Message-ID: <4CBA6FCF.3080301@DataIX.net> In-Reply-To: <i9alfp$c0o$1@dough.gmane.org> References: <i9a4vl$4v2$1@dough.gmane.org> <201010151909.o9FJ9cZf065459@lurza.secnetix.de> <i9alfp$c0o$1@dough.gmane.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 10/15/2010 18:44, David Samms wrote: > On 10/15/10 15:09, Oliver Fromme wrote: >> > ls -kls shawxp/var/amavis/.spamBAD >> > total 13352131 >> > 771 -rw------- 1 110 110 684032 Oct 15 12:02 >> > auto-whitelist >> > 1 -rw------- 1 110 110 40 Oct 15 12:37 >> bayes.lock >> > 1284 -rw------- 1 110 110 1294336 Oct 15 12:38 >> bayes_seen >> > 4229 -rw------- 1 110 110 4227072 Oct 15 12:38 >> bayes_toks >> > 5025103 -rw------- 1 110 110 553184002048 Oct 15 12:38 >> > bayes_toks.expire3515 >> > 8320745 -rw------- 1 110 110 140743122878464 Oct 15 12:14 >> > bayes_toks.expire97254 >> >> Ok, so those files are so-called "sparse" files, i.e. they >> contain holes that don't actually occupy disk space. >> >> The numbers in the first column indicate the amount of >> physical disk space allocated (in KB). That's about 5 GB >> for the first file and 8 GB for the second (this is also >> consistent with the "total" value in the first line of the >> ls output, i.e. about 13 GB). >> >> That's still quite big, but certainly not in the TB range. >> I do not know why amavis creates such large sparse files, >> though. > > Under UFS I don't observe amavis creating the sparse files. The problem > is fairly repeatable, not on demand, but occurs several times a day. > High disk activity is a sure sign amavis is creating large files. Task > shuts down normaly and cpu load is low. > > This is a production server but I have moved all critical customer jails > back to UFS so can do testing if anyone has any ideas of what to look for. > I do not believe this to be a problem with amavis/amavisd, lets get that out of the way. Why I say this is because I have experienced the same problem on a server with UFS and p5-Mail-Spamassassin. I was logged into that server doing some work and there was plenty of disk space to do the work as it consisted of copying a ton of space eating files to the file-system where this happened. After operations were done, cleaned up the disk properly restoring space back for normal usage and then logged out. I came back the next day only to find my INBOX full of error messages and other junk from that server saying things like can't write to disk and such error messages. What it turned out to be was the same files that you had mentioned above but in my ~/.spamassassin/.bayes*. One of the files just like yours had grown to eat up the entire amount of disk space that was left besides the 8% that was reserved. I tried to find the offending email messages that might have caused this to happen but in some odd ~7 hours total time did not come up with a conclusion and could not afford to waste further time on that issue. IMHO this has to do with bayes and the usage of dbm, or db4* and something in a messages header or body that triggers it to happen but do not have enough time or conclusive evidence yet to suggest anything further. Ultimately I removed the offending files and went about business as usual until I see it happen again. Also tuned the quota for myself to be a much lower amount disk space so I should get an email as soon as it happens again without effecting the rest of operations on that disk. Regards, -- jhell,v
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4CBA6FCF.3080301>