From owner-freebsd-questions@FreeBSD.ORG Wed Feb 10 12:44:31 2010 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 213CE1065672 for ; Wed, 10 Feb 2010 12:44:31 +0000 (UTC) (envelope-from freebsd-questions@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id EA8928FC12 for ; Wed, 10 Feb 2010 12:44:29 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1NfBvZ-0002IN-35 for freebsd-questions@freebsd.org; Wed, 10 Feb 2010 13:44:21 +0100 Received: from 62.97.71.6 ([62.97.71.6]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 10 Feb 2010 13:44:21 +0100 Received: from matiassurdi by 62.97.71.6 with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 10 Feb 2010 13:44:21 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-questions@freebsd.org From: Matias Date: Wed, 10 Feb 2010 13:44:00 +0100 Lines: 254 Message-ID: Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 62.97.71.6 User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; es-ES; rv:1.9.1.7) Gecko/20100111 Thunderbird/3.0.1 Sender: news Subject: Filesystem problem X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Feb 2010 12:44:31 -0000 Hi, I know this may not be the correct place to post this, but I feel like I'm going to get more help here than in the ejabberd forums. Maybe you can help me to debug what is going on here: I'm runing ejabberd-2.1-2 on FreeBSD it05 8.0-RELEASE FreeBSD 8.0-RELEASE #0: Sat Nov 21 15:02:08 UTC 2009 root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 Suddenly, my /var partition is running out of space. When I run df -h, i get: ]# df -h Filesystem Size Used Avail Capacity Mounted on /dev/mirror/gm0s1a 496M 373M 83M 82% / devfs 1.0K 1.0K 0B 100% /dev /dev/mirror/gm0s1e 496M 58M 398M 13% /tmp /dev/mirror/gm0s1f 433G 2.6G 396G 1% /usr /dev/mirror/gm0s1d 13G 11G 998M 92% /var procfs 4.0K 4.0K 0B 100% /proc linprocfs 4.0K 4.0K 0B 100% /usr/compat/linux/proc When I cd to /var and run a du -hs I get: # cd /var/ [root@it05 /var]# du -hs * 2.0K account 2.0K agentx 6.0K at 2.0K audit 12K backups 4.0K crash 6.0K cron 168M db 2.0K empty 2.0K games 2.0K heimdal 18K lib 8.8M log 850K mail 4.0K msgs 48K named 4.0K net-snmp 2.0K preserve 54K run 2.0K rwho 3.5M spool 116K tmp 24K yp So, it seems that something is sucking up all the space but doesn't have a file descriptor on the file system, so du is not finding the file. Running lsof|grep /var I get: ... beam.smp 28398 ejabberd 10w VREG 0,93 31895 23581 /var/log/ejabberd/erlang.log beam.smp 28398 ejabberd 11u VREG 0,93 97 47130 /var/spool/ejabberd/LATEST.LOG beam.smp 28398 ejabberd 12w VREG 0,93 11170768676 23582 /var (/dev/mirror/gm0s1d) beam.smp 28398 ejabberd 19u VREG 0,93 119220 47140 /var/spool/ejabberd/offline_msg.DAT beam.smp 28398 ejabberd 20u VREG 0,93 6809 47142 /var/spool/ejabberd/private_storage.DAT beam.smp 28398 ejabberd 22u VREG 0,93 1282598 47148 /var/spool/ejabberd/pubsub_item.DAT beam.smp 28398 ejabberd 23u VREG 0,93 5752 47154 /var/spool/ejabberd/vcard.DAT beam.smp 28398 ejabberd 10w VREG 0,93 31895 23581 /var/log/ejabberd/erlang.log beam.smp 28398 ejabberd 11u VREG 0,93 97 47130 /var/spool/ejabberd/LATEST.LOG beam.smp 28398 ejabberd 12w VREG 0,93 11170768676 23582 /var (/dev/mirror/gm0s1d) beam.smp 28398 ejabberd 19u VREG 0,93 119220 47140 /var/spool/ejabberd/offline_msg.DAT beam.smp 28398 ejabberd 20u VREG 0,93 6809 47142 /var/spool/ejabberd/private_storage.DAT beam.smp 28398 ejabberd 22u VREG 0,93 1282598 47148 /var/spool/ejabberd/pubsub_item.DAT beam.smp 28398 ejabberd 23u VREG 0,93 5752 47154 /var/spool/ejabberd/vcard.DAT beam.smp 28398 ejabberd 10w VREG 0,93 31895 23581 /var/log/ejabberd/erlang.log beam.smp 28398 ejabberd 11u VREG 0,93 97 47130 /var/spool/ejabberd/LATEST.LOG beam.smp 28398 ejabberd 12w VREG 0,93 11170768676 23582 /var (/dev/mirror/gm0s1d) beam.smp 28398 ejabberd 19u VREG 0,93 119220 47140 /var/spool/ejabberd/offline_msg.DAT beam.smp 28398 ejabberd 20u VREG 0,93 6809 47142 /var/spool/ejabberd/private_storage.DAT beam.smp 28398 ejabberd 22u VREG 0,93 1282598 47148 /var/spool/ejabberd/pubsub_item.DAT beam.smp 28398 ejabberd 23u VREG 0,93 5752 47154 /var/spool/ejabberd/vcard.DAT beam.smp 28398 ejabberd 10w VREG 0,93 31895 23581 /var/log/ejabberd/erlang.log beam.smp 28398 ejabberd 11u VREG 0,93 97 47130 /var/spool/ejabberd/LATEST.LOG beam.smp 28398 ejabberd 12w VREG 0,93 11170768676 23582 /var (/dev/mirror/gm0s1d) beam.smp 28398 ejabberd 19u VREG 0,93 119220 47140 /var/spool/ejabberd/offline_msg.DAT beam.smp 28398 ejabberd 20u VREG 0,93 6809 47142 /var/spool/ejabberd/private_storage.DAT beam.smp 28398 ejabberd 22u VREG 0,93 1282598 47148 /var/spool/ejabberd/pubsub_item.DAT beam.smp 28398 ejabberd 23u VREG 0,93 5752 47154 /var/spool/ejabberd/vcard.DAT beam.smp 28398 ejabberd 10w VREG 0,93 31895 23581 /var/log/ejabberd/erlang.log beam.smp 28398 ejabberd 11u VREG 0,93 97 47130 /var/spool/ejabberd/LATEST.LOG beam.smp 28398 ejabberd 12w VREG 0,93 11170768676 23582 /var (/dev/mirror/gm0s1d) beam.smp 28398 ejabberd 19u VREG 0,93 119220 47140 /var/spool/ejabberd/offline_msg.DAT beam.smp 28398 ejabberd 20u VREG 0,93 6809 47142 /var/spool/ejabberd/private_storage.DAT beam.smp 28398 ejabberd 22u VREG 0,93 1282598 47148 /var/spool/ejabberd/pubsub_item.DAT beam.smp 28398 ejabberd 23u VREG 0,93 5752 47154 /var/spool/ejabberd/vcard.DAT beam.smp 28398 ejabberd 10w VREG 0,93 31895 23581 /var/log/ejabberd/erlang.log beam.smp 28398 ejabberd 11u VREG 0,93 97 47130 /var/spool/ejabberd/LATEST.LOG beam.smp 28398 ejabberd 12w VREG 0,93 11170768676 23582 /var (/dev/mirror/gm0s1d) beam.smp 28398 ejabberd 19u VREG 0,93 119220 47140 /var/spool/ejabberd/offline_msg.DAT beam.smp 28398 ejabberd 20u VREG 0,93 6809 47142 /var/spool/ejabberd/private_storage.DAT beam.smp 28398 ejabberd 22u VREG 0,93 1282598 47148 /var/spool/ejabberd/pubsub_item.DAT beam.smp 28398 ejabberd 23u VREG 0,93 5752 47154 /var/spool/ejabberd/vcard.DAT beam.smp 28398 ejabberd 10w VREG 0,93 31895 23581 /var/log/ejabberd/erlang.log beam.smp 28398 ejabberd 11u VREG 0,93 97 47130 /var/spool/ejabberd/LATEST.LOG beam.smp 28398 ejabberd 12w VREG 0,93 11170768676 23582 /var (/dev/mirror/gm0s1d) beam.smp 28398 ejabberd 19u VREG 0,93 119220 47140 /var/spool/ejabberd/offline_msg.DAT beam.smp 28398 ejabberd 20u VREG 0,93 6809 47142 /var/spool/ejabberd/private_storage.DAT beam.smp 28398 ejabberd 22u VREG 0,93 1282598 47148 /var/spool/ejabberd/pubsub_item.DAT beam.smp 28398 ejabberd 23u VREG 0,93 5752 47154 /var/spool/ejabberd/vcard.DAT beam.smp 28398 ejabberd 10w VREG 0,93 31895 23581 /var/log/ejabberd/erlang.log beam.smp 28398 ejabberd 11u VREG 0,93 97 47130 /var/spool/ejabberd/LATEST.LOG beam.smp 28398 ejabberd 12w VREG 0,93 11170768676 23582 /var (/dev/mirror/gm0s1d) beam.smp 28398 ejabberd 19u VREG 0,93 119220 47140 /var/spool/ejabberd/offline_msg.DAT beam.smp 28398 ejabberd 20u VREG 0,93 6809 47142 /var/spool/ejabberd/private_storage.DAT beam.smp 28398 ejabberd 22u VREG 0,93 1282598 47148 /var/spool/ejabberd/pubsub_item.DAT beam.smp 28398 ejabberd 23u VREG 0,93 5752 47154 /var/spool/ejabberd/vcard.DAT beam.smp 28398 ejabberd 10w VREG 0,93 31895 23581 /var/log/ejabberd/erlang.log beam.smp 28398 ejabberd 11u VREG 0,93 97 47130 /var/spool/ejabberd/LATEST.LOG beam.smp 28398 ejabberd 12w VREG 0,93 11170768676 23582 /var (/dev/mirror/gm0s1d) beam.smp 28398 ejabberd 19u VREG 0,93 119220 47140 /var/spool/ejabberd/offline_msg.DAT beam.smp 28398 ejabberd 20u VREG 0,93 6809 47142 /var/spool/ejabberd/private_storage.DAT beam.smp 28398 ejabberd 22u VREG 0,93 1282598 47148 /var/spool/ejabberd/pubsub_item.DAT beam.smp 28398 ejabberd 23u VREG 0,93 5752 47154 /var/spool/ejabberd/vcard.DAT beam.smp 28398 ejabberd 10w VREG 0,93 31895 23581 /var/log/ejabberd/erlang.log beam.smp 28398 ejabberd 11u VREG 0,93 97 47130 /var/spool/ejabberd/LATEST.LOG beam.smp 28398 ejabberd 12w VREG 0,93 11170768676 23582 /var (/dev/mirror/gm0s1d) beam.smp 28398 ejabberd 19u VREG 0,93 119220 47140 /var/spool/ejabberd/offline_msg.DAT beam.smp 28398 ejabberd 20u VREG 0,93 6809 47142 /var/spool/ejabberd/private_storage.DAT beam.smp 28398 ejabberd 22u VREG 0,93 1282598 47148 /var/spool/ejabberd/pubsub_item.DAT beam.smp 28398 ejabberd 23u VREG 0,93 5752 47154 /var/spool/ejabberd/vcard.DAT beam.smp 28398 ejabberd 10w VREG 0,93 31895 23581 /var/log/ejabberd/erlang.log beam.smp 28398 ejabberd 11u VREG 0,93 97 47130 /var/spool/ejabberd/LATEST.LOG beam.smp 28398 ejabberd 12w VREG 0,93 11170768676 23582 /var (/dev/mirror/gm0s1d) beam.smp 28398 ejabberd 19u VREG 0,93 119220 47140 /var/spool/ejabberd/offline_msg.DAT beam.smp 28398 ejabberd 20u VREG 0,93 6809 47142 /var/spool/ejabberd/private_storage.DAT beam.smp 28398 ejabberd 22u VREG 0,93 1282598 47148 /var/spool/ejabberd/pubsub_item.DAT beam.smp 28398 ejabberd 23u VREG 0,93 5752 47154 /var/spool/ejabberd/vcard.DAT beam.smp 28398 ejabberd 10w VREG 0,93 31895 23581 /var/log/ejabberd/erlang.log beam.smp 28398 ejabberd 11u VREG 0,93 97 47130 /var/spool/ejabberd/LATEST.LOG beam.smp 28398 ejabberd 12w VREG 0,93 11170768676 23582 /var (/dev/mirror/gm0s1d) beam.smp 28398 ejabberd 19u VREG 0,93 119220 47140 /var/spool/ejabberd/offline_msg.DAT beam.smp 28398 ejabberd 20u VREG 0,93 6809 47142 /var/spool/ejabberd/private_storage.DAT beam.smp 28398 ejabberd 22u VREG 0,93 1282598 47148 /var/spool/ejabberd/pubsub_item.DAT beam.smp 28398 ejabberd 23u VREG 0,93 5752 47154 /var/spool/ejabberd/vcard.DAT beam.smp 28398 ejabberd 10w VREG 0,93 31895 23581 /var/log/ejabberd/erlang.log beam.smp 28398 ejabberd 11u VREG 0,93 97 47130 /var/spool/ejabberd/LATEST.LOG beam.smp 28398 ejabberd 12w VREG 0,93 11170768676 23582 /var (/dev/mirror/gm0s1d) beam.smp 28398 ejabberd 19u VREG 0,93 119220 47140 /var/spool/ejabberd/offline_msg.DAT beam.smp 28398 ejabberd 20u VREG 0,93 6809 47142 /var/spool/ejabberd/private_storage.DAT beam.smp 28398 ejabberd 22u VREG 0,93 1282598 47148 /var/spool/ejabberd/pubsub_item.DAT beam.smp 28398 ejabberd 23u VREG 0,93 5752 47154 /var/spool/ejabberd/vcard.DAT ... So, it seems that ejabberd is sucking up 10Gb I'm missing on my filesystem, and that space is being reserved into the /var directory. Is this a bug on ejabberd? Is there any way to solve this without having to restart the server (not tried yet, as many people is using it....) Thanks. How