From owner-freebsd-questions@FreeBSD.ORG Sat Feb 20 20:26:19 2010 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CE9EA106566B for ; Sat, 20 Feb 2010 20:26:19 +0000 (UTC) (envelope-from dan@dan.emsphone.com) Received: from email1.allantgroup.com (email1.emsphone.com [199.67.51.115]) by mx1.freebsd.org (Postfix) with ESMTP id 971BB8FC0A for ; Sat, 20 Feb 2010 20:26:19 +0000 (UTC) Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101]) by email1.allantgroup.com (8.14.0/8.14.0) with ESMTP id o1KKQFi9038840 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Sat, 20 Feb 2010 14:26:16 -0600 (CST) (envelope-from dan@dan.emsphone.com) Received: from dan.emsphone.com (smmsp@localhost [127.0.0.1]) by dan.emsphone.com (8.14.4/8.14.3) with ESMTP id o1KKQFTZ010650 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Sat, 20 Feb 2010 14:26:15 -0600 (CST) (envelope-from dan@dan.emsphone.com) Received: (from dan@localhost) by dan.emsphone.com (8.14.4/8.14.3/Submit) id o1KKQFYD010576; Sat, 20 Feb 2010 14:26:15 -0600 (CST) (envelope-from dan) Date: Sat, 20 Feb 2010 14:26:15 -0600 From: Dan Nelson To: Boris Samorodov Message-ID: <20100220202614.GG70798@dan.emsphone.com> References: <97752307@ipt.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <97752307@ipt.ru> X-OS: FreeBSD 7.2-STABLE User-Agent: Mutt/1.5.20 (2009-06-14) X-Virus-Scanned: clamav-milter 0.95.3 at email1.allantgroup.com X-Virus-Status: Clean X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.0.2 (email1.allantgroup.com [199.67.51.78]); Sat, 20 Feb 2010 14:26:16 -0600 (CST) X-Scanned-By: MIMEDefang 2.45 Cc: freebsd-questions@freebsd.org Subject: Re: strange disk activity X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 Feb 2010 20:26:19 -0000 In the last episode (Feb 20), Boris Samorodov said: > Hello List, > > I've got a very strange disk activity: > ----- > % iostat -xw60 da0 > extended device statistics > device r/s w/s kr/s kw/s wait svc_t b > da0 43.2 204.4 971.9 10917.2 0 30.7 30 > extended device statistics > device r/s w/s kr/s kw/s wait svc_t b > da0 5.1 274.6 72.7 15206.2 0 50.4 27 > > If I'm not mistaken it's approx. 15Mb/sec. Which means more than 1000 Tb a > day! > > How can I find which program is trashing the disk? The system is > FreeBSD-7.0 with 15 jails. Thanks for your help. Something like this would be a good start: dtrace -n 'syscall::write:entry { @dist[pid,execname] = sum(arg2); }' Let it run a few seconds, then hit ^C, and it will print the total bytes written by each process during that period. You may also have to trace writev and pwrite, since FreeBSD's dtrace doesn't include the sysinfo provider that would let you count them all at once. Data written to swap or mmap'ped files won't register here, but most heavy writes aren't done with mmap. -- Dan Nelson dnelson@allantgroup.com