From owner-freebsd-bugs@FreeBSD.ORG Tue Apr 17 19:00:09 2007 Return-Path: X-Original-To: freebsd-bugs@hub.freebsd.org Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 44C3616A403 for ; Tue, 17 Apr 2007 19:00:09 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [69.147.83.40]) by mx1.freebsd.org (Postfix) with ESMTP id 2665B13C468 for ; Tue, 17 Apr 2007 19:00:09 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id l3HJ083k000342 for ; Tue, 17 Apr 2007 19:00:09 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id l3HJ08Fu000340; Tue, 17 Apr 2007 19:00:08 GMT (envelope-from gnats) Resent-Date: Tue, 17 Apr 2007 19:00:08 GMT Resent-Message-Id: <200704171900.l3HJ08Fu000340@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Steve Chan Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 47E7A16A416 for ; Tue, 17 Apr 2007 18:50:42 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from www.freebsd.org (www.freebsd.org [69.147.83.33]) by mx1.freebsd.org (Postfix) with ESMTP id 2F8B513C45D for ; Tue, 17 Apr 2007 18:50:42 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from www.freebsd.org (localhost [127.0.0.1]) by www.freebsd.org (8.13.1/8.13.1) with ESMTP id l3HIogGQ043037 for ; Tue, 17 Apr 2007 18:50:42 GMT (envelope-from nobody@www.freebsd.org) Received: (from nobody@localhost) by www.freebsd.org (8.13.1/8.13.1/Submit) id l3HIje87033364; Tue, 17 Apr 2007 18:45:40 GMT (envelope-from nobody) Message-Id: <200704171845.l3HIje87033364@www.freebsd.org> Date: Tue, 17 Apr 2007 18:45:40 GMT From: Steve Chan To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-3.0 Cc: Subject: misc/111782: /sbin/dump fails horribly for large filesystems X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Apr 2007 19:00:09 -0000 >Number: 111782 >Category: misc >Synopsis: /sbin/dump fails horribly for large filesystems >Confidential: no >Severity: critical >Priority: medium >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Tue Apr 17 19:00:08 GMT 2007 >Closed-Date: >Last-Modified: >Originator: Steve Chan >Release: 6.2-RELEASE-p3 >Organization: Lawrence Berkeley Laboratory >Environment: FreeBSD sigma.nersc.gov 6.2-RELEASE-p3 FreeBSD 6.2-RELEASE-p3 #0: Mon Mar 19 20:33:49 PDT 2007 scottc@sigma.nersc.gov:/usr/obj/usr/src/sys/GENERIC i386 >Description: The dump command misreports the dumpsize on large filesystems (seems to work for filesystems up to 283GB, but definitely fails for filesystems of 680GB), and generates bogus dumps for these large filesystems. Here is output from dump reporting on dump sizes for 2 filesystems, along with a df summary of those two filesystems: sigma# df -k /bro /log Filesystem 1K-blocks Used Avail Capacity Mounted on /dev/ad0s1h 14487150 280132 13048046 2% /bro /dev/da0g 680876920 43297592 583109176 7% /log sigma# /sbin/dump -0 -L -S /bro DUMP: Date of this level 0 dump: Tue Apr 17 11:29:19 2007 DUMP: Date of last level 0 dump: the epoch DUMP: Dumping snapshot of /dev/ad0s1h (/bro) to /dev/sa0 DUMP: mapping (Pass I) [regular files] DUMP: mapping (Pass II) [directories] DUMP: estimated 280959 tape blocks on 7.19 tape(s). sigma# /sbin/dump -0 -L -S /log DUMP: WARNING: -L requested but snapshot location /log/.snap DUMP: is not a directory: dump downgraded, -L ignored DUMP: Date of this level 0 dump: Tue Apr 17 11:29:39 2007 DUMP: Date of last level 0 dump: the epoch DUMP: Dumping /dev/da0g (/log) to /dev/sa0 DUMP: mapping (Pass I) [regular files] DUMP: mapping (Pass II) [directories] DUMP: estimated 633671689 tape blocks on 15278.33 tape(s). Doing a little math on the results based on the # of blocks that dump is reporting, and on the actual blocks used by df, it looks like dump is hugely overreporting the amount of space needed. Dump thinks the larger filesystem dump is over 2000x the size of the smaller one, while based on the output from df, it is only 150x larger: sigma# bc bc 1.06 Copyright 1991-1994, 1997, 1998, 2000 Free Software Foundation, Inc. This is free software with ABSOLUTELY NO WARRANTY. For details type `warranty'. scale = 2 633671689 / 280959 2255.38 43297592 / 280132 154.56 We see this problem whether we use the -L flag to dum or not. sigma# /sbin/dump -0 -L -S /log DUMP: WARNING: -L requested but snapshot location /log/.snap DUMP: is not a directory: dump downgraded, -L ignored DUMP: Date of this level 0 dump: Tue Apr 17 11:40:12 2007 DUMP: Date of last level 0 dump: the epoch DUMP: Dumping /dev/da0g (/log) to /dev/sa0 DUMP: mapping (Pass I) [regular files] DUMP: mapping (Pass II) [directories] DUMP: estimated 633671689 tape blocks on 15278.33 tape(s). sigma# /sbin/dump -0 -S /log DUMP: WARNING: should use -L when dumping live read-write filesystems! DUMP: Date of this level 0 dump: Tue Apr 17 11:40:47 2007 DUMP: Date of last level 0 dump: the epoch DUMP: Dumping /dev/da0g (/log) to /dev/sa0 DUMP: mapping (Pass I) [regular files] DUMP: mapping (Pass II) [directories] DUMP: estimated 633671689 tape blocks on 15278.33 tape(s). The backups that are actually generated by dump are way too big and cannot be used. We have several filesystems of the same size on this machine, and the problem occurs on all of them. Smaller filesystems backup fine. The filesystems are ufs, with soft-updates and local. At this point, we cannot do any backups of our larger filesystems using the dump command. We did not see this issue when we ran the same exact hardware under FreeBSD 4.10. >How-To-Repeat: Try running dump on a filesystem that is at least 680GB in capacity. >Fix: >Release-Note: >Audit-Trail: >Unformatted: