From owner-freebsd-current@FreeBSD.ORG Wed Apr 25 18:24:08 2007 Return-Path: X-Original-To: freebsd-current@freebsd.org Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 878BC16A481 for ; Wed, 25 Apr 2007 18:24:08 +0000 (UTC) (envelope-from grafan@gmail.com) Received: from qb-out-0506.google.com (qb-out-0506.google.com [72.14.204.224]) by mx1.freebsd.org (Postfix) with ESMTP id 4848E13C458 for ; Wed, 25 Apr 2007 18:24:08 +0000 (UTC) (envelope-from grafan@gmail.com) Received: by qb-out-0506.google.com with SMTP id e11so6649855qba for ; Wed, 25 Apr 2007 11:24:07 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=AzvZ/N64OC71P7L4nZDSYce/z07SvzUTSIV+VjU8JkbxB9n1FCd4w+UP1cjGdZq47ttF5SdV9CWGoLmbvq9ehFHhzlmuoLyApyLahvQf7U6dMfU0wzrdwBpo0yLij+PebPJbbsoRNcadBvGdVjiMJD19dj0bPt1O8/LOaAIvjqY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=KWeSmSlF+aM3ismRLxWTPAxiBd+79ZrAf6p6+fl0cWl/C2vjOR4EH8Agxy6g5UlqJnyMJIJPo1S82YuIsAU8pelTgc2HYaAoUoEf7gIIWPlWE0dz7y+WE3LaopGX+8DYccVrbDUgty4b38kSaLBrv7ewgic8kMsL8gLb8+nswgM= Received: by 10.65.252.13 with SMTP id e13mr2128157qbs.1177525447501; Wed, 25 Apr 2007 11:24:07 -0700 (PDT) Received: by 10.64.185.12 with HTTP; Wed, 25 Apr 2007 11:24:07 -0700 (PDT) Message-ID: <6eb82e0704251124o53b7bc1aq9836a20ee06fcd11@mail.gmail.com> Date: Thu, 26 Apr 2007 02:24:07 +0800 From: "Rong-en Fan" To: "FreeBSD Current" , pjd@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Cc: Subject: panic: Journal overflow X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2007 18:24:08 -0000 This is a i386 current SMP box as of Apr 14 or 15. Got a panic with geom journal. panic: Journal overflow (joffset=3246704015360 active=3246705852416 inactive=324 cpuid = 2 KDB: stack backtrace: db_trace_self_wrapper(c06b3e2c,e5321944,c04fc86e,c06c70c8,2,...) at db_trace_sel kdb_backtrace(c06c70c8,2,c06ad0d2,e5321950,5,...) at kdb_backtrace+0x2f panic(c06ad0d2,eea3b800,2f3,eebfc000,2f3,...) at panic+0x11f g_journal_check_overflow(c4fc5c00,cb030a00,eb,95428000,eb,...) at g_journal_chec g_journal_flush(c4fc5c00,0,eb,95428000,eb,...) at g_journal_flush+0x60d g_journal_add_current(c4fc5c00,c9cb0948,ca8e818c,c4fc5c00,e5321cbc,...) at g_jou g_journal_release_delayed(c4fc5c00,0,ca8e818c,c4fdadc0,2,...) at g_journal_relea g_journal_flush_send(c4fc5c00,c8521c60,205d0000,1b2,205d4000,...) at g_journal_f g_journal_worker(c4fc5c00,e5321d38,0,0,0,...) at g_journal_worker+0x7f7 fork_exit(c04b5813,c4fc5c00,e5321d38) at fork_exit+0x83 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe5321d70, ebp = 0 --- Sorry, that I don't have core dump available. Will set up next time. After the panic, the system hangs at single user prompt: Trying to mount root from ufs:/dev/da0s1a WARNING: / was not properly dismounted Loading configuration files. kernel dumps on /dev/da0s1b Entropy harvesting: interrupts ethernet point_to_point kickstart. swapon: adding /dev/da0s1b as swap device Starting file system checks: /dev/da0s1a: 1704 files, 56880 used, 196935 free (631 frags, 24538 blocks, 0.2% Can't stat /dev/concat/data.journal: No such file or directory /dev/da0s1f: 237 files, 1208 used, 11877856 free (64 frags, 1484724 blocks, 0.0% GEOM_JOURNAL: Journal concat/data consistent. /dev/da0s1e: UNREF FILE I=548536 OWNER=root MODE=100644 /dev/da0s1e: SIZE=717466 MTIME=Apr 4 07:58 2007 (CLEARED) /dev/da0s1e: FREE BLK COUNT(S) WRONG IN SUPERBLK (SALVAGED) /dev/da0s1e: SUMMARY INFORMATION BAD (SALVAGED) /dev/da0s1e: BLK(S) MISSING IN BIT MAPS (SALVAGED) /dev/da0s1e: 190105 files, 763850 used, 2281197 free (36557 frags, 280580 blocks /dev/da0s1d: 17507 files, 40158 used, 972857 free (641 frags, 121527 blocks, 0.1 THE FOLLOWING FILE SYSTEM HAD AN UNEXPECTED INCONSISTENCY: ufs: /dev/concat/data.journal (/data) Unknown error; help! AEnter full pathname of shell or RETURN for /bin/sh: # mount -a WARNING: R/W mount of /backup denied. Filesystem is not clean - run fsck mount: /dev/concat/data.journal : Operation not permitted # fsck_ffs -p /dev/concat/data.journal I need to issue 'fsck_ffs -p' myself... any idea about why this happens? The geom journal setup: Geom name: gjournal 68372861 ID: 68372861 Providers: 1. Name: concat/data.journal Mediasize: 3246137539072 (3.0T) Sectorsize: 512 Mode: r1w1e1 Consumers: 1. Name: concat/data Mediasize: 3247211281408 (3.0T) Sectorsize: 512 Mode: r1w1e1 Jend: 3247211280896 Jstart: 3246137539072 Role: Data,Journal The gconcat consists two scsi disk (actually, it's raid) da0 and da1. Oh no, it panics with journal overflow again while writing this message :( The data.journal is shared by nfs, and there are two boxes that are doing a tar writing operation on this partition. Regards, Rong-En Fan