From owner-freebsd-fs@FreeBSD.ORG  Wed Dec 12 19:35:44 2012
Return-Path: <owner-freebsd-fs@FreeBSD.ORG>
Delivered-To: fs@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
 by hub.freebsd.org (Postfix) with ESMTP id 47038E52;
 Wed, 12 Dec 2012 19:35:44 +0000 (UTC) (envelope-from dim@FreeBSD.org)
Received: from tensor.andric.com (cl-327.ede-01.nl.sixxs.net
 [IPv6:2001:7b8:2ff:146::2])
 by mx1.freebsd.org (Postfix) with ESMTP id F17A48FC1B;
 Wed, 12 Dec 2012 19:35:43 +0000 (UTC)
Received: from [IPv6:2001:7b8:3a7:0:5821:52ef:5e7d:addd] (unknown
 [IPv6:2001:7b8:3a7:0:5821:52ef:5e7d:addd])
 (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits))
 (No client certificate requested)
 by tensor.andric.com (Postfix) with ESMTPSA id 6DBF45C37;
 Wed, 12 Dec 2012 20:35:42 +0100 (CET)
Message-ID: <50C8DC8E.1080204@FreeBSD.org>
Date: Wed, 12 Dec 2012 20:35:42 +0100
From: Dimitry Andric <dim@FreeBSD.org>
Organization: The FreeBSD Project
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64;
 rv:18.0) Gecko/20121128 Thunderbird/18.0
MIME-Version: 1.0
To: Volodymyr Kostyrko <c.kworr@gmail.com>
Subject: Re: clang compiled kernel panic when mounting zfs root on i386
References: <50b37d46.8584440a.735c.ffffb4e6@mx.google.com>
 <20121126171658.GD3013@kib.kiev.ua> <20121127071243.D1255@besplex.bde.org>
 <20121129232944.GQ3013@kib.kiev.ua>
 <50b8a9c5.e64dec0a.1d88.133a@mx.google.com>
 <20121130164715.GW3013@kib.kiev.ua>
 <50b9cf0c.0fd9650a.5bbf.ffffb9b3@mx.google.com>
 <20121203224132.GJ3013@kib.kiev.ua> <50C880D7.1040907@gmail.com>
In-Reply-To: <50C880D7.1040907@gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Cc: freebsd-current@freebsd.org, fs@freebsd.org
X-BeenThere: freebsd-fs@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: Filesystems <freebsd-fs.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-fs>,
 <mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs>
List-Post: <mailto:freebsd-fs@freebsd.org>
List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
 <mailto:freebsd-fs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 12 Dec 2012 19:35:44 -0000

On 2012-12-12 14:04, Volodymyr Kostyrko wrote:
> 04.12.2012 00:41, Konstantin Belousov:
>> Please try the patch below. It might give an immediate relief, but still
>> there are many offenders in the backtrace.
>
> I'm having almost the same issue and the patch doesn't work for me.
...

Looking at the stack frame addresses, it seems some of them are mangled.
Did you type this by hand?  The differences between subsequent frames
are a bit strange because of it (and because of awk's integer
processing):

_mtx_lock_flags                  40
uma_zalloc_arg                   80
malloc                           48
zfs_kmem_alloc                   20
vdev_mirror_io_start             100
zio_vdev_io_start                -2270966072
zio_execute                      2270966192
spa_load_verify_cb               64
traverse_visitbp                 304
traverse_dnode                   -2129031145
traverse_visitbp                 2129031529
traverse_visitbp                 805306672
traverse_visitbp                 -805306064
traverse_visitbp                 304
traverse_visitbp                 304
traverse_visitbp                 304
traverse_visitbp                 304
traverse_dnode                   80
traverse_visitbp                 304
traverse_impl                    176
traverse_pool                    160
spa_load                         280
spa_load                         280
spa_load_best                    -560
spa_open_common                  740
spa_open                         20
dsl_dir_open_spa                 360
dsl_dataset_hold                 124
dsl_dataset_own                  28
dmu_objset_own                   40
zfsvfs_create                    80
zfs_mount                        560
vfs_donmount                     624
kernel_mount                     64
parse_mount                      280
vfs_mountroot                    616
start_init                       108
fork_exit                        40
fork_trampoline                  0

The kernel stack is just 8,192 bytes; since you can see these routines
are all consuming massive amounts of stack, and the calls are very
deeply nested, it is almost inevitable that it would crash.

Especially the recursive spa_load and traverse_visitbp calls are scary,
because that can grow out of hand very quickly.  It is probably tricky
to remove the recursion...