From owner-freebsd-arm@freebsd.org Tue Sep 1 10:38:48 2015 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F368D9C8153 for ; Tue, 1 Sep 2015 10:38:48 +0000 (UTC) (envelope-from andrew@fubar.geek.nz) Received: from kif.fubar.geek.nz (kif.fubar.geek.nz [178.62.119.249]) by mx1.freebsd.org (Postfix) with ESMTP id C241911EB for ; Tue, 1 Sep 2015 10:38:48 +0000 (UTC) (envelope-from andrew@fubar.geek.nz) Received: from bender (host81-149-102-120.in-addr.btopenworld.com [81.149.102.120]) by kif.fubar.geek.nz (Postfix) with ESMTPSA id 567C0D7FAC; Tue, 1 Sep 2015 10:38:41 +0000 (UTC) Date: Tue, 1 Sep 2015 11:38:38 +0100 From: Andrew Turner To: Julien Grall Cc: freebsd-arm Subject: Re: arm64: userspace broken with jemalloc 4.0.0 Message-ID: <20150901113838.02347efd@bender> In-Reply-To: <55E552CC.9080206@citrix.com> References: <55E22CC0.9000306@citrix.com> <20150831220647.67a4646d@bender.Home> <55E552CC.9080206@citrix.com> X-Mailer: Claws Mail 3.12.0 (GTK+ 2.24.28; amd64-portbld-freebsd10.1) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Sep 2015 10:38:49 -0000 On Tue, 1 Sep 2015 08:25:00 +0100 Julien Grall wrote: > Hi Andrew, > > On 31/08/2015 22:06, Andrew Turner wrote: > > On Sat, 29 Aug 2015 23:05:52 +0100 > > Julien Grall wrote: > >> I've built the latest freebsd master (r287263) for arm64 today. > >> While trying to use the userspace I hit some ASSERT in jemalloc: > >> > >> # ls > >> : /usr/src/freebsd/lib/libc/../../contrib/jemalloc/include/jemalloc/internal/arena.h:571: > >> Failed assertion: "pageind >= map_bias" pid 21 (ls), uid 0: exited > >> on signal 6 Abort trap > >> > >> It's happening every time with the command "ls". > >> > >> I tried to use the previous version of jemalloc (i.e reverting > >> all the patches up to "Update jemalloc to version 4.0.0" included) > >> and everything is working. > >> > >> Note that I'm using Freebsd as a Xen ARM guest although the only > >> difference is the version of jemalloc (4.0.0 vs 3.6.0). > >> > >> Does anyone using arm64 have seen a similar ASSERT? > >> > >> BTW, is there any way to rebuild only the libc rather than doing > >> make buildworld everytime I modified the jemalloc code? > >> > >> Regards, > >> > > > > This is a bug in the runtime linkers handling of tls. The attached > > patch allows me to get to multiuser mode without anything hitting > > the above assert. > > I ended up to a similar patch during the week-end (see below). > Although I was looking to the amd64/i386 definition of > calculate_tls_offset which is the same as ARM64. I didn't understand > why it's working for this architecture but not for ours. > > Is there any possible bug in the amd64/i386 runtime too? amd64 and i386 use a different tls variant. They place the thread pointer at end of the buffer and allocate data before it, where arm64 places it at the start and allocates space after this. I suspect it was previously working by chance. There is a bug with all Variant I tls architectures (arm, arm64, mips, and powerpc) where the thread pointer may be trashed. I expect to fix this soon, and have a more complete fix committed soon after this. (For those interested in the layout of the tls data there is a paper on it at [1], with useful figures on pages 5 and 6). Andrew [1] http://www.akkadia.org/drepper/tls.pdf