From owner-freebsd-stable@FreeBSD.ORG Tue Nov 18 09:20:52 2008 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 08E4E106568B; Tue, 18 Nov 2008 09:20:51 +0000 (UTC) (envelope-from quakelee@geekcn.org) Received: from tarsier.delphij.net (delphij-pt.tunnel.tserv2.fmt.ipv6.he.net [IPv6:2001:470:1f03:2c9::2]) by mx1.freebsd.org (Postfix) with ESMTP id A76FB8FC0C; Tue, 18 Nov 2008 09:20:50 +0000 (UTC) (envelope-from quakelee@geekcn.org) Received: from tarsier.geekcn.org (tarsier.geekcn.org [211.166.10.233]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.delphij.net (Postfix) with ESMTPS id CA3A12844E; Tue, 18 Nov 2008 17:20:49 +0800 (CST) Received: from localhost (tarsier.geekcn.org [211.166.10.233]) by tarsier.geekcn.org (Postfix) with ESMTP id 82555EC68C9; Tue, 18 Nov 2008 17:20:49 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([211.166.10.233]) by localhost (mail.geekcn.org [211.166.10.233]) (amavisd-new, port 10024) with ESMTP id Zk37FIA3ashY; Tue, 18 Nov 2008 17:20:44 +0800 (CST) Received: from qld630 (unknown [222.131.116.4]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id 20C04EC68C6; Tue, 18 Nov 2008 17:20:44 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=geekcn.org; c=nofws; q=dns; h=date:to:subject:from:organization:cc:content-type: mime-version:references:content-transfer-encoding:message-id:in-reply-to:user-agent; b=NACDwVvDzZm7E8sfCXFq2rUcHprEa0JR6x2gPQ/pAd5VoZ4a3HX+0B1VNd3vX5EG8 vsxIFBjMo7sgaDauNUY9A== Date: Tue, 18 Nov 2008 17:20:43 +0800 To: "Pawel Jakub Dawidek" , d@delphij.net From: "Chao Shin" Organization: GeekCN Content-Type: text/plain; format=flowed; delsp=yes; charset=utf-8 MIME-Version: 1.0 References: <491CE71F.2020208@delphij.net> <491CE835.4050504@delphij.net> <20081117155835.GC2101@garage.freebsd.pl> Content-Transfer-Encoding: 8bit Message-ID: In-Reply-To: <20081117155835.GC2101@garage.freebsd.pl> User-Agent: Opera Mail/9.62 (Win32) Cc: FreeBSD Stable Subject: Re: ZFS crashes on heavy threaded environment X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Nov 2008 09:20:52 -0000 On Mon, 17 Nov 2008 23:58:35 +0800，Pawel Jakub Dawidek wrote: > On Thu, Nov 13, 2008 at 06:53:41PM -0800, Xin LI wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> Xin LI wrote: >> > Hi, Pawel, >> > >> > We can still reproduce the ZFS crash (threading+heavy I/O load) on a >> > fresh 7.1-STABLE build, in a few minutes: >> > >> > /usr/local/bin/iozone -M -e -+u -T -t 128 -S 4096 -L 64 -r 4k -s 30g >> -i >> > 0 -i 1 -i 2 -i 8 -+p 70 -C >> > >> > I have included a backtrace output from my colleague who has his hands >> > on the test environment. Should there is more information necessary >> > please let us know and we wish to provide help on this. >> >> Further datapoint. The system used to run with untuned loader.conf, and >> my colleague just reported that with the following loader.conf, the >> problem can be triggered sooner: >> >> vm.kmem_size_max=838860800 >> vm.kmem_size_scale="2" >> >> The system is running FreeBSD/amd64 7.1-PRERELEASE equipped with 8GB of >> RAM with GENERIC kernel. > > With new ZFS I get: > > Memory allocation failed:: Cannot allocate memory > > Is this expected? > At first, Congratulations to you, thanks to your works, well done! I used this command on a FreeBSD 7.1-PRERELEASE amd64 box with 8GB mem, isn't got output like that, but kernel panic. Maybe you should lower the threads and file size, for example: /usr/local/bin/iozone -M -e -+u -T -t 64 -S 4096 -L 64 -r 4k -s 2g -i 0 -i 1 -i 2 -i 8 -+p 70 -C Actually, we had used this command to test a 8-current with zfs v12 patch on July, there is no more panic. So we hope zfs v13 can MFC as soon as possible, because we really need it now. -- The Power to Serve