From owner-freebsd-current@FreeBSD.ORG Thu Nov 24 20:29:50 2011 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C74C0106564A; Thu, 24 Nov 2011 20:29:50 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 632B48FC14; Thu, 24 Nov 2011 20:29:49 +0000 (UTC) Received: from alf.home (alf.kiev.zoral.com.ua [10.1.1.177]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id pAOKTjuu077996 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 24 Nov 2011 22:29:45 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from alf.home (kostik@localhost [127.0.0.1]) by alf.home (8.14.5/8.14.5) with ESMTP id pAOKTjd3049933; Thu, 24 Nov 2011 22:29:45 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by alf.home (8.14.5/8.14.5/Submit) id pAOKTjpX049932; Thu, 24 Nov 2011 22:29:45 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: alf.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 24 Nov 2011 22:29:45 +0200 From: Kostik Belousov To: Maxim Konovalov Message-ID: <20111124202945.GR50300@deviant.kiev.zoral.com.ua> References: <20111122124410.GP50300@deviant.kiev.zoral.com.ua> <20111122154357.GI95664@mdounin.ru> <20111122154935.GR50300@deviant.kiev.zoral.com.ua> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="865IE93Xpm46K7pT" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: arch@freebsd.org, current@freebsd.org, igor@sysoev.ru, Maxim Dounin Subject: Re: RLIMIT_DATA and malloc(3) use of mmap(2) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Nov 2011 20:29:51 -0000 --865IE93Xpm46K7pT Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Nov 24, 2011 at 09:38:19PM +0400, Maxim Konovalov wrote: > Hi Kostik, >=20 > On Wed, 23 Nov 2011, 11:22+0400, Maxim Konovalov wrote: >=20 > > [...] > > > Anyway, the patch needs testers before I will push it forward. > > > > > [igor's email was corrected] > > > > We will test it in out environment and let you know. > > > It seems I don't understand how it works. >=20 > Here is a test program: http://maxim.int.ru/stuff/malloc_test.c.txt >=20 > It allocates successfully 64x1MB chunks by malloc(), reallocf() and > realloc() with the following command line: >=20 > MALLOC_OPTIONS=3DJM limits -d 10m ./malloc_test >=20 > When we add L flag to the MALLOC_OPTIONS it starts to work strange: >=20 > It allocates 64MB via malloc() but fails to allocate more than 3MB by > realloc() and reallocf(). The non-failing 64MB allocation is easily explained by a bug. I forgot to account for the case when existing map entry was expanded, instead of new entry created due to mmap(2). The other part, in particular, the failure after 3MB, is in fact the correct behaviour. Jemalloc() caches allocation of the pages, and it allocates more then asked in the request. ktrace(1) shows the whole history. Malloc() first allocated 4MB for the needs of libc etc. Then, it allocated 4MB chunk which was used for satisfying the requests of 1-3MB. When the 4MB request came in, the allocation for 8MB was attempted, and failed, since 4MB (libc etc) + 8MB =3D 12MB > data limit. >=20 > More funny, the result varies from time to time: I have no explanation for this, and cannot reproduce the issue. Look at the ktrace. Overall, the test is quite curious but absolutely unrealistic. Fixed patch is available at http://people.freebsd.org/~kib/misc/map_datalimit.2.patch --865IE93Xpm46K7pT Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk7OqTkACgkQC3+MBN1Mb4hVFACginFQ/syGlHkbuL2ZtY20QdAX mqgAoKgirRQKnX1phG9DhOMUwQHm/B+C =5PsP -----END PGP SIGNATURE----- --865IE93Xpm46K7pT--