From owner-freebsd-fs@FreeBSD.ORG Wed Apr 8 17:22:55 2015 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B5C7258A; Wed, 8 Apr 2015 17:22:55 +0000 (UTC) Received: from mail-pd0-x230.google.com (mail-pd0-x230.google.com [IPv6:2607:f8b0:400e:c02::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 789CBBD6; Wed, 8 Apr 2015 17:22:55 +0000 (UTC) Received: by pddn5 with SMTP id n5so122075108pdd.2; Wed, 08 Apr 2015 10:22:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=T68rGqi4L9K4jKHo1g20PvXd4bB5Mh/B/FgYnI7PDxM=; b=TjLoJW5GBjbW0rV0FSa2g8pzRG7n/2U5BT41VEU+sZHbwXmhGO7wxHRnAxxULA+96Y WvynTJ6H05hioskTmMop+6QT/He84Ho9ZkrCe4c9q2d0Y1pr9wWVskQBz18kgis7r8zh EukcpyF/j6QEuY4s9GIR8thzGv013SxFhTfgtzWXHd8YOd6vAqlM5vetP3EaPULli3Xb xIKW2JOYFnH+uP4i2Z8pzDeEft3qeaGAMvCeHDmN/bqfXw6EgCcFkIcmldR1AdLYFVlW YZBKFB3ihUE/O6CsM0RdKIzp7vyjFheoy7+xFSd2Uv1DkxSeEnuhlamdSUaqovkbSOik tfuQ== X-Received: by 10.68.102.228 with SMTP id fr4mr47217129pbb.87.1428513775143; Wed, 08 Apr 2015 10:22:55 -0700 (PDT) Received: from ?IPv6:2601:8:ab80:7d6:489a:2140:1bdb:1a5? ([2601:8:ab80:7d6:489a:2140:1bdb:1a5]) by mx.google.com with ESMTPSA id xt9sm11661003pbc.14.2015.04.08.10.22.53 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 08 Apr 2015 10:22:54 -0700 (PDT) Content-Type: multipart/signed; boundary="Apple-Mail=_25C7EA9E-02FC-4071-AD0E-4238DBE008B4"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: FreeBSD/ZFS on 9.3-RELEASE chews up memory with "wide" directories when calling readdir, etc; causes trap 12 panics From: Garrett Cooper In-Reply-To: Date: Wed, 8 Apr 2015 10:22:54 -0700 Message-Id: <81474997-069A-4DA5-9BF7-1EC427D7FFEF@gmail.com> References: To: Alan Somers X-Mailer: Apple Mail (2.1878.6) Cc: freebsd-fs , freebsd-stable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Apr 2015 17:22:55 -0000 --Apple-Mail=_25C7EA9E-02FC-4071-AD0E-4238DBE008B4 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Apr 8, 2015, at 9:15, Alan Somers wrote: > On Tue, Apr 7, 2015 at 12:37 AM, Garrett Cooper = wrote: >> Hi, >> Long story short, I had a lot of mail spooled up in = /var/spool. When I did ls /var/spool, ZFS chewed up almost all 12GB of = my memory in <10 mins (because there were enough files there) and the = system eventually panicked because [I assume that a memory allocation = failed and] a trap 12 panic was caught. I don=92t have the exact = details, but it should be relatively easy to repro (YMMV if you have a = boatload of RAM): >>=20 >> repro_end=3D10000000000 >> for i in $(seq 1 $repro_end); do mktemp tmp.XXXXXXXXXXXX; done >> ls >>=20 >> This might be ameliorated via r281026, but this change is only = available in CURRENT (so far), and I haven=92t tested it. >> Are there any comments about this scalability issue with = FreeBSD/ZFS? >> Thanks, >=20 > I spent the last ~ 24 hours creating 58,567,635 empty files in one > directory. I can ls it without crashing on a machine with 32 GB RAM. >=20 > # /usr/bin/time -l ls /tmp/tmp | wc > 1061.21 real 225.54 user 36.61 sys > 9720268 maximum resident set size > 28 average shared memory size > 8 average unshared data size > 128 average unshared stack size > 2425013 page reclaims > 0 page faults > 0 swaps > 108036 block input operations > 0 block output operations > 0 messages sent > 0 messages received > 0 signals received > 108004 voluntary context switches > 2428 involuntary context switches > 58567635 58567635 644243985 Strange. I wish I could gather more details about how many files were = located there. Distance and time that they were created might manner; in = my case there were _many_ spooled up emails that hadn=92t been sent = because I didn=92t take the time to fix my SMTP settings via = comcast/gmail. RAM amount might matter too. 12GB vs 32GB is a bit of a difference. Thanks! --Apple-Mail=_25C7EA9E-02FC-4071-AD0E-4238DBE008B4 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJVJWPuAAoJEMZr5QU6S73eH0sH/13o74/hVQlD48Owq1o7nutf Lvje62TD0mfhB8owDTUNUGJ/eWun7izjb0BkjVdW1tV6736HCQyzfq7HXLj7yuZH WSzc8fHVI+Oh0E8TOSWn8tN+Th6s1lgab4Fd8Dc/CwAsI7Ai9Q29hM/0sJMjXK9r p94OtaoNO8U9wNbpwrY9DNIoVU6KOr7GsYI9WQw8RqbTYXp3YAmY/k5BmdbOWPiZ ntBYEKg0TwkFvHyiDdQqTArLI0SqmfDSPWCMkztm1JedjT1lM3fk1Vj1RqWisbvb /OAYj5sTjHryqQhvm1LuKuUbxFhWPBcv5i3wAj+tIymAjuKLERiJ24FWLpRB7TU= =Q5id -----END PGP SIGNATURE----- --Apple-Mail=_25C7EA9E-02FC-4071-AD0E-4238DBE008B4--