Date: Wed, 8 Apr 2015 10:22:54 -0700 From: Garrett Cooper <yaneurabeya@gmail.com> To: Alan Somers <asomers@freebsd.org> Cc: freebsd-fs <freebsd-fs@freebsd.org>, freebsd-stable <freebsd-stable@freebsd.org> Subject: Re: FreeBSD/ZFS on 9.3-RELEASE chews up memory with "wide" directories when calling readdir, etc; causes trap 12 panics Message-ID: <81474997-069A-4DA5-9BF7-1EC427D7FFEF@gmail.com> In-Reply-To: <CAOtMX2gyc2_s=2ovhmYBNi0J9jstgAsWoJDu5KDaFwR-%2BEHxxg@mail.gmail.com> References: <F390CDB8-024C-4F6F-8793-752756859B5C@gmail.com> <CAOtMX2gyc2_s=2ovhmYBNi0J9jstgAsWoJDu5KDaFwR-%2BEHxxg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--Apple-Mail=_25C7EA9E-02FC-4071-AD0E-4238DBE008B4 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Apr 8, 2015, at 9:15, Alan Somers <asomers@freebsd.org> wrote: > On Tue, Apr 7, 2015 at 12:37 AM, Garrett Cooper = <yaneurabeya@gmail.com> wrote: >> Hi, >> Long story short, I had a lot of mail spooled up in = /var/spool. When I did ls /var/spool, ZFS chewed up almost all 12GB of = my memory in <10 mins (because there were enough files there) and the = system eventually panicked because [I assume that a memory allocation = failed and] a trap 12 panic was caught. I don=92t have the exact = details, but it should be relatively easy to repro (YMMV if you have a = boatload of RAM): >>=20 >> repro_end=3D10000000000 >> for i in $(seq 1 $repro_end); do mktemp tmp.XXXXXXXXXXXX; done >> ls >>=20 >> This might be ameliorated via r281026, but this change is only = available in CURRENT (so far), and I haven=92t tested it. >> Are there any comments about this scalability issue with = FreeBSD/ZFS? >> Thanks, >=20 > I spent the last ~ 24 hours creating 58,567,635 empty files in one > directory. I can ls it without crashing on a machine with 32 GB RAM. >=20 > # /usr/bin/time -l ls /tmp/tmp | wc > 1061.21 real 225.54 user 36.61 sys > 9720268 maximum resident set size > 28 average shared memory size > 8 average unshared data size > 128 average unshared stack size > 2425013 page reclaims > 0 page faults > 0 swaps > 108036 block input operations > 0 block output operations > 0 messages sent > 0 messages received > 0 signals received > 108004 voluntary context switches > 2428 involuntary context switches > 58567635 58567635 644243985 Strange. I wish I could gather more details about how many files were = located there. Distance and time that they were created might manner; in = my case there were _many_ spooled up emails that hadn=92t been sent = because I didn=92t take the time to fix my SMTP settings via = comcast/gmail. RAM amount might matter too. 12GB vs 32GB is a bit of a difference. Thanks! --Apple-Mail=_25C7EA9E-02FC-4071-AD0E-4238DBE008B4 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJVJWPuAAoJEMZr5QU6S73eH0sH/13o74/hVQlD48Owq1o7nutf Lvje62TD0mfhB8owDTUNUGJ/eWun7izjb0BkjVdW1tV6736HCQyzfq7HXLj7yuZH WSzc8fHVI+Oh0E8TOSWn8tN+Th6s1lgab4Fd8Dc/CwAsI7Ai9Q29hM/0sJMjXK9r p94OtaoNO8U9wNbpwrY9DNIoVU6KOr7GsYI9WQw8RqbTYXp3YAmY/k5BmdbOWPiZ ntBYEKg0TwkFvHyiDdQqTArLI0SqmfDSPWCMkztm1JedjT1lM3fk1Vj1RqWisbvb /OAYj5sTjHryqQhvm1LuKuUbxFhWPBcv5i3wAj+tIymAjuKLERiJ24FWLpRB7TU= =Q5id -----END PGP SIGNATURE----- --Apple-Mail=_25C7EA9E-02FC-4071-AD0E-4238DBE008B4--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?81474997-069A-4DA5-9BF7-1EC427D7FFEF>