Date: Wed, 8 Apr 2015 10:15:01 -0600 From: Alan Somers <asomers@freebsd.org> To: Garrett Cooper <yaneurabeya@gmail.com> Cc: freebsd-fs <freebsd-fs@freebsd.org>, freebsd-stable <freebsd-stable@freebsd.org> Subject: Re: FreeBSD/ZFS on 9.3-RELEASE chews up memory with "wide" directories when calling readdir, etc; causes trap 12 panics Message-ID: <CAOtMX2gyc2_s=2ovhmYBNi0J9jstgAsWoJDu5KDaFwR-%2BEHxxg@mail.gmail.com> In-Reply-To: <F390CDB8-024C-4F6F-8793-752756859B5C@gmail.com> References: <F390CDB8-024C-4F6F-8793-752756859B5C@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Apr 7, 2015 at 12:37 AM, Garrett Cooper <yaneurabeya@gmail.com> wro=
te:
> Hi,
> Long story short, I had a lot of mail spooled up in /var/spool. W=
hen I did ls /var/spool, ZFS chewed up almost all 12GB of my memory in <10 =
mins (because there were enough files there) and the system eventually pani=
cked because [I assume that a memory allocation failed and] a trap 12 panic=
was caught. I don=E2=80=99t have the exact details, but it should be relat=
ively easy to repro (YMMV if you have a boatload of RAM):
>
> repro_end=3D10000000000
> for i in $(seq 1 $repro_end); do mktemp tmp.XXXXXXXXXXXX; done
> ls
>
> This might be ameliorated via r281026, but this change is only av=
ailable in CURRENT (so far), and I haven=E2=80=99t tested it.
> Are there any comments about this scalability issue with FreeBSD/=
ZFS?
> Thanks,
I spent the last ~ 24 hours creating 58,567,635 empty files in one
directory. I can ls it without crashing on a machine with 32 GB RAM.
# /usr/bin/time -l ls /tmp/tmp | wc
1061.21 real 225.54 user 36.61 sys
9720268 maximum resident set size
28 average shared memory size
8 average unshared data size
128 average unshared stack size
2425013 page reclaims
0 page faults
0 swaps
108036 block input operations
0 block output operations
0 messages sent
0 messages received
0 signals received
108004 voluntary context switches
2428 involuntary context switches
58567635 58567635 644243985
-Alan
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOtMX2gyc2_s=2ovhmYBNi0J9jstgAsWoJDu5KDaFwR-%2BEHxxg>