Date: Thu, 12 Mar 2015 09:37:00 -0700 From: Adrian Chadd <adrian@freebsd.org> To: Ryan Stone <rysto32@gmail.com> Cc: FreeBSD Current <freebsd-current@freebsd.org> Subject: Re: [PATCH] Convert the VFS cache lock to an rmlock Message-ID: <CAJ-VmoksXvb1ocC5JdxoP%2B9ki1d7K2ZvtvvOG5N1vRxGnU%2BQFg@mail.gmail.com> In-Reply-To: <CAFMmRNysnUezX9ozGrCpivPCTMYRJtoxm9ijR0yQO03LpXnwBQ@mail.gmail.com> References: <CAFMmRNysnUezX9ozGrCpivPCTMYRJtoxm9ijR0yQO03LpXnwBQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Do you have access to any boxes that have more than 12 cores? (like 36, 64, 80+ ?) -adrian On 12 March 2015 at 08:14, Ryan Stone <rysto32@gmail.com> wrote: > I've just submitted a patch to Differential[1] for review that converts the > VFS cache to use an rmlock in place of the current rwlock. My main > motivation for the change is to fix a priority inversion problem that I saw > recently. A real-time priority thread attempted to acquire a write lock on > the VFS cache lock, but there was already a reader holding it. The reader > was preempted by a normal priority thread, and my real-time thread was > starved. > > [1] https://reviews.freebsd.org/D2051 > > > I was worried about the performance implications of the change, as I wasn't > sure how common write operations on the VFS cache would be. I did a -j12 > buildworld/buildkernel test on a 12-core Haswell Xeon system, as I figured > that would be a reasonable stress test that simultaneously creates lots of > small files and reads a lot of files as well. This actually wound up being > about a 10% performance *increase* (the units below are seconds of elapsed > time as measured by /usr/bin/time, so smaller is better): > > $ ministat -C 1 orig.log rmlock.log > x orig.log > + rmlock.log > +------------------------------------------------------------------------------+ > | + x > | > |++++ x x xxx > | > | |A| > |_________A___M____|| > +------------------------------------------------------------------------------+ > N Min Max Median Avg Stddev > x 6 2710.31 2821.35 2816.75 2798.0617 43.324817 > + 5 2488.25 2500.25 2498.04 2495.756 5.0494782 > Difference at 95.0% confidence > -302.306 +/- 44.4709 > -10.8041% +/- 1.58935% > (Student's t, pooled s = 32.4674) > > The one outlier in the rwlock case does confuse me a bit. What I did was > booted a freshly-built image with the rmlock lock applied, did a git > checkout of head, and then did 5 builds in a row. The git checkout should > have had the effect of priming the disk cache with the source files. Then > I installed the stock head kernel, rebooted, and ran 5 more builds (and > then 1 more when I noticed the outlier). The fast outlier was the *first* > run, which should have been running with a cold disk cache, so I really > don't know why it would be 90 seconds faster. I do see that this run also > had about 500-600 fewer seconds spent in system time: > > x orig.log > +------------------------------------------------------------------------------+ > | > x | > |x x x > xx | > | > |_________________________A__________M_____________|| > +------------------------------------------------------------------------------+ > N Min Max Median Avg Stddev > x 6 3515.23 4121.84 4105.57 4001.71 239.61362 > > I'm not sure how much that I care, given that the rmlock is universally > faster (but maybe I should try the "cold boot" case anyway). > > If anybody had any comments or further testing that they would like to see, > please let me know. > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-VmoksXvb1ocC5JdxoP%2B9ki1d7K2ZvtvvOG5N1vRxGnU%2BQFg>