Date: Tue, 22 Feb 2011 18:47:59 -0800 From: Garrett Cooper <yanegomi@gmail.com> To: Alexander Best <arundel@freebsd.org> Cc: Brandon Gooch <jamesbrandongooch@gmail.com>, Eir Nym <eirnym@gmail.com>, Dimitry Andric <dim@freebsd.org>, FreeBSD Current <freebsd-current@freebsd.org> Subject: Re: Wow... (<-- blown away at performance) Message-ID: <AANLkTi=dvFbKoVkXzZn8%2BdRJB7-mR3m=6SDAu2G1TTKC@mail.gmail.com> In-Reply-To: <20110223020008.GA94832@freebsd.org> References: <AANLkTi=TM8qfSZLmmX_tFmdhd6D3w-=tjZ99kKK_Cw4K@mail.gmail.com> <AANLkTi=CoOe1WOe=-geVZtPwgJpBpE0vg88QG5aiXYRT@mail.gmail.com> <AANLkTik_C9Bsbio62GDaVcs93sWiN82yS9UbhvajzyBk@mail.gmail.com> <20110222203134.GA53262@freebsd.org> <AANLkTinBMr=U6v1er9iFEY25Y_%2BkFZYsCZZWh1wnKHaq@mail.gmail.com> <20110222215036.GA66303@freebsd.org> <20110222223314.GA72748@freebsd.org> <20110223020008.GA94832@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Feb 22, 2011 at 6:00 PM, Alexander Best <arundel@freebsd.org> wrote=
:
> On Tue Feb 22 11, Alexander Best wrote:
>> On Tue Feb 22 11, Alexander Best wrote:
>> > On Tue Feb 22 11, Brandon Gooch wrote:
>> > > On Tue, Feb 22, 2011 at 2:31 PM, Alexander Best <arundel@freebsd.org=
> wrote:
>> > > > On Tue Feb 22 11, Garrett Cooper wrote:
>> > > >> On Tue, Feb 22, 2011 at 2:10 AM, Eir Nym <eirnym@gmail.com> wrote=
:
>> > > >> > On 22 February 2011 11:15, Garrett Cooper <yanegomi@gmail.com> =
wrote:
>> > > >> >> =A0 =A0I don't know what to say, but r218938 screams with flas=
h videos
>> > > >> >> (native Linux speed). Not sure if it's the new binutils or if =
it's the
>> > > >> >> new linuxulator patches, but I can run multiple instances of y=
outube
>> > > >> >> in parallel (5 total with other miscellaneous flash animation)=
without
>> > > >> >> it totally lagging out Firefox/X11, and it appears to close th=
e
>> > > >> >> instances of firefox properly now. Hopefully this version fare=
s better
>> > > >> >> than r218113 did (I think I hit a kernel bug after 2 weeks upt=
ime,
>> > > >> >> where my system just hardlocked for no apparent reason).
>> > > >> >> =A0 =A0Anyhow, hope others have similar results.
>> > > >> >> Cheers!
>> > > >> >> -Garrett
>> > > >> >>
>> > > >> >> $ uname -a
>> > > >> >> FreeBSD bayonetta.local 9.0-CURRENT FreeBSD 9.0-CURRENT #0 r21=
8938M:
>> > > >> >> Mon Feb 21 23:10:51 PST 2011
>> > > >> >> gcooper@bayonetta.local:/usr/obj/usr/src/sys/BAYONETTA =A0amd6=
4
>> > > >> >
>> > > >> > Which FlashPlayer version do you test? Adobe has made significa=
nt
>> > > >> > performance changes in 10.2 (from 10.1). You can search for Sta=
geVideo
>> > > >> > performance to learn more about. Youtube already use them since=
10.2
>> > > >> > beta
>> > > >>
>> > > >> =A0 =A0 linux-f10-flashplugin-10.1r102.65 . The performance incre=
ases are
>> > > >> claimed to be "up to 85%" on the Stage Video site, but I'm seeing=
a
>> > > >> more than 200% increase (now it actually scales between multiple
>> > > >> instances, instead of croaks with one instance, tiling up and dow=
n the
>> > > >> screen when moving the window slider for instance or switching ta=
bs).
>> > > >> Besides, it seems like it needs external support from the video
>> > > >> driver, and I'm not sure that that bridge exists in the linuxulat=
or.
>> > > >> =A0 =A0 Also, it looks like npviewer.bin still hangs to resources=
on until
>> > > >> Firefox closes (or I kill it :)..), so something still needs to b=
e
>> > > >> resolved there, but that isn't a regression (it's acted that way =
for
>> > > >> ages), and shouldn't be too hard to do.
>> > > >
>> > > > i think the problem with npviewer.bin having to be killed by hand =
is a futex
>> > > > issue in combination with a high number of threads. this is the ou=
tput of a
>> > > > test from darren hart's futex test suite under freebsd 9.0:
>> > > >
>> > > > futex_wait: Measure FUTEX_WAIT operations per second
>> > > > =A0 =A0 =A0 =A0Arguments: iterations=3D100000000 threads=3D1
>> > > > Result: 18622 Kiter/s
>> > > > futex_wait: Measure FUTEX_WAIT operations per second
>> > > > =A0 =A0 =A0 =A0Arguments: iterations=3D100000000 threads=3D2
>> > > > Result: 15469 Kiter/s
>> > > > futex_wait: Measure FUTEX_WAIT operations per second
>> > > > =A0 =A0 =A0 =A0Arguments: iterations=3D100000000 threads=3D3
>> > > > Result: 12713 Kiter/s
>> > > > futex_wait: Measure FUTEX_WAIT operations per second
>> > > > =A0 =A0 =A0 =A0Arguments: iterations=3D100000000 threads=3D4
>> > > > Result: 12247 Kiter/s
>> > > > futex_wait: Measure FUTEX_WAIT operations per second
>> > > > =A0 =A0 =A0 =A0Arguments: iterations=3D100000000 threads=3D5
>> > > > Result: 11814 Kiter/s
>> > > > futex_wait: Measure FUTEX_WAIT operations per second
>> > > > =A0 =A0 =A0 =A0Arguments: iterations=3D100000000 threads=3D6
>> > > > Result: 13468 Kiter/s
>> > > > futex_wait: Measure FUTEX_WAIT operations per second
>> > > > =A0 =A0 =A0 =A0Arguments: iterations=3D100000000 threads=3D8
>> > > > Result: 12061 Kiter/s
>> > > > futex_wait: Measure FUTEX_WAIT operations per second
>> > > > =A0 =A0 =A0 =A0Arguments: iterations=3D100000000 threads=3D10
>> > > > Result: 12854 Kiter/s
>> > > > futex_wait: Measure FUTEX_WAIT operations per second
>> > > > =A0 =A0 =A0 =A0Arguments: iterations=3D100000000 threads=3D12
>> > > > Result: 12999 Kiter/s
>> > > > futex_wait: Measure FUTEX_WAIT operations per second
>> > > > =A0 =A0 =A0 =A0Arguments: iterations=3D100000000 threads=3D16
>> > > > Result: 12402 Kiter/s
>> > > > futex_wait: Measure FUTEX_WAIT operations per second
>> > > > =A0 =A0 =A0 =A0Arguments: iterations=3D100000000 threads=3D24
>> > > > Result: 9815 Kiter/s
>> > > > futex_wait: Measure FUTEX_WAIT operations per second
>> > > > =A0 =A0 =A0 =A0Arguments: iterations=3D100000000 threads=3D32
>> > > > Result: 8518 Kiter/s
>> > > > futex_wait: Measure FUTEX_WAIT operations per second
>> > > > =A0 =A0 =A0 =A0Arguments: iterations=3D100000000 threads=3D64
>> > > > =A0 =A0 =A0 =A0ERROR: Resource temporarily unavailable: pthread_cr=
eate
>> > > > Result: ERROR
>> > > > futex_wait: Measure FUTEX_WAIT operations per second
>> > > > =A0 =A0 =A0 =A0Arguments: iterations=3D100000000 threads=3D128
>> > > > =A0 =A0 =A0 =A0ERROR: Resource temporarily unavailable: pthread_cr=
eate
>> > > > Result: ERROR
>> > > > futex_wait: Measure FUTEX_WAIT operations per second
>> > > > =A0 =A0 =A0 =A0Arguments: iterations=3D100000000 threads=3D256
>> > > > =A0 =A0 =A0 =A0ERROR: Resource temporarily unavailable: pthread_cr=
eate
>> > > > Result: ERROR
>> > > > futex_wait: Measure FUTEX_WAIT operations per second
>> > > > =A0 =A0 =A0 =A0Arguments: iterations=3D100000000 threads=3D512
>> > > > =A0 =A0 =A0 =A0ERROR: Resource temporarily unavailable: pthread_cr=
eate
>> > > > Result: ERROR
>> > > > futex_wait: Measure FUTEX_WAIT operations per second
>> > > > =A0 =A0 =A0 =A0Arguments: iterations=3D100000000 threads=3D1024
>> > > > =A0 =A0 =A0 =A0ERROR: Resource temporarily unavailable: pthread_cr=
eate
>> > > > Result: ERROR
>> > > >
>> > > > cheers.
>> > > > alex
>> > >
>> > > Have you found any method or workaround to mitigate this issue?
>>
>> i did a ktrace -i and linux_kdump run and the results are quite interest=
ing.
>> with only 1 thread linux_sys_futex() *never* failed with EAGAIN. startin=
g
>> with 2 threads however i'm seeing an increasing number of linux_sys_fute=
x()
>> failures (all with EAGAIN).
>>
>> the overhead is quite massive. linux_kdump > /tmp/futex-failure produced=
941
>> megs of output. doing
>> grep -v "futex_wait" futex-failure > futex-grep
>> produces an output of 192K!! so it's quite obvious that the futex_wait s=
tuff
>> is responsible for causing massive activity, due to the constant failure=
s and
>> retries.
>
> i ran kdump, instead of linux_kdump and it seems the freebsd function ret=
urning
> EAGAIN is nanosleep(2):
>
> 929 futex_wait CALL =A0nanosleep(0xffffdb5c,0x81,0x1,0,0,0)
> =A09929 futex_wait RET =A0 nanosleep 0
> =A09929 futex_wait CALL =A0nanosleep(0xffffdb5c,0x81,0x1,0,0,0)
> =A09928 futex_wait CALL =A0nanosleep(0xffffdb5c,0x80,0x2,0,0,0)
> =A09929 futex_wait RET =A0 nanosleep 0
> =A09928 futex_wait RET =A0 nanosleep -1 errno 35 Resource temporarily una=
vailable
> =A09929 futex_wait CALL =A0nanosleep(0xffffdb5c,0x81,0x1,0,0,0)
> =A09928 futex_wait CALL =A0nanosleep(0xffffdb5c,0x80,0x2,0,0,0)
> =A09929 futex_wait RET =A0 nanosleep 0
> =A09928 futex_wait RET =A0 nanosleep -1 errno 35 Resource temporarily una=
vailable
> =A09929 futex_wait CALL =A0nanosleep(0xffffdb5c,0x80,0x2,0,0,0)
> =A09928 futex_wait CALL =A0nanosleep(0xffffdb5c,0x81,0x1,0,0,0)
> =A09929 futex_wait RET =A0 nanosleep -1 errno 35 Resource temporarily una=
vailable
> =A09928 futex_wait RET =A0 nanosleep 0
> =A09929 futex_wait CALL =A0nanosleep(0xffffdb5c,0x81,0x1,0,0,0)
> =A09929 futex_wait RET =A0 nanosleep 0
> =A09929 futex_wait CALL =A0nanosleep(0xffffdb5c,0x80,0x2,0,0,0)
> =A09928 futex_wait CALL =A0nanosleep(0xffffdb5c,0x81,0x1,0,0,0)
> =A09929 futex_wait RET =A0 nanosleep -1 errno 35 Resource temporarily una=
vailable
> =A09928 futex_wait RET =A0 nanosleep 0
> =A09929 futex_wait CALL =A0nanosleep(0xffffdb5c,0x81,0x1,0,0,0)
> =A09929 futex_wait RET =A0 nanosleep 0
> =A09928 futex_wait CALL =A0nanosleep(0xffffdb5c,0x81,0x1,0,0,0)
> =A09929 futex_wait CALL =A0nanosleep(0xffffdb5c,0x80,0x2,0,0,0)
> =A09928 futex_wait RET =A0 nanosleep 0
> =A09929 futex_wait RET =A0 nanosleep -1 errno 35 Resource temporarily una=
vailable
> =A09929 futex_wait CALL =A0nanosleep(0xffffdb5c,0x80,0x2,0,0,0)
> =A09928 futex_wait CALL =A0nanosleep(0xffffdb5c,0x81,0x1,0,0,0)
> =A09929 futex_wait RET =A0 nanosleep -1 errno 35 Resource temporarily una=
vailable
>
> so this seems to be the location in linux_futex.c where error gets set to
> EAGAIN:
>
> static int
> futex_sleep(struct futex *f, struct waiting_proc *wp, int timeout)
> {
> =A0 =A0 =A0 =A0int error;
>
> =A0 =A0 =A0 =A0FUTEX_ASSERT_LOCKED(f);
> =A0 =A0 =A0 =A0LINUX_CTR4(sys_futex, "futex_sleep enter uaddr %p wp %p ti=
mo %d ref %d",
> =A0 =A0 =A0 =A0 =A0 =A0f->f_uaddr, wp, timeout, f->f_refcount);
> =A0 =A0 =A0 =A0error =3D sx_sleep(wp, &f->f_lck, PCATCH, "futex", timeout=
);
> =A0 =A0 =A0 =A0^^^^
>
> according to nanosleep(2) however EAGAIN is not a valid return value...i'=
m
> lost. :(
Don't trust the manpage young padawan (I wish one could, but not
all manpages are updated in a timely manner, in particular the kernel
ones I've discovered :(...). The one spot where it could be completely
screwing things up is with timer_create (it's the one logical spot in
sys/kern/... that returns EAGAIN), and this might make sense if our
clock values weren't being translated over properly for the
linuxulator and clocks were getting screwed up somewhere.
Well, that and clock_nanosleep, isn't implemented, and I wouldn't
be so sure if it's sane when compared to what Linux's clocks/timers do
(sane being equivalent, not necessarily right).
Something I ranted about in another PR:
http://www.freebsd.org/cgi/query-pr.cgi?pr=3Dkern/150095 .
So, that might be something worth looking into or not. It's up to
you *shrugs*.
HTH,
-Garrett
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?AANLkTi=dvFbKoVkXzZn8%2BdRJB7-mR3m=6SDAu2G1TTKC>