From owner-freebsd-current@FreeBSD.ORG Wed Feb 23 01:31:10 2011 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id C0AB01065673; Wed, 23 Feb 2011 01:31:10 +0000 (UTC) Date: Wed, 23 Feb 2011 01:31:10 +0000 From: Alexander Best To: Brandon Gooch Message-ID: <20110223013110.GA92284@freebsd.org> References: <20110222203134.GA53262@freebsd.org> <20110222215036.GA66303@freebsd.org> <20110222223314.GA72748@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20110222223314.GA72748@freebsd.org> Cc: Garrett Cooper , Eir Nym , Dimitry Andric , FreeBSD Current Subject: Re: Wow... (<-- blown away at performance) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Feb 2011 01:31:10 -0000 On Tue Feb 22 11, Alexander Best wrote: > On Tue Feb 22 11, Alexander Best wrote: > > On Tue Feb 22 11, Brandon Gooch wrote: > > > On Tue, Feb 22, 2011 at 2:31 PM, Alexander Best wrote: > > > > On Tue Feb 22 11, Garrett Cooper wrote: > > > >> On Tue, Feb 22, 2011 at 2:10 AM, Eir Nym wrote: > > > >> > On 22 February 2011 11:15, Garrett Cooper wrote: > > > >> >>    I don't know what to say, but r218938 screams with flash videos > > > >> >> (native Linux speed). Not sure if it's the new binutils or if it's the > > > >> >> new linuxulator patches, but I can run multiple instances of youtube > > > >> >> in parallel (5 total with other miscellaneous flash animation) without > > > >> >> it totally lagging out Firefox/X11, and it appears to close the > > > >> >> instances of firefox properly now. Hopefully this version fares better > > > >> >> than r218113 did (I think I hit a kernel bug after 2 weeks uptime, > > > >> >> where my system just hardlocked for no apparent reason). > > > >> >>    Anyhow, hope others have similar results. > > > >> >> Cheers! > > > >> >> -Garrett > > > >> >> > > > >> >> $ uname -a > > > >> >> FreeBSD bayonetta.local 9.0-CURRENT FreeBSD 9.0-CURRENT #0 r218938M: > > > >> >> Mon Feb 21 23:10:51 PST 2011 > > > >> >> gcooper@bayonetta.local:/usr/obj/usr/src/sys/BAYONETTA  amd64 > > > >> > > > > >> > Which FlashPlayer version do you test? Adobe has made significant > > > >> > performance changes in 10.2 (from 10.1). You can search for StageVideo > > > >> > performance to learn more about. Youtube already use them since 10.2 > > > >> > beta > > > >> > > > >>     linux-f10-flashplugin-10.1r102.65 . The performance increases are > > > >> claimed to be "up to 85%" on the Stage Video site, but I'm seeing a > > > >> more than 200% increase (now it actually scales between multiple > > > >> instances, instead of croaks with one instance, tiling up and down the > > > >> screen when moving the window slider for instance or switching tabs). > > > >> Besides, it seems like it needs external support from the video > > > >> driver, and I'm not sure that that bridge exists in the linuxulator. > > > >>     Also, it looks like npviewer.bin still hangs to resources on until > > > >> Firefox closes (or I kill it :)..), so something still needs to be > > > >> resolved there, but that isn't a regression (it's acted that way for > > > >> ages), and shouldn't be too hard to do. > > > > > > > > i think the problem with npviewer.bin having to be killed by hand is a futex > > > > issue in combination with a high number of threads. this is the output of a > > > > test from darren hart's futex test suite under freebsd 9.0: > > > > > > > > futex_wait: Measure FUTEX_WAIT operations per second > > > >        Arguments: iterations=100000000 threads=1 > > > > Result: 18622 Kiter/s > > > > futex_wait: Measure FUTEX_WAIT operations per second > > > >        Arguments: iterations=100000000 threads=2 > > > > Result: 15469 Kiter/s > > > > futex_wait: Measure FUTEX_WAIT operations per second > > > >        Arguments: iterations=100000000 threads=3 > > > > Result: 12713 Kiter/s > > > > futex_wait: Measure FUTEX_WAIT operations per second > > > >        Arguments: iterations=100000000 threads=4 > > > > Result: 12247 Kiter/s > > > > futex_wait: Measure FUTEX_WAIT operations per second > > > >        Arguments: iterations=100000000 threads=5 > > > > Result: 11814 Kiter/s > > > > futex_wait: Measure FUTEX_WAIT operations per second > > > >        Arguments: iterations=100000000 threads=6 > > > > Result: 13468 Kiter/s > > > > futex_wait: Measure FUTEX_WAIT operations per second > > > >        Arguments: iterations=100000000 threads=8 > > > > Result: 12061 Kiter/s > > > > futex_wait: Measure FUTEX_WAIT operations per second > > > >        Arguments: iterations=100000000 threads=10 > > > > Result: 12854 Kiter/s > > > > futex_wait: Measure FUTEX_WAIT operations per second > > > >        Arguments: iterations=100000000 threads=12 > > > > Result: 12999 Kiter/s > > > > futex_wait: Measure FUTEX_WAIT operations per second > > > >        Arguments: iterations=100000000 threads=16 > > > > Result: 12402 Kiter/s > > > > futex_wait: Measure FUTEX_WAIT operations per second > > > >        Arguments: iterations=100000000 threads=24 > > > > Result: 9815 Kiter/s > > > > futex_wait: Measure FUTEX_WAIT operations per second > > > >        Arguments: iterations=100000000 threads=32 > > > > Result: 8518 Kiter/s > > > > futex_wait: Measure FUTEX_WAIT operations per second > > > >        Arguments: iterations=100000000 threads=64 > > > >        ERROR: Resource temporarily unavailable: pthread_create > > > > Result: ERROR > > > > futex_wait: Measure FUTEX_WAIT operations per second > > > >        Arguments: iterations=100000000 threads=128 > > > >        ERROR: Resource temporarily unavailable: pthread_create > > > > Result: ERROR > > > > futex_wait: Measure FUTEX_WAIT operations per second > > > >        Arguments: iterations=100000000 threads=256 > > > >        ERROR: Resource temporarily unavailable: pthread_create > > > > Result: ERROR > > > > futex_wait: Measure FUTEX_WAIT operations per second > > > >        Arguments: iterations=100000000 threads=512 > > > >        ERROR: Resource temporarily unavailable: pthread_create > > > > Result: ERROR > > > > futex_wait: Measure FUTEX_WAIT operations per second > > > >        Arguments: iterations=100000000 threads=1024 > > > >        ERROR: Resource temporarily unavailable: pthread_create > > > > Result: ERROR > > > > > > > > cheers. > > > > alex > > > > > > Have you found any method or workaround to mitigate this issue? > > i did a ktrace -i and linux_kdump run and the results are quite interesting. > with only 1 thread linux_sys_futex() *never* failed with EAGAIN. starting > with 2 threads however i'm seeing an increasing number of linux_sys_futex() > failures (all with EAGAIN). it seems however that the cause for the tests with 64-1024 threads to fail is linux_mmap(): 10070 futex_wait CALL linux_clone(0x3d0f00,0xfc20d4b4,0xfc20dbd8,0xffffd9bc,0xfc20dbd8) 10122 futex_wait RET linux_fork 0 10122 futex_wait CALL linux_set_robust_list(0xf820dbe0,0xc) 10122 futex_wait RET linux_set_robust_list 0 10122 futex_wait CALL linux_sys_futex(0xffffdb48,0x80,0,0,0,0) 10070 futex_wait RET linux_clone 10123/0x278b 10123 futex_wait RET linux_fork 0 10070 futex_wait CALL linux_mmap2(0,0x4000000,0x3,0x20022,0xffffffff,0) 10123 futex_wait CALL linux_set_robust_list(0xfc20dbe0,0xc) 10123 futex_wait RET linux_set_robust_list 0 10123 futex_wait CALL linux_sys_futex(0xffffdb48,0x80,0,0,0,0) 10070 futex_wait RET linux_mmap2 -1 errno 12 Cannot allocate memory 10070 futex_wait CALL write(0x2,0xffffb2c8,0x44) 10070 futex_wait GIO fd 2 wrote 68 bytes " \^[[1;33mERROR\^[[0m: Resource temporarily unavailable: pthread_create " 10070 futex_wait RET write 68/0x44 10070 futex_wait CALL linux_sys_futex(0xffffdb48,0x81,0x7fffffff,0,0,0) cheers. alex > > the overhead is quite massive. linux_kdump > /tmp/futex-failure produced 941 > megs of output. doing > grep -v "futex_wait" futex-failure > futex-grep > produces an output of 192K!! so it's quite obvious that the futex_wait stuff > is responsible for causing massive activity, due to the constant failures and > retries. > > here's a bit of output: > > 32126 futex_wait RET linux_sys_futex -1 errno 35 Resource temporarily unavailable > 32125 futex_wait RET linux_sys_futex 0 > 32126 futex_wait CALL linux_sys_futex(0xffffdb5c,0x81,0x1,0,0,0) > 32126 futex_wait RET linux_sys_futex 0 > 32126 futex_wait CALL linux_sys_futex(0xffffdb5c,0x81,0x1,0,0,0) > 32125 futex_wait CALL linux_sys_futex(0xffffdb5c,0x80,0x2,0,0,0) > 32126 futex_wait RET linux_sys_futex 0 > 32125 futex_wait RET linux_sys_futex -1 errno 35 Resource temporarily unavailable > 32126 futex_wait CALL linux_sys_futex(0xffffdb5c,0x80,0x2,0,0,0) > 32125 futex_wait CALL linux_sys_futex(0xffffdb5c,0x81,0x1,0,0,0) > 32126 futex_wait RET linux_sys_futex -1 errno 35 Resource temporarily unavailable > 32125 futex_wait RET linux_sys_futex 0 > 32126 futex_wait CALL linux_sys_futex(0xffffdb5c,0x81,0x1,0,0,0) > 32126 futex_wait RET linux_sys_futex 0 > 32126 futex_wait CALL linux_sys_futex(0xffffdb5c,0x80,0x2,0,0,0) > 32125 futex_wait CALL linux_sys_futex(0xffffdb5c,0x81,0x1,0,0,0) > 32126 futex_wait RET linux_sys_futex -1 errno 35 Resource temporarily unavailable > 32125 futex_wait RET linux_sys_futex 0 > 32126 futex_wait CALL linux_sys_futex(0xffffdb5c,0x81,0x1,0,0,0) > 32126 futex_wait RET linux_sys_futex 0 > 32126 futex_wait CALL linux_sys_futex(0xffffdb5c,0x80,0x2,0,0,0) > 32125 futex_wait CALL linux_sys_futex(0xffffdb5c,0x81,0x1,0,0,0) > 32126 futex_wait RET linux_sys_futex -1 errno 35 Resource temporarily unavailable > 32125 futex_wait RET linux_sys_futex 0 > 32126 futex_wait CALL linux_sys_futex(0xffffdb5c,0x81,0x1,0,0,0) > 32126 futex_wait RET linux_sys_futex 0 > 32126 futex_wait CALL linux_sys_futex(0xffffdb5c,0x81,0x1,0,0,0) > 32125 futex_wait CALL linux_sys_futex(0xffffdb5c,0x80,0x2,0,0,0) > 32126 futex_wait RET linux_sys_futex 0 > 32125 futex_wait RET linux_sys_futex -1 errno 35 Resource temporarily unavailable > 32125 futex_wait CALL linux_sys_futex(0xffffdb5c,0x81,0x1,0,0,0) > 32126 futex_wait CALL linux_sys_futex(0xffffdb5c,0x80,0x2,0,0,0) > 32125 futex_wait RET linux_sys_futex 0 > 32126 futex_wait RET linux_sys_futex -1 errno 35 Resource temporarily unavailable > 32126 futex_wait CALL linux_sys_futex(0xffffdb5c,0x80,0x2,0,0,0) > 32125 futex_wait CALL linux_sys_futex(0xffffdb5c,0x81,0x1,0,0,0) > 32126 futex_wait RET linux_sys_futex -1 errno 35 Resource temporarily unavailable > 32125 futex_wait RET linux_sys_futex 0 > 32126 futex_wait CALL linux_sys_futex(0xffffdb5c,0x81,0x1,0,0,0) > 32126 futex_wait RET linux_sys_futex 0 > 32126 futex_wait CALL linux_sys_futex(0xffffdb5c,0x80,0x2,0,0,0) > 32125 futex_wait CALL linux_sys_futex(0xffffdb5c,0x81,0x1,0,0,0) > 32126 futex_wait RET linux_sys_futex -1 errno 35 Resource temporarily unavailable > 32125 futex_wait RET linux_sys_futex 0 > 32126 futex_wait CALL linux_sys_futex(0xffffdb5c,0x81,0x1,0,0,0) > 32126 futex_wait RET linux_sys_futex 0 > 32126 futex_wait CALL linux_sys_futex(0xffffdb5c,0x80,0x2,0,0,0) > 32125 futex_wait CALL linux_sys_futex(0xffffdb5c,0x81,0x1,0,0,0) > 32126 futex_wait RET linux_sys_futex -1 errno 35 Resource temporarily unavailable > 32125 futex_wait RET linux_sys_futex 0 > 32126 futex_wait CALL linux_sys_futex(0xffffdb5c,0x81,0x1,0,0,0) > 32126 futex_wait RET linux_sys_futex 0 > 32126 futex_wait CALL linux_sys_futex(0xffffdb5c,0x80,0x2,0,0,0) > 32125 futex_wait CALL linux_sys_futex(0xffffdb5c,0x81,0x1,0,0,0) > 32126 futex_wait RET linux_sys_futex -1 errno 35 Resource temporarily unavailable > > cheers. > alex > > > > > no. i've increased the kern.threads.max_threads_per_proc value, but that had > > no effect. so it seems to be a bug in the linuxulator's futex implementation. > > > > cheers. > > alex > > > > > > > > -Brandon > > > > -- > > a13x > > -- > a13x -- a13x