From owner-freebsd-stable@FreeBSD.ORG Tue Nov 15 08:33:20 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C015E10657AB; Tue, 15 Nov 2011 08:33:20 +0000 (UTC) (envelope-from dcherednik@masterhost.ru) Received: from mail.corp.masterhost.ru (wincas02.mail.corp.masterhost.ru [90.156.219.212]) by mx1.freebsd.org (Postfix) with ESMTP id F26658FC0C; Tue, 15 Nov 2011 08:33:19 +0000 (UTC) Received: from kzholnay-pc.masterhost.ru (87.242.97.5) by mail.corp.masterhost.ru (90.156.219.210) with Microsoft SMTP Server (TLS) id 14.1.339.1; Tue, 15 Nov 2011 12:22:20 +0400 Message-ID: <4EC2213B.7030005@masterhost.ru> Date: Tue, 15 Nov 2011 12:22:19 +0400 From: Daniil Cherednik User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.23) Gecko/20110922 Lightning/1.0b2 Thunderbird/3.1.15 MIME-Version: 1.0 To: References: <4EC17AAF.9050807@FreeBSD.org> <201111141556.13295.jhb@freebsd.org> <4EC18F28.20307@FreeBSD.org> In-Reply-To: <4EC18F28.20307@FreeBSD.org> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-Originating-IP: [87.242.97.5] Cc: Doug Barton , Konstantin Belousov , John Baldwin Subject: Re: 8.2 + apache == a LOT of sigprocmask X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Nov 2011 08:33:20 -0000 Hi. I was trying to understand a cause for this problem, and made an ugly hack: diff -u ./rtld_lock.c.orig ./rtld_lock.c --- ./rtld_lock.c.orig 2011-11-15 07:56:14.000000000 +0000 +++ ./rtld_lock.c 2011-11-15 07:54:42.000000000 +0000 @@ -118,7 +118,7 @@ sigset_t tmp_oldsigmask; for ( ; ; ) { - sigprocmask(SIG_BLOCK, &fullsigmask, &tmp_oldsigmask); +// sigprocmask(SIG_BLOCK, &fullsigmask, &tmp_oldsigmask); if (atomic_cmpset_acq_int(&l->lock, 0, WAFLAG)) break; sigprocmask(SIG_SETMASK, &tmp_oldsigmask, NULL); @@ -135,7 +135,7 @@ atomic_add_rel_int(&l->lock, -RC_INCR); else { atomic_add_rel_int(&l->lock, -WAFLAG); - sigprocmask(SIG_SETMASK, &oldsigmask, NULL); +// sigprocmask(SIG_SETMASK, &oldsigmask, NULL); } } this reduced number of syscalls, but I am not sure about stability and correctness of this hack. And performance problems of apache remained (this hack only gave 5-10% increase of performance). Also I found problem in longjmp syscalls. In FreeBSD we are doing this syscalls from gen/setjmp.S without condition check, for example in Linux we are doing it only if stack has been saved. Linux code: if (env[0].__mask_was_saved) /* Restore the saved signal mask. */ (void) __sigprocmask (SIG_SETMASK, &env[0].__saved_mask, (sigset_t *) NULL); After all that I was trying to compare perfomance of return from fork() in Linux and FreeBSD (see http://lists.freebsd.org/pipermail/freebsd-hackers/2011-October/036705.html) and fork() in FreeBSD was slower. I am not trying to start a holy war, but I really need to increase performance of our hosting in FreeBSD. On 15.11.2011 01:59, Doug Barton wrote: > On 11/14/2011 12:56, John Baldwin wrote: >> On Monday, November 14, 2011 3:31:43 pm Doug Barton wrote: >>> Trying to track down a load problem we're seeing on 8.2-RELEASE-p4 i386 >>> in a busy web hosting environment I came across the following post: >>> >>> http://lists.freebsd.org/pipermail/freebsd-questions/2011- >> October/234520.html >>> That basically describes what we're seeing as well, including the >>> "doesn't happen on Linux" part. >>> >>> Does anyone have any ideas about this? >>> >>> With incredibly similar stuff running on 7.x we didn't see this problem, >>> so it seems to be something new in 8. >> I suspect it has to do with some of the changes to rtld such that it now >> always blocks signals while resolving symbols (or something along those >> lines IIRC). It makes throwing exceptions slow as well. > The calls to sigprocmask() in rtld seem to be doing what you suggest > here, but they involve setting and restoring the mask. In my followup > post I pasted what we're seeing, which is different, and much more > voluminous. For example, 13,500 calls in 30 seconds from a single apache > worker process. > > Although this does seem to explain why our test cases have more calls > when compiled with C++ than they do when compiled with C. :) > > Thanks for the response in any case. > > > Doug > -- С уважением, Daniil Cherednik .masterhost