From owner-freebsd-apache@FreeBSD.ORG Mon Jul 19 22:41:27 2010 Return-Path: Delivered-To: apache@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7551B1065675 for ; Mon, 19 Jul 2010 22:41:27 +0000 (UTC) (envelope-from pgollucci@p6m7g8.com) Received: from cell.p6m7g8.net (static-71-178-236-107.washdc.fios.verizon.net [71.178.236.107]) by mx1.freebsd.org (Postfix) with ESMTP id 18FF08FC20 for ; Mon, 19 Jul 2010 22:41:26 +0000 (UTC) Received: from philip.hq.rws (wsip-174-79-184-239.dc.dc.cox.net [174.79.184.239]) (authenticated bits=0) by cell.p6m7g8.net (8.14.4/8.14.3) with ESMTP id o6JMfLXK024837 (version=TLSv1/SSLv3 cipher=DHE-DSS-CAMELLIA256-SHA bits=256 verify=NO) for ; Mon, 19 Jul 2010 22:41:21 GMT (envelope-from pgollucci@p6m7g8.com) Message-ID: <4C44D491.7000901@p6m7g8.com> Date: Mon, 19 Jul 2010 22:41:21 +0000 From: "Philip M. Gollucci" Organization: P6M7G8 Inc. User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.9) Gecko/20100607 Thunderbird/3.0.4 MIME-Version: 1.0 To: apache@FreeBSD.org X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-0.9 required=5.0 tests=BAYES_00,RDNS_DYNAMIC, TO_NO_BRKTS_DYNIP autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on cell.p6m7g8.net Cc: Subject: Fwd: Re: 2.3 upgrade on apache.org X-BeenThere: freebsd-apache@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Support of apache-related ports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Jul 2010 22:41:27 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 FYI - -------- Original Message -------- Subject: Re: 2.3 upgrade on apache.org Date: Sun, 18 Jul 2010 10:58:45 -0700 From: Paul Querna Reply-To: dev@httpd.apache.org To: dev@httpd.apache.org A small update: We thought the load average was a little high, in the 70+ range. We observed massive CPU usage from the httpd processes, but IO wise the machine was fine -- tons of IO bandwidth to go around -- it seemed to be something spinning inside httpd itself. At first I thought it might of been the event mpm bug, so with the awesome MPMs as a module, we switched to worker. Didn't help. I dug out ktrace, and started seeing what we were doing. Every httpd processes I looked at was spinning on sendfile: 33893 httpd CALL sendfile(0x33,0x23,0x1d3000,0xbe6201,0x7ffffd1e9bd0,0x7ffffd1e9bf8,) 33893 httpd RET sendfile -1 errno 35 Resource temporarily unavailable ktrace/dtrace dumps available to any committer who wants to take a look. A similar problem is described in this thread from 2002, about running 2.0.x on Daedalus: I've started tuning up the FreeBSD sysctls for network buffers etc, but it didn't seem to have a significant effect. We have now disabled Sendfile on apache.org, and the load average dropped from ~80 to 0.35. On Sun, Jul 18, 2010 at 3:08 AM, Paul Querna wrote: > www.apache.org is now running trunk @ r965127, using the Event MPM on > FreeBSD 8.1, on a new x86 box. > > Previously it was running on Solaris 10, with 2.2.x on sparc t2000s. > > I have also enabled OCSP stapling on the SSL side: > > > Anyways, I've made a list of the 2.3-ish issues seen during the upgrade. > > httpd itself: > - Using the Define directive to enable SSL was a bad idea. The SSL > init code was only being called once in post_config instead of twice > This is only 'new' because the Define directive is new, so I tried to > use it. Example of how it was used: > > Define SSL > Listen 1.2.3.4: > > > > LoadModule ssl_module .. > > > We do -D MACHINE_NAME at the command line, and I was hoping to just > enable SSL from within the config file, but that didn't work. Adding > -D SSL to the command line fixed the issue. > > - Lots of warnings in our configurations about "Ignoring deprecated > use of DefaultType". > > - Infra already had lots of changes, and those seem to stil be working fine. > > 3rd party: > > - APR's detection of BDB was funky again, I had --with-dbm=db48, which > it was happy about, but until I added --with-berkeley-db=/usr/local, > it wouldn't actually compile up the shared objects for the DB DBM, but > configure reported success. > > - mod_wsgi: Needed to comment out 2 lines debug logging, that were > accessing server_rec->loglevel (which has been removed from 2.3.x). > > - mod_mbox/mod_allowmethods/mod_asf_cgi: Needed build system > adjustments to not look for apr-util. > > Surprisingly painless upgrade after I figured out the mod_ssl config > issue; Most of the work was around moving Solaris -> FreeBSD for our > various rsyncs and crontabs, of course let infra know if you see > anything busted! > > Thanks, > > Paul > -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iD8DBQFMRNSRdbiP+9ubjBwRAippAJ49ODp6c+w8RsGTKYVf0yQblTbpYwCfSt6c od7KirxvHpV2dPTcWxi6wyI= =wrVg -----END PGP SIGNATURE-----