Date: Mon, 19 Jul 2010 22:41:21 +0000 From: "Philip M. Gollucci" <pgollucci@p6m7g8.com> To: apache@FreeBSD.org Subject: Fwd: Re: 2.3 upgrade on apache.org Message-ID: <4C44D491.7000901@p6m7g8.com>
next in thread | raw e-mail | index | archive | help
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 FYI - -------- Original Message -------- Subject: Re: 2.3 upgrade on apache.org Date: Sun, 18 Jul 2010 10:58:45 -0700 From: Paul Querna <paul@querna.org> Reply-To: dev@httpd.apache.org To: dev@httpd.apache.org A small update: We thought the load average was a little high, in the 70+ range. We observed massive CPU usage from the httpd processes, but IO wise the machine was fine -- tons of IO bandwidth to go around -- it seemed to be something spinning inside httpd itself. At first I thought it might of been the event mpm bug, so with the awesome MPMs as a module, we switched to worker. Didn't help. I dug out ktrace, and started seeing what we were doing. Every httpd processes I looked at was spinning on sendfile: 33893 httpd CALL sendfile(0x33,0x23,0x1d3000,0xbe6201,0x7ffffd1e9bd0,0x7ffffd1e9bf8,<invalid=0>) 33893 httpd RET sendfile -1 errno 35 Resource temporarily unavailable ktrace/dtrace dumps available to any committer who wants to take a look. A similar problem is described in this thread from 2002, about running 2.0.x on Daedalus: <http://www.mail-archive.com/dev@httpd.apache.org/msg04928.html>; I've started tuning up the FreeBSD sysctls for network buffers etc, but it didn't seem to have a significant effect. We have now disabled Sendfile on apache.org, and the load average dropped from ~80 to 0.35. On Sun, Jul 18, 2010 at 3:08 AM, Paul Querna <paul@querna.org> wrote: > www.apache.org is now running trunk @ r965127, using the Event MPM on > FreeBSD 8.1, on a new x86 box. > > Previously it was running on Solaris 10, with 2.2.x on sparc t2000s. > > I have also enabled OCSP stapling on the SSL side: > <https://www.apache.org/>; > > Anyways, I've made a list of the 2.3-ish issues seen during the upgrade. > > httpd itself: > - Using the Define directive to enable SSL was a bad idea. The SSL > init code was only being called once in post_config instead of twice > This is only 'new' because the Define directive is new, so I tried to > use it. Example of how it was used: > <IfDefine MACHINE_NAME> > Define SSL > Listen 1.2.3.4: > </IfDefine> > > <IfDefine SSL> > LoadModule ssl_module .. > </IfDefine> > > We do -D MACHINE_NAME at the command line, and I was hoping to just > enable SSL from within the config file, but that didn't work. Adding > -D SSL to the command line fixed the issue. > > - Lots of warnings in our configurations about "Ignoring deprecated > use of DefaultType". > > - Infra already had lots of <IfVersion blocks for most 2.3 authn/z > changes, and those seem to stil be working fine. > > 3rd party: > > - APR's detection of BDB was funky again, I had --with-dbm=db48, which > it was happy about, but until I added --with-berkeley-db=/usr/local, > it wouldn't actually compile up the shared objects for the DB DBM, but > configure reported success. > > - mod_wsgi: Needed to comment out 2 lines debug logging, that were > accessing server_rec->loglevel (which has been removed from 2.3.x). > > - mod_mbox/mod_allowmethods/mod_asf_cgi: Needed build system > adjustments to not look for apr-util. > > Surprisingly painless upgrade after I figured out the mod_ssl config > issue; Most of the work was around moving Solaris -> FreeBSD for our > various rsyncs and crontabs, of course let infra know if you see > anything busted! > > Thanks, > > Paul > -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iD8DBQFMRNSRdbiP+9ubjBwRAippAJ49ODp6c+w8RsGTKYVf0yQblTbpYwCfSt6c od7KirxvHpV2dPTcWxi6wyI= =wrVg -----END PGP SIGNATURE-----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4C44D491.7000901>