From owner-freebsd-stable@FreeBSD.ORG Thu Nov 6 07:11:53 2014 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9B4C1419 for ; Thu, 6 Nov 2014 07:11:53 +0000 (UTC) Received: from smtp.infracaninophile.co.uk (smtp6.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3cd3:cd67:fafa:3d78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.infracaninophile.co.uk", Issuer "ca.infracaninophile.co.uk" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id EBD25F3D for ; Thu, 6 Nov 2014 07:11:52 +0000 (UTC) Received: from seedling.local (seedling.black-earth.co.uk [81.2.117.99]) (authenticated bits=0) by smtp.infracaninophile.co.uk (8.14.9/8.14.9) with ESMTP id sA67Bcjq019770 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Thu, 6 Nov 2014 07:11:40 GMT (envelope-from matthew@FreeBSD.org) DKIM-Filter: OpenDKIM Filter v2.9.2 smtp.infracaninophile.co.uk sA67Bcjq019770 Authentication-Results: smtp.infracaninophile.co.uk/sA67Bcjq019770; dkim=none reason="no signature"; dkim-adsp=none; dkim-atps=neutral X-Authentication-Warning: lucid-nonsense.infracaninophile.co.uk: Host seedling.black-earth.co.uk [81.2.117.99] claimed to be seedling.local Message-ID: <545B1F2A.5010203@FreeBSD.org> Date: Thu, 06 Nov 2014 07:11:38 +0000 From: Matthew Seaman User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Steven Hartland , freebsd-stable@freebsd.org Subject: Re: Varnish proxy goes catatonic under heavy load References: <545A0EB4.4090404@freebsd.org> <545A117B.4080606@multiplay.co.uk> In-Reply-To: <545A117B.4080606@multiplay.co.uk> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="5JvcHa91nkfKeKqGlA07fUbPm4pl86ewF" X-Virus-Scanned: clamav-milter 0.98.4 at lucid-nonsense.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on lucid-nonsense.infracaninophile.co.uk X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Nov 2014 07:11:53 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --5JvcHa91nkfKeKqGlA07fUbPm4pl86ewF Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 05/11/2014 12:00, Steven Hartland wrote: > As a guess you exhausted all mbufs, 10 has much better defaults for > these so I'd recommend updating. >=20 > If you can get in via IPMI or something similar you should be able to > confirm. >=20 > A trick I've used in the past to recover from such a issue is to hard > bounce the nic ports on the switch which seemed to free enough to be > able to ssh in. Yes, you nailed it. We managed to recreate the effect in the lab, and 10.0 behaves much better under horrible overload. While horribly slow, we can still get to a shell prompt via ssh, and when we turn off the load, the system recovers straight away. We'll be upgrading to 10.x ASAP. Cheers, Matthew --=20 Dr Matthew J Seaman MA, D.Phil. PGP: http://www.infracaninophile.co.uk/pgpkey --5JvcHa91nkfKeKqGlA07fUbPm4pl86ewF Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.20 (Darwin) iQJ8BAEBCgBmBQJUWx8qXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ2NTNBNjhCOTEzQTRFNkNGM0UxRTEzMjZC QjIzQUY1MThFMUE0MDEzAAoJELsjr1GOGkATd1oP/3NSbCs+EyrvRuLRpV205Lmx 5FUZUbL0ogP9I3y1M4iY6F8CAPJ24Fo7tUDqy1IEYUoI2sI4p2TzLViMy4oolQty EMsHcxepFk/L4Fh3i/Aik+GliJB3+qyJtSIMIgp2FP4vatvPZcv9NTlkXniB52dW fymcZILshNjq/1t4mG+82unBmkibWUcUnplbTWqjT30qq6Ck+4/9/sfJE6JRZCQD cqsytCnmSJt5kq5ZwlBTFRcj3GoSom6Av3Jx+3N9hxZFE6fvQ1Fd07YIH0/E7o+2 i2Y9l6lom8A1ggGODhXAGJIiqG5QgHGW3+MzSKFsJnWhyDNhC+fTqgBsXSbsts7l rOAVrksnezAtP8Gji/63HK5tRSjUptsAEPvN+y4O2QuG4FczuRuMAfyLfzyGgnAz zH6a8BhSZAoip4dGVQ/na1okEkE+vNzyPSyAiR5YQPF3hgAnFJZv8mzqC0YTrUIQ PL/D5jJMhNyB8k8wmfv26gtZc+VeSIoGyBIEox71rtQViDnpmBILb+MeLKc00vp8 1hUtqD8Wyia4oyJzyjK7tBJ1vhBRIHhfQiA4VWbp2vk+wFcB6oDcgG/ujXbn490x itxMm+lS2G0ueEM8oARqxlH1f4/chBOZKIxYiWEUVuaIWNc48GAsN/og2iRCLPfa LZVXCIjOMblqiYtIrG8o =nVlN -----END PGP SIGNATURE----- --5JvcHa91nkfKeKqGlA07fUbPm4pl86ewF--