From owner-freebsd-hackers@FreeBSD.ORG Fri Jan 18 01:39:08 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id C26D9EBC; Fri, 18 Jan 2013 01:39:08 +0000 (UTC) (envelope-from prvs=17301d565b=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id 08FDE24E; Fri, 18 Jan 2013 01:39:07 +0000 (UTC) Received: from r2d2 ([188.220.16.49]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50001744155.msg; Fri, 18 Jan 2013 01:39:05 +0000 X-Spam-Processed: mail1.multiplay.co.uk, Fri, 18 Jan 2013 01:39:05 +0000 (not processed: message from valid local sender) X-MDRemoteIP: 188.220.16.49 X-Return-Path: prvs=17301d565b=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: From: "Steven Hartland" To: , "'Damien Fleuriot'" References: <9A20C472-593A-4AAB-A0C0-FE6375011DAD@fisglobal.com> <201301171103.41013.jhb@freebsd.org> <796BC3D0-522A-4E1E-8BC6-96741D96E0A4@fisglobal.com> <259B53A09C4141E384BBA78217DD2BBD@multiplay.co.uk> <39477686-B5CC-439B-82A8-EEECBE358A9F@my.gd> <0bbf01cdf507$fddd7e20$f9987a60$@freebsd.org> Subject: Re: disadvantages of running 8.3 kernel on freebsd 8.2 system Date: Fri, 18 Jan 2013 01:39:27 -0000 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_140D_01CDF51C.A7402260" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: freebsd-hackers@freebsd.org, 'Devin Teske' , =?UTF-8?Q?'Ali_Okan_Y=C3=9CKSEL'?= X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Jan 2013 01:39:08 -0000 This is a multi-part message in MIME format. ------=_NextPart_000_140D_01CDF51C.A7402260 Content-Type: text/plain; format=flowed; charset="UTF-8"; reply-type=original Content-Transfer-Encoding: 7bit ----- Original Message ----- From: > But I bet you're not sitting 88 units of Thunderbolt cards that don't work in 8.3. > > 8.3 is also exhibiting major problems with the igb-based NICs on those same 88 units. Only effects igb init but might want to make sure you have r245334 back ported to avoid memory leaks when mbuf clusters are exhausted. 8.3 version of the patch attached ;-) For reference not only does this prevent the nic initialising properly it can also hang the boot process as when routing initialises "route" appears to trigger mbuf allocation with wait and as mbufs are exhaused and not freed correctly this hangs forever. This will happen on an untuned kernel if more than 2 igb nics are configured as each igb requires 8k of mbuf clusters (1k per queue x 8 queues on a machine with 8 or more cores) and the default kern.ipc.nmbclusters is only 25600. For clarity by "configured" I mean if the nic is initialised either by assigning an IP or "ifconfig igbX up" the queues are not allocated if the nic is present but unused. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. ------=_NextPart_000_140D_01CDF51C.A7402260 Content-Type: text/plain; format=flowed; name="igb-mbuf-free-patch-8.3-REL.txt"; reply-type=original Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="igb-mbuf-free-patch-8.3-REL.txt" Fixed mbuf free when receive structures fail to allocate.=0A= =0A= This prevents quad igb card on high core machines, without any = nmbcluster or=0A= igb queue tuning wedging the boot process if all nics are configured.=0A= --- sys/dev/e1000/if_igb.c.orig 2013-01-10 21:44:03.017805977 +0000=0A= +++ sys/dev/e1000/if_igb.c 2013-01-10 21:44:55.751355018 +0000=0A= @@ -4335,8 +4335,8 @@=0A= * the rings that completed, the failing case will have=0A= * cleaned up for itself. 'i' is the endpoint.=0A= */=0A= - for (int j =3D 0; j > i; ++j) {=0A= - rxr =3D &adapter->rx_rings[i];=0A= + for (int j =3D 0; j < i; ++j) {=0A= + rxr =3D &adapter->rx_rings[j];=0A= IGB_RX_LOCK(rxr);=0A= igb_free_receive_ring(rxr);=0A= IGB_RX_UNLOCK(rxr);=0A= ------=_NextPart_000_140D_01CDF51C.A7402260--