From owner-freebsd-net@FreeBSD.ORG Fri May 25 00:48:31 2012 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1523E106567E for ; Fri, 25 May 2012 00:48:31 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-wi0-f172.google.com (mail-wi0-f172.google.com [209.85.212.172]) by mx1.freebsd.org (Postfix) with ESMTP id 94E468FC0C for ; Fri, 25 May 2012 00:48:30 +0000 (UTC) Received: by wibhj8 with SMTP id hj8so5227688wib.13 for ; Thu, 24 May 2012 17:48:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=TlRSJY/I1GqS2gMLputD4HRGCwfjTOlFW69JnyllSTE=; b=iG3oryIc62Wju5bnERIRAzZUp5zeB9FYPcEcGWgDyUg53loYXouicOCHofRNzC8w8/ FFv4SfwZQylBR3N4VDSiqIf5usTJF00iY9FX/gliRDZFWfMitucDVnr//fbGQ3yP0k/B /7g1sf7u+XQgRQvqdVG5ZrISOQvlJ2UIjA/t874JfDH7IAEtVdCTr8VCeIgyi38kiZyN cfMmmtf5ryZu+HPqw1Pvt3kIX6J+WX09Da21FI4AcqgmNQRORVeDPc1GdU/ZFrPgTKyo 9qtXvgtq5h+OIgV4HH7D9YY/buGoIAXqWTFXPE6ezrMPguJnHl3omuucf47/YPDphQKr hAWA== MIME-Version: 1.0 Received: by 10.180.109.197 with SMTP id hu5mr2868973wib.8.1337906903758; Thu, 24 May 2012 17:48:23 -0700 (PDT) Received: by 10.223.155.4 with HTTP; Thu, 24 May 2012 17:48:23 -0700 (PDT) In-Reply-To: References: Date: Thu, 24 May 2012 17:48:23 -0700 Message-ID: From: Kevin Oberman To: "Bjoern A. Zeeb" Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: net@freebsd.org Subject: Re: Major performance hit with ToS setting X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 May 2012 00:48:31 -0000 On Thu, May 24, 2012 at 4:43 PM, Bjoern A. Zeeb wrote: > On 24. May 2012, at 22:55 , Kevin Oberman wrote: > >> When we set the ToS bits for less than best effort (also called >> scavenger) on packets (ToS=3D32), performance on FreeBSD 8.2 is >> terrible. It was as good as best effort on FreeBSD 7.3 (assuming no >> congestion). I will look into what 9 does, but does anyone have an >> idea of why 8.2 behaves so badly when ToS is set to 32? >> Here is an example of what happens to performance: >> nuttcp with ToS=3D0: >> 622.5000 MB / =A0 1.00 sec =3D 5221.7570 Mbps >> 623.3125 MB / =A0 1.00 sec =3D 5228.5883 Mbps >> 624.0000 MB / =A0 1.00 sec =3D 5234.4495 Mbps >> >> With ToS=3D32 (0x20): >> 0.3750 MB / =A0 1.00 sec =3D =A0 =A03.1457 Mbps >> 0.5000 MB / =A0 1.00 sec =3D =A0 =A04.1942 Mbps >> 0.5000 MB / =A0 1.00 sec =3D =A0 =A04.1942 Mbps >> >> This is,of course, on a 10G interface. On 7.3 there is little >> difference between the two. We are using cubic CC on the 8.2 system. > > This looks like a different problem than just TOS. =A0I assume however > that not setting the TOS you get the same as expected performance? > > Which NIC? The two nuttcp runs were identical and we got similar results with iperf. All tests were using Myricom (mxge) cards. --=20 R. Kevin Oberman, Network Engineer E-mail: kob6558@gmail.com