From owner-freebsd-net@freebsd.org Tue Jun 16 21:48:23 2020 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 967283363EF for ; Tue, 16 Jun 2020 21:48:23 +0000 (UTC) (envelope-from prvs=44365f4422=mark.saad@lucera.com) Received: from mx0b-0017d602.pphosted.com (mx0b-0017d602.pphosted.com [148.163.153.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.pphosted.com", Issuer "Thawte RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49mhdy3kTRz3XfZ; Tue, 16 Jun 2020 21:48:22 +0000 (UTC) (envelope-from prvs=44365f4422=mark.saad@lucera.com) Received: from pps.filterd (m0101316.ppops.net [127.0.0.1]) by mx0b-0017d602.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 05GLiL9E014214; Tue, 16 Jun 2020 17:48:21 -0400 Received: from nam11-bn8-obe.outbound.protection.outlook.com (mail-bn8nam11lp2175.outbound.protection.outlook.com [104.47.58.175]) by mx0b-0017d602.pphosted.com with ESMTP id 31q68s00b4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 16 Jun 2020 17:48:21 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kEFU+tvQqyfqEZzoSVvoIOWrlLhr0wiuJQntUwzOwm3tFXvd9tZSU37klDneADMucTGiItiyJDutSYq9XVZVRdBQONX0MWwjv9tuqsfRjf20W+yUCetiOw9CMrLnE2Ju3S6QswCHRezjwSYBS45OnAXzrnDodde+cFq2ruPRxOsWyu+3eKcbGhU6AjGCsSKnmHhnqIebBQLSRV/IX4cBGmCPjFyCLVrEeal+4gYrSSvDUO/3XwbOEmy2wmlJeryXboRm3UGfdOnPljLncpZ58Ss8NbgBH3BdVVaEx+cuIWAOO6prLEcfDfqzzHAQrR9ZAYSSr+1fAvrKeCZxuTkGAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=E/g7mjlfPNh7X3UGKoA/da0ALQy5XjhqKTFdu5DqPrM=; b=gLOxhoGyNJ+1PR5ReN/7nySK7pV2/yINB/Pt3K5nvfA0D18JLSgHe6/mUVeJithzQxRd+M+QrwUYI+VgauYD8Pou2se/Nzs4k/9/aPiwpDsE10sYcLKsVFR7ySSIVOU44iU1gkFUEeSQaUT8g4SUUjvpoDj0OCfbypJH/NhA8tExV07v6ZJvcVCFaHQye9YboWYziyzYCw2vTgACSsAYhHYEIUZ7MIN30x7rnt8LLjot7F/ftWg7/7msdT9hPxdo7gw9E/boe0E9vZyHh56T4Dj7POnoYYq/aLYL6/7DExIU80iC824O75RPG9dJu6ojWijxrKvCbL6wwK65x+OWOA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=lucera.com; dmarc=pass action=none header.from=lucera.com; dkim=pass header.d=lucera.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bgccs.onmicrosoft.com; s=selector1-bgccs-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=E/g7mjlfPNh7X3UGKoA/da0ALQy5XjhqKTFdu5DqPrM=; b=D/ITUXp3akf3Pz384T/gc5dZehUk5Eq6F13aO9BxgWcOHaroaPQkn572Ht09G6YYG3C4a9ePTHSNJWVyf3MEBxpky2zJXfWC7C0mIJ1GIu+5vlfvU4Vd2jvkxOZle1Ad46r21nuQgTNkvpIirDb/m3siN4qNao+F93yO3KFtfHA= Received: from CY4PR12MB1704.namprd12.prod.outlook.com (2603:10b6:903:11d::22) by CY4PR1201MB0229.namprd12.prod.outlook.com (2603:10b6:910:1d::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.21; Tue, 16 Jun 2020 21:48:19 +0000 Received: from CY4PR12MB1704.namprd12.prod.outlook.com ([fe80::c19f:823:48d2:2863]) by CY4PR12MB1704.namprd12.prod.outlook.com ([fe80::c19f:823:48d2:2863%12]) with mapi id 15.20.3088.029; Tue, 16 Jun 2020 21:48:19 +0000 From: "Saad, Mark" To: "Foster, Greg" , Navdeep Parhar CC: "freebsd-net@freebsd.org" Subject: Re: How to Increase TX Queue Priority for LACP Packets Thread-Topic: How to Increase TX Queue Priority for LACP Packets Thread-Index: AdZBE8R05PE/72svTY6segyLIx/vzAAGao0AALpZlVAABDPSKQ== Date: Tue, 16 Jun 2020 21:48:19 +0000 Message-ID: References: <20200613025031.GA34653@x270>, In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [69.123.99.190] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 484e0eeb-fb93-4480-e589-08d8123efbad x-ms-traffictypediagnostic: CY4PR1201MB0229: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:9508; x-forefront-prvs: 04362AC73B x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: Sok+SXpMJP+XZBLtsnCmtQMPWgoM1EulnoUHHE/Z2DV3UCyzhTTPQf1mxXcDRfa7lk35Ywq9cV/1UtBNJSCEs23tm5BaV74a8hRtAUtA+VOmuSap1gayTqPwdJVi9dRZXCkKsHdg2+wyLNyWR4J61R+8Y+gWZ9981d3P3WUoaMAf8ZCrSPi+/irrgw9rZ4kS3PLZaSI7GXGzXQ0Uy3/mt3ZXBPxCK+EDWlSZUZgeoohcLBjc5pWMEcBTemlehtFZT+4wKn3BIbB566GTdefOF3QTGR9yzxXzlvDRihz+LJan8A5DjlQWxP6w20FBSQc/yg8PCLuRaYebBqrbh7HeIQfCp9K4y67+ltaNZZ85iIksClIj9EoMP8QcmmrbN8jxTXrsMLu0bRCCDc80tBEIsg== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CY4PR12MB1704.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(366004)(136003)(39860400002)(396003)(376002)(346002)(2906002)(110136005)(83380400001)(52536014)(478600001)(316002)(186003)(4326008)(86362001)(5660300002)(26005)(7696005)(76116006)(8676002)(66446008)(64756008)(33656002)(66946007)(66556008)(66476007)(966005)(9686003)(6506007)(53546011)(71200400001)(55016002)(8936002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: dmNquHBY9iGyMbirOWjFrZ+UOAk5NXYxBcCjtXhpGFPifXjiBEL+5GO4xxrU60N1mCo4X03oa0mN0chZehzURU6jdCOZl4XUNIL86RdsndGNusGYYOGUQxiJ8oYp5crZyGaxLsjP+Fg169RQvfcbRDSwqfIHvbdE2P2PGvFvo+WoAOEuUA+6gCN1GZHpAayUbTcp+eso6plZFXgH+VKZoHbmYEe+81lBVKm8+vjRngweParoUg2TsA3zxHhBQBsrjU2jolZkOY231HuHfgZBBKagOQiJgweDdxbVCwVALXZaXU2/UXfEUt08kpT1o/k/Qxu3tf4bLqM+HptwikYxFD2/6TQhgKezjbUGUZ7IDCB7/khaPGOEHElQrNZEZhEaAwAr7ICAcpXL+pZHAMLo7z8XyzzDmqqfF+hzLFxcyAhy7wDcIbEmEuMCNQhUFisXlFhCImRCvL9SdRyOt/Uln/WAdOAWclWCNtpTO0qbX1s= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: lucera.com X-MS-Exchange-CrossTenant-Network-Message-Id: 484e0eeb-fb93-4480-e589-08d8123efbad X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Jun 2020 21:48:19.1385 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: cfeb5f5e-839a-44b8-ab46-47157d8f1241 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: drR3xU49sVYGgzH0TlAcwVnGvzi6deYEMKulwyg6nuxWS4YWL1CxczjTEmkG2GBexuM8mRxIR6IFI+Sm5mvMYA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR1201MB0229 X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.216, 18.0.687 definitions=2020-06-16_13:2020-06-16, 2020-06-16 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 bulkscore=0 malwarescore=0 suspectscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 lowpriorityscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2004280000 definitions=main-2006160147 X-Rspamd-Queue-Id: 49mhdy3kTRz3XfZ X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bgccs.onmicrosoft.com header.s=selector1-bgccs-onmicrosoft-com header.b=D/ITUXp3; dmarc=none; spf=pass (mx1.freebsd.org: domain of prvs=44365f4422=mark.saad@lucera.com designates 148.163.153.124 as permitted sender) smtp.mailfrom=prvs=44365f4422=mark.saad@lucera.com X-Spamd-Result: default: False [-4.53 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-0.91)[-0.906]; RCVD_COUNT_FIVE(0.00)[5]; HAS_XOIP(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; MIME_GOOD(-0.10)[text/plain]; R_SPF_ALLOW(-0.20)[+ip4:148.163.153.124]; DMARC_NA(0.00)[lucera.com]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; R_DKIM_ALLOW(-0.20)[bgccs.onmicrosoft.com:s=selector1-bgccs-onmicrosoft-com]; FORGED_SENDER_VERP_SRS(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[bgccs.onmicrosoft.com:+]; NEURAL_HAM_SHORT(-1.02)[-1.020]; NEURAL_HAM_LONG(-1.01)[-1.007]; RCVD_IN_DNSWL_LOW(-0.10)[148.163.153.124:from]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:22843, ipnet:148.163.152.0/22, country:US]; FROM_NEQ_ENVFROM(0.00)[Mark.Saad@lucera.com,prvs=44365f4422=mark.saad@lucera.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[148.163.153.124:from] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Jun 2020 21:48:23 -0000 All Is there any way to make this change on other nic's like Intel ix and Sola= rflare sfxge ? I have seen similar issues on both with 12.1 mainly with solarflare nics. --- Mark Saad mark.saad@lucera.com ________________________________________ From: owner-freebsd-net@freebsd.org on beha= lf of Foster, Greg Sent: Tuesday, June 16, 2020 3:56 PM To: Navdeep Parhar Cc: freebsd-net@freebsd.org Subject: RE: How to Increase TX Queue Priority for LACP Packets HI Navdeep, Thanks for the information! I've integrated the changes and will be testin= g more today. We have seen the LACP port flapping under different scenarios, most we beli= eve are traffic/load based. I did see the flapping unexpectedly when I just enabled LACP debug (e.g., sysctl net.link.lagg.lacp.debug=3D1). Is this a known problem? Thanks Greg -----Original Message----- From: Navdeep Parhar On Behalf Of Navdeep Parhar Sent: Friday, June 12, 2020 7:51 PM To: Foster, Greg Cc: freebsd-net@freebsd.org Subject: Re: How to Increase TX Queue Priority for LACP Packets On Fri, Jun 12, 2020 at 11:47:41PM +0000, Foster, Greg wrote: > FreeBSD Networkers, > > We are seeing LACP port flapping on our FreeBSD 10.4/12.1 systems > under different conditions. > > Can someone explain or point me to the information on how to queue the > LACP packets to a higher priority queue ? > > We are using the Chelsio T580-LP-CR adapter/cxgbe driver. The Cheslio > NICs have 8 TX/RX queues each, but I don't know how to explicitly put > the LACP packets in the higher priority TX queue. > > I've read about PF/ALTQ and think this may be overkill our needs, and > was wondering if there was a simpler method. This is cxgbe specific but that's what you're using so it'll do. Add "hw.cxgbe.rsrv_noflowq=3D1" to your /boot/loader.conf. That reserves o= ne tx queue for non-RSS traffic (like ARP, LACP). You might also want to i= ncrease the number of tx queues to compensate for the one that's now reserv= ed. Use "hw.cxgbe.ntxq=3D9" for that. The ntxq knob might be different on= 10.4 but the man page matching the driver should have its exact name. Regards, Navdeep _______________________________________________ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"