From owner-freebsd-net@freebsd.org Wed Apr 25 22:21:47 2018 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7B679FB1CC6 for ; Wed, 25 Apr 2018 22:21:47 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 1961378968 for ; Wed, 25 Apr 2018 22:21:46 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: by mailman.ysv.freebsd.org (Postfix) id C9A67FB1CBD; Wed, 25 Apr 2018 22:21:45 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8A8CAFB1CBB; Wed, 25 Apr 2018 22:21:45 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-QB1-obe.outbound.protection.outlook.com (mail-eopbgr660054.outbound.protection.outlook.com [40.107.66.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT TLS CA 4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1D12D78960; Wed, 25 Apr 2018 22:21:44 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from YQBPR0101MB1042.CANPRD01.PROD.OUTLOOK.COM (52.132.66.153) by YQBPR0101MB0897.CANPRD01.PROD.OUTLOOK.COM (52.132.66.12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.715.18; Wed, 25 Apr 2018 22:21:43 +0000 Received: from YQBPR0101MB1042.CANPRD01.PROD.OUTLOOK.COM ([fe80::893c:efc2:d71f:945a]) by YQBPR0101MB1042.CANPRD01.PROD.OUTLOOK.COM ([fe80::893c:efc2:d71f:945a%13]) with mapi id 15.20.0715.018; Wed, 25 Apr 2018 22:21:43 +0000 From: Rick Macklem To: Ryan Stone , Konstantin Belousov CC: FreeBSD Current , "menny@mellanox.com" , FreeBSD Net Subject: Re: mlx5(4) jumbo receive Thread-Topic: mlx5(4) jumbo receive Thread-Index: AQHT3NCdClq8g0H1WkiFu85ReYNX16QSDLEp Date: Wed, 25 Apr 2018 22:21:43 +0000 Message-ID: References: <20180424085553.GA6887@kib.kiev.ua>, In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=rmacklem@uoguelph.ca; x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; YQBPR0101MB0897; 7:CyvMySIgonstN1+bHIQ37R8cTLaibTVBFHfSoJASehLM87egVEc01UMhzzKKlcaIcmmzN8YInNn1wWS7Cp7lVTGySfslbuuM+kZtKxujN8oRk3yjOmxl8vEhvaYStbixezmDJBEgRRZm8FvtDCfQrEPYCT116LDgwwEq0IimpQQOhpjyf25UURndOD72/9usKxOtczZYRko3uUh76Nd6GL/bPWDLNOc65iMUcgl8llg2Hr8Vshb6aJlZ2xLDvYId x-ms-exchange-antispam-srfa-diagnostics: SOS; x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(8989080)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(8990040)(2017052603328)(7153060)(7193020); SRVR:YQBPR0101MB0897; x-ms-traffictypediagnostic: YQBPR0101MB0897: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(85827821059158); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040522)(2401047)(5005006)(8121501046)(10201501046)(93006095)(93001095)(3231232)(944501410)(52105095)(3002001)(6041310)(20161123562045)(20161123558120)(20161123560045)(201703131423095)(201702281529075)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(6072148)(201708071742011); SRVR:YQBPR0101MB0897; BCL:0; PCL:0; RULEID:; SRVR:YQBPR0101MB0897; x-forefront-prvs: 06530126A4 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(366004)(346002)(396003)(376002)(39860400002)(39380400002)(199004)(189003)(305945005)(229853002)(2906002)(2900100001)(59450400001)(3280700002)(9686003)(53936002)(55016002)(3660700001)(786003)(86362001)(99286004)(26005)(486006)(11346002)(316002)(102836004)(7696005)(6506007)(186003)(76176011)(476003)(68736007)(54906003)(446003)(110136005)(8936002)(81166006)(5250100002)(81156014)(8676002)(97736004)(106356001)(33656002)(74482002)(39060400002)(4326008)(105586002)(25786009)(6246003)(6436002)(5660300001)(74316002)(14454004)(478600001); DIR:OUT; SFP:1101; SCL:1; SRVR:YQBPR0101MB0897; H:YQBPR0101MB1042.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: uoguelph.ca does not designate permitted sender hosts) x-microsoft-antispam-message-info: fVqqgwQGcL4uONKMbnDsW3NdInPUopTg2DUHQ2GaZp1odsc5NmvYZFq3ZXxiBK/OVJnQyXHjTC3LUSUMbkR0VlfWhIWf5yXoRfAAnhMKPZN4l1yjlasBRDZrrJu9Rv6ridB4L8eisIHYeO8LXyRl+VWr4W9GUu0yEkiqn33BaWdSNPIWXGtLnyJibf0HNGCu spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 64862c4f-1c94-4c4b-c3f4-08d5aafaecc6 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-Network-Message-Id: 64862c4f-1c94-4c4b-c3f4-08d5aafaecc6 X-MS-Exchange-CrossTenant-originalarrivaltime: 25 Apr 2018 22:21:43.3007 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQBPR0101MB0897 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Apr 2018 22:21:47 -0000 Ryan Stone wrote: >On Tue, Apr 24, 2018 at 4:55 AM, Konstantin Belousov >>wrote: >> +#ifndef MLX5E_MAX_RX_BYTES >> +#define MLX5E_MAX_RX_BYTES MCLBYTES >> +#endif > >Why do you use a 2KB buffer rather than a PAGE_SIZE'd buffer? >MJUMPAGESIZE should offer significantly better performance for jumbo >frames without increasing the risk of memory fragmentation. Actually, when I was playing with using jumbo mbuf clusters for NFS, I was = able to get it to fragment to the point where allocations failed when mixing 2K = and 4K mbuf clusters. Admittedly I was using a 256Mbyte i386 and it wasn't easily reproduced, but it was possible. --> Using a mix of 2K and 4K mbuf clusters can result in fragmentation, alt= hough I suspect that it isn't nearly as serious as what can happen when usi= ng 9K mbuf clusters. rick=