From owner-freebsd-net@freebsd.org  Wed Mar 28 13:49:25 2018
Return-Path: <owner-freebsd-net@freebsd.org>
Delivered-To: freebsd-net@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id C7DC4F5F7BC
 for <freebsd-net@mailman.ysv.freebsd.org>;
 Wed, 28 Mar 2018 13:49:25 +0000 (UTC)
 (envelope-from Ming.Fu@esentire.com)
Received: from mail.esentire.com (mail.esentire.com [52.129.34.132])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 72FB06B140
 for <freebsd-net@freebsd.org>; Wed, 28 Mar 2018 13:49:25 +0000 (UTC)
 (envelope-from Ming.Fu@esentire.com)
Received: from exchange.esentire.com (cas01colo01p.internal [10.1.120.115])
 by mail.esentire.com (Postfix) with ESMTPS id D76FA1801CD
 for <freebsd-net@freebsd.org>; Wed, 28 Mar 2018 13:41:27 +0000 (UTC)
Received: from mbx01cmb01p.esentire.local (10.1.120.118) by
 mbx01cmb01p.esentire.local (10.1.120.118) with Microsoft SMTP Server (TLS) id
 15.0.1347.2; Wed, 28 Mar 2018 09:41:27 -0400
Received: from mbx01cmb01p.esentire.local ([fe80::c036:855e:1a82:9d34]) by
 mbx01cmb01p.esentire.local ([fe80::c036:855e:1a82:9d34%14]) with mapi id
 15.00.1347.000; Wed, 28 Mar 2018 09:41:27 -0400
From: Ming Fu <Ming.Fu@esentire.com>
To: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org>
Subject: Netmap on Linux nm_open() fail when receive ring size is set to 4096
Thread-Topic: Netmap on Linux nm_open() fail when receive ring size is set to
 4096
Thread-Index: AdPGmnaafMxJ6MGgQPqo+Ei4Y3bW4Q==
Date: Wed, 28 Mar 2018 13:41:27 +0000
Message-ID: <092c8d937ff6471fb701d64dcff1c756@mbx01cmb01p.esentire.local>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: 
X-MS-TNEF-Correlator: 
x-ms-exchange-transport-fromentityheader: Hosted
x-originating-ip: [10.1.120.131]
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
X-Content-Filtered-By: Mailman/MimeDel 2.1.25
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.25
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net/>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 28 Mar 2018 13:49:26 -0000

Hi,

I was trying netmap on a Linux box with 128G of ram (64G per numa node). If=
 I set ixgbe interface to 4096 ring size, the nm_open will fail with error =
"Cannot allocate memory". What can I tweak to make the card use larger ring=
 size? The following test was run after fresh reboot.

$ ethtool -g enp5s0f0
Ring parameters for enp5s0f0:
Pre-set maximums:
RX:                        4096
RX Mini:               0
RX Jumbo:           0
TX:                        4096
Current hardware settings:
RX:                        512
RX Mini:               0
RX Jumbo:           0
TX:                        512

$ ethtool -G enp5s0f0 rx 1024
$ ./nmtest -i enp5s0f0
^C
$ ethtool -G enp5s0f0 rx 2048
$ nmtest -i enp5s0f0
^C
$ ethtool -G enp5s0f0 rx 4096
$ nmtest -i enp5s0f0
816.039684 nm_open [945] NIOCREGIF failed: Cannot allocate memory netmap:en=
p5s0f0
fail to nm_open(netmap:enp5s0f0 ... ): Cannot allocate memory

Thanks,
Ming