Date: Thu, 28 Dec 2017 17:34:06 +0000 From: Charlie Smurthwaite <charlie@atech.media> To: net@freebsd.org Subject: Linux netmap memory allocation Message-ID: <7b85fc73-9cc8-0a60-5264-d26f47af5eae@atech.media>
next in thread | raw e-mail | index | archive | help
Hi,
I'm just starting to use netmap and it is my intention to do zero-copy
forwarding of frames between a large number of NICs. I am using Intel
i350 (igb) on Linux. I therefore require a large memory area for rings
and buffers.
My calculation:
32 NICs * 2 rings (TX+RX) * 256 frames * 2048 bytes =3D 32MB
I am currently having a problem (or perhaps just a misunderstanding)
regarding allocation of this memory. I am attempting to use the
following code:
void thread_main(int thread_id) {
struct nmreq req; // A struct for the netmap request
int fd; // File descriptor for netmap socket
void * mem; // Pointer to allocated memory area
fd =3D open("/dev/netmap", 0); // Open a generic netmap socket
strcpy(req.nr_name, "enp8s0f0"); // Copy NIC name into request
req.nr_version =3D NETMAP_API; // Set version number
req.nr_flags =3D NR_REG_ONE_NIC; // We will be using a single hw ring
// Select ring 0, disable TX on poll
req.nr_ringid =3D NETMAP_NO_TX_POLL | NETMAP_HW_RING | 0;
// Ask for 64 additional rings to be allocated (32 * (TX+RX))
req.nr_arg1 =3D 64;
// Allocate a separate memory area for each thread
req.nr_arg2 =3D 10 + thread_id;
// Ask for additional buffers (256 per ring)
req.nr_arg3 =3D 64*256;
// Initialize port
ioctl(fd, NIOCREGIF, &req);
// Check the allocated memory size
printf("memsize: %u\n", req.nr_memsize);
// Check the allocated memory area
printf("nr_arg2: %u\n", req.nr_arg2);
}
The output is as follows:
memsize: 4206859
nr_arg2: 10
This is far short of the amount of memory I am hoping to be allocated.
Am I doing something wrong, or is this simply an indication that the
driver is unwilling to allocate more than 4MB?
A secondary (related) problem is that if I don't set arg1,arg2,arg3 in
my code (ie they will be zero), then I get varying output (it varies
between each of the following):
memsize: 4206843
nr_arg2: 0
memsize: 343019520
nr_arg2: 1
Any pointers would be appreciated. Thanks!
Charlie
Charlie Smurthwaite
Technical Director
tel. email. charlie@atech.media<mailto:charlie@atech.media> web. https://a=
tech.media
This e-mail has been sent by aTech Media Limited (or one of its assoicated =
group companys, Dial 9 Communications Limited or Viaduct Hosting Limited). =
Its contents are confidential therefore if you have received this message i=
n error, we would appreciate it if you could let us know and delete the mes=
sage. aTech Media Limited is a UK limited company, registration number 5523=
199. Dial 9 Communications Limited is a UK limited company, registration nu=
mber 7740921. Viaduct Hosting Limited is a UK limited company, registration=
number 8514362. All companies are registered at Unit 9 Winchester Place, N=
orth Street, Poole, Dorset, BH15 1NX.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?7b85fc73-9cc8-0a60-5264-d26f47af5eae>