Date: Tue, 27 Mar 2018 09:53:11 +0000 From: "Kononov, Oleksandr" <oleksandr.kononov@intel.com> To: "freebsd-drivers@freebsd.org" <freebsd-drivers@freebsd.org> Cc: "Vanco, Juraj" <juraj.vanco@intel.com> Subject: FreeBSD 11.1 contigfree performance issue Message-ID: <865AA1660A1A014C99D99B800FA40800813681@IRSMSX101.ger.corp.intel.com>
index | next in thread | raw e-mail
[-- Attachment #1 --]
I am using FreeBSD 11.1 -RELEASE-amd64 running on a single 32 core CPU and am having issues with contigmalloc performance.
Timing the function using rdtsc shows that it uses up on average about 10 million cycles on that function along.
Using the same code, FreeBSD and timing method I ran it on anther machine on two CPU's with a total of 32 cores.
This gave about 12 thousand cycles on that function.
Digging through the source code (on the single CPU) I found that
smp_targeted_tlb_shootdown function in /usr/src/sys/x86/x86/mp_x86.c
cause the majority of performance hit due to some cores remaining in a paused state longer after the
interrupt was send to them.
I attached a sample code and Makefile in this email.
Steps to recreate (and show rdtsc cycles):
$ make
$ kldload ./test.ko
$ dmesg
If anyone has any idea what is the cause of this issue, it would be greatly appreciated.
--------------------------------------------------------------
Intel Research and Development Ireland Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263
This e-mail and any attachments may contain confidential material for the sole
use of the intended recipient(s). Any review or distribution by others is
strictly prohibited. If you are not the intended recipient, please contact the
sender and delete all copies.
[-- Attachment #2 --]
# Note: It is important to make sure you include the <bsd.kmod.mk> makefile after declaring the KMOD and SRCS variables.
# Declare Name of kernel module
KMOD = test
# Enumerate Source files for kernel module
SRCS = test.c
# Include kernel module makefile
# /usr/src/share/mk/bsd.kmod.mk
.include <bsd.kmod.mk>
[-- Attachment #3 --]
#include <sys/param.h>
#include <sys/module.h>
#include <sys/kernel.h>
#include <sys/systm.h>
#include <sys/malloc.h>
static void runTest(void);
MALLOC_DECLARE(M_FOO);
MALLOC_DEFINE(M_FOO, "mfoo", "foo mem");
/* The function called at load/unload. */
static int event_handler(struct module *module, int event, void *arg) {
int e = 0; /* Error, 0 for normal return status */
switch (event) {
case MOD_LOAD:
runTest();
break;
case MOD_UNLOAD:
printf("Done\n");
uprintf("Done\n");
break;
default:
e = EOPNOTSUPP; /* Error, Operation Not Supported */
break;
}
return(e);
}
static void runTest() {
const int pageSize = 4096;
unsigned long size;
void *ptr = NULL;
unsigned long long t1, t2;
for(size = 1; size <= (pageSize -1); size <<= 1) {
t1 = rdtsc();
ptr = contigmalloc(size, M_FOO, M_WAITOK, 0, ~0,
pageSize, 0);
t2 = rdtsc();
printf("contigmalloc cycles: %llu\n", t2-t1);
if(ptr == NULL)
return;
t1 = rdtsc();
contigfree(ptr, size, M_FOO);
t2 = rdtsc();
printf("contigfree cycles: %llu\n\n", t2-t1);
uprintf("Loop complete\n");
}
}
/* The second argument of DECLARE_MODULE. */
static moduledata_t test_conf = {
"test", /* module name */
event_handler, /* event handler */
NULL /* extra data */
};
DECLARE_MODULE(test, test_conf, SI_SUB_DRIVERS, SI_ORDER_MIDDLE);
home |
help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?865AA1660A1A014C99D99B800FA40800813681>