From owner-freebsd-drivers@freebsd.org Wed Aug 12 06:24:26 2020 Return-Path: Delivered-To: freebsd-drivers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 348EA3BF3AD for ; Wed, 12 Aug 2020 06:24:26 +0000 (UTC) (envelope-from julian.grajkowski@gmail.com) Received: from mail-il1-x130.google.com (mail-il1-x130.google.com [IPv6:2607:f8b0:4864:20::130]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BRKRY3hgjz4WBx for ; Wed, 12 Aug 2020 06:24:25 +0000 (UTC) (envelope-from julian.grajkowski@gmail.com) Received: by mail-il1-x130.google.com with SMTP id q14so700565ilj.8 for ; Tue, 11 Aug 2020 23:24:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=6MNoUjIDDx4MSFz/bPPLjYJ7PjI2Grn7u67mEDc2Bw4=; b=T3wRbRL1H8PPCi3z67ha8APlEH1NgdM93BwTY20RRDiwjt3ixZSqVoZaV8OhZAvyyV ARICvnOLPmhPxTbG76MuYVsSPwIEIi3IOfzVMVP7HZp79Y2zaL7Aephe6o6BcREsFmWB lncsnNUmAfOVogOMJlpXCF7W4DBHfBuxCtA477K0E0PsMUkQDXc1Vr3t/ekCo10GJ3ow rX62MTfJyQKZV7+BOdxpbSScChaHh5pKyo4Bc3+akh8hmRSQRNm1iU4MfrgT06TKUR0Z xCGg8UhzqABJ2ofr3ZCFne9+2BE+3njm2Bdx75uSOxHS02hao0/5PKWXl4DxUStJ/OHl frFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=6MNoUjIDDx4MSFz/bPPLjYJ7PjI2Grn7u67mEDc2Bw4=; b=JTzp8/v91+PILJmnJ3c8/NVmKaTh4y7hZZGtu0UcrWIbDsi6akagjPgEX6HJWZpJ0Y KWiJMAyEkbVIWNHWCQ1fiE8lKGuJkFmf7AtV1abWS1o+cHopFaltNgYqmNa7OPHlkuIJ 1HzZ6ouQdtHDM2jPO1Ucrfbu6/7Zh9RAwGbX6xIpNG5vfCwVa6H4dZj635AhHX1SJY0W XJV/f9kEDvIyCHJUC1OR/x+i0KtdVVa5860NiOjgYx1sHSbgVYfxsqKtOcOD/KM0eRzL J3yopYLYfdH7fG+dfscW8JSrzkrIkbkpfKp25fMiLwP+Di1I789xvblz2ykgGxL7BCpk S3mQ== X-Gm-Message-State: AOAM530Psnq8BJhGiTqqdXBdvPdcsuY0fhvyeP9zFRCBWs+onEY46XWU jO/WVFF7AyTqqORF0XQDl70PNUYDqknfKQE+YXNOTXY= X-Google-Smtp-Source: ABdhPJwpHG+FlW+RFwk4YYpdF2gbVz/uDVLFXVdEGURRMxySbnq6lHrGaivfmEDG3xZn/UK9a2iCuLcMHeQYGhZrDho= X-Received: by 2002:a92:874a:: with SMTP id d10mr26951481ilm.273.1597213463948; Tue, 11 Aug 2020 23:24:23 -0700 (PDT) MIME-Version: 1.0 From: Julian Grajkowski Date: Wed, 12 Aug 2020 08:24:12 +0200 Message-ID: Subject: contigmalloc and contigfree issue To: freebsd-drivers@freebsd.org X-Rspamd-Queue-Id: 4BRKRY3hgjz4WBx X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=T3wRbRL1; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of juliangrajkowski@gmail.com designates 2607:f8b0:4864:20::130 as permitted sender) smtp.mailfrom=juliangrajkowski@gmail.com X-Spamd-Result: default: False [-3.09 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.987]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/mixed,multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-drivers@freebsd.org]; TO_DN_NONE(0.00)[]; HAS_ATTACHMENT(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.01)[-1.007]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::130:from]; NEURAL_HAM_SHORT(-0.09)[-0.093]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:~,4:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Aug 2020 06:24:26 -0000 Hi, I am currently developing a driver, which uses quite large (2M+) contiguous memory chunks allocations via contigmalloc. I have noticed that contigmalloc as well as contigfree take a lot of time when dealing with large memory chunks and also a rather significant difference in number of cycles required for freeing the memory via contigfree between FreeBSD 11.3 and 12.1. I have checked this on the same machine with both 11.3-p11 and 12.1-p7 by running contigmalloc (with and without M_NOWAIT) and contigfree in a loop 40 times and the average results are following: *11.3-p11* *M_NOWAIT* *contigmalloc 179112 contigfree 211302* *No flag* *contigmalloc 176541 contigfree 199462* *12.1-p7* *M_NOWAIT* *contigmalloc 171769 contigfree 127562* *No flag* *contigmalloc 171974 contigfree 127448* Following code was used for measuring: [code] static void runTest() { const int pageSize = 4096; unsigned long size; void *ptr = NULL; unsigned long long t1, t2; int i = 0; unsigned long long avg_malloc = 0, avg_free = 0; int probes = 40; size = 2 * 1024 * 1024; for (i = 0; i < probes; i++) { t1 = rdtsc(); ptr = contigmalloc(size, M_FOO, M_NOWAIT, 0, ~0, pageSize, 0); t2 = rdtsc(); avg_malloc += t2 - t1; printf("contigmalloc cycles: %llu\n", t2-t1); if (ptr == NULL) return; t1 = rdtsc(); contigfree(ptr, size, M_FOO); t2 = rdtsc(); avg_free += t2 - t1; printf("contigfree cycles: %llu\n\n", t2-t1); uprintf("Loop complete\n"); } printf("contigmalloc %llu contigfree %llu\n", avg_malloc / probes, avg_free / probes); } [/code] The measurements were done in a following setup: CPU: Intel(R) Atom(TM) CPU C3958 @ 2.00GHz (2000.06-MHz K8-class CPU) FreeBSD/SMP: Multiprocessor System Detected: 16 CPUs 8GB RAM (2x4GB): Type: DDR4 Type Detail: Synchronous Unbuffered (Unregistered) Speed: 2400 MT/s Could someone please provide some explanation of discussed performance of contigmalloc and contigfree? Also, what could be the reason for improved performance of contigfree on 12.1? I still have not figured out what is the source of this behavior. Is performance of contigmalloc and contigfree a known issue? I have not found any info about this problem on the forum. Please find the complete source of the module used for measurements in attachment. Thank you in advance for any help. Kind regards, Julian