From owner-freebsd-drivers@freebsd.org Wed Sep 16 22:36:43 2020 Return-Path: Delivered-To: freebsd-drivers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0CF0D3F3864 for ; Wed, 16 Sep 2020 22:36:43 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gate2.funkthat.com", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BsFMG0kQRz4NX9 for ; Wed, 16 Sep 2020 22:36:41 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.15.2/8.15.2) with ESMTPS id 08GMacMX063526 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 16 Sep 2020 15:36:38 -0700 (PDT) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.15.2/8.15.2/Submit) id 08GMacFC063525; Wed, 16 Sep 2020 15:36:38 -0700 (PDT) (envelope-from jmg) Date: Wed, 16 Sep 2020 15:36:38 -0700 From: John-Mark Gurney To: Warner Losh Cc: Julian Grajkowski , freebsd-drivers@freebsd.org Subject: Re: getpid() performance Message-ID: <20200916223638.GR4213@funkthat.com> Mail-Followup-To: Warner Losh , Julian Grajkowski , freebsd-drivers@freebsd.org References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Operating-System: FreeBSD 11.3-STABLE amd64 X-PGP-Fingerprint: D87A 235F FB71 1F3F 55B7 ED9B D5FF 5A51 C0AC 3D65 X-Files: The truth is out there X-URL: https://www.funkthat.com/ X-Resume: https://www.funkthat.com/~jmg/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.6.1 (2016-04-27) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (gold.funkthat.com [127.0.0.1]); Wed, 16 Sep 2020 15:36:38 -0700 (PDT) X-Rspamd-Queue-Id: 4BsFMG0kQRz4NX9 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of jmg@gold.funkthat.com has no SPF policy when checking 208.87.223.18) smtp.mailfrom=jmg@gold.funkthat.com X-Spamd-Result: default: False [-0.41 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org]; FREEFALL_USER(0.00)[jmg]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[funkthat.com]; AUTH_NA(1.00)[]; NEURAL_SPAM_SHORT(0.03)[0.033]; NEURAL_HAM_LONG(-0.89)[-0.894]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-0.75)[-0.745]; R_SPF_NA(0.00)[no SPF record]; FORGED_SENDER(0.30)[jmg@funkthat.com,jmg@gold.funkthat.com]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:32354, ipnet:208.87.216.0/21, country:US]; FROM_NEQ_ENVFROM(0.00)[jmg@funkthat.com,jmg@gold.funkthat.com]; MAILMAN_DEST(0.00)[freebsd-drivers] X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Sep 2020 22:36:43 -0000 Warner Losh wrote this message on Wed, Sep 16, 2020 at 01:24 -0600: > On Wed, Sep 16, 2020 at 1:15 AM Julian Grajkowski < > julian.grajkowski@gmail.com> wrote: > > > Hi, > > > > I am working on a contiguous memory allocator which frequently calls > > getpid() in user space and I have noticed very poor performance of this > > function call. I measured this call performance using following code: > > > > inline uint64_t rdtsc_start(void) > > { > > uint32_t cycles_high; > > uint32_t cycles_low; > > > > asm volatile("lfence\n\t" > > "rdtscp\n\t" > > "mov %%edx, %0\n\t" > > "mov %%eax, %1\n\t" > > : "=r" (cycles_high), "=r" (cycles_low) > > : : "%rax", "%rdx", "%rcx"); > > > > return (((uint64_t)cycles_high << 32) | cycles_low); > > } > > > > > > inline uint64_t rdtsc_end(void) > > { > > uint32_t cycles_high; > > uint32_t cycles_low; > > > > asm volatile("rdtscp\n\t" > > "mov %%edx, %0\n\t" > > "mov %%eax, %1\n\t" > > "lfence\n\t" > > : "=r" (cycles_high), "=r" (cycles_low) > > : : "%rax", "%rdx", "%rcx"); > > > > return (((uint64_t)cycles_high << 32) | cycles_low); > > } > > > > This way I measured ~320 cycles used for getpid() on FreeBSD 12.1. For > > comparison, in Linux (CentOS 7) this call uses ~10 cycles. I am aware that > > this should not be compared directly. as these are different systems, but > > such a big difference in performance is an issue for me, as getpid() is > > called very often in my code. > > > > Is such a poor performance of getpid() a known problem and is it possible > > that this might be improved in future releases? > > > > glibc optimizes getpid() system call so it only calls it once and returns a > cached value (which is in line with 10 cycles, there's no way you can > save/restore state in 10 cycles, let alone do a dispatch). FreeBSD doesn't. if you really need to see if your process has forked (I assume that is why you're calling getpid so frequently), you can mmap a page, and using minherit's INHERIT_ZERO so that all the data in that page will be zero'd on fork. You can then change your getpid check to something like: pid_t *page_with_inherit_zero_set; pid_t my_getpid() { if (page_with_inherit_zero_set == NULL) allocate_page_and_set_inherit_zero(); if (*page_with_inherit_zero_set == 0) { *page_with_inherit_zero_set = getpid(); return *page_with_inherit_zero_set; } and you should see similar improvements. Though this might allow you to move this logic to a better place in your code. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."