From nobody Mon Jun 28 11:20:25 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4F15511CA30B for ; Mon, 28 Jun 2021 11:20:41 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GD4sj03rmz4nS4; Mon, 28 Jun 2021 11:20:40 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 15SBKQUK001356 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 28 Jun 2021 14:20:29 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 15SBKQUK001356 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 15SBKPxx001339; Mon, 28 Jun 2021 14:20:25 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 28 Jun 2021 14:20:25 +0300 From: Konstantin Belousov To: Dmitry Chagin Cc: freebsd-hackers@freebsd.org Subject: Re: pondering pi futexes Message-ID: References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.5 X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on tom.home X-Rspamd-Queue-Id: 4GD4sj03rmz4nS4 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On Sun, Jun 27, 2021 at 10:39:35PM +0300, Dmitry Chagin wrote: > > Hi, > some time ago I have changed Linuxulator futexes from sx lock to mtx. > sx was used as it allows copyin/copyout with sx lock held. > to use mtx I have changed the code like: > 1. lock mtx; > 2. disable_pagefaults; > 3. copyin() > 4. enable_pagefaults; > 5. if error > - unlock mtx; > copyin(); > if error == 0 goto 1. > > it works (needto replace copyin() by fueword32()), but pondering pi futexes > imlementation, I see that it is not possible to drop the futex lock on a return > from msleep() path. > > below a simplified FUTEX_LOCK_PI operation, where on enter to the kernel current thread: > > 0. acquire futex lock (which is mtx) > 1. cmpset(0 -> current thread TID), return (0) on success; > 2. fetch() from futex *uaddr (for TID of owner): > - check EDEADLK case (the futex word at *uaddr is already locked by the caller); > - check that is no waiters on *uaddr exists which is waiting via FUTEX_WAIT or > FUTEX_WAIT_BITSET, return (EINVAL) if so; > - cmpset(TID -> (FUTEX_WAITERS|TID)); > - on error, the futex owner changed in user-space, repeat from 1. > 3. Here we have: the owner, one waiter (current thread) and 0 or more waiters > sleeping on a waiting_proc. FUTEX_WAITERS bit is set, so any new waiters go to > the kernel and owner should unlock futex via the FUTEX_UNLOCK_PI op; > 4. Try to find the thread which is associated with the owner’s TID: > - on error, something bad happened, owner died? Clean owner state link? > return (ESRCH). Or if no other waiters? Check this... > - on success: > - save owner state link to the struct futex (save priority); > - check the owner's priority, bump it if needed; > - put the current thread to the waiters list in descending priority order; > - change priority of all waiters if needed; > - msleep on a futex waiting_proc; come back with futex lock held; > - restore own priority? If last waiter?; [ponders..] > - on timeout return (ETIMEDOUT); > - the current thread is the new owner: > bah!! - store() the owner TID to *uaddr; [check what should I do on error..] > - release futex lock; > - return (0). > > is it possible to hold *uaddr page to prevent page faults? I did not followed exact algorithm you trying to describe. Still, I can make two points which could be useful for formulation of the working solution. 1. Umtx AKA FreeBSD native implementation of something very similar to futex, has a concept of the umtx queue chain. The chain owns the mutex lock used for 'fast' ops, but for situations like accesses to userspace, we 'busy' the umtxq chain. De-facto busy state is the hand-rolled sleepable lock, with usual interlocking against chain mutex, and msleep/wakeup inter-thread notifications. 2. If you insist on accessing userspace pages while owning non-sleepable locks, or even sleepable locks which do not compose cleanly with whole VM+VFS+network (the later due to NFS) locking protocols, we do have a technique that can be used. Althought it is complicated and sometimes slow. You already noted half of it above, namely disable of the page faults and copyin_nofault-like operations. Second half is prefaulting, which is done by vm_fault_quick_hold_pages(). There are two ways you can use it. Either you hold the target userspace page, then remap it into KVA and access userspace object through the kernel mapping. This assumes that the object fits into page, also it does not interact with parallel forks and CoW, which you either have to accept to recheck after. The usual races, e.g. userspace remapping the held (actually wired) page under us, is there, of course, but it is a race anyway. Or, you hold the page but still access it using copyin_nofault. Then any reported error from copyin becomes fatal instead of an indicator to retry. Error can happen for the same reason, because userspace could unmap or remap this address under us. Umtx implementation only uses busy on chains, might be because _nofault+ vm_fault_quick_hold() appeared later. From nobody Tue Jun 29 21:19:16 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 3D7FB11C8302 for ; Tue, 29 Jun 2021 21:19:30 +0000 (UTC) (envelope-from micchie.gml@gmail.com) Received: from mail-oi1-x22a.google.com (mail-oi1-x22a.google.com [IPv6:2607:f8b0:4864:20::22a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GDy690tV0z4TM9 for ; Tue, 29 Jun 2021 21:19:29 +0000 (UTC) (envelope-from micchie.gml@gmail.com) Received: by mail-oi1-x22a.google.com with SMTP id a133so370527oib.13 for ; Tue, 29 Jun 2021 14:19:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=3nFGwxY12kTgAEb7ubX1p3VYwb5v5GCOtjh0xdPGPXk=; b=SzARAIlYtskQa6n3SMMdMOO4WhN1d8BNHQn86VlBoWKINkpoNaEFIfnELtrBr6m/Y3 wlWke+DkUdOwS5B897t6FyizmHsVZM46Vk6YTFDFql4scL+4iaJXPpIrpkeD7Ev61nAf qs0gCtPlspsYZ3P77Doa/9VBk+0X/wiuDhhhUgJniTPjGP39vv7Jpbr2DKt5/fHprNUJ XudJKbm2xlhlocEnXWSLVlZze8aqZAsgRJ5wf2zRqZRyJvDVwVfNvTgf1Jhg9GBLzzgQ gAnghulHriWhtS8wtZWphHlIxYpcGAZrDmzLhRROFJdHBSW3AhWuTxO+YePX75+dhpDw bAng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=3nFGwxY12kTgAEb7ubX1p3VYwb5v5GCOtjh0xdPGPXk=; b=WNGxrxmGV/xD/aj60tRJCceADF5PtaapXtpW6pLPlvPuIxfZyR9qIUAg04AVHB6rpS XR/W2n/VUrS3HnN6hAWzxFnpNJYmO8irO5cLX8WmbE/5HHSV4FIwBtk+wB4yLgfKvqtr dVqdei10ZC1thLVRxJTYglpbK+uHiK/TefCT6c0+NdeNPvhiW4Gg+okgqgEvMqyT79L3 7OCBKxb0Thj+jPGGEy52/DWG5zg3a76H1/HqNlZWDPwXvl0XuGICAxhJ3pDTFtzfSyY5 0e1cpkdo68hLIm6DhXy/Zf6nQydwzy5vbI4mCLiKVxzTaD3ORfAVGa2X7kA+vK6a8+ig UiwQ== X-Gm-Message-State: AOAM532W3sl4OJVhqoUgcoV6LZ9DECF3dgjqwb8Cq4K2FpF7HWqHOZMC qJ0R1MiE0SGBLL8IqBVcckZx3IeieXu/DuQEIupyEFd/ah4XeQ== X-Google-Smtp-Source: ABdhPJwO7b6ZBvOHOAOAGMzsj8B2R6/9AkCEJgrL7Lf8/04gDd8s3DAyF0Yiz6GzsIeaJ2IpnVLzqlbnNw4QZYeKads= X-Received: by 2002:aca:d481:: with SMTP id l123mr12457125oig.73.1625001567438; Tue, 29 Jun 2021 14:19:27 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 From: Michio Honda Date: Tue, 29 Jun 2021 22:19:16 +0100 Message-ID: Subject: Incorrect credits in iflib_tx_credits_update() in sys/net/iflib.c To: freebsd-hackers@freebsd.org Content-Type: multipart/alternative; boundary="0000000000002fcb0205c5ee2962" X-Rspamd-Queue-Id: 4GDy690tV0z4TM9 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=SzARAIlY; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of micchiegml@gmail.com designates 2607:f8b0:4864:20::22a as permitted sender) smtp.mailfrom=micchiegml@gmail.com X-Spamd-Result: default: False [-2.95 / 15.00]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::22a:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::22a:from:127.0.2.255]; NEURAL_SPAM_SHORT(0.05)[0.054]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::22a:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers] X-ThisMailContainsUnwantedMimeParts: Y --0000000000002fcb0205c5ee2962 Content-Type: text/plain; charset="UTF-8" Hi, I noticed that that function assumes ctx->isc_txd_credits_update() returns at most twice of txq->ift_size (the number of ring buffers) - 1, but I noticed em_isc_txd_credits_update() can return a much larger value. As a result, too large txq->ift_cidx_processed is set and thus iflib_netmap_txsync sets incorrect nr_hwtail. I am not sure this should be fixed in the driver or iflib - could someone take a look at this? Cheers, - Michio --0000000000002fcb0205c5ee2962-- From nobody Wed Jun 30 11:07:56 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 3838811E137D for ; Wed, 30 Jun 2021 11:08:04 +0000 (UTC) (envelope-from dchagin@heemeyer.club) Received: from heemeyer.club (heemeyer.club [IPv6:2001:19f0:6400:80a1:5054:ff:fe7a:a27d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GFJVC1Z3mz3LDT for ; Wed, 30 Jun 2021 11:08:02 +0000 (UTC) (envelope-from dchagin@heemeyer.club) Received: from heemeyer.club (localhost [127.0.0.1]) by heemeyer.club (8.16.1/8.16.1) with ESMTP id 15UB7uSJ048515; Wed, 30 Jun 2021 14:07:56 +0300 (MSK) (envelope-from dchagin@heemeyer.club) Received: (from dchagin@localhost) by heemeyer.club (8.16.1/8.16.1/Submit) id 15UB7u2P048514; Wed, 30 Jun 2021 14:07:56 +0300 (MSK) (envelope-from dchagin) Date: Wed, 30 Jun 2021 14:07:56 +0300 From: Dmitry Chagin To: Konstantin Belousov Cc: freebsd-hackers@freebsd.org Subject: Re: pondering pi futexes Message-ID: References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspamd-Queue-Id: 4GFJVC1Z3mz3LDT X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of dchagin@heemeyer.club has no SPF policy when checking 2001:19f0:6400:80a1:5054:ff:fe7a:a27d) smtp.mailfrom=dchagin@heemeyer.club X-Spamd-Result: default: False [-1.79 / 15.00]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[dchagin]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[freebsd.org]; RBL_DBL_DONT_QUERY_IPS(0.00)[2001:19f0:6400:80a1:5054:ff:fe7a:a27d:from]; AUTH_NA(1.00)[]; SPAMHAUS_ZRD(0.00)[2001:19f0:6400:80a1:5054:ff:fe7a:a27d:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.99)[-0.995]; RCPT_COUNT_TWO(0.00)[2]; R_SPF_NA(0.00)[no SPF record]; FREEMAIL_TO(0.00)[gmail.com]; FORGED_SENDER(0.30)[dchagin@freebsd.org,dchagin@heemeyer.club]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:20473, ipnet:2001:19f0:6400::/38, country:US]; FROM_NEQ_ENVFROM(0.00)[dchagin@freebsd.org,dchagin@heemeyer.club]; MAILMAN_DEST(0.00)[freebsd-hackers]; RCVD_COUNT_TWO(0.00)[2] X-ThisMailContainsUnwantedMimeParts: N On Mon, Jun 28, 2021 at 02:20:25PM +0300, Konstantin Belousov wrote: > On Sun, Jun 27, 2021 at 10:39:35PM +0300, Dmitry Chagin wrote: > > > > Hi, > > some time ago I have changed Linuxulator futexes from sx lock to mtx. > > sx was used as it allows copyin/copyout with sx lock held. > > to use mtx I have changed the code like: > > 1. lock mtx; > > 2. disable_pagefaults; > > 3. copyin() > > 4. enable_pagefaults; > > 5. if error > > - unlock mtx; > > copyin(); > > if error == 0 goto 1. > > > > it works (needto replace copyin() by fueword32()), but pondering pi futexes > > imlementation, I see that it is not possible to drop the futex lock on a return > > from msleep() path. > > > > below a simplified FUTEX_LOCK_PI operation, where on enter to the kernel current thread: > > > > 0. acquire futex lock (which is mtx) > > 1. cmpset(0 -> current thread TID), return (0) on success; > > 2. fetch() from futex *uaddr (for TID of owner): > > - check EDEADLK case (the futex word at *uaddr is already locked by the caller); > > - check that is no waiters on *uaddr exists which is waiting via FUTEX_WAIT or > > FUTEX_WAIT_BITSET, return (EINVAL) if so; > > - cmpset(TID -> (FUTEX_WAITERS|TID)); > > - on error, the futex owner changed in user-space, repeat from 1. > > 3. Here we have: the owner, one waiter (current thread) and 0 or more waiters > > sleeping on a waiting_proc. FUTEX_WAITERS bit is set, so any new waiters go to > > the kernel and owner should unlock futex via the FUTEX_UNLOCK_PI op; > > 4. Try to find the thread which is associated with the owner’s TID: > > - on error, something bad happened, owner died? Clean owner state link? > > return (ESRCH). Or if no other waiters? Check this... > > - on success: > > - save owner state link to the struct futex (save priority); > > - check the owner's priority, bump it if needed; > > - put the current thread to the waiters list in descending priority order; > > - change priority of all waiters if needed; > > - msleep on a futex waiting_proc; come back with futex lock held; > > - restore own priority? If last waiter?; [ponders..] > > - on timeout return (ETIMEDOUT); > > - the current thread is the new owner: > > bah!! - store() the owner TID to *uaddr; [check what should I do on error..] > > - release futex lock; > > - return (0). > > > > is it possible to hold *uaddr page to prevent page faults? > > I did not followed exact algorithm you trying to describe. Still, I can make > two points which could be useful for formulation of the working solution. > > 1. Umtx AKA FreeBSD native implementation of something very similar to > futex, has a concept of the umtx queue chain. The chain owns the mutex > lock used for 'fast' ops, but for situations like accesses to userspace, > we 'busy' the umtxq chain. De-facto busy state is the hand-rolled > sleepable lock, with usual interlocking against chain mutex, and > msleep/wakeup inter-thread notifications. > > 2. If you insist on accessing userspace pages while owning non-sleepable > locks, or even sleepable locks which do not compose cleanly with whole > VM+VFS+network (the later due to NFS) locking protocols, we do have a > technique that can be used. Althought it is complicated and sometimes > slow. > > You already noted half of it above, namely disable of the page faults and > copyin_nofault-like operations. Second half is prefaulting, which is > done by vm_fault_quick_hold_pages(). There are two ways you can use it. > > Either you hold the target userspace page, then remap it into KVA and > access userspace object through the kernel mapping. This assumes that > the object fits into page, also it does not interact with parallel forks > and CoW, which you either have to accept to recheck after. The usual > races, e.g. userspace remapping the held (actually wired) page under us, > is there, of course, but it is a race anyway. > > Or, you hold the page but still access it using copyin_nofault. Then any > reported error from copyin becomes fatal instead of an indicator to retry. > Error can happen for the same reason, because userspace could unmap or > remap this address under us. > > Umtx implementation only uses busy on chains, might be because _nofault+ > vm_fault_quick_hold() appeared later. Thank you for the clarification, I wrote a simple busy code, and I don't like it, will try pre-faulting, thaks for pointing it out. From nobody Thu Jul 1 19:49:15 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 45A8211D4B05 for ; Thu, 1 Jul 2021 19:49:17 +0000 (UTC) (envelope-from leres@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GG8191dRvz4rBc for ; Thu, 1 Jul 2021 19:49:17 +0000 (UTC) (envelope-from leres@freebsd.org) Received: from ice.alameda.xse.com (unknown [IPv6:2600:1700:a570:e20:f2ad:4eff:fe09:150e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: leres) by smtp.freebsd.org (Postfix) with ESMTPSA id E332C20270 for ; Thu, 1 Jul 2021 19:49:16 +0000 (UTC) (envelope-from leres@freebsd.org) From: Craig Leres To: freebsd-hackers@freebsd.org Subject: firefox is slow and uses a lot of memory Message-ID: <37fe2be4-a142-c49e-e1b0-0e9a8fd07a74@freebsd.org> Date: Thu, 1 Jul 2021 12:49:15 -0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-ThisMailContainsUnwantedMimeParts: N Starting with firefox 89 or so, I've found it starts using a LOT of memory and actually makes the system sluggish and causes momentary X freezes. I have 32G of ram/swap, is it reasonable for firefox to use 15G? Craig ice 95 % uname -a FreeBSD ice.alameda.xse.com 12.2-RELEASE-p8 FreeBSD 12.2-RELEASE-p8 r18 LBL amd64 >>>> top -osize last pid: 71913; load averages: 0.52, 0.53, 0.47 up 1+00:29:04 12:46:04 215 processes: 2 running, 206 sleeping, 6 stopped, 1 zombie CPU: 1.9% user, 0.0% nice, 0.3% system, 0.1% interrupt, 97.7% idle Mem: 2931M Active, 3479M Inact, 1874M Laundry, 19G Wired, 1192M Buf, 4116M Free ARC: 13G Total, 3822M MFU, 8686M MRU, 172K Anon, 270M Header, 553M Other 11G Compressed, 81G Uncompressed, 7.38:1 Ratio Swap: 32G Total, 32G Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 63858 leres 35 20 0 15G 506M select 7 0:40 0.08% firefox 3610 leres 3 20 0 6268M 61M select 2 4:54 1.00% Xorg 3159 telegraf 19 23 0 4926M 47M uwait 3 0:26 0.00% telegraf 63845 leres 125 20 0 4612M 1794M select 7 18:49 4.61% firefox 33553 leres 60 20 0 4579M 2077M CPU2 2 9:18 3.10% thunderb 63859 leres 34 20 0 2916M 559M select 3 1:11 0.08% firefox 63857 leres 34 20 0 2812M 466M select 0 2:00 0.18% firefox 63850 leres 37 20 0 2723M 377M select 2 1:51 0.24% firefox 63849 leres 34 20 0 2708M 394M select 3 1:05 0.02% firefox 63856 leres 34 20 0 2608M 275M select 6 0:30 0.01% firefox 63861 leres 34 20 0 2491M 189M select 6 0:05 0.05% firefox 63847 leres 34 20 0 2471M 171M select 1 0:04 0.05% firefox 70198 leres 27 20 0 2436M 142M select 5 0:01 0.05% firefox 63860 leres 26 20 0 2422M 130M select 2 0:01 0.00% firefox 3132 influxd 18 52 0 997M 185M uwait 2 6:16 0.06% influxd 3214 root 6 20 0 510M 491M select 0 2:38 0.19% pcscd 2897 bind 26 52 0 349M 281M sigwai 0 0:15 0.00% named 22334 www 4 52 0 108M 77M select 3 0:02 0.00% httpd From nobody Thu Jul 1 21:53:01 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id E939511DEA68 for ; Thu, 1 Jul 2021 21:53:03 +0000 (UTC) (envelope-from dchagin@heemeyer.club) Received: from heemeyer.club (heemeyer.club [IPv6:2001:19f0:6400:80a1:5054:ff:fe7a:a27d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GGBlz2DDSz3HpF for ; Thu, 1 Jul 2021 21:53:03 +0000 (UTC) (envelope-from dchagin@heemeyer.club) Received: from heemeyer.club (localhost [127.0.0.1]) by heemeyer.club (8.16.1/8.16.1) with ESMTP id 161Lr19M004147; Fri, 2 Jul 2021 00:53:01 +0300 (MSK) (envelope-from dchagin@heemeyer.club) Received: (from dchagin@localhost) by heemeyer.club (8.16.1/8.16.1/Submit) id 161Lr1iq004146; Fri, 2 Jul 2021 00:53:01 +0300 (MSK) (envelope-from dchagin) Date: Fri, 2 Jul 2021 00:53:01 +0300 From: Dmitry Chagin To: Konstantin Belousov Cc: freebsd-hackers@freebsd.org Subject: Re: pondering pi futexes Message-ID: References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspamd-Queue-Id: 4GGBlz2DDSz3HpF X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of dchagin@heemeyer.club has no SPF policy when checking 2001:19f0:6400:80a1:5054:ff:fe7a:a27d) smtp.mailfrom=dchagin@heemeyer.club X-Spamd-Result: default: False [-1.80 / 15.00]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[dchagin]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.997]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[freebsd.org]; RBL_DBL_DONT_QUERY_IPS(0.00)[2001:19f0:6400:80a1:5054:ff:fe7a:a27d:from]; AUTH_NA(1.00)[]; SPAMHAUS_ZRD(0.00)[2001:19f0:6400:80a1:5054:ff:fe7a:a27d:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_TWO(0.00)[2]; R_SPF_NA(0.00)[no SPF record]; FREEMAIL_TO(0.00)[gmail.com]; FORGED_SENDER(0.30)[dchagin@freebsd.org,dchagin@heemeyer.club]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:20473, ipnet:2001:19f0:6400::/38, country:US]; FROM_NEQ_ENVFROM(0.00)[dchagin@freebsd.org,dchagin@heemeyer.club]; MAILMAN_DEST(0.00)[freebsd-hackers]; RCVD_COUNT_TWO(0.00)[2] X-ThisMailContainsUnwantedMimeParts: N On Mon, Jun 28, 2021 at 02:20:25PM +0300, Konstantin Belousov wrote: > On Sun, Jun 27, 2021 at 10:39:35PM +0300, Dmitry Chagin wrote: > > > > Hi, > > some time ago I have changed Linuxulator futexes from sx lock to mtx. > > sx was used as it allows copyin/copyout with sx lock held. > > to use mtx I have changed the code like: > > 1. lock mtx; > > 2. disable_pagefaults; > > 3. copyin() > > 4. enable_pagefaults; > > 5. if error > > - unlock mtx; > > copyin(); > > if error == 0 goto 1. > > > > it works (needto replace copyin() by fueword32()), but pondering pi futexes > > imlementation, I see that it is not possible to drop the futex lock on a return > > from msleep() path. > > > > below a simplified FUTEX_LOCK_PI operation, where on enter to the kernel current thread: > > > > 0. acquire futex lock (which is mtx) > > 1. cmpset(0 -> current thread TID), return (0) on success; > > 2. fetch() from futex *uaddr (for TID of owner): > > - check EDEADLK case (the futex word at *uaddr is already locked by the caller); > > - check that is no waiters on *uaddr exists which is waiting via FUTEX_WAIT or > > FUTEX_WAIT_BITSET, return (EINVAL) if so; > > - cmpset(TID -> (FUTEX_WAITERS|TID)); > > - on error, the futex owner changed in user-space, repeat from 1. > > 3. Here we have: the owner, one waiter (current thread) and 0 or more waiters > > sleeping on a waiting_proc. FUTEX_WAITERS bit is set, so any new waiters go to > > the kernel and owner should unlock futex via the FUTEX_UNLOCK_PI op; > > 4. Try to find the thread which is associated with the owner’s TID: > > - on error, something bad happened, owner died? Clean owner state link? > > return (ESRCH). Or if no other waiters? Check this... > > - on success: > > - save owner state link to the struct futex (save priority); > > - check the owner's priority, bump it if needed; > > - put the current thread to the waiters list in descending priority order; > > - change priority of all waiters if needed; > > - msleep on a futex waiting_proc; come back with futex lock held; > > - restore own priority? If last waiter?; [ponders..] > > - on timeout return (ETIMEDOUT); > > - the current thread is the new owner: > > bah!! - store() the owner TID to *uaddr; [check what should I do on error..] > > - release futex lock; > > - return (0). > > > > is it possible to hold *uaddr page to prevent page faults? > > I did not followed exact algorithm you trying to describe. Still, I can make > two points which could be useful for formulation of the working solution. > > 1. Umtx AKA FreeBSD native implementation of something very similar to > futex, has a concept of the umtx queue chain. The chain owns the mutex > lock used for 'fast' ops, but for situations like accesses to userspace, > we 'busy' the umtxq chain. De-facto busy state is the hand-rolled > sleepable lock, with usual interlocking against chain mutex, and > msleep/wakeup inter-thread notifications. > reading umtx impl I see umtxq_sleep drop lock (PDROP bit is set) and, in case of a signal, reacquire lock and breaks from loop. is there a possible small window for wakeup loss? as ERESTART returned, or the caller should to take care of this? Or I missing something? Im trying to reuse the umtx code for futexes, at least queue chain. Thanks in advance! > 2. If you insist on accessing userspace pages while owning non-sleepable > locks, or even sleepable locks which do not compose cleanly with whole > VM+VFS+network (the later due to NFS) locking protocols, we do have a > technique that can be used. Althought it is complicated and sometimes > slow. > > You already noted half of it above, namely disable of the page faults and > copyin_nofault-like operations. Second half is prefaulting, which is > done by vm_fault_quick_hold_pages(). There are two ways you can use it. > > Either you hold the target userspace page, then remap it into KVA and > access userspace object through the kernel mapping. This assumes that > the object fits into page, also it does not interact with parallel forks > and CoW, which you either have to accept to recheck after. The usual > races, e.g. userspace remapping the held (actually wired) page under us, > is there, of course, but it is a race anyway. > > Or, you hold the page but still access it using copyin_nofault. Then any > reported error from copyin becomes fatal instead of an indicator to retry. > Error can happen for the same reason, because userspace could unmap or > remap this address under us. > > Umtx implementation only uses busy on chains, might be because _nofault+ > vm_fault_quick_hold() appeared later. From nobody Thu Jul 1 23:41:34 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 644E511F08CF for ; Thu, 1 Jul 2021 23:41:38 +0000 (UTC) (envelope-from sblachmann@gmail.com) Received: from mail-lj1-x231.google.com (mail-lj1-x231.google.com [IPv6:2a00:1450:4864:20::231]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GGF9G2HFmz3kTh; Thu, 1 Jul 2021 23:41:38 +0000 (UTC) (envelope-from sblachmann@gmail.com) Received: by mail-lj1-x231.google.com with SMTP id q4so10788256ljp.13; Thu, 01 Jul 2021 16:41:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=tlTGxkCaheZunq8mxxpZIHuatFMT4vrtPv8WgriT3aM=; b=qhHF6HMnxWn/tNs+LicgvCn+Z7Pq8MQUaPAPnh/mKCnPQko/YVYyn3PFyOi36Fwv7t 9O3+F4Vnvd8Pt3vi39oqoV9e5JzrDdvZenYvPkMp+SnouG6mFmX/YIyJqLKFgduK/Cd6 /dT4MbLpo4ZTdMEQOj6w8vbjy2oJ00J+9Y0CMqkt8jyzKaNEU1YnEc+BGv4u+X8hNSon hnP/7BqzDXVIUFsE4murLiA6WJALWorqvSVdHEdB1pDsTD2Eot9rGGlmPA0wckpTbTsZ M7Fq61piv6Ki01jHk2UM1wxgJ/Cloibrhq/Dbx0/vBBQ4pZt9lcvc6y0dYebWTuA7L7U +X0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=tlTGxkCaheZunq8mxxpZIHuatFMT4vrtPv8WgriT3aM=; b=WilXsZ7SYakvOStR/wTOudrt2KvmXjXuuYHUpRbleq+ddlQGF4hi46+nbPTQJTmaH+ HC2tVyNAiyaT2QKeBDkrDW0u6+7JpssvfOQH+wTYsj8RNw3/Xjnp6Xt2bXsaKzsvnz7U fx2jLC+F920KKuCmuRdUQRjpICBEcRvHNNxZfffTqP/syWIWmkHRSgnuFxKwI4+sewLt btBgkj1pyB9Vpn2TwXYzMqIo/rmFVmsRbgwFKv6aV8W+df1/YSL/opv/Y35HD0knWV6e hHrL3YUTPeUTFlKXvn/1WDoFYgjw3tRshcE+2x/mcjEcp3R86fQpHkJnayvD7RGNypmL IUtA== X-Gm-Message-State: AOAM532z59OeocrA4szGLPaT/AP7kA1bZVgTkuMrcIhHbSuJfzhKM4qO idfbn3ZX89kTBoOI3GD5lqMKJcwdwLkvMdIQXR+8mbm1 X-Google-Smtp-Source: ABdhPJzqJ52/TUaMfWPDM/i8fViNgooSslm/q21Ub0tmRvaPoOL+3lc9DCTIor/OViSVZhWqYfFVS87UfOJpYHXPq9U= X-Received: by 2002:a2e:8e2d:: with SMTP id r13mr1594906ljk.262.1625182895914; Thu, 01 Jul 2021 16:41:35 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Received: by 2002:a2e:9c1a:0:0:0:0:0 with HTTP; Thu, 1 Jul 2021 16:41:34 -0700 (PDT) In-Reply-To: <37fe2be4-a142-c49e-e1b0-0e9a8fd07a74@freebsd.org> References: <37fe2be4-a142-c49e-e1b0-0e9a8fd07a74@freebsd.org> From: Stefan Blachmann Date: Fri, 2 Jul 2021 01:41:34 +0200 Message-ID: Subject: Re: firefox is slow and uses a lot of memory To: Craig Leres Cc: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4GGF9G2HFmz3kTh X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N FF is an insane memory hog. Still v. 88 here. According to "ps axu", currently uses ~25GB resident. "virtual size" ~60GB. BTW... # pkg install www/chromium Updating FreeBSD repository catalogue... FreeBSD repository is up to date. All repositories are up to date. pkg: No packages available to install matching 'www/chromium' have been found in the repositories # Any idea why I cannot install Chromium on 12.2? On 7/1/21, Craig Leres wrote: > Starting with firefox 89 or so, I've found it starts using a LOT of > memory and actually makes the system sluggish and causes momentary X > freezes. I have 32G of ram/swap, is it reasonable for firefox to use 15G? > > Craig > > ice 95 % uname -a > FreeBSD ice.alameda.xse.com 12.2-RELEASE-p8 FreeBSD 12.2-RELEASE-p8 r18 > LBL amd64 > > > >>>> top -osize > > last pid: 71913; load averages: 0.52, 0.53, 0.47 up 1+00:29:04 > 12:46:04 > 215 processes: 2 running, 206 sleeping, 6 stopped, 1 zombie > CPU: 1.9% user, 0.0% nice, 0.3% system, 0.1% interrupt, 97.7% idle > Mem: 2931M Active, 3479M Inact, 1874M Laundry, 19G Wired, 1192M Buf, > 4116M Free > ARC: 13G Total, 3822M MFU, 8686M MRU, 172K Anon, 270M Header, 553M Other > 11G Compressed, 81G Uncompressed, 7.38:1 Ratio > Swap: 32G Total, 32G Free > > PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU > COMMAND > 63858 leres 35 20 0 15G 506M select 7 0:40 0.08% > firefox > 3610 leres 3 20 0 6268M 61M select 2 4:54 1.00% > Xorg > 3159 telegraf 19 23 0 4926M 47M uwait 3 0:26 0.00% > telegraf > 63845 leres 125 20 0 4612M 1794M select 7 18:49 4.61% > firefox > 33553 leres 60 20 0 4579M 2077M CPU2 2 9:18 3.10% > thunderb > 63859 leres 34 20 0 2916M 559M select 3 1:11 0.08% > firefox > 63857 leres 34 20 0 2812M 466M select 0 2:00 0.18% > firefox > 63850 leres 37 20 0 2723M 377M select 2 1:51 0.24% > firefox > 63849 leres 34 20 0 2708M 394M select 3 1:05 0.02% > firefox > 63856 leres 34 20 0 2608M 275M select 6 0:30 0.01% > firefox > 63861 leres 34 20 0 2491M 189M select 6 0:05 0.05% > firefox > 63847 leres 34 20 0 2471M 171M select 1 0:04 0.05% > firefox > 70198 leres 27 20 0 2436M 142M select 5 0:01 0.05% > firefox > 63860 leres 26 20 0 2422M 130M select 2 0:01 0.00% > firefox > 3132 influxd 18 52 0 997M 185M uwait 2 6:16 0.06% > influxd > 3214 root 6 20 0 510M 491M select 0 2:38 0.19% > pcscd > 2897 bind 26 52 0 349M 281M sigwai 0 0:15 0.00% > named > 22334 www 4 52 0 108M 77M select 3 0:02 0.00% > httpd > > From nobody Fri Jul 2 06:12:34 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id EBF4411E7F24; Fri, 2 Jul 2021 06:13:01 +0000 (UTC) (envelope-from elenamihailescu22@gmail.com) Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com [IPv6:2a00:1450:4864:20::635]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GGPrs2Jl2z4ltf; Fri, 2 Jul 2021 06:13:00 +0000 (UTC) (envelope-from elenamihailescu22@gmail.com) Received: by mail-ej1-x635.google.com with SMTP id hr1so10989976ejc.1; Thu, 01 Jul 2021 23:13:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to:cc; bh=PhwFme3k+Yw3GKg7HSZ8Ybn9vF15oveZUMiFYxtXOlE=; b=RVLJnHGD5oCu4uk1/01imn6WEPL4UnfMAuepSvJfSI90p7cMzeU+jsgjQ/LR99OjOD oZ2d46SGySy/rlUAHeq94Erj8d1XPQgP0E3tPsf+yMlYaz5efJRew9fUe5cuz3fWbMNg 1cgFMsqBStWY9suvDYsDg/23/ICiSERQbfYINXREb+udmhZCWRRT1xGrajSPGsiqYXX+ qkY51DUjykam3tExIGr59Y9FVdLr7LAil4MvmMbQjYuj+alhbdNLcMG4ZHZPiZsuhwIv BAx+8+vRhlKRqdCNV3OaLD4AcH6MaKKyAMKUyPkh70Vynz6YZXm+IYnUReZ54aCcZv3+ ueDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=PhwFme3k+Yw3GKg7HSZ8Ybn9vF15oveZUMiFYxtXOlE=; b=CclxwaJqZCgs7axL8y9N9BlhzBquLnQHRUbfwgIre2mtT1awU7jD6Iu7ZQUiryjc05 GUxHD73Rzc3bba0VmEEoAu/QHZ3DOm+aMvipfd/f4FPyjC1hfzUez86DASWNoTFMWoAy Y/HvJKxRv5DIhwVVZWDtEJaQC/OJtGrrBi22VUgXSmPOWhLdjqtIWbkxVSG6yWenmkqf 4pkXbi+WV0HxkkzShS/2/lR0pJQxQLmAeexwQDDszJUWGcsl/Lr2fj17WGY0a4DaH1gv Oc1it+IFq+FrZZC8tDFqnH5RgT8S9RqUZMtS8Pa8PORg8P26Gco2NliQhDIo61pCUlnu JNVA== X-Gm-Message-State: AOAM532toKr8geAyWpNp5+betTX6NMeEjYkWSI6/HI1VCjx2mRmPWxpn M86t2gST/PbzHMigWC74y5G/cPUy5Brk3AvrGpLLdFRD X-Google-Smtp-Source: ABdhPJyiGlHvt2rgg7VZHcHaIMqbTBP3zfZGSy5ybQlgvFqp5zkVI188RN8SaX1IelolVG7sKpqpIn2xRr+JDBXVpBQ= X-Received: by 2002:a17:907:d06:: with SMTP id gn6mr3724206ejc.447.1625206379628; Thu, 01 Jul 2021 23:12:59 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 From: Elena Mihailescu Date: Fri, 2 Jul 2021 09:12:34 +0300 Message-ID: Subject: Opened review for bhyve live migration feature To: freebsd-virtualization@freebsd.org, freebsd-hackers@freebsd.org Cc: Mihai Carabas , Darius Mihai Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4GGPrs2Jl2z4ltf X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=RVLJnHGD; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of elenamihailescu22@gmail.com designates 2a00:1450:4864:20::635 as permitted sender) smtp.mailfrom=elenamihailescu22@gmail.com X-Spamd-Result: default: False [-3.66 / 15.00]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.99)[-0.994]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:1450:4864:20::635:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.67)[-0.671]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[2a00:1450:4864:20::635:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::635:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization,freebsd-hackers]; FREEMAIL_CC(0.00)[freebsd.org,gmail.com] X-ThisMailContainsUnwantedMimeParts: N Hello, We have opened a review on Phabricator for a live migration feature for bhyve [1]. This review is based on the warm migration feature (for which we also have an opened review [2]) and works on top of the save/restore functionality for bhyve. Any feedback is welcome. [1] https://reviews.freebsd.org/D30954 [2] https://reviews.freebsd.org/D28270 Thank you, Elena From nobody Fri Jul 2 06:56:10 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 467AE11F4455 for ; Fri, 2 Jul 2021 06:56:13 +0000 (UTC) (envelope-from debdrup@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [96.47.72.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GGQpj1M97z4rS2 for ; Fri, 2 Jul 2021 06:56:13 +0000 (UTC) (envelope-from debdrup@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1625208973; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=d7dHBaeBBkNrB6cLTZZA+tlUnvHjhX37b7j/QaRVhpw=; b=ySgGLgBYOzyNY8+NQZ7fxW0MpKygyuwZmMNjwKq5Z6JT53uvD8BBrwf9PsZtiSrDeositM 16RPAQqn2Zo2/3Quq/hV0tlrsBwFtNyx3XX05wjZokveYSElADkjsgWZVfvZVW6TxUzf2Q 7/tMseGZECBfAhM0XAQ6xGQ2swdegZYQvBjhZrJg9sRs+8bku5CsMbdOfezXeYlQkDLlhf oSNzqhUEuduVYqnl7QPVWAw7PezSwucHedEGx/ZHThqczLkYCoNtj9roVQKBDdlE5019i8 g2NohTvf4+LjDFqm4WoZVKhWUtvECTW1+LtEW3KFUmINMRpEmz4brYzci3mQbg== Received: by freefall.freebsd.org (Postfix, from userid 1471) id 1517CF5A5; Fri, 2 Jul 2021 06:56:13 +0000 (UTC) Date: Fri, 2 Jul 2021 08:56:10 +0200 From: Daniel Ebdrup Jensen To: freebsd-hackers@freebsd.org Subject: Re: firefox is slow and uses a lot of memory Message-ID: <20210702065610.df7pdqutdkvixpgb@nerd-thinkpad.local> Mail-Followup-To: Daniel Ebdrup Jensen , freebsd-hackers@freebsd.org References: <37fe2be4-a142-c49e-e1b0-0e9a8fd07a74@freebsd.org> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="6odkw25y2ax7qlsq" Content-Disposition: inline In-Reply-To: <37fe2be4-a142-c49e-e1b0-0e9a8fd07a74@freebsd.org> ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1625208973; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=d7dHBaeBBkNrB6cLTZZA+tlUnvHjhX37b7j/QaRVhpw=; b=ZA2GOHCePW3FHQtgYtBYYq4cIukDwnX/c2hrzDNqJ3gIy98piVP+efycncWUEyCQ65ruhe CjtsbMxYykrtbhObwI3x1u20InkHftC0nnYp5XzR8DU3tNxnB7lRWXqGtXxu8/Wts/hy8H h+QSVqZlTHsjWDkyOmd2amcIJBFjLSHPWruPOgRgewuW9UicfWrratWrEL3WtvmiI5ceXt spc8g0jdRrWSoTwIueUbbdDEq47SzRbwEMWstafe71hUrngFEgRScg4xjh56k5BxFMXwtG ZiTd4jqSU4GOEk37XtEm0sDbzWJNM8er8eSng+1ZtAeagVB7OJXkh1g+1t06Hg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1625208973; a=rsa-sha256; cv=none; b=FSpNlpI04ihsnQP4jAt0r15R4/w17ba3xRVP03RDyKK0pJ5zzYdGX8E90WGjP9tEwLjGSN 46FD1yxiChtW1SXmd23pFd2nQJbeoQt2tY+lQd75d+Pm6fVSIMppbz9oMSVW6btnPkUPQ0 xMhEptkxywlNxtdDg6n5pJTuVAl2sRvPYCg9m1IoW5tMYFR8DIASyMX4a2kgDUa/ZMlM8k 7cU82f8IkR422bqjql0RN1hXjIogSQEbMf8Gfk5tD8qK+LpDOmvxGI/gGrO+d2SaryZW+q nI8FzXTmbDCZOSjYoSYjE2KDJGK3ZtTVQ3lNjrZ0pIIUsmeGUhU2CZma1+wNPQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N --6odkw25y2ax7qlsq Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jul 01, 2021 at 12:49:15PM -0700, Craig Leres wrote: >Starting with firefox 89 or so, I've found it starts using a LOT of=20 >memory and actually makes the system sluggish and causes momentary X=20 >freezes. I have 32G of ram/swap, is it reasonable for firefox to use=20 >15G? > > Craig [SNIP] Hi folks, I can't help but wonder if this is because of the link-time optimization that was enabled rather recently (with v88, if memory serves). Perhaps you can investigate this? If you just want to mitigate it, there's two ways of achieving it: You can either use limits(1) or use resource control via rctl(8). The second option does mean that you either have to limit the memory use of your user account, or run Firefox in a jail, but using [1] makes that relatively easy. Yours, Daniel Ebdrup Jensen 1: https://wiki.freebsd.org/JailingGUIApplications --6odkw25y2ax7qlsq Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGTBAABCgB9FiEEDonNJPbg/JLIMoS6Ps5hSHzN87oFAmDeuIpfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDBF ODlDRDI0RjZFMEZDOTJDODMyODRCQTNFQ0U2MTQ4N0NDREYzQkEACgkQPs5hSHzN 87o7fgf/bB7PKa42066ZhEjyXyIpMMaCU3BpUA6B6/tYbpnxd3wFKkkE91biN83Z XHRJmgWrB8Wr530PR7yNxiXHmQVgQiCRqI2UNsq/sxintJhAqFGAoM4MUUqbwLUo lC+4kBlQkZYOQsiK3b9SZIsVRvpp5etEaDStzF7mzmfFzEAl8zpTdYSZPgf7nUN0 WgZK17SlBfei+CrWpSVZ/eSx/7vlXXyrztn3YoSGahZHJUFMJXOnd+OqAPpvtYsf kAVmA4mI5SvOpSOrBpkiw95Sga5BKkeDW+dztcDg/yXDagDCANjxTItLmAPUSXN9 SuDtZsKrY4cjsBO9s0R83w99abD1xA== =K/uS -----END PGP SIGNATURE----- --6odkw25y2ax7qlsq-- From nobody Fri Jul 2 07:04:44 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 714B411F5915 for ; Fri, 2 Jul 2021 07:04:57 +0000 (UTC) (envelope-from michaelsprivate@gmail.com) Received: from mail-io1-xd2d.google.com (mail-io1-xd2d.google.com [IPv6:2607:f8b0:4864:20::d2d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GGR0m4YwYz4slS for ; Fri, 2 Jul 2021 07:04:56 +0000 (UTC) (envelope-from michaelsprivate@gmail.com) Received: by mail-io1-xd2d.google.com with SMTP id g5so6918594iox.11 for ; Fri, 02 Jul 2021 00:04:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=IJc27iCCPrG4IuE3rCR/jknp0aeT1lIq7HqODc4zLEY=; b=uGrEcBTlzPas2YvYXBjf0LcY2XIhkMmnB6EPK5SDt9BDtsbvZcEjFQo92RQ64x5yO5 /Mq0pQpXfKNWWshxB5dePIwTouXs8LYSZJRFArhC7QWzp06BVVdqaHZq8kJt3+BEr09D xL58uJrpVS2/bHwoMJ43Egl8EFVKPGa1/tnbXCpiPIoZZ3MNpfcsq0DjvvcPQrWkgyet p5rIA/WirVpjRtYaL1E8JLUDZEJk2FKVqSOuWiVzz+8OXi+ViVFc6e8ME9QciNJQN0Pr PrNik9K8lBiRiQZ/MNBDOg/dC28QzHfYgxNKU+fyQl3mdhZNLxi5rwRnLX75zuLzAri3 qpMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=IJc27iCCPrG4IuE3rCR/jknp0aeT1lIq7HqODc4zLEY=; b=C13jy6Rs86XGe5bwODaGCI2lupMmFoNu1hjeakVhwo6JpsRWevIh90VXOvx8PY1vwC p3rBOGjILExt+5CnpeE9T7us/u4dc8P1tvR4Y7MiTvMBXSFPqYo0tR45brH4R1i2nOG0 rFJUEu7p2ZZZmY4AY+pK3HWx54UqkYy4aCbhCWPP7Lh9K2lio49bb0pJXGpqjH4pLxKI DMA6hwAkcBB9flYHlLFdQHv+HhjKBBJxax8ozPdTlcR0gHZBFhCP/NUiw9kfHhJ4NMBa GgGARKD88iFBAt133oukikEcoxFqQdi1vAMcENSvtcer8O79+q6uaA1rkcG5jil6j7Uy WT0A== X-Gm-Message-State: AOAM5326VoCpLpnUSjJmj97U9zIaKupCcJ3zszABrvzr+ulSwW0p5Dcw O02VVXfiMf3V4rXzLnbiLLigGNUjeCFJJq/7zjZPi4bFgDFzoR6d X-Google-Smtp-Source: ABdhPJxOjApZ+U2L2anHwVrFmRaXZt+FObMDZneonABu0nhzusdiCGKz4gymdBM1vSjDgtXdHGtYoaDSycWiQNgiNDg= X-Received: by 2002:a05:6602:249a:: with SMTP id g26mr2588233ioe.150.1625209495179; Fri, 02 Jul 2021 00:04:55 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: <37fe2be4-a142-c49e-e1b0-0e9a8fd07a74@freebsd.org> <20210702065610.df7pdqutdkvixpgb@nerd-thinkpad.local> In-Reply-To: <20210702065610.df7pdqutdkvixpgb@nerd-thinkpad.local> From: Michael Schuster Date: Fri, 2 Jul 2021 09:04:44 +0200 Message-ID: Subject: Re: firefox is slow and uses a lot of memory To: freebsd-hackers@freebsd.org Content-Type: multipart/alternative; boundary="000000000000a5556b05c61e923d" X-Rspamd-Queue-Id: 4GGR0m4YwYz4slS X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=uGrEcBTl; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of michaelsprivate@gmail.com designates 2607:f8b0:4864:20::d2d as permitted sender) smtp.mailfrom=michaelsprivate@gmail.com X-Spamd-Result: default: False [0.24 / 15.00]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::d2d:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.76)[-0.758]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; NEURAL_SPAM_MEDIUM(1.00)[0.999]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::d2d:from:127.0.2.255]; NEURAL_SPAM_SHORT(1.00)[1.000]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::d2d:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers] X-ThisMailContainsUnwantedMimeParts: Y --000000000000a5556b05c61e923d Content-Type: text/plain; charset="UTF-8" On Fri, Jul 2, 2021 at 8:57 AM Daniel Ebdrup Jensen wrote: > On Thu, Jul 01, 2021 at 12:49:15PM -0700, Craig Leres wrote: > >Starting with firefox 89 or so, I've found it starts using a LOT of > >memory and actually makes the system sluggish and causes momentary X > >freezes. I have 32G of ram/swap, is it reasonable for firefox to use > >15G? > just an idea: If you enabled "Restore previous session" and had a lot of stuff open when you left off the last time, perhaps you want to repeat this test with a "clean" session - probably the easiest would be using a different user account, though I'm sure firefox-savvy people will be able to show you ways of doing this with your own user account w/o having to dump your usual settings. This may show you whether it's an issue with firefox per se or the pages you loaded. HTH Michael > > > Craig > [SNIP] > > Hi folks, > > I can't help but wonder if this is because of the link-time > optimization that was enabled rather recently (with v88, if memory > serves). > > Perhaps you can investigate this? > > If you just want to mitigate it, there's two ways of achieving it: > You can either use limits(1) or use resource control via rctl(8). > > The second option does mean that you either have to limit the > memory use of your user account, or run Firefox in a jail, but > using [1] makes that relatively easy. > > Yours, > Daniel Ebdrup Jensen > > 1: https://wiki.freebsd.org/JailingGUIApplications > -- Michael Schuster http://recursiveramblings.wordpress.com/ recursion, n: see 'recursion' --000000000000a5556b05c61e923d-- From nobody Fri Jul 2 10:51:21 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 42EA211DD50E for ; Fri, 2 Jul 2021 10:51:37 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GGX2K08hrz3rFq; Fri, 2 Jul 2021 10:51:36 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 162ApLYN002305 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Fri, 2 Jul 2021 13:51:24 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 162ApLYN002305 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 162ApL4j002304; Fri, 2 Jul 2021 13:51:21 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 2 Jul 2021 13:51:21 +0300 From: Konstantin Belousov To: Dmitry Chagin Cc: freebsd-hackers@freebsd.org Subject: Re: pondering pi futexes Message-ID: References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.5 X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on tom.home X-Rspamd-Queue-Id: 4GGX2K08hrz3rFq X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On Fri, Jul 02, 2021 at 12:53:01AM +0300, Dmitry Chagin wrote: > On Mon, Jun 28, 2021 at 02:20:25PM +0300, Konstantin Belousov wrote: > > On Sun, Jun 27, 2021 at 10:39:35PM +0300, Dmitry Chagin wrote: > > > > > > Hi, > > > some time ago I have changed Linuxulator futexes from sx lock to mtx. > > > sx was used as it allows copyin/copyout with sx lock held. > > > to use mtx I have changed the code like: > > > 1. lock mtx; > > > 2. disable_pagefaults; > > > 3. copyin() > > > 4. enable_pagefaults; > > > 5. if error > > > - unlock mtx; > > > copyin(); > > > if error == 0 goto 1. > > > > > > it works (needto replace copyin() by fueword32()), but pondering pi futexes > > > imlementation, I see that it is not possible to drop the futex lock on a return > > > from msleep() path. > > > > > > below a simplified FUTEX_LOCK_PI operation, where on enter to the kernel current thread: > > > > > > 0. acquire futex lock (which is mtx) > > > 1. cmpset(0 -> current thread TID), return (0) on success; > > > 2. fetch() from futex *uaddr (for TID of owner): > > > - check EDEADLK case (the futex word at *uaddr is already locked by the caller); > > > - check that is no waiters on *uaddr exists which is waiting via FUTEX_WAIT or > > > FUTEX_WAIT_BITSET, return (EINVAL) if so; > > > - cmpset(TID -> (FUTEX_WAITERS|TID)); > > > - on error, the futex owner changed in user-space, repeat from 1. > > > 3. Here we have: the owner, one waiter (current thread) and 0 or more waiters > > > sleeping on a waiting_proc. FUTEX_WAITERS bit is set, so any new waiters go to > > > the kernel and owner should unlock futex via the FUTEX_UNLOCK_PI op; > > > 4. Try to find the thread which is associated with the owner’s TID: > > > - on error, something bad happened, owner died? Clean owner state link? > > > return (ESRCH). Or if no other waiters? Check this... > > > - on success: > > > - save owner state link to the struct futex (save priority); > > > - check the owner's priority, bump it if needed; > > > - put the current thread to the waiters list in descending priority order; > > > - change priority of all waiters if needed; > > > - msleep on a futex waiting_proc; come back with futex lock held; > > > - restore own priority? If last waiter?; [ponders..] > > > - on timeout return (ETIMEDOUT); > > > - the current thread is the new owner: > > > bah!! - store() the owner TID to *uaddr; [check what should I do on error..] > > > - release futex lock; > > > - return (0). > > > > > > is it possible to hold *uaddr page to prevent page faults? > > > > I did not followed exact algorithm you trying to describe. Still, I can make > > two points which could be useful for formulation of the working solution. > > > > 1. Umtx AKA FreeBSD native implementation of something very similar to > > futex, has a concept of the umtx queue chain. The chain owns the mutex > > lock used for 'fast' ops, but for situations like accesses to userspace, > > we 'busy' the umtxq chain. De-facto busy state is the hand-rolled > > sleepable lock, with usual interlocking against chain mutex, and > > msleep/wakeup inter-thread notifications. > > > reading umtx impl I see umtxq_sleep drop lock (PDROP bit is set) and, in case > of a signal, reacquire lock and breaks from loop. is there a possible > small window for wakeup loss? as ERESTART returned, or the caller should > to take care of this? Or I missing something? > Im trying to reuse the umtx code for futexes, at least queue chain. > Thanks in advance! What is the race you see there? If we get EINTR/ERESTART from msleep, the loop inside umtxq_sleep() is not re-entered, and we do not sleep waiting for the chain to become unbusy anymore. Even if there is a wakeup on the chain, before or after we acquired the lock, it does not matter for us since we are not going to sleep on it anymore. Yes, it is up to the caller to decide what to do with EINTR/ERESTART. Some umtx ops are specified to not react abruptly to signals, they should finish the op anyway. For such ops, specific checks for stops and exit requests are still needed. > > > 2. If you insist on accessing userspace pages while owning non-sleepable > > locks, or even sleepable locks which do not compose cleanly with whole > > VM+VFS+network (the later due to NFS) locking protocols, we do have a > > technique that can be used. Althought it is complicated and sometimes > > slow. > > > > You already noted half of it above, namely disable of the page faults and > > copyin_nofault-like operations. Second half is prefaulting, which is > > done by vm_fault_quick_hold_pages(). There are two ways you can use it. > > > > Either you hold the target userspace page, then remap it into KVA and > > access userspace object through the kernel mapping. This assumes that > > the object fits into page, also it does not interact with parallel forks > > and CoW, which you either have to accept to recheck after. The usual > > races, e.g. userspace remapping the held (actually wired) page under us, > > is there, of course, but it is a race anyway. > > > > Or, you hold the page but still access it using copyin_nofault. Then any > > reported error from copyin becomes fatal instead of an indicator to retry. > > Error can happen for the same reason, because userspace could unmap or > > remap this address under us. > > > > Umtx implementation only uses busy on chains, might be because _nofault+ > > vm_fault_quick_hold() appeared later. From nobody Fri Jul 2 19:51:52 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 1439211E763A for ; Fri, 2 Jul 2021 19:51:56 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GGm1l1YXRz3lYD for ; Fri, 2 Jul 2021 19:51:55 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Received: by mail-wm1-x32d.google.com with SMTP id r9-20020a7bc0890000b02901f347b31d55so6920908wmh.2 for ; Fri, 02 Jul 2021 12:51:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-transfer-encoding:content-language; bh=UAeG7Dg5bI6rYhq2rlaiQT6O9zKu5b00mvNHPykgTv0=; b=L4ri3L/9WujF4XNWEqvVTEnlhIsBUsazxHcSH/REpm+R4IFzsd94Ko1L/GegDMUpkC HJhVz8cq90a/iI87fx4tXX2rCTX/sg1wg0fhlNy2BNU13V4UvR6YbCkynpUbevsJsreC OaohPUkI6IexIn25G8vw7Nc7GEYcQ7YwqKne4S1G/Z1b9jW1zLmbmhHmAPTOKso6L+ye HgrPym/TFLqYUZ+owVY4NiNmlRZf4pGf9d/su61zUkOCUA1Tw5hyk+Di2CrohXbPq6OQ 5nbTk9fpggwaNGOwbpblNLI87W1OOMOw15Lf0/0EeTuN9EenQqIr9eihs+1H3NpmnGnl zgbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=UAeG7Dg5bI6rYhq2rlaiQT6O9zKu5b00mvNHPykgTv0=; b=sjnPtbVCkgZBuzmo5mIubPtboA1NOsKcG86VSLDYth6Qp2+gO4s4vHj7XNzqaJWjLg JAmJXKW59QKbOBjWKt9IE+xj3vPOPoqUH2MzZ9bXXr5iul93ZcpJDpVMf6WHdbNFDR/u Asu441zD6iJka7lTFM5xo0hpczZkNTHAJriNGW2D4HKBRzCgy4GMKPbb0liIWUnxS8uc CRIz/4rVbKu/DnD663FColXJRzNjEUcjyM20y5XgbAEj8mMC17iw6tWJeE17/x91JhO7 GYZA3X7wbC9rtMgArUaK26LlSKV6sYzT8dyFigtiAmoilCMKxrlb/xCb4X4iqv0RuWvo yD2Q== X-Gm-Message-State: AOAM531Prt29PVRbLXnGIRgE1ZSEvFWp9VfinF0YScalr2Og66N5Gqie yBRHlYIyir/uk01wMEFvIBUNKjNSARDvkOoW X-Google-Smtp-Source: ABdhPJxUoEcIxM+RoudPvMFiPoE9WJuMIcZHXAfgKBiFhlLDIlC9k7x40tfEVFasqMs21KXgqW4PDw== X-Received: by 2002:a05:600c:2292:: with SMTP id 18mr827104wmf.179.1625255513359; Fri, 02 Jul 2021 12:51:53 -0700 (PDT) Received: from [192.168.1.11] (88-105-96-80.dynamic.dsl.as9105.com. [88.105.96.80]) by smtp.gmail.com with ESMTPSA id c8sm4069766wri.91.2021.07.02.12.51.52 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 02 Jul 2021 12:51:52 -0700 (PDT) Subject: Re: firefox is slow and uses a lot of memory To: freebsd-hackers@freebsd.org References: <37fe2be4-a142-c49e-e1b0-0e9a8fd07a74@freebsd.org> From: Graham Perrin Message-ID: <521f2dba-fd28-f005-7c27-efd59f056283@gmail.com> Date: Fri, 2 Jul 2021 20:51:52 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 In-Reply-To: <37fe2be4-a142-c49e-e1b0-0e9a8fd07a74@freebsd.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-GB X-Rspamd-Queue-Id: 4GGm1l1YXRz3lYD X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=L4ri3L/9; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of grahamperrin@gmail.com designates 2a00:1450:4864:20::32d as permitted sender) smtp.mailfrom=grahamperrin@gmail.com X-Spamd-Result: default: False [-0.01 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:1450:4864:20::32d:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; RECEIVED_SPAMHAUS_PBL(0.00)[88.105.96.80:received]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; NEURAL_SPAM_MEDIUM(1.00)[0.997]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2a00:1450:4864:20::32d:from:127.0.2.255]; NEURAL_SPAM_SHORT(0.99)[0.990]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::32d:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers] X-ThisMailContainsUnwantedMimeParts: N On 01/07/2021 20:49, Craig Leres wrote: > Starting with firefox 89 or so, I've found it starts using a LOT of > memory and actually makes the system sluggish and causes momentary X > freezes. I have 32G of ram/swap, No such problem here with FreeBSD 14.0-CURRENT. Can you share details of your graphics hardware, and associated software? Maybe of interest: and the subsequent answers about hardware and software. > is it reasonable for firefox to use 15G? > >         Craig > > ice 95 % uname -a > FreeBSD ice.alameda.xse.com 12.2-RELEASE-p8 FreeBSD 12.2-RELEASE-p8 > r18 LBL  amd64 > > > >>>> top -osize > > last pid: 71913;  load averages:  0.52,  0.53,  0.47    up 1+00:29:04 > 12:46:04 > 215 processes: 2 running, 206 sleeping, 6 stopped, 1 zombie > CPU:  1.9% user,  0.0% nice,  0.3% system,  0.1% interrupt, 97.7% idle > Mem: 2931M Active, 3479M Inact, 1874M Laundry, 19G Wired, 1192M Buf, > 4116M Free > ARC: 13G Total, 3822M MFU, 8686M MRU, 172K Anon, 270M Header, 553M Other >      11G Compressed, 81G Uncompressed, 7.38:1 Ratio > Swap: 32G Total, 32G Free > >   PID USERNAME    THR PRI NICE   SIZE    RES STATE    C   TIME WCPU > COMMAND > 63858 leres        35  20    0    15G   506M select   7   0:40 0.08% > firefox >  3610 leres         3  20    0  6268M    61M select   2   4:54 1.00% Xorg >  3159 telegraf     19  23    0  4926M    47M uwait    3   0:26 0.00% > telegraf > 63845 leres       125  20    0  4612M  1794M select   7  18:49 4.61% > firefox > 33553 leres        60  20    0  4579M  2077M CPU2     2   9:18 3.10% > thunderb > 63859 leres        34  20    0  2916M   559M select   3   1:11 0.08% > firefox > 63857 leres        34  20    0  2812M   466M select   0   2:00 0.18% > firefox > 63850 leres        37  20    0  2723M   377M select   2   1:51 0.24% > firefox > 63849 leres        34  20    0  2708M   394M select   3   1:05 0.02% > firefox > 63856 leres        34  20    0  2608M   275M select   6   0:30 0.01% > firefox > 63861 leres        34  20    0  2491M   189M select   6   0:05 0.05% > firefox > 63847 leres        34  20    0  2471M   171M select   1   0:04 0.05% > firefox > 70198 leres        27  20    0  2436M   142M select   5   0:01 0.05% > firefox > 63860 leres        26  20    0  2422M   130M select   2   0:01 0.00% > firefox >  3132 influxd      18  52    0   997M   185M uwait    2   6:16 0.06% > influxd >  3214 root          6  20    0   510M   491M select   0   2:38 0.19% > pcscd >  2897 bind         26  52    0   349M   281M sigwai   0   0:15 0.00% > named > 22334 www           4  52    0   108M    77M select   3   0:02 0.00% > httpd > From nobody Fri Jul 2 20:03:04 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4091711D1ABB for ; Fri, 2 Jul 2021 20:03:14 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GGmGn1PfGz3np1; Fri, 2 Jul 2021 20:03:12 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (v-critter.freebsd.dk [192.168.55.3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by phk.freebsd.dk (Postfix) with ESMTPS id BA72289287; Fri, 2 Jul 2021 20:03:04 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.16.1/8.16.1) with ESMTPS id 162K34T1050616 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Fri, 2 Jul 2021 20:03:04 GMT (envelope-from phk@critter.freebsd.dk) Received: (from phk@localhost) by critter.freebsd.dk (8.16.1/8.16.1/Submit) id 162K34kg050615; Fri, 2 Jul 2021 20:03:04 GMT (envelope-from phk) Message-Id: <202107022003.162K34kg050615@critter.freebsd.dk> To: Craig Leres cc: freebsd-hackers@freebsd.org Subject: Re: firefox is slow and uses a lot of memory In-reply-to: <37fe2be4-a142-c49e-e1b0-0e9a8fd07a74@freebsd.org> From: "Poul-Henning Kamp" References: <37fe2be4-a142-c49e-e1b0-0e9a8fd07a74@freebsd.org> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <50613.1625256184.1@critter.freebsd.dk> Date: Fri, 02 Jul 2021 20:03:04 +0000 X-Rspamd-Queue-Id: 4GGmGn1PfGz3np1 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of phk@critter.freebsd.dk designates 130.225.244.222 as permitted sender) smtp.mailfrom=phk@critter.freebsd.dk X-Spamd-Result: default: False [-2.99 / 15.00]; RCVD_TLS_ALL(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[130.225.244.222:from]; FREEFALL_USER(0.00)[phk]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[freebsd.dk]; ARC_NA(0.00)[]; SPAMHAUS_ZRD(0.00)[130.225.244.222:from:127.0.2.255]; RCVD_COUNT_THREE(0.00)[3]; NEURAL_HAM_MEDIUM(-0.99)[-0.991]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FORGED_SENDER(0.30)[phk@phk.freebsd.dk,phk@critter.freebsd.dk]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:1835, ipnet:130.225.0.0/16, country:EU]; FROM_NEQ_ENVFROM(0.00)[phk@phk.freebsd.dk,phk@critter.freebsd.dk]; MAILMAN_DEST(0.00)[freebsd-hackers] X-Spam: Yes X-ThisMailContainsUnwantedMimeParts: N -------- Craig Leres writes: > Starting with firefox 89 or so, I've found it starts using a LOT of > memory and actually makes the system sluggish and causes momentary X > freezes. I have 32G of ram/swap, is it reasonable for firefox to use 15G? I noticed recently that firefox had started DNS's all bookmarks rather aggressively I found something under about:config which made it stop, (sorry cant remember which setting) but that made it at lot more workable for me. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From nobody Fri Jul 2 23:56:40 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id CFD6111E4CB6 for ; Fri, 2 Jul 2021 23:56:43 +0000 (UTC) (envelope-from gusev.vitaliy@gmail.com) Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com [IPv6:2a00:1450:4864:20::133]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GGsSC1RJ8z4d3C; Fri, 2 Jul 2021 23:56:43 +0000 (UTC) (envelope-from gusev.vitaliy@gmail.com) Received: by mail-lf1-x133.google.com with SMTP id a15so21001945lfr.6; Fri, 02 Jul 2021 16:56:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:mime-version:subject:message-id:date:cc:to; bh=ehlbtiMz3vNMb25E04gIao2phmRWnfWDuFphmj8o6O4=; b=TWzBPwjQ2n7cggFzXgXdBJ93ce2R+fwS4nhQKBHV3pNAVeQdF1Bbe1Ef4M1ulfNqvm onbLzsUsTXLRe5Pkme1qXFoPme5/7zhwuQqoO91Oei7jYWlDK8x4kcN4hLkdNqoXtdYG DqFFo+xfTI2IicGZCbY8dixJywJESAvaDNbgZvgiP8DQZEm803HPLZ6cl9FYqhrvsk4s 9cMzViNNlTSHErHvZJUhfWKjAltwH14OulCOsdeEnsGjXv5Xe498r7i7ZoOfuYxeqxyF MF9mLwmMPDKndvdyl/BzjC+Kt8Yy7wV2znKafokCJx1pQfBh0o1wxjKBIsymkd00At/S 8EPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:mime-version:subject:message-id:date:cc:to; bh=ehlbtiMz3vNMb25E04gIao2phmRWnfWDuFphmj8o6O4=; b=pT2BvVd4g5y/tEb7cSIYLKAQDLr87MayRQWSJCMrLGuzyU01kVtxEF8fdQo8x4t4BU 82UzVO2aTiPwRJPlO+dL2jcd7WE1Plw1hoNdg3HZ07393OfzJxDKH2c65x4n5m3XmREx eCxuP6wtdTRnhpe/9pJrgWGgiov56MdzA3C6flRMzh7kDmSOHDsThIkmixZRjYk8DOJD jS3dMiJPavD7DsSb8miKBP40QMa9Ae+Tn/yg2etAuX8EsxkPcispmk0YHhwjYcHT05NR n4KXBU4W+xTe0lJ1rLA+aYOnoTuRLnEzxbmzANaFcE13eE4nKr5AiIfNGrFMpptcaS52 VFoA== X-Gm-Message-State: AOAM533UK+pRjj+5RjsJIJw/wW7efwE5CLS1iN1GkOO1f5QlOhBLKV19 GSqR/KW+gnUzsVeWFRuRbLLDhQ34/fs= X-Google-Smtp-Source: ABdhPJyQQHgceD1UfkqWqMy5QTUAw1kJJwmW+OW+tj6cYMOscj4qCJotfwSR2stZHX63xz4YHZQtJg== X-Received: by 2002:a05:6512:3186:: with SMTP id i6mr1529497lfe.291.1625270201610; Fri, 02 Jul 2021 16:56:41 -0700 (PDT) Received: from [10.42.0.5] ([188.187.60.230]) by smtp.gmail.com with ESMTPSA id l15sm400552lfh.228.2021.07.02.16.56.40 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 02 Jul 2021 16:56:40 -0700 (PDT) From: Vitaliy Gusev Content-Type: multipart/alternative; boundary="Apple-Mail=_561E82D1-B4FD-48AE-BAD7-E731D542BAD9" List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.7\)) Subject: madvise(MADV_FREE) doesn't work in some cases? Message-Id: Date: Sat, 3 Jul 2021 02:56:40 +0300 Cc: Konstantin Belousov To: freebsd-hackers@freebsd.org X-Mailer: Apple Mail (2.3608.120.23.2.7) X-Rspamd-Queue-Id: 4GGsSC1RJ8z4d3C X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=TWzBPwjQ; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of gusevvitaliy@gmail.com designates 2a00:1450:4864:20::133 as permitted sender) smtp.mailfrom=gusevvitaliy@gmail.com X-Spamd-Result: default: False [2.21 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; HAS_ATTACHMENT(0.00)[]; MV_CASE(0.50)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:~,4:~,5:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; ARC_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:1450:4864:20::133:from]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; RECEIVED_SPAMHAUS_PBL(0.00)[188.187.60.230:received]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain,multipart/mixed]; MIME_BAD_ATTACHMENT(1.60)[c]; NEURAL_SPAM_MEDIUM(0.11)[0.110]; NEURAL_SPAM_SHORT(1.00)[1.000]; SPAMHAUS_ZRD(0.00)[2a00:1450:4864:20::133:from:127.0.2.255]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::133:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers] X-Spam: Yes X-ThisMailContainsUnwantedMimeParts: Y --Apple-Mail=_561E82D1-B4FD-48AE-BAD7-E731D542BAD9 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hi,=20 I came across not expected behaviour with madvise() in FreeBSD. Attached test program mmapfork does: mmap, fork, touch memory and then = madvise(MADV_FREE). Expected behaviour - one process can allocate memory (lazy allocation) = while system is freeing previously allocated memory for a second = process. Current behaviour - system kills one process with message in dmesg: pid 31314 (mmapfork), jid 0, uid 1001, was killed: out of swap space Running this test in Linux or illumos shows expected behaviour with a = little difference in illumos - it frees memory almost immediately, w/o = needs lack of memory in a system. If use MADV_NOTNEED - no changes. If modify program and do not do fork(), but run two instances - that = shows expected behaviour. To reproduce just disable swap, and run program with argument as 1/2 RAM = on a system. For instance, command below will try run and use ~ 2GB area = twice. [vetal@bsdev ~]$ ./mmapfork 2000 Testing program is attached. Note, during testing I disabled swap on all systems: Linux, illumos and = FreeBSD. Does it mean madvise() doesn't work well in FreeBSD or test does = something wrong? Thanks, Vitaliy Gusev --Apple-Mail=_561E82D1-B4FD-48AE-BAD7-E731D542BAD9 Content-Type: multipart/mixed; boundary="Apple-Mail=_1525D0A7-8A3B-4A8C-9910-463C30219109" --Apple-Mail=_1525D0A7-8A3B-4A8C-9910-463C30219109-- --Apple-Mail=_561E82D1-B4FD-48AE-BAD7-E731D542BAD9-- From nobody Sat Jul 3 06:54:20 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 287CC11E26C9 for ; Sat, 3 Jul 2021 06:54:23 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com [IPv6:2a00:1450:4864:20::534]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GH2k70R0cz3rMr for ; Sat, 3 Jul 2021 06:54:22 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: by mail-ed1-x534.google.com with SMTP id w13so16377180edc.0 for ; Fri, 02 Jul 2021 23:54:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; bh=XkLeE2flVx2KlGvoF8AbPUezqxedTIXSaxtRt2B7Plo=; b=PJ3GQW5M5DqS8ZCrrfnLv7b6UDsv8gohups0YJ6XxhdtVWdRtAVfQUYFQMmqR6BZCj rcWSJoUkCmN8/7mQ387DZcI27Qd7aKg6qm2RJ4kaW6YNG4UyHwq4B+GJRQ0MfwM3ud1d C0fYKe8SILu4oB7v08Eh3s0hIRNhA6h7fo/YAJ0ei1r4J0S5RSpae4gHsCGEJ0SJVJAp ebLT/rw6r9iV06gluY2C+pEbYfJQT2/e2x4iiWU9gnL6O4xelJH796w9EcDT8AFqRrT9 956PXrTxMkP4eeK9S8oAsRJCfPNJcTKUMXIz5amhv2mUqyoOBsvi4KEiTf6FEe0kcFtb MfoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:reply-to:mime-version:content-transfer-encoding; bh=XkLeE2flVx2KlGvoF8AbPUezqxedTIXSaxtRt2B7Plo=; b=lvYGuEWNZ2ufSb3CEQkOLcfTlmMV/d/c7UE76kiJX/dh1C/v8PK6uBphKRDn2Y/FXu ZNDGMuTKib7r6qhkIyYW6N8pVKLegTL/pJBmOFvqS8sJl8Q/UoPkvt1Vbt0TasG9QIOK HdYOMI+4sfxYLvjwp224h4aeSsTEzt/7lVH+kUONRPIm1MQDTmsn3Fo7GjBQOj8BL8eE 4a9xpvTL+fHti1KvY2KABc0zY6CFPXRE1iyjljGMgxEXehkN0ayDhJSq98gF8OimIY2K 8cqTEpBHHSlCaPZ7UE8HFkRy37fByiH/uH41Evm+IG91s//i34N5ZLX5PM+KHT2mTLpI vakw== X-Gm-Message-State: AOAM533l5J0xMIRh1LDPxI1QW3CwiWk88SFnrm97OChe/goSVPy63WGZ GrxtxRNusiccbe0qBdz1GTI= X-Google-Smtp-Source: ABdhPJzG+o+MLcdx+LHWzR0A5yTGIRl1P+7p7uLK8KrEXbz1ROCdXErmSl5iSaYJu17EMhs/wMHz+A== X-Received: by 2002:a05:6402:27c7:: with SMTP id c7mr3840483ede.272.1625295261507; Fri, 02 Jul 2021 23:54:21 -0700 (PDT) Received: from ernst.home (pd9e2360f.dip0.t-ipconnect.de. [217.226.54.15]) by smtp.gmail.com with ESMTPSA id i18sm2275635edc.7.2021.07.02.23.54.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Jul 2021 23:54:20 -0700 (PDT) Date: Sat, 3 Jul 2021 08:54:20 +0200 From: Gary Jennejohn To: Vitaliy Gusev Cc: freebsd-hackers@freebsd.org Subject: Re: madvise(MADV_FREE) doesn't work in some cases? Message-ID: <20210703065420.6dbafb5f@ernst.home> In-Reply-To: References: Reply-To: gljennjohn@gmail.com X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.33; amd64-portbld-freebsd14.0) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4GH2k70R0cz3rMr X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; TAGGED_RCPT(0.00)[]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On Sat, 3 Jul 2021 02:56:40 +0300 Vitaliy Gusev wrote: > Hi, > > I came across not expected behaviour with madvise() in FreeBSD. > > Attached test program mmapfork does: mmap, fork, touch memory and then madvise(MADV_FREE). > > Expected behaviour - one process can allocate memory (lazy allocation) while system is freeing previously allocated memory for a second process. > > Current behaviour - system kills one process with message in dmesg: > > pid 31314 (mmapfork), jid 0, uid 1001, was killed: out of swap space > > Running this test in Linux or illumos shows expected behaviour with a little difference in illumos - it frees memory almost immediately, w/o needs lack of memory in a system. > > If use MADV_NOTNEED - no changes. > > If modify program and do not do fork(), but run two instances - that shows expected behaviour. > > To reproduce just disable swap, and run program with argument as 1/2 RAM on a system. For instance, command below will try run and use ~ 2GB area twice. > > [vetal@bsdev ~]$ ./mmapfork 2000 > > Testing program is attached. > > Note, during testing I disabled swap on all systems: Linux, illumos and FreeBSD. > > Does it mean madvise() doesn't work well in FreeBSD or test does something wrong? > The FreeBSD mail server apparently removed the attachment. So either provide it in-line or put it on some pubic file server and add a link to it. -- Gary Jennejohn From nobody Sat Jul 3 08:03:36 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id CE8E611F0A46 for ; Sat, 3 Jul 2021 08:03:45 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com [IPv6:2a00:1450:4864:20::62a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GH4G84pKMz4RZD; Sat, 3 Jul 2021 08:03:44 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: by mail-ej1-x62a.google.com with SMTP id o5so20304388ejy.7; Sat, 03 Jul 2021 01:03:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; bh=y4Y9ejVry/5KYIvdrlemYw3kC5cr/ojJHv4V5C8dZYo=; b=DN7Kz/xUUXq2UhFDahUgn1JcbZXYWJc+gjIk0Vs3p+x77wryn39W7gYN+XSrfUfT37 YVeRyZ5MOV1scbQ2ajXABJeVmQd2KSnreGCOFoE80oowYZKRQSThpffHtYC4PUC3LWYA aQiyqhoiMW0Nev9/0w+LbDpu4JSKYJqdLtqYGzZMLEkgWWs0I3Ar/IqvWhAZC/rcaZZt ML8ZXWOLbWTYfVl9q1s4rJf5hY2lz+R7wY2WzTe4yRwdZ9ww7grkTLPvURZK8KeotsG5 dPuaWEArb1kzJP6/U9GD7XEQcgUz15jAtCLnzG8M+qwYqrxAFVAP8f8nFiUQ7c7r4kDT HL9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:reply-to:mime-version:content-transfer-encoding; bh=y4Y9ejVry/5KYIvdrlemYw3kC5cr/ojJHv4V5C8dZYo=; b=UlOAFNny/5nzdAPMUrWNMbZfNHm2EnAaLk6EHp+Xtc/4IJ508b5Inlt261lUdqGNSf IadEF6ctheOkn92w+cZ7RcxX/Lcq5m8e0YuKHjbjSEpsQULlRfYDu1IUZC10a96ehq+U t6HK961eQQH2Cm90UIYBkFwrszYIuN8ruTDHOjg/fUZcnh8mwmnDFcXfjBW/KboGo+wE goc1rMy0m3h63C6tGFGLNy3LsMdrcFtjbnxxxayRXC4GsLQBlsqWcuS3JrjbpEt1kSzF zN4VE2PTexz/fj4XVaxUKdBB5CjBfS6Y/cKWBc3ZeM4wp5qC/BhY7UBqTgp9QriydECM 3uVQ== X-Gm-Message-State: AOAM53089G6RRBJpbPC4xh0ZvqJxXw8ITbnLfUX1i3BqDchiIBvvzkXv 0DciHKYpy8SHqeYyhRFf9VkBBIPehiU= X-Google-Smtp-Source: ABdhPJwWOEXMisgeJ+qCLMrY/K2C4WISZsAC1QSSlps2HSCgCx+U6rZ5z2yrvWEMolF8YP+ROnC0xA== X-Received: by 2002:a17:907:e8d:: with SMTP id ho13mr3681062ejc.545.1625299416985; Sat, 03 Jul 2021 01:03:36 -0700 (PDT) Received: from ernst.home (pd9e2360f.dip0.t-ipconnect.de. [217.226.54.15]) by smtp.gmail.com with ESMTPSA id q22sm2313097edv.51.2021.07.03.01.03.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 03 Jul 2021 01:03:36 -0700 (PDT) Date: Sat, 3 Jul 2021 10:03:36 +0200 From: Gary Jennejohn To: Craig Leres Cc: freebsd-hackers@freebsd.org Subject: Re: firefox is slow and uses a lot of memory Message-ID: <20210703080336.7a01c22d@ernst.home> In-Reply-To: <202107022003.162K34kg050615@critter.freebsd.dk> References: <37fe2be4-a142-c49e-e1b0-0e9a8fd07a74@freebsd.org> <202107022003.162K34kg050615@critter.freebsd.dk> Reply-To: gljennjohn@gmail.com X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.33; amd64-portbld-freebsd14.0) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4GH4G84pKMz4RZD X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=DN7Kz/xU; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of gljennjohn@gmail.com designates 2a00:1450:4864:20::62a as permitted sender) smtp.mailfrom=gljennjohn@gmail.com X-Spamd-Result: default: False [0.12 / 15.00]; HAS_REPLYTO(0.00)[gljennjohn@gmail.com]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; REPLYTO_ADDR_EQ_FROM(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:1450:4864:20::62a:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; RECEIVED_SPAMHAUS_PBL(0.00)[217.226.54.15:received]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.88)[-0.877]; MIME_GOOD(-0.10)[text/plain]; FREEMAIL_REPLYTO(0.00)[gmail.com]; NEURAL_SPAM_MEDIUM(1.00)[1.000]; NEURAL_SPAM_SHORT(1.00)[1.000]; SPAMHAUS_ZRD(0.00)[2a00:1450:4864:20::62a:from:127.0.2.255]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::62a:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers] X-ThisMailContainsUnwantedMimeParts: N On Fri, 02 Jul 2021 20:03:04 +0000 "Poul-Henning Kamp" wrote: > -------- > Craig Leres writes: > > Starting with firefox 89 or so, I've found it starts using a LOT of > > memory and actually makes the system sluggish and causes momentary X > > freezes. I have 32G of ram/swap, is it reasonable for firefox to use 15G? > > I noticed recently that firefox had started DNS's all bookmarks rather > aggressively I found something under about:config which made it stop, > (sorry cant remember which setting) but that made it at lot more workable > for me. > Another possibility would be to enter about:networking#dns. A button "Clear DNS Cache" will appear. Click on that and check whether that reduces the amount of memory being used by firefox. -- Gary Jennejohn From nobody Sat Jul 3 08:34:50 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 3FAC111F5E6A for ; Sat, 3 Jul 2021 08:34:53 +0000 (UTC) (envelope-from gusev.vitaliy@gmail.com) Received: from mail-lj1-x22c.google.com (mail-lj1-x22c.google.com [IPv6:2a00:1450:4864:20::22c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GH4y50vZ3z4ZjM; Sat, 3 Jul 2021 08:34:52 +0000 (UTC) (envelope-from gusev.vitaliy@gmail.com) Received: by mail-lj1-x22c.google.com with SMTP id r16so16840068ljk.9; Sat, 03 Jul 2021 01:34:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=/t+3DsHlBQDD99isBTOOhsJfCx5CG0ZoCFNJvgLaU98=; b=qs0aEFaUyoV271N1hP1WlyX5TIJ6Kl87R/nzumeVmOwnH15U8jAT82T96jO0Rr8LO/ 918kx+Q+Jvn8Tq9B6NF2ShbJ8jVAfnadGbz3IBovSjbbYXi6NaqR+MLMpztQ6vMollAz cyaOqY8mt5kiUZXWp67vuR76TIE0Fp8YGwyo56oN99ICFIwoXIi4vzXdh3NTGdXJbfPL jjvgltUtRX9+3TwLvxKkk7vMzQUbGGaoQ2OsKdKjrIRg+bNm2Wr357LjzcAeaejLDkeV N0p3S2h9wjJO3Y+d2eMNCVg4vVoCK9lTOc8JwX8BQyXCCH1MMeUklU5jGS6jKjRW5BTq Hzsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=/t+3DsHlBQDD99isBTOOhsJfCx5CG0ZoCFNJvgLaU98=; b=qKnSOnGcp2L3Ki8ppjywznvWiDlPysP91+COWlYTB6uP4Bg9DbbImeQh4T+PhmrRiD Z8R5Dr3A/im9YbYz1KJ/iCnHsM6CVojpgxO/txr2ZrC74nJKYoHsZC/MgcVHUolyFWmW QBMOSd8vnpscIWBa9pyKF8/dDU0mxtFE0vsKLie4HX2pGlhxTqOkRabMjRIF0Yoo4zKA JmCt/zKrt8S2Cvej/VCmpi43a/fKOb8ABnbkPAnGfVrNCRDWXDftYOsKqEIweqFwwwas QDuPk1umZ/MjMhIFN3Qwkb0o6DFbQvFuqPXG4BNqUFIYTYf2Zs37TLT46nQTM3gCBIAe L7Yw== X-Gm-Message-State: AOAM53389b7fKpTi9AVKKwaA8PO05OEWk7GdkVB40puL962JrWmPFR4r 4mNyyNRtQlIEi5bkUMl/fOg= X-Google-Smtp-Source: ABdhPJyRycJLnncugefjBAAwqP741bnakvrjWU9a9mQYJ8rNYOiTN8H0yRAuTbRlJMqgX4zEdFBDyQ== X-Received: by 2002:a05:651c:201e:: with SMTP id s30mr2854990ljo.364.1625301291519; Sat, 03 Jul 2021 01:34:51 -0700 (PDT) Received: from [10.42.0.5] ([188.187.60.230]) by smtp.gmail.com with ESMTPSA id v10sm547609ljp.20.2021.07.03.01.34.50 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sat, 03 Jul 2021 01:34:51 -0700 (PDT) From: Vitaliy Gusev Message-Id: Content-Type: multipart/alternative; boundary="Apple-Mail=_E13708A3-36BC-422B-A795-631242F67439" List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.7\)) Subject: Re: madvise(MADV_FREE) doesn't work in some cases? Date: Sat, 3 Jul 2021 11:34:50 +0300 In-Reply-To: <20210703065420.6dbafb5f@ernst.home> Cc: freebsd-hackers@freebsd.org, "kib@freebsd.org" To: gljennjohn@gmail.com References: <20210703065420.6dbafb5f@ernst.home> X-Mailer: Apple Mail (2.3608.120.23.2.7) X-Rspamd-Queue-Id: 4GH4y50vZ3z4ZjM X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[] X-ThisMailContainsUnwantedMimeParts: Y --Apple-Mail=_E13708A3-36BC-422B-A795-631242F67439 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On 3 Jul 2021, at 09:54, Gary Jennejohn wrote: >=20 > The FreeBSD mail server apparently removed the attachment. >=20 > So either provide it in-line or put it on some pubic file server and > add a link to it. >=20 > --=20 > Gary Jennejohn #include #include #include #include #include #include int main(int argc, char *argv[]) { size_t len =3D (size_t)(argc > 1 ? atoi(argv[1]) : 1024) * 1024 = * 1024; uint8_t *ptr, *end, *p; unsigned pagesz =3D 1<<12; int pid; ptr =3D mmap(NULL, len, PROT_WRITE | PROT_READ, MAP_ANONYMOUS | = MAP_PRIVATE, -1, 0); if (ptr =3D=3D MAP_FAILED) err(1, "cannot mmap"); end =3D ptr + len; printf("mmap %p pid %d\n", ptr, getpid()); printf("end %p len %#lx\n", end, len); fflush(stdout); pid =3D fork(); if (pid < 0) err(1, "cannot fork"); printf("pid %d\n", getpid()); sleep(pid =3D=3D 0 ? 1 : 14); printf("%d: touch\n", getpid()); p =3D ptr; while (p < end) { *p =3D 1; p +=3D pagesz; } printf("%d: sleep before madvise\n", getpid()); sleep(pid =3D=3D 0 ? 1 : 4); printf("%d, madvise\n", getpid()); p =3D ptr; while (p < end) { int error; error =3D madvise(p, pagesz, MADV_FREE); if (error) { err(1, "cannot madvise"); } p +=3D pagesz; } printf("%d: Press enter to exit\n", getpid()); getchar(); } =E2=80=94=E2=80=94 Vitaliy Gusev= --Apple-Mail=_E13708A3-36BC-422B-A795-631242F67439-- From nobody Sat Jul 3 11:35:59 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 1F14A11D7881 for ; Sat, 3 Jul 2021 11:36:15 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GH8zK5NKTz4v9c for ; Sat, 3 Jul 2021 11:36:13 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 163BZxgY062996 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sat, 3 Jul 2021 14:36:02 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 163BZxgY062996 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 163BZx8M062995; Sat, 3 Jul 2021 14:35:59 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 3 Jul 2021 14:35:59 +0300 From: Konstantin Belousov To: Vitaliy Gusev Cc: freebsd-hackers@freebsd.org Subject: Re: madvise(MADV_FREE) doesn't work in some cases? Message-ID: References: <0A95973D-254A-4574-8DC7-9F515F60B873@gmail.com> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <0A95973D-254A-4574-8DC7-9F515F60B873@gmail.com> X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.5 X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on tom.home X-Rspamd-Queue-Id: 4GH8zK5NKTz4v9c X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (mx1.freebsd.org: 2001:470:d5e7:1::1 is neither permitted nor denied by domain of kostikbel@gmail.com) smtp.mailfrom=kostikbel@gmail.com X-Spamd-Result: default: False [2.00 / 15.00]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; HAS_XAW(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all:c]; RCPT_COUNT_TWO(0.00)[2]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RBL_DBL_DONT_QUERY_IPS(0.00)[2001:470:d5e7:1::1:from]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; ARC_NA(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(1.00)[1.000]; NEURAL_HAM_LONG(-1.00)[-0.998]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; NEURAL_SPAM_MEDIUM(1.00)[1.000]; SPAMHAUS_ZRD(0.00)[2001:470:d5e7:1::1:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : No valid SPF, No valid DKIM,none] X-ThisMailContainsUnwantedMimeParts: N On Sat, Jul 03, 2021 at 02:32:01AM +0300, Vitaliy Gusev wrote: > Hi, > > I came across not expected behaviour with madvise() in FreeBSD. > > Attached test program mmapfork does: mmap, fork, touch memory and then madvise(MADV_FREE). > > Expected behaviour - one process can allocate memory (lazy allocation) while system is freeing previously allocated memory for a second process. > > Current behaviour - system kills one process with message in dmesg: > > pid 31314 (mmapfork), jid 0, uid 1001, was killed: out of swap space > > Running this test in Linux or illumos shows expected behaviour with a little difference in illumos - it frees memory almost immediately, w/o needs lack of memory in a system. > > If use MADV_NOTNEED - no changes. > > If modify program and do not do fork(), but run two instances - that shows expected behaviour. > > To reproduce just disable swap, and run program with argument as 1/2 RAM on a system. For instance, command below will try run and use ~ 2GB area twice. > > [vetal@bsdev ~]$ ./mmapfork 2000 > > Testing program is attached. > > Note, during testing I disabled swap on all systems: Linux, illumos and FreeBSD. > > Does it mean madvise() doesn't work well in FreeBSD or test does something wrong? Your program does not exactly what you described above. There is a generic race to consume memory, and some specific details about madvise(2) on FreeBSD. >From the code, you do: - mmap anonymous private region - fork - both child and parent start touching the mmaped region. Two processes race to consume 1/2 of RAM on your system. If one of them happen to execute faster then another, you do get to the case where one of them does madvise(). But it could be that processes execute in lockstep, and try to eat all the memory before going to madvise(). Did you excluded this case? Now, about the specific of madvise(MADV_FREE) on FreeBSD. Due to the way CoW is implemented with the shadow chain of objects, we cannot drop the top of the shadow chain, otherwise instead of returning zeroed pages next time, we would return content back in the time. It was relatively recent discovery, see bf5661f4a1af6931ec4b6, PR 240061. To explain it in simplified form, when there is potential old content under the CoW copy for the mapping, we cannot drop CoW-ed pages. This is the motivation why madvise(MADV_FREE) does nothing for your program. When you run two instances without fork, there is no previous content and no Cow, so madvise() can safely remove the pages from the object, and on the next access they are zero-filled. You can read more details in the referenced commit, as well as some musings about way to make it somewhat better. I must say, that trying to allocated 1/2 + 1/2 of RAM this way, on a system without swap, is the way to ask for troubles anyway. From nobody Sat Jul 3 15:30:52 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 03A4C11F2A2D for ; Sat, 3 Jul 2021 15:30:56 +0000 (UTC) (envelope-from rwmaillists@googlemail.com) Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GHGB713hZz3kb6 for ; Sat, 3 Jul 2021 15:30:54 +0000 (UTC) (envelope-from rwmaillists@googlemail.com) Received: by mail-wr1-x431.google.com with SMTP id p8so16301943wrr.1 for ; Sat, 03 Jul 2021 08:30:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20161025; h=date:from:to:subject:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=8By4PY+dshwsKgq56AwPnLeyY0W6HBGxsCYcU4uXfLs=; b=Ufq01sDzlguXlhFEtDTW+foKxol3hTCiA12NSCSkg3oos/aopliC9wNl5JXQ5pM7ly DQgTWcFBq6uyUJR8w0/C0x4dNr/HHcmDM292KV5akeCuzhD3RW1gQLTbu9er6m74Hndf 0dGzqcSosf9TW9Qw+rVxEBGJ3LNffzNE/7/x1ypstZZe6kKwpZ5Amg7PhWa88m/KlHRK o1Ac1NlwihmXsZaGhH4Iq9TuyoznhDdqvGfmRUhanZW3gLZKDPaGWQ2Y1WrmgX2ehfbZ 65W4IGn0wRx3aIsHr9iEfpoqkt8z9pcv3BCcmoe3Qi7aflwhiWkBCGqC2iBcfa+42/Nx /f0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8By4PY+dshwsKgq56AwPnLeyY0W6HBGxsCYcU4uXfLs=; b=Ft4bS0RZOAedqMhKlk+Ym8m6TZ6RQbco03h1SKH8v5Z74PVOc5nftYEewb1FU5DS/a hVXS+U3CR0DeXSazErbrpwBcBut6/BJ2RKuaR9eiZ2XfSg0hh1iyg5/+DzFwj8LyKdll HpmdVOSrmxanEckq/YFw43h7uiw8PZfXMcW/3X0V6LUjAwJcQllGrnvm9D6fD0UM8vut 7MFhE5J5yU/VcqEqzK7ozwV9P+JzOilY9y+6quN/sCPAkITLaTyI4edOyX7CYoLeBJGc vteSzVxs04/j1rVpzwCiyYMSqqIxgoCQMcuI7Hh7YXkjm6iMOw2wZS9HT0lVCx1cWgJm gTgQ== X-Gm-Message-State: AOAM532HmwNDopdwIsZGrTyqGGObwmnxKln3B9nsjE0fqR0pjuaKVul7 J+2az379TanNmyW7yy3/fpyuVUqmMG8= X-Google-Smtp-Source: ABdhPJy8ID7A/drd0+Q4NnQMEKwULcubkw7DkMFVKfi2vaRV+qtrufazb6bYRDf9dw7wDWO29gqF4Q== X-Received: by 2002:adf:e107:: with SMTP id t7mr5781563wrz.165.1625326253639; Sat, 03 Jul 2021 08:30:53 -0700 (PDT) Received: from gumby.homeunix.com (host-92-1-116-226.as13285.net. [92.1.116.226]) by smtp.gmail.com with ESMTPSA id c9sm6842321wro.5.2021.07.03.08.30.52 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 03 Jul 2021 08:30:53 -0700 (PDT) Date: Sat, 3 Jul 2021 16:30:52 +0100 To: freebsd-hackers@freebsd.org Subject: Re: firefox is slow and uses a lot of memory Message-ID: <20210703163052.64d72443@gumby.homeunix.com> In-Reply-To: <20210703080336.7a01c22d@ernst.home> References: <37fe2be4-a142-c49e-e1b0-0e9a8fd07a74@freebsd.org> <202107022003.162K34kg050615@critter.freebsd.dk> <20210703080336.7a01c22d@ernst.home> X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.33; amd64-portbld-freebsd12.2) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4GHGB713hZz3kb6 X-Spamd-Bar: + Authentication-Results: mx1.freebsd.org; dkim=pass header.d=googlemail.com header.s=20161025 header.b=Ufq01sDz; dmarc=pass (policy=quarantine) header.from=googlemail.com; spf=pass (mx1.freebsd.org: domain of rwmaillists@googlemail.com designates 2a00:1450:4864:20::431 as permitted sender) smtp.mailfrom=rwmaillists@googlemail.com X-Spamd-Result: default: False [2.00 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; FREEMAIL_FROM(0.00)[googlemail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[googlemail.com:+]; DMARC_POLICY_ALLOW(-0.50)[googlemail.com,quarantine]; RECEIVED_SPAMHAUS_PBL(0.00)[92.1.116.226:received]; FROM_EQ_ENVFROM(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:1450:4864:20::431:from]; FREEMAIL_ENVFROM(0.00)[googlemail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; DWL_DNSWL_NONE(0.00)[googlemail.com:dkim]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[googlemail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_SPAM_SHORT(1.00)[1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; NEURAL_SPAM_MEDIUM(1.00)[1.000]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2a00:1450:4864:20::431:from:127.0.2.255]; MIME_TRACE(0.00)[0:+]; NEURAL_SPAM_LONG(1.00)[1.000]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::431:from]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-hackers] Reply-To: rwmaillists@googlemail.com From: RW via freebsd-hackers X-Original-From: RW X-ThisMailContainsUnwantedMimeParts: N On Sat, 3 Jul 2021 10:03:36 +0200 Gary Jennejohn wrote: > On Fri, 02 Jul 2021 20:03:04 +0000 > "Poul-Henning Kamp" wrote: > > > -------- > > Craig Leres writes: > > > Starting with firefox 89 or so, I've found it starts using a LOT > > > of memory and actually makes the system sluggish and causes > > > momentary X freezes. I have 32G of ram/swap, is it reasonable for > > > firefox to use 15G? > > > > I noticed recently that firefox had started DNS's all bookmarks > > rather aggressively I found something under about:config which made > > it stop, (sorry cant remember which setting) but that made it at > > lot more workable for me. > > > > Another possibility would be to enter about:networking#dns. A button > "Clear DNS Cache" will appear. Click on that and check whether that > reduces the amount of memory being used by firefox. There's something deeply wrong if prefetching DNS like that causes high memory use. From nobody Sat Jul 3 17:02:45 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 56FD611D2D68 for ; Sat, 3 Jul 2021 17:02:56 +0000 (UTC) (envelope-from tech-lists@zyxst.net) Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GHJDH4Z1yz3tVZ for ; Sat, 3 Jul 2021 17:02:55 +0000 (UTC) (envelope-from tech-lists@zyxst.net) Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id C7FA05C003E for ; Sat, 3 Jul 2021 13:02:48 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Sat, 03 Jul 2021 13:02:48 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zyxst.net; h= date:from:to:subject:message-id:mime-version:content-type; s= fm3; bh=oSKbcnKlE2RAa7w8xTQzh0EotVpEiEJl2O9Ufitqkz4=; b=EQr8sr5J ERwb5sIKRbe3XJQ1raRIION9qhcq7StEk2upq0qHUpsqG9+CXA/bQ4T9U7o1v5wl lKLIc4J00RU3t0ibuBr67eaMV+AUUP25pZ9t9nsosjY9+4EEekLQogGPYsyOWPcA Z4R5IyzlWEBCzbbT3yxaRLiC/J7yE25LcVbTR1opBBiNW7rtlWPll4bCar2Pzhp3 NxsDkDc6FIOFvtihd4o5lNTLZUGVhmOlSpbgVzWhhFj55zST2imcgN20nFTfBAxw zNjHNZeyHVqr+Q6nFgxgIxG54aZ8pSD8z/ik+72hiU4KpycBwZcRwOKOUTfY4SHH 727QzVCWuOEm4Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:message-id :mime-version:subject:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm3; bh=oSKbcnKlE2RAa7w8xTQzh0EotVpEi EJl2O9Ufitqkz4=; b=wnfMPHYHs0oORD9ruixiqkfWUsorvOqeqLQntPhg1RZt9 JuHVjfeua/3NMuGCcaEKkMWJwazdCb4QwcyT/ce0kB0s7mPnEdoNppdTHkYjDxxG Wc6I/lg5PgXt1E03qaSfWKwPuoCMeZ1raJlrM/ZVnKlqEugKBShZoJ+nPnJSuPSc RjI8RV1UqcQLZJgCzcclK/zC2LFZzWU6loZncetb6EHNi7wiCqte/Lw6TAuqlK3a zivBMFD3HiXn74a1D3ZYQDR1XGh4W4SwRX2fSe2oLoFhIahdwxOZoNp3yWAknDvz 6IbtTLyC/AhmBWnCm097OfgAIX3q8k/l47+O8GVQQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrfeejtddguddtiecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfggtggusehgtdorre dttddvnecuhfhrohhmpehtvggthhdqlhhishhtshcuoehtvggthhdqlhhishhtshesiiih gihsthdrnhgvtheqnecuggftrfgrthhtvghrnheplefffffhjeevueeuuedvleejtefgud ffffektdegffdutdevgeejhefftdegtddunecuvehluhhsthgvrhfuihiivgeptdenucfr rghrrghmpehmrghilhhfrhhomhepthgvtghhqdhlihhsthhsseiihiigshhtrdhnvght X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Sat, 3 Jul 2021 13:02:48 -0400 (EDT) Date: Sat, 3 Jul 2021 18:02:45 +0100 From: tech-lists To: freebsd-hackers@freebsd.org Subject: make optimization setting Message-ID: Mail-Followup-To: freebsd-hackers@freebsd.org List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="zcgiVDOz0g4uN5zN" Content-Disposition: inline X-Rspamd-Queue-Id: 4GHJDH4Z1yz3tVZ X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=zyxst.net header.s=fm3 header.b=EQr8sr5J; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=wnfMPHYH; dmarc=none; spf=pass (mx1.freebsd.org: domain of tech-lists@zyxst.net designates 66.111.4.26 as permitted sender) smtp.mailfrom=tech-lists@zyxst.net X-Spamd-Result: default: False [-5.60 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[66.111.4.26:from]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.26]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[zyxst.net:+,messagingengine.com:+]; NEURAL_HAM_SHORT(-0.91)[-0.907]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[66.111.4.26:from]; ASN(0.00)[asn:11403, ipnet:66.111.0.0/20, country:US]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.26:from]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.996]; R_DKIM_ALLOW(-0.20)[zyxst.net:s=fm3,messagingengine.com:s=fm3]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[zyxst.net]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[66.111.4.26:from:127.0.2.255]; MAILMAN_DEST(0.00)[freebsd-hackers] X-ThisMailContainsUnwantedMimeParts: N --zcgiVDOz0g4uN5zN Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi, In an arm64.aarch64 context[1], would there be measurable benefit[2] compiling world/kernel with -O3 or -0fast, or higher[3]? [1] main/14, generic-nodebug, zfs-on-root, usb3-boot, encrypted swap & / [2] after installation, quicker execution of src binaries than those made with stock settings. [3] how high? thanks --=20 J. --zcgiVDOz0g4uN5zN Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEE8n3tWhxW11Ccvv9/s8o7QhFzNAUFAmDgmC0ACgkQs8o7QhFz NAVNUg//YcMXDwNeANGeQ+PYsvxXX3G+BipYhCBRyQoh4oQhxtUnSvC7QhwTLwrM IPvS5AmombFxumm8RDY3/hNw3xoFtKiMX1YkINoh2ERhf9dFGt4eUbsbXqC5gbHp jBo6ruJdH9BWHgzknDyxI8C48QZxqDGwtbJUr/HSQtHieqe5lVYbyB0KixnuFgpY DxJj1pr4g353X4rhwDzIgXYuoKcsesO27jOzqGYj1IRw8uKyGTXaf223UefROLDR 9TnvEMBiGOXMqgmhD9ReZ7OjOhPnV0hr8pIMRo3O+MX3+5zoUT97QI3bOrpcrSIy 8k8T+qk//+K3UhosduzKY4/NVPDPqsCpMuT1qcJ0A9smHhtrOzk62Tyf9Ae9eMwv DLj/1yToToCV28FbTR3mYsqZZkenicwUGHvzmjo55EkAka84uTjrkkgHKIiDZ4eD i5mc/VClWa1enURJ+gwLDkx3H+8JwMWUr8fCbjPa0IG15Nn7dfMlXv1M5CjDqAYd Ytc44N2ryWvLTr/sTw+kNwr08aAv4/ox7ELqROEuTGu0WZ8iS1iaqHvcrT9gfBuN 7aRzixjhH1R8dBWvZCBtQGhvogkH56Y8CPnpQit7YbY97xxXlpg1NOkAuBN6GUqK /ug13a2P6LgkVDbsjn3ItsmQs9ahZ0qpQOf+EsarlXKl6SyPsqU= =jfA3 -----END PGP SIGNATURE----- --zcgiVDOz0g4uN5zN--