From nobody Sun Aug 20 20:02:31 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RTRNT5mrQz4qf8h for ; Sun, 20 Aug 2023 20:02:33 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-oo1-xc30.google.com (mail-oo1-xc30.google.com [IPv6:2607:f8b0:4864:20::c30]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RTRNT3sDpz3P8l for ; Sun, 20 Aug 2023 20:02:33 +0000 (UTC) (envelope-from mjguzik@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-oo1-xc30.google.com with SMTP id 006d021491bc7-56ce1bd7fc4so1938763eaf.2 for ; Sun, 20 Aug 2023 13:02:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1692561752; x=1693166552; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=YY1SR3y85g+ahr047W/A6oTYhluZvFVEymQ2LpichjI=; b=K97xQr+VohZgHKPkUXs0Bd1REOKZD7QwzEV01i+LPhBzVggCHvVt/tSAhcbM/f0ovt W9dU01Z6g8urILBue3OyL+lPr5M31UWR+Zux0l7QsE6VoAU0ub1u9ioat0V0uHwT58Wm 58Nd6XvYt6WTYBizRHeBpN5rkMzfQOv8ub1wiTFKsn+b+NL+5SllaeF4JzCPCnhfrDi9 lf5MdXiMujRW2v62CRKxUgxRuoXLKPgN5ZpG69dElTPLYFhBfGPM/fh90Z5JeTDyM/w2 xo0RU79syUYKCYBbKX/joqTav3jWDrXZbY1pLZw+SOZ4Bn/zG4YAPxS7CZM4YZ95qIBb Esig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692561752; x=1693166552; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=YY1SR3y85g+ahr047W/A6oTYhluZvFVEymQ2LpichjI=; b=NBWcohykoz9AVGeIJzNpEOfP9wIlAuAI9llxWQpaseWptBGspbGe82MuNCDE+7hOU9 rAi1BedgIACLwf8UiNX404EzxXlAC4w3z7ItuztIgY47IMWKEaH9qwnpsF8rqe71l1uc Wbt1X1dwg0bNqUoUCd4Pde1eioDn4DYwZZ4cF/6esCVg8+/Fxdlrusihlj7d5Es4nowq UeusVsAsX5zvfMB/OlvCt0GvQL6qd2fgGgBkfCjLMC9CeKPbaj5w9cM60la7OuWKvji/ +D1PvLoPe0965maHvQsnP2GxsFspqBC3kmcI8t2HAQ4pADDjaiaIPkiKHObzY1/9Rfac GqTg== X-Gm-Message-State: AOJu0YxRZVqt61b2hiIEfZBsv8XUgQRmSE3QP5UAMVN5dQIhYa8zc4f3 MJs7lxAJtaLBrQJelKNWIw6rD8ahjVwpHQUjWHydKcCSHfM= X-Google-Smtp-Source: AGHT+IGeSkib8+8NiMPhi47CqQudsp7yU7TBZjGgQYfHWSZwy038XcKsd3zGIAHszeHxHdKgBIMrhTsFOx7rYHDkNS0= X-Received: by 2002:a4a:381d:0:b0:56d:2cbf:2315 with SMTP id c29-20020a4a381d000000b0056d2cbf2315mr4322451ooa.9.1692561752198; Sun, 20 Aug 2023 13:02:32 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Received: by 2002:ac9:79d9:0:b0:4f0:1250:dd51 with HTTP; Sun, 20 Aug 2023 13:02:31 -0700 (PDT) In-Reply-To: <73f7c9d3db8f117deb077fb17b1e352a@Leidinger.net> References: <61ca9df1b15c0e5477ff51196d0ec073@Leidinger.net> <88e837aeb5a65c1f001de2077fb7bcbd@Leidinger.net> <4d60bd12b482e020fd4b186a9ec1a250@Leidinger.net> <73f7c9d3db8f117deb077fb17b1e352a@Leidinger.net> From: Mateusz Guzik Date: Sun, 20 Aug 2023 22:02:31 +0200 Message-ID: Subject: Re: Speed improvements in ZFS To: Alexander Leidinger Cc: current@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4RTRNT3sDpz3P8l X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] On 8/20/23, Alexander Leidinger wrote: > Am 2023-08-20 19:10, schrieb Mateusz Guzik: >> On 8/18/23, Alexander Leidinger wrote: > >>> I have a 51MB text file, compressed to about 1MB. Are you interested >>> to >>> get it? >>> >> >> Your problem is not the vnode limit, but nullfs. >> >> https://people.freebsd.org/~mjg/netchild-periodic-find.svg > > 122 nullfs mounts on this system. And every jail I setup has several > null mounts. One basesystem mounted into every jail, and then shared > ports (packages/distfiles/ccache) across all of them. > >> First, some of the contention is notorious VI_LOCK in order to do >> anything. >> >> But more importantly the mind-boggling off-cpu time comes from >> exclusive locking which should not be there to begin with -- as in >> that xlock in stat should be a slock. >> >> Maybe I'm going to look into it later. > > That would be fantastic. > I did a quick test, things are shared locked as expected. However, I found the following: if ((xmp->nullm_flags & NULLM_CACHE) != 0) { mp->mnt_kern_flag |= lowerrootvp->v_mount->mnt_kern_flag & (MNTK_SHARED_WRITES | MNTK_LOOKUP_SHARED | MNTK_EXTENDED_SHARED); } are you using the "nocache" option? it has a side effect of xlocking -- Mateusz Guzik