From nobody Mon Aug 16 19:30:51 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 45D34175DEBB for ; Mon, 16 Aug 2021 19:30:53 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-lf1-x12a.google.com (mail-lf1-x12a.google.com [IPv6:2a00:1450:4864:20::12a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GpPQj0yy5z3HLR; Mon, 16 Aug 2021 19:30:53 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: by mail-lf1-x12a.google.com with SMTP id c24so36561660lfi.11; Mon, 16 Aug 2021 12:30:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=aUshE31Bb+tTsBJe5u3K1ZBUpPGGDq+j2q8w7MIEHmY=; b=aYHDUYPVrUEE4DKecaBPAeCP/Vg3wGeQKaWFAbJyMkOxWhGXffY9n7kibOTnlEUWsA 7IfJVvgvi129zz8vmuUiFiXbqCsRe/kHpyACR8WzqhW0CqHOSaUqbXGsJtBIo7ICivXT RChYE+7gD3ibaUwAOXUlD2H8J0EPYLoOPQIPwaWrVfPsx2d1n8pDS1JZsAPykFwzxECQ ckOR3GFypQv39VbO8JNFqdHx4oLUSyUTMaP5SvzCX1smjklaUhMmtz8TfgxU7ip5c+LV TV6jtQg9iZzS9dsRSYT0JRSrBzAVDpYwIXqWHbR8njn1htk/p5nGS1rGUXbtZooY83UI nsIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=aUshE31Bb+tTsBJe5u3K1ZBUpPGGDq+j2q8w7MIEHmY=; b=cuLQsgxvUD7JotjtLt6OVl5Ng9yKeBI1sD7G0Br7j1UmrXUcNkWF2rDHmaygeJxHhc yYqfJEcj6GxzPdIpQoRWLZeqJTekBgiKMANa6BBPagTgI8iV3PpFDNcf1gmmARawaanQ z5Pw0qZJ9OHwkNg6yy2u1oJfVh+6dEbIN3UeKZjev6xGKapefMWDGHgkSncb8+kyT5Ik 9PYu6812dAFqnvzL9q14d8emPs42f53Pkh/uAdoR1vsr0DIzSggrw0eRFw/ZtOj49pNM AvQV48dDvVbsihsPWHOnqdoX/FJT2VRAitv1z+VdjI6ner/NfWjnDlG7kgdfU/OBN7bp DiBg== X-Gm-Message-State: AOAM531jLp6iuPmRB99uS04GMKIZWjh1kN7MRCEK7ReX+XOiwezqDTJx HbzTiep3oK0GgR03oQBv2z+wKLUkiBAMsPIF2FoF9w9a X-Google-Smtp-Source: ABdhPJzhJbYeDo3QCQpkWMChkHT+4tt405keXnp7QglNtXgp6bmb+3UK6k2VPJzTJBBujaDaAHdM+y+1DLICKWWzvvs= X-Received: by 2002:ac2:4e8c:: with SMTP id o12mr122210lfr.374.1629142251856; Mon, 16 Aug 2021 12:30:51 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Received: by 2002:a2e:6f04:0:0:0:0:0 with HTTP; Mon, 16 Aug 2021 12:30:51 -0700 (PDT) In-Reply-To: References: From: Mateusz Guzik Date: Mon, 16 Aug 2021 21:30:51 +0200 Message-ID: Subject: Re: sysctl is too slow To: Alan Somers Cc: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4GpPQj0yy5z3HLR X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N Last time I checked lookup of a sysctl was very bad with linear scans all over. Short of complete revamp of the entire thing I would start with replacing the scans with a RB tree at each level. As is if you indeed have 5000 datasets, you are doing increasingly longer walks. On 8/16/21, Alan Somers wrote: > ztop feels very sluggish on a server with 5000 ZFS datasets. Dtrace shows > that almost all of its time is spent in sys_sysctl. ktrace shows that both > ztop and sysctl(8) call sys_sysctl a total of five times for each sysctl > they care about: > > 1) To get the next oid > 2) To get the sysctl's name > 3) To get the oidfmt > 4) To get the size of the value > 5) To get the value itself. > > Each of these steps takes about equal time, and together all five take > about 100us. If the time per call is mostly syscall overhead, then the > process could be sped up by 80% by combining all of these things into a > single syscall: return the next oid, its name, its format, the size of its > value, and optimistically the value itself, assuming the user passed a > sufficiently large buffer. > > Am I missing something? Is there any other reason why sysctl is so slow? > Or should I forget about it, and try to export ZFS's dataset stats through > devstat instead? > -Alan > -- Mateusz Guzik