From nobody Mon Aug 16 18:50:46 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 60F49175B29F for ; Mon, 16 Aug 2021 18:51:04 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ot1-f51.google.com (mail-ot1-f51.google.com [209.85.210.51]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GpNXl4w3cz3D9b for ; Mon, 16 Aug 2021 18:51:03 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-ot1-f51.google.com with SMTP id h63-20020a9d14450000b02904ce97efee36so22105859oth.7 for ; Mon, 16 Aug 2021 11:51:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=2facFIvElFB0X9KHlE2/Q+0ZVfqvIiLt+4hKQ6mW0oQ=; b=SIb6IshVvjoPTZcBpl+en84mu4Ek4bematYJ5oOGdGQLoANmZh0hqUdL1ZEG9qFGk5 TiC3qf+Zp4DJKxAWpM0zn6xVXYpOS8iiLiuYKoeK509RJISm+B+s0JHgVLKZSRImCzK7 NoJQcLdDoc4fwkb67lNusbba/hwDdpoURoolJBCRPmEytc+efUnBZLVy7zTh2QPpqTzc I2zRlIlc3iSO15uDch7sgYciesNxf2nmz8Bcs3ODzh2jKYdMHKvC4LawuRTEJPBpAzYx Qv/cHj6YKf1R0+6OQ2aXqKUVwP8WDlbJS9rWRUUWSRQ8CnV4HBBANY3UM6rUpXaY/Qa3 e0yw== X-Gm-Message-State: AOAM530ZtSopy1Uj4WHDJhOAdwd1LPffsC3wAAx5+wwmbXVBfJukTr2v /ZeCbfZSpFSrW/KAhB9eiXwYvloKMPlkDCZBH17crfsc X-Google-Smtp-Source: ABdhPJyxClNYRwfMZQlMiaOonFNJkb3HQrQQJXjz71PYclRxxDuJ9DXU3vLfc7pICrLg3lVWtGHIXJh5zI0sPo5m6Eg= X-Received: by 2002:a9d:450c:: with SMTP id w12mr158546ote.18.1629139857183; Mon, 16 Aug 2021 11:50:57 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 From: Alan Somers Date: Mon, 16 Aug 2021 12:50:46 -0600 Message-ID: Subject: sysctl is too slow To: FreeBSD Hackers Content-Type: multipart/alternative; boundary="0000000000007a17d605c9b1aeb7" X-Rspamd-Queue-Id: 4GpNXl4w3cz3D9b X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.210.51 as permitted sender) smtp.mailfrom=asomers@gmail.com X-Spamd-Result: default: False [-0.25 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.97)[-0.971]; RCVD_COUNT_TWO(0.00)[2]; FREEFALL_USER(0.00)[asomers]; FROM_HAS_DN(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[209.85.210.51:from]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-0.28)[-0.275]; RCVD_TLS_ALL(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[209.85.210.51:from]; NEURAL_SPAM_LONG(1.00)[1.000]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; FREEMAIL_ENVFROM(0.00)[gmail.com]; TO_DOM_EQ_FROM_DOM(0.00)[] X-ThisMailContainsUnwantedMimeParts: Y --0000000000007a17d605c9b1aeb7 Content-Type: text/plain; charset="UTF-8" ztop feels very sluggish on a server with 5000 ZFS datasets. Dtrace shows that almost all of its time is spent in sys_sysctl. ktrace shows that both ztop and sysctl(8) call sys_sysctl a total of five times for each sysctl they care about: 1) To get the next oid 2) To get the sysctl's name 3) To get the oidfmt 4) To get the size of the value 5) To get the value itself. Each of these steps takes about equal time, and together all five take about 100us. If the time per call is mostly syscall overhead, then the process could be sped up by 80% by combining all of these things into a single syscall: return the next oid, its name, its format, the size of its value, and optimistically the value itself, assuming the user passed a sufficiently large buffer. Am I missing something? Is there any other reason why sysctl is so slow? Or should I forget about it, and try to export ZFS's dataset stats through devstat instead? -Alan --0000000000007a17d605c9b1aeb7--