From owner-freebsd-fs@freebsd.org Tue Jun 11 06:12:53 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 15BE415AEFDC for ; Tue, 11 Jun 2019 06:12:53 +0000 (UTC) (envelope-from stilezy@gmail.com) Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A253475E27 for ; Tue, 11 Jun 2019 06:12:51 +0000 (UTC) (envelope-from stilezy@gmail.com) Received: by mail-wr1-x42a.google.com with SMTP id n4so11533020wrs.3 for ; Mon, 10 Jun 2019 23:12:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:date:message-id:user-agent:subject:mime-version; bh=0+CTgm19VAV9c5WHHPGIJ3NW4WZVDOaqggbv8ZMHmw0=; b=b+44y7LtCcqSaYg2NEbBEi1VrATae7VMoyfQxuNWefkO5TCjoPFDYa0F5qb3uwGXAr xYmNQsNOF8CQXI6YvKdt8rkZ0gKdebBQkk8qVs8DIuAp60rm9VvjLNCOnzTpyp/QO/ZF eonmAgBpVqKNRwbHZsKKzMYoXs5mjeQLcs7IQq2QHMV4aBgwmYdoppolKWFZLNUKcdjr IU8ZGZiqw9W/KAOpTRQHXT1BPBU/9G1GS8KNHlpKyr9Y6akLSFGO1sKhf89hlKlLHKqk nb+WU4fT1s45m4XYMKb5HztKteOtFulrl/Bj0HlUp5shwJH49EEh9GSZVuzXdJ+ko9q4 IeCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:date:message-id:user-agent:subject :mime-version; bh=0+CTgm19VAV9c5WHHPGIJ3NW4WZVDOaqggbv8ZMHmw0=; b=ohjRRSKRThjRhEzG5joBfi2UzbHpiUrav6hFLkoTYDbBBk4YtxMdFq1GnZKvFQCQy/ OqcDCQTkKNQH5JjCFmhooyZMPsXAp6stDfOWfFOOb0UY89qLfjhdXBLk7swgWZI4y56c tyytTzv8WQUJhw+jgNRy61hss21HND52xihdAcznigh7JNTkNCvlzZJuUElr/vRCMi6c l81ZJUVPxzde0j15jZx2mllR2bFlq/gJXBA9PV4302sOYWSZavMi5MzvMUejDsQxw+nv zR6Ch8HZH5EZaJd3nsOs6mY4vJ/C6jmRVQWm8ScvVvfzF6ujEhUn/y30mtMv+OC4qaJA f7TQ== X-Gm-Message-State: APjAAAVfuuCHOw2/KqLV+nR8VX+aUmJvfrcq7o+q0XGorYsx+8Q0AwUj HO0TGM2uG8GNK5hwOgkiAADHFhBM X-Google-Smtp-Source: APXvYqxwuzHWrRmc5az0p5SAablyvMRfdxj6R6vZYFeJiIVUaUcVmYKYlrafKKajrgHPM6jYkt4eGw== X-Received: by 2002:adf:8bdd:: with SMTP id w29mr20599215wra.325.1560233568898; Mon, 10 Jun 2019 23:12:48 -0700 (PDT) Received: from [10.193.100.21] ([78.32.235.97]) by smtp.gmail.com with ESMTPSA id h90sm29066776wrh.15.2019.06.10.23.12.48 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 10 Jun 2019 23:12:48 -0700 (PDT) From: Stilez To: Date: Tue, 11 Jun 2019 07:12:47 +0100 Message-ID: <16b452ae318.2783.49a377fccbf53440a4b582c142a1ed88@gmail.com> User-Agent: AquaMail/1.19.0-1434 (build: 101900002) Subject: [ZFS] Interaction between DDT R/W and ashift/block size? MIME-Version: 1.0 X-Rspamd-Queue-Id: A253475E27 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=b+44y7Lt; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of stilezy@gmail.com designates 2a00:1450:4864:20::42a as permitted sender) smtp.mailfrom=stilezy@gmail.com X-Spamd-Result: default: False [-5.83 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.97)[-0.975,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(-2.85)[ip: (-9.52), ipnet: 2a00:1450::/32(-2.38), asn: 15169(-2.29), country: US(-0.06)]; RCVD_IN_DNSWL_NONE(0.00)[a.2.4.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; SUBJECT_ENDS_QUESTION(1.00)[] Content-Type: text/plain; format=flowed; charset="us-ascii" Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Jun 2019 06:12:53 -0000 Can someone please clarify for me, how DDT block R/W is affected by the ashift setting and pool block sizes, as this is fairly deep ZFS stuff. Thanks 1) DDT entries are typically 180-280 bytes on disk. A look at gstat or other disk tools (iostat, dtrace) shows that DDT blocks are typically 4K on disk, on a usual system with ashift=12. - Does this mean that a typical on-disk 4K DDT block will usually contain ~ 10-20 DDT entries? Or do on-disk DDT blocks only store 1 entry per block, regardless of block size, wasting most of their space? 2) Like other blocks, DDT are collated in TXGs before writing out, suggesting they might be written sequentially, in groups, or with multiple entries per block, making larger IO more efficient. - If I increase ashift from 12 to say 13 or 14, is this likely to enhance DDT storage efficiency and DDT record load/save time by cutting IO, or just waste space with no DDT IO benefit? (I appreciate this would impact small files in the pool). What about increasing prefetch for small IO? Both pool and server are specced for and suitable for dedup, being ~ 4x dedup and having ~ 0.25 TB ARC + 0.5 TB of 900p L2ARC, with a metadata reservation in ARC ample for the entire 162M entries in DDT as well as all other metadata. I'm also assuming ashift size applies to DDT blocks in the first place. Thanks for any insight available, as this is fairly deep stuff in the ZFS codebase. Stilez