From owner-svn-src-head@freebsd.org Wed May 22 20:48:37 2019 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E944915B6D63; Wed, 22 May 2019 20:48:36 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-yb1-xb42.google.com (mail-yb1-xb42.google.com [IPv6:2607:f8b0:4864:20::b42]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 581D497E75; Wed, 22 May 2019 20:48:36 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: by mail-yb1-xb42.google.com with SMTP id a21so1391337ybg.9; Wed, 22 May 2019 13:48:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:to:cc:references:from:openpgp:autocrypt:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=ps08Je2LxRAAc2g1TRftceYXyGVuoaLm+xsAB1n7sdg=; b=j0Nkg9+mCMjgCxX/AIm0bv0xuNFDTW1U/1gV3UWsXg6pHC4KzeQInVW/JAAnxzLWso zcZ/m4F5Zmh5j2o3v8F61faZ3Qbj81jj+9v1FqR5oi/SNBOFSj+xGQ8CRzXMayW3cC/U 67qGjetTqv5t8S38SR8d+sgHKd5jjNpt+trVKO1pk1UJg4E9F7jevbrZb8B2kqeuTf03 LTXYLrbkeVB4hjPnAWbrYNOHX3XvrHVb7fMpyX8/G3fvQrjCnEMNOAcqiFr/o/s4EVIY x/MsCoDcmZLZX65z+lpsLNC7HGp+yh634rO+NBd25lOkKHq6LQjfsboZ6lVGbKNdaDWx 9Zow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:to:cc:references:from:openpgp :autocrypt:message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=ps08Je2LxRAAc2g1TRftceYXyGVuoaLm+xsAB1n7sdg=; b=pQavRIE7kIWFfOBCEuFI52+lOSno25hRk7idkAUboAcBEk5gMukp6IOuvqKyAwSDfL gnOdMWUpb5hFpn7mLH9wZ00t9A+/6QBAwEug+10Hr3mcxTra0R65pzulGt3tJz/LOwcO HWYGgJ1zpc7Ahmj+SksuHjDsUrRgtnxuRREccJctGrIWIN/4QwxSXY1cj2UiXTPQftau rBd2UuV9HUVHhRDbzoxEFfBfkrjSv67USrrrRbFWJsHav8KaMmg5BWZWnI9yxOht6oC9 F/+/WD2ZCGni0J9vPMFfRY67ootpTMA2zCDFSYUuExfwd5WT3ZkKHK8KrqmXEeo6sIgE Pv3Q== X-Gm-Message-State: APjAAAUqrCuhRC8el/Lo0GEa+zEuSmjU8Z0k0NNdmPqa1HHYWCBV4Tql Xc5pk3LmSgAkjIg53RgnuuDF0tO6Ixc= X-Google-Smtp-Source: APXvYqy1W+7CbARV/X5bmqWHfXn6fJo7K0oUqA/oIKerznl6tIWbpcc48RYsP0HKK2cN6U/6vjalTw== X-Received: by 2002:a25:7642:: with SMTP id r63mr11058504ybc.253.1558558115236; Wed, 22 May 2019 13:48:35 -0700 (PDT) Received: from mavoffice.ixsystems.com ([12.189.233.129]) by smtp.gmail.com with ESMTPSA id o1sm5288711ywo.79.2019.05.22.13.48.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 22 May 2019 13:48:34 -0700 (PDT) Sender: Alexander Motin Subject: Re: svn commit: r348117 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs To: Slawa Olhovchenkov Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org References: <201905221843.x4MIhmh1012837@repo.freebsd.org> <20190522194914.GF47119@zxy.spb.ru> <4766287e-283a-775d-9e8e-4d0442d802f3@FreeBSD.org> <20190522203424.GC2161@zxy.spb.ru> From: Alexander Motin Openpgp: preference=signencrypt Autocrypt: addr=mav@FreeBSD.org; prefer-encrypt=mutual; keydata= xsBNBFOzxAwBCADkPrax0pI2W/ig0CK9nRJJwsHitAGEZ2HZiFEuti+6/4UVxj81yr4ak/4g 9bKUyC7rMEAp/ZHNhd+MFCPAAcHPvtovnfykqE/vuosCS3wlSLloix2iKVLks0CwbLHGAyne 46lTQW74Xl/33c3W1Z6d8jD9gVFT/xaVzZ0U9xdzOmsYAZaAj4ki0tuxO9F7L+ct9grRe7iP g8t9hai7BL4ee3VRwk2JXnKb7UvBiVITKYWKz1jRvZIrjPokgEcCLOSlv7x/1kjuFnj3xWZU 7HSFFT8J93epBbrSSCsYsppIk2fZH41kaaFXsMQfTPH8wkeM6qwrvOh4HiQM08R+9tThABEB AAHNIUFsZXhhbmRlciBNb3RpbiA8bWF2QEZyZWVCU0Qub3JnPsLAlwQTAQoAQQIbAwULCQgH AwUVCgkICwUWAwIBAAIeAQIXgAIZARYhBOmM88TmnMPNDledVYMYw5VbqyJ/BQJZYMKuBQkN McyiAAoJEIMYw5VbqyJ/tuUIAOG3ONOSNYqjK4eTZ1TVh9jdUBAhWk5nhDFnODN49Wj0AbYm 7aIqy8O1hnCDSZG5LttjSAo3UfXJZDKQM0BLb0gpRMBnAYqO6tdolLNqAbPGJBnGoPjsh24y 6KcbDaNnis+lD4GwPXwQM+92wZGhCUFElPV9NciZGVS65TNIgk7X+yEjjhD1MSWKKijZ1r9Z zIt4OzUTxxNOvzdlABZS88nNRdJkatOQJPmFdd1mpP6UzTNCiLUo1pIqOEtJgvVVDYq5WHY6 tciWWYdmZG/tIBexJmv2mV2OLVjXR6ZeKmntVH14H72/wRHJuYHQC+r5SVRcWWayrThsY6jZ Yr4+raTOwE0EU7PEDAEIAOZgWf2cJIu+58IzP2dkXE/urj3tr4OqrB/yHGWUf71Lz6D0Fi6Z AXgDtmcFLGPfMyWuLAvSM+xmoguk7zC4hRBYvQycmIhuqBq1jO1Wp/Z+lpoPM/1cDYLn8Flv mI/c40MhUZh345DA4jYWWaZNjQHUWVQ1fPf595vdVVMPT/abE8E5DaF6fSkRmqFTmfYRkfbt 3ytU8NdUapDcJVY7cEP2nJBVNZPnOIObR/ZIgSxjjrG5o34yXoqeup8JvwEv+/NylzzuyXEZ R1EdEIzQ/a1nh/0j4NXtzZEqKW4aTWlmSqb6wN8jh1OSOOqkYsfnE3nfxcZbxi4IRoNQYlm5 9R8AEQEAAcLAZQQYAQoADwUCU7PEDAIbDAUJBaOagAAKCRCDGMOVW6sif7FRB/4k9y/GaGqU fcJiXdQHRAKHCUvbKMFgeEDHOg33qx+POS2Ah85/PXVa2jYBldCZDmYc+zl48aEMd163a7s3 0gJaB7CYElwxlKUk6c+5gwoYIJuJJzSzW0JzSD5ch7RIRxbfxrKdsiHrUW8AeduZWzlK6VaW RmWILgLmxfLdhEVFWxbr99GSeVFZaZwn6tl/8CvBcgYoARvJvl0V5zS1akQfEISYkwL9EfUI W44EOHranL5qUXkedXBYp6fRsooGrIimfwYxaC8FbXhk3FMgMjDMRiVq4POHo1iGeYETsUrL NM6184E25gPVtX2fb3RhM8Xh6BkwCZ6ZYbQ+AcD4F/cKwsB8BBgBCgAmAhsMFiEE6YzzxOac w80OV51VgxjDlVurIn8FAllgwtgFCQ0xzMwACgkQgxjDlVurIn9OqAf9FAcKWS95wTTbraXA qg/+bQyHgjlMtGCgkmfxLsbUGeqiFgmSIuoDrF7q6sYPs6p00CXXZRuuNZt0lX7O95re8mgz gxm5iJisZpdbHMVepYlw/AxT2wCHwxGCEe64Lm+A9vjlOd+3D3/6fSLwZ9WFCE6p6lQZ1CDg 09xe+JKSgC+KDqmn0tzGKyfSCuhRAq3XkZyxL1hxBaDeP0eeKlzoy7jXodf3wVvXXc0cmpza B5McuRHK4EU6jIioHo30YqPM4AjPHGxV2X1N6/Aayungzj9EXNZtKCxs6dsTvjniWa5VkZ9F 4SOdSbxEen1DZRYpeWnd7GVmO86n+5USkKCXPg== Message-ID: Date: Wed, 22 May 2019 16:48:34 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <20190522203424.GC2161@zxy.spb.ru> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 581D497E75 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.977,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 May 2019 20:48:37 -0000 On 22.05.2019 16:34, Slawa Olhovchenkov wrote: > On Wed, May 22, 2019 at 04:00:58PM -0400, Alexander Motin wrote: > >> On 22.05.2019 15:49, Slawa Olhovchenkov wrote: >>> On Wed, May 22, 2019 at 06:43:48PM +0000, Alexander Motin wrote: >>>> Author: mav >>>> Date: Wed May 22 18:43:48 2019 >>>> New Revision: 348117 >>>> URL: https://svnweb.freebsd.org/changeset/base/348117 >>>> >>>> Log: >>>> Allocate buffers smaller then ABD chunk size as linear. >>>> >>>> This allows to reduce memory waste by letting UMA to put multiple small >>>> buffers into one memory page slab. The page sharing means that UMA >>>> may not be able to free memory page when some of buffers are freed, but >>>> alternatively memory used by that buffer would just be wasted from the >>>> beginning. >>>> >>>> This change follows alike change in ZoL, but unlike Linux (according to >>>> my understanding of it from comments) FreeBSD never shares slabs bigger >>>> then one memory page, so this should be even less invasive then there. >>>> >>>> MFC after: 2 weeks >>>> Sponsored by: iXsystems, Inc. >>>> >>>> Modified: >>>> head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/abd.c >>>> >>>> Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/abd.c >>>> ============================================================================== >>>> --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/abd.c Wed May 22 17:42:22 2019 (r348116) >>>> +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/abd.c Wed May 22 18:43:48 2019 (r348117) >>>> @@ -290,7 +290,7 @@ abd_free_struct(abd_t *abd) >>>> abd_t * >>>> abd_alloc(size_t size, boolean_t is_metadata) >>>> { >>>> - if (!zfs_abd_scatter_enabled) >>>> + if (!zfs_abd_scatter_enabled || size <= zfs_abd_chunk_size) >>>> return (abd_alloc_linear(size, is_metadata)); >>> >>> may be `size < zfs_abd_chunk_size`? >> >> Why? It should be the same from memory usage, but I see theoretical >> benefits from having linear buffer with one page rather then scatter >> list with one page. Plus ZoL also had there `size <= PAGESIZE`. > > May be malloc() for page size more effictive? Not sure. > Less overhead for kegs and etc. malloc()'s up to 64KB are also backed by UMA. May be it could be optimized on platforms with direct map, and may be after 12 VM subsystem would not be too shocked, but on platforms without direct map requests going around UMA causes huge TLB shootdown on free(). Thinking again, I see one benefit of allocating 4KB buffers not as linear but as ABD -- using same ABD UMA zone for both 4KB and above should improve its caching efficiency. On the other side I already saw lock congestion on UMA locks there, so there may be trade-off. -- Alexander Motin