From owner-freebsd-hackers@freebsd.org Sun Jan 19 18:32:59 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5ED702219F8 for ; Sun, 19 Jan 2020 18:32:59 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: from puchar.net (puchar.net [194.1.144.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4813MF38B8z4YDV for ; Sun, 19 Jan 2020 18:32:57 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: Received: from 127.0.0.1 (localhost [127.0.0.1]) by puchar.net (8.15.2/8.15.2) with ESMTPS id 00JIWr14018348 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Sun, 19 Jan 2020 19:32:54 +0100 (CET) (envelope-from puchar-wojtek@puchar.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=puchar.net; s=default; t=1579458774; bh=E8yj3QoBlTT4rv8BGo3vigT9MwOXEUHfcWFd1DcPAoA=; h=Date:From:To:Subject; b=X89gTJ2UDb3UVuNXWPEK9PkaQzrdBl93/CJzNdG1ca8fDcwhj2WFaxNQ4qhVQ7pHK CEjRLic1NtjVGmKwfN4jF489lSlALPq1GTDb97FsSHeUAXmQ4PRlx78bjsUHQ79ayw lumjxZ3WEq3E+Hut+zN3Fd05pTP0rb7329drEKTE= Received: from localhost (puchar-wojtek@localhost) by puchar.net (8.15.2/8.15.2/Submit) with ESMTP id 00JIWr1U018345 for ; Sun, 19 Jan 2020 19:32:53 +0100 (CET) (envelope-from puchar-wojtek@puchar.net) Date: Sun, 19 Jan 2020 19:32:53 +0100 (CET) From: Wojciech Puchar To: freebsd-hackers@freebsd.org Subject: slow directory operation on huge dirs Message-ID: User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset=US-ASCII X-Rspamd-Queue-Id: 4813MF38B8z4YDV X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=fail (rsa verify failed) header.d=puchar.net header.s=default header.b=X89gTJ2U; dmarc=none; spf=pass (mx1.freebsd.org: domain of wojtek@puchar.net designates 194.1.144.90 as permitted sender) smtp.mailfrom=wojtek@puchar.net X-Spamd-Result: default: False [-3.46 / 15.00]; ARC_NA(0.00)[]; SUBJECT_ENDS_SPACES(0.50)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; R_DKIM_REJECT(1.00)[puchar.net:s=default]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; DMARC_NA(0.00)[puchar.net]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[puchar.net:-]; RCVD_IN_DNSWL_NONE(0.00)[90.144.1.194.list.dnswl.org : 127.0.10.0]; IP_SCORE(-2.66)[ip: (-7.03), ipnet: 194.1.144.0/24(-3.52), asn: 43476(-2.81), country: PL(0.07)]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:43476, ipnet:194.1.144.0/24, country:PL]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 Jan 2020 18:32:59 -0000 i have UFS_DIRHASH enabled and enough large dirhash_maxmem yet when there is heavy I/O on large directory (like dovecot indexing so it reads every file) it randomly works fast or slow. Most often slow. Whole system slows down, and system cpu usage gets above 50% on quad core Xeon. lowering maxvnodes from 500000 to 100000 "fixes" the problem. what is going on here? 11.3-STABLE from maybe 2 weeks ago. From owner-freebsd-hackers@freebsd.org Wed Jan 22 22:34:00 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0BC391FF72A for ; Wed, 22 Jan 2020 22:34:00 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from mailhost.m5p.com (mailhost.m5p.com [74.104.188.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "m5p.com", Issuer "Let's Encrypt Authority X3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4830Yz0Fvgz4d3t for ; Wed, 22 Jan 2020 22:33:55 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from [IPv6:2001:470:1f07:15ff::26] (court.m5p.com [IPv6:2001:470:1f07:15ff:0:0:0:26]) (authenticated bits=0) by mailhost.m5p.com (8.15.2/8.15.2) with ESMTPSA id 00MMXeC6027259 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Wed, 22 Jan 2020 17:33:47 -0500 (EST) (envelope-from george+freebsd@m5p.com) To: FreeBSD Hackers From: George Mitchell Autocrypt: addr=george+freebsd@m5p.com; prefer-encrypt=mutual; keydata= mQINBFgnLnwBEADAJDiBKQX77LFRz9wZW8mz3KvaQol2nIremcws0F1mz/zgFlk6uhQVtwnL wb4XL5LdFwcNE1+QZzPLcbYWoWQlz0lBw1bMuKAgr0S6V2e0+I0DqhKeslVFctcTwtvT6pnK VLZXO/7ZGAaLzG4K5vSPzgoevU+YI/pxNsVCH2UO/c3jQW63uEt25mIZbCF1Pu4jgp4RhIgF ujn877r/j6OwBwjzRUu3E6ADp+U825d+5YCuQMEH0wIPnn9GTpXvfdKdbwOIl2akqXqs4cnk iATWfK3r6D4mvDEj1OPHlTvJYcfic7aOIiAwmx1C1v78GjXOdOOA0SGffNix3C2/8oZUO1+V Aet4MKpUKkduWSvULhIkHNZ5Nu8SIJOqge8pmtHxuNXAMfMrAjMdjPwwBFLsYg3Xa2E2oJwg ehTauwd/EDJFcVCyDCyCAYOi/BH/+XQyxzgDlY9N9qj9tHqhVPI6XK7t8UVffGiZUq4rHp5J RdOToqiTNC6eCJBczhMIW+DuFvWU9e6W708T1dz0Accn6Lrgk4eRIn3GFPBG+TxnpjAqHsbW 607dcnD3YKAqY4e+khczL4EObhe7dC1v2fmZiAC6Ds3WHR11IfqoUgCkIwJ590Ej+ElygJFF XxI82wtEz9hkeLLvItpyEJNVjppViRW+Dgl/U7ypHB3qDgYjgwARAQABtChHZW9yZ2UgTWl0 Y2hlbGwgPGdlb3JnZStmcmVlYnNkQG01cC5jb20+iQJUBBMBCAA+FiEENdM4ZHktsJW5kKZX wRES3m+p4fkFAlhZcR0CGyMFCQlmAYAFCwkIBwIGFQgJCgsCBBYCAwECHgECF4AACgkQwRES 3m+p4flqmw/9Emr/ydTG2n9o/IX1yVCNcHVFenVrcOY0L+DGQYZRO/XpLvsGYcuSIQId1w7h l4HZKI89ri2fF2ks6upMqBajLf8s7a8PnYrbw5bPaoOFyNTjv57GLZVsYw95kmMUpK6siuAA fXvHfKUpC/sThbwSv/1CLryVG74+5vdI8j7cQeDM436FThxlVfHKrILIiL34D4WThFB3hV/Y 2A+mQwXmdLcuQXXeAazqsFJL8sgEKSC7GMcExDkVpGc5Rh2hu97a4Sa7qWX9G/YdZOrcDacJ XxfvePn3m3WfRtXN/r0lUfiVXiqkFfbvqSaZQ0I4UvZXNGd/gH4jKHtX9RTH9G96UZeHNoMo tPw9U0fx8Ceh72nUL9qzqnmok/ryWm+6gt4Q1eRP7QAosOa1g/RgUdS1Z9IuCmbXMDp5kbNw L4ZoDMF5U3mmh8/IOKkhGopNLbNv0mwUgC59pnCptiOVx8DyckXWC4L2r6PKbWGrcGIzsUER 9smfL10gpp5H5agjwwPZI6/kzJ0R5nBzQWAlwqI73YAy6JI0HTD6lvxW7yWm2fGjEfmyaBOU 8OLUin7auoFSn+QmD5yNCUn5Ls77qHARkT1ZGocAnQkvZBGTwXpvyJixygXsm+vSUFDYBOSn cR54vdXOEMqrJk6SGau5YI9V7EhQVveE1BUp8ofWf2oo4RG5Ag0EWCcufAEQALuTOxmqMFE+ ieev/rcL3wVJrcuKS+pBbKCY9IIL0OwVf98HQJJcgdOsdDhruVd19nJNlwZ3Fc34wLw7y2GO 9WrpZiYKnI4n9urhLE5r1ydBInlI/1UKZWgM3/dPjJtcXMsC3vnqR9DmOxW4/SbqJDjP3XzO FleT4yip3AaNhPGwEPTZrubVp5hp/JojaZn690TLRwOFXg8NcjpOEs0Bq9M+OLpmsF0flrgs yDfS7y+SQ17R4Iq9T7RxZvZVAh510yGGIZIETYO/4Dh417VVm+gaksOVh2egetpUUvYYc0Ub KaP+5F/WGNrmRb1F6SKypvLlKkYAHCsUUSzsAGl9gbQhEEpuOMbUKp1979HoRMkW+8046kIo 8BZ6ph8izG/g4dZOaEqKGEhqdhYIB7UwMtFFuPtSs5Nl6JrZYni/nzFtTmtVCgcj9PNqrzqt fNFYhNznD6St6wxp3TOm9D3TQF0dzwBM1jZpb8WvmK3k6oy8hbpjiBzxn3kyRA9Vzy+PdbN/ G5a1k0rpZu/ivpBuLCDVGljUmgQigXg6xkk5UxBoHp7MPvG9prZ5jqdEa2r1KgnGjaq+VJsu Uqrw10dVgeG1NulDU1+sQl+/mwtflbkimhjDDjxsVgfrv7uvV/9be+gGm1KATuqdgCboSb1s QAo5ARfwFfChrnh+fTfPpPKHABEBAAGJAiQEGAEIAA8FAlgnLnwCGwwFCQlmAYAACgkQwRES 3m+p4fno9w/4m+swztkzxSWdutjgSv2mw+PdrKWVGFAUD2HoY1Qpi5LNLE6s9pP3qzwpQYwK viOufVJYWZ540ss6BImZBGJwyHouacqrpZjpRo5+ftj07rY1SNd8QjcHDggPfpgJ1D4Il3Xi vRg5/gzkXnRu8dXeVvMP1Ndk/F5wcoLZlQwFtPfu2xyRYIsveXMoyypAvAFSaAGXU0hRzuDJ fGI3LFvpI9UXU2C4MMzjfyZyD2NJEDKOACTo85QQzxgheTDQaDocXW00wknXFMwEItiXp8dO 2zEml/3Kj4efDfjqGpjNefjK0cnj02Byt7y6GozWXyIylrXu0SN9qWRzUVZH3+q+ijA4q3Gm 9uWzLdpjN4QWAiiaEvMhLPohp9DdLsy3kAWWrA3+pAfHSTZXrobMMbSeBkE9E4/WxdKl0nM7 TNslAWcxkTd/7Ly9cxwT8wFdHuQB1hgCmIQxDNXHL1N1ANTeUYum1w9nUg6e1M0UWu+nk3Cw qL7oL2KZe13mQnU/CFwlhbf+i//j3SXrQLlIVQv9Fn805bxIcVo9yqUZyoiV7EUpvOsxDCZh ej3mNYF5nRCf6trEJQVk0aLC26zJAYExykdUlRqc4I13XPhlt+aFSMMkoL/thYO6e9oNFK6Q aJEKXomzxxqpceJVmPH6zvqJbOboAdE/mOD0PoS1M6saIQ== Subject: Odd svn error Message-ID: <740d3745-7cc8-3640-1701-2efdadf28d03@m5p.com> Date: Wed, 22 Jan 2020 17:33:34 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.3.1 MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="Eoyy1AN7NvmojJaiyysTJ03fyo78kNtnk" X-Spam-Status: No, score=1.2 required=10.0 tests=HELO_MISC_IP,HELO_NO_DOMAIN autolearn=no autolearn_force=no version=3.4.3 X-Spam-Level: * X-Spam-Checker-Version: SpamAssassin 3.4.3 (2019-12-06) on mattapan.m5p.com X-Rspamd-Queue-Id: 4830Yz0Fvgz4d3t X-Spamd-Bar: ------- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of george@m5p.com designates 74.104.188.4 as permitted sender) smtp.mailfrom=george@m5p.com X-Spamd-Result: default: False [-7.22 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+a]; TO_MATCH_ENVRCPT_ALL(0.00)[]; HAS_ATTACHMENT(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; DMARC_NA(0.00)[m5p.com]; TO_DN_ALL(0.00)[]; IP_SCORE(-2.82)[ip: (-9.86), ipnet: 74.104.0.0/16(-4.93), asn: 701(0.76), country: US(-0.05)]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:~]; ASN(0.00)[asn:701, ipnet:74.104.0.0/16, country:US]; TAGGED_FROM(0.00)[freebsd]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jan 2020 22:34:00 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --Eoyy1AN7NvmojJaiyysTJ03fyo78kNtnk Content-Type: multipart/mixed; boundary="kCCjFG57NTCy8S36CiB6f6kNIje8DdTzJ"; protected-headers="v1" From: George Mitchell To: FreeBSD Hackers Message-ID: <740d3745-7cc8-3640-1701-2efdadf28d03@m5p.com> Subject: Odd svn error --kCCjFG57NTCy8S36CiB6f6kNIje8DdTzJ Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable In preparation for doing a base system build on my Raspberry Pi, I have set up a separate /usr/src and /usr/obj tree on an amd86 machine and NFS exported it to my Pi. In that /usr/src tree, the amd86 reports a plausible bunch of stuff about the tree, but on the Pi, "svnlite info" says: svn: E155016: The working copy database at '/usr/src' is corrupt. /usr/src on the Pi is where I NFS-mounted the tree. "ls -l" for the .svn directory looks the same on both machines, and specifically the SHA256 of .svn/wc.db is the same on both machines. Am I running into byte order problems? I thought SQLite took care of that sort of thing. I guess I can just do a fresh checkout, but I had hoped to avoid that step ... -- George --kCCjFG57NTCy8S36CiB6f6kNIje8DdTzJ-- --Eoyy1AN7NvmojJaiyysTJ03fyo78kNtnk Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEENdM4ZHktsJW5kKZXwRES3m+p4fkFAl4ozcQACgkQwRES3m+p 4fmVwBAAsCYTKgD2h1udkPGhK+rInvc2fUZkJbdeZZFg1VxX8U8hqFALz3fh+tCj SpspmxEbSurVd94kYm4aFJro1YRK/x5d66urEe+pz7OroHSYJ0We6oYnNHuOUfuX QqkU9EmYc6Q8D5zj+w3YHMc6Jy9A7JoZ9mniUsORa4/NgLN0rHkB8UDwykE2WNB2 /ncOrMOneFY2KFXhbXm7QOCxflD3BRTDYZWtZcWmUaj2VvbR896b4daHFfFOe9Ec 8nLyQuWY4EOvPjdON75xiyqI69lnJYsLTDJgqsnkc5Ug9vWV5FAxM8Yl+bcXEyI8 D9adPLgV6w+CH3KuGrukf3mtd8emRPnGfo6xj5PT2/WurHzI3NA9Vc5kltLyBoRW MwmfauRYi1zZBGY0zLY7OKNqLLM/sT57YvQeyBuvxEgg36fXdkvPZAXsdEhkY5VU Bko7eB/xb7ULf53aBwWvLvCuJxuwcuBR4p8BGHwjiCexqjM6mBi98cqMuE3ce40V lXW1rhyiP9WPzHaE3dgMDEfGPiM1IyZxPJ7CgRvTlbCfs+MZY3PDp1zFBs+ggB+W NycmbDFdf3fibKiSkc18NBBoxZfCE/Zw2WnpGsARMMNH/UdnW4iIrHZMT67wkvmR 5hoVylxPUILvCa6uwLQ4oc4naRsWIO9egdcWBaVOI3TMSfOJjb4= =4EeR -----END PGP SIGNATURE----- --Eoyy1AN7NvmojJaiyysTJ03fyo78kNtnk-- From owner-freebsd-hackers@freebsd.org Wed Jan 22 22:39:10 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 010DE1FFB35 for ; Wed, 22 Jan 2020 22:39:10 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from mailhost.m5p.com (mailhost.m5p.com [74.104.188.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "m5p.com", Issuer "Let's Encrypt Authority X3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4830gx2fjMz4dHy for ; Wed, 22 Jan 2020 22:39:09 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from [IPv6:2001:470:1f07:15ff::26] (court.m5p.com [IPv6:2001:470:1f07:15ff:0:0:0:26]) (authenticated bits=0) by mailhost.m5p.com (8.15.2/8.15.2) with ESMTPSA id 00MMd0tn027298 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Wed, 22 Jan 2020 17:39:07 -0500 (EST) (envelope-from george+freebsd@m5p.com) Subject: Re: Odd svn error To: freebsd-hackers@freebsd.org References: <740d3745-7cc8-3640-1701-2efdadf28d03@m5p.com> From: George Mitchell Autocrypt: addr=george+freebsd@m5p.com; prefer-encrypt=mutual; keydata= mQINBFgnLnwBEADAJDiBKQX77LFRz9wZW8mz3KvaQol2nIremcws0F1mz/zgFlk6uhQVtwnL wb4XL5LdFwcNE1+QZzPLcbYWoWQlz0lBw1bMuKAgr0S6V2e0+I0DqhKeslVFctcTwtvT6pnK VLZXO/7ZGAaLzG4K5vSPzgoevU+YI/pxNsVCH2UO/c3jQW63uEt25mIZbCF1Pu4jgp4RhIgF ujn877r/j6OwBwjzRUu3E6ADp+U825d+5YCuQMEH0wIPnn9GTpXvfdKdbwOIl2akqXqs4cnk iATWfK3r6D4mvDEj1OPHlTvJYcfic7aOIiAwmx1C1v78GjXOdOOA0SGffNix3C2/8oZUO1+V Aet4MKpUKkduWSvULhIkHNZ5Nu8SIJOqge8pmtHxuNXAMfMrAjMdjPwwBFLsYg3Xa2E2oJwg ehTauwd/EDJFcVCyDCyCAYOi/BH/+XQyxzgDlY9N9qj9tHqhVPI6XK7t8UVffGiZUq4rHp5J RdOToqiTNC6eCJBczhMIW+DuFvWU9e6W708T1dz0Accn6Lrgk4eRIn3GFPBG+TxnpjAqHsbW 607dcnD3YKAqY4e+khczL4EObhe7dC1v2fmZiAC6Ds3WHR11IfqoUgCkIwJ590Ej+ElygJFF XxI82wtEz9hkeLLvItpyEJNVjppViRW+Dgl/U7ypHB3qDgYjgwARAQABtChHZW9yZ2UgTWl0 Y2hlbGwgPGdlb3JnZStmcmVlYnNkQG01cC5jb20+iQJUBBMBCAA+FiEENdM4ZHktsJW5kKZX wRES3m+p4fkFAlhZcR0CGyMFCQlmAYAFCwkIBwIGFQgJCgsCBBYCAwECHgECF4AACgkQwRES 3m+p4flqmw/9Emr/ydTG2n9o/IX1yVCNcHVFenVrcOY0L+DGQYZRO/XpLvsGYcuSIQId1w7h l4HZKI89ri2fF2ks6upMqBajLf8s7a8PnYrbw5bPaoOFyNTjv57GLZVsYw95kmMUpK6siuAA fXvHfKUpC/sThbwSv/1CLryVG74+5vdI8j7cQeDM436FThxlVfHKrILIiL34D4WThFB3hV/Y 2A+mQwXmdLcuQXXeAazqsFJL8sgEKSC7GMcExDkVpGc5Rh2hu97a4Sa7qWX9G/YdZOrcDacJ XxfvePn3m3WfRtXN/r0lUfiVXiqkFfbvqSaZQ0I4UvZXNGd/gH4jKHtX9RTH9G96UZeHNoMo tPw9U0fx8Ceh72nUL9qzqnmok/ryWm+6gt4Q1eRP7QAosOa1g/RgUdS1Z9IuCmbXMDp5kbNw L4ZoDMF5U3mmh8/IOKkhGopNLbNv0mwUgC59pnCptiOVx8DyckXWC4L2r6PKbWGrcGIzsUER 9smfL10gpp5H5agjwwPZI6/kzJ0R5nBzQWAlwqI73YAy6JI0HTD6lvxW7yWm2fGjEfmyaBOU 8OLUin7auoFSn+QmD5yNCUn5Ls77qHARkT1ZGocAnQkvZBGTwXpvyJixygXsm+vSUFDYBOSn cR54vdXOEMqrJk6SGau5YI9V7EhQVveE1BUp8ofWf2oo4RG5Ag0EWCcufAEQALuTOxmqMFE+ ieev/rcL3wVJrcuKS+pBbKCY9IIL0OwVf98HQJJcgdOsdDhruVd19nJNlwZ3Fc34wLw7y2GO 9WrpZiYKnI4n9urhLE5r1ydBInlI/1UKZWgM3/dPjJtcXMsC3vnqR9DmOxW4/SbqJDjP3XzO FleT4yip3AaNhPGwEPTZrubVp5hp/JojaZn690TLRwOFXg8NcjpOEs0Bq9M+OLpmsF0flrgs yDfS7y+SQ17R4Iq9T7RxZvZVAh510yGGIZIETYO/4Dh417VVm+gaksOVh2egetpUUvYYc0Ub KaP+5F/WGNrmRb1F6SKypvLlKkYAHCsUUSzsAGl9gbQhEEpuOMbUKp1979HoRMkW+8046kIo 8BZ6ph8izG/g4dZOaEqKGEhqdhYIB7UwMtFFuPtSs5Nl6JrZYni/nzFtTmtVCgcj9PNqrzqt fNFYhNznD6St6wxp3TOm9D3TQF0dzwBM1jZpb8WvmK3k6oy8hbpjiBzxn3kyRA9Vzy+PdbN/ G5a1k0rpZu/ivpBuLCDVGljUmgQigXg6xkk5UxBoHp7MPvG9prZ5jqdEa2r1KgnGjaq+VJsu Uqrw10dVgeG1NulDU1+sQl+/mwtflbkimhjDDjxsVgfrv7uvV/9be+gGm1KATuqdgCboSb1s QAo5ARfwFfChrnh+fTfPpPKHABEBAAGJAiQEGAEIAA8FAlgnLnwCGwwFCQlmAYAACgkQwRES 3m+p4fno9w/4m+swztkzxSWdutjgSv2mw+PdrKWVGFAUD2HoY1Qpi5LNLE6s9pP3qzwpQYwK viOufVJYWZ540ss6BImZBGJwyHouacqrpZjpRo5+ftj07rY1SNd8QjcHDggPfpgJ1D4Il3Xi vRg5/gzkXnRu8dXeVvMP1Ndk/F5wcoLZlQwFtPfu2xyRYIsveXMoyypAvAFSaAGXU0hRzuDJ fGI3LFvpI9UXU2C4MMzjfyZyD2NJEDKOACTo85QQzxgheTDQaDocXW00wknXFMwEItiXp8dO 2zEml/3Kj4efDfjqGpjNefjK0cnj02Byt7y6GozWXyIylrXu0SN9qWRzUVZH3+q+ijA4q3Gm 9uWzLdpjN4QWAiiaEvMhLPohp9DdLsy3kAWWrA3+pAfHSTZXrobMMbSeBkE9E4/WxdKl0nM7 TNslAWcxkTd/7Ly9cxwT8wFdHuQB1hgCmIQxDNXHL1N1ANTeUYum1w9nUg6e1M0UWu+nk3Cw qL7oL2KZe13mQnU/CFwlhbf+i//j3SXrQLlIVQv9Fn805bxIcVo9yqUZyoiV7EUpvOsxDCZh ej3mNYF5nRCf6trEJQVk0aLC26zJAYExykdUlRqc4I13XPhlt+aFSMMkoL/thYO6e9oNFK6Q aJEKXomzxxqpceJVmPH6zvqJbOboAdE/mOD0PoS1M6saIQ== Message-ID: <8d900391-a95b-51a9-c8bb-fead7c0fc59d@m5p.com> Date: Wed, 22 Jan 2020 17:39:00 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.3.1 MIME-Version: 1.0 In-Reply-To: <740d3745-7cc8-3640-1701-2efdadf28d03@m5p.com> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="vYBY5jJrYaYP7fYH16fKZfd9T934Xj6JA" X-Spam-Status: No, score=1.2 required=10.0 tests=HELO_MISC_IP,HELO_NO_DOMAIN autolearn=no autolearn_force=no version=3.4.3 X-Spam-Level: * X-Spam-Checker-Version: SpamAssassin 3.4.3 (2019-12-06) on mattapan.m5p.com X-Rspamd-Queue-Id: 4830gx2fjMz4dHy X-Spamd-Bar: ------- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of george@m5p.com designates 74.104.188.4 as permitted sender) smtp.mailfrom=george@m5p.com X-Spamd-Result: default: False [-7.22 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+a:c]; TO_MATCH_ENVRCPT_ALL(0.00)[]; HAS_ATTACHMENT(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_DN_NONE(0.00)[]; IP_SCORE(-2.82)[ip: (-9.87), ipnet: 74.104.0.0/16(-4.93), asn: 701(0.75), country: US(-0.05)]; DMARC_NA(0.00)[m5p.com]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:~]; ASN(0.00)[asn:701, ipnet:74.104.0.0/16, country:US]; TAGGED_FROM(0.00)[freebsd]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jan 2020 22:39:10 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --vYBY5jJrYaYP7fYH16fKZfd9T934Xj6JA Content-Type: multipart/mixed; boundary="6CwXvpsX8WPYHp0VtdxZmsi7xsNEg5Swt"; protected-headers="v1" From: George Mitchell To: freebsd-hackers@freebsd.org Message-ID: <8d900391-a95b-51a9-c8bb-fead7c0fc59d@m5p.com> Subject: Re: Odd svn error References: <740d3745-7cc8-3640-1701-2efdadf28d03@m5p.com> In-Reply-To: <740d3745-7cc8-3640-1701-2efdadf28d03@m5p.com> --6CwXvpsX8WPYHp0VtdxZmsi7xsNEg5Swt Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 2020-01-22 17:33, George Mitchell wrote: > [blah blah blah] I think I have a permissions problem; sorry for the noise. -- George --6CwXvpsX8WPYHp0VtdxZmsi7xsNEg5Swt-- --vYBY5jJrYaYP7fYH16fKZfd9T934Xj6JA Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEENdM4ZHktsJW5kKZXwRES3m+p4fkFAl4ozwQACgkQwRES3m+p 4flLdw/+K1QW1cCfaOlJKH87alG2Q4UJ9sw/M+HMlaR3jJDyu29wFHN3xLJ+r6yN 16H3HY8Q8LOCrtUD62R/NYOrnuCApMLrbStrgqpwiHKS4RivVVlUEGJ5zOrnWM/m ABYwkDdybN56OZlY1lCVq1S15IHNG2iXYztP+leIoshbUEEgwZyF3hg1vGniPyiy 4yIfzmm8V8EkIvIdxhu1kHibGpvXlF4AEUOj1VvpEFVQcdV7B0yUJc7Q7wseOaqk hACsLNq56tJD5XFtP4Vzw63zx51k5IGkXxIrQhXc16UBnIaJm13YNip6kZGi/sL5 IKXKEQCeUI97XUR+eQe7NQieDTn87cfPjgzBmI1evHkJelmkuk7FUSc8Af1fG1Xh nvuJFZGGjxahYv0POZvuRpBa/xGKtwA3YehfodZ24iVPS3l5hExxvXYWCfXHBRTn vyenbGzW4AJHoMCCm7lBaL/QnJvKjXU+6ycySRAjpZG6ONzKZvKtFOTQv6GF0B7t AoI0w2XsiWrrb/RWGMBYjajvqH9JqenvUHO46GWuR8i92MAoSJ92HsAvg0+WMcHy w5QXoDUewnEr4x6ablHln3w4u+Iv3fsx2g6J+G67qbV9HCQW0qu54uL2ZxE/jUHk EeqYmNFkhEdH0mBPqMODFiaQI/XthkwYcFvVwV5iRquUABso4xE= =3qbZ -----END PGP SIGNATURE----- --vYBY5jJrYaYP7fYH16fKZfd9T934Xj6JA-- From owner-freebsd-hackers@freebsd.org Thu Jan 23 05:38:48 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D785B22D1C9 for ; Thu, 23 Jan 2020 05:38:48 +0000 (UTC) (envelope-from trond.endrestol@ximalas.info) Received: from enterprise.ximalas.info (enterprise.ximalas.info [IPv6:2001:700:1100:1::8]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "ximalas.info", Issuer "Hostmaster ximalas.info" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 483B070MXWz48Cc for ; Thu, 23 Jan 2020 05:38:46 +0000 (UTC) (envelope-from trond.endrestol@ximalas.info) Received: from enterprise.ximalas.info (Ximalas@localhost [127.0.0.1]) by enterprise.ximalas.info (8.15.2/8.15.2) with ESMTPS id 00N5cWWK077570 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Thu, 23 Jan 2020 06:38:33 +0100 (CET) (envelope-from trond.endrestol@ximalas.info) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ximalas.info; s=default; t=1579757913; bh=XJN2lltOmaEpoLGgRw3VSWpfVGRw0S/DEUoCQCZOJ3E=; h=Date:From:To:cc:Subject:In-Reply-To:References; b=jVWk4e2Fj+K80LH03/Uyo8koRunqbzAT007vRgwqww/XZ2IgfyTL5fkTA6fLq3qGe 77MNSU9gArmefRSo6vcukXYykzzpzVEXOhUdaS542qChGqN2MHfhl7zXuD+La87csU 3IAtdUA+ntYsKCFRXm1R0ysi0rFIP+lgYmbFzOUWkXXX2VTIfcQ9Kgt8g/FRQjJyVg 81A3p53i10MRqTgiju6f4UAHNh8+xomKJw6kJj3wB5FGGvVWsCNhXDyUAVHqNznQlZ A3e1Yb3pcZKR1c9EpgsU3BdtyztoC6fuMXa2kjJ9QUGjeinDhwXjeGKc24puZf8vK7 nV6nnP//GBEeA== Received: from localhost (trond@localhost) by enterprise.ximalas.info (8.15.2/8.15.2/Submit) with ESMTP id 00N5cWYP077567; Thu, 23 Jan 2020 06:38:32 +0100 (CET) (envelope-from trond.endrestol@ximalas.info) X-Authentication-Warning: enterprise.ximalas.info: trond owned process doing -bs Date: Thu, 23 Jan 2020 06:38:32 +0100 (CET) From: =?UTF-8?Q?Trond_Endrest=C3=B8l?= Sender: Trond.Endrestol@ximalas.info To: George Mitchell cc: freebsd-hackers@freebsd.org Subject: Re: Odd svn error In-Reply-To: <8d900391-a95b-51a9-c8bb-fead7c0fc59d@m5p.com> Message-ID: References: <740d3745-7cc8-3640-1701-2efdadf28d03@m5p.com> <8d900391-a95b-51a9-c8bb-fead7c0fc59d@m5p.com> User-Agent: Alpine 2.22 (BSF 395 2020-01-19) OpenPGP: url=http://ximalas.info/about/tronds-openpgp-public-key MIME-Version: 1.0 Content-Type: text/plain; CHARSET=US-ASCII Content-ID: X-Spam-Status: No, score=-0.8 required=5.0 tests=ALL_TRUSTED,DKIM_INVALID, DKIM_SIGNED autolearn=no autolearn_force=no version=3.4.3 X-Spam-Checker-Version: SpamAssassin 3.4.3 (2019-12-06) on enterprise.ximalas.info X-Rspamd-Queue-Id: 483B070MXWz48Cc X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ximalas.info header.s=default header.b=jVWk4e2F; dmarc=pass (policy=none) header.from=ximalas.info; spf=pass (mx1.freebsd.org: domain of trond.endrestol@ximalas.info designates 2001:700:1100:1::8 as permitted sender) smtp.mailfrom=trond.endrestol@ximalas.info X-Spamd-Result: default: False [-5.02 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[ximalas.info:s=default]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+a]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TAGGED_RCPT(0.00)[freebsd]; HAS_XAW(0.00)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[ximalas.info:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[ximalas.info,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:224, ipnet:2001:700::/32, country:NO]; IP_SCORE(-2.02)[ip: (-8.03), ipnet: 2001:700::/32(-1.32), asn: 224(-0.73), country: NO(-0.01)] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2020 05:38:48 -0000 On Wed, 22 Jan 2020 17:39-0500, George Mitchell wrote: > On 2020-01-22 17:33, George Mitchell wrote: > > [blah blah blah] > I think I have a permissions problem; sorry for the noise. -- George Remember to use the mount option nlockd for NFS mounted /usr/src. -- Trond. From owner-freebsd-hackers@freebsd.org Thu Jan 23 11:45:36 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7F0F2236718 for ; Thu, 23 Jan 2020 11:45:36 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: from puchar.net (puchar.net [194.1.144.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 483L7M2CHNz4WTh for ; Thu, 23 Jan 2020 11:45:34 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: Received: from 127.0.0.1 (localhost [127.0.0.1]) by puchar.net (8.15.2/8.15.2) with ESMTPS id 00NBjSnb098758 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 23 Jan 2020 12:45:29 +0100 (CET) (envelope-from puchar-wojtek@puchar.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=puchar.net; s=default; t=1579779929; bh=VeP8sSxtZUaIvPd6FiHqiTvp4ruVJQSwMiDsZXpS/Wo=; h=Date:From:To:cc:Subject:In-Reply-To:References; b=Rq+lw1YGiFiwWvh1BFrAkVvA8NifUAiV76Tkk6T9uzRpJbOX/HzpHV0P0+WJqxHe7 T5N+rWE9AKbKpkapzdfIiehiAyC9TssD/giLFB3Jl/8CvyyMxupUv/UCp3bgSH4WnZ uRsoJ4tx4Lil1G5Miw4NJffQRXEVCeqQ7LYT1/d0= Received: from localhost (puchar-wojtek@localhost) by puchar.net (8.15.2/8.15.2/Submit) with ESMTP id 00NBjSHA098754; Thu, 23 Jan 2020 12:45:28 +0100 (CET) (envelope-from puchar-wojtek@puchar.net) Date: Thu, 23 Jan 2020 12:45:28 +0100 (CET) From: Wojciech Puchar To: Wojciech Puchar cc: freebsd-hackers@freebsd.org Subject: Re: slow directory operation on huge dirs In-Reply-To: Message-ID: References: User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Rspamd-Queue-Id: 483L7M2CHNz4WTh X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=fail (rsa verify failed) header.d=puchar.net header.s=default header.b=Rq+lw1YG; dmarc=none; spf=pass (mx1.freebsd.org: domain of wojtek@puchar.net designates 194.1.144.90 as permitted sender) smtp.mailfrom=wojtek@puchar.net X-Spamd-Result: default: False [-3.45 / 15.00]; ARC_NA(0.00)[]; SUBJECT_ENDS_SPACES(0.50)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; R_DKIM_REJECT(1.00)[puchar.net:s=default]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[puchar.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[puchar.net:-]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[90.144.1.194.list.dnswl.org : 127.0.10.0]; IP_SCORE(-2.65)[ip: (-7.00), ipnet: 194.1.144.0/24(-3.50), asn: 43476(-2.80), country: PL(0.07)]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:43476, ipnet:194.1.144.0/24, country:PL]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2020 11:45:36 -0000 On Sun, 19 Jan 2020, Wojciech Puchar wrote: > i have UFS_DIRHASH enabled and enough large dirhash_maxmem > yet when there is heavy I/O on large directory (like dovecot indexing so it > reads every file) it randomly works fast or slow. Most often slow. > Whole system slows down, and system cpu usage gets above 50% on quad core > Xeon. > > lowering maxvnodes from 500000 to 100000 "fixes" the problem. > > what is going on here? > > 11.3-STABLE from maybe 2 weeks ago. > nobody have idea what's the problem? From owner-freebsd-hackers@freebsd.org Thu Jan 23 13:00:55 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 204CC1F1584 for ; Thu, 23 Jan 2020 13:00:55 +0000 (UTC) (envelope-from SRS0=hgVM=3M=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 483MpG07rRz4bWq for ; Thu, 23 Jan 2020 13:00:53 +0000 (UTC) (envelope-from SRS0=hgVM=3M=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 32FFB28426; Thu, 23 Jan 2020 14:00:51 +0100 (CET) Received: from illbsd.quip.test (ip-62-24-92-232.net.upcbroadband.cz [62.24.92.232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 70D6028417; Thu, 23 Jan 2020 14:00:50 +0100 (CET) Subject: Re: slow directory operation on huge dirs To: Wojciech Puchar Cc: freebsd-hackers@freebsd.org References: From: Miroslav Lachman <000.fbsd@quip.cz> Message-ID: Date: Thu, 23 Jan 2020 14:00:50 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Firefox/52.0 SeaMonkey/2.49.3 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 483MpG07rRz4bWq X-Spamd-Bar: ++++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of SRS0=hgVM=3M=quip.cz=000.fbsd@elsa.codelab.cz has no SPF policy when checking 94.124.105.4) smtp.mailfrom=SRS0=hgVM=3M=quip.cz=000.fbsd@elsa.codelab.cz X-Spamd-Result: default: False [4.06 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; IP_SCORE(0.86)[ip: (0.35), ipnet: 94.124.104.0/21(0.18), asn: 42000(3.70), country: CZ(0.09)]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[quip.cz]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.99)[0.995,0]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[4.105.124.94.list.dnswl.org : 127.0.10.0]; NEURAL_SPAM_LONG(1.00)[1.000,0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[000.fbsd@quip.cz,SRS0=hgVM=3M=quip.cz=000.fbsd@elsa.codelab.cz]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:42000, ipnet:94.124.104.0/21, country:CZ]; FROM_NEQ_ENVFROM(0.00)[000.fbsd@quip.cz,SRS0=hgVM=3M=quip.cz=000.fbsd@elsa.codelab.cz]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2020 13:00:55 -0000 Wojciech Puchar wrote on 2020/01/23 12:45: > > > On Sun, 19 Jan 2020, Wojciech Puchar wrote: > >> i have UFS_DIRHASH enabled and enough large dirhash_maxmem >> yet when there is heavy I/O on large directory (like dovecot indexing >> so it reads every file) it randomly works fast or slow. Most often slow. >> Whole system slows down, and system cpu usage gets above 50% on quad >> core Xeon. >> >> lowering maxvnodes from 500000 to 100000 "fixes" the problem. >> >> what is going on here? >> >> 11.3-STABLE from maybe 2 weeks ago. >> > nobody have idea what's the problem? Is it "just slow" or is your disk too busy? (iostat) Can it be that pre-cacheing on huge directory makes disk too busy because there is too much to read? Miroslav Lachman From owner-freebsd-hackers@freebsd.org Thu Jan 23 13:13:02 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 15AB71F1AA5 for ; Thu, 23 Jan 2020 13:13:02 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: from puchar.net (puchar.net [194.1.144.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 483N4D6bcGz4cM8 for ; Thu, 23 Jan 2020 13:13:00 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: Received: from 127.0.0.1 (localhost [127.0.0.1]) by puchar.net (8.15.2/8.15.2) with ESMTPS id 00NDCsN1064241 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 23 Jan 2020 14:12:55 +0100 (CET) (envelope-from puchar-wojtek@puchar.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=puchar.net; s=default; t=1579785175; bh=MqJmYb+pCgIjfOqx0M1U6sdkldv6xmsCGSXDFPetwTQ=; h=Date:From:To:cc:Subject:In-Reply-To:References; b=Z0/+7egQlQX0OLNrUDHlOmIwbMFuLFmFNZrK70fIwS0sKxIDKCBN2rvldFhEvDUmK 9MayCjBljmr0UW6bwT5m1y/6Q5PmBAbuNlo51BI/kL5y16xGv6ghtZMho7hXzCANP+ 18Vl55JNcEBtC4y+zKCGBViB7RpwMC6nIQQMLTuo= Received: from localhost (puchar-wojtek@localhost) by puchar.net (8.15.2/8.15.2/Submit) with ESMTP id 00NDCsnD064238; Thu, 23 Jan 2020 14:12:54 +0100 (CET) (envelope-from puchar-wojtek@puchar.net) Date: Thu, 23 Jan 2020 14:12:54 +0100 (CET) From: Wojciech Puchar To: Miroslav Lachman <000.fbsd@quip.cz> cc: Wojciech Puchar , freebsd-hackers@freebsd.org Subject: Re: slow directory operation on huge dirs In-Reply-To: Message-ID: References: User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Rspamd-Queue-Id: 483N4D6bcGz4cM8 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=fail (rsa verify failed) header.d=puchar.net header.s=default header.b=Z0/+7egQ; dmarc=none; spf=pass (mx1.freebsd.org: domain of wojtek@puchar.net designates 194.1.144.90 as permitted sender) smtp.mailfrom=wojtek@puchar.net X-Spamd-Result: default: False [-3.94 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+mx:c]; R_DKIM_REJECT(1.00)[puchar.net:s=default]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[puchar.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[puchar.net:-]; RCVD_IN_DNSWL_NONE(0.00)[90.144.1.194.list.dnswl.org : 127.0.10.0]; IP_SCORE(-2.64)[ip: (-6.97), ipnet: 194.1.144.0/24(-3.49), asn: 43476(-2.79), country: PL(0.07)]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:43476, ipnet:194.1.144.0/24, country:PL]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2020 13:13:02 -0000 >>> >> nobody have idea what's the problem? > > Is it "just slow" or is your disk too busy? (iostat) no. disk is idle, system CPU load takes half of system (8 cores)! > > Can it be that pre-cacheing on huge directory makes disk too busy because > there is too much to read? this part works fine. The problem begins where it's already cached. To get the problem you need BOTH of: - large maxvnodes like half million - large directory (in order of 100000 files) From owner-freebsd-hackers@freebsd.org Thu Jan 23 13:27:15 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 205A31F1F91 for ; Thu, 23 Jan 2020 13:27:15 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from mailhost.m5p.com (mailhost.m5p.com [74.104.188.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "m5p.com", Issuer "Let's Encrypt Authority X3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 483NNf3LdFz4d1b for ; Thu, 23 Jan 2020 13:27:14 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from [IPv6:2001:470:1f07:15ff::26] (court.m5p.com [IPv6:2001:470:1f07:15ff:0:0:0:26]) (authenticated bits=0) by mailhost.m5p.com (8.15.2/8.15.2) with ESMTPSA id 00NDR5CH033706 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Thu, 23 Jan 2020 08:27:12 -0500 (EST) (envelope-from george+freebsd@m5p.com) Subject: Re: Odd svn error To: =?UTF-8?Q?Trond_Endrest=c3=b8l?= , George Mitchell Cc: freebsd-hackers@freebsd.org References: <740d3745-7cc8-3640-1701-2efdadf28d03@m5p.com> <8d900391-a95b-51a9-c8bb-fead7c0fc59d@m5p.com> From: George Mitchell Autocrypt: addr=george+freebsd@m5p.com; prefer-encrypt=mutual; keydata= mQINBFgnLnwBEADAJDiBKQX77LFRz9wZW8mz3KvaQol2nIremcws0F1mz/zgFlk6uhQVtwnL wb4XL5LdFwcNE1+QZzPLcbYWoWQlz0lBw1bMuKAgr0S6V2e0+I0DqhKeslVFctcTwtvT6pnK VLZXO/7ZGAaLzG4K5vSPzgoevU+YI/pxNsVCH2UO/c3jQW63uEt25mIZbCF1Pu4jgp4RhIgF ujn877r/j6OwBwjzRUu3E6ADp+U825d+5YCuQMEH0wIPnn9GTpXvfdKdbwOIl2akqXqs4cnk iATWfK3r6D4mvDEj1OPHlTvJYcfic7aOIiAwmx1C1v78GjXOdOOA0SGffNix3C2/8oZUO1+V Aet4MKpUKkduWSvULhIkHNZ5Nu8SIJOqge8pmtHxuNXAMfMrAjMdjPwwBFLsYg3Xa2E2oJwg ehTauwd/EDJFcVCyDCyCAYOi/BH/+XQyxzgDlY9N9qj9tHqhVPI6XK7t8UVffGiZUq4rHp5J RdOToqiTNC6eCJBczhMIW+DuFvWU9e6W708T1dz0Accn6Lrgk4eRIn3GFPBG+TxnpjAqHsbW 607dcnD3YKAqY4e+khczL4EObhe7dC1v2fmZiAC6Ds3WHR11IfqoUgCkIwJ590Ej+ElygJFF XxI82wtEz9hkeLLvItpyEJNVjppViRW+Dgl/U7ypHB3qDgYjgwARAQABtChHZW9yZ2UgTWl0 Y2hlbGwgPGdlb3JnZStmcmVlYnNkQG01cC5jb20+iQJUBBMBCAA+FiEENdM4ZHktsJW5kKZX wRES3m+p4fkFAlhZcR0CGyMFCQlmAYAFCwkIBwIGFQgJCgsCBBYCAwECHgECF4AACgkQwRES 3m+p4flqmw/9Emr/ydTG2n9o/IX1yVCNcHVFenVrcOY0L+DGQYZRO/XpLvsGYcuSIQId1w7h l4HZKI89ri2fF2ks6upMqBajLf8s7a8PnYrbw5bPaoOFyNTjv57GLZVsYw95kmMUpK6siuAA fXvHfKUpC/sThbwSv/1CLryVG74+5vdI8j7cQeDM436FThxlVfHKrILIiL34D4WThFB3hV/Y 2A+mQwXmdLcuQXXeAazqsFJL8sgEKSC7GMcExDkVpGc5Rh2hu97a4Sa7qWX9G/YdZOrcDacJ XxfvePn3m3WfRtXN/r0lUfiVXiqkFfbvqSaZQ0I4UvZXNGd/gH4jKHtX9RTH9G96UZeHNoMo tPw9U0fx8Ceh72nUL9qzqnmok/ryWm+6gt4Q1eRP7QAosOa1g/RgUdS1Z9IuCmbXMDp5kbNw L4ZoDMF5U3mmh8/IOKkhGopNLbNv0mwUgC59pnCptiOVx8DyckXWC4L2r6PKbWGrcGIzsUER 9smfL10gpp5H5agjwwPZI6/kzJ0R5nBzQWAlwqI73YAy6JI0HTD6lvxW7yWm2fGjEfmyaBOU 8OLUin7auoFSn+QmD5yNCUn5Ls77qHARkT1ZGocAnQkvZBGTwXpvyJixygXsm+vSUFDYBOSn cR54vdXOEMqrJk6SGau5YI9V7EhQVveE1BUp8ofWf2oo4RG5Ag0EWCcufAEQALuTOxmqMFE+ ieev/rcL3wVJrcuKS+pBbKCY9IIL0OwVf98HQJJcgdOsdDhruVd19nJNlwZ3Fc34wLw7y2GO 9WrpZiYKnI4n9urhLE5r1ydBInlI/1UKZWgM3/dPjJtcXMsC3vnqR9DmOxW4/SbqJDjP3XzO FleT4yip3AaNhPGwEPTZrubVp5hp/JojaZn690TLRwOFXg8NcjpOEs0Bq9M+OLpmsF0flrgs yDfS7y+SQ17R4Iq9T7RxZvZVAh510yGGIZIETYO/4Dh417VVm+gaksOVh2egetpUUvYYc0Ub KaP+5F/WGNrmRb1F6SKypvLlKkYAHCsUUSzsAGl9gbQhEEpuOMbUKp1979HoRMkW+8046kIo 8BZ6ph8izG/g4dZOaEqKGEhqdhYIB7UwMtFFuPtSs5Nl6JrZYni/nzFtTmtVCgcj9PNqrzqt fNFYhNznD6St6wxp3TOm9D3TQF0dzwBM1jZpb8WvmK3k6oy8hbpjiBzxn3kyRA9Vzy+PdbN/ G5a1k0rpZu/ivpBuLCDVGljUmgQigXg6xkk5UxBoHp7MPvG9prZ5jqdEa2r1KgnGjaq+VJsu Uqrw10dVgeG1NulDU1+sQl+/mwtflbkimhjDDjxsVgfrv7uvV/9be+gGm1KATuqdgCboSb1s QAo5ARfwFfChrnh+fTfPpPKHABEBAAGJAiQEGAEIAA8FAlgnLnwCGwwFCQlmAYAACgkQwRES 3m+p4fno9w/4m+swztkzxSWdutjgSv2mw+PdrKWVGFAUD2HoY1Qpi5LNLE6s9pP3qzwpQYwK viOufVJYWZ540ss6BImZBGJwyHouacqrpZjpRo5+ftj07rY1SNd8QjcHDggPfpgJ1D4Il3Xi vRg5/gzkXnRu8dXeVvMP1Ndk/F5wcoLZlQwFtPfu2xyRYIsveXMoyypAvAFSaAGXU0hRzuDJ fGI3LFvpI9UXU2C4MMzjfyZyD2NJEDKOACTo85QQzxgheTDQaDocXW00wknXFMwEItiXp8dO 2zEml/3Kj4efDfjqGpjNefjK0cnj02Byt7y6GozWXyIylrXu0SN9qWRzUVZH3+q+ijA4q3Gm 9uWzLdpjN4QWAiiaEvMhLPohp9DdLsy3kAWWrA3+pAfHSTZXrobMMbSeBkE9E4/WxdKl0nM7 TNslAWcxkTd/7Ly9cxwT8wFdHuQB1hgCmIQxDNXHL1N1ANTeUYum1w9nUg6e1M0UWu+nk3Cw qL7oL2KZe13mQnU/CFwlhbf+i//j3SXrQLlIVQv9Fn805bxIcVo9yqUZyoiV7EUpvOsxDCZh ej3mNYF5nRCf6trEJQVk0aLC26zJAYExykdUlRqc4I13XPhlt+aFSMMkoL/thYO6e9oNFK6Q aJEKXomzxxqpceJVmPH6zvqJbOboAdE/mOD0PoS1M6saIQ== Message-ID: <60b95bf6-38b7-c592-2222-048041fac7d7@m5p.com> Date: Thu, 23 Jan 2020 08:27:04 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.3.1 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="TtA5sOgNRlBWtP2OClXhA4QuH2IfIg3R8" X-Spam-Status: No, score=1.2 required=10.0 tests=HELO_MISC_IP,HELO_NO_DOMAIN autolearn=no autolearn_force=no version=3.4.3 X-Spam-Level: * X-Spam-Checker-Version: SpamAssassin 3.4.3 (2019-12-06) on mattapan.m5p.com X-Rspamd-Queue-Id: 483NNf3LdFz4d1b X-Spamd-Bar: ------- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of george@m5p.com designates 74.104.188.4 as permitted sender) smtp.mailfrom=george@m5p.com X-Spamd-Result: default: False [-7.22 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+a:c]; IP_SCORE(-2.82)[ip: (-9.87), ipnet: 74.104.0.0/16(-4.93), asn: 701(0.73), country: US(-0.05)]; TAGGED_RCPT(0.00)[freebsd]; HAS_ATTACHMENT(0.00)[]; DMARC_NA(0.00)[m5p.com]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:~]; ASN(0.00)[asn:701, ipnet:74.104.0.0/16, country:US]; TAGGED_FROM(0.00)[freebsd]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2020 13:27:15 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --TtA5sOgNRlBWtP2OClXhA4QuH2IfIg3R8 Content-Type: multipart/mixed; boundary="8hSOwVYaJB2VbFkVccXWvlNdg7K09FObP"; protected-headers="v1" From: George Mitchell To: =?UTF-8?Q?Trond_Endrest=c3=b8l?= , George Mitchell Cc: freebsd-hackers@freebsd.org Message-ID: <60b95bf6-38b7-c592-2222-048041fac7d7@m5p.com> Subject: Re: Odd svn error References: <740d3745-7cc8-3640-1701-2efdadf28d03@m5p.com> <8d900391-a95b-51a9-c8bb-fead7c0fc59d@m5p.com> In-Reply-To: --8hSOwVYaJB2VbFkVccXWvlNdg7K09FObP Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 2020-01-23 00:38, Trond Endrest=C3=B8l wrote: > On Wed, 22 Jan 2020 17:39-0500, George Mitchell wrote: >=20 >> On 2020-01-22 17:33, George Mitchell wrote: >>> [blah blah blah] >> I think I have a permissions problem; sorry for the noise. -- George >=20 > Remember to use the mount option nlockd for NFS mounted /usr/src. >Thanks! I eventually figured that out, after simply doing all my svn operations on the machine from where the disk is exported. -- George --8hSOwVYaJB2VbFkVccXWvlNdg7K09FObP-- --TtA5sOgNRlBWtP2OClXhA4QuH2IfIg3R8 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEENdM4ZHktsJW5kKZXwRES3m+p4fkFAl4pnygACgkQwRES3m+p 4flMjw//SVDPRdxSMgP/XbC4ttlLYw2OBDl0z3oHHr+67BaI3EzCNAoSwBbOUZ9A 3BbMAZuAauY/c4x8FEMtta69LOEIkK7ka72lusxjul8K0jnk5Pe6TowTesu5ni1I DnFx+gA2MVQfUuMDyp+wdHidSFT4e6i5becaS+eAeGH2Ntv+fcpDmfKdXSy4kbZi nRt0lpB/9zGugmOCoin+lN0GlTK9qRZvbhBaftgOymzS6FJa7ftG1K5RhaG7G84z FeGDongzLgHPXctecx+F/TUig+wjhR37XeWTH+4q9XkpB3o0CJpVGBZC+4scCy20 OlSJAW2d18BA/3Ztps3p3TFwPTrjVbDKVg3NRGuJ+x9FzpArDLG9QJhcmrqOLEgs H88ytuve95dc6ZUB28xvXvtBTKyVKAXCB8gOulVYb0oncZGa0CoZuOcYMiZgXIKw slweQDNwPtaLHjn9cRF/5Gsp8IHmTWRtacxSFzFi4O/9vDPCjzwZVmcm77GPaHDD mGAT2auspEi0ArToHU+FhreczcxRXaRwYOJXqLP9IBpDfQPCvM/WUqOnK3DE4q8a tKSOduZ4kJJ87eUm7BAmfik6SauWuG1/5YoTIWtTPHWkEUUtvyBzLVf3T5kbs42C lZgEIzDRQKisTMHZ+sKi8SKLwBQ0noC1jNVm4ZpRgpBnKEcTYZs= =v8/q -----END PGP SIGNATURE----- --TtA5sOgNRlBWtP2OClXhA4QuH2IfIg3R8-- From owner-freebsd-hackers@freebsd.org Thu Jan 23 13:35:16 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9DB631F2454 for ; Thu, 23 Jan 2020 13:35:16 +0000 (UTC) (envelope-from SRS0=hgVM=3M=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 483NYv4mPFz4dk2 for ; Thu, 23 Jan 2020 13:35:15 +0000 (UTC) (envelope-from SRS0=hgVM=3M=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 1979A28416; Thu, 23 Jan 2020 14:35:14 +0100 (CET) Received: from illbsd.quip.test (ip-62-24-92-232.net.upcbroadband.cz [62.24.92.232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 4F02928429; Thu, 23 Jan 2020 14:35:13 +0100 (CET) Subject: Re: slow directory operation on huge dirs To: Wojciech Puchar Cc: freebsd-hackers@freebsd.org References: From: Miroslav Lachman <000.fbsd@quip.cz> Message-ID: <9b426949-be87-106a-46c3-f1b6a2e5bb83@quip.cz> Date: Thu, 23 Jan 2020 14:35:13 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Firefox/52.0 SeaMonkey/2.49.3 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 483NYv4mPFz4dk2 X-Spamd-Bar: ++++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of SRS0=hgVM=3M=quip.cz=000.fbsd@elsa.codelab.cz has no SPF policy when checking 94.124.105.4) smtp.mailfrom=SRS0=hgVM=3M=quip.cz=000.fbsd@elsa.codelab.cz X-Spamd-Result: default: False [4.06 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; IP_SCORE(0.86)[ip: (0.35), ipnet: 94.124.104.0/21(0.17), asn: 42000(3.70), country: CZ(0.09)]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[quip.cz]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(1.00)[0.996,0]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[4.105.124.94.list.dnswl.org : 127.0.10.0]; NEURAL_SPAM_LONG(1.00)[1.000,0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[000.fbsd@quip.cz,SRS0=hgVM=3M=quip.cz=000.fbsd@elsa.codelab.cz]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:42000, ipnet:94.124.104.0/21, country:CZ]; FROM_NEQ_ENVFROM(0.00)[000.fbsd@quip.cz,SRS0=hgVM=3M=quip.cz=000.fbsd@elsa.codelab.cz]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2020 13:35:16 -0000 Wojciech Puchar wrote on 2020/01/23 14:12: >>>> >>> nobody have idea what's the problem? >> >> Is it "just slow" or is your disk too busy? (iostat) > > no. disk is idle, system CPU load takes half of system (8 cores)! > >> >> Can it be that pre-cacheing on huge directory makes disk too busy >> because there is too much to read? > this part works fine. The problem begins where it's already cached. > > To get the problem you need BOTH of: > > - large maxvnodes like half million > - large directory (in order of 100000 files) I remember very similar problem with slow access of directories with 80 000+ files. It was always slow like frozen hell. There is some recommendation to avoid more than 10 000 of file entries in a directory. That is why filesystem caches or sessions in PHP must create more levels of subdirectories. I know you cannot control users of maildirs. I am not sure if this situation can be improved or not. Miroslav Lachman From owner-freebsd-hackers@freebsd.org Thu Jan 23 13:40:23 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id F1CA81F26FD for ; Thu, 23 Jan 2020 13:40:23 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: from puchar.net (puchar.net [194.1.144.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 483Ngq1r16z4f09 for ; Thu, 23 Jan 2020 13:40:22 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: Received: from 127.0.0.1 (localhost [127.0.0.1]) by puchar.net (8.15.2/8.15.2) with ESMTPS id 00NDeHZ9083528 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 23 Jan 2020 14:40:18 +0100 (CET) (envelope-from puchar-wojtek@puchar.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=puchar.net; s=default; t=1579786818; bh=FsFeT3rC7NufXegyyDQ3B9zSdYMGULMDBKECo9V6qxU=; h=Date:From:To:cc:Subject:In-Reply-To:References; b=qcnUnbqgseqViE2p1nzggKsMMo0KdqagHsd6HycC42cGrro3ZIo4qGuKAv/gqum9H ZPimJVQKLN20bWARx/IQA5C2YJExJJGdL1Fdu5zPCxGY28Vz7v/mbt5RWckWHBOx0U hyYhHUb967Z2h/j06g/wfniIgtnxgn3LTIjqF9fE= Received: from localhost (puchar-wojtek@localhost) by puchar.net (8.15.2/8.15.2/Submit) with ESMTP id 00NDeHl3083525; Thu, 23 Jan 2020 14:40:17 +0100 (CET) (envelope-from puchar-wojtek@puchar.net) Date: Thu, 23 Jan 2020 14:40:17 +0100 (CET) From: Wojciech Puchar To: Miroslav Lachman <000.fbsd@quip.cz> cc: Wojciech Puchar , freebsd-hackers@freebsd.org Subject: Re: slow directory operation on huge dirs In-Reply-To: <9b426949-be87-106a-46c3-f1b6a2e5bb83@quip.cz> Message-ID: References: <9b426949-be87-106a-46c3-f1b6a2e5bb83@quip.cz> User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Rspamd-Queue-Id: 483Ngq1r16z4f09 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=fail (rsa verify failed) header.d=puchar.net header.s=default header.b=qcnUnbqg; dmarc=none; spf=pass (mx1.freebsd.org: domain of wojtek@puchar.net designates 194.1.144.90 as permitted sender) smtp.mailfrom=wojtek@puchar.net X-Spamd-Result: default: False [-3.93 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+mx:c]; R_DKIM_REJECT(1.00)[puchar.net:s=default]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[puchar.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[puchar.net:-]; RCVD_IN_DNSWL_NONE(0.00)[90.144.1.194.list.dnswl.org : 127.0.10.0]; IP_SCORE(-2.63)[ip: (-6.95), ipnet: 194.1.144.0/24(-3.47), asn: 43476(-2.78), country: PL(0.07)]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:43476, ipnet:194.1.144.0/24, country:PL]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2020 13:40:24 -0000 >> this part works fine. The problem begins where it's already cached. >> >> To get the problem you need BOTH of: >> >> - large maxvnodes like half million >> - large directory (in order of 100000 files) > > I remember very similar problem with slow access of directories with 80 000+ > files. It was always slow like frozen hell. There is some recommendation to first - we have DIRHASH option and i have it enabled. second - it is not just a problem of huge directory but both huge directory and large maxvnodes settings. Large directories works fast when kern.maxvnodes is small and there is enough dirhash_maxmem. it's clearly an error somewhere. From owner-freebsd-hackers@freebsd.org Thu Jan 23 16:41:52 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A25F51F6E20 for ; Thu, 23 Jan 2020 16:41:52 +0000 (UTC) (envelope-from mike@sentex.net) Received: from pyroxene2a.sentex.ca (pyroxene19.sentex.ca [IPv6:2607:f3e0:0:3::19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "pyroxene.sentex.ca", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 483SjC2b9Pz3M9D for ; Thu, 23 Jan 2020 16:41:51 +0000 (UTC) (envelope-from mike@sentex.net) Received: from [192.168.43.26] (saphire3.sentex.net [192.168.43.26]) by pyroxene2a.sentex.ca (8.15.2/8.15.2) with ESMTP id 00NGfn8o011787; Thu, 23 Jan 2020 11:41:49 -0500 (EST) (envelope-from mike@sentex.net) Subject: Re: slow directory operation on huge dirs To: Wojciech Puchar , Miroslav Lachman <000.fbsd@quip.cz> Cc: freebsd-hackers@freebsd.org References: <9b426949-be87-106a-46c3-f1b6a2e5bb83@quip.cz> From: Mike Tancsa Autocrypt: addr=mike@sentex.net; prefer-encrypt=mutual; keydata= xsBNBEzcA24BCACpwI/iqOrs0GfQSfhA1v6Z8AcXVeGsRyKEKUpxoOYxXWc2z3vndbYlIP6E YJeifzKhS/9E+VjhhICaepLHfw865TDTUPr5D0Ed+edSsKjlnDtb6hfNJC00P7eoiuvi85TW F/gAxRY269A5d856bYrzLbkWp2lKUR3Bg6NnORtflGzx9ZWAltZbjYjjRqegPv0EQNYcHqWo eRpXilEo1ahT6nmOU8V7yEvT2j4wlLcQ6qg7w+N/vcBvyd/weiwHU+vTQ9mT61x5/wUrQhdw 2gJHeQXeDGMJV49RT2EEz+QVxaf477eyWsdQzPVjAKRMT3BVdK8WvpYAEfBAbXmkboOxABEB AAHNHG1pa2UgdGFuY3NhIDxtaWtlQHNlbnRleC5jYT7CwHgEEwECACIFAkzcA24CGwMGCwkI BwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEJXHwM2kc8rX+sMH/2V6pTBKsQ5mpWWLgs6wVP2k BC+6r/YKNXv9Rw/PrC6+9hTbgA+sSjJ+8gxsCbJsOQXZrxF0x3l9oYdYfuKcwdwXFX1/FS8p HfBeDkmlH+dI709xT9wgrR4dS5aMmKp0scPrXPIAKiYVOHjOlNItcLYTEEWEFBepheEVsgmk GrNbcrHwOx/u4igUQ8vcpyXPyUki+BsftPw8ZQvBU887igh0OxaCR8AurJppQ5UQd63r81cX E1ZjoFoWCaGK/SjPb/OhpYpu5swoZIhOxQbn7OtakYPsDd5t2A5KhvjI8BMTnd5Go+2xsCmr jlIEq8Bi29gCcfQUvNiClevi13ifmnnOwE0ETNwDbgEIALWGNJHRAhpd0A4vtd3G0oRqMBcM FGThQr3qORmEBTPPEomTdBaHcn+Xl+3YUvTBD/67/mutWBwgp2R5gQOSqcM7axvgMSHbKqBL 9sd1LsLw0UT2O5AYxv3EwzhG84pwRg3XcUqvWA4lA8tIj/1q4Jzi5qOkg1zxq4W9qr9oiYK5 bBR638JUvr3eHMaz/Nz+sDVFgwHmXZj3M6aE5Ce9reCGbvrae7H5D5PPvtT3r22X8SqfVAiO TFKedCf/6jbSOedPN931FJQYopj9P6b3m0nI3ZiCDVSqeyOAIBLzm+RBUIU3brzoxDhYR8pz CJc2sK8l6YjqivPakrD86bFDff8AEQEAAcLAXwQYAQIACQUCTNwDbgIbDAAKCRCVx8DNpHPK 1+iQB/99aqNtez9ZTBWELj269La8ntuRx6gCpzfPXfn6SDIfTItDxTh1hrdRVP5QNGGF5wus N4EMwXouskva1hbFX3Pv72csYSxxEJXjW16oV8WK4KjKXoskLg2RyRP4uXqL7Mp2ezNtVY5F 9nu3fj4ydpHCSaqKy5xd70A8D50PfZsFgkrsa5gdQhPiGGEdxhq/XSeAAnZ4uVLJKarH+mj5 MEhgZPEBWkGrbDZpezl9qbFcUem/uT9x8FYT/JIztMVh9qDcdP5tzANW5J7nvgXjska+VFGY ryZK4SPDczh74mn6GI/+RBi7OUzXXPgpPBrhS5FByjwCqjjsSpTjTds+NGIY Organization: Sentex Communications Message-ID: <7685f996-c1b8-e11b-aaf6-b59c001ce226@sentex.net> Date: Thu, 23 Jan 2020 16:41:47 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.4.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 483SjC2b9Pz3M9D X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of mike@sentex.net designates 2607:f3e0:0:3::19 as permitted sender) smtp.mailfrom=mike@sentex.net X-Spamd-Result: default: False [-2.71 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ptr]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; HFILTER_HELO_IP_A(1.00)[pyroxene2a.sentex.ca]; TO_DN_SOME(0.00)[]; DMARC_NA(0.00)[sentex.net]; HAS_ORG_HEADER(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; HFILTER_HELO_NORES_A_OR_MX(0.30)[pyroxene2a.sentex.ca]; IP_SCORE(-1.71)[ipnet: 2607:f3e0::/32(-4.93), asn: 11647(-3.54), country: CA(-0.09)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11647, ipnet:2607:f3e0::/32, country:CA]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2020 16:41:52 -0000 On 1/23/2020 1:40 PM, Wojciech Puchar wrote: > > first - we have DIRHASH option and i have it enabled. > second - it is not just a problem of huge directory but both huge > directory and large maxvnodes settings. > > Large directories works fast when kern.maxvnodes is small and there is > enough dirhash_maxmem. > > it's clearly an error somewhere. Dont know about the vnodes adjustment, and its probably not related. But on our largish mail servers, we need to change the defaults in nsswitch as ls takes forever on large directories otherwise. --- /usr/src/etc/nsswitch.conf  2019-07-10 14:46:40.871619000 -0400 +++ /etc/nsswitch.conf  2016-01-27 14:21:30.400226032 -0500 @@ -1,13 +1,12 @@  #  # nsswitch.conf(5) - name service switch configuration file -# $FreeBSD: stable/11/etc/nsswitch.conf 301711 2016-06-09 01:28:44Z markj $ +# $FreeBSD: src/etc/nsswitch.conf,v 1.1.10.1 2009/08/03 08:13:06 kensmith Exp $  # -group: compat +group: files  group_compat: nis  hosts: files dns -netgroup: compat  networks: files -passwd: compat +passwd: files  passwd_compat: nis  shells: files  services: compat     ---Mike From owner-freebsd-hackers@freebsd.org Thu Jan 23 18:42:48 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C89DA1FA305 for ; Thu, 23 Jan 2020 18:42:48 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from smtp-out-so.shaw.ca (smtp-out-so.shaw.ca [64.59.136.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 483WNl5Sxzz3yRr for ; Thu, 23 Jan 2020 18:42:47 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from spqr.komquats.com ([70.67.125.17]) by shaw.ca with ESMTPA id uhRBiXnQk17ZDuhRDinaQJ; Thu, 23 Jan 2020 11:42:45 -0700 X-Authority-Analysis: v=2.3 cv=ZsqT1OzG c=1 sm=1 tr=0 a=VFtTW3WuZNDh6VkGe7fA3g==:117 a=VFtTW3WuZNDh6VkGe7fA3g==:17 a=jpOVt7BSZ2e4Z31A5e1TngXxSK0=:19 a=IkcTkHD0fZMA:10 a=Jdjhy38mL1oA:10 a=_8di5KHbAAAA:8 a=6I5d2MoRAAAA:8 a=YxBL1-UpAAAA:8 a=lI3h0bHuk6_Q36RTsWsA:9 a=QEXdDO2ut3YA:10 a=z0NWOAd74rtbdv5dRerE:22 a=IjZwj45LgO3ly-622nXo:22 a=Ia-lj3WSrqcvXOmTRaiG:22 Received: from Resas-iPad.esitwifi.local (S0106788a207e2972.gv.shawcable.net [70.66.154.233]) by spqr.komquats.com (Postfix) with ESMTPSA id 81C2D330; Thu, 23 Jan 2020 10:42:40 -0800 (PST) Date: Thu, 23 Jan 2020 10:22:54 -0800 User-Agent: K-9 Mail for Android In-Reply-To: <7685f996-c1b8-e11b-aaf6-b59c001ce226@sentex.net> References: <9b426949-be87-106a-46c3-f1b6a2e5bb83@quip.cz> <7685f996-c1b8-e11b-aaf6-b59c001ce226@sentex.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Re: slow directory operation on huge dirs To: freebsd-hackers@freebsd.org, Mike Tancsa , Wojciech Puchar , Miroslav Lachman <000.fbsd@quip.cz> From: Cy Schubert Message-ID: X-CMAE-Envelope: MS4wfIT6bhOYo8HbK5GDjdGc7KgSMFAHiF1cFf5nMTW6J3ivIJF3GjOJCl0MsF8x3mBvQbS0Jy4l0polbjnisLAURI2jcF/MZNWQ6taqGKk4NSTWoKVHS7NF pqQEUrKcC5JSKadlfD8UarhoWPA9JS2whJswvRgoc53LkeybWrneLU6iuKBUopyCv3TOFBo/WUZdgXRVsDHbQhgOjCTU9vJ3wEZ6RTyzw5Ht+f0elPoD1nKf n9I+2klPd9OBZQvB6qM9fOGPyzuiumFNdpteH5t0k9s= X-Rspamd-Queue-Id: 483WNl5Sxzz3yRr X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=none; spf=none (mx1.freebsd.org: domain of cy.schubert@cschubert.com has no SPF policy when checking 64.59.136.137) smtp.mailfrom=cy.schubert@cschubert.com X-Spamd-Result: default: False [-4.58 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[137.136.59.64.rep.mailspike.net : 127.0.0.17]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[17.125.67.70.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.11,233.154.66.70.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.11]; R_SPF_NA(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[137.136.59.64.list.dnswl.org : 127.0.5.1]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:6327, ipnet:64.59.128.0/20, country:CA]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(-2.38)[ip: (-6.18), ipnet: 64.59.128.0/20(-3.17), asn: 6327(-2.47), country: CA(-0.09)]; FROM_EQ_ENVFROM(0.00)[] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2020 18:42:48 -0000 On January 23, 2020 8:41:47 AM PST, Mike Tancsa wrote: >On 1/23/2020 1:40 PM, Wojciech Puchar wrote: >> >> first - we have DIRHASH option and i have it enabled=2E >> second - it is not just a problem of huge directory but both huge >> directory and large maxvnodes settings=2E >> >> Large directories works fast when kern=2Emaxvnodes is small and there >is >> enough dirhash_maxmem=2E >> >> it's clearly an error somewhere=2E=20 > > >Dont know about the vnodes adjustment, and its probably not related=2E >But >on our largish mail servers, we need to change the defaults in nsswitch >as ls takes forever on large directories otherwise=2E > >--- /usr/src/etc/nsswitch=2Econf=C2=A0 2019-07-10 14:46:40=2E871619000 -0= 400 >+++ /etc/nsswitch=2Econf=C2=A0 2016-01-27 14:21:30=2E400226032 -0500 >@@ -1,13 +1,12 @@ >=C2=A0# >=C2=A0# nsswitch=2Econf(5) - name service switch configuration file >-# $FreeBSD: stable/11/etc/nsswitch=2Econf 301711 2016-06-09 01:28:44Z >markj $ >+# $FreeBSD: src/etc/nsswitch=2Econf,v 1=2E1=2E10=2E1 2009/08/03 08:13:06 >kensmith Exp $ >=C2=A0# >-group: compat >+group: files >=C2=A0group_compat: nis >=C2=A0hosts: files dns >-netgroup: compat >=C2=A0networks: files >-passwd: compat >+passwd: files >=C2=A0passwd_compat: nis >=C2=A0shells: files >=C2=A0services: compat > > >=C2=A0=C2=A0=C2=A0 ---Mike > >_______________________________________________ >freebsd-hackers@freebsd=2Eorg mailing list >https://lists=2Efreebsd=2Eorg/mailman/listinfo/freebsd-hackers >To unsubscribe, send any mail to >"freebsd-hackers-unsubscribe@freebsd=2Eorg" Do you use a directory service such as ldap, nis, or a/d? --=20 Pardon the typos and autocorrect, small keyboard in use=2E=20 Cy Schubert FreeBSD UNIX: Web: https://www=2EFreeBSD=2Eorg The need of the many outweighs the greed of the few=2E Sent from my Android device with K-9 Mail=2E Please excuse my brevity=2E From owner-freebsd-hackers@freebsd.org Thu Jan 23 18:46:17 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 806431FA4BA for ; Thu, 23 Jan 2020 18:46:17 +0000 (UTC) (envelope-from mike@sentex.net) Received: from pyroxene2a.sentex.ca (pyroxene19.sentex.ca [IPv6:2607:f3e0:0:3::19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "pyroxene.sentex.ca", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 483WSm4rj2z3ycl for ; Thu, 23 Jan 2020 18:46:16 +0000 (UTC) (envelope-from mike@sentex.net) Received: from [IPv6:2607:f3e0:0:4:885c:d25e:5e81:ec80] ([IPv6:2607:f3e0:0:4:885c:d25e:5e81:ec80]) by pyroxene2a.sentex.ca (8.15.2/8.15.2) with ESMTPS id 00NIkCYS021331 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO); Thu, 23 Jan 2020 13:46:13 -0500 (EST) (envelope-from mike@sentex.net) Subject: Re: slow directory operation on huge dirs To: Cy Schubert , freebsd-hackers@freebsd.org, Wojciech Puchar , Miroslav Lachman <000.fbsd@quip.cz> References: <9b426949-be87-106a-46c3-f1b6a2e5bb83@quip.cz> <7685f996-c1b8-e11b-aaf6-b59c001ce226@sentex.net> From: mike tancsa Autocrypt: addr=mike@sentex.net; keydata= mQENBFywzOMBCACoNFpwi5MeyEREiCeHtbm6pZJI/HnO+wXdCAWtZkS49weOoVyUj5BEXRZP xflV2ib2hflX4nXqhenaNiia4iaZ9ft3I1ebd7GEbGnsWCvAnob5MvDZyStDAuRxPJK1ya/s +6rOvr+eQiXYNVvfBhrCfrtR/esSkitBGxhUkBjOti8QwzD71JVF5YaOjBAs7jZUKyLGj0kW yDg4jUndudWU7G2yc9GwpHJ9aRSUN8e/mWdIogK0v+QBHfv/dsI6zVB7YuxCC9Fx8WPwfhDH VZC4kdYCQWKXrm7yb4TiVdBh5kgvlO9q3js1yYdfR1x8mjK2bH2RSv4bV3zkNmsDCIxjABEB AAG0HW1pa2UgdGFuY3NhIDxtaWtlQHNlbnRleC5uZXQ+iQFUBBMBCAA+FiEEmuvCXT0aY6hs 4SbWeVOEFl5WrMgFAlywzOYCGwMFCQHhM4AFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQ eVOEFl5WrMhnPAf7Bf+ola0V9t4i8rwCMGvzkssGaxY/5zNSZO9BgSgfN0WzgmBEOy/3R4km Yn5KH94NltJYAAE5hqkFmAwK6psOqAR9cxHrRfU+gV2KO8pCDc6K/htkQcd/mclJYpCHp6Eq EVJOiAxcNaYuHZkeMdXDuvvI5Rk82VHk84BGgxIqIrhLlkguoPbXOOa+8c/Mpb1sRAGZEOuX EzKNC49+GS9gKW6ISbanyPsGEcFyP7GKMzcHBPf3cPrewZQZ6gBoNscasL6IJeAQDqzQAxbU GjO0qBSMRgnLXK7+DJlxrYdHGXqNbV6AYsmHJ6c2WWWiuRviFBqXinlgJ2FnYebZPAfWibkB DQRcsMzkAQgA1Dpo/xWS66MaOJLwA28sKNMwkEk1Yjs+okOXDOu1F+0qvgE8sVmrOOPvvWr4 axtKRSG1t2QUiZ/ZkW/x/+t0nrM39EANV1VncuQZ1ceIiwTJFqGZQ8kb0+BNkwuNVFHRgXm1 qzAJweEtRdsCMohB+H7BL5LGCVG5JaU0lqFU9pFP40HxEbyzxjsZgSE8LwkI6wcu0BLv6K6c Lm0EiHPOl5G8kgRi38PS7/6s3R8QDsEtbGsYy6O82k3zSLIjuDBwA9GRaeigGppTxzAHVjf5 o9KKu4O7gC2KKVHPegbXS+GK7DU0fjzX57H5bZ6komE5eY4p3oWT/CwVPSGfPs8jOwARAQAB iQE8BBgBCAAmFiEEmuvCXT0aY6hs4SbWeVOEFl5WrMgFAlywzOQCGwwFCQHhM4AACgkQeVOE Fl5WrMhmjQf/dBCjAVn1J0GzSsHiLvSAQz1cchbdy8LD0Tnpzjgp5KLU7sNojbI8vqt4yKAi cayI88j8+xxNXPMWM4pHELuUuVHS5XTpHa/wwulUtI5w/zyKlUDsIvqTPZLUEwH7DfNBueVM WyNaIjV2kxSmM8rNMC+RkgyfbjGLCkmWsMRVuLIUYpl5D9WHmenUbiErlKU2KvEEXEg/aLKq 3m/AdM9RAYsP9O4l+sAZEfyYoNJzDhTZMzn/9Q0uFPLK9smDQh4WBTFaApveVJPHRKmHPoNF Xxj+yScYdQ4SKH34WnhNSELvnZQ3ulH5tpASmm0w+GxfZqSc8+QCwoKtBRDUxoE56A== Message-ID: Date: Thu, 23 Jan 2020 13:46:13 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.4.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 483WSm4rj2z3ycl X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of mike@sentex.net designates 2607:f3e0:0:3::19 as permitted sender) smtp.mailfrom=mike@sentex.net X-Spamd-Result: default: False [-2.71 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ptr]; NEURAL_HAM_LONG(-1.00)[-0.997,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[sentex.net]; HFILTER_HELO_IP_A(1.00)[pyroxene2a.sentex.ca]; TO_DN_SOME(0.00)[]; HFILTER_HELO_NORES_A_OR_MX(0.30)[pyroxene2a.sentex.ca]; TO_MATCH_ENVRCPT_SOME(0.00)[]; IP_SCORE(-1.71)[ipnet: 2607:f3e0::/32(-4.93), asn: 11647(-3.54), country: CA(-0.09)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11647, ipnet:2607:f3e0::/32, country:CA]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2020 18:46:17 -0000 On 1/23/2020 1:22 PM, Cy Schubert wrote: > >> Dont know about the vnodes adjustment, and its probably not related. >> But >> on our largish mail servers, we need to change the defaults in nsswitch >> as ls takes forever on large directories otherwise. >> >> --- /usr/src/etc/nsswitch.conf  2019-07-10 14:46:40.871619000 -0400 >> +++ /etc/nsswitch.conf  2016-01-27 14:21:30.400226032 -0500 >> @@ -1,13 +1,12 @@ >>  # >>  # nsswitch.conf(5) - name service switch configuration file >> -# $FreeBSD: stable/11/etc/nsswitch.conf 301711 2016-06-09 01:28:44Z >> markj $ >> +# $FreeBSD: src/etc/nsswitch.conf,v 1.1.10.1 2009/08/03 08:13:06 >> kensmith Exp $ >>  # >> -group: compat >> +group: files >>  group_compat: nis >>  hosts: files dns >> -netgroup: compat >>  networks: files >> -passwd: compat >> +passwd: files >>  passwd_compat: nis >>  shells: files >>  services: compat >> >> >> Do you use a directory service such as ldap, nis, or a/d? No. Just a lot of users in the master.passwd file.     ---Mike From owner-freebsd-hackers@freebsd.org Thu Jan 23 19:13:15 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4DF4D1FB016 for ; Thu, 23 Jan 2020 19:13:15 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: from puchar.net (puchar.net [194.1.144.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 483X3t146Nz416C for ; Thu, 23 Jan 2020 19:13:13 +0000 (UTC) (envelope-from wojtek@puchar.net) Received: Received: from 127.0.0.1 (localhost [127.0.0.1]) by puchar.net (8.15.2/8.15.2) with ESMTPS id 00NJD87A018524 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 23 Jan 2020 20:13:08 +0100 (CET) (envelope-from puchar-wojtek@puchar.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=puchar.net; s=default; t=1579806788; bh=bMzqc8h5SA99w9Ysd0+QrvOqZJysTjB8ddF7RPfGi+U=; h=Date:From:To:cc:Subject:In-Reply-To:References; b=dIfUuVeKsJ/mhYm3zknAlAzKZIhHdABnS71lz5+faadesKa5PQpwoWxF+ccm+i8jQ 1GWUWeK6fV27n452C1g5vRrXlWyJGUN4UgPj0EjKZnvttYC+lnGJEpEvKxm6PCeoOp PYyvgvLHdS2JtmQp2ACfP4O6Yi9e9MCJfGffET7w= Received: from localhost (puchar-wojtek@localhost) by puchar.net (8.15.2/8.15.2/Submit) with ESMTP id 00NJD7op018521; Thu, 23 Jan 2020 20:13:07 +0100 (CET) (envelope-from puchar-wojtek@puchar.net) Date: Thu, 23 Jan 2020 20:13:07 +0100 (CET) From: Wojciech Puchar To: Mike Tancsa cc: Wojciech Puchar , Miroslav Lachman <000.fbsd@quip.cz>, freebsd-hackers@freebsd.org Subject: Re: slow directory operation on huge dirs In-Reply-To: <7685f996-c1b8-e11b-aaf6-b59c001ce226@sentex.net> Message-ID: References: <9b426949-be87-106a-46c3-f1b6a2e5bb83@quip.cz> <7685f996-c1b8-e11b-aaf6-b59c001ce226@sentex.net> User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Rspamd-Queue-Id: 483X3t146Nz416C X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=fail (rsa verify failed) header.d=puchar.net header.s=default header.b=dIfUuVeK; dmarc=none; spf=pass (mx1.freebsd.org: domain of wojtek@puchar.net designates 194.1.144.90 as permitted sender) smtp.mailfrom=wojtek@puchar.net X-Spamd-Result: default: False [-3.92 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+mx]; R_DKIM_REJECT(1.00)[puchar.net:s=default]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[puchar.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[puchar.net:-]; RCVD_IN_DNSWL_NONE(0.00)[90.144.1.194.list.dnswl.org : 127.0.10.0]; IP_SCORE(-2.62)[ip: (-6.92), ipnet: 194.1.144.0/24(-3.46), asn: 43476(-2.77), country: PL(0.07)]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:43476, ipnet:194.1.144.0/24, country:PL]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2020 19:13:15 -0000 >> enough dirhash_maxmem. >> >> it's clearly an error somewhere. > > > Dont know about the vnodes adjustment, and its probably not related. But yes it is. of course it should not. From owner-freebsd-hackers@freebsd.org Fri Jan 24 14:48:39 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 81E931F203F for ; Fri, 24 Jan 2020 14:48:39 +0000 (UTC) (envelope-from steffen@sdaoden.eu) Received: from sdaoden.eu (sdaoden.eu [217.144.132.164]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4842864WkFz47CT for ; Fri, 24 Jan 2020 14:48:38 +0000 (UTC) (envelope-from steffen@sdaoden.eu) Received: by sdaoden.eu (Postfix, from userid 1000) id D872C16054; Fri, 24 Jan 2020 15:48:30 +0100 (CET) Date: Fri, 24 Jan 2020 15:48:29 +0100 From: Steffen Nurpmeso To: freebsd-hackers@freebsd.org Subject: Long (-- style) options (.Fl) with mdoc(7) Message-ID: <20200124144829.49360%steffen@sdaoden.eu> Mail-Followup-To: freebsd-hackers@freebsd.org User-Agent: s-nail v14.9.16-79-g49f65d86-dirty OpenPGP: id=EE19E1C1F2F7054F8D3954D8308964B51883A0DD; url=https://ftp.sdaoden.eu/steffen.asc; preference=signencrypt BlahBlahBlah: Any stupid boy can crush a beetle. But all the professors in the world can make no bugs. X-Rspamd-Queue-Id: 4842864WkFz47CT X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of steffen@sdaoden.eu designates 217.144.132.164 as permitted sender) smtp.mailfrom=steffen@sdaoden.eu X-Spamd-Result: default: False [-0.73 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.83)[-0.834,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+a]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[sdaoden.eu]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-0.91)[-0.905,0]; MID_CONTAINS_FROM(1.00)[]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15987, ipnet:217.144.132.0/24, country:DE]; IP_SCORE(0.31)[asn: 15987(1.58), country: DE(-0.02)] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Jan 2020 14:48:39 -0000 Hello. The very moment i have seen a showmount(8) change fly by, and wanted to make you aware of the persuading approach that NetBSD's wizd(8) uses, namely ".Fl Fl long-option". Different to ".Fl -long-option" this produces visually appealing results on all output devices. Ciao, --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) From owner-freebsd-hackers@freebsd.org Fri Jan 24 17:58:26 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AC04F1F6ED7 for ; Fri, 24 Jan 2020 17:58:26 +0000 (UTC) (envelope-from yp@xvoid.org) Received: from new4-smtp.messagingengine.com (new4-smtp.messagingengine.com [66.111.4.230]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4846M54HPmz4L9s for ; Fri, 24 Jan 2020 17:58:25 +0000 (UTC) (envelope-from yp@xvoid.org) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailnew.nyi.internal (Postfix) with ESMTP id 017B86764 for ; Fri, 24 Jan 2020 12:58:25 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Fri, 24 Jan 2020 12:58:25 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xvoid.org; h= subject:to:references:from:message-id:date:mime-version :in-reply-to:content-type:content-transfer-encoding; s=fm1; bh=P tlRJdEoyBGpMU+0vwgePsECG4ihzqT/Efcv2QwhSs4=; b=CfzQFKxKtqt6opTx7 UAuPOA0MSPWwLIZcN+OPVwxw+qiZjSiUw56VNkeLuJlLPLd5sKKSKmrqd5lpUBBq tjulBV9xnmOOVRGtb6kksP6OWXDIedbBSc5oxy0vYkuo//VpIc7KznWH2YSf01ED UjNsuEXePFe6BsT+b2FFy+nFInnP20zK66HUiXN6v1I++dqzc9DwALNq3ltkkJSs DpI9Sawm/FRi6LGBYy/bYTwjTzA9cs+6EWEUahE+vfOs8K6lb2DxYRWg7gDuqdQc HiPzhcXIlFsCUWUIGeE9m23pMOfjSWXTgsYkndpqxVON5AqL1mUwa1sFAjgUk4EF TVhUg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=PtlRJdEoyBGpMU+0vwgePsECG4ihzqT/Efcv2QwhS s4=; b=lyxejCwkwkPqu+U8UfY5BOwjzBe9WDZi+CMhCzG9p6xTfzwMQQLR3+JB0 CekvVBiJK+5+xHWnVgrHjtiU7Hu/q8uvlYxb8KIwGMl56EuHfkhddNkpQJ+oPnS4 wiDj79GwGiqFsNW6Y/jb3rDJPeJVfiZnoIii08RxdBG7LNLTQrSNP+gWFon8VyUg Grgn4GK0h8hUSsH8rrZji5DapLnT97xAMzMJvvPU6Nt3JRM5JYSyoHHB51YhKUVA GK4V0/aONM2FlCxXJir2xEJDrNubRj4n2sJ4+7LXU5/862nk40WUWwQksndNJ2tP v2lTMSivRYbb1lOZGdsOfxPpLNfAA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedugedrvdeggddutdeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefuvfhfhffkffgfgggjtgfgsehtje ertddtfeejnecuhfhrohhmpegjuhhrihcurfgrnhhkohhvuceohihpseigvhhoihgurdho rhhgqeenucffohhmrghinhepsghsugdrlhhvnecukfhppedujeekrdduheehrdegrdekle enucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpeihphes gihvohhiugdrohhrgh X-ME-Proxy: Received: from earth.xvoid.org (unknown [178.155.4.89]) by mail.messagingengine.com (Postfix) with ESMTPA id E09FA3061173 for ; Fri, 24 Jan 2020 12:58:23 -0500 (EST) Subject: Re: Long (-- style) options (.Fl) with mdoc(7) To: freebsd-hackers@freebsd.org References: <20200124144829.49360%steffen@sdaoden.eu> From: Yuri Pankov Message-ID: Date: Fri, 24 Jan 2020 20:58:22 +0300 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:68.0) Gecko/20100101 Thunderbird/68.4.1 MIME-Version: 1.0 In-Reply-To: <20200124144829.49360%steffen@sdaoden.eu> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4846M54HPmz4L9s X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=xvoid.org header.s=fm1 header.b=CfzQFKxK; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=lyxejCwk; dmarc=none; spf=pass (mx1.freebsd.org: domain of yp@xvoid.org designates 66.111.4.230 as permitted sender) smtp.mailfrom=yp@xvoid.org X-Spamd-Result: default: False [0.97 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(0.00)[xvoid.org:s=fm1,messagingengine.com:s=fm1]; NEURAL_HAM_LONG(-0.14)[-0.143,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(0.00)[+ip4:66.111.4.230]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_DN_NONE(0.00)[]; RECEIVED_SPAMHAUS_XBL(5.00)[89.4.155.178.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.4]; RCPT_COUNT_ONE(0.00)[1]; BAD_REP_POLICIES(0.10)[]; RCVD_COUNT_THREE(0.00)[4]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[xvoid.org:+,messagingengine.com:+]; DMARC_NA(0.00)[xvoid.org]; NEURAL_HAM_MEDIUM(-0.29)[-0.290,0]; IP_SCORE(-3.49)[ip: (-9.83), ipnet: 66.111.4.0/24(-4.90), asn: 11403(-2.68), country: US(-0.05)]; RCVD_IN_DNSWL_LOW(-0.10)[230.4.111.66.list.dnswl.org : 127.0.5.1]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[89.4.155.178.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.11] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Jan 2020 17:58:26 -0000 Steffen Nurpmeso wrote: > Hello. > > The very moment i have seen a showmount(8) change fly by, and > wanted to make you aware of the persuading approach that NetBSD's > wizd(8) uses, namely ".Fl Fl long-option". Different to ".Fl > -long-option" this produces visually appealing results on all > output devices. mandoc style guide advises differently: https://mandoc.bsd.lv/mdoc/style/options.html It's not authoritative, of course, but I'm not aware of any other mdoc style guide. May be you could take it up to style guide/mandoc authors? From owner-freebsd-hackers@freebsd.org Fri Jan 24 18:31:35 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8CC5C1F7ED9 for ; Fri, 24 Jan 2020 18:31:35 +0000 (UTC) (envelope-from steffen@sdaoden.eu) Received: from sdaoden.eu (sdaoden.eu [217.144.132.164]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48475L1rw2z4NBK for ; Fri, 24 Jan 2020 18:31:34 +0000 (UTC) (envelope-from steffen@sdaoden.eu) Received: by sdaoden.eu (Postfix, from userid 1000) id 82B7816054; Fri, 24 Jan 2020 19:31:32 +0100 (CET) Date: Fri, 24 Jan 2020 19:31:32 +0100 From: Steffen Nurpmeso To: Yuri Pankov Cc: freebsd-hackers@freebsd.org, Ingo Schwarze Subject: Re: Long (-- style) options (.Fl) with mdoc(7) Message-ID: <20200124183132.-nDI2%steffen@sdaoden.eu> In-Reply-To: References: <20200124144829.49360%steffen@sdaoden.eu> Mail-Followup-To: Yuri Pankov , freebsd-hackers@freebsd.org, Ingo Schwarze User-Agent: s-nail v14.9.16-79-g49f65d86-dirty OpenPGP: id=EE19E1C1F2F7054F8D3954D8308964B51883A0DD; url=https://ftp.sdaoden.eu/steffen.asc; preference=signencrypt BlahBlahBlah: Any stupid boy can crush a beetle. But all the professors in the world can make no bugs. X-Rspamd-Queue-Id: 48475L1rw2z4NBK X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of steffen@sdaoden.eu designates 217.144.132.164 as permitted sender) smtp.mailfrom=steffen@sdaoden.eu X-Spamd-Result: default: False [0.18 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.63)[-0.626,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+a]; NEURAL_HAM_LONG(-0.20)[-0.201,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[sdaoden.eu]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MID_CONTAINS_FROM(1.00)[]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15987, ipnet:217.144.132.0/24, country:DE]; IP_SCORE(0.30)[asn: 15987(1.53), country: DE(-0.02)] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Jan 2020 18:31:35 -0000 Yuri Pankov wrote in : |Steffen Nurpmeso wrote: |> The very moment i have seen a showmount(8) change fly by, and |> wanted to make you aware of the persuading approach that NetBSD's |> wizd(8) uses, namely ".Fl Fl long-option". Different to ".Fl |> -long-option" this produces visually appealing results on all |> output devices. | |mandoc style guide advises differently: | |https://mandoc.bsd.lv/mdoc/style/options.html | |It's not authoritative, of course, but I'm not aware of any other mdoc |style guide. May be you could take it up to style guide/mandoc authors? I can Cc: him. That is _he_, who is totally opposed against long- options, isn't he? Whatever this guide says, if you do Postscript/PDF output (with groff) then Fl becomes a nice dash, whereas the other thing is a hyphen-minus. I am using Fl Fl ever since i saw wizd(8)'s comment, tried it, and was satisfied by the results. Despite that Fl Fl is ugly but logically deducable, too. --End of --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) From owner-freebsd-hackers@freebsd.org Fri Jan 24 18:56:09 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BCC331F8DB5 for ; Fri, 24 Jan 2020 18:56:09 +0000 (UTC) (envelope-from yp@xvoid.org) Received: from wnew1-smtp.messagingengine.com (wnew1-smtp.messagingengine.com [64.147.123.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4847dh639Fz4PQl for ; Fri, 24 Jan 2020 18:56:08 +0000 (UTC) (envelope-from yp@xvoid.org) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailnew.west.internal (Postfix) with ESMTP id C83AC583; Fri, 24 Jan 2020 13:56:06 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Fri, 24 Jan 2020 13:56:06 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xvoid.org; h= subject:to:references:from:message-id:date:mime-version :in-reply-to:content-type:content-transfer-encoding; s=fm1; bh=0 d31A0935S+GsMzqBlykbV+zMyRcXaZUG+dD8AWb7vw=; b=G4zTP81MXtqDQKiSz gJOXNICEUgv7GU4KU+cejmOOW/gExqja+XRX9yQgbBZ9S1BN9f4qz3E/d+wJPkKG xMXktUfq7Y7TtY9iNwj9ofRdB0rCpvC5uRevtsLEa4I7Losu6dzVf1h9uN38cZSE 04vFaBbweqrf6wXTe5tiqWTpZxTLQmvLbg/nHDZci0IoXuq2DJGTSD21CykFK+T9 hZRbHC+y5O76Z1TDX8SybOT53aZUoCMRTBlZsL1Vl6k4ZAAcbNfezEkCKwZaVGqb pnYKQQoIRehIBAvOmITR/TYlKewBpYEgHTmQk5FDp1BbYSPprKNrdVXLYN4Puh1f oKtPQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=0d31A0935S+GsMzqBlykbV+zMyRcXaZUG+dD8AWb7 vw=; b=jsHVnFapc1uNtiUiGAuhr8mhtfI6V1j5cVflZeqd+ORqLpZ9rsySWP39W gFXPW5S/bAhL+xlmGIAbClnqgIumXWJ/oxr3awhzkxtf6fJ/v9qbQ06MXUiPF98z Pg3XUjtHi+Zt3LO7l1ye971iTsxkbR9MHH2ck3BhkVvHGtFqM9o/tz3GbYG2gxfM lSoVi6XrPddQ2i3AAVk4CIw0beR90On2IfgTrnkZjV6QIpZnDqgaNjAXz2Il7KM1 2ZwwuwhHQLIpgiV0dS1+v2vaBp0TQxMWD/aETDxWlAmW8ARAKUZDYp4ZOISO+yCs pkiofkBFCyNgRb8xdGzjNZ8HUzETg== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedugedrvdeggdduudejucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepuffvfhfhkffffgggjggtgfesthejredttdefjeenucfhrhhomhepjghurhhi ucfrrghnkhhovhcuoeihphesgihvohhiugdrohhrgheqnecuffhomhgrihhnpegsshgurd hlvhenucfkphepudejkedrudehhedrgedrkeelnecuvehluhhsthgvrhfuihiivgeptden ucfrrghrrghmpehmrghilhhfrhhomhephihpseigvhhoihgurdhorhhg X-ME-Proxy: Received: from earth.xvoid.org (unknown [178.155.4.89]) by mail.messagingengine.com (Postfix) with ESMTPA id 12EC530617E6; Fri, 24 Jan 2020 13:56:04 -0500 (EST) Subject: Re: Long (-- style) options (.Fl) with mdoc(7) To: freebsd-hackers@freebsd.org, Ingo Schwarze References: <20200124144829.49360%steffen@sdaoden.eu> <20200124183132.-nDI2%steffen@sdaoden.eu> From: Yuri Pankov Message-ID: <6ab0ed49-0fa4-8028-8f31-a4f4daa5e538@xvoid.org> Date: Fri, 24 Jan 2020 21:56:03 +0300 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:68.0) Gecko/20100101 Thunderbird/68.4.1 MIME-Version: 1.0 In-Reply-To: <20200124183132.-nDI2%steffen@sdaoden.eu> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4847dh639Fz4PQl X-Spamd-Bar: ++++ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=xvoid.org header.s=fm1 header.b=G4zTP81M; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=jsHVnFap; dmarc=none; spf=pass (mx1.freebsd.org: domain of yp@xvoid.org designates 64.147.123.26 as permitted sender) smtp.mailfrom=yp@xvoid.org X-Spamd-Result: default: False [4.38 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(0.00)[xvoid.org:s=fm1,messagingengine.com:s=fm1]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(0.00)[+ip4:64.147.123.26:c]; RECEIVED_SPAMHAUS_XBL(5.00)[89.4.155.178.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.4]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[xvoid.org]; NEURAL_SPAM_MEDIUM(0.18)[0.177,0]; BAD_REP_POLICIES(0.10)[]; RCVD_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[xvoid.org:+,messagingengine.com:+]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_SPAM_LONG(0.83)[0.831,0]; IP_SCORE(-1.53)[ipnet: 64.147.123.0/24(-4.92), asn: 11403(-2.68), country: US(-0.05)]; RCVD_IN_DNSWL_LOW(-0.10)[26.123.147.64.list.dnswl.org : 127.0.5.1]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11403, ipnet:64.147.123.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[89.4.155.178.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.11] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Jan 2020 18:56:09 -0000 Steffen Nurpmeso wrote: > Yuri Pankov wrote in : > |Steffen Nurpmeso wrote: > |> The very moment i have seen a showmount(8) change fly by, and > |> wanted to make you aware of the persuading approach that NetBSD's > |> wizd(8) uses, namely ".Fl Fl long-option". Different to ".Fl > |> -long-option" this produces visually appealing results on all > |> output devices. > | > |mandoc style guide advises differently: > | > |https://mandoc.bsd.lv/mdoc/style/options.html > | > |It's not authoritative, of course, but I'm not aware of any other mdoc > |style guide. May be you could take it up to style guide/mandoc authors? > > I can Cc: him. That is _he_, who is totally opposed against long- > options, isn't he? > Whatever this guide says, if you do Postscript/PDF output (with > groff) then Fl becomes a nice dash, whereas the other thing is > a hyphen-minus. I am using Fl Fl ever since i saw wizd(8)'s > comment, tried it, and was satisfied by the results. Despite that > Fl Fl is ugly but logically deducable, too. Note that I don't disagree on using Fl Fl, and quite some of our man pages already do it, I just want it documented at least somewhere so it's possible to link to it if there's a question how something should be done. From owner-freebsd-hackers@freebsd.org Fri Jan 24 23:03:05 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 948901FF0D2 for ; Fri, 24 Jan 2020 23:03:05 +0000 (UTC) (envelope-from yp@xvoid.org) Received: from wnew2-smtp.messagingengine.com (wnew2-smtp.messagingengine.com [64.147.123.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 484F6c3Kysz3C2h for ; Fri, 24 Jan 2020 23:03:04 +0000 (UTC) (envelope-from yp@xvoid.org) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailnew.west.internal (Postfix) with ESMTP id ED8F64E6 for ; Fri, 24 Jan 2020 18:03:02 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Fri, 24 Jan 2020 18:03:03 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xvoid.org; h= subject:references:to:from:message-id:date:mime-version :in-reply-to:content-type:content-transfer-encoding; s=fm1; bh=X jbQgv8szJsTXvgYpaaxl+jYgxyqLOdgyx/RlC4jIK0=; b=uEty432JC2wW7v4o3 8TgrCQj0+NOlQ2kL3g8bYxp2IZ7hRkpl2XsUSwY5JA/xJkQznA6ULh7h5kyqrkc0 tJqmCuotUIcj9on1xomTHwaiyhQgVX0v/0Oxbns5j1FujYgoMOx9jrv3MPUGylk7 Uj+poQLa1P9snnp94Mk4Gcd6x2LxNzXLXEIAhvmL8B+Ke/Huvzn1iEQrnjBnlnPu x0i4Dyqzvr0FN8Gkgf0mHLiQRBPkZW5SROhYQBP0S8GGDL3X4QMwb3IN4EGV7y3b dcjul+w/z4trVspJp4jNf9HMmhaUEHyvmFnStA+5Xde1sGueMxZSJ2S0RFXX18Ke jKvcw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=XjbQgv8szJsTXvgYpaaxl+jYgxyqLOdgyx/RlC4jI K0=; b=sVncuzEErffZSMP2YS9fS3hyillYEksIOID5/dIS8rItGPNOulQTUcCg6 HMauc0mn+OPS5MIn3/CVPO1Y3R4SesOjWisyrwgaITpfgvKc2rqX8C5UoUkp4qTa YTZh/k/YoKODo1vATqTpe9166XJcc8mS5XiL9jDYHk/gJmr3oHvtkrlCGj43c/Jv 0rnrQUYryzT6M5zGPUB47d1yJJ7TU9E2UgGOb4XjQpUWsSqiGl9WP09+WgUO40h6 ne5vkJlmzd9nElqoeT6rK0v8lfcUUKzxKS/mpQMCZJsuSWdPT3MmOdh/SUPUDb04 Kj7qZt6a5Rwq7HZYG0osN+t2rHTBA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedugedrvdeigddthecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepufhfvffhkffffgggjggtgfesthejre dttdefheenucfhrhhomhepjghurhhiucfrrghnkhhovhcuoeihphesgihvohhiugdrohhr gheqnecuffhomhgrihhnpegsshgurdhlvhenucfkphepudejkedrudehhedrgedrkeelne cuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhephihpseig vhhoihgurdhorhhg X-ME-Proxy: Received: from [192.168.1.6] (unknown [178.155.4.89]) by mail.messagingengine.com (Postfix) with ESMTPA id ACB323062F4B for ; Fri, 24 Jan 2020 18:03:01 -0500 (EST) Subject: Fwd: Re: Long (-- style) options (.Fl) with mdoc(7) References: <20200124194444.GK35815@athene.usta.de> To: freebsd-hackers@freebsd.org From: Yuri Pankov X-Forwarded-Message-Id: <20200124194444.GK35815@athene.usta.de> Message-ID: Date: Sat, 25 Jan 2020 02:02:59 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:68.0) Gecko/20100101 Thunderbird/68.4.1 MIME-Version: 1.0 In-Reply-To: <20200124194444.GK35815@athene.usta.de> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 484F6c3Kysz3C2h X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=xvoid.org header.s=fm1 header.b=uEty432J; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=sVncuzEE; dmarc=none; spf=pass (mx1.freebsd.org: domain of yp@xvoid.org designates 64.147.123.27 as permitted sender) smtp.mailfrom=yp@xvoid.org X-Spamd-Result: default: False [0.90 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(0.00)[+ip4:64.147.123.27]; TO_DN_NONE(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[xvoid.org:+,messagingengine.com:+]; RECEIVED_SPAMHAUS_PBL(0.00)[89.4.155.178.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.11]; RCVD_IN_DNSWL_LOW(-0.10)[27.123.147.64.list.dnswl.org : 127.0.5.1]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:11403, ipnet:64.147.123.0/24, country:US]; MID_RHS_MATCH_FROM(0.00)[]; SH_EMAIL_ZRD(0.00)[0.0.0.0]; ARC_NA(0.00)[]; RECEIVED_SPAMHAUS_XBL(5.00)[89.4.155.178.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.4]; R_DKIM_ALLOW(0.00)[xvoid.org:s=fm1,messagingengine.com:s=fm1]; NEURAL_HAM_MEDIUM(-0.22)[-0.216,0]; FROM_HAS_DN(0.00)[]; SH_EMAIL_DBL_DONT_QUERY_IPS(0.00)[0.0.0.0]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.31)[-0.311,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[xvoid.org]; RCPT_COUNT_ONE(0.00)[1]; BAD_REP_POLICIES(0.10)[]; MIME_TRACE(0.00)[0:+]; IP_SCORE(-3.47)[ip: (-9.70), ipnet: 64.147.123.0/24(-4.92), asn: 11403(-2.68), country: US(-0.05)] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Jan 2020 23:03:05 -0000 Forwarding on behalf of Ingo as he isn't subscribed, and the message provides useful content. -------- Forwarded Message -------- Subject: Re: Long (-- style) options (.Fl) with mdoc(7) Date: Fri, 24 Jan 2020 20:44:44 +0100 From: Ingo Schwarze To: Yuri Pankov CC: freebsd-hackers@freebsd.org Hi Yuri, Yuri Pankov wrote on Fri, Jan 24, 2020 at 09:56:03PM +0300: > Steffen Nurpmeso wrote: >> Yuri Pankov wrote in : >>> Steffen Nurpmeso wrote: >>>> The very moment i have seen a showmount(8) change fly by, and >>>> wanted to make you aware of the persuading approach that NetBSD's >>>> wizd(8) uses, namely ".Fl Fl long-option". The reason why that is a bad idea is that it is semantically wrong. The meaning of ".Fl Fl long-option" is: The option "-" (as in the traditional "su -") followed by the other, additional option "-long-option" >>>> Different to ".Fl -long-option" this produces visually appealing >>>> results on all output devices. That is doubtful: $ echo .Fl Fl long-option | mandoc -mdoc -Thtml [...] --long-option [...] You get two separate elements which can screw CSS markup. >>> mandoc style guide advises differently: >>> https://mandoc.bsd.lv/mdoc/style/options.html >>> It's not authoritative, of course, but I'm not aware of any other mdoc >>> style guide. May be you could take it up to style guide/mandoc authors? >> I can Cc: him. That is _he_, who is totally opposed against long- >> options, isn't he? Yes. I consider long options a scourge - hard to document, laborious to type, and encouraging poor CLI design because they encourage excessive numbers of command line options, so program authors tend to stop considering whether yet another option is really needed. Usually, less is more. Besides, i'm not alone with that opinion: POSIX also discourages them. However, the many downsides of long options have no bearing on the fact that where they exist and do not have short aliases, they need to be documented. >> Whatever this guide says, if you do Postscript/PDF output (with >> groff) then Fl becomes a nice dash, whereas the other thing is >> a hyphen-minus. That just isn't true: $ cat tmp.mdoc .Fl \-long\-option $ groff -mdoc -Tps tmp.mdoc [...] /F0 10/Courier-Bold@0 SF(\255\255long\255option)73.666 12 Q 0 Cg EP All three dashes become the same character in -T ps output, and that is not a coincidence because the groff_mdoc(7) macros implement the .Fl macro in terms of \-: $ less /co/groff/tmac/doc.tmac-u [...] .de Fl [...] . nop \|\-\f[]\s[0]\c [...] So, *typographically*, .Fl Fl long\-option and .Fl \-long\-option produce exactly the same when disregarding \| spaces. The difference is purely *semantic*, and in that respect, .Fl Fl is clearly wrong. > Note that I don't disagree on using Fl Fl, and quite some of our man > pages already do it, I just want it documented at least somewhere so > it's possible to link to it if there's a question how something should > be done. The "Fl Fl" idiom is not so bad that it is urgent to fix it, i think there are some "Fl Fl" in OpenBSD too, but it is better avoided. Yours, Ingo