From nobody Wed Jan 5 21:04:14 2022 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id D56DA1947943 for ; Wed, 5 Jan 2022 21:04:23 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-QB1-obe.outbound.protection.outlook.com (mail-eopbgr660061.outbound.protection.outlook.com [40.107.66.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "DigiCert Cloud Services CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JThn26NWZz51Xc for ; Wed, 5 Jan 2022 21:04:22 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jrBt55VrkM/BBu2dv8K2QTepwgdQ9nrEqiH+XZQBjfIjCnDAlHGKsOFsJfuQ2TyI9puPj+aY4mrXHKsKKFN31Fgiu6Gn0t+T72ww/A+d/EuVC0cFV36Z4+ilBkJkFdAwVEcNtc+uJNZkhu0g8gJaEWVXqRzZdcvqAkSsEiYTlX3izYr2ukHiAg6B4ngouvmaetL+GJykbsi52Mk9V8FpNO2/j1fYA9DgWtDBkXL+TjLfmxEn5Y/f3C+RbR77y9301uSrQDnc4xtM5m8vrtDgVv28wAmYaVjw1d+cTzPfrxO2FUgNo//2iaAEFKxgjuI6YgXPjofRHJpMR86dGg8ZoA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=LuP+hXdAUwpaM3bL/ByNeiFC+8cOIkqj+FNUo0oKDeQ=; b=OeDVGr5FcAg5FzDDi877FnvV2S+AJbaHCOWBlhrMMicYJd+S86Efgov/tQhUp3OK3YZJw5tzer4DPhyd2kjsKult9yHoGH6rfQO4N3pKAxkittCwpuhvZMQav9gBBb3vYmKgpiQx/7we5BGA5/m/xVuutEBp/QTBTOyjBdqKRggxojThvG754zMk44qqDptPCO1isEyCF/IliuPBUpGmZ0q77dItUtNqp+ojgy+7vxAE08JQW8lPH+G7iTGyvl/ETvRYMPFHJTyJzGaBhq2F8dIe5GQPEgvsbCVkSdBPY5M5oQk+q2RfUVe5svteN6LYmtj2hPtJS2JB1Hy+52jvmg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uoguelph.ca; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LuP+hXdAUwpaM3bL/ByNeiFC+8cOIkqj+FNUo0oKDeQ=; b=kSJvxHKyV2wQMN7uL0WzYVB4AfGvkFGn7LHmt8O9+rwdc3tI53FVkFh3PgFMASiTMFEHnWfk67f7dPQCcxmq0ETf1yrAUlXCK50PcfnBefjNv96iVkJ7Gwk2hNV1RAEicuNtOTND5awr9GT8JKihYySBkfsAnSz3s2JCFosfPg6pDniBmQ5TTK0kF81OtU0Vd0igeaSNORBK2FUuJU5/onQLe3azCN5Ywakhm18MP4/CxdRszmfGjSPkANvQhs91Sch31elvaWgyNQvJZVPFdAczrHPrDJBZFRiensSxRBdI5ABa8x/ZUdkFO636cKdKwQtVtmitnirNqu7YP6vaTg== Received: from YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:19::29) by YQBPR0101MB2276.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:5::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4867.9; Wed, 5 Jan 2022 21:04:14 +0000 Received: from YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM ([fe80::50bf:ecf6:9d13:fd03]) by YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM ([fe80::50bf:ecf6:9d13:fd03%4]) with mapi id 15.20.4844.017; Wed, 5 Jan 2022 21:04:14 +0000 From: Rick Macklem To: Zaphod Beeblebrox , freebsd-current Subject: Re: Writing large build logs to NFS extremely slow? Thread-Topic: Writing large build logs to NFS extremely slow? Thread-Index: AQHXvWMEhXhFzFqRLESM9Us/FueWtqxVcvPx Date: Wed, 5 Jan 2022 21:04:14 +0000 Message-ID: References: <20211007021643.bwglyvrswk2nm3fl@nexus.home.palmen-it.de> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: suggested_attachment_session_id: 181dcbc6-2538-d681-6ad9-d57d3898a296 x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 660c07aa-d402-41ed-9213-08d9d08eee07 x-ms-traffictypediagnostic: YQBPR0101MB2276:EE_ x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2958; x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: GEaSCr5hITVGhTuvS4cUMDwo2yB40U7tJGOFPpHDnZkkGARAwetBAR+7yKIViBaTu/7iVYPAkQ65MIyFG57OC4Q3CzzYNrk9z2awGMyO3E19zQJMs4iqoLKrsZ5Nyr67dbkbBg0VeaC/+H35hAhDTgKzdoH0LKx6Ywc68vrnnuAK5yOUUjzyg8BVYXN6M0hRMLjF89ezTeAX5HSdKw+NM06YLq8sKwYPUu8mD9KAo7WYGX+ikwifyRB20IklEgzf1uBynyk2bfaSKshDfHa3stLPcnfEKzfwGpYfzr6UTx5FtQ9pfOH1M6VqMahciUOmjA7XJzI28rlWCxVmf21FcHcLiyPf/IoowZn+6CpLiLrTkxATS+s0+8Ds7LP16IAeGpnyrtrTqNdPqHIHBXhewLpU4ssmOQ4Cc6Gx6DgwKZ20q7MCW3Y79VICFhFopFWCG68j5JBD+sYyAEOIrFWa9NR/cLdJRbvdGitzYEMfkJ6WZb+NBcfxTvIisomD2S/vH8eaIod+HcXKUCwF4yvlsS6359/of89IlmIKl5p5HOGa7GnYlO39uVeU/5YmEfe6L11ybX31r7N5sejXgm5loiJvqflrf9qnZ6AZW1I/bdRSjitdXvAmE3On6dJ3fOK0D8l4Liy52xVLUPjDBVH0ZtiqGxdyQ8pt+qqPcRZTLqsixcaXl7vfs7QwAi6Qd4tcYzB4qIGbKCHEFLocR9FE9Ze8T3BiypqUscO3zSdAZQ5Nbb6uFZfHFThOw0m7uxxJQ3RmTf1NqNGbYrmE68IDmpbh/LI2tEly2ZM8LXgaEb4= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(366004)(508600001)(2906002)(110136005)(83380400001)(966005)(7696005)(38070700005)(8676002)(786003)(6506007)(53546011)(8936002)(52536014)(316002)(38100700002)(9686003)(55016003)(66556008)(186003)(64756008)(66446008)(71200400001)(5660300002)(33656002)(66476007)(122000001)(76116006)(66946007)(91956017)(86362001);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?Windows-1252?Q?zCXWUXdKioe5ETAaY3AekIA7tmICljSdgeWA5K1+jHXbzWWiytpiZAEZ?= =?Windows-1252?Q?ND0m1R1vdpWfLakeaoA+Vh09CMGIJIW+wVC61O5J79nLzRFNgRDNh3Dj?= =?Windows-1252?Q?aLATn48YbDyV70qZCqqkQu/6VmsXT3pkHAXR5ifJmyDD207+qf2GTCTX?= =?Windows-1252?Q?kjhpBPKK8pYDrMdKA6aibtXn31OLgUtMsmP946E+TD30SwKSFCriiIln?= =?Windows-1252?Q?C6fljN8PwwR0Qszgqm/JNz7oRJzNCCmb+sv5UFfTFHP3TGzK3HZ7klqV?= =?Windows-1252?Q?wlauZAOI9BzpyuckKo3t5qaWD0+C2wbTUqDdjAL+DJ1I5k7LavWi/K6B?= =?Windows-1252?Q?PAUOUk9ejR/jOzy2jj5Eli7ACngs91CBeZe2EUF8Cf838jbByZZgnZNW?= =?Windows-1252?Q?i3Fxi+AGnWxJTuzh0IfeE7qSAkjr+cQumBASCRY11ut4ccpgw0RrLN9+?= =?Windows-1252?Q?Cc2mBIJF4/BNxkNrc9he+qMcw33gq1Amji8MHpk/Vu4ED0e0F3xfkYL7?= =?Windows-1252?Q?Pwlex7qNsV+JU1h9M+znzhO3KVU+he/P0snpDkWR4xMkOL6avS3k1t4Q?= =?Windows-1252?Q?MYDuDx6TM0oaNLIXObmZn6ju3rEeCSOO1QGocbwlQtlQr2aI9+eCck9H?= =?Windows-1252?Q?1UX40lQRORcWl8jxwXl4B+82FHxcYBsf3p12bcIgr8Iv1kmDEAaSrN53?= =?Windows-1252?Q?sTlbjEKhz1WxWp5sdjU6OWPxk+CO0OzGbUjNg5jEQZIVCAjquNxUHpxV?= =?Windows-1252?Q?YWUAai4JlhPSYE7NCdUdSe0e5W419h6VZ7+doY0lsPKyTA64VdyBr6Wp?= =?Windows-1252?Q?ZVkdhiXDUlHsI9yWEp0wahm45RsMbCkZK+KR8W+cDYw3Q3/3ke3doD6u?= =?Windows-1252?Q?A9VoeZ7+u1lwgPviM5S7bSAIR5hfxekLpr8nZfVv8iS6CYCE0jIRsUjZ?= =?Windows-1252?Q?ijdcLvVi72oBJ98i2qSZZZbV4+DcgUYd22j+UturritMKJ3Ie3H+GE4I?= =?Windows-1252?Q?SV70Ui0PqVyR1u86Lo5OtLDbfZsxOxJz8s6zGW7fZwrrIOKOaTHBqHAn?= =?Windows-1252?Q?498NVcpNgjm5c3plEX0/PrG+PQh8//i3+bppWaPkv3tlrrK+PeGN46Pj?= =?Windows-1252?Q?VQwEcP41Bv+OjPs7MFjn0qS1UTsn0NkOUNfDrsBY7W4a4ohdZ3BDUn1i?= =?Windows-1252?Q?Jq182JZml4zX+O/CLeJ4jRtK4vH6UD+9zCJohWnjH4QKt67wINoZ59/c?= =?Windows-1252?Q?UcXDVI5nOa9Kt6FYf7nXlJQ/CLbfnbjg4SzF35hSt+XULqJ43UEG8Ezd?= =?Windows-1252?Q?0Nv3ywmTagoqILJnewequE5MkeAyRm4a2i90gr6ZdXcB+tRdg5xNS5xB?= =?Windows-1252?Q?8FtrOokMfz9d1cUpIoHUHIqCWqjJ6D2tdT9pUSr+QoB4R6AZOfNGkyv1?= =?Windows-1252?Q?giemIB+xcyfL5CBclyx3BNJti8tKm3j408gDk4LJiSdVKuskHKwSzqzu?= =?Windows-1252?Q?fS/XM7RP3zDTGVkIHqR+atJxi3wn55nRwPCVhSuZ6YJJ1qq4cTjApKEm?= =?Windows-1252?Q?MWPnl2TaqoheXPDfyNxSdIud67G4CUgLOiRk+3JC729+aBTn6A87WWzL?= =?Windows-1252?Q?LZ91wLzynty/YdHuQZbP5KAoh+vKy4sq4lbIa8gGQfO/R8lUhDLU0li9?= =?Windows-1252?Q?G7VA6ukYrJUyxcZYBmIw+HyLwXMS97ca669lVS7v0mZpZOeIAatLQA7K?= =?Windows-1252?Q?pcoKHJNe3kqhPW9sXq4u+gQSOeZwvuS35eNP/cEJM5s03cCRceH/p9Dp?= =?Windows-1252?Q?d6WVrA=3D=3D?= Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: 660c07aa-d402-41ed-9213-08d9d08eee07 X-MS-Exchange-CrossTenant-originalarrivaltime: 05 Jan 2022 21:04:14.6574 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Ai+mrrHQkQmQe3qZUvxLR3/ZAo+TYlRVb4TTdliy3bHdJ10E9zGNeoCxVUyYQKoUmC4DhYQp6wjuJPU4inYt6A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQBPR0101MB2276 X-Rspamd-Queue-Id: 4JThn26NWZz51Xc X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=uoguelph.ca header.s=selector2 header.b=kSJvxHKy; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=none) header.from=uoguelph.ca; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 40.107.66.61 as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [-3.01 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[uoguelph.ca:s=selector2]; FREEFALL_USER(0.00)[rmacklem]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:40.107.0.0/16]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; NEURAL_SPAM_SHORT(0.99)[0.987]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; DWL_DNSWL_LOW(-1.00)[uoguelph.ca:dkim]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[uoguelph.ca:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[40.107.66.61:from]; DMARC_POLICY_ALLOW(-0.50)[uoguelph.ca,none]; FREEMAIL_TO(0.00)[gmail.com,freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:8075, ipnet:40.104.0.0/14, country:US]; RCVD_TLS_LAST(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[40.107.66.61:from] X-ThisMailContainsUnwantedMimeParts: N Zaphod Beeblebrox wrote:=0A= > Is the NFS mounted filesystem NFS?=0A= > I've found NFS mounted ZFS has several=0A= > pathologies like this when there is no SSD cache and/or log vdevs attache= d.=0A= Just fyi, I've recently committed a change to main that changes the NFS=0A= client so that it does append writes directly to the NFS server.=0A= This improved the append write performance by 10x for UFS, but only=0A= improved 30% for ZFS without any ZIL log and sync=3Denabled.=0A= Append writes over NFS is still MUCH slower than a local file system.=0A= (Partially because it has to do every write to the serve FileSync, which me= ans=0A= the server must commit the changes to stable storage before replying to th= e=0A= RPC. It also must do a Getattr RPC against the server before every write,= =0A= to get the up-to-date file size.)=0A= =0A= This change won't by MFC'd for a couple of months and is somewhat broken=0A= in main at this time. I am working with kib@ to fix it.=0A= =0A= rick=0A= =0A= On Wed, Oct 6, 2021 at 10:18 PM Felix Palmen wrote:=0A= =0A= > Hi all,=0A= >=0A= > I use a -CURRENT bhyve vm for testing port builds with poudriere. As=0A= > this vm is only running when needed, but I want to always have access to= =0A= > the build logs, I use NFS to mount /usr/local/poudriere/data/logs from=0A= > the host.=0A= >=0A= > I noticed some few ports take ridiculously long to build while barely=0A= > using any CPU time at all. On a closer look, that's all ports producing= =0A= > a lot of compiler (warning) output, e.g. gcc, gnutls, gtk2, =85=0A= >=0A= > So I assume appending to a large file via NFS gets slower and slower. Is= =0A= > there any mount option I could try to fix this? Right now I only have=0A= > `nolockd`, I also tried `noncontigwr` which didn't change anything.=0A= >=0A= > Thinking about alternatives to NFS, are there any news for client-side=0A= > 9p virtfs? I found which=0A= > still builds with a few minor adaptions, but trying to mount a 9p share= =0A= > freezes the machine.=0A= >=0A= > Would you suggest a different mailing list to ask?=0A= >=0A= > BR, Felix=0A= >=0A= > --=0A= > Dipl.-Inform. Felix Palmen ,.//..........=0A= > {web} http://palmen-it.de {jabber} [see email] ,//palmen-it.de=0A= > {pgp public key} http://palmen-it.de/pub.txt // """""""""""=0A= > {pgp fingerprint} A891 3D55 5F2E 3A74 3965 B997 3EF2 8B0A BC02 DA2A=0A= >=0A=