From nobody Sun Aug 21 00:02:48 2022 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4M9G0D6Z50z4Zdvq for ; Sun, 21 Aug 2022 00:02:52 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-YQB-obe.outbound.protection.outlook.com (mail-yqbcan01on2078.outbound.protection.outlook.com [40.107.116.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "DigiCert Cloud Services CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4M9G0C5lLgz47VY for ; Sun, 21 Aug 2022 00:02:51 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=GO6Lmx3IOKEFJ1GjYZ0UfiUzv/EDWanJ2+etbBPTduGnljHVG2v9ut2uXKsHoyQFcZWlFuYnQjwzaBHraFm5kBLMmK35wPcJzaA71g76v3/nvy6Uavvj5XlGOIwpMCN02/WBQfvaIVyhefbjCu1CzhQvKv4fpm/Gffgwy9wXFwDBKDP5xRN7GT7aRwxrdJcbCsVR/M/EryKVLSu6Gb1sajGKqiViu8vH7CEWue+mMRiPCzTVAM49f48b7qZU3kLcqWiI6EWH/4OINT3oyPX0FME10FaiuJOlSKG33OtUYeUxGg9b4dwOYyE3eaH/VAVP3fc+NCdgggAeFDJS6z7D/w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4dsHbxf/Qx6U4WICZFXQB2fdxiRlrJjwsgVQBaNrypM=; b=hX9Jau6juC/JB7TSgxAOM6wv70lsUNvxx+FaZ83SRJMOyV2jOC1zPz9FuiwfCslhiAW/qjYlHR+CBjpnTgLjl2AcUhyOaVRjDepTXiZW5aXaKPx0+t500gKzgK6vXEfs0qV0HilupdWHu4oQEoIcgbrFEn/t+yf68xFgeVzIb122oNm5bU1EnOUadOPDUP4oiFQj0Lyqm4eVDNgsHQ0BxWZpc44rdLPDKno+IbCRTM05eM1Py2HVBC2KmXJfbvaUyBTtsTrs+ByE8UVN+RC8+QpeJFxSsk3oVZqH9hoNDPXOCdkHAGARr/GjujjDsGW61ke1VNnpAe868h7csZB8cQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uoguelph.ca; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4dsHbxf/Qx6U4WICZFXQB2fdxiRlrJjwsgVQBaNrypM=; b=QYCRhGvCfpWJ1bOqdrRiKlDnWy9QgzArjFnMbXPefpV4KrhphC2PoArGDgMUHa8UrrHhpdbuZVVjwlpfviN0LBCCOiDFiTZqxTu+IyvoT/QBDtPuRyxHi9mfXNm25IZqOq9O4YucLdt2ez9IaM846eYV360hUhjJO44puKh18+wWGq5fzrh8uI1zxBVWeS6F7ksFCJr7msbhkQi0oSttLVLuo5k1U4EIMTuJRZCm4pSluLgWn+8jXmGC7EhN6FpWu9PEHg2rHbjxCsI/L3UHUj9AxpRATCubdaOV1NlqBS8hHjppG+JIBcnM0qdtGOQx2B76l1V0RAuM92D2awAVew== Received: from YT4PR01MB9736.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:e6::17) by YQXPR01MB3126.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:4d::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5546.16; Sun, 21 Aug 2022 00:02:49 +0000 Received: from YT4PR01MB9736.CANPRD01.PROD.OUTLOOK.COM ([fe80::4c04:eddf:6168:b3ec]) by YT4PR01MB9736.CANPRD01.PROD.OUTLOOK.COM ([fe80::4c04:eddf:6168:b3ec%3]) with mapi id 15.20.5546.021; Sun, 21 Aug 2022 00:02:48 +0000 From: Rick Macklem To: Konstantin Belousov CC: FreeBSD Filesystems Subject: Re: SEEK_DATA/SEEK_HOLE with vnode locked Thread-Topic: SEEK_DATA/SEEK_HOLE with vnode locked Thread-Index: AQHYrQY7fWQ5YS0dF02y4OgRBDgrJa2owxGAgA/DcfU= Date: Sun, 21 Aug 2022 00:02:48 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 22356ee7-b7f1-41ef-bcb2-08da83087bf3 x-ms-traffictypediagnostic: YQXPR01MB3126:EE_ x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: ObyW3NCUmkJKGPEYNo+KrH2atHKKXlvhoUwlLEhv7uZ41+XcWTd9o3F/A0vUS8dLhe5GcyzcCDT/JiIqjG9wIDVvtjVIS3JOB2pi5REqDxq7/WE+K/+1wK6V/DeJFIMbITGFrEQJw/ltrtgES6ZO1hMpcl7J6m7hMKczxRD6HwfyaW7u5/nO/7MDjo6Uhdt39iVcqNhCCevWXlE3S7Xt8pYk2O0AvlDDHTy3v1Q3fFRIW+Hv4LD4tOhEJwyjegqdOE5tJ8rWko4lAXk6eZ9tFWLQxe2eseK1SbA0yyU7pKeysUUcqEXupCgt7t5I4+369tcpGOU44RQ17HYpxsSGFLKxW8fYuOOrmRkF9p1lXWRYpzCJ+nxk/Kq4sIefKxxZM98RGRQ5gvmW1itoIrPU/pmLULqvIN/FpK0KTE5WjWKkUkzf2FnfJ2vhTZOECf6hA2igYMqgGA6TI+ZwyJ3izWCeQpYbakn7iuJ4LIctWhHGzlbd/kl3MLBj7rR0nvKXe+zQ44cE59uXkVKj1F5Z5b2vm1sFfP4gfb3/yk+4ElhBQtiWKhy3G41stBHyg4EidDVHiv54ZG9GWqpjHIML4rTzmwxbu360MIP4Wp+Z+XwUe6tSVEMxSEVPgL4x9p/8/4414ElpWSvNhZmh0s325fnUQosJ6nrBkb8REGTTJ93Ef8gEZxaVGLMqYOqUHxVhjPoFHKk4tA7uovz1d7y8yERYwatiQuJDG7uCBzHDMSAqELde757GdeTpJNNSbsIHxFVki/x/bVf1N3aO5Clp/Q== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:YT4PR01MB9736.CANPRD01.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230016)(4636009)(346002)(396003)(376002)(366004)(39860400002)(136003)(66476007)(8936002)(55016003)(8676002)(316002)(66446008)(76116006)(91956017)(66946007)(86362001)(122000001)(66556008)(64756008)(5660300002)(6916009)(2906002)(52536014)(4326008)(786003)(53546011)(6506007)(478600001)(33656002)(7696005)(71200400001)(41320700001)(186003)(41300700001)(83380400001)(9686003)(38070700005)(38100700002);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?2BcB8gO4/TcYBuzO20YdYvIPq9RlXmsfrXjel+JSugDlmnhv7RoqUP0/Z208?= =?us-ascii?Q?2lK6bXN8/zj0wzzF6zGz0DMgV85YUyUuStespmzrM/4cN0eMgwmeF6DRlU4G?= =?us-ascii?Q?8YBIIW6Lwp+P26LWz5J9dTX1dLlk5p5FOz+OanVEVGq+ZrESziy/Rm2QIf7U?= =?us-ascii?Q?5ey4lOBnke9kio9tdCihPGiujtpTD0LISjgu/06TRT2aaMMYhG4xpyfWSvdZ?= =?us-ascii?Q?Ol+u3YdsKrVksIPqmMtQkkene9qTGZWyMFg0axcXeza6w6jXC8cVkegiPrlf?= =?us-ascii?Q?JvIvdqnfniMSMqATcX7Ij+rSYU/FKdE6NpcvCa4R5Au7S7UiNluS8GubRBnk?= =?us-ascii?Q?nW0dQuNWvqLR3+c9EvvKFKDDqUSqUfsvgU5Rk9Ifi6qk1awwTuOdJS5CSzGQ?= =?us-ascii?Q?wYRKbe0B8Li6YcnpWh/5C+ZAU3pA9xeTJdQ6pMc+JPDGA/1u4n7Gg56TV7fi?= =?us-ascii?Q?e5cQgK6+7MlWTbEbLYm1iiXjwxh2pHT0NFqaRXVJkONcAd4vs4K9TyeMqRHD?= =?us-ascii?Q?GjZYhlAO2Xki3JxOTXD9dUfSz+bvDjlz8oj6xZwt3RNEZsGnQlx2bxpItXgo?= =?us-ascii?Q?PBg8SGZbNYViv4mRCPtJJbXwX8cd6q9CkRBNtpo66UdmIp7c6JnezxpA1hVy?= =?us-ascii?Q?mavZqUnZuXrwBJ7jtzGwcpAFScMCpz5wtYXluHIb1svSyzqH/O+rVrwrsUko?= =?us-ascii?Q?RZ6tlQGmbSBppY/gigqPqglkg2HSgcKX6DGmngEnGseLOZnwq7YBtqU18uzR?= =?us-ascii?Q?u6nCZXSg16P4nbM7yC5yDTj2Sjcc+/8gE5kcem2ANxAbzXtTbSalbED3awlH?= =?us-ascii?Q?WkbDsmmXP7acU3DRcEtBSpbx2ru0NxHA46DQZqDg4H0fXQjglFjMmqSyAhSo?= =?us-ascii?Q?QTYLJ/Lb9T1+JGDZV3HTCGdsEazvP20tCjK0tWCrG4Nh2NonUTxvPi6rf3mx?= =?us-ascii?Q?3Sx1VWwGy/gvnJd5eWpSBQx0HZvzWe2cij/XS8OZtvf7s1Si+4DtgDcYnBIb?= =?us-ascii?Q?jTJsTW+1WtG0W4job+Bzinn7OIEL4y4f4igmlvVUE/4Sc+yOoCGpkQFjbZ6x?= =?us-ascii?Q?u48DgVCGZBOMd8deXB15mZB5krek9dmMnkjDrX+HyvBE48bhDIlqmVi0tC4D?= =?us-ascii?Q?S1mnOnUAeE1J+hheWr6FP9J8mfdAfm/tHwveHLnqP8WWONS4Zxb7B8Ceq1SK?= =?us-ascii?Q?P+bxLhWV+MySPITGrDGNOro9mQE0opQm2tev6poE5BBy6U9bE3oalRFjj0hX?= =?us-ascii?Q?DdQ1a5xTKhF6V56xRDILSdyBXi0qR/61IU0qeAkHHLRZh/Dmkm/ShlG+ws3P?= =?us-ascii?Q?MHFMP+7vTDKXBBrNIx9Z4uGFLanSK3ET7PHw5uDcwJPDC2ew7wJNHBtpXznd?= =?us-ascii?Q?Q+nurdH6PdLKLZjNutHfKzMoXPo3q3nZGQ+XLU0v0Lnc+fZW2eVcc04xrfKs?= =?us-ascii?Q?7Dt6zj4D6ZKi9Qirq+KxwAkzMRrdArRp7+HssnshH+FA2BEwqURPL0sGfCi0?= =?us-ascii?Q?nKCtj0nQUDWL2VKNLjZz+MIsUGmhYyhZf2WztlTqGDzEFqPwBIz/hU81vZ0q?= =?us-ascii?Q?p6rREA9eeo5AfZVUd+YqHdUvp0lzjMrgKh4MDSQbriixTRLjecCCp6gvSTYd?= =?us-ascii?Q?jwv/tIV6hNSdi978452G6d81QW72pfdNHnTY3s2/tT7s?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: YT4PR01MB9736.CANPRD01.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: 22356ee7-b7f1-41ef-bcb2-08da83087bf3 X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Aug 2022 00:02:48.8425 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: DFPrItX5dExxzLuNsSFkDbHkj2ObEioq6NebOshlGdv20pGz8wu/Lj2uBIibYI3LUzfpp9oyZhFhlnJQzOWSQA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQXPR01MB3126 X-Rspamd-Queue-Id: 4M9G0C5lLgz47VY X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=uoguelph.ca header.s=selector2 header.b=QYCRhGvC; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=none) header.from=uoguelph.ca; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 40.107.116.78 as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [-6.00 / 15.00]; DWL_DNSWL_LOW(-1.00)[uoguelph.ca:dkim]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[uoguelph.ca,none]; R_DKIM_ALLOW(-0.20)[uoguelph.ca:s=selector2]; R_SPF_ALLOW(-0.20)[+ip4:40.107.0.0/16]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:8075, ipnet:40.104.0.0/14, country:US]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[uoguelph.ca:+]; TO_DN_ALL(0.00)[]; FREEFALL_USER(0.00)[rmacklem]; RCPT_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; RWL_MAILSPIKE_POSSIBLE(0.00)[40.107.116.78:from] X-ThisMailContainsUnwantedMimeParts: N Just to summarize this... I was able to do a VOP_SEEK() which would be called with a LK_SHARED locked vnode and it seemed to work fine. However, ReadPlus (which is like Read, but allows for holes to be represented as in the reply instead of a stream of 0 bytes) seems to be a performance dud. I was surprised how poorly it performed compares to ordinary Read. Typically it would take 60% longer to read a file. I tried sparse and non-sparse files of various sizes and they always took longer. (If I disabled SEEK_DATA/SEEK_HOLE in the server code, so it never actually did holes, it worked comparably to regular Read, so somehow the overhead of doing SEEK_DATA/SEEK_HOLE was a big performance hit. It was using LK_SHARED locks, so it wasn't serializing the reads, but I don't really know why it performed so poorly?) Anyhow, unless the performance issue gets resolved, there is no reason to commit the code to FreeBSD's main. (NFSv4.2 operations, like ReadPlus, are all optional and are not required for an RFC conformant implementation.) rick ________________________________________ From: Konstantin Belousov Sent: Wednesday, August 10, 2022 7:11 PM To: Rick Macklem Cc: FreeBSD Filesystems Subject: Re: SEEK_DATA/SEEK_HOLE with vnode locked CAUTION: This email originated from outside of the University of Guelph. Do= not click links or open attachments unless you recognize the sender and kn= ow the content is safe. If in doubt, forward suspicious emails to IThelp@uo= guelph.ca On Wed, Aug 10, 2022 at 10:17:36PM +0000, Rick Macklem wrote: > Hi, > > When implementing what is called Read_Plus for the NFSv4.2 > server, I need to do SEEK_DATA/SEEK_HOLE. However, if I > use VOP_IOCTL(), I need to unlock/relock the vnode. > > This can result in problems if another RPC changes the size > of the file or allocates/deallocates data in the file while the > vnode is unlocked. > > >From what I can see, UFS does SEEK_DATA/SEEK_HOLE with > the vnode locked and ZFS doesn't seem to care/notice if it > is locked. > (Actually, ZFS looks like it might be unsafe, since it seems to > assume it can use the unlocked vnode that might be doomed, > but I do not know ZFS.) The statement about UFS needs more precision. UFS VOP_IOCTL() takes unlocked but references vnode, and the first thing vn_bmap_seekhole/data() and ufs_vmap_seekdata() do is vn_lock(vp, LK_SHARED). Since LK_RETRY flag is not specified, doomed vnodes result in ENOENT. I believe this interface was done because I wrote bmap-based seekhold/data while ZFS had its implementation already in place. It avoided taking the vnode lock, so UFS needed to do the same. I am not sure was the ZFS interface decision due to some internal consisten= cy guarantees already present, or because internal locking causing ordering issue with the vnode lock. > > Anyhow, does implementing a new vnode op VOP_SEEK(), which > takes a locked vnode and does SEEK_DATA/SEEK_HOLE sound > reasonable? You need to make some decision about ZFS first, I believe. UFS and generic buffer cache consumers fs are trivial to adopt, of course.