From owner-freebsd-fs@freebsd.org Fri Dec 11 01:21:14 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4162A4BE7C2 for ; Fri, 11 Dec 2020 01:21:14 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-TO1-obe.outbound.protection.outlook.com (mail-to1can01on0612.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe5d::612]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CsXzs2LdWz4btX for ; Fri, 11 Dec 2020 01:21:13 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Vrrf9k6/Wz9BdNzZMVfGAYymGK+Z6g9+Cr8FOcaP1rZTVMwjq2t5Ye92HqoXjR6qteEIR+hSqkwGzfykcEeJF53oT9I9uBNviwHbxe3PDBqfQh7+LjTNM2sBw4nfXVIt9oNJPW1JqEQklF5d+Lki4TSxhyjHXjGZXQyu+AyrmhTVTy8b8QVj2B54TYGA8CtE2KCMytxB6i/d3Mkl8CaImZKmmzkIeOxoibDgYnQjE2Y6pYOXSvfOhlqDLzTl8Qk6WouNKScBblVnVcnPyV57D8xMlBHyOKkILKZgtRF5HJGoA3zncIm/EtToOKZeg/05S7wHniGOxgF8hfKB1uMEiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vFFcG++IOfEvSZQaTvPEJIUlgClTcaYm/sMn7YzTBOk=; b=ktLPEr/3XIfu6zjVifLhC85MYZpJEtgNfk1qmVssreuz+07PIrHns91EbYUC5KTUFYU/y6AieLamVLcL7QPuZcNAprFISQJ91T47F+tyxHUx+McDGhOL+UU+q+m984YPVq3BPDeB2NPZOh3vrVd/gnnyp+WWwolfXDPZU6VVlyON+EeMeGS+N+bqZbA+fT4cTmTq9dpq8z/kBgbwMDPxRSWJUUHrq4NopjnskWU0a5a55A2W/N4HtXrIdoE2aTVRRgb9VVfNvQQibYTlminXXL33078UHOs9fhwcT0eqqNA7hcATWGBMCn1KmU5hjNESMnEeGbTHBTUxTsBsBAnLtg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uoguelph.ca; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vFFcG++IOfEvSZQaTvPEJIUlgClTcaYm/sMn7YzTBOk=; b=KY5OruZWTL2jWhRcUIrUtibMX/nJ4YRK0Lxb5fuXcfKvsTggT7Gh6qy6xQn0wEwjVE/OgA8s/fBORwkakvUX8ijJulDNUi6EAn8K3iI/Jw8NAeKvlRO/Xci0UGoA10wvUq+BpkAq5geBffhF/K+i9HXUagXJGlHWOets/8TnDpxob+o3SHTx5lSVWI/aYsBowS6qsLhUbrHqWiI0IA4DYbzckZ8im6eWhb1Exg5wT6xzRo3MsSQQ/Z6VXgseAZQCBEha0ki/wlhxNNUyxDvA/Mw2zCaa52eCdSZxnRzzxwbZyQ7SeMIJhlG1ra3vKxTGcUCvXt2UUdpon+APWBQBQg== Received: from YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:19::29) by YQXPR01MB2406.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:51::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3632.21; Fri, 11 Dec 2020 01:21:11 +0000 Received: from YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM ([fe80::7d6b:aa68:78f4:5d94]) by YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM ([fe80::7d6b:aa68:78f4:5d94%6]) with mapi id 15.20.3632.027; Fri, 11 Dec 2020 01:21:11 +0000 From: Rick Macklem To: J David , Peter Eriksson CC: FreeBSD FS Subject: Re: Major issues with nfsv4 Thread-Topic: Major issues with nfsv4 Thread-Index: AQHWzw/HDat+dHoH9kKG5K3Xpd53kqnwv/2AgAAIjACAAE0Npg== Date: Fri, 11 Dec 2020 01:21:11 +0000 Message-ID: References: <976EC1BD-AB37-478C-B567-E8013E80F071@lysator.liu.se>, In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: aefd4e19-e512-40dd-22db-08d89d730ba7 x-ms-traffictypediagnostic: YQXPR01MB2406: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: gbfFnof0OGZSW0lQBB03OD7hJs4JsxNnFA8EDAUshC58eUGEheuidgJw40U/BfYfsavUXl28J866aE1vH35mNfcc+5IfAXSNLPWjWU+Xvb3DWrnYOFbzlV0+mn+owPESbvo88IIEVhf1leDQ75EXujplo7qnQR4uPLTTk4Tcm6q+a7GiBrRcIQsnczo5dwSeQxHwyRnLKqHJU2Qg/05zLjrjiVtgV1VqjWpeTbLiaEQKFf/2Ott7af2hozAza3YkHHd/FrnH/Ebepgn4NZOeslTO/fyGQ6K2gT/wduStXBjSbwfDzlLUQ6QMK/RaaSOF8htVWMJURLmEx/Nv92VgEBuw/6I45sPjhhrGx2UcqBgk+xHWvkAl+v8k5GP5WSu+om3Fk5uYDJSq6Q5Awswoww== x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(376002)(136003)(346002)(366004)(186003)(76116006)(91956017)(66946007)(7696005)(52536014)(66476007)(66556008)(33656002)(786003)(8936002)(2906002)(71200400001)(5660300002)(110136005)(9686003)(55016002)(83380400001)(966005)(64756008)(66446008)(6506007)(86362001)(4326008)(508600001)(8676002); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata: =?iso-8859-1?Q?iwfnVF6P1m2tZg/gDkUojXoHHwOKbK+YX83ouyXVVKYjK6AcgpVfoHYtu/?= =?iso-8859-1?Q?GL1ZQWZJWwUEGF+bHsWQiXMKqu2btLkrrjCHv4Eqolh0oXSa/y0Nio+oXZ?= =?iso-8859-1?Q?DefcnNS1T0DsEfMBLHJVExehsSdaDZhFWLxMjpNcaMDOn5iJ9EEPwPML+r?= =?iso-8859-1?Q?Z5U9X/0LiJZmEyRP5xWV6Utk3PGPdmee89wL0ldsQwgF6nP3XxNBJKLvo9?= =?iso-8859-1?Q?68czaVsA5a0IKkQwW4sb7eX/Ssne1+EgOwMGoRufruYHny0ScNa06Qk0I5?= =?iso-8859-1?Q?zqa89krl2re8TfNQy9hC73YqDd0ElfvX3K7fiffMhLpx1mVZOCrVL0oCc2?= =?iso-8859-1?Q?R91vxfx4rV130gHf7wxWlfPMLib9Cn9yOxbE1Z+W8ITqOZ7/d3S5fScwmr?= =?iso-8859-1?Q?qMlGy43IeUrMaHUieyBUpHyAGA+wzToBV/0XM5oEpw+uYpRfPyLBT0XfnG?= =?iso-8859-1?Q?ffm3Bx7kCEQv5t7syN6J93XZMC6osUnnbDq7DyF+PmQB6hogRx2g35OtE0?= =?iso-8859-1?Q?zzIKe9EWYuIEpHMUOGZbvWIOSSv8uWkQ2wSKaPDCCVVTPdSfr7wcrcUh04?= =?iso-8859-1?Q?O0+6D7HDEnyLWxZ6uXhFevh17dw/sIVoI5YUi2VLaTSocRRAKNuIac8KH/?= =?iso-8859-1?Q?CWq2B1NzeQDWDdwM+VkrMA1TiyINYn9/Y+ywWY8nCO1JQ262MhemJabIrk?= =?iso-8859-1?Q?LuJrzWXvEJmFJqi5yU3dGpm5QU3FMnMi58ukAtt3MwIkjHmyooLIMFSkRM?= =?iso-8859-1?Q?JaVxf48fq1F8R3bkwEMjVujFCukN+ggAf49OYNILUZ9d6AGNc8EddlWc5K?= =?iso-8859-1?Q?gM7AP19oA/OSv/N6ORIzqFBaSVgsTYfhhUvqg4gWP7UhvdpLCX3z2ZU+l8?= =?iso-8859-1?Q?AvULSsEltNugVDwalz8MAX4s4lH27+MugvriWoR0xXKpmxBZNV1i8PFlgm?= =?iso-8859-1?Q?xvixk7CCQViC2ggUUSqnfdX4fGHd9rUX1S5plzrwf2hBs9+K6KKN8CAleC?= =?iso-8859-1?Q?qNW3aRAuvrFI/CTou1Jg5epZRengByEWWQrQ49qSnRvijyUQYkNrtEObz/?= =?iso-8859-1?Q?3L1YJ4f+Nr024QYhOGV59b4=3D?= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: aefd4e19-e512-40dd-22db-08d89d730ba7 X-MS-Exchange-CrossTenant-originalarrivaltime: 11 Dec 2020 01:21:11.4058 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 4edoaf3ck/tYWUBxh8ZWfjYnAC1HGTy5UEJ5SRtac1RqYStW7EAZLKy/UX6zV3whjVtR3i8b84BQCewVx2JASQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQXPR01MB2406 X-Rspamd-Queue-Id: 4CsXzs2LdWz4btX X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=uoguelph.ca header.s=selector1 header.b=KY5OruZW; arc=pass (microsoft.com:s=arcselector9901:i=1); dmarc=pass (policy=none) header.from=uoguelph.ca; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 2a01:111:f400:fe5d::612 as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a01:111:f400:fe5d::612:from]; R_DKIM_ALLOW(-0.20)[uoguelph.ca:s=selector1]; FREEFALL_USER(0.00)[rmacklem]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip6:2a01:111:f400::/48]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DWL_DNSWL_LOW(-1.00)[uoguelph.ca:dkim]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[uoguelph.ca:+]; DMARC_POLICY_ALLOW(-0.50)[uoguelph.ca,none]; SPAMHAUS_ZRD(0.00)[2a01:111:f400:fe5d::612:from:127.0.2.255]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_SPAM_LONG(1.00)[1.000]; FREEMAIL_TO(0.00)[gmail.com,lysator.liu.se]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:8075, ipnet:2a01:111:f000::/36, country:US]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; MAILMAN_DEST(0.00)[freebsd-fs] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Dec 2020 01:21:14 -0000 J. David wrote:=0A= >Ah, oops. The "12.2 servers" referred to at the top of the message=0A= >are the NFS *clients* in this scenario. They are application servers,=0A= >not NFS servers. Sorry for the confusing overloaded usage of "server"=0A= >there!=0A= So what is your NFS server running?=0A= =0A= Btw, if it happens to be a Linux system and you aren't using Kerberos,=0A= it will expect Users/Groups as the numbers in strings by default.=0A= To do that, do not start the nfsuserd(8) daemon on the client and=0A= instead add the following line to the client's /etc/sysctl.conf file:=0A= vfs.nfs.enable_uidtostring=3D1=0A= =0A= When User/Group mapping is broken, you'll see lots of files owned=0A= by "nobody".=0A= =0A= Also, if you do want to see what the NFS packets look like, you can=0A= capture packets with tcpdump, but then look at them in wireshark.=0A= # tcpdump -s 0 -w out.pcap host =0A= - then look at out.pcap in wireshark. Unlike tcpdump, wireshark=0A= knows how to parse NFS messages properly.=0A= =0A= rick=0A= ps: Once you have switched to NFSv4.1 and have User/Group=0A= mapping working, I suspect the NFS clients will be ok.=0A= Using NFSv4.1 also avoids FreeBSD NFS server issues w.r.t.=0A= tuning the DRC, since it is not used by NFSv4.1 (again, fixed=0A= by sessions).=0A= =0A= Everything in the message (dmesg, tcpdump, nfsstat, etc.) is from the=0A= perspective of a FreeBSD 12.2 NFS client, which is where the problems=0A= are occurring.=0A= =0A= Our Linux servers (machines? instances? hosts? nodes?) that are NFS=0A= clients have been running NFSv4 against the same servers for many=0A= years without incident.=0A= =0A= Thanks!=0A= _______________________________________________=0A= freebsd-fs@freebsd.org mailing list=0A= https://lists.freebsd.org/mailman/listinfo/freebsd-fs=0A= To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"=0A= =0A=