From owner-freebsd-current@freebsd.org Fri Mar 17 01:57:26 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B4956D0FC09 for ; Fri, 17 Mar 2017 01:57:26 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-TO1-obe.outbound.protection.outlook.com (mail-eopbgr670059.outbound.protection.outlook.com [40.107.67.59]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6D94F1DEF; Fri, 17 Mar 2017 01:57:25 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM (10.165.218.133) by YTXPR01MB0191.CANPRD01.PROD.OUTLOOK.COM (10.165.218.135) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.961.17; Fri, 17 Mar 2017 01:57:23 +0000 Received: from YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM ([10.165.218.133]) by YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM ([10.165.218.133]) with mapi id 15.01.0961.022; Fri, 17 Mar 2017 01:57:23 +0000 From: Rick Macklem To: Dimitry Andric , Ian Lepore CC: Gergely Czuczy , FreeBSD Current Subject: Re: process killed: text file modification Thread-Topic: process killed: text file modification Thread-Index: AQHSnqPLfXcZwdtVHkecT5jK6Yv9dKGYQJIX Date: Fri, 17 Mar 2017 01:57:23 +0000 Message-ID: References: <646c1395-9482-b214-118c-01573243ae5a@harmless.hu> <45436522-77df-f894-0569-737a6a74958f@harmless.hu> <55189643.aaZPuY77s8@ralph.baldwin.cx> <3ed3e4a3-23af-7267-39f1-9090093c9c1e@harmless.hu> <5ac94b9a-7ced-9eff-d746-7dddaaeca516@harmless.hu> <1489340839.40576.82.camel@freebsd.org>, In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: harmless.hu; dkim=none (message not signed) header.d=none;harmless.hu; dmarc=none action=none header.from=uoguelph.ca; x-ms-office365-filtering-correlation-id: f6a7712a-ecd9-46e3-9d1e-08d46cd8f4a6 x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001);SRVR:YTXPR01MB0191; x-microsoft-exchange-diagnostics: 1; YTXPR01MB0191; 7:UK9g98OHegZFbT37cYu72jKDHRKGc74BMWrAjpg8cDwItd/fA8l5foqu3o7mUm/y+5JpiNq0PsQtYucdWYAxP6qA43UvSG5qSUkBnzBS1C58ve4qD0a6yMWCk/uh4Da0lnbmYrIxv1/KyvRkKJF3RcsuPIeaIwgZVWi7xiQ8dYo1MHIN2udv5aQ+BpJZ6tR8aCkR6eWuztFGBTVQDpu7iAzGdx6ZpZh0L+vpdkv6ZEj9HuMtIswpq2cTQn0QBkbFfDsAIttutFRgh8ed5F37MBlCfUh8PVakd5coNm5sPy6C1Wkmhmf4OzDGJlXtss9KLvntFx/Z2IZ+9tr6UqEXyQ== x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040375)(2401047)(5005006)(8121501046)(10201501046)(3002001)(6041248)(20161123555025)(20161123562025)(20161123558025)(20161123564025)(20161123560025)(6072148); SRVR:YTXPR01MB0191; BCL:0; PCL:0; RULEID:; SRVR:YTXPR01MB0191; x-forefront-prvs: 0249EFCB0B x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(39410400002)(39830400002)(39450400003)(24454002)(2900100001)(229853002)(93886004)(54356999)(76176999)(50986999)(6436002)(33656002)(102836003)(6306002)(6506006)(55016002)(77096006)(54906002)(74482002)(74316002)(9686003)(305945005)(53936002)(7696004)(8936002)(2950100002)(122556002)(81166006)(86362001)(3660700001)(8676002)(2906002)(5660300001)(38730400002)(189998001)(6246003)(966004)(4326008)(3280700002); DIR:OUT; SFP:1101; SCL:1; SRVR:YTXPR01MB0191; H:YTXPR01MB0189.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; MLV:sfv; LANG:en; spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Mar 2017 01:57:23.7971 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-Transport-CrossTenantHeadersStamped: YTXPR01MB0191 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Mar 2017 01:57:26 -0000 Dimitry Andric wrote: [lots of stuff snipped] > I'm also running into this problem, but while using lld. I must set > vfs.timestamp_precision to 1 (e.g. sec + ns accurate to 1/HZ) on both > the client and the server, to make it work. >=20 > Instead of GNU ld, lld uses mmap to write to the output executable. If > this executable is more than one page, and resides on an NFS share, > running it will almost always result in "text file modification", if > vfs_timestamp_precision >=3D 2. >=20 > A small test case: http://www.andric.com/freebsd/test-mmap-write.c, > which writes a simple "hello world" i386-freebsd executable file, using > the sequence: open() -> ftruncate() -> mmap() -> memcpy() -> munmap() -> > close(). Hopefully Kostik will correct me if I have this wrong, but I don't believe = any of the above syscalls guarantee that dirty pages have been flushed. At least for cases without munmap(), the writes of dirty pages can occur af= ter the file descriptor is closed. I run into this in NFSv4, where there is a C= lose (NFSv4 one) that can't be done until VOP_INACTIVE(). If you look in the NFS VOP_INACTIVE() { called ncl_inactive() } you'll see: if (NFS_ISV4(vp) && vp->v_type =3D=3D VREG) { 237 /* 238 * Since mmap()'d files do I/O after VOP_CLOSE(), the = NFSv4 239 * Close operations are delayed until now. Any dirty 240 * buffers/pages must be flushed before the close, so = that the 241 * stateid is available for the writes. 242 */ 243 if (vp->v_object !=3D NULL) { 244 VM_OBJECT_WLOCK(vp->v_object); 245 retv =3D vm_object_page_clean(vp->v_object, 0,= 0, 246 OBJPC_SYNC); 247 VM_OBJECT_WUNLOCK(vp->v_object); 248 } else 249 retv =3D TRUE; 250 if (retv =3D=3D TRUE) { 251 (void)ncl_flush(vp, MNT_WAIT, NULL, ap->a_td, = 1, 0); 252 (void)nfsrpc_close(vp, 1, ap->a_td); 253 } 254 } Note that nothing like this is done for NFSv3. What might work is implementing a VOP_SET_TEXT() vnode op for the NFS client that does most of the above (except for nfsrpc_close()) and then set= s VV_TEXT. --> That way, all the dirty pages will be flushed to the server before the = executable starts executing. > Running this on an NFS share, and then attempting to run the resulting > 'helloworld' executable will result in the "text file modification" > error, and it will be killed. But if you simply copy the executable to > something else, then it runs fine, even if you use -p to retain the > properties! > > IMHO this is a rather surprising problem with the NFS code, and Kostik > remarked that the problem seems to be that the VV_TEXT flag is set too > early, before the nfs cache is invalidated. Rick, do you have any ideas > about this? I don't think it is the "nfs cache" that needs invalidation, but the dirty pages written by the mmap'd file need to be flushed, before the VV_TEXT is set, I think? If Kostik meant "attribute cache" when he said "nfs cache", I'll note that = the cached attributes (including mtime) are updated by the reply to every write= . (As such, I think it is the writes that must be done before setting VV_TEXT that is needed.) It is a fairly simple patch to create. I'll post one to this thread in a da= y or so unless Kostik thinks this isn't correct and not worth trying. rick