From owner-svn-src-head@freebsd.org Mon Mar 30 21:57:38 2020 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 36F2D26837C; Mon, 30 Mar 2020 21:57:38 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48rmXc6Tsjz4SlC; Mon, 30 Mar 2020 21:57:36 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id F1775250EE; Mon, 30 Mar 2020 21:57:28 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 02ULvSg8038069; Mon, 30 Mar 2020 21:57:28 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 02ULvS5J038068; Mon, 30 Mar 2020 21:57:28 GMT (envelope-from kib@FreeBSD.org) Message-Id: <202003302157.02ULvS5J038068@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Mon, 30 Mar 2020 21:57:28 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r359469 - head/sys/kern X-SVN-Group: head X-SVN-Commit-Author: kib X-SVN-Commit-Paths: head/sys/kern X-SVN-Commit-Revision: 359469 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Mar 2020 21:57:38 -0000 Author: kib Date: Mon Mar 30 21:57:28 2020 New Revision: 359469 URL: https://svnweb.freebsd.org/changeset/base/359469 Log: kern_sendfile.c: wait for all in-flight ios completion before unwiring pages. Reviewed by: glebius, markj Tested by: pho Sponsored by: The FreeBSD Foundation MFC after: 1 week Differential revision: https://reviews.freebsd.org/D24038 Modified: head/sys/kern/kern_sendfile.c Modified: head/sys/kern/kern_sendfile.c ============================================================================== --- head/sys/kern/kern_sendfile.c Mon Mar 30 21:50:51 2020 (r359468) +++ head/sys/kern/kern_sendfile.c Mon Mar 30 21:57:28 2020 (r359469) @@ -256,6 +256,17 @@ fixspace(int old, int new, off_t off, int *space) } /* + * Wait for all in-flight ios to complete, we must not unwire pages + * under them. + */ +static void +sendfile_iowait(struct sf_io *sfio, const char *wmesg) +{ + while (atomic_load_int(&sfio->nios) != 1) + pause(wmesg, 1); +} + +/* * I/O completion callback. */ static void @@ -437,6 +448,8 @@ sendfile_swapin(vm_object_t obj, struct sf_io *sfio, i i + count == npages ? &rhpages : NULL, &sendfile_iodone, sfio); if (__predict_false(rv != VM_PAGER_OK)) { + sendfile_iowait(sfio, "sferrio"); + /* * Perform full pages recovery before returning EIO. * Pages from 0 to npages are wired. @@ -967,6 +980,7 @@ retry_space: m != NULL ? SFB_NOWAIT : SFB_CATCH); if (sf == NULL) { SFSTAT_INC(sf_allocfail); + sendfile_iowait(sfio, "sfnosf"); for (int j = i; j < npages; j++) vm_page_unwire(pa[j], PQ_INACTIVE); if (m == NULL)