From owner-svn-src-all@freebsd.org Wed Nov 6 23:45:43 2019 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E54BD17C21A; Wed, 6 Nov 2019 23:45:43 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 477jpH5lMgz3JMF; Wed, 6 Nov 2019 23:45:43 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9049C4F56; Wed, 6 Nov 2019 23:45:43 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id xA6Njhh3099775; Wed, 6 Nov 2019 23:45:43 GMT (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id xA6Njh9i099774; Wed, 6 Nov 2019 23:45:43 GMT (envelope-from glebius@FreeBSD.org) Message-Id: <201911062345.xA6Njh9i099774@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: glebius set sender to glebius@FreeBSD.org using -f From: Gleb Smirnoff Date: Wed, 6 Nov 2019 23:45:43 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r354419 - head/sys/kern X-SVN-Group: head X-SVN-Commit-Author: glebius X-SVN-Commit-Paths: head/sys/kern X-SVN-Commit-Revision: 354419 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Nov 2019 23:45:44 -0000 Author: glebius Date: Wed Nov 6 23:45:43 2019 New Revision: 354419 URL: https://svnweb.freebsd.org/changeset/base/354419 Log: If vm_pager_get_pages_async() returns an error synchronously we leak wired and busy pages. Add code that would carefully cleanups the state in case of synchronous error return. Cover a case when a first I/O went on asynchronously, but second or N-th returned error synchronously. In collaboration with: chs Reviewed by: jtl, kib Modified: head/sys/kern/kern_sendfile.c Modified: head/sys/kern/kern_sendfile.c ============================================================================== --- head/sys/kern/kern_sendfile.c Wed Nov 6 23:44:38 2019 (r354418) +++ head/sys/kern/kern_sendfile.c Wed Nov 6 23:45:43 2019 (r354419) @@ -269,6 +269,16 @@ sendfile_iodone(void *arg, vm_page_t *pg, int count, i if (!refcount_release(&sfio->nios)) return; + if (__predict_false(sfio->error && sfio->m == NULL)) { + /* + * I/O operation failed, but pru_send hadn't been executed - + * nothing had been sent to the socket. The syscall has + * returned error to the user. + */ + free(sfio, M_TEMP); + return; + } + #if defined(KERN_TLS) && defined(INVARIANTS) if ((sfio->m->m_flags & M_EXT) != 0 && sfio->m->m_ext.ext_type == EXT_PGS) @@ -279,7 +289,7 @@ sendfile_iodone(void *arg, vm_page_t *pg, int count, i ("non-ext_pgs mbuf with TLS session")); #endif CURVNET_SET(so->so_vnet); - if (sfio->error) { + if (__predict_false(sfio->error)) { /* * I/O operation failed. The state of data in the socket * is now inconsistent, and all what we can do is to tear @@ -414,10 +424,25 @@ sendfile_swapin(vm_object_t obj, struct sf_io *sfio, i rv = vm_pager_get_pages_async(obj, pa + i, count, NULL, i + count == npages ? &rhpages : NULL, &sendfile_iodone, sfio); - if (rv != VM_PAGER_OK) { - for (j = i; j < i + count; j++) { - if (pa[j] != bogus_page) - vm_page_unwire(pa[j], PQ_INACTIVE); + if (__predict_false(rv != VM_PAGER_OK)) { + /* + * Perform full pages recovery before returning EIO. + * Pages from 0 to npages are wired. + * Pages from i to npages are also busied. + * Pages from (i + 1) to (i + count - 1) may be + * substituted to bogus page, and not busied. + */ + for (j = 0; j < npages; j++) { + if (j > i && j < i + count - 1 && + pa[j] == bogus_page) + pa[j] = vm_page_lookup(obj, + OFF_TO_IDX(vmoff(j, off))); + else if (j >= i) + vm_page_xunbusy(pa[j]); + KASSERT(pa[j] != NULL && pa[j] != bogus_page, + ("%s: page %p[%d] I/O recovery failure", + __func__, pa, j)); + vm_page_unwire(pa[j], PQ_INACTIVE); } VM_OBJECT_WUNLOCK(obj); return (EIO); @@ -806,7 +831,8 @@ retry_space: if (error != 0) { if (vp != NULL) VOP_UNLOCK(vp, 0); - free(sfio, M_TEMP); + sfio->m = NULL; + sendfile_iodone(sfio, NULL, 0, error); goto done; }