From owner-svn-src-all@freebsd.org Thu Nov 24 06:01:30 2016 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E43DFC527B4; Thu, 24 Nov 2016 06:01:30 +0000 (UTC) (envelope-from sephe@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A513DC8C; Thu, 24 Nov 2016 06:01:30 +0000 (UTC) (envelope-from sephe@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id uAO61Tff067183; Thu, 24 Nov 2016 06:01:29 GMT (envelope-from sephe@FreeBSD.org) Received: (from sephe@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id uAO61TE2067181; Thu, 24 Nov 2016 06:01:29 GMT (envelope-from sephe@FreeBSD.org) Message-Id: <201611240601.uAO61TE2067181@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: sephe set sender to sephe@FreeBSD.org using -f From: Sepherosa Ziehau Date: Thu, 24 Nov 2016 06:01:29 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r309083 - head/sys/dev/hyperv/vmbus X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Nov 2016 06:01:31 -0000 Author: sephe Date: Thu Nov 24 06:01:29 2016 New Revision: 309083 URL: https://svnweb.freebsd.org/changeset/base/309083 Log: hyperv/vmbus: Fix the multi-channel revoking on vmbus side. - Reference count the sub-channel when channel offer message is processed, so that immediate rescind message on the same channel will not race sub-channel open on driver side. - Drop the above reference when sub-channel is closed, this closely mimics the hypervisor's reaction when primary channel is closed on the VM side. No drivers use sub-channel after primary channel is closed. MFC after: 1 week Sponsored by: Microsoft Differential Revision: https://reviews.freebsd.org/D8546 Modified: head/sys/dev/hyperv/vmbus/vmbus_chan.c head/sys/dev/hyperv/vmbus/vmbus_chanvar.h Modified: head/sys/dev/hyperv/vmbus/vmbus_chan.c ============================================================================== --- head/sys/dev/hyperv/vmbus/vmbus_chan.c Thu Nov 24 05:52:28 2016 (r309082) +++ head/sys/dev/hyperv/vmbus/vmbus_chan.c Thu Nov 24 06:01:29 2016 (r309083) @@ -65,6 +65,7 @@ static void vmbus_chan_cpu_default(str static int vmbus_chan_release(struct vmbus_channel *); static void vmbus_chan_set_chmap(struct vmbus_channel *); static void vmbus_chan_clear_chmap(struct vmbus_channel *); +static void vmbus_chan_detach(struct vmbus_channel *); static void vmbus_chan_ins_prilist(struct vmbus_softc *, struct vmbus_channel *); @@ -628,6 +629,32 @@ vmbus_chan_gpadl_disconnect(struct vmbus } static void +vmbus_chan_detach(struct vmbus_channel *chan) +{ + int refs; + + KASSERT(chan->ch_refs > 0, ("chan%u: invalid refcnt %d", + chan->ch_id, chan->ch_refs)); + refs = atomic_fetchadd_int(&chan->ch_refs, -1); +#ifdef INVARIANTS + if (VMBUS_CHAN_ISPRIMARY(chan)) { + KASSERT(refs == 1, ("chan%u: invalid refcnt %d for prichan", + chan->ch_id, refs + 1)); + } +#endif + if (refs == 1) { + /* + * Detach the target channel. + */ + if (bootverbose) { + vmbus_chan_printf(chan, "chan%u detached\n", + chan->ch_id); + } + taskqueue_enqueue(chan->ch_mgmt_tq, &chan->ch_detach_task); + } +} + +static void vmbus_chan_clrchmap_task(void *xchan, int pending __unused) { struct vmbus_channel *chan = xchan; @@ -752,8 +779,15 @@ vmbus_chan_close(struct vmbus_channel *c int i; subchan = vmbus_subchan_get(chan, subchan_cnt); - for (i = 0; i < subchan_cnt; ++i) + for (i = 0; i < subchan_cnt; ++i) { vmbus_chan_close_internal(subchan[i]); + /* + * This sub-channel is referenced, when it is + * linked to the primary channel; drop that + * reference now. + */ + vmbus_chan_detach(subchan[i]); + } vmbus_subchan_rel(subchan, subchan_cnt); } @@ -1114,6 +1148,7 @@ vmbus_chan_alloc(struct vmbus_softc *sc) return NULL; } + chan->ch_refs = 1; chan->ch_vmbus = sc; mtx_init(&chan->ch_subchan_lock, "vmbus subchan", NULL, MTX_DEF); sx_init(&chan->ch_orphan_lock, "vmbus chorphan"); @@ -1137,6 +1172,8 @@ vmbus_chan_free(struct vmbus_channel *ch VMBUS_CHAN_ST_ONLIST)) == 0, ("free busy channel")); KASSERT(chan->ch_orphan_xact == NULL, ("still has orphan xact installed")); + KASSERT(chan->ch_refs == 0, ("chan%u: invalid refcnt %d", + chan->ch_id, chan->ch_refs)); hyperv_dmamem_free(&chan->ch_monprm_dma, chan->ch_monprm); mtx_destroy(&chan->ch_subchan_lock); @@ -1213,6 +1250,14 @@ vmbus_chan_add(struct vmbus_channel *new ("new channel is not sub-channel")); KASSERT(prichan != NULL, ("no primary channel")); + /* + * Reference count this sub-channel; it will be dereferenced + * when this sub-channel is closed. + */ + KASSERT(newchan->ch_refs == 1, ("chan%u: invalid refcnt %d", + newchan->ch_id, newchan->ch_refs)); + atomic_add_int(&newchan->ch_refs, 1); + newchan->ch_prichan = prichan; newchan->ch_dev = prichan->ch_dev; @@ -1359,6 +1404,7 @@ vmbus_chan_msgproc_choffer(struct vmbus_ if (error) { device_printf(sc->vmbus_dev, "add chan%u failed: %d\n", chan->ch_id, error); + atomic_subtract_int(&chan->ch_refs, 1); vmbus_chan_free(chan); return; } @@ -1426,9 +1472,7 @@ vmbus_chan_msgproc_chrescind(struct vmbu if (bootverbose) vmbus_chan_printf(chan, "chan%u revoked\n", note->chm_chanid); - - /* Detach the target channel. */ - taskqueue_enqueue(chan->ch_mgmt_tq, &chan->ch_detach_task); + vmbus_chan_detach(chan); } static int Modified: head/sys/dev/hyperv/vmbus/vmbus_chanvar.h ============================================================================== --- head/sys/dev/hyperv/vmbus/vmbus_chanvar.h Thu Nov 24 05:52:28 2016 (r309082) +++ head/sys/dev/hyperv/vmbus/vmbus_chanvar.h Thu Nov 24 06:01:29 2016 (r309083) @@ -142,6 +142,8 @@ struct vmbus_channel { struct sx ch_orphan_lock; struct vmbus_xact_ctx *ch_orphan_xact; + int ch_refs; + struct sysctl_ctx_list ch_sysctl_ctx; } __aligned(CACHE_LINE_SIZE);