From owner-svn-src-all@FreeBSD.ORG Tue Sep 20 23:44:35 2011 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6BDD1106564A; Tue, 20 Sep 2011 23:44:35 +0000 (UTC) (envelope-from gibbs@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 5A33A8FC08; Tue, 20 Sep 2011 23:44:35 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id p8KNiZFW049838; Tue, 20 Sep 2011 23:44:35 GMT (envelope-from gibbs@svn.freebsd.org) Received: (from gibbs@localhost) by svn.freebsd.org (8.14.4/8.14.4/Submit) id p8KNiZ9O049832; Tue, 20 Sep 2011 23:44:35 GMT (envelope-from gibbs@svn.freebsd.org) Message-Id: <201109202344.p8KNiZ9O049832@svn.freebsd.org> From: "Justin T. Gibbs" Date: Tue, 20 Sep 2011 23:44:35 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r225704 - in head/sys: dev/xen/control xen/xenbus xen/xenstore X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Sep 2011 23:44:35 -0000 Author: gibbs Date: Tue Sep 20 23:44:34 2011 New Revision: 225704 URL: http://svn.freebsd.org/changeset/base/225704 Log: Properly handle suspend/resume events in the Xen device framework. Sponsored by: BQ Internet sys/xen/xenbus/xenbusb.c: o In xenbusb_resume(), publish the state transition of the resuming device into XenbusStateIntiailising so that the remote peer can see it. Recording the state locally is not sufficient to trigger a re-connect sequence. o In xenbusb_resume(), defer new-bus resume processing until after the remote peer's XenStore address has been updated. The drivers may need to refer to this information during resume processing. sys/xen/xenbus/xenbusb_back.c: sys/xen/xenbus/xenbusb_front.c: Register xenbusb_resume() rather than bus_generic_resume() as the handler for device_resume events. sys/xen/xenstore/xenstore.c: o Fix grammer in a comment. o In xs_suspend(), pass suspend events on to the child devices (e.g. xenbusb_front/back, that are attached to the XenStore. Approved by: re MFC after: 1 week Modified: head/sys/dev/xen/control/control.c head/sys/xen/xenbus/xenbusb.c head/sys/xen/xenbus/xenbusb_back.c head/sys/xen/xenbus/xenbusb_front.c head/sys/xen/xenstore/xenstore.c Modified: head/sys/dev/xen/control/control.c ============================================================================== --- head/sys/dev/xen/control/control.c Tue Sep 20 23:22:58 2011 (r225703) +++ head/sys/dev/xen/control/control.c Tue Sep 20 23:44:34 2011 (r225704) @@ -115,6 +115,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -201,6 +202,8 @@ xctrl_suspend() int i, j, k, fpp; unsigned long max_pfn, start_info_mfn; + EVENTHANDLER_INVOKE(power_suspend); + #ifdef SMP struct thread *td; cpuset_t map; @@ -221,7 +224,13 @@ xctrl_suspend() stop_cpus(map); #endif + /* + * Be sure to hold Giant across DEVICE_SUSPEND/RESUME since non-MPSAFE + * drivers need this. + */ + mtx_lock(&Giant); if (DEVICE_SUSPEND(root_bus) != 0) { + mtx_unlock(&Giant); printf("xen_suspend: device_suspend failed\n"); #ifdef SMP if (!CPU_EMPTY(&map)) @@ -229,6 +238,7 @@ xctrl_suspend() #endif return; } + mtx_unlock(&Giant); local_irq_disable(); @@ -283,11 +293,14 @@ xctrl_suspend() vcpu_prepare(i); #endif + /* * Only resume xenbus /after/ we've prepared our VCPUs; otherwise * the VCPU hotplug callback can race with our vcpu_prepare */ + mtx_lock(&Giant); DEVICE_RESUME(root_bus); + mtx_unlock(&Giant); #ifdef SMP thread_lock(curthread); @@ -296,6 +309,7 @@ xctrl_suspend() if (!CPU_EMPTY(&map)) restart_cpus(map); #endif + EVENTHANDLER_INVOKE(power_resume); } static void @@ -322,39 +336,47 @@ xctrl_suspend() { int suspend_cancelled; + EVENTHANDLER_INVOKE(power_suspend); + + /* + * Be sure to hold Giant across DEVICE_SUSPEND/RESUME since non-MPSAFE + * drivers need this. + */ + mtx_lock(&Giant); if (DEVICE_SUSPEND(root_bus)) { + mtx_unlock(&Giant); printf("xen_suspend: device_suspend failed\n"); return; } - - /* - * Make sure we don't change cpus or switch to some other - * thread. for the duration. - */ - critical_enter(); + mtx_unlock(&Giant); /* * Prevent any races with evtchn_interrupt() handler. */ - irq_suspend(); disable_intr(); + irq_suspend(); suspend_cancelled = HYPERVISOR_suspend(0); - if (!suspend_cancelled) + if (suspend_cancelled) + irq_resume(); + else xenpci_resume(); /* * Re-enable interrupts and put the scheduler back to normal. */ enable_intr(); - critical_exit(); /* * FreeBSD really needs to add DEVICE_SUSPEND_CANCEL or * similar. */ + mtx_lock(&Giant); if (!suspend_cancelled) DEVICE_RESUME(root_bus); + mtx_unlock(&Giant); + + EVENTHANDLER_INVOKE(power_resume); } #endif Modified: head/sys/xen/xenbus/xenbusb.c ============================================================================== --- head/sys/xen/xenbus/xenbusb.c Tue Sep 20 23:22:58 2011 (r225703) +++ head/sys/xen/xenbus/xenbusb.c Tue Sep 20 23:44:34 2011 (r225704) @@ -773,7 +773,7 @@ xenbusb_resume(device_t dev) ivars = device_get_ivars(kids[i]); xs_unregister_watch(&ivars->xd_otherend_watch); - ivars->xd_state = XenbusStateInitialising; + xenbus_set_state(kids[i], XenbusStateInitialising); /* * Find the new backend details and @@ -783,16 +783,16 @@ xenbusb_resume(device_t dev) if (error) return (error); - DEVICE_RESUME(kids[i]); - statepath = malloc(ivars->xd_otherend_path_len + strlen("/state") + 1, M_XENBUS, M_WAITOK); sprintf(statepath, "%s/state", ivars->xd_otherend_path); free(ivars->xd_otherend_watch.node, M_XENBUS); ivars->xd_otherend_watch.node = statepath; - xs_register_watch(&ivars->xd_otherend_watch); + DEVICE_RESUME(kids[i]); + + xs_register_watch(&ivars->xd_otherend_watch); #if 0 /* * Can't do this yet since we are running in Modified: head/sys/xen/xenbus/xenbusb_back.c ============================================================================== --- head/sys/xen/xenbus/xenbusb_back.c Tue Sep 20 23:22:58 2011 (r225703) +++ head/sys/xen/xenbus/xenbusb_back.c Tue Sep 20 23:44:34 2011 (r225704) @@ -292,7 +292,7 @@ static device_method_t xenbusb_back_meth DEVMETHOD(device_detach, bus_generic_detach), DEVMETHOD(device_shutdown, bus_generic_shutdown), DEVMETHOD(device_suspend, bus_generic_suspend), - DEVMETHOD(device_resume, bus_generic_resume), + DEVMETHOD(device_resume, xenbusb_resume), /* Bus Interface */ DEVMETHOD(bus_print_child, xenbusb_print_child), Modified: head/sys/xen/xenbus/xenbusb_front.c ============================================================================== --- head/sys/xen/xenbus/xenbusb_front.c Tue Sep 20 23:22:58 2011 (r225703) +++ head/sys/xen/xenbus/xenbusb_front.c Tue Sep 20 23:44:34 2011 (r225704) @@ -171,7 +171,7 @@ static device_method_t xenbusb_front_met DEVMETHOD(device_detach, bus_generic_detach), DEVMETHOD(device_shutdown, bus_generic_shutdown), DEVMETHOD(device_suspend, bus_generic_suspend), - DEVMETHOD(device_resume, bus_generic_resume), + DEVMETHOD(device_resume, xenbusb_resume), /* Bus Interface */ DEVMETHOD(bus_print_child, xenbusb_print_child), Modified: head/sys/xen/xenstore/xenstore.c ============================================================================== --- head/sys/xen/xenstore/xenstore.c Tue Sep 20 23:22:58 2011 (r225703) +++ head/sys/xen/xenstore/xenstore.c Tue Sep 20 23:44:34 2011 (r225704) @@ -721,8 +721,8 @@ xs_reply_filter(uint32_t request_msg_typ /* * The count of transactions drops if we attempted * to end a transaction (even if that attempt fails - * in error), we receive a transaction end acknowledgement - * or if our attempt to begin a transactionfails. + * in error), we receive a transaction end acknowledgement, + * or if our attempt to begin a transaction fails. */ if (request_msg_type == XS_TRANSACTION_END || (request_reply_error == 0 && reply_msg_type == XS_TRANSACTION_END) @@ -1194,8 +1194,14 @@ xs_attach(device_t dev) * all transactions and individual requests have completed. */ static int -xs_suspend(device_t dev __unused) +xs_suspend(device_t dev) { + int error; + + /* Suspend child Xen devices. */ + error = bus_generic_suspend(dev); + if (error != 0) + return (error); sx_xlock(&xs.suspend_mutex); sx_xlock(&xs.request_mutex); @@ -1227,6 +1233,9 @@ xs_resume(device_t dev __unused) sx_xunlock(&xs.suspend_mutex); + /* Resume child Xen devices. */ + bus_generic_resume(dev); + return (0); }