From owner-freebsd-arch@freebsd.org Fri Dec 16 04:34:54 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8F3B1C767C8 for ; Fri, 16 Dec 2016 04:34:54 +0000 (UTC) (envelope-from chmeeedalf@gmail.com) Received: from mail-it0-x242.google.com (mail-it0-x242.google.com [IPv6:2607:f8b0:4001:c0b::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 65C8A1011 for ; Fri, 16 Dec 2016 04:34:54 +0000 (UTC) (envelope-from chmeeedalf@gmail.com) Received: by mail-it0-x242.google.com with SMTP id c20so1301993itb.0 for ; Thu, 15 Dec 2016 20:34:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=cc:message-id:from:to:in-reply-to:content-transfer-encoding :mime-version:subject:date:references; bh=IHcz/Q/aic0P6czdPho1I4oh+RcVapQlQdWLcnoq9zk=; b=vMykM9O/FFrxXs2HWrHfjpT6UEMlSQtLp4ceIjmfBKr2n70dKsQj3VzttJCoojMW1G bHZoxAFn0jUydb0vAxPBczXIr34gr5ZW/59sXP+jh6Hq+4J7wlRBNcbMD6PQX4Fp/8Mr X7CwQ+mw48DPfIFeWgHYnXMFUEGMa+XTt2bWCpglD4JyC9R9+3KdsimWyHgfkxf2tBZU 98Q8rxnnWYJmCEnLLqBbvOA9I+CIMj0N1W1eK5pNva/SWnTeIl6Yvn9QF684JMgwveeW R2eUwTE/l3Rab94FyghWvskLsQsz6rgQCuKCji2RXYqwMqYmC8Be21kpx4fKIFPdGJPv cotg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:cc:message-id:from:to:in-reply-to :content-transfer-encoding:mime-version:subject:date:references; bh=IHcz/Q/aic0P6czdPho1I4oh+RcVapQlQdWLcnoq9zk=; b=mxzWVZbFreeG/iSJ1nuETUHo0oJk3SBoKnxuZZAcxnaI5jgbW+tEHK2jLeH/wFfZ0r VpShQMHFRi2T99T+ZlTW6a2ZBg5e3ruVHGVAtXpFfgVg7EuB1lh5bnYTDpITs1gpYchd 5+HnLTgL1oxbeR/vZrLz7Vtq7BMNx5cQAfyjxRZcOHzDs8VgGX/wETpfhM26uku4cqtN SqakIHqGCuIJrwQSGAPCCCMGDG6umLzXXxzTJxnF6UjcKDGrAjl8Fd9R4ceQV9sSFQXY /AmViYRpaEPwyjw2i8iJfcBs/lgTBp45CeTnR2ivzxNSOE0iBvL1h4xz3vFvh+/MJZn2 8fcQ== X-Gm-Message-State: AKaTC01QFuaRui8rAg9QIB/oT8i1klC9oSFUU9fwp+b2oLc69McQfI/TRZ3XCQSK2kOfzg== X-Received: by 10.36.68.146 with SMTP id o140mr1092800ita.33.1481862893574; Thu, 15 Dec 2016 20:34:53 -0800 (PST) Received: from blackstar.knownspace (50-80-150-234.client.mchsi.com. [50.80.150.234]) by smtp.gmail.com with ESMTPSA id u12sm763022ita.1.2016.12.15.20.34.52 (version=TLS1 cipher=AES128-SHA bits=128/128); Thu, 15 Dec 2016 20:34:52 -0800 (PST) Cc: FreeBSD Arch Message-Id: From: Justin Hibbits To: =?ISO-8859-1?Q?Roger_Pau_Monn=E9?= In-Reply-To: <7469755.xT5lfhErkd@ralph.baldwin.cx> Content-Type: text/plain; charset=ISO-8859-1; format=flowed; delsp=yes Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Apple Message framework v936) Subject: Re: Order of device suspend/resume Date: Thu, 15 Dec 2016 22:34:51 -0600 References: <20161215114033.r33nt3fqhnfi7hqw@dhcp-3-221.uk.xensource.com> <7469755.xT5lfhErkd@ralph.baldwin.cx> X-Mailer: Apple Mail (2.936) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2016 04:34:54 -0000 On Dec 15, 2016, at 3:38 PM, John Baldwin wrote: > On Thursday, December 15, 2016 11:40:33 AM Roger Pau Monn=E9 wrote: >> Hello, >> >> I'm currently dealing with a bug in the Xen suspend/resume =20 >> sequence, and I've >> found that lacking a way to order device priority during suspend/=20 >> resume is >> proving quite harmful for Xen (and maybe other systems too). The =20 >> current >> suspend/resume code simply scans the root bus, and suspends/resumes =20= >> every device >> based on the order they are attached to their parents. The problem =20= >> here is that >> there's no way to tell that some devices should be resumed before =20 >> others, for >> example the event timers/time counters/uarts should definitely be =20 >> resume before >> other devices, but that's seems to happens mostly out of chance. >> >> Currently most time related devices are attached directly to the =20 >> nexus, which >> means they will get resumed first, but for example the uart is =20 >> currently >> attached to the pci bus IIRC, which means it gets resumed quite =20 >> late. On Xen >> systems, this is even worse. The Xen PV bus (that contains all Xen-=20= >> related >> devices) is attached the last one (because it tends to pick up =20 >> unused memory >> regions for it's own usage) and this bus also contains the PV =20 >> timecounter which >> should be resumed _before_ other devices, or else timecounting will =20= >> be >> completely screwed and things can get stuck in indefinitely long =20 >> loops (due to >> the fact that the timecounter is implemented based on the uptime of =20= >> the host, >> and that changes from host-to-host). >> >> In order to solve this I could add a hack to the Xen resume process =20= >> (which is >> already different from the ACPI one), but this looks gross. I could =20= >> also attach >> the Xen PV timer to the nexus directly (as it was done before), but =20= >> I also >> prefer to keep all Xen-related devices in the same bus for =20 >> coherency. Last >> option would be to add some kind of suspend/resume priorities to =20 >> the devices, >> and do more than one suspend/resume pass. This is more complex and =20= >> requires more >> changes, so I would like to know if it would be helpful for other =20 >> systems, or if >> someone has already attempted to do it. > > I think Justin Hibbits had some patches to make use of the boot-time =20= > new-bus > passes for suspend and resume which I think would help with this. =20 > You suspend > things in the reverse order of boot and resume operates in the same =20= > order as > boot. > > --=20 > John Baldwin John is right. I have a (somewhat abandoned due to time and focus) =20 branch, https://svnweb.freebsd.org/base/projects/pmac_pmu/ which has =20 the necessary code working mostly on PowerPC. The diff can be found =20 at https://reviews.freebsd.org/D203 too. - Justin