From owner-cvs-src-old@FreeBSD.ORG Tue Dec 7 12:25:43 2010 Return-Path: Delivered-To: cvs-src-old@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3A03E1065675 for ; Tue, 7 Dec 2010 12:25:43 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from repoman.freebsd.org (repoman.freebsd.org [IPv6:2001:4f8:fff6::29]) by mx1.freebsd.org (Postfix) with ESMTP id 0E53D8FC21 for ; Tue, 7 Dec 2010 12:25:42 +0000 (UTC) Received: from repoman.freebsd.org (localhost [127.0.0.1]) by repoman.freebsd.org (8.14.4/8.14.4) with ESMTP id oB7CPgAB066908 for ; Tue, 7 Dec 2010 12:25:42 GMT (envelope-from avg@repoman.freebsd.org) Received: (from svn2cvs@localhost) by repoman.freebsd.org (8.14.4/8.14.4/Submit) id oB7CPgJW066907 for cvs-src-old@freebsd.org; Tue, 7 Dec 2010 12:25:42 GMT (envelope-from avg@repoman.freebsd.org) Message-Id: <201012071225.oB7CPgJW066907@repoman.freebsd.org> X-Authentication-Warning: repoman.freebsd.org: svn2cvs set sender to avg@repoman.freebsd.org using -f From: Andriy Gapon Date: Tue, 7 Dec 2010 12:25:26 +0000 (UTC) To: cvs-src-old@freebsd.org X-FreeBSD-CVS-Branch: HEAD Subject: cvs commit: src/sys/cddl/compat/opensolaris/sys cyclic_impl.h src/sys/cddl/dev/cyclic cyclic.c X-BeenThere: cvs-src-old@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: **OBSOLETE** CVS commit messages for the src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Dec 2010 12:25:43 -0000 avg 2010-12-07 12:25:26 UTC FreeBSD src repository Modified files: sys/cddl/compat/opensolaris/sys cyclic_impl.h sys/cddl/dev/cyclic cyclic.c Log: SVN rev 216254 on 2010-12-07 12:25:26Z by avg opensolaris cyclic: fix deadlock and make a little bit closer to upstream The dealock was caused in the following way: - thread T1 on CPU C1 holds a spin mutex, IPIs CPU C2 and waits for the IPI to be handled - C2 executes timer interrupt filter, thus has interrupts disabled, and gets blocked on the spin mutex held by T1 The problem seems to have been introduced by simplifications made to OpenSolaris code during porting. The problem is fixed by reorganizing the code to more closely resemble the upstream version. Interrupt filter (cyclic_fire) now doesn't acquire any locks, all per-CPU data accesses are performed on a target CPU with preemption and interrupts disabled thus precluding concurrent access to the data. cyp_mtx spin mutex is used to disable preemtion and interrupts; it's not used for classical mutual exclusion, because xcall already serializes calls to a CPU. It's an emulation of OpenSolaris cyb_set_level(CY_HIGH_LEVEL) call, the spin mutexes could probably be reduced to just a spinlock_enter()/_exit() pair. Diff with upstream version is now reduced by ~500 lines, however it still remains quite large - many things that are not needed (at the moment) or are irrelevant on FreeBSD were simply ripped out during porting. Examples of such things: - support for CPU onlining/offlining - support for suspend/resume - support for running callouts at soft interrupt levels - support for callout rebinding from CPU to CPU - support for CPU partitions Tested by: Artem Belevich MFC after: 3 weeks X-MFC with: r216252 Revision Changes Path 1.2 +8 -1 src/sys/cddl/compat/opensolaris/sys/cyclic_impl.h 1.4 +164 -175 src/sys/cddl/dev/cyclic/cyclic.c