From owner-freebsd-current@FreeBSD.ORG Thu Aug 26 22:24:39 2010 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 97C8C10656A9 for ; Thu, 26 Aug 2010 22:24:39 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-ew0-f54.google.com (mail-ew0-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id 271908FC13 for ; Thu, 26 Aug 2010 22:24:38 +0000 (UTC) Received: by ewy4 with SMTP id 4so1827020ewy.13 for ; Thu, 26 Aug 2010 15:24:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=AFvfxNmUuCyjItBzkqTouJJBJdEG6egXItEN1Lan0L4=; b=shL/D3EWfkK487rhdxYhBK+cuJtSsJVo/oR97AnVJbjORkwB7/NSryVfeYMhGVDbXg 3N4Dv+UJ5dAVSNCshMUS59FXkwdar2K6RRlYbCxBfqzVwo1VhpxyYEwvWwh3temzb9NR fcG/Li3H+m9VflQB/NaK24IVecA84V065VuT8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=TnJLpKgOIQRh184K0Cl6+Jg7HaTNSsRnVSquZWxG3UJ0LyB0+ptcvsvY6o/ZuAnxr9 66Z6zUUV1Ftet/0Fl9ahOjTuLoHXZRFA7Iobieb/83RVDz1dErKmxYMVn80yyM+a5paK BGr00Gz+b/Osytp2ov47hd25LD+jN0nKSuvHY= MIME-Version: 1.0 Received: by 10.213.47.76 with SMTP id m12mr1453405ebf.43.1282861477922; Thu, 26 Aug 2010 15:24:37 -0700 (PDT) Sender: mdf356@gmail.com Received: by 10.213.20.144 with HTTP; Thu, 26 Aug 2010 15:24:37 -0700 (PDT) In-Reply-To: <4C76E661.60600@icyb.net.ua> References: <201008261649.20543.jhb@freebsd.org> <4C76E661.60600@icyb.net.ua> Date: Thu, 26 Aug 2010 15:24:37 -0700 X-Google-Sender-Auth: fDOKo6L0PnjExsqxxCWDHpLmWig Message-ID: From: mdf@FreeBSD.org To: Andriy Gapon Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-current@freebsd.org Subject: Re: sched_pin() bug in SCHED_ULE X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Aug 2010 22:24:39 -0000 On Thu, Aug 26, 2010 at 3:10 PM, Andriy Gapon wrote: > on 27/08/2010 00:20 mdf@FreeBSD.org said the following: >> >> I tried making sched_pin() a real function which used >> intr_disable/intr_restore around saving off td->td_oncpu, >> td->td_lastcpu and ts->ts_cpu, and the stack at the time of call. =A0In >> sched_switch when I saw an unexpected migration I printed all that >> out. =A0I found that on my boxes, at sched_pin() time ts_cpu was already >> different from td->td_oncpu, so the specific problem I was having was >> that while another thread can change ts_cpu it has no way to force >> that thread to immediately migrate. > > Like e.g. in sched_affinity where ts_cpu is first changed and then the ol= d cpu > is ipi-ed? That could do it. I didn't follow the stack of the places that were touching ts_cpu for non-curthread. >> I believe the below patch fixes the issue, though I'm open to other meth= ods: >> >> >> Index: kern/sched_ule.c >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> --- kern/sched_ule.c =A0(.../head/src/sys) =A0 =A0 =A0(revision 158279) >> +++ kern/sched_ule.c =A0(.../branches/BR_BUG_67957/src/sys) =A0 =A0 (rev= ision 158279) >> @@ -112,6 +112,7 @@ >> =A0/* flags kept in ts_flags */ >> =A0#define =A0 =A0 =A0TSF_BOUND =A0 =A0 =A0 0x0001 =A0 =A0 =A0 =A0 =A0/*= Thread can not migrate. */ >> =A0#define =A0 =A0 =A0TSF_XFERABLE =A0 =A00x0002 =A0 =A0 =A0 =A0 =A0/* T= hread was added as transferable. */ >> +#define =A0 =A0 =A0TSF_BINDING =A0 =A0 0x0004 =A0 =A0 =A0 =A0 =A0/* Thr= ead is being bound. */ > > I don't really follow why TSF_BINDING is needed, i.e. why TSF_BOUND is no= t > sufficient in this case? I wanted to tighten up the asserts, so that the only time it was okay for a td_pinned thread to migrate was the one time it was moving to the target cpu. Having a separate flag allows that. Thanks, matthew