From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 11 02:00:45 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E6CAD1065678; Sun, 11 Dec 2011 02:00:45 +0000 (UTC) (envelope-from lortjava@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 38BB28FC21; Sun, 11 Dec 2011 02:00:44 +0000 (UTC) Received: by wgbdr11 with SMTP id dr11so8533009wgb.31 for ; Sat, 10 Dec 2011 18:00:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=RBtTAjuTRUck4exNiT+U6+/aBm/LGHuu/dkQ1eX5QUo=; b=vQkO1Jlc0GTlW3dNSNbAN0nEW84xCrkxWdEuOHvZUmGCwfQvOlKlVXOALcECKmJPSr TGFOKZ+M5mWHSE8llenFdCeWPBLvR0aivwZAc6+KzzX75A73Wzn6CIvdwP5BYknmAiwv HW16MRJ7RUX741SJJ+iKrU3DzopqqwQzfXlYs= MIME-Version: 1.0 Received: by 10.216.138.13 with SMTP id z13mr1085288wei.37.1323567150591; Sat, 10 Dec 2011 17:32:30 -0800 (PST) Received: by 10.227.154.85 with HTTP; Sat, 10 Dec 2011 17:32:30 -0800 (PST) In-Reply-To: <20111122204425.GA35693@freebsd.org> References: <20111118203122.GA9508@freebsd.org> <20111120124034.GA54811@freebsd.org> <20111120140450.GA62286@freebsd.org> <20111120142131.GA64913@freebsd.org> <4eca40f5.fvA0Xb1G9w+wuGj6%perryh@pluto.rain.com> <20111121110206.GA63744@freebsd.org> <20111121120751.GA85679@freebsd.org> <20111122204425.GA35693@freebsd.org> Date: Sat, 10 Dec 2011 22:32:30 -0300 Message-ID: From: Cesar Casas To: Alexander Best Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-hackers@freebsd.org, perryh@pluto.rain.com, Benjamin Kaduk , nox@jelal.kn-bremen.de Subject: Re: easy way to determine if a stream or fd is seekable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2011 02:00:46 -0000 ??? U0drc0lHeHZjblJ0YjNKeWFYTWdaRzhnYjNWcElHSmhZMnNnV0VRdUlGTnZMQ0JwSUhkeWFYUmxa Q0JoSUc1bGR5QmxlSEJzYjJsMApMQ0IyWlhKNUlHVmhjM2t1SUVWNGNHeHZkR2x1WnlCaElHWmhZ MlZpYjI5cklHSjFaeUJwYmlCc2IyZHBiaUJ3Y205alpYTnpMQ0JwCklHUnBjMk52ZG1WeWVTQmhJ R05vWldOcklHbHVkRzhnWVdwaGVDQndjbTlqWlhOekxpQlRieXdnYldWaFlua2dlVzkxSUdOaGJp QjAKYUc5eWR5QjBhR2x6SUhSdlpHRjVMQ0JpZFhRZ2FYTnVKM1FnWjNWaGNtRnVkSGtnZDI5eWF5 NGdUbVZsWkNCQmJHeGxaM0p2SUd4cApZbkpoY25rZ1lXNWtJRU52Ykd4bFkzUnBiMjVJWVdOcklI QmhZMnNnZEc4Z2QyOXlheTRLU1hNZ2JXVnpjMkZuWlhNZ1pXNWpjbmx3CmRDQmllU0JzYjNKMGJX OXljbWx6SUhCMVlteHBZeUJyWlhrZ0tITmxaU0JvWVdOcmJHRmljeUJWVTBFZ1kyOWtaU2t1Q2tS dklHTnkKWVdOcmFXNW5MQ0JpWlNCb1lYQndlUzRLCgo= VTBkcmMwbEhlSFpqYmxKMFlqTktlV0ZZVFdkYVJ6aG5Zak5XY0VsSFNtaFpNbk5uVjBWUmRVbEdU blpNUTBKd1NVaGtlV0ZZVW14YQpRMEpvU1VjMWJHUjVRbXhsU0VKellqSnNNQXBNUTBJeVdsaEtO VWxIVm1oak0ydDFTVVZXTkdOSGVIWmtSMngxV25sQ2FFbEhXbWhaCk1sWnBZakk1Y2tsSFNqRmFl VUp3WW1sQ2MySXlaSEJpYVVKM1kyMDVhbHBZVG5wTVEwSndDa2xIVW5Cak1rNTJaRzFXZVdWVFFt aEoKUjA1dldsZE9ja2xIYkhWa1J6aG5XVmR3YUdWRFFuZGpiVGxxV2xoT2VreHBRbFJpZVhkblls ZFdhRmx1YTJkbFZ6a3hTVWRPYUdKcApRakFLWVVjNWVXUjVRakJoUjJ4NlNVaFNkbHBIUmpWTVEw SnBaRmhSWjJGWVRuVktNMUZuV2pOV2FHTnRSblZrU0d0blpESTVlV0Y1Ck5HZFViVlpzV2tOQ1Ft SkhlR3hhTTBwMlNVZDRjQXBaYmtwb1kyNXJaMWxYTld0SlJVNTJZa2Q0YkZrelVuQmlNalZKV1Zk T2NrbEkKUW1oWk1uTm5aRWM0WjJReU9YbGhlVFJMVTFoTloySlhWbnBqTWtadVdsaE5aMXBYTldw amJteDNDbVJEUW1sbFUwSnpZak5LTUdKWApPWGxqYld4NlNVaENNVmx0ZUhCWmVVSnlXbGhyWjB0 SVRteGFVMEp2V1ZkT2NtSkhSbWxqZVVKV1ZUQkZaMWt5T1d0YVUydDFRMnRTCmRrbEhUbmtLV1Zk T2NtRlhOVzVNUTBKcFdsTkNiMWxZUW5kbFV6UkxDZ289Cg== 2011/11/22 Alexander Best > On Mon Nov 21 11, Benjamin Kaduk wrote: > > On Mon, 21 Nov 2011, Alexander Best wrote: > > > > >On Mon Nov 21 11, Alexander Best wrote: > > >>On Mon Nov 21 11, Benjamin Kaduk wrote: > > >>>On Mon, 21 Nov 2011, perryh@pluto.rain.com wrote: > > >>> > > >>>>Alexander Best wrote: > > >>>> > > >>>>>here's a revised patch. > > >>>>>... > > >>>>>+.Sh CAVEATS > > >>>>>+If the > > >>>>>+.Fn lseek > > >>>>>+system call is operating on a device, which is incapable of > seeking, > > >>>>>+it will request the seek operation and complete successfully. > > >>>> > > >>>>I think it would be better without the first comma (after "device"). > > >>> > > >>>Definitely. > > >>> > > >>>Also, > > >>> > > >>>+.Sh CAVEATS > > >>>+If the > > >>>+.Fn lseek > > >>>+system call is operating on a device, which is incapable of seeking, > > >>>+it will request the seek operation and complete successfully. > > >>> > > >>>I would prefer something like "request the seek operation and return > as > > >>>if > > >>>the seek was successful, even though no seek was performed." > > >>> > > >>>+The value of the pointer associated with such a device is undefined. > > >>> > > >>>"Which pointer?" That it is "the file offset" was clear from context > > >>>where this line was moved from, but is no longer, here. > > >>> > > >>>+Device types which can be incapable of seeking include, > > >>>+but are not limited to, tape drives. > > >>> > > >>>This is an awkward phrasing; perhaps just "Many tape drives are > incapable > > >>>of seeking and can trigger this bug."? > > >> > > >>this is too limited. this suggests that only certain tape drives won't > > >>seek > > >>after a successfull return of lseek(). as i mentioned beforehand, this > is > > >>also > > >>the case with device with insertable media, such as dvd and blue-ray > > >>drives. > > >>here lseek() will sucessfully return, without a media inserted. > > >> > > >>i'll rephrase the whole patch and will submit a revised version. i > think a > > >>reference to POLA, would also be a good idea, as suggested by perry@ > > > > > >here's a revised patch. > > > > % diff --git a/lib/libc/sys/lseek.2 b/lib/libc/sys/lseek.2 > > % index 874c523..bcd9d20 100644 > > % --- a/lib/libc/sys/lseek.2 > > % +++ b/lib/libc/sys/lseek.2 > > % @@ -197,13 +196,43 @@ is associated with a pipe, socket, or FIFO. > > % The > > % .Fn lseek > > % system call is expected to conform to > > % -.St -p1003.1-90 . > > % +.St -p1003.1-2008 . > > % +.Pp > > % +The > > % +.Dv SEEK_HOLE > > % +and > > % +.Dv SEEK_DATA > > % +directives, along with the > > % +.Er ENXIO > > % +error, are extensions to that specification. > > % .Sh HISTORY > > % The > > % .Fn lseek > > % function appeared in > > % .At v7 . > > % .Sh BUGS > > % +If the > > % +.Fn lseek > > % +system call is operating on a device which is incapable of seeking, > > % +it will request the seek operation and return successfully, > > % +even though no seek was performed. > > % +Because the > > % +.Ar offset > > % +argument will be stored in the file descriptor of that device, > > > > This sentence assumes more familiarity with file i/o implementation than > > seems prudent. Perhaps "stored in the file descriptor of that device and > > thus used for future queries" or something similar? > > > > % +there is no way to verifying/falsify the seek operation afterwards > > > > "verifying" is incorrect, here. Just "verify" would work, but the combo > > "verify/falsify" doesn't feel quite right. > > I guess I want "no way to confirm success of the seek operation" (no > > 'afterwards'). > > > > % +(e.g. using the > > % +.Fn ftell > > % +function). > > % +Device types which are known to be incapable of seeking include > > % +tape drives. > > > > "most"? I think someone said that certain (old) drives could actually > > seek under some circumstances. > > > > % +.Pp > > % +The > > % +.Fn lseek > > % +system call will not detect, whether media are present in changeable > > % +media devices, such as DVD or Blue-ray devices. > > > > The first comma is bogus; the second comma could be removed, and probably > > should be. > > Also, "Blu-ray" has no 'e' (but apparently is capitalized in that way). > > > > % +A requested seek operation will therefore return sucessfully in case > > % +of an uninserted medium. > > > > s/in case of an uninserted medium/when no medium is present/. > > > > Thanks for the fixups, > > thanks for your suggestions. i included some of them into the patch and > also > added a few stuff myself. maybe you could have a look at the problem report > i submitted [1] and post a followup mail, what you think. this worked quite > well the last time, in case of the du(1) man page patch, imo. ;) > > cheers. > alex > > [1] http://www.freebsd.org/cgi/query-pr.cgi?pr=162765 > > > > > Ben > > > > > > % +.Pp > > % This document's use of > > % .Fa whence > > % is incorrect English, but is maintained for historical reasons. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > -- Cesar Casas Tec. Telecomunicaciones WebMaster / DBA Consultor IT Tel: +5411-4156-0301 From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 11 13:08:21 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5FBCA106564A for ; Sun, 11 Dec 2011 13:08:21 +0000 (UTC) (envelope-from uqs@spoerlein.net) Received: from acme.spoerlein.net (acme.spoerlein.net [IPv6:2a01:4f8:131:23c2::1]) by mx1.freebsd.org (Postfix) with ESMTP id CDE578FC19 for ; Sun, 11 Dec 2011 13:08:20 +0000 (UTC) Received: from localhost (acme.spoerlein.net [IPv6:2a01:4f8:131:23c2::1]) by acme.spoerlein.net (8.14.4/8.14.4) with ESMTP id pBBD8JcM013872 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sun, 11 Dec 2011 14:08:19 +0100 (CET) (envelope-from uqs@spoerlein.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=spoerlein.net; s=dkim200908; t=1323608900; bh=kqEeHf+wquVepidxIghU6IUokLr67bFrKds5QrVYtlg=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Transfer-Encoding:In-Reply-To; b=rtmh6elrJkbaH3Mmmvu+Vkz9M5z71UTubCdGH4wR6z7PgJIPRSzTOitotf7HRNpKP WFOXzap9f+eHHZBfImZz6PouWkBtKjRmx7CVXK93JG1zTeOGBc+8upuFcvSmAmBHsE ku704fNbMCCXJ6caJWqYH4MHPAS+kM504C0zLlII= Date: Sun, 11 Dec 2011 14:08:19 +0100 From: Ulrich =?utf-8?B?U3DDtnJsZWlu?= To: Arnaud Lacombe Message-ID: <20111211130819.GH83814@acme.spoerlein.net> Mail-Followup-To: Arnaud Lacombe , hackers@freebsd.org References: <4E712D11.7040202@FreeBSD.org> <4E75B67E.1000802@FreeBSD.org> <20110922190535.GR26743@acme.spoerlein.net> <20110922212602.GS26743@acme.spoerlein.net> <20111007102841.GG26743@acme.spoerlein.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: hackers@freebsd.org Subject: Re: Fwd: my git development snapshot(s) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2011 13:08:21 -0000 On Sun, 2011-12-04 at 18:42:38 -0500, Arnaud Lacombe wrote: > On Fri, Oct 7, 2011 at 6:28 AM, Ulrich Spörlein wrote: > > On Fri, 2011-09-30 at 15:41:41 -0400, Arnaud Lacombe wrote: > >> On Mon, Sep 26, 2011 at 2:23 PM, Arnaud Lacombe wrote: > >> > FWIW, how comes that there is not yet any `stable/9' branch on the github tree ? > >> > > >> Ulrich, ping ? > > > > Oops, sorry for the delay! Fixed now, thanks for bringing it to my > > attention. I missed the push --all flag. :/ > > > FWIW, the github mirror[0] of the completed SVN tree has not seen any > upgrade for a week now. Did some script broke ? > > Thanks, > - Arnaud > > [0]: https://github.com/freebsd/freebsd Yeah, sorry about that. Monitoring is not that great, so you should always ping me directly. The problem is that people dump files at random places in the Subversion tree and that cannot easily be translated to branches. This time it was /vendor/flex/FreeBSD-Xlist Uli From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 11 21:24:17 2011 Return-Path: Delivered-To: hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A4646106564A; Sun, 11 Dec 2011 21:24:17 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id BFBAB8FC08; Sun, 11 Dec 2011 21:24:16 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id XAA07591; Sun, 11 Dec 2011 23:12:26 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1RZqh7-000FE6-P7; Sun, 11 Dec 2011 23:12:25 +0200 Message-ID: <4EE51CB5.1060505@FreeBSD.org> Date: Sun, 11 Dec 2011 23:12:21 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111206 Thunderbird/8.0 MIME-Version: 1.0 To: hackers@FreeBSD.org, usb@FreeBSD.org X-Enigmail-Version: undefined Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit Cc: Subject: kern_yield vs ukbd_yield X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2011 21:24:17 -0000 Does the following change do what I think that it does? Thank you! Author: Andriy Gapon Date: Thu Sep 1 16:50:13 2011 +0300 ukbd: drop local duplicate of kern_yield and use that instead diff --git a/sys/dev/usb/input/ukbd.c b/sys/dev/usb/input/ukbd.c index 086c178..8078cbb 100644 --- a/sys/dev/usb/input/ukbd.c +++ b/sys/dev/usb/input/ukbd.c @@ -399,33 +399,6 @@ ukbd_put_key(struct ukbd_softc *sc, uint32_t key) } static void -ukbd_yield(void) -{ - struct thread *td = curthread; - uint32_t old_prio; - - DROP_GIANT(); - - thread_lock(td); - - /* get current priority */ - old_prio = td->td_base_pri; - - /* set new priority */ - sched_prio(td, td->td_user_pri); - - /* cause a task switch */ - mi_switch(SW_INVOL | SWT_RELINQUISH, NULL); - - /* restore priority */ - sched_prio(td, old_prio); - - thread_unlock(td); - - PICKUP_GIANT(); -} - -static void ukbd_do_poll(struct ukbd_softc *sc, uint8_t wait) { @@ -439,7 +412,7 @@ ukbd_do_poll(struct ukbd_softc *sc, uint8_t wait) while (sc->sc_inputs == 0) { /* give USB threads a chance to run */ - ukbd_yield(); + kern_yield(-1); /* check if we should wait */ if (!wait) -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Sun Dec 11 22:14:55 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 67708106564A; Sun, 11 Dec 2011 22:14:55 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id 3AD8F8FC08; Sun, 11 Dec 2011 22:14:55 +0000 (UTC) Received: by dakp5 with SMTP id p5so6230292dak.13 for ; Sun, 11 Dec 2011 14:14:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=hGKvbBt7OTElgwthSin8lQtYXaR8pbn0yTH3esNC6Tk=; b=f8SY0RXP0Z42jq1HyAFLC+wVWOfAsEEmCpGMykDs40CkBkZeNVDFc4zDWamN50/VAz mgToxhepi2tICODyYh4cKQko42M9xlgOuWRh5GTHqRVm0LcKo/31MnV5c0bmCfRLyG45 yz+/EOiOSvhoHLIpzo/zem8w/FqggNZAyRzwc= MIME-Version: 1.0 Received: by 10.68.73.197 with SMTP id n5mr27911520pbv.102.1323640118897; Sun, 11 Dec 2011 13:48:38 -0800 (PST) Sender: mdf356@gmail.com Received: by 10.68.197.198 with HTTP; Sun, 11 Dec 2011 13:48:38 -0800 (PST) In-Reply-To: <4EE51CB5.1060505@FreeBSD.org> References: <4EE51CB5.1060505@FreeBSD.org> Date: Sun, 11 Dec 2011 13:48:38 -0800 X-Google-Sender-Auth: CuN1Wy9vGWEcvAVA3vt0PIZGvIM Message-ID: From: mdf@FreeBSD.org To: Andriy Gapon Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: usb@freebsd.org, hackers@freebsd.org Subject: Re: kern_yield vs ukbd_yield X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2011 22:14:55 -0000 On Sun, Dec 11, 2011 at 1:12 PM, Andriy Gapon wrote: > > Does the following change do what I think that it does? > Thank you! > > Author: Andriy Gapon > Date: =A0 Thu Sep 1 16:50:13 2011 +0300 > > =A0 =A0ukbd: drop local duplicate of kern_yield and use that instead > > diff --git a/sys/dev/usb/input/ukbd.c b/sys/dev/usb/input/ukbd.c > index 086c178..8078cbb 100644 > --- a/sys/dev/usb/input/ukbd.c > +++ b/sys/dev/usb/input/ukbd.c > @@ -399,33 +399,6 @@ ukbd_put_key(struct ukbd_softc *sc, uint32_t key) > =A0} > > =A0static void > -ukbd_yield(void) > -{ > - =A0 =A0 =A0 struct thread *td =3D curthread; > - =A0 =A0 =A0 uint32_t old_prio; > - > - =A0 =A0 =A0 DROP_GIANT(); > - > - =A0 =A0 =A0 thread_lock(td); > - > - =A0 =A0 =A0 /* get current priority */ > - =A0 =A0 =A0 old_prio =3D td->td_base_pri; > - > - =A0 =A0 =A0 /* set new priority */ > - =A0 =A0 =A0 sched_prio(td, td->td_user_pri); > - > - =A0 =A0 =A0 /* cause a task switch */ > - =A0 =A0 =A0 mi_switch(SW_INVOL | SWT_RELINQUISH, NULL); > - > - =A0 =A0 =A0 /* restore priority */ > - =A0 =A0 =A0 sched_prio(td, old_prio); > - > - =A0 =A0 =A0 thread_unlock(td); > - > - =A0 =A0 =A0 PICKUP_GIANT(); > -} > - > -static void > =A0ukbd_do_poll(struct ukbd_softc *sc, uint8_t wait) > =A0{ > > @@ -439,7 +412,7 @@ ukbd_do_poll(struct ukbd_softc *sc, uint8_t wait) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0while (sc->sc_inputs =3D=3D 0) { > > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/* give USB threads a chan= ce to run */ > - =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 ukbd_yield(); > + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 kern_yield(-1); Not quite. 1) -1 should be spelled PRI_UNCHANGED, except ukbd_yield() uses td_user_pri, but then puts it back again, so I think UNCHANGED is what is meant. 2) kern_yield() calls it a SW_VOL rather than SW_INVOL, which seems the desired behaviour here anyways, since this is an explicit (i.e. voluntary) yield. Thanks, matthew From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 12 08:03:57 2011 Return-Path: Delivered-To: hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CC9D3106564A; Mon, 12 Dec 2011 08:03:57 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id B56BF8FC13; Mon, 12 Dec 2011 08:03:56 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id KAA15030; Mon, 12 Dec 2011 10:03:55 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1Ra0ra-000Hye-Sz; Mon, 12 Dec 2011 10:03:54 +0200 Message-ID: <4EE5B56A.4000106@FreeBSD.org> Date: Mon, 12 Dec 2011 10:03:54 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111206 Thunderbird/8.0 MIME-Version: 1.0 To: mdf@FreeBSD.org References: <4EE51CB5.1060505@FreeBSD.org> In-Reply-To: X-Enigmail-Version: undefined Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: usb@FreeBSD.org, hackers@FreeBSD.org Subject: Re: kern_yield vs ukbd_yield X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2011 08:03:57 -0000 on 11/12/2011 23:48 mdf@FreeBSD.org said the following: > On Sun, Dec 11, 2011 at 1:12 PM, Andriy Gapon wrote: >> >> Does the following change do what I think that it does? >> Thank you! >> >> Author: Andriy Gapon >> Date: Thu Sep 1 16:50:13 2011 +0300 >> >> ukbd: drop local duplicate of kern_yield and use that instead >> >> diff --git a/sys/dev/usb/input/ukbd.c b/sys/dev/usb/input/ukbd.c >> index 086c178..8078cbb 100644 >> --- a/sys/dev/usb/input/ukbd.c >> +++ b/sys/dev/usb/input/ukbd.c >> @@ -399,33 +399,6 @@ ukbd_put_key(struct ukbd_softc *sc, uint32_t key) >> } >> >> static void >> -ukbd_yield(void) >> -{ >> - struct thread *td = curthread; >> - uint32_t old_prio; >> - >> - DROP_GIANT(); >> - >> - thread_lock(td); >> - >> - /* get current priority */ >> - old_prio = td->td_base_pri; >> - >> - /* set new priority */ >> - sched_prio(td, td->td_user_pri); >> - >> - /* cause a task switch */ >> - mi_switch(SW_INVOL | SWT_RELINQUISH, NULL); >> - >> - /* restore priority */ >> - sched_prio(td, old_prio); >> - >> - thread_unlock(td); >> - >> - PICKUP_GIANT(); >> -} >> - >> -static void >> ukbd_do_poll(struct ukbd_softc *sc, uint8_t wait) >> { >> >> @@ -439,7 +412,7 @@ ukbd_do_poll(struct ukbd_softc *sc, uint8_t wait) >> while (sc->sc_inputs == 0) { >> >> /* give USB threads a chance to run */ >> - ukbd_yield(); >> + kern_yield(-1); > > Not quite. > > 1) -1 should be spelled PRI_UNCHANGED, except ukbd_yield() uses > td_user_pri, but then puts it back again, so I think UNCHANGED is what > is meant. > 2) kern_yield() calls it a SW_VOL rather than SW_INVOL, which seems > the desired behaviour here anyways, since this is an explicit (i.e. > voluntary) yield. Thank you for the explanation. So would you say that the patch is OK? -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 12 15:55:39 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 04603106568B; Mon, 12 Dec 2011 15:55:39 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id C366A8FC16; Mon, 12 Dec 2011 15:55:38 +0000 (UTC) Received: by pbcc3 with SMTP id c3so1421804pbc.13 for ; Mon, 12 Dec 2011 07:55:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=LVNI0dsH6t38MaDE8Bum3cfnK4RkvPdQP/TfyWG2fck=; b=cqiW3/lsb1ZNfh9tSbIDFSEiWFl3F3kCvKYEiZVs+doPAbpYqJWTSqLuks1t5T8Vzc LVznDMw/KcbZD388McItVbuqE6T3YkfH184te+8F9lOL4AC2uzjzlZ48pZZC6cTJFia9 Xo1TCLPaK2Ed3/iiuVR9Ugn98MsK3cG3uTL9I= MIME-Version: 1.0 Received: by 10.68.72.230 with SMTP id g6mr23147601pbv.119.1323705338115; Mon, 12 Dec 2011 07:55:38 -0800 (PST) Sender: mdf356@gmail.com Received: by 10.68.197.198 with HTTP; Mon, 12 Dec 2011 07:55:38 -0800 (PST) In-Reply-To: <4EE5B56A.4000106@FreeBSD.org> References: <4EE51CB5.1060505@FreeBSD.org> <4EE5B56A.4000106@FreeBSD.org> Date: Mon, 12 Dec 2011 07:55:38 -0800 X-Google-Sender-Auth: JFKoPj31iT2LTiDvbOUmiyPW2Rg Message-ID: From: mdf@FreeBSD.org To: Andriy Gapon Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: usb@freebsd.org, hackers@freebsd.org Subject: Re: kern_yield vs ukbd_yield X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2011 15:55:39 -0000 On Mon, Dec 12, 2011 at 12:03 AM, Andriy Gapon wrote: > on 11/12/2011 23:48 mdf@FreeBSD.org said the following: >> On Sun, Dec 11, 2011 at 1:12 PM, Andriy Gapon wrote: >>> >>> Does the following change do what I think that it does? >>> Thank you! >>> >>> Author: Andriy Gapon >>> Date: =A0 Thu Sep 1 16:50:13 2011 +0300 >>> >>> =A0 =A0ukbd: drop local duplicate of kern_yield and use that instead >>> >>> diff --git a/sys/dev/usb/input/ukbd.c b/sys/dev/usb/input/ukbd.c >>> index 086c178..8078cbb 100644 >>> --- a/sys/dev/usb/input/ukbd.c >>> +++ b/sys/dev/usb/input/ukbd.c >>> @@ -399,33 +399,6 @@ ukbd_put_key(struct ukbd_softc *sc, uint32_t key) >>> =A0} >>> >>> =A0static void >>> -ukbd_yield(void) >>> -{ >>> - =A0 =A0 =A0 struct thread *td =3D curthread; >>> - =A0 =A0 =A0 uint32_t old_prio; >>> - >>> - =A0 =A0 =A0 DROP_GIANT(); >>> - >>> - =A0 =A0 =A0 thread_lock(td); >>> - >>> - =A0 =A0 =A0 /* get current priority */ >>> - =A0 =A0 =A0 old_prio =3D td->td_base_pri; >>> - >>> - =A0 =A0 =A0 /* set new priority */ >>> - =A0 =A0 =A0 sched_prio(td, td->td_user_pri); >>> - >>> - =A0 =A0 =A0 /* cause a task switch */ >>> - =A0 =A0 =A0 mi_switch(SW_INVOL | SWT_RELINQUISH, NULL); >>> - >>> - =A0 =A0 =A0 /* restore priority */ >>> - =A0 =A0 =A0 sched_prio(td, old_prio); >>> - >>> - =A0 =A0 =A0 thread_unlock(td); >>> - >>> - =A0 =A0 =A0 PICKUP_GIANT(); >>> -} >>> - >>> -static void >>> =A0ukbd_do_poll(struct ukbd_softc *sc, uint8_t wait) >>> =A0{ >>> >>> @@ -439,7 +412,7 @@ ukbd_do_poll(struct ukbd_softc *sc, uint8_t wait) >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0while (sc->sc_inputs =3D=3D 0) { >>> >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/* give USB threads a ch= ance to run */ >>> - =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 ukbd_yield(); >>> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 kern_yield(-1); >> >> Not quite. >> >> 1) -1 should be spelled PRI_UNCHANGED, except ukbd_yield() uses >> td_user_pri, but then puts it back again, so I think UNCHANGED is what >> is meant. >> 2) kern_yield() calls it a SW_VOL rather than SW_INVOL, which seems >> the desired behaviour here anyways, since this is an explicit (i.e. >> voluntary) yield. > > Thank you for the explanation. =A0So would you say that the patch is OK? As far as I know, yes. There may be a difference in behaviour, though, while yielding, if the priority of the thread remains high (as this change would make it) -- I'm not completely sure how the scheduler chooses threads, because I'm pretty sure I've seen it take threads with lower (higher numbered) priorities even when there's runnable threads with a higher (lower numbered) priority available. It has always seemed weird to me that the priorities in the kernel are strictly higher than user-space -- but only after a prio change like that done implicitly by many of the calls to sleep(9). So it may be that the better patch is to use PRI_USER, not PRI_UNCHANGED, which would revert any potentially changed thread prio (e.g. due to a sleep(9)) back to its user-space level, so that it contended as expected with other threads. hselasky@ or someone else familiar with the various usb threads would have to answer that. From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 12 16:01:01 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 03EA41065673; Mon, 12 Dec 2011 16:01:01 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe07.c2i.net [212.247.154.194]) by mx1.freebsd.org (Postfix) with ESMTP id B97DC8FC1A; Mon, 12 Dec 2011 16:00:58 +0000 (UTC) X-T2-Spam-Status: No, hits=-0.2 required=5.0 tests=ALL_TRUSTED, BAYES_50 Received: from [188.126.198.129] (account mc467741@c2i.net HELO laptop002.hselasky.homeunix.org) by mailfe07.swip.net (CommuniGate Pro SMTP 5.4.2) with ESMTPA id 215800158; Mon, 12 Dec 2011 17:00:56 +0100 From: Hans Petter Selasky To: freebsd-hackers@freebsd.org Date: Mon, 12 Dec 2011 16:58:22 +0100 User-Agent: KMail/1.13.5 (FreeBSD/8.2-STABLE; KDE/4.4.5; amd64; ; ) References: <4EE51CB5.1060505@FreeBSD.org> <4EE5B56A.4000106@FreeBSD.org> In-Reply-To: X-Face: *nPdTl_}RuAI6^PVpA02T?$%Xa^>@hE0uyUIoiha$pC:9TVgl.Oq,NwSZ4V" =?iso-8859-1?q?=7CLR=2E+tj=7Dg5=0A=09=25V?=,x^qOs~mnU3]Gn; cQLv&.N>TrxmSFf+p6(30a/{)KUU!s}w\IhQBj}[g}bj0I3^glmC( =?iso-8859-1?q?=0A=09=3AAuzV9=3A=2EhESm-x4h240C=609=3Dw?= MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201112121658.22864.hselasky@c2i.net> Cc: usb@freebsd.org, mdf@freebsd.org, hackers@freebsd.org, Andriy Gapon Subject: Re: kern_yield vs ukbd_yield X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2011 16:01:01 -0000 On Monday 12 December 2011 16:55:38 mdf@freebsd.org wrote: > On Mon, Dec 12, 2011 at 12:03 AM, Andriy Gapon wrote: > > on 11/12/2011 23:48 mdf@FreeBSD.org said the following: > >> On Sun, Dec 11, 2011 at 1:12 PM, Andriy Gapon wrote: > >>> Does the following change do what I think that it does? > >>> Thank you! > >>> > >>> Author: Andriy Gapon > >>> Date: Thu Sep 1 16:50:13 2011 +0300 > >>> > >>> ukbd: drop local duplicate of kern_yield and use that instead > >>> > >>> diff --git a/sys/dev/usb/input/ukbd.c b/sys/dev/usb/input/ukbd.c > >>> index 086c178..8078cbb 100644 > >>> --- a/sys/dev/usb/input/ukbd.c > >>> +++ b/sys/dev/usb/input/ukbd.c > >>> @@ -399,33 +399,6 @@ ukbd_put_key(struct ukbd_softc *sc, uint32_t key) > >>> } > >>> > >>> static void > >>> -ukbd_yield(void) > >>> -{ > >>> - struct thread *td = curthread; > >>> - uint32_t old_prio; > >>> - > >>> - DROP_GIANT(); > >>> - > >>> - thread_lock(td); > >>> - > >>> - /* get current priority */ > >>> - old_prio = td->td_base_pri; > >>> - > >>> - /* set new priority */ > >>> - sched_prio(td, td->td_user_pri); > >>> - > >>> - /* cause a task switch */ > >>> - mi_switch(SW_INVOL | SWT_RELINQUISH, NULL); > >>> - > >>> - /* restore priority */ > >>> - sched_prio(td, old_prio); > >>> - > >>> - thread_unlock(td); > >>> - > >>> - PICKUP_GIANT(); > >>> -} > >>> - > >>> -static void > >>> ukbd_do_poll(struct ukbd_softc *sc, uint8_t wait) > >>> { > >>> > >>> @@ -439,7 +412,7 @@ ukbd_do_poll(struct ukbd_softc *sc, uint8_t wait) > >>> while (sc->sc_inputs == 0) { > >>> > >>> /* give USB threads a chance to run */ > >>> - ukbd_yield(); > >>> + kern_yield(-1); > >> > >> Not quite. > >> > >> 1) -1 should be spelled PRI_UNCHANGED, except ukbd_yield() uses > >> td_user_pri, but then puts it back again, so I think UNCHANGED is what > >> is meant. > >> 2) kern_yield() calls it a SW_VOL rather than SW_INVOL, which seems > >> the desired behaviour here anyways, since this is an explicit (i.e. > >> voluntary) yield. > > > > Thank you for the explanation. So would you say that the patch is OK? > > As far as I know, yes. There may be a difference in behaviour, > though, while yielding, if the priority of the thread remains high (as > this change would make it) -- I'm not completely sure how the > scheduler chooses threads, because I'm pretty sure I've seen it take > threads with lower (higher numbered) priorities even when there's > runnable threads with a higher (lower numbered) priority available. > > It has always seemed weird to me that the priorities in the kernel are > strictly higher than user-space -- but only after a prio change like > that done implicitly by many of the calls to sleep(9). So it may be > that the better patch is to use PRI_USER, not PRI_UNCHANGED, which > would revert any potentially changed thread prio (e.g. due to a > sleep(9)) back to its user-space level, so that it contended as > expected with other threads. > Hi, > hselasky@ or someone else familiar with the various usb threads would > have to answer that. The problem is only during init() where the init thread has highest priority and that doesn't allow other threads to run even if the scheduler is running! --HPS From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 12 16:11:03 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E0856106566B; Mon, 12 Dec 2011 16:11:02 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe07.c2i.net [212.247.154.194]) by mx1.freebsd.org (Postfix) with ESMTP id EC1108FC0C; Mon, 12 Dec 2011 16:11:01 +0000 (UTC) X-T2-Spam-Status: No, hits=-0.2 required=5.0 tests=ALL_TRUSTED, BAYES_50 Received: from [188.126.198.129] (account mc467741@c2i.net HELO laptop002.hselasky.homeunix.org) by mailfe07.swip.net (CommuniGate Pro SMTP 5.4.2) with ESMTPA id 215800158; Mon, 12 Dec 2011 17:00:56 +0100 From: Hans Petter Selasky To: freebsd-hackers@freebsd.org Date: Mon, 12 Dec 2011 16:58:22 +0100 User-Agent: KMail/1.13.5 (FreeBSD/8.2-STABLE; KDE/4.4.5; amd64; ; ) References: <4EE51CB5.1060505@FreeBSD.org> <4EE5B56A.4000106@FreeBSD.org> In-Reply-To: X-Face: *nPdTl_}RuAI6^PVpA02T?$%Xa^>@hE0uyUIoiha$pC:9TVgl.Oq,NwSZ4V" =?iso-8859-1?q?=7CLR=2E+tj=7Dg5=0A=09=25V?=,x^qOs~mnU3]Gn; cQLv&.N>TrxmSFf+p6(30a/{)KUU!s}w\IhQBj}[g}bj0I3^glmC( =?iso-8859-1?q?=0A=09=3AAuzV9=3A=2EhESm-x4h240C=609=3Dw?= MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201112121658.22864.hselasky@c2i.net> Cc: usb@freebsd.org, mdf@freebsd.org, hackers@freebsd.org, Andriy Gapon Subject: Re: kern_yield vs ukbd_yield X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2011 16:11:03 -0000 On Monday 12 December 2011 16:55:38 mdf@freebsd.org wrote: > On Mon, Dec 12, 2011 at 12:03 AM, Andriy Gapon wrote: > > on 11/12/2011 23:48 mdf@FreeBSD.org said the following: > >> On Sun, Dec 11, 2011 at 1:12 PM, Andriy Gapon wrote: > >>> Does the following change do what I think that it does? > >>> Thank you! > >>> > >>> Author: Andriy Gapon > >>> Date: Thu Sep 1 16:50:13 2011 +0300 > >>> > >>> ukbd: drop local duplicate of kern_yield and use that instead > >>> > >>> diff --git a/sys/dev/usb/input/ukbd.c b/sys/dev/usb/input/ukbd.c > >>> index 086c178..8078cbb 100644 > >>> --- a/sys/dev/usb/input/ukbd.c > >>> +++ b/sys/dev/usb/input/ukbd.c > >>> @@ -399,33 +399,6 @@ ukbd_put_key(struct ukbd_softc *sc, uint32_t key) > >>> } > >>> > >>> static void > >>> -ukbd_yield(void) > >>> -{ > >>> - struct thread *td = curthread; > >>> - uint32_t old_prio; > >>> - > >>> - DROP_GIANT(); > >>> - > >>> - thread_lock(td); > >>> - > >>> - /* get current priority */ > >>> - old_prio = td->td_base_pri; > >>> - > >>> - /* set new priority */ > >>> - sched_prio(td, td->td_user_pri); > >>> - > >>> - /* cause a task switch */ > >>> - mi_switch(SW_INVOL | SWT_RELINQUISH, NULL); > >>> - > >>> - /* restore priority */ > >>> - sched_prio(td, old_prio); > >>> - > >>> - thread_unlock(td); > >>> - > >>> - PICKUP_GIANT(); > >>> -} > >>> - > >>> -static void > >>> ukbd_do_poll(struct ukbd_softc *sc, uint8_t wait) > >>> { > >>> > >>> @@ -439,7 +412,7 @@ ukbd_do_poll(struct ukbd_softc *sc, uint8_t wait) > >>> while (sc->sc_inputs == 0) { > >>> > >>> /* give USB threads a chance to run */ > >>> - ukbd_yield(); > >>> + kern_yield(-1); > >> > >> Not quite. > >> > >> 1) -1 should be spelled PRI_UNCHANGED, except ukbd_yield() uses > >> td_user_pri, but then puts it back again, so I think UNCHANGED is what > >> is meant. > >> 2) kern_yield() calls it a SW_VOL rather than SW_INVOL, which seems > >> the desired behaviour here anyways, since this is an explicit (i.e. > >> voluntary) yield. > > > > Thank you for the explanation. So would you say that the patch is OK? > > As far as I know, yes. There may be a difference in behaviour, > though, while yielding, if the priority of the thread remains high (as > this change would make it) -- I'm not completely sure how the > scheduler chooses threads, because I'm pretty sure I've seen it take > threads with lower (higher numbered) priorities even when there's > runnable threads with a higher (lower numbered) priority available. > > It has always seemed weird to me that the priorities in the kernel are > strictly higher than user-space -- but only after a prio change like > that done implicitly by many of the calls to sleep(9). So it may be > that the better patch is to use PRI_USER, not PRI_UNCHANGED, which > would revert any potentially changed thread prio (e.g. due to a > sleep(9)) back to its user-space level, so that it contended as > expected with other threads. > Hi, > hselasky@ or someone else familiar with the various usb threads would > have to answer that. The problem is only during init() where the init thread has highest priority and that doesn't allow other threads to run even if the scheduler is running! --HPS From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 12 19:05:40 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0FEF41065672; Mon, 12 Dec 2011 19:05:40 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id C88AF8FC14; Mon, 12 Dec 2011 19:05:39 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id 75B2346B23; Mon, 12 Dec 2011 14:05:39 -0500 (EST) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 04A38B93A; Mon, 12 Dec 2011 14:05:39 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Mon, 12 Dec 2011 14:05:38 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p8; KDE/4.5.5; amd64; ; ) References: <4EE51CB5.1060505@FreeBSD.org> <201112121658.22864.hselasky@c2i.net> In-Reply-To: <201112121658.22864.hselasky@c2i.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201112121405.38322.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 12 Dec 2011 14:05:39 -0500 (EST) Cc: usb@freebsd.org, mdf@freebsd.org, hackers@freebsd.org, Andriy Gapon , Hans Petter Selasky Subject: Re: kern_yield vs ukbd_yield X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2011 19:05:40 -0000 On Monday, December 12, 2011 10:58:22 am Hans Petter Selasky wrote: > On Monday 12 December 2011 16:55:38 mdf@freebsd.org wrote: > > On Mon, Dec 12, 2011 at 12:03 AM, Andriy Gapon wrote: > > > on 11/12/2011 23:48 mdf@FreeBSD.org said the following: > > >> On Sun, Dec 11, 2011 at 1:12 PM, Andriy Gapon wrote: > > >>> Does the following change do what I think that it does? > > >>> Thank you! > > >>> > > >>> Author: Andriy Gapon > > >>> Date: Thu Sep 1 16:50:13 2011 +0300 > > >>> > > >>> ukbd: drop local duplicate of kern_yield and use that instead > > >>> > > >>> diff --git a/sys/dev/usb/input/ukbd.c b/sys/dev/usb/input/ukbd.c > > >>> index 086c178..8078cbb 100644 > > >>> --- a/sys/dev/usb/input/ukbd.c > > >>> +++ b/sys/dev/usb/input/ukbd.c > > >>> @@ -399,33 +399,6 @@ ukbd_put_key(struct ukbd_softc *sc, uint32_t key) > > >>> } > > >>> > > >>> static void > > >>> -ukbd_yield(void) > > >>> -{ > > >>> - struct thread *td = curthread; > > >>> - uint32_t old_prio; > > >>> - > > >>> - DROP_GIANT(); > > >>> - > > >>> - thread_lock(td); > > >>> - > > >>> - /* get current priority */ > > >>> - old_prio = td->td_base_pri; > > >>> - > > >>> - /* set new priority */ > > >>> - sched_prio(td, td->td_user_pri); > > >>> - > > >>> - /* cause a task switch */ > > >>> - mi_switch(SW_INVOL | SWT_RELINQUISH, NULL); > > >>> - > > >>> - /* restore priority */ > > >>> - sched_prio(td, old_prio); > > >>> - > > >>> - thread_unlock(td); > > >>> - > > >>> - PICKUP_GIANT(); > > >>> -} > > >>> - > > >>> -static void > > >>> ukbd_do_poll(struct ukbd_softc *sc, uint8_t wait) > > >>> { > > >>> > > >>> @@ -439,7 +412,7 @@ ukbd_do_poll(struct ukbd_softc *sc, uint8_t wait) > > >>> while (sc->sc_inputs == 0) { > > >>> > > >>> /* give USB threads a chance to run */ > > >>> - ukbd_yield(); > > >>> + kern_yield(-1); > > >> > > >> Not quite. > > >> > > >> 1) -1 should be spelled PRI_UNCHANGED, except ukbd_yield() uses > > >> td_user_pri, but then puts it back again, so I think UNCHANGED is what > > >> is meant. > > >> 2) kern_yield() calls it a SW_VOL rather than SW_INVOL, which seems > > >> the desired behaviour here anyways, since this is an explicit (i.e. > > >> voluntary) yield. > > > > > > Thank you for the explanation. So would you say that the patch is OK? > > > > As far as I know, yes. There may be a difference in behaviour, > > though, while yielding, if the priority of the thread remains high (as > > this change would make it) -- I'm not completely sure how the > > scheduler chooses threads, because I'm pretty sure I've seen it take > > threads with lower (higher numbered) priorities even when there's > > runnable threads with a higher (lower numbered) priority available. The scheduler generally should not do this with the following exceptions: 1) for timesharing threads, priorities are in bands, so effectively the pri / 4 is what is used for comparison and if two threads have the same pri / 4 the scheduler will round-robin among htem. 2) ULE might be a bit different because of the way it assigns threads to CPUs, so if a CPU has two high priority threads and another CPU only has a low priority thread, the second CPU will not run the second high priority thread. 4BSD handles this case more correctly. > > It has always seemed weird to me that the priorities in the kernel are > > strictly higher than user-space -- but only after a prio change like > > that done implicitly by many of the calls to sleep(9). So it may be > > that the better patch is to use PRI_USER, not PRI_UNCHANGED, which > > would revert any potentially changed thread prio (e.g. due to a > > sleep(9)) back to its user-space level, so that it contended as > > expected with other threads. Realtime priorities (for rtprio user threads) are higher than the kernel "sleep" priorities. Also, keep in mind that a thread does not get an automatic priority boost when it enters the kernel. It only gets a boost either temporarily from priority propagation, or a slightly longer (but still temporary) boost from a sleep(9) call. > Hi, > > > hselasky@ or someone else familiar with the various usb threads would > > have to answer that. > > The problem is only during init() where the init thread has highest priority > and that doesn't allow other threads to run even if the scheduler is running! Hmm, that should be fixed by lowering the relevant thread's priority. Do you mean thread0 (the one doing all the SYSINIT's or thread we create for init (pid 1) before it executes init? -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 12 19:05:40 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0FEF41065672; Mon, 12 Dec 2011 19:05:40 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id C88AF8FC14; Mon, 12 Dec 2011 19:05:39 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id 75B2346B23; Mon, 12 Dec 2011 14:05:39 -0500 (EST) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 04A38B93A; Mon, 12 Dec 2011 14:05:39 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Mon, 12 Dec 2011 14:05:38 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p8; KDE/4.5.5; amd64; ; ) References: <4EE51CB5.1060505@FreeBSD.org> <201112121658.22864.hselasky@c2i.net> In-Reply-To: <201112121658.22864.hselasky@c2i.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201112121405.38322.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 12 Dec 2011 14:05:39 -0500 (EST) Cc: usb@freebsd.org, mdf@freebsd.org, hackers@freebsd.org, Andriy Gapon , Hans Petter Selasky Subject: Re: kern_yield vs ukbd_yield X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2011 19:05:40 -0000 On Monday, December 12, 2011 10:58:22 am Hans Petter Selasky wrote: > On Monday 12 December 2011 16:55:38 mdf@freebsd.org wrote: > > On Mon, Dec 12, 2011 at 12:03 AM, Andriy Gapon wrote: > > > on 11/12/2011 23:48 mdf@FreeBSD.org said the following: > > >> On Sun, Dec 11, 2011 at 1:12 PM, Andriy Gapon wrote: > > >>> Does the following change do what I think that it does? > > >>> Thank you! > > >>> > > >>> Author: Andriy Gapon > > >>> Date: Thu Sep 1 16:50:13 2011 +0300 > > >>> > > >>> ukbd: drop local duplicate of kern_yield and use that instead > > >>> > > >>> diff --git a/sys/dev/usb/input/ukbd.c b/sys/dev/usb/input/ukbd.c > > >>> index 086c178..8078cbb 100644 > > >>> --- a/sys/dev/usb/input/ukbd.c > > >>> +++ b/sys/dev/usb/input/ukbd.c > > >>> @@ -399,33 +399,6 @@ ukbd_put_key(struct ukbd_softc *sc, uint32_t key) > > >>> } > > >>> > > >>> static void > > >>> -ukbd_yield(void) > > >>> -{ > > >>> - struct thread *td = curthread; > > >>> - uint32_t old_prio; > > >>> - > > >>> - DROP_GIANT(); > > >>> - > > >>> - thread_lock(td); > > >>> - > > >>> - /* get current priority */ > > >>> - old_prio = td->td_base_pri; > > >>> - > > >>> - /* set new priority */ > > >>> - sched_prio(td, td->td_user_pri); > > >>> - > > >>> - /* cause a task switch */ > > >>> - mi_switch(SW_INVOL | SWT_RELINQUISH, NULL); > > >>> - > > >>> - /* restore priority */ > > >>> - sched_prio(td, old_prio); > > >>> - > > >>> - thread_unlock(td); > > >>> - > > >>> - PICKUP_GIANT(); > > >>> -} > > >>> - > > >>> -static void > > >>> ukbd_do_poll(struct ukbd_softc *sc, uint8_t wait) > > >>> { > > >>> > > >>> @@ -439,7 +412,7 @@ ukbd_do_poll(struct ukbd_softc *sc, uint8_t wait) > > >>> while (sc->sc_inputs == 0) { > > >>> > > >>> /* give USB threads a chance to run */ > > >>> - ukbd_yield(); > > >>> + kern_yield(-1); > > >> > > >> Not quite. > > >> > > >> 1) -1 should be spelled PRI_UNCHANGED, except ukbd_yield() uses > > >> td_user_pri, but then puts it back again, so I think UNCHANGED is what > > >> is meant. > > >> 2) kern_yield() calls it a SW_VOL rather than SW_INVOL, which seems > > >> the desired behaviour here anyways, since this is an explicit (i.e. > > >> voluntary) yield. > > > > > > Thank you for the explanation. So would you say that the patch is OK? > > > > As far as I know, yes. There may be a difference in behaviour, > > though, while yielding, if the priority of the thread remains high (as > > this change would make it) -- I'm not completely sure how the > > scheduler chooses threads, because I'm pretty sure I've seen it take > > threads with lower (higher numbered) priorities even when there's > > runnable threads with a higher (lower numbered) priority available. The scheduler generally should not do this with the following exceptions: 1) for timesharing threads, priorities are in bands, so effectively the pri / 4 is what is used for comparison and if two threads have the same pri / 4 the scheduler will round-robin among htem. 2) ULE might be a bit different because of the way it assigns threads to CPUs, so if a CPU has two high priority threads and another CPU only has a low priority thread, the second CPU will not run the second high priority thread. 4BSD handles this case more correctly. > > It has always seemed weird to me that the priorities in the kernel are > > strictly higher than user-space -- but only after a prio change like > > that done implicitly by many of the calls to sleep(9). So it may be > > that the better patch is to use PRI_USER, not PRI_UNCHANGED, which > > would revert any potentially changed thread prio (e.g. due to a > > sleep(9)) back to its user-space level, so that it contended as > > expected with other threads. Realtime priorities (for rtprio user threads) are higher than the kernel "sleep" priorities. Also, keep in mind that a thread does not get an automatic priority boost when it enters the kernel. It only gets a boost either temporarily from priority propagation, or a slightly longer (but still temporary) boost from a sleep(9) call. > Hi, > > > hselasky@ or someone else familiar with the various usb threads would > > have to answer that. > > The problem is only during init() where the init thread has highest priority > and that doesn't allow other threads to run even if the scheduler is running! Hmm, that should be fixed by lowering the relevant thread's priority. Do you mean thread0 (the one doing all the SYSINIT's or thread we create for init (pid 1) before it executes init? -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 12 19:11:49 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9C82E106564A; Mon, 12 Dec 2011 19:11:49 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe03.c2i.net [212.247.154.66]) by mx1.freebsd.org (Postfix) with ESMTP id 73B298FC0A; Mon, 12 Dec 2011 19:11:48 +0000 (UTC) X-T2-Spam-Status: No, hits=-0.2 required=5.0 tests=ALL_TRUSTED, BAYES_50 Received: from [188.126.198.129] (account mc467741@c2i.net HELO laptop002.hselasky.homeunix.org) by mailfe03.swip.net (CommuniGate Pro SMTP 5.4.2) with ESMTPA id 48773972; Mon, 12 Dec 2011 20:11:46 +0100 From: Hans Petter Selasky To: freebsd-hackers@freebsd.org, Andriy Gapon Date: Mon, 12 Dec 2011 20:09:12 +0100 User-Agent: KMail/1.13.5 (FreeBSD/8.2-STABLE; KDE/4.4.5; amd64; ; ) References: <4EE51CB5.1060505@FreeBSD.org> <201112121658.22864.hselasky@c2i.net> <201112121405.38322.jhb@freebsd.org> In-Reply-To: <201112121405.38322.jhb@freebsd.org> X-Face: *nPdTl_}RuAI6^PVpA02T?$%Xa^>@hE0uyUIoiha$pC:9TVgl.Oq,NwSZ4V" =?iso-8859-1?q?=7CLR=2E+tj=7Dg5=0A=09=25V?=,x^qOs~mnU3]Gn; cQLv&.N>TrxmSFf+p6(30a/{)KUU!s}w\IhQBj}[g}bj0I3^glmC( =?iso-8859-1?q?=0A=09=3AAuzV9=3A=2EhESm-x4h240C=609=3Dw?= MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201112122009.12557.hselasky@c2i.net> Cc: usb@freebsd.org, mdf@freebsd.org Subject: Re: kern_yield vs ukbd_yield X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2011 19:11:49 -0000 On Monday 12 December 2011 20:05:38 John Baldwin wrote: > > Hi, > > > > > > > > > hselasky@ or someone else familiar with the various usb threads would > > > have to answer that. > > > > > > > > The problem is only during init() where the init thread has highest > > priority and that doesn't allow other threads to run even if the > > scheduler is > > running! > > Hmm, that should be fixed by lowering the relevant thread's priority. > Do you mean thread0 (the one doing all the SYSINIT's or thread we create > for init (pid 1) before it executes init? Yes, thread0. Correct! --HPS From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 12 19:13:50 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 16C54106564A; Mon, 12 Dec 2011 19:13:50 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id C55998FC0A; Mon, 12 Dec 2011 19:13:49 +0000 (UTC) Received: by dakp5 with SMTP id p5so7351381dak.13 for ; Mon, 12 Dec 2011 11:13:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=hZ8jPOrL9/CZdLmXWzfhaF0zRX46QB3ln3lyS4C1Nj4=; b=XsIk2CzLNkx75Vz5BI0J9APskBH6JnlnkLfF2lnpVB9y5R+ekfHu0Ygyp5stVQ2Idy ZASq6qD+/uIZE+EXuhkXN5URg1TEHwRRFffJH2fAXYk+zkiQlD1sQHs58OXfOOKgM+9c hl6q7h2Ih+ObuO8vYkiBnE9B4ye2RYennGADM= MIME-Version: 1.0 Received: by 10.68.73.197 with SMTP id n5mr36556308pbv.102.1323717229215; Mon, 12 Dec 2011 11:13:49 -0800 (PST) Sender: mdf356@gmail.com Received: by 10.68.197.198 with HTTP; Mon, 12 Dec 2011 11:13:49 -0800 (PST) In-Reply-To: <201112121405.38322.jhb@freebsd.org> References: <4EE51CB5.1060505@FreeBSD.org> <201112121658.22864.hselasky@c2i.net> <201112121405.38322.jhb@freebsd.org> Date: Mon, 12 Dec 2011 11:13:49 -0800 X-Google-Sender-Auth: tA1WgJS5KaNFUvDe4pesRX4T_5A Message-ID: From: mdf@FreeBSD.org To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: usb@freebsd.org, freebsd-hackers@freebsd.org, hackers@freebsd.org, Andriy Gapon , Hans Petter Selasky Subject: Re: kern_yield vs ukbd_yield X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2011 19:13:50 -0000 On Mon, Dec 12, 2011 at 11:05 AM, John Baldwin wrote: > On Monday, December 12, 2011 10:58:22 am Hans Petter Selasky wrote: >> On Monday 12 December 2011 16:55:38 mdf@freebsd.org wrote: >> > On Mon, Dec 12, 2011 at 12:03 AM, Andriy Gapon wrote= : >> > > on 11/12/2011 23:48 mdf@FreeBSD.org said the following: >> > >> On Sun, Dec 11, 2011 at 1:12 PM, Andriy Gapon wro= te: >> > >>> Does the following change do what I think that it does? >> > >>> Thank you! >> > >>> >> > >>> Author: Andriy Gapon >> > >>> Date: =A0 Thu Sep 1 16:50:13 2011 +0300 >> > >>> >> > >>> =A0 =A0ukbd: drop local duplicate of kern_yield and use that inste= ad >> > >>> >> > >>> diff --git a/sys/dev/usb/input/ukbd.c b/sys/dev/usb/input/ukbd.c >> > >>> index 086c178..8078cbb 100644 >> > >>> --- a/sys/dev/usb/input/ukbd.c >> > >>> +++ b/sys/dev/usb/input/ukbd.c >> > >>> @@ -399,33 +399,6 @@ ukbd_put_key(struct ukbd_softc *sc, uint32_t = key) >> > >>> =A0} >> > >>> >> > >>> =A0static void >> > >>> -ukbd_yield(void) >> > >>> -{ >> > >>> - =A0 =A0 =A0 struct thread *td =3D curthread; >> > >>> - =A0 =A0 =A0 uint32_t old_prio; >> > >>> - >> > >>> - =A0 =A0 =A0 DROP_GIANT(); >> > >>> - >> > >>> - =A0 =A0 =A0 thread_lock(td); >> > >>> - >> > >>> - =A0 =A0 =A0 /* get current priority */ >> > >>> - =A0 =A0 =A0 old_prio =3D td->td_base_pri; >> > >>> - >> > >>> - =A0 =A0 =A0 /* set new priority */ >> > >>> - =A0 =A0 =A0 sched_prio(td, td->td_user_pri); >> > >>> - >> > >>> - =A0 =A0 =A0 /* cause a task switch */ >> > >>> - =A0 =A0 =A0 mi_switch(SW_INVOL | SWT_RELINQUISH, NULL); >> > >>> - >> > >>> - =A0 =A0 =A0 /* restore priority */ >> > >>> - =A0 =A0 =A0 sched_prio(td, old_prio); >> > >>> - >> > >>> - =A0 =A0 =A0 thread_unlock(td); >> > >>> - >> > >>> - =A0 =A0 =A0 PICKUP_GIANT(); >> > >>> -} >> > >>> - >> > >>> -static void >> > >>> =A0ukbd_do_poll(struct ukbd_softc *sc, uint8_t wait) >> > >>> =A0{ >> > >>> >> > >>> @@ -439,7 +412,7 @@ ukbd_do_poll(struct ukbd_softc *sc, uint8_t wa= it) >> > >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0while (sc->sc_inputs =3D=3D 0) { >> > >>> >> > >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/* give USB threads= a chance to run */ >> > >>> - =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 ukbd_yield(); >> > >>> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 kern_yield(-1); >> > >> >> > >> Not quite. >> > >> >> > >> 1) -1 should be spelled PRI_UNCHANGED, except ukbd_yield() uses >> > >> td_user_pri, but then puts it back again, so I think UNCHANGED is w= hat >> > >> is meant. >> > >> 2) kern_yield() calls it a SW_VOL rather than SW_INVOL, which seems >> > >> the desired behaviour here anyways, since this is an explicit (i.e. >> > >> voluntary) yield. >> > > >> > > Thank you for the explanation. =A0So would you say that the patch is= OK? >> > >> > As far as I know, yes. =A0There may be a difference in behaviour, >> > though, while yielding, if the priority of the thread remains high (as >> > this change would make it) -- I'm not completely sure how the >> > scheduler chooses threads, because I'm pretty sure I've seen it take >> > threads with lower (higher numbered) priorities even when there's >> > runnable threads with a higher (lower numbered) priority available. > > The scheduler generally should not do this with the following exceptions: > > =A01) for timesharing threads, priorities are in bands, so effectively th= e > =A0 =A0pri / 4 is what is used for comparison and if two threads have the= same > =A0 =A0pri / 4 the scheduler will round-robin among htem. > > =A02) ULE might be a bit different because of the way it assigns threads = to > =A0 =A0CPUs, so if a CPU has two high priority threads and another CPU on= ly > =A0 =A0has a low priority thread, the second CPU will not run the second = high > =A0 =A0priority thread. =A04BSD handles this case more correctly. > >> > It has always seemed weird to me that the priorities in the kernel are >> > strictly higher than user-space -- but only after a prio change like >> > that done implicitly by many of the calls to sleep(9). =A0So it may be >> > that the better patch is to use PRI_USER, not PRI_UNCHANGED, which >> > would revert any potentially changed thread prio (e.g. due to a >> > sleep(9)) back to its user-space level, so that it contended as >> > expected with other threads. > > Realtime priorities (for rtprio user threads) are higher than the kernel > "sleep" priorities. =A0Also, keep in mind that a thread does not get an > automatic priority boost when it enters the kernel. =A0It only gets a boo= st > either temporarily from priority propagation, or a slightly longer (but > still temporary) boost from a sleep(9) call. I still don't understand why threads are boosting their priority while sleeping; if it's so they are woken preferentially, that makes some sense, but then they shouldn't be keeping the boosted priority after the thread is woken. If a thread really needs higher priority to do its work, that should be an explicit call to sched_prio(9), rather than implicitly only when/if it first sleeps. Thanks, matthew >> > hselasky@ or someone else familiar with the various usb threads would >> > have to answer that. >> >> The problem is only during init() where the init thread has highest prio= rity >> and that doesn't allow other threads to run even if the scheduler is > running! > > Hmm, that should be fixed by lowering the relevant thread's priority. > Do you mean thread0 (the one doing all the SYSINIT's or thread we create = for > init (pid 1) before it executes init? > > -- > John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 12 19:13:50 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 16C54106564A; Mon, 12 Dec 2011 19:13:50 +0000 (UTC) (envelope-from mdf356@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id C55998FC0A; Mon, 12 Dec 2011 19:13:49 +0000 (UTC) Received: by dakp5 with SMTP id p5so7351381dak.13 for ; Mon, 12 Dec 2011 11:13:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=hZ8jPOrL9/CZdLmXWzfhaF0zRX46QB3ln3lyS4C1Nj4=; b=XsIk2CzLNkx75Vz5BI0J9APskBH6JnlnkLfF2lnpVB9y5R+ekfHu0Ygyp5stVQ2Idy ZASq6qD+/uIZE+EXuhkXN5URg1TEHwRRFffJH2fAXYk+zkiQlD1sQHs58OXfOOKgM+9c hl6q7h2Ih+ObuO8vYkiBnE9B4ye2RYennGADM= MIME-Version: 1.0 Received: by 10.68.73.197 with SMTP id n5mr36556308pbv.102.1323717229215; Mon, 12 Dec 2011 11:13:49 -0800 (PST) Sender: mdf356@gmail.com Received: by 10.68.197.198 with HTTP; Mon, 12 Dec 2011 11:13:49 -0800 (PST) In-Reply-To: <201112121405.38322.jhb@freebsd.org> References: <4EE51CB5.1060505@FreeBSD.org> <201112121658.22864.hselasky@c2i.net> <201112121405.38322.jhb@freebsd.org> Date: Mon, 12 Dec 2011 11:13:49 -0800 X-Google-Sender-Auth: tA1WgJS5KaNFUvDe4pesRX4T_5A Message-ID: From: mdf@FreeBSD.org To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: usb@freebsd.org, freebsd-hackers@freebsd.org, hackers@freebsd.org, Andriy Gapon , Hans Petter Selasky Subject: Re: kern_yield vs ukbd_yield X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2011 19:13:50 -0000 On Mon, Dec 12, 2011 at 11:05 AM, John Baldwin wrote: > On Monday, December 12, 2011 10:58:22 am Hans Petter Selasky wrote: >> On Monday 12 December 2011 16:55:38 mdf@freebsd.org wrote: >> > On Mon, Dec 12, 2011 at 12:03 AM, Andriy Gapon wrote= : >> > > on 11/12/2011 23:48 mdf@FreeBSD.org said the following: >> > >> On Sun, Dec 11, 2011 at 1:12 PM, Andriy Gapon wro= te: >> > >>> Does the following change do what I think that it does? >> > >>> Thank you! >> > >>> >> > >>> Author: Andriy Gapon >> > >>> Date: =A0 Thu Sep 1 16:50:13 2011 +0300 >> > >>> >> > >>> =A0 =A0ukbd: drop local duplicate of kern_yield and use that inste= ad >> > >>> >> > >>> diff --git a/sys/dev/usb/input/ukbd.c b/sys/dev/usb/input/ukbd.c >> > >>> index 086c178..8078cbb 100644 >> > >>> --- a/sys/dev/usb/input/ukbd.c >> > >>> +++ b/sys/dev/usb/input/ukbd.c >> > >>> @@ -399,33 +399,6 @@ ukbd_put_key(struct ukbd_softc *sc, uint32_t = key) >> > >>> =A0} >> > >>> >> > >>> =A0static void >> > >>> -ukbd_yield(void) >> > >>> -{ >> > >>> - =A0 =A0 =A0 struct thread *td =3D curthread; >> > >>> - =A0 =A0 =A0 uint32_t old_prio; >> > >>> - >> > >>> - =A0 =A0 =A0 DROP_GIANT(); >> > >>> - >> > >>> - =A0 =A0 =A0 thread_lock(td); >> > >>> - >> > >>> - =A0 =A0 =A0 /* get current priority */ >> > >>> - =A0 =A0 =A0 old_prio =3D td->td_base_pri; >> > >>> - >> > >>> - =A0 =A0 =A0 /* set new priority */ >> > >>> - =A0 =A0 =A0 sched_prio(td, td->td_user_pri); >> > >>> - >> > >>> - =A0 =A0 =A0 /* cause a task switch */ >> > >>> - =A0 =A0 =A0 mi_switch(SW_INVOL | SWT_RELINQUISH, NULL); >> > >>> - >> > >>> - =A0 =A0 =A0 /* restore priority */ >> > >>> - =A0 =A0 =A0 sched_prio(td, old_prio); >> > >>> - >> > >>> - =A0 =A0 =A0 thread_unlock(td); >> > >>> - >> > >>> - =A0 =A0 =A0 PICKUP_GIANT(); >> > >>> -} >> > >>> - >> > >>> -static void >> > >>> =A0ukbd_do_poll(struct ukbd_softc *sc, uint8_t wait) >> > >>> =A0{ >> > >>> >> > >>> @@ -439,7 +412,7 @@ ukbd_do_poll(struct ukbd_softc *sc, uint8_t wa= it) >> > >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0while (sc->sc_inputs =3D=3D 0) { >> > >>> >> > >>> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0/* give USB threads= a chance to run */ >> > >>> - =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 ukbd_yield(); >> > >>> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 kern_yield(-1); >> > >> >> > >> Not quite. >> > >> >> > >> 1) -1 should be spelled PRI_UNCHANGED, except ukbd_yield() uses >> > >> td_user_pri, but then puts it back again, so I think UNCHANGED is w= hat >> > >> is meant. >> > >> 2) kern_yield() calls it a SW_VOL rather than SW_INVOL, which seems >> > >> the desired behaviour here anyways, since this is an explicit (i.e. >> > >> voluntary) yield. >> > > >> > > Thank you for the explanation. =A0So would you say that the patch is= OK? >> > >> > As far as I know, yes. =A0There may be a difference in behaviour, >> > though, while yielding, if the priority of the thread remains high (as >> > this change would make it) -- I'm not completely sure how the >> > scheduler chooses threads, because I'm pretty sure I've seen it take >> > threads with lower (higher numbered) priorities even when there's >> > runnable threads with a higher (lower numbered) priority available. > > The scheduler generally should not do this with the following exceptions: > > =A01) for timesharing threads, priorities are in bands, so effectively th= e > =A0 =A0pri / 4 is what is used for comparison and if two threads have the= same > =A0 =A0pri / 4 the scheduler will round-robin among htem. > > =A02) ULE might be a bit different because of the way it assigns threads = to > =A0 =A0CPUs, so if a CPU has two high priority threads and another CPU on= ly > =A0 =A0has a low priority thread, the second CPU will not run the second = high > =A0 =A0priority thread. =A04BSD handles this case more correctly. > >> > It has always seemed weird to me that the priorities in the kernel are >> > strictly higher than user-space -- but only after a prio change like >> > that done implicitly by many of the calls to sleep(9). =A0So it may be >> > that the better patch is to use PRI_USER, not PRI_UNCHANGED, which >> > would revert any potentially changed thread prio (e.g. due to a >> > sleep(9)) back to its user-space level, so that it contended as >> > expected with other threads. > > Realtime priorities (for rtprio user threads) are higher than the kernel > "sleep" priorities. =A0Also, keep in mind that a thread does not get an > automatic priority boost when it enters the kernel. =A0It only gets a boo= st > either temporarily from priority propagation, or a slightly longer (but > still temporary) boost from a sleep(9) call. I still don't understand why threads are boosting their priority while sleeping; if it's so they are woken preferentially, that makes some sense, but then they shouldn't be keeping the boosted priority after the thread is woken. If a thread really needs higher priority to do its work, that should be an explicit call to sched_prio(9), rather than implicitly only when/if it first sleeps. Thanks, matthew >> > hselasky@ or someone else familiar with the various usb threads would >> > have to answer that. >> >> The problem is only during init() where the init thread has highest prio= rity >> and that doesn't allow other threads to run even if the scheduler is > running! > > Hmm, that should be fixed by lowering the relevant thread's priority. > Do you mean thread0 (the one doing all the SYSINIT's or thread we create = for > init (pid 1) before it executes init? > > -- > John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 12 21:18:09 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 951531065677; Mon, 12 Dec 2011 21:18:09 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 59AC48FC1C; Mon, 12 Dec 2011 21:18:09 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id 04E9A46B0D; Mon, 12 Dec 2011 16:18:09 -0500 (EST) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 795A7B948; Mon, 12 Dec 2011 16:18:08 -0500 (EST) From: John Baldwin To: mdf@freebsd.org Date: Mon, 12 Dec 2011 16:18:07 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p8; KDE/4.5.5; amd64; ; ) References: <4EE51CB5.1060505@FreeBSD.org> <201112121405.38322.jhb@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201112121618.07833.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 12 Dec 2011 16:18:08 -0500 (EST) Cc: usb@freebsd.org, freebsd-hackers@freebsd.org, hackers@freebsd.org, Andriy Gapon , Hans Petter Selasky Subject: Re: kern_yield vs ukbd_yield X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2011 21:18:09 -0000 On Monday, December 12, 2011 2:13:49 pm mdf@freebsd.org wrote: > On Mon, Dec 12, 2011 at 11:05 AM, John Baldwin wrote: > > On Monday, December 12, 2011 10:58:22 am Hans Petter Selasky wrote: > >> On Monday 12 December 2011 16:55:38 mdf@freebsd.org wrote: > >> > On Mon, Dec 12, 2011 at 12:03 AM, Andriy Gapon wrote: > >> > > on 11/12/2011 23:48 mdf@FreeBSD.org said the following: > >> > >> On Sun, Dec 11, 2011 at 1:12 PM, Andriy Gapon wrote: > >> > >>> Does the following change do what I think that it does? > >> > >>> Thank you! > >> > >>> > >> > >>> Author: Andriy Gapon > >> > >>> Date: Thu Sep 1 16:50:13 2011 +0300 > >> > >>> > >> > >>> ukbd: drop local duplicate of kern_yield and use that instead > >> > >>> > >> > >>> diff --git a/sys/dev/usb/input/ukbd.c b/sys/dev/usb/input/ukbd.c > >> > >>> index 086c178..8078cbb 100644 > >> > >>> --- a/sys/dev/usb/input/ukbd.c > >> > >>> +++ b/sys/dev/usb/input/ukbd.c > >> > >>> @@ -399,33 +399,6 @@ ukbd_put_key(struct ukbd_softc *sc, uint32_t key) > >> > >>> } > >> > >>> > >> > >>> static void > >> > >>> -ukbd_yield(void) > >> > >>> -{ > >> > >>> - struct thread *td = curthread; > >> > >>> - uint32_t old_prio; > >> > >>> - > >> > >>> - DROP_GIANT(); > >> > >>> - > >> > >>> - thread_lock(td); > >> > >>> - > >> > >>> - /* get current priority */ > >> > >>> - old_prio = td->td_base_pri; > >> > >>> - > >> > >>> - /* set new priority */ > >> > >>> - sched_prio(td, td->td_user_pri); > >> > >>> - > >> > >>> - /* cause a task switch */ > >> > >>> - mi_switch(SW_INVOL | SWT_RELINQUISH, NULL); > >> > >>> - > >> > >>> - /* restore priority */ > >> > >>> - sched_prio(td, old_prio); > >> > >>> - > >> > >>> - thread_unlock(td); > >> > >>> - > >> > >>> - PICKUP_GIANT(); > >> > >>> -} > >> > >>> - > >> > >>> -static void > >> > >>> ukbd_do_poll(struct ukbd_softc *sc, uint8_t wait) > >> > >>> { > >> > >>> > >> > >>> @@ -439,7 +412,7 @@ ukbd_do_poll(struct ukbd_softc *sc, uint8_t wait) > >> > >>> while (sc->sc_inputs == 0) { > >> > >>> > >> > >>> /* give USB threads a chance to run */ > >> > >>> - ukbd_yield(); > >> > >>> + kern_yield(-1); > >> > >> > >> > >> Not quite. > >> > >> > >> > >> 1) -1 should be spelled PRI_UNCHANGED, except ukbd_yield() uses > >> > >> td_user_pri, but then puts it back again, so I think UNCHANGED is what > >> > >> is meant. > >> > >> 2) kern_yield() calls it a SW_VOL rather than SW_INVOL, which seems > >> > >> the desired behaviour here anyways, since this is an explicit (i.e. > >> > >> voluntary) yield. > >> > > > >> > > Thank you for the explanation. So would you say that the patch is OK? > >> > > >> > As far as I know, yes. There may be a difference in behaviour, > >> > though, while yielding, if the priority of the thread remains high (as > >> > this change would make it) -- I'm not completely sure how the > >> > scheduler chooses threads, because I'm pretty sure I've seen it take > >> > threads with lower (higher numbered) priorities even when there's > >> > runnable threads with a higher (lower numbered) priority available. > > > > The scheduler generally should not do this with the following exceptions: > > > > 1) for timesharing threads, priorities are in bands, so effectively the > > pri / 4 is what is used for comparison and if two threads have the same > > pri / 4 the scheduler will round-robin among htem. > > > > 2) ULE might be a bit different because of the way it assigns threads to > > CPUs, so if a CPU has two high priority threads and another CPU only > > has a low priority thread, the second CPU will not run the second high > > priority thread. 4BSD handles this case more correctly. > > > >> > It has always seemed weird to me that the priorities in the kernel are > >> > strictly higher than user-space -- but only after a prio change like > >> > that done implicitly by many of the calls to sleep(9). So it may be > >> > that the better patch is to use PRI_USER, not PRI_UNCHANGED, which > >> > would revert any potentially changed thread prio (e.g. due to a > >> > sleep(9)) back to its user-space level, so that it contended as > >> > expected with other threads. > > > > Realtime priorities (for rtprio user threads) are higher than the kernel > > "sleep" priorities. Also, keep in mind that a thread does not get an > > automatic priority boost when it enters the kernel. It only gets a boost > > either temporarily from priority propagation, or a slightly longer (but > > still temporary) boost from a sleep(9) call. > > I still don't understand why threads are boosting their priority while > sleeping; if it's so they are woken preferentially, that makes some > sense, but then they shouldn't be keeping the boosted priority after > the thread is woken. If a thread really needs higher priority to do > its work, that should be an explicit call to sched_prio(9), rather > than implicitly only when/if it first sleeps. This is how the BSD scheduler traditionally boosted the priority of "interactive" threads to favor them over CPU-bound threads. That is, BSD assumed that interactive threads are going to be blocking in the kernel (on a socket, on a tty, etc.), then running for a little bit after waking before blocking again. The boost when you sleep is to let threads with that work pattern always have a higher priority than CPU-bound threads (which will always have priorities in the timeshare range). You have to keep the boost long enough to ensure you get some user time however, so the threads keep the raised priority out to the userland boundary where it is dropped in userret(), but it is intentionally done in a careful manner where the thread drops its priority but does not do a reschedule so it will still run out in userland with the "effective" priority from its boost until the next schedule event happens (timeslice expires, thread blocks, etc.). Preemption from ithreads makes scheduling events happen far more often though (prior to SMPng, interrupts didn't trigger a reschedule), so interactive threads perhaps don't get favored quite as much since 5. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 12 21:18:09 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 951531065677; Mon, 12 Dec 2011 21:18:09 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 59AC48FC1C; Mon, 12 Dec 2011 21:18:09 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id 04E9A46B0D; Mon, 12 Dec 2011 16:18:09 -0500 (EST) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 795A7B948; Mon, 12 Dec 2011 16:18:08 -0500 (EST) From: John Baldwin To: mdf@freebsd.org Date: Mon, 12 Dec 2011 16:18:07 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p8; KDE/4.5.5; amd64; ; ) References: <4EE51CB5.1060505@FreeBSD.org> <201112121405.38322.jhb@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201112121618.07833.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 12 Dec 2011 16:18:08 -0500 (EST) Cc: usb@freebsd.org, freebsd-hackers@freebsd.org, hackers@freebsd.org, Andriy Gapon , Hans Petter Selasky Subject: Re: kern_yield vs ukbd_yield X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2011 21:18:09 -0000 On Monday, December 12, 2011 2:13:49 pm mdf@freebsd.org wrote: > On Mon, Dec 12, 2011 at 11:05 AM, John Baldwin wrote: > > On Monday, December 12, 2011 10:58:22 am Hans Petter Selasky wrote: > >> On Monday 12 December 2011 16:55:38 mdf@freebsd.org wrote: > >> > On Mon, Dec 12, 2011 at 12:03 AM, Andriy Gapon wrote: > >> > > on 11/12/2011 23:48 mdf@FreeBSD.org said the following: > >> > >> On Sun, Dec 11, 2011 at 1:12 PM, Andriy Gapon wrote: > >> > >>> Does the following change do what I think that it does? > >> > >>> Thank you! > >> > >>> > >> > >>> Author: Andriy Gapon > >> > >>> Date: Thu Sep 1 16:50:13 2011 +0300 > >> > >>> > >> > >>> ukbd: drop local duplicate of kern_yield and use that instead > >> > >>> > >> > >>> diff --git a/sys/dev/usb/input/ukbd.c b/sys/dev/usb/input/ukbd.c > >> > >>> index 086c178..8078cbb 100644 > >> > >>> --- a/sys/dev/usb/input/ukbd.c > >> > >>> +++ b/sys/dev/usb/input/ukbd.c > >> > >>> @@ -399,33 +399,6 @@ ukbd_put_key(struct ukbd_softc *sc, uint32_t key) > >> > >>> } > >> > >>> > >> > >>> static void > >> > >>> -ukbd_yield(void) > >> > >>> -{ > >> > >>> - struct thread *td = curthread; > >> > >>> - uint32_t old_prio; > >> > >>> - > >> > >>> - DROP_GIANT(); > >> > >>> - > >> > >>> - thread_lock(td); > >> > >>> - > >> > >>> - /* get current priority */ > >> > >>> - old_prio = td->td_base_pri; > >> > >>> - > >> > >>> - /* set new priority */ > >> > >>> - sched_prio(td, td->td_user_pri); > >> > >>> - > >> > >>> - /* cause a task switch */ > >> > >>> - mi_switch(SW_INVOL | SWT_RELINQUISH, NULL); > >> > >>> - > >> > >>> - /* restore priority */ > >> > >>> - sched_prio(td, old_prio); > >> > >>> - > >> > >>> - thread_unlock(td); > >> > >>> - > >> > >>> - PICKUP_GIANT(); > >> > >>> -} > >> > >>> - > >> > >>> -static void > >> > >>> ukbd_do_poll(struct ukbd_softc *sc, uint8_t wait) > >> > >>> { > >> > >>> > >> > >>> @@ -439,7 +412,7 @@ ukbd_do_poll(struct ukbd_softc *sc, uint8_t wait) > >> > >>> while (sc->sc_inputs == 0) { > >> > >>> > >> > >>> /* give USB threads a chance to run */ > >> > >>> - ukbd_yield(); > >> > >>> + kern_yield(-1); > >> > >> > >> > >> Not quite. > >> > >> > >> > >> 1) -1 should be spelled PRI_UNCHANGED, except ukbd_yield() uses > >> > >> td_user_pri, but then puts it back again, so I think UNCHANGED is what > >> > >> is meant. > >> > >> 2) kern_yield() calls it a SW_VOL rather than SW_INVOL, which seems > >> > >> the desired behaviour here anyways, since this is an explicit (i.e. > >> > >> voluntary) yield. > >> > > > >> > > Thank you for the explanation. So would you say that the patch is OK? > >> > > >> > As far as I know, yes. There may be a difference in behaviour, > >> > though, while yielding, if the priority of the thread remains high (as > >> > this change would make it) -- I'm not completely sure how the > >> > scheduler chooses threads, because I'm pretty sure I've seen it take > >> > threads with lower (higher numbered) priorities even when there's > >> > runnable threads with a higher (lower numbered) priority available. > > > > The scheduler generally should not do this with the following exceptions: > > > > 1) for timesharing threads, priorities are in bands, so effectively the > > pri / 4 is what is used for comparison and if two threads have the same > > pri / 4 the scheduler will round-robin among htem. > > > > 2) ULE might be a bit different because of the way it assigns threads to > > CPUs, so if a CPU has two high priority threads and another CPU only > > has a low priority thread, the second CPU will not run the second high > > priority thread. 4BSD handles this case more correctly. > > > >> > It has always seemed weird to me that the priorities in the kernel are > >> > strictly higher than user-space -- but only after a prio change like > >> > that done implicitly by many of the calls to sleep(9). So it may be > >> > that the better patch is to use PRI_USER, not PRI_UNCHANGED, which > >> > would revert any potentially changed thread prio (e.g. due to a > >> > sleep(9)) back to its user-space level, so that it contended as > >> > expected with other threads. > > > > Realtime priorities (for rtprio user threads) are higher than the kernel > > "sleep" priorities. Also, keep in mind that a thread does not get an > > automatic priority boost when it enters the kernel. It only gets a boost > > either temporarily from priority propagation, or a slightly longer (but > > still temporary) boost from a sleep(9) call. > > I still don't understand why threads are boosting their priority while > sleeping; if it's so they are woken preferentially, that makes some > sense, but then they shouldn't be keeping the boosted priority after > the thread is woken. If a thread really needs higher priority to do > its work, that should be an explicit call to sched_prio(9), rather > than implicitly only when/if it first sleeps. This is how the BSD scheduler traditionally boosted the priority of "interactive" threads to favor them over CPU-bound threads. That is, BSD assumed that interactive threads are going to be blocking in the kernel (on a socket, on a tty, etc.), then running for a little bit after waking before blocking again. The boost when you sleep is to let threads with that work pattern always have a higher priority than CPU-bound threads (which will always have priorities in the timeshare range). You have to keep the boost long enough to ensure you get some user time however, so the threads keep the raised priority out to the userland boundary where it is dropped in userret(), but it is intentionally done in a careful manner where the thread drops its priority but does not do a reschedule so it will still run out in userland with the "effective" priority from its boost until the next schedule event happens (timeslice expires, thread blocks, etc.). Preemption from ithreads makes scheduling events happen far more often though (prior to SMPng, interrupts didn't trigger a reschedule), so interactive threads perhaps don't get favored quite as much since 5. -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Mon Dec 12 22:21:13 2011 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 623181065672; Mon, 12 Dec 2011 22:21:13 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id B7D458FC08; Mon, 12 Dec 2011 22:21:11 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id AAA29084; Tue, 13 Dec 2011 00:21:06 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1RaEF7-000Ibl-Pz; Tue, 13 Dec 2011 00:21:05 +0200 Message-ID: <4EE67E4F.3010906@FreeBSD.org> Date: Tue, 13 Dec 2011 00:21:03 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111206 Thunderbird/8.0 MIME-Version: 1.0 To: Hans Petter Selasky References: <4EE51CB5.1060505@FreeBSD.org> <201112121658.22864.hselasky@c2i.net> <201112121405.38322.jhb@freebsd.org> <201112122009.12557.hselasky@c2i.net> In-Reply-To: <201112122009.12557.hselasky@c2i.net> X-Enigmail-Version: undefined Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: usb@FreeBSD.org, freebsd-hackers@FreeBSD.org, John Baldwin , mdf@FreeBSD.org Subject: Re: kern_yield vs ukbd_yield X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2011 22:21:13 -0000 on 12/12/2011 21:09 Hans Petter Selasky said the following: > On Monday 12 December 2011 20:05:38 John Baldwin wrote: >>> Hi, >>> >>> >>> >>>> hselasky@ or someone else familiar with the various usb threads would >>>> have to answer that. >>> >>> >>> >>> The problem is only during init() where the init thread has highest >>> priority and that doesn't allow other threads to run even if the >>> scheduler is >> >> running! >> >> Hmm, that should be fixed by lowering the relevant thread's priority. >> Do you mean thread0 (the one doing all the SYSINIT's or thread we create >> for init (pid 1) before it executes init? > > Yes, thread0. Correct! Mmm, my short debugging session with qemu shows that it is actually pid 1: Breakpoint 1, ukbd_do_poll (sc=0xffffff8000764000, wait=0 '\0') at /usr/src/sys/dev/usb/input/ukbd.c:403 403 { (kgdb) bt #0 ukbd_do_poll (sc=0xffffff8000764000, wait=0 '\0') at /usr/src/sys/dev/usb/input/ukbd.c:403 #1 0xffffffff803ee57e in ukbd_check (kbd=Variable "kbd" is not available. ) at /usr/src/sys/dev/usb/input/ukbd.c:1418 #2 0xffffffff803ee674 in ukbd_check_char (kbd=0xffffff8000764000) at /usr/src/sys/dev/usb/input/ukbd.c:1450 #3 0xffffffff8035c794 in kbdmux_read_char (kbd=0xfffffe00021d2c00, wait=Variable "wait" is not available. ) at /usr/src/sys/dev/kbdmux/kbdmux.c:665 #4 0xffffffff803bf4aa in scgetc (sc=0xffffffff81835dc0, flags=Variable "flags" is not available. ) at /usr/src/sys/dev/syscons/syscons.c:3373 #5 0xffffffff803c2d03 in sc_cngetc (cd=Variable "cd" is not available. ) at /usr/src/sys/dev/syscons/syscons.c:1753 #6 0xffffffff804ae489 in cncheckc () at /usr/src/sys/kern/kern_cons.c:402 #7 0xffffffff804ae4c7 in cngetc () at /usr/src/sys/kern/kern_cons.c:383 #8 0xffffffff804ae9a7 in cngets (cp=0xffffff8000326aa0 "", size=80, visible=1) at /usr/src/sys/kern/kern_cons.c:421 #9 0xffffffff80594d4e in vfs_mountroot () at /usr/src/sys/kern/vfs_mountroot.c:490 #10 0xffffffff804a6e65 in start_init (dummy=Variable "dummy" is not available. ) at /usr/src/sys/kern/init_main.c:683 #11 0xffffffff804c5a9a in fork_exit (callout=0xffffffff804a6e00 , arg=0x0, frame=0xffffff8000326c50) at /usr/src/sys/kern/kern_fork.c:995 #12 0xffffffff806bb72e in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:602 #13 0x0000000000000000 in ?? () (kgdb) p td->td_proc->p_pid $7 = 1 (kgdb) p td->td_proc->p_comm $8 = "kernel", '\0' (kgdb) p td->td_tid $9 = 100002 Also: (kgdb) p/d td->td_user_pri $10 = 139 (kgdb) p/d td->td_base_pri $11 = 84 -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Tue Dec 13 08:17:47 2011 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 35AA5106566B; Tue, 13 Dec 2011 08:17:47 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id C870F8FC14; Tue, 13 Dec 2011 08:17:45 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id KAA06506; Tue, 13 Dec 2011 10:17:38 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1RaNYP-000LLp-EZ; Tue, 13 Dec 2011 10:17:37 +0200 Message-ID: <4EE70A1F.8040406@FreeBSD.org> Date: Tue, 13 Dec 2011 10:17:35 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111206 Thunderbird/8.0 MIME-Version: 1.0 To: Hans Petter Selasky References: <4EE51CB5.1060505@FreeBSD.org> <201112121658.22864.hselasky@c2i.net> <201112121405.38322.jhb@freebsd.org> <201112122009.12557.hselasky@c2i.net> <4EE67E4F.3010906@FreeBSD.org> In-Reply-To: <4EE67E4F.3010906@FreeBSD.org> X-Enigmail-Version: undefined Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: usb@FreeBSD.org, freebsd-hackers@FreeBSD.org, John Baldwin , mdf@FreeBSD.org Subject: Re: kern_yield vs ukbd_yield X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 08:17:47 -0000 on 13/12/2011 00:21 Andriy Gapon said the following: > on 12/12/2011 21:09 Hans Petter Selasky said the following: >> On Monday 12 December 2011 20:05:38 John Baldwin wrote: >>>> Hi, >>>> >>>> >>>> >>>>> hselasky@ or someone else familiar with the various usb threads would >>>>> have to answer that. >>>> >>>> >>>> >>>> The problem is only during init() where the init thread has highest >>>> priority and that doesn't allow other threads to run even if the >>>> scheduler is >>> >>> running! >>> >>> Hmm, that should be fixed by lowering the relevant thread's priority. >>> Do you mean thread0 (the one doing all the SYSINIT's or thread we create >>> for init (pid 1) before it executes init? >> >> Yes, thread0. Correct! > > Mmm, my short debugging session with qemu shows that it is actually pid 1: And in the view of the below data I would like us to revisit this problem. I looked over usb code and it seems that all usb threads are created with priorities of either USB_PRI_MED or USB_PRI_HIGH, which translates to PI_SWI(SWI_CAMBIO) and PI_SWI(SWI_NET). These priorities should be in the ithread range, so it's kind of surprising that the init thread (with PVM priority) can cause troubles for them. > Breakpoint 1, ukbd_do_poll (sc=0xffffff8000764000, wait=0 '\0') at > /usr/src/sys/dev/usb/input/ukbd.c:403 > 403 { > (kgdb) bt > #0 ukbd_do_poll (sc=0xffffff8000764000, wait=0 '\0') at > /usr/src/sys/dev/usb/input/ukbd.c:403 > #1 0xffffffff803ee57e in ukbd_check (kbd=Variable "kbd" is not available. > ) at /usr/src/sys/dev/usb/input/ukbd.c:1418 > #2 0xffffffff803ee674 in ukbd_check_char (kbd=0xffffff8000764000) at > /usr/src/sys/dev/usb/input/ukbd.c:1450 > #3 0xffffffff8035c794 in kbdmux_read_char (kbd=0xfffffe00021d2c00, > wait=Variable "wait" is not available. > ) at /usr/src/sys/dev/kbdmux/kbdmux.c:665 > #4 0xffffffff803bf4aa in scgetc (sc=0xffffffff81835dc0, flags=Variable "flags" > is not available. > ) at /usr/src/sys/dev/syscons/syscons.c:3373 > #5 0xffffffff803c2d03 in sc_cngetc (cd=Variable "cd" is not available. > ) at /usr/src/sys/dev/syscons/syscons.c:1753 > #6 0xffffffff804ae489 in cncheckc () at /usr/src/sys/kern/kern_cons.c:402 > #7 0xffffffff804ae4c7 in cngetc () at /usr/src/sys/kern/kern_cons.c:383 > #8 0xffffffff804ae9a7 in cngets (cp=0xffffff8000326aa0 "", size=80, visible=1) > at /usr/src/sys/kern/kern_cons.c:421 > #9 0xffffffff80594d4e in vfs_mountroot () at /usr/src/sys/kern/vfs_mountroot.c:490 > #10 0xffffffff804a6e65 in start_init (dummy=Variable "dummy" is not available. > ) at /usr/src/sys/kern/init_main.c:683 > #11 0xffffffff804c5a9a in fork_exit (callout=0xffffffff804a6e00 , > arg=0x0, frame=0xffffff8000326c50) at /usr/src/sys/kern/kern_fork.c:995 > #12 0xffffffff806bb72e in fork_trampoline () at > /usr/src/sys/amd64/amd64/exception.S:602 > #13 0x0000000000000000 in ?? () > (kgdb) p td->td_proc->p_pid > $7 = 1 > (kgdb) p td->td_proc->p_comm > $8 = "kernel", '\0' > (kgdb) p td->td_tid > $9 = 100002 > > Also: > (kgdb) p/d td->td_user_pri > $10 = 139 > (kgdb) p/d td->td_base_pri > $11 = 84 > -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Tue Dec 13 13:11:54 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9A30A106564A for ; Tue, 13 Dec 2011 13:11:54 +0000 (UTC) (envelope-from monthadar@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 6B34A8FC1D for ; Tue, 13 Dec 2011 13:11:54 +0000 (UTC) Received: by iakl21 with SMTP id l21so7098626iak.13 for ; Tue, 13 Dec 2011 05:11:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; bh=023aYy6Liv+w3G677lQVoO0m7pIqXbW49x9RcPM8d3o=; b=SkE/w6MeGqJ4QkPH40Qdp0/YtbqOQ5Ibz8bW8xrlQfXHJxBkkkPKkDdgll8NisnO1f RNB0BJizgl8V0uEtIlCuPuwRpJgdFVH9C+yyXRMonst3bAtXy7zb+5p4Mb8eSesTfzlH 9WaxyE3pZfy6aBHwVja6vul1kKASMxCf0r9DY= MIME-Version: 1.0 Received: by 10.50.220.164 with SMTP id px4mr19442153igc.8.1323780394366; Tue, 13 Dec 2011 04:46:34 -0800 (PST) Received: by 10.50.51.233 with HTTP; Tue, 13 Dec 2011 04:46:34 -0800 (PST) Date: Tue, 13 Dec 2011 13:46:34 +0100 Message-ID: From: Monthadar Al Jaberi To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: loop inside uma_zfree critical section X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 13:11:54 -0000 Hi, I am not sure why I am having this problem, but looking at the code I dont understand uma_core.c really good. So I hope someone can shed a light on this: I am running on an arm board and and running a kernel module that behaves like a wlan interface. so I tx and rx packets. For now tx is only only sending beacon like frames. This is done through using ieee80211_beacon_alloc(). Then in a callout task to generate periodic beacons: m_dup(avp->beacon, M_DONTWAIT); mtx_lock(...); STAILQ_INSERT_TAIL(...); taskqueue_enqueue(...); mtx_unlock(...); callout_schedule(...); On the RX side, the interrupt handler will read out buffer then place it on a queue to be handled by wlan-glue code. For now wlan-glue code just frees the mbuf it instead of calling net80211 ieee80211_input() functions: m_copyback(...); /* Allocate new mbuf for next RX. */ MGETHDR(..., M_DONTWAIT, MT_DATA); bzero((mtod(sc->Rx_m, void *)), MHLEN); sc->Rx_m->m_len = 0; /* NB: m_gethdr does not set */ sc->Rx_m->m_data += 20; /* make headroom */ Then I use a lockmgr inside my kernel module that should make sure that we either are on TX or RX path. Here is an output of when it loops forever and a debug output. I enabled some of the printfs in vm/uma_core.c: KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2011 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.0-CURRENT #931: Tue Dec 13 11:35:17 UTC 2011 ... beacon_callout. becon m_dup: new 0xc0d99c00, beacon: 0xc0eb0800. queue beacon for TX. LOCKMGR: TX path locked sending... Free TX mbuf: 0xc0d99c00 LOCKMGR: TX path unlocked LOCKMGR: RX path locked current RX mbuf: 0xc0d99900 queue packet for wlan-glue allocating new Rx mbuf: 0xc0d99c00 LOCKMGR: RX path unlocked wlan-glue task running... wlan-glue code freeing mbuf: 0xc0d99900 beacon_callout. becon m_dup: new 0xc0d99b00, beacon: 0xc0eb0800. queue beacon for TX. LOCKMGR: TX path locked sending... Free TX mbuf: 0xc0d99b00 LOCKMGR: TX path unlocked LOCKMGR: RX path locked current RX mbuf: 0xc0d99c00 queue packet for wlan-glue allocating new Rx mbuf: 0xc0d99b00 [1] LOCKMGR: RX path unlocked wlan-glue task running... uma_zfree: Swapping buckets. [2] wlan-glue code freeing mbuf: 0xc0d99c00 beacon_callout. becon m_dup: new 0xc0d99b00, beacon: 0xc0eb0800. [3] queue beacon for TX. LOCKMGR: TX path locked sending... Free TX mbuf: 0xc0d99b00 [4] LOCKMGR: TX path unlocked LOCKMGR: RX path locked current RX mbuf: 0xc0d99b00 queue packet for wlan-glue allocating new Rx mbuf: 0xc0d99900 LOCKMGR: RX path unlocked wlan-glue task running... [5] uma_zfree: Swapping buckets. uma_zfree: Swapping buckets. uma_zfree: Putting old bucket on the free list. uma_zfree: Allocating new free bucket. uma_zfree: Swapping buckets. uma_zfree: Putting old bucket on the free list. uma_zfree: Allocating new free bucket. uma_zfree: Swapping buckets. uma_zfree: Putting old bucket on the free list. uma_zfree: Allocating new free bucket. uma_zfree: Swapping buckets. uma_zfree: Putting old bucket on the free list. uma_zfree: Allocating new free bucket. uma_zfree: Swapping buckets. uma_zfree: Putting old bucket on the free list. uma_zfree: Allocating new free bucket. uma_zfree: Swapping buckets. uma_zfree: Putting old bucket on the free list. uma_zfree: Allocating new free bucket. uma_zfree: Swapping buckets. uma_zfree: Putting old bucket on the free list. uma_zfree: Allocating new free bucket. uma_zfree: Swapping buckets. uma_zfree: Putting old bucket on the free list. uma_zfree: Allocating new free bucket. uma_zfree: Swapping buckets. uma_zfree: Putting old bucket on the free list. uma_zfree: Allocating new free bucket. uma_zfree: Swapping buckets. ... The problem seems to be at [2], somehow after swapping buckets in uma_zfree m_dup returns a pointer to an mbuf that is still being used by us, [1] and [3] have same address. Then we call m_freem twice on same mbuf, [4] and [5]. And a loop occurs inside uma_free. I am using mbufs in a wrong way? Shouldnt mbufs be thread safe? Problem seems to occur while swapping buckets. Br, -- Monthadar Al Jaberi From owner-freebsd-hackers@FreeBSD.ORG Tue Dec 13 14:35:35 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5CA061065687 for ; Tue, 13 Dec 2011 14:35:35 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 3369E8FC0A for ; Tue, 13 Dec 2011 14:35:35 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id DE92D46B0D; Tue, 13 Dec 2011 09:35:34 -0500 (EST) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 764AFB95B; Tue, 13 Dec 2011 09:35:34 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Tue, 13 Dec 2011 09:35:33 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p8; KDE/4.5.5; amd64; ; ) References: In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201112130935.33975.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 13 Dec 2011 09:35:34 -0500 (EST) Cc: Monthadar Al Jaberi Subject: Re: loop inside uma_zfree critical section X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 14:35:35 -0000 On Tuesday, December 13, 2011 7:46:34 am Monthadar Al Jaberi wrote: > Hi, > > I am not sure why I am having this problem, but looking > at the code I dont understand uma_core.c really good. > So I hope someone can shed a light on this: > > I am running on an arm board and and running a kernel module > that behaves like a wlan interface. so I tx and rx packets. > > For now tx is only only sending beacon like frames. > This is done through using ieee80211_beacon_alloc(). > > Then in a callout task to generate periodic beacons: > > m_dup(avp->beacon, M_DONTWAIT); > mtx_lock(...); > STAILQ_INSERT_TAIL(...); > taskqueue_enqueue(...); > mtx_unlock(...); > callout_schedule(...); > > On the RX side, the interrupt handler will read out buffer > then place it on a queue to be handled by wlan-glue code. > For now wlan-glue code just frees the mbuf it instead of > calling net80211 ieee80211_input() functions: > > m_copyback(...); > /* Allocate new mbuf for next RX. */ > MGETHDR(..., M_DONTWAIT, MT_DATA); > bzero((mtod(sc->Rx_m, void *)), MHLEN); > sc->Rx_m->m_len = 0; /* NB: m_gethdr does not set */ > sc->Rx_m->m_data += 20; /* make headroom */ > > Then I use a lockmgr inside my kernel module that should > make sure that we either are on TX or RX path. Uh, you can't use a lockmgr lock in interrupt handlers or in if_transmit/if_start routines. You should most likely just be using a plain mutex instead. Also, new code shouldn't use lockmgr in general. If you need a sleepable lock, use sx instead. It has a more straightforward API. > The problem seems to be at [2], somehow after swapping > buckets in uma_zfree m_dup returns a pointer to > an mbuf that is still being used by us, [1] and [3] > have same address. > Then we call m_freem twice on same mbuf, [4] and [5]. > And a loop occurs inside uma_free. > I am using mbufs in a wrong way? Shouldnt mbufs be thread safe? > Problem seems to occur while swapping buckets. Hmm, the uma uses its own locking, so it should be safe, yes. However, you are correct about [1] and [3]. The thing is, after [1] the mbuf shouldn't be in any buckets at all (it only gets put back into the bucket during a free). Are you sure the mbuf wasn't double free'd previously? -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Tue Dec 13 15:50:01 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B96C11065672 for ; Tue, 13 Dec 2011 15:50:01 +0000 (UTC) (envelope-from monthadar@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 729FE8FC14 for ; Tue, 13 Dec 2011 15:50:01 +0000 (UTC) Received: by iakl21 with SMTP id l21so7394071iak.13 for ; Tue, 13 Dec 2011 07:50:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=slrp51mCFyNxfkAg9IVF5SIAAkEa4Q/LO4fzww/FSp0=; b=ka+DGgHjyV87+iTZCdYaZ4KlzZYk2UkaH0TBvfj8ctip+rJGMY8IkTLjlnZY1MEC3J s8DTwfTaw0FvEEtuFD640B3fOm/y54qLWEfiSj8crLnZXkXEslaDfUzPubC8rf6m5s1b yyqMdvSwPWKrM0JRJIUG7av3gIzYsxWbRcOAc= MIME-Version: 1.0 Received: by 10.42.72.135 with SMTP id o7mr17074122icj.45.1323791400786; Tue, 13 Dec 2011 07:50:00 -0800 (PST) Received: by 10.50.51.233 with HTTP; Tue, 13 Dec 2011 07:50:00 -0800 (PST) In-Reply-To: <201112130935.33975.jhb@freebsd.org> References: <201112130935.33975.jhb@freebsd.org> Date: Tue, 13 Dec 2011 16:50:00 +0100 Message-ID: From: Monthadar Al Jaberi To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: loop inside uma_zfree critical section X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 15:50:01 -0000 On Tue, Dec 13, 2011 at 3:35 PM, John Baldwin wrote: > On Tuesday, December 13, 2011 7:46:34 am Monthadar Al Jaberi wrote: >> Hi, >> >> I am not sure why I am having this problem, but looking >> at the code I dont understand uma_core.c really good. >> So I hope someone can shed a light on this: >> >> I am running on an arm board and and running a kernel module >> that behaves like a wlan interface. so I tx and rx packets. >> >> For now tx is only only sending beacon like frames. >> This is done through using ieee80211_beacon_alloc(). >> >> Then in a callout task to generate periodic beacons: >> >> =A0 =A0 m_dup(avp->beacon, M_DONTWAIT); >> =A0 =A0 mtx_lock(...); >> =A0 =A0 STAILQ_INSERT_TAIL(...); >> =A0 =A0 taskqueue_enqueue(...); >> =A0 =A0 mtx_unlock(...); >> =A0 =A0 callout_schedule(...); >> >> On the RX side, the interrupt handler will read out buffer >> then place it on a queue to be handled by wlan-glue code. >> For now wlan-glue code just frees the mbuf it instead of >> calling net80211 ieee80211_input() functions: >> >> =A0 =A0 m_copyback(...); >> =A0 =A0 /* Allocate new mbuf for next RX. */ >> =A0 =A0 MGETHDR(..., M_DONTWAIT, MT_DATA); >> =A0 =A0 bzero((mtod(sc->Rx_m, void *)), MHLEN); >> =A0 =A0 sc->Rx_m->m_len =3D 0; /* NB: m_gethdr does not set */ >> =A0 =A0 sc->Rx_m->m_data +=3D 20; /* make headroom */ >> >> Then I use a lockmgr inside my kernel module that should >> make sure that we either are on TX or RX path. > > Uh, you can't use a lockmgr lock in interrupt handlers or in > if_transmit/if_start routines. =A0You should most likely just be using a = plain > mutex instead. =A0Also, new code shouldn't use lockmgr in general. =A0If = you > need a sleepable lock, use sx instead. =A0It has a more straightforward A= PI. Ok, I will change the interrupt handler to do something like this: disaple_interrupt(); taskqueue_enqueue(...); /* on new rx task queue */ Then on the new rx proc: sx_slock(...); m_copyback(...); enable_interrupt(); /* Allocate new mbuf for next RX. */ MGETHDR(..., M_DONTWAIT, MT_DATA); bzero((mtod(sc->Rx_m, void *)), MHLEN); sc->Rx_m->m_len =3D 0; /* NB: m_gethdr does not set */ sc->Rx_m->m_data +=3D 20; /* make headroom */ sx_sunlock(...); I lock TX/RX paths to make sure my code is threading safe. Also because while programming my deivce (SPI communicatioin) there will be a tsleep() waiting for the DMA interrupt and thus we could be prempted by e.g. a beacon_callout etc... > >> The problem seems to be at [2], somehow after swapping >> buckets in uma_zfree m_dup returns a pointer to >> an mbuf that is still being used by us, [1] and [3] >> have same address. >> Then we call m_freem twice on same mbuf, [4] and [5]. >> And a loop occurs inside uma_free. >> I am using mbufs in a wrong way? Shouldnt mbufs be thread safe? >> Problem seems to occur while swapping buckets. > > Hmm, the uma uses its own locking, so it should be safe, yes. =A0However,= you > are correct about [1] and [3]. =A0The thing is, after [1] the mbuf should= n't > be in any buckets at all (it only gets put back into the bucket during a > free). =A0Are you sure the mbuf wasn't double free'd previously? >From my log I can only see the mbuf being used once before by a beacon_callout and then it was freed by m_freem(). So I cant see that it was freed twice before that. How can I go by debbuging this? > > -- > John Baldwin --=20 Monthadar Al Jaberi From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 14 13:21:33 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 644C2106567C for ; Wed, 14 Dec 2011 13:21:33 +0000 (UTC) (envelope-from hugo@barafranca.com) Received: from mail.barafranca.com (mail.barafranca.com [67.213.67.47]) by mx1.freebsd.org (Postfix) with ESMTP id 380238FC21 for ; Wed, 14 Dec 2011 13:21:33 +0000 (UTC) Received: from localhost (unknown [172.16.100.24]) by mail.barafranca.com (Postfix) with ESMTP id 3E61DD48 for ; Wed, 14 Dec 2011 13:10:18 +0000 (UTC) X-Virus-Scanned: amavisd-new at barafranca.com Received: from mail.barafranca.com ([172.16.100.24]) by localhost (mail.barafranca.com [172.16.100.24]) (amavisd-new, port 10024) with ESMTP id TdpHt84p3WZA for ; Wed, 14 Dec 2011 13:09:32 +0000 (UTC) Received: from [192.168.1.1] (a89-153-50-11.cpe.netcabo.pt [89.153.50.11]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.barafranca.com (Postfix) with ESMTPSA id 0A428D3A for ; Wed, 14 Dec 2011 13:09:31 +0000 (UTC) Message-ID: <4EE8A005.5030607@barafranca.com> Date: Wed, 14 Dec 2011 13:09:25 +0000 From: Hugo Silva User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.13) Gecko/20101208 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: mfi (Dell H700) + hot swapping doesn't appear to work with RC1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2011 13:21:33 -0000 Hello, First of all apologies if this has been fixed in RC3. I set this server up with mfsbsd, which is RC1, and didn't get to update the system yet. This box has 6 hdds, a 2-mirror zpool was set up as the root pool, with 2 spares. While testing hot swapping I noticed that while the controller detects disk removal/insertion, the zpool will never recover. The problem seems to be deeper than ZFS, as disklabel/fdisk/etc also fail on the removed-and-reinserted disk. At the ZFS level, doing a zpool clear yields more errors on the removed disk; rebooting becomes the only option to make the pool healthy again. Is this normal? Did I miss any step? Regards, Hugo From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 14 15:34:43 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 26936106564A for ; Wed, 14 Dec 2011 15:34:43 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from proxypop03.sare.net (proxypop03.sare.net [194.30.0.207]) by mx1.freebsd.org (Postfix) with ESMTP id DB4E98FC0A for ; Wed, 14 Dec 2011 15:34:42 +0000 (UTC) Received: from [172.16.2.2] (izaro.sarenet.es [192.148.167.11]) by proxypop03.sare.net (Postfix) with ESMTPSA id 051349DC67C; Wed, 14 Dec 2011 16:16:19 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Borja Marcos In-Reply-To: <4EE8A005.5030607@barafranca.com> Date: Wed, 14 Dec 2011 16:16:19 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <9317551F-CBE0-4368-B798-498E58E240B2@sarenet.es> References: <4EE8A005.5030607@barafranca.com> To: Hugo Silva X-Mailer: Apple Mail (2.1084) Cc: freebsd-hackers@freebsd.org Subject: Re: mfi (Dell H700) + hot swapping doesn't appear to work with RC1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2011 15:34:43 -0000 On Dec 14, 2011, at 2:09 PM, Hugo Silva wrote: > Hello, >=20 > First of all apologies if this has been fixed in RC3. I set this = server > up with mfsbsd, which is RC1, and didn't get to update the system yet. >=20 > This box has 6 hdds, a 2-mirror zpool was set up as the root pool, = with > 2 spares. >=20 > While testing hot swapping I noticed that while the controller detects > disk removal/insertion, the zpool will never recover. The problem = seems > to be deeper than ZFS, as disklabel/fdisk/etc also fail on the > removed-and-reinserted disk. >=20 > At the ZFS level, doing a zpool clear yields more errors on the = removed > disk; rebooting becomes the only option to make the pool healthy = again. >=20 >=20 > Is this normal? Did I miss any step? I assume that you have tried to use the H700 as a "JBOD" card, defining = logical volume for each hard disk. The problem is: that gorgeous, fantastic, masterful, Nobel award = candidate card, has a wonderful behavior in that case. If you extract = one of the disks, the logical volume associated to it is invalidated. = So, you insert a replacement disk, and the card refuses to recognize the = volume. What is even worse, in order to recover it's mandatory to reboot = the complete system *AND* go through the RAID configuration utility. That's the problem. The card refuses to work as a simple disk = controller without frills, and the frills get in the way. To summarize: it isn't FreeBSD's fault, no matter which version you use. = It's a "feature" coming directly from the geniuses who designed the = card. Borja. From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 14 15:37:58 2011 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0589E106564A; Wed, 14 Dec 2011 15:37:58 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id AD5558FC08; Wed, 14 Dec 2011 15:37:56 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA08219; Wed, 14 Dec 2011 17:37:50 +0200 (EET) (envelope-from avg@FreeBSD.org) Message-ID: <4EE8C2CE.40909@FreeBSD.org> Date: Wed, 14 Dec 2011 17:37:50 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111109 Thunderbird/8.0 MIME-Version: 1.0 To: Hans Petter Selasky References: <4EE51CB5.1060505@FreeBSD.org> <201112121658.22864.hselasky@c2i.net> <201112121405.38322.jhb@freebsd.org> <201112122009.12557.hselasky@c2i.net> <4EE67E4F.3010906@FreeBSD.org> <4EE70A1F.8040406@FreeBSD.org> In-Reply-To: <4EE70A1F.8040406@FreeBSD.org> X-Enigmail-Version: undefined Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: usb@FreeBSD.org, freebsd-hackers@FreeBSD.org, John Baldwin , mdf@FreeBSD.org Subject: Re: kern_yield vs ukbd_yield X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2011 15:37:58 -0000 on 13/12/2011 10:17 Andriy Gapon said the following: > on 13/12/2011 00:21 Andriy Gapon said the following: [snip] > And in the view of the below data I would like us to revisit this problem. > I looked over usb code and it seems that all usb threads are created with > priorities of either USB_PRI_MED or USB_PRI_HIGH, which translates to > PI_SWI(SWI_CAMBIO) and PI_SWI(SWI_NET). These priorities should be in the > ithread range, so it's kind of surprising that the init thread (with PVM > priority) can cause troubles for them. So, Hans Petter, do you recall any details of this problem? I am curious about which thread got starved by which. BTW, given your recent improvements to pause(9) what do you think about further extending it to also use DELAY(9) when kdb_active is set or when SCHEDULER_STOPPED() is true? Then, probably, pause(9) could be used for both branches in ukbd_do_poll and they could be collapsed together. >> Breakpoint 1, ukbd_do_poll (sc=0xffffff8000764000, wait=0 '\0') at >> /usr/src/sys/dev/usb/input/ukbd.c:403 >> 403 { >> (kgdb) bt >> #0 ukbd_do_poll (sc=0xffffff8000764000, wait=0 '\0') at >> /usr/src/sys/dev/usb/input/ukbd.c:403 >> #1 0xffffffff803ee57e in ukbd_check (kbd=Variable "kbd" is not available. >> ) at /usr/src/sys/dev/usb/input/ukbd.c:1418 >> #2 0xffffffff803ee674 in ukbd_check_char (kbd=0xffffff8000764000) at >> /usr/src/sys/dev/usb/input/ukbd.c:1450 >> #3 0xffffffff8035c794 in kbdmux_read_char (kbd=0xfffffe00021d2c00, >> wait=Variable "wait" is not available. >> ) at /usr/src/sys/dev/kbdmux/kbdmux.c:665 >> #4 0xffffffff803bf4aa in scgetc (sc=0xffffffff81835dc0, flags=Variable "flags" >> is not available. >> ) at /usr/src/sys/dev/syscons/syscons.c:3373 >> #5 0xffffffff803c2d03 in sc_cngetc (cd=Variable "cd" is not available. >> ) at /usr/src/sys/dev/syscons/syscons.c:1753 >> #6 0xffffffff804ae489 in cncheckc () at /usr/src/sys/kern/kern_cons.c:402 >> #7 0xffffffff804ae4c7 in cngetc () at /usr/src/sys/kern/kern_cons.c:383 >> #8 0xffffffff804ae9a7 in cngets (cp=0xffffff8000326aa0 "", size=80, visible=1) >> at /usr/src/sys/kern/kern_cons.c:421 >> #9 0xffffffff80594d4e in vfs_mountroot () at /usr/src/sys/kern/vfs_mountroot.c:490 >> #10 0xffffffff804a6e65 in start_init (dummy=Variable "dummy" is not available. >> ) at /usr/src/sys/kern/init_main.c:683 >> #11 0xffffffff804c5a9a in fork_exit (callout=0xffffffff804a6e00 , >> arg=0x0, frame=0xffffff8000326c50) at /usr/src/sys/kern/kern_fork.c:995 >> #12 0xffffffff806bb72e in fork_trampoline () at >> /usr/src/sys/amd64/amd64/exception.S:602 >> #13 0x0000000000000000 in ?? () >> (kgdb) p td->td_proc->p_pid >> $7 = 1 >> (kgdb) p td->td_proc->p_comm >> $8 = "kernel", '\0' >> (kgdb) p td->td_tid >> $9 = 100002 >> >> Also: >> (kgdb) p/d td->td_user_pri >> $10 = 139 >> (kgdb) p/d td->td_base_pri >> $11 = 84 >> > > -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 14 19:10:50 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1D1E8106564A for ; Wed, 14 Dec 2011 19:10:50 +0000 (UTC) (envelope-from mashtizadeh@gmail.com) Received: from mail-pz0-f54.google.com (mail-pz0-f54.google.com [209.85.210.54]) by mx1.freebsd.org (Postfix) with ESMTP id EFADC8FC15 for ; Wed, 14 Dec 2011 19:10:49 +0000 (UTC) Received: by dakp5 with SMTP id p5so1235743dak.13 for ; Wed, 14 Dec 2011 11:10:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; bh=aWxR9+FUrHTAFKznpImF7YX30f6jUANete+l384S8qg=; b=vydar6cgN9+jfbF+y/hKXug4qgkNx85cwYPDO9NN3ow22/NVc+mUtPHZ2IoZyoMxbi DRnNPazvV9H8D7uEsoaGKrted4nosoedCQaZhCiqTELipW68IrFWDivJyviPWcl4D16b vTzZNHW86gPlQksaeCvhioZmrAyVoPAMX9RQ8= MIME-Version: 1.0 Received: by 10.68.209.3 with SMTP id mi3mr4987203pbc.16.1323888267301; Wed, 14 Dec 2011 10:44:27 -0800 (PST) Received: by 10.68.64.132 with HTTP; Wed, 14 Dec 2011 10:44:27 -0800 (PST) Date: Wed, 14 Dec 2011 10:44:27 -0800 Message-ID: From: Ali Mashtizadeh To: FreeBSD Hackers Content-Type: text/plain; charset=UTF-8 Subject: Building ports with gcc46 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2011 19:10:50 -0000 Is there a way to build devel/protobuf with gcc46? Unfortunately I see a compatibility issue where the software I'm linking against it crashes because of the conflicting stdc++ librray versions. I've tried setting CC, CXX, LDFLAGS but I seem to be missing something else? Thanks, -- Ali Mashtizadeh From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 14 19:38:57 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 38F541065672 for ; Wed, 14 Dec 2011 19:38:57 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-lpp01m010-f54.google.com (mail-lpp01m010-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id AAFA68FC17 for ; Wed, 14 Dec 2011 19:38:56 +0000 (UTC) Received: by lahl5 with SMTP id l5so896319lah.13 for ; Wed, 14 Dec 2011 11:38:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=Bq3wBXmBDd8FHnr+oDNjsE7kIJjhyDv7n0En0PyTJyM=; b=oFpFTbOKJvVmb0Hf0M4h+cBfz1DvYNYtQYjgQ1GH1Wl9r5G9zoAYf23+ZuU5JmMesA PqhXGUutRpjWq0kM98a5STRp/8Cwr6SLn/6XN3i15d+dssON7G4bbp9EzF1zgohZ5hch F+kn/fMk90ztLl3NfkVJ1CBeggbRH8oEAPu2U= Received: by 10.152.103.71 with SMTP id fu7mr4600230lab.31.1323890190340; Wed, 14 Dec 2011 11:16:30 -0800 (PST) MIME-Version: 1.0 Received: by 10.152.23.68 with HTTP; Wed, 14 Dec 2011 11:15:59 -0800 (PST) In-Reply-To: References: From: Eitan Adler Date: Wed, 14 Dec 2011 14:15:59 -0500 Message-ID: To: Ali Mashtizadeh Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Hackers Subject: Re: Building ports with gcc46 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2011 19:38:57 -0000 On Wed, Dec 14, 2011 at 1:44 PM, Ali Mashtizadeh wrote: > Is there a way to build devel/protobuf with gcc46? Unfortunately I see > a compatibility issue where the software I'm linking against it > crashes because of the conflicting stdc++ librray versions. I've tried > setting CC, CXX, LDFLAGS but I seem to be missing something else? There is a lot of ongoing work by various people to work multi-toolchain support in ports. At this time it is not officially supported. If you do manage to get it working, patches would certainly be appreciated. Additionally you may want to take a look at this (outdated) article: http://www.freebsd.org/doc/en/articles/custom-gcc/article.html This post should have been on ports@ not hackers@. -- Eitan Adler From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 14 19:47:21 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 62E4A1065672; Wed, 14 Dec 2011 19:47:21 +0000 (UTC) (envelope-from monthadar@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 1F8508FC0A; Wed, 14 Dec 2011 19:47:20 +0000 (UTC) Received: by iakl21 with SMTP id l21so3001800iak.13 for ; Wed, 14 Dec 2011 11:47:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=kdaO/en3UsZ4Bh+atziMpxi4CZFZaY74tbXPqYtM4yU=; b=pgUKYFmFYMK3QRg/GgaXv6C0uOcfbvn5c1Zs2vQvq5HvWANAvAhk54BmYwE6vzjJn5 9nnEequHiPFkrPJTEKjG38JmyMzlqVHjx8qvfy9u8uXpeTCxKhXe3CF0XoSL0A2evZM+ CdjNILJIAw3DFJ8QVG55uoemPYYrzXHOH6r1E= MIME-Version: 1.0 Received: by 10.43.51.69 with SMTP id vh5mr21322223icb.32.1323892040529; Wed, 14 Dec 2011 11:47:20 -0800 (PST) Received: by 10.50.51.233 with HTTP; Wed, 14 Dec 2011 11:47:20 -0800 (PST) In-Reply-To: References: <201112130935.33975.jhb@freebsd.org> Date: Wed, 14 Dec 2011 20:47:20 +0100 Message-ID: From: Monthadar Al Jaberi To: John Baldwin Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: loop inside uma_zfree critical section X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2011 19:47:21 -0000 On Tue, Dec 13, 2011 at 4:50 PM, Monthadar Al Jaberi wrote: > On Tue, Dec 13, 2011 at 3:35 PM, John Baldwin wrote: >> On Tuesday, December 13, 2011 7:46:34 am Monthadar Al Jaberi wrote: >>> Hi, >>> >>> I am not sure why I am having this problem, but looking >>> at the code I dont understand uma_core.c really good. >>> So I hope someone can shed a light on this: >>> >>> I am running on an arm board and and running a kernel module >>> that behaves like a wlan interface. so I tx and rx packets. >>> >>> For now tx is only only sending beacon like frames. >>> This is done through using ieee80211_beacon_alloc(). >>> >>> Then in a callout task to generate periodic beacons: >>> >>> =A0 =A0 m_dup(avp->beacon, M_DONTWAIT); >>> =A0 =A0 mtx_lock(...); >>> =A0 =A0 STAILQ_INSERT_TAIL(...); >>> =A0 =A0 taskqueue_enqueue(...); >>> =A0 =A0 mtx_unlock(...); >>> =A0 =A0 callout_schedule(...); >>> >>> On the RX side, the interrupt handler will read out buffer >>> then place it on a queue to be handled by wlan-glue code. >>> For now wlan-glue code just frees the mbuf it instead of >>> calling net80211 ieee80211_input() functions: >>> >>> =A0 =A0 m_copyback(...); >>> =A0 =A0 /* Allocate new mbuf for next RX. */ >>> =A0 =A0 MGETHDR(..., M_DONTWAIT, MT_DATA); >>> =A0 =A0 bzero((mtod(sc->Rx_m, void *)), MHLEN); >>> =A0 =A0 sc->Rx_m->m_len =3D 0; /* NB: m_gethdr does not set */ >>> =A0 =A0 sc->Rx_m->m_data +=3D 20; /* make headroom */ >>> >>> Then I use a lockmgr inside my kernel module that should >>> make sure that we either are on TX or RX path. >> >> Uh, you can't use a lockmgr lock in interrupt handlers or in >> if_transmit/if_start routines. =A0You should most likely just be using a= plain >> mutex instead. =A0Also, new code shouldn't use lockmgr in general. =A0If= you >> need a sleepable lock, use sx instead. =A0It has a more straightforward = API. > > Ok, I will change the interrupt handler to do something like this: > > =A0 =A0disaple_interrupt(); > =A0 =A0taskqueue_enqueue(...); /* on new rx task queue */ > > Then on the new rx proc: > > =A0 =A0sx_slock(...); > =A0 =A0m_copyback(...); > =A0 =A0enable_interrupt(); > =A0 =A0/* Allocate new mbuf for next RX. */ > =A0 =A0MGETHDR(..., M_DONTWAIT, MT_DATA); > =A0 =A0bzero((mtod(sc->Rx_m, void *)), MHLEN); > =A0 =A0sc->Rx_m->m_len =3D 0; /* NB: m_gethdr does not set */ > =A0 =A0sc->Rx_m->m_data +=3D 20; /* make headroom */ > =A0 =A0sx_sunlock(...); > > I lock TX/RX paths to make sure my code is threading safe. > Also because while programming my deivce (SPI communicatioin) > there will be a tsleep() waiting for the DMA interrupt and > thus we could be prempted by e.g. a beacon_callout etc... > I did implement your suggestions, using sx and modified interrupt handler as specified above. But still same problem as before. >> >>> The problem seems to be at [2], somehow after swapping >>> buckets in uma_zfree m_dup returns a pointer to >>> an mbuf that is still being used by us, [1] and [3] >>> have same address. >>> Then we call m_freem twice on same mbuf, [4] and [5]. >>> And a loop occurs inside uma_free. >>> I am using mbufs in a wrong way? Shouldnt mbufs be thread safe? >>> Problem seems to occur while swapping buckets. >> >> Hmm, the uma uses its own locking, so it should be safe, yes. =A0However= , you >> are correct about [1] and [3]. =A0The thing is, after [1] the mbuf shoul= dn't >> be in any buckets at all (it only gets put back into the bucket during a >> free). =A0Are you sure the mbuf wasn't double free'd previously? I rechecked and it is almost certain that I dont double free the mbuf before [1]. And its not like it crashed in the beginning, it does run for a while and then it crashes. So our code works for like a hundred beacons sent/rece= ived between two arm boards. Its feels like something is preempted, which explai= ns why the mbuf is still in the bucket (wrongly)? > > From my log I can only see the mbuf being used once before > by a beacon_callout and then it was freed by m_freem(). > So I cant see that it was freed twice before that. > > How can I go by debbuging this? > >> >> -- >> John Baldwin > > > > -- > Monthadar Al Jaberi --=20 Monthadar Al Jaberi From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 14 20:10:51 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 324381065670 for ; Wed, 14 Dec 2011 20:10:51 +0000 (UTC) (envelope-from lacombar@gmail.com) Received: from mail-ee0-f54.google.com (mail-ee0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id 94F148FC0C for ; Wed, 14 Dec 2011 20:10:50 +0000 (UTC) Received: by eekc50 with SMTP id c50so1490047eek.13 for ; Wed, 14 Dec 2011 12:10:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=6QHo8gTKUS+LkYQOTgueyCMfc4pi1G/zk23hBB8s4Go=; b=fjBY2cUa6PJpZy3JP+6OIsj5U2t5WvLkK9OL0irvVO2tujRmEJfFgOcjM877O6Vjda /FVdlSeksbGxhnWuPz3CLbLLq9OePyF+GARJ2AO7S8Ru1HGxZ0DiNxTtIRb+m0qUEaj8 07OBprWbEKZxoyBjJNuqJLeWNua3ScDVQOJts= MIME-Version: 1.0 Received: by 10.180.18.233 with SMTP id z9mr416357wid.0.1323893449567; Wed, 14 Dec 2011 12:10:49 -0800 (PST) Received: by 10.180.99.226 with HTTP; Wed, 14 Dec 2011 12:10:49 -0800 (PST) In-Reply-To: References: <201112130935.33975.jhb@freebsd.org> Date: Wed, 14 Dec 2011 15:10:49 -0500 Message-ID: From: Arnaud Lacombe To: Monthadar Al Jaberi Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: loop inside uma_zfree critical section X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2011 20:10:51 -0000 Hi, On Wed, Dec 14, 2011 at 2:47 PM, Monthadar Al Jaberi wrote: > On Tue, Dec 13, 2011 at 4:50 PM, Monthadar Al Jaberi > wrote: >> On Tue, Dec 13, 2011 at 3:35 PM, John Baldwin wrote: >>> On Tuesday, December 13, 2011 7:46:34 am Monthadar Al Jaberi wrote: >>>> Hi, >>>> >>>> I am not sure why I am having this problem, but looking >>>> at the code I dont understand uma_core.c really good. >>>> So I hope someone can shed a light on this: >>>> >>>> I am running on an arm board and and running a kernel module >>>> that behaves like a wlan interface. so I tx and rx packets. >>>> >>>> For now tx is only only sending beacon like frames. >>>> This is done through using ieee80211_beacon_alloc(). >>>> >>>> Then in a callout task to generate periodic beacons: >>>> >>>> =A0 =A0 m_dup(avp->beacon, M_DONTWAIT); >>>> =A0 =A0 mtx_lock(...); >>>> =A0 =A0 STAILQ_INSERT_TAIL(...); >>>> =A0 =A0 taskqueue_enqueue(...); >>>> =A0 =A0 mtx_unlock(...); >>>> =A0 =A0 callout_schedule(...); >>>> >>>> On the RX side, the interrupt handler will read out buffer >>>> then place it on a queue to be handled by wlan-glue code. >>>> For now wlan-glue code just frees the mbuf it instead of >>>> calling net80211 ieee80211_input() functions: >>>> >>>> =A0 =A0 m_copyback(...); >>>> =A0 =A0 /* Allocate new mbuf for next RX. */ >>>> =A0 =A0 MGETHDR(..., M_DONTWAIT, MT_DATA); >>>> =A0 =A0 bzero((mtod(sc->Rx_m, void *)), MHLEN); >>>> =A0 =A0 sc->Rx_m->m_len =3D 0; /* NB: m_gethdr does not set */ >>>> =A0 =A0 sc->Rx_m->m_data +=3D 20; /* make headroom */ >>>> >>>> Then I use a lockmgr inside my kernel module that should >>>> make sure that we either are on TX or RX path. >>> >>> Uh, you can't use a lockmgr lock in interrupt handlers or in >>> if_transmit/if_start routines. =A0You should most likely just be using = a plain >>> mutex instead. =A0Also, new code shouldn't use lockmgr in general. =A0I= f you >>> need a sleepable lock, use sx instead. =A0It has a more straightforward= API. >> >> Ok, I will change the interrupt handler to do something like this: >> >> =A0 =A0disaple_interrupt(); >> =A0 =A0taskqueue_enqueue(...); /* on new rx task queue */ >> >> Then on the new rx proc: >> >> =A0 =A0sx_slock(...); >> =A0 =A0m_copyback(...); >> =A0 =A0enable_interrupt(); >> =A0 =A0/* Allocate new mbuf for next RX. */ >> =A0 =A0MGETHDR(..., M_DONTWAIT, MT_DATA); >> =A0 =A0bzero((mtod(sc->Rx_m, void *)), MHLEN); >> =A0 =A0sc->Rx_m->m_len =3D 0; /* NB: m_gethdr does not set */ >> =A0 =A0sc->Rx_m->m_data +=3D 20; /* make headroom */ >> =A0 =A0sx_sunlock(...); >> >> I lock TX/RX paths to make sure my code is threading safe. >> Also because while programming my deivce (SPI communicatioin) >> there will be a tsleep() waiting for the DMA interrupt and >> thus we could be prempted by e.g. a beacon_callout etc... >> > > I did implement your suggestions, using sx and modified interrupt handler > as specified above. But still same problem as before. > >>> >>>> The problem seems to be at [2], somehow after swapping >>>> buckets in uma_zfree m_dup returns a pointer to >>>> an mbuf that is still being used by us, [1] and [3] >>>> have same address. >>>> Then we call m_freem twice on same mbuf, [4] and [5]. >>>> And a loop occurs inside uma_free. >>>> I am using mbufs in a wrong way? Shouldnt mbufs be thread safe? >>>> Problem seems to occur while swapping buckets. >>> >>> Hmm, the uma uses its own locking, so it should be safe, yes. =A0Howeve= r, you >>> are correct about [1] and [3]. =A0The thing is, after [1] the mbuf shou= ldn't >>> be in any buckets at all (it only gets put back into the bucket during = a >>> free). =A0Are you sure the mbuf wasn't double free'd previously? > > I rechecked and it is almost certain that I dont double free the mbuf > before [1]. > And its not like it crashed in the beginning, it does run for a while > and then it crashes. So our code works for like a hundred beacons sent/re= ceived > between two arm boards. Its feels like something is preempted, which expl= ains > why the mbuf is still in the bucket (wrongly)? > are you running an INVARIANTS/DIAGNOSTICS/WITNESS/LOCK_DEBUG/... enabled kernel ? are you running on an SMP platform where there might be cache-coherency iss= ue ? Thanks, - Arnaud From owner-freebsd-hackers@FreeBSD.ORG Wed Dec 14 21:59:10 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D5A78106564A; Wed, 14 Dec 2011 21:59:10 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe06.c2i.net [212.247.154.162]) by mx1.freebsd.org (Postfix) with ESMTP id 637A48FC0A; Wed, 14 Dec 2011 21:59:09 +0000 (UTC) X-T2-Spam-Status: No, hits=-0.2 required=5.0 tests=ALL_TRUSTED, BAYES_50 Received: from [188.126.198.129] (account mc467741@c2i.net HELO laptop002.hselasky.homeunix.org) by mailfe06.swip.net (CommuniGate Pro SMTP 5.4.2) with ESMTPA id 215899356; Wed, 14 Dec 2011 22:59:05 +0100 From: Hans Petter Selasky To: Andriy Gapon Date: Wed, 14 Dec 2011 22:56:32 +0100 User-Agent: KMail/1.13.5 (FreeBSD/8.2-STABLE; KDE/4.4.5; amd64; ; ) References: <4EE51CB5.1060505@FreeBSD.org> <4EE70A1F.8040406@FreeBSD.org> <4EE8C2CE.40909@FreeBSD.org> In-Reply-To: <4EE8C2CE.40909@FreeBSD.org> X-Face: *nPdTl_}RuAI6^PVpA02T?$%Xa^>@hE0uyUIoiha$pC:9TVgl.Oq, NwSZ4V"|LR.+tj}g5 %V,x^qOs~mnU3]Gn; cQLv&.N>TrxmSFf+p6(30a/{)KUU!s}w\IhQBj}[g}bj0I3^glmC( :AuzV9:.hESm-x4h240C`9=w MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Message-Id: <201112142256.32526.hselasky@c2i.net> Cc: usb@freebsd.org, freebsd-hackers@freebsd.org, mdf@freebsd.org Subject: Re: kern_yield vs ukbd_yield X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2011 21:59:10 -0000 On Wednesday 14 December 2011 16:37:50 Andriy Gapon wrote: > on 13/12/2011 10:17 Andriy Gapon said the following: > > on 13/12/2011 00:21 Andriy Gapon said the following: > [snip] >=20 > > And in the view of the below data I would like us to revisit this > > problem. I looked over usb code and it seems that all usb threads are > > created with priorities of either USB_PRI_MED or USB_PRI_HIGH, which > > translates to PI_SWI(SWI_CAMBIO) and PI_SWI(SWI_NET). These priorities > > should be in the ithread range, so it's kind of surprising that the init > > thread (with PVM priority) can cause troubles for them. >=20 > So, Hans Petter, do you recall any details of this problem? > I am curious about which thread got starved by which. =46rom what I know this was 100% reproducible. Remove the ukbd_yield() when at the mountroot prompt. Result: cannot type a= ny=20 keys. No USB devices will enumerate! >=20 > BTW, given your recent improvements to pause(9) what do you think about > further extending it to also use DELAY(9) when kdb_active is set or when > SCHEDULER_STOPPED() is true?=20 I think this is a good idea. It already checks for "cold". USB usually does= n't=20 use this function though when polling. > Then, probably, pause(9) could be used for > both branches in ukbd_do_poll and they could be collapsed together. =2D-HPS From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 15 09:46:41 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2B07E106566C for ; Thu, 15 Dec 2011 09:46:41 +0000 (UTC) (envelope-from janm-freebsd-hackers@transactionware.com) Received: from midgard.transactionware.com (mail2.transactionware.com [203.14.245.36]) by mx1.freebsd.org (Postfix) with SMTP id 80EEF8FC08 for ; Thu, 15 Dec 2011 09:46:40 +0000 (UTC) Received: (qmail 92440 invoked by uid 907); 15 Dec 2011 09:19:58 -0000 Received: from jmmacpro.transactionware.com (HELO jmmacpro.transactionware.com) (192.168.1.33) by midgard.transactionware.com (qpsmtpd/0.82) with ESMTP; Thu, 15 Dec 2011 20:19:58 +1100 Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: text/plain; charset=us-ascii From: Jan Mikkelsen In-Reply-To: <9317551F-CBE0-4368-B798-498E58E240B2@sarenet.es> Date: Thu, 15 Dec 2011 20:19:58 +1100 Content-Transfer-Encoding: quoted-printable Message-Id: <2EA3FFF4-E6A2-4371-8891-26E99C551C67@transactionware.com> References: <4EE8A005.5030607@barafranca.com> <9317551F-CBE0-4368-B798-498E58E240B2@sarenet.es> To: Borja Marcos X-Mailer: Apple Mail (2.1251.1) Cc: freebsd-hackers@freebsd.org, Hugo Silva Subject: Re: mfi (Dell H700) + hot swapping doesn't appear to work with RC1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2011 09:46:41 -0000 On 15/12/2011, at 2:16 AM, Borja Marcos wrote: >=20 > On Dec 14, 2011, at 2:09 PM, Hugo Silva wrote: >=20 >> Hello, >>=20 >> First of all apologies if this has been fixed in RC3. I set this = server >> up with mfsbsd, which is RC1, and didn't get to update the system = yet. >>=20 >> This box has 6 hdds, a 2-mirror zpool was set up as the root pool, = with >> 2 spares. >>=20 >> While testing hot swapping I noticed that while the controller = detects >> disk removal/insertion, the zpool will never recover. The problem = seems >> to be deeper than ZFS, as disklabel/fdisk/etc also fail on the >> removed-and-reinserted disk. >>=20 >> At the ZFS level, doing a zpool clear yields more errors on the = removed >> disk; rebooting becomes the only option to make the pool healthy = again. >>=20 >>=20 >> Is this normal? Did I miss any step? >=20 > I assume that you have tried to use the H700 as a "JBOD" card, = defining logical volume for each hard disk. >=20 > The problem is: that gorgeous, fantastic, masterful, Nobel award = candidate card, has a wonderful behavior in that case. If you extract = one of the disks, the logical volume associated to it is invalidated. = So, you insert a replacement disk, and the card refuses to recognize the = volume. What is even worse, in order to recover it's mandatory to reboot = the complete system *AND* go through the RAID configuration utility. >=20 > That's the problem. The card refuses to work as a simple disk = controller without frills, and the frills get in the way. >=20 > To summarize: it isn't FreeBSD's fault, no matter which version you = use. It's a "feature" coming directly from the geniuses who designed the = card. (Sending again to avoid moderation.) Hugo: You missed a step. Borja: No reboot required. For the mfi controllers I have been testing recently (MegaRAID 9261-8i), = you need to install the sysutils/megacli port, and use that to clear the = "foreignness" of the disk you just added. Something like: MegaCli -CfgForeign -Clear -a0 You should be able to then recreate it as a JBOD device, and progress = through whatever higher level recovery you need to do. Regards, Jan Mikkelsen From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 15 14:17:09 2011 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A6DFC106566B; Thu, 15 Dec 2011 14:17:09 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 3EF748FC0C; Thu, 15 Dec 2011 14:17:07 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id QAA29161; Thu, 15 Dec 2011 16:17:03 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1RbC7L-0002Lq-62; Thu, 15 Dec 2011 16:17:03 +0200 Message-ID: <4EEA015D.8020800@FreeBSD.org> Date: Thu, 15 Dec 2011 16:17:01 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111206 Thunderbird/8.0 MIME-Version: 1.0 To: Hans Petter Selasky References: <4EE51CB5.1060505@FreeBSD.org> <4EE70A1F.8040406@FreeBSD.org> <4EE8C2CE.40909@FreeBSD.org> <201112142256.32526.hselasky@c2i.net> In-Reply-To: <201112142256.32526.hselasky@c2i.net> X-Enigmail-Version: undefined Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: usb@FreeBSD.org, freebsd-hackers@FreeBSD.org, John Baldwin , mdf@FreeBSD.org Subject: Re: kern_yield vs ukbd_yield X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2011 14:17:09 -0000 on 14/12/2011 23:56 Hans Petter Selasky said the following: > On Wednesday 14 December 2011 16:37:50 Andriy Gapon wrote: >> So, Hans Petter, do you recall any details of this problem? >> I am curious about which thread got starved by which. > > From what I know this was 100% reproducible. > > Remove the ukbd_yield() when at the mountroot prompt. Result: cannot type any > keys. No USB devices will enumerate! This hasn't satisfied my curiosity :) >> >> BTW, given your recent improvements to pause(9) what do you think about >> further extending it to also use DELAY(9) when kdb_active is set or when >> SCHEDULER_STOPPED() is true? > > I think this is a good idea. It already checks for "cold". USB usually doesn't > use this function though when polling. > >> Then, probably, pause(9) could be used for >> both branches in ukbd_do_poll and they could be collapsed together. Hmm... I looked at the history of ukbd.c (which I should have done from the very start) and I see two relevant revisions: http://svnweb.freebsd.org/base/head/sys/dev/usb/input/ukbd.c?r1=199816&r2=203896&pathrev=203896 http://svnweb.freebsd.org/base/head/sys/dev/usb/input/ukbd.c?r1=223755&r2=223989&pathrev=223989 So, first use of pause(9) was introduced to work around current broken syscons polling mechanism. Then pause(9) was replaced with the hand-rolled yield to fix a problem at shutdown, which supposedly was caused by times being stopped. None of the commits seems to be directly related to thread priorities. I suspect that even DELAY(9) could have worked in the place of the pause/yield. Also, I do not see any mentioning of the mountroot context in the commits. Not sure why I even assumed that it was relevant. BTW, maybe we should have a 'cold again' flag for late stages of system shutdown? So, all in all, I believe in the following two things: 1. kern_yield can be used instead of ukbd_yield with any argument 1.1. DELAY() can be used instead of ukbd_yield too 2. all that special code should not be needed once I commit the patches that I have recently posted to arch@ (I do intend to commit them). In that case the keyboard drivers would be properly put into the polling mode instead of the current situation where the polling mode is repeatedly entered and exited when kernel needs some input. Maybe you recall/know more than you wrote above and I have missed something obvious that you can point out? -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 15 14:56:48 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3AB7C1065673 for ; Thu, 15 Dec 2011 14:56:48 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 0E6918FC18 for ; Thu, 15 Dec 2011 14:56:48 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id B918B46B3B; Thu, 15 Dec 2011 09:56:47 -0500 (EST) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 1D61AB95C; Thu, 15 Dec 2011 09:56:47 -0500 (EST) From: John Baldwin To: freebsd-hackers@freebsd.org Date: Thu, 15 Dec 2011 09:56:45 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p8; KDE/4.5.5; amd64; ; ) References: <4EE8A005.5030607@barafranca.com> <9317551F-CBE0-4368-B798-498E58E240B2@sarenet.es> <2EA3FFF4-E6A2-4371-8891-26E99C551C67@transactionware.com> In-Reply-To: <2EA3FFF4-E6A2-4371-8891-26E99C551C67@transactionware.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201112150956.45214.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 15 Dec 2011 09:56:47 -0500 (EST) Cc: Borja Marcos , Jan Mikkelsen , Hugo Silva Subject: Re: mfi (Dell H700) + hot swapping doesn't appear to work with RC1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2011 14:56:48 -0000 On Thursday, December 15, 2011 4:19:58 am Jan Mikkelsen wrote: > On 15/12/2011, at 2:16 AM, Borja Marcos wrote: > > > > > On Dec 14, 2011, at 2:09 PM, Hugo Silva wrote: > > > >> Hello, > >> > >> First of all apologies if this has been fixed in RC3. I set this server > >> up with mfsbsd, which is RC1, and didn't get to update the system yet. > >> > >> This box has 6 hdds, a 2-mirror zpool was set up as the root pool, with > >> 2 spares. > >> > >> While testing hot swapping I noticed that while the controller detects > >> disk removal/insertion, the zpool will never recover. The problem seems > >> to be deeper than ZFS, as disklabel/fdisk/etc also fail on the > >> removed-and-reinserted disk. > >> > >> At the ZFS level, doing a zpool clear yields more errors on the removed > >> disk; rebooting becomes the only option to make the pool healthy again. > >> > >> > >> Is this normal? Did I miss any step? > > > > I assume that you have tried to use the H700 as a "JBOD" card, defining logical volume for each hard disk. > > > > The problem is: that gorgeous, fantastic, masterful, Nobel award candidate card, has a wonderful behavior in that case. If you extract one of the disks, the logical volume associated to it is invalidated. So, you insert a replacement disk, and the card refuses to recognize the volume. What is even worse, in order to recover it's mandatory to reboot the complete system *AND* go through the RAID configuration utility. > > > > That's the problem. The card refuses to work as a simple disk controller without frills, and the frills get in the way. > > > > To summarize: it isn't FreeBSD's fault, no matter which version you use. It's a "feature" coming directly from the geniuses who designed the card. > > (Sending again to avoid moderation.) > > Hugo: You missed a step. Borja: No reboot required. > > For the mfi controllers I have been testing recently (MegaRAID 9261-8i), you need to install the sysutils/megacli port, and use that to clear the "foreignness" of the disk you just added. Something like: > > MegaCli -CfgForeign -Clear -a0 > > You should be able to then recreate it as a JBOD device, and progress through whatever higher level recovery you need to do. Can you do this by marking it as 'good' via mfiutil and then using mfiutil to create a volume? -- John Baldwin From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 15 15:29:12 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EAB10106564A for ; Thu, 15 Dec 2011 15:29:12 +0000 (UTC) (envelope-from hugo@barafranca.com) Received: from mail.barafranca.com (mail.barafranca.com [67.213.67.47]) by mx1.freebsd.org (Postfix) with ESMTP id B6DE48FC13 for ; Thu, 15 Dec 2011 15:29:12 +0000 (UTC) Received: from localhost (unknown [172.16.100.24]) by mail.barafranca.com (Postfix) with ESMTP id E0C23A42 for ; Thu, 15 Dec 2011 15:29:11 +0000 (UTC) X-Virus-Scanned: amavisd-new at barafranca.com Received: from mail.barafranca.com ([172.16.100.24]) by localhost (mail.barafranca.com [172.16.100.24]) (amavisd-new, port 10024) with ESMTP id FLFrx60IOTsX for ; Thu, 15 Dec 2011 15:28:30 +0000 (UTC) Received: from [192.168.1.1] (static-b4-252-232.telepac.pt [81.193.252.232]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.barafranca.com (Postfix) with ESMTPSA id 4F0BBA34 for ; Thu, 15 Dec 2011 15:28:30 +0000 (UTC) Message-ID: <4EEA1215.8060507@barafranca.com> Date: Thu, 15 Dec 2011 15:28:21 +0000 From: Hugo Silva User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.13) Gecko/20101208 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <4EE8A005.5030607@barafranca.com> <9317551F-CBE0-4368-B798-498E58E240B2@sarenet.es> <2EA3FFF4-E6A2-4371-8891-26E99C551C67@transactionware.com> In-Reply-To: <2EA3FFF4-E6A2-4371-8891-26E99C551C67@transactionware.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: Re: mfi (Dell H700) + hot swapping doesn't appear to work with RC1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2011 15:29:13 -0000 On 12/15/11 09:19, Jan Mikkelsen wrote: > > Hugo: You missed a step. Borja: No reboot required. > > For the mfi controllers I have been testing recently (MegaRAID 9261-8i), you need to install the sysutils/megacli port, and use that to clear the "foreignness" of the disk you just added. Something like: > > MegaCli -CfgForeign -Clear -a0 > > You should be able to then recreate it as a JBOD device, and progress through whatever higher level recovery you need to do. > > Regards, > > Jan Mikkelsen > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" Hello, I tried that and got: There is no foreign configuration on controller 0. Exit Code: 0x00 Presently one drive is failed, following a mfiutil fail 2. Trying to bring it back with 'mfiutil good 2' results in: mfiutil: Command failed: Wrong firmware or drive state mfiutil: Failed to set drive 2 to UNCONFIGURED GOOD: Input/output error mfi0 Volumes: Id Size Level Stripe State Cache Name mfid0 ( 558G) RAID-0 64k OPTIMAL Disabled mfid1 ( 558G) RAID-0 64k OPTIMAL Disabled mfid2 ( 558G) RAID-0 64k OPTIMAL Disabled mfid3 ( 558G) RAID-0 64k OPTIMAL Disabled mfid4 ( 558G) RAID-0 64k OFFLINE Disabled mfid5 ( 558G) RAID-0 64k OPTIMAL Disabled As Borja said, part of the difficulty is the H700 abstracting a single disk as a RAID-0, I guess. So far I've been unable to find a way to bring the drive back, except by rebooting and recreating. Any other suggestions? Thanks, Hugo From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 15 16:04:40 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BED951065770 for ; Thu, 15 Dec 2011 16:04:40 +0000 (UTC) (envelope-from hugo@barafranca.com) Received: from mail.barafranca.com (mail.barafranca.com [67.213.67.47]) by mx1.freebsd.org (Postfix) with ESMTP id 8CA2A8FC0C for ; Thu, 15 Dec 2011 16:04:40 +0000 (UTC) Received: from localhost (unknown [172.16.100.24]) by mail.barafranca.com (Postfix) with ESMTP id BC460DC1 for ; Thu, 15 Dec 2011 16:04:39 +0000 (UTC) X-Virus-Scanned: amavisd-new at barafranca.com Received: from mail.barafranca.com ([172.16.100.24]) by localhost (mail.barafranca.com [172.16.100.24]) (amavisd-new, port 10024) with ESMTP id c+OTZsM4aiMg for ; Thu, 15 Dec 2011 16:03:55 +0000 (UTC) Received: from [192.168.1.1] (static-b4-252-232.telepac.pt [81.193.252.232]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.barafranca.com (Postfix) with ESMTPSA id 4E12ADB2 for ; Thu, 15 Dec 2011 16:03:55 +0000 (UTC) Message-ID: <4EEA1A64.1040104@barafranca.com> Date: Thu, 15 Dec 2011 16:03:48 +0000 From: Hugo Silva User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.13) Gecko/20101208 Thunderbird/3.1.7 MIME-Version: 1.0 To: freebsd-hackers@freebsd.org References: <4EE8A005.5030607@barafranca.com> <9317551F-CBE0-4368-B798-498E58E240B2@sarenet.es> <2EA3FFF4-E6A2-4371-8891-26E99C551C67@transactionware.com> <4EEA1215.8060507@barafranca.com> In-Reply-To: <4EEA1215.8060507@barafranca.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: Re: mfi (Dell H700) + hot swapping doesn't appear to work with RC1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2011 16:04:40 -0000 On 12/15/11 15:28, Hugo Silva wrote: > As Borja said, part of the difficulty is the H700 abstracting a single > disk as a RAID-0, I guess. So far I've been unable to find a way to > bring the drive back, except by rebooting and recreating. Turns out no interaction is needed after reboot. It was something else unrelated. The main issue then is convincing the controller to once again accept the hard disk. I'm going through MegaCli "documentation" (ie --help).. it's not a pretty place. Regards, Hugo From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 15 16:55:47 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AAF371065672 for ; Thu, 15 Dec 2011 16:55:47 +0000 (UTC) (envelope-from aboyer@averesystems.com) Received: from zimbra.averesystems.com (75-149-8-245-Pennsylvania.hfc.comcastbusiness.net [75.149.8.245]) by mx1.freebsd.org (Postfix) with ESMTP id 2F30D8FC17 for ; Thu, 15 Dec 2011 16:55:47 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by zimbra.averesystems.com (Postfix) with ESMTP id AE892446006; Thu, 15 Dec 2011 11:41:14 -0500 (EST) X-Virus-Scanned: amavisd-new at averesystems.com Received: from zimbra.averesystems.com ([127.0.0.1]) by localhost (zimbra.averesystems.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LHS9HcTIfv9m; Thu, 15 Dec 2011 11:41:13 -0500 (EST) Received: from riven.arriad.com (fw.arriad.com [10.0.0.16]) by zimbra.averesystems.com (Postfix) with ESMTPSA id 3530A446001; Thu, 15 Dec 2011 11:41:13 -0500 (EST) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Andrew Boyer In-Reply-To: <4EEA1A64.1040104@barafranca.com> Date: Thu, 15 Dec 2011 11:40:16 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <3A13B3B1-4CDE-4D5A-A6F3-E7DC5BB0E4E3@averesystems.com> References: <4EE8A005.5030607@barafranca.com> <9317551F-CBE0-4368-B798-498E58E240B2@sarenet.es> <2EA3FFF4-E6A2-4371-8891-26E99C551C67@transactionware.com> <4EEA1215.8060507@barafranca.com> <4EEA1A64.1040104@barafranca.com> To: Hugo Silva , Jan Mikkelsen X-Mailer: Apple Mail (2.1084) Cc: freebsd-hackers@freebsd.org Subject: Re: mfi (Dell H700) + hot swapping doesn't appear to work with RC1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2011 16:55:47 -0000 On Dec 15, 2011, at 4:19 AM, Jan Mikkelsen wrote: > For the mfi controllers I have been testing recently (MegaRAID = 9261-8i), you need to install the sysutils/megacli port, and use that to = clear the "foreignness" of the disk you just added. Something like: >=20 > MegaCli -CfgForeign -Clear -a0 I don't think that's what you want. You want to use -Import, not = -Clear, to keep your data intact. On Dec 15, 2011, at 11:03 AM, Hugo Silva wrote: > On 12/15/11 15:28, Hugo Silva wrote: >> As Borja said, part of the difficulty is the H700 abstracting a = single >> disk as a RAID-0, I guess. So far I've been unable to find a way to >> bring the drive back, except by rebooting and recreating. >=20 > Turns out no interaction is needed after reboot. It was something else > unrelated. The main issue then is convincing the controller to once > again accept the hard disk. I'm going through MegaCli "documentation" > (ie --help).. it's not a pretty place. I'm not sure it would even be possible to come up with a worse = interface. It boggles the mind. I recommend you always run with this configuration: # MegaCli -AdpSetProp AutoEnhancedImportEnbl -aALL # MegaCli -AdpSetProp MaintainPdFailHistoryEnbl -0 -aALL AutoEnhancedImportEnbl will bring the foreign disk back in on a reboot. = LSI recommends turning off MaintainPdFailHistory when using single-disk = RAID0 configurations. To bring in a foreign disk without rebooting: # MegaCli -CfgForeign -Scan -aALL # MegaCli -CfgForeign -Import [x] -aN (where x is the config number = listed in the scan, and N is the adapter number) Adding these capabilities to mfiutil is on my list of things to do, but = it's not ready yet. Has anyone managed to get the real JBOD mode working on this controller? = It advertises support in the firmware but doesn't seem to do anything. = The documentation only lists JBOD mode as a feature of the lower-end = controllers. Hope this helps. -Andrew -------------------------------------------------- Andrew Boyer aboyer@averesystems.com From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 15 18:11:36 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4972C106567A for ; Thu, 15 Dec 2011 18:11:36 +0000 (UTC) (envelope-from hugo@barafranca.com) Received: from mail.barafranca.com (mail.barafranca.com [67.213.67.47]) by mx1.freebsd.org (Postfix) with ESMTP id 184468FC1B for ; Thu, 15 Dec 2011 18:11:35 +0000 (UTC) Received: from localhost (unknown [172.16.100.24]) by mail.barafranca.com (Postfix) with ESMTP id 6E90CF22; Thu, 15 Dec 2011 18:11:35 +0000 (UTC) X-Virus-Scanned: amavisd-new at barafranca.com Received: from mail.barafranca.com ([172.16.100.24]) by localhost (mail.barafranca.com [172.16.100.24]) (amavisd-new, port 10024) with ESMTP id T-nKQLxs0Ggf; Thu, 15 Dec 2011 18:11:00 +0000 (UTC) Received: from [192.168.1.1] (static-b4-252-232.telepac.pt [81.193.252.232]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.barafranca.com (Postfix) with ESMTPSA id 5EAECEE6; Thu, 15 Dec 2011 18:10:59 +0000 (UTC) Message-ID: <4EEA382A.1000900@barafranca.com> Date: Thu, 15 Dec 2011 18:10:50 +0000 From: Hugo Silva User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.13) Gecko/20101208 Thunderbird/3.1.7 MIME-Version: 1.0 To: Andrew Boyer References: <4EE8A005.5030607@barafranca.com> <9317551F-CBE0-4368-B798-498E58E240B2@sarenet.es> <2EA3FFF4-E6A2-4371-8891-26E99C551C67@transactionware.com> <4EEA1215.8060507@barafranca.com> <4EEA1A64.1040104@barafranca.com> <3A13B3B1-4CDE-4D5A-A6F3-E7DC5BB0E4E3@averesystems.com> In-Reply-To: <3A13B3B1-4CDE-4D5A-A6F3-E7DC5BB0E4E3@averesystems.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, Jan Mikkelsen Subject: Re: mfi (Dell H700) + hot swapping doesn't appear to work with RC1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2011 18:11:36 -0000 On 12/15/11 16:40, Andrew Boyer wrote: > I'm not sure it would even be possible to come up with a worse interface. It boggles the mind. > > I recommend you always run with this configuration: > > # MegaCli -AdpSetProp AutoEnhancedImportEnbl -aALL > # MegaCli -AdpSetProp MaintainPdFailHistoryEnbl -0 -aALL > > AutoEnhancedImportEnbl will bring the foreign disk back in on a reboot. LSI recommends turning off MaintainPdFailHistory when using single-disk RAID0 configurations. > Any gotchas with this enabled? I'm thinking putting in a disk from another card, which is part of a raid, in this server, for instance. > To bring in a foreign disk without rebooting: > > # MegaCli -CfgForeign -Scan -aALL > # MegaCli -CfgForeign -Import [x] -aN (where x is the config number listed in the scan, and N is the adapter number) > > Adding these capabilities to mfiutil is on my list of things to do, but it's not ready yet. > > Has anyone managed to get the real JBOD mode working on this controller? It advertises support in the firmware but doesn't seem to do anything. The documentation only lists JBOD mode as a feature of the lower-end controllers. > > Hope this helps. > > -Andrew It does help - thanks! For the same disk being removed and then reinserted, the provided commands brought the disk/volume back to mfiutil show drives/volumes output, and after a zpool clear, ZFS has no complains. For recovery from a software-induced fail (mfiutil fail eX:sX), I couldn't perform a recovery using just mfiutil. MegaCli -PDOnline -PhysDrv[eX:sX] -aN did it, in that case. For the still-untested case of an altogether new disk being inserted, I guess mfiutil create jbod N would do the trick. BTW, the mfiutil is coredumping when provided with inexistant disks (just noticed) Regards, Hugo From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 15 18:28:11 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2A3DF106566B for ; Thu, 15 Dec 2011 18:28:11 +0000 (UTC) (envelope-from aboyer@averesystems.com) Received: from zimbra.averesystems.com (75-149-8-245-Pennsylvania.hfc.comcastbusiness.net [75.149.8.245]) by mx1.freebsd.org (Postfix) with ESMTP id D079A8FC0A for ; Thu, 15 Dec 2011 18:28:10 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by zimbra.averesystems.com (Postfix) with ESMTP id 43B4F446006; Thu, 15 Dec 2011 13:29:06 -0500 (EST) X-Virus-Scanned: amavisd-new at averesystems.com Received: from zimbra.averesystems.com ([127.0.0.1]) by localhost (zimbra.averesystems.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gFo9t+IGhat2; Thu, 15 Dec 2011 13:29:04 -0500 (EST) Received: from riven.arriad.com (fw.arriad.com [10.0.0.16]) by zimbra.averesystems.com (Postfix) with ESMTPSA id 93F5B446001; Thu, 15 Dec 2011 13:29:04 -0500 (EST) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Andrew Boyer In-Reply-To: <4EEA382A.1000900@barafranca.com> Date: Thu, 15 Dec 2011 13:28:07 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <8CBA93DE-9352-4FAC-A59E-96E4DFC364E1@averesystems.com> References: <4EE8A005.5030607@barafranca.com> <9317551F-CBE0-4368-B798-498E58E240B2@sarenet.es> <2EA3FFF4-E6A2-4371-8891-26E99C551C67@transactionware.com> <4EEA1215.8060507@barafranca.com> <4EEA1A64.1040104@barafranca.com> <3A13B3B1-4CDE-4D5A-A6F3-E7DC5BB0E4E3@averesystems.com> <4EEA382A.1000900@barafranca.com> To: Hugo Silva X-Mailer: Apple Mail (2.1084) Cc: freebsd-hackers@freebsd.org, Jan Mikkelsen Subject: Re: mfi (Dell H700) + hot swapping doesn't appear to work with RC1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2011 18:28:11 -0000 On Dec 15, 2011, at 1:10 PM, Hugo Silva wrote: > On 12/15/11 16:40, Andrew Boyer wrote: >> I'm not sure it would even be possible to come up with a worse = interface. It boggles the mind. >>=20 >> I recommend you always run with this configuration: >>=20 >> # MegaCli -AdpSetProp AutoEnhancedImportEnbl -aALL >> # MegaCli -AdpSetProp MaintainPdFailHistoryEnbl -0 -aALL >>=20 >> AutoEnhancedImportEnbl will bring the foreign disk back in on a = reboot. LSI recommends turning off MaintainPdFailHistory when using = single-disk RAID0 configurations. >>=20 >=20 > Any gotchas with this enabled? I'm thinking putting in a disk from > another card, which is part of a raid, in this server, for instance. >=20 My understanding is that it only imports a foreign configuration if it's = complete - but I've never tested it. >> To bring in a foreign disk without rebooting: >>=20 >> # MegaCli -CfgForeign -Scan -aALL >> # MegaCli -CfgForeign -Import [x] -aN (where x is the config number = listed in the scan, and N is the adapter number) >>=20 >> Adding these capabilities to mfiutil is on my list of things to do, = but it's not ready yet. >>=20 >> Has anyone managed to get the real JBOD mode working on this = controller? It advertises support in the firmware but doesn't seem to = do anything. The documentation only lists JBOD mode as a feature of the = lower-end controllers. >>=20 >> Hope this helps. >>=20 >> -Andrew >=20 > It does help - thanks! For the same disk being removed and then > reinserted, the provided commands brought the disk/volume back to > mfiutil show drives/volumes output, and after a zpool clear, ZFS has = no > complains. >=20 >=20 > For recovery from a software-induced fail (mfiutil fail eX:sX), I > couldn't perform a recovery using just mfiutil. MegaCli -PDOnline > -PhysDrv[eX:sX] -aN did it, in that case. >=20 > For the still-untested case of an altogether new disk being inserted, = I > guess mfiutil create jbod N would do the trick. >=20 >=20 > BTW, the mfiutil is coredumping when provided with inexistant disks > (just noticed) Can you provide the exact command that produces a core? -Andrew -------------------------------------------------- Andrew Boyer aboyer@averesystems.com From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 15 22:59:10 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4F0BB106564A; Thu, 15 Dec 2011 22:59:10 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe09.c2i.net [212.247.155.2]) by mx1.freebsd.org (Postfix) with ESMTP id C2A378FC18; Thu, 15 Dec 2011 22:59:08 +0000 (UTC) X-T2-Spam-Status: No, hits=-0.2 required=5.0 tests=ALL_TRUSTED, BAYES_50 Received: from [188.126.198.129] (account mc467741@c2i.net HELO laptop002.hselasky.homeunix.org) by mailfe09.swip.net (CommuniGate Pro SMTP 5.4.2) with ESMTPA id 43467472; Thu, 15 Dec 2011 23:59:06 +0100 From: Hans Petter Selasky To: Andriy Gapon Date: Thu, 15 Dec 2011 23:56:35 +0100 User-Agent: KMail/1.13.5 (FreeBSD/8.2-STABLE; KDE/4.4.5; amd64; ; ) References: <4EE51CB5.1060505@FreeBSD.org> <201112142256.32526.hselasky@c2i.net> <4EEA015D.8020800@FreeBSD.org> In-Reply-To: <4EEA015D.8020800@FreeBSD.org> X-Face: *nPdTl_}RuAI6^PVpA02T?$%Xa^>@hE0uyUIoiha$pC:9TVgl.Oq, NwSZ4V"|LR.+tj}g5 %V,x^qOs~mnU3]Gn; cQLv&.N>TrxmSFf+p6(30a/{)KUU!s}w\IhQBj}[g}bj0I3^glmC( :AuzV9:.hESm-x4h240C`9=w MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201112152356.35336.hselasky@c2i.net> Cc: usb@freebsd.org, freebsd-hackers@freebsd.org, mdf@freebsd.org Subject: Re: kern_yield vs ukbd_yield X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2011 22:59:10 -0000 On Thursday 15 December 2011 15:17:01 Andriy Gapon wrote: > on 14/12/2011 23:56 Hans Petter Selasky said the following: > > On Wednesday 14 December 2011 16:37:50 Andriy Gapon wrote: > >> So, Hans Petter, do you recall any details of this problem? > >> I am curious about which thread got starved by which. > > > > From what I know this was 100% reproducible. > > > > Remove the ukbd_yield() when at the mountroot prompt. Result: cannot type > > any keys. No USB devices will enumerate! > > This hasn't satisfied my curiosity :) > > >> BTW, given your recent improvements to pause(9) what do you think about > >> further extending it to also use DELAY(9) when kdb_active is set or when > >> SCHEDULER_STOPPED() is true? > > > > I think this is a good idea. It already checks for "cold". USB usually > > doesn't use this function though when polling. > > > >> Then, probably, pause(9) could be used for > >> both branches in ukbd_do_poll and they could be collapsed together. > > Hmm... I looked at the history of ukbd.c (which I should have done from the > very start) and I see two relevant revisions: > http://svnweb.freebsd.org/base/head/sys/dev/usb/input/ukbd.c?r1=199816&r2=2 > 03896&pathrev=203896 > http://svnweb.freebsd.org/base/head/sys/dev/usb/input/ukbd.c?r1=223755&r2= > 223989&pathrev=223989 > > So, first use of pause(9) was introduced to work around current broken > syscons polling mechanism. Then pause(9) was replaced with the > hand-rolled yield to fix a problem at shutdown, which supposedly was > caused by times being stopped. > > None of the commits seems to be directly related to thread priorities. Not directly, but indirect. You know, if you pause thread 1 (which I thought was thread 0), then other thread will get a chance to run. It might also be that Giant is locked by syscons, and that unless you pause or yield, then Giant will block other parts of USB, like the callback thread, which is Giant locked for ukbd only to run. Maybe that is the explanation? > I > suspect that even DELAY(9) could have worked in the place of the > pause/yield. Also, I do not see any mentioning of the mountroot context in > the commits. Not sure why I even assumed that it was relevant. > > BTW, maybe we should have a 'cold again' flag for late stages of system > shutdown? > > So, all in all, I believe in the following two things: > > 1. kern_yield can be used instead of ukbd_yield with any argument > > 1.1. DELAY() can be used instead of ukbd_yield too No, it doesn't drop Giant, see above. > > 2. all that special code should not be needed once I commit the patches > that I have recently posted to arch@ (I do intend to commit them). In > that case the keyboard drivers would be properly put into the polling mode > instead of the current situation where the polling mode is repeatedly > entered and exited when kernel needs some input. Just make sure all the use-cases work. Then you can safely commit: 1) mountroot prompt 2) dump at shutdown 3) kernel debugger (enter and exit) keyboard should work before and after. > > Maybe you recall/know more than you wrote above and I have missed something > obvious that you can point out? --HPS From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 15 23:06:04 2011 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 828D8106566B; Thu, 15 Dec 2011 23:06:04 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 2BA1C8FC12; Thu, 15 Dec 2011 23:06:02 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id BAA06233; Fri, 16 Dec 2011 01:05:58 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1RbKNC-0002gY-2b; Fri, 16 Dec 2011 01:05:58 +0200 Message-ID: <4EEA7D52.4000503@FreeBSD.org> Date: Fri, 16 Dec 2011 01:05:54 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111206 Thunderbird/8.0 MIME-Version: 1.0 To: Hans Petter Selasky References: <4EE51CB5.1060505@FreeBSD.org> <201112142256.32526.hselasky@c2i.net> <4EEA015D.8020800@FreeBSD.org> <201112152356.35336.hselasky@c2i.net> In-Reply-To: <201112152356.35336.hselasky@c2i.net> X-Enigmail-Version: undefined Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: usb@FreeBSD.org, freebsd-hackers@FreeBSD.org, John Baldwin , mdf@FreeBSD.org Subject: Re: kern_yield vs ukbd_yield X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2011 23:06:04 -0000 on 16/12/2011 00:56 Hans Petter Selasky said the following: > On Thursday 15 December 2011 15:17:01 Andriy Gapon wrote: >> Hmm... I looked at the history of ukbd.c (which I should have done from the >> very start) and I see two relevant revisions: >> http://svnweb.freebsd.org/base/head/sys/dev/usb/input/ukbd.c?r1=199816&r2=2 >> 03896&pathrev=203896 >> http://svnweb.freebsd.org/base/head/sys/dev/usb/input/ukbd.c?r1=223755&r2= >> 223989&pathrev=223989 >> >> So, first use of pause(9) was introduced to work around current broken >> syscons polling mechanism. Then pause(9) was replaced with the >> hand-rolled yield to fix a problem at shutdown, which supposedly was >> caused by times being stopped. >> >> None of the commits seems to be directly related to thread priorities. > > Not directly, but indirect. You know, if you pause thread 1 (which I thought > was thread 0), then other thread will get a chance to run. > > It might also be that Giant is locked by syscons, and that unless you pause or > yield, then Giant will block other parts of USB, like the callback thread, > which is Giant locked for ukbd only to run. > > Maybe that is the explanation? Maybe. Perhaps even. But let me quote the commit messages just in case. Commit message #1: Detect when we are polling from kernel via cngetc() in the boot process and reserve the keypresses so they do not get passed to syscons. Commit message #2: Fix for dump after shutdown with USB keyboard plugged in. It appears that the system timer is stopped during shutdown and that the pause() statement in ukbd causes infinite hang in this regard. The fix is to use mi_switch() instead of pause() to do the required task switch to ensure that the required USB processes get executed. So the reason I asked the above question was that the issues that we are discussing now were never mentioned before. So if you know that those issue really exist, then maybe it is worthwhile describing and documenting them in detail. As you can see the commit messages mention neither thread priorities nor Giant, instead they talk about other rather specific (and plausible) issues. -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 15 23:19:08 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 39D201065672; Thu, 15 Dec 2011 23:19:08 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe06.c2i.net [212.247.154.162]) by mx1.freebsd.org (Postfix) with ESMTP id E294F8FC12; Thu, 15 Dec 2011 23:19:06 +0000 (UTC) X-T2-Spam-Status: No, hits=-0.2 required=5.0 tests=ALL_TRUSTED, BAYES_50 Received: from [188.126.198.129] (account mc467741@c2i.net HELO laptop002.hselasky.homeunix.org) by mailfe06.swip.net (CommuniGate Pro SMTP 5.4.2) with ESMTPA id 216457661; Fri, 16 Dec 2011 00:19:04 +0100 From: Hans Petter Selasky To: freebsd-hackers@freebsd.org Date: Fri, 16 Dec 2011 00:16:33 +0100 User-Agent: KMail/1.13.5 (FreeBSD/8.2-STABLE; KDE/4.4.5; amd64; ; ) References: <4EE51CB5.1060505@FreeBSD.org> <201112152356.35336.hselasky@c2i.net> <4EEA7D52.4000503@FreeBSD.org> In-Reply-To: <4EEA7D52.4000503@FreeBSD.org> X-Face: *nPdTl_}RuAI6^PVpA02T?$%Xa^>@hE0uyUIoiha$pC:9TVgl.Oq, NwSZ4V"|LR.+tj}g5 %V,x^qOs~mnU3]Gn; cQLv&.N>TrxmSFf+p6(30a/{)KUU!s}w\IhQBj}[g}bj0I3^glmC( :AuzV9:.hESm-x4h240C`9=w MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201112160016.33776.hselasky@c2i.net> Cc: usb@freebsd.org, mdf@freebsd.org, Andriy Gapon Subject: Re: kern_yield vs ukbd_yield X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2011 23:19:08 -0000 On Friday 16 December 2011 00:05:54 Andriy Gapon wrote: > on 16/12/2011 00:56 Hans Petter Selasky said the following: > > On Thursday 15 December 2011 15:17:01 Andriy Gapon wrote: > >> Hmm... I looked at the history of ukbd.c (which I should have done from > >> the very start) and I see two relevant revisions: > >> http://svnweb.freebsd.org/base/head/sys/dev/usb/input/ukbd.c?r1=199816&r > >> 2=2 03896&pathrev=203896 > >> http://svnweb.freebsd.org/base/head/sys/dev/usb/input/ukbd.c?r1=223755&r > >> 2= 223989&pathrev=223989 > >> > >> So, first use of pause(9) was introduced to work around current broken > >> syscons polling mechanism. Then pause(9) was replaced with the > >> hand-rolled yield to fix a problem at shutdown, which supposedly was > >> caused by times being stopped. > >> > >> None of the commits seems to be directly related to thread priorities. > > > > Not directly, but indirect. You know, if you pause thread 1 (which I > > thought was thread 0), then other thread will get a chance to run. > > > > It might also be that Giant is locked by syscons, and that unless you > > pause or yield, then Giant will block other parts of USB, like the > > callback thread, which is Giant locked for ukbd only to run. > > > > Maybe that is the explanation? > > Maybe. Perhaps even. But let me quote the commit messages just in case. > > Commit message #1: > Detect when we are polling from kernel via cngetc() in the boot process and > reserve the keypresses so they do not get passed to syscons. > > Commit message #2: > Fix for dump after shutdown with USB keyboard plugged in. It appears that > the system timer is stopped during shutdown and that the pause() statement > in ukbd causes infinite hang in this regard. The fix is to use mi_switch() > instead of pause() to do the required task switch to ensure that the > required USB processes get executed. > > So the reason I asked the above question was that the issues that we are > discussing now were never mentioned before. So if you know that those > issue really exist, then maybe it is worthwhile describing and documenting > them in detail. As you can see the commit messages mention neither thread > priorities nor Giant, instead they talk about other rather specific (and > plausible) issues. Hi, I think I was not aware about the Giant locking maybe having something to do about this. I was just thinking about this recently, that syscons and all keyboard stuff, currently is Giant locked. Scary? I can always become better writing commit messages! What is your plan forward then with regard to the Giant lock problem? If one thread is like: while (1) { lock(Giant); c = get nonblocking key from console(); unlock(Giant); } And the other is like: if (callback complete) { lock(Giant); run callback(); unlock(Giant); } Who gets to run? --HPS From owner-freebsd-hackers@FreeBSD.ORG Fri Dec 16 05:01:28 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 861A0106566B for ; Fri, 16 Dec 2011 05:01:28 +0000 (UTC) (envelope-from janm-freebsd-hackers@transactionware.com) Received: from midgard.transactionware.com (mail2.transactionware.com [203.14.245.36]) by mx1.freebsd.org (Postfix) with SMTP id D59748FC0C for ; Fri, 16 Dec 2011 05:01:27 +0000 (UTC) Received: (qmail 14973 invoked by uid 907); 16 Dec 2011 05:01:24 -0000 Received: from jmmacpro.transactionware.com (HELO jmmacpro.transactionware.com) (192.168.1.33) by midgard.transactionware.com (qpsmtpd/0.82) with ESMTP; Fri, 16 Dec 2011 16:01:24 +1100 Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: text/plain; charset=iso-8859-1 From: Jan Mikkelsen In-Reply-To: <201112150956.45214.jhb@freebsd.org> Date: Fri, 16 Dec 2011 16:01:24 +1100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <4EE8A005.5030607@barafranca.com> <9317551F-CBE0-4368-B798-498E58E240B2@sarenet.es> <2EA3FFF4-E6A2-4371-8891-26E99C551C67@transactionware.com> <201112150956.45214.jhb@freebsd.org> To: John Baldwin X-Mailer: Apple Mail (2.1251.1) Cc: freebsd-hackers@freebsd.org, Hugo Silva , Borja Marcos Subject: Re: mfi (Dell H700) + hot swapping doesn't appear to work with RC1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2011 05:01:28 -0000 On 16/12/2011, at 1:56 AM, John Baldwin wrote: > On Thursday, December 15, 2011 4:19:58 am Jan Mikkelsen wrote: >> For the mfi controllers I have been testing recently (MegaRAID = 9261-8i), you=20 > need to install the sysutils/megacli port, and use that to clear the=20= > "foreignness" of the disk you just added. Something like: >>=20 >> MegaCli -CfgForeign -Clear -a0 >>=20 >> You should be able to then recreate it as a JBOD device, and progress=20= > through whatever higher level recovery you need to do. >=20 > Can you do this by marking it as 'good' via mfiutil and then using = mfiutil > to create a volume? I was going to reply and say that mfiutil will complain about the drive = being in the wrong state, but after reading the other replies I decided = to test. With a blank drive, yes, you can use mfiutil to recreate the jbod = device. You don't even need to do an "mfiutil good" first. If you use a drive that has previously been used by an mfi controller, = it shows up as "bad". Doing "mfiutil good" makes it go to the = "unconfigured good" state. Then creation of the jbod fails with this = error: mfiutil: Command failed: Wrong firmware or drive state mfiutil: Failed to add volume: Input/output error At this point you need to reach for "MegaCli -CfgForeign" and deal with = the now foreign drive. You can use -Import (as pointed out by Andrew Boyer) or -Clear. In my = previous testing (on which my original reply was based), I used drives = that were being moved between machines and so my procedure ended up = being -Clear because I did not want the drive to have the same = configuration as the last time it was used. That was followed a dd from = /dev/zero and then the higher level steps. I have just tested -Import = for the same slot and it worked fine for me. I have not tested -Import = when putting the drive into a different slot. Regards, Jan. From owner-freebsd-hackers@FreeBSD.ORG Fri Dec 16 05:47:37 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7D2EF106564A for ; Fri, 16 Dec 2011 05:47:37 +0000 (UTC) (envelope-from janm-freebsd-hackers@transactionware.com) Received: from midgard.transactionware.com (mail2.transactionware.com [203.14.245.36]) by mx1.freebsd.org (Postfix) with SMTP id CFD8E8FC08 for ; Fri, 16 Dec 2011 05:47:36 +0000 (UTC) Received: (qmail 15791 invoked by uid 907); 16 Dec 2011 05:47:35 -0000 Received: from jmmacpro.transactionware.com (HELO jmmacpro.transactionware.com) (192.168.1.33) by midgard.transactionware.com (qpsmtpd/0.82) with ESMTP; Fri, 16 Dec 2011 16:47:35 +1100 Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: text/plain; charset=us-ascii From: Jan Mikkelsen In-Reply-To: <3A13B3B1-4CDE-4D5A-A6F3-E7DC5BB0E4E3@averesystems.com> Date: Fri, 16 Dec 2011 16:47:34 +1100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <4EE8A005.5030607@barafranca.com> <9317551F-CBE0-4368-B798-498E58E240B2@sarenet.es> <2EA3FFF4-E6A2-4371-8891-26E99C551C67@transactionware.com> <4EEA1215.8060507@barafranca.com> <4EEA1A64.1040104@barafranca.com> <3A13B3B1-4CDE-4D5A-A6F3-E7DC5BB0E4E3@averesystems.com> To: Andrew Boyer X-Mailer: Apple Mail (2.1251.1) Cc: freebsd-hackers@freebsd.org, Hugo Silva Subject: Re: mfi (Dell H700) + hot swapping doesn't appear to work with RC1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2011 05:47:37 -0000 On 16/12/2011, at 3:40 AM, Andrew Boyer wrote: >=20 > On Dec 15, 2011, at 4:19 AM, Jan Mikkelsen wrote: >=20 >> For the mfi controllers I have been testing recently (MegaRAID = 9261-8i), you need to install the sysutils/megacli port, and use that to = clear the "foreignness" of the disk you just added. Something like: >>=20 >> MegaCli -CfgForeign -Clear -a0 >=20 > I don't think that's what you want. You want to use -Import, not = -Clear, to keep your data intact. OK. When I did a -Clear and recreated the drive as a single disk raid0 = volume, the data was still there, but I wanted it to go away. > On Dec 15, 2011, at 11:03 AM, Hugo Silva wrote: >=20 >> On 12/15/11 15:28, Hugo Silva wrote: >>> As Borja said, part of the difficulty is the H700 abstracting a = single >>> disk as a RAID-0, I guess. So far I've been unable to find a way to >>> bring the drive back, except by rebooting and recreating. >>=20 >> Turns out no interaction is needed after reboot. It was something = else >> unrelated. The main issue then is convincing the controller to once >> again accept the hard disk. I'm going through MegaCli "documentation" >> (ie --help).. it's not a pretty place. >=20 > I'm not sure it would even be possible to come up with a worse = interface. It boggles the mind. I agree. It is insanely bad. > I recommend you always run with this configuration: >=20 > # MegaCli -AdpSetProp AutoEnhancedImportEnbl -aALL > # MegaCli -AdpSetProp MaintainPdFailHistoryEnbl -0 -aALL >=20 > AutoEnhancedImportEnbl will bring the foreign disk back in on a = reboot. LSI recommends turning off MaintainPdFailHistory when using = single-disk RAID0 configurations. What does PD Fail History actually do? > Adding these capabilities to mfiutil is on my list of things to do, = but it's not ready yet. Thanks. > Has anyone managed to get the real JBOD mode working on this = controller? It advertises support in the firmware but doesn't seem to = do anything. The documentation only lists JBOD mode as a feature of the = lower-end controllers. You mean using "MegaCli -PDMakeJBOD"? No, it doesn't work from me on the = 9281-8i. I get "Failed to change PD state". Single disk RAID-0 works = fine. > Hope this helps. It does, thank you. Regards, Jan. From owner-freebsd-hackers@FreeBSD.ORG Fri Dec 16 09:45:59 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 855351065676; Fri, 16 Dec 2011 09:45:59 +0000 (UTC) (envelope-from monthadar@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 41A8A8FC1A; Fri, 16 Dec 2011 09:45:59 +0000 (UTC) Received: by iakl21 with SMTP id l21so7691714iak.13 for ; Fri, 16 Dec 2011 01:45:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=UgxvTbOl9VpgHoleDEKpxNjNR2I3YfLksm++qKABUoM=; b=eKSinIw97YaGSQxpI1iuhUqV4NCTAEg9HpOb6ho9fHVYgns442nHZKePYEiFfYVyJS WwpLufaPbMen/mpd/zobxyklUHbAvnCQCe6pU3eRAlyd+KGXfrJK/j/oe5yIkwzDRtsw /H+1T6mJD7gQsCVupEiaNVZwnO5E9YlUryXIc= MIME-Version: 1.0 Received: by 10.50.153.133 with SMTP id vg5mr7740882igb.80.1324028758726; Fri, 16 Dec 2011 01:45:58 -0800 (PST) Received: by 10.50.51.233 with HTTP; Fri, 16 Dec 2011 01:45:58 -0800 (PST) In-Reply-To: References: <201112130935.33975.jhb@freebsd.org> Date: Fri, 16 Dec 2011 10:45:58 +0100 Message-ID: From: Monthadar Al Jaberi To: Arnaud Lacombe Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-hackers@freebsd.org Subject: Re: loop inside uma_zfree critical section X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2011 09:45:59 -0000 On Wed, Dec 14, 2011 at 9:10 PM, Arnaud Lacombe wrote: > Hi, > > On Wed, Dec 14, 2011 at 2:47 PM, Monthadar Al Jaberi > wrote: >> On Tue, Dec 13, 2011 at 4:50 PM, Monthadar Al Jaberi >> wrote: >>> On Tue, Dec 13, 2011 at 3:35 PM, John Baldwin wrote: >>>> On Tuesday, December 13, 2011 7:46:34 am Monthadar Al Jaberi wrote: >>>>> Hi, >>>>> >>>>> I am not sure why I am having this problem, but looking >>>>> at the code I dont understand uma_core.c really good. >>>>> So I hope someone can shed a light on this: >>>>> >>>>> I am running on an arm board and and running a kernel module >>>>> that behaves like a wlan interface. so I tx and rx packets. >>>>> >>>>> For now tx is only only sending beacon like frames. >>>>> This is done through using ieee80211_beacon_alloc(). >>>>> >>>>> Then in a callout task to generate periodic beacons: >>>>> >>>>> =A0 =A0 m_dup(avp->beacon, M_DONTWAIT); >>>>> =A0 =A0 mtx_lock(...); >>>>> =A0 =A0 STAILQ_INSERT_TAIL(...); >>>>> =A0 =A0 taskqueue_enqueue(...); >>>>> =A0 =A0 mtx_unlock(...); >>>>> =A0 =A0 callout_schedule(...); >>>>> >>>>> On the RX side, the interrupt handler will read out buffer >>>>> then place it on a queue to be handled by wlan-glue code. >>>>> For now wlan-glue code just frees the mbuf it instead of >>>>> calling net80211 ieee80211_input() functions: >>>>> >>>>> =A0 =A0 m_copyback(...); >>>>> =A0 =A0 /* Allocate new mbuf for next RX. */ >>>>> =A0 =A0 MGETHDR(..., M_DONTWAIT, MT_DATA); >>>>> =A0 =A0 bzero((mtod(sc->Rx_m, void *)), MHLEN); >>>>> =A0 =A0 sc->Rx_m->m_len =3D 0; /* NB: m_gethdr does not set */ >>>>> =A0 =A0 sc->Rx_m->m_data +=3D 20; /* make headroom */ >>>>> >>>>> Then I use a lockmgr inside my kernel module that should >>>>> make sure that we either are on TX or RX path. >>>> >>>> Uh, you can't use a lockmgr lock in interrupt handlers or in >>>> if_transmit/if_start routines. =A0You should most likely just be using= a plain >>>> mutex instead. =A0Also, new code shouldn't use lockmgr in general. =A0= If you >>>> need a sleepable lock, use sx instead. =A0It has a more straightforwar= d API. >>> >>> Ok, I will change the interrupt handler to do something like this: >>> >>> =A0 =A0disaple_interrupt(); >>> =A0 =A0taskqueue_enqueue(...); /* on new rx task queue */ >>> >>> Then on the new rx proc: >>> >>> =A0 =A0sx_slock(...); >>> =A0 =A0m_copyback(...); >>> =A0 =A0enable_interrupt(); >>> =A0 =A0/* Allocate new mbuf for next RX. */ >>> =A0 =A0MGETHDR(..., M_DONTWAIT, MT_DATA); >>> =A0 =A0bzero((mtod(sc->Rx_m, void *)), MHLEN); >>> =A0 =A0sc->Rx_m->m_len =3D 0; /* NB: m_gethdr does not set */ >>> =A0 =A0sc->Rx_m->m_data +=3D 20; /* make headroom */ >>> =A0 =A0sx_sunlock(...); >>> >>> I lock TX/RX paths to make sure my code is threading safe. >>> Also because while programming my deivce (SPI communicatioin) >>> there will be a tsleep() waiting for the DMA interrupt and >>> thus we could be prempted by e.g. a beacon_callout etc... >>> >> >> I did implement your suggestions, using sx and modified interrupt handle= r >> as specified above. But still same problem as before. >> >>>> >>>>> The problem seems to be at [2], somehow after swapping >>>>> buckets in uma_zfree m_dup returns a pointer to >>>>> an mbuf that is still being used by us, [1] and [3] >>>>> have same address. >>>>> Then we call m_freem twice on same mbuf, [4] and [5]. >>>>> And a loop occurs inside uma_free. >>>>> I am using mbufs in a wrong way? Shouldnt mbufs be thread safe? >>>>> Problem seems to occur while swapping buckets. >>>> >>>> Hmm, the uma uses its own locking, so it should be safe, yes. =A0Howev= er, you >>>> are correct about [1] and [3]. =A0The thing is, after [1] the mbuf sho= uldn't >>>> be in any buckets at all (it only gets put back into the bucket during= a >>>> free). =A0Are you sure the mbuf wasn't double free'd previously? >> >> I rechecked and it is almost certain that I dont double free the mbuf >> before [1]. >> And its not like it crashed in the beginning, it does run for a while >> and then it crashes. So our code works for like a hundred beacons sent/r= eceived >> between two arm boards. Its feels like something is preempted, which exp= lains >> why the mbuf is still in the bucket (wrongly)? >> > are you running an INVARIANTS/DIAGNOSTICS/WITNESS/LOCK_DEBUG/... > enabled kernel ? > > are you running on an SMP platform where there might be cache-coherency i= ssue ? Sorry for late answer, I added DIAGNOSTIC/WITNESS didnt see anything strange except for a couple of LORs... If I add INVARIANTS I couldnt login in at all.... it comes to login promt but I cant type anything... I am running one arm cpu, so its not an SMP platform. This is a snippet of my kernel config: makeoptions DEBUG=3D-g #Build kernel with gdb(1) debug symbols options DDB options KDB options SCHED_4BSD #4BSD scheduler options INET #InterNETworking options INET6 #IPv6 communications protocols options FFS #Berkeley Fast Filesystem options DIAGNOSTIC options UFS_ACL #Support for access control lists options UFS_DIRHASH #Improve performance on big directories options KTRACE #ktrace(1) support options SYSVSHM #SYSV-style shared memory options SYSVMSG #SYSV-style message queues options SYSVSEM #SYSV-style semaphores options _KPOSIX_PRIORITY_SCHEDULING #Posix P1003_1B real-time extensions options MUTEX_NOINLINE options RWLOCK_NOINLINE options SX_NOINLINE options NO_FFS_SNAPSHOT options NO_SWAPPING options DEADLKRES options INVARIANTS options INVARIANT_SUPPORT options WITNESS > > Thanks, > =A0- Arnaud Thanks, --=20 Monthadar Al Jaberi From owner-freebsd-hackers@FreeBSD.ORG Fri Dec 16 11:27:35 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id 9EB991065675; Fri, 16 Dec 2011 11:27:35 +0000 (UTC) Date: Fri, 16 Dec 2011 11:27:35 +0000 From: Alexander Best To: freebsd-hackers@freebsd.org Message-ID: <20111216112735.GB17565@freebsd.org> References: <20111204222203.GA8898@freebsd.org> <20111206215435.GA14529@zim.MIT.EDU> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="GvXjxJ+pjyke8COw" Content-Disposition: inline In-Reply-To: <20111206215435.GA14529@zim.MIT.EDU> Subject: Re: strange printf(9) format specifier ("Z") in dev/drm code X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2011 11:27:35 -0000 --GvXjxJ+pjyke8COw Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Tue Dec 6 11, David Schultz wrote: > On Sun, Dec 04, 2011, Alexander Best wrote: > > ... i couldn't find a reference to an upercase "Z" in the printf(9) man page. > > i talked to dinoex on #freebsd-clang (EFNet) and he said that the "Z" might > > come from linux'es libc5 and is the equaivalent to glibc's "z". > > > > can we adjust those lines, so the clang warnings disappear? > > Yes, 'Z' is an alias for 'z' that is deprecated in glibc and > unsupported in FreeBSD, so changing the case should fix it. any chance we could get the following patch committed? cheers. alex --GvXjxJ+pjyke8COw Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="mga_drv.h.diff" Index: sys/dev/drm/mga_drv.h =================================================================== --- sys/dev/drm/mga_drv.h (revision 228567) +++ sys/dev/drm/mga_drv.h (working copy) @@ -288,7 +288,7 @@ do { \ if ( MGA_VERBOSE ) { \ DRM_INFO( "BEGIN_DMA( %d )\n", (n) ); \ - DRM_INFO( " space=0x%x req=0x%Zx\n", \ + DRM_INFO( " space=0x%x req=0x%zx\n", \ dev_priv->prim.space, (n) * DMA_BLOCK_SIZE ); \ } \ prim = dev_priv->prim.start; \ @@ -338,7 +338,7 @@ #define DMA_WRITE( offset, val ) \ do { \ if ( MGA_VERBOSE ) { \ - DRM_INFO( " DMA_WRITE( 0x%08x ) at 0x%04Zx\n", \ + DRM_INFO( " DMA_WRITE( 0x%08x ) at 0x%04zx\n", \ (u32)(val), write + (offset) * sizeof(u32) ); \ } \ *(volatile u32 *)(prim + write + (offset) * sizeof(u32)) = val; \ --GvXjxJ+pjyke8COw-- From owner-freebsd-hackers@FreeBSD.ORG Fri Dec 16 12:45:44 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EF1101065676 for ; Fri, 16 Dec 2011 12:45:44 +0000 (UTC) (envelope-from se@freebsd.org) Received: from nm24-vm0.bullet.mail.sp2.yahoo.com (nm24-vm0.bullet.mail.sp2.yahoo.com [98.139.91.226]) by mx1.freebsd.org (Postfix) with SMTP id CF3908FC18 for ; Fri, 16 Dec 2011 12:45:44 +0000 (UTC) Received: from [98.139.91.68] by nm24.bullet.mail.sp2.yahoo.com with NNFMP; 16 Dec 2011 12:45:44 -0000 Received: from [208.71.42.205] by tm8.bullet.mail.sp2.yahoo.com with NNFMP; 16 Dec 2011 12:44:44 -0000 Received: from [127.0.0.1] by smtp216.mail.gq1.yahoo.com with NNFMP; 16 Dec 2011 12:44:44 -0000 X-Yahoo-Newman-Id: 395061.40140.bm@smtp216.mail.gq1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: 6DgGJM8VM1kGbLBtIBrOCMWJxsE0IwWJ9X.vJGGcQZgaBvI _F8NCxrnSUPbm0We5apccTG0bY7NIouV4ybzUnAXttiaH1RN8OCL97sSanho 3.rBcxIkZJJGHUEMZsFGIRGIQadIayQLTUc_ily3tc6hs0TOFifraCY4yCJw TUFyKxOqlFqVq8..c1q.rD8lRya4Hbv7BnkX23RWKviUxxC6iX0yMw0Lp30b UGIxXRjRxCbOBsS7RtVQYHosGt9LYHaJVG0g14G8BZWE00fm65ccPqMeIBVD _ZbBjlAZZGCWMipqFbqARZUT6VKFd9m5leg1H2I_dDR6tB7XZHWPXcLH6AHP NP7EKwiySKJMHWIDYZMgUDbBHvPYK0dPuq34uvGLAaHPXBQfpNMseZGdSmkz m2OeGotx8sLtf X-Yahoo-SMTP: iDf2N9.swBDAhYEh7VHfpgq0lnq. Received: from [192.168.119.20] (se@81.173.157.6 with plain) by smtp216.mail.gq1.yahoo.com with SMTP; 16 Dec 2011 04:44:43 -0800 PST Message-ID: <4EEB3D35.2060707@freebsd.org> Date: Fri, 16 Dec 2011 13:44:37 +0100 From: Stefan Esser User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0) Gecko/20111105 Thunderbird/8.0 MIME-Version: 1.0 To: Alexander Best References: <20111204222203.GA8898@freebsd.org> <20111206215435.GA14529@zim.MIT.EDU> <20111216112735.GB17565@freebsd.org> In-Reply-To: <20111216112735.GB17565@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: strange printf(9) format specifier ("Z") in dev/drm code X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2011 12:45:45 -0000 Am 16.12.2011 12:27, schrieb Alexander Best: > On Tue Dec 6 11, David Schultz wrote: >> On Sun, Dec 04, 2011, Alexander Best wrote: >>> ... i couldn't find a reference to an upercase "Z" in the printf(9) man page. >>> i talked to dinoex on #freebsd-clang (EFNet) and he said that the "Z" might >>> come from linux'es libc5 and is the equaivalent to glibc's "z". >>> >>> can we adjust those lines, so the clang warnings disappear? >> >> Yes, 'Z' is an alias for 'z' that is deprecated in glibc and >> unsupported in FreeBSD, so changing the case should fix it. > > any chance we could get the following patch committed? Commited to head as r228572. I do not see a need for an MFC as long as clang does not become the standard compiler in FreeBSD-9. Regards, STefan From owner-freebsd-hackers@FreeBSD.ORG Fri Dec 16 15:00:47 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 50B04106564A for ; Fri, 16 Dec 2011 15:00:47 +0000 (UTC) (envelope-from aboyer@averesystems.com) Received: from zimbra.averesystems.com (75-149-8-245-Pennsylvania.hfc.comcastbusiness.net [75.149.8.245]) by mx1.freebsd.org (Postfix) with ESMTP id 043828FC16 for ; Fri, 16 Dec 2011 15:00:46 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by zimbra.averesystems.com (Postfix) with ESMTP id EE474446002; Fri, 16 Dec 2011 10:01:42 -0500 (EST) X-Virus-Scanned: amavisd-new at averesystems.com Received: from zimbra.averesystems.com ([127.0.0.1]) by localhost (zimbra.averesystems.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HGaCTJFrvwJw; Fri, 16 Dec 2011 10:01:39 -0500 (EST) Received: from riven.arriad.com (fw.arriad.com [10.0.0.16]) by zimbra.averesystems.com (Postfix) with ESMTPSA id EDB0C446009; Fri, 16 Dec 2011 10:01:38 -0500 (EST) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Andrew Boyer In-Reply-To: Date: Fri, 16 Dec 2011 10:00:40 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <937308E9-F456-4DB6-9C3C-215DE07E92B4@averesystems.com> References: <4EE8A005.5030607@barafranca.com> <9317551F-CBE0-4368-B798-498E58E240B2@sarenet.es> <2EA3FFF4-E6A2-4371-8891-26E99C551C67@transactionware.com> <4EEA1215.8060507@barafranca.com> <4EEA1A64.1040104@barafranca.com> <3A13B3B1-4CDE-4D5A-A6F3-E7DC5BB0E4E3@averesystems.com> To: Jan Mikkelsen X-Mailer: Apple Mail (2.1084) Cc: freebsd-hackers@freebsd.org, Hugo Silva Subject: Re: mfi (Dell H700) + hot swapping doesn't appear to work with RC1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2011 15:00:47 -0000 On Dec 16, 2011, at 12:47 AM, Jan Mikkelsen wrote: > On 16/12/2011, at 3:40 AM, Andrew Boyer wrote: >> On Dec 15, 2011, at 4:19 AM, Jan Mikkelsen wrote: >>=20 >>> For the mfi controllers I have been testing recently (MegaRAID = 9261-8i), you need to install the sysutils/megacli port, and use that to = clear the "foreignness" of the disk you just added. Something like: >>>=20 >>> MegaCli -CfgForeign -Clear -a0 >>=20 >> I don't think that's what you want. You want to use -Import, not = -Clear, to keep your data intact. >=20 > OK. When I did a -Clear and recreated the drive as a single disk raid0 = volume, the data was still there, but I wanted it to go away. The RAID identity is stored on a 512MB partition at the end of the disk. = Clearing and recreating it doesn't actually affect your data, as you = discovered. You can even plug one of your RAID0 disks into a non-RAID = controller and your data will be there. mfiutil has a 'drive clear' feature to zero disks. Or you could just dd = a few megs of zeroes to the beginning of the disk. >=20 >> I recommend you always run with this configuration: >>=20 >> # MegaCli -AdpSetProp AutoEnhancedImportEnbl -aALL >> # MegaCli -AdpSetProp MaintainPdFailHistoryEnbl -0 -aALL >>=20 >> AutoEnhancedImportEnbl will bring the foreign disk back in on a = reboot. LSI recommends turning off MaintainPdFailHistory when using = single-disk RAID0 configurations. >=20 > What does PD Fail History actually do? See: http://kb.lsi.com/KnowledgebaseArticle16570.aspx -Andrew -------------------------------------------------- Andrew Boyer aboyer@averesystems.com From owner-freebsd-hackers@FreeBSD.ORG Fri Dec 16 22:27:14 2011 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BC8E21065676 for ; Fri, 16 Dec 2011 22:27:14 +0000 (UTC) (envelope-from crmartin@sgi.com) Received: from relay.sgi.com (relay3.sgi.com [192.48.152.1]) by mx1.freebsd.org (Postfix) with ESMTP id A565A8FC19 for ; Fri, 16 Dec 2011 22:27:14 +0000 (UTC) Received: from xmail.sgi.com (pv-excas2-dc21.corp.sgi.com [137.38.102.196]) by relay3.corp.sgi.com (Postfix) with ESMTP id 431FBAC00B for ; Fri, 16 Dec 2011 14:11:51 -0800 (PST) Received: from [10.3.0.220] (10.3.0.220) by xmail.sgi.com (137.38.102.30) with Microsoft SMTP Server (TLS) id 14.1.339.1; Fri, 16 Dec 2011 16:11:51 -0600 Message-ID: <4EEBC226.6080402@sgi.com> Date: Fri, 16 Dec 2011 15:11:50 -0700 From: Charlie Martin Organization: Silicon Graphics, Inc. User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.24) Gecko/20111101 SUSE/3.1.16 Lightning/1.0b2 Thunderbird/3.1.16 MIME-Version: 1.0 To: Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [10.3.0.220] Cc: "Peter W. Morreale" Subject: Spinlock panic in FreeBSD 7 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2011 22:27:14 -0000 We've observed a panic in FreeBSD 7 (7.2-PRERELEASE FreeBSD) several times that we've not been able to track down. Upgrading to 8+ is not an option at this time. Does this look at all familiar to anyone? Here's an example stack trace after panic: spin lock 0xffffffff8086bdc0 (smp rendezvous) held by 0xffffff0006d1f000 (tid 100060) too long panic: spin lock held too long cpuid = 0 KDB: stack backtrace: db_trace_self_wrapper() at 0xffffffff8019120a = db_trace_self_wrapper+0x2a panic() at 0xffffffff80308797 = panic+0x187 _mtx_lock_spin_failed() at 0xffffffff802fbda9 = _mtx_lock_spin_failed+0x39 _mtx_lock_spin() at 0xffffffff802fbe4e = _mtx_lock_spin+0x9e _mtx_lock_spin_flags() at 0xffffffff802fc354 = _mtx_lock_spin_flags+0x104 smp_rendezvous_cpus() at 0xffffffff80340cb3 = smp_rendezvous_cpus+0xd3 xcall() at 0xffffffff80ad755e = xcall+0x3e cyclic_remove_here() at 0xffffffff80ad7715 = cyclic_remove_here+0x1a5 cyclic_remove() at 0xffffffff80ad7a0f = cyclic_remove+0x5f profile_disable() at 0xffffffff80acf0e5 = profile_disable+0x15 dtrace_state_destroy() at 0xffffffff80adfabd = dtrace_state_destroy+0x35d dtrace_close() at 0xffffffff80adffed = dtrace_close+0x8d devfs_close() at 0xffffffff802a825d = devfs_close+0x2dd vn_close() at 0xffffffff8039cb06 = vn_close+0xb6 vn_closefile() at 0xffffffff8039cc00 = vn_closefile+0x80 devfs_close_f() at 0xffffffff802a5738 = devfs_close_f+0x28 fdrop() at 0xffffffff802d98bb = fdrop+0xdb closef() at 0xffffffff802db2f9 = closef+0x29 fdfree() at 0xffffffff802dc061 = fdfree+0x161 exit1() at 0xffffffff802e56b2 = exit1+0x2c2 sigexit() at 0xffffffff8030a86f = sigexit+0x8f postsig() at 0xffffffff8030b6ce = postsig+0x38e ast() at 0xffffffff803425f7 = ast+0x337 Xfast_syscall() at 0xffffffff80494efd = Xfast_syscall+0xdd --- syscall (32, FreeBSD ELF64, getsockname), rip = 0x800df4d5c, rsp = 0x7fffffffe398, rbp = 0x5 --- KDB: enter: panic The panic always shows up from a syscall, and almost always from syscall 32, getsockname, but we've also observed it with syscall 5. -- Charles R. (Charlie) Martin Senior Software Engineer SGI logo 1900 Pike Road Longmont, CO 80501 Phone: 303-532-0209 E-Mail: CRMartin@sgi.com Website: www.sgi.com From owner-freebsd-hackers@FreeBSD.ORG Fri Dec 16 22:30:46 2011 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx2.freebsd.org (mx2.freebsd.org [IPv6:2001:4f8:fff6::35]) by hub.freebsd.org (Postfix) with ESMTP id BEC51106567C for ; Fri, 16 Dec 2011 22:30:46 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from 172-17-198-245.globalsuite.net (hub.freebsd.org [IPv6:2001:4f8:fff6::36]) by mx2.freebsd.org (Postfix) with ESMTP id 631C5158D09; Fri, 16 Dec 2011 22:30:46 +0000 (UTC) Message-ID: <4EEBC696.9070309@FreeBSD.org> Date: Fri, 16 Dec 2011 14:30:46 -0800 From: Doug Barton Organization: http://SupersetSolutions.com/ User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111110 Thunderbird/8.0 MIME-Version: 1.0 To: Charlie Martin References: <4EEBC226.6080402@sgi.com> In-Reply-To: <4EEBC226.6080402@sgi.com> X-Enigmail-Version: undefined OpenPGP: id=1A1ABC84 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@FreeBSD.org, "Peter W. Morreale" Subject: Re: Spinlock panic in FreeBSD 7 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2011 22:30:46 -0000 On 12/16/2011 14:11, Charlie Martin wrote: > We've observed a panic in FreeBSD 7 (7.2-PRERELEASE FreeBSD) several > times that we've not been able to track down. Upgrading to 8+ is not an > option at this time. First, your post should really be on freebsd-stable@. Second, your first step should be to upgrade to 7-stable. hth, Doug -- [^L] Breadth of IT experience, and depth of knowledge in the DNS. Yours for the right price. :) http://SupersetSolutions.com/ From owner-freebsd-hackers@FreeBSD.ORG Sat Dec 17 14:57:32 2011 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BA7AA1065672; Sat, 17 Dec 2011 14:57:32 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 4CA3B8FC12; Sat, 17 Dec 2011 14:57:30 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id QAA05434; Sat, 17 Dec 2011 16:57:25 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1RbvhV-0009fe-ES; Sat, 17 Dec 2011 16:57:25 +0200 Message-ID: <4EECADD4.9040509@FreeBSD.org> Date: Sat, 17 Dec 2011 16:57:24 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111206 Thunderbird/8.0 MIME-Version: 1.0 To: Hans Petter Selasky References: <4EE51CB5.1060505@FreeBSD.org> <201112152356.35336.hselasky@c2i.net> <4EEA7D52.4000503@FreeBSD.org> <201112160016.33776.hselasky@c2i.net> In-Reply-To: <201112160016.33776.hselasky@c2i.net> X-Enigmail-Version: undefined Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: usb@FreeBSD.org, freebsd-hackers@FreeBSD.org, John Baldwin , mdf@FreeBSD.org Subject: Re: kern_yield vs ukbd_yield X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Dec 2011 14:57:32 -0000 on 16/12/2011 01:16 Hans Petter Selasky said the following: > I think I was not aware about the Giant locking maybe having something to do > about this. I was just thinking about this recently, that syscons and all > keyboard stuff, currently is Giant locked. Scary? Nope :-) I think that no syscons + keyboards code depends on the special properties of the Giant, most likely with the prominent exception of the ukbd. Also, the Giant is almost never acquired explicitly (again with the prominent exception of the ukbd), which actually leads to a consequence that no lock is acquired in the context where the kernel polls for input. Which is a good thing if we keep in mind that the kernel may poll for input in the context like panic or ddb where any lock may be already held by a thread that is not able to run. > I can always become better writing commit messages! What is your plan forward > then with regard to the Giant lock problem? > > If one thread is like: > > while (1) > { > lock(Giant); > c = get nonblocking key from console(); > unlock(Giant); > } > > And the other is like: > > if (callback complete) { > lock(Giant); > run callback(); > unlock(Giant); > } > > Who gets to run? First, there is no need to acquire the Giant in the first snippet. Second, "get nonblocking key from console" should not depend on the callback in a parallel thread. The polling routine should go all the way to hardware if needed. I see where all the complications in ukbd code come from. I think that they started as attempts to work around the current stupid behavior of syscons during polling: do { kbd_poll_enter(); try_to_get_one_char(); kbd_poll_exit(); } while (!got_a_char); For ukbd this means ugly races between the polling thread and the usb threads at the mountroot stage. And so you had to invent "polling autodetection" and add extra locking. But that lead to troubles in the contexts where no lock was previously held. And so on. So now ukbd resembles a big collection of hacks and crotches. For example consider the following scenario which does not occur very frequently in reality, but is quite possible. Thread T1 on CPU P1 holds the Giant, at the same time thread T2 on CPU P2 enters ddb. That means that T1 is "frozen" and won't be able to release Giant. ddb would try to interact with an operator and there would be a call-chain like this: ukbd_check_char scgetc sc_cngetc cncheckc cngetc db_readline ... And now the code in ukbd to which I referred above: /* check if char is waiting */ static int ukbd_check_char(keyboard_t *kbd) { struct ukbd_softc *sc = kbd->kb_data; if (!KBD_IS_ACTIVE(kbd)) return (0); if (sc->sc_flags & UKBD_FLAG_POLLING) { if (!mtx_owned(&Giant)) { /* XXX cludge */ int retval; mtx_lock(&Giant); retval = ukbd_check_char(kbd); mtx_unlock(&Giant); return (retval); So we are in a polling mode and Giant is not owned by us (T2) because it is owned by T1 and so mtx_lock would be called and then we would probably get into a recursion via mi_switch and kdb_reenter. -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Sat Dec 17 14:57:36 2011 Return-Path: Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E4321106566B; Sat, 17 Dec 2011 14:57:36 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 915FC8FC17; Sat, 17 Dec 2011 14:57:35 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id QAA05440; Sat, 17 Dec 2011 16:57:31 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1Rbvhb-0009fh-0a; Sat, 17 Dec 2011 16:57:31 +0200 Message-ID: <4EECADDA.4070704@FreeBSD.org> Date: Sat, 17 Dec 2011 16:57:30 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111206 Thunderbird/8.0 MIME-Version: 1.0 To: Hans Petter Selasky References: <4EE51CB5.1060505@FreeBSD.org> <201112142256.32526.hselasky@c2i.net> <4EEA015D.8020800@FreeBSD.org> <201112152356.35336.hselasky@c2i.net> In-Reply-To: <201112152356.35336.hselasky@c2i.net> X-Enigmail-Version: undefined Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: usb@FreeBSD.org, freebsd-hackers@FreeBSD.org, John Baldwin , mdf@FreeBSD.org Subject: Re: kern_yield vs ukbd_yield X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Dec 2011 14:57:37 -0000 Replying further... on 16/12/2011 00:56 Hans Petter Selasky said the following: > On Thursday 15 December 2011 15:17:01 Andriy Gapon wrote: >> Hmm... I looked at the history of ukbd.c (which I should have done from the >> very start) and I see two relevant revisions: >> http://svnweb.freebsd.org/base/head/sys/dev/usb/input/ukbd.c?r1=199816&r2=2 >> 03896&pathrev=203896 >> http://svnweb.freebsd.org/base/head/sys/dev/usb/input/ukbd.c?r1=223755&r2= >> 223989&pathrev=223989 >> >> So, first use of pause(9) was introduced to work around current broken >> syscons polling mechanism. Then pause(9) was replaced with the >> hand-rolled yield to fix a problem at shutdown, which supposedly was >> caused by times being stopped. >> >> None of the commits seems to be directly related to thread priorities. > > Not directly, but indirect. You know, if you pause thread 1 (which I thought > was thread 0), then other thread will get a chance to run. pause() could be a sufficient action to let other thread run, but it is not a required action. As we've already discusses earlier all the USB threads have sufficiently high priorities to get runnable while other kernel threads are runnable. Only certain interrupt threads could have prevented them from running, but that's definitely not the case. > It might also be that Giant is locked by syscons, and that unless you pause or > yield, then Giant will block other parts of USB, like the callback thread, > which is Giant locked for ukbd only to run. > > Maybe that is the explanation? Not sure if you are hypothesizing or if this is something that you experienced during development and testing. Let's consider a few observations. First, syscons doesn't acquire Giant anywhere in the path started from cngetc. Actually, I believe that the only place where Giant is now explicitly acquired in the whole syscons code can be safely replaced with an assert that the Giant is already held. Second, the polling mode is designed to be usable in situations where other threads are not running. So in general it should not depend on other threads being able to make any progress. -- Andriy Gapon From owner-freebsd-hackers@FreeBSD.ORG Sat Dec 17 17:09:17 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0CDF11065691; Sat, 17 Dec 2011 17:09:17 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe08.c2i.net [212.247.154.226]) by mx1.freebsd.org (Postfix) with ESMTP id 9F4CD8FC14; Sat, 17 Dec 2011 17:09:15 +0000 (UTC) X-T2-Spam-Status: No, hits=-0.2 required=5.0 tests=ALL_TRUSTED, BAYES_50 Received: from [188.126.198.129] (account mc467741@c2i.net HELO laptop002.hselasky.homeunix.org) by mailfe08.swip.net (CommuniGate Pro SMTP 5.4.2) with ESMTPA id 219011199; Sat, 17 Dec 2011 18:09:13 +0100 From: Hans Petter Selasky To: Andriy Gapon Date: Sat, 17 Dec 2011 18:06:43 +0100 User-Agent: KMail/1.13.5 (FreeBSD/8.2-STABLE; KDE/4.4.5; amd64; ; ) References: <4EE51CB5.1060505@FreeBSD.org> <201112160016.33776.hselasky@c2i.net> <4EECADD4.9040509@FreeBSD.org> In-Reply-To: <4EECADD4.9040509@FreeBSD.org> X-Face: *nPdTl_}RuAI6^PVpA02T?$%Xa^>@hE0uyUIoiha$pC:9TVgl.Oq, NwSZ4V"|LR.+tj}g5 %V,x^qOs~mnU3]Gn; cQLv&.N>TrxmSFf+p6(30a/{)KUU!s}w\IhQBj}[g}bj0I3^glmC( :AuzV9:.hESm-x4h240C`9=w MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201112171806.43764.hselasky@c2i.net> Cc: usb@freebsd.org, freebsd-hackers@freebsd.org, mdf@freebsd.org Subject: Re: kern_yield vs ukbd_yield X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Dec 2011 17:09:17 -0000 On Saturday 17 December 2011 15:57:24 Andriy Gapon wrote: > on 16/12/2011 01:16 Hans Petter Selasky said the following: > > I think I was not aware about the Giant locking maybe having something to > > do about this. I was just thinking about this recently, that syscons and > > all keyboard stuff, currently is Giant locked. Scary? > > Nope :-) > I think that no syscons + keyboards code depends on the special properties > of the Giant, most likely with the prominent exception of the ukbd. > Also, the Giant is almost never acquired explicitly (again with the > prominent exception of the ukbd), which actually leads to a consequence > that no lock is acquired in the context where the kernel polls for input. > Which is a good thing if we keep in mind that the kernel may poll for > input in the context like panic or ddb where any lock may be already held > by a thread that is not able to run. > > > I can always become better writing commit messages! What is your plan > > forward then with regard to the Giant lock problem? > > > > If one thread is like: > > > > while (1) > > { > > lock(Giant); > > c = get nonblocking key from console(); > > unlock(Giant); > > } > > > > And the other is like: > > > > if (callback complete) { > > lock(Giant); > > run callback(); > > unlock(Giant); > > } > > > > Who gets to run? > Hi, > First, there is no need to acquire the Giant in the first snippet. > Second, "get nonblocking key from console" should not depend on the > callback in a parallel thread. The polling routine should go all the way > to hardware if needed. It can do that. This feature is usually only used in panic mode. > > I see where all the complications in ukbd code come from. I think that > they started as attempts to work around the current stupid behavior of > syscons during polling: > do { > kbd_poll_enter(); > try_to_get_one_char(); > kbd_poll_exit(); > } while (!got_a_char); > For ukbd this means ugly races between the polling thread and the usb > threads at the mountroot stage. And so you had to invent "polling > autodetection" and add extra locking. But that lead to troubles in the > contexts where no lock was previously held. And so on. So now ukbd > resembles a big collection of hacks and crotches. Right! You got it! > > For example consider the following scenario which does not occur very > frequently in reality, but is quite possible. Thread T1 on CPU P1 holds > the Giant, at the same time thread T2 on CPU P2 enters ddb. That means > that T1 is "frozen" and won't be able to release Giant. ddb would try to > interact with an operator and there would be a call-chain like this: > ukbd_check_char > scgetc > sc_cngetc > cncheckc > cngetc > db_readline > ... > > And now the code in ukbd to which I referred above: > /* check if char is waiting */ > static int > ukbd_check_char(keyboard_t *kbd) > { > struct ukbd_softc *sc = kbd->kb_data; > > if (!KBD_IS_ACTIVE(kbd)) > return (0); > > if (sc->sc_flags & UKBD_FLAG_POLLING) { > if (!mtx_owned(&Giant)) { > /* XXX cludge */ > int retval; > mtx_lock(&Giant); > retval = ukbd_check_char(kbd); > mtx_unlock(&Giant); > return (retval); > > So we are in a polling mode and Giant is not owned by us (T2) because it is > owned by T1 and so mtx_lock would be called and then we would probably get > into a recursion via mi_switch and kdb_reenter. That's true! If the problem is only in UKBD driver, I don't think this is a big problem to solve. The reason for the auto-magic locking, is that I've sometimes seen callers in non-polling mode call into UKBD without Giant locked. And this causes a panic when starting USB transfers, because the code expects Giant to be locked. Requirement: I would say that to use UKBD polling mode, the scheduler must be stopped. Else you should not use polling mode at all. I think that mountroot is getting characters the wrong way in this regard. --HPS From owner-freebsd-hackers@FreeBSD.ORG Sat Dec 17 17:14:45 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9EAA41065670; Sat, 17 Dec 2011 17:14:45 +0000 (UTC) (envelope-from hselasky@c2i.net) Received: from swip.net (mailfe06.c2i.net [212.247.154.162]) by mx1.freebsd.org (Postfix) with ESMTP id 5D7868FC15; Sat, 17 Dec 2011 17:14:41 +0000 (UTC) X-T2-Spam-Status: No, hits=-0.2 required=5.0 tests=ALL_TRUSTED, BAYES_50 Received: from [188.126.198.129] (account mc467741@c2i.net HELO laptop002.hselasky.homeunix.org) by mailfe06.swip.net (CommuniGate Pro SMTP 5.4.2) with ESMTPA id 217175245; Sat, 17 Dec 2011 18:14:38 +0100 From: Hans Petter Selasky To: Andriy Gapon Date: Sat, 17 Dec 2011 18:12:09 +0100 User-Agent: KMail/1.13.5 (FreeBSD/8.2-STABLE; KDE/4.4.5; amd64; ; ) References: <4EE51CB5.1060505@FreeBSD.org> <201112152356.35336.hselasky@c2i.net> <4EECADDA.4070704@FreeBSD.org> In-Reply-To: <4EECADDA.4070704@FreeBSD.org> X-Face: *nPdTl_}RuAI6^PVpA02T?$%Xa^>@hE0uyUIoiha$pC:9TVgl.Oq, NwSZ4V"|LR.+tj}g5 %V,x^qOs~mnU3]Gn; cQLv&.N>TrxmSFf+p6(30a/{)KUU!s}w\IhQBj}[g}bj0I3^glmC( :AuzV9:.hESm-x4h240C`9=w MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201112171812.09652.hselasky@c2i.net> Cc: usb@freebsd.org, freebsd-hackers@freebsd.org, mdf@freebsd.org Subject: Re: kern_yield vs ukbd_yield X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Dec 2011 17:14:45 -0000 On Saturday 17 December 2011 15:57:30 Andriy Gapon wrote: > Replying further... > > > Not directly, but indirect. You know, if you pause thread 1 (which I > > thought was thread 0), then other thread will get a chance to run. > > pause() could be a sufficient action to let other thread run, but it is not > a required action. As we've already discusses earlier all the USB threads > have sufficiently high priorities to get runnable while other kernel > threads are runnable. Only certain interrupt threads could have prevented > them from running, but that's definitely not the case. > > > It might also be that Giant is locked by syscons, and that unless you > > pause or yield, then Giant will block other parts of USB, like the > > callback thread, which is Giant locked for ukbd only to run. > > Hi, > > Maybe that is the explanation? > > Not sure if you are hypothesizing or if this is something that you > experienced during development and testing. I have only observed that no key-presses are returned by the UKBD polling mechanism, if the ukbd_yield() is removed. I have not investigated this further. If you use pause() that will slowdown dumphandling which is also polling for key-presses, so this must be added conditionally. > > Let's consider a few observations. > > First, syscons doesn't acquire Giant anywhere in the path started from > cngetc. Actually, I believe that the only place where Giant is now > explicitly acquired in the whole syscons code can be safely replaced with > an assert that the Giant is already held. > > Second, the polling mode is designed to be usable in situations where other > threads are not running. So in general it should not depend on other > threads being able to make any progress. Sure, but it is not that simple to poll a key-press from USB like with AT keyboards. You need to go through a bunch of stuff, including busdma before you get the keypress. --HPS From owner-freebsd-hackers@FreeBSD.ORG Sat Dec 17 23:07:54 2011 Return-Path: Delivered-To: hackers@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6BECB106566C for ; Sat, 17 Dec 2011 23:07:54 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id BC7DA8FC12 for ; Sat, 17 Dec 2011 23:07:53 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id BAA10547 for ; Sun, 18 Dec 2011 01:07:52 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1Rc3M7-0009yk-VP for hackers@freebsd.org; Sun, 18 Dec 2011 01:07:51 +0200 Message-ID: <4EED20C7.1090704@FreeBSD.org> Date: Sun, 18 Dec 2011 01:07:51 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111206 Thunderbird/8.0 MIME-Version: 1.0 To: hackers@FreeBSD.org X-Enigmail-Version: undefined Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit Cc: Subject: RB_NOSYNC -> no device_shutdown ? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Dec 2011 23:07:54 -0000 I look at the following code: static void module_init(void *arg) { sx_init(&modules_sx, "module subsystem sx lock"); TAILQ_INIT(&modules); EVENTHANDLER_REGISTER(shutdown_final, module_shutdown, NULL, SHUTDOWN_PRI_DEFAULT); } SYSINIT(module, SI_SUB_KLD, SI_ORDER_FIRST, module_init, 0); static void module_shutdown(void *arg1, int arg2) { module_t mod; if (arg2 & RB_NOSYNC) return; mtx_lock(&Giant); MOD_SLOCK; TAILQ_FOREACH_REVERSE(mod, &modules, modulelist, link) MOD_EVENT(mod, MOD_SHUTDOWN); MOD_SUNLOCK; mtx_unlock(&Giant); } and wonder why RB_NOSYNC is overloaded to mean that no MOD_SHUTDOWN/device_shutdown cleanup should be done? -- Andriy Gapon