From owner-freebsd-arch@FreeBSD.ORG Sun Dec 11 21:45:17 2011 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 85181106566B; Sun, 11 Dec 2011 21:45:17 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id A3A208FC15; Sun, 11 Dec 2011 21:45:16 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id XAA08031; Sun, 11 Dec 2011 23:45:14 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1RZrCs-000FGM-B8; Sun, 11 Dec 2011 23:45:14 +0200 Message-ID: <4EE52469.3050201@FreeBSD.org> Date: Sun, 11 Dec 2011 23:45:13 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111206 Thunderbird/8.0 MIME-Version: 1.0 To: arch@FreeBSD.org, FreeBSD current X-Enigmail-Version: undefined Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit Cc: Subject: grabbing console (syscons) in kernel X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2011 21:45:17 -0000 There are a few cases when the kernel needs to interact with a user via syscons. This is the cases where the kernel not only spews some output but also expects some input. Some examples are: - asking for a root filesystem specification - entering ddb - asking to press a key for reboot In this cases the kernel implicitly grabs the console for its own use. I'd like to make action more explicit. What do you think about the approach and implementation in the following patches? Thank you! https://gitorious.org/~avg/freebsd/avgbsd/commit/5248b49ebf84d98a0597fa5aa4d813a38f581acc https://gitorious.org/~avg/freebsd/avgbsd/commit/a0849c52242378474bb2eaa41726376fbc4c5bf6 https://gitorious.org/~avg/freebsd/avgbsd/commit/a67515cbd720b16f03ba435ed182966a8a338b15 https://gitorious.org/~avg/freebsd/avgbsd/commit/b8864b68b4c0e26ece065a38301c305833be32eb https://gitorious.org/~avg/freebsd/avgbsd/commit/1017ae425d8abecd7482bd6c6deaaf9f25f5c6cd P.S. one of the benefits is that a keyboard is put into and out of the polling mode before getting all the required input and after that; not around each character as it is done now in rather twisted way. -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Sun Dec 11 22:35:10 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 319431065676; Sun, 11 Dec 2011 22:35:10 +0000 (UTC) (envelope-from yerenkow@gmail.com) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id E38338FC20; Sun, 11 Dec 2011 22:35:08 +0000 (UTC) Received: by iakl21 with SMTP id l21so2784995iak.13 for ; Sun, 11 Dec 2011 14:35:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=H1BTTpAmPL4oyToD3XMZRRQfD2DN9q62C8+r5096wq8=; b=Pxs8YByTbyk6sxnALIuZ7k2mlvb65aBjW8nSwmq1gswveZajYjQGOG3pjrof6u5yw8 TccOgG4oxLQpXCTco3Ng/dJm5ggDRA8M6iN7900FKZdMB27BzTQbgUPfskZDeaqFYR1r jXCp/3Wh/sgLgXmf2sOrgXEusozvz5a4lqLgo= MIME-Version: 1.0 Received: by 10.182.45.102 with SMTP id l6mr2360101obm.0.1323641200271; Sun, 11 Dec 2011 14:06:40 -0800 (PST) Received: by 10.182.36.133 with HTTP; Sun, 11 Dec 2011 14:06:40 -0800 (PST) In-Reply-To: <4EE52469.3050201@FreeBSD.org> References: <4EE52469.3050201@FreeBSD.org> Date: Mon, 12 Dec 2011 00:06:40 +0200 Message-ID: From: Alexander Yerenkow To: Andriy Gapon Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: arch@freebsd.org, FreeBSD current Subject: Re: grabbing console (syscons) in kernel X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2011 22:35:10 -0000 2011/12/11 Andriy Gapon > > There are a few cases when the kernel needs to interact with a user via > syscons. > This is the cases where the kernel not only spews some output but also > expects > some input. Some examples are: > - asking for a root filesystem specification > - entering ddb > - asking to press a key for reboot > > In this cases the kernel implicitly grabs the console for its own use. > I'd like to make action more explicit. > > What do you think about the approach and implementation in the following > patches? > Thank you! > > > https://gitorious.org/~avg/freebsd/avgbsd/commit/5248b49ebf84d98a0597fa5aa4d813a38f581acc > > https://gitorious.org/~avg/freebsd/avgbsd/commit/a0849c52242378474bb2eaa41726376fbc4c5bf6 > > https://gitorious.org/~avg/freebsd/avgbsd/commit/a67515cbd720b16f03ba435ed182966a8a338b15 > > https://gitorious.org/~avg/freebsd/avgbsd/commit/b8864b68b4c0e26ece065a38301c305833be32eb > > https://gitorious.org/~avg/freebsd/avgbsd/commit/1017ae425d8abecd7482bd6c6deaaf9f25f5c6cd > > P.S. one of the benefits is that a keyboard is put into and out of the > polling > mode before getting all the required input and after that; not around each > character as it is done now in rather twisted way. > Does asking for geli password is such case too? For example, I have mouse, which can disappear/reappear while laying still; and when geli asks password, ums0 connect message can appear, which moves line "Please enter geli password..." to top, and it can be missed; This behavior somehow change? > > -- > Andriy Gapon > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > -- Regards, Alexander Yerenkow From owner-freebsd-arch@FreeBSD.ORG Mon Dec 12 07:52:57 2011 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C616A1065673; Mon, 12 Dec 2011 07:52:57 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id E08508FC0A; Mon, 12 Dec 2011 07:52:56 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id JAA14892; Mon, 12 Dec 2011 09:52:53 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1Ra0gv-000HyJ-K2; Mon, 12 Dec 2011 09:52:53 +0200 Message-ID: <4EE5B2D4.9050705@FreeBSD.org> Date: Mon, 12 Dec 2011 09:52:52 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111206 Thunderbird/8.0 MIME-Version: 1.0 To: Alexander Yerenkow References: <4EE52469.3050201@FreeBSD.org> In-Reply-To: X-Enigmail-Version: undefined Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: arch@FreeBSD.org, FreeBSD current Subject: Re: grabbing console (syscons) in kernel X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2011 07:52:57 -0000 on 12/12/2011 00:06 Alexander Yerenkow said the following: > > > 2011/12/11 Andriy Gapon > > > > There are a few cases when the kernel needs to interact with a user via syscons. > This is the cases where the kernel not only spews some output but also expects > some input. Some examples are: > - asking for a root filesystem specification > - entering ddb > - asking to press a key for reboot > > In this cases the kernel implicitly grabs the console for its own use. > I'd like to make action more explicit. > > What do you think about the approach and implementation in the following > patches? > Thank you! > > https://gitorious.org/~avg/freebsd/avgbsd/commit/5248b49ebf84d98a0597fa5aa4d813a38f581acc > https://gitorious.org/~avg/freebsd/avgbsd/commit/a0849c52242378474bb2eaa41726376fbc4c5bf6 > https://gitorious.org/~avg/freebsd/avgbsd/commit/a67515cbd720b16f03ba435ed182966a8a338b15 > https://gitorious.org/~avg/freebsd/avgbsd/commit/b8864b68b4c0e26ece065a38301c305833be32eb > https://gitorious.org/~avg/freebsd/avgbsd/commit/1017ae425d8abecd7482bd6c6deaaf9f25f5c6cd > > P.S. one of the benefits is that a keyboard is put into and out of the polling > mode before getting all the required input and after that; not around each > character as it is done now in rather twisted way. > > > Does asking for geli password is such case too? Yes. > For example, I have mouse, which can disappear/reappear while laying still; and > when geli asks password, ums0 connect message can appear, which moves line > "Please enter geli password..." to top, and it can be missed; > This behavior somehow change? No. The patches do not contain user-visible changes. They make things easier for keyboard drivers and hopefully provide a more logical interface to kernel. Their primary target is the input path, they do not introduce any changes to the output path. -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Tue Dec 13 17:49:11 2011 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 950471065672; Tue, 13 Dec 2011 17:49:11 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id B490D8FC17; Tue, 13 Dec 2011 17:49:10 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA18562; Tue, 13 Dec 2011 19:49:08 +0200 (EET) (envelope-from avg@FreeBSD.org) Message-ID: <4EE79014.3070006@FreeBSD.org> Date: Tue, 13 Dec 2011 19:49:08 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111109 Thunderbird/8.0 MIME-Version: 1.0 To: arch@FreeBSD.org, FreeBSD current References: <4EE52469.3050201@FreeBSD.org> In-Reply-To: <4EE52469.3050201@FreeBSD.org> X-Enigmail-Version: undefined Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: Subject: Re: grabbing console (syscons) in kernel X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 17:49:11 -0000 on 11/12/2011 23:45 Andriy Gapon said the following: > > There are a few cases when the kernel needs to interact with a user via syscons. > This is the cases where the kernel not only spews some output but also expects > some input. Some examples are: > - asking for a root filesystem specification > - entering ddb > - asking to press a key for reboot > > In this cases the kernel implicitly grabs the console for its own use. > I'd like to make action more explicit. > > What do you think about the approach and implementation in the following patches? > Thank you! > > https://gitorious.org/~avg/freebsd/avgbsd/commit/5248b49ebf84d98a0597fa5aa4d813a38f581acc > https://gitorious.org/~avg/freebsd/avgbsd/commit/a0849c52242378474bb2eaa41726376fbc4c5bf6 > https://gitorious.org/~avg/freebsd/avgbsd/commit/a67515cbd720b16f03ba435ed182966a8a338b15 > https://gitorious.org/~avg/freebsd/avgbsd/commit/b8864b68b4c0e26ece065a38301c305833be32eb > https://gitorious.org/~avg/freebsd/avgbsd/commit/1017ae425d8abecd7482bd6c6deaaf9f25f5c6cd I was advised that the above links might not be the best way to present the patches for review, so here are them as a single diff file: http://people.freebsd.org/~avg/cngrab.diff > P.S. one of the benefits is that a keyboard is put into and out of the polling > mode before getting all the required input and after that; not around each > character as it is done now in rather twisted way. > -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Tue Dec 13 22:00:22 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 388791065673 for ; Tue, 13 Dec 2011 22:00:22 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (relay02.stack.nl [IPv6:2001:610:1108:5010::104]) by mx1.freebsd.org (Postfix) with ESMTP id BB4AA8FC08 for ; Tue, 13 Dec 2011 22:00:21 +0000 (UTC) Received: from snail.stack.nl (snail.stack.nl [IPv6:2001:610:1108:5010::131]) by mx1.stack.nl (Postfix) with ESMTP id 1FF3C359331 for ; Tue, 13 Dec 2011 23:00:21 +0100 (CET) Received: by snail.stack.nl (Postfix, from userid 1677) id 06ED128468; Tue, 13 Dec 2011 23:00:21 +0100 (CET) Date: Tue, 13 Dec 2011 23:00:20 +0100 From: Jilles Tjoelker To: freebsd-arch@freebsd.org Message-ID: <20111213220020.GA42285@stack.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Subject: Some sh(1) plans for 10.0 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 22:00:22 -0000 Here are some changes that may happen to sh(1) in 10.0. * Report output errors in builtins (error message and exit status 2). For example: echo >&-. PR bin/158206. * Special expansion for assignments in export/readonly/local. This expands assignments in a command like export a=~ b=$1 as if the "export" were not there (tilde expansion after = and :, no pathname generation, no word splitting). This feature has been available in ksh/bash/zsh for a long time and has been proposed and defined for a new version of POSIX at http://austingroupbugs.net/view.php?id=351 Users sometimes get confused by this not working and if POSIX plans to add it, why not add it here? * 'time' keyword, allowing timing of pipelines and compound commands. This works such that if any options are specified, it will fall back to /usr/bin/time. The proposed changes to POSIX are fairly extensive including changes to the ! keyword which will break some of our scripts and a TIMEFORMAT variable. A somewhat stripped version may be useful. Note that a shell-based version cannot and will not support SIGINFO. This is also the case with the versions in tcsh, bash and zsh. http://austingroupbugs.net/view.php?id=267 * Conditional command [[ ... ]] much like ksh. This is being discussed in the Austin Group at this time. I don't really like the duplication with [ ... ] and case, considering that they work well in the POSIX spec and our implementation. Some reasons to add it anyway are that it is slightly easier to use and cleaner in syntax, that < and > can use locale-specific ordering (strcoll) (but when are strings compared for greater-than in a shell script?) and that a regex match could be added. A slightly older proposal is at https://docs.google.com/document/d/1Gd9r0f0rmmUIZlBnO4NyhTz2q-_1PHHGVCHAglFbJoY/edit * vfork support, with an environment variable to opt out. * Executing commands in subshell environments without forking in more situations (saving and restoring state manually instead). * Here document expansion without a fork. A simple implementation would make expansion side effects persistent and errors fatal while a more complicated implementation could avoid that. A fork is still needed if the here document cannot be written into a pipe without blocking. (This could be avoided with ugly SIGIO magic but the case seems too rare to bother.) * If execve() fails even though the file exists and is executable, and it should not be run using /bin/sh ([ENOEXEC] and text file), give a clearer error message. Example: #! interpreter does not exist. Some rework could make all execve() errors detected in the child process of this kind. -- Jilles Tjoelker From owner-freebsd-arch@FreeBSD.ORG Wed Dec 14 14:48:50 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 88822106566B; Wed, 14 Dec 2011 14:48:50 +0000 (UTC) (envelope-from jrid@cubinlab.ee.unimelb.edu.au) Received: from mail-gw1.its.unimelb.edu.au (mail-gw1.its.unimelb.edu.au [128.250.5.150]) by mx1.freebsd.org (Postfix) with ESMTP id 30CF48FC13; Wed, 14 Dec 2011 14:48:50 +0000 (UTC) Received: from emu.cubinlab.ee.unimelb.edu.au (cubinlab.ee.unimelb.edu.au [128.250.80.33]) by mail-gw1.its.unimelb.edu.au (Postfix) with ESMTPS id BE9981FFE; Thu, 15 Dec 2011 01:30:16 +1100 (EST) Received: from jrid.cubinlab.ee.unimelb.edu.au (jrid.cubinlab.ee.unimelb.edu.au [10.0.1.128]) (authenticated bits=0) by emu.cubinlab.ee.unimelb.edu.au (8.14.4/8.14.4) with ESMTP id pBEEUE3W044929 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES128-SHA bits=128 verify=NO); Thu, 15 Dec 2011 01:30:15 +1100 (EST) (envelope-from jrid@cubinlab.ee.unimelb.edu.au) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Julien Ridoux In-Reply-To: <4EDB6814.9040403@freebsd.org> Date: Thu, 15 Dec 2011 01:30:14 +1100 Content-Transfer-Encoding: quoted-printable Message-Id: <405E9C58-F999-45C2-BC20-81ED4CA8A3E9@cubinlab.ee.unimelb.edu.au> References: <201111210417.pAL4HOdi023556@svn.freebsd.org> <4ED8575D.2010405@freebsd.org> <201112021927.25234.jkim@FreeBSD.org> <201112022002.49618.jkim@FreeBSD.org> <4EDB6814.9040403@freebsd.org> To: Jung-uk Kim X-Mailer: Apple Mail (2.1084) Cc: svn-src-head@freebsd.org, Lawrence Stewart , Benjamin Kaduk , freebsd-arch@freebsd.org Subject: Re: svn commit: r227778 - head/sys/net X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2011 14:48:50 -0000 On 04/12/2011, at 11:31 PM, Lawrence Stewart wrote: > On 12/03/11 12:02, Jung-uk Kim wrote: >> On Friday 02 December 2011 07:27 pm, Jung-uk Kim wrote: >>> On Thursday 01 December 2011 11:43 pm, Lawrence Stewart wrote: >>>> On 12/02/11 03:43, Jung-uk Kim wrote: >>>>> On Thursday 01 December 2011 10:11 am, Lawrence Stewart wrote: >>>>>> On 11/30/11 05:09, Jung-uk Kim wrote: >>>>>>> On Tuesday 29 November 2011 11:13 am, Lawrence Stewart wrote: > [snip] >>>>>>>> Here's the first of the patches: >>>>>>>>=20 >>>>>>>> http://people.freebsd.org/~lstewart/patches/misc/ffclock_bpf >>>>>>>> _i nt act abi_10.x.r228130.patch >>>>>>>=20 >>>>>>> I only glanced at it but it looks very close to what I wanted >>>>>>> to suggest. >>>>>>=20 >>>>>> Final candidate patch is at: >>>>>>=20 >>>>>> http://people.freebsd.org/~lstewart/patches/misc/ffclock_bpf_i >>>>>> nt act abi_10.x.r228180.patch >>>>>>=20 >>>>>> Assuming it passes the "make tinderbox" build I'm currently >>>>>> running and no further input is received from interested >>>>>> parties, I plan to commit it in ~10 hours. >>>>>>=20 >>>>>> Changes since the r228130 patch I sent previously: >>>>>>=20 >>>>>> - The new flags in bpf.h are added unconditionally so that >>>>>> they can always be referenced at compile time and a decision >>>>>> made at runtime as to whether a flag will be set or not. Using >>>>>> one of the new flags when the kernel doesn't have FFCLOCK >>>>>> compiled in results in the flag being ignored. An app should >>>>>> check for the existence of the "ffclock" kernel feature or the >>>>>> "kern.sysclock" sysctl tree before attempting to use the >>>>>> flags. >>>>>>=20 >>>>>> - This patch will hopefully be MFCed at some point, so I added >>>>>> a CTASSERT to bpf.c to ensure that the ABI of structs >>>>>> bpf_hdr32, bpf_hdr and bpf_xhdr remains intact when FFCLOCK is >>>>>> enabled and the union of a ffcounter and struct >>>>>> timeval32/timeval/bpf_ts is switched in. >>>>>>=20 >>>>>> - bpf_gettime() more comprehensively covers all the possible >>>>>> cases of flag combination and does sensible things for each >>>>>> case (even though some cases are rather silly). >>>>>>=20 >>>>>> - The snippet of code at the beginning of catchpacket() that >>>>>> was manipulating the struct bintime derived from bpf_gettime() >>>>>> was gross and has been removed in favour of selecting the >>>>>> right {get}bin{up}time() function call in bpf_gettime(). >>>>>=20 >>>>> I did that to reduce branching. Since you are introducing more >>>>> branches, it warrants a function pointer now. >>>=20 >>> There's another reason, BTW. If mbufs are tagged with timestamps >>> (where only monotonic timestamps are allowed), they must be >>> converted within the bpf.c. I forgot all the details. :-( >=20 > We should document this knowledge in some code comments. >=20 >>>> I see, thanks for the explanation. Could you elaborate a bit more >>>> about how you would implement the function pointer idea? I'm also >>>> curious in the !FFCLOCK case just how much overhead having the >>>> 2-layer nested if/else adds. I'm not an very optimisation savvy >>>> person, but I'm wondering if it's actually worth micro-optimising >>>> this code. >>>>=20 >>>> My initial thoughts about your function pointer idea lead to >>>> adding a function pointer in the bpf_d and setting it to the >>>> appropriate function to get the timestamp from at bpf_d creation >>>> or ioctl time. Whilst I like this idea, I can't see how it would >>>> work given that the various functions involved in time/ffcounter >>>> stamp generation all have different signatures. >>>>=20 >>>> We could have multiple variants of bpf_gettime() which each call >>>> the appropriate underlying functions to generate the appropriate >>>> stamp. Would add quite a lot of code but would reduce the >>>> overhead of calling bpf_gettime() to an indirect function call + >>>> the underlying stamp generation function call. This also solves >>>> the problem of multiple function signatures. >>>=20 >>> Please see my patch: >>>=20 >>> http://people.freebsd.org/~jkim/bpf_ffclock.diff >>=20 >> I booted up the kernel and found it just crashes because of stupid >> typos. Now a new patch is uploaded in place. Sorry. >>=20 >> Anyway, I see no regression nor ABI breakage. :-) >=20 > struct bpf_d being part of the ABI was the main thing I was concerned = about, so given that it isn't I like your approach a lot. >=20 > As noted by Julien, this approach does introduce problems with respect = to the follow up patch that adds a permanent bh_ffcounter member to the = bpf header. I thought the fromclock() API would sufficiently meet the = needs of consumers like BPF, but if we were to proceed with something = like Jung-uk's proposed patch I don't think that's true anymore. >=20 > We decided to bite the bullet and devise an API that is more compact = and can return all appropriate time information from all underlying = sysclocks in an efficient manner - something like a more generic version = of ffclock_abstime(). Julien and I spent quite some time today nutting = out details, and Julien has done a proof of concept patch against my = proposed BPF patch which looks good as a starting point for discussion: >=20 > = http://www.cubinlab.ee.unimelb.edu.au/~jrid/patches/r228254/sysclock_snap.= patch >=20 > It needs a bit more work and should be split into two patches (one = introducing the API and the other being the BPF changes). >=20 > With something like this though, the BPF patch becomes radically = simplified in the FFCLOCK case. We don't even need a function pointer = cached in the bpf_d anymore, but could cache a struct sysclock_snap = there instead if we really wanted to minimise overhead in bpf_gettime(). >=20 > We'll have a go at refining the patch tomorrow hopefully, but wanted = to put this out there for early consideration. >=20 > Cheers, > Lawrence Hi Jung-uk (and all), It took us a bit of time but we have a new patch. There are a lot of = changes in the patch, and I will start by summarising our motivation for = the changes. Hopefully I won't forget anything, but if I leave something = unclear, don't hesitate to ask Lawrence or me. Starting with the easy bits, we drop the union in the BPF header and = followed something similar to what Ben and you proposed. Next, the core motivation was to ensure that the timestamping function = be called once only per packet captured, while keeping the ability to = read both clocks and return the time from one of the two to the user. = This forced us to redesign the timestamping function and one thing = leading to another a fair bit of the BPF code. There 3 orthogonal parameters to the timestamping function in the BPF = context: - which clock is used to return the time - what is the format of the value returned - and a performance issue (nothing to do with timestamping or = timekeeping): reading the hardware counter or not We handle the first one with a new sysclock_getsnapshot() which saves = both clock parameters when timestamping and potentially reads the = hardware counter. The second one has been covered in previous discussion with the addition = of BPF_T_FFCOUNTER. The performance issue created problems. In the current code, the order = with which the BPF devices are open on one interface impacts the = timestamping function. Let's consider two BPF devices respectively open = with BPF_TSTAMP_NORMAL and BPF_TSTAMP_FAST quality. If NORMAL is open = after FAST, it is at the head of the list, the hardware counter is read, = and FAST benefits from higher quality timestamps. If FAST is open after = NORMAL, the timestamping function is called once and return last kernel = tick time, then it is called a second time for NORMAL, this time = accessing the hardware counter. None of these cases are satisfactory. This lead to inconsistent = behaviour based on the order the BPF are open. An application can impact = the settings of other BPF devices and other applications / kernel = consumers. The timestamping suffers from higher latency and worse, = higher variability of the latency, which is bad. We then took a different approach, which is inspired by your work on = BPF. We believe the FAST/NORMAL settings should have 2 level of = granularity: system wide and per interface. The system wide is the = default behaviour (for example all BPF devices are NORMAL) and the = interface one super-seeds the system one. This is implemented by the use = of new sysctl living in the net.bpf tree. This enables many scenarios such as: - system default is FAST to improve performance because the system uses = a slow timecounter (eg HPET or ACPI). - the system has 3 interfaces. Interface 2 is put in NORMAL mode (need = for accuracy but not much traffic), and the last can do hardware = timestamping and is put in BPF_TSTAMP_EXTERNAL mode. This then naturally led to store the performance setting = (none/fast/normal/external) for each interface instead of each BPF = descriptor. Each BPF descriptor still maintains the other 2 settings: the choice of = clock and the format of the timestamp independently from each other. We believe this new patch keeps the spirit of the changes you proposed = and improves it while accomodating the changes required by the = feed-forward clock and future evolutions with things like IEEE 1588 = compatible NICs. We plan to push this patch as soon as possible. We would really = appreciate if you could comment on this long email and pach very soon. The patch can be found here: = http://www.cubinlab.ee.unimelb.edu.au/~jrid/patches/r228429-v4/ffclock_bpf= _intactabi.patch We will adjust the documentation as well once we have reached consensus. Julien (and Lawrence of course) From owner-freebsd-arch@FreeBSD.ORG Wed Dec 14 14:59:09 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 56630106564A for ; Wed, 14 Dec 2011 14:59:09 +0000 (UTC) (envelope-from uqs@spoerlein.net) Received: from acme.spoerlein.net (acme.spoerlein.net [IPv6:2a01:4f8:131:23c2::1]) by mx1.freebsd.org (Postfix) with ESMTP id D5DBC8FC16 for ; Wed, 14 Dec 2011 14:59:08 +0000 (UTC) Received: from localhost (acme.spoerlein.net [IPv6:2a01:4f8:131:23c2::1]) by acme.spoerlein.net (8.14.4/8.14.4) with ESMTP id pBEEx7xA019669 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Wed, 14 Dec 2011 15:59:07 +0100 (CET) (envelope-from uqs@spoerlein.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=spoerlein.net; s=dkim200908; t=1323874747; bh=xdU2/p3QZmAjVZwr2cAvYki4glch9glZXtyzKkglOGU=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:In-Reply-To; b=gmiT8Uo+fB5rNVkU+9k2U30QMph/2eOj48ztDAXG2sPZhpvjyZs+BakptUE+xTT3f xyDHX4jMSFB4sOXb2dQYDDc6Xa/BRnzzKQ5Qv/sV+RgX6Qobu0dqdV45zHC6YUs3uj dHXpD58tLVF4/9TQWAPkOIpe/VitJAKizvNoQm9w= Date: Wed, 14 Dec 2011 15:59:07 +0100 From: Ulrich =?utf-8?B?U3DDtnJsZWlu?= To: Jilles Tjoelker Message-ID: <20111214145907.GN83814@acme.spoerlein.net> Mail-Followup-To: Jilles Tjoelker , freebsd-arch@freebsd.org References: <20111213220020.GA42285@stack.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20111213220020.GA42285@stack.nl> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-arch@freebsd.org Subject: Re: Some sh(1) plans for 10.0 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2011 14:59:09 -0000 On Tue, 2011-12-13 at 23:00:20 +0100, Jilles Tjoelker wrote: > Here are some changes that may happen to sh(1) in 10.0. > > * Report output errors in builtins (error message and exit status 2). > For example: echo >&-. PR bin/158206. > > * Special expansion for assignments in export/readonly/local. > This expands assignments in a command like export a=~ b=$1 as if the > "export" were not there (tilde expansion after = and :, no pathname > generation, no word splitting). > This feature has been available in ksh/bash/zsh for a long time and > has been proposed and defined for a new version of POSIX at > http://austingroupbugs.net/view.php?id=351 > Users sometimes get confused by this not working and if POSIX plans to > add it, why not add it here? Woa, looks like thats a fucked up situation overall ... > * 'time' keyword, allowing timing of pipelines and compound commands. > This works such that if any options are specified, it will fall back > to /usr/bin/time. The proposed changes to POSIX are fairly extensive > including changes to the ! keyword which will break some of our > scripts and a TIMEFORMAT variable. A somewhat stripped version may be > useful. Note that a shell-based version cannot and will not support > SIGINFO. This is also the case with the versions in tcsh, bash and > zsh. > http://austingroupbugs.net/view.php?id=267 > > * Conditional command [[ ... ]] much like ksh. This is being discussed > in the Austin Group at this time. I don't really like the duplication > with [ ... ] and case, considering that they work well in the POSIX > spec and our implementation. Some reasons to add it anyway are that it > is slightly easier to use and cleaner in syntax, that < and > can use > locale-specific ordering (strcoll) (but when are strings compared for > greater-than in a shell script?) and that a regex match could be > added. > A slightly older proposal is at > https://docs.google.com/document/d/1Gd9r0f0rmmUIZlBnO4NyhTz2q-_1PHHGVCHAglFbJoY/edit > > * vfork support, with an environment variable to opt out. > > * Executing commands in subshell environments without forking in more > situations (saving and restoring state manually instead). Is this really worth the effort and the added complexity? I hope there will be benchmarks showing that this is actually saving us non-negligible time. > * Here document expansion without a fork. A simple implementation would > make expansion side effects persistent and errors fatal while a more > complicated implementation could avoid that. A fork is still needed if > the here document cannot be written into a pipe without blocking. > (This could be avoided with ugly SIGIO magic but the case seems too > rare to bother.) > > * If execve() fails even though the file exists and is executable, and > it should not be run using /bin/sh ([ENOEXEC] and text file), give a > clearer error message. Example: #! interpreter does not exist. > Some rework could make all execve() errors detected in the child > process of this kind. Good stuff overall, keep up the good work! Uli From owner-freebsd-arch@FreeBSD.ORG Wed Dec 14 16:41:46 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 69B251065678 for ; Wed, 14 Dec 2011 16:41:46 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 295278FC1C for ; Wed, 14 Dec 2011 16:41:46 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id 0665046B52 for ; Wed, 14 Dec 2011 11:41:44 -0500 (EST) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 6B7ABB96C for ; Wed, 14 Dec 2011 11:41:43 -0500 (EST) From: John Baldwin To: arch@freebsd.org Date: Wed, 14 Dec 2011 11:41:41 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p8; KDE/4.5.5; amd64; ; ) MIME-Version: 1.0 Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-Id: <201112141141.41168.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 14 Dec 2011 11:41:43 -0500 (EST) Cc: Subject: Changing lseek() to KNOTE on the vnode when seeking on a file X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2011 16:41:46 -0000 A co-worker ran into an issue with using an EVFILT_READ kevent on a regular file recently. Specifically, in the manpage it says: EVFILT_READ Takes a descriptor as the identifier, and returns whenever there is data available to read. The behavior of the fil- ter is slightly different depending on the descriptor type. ... Vnodes Returns when the file pointer is not at the end of file. data contains the offset from current position to end of file, and may be negative. He was then working on a program that read to EOF, then seeked back into the file. He was expecting to get a new kevent after seeking back into the file since for his file descriptor after the lseek "there is data available to read" and "the file pointer is not at the end of file". I have a patch to fix this by doing a KNOTE() on a vnode after a successful seek. I checked OS X and it looks like they added this to their lseek() in Snow Leopard (http://fxr.watson.org/fxr/source/bsd/vfs/vfs_syscalls.c?v=xnu-1699.24.8#L4182). The one patch to fix this is below along with a test. Note that unlike OS X I did not add a new NOTE_NONE for this case. OS X has logic in their VFS filter operations that make special assumptions about a hint value of 0, so they had to add NOTE_NONE as a hack. We do not have the same special assumptions about a hint of 0, so we can just use "0". Without this fix the test below complains about missing events for the "after seek" and "after third read" cases. Index: vfs_syscalls.c =================================================================== --- vfs_syscalls.c (revision 228311) +++ vfs_syscalls.c (working copy) @@ -2049,6 +2049,7 @@ sys_lseek(td, uap) if (error != 0) goto drop; fp->f_offset = offset; + VFS_KNOTE_UNLOCKED(vp, 0); *(off_t *)(td->td_retval) = fp->f_offset; drop: fdrop(fp, td); /*- * Test to see if lseek(2) provokes an updated kevent on a regular * file descriptor. */ #include #include #include #include #include #include #include #include #include char template[] = "/tmp/kevent_lseek.XXXXXX"; static int fd, kq; static void check_event(bool expected, off_t offset, const char *desc) { struct timespec ts = { 0, 0 }; struct kevent ev; int retval; retval = kevent(kq, NULL, 0, &ev, 1, &ts); if (retval < 0) err(1, "kevent"); if (!expected) { if (retval != 0) printf("Unexpected kevent: %s\n", desc); } else { if (retval == 0) printf("Missing kevent: %s\n", desc); else if (ev.data != offset) printf("Wrong offset (%jd vs %jd): %s\n", (intmax_t)ev.data, (intmax_t)offset, desc); } } static void readn(size_t count) { char buf[count]; ssize_t nread; nread = read(fd, buf, count); if (nread < 0) err(1, "read"); if (nread != count) errx(1, "short read: %zd vs %zu\n", nread, count); } int main(int ac, char **av) { struct kevent ev; kq = kqueue(); if (kq < 0) err(1, "kqueue"); fd = mkstemp(template); if (fd < 0) err(1, "mkstemp"); if (unlink(template) < 0) err(1, "unlink"); EV_SET(&ev, fd, EVFILT_READ, EV_ADD, 0, 0, 0); if (kevent(kq, &ev, 1, NULL, 0, NULL) < 0) err(1, "kevent(EV_ADD)"); check_event(false, 0, "initial check"); if (ftruncate(fd, 2048) < 0) err(1, "ftruncate(grow)"); check_event(true, 2048, "after grow"); readn(512); check_event(true, 2048 - 512, "after read"); readn(2048 - 512); check_event(false, 0, "after read to EOF"); if (lseek(fd, 1024, SEEK_SET) < 0) err(1, "lseek"); check_event(true, 2048 - 1024, "after seek"); readn(512); check_event(true, 2048 - 1024 - 512, "after third read"); readn(512); check_event(false, 0, "after fourth read to EOF"); close(fd); close(kq); return (0); } -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Wed Dec 14 16:58:39 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 40BF4106564A; Wed, 14 Dec 2011 16:58:39 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id D177B8FC14; Wed, 14 Dec 2011 16:58:38 +0000 (UTC) Received: from alf.home (alf.kiev.zoral.com.ua [10.1.1.177]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id pBEGwURr092286 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 14 Dec 2011 18:58:30 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from alf.home (kostik@localhost [127.0.0.1]) by alf.home (8.14.5/8.14.5) with ESMTP id pBEGwUsl044803; Wed, 14 Dec 2011 18:58:30 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by alf.home (8.14.5/8.14.5/Submit) id pBEGwU2h044802; Wed, 14 Dec 2011 18:58:30 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: alf.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 14 Dec 2011 18:58:30 +0200 From: Kostik Belousov To: John Baldwin Message-ID: <20111214165830.GG50300@deviant.kiev.zoral.com.ua> References: <201112141141.41168.jhb@freebsd.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="hXPCCV0+y/03K5AN" Content-Disposition: inline In-Reply-To: <201112141141.41168.jhb@freebsd.org> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: arch@freebsd.org Subject: Re: Changing lseek() to KNOTE on the vnode when seeking on a file X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2011 16:58:39 -0000 --hXPCCV0+y/03K5AN Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Dec 14, 2011 at 11:41:41AM -0500, John Baldwin wrote: > A co-worker ran into an issue with using an EVFILT_READ kevent on a regul= ar=20 > file recently. Specifically, in the manpage it says: >=20 > EVFILT_READ Takes a descriptor as the identifier, and returns whe= never > there is data available to read. The behavior of the= fil- > ter is slightly different depending on the descriptor > type. >=20 > ... >=20 > Vnodes > Returns when the file pointer is not at the end of > file. data contains the offset from current posi= tion > to end of file, and may be negative. >=20 > He was then working on a program that read to EOF, then seeked back into = the > file. He was expecting to get a new kevent after seeking back into the f= ile > since for his file descriptor after the lseek "there is data available to= =20 > read" and "the file pointer is not at the end of file". I have a patch t= o fix=20 > this by doing a KNOTE() on a vnode after a successful seek. I checked OS= X=20 > and it looks like they added this to their lseek() in Snow Leopard > (http://fxr.watson.org/fxr/source/bsd/vfs/vfs_syscalls.c?v=3Dxnu-1699.24.= 8#L4182). >=20 > The one patch to fix this is below along with a test. Note that unlike O= S X > I did not add a new NOTE_NONE for this case. OS X has logic in their VFS > filter operations that make special assumptions about a hint value of 0, = so > they had to add NOTE_NONE as a hack. We do not have the same special=20 > assumptions about a hint of 0, so we can just use "0". Without this fix = the > test below complains about missing events for the "after seek" and "after= =20 > third read" cases. Just curious - wouldn't it generate a spurious event if lseek is performed with zero offset, e.g. SEEK_CUR with offset 0 ? --hXPCCV0+y/03K5AN Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk7o1bYACgkQC3+MBN1Mb4hU0wCeMul5vUVedn8Ujcs7AXDNnY9f jXcAn16gdJFhDhRDXYdARMYi7bblzJQG =hP4y -----END PGP SIGNATURE----- --hXPCCV0+y/03K5AN-- From owner-freebsd-arch@FreeBSD.ORG Wed Dec 14 20:43:16 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 88CDD1065675 for ; Wed, 14 Dec 2011 20:43:16 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 5F6CC8FC19 for ; Wed, 14 Dec 2011 20:43:16 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id D8B8D46B43; Wed, 14 Dec 2011 15:43:15 -0500 (EST) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 5031AB91A; Wed, 14 Dec 2011 15:43:15 -0500 (EST) From: John Baldwin To: Kostik Belousov Date: Wed, 14 Dec 2011 15:43:13 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p8; KDE/4.5.5; amd64; ; ) References: <201112141141.41168.jhb@freebsd.org> <20111214165830.GG50300@deviant.kiev.zoral.com.ua> In-Reply-To: <20111214165830.GG50300@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201112141543.14130.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 14 Dec 2011 15:43:15 -0500 (EST) Cc: arch@freebsd.org Subject: Re: Changing lseek() to KNOTE on the vnode when seeking on a file X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2011 20:43:16 -0000 On Wednesday, December 14, 2011 11:58:30 am Kostik Belousov wrote: > On Wed, Dec 14, 2011 at 11:41:41AM -0500, John Baldwin wrote: > > A co-worker ran into an issue with using an EVFILT_READ kevent on a regular > > file recently. Specifically, in the manpage it says: > > > > EVFILT_READ Takes a descriptor as the identifier, and returns whenever > > there is data available to read. The behavior of the fil- > > ter is slightly different depending on the descriptor > > type. > > > > ... > > > > Vnodes > > Returns when the file pointer is not at the end of > > file. data contains the offset from current position > > to end of file, and may be negative. > > > > He was then working on a program that read to EOF, then seeked back into the > > file. He was expecting to get a new kevent after seeking back into the file > > since for his file descriptor after the lseek "there is data available to > > read" and "the file pointer is not at the end of file". I have a patch to fix > > this by doing a KNOTE() on a vnode after a successful seek. I checked OS X > > and it looks like they added this to their lseek() in Snow Leopard > > (http://fxr.watson.org/fxr/source/bsd/vfs/vfs_syscalls.c?v=xnu-1699.24.8#L4182). > > > > The one patch to fix this is below along with a test. Note that unlike OS X > > I did not add a new NOTE_NONE for this case. OS X has logic in their VFS > > filter operations that make special assumptions about a hint value of 0, so > > they had to add NOTE_NONE as a hack. We do not have the same special > > assumptions about a hint of 0, so we can just use "0". Without this fix the > > test below complains about missing events for the "after seek" and "after > > third read" cases. > > Just curious - wouldn't it generate a spurious event if lseek is > performed with zero offset, e.g. SEEK_CUR with offset 0 ? Yes, it would, though if you aren't specifying EV_CLEAR and haven't gotten to EOF yet, then it would already be pending anyway. In the case of EV_CLEAR this could cause a new event to fire, yes. However, judging by OS X's implementation of lseek(), they would do the same. Also, you won't get an event if you are at EOF and merely seek back to EOF. I suppose we could check for the case where the new offset == the old one, but I'm not sure it is a common enough case in conjunction with use of kevents() to really warrant that? Note you can already get spurious events on an EVFILT_READ filter if some other process creates a new link to the file, or does a chown, chmod, etc. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Wed Dec 14 20:46:06 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CE7DF106566B; Wed, 14 Dec 2011 20:46:06 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 4E3EF8FC1A; Wed, 14 Dec 2011 20:46:05 +0000 (UTC) Received: from alf.home (alf.kiev.zoral.com.ua [10.1.1.177]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id pBEKjxCA009938 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 14 Dec 2011 22:46:00 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from alf.home (kostik@localhost [127.0.0.1]) by alf.home (8.14.5/8.14.5) with ESMTP id pBEKjxYT045774; Wed, 14 Dec 2011 22:45:59 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by alf.home (8.14.5/8.14.5/Submit) id pBEKjxBJ045773; Wed, 14 Dec 2011 22:45:59 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: alf.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 14 Dec 2011 22:45:59 +0200 From: Kostik Belousov To: John Baldwin Message-ID: <20111214204559.GI50300@deviant.kiev.zoral.com.ua> References: <201112141141.41168.jhb@freebsd.org> <20111214165830.GG50300@deviant.kiev.zoral.com.ua> <201112141543.14130.jhb@freebsd.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="M3mBFhrxhGLhlzsC" Content-Disposition: inline In-Reply-To: <201112141543.14130.jhb@freebsd.org> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: arch@freebsd.org Subject: Re: Changing lseek() to KNOTE on the vnode when seeking on a file X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2011 20:46:06 -0000 --M3mBFhrxhGLhlzsC Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Dec 14, 2011 at 03:43:13PM -0500, John Baldwin wrote: > On Wednesday, December 14, 2011 11:58:30 am Kostik Belousov wrote: > > On Wed, Dec 14, 2011 at 11:41:41AM -0500, John Baldwin wrote: > > > A co-worker ran into an issue with using an EVFILT_READ kevent on a r= egular=20 > > > file recently. Specifically, in the manpage it says: > > >=20 > > > EVFILT_READ Takes a descriptor as the identifier, and returns= whenever > > > there is data available to read. The behavior of= the fil- > > > ter is slightly different depending on the descri= ptor > > > type. > > >=20 > > > ... > > >=20 > > > Vnodes > > > Returns when the file pointer is not at the e= nd of > > > file. data contains the offset from current = position > > > to end of file, and may be negative. > > >=20 > > > He was then working on a program that read to EOF, then seeked back i= nto the > > > file. He was expecting to get a new kevent after seeking back into t= he file > > > since for his file descriptor after the lseek "there is data availabl= e to=20 > > > read" and "the file pointer is not at the end of file". I have a pat= ch to fix=20 > > > this by doing a KNOTE() on a vnode after a successful seek. I checke= d OS X=20 > > > and it looks like they added this to their lseek() in Snow Leopard > > > (http://fxr.watson.org/fxr/source/bsd/vfs/vfs_syscalls.c?v=3Dxnu-1699= .24.8#L4182). > > >=20 > > > The one patch to fix this is below along with a test. Note that unli= ke OS X > > > I did not add a new NOTE_NONE for this case. OS X has logic in their= VFS > > > filter operations that make special assumptions about a hint value of= 0, so > > > they had to add NOTE_NONE as a hack. We do not have the same special= =20 > > > assumptions about a hint of 0, so we can just use "0". Without this = fix the > > > test below complains about missing events for the "after seek" and "a= fter=20 > > > third read" cases. > >=20 > > Just curious - wouldn't it generate a spurious event if lseek is > > performed with zero offset, e.g. SEEK_CUR with offset 0 ? >=20 > Yes, it would, though if you aren't specifying EV_CLEAR and haven't gotte= n to > EOF yet, then it would already be pending anyway. In the case of EV_CLEAR > this could cause a new event to fire, yes. However, judging by OS X's > implementation of lseek(), they would do the same. Also, you won't get an > event if you are at EOF and merely seek back to EOF. I suppose we could > check for the case where the new offset =3D=3D the old one, but I'm not s= ure it > is a common enough case in conjunction with use of kevents() to really > warrant that? Note you can already get spurious events on an EVFILT_READ > filter if some other process creates a new link to the file, or does a > chown, chmod, etc. I think getting the note is fine, I mostly checked my understanding of the patch. I do not see any value in checking the specific case to not send the note. An application shall be ready to get the file extended and truncated by other process anyway. --M3mBFhrxhGLhlzsC Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk7pCwcACgkQC3+MBN1Mb4hzNgCcDnZIg9EJ3EndhVnzhLzUrDWu 2LYAoMr1xiI/K8BImkCLg5iqkh9/1wjM =TTOk -----END PGP SIGNATURE----- --M3mBFhrxhGLhlzsC-- From owner-freebsd-arch@FreeBSD.ORG Thu Dec 15 03:40:57 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 430761065673 for ; Thu, 15 Dec 2011 03:40:57 +0000 (UTC) (envelope-from lstewart@freebsd.org) Received: from lauren.room52.net (lauren.room52.net [210.50.193.198]) by mx1.freebsd.org (Postfix) with ESMTP id ABADD8FC0C for ; Thu, 15 Dec 2011 03:40:56 +0000 (UTC) Received: from lstewart.caia.swin.edu.au (lstewart.caia.swin.edu.au [136.186.229.95]) by lauren.room52.net (Postfix) with ESMTPSA id E76967E824; Thu, 15 Dec 2011 14:20:58 +1100 (EST) Message-ID: <4EE9679A.70909@freebsd.org> Date: Thu, 15 Dec 2011 14:20:58 +1100 From: Lawrence Stewart User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111127 Thunderbird/8.0 MIME-Version: 1.0 To: freebsd-arch@freebsd.org References: <201111210417.pAL4HOdi023556@svn.freebsd.org> <4ED8575D.2010405@freebsd.org> <201112021927.25234.jkim@FreeBSD.org> <201112022002.49618.jkim@FreeBSD.org> <4EDB6814.9040403@freebsd.org> <405E9C58-F999-45C2-BC20-81ED4CA8A3E9@cubinlab.ee.unimelb.edu.au> In-Reply-To: <405E9C58-F999-45C2-BC20-81ED4CA8A3E9@cubinlab.ee.unimelb.edu.au> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.0 required=5.0 tests=UNPARSEABLE_RELAY autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lauren.room52.net Cc: Julien Ridoux , Benjamin Kaduk , Jung-uk Kim Subject: System clock APIs and feed-forward clock integration with BPF (was "Re: svn commit: r227778 - head/sys/net") X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2011 03:40:57 -0000 [Moving discussion to freebsd-arch@ and renaming thread] On 12/15/11 01:30, Julien Ridoux wrote: > > On 04/12/2011, at 11:31 PM, Lawrence Stewart wrote: > >> On 12/03/11 12:02, Jung-uk Kim wrote: >>> On Friday 02 December 2011 07:27 pm, Jung-uk Kim wrote: >>>> On Thursday 01 December 2011 11:43 pm, Lawrence Stewart wrote: >>>>> On 12/02/11 03:43, Jung-uk Kim wrote: >>>>>> On Thursday 01 December 2011 10:11 am, Lawrence Stewart wrote: >>>>>>> On 11/30/11 05:09, Jung-uk Kim wrote: >>>>>>>> On Tuesday 29 November 2011 11:13 am, Lawrence Stewart wrote: >> [snip] >>>>>>>>> Here's the first of the patches: >>>>>>>>> >>>>>>>>> http://people.freebsd.org/~lstewart/patches/misc/ffclock_bpf >>>>>>>>> _i nt act abi_10.x.r228130.patch >>>>>>>> >>>>>>>> I only glanced at it but it looks very close to what I wanted >>>>>>>> to suggest. >>>>>>> >>>>>>> Final candidate patch is at: >>>>>>> >>>>>>> http://people.freebsd.org/~lstewart/patches/misc/ffclock_bpf_i >>>>>>> nt act abi_10.x.r228180.patch >>>>>>> >>>>>>> Assuming it passes the "make tinderbox" build I'm currently >>>>>>> running and no further input is received from interested >>>>>>> parties, I plan to commit it in ~10 hours. >>>>>>> >>>>>>> Changes since the r228130 patch I sent previously: >>>>>>> >>>>>>> - The new flags in bpf.h are added unconditionally so that >>>>>>> they can always be referenced at compile time and a decision >>>>>>> made at runtime as to whether a flag will be set or not. Using >>>>>>> one of the new flags when the kernel doesn't have FFCLOCK >>>>>>> compiled in results in the flag being ignored. An app should >>>>>>> check for the existence of the "ffclock" kernel feature or the >>>>>>> "kern.sysclock" sysctl tree before attempting to use the >>>>>>> flags. >>>>>>> >>>>>>> - This patch will hopefully be MFCed at some point, so I added >>>>>>> a CTASSERT to bpf.c to ensure that the ABI of structs >>>>>>> bpf_hdr32, bpf_hdr and bpf_xhdr remains intact when FFCLOCK is >>>>>>> enabled and the union of a ffcounter and struct >>>>>>> timeval32/timeval/bpf_ts is switched in. >>>>>>> >>>>>>> - bpf_gettime() more comprehensively covers all the possible >>>>>>> cases of flag combination and does sensible things for each >>>>>>> case (even though some cases are rather silly). >>>>>>> >>>>>>> - The snippet of code at the beginning of catchpacket() that >>>>>>> was manipulating the struct bintime derived from bpf_gettime() >>>>>>> was gross and has been removed in favour of selecting the >>>>>>> right {get}bin{up}time() function call in bpf_gettime(). >>>>>> >>>>>> I did that to reduce branching. Since you are introducing more >>>>>> branches, it warrants a function pointer now. >>>> >>>> There's another reason, BTW. If mbufs are tagged with timestamps >>>> (where only monotonic timestamps are allowed), they must be >>>> converted within the bpf.c. I forgot all the details. :-( >> >> We should document this knowledge in some code comments. >> >>>>> I see, thanks for the explanation. Could you elaborate a bit more >>>>> about how you would implement the function pointer idea? I'm also >>>>> curious in the !FFCLOCK case just how much overhead having the >>>>> 2-layer nested if/else adds. I'm not an very optimisation savvy >>>>> person, but I'm wondering if it's actually worth micro-optimising >>>>> this code. >>>>> >>>>> My initial thoughts about your function pointer idea lead to >>>>> adding a function pointer in the bpf_d and setting it to the >>>>> appropriate function to get the timestamp from at bpf_d creation >>>>> or ioctl time. Whilst I like this idea, I can't see how it would >>>>> work given that the various functions involved in time/ffcounter >>>>> stamp generation all have different signatures. >>>>> >>>>> We could have multiple variants of bpf_gettime() which each call >>>>> the appropriate underlying functions to generate the appropriate >>>>> stamp. Would add quite a lot of code but would reduce the >>>>> overhead of calling bpf_gettime() to an indirect function call + >>>>> the underlying stamp generation function call. This also solves >>>>> the problem of multiple function signatures. >>>> >>>> Please see my patch: >>>> >>>> http://people.freebsd.org/~jkim/bpf_ffclock.diff >>> >>> I booted up the kernel and found it just crashes because of stupid >>> typos. Now a new patch is uploaded in place. Sorry. >>> >>> Anyway, I see no regression nor ABI breakage. :-) >> >> struct bpf_d being part of the ABI was the main thing I was concerned about, so given that it isn't I like your approach a lot. >> >> As noted by Julien, this approach does introduce problems with respect to the follow up patch that adds a permanent bh_ffcounter member to the bpf header. I thought the fromclock() API would sufficiently meet the needs of consumers like BPF, but if we were to proceed with something like Jung-uk's proposed patch I don't think that's true anymore. >> >> We decided to bite the bullet and devise an API that is more compact and can return all appropriate time information from all underlying sysclocks in an efficient manner - something like a more generic version of ffclock_abstime(). Julien and I spent quite some time today nutting out details, and Julien has done a proof of concept patch against my proposed BPF patch which looks good as a starting point for discussion: >> >> http://www.cubinlab.ee.unimelb.edu.au/~jrid/patches/r228254/sysclock_snap.patch >> >> It needs a bit more work and should be split into two patches (one introducing the API and the other being the BPF changes). >> >> With something like this though, the BPF patch becomes radically simplified in the FFCLOCK case. We don't even need a function pointer cached in the bpf_d anymore, but could cache a struct sysclock_snap there instead if we really wanted to minimise overhead in bpf_gettime(). >> >> We'll have a go at refining the patch tomorrow hopefully, but wanted to put this out there for early consideration. >> >> Cheers, >> Lawrence > > > Hi Jung-uk (and all), > > It took us a bit of time but we have a new patch. There are a lot of changes in the patch, and I will start by summarising our motivation for the changes. Hopefully I won't forget anything, but if I leave something unclear, don't hesitate to ask Lawrence or me. > > Starting with the easy bits, we drop the union in the BPF header and followed something similar to what Ben and you proposed. > > Next, the core motivation was to ensure that the timestamping function be called once only per packet captured, while keeping the ability to read both clocks and return the time from one of the two to the user. This forced us to redesign the timestamping function and one thing leading to another a fair bit of the BPF code. > > There 3 orthogonal parameters to the timestamping function in the BPF context: > - which clock is used to return the time > - what is the format of the value returned > - and a performance issue (nothing to do with timestamping or timekeeping): reading the hardware counter or not > > We handle the first one with a new sysclock_getsnapshot() which saves both clock parameters when timestamping and potentially reads the hardware counter. > The second one has been covered in previous discussion with the addition of BPF_T_FFCOUNTER. > > The performance issue created problems. In the current code, the order with which the BPF devices are open on one interface impacts the timestamping function. Let's consider two BPF devices respectively open with BPF_TSTAMP_NORMAL and BPF_TSTAMP_FAST quality. If NORMAL is open after FAST, it is at the head of the list, the hardware counter is read, and FAST benefits from higher quality timestamps. If FAST is open after NORMAL, the timestamping function is called once and return last kernel tick time, then it is called a second time for NORMAL, this time accessing the hardware counter. > > None of these cases are satisfactory. This lead to inconsistent behaviour based on the order the BPF are open. An application can impact the settings of other BPF devices and other applications / kernel consumers. The timestamping suffers from higher latency and worse, higher variability of the latency, which is bad. > > We then took a different approach, which is inspired by your work on BPF. We believe the FAST/NORMAL settings should have 2 level of granularity: system wide and per interface. The system wide is the default behaviour (for example all BPF devices are NORMAL) and the interface one super-seeds the system one. This is implemented by the use of new sysctl living in the net.bpf tree. > > This enables many scenarios such as: > - system default is FAST to improve performance because the system uses a slow timecounter (eg HPET or ACPI). > - the system has 3 interfaces. Interface 2 is put in NORMAL mode (need for accuracy but not much traffic), and the last can do hardware timestamping and is put in BPF_TSTAMP_EXTERNAL mode. > > This then naturally led to store the performance setting (none/fast/normal/external) for each interface instead of each BPF descriptor. > > Each BPF descriptor still maintains the other 2 settings: the choice of clock and the format of the timestamp independently from each other. > > We believe this new patch keeps the spirit of the changes you proposed and improves it while accomodating the changes required by the feed-forward clock and future evolutions with things like IEEE 1588 compatible NICs. > > We plan to push this patch as soon as possible. We would really appreciate if you could comment on this long email and pach very soon. > > The patch can be found here: > http://www.cubinlab.ee.unimelb.edu.au/~jrid/patches/r228429-v4/ffclock_bpf_intactabi.patch > > We will adjust the documentation as well once we have reached consensus. > > Julien (and Lawrence of course) From owner-freebsd-arch@FreeBSD.ORG Fri Dec 16 09:47:47 2011 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1D5EA1065783; Fri, 16 Dec 2011 09:47:47 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 3411E8FC1A; Fri, 16 Dec 2011 09:47:45 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id LAA15209; Fri, 16 Dec 2011 11:47:43 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1RbUOE-0005RF-Hg; Fri, 16 Dec 2011 11:47:42 +0200 Message-ID: <4EEB13BD.3060401@FreeBSD.org> Date: Fri, 16 Dec 2011 11:47:41 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:8.0) Gecko/20111206 Thunderbird/8.0 MIME-Version: 1.0 To: arch@FreeBSD.org, FreeBSD current References: <4EE52469.3050201@FreeBSD.org> <4EE79014.3070006@FreeBSD.org> In-Reply-To: <4EE79014.3070006@FreeBSD.org> X-Enigmail-Version: undefined Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Subject: Re: grabbing console (syscons) in kernel X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2011 09:47:47 -0000 on 13/12/2011 19:49 Andriy Gapon said the following: > on 11/12/2011 23:45 Andriy Gapon said the following: >> >> There are a few cases when the kernel needs to interact with a user via syscons. >> This is the cases where the kernel not only spews some output but also expects >> some input. Some examples are: >> - asking for a root filesystem specification >> - entering ddb >> - asking to press a key for reboot >> >> In this cases the kernel implicitly grabs the console for its own use. >> I'd like to make action more explicit. >> >> What do you think about the approach and implementation in the following patches? >> Thank you! >> >> https://gitorious.org/~avg/freebsd/avgbsd/commit/5248b49ebf84d98a0597fa5aa4d813a38f581acc >> https://gitorious.org/~avg/freebsd/avgbsd/commit/a0849c52242378474bb2eaa41726376fbc4c5bf6 >> https://gitorious.org/~avg/freebsd/avgbsd/commit/a67515cbd720b16f03ba435ed182966a8a338b15 >> https://gitorious.org/~avg/freebsd/avgbsd/commit/b8864b68b4c0e26ece065a38301c305833be32eb >> https://gitorious.org/~avg/freebsd/avgbsd/commit/1017ae425d8abecd7482bd6c6deaaf9f25f5c6cd > > > I was advised that the above links might not be the best way to present the > patches for review, so here are them as a single diff file: > http://people.freebsd.org/~avg/cngrab.diff I plan to start committing these changes tomorrow. So if this is "your area" or you want to be sure that nothing gets broken in this area, then please review the changes or let me know your review plans. Post-commit reviews are of course welcome as usual, but pre-commit reviews are preferred. Thank you. >> P.S. one of the benefits is that a keyboard is put into and out of the polling >> mode before getting all the required input and after that; not around each >> character as it is done now in rather twisted way. -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Fri Dec 16 20:59:38 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 15EC3106566C for ; Fri, 16 Dec 2011 20:59:38 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id E46F68FC08 for ; Fri, 16 Dec 2011 20:59:37 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [96.47.65.170]) by cyrus.watson.org (Postfix) with ESMTPSA id 9CE2446B0D for ; Fri, 16 Dec 2011 15:59:37 -0500 (EST) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 2D81DB915 for ; Fri, 16 Dec 2011 15:59:37 -0500 (EST) From: John Baldwin To: arch@freebsd.org Date: Fri, 16 Dec 2011 15:59:36 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p8; KDE/4.5.5; amd64; ; ) MIME-Version: 1.0 Content-Type: Text/Plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Message-Id: <201112161559.36428.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 16 Dec 2011 15:59:37 -0500 (EST) Cc: Subject: TASK_INITIALIZER() X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2011 20:59:38 -0000 Any objection to adding a macro to make it easy to statically initialize task structures (similar to the initializer macros in )? This allows global tasks to be statically initalized without requiring a dedicated SYSINIT() routine. Index: share/man/man9/Makefile =================================================================== --- share/man/man9/Makefile (revision 228534) +++ share/man/man9/Makefile (working copy) @@ -1250,6 +1250,7 @@ sysctl_ctx_init.9 sysctl_ctx_free.9 MLINKS+=SYSINIT.9 SYSUNINIT.9 MLINKS+=taskqueue.9 TASK_INIT.9 \ + taskqueue.9 TASK_INITIALIZER.9 \ taskqueue.9 taskqueue_cancel.9 \ taskqueue.9 taskqueue_create.9 \ taskqueue.9 taskqueue_create_fast.9 \ Index: share/man/man9/taskqueue.9 =================================================================== --- share/man/man9/taskqueue.9 (revision 228534) +++ share/man/man9/taskqueue.9 (working copy) @@ -80,6 +80,7 @@ .Ft void .Fn taskqueue_run "struct taskqueue *queue" .Fn TASK_INIT "struct task *task" "int priority" "task_fn_t func" "void *context" +.Fn TASK_INITIALIZER "int priority" "task_fn_t func" "void *context" .Fn TASKQUEUE_DECLARE "name" .Fn TASKQUEUE_DEFINE "name" "taskqueue_enqueue_fn enqueue" "void *context" "init" .Fn TASKQUEUE_FAST_DEFINE "name" "taskqueue_enqueue_fn enqueue" "void *context" "init" @@ -243,9 +244,14 @@ is provided to initialise a .Va task structure. +The +.Fn TASK_INITIALIZER +macro generates an initializer for a task structure. A macro .Fn TIMEOUT_TASK_INIT "queue" "timeout_task" "priority" "func" "context" -initializes the timeout_task structure. +initializes the +.Va timeout_task +structure. The values of .Va priority , .Va func , Index: sys/sys/taskqueue.h =================================================================== --- sys/sys/taskqueue.h (revision 228534) +++ sys/sys/taskqueue.h (working copy) @@ -77,6 +77,12 @@ void taskqueue_unblock(struct taskqueue *queue); int taskqueue_member(struct taskqueue *queue, struct thread *td); +#define TASK_INITIALIZER(priority, func, context) \ + { .ta_pending = 0, \ + .ta_priority = (priority), \ + .ta_func = (func), \ + .ta_context = (context) } + /* * Functions for dedicated thread taskqueues */ -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri Dec 16 21:49:15 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A4B6910656A7; Fri, 16 Dec 2011 21:49:15 +0000 (UTC) (envelope-from ed@hoeg.nl) Received: from mx0.hoeg.nl (mx0.hoeg.nl [IPv6:2a01:4f8:101:5343::aa]) by mx1.freebsd.org (Postfix) with ESMTP id 7B8B08FC12; Fri, 16 Dec 2011 21:49:14 +0000 (UTC) Received: by mx0.hoeg.nl (Postfix, from userid 1000) id 627E92A28E79; Fri, 16 Dec 2011 22:49:13 +0100 (CET) Date: Fri, 16 Dec 2011 22:49:13 +0100 From: Ed Schouten To: threads@freebsd.org, arch@freebsd.org Message-ID: <20111216214913.GA1771@hoeg.nl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="MvEh5MSbieV/1Yst" Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Subject: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2011 21:49:15 -0000 --MvEh5MSbieV/1Yst Content-Type: multipart/mixed; boundary="HSVPcR81XVhxeM7P" Content-Disposition: inline --HSVPcR81XVhxeM7P Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi folks, We can expect C11 (or C12) to be released in the nearby future. I already took the liberty of adding fallbacks for some of the language keywords to , but it seems the standard also includes a new threading API. In my opinion the ISO folks suffer a bit from the Not Invented Here syndrome. In an earlier revision of the C1X specification, they even described a `struct xtime', which had a purpose identical to `struct timespec'. The same holds for the threading API. It can be 1:1 mapped to a subset of pthread -- why not simply standardize that subset then? Putting my personal objections aside, I do think we should add support for the API by the time C1X is final. Attached is a patch that adds the new API to libthr. It simply wraps all the calls around pthread. Even the objects used by the API are type compatible with the ones used by pthread. The questions: - As this API is just a second-class citizen and will not be used by the C library, I suspect there is no need for underscore prefixing and weak aliasing. Is this correct? - When accompanied with man pages, are there any objections if I commit this to SVN when C1X is official? Thanks, --=20 Ed Schouten WWW: http://80386.nl/ --HSVPcR81XVhxeM7P Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="c1x-threads.diff" Content-Transfer-Encoding: quoted-printable Index: include/threads.h =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- include/threads.h (revision 0) +++ include/threads.h (working copy) @@ -0,0 +1,106 @@ +/*- + * Copyright (c) 2012 Ed Schouten + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURP= OSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENT= IAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STR= ICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY W= AY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#ifndef _THREADS_H_ +#define _THREADS_H_ + +#include + +/* + * The C1X threads interface. + * + * This interface is implemented as a light-weight wrapper around + * . To prevent namespace pollution, the once_flag object, + * its corresponding ONCE_FLAG_INIT and TSS_DTOR_ITERATIONS have been + * copied from this header file. They must be kept in sync. + */ + +typedef struct pthread_cond *cnd_t; +typedef struct pthread_mutex *mtx_t; +typedef struct pthread *thrd_t; +typedef int tss_t; + +typedef struct { + int __state; + mtx_t __mutex; +} once_flag; + +typedef void (*tss_dtor_t)(void *); +typedef int (*thrd_start_t)(void *); + +enum { + mtx_plain =3D 0x1, + mtx_recursive =3D 0x2, + mtx_timed =3D 0x4 +}; + +enum { + thrd_busy =3D 1, + thrd_error =3D 2, + thrd_nomem =3D 3, + thrd_success =3D 4, + thrd_timedout =3D 5 +}; + +#if !defined(__cplusplus) || __cplusplus < 201103L +#define thread_local _Thread_local +#endif +#define ONCE_FLAG_INIT { 0, NULL } +#define TSS_DTOR_ITEARTIONS 4 + +__BEGIN_DECLS +void call_once(once_flag *, void (*)(void)); +int cnd_broadcast(cnd_t *); +void cnd_destroy(cnd_t *); +int cnd_init(cnd_t *); +int cnd_signal(cnd_t *); +int cnd_timedwait(cnd_t *__restrict, mtx_t *__restrict, + const struct timespec *__restrict); +int cnd_wait(cnd_t *, mtx_t *); +void mtx_destroy(mtx_t *); +int mtx_init(mtx_t *, int); +int mtx_lock(mtx_t *); +int mtx_timedlock(mtx_t *__restrict, const struct timespec *__restrict); +int mtx_trylock(mtx_t *); +int mtx_unlock(mtx_t *); +int thrd_create(thrd_t *, thrd_start_t, void *); +thrd_t thrd_current(void); +int thrd_detach(thrd_t); +int thrd_equal(thrd_t, thrd_t); +_Noreturn void + thrd_exit(int); +int thrd_join(thrd_t, int *); +int thrd_sleep(const struct timespec *, struct timespec *); +void thrd_yield(void); +int tss_create(tss_t *, tss_dtor_t); +void tss_delete(tss_t); +void * tss_get(tss_t); +int tss_set(tss_t, void *); +__END_DECLS + +#endif /* !_THREADS_H_ */ Index: lib/libthr/c1x/c1x_cnd.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- lib/libthr/c1x/c1x_cnd.c (revision 0) +++ lib/libthr/c1x/c1x_cnd.c (working copy) @@ -0,0 +1,89 @@ +/*- + * Copyright (c) 2012 Ed Schouten + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURP= OSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENT= IAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STR= ICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY W= AY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#include +__FBSDID("$FreeBSD$"); + +#include "namespace.h" +#include +#include +#include +#include "un-namespace.h" + +int +cnd_broadcast(cnd_t *cond) +{ + + if (_pthread_cond_broadcast(cond) !=3D 0) + return (thrd_error); + return (thrd_success); +} + +void +cnd_destroy(cnd_t *cond) +{ + + (void)_pthread_cond_destroy(cond); +} + +int +cnd_init(cnd_t *cond) +{ + + if (_pthread_cond_init(cond, NULL) !=3D 0) + return (errno =3D=3D ENOMEM ? thrd_nomem : thrd_error); + return (thrd_success); +} + +int +cnd_signal(cnd_t *cond) +{ + + if (_pthread_cond_signal(cond) !=3D 0) + return (thrd_error); + return (thrd_success); +} + +int +cnd_timedwait(cnd_t *restrict cond, mtx_t *restrict mtx, + const struct timespec *restrict ts) +{ + + if (_pthread_cond_timedwait(cond, mtx, ts) !=3D 0) + return (errno =3D=3D ETIMEDOUT ? thrd_timedout : thrd_error); + return (thrd_success); +} + +int +cnd_wait(cnd_t *cond, mtx_t *mtx) +{ + + if (_pthread_cond_wait(cond, mtx) !=3D 0) + return (thrd_error); + return (thrd_success); +} Index: lib/libthr/c1x/c1x_mtx.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- lib/libthr/c1x/c1x_mtx.c (revision 0) +++ lib/libthr/c1x/c1x_mtx.c (working copy) @@ -0,0 +1,108 @@ +/*- + * Copyright (c) 2012 Ed Schouten + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURP= OSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENT= IAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STR= ICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY W= AY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#include +__FBSDID("$FreeBSD$"); + +#include "namespace.h" +#include +#include +#include +#include "un-namespace.h" + +void +mtx_destroy(mtx_t *mtx) +{ + + (void)_pthread_mutex_destroy(mtx); +} + +int +mtx_init(mtx_t *mtx, int type) +{ + pthread_mutexattr_t attr; + int mt; + + switch (type) { + case mtx_plain: + case mtx_timed: + mt =3D PTHREAD_MUTEX_NORMAL; + break; + case mtx_plain | mtx_recursive: + case mtx_timed | mtx_recursive: + mt =3D PTHREAD_MUTEX_RECURSIVE; + break; + default: + errno =3D EINVAL; + return (thrd_error); + } + + if (_pthread_mutexattr_init(&attr) !=3D 0) + return (thrd_error); + if (_pthread_mutexattr_settype(&attr, mt) !=3D 0) + return (thrd_error); + if (_pthread_mutex_init(mtx, &attr) !=3D 0) + return (thrd_error); + return (thrd_success); +} + +int +mtx_lock(mtx_t *mtx) +{ + + if (_pthread_mutex_lock(mtx) !=3D 0) + return (thrd_error); + return (thrd_success); +} + +int +mtx_timedlock(mtx_t *restrict mtx, const struct timespec *restrict ts) +{ + + if (_pthread_mutex_timedlock(mtx, ts) !=3D 0) + return (errno =3D=3D ETIMEDOUT ? thrd_timedout : thrd_error); + return (thrd_success); +} + +int +mtx_trylock(mtx_t *mtx) +{ + + if (_pthread_mutex_lock(mtx) !=3D 0) + return (errno =3D=3D EBUSY ? thrd_busy : thrd_error); + return (thrd_success); +} + +int +mtx_unlock(mtx_t *mtx) +{ + + if (_pthread_mutex_unlock(mtx) !=3D 0) + return (thrd_error); + return (thrd_success); +} Index: lib/libthr/c1x/c1x_tss.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- lib/libthr/c1x/c1x_tss.c (revision 0) +++ lib/libthr/c1x/c1x_tss.c (working copy) @@ -0,0 +1,67 @@ +/*- + * Copyright (c) 2012 Ed Schouten + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURP= OSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENT= IAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STR= ICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY W= AY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#include +__FBSDID("$FreeBSD$"); + +#include "namespace.h" +#include +#include +#include "un-namespace.h" + +int +tss_create(tss_t *key, tss_dtor_t dtor) +{ + + if (_pthread_key_create(key, dtor) !=3D 0) + return (thrd_error); + return (thrd_success); +} + +void +tss_delete(tss_t key) +{ + + (void)_pthread_key_delete(key); +} + +void * +tss_get(tss_t key) +{ + + return (_pthread_getspecific(key)); +} + +int +tss_set(tss_t key, void *val) +{ + + if (_pthread_setspecific(key, val) !=3D 0) + return (thrd_error); + return (thrd_success); +} Index: lib/libthr/c1x/Makefile.inc =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- lib/libthr/c1x/Makefile.inc (revision 0) +++ lib/libthr/c1x/Makefile.inc (working copy) @@ -0,0 +1,11 @@ +# $FreeBSD: head/lib/libthr/thread/Makefile.inc 218414 2011-02-07 21:26:46= Z jkim $ + +# thr sources +.PATH: ${.CURDIR}/c1x + +SRCS+=3D \ + c1x_call_once.c \ + c1x_cnd.c \ + c1x_mtx.c \ + c1x_thrd.c \ + c1x_tss.c Index: lib/libthr/c1x/c1x_call_once.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- lib/libthr/c1x/c1x_call_once.c (revision 0) +++ lib/libthr/c1x/c1x_call_once.c (working copy) @@ -0,0 +1,42 @@ +/*- + * Copyright (c) 2012 Ed Schouten + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURP= OSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENT= IAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STR= ICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY W= AY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#include +__FBSDID("$FreeBSD$"); + +#include "namespace.h" +#include +#include +#include "un-namespace.h" + +void +call_once(once_flag *flag, void (*func)(void)) +{ + + (void)_pthread_once((pthread_once_t *)flag, func); +} Index: lib/libthr/c1x/c1x_thrd.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- lib/libthr/c1x/c1x_thrd.c (revision 0) +++ lib/libthr/c1x/c1x_thrd.c (working copy) @@ -0,0 +1,128 @@ +/*- + * Copyright (c) 2012 Ed Schouten + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURP= OSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENT= IAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STR= ICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY W= AY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * $FreeBSD$ + */ + +#include +__FBSDID("$FreeBSD$"); + +#include "namespace.h" +#include +#include +#include +#include +#include "un-namespace.h" + +struct thrd_param { + thrd_start_t func; + void *arg; +}; + +static void * +thrd_entry(void *arg) +{ + struct thrd_param tp; + + tp =3D *(struct thrd_param *)arg; + free(arg); + return ((void *)(intptr_t)tp.func(tp.arg)); +} + +int +thrd_create(thrd_t *thr, thrd_start_t func, void *arg) +{ + struct thrd_param *tp; + + /* + * Work around return type inconsistency. Wrap execution using + * a function conforming to pthread_create()'s start_routine. + */ + tp =3D malloc(sizeof(*tp)); + if (tp =3D=3D NULL) + return (thrd_nomem); + tp->func =3D func; + tp->arg =3D arg; + if (_pthread_create(thr, NULL, thrd_entry, tp) !=3D 0) { + free(tp); + return (thrd_error); + } + return (thrd_success); +} + +thrd_t +thrd_current(void) +{ + + return (_pthread_self()); +} + +int +thrd_detach(thrd_t thr) +{ + + if (_pthread_detach(thr) !=3D 0) + return (thrd_error); + return (thrd_success); +} + +int +thrd_equal(thrd_t thr0, thrd_t thr1) +{ + + return (_pthread_equal(thr0, thr1)); +} + +_Noreturn void +thrd_exit(int res) +{ + + _pthread_exit((void *)(intptr_t)res); +} + +int +thrd_join(thrd_t thr, int *res) +{ + void *value_ptr; + + if (_pthread_join(thr, &value_ptr) !=3D 0) + return (thrd_error); + *res =3D (intptr_t)value_ptr; + return (thrd_success); +} + +int +thrd_sleep(const struct timespec *duration, struct timespec *remaining) +{ + + return (_nanosleep(duration, remaining)); +} + +void +thrd_yield(void) +{ + + _pthread_yield(); +} Index: lib/libthr/pthread.map =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- lib/libthr/pthread.map (revision 228590) +++ lib/libthr/pthread.map (working copy) @@ -408,3 +408,31 @@ setcontext; swapcontext; }; + +FBSD_1.3 { + call_once; + cnd_broadcast; + cnd_destroy; + cnd_init; + cnd_signal; + cnd_timedwait; + cnd_wait; + mtx_destroy; + mtx_init; + mtx_lock; + mtx_timedlock; + mtx_trylock; + mtx_unlock; + thrd_create; + thrd_current; + thrd_detach; + thrd_equal; + thrd_exit; + thrd_join; + thrd_sleep; + thrd_yield; + tss_create; + tss_delete; + tss_get; + tss_set; +}; Index: lib/libthr/Makefile =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- lib/libthr/Makefile (revision 228590) +++ lib/libthr/Makefile (working copy) @@ -47,6 +47,7 @@ .PATH: ${.CURDIR}/arch/${MACHINE_CPUARCH}/${MACHINE_CPUARCH} =20 .include "${.CURDIR}/arch/${MACHINE_CPUARCH}/Makefile.inc" +.include "${.CURDIR}/c1x/Makefile.inc" .include "${.CURDIR}/sys/Makefile.inc" .include "${.CURDIR}/thread/Makefile.inc" =20 --HSVPcR81XVhxeM7P-- --MvEh5MSbieV/1Yst Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iQIbBAEBAgAGBQJO67zZAAoJEG5e2P40kaK7n7sP+KGuwANm6TUNjPHOMFfMewI6 4BBQjsJTzTzdFbqEEqQa5dNwd6fQOparlJ8A9jVHvIS/MD+IHGtbV8R9znb9a2um Fpsy0VCqt7jEcqOXARMYaupO0w8oVbwwSiuWEiIahYRkgxq76z7jEL80sM7n5HbN X7ftkjEQ5pcjEPqTHlSKQdD6HVP7M8qUqoanZ+r+HMHNgqwMKJuOXBHxKIpyMolC xZbGyH4etKGmLIiRVGqCOPrkzHDck/wMRb6OahZa0wuNNM7tRY//xOmw3OPw7P+0 +Br8jm9CAaDYWq4ajvPqLWOELlMY3Mm8rOaN5lrtFN8qk5Z5GE6LJG1QZIHqgmNR SGLvdaDskqi6QVGq4hTOaKbANTl+MJ0O+XWEBVq2yasQSYQt1b1jlBPIw3KIpAXh PsIolPEycGhwpvmhmOvfBGnVddh1HpFYIcMeq/N33R3CIDhtQtrypbK2H/t3qpN2 5AV1WyTqeLelfBeb6SNF4TM5BSewA1Bo2okx8ZGynzw5AjN8EXNnSgC6JCqMLxla u+Whj0bPNU+QT8BZfNO8Ci+wrvEcz1Vx15vfT5MJo/wRRW/STXhRuo4qzlwwMrK3 VxlLtCXUguFEvWxey/InuecVykZF4Fy9HIiF7MOxxpJhSYXHOtBODFg7p8qMHuAE UIX5gmzMOAXdCUnLUH0= =g8V1 -----END PGP SIGNATURE----- --MvEh5MSbieV/1Yst-- From owner-freebsd-arch@FreeBSD.ORG Fri Dec 16 22:09:23 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 40231106567A for ; Fri, 16 Dec 2011 22:09:23 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 97F0F8FC13 for ; Fri, 16 Dec 2011 22:09:21 +0000 (UTC) Received: from alf.home (alf.kiev.zoral.com.ua [10.1.1.177]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id pBGM9F3W054901 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 17 Dec 2011 00:09:15 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from alf.home (kostik@localhost [127.0.0.1]) by alf.home (8.14.5/8.14.5) with ESMTP id pBGM9FOB054834; Sat, 17 Dec 2011 00:09:15 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by alf.home (8.14.5/8.14.5/Submit) id pBGM9EbT054833; Sat, 17 Dec 2011 00:09:14 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: alf.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 17 Dec 2011 00:09:14 +0200 From: Kostik Belousov To: Ed Schouten Message-ID: <20111216220914.GW50300@deviant.kiev.zoral.com.ua> References: <20111216214913.GA1771@hoeg.nl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ubsQ1kiTK0VEt5fK" Content-Disposition: inline In-Reply-To: <20111216214913.GA1771@hoeg.nl> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: threads@freebsd.org, arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2011 22:09:23 -0000 --ubsQ1kiTK0VEt5fK Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Dec 16, 2011 at 10:49:13PM +0100, Ed Schouten wrote: > Hi folks, >=20 > We can expect C11 (or C12) to be released in the nearby future. I > already took the liberty of adding fallbacks for some of the language > keywords to , but it seems the standard also includes a new > threading API. >=20 > In my opinion the ISO folks suffer a bit from the Not Invented Here > syndrome. In an earlier revision of the C1X specification, they even > described a `struct xtime', which had a purpose identical to `struct > timespec'. The same holds for the threading API. It can be 1:1 mapped to > a subset of pthread -- why not simply standardize that subset then? >=20 > Putting my personal objections aside, I do think we should add support > for the API by the time C1X is final. Attached is a patch that adds the > new API to libthr. It simply wraps all the calls around pthread. Even > the objects used by the API are type compatible with the ones used by > pthread. >=20 > The questions: >=20 > - As this API is just a second-class citizen and will not be used by the > C library, I suspect there is no need for underscore prefixing and > weak aliasing. Is this correct? If application that does not use the new interface supposed to be able to implement function with new names, then the not-underscored symbols must be weak. >=20 > - When accompanied with man pages, are there any objections if I commit > this to SVN when C1X is official? +int +cnd_init(cnd_t *cond) +{ + + if (_pthread_cond_init(cond, NULL) !=3D 0) + return (errno =3D=3D ENOMEM ? thrd_nomem : thrd_error); + return (thrd_success); +} pthread_cond_init does not set errno, it returns error. There are several more instances of such errno use in the patch. Do you have reference to the draft ? --ubsQ1kiTK0VEt5fK Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk7rwYoACgkQC3+MBN1Mb4gs5wCgzHFzXHOpcpSYL8VL9n9Zm2nz V7oAnjAWXKFwKlFwG6T3hXn6TF6qTMBh =llEm -----END PGP SIGNATURE----- --ubsQ1kiTK0VEt5fK-- From owner-freebsd-arch@FreeBSD.ORG Fri Dec 16 22:20:00 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A1126106566B; Fri, 16 Dec 2011 22:20:00 +0000 (UTC) (envelope-from ed@hoeg.nl) Received: from mx0.hoeg.nl (mx0.hoeg.nl [IPv6:2a01:4f8:101:5343::aa]) by mx1.freebsd.org (Postfix) with ESMTP id 653148FC15; Fri, 16 Dec 2011 22:20:00 +0000 (UTC) Received: by mx0.hoeg.nl (Postfix, from userid 1000) id CA3382A28E7C; Fri, 16 Dec 2011 23:19:59 +0100 (CET) Date: Fri, 16 Dec 2011 23:19:59 +0100 From: Ed Schouten To: Kostik Belousov Message-ID: <20111216221959.GB1771@hoeg.nl> References: <20111216214913.GA1771@hoeg.nl> <20111216220914.GW50300@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="8Bkcj4NmnZ79d4W7" Content-Disposition: inline In-Reply-To: <20111216220914.GW50300@deviant.kiev.zoral.com.ua> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: threads@freebsd.org, arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2011 22:20:00 -0000 --8Bkcj4NmnZ79d4W7 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello Kostik, * Kostik Belousov , 20111216 23:09: > If application that does not use the new interface supposed to be > able to implement function with new names, then the not-underscored > symbols must be weak. For example when an application wants to implement its own functions that are named thrd_*(), for example? > pthread_cond_init does not set errno, it returns error. > There are several more instances of such errno use in the patch. Oh wait. It returns the errno. Ugh. Fixed! > Do you have reference to the draft ? Yes, sure: http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf Groetjes, --=20 Ed Schouten WWW: http://80386.nl/ --8Bkcj4NmnZ79d4W7 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iQIcBAEBAgAGBQJO68QPAAoJEG5e2P40kaK7CT4P/ive+2G9Ho1eisHxqcIZydqV DM6ai+WYSL7xC5OyZm56c5wOwpkfYH5NiEefQGCk82FlO+ULViW5FdmA2isll3tF 8dJ10hExNhAo7ZnfTvA6CsgRbvGjLluHEb7ELHYoMNHz68j64xmvA9wREQ06hPF0 OSZrAbsTxT6vBXSv/ABP54ovflFLrpfYyZAlqz1VNXFlwSvfZexTHIgtMmUO10jg cEwJUtkdT6xM/0XkoMz0Ho+igfXEYGzZTaDiGZKi+AdODKe+VzWIINk9P2U5Jhud cT5Q/wlHicEvlhIxCIpiQxhDYGndIsHBAf1Jp2iSo09gTQTSTlRLofeOUP1wbb93 pmTSKhgm3suXTunygHnIw1Wno2NFSuf1fbEvhlcvte55La0fTmPwkFOwD/YRom19 234MbHdoMw/PrtdzJWwkv6bNrqwUdlZQii1oOMv8uRClrawxOsLfl6G85WN0Fc// Z//VYVrs8YRT5wWdXKUMNA6KHz7bmiGbDLoliRkZteU8h7oD99eR6EODP2iOAZ+/ tTKYC+ejozwDTaFPyGYDAXb8wQHgNO2WktlGr5Lj65EAkLPVMpHggz31QNEHBwvf y+28MJ/UxDIoKaMBd3kn4Amwk2yUM/DCMauJWkk7/USyS/uwpIiiPDHUsWIuRENJ hnkH0GJV7UD91o+cbctI =W6qY -----END PGP SIGNATURE----- --8Bkcj4NmnZ79d4W7-- From owner-freebsd-arch@FreeBSD.ORG Fri Dec 16 22:31:32 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9198C106566B; Fri, 16 Dec 2011 22:31:32 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id E6F408FC1D; Fri, 16 Dec 2011 22:31:31 +0000 (UTC) Received: from alf.home (alf.kiev.zoral.com.ua [10.1.1.177]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id pBGMVQ3j056545 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 17 Dec 2011 00:31:26 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from alf.home (kostik@localhost [127.0.0.1]) by alf.home (8.14.5/8.14.5) with ESMTP id pBGMVQ7O054949; Sat, 17 Dec 2011 00:31:26 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by alf.home (8.14.5/8.14.5/Submit) id pBGMVQs8054948; Sat, 17 Dec 2011 00:31:26 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: alf.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 17 Dec 2011 00:31:26 +0200 From: Kostik Belousov To: Ed Schouten Message-ID: <20111216223126.GX50300@deviant.kiev.zoral.com.ua> References: <20111216214913.GA1771@hoeg.nl> <20111216220914.GW50300@deviant.kiev.zoral.com.ua> <20111216221959.GB1771@hoeg.nl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="GJTx2ed35C3D1GF1" Content-Disposition: inline In-Reply-To: <20111216221959.GB1771@hoeg.nl> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: threads@freebsd.org, arch@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Dec 2011 22:31:32 -0000 --GJTx2ed35C3D1GF1 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Dec 16, 2011 at 11:19:59PM +0100, Ed Schouten wrote: > Hello Kostik, >=20 > * Kostik Belousov , 20111216 23:09: > > If application that does not use the new interface supposed to be > > able to implement function with new names, then the not-underscored > > symbols must be weak. >=20 > For example when an application wants to implement its own functions > that are named thrd_*(), for example? Yes. The realistic example is the code written to C99/SUSv4 conformance that happens to define thrd_. It might be that easiest solution is to put the functions into separate library, besides defining them weak. >=20 > > pthread_cond_init does not set errno, it returns error. > > There are several more instances of such errno use in the patch. >=20 > Oh wait. It returns the errno. Ugh. Fixed! >=20 > > Do you have reference to the draft ? >=20 > Yes, sure: >=20 > http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf BTW, it looks not very useful to add a bunch of threading functions without at least trying to specify the memory model. --GJTx2ed35C3D1GF1 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk7rxr4ACgkQC3+MBN1Mb4jqNwCff+EFGr2qR0uU9jPSjtCQaTA5 JfcAoJa6EjoIPT4sttlyWCQ4n8qXEG/o =LlXk -----END PGP SIGNATURE----- --GJTx2ed35C3D1GF1-- From owner-freebsd-arch@FreeBSD.ORG Sat Dec 17 00:40:07 2011 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2AC4B106566B; Sat, 17 Dec 2011 00:40:07 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 989B58FC12; Sat, 17 Dec 2011 00:40:03 +0000 (UTC) Received: from [10.0.0.33] (99-195-103-194.dyn.centurytel.net [99.195.103.194]) (authenticated bits=0) by harmony.bsdimp.com (8.14.4/8.14.3) with ESMTP id pBH0VIhF018743 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES128-SHA bits=128 verify=NO); Fri, 16 Dec 2011 17:31:20 -0700 (MST) (envelope-from imp@bsdimp.com) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <20111216223126.GX50300@deviant.kiev.zoral.com.ua> Date: Fri, 16 Dec 2011 17:31:14 -0700 Content-Transfer-Encoding: 7bit Message-Id: References: <20111216214913.GA1771@hoeg.nl> <20111216220914.GW50300@deviant.kiev.zoral.com.ua> <20111216221959.GB1771@hoeg.nl> <20111216223126.GX50300@deviant.kiev.zoral.com.ua> To: Kostik Belousov X-Mailer: Apple Mail (2.1084) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (harmony.bsdimp.com [10.0.0.6]); Fri, 16 Dec 2011 17:31:22 -0700 (MST) Cc: Ed Schouten , arch@FreeBSD.org, threads@FreeBSD.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Dec 2011 00:40:07 -0000 On Dec 16, 2011, at 3:31 PM, Kostik Belousov wrote: > On Fri, Dec 16, 2011 at 11:19:59PM +0100, Ed Schouten wrote: >> Hello Kostik, >> >> * Kostik Belousov , 20111216 23:09: >>> If application that does not use the new interface supposed to be >>> able to implement function with new names, then the not-underscored >>> symbols must be weak. >> >> For example when an application wants to implement its own functions >> that are named thrd_*(), for example? > Yes. The realistic example is the code written to C99/SUSv4 conformance > that happens to define thrd_. > > It might be that easiest solution is to put the functions into > separate library, besides defining them weak. I thought the canonical solution here was to say #if POSIX_VISIBLE >= 201201 #endif Except this isn't posix. :( Warner From owner-freebsd-arch@FreeBSD.ORG Sat Dec 17 03:38:47 2011 Return-Path: Delivered-To: arch@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D00C1106564A; Sat, 17 Dec 2011 03:38:47 +0000 (UTC) (envelope-from das@FreeBSD.ORG) Received: from zim.MIT.EDU (ZIM.MIT.EDU [18.95.3.101]) by mx1.freebsd.org (Postfix) with ESMTP id 920A98FC08; Sat, 17 Dec 2011 03:38:47 +0000 (UTC) Received: from zim.MIT.EDU (localhost [127.0.0.1]) by zim.MIT.EDU (8.14.5/8.14.2) with ESMTP id pBH3BVYb015232; Fri, 16 Dec 2011 22:11:31 -0500 (EST) (envelope-from das@FreeBSD.ORG) Received: (from das@localhost) by zim.MIT.EDU (8.14.5/8.14.2/Submit) id pBH3BVuQ015231; Fri, 16 Dec 2011 22:11:31 -0500 (EST) (envelope-from das@FreeBSD.ORG) Date: Fri, 16 Dec 2011 22:11:31 -0500 From: David Schultz To: Warner Losh Message-ID: <20111217031131.GA15194@zim.MIT.EDU> Mail-Followup-To: Warner Losh , Kostik Belousov , Ed Schouten , arch@freebsd.org, threads@freebsd.org References: <20111216214913.GA1771@hoeg.nl> <20111216220914.GW50300@deviant.kiev.zoral.com.ua> <20111216221959.GB1771@hoeg.nl> <20111216223126.GX50300@deviant.kiev.zoral.com.ua> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Cc: Kostik Belousov , Ed Schouten , threads@FreeBSD.ORG, arch@FreeBSD.ORG Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Dec 2011 03:38:47 -0000 On Fri, Dec 16, 2011, Warner Losh wrote: > > On Dec 16, 2011, at 3:31 PM, Kostik Belousov wrote: > > > On Fri, Dec 16, 2011 at 11:19:59PM +0100, Ed Schouten wrote: > >> Hello Kostik, > >> > >> * Kostik Belousov , 20111216 23:09: > >>> If application that does not use the new interface supposed to be > >>> able to implement function with new names, then the not-underscored > >>> symbols must be weak. > >> > >> For example when an application wants to implement its own functions > >> that are named thrd_*(), for example? > > Yes. The realistic example is the code written to C99/SUSv4 conformance > > that happens to define thrd_. > > > > It might be that easiest solution is to put the functions into > > separate library, besides defining them weak. > > I thought the canonical solution here was to say > > #if POSIX_VISIBLE >= 201201 > > #endif > > Except this isn't posix. :( In this case it's #if __ISO_C_VISIBLE >= x (with the appropriate changes in for the new standard). That deals with visibility issues in the compiler. The weak symbols deal with visibility issues in the linker. From owner-freebsd-arch@FreeBSD.ORG Sat Dec 17 11:31:57 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 55A59106566B for ; Sat, 17 Dec 2011 11:31:57 +0000 (UTC) (envelope-from gljennjohn@googlemail.com) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id DF77C8FC13 for ; Sat, 17 Dec 2011 11:31:56 +0000 (UTC) Received: by eaaf13 with SMTP id f13so5047918eaa.13 for ; Sat, 17 Dec 2011 03:31:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=date:from:to:subject:message-id:in-reply-to:references:reply-to :x-mailer:mime-version:content-type:content-transfer-encoding; bh=CQFfUT/ksOd9Mr3SzmnCRDWtLSfGG+odMnAeDPxD5mM=; b=FliIjB5Z2dfLNPK/bgCTuAyjtMKBQIdeyQdqubtysL1xgKys7FWZZwOohJJO3lqJRd ujD0MhIAgk20ZseLTy6G8mmjY696VtxjBJzpzKEAszn4sT0qRqn6fKRKWn1Ioi9V6cmx tMC+kMyVCviD98ZEb6jtdEGOM2rOIQhGZ4YsY= Received: by 10.213.21.147 with SMTP id j19mr1100978ebb.149.1324120102875; Sat, 17 Dec 2011 03:08:22 -0800 (PST) Received: from ernst.jennejohn.org (p578E2767.dip.t-dialin.net. [87.142.39.103]) by mx.google.com with ESMTPS id z54sm12741322eeh.5.2011.12.17.03.08.21 (version=TLSv1/SSLv3 cipher=OTHER); Sat, 17 Dec 2011 03:08:22 -0800 (PST) Date: Sat, 17 Dec 2011 12:08:20 +0100 From: Gary Jennejohn To: freebsd-arch@freebsd.org Message-ID: <20111217120820.5c2d0ee4@ernst.jennejohn.org> In-Reply-To: <201112161559.36428.jhb@freebsd.org> References: <201112161559.36428.jhb@freebsd.org> X-Mailer: Claws Mail 3.7.10 (GTK+ 2.24.6; amd64-portbld-freebsd10.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: TASK_INITIALIZER() X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: gljennjohn@googlemail.com List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Dec 2011 11:31:57 -0000 On Fri, 16 Dec 2011 15:59:36 -0500 John Baldwin wrote: > Any objection to adding a macro to make it easy to statically initialize task > structures (similar to the initializer macros in )? This allows > global tasks to be statically initalized without requiring a dedicated > SYSINIT() routine. > > Index: share/man/man9/Makefile > =================================================================== > --- share/man/man9/Makefile (revision 228534) > +++ share/man/man9/Makefile (working copy) > @@ -1250,6 +1250,7 @@ > sysctl_ctx_init.9 sysctl_ctx_free.9 > MLINKS+=SYSINIT.9 SYSUNINIT.9 > MLINKS+=taskqueue.9 TASK_INIT.9 \ > + taskqueue.9 TASK_INITIALIZER.9 \ > taskqueue.9 taskqueue_cancel.9 \ > taskqueue.9 taskqueue_create.9 \ > taskqueue.9 taskqueue_create_fast.9 \ > Index: share/man/man9/taskqueue.9 > =================================================================== > --- share/man/man9/taskqueue.9 (revision 228534) > +++ share/man/man9/taskqueue.9 (working copy) > @@ -80,6 +80,7 @@ > .Ft void > .Fn taskqueue_run "struct taskqueue *queue" > .Fn TASK_INIT "struct task *task" "int priority" "task_fn_t func" "void > *context" > +.Fn TASK_INITIALIZER "int priority" "task_fn_t func" "void *context" > .Fn TASKQUEUE_DECLARE "name" > .Fn TASKQUEUE_DEFINE "name" "taskqueue_enqueue_fn enqueue" "void *context" > "init" > .Fn TASKQUEUE_FAST_DEFINE "name" "taskqueue_enqueue_fn enqueue" "void > *context" "init" > @@ -243,9 +244,14 @@ > is provided to initialise a > .Va task > structure. > +The > +.Fn TASK_INITIALIZER > +macro generates an initializer for a task structure. > A macro > .Fn TIMEOUT_TASK_INIT "queue" "timeout_task" "priority" "func" "context" > -initializes the timeout_task structure. > +initializes the > +.Va timeout_task > +structure. > The values of > .Va priority , > .Va func , > Index: sys/sys/taskqueue.h > =================================================================== > --- sys/sys/taskqueue.h (revision 228534) > +++ sys/sys/taskqueue.h (working copy) > @@ -77,6 +77,12 @@ > void taskqueue_unblock(struct taskqueue *queue); > int taskqueue_member(struct taskqueue *queue, struct thread *td); > > +#define TASK_INITIALIZER(priority, func, context) \ > + { .ta_pending = 0, \ > + .ta_priority = (priority), \ > + .ta_func = (func), \ > + .ta_context = (context) } > + > /* > * Functions for dedicated thread taskqueues > */ > Seems like an excellent idea. Are you also planning to replace the existing SYSINIT() entries? -- Gary Jennejohn From owner-freebsd-arch@FreeBSD.ORG Sat Dec 17 21:22:01 2011 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E4126106567A; Sat, 17 Dec 2011 21:22:01 +0000 (UTC) (envelope-from phk@phk.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.freebsd.org (Postfix) with ESMTP id A03678FC12; Sat, 17 Dec 2011 21:22:01 +0000 (UTC) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.61.3]) by phk.freebsd.dk (Postfix) with ESMTP id 8377F5DAA; Sat, 17 Dec 2011 21:02:17 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.14.5/8.14.5) with ESMTP id pBHL2HRE085478; Sat, 17 Dec 2011 21:02:17 GMT (envelope-from phk@phk.freebsd.dk) To: Kostik Belousov From: "Poul-Henning Kamp" In-Reply-To: Your message of "Sat, 17 Dec 2011 00:31:26 +0200." <20111216223126.GX50300@deviant.kiev.zoral.com.ua> Content-Type: text/plain; charset=ISO-8859-1 Date: Sat, 17 Dec 2011 21:02:17 +0000 Message-ID: <85477.1324155737@critter.freebsd.dk> Cc: arch@freebsd.org, Ed Schouten , threads@freebsd.org Subject: Re: [Patch] C1X threading support X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Dec 2011 21:22:02 -0000 In message <20111216223126.GX50300@deviant.kiev.zoral.com.ua>, Kostik Belousov writes: >> http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf >BTW, it looks not very useful to add a bunch of threading functions >without at least trying to specify the memory model. I have never fully understood what goal these sequential obesity-binges from ISO-C serve. Structure packing ? Nahh, nobody uses that. Big/Little Endian API ? Naah, nobody moves binary data between computers. Ohhh, but I know: Lets make a rival to the POSIX threads, we can do it much better and slightly incompatible, big market there I'm sure. What ? A "assert mutex is held" facility ? Why would you want that ? Just write perfect code to begin with! Bang! Bang! Bang! &c &c... -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence.