From owner-freebsd-arch@FreeBSD.ORG Mon Jul 29 14:25:50 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 29707A96; Mon, 29 Jul 2013 14:25:50 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id CD6D329B8; Mon, 29 Jul 2013 14:25:48 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA06580; Mon, 29 Jul 2013 17:25:40 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1V3oOJ-00072l-ND; Mon, 29 Jul 2013 17:25:39 +0300 Message-ID: <51F67B2A.3040302@FreeBSD.org> Date: Mon, 29 Jul 2013 17:24:42 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130708 Thunderbird/17.0.7 MIME-Version: 1.0 To: freebsd-arch@FreeBSD.org Subject: translate INVARIANTS to DEBUG for code from OpenSolaris X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jul 2013 14:25:50 -0000 [zfs-devel@, fs@, dtrace@ are Bcc-ed] In OpenSolaris and its descendants DEBUG is used in a fashion similar to our INVARIANTS. For example, ASSERT macros are enabled by it. In our kernel code DEBUG has a different meaning and enables far too verbose or far too obscure code and, as such, it is very rarely enabled. The idea of a change that I would like to propose is to translate INVARIANTS kernel option into DEBUG for the files that originated from OpenSolaris (and hopefully only for them). The change: opensolaris code: translate INVARIANTS to DEBUG and ZFS_DEBUG do this by forcing inclusion of sys/cddl/compat/opensolaris/sys/debug_compat.h via -include option into all source files from OpenSolaris. Note that this -include option must always be after -include opt_global.h. Additionally, remove forced definition of DEBUG for some modules and fix their build without DEBUG. Also, meaning of DEBUG was overloaded to enable WITNESS support for some OpenSolaris (primarily ZFS) locks. Now this overloading is removed and that use of DEBUG is replaced with a new option OPENSOLARIS_WITNESS. http://people.freebsd.org/~avg/osol-invariants-debug.diff I would like to ask for your feedback on the soundness of the whole idea. Also on the name, location and style of inclusion for sys/cddl/compat/opensolaris/sys/debug_compat.h. And on any other details of the proposed change. Testing is also welcome, of course. Thank you very much. -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Mon Jul 29 18:25:24 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id EC8527E8; Mon, 29 Jul 2013 18:25:23 +0000 (UTC) (envelope-from prvs=1922be3455=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6A5882728; Mon, 29 Jul 2013 18:25:23 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50005209219.msg; Mon, 29 Jul 2013 19:25:20 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Mon, 29 Jul 2013 19:25:20 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=1922be3455=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: <69CEF0CC3C8C4B9BBA8E3ACACB18DC2B@multiplay.co.uk> From: "Steven Hartland" To: "Andriy Gapon" , References: <51F67B2A.3040302@FreeBSD.org> Subject: Re: translate INVARIANTS to DEBUG for code from OpenSolaris Date: Mon, 29 Jul 2013 19:25:54 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jul 2013 18:25:24 -0000 ----- Original Message ----- From: "Andriy Gapon" > [zfs-devel@, fs@, dtrace@ are Bcc-ed] > > In OpenSolaris and its descendants DEBUG is used in a fashion similar to our > INVARIANTS. For example, ASSERT macros are enabled by it. > In our kernel code DEBUG has a different meaning and enables far too verbose or > far too obscure code and, as such, it is very rarely enabled. > > The idea of a change that I would like to propose is to translate INVARIANTS > kernel option into DEBUG for the files that originated from OpenSolaris (and > hopefully only for them). > > The change: > opensolaris code: translate INVARIANTS to DEBUG and ZFS_DEBUG > > do this by forcing inclusion of > sys/cddl/compat/opensolaris/sys/debug_compat.h > via -include option into all source files from OpenSolaris. > Note that this -include option must always be after -include opt_global.h. > > Additionally, remove forced definition of DEBUG for some modules and fix > their build without DEBUG. > > Also, meaning of DEBUG was overloaded to enable WITNESS support for some > OpenSolaris (primarily ZFS) locks. Now this overloading is removed and > that use of DEBUG is replaced with a new option OPENSOLARIS_WITNESS. > > http://people.freebsd.org/~avg/osol-invariants-debug.diff > > I would like to ask for your feedback on the soundness of the whole idea. > Also on the name, location and style of inclusion for > sys/cddl/compat/opensolaris/sys/debug_compat.h. > And on any other details of the proposed change. > > Testing is also welcome, of course. > > Thank you very much. I'm not sure you need #define ZFS_DEBUG as its already a dependency on DEBUG see: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_debug.h Should this connection be broken and INVARIANTS only trigger DEBUG by default? One potential reason for this distinction is that ZFS_DEBUG brings with it some potentially quite heavy weight validation such as dnode_verify. If so I have small list of additional fixes here that where required when I tested enabling ASSERTS checking without ZFS_DEBUG. What do others think, should INVARIANTS trigger ZFS_DEBUG or should that be kept seperate? Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-freebsd-arch@FreeBSD.ORG Mon Jul 29 18:36:27 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id AA4ADBD1 for ; Mon, 29 Jul 2013 18:36:27 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id D614927CF for ; Mon, 29 Jul 2013 18:36:26 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id VAA08937; Mon, 29 Jul 2013 21:36:24 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1V3sIy-0007NI-44; Mon, 29 Jul 2013 21:36:24 +0300 Message-ID: <51F6B5EF.5020407@FreeBSD.org> Date: Mon, 29 Jul 2013 21:35:27 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130708 Thunderbird/17.0.7 MIME-Version: 1.0 To: Steven Hartland Subject: Re: translate INVARIANTS to DEBUG for code from OpenSolaris References: <51F67B2A.3040302@FreeBSD.org> <69CEF0CC3C8C4B9BBA8E3ACACB18DC2B@multiplay.co.uk> In-Reply-To: <69CEF0CC3C8C4B9BBA8E3ACACB18DC2B@multiplay.co.uk> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jul 2013 18:36:27 -0000 on 29/07/2013 21:25 Steven Hartland said the following: > ----- Original Message ----- From: "Andriy Gapon" > >> [zfs-devel@, fs@, dtrace@ are Bcc-ed] >> >> In OpenSolaris and its descendants DEBUG is used in a fashion similar to our >> INVARIANTS. For example, ASSERT macros are enabled by it. >> In our kernel code DEBUG has a different meaning and enables far too verbose or >> far too obscure code and, as such, it is very rarely enabled. >> >> The idea of a change that I would like to propose is to translate INVARIANTS >> kernel option into DEBUG for the files that originated from OpenSolaris (and >> hopefully only for them). >> >> The change: >> opensolaris code: translate INVARIANTS to DEBUG and ZFS_DEBUG >> >> do this by forcing inclusion of >> sys/cddl/compat/opensolaris/sys/debug_compat.h >> via -include option into all source files from OpenSolaris. >> Note that this -include option must always be after -include opt_global.h. >> >> Additionally, remove forced definition of DEBUG for some modules and fix >> their build without DEBUG. >> >> Also, meaning of DEBUG was overloaded to enable WITNESS support for some >> OpenSolaris (primarily ZFS) locks. Now this overloading is removed and >> that use of DEBUG is replaced with a new option OPENSOLARIS_WITNESS. >> >> http://people.freebsd.org/~avg/osol-invariants-debug.diff >> >> I would like to ask for your feedback on the soundness of the whole idea. >> Also on the name, location and style of inclusion for >> sys/cddl/compat/opensolaris/sys/debug_compat.h. >> And on any other details of the proposed change. >> >> Testing is also welcome, of course. >> >> Thank you very much. > > I'm not sure you need #define ZFS_DEBUG as its already a dependency on > DEBUG see: > sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_debug.h Good point. Thank you. > Should this connection be broken and INVARIANTS only trigger DEBUG by > default? One potential reason for this distinction is that ZFS_DEBUG brings > with it some potentially quite heavy weight validation such as dnode_verify. > > If so I have small list of additional fixes here that where required when > I tested enabling ASSERTS checking without ZFS_DEBUG. > > What do others think, should INVARIANTS trigger ZFS_DEBUG or should that > be kept seperate? I would prefer to follow what upstream does in this respect. If DEBUG implies ZFS_DEBUG upstream, then I think that we should do the same for INVARIANTS -> DEBUG -> ZFS_DEBUG. Unless there are strong reasons to do otherwise. -- Andriy Gapon From owner-freebsd-arch@FreeBSD.ORG Mon Jul 29 18:47:16 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 6B316F40 for ; Mon, 29 Jul 2013 18:47:16 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-pb0-x235.google.com (mail-pb0-x235.google.com [IPv6:2607:f8b0:400e:c01::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3CA8C2846 for ; Mon, 29 Jul 2013 18:47:16 +0000 (UTC) Received: by mail-pb0-f53.google.com with SMTP id up15so4326512pbc.26 for ; Mon, 29 Jul 2013 11:47:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=S5o0Gj1KW/f2aa0fu1EVPmN7mktX8qKyghsgbpGAo8Y=; b=WNwh4VugdgHzBz4RdOc+C6hOKTrIGg+zJmTYy4K59MBFdK08Y18ChRs/h2LSeohB04 3yt3OcmcDeMg1fNLemMLlBH39fBhx+tqofHTvX6NSztbcUsnSpyPs6kP1a05JjDGkHCV jz21uv/WLLdfBUKTBsJZMdHxO5vpdC6C6iDes= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:x-gm-message-state; bh=S5o0Gj1KW/f2aa0fu1EVPmN7mktX8qKyghsgbpGAo8Y=; b=Ylf6Ux3of//2hknxNQbeXzjdS/FTfymYmclPBKwDQp051GcZK2/SoUPDNNy0wrIJ8b dQvcRenTiEXAXDSwYVD6+jup2qR7rnzwJ2XW8IIr46dJaVaAq1ATvJbaT16tYYX7Fscr K+Shvi7r97V+9T3Ml+BONQN4eAwcWP6SON6lX0KIyJvtiMgTnxbtXNkU8PodYcxvd/9f +ucocLjX6n0FAHErz4/qaEvu7Kq8TKEYIHCCwAqdCe5QhnEl2ZWRxeZb4uYDX+MTIYW4 ap3bIJ7q8wr5VHsaKtIClS6Wl46j65V0vP9prz30YDKB8pILThARMkU7YO5WO/Xw/2t4 +U3Q== MIME-Version: 1.0 X-Received: by 10.66.246.133 with SMTP id xw5mr69261373pac.114.1375123635826; Mon, 29 Jul 2013 11:47:15 -0700 (PDT) Received: by 10.70.132.66 with HTTP; Mon, 29 Jul 2013 11:47:15 -0700 (PDT) In-Reply-To: <51F67B2A.3040302@FreeBSD.org> References: <51F67B2A.3040302@FreeBSD.org> Date: Mon, 29 Jul 2013 11:47:15 -0700 Message-ID: Subject: Re: translate INVARIANTS to DEBUG for code from OpenSolaris From: Matthew Ahrens To: Andriy Gapon X-Gm-Message-State: ALoCoQn077IPMpapGnyIJEUQ++KQXYoKZ963UN1fHEvnG6ip23D6FdiKBzV9UbSd3P02deQ7wiH6 Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jul 2013 18:47:16 -0000 >From my perspective (working primarily with the OpenSolaris / illumos code), this approach makes sense. --matt On Mon, Jul 29, 2013 at 7:24 AM, Andriy Gapon wrote: > [zfs-devel@, fs@, dtrace@ are Bcc-ed] > > In OpenSolaris and its descendants DEBUG is used in a fashion similar to > our > INVARIANTS. For example, ASSERT macros are enabled by it. > In our kernel code DEBUG has a different meaning and enables far too > verbose or > far too obscure code and, as such, it is very rarely enabled. > > The idea of a change that I would like to propose is to translate > INVARIANTS > kernel option into DEBUG for the files that originated from OpenSolaris > (and > hopefully only for them). > > The change: > opensolaris code: translate INVARIANTS to DEBUG and ZFS_DEBUG > > do this by forcing inclusion of > sys/cddl/compat/opensolaris/sys/debug_compat.h > via -include option into all source files from OpenSolaris. > Note that this -include option must always be after -include > opt_global.h. > > Additionally, remove forced definition of DEBUG for some modules and > fix > their build without DEBUG. > > Also, meaning of DEBUG was overloaded to enable WITNESS support for > some > OpenSolaris (primarily ZFS) locks. Now this overloading is removed and > that use of DEBUG is replaced with a new option OPENSOLARIS_WITNESS. > > http://people.freebsd.org/~avg/osol-invariants-debug.diff > > I would like to ask for your feedback on the soundness of the whole idea. > Also on the name, location and style of inclusion for > sys/cddl/compat/opensolaris/sys/debug_compat.h. > And on any other details of the proposed change. > > Testing is also welcome, of course. > > Thank you very much. > -- > Andriy Gapon > > _______________________________________________ > freebsd-dtrace@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-dtrace > To unsubscribe, send any mail to "freebsd-dtrace-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Tue Jul 30 02:39:14 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id B3E819E3; Tue, 30 Jul 2013 02:39:14 +0000 (UTC) (envelope-from gibbs@FreeBSD.org) Received: from aslan.scsiguy.com (aslan.scsiguy.com [70.89.174.89]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6E48C2DA3; Tue, 30 Jul 2013 02:39:14 +0000 (UTC) Received: from [192.168.0.99] (macbook.scsiguy.com [192.168.0.99]) (authenticated bits=0) by aslan.scsiguy.com (8.14.7/8.14.5) with ESMTP id r6U2dCOi034033 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Tue, 30 Jul 2013 02:39:12 GMT (envelope-from gibbs@FreeBSD.org) Subject: Re: translate INVARIANTS to DEBUG for code from OpenSolaris Mime-Version: 1.0 (Mac OS X Mail 6.5 \(1508\)) Content-Type: text/plain; charset=us-ascii From: "Justin T. Gibbs" X-Priority: 3 In-Reply-To: <69CEF0CC3C8C4B9BBA8E3ACACB18DC2B@multiplay.co.uk> Date: Mon, 29 Jul 2013 20:39:12 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <751C9B66-0476-4E3B-AF07-FAFF2F54FEBB@FreeBSD.org> References: <51F67B2A.3040302@FreeBSD.org> <69CEF0CC3C8C4B9BBA8E3ACACB18DC2B@multiplay.co.uk> To: Steven Hartland X-Mailer: Apple Mail (2.1508) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (aslan.scsiguy.com [70.89.174.89]); Tue, 30 Jul 2013 02:39:12 +0000 (UTC) Cc: Andriy Gapon , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jul 2013 02:39:14 -0000 On Jul 29, 2013, at 12:25 PM, Steven Hartland = wrote: > ----- Original Message ----- From: "Andriy Gapon" >=20 >> [zfs-devel@, fs@, dtrace@ are Bcc-ed] >> In OpenSolaris and its descendants DEBUG is used in a fashion similar = to our >> INVARIANTS. For example, ASSERT macros are enabled by it. >> In our kernel code DEBUG has a different meaning and enables far too = verbose or >> far too obscure code and, as such, it is very rarely enabled. >> The idea of a change that I would like to propose is to translate = INVARIANTS >> kernel option into DEBUG for the files that originated from = OpenSolaris (and >> hopefully only for them). >> The change: >> opensolaris code: translate INVARIANTS to DEBUG and ZFS_DEBUG >> do this by forcing inclusion of >> sys/cddl/compat/opensolaris/sys/debug_compat.h >> via -include option into all source files from OpenSolaris. >> Note that this -include option must always be after -include = opt_global.h. >> Additionally, remove forced definition of DEBUG for some modules = and fix >> their build without DEBUG. >> Also, meaning of DEBUG was overloaded to enable WITNESS support for = some >> OpenSolaris (primarily ZFS) locks. Now this overloading is removed = and >> that use of DEBUG is replaced with a new option = OPENSOLARIS_WITNESS. >> http://people.freebsd.org/~avg/osol-invariants-debug.diff >> I would like to ask for your feedback on the soundness of the whole = idea. >> Also on the name, location and style of inclusion for >> sys/cddl/compat/opensolaris/sys/debug_compat.h. >> And on any other details of the proposed change. >> Testing is also welcome, of course. >> Thank you very much. >=20 > I'm not sure you need #define ZFS_DEBUG as its already a dependency on > DEBUG see: > sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zfs_debug.h >=20 > Should this connection be broken and INVARIANTS only trigger DEBUG by > default? One potential reason for this distinction is that ZFS_DEBUG = brings > with it some potentially quite heavy weight validation such as = dnode_verify. I would prefer to have INVARIANTS enable all of the ZFS invariant code, = not just the asserts. Folks already know that INVARIANTS means (sometimes quite significantly) lower performance. This is a small price to pay = for catching defects before a release. -- Justin= From owner-freebsd-arch@FreeBSD.ORG Tue Jul 30 19:58:02 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from hammer.pct.niksun.com (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by hub.freebsd.org (Postfix) with ESMTP id E67222C3; Tue, 30 Jul 2013 19:58:01 +0000 (UTC) (envelope-from jkim@FreeBSD.org) Message-ID: <51F81A74.4030009@FreeBSD.org> Date: Tue, 30 Jul 2013 15:56:36 -0400 From: Jung-uk Kim User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130715 Thunderbird/17.0.7 MIME-Version: 1.0 To: freebsd-arch@FreeBSD.org Subject: [PATCH] Atomic swap and test-and-set X-Enigmail-Version: 1.5.1 Content-Type: multipart/mixed; boundary="------------050506090501030302020606" Cc: kib@FreeBSD.org, dumbbell@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jul 2013 19:58:02 -0000 This is a multi-part message in MIME format. --------------050506090501030302020606 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The attached patch implements atomic_swap_() and atomic_testandset_() for x86. is int or long ATM. The semantics of each operation (in atomic(9) style): atomic_swap(p, v) r = *p; *p = v; return (r); atomic_testandset(p, s) v = ()1 << s % (sizeof() * NBBY); r = (*p & v) != 0; *p |= v; return (r); Note atomic_readandclear_() is now obsolete and implemented as a macro around atomic_swap_() where the new value v is 0. Please let me know if you have any objection. Thanks! Jung-uk Kim * Note 1: This patch is also available from here. http://people.freebsd.org/~jkim/atomic.diff * Note 2: These atomic ops are useful for porting Linux drivers and it may simplify diffs, e.g., drm2 for AMD GPUs project. Currently, I am using the following wrapper: http://people.freebsd.org/~jkim/drm_atomic.h A full patch against kms-radeon tree is here: http://people.freebsd.org/~jkim/drm_atomic.diff -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (FreeBSD) iQEcBAEBAgAGBQJR+Bp0AAoJECXpabHZMqHOpIsIAMqEjNehGRe7q7B7CPC2cqdY aEWiRlKXLa0K3Vi2f9asYFlbhIm+PstV8Juhw2ijT/P3i55xoQTc4DqP6H4ISUgK pKM8TLAFJWlItf3o8HV/q7Tns/iaqb8cnB0D2jLUtheeM8UTXEim+XOSWILL+csR zuXGDSDGC0vTxip00ocPUdtdG/Y5Sg/Eq+5CZYp58DSoEBDfuKhGJc3xrC6QrMGx zwBkmAp5escfFkyE1WLI99tJevrQZpqJViD+66dDcLW56w4Aqpqib69sMMEs/meJ KrwKRwt9+kgvwEFiDGXzDsjVfcTky9Cb8zn3jbBrGxtNxd4aHgCNY1xeG98kHaM= =6LmO -----END PGP SIGNATURE----- --------------050506090501030302020606 Content-Type: text/x-patch; name="atomic.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="atomic.diff" Index: sys/amd64/include/atomic.h =================================================================== --- sys/amd64/include/atomic.h (revision 253812) +++ sys/amd64/include/atomic.h (working copy) @@ -54,6 +54,7 @@ * atomic_clear_int(P, V) (*(u_int *)(P) &= ~(V)) * atomic_add_int(P, V) (*(u_int *)(P) += (V)) * atomic_subtract_int(P, V) (*(u_int *)(P) -= (V)) + * atomic_swap_int(P, V) (return (*(u_int *)(P)); *(u_int *)(P) = (V);) * atomic_readandclear_int(P) (return (*(u_int *)(P)); *(u_int *)(P) = 0;) * * atomic_set_long(P, V) (*(u_long *)(P) |= (V)) @@ -60,6 +61,7 @@ * atomic_clear_long(P, V) (*(u_long *)(P) &= ~(V)) * atomic_add_long(P, V) (*(u_long *)(P) += (V)) * atomic_subtract_long(P, V) (*(u_long *)(P) -= (V)) + * atomic_swap_long(P, V) (return (*(u_long *)(P)); *(u_long *)(P) = (V);) * atomic_readandclear_long(P) (return (*(u_long *)(P)); *(u_long *)(P) = 0;) */ @@ -80,6 +82,8 @@ int atomic_cmpset_int(volatile u_int *dst, u_int e int atomic_cmpset_long(volatile u_long *dst, u_long expect, u_long src); u_int atomic_fetchadd_int(volatile u_int *p, u_int v); u_long atomic_fetchadd_long(volatile u_long *p, u_long v); +int atomic_testandset_int(volatile u_int *p, int v); +int atomic_testandset_long(volatile u_long *p, int v); #define ATOMIC_LOAD(TYPE, LOP) \ u_##TYPE atomic_load_acq_##TYPE(volatile u_##TYPE *p) @@ -211,6 +215,44 @@ atomic_fetchadd_long(volatile u_long *p, u_long v) return (v); } +static __inline int +atomic_testandset_int(volatile u_int *p, int v) +{ + u_char res; + + __asm __volatile( + " " MPLOCKED " " + " btsl %2, %1 ; " + " setc %0 ; " + "# atomic_testandset_int" + : "=r" (res), /* 0 */ + "=m" (*p) /* 1 */ + : "r" (v), /* 2 */ + "m" (*p) + : "cc"); + + return (res); +} + +static __inline int +atomic_testandset_long(volatile u_long *p, int v) +{ + u_char res; + + __asm __volatile( + " " MPLOCKED " " + " btsq %2, %1 ; " + " setc %0 ; " + "# atomic_testandset_long" + : "=r" (res), /* 0 */ + "=m" (*p) /* 1 */ + : "r" ((u_long)v), /* 2 */ + "m" (*p) + : "cc"); + + return (res); +} + /* * We assume that a = b will do atomic loads and stores. Due to the * IA32 memory model, a simple store guarantees release semantics. @@ -303,45 +345,41 @@ ATOMIC_STORE(long); #ifndef WANT_FUNCTIONS -/* Read the current value and store a zero in the destination. */ +/* Read the current value and store a new value in the destination. */ #ifdef __GNUCLIKE_ASM static __inline u_int -atomic_readandclear_int(volatile u_int *addr) +atomic_swap_int(volatile u_int *p, u_int v) { - u_int res; - res = 0; __asm __volatile( - " xchgl %1,%0 ; " - "# atomic_readandclear_int" - : "+r" (res), /* 0 */ - "=m" (*addr) /* 1 */ - : "m" (*addr)); + " xchgl %1, %0 ; " + "# atomic_swap_int" + : "+r" (v), /* 0 */ + "=m" (*p) /* 1 */ + : "m" (*p)); - return (res); + return (v); } static __inline u_long -atomic_readandclear_long(volatile u_long *addr) +atomic_swap_long(volatile u_long *p, u_long v) { - u_long res; - res = 0; __asm __volatile( - " xchgq %1,%0 ; " - "# atomic_readandclear_long" - : "+r" (res), /* 0 */ - "=m" (*addr) /* 1 */ - : "m" (*addr)); + " xchgq %1, %0 ; " + "# atomic_swap_long" + : "+r" (v), /* 0 */ + "=m" (*p) /* 1 */ + : "m" (*p)); - return (res); + return (v); } #else /* !__GNUCLIKE_ASM */ -u_int atomic_readandclear_int(volatile u_int *addr); -u_long atomic_readandclear_long(volatile u_long *addr); +u_int atomic_swap_int(volatile u_int *, u_int); +u_long atomic_swap_long(volatile u_long *, u_long); #endif /* __GNUCLIKE_ASM */ @@ -385,6 +423,9 @@ static __inline u_long #define atomic_cmpset_acq_long atomic_cmpset_long #define atomic_cmpset_rel_long atomic_cmpset_long +#define atomic_readandclear_int(p) atomic_swap_int(p, 0) +#define atomic_readandclear_long(p) atomic_swap_long(p, 0) + /* Operations on 8-bit bytes. */ #define atomic_set_8 atomic_set_char #define atomic_set_acq_8 atomic_set_acq_char @@ -435,8 +476,10 @@ static __inline u_long #define atomic_cmpset_32 atomic_cmpset_int #define atomic_cmpset_acq_32 atomic_cmpset_acq_int #define atomic_cmpset_rel_32 atomic_cmpset_rel_int +#define atomic_swap_32 atomic_swap_int #define atomic_readandclear_32 atomic_readandclear_int #define atomic_fetchadd_32 atomic_fetchadd_int +#define atomic_testandset_32 atomic_testandset_int /* Operations on 64-bit quad words. */ #define atomic_set_64 atomic_set_long @@ -456,7 +499,9 @@ static __inline u_long #define atomic_cmpset_64 atomic_cmpset_long #define atomic_cmpset_acq_64 atomic_cmpset_acq_long #define atomic_cmpset_rel_64 atomic_cmpset_rel_long +#define atomic_swap_64 atomic_swap_long #define atomic_readandclear_64 atomic_readandclear_long +#define atomic_testandset_64 atomic_testandset_long /* Operations on pointers. */ #define atomic_set_ptr atomic_set_long @@ -476,6 +521,7 @@ static __inline u_long #define atomic_cmpset_ptr atomic_cmpset_long #define atomic_cmpset_acq_ptr atomic_cmpset_acq_long #define atomic_cmpset_rel_ptr atomic_cmpset_rel_long +#define atomic_swap_ptr atomic_swap_long #define atomic_readandclear_ptr atomic_readandclear_long #endif /* !WANT_FUNCTIONS */ Index: sys/i386/include/atomic.h =================================================================== --- sys/i386/include/atomic.h (revision 253812) +++ sys/i386/include/atomic.h (working copy) @@ -54,6 +54,7 @@ * atomic_clear_int(P, V) (*(u_int *)(P) &= ~(V)) * atomic_add_int(P, V) (*(u_int *)(P) += (V)) * atomic_subtract_int(P, V) (*(u_int *)(P) -= (V)) + * atomic_swap_int(P, V) (return (*(u_int *)(P)); *(u_int *)(P) = (V);) * atomic_readandclear_int(P) (return (*(u_int *)(P)); *(u_int *)(P) = 0;) * * atomic_set_long(P, V) (*(u_long *)(P) |= (V)) @@ -60,6 +61,7 @@ * atomic_clear_long(P, V) (*(u_long *)(P) &= ~(V)) * atomic_add_long(P, V) (*(u_long *)(P) += (V)) * atomic_subtract_long(P, V) (*(u_long *)(P) -= (V)) + * atomic_swap_long(P, V) (return (*(u_long *)(P)); *(u_long *)(P) = (V);) * atomic_readandclear_long(P) (return (*(u_long *)(P)); *(u_long *)(P) = 0;) */ @@ -78,6 +80,7 @@ void atomic_##NAME##_barr_##TYPE(volatile u_##TYPE int atomic_cmpset_int(volatile u_int *dst, u_int expect, u_int src); u_int atomic_fetchadd_int(volatile u_int *p, u_int v); +int atomic_testandset_int(volatile u_int *p, int v); #define ATOMIC_LOAD(TYPE, LOP) \ u_##TYPE atomic_load_acq_##TYPE(volatile u_##TYPE *p) @@ -281,6 +284,25 @@ atomic_fetchadd_int(volatile u_int *p, u_int v) return (v); } +static __inline int +atomic_testandset_int(volatile u_int *p, int v) +{ + u_char res; + + __asm __volatile( + " " MPLOCKED " " + " btsl %2, %1 ; " + " setc %0 ; " + "# atomic_testandset_int" + : "=r" (res), /* 0 */ + "=m" (*p) /* 1 */ + : "r" (v), /* 2 */ + "m" (*p) + : "cc"); + + return (res); +} + /* * We assume that a = b will do atomic loads and stores. Due to the * IA32 memory model, a simple store guarantees release semantics. @@ -393,45 +415,41 @@ atomic_fetchadd_long(volatile u_long *p, u_long v) return (atomic_fetchadd_int((volatile u_int *)p, (u_int)v)); } -/* Read the current value and store a zero in the destination. */ +static __inline int +atomic_testandset_long(volatile u_long *p, int v) +{ + + return (atomic_testandset_int((volatile u_int *)p, v)); +} + +/* Read the current value and store a new value in the destination. */ #ifdef __GNUCLIKE_ASM static __inline u_int -atomic_readandclear_int(volatile u_int *addr) +atomic_swap_int(volatile u_int *p, u_int v) { - u_int res; - res = 0; __asm __volatile( - " xchgl %1,%0 ; " - "# atomic_readandclear_int" - : "+r" (res), /* 0 */ - "=m" (*addr) /* 1 */ - : "m" (*addr)); + " xchgl %1, %0 ; " + "# atomic_swap_int" + : "+r" (v), /* 0 */ + "=m" (*p) /* 1 */ + : "m" (*p)); - return (res); + return (v); } static __inline u_long -atomic_readandclear_long(volatile u_long *addr) +atomic_swap_long(volatile u_long *p, u_long v) { - u_long res; - res = 0; - __asm __volatile( - " xchgl %1,%0 ; " - "# atomic_readandclear_long" - : "+r" (res), /* 0 */ - "=m" (*addr) /* 1 */ - : "m" (*addr)); - - return (res); + return (atomic_swap_int((volatile u_int *)p, (u_int)v)); } #else /* !__GNUCLIKE_ASM */ -u_int atomic_readandclear_int(volatile u_int *addr); -u_long atomic_readandclear_long(volatile u_long *addr); +u_int atomic_swap_int(volatile u_int *, u_int); +u_long atomic_swap_long(volatile u_long *, u_long); #endif /* __GNUCLIKE_ASM */ @@ -475,6 +493,9 @@ static __inline u_long #define atomic_cmpset_acq_long atomic_cmpset_long #define atomic_cmpset_rel_long atomic_cmpset_long +#define atomic_readandclear_int(p) atomic_swap_int(p, 0) +#define atomic_readandclear_long(p) atomic_swap_long(p, 0) + /* Operations on 8-bit bytes. */ #define atomic_set_8 atomic_set_char #define atomic_set_acq_8 atomic_set_acq_char @@ -525,8 +546,10 @@ static __inline u_long #define atomic_cmpset_32 atomic_cmpset_int #define atomic_cmpset_acq_32 atomic_cmpset_acq_int #define atomic_cmpset_rel_32 atomic_cmpset_rel_int +#define atomic_swap_32 atomic_swap_int #define atomic_readandclear_32 atomic_readandclear_int #define atomic_fetchadd_32 atomic_fetchadd_int +#define atomic_testandset_32 atomic_testandset_int /* Operations on pointers. */ #define atomic_set_ptr(p, v) \ @@ -565,6 +588,8 @@ static __inline u_long #define atomic_cmpset_rel_ptr(dst, old, new) \ atomic_cmpset_rel_int((volatile u_int *)(dst), (u_int)(old), \ (u_int)(new)) +#define atomic_swap_ptr(p, v) \ + atomic_swap_int((volatile u_int *)(p), (u_int)(v)) #define atomic_readandclear_ptr(p) \ atomic_readandclear_int((volatile u_int *)(p)) --------------050506090501030302020606-- From owner-freebsd-arch@FreeBSD.ORG Wed Jul 31 09:10:11 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 76AA2DE8; Wed, 31 Jul 2013 09:10:11 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [46.4.40.135]) by mx1.freebsd.org (Postfix) with ESMTP id 39B0D23DC; Wed, 31 Jul 2013 09:10:10 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:2418:88c5:96ef:35b4]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPSA id EB9D54AC57; Wed, 31 Jul 2013 13:10:01 +0400 (MSK) Date: Wed, 31 Jul 2013 13:09:56 +0400 From: Lev Serebryakov Organization: FreeBSD X-Priority: 3 (Normal) Message-ID: <447183917.20130731130956@serebryakov.spb.ru> To: freebsd-geom@freebsd.org, freebsd-arch@FreeBSD.org Subject: [PROPOSAL] GEOM probing/tasting firewall MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 09:10:11 -0000 Hello, Freebsd-geom. After message to GEOM list by Rotate 13 I was "forced" to write this proposal. For first time this idea was formulated in Jabber talk with friend of mine, who uses FreeBSD for massive iSCSI hosting on ZVOLs. He has problems with tasting these ZVOLs, which contain different types of data (Windows disks, Linux disks, FreeBSD disks, etc). Here are label conflicts, strange messages about corrupted GPTs, etc. So, it looks like to have configurable way to prevent some GEOM tasting is good idea. I propose to have mechanism, modelled after ipfw, to filter all new GEOMs before passing it to class tasting mechanisms. It will be chain of (numbered) rules with format: rule ::= (enable | disable) ['taste'] ['by'] [['of'] ] consumer-spec ::= 'all' | conditions ::= ('and' )* condition ::= [not] ( 'class' ' | 'name' | 'path' ) mask ::= 'path' means "path in /dev hierarchy' here. Of course, default last (and only one, if user does nothing) rule must be "enable taste by all" Maybe, could be expanded to full-featured boolean expressions, with parenthesis and 'or' operator, it is possible, but ommitted in this proposal to save time (I don't want to write out full expression EBNF now). Every added rule should trigger spoiling and re-tasting of all providers, which are not opened by upper level (so, already mounted file systems will prevent GEOMs from disappearing even if new rule forbid such tasting). I could write prototype code for this proposal, it it is not rejected right now :) -- // Black Lion AKA Lev Serebryakov From owner-freebsd-arch@FreeBSD.ORG Wed Jul 31 11:58:24 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id CD724461; Wed, 31 Jul 2013 11:58:24 +0000 (UTC) (envelope-from rabgvzr@gmail.com) Received: from mail-qe0-x22e.google.com (mail-qe0-x22e.google.com [IPv6:2607:f8b0:400d:c02::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 6F7DD2D70; Wed, 31 Jul 2013 11:58:24 +0000 (UTC) Received: by mail-qe0-f46.google.com with SMTP id i11so302722qej.5 for ; Wed, 31 Jul 2013 04:58:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=dK5TLSXbSx7vZSGSu8yK3Dj4fXZ+oyRsZxEycztGB18=; b=i8nt93h14JE8KMALHZa7S73IF09hCrIz7I8UnA7KCaC5R1UiHJLrhvaUgo2Jnmmlk4 Fk+I61bpol72A0DZh5NVEqDslYr37I82IIYkfzGfSvn4JcPNFTm0YyvkxhI7i3bJo61e cEfhpxeBnFWPuZaD7rd1owB1DI5l2SNJC0fMDuIex/K5/ENF7gaRApLts857AiNM3RPm QshCxaf2uaVATDewAEzjkwue7xL/DucIKLuWbbGvDroq2u2xqxoAw4SeeCjFozBuV6/m j5b2dcVe92n8KL7U35JOtAH3cO4if57DU0xZ3HweC9tlQ2E1f4HH2wWTUDGCb0FES3gj Z75A== MIME-Version: 1.0 X-Received: by 10.224.75.196 with SMTP id z4mr11937228qaj.20.1375271903589; Wed, 31 Jul 2013 04:58:23 -0700 (PDT) Received: by 10.49.74.1 with HTTP; Wed, 31 Jul 2013 04:58:23 -0700 (PDT) In-Reply-To: <447183917.20130731130956@serebryakov.spb.ru> References: <447183917.20130731130956@serebryakov.spb.ru> Date: Wed, 31 Jul 2013 11:58:23 +0000 Message-ID: Subject: Re: [PROPOSAL] GEOM probing/tasting firewall From: Rotate 13 To: Lev Serebryakov Content-Type: text/plain; charset=UTF-8 Cc: freebsd-arch@freebsd.org, freebsd-geom@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 11:58:24 -0000 TL;DR: Please, make not the plugging in of arbitrary drive to cause kernel to stick ring0 tongue in it. lev@ proposal is very good from my userland perspective. Commentary: On Wed, 31 Jul 2013 13:09:56 +0400, Lev Serebryakov wrote: > condition ::= [not] ( 'class' ' | 'name' | 'path' ) > mask ::= > > 'path' means "path in /dev hierarchy' here. User comment: Need means to disable all taste on specific *physical* hotplug port. I understand this is problem involving different subsystems, and should be solved there; see recurrent threads on freebsd-fs etc. about "wiring down" SATA ports, and let's not start talking USB! But is not good when the emergency red port on front panel could be /dev/da0, /dev/da1... (or if eSATA: /dev/ada8, /dev/ada9... etc.). Class and name don't seem to solve this either? > Of course, default last (and only one, if user does nothing) rule must be > > "enable taste by all" User comment: Please make easiest to set "default deny" after startup! (Where to trigger this is userland problem, and good to solve in rc scripts.) Simplest and best, and solves above problem with device identification (from my perspective). I lack kernel skills to build this bigger-thing-than-bikeshed; so I will not talk more about the colour, except to say please "fail closed". But, just a few words on why lev@ idea is The Right Thing from my userland perspective. Taste system is very convenient, good idea. I like it when I build new system, or at boot without config headache, and the GEOM layers magically snap together like Lego bricks. But it needs ability to be turned off completely, when desired. I just looked for global "enable tasting" sysctl, which can be set to false; specific already exist in graid. And lev@ suggestion of the fine-grain control is best, if easy/not complicated to switch on global "default deny" and then open holes (i.e.: like firewall rules good, like devfs rules bad!). Think use cases: * USB flash drive found in parking lot. (Set rule to wire down specific USB port for untrusted media.) * Data recovery from known infected Windows hard disk (with unknown evilness), attached through eSATA (so no good to hack up umass(4)). * Bob locks terminal, walks to water cooler. Mallory found bug in some obscure GEOM, now has 30 seconds to stick in USB drive, pull it out, and slip away. * Sysadmin who is just a control freak. Isn't FreeBSD appeal to people who want system to do what it is told, *and nothing else*? I am not kernel hacker, and I know very smart people write GEOMs. But also, very smart people write VM subsystem (SA-13:06.mmap), ring-change codes (SA-12:04.sysret), and filesystems (see mount(2) manpage quoted at end here). (Not mentioning SA-11:05.unix, which was really stupid.) And there are lot of GEOMs in base system, not mentioning modular design encourages roll your own kld. Not all GEOMs are written by the well-known GEOM hackers who write work-of-art C code. My policies: (a) Nothing to run automatically after the carefully-configured boot+rc process, as much as possible. (b) Everything do from userland, which can be done from userland. (c) Trust system to not be "helpful" behind my back. ( Referenced above: mount(2) manpage, emphasis on last sentence wise words: BUGS Some of the error codes need translation to more obvious messages. Allowing untrusted users to mount arbitrary media, e.g. by enabling vfs.usermount, should not be considered safe. Most file systems in FreeBSD were not built to safeguard against malicious devices. Taste codes much more simple than filesystems, yes; but is g_foo tasting guaranteed absolutely bug-free? I don't care how simple it is: It runs in the kernel, and licks the USB stick from parking lot! Ewww. ) From owner-freebsd-arch@FreeBSD.ORG Wed Jul 31 14:31:28 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 3A1C6374; Wed, 31 Jul 2013 14:31:28 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id EFEFC2565; Wed, 31 Jul 2013 14:31:27 +0000 (UTC) Received: from dommail.onthenet.com.au (dommail.OntheNet.com.au [203.13.70.57]) by alto.onthenet.com.au (Postfix) with ESMTPS id 250B3125AC; Thu, 1 Aug 2013 00:31:20 +1000 (EST) Received: from Peter-Grehans-MacBook-Pro-2.local (c-67-161-27-37.hsd1.ca.comcast.net [67.161.27.37]) by dommail.onthenet.com.au (MOS 4.2.4-GA) with ESMTP id BNS43095 (AUTH peterg@ptree32.com.au); Thu, 1 Aug 2013 00:31:19 +1000 Message-ID: <51F91FAC.60905@freebsd.org> Date: Wed, 31 Jul 2013 07:31:08 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:17.0) Gecko/20130620 Thunderbird/17.0.7 MIME-Version: 1.0 To: lev@FreeBSD.org Subject: Re: [PROPOSAL] GEOM probing/tasting firewall References: <447183917.20130731130956@serebryakov.spb.ru> In-Reply-To: <447183917.20130731130956@serebryakov.spb.ru> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-arch@FreeBSD.org, freebsd-geom@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 14:31:28 -0000 > For first time this idea was formulated in Jabber talk with friend of > mine, who uses FreeBSD for massive iSCSI hosting on ZVOLs. He has problems > with tasting these ZVOLs, which contain different types of data (Windows > disks, Linux disks, FreeBSD disks, etc). Here are label conflicts, strange > messages about corrupted GPTs, etc. So, it looks like to have configurable > way to prevent some GEOM tasting is good idea. I'm all for this. bhyve has the exact same problem with unnecessary tasting of zvols and raw volumes being used by guest o/s's. later, Peter. From owner-freebsd-arch@FreeBSD.ORG Wed Jul 31 17:32:52 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id DB791C9E; Wed, 31 Jul 2013 17:32:52 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 550AE2EFA; Wed, 31 Jul 2013 17:32:52 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.7/8.14.7) with ESMTP id r6VHWlte080920; Wed, 31 Jul 2013 20:32:47 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua r6VHWlte080920 Received: (from kostik@localhost) by tom.home (8.14.7/8.14.7/Submit) id r6VHWl2J080919; Wed, 31 Jul 2013 20:32:47 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 31 Jul 2013 20:32:47 +0300 From: Konstantin Belousov To: Jung-uk Kim Subject: Re: [PATCH] Atomic swap and test-and-set Message-ID: <20130731173247.GK4972@kib.kiev.ua> References: <51F81A74.4030009@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="8BaRf9C5c8wiXWYy" Content-Disposition: inline In-Reply-To: <51F81A74.4030009@FreeBSD.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: dumbbell@FreeBSD.org, freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 17:32:52 -0000 --8BaRf9C5c8wiXWYy Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jul 30, 2013 at 03:56:36PM -0400, Jung-uk Kim wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 >=20 > The attached patch implements atomic_swap_() and > atomic_testandset_() for x86. is int or long ATM. >=20 > The semantics of each operation (in atomic(9) style): >=20 > atomic_swap(p, v) > r =3D *p; > *p =3D v; > return (r); >=20 > atomic_testandset(p, s) > v =3D ()1 << s % (sizeof() * NBBY); > r =3D (*p & v) !=3D 0; > *p |=3D v; > return (r); >=20 > Note atomic_readandclear_() is now obsolete and implemented as a > macro around atomic_swap_() where the new value v is 0. >=20 > Please let me know if you have any objection. I think that the atomic.diff is fine. You did not documented swap and testandset, which would be good to have in the manpage. I think that constraints for all added inlines contradict the explicit requirement in the gcc documentation, which states that rmw operands should be specified using the 'same location' output operand for the input operand (hopefully this can be parsed). It just happens so that for "m" compiler cannot do anything else then passing the address. The problem is not specific to the new functions, it seems to be present in other functions as well, e.g. cmpset. This probably can be postponed. --8BaRf9C5c8wiXWYy Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (FreeBSD) iQIcBAEBAgAGBQJR+Uo+AAoJEJDCuSvBvK1BNwIP/A6R23nUTIikAx0NRkjQzmlw eksqbyOIxOKDvWhiIjXxhFbxuU0y55usIiXS0mLQS/6NcaPkGmJWYouCvBbK5ION FbyXhjbN0yVWa9qYnCn3aipbMrmVw+yTNYkmerEBkv/i/0TGNRfaDPnKzZt29u/K dsl2aTeTllugh4Zce9JiiNgK6BiBFzsVSiAGFgXqEvbfsGM4waTisH9nS4ReeQyx ryteWLHjbCWHLJX/hhqSDjL/zYkD767XXlmzdCfLbD+FWZh4gxsgFz/LL3Y+ugwy ie292+xnDbCVNGz60W8zizuYMTHXiH9J2J51RY8wlJ0RcKfpmNyy2DMqLlMqeCr0 SgQ1/UhKVNDcsXlAm9l8qEgmjRi7p38IXEw4Xlz3mcBGaM8kv/gwAVJwKa4p9CK9 d8OzdBOaV7KKhQrt3defuYs2qCLJk/9U/dYjKpqD4ZXp13Q2XYFRbPzsU1Z+Mt1Y gRR1OSbBbsLe+aV0BysFlHIntS3+5TI1aV1OIZuAb/ofZ8OSuhegZUwkKT2Peq6n OqpfIs5EMuLFhYBGOuTHGFXXqzntCSR+pmr11r2QEDg8g/y7H/Pc0qmYdvBpUkbg DYwcyoUss+BozChMNflQOCd4/Tnjg8gV6Q7cEzhAKi2j+wxqsKWGHhDDR6EU3lUy R6Dzj7lIBH1jysnbGiOC =0S+E -----END PGP SIGNATURE----- --8BaRf9C5c8wiXWYy-- From owner-freebsd-arch@FreeBSD.ORG Wed Jul 31 20:03:16 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from hammer.pct.niksun.com (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by hub.freebsd.org (Postfix) with ESMTP id C639BD19; Wed, 31 Jul 2013 20:03:15 +0000 (UTC) (envelope-from jkim@FreeBSD.org) Message-ID: <51F96D2C.6030109@FreeBSD.org> Date: Wed, 31 Jul 2013 16:01:48 -0400 From: Jung-uk Kim User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130715 Thunderbird/17.0.7 MIME-Version: 1.0 To: Konstantin Belousov Subject: Re: [PATCH] Atomic swap and test-and-set References: <51F81A74.4030009@FreeBSD.org> <20130731173247.GK4972@kib.kiev.ua> In-Reply-To: <20130731173247.GK4972@kib.kiev.ua> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: dumbbell@FreeBSD.org, freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 20:03:16 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2013-07-31 13:32:47 -0400, Konstantin Belousov wrote: > On Tue, Jul 30, 2013 at 03:56:36PM -0400, Jung-uk Kim wrote: >> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 >> >> The attached patch implements atomic_swap_() and >> atomic_testandset_() for x86. is int or long ATM. >> >> The semantics of each operation (in atomic(9) style): >> >> atomic_swap(p, v) r = *p; *p = v; return (r); >> >> atomic_testandset(p, s) v = ()1 << s % (sizeof() * >> NBBY); r = (*p & v) != 0; *p |= v; return (r); >> >> Note atomic_readandclear_() is now obsolete and implemented >> as a macro around atomic_swap_() where the new value v is >> 0. >> >> Please let me know if you have any objection. > I think that the atomic.diff is fine. > > You did not documented swap and testandset, which would be good to > have in the manpage. Sure. > I think that constraints for all added inlines contradict the > explicit requirement in the gcc documentation, which states that > rmw operands should be specified using the 'same location' output > operand for the input operand (hopefully this can be parsed). It > just happens so that for "m" compiler cannot do anything else then > passing the address. > > The problem is not specific to the new functions, it seems to be > present in other functions as well, e.g. cmpset. This probably can > be postponed. What a coincidence! I was cleaning up the constraints before I read your e-mail. Also, I am implementing 64-bit versions for i386 and moving atomic_cmpset_64 from pmap.h to atomic.h. I'll post new patches when I am done. Jung-uk Kim -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (FreeBSD) iQEcBAEBAgAGBQJR+W0sAAoJECXpabHZMqHOaVsH/1CzPZiPfpWXQefh08wOPSMV bF/M0UrevFKE72QnRTiB00D6aY9vfNRmpYf3oceyMoZl3iYytRfqmNCawCUwMt+E suUrIsLNF+Kzjq5EcuZRDztAaphRUbr3CcFQFiE82nD6HE1gcsYmdj8i5e93OPgS B9MH7mgJGbct5q5eyM6BdJJSMltbKg2JPEWTlcMIly1P5FrHablQd53iqhHIIkwx JHWJNxcMJsv+EEd8cCHepNSODEI7XoCuiOpkm4Y7k4Ezy/82pWUVG7/WcBvzSNt1 Yxq66TxloJzPzT67bOSvqohTx+9mWX+blPBRJGhZzOloo9KG7sF9IdGoUkwcoqg= =C2G6 -----END PGP SIGNATURE----- From owner-freebsd-arch@FreeBSD.ORG Wed Jul 31 20:42:34 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 162A2591; Wed, 31 Jul 2013 20:42:34 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail108.syd.optusnet.com.au (mail108.syd.optusnet.com.au [211.29.132.59]) by mx1.freebsd.org (Postfix) with ESMTP id B489D26B1; Wed, 31 Jul 2013 20:42:33 +0000 (UTC) Received: from c122-106-156-23.carlnfd1.nsw.optusnet.com.au (c122-106-156-23.carlnfd1.nsw.optusnet.com.au [122.106.156.23]) by mail108.syd.optusnet.com.au (Postfix) with ESMTPS id A252A1A12B7; Thu, 1 Aug 2013 06:42:22 +1000 (EST) Date: Thu, 1 Aug 2013 06:42:21 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Konstantin Belousov Subject: Re: [PATCH] Atomic swap and test-and-set In-Reply-To: <20130731173247.GK4972@kib.kiev.ua> Message-ID: <20130801055421.R2570@besplex.bde.org> References: <51F81A74.4030009@FreeBSD.org> <20130731173247.GK4972@kib.kiev.ua> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=DstvpgP+ c=1 sm=1 tr=0 a=ebeQFi2P/qHVC0Yw9JDJ4g==:117 a=PO7r1zJSAAAA:8 a=1OkM0EXDtuEA:10 a=kj9zAlcOel0A:10 a=JzwRw_2MAAAA:8 a=TAuQMiTarRgA:10 a=ltDKUUI1AAAA:8 a=ZkdbeemF0AEiy2tZeIwA:9 a=4RcLnnIehuRn5zdd:21 a=lQd6o-TnKxyesf0K:21 a=CjuIK1q_8ugA:10 a=kT6dOc83ZcwA:10 Cc: dumbbell@FreeBSD.org, Jung-uk Kim , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 20:42:34 -0000 On Wed, 31 Jul 2013, Konstantin Belousov wrote: > On Tue, Jul 30, 2013 at 03:56:36PM -0400, Jung-uk Kim wrote: >> ... >> Please let me know if you have any objection. > I think that the atomic.diff is fine. > > You did not documented swap and testandset, which would be good to have > in the manpage. > > I think that constraints for all added inlines contradict the explicit > requirement in the gcc documentation, which states that rmw operands > should be specified using the 'same location' output operand for the > input operand (hopefully this can be parsed). It just happens so that > for "m" compiler cannot do anything else then passing the address. > > The problem is not specific to the new functions, it seems to be present > in other functions as well, e.g. cmpset. This probably can be postponed. The problem is the opposite. The other functions used to use the '+' constraint to indicate input-output operands, but gcc broke this so it was changed in 2005. gcc.info still says fuzzily that the '+' constraint "should" only be used for operands that can be allocated in a register: % The ordinary output operands must be write-only; GCC will assume that % the values in these operands before the instruction are dead and need % not be generated. Extended asm supports input-output or read-write % operands. Use the constraint character `+' to indicate such an operand % and list it with the output operands. You should only use read-write ^^^^^^ % operands when the constraints for the operand (or the operand in which % only some of the bits are to be changed) allow a register. ^^^^^^^^^^^^^^^^ but using it for non-register operands caused problems, so the requirement was "shall". % You may, as an alternative, logically split its function into two ^^^^^^^^^^^^^^^^^ % separate operands, one input operand and one write-only output operand. % The connection between them is expressed by constraints which say they % need to be in the same location when the instruction executes. You can % use the same C expression for both operands, or different expressions. % For example, here we write the (fictitious) `combine' instruction with % `bar' as its read-only source operand and `foo' as its read-write % destination: % % asm ("combine %2,%0" : "=r" (foo) : "0" (foo), "g" (bar)); % % The constraint `"0"' for operand 1 says that it must occupy the same % location as operand 0. A number in constraint is allowed only in an % input operand and it must refer to an output operand. This says the '0' constraint is just an alternative (worse) way of writing the '+' constraint. The '+' and '0' constraints should be allowed for all types of operands. If the compiler doesn't need them for correctness, then they should work as syntactic sugar for avoiding repeating operands. (The '0' constraint is bad since it works as syntactic salt and increases the number of operands. The magic numbers for a large number of operands are hard enough to keep track of without unnecessary ones increasing the number.) But I think constrains indicating input-output operands, and even more, are needed for correctness for memory operands. Without them, there seems to be nothing (except volatile qualifiers and optimization) to prevent the compiler copying an input operand to a different place in memory. It would have to copy the input operand to memory if it is not already in memory (and the constraint says memory), and it is not clear that it can't do this for operands of the form (*dst). Using the pointer and hiding the accesses through it in the asm is no good as a fix, since it now requires memory clobbers and perhaps another register to hold the pointer. Perhaps this is fixed in clang or in newer gcc's, but it is hard to keep down with compiler bugs. It actually became worse between gcc-3 and gcc-4. In gcc-3, gcc.info said that the '+' constraint was required for input-output operands (with no restriction to register operands), and that the '0' constraint was an alternative restricted to register operands. Now it says the above (that '+' "should" only be used for register operands, and that '0' is an alternative with no other restrictions). Hmm, another reading of the gcc-4 info is that it says that it is input-output operands that require a register operand, so that it is just invalid to try to make a memory operand input-output. Bruce From owner-freebsd-arch@FreeBSD.ORG Thu Aug 1 12:43:21 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id B9F3F5BD for ; Thu, 1 Aug 2013 12:43:21 +0000 (UTC) (envelope-from piyuskedia@gmail.com) Received: from mail-pa0-x243.google.com (mail-pa0-x243.google.com [IPv6:2607:f8b0:400e:c03::243]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 93F432FEA for ; Thu, 1 Aug 2013 12:43:21 +0000 (UTC) Received: by mail-pa0-f67.google.com with SMTP id bg4so1033834pad.6 for ; Thu, 01 Aug 2013 05:43:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=mafbX2W7S1JeeQSYZ6A8Slti3Xv6Jax0a94Jr5CIAog=; b=eo9+O3rU8PTH0M5XRiwr7ISjR8BCSCrlMvuY1/2fTaCwYgGl5ehqfAsLlKHe7gtO5Q c2RphFu+2RIk3A0zXHnwQ9eohM1/ZlssdhBiPdvZ27RPtC1V1c73EU1533unYyv93Dvp VCYOl40fV103cAfymqbVdBB0rbmmTmwTsn9fMz1pyKJZaWBoJoWqQtCxuPjb4ZGtGMnI C6Ip5f7dNmJjl6zzTqqAh047bzHRk1ZVKtFpTRMGXKrQ3ix6j+yOfogOJCZAd2XUFGu/ +7G/LoYU9B8A3XxZsiLJnVYrJQT0QqTegaHKCIl41wJVVd2/VbCDXh2eLnIsIAURPGWf VwPQ== MIME-Version: 1.0 X-Received: by 10.66.163.229 with SMTP id yl5mr3953991pab.69.1375361001296; Thu, 01 Aug 2013 05:43:21 -0700 (PDT) Received: by 10.70.7.65 with HTTP; Thu, 1 Aug 2013 05:43:21 -0700 (PDT) In-Reply-To: References: Date: Thu, 1 Aug 2013 21:43:21 +0900 Message-ID: Subject: Fwd: Interrupt Handlers From: Piyus Kedia To: freebsd-arch@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Aug 2013 12:43:21 -0000 Hi All, I am working on dynamic binary translator. Please tell me if anybody know if in freebsd interrupted kernel PC is ever used in interrupt handlers? Thanks and regards, Piyus From owner-freebsd-arch@FreeBSD.ORG Fri Aug 2 00:41:29 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from hammer.pct.niksun.com (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by hub.freebsd.org (Postfix) with ESMTP id D766E937; Fri, 2 Aug 2013 00:41:28 +0000 (UTC) (envelope-from jkim@FreeBSD.org) Message-ID: <51FAFFDF.7040309@FreeBSD.org> Date: Thu, 01 Aug 2013 20:39:59 -0400 From: Jung-uk Kim User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130715 Thunderbird/17.0.7 MIME-Version: 1.0 To: Konstantin Belousov Subject: Re: [PATCH] Atomic swap and test-and-set References: <51F81A74.4030009@FreeBSD.org> <20130731173247.GK4972@kib.kiev.ua> <51F96D2C.6030109@FreeBSD.org> In-Reply-To: <51F96D2C.6030109@FreeBSD.org> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: bde@FreeBSD.org, dumbbell@FreeBSD.org, freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Aug 2013 00:41:29 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2013-07-31 16:01:48 -0400, Jung-uk Kim wrote: > On 2013-07-31 13:32:47 -0400, Konstantin Belousov wrote: >> On Tue, Jul 30, 2013 at 03:56:36PM -0400, Jung-uk Kim wrote: >>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 >>> >>> The attached patch implements atomic_swap_() and >>> atomic_testandset_() for x86. is int or long >>> ATM. >>> >>> The semantics of each operation (in atomic(9) style): >>> >>> atomic_swap(p, v) r = *p; *p = v; return (r); >>> >>> atomic_testandset(p, s) v = ()1 << s % (sizeof() * >>> NBBY); r = (*p & v) != 0; *p |= v; return (r); >>> >>> Note atomic_readandclear_() is now obsolete and >>> implemented as a macro around atomic_swap_() where the >>> new value v is 0. >>> >>> Please let me know if you have any objection. >> I think that the atomic.diff is fine. > >> You did not documented swap and testandset, which would be good >> to have in the manpage. > > Sure. > >> I think that constraints for all added inlines contradict the >> explicit requirement in the gcc documentation, which states that >> rmw operands should be specified using the 'same location' >> output operand for the input operand (hopefully this can be >> parsed). It just happens so that for "m" compiler cannot do >> anything else then passing the address. > >> The problem is not specific to the new functions, it seems to be >> present in other functions as well, e.g. cmpset. This probably >> can be postponed. > > What a coincidence! I was cleaning up the constraints before I > read your e-mail. Also, I am implementing 64-bit versions for i386 > and moving atomic_cmpset_64 from pmap.h to atomic.h. > > I'll post new patches when I am done. I just created a branch and committed the patches because the final version grew too big. http://svnweb.freebsd.org/base/projects/atomic64 Please let me know what you think. Especially, I need more eyes to review the last two changes: http://svnweb.freebsd.org/changeset/base/253876 http://svnweb.freebsd.org/changeset/base/253877 Feel free to implement the new atomic ops for other architectures, BTW. :-) Thanks! Jung-uk Kim * Note: drm2 patches for AMD GPUs were updated as well: http://people.freebsd.org/~jkim/drm2_atomic.diff http://people.freebsd.org/~jkim/drm2_radeon.diff The second patch is a new addition, which makes it build on i386. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (FreeBSD) iQEcBAEBAgAGBQJR+v/fAAoJECXpabHZMqHOL3kH/1Ak7vGgolriV3XP3vontBjf 6Wj0MkuzA3ytF6uhdoV5XQaDT6De2hTOOJfSDzLvaUiRI7gwlGW+WlGgE03Raly6 bsqcEUID9JCSUDpLc+MaxVBnbSZM6bkenTaSgDzMVCqtHt28BLsFrEHP+yModqaY f/FrVR8RRJc7kjpKu0kpbSwTAUBVR73oJeWP4z58dQvN4EjRd3VbrZDrP42ppzjq CEKjaub4pd7AEOPH18aB4j/WNp1uY04ybsbFubsOdfjOJbSz7oDRhC+rM8rKEuYL XBkNFE5xRL14CxPAluEkgrSNd4RQYTlPPQbup998tgN5gOu9M2wVYPtYlUXQCYk= =CzrX -----END PGP SIGNATURE----- From owner-freebsd-arch@FreeBSD.ORG Fri Aug 2 08:38:05 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 961ED63B for ; Fri, 2 Aug 2013 08:38:05 +0000 (UTC) (envelope-from leaf-sapporo@www2865.sakura.ne.jp) Received: from www2865.sakura.ne.jp (www2865.sakura.ne.jp [IPv6:2403:3a00:201:1c:49:212:198:75]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 473E22B51 for ; Fri, 2 Aug 2013 08:38:05 +0000 (UTC) Received: from www2865.sakura.ne.jp (localhost [127.0.0.1]) by www2865.sakura.ne.jp (8.14.4/8.14.4) with ESMTP id r728bwca033946 for ; Fri, 2 Aug 2013 17:37:58 +0900 (JST) (envelope-from leaf-sapporo@www2865.sakura.ne.jp) Received: (from leaf-sapporo@localhost) by www2865.sakura.ne.jp (8.14.4/8.14.4/Submit) id r728bwxw033943; Fri, 2 Aug 2013 17:37:58 +0900 (JST) (envelope-from leaf-sapporo) Date: Fri, 2 Aug 2013 17:37:58 +0900 (JST) To: freebsd-arch@freebsd.org Subject: =?utf-8?B?0JDQu9GM0YLQtdGA0L3QsNGC0LjQstC90YvQuSDQuNGB0YLQvtGH0L3QuNC6INGD0LLQtdC70LjRh9C10L3QuNGPINC/0L7RgdC10YnQsNC10LzQvtGB0YLQuA==?= From: =?utf-8?B?0JTQvNC40YLRgNC40Lk=?= Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Aug 2013 08:38:05 -0000 Здравствуйте! Хочу предложить для Вашего сайта целевых посетителей. Источником являются электронные рассылки. Наши преимущества: - Возможность таргетинга по любому городу России; - Просмотр статистики в личном кабинете нашего сервиса; - Стоимость клика гораздо дешевле Яндекс.Директа; - Размер объявления до 200 символов. Обращайтесь по любым вопросам по телефону: 7 (Ч 95) 54 2 -3 9 ~ 87 From owner-freebsd-arch@FreeBSD.ORG Fri Aug 2 08:53:20 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 5DD3DADE; Fri, 2 Aug 2013 08:53:20 +0000 (UTC) (envelope-from tijl@coosemans.org) Received: from mailrelay009.isp.belgacom.be (mailrelay009.isp.belgacom.be [195.238.6.176]) by mx1.freebsd.org (Postfix) with ESMTP id A3E692C11; Fri, 2 Aug 2013 08:53:19 +0000 (UTC) X-Belgacom-Dynamic: yes X-Cloudmark-SP-Filtered: true X-Cloudmark-SP-Result: v=1.1 cv=YNYxFN+OaEw7Bvgx8BtAje2BdF1dMiUvAYP8MJfhIY0= c=1 sm=2 a=1OkM0EXDtuEA:10 a=ZxPGInapWdQA:10 a=ltDKUUI1AAAA:8 a=mDV3o1hIAAAA:8 a=gMpRr5FI_dPS_sViXIkA:9 a=CjuIK1q_8ugA:10 a=JbAvG2SwbSEA:10 a=npukW5hYu58A:10 a=kT6dOc83ZcwA:10 a=utvzSIibA5NBc_NGTNcA:9 a=0CaTwavtbm8A:10 a=_G8qimlXLBUA:10 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AlgGAA1y+1FR8ZNi/2dsb2JhbABbgwY1rjyQYYEaF3SCJAEBBVYjEAsOCgklDyoeBognCLkQjS+CWAeEDAOQEodMkU+DGTo Received: from 98.147-241-81.adsl-dyn.isp.belgacom.be (HELO kalimero.tijl.coosemans.org) ([81.241.147.98]) by relay.skynet.be with ESMTP; 02 Aug 2013 10:52:08 +0200 Received: from kalimero.tijl.coosemans.org (kalimero.tijl.coosemans.org [127.0.0.1]) by kalimero.tijl.coosemans.org (8.14.7/8.14.7) with ESMTP id r728q6UG001866; Fri, 2 Aug 2013 10:52:07 +0200 (CEST) (envelope-from tijl@coosemans.org) Date: Fri, 2 Aug 2013 10:52:00 +0200 From: Tijl Coosemans To: Bruce Evans Subject: Re: [PATCH] Atomic swap and test-and-set Message-ID: <20130802105200.29672730@kalimero.tijl.coosemans.org> In-Reply-To: <20130801055421.R2570@besplex.bde.org> References: <51F81A74.4030009@FreeBSD.org> <20130731173247.GK4972@kib.kiev.ua> <20130801055421.R2570@besplex.bde.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=PGP-SHA256; boundary="Sig_/BWkRovEEKzbI/YgkEZrBK1w"; protocol="application/pgp-signature" Cc: Konstantin Belousov , Jung-uk Kim , dumbbell@FreeBSD.org, freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Aug 2013 08:53:20 -0000 --Sig_/BWkRovEEKzbI/YgkEZrBK1w Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Thu, 1 Aug 2013 06:42:21 +1000 (EST) Bruce Evans wrote: > On Wed, 31 Jul 2013, Konstantin Belousov wrote: >> I think that constraints for all added inlines contradict the explicit >> requirement in the gcc documentation, which states that rmw operands >> should be specified using the 'same location' output operand for the >> input operand (hopefully this can be parsed). It just happens so that >> for "m" compiler cannot do anything else then passing the address. >> >> The problem is not specific to the new functions, it seems to be present >> in other functions as well, e.g. cmpset. This probably can be postponed. >=20 > The problem is the opposite. The other functions used to use the '+' > constraint to indicate input-output operands, but gcc broke this so it > was changed in 2005. gcc.info still says fuzzily that the '+' constraint > "should" only be used for operands that can be allocated in a register: That text has changed for gcc 4.8.0: http://gcc.gnu.org/ml/gcc-patches/2012-10/msg02130.html It refers to a patch from 2004: http://gcc.gnu.org/ml/gcc-patches/2004-05/msg00438.html That patch appears to be in all gcc 4.x releases. --Sig_/BWkRovEEKzbI/YgkEZrBK1w Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (FreeBSD) iF4EAREIAAYFAlH7czUACgkQfoCS2CCgtisFqAD/RwXejdmz2/29RZ0yUB2Y0Qj5 BEBwQb/t9sQy9nULhAoA/1zwDK+Z8Tao3xUjXBnM+goR0uF/9pReH6MWkEgZybiJ =6d4o -----END PGP SIGNATURE----- --Sig_/BWkRovEEKzbI/YgkEZrBK1w-- From owner-freebsd-arch@FreeBSD.ORG Fri Aug 2 10:06:12 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 5C112834; Fri, 2 Aug 2013 10:06:12 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail106.syd.optusnet.com.au (mail106.syd.optusnet.com.au [211.29.132.42]) by mx1.freebsd.org (Postfix) with ESMTP id 1C30E2EEC; Fri, 2 Aug 2013 10:06:11 +0000 (UTC) Received: from c122-106-156-23.carlnfd1.nsw.optusnet.com.au (c122-106-156-23.carlnfd1.nsw.optusnet.com.au [122.106.156.23]) by mail106.syd.optusnet.com.au (Postfix) with ESMTPS id 52C5E3C172B; Fri, 2 Aug 2013 20:05:26 +1000 (EST) Date: Fri, 2 Aug 2013 20:05:18 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Tijl Coosemans Subject: Re: [PATCH] Atomic swap and test-and-set In-Reply-To: <20130802105200.29672730@kalimero.tijl.coosemans.org> Message-ID: <20130802195156.Q1047@besplex.bde.org> References: <51F81A74.4030009@FreeBSD.org> <20130731173247.GK4972@kib.kiev.ua> <20130801055421.R2570@besplex.bde.org> <20130802105200.29672730@kalimero.tijl.coosemans.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=DstvpgP+ c=1 sm=1 tr=0 a=ebeQFi2P/qHVC0Yw9JDJ4g==:117 a=PO7r1zJSAAAA:8 a=1OkM0EXDtuEA:10 a=kj9zAlcOel0A:10 a=JzwRw_2MAAAA:8 a=TAuQMiTarRgA:10 a=ltDKUUI1AAAA:8 a=mDV3o1hIAAAA:8 a=Chh8ltowLiiSAWs7W10A:9 a=CjuIK1q_8ugA:10 a=JbAvG2SwbSEA:10 a=npukW5hYu58A:10 a=kT6dOc83ZcwA:10 Cc: Konstantin Belousov , Jung-uk Kim , dumbbell@FreeBSD.org, freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Aug 2013 10:06:12 -0000 On Fri, 2 Aug 2013, Tijl Coosemans wrote: > On Thu, 1 Aug 2013 06:42:21 +1000 (EST) Bruce Evans wrote: >> On Wed, 31 Jul 2013, Konstantin Belousov wrote: >>> I think that constraints for all added inlines contradict the explicit >>> requirement in the gcc documentation, which states that rmw operands >>> should be specified using the 'same location' output operand for the >>> input operand (hopefully this can be parsed). It just happens so that >>> for "m" compiler cannot do anything else then passing the address. >>> >>> The problem is not specific to the new functions, it seems to be present >>> in other functions as well, e.g. cmpset. This probably can be postponed. >> >> The problem is the opposite. The other functions used to use the '+' >> constraint to indicate input-output operands, but gcc broke this so it >> was changed in 2005. gcc.info still says fuzzily that the '+' constraint >> "should" only be used for operands that can be allocated in a register: > > That text has changed for gcc 4.8.0: > http://gcc.gnu.org/ml/gcc-patches/2012-10/msg02130.html > > It refers to a patch from 2004: > http://gcc.gnu.org/ml/gcc-patches/2004-05/msg00438.html > > That patch appears to be in all gcc 4.x releases. Actually, only a fixed version of it seems to be there. The version of gcc-4.2.1 in FreeBSD doesn't have a Changelog entry matching msg00438.html, but it has a patch that does more. The followup to msg00438.html pointed out a problem in the original patch (that it generates a matching input constraint too blindly and thus breaks the case of multiple alternatives). This seems to be fixed in the version in FreeBSD, but the followup stops at that point. So the pointer to the patch seems to be off by 1. msg02130.html gives the fuzzier pointer "back in 2004". These are still better in log messages than vcs-specific revision numbers. Bruce From owner-freebsd-arch@FreeBSD.ORG Fri Aug 2 10:08:09 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 1977B919 for ; Fri, 2 Aug 2013 10:08:09 +0000 (UTC) (envelope-from piyuskedia@gmail.com) Received: from mail-pd0-x233.google.com (mail-pd0-x233.google.com [IPv6:2607:f8b0:400e:c02::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E77CF2F09 for ; Fri, 2 Aug 2013 10:08:08 +0000 (UTC) Received: by mail-pd0-f179.google.com with SMTP id v10so503282pde.10 for ; Fri, 02 Aug 2013 03:08:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=jl18ZPqEMuIGEcOhqTtWpNaO1mSPas9dWzzIOmaCFak=; b=nNgSsdeFJXqp2pNAYRa9Jgubz1KDJF2kQDjn4lQMbDLl/85T5AHSospWAYJ9+X/nzD 7XSVucEain7DYlLZBcjCV/4F01pYn0JbFTJtspXdeBH9mwz7cndw4BXxxD0lx7pBMBxs 7/ifK7lHjNpjSoUwrsjvEwU4XIhlShxKSgA3Cy03qAv7vS/Co/0gtOIknaIxKpjPH3A5 MS0U/BZvpnDXutiCNjXRR4Cilj+jaGxkMLaiRnssWufgLXSvvJ5NCDykHYAELgWJEgwd j9rfiz4uXIyl9BsxyzcflfVXlyO97KyOWXFFYqXUg8HDtKgz7djUzPw82rDckI6inKFS 6DPg== MIME-Version: 1.0 X-Received: by 10.68.255.1 with SMTP id am1mr6838441pbd.68.1375438088484; Fri, 02 Aug 2013 03:08:08 -0700 (PDT) Received: by 10.70.7.65 with HTTP; Fri, 2 Aug 2013 03:08:08 -0700 (PDT) In-Reply-To: References: Date: Fri, 2 Aug 2013 19:08:08 +0900 Message-ID: Subject: Fwd: Use of the PC value in interrupt/exception handlers From: Piyus Kedia To: freebsd-arch@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Sorav Bansal X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Aug 2013 10:08:09 -0000 Dear all, We are working on developing a dynamic binary translator for the kernel. Towards this, we wanted to confirm if the interrupted PC value pushed on stack by an interrupt/exception is used by the interrupt/exception handlers? For example, is the PC value compared against a fixed address to determine the handler behaviour (like Linux's page fault handler compares the faulting PC against an exception table, to allow functions like copy_from_user to fault). Basically, we are wondering if it is safe to replace the pushed PC value on stack by another value. This would be safe if the PC value is only used for returning from interrupt, or for reading contents at that PC address (e.g., to decode the instruction at current PC). It would be unsafe if the value of the address itself is meaningful to the handler. We found that in FreeBSD segment-not-present exception handler checks the trapped PC value against some fixed kernel PC by looking at the code, except that it is only used for debugging purposes. It would be nice if somebody could also confirm this. Thanks, Piyus From owner-freebsd-arch@FreeBSD.ORG Fri Aug 2 12:06:00 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 13545645 for ; Fri, 2 Aug 2013 12:06:00 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 8B7E723EB for ; Fri, 2 Aug 2013 12:05:59 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.7/8.14.7) with ESMTP id r72C45Z0013169; Fri, 2 Aug 2013 15:04:05 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua r72C45Z0013169 Received: (from kostik@localhost) by tom.home (8.14.7/8.14.7/Submit) id r72C45xb013168; Fri, 2 Aug 2013 15:04:05 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 2 Aug 2013 15:04:05 +0300 From: Konstantin Belousov To: Piyus Kedia Subject: Re: Fwd: Use of the PC value in interrupt/exception handlers Message-ID: <20130802120405.GY4972@kib.kiev.ua> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="64C3cv1XTIf1+2px" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: Sorav Bansal , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Aug 2013 12:06:00 -0000 --64C3cv1XTIf1+2px Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Aug 02, 2013 at 07:08:08PM +0900, Piyus Kedia wrote: > Dear all, >=20 > We are working on developing a dynamic binary translator for the kernel. > Towards this, we wanted to confirm if the interrupted PC value pushed on > stack by an interrupt/exception is used by the interrupt/exception > handlers? For example, is the PC value compared against a fixed address to > determine the handler behaviour (like > Linux's page fault handler compares the faulting PC against an exception > table, to allow functions like copy_from_user to fault). >=20 > Basically, we are wondering if it is safe to replace the pushed PC value = on > stack by another value. This would be safe if the PC value is only used f= or > returning from interrupt, or for reading contents at that PC address (e.g= =2E, > to decode the instruction at current PC). It would be unsafe if the value > of the address itself is meaningful to the handler. >=20 > We found that in FreeBSD segment-not-present exception handler checks the > trapped PC value against some fixed kernel PC by looking at the code, > except that it is only used for debugging purposes. It would be nice if > somebody could also confirm this. You did not specified which architecture you are talking about. There are subtle differences between i386 and amd64 in this area, and I have no knowledge of other architectures. The answer to your question if very machine-specific. Yes, the most obvious place where the instruction pointer is replaced if the trap(). There, one case is when saved %rip points to the predefined list of the instructions which might fault because their operations are inherently non-safe. This list is mostly populated with addresses from the interrupt return sequence. Another mechanism, implemented in the trap() but actually used by other code, in particular, the copyin/copyout, but also other functions, is the pcb_onfault handler. If e.g. page fault happens during the copying, the control stream is passed to the handler specified in pcb_onfault. Look at the support.s for examples. I am not sure if this use is relevant to you. The ddb looks for the special rip values to properly step over the trap or interrupt frames, since these frames do not follow the 'normal' kernel frame layout (not quite normal, because we deviate from the ABI-mandated backtrace interface in the kernel). Are you only interested in the kernel side of things ? Usermode with the signal delivery/sigreturn(2), ucontext(2) and ptrace(2) could be also the interesting situation to consider. --64C3cv1XTIf1+2px Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (FreeBSD) iQIcBAEBAgAGBQJR+6A1AAoJEJDCuSvBvK1BGw8P/AtUM1aWnDviNAUeaY658SJu uuo8W0eYXoTLgTe4vHxI5N/vF0OA28N97pPHrK0+2ns9JrlXxdszk9qLGUwAtOL7 VOff3qwxJdw1ahflvyCDuwFVp9BUYju/VFLwhBzNtXUsc0+Iv0x0NrkcZ0aZU9ax tD+x3wxic/qOM/M6yHcFs1r4Ts27ZHarQtCjfJckCIWI4+2RPLeT3J7g8zsauy4j lUpeqBUmRBASOg7pD3FDTWYx/M87GlPpwQS8lLrbPnwnEoAimyE3lDkh4yq/tywR aI63ejn9q/1Qo4FAQ04GRa3QzpaofKVjiRSdv97zuZgyHdz5knt+PpfFu3qq4B/A ws89nOl3Lrxb1jwDzprOUg48U/jLCAc+ROayIT0x7pl8gz7ibu7uo/olX65y/qO8 ty4/cnGLO5QoJjE4OmQ16hXJN2GqJFf+CuAY8gp6o7YvDbNAnsUr+ff0N6VydrV+ yxgIGPzuS+hipmN6IMjlBLoxjEKM1YmgH9ZKHnl+XZnMpXkEvb0TEIs+6LfBVkwH /4ayhD7GSSY77OLpwPqv+999VuszW2JHEv5DiY37s9TH2yeLdKIlTjZ/V0taM0hU GNkarRHpN/e5Tj/PenUceyZ35u4l2cxc+2p+2ZxjST+IesolinIz4z2MSyMFwxZQ uNAX/sIGtAubtTbVC23H =vHVz -----END PGP SIGNATURE----- --64C3cv1XTIf1+2px-- From owner-freebsd-arch@FreeBSD.ORG Fri Aug 2 12:40:54 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 21058AFF for ; Fri, 2 Aug 2013 12:40:54 +0000 (UTC) (envelope-from piyuskedia@gmail.com) Received: from mail-pb0-x232.google.com (mail-pb0-x232.google.com [IPv6:2607:f8b0:400e:c01::232]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id EE526250C for ; Fri, 2 Aug 2013 12:40:53 +0000 (UTC) Received: by mail-pb0-f50.google.com with SMTP id uo5so636845pbc.23 for ; Fri, 02 Aug 2013 05:40:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=BFtiYAEVc9EE12SZ2X3VsAcqUvKR8DUG2Rtin83wiEw=; b=EV9y7rYfXCWssDpEBb4+JAREht2O3RmPYBRyXhG7n+5N6vpTsFZN/667AvbOXE9qH4 MBw0W1s6RLPW6zdXq8FFfBBs2pW62KvSdWFYr2dc66Xg5ozlZ8pgJT4UCNqyQ2ghReC2 IzDov73Ia38wf10fi3TMuCNwo1OiG1lKOYc4nRv+B8vmn32dP+DDbIMykUCQPvBRH5uA O7nI5bpjKEMq3Ol4XH4K4j1NqG5e8+o2089Fk0uF/i7TN+XLwVlaTmav2977nykWe21u UISK0W+TzI+w4hZuIkWiNTDRDQCwG7nkXn/Nrg6outFF+6TEA/iGxKl8d0MzviS41rjZ +VYg== MIME-Version: 1.0 X-Received: by 10.68.201.98 with SMTP id jz2mr7587460pbc.56.1375447253654; Fri, 02 Aug 2013 05:40:53 -0700 (PDT) Received: by 10.70.7.65 with HTTP; Fri, 2 Aug 2013 05:40:53 -0700 (PDT) In-Reply-To: <20130802120405.GY4972@kib.kiev.ua> References: <20130802120405.GY4972@kib.kiev.ua> Date: Fri, 2 Aug 2013 21:40:53 +0900 Message-ID: Subject: Re: Fwd: Use of the PC value in interrupt/exception handlers From: Piyus Kedia To: Konstantin Belousov Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Aug 2013 12:40:54 -0000 Thanks, that helped a lot. On Fri, Aug 2, 2013 at 9:04 PM, Konstantin Belousov wrote: > On Fri, Aug 02, 2013 at 07:08:08PM +0900, Piyus Kedia wrote: > > Dear all, > > > > We are working on developing a dynamic binary translator for the kernel. > > Towards this, we wanted to confirm if the interrupted PC value pushed on > > stack by an interrupt/exception is used by the interrupt/exception > > handlers? For example, is the PC value compared against a fixed address > to > > determine the handler behaviour (like > > Linux's page fault handler compares the faulting PC against an exception > > table, to allow functions like copy_from_user to fault). > > > > Basically, we are wondering if it is safe to replace the pushed PC value > on > > stack by another value. This would be safe if the PC value is only used > for > > returning from interrupt, or for reading contents at that PC address > (e.g., > > to decode the instruction at current PC). It would be unsafe if the value > > of the address itself is meaningful to the handler. > > > > We found that in FreeBSD segment-not-present exception handler checks the > > trapped PC value against some fixed kernel PC by looking at the code, > > except that it is only used for debugging purposes. It would be nice if > > somebody could also confirm this. > > You did not specified which architecture you are talking about. There > are subtle differences between i386 and amd64 in this area, and I have > no knowledge of other architectures. The answer to your question if very > machine-specific. > > Yes, the most obvious place where the instruction pointer is replaced if > the trap(). There, one case is when saved %rip points to the predefined > list of the instructions which might fault because their operations are > inherently non-safe. This list is mostly populated with addresses from > the interrupt return sequence. > > Another mechanism, implemented in the trap() but actually used by other > code, in particular, the copyin/copyout, but also other functions, is > the pcb_onfault handler. If e.g. page fault happens during the copying, > the control stream is passed to the handler specified in pcb_onfault. > Look at the support.s for examples. I am not sure if this use is relevant > to you. > > The ddb looks for the special rip values to properly step over the trap > or interrupt frames, since these frames do not follow the 'normal' kernel > frame layout (not quite normal, because we deviate from the ABI-mandated > backtrace interface in the kernel). > > Are you only interested in the kernel side of things ? Usermode with the > signal delivery/sigreturn(2), ucontext(2) and ptrace(2) could be also > the interesting situation to consider. > From owner-freebsd-arch@FreeBSD.ORG Fri Aug 2 15:32:21 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 73A63F50 for ; Fri, 2 Aug 2013 15:32:21 +0000 (UTC) (envelope-from ian@FreeBSD.org) Received: from mho-02-ewr.mailhop.org (mho-02-ewr.mailhop.org [204.13.248.72]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 47D602C9F for ; Fri, 2 Aug 2013 15:32:20 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-02-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1V5HL1-0009Ml-Tp; Fri, 02 Aug 2013 15:32:20 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id r72FVYEP022775; Fri, 2 Aug 2013 09:31:34 -0600 (MDT) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX1/W/1+LO8epj8i6ZroHx49H Subject: Re: Fwd: Use of the PC value in interrupt/exception handlers From: Ian Lepore To: Piyus Kedia In-Reply-To: References: Content-Type: text/plain; charset="us-ascii" Date: Fri, 02 Aug 2013 09:31:33 -0600 Message-ID: <1375457493.45247.249.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: Sorav Bansal , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Aug 2013 15:32:21 -0000 On Fri, 2013-08-02 at 19:08 +0900, Piyus Kedia wrote: > Dear all, > > We are working on developing a dynamic binary translator for the kernel. > Towards this, we wanted to confirm if the interrupted PC value pushed on > stack by an interrupt/exception is used by the interrupt/exception > handlers? For example, is the PC value compared against a fixed address to > determine the handler behaviour (like > Linux's page fault handler compares the faulting PC against an exception > table, to allow functions like copy_from_user to fault). > > Basically, we are wondering if it is safe to replace the pushed PC value on > stack by another value. This would be safe if the PC value is only used for > returning from interrupt, or for reading contents at that PC address (e.g., > to decode the instruction at current PC). It would be unsafe if the value > of the address itself is meaningful to the handler. > > We found that in FreeBSD segment-not-present exception handler checks the > trapped PC value against some fixed kernel PC by looking at the code, > except that it is only used for debugging purposes. It would be nice if > somebody could also confirm this. > > Thanks, > Piyus For the ARM architectures which use Restartable Atomic Sequences (RAS) to implement atomic operations, examining the value of the saved PC and possibly modifying it is how RAS works. See the PUSHFRAMEINSVC macro in sys/arm/include/asmacros.h. In a nutshell, the RAS code works by having userland code store the begin/end addresses of a small block of code that must be executed to completion without interruption to be correct. If an exception or interrupt happens while the PC is in that range, the exception-entry code implemented by PUSHFRAMEINSVC modifies the saved PC so that on return to userland, execution resumes at the beginning of the atomic sequence. -- Ian From owner-freebsd-arch@FreeBSD.ORG Fri Aug 2 16:59:44 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 7F77B4DF; Fri, 2 Aug 2013 16:59:44 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B070C20D7; Fri, 2 Aug 2013 16:59:13 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.7/8.14.7) with ESMTP id r72Gx7QY001570; Fri, 2 Aug 2013 19:59:07 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua r72Gx7QY001570 Received: (from kostik@localhost) by tom.home (8.14.7/8.14.7/Submit) id r72Gwax0001554; Fri, 2 Aug 2013 19:58:36 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 2 Aug 2013 19:58:36 +0300 From: Konstantin Belousov To: Ian Lepore Subject: Re: Fwd: Use of the PC value in interrupt/exception handlers Message-ID: <20130802165836.GC4972@kib.kiev.ua> References: <1375457493.45247.249.camel@revolution.hippie.lan> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="c/lwuELUGxxkaMpf" Content-Disposition: inline In-Reply-To: <1375457493.45247.249.camel@revolution.hippie.lan> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: freebsd-arch@FreeBSD.org, Sorav Bansal , Piyus Kedia X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Aug 2013 16:59:44 -0000 --c/lwuELUGxxkaMpf Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Aug 02, 2013 at 09:31:33AM -0600, Ian Lepore wrote: > On Fri, 2013-08-02 at 19:08 +0900, Piyus Kedia wrote: > > Dear all, > >=20 > > We are working on developing a dynamic binary translator for the kernel. > > Towards this, we wanted to confirm if the interrupted PC value pushed on > > stack by an interrupt/exception is used by the interrupt/exception > > handlers? For example, is the PC value compared against a fixed address= to > > determine the handler behaviour (like > > Linux's page fault handler compares the faulting PC against an exception > > table, to allow functions like copy_from_user to fault). > >=20 > > Basically, we are wondering if it is safe to replace the pushed PC valu= e on > > stack by another value. This would be safe if the PC value is only used= for > > returning from interrupt, or for reading contents at that PC address (e= =2Eg., > > to decode the instruction at current PC). It would be unsafe if the val= ue > > of the address itself is meaningful to the handler. > >=20 > > We found that in FreeBSD segment-not-present exception handler checks t= he > > trapped PC value against some fixed kernel PC by looking at the code, > > except that it is only used for debugging purposes. It would be nice if > > somebody could also confirm this. > >=20 > > Thanks, > > Piyus >=20 > For the ARM architectures which use Restartable Atomic Sequences (RAS) > to implement atomic operations, examining the value of the saved PC and > possibly modifying it is how RAS works. See the PUSHFRAMEINSVC macro in > sys/arm/include/asmacros.h. =20 >=20 > In a nutshell, the RAS code works by having userland code store the > begin/end addresses of a small block of code that must be executed to > completion without interruption to be correct. If an exception or > interrupt happens while the PC is in that range, the exception-entry > code implemented by PUSHFRAMEINSVC modifies the saved PC so that on > return to userland, execution resumes at the beginning of the atomic > sequence. This reminds of me the following MIPS code: http://svnweb.freebsd.org/base?view=3Drevision&revision=3D226517 --c/lwuELUGxxkaMpf Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (FreeBSD) iQIcBAEBAgAGBQJR++U8AAoJEJDCuSvBvK1BfG4P/1k47DWq0o1wSy3K0+9ri0g3 388Q1phR8IdJGDVTrf5ZzbuQi0tNL3Knyu9hL3deMG/1LXWByojOVxffycLwYCJo K+8Wie47QJO9ZFN6mks0NN7Q3NBeXC07RIN7ShPg+bTEUIzNbsytG+WlMNAqddis NP9l/XEYWHlXZNOGzuQhk9IrXKqI1w+9IY0xYOzpnre9YSHeVGDyzuhroX0xKXpP l5wbki2VKBncXb/4lpSNoQP50UgxiQgZqjsuQG9LTa1PhwyxJBjiVcjwFkDjs1Od pcKJdOnbjKzfQP+BDwS2x6v9Wl6ikxixIJ0WS9Eb6RuhiwRX8kpYGBdhlJ7Qms3f U4L+xYZriycnozUh4b3vzVfU7WwT4OBJysT7plqzMbfUNPa3N5lUgQj66SGjH0KY eV1BKsw1MPIIszAWRZ6vm4Ryri5rqvpj3/WiWoh2w6LizCNFQVhHmfnHaepYoNyx bfuUKVOdLgSZl7B7jcYygrybPFauakFTD7BbJhYObWZlXHCTirhK/8qRZAE3bhgq nuREPzE4x1s8elWvZIyy+jrwFseQvAcuYUnumXy0knylwfJ76qszfmUZ4BtVxVT7 fr8TDjNz9npVu7K4dN5K8hFg9D96pQqvfNsj7qJIc3/Rh4YP6FezxNZza/5h6HDx 42+ZbLht+U9okAeMkICO =EBhZ -----END PGP SIGNATURE----- --c/lwuELUGxxkaMpf-- From owner-freebsd-arch@FreeBSD.ORG Fri Aug 2 17:05:06 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id DE76E5D4; Fri, 2 Aug 2013 17:05:06 +0000 (UTC) (envelope-from piyuskedia@gmail.com) Received: from mail-pb0-x230.google.com (mail-pb0-x230.google.com [IPv6:2607:f8b0:400e:c01::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B0D812102; Fri, 2 Aug 2013 17:05:06 +0000 (UTC) Received: by mail-pb0-f48.google.com with SMTP id ma3so899059pbc.21 for ; Fri, 02 Aug 2013 10:05:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=fvz7dbIXEIXXfyP7q1x1NhsNnJAhdyXl0/aslvUgqc8=; b=TZmlh212+s0+D8c37eQphsYMeQ3JMJJvWZKDpGbUlh0zDp10iIXRxb1OZeIvZ+5aCd 13tJXhC47dj0ZIQkD3Sz9tJzeHn1uMcHAh6tg50EtikqxKupmzY/6CSrtBkB4ExloEOe LSEM5dTvufsziHYnEwR7lSR3mJsBd4h2CPAXdqEnTgUY4SC5aYlpqv8zv2FcxrOnacmL 6COyXGkkgSMOM7ANzSMjWjRRdaY4g3fzsPUMTuE6bB+Xy82gEMpwH0+iw5E2bVhk9d3y yLApF+6xMNOEBEZZY8Nf8DuTVedwWt0gcRlWnKS7pG/aPI2XCIae61Jw5ZBjkdZFrki3 KUPQ== MIME-Version: 1.0 X-Received: by 10.66.141.232 with SMTP id rr8mr11555601pab.184.1375463106206; Fri, 02 Aug 2013 10:05:06 -0700 (PDT) Received: by 10.70.7.65 with HTTP; Fri, 2 Aug 2013 10:05:06 -0700 (PDT) In-Reply-To: <20130802165836.GC4972@kib.kiev.ua> References: <1375457493.45247.249.camel@revolution.hippie.lan> <20130802165836.GC4972@kib.kiev.ua> Date: Sat, 3 Aug 2013 02:05:06 +0900 Message-ID: Subject: Re: Fwd: Use of the PC value in interrupt/exception handlers From: Piyus Kedia To: Konstantin Belousov Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-arch@freebsd.org, Ian Lepore , Sorav Bansal X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Aug 2013 17:05:06 -0000 Thank you, this explains things very clearly. It helped us a lot. On Sat, Aug 3, 2013 at 1:58 AM, Konstantin Belousov wrote: > On Fri, Aug 02, 2013 at 09:31:33AM -0600, Ian Lepore wrote: > > On Fri, 2013-08-02 at 19:08 +0900, Piyus Kedia wrote: > > > Dear all, > > > > > > We are working on developing a dynamic binary translator for the > kernel. > > > Towards this, we wanted to confirm if the interrupted PC value pushed > on > > > stack by an interrupt/exception is used by the interrupt/exception > > > handlers? For example, is the PC value compared against a fixed > address to > > > determine the handler behaviour (like > > > Linux's page fault handler compares the faulting PC against an > exception > > > table, to allow functions like copy_from_user to fault). > > > > > > Basically, we are wondering if it is safe to replace the pushed PC > value on > > > stack by another value. This would be safe if the PC value is only > used for > > > returning from interrupt, or for reading contents at that PC address > (e.g., > > > to decode the instruction at current PC). It would be unsafe if the > value > > > of the address itself is meaningful to the handler. > > > > > > We found that in FreeBSD segment-not-present exception handler checks > the > > > trapped PC value against some fixed kernel PC by looking at the code, > > > except that it is only used for debugging purposes. It would be nice if > > > somebody could also confirm this. > > > > > > Thanks, > > > Piyus > > > > For the ARM architectures which use Restartable Atomic Sequences (RAS) > > to implement atomic operations, examining the value of the saved PC and > > possibly modifying it is how RAS works. See the PUSHFRAMEINSVC macro in > > sys/arm/include/asmacros.h. > > > > In a nutshell, the RAS code works by having userland code store the > > begin/end addresses of a small block of code that must be executed to > > completion without interruption to be correct. If an exception or > > interrupt happens while the PC is in that range, the exception-entry > > code implemented by PUSHFRAMEINSVC modifies the saved PC so that on > > return to userland, execution resumes at the beginning of the atomic > > sequence. > > This reminds of me the following MIPS code: > http://svnweb.freebsd.org/base?view=revision&revision=226517 > From owner-freebsd-arch@FreeBSD.ORG Fri Aug 2 19:04:05 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 90A0958F; Fri, 2 Aug 2013 19:04:05 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.dawidek.net (garage.dawidek.net [91.121.88.72]) by mx1.freebsd.org (Postfix) with ESMTP id 5698B2614; Fri, 2 Aug 2013 19:04:05 +0000 (UTC) Received: from localhost (host-188-252-7-16.idev.pl [188.252.7.16]) by mail.dawidek.net (Postfix) with ESMTPSA id E63E5B37; Fri, 2 Aug 2013 20:58:54 +0200 (CEST) Date: Fri, 2 Aug 2013 21:04:32 +0200 From: Pawel Jakub Dawidek To: Peter Grehan Subject: Re: [PROPOSAL] GEOM probing/tasting firewall Message-ID: <20130802190431.GH5771@garage.freebsd.pl> References: <447183917.20130731130956@serebryakov.spb.ru> <51F91FAC.60905@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="7J16OGEJ/mt06A90" Content-Disposition: inline In-Reply-To: <51F91FAC.60905@freebsd.org> X-OS: FreeBSD 10.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-geom@freebsd.org, lev@FreeBSD.org, freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Aug 2013 19:04:05 -0000 --7J16OGEJ/mt06A90 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Jul 31, 2013 at 07:31:08AM -0700, Peter Grehan wrote: > > For first time this idea was formulated in Jabber talk with friend of > > mine, who uses FreeBSD for massive iSCSI hosting on ZVOLs. He has pro= blems > > with tasting these ZVOLs, which contain different types of data (Wind= ows > > disks, Linux disks, FreeBSD disks, etc). Here are label conflicts, st= range > > messages about corrupted GPTs, etc. So, it looks like to have configu= rable > > way to prevent some GEOM tasting is good idea. >=20 > I'm all for this. bhyve has the exact same problem with unnecessary=20 > tasting of zvols and raw volumes being used by guest o/s's. Firewall idea is overkill for my taste. I'd much prefer to have a flag which would tell GEOM not to present GEOM provider I'm creating for tasting. This also means it would not be available via /dev/. We would still need a way to selectively make those providers available via /dev/ or just presented for tasting, but ZVOL snapshots seems to be good candidates for such a flag. For regular ZFS file systems there is 'canmount' property which controls if the given file system should be mounted automatically or not. Maybe we need similar property for ZVOL snapshots that would enable/disable GEOM tasting. Another idea is to implement lazy device creation in /dev/ - when provider is created with this don't-taste flag its corresponding /dev/ entry is not created, because the DEV GEOM class didn't taste it. But DEV class could respond to devfs lookups by trying to find provider by name (there is function for that already) and when found, create /dev/ entry for it. This would make providers that don't like to be tasted still available through /dev/. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://mobter.com --7J16OGEJ/mt06A90 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (FreeBSD) iEYEARECAAYFAlH8Ar8ACgkQForvXbEpPzRQvwCeKciHHr6IeDDuFPy9L8LtWlnk regAoJSmkPC8+IwR5jhBnMMlEkg3Pm6d =E1cC -----END PGP SIGNATURE----- --7J16OGEJ/mt06A90-- From owner-freebsd-arch@FreeBSD.ORG Fri Aug 2 19:57:31 2013 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 90FE368B; Fri, 2 Aug 2013 19:57:31 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id 533B82885; Fri, 2 Aug 2013 19:57:31 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:2418:88c5:96ef:35b4]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPSA id EEB374AC57; Fri, 2 Aug 2013 23:57:27 +0400 (MSK) Date: Fri, 2 Aug 2013 23:57:20 +0400 From: Lev Serebryakov Organization: FreeBSD Project X-Priority: 3 (Normal) Message-ID: <1472113917.20130802235720@serebryakov.spb.ru> To: Pawel Jakub Dawidek Subject: Re: [PROPOSAL] GEOM probing/tasting firewall In-Reply-To: <20130802190431.GH5771@garage.freebsd.pl> References: <447183917.20130731130956@serebryakov.spb.ru> <51F91FAC.60905@freebsd.org> <20130802190431.GH5771@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-geom@freebsd.org, Peter Grehan , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Aug 2013 19:57:31 -0000 Hello, Pawel. You wrote 2 =D0=B0=D0=B2=D0=B3=D1=83=D1=81=D1=82=D0=B0 2013 =D0=B3., 23:04:= 32: PJD> Firewall idea is overkill for my taste. I'd much prefer to have a flag PJD> which would tell GEOM not to present GEOM provider I'm creating for PJD> tasting. This also means it would not be available via /dev/. I've thought about cases like "GPT in gmirror" and, worse, "MBR in gmirror" to solve by such "overkill" solution -- you don't allow geom_label to taste devices, which are included in mirror (or other RAID), but allow to taste them by anything else, so they have /dev entries and picked up by soft-RAID class they belong to... There was MANY threads about such problems in the past, and I don';t remember to have good solution but "make mirrors out of partitions". Several solutions were discussed, like "levels" of classes (to make tasting order predicatable), but, as far as I understand, nothing was done, and no good solution was found. --=20 // Black Lion AKA Lev Serebryakov From owner-freebsd-arch@FreeBSD.ORG Fri Aug 2 20:24:57 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id C054AA97 for ; Fri, 2 Aug 2013 20:24:57 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) by mx1.freebsd.org (Postfix) with ESMTP id 7A072298C for ; Fri, 2 Aug 2013 20:24:57 +0000 (UTC) Received: from slw by zxy.spb.ru with local (Exim 4.69 (FreeBSD)) (envelope-from ) id 1V5Lv2-000EzX-2W for freebsd-arch@freebsd.org; Sat, 03 Aug 2013 00:25:48 +0400 Date: Sat, 3 Aug 2013 00:25:48 +0400 From: Slawa Olhovchenkov To: freebsd-arch@freebsd.org Subject: Re: [PROPOSAL] GEOM probing/tasting firewall Message-ID: <20130802202548.GA57445@zxy.spb.ru> References: <447183917.20130731130956@serebryakov.spb.ru> <51F91FAC.60905@freebsd.org> <20130802190431.GH5771@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20130802190431.GH5771@garage.freebsd.pl> User-Agent: Mutt/1.5.21 (2010-09-15) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Aug 2013 20:24:57 -0000 On Fri, Aug 02, 2013 at 09:04:32PM +0200, Pawel Jakub Dawidek wrote: > On Wed, Jul 31, 2013 at 07:31:08AM -0700, Peter Grehan wrote: > > > For first time this idea was formulated in Jabber talk with friend of > > > mine, who uses FreeBSD for massive iSCSI hosting on ZVOLs. He has problems > > > with tasting these ZVOLs, which contain different types of data (Windows > > > disks, Linux disks, FreeBSD disks, etc). Here are label conflicts, strange > > > messages about corrupted GPTs, etc. So, it looks like to have configurable > > > way to prevent some GEOM tasting is good idea. > > > > I'm all for this. bhyve has the exact same problem with unnecessary > > tasting of zvols and raw volumes being used by guest o/s's. > > Firewall idea is overkill for my taste. I'd much prefer to have a flag > which would tell GEOM not to present GEOM provider I'm creating for > tasting. This also means it would not be available via /dev/. In this case this is don't allowed export such zvol over iSCSI or in the hypervisor. > We would still need a way to selectively make those providers available > via /dev/ or just presented for tasting, but ZVOL snapshots seems to be > good candidates for such a flag. Massive ZVOL snapshots is other case (and this case don't resolved by firewall). > For regular ZFS file systems there is 'canmount' property which controls > if the given file system should be mounted automatically or not. Maybe > we need similar property for ZVOL snapshots that would enable/disable > GEOM tasting. > > Another idea is to implement lazy device creation in /dev/ - when > provider is created with this don't-taste flag its corresponding /dev/ > entry is not created, because the DEV GEOM class didn't taste it. > But DEV class could respond to devfs lookups by trying to find provider > by name (there is function for that already) and when found, create > /dev/ entry for it. This would make providers that don't like to be > tasted still available through /dev/. And I don't allowed to see by `ls` list of created ZVOL? Bad. From owner-freebsd-arch@FreeBSD.ORG Fri Aug 2 20:33:14 2013 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 4751ECDF for ; Fri, 2 Aug 2013 20:33:14 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [46.4.40.135]) by mx1.freebsd.org (Postfix) with ESMTP id 0860029CB for ; Fri, 2 Aug 2013 20:33:13 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:2418:88c5:96ef:35b4]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPSA id 812B84AC58; Sat, 3 Aug 2013 00:33:12 +0400 (MSK) Date: Sat, 3 Aug 2013 00:33:05 +0400 From: Lev Serebryakov Organization: FreeBSD X-Priority: 3 (Normal) Message-ID: <682716067.20130803003305@serebryakov.spb.ru> To: Slawa Olhovchenkov Subject: Re: [PROPOSAL] GEOM probing/tasting firewall In-Reply-To: <20130802202548.GA57445@zxy.spb.ru> References: <447183917.20130731130956@serebryakov.spb.ru> <51F91FAC.60905@freebsd.org> <20130802190431.GH5771@garage.freebsd.pl> <20130802202548.GA57445@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Aug 2013 20:33:14 -0000 Hello, Slawa. You wrote 3 =D0=B0=D0=B2=D0=B3=D1=83=D1=81=D1=82=D0=B0 2013 =D0=B3., 0:25:4= 8: SO> Massive ZVOL snapshots is other case (and this case don't resolved by f= irewall). This case SHOULD be resolved by firewall. You "deny" tasting of snapshot by name mask for anything but DEV class. All /dev entries will be created, but its all -- no geom / raid / mirror / stripe / whatever tasting. --=20 // Black Lion AKA Lev Serebryakov