From owner-freebsd-hackers@freebsd.org Sun Oct 14 06:14:42 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1B6D710C6E8B
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Sun, 14 Oct 2018 06:14:42 +0000 (UTC)
(envelope-from srs0=ssn3=m2=sigsegv.be=kristof@codepro.be)
Received: from venus.codepro.be (venus.codepro.be [IPv6:2a01:4f8:162:1127::2])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256
bits)) (Client CN "smtp.codepro.be",
Issuer "Let's Encrypt Authority X3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id A234E7B10D;
Sun, 14 Oct 2018 06:14:41 +0000 (UTC)
(envelope-from srs0=ssn3=m2=sigsegv.be=kristof@codepro.be)
Received: from [192.168.228.1] (unknown [12.203.80.130])
(Authenticated sender: kp)
by venus.codepro.be (Postfix) with ESMTPSA id 603A534253;
Sun, 14 Oct 2018 08:14:38 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sigsegv.be; s=mail;
t=1539497679; bh=45aDuJ7ynYYkWhFlgasdK/KwjpVmfk8jku5nA/CzVlM=;
h=From:To:Cc:Subject:Date:In-Reply-To:References;
b=JRyfHenKgARpre92K4pTzxzWsSxdU4wA9uqkCtp3mdKRzO3svykKEGCT0A6DWzeXa
ZMCngE5hMokOPGsAMjcC1IqyeR4Efw/DGK+OkVk+3wxtEhPQ78occELfYa2LmDG9fD
lQ5nxOSKA5OqHHNXVT1LSEdWfm4L9ab5W74XScn8=
From: "Kristof Provost" <kristof@sigsegv.be>
To: "Lev Serebryakov" <lev@FreeBSD.org>
Cc: freebsd-hackers@FreeBSD.org
Subject: Re: What are ck_queue.h guarantees?
Date: Sat, 13 Oct 2018 23:14:31 -0700
X-Mailer: MailMate (2.0BETAr6123)
Message-ID: <AD1EC8D5-C137-4128-90C9-0FCAECC5754C@sigsegv.be>
In-Reply-To: <1551957390.20181013184006@serebryakov.spb.ru>
References: <1551957390.20181013184006@serebryakov.spb.ru>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Content-Filtered-By: Mailman/MimeDel 2.1.27
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sun, 14 Oct 2018 06:14:42 -0000
On 13 Oct 2018, at 8:40, Lev Serebryakov wrote:
> Hello Freebsd-hackers,
>
> Concurrency Kit documentation says:
>
> ====
> ck_queue is a queue.h-compatible implementation of many-reader-single-
> writer queues. It allows for safe concurrent iteration, peeking and
> read-
> side access in the presence of a single concurrent writer without any
> usage of locks.
> ====
>
> But in all places at kernel I peeked, CK_XXXX macros are protected by
> locks. Yes, even read ones.
>
Note that the implementation of if_maddr_rlock() doesn’t actually take
a lock. Instead it calls epoch_enter_preempt().
> Why is it so? Why do we bother to use CK_XXX API (which adds all
> needed
> barriers and uses CASes) if all accesses are protected by locks,
> anyway?
>
ck_queues are safe to use, even when elements are being added or
removed. Missing new elements is usually fine, but what happens if an
element we’re looking at right now is being removed by a different
thread?
We might still be using it when the removing thread frees it. That’s
what the epoch code protects against. It allows the removing thread to
know when no other thread is using the removed item any more (and thus
when it’s safe to actually delete it).
Hence the ‘lock’ and ‘unlock’ calls. They don’t actually take
a lock, and there’s no contention. Many threads can enter the section
between lock and unlock at the same time.
I suspect the ‘lock’/‘unlock’ naming is mostly historical here:
i.e. it used to be a real lock, and when it was replaced by the
epoch-based approach the functions were not renamed.
Best regards,
Kristof
From owner-freebsd-hackers@freebsd.org Sun Oct 14 09:58:22 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9C12F10CD04D
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Sun, 14 Oct 2018 09:58:22 +0000 (UTC)
(envelope-from munro@penski.net)
Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com
[IPv6:2a00:1450:4864:20::531])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(Client CN "smtp.gmail.com",
Issuer "Google Internet Authority G3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 1B11D819A4
for <freebsd-hackers@freebsd.org>; Sun, 14 Oct 2018 09:58:21 +0000 (UTC)
(envelope-from munro@penski.net)
Received: by mail-ed1-x531.google.com with SMTP id b7-v6so15135213edd.9
for <freebsd-hackers@freebsd.org>; Sun, 14 Oct 2018 02:58:21 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=ip9-org.20150623.gappssmtp.com; s=20150623;
h=mime-version:references:in-reply-to:from:date:message-id:subject:to
:cc; bh=vRjmFTiMhk7IeHcdK2cP4kwouWCZZHTG8YoEM0c+bAc=;
b=TyegoWZVD9n7y3axDYsCpc3Nti5HfFPAmfwODYeRfCxBVcDWAr1pVi8lLinPnIu0mu
DmGtDYtzl85i3qF1W7NKCuIDaeVfg7tMcRihv+QSQEtyVNHw83uOEGbCSkXObYfSTZZ7
IJOGJ4+QuAqpG9Ot0/BtYUTT4qnKyqKNzfdorn0Vp8J4ZN8YBa4xLyui1QBqtHcy63bm
+XDLz1G8ubpFGYu3O7/+RRnmymTFi7+SBjDGcWWkRfBtWIdbnOvEX5fGU/krc7mWAKi0
8P/8vynyfpQBvitCcJzV1u159+lyo/HqvKkk8gGsQwibrwTymdUynPGI3ZLf1lkbhd6H
7J8A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:mime-version:references:in-reply-to:from:date
:message-id:subject:to:cc;
bh=vRjmFTiMhk7IeHcdK2cP4kwouWCZZHTG8YoEM0c+bAc=;
b=nMisesEMxDqsXC1YZqjVkDWCDAnv8XkhMc6Xy8pR5IIf0Or0T0Nx+EGZwNX9RBuIuI
teFYb+JBS3FLoXcMztMsiGqEapaKjxaEukuZK/4reGQZN6hbm3xPqiC+Er1wNslsM0zt
rq158Y0Fq4hkz0H9fbu43HcvBYdqyAUQSz68DoOq2DMJdn0uQs35rkCtVu1kdMOYPHlE
GV7AWK/94s/tQEPbfSolwbglvK72KGmV05JhZccjpKpmKVAV8cQCoQpSMfFk++RYhB4t
yL+PhfLL4N4TbC8wbTsHhFOQ5Iy5NhUqEwW4egQKuvkjPkJizmDsmtzMLkt+BW5yIEvi
7Xmw==
X-Gm-Message-State: ABuFfojQE7DYTsMaNBlGPVAVCU3seSTUpS14UmhZsLxbhKaE6sEP21zw
xdt1e82QEJ2SH7FK02qaWZVXc+0iM9Llz6mW2jfbDw==
X-Google-Smtp-Source: ACcGV60SksBgwkqDrXf57IQxQOq/hI5rpc84kBDoWf5G8vtDw+HudYayOsGnatdkihKFgJskeDH3P3tnieZW5+xW6ds=
X-Received: by 2002:a17:906:6983:: with SMTP id
i3-v6mr14813721ejr.141.1539511100565;
Sun, 14 Oct 2018 02:58:20 -0700 (PDT)
MIME-Version: 1.0
References: <CADLWmXU=7QM-oHmY=TMAQanQE-dnXY4v74Zm1kkEz3Gc=ip21A@mail.gmail.com>
<20181011001954.GV5335@kib.kiev.ua>
<CADLWmXWS6qjt02bxUkd7BewfhYw69at8OYe+h15+1OCpnpi6ng@mail.gmail.com>
<20181013235021.GX5335@kib.kiev.ua>
In-Reply-To: <20181013235021.GX5335@kib.kiev.ua>
From: Thomas Munro <munro@ip9.org>
Date: Sun, 14 Oct 2018 22:58:08 +1300
Message-ID: <CADLWmXVN=anzRfP0kGkCVW2NR+u9Gcx=O1GVdbn_ZJuvzC7gHg@mail.gmail.com>
Subject: Re: PostgresSQL vs super pages
To: Konstantin Belousov <kostikbel@gmail.com>
Cc: freebsd-hackers@freebsd.org, alc@freebsd.org, mjg@freebsd.org
Content-Type: text/plain; charset="UTF-8"
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sun, 14 Oct 2018 09:58:22 -0000
On Sun, 14 Oct 2018 at 12:50, Konstantin Belousov <kostikbel@gmail.com> wrote:
> On Thu, Oct 11, 2018 at 02:01:20PM +1300, Thomas Munro wrote:
> > On Thu, 11 Oct 2018 at 13:20, Konstantin Belousov <kostikbel@gmail.com> wrote:
> > > On Thu, Oct 11, 2018 at 12:59:41PM +1300, Thomas Munro wrote:
> > > > shm_open("/PostgreSQL.1721888107",O_RDWR|O_CREAT|O_EXCL,0600) = 46 (0x2e)
> > > > ftruncate(46,0x400000) = 0 (0x0)
> > > Try to write zeroes instead of truncating.
> > > This should activate the fast path in the fault handler, and if the
> > > pages allocated for backing store of the shm object were from reservation,
> > > you should get superpage mapping on the first fault without promotion.
> >
> > If you just write() to a newly shm_open()'d fd you get a return code
> > of 0 so I assume that doesn't work. If you ftruncate() to the desired
> > size first, then loop writing 8192 bytes of zeroes at a time, it
> > works. But still no super pages. I tried also with a write buffer of
> > 2MB of zeroes, but still no super pages. I tried abandoning
> > shm_open() and instead using a mapped file, and still no super pages.
>
> I did not quite scientific experiment, but you would need to try to find
> the differences between what I did and what you observe. Below is the
> naive test program that directly implements my suggestion, and the
> output from the procstat -v for it after all things were set up.
>
...
> 98579 0x800e00000 0x801200000 rw- 1024 1030 3 0 --S- df
Huh. Your program doesn't result in an S mapping on my laptop, but I
tried on an EC2 t2.2xlarge machine and there it promotes to S, even if
I comment out the write() loop (the loop that assigned to every byte
is enough). The difference might be the amount of memory on the
system: on my 4GB laptop, it is very reluctant to use super pages (but
I have seen it do it, so I know it can). On a 32GB system, it does it
immediately, and it works nicely for PostgreSQL too. So perhaps my
problem is testing on a small RAM system, though I don't understand
why.
From owner-freebsd-hackers@freebsd.org Sun Oct 14 09:58:31 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 06D4810CD050
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Sun, 14 Oct 2018 09:58:31 +0000 (UTC) (envelope-from lev@FreeBSD.org)
Received: from onlyone.not-for.work (onlyone.not-for.work [148.251.9.81])
by mx1.freebsd.org (Postfix) with ESMTP id A0271819AB
for <freebsd-hackers@FreeBSD.org>; Sun, 14 Oct 2018 09:58:30 +0000 (UTC)
(envelope-from lev@FreeBSD.org)
Received: from lion.home.serebryakov.spb.ru (unknown
[IPv6:2001:470:923f:1:c146:1b80:de86:38c8])
(Authenticated sender: lev@serebryakov.spb.ru)
by onlyone.not-for.work (Postfix) with ESMTPSA id 67E0D899;
Sun, 14 Oct 2018 12:58:23 +0300 (MSK)
Date: Sun, 14 Oct 2018 12:58:25 +0300
From: Lev Serebryakov <lev@FreeBSD.org>
Reply-To: Lev Serebryakov <lev@FreeBSD.org>
Organization: FreeBSD
Message-ID: <1822540947.20181014125825@serebryakov.spb.ru>
To: "Kristof Provost" <kristof@sigsegv.be>
CC: freebsd-hackers@FreeBSD.org
Subject: Re: What are ck_queue.h guarantees?
In-Reply-To: <AD1EC8D5-C137-4128-90C9-0FCAECC5754C@sigsegv.be>
References: <1551957390.20181013184006@serebryakov.spb.ru>
<AD1EC8D5-C137-4128-90C9-0FCAECC5754C@sigsegv.be>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: base64
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sun, 14 Oct 2018 09:58:31 -0000
SGVsbG8gS3Jpc3RvZiwNCg0KU3VuZGF5LCBPY3RvYmVyIDE0LCAyMDE4LCA5OjE0OjMxIEFN
LCB5b3Ugd3JvdGU6DQoNCj4+ICAgQnV0IGluIGFsbCBwbGFjZXMgYXQga2VybmVsIEkgcGVl
a2VkLCBDS19YWFhYIG1hY3JvcyBhcmUgcHJvdGVjdGVkIGJ5DQo+PiAgbG9ja3MuIFllcywg
ZXZlbiByZWFkIG9uZXMuDQoNCj4gTm90ZSB0aGF0IHRoZSBpbXBsZW1lbnRhdGlvbiBvZiBp
Zl9tYWRkcl9ybG9jaygpIGRvZXNu4oCZdCBhY3R1YWxseSB0YWtlIGENCj4gbG9jay4gSW5z
dGVhZCBpdCBjYWxscyBlcG9jaF9lbnRlcl9wcmVlbXB0KCkuDQogT2gsIEkndmUgc2hvdWxk
IGxvb2sgaW5zaWRlIHRoZXNlIGZ1bmN0aW9ucyBiZWZvcmUgYXNraW5nLiBUaGFuayB5b3Ug
Zm9yDQpwb2ludGluZyBtZSBvdXQuDQoNCj4gSSBzdXNwZWN0IHRoZSDigJhsb2Nr4oCZL+KA
mHVubG9ja+KAmSBuYW1pbmcgaXMgbW9zdGx5IGhpc3RvcmljYWwgaGVyZTogaS5lLiBpdA0K
PiB1c2VkIHRvIGJlIGEgcmVhbCBsb2NrLCBhbmQgd2hlbiBpdCB3YXMgcmVwbGFjZWQgYnkg
dGhlIGVwb2NoLWJhc2VkDQo+IGFwcHJvYWNoIHRoZSBmdW5jdGlvbnMgd2VyZSBub3QgcmVu
YW1lZC4NCiAgWWVwLCBhbmQgaXQgaXMgdmVyeSBtaXNsZWFkaW5nIDotKQ0KDQoNCi0tIA0K
QmVzdCByZWdhcmRzLA0KIExldiAgICAgICAgICAgICAgICAgICAgICAgICAgICBtYWlsdG86
bGV2QEZyZWVCU0Qub3Jn
From owner-freebsd-hackers@freebsd.org Sun Oct 14 11:46:00 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8932710D007E
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Sun, 14 Oct 2018 11:46:00 +0000 (UTC)
(envelope-from kostikbel@gmail.com)
Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 14D6184792;
Sun, 14 Oct 2018 11:45:59 +0000 (UTC)
(envelope-from kostikbel@gmail.com)
Received: from tom.home (kib@localhost [127.0.0.1])
by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id w9EBjjdj013383
(version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO);
Sun, 14 Oct 2018 14:45:48 +0300 (EEST)
(envelope-from kostikbel@gmail.com)
DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua w9EBjjdj013383
Received: (from kostik@localhost)
by tom.home (8.15.2/8.15.2/Submit) id w9EBjivo013382;
Sun, 14 Oct 2018 14:45:44 +0300 (EEST)
(envelope-from kostikbel@gmail.com)
X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com
using -f
Date: Sun, 14 Oct 2018 14:45:44 +0300
From: Konstantin Belousov <kostikbel@gmail.com>
To: Thomas Munro <munro@ip9.org>
Cc: freebsd-hackers@freebsd.org, alc@freebsd.org, mjg@freebsd.org
Subject: Re: PostgresSQL vs super pages
Message-ID: <20181014114544.GA5335@kib.kiev.ua>
References: <CADLWmXU=7QM-oHmY=TMAQanQE-dnXY4v74Zm1kkEz3Gc=ip21A@mail.gmail.com>
<20181011001954.GV5335@kib.kiev.ua>
<CADLWmXWS6qjt02bxUkd7BewfhYw69at8OYe+h15+1OCpnpi6ng@mail.gmail.com>
<20181013235021.GX5335@kib.kiev.ua>
<CADLWmXVN=anzRfP0kGkCVW2NR+u9Gcx=O1GVdbn_ZJuvzC7gHg@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CADLWmXVN=anzRfP0kGkCVW2NR+u9Gcx=O1GVdbn_ZJuvzC7gHg@mail.gmail.com>
User-Agent: Mutt/1.10.1 (2018-07-13)
X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00,
DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM,
NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sun, 14 Oct 2018 11:46:00 -0000
On Sun, Oct 14, 2018 at 10:58:08PM +1300, Thomas Munro wrote:
> On Sun, 14 Oct 2018 at 12:50, Konstantin Belousov <kostikbel@gmail.com> wrote:
> > On Thu, Oct 11, 2018 at 02:01:20PM +1300, Thomas Munro wrote:
> > > On Thu, 11 Oct 2018 at 13:20, Konstantin Belousov <kostikbel@gmail.com> wrote:
> > > > On Thu, Oct 11, 2018 at 12:59:41PM +1300, Thomas Munro wrote:
> > > > > shm_open("/PostgreSQL.1721888107",O_RDWR|O_CREAT|O_EXCL,0600) = 46 (0x2e)
> > > > > ftruncate(46,0x400000) = 0 (0x0)
> > > > Try to write zeroes instead of truncating.
> > > > This should activate the fast path in the fault handler, and if the
> > > > pages allocated for backing store of the shm object were from reservation,
> > > > you should get superpage mapping on the first fault without promotion.
> > >
> > > If you just write() to a newly shm_open()'d fd you get a return code
> > > of 0 so I assume that doesn't work. If you ftruncate() to the desired
> > > size first, then loop writing 8192 bytes of zeroes at a time, it
> > > works. But still no super pages. I tried also with a write buffer of
> > > 2MB of zeroes, but still no super pages. I tried abandoning
> > > shm_open() and instead using a mapped file, and still no super pages.
> >
> > I did not quite scientific experiment, but you would need to try to find
> > the differences between what I did and what you observe. Below is the
> > naive test program that directly implements my suggestion, and the
> > output from the procstat -v for it after all things were set up.
> >
> ...
> > 98579 0x800e00000 0x801200000 rw- 1024 1030 3 0 --S- df
>
> Huh. Your program doesn't result in an S mapping on my laptop, but I
> tried on an EC2 t2.2xlarge machine and there it promotes to S, even if
> I comment out the write() loop (the loop that assigned to every byte
> is enough). The difference might be the amount of memory on the
> system: on my 4GB laptop, it is very reluctant to use super pages (but
> I have seen it do it, so I know it can). On a 32GB system, it does it
> immediately, and it works nicely for PostgreSQL too. So perhaps my
> problem is testing on a small RAM system, though I don't understand
> why.
How many free memory does your system have ? Free as reported by top. If
the free memory is low and fragmented, and I suppose it is on 4G laptop
which you use with X, browser and other memory-consuming applications,
system would have troubles filling the reverve, i.e reserving 2M of
2M-aligned physical pages.
You can try the test programs right after booting into single user mode.
From owner-freebsd-hackers@freebsd.org Sun Oct 14 22:42:29 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8A6CC10C55D0
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Sun, 14 Oct 2018 22:42:29 +0000 (UTC)
(envelope-from munro@penski.net)
Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com
[IPv6:2a00:1450:4864:20::529])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(Client CN "smtp.gmail.com",
Issuer "Google Internet Authority G3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 0B19D788B1
for <freebsd-hackers@freebsd.org>; Sun, 14 Oct 2018 22:42:28 +0000 (UTC)
(envelope-from munro@penski.net)
Received: by mail-ed1-x529.google.com with SMTP id g32-v6so16064193edg.13
for <freebsd-hackers@freebsd.org>; Sun, 14 Oct 2018 15:42:28 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=ip9-org.20150623.gappssmtp.com; s=20150623;
h=mime-version:references:in-reply-to:from:date:message-id:subject:to
:cc; bh=JhoGCNnjojkZgEkWlJnQGBtW6M+b+ywZZmJcLQrcgfY=;
b=wPN3PEgHllhi9mWBJyzzmKXqC9x4aMKw2tBvipYmHKsZYASDZAQUVebWPfgWAQkQJY
EKXqg2JUecluT3/AgusgfFznYCKQCxRskbDmue427/a2vQK76l5ubtXIdI2kSQ9x//76
oK1vM5CXWIQyMWbHmuslVezO2Gpv84+itVpU3TzlMAE7Ghpa05Cbeyeud7fZS3MGVedU
ZWeyrTUPmwe6A1cYpZjPXJUCJBPQrO7qkMMPJGFv2WyVcuM+MDbowExt/Aa6fmYSiHz+
At4jTQPZq7RVsBXdN3kv5gkR9kcICFOKTe/bP/1lzCY5NFqmt/K4sjaxsyHVYi2Z0Lbw
YPGQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:mime-version:references:in-reply-to:from:date
:message-id:subject:to:cc;
bh=JhoGCNnjojkZgEkWlJnQGBtW6M+b+ywZZmJcLQrcgfY=;
b=WBgR6KtsAOmcT3RuaCIGcZQRsju61oZM38KDSbptdM0U0HE5IgtMzExXY6hYYLXQ7w
/66Ob2G3snfazMEPMNtHNczI/2Az73F8zHvZ2KmrqKUtKj+puaGuXSmGZfZnPZgJlGEv
+dsx1uc7rLPpnpsOthsOp5Dikng+7YZdccLZ64lgw4AoG++65NQPyHApB5lgUENWtnQM
0o6Bk8+o/4EBfNuUebvxbnjQKxNh/i+YLO3d2FigXGqWd0nshziH1GB7NnPGU8CVeqgj
9QosjF4Hr1y9QiRqn2m68U0JnBEgNpE9D56yZB/c29w4BvbyiPvJSl3T677Y4l7YFlbV
vM6A==
X-Gm-Message-State: ABuFfohJ5kD/dM2Ni6BSRRzAu+x1TWg0eka/Ifvm7fRQiyfbOyETPFo5
vTUcsw5qPVaPHofb7g2ex9rqyz2ThLHjuzxytZrs0A==
X-Google-Smtp-Source: ACcGV63KZwb96A4SXeBxwwzE+ftPvaHSiRfb1hXw6VQPRdcFehXuwBLEEG4jsCjUXMSv5+0MzlLLFxzQNzQabNCnH60=
X-Received: by 2002:a50:9806:: with SMTP id
g6-v6mr20793204edb.235.1539556947467;
Sun, 14 Oct 2018 15:42:27 -0700 (PDT)
MIME-Version: 1.0
References: <CADLWmXU=7QM-oHmY=TMAQanQE-dnXY4v74Zm1kkEz3Gc=ip21A@mail.gmail.com>
<20181011001954.GV5335@kib.kiev.ua>
<CADLWmXWS6qjt02bxUkd7BewfhYw69at8OYe+h15+1OCpnpi6ng@mail.gmail.com>
<20181013235021.GX5335@kib.kiev.ua>
<CADLWmXVN=anzRfP0kGkCVW2NR+u9Gcx=O1GVdbn_ZJuvzC7gHg@mail.gmail.com>
<20181014114544.GA5335@kib.kiev.ua> <20181014223349.GA9022@raichu>
In-Reply-To: <20181014223349.GA9022@raichu>
From: Thomas Munro <munro@ip9.org>
Date: Mon, 15 Oct 2018 11:42:15 +1300
Message-ID: <CADLWmXUaJMBSV=W7qSGk5kKe14jSKhM7yqH8bKRrrFO0qCi=cw@mail.gmail.com>
Subject: Re: PostgresSQL vs super pages
To: markj@freebsd.org
Cc: Konstantin Belousov <kostikbel@gmail.com>, alc@freebsd.org,
freebsd-hackers@freebsd.org, mjg@freebsd.org
Content-Type: text/plain; charset="UTF-8"
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sun, 14 Oct 2018 22:42:29 -0000
On Mon, 15 Oct 2018 at 11:33, Mark Johnston <markj@freebsd.org> wrote:
> On Sun, Oct 14, 2018 at 02:45:44PM +0300, Konstantin Belousov wrote:
> > On Sun, Oct 14, 2018 at 10:58:08PM +1300, Thomas Munro wrote:
> > > Huh. Your program doesn't result in an S mapping on my laptop, but I
> > > tried on an EC2 t2.2xlarge machine and there it promotes to S, even if
> > > I comment out the write() loop (the loop that assigned to every byte
> > > is enough). The difference might be the amount of memory on the
> > > system: on my 4GB laptop, it is very reluctant to use super pages (but
> > > I have seen it do it, so I know it can). On a 32GB system, it does it
> > > immediately, and it works nicely for PostgreSQL too. So perhaps my
> > > problem is testing on a small RAM system, though I don't understand
> > > why.
> > How many free memory does your system have ? Free as reported by top. If
> > the free memory is low and fragmented, and I suppose it is on 4G laptop
> > which you use with X, browser and other memory-consuming applications,
> > system would have troubles filling the reverve, i.e reserving 2M of
> > 2M-aligned physical pages.
>
> BTW, this can be explicitly verified with the sysctl vm.phys_free
> sysctl. Superpage promotion requires free 2MB chunks from freelist 0,
> pool 0.
Ah, I see. Straight after rebooting without X I get super pages and
vm.phys_free looks more healthy. I'd observed the same problem on
other machines including servers with a bit (but not a lot) more
memory, but clearly none of my FreeBSD systems are currently big
enough to keep suitable chunks around on the freelist. I wonder if
ZFS is a factor. Well, this was educational. Thanks very much for
your help!
From owner-freebsd-hackers@freebsd.org Sun Oct 14 22:33:59 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id DFA5A10C4CE8
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Sun, 14 Oct 2018 22:33:58 +0000 (UTC)
(envelope-from markjdb@gmail.com)
Received: from mail-pf1-x441.google.com (mail-pf1-x441.google.com
[IPv6:2607:f8b0:4864:20::441])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(Client CN "smtp.gmail.com",
Issuer "Google Internet Authority G3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 4CEB77812F;
Sun, 14 Oct 2018 22:33:58 +0000 (UTC)
(envelope-from markjdb@gmail.com)
Received: by mail-pf1-x441.google.com with SMTP id l17-v6so8687253pff.2;
Sun, 14 Oct 2018 15:33:58 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
h=sender:date:from:to:cc:subject:message-id:references:mime-version
:content-disposition:in-reply-to:user-agent;
bh=wJ/Eu9M8pL1T+BEmcfAbfudEWLX5p3XhimN/kHJ0ZJc=;
b=UBMA+pNCJKKpzrpYtVosy9a8jxZrZcV1J8KZsyctuoudYMyY/+PVA4vt7uIyrwchQ+
tf0YPdVVsIEHo1yWs9NesaqVlGpplwpE6foATryQGwPApdfQeunskO0CZbRLUN7Achpz
9gI7lAPaGc1U0OWjnwTHPEn5HoTHkGVo/GctrJo4briGCngtG4SRnbICyL8I2v1IoR+x
1U7jRnL7lIPrO0Q99rG6AJKPv+3B5jwSOY1dMqjWvZ8gAEu+tS/xkMGnO4Xwe8Ae+zEl
jznSozv8BraKE803K67TIIgYEyai7GbdznTfqS4JBgh7ZfO+MVP0crEiJ90xnggPUoYT
4JZg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:sender:date:from:to:cc:subject:message-id
:references:mime-version:content-disposition:in-reply-to:user-agent;
bh=wJ/Eu9M8pL1T+BEmcfAbfudEWLX5p3XhimN/kHJ0ZJc=;
b=mQhuCJVnGAOlkySwomIQCfkylEax4g0EDBp/yNBM433jq2iDpQOcY2fnU3Zqv+uCKV
S1fO2G4cjJgfyr1m7olZDLU6pHvQ3fboRE8AOJPDw3pPVAvNBnlIGdkE+YKJWWSoRloD
74BFBJYOsO9zP7A3xW1RiXR/bOx0ZSKr75MC5hQ2FiCJL2g9r7Tt2bfR0TSKup8ei+eI
S76rDHs90YgNtd30ad2L8F9uwWfLshsKdRhnCo83WHMPcDhlUf9Xna0060SmR1m1jnLt
fnEBuFchxQR7z5JTYuiBz+rJYYFLr2HRFwMwyz/ahve1DPYq2GXI4wC0nZ+r651d3wuQ
Xo4g==
X-Gm-Message-State: ABuFfohWopy1q9yR4B0o0nzwNJ5+P71RcBPu5PRrz7lhvv8/IYSaFXZn
1wZAy52p/RlP0i+XLZCMkVL7y+LR
X-Google-Smtp-Source: ACcGV60kZfdrB6vswa+8PmIEorKePATWYMf0pjpS/ofCLLvB7UASOPiQWLlwMLTYf0oGC7FjIBXXGw==
X-Received: by 2002:a62:68c3:: with SMTP id
d186-v6mr6813478pfc.195.1539556437115;
Sun, 14 Oct 2018 15:33:57 -0700 (PDT)
Received: from raichu (toroon0560w-lp130-09-70-52-226-56.dsl.bell.ca.
[70.52.226.56])
by smtp.gmail.com with ESMTPSA id 84-v6sm10374363pfv.33.2018.10.14.15.33.54
(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
Sun, 14 Oct 2018 15:33:55 -0700 (PDT)
Sender: Mark Johnston <markjdb@gmail.com>
Date: Sun, 14 Oct 2018 18:33:49 -0400
From: Mark Johnston <markj@freebsd.org>
To: Konstantin Belousov <kostikbel@gmail.com>
Cc: Thomas Munro <munro@ip9.org>, alc@freebsd.org,
freebsd-hackers@freebsd.org, mjg@freebsd.org
Subject: Re: PostgresSQL vs super pages
Message-ID: <20181014223349.GA9022@raichu>
References: <CADLWmXU=7QM-oHmY=TMAQanQE-dnXY4v74Zm1kkEz3Gc=ip21A@mail.gmail.com>
<20181011001954.GV5335@kib.kiev.ua>
<CADLWmXWS6qjt02bxUkd7BewfhYw69at8OYe+h15+1OCpnpi6ng@mail.gmail.com>
<20181013235021.GX5335@kib.kiev.ua>
<CADLWmXVN=anzRfP0kGkCVW2NR+u9Gcx=O1GVdbn_ZJuvzC7gHg@mail.gmail.com>
<20181014114544.GA5335@kib.kiev.ua>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20181014114544.GA5335@kib.kiev.ua>
User-Agent: Mutt/1.10.1 (2018-07-13)
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sun, 14 Oct 2018 22:33:59 -0000
On Sun, Oct 14, 2018 at 02:45:44PM +0300, Konstantin Belousov wrote:
> On Sun, Oct 14, 2018 at 10:58:08PM +1300, Thomas Munro wrote:
> > On Sun, 14 Oct 2018 at 12:50, Konstantin Belousov <kostikbel@gmail.com> wrote:
> > > On Thu, Oct 11, 2018 at 02:01:20PM +1300, Thomas Munro wrote:
> > > > On Thu, 11 Oct 2018 at 13:20, Konstantin Belousov <kostikbel@gmail.com> wrote:
> > > > > On Thu, Oct 11, 2018 at 12:59:41PM +1300, Thomas Munro wrote:
> > > > > > shm_open("/PostgreSQL.1721888107",O_RDWR|O_CREAT|O_EXCL,0600) = 46 (0x2e)
> > > > > > ftruncate(46,0x400000) = 0 (0x0)
> > > > > Try to write zeroes instead of truncating.
> > > > > This should activate the fast path in the fault handler, and if the
> > > > > pages allocated for backing store of the shm object were from reservation,
> > > > > you should get superpage mapping on the first fault without promotion.
> > > >
> > > > If you just write() to a newly shm_open()'d fd you get a return code
> > > > of 0 so I assume that doesn't work. If you ftruncate() to the desired
> > > > size first, then loop writing 8192 bytes of zeroes at a time, it
> > > > works. But still no super pages. I tried also with a write buffer of
> > > > 2MB of zeroes, but still no super pages. I tried abandoning
> > > > shm_open() and instead using a mapped file, and still no super pages.
> > >
> > > I did not quite scientific experiment, but you would need to try to find
> > > the differences between what I did and what you observe. Below is the
> > > naive test program that directly implements my suggestion, and the
> > > output from the procstat -v for it after all things were set up.
> > >
> > ...
> > > 98579 0x800e00000 0x801200000 rw- 1024 1030 3 0 --S- df
> >
> > Huh. Your program doesn't result in an S mapping on my laptop, but I
> > tried on an EC2 t2.2xlarge machine and there it promotes to S, even if
> > I comment out the write() loop (the loop that assigned to every byte
> > is enough). The difference might be the amount of memory on the
> > system: on my 4GB laptop, it is very reluctant to use super pages (but
> > I have seen it do it, so I know it can). On a 32GB system, it does it
> > immediately, and it works nicely for PostgreSQL too. So perhaps my
> > problem is testing on a small RAM system, though I don't understand
> > why.
> How many free memory does your system have ? Free as reported by top. If
> the free memory is low and fragmented, and I suppose it is on 4G laptop
> which you use with X, browser and other memory-consuming applications,
> system would have troubles filling the reverve, i.e reserving 2M of
> 2M-aligned physical pages.
BTW, this can be explicitly verified with the sysctl vm.phys_free
sysctl. Superpage promotion requires free 2MB chunks from freelist 0,
pool 0.
>
> You can try the test programs right after booting into single user mode.
From owner-freebsd-hackers@freebsd.org Mon Oct 15 05:40:35 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8AA3410CF872
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Mon, 15 Oct 2018 05:40:35 +0000 (UTC)
(envelope-from sebastian.huber@embedded-brains.de)
Received: from dedi548.your-server.de (dedi548.your-server.de
[IPv6:2a01:4f8:d0a:2645::2])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 2427284735
for <freebsd-hackers@freebsd.org>; Mon, 15 Oct 2018 05:40:34 +0000 (UTC)
(envelope-from sebastian.huber@embedded-brains.de)
Received: from [78.46.172.3] (helo=sslproxy06.your-server.de)
by dedi548.your-server.de with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256)
(Exim 4.89_1) (envelope-from <sebastian.huber@embedded-brains.de>)
id 1gBvcA-00049i-PU
for freebsd-hackers@freebsd.org; Mon, 15 Oct 2018 07:40:26 +0200
Received: from [82.135.62.35] (helo=mail.embedded-brains.de)
by sslproxy06.your-server.de with esmtpsa
(TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89)
(envelope-from <sebastian.huber@embedded-brains.de>)
id 1gBvcA-00040D-KF
for freebsd-hackers@freebsd.org; Mon, 15 Oct 2018 07:40:26 +0200
Received: from localhost (localhost.localhost [127.0.0.1])
by mail.embedded-brains.de (Postfix) with ESMTP id 5438F2A0043
for <freebsd-hackers@freebsd.org>; Mon, 15 Oct 2018 07:40:25 +0200 (CEST)
Received: from mail.embedded-brains.de ([127.0.0.1])
by localhost (zimbra.eb.localhost [127.0.0.1]) (amavisd-new, port 10032)
with ESMTP id kvif6HMS__03 for <freebsd-hackers@freebsd.org>;
Mon, 15 Oct 2018 07:40:23 +0200 (CEST)
Received: from localhost (localhost.localhost [127.0.0.1])
by mail.embedded-brains.de (Postfix) with ESMTP id 200E62A165C
for <freebsd-hackers@freebsd.org>; Mon, 15 Oct 2018 07:40:23 +0200 (CEST)
X-Virus-Scanned: amavisd-new at zimbra.eb.localhost
Received: from mail.embedded-brains.de ([127.0.0.1])
by localhost (zimbra.eb.localhost [127.0.0.1]) (amavisd-new, port 10026)
with ESMTP id e81Csj7H17qV for <freebsd-hackers@freebsd.org>;
Mon, 15 Oct 2018 07:40:23 +0200 (CEST)
Received: from huber-nb-linux.suse (unknown [192.168.96.161])
by mail.embedded-brains.de (Postfix) with ESMTPSA id 0318C2A0043
for <freebsd-hackers@freebsd.org>; Mon, 15 Oct 2018 07:40:23 +0200 (CEST)
To: FreeBSD <freebsd-hackers@freebsd.org>
From: Sebastian Huber <sebastian.huber@embedded-brains.de>
Subject: Why SPDX-License-Identifier: BSD-2-Clause-FreeBSD?
Message-ID: <f396d707-a707-837e-e76f-fec3a7fba2cf@embedded-brains.de>
Date: Mon, 15 Oct 2018 07:40:23 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
Thunderbird/60.2.1
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: quoted-printable
X-Authenticated-Sender: smtp-embedded@poldinet.de
X-Virus-Scanned: Clear (ClamAV 0.100.1/25038/Mon Oct 15 06:54:43 2018)
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 15 Oct 2018 05:40:35 -0000
Hello,
I noticed that a lot of files have this SPDX-License-Identifier:
BSD-2-Clause-FreeBSD
I didn't find this identifier at https://opensource.org/licenses, e.g.
https://opensource.org/licenses/BSD-2-Clause-FreeBSD
doesn't exist. I didn't find a difference to
https://opensource.org/licenses/BSD-2-Clause
What is the reason for this specialized identifier?
--=20
Sebastian Huber, embedded brains GmbH
Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone : +49 89 189 47 41-16
Fax : +49 89 189 47 41-09
E-Mail : sebastian.huber@embedded-brains.de
PGP : Public key available on request.
Diese Nachricht ist keine gesch=C3=A4ftliche Mitteilung im Sinne des EHUG=
.
From owner-freebsd-hackers@freebsd.org Mon Oct 15 05:45:09 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 366BE10CFC45
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Mon, 15 Oct 2018 05:45:09 +0000 (UTC)
(envelope-from allanjude@freebsd.org)
Received: from mx1.scaleengine.net (mx1.scaleengine.net [209.51.186.6])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id BDF2084B58
for <freebsd-hackers@freebsd.org>; Mon, 15 Oct 2018 05:45:08 +0000 (UTC)
(envelope-from allanjude@freebsd.org)
Received: from [10.1.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28])
(Authenticated sender: allanjude.freebsd@scaleengine.com)
by mx1.scaleengine.net (Postfix) with ESMTPSA id 533221E8FB
for <freebsd-hackers@freebsd.org>; Mon, 15 Oct 2018 05:45:02 +0000 (UTC)
Subject: Re: Why SPDX-License-Identifier: BSD-2-Clause-FreeBSD?
To: freebsd-hackers@freebsd.org
References: <f396d707-a707-837e-e76f-fec3a7fba2cf@embedded-brains.de>
From: Allan Jude <allanjude@freebsd.org>
Openpgp: preference=signencrypt
Autocrypt: addr=allanjude@freebsd.org; prefer-encrypt=mutual; keydata=
xsFNBFVwZcYBEADwrZDH0xe0ZVjc9ORCc6PcBLwS/RTXA6NkvpD6ea02pZ8lPOVgteuuugFc
D34LdDbiWr+479vfrKBh+Y38GL0oZ0/13j10tIlDMHSa5BU0y6ACtnhupFvVlQ57+XaJAb/q
7qkfSiuxVwQ3FY3PL3cl1RrIP5eGHLA9hu4eVbu+FOX/q/XVKz49HaeIaxzo2Q54572VzIo6
C28McX9m65UL5fXMUGJDDLCItLmehZlHsQQ+uBxvODLFpVV2lUgDR/0rDa0B9zHZX8jY8qQ7
ZdCSy7CwClXI054CkXZCaBzgxYh/CotdI8ezmaw7NLs5vWNTxaDEFXaFMQtMVhvqQBpHkfOD
7rjjOmFw00nJL4FuPE5Yut0CPyx8vLjVmNJSt/Y8WxxmhutsqJYFgYfWl/vaWkrFLur/Zcmz
IklwLw35HLsCZytCN5A3rGKdRbQjD6QPXOTJu0JPrJF6t2xFkWAT7oxnSV0ELhl2g+JfMMz2
Z1PDmS3NRnyEdqEm7NoRGXJJ7bgxDbN+9SXTyOletqGNXj/bSrBvhvZ0RQrzdHAPwQUfVSU2
qBhQEi2apSZstgVNMan0GUPqCdbE2zpysg+zT7Yhvf9EUQbzPL4LpdK1llT9fZbrdMzEXvEF
oSvwJFdV3sqKmZc7b+E3PuxK6GTsKqaukd/3Cj8aLHG1T1im1QARAQABzSJBbGxhbiBKdWRl
IDxhbGxhbmp1ZGVAZnJlZWJzZC5vcmc+wsF/BBMBAgApBQJVcGXGAhsjBQkSzAMABwsJCAcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQGZU1PhKYC34Muw/+JOKpSfhhysWFYiRXynGRDe07
Z6pVsn7DzrPUMRNZfHu8Uujmmy3p2nx9FelIY9yjd2UKHhug+whM54MiIFs90eCRVa4XEsPR
4FFAm0DAWrrb7qhZFcE/GhHdRWpZ341WAElWf6Puj2devtRjfYbikvj5+1V1QmDbju7cEw5D
mEET44pTuD2VMRJpu2yZZzkM0i+wKFuPxlhqreufA1VNkZXI/rIfkYWK+nkXd9Efw3YdCyCQ
zUgTUCb88ttSqcyhik/li1CDbXBpkzDCKI6I/8fAb7jjOC9LAtrZJrdgONywcVFoyK9ZN7EN
AVA+xvYCmuYhR/3zHWH1g4hAm1v1+gIsufhajhfo8/wY1SetlzPaYkSkVQLqD8T6zZyhf+AN
bC7ci44UsiKGAplB3phAXrtSPUEqM86kbnHg3fSx37kWKUiYNOnx4AC2VXvEiKsOBlpyt3dw
WQbOtOYM+vkfbBwDtoGOOPYAKxc4LOIt9r+J8aD+gTooi9Eo5tvphATf9WkCpl9+aaGbSixB
tUpvQMRnSMqTqq4Z7DeiG6VMRQIjsXDSLJEUqcfhnLFo0Ko/RiaHd5xyAQ4DhQ9QpkyQjjNf
/3f/dYG7JAtoD30txaQ5V8uHrz210/77DRRX+HJjEj6xCxWUGvQgvEZf5XXyxeePvqZ+zQyT
DX61bYw6w6bOwU0EVXBlxgEQAMy7YVnCCLN4oAOBVLZ5nUbVPvpUhsdA94/0/P+uqCIh28Cz
ar56OCX0X19N/nAWecxL4H32zFbIRyDB2V/MEh4p9Qvyu/j4i1r3Ex5GhOT2hnit43Ng46z5
29Es4TijrHJP4/l/rB2VOqMKBS7Cq8zk1cWqaI9XZ59imxDNjtLLPPM+zQ1yE3OAMb475QwN
UgWxTMw8rkA7CEaqeIn4sqpTSD5C7kT1Bh26+rbgJDZ77D6Uv1LaCZZOaW52okW3bFbdozV8
yM2u+xz2Qs8bHz67p+s+BlygryiOyYytpkiK6Iy4N7FTolyj5EIwCuqzfk0SaRHeOKX2ZRjC
qatkgoD/t13PNT38V9tw3qZVOJDS0W6WM8VSg+F+bkM9LgJ8CmKV+Hj0k3pfGfYPOZJ/v18i
+SmZmL/Uw2RghnwDWGAsPCKu4uZR777iw7n9Io6Vfxndw2dcS0e9klvFYoaGS6H2F13Asygr
WBzFNGFQscN4mUW+ZYBzpTOcHkdT7w8WS55BmXYLna+dYer9/HaAuUrONjujukN4SPS1fMJ2
/CS/idAUKyyVVX5vozoNK2JVC1h1zUAVsdnmhEzNPsvBoqcVNfyqBFROEVLIPwq+lQMGNVjH
ekLTKRWf59MEhUC2ztjSKkGmwdg73d6xSXMuq45EgIJV2wPvOgWQonoHH/kxABEBAAHCwWUE
GAECAA8FAlVwZcYCGwwFCRLMAwAACgkQGZU1PhKYC34w5A//YViBtZyDV5O+SJT9FFO3lb9x
Zdxf0trA3ooCt7gdBkdnBM6T5EmjgVZ3KYYyFfwXZVkteuCCycMF/zVw5eE9FL1+zz9gg663
nY9q2F77TZTKXVWOLlOV2bY+xaK94U4ytogOGhh9b4UnQ/Ct3+6aviCF78Go608BXbmF/GVT
7uhddemk7ItxM1gE5Hscx3saxGKlayaOsdPKeGTVJCDEtHDuOc7/+jGh5Zxpk/Hpi+DUt1ot
8e6hPYLIQa4uVx4f1xxxV858PQ7QysSLr9pTV7FAQ18JclCaMc7JWIa3homZQL/MNKOfST0S
2e+msuRwQo7AnnfFKBUtb02KwpA4GhWryhkjUh/kbVc1wmGxaU3DgXYQ5GV5+Zf4kk/wqr/7
KG0dkTz6NLCVLyDlmAzuFhf66DJ3zzz4yIo3pbDYi3HB/BwJXVSKB3Ko0oUo+6/qMrOIS02L
s++QE/z7K12CCcs7WwOjfCYHK7VtE0Sr/PfybBdTbuDncOuAyAIeIKxdI2nmQHzl035hhvQX
s4CSghsP319jAOQiIolCeSbTMD4QWMK8RL/Pe1FI1jC3Nw9s+jq8Dudtbcj2UwAP/STUEbJ9
5rznzuuhPjE0e++EU/RpWmcaIMK/z1zZDMN+ce2v1qzgV936ZhJ3iaVzyqbEE81gDxg3P+IM
kiYh4ZtPB4Q=
Message-ID: <149f117b-bf85-0dce-e26c-c960ad93f7a4@freebsd.org>
Date: Mon, 15 Oct 2018 01:44:58 -0400
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101
Thunderbird/52.9.1
MIME-Version: 1.0
In-Reply-To: <f396d707-a707-837e-e76f-fec3a7fba2cf@embedded-brains.de>
Content-Type: multipart/signed; micalg=pgp-sha1;
protocol="application/pgp-signature";
boundary="wEZxUCpt2eOGVkoTnPTJ0WeX8Qg0QNwHO"
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 15 Oct 2018 05:45:09 -0000
This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--wEZxUCpt2eOGVkoTnPTJ0WeX8Qg0QNwHO
Content-Type: multipart/mixed; boundary="RBRRas9eht5LmgtUCHdHcWZZQJksObmVO";
protected-headers="v1"
From: Allan Jude <allanjude@freebsd.org>
To: freebsd-hackers@freebsd.org
Message-ID: <149f117b-bf85-0dce-e26c-c960ad93f7a4@freebsd.org>
Subject: Re: Why SPDX-License-Identifier: BSD-2-Clause-FreeBSD?
References: <f396d707-a707-837e-e76f-fec3a7fba2cf@embedded-brains.de>
In-Reply-To: <f396d707-a707-837e-e76f-fec3a7fba2cf@embedded-brains.de>
--RBRRas9eht5LmgtUCHdHcWZZQJksObmVO
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
On 2018-10-15 01:40, Sebastian Huber wrote:
> Hello,
>=20
> I noticed that a lot of files have this SPDX-License-Identifier:
>=20
> BSD-2-Clause-FreeBSD
>=20
> I didn't find this identifier at https://opensource.org/licenses, e.g.
>=20
> https://opensource.org/licenses/BSD-2-Clause-FreeBSD
>=20
> doesn't exist. I didn't find a difference to
>=20
> https://opensource.org/licenses/BSD-2-Clause
>=20
> What is the reason for this specialized identifier?
>=20
When opensource.org created their 'official' BSD license, they deviated
from the wording used in the real BSD license.
The FreeBSD license specifically says "PROVIDED BY THE AUTHOR AND
CONTRIBUTORS" and the OSI version says "PROVIDED BY THE COPYRIGHT
HOLDERS AND CONTRIBUTORS"
There can be a significant difference between those two terms, making it
an entirely different license.
--=20
Allan Jude
--RBRRas9eht5LmgtUCHdHcWZZQJksObmVO--
--wEZxUCpt2eOGVkoTnPTJ0WeX8Qg0QNwHO
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (MingW32)
iQIcBAEBAgAGBQJbxCldAAoJEBmVNT4SmAt+Q6cP/3R1MYvU4kKIKU2lUFCOF1w7
IsEFyGsCOdcr5PnTybcYZrBQ2+3ykNCXqNFEHe1YtIoPjDA1X43li67IkH/9w6x3
CgEpMvQIfewPuc1GkoREh51sv8H9zklTWPJao/dlTauZf3jjMdbpIRG5Nf0gJgc+
kHoBvPz1jjyUTlFXwDyTsIidSyy3T3yB2zx1/FumogDb51jrbUafQ19kECuVDOD5
7GdnaWkHYf7raB2foB5oFUkqLWpqSr+8q3zmwJHG52qQHQqQ2IzApKcHRQo+NYf8
wp1W3YX7DocpxhcqjlkpQnR1oit2zS+iN/9Hy0jwhZ+UMazZHszUgJgpVtmT+ygW
MLU3444Weti3rCBW6WmSf9BFBgob9/ZAP/dNIURLyuoWewoviwRGEh6mEiRSOTRw
J0l255dQZIhYGjZ7fAdeEIWU7oRT5q1max/3nPjrLbse9MSwWJ7+xmKrs8kjtuGF
tSCzxO5B9x4qhMsHsPgQ0C8+xRliTKDr6ngtaC1Q80Ifkcj3BGIskD5el907vgG2
wUY3rEmoXPuDPVwBHBsZ9VglvPV06wB7a6mSLX5VzQmiuFtT+jiOD5Y95uhPe94T
mJll1yuRTvyfkrHwbO/kPFHgmsepjaaMy6VtxzuMetZUR+5LM8yFqtmSGGLkqnuo
+PZC+TcyAmmDk+E0XzGN
=chdA
-----END PGP SIGNATURE-----
--wEZxUCpt2eOGVkoTnPTJ0WeX8Qg0QNwHO--
From owner-freebsd-hackers@freebsd.org Mon Oct 15 07:03:49 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id D632410D178B
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Mon, 15 Oct 2018 07:03:49 +0000 (UTC)
(envelope-from sebastian.huber@embedded-brains.de)
Received: from dedi548.your-server.de (dedi548.your-server.de [85.10.215.148])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256
bits)) (Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 5688F87138;
Mon, 15 Oct 2018 07:03:49 +0000 (UTC)
(envelope-from sebastian.huber@embedded-brains.de)
Received: from [78.46.172.2] (helo=sslproxy05.your-server.de)
by dedi548.your-server.de with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256)
(Exim 4.89_1) (envelope-from <sebastian.huber@embedded-brains.de>)
id 1gBwup-000693-4S; Mon, 15 Oct 2018 09:03:47 +0200
Received: from [82.135.62.35] (helo=mail.embedded-brains.de)
by sslproxy05.your-server.de with esmtpsa
(TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89)
(envelope-from <sebastian.huber@embedded-brains.de>)
id 1gBwuo-0000vP-Uo; Mon, 15 Oct 2018 09:03:46 +0200
Received: from localhost (localhost.localhost [127.0.0.1])
by mail.embedded-brains.de (Postfix) with ESMTP id ABD9C2A0043;
Mon, 15 Oct 2018 09:03:45 +0200 (CEST)
Received: from mail.embedded-brains.de ([127.0.0.1])
by localhost (zimbra.eb.localhost [127.0.0.1]) (amavisd-new, port 10032)
with ESMTP id pHy50XdZF-Xo; Mon, 15 Oct 2018 09:03:43 +0200 (CEST)
Received: from localhost (localhost.localhost [127.0.0.1])
by mail.embedded-brains.de (Postfix) with ESMTP id 7ADF22A165C;
Mon, 15 Oct 2018 09:03:43 +0200 (CEST)
X-Virus-Scanned: amavisd-new at zimbra.eb.localhost
Received: from mail.embedded-brains.de ([127.0.0.1])
by localhost (zimbra.eb.localhost [127.0.0.1]) (amavisd-new, port 10026)
with ESMTP id UZf53bMlJCZm; Mon, 15 Oct 2018 09:03:43 +0200 (CEST)
Received: from huber-nb-linux.suse (unknown [192.168.96.161])
by mail.embedded-brains.de (Postfix) with ESMTPSA id 69EE12A0043;
Mon, 15 Oct 2018 09:03:43 +0200 (CEST)
Subject: Re: Why SPDX-License-Identifier: BSD-2-Clause-FreeBSD?
To: Allan Jude <allanjude@freebsd.org>, freebsd-hackers@freebsd.org
References: <f396d707-a707-837e-e76f-fec3a7fba2cf@embedded-brains.de>
<149f117b-bf85-0dce-e26c-c960ad93f7a4@freebsd.org>
From: Sebastian Huber <sebastian.huber@embedded-brains.de>
Message-ID: <fa33ae00-5772-a562-40c8-ffd4a6b0dc8a@embedded-brains.de>
Date: Mon, 15 Oct 2018 09:03:44 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
Thunderbird/60.2.1
MIME-Version: 1.0
In-Reply-To: <149f117b-bf85-0dce-e26c-c960ad93f7a4@freebsd.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: quoted-printable
X-Authenticated-Sender: smtp-embedded@poldinet.de
X-Virus-Scanned: Clear (ClamAV 0.100.1/25038/Mon Oct 15 06:54:43 2018)
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 15 Oct 2018 07:03:50 -0000
On 15/10/2018 07:44, Allan Jude wrote:
> On 2018-10-15 01:40, Sebastian Huber wrote:
>> Hello,
>>
>> I noticed that a lot of files have this SPDX-License-Identifier:
>>
>> BSD-2-Clause-FreeBSD
>>
>> I didn't find this identifier at https://opensource.org/licenses, e.g.
>>
>> https://opensource.org/licenses/BSD-2-Clause-FreeBSD
>>
>> doesn't exist. I didn't find a difference to
>>
>> https://opensource.org/licenses/BSD-2-Clause
>>
>> What is the reason for this specialized identifier?
>>
> When opensource.org created their 'official' BSD license, they deviated
> from the wording used in the real BSD license.
>
> The FreeBSD license specifically says "PROVIDED BY THE AUTHOR AND
> CONTRIBUTORS" and the OSI version says "PROVIDED BY THE COPYRIGHT
> HOLDERS AND CONTRIBUTORS"
>
> There can be a significant difference between those two terms, making i=
t
> an entirely different license.
In case the author of a FreeBSD source file is not the copyright holder,=20
then doesn't have FreeBSD a problem here?
--=20
Sebastian Huber, embedded brains GmbH
Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone : +49 89 189 47 41-16
Fax : +49 89 189 47 41-09
E-Mail : sebastian.huber@embedded-brains.de
PGP : Public key available on request.
Diese Nachricht ist keine gesch=C3=A4ftliche Mitteilung im Sinne des EHUG=
.
From owner-freebsd-hackers@freebsd.org Mon Oct 15 10:04:27 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5342010D5C18
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Mon, 15 Oct 2018 10:04:27 +0000 (UTC) (envelope-from joerg@bec.de)
Received: from relay3-d.mail.gandi.net (relay3-d.mail.gandi.net
[217.70.183.195])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id D10628D5EB
for <freebsd-hackers@freebsd.org>; Mon, 15 Oct 2018 10:04:26 +0000 (UTC)
(envelope-from joerg@bec.de)
X-Originating-IP: 93.205.162.222
Received: from britannica.bec.de (p5DCDA2DE.dip0.t-ipconnect.de
[93.205.162.222]) (Authenticated sender: joerg@bec.de)
by relay3-d.mail.gandi.net (Postfix) with ESMTPSA id 5031460019
for <freebsd-hackers@freebsd.org>; Mon, 15 Oct 2018 10:03:58 +0000 (UTC)
Date: Mon, 15 Oct 2018 12:03:56 +0200
From: Joerg Sonnenberger <joerg@bec.de>
To: freebsd-hackers@freebsd.org
Subject: Re: Why SPDX-License-Identifier: BSD-2-Clause-FreeBSD?
Message-ID: <20181015100356.GA26509@britannica.bec.de>
Mail-Followup-To: freebsd-hackers@freebsd.org
References: <f396d707-a707-837e-e76f-fec3a7fba2cf@embedded-brains.de>
<149f117b-bf85-0dce-e26c-c960ad93f7a4@freebsd.org>
<fa33ae00-5772-a562-40c8-ffd4a6b0dc8a@embedded-brains.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <fa33ae00-5772-a562-40c8-ffd4a6b0dc8a@embedded-brains.de>
User-Agent: Mutt/1.9.5 (2018-04-13)
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 15 Oct 2018 10:04:27 -0000
On Mon, Oct 15, 2018 at 09:03:44AM +0200, Sebastian Huber wrote:
> In case the author of a FreeBSD source file is not the copyright holder,
> then doesn't have FreeBSD a problem here?
I could submit a file (--> I'm the author) but assign the copyright to
the FreeBSD Foundation (--> TFF is the copyright holder).
Joerg
From owner-freebsd-hackers@freebsd.org Mon Oct 15 10:45:20 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7116310D6F11
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Mon, 15 Oct 2018 10:45:20 +0000 (UTC)
(envelope-from sebastian.huber@embedded-brains.de)
Received: from dedi548.your-server.de (dedi548.your-server.de [85.10.215.148])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256
bits)) (Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 0A63E8F07B
for <freebsd-hackers@freebsd.org>; Mon, 15 Oct 2018 10:45:19 +0000 (UTC)
(envelope-from sebastian.huber@embedded-brains.de)
Received: from [78.46.172.2] (helo=sslproxy05.your-server.de)
by dedi548.your-server.de with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256)
(Exim 4.89_1) (envelope-from <sebastian.huber@embedded-brains.de>)
id 1gC0NB-0004Sb-EH
for freebsd-hackers@freebsd.org; Mon, 15 Oct 2018 12:45:17 +0200
Received: from [82.135.62.35] (helo=mail.embedded-brains.de)
by sslproxy05.your-server.de with esmtpsa
(TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89)
(envelope-from <sebastian.huber@embedded-brains.de>)
id 1gC0NB-0004Bq-91
for freebsd-hackers@freebsd.org; Mon, 15 Oct 2018 12:45:17 +0200
Received: from localhost (localhost.localhost [127.0.0.1])
by mail.embedded-brains.de (Postfix) with ESMTP id 328342A0043
for <freebsd-hackers@freebsd.org>; Mon, 15 Oct 2018 12:45:16 +0200 (CEST)
Received: from mail.embedded-brains.de ([127.0.0.1])
by localhost (zimbra.eb.localhost [127.0.0.1]) (amavisd-new, port 10032)
with ESMTP id iEO5PMhX9_s8 for <freebsd-hackers@freebsd.org>;
Mon, 15 Oct 2018 12:45:15 +0200 (CEST)
Received: from localhost (localhost.localhost [127.0.0.1])
by mail.embedded-brains.de (Postfix) with ESMTP id D93E92A165C
for <freebsd-hackers@freebsd.org>; Mon, 15 Oct 2018 12:45:15 +0200 (CEST)
X-Virus-Scanned: amavisd-new at zimbra.eb.localhost
Received: from mail.embedded-brains.de ([127.0.0.1])
by localhost (zimbra.eb.localhost [127.0.0.1]) (amavisd-new, port 10026)
with ESMTP id T6DM6yklf5r4 for <freebsd-hackers@freebsd.org>;
Mon, 15 Oct 2018 12:45:15 +0200 (CEST)
Received: from huber-nb-linux.suse (unknown [192.168.96.161])
by mail.embedded-brains.de (Postfix) with ESMTPSA id BE4AE2A0043
for <freebsd-hackers@freebsd.org>; Mon, 15 Oct 2018 12:45:15 +0200 (CEST)
Subject: Re: Why SPDX-License-Identifier: BSD-2-Clause-FreeBSD?
To: freebsd-hackers@freebsd.org
References: <f396d707-a707-837e-e76f-fec3a7fba2cf@embedded-brains.de>
<149f117b-bf85-0dce-e26c-c960ad93f7a4@freebsd.org>
<fa33ae00-5772-a562-40c8-ffd4a6b0dc8a@embedded-brains.de>
<20181015100356.GA26509@britannica.bec.de>
From: Sebastian Huber <sebastian.huber@embedded-brains.de>
Message-ID: <8a56c350-fc18-aee1-f392-f0e52c629290@embedded-brains.de>
Date: Mon, 15 Oct 2018 12:45:16 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
Thunderbird/60.2.1
MIME-Version: 1.0
In-Reply-To: <20181015100356.GA26509@britannica.bec.de>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: quoted-printable
X-Authenticated-Sender: smtp-embedded@poldinet.de
X-Virus-Scanned: Clear (ClamAV 0.100.1/25038/Mon Oct 15 06:54:43 2018)
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 15 Oct 2018 10:45:20 -0000
On 15/10/2018 12:03, Joerg Sonnenberger wrote:
> On Mon, Oct 15, 2018 at 09:03:44AM +0200, Sebastian Huber wrote:
>> In case the author of a FreeBSD source file is not the copyright holde=
r,
>> then doesn't have FreeBSD a problem here?
> I could submit a file (--> I'm the author) but assign the copyright to
> the FreeBSD Foundation (--> TFF is the copyright holder).
How do you know that such a contract exists if you only look at the file?
In case the copyright holder is not the author, then is the text
=C2=A0"THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'=
' AND
=C2=A0=C2=A0 ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMIT=
ED TO, THE
=C2=A0=C2=A0 IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PAR=
TICULAR=20
PURPOSE
=C2=A0=C2=A0 ARE DISCLAIMED.=C2=A0 IN NO EVENT SHALL THE AUTHOR OR CONTR=
IBUTORS BE LIABLE
=C2=A0=C2=A0 FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, O=
R=20
CONSEQUENTIAL
=C2=A0=C2=A0 DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBS=
TITUTE GOODS
=C2=A0=C2=A0 OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INT=
ERRUPTION)
=C2=A0=C2=A0 HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN C=
ONTRACT,=20
STRICT
=C2=A0=C2=A0 LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARIS=
ING IN=20
ANY WAY
=C2=A0=C2=A0 OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POS=
SIBILITY OF
=C2=A0=C2=A0 SUCH DAMAGE."
applicable to the copyright holder? If not, is it liable for the=20
software due to some default legislation?
--=20
Sebastian Huber, embedded brains GmbH
Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone : +49 89 189 47 41-16
Fax : +49 89 189 47 41-09
E-Mail : sebastian.huber@embedded-brains.de
PGP : Public key available on request.
Diese Nachricht ist keine gesch=C3=A4ftliche Mitteilung im Sinne des EHUG=
.
From owner-freebsd-hackers@freebsd.org Mon Oct 15 10:50:51 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8AF5810D72A1
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Mon, 15 Oct 2018 10:50:51 +0000 (UTC)
(envelope-from phk@critter.freebsd.dk)
Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222])
by mx1.freebsd.org (Postfix) with ESMTP id 283DE8F3D4
for <freebsd-hackers@freebsd.org>; Mon, 15 Oct 2018 10:50:50 +0000 (UTC)
(envelope-from phk@critter.freebsd.dk)
Received: from critter.freebsd.dk (unknown [192.168.55.3])
by phk.freebsd.dk (Postfix) with ESMTP id 405BD14852;
Mon, 15 Oct 2018 10:50:44 +0000 (UTC)
Received: from critter.freebsd.dk (localhost [127.0.0.1])
by critter.freebsd.dk (8.15.2/8.15.2) with ESMTPS id w9FAohJc001505
(version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO);
Mon, 15 Oct 2018 10:50:43 GMT (envelope-from phk@critter.freebsd.dk)
Received: (from phk@localhost)
by critter.freebsd.dk (8.15.2/8.15.2/Submit) id w9FAogAS001504;
Mon, 15 Oct 2018 10:50:42 GMT (envelope-from phk)
To: Sebastian Huber <sebastian.huber@embedded-brains.de>
cc: freebsd-hackers@freebsd.org
Subject: Re: Why SPDX-License-Identifier: BSD-2-Clause-FreeBSD?
In-reply-to: <8a56c350-fc18-aee1-f392-f0e52c629290@embedded-brains.de>
From: "Poul-Henning Kamp" <phk@phk.freebsd.dk>
References: <f396d707-a707-837e-e76f-fec3a7fba2cf@embedded-brains.de>
<149f117b-bf85-0dce-e26c-c960ad93f7a4@freebsd.org>
<fa33ae00-5772-a562-40c8-ffd4a6b0dc8a@embedded-brains.de>
<20181015100356.GA26509@britannica.bec.de>
<8a56c350-fc18-aee1-f392-f0e52c629290@embedded-brains.de>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <1502.1539600642.1@critter.freebsd.dk>
Content-Transfer-Encoding: quoted-printable
Date: Mon, 15 Oct 2018 10:50:42 +0000
Message-ID: <1503.1539600642@critter.freebsd.dk>
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 15 Oct 2018 10:50:51 -0000
--------
In message <8a56c350-fc18-aee1-f392-f0e52c629290@embedded-brains.de>, Seba=
stian
Huber writes:
>> I could submit a file (--> I'm the author) but assign the copyright to
>> the FreeBSD Foundation (--> TFF is the copyright holder).
>
>How do you know that such a contract exists if you only look at the file?
As a user licensing the code under the BSD2 copyright, that does
not concern you in any way, unless you have reason to suspect that
the contract does not exist and the license therefore may be
fraudulent.
-- =
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe =
Never attribute to malice what can adequately be explained by incompetence=
.
From owner-freebsd-hackers@freebsd.org Mon Oct 15 15:07:20 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 206B010DDE85
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Mon, 15 Oct 2018 15:07:20 +0000 (UTC) (envelope-from lev@FreeBSD.org)
Received: from onlyone.not-for.work (onlyone.not-for.work [148.251.9.81])
by mx1.freebsd.org (Postfix) with ESMTP id 93FA177B69
for <freebsd-hackers@freebsd.org>; Mon, 15 Oct 2018 15:07:19 +0000 (UTC)
(envelope-from lev@FreeBSD.org)
Received: from [192.168.23.186] (unknown [89.113.128.32])
(Authenticated sender: lev@serebryakov.spb.ru)
by onlyone.not-for.work (Postfix) with ESMTPSA id 77418B41
for <freebsd-hackers@freebsd.org>; Mon, 15 Oct 2018 18:07:18 +0300 (MSK)
To: freebsd-hackers@freebsd.org
Reply-To: lev@FreeBSD.org
From: Lev Serebryakov <lev@FreeBSD.org>
Subject: Is it bad idea to use transforms from sys/crypto directly?
Openpgp: preference=signencrypt
Autocrypt: addr=lev@FreeBSD.org; prefer-encrypt=mutual; keydata=
xsFNBFKbGksBEADeguVs+XyJc3mL3iiOBqDd16wSk97YTJYOi4VsHsINzJr09oFvNDiaDBIi
fLn2p8XcJvehcsF2GSgrfXfw+uK4O1jyNIKJmiYA0EtE+ZbRtvDrrE0w6Q8+SDeKA21SWh3Y
vSQ0DJUontbgW55ER2CbEiIUTIn34uQ0kmESAaw/v5p/9ue8yPTmURvv130FqPFz8VPzltqL
NxyGt54TxPfKAzAHEIwxlEZ63JOwzloKh1UDBExcsf9nJO08/TAVgR5UZ5njFBPzaaquhRoP
qPJLEQQDqxPIlvMNtHKf7iIebE4BHeqgCdJA0BoiR6gpa0wlsZtdrTPK3n4wYSphLvGbhfOZ
YW/hbcu7HYS/FImkVxB3iY17kcC1UTnx4ZaYeASPBGOOPbXky1lLfmDGWIFT//70yx+G17qD
OZzF1SvJJhGvh6ilFYaWMX7T+nIp6Mcafc4D7AakXM+XdubNXOMlCJhzPcZ0skgAEnYV587w
V7em5fDVwQccwvtfezzqKeJAU5TGiywBHSR5Svzk2FwRNf6M//hWkpq0SRR63iOhkHGOAEBi
69GfEIwH2/w24rLxP0E+Hqq8n+EWNkPatw1Mhcl5PKkdvGCjJUaGNMkpBffjyYo254JXRscR
eEnwdIkJt4ErDvjb2/UrOFq31wWMOiLzJeVchAgvTHBMRfP9aQARAQABzShMZXYgU2VyZWJy
eWFrb3YgPGxldkBzZXJlYnJ5YWtvdi5zcGIucnU+wsGCBBMBCAAsAhsDBwsJCAcDAgEGFQgC
CQoLBBYCAwECHgECF4ACGQEFAlKbP8wFCQlmJwEACgkQ6rA8WL/cR4/6VBAAjRMyyX3PBFx/
HxyiIZ698EfwlWUua8Ft4crtrdK52m0qNkbBB9BH8xQgBHG32A1CwyzQnzxHgZuoOWMjh+Qq
WJv7dmpM/q/c1GCJHhlPgewXrciTwpAamZILN071u+1GCPWwGRPzfQ/U+k63KJWx9ozf4doM
WTTom6Cqcssi4J1u5kkt52a5ZRhsCK9pEVGilk36XTP9BakGrnMSIxF/NK4xeZVX2q+Nuqvf
RchyofKXVgLEDLwb1cd/baLtBpDzy0PTN2Zl2lX4kOA6jwTKsqRya9A1Vui1KXwPh2XViTQ1
7Y3l5qg/M+sR73DohezP6bO6huOnLhty17jAqHPNlD6RonDo+j8uIlEg4iMSTN3MhzkBAu0Q
pe3ucQ0o1767JiXN3fsNvRzSFhLVNDqPLce4uKlMogsbreXWvdgHGTN1ybOHGbybZnP77yHz
uNBacbmG3vL/OLXMqwLdL2JXoiec4DmXjjCdhTBl5xLV9Hz/6VWKqElteg8QFVvHB3tHWzJ4
/rpiVEixytCIII6DS33BXZ0h2EOkK/6AYA2SJxy1vgOH4SZBtDBHoezmHV2nFnq5O0c7AuAB
7WPWgQG0sEwHQPZmg/baRGitRJnaxf/Gvf1DeD1x1VrcoVke2vwBcgDM3kugP8L9hsqic2D3
dI+gP76haeuvNNZr3y9L9zvOwU0EUpsaSwEQALRr3B+OjY/cnJPstz5CVsVWyEZtJtrNviZr
tBgbkhlkPm98sEWR4+gbpyeufdYJengDjeGzMDKcLB7h5fICS/j6A8XdlJ40TlbPfNgb6OHa
ebaIYKTJpXKR9sD7ZyGivYMofm0em40wGUX7BIkdkomaWj+wUiS0CdXU0FWDj9wv73+Eim+X
zZyXeFgIPv97v+pET7DfwKkADOfrkW9s4OfvGVjd+wm35wc8EngQEz0qdPBxx74X7vZFAxlA
SXu8gDBJGYt2Bkc3QwULnfeXrZJWgqNPR5o44gGu96yaiOFaN/C6CJtev5ZEX+0ZxbvsHHB7
Z5AtsRURKpZ4w5HFHGhzHtDtoAKgeZ/gbhTVXPHvNQR818eN+Nl5BV8BRF/8yhR6VlJb8GYw
h8oKDeVGVYC34+raHZQAM9WoBnN7jlt4T9zzPwtmw5mIahGFgvw1KDr7OItN2ZgtZ20UYC5m
Go602nmHq0aPbU6SwGi1xohrliNsKaaciYiMaVIGRQq8iGr9Fe2HlvaA3BpB275i/gCVlUdG
y5XLAv+yQMUvn5Z7XVsMroxDk/O+ae1ElyBvKiKyfWGJXTg5XUukkkyQmfWPxWUGoNA1P/P4
GMHSu7/Rqe/7m4uPu/RyTTqsSjjKJdP9kBwEzvqPtXsVoZuShtrptRQJDYflhgE4qmKSMKen
ABEBAAHCwWUEGAECAA8FAlKbGksCGwwFCRLMAwAACgkQ6rA8WL/cR48RkA//SNzeW3CI8KHx
rA0aeHW6Nb5ieoqVRBGLyjBM06RX6vHB9v4dJL6Z+yV2jGN2s+XZX2HILbuTOwcTxGkI3xTT
e0cDXVaF5K8R/liigUjtwuC2v/sWgoWyUmK1Cy9CPYdcXmFq6nESfkUe8DYiGOUULdHq5w63
F53yOZ72iXRBQBZgkhPtRFu4lPYIzOsMag9DIJ9CthR1r0ziqU/keb94Qt3l+aXK7CwGdY7X
T4zUIMHNYsuAuyX+NJIXfsN68TT6m7QmlUwxPs13nxmoVQzm4ruV+hlQKh1MtbsjWRkNgPxF
IPiqoAEhy8QoddlSvRTwL5Z7zFQiwMdiXU7toL8pfzj/zJR1jELXKMipijrt5MLrV8XX3OPN
yZZvh95VIl8mv+iAqwSZUufd2EJnvj5TObB0eH+a+34NWf/XqA3fPjE6KHzmdnw9PZjPEjlx
JCPECSs+6gse1+GaEfKYuXzB/ENe2ctlcfx5iQJXFc+/+zG/uU/JX/pXJHA12CUfB5g7lH6X
BZIHvRo3VTCDjXgbF5xxDAe5V4exf8d4oSNjQIFLYxxN7zkvH89EN6RPfRgsWN7bYArCwfS9
MOgs9pFeCOewR6qieK150aoqNENGfKFXJup+5VVl6I0mU+j0rgVDZDht2/QgP/Tb4lGBe+ai
pOGaK/GYNR+Ad6bUmokKsx4=
Organization: FreeBSD
Message-ID: <151e4875-8dcf-2114-aaef-bcbea5600ccf@FreeBSD.org>
Date: Mon, 15 Oct 2018 18:07:18 +0300
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101
Thunderbird/52.9.1
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 8bit
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 15 Oct 2018 15:07:20 -0000
I need very fixed set of crypto primitives in kernel.
I could use opencrypto API, of course.
But looks like IPsec uses enc_xform_XXX directly.
Is it Ok to use enc_xform_XXX directly, or is it considered Very Bad Style?
--
// Lev Serebryakov
From owner-freebsd-hackers@freebsd.org Mon Oct 15 19:42:53 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4346E10C6F55
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Mon, 15 Oct 2018 19:42:53 +0000 (UTC)
(envelope-from markjdb@gmail.com)
Received: from mail-it1-x12d.google.com (mail-it1-x12d.google.com
[IPv6:2607:f8b0:4864:20::12d])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(Client CN "smtp.gmail.com",
Issuer "Google Internet Authority G3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id B59E982273
for <freebsd-hackers@freebsd.org>; Mon, 15 Oct 2018 19:42:52 +0000 (UTC)
(envelope-from markjdb@gmail.com)
Received: by mail-it1-x12d.google.com with SMTP id i191-v6so29307183iti.5
for <freebsd-hackers@freebsd.org>; Mon, 15 Oct 2018 12:42:52 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
h=sender:date:from:to:subject:message-id:mime-version
:content-disposition:user-agent;
bh=Ot66AOUJdGmE100VLodY/oOsJ6BbUnxlbNmFdC68FQQ=;
b=rwxZgehWJ/Qm7ODbQCRcgbrWkrEWxvmg1QS9ZOQCyJ8OJ6JpiE6YG9f+YpVCPOFlMP
WxAYjvv3tY3ygY3C/esLnmkrjpc3Qw2kOcMbAWwNqP0KDQzZxeSbo/YrpTXqQGlzPWDl
iAulbFUkfDIu3ujFUbAlQk7fXgIOOVrqc1zOXMBKYh977BQZwvhhbJczmUdyuvPhn2/R
C2ISTjgCczlivrrejaZTI0XYPJ5oLh7eVMKOJk2xua06vCHmpmoZwQnYS5jdtAC8bnLB
QC77/rA28nAMMGrakrNTmAwuAOB6VJZAiJEcSsoV9CskaXBdhh8fC2s7hWa0h/jaodVm
d0hw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:sender:date:from:to:subject:message-id
:mime-version:content-disposition:user-agent;
bh=Ot66AOUJdGmE100VLodY/oOsJ6BbUnxlbNmFdC68FQQ=;
b=MTRGFWIl2xv01L0Wtk7qxMfVP8+lJKmX6upFYeTRHFdDqeceoTCE0wiEoj2LAoGZIs
cJ27PXkThiDzY6+tjOchc72FN5Dkxiq0XsLjMMCsbcO+B6HyeenWdcFIhgFpM2gyBtNG
aUqx4q5k4KgvfITCaS5vcfh2PWtmXuKeU3OuTnGvvMYaFRFyVo1ngsBXjjLYpTh/n7WF
Gj9NKJ8CqBfkzMSmmwdDnLdtBtnl90qbyMTUFhU4HmX954M/5h4moxTbECsXa814RBJr
fkS+ckXOubLnToHYsOOTDZt6X5lYzbWY84qmA9AexTKWM4pKrK+9NR1HXKcc7IGaAqL4
+KqQ==
X-Gm-Message-State: ABuFfojHTge92kKCFV3jnzOXHGikwfJIxwnH6nd1dIL/fG/M6/8Z+Ux7
5oS77nOC6NPUQMLiLEDm+KC/2Anc
X-Google-Smtp-Source: ACcGV616Udm08YEU9pQ6JMgt/BTTaKpbcSn4b2FMnkkxIUORVqrlM/ACo9tFMOpNQggbXo/K3igBIw==
X-Received: by 2002:a24:16ca:: with SMTP id
a193-v6mr13771306ita.134.1539632571827;
Mon, 15 Oct 2018 12:42:51 -0700 (PDT)
Received: from spy (ip-24-50-182-247.user.start.ca. [24.50.182.247])
by smtp.gmail.com with ESMTPSA id p136-v6sm5282444itb.37.2018.10.15.12.42.50
for <freebsd-hackers@freebsd.org>
(version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
Mon, 15 Oct 2018 12:42:51 -0700 (PDT)
Sender: Mark Johnston <markjdb@gmail.com>
Date: Mon, 15 Oct 2018 15:42:45 -0400
From: Mark Johnston <markj@freebsd.org>
To: freebsd-hackers@freebsd.org
Subject: [CFT] capsicum patches for rtsol(8) and rtsold(8)
Message-ID: <20181015194212.GA2751@spy>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.10.1 (2018-07-13)
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 15 Oct 2018 19:42:53 -0000
Hi,
Last week I spent some time Capsicumizing rtsol(8) and rtsold(8). The
code for these programs is relatively straightforward, and seems like an
ideal candidate for sandboxing given that it parses ND6 RAs while
running with privileges (which has led to one SA in the past).
I currently don't run rtsold as my home ISP only gives me an IPv4
address. I did a fair amount of testing on an internal network and used
packet captures to verify everything, but I was hoping that some folks
who actually rely on rtsol(d) would be willing to test the patch before
I try to get it reviewed.
The patch is here:
https://people.freebsd.org/~markj/patches/rtsold_capsicum.diff
and a review and description for the capsicum portions is here, if
anyone is interested:
https://reviews.freebsd.org/D17572
The patch is a superset of the review contents; it contains some
unrelated fixes (for e.g., Coverity bugs) and cleanups.
To test the patch, apply it, rebuild sbin/rtsol and usr.sbin/rtsold,
and restart rtsold. It would be helpful to restart networking at this
point, or simply reboot. If the patch works properly, you shouldn't
notice any changes in behaviour. I'd be particularly interested in
hearing reports from anyone that uses -m or -O, or who depends upon the
processing of the RDNSS and DNSSL RA options.
From owner-freebsd-hackers@freebsd.org Mon Oct 15 21:05:57 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id BD2A410C91B0
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Mon, 15 Oct 2018 21:05:57 +0000 (UTC)
(envelope-from james.shank@gmail.com)
Received: from mail-qt1-x82f.google.com (mail-qt1-x82f.google.com
[IPv6:2607:f8b0:4864:20::82f])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(Client CN "smtp.gmail.com",
Issuer "Google Internet Authority G3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 5B0EB84D20
for <freebsd-hackers@freebsd.org>; Mon, 15 Oct 2018 21:05:57 +0000 (UTC)
(envelope-from james.shank@gmail.com)
Received: by mail-qt1-x82f.google.com with SMTP id q40-v6so23243502qte.0
for <freebsd-hackers@freebsd.org>; Mon, 15 Oct 2018 14:05:57 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
h=from:content-transfer-encoding:mime-version:date:subject:message-id
:to; bh=GcYyukUk4QPwNM41ccv3hCTmeZ61rYMQIdky4tnI2wc=;
b=fS++5FD6hUADGeiyPVigjxuGh5qm1XDBLpfq2WnbS2gXS9KWPvh4ZV221XnexzEMv4
xkY1mY7XGDgA50pE3v7J60dOjsmWKGkSYdaT3o8E1h4lTwvkbnSjNRDgLwFzhAK0vQ1J
QL4PyXQRBczbq4RZrRTDuNpwC/RudPvFs526E3Z/+YcrqqNejrtYiUNYWjNlxzTwQ5bj
s1EVY1UXwU0TfzLJUF7BNeGbUnEC05pdtLRC0jSoMVWcIFVWDv0gtV+0nVp1qk5DS9AB
3jUX3SIsOu7i5/RYf/a+KiesXBhJv1wNt61Xf1tvPiEAvCM9O/bA7KJhZ7uGObpZLmC9
ppSA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:from:content-transfer-encoding:mime-version:date
:subject:message-id:to;
bh=GcYyukUk4QPwNM41ccv3hCTmeZ61rYMQIdky4tnI2wc=;
b=SJeF1yf0GLIYinTdnlRQV7vgyoBX/+LAF7hQYHZTCrBvGIRySz7yMVj7IbqQot5TMy
ta1tC6G/ap2GDOa8wr2FI58cP3Ds1nuSlPm5R72v8oGTgd6Uvz5FXauKo2gbsgBq5SbK
FXbKSH/A1TIpKyj68UcIm8+BrqsGqSc1kk/R0+SCMOQqgKrTD0gFDcIezBYN+Gk5fJf7
gcM+0qQ61JttnmBiP0NQT3HAg6dLPwyxsT3mbzJ/Ts1yeKBlFoj+KGeo/fgbAWEon08z
JUpq6vto6O4DP4AqR2WLp8lt28wv3MOdKaDqU2XrhXAaPLYG9DEiu2TbKxzdilTo72a1
OSxg==
X-Gm-Message-State: ABuFfogbG+lZN+HO/2Ebx42K7yzfXuedqGujEHkQX1tj3WnR2dV23TZu
4KajlbZn0IgTmI5lIyhJ/MdsXsCf
X-Google-Smtp-Source: ACcGV61HtiJJrCs/f2n9M92tjN5ctEQjbiDUyzd9dOnGUn75tDl9mhC6cuflIpQY0mB3MzI801eqwQ==
X-Received: by 2002:ac8:2092:: with SMTP id
18-v6mr17573216qtd.192.1539637556674;
Mon, 15 Oct 2018 14:05:56 -0700 (PDT)
Received: from ?IPv6:2600:380:944e:d585:41a9:7ef7:e8ee:334c?
([2600:380:944e:d585:41a9:7ef7:e8ee:334c])
by smtp.gmail.com with ESMTPSA id r57-v6sm8569156qtc.36.2018.10.15.14.05.55
for <freebsd-hackers@freebsd.org>
(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
Mon, 15 Oct 2018 14:05:55 -0700 (PDT)
From: James Shank <james.shank@gmail.com>
Content-Type: text/plain;
charset=utf-8
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (1.0)
Date: Mon, 15 Oct 2018 17:05:54 -0400
Subject: High I/O RAID on UFS2
Message-Id: <1BA62E58-93D7-4763-B27A-E21B7DA362B2@gmail.com>
To: freebsd-hackers@freebsd.org
X-Mailer: iPhone Mail (16A404)
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 15 Oct 2018 21:05:58 -0000
Hi FreeBSD Hackers,
I am running FreeBSD 11.1-RELEASE-p15 with a UFS2 filesystem block size 64K,=
with a RAID 6 of 16 SSD drives on a MRSAS compatible RAID card stripe size o=
f 256K (and yes, I know about the misalignment) and 384GB memory on 48 real c=
ores with hyper threading enabled. I invite critiques of this configuration=
, but am more keen on getting help making it run efficiently.
I am running 11.1 because 11.2-RELEASE repeatably panics after failing somew=
here after allocbuf() tries to allocate a page (sorry, I don=E2=80=99t still=
have the output from this state).
I am doing very read heavy operations, via multithreaded readers. My writes=
tend to be small, but come in every 15 seconds or so. I=E2=80=99d estimate=
read to write ratio of about 100:1 or more.
What I see happening is pagedaemon coming alive and bufferspacedaemon doing t=
he same to clear the corresponding resources. bufferspacedaemon runs nearly=
all the time at 100%. I haven=E2=80=99t yet had much success with tuning t=
hings to keep enough resources free to allow my read rates to be sustained.
My read rates can hit and exceed 6GBps when everything is humming along, but=
only for short periods of time, then they drop off to much lower rates afte=
r the kernel starts shuffling resources. For long periods of time, my best s=
ustainable rate seems to be 3.5GBps, and I can=E2=80=99t seem to get the ful=
l read rate for sustained periods.
I have tried several things, but I believe now that I may be missing somethi=
ng that the more experienced gurus will find obvious.
Any pointers on where I should look next?
Thanks!
James
From owner-freebsd-hackers@freebsd.org Tue Oct 16 04:00:25 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 62C4C10DFE76
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Tue, 16 Oct 2018 04:00:25 +0000 (UTC)
(envelope-from marklmi@yahoo.com)
Received: from sonic306-21.consmr.mail.gq1.yahoo.com
(sonic306-21.consmr.mail.gq1.yahoo.com [98.137.68.84])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id B814978E47
for <freebsd-hackers@freebsd.org>; Tue, 16 Oct 2018 04:00:24 +0000 (UTC)
(envelope-from marklmi@yahoo.com)
X-YMail-OSG: z17AQoIVM1mbFSZ8AMx5keokcT1PbUjyxr1ezsLrKpbUDw5qAmuAM.VMo5Gik4t
N4sE2VL7jB8Ygf0FccRtQ1TUoLmCWb_5hIVIfvJPbxR16N7VSVKW03uZ9QHA1.kz7nQE8wGVI_GP
JY__t3tiCQSCIXa7WdwG5.RJxL8QgO.fR4WJ2QdrT4sokem.XdoMSVpu_UUBCUJ1hGgMBWOkyWsU
eTRS9VvmxrEpYqYSOpkfe11kL_ILSVqrSVT2PmHYMGt7TT3aya0ksou.7jFUXl.Hc2ZO1sGYjms6
Wf56SBolvP0hXq34x0jKKu43HjWtibzwXjjuGr28IXEvxt.anLH1_0CDbzLUZVYWhXcO9RuqKAXR
bMCJhTBRh1ru2iSiZPGWzkM.MWcdnjOF.VcPr4y.qAQNFdqDiG6LeppiIMta6wueo1SdAufDW8PV
.dkSXVwdCc_lVQTeXRGyIY21V1J_KmyiuigFMZ6mdWmrP99dpsFXp52BylCfibqIdW.VQ55AcKQv
CXBkHd9fXPbcwP.njaV7SZywy2UqrxB9AUWjJppbVVHO0BGvB9yK.pHZvsD8ZG1w8nhc7roWL0WF
ArMFytAZut2mWOHTcNZ6jqKXcDnE41fh.If0YtpslBt0HFqfyv.vnIlsZOuM.WbRjGO0hzziRyIq
8VTwaDICKgwR8v4Imp4nEyQxYvFHd6o12SAT4nLpCrH.6sV73sPHYo6nQrTDX8_F_.9xDn.ojwk8
uKufXRQexAnhyQMv7f_DOtRDrF3kwjzX1Aoaq7vBU9XXcKMnX4BSYNfgr9vhJJqYtYiFtAUPUWMO
ho3NherbyWZlGpbo6OkKmh0w_hvkod2BIdi5y5jJsQF2Qi990FmZP3rlWTEVFGJF4zw.PF5LNVfv
ROek40xAQgUI3LV7ka7iEl48dTcPvxarkbzSma.FtL3tDP2DenTktDUUHdJHMjvwtkse6_iPFqJa
1D2HQxyURmkF6CXsXMHgba0wgF7wEHQ_00FMYTJVgPRE_ein_dxqoBmCD4Hiedjoo6GdzPZ62ook
skq.TEmJN93Mehsc0Q_1.zMLK9efPp8TUnm55OmqyJj4H_pQuiXRgvdSoOWi_oUZmVCNUefNRgzl
NoJszC.0dXmfNYA--
Received: from sonic.gate.mail.ne1.yahoo.com by
sonic306.consmr.mail.gq1.yahoo.com with HTTP; Tue, 16 Oct 2018 04:00:18 +0000
Received: from c-76-115-7-162.hsd1.or.comcast.net (EHLO [192.168.1.25])
([76.115.7.162])
by smtp431.mail.gq1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID
dd93d55728f71f0de1169e4749adc2d5;
Tue, 16 Oct 2018 04:00:17 +0000 (UTC)
Content-Type: text/plain;
charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\))
Subject: Re: FYI: powerpc64 headbuilt via devel/powerpc64-xtoolchain-gcc and
C++ exceptions for user code built by system-clang or devel/powerpc64-gcc (as
of head -r339076 and ports -r480180)
From: Mark Millard <marklmi@yahoo.com>
In-Reply-To: <0539C16B-1603-4639-914A-0308578C7262@yahoo.com>
Date: Mon, 15 Oct 2018 21:00:17 -0700
Cc: Justin Hibbits <jrh29@alumni.cwru.edu>
Content-Transfer-Encoding: quoted-printable
Message-Id: <3B69D483-AEBA-47CA-B140-7445089EB064@yahoo.com>
References: <E1167EAA-3F90-4F0A-9F51-53CFE1461617@yahoo.com>
<0539C16B-1603-4639-914A-0308578C7262@yahoo.com>
To: FreeBSD PowerPC ML <freebsd-ppc@freebsd.org>,
FreeBSD Toolchain <freebsd-toolchain@freebsd.org>,
FreeBSD Mailing List <freebsd-hackers@freebsd.org>
X-Mailer: Apple Mail (2.3445.9.1)
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 16 Oct 2018 04:00:25 -0000
[I've found the problem at the low level for
my context of using WITH_LIBCPLUSPLUS=3D with
the likes of devel/powerpc64-gcc but I do
not have a solution for WITH_LIBCPLUSPLUS=3D
so far. I give details of what I found.]
On 2018-Oct-14, at 12:40 AM, Mark Millard <marklmi at yahoo.com> wrote:
> On 2018-Oct-12, at 1:59 PM, Mark Millard <marklmi at yahoo.com> wrote:
>=20
>> I built a powerpc64 head -r339076 via devel/powerpc64-gcc
>> and the like that built clang as cc as well (and
>> WITHOUT_LIB32). This included use of base/binutils to
>> the the powerpc64 set up. The system and kernel are
>> non-debug builds (but with symbols). [system-clang is not
>> used for buildworld buildkernel because of known
>> issues (last I tried).]
>>=20
>> booting, buildworld, buildkernel, poudriere building
>> what totaled to be somewhat under 400 ports all seem
>> to work. But . . .
>>=20
>> It been a long time since I've done something analogous
>> and a significant item in the result is different than in
>> the past once I started testing the throwing of C++
>> exceptions in code produced by system-clang or by
>> devel/powerpc64-gcc :
>>=20
>> Such code ends up stuck using around 100% of a CPU.
>> An example is the program:
>>=20
>> # more exception_test.cpp
>> #include <exception>
>>=20
>> int main(void)
>> {
>> try { throw std::exception(); }
>> catch (std::exception& e) {}
>> return 0;
>> }
>>=20
>> For system-clang it ended up with:
>>=20
>> # ldd a.out
>> a.out:
>> libc++.so.1 =3D> /usr/lib/libc++.so.1 (0x81006d000)
>> libcxxrt.so.1 =3D> /lib/libcxxrt.so.1 (0x810184000)
>> libm.so.5 =3D> /lib/libm.so.5 (0x8101ab000)
>> libc.so.7 =3D> /lib/libc.so.7 (0x8101eb000)
>> libgcc_s.so.1 =3D> /lib/libgcc_s.so.1 (0x810554000)
>>=20
>> That program goes into an possibly unbounded execution.
>> (Historically when this program had problems it would
>> stop and produce a core file.)
>>=20
>> When compiled by devel/powerpc64-gcc the a.out that results
>> does the same thing. ( =
/usr/local/bin/powerpc64-unknown-freebsd12.0-c++=20
>> as the compiler path ) So this is not really clang specific
>> in any way. This ended up with:
>>=20
>> # ldd a.out
>> a.out:
>> libc++.so.1 =3D> /usr/lib/libc++.so.1 (0x81006d000)
>> libcxxrt.so.1 =3D> /lib/libcxxrt.so.1 (0x810184000)
>> libm.so.5 =3D> /lib/libm.so.5 (0x8101ab000)
>> libgcc_s.so.1 =3D> /lib/libgcc_s.so.1 (0x8101eb000)
>> libc.so.7 =3D> /lib/libc.so.7 (0x810211000)
>>=20
>> (That should not have involved clang or llvm at all.)
>>=20
>> But compiled by lang/gcc8's g++8 the a.out that results works
>> fine. This ends up with:
>>=20
>> # ldd a.out
>> a.out:
>> libstdc++.so.6 =3D> /usr/local/lib/gcc8/libstdc++.so.6 =
(0x81006e000)
>> libm.so.5 =3D> /lib/libm.so.5 (0x8102c7000)
>> libgcc_s.so.1 =3D> /lib/libgcc_s.so.1 (0x810307000)
>> libc.so.7 =3D> /lib/libc.so.7 (0x81032d000)
>>=20
>> It is not clear if using base/gcc as system cc
>> would do any better than using system-clang does
>> or than devel/powerpc64-gcc does: it is sort of
>> a variant of devel/powerpc64-gcc .
>>=20
>> It will probably be some time before I figure out
>> much about what is going on.
>>=20
>> Two things common to the problem cases are:
>>=20
>> libc++.so.1 =3D> /usr/lib/libc++.so.1 (0x81006d000)
>> libcxxrt.so.1 =3D> /lib/libcxxrt.so.1 (0x810184000)
>>=20
>> lang/gcc8 avoids those being involved.
>>=20
>>=20
>> Notes:
>>=20
>> . . .
>>=20
>> WITHOUT_LIB32=3D is because, for every post-gcc 4.2.1
>> that I've tried, the lib32 produced misuses R30 in
>> crtbeginS code (vs. the ABI for FreeBSD) and 32-bit
>> code just produces core files from the bad so-called
>> address dereference that results.
>>=20
>> I'd rather have throwing C++ exceptions working and
>> lack of lib32 than have lib32 but not have throwing
>> C++ exceptions working. But at the moment how to have
>> such is not obvious when fairly modern compilers
>> and toolchains are involved.=20
>=20
> Here is what I've found so far.
>=20
> The code is looping in the following routine.
> (I've inserted 2 NOTE: lines for what the
> sustained looping is like.)
>=20
. . . (the routine was _Unwind_RaiseException) . . .
So far I've found that the following in
_Unwind_RaiseException stays invariant once
initialized --despite the uw_frame_state_for
and uw_update_context calls in
_Unwind_RaiseException 's loop that normally
update fs:
(gdb) print fs
$15 =3D {regs =3D {reg =3D {{loc =3D {reg =3D 0, offset =3D 0, exp =3D =
0x0}, how =3D REG_UNSAVED} <repeats 31 times>, {loc =3D {reg =3D =
18446744073709551608, offset =3D -8,=20
exp =3D 0xfffffffffffffff8 <Error reading address =
0xfffffffffffffff8: Bad address>}, how =3D REG_SAVED_OFFSET}, {loc =3D =
{reg =3D 0, offset =3D 0, exp =3D 0x0}, how =3D REG_UNSAVED} <repeats 33 =
times>, {
loc =3D {reg =3D 16, offset =3D 16, exp =3D 0x10 <Error reading =
address 0x10: Bad address>}, how =3D REG_SAVED_OFFSET}, {loc =3D {reg =3D =
0, offset =3D 0, exp =3D 0x0}, how =3D REG_UNSAVED} <repeats 80 times>},=20=
prev =3D 0x0}, cfa_offset =3D 0, cfa_reg =3D 1, cfa_exp =3D 0x0, =
cfa_how =3D CFA_REG_OFFSET, pc =3D 0x8101999f8, personality =3D 0, =
data_align =3D -8, code_align =3D 4, retaddr_column =3D 65,=20
fde_encoding =3D 27 '\033', lsda_encoding =3D 255 '?', saw_z =3D 1 =
'\001', signal_frame =3D 0 '\0', eh_ptr =3D 0x0}
It turns out that pc value 0x8101999f8 is a little after the libcxxrt.so
call to _Unwind_RaiseException that is in throw_exception. But
_Unwind_RaiseException returning would be a failure and would end up
in a non-returning, error-reporting code path.
In other words: this is not an appropriate context for following the
return path to unwind out of _Unwind_RaiseException and its internal
caller (throw_exception).
It got to that address from lr containing the address of the
instruction after the one that does bl to the
_Unwind_RaiseException routine.
Overall it needs to unwind past this in the normal case but is stuck
handling the error/no-return case as "the" case.
Supporting details follow.
What lead up to 0x8101999f8 for initialization was
the lr value related to calling _Unwind_RaiseException
(see the address in lr below, also copied to r5):
. . .
(gdb) c
Continuing.
Breakpoint 9, <signal handler called>
1: x/i $pc 0x8101f35d8 <_Unwind_RaiseException+216>: bl =
0x8101f2bc0 <uw_init_context_1>
Current language: auto; currently minimal
where the register values being supplied are (see
r5 and lr):
(gdb) info reg
r0 0x8101999f0 34629851632
r1 0x3fffffffffffc320 4611686018427372320
r2 0x810217900 34630367488
r3 0x3fffffffffffd280 4611686018427376256
r4 0x3fffffffffffd930 4611686018427377968
r5 0x8101999f0 34629851632
r6 0xa0 160
r7 0x0 0
r8 0x1 1
r9 0x8101aac10 34629921808
r10 0x1 1
r11 0x28 40
r12 0x28000282 671089282
r13 0x81005d020 34628554784
r14 0x0 0
r15 0x0 0
r16 0x0 0
r17 0x0 0
r18 0x0 0
r19 0x0 0
r20 0x0 0
r21 0x0 0
r22 0x0 0
r23 0x0 0
r24 0x0 0
r25 0x0 0
r26 0x0 0
r27 0x3fffffffffffd280 4611686018427376256
r28 0x810041060 34628440160
r29 0x3fffffffffffc390 4611686018427372432
r30 0x3fffffffffffcd10 4611686018427374864
r31 0x810041008 34628440072
pc 0x8101f35d8 34630219224
ps 0x0 0
cr 0x0 0
lr 0x8101999f0 34629851632
ctr 0x8101f3500 34630219008
xer 0x0 0
fpscr 0xfff80000 -524288
vscr 0x0 0
vrsave 0x0 0
The pc listed in print fs (0x8101999f8) is in the
following from libcxxrt, as is the value in r5 and
lr (0x8101999f0): (some blank lines inserted to
help identify the area and some related material)
(gdb) disass throw_exception
Dump of assembler code for function throw_exception:
0x0000000810199960 <throw_exception+0>: mflr r0
0x0000000810199964 <throw_exception+4>: std r31,-8(r1)
0x0000000810199968 <throw_exception+8>: mr r31,r3
0x000000081019996c <throw_exception+12>: std r0,16(r1)
0x0000000810199970 <throw_exception+16>: stdu r1,-128(r1)
0x0000000810199974 <throw_exception+20>: bl 0x810197ab0 =
<thread_info>
0x0000000810199978 <throw_exception+24>: ld r10,8(r3)
0x000000081019997c <throw_exception+28>: mr r9,r3
0x0000000810199980 <throw_exception+32>: cmpdi cr7,r10,0
0x0000000810199984 <throw_exception+36>: std r10,24(r31)
0x0000000810199988 <throw_exception+40>: beq- cr7,0x810199a10 =
<throw_exception+176>
0x000000081019998c <throw_exception+44>: ld r10,0(r9)
0x0000000810199990 <throw_exception+48>: cmpdi cr7,r10,0
0x0000000810199994 <throw_exception+52>: std r10,32(r31)
0x0000000810199998 <throw_exception+56>: beq- cr7,0x8101999d0 =
<throw_exception+112>
0x000000081019999c <throw_exception+60>: lwz r10,48(r9)
0x00000008101999a0 <throw_exception+64>: addi r3,r31,88
0x00000008101999a4 <throw_exception+68>: addi r10,r10,1
0x00000008101999a8 <throw_exception+72>: stw r10,48(r9)
0x00000008101999ac <throw_exception+76>: bl 0x81018e500 =
<00000018.plt_call._Unwind_RaiseException@@GCC_3.0>
0x00000008101999b0 <throw_exception+80>: ld r2,40(r1)
0x00000008101999b4 <throw_exception+84>: addi r1,r1,128
0x00000008101999b8 <throw_exception+88>: mr r4,r31
0x00000008101999bc <throw_exception+92>: ld r0,16(r1)
0x00000008101999c0 <throw_exception+96>: ld r31,-8(r1)
0x00000008101999c4 <throw_exception+100>: mtlr r0
0x00000008101999c8 <throw_exception+104>: b 0x8101996b0 =
<report_failure>
0x00000008101999cc <throw_exception+108>: nop
0x00000008101999d0 <throw_exception+112>: nop
0x00000008101999d4 <throw_exception+116>: addi r3,r31,88
0x00000008101999d8 <throw_exception+120>: ld r10,-30008(r2)
0x00000008101999dc <throw_exception+124>: std r10,32(r31)
0x00000008101999e0 <throw_exception+128>: lwz r10,48(r9)
0x00000008101999e4 <throw_exception+132>: addi r10,r10,1
0x00000008101999e8 <throw_exception+136>: stw r10,48(r9)
0x00000008101999ec <throw_exception+140>: bl 0x81018e500 =
<00000018.plt_call._Unwind_RaiseException@@GCC_3.0>
0x00000008101999f0 <throw_exception+144>: ld r2,40(r1)
0x00000008101999f4 <throw_exception+148>: addi r1,r1,128
0x00000008101999f8 <throw_exception+152>: mr r4,r31
0x00000008101999fc <throw_exception+156>: ld r0,16(r1)
0x0000000810199a00 <throw_exception+160>: ld r31,-8(r1)
0x0000000810199a04 <throw_exception+164>: mtlr r0
0x0000000810199a08 <throw_exception+168>: b 0x8101996b0 =
<report_failure>
0x0000000810199a0c <throw_exception+172>: nop
0x0000000810199a10 <throw_exception+176>: nop
0x0000000810199a14 <throw_exception+180>: ld r10,-30000(r2)
0x0000000810199a18 <throw_exception+184>: std r10,24(r31)
0x0000000810199a1c <throw_exception+188>: b 0x81019998c =
<throw_exception+44>
0x0000000810199a20 <throw_exception+192>: .long 0x0
0x0000000810199a24 <throw_exception+196>: .long 0x90001
0x0000000810199a28 <throw_exception+200>: lwz r0,0(r1)
End of assembler dump.
For: 0x00000008101999f8 (above)
objdump shows: 00000000000159f8 (below):
0000000000015960 <.__cxa_end_catch+0x460> mflr r0
0000000000015964 <.__cxa_end_catch+0x464> std r31,-8(r1)
0000000000015968 <.__cxa_end_catch+0x468> mr r31,r3
000000000001596c <.__cxa_end_catch+0x46c> std r0,16(r1)
0000000000015970 <.__cxa_end_catch+0x470> stdu r1,-128(r1)
0000000000015974 <.__cxa_end_catch+0x474> bl 0000000000013ab0 =
<._ZdaPv+0x590>
0000000000015978 <.__cxa_end_catch+0x478> ld r10,8(r3)
000000000001597c <.__cxa_end_catch+0x47c> mr r9,r3
0000000000015980 <.__cxa_end_catch+0x480> cmpdi cr7,r10,0
0000000000015984 <.__cxa_end_catch+0x484> std r10,24(r31)
0000000000015988 <.__cxa_end_catch+0x488> beq cr7,0000000000015a10 =
<.__cxa_end_catch+0x510>
000000000001598c <.__cxa_end_catch+0x48c> ld r10,0(r9)
0000000000015990 <.__cxa_end_catch+0x490> cmpdi cr7,r10,0
0000000000015994 <.__cxa_end_catch+0x494> std r10,32(r31)
0000000000015998 <.__cxa_end_catch+0x498> beq cr7,00000000000159d0 =
<.__cxa_end_catch+0x4d0>
000000000001599c <.__cxa_end_catch+0x49c> lwz r10,48(r9)
00000000000159a0 <.__cxa_end_catch+0x4a0> addi r3,r31,88
00000000000159a4 <.__cxa_end_catch+0x4a4> addi r10,r10,1
00000000000159a8 <.__cxa_end_catch+0x4a8> stw r10,48(r9)
00000000000159ac <.__cxa_end_catch+0x4ac> bl 000000000000a500 =
<CXXABI_1.3@@CXXABI_1.3+0xa500>
00000000000159b0 <.__cxa_end_catch+0x4b0> ld r2,40(r1)
00000000000159b4 <.__cxa_end_catch+0x4b4> addi r1,r1,128
00000000000159b8 <.__cxa_end_catch+0x4b8> mr r4,r31
00000000000159bc <.__cxa_end_catch+0x4bc> ld r0,16(r1)
00000000000159c0 <.__cxa_end_catch+0x4c0> ld r31,-8(r1)
00000000000159c4 <.__cxa_end_catch+0x4c4> mtlr r0
00000000000159c8 <.__cxa_end_catch+0x4c8> b 00000000000156b0 =
<.__cxa_end_catch+0x1b0>
00000000000159cc <.__cxa_end_catch+0x4cc> nop
00000000000159d0 <.__cxa_end_catch+0x4d0> nop
00000000000159d4 <.__cxa_end_catch+0x4d4> addi r3,r31,88
00000000000159d8 <.__cxa_end_catch+0x4d8> ld r10,-30008(r2)
00000000000159dc <.__cxa_end_catch+0x4dc> std r10,32(r31)
00000000000159e0 <.__cxa_end_catch+0x4e0> lwz r10,48(r9)
00000000000159e4 <.__cxa_end_catch+0x4e4> addi r10,r10,1
00000000000159e8 <.__cxa_end_catch+0x4e8> stw r10,48(r9)
00000000000159ec <.__cxa_end_catch+0x4ec> bl 000000000000a500 =
<CXXABI_1.3@@CXXABI_1.3+0xa500>
00000000000159f0 <.__cxa_end_catch+0x4f0> ld r2,40(r1)
00000000000159f4 <.__cxa_end_catch+0x4f4> addi r1,r1,128
00000000000159f8 <.__cxa_end_catch+0x4f8> mr r4,r31
00000000000159fc <.__cxa_end_catch+0x4fc> ld r0,16(r1)
0000000000015a00 <.__cxa_end_catch+0x500> ld r31,-8(r1)
0000000000015a04 <.__cxa_end_catch+0x504> mtlr r0
0000000000015a08 <.__cxa_end_catch+0x508> b 00000000000156b0 =
<.__cxa_end_catch+0x1b0>
0000000000015a0c <.__cxa_end_catch+0x50c> nop
0000000000015a10 <.__cxa_end_catch+0x510> nop
0000000000015a14 <.__cxa_end_catch+0x514> ld r10,-30000(r2)
0000000000015a18 <.__cxa_end_catch+0x518> std r10,24(r31)
0000000000015a1c <.__cxa_end_catch+0x51c> b 000000000001598c =
<.__cxa_end_catch+0x48c>
0000000000015a20 <.__cxa_end_catch+0x520> .long 0x0
0000000000015a24 <.__cxa_end_catch+0x524> .long 0x90001
0000000000015a28 <.__cxa_end_catch+0x528> lwz r0,0(r1)
And dwarfdump shows starting at 0x00015960 as:
< 117><0x00015960:0x00015a2c><><cie offset 0x0000101c::cie index =
0><fde offset 0x00001018 length: 0x00000030>
<eh aug data len 0x0>
0x00015960: <off cfa=3D00(r1) >=20
0x00015968: <off cfa=3D00(r1) > <off r31=3D-8(cfa) > <off r65=3Dr0=
>=20
0x00015974: <off cfa=3D128(r1) > <off r31=3D-8(cfa) > <off =
r65=3D16(cfa) >=20
0x000159b8: <off cfa=3D00(r1) > <off r31=3D-8(cfa) > <off =
r65=3D16(cfa) >=20
0x000159c8: <off cfa=3D00(r1) >=20
0x000159d0: <off cfa=3D128(r1) > <off r31=3D-8(cfa) > <off =
r65=3D16(cfa) >=20
0x000159f8: <off cfa=3D00(r1) > <off r31=3D-8(cfa) > <off =
r65=3D16(cfa) >=20
0x00015a08: <off cfa=3D00(r1) >=20
0x00015a10: <off cfa=3D128(r1) > <off r31=3D-8(cfa) > <off =
r65=3D16(cfa) >=20
fde section offset 4120 0x00001018 cie offset for fde: 4124 0x0000101c
0 DW_CFA_advance_loc 8 (2 * 4)
1 DW_CFA_register r65 =3D r0
4 DW_CFA_offset r31 -8 (1 * -8)
6 DW_CFA_advance_loc 12 (3 * 4)
7 DW_CFA_def_cfa_offset 128
10 DW_CFA_offset_extended_sf r65 16 (-2 * -8)
13 DW_CFA_advance_loc 68 (17 * 4)
14 DW_CFA_remember_state
15 DW_CFA_def_cfa_offset 0
17 DW_CFA_advance_loc 16 (4 * 4)
18 DW_CFA_restore_extended r65
20 DW_CFA_restore r31
21 DW_CFA_advance_loc 8 (2 * 4)
22 DW_CFA_restore_state
23 DW_CFA_advance_loc 40 (10 * 4)
24 DW_CFA_remember_state
25 DW_CFA_def_cfa_offset 0
27 DW_CFA_advance_loc 16 (4 * 4)
28 DW_CFA_restore_extended r65
30 DW_CFA_restore r31
31 DW_CFA_advance_loc 8 (2 * 4)
32 DW_CFA_restore_state
33 DW_CFA_nop
34 DW_CFA_nop
/usr/src/contrib/libstdc++/libsupc++/eh_throw.cc has something
that /usr/src/contrib/libcxxrt/exception.cc does not have in
its error handling code path: a use of __cxa_begin_catch=20
in __cxxabiv1::__cxa_throw :
#ifdef _GLIBCXX_SJLJ_EXCEPTIONS
_Unwind_SjLj_RaiseException (&header->unwindHeader);
#else
_Unwind_RaiseException (&header->unwindHeader);
#endif
// Some sort of unwinding error. Note that terminate is a handler.
__cxa_begin_catch (&header->unwindHeader);
std::terminate ();
It looks to me like __cxa_begin_catch might do either of
terminate or _Unwind_Resume and that the (conditional)
_Unwind_Resume case is possibly needed here for the
normal execution.
There are also no other calls (bl's) before the terminate:
more directly indicated to not return.
I do not know if one or both of those points helped the code
unwind correctly or not. But it seems suggestive.
Other notes:
I've demonstrated the problem on my prior head
-r333594 environment that had been build via a
6.3 vintage of devel/powerpc64-gcc (but that was
then updated to ports -r469844 and so had 6.4 of
devel/powerpc64-gcc installed). Also: base/binutils
was of a 2.30 vintage and base/gcc was of a 6.3
vintage. (system-clang was not cc, base/gcc
provided system-cc.)
Compiling to produce the a.out via:
/usr/bin/powerpc64-unknown-freebsd12.0-g++
(and so via a 6.3 vintage g++) made no
difference. It still had the problem.
I have taken to having buildworld buildkernel
use -gdwarf-2 so that /usr/libexec/gdb has
access to the information in a format it is
(mostly) designed for. (/usr/local/bin/gdb is
broken by the thrown-C++-exception problem that
I'm investigating: gdb internally throws
exceptions in normal operation.)
=3D=3D=3D
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)
From owner-freebsd-hackers@freebsd.org Tue Oct 16 05:26:00 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id D02E010E200B
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Tue, 16 Oct 2018 05:26:00 +0000 (UTC)
(envelope-from sebastian.huber@embedded-brains.de)
Received: from dedi548.your-server.de (dedi548.your-server.de [85.10.215.148])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256
bits)) (Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 4E2BC7B81D
for <freebsd-hackers@freebsd.org>; Tue, 16 Oct 2018 05:26:00 +0000 (UTC)
(envelope-from sebastian.huber@embedded-brains.de)
Received: from [78.46.172.2] (helo=sslproxy05.your-server.de)
by dedi548.your-server.de with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256)
(Exim 4.89_1) (envelope-from <sebastian.huber@embedded-brains.de>)
id 1gCHrh-0002nq-6n
for freebsd-hackers@freebsd.org; Tue, 16 Oct 2018 07:25:57 +0200
Received: from [82.135.62.35] (helo=mail.embedded-brains.de)
by sslproxy05.your-server.de with esmtpsa
(TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89)
(envelope-from <sebastian.huber@embedded-brains.de>)
id 1gCHrh-000NeZ-1n
for freebsd-hackers@freebsd.org; Tue, 16 Oct 2018 07:25:57 +0200
Received: from localhost (localhost.localhost [127.0.0.1])
by mail.embedded-brains.de (Postfix) with ESMTP id CE1522A0043
for <freebsd-hackers@freebsd.org>; Tue, 16 Oct 2018 07:25:56 +0200 (CEST)
Received: from mail.embedded-brains.de ([127.0.0.1])
by localhost (zimbra.eb.localhost [127.0.0.1]) (amavisd-new, port 10032)
with ESMTP id ihAY8Yjrq4e0 for <freebsd-hackers@freebsd.org>;
Tue, 16 Oct 2018 07:25:56 +0200 (CEST)
Received: from localhost (localhost.localhost [127.0.0.1])
by mail.embedded-brains.de (Postfix) with ESMTP id 4E4AC2A165C
for <freebsd-hackers@freebsd.org>; Tue, 16 Oct 2018 07:25:56 +0200 (CEST)
X-Virus-Scanned: amavisd-new at zimbra.eb.localhost
Received: from mail.embedded-brains.de ([127.0.0.1])
by localhost (zimbra.eb.localhost [127.0.0.1]) (amavisd-new, port 10026)
with ESMTP id voqwhLRRrt-U for <freebsd-hackers@freebsd.org>;
Tue, 16 Oct 2018 07:25:56 +0200 (CEST)
Received: from huber-nb-linux.suse (unknown [192.168.96.161])
by mail.embedded-brains.de (Postfix) with ESMTPSA id 389802A0043
for <freebsd-hackers@freebsd.org>; Tue, 16 Oct 2018 07:25:56 +0200 (CEST)
Subject: Re: Why SPDX-License-Identifier: BSD-2-Clause-FreeBSD?
From: Sebastian Huber <sebastian.huber@embedded-brains.de>
To: FreeBSD <freebsd-hackers@freebsd.org>
References: <f396d707-a707-837e-e76f-fec3a7fba2cf@embedded-brains.de>
Message-ID: <a32726a3-762c-6196-0def-f79e4e025a4c@embedded-brains.de>
Date: Tue, 16 Oct 2018 07:25:55 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
Thunderbird/60.2.1
MIME-Version: 1.0
In-Reply-To: <f396d707-a707-837e-e76f-fec3a7fba2cf@embedded-brains.de>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: quoted-printable
X-Authenticated-Sender: smtp-embedded@poldinet.de
X-Virus-Scanned: Clear (ClamAV 0.100.1/25040/Mon Oct 15 23:01:09 2018)
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 16 Oct 2018 05:26:01 -0000
Hello,
it seems the master list of SPDX Identifiers is maintained by the Linux=20
Foundation and not OSI:
https://spdx.org/licenses/
The BSD-2-Clause-FreeBSD is there:
https://spdx.org/licenses/BSD-2-Clause-FreeBSD.html
--=20
Sebastian Huber, embedded brains GmbH
Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone : +49 89 189 47 41-16
Fax : +49 89 189 47 41-09
E-Mail : sebastian.huber@embedded-brains.de
PGP : Public key available on request.
Diese Nachricht ist keine gesch=C3=A4ftliche Mitteilung im Sinne des EHUG=
.
From owner-freebsd-hackers@freebsd.org Tue Oct 16 10:03:10 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2FD7210C824B
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Tue, 16 Oct 2018 10:03:10 +0000 (UTC)
(envelope-from marklmi@yahoo.com)
Received: from sonic317-29.consmr.mail.bf2.yahoo.com
(sonic317-29.consmr.mail.bf2.yahoo.com [74.6.129.84])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id C612F8478A
for <freebsd-hackers@freebsd.org>; Tue, 16 Oct 2018 10:03:09 +0000 (UTC)
(envelope-from marklmi@yahoo.com)
X-YMail-OSG: rJfGS9MVM1kXWwo4DiGN6NERK8c6c9xS.dHBPuUxPgJkU9DpeUjPl_expNe7eYy
7xnZAZ9msv2ueSY9sRiXa3k26BOJqgl0GHULCcJwqHkmrXeS.EOYH2KKUjt0s.zov5Dk8HleST_n
R3b6ZdInReAb1CDKRc4dDjKYyGopLUUEXoh6R8KaIclBJmaPOLaU0Cy6JoHtCudFAbUzaNKTfvnm
Lz1x9gRNHVncfHdZCYx55InBiKw3wXw5E8WOqwTSxOjr.CXP69OMT65WQCX3BAIJhrDchmTQIiqc
CIx7ChcwsvUepS6we9z5r03YgS.9AjHfdfwUjw4BSCd57cuVVwSFM.YSOe9_9OpG2LVLXzYGwUt7
yUhxbaeOOWkltMpPSfbSn2l82SEFc_OuBU0L7GEgZFhkee3eSwkHoylsr6QITpo7fBDmEKusSdzg
q7me5D9v1L4YB5cnHPVemP2N0Ez0qCLbWAgcc29ReBFFACXsWV9.7_L3pIhLr34lp2mWtwZbbAk.
1zzlDxVfcClmGLdXP7RQZTNm9H6rRQvhtVBoG0TrUg29sNO_OXwaVQoaa70fsDceloEH2RVIoB67
qPo9WHgnljuQOFOZkpNdeDlyLWULNn2AouUjl4h5DKHD0xkIewYZn5SQp_w.lo8I.zNXshpcyuy_
z1jkXj5q3Ks6yaz8WCF8O58dPAHkkPfZMZK5vABMQ6o9awIkcairktmX9xWnKTiotifLvYM4ZWWx
QDZa47befGb8EymlBKCkd54Xzlgkffo2qdUhcClEcVPKc.uC0AxnGTrVbypVtvzp71k3l.B2ImiD
u9laZEZujwKFcw0LcCxrNNjvv21cEF1PHSwtoVIYMSIqQ7kptw8vQWyajuJ7XGzxyIO7h7bQ9IPP
bq_poOba7MLMflQD6c6dI9nSHA_KQp9V9j3G89kDEVkJ371nhBJ7AQx_IRyQkX8VH5hjYWkHaJRO
enhllxAOsUcfO2StBxMB7B5WsyB3CVtJ2wHt_H9QuLstOqQZw9fzBKBdcqvWClAexcaErmB0reJ3
IM5dgnLbkOkw29Sv39p0CCKQvRPmUv7hnryiP81QaQBYKGflsi7869IcyJ9mgZK9IbsS9LqfN2Ru
n84vXc7Y-
Received: from sonic.gate.mail.ne1.yahoo.com by
sonic317.consmr.mail.bf2.yahoo.com with HTTP; Tue, 16 Oct 2018 10:03:03 +0000
Received: from c-76-115-7-162.hsd1.or.comcast.net (EHLO [192.168.1.25])
([76.115.7.162])
by smtp424.mail.bf1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID
74cf5700833911eba1085e7c2c1aa0d0;
Tue, 16 Oct 2018 10:02:59 +0000 (UTC)
Content-Type: text/plain;
charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\))
Subject: Re: Why SPDX-License-Identifier: BSD-2-Clause-FreeBSD?
From: Mark Millard <marklmi@yahoo.com>
In-Reply-To: <a32726a3-762c-6196-0def-f79e4e025a4c@embedded-brains.de>
Date: Tue, 16 Oct 2018 03:02:56 -0700
Cc: FreeBSD <freebsd-hackers@freebsd.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <8A9D0204-17A7-480D-AD51-58A518AE6E27@yahoo.com>
References: <f396d707-a707-837e-e76f-fec3a7fba2cf@embedded-brains.de>
<a32726a3-762c-6196-0def-f79e4e025a4c@embedded-brains.de>
To: Sebastian Huber <sebastian.huber@embedded-brains.de>
X-Mailer: Apple Mail (2.3445.9.1)
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 16 Oct 2018 10:03:10 -0000
On 2018-Oct-15, at 10:25 PM, Sebastian Huber =
<sebastian.huber@embedded-brains.de> wrote:
> Hello,
>=20
> it seems the master list of SPDX Identifiers is maintained by the =
Linux Foundation and not OSI:
>=20
> https://spdx.org/licenses/
>=20
> The BSD-2-Clause-FreeBSD is there:
>=20
> https://spdx.org/licenses/BSD-2-Clause-FreeBSD.html
And it does not agree with the other 2, listing from the above
first:
PROVIDED BY THE FREEBSD PROJECT
IN NO EVENT SHALL THE FREEBSD PROJECT OR CONTRIBUTORS BE LIABLE
(yep: no "AND CONTRIBUTORS" for the first line that I listed)
vs.
PROVIDED BY THE AUTHOR AND CONTRIBUTORS
IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
vs.
PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
Messy. (I've not double checked the rest of the wording.)
=3D=3D=3D
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)
From owner-freebsd-hackers@freebsd.org Tue Oct 16 15:13:47 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 41A5310D66F9
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Tue, 16 Oct 2018 15:13:47 +0000 (UTC)
(envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net)
Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140])
(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id B785D70C19
for <freebsd-hackers@freebsd.org>; Tue, 16 Oct 2018 15:13:46 +0000 (UTC)
(envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net)
Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1])
by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w9GFDPQI071556;
Tue, 16 Oct 2018 08:13:25 -0700 (PDT)
(envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net)
Received: (from freebsd-rwg@localhost)
by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w9GFDPCA071555;
Tue, 16 Oct 2018 08:13:25 -0700 (PDT) (envelope-from freebsd-rwg)
From: "Rodney W. Grimes" <freebsd-rwg@pdx.rh.CN85.dnsmgr.net>
Message-Id: <201810161513.w9GFDPCA071555@pdx.rh.CN85.dnsmgr.net>
Subject: Re: Why SPDX-License-Identifier: BSD-2-Clause-FreeBSD?
In-Reply-To: <8A9D0204-17A7-480D-AD51-58A518AE6E27@yahoo.com>
To: Mark Millard <marklmi@yahoo.com>
Date: Tue, 16 Oct 2018 08:13:24 -0700 (PDT)
CC: Sebastian Huber <sebastian.huber@embedded-brains.de>,
FreeBSD <freebsd-hackers@freebsd.org>
X-Mailer: ELM [version 2.4ME+ PL121h (25)]
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 16 Oct 2018 15:13:47 -0000
> On 2018-Oct-15, at 10:25 PM, Sebastian Huber <sebastian.huber@embedded-brains.de> wrote:
>
> > Hello,
> >
> > it seems the master list of SPDX Identifiers is maintained by the Linux Foundation and not OSI:
> >
> > https://spdx.org/licenses/
> >
> > The BSD-2-Clause-FreeBSD is there:
> >
> > https://spdx.org/licenses/BSD-2-Clause-FreeBSD.html
>
> And it does not agree with the other 2, listing from the above
> first:
>
> PROVIDED BY THE FREEBSD PROJECT
> IN NO EVENT SHALL THE FREEBSD PROJECT OR CONTRIBUTORS BE LIABLE
>
> (yep: no "AND CONTRIBUTORS" for the first line that I listed)
>
> vs.
>
> PROVIDED BY THE AUTHOR AND CONTRIBUTORS
> IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
>
> vs.
>
> PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
>
>
> Messy. (I've not double checked the rest of the wording.)
And this is why ultimately SPDX tagging should leave the original
license in a file in place, this dis-joint nature creates space
for these types of problems. Rather simple to just not have
them and leave the license in the file as has been done for 30
years.
--
Rod Grimes rgrimes@freebsd.org
From owner-freebsd-hackers@freebsd.org Tue Oct 16 16:06:56 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id AE81110D8C46
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Tue, 16 Oct 2018 16:06:56 +0000 (UTC)
(envelope-from bzeeb-lists@lists.zabbadoz.net)
Received: from mx1.sbone.de (cross.sbone.de [195.201.62.131])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified))
by mx1.freebsd.org (Postfix) with ESMTPS id 361EA7376E;
Tue, 16 Oct 2018 16:06:55 +0000 (UTC)
(envelope-from bzeeb-lists@lists.zabbadoz.net)
Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587])
(using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits))
(No client certificate requested)
by mx1.sbone.de (Postfix) with ESMTPS id 43DA48D4A163;
Tue, 16 Oct 2018 16:06:48 +0000 (UTC)
Received: from content-filter.sbone.de (content-filter.sbone.de
[IPv6:fde9:577b:c1a9:31::2013:2742])
(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
(No client certificate requested)
by mail.sbone.de (Postfix) with ESMTPS id 5DE61D1F851;
Tue, 16 Oct 2018 16:06:47 +0000 (UTC)
X-Virus-Scanned: amavisd-new at sbone.de
Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587])
by content-filter.sbone.de (content-filter.sbone.de
[fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024)
with ESMTP id 5S1Zpe8ejr4D; Tue, 16 Oct 2018 16:06:45 +0000 (UTC)
Received: from [10.248.106.11] (fresh-ayiya.sbone.de
[IPv6:fde9:577b:c1a9:f001::2])
(using TLSv1 with cipher AES256-SHA (256/256 bits))
(No client certificate requested)
by mail.sbone.de (Postfix) with ESMTPSA id 102B8D1F835;
Tue, 16 Oct 2018 16:06:44 +0000 (UTC)
From: "Bjoern A. Zeeb" <bzeeb-lists@lists.zabbadoz.net>
To: "Mark Johnston" <markj@freebsd.org>
Cc: freebsd-hackers@freebsd.org
Subject: Re: [CFT] capsicum patches for rtsol(8) and rtsold(8)
Date: Tue, 16 Oct 2018 16:06:43 +0000
X-Mailer: MailMate (2.0BETAr6123)
Message-ID: <E9282C14-3AEE-4D07-BD0A-38550F974DA9@lists.zabbadoz.net>
In-Reply-To: <20181015194212.GA2751@spy>
References: <20181015194212.GA2751@spy>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 16 Oct 2018 16:06:56 -0000
On 15 Oct 2018, at 19:42, Mark Johnston wrote:
> https://people.freebsd.org/~markj/patches/rtsold_capsicum.diff
(0) the git rename doesn’t really work when applying the diff with
FreeBSD’s patch so the mv has to be done manually
(1) the rtsol Makefile also needs cap_syslog and util to link to
otherwise rtsold.c has unresolved symbols
(2) rtsol seem to have worked when manually invoked; /etc/resolv.conf
was created (I had rm’ed it) and the 3 nameserver lines re-appeared;
sorry can’t test the search string here
(3) rtsold crashes:
root@i386-a3-carp:/ # gdb82 /usr/sbin/rtsold /rtsold.core
GNU gdb (GDB) 8.2 [GDB v8.2 for FreeBSD]
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "i386-portbld-freebsd12.0".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/sbin/rtsold...Reading symbols from
/usr/lib/debug//usr/sbin/rtsold.debug...done.
done.
[New LWP 100091]
Core was generated by `/usr/sbin/rtsold -a'.
Program terminated with signal SIGABRT, Aborted.
#0 thr_kill () at thr_kill.S:3
3 thr_kill.S: No such file or directory.
(gdb) where
#0 thr_kill () at thr_kill.S:3
#1 0x281ca3a1 in __raise (s=6) at /usr/src/lib/libc/gen/raise.c:52
#2 0x281ca2e3 in abort () at /usr/src/lib/libc/stdlib/abort.c:67
#3 0x28246a98 in __assert (func=0x28092479 "cap_xfer_nvlist",
file=0x280925a3 "/usr/src/lib/libcasper/libcasper/libcasper.c",
line=304, failedexpr=0x280924d6 "chan != NULL") at
/usr/src/lib/libc/gen/assert.c:51
#4 0x28095a60 in cap_xfer_nvlist (chan=0x0, nvl=0x282940a0) at
/usr/src/lib/libcasper/libcasper/libcasper.c:304
#5 0x2809a1ce in cap_vsyslog (chan=0x0, priority=3, fmt=0xffbfe7cc
"<main> failed to initialize capabilities: %s", ap=0xffbfebec
"\210\266'( \354\277\377 ") at
/usr/src/lib/libcasper/services/cap_syslog/cap_syslog.c:72
#6 0x0804af2f in warnmsg (priority=3, func=0x804f163 "main",
msg=0x804f168 "failed to initialize capabilities: %s") at
/usr/src/usr.sbin/rtsold/rtsold.c:787
#7 0x0804abb3 in main (argc=<optimized out>, argv=<optimized out>) at
/usr/src/usr.sbin/rtsold/rtsold.c:255
/bz
From owner-freebsd-hackers@freebsd.org Tue Oct 16 16:53:14 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id DFD2810DA5FE
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Tue, 16 Oct 2018 16:53:13 +0000 (UTC)
(envelope-from markjdb@gmail.com)
Received: from mail-pg1-x541.google.com (mail-pg1-x541.google.com
[IPv6:2607:f8b0:4864:20::541])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(Client CN "smtp.gmail.com",
Issuer "Google Internet Authority G3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 5D6A7754EB
for <freebsd-hackers@freebsd.org>; Tue, 16 Oct 2018 16:53:13 +0000 (UTC)
(envelope-from markjdb@gmail.com)
Received: by mail-pg1-x541.google.com with SMTP id y18-v6so11145013pge.0
for <freebsd-hackers@freebsd.org>; Tue, 16 Oct 2018 09:53:13 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
h=sender:date:from:to:cc:subject:message-id:references:mime-version
:content-disposition:content-transfer-encoding:in-reply-to
:user-agent; bh=fdqwFYb+AdEEbEB+QmOiDrftzcotK5LmnWnhXLVH9mc=;
b=ACW5BxX8EH62bCSKu29pjCd8Y2hjnE6cyB9VnAzFQe+cCDS0PVGmazwIOZJVEDmLJa
npeWkfAX45j2MxM0KwLTlz1do9Xo7edjPmJ7wTEFuBPhWP/fQ1VC1o2OLIMZrfkcJ9cu
7F5Z8tpPEMuiXWtdgwEdwq5+ZsurkUmkdRdcE9FqdagSoV9HpL7IqiAHw3iwo96MTzxF
PokMXO+ADJ0dviiU0wbbqyzpCLGQ43u7H4IkQE6kXHWTGlpK8KC4mO/LBiMp+hSAnLS+
Dp5iTyuCaVyysbglCAVIVWLtDDPI10/xtE2INtk53oRg/5J0SFZEVLwX/R1KJvPW0E4W
oeSQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:sender:date:from:to:cc:subject:message-id
:references:mime-version:content-disposition
:content-transfer-encoding:in-reply-to:user-agent;
bh=fdqwFYb+AdEEbEB+QmOiDrftzcotK5LmnWnhXLVH9mc=;
b=mLcowJUES4sZGKLkmoqDblZnLSFgCmZhHuJatr8RoklDjq+4Xnv+pX268pcgMO3g/c
kP4c1CMb3Q2Ek7c2oMC8ePPRkaQIuOjst64qxncy+LGqgDseYnAlNisZ7xImrmInnpC/
mRtIGrjbBLv2Yu4QYf97b5OtCDUk5MJzdSVBs2wMm7mDkhcpvzK3ebK0GuvakK4sopmH
33MCl3AB6wxD5YbHiKHf6Lx+8fn5kdB8Xb9kLkyoiJTXl6peHpQB63fQux2vVnVWcGko
uP6jI3T+FeYzqo7jHM8Mq6YpS29vEcRQzH3vOSoFPRUTPeBKvwTh9AzDWPoi1NTwVW0a
NQdA==
X-Gm-Message-State: ABuFfojZbCwG7R2O03bwJRrQfHUoebTIootJTyB0JtHaMdxbVHu7Fxet
YoD/XZJww+JK3QGW9ed9Iw4NIUXoZ60=
X-Google-Smtp-Source: ACcGV63637wVCFxE+/cHW2OcDXVmAn798m0Y8h7hHGm9tPI4TG7fCeTqPzl5POH38PyxMxS9xlYXJA==
X-Received: by 2002:a63:510a:: with SMTP id
f10-v6mr20729888pgb.31.1539708792206;
Tue, 16 Oct 2018 09:53:12 -0700 (PDT)
Received: from raichu (toroon0560w-lp130-09-70-52-226-56.dsl.bell.ca.
[70.52.226.56])
by smtp.gmail.com with ESMTPSA id l71-v6sm28030249pge.89.2018.10.16.09.53.10
(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
Tue, 16 Oct 2018 09:53:11 -0700 (PDT)
Sender: Mark Johnston <markjdb@gmail.com>
Date: Tue, 16 Oct 2018 12:53:08 -0400
From: Mark Johnston <markj@freebsd.org>
To: "Bjoern A. Zeeb" <bzeeb-lists@lists.zabbadoz.net>
Cc: freebsd-hackers@freebsd.org
Subject: Re: [CFT] capsicum patches for rtsol(8) and rtsold(8)
Message-ID: <20181016165308.GB5066@raichu>
References: <20181015194212.GA2751@spy>
<E9282C14-3AEE-4D07-BD0A-38550F974DA9@lists.zabbadoz.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <E9282C14-3AEE-4D07-BD0A-38550F974DA9@lists.zabbadoz.net>
User-Agent: Mutt/1.10.1 (2018-07-13)
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 16 Oct 2018 16:53:14 -0000
On Tue, Oct 16, 2018 at 04:06:43PM +0000, Bjoern A. Zeeb wrote:
> On 15 Oct 2018, at 19:42, Mark Johnston wrote:
>
> > https://people.freebsd.org/~markj/patches/rtsold_capsicum.diff
>
> (0) the git rename doesn’t really work when applying the diff with
> FreeBSD’s patch so the mv has to be done manually
>
> (1) the rtsol Makefile also needs cap_syslog and util to link to
> otherwise rtsold.c has unresolved symbols
>
> (2) rtsol seem to have worked when manually invoked; /etc/resolv.conf
> was created (I had rm’ed it) and the 3 nameserver lines re-appeared;
> sorry can’t test the search string here
>
> (3) rtsold crashes:
Thanks. I made some last-minute changes and forgot to retest, of
course. :(
I uploaded a new patch which should fix all of these issues - could you
give it a try?
From owner-freebsd-hackers@freebsd.org Tue Oct 16 18:29:57 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 731A110DD085
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Tue, 16 Oct 2018 18:29:57 +0000 (UTC)
(envelope-from bzeeb-lists@lists.zabbadoz.net)
Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:13b:39f::9f:25])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified))
by mx1.freebsd.org (Postfix) with ESMTPS id 0F2547947A;
Tue, 16 Oct 2018 18:29:56 +0000 (UTC)
(envelope-from bzeeb-lists@lists.zabbadoz.net)
Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587])
(using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits))
(No client certificate requested)
by mx1.sbone.de (Postfix) with ESMTPS id 351C58D4A12D;
Tue, 16 Oct 2018 18:29:54 +0000 (UTC)
Received: from content-filter.sbone.de (content-filter.sbone.de
[IPv6:fde9:577b:c1a9:31::2013:2742])
(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
(No client certificate requested)
by mail.sbone.de (Postfix) with ESMTPS id E0B76D1F83F;
Tue, 16 Oct 2018 18:29:53 +0000 (UTC)
X-Virus-Scanned: amavisd-new at sbone.de
Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587])
by content-filter.sbone.de (content-filter.sbone.de
[fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024)
with ESMTP id WH_VTR52GXzt; Tue, 16 Oct 2018 18:29:51 +0000 (UTC)
Received: from [10.248.106.11] (fresh-ayiya.sbone.de
[IPv6:fde9:577b:c1a9:f001::2])
(using TLSv1 with cipher AES256-SHA (256/256 bits))
(No client certificate requested)
by mail.sbone.de (Postfix) with ESMTPSA id 26900D1F835;
Tue, 16 Oct 2018 18:29:50 +0000 (UTC)
From: "Bjoern A. Zeeb" <bzeeb-lists@lists.zabbadoz.net>
To: "Mark Johnston" <markj@freebsd.org>
Cc: freebsd-hackers@freebsd.org
Subject: Re: [CFT] capsicum patches for rtsol(8) and rtsold(8)
Date: Tue, 16 Oct 2018 18:29:49 +0000
X-Mailer: MailMate (2.0BETAr6123)
Message-ID: <86D87437-BD34-489A-87B7-33F1089080EE@lists.zabbadoz.net>
In-Reply-To: <20181016165308.GB5066@raichu>
References: <20181015194212.GA2751@spy>
<E9282C14-3AEE-4D07-BD0A-38550F974DA9@lists.zabbadoz.net>
<20181016165308.GB5066@raichu>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 16 Oct 2018 18:29:57 -0000
On 16 Oct 2018, at 16:53, Mark Johnston wrote:
> On Tue, Oct 16, 2018 at 04:06:43PM +0000, Bjoern A. Zeeb wrote:
>> On 15 Oct 2018, at 19:42, Mark Johnston wrote:
>>
>>> https://people.freebsd.org/~markj/patches/rtsold_capsicum.diff
>>
>> (0) the git rename doesn’t really work when applying the diff with
>> FreeBSD’s patch so the mv has to be done manually
>>
>> (1) the rtsol Makefile also needs cap_syslog and util to link to
>> otherwise rtsold.c has unresolved symbols
>>
>> (2) rtsol seem to have worked when manually invoked;
>> /etc/resolv.conf
>> was created (I had rm’ed it) and the 3 nameserver lines
>> re-appeared;
>> sorry can’t test the search string here
>>
>> (3) rtsold crashes:
>
> Thanks. I made some last-minute changes and forgot to retest, of
> course. :(
>
> I uploaded a new patch which should fix all of these issues - could
> you
> give it a try?
With the old and new patch:
root@i386-a3-carp:/usr/src/sbin/rtsol # rtsol vtnet0
failed to run script: Invalid argument
Hadn’t noticed that before.
Also on a running system:
root@i386-a3-carp:/ # rm /etc/resolv.conf
root@i386-a3-carp:/ # cat /etc/resolv.conf
cat: /etc/resolv.conf: No such file or directory
root@i386-a3-carp:/ # sh /etc/rc.d/rtsold restart
Stopping rtsold.
Waiting for PIDS: 1047.
Starting rtsold.
root@i386-a3-carp:/ # cat /etc/resolv.conf
cat: /etc/resolv.conf: No such file or directory
After a reboot:
root@i386-a3-carp:~ # ps ax | grep rts
372 - SCs 0:00.01 /usr/sbin/rtsold -a
375 - Ss 0:00.01 rtsold: system.syslog (rtsold)
376 - S 0:00.00 rtsold: rtsold.script (rtsold)
693 u0 R+ 0:00.01 grep rts
root@i386-a3-carp:~ # cat /etc/resolv.conf
# Generated by resolvconf
nameserver ..
root@i386-a3-carp:~ # rtsol vtnet0
failed to run script: Invalid argument
root@i386-a3-carp:~ # sh /etc/rc.d/rtsold restart
Stopping rtsold.
Waiting for PIDS: 372.
Starting rtsold.
Oct 16 18:28:33 i386-a3-carp rtsold[734]: <call_script> failed to run
script: Invalid argument
/bz
From owner-freebsd-hackers@freebsd.org Tue Oct 16 19:00:00 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id CCAC410DDDA9;
Tue, 16 Oct 2018 19:00:00 +0000 (UTC) (envelope-from lev@FreeBSD.org)
Received: from onlyone.not-for.work (onlyone.not-for.work [148.251.9.81])
by mx1.freebsd.org (Postfix) with ESMTP id 5C2687A51E;
Tue, 16 Oct 2018 19:00:00 +0000 (UTC) (envelope-from lev@FreeBSD.org)
Received: from [192.168.23.186] (unknown [89.113.128.32])
(Authenticated sender: lev@serebryakov.spb.ru)
by onlyone.not-for.work (Postfix) with ESMTPSA id 71FFDD33;
Tue, 16 Oct 2018 21:59:59 +0300 (MSK)
Reply-To: lev@FreeBSD.org
To: freebsd-hackers@FreeBSD.org, freebsd-security@freebsd.org
From: Lev Serebryakov <lev@FreeBSD.org>
Subject: =?UTF-8?Q?cryptodev_/_softcrypto_=e2=80=94_are_here_any_plans_to_cl?=
=?UTF-8?Q?eanup_it=3f?=
Openpgp: preference=signencrypt
Autocrypt: addr=lev@FreeBSD.org; prefer-encrypt=mutual; keydata=
xsFNBFKbGksBEADeguVs+XyJc3mL3iiOBqDd16wSk97YTJYOi4VsHsINzJr09oFvNDiaDBIi
fLn2p8XcJvehcsF2GSgrfXfw+uK4O1jyNIKJmiYA0EtE+ZbRtvDrrE0w6Q8+SDeKA21SWh3Y
vSQ0DJUontbgW55ER2CbEiIUTIn34uQ0kmESAaw/v5p/9ue8yPTmURvv130FqPFz8VPzltqL
NxyGt54TxPfKAzAHEIwxlEZ63JOwzloKh1UDBExcsf9nJO08/TAVgR5UZ5njFBPzaaquhRoP
qPJLEQQDqxPIlvMNtHKf7iIebE4BHeqgCdJA0BoiR6gpa0wlsZtdrTPK3n4wYSphLvGbhfOZ
YW/hbcu7HYS/FImkVxB3iY17kcC1UTnx4ZaYeASPBGOOPbXky1lLfmDGWIFT//70yx+G17qD
OZzF1SvJJhGvh6ilFYaWMX7T+nIp6Mcafc4D7AakXM+XdubNXOMlCJhzPcZ0skgAEnYV587w
V7em5fDVwQccwvtfezzqKeJAU5TGiywBHSR5Svzk2FwRNf6M//hWkpq0SRR63iOhkHGOAEBi
69GfEIwH2/w24rLxP0E+Hqq8n+EWNkPatw1Mhcl5PKkdvGCjJUaGNMkpBffjyYo254JXRscR
eEnwdIkJt4ErDvjb2/UrOFq31wWMOiLzJeVchAgvTHBMRfP9aQARAQABzShMZXYgU2VyZWJy
eWFrb3YgPGxldkBzZXJlYnJ5YWtvdi5zcGIucnU+wsGCBBMBCAAsAhsDBwsJCAcDAgEGFQgC
CQoLBBYCAwECHgECF4ACGQEFAlKbP8wFCQlmJwEACgkQ6rA8WL/cR4/6VBAAjRMyyX3PBFx/
HxyiIZ698EfwlWUua8Ft4crtrdK52m0qNkbBB9BH8xQgBHG32A1CwyzQnzxHgZuoOWMjh+Qq
WJv7dmpM/q/c1GCJHhlPgewXrciTwpAamZILN071u+1GCPWwGRPzfQ/U+k63KJWx9ozf4doM
WTTom6Cqcssi4J1u5kkt52a5ZRhsCK9pEVGilk36XTP9BakGrnMSIxF/NK4xeZVX2q+Nuqvf
RchyofKXVgLEDLwb1cd/baLtBpDzy0PTN2Zl2lX4kOA6jwTKsqRya9A1Vui1KXwPh2XViTQ1
7Y3l5qg/M+sR73DohezP6bO6huOnLhty17jAqHPNlD6RonDo+j8uIlEg4iMSTN3MhzkBAu0Q
pe3ucQ0o1767JiXN3fsNvRzSFhLVNDqPLce4uKlMogsbreXWvdgHGTN1ybOHGbybZnP77yHz
uNBacbmG3vL/OLXMqwLdL2JXoiec4DmXjjCdhTBl5xLV9Hz/6VWKqElteg8QFVvHB3tHWzJ4
/rpiVEixytCIII6DS33BXZ0h2EOkK/6AYA2SJxy1vgOH4SZBtDBHoezmHV2nFnq5O0c7AuAB
7WPWgQG0sEwHQPZmg/baRGitRJnaxf/Gvf1DeD1x1VrcoVke2vwBcgDM3kugP8L9hsqic2D3
dI+gP76haeuvNNZr3y9L9zvOwU0EUpsaSwEQALRr3B+OjY/cnJPstz5CVsVWyEZtJtrNviZr
tBgbkhlkPm98sEWR4+gbpyeufdYJengDjeGzMDKcLB7h5fICS/j6A8XdlJ40TlbPfNgb6OHa
ebaIYKTJpXKR9sD7ZyGivYMofm0em40wGUX7BIkdkomaWj+wUiS0CdXU0FWDj9wv73+Eim+X
zZyXeFgIPv97v+pET7DfwKkADOfrkW9s4OfvGVjd+wm35wc8EngQEz0qdPBxx74X7vZFAxlA
SXu8gDBJGYt2Bkc3QwULnfeXrZJWgqNPR5o44gGu96yaiOFaN/C6CJtev5ZEX+0ZxbvsHHB7
Z5AtsRURKpZ4w5HFHGhzHtDtoAKgeZ/gbhTVXPHvNQR818eN+Nl5BV8BRF/8yhR6VlJb8GYw
h8oKDeVGVYC34+raHZQAM9WoBnN7jlt4T9zzPwtmw5mIahGFgvw1KDr7OItN2ZgtZ20UYC5m
Go602nmHq0aPbU6SwGi1xohrliNsKaaciYiMaVIGRQq8iGr9Fe2HlvaA3BpB275i/gCVlUdG
y5XLAv+yQMUvn5Z7XVsMroxDk/O+ae1ElyBvKiKyfWGJXTg5XUukkkyQmfWPxWUGoNA1P/P4
GMHSu7/Rqe/7m4uPu/RyTTqsSjjKJdP9kBwEzvqPtXsVoZuShtrptRQJDYflhgE4qmKSMKen
ABEBAAHCwWUEGAECAA8FAlKbGksCGwwFCRLMAwAACgkQ6rA8WL/cR48RkA//SNzeW3CI8KHx
rA0aeHW6Nb5ieoqVRBGLyjBM06RX6vHB9v4dJL6Z+yV2jGN2s+XZX2HILbuTOwcTxGkI3xTT
e0cDXVaF5K8R/liigUjtwuC2v/sWgoWyUmK1Cy9CPYdcXmFq6nESfkUe8DYiGOUULdHq5w63
F53yOZ72iXRBQBZgkhPtRFu4lPYIzOsMag9DIJ9CthR1r0ziqU/keb94Qt3l+aXK7CwGdY7X
T4zUIMHNYsuAuyX+NJIXfsN68TT6m7QmlUwxPs13nxmoVQzm4ruV+hlQKh1MtbsjWRkNgPxF
IPiqoAEhy8QoddlSvRTwL5Z7zFQiwMdiXU7toL8pfzj/zJR1jELXKMipijrt5MLrV8XX3OPN
yZZvh95VIl8mv+iAqwSZUufd2EJnvj5TObB0eH+a+34NWf/XqA3fPjE6KHzmdnw9PZjPEjlx
JCPECSs+6gse1+GaEfKYuXzB/ENe2ctlcfx5iQJXFc+/+zG/uU/JX/pXJHA12CUfB5g7lH6X
BZIHvRo3VTCDjXgbF5xxDAe5V4exf8d4oSNjQIFLYxxN7zkvH89EN6RPfRgsWN7bYArCwfS9
MOgs9pFeCOewR6qieK150aoqNENGfKFXJup+5VVl6I0mU+j0rgVDZDht2/QgP/Tb4lGBe+ai
pOGaK/GYNR+Ad6bUmokKsx4=
Organization: FreeBSD
Message-ID: <e7cbc29c-2095-00e0-76cd-b2bfe5598a5a@FreeBSD.org>
Date: Tue, 16 Oct 2018 21:59:58 +0300
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101
Thunderbird/52.9.1
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
protocol="application/pgp-signature";
boundary="SlhqVpYD81pXqqbuVsGylhY9yPEmNDF0y"
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 16 Oct 2018 19:00:01 -0000
This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--SlhqVpYD81pXqqbuVsGylhY9yPEmNDF0y
Content-Type: multipart/mixed; boundary="rlo8v78kM9GO3VaQA0xO1kGEI5BitlY2L";
protected-headers="v1"
From: Lev Serebryakov <lev@FreeBSD.org>
Reply-To: lev@FreeBSD.org
To: freebsd-hackers@FreeBSD.org, freebsd-security@freebsd.org
Message-ID: <e7cbc29c-2095-00e0-76cd-b2bfe5598a5a@FreeBSD.org>
Subject: =?UTF-8?Q?cryptodev_/_softcrypto_=e2=80=94_are_here_any_plans_to_cl?=
=?UTF-8?Q?eanup_it=3f?=
--rlo8v78kM9GO3VaQA0xO1kGEI5BitlY2L
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
To be honest, I'm surprised by inconsistency of our kernel crypto
infrastructure.
"struct enc_xform" contains context size, but "struct auth_hash" doesn'=
t.
Memory management is different for auth algorithms and encryption
algorithms.
There is Setkey for auth algorithms, but it is mostly unused.
There is no way to re-key encryption without re-allocating context
("key" or "schedule", even naming is not consistent). Ouch.
As I could see by commits, there was some simplifications , but,
maybe, here is project to cleanup this subsystem?
--=20
// Lev Serebryakov
--rlo8v78kM9GO3VaQA0xO1kGEI5BitlY2L--
--SlhqVpYD81pXqqbuVsGylhY9yPEmNDF0y
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQKTBAEBCgB9FiEE+W0coLX0MYtnSzMK6rA8WL/cR48FAlvGNS5fFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEY5
NkQxQ0EwQjVGNDMxOEI2NzRCMzMwQUVBQjAzQzU4QkZEQzQ3OEYACgkQ6rA8WL/c
R4+ZQA//VPjxdBD+p55GXyLyDUDcGes/AQ0hmoZM29LluvoolQeWTmP23YfGkXv6
11uE4F/lGy0WCGFok24MVwtjMwS8RuDuTuIfahelZYJQc0IReImab4DP8KR+x23h
tqi0oUj+iunMSebLHFTNEtu0yqoKKBtjWJo24afn9gAyLisxit2s+U9wfqzFxUIO
fSf9HOlMJSmHBrSQg919Hs/QDlL1+QxiKLP2V8aeu6kCrhYao/bqYwHFW/BokYUk
R6m2FcgoiNupRM6P4V/gk3qGIRyaRj0V5/MZSK87a5grLKqLkbjoIeXzbX3TS0p8
zTa90fDSB0GGDkw/k4GbHsKkt0d+eRZbm+krTnCGd3NhZ+ByM9/WC8OjMLW9lBUU
FRDsQyJPowYvpmL7m3nnGN+i+h7PbkemypuRw4ZQ+py/Fha0acFq6ef8NVJnIzN1
ZEHZjdjyFVzv4CAE5G0FLSo2za4pMw3BczPr9unyxSyPytYBR6LzuEZLWcmcVQ+2
HAAuiS/8vcpWHz8tDTrFubAgpQP2so0930UQM31mqKxIq75o4ckZVRdvmmvgiTv/
eW8kDFT5tDLSLUPVbvKuwB4VnSKE6YKypeylVXNwq75V4ln3HgrYZ6HXhkr+eNUg
ypERNwkG7wVFIQcjyYhisXJ3n6oyecjA3KPFtYugFgneGu/vUN4=
=Feid
-----END PGP SIGNATURE-----
--SlhqVpYD81pXqqbuVsGylhY9yPEmNDF0y--
From owner-freebsd-hackers@freebsd.org Tue Oct 16 20:04:20 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id EBED410E0421
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Tue, 16 Oct 2018 20:04:19 +0000 (UTC)
(envelope-from markjdb@gmail.com)
Received: from mail-it1-x144.google.com (mail-it1-x144.google.com
[IPv6:2607:f8b0:4864:20::144])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(Client CN "smtp.gmail.com",
Issuer "Google Internet Authority G3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 54C5A7D4A0
for <freebsd-hackers@freebsd.org>; Tue, 16 Oct 2018 20:04:19 +0000 (UTC)
(envelope-from markjdb@gmail.com)
Received: by mail-it1-x144.google.com with SMTP id c85-v6so11506766itd.1
for <freebsd-hackers@freebsd.org>; Tue, 16 Oct 2018 13:04:19 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
h=sender:date:from:to:cc:subject:message-id:references:mime-version
:content-disposition:content-transfer-encoding:in-reply-to
:user-agent; bh=qmumv9Yp9z30KBVBz7np+2DcRfp9LQGJa8gUt8QCx6M=;
b=lv6+H70Z9jyUqlfwpnIzKFGhC3TZ6df4v40tE7H0uVHqsNK9onyaOdvPsKgtS/QqAa
QiwXCrs8TaoVm2PN4ZhErwP7s1+OJyYTLvKIMx7csgs118LMFq3HOZOfpZJAcI1FLciN
5NZCL+dOrHf//DYvcI8PEnYF31+JuP6B64/q1+2tRMavv3SSAoveFP+4z3N+MjUHFyR1
c1+RH8/FQJvTh2RcWNNzOlpEJwatbolAb51nagtBc5kRBxVZ19pYz9A0NmOqGcJoK8VP
vi2F9jBluh5BchlgA7jRXF6f7vLTQkSxvM1GFUgXMNMI/sEDq2rWdqvFhL0B6u5mUhg7
AWhA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:sender:date:from:to:cc:subject:message-id
:references:mime-version:content-disposition
:content-transfer-encoding:in-reply-to:user-agent;
bh=qmumv9Yp9z30KBVBz7np+2DcRfp9LQGJa8gUt8QCx6M=;
b=AnZWjE5fQH2dNV2TrGjaIF1u+cihkpmwPn852keot5XEKJ38hTruaoEzlddgQMaU0B
9Duve9qYn8Ls84BwDggwUPa0mH8+BvSJH7REI+RwFs85BmRAgmRAY/RZBlQlyrTJQ6+m
iupR6Btg8rOjZa/f494YTveRInyWbbco8cBaHfeETpEjOYTSqBluWtwzpheftsDmTlOA
ru0rSxViPNOBkwerNmAhzRjp5EGyINas3Luh4f0zjT00/ueFR5bEbEgcBliCgmHN19sf
qfkNjDMzEa6uUB5ukzr4ZbELDX0JRtwZ6y92wydicGO3st571hZr1mCo3d6hsuyLBOkv
z7nQ==
X-Gm-Message-State: ABuFfojw807D5HHibY33afGxEteKEih0VwslgCid2mWeQ/WRyFV57y0s
lp3Hdiim/W+DpdmOHE7Go52yvxafTi8=
X-Google-Smtp-Source: ACcGV62YeUWMWr+Dvpe8jpLHD67Ab33BR8srfNf4KdOMoaWvI40L/cvmicp2acPDoaF/sU9k86iEaw==
X-Received: by 2002:a24:2993:: with SMTP id
p141-v6mr17129690itp.119.1539720257782;
Tue, 16 Oct 2018 13:04:17 -0700 (PDT)
Received: from raichu (toroon0560w-lp130-09-70-52-226-56.dsl.bell.ca.
[70.52.226.56])
by smtp.gmail.com with ESMTPSA id c26-v6sm5060359itd.18.2018.10.16.13.04.16
(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
Tue, 16 Oct 2018 13:04:16 -0700 (PDT)
Sender: Mark Johnston <markjdb@gmail.com>
Date: Tue, 16 Oct 2018 16:04:14 -0400
From: Mark Johnston <markj@freebsd.org>
To: "Bjoern A. Zeeb" <bzeeb-lists@lists.zabbadoz.net>
Cc: freebsd-hackers@freebsd.org
Subject: Re: [CFT] capsicum patches for rtsol(8) and rtsold(8)
Message-ID: <20181016200414.GD5066@raichu>
References: <20181015194212.GA2751@spy>
<E9282C14-3AEE-4D07-BD0A-38550F974DA9@lists.zabbadoz.net>
<20181016165308.GB5066@raichu>
<86D87437-BD34-489A-87B7-33F1089080EE@lists.zabbadoz.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <86D87437-BD34-489A-87B7-33F1089080EE@lists.zabbadoz.net>
User-Agent: Mutt/1.10.1 (2018-07-13)
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 16 Oct 2018 20:04:20 -0000
On Tue, Oct 16, 2018 at 06:29:49PM +0000, Bjoern A. Zeeb wrote:
> On 16 Oct 2018, at 16:53, Mark Johnston wrote:
>
> > On Tue, Oct 16, 2018 at 04:06:43PM +0000, Bjoern A. Zeeb wrote:
> >> On 15 Oct 2018, at 19:42, Mark Johnston wrote:
> >>
> >>> https://people.freebsd.org/~markj/patches/rtsold_capsicum.diff
> >>
> >> (0) the git rename doesn’t really work when applying the diff with
> >> FreeBSD’s patch so the mv has to be done manually
> >>
> >> (1) the rtsol Makefile also needs cap_syslog and util to link to
> >> otherwise rtsold.c has unresolved symbols
> >>
> >> (2) rtsol seem to have worked when manually invoked;
> >> /etc/resolv.conf
> >> was created (I had rm’ed it) and the 3 nameserver lines
> >> re-appeared;
> >> sorry can’t test the search string here
> >>
> >> (3) rtsold crashes:
> >
> > Thanks. I made some last-minute changes and forgot to retest, of
> > course. :(
> >
> > I uploaded a new patch which should fix all of these issues - could
> > you
> > give it a try?
>
> With the old and new patch:
>
> root@i386-a3-carp:/usr/src/sbin/rtsol # rtsol vtnet0
> failed to run script: Invalid argument
>
> Hadn’t noticed that before.
That's a cosmetic bug. I uploaded a new patch which should fix it.
> Also on a running system:
>
> root@i386-a3-carp:/ # rm /etc/resolv.conf
> root@i386-a3-carp:/ # cat /etc/resolv.conf
> cat: /etc/resolv.conf: No such file or directory
> root@i386-a3-carp:/ # sh /etc/rc.d/rtsold restart
> Stopping rtsold.
> Waiting for PIDS: 1047.
> Starting rtsold.
> root@i386-a3-carp:/ # cat /etc/resolv.conf
> cat: /etc/resolv.conf: No such file or directory
resolvconf -a will only update /etc/resolv.conf if the info in
/var/run/resolvconf/interfaces/vtnet0 has changed, I believe. Try
deleting that file too, and then try running rtsol.
> After a reboot:
>
> root@i386-a3-carp:~ # ps ax | grep rts
> 372 - SCs 0:00.01 /usr/sbin/rtsold -a
> 375 - Ss 0:00.01 rtsold: system.syslog (rtsold)
> 376 - S 0:00.00 rtsold: rtsold.script (rtsold)
> 693 u0 R+ 0:00.01 grep rts
> root@i386-a3-carp:~ # cat /etc/resolv.conf
> # Generated by resolvconf
> nameserver ..
>
> root@i386-a3-carp:~ # rtsol vtnet0
> failed to run script: Invalid argument
> root@i386-a3-carp:~ # sh /etc/rc.d/rtsold restart
> Stopping rtsold.
> Waiting for PIDS: 372.
> Starting rtsold.
> Oct 16 18:28:33 i386-a3-carp rtsold[734]: <call_script> failed to run
> script: Invalid argument
>
>
> /bz
From owner-freebsd-hackers@freebsd.org Tue Oct 16 22:29:56 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id A2EF410E3B4B
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Tue, 16 Oct 2018 22:29:56 +0000 (UTC)
(envelope-from marklmi@yahoo.com)
Received: from sonic314-21.consmr.mail.ne1.yahoo.com
(sonic314-21.consmr.mail.ne1.yahoo.com [66.163.189.147])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id A80A582EE4
for <freebsd-hackers@freebsd.org>; Tue, 16 Oct 2018 22:29:55 +0000 (UTC)
(envelope-from marklmi@yahoo.com)
X-YMail-OSG: H3tKvo8VM1lL61yMh6CAH_DAz0RjTyHGHoAL3Cl41607ETYPC6GQ4Ay.ZiIddB3
lwDmt6xgyj1vd27RIk_uht3rn2WaTSqkjfIlNGCS2h1fOkH16zcXkwkQE4Nab4QZU3yrfLhON7I9
fPRzZxgv_XdMMKaHoE7A1y9Gq3UmCuguJtgU0qkOeuGBjosvCbK.8_JlrkfPDogmAJmgKAWgguZp
CTf_j3qBWQT77b3yTwWZH9KjTMvwNPFPA9hT5rW.ntq.mf8eKpvdVMefPATec02BFPCXGzFUvo.g
3QAOuuiMgsl5BhAhcgunQ50kG0mBLbFYbE62ts8P7ZLzT6193MBbsTFIeZcSVaprvDA9uWKMokQ4
SonwpY1ctf95rU7f7roYBa5MQ4VgT0SjJRWa1rlQtInql9SRVNbfCUSO11bqOZDSdVzwiuKQ9iD5
IyJ1QMNpfHVW.AqAAxq_YYEO4Ut3NYR2piJhtmsLXlvSbGX99Lf8jS5n6ztnwjNcCLsAgrfMKFAx
Bh45DK4JA6Vj3a.VuvscMmUVG8x0Qafbz1MaHB_XkvBonwKefZ8FrB2gDvWaJ3z7i2Bc_WRE5sPA
2NJ_LICzvuWO1flzNOxVW9368VA4ldM90BdwjN_BwbBtxFfKQqxU3kLNFVzOx5BSfkEvrdVNmtf1
FWZuEgB2.u3Kd_t1NgWPYLEK6vXOweuKvWT65QJubcE3TKgiGmfdDsAWkiScalJZHu_KEhopJqUh
bfN13MlcezuRAQAPtm4wcAOSZ1chlKeKyJ.iQ5jVIozGT8YVI6LfoSljXOpOxOhQIyUwzNh6oCkp
BnbFqGyZjjij9SdI4ziNf.dSpFVI19xxF..WAKbDAnYzTLaix.zmctf_E04UOlIAcCktfhafISrF
DXpcsQie2tVifS2dXLqWBtfUONWm7y.FC_IaYFQHCISddhjO5CZ8Kphp_L2uEIg.5obJlX5fw0kp
Qca3bLjPnNU.Yw1qjrFvYAnprzJ_cZhBlXHEvuKC_9cFMx6zHf9rwnVFAumuU5pjec_oQw5HEBFJ
z3HTRXahguBEb.kkXixI3DE4x_zrprV7EM_YG1hB475iqsKySkvKeIzXuXonh4VPFaNdGYaARzmZ
o6SHQABNiz3SjrVga7Q--
Received: from sonic.gate.mail.ne1.yahoo.com by
sonic314.consmr.mail.ne1.yahoo.com with HTTP; Tue, 16 Oct 2018 22:29:48 +0000
Received: from c-76-115-7-162.hsd1.or.comcast.net (EHLO [192.168.1.25])
([76.115.7.162])
by smtp412.mail.ne1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID
513438678ecee606fe797d481f79ca2e;
Tue, 16 Oct 2018 22:29:46 +0000 (UTC)
From: Mark Millard <marklmi@yahoo.com>
Content-Type: text/plain;
charset=us-ascii
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\))
Subject: How to get devel/powerpc-gcc based WITH_LIBCPLUSPLUS= buildworld to
have some throwing of C++ exceptions work (patch)
Message-Id: <86E4687B-280A-4625-A56E-8D6FC4C4675B@yahoo.com>
Date: Tue, 16 Oct 2018 15:29:45 -0700
Cc: Justin Hibbits <chmeeedalf@gmail.com>
To: FreeBSD PowerPC ML <freebsd-ppc@freebsd.org>,
FreeBSD Toolchain <freebsd-toolchain@freebsd.org>,
FreeBSD <freebsd-hackers@freebsd.org>
X-Mailer: Apple Mail (2.3445.9.1)
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 16 Oct 2018 22:29:56 -0000
I now have a patch that gets some basic C++ exception throwing
going for WITH_LIBCPLUSPLUS=3D use when building via
devel/powerpc64-gcc . But the overall mechanism seems to mess
up the handling of powerpc64's "red zone" style of stack
processing in various cases.
I've had recent list submittals reporting that buildworld
using WITH_LIBCPLUSPLUS=3D based on devel/powerpc64-gcc would
get stuck looping in _Unwind_RaiseException. This prevented
use of devel/gdb --which makes extensive use of throwing C++
exceptions in normal operation.
Well, I now have a patch that avoids the problem in
libcxxrt's throw_exception itself that made all throws
get stuck --and so allows some C++ exceptions to be thrown.
See below. (I'm not sure leading spaces will all be
preserved.) Most of text is commentary, not code.
# svnlite diff /usr/src/contrib/libcxxrt/
Index: /usr/src/contrib/libcxxrt/exception.cc
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
--- /usr/src/contrib/libcxxrt/exception.cc (revision 339076)
+++ /usr/src/contrib/libcxxrt/exception.cc (working copy)
@@ -772,10 +772,71 @@
info->globals.uncaughtExceptions++;
=20
_Unwind_Reason_Code err =3D =
_Unwind_RaiseException(&ex->unwindHeader);
+#if !defined(__powerpc64__) && !defined(__ppc64__)
// The _Unwind_RaiseException() function should not return, it =
should
// unwind the stack past this function. If it does return, then =
something
// has gone wrong.
report_failure(err, ex);
+#else
+// NOTE: Only tested for devel/powerpc64-gcc based buildworld
+// because clang still silently ignores
+// __builtin_eh_return(offset,handler) for powerpc64
+// (and powerpc), thus not generating correct output.
+//
+// NOTE: I've no clue if other archtiectures might have
+// analogous issues to powerpc64. I'm not sure
+// about powerpc because of it still being stuck
+// at gcc 4.2.1 . (clang problems and no devel/powerpc-gcc .)
+//
+// The above/normal code produced the following sort of structure
+// for throw_exception. r1 is the stack pointer, note its adjustments
+// via stdu r1,-128(r1) and via addi r1,r1,128 .
+//
+// <throw_exception+0>: mflr r0
+// <throw_exception+4>: std r31,-8(r1)
+// <throw_exception+8>: mr r31,r3
+// <throw_exception+12>: std r0,16(r1)
+// <throw_exception+16>: stdu r1,-128(r1)
+// . . .
+// <throw_exception+140>: bl =
<00000018.plt_call._Unwind_RaiseException@@GCC_3.0>
+// <throw_exception+144>: ld r2,40(r1)
+// <throw_exception+148>: addi r1,r1,128
+// <throw_exception+152>: mr r4,r31
+// <throw_exception+156>: ld r0,16(r1)
+// <throw_exception+160>: ld r31,-8(r1)
+// <throw_exception+164>: mtlr r0
+// <throw_exception+168>: b <report_failure>
+//
+// The loop in __Unwind_RaiseException had its "fs"
+// used with uw_frame_state_for and uw_update_context get
+// stuck with the pc field having the address for
+// throw_exception+152 (just after the stack adjustment
+// addi r1,r1,128). Effectively, throw_exception unwinds
+// its stack use before calling report_failure in a
+// way that throw_exception is no longer on the stack.
+// The exception unwinding logic did not handle this
+// correctly and got stuck looping.
+//
+// The below avoids having any such stack adjustment here
+// by avoiding the report_failure call and directly doing
+// what case _URC_END_OF_STACK in report_failure does for
+// its first couple of lines. (It is also the kind of
+// thing that src/contrib/libstdc++/libsupc++/eh_throw.cc
+// has in its __cxxabiv1::__cxa_throw after the
+// _Unwind_RaiseException call.)
+//
+// Another option could be to turn report_failure into
+// a macro so that no subroutine call could be involved.
+// That should avoid the early stack pointer kadjsutment.
+//
+// Also: For the other archtiectures that I looked at, no
+// such stack adjsutments were involved in the code
+// generated (or the matching dwarfdump output).
+// But I did not look at many.
+
+ __cxa_begin_catch (&(ex->unwindHeader));
+ std::terminate();
+#endif
}
=20
=20
However, code such as the following from devel/kyua leads to
other examples of _Unwind_RaiseException looping without
making progress. Note the stdu r1,-368(r1) and the
addi r1,r1,368 stack pointer adjustments and their timing
relative to stack usage (the "red zone" style used for
FreeBSD's powerpc64 ABI):
(gdb) x/64i 0x100a8528-88
0x100a84d0 <utils::fs::mkdir(utils::fs::path const&, int)>: mflr =
r0
0x100a84d4 <utils::fs::mkdir(utils::fs::path const&, int)+4>: =
std r30,-16(r1)
0x100a84d8 <utils::fs::mkdir(utils::fs::path const&, int)+8>: =
std r31,-8(r1)
0x100a84dc <utils::fs::mkdir(utils::fs::path const&, int)+12>: =
std r29,-24(r1)
0x100a84e0 <utils::fs::mkdir(utils::fs::path const&, int)+16>: =
mr r31,r4
0x100a84e4 <utils::fs::mkdir(utils::fs::path const&, int)+20>: =
std r0,16(r1)
0x100a84e8 <utils::fs::mkdir(utils::fs::path const&, int)+24>: =
stdu r1,-368(r1)
0x100a84ec <utils::fs::mkdir(utils::fs::path const&, int)+28>: =
mr r30,r3
0x100a84f0 <utils::fs::mkdir(utils::fs::path const&, int)+32>: =
bl 0x100abab0 <utils::fs::path::c_str() const>
0x100a84f4 <utils::fs::mkdir(utils::fs::path const&, int)+36>: =
nop
0x100a84f8 <utils::fs::mkdir(utils::fs::path const&, int)+40>: =
clrlwi r4,r31,16
0x100a84fc <utils::fs::mkdir(utils::fs::path const&, int)+44>: =
bl 0x10009fc0 <000000af.plt_call.mkdir@@FBSD_1.0>
0x100a8500 <utils::fs::mkdir(utils::fs::path const&, int)+48>: =
ld r2,40(r1)
0x100a8504 <utils::fs::mkdir(utils::fs::path const&, int)+52>: =
cmpwi cr7,r3,-1
0x100a8508 <utils::fs::mkdir(utils::fs::path const&, int)+56>: =
beq cr7,0x100a8528 <utils::fs::mkdir(utils::fs::path const&, =
int)+88>
0x100a850c <utils::fs::mkdir(utils::fs::path const&, int)+60>: =
addi r1,r1,368
0x100a8510 <utils::fs::mkdir(utils::fs::path const&, int)+64>: =
ld r0,16(r1)
0x100a8514 <utils::fs::mkdir(utils::fs::path const&, int)+68>: =
ld r29,-24(r1)
0x100a8518 <utils::fs::mkdir(utils::fs::path const&, int)+72>: =
ld r30,-16(r1)
0x100a851c <utils::fs::mkdir(utils::fs::path const&, int)+76>: =
ld r31,-8(r1)
0x100a8520 <utils::fs::mkdir(utils::fs::path const&, int)+80>: =
mtlr r0
0x100a8524 <utils::fs::mkdir(utils::fs::path const&, int)+84>: =
blr
0x100a8528 <utils::fs::mkdir(utils::fs::path const&, int)+88>: =
bl 0x10009d40 <000000af.plt_call.__error@@FBSD_1.0>
. . . (more not shown here) . . .
This goes along with (darfdump -v -v -F output):
< 1323><0x100a84d0:0x100a8670><mkdir><cie offset 0x0000f100::cie index =
1><fde offset 0x0000f3d8 length: 0x00000034>
<eh aug data len 0x8 bytes 0x00 00 00 00 00 00 f3 c7 >
0x100a84d0: <off cfa=3D00(r1) >=20
0x100a84e0: <off cfa=3D00(r1) > <off r29=3D-24(cfa) > <off =
r30=3D-16(cfa) > <off r31=3D-8(cfa) > <off r65=3Dr0 >=20
0x100a84ec: <off cfa=3D368(r1) > <off r29=3D-24(cfa) > <off =
r30=3D-16(cfa) > <off r31=3D-8(cfa) > <off r65=3D16(cfa) >=20
0x100a8510: <off cfa=3D00(r1) > <off r29=3D-24(cfa) > <off =
r30=3D-16(cfa) > <off r31=3D-8(cfa) > <off r65=3D16(cfa) >=20
0x100a8524: <off cfa=3D00(r1) >=20
0x100a8528: <off cfa=3D368(r1) > <off r29=3D-24(cfa) > <off =
r30=3D-16(cfa) > <off r31=3D-8(cfa) > <off r65=3D16(cfa) >=20
fde section offset 62424 0x0000f3d8 cie offset for fde: 61696 =
0x0000f100
0 DW_CFA_advance_loc 16 (4 * 4)
1 DW_CFA_register r65 =3D r0
4 DW_CFA_offset r30 -16 (2 * -8)
6 DW_CFA_offset r31 -8 (1 * -8)
8 DW_CFA_offset r29 -24 (3 * -8)
10 DW_CFA_advance_loc 12 (3 * 4)
11 DW_CFA_def_cfa_offset 368
14 DW_CFA_offset_extended_sf r65 16 (-2 * -8)
17 DW_CFA_advance_loc 36 (9 * 4)
18 DW_CFA_remember_state
19 DW_CFA_def_cfa_offset 0
21 DW_CFA_advance_loc 20 (5 * 4)
22 DW_CFA_restore_extended r65
24 DW_CFA_restore r31
25 DW_CFA_restore r30
26 DW_CFA_restore r29
27 DW_CFA_advance_loc 4 (1 * 4)
28 DW_CFA_restore_state
29 DW_CFA_nop
30 DW_CFA_nop
The _Unwind_RaiseException's fs ends up reaching and the
holding the following value in my attempted devel/kyua
run for FreeBSD's test suite:
(gdb) print fs
$9 =3D {regs =3D {reg =3D {{loc =3D {reg =3D 0, offset =3D 0, exp =3D =
0x0}, how =3D REG_UNSAVED} <repeats 29 times>, {loc =3D {reg =3D =
18446744073709551592, offset =3D -24,=20
exp =3D 0xffffffffffffffe8 <error: Cannot access memory at =
address 0xffffffffffffffe8>}, how =3D REG_SAVED_OFFSET}, {loc =3D {reg =3D=
18446744073709551600, offset =3D -16,=20
exp =3D 0xfffffffffffffff0 <error: Cannot access memory at =
address 0xfffffffffffffff0>}, how =3D REG_SAVED_OFFSET}, {loc =3D {reg =3D=
18446744073709551608, offset =3D -8,=20
exp =3D 0xfffffffffffffff8 <error: Cannot access memory at =
address 0xfffffffffffffff8>}, how =3D REG_SAVED_OFFSET}, {loc =3D {reg =3D=
0, offset =3D 0, exp =3D 0x0},=20
how =3D REG_UNSAVED} <repeats 33 times>, {loc =3D {reg =3D 16, =
offset =3D 16, exp =3D 0x10 <error: Cannot access memory at address =
0x10>}, how =3D REG_SAVED_OFFSET}, {loc =3D {reg =3D 0, offset =3D 0,=20
exp =3D 0x0}, how =3D REG_UNSAVED} <repeats 80 times>}, prev =3D=
0x0}, cfa_offset =3D 0, cfa_reg =3D 1, cfa_exp =3D 0x0, cfa_how =3D =
CFA_REG_OFFSET,=20
pc =3D 0x100a8528 <utils::fs::mkdir(utils::fs::path const&, int)+88>,=20=
personality =3D @0x810549c40: 0x81053c900 <__gxx_personality_v0(int, =
_Unwind_Action, uint64_t, _Unwind_Exception*, _Unwind_Context*)>, =
data_align =3D -8, code_align =3D 4, retaddr_column =3D 65,=20
fde_encoding =3D 27 '\033', lsda_encoding =3D 20 '\024', saw_z =3D 1 =
'\001', signal_frame =3D 0 '\000', eh_ptr =3D 0x0}
Note the invariant value it is looping with (fs.pc value):
pc =3D 0x100a8528 <utils::fs::mkdir(utils::fs::path const&, int)+88>
But the routine is at 0x00000000100a85ec :
(gdb) bt
#0 _Unwind_RaiseException (exc=3D0x8109582e0) at =
/usr/src/gnu/lib/libgcc/../../../contrib/gcc/unwind.inc:103
#1 0x000000081053d704 in throw_exception (ex=3D0x810958288) at =
/usr/src/contrib/libcxxrt/exception.cc:774
#2 0x00000000100a85ec in utils::fs::mkdir (dir=3D..., =
mode=3Dmode@entry=3D493) at utils/fs/operations.cpp:484
#3 0x00000000100a8694 in utils::fs::mkdir_p (dir=3D..., mode=3D<optimized=
out>) at utils/fs/operations.cpp:502
#4 0x000000001000cbf4 in (anonymous namespace)::safe_main =
(mock_command=3D..., argv=3D0x3fffffffffffdb88, argc=3D4, =
ui=3D0x3fffffffffffd960, this=3D<optimized out>, this=3D<optimized out>) =
at cli/main.cpp:207
#5 cli::main (ui=3Dui@entry=3D0x3fffffffffffd960, argc=3Dargc@entry=3D4, =
argv=3Dargv@entry=3D0x3fffffffffffdb88, mock_command=3D...) at =
cli/main.cpp:280
#6 0x000000001000e9dc in cli::main (argc=3D<optimized out>, =
argv=3D0x3fffffffffffdb88) at cli/main.cpp:353
#7 0x000000001000c570 in main (argc=3D<optimized out>, argv=3D<optimized =
out>) at main.cpp:49
where utils::fs::mkdir:
0x00000000100a85d0 <+256>: bne 0x100a85f0 =
<utils::fs::mkdir(utils::fs::path const&, int)+288>
0x00000000100a85d4 <+260>: addis r9,r2,-1
0x00000000100a85d8 <+264>: addis r10,r2,-1
0x00000000100a85dc <+268>: mr r3,r29
0x00000000100a85e0 <+272>: addi r5,r9,20864
0x00000000100a85e4 <+276>: addi r4,r10,-12216
0x00000000100a85e8 <+280>: bl 0x1000a140 =
<000000af.plt_call.__cxa_throw@@CXXABI_1.3>
=3D> 0x00000000100a85ec <+284>: ld r2,40(r1)
0x00000000100a85f0 <+288>: ld r3,320(r1)
0x00000000100a85f4 <+292>: bl 0x1000a8c0 =
<000000af.plt_call._ZdlPv>
0x00000000100a85f8 <+296>: ld r2,40(r1)
0x00000000100a85fc <+300>: b 0x100a85d4 =
<utils::fs::mkdir(utils::fs::path const&, int)+260>
This is for the devel/kyua code:
void
fs::mkdir(const fs::path& dir, const int mode)
{
if (::mkdir(dir.c_str(), static_cast< mode_t >(mode)) =3D=3D -1) {
const int original_errno =3D errno;
throw fs::system_error(F("Failed to create directory %s") % dir,
original_errno);
}
}
I've not figured out how to make general throwing of C++
exceptions avoid this _Unwind_RaiseException unbounded
looping with a fixed fs.pc value.
But at least now I can use devel/gdb for some of the
investigation's activity.
Other notes:
WITH_LLVM_LIBUNWIND=3D does not even compile for targeting
powerpc64 --so that is not currently a way around the
problem.
WITHOUT_LIB32=3D yse is because, for every post-gcc 4.2.1
that I've tried, the lib32 produced misuses R30 in
crtbeginS code (vs. the ABI for FreeBSD) and 32-bit
code just produces core files from a bad so-called
address that is dereferenced via R30 content.
=3D=3D=3D
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)
From owner-freebsd-hackers@freebsd.org Wed Oct 17 02:25:49 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7813C10E7FAF
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Wed, 17 Oct 2018 02:25:49 +0000 (UTC)
(envelope-from darek@tramada.com)
Received: from mail-io1-xd32.google.com (mail-io1-xd32.google.com
[IPv6:2607:f8b0:4864:20::d32])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(Client CN "smtp.gmail.com",
Issuer "Google Internet Authority G3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 123318BB01
for <freebsd-hackers@freebsd.org>; Wed, 17 Oct 2018 02:25:48 +0000 (UTC)
(envelope-from darek@tramada.com)
Received: by mail-io1-xd32.google.com with SMTP id n18-v6so17877348ioa.9
for <freebsd-hackers@freebsd.org>; Tue, 16 Oct 2018 19:25:48 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=tramada-com.20150623.gappssmtp.com; s=20150623;
h=mime-version:from:date:message-id:subject:to;
bh=9RtI8k4tper0rnmn+kDDESQPQKaCg9Cxzvwnt+x4qzo=;
b=ylPR4WcBWmexa+9UnsBqXv4yPFX0RxN+iOpHFUC30FnJ+Qyn726M/vdUSKX6tEE//X
HrgtVf/aA2EZuBlkBmjT9QwefHsB/XIqLVXs7Te6NfpKPcg7dsF0vIOIkee+oDzuRrdz
pwBbt2H5+Oas4+9RZCL5f7Qh9Sufgl6pSkAE7UC+ugLqaUDSL/xx8O7Pdahf96M/gyLW
BtW19i9jAhaNfr9RHci2GwctpXhqH30G+HbjX64TRGXUCaXsw+Ctsuy0vJvNb86dO4aZ
PReVBnhPxgw+b1D/Ub72n7k+C4Xs8XUGnDsjUVU3ATSBcWfD0RA5WUwBYxBBK79Z3+Bi
IvVw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:mime-version:from:date:message-id:subject:to;
bh=9RtI8k4tper0rnmn+kDDESQPQKaCg9Cxzvwnt+x4qzo=;
b=eieOd3iM/2LUCBsRSjMrZhMnObrQqkvTV84GZaQBRFYc7LylOH2PDao33uX98JT7bW
/i4zFCc20qkOYsLpxYP1PjFTeuIGE5sJl4EBz3VN5yD27fzBNq3pUXH7eDu2eNhGNFBL
aM9kh+feoquUFm57hbaf/1l6Es76PWM3q0I31dQdkOaXmkJ51+/R31F2btrJv9w0AROD
0lxIMlW7OlwjD8sfNn1wbOpPMVwbYQgHThQJ11V8sgIrjENTFaTAtj8deIInmwvcpFJX
z5xLRzIQppehCQLgQzv5YAf+7Cm+Ut/VpMpuMf+vkIDy0ZNEf283YxzzjtHjCvWuFUJ9
dTjQ==
X-Gm-Message-State: ABuFfojHAu11vu8ElmfMckBgVweaVizPxoYQzioAVMbWgPxwTwljljKC
IQbE6VL+yu+vDq4jhKWavEOvLH1G8kwVURSk4R3MQRuloS0=
X-Google-Smtp-Source: ACcGV62EXanDX7ql3IAr9ScFOkeWz5FxcM1OVRmxQhoqeSTl0JgmioyyCPTgTskYPedaduLJ0bDGNQ7Titmm9SR3v2A=
X-Received: by 2002:a6b:c8d2:: with SMTP id
y201-v6mr16029280iof.286.1539743147464;
Tue, 16 Oct 2018 19:25:47 -0700 (PDT)
MIME-Version: 1.0
From: Darek Margas <darek@tramada.com>
Date: Wed, 17 Oct 2018 13:25:31 +1100
Message-ID: <CAG0rGZecYsycwuBzhRBngnBc7TG5Y5913VmdLPPhCbodZPKu8Q@mail.gmail.com>
Subject: High load and MySQL slow without apparent reason
To: freebsd-hackers@freebsd.org
Content-Type: text/plain; charset="UTF-8"
X-Content-Filtered-By: Mailman/MimeDel 2.1.27
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 17 Oct 2018 02:25:49 -0000
Hi Everyone,
I'm trying to refresh my old FreeBSD experience by moving MySQL platform
from Linux onto FreebSD+ZFS.
Before I ask for your help I would like to give you some context.
The machine is Dell server 2x20 cores, Intel IXL NIC, 1TB of RAM and lots
of SAS SSD drives.
The kernel is slightly modified by removing some unused stuff, replacing
ixl driver with latest from Intel website and enabling NUMA.
The whole thing runs number of MySQL daemons packed in jails (bridged
network) with settings optimized for ZFS ARC caching (O_DIRECT, small
buffers, etc).
This is 11.2-RELEASE.
When I tested it first time I found troubles with back pressure on ARC
whilst short in memory leading machine do death. I also found that
disabling ARC compression solved silent death but decided to make some
tunes to keep more memory free for sudden need.
Ran some tests, used it for replication salves, etc.
Here is the thing - how I crashed this machine without understanding what
has happened.
First my tunes. I adjusted v_free_target and v_free_min aiming to 128G and
64G respectively. However, I overlooked fact that this is in pages not in
1k blocks. As result I set:
- 700G max ARC size
- 512G v_free_target
- 256G v_free_min
Obviously this is a nonsense, however, the machine worked calm until ARC
got half of memory. Then shit happened. As I made machine with no swap at
all I have got number of zombies and problems with reclaiming console (say,
open VI which works, then exit and VI stays on console while became zombie).
That was "fixed" by disabling swapping via sysctl. I also noticed 25% of
CPU taken by "system" with nothing popping in top except pagedaemon and zfs
(on arc_reclaim).
I have added 40G of swap, rebooted machine but kept wrong settings.
It was again calm until ARC got half of memory. This is when I found what I
did and fixed v_free stuff to be
- 128G v_free_target
- 64G v_free_min
The machine started managing memory the right way, wiping inactive to
laundry and laundering only when needed. I still observed 25% of
unexplained load from "system" (floating 5-60%) but all seemed OK.
At this point I switched one replica to be master and put production
queries on it.
Summarizing the above - the machine had issues and has not been rebooted
but seemed OK with memory management while having unexplained system load.
Once I switched my SQLs from Linux master to FreeBSD I noticed slow
performance. There is stored proc called every 15 minutes. On old machine
and all others it takes around 30-40s to complete and previous master had
spike in ROW executions to 650kps (one minute sample) while new one got it
up to 350kps and run for nearly 3 minutes.
I started looking deeper and found:
- Made all MySQL settings the same (when possible as some follow platform)
with no improvement
- MySQL reload did not help
- Stopping all replicas running around on the same machine (5 of them) to
release resources made it worse (over 5 minutes to complete call). Starting
replicas made it better again by one minute.
BTW - jail was limited to one NUMA zone and half cores. Not all replicas
had the same NUMA and CPU group.
I copied ZFS content to test machine which is exactly the same and kicked
the same MySQL in same jail and with same settings.
- Test instance ran correctly within similar completion time to old Linux
master
- ARC on test machine was loaded up to 700G so I thought it would be good
enough to compare but machine still had lots of memory
To make it closer I compiled "memory allocator" which simply allocates and
fills memory until killed or system dies.
Run it on test machine first:
- No effect until v_mem_target passed
- Once passed pagedaemon kicked in, memory got wiped and shifted, swap got
full (paging only anyway)
- Load around 20% appeared from system, similar to broken production machine
- Got down to 50G passing v_free_min
- KIlled allocator
- After 1-2s freezing all got back to normal, load from system was gone.
- Swap was in use for some time after but finally got clean (that was only
4G swap on test machine)
- After some time machine is still calm and MySQL fast
Repeated the same on production machine:
- All as above, except:
- after killing allocator machine got frozen for, say, 10-15s
- memory was released but load did not change - neither got much higher
while allocating memory nor lower after.
- Machine remained slow
Finally I rebooted whole machine and now it is fast while building ARC. I
believe it won't have the same issue soon as v_free stuff is set correctly,
however, I need to understand why this MySQL process suffered and whether
it was possible to recover it without reboot. I can imagine it was
something running in a loop or contention on something otherwise unused or
simply another clash in settings triggering something in unusual way but
have no idea where to look to investigate it. Well, it's possible that
there is a bug too.
Before reboot I collected various vmstats, tops, ran ktrace on MySQL and
sysctl to dump settings. Not posting as don't know what would be useful.
Could you please point me in right direction?
Cheers,
Darek
From owner-freebsd-hackers@freebsd.org Wed Oct 17 03:31:56 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 56CBE10C19D1
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Wed, 17 Oct 2018 03:31:56 +0000 (UTC)
(envelope-from allanjude@freebsd.org)
Received: from mx1.scaleengine.net (mx1.scaleengine.net [209.51.186.6])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id E1A158DB3C
for <freebsd-hackers@freebsd.org>; Wed, 17 Oct 2018 03:31:55 +0000 (UTC)
(envelope-from allanjude@freebsd.org)
Received: from [10.1.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28])
(Authenticated sender: allanjude.freebsd@scaleengine.com)
by mx1.scaleengine.net (Postfix) with ESMTPSA id 15D661F01F
for <freebsd-hackers@freebsd.org>; Wed, 17 Oct 2018 03:31:49 +0000 (UTC)
Subject: Re: High load and MySQL slow without apparent reason
To: freebsd-hackers@freebsd.org
References: <CAG0rGZecYsycwuBzhRBngnBc7TG5Y5913VmdLPPhCbodZPKu8Q@mail.gmail.com>
From: Allan Jude <allanjude@freebsd.org>
Openpgp: preference=signencrypt
Autocrypt: addr=allanjude@freebsd.org; prefer-encrypt=mutual; keydata=
xsFNBFVwZcYBEADwrZDH0xe0ZVjc9ORCc6PcBLwS/RTXA6NkvpD6ea02pZ8lPOVgteuuugFc
D34LdDbiWr+479vfrKBh+Y38GL0oZ0/13j10tIlDMHSa5BU0y6ACtnhupFvVlQ57+XaJAb/q
7qkfSiuxVwQ3FY3PL3cl1RrIP5eGHLA9hu4eVbu+FOX/q/XVKz49HaeIaxzo2Q54572VzIo6
C28McX9m65UL5fXMUGJDDLCItLmehZlHsQQ+uBxvODLFpVV2lUgDR/0rDa0B9zHZX8jY8qQ7
ZdCSy7CwClXI054CkXZCaBzgxYh/CotdI8ezmaw7NLs5vWNTxaDEFXaFMQtMVhvqQBpHkfOD
7rjjOmFw00nJL4FuPE5Yut0CPyx8vLjVmNJSt/Y8WxxmhutsqJYFgYfWl/vaWkrFLur/Zcmz
IklwLw35HLsCZytCN5A3rGKdRbQjD6QPXOTJu0JPrJF6t2xFkWAT7oxnSV0ELhl2g+JfMMz2
Z1PDmS3NRnyEdqEm7NoRGXJJ7bgxDbN+9SXTyOletqGNXj/bSrBvhvZ0RQrzdHAPwQUfVSU2
qBhQEi2apSZstgVNMan0GUPqCdbE2zpysg+zT7Yhvf9EUQbzPL4LpdK1llT9fZbrdMzEXvEF
oSvwJFdV3sqKmZc7b+E3PuxK6GTsKqaukd/3Cj8aLHG1T1im1QARAQABzSJBbGxhbiBKdWRl
IDxhbGxhbmp1ZGVAZnJlZWJzZC5vcmc+wsF/BBMBAgApBQJVcGXGAhsjBQkSzAMABwsJCAcD
AgEGFQgCCQoLBBYCAwECHgECF4AACgkQGZU1PhKYC34Muw/+JOKpSfhhysWFYiRXynGRDe07
Z6pVsn7DzrPUMRNZfHu8Uujmmy3p2nx9FelIY9yjd2UKHhug+whM54MiIFs90eCRVa4XEsPR
4FFAm0DAWrrb7qhZFcE/GhHdRWpZ341WAElWf6Puj2devtRjfYbikvj5+1V1QmDbju7cEw5D
mEET44pTuD2VMRJpu2yZZzkM0i+wKFuPxlhqreufA1VNkZXI/rIfkYWK+nkXd9Efw3YdCyCQ
zUgTUCb88ttSqcyhik/li1CDbXBpkzDCKI6I/8fAb7jjOC9LAtrZJrdgONywcVFoyK9ZN7EN
AVA+xvYCmuYhR/3zHWH1g4hAm1v1+gIsufhajhfo8/wY1SetlzPaYkSkVQLqD8T6zZyhf+AN
bC7ci44UsiKGAplB3phAXrtSPUEqM86kbnHg3fSx37kWKUiYNOnx4AC2VXvEiKsOBlpyt3dw
WQbOtOYM+vkfbBwDtoGOOPYAKxc4LOIt9r+J8aD+gTooi9Eo5tvphATf9WkCpl9+aaGbSixB
tUpvQMRnSMqTqq4Z7DeiG6VMRQIjsXDSLJEUqcfhnLFo0Ko/RiaHd5xyAQ4DhQ9QpkyQjjNf
/3f/dYG7JAtoD30txaQ5V8uHrz210/77DRRX+HJjEj6xCxWUGvQgvEZf5XXyxeePvqZ+zQyT
DX61bYw6w6bOwU0EVXBlxgEQAMy7YVnCCLN4oAOBVLZ5nUbVPvpUhsdA94/0/P+uqCIh28Cz
ar56OCX0X19N/nAWecxL4H32zFbIRyDB2V/MEh4p9Qvyu/j4i1r3Ex5GhOT2hnit43Ng46z5
29Es4TijrHJP4/l/rB2VOqMKBS7Cq8zk1cWqaI9XZ59imxDNjtLLPPM+zQ1yE3OAMb475QwN
UgWxTMw8rkA7CEaqeIn4sqpTSD5C7kT1Bh26+rbgJDZ77D6Uv1LaCZZOaW52okW3bFbdozV8
yM2u+xz2Qs8bHz67p+s+BlygryiOyYytpkiK6Iy4N7FTolyj5EIwCuqzfk0SaRHeOKX2ZRjC
qatkgoD/t13PNT38V9tw3qZVOJDS0W6WM8VSg+F+bkM9LgJ8CmKV+Hj0k3pfGfYPOZJ/v18i
+SmZmL/Uw2RghnwDWGAsPCKu4uZR777iw7n9Io6Vfxndw2dcS0e9klvFYoaGS6H2F13Asygr
WBzFNGFQscN4mUW+ZYBzpTOcHkdT7w8WS55BmXYLna+dYer9/HaAuUrONjujukN4SPS1fMJ2
/CS/idAUKyyVVX5vozoNK2JVC1h1zUAVsdnmhEzNPsvBoqcVNfyqBFROEVLIPwq+lQMGNVjH
ekLTKRWf59MEhUC2ztjSKkGmwdg73d6xSXMuq45EgIJV2wPvOgWQonoHH/kxABEBAAHCwWUE
GAECAA8FAlVwZcYCGwwFCRLMAwAACgkQGZU1PhKYC34w5A//YViBtZyDV5O+SJT9FFO3lb9x
Zdxf0trA3ooCt7gdBkdnBM6T5EmjgVZ3KYYyFfwXZVkteuCCycMF/zVw5eE9FL1+zz9gg663
nY9q2F77TZTKXVWOLlOV2bY+xaK94U4ytogOGhh9b4UnQ/Ct3+6aviCF78Go608BXbmF/GVT
7uhddemk7ItxM1gE5Hscx3saxGKlayaOsdPKeGTVJCDEtHDuOc7/+jGh5Zxpk/Hpi+DUt1ot
8e6hPYLIQa4uVx4f1xxxV858PQ7QysSLr9pTV7FAQ18JclCaMc7JWIa3homZQL/MNKOfST0S
2e+msuRwQo7AnnfFKBUtb02KwpA4GhWryhkjUh/kbVc1wmGxaU3DgXYQ5GV5+Zf4kk/wqr/7
KG0dkTz6NLCVLyDlmAzuFhf66DJ3zzz4yIo3pbDYi3HB/BwJXVSKB3Ko0oUo+6/qMrOIS02L
s++QE/z7K12CCcs7WwOjfCYHK7VtE0Sr/PfybBdTbuDncOuAyAIeIKxdI2nmQHzl035hhvQX
s4CSghsP319jAOQiIolCeSbTMD4QWMK8RL/Pe1FI1jC3Nw9s+jq8Dudtbcj2UwAP/STUEbJ9
5rznzuuhPjE0e++EU/RpWmcaIMK/z1zZDMN+ce2v1qzgV936ZhJ3iaVzyqbEE81gDxg3P+IM
kiYh4ZtPB4Q=
Message-ID: <248cd85b-f36e-58ea-873d-8d89846f1c93@freebsd.org>
Date: Tue, 16 Oct 2018 23:31:44 -0400
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101
Thunderbird/52.9.1
MIME-Version: 1.0
In-Reply-To: <CAG0rGZecYsycwuBzhRBngnBc7TG5Y5913VmdLPPhCbodZPKu8Q@mail.gmail.com>
Content-Type: multipart/signed; micalg=pgp-sha1;
protocol="application/pgp-signature";
boundary="NztwEwbgfJvkJ04HVBJP9GJl1lMRV0Htb"
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 17 Oct 2018 03:31:56 -0000
This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--NztwEwbgfJvkJ04HVBJP9GJl1lMRV0Htb
Content-Type: multipart/mixed; boundary="iVJ1ytEsUb9aACkVJa83rt1B8JQKRSmWE";
protected-headers="v1"
From: Allan Jude <allanjude@freebsd.org>
To: freebsd-hackers@freebsd.org
Message-ID: <248cd85b-f36e-58ea-873d-8d89846f1c93@freebsd.org>
Subject: Re: High load and MySQL slow without apparent reason
References: <CAG0rGZecYsycwuBzhRBngnBc7TG5Y5913VmdLPPhCbodZPKu8Q@mail.gmail.com>
In-Reply-To: <CAG0rGZecYsycwuBzhRBngnBc7TG5Y5913VmdLPPhCbodZPKu8Q@mail.gmail.com>
--iVJ1ytEsUb9aACkVJa83rt1B8JQKRSmWE
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
On 2018-10-16 22:25, Darek Margas wrote:
> Hi Everyone,
>=20
> I'm trying to refresh my old FreeBSD experience by moving MySQL platfor=
m
> from Linux onto FreebSD+ZFS.
>=20
> Before I ask for your help I would like to give you some context.
>=20
> The machine is Dell server 2x20 cores, Intel IXL NIC, 1TB of RAM and lo=
ts
> of SAS SSD drives.
> The kernel is slightly modified by removing some unused stuff, replacin=
g
> ixl driver with latest from Intel website and enabling NUMA.
> The whole thing runs number of MySQL daemons packed in jails (bridged
> network) with settings optimized for ZFS ARC caching (O_DIRECT, small
> buffers, etc).
>=20
> This is 11.2-RELEASE.
>=20
> When I tested it first time I found troubles with back pressure on ARC
> whilst short in memory leading machine do death. I also found that
> disabling ARC compression solved silent death but decided to make some
> tunes to keep more memory free for sudden need.
>=20
> Ran some tests, used it for replication salves, etc.
>=20
> Here is the thing - how I crashed this machine without understanding wh=
at
> has happened.
>=20
> First my tunes. I adjusted v_free_target and v_free_min aiming to 128G =
and
> 64G respectively. However, I overlooked fact that this is in pages not =
in
> 1k blocks. As result I set:
>=20
> - 700G max ARC size
> - 512G v_free_target
> - 256G v_free_min
You likely want to tune 'vfs.zfs.arc_free_target' to a value very close
to v_free_target or atleast v_free_min to cause ZFS to give back memory
at that level of memory shortage as well.
>=20
> Obviously this is a nonsense, however, the machine worked calm until AR=
C
> got half of memory. Then shit happened. As I made machine with no swap =
at
> all I have got number of zombies and problems with reclaiming console (=
say,
> open VI which works, then exit and VI stays on console while became zom=
bie).
> That was "fixed" by disabling swapping via sysctl. I also noticed 25% o=
f
> CPU taken by "system" with nothing popping in top except pagedaemon and=
zfs
> (on arc_reclaim).
>=20
> I have added 40G of swap, rebooted machine but kept wrong settings.
>=20
> It was again calm until ARC got half of memory. This is when I found wh=
at I
> did and fixed v_free stuff to be
>=20
> - 128G v_free_target
> - 64G v_free_min
>=20
> The machine started managing memory the right way, wiping inactive to
> laundry and laundering only when needed. I still observed 25% of
> unexplained load from "system" (floating 5-60%) but all seemed OK.
>=20
> At this point I switched one replica to be master and put production
> queries on it.
>=20
> Summarizing the above - the machine had issues and has not been reboote=
d
> but seemed OK with memory management while having unexplained system lo=
ad.
>=20
> Once I switched my SQLs from Linux master to FreeBSD I noticed slow
> performance. There is stored proc called every 15 minutes. On old machi=
ne
> and all others it takes around 30-40s to complete and previous master h=
ad
> spike in ROW executions to 650kps (one minute sample) while new one got=
it
> up to 350kps and run for nearly 3 minutes.
>=20
> I started looking deeper and found:
> - Made all MySQL settings the same (when possible as some follow platfo=
rm)
> with no improvement
> - MySQL reload did not help
> - Stopping all replicas running around on the same machine (5 of them) =
to
> release resources made it worse (over 5 minutes to complete call). Star=
ting
> replicas made it better again by one minute.
>=20
> BTW - jail was limited to one NUMA zone and half cores. Not all replica=
s
> had the same NUMA and CPU group.
>=20
> I copied ZFS content to test machine which is exactly the same and kick=
ed
> the same MySQL in same jail and with same settings.
> - Test instance ran correctly within similar completion time to old Lin=
ux
> master
> - ARC on test machine was loaded up to 700G so I thought it would be go=
od
> enough to compare but machine still had lots of memory
>=20
> To make it closer I compiled "memory allocator" which simply allocates =
and
> fills memory until killed or system dies.
>=20
> Run it on test machine first:
> - No effect until v_mem_target passed
> - Once passed pagedaemon kicked in, memory got wiped and shifted, swap =
got
> full (paging only anyway)
> - Load around 20% appeared from system, similar to broken production ma=
chine
> - Got down to 50G passing v_free_min
> - KIlled allocator
> - After 1-2s freezing all got back to normal, load from system was gone=
=2E
> - Swap was in use for some time after but finally got clean (that was o=
nly
> 4G swap on test machine)
> - After some time machine is still calm and MySQL fast
>=20
> Repeated the same on production machine:
> - All as above, except:
> - after killing allocator machine got frozen for, say, 10-15s
> - memory was released but load did not change - neither got much higher=
> while allocating memory nor lower after.
> - Machine remained slow
>=20
> Finally I rebooted whole machine and now it is fast while building ARC.=
I
> believe it won't have the same issue soon as v_free stuff is set correc=
tly,
> however, I need to understand why this MySQL process suffered and wheth=
er
> it was possible to recover it without reboot. I can imagine it was
> something running in a loop or contention on something otherwise unused=
or
> simply another clash in settings triggering something in unusual way bu=
t
> have no idea where to look to investigate it. Well, it's possible that
> there is a bug too.
>=20
> Before reboot I collected various vmstats, tops, ran ktrace on MySQL an=
d
> sysctl to dump settings. Not posting as don't know what would be useful=
=2E
>=20
> Could you please point me in right direction?
>=20
> Cheers,
> Darek
> _______________________________________________
> freebsd-hackers@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers
> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.o=
rg"
>=20
--=20
Allan Jude
--iVJ1ytEsUb9aACkVJa83rt1B8JQKRSmWE--
--NztwEwbgfJvkJ04HVBJP9GJl1lMRV0Htb
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (MingW32)
iQIcBAEBAgAGBQJbxq0kAAoJEBmVNT4SmAt+PHUQALxpVZdNoGRBNv0nDMw86Fzh
yG/he3JC8eEqlIi+t34sPtTkwINc6F9QgRCSkWAe1DyCLDkVgZ8AHZgSeuiNFQVW
tPP4UL5h33fjYO+BMEZy6hdVkHQZivZ9YjyhDuo/s9NjKTekpjk2V8ngOe2W6KD5
vt7GgN04jp43lCtQ4RR3toCjZzkMOZHgaMJZ34n9AOlb1YflJrAYbJpGt4eVnTPN
0DD+hq9RXkAqzPxBfQsCZLB2vFezAgFrv2GZ0AP0otKkZgpe9ahHPzk5899AvRm9
lBMzlW5Qh0I1cs+yfb3Uhb1VefQIuIuAPjSJQjengOdSdcEZWZQCU37IMGSrurm7
22HS3f65OGrId/dE9si4+nX6Vg/ZcSxNnsxt8bYS52Yq6q01HKWZFXp1728vCNvc
hJ+7QN5AnCBPjFpUMHTRmzXLXulRdM3tIsRkFNn3n1FvCnk+SqnoQ8rOs3lpb2yp
Xs/4z5cMahEggqIu6eukJMqo/cxxOHtIQ/0FL6EXndu6OrWJllDOZtnXtYRIEH5x
0M21Mi44h7WvLnyl/SDEYhzTxvPe+/DwrKTKfF4kWlf8wPJcgGP5uUCD+lcQI0ZM
+NsZIDlsqqHuukHQ4Kho+kjZzo8neMZiCEBZRqWX3R0iM/00M2/uB8qNZtEFRMB6
25VJqU04/4DRiPgwPuCg
=PYHj
-----END PGP SIGNATURE-----
--NztwEwbgfJvkJ04HVBJP9GJl1lMRV0Htb--
From owner-freebsd-hackers@freebsd.org Wed Oct 17 20:25:49 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9020210DE801
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Wed, 17 Oct 2018 20:25:49 +0000 (UTC)
(envelope-from marklmi@yahoo.com)
Received: from sonic301-22.consmr.mail.gq1.yahoo.com
(sonic301-22.consmr.mail.gq1.yahoo.com [98.137.64.148])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 09324905D5
for <freebsd-hackers@freebsd.org>; Wed, 17 Oct 2018 20:25:48 +0000 (UTC)
(envelope-from marklmi@yahoo.com)
X-YMail-OSG: .17zpVUVM1mNnfw6qZnnlw061.Segc8FdcVUgNsvTmB9z6anzaXPsRxzu.xUlWG
XoykGuRCrwNL0IpO90jz27_P3TFPN9lLo7HRFoBWQVpXmz7Yd6WfYkXvtjCexuDIzNKyUW0ucnkO
OBAnbi4xXmqsDGTcLVfe4lc7CW90AyQPvGnVrxJGSVCQxTSBKX2j7723LSPb5dBOcF0bi7vHs7x2
dD6KRV.mrt8tR.m6ki7PMkCWqzyOUIt_wXUUslaUcpE5aGbIJS92M.CXPybvCePGE0646iFSu_ka
giF0dGnTpUD5fw6Hi.6i1kP4rPBx7ZJeRxLpS4cBgdNW3iciKzzDmapspgPWsP0CHn.9jqVN9JKB
64Vv3xWglFOOvTnYEQUgnV_LGHm9SFFWBCEGjI.1Vqjl5tYnzDq816F0FngxTJqbK_Bx9eOHha1e
2tdJ_PIJNj6V3pxEsDQZ12D8E2HDuF7Ix2E8RkTMPHDVOB2R9rMQXWRaw9jxJECUUAuc7Ul27iYG
plKsEYS2L_bLTdVORUnPFDqUfx6XHJ9aXvLW2uJRe76pc6MKxevZnIxACvEiB9tQBQIKsZFQswJo
3bRnmyvPRzzyVwP0he80jLvT4MXIe6Fsg2GTyO4fiYolX07fig1KnIZYq27HhbxlJC_P4k1._FlZ
L0HtMQTHGGr9ggWvww7w7mYe0ecyEGxq4Q8moct1WsQWhrOf4rxGCLIfZWnu75P5MGvLVAvkVNCO
DpSa01D8nx44xnRoMQA1gpMuW5G2_OUbLLRLeKUfKyEeysslm4fSmF__JtdzYKXN9QnJ6YWfgB1C
9Ea1mApJvKPxEqbViksKSxu7ZzTSUlOXBX4sXkgzBIVEGshktXq2v644DcxnIUzfdDGLia6WS_zw
aZhT9rj8cjy48hIFaa7YlSjRQWLT938oSOcdFNuRkv2KgHIudc1o5mXC4Q6Eh.sp6nNxWIzboF2x
CLPkdN8PcPbSfJpl.GfEXSE4Sz3IhNfLGXLQzk6UG9AMiopmJFbsZsDW7oHJSaRwtPZT.uKAjfBy
jgtyAU_d4.ohptH4OsDwIYIdQbRjLTU9gEoGmIIucszgdt0YlN6oYYz6en_sMqVzPbMeowXQX9iV
LMhA3zBqH9RSen_4-
Received: from sonic.gate.mail.ne1.yahoo.com by
sonic301.consmr.mail.gq1.yahoo.com with HTTP; Wed, 17 Oct 2018 20:25:42 +0000
Received: from c-76-115-7-162.hsd1.or.comcast.net (EHLO [192.168.1.25])
([76.115.7.162])
by smtp428.mail.gq1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID
80e922adb91863636bdf56988756cc64;
Wed, 17 Oct 2018 20:25:40 +0000 (UTC)
From: Mark Millard <marklmi@yahoo.com>
Content-Type: text/plain;
charset=us-ascii
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\))
Subject: What is incomplete for /lib/libgcc_s.so-based C++ exception handling
(where WITH_LLVM_LIBUNWIND= and /usr/local/lib/gcc*/libgcc_s.so are
not used)
Message-Id: <0379371E-0541-42DD-93EF-BEE2E9DE3FBC@yahoo.com>
Date: Wed, 17 Oct 2018 13:25:39 -0700
Cc: FreeBSD PowerPC ML <freebsd-ppc@freebsd.org>
To: FreeBSD Toolchain <freebsd-toolchain@freebsd.org>,
FreeBSD <freebsd-hackers@freebsd.org>, freebsd-arch@freebsd.org
X-Mailer: Apple Mail (2.3445.9.1)
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 17 Oct 2018 20:25:49 -0000
[This summarizes other results without the code
and debugger evidence and such from my recent
explorations. It should be much easier to
follow than my exploration reports.]
Documents like DWARF5.pdf document the "row" vs. Location
information for Call Frame Information as (also used
for .eh_frame like materials for C++ exception handling):
(CFA: Cannonical Frame Address)
QUOTE ("Structure of Call Frame Information")
LOC CFA R0 R1...RN
L0
L1
...
LN
END QUOTE
Note that the CFA is conceptually one of the
registers in each row, even though it is not a
machine register but a way to calculate the
conceptual register's value from machine
registers.
The information for the machine registers are
typically based on the earlier CFA value (from
the same row!). Absent a correct CFA cell in
a row, most potential use of that row is likely
messed up.
One way CFA is found is by adding an offset to
the value of a machine register for the range
in question, Ln up to L(n+1) [or based on the
end of the overall range for the last Ln]. I
will use that for illustration because there
are examples of this in my testing.
/lib/libgcc_s.so.1 does not implement this
fully for some DW_CFA_* operations:
QUOTE (note the "every register" reference, so including CFA)
DW_CFA_remember_state
The DW_CFA_remember_state instruction takes no operands. The required =
action is to push the set of rules for every register onto an implicit =
stack.
DW_CFA_restore_state
The DW_CFA_restore_state instruction takes no operands. The required =
action is to pop the set of rules off the implicit stack and place them =
in the current row.
END QUOTE
In other words: push and pop a complete row,
not just machine registers information from
the row.
For example, the the "cfa_offset" for computing the CFA
value from from a register is not saved and restored.
Nor is which register the offset is based on. (This
can vary, though not in my examples.) In general the
CFA cell is not saved and restored, what ever its
contents.
So any compiler that produces code depending on
DW_CFA_remember_state and DW_CFA_restore_state
for .eh_frame like material ends up with C++
exception handling messed up when the
DW_CFA_restore_state should change the CFA to a=20
non-default one (from the prior
DW_CFA_remember_state).
This prevents reliable use of throwing C++ exceptions
when building via the likes of devel/powerpc64-gcc
or lang/gcc8 ( when not using
-Wl,-rpath=3D-Wl,-rpath=3D/usr/local/lib/gcc8 so that
/lib/libgcc_s.so.1 ends up being used). One result
can be _Unwind_RaiseException looping looking at
the same frame over and over instead of progressing
to the next frame. For example, this happens via
cfa_offset 0 being used. devel/powerpc64-gcc -O2
code tends to get that.
Notes:
For powerpc64, clang++ tends to use another register
(%r31) with the old value (of %r1, the stack pointer)
instead of involving the
DW_CFA_remember_state/DW_CFA_restore_state pair
based on just %r1. (clang has other problems
relative to sue for buildworld buildkernel.)
Code generation styles matter for if the incomplete
coverage by /lib/libgcc_s.so will be visible or not.
At this stage, WITH_LLVM_LIBUNWIND=3D builds
targeting powerpc64 do not even compile/assemble
the relevant code, apparently both because of
darwin specific assembler code and FreeBSD's build
not using the C-preprocessor on the .S file as
required. (There could be more to getting it
working.)
I do not know about other architecture/compiler
(or toolchain) combinations that may not yet be
able to use WITH_LLVM_LIBUNWIND=3D . But I'd
expect a potentially similar status from some.
A range of modern /usr/local/lib/gcc*/libgcc_s.so
do implement DW_CFA_remember_state/DW_CFA_restore_state
operations and they are put to use. So using the likes
of -Wl,-rpath=3D/usr/local/lib/gcc8 works for g++8 C++
exception handling (but is problematical for buildworld
buildkernel).
I made a similar exploration of the issue in around
early 2016 and got basically the same results, not that
I remembered much. But I now have a small source code
example that shows the cfa_offset issue for the likes
of devel/powerpc64-gcc output.
The standard source for throw_exception in
/lib/libgcc_s.so produces the cfa_offset problem
when devel/powerpc64-gcc is used to buildworld.
This turns all thrown C++ exceptions in to
unbounded looping in _Unwind_RaiseException for
that kind of context.
=3D=3D=3D
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)
From owner-freebsd-hackers@freebsd.org Thu Oct 18 00:25:37 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8D9E510E2ADA
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Thu, 18 Oct 2018 00:25:37 +0000 (UTC)
(envelope-from yuripv@yuripv.net)
Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com
[66.111.4.27])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 00B4296CA0
for <freebsd-hackers@freebsd.org>; Thu, 18 Oct 2018 00:25:36 +0000 (UTC)
(envelope-from yuripv@yuripv.net)
Received: from compute5.internal (compute5.nyi.internal [10.202.2.45])
by mailout.nyi.internal (Postfix) with ESMTP id 6B0142222C
for <freebsd-hackers@freebsd.org>; Wed, 17 Oct 2018 20:25:36 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163])
by compute5.internal (MEProxy); Wed, 17 Oct 2018 20:25:36 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yuripv.net; h=to
:from:subject:message-id:date:mime-version:content-type; s=fm3;
bh=L5CTh8yOwHUz9huACx1hIwRK/ZxMMPOsdalhFhNsiyU=; b=pT/8san77l55
WrcD6c7GuuvkdgZ/1XQKq4dVn0lIcHgQ/2BSLVj/7pRegfLR4lQZNJota2q4LLMx
CA3/rflmmqg53N25/c5TStXzJ+NLfkc37VxNVsWWjMOUEMtetTyygvEL2keJBnm+
f1C+yoFdtFVaQeSXf3vB+TdQO1MVkSFESL18DUKr7h91S19dTbwbtoxqN9YHWGHn
i5AdC6XCy6mvFAj4AG7GTTmYPDjMBIHNTRo4Y4otqPzTllKR37rYuEFKwN+on9hQ
5ow3XQWqcFiNxOiBM0WQK2yZcxPtCBDgA0k5L/H0tc2J17dgOkH9QkYXZ2vcMsMU
3yKHUE2TjA==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=content-type:date:from:message-id
:mime-version:subject:to:x-me-proxy:x-me-proxy:x-me-sender
:x-me-sender:x-sasl-enc; s=fm1; bh=L5CTh8yOwHUz9huACx1hIwRK/ZxMM
POsdalhFhNsiyU=; b=P/G015U6u4JO2fsJIXmdc+7meydebXpJutBpFw/OyC5YY
yqoYnMoZfV4f0odnY9oyJnTJiPiq+Ngujd3ZbLHCSyS5QefiwWi8CU5g+Cgfs2sB
8DEVnww6UZMdlR8hdLVVj3pZUb96K2SNu8fMg2Y3IxN8auRfT3ArvAFDBckAYy/e
3mLK9CKiEn0GwNjaNH1xvKkF/WK7wisQlIGyENWg8wRzYwBBlVYY7CZ9qAt/fvqr
ROiahlMJW1R4VhQXoqoyBz4JGVZQz8K3oyfYF5ZfgeoKEGGW97SX3thkeyFs/Jih
qPVnNctHZ+IpbfRb/1WDSrtkSiIXr3BwDaNw2xKXQ==
X-ME-Sender: <xms:ANPHWzK30ucCLg6tRL-36fMDYTB_BjXfnRK3wJ-tTW3fUqqCl3rQqQ>
X-ME-Proxy: <xmx:ANPHWzGvMRVd_kt1sj2kr3-NIfhAhbx93EocZREYvt7hgdrMyBTodw>
<xmx:ANPHWwjR4iwDab2u2XSdci2YvXnRGiJqk0GW1k01-kBdjFp8OTeitw>
<xmx:ANPHWzc_LyXlOgKwuMBdvqAJYeOfDC4DatE4NzwIiX76zc6OajJJlw>
<xmx:ANPHW-2jd2CCltb1sZz5lJsa_Om3oBTB44jurKyK-LVEwP5fta9q7g>
<xmx:ANPHW7usRZWN7qlctYHA0Qa-9NLoyoXT-Bem-F610bjrHn-_nG73-A>
<xmx:ANPHW1uikEwEAjkFhbiUS6sABhQnpGQP2sop96YjygstnO6s1kw76g>
Received: from [192.168.1.2] (unknown [85.173.27.113])
by mail.messagingengine.com (Postfix) with ESMTPA id D04C4102DE
for <freebsd-hackers@freebsd.org>; Wed, 17 Oct 2018 20:25:34 -0400 (EDT)
To: freebsd-hackers <freebsd-hackers@freebsd.org>
From: Yuri Pankov <yuripv@yuripv.net>
Subject: embed endian info in locale data files magic (PR231965)
Openpgp: preference=signencrypt
Autocrypt: addr=yuripv@yuripv.net; keydata=
xsBNBFu8u6IBCADB11gP0QwnorrHjqAtKLHKHNHskhy0s7jqJKfx0YqXgVBKGLJ9/mjLAz0F
CBNvemHSDDTs0mEZ9cBKKi6cmsav6+UQgr//yai6hvXLBJqKchSFO4MhmdvBtsGFq1yKz5Zi
uhjmimKyIpgBgvMdbgGbGq6cnSB2uEPmZuJr419SVRODOkXukU+F5WHgaHzDdHAIu1asCt2B
+6msxqIqlFWcXyZyTGicTGGvC/PFIsVRUtD1dIJANTC876g7DTb7LZXWiWwJpSJ4GKMXMHVX
Ct9BoQ4i3nhKbOxb6Io1wsy+NFyWsTJ9KYrxKKPJP3oG8BWb/cqlFqnE4eNSsiq2q7krABEB
AAHNH1l1cmkgUGFua292IDx5dXJpcHZAeXVyaXB2Lm5ldD7CwJcEEwEIAEECGwMFCQWjmoAF
CwkIBwMFFQoJCAsFFgMCAQACHgECF4AWIQT4arc+w94tPi0v/3CTi+B/sSrhbAUCW708wAIZ
AQAKCRCTi+B/sSrhbPxBB/961alcU091O+yKT5/oReHVc/PX0Tz4sW3V44AcgLfYlrZavCro
EFz90qmCrl0xqEwuAKcC4bjmL8SjPWAhSN6IH9nxdw+HeZnAPiHm/q679Bu47+nHBl3qD/9p
+t1PkKeKZfaWToFMt1nq06ytSu6VLMCwLdlDNe6DReX0ex/afEqKsuaIZSKL4UYjRwklp8PU
Uf98QkrfapyHB67hQMzfI4tPeJaYyv0cTgfq3kUWJx1V6Xi0b6Zxj4ZrB2TXvaMO5g7yhU9E
E3WWAvoe4FgB3a7dHe8atnHhq5+Cuvm6+LD4Jh7jvMAE5UMN+xxQpnGpNghHjaCy4vXrLRBZ
nhRYzsBNBFu8u6IBCADKih3Q933rDNj4ZA8FhBQ2RlmBgvwOLcDPIL3h0V7h38y3+HisgFSc
XACDsdrTlYZ1bRXkD9FHENynBcv0l/3uGJDk8jaGIDE0TP8OQBRp+IaU9/BHnAqrKxTJGIol
Dahy2m+yx2yhdc6B4ujWMDqCF1rWOD+ymOWw+VLllOkrHcZa5PJtX9UOGbApZl8ZTM8El4CA
NN8F1bg9MWzUi+8LYoGWGc+BwsFS1OUB1c4SPgMu5fD4Wfsr9yRl06fdpEA2YT7B/j5/5RSC
0sE2Zs/tmJ/JRflHJ12ycj59ma2xQMfEJF40hZDpMFQmZvbVqgEg3ocQcltjbxlIKZ/mjC4z
ABEBAAHCwHwEGAEKACYWIQT4arc+w94tPi0v/3CTi+B/sSrhbAUCW7y7ogIbDAUJBaOagAAK
CRCTi+B/sSrhbIDcCACqAZMcoxUBLZa40a5b24j5i1jplvCYYb3h+Q5lt5+BFJ87kCb4dJuU
D3kh2i29BrxWQWa9WNue9ozxeYkbkfXubQYXexVolRsnh64OdGsE8KvorBFBB3zdK/GRt2Jy
+jsnTfUWuQllbzMP0MfhCDMk1Mo8WvDH2/cOEP/yLKf20a+cd6nLs7bidjmGXo9pyuBKAtV6
Kv+VRu54AL+A/UBYu/eB3Dtvzcnut+1Zq6KaP++kUwPwINLIk04OBDwN0zRNTiqMAFYYyz2v
ZHBB6E1th/l//ZC5b9Dk0ZpFI1bYdL9ymnrZe1MqbGPnDCToQxu00T/pZCm6Z92YrZQYuNwl
Message-ID: <f081ab81-56d0-49cc-92b0-fadc09618d31@yuripv.net>
Date: Thu, 18 Oct 2018 03:25:26 +0300
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101
Thunderbird/60.2.1
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
protocol="application/pgp-signature";
boundary="RTL9UyG6YiKgof5xLKUFl5STeAOQNr1v0"
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 18 Oct 2018 00:25:37 -0000
This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--RTL9UyG6YiKgof5xLKUFl5STeAOQNr1v0
Content-Type: multipart/mixed; boundary="5oxIDsZ06G3YJqyqE6v4e2eYEjCaO8uTB";
protected-headers="v1"
From: Yuri Pankov <yuripv@yuripv.net>
To: freebsd-hackers <freebsd-hackers@freebsd.org>
Message-ID: <f081ab81-56d0-49cc-92b0-fadc09618d31@yuripv.net>
Subject: embed endian info in locale data files magic (PR231965)
--5oxIDsZ06G3YJqyqE6v4e2eYEjCaO8uTB
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D231965 ([PowerPC64]
Cross compiling powerpc64 from amd64 results in nonfunctional locale
installations), describes the issue with locale data built on LE system
(amd64) when used on BE system (powerpc).
Fix introduced in rS308170 depends on the fact that locale data is built
on LE system, and will likely fail when it's built natively on mips
(please correct me if I'm wrong). More so, we shouldn't be hardcoding
the conversion in libc, and I see 2 options here:
1. fix localedef to output data in target's system endian
2. embed the endian info in locale data files (updating magic signature)
and ehhance the previous fix with runtime selection of needed
conversion
I have put the change for #2 together at
https://reviews.freebsd.org/D17603 (more a PoC at the moment than a real
review), and wondering if it looks sane enough or if there's anything
obvious I'm missing.
I have briefly tested the libc and locale files (LC_CTYPE and
LC_COLLATE) built on amd64 on a powerpc system, and it seems to work.
TIA
--5oxIDsZ06G3YJqyqE6v4e2eYEjCaO8uTB--
--RTL9UyG6YiKgof5xLKUFl5STeAOQNr1v0
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCAAdFiEE+Gq3PsPeLT4tL/9wk4vgf7Eq4WwFAlvH0vYACgkQk4vgf7Eq
4Wy+XQf8DvLLWf5bYetZTJiTR7vlNSiZQLxjbF72CDcq60puz7mmCmGLlqUCYsSZ
3uPpU0G90LpQ8hY95Y1RvCX8MXet7YheiOxv550jvCkER9khiGCHILjkGVG+P2ae
DJLuB/aKI/+q4WyKRC7IF+y3Yk/WaLymkEIy984buR/0H9fTpbVtlB8TPQQz/qm2
olNXa+SP2xTFJPP0lEd+V92k8kqLk0fS+uLwOUmlEjUoqrr20uQrGtRaPhXGVvmg
5WfVF8JU63vHUL2AC5lpKwmW7CzOKJj2p0dEFkLPtHVyNE6IVYxc/jPRg8GTef0B
FgLBI0xLxu2Ui7dLOyRue8MwxFPknA==
=sll+
-----END PGP SIGNATURE-----
--RTL9UyG6YiKgof5xLKUFl5STeAOQNr1v0--
From owner-freebsd-hackers@freebsd.org Thu Oct 18 04:21:18 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id E5C1910C895D
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Thu, 18 Oct 2018 04:21:17 +0000 (UTC)
(envelope-from wlosh@bsdimp.com)
Received: from mail-ua1-x930.google.com (mail-ua1-x930.google.com
[IPv6:2607:f8b0:4864:20::930])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(Client CN "smtp.gmail.com",
Issuer "Google Internet Authority G3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 6DCEC791B8
for <freebsd-hackers@freebsd.org>; Thu, 18 Oct 2018 04:21:17 +0000 (UTC)
(envelope-from wlosh@bsdimp.com)
Received: by mail-ua1-x930.google.com with SMTP id y5so5651133uap.4
for <freebsd-hackers@freebsd.org>; Wed, 17 Oct 2018 21:21:17 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=bsdimp-com.20150623.gappssmtp.com; s=20150623;
h=mime-version:references:in-reply-to:from:date:message-id:subject:to
:cc; bh=m6lIHcCVwLEcHhQYpSqKGNLp7+AYA1ODnWO8qmVRyYc=;
b=NiJPTATEqq21MIyO5qRlzPl6jnDp5Rk/SPHUtIniv4bZQrs2Ccz5SZxX2sb/ewqHN/
UnDqa4fWhMQl5IBv4+yg2KNXjF8zs4lglRQZcc9P+rgrJBGy9XBWpiWym+jylSU3WJ6W
Q9bDNWIIxUpe7ENqgwpgGtkcIS9zW8+fTK1YOYRUnCmzoZ73E9V2vfHCsHQkj9/0LrLD
LTvhrExYeNgaA4NKnExSmN5d/4JOYTiBvYS41x/uA0Lc+cNJ9TV2j5+r+RteBh2Ad8Oq
HAwFioWHZ11LvFnUMF1vwHQCxXx1cmYHDgDD9uFG1VmPfOWEfSC15SyYgQZiq6oftPJu
z9PA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:mime-version:references:in-reply-to:from:date
:message-id:subject:to:cc;
bh=m6lIHcCVwLEcHhQYpSqKGNLp7+AYA1ODnWO8qmVRyYc=;
b=GgGxZ46tXrOmD7RYThLMk2kZ3/wZ+ALJd4Iei1imW/YdfVIjbfWcYcoQ4h1QgKtjR9
KQSufSC1Dw+2BzYr5SLQlNWkdPbXGJrxqw90T1ILJ90Uqja2+s4+3o3kdr2s3BiVch9y
0qZTGh1wj0o209ERrGQfTWgtTUJdo7zTxAI0iSG3FUIYp1FdKWH8X0vHy7PSqzrftgXX
V0pGEOYOF9ZxXlO63wrhlU8JtLN/l2T2noUgXD9zg4IIEHfofFTthW1tM58oNxnJEoks
DDHzEDMMlHepgSuK2P9dF4rt1AP/uYFEs1lsNaXevDoEQnAiK0IDdfIRtZx15WHG5OuO
ucaw==
X-Gm-Message-State: ABuFfogVkyI2GA2Ak0jWwVqEYQETk6WuJNLnTnuk4ctlMa33bJYsbpqf
92F03ZY2SJbx4ENF5jpmHxXFxhCicQt8JS1x8FGVqA==
X-Google-Smtp-Source: ACcGV62HEYROLEeN4DuZOqzz3ExSiZoAmcWYlCT/AwLxsMz8TZ/rt0ojyb0WlDIXkoVHkLwYscTYClDgOoBeyoyBU7o=
X-Received: by 2002:ab0:a97:: with SMTP id d23mr12588302uak.39.1539836476526;
Wed, 17 Oct 2018 21:21:16 -0700 (PDT)
MIME-Version: 1.0
References: <f081ab81-56d0-49cc-92b0-fadc09618d31@yuripv.net>
<abaa1192-defc-973e-8d0f-b8e5a2437c0d@yuripv.net>
In-Reply-To: <abaa1192-defc-973e-8d0f-b8e5a2437c0d@yuripv.net>
From: Warner Losh <imp@bsdimp.com>
Date: Wed, 17 Oct 2018 22:21:05 -0600
Message-ID: <CANCZdfpuUhJTxeUrEn1jrphSVaPqWw86Xm-e-rdX_mwC_kJO9w@mail.gmail.com>
Subject: Re: embed endian info in locale data files magic (PR231965)
To: Yuri Pankov <yuripv@yuripv.net>
Cc: "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org>
Content-Type: text/plain; charset="UTF-8"
X-Content-Filtered-By: Mailman/MimeDel 2.1.27
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 18 Oct 2018 04:21:18 -0000
On Wed, Oct 17, 2018 at 10:18 PM Yuri Pankov <yuripv@yuripv.net> wrote:
> Yuri Pankov wrote:
> > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=231965 ([PowerPC64]
> > Cross compiling powerpc64 from amd64 results in nonfunctional locale
> > installations), describes the issue with locale data built on LE system
> > (amd64) when used on BE system (powerpc).
> >
> > Fix introduced in rS308170 depends on the fact that locale data is built
> > on LE system, and will likely fail when it's built natively on mips
> > (please correct me if I'm wrong). More so, we shouldn't be hardcoding
> > the conversion in libc, and I see 2 options here:
> >
> > 1. fix localedef to output data in target's system endian
> > 2. embed the endian info in locale data files (updating magic signature)
> > and ehhance the previous fix with runtime selection of needed
> > conversion
>
> Thinking more about this, or:
>
> 3. Always store the data in LE (or BE, doesn't matter), and
> appropriately convert while reading. This will likely require least
> change.
>
I like this.
Warner
From owner-freebsd-hackers@freebsd.org Thu Oct 18 01:59:11 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 044B110E6B11
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Thu, 18 Oct 2018 01:59:11 +0000 (UTC)
(envelope-from yuripv@yuripv.net)
Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com
[66.111.4.27])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 979E8731A1
for <freebsd-hackers@freebsd.org>; Thu, 18 Oct 2018 01:59:10 +0000 (UTC)
(envelope-from yuripv@yuripv.net)
Received: from compute5.internal (compute5.nyi.internal [10.202.2.45])
by mailout.nyi.internal (Postfix) with ESMTP id 1D00222182
for <freebsd-hackers@freebsd.org>; Wed, 17 Oct 2018 21:59:10 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163])
by compute5.internal (MEProxy); Wed, 17 Oct 2018 21:59:10 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yuripv.net; h=
subject:from:to:references:message-id:date:mime-version
:in-reply-to:content-type; s=fm3; bh=gsh5Ufv+qkLqfJSOrN4/HCOmqd5
pjHFWM3YZznMFqtc=; b=eaxi0oqhOzOn+joYie4gNbEQWggg2bgZ6nKjjro9Nu9
sL8OlrmdklORsIEsrZCUjs+PTdIAXx4MXEZAujHONFYglLCiAEakKKMMU4PIa5Zx
+iAz9L8V8eqshpZVw3L+cy0kbT9a5xmswnCKfIHz8sXpbzSB4g4qAavzcvGQodGL
CnaK0XMxUvvO/8/Jao7Gr0TKVkQrDLElmexyqt1by6igH6JLtItBSN6RpYvE0UDn
ytbFipw52atrhxWkQV5uHX+M3tKplWpd3e9QorRpt2jpPC8c+b8NkIOLDwn58g6Z
wkZh1RFq/t4qdo9Ncfvh3qvH0Gbs/5GQmskJL596frg==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=content-type:date:from:in-reply-to
:message-id:mime-version:references:subject:to:x-me-proxy
:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; bh=gsh5Uf
v+qkLqfJSOrN4/HCOmqd5pjHFWM3YZznMFqtc=; b=H48FLgYFGOIbEtbvW6n+Ab
Zm+LY8vaUFvNdPFIUrsEvfm9PE2Kx7iKAwf2FrHSIkeAcl3Hsl7vaphdpRE3b1ca
fU+j1xLPwfmTaa4XZNOeOPtcrEH0uyEZtyTXbSewUyMDrsS71lzo4JcfCWnSJCAx
u4Ea6kY0H6YrKezV+B201OOgfKO1LUF8bJKs2TQmghc65CyQ2NpbFsF1KgTnFrLD
pJIJg3cP0bGtZ7Qpw9zScyw/1rpwjrs/4IFCqsKKH3O5PKVaYBJY/RYcBuvoJFO4
lGgEt/IYqNzWnMFV+uLxzVdHK1M7ZqhStcLxsnf6/wJQEpbz9abguKCHw4RdBiIA
==
X-ME-Sender: <xms:7ejHW72bal2hBn3ssIzXhMCgomfuqIWLbueX4wv0Uq1izvVfDAArIQ>
X-ME-Proxy: <xmx:7ejHW_pa4bKA1VGQwAOakAKO9S_iK96Xk3K2nTB_XY5WsG7_SFdpsA>
<xmx:7ejHW7L0CQ-UYDFFvQWF5ap5E73mff05rwk8tO4SZ-uVZU-ColjdIw>
<xmx:7ejHW5300muOWri6VNP7EH00EEKRhPIs716mW_bN8APx2noeqjfxvg>
<xmx:7ejHW6XnEiTsUUNlev-wCm-IhhKd6n3fCUfv2aLqyp9pFo3BRRDfSg>
<xmx:7ejHW_V6b5eXMxFZgYX-kNf3tfp2DDebYwLke0OCtYiejv4RtmEl2A>
<xmx:7ujHW7WOhJ_5OKRnIN_GUfn92qnG7j2DIM2ke5RZbC_8eapa-Mcf7A>
Received: from [192.168.1.2] (unknown [85.173.27.113])
by mail.messagingengine.com (Postfix) with ESMTPA id 7E2B9102E9
for <freebsd-hackers@freebsd.org>; Wed, 17 Oct 2018 21:59:08 -0400 (EDT)
Subject: Re: embed endian info in locale data files magic (PR231965)
From: Yuri Pankov <yuripv@yuripv.net>
To: freebsd-hackers <freebsd-hackers@freebsd.org>
References: <f081ab81-56d0-49cc-92b0-fadc09618d31@yuripv.net>
Openpgp: preference=signencrypt
Autocrypt: addr=yuripv@yuripv.net; keydata=
xsBNBFu8u6IBCADB11gP0QwnorrHjqAtKLHKHNHskhy0s7jqJKfx0YqXgVBKGLJ9/mjLAz0F
CBNvemHSDDTs0mEZ9cBKKi6cmsav6+UQgr//yai6hvXLBJqKchSFO4MhmdvBtsGFq1yKz5Zi
uhjmimKyIpgBgvMdbgGbGq6cnSB2uEPmZuJr419SVRODOkXukU+F5WHgaHzDdHAIu1asCt2B
+6msxqIqlFWcXyZyTGicTGGvC/PFIsVRUtD1dIJANTC876g7DTb7LZXWiWwJpSJ4GKMXMHVX
Ct9BoQ4i3nhKbOxb6Io1wsy+NFyWsTJ9KYrxKKPJP3oG8BWb/cqlFqnE4eNSsiq2q7krABEB
AAHNH1l1cmkgUGFua292IDx5dXJpcHZAeXVyaXB2Lm5ldD7CwJcEEwEIAEECGwMFCQWjmoAF
CwkIBwMFFQoJCAsFFgMCAQACHgECF4AWIQT4arc+w94tPi0v/3CTi+B/sSrhbAUCW708wAIZ
AQAKCRCTi+B/sSrhbPxBB/961alcU091O+yKT5/oReHVc/PX0Tz4sW3V44AcgLfYlrZavCro
EFz90qmCrl0xqEwuAKcC4bjmL8SjPWAhSN6IH9nxdw+HeZnAPiHm/q679Bu47+nHBl3qD/9p
+t1PkKeKZfaWToFMt1nq06ytSu6VLMCwLdlDNe6DReX0ex/afEqKsuaIZSKL4UYjRwklp8PU
Uf98QkrfapyHB67hQMzfI4tPeJaYyv0cTgfq3kUWJx1V6Xi0b6Zxj4ZrB2TXvaMO5g7yhU9E
E3WWAvoe4FgB3a7dHe8atnHhq5+Cuvm6+LD4Jh7jvMAE5UMN+xxQpnGpNghHjaCy4vXrLRBZ
nhRYzsBNBFu8u6IBCADKih3Q933rDNj4ZA8FhBQ2RlmBgvwOLcDPIL3h0V7h38y3+HisgFSc
XACDsdrTlYZ1bRXkD9FHENynBcv0l/3uGJDk8jaGIDE0TP8OQBRp+IaU9/BHnAqrKxTJGIol
Dahy2m+yx2yhdc6B4ujWMDqCF1rWOD+ymOWw+VLllOkrHcZa5PJtX9UOGbApZl8ZTM8El4CA
NN8F1bg9MWzUi+8LYoGWGc+BwsFS1OUB1c4SPgMu5fD4Wfsr9yRl06fdpEA2YT7B/j5/5RSC
0sE2Zs/tmJ/JRflHJ12ycj59ma2xQMfEJF40hZDpMFQmZvbVqgEg3ocQcltjbxlIKZ/mjC4z
ABEBAAHCwHwEGAEKACYWIQT4arc+w94tPi0v/3CTi+B/sSrhbAUCW7y7ogIbDAUJBaOagAAK
CRCTi+B/sSrhbIDcCACqAZMcoxUBLZa40a5b24j5i1jplvCYYb3h+Q5lt5+BFJ87kCb4dJuU
D3kh2i29BrxWQWa9WNue9ozxeYkbkfXubQYXexVolRsnh64OdGsE8KvorBFBB3zdK/GRt2Jy
+jsnTfUWuQllbzMP0MfhCDMk1Mo8WvDH2/cOEP/yLKf20a+cd6nLs7bidjmGXo9pyuBKAtV6
Kv+VRu54AL+A/UBYu/eB3Dtvzcnut+1Zq6KaP++kUwPwINLIk04OBDwN0zRNTiqMAFYYyz2v
ZHBB6E1th/l//ZC5b9Dk0ZpFI1bYdL9ymnrZe1MqbGPnDCToQxu00T/pZCm6Z92YrZQYuNwl
Message-ID: <abaa1192-defc-973e-8d0f-b8e5a2437c0d@yuripv.net>
Date: Thu, 18 Oct 2018 04:58:58 +0300
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101
Thunderbird/60.2.1
MIME-Version: 1.0
In-Reply-To: <f081ab81-56d0-49cc-92b0-fadc09618d31@yuripv.net>
Content-Type: multipart/signed; micalg=pgp-sha256;
protocol="application/pgp-signature";
boundary="1PLmTiomysntfNknvXhyS0R0pJqI4h7pF"
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 18 Oct 2018 01:59:11 -0000
This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--1PLmTiomysntfNknvXhyS0R0pJqI4h7pF
Content-Type: multipart/mixed; boundary="V7dAMGwHWZv6qJlfJX3KibuOQEFFSHKLP";
protected-headers="v1"
From: Yuri Pankov <yuripv@yuripv.net>
To: freebsd-hackers <freebsd-hackers@freebsd.org>
Message-ID: <abaa1192-defc-973e-8d0f-b8e5a2437c0d@yuripv.net>
Subject: Re: embed endian info in locale data files magic (PR231965)
References: <f081ab81-56d0-49cc-92b0-fadc09618d31@yuripv.net>
In-Reply-To: <f081ab81-56d0-49cc-92b0-fadc09618d31@yuripv.net>
--V7dAMGwHWZv6qJlfJX3KibuOQEFFSHKLP
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
Yuri Pankov wrote:
> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D231965 ([PowerPC64]=
> Cross compiling powerpc64 from amd64 results in nonfunctional locale
> installations), describes the issue with locale data built on LE system=
> (amd64) when used on BE system (powerpc).
>=20
> Fix introduced in rS308170 depends on the fact that locale data is buil=
t
> on LE system, and will likely fail when it's built natively on mips
> (please correct me if I'm wrong). More so, we shouldn't be hardcoding
> the conversion in libc, and I see 2 options here:
>=20
> 1. fix localedef to output data in target's system endian
> 2. embed the endian info in locale data files (updating magic signature=
)
> and ehhance the previous fix with runtime selection of needed
> conversion
Thinking more about this, or:
3. Always store the data in LE (or BE, doesn't matter), and
appropriately convert while reading. This will likely require least chan=
ge.
> I have put the change for #2 together at
> https://reviews.freebsd.org/D17603 (more a PoC at the moment than a rea=
l
> review), and wondering if it looks sane enough or if there's anything
> obvious I'm missing.
>=20
> I have briefly tested the libc and locale files (LC_CTYPE and
> LC_COLLATE) built on amd64 on a powerpc system, and it seems to work.
>=20
> TIA
>=20
--V7dAMGwHWZv6qJlfJX3KibuOQEFFSHKLP--
--1PLmTiomysntfNknvXhyS0R0pJqI4h7pF
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCAAdFiEE+Gq3PsPeLT4tL/9wk4vgf7Eq4WwFAlvH6OIACgkQk4vgf7Eq
4WxjrggAhY+fwpVV+PI7cSvxAJayX5bf4qtXjploX2GsLcA/OK3ZpVFPtlPmgrHK
P8tENd7U34pufS5lTrnov7Uv7OEWsiOtK890NyNQFYMeqWDdVsNIctzfteu2sR8c
wUOvnEcZ6+uANDWH2owlB7K+caM1nhhkT184tlpKGM+RGoeg13OGEib9lcJvDvqQ
8e771orSth2wMNqFmamYlxGj/llx5OjCkj+nyjhN8c2eaAlxw2YbJoAEt7MF6axk
+ZSrlbrxwEjeh9r4ifz2FowviHjw+4/kR0WGkLmIPvnVT7Sg+6pDbG76lHYQ/CuU
5eqOWDSqYajbi7Kb3qWEZuj+xCoEkw==
=rtsl
-----END PGP SIGNATURE-----
--1PLmTiomysntfNknvXhyS0R0pJqI4h7pF--
From owner-freebsd-hackers@freebsd.org Thu Oct 18 06:16:35 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6C52C10CF1AE
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Thu, 18 Oct 2018 06:16:35 +0000 (UTC)
(envelope-from phk@critter.freebsd.dk)
Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222])
by mx1.freebsd.org (Postfix) with ESMTP id 0608B7D6C7
for <freebsd-hackers@freebsd.org>; Thu, 18 Oct 2018 06:16:34 +0000 (UTC)
(envelope-from phk@critter.freebsd.dk)
Received: from critter.freebsd.dk (unknown [192.168.55.3])
by phk.freebsd.dk (Postfix) with ESMTP id 51E4214816;
Thu, 18 Oct 2018 06:16:27 +0000 (UTC)
Received: from critter.freebsd.dk (localhost [127.0.0.1])
by critter.freebsd.dk (8.15.2/8.15.2) with ESMTPS id w9I6GQMI092616
(version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO);
Thu, 18 Oct 2018 06:16:26 GMT (envelope-from phk@critter.freebsd.dk)
Received: (from phk@localhost)
by critter.freebsd.dk (8.15.2/8.15.2/Submit) id w9I6GP3I092615;
Thu, 18 Oct 2018 06:16:25 GMT (envelope-from phk)
To: Warner Losh <imp@bsdimp.com>
cc: Yuri Pankov <yuripv@yuripv.net>,
"freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org>
Subject: Re: embed endian info in locale data files magic (PR231965)
In-reply-to: <CANCZdfpuUhJTxeUrEn1jrphSVaPqWw86Xm-e-rdX_mwC_kJO9w@mail.gmail.com>
From: "Poul-Henning Kamp" <phk@phk.freebsd.dk>
References: <f081ab81-56d0-49cc-92b0-fadc09618d31@yuripv.net>
<abaa1192-defc-973e-8d0f-b8e5a2437c0d@yuripv.net>
<CANCZdfpuUhJTxeUrEn1jrphSVaPqWw86Xm-e-rdX_mwC_kJO9w@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <92613.1539843385.1@critter.freebsd.dk>
Content-Transfer-Encoding: quoted-printable
Date: Thu, 18 Oct 2018 06:16:25 +0000
Message-ID: <92614.1539843385@critter.freebsd.dk>
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 18 Oct 2018 06:16:35 -0000
--------
In message <CANCZdfpuUhJTxeUrEn1jrphSVaPqWw86Xm-e-rdX_mwC_kJO9w@mail.gmail=
.com>
, Warner Losh writes:
>> 3. Always store the data in LE (or BE, doesn't matter), and
>> appropriately convert while reading. This will likely require least
>> change.
>
>I like this.
The locale stuff can have a surprisingly large performance impact, so
it might be a good idea to run some benchmarks first.
-- =
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk@FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe =
Never attribute to malice what can adequately be explained by incompetence=
.
From owner-freebsd-hackers@freebsd.org Thu Oct 18 07:57:30 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 989C410D2C3E
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Thu, 18 Oct 2018 07:57:30 +0000 (UTC)
(envelope-from bapt@FreeBSD.org)
Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "smtp.freebsd.org",
Issuer "Let's Encrypt Authority X3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 4BFC68110E;
Thu, 18 Oct 2018 07:57:30 +0000 (UTC)
(envelope-from bapt@FreeBSD.org)
Received: from ivaldir.etoilebsd.net (etoilebsd.net [178.32.217.76])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate) (Authenticated sender: bapt)
by smtp.freebsd.org (Postfix) with ESMTPSA id 30A7F1277F;
Thu, 18 Oct 2018 07:57:30 +0000 (UTC)
(envelope-from bapt@FreeBSD.org)
Received: by ivaldir.etoilebsd.net (Postfix, from userid 1001)
id 2EE738C333; Thu, 18 Oct 2018 09:57:29 +0200 (CEST)
Date: Thu, 18 Oct 2018 09:57:29 +0200
From: Baptiste Daroussin <bapt@FreeBSD.org>
To: Yuri Pankov <yuripv@yuripv.net>
Cc: freebsd-hackers <freebsd-hackers@freebsd.org>
Subject: Re: embed endian info in locale data files magic (PR231965)
Message-ID: <20181018075728.sxbvrtnqorrieuu2@ivaldir.net>
References: <f081ab81-56d0-49cc-92b0-fadc09618d31@yuripv.net>
<abaa1192-defc-973e-8d0f-b8e5a2437c0d@yuripv.net>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
protocol="application/pgp-signature"; boundary="jkvypxlwu4xz5ve5"
Content-Disposition: inline
In-Reply-To: <abaa1192-defc-973e-8d0f-b8e5a2437c0d@yuripv.net>
User-Agent: NeoMutt/20180716
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.27
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 18 Oct 2018 07:57:30 -0000
--jkvypxlwu4xz5ve5
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Thu, Oct 18, 2018 at 04:58:58AM +0300, Yuri Pankov wrote:
> Yuri Pankov wrote:
> > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D231965 ([PowerPC64]
> > Cross compiling powerpc64 from amd64 results in nonfunctional locale
> > installations), describes the issue with locale data built on LE system
> > (amd64) when used on BE system (powerpc).
> >=20
> > Fix introduced in rS308170 depends on the fact that locale data is built
> > on LE system, and will likely fail when it's built natively on mips
> > (please correct me if I'm wrong). More so, we shouldn't be hardcoding
> > the conversion in libc, and I see 2 options here:
> >=20
> > 1. fix localedef to output data in target's system endian
> > 2. embed the endian info in locale data files (updating magic signature)
> > and ehhance the previous fix with runtime selection of needed
> > conversion
>=20
> Thinking more about this, or:
>=20
> 3. Always store the data in LE (or BE, doesn't matter), and
> appropriately convert while reading. This will likely require least chan=
ge.
>=20
I agree on that approach, just some benchmarking would be needed to make su=
re
we are not killing the performance
Best regards,
Bapt
--jkvypxlwu4xz5ve5
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEEgOTj3suS2urGXVU3Y4mL3PG3PloFAlvIPMMACgkQY4mL3PG3
PlpLqg/+PnpFP8OZ0lB97JD9lAdk+gkyMQ6FS8s9z4lBrfUReMcC9QOGJGmMHPTx
nD3Gb6XeQFwi89W9KDPJnI0h1L7nowDULyp8bAqeTO0QMEkESWb9VnvnmrGLbg/0
zrRRaxXuZnRcOTiSMGaXh01ENv9+G9Ja/FCF1xadbk0oTwiR9/Lt58Fr6zV83RmY
Pu6lxQI+xrWR04l5ORtovvv19DutZ0GhO/oEFzqy8n+Qa4KnFFxtCxf+ioNaQHRX
zwf4ATutmesaIGd+Y46GtpyQCN07P+lLbfaRxDqoXPjjjACpqa5qFXi7ZauVRnVW
aZdZyE67jKO+fUaInUn7H6q4NKLcHzIwDkk6RhDsn1Uxaa1deorJ2CTmUbLOQsiF
hPy+E3DavLLL8QiwlF7n75oCSqaYbBxKxf8Q314PGymjzzlCLk2KSKeAAfRpUxt+
VtlH/pq7kYF6iy5qCKTluoIW+vj0PLpsLXi9CnadcknhlOsGMfg3YGCjkOvOMz5p
wSFxjqaypsM2+oUNp99byb8HrPSR4K8PHAGr/StBbH/lGTCM4ubP7t0vQVaXQ5ez
adG7v3kWFu1sgW83coHYiLc6oTRXPo3/q6/oTXUZaKtOMR0Pf+aJzP/uulnd8zWE
sp6mFG8kaqX86HrF+f0pb8Z+8z+KxOJtN3Yh4SZxJviyK5n6iTM=
=fX4N
-----END PGP SIGNATURE-----
--jkvypxlwu4xz5ve5--
From owner-freebsd-hackers@freebsd.org Thu Oct 18 17:14:52 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8B542EFB95B
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Thu, 18 Oct 2018 17:14:52 +0000 (UTC)
(envelope-from brooks@spindle.one-eyed-alien.net)
Received: from spindle.one-eyed-alien.net (spindle.one-eyed-alien.net
[199.48.129.229])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 2AEEA73D7C;
Thu, 18 Oct 2018 17:14:52 +0000 (UTC)
(envelope-from brooks@spindle.one-eyed-alien.net)
Received: by spindle.one-eyed-alien.net (Postfix, from userid 3001)
id EDC553C475F; Thu, 18 Oct 2018 17:14:50 +0000 (UTC)
Date: Thu, 18 Oct 2018 17:14:50 +0000
From: Brooks Davis <brooks@freebsd.org>
To: Baptiste Daroussin <bapt@FreeBSD.org>
Cc: Yuri Pankov <yuripv@yuripv.net>,
freebsd-hackers <freebsd-hackers@freebsd.org>
Subject: Re: embed endian info in locale data files magic (PR231965)
Message-ID: <20181018171450.GA18726@spindle.one-eyed-alien.net>
References: <f081ab81-56d0-49cc-92b0-fadc09618d31@yuripv.net>
<abaa1192-defc-973e-8d0f-b8e5a2437c0d@yuripv.net>
<20181018075728.sxbvrtnqorrieuu2@ivaldir.net>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha1;
protocol="application/pgp-signature"; boundary="a8Wt8u1KmwUX3Y2C"
Content-Disposition: inline
In-Reply-To: <20181018075728.sxbvrtnqorrieuu2@ivaldir.net>
User-Agent: Mutt/1.9.4 (2018-02-28)
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 18 Oct 2018 17:14:52 -0000
--a8Wt8u1KmwUX3Y2C
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Thu, Oct 18, 2018 at 09:57:29AM +0200, Baptiste Daroussin wrote:
> On Thu, Oct 18, 2018 at 04:58:58AM +0300, Yuri Pankov wrote:
> > Yuri Pankov wrote:
> > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D231965 ([PowerPC6=
4]
> > > Cross compiling powerpc64 from amd64 results in nonfunctional locale
> > > installations), describes the issue with locale data built on LE syst=
em
> > > (amd64) when used on BE system (powerpc).
> > >=20
> > > Fix introduced in rS308170 depends on the fact that locale data is bu=
ilt
> > > on LE system, and will likely fail when it's built natively on mips
> > > (please correct me if I'm wrong). More so, we shouldn't be hardcoding
> > > the conversion in libc, and I see 2 options here:
> > >=20
> > > 1. fix localedef to output data in target's system endian
> > > 2. embed the endian info in locale data files (updating magic signatu=
re)
> > > and ehhance the previous fix with runtime selection of needed
> > > conversion
> >=20
> > Thinking more about this, or:
> >=20
> > 3. Always store the data in LE (or BE, doesn't matter), and
> > appropriately convert while reading. This will likely require least ch=
ange.
> >=20
> I agree on that approach, just some benchmarking would be needed to make =
sure
> we are not killing the performance
While not a bad idea, we should probably focus on benchmarking modern
power systems. None of the other BE systems are all that important IMO
(I say this as someone who uses BE mips64 daily.)
-- Brooks
--a8Wt8u1KmwUX3Y2C
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQEcBAEBAgAGBQJbyL+KAAoJEKzQXbSebgfA9m0H/3tvNTIPY/necEPmXnUUwffR
lpkmz/AQoVRjgqo+VSxcmliPya0++o+jMHck1/c4yROZwAkBzZ5RugEnu5LT7tRX
hgOfsqcpGm6jQA571di5S2gFfIQybgYvv/mDUU/PFygf4lnMS2kaD5uFg9BjRbo5
b1mYHED7YhKZdEZcAcriQJbqIYa9xwios8BkX37Ggsm0upuA2uLsQfHQ8xpuNezf
N9piug6lr9FYtdAUkWgH0hBo37lYEgttQPux4+V0oSASre6bGvaOv/YpQxVGccQf
bG0hbUm4zQni3H84r+S+AejmEMbSM+C8WgKN8Daw//efwh3Ot223z3qH3B9Ehco=
=rMcS
-----END PGP SIGNATURE-----
--a8Wt8u1KmwUX3Y2C--
From owner-freebsd-hackers@freebsd.org Thu Oct 18 20:52:23 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id DF703F74CF5
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Thu, 18 Oct 2018 20:52:22 +0000 (UTC)
(envelope-from yuripv@yuripv.net)
Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com
[66.111.4.28])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 7C86B7E43F
for <freebsd-hackers@freebsd.org>; Thu, 18 Oct 2018 20:52:22 +0000 (UTC)
(envelope-from yuripv@yuripv.net)
Received: from compute5.internal (compute5.nyi.internal [10.202.2.45])
by mailout.nyi.internal (Postfix) with ESMTP id AF87822B92
for <freebsd-hackers@freebsd.org>; Thu, 18 Oct 2018 16:52:21 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163])
by compute5.internal (MEProxy); Thu, 18 Oct 2018 16:52:21 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yuripv.net; h=
subject:references:to:from:message-id:date:mime-version
:in-reply-to:content-type; s=fm3; bh=Te4ueGyHXySnT0f/4qAVuhm+QGZ
+tVbZNIWdNaVFG/A=; b=SDncs8KUQ6UB7JsUI1qUibcIKE8YQZ2p6PtE+N+6Tk7
XnzDfajbwso4B1QCoV1hWGvJYM2YdSijPeBeD5SPgSqOSP+8ZHvq5u5Ok9OKlRvq
omp8yqANDdYBoBbUAHwaJ4uA9qlyAJo+kuk6/xep16iTIJcxGDmmsd3eSwZuzRvX
DDbkd+iico3iFiHeOHu3k3AqC6PFzn86JQQLA3unirI9uKejE5xYQPOliD44IYZL
D74a8ANjKOdhtiOs21uc15p5VqK5eG/g18WuALiNRc4Z2GtC0nx3E6Vt/S2EiSUr
gCU6KxTkZQri8BZ5ndqVUeeJWpFxDPaSzvmcSb6hoBA==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=content-type:date:from:in-reply-to
:message-id:mime-version:references:subject:to:x-me-proxy
:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; bh=Te4ueG
yHXySnT0f/4qAVuhm+QGZ+tVbZNIWdNaVFG/A=; b=osCpAefovInNNs5LnIxHlz
/URA/yQmQ4CYfR42WSMZWJAqo+B4d3u8v8VCYxE7bhd50wCcfQSjhLAV326+C/yV
Xp8HkpGOyzv5Ob8ZA/GUt9RS3ZdigSQuDC3JVDacBV6sgy+uDytefSD4ZTD9uI+i
OsMRVqojTvhie4S/UwuIE/nmlJdGrqNYVnr9EJA5tlaysl3YJnHr35p0XJdocGde
H11CzGD4lcF+H70UyQff9/+/ZMiUVSK4vETkU7jeaglQYSExW3J4/A/YBSqIPv4h
5OlGkcEjyJhuDMqPbvPXfEjQTkflLypz0RZWxu9mcKFatr0y5hixkJKDthys1nXQ
==
X-ME-Sender: <xms:hfLIWwoTJ75-j6jG6Lz01kzuIF9EaKmaFv4yXGN1H-KbKHLmbWUuNA>
X-ME-Proxy: <xmx:hfLIW9WdDZ9xRYFccxE4rWTUZWNfHR7Yto8CNV7lmibB1Ub5IfhdRw>
<xmx:hfLIW3mv3p_YZ6ilocoJgpSmvNZ5SpCzcMj9mnOAfRYa-Q1_yytHZQ>
<xmx:hfLIWwD7Qf3ThQwTOup6Z7FDaEezMYPFL2BR00S9tAuBVwtzJnUnWA>
<xmx:hfLIW-jcUWHMtBUnT7HgdAARwFPc_knecnmpY7-JHPqtLJpTY_w_WA>
<xmx:hfLIW1zP9lRVpPqCzyJFwQ9-J-_7ijEuPpxBT5KrsGpZM1XlfAIbwg>
<xmx:hfLIW8TYMFvUIwBqgqOxDrfmCvEOqNInBEbQIr-OE6u4BJ4MZ03xbA>
Received: from [192.168.1.2] (unknown [85.173.27.113])
by mail.messagingengine.com (Postfix) with ESMTPA id 34859102E2
for <freebsd-hackers@freebsd.org>; Thu, 18 Oct 2018 16:52:20 -0400 (EDT)
Subject: Re: embed endian info in locale data files magic (PR231965)
References: <f081ab81-56d0-49cc-92b0-fadc09618d31@yuripv.net>
<abaa1192-defc-973e-8d0f-b8e5a2437c0d@yuripv.net>
<CANCZdfpuUhJTxeUrEn1jrphSVaPqWw86Xm-e-rdX_mwC_kJO9w@mail.gmail.com>
<92614.1539843385@critter.freebsd.dk>
To: freebsd-hackers <freebsd-hackers@freebsd.org>
From: Yuri Pankov <yuripv@yuripv.net>
Openpgp: preference=signencrypt
Autocrypt: addr=yuripv@yuripv.net; keydata=
xsBNBFu8u6IBCADB11gP0QwnorrHjqAtKLHKHNHskhy0s7jqJKfx0YqXgVBKGLJ9/mjLAz0F
CBNvemHSDDTs0mEZ9cBKKi6cmsav6+UQgr//yai6hvXLBJqKchSFO4MhmdvBtsGFq1yKz5Zi
uhjmimKyIpgBgvMdbgGbGq6cnSB2uEPmZuJr419SVRODOkXukU+F5WHgaHzDdHAIu1asCt2B
+6msxqIqlFWcXyZyTGicTGGvC/PFIsVRUtD1dIJANTC876g7DTb7LZXWiWwJpSJ4GKMXMHVX
Ct9BoQ4i3nhKbOxb6Io1wsy+NFyWsTJ9KYrxKKPJP3oG8BWb/cqlFqnE4eNSsiq2q7krABEB
AAHNH1l1cmkgUGFua292IDx5dXJpcHZAeXVyaXB2Lm5ldD7CwJcEEwEIAEECGwMFCQWjmoAF
CwkIBwMFFQoJCAsFFgMCAQACHgECF4AWIQT4arc+w94tPi0v/3CTi+B/sSrhbAUCW708wAIZ
AQAKCRCTi+B/sSrhbPxBB/961alcU091O+yKT5/oReHVc/PX0Tz4sW3V44AcgLfYlrZavCro
EFz90qmCrl0xqEwuAKcC4bjmL8SjPWAhSN6IH9nxdw+HeZnAPiHm/q679Bu47+nHBl3qD/9p
+t1PkKeKZfaWToFMt1nq06ytSu6VLMCwLdlDNe6DReX0ex/afEqKsuaIZSKL4UYjRwklp8PU
Uf98QkrfapyHB67hQMzfI4tPeJaYyv0cTgfq3kUWJx1V6Xi0b6Zxj4ZrB2TXvaMO5g7yhU9E
E3WWAvoe4FgB3a7dHe8atnHhq5+Cuvm6+LD4Jh7jvMAE5UMN+xxQpnGpNghHjaCy4vXrLRBZ
nhRYzsBNBFu8u6IBCADKih3Q933rDNj4ZA8FhBQ2RlmBgvwOLcDPIL3h0V7h38y3+HisgFSc
XACDsdrTlYZ1bRXkD9FHENynBcv0l/3uGJDk8jaGIDE0TP8OQBRp+IaU9/BHnAqrKxTJGIol
Dahy2m+yx2yhdc6B4ujWMDqCF1rWOD+ymOWw+VLllOkrHcZa5PJtX9UOGbApZl8ZTM8El4CA
NN8F1bg9MWzUi+8LYoGWGc+BwsFS1OUB1c4SPgMu5fD4Wfsr9yRl06fdpEA2YT7B/j5/5RSC
0sE2Zs/tmJ/JRflHJ12ycj59ma2xQMfEJF40hZDpMFQmZvbVqgEg3ocQcltjbxlIKZ/mjC4z
ABEBAAHCwHwEGAEKACYWIQT4arc+w94tPi0v/3CTi+B/sSrhbAUCW7y7ogIbDAUJBaOagAAK
CRCTi+B/sSrhbIDcCACqAZMcoxUBLZa40a5b24j5i1jplvCYYb3h+Q5lt5+BFJ87kCb4dJuU
D3kh2i29BrxWQWa9WNue9ozxeYkbkfXubQYXexVolRsnh64OdGsE8KvorBFBB3zdK/GRt2Jy
+jsnTfUWuQllbzMP0MfhCDMk1Mo8WvDH2/cOEP/yLKf20a+cd6nLs7bidjmGXo9pyuBKAtV6
Kv+VRu54AL+A/UBYu/eB3Dtvzcnut+1Zq6KaP++kUwPwINLIk04OBDwN0zRNTiqMAFYYyz2v
ZHBB6E1th/l//ZC5b9Dk0ZpFI1bYdL9ymnrZe1MqbGPnDCToQxu00T/pZCm6Z92YrZQYuNwl
Message-ID: <feb7f5ab-9818-72f1-98a1-848112d6c4ec@yuripv.net>
Date: Thu, 18 Oct 2018 23:52:10 +0300
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101
Thunderbird/60.2.1
MIME-Version: 1.0
In-Reply-To: <92614.1539843385@critter.freebsd.dk>
Content-Type: multipart/signed; micalg=pgp-sha256;
protocol="application/pgp-signature";
boundary="tF7bYPLEyp2hmgWJvyuc0MfyDwEogvTGq"
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 18 Oct 2018 20:52:23 -0000
This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--tF7bYPLEyp2hmgWJvyuc0MfyDwEogvTGq
Content-Type: multipart/mixed; boundary="zeVC69HDKDu7ivzgLoHj1A0PJ4E0nGZsv";
protected-headers="v1"
From: Yuri Pankov <yuripv@yuripv.net>
To: freebsd-hackers <freebsd-hackers@freebsd.org>
Message-ID: <feb7f5ab-9818-72f1-98a1-848112d6c4ec@yuripv.net>
Subject: Re: embed endian info in locale data files magic (PR231965)
References: <f081ab81-56d0-49cc-92b0-fadc09618d31@yuripv.net>
<abaa1192-defc-973e-8d0f-b8e5a2437c0d@yuripv.net>
<CANCZdfpuUhJTxeUrEn1jrphSVaPqWw86Xm-e-rdX_mwC_kJO9w@mail.gmail.com>
<92614.1539843385@critter.freebsd.dk>
In-Reply-To: <92614.1539843385@critter.freebsd.dk>
--zeVC69HDKDu7ivzgLoHj1A0PJ4E0nGZsv
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
Poul-Henning Kamp wrote:
> --------
> In message <CANCZdfpuUhJTxeUrEn1jrphSVaPqWw86Xm-e-rdX_mwC_kJO9w@mail.gm=
ail.com>
> , Warner Losh writes:
>=20
>>> 3. Always store the data in LE (or BE, doesn't matter), and
>>> appropriately convert while reading. This will likely require least
>>> change.
>>
>> I like this.
>=20
> The locale stuff can have a surprisingly large performance impact, so
> it might be a good idea to run some benchmarks first.
Thanks for the answers, and especially for mentioning the magic word
which is "performance" that I seem to be forgetting about. I have
updated the change to skip any runtime conversions and rather make
localedef output data in target system's byte order.
If anyone wants to take a look, it's still at:
https://reviews.freebsd.org/D17603
--zeVC69HDKDu7ivzgLoHj1A0PJ4E0nGZsv--
--tF7bYPLEyp2hmgWJvyuc0MfyDwEogvTGq
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCAAdFiEE+Gq3PsPeLT4tL/9wk4vgf7Eq4WwFAlvI8noACgkQk4vgf7Eq
4WzMXwgAsxqXQsV+G+JzF9kM65lTkbZAg16/MwKJtVPhDGgqmK1v0VIIwL/Malyq
GRaMtsWRZB2wBLiffj3X6vOwTrTEHcTmHDpB8eWG4dFicM6Owt3Mh5NXp4+VD2wy
2EPgM8deLgeSKVQ9mSXk9vqiu2FQfQoXHR12erlSrzcfa6iudbAQcBtkhKBjKiRx
dVIdimfFGjQcaAZxX0EfV9GcRbd4T+HGRb0LXkldNK4LJYw5954YS59Hp1qMgtWB
i5pezQdjnMPtCJegYenw8j5Fpkp9ZjFDxdFGIkTNMOr+14vBTZ0AhQi2TTjcBEN/
xvRD3yndMLE9uiKmxjdgek46hcSvxQ==
=r82M
-----END PGP SIGNATURE-----
--tF7bYPLEyp2hmgWJvyuc0MfyDwEogvTGq--
From owner-freebsd-hackers@freebsd.org Thu Oct 18 23:43:54 2018
Return-Path: <owner-freebsd-hackers@freebsd.org>
Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 37983F78C81
for <freebsd-hackers@mailman.ysv.freebsd.org>;
Thu, 18 Oct 2018 23:43:54 +0000 (UTC)
(envelope-from marklmi@yahoo.com)
Received: from sonic306-4.consmr.mail.bf2.yahoo.com
(sonic306-4.consmr.mail.bf2.yahoo.com [74.6.132.43])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 6D63A8408F
for <freebsd-hackers@freebsd.org>; Thu, 18 Oct 2018 23:43:53 +0000 (UTC)
(envelope-from marklmi@yahoo.com)
X-YMail-OSG: q1ZaTX8VM1l3kULXorahx9331cWS_TDGqXFHpdZe4jdHODYK1JGvK3PdZn7BQM7
yzqe323UC4mpXFXu1VsfRMvY5piF50xHWqGuV_4vOlrZYB2wJezl4nhsroZhsxgltp9JQY0wa4FL
1aLHI_Lf8ltoqamWWyvXksw7.dx8NR4jmG1Sb0w5iXNjJNPXbaK2IPSfPDFkpzDpzR_0wv1ZnoYS
MXwIrvbGUbYEGMdz.cpO6eTiXLJ_lG3KLOjpFzpGlp0hpz_xgzf7y85fE2iqqvNTPOYLvyblJpgp
599EbgmL5oGvA3vwf7I1vbrLE1ST9yHx1pFp1WSUu5gARHMmKVG7Dm6.3X4vs4FS1J2MgwIOA7Ql
G7XuSppDbT2Kv.NZvkO1zHmtAbezz0syLm5nSHq8YswDfQpRfqsL7rCVnE0Z0RNgi8nUn1JW_FIM
kMJFvhUAFDnBlTTGnNcn85yrVnom5URogg_5mxsEhNDlXpeE0Jmjk8DsDx8h7dUe5XQE5vdG12uX
IZ1rnDYCDWT7ybthDG1abIQNPaAaprSnUDcSwQTHfChNCuD85jocqgrMjqMszJW.QL5ZMmVET3sl
pM._9xn0A_w8BZLrUGWqqlXthuO8gWC4wliouHZuzHqC2K7ze1nwdotbyVrboA4yOHsRKjwzjb5s
hDvDB3boJV9KIu1MdoXreB2FDVpJqE0n2O.amHXr81t63_d7V5eqFhTxf4DLY6a2wweIH6F0U6mp
HU6BCM5oL3c7Y4ywH7UQ9OfPSKK1S_kuujsmhSQEmskSa7HqQ4EhejftIvralY0jMxRy05XeshBY
uRIvI566oikq3RNLAH0f5y6.THssVTQFBa3hyvUUj62hPD2ZwqSzkXJUJiA3_aDvgLRWa9q.8nbu
3FywMvqn4SSCgsuEtJE.1EuK4cfNaZAV8YVapjiLquvSOtIZ3VpUvhHGvHaZU.gVm0R64dHaru6o
Az6RWfDrOn8f6BWr2JNHac_OqORB6GY0N3QCoAkpu20_6d85QZrAimCuucqhFEnaUeYBJfDtn2tZ
3wQwowzR6be55c340PDPsTYCgAP_j0cWGVEoXkitfEHFDnVWTvWAwf3vR_9n0X9UNsz0vAC0ARlX
3sweLJLo4pDx_qzVqHA--
Received: from sonic.gate.mail.ne1.yahoo.com by
sonic306.consmr.mail.bf2.yahoo.com with HTTP; Thu, 18 Oct 2018 23:43:46 +0000
Received: from c-76-115-7-162.hsd1.or.comcast.net (EHLO [192.168.1.25])
([76.115.7.162])
by smtp416.mail.bf1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID
a646330be3e13d20eaceb7b17daac9d4;
Thu, 18 Oct 2018 23:43:43 +0000 (UTC)
Content-Type: text/plain;
charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\))
Subject: Re: What is incomplete for /lib/libgcc_s.so-based C++ exception
handling (where WITH_LLVM_LIBUNWIND= and /usr/local/lib/gcc*/libgcc_s.so are
not used)
From: Mark Millard <marklmi@yahoo.com>
In-Reply-To: <0379371E-0541-42DD-93EF-BEE2E9DE3FBC@yahoo.com>
Date: Thu, 18 Oct 2018 16:43:40 -0700
Cc: FreeBSD PowerPC ML <freebsd-ppc@freebsd.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <B8C7F4C3-C077-4166-86D4-180EF23F4905@yahoo.com>
References: <0379371E-0541-42DD-93EF-BEE2E9DE3FBC@yahoo.com>
To: FreeBSD Toolchain <freebsd-toolchain@freebsd.org>,
FreeBSD <freebsd-hackers@freebsd.org>, freebsd-arch@freebsd.org
X-Mailer: Apple Mail (2.3445.9.1)
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
<freebsd-hackers.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 18 Oct 2018 23:43:54 -0000
[I add a note indicating that clang++ does
generate examples of DW_CFA_remember_state
and DW_CFA_restore_state use for pwoerpc64
that lead to /lib/libgcc_s.so messing up
the exception handling.]
On 2018-Oct-17, at 1:25 PM, Mark Millard <marklmi@yahoo.com> wrote:
> [This summarizes other results without the code
> and debugger evidence and such from my recent
> explorations. It should be much easier to
> follow than my exploration reports.]
>=20
> Documents like DWARF5.pdf document the "row" vs. Location
> information for Call Frame Information as (also used
> for .eh_frame like materials for C++ exception handling):
> (CFA: Cannonical Frame Address)
>=20
> QUOTE ("Structure of Call Frame Information")
> LOC CFA R0 R1...RN
>=20
> L0
>=20
> L1
>=20
> ...
>=20
> LN
> END QUOTE
>=20
> Note that the CFA is conceptually one of the
> registers in each row, even though it is not a
> machine register but a way to calculate the
> conceptual register's value from machine
> registers.
>=20
> The information for the machine registers are
> typically based on the earlier CFA value (from
> the same row!). Absent a correct CFA cell in
> a row, most potential use of that row is likely
> messed up.
>=20
> One way CFA is found is by adding an offset to
> the value of a machine register for the range
> in question, Ln up to L(n+1) [or based on the
> end of the overall range for the last Ln]. I
> will use that for illustration because there
> are examples of this in my testing.
>=20
> /lib/libgcc_s.so.1 does not implement this
> fully for some DW_CFA_* operations:
>=20
> QUOTE (note the "every register" reference, so including CFA)
> DW_CFA_remember_state
>=20
> The DW_CFA_remember_state instruction takes no operands. The required =
action is to push the set of rules for every register onto an implicit =
stack.
>=20
> DW_CFA_restore_state
>=20
> The DW_CFA_restore_state instruction takes no operands. The required =
action is to pop the set of rules off the implicit stack and place them =
in the current row.
> END QUOTE
>=20
> In other words: push and pop a complete row,
> not just machine registers information from
> the row.
>=20
> For example, the the "cfa_offset" for computing the CFA
> value from from a register is not saved and restored.
> Nor is which register the offset is based on. (This
> can vary, though not in my examples.) In general the
> CFA cell is not saved and restored, what ever its
> contents.
>=20
> So any compiler that produces code depending on
> DW_CFA_remember_state and DW_CFA_restore_state
> for .eh_frame like material ends up with C++
> exception handling messed up when the
> DW_CFA_restore_state should change the CFA to a=20
> non-default one (from the prior
> DW_CFA_remember_state).
>=20
> This prevents reliable use of throwing C++ exceptions
> when building via the likes of devel/powerpc64-gcc
> or lang/gcc8 ( when not using
> -Wl,-rpath=3D-Wl,-rpath=3D/usr/local/lib/gcc8 so that
> /lib/libgcc_s.so.1 ends up being used). One result
> can be _Unwind_RaiseException looping looking at
> the same frame over and over instead of progressing
> to the next frame. For example, this happens via
> cfa_offset 0 being used. devel/powerpc64-gcc -O2
> code tends to get that.
>=20
>=20
> Notes:
>=20
> For powerpc64, clang++ tends to use another register
> (%r31) with the old value (of %r1, the stack pointer)
> instead of involving the
> DW_CFA_remember_state/DW_CFA_restore_state pair
> based on just %r1. (clang has other problems
> relative to sue for buildworld buildkernel.)
/usr/tests/lib/atf/libatf-c++/detail/exceptions_test
has examples were clang++ generated use of
DW_CFA_remember_state and DW_CFA_restore_state and
/lib/libgcc_s.so 's _Unwind_RaiseException ends
up stuck looping. There are other examples as well.
The problem is not limited to devel/powerpc64-gcc
or other g++ use that uses /lib/libgcc_s.so .
> Code generation styles matter for if the incomplete
> coverage by /lib/libgcc_s.so will be visible or not.
>=20
> At this stage, WITH_LLVM_LIBUNWIND=3D builds
> targeting powerpc64 do not even compile/assemble
> the relevant code, apparently both because of
> darwin specific assembler code and FreeBSD's build
> not using the C-preprocessor on the .S file as
> required. (There could be more to getting it
> working.)
>=20
> I do not know about other architecture/compiler
> (or toolchain) combinations that may not yet be
> able to use WITH_LLVM_LIBUNWIND=3D . But I'd
> expect a potentially similar status from some.
>=20
> A range of modern /usr/local/lib/gcc*/libgcc_s.so
> do implement DW_CFA_remember_state/DW_CFA_restore_state
> operations and they are put to use. So using the likes
> of -Wl,-rpath=3D/usr/local/lib/gcc8 works for g++8 C++
> exception handling (but is problematical for buildworld
> buildkernel).
>=20
> I made a similar exploration of the issue in around
> early 2016 and got basically the same results, not that
> I remembered much. But I now have a small source code
> example that shows the cfa_offset issue for the likes
> of devel/powerpc64-gcc output.
>=20
> The standard source for throw_exception in
> /lib/libgcc_s.so produces the cfa_offset problem
> when devel/powerpc64-gcc is used to buildworld.
> This turns all thrown C++ exceptions in to
> unbounded looping in _Unwind_RaiseException for
> that kind of context.
=3D=3D=3D
Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)