From owner-freebsd-arch@FreeBSD.ORG Sun Apr 5 00:22:31 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C4621498 for ; Sun, 5 Apr 2015 00:22:31 +0000 (UTC) Received: from mail-ig0-x22b.google.com (mail-ig0-x22b.google.com [IPv6:2607:f8b0:4001:c05::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 892593F5 for ; Sun, 5 Apr 2015 00:22:31 +0000 (UTC) Received: by igbqf9 with SMTP id qf9so1234390igb.1 for ; Sat, 04 Apr 2015 17:22:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=0oyeyxVroRWEGo1HLjm74xyuQNJGbo+C44Y6UQ5SL24=; b=XrkhqSztZXzbZx0bxzaW8LkxZv6ugNsJSi4kyeg3tWTRtl2ePIz5yg0CwewzJd4cru kPPUPVsspARmHw9bVFtIPPgy0E1w6OBNsyyyjKO1OxcgASpmVoyZ2tMU+A9z3BLQXa7V 0Fwgnyz3yxSGuSKJKXB1dtvbuTVGXaXryBJDMk0Q5JhoD0BbVbYgf+bjNHSQxA/FOGzl xXEr/igU3sPGMyVid1NzN3976LjZkS+MAeSjrQHDxFgAu0LbakAhVeHRRAPTQLdAO0Du YoRNkidsRkoJzDvQ0UttSNz6pSVHfElflIvfeVbAwGRrg4uS8QQSiTXsHCkR1xtXs+Pi KK5g== MIME-Version: 1.0 X-Received: by 10.43.163.129 with SMTP id mo1mr137018icc.61.1428193350894; Sat, 04 Apr 2015 17:22:30 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.36.17.194 with HTTP; Sat, 4 Apr 2015 17:22:30 -0700 (PDT) In-Reply-To: References: Date: Sat, 4 Apr 2015 17:22:30 -0700 X-Google-Sender-Auth: hOEwfbPfWmjmTqgBHh2ACEKJtkc Message-ID: Subject: Re: make universe failures? From: Adrian Chadd To: Garrett Cooper Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Apr 2015 00:22:31 -0000 On 4 April 2015 at 16:33, Garrett Cooper wrote: > On Apr 4, 2015, at 16:28, Adrian Chadd wrote: > >> Hi, >> >> cc -O -pipe -I/usr/home/adrian/work/freebsd/uverse/head/lib/libcrypt >> -std=3Dgnu99 -Qunused-arguments >> -L/home/adrian/work/freebsd/uverse/obj/arm.arm/usr/home/adrian/work/free= bsd/uverse/head/tmp/usr/lib/private >> -rpath /usr/lib/private -rpath /usr/lib/private -o crypt_tests >> crypt_tests.o /home/adrian/work/freebsd/uverse/obj/arm.arm/usr/home/adri= an/work/freebsd/uverse/head/lib/atf/libatf-c/libatf-c.so >> -lcrypt >> cc: error: no such file or directory: >> '/home/adrian/work/freebsd/uverse/obj/arm.arm/usr/home/adrian/work/freeb= sd/uverse/head/lib/atf/libatf-c/libatf-c.so' >> >> .. this happens for make universe on at least amd64, arm and armeb. >> I'm still waiting for the others to (not) complete. > > Might be related to r280179 and r273449, but more info=E2=80=99s needed. = Your src.conf for the build host and targets would be helpful. > Thanks! adrian@bruce:~/work/freebsd/uverse/head % cat /etc/make.conf cat: /etc/make.conf: No such file or directory adrian@bruce:~/work/freebsd/uverse/head % cat /etc/src.conf cat: /etc/src.conf: No such file or directory adrian@bruce:~/work/freebsd/uverse/head % env MAKEOBJDIRPREFIX=3D/home/adrian/work/freebsd/uverse/obj make -j2 universe JFLAG=3D-j8 Thanks for replying so quickly! -adrian From owner-freebsd-arch@FreeBSD.ORG Sun Apr 5 00:25:51 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 979967BB; Sun, 5 Apr 2015 00:25:51 +0000 (UTC) Received: from mail-pa0-x22a.google.com (mail-pa0-x22a.google.com [IPv6:2607:f8b0:400e:c03::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5CEEA639; Sun, 5 Apr 2015 00:25:51 +0000 (UTC) Received: by patj18 with SMTP id j18so3439267pat.2; Sat, 04 Apr 2015 17:25:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=45NmAy2jhHG8kLIepRKSifHDYgS48smZUuFhK7yFJS4=; b=oHkkd7+JSTLqyyYdYH1e9EEVLl4lRQDy20+XEngWi1LODQWZbzQaaPSbcz+prNCwWg 2rli6hM18GTnuWKPXKC/yKgc4EMe8zB1eyg8T35U19x0syJv6eke2yGCo7c/H7mVf50v Q+dCVChu9X/W+em6E7KK7DXGMZaJI3SUjqtS+uKkD5DQxGDJtLlqpjJWnozwbYAQea8t WcwLpp58IhfaRZ4D0qOXNQ3xXaBvQh/HraaNwD6MWHBtnpRLHrvOe3+sd+KfBa1/D8Zx ptUPQGPKXy4f76ZfmUvAdLf5Qa6rzhkGK6AN3d6+nTdYYjUT0kdox2P+dxkXW+AZweQi /wRg== X-Received: by 10.70.131.15 with SMTP id oi15mr15679740pdb.161.1428193550596; Sat, 04 Apr 2015 17:25:50 -0700 (PDT) Received: from ?IPv6:2601:8:ab80:7d6:981:fa61:f8b7:c531? ([2601:8:ab80:7d6:981:fa61:f8b7:c531]) by mx.google.com with ESMTPSA id l7sm109139pdp.71.2015.04.04.17.25.49 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 04 Apr 2015 17:25:49 -0700 (PDT) Content-Type: multipart/signed; boundary="Apple-Mail=_DC97B507-E8A5-49C5-A00D-EB435BEBA1F0"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: make universe failures? From: Garrett Cooper In-Reply-To: Date: Sat, 4 Apr 2015 17:25:48 -0700 Message-Id: <310573CF-443F-47BA-9F96-630107A4BD52@gmail.com> References: To: Adrian Chadd X-Mailer: Apple Mail (2.1878.6) Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Apr 2015 00:25:51 -0000 --Apple-Mail=_DC97B507-E8A5-49C5-A00D-EB435BEBA1F0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Apr 4, 2015, at 17:22, Adrian Chadd wrote: > On 4 April 2015 at 16:33, Garrett Cooper = wrote: >> On Apr 4, 2015, at 16:28, Adrian Chadd wrote: >>=20 >>> Hi, >>>=20 >>> cc -O -pipe = -I/usr/home/adrian/work/freebsd/uverse/head/lib/libcrypt >>> -std=3Dgnu99 -Qunused-arguments >>> = -L/home/adrian/work/freebsd/uverse/obj/arm.arm/usr/home/adrian/work/freebs= d/uverse/head/tmp/usr/lib/private >>> -rpath /usr/lib/private -rpath /usr/lib/private -o crypt_tests >>> crypt_tests.o = /home/adrian/work/freebsd/uverse/obj/arm.arm/usr/home/adrian/work/freebsd/= uverse/head/lib/atf/libatf-c/libatf-c.so >>> -lcrypt >>> cc: error: no such file or directory: >>> = '/home/adrian/work/freebsd/uverse/obj/arm.arm/usr/home/adrian/work/freebsd= /uverse/head/lib/atf/libatf-c/libatf-c.so' >>>=20 >>> .. this happens for make universe on at least amd64, arm and armeb. >>> I'm still waiting for the others to (not) complete. >>=20 >> Might be related to r280179 and r273449, but more info=92s needed. = Your src.conf for the build host and targets would be helpful. >> Thanks! >=20 > adrian@bruce:~/work/freebsd/uverse/head % cat /etc/make.conf > cat: /etc/make.conf: No such file or directory > adrian@bruce:~/work/freebsd/uverse/head % cat /etc/src.conf > cat: /etc/src.conf: No such file or directory >=20 > adrian@bruce:~/work/freebsd/uverse/head % env > MAKEOBJDIRPREFIX=3D/home/adrian/work/freebsd/uverse/obj make -j2 > universe JFLAG=3D-j8 >=20 > Thanks for replying so quickly! -j2 and JFLAG probably isn=92t the best idea in the world. JFLAG should = be ok... --Apple-Mail=_DC97B507-E8A5-49C5-A00D-EB435BEBA1F0 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJVIIENAAoJEMZr5QU6S73eGlcH/3gq6p8RuRZJfC6ckHA6gs7u X0NixejjM04UfyrOkAgTsmhMr0dd0dmdozNxHNz2QBbYxbS+fbYrYgzBTDjzFx5w 2+GME4Bp/plDWjqCOgl0b1BNPKAPlGjyBZVoxJ7te6LA0vKdGZkSFkiy5vFyOCS4 f1Xxv2cLi7MabNCA9sXVmN/m2BnqjTk9GcBtWp9fQe04rpNXqJXpoyn01FYPPu6l 7T1se2RpsdnwsY9Ki8xj4SdbLF034/8In5JTaZUKrniV5wudHfAUhubnvuvf7+kO m+S9pvOlbNLpMxIy/UEIKftg3llOuw7s17/+Lxxs0865I6Fmu1lUTCZ4siMlAS0= =dUlE -----END PGP SIGNATURE----- --Apple-Mail=_DC97B507-E8A5-49C5-A00D-EB435BEBA1F0-- From owner-freebsd-arch@FreeBSD.ORG Sun Apr 5 00:33:33 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D3934AF0 for ; Sun, 5 Apr 2015 00:33:33 +0000 (UTC) Received: from mail-ig0-x22e.google.com (mail-ig0-x22e.google.com [IPv6:2607:f8b0:4001:c05::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 971AD7BD for ; Sun, 5 Apr 2015 00:33:33 +0000 (UTC) Received: by igcau2 with SMTP id au2so1395678igc.1 for ; Sat, 04 Apr 2015 17:33:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=VvOxyt6rFq3sYqcoQHoNltgUfFAK9rIJbnlhUJDfbUc=; b=z9J9sAVMtXW38X5stYO7VII+GexDq7XRZjTBz0jqq83XoSHBBa0ZRjnKE0vsGreIM2 4bL1VCTj9tgVbDNNZM5cUk5B54OCxG5plfrU/+SUsGVJuwrCAxRNmk2sivu93ZrC3izr 21P1FlF3P8l6qPykCcGJ4G4ocEUQtX8L8T9hcW0QdK6u5kTTe1mPfVGUMS25JOqKgWWb s8vfRyyGY1YCkhlYIgstuTg4zzBxEmNf2mWAF1JOiG/fZtHknu/+Pg0iBvpBj8vK4q+C aOmZ+l6pPrYU+l//NnaQYXiWlxzfcBbem8kiBArk09V0eIQmiUGVwUFtM+QSt3I2DLgg oq+A== MIME-Version: 1.0 X-Received: by 10.50.73.168 with SMTP id m8mr30279552igv.32.1428194012974; Sat, 04 Apr 2015 17:33:32 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.36.17.194 with HTTP; Sat, 4 Apr 2015 17:33:32 -0700 (PDT) In-Reply-To: <310573CF-443F-47BA-9F96-630107A4BD52@gmail.com> References: <310573CF-443F-47BA-9F96-630107A4BD52@gmail.com> Date: Sat, 4 Apr 2015 17:33:32 -0700 X-Google-Sender-Auth: l0CPxRw50P3gYkIBG91TmSqX0Yc Message-ID: Subject: Re: make universe failures? From: Adrian Chadd To: Garrett Cooper Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Apr 2015 00:33:33 -0000 On 4 April 2015 at 17:25, Garrett Cooper wrote: > On Apr 4, 2015, at 17:22, Adrian Chadd wrote: > >> On 4 April 2015 at 16:33, Garrett Cooper wrote: >>> On Apr 4, 2015, at 16:28, Adrian Chadd wrote: >>> >>>> Hi, >>>> >>>> cc -O -pipe -I/usr/home/adrian/work/freebsd/uverse/head/lib/libcryp= t >>>> -std=3Dgnu99 -Qunused-arguments >>>> -L/home/adrian/work/freebsd/uverse/obj/arm.arm/usr/home/adrian/work/fr= eebsd/uverse/head/tmp/usr/lib/private >>>> -rpath /usr/lib/private -rpath /usr/lib/private -o crypt_tests >>>> crypt_tests.o /home/adrian/work/freebsd/uverse/obj/arm.arm/usr/home/ad= rian/work/freebsd/uverse/head/lib/atf/libatf-c/libatf-c.so >>>> -lcrypt >>>> cc: error: no such file or directory: >>>> '/home/adrian/work/freebsd/uverse/obj/arm.arm/usr/home/adrian/work/fre= ebsd/uverse/head/lib/atf/libatf-c/libatf-c.so' >>>> >>>> .. this happens for make universe on at least amd64, arm and armeb. >>>> I'm still waiting for the others to (not) complete. >>> >>> Might be related to r280179 and r273449, but more info=E2=80=99s needed= . Your src.conf for the build host and targets would be helpful. >>> Thanks! >> >> adrian@bruce:~/work/freebsd/uverse/head % cat /etc/make.conf >> cat: /etc/make.conf: No such file or directory >> adrian@bruce:~/work/freebsd/uverse/head % cat /etc/src.conf >> cat: /etc/src.conf: No such file or directory >> >> adrian@bruce:~/work/freebsd/uverse/head % env >> MAKEOBJDIRPREFIX=3D/home/adrian/work/freebsd/uverse/obj make -j2 >> universe JFLAG=3D-j8 >> >> Thanks for replying so quickly! > > -j2 and JFLAG probably isn=E2=80=99t the best idea in the world. JFLAG sh= ould be ok... Why not? -a From owner-freebsd-arch@FreeBSD.ORG Sun Apr 5 00:36:04 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E8512CF6; Sun, 5 Apr 2015 00:36:04 +0000 (UTC) Received: from mail-pd0-x229.google.com (mail-pd0-x229.google.com [IPv6:2607:f8b0:400e:c02::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AC8347FB; Sun, 5 Apr 2015 00:36:04 +0000 (UTC) Received: by pddn5 with SMTP id n5so3644397pdd.2; Sat, 04 Apr 2015 17:36:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=6tWouP+EDT4girZy5S/iMxpDrnoS1PSTQMlt4TXKH8w=; b=jSin6+iyrwPovmpbWMohdv9t+y9dtTp+idYRW5YJKOXFdP8OjLJA8rybNjyOTiE3hs pdf0YsxC7h9WdLQ/1qEJLsI8EdQ906dhV316vyZYvqGVwYa+RXvfSvrnTpkOZ0WsujNJ X2/eaXffenCZFmvmtwjW/DhiaPbDpKbeY6yIHCpmT3Ea01tsv/0KG2Bv3U1Wn4s7eyJM RyipWL4IJJZfHFZbAHTaP3fKXzLU0pv6I1gekcK6siIUU6Iw0i6re+6spqvJZpc5y/op vSPolcJ9tLl0hXl0hDNCvGNrsZ5Pl5QKK5403K/P0OqOXCjzV8iyc9kFT4UUhgjxacI3 aliA== X-Received: by 10.68.69.45 with SMTP id b13mr15911128pbu.150.1428194164145; Sat, 04 Apr 2015 17:36:04 -0700 (PDT) Received: from ?IPv6:2601:8:ab80:7d6:981:fa61:f8b7:c531? ([2601:8:ab80:7d6:981:fa61:f8b7:c531]) by mx.google.com with ESMTPSA id g10sm132934pdm.29.2015.04.04.17.36.03 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 04 Apr 2015 17:36:03 -0700 (PDT) Content-Type: multipart/signed; boundary="Apple-Mail=_31F18AF5-0939-4C0A-ADFB-6879430D0460"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: make universe failures? From: Garrett Cooper In-Reply-To: Date: Sat, 4 Apr 2015 17:36:03 -0700 Message-Id: <43D75544-1458-4AE1-BCC1-DC5DB5AB72DA@gmail.com> References: <310573CF-443F-47BA-9F96-630107A4BD52@gmail.com> To: Adrian Chadd X-Mailer: Apple Mail (2.1878.6) Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Apr 2015 00:36:05 -0000 --Apple-Mail=_31F18AF5-0939-4C0A-ADFB-6879430D0460 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Apr 4, 2015, at 17:33, Adrian Chadd wrote: > On 4 April 2015 at 17:25, Garrett Cooper = wrote: >> On Apr 4, 2015, at 17:22, Adrian Chadd wrote: >>=20 >>> On 4 April 2015 at 16:33, Garrett Cooper = wrote: >>>> On Apr 4, 2015, at 16:28, Adrian Chadd wrote: >>>>=20 >>>>> Hi, >>>>>=20 >>>>> cc -O -pipe = -I/usr/home/adrian/work/freebsd/uverse/head/lib/libcrypt >>>>> -std=3Dgnu99 -Qunused-arguments >>>>> = -L/home/adrian/work/freebsd/uverse/obj/arm.arm/usr/home/adrian/work/freebs= d/uverse/head/tmp/usr/lib/private >>>>> -rpath /usr/lib/private -rpath /usr/lib/private -o crypt_tests >>>>> crypt_tests.o = /home/adrian/work/freebsd/uverse/obj/arm.arm/usr/home/adrian/work/freebsd/= uverse/head/lib/atf/libatf-c/libatf-c.so >>>>> -lcrypt >>>>> cc: error: no such file or directory: >>>>> = '/home/adrian/work/freebsd/uverse/obj/arm.arm/usr/home/adrian/work/freebsd= /uverse/head/lib/atf/libatf-c/libatf-c.so' >>>>>=20 >>>>> .. this happens for make universe on at least amd64, arm and = armeb. >>>>> I'm still waiting for the others to (not) complete. >>>>=20 >>>> Might be related to r280179 and r273449, but more info=92s needed. = Your src.conf for the build host and targets would be helpful. >>>> Thanks! >>>=20 >>> adrian@bruce:~/work/freebsd/uverse/head % cat /etc/make.conf >>> cat: /etc/make.conf: No such file or directory >>> adrian@bruce:~/work/freebsd/uverse/head % cat /etc/src.conf >>> cat: /etc/src.conf: No such file or directory >>>=20 >>> adrian@bruce:~/work/freebsd/uverse/head % env >>> MAKEOBJDIRPREFIX=3D/home/adrian/work/freebsd/uverse/obj make -j2 >>> universe JFLAG=3D-j8 >>>=20 >>> Thanks for replying so quickly! >>=20 >> -j2 and JFLAG probably isn=92t the best idea in the world. JFLAG = should be ok... >=20 > Why not? Based on personal experience, parallelizing on two levels just seems = like a bad idea, especially with recursive make and when dealing with = .PHONY targets. I haven=92t followed through the logic exactly to know = where it falls apart, but I know I always run into issues when I use -j = instead of JFLAG for specifying -j. Cheers, --Apple-Mail=_31F18AF5-0939-4C0A-ADFB-6879430D0460 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJVIINzAAoJEMZr5QU6S73eGvIH/1SDfc8HvTq3iEB8Wd8VOmSm ArvNzgsjH7dUzjCwO3gttdbhvT+3eYReLBztFO1Rfpick5HXWEBg40sjWIFN09eA cM+rt5qRMbzWUH99qZ2frTrbCgSdLEbQPlrONCR0R8FwZA/KT5/aw1lUWxKkSKYJ tViqZD1xlCExEuBFL+SMoHT1RB/neJlud4mCwurevvH33EaeOro8UUE/JyqOk7fd wB6VRA+73qpoCbodagyp8i+z8YPB58rcKzIqQVKzcCAvhSSIG+hAK8MtC5ef3bCo tncq1WmGKqcEv1G0jgMayCX4mS+cVp12cdjjy+e9EZ/umfUgEOzmHIsGkkRFAOs= =7H/E -----END PGP SIGNATURE----- --Apple-Mail=_31F18AF5-0939-4C0A-ADFB-6879430D0460-- From owner-freebsd-arch@FreeBSD.ORG Sun Apr 5 00:38:07 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C89A7DD7; Sun, 5 Apr 2015 00:38:07 +0000 (UTC) Received: from relay.mailchannels.net (tkt-001-i373.relay.mailchannels.net [174.136.5.175]) by mx1.freebsd.org (Postfix) with ESMTP id B1B5381B; Sun, 5 Apr 2015 00:38:06 +0000 (UTC) X-Sender-Id: duocircle|x-authuser|hippie Received: from smtp5.ore.mailhop.org (ip-10-229-11-165.us-west-2.compute.internal [10.229.11.165]) by relay.mailchannels.net (Postfix) with ESMTPA id 083AB1D0449; Sun, 5 Apr 2015 00:37:58 +0000 (UTC) X-Sender-Id: duocircle|x-authuser|hippie Received: from smtp5.ore.mailhop.org (smtp5.ore.mailhop.org [10.83.15.107]) (using TLSv1 with cipher DHE-RSA-AES256-SHA) by 0.0.0.0:2500 (trex/5.4.8); Sun, 05 Apr 2015 00:37:58 +0000 X-MC-Relay: Neutral X-MailChannels-SenderId: duocircle|x-authuser|hippie X-MailChannels-Auth-Id: duocircle X-MC-Loop-Signature: 1428194278099:2386893837 X-MC-Ingress-Time: 1428194278099 Received: from c-73-34-117-227.hsd1.co.comcast.net ([73.34.117.227] helo=ilsoft.org) by smtp5.ore.mailhop.org with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.82) (envelope-from ) id 1YeYZZ-0002uh-Cn; Sun, 05 Apr 2015 00:37:57 +0000 Received: from revolution.hippie.lan (revolution.hippie.lan [172.22.42.240]) by ilsoft.org (8.14.9/8.14.9) with ESMTP id t350buuM069323; Sat, 4 Apr 2015 18:37:56 -0600 (MDT) (envelope-from ian@freebsd.org) X-Mail-Handler: DuoCircle Outbound SMTP X-Originating-IP: 73.34.117.227 X-Report-Abuse-To: abuse@duocircle.com (see https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information for abuse reporting information) X-MHO-User: U2FsdGVkX1+9NLY0ZWFT4aZuyIbyDWO0 Message-ID: <1428194276.82583.144.camel@freebsd.org> Subject: Re: make universe failures? From: Ian Lepore To: Garrett Cooper Date: Sat, 04 Apr 2015 18:37:56 -0600 In-Reply-To: <43D75544-1458-4AE1-BCC1-DC5DB5AB72DA@gmail.com> References: <310573CF-443F-47BA-9F96-630107A4BD52@gmail.com> <43D75544-1458-4AE1-BCC1-DC5DB5AB72DA@gmail.com> Content-Type: text/plain; charset="iso-8859-13" X-Mailer: Evolution 3.12.10 FreeBSD GNOME Team Port Mime-Version: 1.0 X-AuthUser: hippie Content-Transfer-Encoding: quoted-printable Cc: Adrian Chadd , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Apr 2015 00:38:08 -0000 On Sat, 2015-04-04 at 17:36 -0700, Garrett Cooper wrote: > On Apr 4, 2015, at 17:33, Adrian Chadd wrote: >=20 > > On 4 April 2015 at 17:25, Garrett Cooper wrot= e: > >> On Apr 4, 2015, at 17:22, Adrian Chadd wrote: > >>=20 > >>> On 4 April 2015 at 16:33, Garrett Cooper wr= ote: > >>>> On Apr 4, 2015, at 16:28, Adrian Chadd wrote: > >>>>=20 > >>>>> Hi, > >>>>>=20 > >>>>> cc -O -pipe -I/usr/home/adrian/work/freebsd/uverse/head/lib/li= bcrypt > >>>>> -std=3Dgnu99 -Qunused-arguments > >>>>> -L/home/adrian/work/freebsd/uverse/obj/arm.arm/usr/home/adrian/wo= rk/freebsd/uverse/head/tmp/usr/lib/private > >>>>> -rpath /usr/lib/private -rpath /usr/lib/private -o crypt_tests > >>>>> crypt_tests.o /home/adrian/work/freebsd/uverse/obj/arm.arm/usr/ho= me/adrian/work/freebsd/uverse/head/lib/atf/libatf-c/libatf-c.so > >>>>> -lcrypt > >>>>> cc: error: no such file or directory: > >>>>> '/home/adrian/work/freebsd/uverse/obj/arm.arm/usr/home/adrian/wor= k/freebsd/uverse/head/lib/atf/libatf-c/libatf-c.so' > >>>>>=20 > >>>>> .. this happens for make universe on at least amd64, arm and arme= b. > >>>>> I'm still waiting for the others to (not) complete. > >>>>=20 > >>>> Might be related to r280179 and r273449, but more info=FFs needed.= Your src.conf for the build host and targets would be helpful. > >>>> Thanks! > >>>=20 > >>> adrian@bruce:~/work/freebsd/uverse/head % cat /etc/make.conf > >>> cat: /etc/make.conf: No such file or directory > >>> adrian@bruce:~/work/freebsd/uverse/head % cat /etc/src.conf > >>> cat: /etc/src.conf: No such file or directory > >>>=20 > >>> adrian@bruce:~/work/freebsd/uverse/head % env > >>> MAKEOBJDIRPREFIX=3D/home/adrian/work/freebsd/uverse/obj make -j2 > >>> universe JFLAG=3D-j8 > >>>=20 > >>> Thanks for replying so quickly! > >>=20 > >> -j2 and JFLAG probably isn=FFt the best idea in the world. JFLAG sho= uld be ok... > >=20 > > Why not? >=20 > Based on personal experience, parallelizing on two levels just seems li= ke a bad idea, especially with recursive make and when dealing with .PHON= Y targets. I haven=FFt followed through the logic exactly to know where i= t falls apart, but I know I always run into issues when I use -j instead = of JFLAG for specifying -j. > Cheers, Strange. I've never heard of JFLAG and have only ever used -j -- Ian From owner-freebsd-arch@FreeBSD.ORG Sun Apr 5 00:38:54 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 067EBE8A for ; Sun, 5 Apr 2015 00:38:54 +0000 (UTC) Received: from mail-ig0-x234.google.com (mail-ig0-x234.google.com [IPv6:2607:f8b0:4001:c05::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BA88C827 for ; Sun, 5 Apr 2015 00:38:53 +0000 (UTC) Received: by igbqf9 with SMTP id qf9so1329192igb.1 for ; Sat, 04 Apr 2015 17:38:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=u9gnRj3+fRVTsR5bT4+lGzM4hAcJnJGXgj3N7Gw5bQo=; b=pd+whKVVoafXOvqHhfzpEPSdAYH6a+CWbQzP45Qjyd2bOwobQEIJMqQ5oaYCj1Uu8u r9j2EIx+UytJ3esH1Q71lvRl5Em2TX6M6syXilXJufAqU75gZml/03a0xB8kIHizRlUR bn3y82lIle05GHRkQciqXDakoswzRb/Y62oJkI8S5+gl2FjnZ2nRLYuYHID1LXbtcpDX /C6JUyex4S5aDOwsy5Iv3jk+xhBepoDv8k6PL3zl4kWrTn034H3+vRtN6wwAtuPcr5Wq R88j8J4OnY3Rkjl1dGTjrtGbpmsKC00s3INMX/x1bph1wQ7I8021c7RbLK5SQ2JKCcCh QG3g== MIME-Version: 1.0 X-Received: by 10.50.62.112 with SMTP id x16mr7106737igr.32.1428194333181; Sat, 04 Apr 2015 17:38:53 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.36.17.194 with HTTP; Sat, 4 Apr 2015 17:38:53 -0700 (PDT) In-Reply-To: <43D75544-1458-4AE1-BCC1-DC5DB5AB72DA@gmail.com> References: <310573CF-443F-47BA-9F96-630107A4BD52@gmail.com> <43D75544-1458-4AE1-BCC1-DC5DB5AB72DA@gmail.com> Date: Sat, 4 Apr 2015 17:38:53 -0700 X-Google-Sender-Auth: B0VwTYdPTtu3XUxkMSA7hqdvsyg Message-ID: Subject: Re: make universe failures? From: Adrian Chadd To: Garrett Cooper Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Apr 2015 00:38:54 -0000 On 4 April 2015 at 17:36, Garrett Cooper wrote: > > Based on personal experience, parallelizing on two levels just seems like= a bad idea, especially with recursive make and when dealing with .PHONY ta= rgets. I haven=E2=80=99t followed through the logic exactly to know where i= t falls apart, but I know I always run into issues when I use -j instead of= JFLAG for specifying -j. Hm, it'd be nice to specify top level and per-build parallelism. Oh well! Thanks, -adrian From owner-freebsd-arch@FreeBSD.ORG Sun Apr 5 00:41:36 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6F81CF92; Sun, 5 Apr 2015 00:41:36 +0000 (UTC) Received: from mail-pa0-x236.google.com (mail-pa0-x236.google.com [IPv6:2607:f8b0:400e:c03::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 31CAA8E6; Sun, 5 Apr 2015 00:41:36 +0000 (UTC) Received: by pacyx8 with SMTP id yx8so3358715pac.1; Sat, 04 Apr 2015 17:41:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=Q7zUZdWaESmvJVe0OqWdxsx2LFcasj8hQ62LAa4D3ko=; b=zK/3WFleOV/nZpDBYJp3mSx1OK8gL1brksKv6wn31ofCxRsMgXeQ03smvDNyxRFFnQ Tr6ARoCFBo9uOhhcNLCaRM9atXk82tRprbmyzAOiRnoZtqs44KU3HcaFUXblBMh+ave6 SQlxBiQCLFXQ4nEq8osss9K4Nc5BGhbqc2OkrV9RMYWgDkU3kOT3YFFDoxWr6rLHm9vJ T6DjHHAEJyPzAqXBHMqUe9up9QHmZUPAx5oNB/uHittY0819Pe5WWZ5wd4QhwgJ5xAq0 n5h9P/8tZia4vgpYuX0xK5h1i3sA0tNCI7ufewKAx+nUuYPuV+ocQKGyM/DgQmcXgG81 e8ww== X-Received: by 10.70.131.227 with SMTP id op3mr16110036pdb.12.1428194495723; Sat, 04 Apr 2015 17:41:35 -0700 (PDT) Received: from ?IPv6:2601:8:ab80:7d6:981:fa61:f8b7:c531? ([2601:8:ab80:7d6:981:fa61:f8b7:c531]) by mx.google.com with ESMTPSA id eh4sm126688pbd.69.2015.04.04.17.41.34 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 04 Apr 2015 17:41:35 -0700 (PDT) Content-Type: multipart/signed; boundary="Apple-Mail=_4B532302-9B7C-4DBC-8858-587AA842B314"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: make universe failures? From: Garrett Cooper In-Reply-To: <1428194276.82583.144.camel@freebsd.org> Date: Sat, 4 Apr 2015 17:41:34 -0700 Message-Id: References: <310573CF-443F-47BA-9F96-630107A4BD52@gmail.com> <43D75544-1458-4AE1-BCC1-DC5DB5AB72DA@gmail.com> <1428194276.82583.144.camel@freebsd.org> To: Ian Lepore X-Mailer: Apple Mail (2.1878.6) Cc: Adrian Chadd , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Apr 2015 00:41:36 -0000 --Apple-Mail=_4B532302-9B7C-4DBC-8858-587AA842B314 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Apr 4, 2015, at 17:37, Ian Lepore wrote: > On Sat, 2015-04-04 at 17:36 -0700, Garrett Cooper wrote: >> On Apr 4, 2015, at 17:33, Adrian Chadd wrote: >>=20 >>> On 4 April 2015 at 17:25, Garrett Cooper = wrote: >>>> On Apr 4, 2015, at 17:22, Adrian Chadd wrote: >>>>=20 >>>>> On 4 April 2015 at 16:33, Garrett Cooper = wrote: >>>>>> On Apr 4, 2015, at 16:28, Adrian Chadd = wrote: >>>>>>=20 >>>>>>> Hi, >>>>>>>=20 >>>>>>> cc -O -pipe = -I/usr/home/adrian/work/freebsd/uverse/head/lib/libcrypt >>>>>>> -std=3Dgnu99 -Qunused-arguments >>>>>>> = -L/home/adrian/work/freebsd/uverse/obj/arm.arm/usr/home/adrian/work/freebs= d/uverse/head/tmp/usr/lib/private >>>>>>> -rpath /usr/lib/private -rpath /usr/lib/private -o crypt_tests >>>>>>> crypt_tests.o = /home/adrian/work/freebsd/uverse/obj/arm.arm/usr/home/adrian/work/freebsd/= uverse/head/lib/atf/libatf-c/libatf-c.so >>>>>>> -lcrypt >>>>>>> cc: error: no such file or directory: >>>>>>> = '/home/adrian/work/freebsd/uverse/obj/arm.arm/usr/home/adrian/work/freebsd= /uverse/head/lib/atf/libatf-c/libatf-c.so' >>>>>>>=20 >>>>>>> .. this happens for make universe on at least amd64, arm and = armeb. >>>>>>> I'm still waiting for the others to (not) complete. >>>>>>=20 >>>>>> Might be related to r280179 and r273449, but more info=B4s = needed. Your src.conf for the build host and targets would be helpful. >>>>>> Thanks! >>>>>=20 >>>>> adrian@bruce:~/work/freebsd/uverse/head % cat /etc/make.conf >>>>> cat: /etc/make.conf: No such file or directory >>>>> adrian@bruce:~/work/freebsd/uverse/head % cat /etc/src.conf >>>>> cat: /etc/src.conf: No such file or directory >>>>>=20 >>>>> adrian@bruce:~/work/freebsd/uverse/head % env >>>>> MAKEOBJDIRPREFIX=3D/home/adrian/work/freebsd/uverse/obj make -j2 >>>>> universe JFLAG=3D-j8 >>>>>=20 >>>>> Thanks for replying so quickly! >>>>=20 >>>> -j2 and JFLAG probably isn=B4t the best idea in the world. JFLAG = should be ok... >>>=20 >>> Why not? >>=20 >> Based on personal experience, parallelizing on two levels just seems = like a bad idea, especially with recursive make and when dealing with = .PHONY targets. I haven=B4t followed through the logic exactly to know = where it falls apart, but I know I always run into issues when I use -j = instead of JFLAG for specifying -j. >> Cheers, >=20 > Strange. I've never heard of JFLAG and have only ever used -j JFLAG=92s a make tinderbox/universe thing. -j would build multiple = universe targets in parallel with up to -j jobs each. Again, I don=92t know where and why things fall apart=85 it=92s just = that running multiple universes for me -j tends to get me in = trouble. I=92d have to go through and audit uses of ${.OBJDIR} to get a = better idea of why things fail... HTH! --Apple-Mail=_4B532302-9B7C-4DBC-8858-587AA842B314 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJVIIS+AAoJEMZr5QU6S73eigEH/2WAwFGFshLwckeVZJTBIUHM 5TKztEwriwsLHjF66SqoD/6XZ7K91SuKwRzMO/+NnUJLr6dn0Wklv+oOddouT5uZ CKDpUKsNNpHOibhHFTXp1R8EKI5F/LMJNlyh0DSPZ0efZKOVXcyd2ayh+nuzbmgr I1ElKLsQyarxu5UJqoGaT3zS6KlNaTRSz5STDI/cSC006xFI6mDV64Thceh1d10G E7AqzSHc9DEkbPN8FUKdrPVkkGp9LqJOgH3TEZ0/NoBu5Y/iROc3B5MQ3itNlBjG y4H7GC/85TlRdo7prOCS6KZi4TvVYfuzMLQByhO3gCalBKunTdQUoLnMXpHGKQc= =QHiu -----END PGP SIGNATURE----- --Apple-Mail=_4B532302-9B7C-4DBC-8858-587AA842B314-- From owner-freebsd-arch@FreeBSD.ORG Sun Apr 5 00:49:58 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8BBCF1CB; Sun, 5 Apr 2015 00:49:58 +0000 (UTC) Received: from relay.mailchannels.net (aso-006-i437.relay.mailchannels.net [23.91.64.118]) by mx1.freebsd.org (Postfix) with ESMTP id AE71A95A; Sun, 5 Apr 2015 00:49:57 +0000 (UTC) X-Sender-Id: duocircle|x-authuser|hippie Received: from smtp4.ore.mailhop.org (ip-10-229-11-165.us-west-2.compute.internal [10.229.11.165]) by relay.mailchannels.net (Postfix) with ESMTPA id 6159842C2; Sun, 5 Apr 2015 00:49:48 +0000 (UTC) X-Sender-Id: duocircle|x-authuser|hippie Received: from smtp4.ore.mailhop.org (smtp4.ore.mailhop.org [10.83.15.107]) (using TLSv1 with cipher DHE-RSA-AES256-SHA) by 0.0.0.0:2500 (trex/5.4.8); Sun, 05 Apr 2015 00:49:49 +0000 X-MC-Relay: Neutral X-MailChannels-SenderId: duocircle|x-authuser|hippie X-MailChannels-Auth-Id: duocircle X-MC-Loop-Signature: 1428194988475:4099112875 X-MC-Ingress-Time: 1428194988475 Received: from c-73-34-117-227.hsd1.co.comcast.net ([73.34.117.227] helo=ilsoft.org) by smtp4.ore.mailhop.org with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.82) (envelope-from ) id 1YeYl1-00008i-EP; Sun, 05 Apr 2015 00:49:47 +0000 Received: from revolution.hippie.lan (revolution.hippie.lan [172.22.42.240]) by ilsoft.org (8.14.9/8.14.9) with ESMTP id t350nkde069355; Sat, 4 Apr 2015 18:49:46 -0600 (MDT) (envelope-from ian@freebsd.org) X-Mail-Handler: DuoCircle Outbound SMTP X-Originating-IP: 73.34.117.227 X-Report-Abuse-To: abuse@duocircle.com (see https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information for abuse reporting information) X-MHO-User: U2FsdGVkX19ljUU3GS0qn4VTsKyGvj1g Message-ID: <1428194986.82583.149.camel@freebsd.org> Subject: Re: make universe failures? From: Ian Lepore To: Garrett Cooper Date: Sat, 04 Apr 2015 18:49:46 -0600 In-Reply-To: References: <310573CF-443F-47BA-9F96-630107A4BD52@gmail.com> <43D75544-1458-4AE1-BCC1-DC5DB5AB72DA@gmail.com> <1428194276.82583.144.camel@freebsd.org> Content-Type: text/plain; charset="iso-2022-jp" X-Mailer: Evolution 3.12.10 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-AuthUser: hippie Cc: Adrian Chadd , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Apr 2015 00:49:58 -0000 On Sat, 2015-04-04 at 17:41 -0700, Garrett Cooper wrote: > On Apr 4, 2015, at 17:37, Ian Lepore wrote: > > > On Sat, 2015-04-04 at 17:36 -0700, Garrett Cooper wrote: > >> On Apr 4, 2015, at 17:33, Adrian Chadd wrote: > >> > >>> On 4 April 2015 at 17:25, Garrett Cooper wrote: > >>>> On Apr 4, 2015, at 17:22, Adrian Chadd wrote: > >>>> > >>>>> On 4 April 2015 at 16:33, Garrett Cooper wrote: > >>>>>> On Apr 4, 2015, at 16:28, Adrian Chadd wrote: > >>>>>> > >>>>>>> Hi, > >>>>>>> > >>>>>>> cc -O -pipe -I/usr/home/adrian/work/freebsd/uverse/head/lib/libcrypt > >>>>>>> -std=gnu99 -Qunused-arguments > >>>>>>> -L/home/adrian/work/freebsd/uverse/obj/arm.arm/usr/home/adrian/work/freebsd/uverse/head/tmp/usr/lib/private > >>>>>>> -rpath /usr/lib/private -rpath /usr/lib/private -o crypt_tests > >>>>>>> crypt_tests.o /home/adrian/work/freebsd/uverse/obj/arm.arm/usr/home/adrian/work/freebsd/uverse/head/lib/atf/libatf-c/libatf-c.so > >>>>>>> -lcrypt > >>>>>>> cc: error: no such file or directory: > >>>>>>> '/home/adrian/work/freebsd/uverse/obj/arm.arm/usr/home/adrian/work/freebsd/uverse/head/lib/atf/libatf-c/libatf-c.so' > >>>>>>> > >>>>>>> .. this happens for make universe on at least amd64, arm and armeb. > >>>>>>> I'm still waiting for the others to (not) complete. > >>>>>> > >>>>>> Might be related to r280179 and r273449, but more info´s needed. Your src.conf for the build host and targets would be helpful. > >>>>>> Thanks! > >>>>> > >>>>> adrian@bruce:~/work/freebsd/uverse/head % cat /etc/make.conf > >>>>> cat: /etc/make.conf: No such file or directory > >>>>> adrian@bruce:~/work/freebsd/uverse/head % cat /etc/src.conf > >>>>> cat: /etc/src.conf: No such file or directory > >>>>> > >>>>> adrian@bruce:~/work/freebsd/uverse/head % env > >>>>> MAKEOBJDIRPREFIX=/home/adrian/work/freebsd/uverse/obj make -j2 > >>>>> universe JFLAG=-j8 > >>>>> > >>>>> Thanks for replying so quickly! > >>>> > >>>> -j2 and JFLAG probably isn´t the best idea in the world. JFLAG should be ok... > >>> > >>> Why not? > >> > >> Based on personal experience, parallelizing on two levels just seems like a bad idea, especially with recursive make and when dealing with .PHONY targets. I haven´t followed through the logic exactly to know where it falls apart, but I know I always run into issues when I use -j instead of JFLAG for specifying -j. > >> Cheers, > > > > Strange. I've never heard of JFLAG and have only ever used -j > > JFLAG’s a make tinderbox/universe thing. -j would build multiple universe targets in parallel with up to -j jobs each. > > Again, I don’t know where and why things fall apart… it’s just that running multiple universes for me -j tends to get me in trouble. I’d have to go through and audit uses of ${.OBJDIR} to get a better idea of why things fail... > > HTH! Maybe the j*j thing happened with fmake, I vaguely remember that. With bmake the -j value propagates in some magic way so that even though you're building arches in parallel, the total number of jobs is the -j value (which you can see pretty clearly in top, after a few minutes the load averages pretty much lock onto the -j value plus some fraction). I've never had any problems with objdir stuff building all the arches in parallel at any value for -j (but I typically never use numbers higher than 14 unless I'm testing build stuff). -- Ian From owner-freebsd-arch@FreeBSD.ORG Sun Apr 5 00:54:57 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B38044B4; Sun, 5 Apr 2015 00:54:57 +0000 (UTC) Received: from mail-pd0-x22d.google.com (mail-pd0-x22d.google.com [IPv6:2607:f8b0:400e:c02::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7563FA62; Sun, 5 Apr 2015 00:54:57 +0000 (UTC) Received: by pdbni2 with SMTP id ni2so3996525pdb.1; Sat, 04 Apr 2015 17:54:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=IGgvbP/dQLBcvS1XrYzV1BjyppNN0AaBjXmu2yttuk0=; b=jpyyfrpw7cUcurUYEpZfuXSSAwbpm4mRSU7b832XMpJcvaInDgAv/PAv5RN2ngKy5T MOxgBRTk6xMtu7JLynP9kMMUUKVPDVtXmq9j003ZT+TkI3mbFPVu+AUwxbb3JLv7Qz0W XKloLyIwPsXikkSHeaw85LVPBZkdHia6E1Rm0bglCyL2RgR//mgvJwUEMN//MrOogea5 rulhb+iK91RRz5k2jeYckiqllwCmcCIXtKvzzwzuZwUYbuYXVSKDrD2CHMl1WwNv8H+q b9c94joaPLNoQ5vbaHvwA6S8ZyENhYyLjuuR9WsUX1iNhkQF9lEFfwngAjXapEwS0ymk fybw== X-Received: by 10.66.241.36 with SMTP id wf4mr16101940pac.8.1428195296925; Sat, 04 Apr 2015 17:54:56 -0700 (PDT) Received: from ?IPv6:2601:8:ab80:7d6:981:fa61:f8b7:c531? ([2601:8:ab80:7d6:981:fa61:f8b7:c531]) by mx.google.com with ESMTPSA id rd7sm139696pdb.85.2015.04.04.17.54.55 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 04 Apr 2015 17:54:56 -0700 (PDT) Content-Type: multipart/signed; boundary="Apple-Mail=_0EF9CC41-D27C-401D-97ED-A47CBDF5601D"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: make universe failures? From: Garrett Cooper In-Reply-To: <1428194986.82583.149.camel@freebsd.org> Date: Sat, 4 Apr 2015 17:54:54 -0700 Message-Id: <97DF2A69-4E1E-4F0D-95BC-7C97ECC0D3B5@gmail.com> References: <310573CF-443F-47BA-9F96-630107A4BD52@gmail.com> <43D75544-1458-4AE1-BCC1-DC5DB5AB72DA@gmail.com> <1428194276.82583.144.camel@freebsd.org> <1428194986.82583.149.camel@freebsd.org> To: Ian Lepore X-Mailer: Apple Mail (2.1878.6) Cc: Adrian Chadd , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Apr 2015 00:54:57 -0000 --Apple-Mail=_0EF9CC41-D27C-401D-97ED-A47CBDF5601D Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=iso-2022-jp On Apr 4, 2015, at 17:49, Ian Lepore wrote: … > Maybe the j*j thing happened with fmake, I vaguely remember that. With > bmake the -j value propagates in some magic way so that even though > you're building arches in parallel, the total number of jobs is the -j > value (which you can see pretty clearly in top, after a few minutes the > load averages pretty much lock onto the -j value plus some fraction). > I've never had any problems with objdir stuff building all the arches in > parallel at any value for -j (but I typically never use numbers higher > than 14 unless I'm testing build stuff). That’s a part of it too. I use -j12/-j16 with JFLAG. --Apple-Mail=_0EF9CC41-D27C-401D-97ED-A47CBDF5601D Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJVIIffAAoJEMZr5QU6S73eTgoIALQH1gKHfm9VTd7lrEDYzOcA KyXNgMgxtv9fR15s4CQUOw/Gy2VWsJXQiu87zEVoSMRhW8XCOwBDq1UAd38MxG1F m5teEgsEvvz+YMgVHO+k10LG1Sm/h9Xd5QJjFK5w9QAjnpX/zfxWsZ4jry1HU9Qf iH7Sk3mu+CJ9Q7bPV5g6h5sBSEwtEVNOGBCPsDMKJNT6tm6q4/oz+KWD5YRIZXzN mCiErG72JDZqcHBFp/WwzF30l2s9EZur1fOrqkTPC++qs5S/F3y/VUDGrV5FDuYK wChhWyYNWrLvRfihfzzch1zdH4s0jhr9u9r9mdcgFSJECtWV1yDmkNuVledgGW8= =jiSh -----END PGP SIGNATURE----- --Apple-Mail=_0EF9CC41-D27C-401D-97ED-A47CBDF5601D-- From owner-freebsd-arch@FreeBSD.ORG Sun Apr 5 02:29:06 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 43FD48C4; Sun, 5 Apr 2015 02:29:06 +0000 (UTC) Received: from mail-ie0-x22e.google.com (mail-ie0-x22e.google.com [IPv6:2607:f8b0:4001:c03::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 043B77CF; Sun, 5 Apr 2015 02:29:06 +0000 (UTC) Received: by iebrs15 with SMTP id rs15so2152789ieb.3; Sat, 04 Apr 2015 19:29:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=2tFhhpZ8fdalLhmQnTXX9bjwhu53COhUO7mcggobyz4=; b=VmBi5KahL2HLqz61TRampGycpPsieZIXaKwWFkbtLOxNSb6x5EVaYRi7Ckb9ln7LaE 68PnbbEqClVYCZUkOAqjMNlvPTJn+ChzLakD7n0+Zv0tnaiNykxLOxmfnvxUm4KtF04E CYmSUFZXYPwD2zOdlfeyOjF8ou7fFzFg8JJyjtGb7+eXPPg4mFGEuucUy18ctQ+W5THz HyA06szksK1lgzqw0PjjVEzYrPXi+nr6diaEE47Onm6vTSTclA4VS6DQ4rSNXg/wHXFn 5+B2T2NUlujJNctyn9Zhz12C6nkQpoYmz9etf8lQ79Ln7clAawiSfi8D7U9prftGiXBi boXw== MIME-Version: 1.0 X-Received: by 10.107.136.25 with SMTP id k25mr12956987iod.88.1428200945453; Sat, 04 Apr 2015 19:29:05 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.36.17.194 with HTTP; Sat, 4 Apr 2015 19:29:05 -0700 (PDT) In-Reply-To: <97DF2A69-4E1E-4F0D-95BC-7C97ECC0D3B5@gmail.com> References: <310573CF-443F-47BA-9F96-630107A4BD52@gmail.com> <43D75544-1458-4AE1-BCC1-DC5DB5AB72DA@gmail.com> <1428194276.82583.144.camel@freebsd.org> <1428194986.82583.149.camel@freebsd.org> <97DF2A69-4E1E-4F0D-95BC-7C97ECC0D3B5@gmail.com> Date: Sat, 4 Apr 2015 19:29:05 -0700 X-Google-Sender-Auth: G0vuCm3cYpp0yolkqteYhvSxyGM Message-ID: Subject: Re: make universe failures? From: Adrian Chadd To: Garrett Cooper Content-Type: text/plain; charset=UTF-8 Cc: Ian Lepore , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Apr 2015 02:29:06 -0000 Still getting it with make -j16 universe -a From owner-freebsd-arch@FreeBSD.ORG Sun Apr 5 03:28:29 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 728995D9; Sun, 5 Apr 2015 03:28:29 +0000 (UTC) Received: from mail-pd0-x230.google.com (mail-pd0-x230.google.com [IPv6:2607:f8b0:400e:c02::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 312F6E21; Sun, 5 Apr 2015 03:28:29 +0000 (UTC) Received: by pddn5 with SMTP id n5so6730867pdd.2; Sat, 04 Apr 2015 20:28:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=abecPGoOPMeqdaULd+PpEf2h/HaffzInJ5YebOVBmF0=; b=hfkXv/65DTy0mWZ3LrrEd+nrkM4ZWH7hY41r4qcpLblbvB1vKB2OgcKAjyeXj9lJHv EEeqPMbgaxhpbBwIIH5roWwjwwHxs4kJQhwNqghtnVjZ48N5ng2QnUd7svtXymFMX9At dMHOr7qV6+4dIY4WeykKJ2nrMchAjuK5skl0KyTazVADhbtPGlg/n/K54+DT9diFq7m7 M4N0mK/ganD9NxziDV/mOFlyOHnggMu3jQBAYIhQJeffA9dCPaXs6hOJINW4CMPMh5jb v1El2eh5RvCg6cJjXa+zf1D6s30wpBBQIt15lzz2zHS9Bo2UxNEOh6+bdDuy7NfYok05 Ci5Q== X-Received: by 10.68.177.36 with SMTP id cn4mr16705339pbc.114.1428204508215; Sat, 04 Apr 2015 20:28:28 -0700 (PDT) Received: from [192.168.20.5] (c-98-247-240-204.hsd1.wa.comcast.net. [98.247.240.204]) by mx.google.com with ESMTPSA id xj1sm343598pbb.92.2015.04.04.20.28.27 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 04 Apr 2015 20:28:27 -0700 (PDT) Content-Type: multipart/signed; boundary="Apple-Mail=_FE61AF58-A1E9-4E4B-9956-B23666BC8F09"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: make universe failures? From: Garrett Cooper In-Reply-To: Date: Sat, 4 Apr 2015 20:28:27 -0700 Message-Id: <6D68B888-AFCC-4BE5-974C-BE616582C8AC@gmail.com> References: <310573CF-443F-47BA-9F96-630107A4BD52@gmail.com> <43D75544-1458-4AE1-BCC1-DC5DB5AB72DA@gmail.com> <1428194276.82583.144.camel@freebsd.org> <1428194986.82583.149.camel@freebsd.org> <97DF2A69-4E1E-4F0D-95BC-7C97ECC0D3B5@gmail.com> To: Adrian Chadd X-Mailer: Apple Mail (2.1878.6) Cc: Ian Lepore , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Apr 2015 03:28:29 -0000 --Apple-Mail=_FE61AF58-A1E9-4E4B-9956-B23666BC8F09 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On Apr 4, 2015, at 19:29, Adrian Chadd wrote: > Still getting it with make -j16 universe If it=92s the same issues, it might be the fact that the beforementioned = commits =93just worked=94 with lower -j values. I=92d need to double = check to be sure though=85 Please post -j bugs somewhere though. Are you trying to replicate them = on ref10-amd64.freebsd.org? --Apple-Mail=_FE61AF58-A1E9-4E4B-9956-B23666BC8F09 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJVIKvbAAoJEMZr5QU6S73eOSIIAJcM8n5GwufPm/xKvYCowTAe bpgLYtTrSkLdjQpkX9j/Gh7h3hwtO3qc8wzKAbfPHXJYLGtYioOkRjy4YPBnBGci T/hWglVMX4IQGgBTZ09Ue/6AzF2ToFUHskrXCVpwkRSoJEKd/4mL0btMSXPrs2po D0s7bbhP7J/q6tQNbzIY5fGDWZ1tbxem7Z23U639ExNzO6gM8AsZVNsuZKkDKIrT JpETSbks4UiOAfQ7wLU7LdPT3+SgpNzGKU7Fs29n/Kmv5uCrPpgLT44nEiS2hDIF ThhKwDVz0eSK5ilcPqeInT1hZTP6mpbsYik8zJN5h2rFnWVCSrF+7EQaTwGS3F8= =9SUB -----END PGP SIGNATURE----- --Apple-Mail=_FE61AF58-A1E9-4E4B-9956-B23666BC8F09-- From owner-freebsd-arch@FreeBSD.ORG Sun Apr 5 04:19:02 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 55A55D77; Sun, 5 Apr 2015 04:19:02 +0000 (UTC) Received: from mail-ig0-x231.google.com (mail-ig0-x231.google.com [IPv6:2607:f8b0:4001:c05::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0E7A52E6; Sun, 5 Apr 2015 04:19:02 +0000 (UTC) Received: by igblo3 with SMTP id lo3so2578712igb.0; Sat, 04 Apr 2015 21:19:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=9btDvfzrqtjnOC2W3mU+tcbo7bHCafm5m7BDHea5Tno=; b=HT/WojFM2IRxa4//OEs1o1Qml4lNWV+QOnh1KKGJCzJC1vHz7byIOinBAk2gm1RAT4 LebJJQcmis4f+7vpMWdhsT3v06xyAnRG/KAac/A6cs5mjJ5+Y/HLao2wK79HCahsJQYK g44p3wObGs/cc/CJBQWby5/L67sIzBCVi0M2fejIR64g2HiHfoalmIoDNyDlm2fDkyIk RAGQ3YeLOtgIjxd4Zs1SfhWDurMm+e57G43rykqtKT70jzLFt9yt5XtNpHgngfrcwFdX bW6abKFI7XyuiyRN8mWNAaycz1C3KTBsj5dUDwHEQT/l7iAGnxXibWy0l1KDem0Z0IyV kvPw== MIME-Version: 1.0 X-Received: by 10.42.137.202 with SMTP id z10mr2445727ict.37.1428207541583; Sat, 04 Apr 2015 21:19:01 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.36.17.194 with HTTP; Sat, 4 Apr 2015 21:19:01 -0700 (PDT) In-Reply-To: <6D68B888-AFCC-4BE5-974C-BE616582C8AC@gmail.com> References: <310573CF-443F-47BA-9F96-630107A4BD52@gmail.com> <43D75544-1458-4AE1-BCC1-DC5DB5AB72DA@gmail.com> <1428194276.82583.144.camel@freebsd.org> <1428194986.82583.149.camel@freebsd.org> <97DF2A69-4E1E-4F0D-95BC-7C97ECC0D3B5@gmail.com> <6D68B888-AFCC-4BE5-974C-BE616582C8AC@gmail.com> Date: Sat, 4 Apr 2015 21:19:01 -0700 X-Google-Sender-Auth: 5gjjgYQJYQcwsxKawQ136vWmir4 Message-ID: Subject: Re: make universe failures? From: Adrian Chadd To: Garrett Cooper Content-Type: text/plain; charset=UTF-8 Cc: Ian Lepore , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Apr 2015 04:19:02 -0000 Hi, Nope, I have some dual-socket, 8-core sandybridge Dell boxes that I am testing on. -adrian From owner-freebsd-arch@FreeBSD.ORG Mon Apr 6 07:01:20 2015 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A644F116 for ; Mon, 6 Apr 2015 07:01:20 +0000 (UTC) Received: from mail-ob0-x242.google.com (mail-ob0-x242.google.com [IPv6:2607:f8b0:4003:c01::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 69785D0F for ; Mon, 6 Apr 2015 07:01:20 +0000 (UTC) Received: by obbgq1 with SMTP id gq1so1883680obb.2 for ; Mon, 06 Apr 2015 00:01:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=jx/YVUUyTXK5oityqqmAnrHjRsb+muhK6kdLa9rJWtI=; b=hS0AsxnbxdnLfM/XGw45O5hpIzvKjfOJ275BDE+/YHt+M/QWfNJDzsSRboLNAi3v++ r0nZoP91IsSdvmMWjtzU0uXd96i0bXyORZy3Kd3QtpAvXAb6Dj8bGcV0gT+yDruku+nP nHQvygLsXAMfOgjAs/Qg0lRzIwQNtOhlOHhc++6uG5KeVq5oj7Goq4m1ZbWQMPx1VEH3 fpfng/+GrdsQhCrVstVwj8aPiT0GRF9lVMx+D4RWGjM+rPluryVJs1kzQ+vrhDHdpMZ1 6kzS7IbITrvCL0tsj8GCM2EmmuX9jRHoA9RBcdUiIUcUtO98LLdiQTeqPwVeKhtKy5gL jRig== MIME-Version: 1.0 X-Received: by 10.60.223.228 with SMTP id qx4mr4527826oec.24.1428303679825; Mon, 06 Apr 2015 00:01:19 -0700 (PDT) Received: by 10.76.75.130 with HTTP; Mon, 6 Apr 2015 00:01:19 -0700 (PDT) Received: by 10.76.75.130 with HTTP; Mon, 6 Apr 2015 00:01:19 -0700 (PDT) Date: Mon, 6 Apr 2015 00:01:19 -0700 Message-ID: Subject: From: Tyrone Pattison To: freebsd-arch@FreeBSD.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Apr 2015 07:01:20 -0000 Tyrone0066@gmail. Com From owner-freebsd-arch@FreeBSD.ORG Mon Apr 6 07:21:30 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 51FC2D69 for ; Mon, 6 Apr 2015 07:21:30 +0000 (UTC) Received: from mail-ie0-x231.google.com (mail-ie0-x231.google.com [IPv6:2607:f8b0:4001:c03::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1AF81F12 for ; Mon, 6 Apr 2015 07:21:30 +0000 (UTC) Received: by iedfl3 with SMTP id fl3so18053894ied.1 for ; Mon, 06 Apr 2015 00:21:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:message-id:subject:from:to:content-type; bh=Ygj1Iwr3IGd094X+E0yNJm3zpCcLcfcXCVBXRagVHj0=; b=Or/06cSWt79TIj/p4eP7bvmxa0nzRshxOzhuXjRLnV1hoj3qbB10kXlWi7QTkHQ53+ k68WvlI+n983NHMg/0c7/Pvd3LcD4HqyME9ux0ikucKlQkWjU2TgHwEeW2MowCEiuYah ffS1i6J8us9fN8KIPK90hBQEpLRrBeF7cMbBhgniw+8LyBm1Jdk46S75iRmKaF0k+1MY UHwR5vYju+qxfHFXDiZytFDZkR0CUrJ6YzYYnGwFFdmebRJCi3OV2SherDkorToVJyvf ElT4FFzkoagjFVWSaDx+6QeIgDMFC1E3xQ2EA5NM9FQt1UuRaTkw4ILxSjU1e7/o7JFe l1Yw== MIME-Version: 1.0 X-Received: by 10.50.107.7 with SMTP id gy7mr21131544igb.49.1428304889582; Mon, 06 Apr 2015 00:21:29 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.36.17.194 with HTTP; Mon, 6 Apr 2015 00:21:29 -0700 (PDT) Date: Mon, 6 Apr 2015 00:21:29 -0700 X-Google-Sender-Auth: cAupZCPDefq6O_xhicWASYZkSVI Message-ID: Subject: x86: finding interrupts that aren't being accounted for? From: Adrian Chadd To: "freebsd-arch@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Apr 2015 07:21:30 -0000 Hi, I have an .. odd problem on a Lenovo X230. I just threw in a very old wifi card (Intel 3945) into the expresscard (pcie) slot. Now, we don't have any pcie-hp support in -HEAD just yet, but i wasn't expecting the system to crawl to a halt. When I unplug it, everything returns to normal. Other cards don't do this. So, I figured it may be interrupt spam - but vmstat -ia shows no interrupts going crazy. pmcstat -S CPU_CLK_UNHALTED_CORE -T -w 5 doesn't register anything either - only a handful of background samples. However, /counter/ mode pmc tells a different story - pmcstat -s CPU_CLK_UNHALTED_CORE -w 1 shows all four cores going at 110% when the card is inserted, with brief periods of idle. Once I remove the card, the counters go back down to zero. My working theory is: something is chewing CPU and it's likely interrupts, but if it is, it's something far, far earlier than the x86 interrupt C code, which counts interrupts and spurious events. So - has anyone diagnosed this stuff on FreeBSD/x86 before? I was kind of hoping we'd at least get accurate statistics about spurious interrupts, and if we don't, I'd like to understand why. Thanks! -adrian From owner-freebsd-arch@FreeBSD.ORG Mon Apr 6 19:19:25 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 50240AC9; Mon, 6 Apr 2015 19:19:25 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 28C767E3; Mon, 6 Apr 2015 19:19:25 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-54-116-245.nwrknj.fios.verizon.net [173.54.116.245]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id A4EFBB945; Mon, 6 Apr 2015 15:19:23 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Subject: Re: x86: finding interrupts that aren't being accounted for? Date: Mon, 06 Apr 2015 15:18:23 -0400 Message-ID: <1858440.dQ4AvDcZf7@ralph.baldwin.cx> User-Agent: KMail/4.14.2 (FreeBSD/10.1-STABLE; KDE/4.14.2; amd64; ; ) In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 06 Apr 2015 15:19:23 -0400 (EDT) Cc: Adrian Chadd X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Apr 2015 19:19:25 -0000 On Monday, April 06, 2015 12:21:29 AM Adrian Chadd wrote: > Hi, > > I have an .. odd problem on a Lenovo X230. > > I just threw in a very old wifi card (Intel 3945) into the expresscard > (pcie) slot. Now, we don't have any pcie-hp support in -HEAD just yet, > but i wasn't expecting the system to crawl to a halt. > > When I unplug it, everything returns to normal. > > Other cards don't do this. > > So, I figured it may be interrupt spam - but vmstat -ia shows no > interrupts going crazy. > > pmcstat -S CPU_CLK_UNHALTED_CORE -T -w 5 doesn't register anything > either - only a handful of background samples. > > However, /counter/ mode pmc tells a different story - pmcstat -s > CPU_CLK_UNHALTED_CORE -w 1 shows all four cores going at 110% when the > card is inserted, with brief periods of idle. Once I remove the card, > the counters go back down to zero. > > My working theory is: something is chewing CPU and it's likely > interrupts, but if it is, it's something far, far earlier than the x86 > interrupt C code, which counts interrupts and spurious events. > > So - has anyone diagnosed this stuff on FreeBSD/x86 before? I was kind > of hoping we'd at least get accurate statistics about spurious > interrupts, and if we don't, I'd like to understand why. > > Thanks! SMM? Perhaps SMM doesn't hide itself from PMC counters (but it can hide itself from samples). If it is SMM there's not really anything you can do about it. Try getting a KTR_SCHED trace and looking at it in schedgraph. When I've seen SMM isuses in the past it shows up as hole in the graph where nothing happens in the system. In your case you could perhaps be getting PCI errors that are triggering the SMM handler. Perhaps compare pciconf -le before and after to see if there are any changes. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Mon Apr 6 19:19:25 2015 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 92EB1ACC; Mon, 6 Apr 2015 19:19:25 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 68AF47E5; Mon, 6 Apr 2015 19:19:25 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-54-116-245.nwrknj.fios.verizon.net [173.54.116.245]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 4C8A5B913; Mon, 6 Apr 2015 15:19:24 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Subject: Re: RFC: chgsbsize doesn't handle -ve sbsize correctly Date: Mon, 06 Apr 2015 15:11:47 -0400 Message-ID: <476184366.nOLsX7PO1g@ralph.baldwin.cx> User-Agent: KMail/4.14.2 (FreeBSD/10.1-STABLE; KDE/4.14.2; amd64; ; ) In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 06 Apr 2015 15:19:24 -0400 (EDT) Cc: "arch@FreeBSD.org" , Anuranjan Shukla , Simon Gerraty X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Apr 2015 19:19:25 -0000 On Friday, April 03, 2015 07:38:59 AM Anuranjan Shukla wrote: > Hi, > In struct uidinfo{}, ui_sbsize member is an int. In a system with large > number of sockets owned by a user it's possible for ui_sbsize to roll over > to a negative value. When that happens, further calls to _increase_ sbsize > don't detect this condition as they should per design. But when the > sockets start shutting down this condition would be detected and you'll > see the console prints ("negative sbsize for uid =") happen. As a worst > case the console being flooded with these prints can create a DoS scenario > (specially on single CPU systems). > > Regardless of the end result, the code itself needs to be fixed for > correctness. There are two things to note: > - Int doesn't seem to be the correct type for ui_sbsize. Should be a u_int > atleast. > - Since there's no real prevention of the overflow as it happens, the > warning print isn't helpful and should be a DEBUG level log at best. If we > change ui_sbsize to be a u_int, the negative check itself can go. > > > int > chgsbsize(uip, hiwat, to, max) > { > int diff; > > diff = to - *hiwat; > if (diff > 0) { > if (atomic_fetchadd_long(&uip->ui_sbsize, (long)diff) + > diff > max) { > <======= -ve ui_sbsize goes undetected and we'll pass the above > check > atomic_subtract_long(&uip->ui_sbsize, (long)diff); > return (0); > } > } else { > atomic_add_long(&uip->ui_sbsize, (long)diff); > if (uip->ui_sbsize < 0) > printf("negative sbsize for uid = %d\n", > uip->ui_uid); > > <==== Now we'll detect, far too late, that sbsize went -ve Note that ui_sbsize is long, not int. That doesn't help you on 32-bit platforms, but you are also stuck with 32 bits since you typically don't have 64-bit atomics on 32-bit platforms. Making it unsigned just means you can't detect overflow anymore. Instead I think it should be fixed to detect the overflow when increasing the size and fail then. Several nearby functions would need the same fix btw: Index: kern_resource.c =================================================================== --- kern_resource.c (revision 281146) +++ kern_resource.c (working copy) @@ -1380,17 +1380,18 @@ chgproccnt(struct uidinfo *uip, int diff, rlim_t m int chgsbsize(struct uidinfo *uip, u_int *hiwat, u_int to, rlim_t max) { + long new; int diff; diff = to - *hiwat; + new = atomic_fetchadd_long(&uip->ui_sbsize, (long)diff) + diff; if (diff > 0) { - if (atomic_fetchadd_long(&uip->ui_sbsize, (long)diff) + diff > max) { + if (new < 0 || new > max) { atomic_subtract_long(&uip->ui_sbsize, (long)diff); return (0); } } else { - atomic_add_long(&uip->ui_sbsize, (long)diff); - if (uip->ui_sbsize < 0) + if (new < 0) printf("negative sbsize for uid = %d\n", uip->ui_uid); } *hiwat = to; -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Mon Apr 6 20:01:54 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EFAD69CB for ; Mon, 6 Apr 2015 20:01:54 +0000 (UTC) Received: from st11p00mm-asmtp003.mac.com (st11p00mm-asmtpout003.mac.com [17.172.81.2]) (using TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C3299CFE for ; Mon, 6 Apr 2015 20:01:54 +0000 (UTC) Received: from akita.localnet (209-23-203-214-Illinois.hfc.comcastbusiness.net [209.23.203.214]) by st11p00mm-asmtp003.mac.com (Oracle Communications Messaging Server 7.0.5.35.0 64bit (built Dec 4 2014)) with ESMTPSA id <0NME0015ZFIX1E20@st11p00mm-asmtp003.mac.com> for freebsd-arch@freebsd.org; Mon, 06 Apr 2015 19:01:47 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.13.68,1.0.33,0.0.0000 definitions=2015-04-06_04:2015-04-06,2015-04-06,1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=4 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1412110000 definitions=main-1504060176 From: Rui Paulo To: freebsd-arch@freebsd.org Subject: Re: x86: finding interrupts that aren't being accounted for? Date: Mon, 06 Apr 2015 12:01:45 -0700 Message-id: <8818748.ffDquhmdWF@akita> User-Agent: KMail/4.14.3 (FreeBSD/11.0-CURRENT; KDE/4.14.3; amd64; ; ) In-reply-to: References: MIME-version: 1.0 Content-transfer-encoding: 7Bit Content-type: text/plain; charset=us-ascii X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Apr 2015 20:01:55 -0000 On Monday 06 April 2015 00:21:29 Adrian Chadd wrote: > Hi, > > I have an .. odd problem on a Lenovo X230. > > I just threw in a very old wifi card (Intel 3945) into the expresscard > (pcie) slot. Now, we don't have any pcie-hp support in -HEAD just yet, > but i wasn't expecting the system to crawl to a halt. > > When I unplug it, everything returns to normal. > > Other cards don't do this. > > So, I figured it may be interrupt spam - but vmstat -ia shows no > interrupts going crazy. > > pmcstat -S CPU_CLK_UNHALTED_CORE -T -w 5 doesn't register anything > either - only a handful of background samples. > > However, /counter/ mode pmc tells a different story - pmcstat -s > CPU_CLK_UNHALTED_CORE -w 1 shows all four cores going at 110% when the > card is inserted, with brief periods of idle. Once I remove the card, > the counters go back down to zero. > > My working theory is: something is chewing CPU and it's likely > interrupts, but if it is, it's something far, far earlier than the x86 > interrupt C code, which counts interrupts and spurious events. > > So - has anyone diagnosed this stuff on FreeBSD/x86 before? I was kind > of hoping we'd at least get accurate statistics about spurious > interrupts, and if we don't, I'd like to understand why. If the cores are being used, you should be getting some samples as to where the PC is. pmcstat doesn't show that? How about DTrace? -- Rui Paulo From owner-freebsd-arch@FreeBSD.ORG Mon Apr 6 20:27:11 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1783C64D; Mon, 6 Apr 2015 20:27:11 +0000 (UTC) Received: from na01-bn1-obe.outbound.protection.outlook.com (mail-bn1bn0108.outbound.protection.outlook.com [157.56.110.108]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 73E39FC6; Mon, 6 Apr 2015 20:27:09 +0000 (UTC) Received: from CO2PR0501MB1063.namprd05.prod.outlook.com (25.160.7.20) by CY1PR0501MB1547.namprd05.prod.outlook.com (25.161.161.145) with Microsoft SMTP Server (TLS) id 15.1.130.23; Mon, 6 Apr 2015 20:27:03 +0000 Received: from CO2PR0501MB1063.namprd05.prod.outlook.com ([25.160.7.20]) by CO2PR0501MB1063.namprd05.prod.outlook.com ([25.160.7.20]) with mapi id 15.01.0125.002; Mon, 6 Apr 2015 20:27:02 +0000 From: Anuranjan Shukla To: John Baldwin , "freebsd-arch@freebsd.org" Subject: Re: RFC: chgsbsize doesn't handle -ve sbsize correctly Thread-Topic: RFC: chgsbsize doesn't handle -ve sbsize correctly Thread-Index: AQHQbeE/f9SgtMJNR02J2FpJ+QyWpp1AX0uA//+fsAA= Date: Mon, 6 Apr 2015 20:27:02 +0000 Message-ID: References: <476184366.nOLsX7PO1g@ralph.baldwin.cx> In-Reply-To: <476184366.nOLsX7PO1g@ralph.baldwin.cx> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.8.150116 x-ms-exchange-messagesentrepresentingtype: 1 x-originating-ip: [66.129.239.12] authentication-results: freebsd.org; dkim=none (message not signed) header.d=none; x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:CY1PR0501MB1547; x-microsoft-antispam-prvs: x-forefront-antispam-report: BMV:1; SFV:NSPM; SFS:(10019020)(6009001)(24454002)(51704005)(479174004)(377454003)(2501003)(19580405001)(2900100001)(46102003)(50986999)(76176999)(40100003)(54356999)(2950100001)(83506001)(450100001)(19580395003)(106116001)(102836002)(92566002)(77156002)(62966003)(86362001)(66066001)(122556002)(99286002)(36756003)(87936001)(2656002); DIR:OUT; SFP:1102; SCL:1; SRVR:CY1PR0501MB1547; H:CO2PR0501MB1063.namprd05.prod.outlook.com; FPR:; SPF:None; MLV:sfv; LANG:en; x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(601004)(5005006)(5002010); SRVR:CY1PR0501MB1547; BCL:0; PCL:0; RULEID:; SRVR:CY1PR0501MB1547; x-forefront-prvs: 0538A71254 Content-Type: text/plain; charset="us-ascii" Content-ID: <960BBF627E9C8A42BCA8FB358DAB6908@namprd05.prod.outlook.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: juniper.net X-MS-Exchange-CrossTenant-originalarrivaltime: 06 Apr 2015 20:27:02.2870 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: bea78b3c-4cdb-4130-854a-1d193232e5f4 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR0501MB1547 Cc: "arch@FreeBSD.org" , Simon Gerraty X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Apr 2015 20:27:11 -0000 Hi John, Correct, I'd miswrote int. I'm curious as to what point there is in keeping sbsize signed in the first place. If we make it unsigned long, on 32 bit also you get 'more', and overflow could be detected someway like 'if (diff > 0 && new < ui_sbsize)'. I don't see much of a value with the signed sbsize. Any particular reason you would favor otherwise? Thanks for get back on this. Anu On 4/6/15, 12:11 PM, "John Baldwin" wrote: >On Friday, April 03, 2015 07:38:59 AM Anuranjan Shukla wrote: >> Hi, >> In struct uidinfo{}, ui_sbsize member is an int. In a system with large >> number of sockets owned by a user it's possible for ui_sbsize to roll >>over >> to a negative value. When that happens, further calls to _increase_ >>sbsize >> don't detect this condition as they should per design. But when the >> sockets start shutting down this condition would be detected and you'll >> see the console prints ("negative sbsize for uid =3D") happen. As a wors= t >> case the console being flooded with these prints can create a DoS >>scenario >> (specially on single CPU systems). >>=20 >> Regardless of the end result, the code itself needs to be fixed for >> correctness. There are two things to note: >> - Int doesn't seem to be the correct type for ui_sbsize. Should be a >>u_int >> atleast. >> - Since there's no real prevention of the overflow as it happens, the >> warning print isn't helpful and should be a DEBUG level log at best. If >>we >> change ui_sbsize to be a u_int, the negative check itself can go. >>=20 >>=20 >> int >> chgsbsize(uip, hiwat, to, max) >> { >> int diff; >>=20 >> diff =3D to - *hiwat; >> if (diff > 0) { >> if (atomic_fetchadd_long(&uip->ui_sbsize, (long)diff) + >> diff > max) { >> <=3D=3D=3D=3D=3D=3D=3D -ve ui_sbsize goes undetected and we'll p= ass the above >> check=20 >> atomic_subtract_long(&uip->ui_sbsize, >>(long)diff); >> return (0); >> } >> } else { >> atomic_add_long(&uip->ui_sbsize, (long)diff); >> if (uip->ui_sbsize < 0) >> printf("negative sbsize for uid =3D %d\n", >> uip->ui_uid); >>=20 >> <=3D=3D=3D=3D Now we'll detect, far too late, that sbsiz= e went >>-ve > >Note that ui_sbsize is long, not int. That doesn't help you on 32-bit >platforms, but you are also stuck with 32 bits since you typically don't >have 64-bit atomics on 32-bit platforms. > >Making it unsigned just means you can't detect overflow anymore. Instead >I think it should be fixed to detect the overflow when increasing the size >and fail then. Several nearby functions would need the same fix btw: > >Index: kern_resource.c >=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >--- kern_resource.c (revision 281146) >+++ kern_resource.c (working copy) >@@ -1380,17 +1380,18 @@ chgproccnt(struct uidinfo *uip, int diff, rlim_t m > int > chgsbsize(struct uidinfo *uip, u_int *hiwat, u_int to, rlim_t max) > { >+ long new; > int diff; >=20 > diff =3D to - *hiwat; >+ new =3D atomic_fetchadd_long(&uip->ui_sbsize, (long)diff) + diff; > if (diff > 0) { >- if (atomic_fetchadd_long(&uip->ui_sbsize, (long)diff) + diff > max) { >+ if (new < 0 || new > max) { > atomic_subtract_long(&uip->ui_sbsize, (long)diff); > return (0); > } > } else { >- atomic_add_long(&uip->ui_sbsize, (long)diff); >- if (uip->ui_sbsize < 0) >+ if (new < 0) > printf("negative sbsize for uid =3D %d\n", uip->ui_uid); > } > *hiwat =3D to; > > >--=20 >John Baldwin From owner-freebsd-arch@FreeBSD.ORG Mon Apr 6 20:37:44 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CF3AA8C7 for ; Mon, 6 Apr 2015 20:37:44 +0000 (UTC) Received: from mail-ig0-x22e.google.com (mail-ig0-x22e.google.com [IPv6:2607:f8b0:4001:c05::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 939A4128 for ; Mon, 6 Apr 2015 20:37:44 +0000 (UTC) Received: by igblo3 with SMTP id lo3so28943987igb.0 for ; Mon, 06 Apr 2015 13:37:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=SpxPc8W06ujAx0lDvN+5kRqeEVomOvlgbSbqZ+ILagI=; b=QVrUvsAmuiHmtTeR7AJZGmXkyRazxZloOIZbP/j5z9Oyl5CRriEYOFigHVPfXcJR4N yczAAshI+Ohs5xiL2G0/Nv7iYpkigUwsdY9tP8h8KpRX7/RO05EhfeX9yg1+JOC9pCsY RE4ukVoClFyQwM/12mVrmktjITjL0tx9dovSqGF44WW7Sj9SpzP0ca56ASQ0/mcvbtj4 RP1U9W121J4RCsKCimNx59S7qsr8OLqerSH3XBP/PVvBa1wdhvnWKvbx9NTi44HXjdms FX997t8p6D/13ARX1qrSm6Fko2ADdQrigqmzCa2saSv6vK3fxSeuUMOCJAn9pwSlb/dz V6gw== MIME-Version: 1.0 X-Received: by 10.107.155.13 with SMTP id d13mr25087212ioe.29.1428352664075; Mon, 06 Apr 2015 13:37:44 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.36.17.194 with HTTP; Mon, 6 Apr 2015 13:37:44 -0700 (PDT) In-Reply-To: <8818748.ffDquhmdWF@akita> References: <8818748.ffDquhmdWF@akita> Date: Mon, 6 Apr 2015 13:37:44 -0700 X-Google-Sender-Auth: mjVf3IC6t6W1-EexlocwfZGlrSU Message-ID: Subject: Re: x86: finding interrupts that aren't being accounted for? From: Adrian Chadd To: Rui Paulo Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Apr 2015 20:37:44 -0000 On 6 April 2015 at 12:01, Rui Paulo wrote: > On Monday 06 April 2015 00:21:29 Adrian Chadd wrote: >> Hi, >> >> I have an .. odd problem on a Lenovo X230. >> >> I just threw in a very old wifi card (Intel 3945) into the expresscard >> (pcie) slot. Now, we don't have any pcie-hp support in -HEAD just yet, >> but i wasn't expecting the system to crawl to a halt. >> >> When I unplug it, everything returns to normal. >> >> Other cards don't do this. >> >> So, I figured it may be interrupt spam - but vmstat -ia shows no >> interrupts going crazy. >> >> pmcstat -S CPU_CLK_UNHALTED_CORE -T -w 5 doesn't register anything >> either - only a handful of background samples. >> >> However, /counter/ mode pmc tells a different story - pmcstat -s >> CPU_CLK_UNHALTED_CORE -w 1 shows all four cores going at 110% when the >> card is inserted, with brief periods of idle. Once I remove the card, >> the counters go back down to zero. >> >> My working theory is: something is chewing CPU and it's likely >> interrupts, but if it is, it's something far, far earlier than the x86 >> interrupt C code, which counts interrupts and spurious events. >> >> So - has anyone diagnosed this stuff on FreeBSD/x86 before? I was kind >> of hoping we'd at least get accurate statistics about spurious >> interrupts, and if we don't, I'd like to understand why. > > If the cores are being used, you should be getting some samples as to where > the PC is. pmcstat doesn't show that? How about DTrace? Nope, nothing. Nothing at all shows up in the sample based checks. -adrian From owner-freebsd-arch@FreeBSD.ORG Mon Apr 6 20:38:29 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0FB07975; Mon, 6 Apr 2015 20:38:29 +0000 (UTC) Received: from mail-ig0-x236.google.com (mail-ig0-x236.google.com [IPv6:2607:f8b0:4001:c05::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C9313130; Mon, 6 Apr 2015 20:38:28 +0000 (UTC) Received: by igblo3 with SMTP id lo3so30763073igb.1; Mon, 06 Apr 2015 13:38:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=+NJ750icqisf56XHqN/ugpJj/xADn7OHiQ5472wc9TI=; b=tpEwcFMCgCVAsn9JhtJr+xCopCQBVCKESGfNSuJltt+lDiuODS1cdoiAtXImy7lvJR xIDpiWQ7aWG1Y/7lxngrDDsh/pQBUOJwahb+MjRe1eTJOtLcKflRgZfBmQjS2ammOK29 Yu1KCoPxrH2SUqeMxPxxr9cq3nwoArQPLt2gtBn2CSj2WAXewUluCNSAQS0dNiApcM0o Uep3Y4BjdqYcKxmf3sW3ZhgPePA+FEUtQgOJ1oAxJ/57vO7W2Wj4WoICCZPetkzBFZ+N EPYsFM2jYMGWyYk+KVLtbKXuGAZgl4cCmiQYlWjRCwpLljgQSG3pWdNAJDDU0rzOuyEJ bgUQ== MIME-Version: 1.0 X-Received: by 10.43.163.129 with SMTP id mo1mr10312028icc.61.1428352708213; Mon, 06 Apr 2015 13:38:28 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.36.17.194 with HTTP; Mon, 6 Apr 2015 13:38:28 -0700 (PDT) In-Reply-To: <1858440.dQ4AvDcZf7@ralph.baldwin.cx> References: <1858440.dQ4AvDcZf7@ralph.baldwin.cx> Date: Mon, 6 Apr 2015 13:38:28 -0700 X-Google-Sender-Auth: G3IQqydW80K-_si1QdHIZTV4cxM Message-ID: Subject: Re: x86: finding interrupts that aren't being accounted for? From: Adrian Chadd To: John Baldwin Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Apr 2015 20:38:29 -0000 On 6 April 2015 at 12:18, John Baldwin wrote: > On Monday, April 06, 2015 12:21:29 AM Adrian Chadd wrote: >> Hi, >> >> I have an .. odd problem on a Lenovo X230. >> >> I just threw in a very old wifi card (Intel 3945) into the expresscard >> (pcie) slot. Now, we don't have any pcie-hp support in -HEAD just yet, >> but i wasn't expecting the system to crawl to a halt. >> >> When I unplug it, everything returns to normal. >> >> Other cards don't do this. >> >> So, I figured it may be interrupt spam - but vmstat -ia shows no >> interrupts going crazy. >> >> pmcstat -S CPU_CLK_UNHALTED_CORE -T -w 5 doesn't register anything >> either - only a handful of background samples. >> >> However, /counter/ mode pmc tells a different story - pmcstat -s >> CPU_CLK_UNHALTED_CORE -w 1 shows all four cores going at 110% when the >> card is inserted, with brief periods of idle. Once I remove the card, >> the counters go back down to zero. >> >> My working theory is: something is chewing CPU and it's likely >> interrupts, but if it is, it's something far, far earlier than the x86 >> interrupt C code, which counts interrupts and spurious events. >> >> So - has anyone diagnosed this stuff on FreeBSD/x86 before? I was kind >> of hoping we'd at least get accurate statistics about spurious >> interrupts, and if we don't, I'd like to understand why. >> >> Thanks! > > SMM? Perhaps SMM doesn't hide itself from PMC counters (but it can hide itself > from samples). > > If it is SMM there's not really anything you can do about it. Try getting a > KTR_SCHED trace and looking at it in schedgraph. When I've seen SMM isuses in > the past it shows up as hole in the graph where nothing happens in the system. > > In your case you could perhaps be getting PCI errors that are triggering the > SMM handler. Perhaps compare pciconf -le before and after to see if there are > any changes. Hm, ok. Can we extract PCIe errors yet? -adrian From owner-freebsd-arch@FreeBSD.ORG Mon Apr 6 21:15:35 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B8FB981D; Mon, 6 Apr 2015 21:15:35 +0000 (UTC) Received: from st11p02mm-asmtp002.mac.com (st11p02mm-asmtp002.mac.com [17.172.220.237]) (using TLSv1.2 with cipher DHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8BE647D5; Mon, 6 Apr 2015 21:15:35 +0000 (UTC) Received: from fukuyama.hsd1.ca.comcast.net (c-73-162-13-215.hsd1.ca.comcast.net [73.162.13.215]) by st11p02mm-asmtp002.mac.com (Oracle Communications Messaging Server 7.0.5.35.0 64bit (built Dec 4 2014)) with ESMTPSA id <0NME007A8LPEBD40@st11p02mm-asmtp002.mac.com>; Mon, 06 Apr 2015 21:15:16 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:5.13.68,1.0.33,0.0.0000 definitions=2015-04-06_05:2015-04-06,2015-04-06,1970-01-01 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 suspectscore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1412110000 definitions=main-1504060195 Content-type: text/plain; charset=us-ascii MIME-version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: x86: finding interrupts that aren't being accounted for? From: Rui Paulo In-reply-to: Date: Mon, 06 Apr 2015 14:15:13 -0700 Content-transfer-encoding: quoted-printable Message-id: References: <1858440.dQ4AvDcZf7@ralph.baldwin.cx> To: Adrian Chadd X-Mailer: Apple Mail (2.2070.6) Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Apr 2015 21:15:35 -0000 > On Apr 6, 2015, at 13:38, Adrian Chadd wrote: >=20 > On 6 April 2015 at 12:18, John Baldwin wrote: >> On Monday, April 06, 2015 12:21:29 AM Adrian Chadd wrote: >>> Hi, >>>=20 >>> I have an .. odd problem on a Lenovo X230. >>>=20 >>> I just threw in a very old wifi card (Intel 3945) into the = expresscard >>> (pcie) slot. Now, we don't have any pcie-hp support in -HEAD just = yet, >>> but i wasn't expecting the system to crawl to a halt. >>>=20 >>> When I unplug it, everything returns to normal. >>>=20 >>> Other cards don't do this. >>>=20 >>> So, I figured it may be interrupt spam - but vmstat -ia shows no >>> interrupts going crazy. >>>=20 >>> pmcstat -S CPU_CLK_UNHALTED_CORE -T -w 5 doesn't register anything >>> either - only a handful of background samples. >>>=20 >>> However, /counter/ mode pmc tells a different story - pmcstat -s >>> CPU_CLK_UNHALTED_CORE -w 1 shows all four cores going at 110% when = the >>> card is inserted, with brief periods of idle. Once I remove the = card, >>> the counters go back down to zero. >>>=20 >>> My working theory is: something is chewing CPU and it's likely >>> interrupts, but if it is, it's something far, far earlier than the = x86 >>> interrupt C code, which counts interrupts and spurious events. >>>=20 >>> So - has anyone diagnosed this stuff on FreeBSD/x86 before? I was = kind >>> of hoping we'd at least get accurate statistics about spurious >>> interrupts, and if we don't, I'd like to understand why. >>>=20 >>> Thanks! >>=20 >> SMM? Perhaps SMM doesn't hide itself from PMC counters (but it can = hide itself >> from samples). >>=20 >> If it is SMM there's not really anything you can do about it. Try = getting a >> KTR_SCHED trace and looking at it in schedgraph. When I've seen SMM = isuses in >> the past it shows up as hole in the graph where nothing happens in = the system. >>=20 >> In your case you could perhaps be getting PCI errors that are = triggering the >> SMM handler. Perhaps compare pciconf -le before and after to see if = there are >> any changes. >=20 > Hm, ok. Can we extract PCIe errors yet? Yes, check pciconf. -- Rui Paulo From owner-freebsd-arch@FreeBSD.ORG Mon Apr 6 21:16:24 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DA26F8D5; Mon, 6 Apr 2015 21:16:23 +0000 (UTC) Received: from mail-ig0-x22c.google.com (mail-ig0-x22c.google.com [IPv6:2607:f8b0:4001:c05::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9DA4F7E6; Mon, 6 Apr 2015 21:16:23 +0000 (UTC) Received: by igblo3 with SMTP id lo3so29701805igb.0; Mon, 06 Apr 2015 14:16:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=pho1ZANy1J6rGMxMAzjyRNmrrMOYi3VTTDFa5XE/T44=; b=EQ/ca85iWUIeT8QXiMAtyNC9lzZNnFnwWUgEz4fb7sQqgD2NN71paEqjt8Uh69Nb0I xoBg5K2nkQ0ZwpIWmrDEwtdb9Dar9M5YFgscwmuuir7h2W/zbstfAJE8yynJ3chVMJEH tVC9SrP+pRJjJ4xwDOZ9PvQFbsrEbTZ6G0jRGB6Jr5gLrUo4+djrsGbn3KxGWUxL7KvI TRDOtOZkOSZBJ5I9wa1or1UHgtNNe7JJn1Ei2D1KEZ4ELJFPvDZMbmfIu8Qd9zBRMeFN PG7HppWnVGM6VCBkMFfF9FeN4jEXeCyVwd0v7ktDdU0NAlfvowgIM05Rsib+1HqV+db7 lJEQ== MIME-Version: 1.0 X-Received: by 10.50.73.168 with SMTP id m8mr497385igv.32.1428354983142; Mon, 06 Apr 2015 14:16:23 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.36.17.194 with HTTP; Mon, 6 Apr 2015 14:16:23 -0700 (PDT) In-Reply-To: References: <1858440.dQ4AvDcZf7@ralph.baldwin.cx> Date: Mon, 6 Apr 2015 14:16:23 -0700 X-Google-Sender-Auth: dSWzM1QFujR8It4wCe39HjRMjFA Message-ID: Subject: Re: x86: finding interrupts that aren't being accounted for? From: Adrian Chadd To: Rui Paulo Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Apr 2015 21:16:24 -0000 On 6 April 2015 at 14:15, Rui Paulo wrote: > >> On Apr 6, 2015, at 13:38, Adrian Chadd wrote: >> >> On 6 April 2015 at 12:18, John Baldwin wrote: >>> On Monday, April 06, 2015 12:21:29 AM Adrian Chadd wrote: >>>> Hi, >>>> >>>> I have an .. odd problem on a Lenovo X230. >>>> >>>> I just threw in a very old wifi card (Intel 3945) into the expresscard >>>> (pcie) slot. Now, we don't have any pcie-hp support in -HEAD just yet, >>>> but i wasn't expecting the system to crawl to a halt. >>>> >>>> When I unplug it, everything returns to normal. >>>> >>>> Other cards don't do this. >>>> >>>> So, I figured it may be interrupt spam - but vmstat -ia shows no >>>> interrupts going crazy. >>>> >>>> pmcstat -S CPU_CLK_UNHALTED_CORE -T -w 5 doesn't register anything >>>> either - only a handful of background samples. >>>> >>>> However, /counter/ mode pmc tells a different story - pmcstat -s >>>> CPU_CLK_UNHALTED_CORE -w 1 shows all four cores going at 110% when the >>>> card is inserted, with brief periods of idle. Once I remove the card, >>>> the counters go back down to zero. >>>> >>>> My working theory is: something is chewing CPU and it's likely >>>> interrupts, but if it is, it's something far, far earlier than the x86 >>>> interrupt C code, which counts interrupts and spurious events. >>>> >>>> So - has anyone diagnosed this stuff on FreeBSD/x86 before? I was kind >>>> of hoping we'd at least get accurate statistics about spurious >>>> interrupts, and if we don't, I'd like to understand why. >>>> >>>> Thanks! >>> >>> SMM? Perhaps SMM doesn't hide itself from PMC counters (but it can hide itself >>> from samples). >>> >>> If it is SMM there's not really anything you can do about it. Try getting a >>> KTR_SCHED trace and looking at it in schedgraph. When I've seen SMM isuses in >>> the past it shows up as hole in the graph where nothing happens in the system. >>> >>> In your case you could perhaps be getting PCI errors that are triggering the >>> SMM handler. Perhaps compare pciconf -le before and after to see if there are >>> any changes. >> >> Hm, ok. Can we extract PCIe errors yet? > > Yes, check pciconf. I'll try, but the system is pretty unusable whilst the card is plugged in... Thanks! -a From owner-freebsd-arch@FreeBSD.ORG Mon Apr 6 22:29:30 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 86738B66 for ; Mon, 6 Apr 2015 22:29:30 +0000 (UTC) Received: from elf.torek.net (mail.torek.net [96.90.199.121]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 694B0F2C for ; Mon, 6 Apr 2015 22:29:29 +0000 (UTC) Received: from elf.torek.net (localhost [127.0.0.1]) by elf.torek.net (8.14.5/8.14.5) with ESMTP id t36MTMlJ024359 for ; Mon, 6 Apr 2015 16:29:22 -0600 (MDT) (envelope-from torek@torek.net) Message-Id: <201504062229.t36MTMlJ024359@elf.torek.net> From: Chris Torek To: freebsd-arch@freebsd.org Subject: sysctl output formatting MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <24357.1428359362.1@elf.torek.net> Date: Mon, 06 Apr 2015 15:29:22 -0700 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (elf.torek.net [127.0.0.1]); Mon, 06 Apr 2015 16:29:22 -0600 (MDT) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Apr 2015 22:29:30 -0000 We had a side discussion at $work about a private sysctl emulation (so our side thing doesn't actually affect sysctl itself at all) where it was suggested that some simple numeric sysctls are "best displayed in hex". Consider, e.g.: $ sysctl kern.timecounter.tc.i8254 kern.timecounter.tc.i8254.mask: 65535 kern.timecounter.tc.i8254.counter: 25822 kern.timecounter.tc.i8254.frequency: 1193182 kern.timecounter.tc.i8254.quality: 0 The "mask" here actually makes more sense displayed in hex. One can of course use: $ sysctl -x kern.timecounter.tc.i8254 kern.timecounter.tc.i8254.mask: 0x0000ffff kern.timecounter.tc.i8254.counter: 0x0000786c kern.timecounter.tc.i8254.frequency: 0x00000000001234de kern.timecounter.tc.i8254.quality: 0000000000 but now the mask is shown in hex (yay) but the others are shown in hex too (boo). The suggestion made (note that I'm carefully filing off names via passive voice :-) ) is that there could be a sysctl flag: #define CTLFLAG_DEFHEX 0x00000800 that says that, without any additional formatting directive, the user-level "sysctl" command should output this particular value in hex. Obviously this eats up another flag bit (but there are a few left) and might require a new argument to the sysctl command ("force output to be decimal", for compatibility or whatever). Then someone said "but what about suggesting that the output be in octal, or binary, or ..." in which case this would have to be a bit-field rather than a single bit (a la the CTLMASK_SECURE field). Anyway, I volunteered to send in the idea to be bikeshedded :-) here. It does seem like a reasonable special case to add, though I admit it buys much more with certain huge 64-bit byte-grouped values we have (0x0303030103030300 reads so much nicer than 217020509924295424 for instance). If people like the idea, I could code it up as well. Chris From owner-freebsd-arch@FreeBSD.ORG Mon Apr 6 23:07:10 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 94E65306 for ; Mon, 6 Apr 2015 23:07:10 +0000 (UTC) Received: from mail-la0-x229.google.com (mail-la0-x229.google.com [IPv6:2a00:1450:4010:c03::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 03F1236F for ; Mon, 6 Apr 2015 23:07:10 +0000 (UTC) Received: by labbd9 with SMTP id bd9so18422177lab.2 for ; Mon, 06 Apr 2015 16:07:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=gcFiBw/w6GAUJwaCowdF2NJuA+qbzBoH48XRIo/ztcs=; b=pnWULs/ZiAxKk0qUWLIi3wMyyi4Roz9PQp83FtEcA5yraxewCX/k2SZ2MO1irjD7o6 tkcrazitTp8zOOcyJG9Bh/pyeRqqdRWH9HB9HKA0j/zMd/1ug0rQg1EgbHPKomAapLav MMYEFcS/t410Aq96Y/sJbm9wOIhH0+ZrIC3bL5GGoKDs0ErNqu8k35osy44+E8I0z9Z3 8NzwCR+1G5pjK3aaX49Vk69Ir9s0zF8Kfyrp13f0TqZmDWFSmW9HLdkh+GGq6FqemDj9 ycP10Lm2U06HJysqJGgjgrWm3bJwJnwB0YzXujMkxPcz1mWSZ72uOZGWhgg7+Bt8Sb7T eQ0Q== MIME-Version: 1.0 X-Received: by 10.153.5.8 with SMTP id ci8mr11095678lad.62.1428361627227; Mon, 06 Apr 2015 16:07:07 -0700 (PDT) Sender: rizzo.unipi@gmail.com Received: by 10.114.180.4 with HTTP; Mon, 6 Apr 2015 16:07:07 -0700 (PDT) In-Reply-To: <201504062229.t36MTMlJ024359@elf.torek.net> References: <201504062229.t36MTMlJ024359@elf.torek.net> Date: Tue, 7 Apr 2015 01:07:07 +0200 X-Google-Sender-Auth: sytoRLkgYUfMXiMICZqQ79vDirA Message-ID: Subject: Re: sysctl output formatting From: Luigi Rizzo To: Chris Torek Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Apr 2015 23:07:10 -0000 On Tue, Apr 7, 2015 at 12:29 AM, Chris Torek wrote: > We had a side discussion at $work about a private sysctl emulation > (so our side thing doesn't actually affect sysctl itself at all) > where it was suggested that some simple numeric sysctls are "best > displayed in hex". > > Consider, e.g.: > > $ sysctl kern.timecounter.tc.i8254 > kern.timecounter.tc.i8254.mask: 65535 > kern.timecounter.tc.i8254.counter: 25822 > kern.timecounter.tc.i8254.frequency: 1193182 > kern.timecounter.tc.i8254.quality: 0 > > The "mask" here actually makes more sense displayed in hex. One > can of course use: > > $ sysctl -x kern.timecounter.tc.i8254 > kern.timecounter.tc.i8254.mask: 0x0000ffff > kern.timecounter.tc.i8254.counter: 0x0000786c > kern.timecounter.tc.i8254.frequency: 0x00000000001234de > kern.timecounter.tc.i8254.quality: 0000000000 > > but now the mask is shown in hex (yay) but the others are shown > in hex too (boo). > > The suggestion made (note that I'm carefully filing off names via > passive voice :-) ) is that there could be a sysctl flag: > > #define CTLFLAG_DEFHEX 0x00000800 > > that says that, without any additional formatting directive, the > user-level "sysctl" command should output this particular value in > hex. Obviously this eats up another flag bit (but there are a few > left) and might require a new argument to the sysctl command > ("force output to be decimal", for compatibility or whatever). > > Then someone said "but what about suggesting that the output be in > octal, or binary, or ..." in which case this would have to be a > bit-field rather than a single bit (a la the CTLMASK_SECURE > field). > > Anyway, I volunteered to send in the idea to be bikeshedded :-) > here. It does seem like a reasonable special case to add, though > I admit it buys much more with certain huge 64-bit byte-grouped > values we have (0x0303030103030300 reads so much nicer than > 217020509924295424 for instance). > > since we are in bikeshed territory how about instead (or in addition) building a small wrapper/sysctl extension that eats sysctl's output, an exception list that specifies which variables should be converted and to what format, and does the conversion. I understand that this is potentially N*M complexity (N sysctl variables times M entries in the exception list) if we allow exceptions to be regexp, but for the numbers at hand it is probably acceptable. The advantage is that you can run it on existing kernels and can be easily customized. c heers luigi From owner-freebsd-arch@FreeBSD.ORG Tue Apr 7 04:24:12 2015 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 032DF7DC; Tue, 7 Apr 2015 04:24:12 +0000 (UTC) Received: from mail109.syd.optusnet.com.au (mail109.syd.optusnet.com.au [211.29.132.80]) by mx1.freebsd.org (Postfix) with ESMTP id A44989C6; Tue, 7 Apr 2015 04:24:10 +0000 (UTC) Received: from c211-30-166-197.carlnfd1.nsw.optusnet.com.au (c211-30-166-197.carlnfd1.nsw.optusnet.com.au [211.30.166.197]) by mail109.syd.optusnet.com.au (Postfix) with ESMTPS id 83FA0D634E4; Tue, 7 Apr 2015 14:24:01 +1000 (AEST) Date: Tue, 7 Apr 2015 14:24:00 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: John Baldwin Subject: Re: RFC: chgsbsize doesn't handle -ve sbsize correctly In-Reply-To: <476184366.nOLsX7PO1g@ralph.baldwin.cx> Message-ID: <20150407114358.B1007@besplex.bde.org> References: <476184366.nOLsX7PO1g@ralph.baldwin.cx> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=Za4kaKlA c=1 sm=1 tr=0 a=KA6XNC2GZCFrdESI5ZmdjQ==:117 a=PO7r1zJSAAAA:8 a=kj9zAlcOel0A:10 a=JzwRw_2MAAAA:8 a=my4DDG8z2af2SRqF6XQA:9 a=Il-x-ZMqIkCfMp2R:21 a=t3AzjiDFJg2ZZeVH:21 a=CjuIK1q_8ugA:10 Cc: Simon Gerraty , "arch@FreeBSD.org" , Anuranjan Shukla , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Apr 2015 04:24:12 -0000 On Mon, 6 Apr 2015, John Baldwin wrote: > On Friday, April 03, 2015 07:38:59 AM Anuranjan Shukla wrote: >> Hi, >> In struct uidinfo{}, ui_sbsize member is an int. In a system with large >> number of sockets owned by a user it's possible for ui_sbsize to roll over >> to a negative value. When that happens, further calls to _increase_ sbsize >> don't detect this condition as they should per design. But when the >> sockets start shutting down this condition would be detected and you'll >> see the console prints ("negative sbsize for uid =") happen. As a worst >> case the console being flooded with these prints can create a DoS scenario >> (specially on single CPU systems). >> >> Regardless of the end result, the code itself needs to be fixed for >> correctness. There are two things to note: >> - Int doesn't seem to be the correct type for ui_sbsize. Should be a u_int >> atleast. >> - Since there's no real prevention of the overflow as it happens, the >> warning print isn't helpful and should be a DEBUG level log at best. If we >> change ui_sbsize to be a u_int, the negative check itself can go. It could be an unconditional panic. >> ... > > Note that ui_sbsize is long, not int. That doesn't help you on 32-bit > platforms, but you are also stuck with 32 bits since you typically don't > have 64-bit atomics on 32-bit platforms. > > Making it unsigned just means you can't detect overflow anymore. Instead You can detect it as wrap. I don't like letting the long actually overflow, bit this seems simplest. > I think it should be fixed to detect the overflow when increasing the size > and fail then. Several nearby functions would need the same fix btw: > > Index: kern_resource.c > =================================================================== > --- kern_resource.c (revision 281146) > +++ kern_resource.c (working copy) > @@ -1380,17 +1380,18 @@ chgproccnt(struct uidinfo *uip, int diff, rlim_t m > int > chgsbsize(struct uidinfo *uip, u_int *hiwat, u_int to, rlim_t max) > { > + long new; > int diff; > > diff = to - *hiwat; Note that the apparently-simple subtraction here is hard to understand due to the misused of unsigned types. For negative adjustments, the RHS is a huge unsigned value. This is converted to a non-huge negative value by assignment to diff. Callers must ensure that the unsigned values are non-huge so that they would have fitted in ints for the final step to work. > + new = atomic_fetchadd_long(&uip->ui_sbsize, (long)diff) + diff; > if (diff > 0) { > - if (atomic_fetchadd_long(&uip->ui_sbsize, (long)diff) + diff > max) { > + if (new < 0 || new > max) { > atomic_subtract_long(&uip->ui_sbsize, (long)diff); > return (0); > } > } else { > - atomic_add_long(&uip->ui_sbsize, (long)diff); > - if (uip->ui_sbsize < 0) > + if (new < 0) > printf("negative sbsize for uid = %d\n", uip->ui_uid); > } > *hiwat = to; It is simpler to fix up any mess in both cases. The above could panic when the size goes negative, but it uses extra logic to continue with a corrupted size. Also remove the redundant casts. This code can be assumed to not be compiled with a K&R compiler or otherwise not have a prototype for the (inline) atomic functions in scope. Also partially fix the printf format error for the uid. uid_t is unsigned and can have any size, but its promotion was assumed to be signed int. Keep assuming that its promition has the same size as u_int. new = atomic_fetchadd_long(&uip->ui_sbsize, diff) + diff; if (new < 0 || new > max) { atomic_subtract_long(&uip->ui_sbsize, diff); if (diff < 0 && new < 0) printf("negative sbsize for uid = %u\n", uip->ui_uid); return (0); } *hiwat = to; Note: it is dangerous to let ui_sbsize actually overflow, but this can't happen unless there is a bug in the accounting. 'max' is usually RLIM_INFINITY when the size is being reduced, but really shouldn't be used in that case. The old code doesn't use it, but my new code depends on it being the max value of an rlim_t which it normally is. When the size is being increased, 'max' is the user's sbsize rlimit. That defaults to RLIM_INFINITY, so it is no limit at all, and is 2**31 times larger that the virtual address space on 32-bit arches. The size is also limited by the 32-bit args. On 64-bit arches, you just can't add up enough u_ints for ui_sbsize to get near overflowing (physical resources are probably at most a few petabytes, so they run out first by a factor of 2**20 or so). On 32-bit systems, with the default maxsockbuf limit of 2M, you need just 1024 maximally sized sockets to reach the limit. There are still races in the above on 32-bit arches. Physical limits would run out long before overflow occurs. However: - 1024 threads running in parallel might cause the overflow before any resources are allocated - just 1 more thread running in parallel and trying to reduce the count might see it remain negative. This happens for the order: - 1 thread allocates uncontended - 1024 threads increase the count to 2 step past overflow - first thread decreases the count to 1 step past overflow - another 1024 threads running in parallel might wrap the count completely (1024 steps to overflow to LONG_MAX + 1 = negative; another 1024 steps to overflow to -1 + 1 = 0). This breaks the overflow detection. Using signed types gives more chance for the overflow detection to work, unless someone shoots themself in the foot by increasing sb_max to LONG_MAX. (BTW, sb_max has a bogus type (u_long), the sysctl for changing it has a bad name (not matching the variable name), and the sysctl is badly implemented (it uses lots of code to just prevent the foot-shooting of setting the limit to a small value; large values are allowed and sb_max is changed non-atomically. I prefer to allow any foot shooting. This is most useful for testing limits.) I first thought that the races could be fixed by limiting ui_sbsize to a value much smaller than the max of its type (say LONG_MAX / 2). This doesn't quite work due to the possibility of conconcurrent accesses. The atomic accesses would have to be slightly more careful to prevent the count ever exceeding a limit. E.g., read the value non-atomically, check that it is not too large, add in a local variable, and use atomic_cmpset to store the result if the value read didn't change. This may be simpler anyway since it doesn't need the fixup step. Probably-buggy implementation: if (diff < -LONG_MAX / 2 || diff > LONG_MAX / 2) return (1); /* XXX callers should limit it. */ for (;;) { old = uip->ui_sbsize; /* May be stale even with atom. read. */ new = old + diff; /* XXX bogus overlow detection next. */ KASSERT(new >= 0, ("negative total sbsize for uid = %u\n", uip->ui_uid)); if (new > max || new > LONG_MAX / 2) return (0); /* Give up a bit too easily. */ if (atomic_cmpset_long(&uip->ui_sbsize, old, new)) { *hiwat = to; return (1); } } Remove the hackish error checking to make this easier to understand. It should be removed in the final version. I don't like KASSERT()s taking more space than the main code to check for things that can't happen. Bruce From owner-freebsd-arch@FreeBSD.ORG Tue Apr 7 07:11:22 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D50F92DE for ; Tue, 7 Apr 2015 07:11:22 +0000 (UTC) Received: from mail107.syd.optusnet.com.au (mail107.syd.optusnet.com.au [211.29.132.53]) by mx1.freebsd.org (Postfix) with ESMTP id 67EB2D23 for ; Tue, 7 Apr 2015 07:11:21 +0000 (UTC) Received: from c211-30-166-197.carlnfd1.nsw.optusnet.com.au (c211-30-166-197.carlnfd1.nsw.optusnet.com.au [211.30.166.197]) by mail107.syd.optusnet.com.au (Postfix) with ESMTPS id CA0FCD4799C; Tue, 7 Apr 2015 17:11:11 +1000 (AEST) Date: Tue, 7 Apr 2015 17:11:10 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Chris Torek Subject: Re: sysctl output formatting In-Reply-To: <201504062229.t36MTMlJ024359@elf.torek.net> Message-ID: <20150407144043.W1410@besplex.bde.org> References: <201504062229.t36MTMlJ024359@elf.torek.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=Za4kaKlA c=1 sm=1 tr=0 a=KA6XNC2GZCFrdESI5ZmdjQ==:117 a=PO7r1zJSAAAA:8 a=kj9zAlcOel0A:10 a=JzwRw_2MAAAA:8 a=1zTvKlfEcTyH67yu42YA:9 a=CjuIK1q_8ugA:10 Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Apr 2015 07:11:23 -0000 On Mon, 6 Apr 2015, Chris Torek wrote: > We had a side discussion at $work about a private sysctl emulation > (so our side thing doesn't actually affect sysctl itself at all) > where it was suggested that some simple numeric sysctls are "best > displayed in hex". > > Consider, e.g.: > > $ sysctl kern.timecounter.tc.i8254 > kern.timecounter.tc.i8254.mask: 65535 > kern.timecounter.tc.i8254.counter: 25822 > kern.timecounter.tc.i8254.frequency: 1193182 > kern.timecounter.tc.i8254.quality: 0 > > The "mask" here actually makes more sense displayed in hex. One > can of course use: > > $ sysctl -x kern.timecounter.tc.i8254 > kern.timecounter.tc.i8254.mask: 0x0000ffff > kern.timecounter.tc.i8254.counter: 0x0000786c > kern.timecounter.tc.i8254.frequency: 0x00000000001234de > kern.timecounter.tc.i8254.quality: 0000000000 > > but now the mask is shown in hex (yay) but the others are shown > in hex too (boo). > > The suggestion made (note that I'm carefully filing off names via > passive voice :-) ) is that there could be a sysctl flag: > > #define CTLFLAG_DEFHEX 0x00000800 > > that says that, without any additional formatting directive, the > user-level "sysctl" command should output this particular value in > hex. Obviously this eats up another flag bit (but there are a few > left) and might require a new argument to the sysctl command > ("force output to be decimal", for compatibility or whatever). The format arg still exists. It used to be used to get unsigned and hex formats, but is redundant for unsigned format and was rarely used to specify hex format. Now it seems to only be used: - for input, to distinguish between "IK" and other formats for CTLTYPE_INT. ("IK" with other CTLTYPEs is only supported for output) - for output, to support 'K' and even more unparsable formats like "S,clockinfo" and "S,vmtotal" Getting hex format for a single special sysctl would have been almost trivial when the X flag in the format was supported. Special cases tend to need specially bloated dynamic attachment that needs 1 byte to be changed to change the format string. For the example with timecounters above: X SYSCTL_ADD_UINT(NULL, SYSCTL_CHILDREN(tc_root), OID_AUTO, X "mask", CTLFLAG_RD, &(tc->tc_counter_mask), 0, X "mask for implemented bits"); SYSCTL_ADD_UINT() defaults a couple of parameters. Not enough to make it less bloated, but enough to prevent you controlling its format string. X SYSCTL_ADD_PROC(NULL, SYSCTL_CHILDREN(tc_root), OID_AUTO, X "counter", CTLTYPE_UINT | CTLFLAG_RD, tc, sizeof(*tc), X sysctl_kern_timecounter_get, "IU", "current timecounter value"); Here it is trivial to change "IU" to "IX". X SYSCTL_ADD_PROC(NULL, SYSCTL_CHILDREN(tc_root), OID_AUTO, X "frequency", CTLTYPE_U64 | CTLFLAG_RD, tc, sizeof(*tc), X sysctl_kern_timecounter_freq, "QU", "timecounter frequency"); This should use SYSCTL_ADD_UQUAD(). (The only differences are that it doesn't falsely claim to be MPSAFE, and has a garbage size value. Neither is MP safe on 32-bit arches, and the Giant locking given by not claiming to be MPSAFE doesn't give any safeness for the 64-bit variable accessed. The garbage size value is apparently unused. The similar x86 machdep.tsc_freq sysctl has a garbage size value of 0, but this seems to work too. In old versions, larger inconsistencies in the types and sizes caused garbage reading the top 32 bits, but also made it impossible to write these bits. SYSCTL_*PROC() is harder to use than SYSCTL_*INTTYPE(), but using it to specify a special ctltype or format is easy enough for a few uses. Bruce From owner-freebsd-arch@FreeBSD.ORG Wed Apr 8 16:12:19 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1D58EA7C; Wed, 8 Apr 2015 16:12:19 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E625DF65; Wed, 8 Apr 2015 16:12:18 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-54-116-245.nwrknj.fios.verizon.net [173.54.116.245]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id E0621B953; Wed, 8 Apr 2015 12:12:17 -0400 (EDT) From: John Baldwin To: Adrian Chadd Subject: Re: x86: finding interrupts that aren't being accounted for? Date: Mon, 06 Apr 2015 17:28:02 -0400 Message-ID: <71486315.GsjOnd645i@ralph.baldwin.cx> User-Agent: KMail/4.14.2 (FreeBSD/10.1-STABLE; KDE/4.14.2; amd64; ; ) In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 08 Apr 2015 12:12:18 -0400 (EDT) Cc: "freebsd-arch@freebsd.org" , Rui Paulo X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Apr 2015 16:12:19 -0000 On Monday, April 06, 2015 02:16:23 PM Adrian Chadd wrote: > On 6 April 2015 at 14:15, Rui Paulo wrote: > > > >> On Apr 6, 2015, at 13:38, Adrian Chadd wrote: > >> > >> On 6 April 2015 at 12:18, John Baldwin wrote: > >>> On Monday, April 06, 2015 12:21:29 AM Adrian Chadd wrote: > >>>> Hi, > >>>> > >>>> I have an .. odd problem on a Lenovo X230. > >>>> > >>>> I just threw in a very old wifi card (Intel 3945) into the expresscard > >>>> (pcie) slot. Now, we don't have any pcie-hp support in -HEAD just yet, > >>>> but i wasn't expecting the system to crawl to a halt. > >>>> > >>>> When I unplug it, everything returns to normal. > >>>> > >>>> Other cards don't do this. > >>>> > >>>> So, I figured it may be interrupt spam - but vmstat -ia shows no > >>>> interrupts going crazy. > >>>> > >>>> pmcstat -S CPU_CLK_UNHALTED_CORE -T -w 5 doesn't register anything > >>>> either - only a handful of background samples. > >>>> > >>>> However, /counter/ mode pmc tells a different story - pmcstat -s > >>>> CPU_CLK_UNHALTED_CORE -w 1 shows all four cores going at 110% when the > >>>> card is inserted, with brief periods of idle. Once I remove the card, > >>>> the counters go back down to zero. > >>>> > >>>> My working theory is: something is chewing CPU and it's likely > >>>> interrupts, but if it is, it's something far, far earlier than the x86 > >>>> interrupt C code, which counts interrupts and spurious events. > >>>> > >>>> So - has anyone diagnosed this stuff on FreeBSD/x86 before? I was kind > >>>> of hoping we'd at least get accurate statistics about spurious > >>>> interrupts, and if we don't, I'd like to understand why. > >>>> > >>>> Thanks! > >>> > >>> SMM? Perhaps SMM doesn't hide itself from PMC counters (but it can hide itself > >>> from samples). > >>> > >>> If it is SMM there's not really anything you can do about it. Try getting a > >>> KTR_SCHED trace and looking at it in schedgraph. When I've seen SMM isuses in > >>> the past it shows up as hole in the graph where nothing happens in the system. > >>> > >>> In your case you could perhaps be getting PCI errors that are triggering the > >>> SMM handler. Perhaps compare pciconf -le before and after to see if there are > >>> any changes. > >> > >> Hm, ok. Can we extract PCIe errors yet? > > > > Yes, check pciconf. > > I'll try, but the system is pretty unusable whilst the card is plugged in... PCI errors latch. You can run 'pciconf -le' after you yank the card back out. I would just do this: 'pciconf -le > before' 'pciconf -le > after' Compare before and after using something like 'kompare'. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Thu Apr 9 06:14:54 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BBB3258C; Thu, 9 Apr 2015 06:14:54 +0000 (UTC) Received: from mail-wg0-x22d.google.com (mail-wg0-x22d.google.com [IPv6:2a00:1450:400c:c00::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5F669BC2; Thu, 9 Apr 2015 06:14:54 +0000 (UTC) Received: by wgbdm7 with SMTP id dm7so109422587wgb.1; Wed, 08 Apr 2015 23:14:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=Kt82c2Y7Fi6dloPDczuFMn5c6Rb0tMk5OwpSZegWwQY=; b=ROdZfEM3B1tVaqd9JHmNKKRvhFwvTiuFsNbiYZ9DFNMDhPixgV2ddriFgBiYw7cY1+ uBYOqar4ABV3T5oMJDTXyiAt1oS2+NzJBmTgvpZ1u7EmvfeggZkqJnRZJdjzhmbzzb2T CK0jVkjHaFe6DFlBQnPWRQbvOt2f4xJxVYGZNtITiie3rPuKzI60BOq3oLpvtkpvrmPo fM23m6PDn/4kquQbZKXUrWvW4wJ8s9MyIAJnxV70RVaEHdDcsQqBL8WEciH9H1Yslixz vL88DMWk0N4w6cgL2S2zMiarVZFVlgKL1P/kAEVvKkXSCkVHY0WpyzlrXipXoIDUFMvP WVCQ== X-Received: by 10.180.96.65 with SMTP id dq1mr3232088wib.46.1428560092972; Wed, 08 Apr 2015 23:14:52 -0700 (PDT) Received: from dft-labs.eu (n1x0n-1-pt.tunnel.tserv5.lon1.ipv6.he.net. [2001:470:1f08:1f7::2]) by mx.google.com with ESMTPSA id e2sm18643873wij.5.2015.04.08.23.14.51 (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Wed, 08 Apr 2015 23:14:51 -0700 (PDT) Date: Thu, 9 Apr 2015 08:14:49 +0200 From: Mateusz Guzik To: freebsd-arch@freebsd.org Subject: Re: atomic ops Message-ID: <20150409061448.GB6086@dft-labs.eu> References: <20141028025222.GA19223@dft-labs.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20141028025222.GA19223@dft-labs.eu> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Attilio Rao , adrian@freebsd.org, Konstantin Belousov , Alan Cox X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Apr 2015 06:14:54 -0000 On Tue, Oct 28, 2014 at 03:52:22AM +0100, Mateusz Guzik wrote: [scratching old content so that I hopefully re-state it nicer] I would like to reviwe the discussion about memory barriers provided in the kernel. The kernel (at least on amd64) lacks lightweight barriers providing only following guarantees: - all writes are completed prior to given point - all reads are completed prior to given point On amd64 such barriers require only compiler barrier, and as such obviously beat currently used operations like load_acq (which uses cmpxchg). Example consumer which would benefit greatly from such barriers is seq.h: https://svnweb.freebsd.org/base/head/sys/sys/seq.h?view=markup _load_acq on amd64 provides full barrier and it was noted we should not change that in order to not break possible 3rd party consumers. Also I don't see any alternative naming convention trying to stick to this scheme that we could use. As such I propose stealing naming from Linux and introduction of smp_wmb and smp_rmb macros providing aforementioned funcionality. So for amd64 this would be: #define smp_wmb() __compiler_membar() #define smp_rmb() __compiler_membar() Any objections? I'm happy to talk to arch maintainers in order to get relevant implementations for all architectures. -- Mateusz Guzik From owner-freebsd-arch@FreeBSD.ORG Thu Apr 9 08:17:11 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1426A69D; Thu, 9 Apr 2015 08:17:11 +0000 (UTC) Received: from pp2.rice.edu (proofpoint2.mail.rice.edu [128.42.201.101]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CDAB3B63; Thu, 9 Apr 2015 08:17:10 +0000 (UTC) Received: from pps.filterd (pp2.rice.edu [127.0.0.1]) by pp2.rice.edu (8.14.5/8.14.5) with SMTP id t398H9jx001935; Thu, 9 Apr 2015 03:17:09 -0500 Received: from mh11.mail.rice.edu (mh11.mail.rice.edu [128.42.199.30]) by pp2.rice.edu with ESMTP id 1tn8hm0ddf-1; Thu, 09 Apr 2015 03:17:08 -0500 X-Virus-Scanned: by amavis-2.7.0 at mh11.mail.rice.edu, auth channel Received: from 108-254-203-201.lightspeed.hstntx.sbcglobal.net (108-254-203-201.lightspeed.hstntx.sbcglobal.net [108.254.203.201]) (using TLSv1 with cipher RC4-MD5 (128/128 bits)) (No client certificate requested) (Authenticated sender: alc) by mh11.mail.rice.edu (Postfix) with ESMTPSA id CA5434C01FE; Thu, 9 Apr 2015 03:17:04 -0500 (CDT) Message-ID: <5526357D.3080403@rice.edu> Date: Thu, 09 Apr 2015 03:17:01 -0500 From: Alan Cox User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: Mateusz Guzik , freebsd-arch@freebsd.org Subject: Re: atomic ops References: <20141028025222.GA19223@dft-labs.eu> <20150409061448.GB6086@dft-labs.eu> In-Reply-To: <20150409061448.GB6086@dft-labs.eu> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 kscore.is_bulkscore=2.26150542737003e-10 kscore.compositescore=0.247412313810446 circleOfTrustscore=0 compositescore=0.600409725240766 suspectscore=3 recipient_domain_to_sender_totalscore=0 phishscore=0 bulkscore=0 kscore.is_spamscore=0 rbsscore=0.600409725240766 recipient_to_sender_totalscore=0 recipient_domain_to_sender_domain_totalscore=0 spamscore=0 recipient_to_sender_domain_totalscore=0 urlsuspectscore=0.000409725240765767 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=7.0.1-1402240000 definitions=main-1504090074 Cc: Attilio Rao , adrian@freebsd.org, Konstantin Belousov X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Apr 2015 08:17:11 -0000 On 04/09/2015 01:14, Mateusz Guzik wrote: > On Tue, Oct 28, 2014 at 03:52:22AM +0100, Mateusz Guzik wrote: > [scratching old content so that I hopefully re-state it nicer] > > I would like to reviwe the discussion about memory barriers provided in= > the kernel. > > The kernel (at least on amd64) lacks lightweight barriers providing onl= y > following guarantees: > - all writes are completed prior to given point > - all reads are completed prior to given point > > On amd64 such barriers require only compiler barrier, and as such > obviously beat currently used operations like load_acq (which uses > cmpxchg). > > Example consumer which would benefit greatly from such barriers is > seq.h: > https://svnweb.freebsd.org/base/head/sys/sys/seq.h?view=3Dmarkup > > _load_acq on amd64 provides full barrier and it was noted we should not= > change that in order to not break possible 3rd party consumers. > Also I don't see any alternative naming convention trying to stick to > this scheme that we could use. > > As such I propose stealing naming from Linux and introduction of smp_wm= b > and smp_rmb macros providing aforementioned funcionality. > > So for amd64 this would be: > #define smp_wmb() __compiler_membar() > #define smp_rmb() __compiler_membar() > > Any objections? > > I'm happy to talk to arch maintainers in order to get relevant > implementations for all architectures. > How about stealing from C11's stdatomic.h instead of Linux. C11's model for expressing memory access ordering requirements is, like our atomic.h, inspired by the release consistency model. And, stdatomic.h has an operation, atomic_thread_fence(), that allows you to express the need for acquire and/or release ordering at some point in your program without an associated memory access. From owner-freebsd-arch@FreeBSD.ORG Thu Apr 9 13:06:15 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A23915EE for ; Thu, 9 Apr 2015 13:06:15 +0000 (UTC) Received: from mail-pd0-f181.google.com (mail-pd0-f181.google.com [209.85.192.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6CEB464A for ; Thu, 9 Apr 2015 13:06:15 +0000 (UTC) Received: by pddn5 with SMTP id n5so152493376pdd.2 for ; Thu, 09 Apr 2015 06:06:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:mime-version:content-type:from :in-reply-to:date:cc:message-id:references:to; bh=5Wj1E8gctdY3bl3NwTaRmRA3fHEBZRXgi1tbzaBZUV4=; b=gK7Pk2jMP/pgIYkFvTO2SgaVhJt0EUJv2p4/07QZQPcMPE0RWTVl/QzcAcM+GRwf5u aFP3GflSgUML42iUcIZBH5ZqN4dGO87ZQU3gzclUfGgXytHcFYixk1YUgTplQ+OswS6e GZEJDmra3evntrH/NwEQOBYMuYoUk/P6F1XrQYAYjWg1FfGVt05oike7hApP/l33roBr 6LhWdvy+weYrtGwmQs/A5Ds8jvYpStJqyZ1LWr8MjeUZjjH/hpWLqmkdjHEJz0izczP5 Ikr9uERkdMhUnCVgDOGWmlPdE0CJcb4ZR4/c+AZ5S6wygjbkptM/4oChTMVRW+X1IsTQ xv9A== X-Gm-Message-State: ALoCoQkJU5ucZO21iZdnBLIB+zRiS8S39SxtoT/ItqAN57UlwKtrtO2U77W1CMenA4E+L1XO/K8K X-Received: by 10.68.231.40 with SMTP id td8mr55085880pbc.53.1428584769011; Thu, 09 Apr 2015 06:06:09 -0700 (PDT) Received: from [10.64.24.41] ([69.53.236.236]) by mx.google.com with ESMTPSA id cr1sm14576486pdb.7.2015.04.09.06.06.06 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 09 Apr 2015 06:06:08 -0700 (PDT) Sender: Warner Losh Subject: Re: atomic ops Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Content-Type: multipart/signed; boundary="Apple-Mail=_F8FEC27C-BD9E-4735-8904-E51A12661A9D"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Pgp-Agent: GPGMail 2.5b6 From: Warner Losh In-Reply-To: <20150409061448.GB6086@dft-labs.eu> Date: Thu, 9 Apr 2015 07:06:03 -0600 Message-Id: References: <20141028025222.GA19223@dft-labs.eu> <20150409061448.GB6086@dft-labs.eu> To: Mateusz Guzik X-Mailer: Apple Mail (2.2070.6) Cc: Attilio Rao , adrian@freebsd.org, Alan Cox , Konstantin Belousov , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Apr 2015 13:06:15 -0000 --Apple-Mail=_F8FEC27C-BD9E-4735-8904-E51A12661A9D Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Apr 9, 2015, at 12:14 AM, Mateusz Guzik wrote: >=20 > On Tue, Oct 28, 2014 at 03:52:22AM +0100, Mateusz Guzik wrote: > [scratching old content so that I hopefully re-state it nicer] >=20 > I would like to reviwe the discussion about memory barriers provided = in > the kernel. >=20 > The kernel (at least on amd64) lacks lightweight barriers providing = only > following guarantees: > - all writes are completed prior to given point > - all reads are completed prior to given point What does completed mean? That=E2=80=99s a very crappy definition since = it means a range of things. (1) The CPU has pushed the writes to cache, but the cache is coherent = across the whole CPU complex. (2) The cache has flushed it to the memory controller / PCIe bridge (3) The memory has actually been updated / The PCIe bridge has pushed = the write to the card. (4) The card has completed its transaction. Which one is it? What=E2=80=99s its purpose? The efficient = implementation requires the definition be towards the start of the list. Convenient programming = of certain devices, however, requires the end=E2=80=A6 Warner --Apple-Mail=_F8FEC27C-BD9E-4735-8904-E51A12661A9D Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJVJnk7AAoJEGwc0Sh9sBEA7NkP/jUBhUi64iqzieCh1IgCrg/l Dpecq/QEYmC2XtOVm9YVyoqLQptrxb2zg7LjKoAcjMA8Io9EpxZucsZYohaZnos4 3jAEklukZpxZZ71ulhWBYrmvccn6atIPyzeoSWPWS+okTvDXBRYZ4OYAlPKvTMso PLM9aSAvQ6TxSzsR0LOjYAWax9aM3MHxoqWjgS5poemAkSu+qA+gMTaW0bM/7TEF FgGrEuIrjEwSF2hs2THfytQj/2otv8wPLVALlzgd9W9+DbnqCC+mKMSiXr9yEHCv qeuz5kmEx6QRJs7wb3OYn/st4JTUoIoLu7kNHYGlqYqNCBKzl3GxL2HH58uPasv1 j/FHRNwg2+HVDF/7I5X2NzXwoa9MXeL5WCgZQj6We9DL+qPXutvofzF6MoCLTjmy 9Hr884IXG6APvozopvEp0Xm1FI+UEVeiXSM22SDeqaPH/d/4/A5V3nxdpPzDEjQP uDyrrRW7AWK/h3Ad1UBFVebclnaUILqIR9nzLV8CNXl4J5R4CFM7ae0QEq0H4O7c z8TfyHyQnTaM7XY0FtZ9TOWY1UlzgmPordZtA5EwUvyYz/zrfR300D4lgOLatY51 rYJkwlSEEWJlbxDQnLuw+qFYyt0Kx+2TpifQ9EouiCPnE8hpb6tsVq9YjA7V1jlQ L8wGp9mQINM/E7BBu+cd =IJLJ -----END PGP SIGNATURE----- --Apple-Mail=_F8FEC27C-BD9E-4735-8904-E51A12661A9D-- From owner-freebsd-arch@FreeBSD.ORG Thu Apr 9 15:51:52 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 04929868 for ; Thu, 9 Apr 2015 15:51:52 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D201BD0C for ; Thu, 9 Apr 2015 15:51:51 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-54-116-245.nwrknj.fios.verizon.net [173.54.116.245]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id C2613B926; Thu, 9 Apr 2015 11:51:49 -0400 (EDT) From: John Baldwin To: freebsd-arch@freebsd.org Subject: Re: sysctl output formatting Date: Thu, 09 Apr 2015 11:51:17 -0400 Message-ID: <4560967.0vtmTZXoz1@ralph.baldwin.cx> User-Agent: KMail/4.14.2 (FreeBSD/10.1-STABLE; KDE/4.14.2; amd64; ; ) In-Reply-To: <201504062229.t36MTMlJ024359@elf.torek.net> References: <201504062229.t36MTMlJ024359@elf.torek.net> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 09 Apr 2015 11:51:49 -0400 (EDT) Cc: Chris Torek X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Apr 2015 15:51:52 -0000 On Monday, April 06, 2015 03:29:22 PM Chris Torek wrote: > We had a side discussion at $work about a private sysctl emulation > (so our side thing doesn't actually affect sysctl itself at all) > where it was suggested that some simple numeric sysctls are "best > displayed in hex". We had this feature for hex and it was removed. 8.x still has SYSCTL_XINT() and friends. It was removed in https://svnweb.freebsd.org/base?view=revision&revision=217586 The argument in that comment is that you should use sysctl -x. However, I think that doesn't really wash. That is fine if you are looking at a single node, but I think SYSCTL_XINT() was useful to set the default format that was used for sysctl -a, etc. You could then have flags to sysctl to override it for individual nodes for use in scripts, etc. > Consider, e.g.: > > $ sysctl kern.timecounter.tc.i8254 > kern.timecounter.tc.i8254.mask: 65535 > kern.timecounter.tc.i8254.counter: 25822 > kern.timecounter.tc.i8254.frequency: 1193182 > kern.timecounter.tc.i8254.quality: 0 > > The "mask" here actually makes more sense displayed in hex. One > can of course use: > > $ sysctl -x kern.timecounter.tc.i8254 > kern.timecounter.tc.i8254.mask: 0x0000ffff > kern.timecounter.tc.i8254.counter: 0x0000786c > kern.timecounter.tc.i8254.frequency: 0x00000000001234de > kern.timecounter.tc.i8254.quality: 0000000000 > > but now the mask is shown in hex (yay) but the others are shown > in hex too (boo). Exactly. I would support bringing the XINT variants back. Note that you would not want to revert the above commit, but you would want to make the XINT variants use a format of "X" and then fix sysctl(8) to handle "X" formats again. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Thu Apr 9 17:37:35 2015 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8BB73552 for ; Thu, 9 Apr 2015 17:37:35 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 698D1C06 for ; Thu, 9 Apr 2015 17:37:35 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.9/8.14.9) with ESMTP id t39HbZS9036232 for ; Thu, 9 Apr 2015 17:37:35 GMT (envelope-from bdrewery@freefall.freebsd.org) Received: (from bdrewery@localhost) by freefall.freebsd.org (8.14.9/8.14.9/Submit) id t39HbZL1036228 for arch@freebsd.org; Thu, 9 Apr 2015 17:37:35 GMT (envelope-from bdrewery) Received: (qmail 93212 invoked from network); 9 Apr 2015 12:37:30 -0500 Received: from unknown (HELO ?10.10.1.139?) (freebsd@shatow.net@10.10.1.139) by sweb.xzibition.com with ESMTPA; 9 Apr 2015 12:37:30 -0500 Message-ID: <5526B8E0.1090403@FreeBSD.org> Date: Thu, 09 Apr 2015 12:37:36 -0500 From: Bryan Drewery Organization: FreeBSD User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: arch@freebsd.org Subject: [RFC] SA/EN ABI/Lib flagging for package rebuilding OpenPGP: id=F9173CB2C3AAEA7A5C8A1F0935D771BB6E4697CF; url=http://www.shatow.net/bryan/bryan2.asc Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="oVEQeu3HqjtX9WVb7XsxUcrbpVRd3PDoo" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Apr 2015 17:37:35 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --oVEQeu3HqjtX9WVb7XsxUcrbpVRd3PDoo Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Currently for any SA/EN we force rebuild all packages for pkg.freebsd.org. This is a very time-consuming process at 24 hours for each set and 10 (non-head) sets to build. We really only need to rebuild packages if ABI/KBI is changed (which I would think would happen almost never) or if a library is updated. The library being updated needs a rebuild in case of packages linking against the library statically. There's no quick way to determine what package may be infected by a static linkage to a library so we just rebuild them all. There are 2 somewhat conflicting goals here. We need head packages to be usable on all ABI/KBI changes. The simplest here was decided to just rebuilding head packages always. This is a compromise I am willing to accept. I had wanted to use __FreeBSD_version for this in the past but it tends to be missed or bumped gratuitously. My idea to cover both cases so that packages will only rebuild if the public API or libraries change in the jail or anything else which affects the resulting binaries. I will take a checksum of the following and if anything differs then do a rebuild: /lib /usr/lib /libexec (.so included due to symbol versioning and detecting added/removed libraries which can change packages) /usr/include (all public API for the system including kernel API for modules) /usr/libdata That is all I am aware of for the public API/KPI/ABI/KBI that is relevant for packages. I also realize I need to include some of the binutils files as they may change how binaries are produced. This is all of 'ar as ld nm objcopy objdump ranlib readelf strip' in /usr/bin/. Also all of GCC/Clang binaries in /usr/bin: 'cc clang clang++ clang-cpp clang-tblgen cpp c++filt CC c++ g++ gcc gnu-ar gnu-ranlib'. I don't want to consider all of /bin /usr/bin /sbin and /usr/sbin as it catches SA that do not really need a rebuild, such as FreeBSD-SA-15:07.ntp which modified /usr/sbin/ntpd. By having a whitelist of files from *bin* I risk not rebuilding in a rare case where /bin/sh or /bin/cp (random example) had a vulnerability fixed in them that could change resulting binaries. I consider that to be very unlikely though. Perhaps for head packages it is more relevant but I still consider it unlikely other changes will require rebuilding packages. If I forgot other critical pieces from *bin* please list them. An alternative would be for there to be a reliable flag in head and SA/EN noting whether binaries need to be rebuilt. There had been a technical limitation to bumping __FreeBSD_version for SA as it would modify all binaries. I think that was fixed. Portmgr fought hard to always have __FreeBSD_version bumped but that's a lost cause. As I mentioned I don't trust this flag in head either. All feedback and other ideas welcome. Thanks, Bryan Drewery --oVEQeu3HqjtX9WVb7XsxUcrbpVRd3PDoo Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAEBAgAGBQJVJrjgAAoJEDXXcbtuRpfP5vEIAJtLgBiNzK6v4JM0gsqOQs+Z uPfaiuIAOBOqkZ2c0gOEgmNDvT8UHq7C7+NyuGmhM9lCsi1e0vZ5dN9ca95x0Qq6 af9crdygPEc+mtWtGCDgJMQ99fSaoazfb4ZJOnthEvcsP/zo9gCm2ufNUV0ZYzXF rR2ATCeBca5mMUmLaTGRGionjBnGQYrXZZ6oTf1xPZBtPDRig7YYdehHS/1e0JTe A9hdUGL7qpV3nwd/HFudTrB9TDTGkJw8oXvwaxdgIp4l7CxBtMi+nc8TyzqxRjBA RJR0pu9HlOpKuqfLet9XsHC7y9559YcgFL8Km0LtECxIhvpc3aDFgWKYH7CPjwQ= =bfLQ -----END PGP SIGNATURE----- --oVEQeu3HqjtX9WVb7XsxUcrbpVRd3PDoo-- From owner-freebsd-arch@FreeBSD.ORG Sat Apr 11 09:18:59 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A1648916 for ; Sat, 11 Apr 2015 09:18:59 +0000 (UTC) Received: from mail-pd0-x22a.google.com (mail-pd0-x22a.google.com [IPv6:2607:f8b0:400e:c02::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6FCB3883 for ; Sat, 11 Apr 2015 09:18:59 +0000 (UTC) Received: by pdbqa5 with SMTP id qa5so48526677pdb.1 for ; Sat, 11 Apr 2015 02:18:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:from:date:message-id:subject:to:content-type; bh=+m/Aw1ByPVkX8y0Q4tJxEf4gGzm27sivx58/OhSJ5+4=; b=emaPJOrgGPBS8txGxbCOepdJpfNezoWVCp9QevKLJqs4bU+ukcahPEH+zBYZpJUMFA jHYUlNuGqjn5yLLtE8WYTO+NP5+fjUQ9QE7EZrHCtladHjbGsVcJmzZmSP+ozFSmByW4 srLo5UL0HPxAHHh3vBs6SKQnlfd7T9yucTMF1bBBAuVvMyq3MODjy+V8R/YSCuagaIIS gBGyS87ymoO/C4/BTqgARLyYvdxOcxIM+DOYioaRP2J+fBQQAbsXzOkZVeGCM7JO5mj5 vGa08nSpzJ3PaSbBypITTNp/ehTSO5j1i3Ag4RpbVlGaJPjjFa7hh1haIvr8xbSAJwLT 3JUg== X-Received: by 10.66.141.77 with SMTP id rm13mr9709914pab.14.1428743938941; Sat, 11 Apr 2015 02:18:58 -0700 (PDT) MIME-Version: 1.0 Sender: ycyc321@gmail.com Received: by 10.67.2.42 with HTTP; Sat, 11 Apr 2015 02:18:28 -0700 (PDT) From: Yue Chen Date: Sat, 11 Apr 2015 05:18:28 -0400 X-Google-Sender-Auth: u9fTaWSOCxNlA8X1uXIT-sWoPL0 Message-ID: Subject: Situations about PC values in kernel data segments To: freebsd-arch@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Apr 2015 09:18:59 -0000 Dear all, We are working on a project about OS security. We wonder in which situations the program counter (PC) value (e.g., the value in %RIP on x86_64, i.e, instruction address) could be in kernel (module) data segments (including stack, heap, etc.). Here we mainly care about the address/value that are NOT function entry points since there exist a number of function pointers. Also, we only consider the normal cases because one can write arbitrary values into a variable/pointer. And we mainly consider i386, AMD64 and ARM. Here are some situations I can think about: function/interrupt/exception/syscall return address on stack; switch/case jump table target; page fault handler (pcb_onfault on *BSD); restartable atomic sequences (RAS) registry; thread/process context structure like Task state segment (TSS), process control block (PCB) and thread control block (TCB); situations for debugging purposes (e.g., like those in ``segment not present'' exception handler). Additionally, does any of these addresses have offset formats or special encodings? For example, on x86_64, we may use 32-bit RIP-relative (addressing) offset to represent a 64-bit full address. In glibc's setjmp/longjmp jmp_buf, they use a special encoding (PTR_MANGLE) for saved register values. Best thanks and regards, Yue From owner-freebsd-arch@FreeBSD.ORG Sat Apr 11 14:28:42 2015 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0ACEA937 for ; Sat, 11 Apr 2015 14:28:42 +0000 (UTC) Received: from mail-vn0-x229.google.com (mail-vn0-x229.google.com [IPv6:2607:f8b0:400c:c0f::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C2455A61 for ; Sat, 11 Apr 2015 14:28:41 +0000 (UTC) Received: by vnbg62 with SMTP id g62so11443967vnb.6 for ; Sat, 11 Apr 2015 07:28:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=RGASW0jVyIRtDA9YBYk6u8dAR9GQ7GD5s/QZeYhuk6Q=; b=uZwQpGNVvAJJr5pl35bHSznxxpCDdueeeIFp7s7/LrVIU9UWwNcLkdpRKSvQ8SPIme BN+uT6pxa6JC5nL5WljFODmflm74O9xxXmoiQ3tAcgaFRfTmpxY+kdaA7089P24w6l9u Jptr+3jGebdCdFL7jsznw+XLgcAo0kvxXFVzJnSh/2DQ+qt49r9XQsdDRBYpZjS0O7je otLuCK+dCp0aMYns65T19/2daSn1FSI+N8XiSy5md7zkOESX+4u2tbCmH/aDNA6GMIjE 1T8BG5MBjSAYYdMzCW7ISio/DtIXT2f29deWhav8fZeK7/l0DOu3JMLmGTAKHaYDVK4N oZWw== X-Received: by 10.52.230.2 with SMTP id su2mr7135567vdc.4.1428762520662; Sat, 11 Apr 2015 07:28:40 -0700 (PDT) Received: from ivaldir.etoilebsd.net ([2001:41d0:8:db4c::1]) by mx.google.com with ESMTPSA id cf6sm368579vdc.15.2015.04.11.07.28.38 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 11 Apr 2015 07:28:39 -0700 (PDT) Sender: Baptiste Daroussin Date: Sat, 11 Apr 2015 16:28:35 +0200 From: Baptiste Daroussin To: arch@FreeBSD.org Subject: RFC: Alternative to PRIVATELIB Message-ID: <20150411142835.GE65320@ivaldir.etoilebsd.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="uCPdOCrL+PnN2Vxy" Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Apr 2015 14:28:42 -0000 --uCPdOCrL+PnN2Vxy Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi, I would like to propose to replace PRIVATELIB with something more convenient. First what is PRIVATELIB is trying to solve: We are maintaining stable ABI over branches but some third parties sources are not really good at maintaining stable ABI, so we do hide them into private where nothing can use it. We do not provide headers for that and we add rpath to every binaries that needs to link to those. What is the issues of PRIVATELIB: any application linking to a library from base (a regular one) that does itself links to a PRIVATELIB cannot anymore statically link to the said application. The is no mechanism to handle PRIVATELIBS in compat*x ports which can be a problem if one of our regular lib is linked to a privatelib and ends up into compat one day. It prevents easy linking for 3rd party application using those privatelibs on purpose (aka with the knowledge abi can break) like libbsdstat. What I would like to propose is the following: Create in bsd.lib.mk support for PRIVATE knobs (what ever name you do prefer) It will just prefix the name of the library with "private" but install it in the regular place It will automatically decide to install the headers into /usr/include/private/${LIB}/ Each private library headers in a custom place to avoid an application that deliberatly use a given privatelib to find another one Prefix all manpage with private_ so that we can provide the documentation for the said libs for the version we ship but if another version is shipped by ports then we can easily access both documentation. As a result bsd.lib.mk will be simpler, we could again static link against everything we ship in base, we can provide documentation for those libs and we can easily isolate them anyway from the ports. I plan to start working on this in a week. Best regards, Bapt --uCPdOCrL+PnN2Vxy Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAlUpL5MACgkQ8kTtMUmk6Ey9QgCcDAB/Ox6f3g2Oj0hq8NVst14A iOIAn0wVeO32Cyqje8Ewlds+sSIe53CY =WI1l -----END PGP SIGNATURE----- --uCPdOCrL+PnN2Vxy-- From owner-freebsd-arch@FreeBSD.ORG Sat Apr 11 15:14:23 2015 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A694842E for ; Sat, 11 Apr 2015 15:14:23 +0000 (UTC) Received: from mail-vn0-x22f.google.com (mail-vn0-x22f.google.com [IPv6:2607:f8b0:400c:c0f::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 562CFF27 for ; Sat, 11 Apr 2015 15:14:23 +0000 (UTC) Received: by vnbg7 with SMTP id g7so11643480vnb.10 for ; Sat, 11 Apr 2015 08:14:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=2jOdJsyhoz2Rn9A68K4HXkU4Aovg/Yy+uhg5S9UYLTI=; b=dPa/5f3XAwewN2AMpLdq19C2p7bMCzNVSQvQHpq//SbB0sPMlb9Pk+NfcLe78RDrWO vxUJSPGpC/+nBHxyiRwEC3wgGLtFN1HynEkPn0gl4ImB3BVqrpJTAafuJoH9I/GK2Dkk UXadsb5H/2nHTIsM13DMJ/vbeeb0wmc2A8Weshu73tdWsKY3zINpv/BFWQk64kLo+iYs tH+avaEhEEz7Lh7PFAKar2vp94tTMVzkqQsWPL4DFaWNWH67oBjw4tP+hgaf3GO33fHc Xm8/bi4eRPBK0vTzpj6Sme72mVClnqaZriog/8EIRoSbTD2Aml0eONQRXzgen48uz3wY V9NQ== X-Received: by 10.52.99.226 with SMTP id et2mr7338470vdb.14.1428765262454; Sat, 11 Apr 2015 08:14:22 -0700 (PDT) Received: from ivaldir.etoilebsd.net ([2001:41d0:8:db4c::1]) by mx.google.com with ESMTPSA id s2sm392329vdh.8.2015.04.11.08.14.15 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 11 Apr 2015 08:14:16 -0700 (PDT) Sender: Baptiste Daroussin Date: Sat, 11 Apr 2015 17:14:12 +0200 From: Baptiste Daroussin To: arch@FreeBSD.org Subject: Re: RFC: Alternative to PRIVATELIB Message-ID: <20150411151412.GF65320@ivaldir.etoilebsd.net> References: <20150411142835.GE65320@ivaldir.etoilebsd.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="xaMk4Io5JJdpkLEb" Content-Disposition: inline In-Reply-To: <20150411142835.GE65320@ivaldir.etoilebsd.net> User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Apr 2015 15:14:23 -0000 --xaMk4Io5JJdpkLEb Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Apr 11, 2015 at 04:28:35PM +0200, Baptiste Daroussin wrote: > Hi, >=20 > I would like to propose to replace PRIVATELIB with something more conveni= ent. >=20 > First what is PRIVATELIB is trying to solve: > We are maintaining stable ABI over branches but some third parties sources > are not really good at maintaining stable ABI, so we do hide them into pr= ivate > where nothing can use it. We do not provide headers for that and we add r= path to > every binaries that needs to link to those. >=20 > What is the issues of PRIVATELIB: > any application linking to a library from base (a regular one) that does = itself > links to a PRIVATELIB cannot anymore statically link to the said applicat= ion. >=20 > The is no mechanism to handle PRIVATELIBS in compat*x ports which can be a > problem if one of our regular lib is linked to a privatelib and ends up i= nto > compat one day. >=20 > It prevents easy linking for 3rd party application using those privatelib= s on > purpose (aka with the knowledge abi can break) like libbsdstat. >=20 > What I would like to propose is the following: >=20 > Create in bsd.lib.mk support for PRIVATE knobs (what ever name you do pre= fer) >=20 > It will just prefix the name of the library with "private" but install it= in the > regular place >=20 > It will automatically decide to install the headers into /usr/include/pri= vate/${LIB}/ > Each private library headers in a custom place to avoid an application th= at > deliberatly use a given privatelib to find another one >=20 > Prefix all manpage with private_ so that we can provide the documentation= for > the said libs for the version we ship but if another version is shipped b= y ports > then we can easily access both documentation. >=20 > As a result bsd.lib.mk will be simpler, we could again static link against > everything we ship in base, we can provide documentation for those libs a= nd we > can easily isolate them anyway from the ports. >=20 > I plan to start working on this in a week. >=20 > Best regards, > Bapt This patch shows an example with libucl: https://people.freebsd.org/~bapt/private.diff Best regards, Bapt --xaMk4Io5JJdpkLEb Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAlUpOkQACgkQ8kTtMUmk6Exm3QCfayHKSrr5Xmv9SQJP/ev1Kalx s4IAoJSZ6j1IOLDJrnTt7fsueC2tE6cp =ahkd -----END PGP SIGNATURE----- --xaMk4Io5JJdpkLEb--