From owner-freebsd-hackers@freebsd.org Sun Apr 2 00:29:44 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 61147D2ACB6 for ; Sun, 2 Apr 2017 00:29:44 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from dmz-mailsec-scanner-4.mit.edu (dmz-mailsec-scanner-4.mit.edu [18.9.25.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 12713238 for ; Sun, 2 Apr 2017 00:29:43 +0000 (UTC) (envelope-from kaduk@mit.edu) X-AuditID: 1209190f-d43ff70000007382-c4-58e044bdeea8 Received: from mailhub-auth-2.mit.edu ( [18.7.62.36]) (using TLS with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by (Symantec Messaging Gateway) with SMTP id FE.26.29570.DB440E85; Sat, 1 Apr 2017 20:24:29 -0400 (EDT) Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by mailhub-auth-2.mit.edu (8.13.8/8.9.2) with ESMTP id v320OSrb028385; Sat, 1 Apr 2017 20:24:29 -0400 Received: from kduck.kaduk.org (24-107-191-124.dhcp.stls.mo.charter.com [24.107.191.124]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id v320OOSM005165 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Sat, 1 Apr 2017 20:24:27 -0400 Date: Sat, 1 Apr 2017 19:24:24 -0500 From: Benjamin Kaduk To: Yao Bao Cc: freebsd-hackers@freebsd.org Subject: Re: make uninstall /usr/src/share/doc/ Message-ID: <20170402002423.GB30306@kduck.kaduk.org> References: <201704010056.v310u4iB078722@meetlost.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201704010056.v310u4iB078722@meetlost.com> User-Agent: Mutt/1.6.1 (2016-04-27) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrGIsWRmVeSWpSXmKPExsUixG6norvX5UGEwYOd1hbfZhxjsti++R+j A5PHjE/zWTy2LFnEGsAUxWWTkpqTWZZapG+XwJWx/PdStoLpbBUnHyxhbGD8wNLFyMkhIWAi MX3hRzYQW0igjUlix33dLkYuIHsDo8SiHV/YIZwrTBLNU9YxgVSxCKhI3N3cwQhiswHZDd2X mUFsEQFZiRtvtoNNZRaQl3i4/QyYLSygL7H57QawGl6gbX0HDrJDbDOT+Hz0KVRcUOLkzCdQ vVoSN/69BNrFAWRLSyz/xwES5hQwl1h7uQesVVRAWaJhxgPmCYwCs5B0z0LSPQuhewEj8ypG 2ZTcKt3cxMyc4tRk3eLkxLy81CJdE73czBK91JTSTYygIOWU5N/BOKfB+xCjAAejEg/vh6f3 I4RYE8uKK3MPMUpyMCmJ8n4vvhchxJeUn1KZkVicEV9UmpNafIhRgoNZSYS39glQOW9KYmVV alE+TEqag0VJnFdcozFCSCA9sSQ1OzW1ILUIJivDwaEkwRvl/CBCSLAoNT21Ii0zpwQhzcTB CTKcB2j4ApAa3uKCxNzizHSI/ClGXY4bxw+8YRJiycvPS5US5zUHKRIAKcoozYObA0ouEtn7 a14xigO9JczbClLFA0xMcJNeAS1hAlpi8fUuyJKSRISUVANjlhGLbMki5ckVoXOmT2BmP5fQ cTtv9tbqj0Y9ptE7UpW/3NmcqDOZoXuCVUrIXNaSTQsTZk/cImC5gPHmrU3POl818k/87L1n eaZoi03k+arf/7gbDx2Y9G/zVAcGOxO39poY18iZaff/fjvzbuupq89tt1/Suyrb7jdncqzj CaaHqaedrkrmKLEUZyQaajEXFScCAIbL1NEJAwAA X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Apr 2017 00:29:44 -0000 On Sat, Apr 01, 2017 at 12:56:04AM +0000, Yao Bao wrote: > Hi, > > I installed the docs under /usr/src/share/doc/ with commands below: > make > make install > make clean > > And I am thinking about is there anyway to unsintall the docs I just installed? > I tried make uninstall but no luck. > > Can anyone provide some tips on this? There is no 'uninstall' target for the bsd.prog.mk, bsd.doc.mk, etc. makefile includes that are used by src/share/doc/. One thing that could be done is 'touch stampfile; make install; find /usr/share -newer stampfile' to get a list of files that are probably the installed ones. Sometimes 'make -n install' is also useful for generating a list of what files would have been installed, but only if submakes are not in use. -Ben From owner-freebsd-hackers@freebsd.org Sun Apr 2 06:03:36 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 63A58D29CCA for ; Sun, 2 Apr 2017 06:03:36 +0000 (UTC) (envelope-from by@meetlost.com) Received: from meetlost.com (freebsd.meetlost.com [IPv6:2403:2500:8000:1::962]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "freebsd103", Issuer "freebsd103" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id E537BAF1 for ; Sun, 2 Apr 2017 06:03:35 +0000 (UTC) (envelope-from by@meetlost.com) Received: from [10.47.158.7] ([117.136.40.229]) (authenticated bits=0) by meetlost.com (8.15.2/8.15.2) with ESMTPSA id v322jeDd087712 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 2 Apr 2017 02:45:41 GMT (envelope-from by@meetlost.com) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: make uninstall /usr/src/share/doc/ From: by X-Mailer: iPhone Mail (14A456) In-Reply-To: <20170402002423.GB30306@kduck.kaduk.org> Date: Sun, 2 Apr 2017 14:03:22 +0800 Cc: freebsd-hackers@freebsd.org Content-Transfer-Encoding: 7bit Message-Id: <5E980365-8479-4E53-818A-F5D3D8B337C7@meetlost.com> References: <201704010056.v310u4iB078722@meetlost.com> <20170402002423.GB30306@kduck.kaduk.org> To: Benjamin Kaduk X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Apr 2017 06:03:36 -0000 Hi, Thanks for the reply. The "touch stampfile" command sequence works as expected. by > > There is no 'uninstall' target for the bsd.prog.mk, bsd.doc.mk, etc. > makefile includes that are used by src/share/doc/. One thing that > could be done is 'touch stampfile; make install; find /usr/share > -newer stampfile' to get a list of files that are probably the > installed ones. Sometimes 'make -n install' is also useful for > generating a list of what files would have been installed, but only > if submakes are not in use. > > -Ben From owner-freebsd-hackers@freebsd.org Sun Apr 2 06:13:28 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B2C06D290BC for ; Sun, 2 Apr 2017 06:13:28 +0000 (UTC) (envelope-from kczekirda@gmail.com) Received: from mail-qt0-x229.google.com (mail-qt0-x229.google.com [IPv6:2607:f8b0:400d:c0d::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7FE64F79 for ; Sun, 2 Apr 2017 06:13:28 +0000 (UTC) (envelope-from kczekirda@gmail.com) Received: by mail-qt0-x229.google.com with SMTP id r45so91984859qte.3 for ; Sat, 01 Apr 2017 23:13:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=p8mLuIn95AITjpcYCk39ow0Obc7RfkkMl0YP4GKfloQ=; b=jixiDwQrlP3U2yB1zpjdWHS8uBBS71KvLc6V19R4NaHWyNOQA7qD2dXUJGYiNYkqAg KdqavvwQ1wrcdzPceBFudRVE1kmIBMxmUzvqWGSUSjCC0XnRw3j3vzoyaXD+cxizRlBK 2uH/G4fYrbHjniRz3D/D7NAbFvPGOi4EIUxhlYezZ8D1f3+f/aHObxdhCILKbTBWWQx3 AcfHaTDNPHCEql4zqKa9xfKEBMU0RzgXePFl/D7eSGVaQlh1Uq25kZuhoK9Im4Ddqf3+ 9pn3dpRHETXTUPXuktB95/qQKoJsb6ksyw5TF3xqm9Kw24RrEVZxn2Ox0W+FS8iQTz5E OGFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=p8mLuIn95AITjpcYCk39ow0Obc7RfkkMl0YP4GKfloQ=; b=G74teXMuv2sCe6D//HpDdbKVppt2IY+HIpOY1HHf7CiMXejGO1ckXKBia3jkvi/4WC nKia/kDAL/NBrsNPNFt+DvQwsVCH/LDnDQfL56F6QTTiCZIybWi7EIySsU/rkKTus94K mwkx7YaMhOsDBfszvRFCphatqchewk0/44lON5aHLeDqgAoOqfU4kED5O9RAJippjxUW ta1TSGuLx/EYe0gnNO3vGfJMbxa8rRxajoK84uJtGXgXvSoZUZUhoUYEjKTjN6EMz15z y0W1WuQtLc7Epz6/ReQqaLk42iBfMLpP9KIoAFUKjDjlgG4EVojlan2ejRwq3bhaec1b FMlw== X-Gm-Message-State: AFeK/H1WKnH4QpMzWvQKp++/BAu53JNJrLqzdBhnNjAJ+G3i2tuCvs8ybsbhdBRq1M6Y6p3Dr26I2naRHoFaZw== X-Received: by 10.237.44.229 with SMTP id g92mr10460279qtd.204.1491113607459; Sat, 01 Apr 2017 23:13:27 -0700 (PDT) MIME-Version: 1.0 Received: by 10.200.44.154 with HTTP; Sat, 1 Apr 2017 23:12:57 -0700 (PDT) In-Reply-To: <1491049373.5625.1.camel@ovh.fr> References: <1491049373.5625.1.camel@ovh.fr> From: Kamil Czekirda Date: Sun, 2 Apr 2017 08:12:57 +0200 Message-ID: Subject: Re: at home server without screen blocked by bad ipfw conf -- live boot usb with sshd To: Orka Edison Cc: freebsd-hackers@freebsd.org X-Mailman-Approved-At: Sun, 02 Apr 2017 10:49:10 +0000 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Apr 2017 06:13:28 -0000 Hi, please take a look at mfsbsd http://mfsbsd.vx.sk/ Kamil 2017-04-01 14:22 GMT+02:00 Orka Edison : > hi, > > i brink my at home by a bad ipfw.rules... > > how can i cr=C3=A9ate an usb boot key with sshd for access to my server ? > with an fixed IP and tools-box for repair my machine. > > best regard, > Orka Edison. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " From owner-freebsd-hackers@freebsd.org Sun Apr 2 11:29:31 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B2CBFD2A6F3 for ; Sun, 2 Apr 2017 11:29:31 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: from smtprelay02.ispgateway.de (smtprelay02.ispgateway.de [80.67.18.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 766B762A for ; Sun, 2 Apr 2017 11:29:30 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: from [78.35.156.238] (helo=fabiankeil.de) by smtprelay02.ispgateway.de with esmtpsa (TLSv1.2:AES256-GCM-SHA384:256) (Exim 4.84) (envelope-from ) id 1cudSy-0007Pr-Be for freebsd-hackers@freebsd.org; Sun, 02 Apr 2017 13:14:40 +0200 Date: Sun, 2 Apr 2017 13:14:40 +0200 From: Fabian Keil To: freebsd-hackers@freebsd.org Subject: Re: at home server without screen blocked by bad ipfw conf -- live boot usb with sshd Message-ID: <20170402130647.2f063868@fabiankeil.de> In-Reply-To: <1491049373.5625.1.camel@ovh.fr> References: <1491049373.5625.1.camel@ovh.fr> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; boundary="Sig_/6UbptWHF7g4Y_A0Jm38QqCu"; protocol="application/pgp-signature" X-Df-Sender: Nzc1MDY3 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Apr 2017 11:29:31 -0000 --Sig_/6UbptWHF7g4Y_A0Jm38QqCu Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Orka Edison wrote: > i brink my at home by a bad ipfw.rules... >=20 > how can i cr=C3=A9ate an usb boot key with sshd for access to my server ? > with an fixed IP and tools-box for repair my machine. I use a script to adjust the UFS partition on an image created by the "memstick" target after it has been copied to the USB stick. The relevant script content: set -e -x mount "${device}" /mnt/ [...] mkdir -p /mnt/root/.ssh echo 'ssh-ed25519 [...]' > /mnt/root/.ssh/authorized_keys chmod -R go-rwx /mnt/root/.ssh echo 'PermitRootLogin yes' >> /mnt/etc/ssh/sshd_config echo 'ifconfig_re0=3D192.168.5.48' >> /mnt/etc/rc.conf echo 'sshd_enable=3D"YES"' >> /mnt/etc/rc.conf sed -e 's@ro,@rw,@' -i.bak /mnt/etc/fstab cat /mnt/etc/fstab umount /mnt You'll have to add your own public ssh key, adjust the rc.conf modification and maybe add a default router if needed. If you simply messed up the ipfw rules the memstick image should contain everything you need to fix it. If you are in a hurry and don't mind using unreproducible binaries built by third parties you could download a memstick image from freebsd.org instead of building it yourself. Fabian --Sig_/6UbptWHF7g4Y_A0Jm38QqCu Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iF0EARECAB0WIQTKUNd6H/m3+ByGULIFiohV/3dUnQUCWODdIAAKCRAFiohV/3dU nSKGAJ9LlfcbYnwUTHyzjWn4q36I5WK+GACgmpMqT9YysrCbIQCH5Rtomab7K9A= =1veK -----END PGP SIGNATURE----- --Sig_/6UbptWHF7g4Y_A0Jm38QqCu-- From owner-freebsd-hackers@freebsd.org Sun Apr 2 20:12:14 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 50668D2B7DB for ; Sun, 2 Apr 2017 20:12:14 +0000 (UTC) (envelope-from kneit@pdx.edu) Received: from mail-pg0-x244.google.com (mail-pg0-x244.google.com [IPv6:2607:f8b0:400e:c05::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 25D2D2A5 for ; Sun, 2 Apr 2017 20:12:13 +0000 (UTC) (envelope-from kneit@pdx.edu) Received: by mail-pg0-x244.google.com with SMTP id g2so25203692pge.2 for ; Sun, 02 Apr 2017 13:12:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pdx-edu.20150623.gappssmtp.com; s=20150623; h=to:from:subject:message-id:date:user-agent:mime-version; bh=ZM5mYF00R5y79kbiB6rIZFiR0mooinUkTxTQ5ngq/Mc=; b=fUO3p9qa0+wm7kEqYRC1kulMac0A/MkmQbIRKu05BNdivD++bG1WYAysCWyaXXJ8nh 4nZ4R8G/IBCNQATHJey4wDNku/ZGEQDSOWHhysv+TWuEt24fJTT2K2U2CMFdsUpGqu02 OLu/jiARNyOjUcdoYUkeFP9ly4Cwf11hoG3UAoOn6Ao8I+2wPqd6VVBe+6/hF6PUEej7 HEPKpGWRM1ptEB+OEENDVp7P5qxr0NqQ5GPtTNPjQF4lYTb1EWi9c18gKTV4wU6Z0/iB HgHw1oeKuKFFcXP0SZ61G8cVkT9ixpoX/caEsZGH67rLZJk4aAmdELmRAPPGxiBFNlvl 9Ryg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:from:subject:message-id:date:user-agent :mime-version; bh=ZM5mYF00R5y79kbiB6rIZFiR0mooinUkTxTQ5ngq/Mc=; b=suI/Tv4vPfj1QqnKUv4bzBlukY+WnqnouP9wE/XIC5Yr1d8yBlH4eybLQkoPLB81EV lB1Nq0gsle4HAj+Ev4e/zvHPD66Ob3h0pDpoj9/JNiFRHU9rdLa3c7KJIn9E/H8eJtjx Sc1vsUuxQ4TymqvJZxM2g93fPTNikYsfDj5RgGemFBNEzwQ9cIsHuu+3oRmtexwKIz5n M0ZFwNDTAHolu6FK2tHeCScfxXvo//ZHkD9QVsldthZI5whvhA9cKlEXPW1KO0CxCiYv ENEjin+8iRfgw4zfggO4ScdX4hcoh47ztSxUqyfIlLcUiRMETUWu2cQqNSnX6FaqAU7/ CzYg== X-Gm-Message-State: AFeK/H1EwW3qOKzc+HXjh9t/QKKsXvhawA0D9N6S+oRLYTA9NcglcpNiljBV0Mu7Kl1RWv/l X-Received: by 10.99.151.18 with SMTP id n18mr2767597pge.199.1491163933087; Sun, 02 Apr 2017 13:12:13 -0700 (PDT) Received: from ?IPv6:2601:1c2:1302:6d10::8? ([2601:1c2:1302:6d10::8]) by smtp.gmail.com with ESMTPSA id t6sm21543938pgo.42.2017.04.02.13.12.12 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 02 Apr 2017 13:12:12 -0700 (PDT) To: freebsd-hackers@freebsd.org From: Kyle Kneitinger Subject: GSoC 2017: Introduction and Draft Proposal [beadm replacement] Message-ID: <3a191979-d925-c7bd-0de5-389d82abb21d@pdx.edu> Date: Sun, 2 Apr 2017 13:12:11 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="AiFNpH6VJWE4J6SPPvnhDCC72ms55v76h" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 Apr 2017 20:12:14 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --AiFNpH6VJWE4J6SPPvnhDCC72ms55v76h Content-Type: multipart/mixed; boundary="br7Prr8V6Mgwg8vFNRDEKidWCKmwb9R1q"; protected-headers="v1" From: Kyle Kneitinger To: freebsd-hackers@freebsd.org Message-ID: <3a191979-d925-c7bd-0de5-389d82abb21d@pdx.edu> Subject: GSoC 2017: Introduction and Draft Proposal [beadm replacement] --br7Prr8V6Mgwg8vFNRDEKidWCKmwb9R1q Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hello All, First, I'd like to introduce myself. I'm Kyle Kneitinger, a senior at Portland State University, in Portland, OR, US. I've been user of FreeBSD for a couple of years now, so since I finally had a summer in which my work/life/schoo= l schedule is compatible with GSoC, I was extremely pleased to browse through the FreeBSD ideas. I strongly gravitated towards the beadm replacement project. I really enj= oy writing libraries and system utilities, and the opportunity to do this fo= r something as important to me as ZFS was very appealing. I corresponded wi= th Allan Jude a few times for clarification about the intended idea and requirements, but I would like to give anyone else the opportunity to weigh in on features, clarifications, or just general feedback on my draft proposa= l. Thank you in advance, Kyle Kneitinger Google Summer of Code 2017 Proposal: A New ZFS Boot Environment Management Library & Tool For FreeBSD Student: Kyle J. Kneitinger Mentor: Allan Jude [[ contact info redacted on draft proposal ]] Synopsis The purpose of this project is to build a convenient framework for managing ZFS boot environments in =3D=3D=3D=3DSynopsis=3D=3D=3D=3D The purpose of this project is to build a convenient framework for managing ZFS boot environments in FreeBSD. Currently, boot environments are managed manually, or with scripts such as `beadm`. The approach that this projec= t will take is to first create a boot environment library to offer simplifi= ed boot environment interaction to userspace applications (for example, True= OS' SysAdm perhaps), and then write an application that provides users with a= command (tentatively referred to as `bootenvmgr` for the remainder of thi= s proposal) that retains all of the features of `beadm`, with some added functionality. Among the new features are: the ability to recursively cre= ate boot environments containing child datasets, the ability to activate an environment for the next boot only, and the ability to attach/detach a bo= ot environment to a jail. =3D=3D=3D=3DTimeline and Deliverables=3D=3D=3D=3D ----Final Deliverable---- The final deliverable is the `bootenvmgr`: a command with a superset of `beadm` features (illustrated below in the usage and clarifying examples), and the library, tentatively referred to as libzfs_bootenv that provides the features to the command, and the userspace ecosystem. $ bootenvmgr -h usage: bootenvmgr command args ... where 'command' is one of the following: activate [-t] create [-r] [-e nonActiveBe | -e bootenv@snapshot] create =E2=9F=A8beName@snapshot=E2=9F=A9 destroy [-F] [bootenv|bootenv@snapshot]* list [-a] [-D] [-H] [-s] rename mount [mountpoint] unmount|umount [-f] jail unjail Each bootenv is a name of a dataset in (or to be created in $POOLNAME/bootenv/ Examples illustrating optional arguments and flags that extend from `bead= m`: -- `bootenvmgr create -r foo` recursively creates a boot environment of= / -- `bootenvmgr activate -t foo` uses the zpool_nextboot function (originally implemented for `zfsbootcfg`) to boot into the foo environment on the next boot only. -- `bootenvmgr jail bars foo` attaches the foo boot environment to the jail named bars. See `man beadm` for extended descriptions of remaining options ----Milestones---- *May 4 - May 30, 2017: Community Bonding Period* Tasks: -- In addition to all usual community bonding tasks, solicit feedback f= rom users of boot environments and `beadm` on features and user experien= ce of the command. -- Replicate various boot environment use cases. Deliverables: -- All items on FreeBSD GSoC Student Checklist Community Bonding sectio= n completed (accounts, pages, repos, etc. created). -- Final requirements solidified in the form of a preliminary man page.= *May 30 - June 30, 2017: By 1st Evaluation* Note: My University is on a term-based system as opposed to semesters, and therefore I am still in classes for the first 2 weeks of June Focus: Design and research. Tasks: -- Prototype recursive boot environment mechanics. -- Craft detailed design. -- Begin Coding library component. Deliverables: -- Makefile and headers for library complete. Skeleton, documented, implementation files created (and if time permits, coding has begun)= =2E *June 30 - July 28, 2017: By 2nd Evaluation* Focus: Completion of library, design of user command. Tasks: -- Complete library implementation. -- Complete all non-functional requirements for library (developer documentation, testing). -- Create detailed design of user command. Deliverables: -- Completed library implementation and documentation. -- User command headers, Makefile, and Skeleton implementation file. *June 28 - August 29, 2017: By Code Submission and Final Evaluation* Focus: Completion of user command Deliverables: -- User command, updated manpage, and thus all project requirements =3D=3D=3D=3DAbout Me & Experience=3D=3D=3D=3D I am currently a Senior at Portland State University in Portland, OR, US,= focusing my studies on operating systems, language design and implementat= ion (especially grammars and parsers), and when the opportunity arises, digit= al music creation. I believe to some extent, my dedication to school is a la= rge factor in why I use FreeBSD on my personal computers, as I wouldn't trust= my in-progress assignments to anything else. With my free time I enjoy playing music, gardening, creating software synthesizers and hardware to control them, hiking, and going to the various OSS events and meetups that I am fortunate enough to live near. Throughout my studies over the past three years, I had the opportunity to grow as a developer in ways both conceptually and practically. In the context = of this project, I would say my most relevant experiences have been my time = as team lead on one of the CS program's final projects, my time participating in an 18-month cooperative learning program with 3 different companies, and = of course, some of my open source work. ---Senior Project Team Lead--- I led a team of 6 other students through our task of reimplementing the Cairo vector graphics library in rust, with Cairo's creator, Carl Worth, as the= project sponsor. This was an invaluable experience that allowed my team and I to grow as developers, communicators, and collaborators. We focused heavily= on design, with the main goal of creating an interface that is familiar and comfortable to those with experience using Cairo, balanced with crafting = an idiomatic, 'rustic', feel to those who write a lot of rust. In the 14 we= eks we had, we surpassed all goals and stretch goals defined by Mr. Worth. T= he academic project wrapped up just a week and half before the writing of th= is proposal, and now myself and a subset of the team are working on outreach and infrastructure to turn this into an educational open source project for n= ew developers, or just those new to rust programming. ---Cooperative Education Program--- I am a month away from completing my 18 months of my participation in Portland State University's PCEP (PSU/Portland Cooperative Education Program). Through this program I worked 20-hours a week at 3 different Portland-based tech companies in three different roles: Quality Engineer, Developer, and DevO= ps Engineer. [[ JOB DETAILS REDACTED FOR DRAFT PROPOSAL ]] ---Open Source--- -- i3wm window manager: Numerous bug fixes in the configuration file parser's handling of variables, documentation corrections, a new feature to allow the passing of a new config file path on soft reloads.=20 Technologies and themes: c, perl, yacc, flex, domain-specific languages, user documentation. -- liblo lightweight OSC (Open Sound Control) library: participated in discussion and drafted proof of concept code to implement a feature for control path discovery. Technologies and themes: c, networking, UDP, API design. -- stk Synthesis Toolkit library: Updates to Makefile after update brok= e compatibility with certain distributions. Technologies and themes: Makefiles, build tools. ---Personal Open Source Projects--- -- RPiShift: A python module for interfacing a Raspberry Pi with an HC5= 95 shift register. Provides simplified API, as well as advanced interaction. Technologies and themes: Python, hardware, documentation, packaging.= -- markoff/slackov/gramov: A library for simple novelty text generation from an updatable corpus and Markov chains, and 2 example applications for popular messaging services. Technologies and themes: Python, API design, API use, packaging. --br7Prr8V6Mgwg8vFNRDEKidWCKmwb9R1q-- --AiFNpH6VJWE4J6SPPvnhDCC72ms55v76h Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQFCBAEBCAAsFiEEJGjKTob84Jt6mBwZrtJnrVK0txEFAljhWxsOHGtuZWl0QHBk eC5lZHUACgkQrtJnrVK0txHZNQf+IA9Jje840BI7IbtsKkQjPmg7NCQFzQeShQod ThY4AY1Ysn8YSnHA0HMY8jQKTErhg6R8Z8mbeA0AmBda6HahFREN9S48hqmGnOme TStmtHZJAJ6elIcK1eX1qIIAtP+2iUR8IRBQF6WHHZ+e3GPdo7/3wR1Y6LCEtwMq 45tLctA5xMGTgDJGK0wn15PplbPEYWsflwjyV16+4u8ko6DBOgR2NdHXYET3UAqs /rCmk7F3J5yKWMqEFuN3oBRlZ1DeT9D0dKn8M2sKnlvzgxsqu3Xq+FyOQ4EIp3TA /Ni1VbiZV6Sa3eBWPrYOnHaay6yUfuZcBl1nbT16nDA57jbeXA== =UmOk -----END PGP SIGNATURE----- --AiFNpH6VJWE4J6SPPvnhDCC72ms55v76h-- From owner-freebsd-hackers@freebsd.org Mon Apr 3 14:28:47 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 188EED2BFFA for ; Mon, 3 Apr 2017 14:28:47 +0000 (UTC) (envelope-from kczekirda@gmail.com) Received: from mail-qt0-x231.google.com (mail-qt0-x231.google.com [IPv6:2607:f8b0:400d:c0d::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BCCDBD85 for ; Mon, 3 Apr 2017 14:28:46 +0000 (UTC) (envelope-from kczekirda@gmail.com) Received: by mail-qt0-x231.google.com with SMTP id r45so113242142qte.3 for ; Mon, 03 Apr 2017 07:28:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=2bY+lb0At9XjBycQhnuYZjEwJWWFUwoirYKswCLmGQg=; b=NnBM9J/sKmHZo2UJjpVD14Hh2eYcSSktOz5LlcPFT6eQpx0TNinXUa8bpAzBkLiZCe VrcM7Qdrb5m4gwCzGDVs67+dcwk4Yqu7j1FAtXtXCm93721+8fIWYtxuJ6sdCKMdBH4G vl+/+O/GPmAfIQkYjzy0rpdjmYjT1thqgIEmch6qHroiQywNuKvRiDzmW64nC6ZDcIJt fmroVfhkieyeFuvOsk5IZaj/QmEfc3BTMEqAry3AXsvGPiy81erhEqYO/lvmC//bBV/N Ysx2N2+LEAplCOiCToOV2q6d1xEiPSpZaO7XXzYjDjXgaA323iRH5boOMtVwfJC2wFPU /FAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=2bY+lb0At9XjBycQhnuYZjEwJWWFUwoirYKswCLmGQg=; b=eTPBTTNUJ79wo0QBAK2OqExmZSCQ7YeF9EI+s7iepED8ccsJ8GnFv9dQmrkIi0WlHI FDE+QiFYANFWvYQ1p31jbVrSS615KCoZdrhec/fXJLV4QBJsrXSTQRE657RKHLfdt75v 5gy3NTg+izXbWX088jhJrpM+4CcxA5dgKvu7bI2b6fdet8/xd9k2Q3+6r+Rzm0ZuCgwd Z1/z5vqpCpR9roQkDJ5iy+8DDEaVQUxwTV0U4VL3/sHCD25hTAOHzX/H60XpQI9qYKKW LNhU/uV1tBAy2x30KVmYyLLRbSwJPlogAbTvYRlzKmih4+0dQWYebPJCYFlPYY6tltOo VMOA== X-Gm-Message-State: AFeK/H1rU2mwGSg30eZJG6Lw/sUOVB31ovBfOzYXDklk5DtKbyy+Jirn5w3fNOjmb5/05LbPQKYt5Nkb6UO0dA== X-Received: by 10.200.44.36 with SMTP id d33mr17937925qta.198.1491229725717; Mon, 03 Apr 2017 07:28:45 -0700 (PDT) MIME-Version: 1.0 Sender: kczekirda@gmail.com Received: by 10.200.44.154 with HTTP; Mon, 3 Apr 2017 07:28:15 -0700 (PDT) In-Reply-To: <1491049373.5625.1.camel@ovh.fr> References: <1491049373.5625.1.camel@ovh.fr> From: Kamil Czekirda Date: Mon, 3 Apr 2017 16:28:15 +0200 X-Google-Sender-Auth: grs4QgkTiY0P4iJJ0kmRTbzFwrY Message-ID: Subject: Re: at home server without screen blocked by bad ipfw conf -- live boot usb with sshd To: Orka Edison Cc: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Apr 2017 14:28:47 -0000 Hi, please take a look at mfsbsd http://mfsbsd.vx.sk/ 2017-04-01 14:22 GMT+02:00 Orka Edison : > hi, > > i brink my at home by a bad ipfw.rules... > > how can i cr=C3=A9ate an usb boot key with sshd for access to my server ? > with an fixed IP and tools-box for repair my machine. > > best regard, > Orka Edison. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " From owner-freebsd-hackers@freebsd.org Tue Apr 4 06:50:51 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1C759D2D587 for ; Tue, 4 Apr 2017 06:50:51 +0000 (UTC) (envelope-from aijazbaig1@gmail.com) Received: from mail-vk0-x232.google.com (mail-vk0-x232.google.com [IPv6:2607:f8b0:400c:c05::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CC9A621F for ; Tue, 4 Apr 2017 06:50:50 +0000 (UTC) (envelope-from aijazbaig1@gmail.com) Received: by mail-vk0-x232.google.com with SMTP id z204so164828219vkd.1 for ; Mon, 03 Apr 2017 23:50:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=E75Xm5VA55zPKFsU61m7NuXTc2rBqf+Z3Dv2Zof+yt0=; b=R1TkJmMfUVR72Ay96qdW2sbapZnDjQYU0wyQJY0L/DGgarVGc8+BJ5+ekzRv61bPz3 +wVKpGZUwKqJ7eAyLqnmWesNJifvN89pZvdBbioASBX/6iJKlSeKmKygJfkJt51dtpLi GtujBnlw4NBYb/iSe/3KOpouHWlZchR5fUKKnz+3hKX1NBH2WU3u5/wcZ+MvMinBlFNa co1EtZ38SkB/J9MpwlaAEDzMHFDBo6ndlpYiyx5O5UeN3IoL3zBsd11ohYzqh7WX0Tbc YemLx3Z71f1FhsgnA/nNC+JdEFsjBkkLJcKuq9VrlTh3sMSXfwlkIiWVZd2UGME7VmQp GykQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=E75Xm5VA55zPKFsU61m7NuXTc2rBqf+Z3Dv2Zof+yt0=; b=aiVnK4B0B32Lsv9ncAbZQmddJn64sprAcwskfTUNauSKoCep0vMSODuc8rbmSNcgM+ eQQmsRDu2ZYXtw0AfnEGnXjz5hGOVoN63kT+VXQXAVew9hnTT+kCyEzYXqD482OCWxEU vzZo0DZxiRyk1dWbhkCH0nGQ3+71oo6CINERXgM451ghbdFOLcp2+Mk3tc0iTRI06aLh PGyZVicm+2I1hcOUJDvHlkoGQil3MR6Hizy5M9ERIv95jZgRLQT807pvu5W7uhk+AAQE AdUmgvPM6Ic2n1leZJKqLY+Lfu936O57Qtcp8MFyCn/BYehlhpYhEDiNqcyitKtGbmqc 6giA== X-Gm-Message-State: AFeK/H0b5I6GtpP4GBkK+sgVnPtZeWg312EfH36X80uFiIr7D+0JycPwP+M/Wfb5v2eh4Hd/Vl9/upsiytpiGg== X-Received: by 10.159.40.135 with SMTP id d7mr11232499uad.122.1491288649438; Mon, 03 Apr 2017 23:50:49 -0700 (PDT) MIME-Version: 1.0 Received: by 10.31.50.138 with HTTP; Mon, 3 Apr 2017 23:50:49 -0700 (PDT) From: Aijaz Baig Date: Tue, 4 Apr 2017 12:20:49 +0530 Message-ID: Subject: Debugging KLD -- cannot access memory at 'address' To: FreeBSD Hackers Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Apr 2017 06:50:51 -0000 Hello. I am unable to get quite a 'handle' on debugging KLDs. I have compiled my modules with the DEBUG_FLAGS set to -g3 and have also installed it using the same flags so that /boot/kernel/ contains not only the module binary (kernel object) but also the symbol file. So I do a make DEBUG_FLAGS=-g3 make install DEBUG_FLAGS=-g3 under the directory containing the KLD which ultimately copies the binary as well as the symbol file. After connecting kgdb to the remote (debugged) machine, I load my module which automatically asks me if I need to load the symbols from the symbol file, to which I click yes sudo kgdb kernel.debug (kgdb) set remotebaud 9600 (kgdb) target remote /dev/cuau0 Remote debugging using /dev/cuau0 kdb_sysctl_enter (oidp=, arg1=, arg2=0xfffffe001e3397f0, req=) at /usr/src/sys/kern/subr_kdb.c:446 446 kdb_why = KDB_WHY_UNSET; Current language: auto; currently minimal (kgdb) kldstat Id Refs Address Size Name 1 8 0x80200000 17e10c8 kernel 2 1 0x819e2000 4cf0 vmxnet.ko 3 1 0x81c11000 23dc vmmemctl.ko 4 1 0x81c14000 47a echo.ko (kgdb) add-kld /boot/kernel/echo.ko add symbol table from file "/boot/kernel/echo.ko.symbols" at .text_addr = 0xffffffff81c14000 set_modmetadata_set_addr = 0xffffffff81c14228 set_sysinit_set_addr = 0xffffffff81c14238 .rodata.str1.1_addr = 0xffffffff81c14240 .data_addr = 0xffffffff81c142f8 .bss_addr = 0xffffffff81c14420 (y or n) y Reading symbols from /boot/kernel/echo.ko.symbols... location expression too complex...done. (kgdb) b echo_write Breakpoint 1 at 0xffffffff81c14161: file echo.c, line 89. (kgdb) getsyms Id Refs Address Size Name 1 8 0x80200000 17e10c8 kernel 2 1 0x819e2000 4cf0 vmxnet.ko 3 1 0x81c11000 23dc vmmemctl.ko 4 1 0x81c14000 47a echo.ko Select the list above with the mouse, paste into the screen and then press ^D. Yes, this is annoying. 1 8 0x80200000 17e10c8 kernel 2 1 0x819e2000 4cf0 vmxnet.ko 3 1 0x81c11000 23dc vmmemctl.ko 4 1 0x81c14000 47a echo.ko (kgdb) n 95 strlen += MIN(uio->uio_iov->iov_len, bufferspace-1); (kgdb) x/d strlen Cannot access memory at address 0x0 (kgdb) info locals strlen = Cannot access memory at address 0x0 I am unable to fathom what am I missing. Keen to hear -- Best Regards, Aijaz Baig From owner-freebsd-hackers@freebsd.org Tue Apr 4 08:14:28 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AD82AD2D8F2 for ; Tue, 4 Apr 2017 08:14:28 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from smtp-a.userve.net (smtp-outbound.userve.net [217.196.1.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.userve.net", Issuer "Go Daddy Secure Certificate Authority - G2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4433AB97; Tue, 4 Apr 2017 08:14:26 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from owa.usd-group.com (owa.usd-group.com [217.196.1.2]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp-a.userve.net (Postfix) with ESMTPS id 64142A8D; Tue, 4 Apr 2017 09:14:19 +0100 (BST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=userve.net; s=201508; t=1491293659; bh=V+KCVs541Q4cMkk7brcTgTK2ITwWis6aH/z+4D5OSAU=; h=From:To:CC:Subject:Date:References:In-Reply-To; b=eR+I/QXgCiQX6OFeJ4sfsWIuctv9sy4nKEd+XyzNrJxWzQT+RkfYuQHhZwGCxFZLD DYBqhnDG+ukgKmEWC6wKtyiW1aupAyrKgakJ2priixxPxZvLa/J/uT9ZNID4Y5au2f TgEwjHLaF9Ugl+kRZbKCeWH8L4qNzgJo/+qc9nX8= Received: from SERVER.ad.usd-group.com (192.168.0.1) by SERVER.ad.usd-group.com (192.168.0.1) with Microsoft SMTP Server (TLS) id 15.0.847.32; Tue, 4 Apr 2017 09:14:18 +0100 Received: from SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9]) by SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9%12]) with mapi id 15.00.0847.030; Tue, 4 Apr 2017 09:14:18 +0100 From: Matt Churchyard To: John CC: "freebsd-hackers@freebsd.org" Subject: RE: Sendmail conf files Thread-Topic: Sendmail conf files Thread-Index: AQHSqYhJA05PurJ9FUaUzW3xtZ+p6KG0R/gAgACU6FA= Date: Tue, 4 Apr 2017 08:14:18 +0000 Message-ID: References: <481fef207e2e457cb4f8a689d0ce4373@SERVER.ad.usd-group.com> <20170403235624.GA94548@FreeBSD.org> In-Reply-To: <20170403235624.GA94548@FreeBSD.org> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.0.10] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailman-Approved-At: Tue, 04 Apr 2017 09:30:51 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Apr 2017 08:14:28 -0000 ----- Matt Churchyard's Original Message ----- > Hello, >=20 > Not sure if this is the right list for this.. >=20 > For me the most awkward part of updating a system using freebsd-update is= when it comes to merging files. The most common files that pop up seem to = be /etc/mail/sendmail.cf & /etc/mail/submit.cf, because these are included = in the base distribution and almost certainly change if you actually use Se= ndmail. > Simply don't use the same names.... in /etc/make.conf: > SENDMAIL_MC =3D /etc/mail/prodconf.mc > and the .cf file(s) will be regenerated. >YMMV, >Cheers I appreciate the response and could use that method on my own systems (alth= ough I probably won't need to now as I pretty much always use Postfix). It = doesn't really solve the original issue though. It's not exactly ideal to tell all users they should edit /etc/make.conf ju= st so they can use Sendmail and not have to mess about with cf files on eve= ry upgrade. (To be honest I got to the point where I used to just accept th= e merge even though it left errors in the config file, then went and ran /e= tc/mail/make to re-create everything. For a user that's easier that trying = to merge files in a terminal editor). As I said there is/has been a large e= ffort to make sure users don't have to edit base config files in FreeBSD so= that base can be updated without affecting user config or requiring mergin= g. Unless there's some awkward issue I'm not aware of, I don't really see w= hy the two .cf files are included in base in the first place, rather than g= enerated at runtime. I didn't really post this to fix my own problem, I'm just trying to suggest= something that may make upgrades easier for everyone. When running freebsd= -update these cf files are some of the very few files that regularly requir= e manual intervention. Matt > In most cases this sort of issue has been solved by providing "default fi= les" such as /etc/defaults/rc.conf, and letting the user override this with= files they create themselves. Obviously there is a concerted effort to mak= e sure users don't have to edit base files where possible so that they don'= t get these sort of issues. >=20 >=20 > For Sendmail, would it not make sense to remove these 2 .cf files from ba= se and update the sendmail rc.d script to run 'make install' in /etc/mail i= f they don't exist? It may also be nice if the freebsd.* files were stored = somewhere else such as /usr/share/sendmail, as these just causes confusion = about which files are actually used if you're new to it. >=20 >=20 > Personally I'm on the side that would rather have Sendmail removed entire= ly and replaced with a simple smtp submission daemon/lda but I think that d= iscussion has already been had. >=20 >=20 > - >=20 > Matt Churchyard From owner-freebsd-hackers@freebsd.org Tue Apr 4 12:35:07 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 32FAAD2DF11 for ; Tue, 4 Apr 2017 12:35:07 +0000 (UTC) (envelope-from eric@metricspace.net) Received: from mail.metricspace.net (mail.metricspace.net [IPv6:2001:470:1f11:617::107]) by mx1.freebsd.org (Postfix) with ESMTP id 0D9E36C2 for ; Tue, 4 Apr 2017 12:35:07 +0000 (UTC) (envelope-from eric@metricspace.net) Received: from [IPv6:2001:470:1f11:617:3210:b3ff:fe77:ca3f] (unknown [IPv6:2001:470:1f11:617:3210:b3ff:fe77:ca3f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: eric) by mail.metricspace.net (Postfix) with ESMTPSA id C98B818EE for ; Tue, 4 Apr 2017 12:34:59 +0000 (UTC) To: "freebsd-hackers@freebsd.org" From: Eric McCorkle Subject: Source of QEMU woes: CPUTYPE Message-ID: Date: Tue, 4 Apr 2017 08:34:56 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="9i7th3aWbum4cTA5Pj6liSncgkta9kE3B" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Apr 2017 12:35:07 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --9i7th3aWbum4cTA5Pj6liSncgkta9kE3B Content-Type: multipart/mixed; boundary="n3e7TdFMkt97h5qmBNVxNSwLE9h2RKmnV"; protected-headers="v1" From: Eric McCorkle To: "freebsd-hackers@freebsd.org" Message-ID: Subject: Source of QEMU woes: CPUTYPE --n3e7TdFMkt97h5qmBNVxNSwLE9h2RKmnV Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable A while ago I posted on here about some problems I'd had with testing boot loader modifications on QEMU, and which also showed up on an unmodified HEAD. I ultimately tracked down the source of the problem: I had CPUTYPE?=3Dnative set in my /etc/make.conf. As my CPU is relatively recent, this caused some instructions that QEMU doesn't support to be generated in various places (most notoriously, in strlen), which would trigger illegal instruction exceptions. Removing the CPUTYPE?=3Dnative line restores things to working order. I'm posting this here, as it's somewhat non-obvious, and probably ought to be documented somewhere. --n3e7TdFMkt97h5qmBNVxNSwLE9h2RKmnV-- --9i7th3aWbum4cTA5Pj6liSncgkta9kE3B Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEzzhiNNveVG6nWjcH1w0wQIFco2cFAljjkvAACgkQ1w0wQIFc o2d5sg/+LjUK97GGMM7APkDx+Z7ZkCGswppidSIcx9Xulehq2pGUhNE4OPXyV1RN iBXIWTRt6IeBH7OPeRX9q1xEYoZDi8E7XeTNJZbzxze2aWVnP5nPl0BuAd54Z19T /znyOkNsP0cmgFX/piwabZvGQgsEq9D3INtYQmfN7RpCel3sWEfnHNNlLHiIBXoT 7W5Zv/z34oJTZq3WmBzwAnz0XqXPCUN5dbQdJ2o5PU1IdXodAbACeQSSM/OO6c/l Zx5e1L4IRbWmoPH0rYvbnOZUewSabkA1fGONL7fjbAWlw/65Pdgtgi1GxCpzXOsD zqShKchMSqTtYgpBAsbOEFUuokIKNpU3UT4SROdwZ8ktOZtJoc10TW9ZFZHntoJV QkYOmQFp2e7Ibo2kYmWnK2xCjOIQ+WwLYFmx3vD54hICSzVKEsikpGbtUhR+3dWz uAAqAY6Eqpuhg81eNlwSSQjExVT01LWf/hCtgPoopDObxk6RNU4wKlyzZsfhnPwi ZLi3nF8lj0cQ5r5AxNhuTJODl7uepfGGjC9v1CZfTdwtWbHwsfMuQzWIYaAhGkLb Zu9HGjr423yqpOGk/rKoGHoY3A2cAwE9srUaX0lp4yKpIq/EmUrET8vD6vIT8Fm4 he91IeVmWSyUMpa2T+rP4sdZ2sfGq8h0yzw1oA94Trxjz18EqBQ= =slnC -----END PGP SIGNATURE----- --9i7th3aWbum4cTA5Pj6liSncgkta9kE3B-- From owner-freebsd-hackers@freebsd.org Tue Apr 4 18:32:11 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1CDEFD2EBCA for ; Tue, 4 Apr 2017 18:32:11 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "tensor.andric.com", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CE396BF7 for ; Tue, 4 Apr 2017 18:32:10 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from [IPv6:2001:470:7a58::988f:bd7f:a95:844] (unknown [IPv6:2001:470:7a58:0:988f:bd7f:a95:844]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 8D9A636EBA; Tue, 4 Apr 2017 20:32:01 +0200 (CEST) From: Dimitry Andric Message-Id: Content-Type: multipart/signed; boundary="Apple-Mail=_C01E471F-D4CA-4FDB-B623-75CED0A40087"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: Source of QEMU woes: CPUTYPE Date: Tue, 4 Apr 2017 20:31:50 +0200 In-Reply-To: Cc: "freebsd-hackers@freebsd.org" To: Eric McCorkle References: X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Apr 2017 18:32:11 -0000 --Apple-Mail=_C01E471F-D4CA-4FDB-B623-75CED0A40087 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii On 4 Apr 2017, at 14:34, Eric McCorkle wrote: > > A while ago I posted on here about some problems I'd had with testing > boot loader modifications on QEMU, and which also showed up on an > unmodified HEAD. > > I ultimately tracked down the source of the problem: I had > CPUTYPE?=native set in my /etc/make.conf. As my CPU is relatively > recent, this caused some instructions that QEMU doesn't support to be > generated in various places (most notoriously, in strlen), which would > trigger illegal instruction exceptions. Out of interest, what does "llvm-tblgen -version | grep 'Host CPU'" show? (This is a simple way to see what LLVM auto-detects.) > I'm posting this here, as it's somewhat non-obvious, and probably ought > to be documented somewhere. I usually find it clearer to specify the exact CPU type myself, for example CPUTYPE?=core-avx2 (which is an alias for "haswell"). You can also specify a lower CPUTYPE to build the world that you are going to run inside QEMU. -Dimitry --Apple-Mail=_C01E471F-D4CA-4FDB-B623-75CED0A40087 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.30 iEYEARECAAYFAljj5qEACgkQsF6jCi4glqN04QCbBmPw//EYF5txrecmdiRq060M SFcAn226Jwp6HtgLpANSBPqf0pgijGUM =KP2T -----END PGP SIGNATURE----- --Apple-Mail=_C01E471F-D4CA-4FDB-B623-75CED0A40087-- From owner-freebsd-hackers@freebsd.org Tue Apr 4 18:34:35 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0FEDDD2ED3C for ; Tue, 4 Apr 2017 18:34:35 +0000 (UTC) (envelope-from eric@metricspace.net) Received: from mail.metricspace.net (mail.metricspace.net [IPv6:2001:470:1f11:617::107]) by mx1.freebsd.org (Postfix) with ESMTP id DA93CE0A; Tue, 4 Apr 2017 18:34:34 +0000 (UTC) (envelope-from eric@metricspace.net) Received: from [IPv6:2001:470:1f11:617:3210:b3ff:fe77:ca3f] (unknown [IPv6:2001:470:1f11:617:3210:b3ff:fe77:ca3f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: eric) by mail.metricspace.net (Postfix) with ESMTPSA id 113491A65; Tue, 4 Apr 2017 18:34:33 +0000 (UTC) Subject: Re: Source of QEMU woes: CPUTYPE To: Dimitry Andric References: Cc: "freebsd-hackers@freebsd.org" From: Eric McCorkle Message-ID: <25c71912-7eec-a174-9d9f-50280c3435e8@metricspace.net> Date: Tue, 4 Apr 2017 14:34:29 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="6O1qIcNBSMugx293PD2wOTgK1r00q2PgF" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Apr 2017 18:34:35 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --6O1qIcNBSMugx293PD2wOTgK1r00q2PgF Content-Type: multipart/mixed; boundary="gWvtJaQcvJSV6iXno68GOik4fiAJHBUSm"; protected-headers="v1" From: Eric McCorkle To: Dimitry Andric Cc: "freebsd-hackers@freebsd.org" Message-ID: <25c71912-7eec-a174-9d9f-50280c3435e8@metricspace.net> Subject: Re: Source of QEMU woes: CPUTYPE References: In-Reply-To: --gWvtJaQcvJSV6iXno68GOik4fiAJHBUSm Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 04/04/2017 14:31, Dimitry Andric wrote: > On 4 Apr 2017, at 14:34, Eric McCorkle wrote: >=20 > Out of interest, what does "llvm-tblgen -version | grep 'Host CPU'" > show? (This is a simple way to see what LLVM auto-detects.) broadwell >=20 >> I'm posting this here, as it's somewhat non-obvious, and probably ough= t >> to be documented somewhere. >=20 > I usually find it clearer to specify the exact CPU type myself, for > example CPUTYPE?=3Dcore-avx2 (which is an alias for "haswell"). You ca= n > also specify a lower CPUTYPE to build the world that you are going to > run inside QEMU. >=20 I have a standard config for my laptops, so that's why it has CPUTYPE?=3Dnative --gWvtJaQcvJSV6iXno68GOik4fiAJHBUSm-- --6O1qIcNBSMugx293PD2wOTgK1r00q2PgF Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEzzhiNNveVG6nWjcH1w0wQIFco2cFAljj5zUACgkQ1w0wQIFc o2d2fQ/+O2WaXq7gEgeHeNZuea7n6z7VbE1oAKey93wP8bHoJslIHUYcKX68DQK+ Pw5medo2hHLuvJhgdkRDiG8LyvwBXO6VIaHWUTU+jRKtC+DC1u97pr3LplAciWZl bYlBT1QdUaNx6tWI/W8m7J5vc6IOSHTwdpzwXPIbWAqv9mTrKWFH3KwLk5LUyQhC mdCjoFr/WbTGzRgRtU4nlja7g54hSZ3KV/9RJ/ill4BvXjuNeq+jIqAyYicp1aqW uvePggvJo46RihrLoEdsiyOfWmf62ZrlqSl37zEQ1yEKVsIs9plbZDlJxFlt35jy euORjA6z/5hOZEyzWtbHKntM2XzR88jQAxXP22ghZbVNM7AGhnep5MbvpsiZMn86 ginb2busEhEWi9ANqWXCRKzHIIs7GjB3sEBsIW8rmTLJ5Z4kXjOreVFWGxa52H4b 5xMTiGp/gq48RYYAIHdVVjd0yrKTMRgvPG+ExKb/I8I5Mjws8pg43qOQvBEAvZk0 gDkBfznCXCe2l5fnWQWJCZIDW1H/yRkEAd3OuKz0Vk40bVx7zfsgT0xzjDAMjR0d jCfmpOWyFMCI7UlVC+AAKLcdPeEEJOs6D8PLN++sC+b9/i8OdtFIgfPzqZFP6Tir aEy+gsygtsoAqRLMjt8lHh0JhyAXbrBjVKf0Rx7+YxrMut+jXa4= =Tw7J -----END PGP SIGNATURE----- --6O1qIcNBSMugx293PD2wOTgK1r00q2PgF-- From owner-freebsd-hackers@freebsd.org Wed Apr 5 01:07:22 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8937DD2D45D for ; Wed, 5 Apr 2017 01:07:22 +0000 (UTC) (envelope-from jaguar.darocha@yandex.com) Received: from forward10h.cmail.yandex.net (forward10h.cmail.yandex.net [IPv6:2a02:6b8:0:f35::ec]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Yandex CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4565E30B; Wed, 5 Apr 2017 01:07:21 +0000 (UTC) (envelope-from jaguar.darocha@yandex.com) Received: from mxback4g.mail.yandex.net (mxback4g.mail.yandex.net [IPv6:2a02:6b8:0:1472:2741:0:8b7:165]) by forward10h.cmail.yandex.net (Yandex) with ESMTP id 3ADCB22377; Wed, 5 Apr 2017 04:07:09 +0300 (MSK) Received: from web45g.yandex.ru (web45g.yandex.ru [95.108.252.215]) by mxback4g.mail.yandex.net (nwsmtp/Yandex) with ESMTP id U5L4zhLoWK-78MSX38l; Wed, 05 Apr 2017 04:07:08 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.com; s=mail; t=1491354428; bh=+VDfbJLXzQDP8f8ISf+Oe9uyhfuiKGJjfZND+JPgiZs=; h=From:To:Cc:In-Reply-To:References:Subject:Message-Id:Date; b=dtfGt+WPJRZk/Axh6NF7O9ap+JHj8jKuSq50SCFOeasE/Ac51m7vvCmFSff9C+abR EJp6AzMUZvqk0KoChco9qJN2vQJ0BW7xYc3c/8h/fn7dOFGxkVes7tAdZZdxuWH4EG iGY53o0l/En4y4hMIdRiSNlSrRkh3uM4M8aBLWsc= Authentication-Results: mxback4g.mail.yandex.net; dkim=pass header.i=@yandex.com Received: by web45g.yandex.ru with HTTP; Wed, 05 Apr 2017 04:07:08 +0300 From: Jaguar DaRocha Envelope-From: jaguar-darocha@yandex.com To: Eric McCorkle Cc: "freebsd-hackers@freebsd.org" , Dimitry Andric In-Reply-To: <25c71912-7eec-a174-9d9f-50280c3435e8@metricspace.net> References: <25c71912-7eec-a174-9d9f-50280c3435e8@metricspace.net> Subject: Re:Source of QEMU woes: CPUTYPE MIME-Version: 1.0 Message-Id: <4098801491354428@web45g.yandex.ru> X-Mailer: Yamail [ http://yandex.ru ] 5.0 Date: Tue, 04 Apr 2017 19:07:08 -0600 Content-Transfer-Encoding: 7bit Content-Type: text/plain X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Apr 2017 01:07:22 -0000 There is a mailing list for qemu. Have you tried there? > On 04/04/2017 14:31, Dimitry Andric wrote: > >> On 4 Apr 2017, at 14:34, Eric McCorkle wrote: >> >> Out of interest, what does "llvm-tblgen -version | grep 'Host CPU'" >> show? (This is a simple way to see what LLVM auto-detects.) > > broadwell > >>> I'm posting this here, as it's somewhat non-obvious, and probably ought >>> to be documented somewhere. >> >> I usually find it clearer to specify the exact CPU type myself, for >> example CPUTYPE?=core-avx2 (which is an alias for "haswell"). You can >> also specify a lower CPUTYPE to build the world that you are going to >> run inside QEMU. > > I have a standard config for my laptops, so that's why it has > CPUTYPE?=native From owner-freebsd-hackers@freebsd.org Wed Apr 5 01:56:59 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 80F12D2E8E1; Wed, 5 Apr 2017 01:56:59 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from dmz-mailsec-scanner-4.mit.edu (dmz-mailsec-scanner-4.mit.edu [18.9.25.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 18C3022A; Wed, 5 Apr 2017 01:56:58 +0000 (UTC) (envelope-from kaduk@mit.edu) X-AuditID: 1209190f-077ff700000002bc-1f-58e44ede61eb Received: from mailhub-auth-2.mit.edu ( [18.7.62.36]) (using TLS with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by (Symantec Messaging Gateway) with SMTP id A4.06.00700.EDE44E85; Tue, 4 Apr 2017 21:56:47 -0400 (EDT) Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by mailhub-auth-2.mit.edu (8.13.8/8.9.2) with ESMTP id v351uj8s019356; Tue, 4 Apr 2017 21:56:46 -0400 Received: from kduck.kaduk.org (24-107-191-124.dhcp.stls.mo.charter.com [24.107.191.124]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id v351ugs6007819 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Tue, 4 Apr 2017 21:56:45 -0400 Date: Tue, 4 Apr 2017 20:56:42 -0500 From: Benjamin Kaduk To: freebsd-hackers@FreeBSD.org Cc: freebsd-current@FreeBSD.org Subject: FINAL CALL for 2017Q1 quarterly status reports Message-ID: <20170405015641.GG30306@kduck.kaduk.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.6.1 (2016-04-27) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrHIsWRmVeSWpSXmKPExsUixG6nonvf70mEQXObrsWcNx+YLLZv/sfo wOQx49N8lgDGKC6blNSczLLUIn27BK6Mtf92shVM5anYv3wXawNjJ1cXIyeHhICJxOnpB1m6 GLk4hATamCSat/xkh3A2MEo8ez6LEcK5wiRx8OsMFpAWFgEVifZv3awgNpuAmsT6FdeYQWwR AXmJfU3v2UFsZiD719YmMFtYwFzi1bxvTCA2L9C6C0/+sUPYghInZz5hgajXkrjx7yVQDQeQ LS2x/B8HSFhUQFmiYcYD5gmMfLOQdMxC0jELoWMBI/MqRtmU3Crd3MTMnOLUZN3i5MS8vNQi XRO93MwSvdSU0k2MoGDjlOTfwTinwfsQowAHoxIPb8W0xxFCrIllxZW5hxglOZiURHkVfJ5E CPEl5adUZiQWZ8QXleakFh9ilOBgVhLhzVMCyvGmJFZWpRblw6SkOViUxHnFNRojhATSE0tS s1NTC1KLYLIyHBxKErwffIEaBYtS01Mr0jJzShDSTBycIMN5gIbngtTwFhck5hZnpkPkTzEq SonzTgZJCIAkMkrz4HpByUAie3/NK0ZxoFeEefNBqniAiQSu+xXQYCagwU/uPAQZXJKIkJJq YKxYJthm2nh/9dZ/T5T3H5W9wdfdfHbZyUWrhN6uCM9rCN/xIjvqdfi0q49u5qc1qb6dHqn3 omzxglNfLvAcv23wNGvn2rarTO/z95wx4Upf5Zq9dHLziTAfgZkp75frH5atXSvYMemaSu39 DCeDuiJd/e9pK6XzBValchgsYvv3dkpdx2E3vhIlluKMREMt5qLiRABKfzL+4QIAAA== X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Apr 2017 01:56:59 -0000 Dear FreeBSD Community, The deadline for the next FreeBSD Quarterly Status update is April 7, 2017, for work done in January through March. Status report submissions do not need to be very long. They may be about anything happening in the FreeBSD project and community, and provide a great way to inform FreeBSD users and developers about work that is underway and completed. Submission of reports is not restricted to committers; anyone doing anything interesting and FreeBSD related can -- and should -- write one! The preferred and easiest submission method is to use the XML generator [1] with the results emailed to the status report team at monthly@FreeBSD.org . (Do be sure, though, to save the form output and not the form itself!) There is also an XML template [2] that can be filled out manually and attached if preferred. For the expected content and style, please study our guidelines on how to write a good status report [3]. You can also review previous issues [4][5] for ideas on the style and format. Please let us know if you are interested in submitting an entry, but are worried about being able to make the deadline. We look forward to seeing your 2017Q1 reports! Thanks, Ben (on behalf of monthly@) [1] https://www.FreeBSD.org/cgi/monthly.cgi [2] https://www.FreeBSD.org/news/status/report-sample.xml [3] https://www.FreeBSD.org/news/status/howto.html [4] https://www.FreeBSD.org/news/status/report-2016-07-2016-09.html [5] https://www.FreeBSD.org/news/status/report-2016-10-2016-12.html From owner-freebsd-hackers@freebsd.org Wed Apr 5 03:00:49 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 82461D2FE9D for ; Wed, 5 Apr 2017 03:00:49 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-82.reflexion.net [208.70.210.82]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48794CAD for ; Wed, 5 Apr 2017 03:00:48 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 5825 invoked from network); 5 Apr 2017 03:00:47 -0000 Received: from unknown (HELO rtc-sm-01.app.dca.reflexion.local) (10.81.150.1) by 0 (rfx-qmail) with SMTP; 5 Apr 2017 03:00:47 -0000 Received: by rtc-sm-01.app.dca.reflexion.local (Reflexion email security v8.40.0) with SMTP; Tue, 04 Apr 2017 23:00:47 -0400 (EDT) Received: (qmail 14817 invoked from network); 5 Apr 2017 03:00:47 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 5 Apr 2017 03:00:47 -0000 Received: from [192.168.1.119] (c-67-170-167-181.hsd1.or.comcast.net [67.170.167.181]) by iron2.pdx.net (Postfix) with ESMTPSA id 83DDCEC88F3; Tue, 4 Apr 2017 20:00:46 -0700 (PDT) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: The arm64 fork-then-swap-out-then-swap-in failures: a program source for exploring them Message-Id: <4DEA2D76-9F27-426D-A8D2-F07B16575FB9@dsl-only.net> Date: Tue, 4 Apr 2017 20:00:45 -0700 To: freebsd-arm , freebsd-hackers@freebsd.org X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Apr 2017 03:00:49 -0000 Uncommenting/commenting parts of the below program allows exploring the problems with fork-then-swap-out-then-in on arm64. Note: By swap-out I mean that zero RES(ident memory) results, for the process(s) of interest, as shown by "top -PCwaopid" . I discovered recently that swapping-out just before the fork() prevents the failure from the swapping after the fork(). Note: Without the fork() no problem happens. Without the later swap-out no problem happens. Both are required. But some activities before the fork() or between fork() and the swap-out prevent the failures. Some of the comments are based on a pine64+ 2GB context. I use stress to force swap-outs during some sleeps in the program. See also Buzilla 217239 and 217138. (I now expect that they have the same cause.) In my environment I've seen the fork-then-swap-out/swap-in failures on a pine64+ 2GB and a rpi3. They are repeatable on both. I do not have access to server-class machines, or any other arm64 machines. // swap_testing5.c // Built via (cc was clang 4.0 in my case): // // cc -g -std=3Dc11 -Wpedantic -o swaptesting5 swap_testing5.c // -O0 and -O2 also gets the problem. // Note: jemalloc's tcache needs to be enabled to get the failure. // But FreeBSD can get into a state were /etc/malloc.conf // -> 'tcache:false' is ineffective. Also: the allocation // size needs to by sufficiently small (<=3D SMALL_MAXCLASS) // to see the problem. Other comments are based on a specific // context (pine64+ 2GB). #include // for raise(.), SIGABRT (induce core dump) #include // for fork(), sleep(.) #include // for pid_t #include // for wait(.) extern void test_setup(void); // Sets up the memory byte = patterns. extern void test_check(void); // Tests the memory byte patterns. extern void memory_willneed(void); // For seeing if // = posix_madvise(.,.,POSIX_MADV_WILLNEED) // makes a difference. int main(void) { sleep(30); // Potentialy force swap-out here. // [Swap-out here does not avoid later failures.] test_setup(); test_check(); // Before potential sleep(.)/swap-out or fork(.) = [passes] sleep(30); // Potentialy force swap-out here. // [Everything below passes if swapped-out here, // no matter if there are later swap-outs // or not.] pid_t pid =3D fork(); // To test no-fork use: =3D 0; no-fork does = not fail. int wait_status =3D 0; // HERE: After fork; before sleep/swap-out/wait. // if (0 < pid) memory_willneed(); // Does not prevent either = parent or // child failure if enabled. // if (0 =3D=3D pid) memory_willneed(); // Prevents both the parent = and the // child failure. Disable to see // failure of both parent and = child. // [Presuming no prior swap-out: = that // would make everything pass.] // During sleep/wait: manually force this process to // swap out. I use something like: // stress -m 1 --vm-bytes 1800M // in another shell and ^C'ing it after top shows the // swapped status desired. 1800M just happened to work // on the Pine64+ 2GB that I was using. I watch with // top -PCwaopid [checking for zero RES(ident memory)]. if (0 < pid) { sleep(30); // Intend to swap-out during sleep. // test_check(); // Test in parent before child runs (longer = sleep). // This test fails if run for a failing = region_size // unless earlier preventing-activity happened. wait(&wait_status); // Only if test_check above passes or is // disabled above. } if (-1 !=3D wait_status && 0 <=3D pid) { if (0 =3D=3D pid) { sleep(90); } // Intend to swap-out during = sleep. test_check(); // Fails for small-enough region_size, both // parent and child processes, unless earlier // preventing-activty happened. } } // The memory and test code follows. #include // for size_t, NULL #include // for malloc(.), free(.) #include // for POSIX_MADV_WILLNEED, posix_madvise(.,.,.) #define region_size (14u*1024u) // Bad dyn_region pattern, parent and child processes examples: // 256u, 2u*1024u, 4u*1024u, 8u*1024u, 9u*1024u, 12u*1024u, = 14u*1024u // No failure examples: // 14u*1024u+1u, 15u*1024u, 16u*1024u, 32u*1024u, = 256u*1024u*1024u #define num_regions (256u*1024u*1024u/region_size) typedef volatile unsigned char value_type; struct region_struct { value_type array[region_size]; }; typedef struct region_struct region; static region * volatile dyn_regions[num_regions] =3D {NULL,}; static value_type value(size_t v) { return (value_type)((v&0xFEu)|0x1u); = } // value avoids zero values: the bad values are zeros. void test_setup(void) { for(size_t i=3D0u; i Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 75351D30748 for ; Wed, 5 Apr 2017 16:22:19 +0000 (UTC) (envelope-from lobo@bsd.com.br) Received: from mail-qk0-x241.google.com (mail-qk0-x241.google.com [IPv6:2607:f8b0:400d:c09::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3C6A3DA5 for ; Wed, 5 Apr 2017 16:22:18 +0000 (UTC) (envelope-from lobo@bsd.com.br) Received: by mail-qk0-x241.google.com with SMTP id p68so2352696qke.1 for ; Wed, 05 Apr 2017 09:22:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsd.com.br; s=capeta; h=date:from:to:cc:subject:message-id:organization:mime-version :content-transfer-encoding; bh=tPwpl3z/i6Jz6IXtSgCjuDCYWD+g5+I+FLmZp5BgRwo=; b=ddbtsvoOJDJLGJmGCk4gBrnhWHyNu5DNuhixCov4rE6TRgX8/+odcQJ9iSR/j6KQpl MLV1vhsAa65AZVC3NktZ82Ft90/WjhvrW/ZV2W5W4gkZpZORu11iHBiWPB3Jrud7ggmn F0RH5KTI/k+E6BdDwyePx3oYmDfp+P8Oz3+WY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:organization :mime-version:content-transfer-encoding; bh=tPwpl3z/i6Jz6IXtSgCjuDCYWD+g5+I+FLmZp5BgRwo=; b=Znn6xzQZrDaIt62t95R89MSCiANvR9aIH9U5LOdMPJmefka/rm6ohk72SpuOr6FeR1 6MqC+mglwIQTei4ZZ34DlkE65tGxjy9imQNdprud81mlWvCio1Yc97WNiHzL+JmUjAIs Hd1VT46Wvvmk9j2zVAclt0UgTp0Y2Rx7t+Xtg8Wxu3XeeihsF0ffQ0TA2hr6Jr3rIYNF SdAID38tBGBU523ABJtkjYqpw/kdq5YjJPlkOdz2bFixuGKHW5f9ec9VQX2dS64G0yKX vpzRMN8R+j7XzIrYgzfKZd5OM/vephD67bkdW82cZlWXJITY5r7aVffwZvKiyaCscqX3 Jl8Q== X-Gm-Message-State: AFeK/H0uIZDZIJuhI3fvw30AOV3QYeSqOea6+THM42dNXQF7j25bKnMpyR4PVrgz6kVt8Q== X-Received: by 10.55.168.11 with SMTP id r11mr30608858qke.239.1491409337833; Wed, 05 Apr 2017 09:22:17 -0700 (PDT) Received: from Papi ([177.98.129.212]) by smtp.gmail.com with ESMTPSA id d142sm11069684qkc.32.2017.04.05.09.22.16 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 05 Apr 2017 09:22:17 -0700 (PDT) Date: Wed, 5 Apr 2017 13:23:16 -0300 From: Mario Lobo To: freebsd-questions@freebsd.org Cc: FreeBSD Hackers Subject: [OFF-TOPIC] C question Message-ID: <20170405132251.536ab064@Papi> Organization: BSD X-Mailer: Claws Mail 3.14.1 (GTK+ 2.24.29; amd64-portbld-freebsd11.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Apr 2017 16:22:19 -0000 Hi There ! I don't know if this list is appropriate for this.=20 if it isn't, please point me to the right direction. Since this is for a Freebsd program, I decided to start here, hoping to find some C gurus who can help me out. Here it goes: 1) I have this; typedef struct { ulong me; ulong record; char string[256]; }KFNODE; KFNODE ***Nodes; 2) Then allocate for the pointers; Nodes =3D (KFNODE ***) malloc(5 * sizeof(KFNODE **)); memset( Nodes, 0x00, (5 * sizeof(KFNODE **))); =20 for(a =3D 0; a < 5; a++) { Nodes[a] =3D (KFNODE **) malloc(43 * sizeof(KFNODE *)); memset( Nodes[a], 0x00, (43 * sizeof(KFNODE *))); } 3) Allocate for the structs for(a =3D 0; a < 5; a++) { for(b =3D 0; b < 43; b++) {=20 Nodes[a][b] =3D (KFNODE *) malloc(sizeof(KFNODE)); memset( Nodes[a][b], 0x00, sizeof(KFNODE)); } } =46rom this point on, I can access any Nodes[a<5][b<43]. This works fine as long as I previously know is a 2D array. My question is: The dimensions are not known before hand and have to be found at run time. Suppose I find out I'll need a 3D array, like Nodes[a][b][c]? How can I dynamically change the level of indirection? How can I dynamically switch: ---------------- KFNODE ***Nodes; to KFNODE ****Nodes; ---------------- Nodes =3D (KFNODE ***) malloc(5 * sizeof(KFNODE **)); to Nodes =3D (KFNODE ****) malloc(5 * sizeof(KFNODE ***)); ---------------- and so forth? Is this possible at all? Thanks --=20 Mario Lobo http://www.mallavoodoo.com.br FreeBSD since 2.2.8 [not Pro-Audio.... YET!!] =20 "UNIX was not designed to stop you from doing stupid things,=20 because that would also stop you from doing clever things." From owner-freebsd-hackers@freebsd.org Wed Apr 5 23:11:54 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AFDC1D306EC for ; Wed, 5 Apr 2017 23:11:54 +0000 (UTC) (envelope-from bsd-lists@bsdforge.com) Received: from udns.ultimatedns.net (static-24-113-41-81.wavecable.com [24.113.41.81]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7A6D19DE for ; Wed, 5 Apr 2017 23:11:53 +0000 (UTC) (envelope-from bsd-lists@bsdforge.com) Received: from ultimatedns.net (localhost [127.0.0.1]) by udns.ultimatedns.net (8.14.9/8.14.9) with ESMTP id v35ND3s4083989 for ; Wed, 5 Apr 2017 16:13:09 -0700 (PDT) (envelope-from bsd-lists@bsdforge.com) To: In-Reply-To: References: <481fef207e2e457cb4f8a689d0ce4373@SERVER.ad.usd-group.com> <20170403235624.GA94548@FreeBSD.org>, From: "Chris H" Subject: Re: Sendmail conf files Date: Wed, 05 Apr 2017 16:13:09 -0700 Content-Type: text/plain; charset=UTF-8; format=fixed MIME-Version: 1.0 Message-id: <0ce8655a2bcddc3b5b0e226fa90fd1f8@ultimatedns.net> Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Apr 2017 23:11:54 -0000 On Tue, 4 Apr 2017 08:14:18 +0000 Matt Churchyard via freebsd-hackers wrote > ----- Matt Churchyard's Original Message ----- > > Hello, > > > > Not sure if this is the right list for this.. > > > > For me the most awkward part of updating a system using freebsd-update is > > when it comes to merging files. The most common files that pop up seem to be > > /etc/mail/sendmail.cf & /etc/mail/submit.cf, because these are included in > > the base distribution and almost certainly change if you actually use > > Sendmail. > > > Simply don't use the same names.... in /etc/make.conf: > > > SENDMAIL_MC = /etc/mail/prodconf.mc > > > and the .cf file(s) will be regenerated. > > >YMMV, > >Cheers > > I appreciate the response and could use that method on my own systems > (although I probably won't need to now as I pretty much always use Postfix). > It doesn't really solve the original issue though. > > It's not exactly ideal to tell all users they should edit /etc/make.conf just > so they can use Sendmail and not have to mess about with cf files on every > upgrade. (To be honest I got to the point where I used to just accept the > merge even though it left errors in the config file, then went and ran > /etc/mail/make to re-create everything. For a user that's easier that trying > to merge files in a terminal editor). As I said there is/has been a large > effort to make sure users don't have to edit base config files in FreeBSD so > that base can be updated without affecting user config or requiring merging. > Unless there's some awkward issue I'm not aware of, I don't really see why > the two .cf files are included in base in the first place, rather than > generated at runtime. > > I didn't really post this to fix my own problem, I'm just trying to suggest > something that may make upgrades easier for everyone. When running > freebsd-update these cf files are some of the very few files that regularly > require manual intervention. > > Matt Include it/them in /etc/mergemaster.rc, as IGNORE_FILES? eg; IGNORE_FILES='/etc/mail/sendmail.cf /etc/mail/submit.cf /some/other/file' Just a thought && HTH! --Chris > > > In most cases this sort of issue has been solved by providing "default > > files" such as /etc/defaults/rc.conf, and letting the user override this with > > files they create themselves. Obviously there is a concerted effort to make > > sure users don't have to edit base files where possible so that they don't > > get these sort of issues. > > > > For Sendmail, would it not make sense to remove these 2 .cf files from base > > and update the sendmail rc.d script to run 'make install' in /etc/mail if > > they don't exist? It may also be nice if the freebsd.* files were stored > > somewhere else such as /usr/share/sendmail, as these just causes confusion > > about which files are actually used if you're new to it. > > > > Personally I'm on the side that would rather have Sendmail removed entirely > > and replaced with a simple smtp submission daemon/lda but I think that > > discussion has already been had. > > > > - > > > > Matt Churchyard From owner-freebsd-hackers@freebsd.org Thu Apr 6 00:30:18 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 87A2BD30D5A for ; Thu, 6 Apr 2017 00:30:18 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from mx1.scaleengine.net (mx1.scaleengine.net [209.51.186.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CF6461 for ; Thu, 6 Apr 2017 00:30:17 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from [10.1.1.2] (unknown [10.1.1.2]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id CC561D009 for ; Thu, 6 Apr 2017 00:30:10 +0000 (UTC) Subject: Re: Sendmail conf files To: freebsd-hackers@freebsd.org References: <481fef207e2e457cb4f8a689d0ce4373@SERVER.ad.usd-group.com> From: Allan Jude Message-ID: <24d5d278-0e95-15bb-f6a1-7132c861ce4e@freebsd.org> Date: Wed, 5 Apr 2017 20:29:54 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <481fef207e2e457cb4f8a689d0ce4373@SERVER.ad.usd-group.com> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="fRlBBNMXXmXJ4128MKn32pwcmchE2BHiX" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Apr 2017 00:30:18 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --fRlBBNMXXmXJ4128MKn32pwcmchE2BHiX Content-Type: multipart/mixed; boundary="ncqiBbPimLdILGhuueXVELNBgnN8iWcX1"; protected-headers="v1" From: Allan Jude To: freebsd-hackers@freebsd.org Message-ID: <24d5d278-0e95-15bb-f6a1-7132c861ce4e@freebsd.org> Subject: Re: Sendmail conf files References: <481fef207e2e457cb4f8a689d0ce4373@SERVER.ad.usd-group.com> In-Reply-To: <481fef207e2e457cb4f8a689d0ce4373@SERVER.ad.usd-group.com> --ncqiBbPimLdILGhuueXVELNBgnN8iWcX1 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 2017-03-30 15:19, Matt Churchyard via freebsd-hackers wrote: > Hello, >=20 >=20 > Not sure if this is the right list for this.. >=20 >=20 > For me the most awkward part of updating a system using freebsd-update = is when it comes to merging files. The most common files that pop up seem= to be /etc/mail/sendmail.cf & /etc/mail/submit.cf, because these are inc= luded in the base distribution and almost certainly change if you actuall= y use Sendmail. >=20 >=20 > In most cases this sort of issue has been solved by providing "default = files" such as /etc/defaults/rc.conf, and letting the user override this = with files they create themselves. Obviously there is a concerted effort = to make sure users don't have to edit base files where possible so that t= hey don't get these sort of issues. >=20 >=20 > For Sendmail, would it not make sense to remove these 2 .cf files from = base and update the sendmail rc.d script to run 'make install' in /etc/ma= il if they don't exist? It may also be nice if the freebsd.* files were s= tored somewhere else such as /usr/share/sendmail, as these just causes co= nfusion about which files are actually used if you're new to it. >=20 >=20 > Personally I'm on the side that would rather have Sendmail removed enti= rely and replaced with a simple smtp submission daemon/lda but I think th= at discussion has already been had. >=20 >=20 > - >=20 > Matt Churchyard > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.o= rg" >=20 I have a background project going on to replace as many config files as I can with new ones written in a common syntax called UCL (Universal Config Language, what pkg uses). The idea will be to have a defaults file, then override it with your own file, and/or a directory of fragments (/etc/foo.d/*.conf). Sendmail is not on the list to get fixed, but most other tools are. --=20 Allan Jude --ncqiBbPimLdILGhuueXVELNBgnN8iWcX1-- --fRlBBNMXXmXJ4128MKn32pwcmchE2BHiX Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJY5YwHAAoJEBmVNT4SmAt+Y/sP/3O+KX3u+w0XPTW7khnW3BeE WWAh90VIJuTZlLgfekR7zvo+f0GyX7++gNJ+JOz/A17bF7+qHZaheVpXL1dCkxpf zX/nUGZRdxqo7Ag1XagwJR/nSTgyMGAVf8AITut1XKRtgVoB+qt5w0aGMcAub6Yc d367WKywHYsnPxuqvMZxAiAIHpZffwj0GTWT5sQZF/4j9HkFUrZZFckf84kuPJwH m2nWS6e2P//QlEcU7j2OdAGW4qhp6r8b2hQi0oIRGyKDb2oxnqvf5nzKyVmmfbLt OZbdpm2Oc++rLQIMjrQurVvuInBPotvgf5nCIkMlZ9clkN9BaMG+AD5umXOBplJW e9nbs1+8/PTx4Sd3T+Y+iy3JAu+lRGGlfN8WUyzH5vQ5EHCdmAAf/fJ3xCUhBjAd 4Qsy7d5AelQBIQaHcr0uIV1bkqUb+4Dy4FLZTuutLA6NI58nM4ln6OTFB985Qc/7 fQTGAUjIKEGWdwOalFVlogHLk1fxC/XxIkGnCRQa3vMS1QQT5cGPOVOLHtgeBpQP XF37Fn2rqNk3lNuakJ7heC+gzvdbj3L406MGW/VIzGwR28xrXA/bMIfH5ELixRUl Sf29e77sFmHVgo1xNZPk7+6Se+ZB3q2cHUPFTVn7DzaaOnYk0WJ4nl6OFyoCTzMX SFZ+gRoYmOot39Atdmbi =pNVp -----END PGP SIGNATURE----- --fRlBBNMXXmXJ4128MKn32pwcmchE2BHiX-- From owner-freebsd-hackers@freebsd.org Thu Apr 6 00:58:55 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B398BD31579; Thu, 6 Apr 2017 00:58:55 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from dmz-mailsec-scanner-6.mit.edu (dmz-mailsec-scanner-6.mit.edu [18.7.68.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4F62B14C; Thu, 6 Apr 2017 00:58:54 +0000 (UTC) (envelope-from kaduk@mit.edu) X-AuditID: 12074423-9c3ff700000003ba-87-58e591983499 Received: from mailhub-auth-1.mit.edu ( [18.9.21.35]) (using TLS with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by (Symantec Messaging Gateway) with SMTP id 51.B5.00954.89195E85; Wed, 5 Apr 2017 20:53:45 -0400 (EDT) Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) by mailhub-auth-1.mit.edu (8.13.8/8.9.2) with ESMTP id v360rh1U031895; Wed, 5 Apr 2017 20:53:44 -0400 Received: from kduck.kaduk.org (24-107-191-124.dhcp.stls.mo.charter.com [24.107.191.124]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id v360rc4Z005338 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 5 Apr 2017 20:53:42 -0400 Date: Wed, 5 Apr 2017 19:53:38 -0500 From: Benjamin Kaduk To: Mario Lobo Cc: freebsd-questions@freebsd.org, FreeBSD Hackers Subject: Re: [OFF-TOPIC] C question Message-ID: <20170406005338.GR30306@kduck.kaduk.org> References: <20170405132251.536ab064@Papi> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170405132251.536ab064@Papi> User-Agent: Mutt/1.6.1 (2016-04-27) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrHIsWRmVeSWpSXmKPExsUixCmqrDtz4tMIg687ZC22b/7HaPHy6yYW i8/rVzA6MHtsnf6H1WPGp/ksAUxRXDYpqTmZZalF+nYJXBnLP6xhLGjjrdjVHNnAuJSri5GT Q0LAROLD6UPsXYxcHEICbUwSx3e8YIVwNjBKXPnxiwXCucIkcX7jZiaQFhYBFYnNfUeYQWw2 ILuh+zKYLSKgIDHp1QY2EJtZIFJi1uHVYPXCAsoSrz43AtkcHLxA6+6ezQcJCwloSZy8dgis lVdAUOLkzCcsEK1aEjf+vQQrZxaQllj+jwMkzCmgLdG66yEjiC0KNLFhxgPmCYwCs5B0z0LS PQuhewEj8ypG2ZTcKt3cxMyc4tRk3eLkxLy81CJdM73czBK91JTSTYygYGV3Ud7B+LLP+xCj AAejEg+vx+MnEUKsiWXFlbmHGCU5mJREeRV8gEJ8SfkplRmJxRnxRaU5qcWHGCU4mJVEeF+0 P40Q4k1JrKxKLcqHSUlzsCiJ84prNEYICaQnlqRmp6YWpBbBZGU4OJQkeAMmADUKFqWmp1ak ZeaUIKSZODhBhvMADe8CqeEtLkjMLc5Mh8ifYlSUEue92g+UEABJZJTmwfWCkolE9v6aV4zi QK8I8x4CaecBJiK47ldAg5mABj+58xBkcEkiQkqqgVH2+5yMO+Ehmvz9Vydk9lTulpGd7sV9 +aGXrElc324hm+5b66dpitmyZauYZczX/s9yRGjC509Z18REeQQqeBeUSV/t+XOunpdta3eo iEX/nD+HtmhXa6oqSkz8YRKxtco2sfRG7uelZ248mGb/fM0Mq90dlY+CZbaX5DQt1tLZxNR9 dbcgrxJLcUaioRZzUXEiAD5vuYkBAwAA X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Apr 2017 00:58:55 -0000 On Wed, Apr 05, 2017 at 01:23:16PM -0300, Mario Lobo wrote: > Hi There ! > > I don't know if this list is appropriate for this. > > if it isn't, please point me to the right direction. It would probably be more appropriate on something like StackOverflow. > > How can I dynamically change the level of indirection? > > > How can I dynamically switch: > > ---------------- > KFNODE ***Nodes; to KFNODE ****Nodes; > ---------------- > Nodes = (KFNODE ***) malloc(5 * sizeof(KFNODE **)); > to Nodes = (KFNODE ****) malloc(5 * sizeof(KFNODE ***)); > ---------------- > > and so forth? > > Is this possible at all? It is a rather unnatural thing to want to do, so the solution would also be rather unnatural. It seems, though, that you could have a loop construct that uses void* and void** for all levels except for the leaf, since the underlying property of the non-leaf allocations is that they are to hold pointers. As you prepare to go to the next layer of indirection you cast the pointers from void* to void** and dereference them. Once your depth gets to the last level then you can cast he void* to KFNODE* and actually handle the leaf elements. But as I said, this is rather unusual style to do, and would require replacing the x[a][b][c] with a loop that dereferences successive pointers. -Ben P.S. All those extra allocations for intermediate arrays add up to a fair bit of storage, and the extra cost of all the pointer indirections does, too. Sometimes it's more efficent to just allocate a "flat" array of KFDNODE[x*y*z] and manually do index arithmetic to emulate a multi-dimensional array. From owner-freebsd-hackers@freebsd.org Thu Apr 6 09:16:48 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 82A09D3087F for ; Thu, 6 Apr 2017 09:16:48 +0000 (UTC) (envelope-from ablacktshirt@gmail.com) Received: from mail-pg0-x22f.google.com (mail-pg0-x22f.google.com [IPv6:2607:f8b0:400e:c05::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4A303A3B for ; Thu, 6 Apr 2017 09:16:48 +0000 (UTC) (envelope-from ablacktshirt@gmail.com) Received: by mail-pg0-x22f.google.com with SMTP id x125so30745391pgb.0 for ; Thu, 06 Apr 2017 02:16:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=to:from:subject:message-id:date:user-agent:mime-version :content-transfer-encoding; bh=eVO+UCjZhI5XR1rXBA3ZpDqXHsskPkTAZ6a/eefct7o=; b=u9GNaI94oeOqNcY6X0wfE1j+2dGq4xoAkBb2CgV45ftQu9yL3k594JJTXYjtQMdXg4 n7FU2Cvfgylwe2c7an4iV03CE8lo91Sdx8rMwa8OrPujiWfctwnQUTqX769gH7uBuvvc uVYVo7y2pg6AOljyMNwEBJWiz1is8aWC8JQWi45E/t6IkQiXhdqhI7GW1jPqZ/ydjZC2 tSRYCuH3TpRSxP8cOGYWLzje0TDez8IFfDqNYJSqaU9w+bSnKMWQu4vBdn/qShCVxeeU vIxInh5RkLLozul0ndEJNbJIbdzcLhM1EVF6ZcCkEA5mKSUc/5EZcZvnkGeZFDSdbzvj s3Ow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:from:subject:message-id:date:user-agent :mime-version:content-transfer-encoding; bh=eVO+UCjZhI5XR1rXBA3ZpDqXHsskPkTAZ6a/eefct7o=; b=IvOgF3V7JpktknGUYv5CpDzNMNatQ1cb2ZA224wgUGjhJ1EL4ZHQ96hZ97AZANRQpB 9tNvTeFpdatTNbMsBOB7OQl1NQI18lvphvaU7TUXx8ULIt7fldkjJpEBUR2obiv6GCZ9 Hx7BZ+6zqRgMClopMallpxi5y4QiE7nLUSK6uwBaC7dbLYar4CasB37OkxBxucrkN80c Pj/ZqRn3YmHtz6g2YRIMq0OKuAFOQzcmIlCaMZSMTUawRy4NwwvxMIYN/AtnkF3w3dra wHLBnYbE+Ak3geZpFyMn2lETfihuJp6/+yD4oiFwErtQHKcNdSPZhjP29nHPAMNbTV4J TPPw== X-Gm-Message-State: AFeK/H1j3WzviPjmA/scavM8/O5FNiM/zoq8IPeeW+351VvsA7B0wHl3UMVguKaBiYaOAA== X-Received: by 10.98.75.221 with SMTP id d90mr33538287pfj.107.1491470207574; Thu, 06 Apr 2017 02:16:47 -0700 (PDT) Received: from [192.168.2.211] ([116.56.129.146]) by smtp.gmail.com with ESMTPSA id x204sm2052082pgx.63.2017.04.06.02.16.45 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 06 Apr 2017 02:16:46 -0700 (PDT) To: freebsd-hackers@freebsd.org From: Yubin Ruan Subject: Understanding the FreeBSD locking mechanism Message-ID: Date: Thu, 6 Apr 2017 17:16:45 +0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Apr 2017 09:16:48 -0000 Hi all freebsd hackers, I am reading the FreeBSD source code related to its locking mechanism. I have done some researches but still cannot understand some codes. Let's take `spinlock' for example. I know there are different kinds of mutex in FreeBSD: spin mutex and other kinds of mutex. I try to locate the source of spin mutex but what I find all look very weird to me. For example, the `spinlock_enter()` 1816 void 1817 spinlock_enter(void) 1818 { 1819 struct thread *td; 1820 register_t flags; 1821 1822 td = curthread; 1823 if (td->td_md.md_spinlock_count == 0) { 1824 flags = intr_disable(); 1825 td->td_md.md_spinlock_count = 1; 1826 td->td_md.md_saved_flags = flags; 1827 } else 1828 td->td_md.md_spinlock_count++; 1829 critical_enter(); 1830 } Does this function provides the ordinary "spinlock" functionality? There is no special "test-and-set" instruction, and neither any extra locking to protect internal data structure manipulation. Isn't this subjected to race condition? I also checked the `mtx_lock()`, but neither can't find a seemingly correct implementation. Do I miss anything? Which is the real implementation of the spin lock in FreeBSD? Thanks Yubin Ruan From owner-freebsd-hackers@freebsd.org Thu Apr 6 09:31:54 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BDE20D30FBE for ; Thu, 6 Apr 2017 09:31:54 +0000 (UTC) (envelope-from ed@nuxi.nl) Received: from mail-yb0-x22d.google.com (mail-yb0-x22d.google.com [IPv6:2607:f8b0:4002:c09::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8852B287 for ; Thu, 6 Apr 2017 09:31:54 +0000 (UTC) (envelope-from ed@nuxi.nl) Received: by mail-yb0-x22d.google.com with SMTP id i124so9087440ybc.3 for ; Thu, 06 Apr 2017 02:31:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nuxi-nl.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=eRhB0bem/7gJB/0kiC6FRHJ8I8FzCMOr2/pcdmoYpFQ=; b=LOBhCltmBRh46EkaCf0Af51cgBPPhQhboi1K5lbr1TIANJsyMvthHb9KZwgn0Q4HJU dUhSGLd6Kbm3hlepY0E7pjMTEq4RyoWyOUPj4I6Ap7ssPgkRTjxZmK4mbQ3+UHVrpw06 PJkUOl6/YneWGqCrdlx2YQYf5lXqUvDk77/VYG2DU6t3syCjGDyACobWrXJOREuaUGvC nbw7ACTqA9vGgvxdmob2kj3mMIv/TJjeZu1JUx7puFu3URIy7GVfbXHrSHNDxVeMCozm xIa1gaO5OreYIzDUFo9d+ZNH7bcHVyOya8nhdknLlT5rihx98tBQfJuf2WwM5V6++LhA zZaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=eRhB0bem/7gJB/0kiC6FRHJ8I8FzCMOr2/pcdmoYpFQ=; b=cEIbFvDwwJHOrWT13E0V38M4470gMw6eEwGnObgd6BtdLLjd7t/jM6WXXy6MpFAfWd e66V49o1WxQIMWfjFuit6Ibb+0W33IDr4NM8+3OwuFlsLfp6XFEUctovUnqPyf7J8DQu vuHkyTskWY470lnJIgyf7WcNSFWhMEuYXfW5j/Lu16je0yA0L8UEmc8Wnhh98txI3kOd THQbwPyKPUapew0v8hxyno07/QQY8+Dxr/rNTb4sDxsaqvRSXoRupll4vn6sast96A0f xImrC6FF88zeRav8IHUGn1c6Y2h0fuj9WODnAoQFD9u8zY2Qd6YLwyXzZYxRituDRL5D sqRw== X-Gm-Message-State: AFeK/H361btvKHMoNQsj0qlsAKYiX8ysyo6w71QTibRkBvkadgkZ3Zp8w9zqPm4WFYIUb2zMvRWnGP1SGMcGpg== X-Received: by 10.37.230.143 with SMTP id d137mr21197755ybh.168.1491471113353; Thu, 06 Apr 2017 02:31:53 -0700 (PDT) MIME-Version: 1.0 Received: by 10.129.51.198 with HTTP; Thu, 6 Apr 2017 02:31:23 -0700 (PDT) In-Reply-To: References: From: Ed Schouten Date: Thu, 6 Apr 2017 11:31:23 +0200 Message-ID: Subject: Re: Understanding the FreeBSD locking mechanism To: Yubin Ruan Cc: FreeBSD Hackers Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Apr 2017 09:31:54 -0000 Hi Yubin, 2017-04-06 11:16 GMT+02:00 Yubin Ruan : > Does this function provides the ordinary "spinlock" functionality? There > is no special "test-and-set" instruction, and neither any extra locking > to protect internal data structure manipulation. Isn't this subjected to > race condition? Locking a spinlock is done through macro mtx_lock_spin(), which expands to __mtx_lock_spin() in sys/sys/mutex.h. That macro first calls into the function you looked at, spinlock_enter(), to disable interrupts. It then calls into the _mtx_obtain_lock_fetch() to do the test-and-set operation you were looking for. Best regards, -- Ed Schouten Nuxi, 's-Hertogenbosch, the Netherlands KvK-nr.: 62051717 From owner-freebsd-hackers@freebsd.org Fri Apr 7 08:43:33 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0CFF5D33148 for ; Fri, 7 Apr 2017 08:43:33 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-8.reflexion.net [208.70.210.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BCAC6153B for ; Fri, 7 Apr 2017 08:43:32 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 28594 invoked from network); 7 Apr 2017 08:16:13 -0000 Received: from unknown (HELO mail-cs-02.app.dca.reflexion.local) (10.81.19.2) by 0 (rfx-qmail) with SMTP; 7 Apr 2017 08:16:13 -0000 Received: by mail-cs-02.app.dca.reflexion.local (Reflexion email security v8.40.0) with SMTP; Fri, 07 Apr 2017 04:16:13 -0400 (EDT) Received: (qmail 14788 invoked from network); 7 Apr 2017 08:16:13 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 7 Apr 2017 08:16:13 -0000 Received: from [192.168.1.106] (c-76-115-7-162.hsd1.or.comcast.net [76.115.7.162]) by iron2.pdx.net (Postfix) with ESMTPSA id 7D2D1EC8937; Fri, 7 Apr 2017 01:16:12 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: The arm64 fork-then-swap-out-then-swap-in failures: a program source for exploring them From: Mark Millard In-Reply-To: <4DEA2D76-9F27-426D-A8D2-F07B16575FB9@dsl-only.net> Date: Fri, 7 Apr 2017 01:16:12 -0700 Cc: andrew@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <163B37B0-55D6-498E-8F52-9A95C036CDFA@dsl-only.net> References: <4DEA2D76-9F27-426D-A8D2-F07B16575FB9@dsl-only.net> To: freebsd-arm , freebsd-hackers@freebsd.org X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Apr 2017 08:43:33 -0000 [I now can: (A) crudely control the number of allocated pages that get zeros (that should not). (B) Watch a "top -PCwaopid" display and predict if the test-architecture will fail or not before the fork() or swap-out happens.] On 2017-Apr-4, at 8:00 PM, Mark Millard wrote: > Uncommenting/commenting parts of the below program allows > exploring the problems with fork-then-swap-out-then-in on > arm64. >=20 > Note: By swap-out I mean that zero RES(ident memory) results, > for the process(s) of interest, as shown by > "top -PCwaopid" . >=20 > I discovered recently that swapping-out just before the > fork() prevents the failure from the swapping after the > fork(). >=20 > Note: > Without the fork() no problem happens. Without the later > swap-out no problem happens. Both are required. But some > activities before the fork() or between fork() and the > swap-out prevent the failures. >=20 > Some of the comments are based on a pine64+ 2GB context. > I use stress to force swap-outs during some sleeps in > the program. See also Buzilla 217239 and 217138. (I now > expect that they have the same cause.) >=20 > In my environment I've seen the fork-then-swap-out/swap-in > failures on a pine64+ 2GB and a rpi3. They are repeatable > on both. I do not have access to server-class machines, or > any other arm64 machines. >=20 >=20 > // swap_testing5.c >=20 > // Built via (cc was clang 4.0 in my case): > // > // cc -g -std=3Dc11 -Wpedantic -o swaptesting5 swap_testing5.c > // -O0 and -O2 also gets the problem. >=20 > // Note: jemalloc's tcache needs to be enabled to get the failure. > // But FreeBSD can get into a state were /etc/malloc.conf > // -> 'tcache:false' is ineffective. Also: the allocation > // size needs to by sufficiently small (<=3D SMALL_MAXCLASS) > // to see the problem. Other comments are based on a specific > // context (pine64+ 2GB). >=20 > #include // for raise(.), SIGABRT (induce core dump) > #include // for fork(), sleep(.) > #include // for pid_t > #include // for wait(.) >=20 > extern void test_setup(void); // Sets up the memory byte = patterns. > extern void test_check(void); // Tests the memory byte = patterns. > extern void memory_willneed(void); // For seeing if > // = posix_madvise(.,.,POSIX_MADV_WILLNEED) > // makes a difference. >=20 > int main(void) { > sleep(30); // Potentialy force swap-out here. > // [Swap-out here does not avoid later failures.] >=20 > test_setup(); > test_check(); // Before potential sleep(.)/swap-out or fork(.) = [passes] >=20 > sleep(30); // Potentialy force swap-out here. > // [Everything below passes if swapped-out here, > // no matter if there are later swap-outs > // or not.] >=20 > pid_t pid =3D fork(); // To test no-fork use: =3D 0; no-fork does = not fail. > int wait_status =3D 0; >=20 > // HERE: After fork; before sleep/swap-out/wait. >=20 > // if (0 < pid) memory_willneed(); // Does not prevent either = parent or > // child failure if enabled. >=20 > // if (0 =3D=3D pid) memory_willneed(); // Prevents both the parent = and the > // child failure. Disable to see > // failure of both parent and = child. > // [Presuming no prior swap-out: = that > // would make everything pass.] >=20 > // During sleep/wait: manually force this process to > // swap out. I use something like: > // stress -m 1 --vm-bytes 1800M > // in another shell and ^C'ing it after top shows the > // swapped status desired. 1800M just happened to work > // on the Pine64+ 2GB that I was using. I watch with > // top -PCwaopid [checking for zero RES(ident memory)]. >=20 > if (0 < pid) { > sleep(30); // Intend to swap-out during sleep. > // test_check(); // Test in parent before child runs (longer = sleep). > // This test fails if run for a failing = region_size > // unless earlier preventing-activity happened. > wait(&wait_status); // Only if test_check above passes or is > // disabled above. > } > if (-1 !=3D wait_status && 0 <=3D pid) { > if (0 =3D=3D pid) { sleep(90); } // Intend to swap-out during = sleep. > test_check(); // Fails for small-enough region_size, both > // parent and child processes, unless earlier > // preventing-activty happened. > } > } >=20 > // The memory and test code follows. >=20 > #include // for size_t, NULL > #include // for malloc(.), free(.) > #include // for POSIX_MADV_WILLNEED, = posix_madvise(.,.,.) >=20 > #define region_size (14u*1024u) > // Bad dyn_region pattern, parent and child processes examples: > // 256u, 2u*1024u, 4u*1024u, 8u*1024u, 9u*1024u, 12u*1024u, = 14u*1024u > // No failure examples: > // 14u*1024u+1u, 15u*1024u, 16u*1024u, 32u*1024u, = 256u*1024u*1024u > #define num_regions (256u*1024u*1024u/region_size) >=20 > typedef volatile unsigned char value_type; > struct region_struct { value_type array[region_size]; }; > typedef struct region_struct region; > static region * volatile dyn_regions[num_regions] =3D {NULL,}; >=20 > static value_type value(size_t v) { return = (value_type)((v&0xFEu)|0x1u); } > // value avoids zero values: the bad values are = zeros. >=20 > void test_setup(void) { > for(size_t i=3D0u; i dyn_regions[i] =3D malloc(sizeof(region)); > if (!dyn_regions[i]) raise(SIGABRT); >=20 > for(size_t j=3D0u; j (*dyn_regions[i]).array[j] =3D value(j); > } > } > } >=20 > void memory_willneed(void) { > for(size_t i=3D0u; i (void) posix_madvise(dyn_regions[i], region_size, = POSIX_MADV_WILLNEED); > } > } >=20 > static volatile size_t first_failure_idx =3D 0u; // dyn_regions index > static volatile size_t first_failure_pos =3D 0u; // sub-array index > static volatile size_t after_bad_idx =3D 0u; // dyn_regions index > static volatile size_t after_bad_pos =3D 0u; // sub-array index > static volatile size_t after_good_idx =3D 0u; // dyn_regions index > static volatile size_t after_good_pos =3D 0u; // sub-array index >=20 > // Note: Some failing cases get (conjunctive notation): > // > // 0 =3D=3D first_failure_idx < after_bad_idx < after_good_idx =3D=3D= num_regions > // && 0 =3D=3D first_failure_pos && 0<=3Dafter_bad_pos<=3Dregion_size = && after_good_idx=3D=3D0 > // && (after_bad_pos is a multiple of the page size in Bytes, here: > // after_bad_pos=3D=3DN*4096 for some non-negative integral value = N) > // > // other failing cases instead fail with: > // > // 0 =3D=3D first_failure && num_regions =3D=3D after_bad_idx =3D=3D = after_good_idx > // && 0 =3D=3D first_failure_pos =3D=3D after_bad_pos =3D=3D = after_good_idx > // > // after_bad_idx strongly tends to vary from failing run to failing = run > // as does after_bad_pos. >=20 > // Note: The working cases get: > // > // num_regions =3D=3D first_failure =3D=3D after_bad_idx =3D=3D = after_good_idx > // && 0 =3D=3D first_failure_pos =3D=3D after_bad_pos =3D=3D = after_good_idx >=20 > void test_check(void) { > first_failure_idx =3D first_failure_pos =3D 0u; >=20 > while (first_failure_idx < num_regions) { > while ( first_failure_pos < region_size > && ( value(first_failure_pos) > =3D=3D = (*dyn_regions[first_failure_idx]).array[first_failure_pos] > ) > ) { > first_failure_pos++; > } >=20 > if (region_size !=3D first_failure_pos) break; >=20 > first_failure_idx++; > first_failure_pos =3D 0u; > } >=20 > after_bad_idx =3D first_failure_idx; > after_bad_pos =3D first_failure_pos; >=20 > while (after_bad_idx < num_regions) { > while ( after_bad_pos < region_size > && ( value(after_bad_pos) > !=3D = (*dyn_regions[after_bad_idx]).array[after_bad_pos] > ) > ) { > after_bad_pos++; > } >=20 > if(region_size !=3D after_bad_pos) break; >=20 > after_bad_idx++; > after_bad_pos =3D 0u; > } >=20 > after_good_idx =3D after_bad_idx; > after_good_pos =3D after_bad_pos; >=20 > while (after_good_idx < num_regions) { > while ( after_good_pos < region_size > && ( value(after_good_pos) > =3D=3D = (*dyn_regions[after_good_idx]).array[after_good_pos] > ) > ) { > after_good_pos++; > } >=20 > if(region_size !=3D after_good_pos) break; >=20 > after_good_idx++; > after_good_pos =3D 0u; > } >=20 > if (num_regions !=3D first_failure_idx) raise(SIGABRT); > } I've found that for the above swap_testing5.c I can make variations that change how much of the allocated region prefix ends up zero vs. stays good. I vary the sleep time between testing the initialized allocations and doing the fork. The longer the sleep the more zero pages show up (be sure to read the comments): # diff swap_testing[56].c = = 1c1 < // swap_testing5.c --- > // swap_testing6.c 5c5 < // cc -g -std=3Dc11 -Wpedantic -o swaptesting5 swap_testing5.c --- > // cc -g -std=3Dc11 -Wpedantic -o swaptesting5 swap_testing6.c 33c33 < sleep(30); // Potentialy force swap-out here. --- > sleep(150); // Potentialy force swap-out here. 37a38,48 > // For no-swap-out here cases: > // > // The longer the sleep here the more allocations > // that end up as zero. > // > // top's Mem Active, Inact, Wired, Bug, Free and > // Swap Total, Used, and Free stay unchanged. > // What does change is the process RES decreases > // while the process SIZE and SWAP stay unchanged > // during this sleep. >=20 NOTE: On other architectures that I've tried (such as armv6/v7) RES does not decrease during the sleep --and the problem does not happen even for as long of sleeps as I've tried. (I use "stress -m 2 --vm-bytes 900M" on armv6/v7 instead of -m 1 --vm-bytes 1800M because that large in one process is not allowed.) So watching top's RES during the sleep (longer than a few seconds) just before the fork() predicts the later fails-vs.-not status: If RES decreases (while other things associated with the process status stay the same) then there will be a failure. At this point I've no clue why the sleeping process has a decreasing RES(ident memory) size. I infer that without the sleep there still is a small amount of loss of RES but on too short of a timescale to observe in a "top -PCwaopid" or other such: in other words that the same behavior is causing the failure then as well, possibly for a loss of only one page of RES. =3D=3D=3D Mark Millard markmi at dsl-only.net From owner-freebsd-hackers@freebsd.org Fri Apr 7 14:19:58 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 442CCD31DCB for ; Fri, 7 Apr 2017 14:19:58 +0000 (UTC) (envelope-from kevans91@ksu.edu) Received: from NAM01-BN3-obe.outbound.protection.outlook.com (mail-bn3nam01on0051.outbound.protection.outlook.com [104.47.33.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AEABB810; Fri, 7 Apr 2017 14:19:57 +0000 (UTC) (envelope-from kevans91@ksu.edu) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ksu.edu; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=+W4UZ5ovFaKZaS0IW962AaoGR+YOpLy2mG62JBoSj5M=; b=QB+rsRZRfKosV/PgV7+lgWpi1IzzMQhV8NXDcOjq/tRkRo20H1sIVmEQdtl3nyOU+/icSoG8vLekGnwCT6Ecp770om/tGdmDZ26nUYMtJyD5d6VVP5YOGilOjeVvHP9IBwoGBy7gttCTo9FyG2cVqLigWcCOhNB5pSWmpp3kBaY= Received: from BLUPR05CA0075.namprd05.prod.outlook.com (10.141.20.45) by BY1PR0501MB1110.namprd05.prod.outlook.com (10.160.103.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1019.8; Fri, 7 Apr 2017 14:19:55 +0000 Received: from CY1NAM02FT041.eop-nam02.prod.protection.outlook.com (2a01:111:f400:7e45::201) by BLUPR05CA0075.outlook.office365.com (2a01:111:e400:855::45) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1019.8 via Frontend Transport; Fri, 7 Apr 2017 14:19:55 +0000 Authentication-Results: spf=pass (sender IP is 129.130.18.151) smtp.mailfrom=ksu.edu; freebsd.org; dkim=none (message not signed) header.d=none;freebsd.org; dmarc=bestguesspass action=none header.from=ksu.edu; Received-SPF: Pass (protection.outlook.com: domain of ksu.edu designates 129.130.18.151 as permitted sender) receiver=protection.outlook.com; client-ip=129.130.18.151; helo=ome-vm-smtp2.campus.ksu.edu; Received: from ome-vm-smtp2.campus.ksu.edu (129.130.18.151) by CY1NAM02FT041.mail.protection.outlook.com (10.152.74.156) with Microsoft SMTP Server id 15.1.1019.14 via Frontend Transport; Fri, 7 Apr 2017 14:19:54 +0000 Received: from calypso.engg.ksu.edu (calypso.engg.ksu.edu [129.130.43.181]) by ome-vm-smtp2.campus.ksu.edu (8.14.4/8.14.4/Debian-2ubuntu2.1) with ESMTP id v37EJrGL018938; Fri, 7 Apr 2017 09:19:54 -0500 Received: by calypso.engg.ksu.edu (Postfix, from userid 110) id B87A024830B; Fri, 7 Apr 2017 09:19:53 -0500 (CDT) Received: from mail-wr0-f173.google.com (mail-wr0-f173.google.com [209.85.128.173]) by calypso.engg.ksu.edu (Postfix) with ESMTPA id 62DA4248305; Fri, 7 Apr 2017 09:19:51 -0500 (CDT) Received: by mail-wr0-f173.google.com with SMTP id g19so66366486wrb.0; Fri, 07 Apr 2017 07:19:51 -0700 (PDT) X-Gm-Message-State: AFeK/H0vEexzL45agHZ1W8w0ybwn1vUuOAUpgn+NBJQJVS87KeUuSGGwK7epxxLrs0APHsofaaYI9YKxjubyWQ== X-Received: by 10.223.151.80 with SMTP id r74mr13089975wrb.6.1491574790172; Fri, 07 Apr 2017 07:19:50 -0700 (PDT) MIME-Version: 1.0 Received: by 10.28.39.134 with HTTP; Fri, 7 Apr 2017 07:19:29 -0700 (PDT) From: Kyle Evans Date: Fri, 7 Apr 2017 09:19:29 -0500 X-Gmail-Original-Message-ID: Message-ID: Subject: regex(3)/grot To: CC: Ngie Cooper X-EOPAttributedMessage: 0 X-Forefront-Antispam-Report: CIP:129.130.18.151; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(39860400002)(39450400003)(39840400002)(39850400002)(39410400002)(39400400002)(2980300002)(438002)(199003)(189002)(106466001)(8936002)(8576002)(2351001)(189998001)(8676002)(54206008)(498394004)(42186005)(236005)(9686003)(6306002)(606005)(6916009)(50986999)(63696999)(54356999)(93516999)(55446002)(90966002)(88552002)(59536001)(2906002)(305945005)(512874002)(5660300001)(84326002)(61726006)(38730400002)(9896002)(110136004)(4326008)(450100002)(45336002)(86362001)(46386002)(75432002)(7906003)(356003)(55456009); DIR:OUT; SFP:1101; SCL:1; SRVR:BY1PR0501MB1110; H:ome-vm-smtp2.campus.ksu.edu; FPR:; SPF:Pass; MLV:sfv; A:1; MX:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; CY1NAM02FT041; 1:3mgEozYs1u+G10NSP6GJcKYGlycLHz/0fRqGX71a3NAaAP1KKJaI5k7Yk6/XVnSKl6g8khUqgHNC+lDCQKgmjLh8laI5BOVqr72baVkNAGNfkjadCDz9pGKF+S4zB60tOOa2iXZRMV03Ub8ToJ59k1yQIUsmjazd7bIVFLNGozqxoBHoNiMTNWeQg/wyRa7w1rCHQcUMFt3mCY2JxDnNFpHaHC6Rq84Z0CYMP/o6AtwshVCFiXzIVyqRB9Tek5XQ6Dx/+dgNbBH9OymYfGmMbEeRi65uzMFzFiHRnHvoeUbLGUGLTN8e4wioJw3+a2xaMJ5adbGK6eSYREb1uYURm1Dh1pk5s5b0jFWza/GHY54fkvnZQg2yWyZao6S9hCKKVe88JJzbhlrNVoFSiLy19BNj8DsX0A3lGzUk1Nuehx5LyXO9MmR/doIUiOoz2xh3jRY62jSWc+knVfTmRLwJk74cSg4MvmBqWf/0AwpXMzwYIvHnLl4gwj9gUOI85omj+v0dafebNPXLK6f3L3e4mKHgQ6ylGWTPRKr/v4tAHgE= X-MS-Office365-Filtering-Correlation-Id: 06b6b6d9-b513-4476-0e1f-08d47dc129a0 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(8251501002)(2017030254075)(201703131423075)(201703031133081); SRVR:BY1PR0501MB1110; X-Microsoft-Exchange-Diagnostics: 1; BY1PR0501MB1110; 3:wO3BaIpdPm+NpBJFWul5gd+tE95jo/+0CADpFQnqKTa9vE9K6raEh1l1AS7i70eSkNQl8KiCW6lMCXNI/cz0gMDyzt3qDIiztfz46AGxE/xRRVmzBCs6O9z1xZS+4mDnK68dEXK8kfGaGPGt7dwVVXxCToHA/VGVQoBU57rSyzf9qrW7RpxZOuH6vH1PbX7msNalVE2/1CDjqw9mSQDbJumUa7a0iGvCl/zQmcfezpyHaom4KHDg3IsaLqAAC2UfT9cjP6rcPoDZmTZWzcQ+fBS8Ex5LK+ePeQ/uO3LfdMqYXN9iDKmpde/RH0fJRsYPQQ0xwBeZ31aJuYQ+6AfnlR3tIbFEN/Masha1a+fIxwJyS/4v369ynaF5G6opHNpvS4FiqFYeou1x2YHk4qjOyXzIMfp2RNQQgfPbIiW3r3ya1JNW9gBJ4y8X4o4D7v2KsYJuUmOJBil+qr+oJw10rH9Mvrmw55AgC9knsCBJKh+0SIDFCHxUzjFAOPxbLEfL X-Microsoft-Exchange-Diagnostics: 1; BY1PR0501MB1110; 25:zysAe6UQ/4fqMn1YwGiWjzf2iUS/E0iIXJ3pCKsuVs4Ypwr9Ym4/DKhNZ65Mg5INIFTknTBxQx2OsgRGSNmIySFBc4YbbOGWqTek5sTDU04f4zl7v1bEPh0VOZov8Z+QA4sVRgmYDJmmr1LfCDpvNN+fJ0A2JO5GufHAxHAXwpKmz4nqpENsRbeTwBYE3wNgqgmMvGwcDbL+fb7kkkT4YutA96thjObUThvjISykq1tFIDVGk26yfiICqR23nT9QPloBG6vEIEhNgdeyMrdAlWWey7QRIbOchL+9A/YAtTtnA73WCTGLqWP6E8p+Q5dNfALkmY6miGLtIyLdQMmJ1/Htlbqnt3rMdvC5yQyCt/1m3ZMnN+4zNp2RvBRQFgKmB39TWbEyW+bjBK2dSY7eogjWJjyHQVELs3q+7ZIhXBPF0NynBTOUStnYBBuz2/+2euqo4m/JyiYw4u+01eMFmA==; 31:XDddskFb1IOJ3Domnc5r6RI2KKnwvpDJFSfYXsHpCpnwLTW734GGrAKDIue7wMnkZEMwfsC9O+Q27lfJHdL5jsHp9Vwv9EtNZmhwhaYdQnDom3Tpuxw0ezmYw6HT+Org0S4X3KbK6EMLRtahIkGFK8L02gHO8Hv2WeZS7R7bBgBKJl2f+qymcAc1FgK0Yzgx9YS1nce80qtnn952IpnivMMr0TKYU+uqo2SRsJS6tH6JMpoepbSqZQJGl+QeIZlcKccTCzHnYmXcdnDNkA6uzn3nPZMdDXcmpLvxvjdau1U= X-Microsoft-Exchange-Diagnostics: 1; BY1PR0501MB1110; 20:OVIuQlib/lF3ZIMtWDNXRoUYr9C84q4oSZHQuH6Xb8d1TQ92rOT68B7jTJXb1OQveDiUYJuJCQz0GVmhFVg6tBc2WbgQ3PRxrGY7396i0nh+5Jak6yoKgUzKNOrl4ojCgVV1FzXEs/+2kHAXQta5E32Vmu/66tfv7Buq6/qC5cG8l/QiMgx5SGk+zIAGoHjQclExFM5+2ce00ELvT71bdja+2RCiHT/DfGAPR71i/YCnIJ+CABlPr6BIZr6Uru+5xFZahnk7bunkTqJGg3ZcwO/TMeyKh5nc1z8t7qP33RdCfZe0uEcxl1WSazbCCsa5DfqF2h0Csv9WfkyN5ZAKCeI2DbiY08+WP2tHzxQiPlNOVHFWfJJB805P0Lu6A8BBlzz6nNt/gC0QDJi0iTVqv4yBq0XB7z+1iOEVBKLryTg/x4tOpmjGUxXqlhDn9qNFg44EcCSHaNhKiDc+OwmiUqaOM3tmJQDvyUFd3WiO9X7Ik3CuWUIaQxIKps+ifRLJ X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(56005881305849); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040450)(601004)(2401047)(13015025)(8121501046)(5005006)(13017025)(13024025)(13018025)(13023025)(93006095)(93004095)(10201501046)(3002001)(6041248)(20161123560025)(20161123555025)(20161123564025)(201703131423075)(201702281529075)(201702281528075)(201703061421075)(20161123562025)(6072148); SRVR:BY1PR0501MB1110; BCL:0; PCL:0; RULEID:; SRVR:BY1PR0501MB1110; X-Microsoft-Exchange-Diagnostics: 1; BY1PR0501MB1110; 4:zmBEVgyXS+6YhOohCrZWb9dQUexoxysq61IxpFMLAtC6oVnk73Z+dmOWdaXp8f/O5TkwDAVQMozmBKDdPR1lZubjwmuRTaCip9wJzZGnQ4eS55YyxqbSZOea95heJzkjcvk79NDsSacAo4vAeo83dvfpwTZsJItpxETGKsRFbSvb+tCO7UN1ywomTbM3wZr9+9qT4emjUUfzSCGomuvHdZFLzzK5UGMoHDimKAFlYKCn3znGRzR/aNxdaOKYf6WhQzXQsChXwh817ezZo/RQ8DC/VgxRa4A/dAj9lrL0MESLT+2Wo2Ma1bHdN0znWwLSUqEAVQ4q2dLF5xclDj/SuaD3LGq/6Df3u3ID64Umm+FvQYyZnWrSG1RXUUe4k5e7tqoFNxCE6Y9+9lPrq/ozWp46zXRod1ODhWiIV5hz9hG6Fp+5zo4dFlIPJwYnS+Vo/D60G6XFSbRowKeiRofEnSAWJrIrU/qr/YdibLig5xMcWV8LXLEmlhnVdAmfiDolty2UKcyuyzboWk6twpd9nsJ3uEAr8qKdiDY7/1Z2wlRMJ4RqHtOwePBDBXDJUEIJiY9shPTN3QTRChTJonO7Fg8j1fzaxAFy1paO3AvMLKpeQIBhZDISQt8cSxdV8N+dn+2ILRylF4HkugdHZW9HhisoSDSb6q5BeDhy8D+lSUd6agMm/js6lON5WFadfPTQn32T0bw6oW1tpvlSb+m6X+CG/aFEx1aW5QSjmm2MmPxGbh3LjtlV3UmlsPUdZcMhSxmnJDmOkvZI6ymy8JRFUA2P12zsCljBaTfLfBlGOVD8HBcNj3bAakHUJXSv7KZgzuRg//TRQ5iWcHTO5/4vkEcfB5B0L67XvZ0RnX+7DdSb/SZV30Q8OUh04sh6OnF4gL9q4WXr7VtZP+w3MAo1Gg== X-Forefront-PRVS: 0270ED2845 X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BY1PR0501MB1110; 23:tgJZ3d4TO5hbcZxDUZiIiGcrYg55qi03jzNlj8Y?= =?us-ascii?Q?bPC9I+De+/ChhyVJycDlogsntvWjpV5WSinIHdcoteOS13noT8HqyHNAZOsw?= =?us-ascii?Q?FWgxnbM/mhm9YLGbpleWoXXLurtKZAZKGt578ioq3V2DwY/zIQP3YCbkhULi?= =?us-ascii?Q?auzjoIfCNTctBt92+HvLT6vb33lCOiptcMmXECCSUJE2v6F6h6mLaX8JsM6f?= =?us-ascii?Q?N85s/ulBIUA984hGcU1+cHjbbek/WPiexefX0swr7HQeBoT/ZLiUHwPm4jrJ?= =?us-ascii?Q?JGo75xiUWs7nGOkNywTE8Q+Qw3XXtl0Q8DRbPeuG0B2L4mizCAuVecOwW8jR?= =?us-ascii?Q?pbgcEyIYkNP2v+uFDEoCke1rOtpm91J0L76da8BCerUqr+LFEyDG4AvnASnU?= =?us-ascii?Q?LjT0ZE7ZkBIoDUrXg8ipYkEBumZAY+467EsiZYXseStAcZfxEP6EDgG9YE1x?= =?us-ascii?Q?vwI61kN9xQMDL96sksmeAsGRT889/8s54+KY+qwuzSnyGoG27I6reLGnFS2X?= =?us-ascii?Q?2DNVZq/SdqiDrvxKWSv79P9sIXhT+Ax2FptwOz99L8HjxNX/KPni9KFRQe8Z?= =?us-ascii?Q?Xd59uMwj5lDKd4MxBnSZOUFiFzFbKiu8pRxxgP6/OMirXzU8R9rdTBYmtiX5?= =?us-ascii?Q?y7Pe7lLziabeTxbMK8/qvWcwAiECn98bW1XkuhJtA1T+wqBCl54jbJjzDnil?= =?us-ascii?Q?ch//KUvhigEbdvTJJqcgh9KWA3MXmR14wuxGSRvDP5S4S5hC7jEn+ImNcODp?= =?us-ascii?Q?OwWADEvl3U6GwoKzSCYlyvcAWVXdLMOrn0dUJuJLBBEmQilt4uKIwCDh2KcG?= =?us-ascii?Q?XtUCT/pkI4U0ZU7jgZkyfD+RwFfKKxTsqBoMELkZU6+uFWYuL8LuDANQHfy6?= =?us-ascii?Q?D76pZ1SoL357ttbTdpPfpswfYiDvo1oRJX0Ke2IuWOFM90vpEqjpjw6RXsAG?= =?us-ascii?Q?yieIvMe4rc5bPxF5aN5NKa6k5MmpDZwYQHjB3pRMbvGnwyPcwwVKNJVyK8sr?= =?us-ascii?Q?XnzeySpxTKk2HYwaqozzYnCLyZ/kjdkw8TCXGym0bgDZKZEG8Ub8ukex6Mha?= =?us-ascii?Q?z0iVa+Vz3pVxu611/j/pu1rrjgSSC8SaT9iqKKA4XKL9iCtit3bRMgE1Vb60?= =?us-ascii?Q?FqKH8F86q+gC0FVEjEu8ShS2ebFebEeAKGi40vMHl+/0MQVOGjk9aVxKj4+6?= =?us-ascii?Q?XFswJgPpnr2WJup3tcfjW5F64YYBHvRfqrNAp?= X-Microsoft-Exchange-Diagnostics: 1; BY1PR0501MB1110; 6:A8MhvWw/l+6XdqavHF8UYBc/+c+oaLvxksNmdq+fuip6f9qi7wohP927fc8hl6h5uRX2L8Xqxe9qJP4r7pcQ4PrkOG3TjZMBgQh1XpRJxq3l2xEFt7ydqtA8g/EHFwc2xSHS505jLAYr/h5HLnB2BoeOBY2U9KdlJg40h5r4qvjb2zKlDdbtA4dF8+P5EELYOjBb8JL9tvaMyzpvZHg1ZXXR/MZqMYzIpqfphNBmRey15AasaQsa5m7cnSIkzJDgljtS+TYhH9X7LtEQr163OwckB4mJnyXFzd8S2VEHubFT9zVFhwdFOTvFRTi3hI1Xs5fWNzkX3YeYdVXEb1XtkjCSsQWTY6HdNvUUrI8Q4dKrj4lOJ3u4KgKi6RVQR5mMIJ0o4HmfXNwBMsJugCEXUw==; 5:GJeTx3D+pIjLPVJgEzLWoh1EKV6okVf9/AgbExyUJy5lO1oKEcc/NcVSydZXokdfp207n2HYU1YJo2Qv0kg8i5UUJu8111DLBMuDEf2tHuK/7TGW69D0TY9Ba2cTSthIc7fPF4vRtW6Pw2y8cWth+g==; 24:4og2dk2R0iO4eGYE76IU+A2Yorzl8XTgzkYev+o0I7bwE5lcyzmK2NQJelFWB2bxjmADvGrJaVQA4CpaLIdHnxVHq1s+5n+/8NhS/jaygWk= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; BY1PR0501MB1110; 7:0nbJe6ATrYeoPC7IptWqqM5U9FqIb7BGi53cmYZbCzb25/9Ltuqd+1OWmYwGScDhyxtS59UP2Y+KWEjrzgUrGScdcMfGMq2HF+jkIXHU+8SE1BNNv4AxFC/rsFaaeYbjlFjM6OsUC661r5us2sN3cROYG8kmPAqtw1GlGXIyWzA0u2US2urD/Ct/psEgVRhqnSDShRa7nWfDnne2GkudzTFxUIMOqtYgseUU7yGQyhjszv1I2i78fOMfKcwMm/rUR+8vy1GJslQMaMh6T3qFBw56XuOnJX2iIDHPr3x4F4DMJz6tij+lRwDzQbM9fu96t+3PX4IhPPdxld/7WezQ5Q==; 20:Lms/5gKI75SCxT8ql+ooA4JnTyvDVBfIFCSpeM8DUdOCIDMwXH0eDbiWdEemRVj1ilvSZhGtTj0LhAWwboUUHLbkecOpaV1Ha3t0VONhhlvGeZhd2SF/iCkk4dAY+kXa05EDtoJTvCbWPMk2oDeVgSg3tUdanDvqXj9xfpsdn88= X-OriginatorOrg: ksu.edu X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Apr 2017 14:19:54.4410 (UTC) X-MS-Exchange-CrossTenant-Id: d9a2fa71-d67d-4cb6-b541-06ccaa8013fb X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=d9a2fa71-d67d-4cb6-b541-06ccaa8013fb; Ip=[129.130.18.151]; Helo=[ome-vm-smtp2.campus.ksu.edu] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY1PR0501MB1110 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Apr 2017 14:19:58 -0000 Hello! Over the next week or so, I plan to address inconsistencies in regex(3) handling of invalid bounds/atom constructs [1] between BREs and EREs, and found grot [2]. This appears to have once upon a time been the vehicle for regression testing of regex(3), but seems to have been replaced by the netbsd test suite (see: [3]). Does grot still serve a purpose, or can it go away? All of its tests seem to have been split out into individual tests grouped by functionality in the netbsd-tests suite [4], which is a model that seems ideal. On top of that, I'm not smart enough to actually build any of the targets in its Makefile and I don't feel a compelling urge to make it work on its own. Thanks, Kyle Evans [1] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=166861 [2] https://svnweb.freebsd.org/base/head/lib/libc/regex/grot/ [3] https://svnweb.freebsd.org/base/head/lib/libc/tests/regex/Makefile?view=markup [4] https://svnweb.freebsd.org/base/head/contrib/netbsd-tests/lib/libc/regex/data/ From owner-freebsd-hackers@freebsd.org Fri Apr 7 14:57:50 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 965C9D323A4 for ; Fri, 7 Apr 2017 14:57:50 +0000 (UTC) (envelope-from kevans91@ksu.edu) Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on0043.outbound.protection.outlook.com [104.47.36.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2B3F3DC5; Fri, 7 Apr 2017 14:57:49 +0000 (UTC) (envelope-from kevans91@ksu.edu) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ksu.edu; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=mBnMFi7WwedXqng+XMBBtcxP4d4jv83VDK3d1qBQqwk=; b=CPIX1FxG0XbIvZvm2JsBHwJ1aRka28JGfpGu6IW3WKMQt/CZ6JjiuxX6TxnAKJeB7MEfIND64GSytHrp06/9yHNs5RJkDPd6XYyfIzS/zGLp4CHr8Vl3NiVGD6tzPfkQMrEnnRFmeKDir+i0+I1Zsk1EFKC72UYUD+g98TE8Kuw= Received: from CO2PR05CA0072.namprd05.prod.outlook.com (10.166.88.168) by BY1PR0501MB1112.namprd05.prod.outlook.com (10.160.103.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1019.8; Fri, 7 Apr 2017 14:57:47 +0000 Received: from BL2NAM02FT058.eop-nam02.prod.protection.outlook.com (2a01:111:f400:7e46::201) by CO2PR05CA0072.outlook.office365.com (2603:10b6:102:2::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1034.5 via Frontend Transport; Fri, 7 Apr 2017 14:57:47 +0000 Authentication-Results: spf=pass (sender IP is 129.130.18.151) smtp.mailfrom=ksu.edu; freebsd.org; dkim=none (message not signed) header.d=none;freebsd.org; dmarc=bestguesspass action=none header.from=ksu.edu; Received-SPF: Pass (protection.outlook.com: domain of ksu.edu designates 129.130.18.151 as permitted sender) receiver=protection.outlook.com; client-ip=129.130.18.151; helo=ome-vm-smtp2.campus.ksu.edu; Received: from ome-vm-smtp2.campus.ksu.edu (129.130.18.151) by BL2NAM02FT058.mail.protection.outlook.com (10.152.76.176) with Microsoft SMTP Server id 15.1.1019.14 via Frontend Transport; Fri, 7 Apr 2017 14:57:46 +0000 Received: from calypso.engg.ksu.edu (calypso.engg.ksu.edu [129.130.43.181]) by ome-vm-smtp2.campus.ksu.edu (8.14.4/8.14.4/Debian-2ubuntu2.1) with ESMTP id v37Evk5f011929; Fri, 7 Apr 2017 09:57:46 -0500 Received: by calypso.engg.ksu.edu (Postfix, from userid 110) id 7284B24830B; Fri, 7 Apr 2017 09:57:46 -0500 (CDT) Received: from mail-wm0-f49.google.com (mail-wm0-f49.google.com [74.125.82.49]) by calypso.engg.ksu.edu (Postfix) with ESMTPA id 1F945248305; Fri, 7 Apr 2017 09:57:44 -0500 (CDT) Received: by mail-wm0-f49.google.com with SMTP id w64so720245wma.0; Fri, 07 Apr 2017 07:57:44 -0700 (PDT) X-Gm-Message-State: AFeK/H1eKxJSxZYWCIKzb4JrrTb50UMuGzSvJWqNYd9RXC11xdGxnI0f 2B1gHdlO3aGXG7AfRh2MVbQDhXpMxw== X-Received: by 10.28.91.1 with SMTP id p1mr28620948wmb.63.1491577062935; Fri, 07 Apr 2017 07:57:42 -0700 (PDT) MIME-Version: 1.0 Received: by 10.28.39.134 with HTTP; Fri, 7 Apr 2017 07:57:22 -0700 (PDT) In-Reply-To: References: From: Kyle Evans Date: Fri, 7 Apr 2017 09:57:22 -0500 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: regex(3)/grot To: CC: Ngie Cooper X-EOPAttributedMessage: 0 X-Forefront-Antispam-Report: CIP:129.130.18.151; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(39410400002)(39850400002)(39400400002)(39840400002)(39450400003)(39860400002)(2980300002)(438002)(24454002)(377454003)(450100002)(4326008)(229853002)(8676002)(45336002)(606005)(9896002)(110136004)(5660300001)(189998001)(38730400002)(498394004)(84326002)(61266001)(8576002)(6306002)(236005)(8936002)(9686003)(6246003)(86362001)(512874002)(106466001)(93516999)(6916009)(75432002)(2950100002)(356003)(90966002)(42186005)(61726006)(63696999)(54356999)(55446002)(50986999)(76176999)(7906003)(305945005)(2906002)(88552002)(46386002)(53546009)(2351001); DIR:OUT; SFP:1101; SCL:1; SRVR:BY1PR0501MB1112; H:ome-vm-smtp2.campus.ksu.edu; FPR:; SPF:Pass; MLV:sfv; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BL2NAM02FT058; 1:hl5Ypcj6c4u/kpb3Y9BSgKD38GVRzLjCYV4rumlxerOezOpKh6rstwpplD5WhJZJTtNG0PZxlCrFtYodZKOvfS4F0MYRNMXPOw6oVbNSoWs8bfaA547rIcYYtFlr9vLjejYn7oX61gdjKm2BsJvckrN35pEE9qfVSIYEsVsJ+8jiqNFtCdVdtDkGtObu4FM2QCX6n+E4hNAEOaFQku+UQITOCAiO4lQvZMSCryLhfcxxrHG64RqQqEWe/GZPH9sLDW66L55fDGUUXSLJZ5Bru8TuaTEUQeX5mnWx0B3XHgVsEgwcoegfGHcukMiX7+hhpoOTisUXB0c7X6F2d9BO/s1E3OwxOnbLPEv6R6e30MNeIS1/Xa5L3Q5/Y8MuZL0Cu86qVs+v5IbrPFsGXoKdQlIcoR+4ckkcbVGnEUOBK7ZSvmAwnEuDaoEpAGIJOPHC5Op+h7h0VMwZWLWJDqSbRa11HvOtJfd15vOqlyA/Yc1x8YgeVTsiSdmUgkLVhAa0lFBsIwYX0g6gXfyRVmxzvnDIfFybRWheIJtmfnvC7VM= X-MS-Office365-Filtering-Correlation-Id: 11e22738-c0cf-40d3-63d7-08d47dc6740a X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(8251501002)(2017030254075)(201703131423075)(201703031133081); SRVR:BY1PR0501MB1112; X-Microsoft-Exchange-Diagnostics: 1; BY1PR0501MB1112; 3:az0/4SUgjZhh7L5gwp9qA0r9a2wmoe0qbBEII24zbQ5QZT2FHmtwgsqJUZq2qkcVEOhQM1kcGVR/uALgY26XNVoSxdFrVvaZ00AItWuKF7RL4qIW21ndALd+ZwGNHFAK+dpDnsLUkzETkl7AwdzPHapCGGEDlsqpycR1Xa1Coc/kRMfrEYW7IscfIu1jq+9vUz4DFnMAblLVrLkXA2rsJc8b+b79n7s39KJbRSP+XeH2mJPkRTV4A5PqIGqBay0/pwnlsPIgrsaebCFRB9saInCtkwSPBnM7SAcBoIpwPRVaQZg4HHKciqm2YUAn94bDzFCqiWhtGwNvci4jt3TFhvCGKRaEZ53vJADFnLs7Kz+4B1/w3trz58WO/3N1+TBATWh6RpPq5nBzY5qvJeSLIXfLbB+OL7P8Zke4qXjOaIgWl/9VEf8yVUUOaXxbiarwvEW/iEMsrBRUJiLWXRKoHBTBnx5G+U1mYxjBcJW+rQDUCPfZgTj6Hi0J/cJ3Tjrf X-Microsoft-Exchange-Diagnostics: 1; BY1PR0501MB1112; 25:4lC58rA6tRqs2dMbKIyny3mw+v8q5LgHgLSSDD08loKYZXMmJ9WI96bGHVPNCVJRyzc5An/NILTdaYtiuHOXbmDjq4rPmBMUSP6hVPeHO6D6Wg66yGMgTtsXqkKOLbVz7GLQlBnmY0JAt5+oYW0FwlHltBRr+GThz/FQGVpP2Gn+kjXv3sKTNkpeOuizHBToaKWakh8hxCuF5n0HhDDXdysM5nmJg8M8Vg04CQ1DuXnekX1aSAgBzD9oI5hdaIPB7csfEuM6F1S6vogMpMvVP63xQwXsW/c5sc0Tqe6E80/1oTj/8TWFQccpW9keWnVcjJhScSUr7ySMmWSFeq8/JK4KD1AFjf+t4l4r6NZxFRtsbhN8bFoTBFiiVU7k8pOfgihNON1QERCty+U4onS7yXLz4G1rXWEEOGGCwKvb5S8yp8m3zixUdBdhMm8muC0GmiKPnz5N75mIcG1cZwLEAw==; 31:KynAoCssn2v2HXtUrPXzzwJslaTf1NqP34djdoqZsAxLXiqBiBYdyv4+VUo83EWgGvwY5kvYLxgDBfJCKaLo8b0LD6KGl7YLO0xGP8Cfwixe8og473RKPeDd7YWTxgARt0+T0RJzMM2f/OQ30SUSlo5Uw02PnnVT5ZkJZ7GaD9Vm0grArCQeq65xT3wC13dC37KsWMvrlYb4tC7HQCRPrJqWR3xtQMYb4rFmp8431HqnaZE+1iyW2vdHniVcp+sHw+cufW8cTZAbsfvtsDQhqRCNNE3NeXeR9tBUFgaiFNU= X-Microsoft-Exchange-Diagnostics: 1; BY1PR0501MB1112; 20:6lZx45/NxoZ+5Z6nvaLG8xyBeauGFeqxT/JSOJK9Ve3TdhN9o2CArbKSkLjXQMaPNcvDeuT07++MEubpi7KyqgES8SUJuYTT5Kk6U2iuu4xe5QRS+6aWA/82MRKVSBc2uPMK+AYJxU/ii5ouRpPb4RIRl0fBWJWDkhkwKJ+aZffw25ytkUPmmYJSEe90avy5wjbE/x9SbsiVTyvVWhDXZw31dIsEyH8jM6dchxicVKTP2S/9LZvSDhdIcsXELEpFu69ovTZ0cV6IWsUC+C6V6e5MRv9bJBsjKkNW83+++M+anp0vexTqISjTgY5gBXEUVlexDRqL1Dy2Au+4V/Pk+NjG1DEkfUj5xDGisqzlbzkIpukY/p6d6TyaB7DNbERp+Wwits7XhHTMsHDUpF6Mv+VMXc8Rf3apZTMYZ+nSldNwR2XC5mNvsB3xlLL4tVusJ7JNfhrXgXsYMQ9Lg3HR3fI5xG06G9EJkHV3uzOBdVDJaRWLorvjtLJuTd0iFOXL X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(56005881305849)(112903893386949); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040450)(601004)(2401047)(13023025)(13024025)(13018025)(13017025)(5005006)(8121501046)(13015025)(3002001)(10201501046)(93006095)(93004095)(6041248)(20161123564025)(201703131423075)(201702281529075)(201702281528075)(201703061421075)(20161123562025)(20161123560025)(20161123555025)(6072148); SRVR:BY1PR0501MB1112; BCL:0; PCL:0; RULEID:; SRVR:BY1PR0501MB1112; X-Microsoft-Exchange-Diagnostics: 1; BY1PR0501MB1112; 4:jDnzbgzm48qETyFk4z9aatpgr3ynyGPJoTYN3CA4vJXQ8u/QLQ8amx8iJlM/WYTZeY8G4rxVqY2RUZEdGFcs+RiBIe9cHB6ixdxORAyR084RIjdjONm2Xp+U5LnI+lSrCiVsPXWuZGDFMAkA0kSjf3RyBL7y+DkCXfg14lsGJ8AtWKD+wxGe02z8fKqxibGE7jDjz5z4y/JwMgl0aa0sYuJllqC7E8dVjNFOA7c8oqa6W2oPNnG5gXkoOZwIMnROI5KGJatQ5ntdtjW3W06ysiLnmlusMLGa+rw2qv06WhifPZwcg/zGTSxvTAUjCRLQOCtdi9P943+JtlvS/mdzaTyJw+CZRCKZjwnm+hjVm3YfL9hHq7Qqji733jtVw/dttq5+58a8yPpZlFCoy+VzFWMzWU+sqLvTykta3ovR6EUvhvDmEnULs+uPrPRB/nydl2xdwk3tSk5IYoHINoAD8+AkBL8HoiJYmk975Q0OSq2xBrbwvOW8ccC8hiYR+e+6BuSqTyvMolnCFo7Rk5mK1jeiCQZ1czRPWkcVSNNhw9AIfWwMYAMvxvWOxFzO4yIAHYLpOBV6tHMjoyjSo0nB1G7SAw9QPBq6a3cFoBRNIKE2A57cnfFsHhQvx0yL+GHhlrzM6ou6RIpVaEv3J1Kzw92gbLWf8LgrAkxeWdd37GH/jepZrd8SFiaBx5+RM9a1oGd623W0geoyOphPvmofvXROYsNrII2DKFgUIX4ETkVfJosVP8AoW8ialtQsMnqqwq+Zwi5ykib7xrmZX7Z2bF1o7joPZYhdWPGFSGRdgfG7qVPQJUihtxCwprv6qTp2m4AoAWt+185FZAh9MxbcKwmyPIpKb3wBrTXfexxj25gHdDPKW9z2vPFhHa1hU4cLeTSFlDu12p77RfvBvNi6U8APhSrP0iagbReickMoKOk= X-Forefront-PRVS: 0270ED2845 X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BY1PR0501MB1112; 23:jNemWK1Qa5S1+lkSO368cuXykv1j1gltT8/6YQc?= =?us-ascii?Q?pLr/uGcMMelhnQHz6r2KZXhK85s6Eg6bTdi2K5A9/gO8aSbMPo33btYz47s6?= =?us-ascii?Q?94carq/kPoWPPuKuo4p+oOuaMqXfCm130Yb5EswIxo04RWlfJ5VEu25gEfb9?= =?us-ascii?Q?6dgY3X5nuraFOZKV+ye6NaXdjjaMt2VnmKBWpnHJ6OJigXnx2VKcXBobRv3Z?= =?us-ascii?Q?RBd9iJWNykW32KeaL3qYfNHOHy+KSqpcchqmW0Vhuv1VIlI4bYHUvEjM1hII?= =?us-ascii?Q?ZA/YcV/Oys6Iea3DAVJt0Cw2nfQfSdhYB2h7MEycjc9naaHJPTILkdyd+6NQ?= =?us-ascii?Q?O9RqtsbtD0E1TU99bPzIJIAnOOlNgzBSmVcnAbueWVGg7rXGVDdATCLYzAD8?= =?us-ascii?Q?nY4X4SEtdt4ar3kf+f4gA5cH7HMlAAndnLpau5wQkH11OtIBmqmbti31OJSy?= =?us-ascii?Q?zTAXApKAF768u7DhoACinD7ct6GiH2vX66Ry+P5J8N9wQjhAH9TRc7GZ9/Qv?= =?us-ascii?Q?oujz33nIqSMdbXoa9RGMkj4AotblylOngN2lgKT4drUeU29Y9mqC/fgIx3TT?= =?us-ascii?Q?TdAb/cLA70gNyGwDMuEUE2DbCtzXTUZrutplvg94Eylxg1q22iV4AoaBLt8x?= =?us-ascii?Q?d/uv7HjJoWsGJ3lIeLcbU4iIFUl1/B67hHjTfgBe2uP80Ns5O6Ul6ifb+xGS?= =?us-ascii?Q?H7wwB2b3Ow1/IhYhBxMpn3IfujcDFaMvN5NYlcFx1PhEVA5BIfbtFoJuzivX?= =?us-ascii?Q?7B21iQaA/pr5PVDqWTa/IvtXz3HxafzY5ohEFEOVHZuHbcuhmrC6mwy20jNr?= =?us-ascii?Q?yowVHs6K2A3WPQ5V5la5KGhhr3LzzzgkQPsUG/MPH3GHtYw0a9tWyaRty/3e?= =?us-ascii?Q?7MKwfRvEdtfW6xAiB2Mx1EckyON9/KE/Na367bIKgaE2F7tK6hfkl/hBqiAM?= =?us-ascii?Q?pa3ZAIPNSHI9rzliI3Z64dGlDa7g8d7QrMcQGhWkgFRhW2MEXtmKsDTHvJEu?= =?us-ascii?Q?Bd3cFYxtYr7gW4vYgGF7BvXq6Kd0i0fUeQVmnPWPUZ0JmHMWdIfFcMub1geb?= =?us-ascii?Q?eIZCMopWWLqhTsBg3lHsd7vsWJ4FhTf8pgao/AjInbwLHBSgXZw/sa+83OTc?= =?us-ascii?Q?jdbzgLLKBPPBxfTC6dbY7evMf2NI/Ov5IRRFvhsQBotZXqK+cuEfjxBDZMzq?= =?us-ascii?Q?yIIXu6+ARVutz6ZXCicnXsp5UUldXQ72ifqBAkeydrfIpn+4E8Bm/uIWUqT3?= =?us-ascii?Q?xBIQcZ0MjviBcuTsE40pYoE5aWq8lx3YfNDsBowU7fLfrY/B4UQ922i337K7?= =?us-ascii?Q?1YQ=3D=3D?= X-Microsoft-Exchange-Diagnostics: 1; BY1PR0501MB1112; 6:IO/UsN7s0i1CuSufCYJ3qq+vb/NQazlv9VlYH0RqVZ46awNreWG6YFkCHJXSFgGqd+S3Gl3YHK7ZflW6MwWsjh/7U2q89HZNeRT5N45rPP5HUrpcMqHOVTD2z/6PkNqi6irxp7owk+6Fr1+E7WP5eoH86ds0B+P0fELjrKYnSQFe+s2rjFF66S/5vGnV8TeH2WN/3X9Wk9mw9zEfVtNjcMHVgMPvxTD5lvbpfebfQ8IIK0yNhyLd13NPA+bAWTLtYugB8fhTW5CNZzeA82k4uc75FGmVre9l8LOb83SX3ni83t2a6NNMiJK+kIuySz61y7nSXwJ2iRw9OGtHz6N5dT1AAWSR1aewb0ZMnZOB0FNxZT6rLRoigavr9HB9LWSOZAvoxUVXbGQRXhfW6UYvWg==; 5:BWfZdM0e7Bm4/aHM5S4ancpKBdlspcOjcXKNEHx6deSp4w3eIokI90PzYMtBZTJYTOieg0Y1KR/6LOjVAm/ev7hujLyEl74JYj50L6OAnirc2BZE8bJ6WpQg1bBt7C57E9I2DaWnRwpkA4LHggyHhw==; 24:vn8Lssz+X+4j2rLQGk1XRmvs3AQaxOb3BvfSVBJD7vQQyNRcGg1/K+5kt+QPbwqtKHHQ5x4zjXIFWxAv2NPU9c4lwsYwycgdwVb7Ut7MVSo= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; BY1PR0501MB1112; 7:Lrgqn/PqrveHPXB3SdS4SScycDVC+XRJ9iMoYcpI8IKBSzbEVNMLL429n2xzSNvdvgWJUjwYPe9tB468BVhcalCrPX+3tU/29r1OG0XkGvKIEzYJ4RZZ1SBFUD8tjxOBlW22+fptAkfN8GzMctGsuS5gVL9JTQXFUHKnLvioEEzJGDHQy25jZii/BeH2io1lYnd29AuvsmENyyMxCPqZMuF77z9kPxfq+CeZe8OV2qBuEHnFKVGsYxo5V9fQxUKHaKquCIl8ljxYJKPyIN7qaEsUt63mHsT+FsdE8yZRLSPimdv05+9aApriqoz8Jl/VAmpH4rJu+c5c+N/B4CWvMw==; 20:oIbpbU2Xr7pQSoxPWQ3aHvzrdM67dCYlWfS1Tt+yjBSsR/Gact+0bJVYwYRj7hmUW9R2J1ST4I5Idw/lj1Zromn0H9IE7woVTEcmxCBziWmUzJSz1MDBhJQ0PKFvMPVPLVOghQ62JAcYuzbKba5RCKvsxqKKX+PWN1+DCY3CIi0= X-OriginatorOrg: ksu.edu X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Apr 2017 14:57:46.8198 (UTC) X-MS-Exchange-CrossTenant-Id: d9a2fa71-d67d-4cb6-b541-06ccaa8013fb X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=d9a2fa71-d67d-4cb6-b541-06ccaa8013fb; Ip=[129.130.18.151]; Helo=[ome-vm-smtp2.campus.ksu.edu] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY1PR0501MB1112 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Apr 2017 14:57:50 -0000 On Fri, Apr 7, 2017 at 9:19 AM, Kyle Evans wrote: > Hello! > > Over the next week or so, I plan to address inconsistencies in regex(3) > handling of invalid bounds/atom constructs [1] between BREs and EREs, and > found grot [2]. This appears to have once upon a time been the vehicle for > regression testing of regex(3), but seems to have been replaced by the > netbsd test suite (see: [3]). > > Does grot still serve a purpose, or can it go away? All of its tests seem > to have been split out into individual tests grouped by functionality in > the netbsd-tests suite [4], which is a model that seems ideal. On top of > that, I'm not smart enough to actually build any of the targets in its > Makefile and I don't feel a compelling urge to make it work on its own. > > Thanks, > > Kyle Evans > > [1] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=166861 > [2] https://svnweb.freebsd.org/base/head/lib/libc/regex/grot/ > [3] https://svnweb.freebsd.org/base/head/lib/libc/tests/ > regex/Makefile?view=markup > [4] https://svnweb.freebsd.org/base/head/contrib/netbsd- > tests/lib/libc/regex/data/ > Further inspection revealed that the netbsd-tests bits are actually just an obviously derived/improved version of grot/, leading me to further believe that grot/ can/should go away. From owner-freebsd-hackers@freebsd.org Sat Apr 8 11:39:30 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 98862D34BFE; Sat, 8 Apr 2017 11:39:30 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: from mail-wm0-x241.google.com (mail-wm0-x241.google.com [IPv6:2a00:1450:400c:c09::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2FAED1DC; Sat, 8 Apr 2017 11:39:30 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: by mail-wm0-x241.google.com with SMTP id d79so2165782wmi.2; Sat, 08 Apr 2017 04:39:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:mail-followup-to :references:mime-version:content-disposition:in-reply-to:user-agent; bh=i2+i21H7oFNUJzp/Dy4+LY4YaX0mUfWAahwmt8ZUCtM=; b=cowhy8IOu/8d83ZOu2YtmHrP0bkr+oaQ6NL7/0mZT66lbnkS7q0i3sEKn9lMR+f5w7 WJN2YzqYBX2Oyi2alNX0fJ47K0zIBGS4L+YbURM4eiAdvlpuOLwK05UozAKJgZt4ycW3 BBqAseuHUuRaJuCIEpaVCdyGfa5VY6IXG7YeeEx2djmZIIwijY0CvanZy4nlEwy4eSms 2qs4OTZiNXfYwYVqgQebZ3wK9hQpfZm7yczH5eNpE21RBLQlQUUuBJelWT/c2JPC1WXL Xon1J+lRiFn8jkQIXPu26AO8AvjB5fByzHDdMHo/9nPmc+7AlfA9u849vEef4hwW/ZKV gKyQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :mail-followup-to:references:mime-version:content-disposition :in-reply-to:user-agent; bh=i2+i21H7oFNUJzp/Dy4+LY4YaX0mUfWAahwmt8ZUCtM=; b=NbG9ydy+jPLbtffeEEck4YeKP8ToE4Hr8kpzeAayYOz3nLuv1jeAwgsT0gl3mrC5rX T5LFHyeTW6bnKAo++fLGBjTxRVKNzAseYrLogfLFdpPqDnXtKsiueMuBOwpkrGuNcD2T X/BOAag7+DrK9p5VzH7WSb2Ul8utLdbiNXMZHatosI3iee/bzXN11X/3rrI3C1Jfw/bQ gzmAoeEnovgCM6p6+fbWWFi/jjFKo3DOAwhp2JwSPsNXFYIdQ9mXVAax2Jska3+5272z /Ttqb/afRvo/8G8QdsgXjpY2sNsvznE9/sjJ3bNrATVkozVjP+/kXzhse2HstJSnLRoE AKdw== X-Gm-Message-State: AN3rC/4Yl1B+6t9Cb9KdFPYofePyzfTnA7tkTL4VEuKFBdrt/K6fK3BHTIeeolvbxcS7Lg== X-Received: by 10.28.186.3 with SMTP id k3mr1669973wmf.74.1491651568551; Sat, 08 Apr 2017 04:39:28 -0700 (PDT) Received: from brick (cpc92310-cmbg19-2-0-cust934.5-4.cable.virginm.net. [82.9.227.167]) by smtp.gmail.com with ESMTPSA id u36sm5399485wrc.20.2017.04.08.04.39.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 08 Apr 2017 04:39:28 -0700 (PDT) Sender: =?UTF-8?Q?Edward_Tomasz_Napiera=C5=82a?= Date: Sat, 8 Apr 2017 12:11:44 +0100 From: Edward Tomasz =?utf-8?Q?Napiera=C5=82a?= To: Eric McCorkle Cc: "freebsd-hackers@freebsd.org" , freebsd-security@freebsd.org Subject: Re: Proposal for a design for signed kernel/modules/etc Message-ID: <20170408111144.GC14604@brick> Mail-Followup-To: Eric McCorkle , "freebsd-hackers@freebsd.org" , freebsd-security@freebsd.org References: <6f6b47ed-84e0-e4c0-9df5-350620cff45b@metricspace.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <6f6b47ed-84e0-e4c0-9df5-350620cff45b@metricspace.net> User-Agent: Mutt/1.8.0 (2017-02-23) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Apr 2017 11:39:30 -0000 On 0327T1354, Eric McCorkle wrote: > Hello everyone, > > The following is a design proposal for signed kernel and kernel module > loading, both at boot- and runtime (with the possibility open for signed > executables and libraries if someone wanted to go that route). I'm > interested in feedback on the idea before I start actually writing code > for it. I see two potential problems with this. First, our current loader(8) depends heavily on Forth code. By making it load modified 4th files, you can do absolutely anything you want; AFAIK they have unrestricted access to hardware. So you should preferably be able to sign them as well. You _might_ (not sure on this one) also want to be able to restrict access to some of the loader configuration variables. Second - given OpenSSL track record, moving signature verification and the x.509 stuff into the kernel (to verify userland) and loader (to verify the kernel and modules)... well, it just doesn't seem to be a good idea. Also: do you know about veriexec? https://reviews.freebsd.org/D8575 From owner-freebsd-hackers@freebsd.org Sat Apr 8 12:03:46 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ED0A0D34E54; Sat, 8 Apr 2017 12:03:46 +0000 (UTC) (envelope-from eric@metricspace.net) Received: from mail.metricspace.net (mail.metricspace.net [IPv6:2001:470:1f11:617::107]) by mx1.freebsd.org (Postfix) with ESMTP id AA9D9861; Sat, 8 Apr 2017 12:03:46 +0000 (UTC) (envelope-from eric@metricspace.net) Received: from [172.16.0.198] (unknown [172.16.0.198]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: eric) by mail.metricspace.net (Postfix) with ESMTPSA id F339316E6; Sat, 8 Apr 2017 12:03:45 +0000 (UTC) Subject: Re: Proposal for a design for signed kernel/modules/etc To: "freebsd-hackers@freebsd.org" , freebsd-security@freebsd.org References: <6f6b47ed-84e0-e4c0-9df5-350620cff45b@metricspace.net> <20170408111144.GC14604@brick> From: Eric McCorkle Message-ID: <181f7b78-64c3-53a6-a143-721ef0cb5186@metricspace.net> Date: Sat, 8 Apr 2017 08:03:40 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <20170408111144.GC14604@brick> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="qBnpgPE5iiBH2Q2URRsv4WA5VCo3qAhHN" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Apr 2017 12:03:47 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --qBnpgPE5iiBH2Q2URRsv4WA5VCo3qAhHN Content-Type: multipart/mixed; boundary="KQKRohTHn9LW4Ce3DD5LtEgm1pORiCnQU"; protected-headers="v1" From: Eric McCorkle To: "freebsd-hackers@freebsd.org" , freebsd-security@freebsd.org Message-ID: <181f7b78-64c3-53a6-a143-721ef0cb5186@metricspace.net> Subject: Re: Proposal for a design for signed kernel/modules/etc References: <6f6b47ed-84e0-e4c0-9df5-350620cff45b@metricspace.net> <20170408111144.GC14604@brick> In-Reply-To: <20170408111144.GC14604@brick> --KQKRohTHn9LW4Ce3DD5LtEgm1pORiCnQU Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 04/08/2017 07:11, Edward Tomasz Napiera=C5=82a wrote: > On 0327T1354, Eric McCorkle wrote: >> Hello everyone, >> >> The following is a design proposal for signed kernel and kernel module= >> loading, both at boot- and runtime (with the possibility open for sign= ed >> executables and libraries if someone wanted to go that route). I'm >> interested in feedback on the idea before I start actually writing cod= e >> for it. >=20 > I see two potential problems with this. >=20 > First, our current loader(8) depends heavily on Forth code. By making > it load modified 4th files, you can do absolutely anything you want; > AFAIK they have unrestricted access to hardware. So you should prefera= bly > be able to sign them as well. You _might_ (not sure on this one) also > want to be able to restrict access to some of the loader configuration > variables. Loader is handled by the UEFI secure boot framework, though the concerns about the 4th code are still valid. In a secure system, you'd want to do something about that, but the concerns are different enough (and it's isolated enough) that it could be done separately. > Second - given OpenSSL track record, moving signature verification > and the x.509 stuff into the kernel (to verify userland) and loader > (to verify the kernel and modules)... well, it just doesn't seem > to be a good idea. Integrating all of OpenSSL would be massively overkill. All you need is RSA/Ed25519 signature verification and parsing a subset of PKCS#7. My thoughts here are to grab the RSA/Ed25519 implementations from libsodium and just write a minimal PKCS#7 parser. > Also: do you know about veriexec? >=20 > https://reviews.freebsd.org/D8575 Is there some documentation of this other than a code review? --KQKRohTHn9LW4Ce3DD5LtEgm1pORiCnQU-- --qBnpgPE5iiBH2Q2URRsv4WA5VCo3qAhHN Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEzzhiNNveVG6nWjcH1w0wQIFco2cFAljo0ZwACgkQ1w0wQIFc o2eLWhAAyJJZKs/vQzV+/CLs2gm+xA4F+kkYFnTiuPgzanOnE0N24DBfQ01ODIwI bV9M+yXxJMcmNbLsmjQs3GQjrXF9eclobs2q9juhjm3hcs0XxV0i6j00v0TtIHxW xCyqEt506u6FsAApasBj6s+cvDL/KeSnXIZTB8VeMriXV+SRK2yl6rPRch51mjvC Ph88Rvxe9i2G49DqRigpsbMYgvd/Q/60cPdciLLq2KJYbgMKJY7nejZJF3A0L5bS 9S5dbkl9kmMtNBknOeQZxF9JcuIesymrz0WOjtPpB837lDjOtLhrtrbCcvZ5lYzo Uw6qLS5junOPNQi+xsSW14EnxgIMIMMvd9WqBRh0Jl+mzHiZDUY83SnvwEu48Nzp 5FBbhbv4cH5wrXpHzjAFt2eKRdnksSFG2xGGuRXAIf81xzNmZGfsM1+Q6ms/sBu9 BNxdgIoZdzcawA+ItJVplrMXTTfjJ94cwUPMUXm1F1MJNvS8c4wZr9Velvq+gF4b 9dsoN6/JlmjKkbZPpot+UvVkMUtGOFUBQf/Gcu+L3cM6NTIDLzSpgoVRzwbdMm/h AIlsFF3r664qntaT1cgOQcw5IN9k3rVXmhCm31XFoZzqFQdsL+AcrRSc0UiKrk29 OozqrwCyOX0brwQQixBLERu86nzacsQl3/8L1EMFY7NiFQvvqh0= =jbMq -----END PGP SIGNATURE----- --qBnpgPE5iiBH2Q2URRsv4WA5VCo3qAhHN-- From owner-freebsd-hackers@freebsd.org Sat Apr 8 12:20:09 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 22389D33469; Sat, 8 Apr 2017 12:20:09 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: from mail-wr0-x235.google.com (mail-wr0-x235.google.com [IPv6:2a00:1450:400c:c0c::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AE290FB9; Sat, 8 Apr 2017 12:20:08 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: by mail-wr0-x235.google.com with SMTP id o21so94984618wrb.2; Sat, 08 Apr 2017 05:20:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:mail-followup-to :references:mime-version:content-disposition :content-transfer-encoding:in-reply-to:user-agent; bh=XobHHXa5gBlO4FgGmMJm1dVEnAWAGgT6oEn2g9zMgHQ=; b=q+dxSTBUltZc6KrMFf7VG4rlbkMT4j+M/0rjN9jRMxpQTy9WGxOJFTWYfklePdWYgH E2fy4LXY7cf8v+tm80msWyKhTODP4J+5UD+J+wLgm6ednjMrF0su+rhR8ApQFohHfwan FgIOG2bvcay416ZvCT73+BtUQgKqxo0ctOsnWtuTQeVKmrVzFzyq+vNlGpnrqe3lyvUy ORUqWYE6/zIK14Kjce2ZT1rt4netXylNAX03+f4I1r5b1aKsQ0gfyxLua1tHQ+gVpix9 BWrUy30rZG8IZV3iTFe3JNe+37/yd4yzs3hKoxOuaJDILNHTc7pfjcY0aD/FWSMJMiU1 yiOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :mail-followup-to:references:mime-version:content-disposition :content-transfer-encoding:in-reply-to:user-agent; bh=XobHHXa5gBlO4FgGmMJm1dVEnAWAGgT6oEn2g9zMgHQ=; b=Rxz/LCb1rFDNsM4JAKWKbHhrmFFNClqMgdlf8USq2nobBakBQmGNQ/fsQOZlkzh4wy SKj/nvMlGATeH8dVZaGtJVxomdBHvbverRlcW8ua6hmyEf+OV/Oizp/Z+RCn43WA17no xWZEqyLW3XTpGQbCRxxkJEOsdVl1qU//1BVgM/ja9d1bgAJLKMzzCel2q0dCWH6CVL2X wmaLnExo8QqvTZGCbuv15qYB5TgRFBr3Qcjegi5px7A+gpxsUV6nEgC9OEf54c+5D58R ckLdzrL5e2eqmUqdlJleh3l9iK0DiGzZ2UrXApalYnS6mE0aU7/lzhoS8s4GhLJjOOvn 73Fw== X-Gm-Message-State: AFeK/H3/QINj4ZaLKZqjpRJgFhtwyHnz5R1wUi3TIDTDYgxTN1OmnrdBp7nBwinzmrk/ZA== X-Received: by 10.223.175.207 with SMTP id y15mr37082764wrd.63.1491654005920; Sat, 08 Apr 2017 05:20:05 -0700 (PDT) Received: from brick (cpc92310-cmbg19-2-0-cust934.5-4.cable.virginm.net. [82.9.227.167]) by smtp.gmail.com with ESMTPSA id 51sm9707510wrx.38.2017.04.08.05.20.05 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 08 Apr 2017 05:20:05 -0700 (PDT) Sender: =?UTF-8?Q?Edward_Tomasz_Napiera=C5=82a?= Date: Sat, 8 Apr 2017 12:52:22 +0100 From: Edward Tomasz =?utf-8?Q?Napiera=C5=82a?= To: Eric McCorkle Cc: "freebsd-hackers@freebsd.org" , freebsd-security@freebsd.org Subject: Re: Proposal for a design for signed kernel/modules/etc Message-ID: <20170408115222.GA64207@brick> Mail-Followup-To: Eric McCorkle , "freebsd-hackers@freebsd.org" , freebsd-security@freebsd.org References: <6f6b47ed-84e0-e4c0-9df5-350620cff45b@metricspace.net> <20170408111144.GC14604@brick> <181f7b78-64c3-53a6-a143-721ef0cb5186@metricspace.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <181f7b78-64c3-53a6-a143-721ef0cb5186@metricspace.net> User-Agent: Mutt/1.8.0 (2017-02-23) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Apr 2017 12:20:09 -0000 On 0408T0803, Eric McCorkle wrote: > On 04/08/2017 07:11, Edward Tomasz Napierała wrote: > > On 0327T1354, Eric McCorkle wrote: > >> Hello everyone, > >> > >> The following is a design proposal for signed kernel and kernel module > >> loading, both at boot- and runtime (with the possibility open for signed > >> executables and libraries if someone wanted to go that route). I'm > >> interested in feedback on the idea before I start actually writing code > >> for it. > > > > I see two potential problems with this. > > > > First, our current loader(8) depends heavily on Forth code. By making > > it load modified 4th files, you can do absolutely anything you want; > > AFAIK they have unrestricted access to hardware. So you should preferably > > be able to sign them as well. You _might_ (not sure on this one) also > > want to be able to restrict access to some of the loader configuration > > variables. > > Loader is handled by the UEFI secure boot framework, though the concerns > about the 4th code are still valid. In a secure system, you'd want to > do something about that, but the concerns are different enough (and it's > isolated enough) that it could be done separately. Unless the way to address those ends up being a signature mechanism that doesn't depend on the format of the files being signed. > > Second - given OpenSSL track record, moving signature verification > > and the x.509 stuff into the kernel (to verify userland) and loader > > (to verify the kernel and modules)... well, it just doesn't seem > > to be a good idea. > > Integrating all of OpenSSL would be massively overkill. All you need is > RSA/Ed25519 signature verification and parsing a subset of PKCS#7. > > My thoughts here are to grab the RSA/Ed25519 implementations from > libsodium and just write a minimal PKCS#7 parser. Ok, that seems to be a reasonable idea. > > Also: do you know about veriexec? > > > > https://reviews.freebsd.org/D8575 > > Is there some documentation of this other than a code review? Not sure; it might be best to just ask the author. Note that there are some manual pages in there, and also that it's not a single review - follow the chain of "Depends on", there's a lot of stuff there. From owner-freebsd-hackers@freebsd.org Sat Apr 8 14:02:38 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 53184D13EFC for ; Sat, 8 Apr 2017 14:02:38 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from smtp.digiware.nl (smtp.digiware.nl [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1DD2AB08; Sat, 8 Apr 2017 14:02:37 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from router.digiware.nl (localhost.digiware.nl [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 0ACEE3DBA1; Sat, 8 Apr 2017 16:02:34 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.com Received: from smtp.digiware.nl ([127.0.0.1]) by router.digiware.nl (router.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ahgnGl4OtpyH; Sat, 8 Apr 2017 16:02:33 +0200 (CEST) Received: from [192.168.10.67] (opteron [192.168.10.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 1EB2A3DB9F; Sat, 8 Apr 2017 16:02:33 +0200 (CEST) Subject: Re: Source of QEMU woes: CPUTYPE To: Dimitry Andric , Eric McCorkle References: Cc: "freebsd-hackers@freebsd.org" From: Willem Jan Withagen Message-ID: Date: Sat, 8 Apr 2017 16:02:31 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Apr 2017 14:02:38 -0000 On 4-4-2017 20:31, Dimitry Andric wrote: > On 4 Apr 2017, at 14:34, Eric McCorkle wrote: >> >> A while ago I posted on here about some problems I'd had with testing >> boot loader modifications on QEMU, and which also showed up on an >> unmodified HEAD. >> >> I ultimately tracked down the source of the problem: I had >> CPUTYPE?=native set in my /etc/make.conf. As my CPU is relatively >> recent, this caused some instructions that QEMU doesn't support to be >> generated in various places (most notoriously, in strlen), which would >> trigger illegal instruction exceptions. > > Out of interest, what does "llvm-tblgen -version | grep 'Host CPU'" > show? (This is a simple way to see what LLVM auto-detects.) > > >> I'm posting this here, as it's somewhat non-obvious, and probably ought >> to be documented somewhere. > > I usually find it clearer to specify the exact CPU type myself, for > example CPUTYPE?=core-avx2 (which is an alias for "haswell"). You can > also specify a lower CPUTYPE to build the world that you are going to > run inside QEMU. So what does: Host CPU: bdver1 tell me? It actually is a: CPU: AMD FX-8370 Eight-Core Processor (4013.71-MHz K8-class CPU) Origin="AuthenticAMD" Id=0x600f20 Family=0x15 Model=0x2 Stepping=0 Features=0x178bfbff Features2=0x3e98320b AMD Features=0x2e500800 AMD Features2=0x1ebbfff Structured Extended Features=0x8 SVM: Features=0x1cff,PauseFilterThreshold> Revision=1, ASIDs=65536 --WjW From owner-freebsd-hackers@freebsd.org Sat Apr 8 16:37:24 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7DEB6D35198 for ; Sat, 8 Apr 2017 16:37:24 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mail-wr0-f170.google.com (mail-wr0-f170.google.com [209.85.128.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0F7BBF94 for ; Sat, 8 Apr 2017 16:37:23 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mail-wr0-f170.google.com with SMTP id g19so81612604wrb.0 for ; Sat, 08 Apr 2017 09:37:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:reply-to:in-reply-to:references :from:date:message-id:subject:to:cc; bh=F+vQrIMmwpyx4hdMpK03FgG7x7E/XmTFFg/jAaJUr7w=; b=XuEC/TSXTgJM5xDFw00+/LokLJd8ZCp1MCnNTAWIC5C+mm/xnjGI5QmUSaH19yLEbV 5jBzokjFhnEi4EpYCergv6eBO3JguFiHPAwlWZpUN9/T9U6JO0V/9wkODmTmnLnEOdZp lpWl22vBfEglYDcsZuqq6zuTJ2CNunJGiEWfxtKP95LbHcCIq6xN41P8cEARa0k3P1YM o9Iw/uVnjeoNSQPHwitig6JBPveNhqxIi3GlhHoTYTh9JSt51ren0nhzfFDHlsF2VLNu IsqxGr5xna+GgHDqbFoW4amrq7d8tOMSiuaPqItQ2pM5WHD2L9g2eTkx3dLHKrkr8UmN XeAw== X-Gm-Message-State: AFeK/H2gmKuCC6VDHL10CoHozlFEUFLp93IeZLwGyELv9Rd4PN7W33mIZ1silQJ+yEVXxA== X-Received: by 10.223.133.35 with SMTP id 32mr36669526wrh.129.1491669435894; Sat, 08 Apr 2017 09:37:15 -0700 (PDT) Received: from mail-wr0-f176.google.com (mail-wr0-f176.google.com. [209.85.128.176]) by smtp.gmail.com with ESMTPSA id z38sm10484773wrc.36.2017.04.08.09.37.15 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 08 Apr 2017 09:37:15 -0700 (PDT) Received: by mail-wr0-f176.google.com with SMTP id g19so81612503wrb.0 for ; Sat, 08 Apr 2017 09:37:15 -0700 (PDT) X-Received: by 10.223.163.136 with SMTP id l8mr41474504wrb.57.1491669435256; Sat, 08 Apr 2017 09:37:15 -0700 (PDT) MIME-Version: 1.0 Reply-To: cem@freebsd.org Received: by 10.80.169.4 with HTTP; Sat, 8 Apr 2017 09:37:14 -0700 (PDT) In-Reply-To: References: From: Conrad Meyer Date: Sat, 8 Apr 2017 09:37:14 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Source of QEMU woes: CPUTYPE To: Willem Jan Withagen Cc: "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Apr 2017 16:37:24 -0000 On Sat, Apr 8, 2017 at 7:02 AM, Willem Jan Withagen wrote: > On 4-4-2017 20:31, Dimitry Andric wrote: >> Out of interest, what does "llvm-tblgen -version | grep 'Host CPU'" >> show? (This is a simple way to see what LLVM auto-detects.) >> >... > > So what does: > Host CPU: bdver1 > tell me? Bulldozer version 1, I guess. (As opposed to Piledriver, Steamroller, or Excavator.) > It actually is a: > > CPU: AMD FX-8370 Eight-Core Processor (4013.71-MHz > K8-class CPU) > Origin="AuthenticAMD" Id=0x600f20 Family=0x15 Model=0x2 Stepping=0 Best, Conrad