From owner-freebsd-stable@freebsd.org Sun Sep 24 05:48:42 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 989E5E1FCD5 for ; Sun, 24 Sep 2017 05:48:42 +0000 (UTC) (envelope-from graham@menhennitt.com.au) Received: from hapkido.dreamhost.com (hapkido.dreamhost.com [66.33.216.122]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6373C6AE8B for ; Sun, 24 Sep 2017 05:48:41 +0000 (UTC) (envelope-from graham@menhennitt.com.au) Received: from homiemail-a47.g.dreamhost.com (sub5.mail.dreamhost.com [208.113.200.129]) by hapkido.dreamhost.com (Postfix) with ESMTP id A485E8BF1C for ; Sat, 23 Sep 2017 22:48:35 -0700 (PDT) Received: from homiemail-a47.g.dreamhost.com (localhost [127.0.0.1]) by homiemail-a47.g.dreamhost.com (Postfix) with ESMTP id 698CF4327 for ; Sat, 23 Sep 2017 22:48:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=menhennitt.com.au; h=to :from:subject:message-id:date:mime-version:content-type :content-transfer-encoding; s=menhennitt.com.au; bh=xW4KVrsJ9TuU b8P553d9kEAikRw=; b=LWEjokiBIxx4vmJ6QQ6bmF05k+ddfszHwwyjrs+VUNiV EApgUajZn2RH+FjPYS2ZwshoyIEpqNohaPNDzX7aY1r554AAml9QRtOb292LKfqb 0PMAZbqwEYKx+goIqGKFdvRQhMi4CxmrCSikF3RaIl/DRGCyEakznIbA2Aecm/k= Received: from [203.2.73.68] (c122-107-208-156.mckinn3.vic.optusnet.com.au [122.107.208.156]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: graham@menhennitt.com.au) by homiemail-a47.g.dreamhost.com (Postfix) with ESMTPSA id 03DE74326 for ; Sat, 23 Sep 2017 22:48:28 -0700 (PDT) To: FreeBSD stable From: Graham Menhennitt Subject: disk errors: CAM status: Uncorrectable parity/CRC error Message-ID: <206df4a1-4666-f0db-12e4-ec7958d9f7b3@menhennitt.com.au> Date: Sun, 24 Sep 2017 15:48:26 +1000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Sep 2017 05:48:42 -0000 G'day all, I'm setting up a machine running 11-Stable on a PC Engines APU2C board.=20 It has a 16Gb SSD as its first disk (ada0), and a Seagate 2Tb SATA-3=20 disk as its second (ada1). I'm getting lots of read errors on the second=20 disk. They appear on the console as: (ada1:ahcich1:0:0:0): READ_FPDMA_QUEUED. ACB: 60 00 00 6b 02 40 00 00 00=20 01 00 00 (ada1:ahcich1:0:0:0): CAM status: Uncorrectable parity/CRC error (ada1:ahcich1:0:0:0): Retrying command dmesg: ACS-2 ATA SATA 3.x device ada1: Serial Number XXXXXXXX ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) ada1: Command Queueing enabled ada1: 1907729MB (3907029168 512 byte sectors) ada1: quirks=3D0x1<4K> I've replaced the disk and the cable without improvement. Unfortunately,=20 I don't have a second CPU board to try. Is it possible that the board can't keep up with the data coming from=20 the disk? If so, can I try slowing it down somehow? Any other suggestions, please? Thanks, =C2=A0=C2=A0=C2=A0 Graham From owner-freebsd-stable@freebsd.org Sun Sep 24 09:50:20 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EF1E7E24120 for ; Sun, 24 Sep 2017 09:50:20 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 57C3070425; Sun, 24 Sep 2017 09:50:19 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id v8O9kJDG030527; Sun, 24 Sep 2017 19:46:19 +1000 (EST) (envelope-from smithi@nimnet.asn.au) Date: Sun, 24 Sep 2017 19:46:19 +1000 (EST) From: Ian Smith To: Marius Strobl cc: freebsd-stable@freebsd.org, FreeBSD Release Engineering Team Subject: Re: FreeBSD 10.4-RC2 Now Available In-Reply-To: <20170923175450.GX20729@alchemy.franken.de> Message-ID: <20170924193857.D35468@sola.nimnet.asn.au> References: <20170923175450.GX20729@alchemy.franken.de> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Sep 2017 09:50:21 -0000 On Sat, 23 Sep 2017 19:54:50 +0200, Marius Strobl wrote: > o Given that the amd64 disc1 image was overflowing, more of the base > components installed into the disc1 (live) file systems had to be > disabled. Most notably, this removed the compiler toolchain from > the disc1 images. All disabled tools are still available with the > dvd1 images, though. 1) Does that also apply to the components installed on the memstick.img? 2) Are we any closer to gettng a larger memstick.img with dvd1 contents? cheers, Ian From owner-freebsd-stable@freebsd.org Sun Sep 24 10:48:05 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4E43DE2520D for ; Sun, 24 Sep 2017 10:48:05 +0000 (UTC) (envelope-from steven@multiplay.co.uk) Received: from mail-wr0-x22c.google.com (mail-wr0-x22c.google.com [IPv6:2a00:1450:400c:c0c::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D5B4771849 for ; Sun, 24 Sep 2017 10:48:04 +0000 (UTC) (envelope-from steven@multiplay.co.uk) Received: by mail-wr0-x22c.google.com with SMTP id k20so3577626wre.4 for ; Sun, 24 Sep 2017 03:48:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=zqCdNm6MO+Cv6MgvbMYJsatua9VQOyqVXgUUHCIMCLs=; b=OH/gijMoTrsrNaAgImZMCMO6re/pw3okNbs79r1CvYxmwHOibmEvIzjPebtpifidsn m8wbNINnoyqb8fXFN/OruJPX8zvDUmaxBwND4ZaHjqtbUcxJCuN9Ukd38l+c3Exq/pH2 FKrWv5hOr93tcVecsMUX2Tua7SXeHWuPIONdA6WIOH9ZF9sb8qZn0xdeowEtsKV24CEa qEfgH16O+oCf1n98TgXC+Nd1Y7EiRbV0KZcx8vyDIjW6IDB7StLHpSthlrFYEJxnmRC+ wjjsg1FX1W2hLimRDUsP1/phVcE70ng2nqasO4rgwfMCjSFr8esdLMdqO2gsvre5+bIO Lw3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=zqCdNm6MO+Cv6MgvbMYJsatua9VQOyqVXgUUHCIMCLs=; b=cGIrz8YqW6Rq7JnNHEYPiw6NF0gr2p7wrV7j8Uz4g4aTsw+kzlA3qE4lisSIyvxwPp JUw/E8IeN3Yhq9QUoBJiqRs3k0AjxB6idTDsnABTh9agDCTjWslykRNvZbJTFEB8ZqM8 UksF07IPvcjF1zNDfRcDpoCc1AEDVH6gpxQgUhQ9zlkS79MTAyx0z7EfqLFfYkCmqgTa YKueJsvIbhA+/YaMe1iJqiY3kvbKx8j9vXOpkkdUPEMJM7Nila38yIagULqhKxyAytRJ GBwKtSOmtOPppeEKmFE6D+za0res5AfQkNvbcTc/l1eWB9bglnGs8dRLnpxSuKTTnRJE GDdA== X-Gm-Message-State: AHPjjUi7SVY02ZnrBfr/qzI5jrYzkKUvrxbvwv3RqNq6Rsrq0woMXIbY sP5U9S2HHjYorCiZBO4BQLQsdJZcOWMM8E1nzA3qa6dJ X-Google-Smtp-Source: AOwi7QCZ+4Um4BxFwKxnvhWB/2QRcflSQ0H9igeoS6hsjpAF603Nqvk+uFx96vgkkWsJgIX6/qVsgRcwD+xK0a9sEFc= X-Received: by 10.25.213.143 with SMTP id m137mr1266935lfg.126.1506250081670; Sun, 24 Sep 2017 03:48:01 -0700 (PDT) MIME-Version: 1.0 References: <206df4a1-4666-f0db-12e4-ec7958d9f7b3@menhennitt.com.au> In-Reply-To: <206df4a1-4666-f0db-12e4-ec7958d9f7b3@menhennitt.com.au> From: Steven Hartland Date: Sun, 24 Sep 2017 10:47:50 +0000 Message-ID: Subject: Re: disk errors: CAM status: Uncorrectable parity/CRC error To: FreeBSD stable , Graham Menhennitt Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Sep 2017 10:48:05 -0000 Try reducing the disk connection speed down to see if that helps. On Sun, 24 Sep 2017 at 06:49, Graham Menhennitt wrote: > G'day all, > > I'm setting up a machine running 11-Stable on a PC Engines APU2C board. > It has a 16Gb SSD as its first disk (ada0), and a Seagate 2Tb SATA-3 > disk as its second (ada1). I'm getting lots of read errors on the second > disk. They appear on the console as: > > (ada1:ahcich1:0:0:0): READ_FPDMA_QUEUED. ACB: 60 00 00 6b 02 40 00 00 00 > 01 00 00 > (ada1:ahcich1:0:0:0): CAM status: Uncorrectable parity/CRC error > (ada1:ahcich1:0:0:0): Retrying command > > dmesg: > > ACS-2 ATA SATA 3.x device > ada1: Serial Number XXXXXXXX > ada1: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) > ada1: Command Queueing enabled > ada1: 1907729MB (3907029168 512 byte sectors) > ada1: quirks=0x1<4K> > > I've replaced the disk and the cable without improvement. Unfortunately, > I don't have a second CPU board to try. > > Is it possible that the board can't keep up with the data coming from > the disk? If so, can I try slowing it down somehow? > > Any other suggestions, please? > > Thanks, > > Graham > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Sun Sep 24 13:14:42 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 90B48E28232 for ; Sun, 24 Sep 2017 13:14:42 +0000 (UTC) (envelope-from andrei@fazik.net.ua) Received: from mail.fazik.net.ua (mail.fazik.net.ua [176.9.63.60]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 573AC75628 for ; Sun, 24 Sep 2017 13:14:41 +0000 (UTC) (envelope-from andrei@fazik.net.ua) Mime-Version: 1.0 DKIM-Filter: OpenDKIM Filter v2.10.3 mail.fazik.net.ua 712165F04D Date: Sun, 24 Sep 2017 13:14:31 +0000 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: andrei@fazik.net.ua Message-ID: <4ced7d7f82a09156b3c697cb759b41f5@fazik.net.ua> Subject: Re: CAM status: Command timeout To: khellman@mcprogramming.com Cc: freebsd-stable@freebsd.org In-Reply-To: <20170921233416.GC2994@dane.localdomain> References: <20170921233416.GC2994@dane.localdomain> X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Sep 2017 13:14:42 -0000 22 =D1=81=D0=B5=D0=BD=D1=82=D1=8F=D0=B1=D1=80=D1=8F 2017 =D0=B3., 02:35, = "Keith Hellman" =D0=BD=D0=B0=D0=BF=D0=B8=D1= =81=D0=B0=D0=BB:=0A=0A> On Wed, Sep 20, 2017 at 11:06:11PM +0000, freebsd= -stable List wrote:=0A> =0A>> Sometimes it even can boot, but in few minu= tes will hang with same errors.=0A>> =0A>> Hardware: Supermicro X8DTN+-F = / 6xWD1502FYPS-02W3B0 /2xE5649=0A>> HDDs connected to sata ports on baseb= oard.=0A>> If I add=0A>> hint.ata.2.mode=3DPIO4=0A>> hint.ata.3.mode=3DPI= O4=0A>> hint.ata.4.mode=3DPIO4=0A>> hint.ata.5.mode=3DPIO4=0A>> to device= .hints I'm able to boot but performance of IO becomes really disappointin= g.=0A>> Also, if I add something like "find / -name something" to zfs rc = script, than boots fine.=0A> =0A> Boots fine but hangs in a few minutes? = Or all is good to go? Here is a=0A> thught: it could be the find / ... co= mmand is causing enough delay to=0A> let some hardware "settle down" and = behave more reasonably. Can you=0A> replace the find cmd with an equivale= nt sleep(1) command of like=0A> duration and get the same results?=0A> = =0A>> If I roll back system to 11.0 all works fine again.=0A> =0A> Seemin= gly minor deltas in the system could have had the needed delay=0A> as an = implicit part of the system...=0A> =0A> Just my 2c=0ANope, it can't boot = fine at all, or init will get segfault or ttys or just system becomes unr= esponsive and I can't even login, so this heppens during start of service= s. I've also tried to disable all "non-base" services, but without succes= s. =0AAs for trying just sleep - it doesn't helps. Seems that find helps = because of caching by system triggered by find.=0A=0A=0A=0A> --=0A> Keith= Hellman #include =0A> khellman@mcprogramming.com from disc= laimer import standard=0A> khellman@mines.edu gpg key 9FCF40FD=0A> freeno= de.net as mrtuple=0A> =0A> "The First Python function ever written (takes= place in the Garden of Eden)"=0A> =0A> Guido sayeth "I will write def fo= o():"=0A> "Hmm, I could use an import, or two",=0A> Satan said, in a whir= l, "Why not write it in Perl?",=0A> and the second function ever written = - def foo_you():=0A> =0A> -- Python Limmerick Contest submission by cappy= 2112=0A> http://groups-beta.google.com/group/comp.lang.python/browse_thre= ad/thread/d7a780beaff2e88a From owner-freebsd-stable@freebsd.org Sun Sep 24 15:30:16 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 88643E2A02F for ; Sun, 24 Sep 2017 15:30:16 +0000 (UTC) (envelope-from marius@alchemy.franken.de) Received: from alchemy.franken.de (alchemy.franken.de [194.94.249.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "alchemy.franken.de", Issuer "alchemy.franken.de" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 105C97CB82; Sun, 24 Sep 2017 15:30:15 +0000 (UTC) (envelope-from marius@alchemy.franken.de) Received: from alchemy.franken.de (localhost [127.0.0.1]) by alchemy.franken.de (8.15.2/8.15.2/ALCHEMY.FRANKEN.DE) with ESMTPS id v8OFUBXQ019592 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 24 Sep 2017 17:30:11 +0200 (CEST) (envelope-from marius@alchemy.franken.de) Received: (from marius@localhost) by alchemy.franken.de (8.15.2/8.15.2/Submit) id v8OFUAqH019591; Sun, 24 Sep 2017 17:30:10 +0200 (CEST) (envelope-from marius) Date: Sun, 24 Sep 2017 17:30:10 +0200 From: Marius Strobl To: Ian Smith Cc: freebsd-stable@freebsd.org, FreeBSD Release Engineering Team Subject: Re: FreeBSD 10.4-RC2 Now Available Message-ID: <20170924153010.GY20729@alchemy.franken.de> References: <20170923175450.GX20729@alchemy.franken.de> <20170924193857.D35468@sola.nimnet.asn.au> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; x-action=pgp-signed In-Reply-To: <20170924193857.D35468@sola.nimnet.asn.au> User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Sep 2017 15:30:16 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 On Sun, Sep 24, 2017 at 07:46:19PM +1000, Ian Smith wrote: > On Sat, 23 Sep 2017 19:54:50 +0200, Marius Strobl wrote: > > > o Given that the amd64 disc1 image was overflowing, more of the base > > components installed into the disc1 (live) file systems had to be > > disabled. Most notably, this removed the compiler toolchain from > > the disc1 images. All disabled tools are still available with the > > dvd1 images, though. > > 1) Does that also apply to the components installed on the memstick.img? Yes, given that the disc1 file systems are also used to create the memstick images. > 2) Are we any closer to gettng a larger memstick.img with dvd1 contents? No, given that we'd likely want to have a variant of the memstick images which include all base tools also for snapshots. However, so far the dvd1 images are only created for releases and around the notion to contain packages, i. e. it's not as simple as creating additional memstick images from the dvd1 file system. Also, given that with head the amd64 disc1 image is overflowing despite all of the stripping, the way to go for 12.0 likely is to drop the historical limits on images sizes along with the disc1 images and instead provide DVD images with all base tools and for releases, additional DVD images with packages on top. Similar for memstick images. All of this probably boils down to adding options to both makefs(8) and mkimg(1) for simply excluding the packages directory (as for makefs(8), simpler than via an mtree(8) file) first, so we neither need to employ hacks in the release scripts nor build yet another file system set. Marius -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQJ8BAEBCgBmBQJZx89+XxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ1M0Q5QjQzNTVGOTU5ODBGQzVENzZCMDIy MEI3MERFMTNGMUQxRTRGAAoJECC3DeE/HR5PrNwQAI7hNUN3IZRFcV3fzAl88GKP B6FmZKSWogxTwqLJCQgglJXoUkid5FfzMGH9qDjEE1g6W0sN4Q5aysv5+5s+WPWU Xfhl5ivm070dYDBaqbR0cxCaK7cbIScf14dbpxE6hNUYHV5hnQZp8kT+vvvWYIQz TZlHGiK1dccYH5WOg/UgR7nvs6piXI5XzmvP7o1mWv2TdT0DwaNXjuHznwZnAdW7 rStMPpvlC99a6+iAU74cbcBQi9w6Uuro+7V+DXZSwQRdP0PQsx1zOxkwAeY5+BKz PFK7w0SC5GB1ym7ddQSyY3+8bQOEelwOX99yqU9SCqPnqyMOl+l+CAv8gCenyoBl BgU4fTAAtsxNgne065/cGA+EEMDZtI2mdleGV8xXQGUFu2Dp8+HOhKBID95SW7LJ +WDAKUvbkQvw5URjFZ3TtsbqZQptGji/Xze8i6nJ1dv0aU2ajU3sjR/N5yy+wg9i KUK7K53Wx0TKVUkd4oUDePEUSjnRzalO8BPk4oLi08EvmX/yBMVswdaV4GIhCy5+ hSzud+98xmq3eDhoRsLqE2eRUbP9BegbM7VappEqFRE1I0mLQG9A/8DR49aZhzA2 TFg+rnYhjjW3gW3GtgZteHeRYIlk9ADsTxvbz2CEL2JpuUIFeb6g2OPPXeqTXZQZ hQPI71i7MhhJXRg+sQdM =nrnb -----END PGP SIGNATURE----- From owner-freebsd-stable@freebsd.org Mon Sep 25 06:30:47 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3DD71E0ACDA for ; Mon, 25 Sep 2017 06:30:47 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from kagate.punkt.de (kagate.punkt.de [217.29.33.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C68286FF5F for ; Mon, 25 Sep 2017 06:30:46 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from hugo10.ka.punkt.de (hugo10.ka.punkt.de [217.29.44.10]) by gate2.intern.punkt.de with ESMTP id v8P6UhAM062293 for ; Mon, 25 Sep 2017 08:30:43 +0200 (CEST) Received: from [217.29.44.197] ([217.29.44.197]) by hugo10.ka.punkt.de (8.14.2/8.14.2) with ESMTP id v8P6Uh8k057401 for ; Mon, 25 Sep 2017 08:30:43 +0200 (CEST) (envelope-from hausen@punkt.de) From: "Patrick M. Hausen" Content-Type: multipart/signed; boundary="Apple-Mail=_0A5EF0ED-6F3F-465D-AD86-AA10EAFF7C33"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: ABI changes within stable branch Date: Mon, 25 Sep 2017 08:30:44 +0200 References: <1b07bf49-508a-c6b4-e805-df7d43230f81@ish.com.au> <20170919081532.GB2170@home.opsec.eu> <20170920172734.GC10570@lonesome.com> To: freebsd-stable In-Reply-To: <20170920172734.GC10570@lonesome.com> Message-Id: <459A760F-DFD4-4EAA-9D16-75D2157A2E3E@punkt.de> X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Sep 2017 06:30:47 -0000 --Apple-Mail=_0A5EF0ED-6F3F-465D-AD86-AA10EAFF7C33 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Morning, > Am 20.09.2017 um 19:27 schrieb Mark Linimon : >=20 > On Tue, Sep 19, 2017 at 10:15:32AM +0200, Kurt Jaeger wrote: >> A pointer to the official policy would be nice 8-} >=20 > 3rd paragraph of: >=20 > http://www.freebsd.org/portmgr/policies_eol.html One comment: it's easy to overlook the implications of the word = "supported" in "Package builds will use the oldest supported minor release = within each major branch ..." Thanks for the link. Patrick -- punkt.de GmbH Internet - Dienstleistungen - Beratung Kaiserallee 13a Tel.: 0721 9109-0 Fax: -100 76133 Karlsruhe info@punkt.de http://punkt.de AG Mannheim 108285 Gf: Juergen Egeling --Apple-Mail=_0A5EF0ED-6F3F-465D-AD86-AA10EAFF7C33 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQEcBAEBCgAGBQJZyKKUAAoJEJBvLuLt2olc3OYH/jQxBxVF7KtBZOGy8XxY2NHv bxKldzuh/9DG+nv7ixVvsu2hjwa0jz5RIV6Pr4qBE5cH5C8yctEauG8DtmeeSuc3 iO/uCm1Wg758C6HM458DnIslI7W3Z8VQpFteqyRXubBdKPuFC5X8zee7WVRs3qMy buZul4CoaGK3fsIUTeq0PdWxBscEmRBWDisQeM4N8LmKqpWudt5R7aboE7Hlrpky ZXAIeUMHCZdxNUOj+s+PbGLIicH+Z3ohz1ykOrIqx5qvywIOks8RRdiAIFWi8Dnr eiaz5zye4R1r+wC+df02k6x6qUkTujJM+010TzOuJGjGrOaWi/jWs0fFC9s4dJw= =o2yQ -----END PGP SIGNATURE----- --Apple-Mail=_0A5EF0ED-6F3F-465D-AD86-AA10EAFF7C33-- From owner-freebsd-stable@freebsd.org Wed Sep 27 11:07:39 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A0831E30FDA for ; Wed, 27 Sep 2017 11:07:39 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: from mail-vk0-x230.google.com (mail-vk0-x230.google.com [IPv6:2607:f8b0:400c:c05::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 498CE6DAEA for ; Wed, 27 Sep 2017 11:07:39 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: by mail-vk0-x230.google.com with SMTP id d12so6850398vkf.1 for ; Wed, 27 Sep 2017 04:07:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=pn1Cr93OOGoUliWajVrgAnnrCxe3/Clv5d+UctasPhw=; b=cmgvRxuypNp+MYE/G/MtTPkoTXMcRPGBHXoXdw6gjnzarxw1utme/Bkgqg5dRE5cAl RGQTrAbsSvX+R/LXX1/InTx/Im32s0DZHmAKOCDg15V/8K3kIDpWk1akWF/5wDwKDe9s FHNGOlM1r9Ubm2Pf/WvZ3TbKA1jQ8sT8Un22F4bImMBRQM4XgxoiDuJoeo5W1Hk+qK/o IxhZelVLSJ3ubTJQGkhOBanvaY1evka/5YhMUZt8pc6LfQZVV3Jr0Bih1ITQTMRJJjML UA1AUEqeEGrgzdmKyqcHVHr7TSewLb9AqC852okxn/LP19rotcvNQZiMjOgFO5hWiwRj ht9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=pn1Cr93OOGoUliWajVrgAnnrCxe3/Clv5d+UctasPhw=; b=UXeuqhE8+QfvHmkwdmmuqZyuvBSl9t2bWufNfAdicG+l7A2uuewPnfALTcf61MvGR5 snTtFLj50IzQtlGUssiu55wUI5uuj6XtLzljpzi4tXRXRXOCK66BdMTtq46j6KaG3EgF XppLBXmXvW9IjAtIDr/mp9vkArZMOnRNudmFsoOafzZEcgz+sHB3sXX76AgAHp/ffslt /RMVIfhPh+AXnFWlUV6yRxQ64tzO8zWWY6WdeDffZWD5ZhY2dBidGSin+CJXBMs+2SQo d+kw2ZL11d6gBzkSazePYwsTjtmJf33DZFdVU8eChZgVI9MIbYgJ6FPKQt8ejokvFauP Sbdg== X-Gm-Message-State: AHPjjUjSD2Ma9n3pJf75MGk65alf1ja8Rfj4DRSf/kGBvPhmSNVz424V omAUN0SHiYsnWcG5mtGNIfVlwY+FnIziiBrrkhufHw== X-Google-Smtp-Source: AOwi7QCArz3ciEqy9VvQw5UG1+jqWutj5kmpM/2cAQ1PmONBqXnk6pcElna5ntF0uav3S4hRyw0tgsQwOWISeeGeTgg= X-Received: by 10.31.178.200 with SMTP id b191mr557809vkf.46.1506510458419; Wed, 27 Sep 2017 04:07:38 -0700 (PDT) MIME-Version: 1.0 Sender: etnapierala@gmail.com Received: by 10.159.51.232 with HTTP; Wed, 27 Sep 2017 04:07:38 -0700 (PDT) In-Reply-To: References: From: Edward Napierala Date: Wed, 27 Sep 2017 13:07:38 +0200 X-Google-Sender-Auth: bxVQMui_bU9omalkt3lTG5Swr24 Message-ID: Subject: Re: zfs, iSCSI and volmode=dev To: "Eugene M. Zheganin" Cc: FreeBSD Stable Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Sep 2017 11:07:39 -0000 2017-08-30 11:45 GMT+02:00 Eugene M. Zheganin : > Hi, > > > I have an iSCSI production system that exports a large number of zvols as > the iSCSI targets. System is running FreeBSD 11.0-RELEASE-p7 and initially > all of the zvols were confugured with default volmode. I've read that it's > recommended to use them in dev mode, so the system isn't bothered with all > of these geom structures, so I've switched all of the zvols to dev mode, > then I exported/imported the pools back. Surprisingly, the performance has > fallen down like 10 times (200-300 Mbits/sec against 3-4 Gbits/sec > previously). After observing for 5 minutes the ESXes trying to boot up, and > doing this extremely slowly, I switched the volmode back to default, then > again exported/imported the pools. The performance went back to normal. > > > So... why did this happen ? The result seems to be counter-intuitive. At > least not obvious to me. I don't really have an answer - mav@ would be the best person to ask. Based on his description, "ZVOLs in GEOM mode don't support DPO/FUA cache control bits, had to chunk large I/Os into MAXPHYS-sized pieces and go through GEOM." There also used to be so that TRIM was only supported in the "dev" mode, but that changed a while ago. From owner-freebsd-stable@freebsd.org Wed Sep 27 17:41:08 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6B0B4E09C79 for ; Wed, 27 Sep 2017 17:41:08 +0000 (UTC) (envelope-from chris@vindaloo.com) Received: from yavin.vindaloo.com (yavin.vindaloo.com [173.199.117.73]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "yavin.vindaloo.com", Issuer "Vindaloo Sign CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 436747FCC1 for ; Wed, 27 Sep 2017 17:41:07 +0000 (UTC) (envelope-from chris@vindaloo.com) Received: from anza.vindaloo.com (ool-45714982.dyn.optonline.net [69.113.73.130]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "smtp.vindaloo.com", Issuer "Vindaloo Sign CA" (verified OK)) by yavin.vindaloo.com (Postfix) with ESMTPS id 15D9BD7A9F for ; Wed, 27 Sep 2017 13:35:30 -0400 (EDT) Received: from kessel.vindaloo.com (kessel.vindaloo.com [172.24.145.71]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by anza.vindaloo.com (Postfix) with ESMTPSA id 95302101E2; Wed, 27 Sep 2017 13:35:29 -0400 (EDT) Date: Wed, 27 Sep 2017 13:35:25 -0400 From: Christopher Sean Hilton To: freebsd-stable@freebsd.org Subject: Bind9 + TCP_FASTOPEN => no rndc Message-ID: <20170927173525.bspia3tpcu35yng3@kessel.vindaloo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: NeoMutt/20170714 (1.8.3) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Sep 2017 17:41:08 -0000 I'm trying to configure bind 9.11 as a nameserver on FreeBSD 11-STABLE. When the bind9 port compile it enables TCP_FASTOPEN but the changes haven't yet been baked into the GENERIC Kernel. I can't find a way to disable the use of TCP_FASTOPEN in bind at startup. Is the only way to fix this problem to build a new kernel with TCP_FASTOPEN enabled? -- Chris -- Chris __o "All I was trying to do was get home from work." _`\<,_ -Rosa Parks ___(*)/_(*)____.___o____..___..o...________ooO..._____________________ Christopher Sean Hilton [chris/at/vindaloo/dot/com] From owner-freebsd-stable@freebsd.org Wed Sep 27 19:20:03 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B324FE0CEFB for ; Wed, 27 Sep 2017 19:20:03 +0000 (UTC) (envelope-from hiren@strugglingcoder.info) Received: from mail.strugglingcoder.info (strugglingcoder.info [104.236.146.68]) by mx1.freebsd.org (Postfix) with ESMTP id A548883042 for ; Wed, 27 Sep 2017 19:20:03 +0000 (UTC) (envelope-from hiren@strugglingcoder.info) Received: from localhost (unknown [10.1.1.3]) (Authenticated sender: hiren@strugglingcoder.info) by mail.strugglingcoder.info (Postfix) with ESMTPA id 48A2917F7D; Wed, 27 Sep 2017 12:12:01 -0700 (PDT) Date: Wed, 27 Sep 2017 12:12:01 -0700 From: hiren panchasara To: Christopher Sean Hilton Cc: freebsd-stable@freebsd.org Subject: Re: Bind9 + TCP_FASTOPEN => no rndc Message-ID: <20170927191201.GE5755@strugglingcoder.info> References: <20170927173525.bspia3tpcu35yng3@kessel.vindaloo.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="tMbDGjvJuJijemkf" Content-Disposition: inline In-Reply-To: <20170927173525.bspia3tpcu35yng3@kessel.vindaloo.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Sep 2017 19:20:03 -0000 --tMbDGjvJuJijemkf Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On 09/27/17 at 01:35P, Christopher Sean Hilton wrote: > I'm trying to configure bind 9.11 as a nameserver on FreeBSD > 11-STABLE. When the bind9 port compile it enables TCP_FASTOPEN but the > changes haven't yet been baked into the GENERIC Kernel. I can't find a > way to disable the use of TCP_FASTOPEN in bind at startup. Is the only > way to fix this problem to build a new kernel with TCP_FASTOPEN > enabled? afaik, yes. options TCP_RFC7413 in kernconf. It defaults to off even with the option. net.inet.tcp.fastopen.enabled=1 will enable actual use. Cheers, Hiren --tMbDGjvJuJijemkf Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQF8BAABCgBmBQJZy/f+XxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRBNEUyMEZBMUQ4Nzg4RjNGMTdFNjZGMDI4 QjkyNTBFMTU2M0VERkU1AAoJEIuSUOFWPt/l1tUIAIM/P3VAz+VQiwvJ9QNpx3Pp hxFs5r1cZv2LGLjoVl7YXjWGx0+vkFEyBPqqGSgFTYcaH9OOh4NMEu5wEzkPTqXv 2J83rBzpuHtwur4vv6G/NHnyXdVRxPFSBCU4piuLpJzzpE39g8YzGC2vk3rRAWA9 5AS47f5JjbTyvXXnjqaek6/w/kAmui68TIMCzMZvzxfpO4I1p7LULulgRixrHRfP KqaNIoRVPgpE3BZrFDAgYfngLpAaD4anU1+/DGdEyPpR25pz9vFvrMiufOPcFbB+ UFg+OYhvldfxTuBPG0OE7IWc9H16gouCQbkWiXAs8dagdMAITUX1R9dt80ANsUY= =YM7U -----END PGP SIGNATURE----- --tMbDGjvJuJijemkf-- From owner-freebsd-stable@freebsd.org Wed Sep 27 20:00:54 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 53061E0E19D for ; Wed, 27 Sep 2017 20:00:54 +0000 (UTC) (envelope-from chris@vindaloo.com) Received: from yavin.vindaloo.com (yavin.vindaloo.com [173.199.117.73]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "yavin.vindaloo.com", Issuer "Vindaloo Sign CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 30BE784E19 for ; Wed, 27 Sep 2017 20:00:53 +0000 (UTC) (envelope-from chris@vindaloo.com) Received: from anza.vindaloo.com (ool-45714982.dyn.optonline.net [69.113.73.130]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "smtp.vindaloo.com", Issuer "Vindaloo Sign CA" (verified OK)) by yavin.vindaloo.com (Postfix) with ESMTPS id 32A1ED7B3C; Wed, 27 Sep 2017 16:00:52 -0400 (EDT) Received: from csh-desktop-vm00.loopone.com (h4.82.141.40.ip.windstream.net [40.141.82.4]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client did not present a certificate) by anza.vindaloo.com (Postfix) with ESMTPSA id A501C101E2; Wed, 27 Sep 2017 16:00:51 -0400 (EDT) Date: Wed, 27 Sep 2017 16:00:49 -0400 From: Christopher Sean Hilton To: David Wolfskill Cc: freebsd-stable@freebsd.org Subject: Re: Bind9 + TCP_FASTOPEN => no rndc Message-ID: <20170927200033.pqwweoncqm4k627d@csh-desktop-vm00.loopone.com> References: <20170927173525.bspia3tpcu35yng3@kessel.vindaloo.com> <20170927175131.GE1165@albert.catwhisker.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: <20170927175131.GE1165@albert.catwhisker.org> User-Agent: NeoMutt/20170914 (1.9.0) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Sep 2017 20:00:54 -0000 On Wed, Sep 27, 2017 at 05:51:31PM +0000, David Wolfskill wrote: > On Wed, Sep 27, 2017 at 01:35:25PM -0400, Christopher Sean Hilton wrote: > > I'm trying to configure bind 9.11 as a nameserver on FreeBSD > > 11-STABLE. When the bind9 port compile it enables TCP_FASTOPEN but the > > changes haven't yet been baked into the GENERIC Kernel. I can't find a > > way to disable the use of TCP_FASTOPEN in bind at startup. Is the only > > way to fix this problem to build a new kernel with TCP_FASTOPEN > > enabled? > >=20 > > -- Chris > > .... >=20 > ? I'm running bind99-9.9.11 (dns/bind99) on a couple systems running > stable/11 (amd64; currently r323950). The kernels are (lightly) > customized, based on GENERIC. I don't recall setting anything involving > TCP_FASTOPEN on anything, and have used rndc without issue.... >=20 > Perhaps you could elaborate a bit on exactly what you are trying to do > and how the system responds? (The systems in question run kernels that > are built on a dedicated "build machine" -- which is presently powered > off for the day. I can bring it up for a reality check, should that be > wanted.) >=20 Good afternoon David, Thanks for the help! I'm running ports ?net?/bind911 of FreeBSD 11-STABLE with the GENERIC kernel. When I start bind, I get this in my logs: Sep 27 13:16:13 alderaan named[30169]: starting BIND 9.11.2 Sep 27 13:16:13 alderaan named[30169]: running on FreeBSD amd64 11.1-PREREL= EASE FreeBSD 11.1-PRERELEASE #2 r321128: Tue Jul 18 11:30:08 EDT 2017 r= oot@freebsd-mule:/usr/obj/usr/src/sys/GENERIC Sep 27 13:16:13 alderaan named[30169]: built with '--localstatedir=3D/var' = '--disable-linux-caps' '--disable-symtable' '--with-randomdev=3D/dev/random= ' '--with-libxml2=3D/usr/local' '--with-readline=3D-L/usr/local/lib -ledit'= '--with-dlopen=3Dyes' '--sysconfdir=3D/usr/local/etc/namedb' '--disable-dn= stap' '--disable-filter-aaaa' '--disable-fixed-rrset' '--without-geoip' '--= with-idn=3D/usr/local' '--enable-ipv6' '--with-libjson' '--disable-largefil= e' '--with-lmdb' '--without-python' '--disable-querytrace' '--enable-rpz-ns= dname' '--enable-rpz-nsip' 'STD_CDEFINES=3D-DDIG_SIGCHASE=3D1' '--enable-th= reads' '--without-gssapi' '--with-openssl=3D/usr' '--disable-native-pkcs11'= '--with-dlz-filesystem=3Dyes' '--without-gost' '--prefix=3D/usr/local' '--= mandir=3D/usr/local/man' '--infodir=3D/usr/local/info/' '--build=3Damd64-po= rtbld-freebsd11.0' 'build_alias=3Damd64-portbld-freebsd11.0' 'CC=3Dcc' 'CFL= AGS=3D-O2 -pipe -DLIBICONV_PLUG -fstack-protector -isystem /usr/local/inclu= de -fno-strict-aliasing' 'LDFLAGS=3D -fstack-protector' 'LIBS=3D-L/usr/loca= l/lib' 'CPPFLAGS=3D-D Sep 27 13:16:13 alderaan named[30169]: running as: named -t /var/named -u b= ind -c /etc/namedb/named.conf Sep 27 13:16:13 alderaan named[30169]: ------------------------------------= ---------------- Sep 27 13:16:13 alderaan named[30169]: BIND 9 is maintained by Internet Sys= tems Consortium, Sep 27 13:16:13 alderaan named[30169]: Inc. (ISC), a non-profit 501(c)(3) p= ublic-benefit=20 Sep 27 13:16:13 alderaan named[30169]: corporation. Support and training f= or BIND 9 are=20 Sep 27 13:16:13 alderaan named[30169]: available at https://www.isc.org/sup= port Sep 27 13:16:13 alderaan named[30169]: ------------------------------------= ---------------- Sep 27 13:16:13 alderaan named[30169]: socket.c:5695: unexpected error: Sep 27 13:16:13 alderaan named[30169]: setsockopt(21, TCP_FASTOPEN) failed = with Protocol not available Sep 27 13:16:13 alderaan named[30169]: socket.c:5695: unexpected error: Sep 27 13:16:13 alderaan named[30169]: setsockopt(22, TCP_FASTOPEN) failed = with Protocol not available Sep 27 13:16:13 alderaan named[30169]: socket.c:5695: unexpected error: Sep 27 13:16:13 alderaan named[30169]: setsockopt(23, TCP_FASTOPEN) failed = with Protocol not available Sep 27 13:16:13 alderaan named[30169]: socket.c:5695: unexpected error: Sep 27 13:16:13 alderaan named[30169]: setsockopt(24, TCP_FASTOPEN) failed = with Protocol not available Sep 27 13:16:13 alderaan named[30169]: couldn't add command channel 127.0.0= =2E1#953: file not found Sep 27 13:16:13 alderaan named[30169]: couldn't add command channel ::1#953= : file not found Sep 27 13:16:13 alderaan named[30169]: all zones loaded I haven't read the bind source code yet but I'm assuming that the inability to start rndc at 127.0.0.1#953 is related to the TCP_FASTOPEN error from the log above. Not much Google reveals this thread:=20 https://forums.freebsd.org/threads/59367/ Which talks about the problem and mentions one, and only one, solution of rebuilding the kernel to support TCP_FASTOPEN. That solution is kind of heavyweight for me. If you read more about tcp_fastopen, you'll get indications that the code may be too green right now to be enabled by default. Please pardon any file blunders here, I'm at work so it's not easy to research this completely. From what I can see though, with the option id defined in but it needs to be compiled in and then enabled via sysctl if you want to actually use it.=20 I was hoping that bind had a runtime option disable this feature but I can't find it anywhere. I'll look at the bind source code tonight. I'll be hoping to find a config switch or something that can turn TCP_FASTOPEN off even if the header files say that it's available. If it's there, I'll submit a patch to the port's config to toggle that switch at compile time. --=20 Chris __o "All I was trying to do was get home from work." _`\<,_ -Rosa Parks ___(*)/_(*)____.___o____..___..o...________ooO..._____________________ Christopher Sean Hilton [chris/at/vindaloo/dot/com] From owner-freebsd-stable@freebsd.org Wed Sep 27 21:17:34 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 95941E0F82F for ; Wed, 27 Sep 2017 21:17:34 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (tensor.andric.com [IPv6:2001:470:7a58:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "tensor.andric.com", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5EDFB2579 for ; Wed, 27 Sep 2017 21:17:34 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from [10.194.10.54] (unknown [185.93.6.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 46E594759A; Wed, 27 Sep 2017 23:17:31 +0200 (CEST) From: Dimitry Andric Message-Id: <5CF82983-FDA1-4F83-9D47-D36845A12B97@FreeBSD.org> Content-Type: multipart/signed; boundary="Apple-Mail=_CC06C6CF-F423-405E-B94A-8ADAEE72212F"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: Bind9 + TCP_FASTOPEN => no rndc Date: Wed, 27 Sep 2017 23:17:29 +0200 In-Reply-To: <20170927173525.bspia3tpcu35yng3@kessel.vindaloo.com> Cc: freebsd-stable@freebsd.org To: Christopher Sean Hilton References: <20170927173525.bspia3tpcu35yng3@kessel.vindaloo.com> X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Sep 2017 21:17:34 -0000 --Apple-Mail=_CC06C6CF-F423-405E-B94A-8ADAEE72212F Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii On 27 Sep 2017, at 19:35, Christopher Sean Hilton = wrote: >=20 > I'm trying to configure bind 9.11 as a nameserver on FreeBSD > 11-STABLE. When the bind9 port compile it enables TCP_FASTOPEN but the > changes haven't yet been baked into the GENERIC Kernel. I can't find a > way to disable the use of TCP_FASTOPEN in bind at startup. Is the only > way to fix this problem to build a new kernel with TCP_FASTOPEN > enabled? It looks like bind enables use of TCP_FASTOPEN whenever its configure script finds the define in the system headers. But it does not check whether the functionality actually works with setsockopt. In any case, the message is harmless noise, as any errors are ignored: #if defined(ISC_PLATFORM_HAVETFO) && defined(TCP_FASTOPEN) #ifdef __APPLE__ backlog =3D 1; #else backlog =3D backlog / 2; if (backlog =3D=3D 0) backlog =3D 1; #endif if (setsockopt(sock->fd, IPPROTO_TCP, TCP_FASTOPEN, (void *)&backlog, sizeof(backlog)) < 0) { isc__strerror(errno, strbuf, sizeof(strbuf)); UNEXPECTED_ERROR(__FILE__, __LINE__, "setsockopt(%d, TCP_FASTOPEN) failed = with %s", sock->fd, strbuf); /* TCP_FASTOPEN is experimental so ignore failures */ } #endif -Dimitry --Apple-Mail=_CC06C6CF-F423-405E-B94A-8ADAEE72212F Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.2 iF0EARECAB0WIQR6tGLSzjX8bUI5T82wXqMKLiCWowUCWcwVaQAKCRCwXqMKLiCW o2MOAJ9aN9jtBYJ4oTdzXDja6ontQJrEiQCgzjdDxLUVW+7aOhEYb935UBgJhjc= =EHXW -----END PGP SIGNATURE----- --Apple-Mail=_CC06C6CF-F423-405E-B94A-8ADAEE72212F-- From owner-freebsd-stable@freebsd.org Wed Sep 27 23:06:58 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C993DE12A72; Wed, 27 Sep 2017 23:06:58 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from dmz-mailsec-scanner-5.mit.edu (dmz-mailsec-scanner-5.mit.edu [18.7.68.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1AC0B6474C; Wed, 27 Sep 2017 23:06:54 +0000 (UTC) (envelope-from kaduk@mit.edu) X-AuditID: 12074422-8b1ff70000004e42-dd-59cc2dd90556 Received: from mailhub-auth-3.mit.edu ( [18.9.21.43]) (using TLS with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by dmz-mailsec-scanner-5.mit.edu (Symantec Messaging Gateway) with SMTP id 58.1E.20034.9DD2CC95; Wed, 27 Sep 2017 19:01:45 -0400 (EDT) Received: from outgoing.mit.edu (OUTGOING-AUTH-1.MIT.EDU [18.9.28.11]) by mailhub-auth-3.mit.edu (8.13.8/8.9.2) with ESMTP id v8RN1ivs018522; Wed, 27 Sep 2017 19:01:44 -0400 Received: from kduck.kaduk.org (24-107-191-124.dhcp.stls.mo.charter.com [24.107.191.124]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id v8RN1dKE001435 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 27 Sep 2017 19:01:41 -0400 Date: Wed, 27 Sep 2017 18:01:39 -0500 From: Benjamin Kaduk To: freebsd-hackers@FreeBSD.org Cc: freebsd-current@FreeBSD.org, freebsd-stable@FreeBSD.org Subject: FreeBSD Quarterly Status Report - Second Quarter 2017 Message-ID: <20170927230138.GB96685@kduck.kaduk.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="HeFlAV5LIbMFYYuh" Content-Disposition: inline User-Agent: Mutt/1.8.3 (2017-05-23) X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrPKsWRmVeSWpSXmKPExsUixCmqrXtT90ykwYujWha7rp1mt5jz5gOT xfbN/xgtDjcLObB4zPg0nyWAMYrLJiU1J7MstUjfLoEr4/GbQ+wFixazViz9/YCxgXH9RJYu Rg4OCQETicZPaV2MXBxCAouZJL7NnMkM4WxklNjRN58JwrnKJHHu4QugDk4OFgFViefrFrCD 2GwCahLrV1xjBrFFBOQl9jW9B4szC1hL/HvQCBYXFrCTmHFwJROIzQu0rW35PGYIW1Di5Mwn LBD1ZRKf9qxiA7mIWUBaYvk/DpCwqICyxLx9q9gmMPLNQtIxC0nHLIQOiLC6xJ95l5gxhLUl li18zQxh20qsW/eeZQEj+ypG2ZTcKt3cxMyc4tRk3eLkxLy81CJdU73czBK91JTSTYyg8GZ3 UdrBOPGf1yFGAQ5GJR7eCwtORwqxJpYVV+YeYpTkYFIS5W1UPhMpxJeUn1KZkVicEV9UmpNa fIhRBWjXow2rLzBKseTl56UqifCeZwaq401JrKxKLcqHKZPmYFES590WtCtSSCA9sSQ1OzW1 ILUIJivDwaEkwftbB6hRsCg1PbUiLTOnBCHNxMF5iFGCgwdoeCpIDW9xQWJucWY6RP4UYyxH 14qLf5g4jm26DCQPTLgCJPfdvAMkH924CyTn3ASRm8Dkiv/3gOSG7w/+MAmBXSwlzvsJZKgA yNCM0jy4vaB0KJG9v+YVozgwSIR5k4HJUYgHmErhNr8COooJ6KjeqSdAjipJREhJNTC6V1b9 nmBxaOKR6/84T5n9vmm2U3fBI8uXdklxXKtWKd5K/Z3wyz/mb9q2MyeKfyxwty5t+xhwnWnO gvO776woKHkXMm/JOpUkL8s9bAwMTnxPWtSyCxvn6962OZezLztVI7h8Pdvv6+8kVW+e/BIm rvBLbD/7VuuVbFrCVjLl/lJf64rY1m9XYinOSDTUYi4qTgQAcmBcT2IDAAA= X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Sep 2017 23:06:58 -0000 --HeFlAV5LIbMFYYuh Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable FreeBSD Quarterly Status Report - 2nd Quarter 2017 FreeBSD continues to defy the rumors of its demise. Much of the development work done this quarter was not particularly visible, especially the effort needed to ensure the upcoming 11.1 release has as few regressions as possible. Planning is also well under way for the 10.4 maintenance release which will quickly follow it. Further work focused on moving the arm architectures' support closer to tier-1 status and improving documentation. In addition, large changes were made to the src and ports trees. These projects and others are further detailed below. --Mark Linimon __________________________________________________________________ The deadline for submissions covering the period from July to September 2017 is October 21, 2017. __________________________________________________________________ FreeBSD Team Reports * FreeBSD Release Engineering Team * Ports Collection * The FreeBSD Core Team * The FreeBSD Foundation * The Postmaster Team Projects * 64-bit Inode Numbers * Capability-Based Network Communication for Capsicum/CloudABI * Ceph on FreeBSD * DTS Updates Kernel * Coda revival * FreeBSD Driver for the Annapurna Labs ENA * Intel 10G Driver Update * pNFS Server Plan B Architectures * FreeBSD on Marvell Armada38x * FreeBSD/arm64 Userland Programs * DTC * Using LLVM's LLD Linker as FreeBSD's System Linker Ports * A New USES Macro for Porting Cargo-Based Rust Applications * GCC (GNU Compiler Collection) * GNOME on FreeBSD * KDE on FreeBSD * New Port: FRRouting * PHP Ports: Help Improving QA * Rust * sndio Support in the FreeBSD Ports Collection * TensorFlow * Updating Port Metadata for non-x86 Architectures * Xfce on FreeBSD Documentation * Absolute FreeBSD, 3rd Edition * Doc Version Strings Improved by Their Absence * New Xen Handbook Section Miscellaneous * BSD Meetups at Rennes (France) Third-Party Projects * HardenedBSD __________________________________________________________________ FreeBSD Team Reports FreeBSD Release Engineering Team Links FreeBSD 11.1-RELEASE Schedule URL: https://www.FreeBSD.org/releases/11.1R/schedule.html FreeBSD Development Snapshots URL: https://download.FreeBSD.org/ftp/snapshots/ISO-IMAGES/ Contact: FreeBSD Release Engineering Team The FreeBSD Release Engineering Team is responsible for setting and publishing release schedules for official project releases of FreeBSD, announcing code freezes, and maintaining the respective branches, among other things. The FreeBSD 11.1-RELEASE cycle started on May 19, and continued as scheduled. FreeBSD consumers are urged to test whenever possible to help ensure the reliability and stability of the upcoming second release from the stable/11 branch. This project was sponsored by The FreeBSD Foundation. __________________________________________________________________ Ports Collection Links About FreeBSD Ports URL: https://www.FreeBSD.org/ports/ Contributing to Ports URL: https://www.freebsd.org/doc/en_US.ISO8859-1/articles/contributing/= ports-contributing.html FreeBSD Ports Monitoring URL: http://portsmon.freebsd.org/index.html Ports Management Team Website URL: https://www.freebsd.org/portmgr/index.html FreeBSD portmgr on Twitter (@freebsd_portmgr) URL: https://twitter.com/freebsd_portmgr/ FreeBSD Ports Management Team on Facebook URL: https://www.facebook.com/portmgr FreeBSD Ports Management Team on Google+ URL: https://plus.google.com/communities/108335846196454338383 Contact: Ren=C3=A9 Ladan Contact: FreeBSD Ports Management Team This quarter, 2017Q2, broke the 30,000 ports landmark for the first time. The PR count is currently just under 2,500, with almost 600 of them unassigned. This quarter saw almost 7,400 commits from 171 committers. More PRs got closed this quarter than last quarter, but also more PRs got sent in, both of which are good to see. Over the past three months, we welcomed four new committers: Bradley T. Hughes (bhughes@), Danilo G. Baio (dbaio@), Jochen Neumeister (joneum@), and Richard Gallamore (ultima@). kan@ re-joined us as a ports committer. One commit bit, that of bf@, was taken in for safekeeping after a long period of inactivity. On the management side, the Ports Management Team welcomed back bapt@, who is working on several new features for the Ports Tree. The Ports Management Team also had its annual real-life meeting during BSDCan. On the infrastructure side, three new USES values were introduced: * cargo, to ease the porting of Rust packages or binaries using the cargo command (also covered separately in this report) * groff, to handle a dependency on the groff document formatting system, that has been removed from the base system for FreeBSD 12 * meson, to provide support for projects based on Meson The default version of PostgreSQL switched from 9.3 to 9.5, and that of Python3 from 3.5 to 3.6. The default generator for ports using cmake has been switched to ninja. Some major version updates are: pkg 1.10.1, Firefox 54.0.1, and Chromium 59.0.3071.115. Behind the scenes, antoine@ ran 36 exp-runs to test version updates, make the CRAN ports platform-independent, test installing bsdgrep(1) as /usr/bin/grep, test LLVM updates, test the ino64 project, and perform Makefile cleanups. __________________________________________________________________ The FreeBSD Core Team Contact: FreeBSD Core Team Core's activities during the second quarter culminated in the introduction of two new initiatives during BSDCan: * Extending FreeBSD Project Membership * The FreeBSD Community Process FreeBSD Project Members FreeBSD Project Membership being extended to more than just committers is a step that enables the Project to recognise and reward people who support us in ways other than by writing code. People that organise conferences or user groups; who are prominent supporters on social media; who triage bug reports and who test changes; and many others who contribute in various ways, are deserving of recognition for the support that they give to the Project. Core hopes that this will both encourage more people to volunteer their time and effort on behalf of the Project, and encourage those who already do to stick with the Project, if not become more deeply involved. The naming for the new group of non-committer Project members took a few tries to get right: having tried, and rejected, "Contributor" and then "Associate", Core took the view that since what they were offerring was formal Project Membership, then that was the right thing to call it. Committers thus become those Project Members with access to commit to the Project's code repositories. Project Members receive an @FreeBSD.org e-mail address, access to various Project hardware, access to internal mailing lists and other communications channels, and invitations to attend Developer Summits in their own right. Committers in addition have commit rights in the Subversion repositories and GitHub, and active Committers can vote in Core team elections. The FreeBSD Community Process This is an idea that has a long pedigree within other projects, and FreeBSD is very consciously modelling its implementation on what has worked elsewhere. When a significantly disruptive or wide-scale change is proposed, we should have a formal mechanism for documenting the change and what it implies. Interested parties can then respond and the change can be evolved into the best fit for all users, or else it can be found to be impracticable and withdrawn. The documentation of the change will remain as a point of reference should the same or a similar proposal come up in the future. Creating a more formal process should help avoid endless sterile arguments about what needs to be done, without anyone feeling they have sufficient investment in the idea nor backing from the majority of the project to justify putting in the work to achieve the desired result. The very first FCP -- FCP 0 -- describes the process itself. At the time of this writing, Core is voting on accepting the initial document, which can be viewed in the Project's Github repository. Two new mailing lists have been created: fcp@FreeBSD.org is the channel for receiving notifications of new FCP proposals and discussing their content, whilst fcp-editors@FreeBSD.org exists to provide help with the process of drafting the FCP documents. Other Core activities Core is delighted to announce that Gordon Tetlow has joined the Security Officer team, and will be working on managing the Security Team caseload, freeing up other members to concentrate on the more technical aspects of vulnerability remediation. In addition, Ed Maste has joined the Security Team and is available to assist the Security Officers where necessary. Although Florian Smeets had to step down, the postmaster team has recruited three new members and is now back up to strength. Considering the desirability of a number of fixes that have been merged into 10-STABLE since the 10.3 release, core has approved a 10.4 release to occur shortly after the 11.1 release. This will be a normal support-lifetime release, unlike the extended lifetime of the 10.3 release, so the overall support lifetime for the 10.x branch will not be significantly extended. During this quarter, Core has approved issuing three new commit bits. Please welcome: * Vladimir Kondratyev (wulf@) * Ryan Libby (rlibby@) * Kyle Evans (kevans@) Also, during this quarter, we had one person give up their commit bit: * Jordan Hubbard (jkh@) It is always unsettling when one of the Project's founding members decides to move on, but Jordan's interests have migrated away from FreeBSD-related projects and he has decided to hang up his bit once and for all. Core would like to thank NTTA (formerly Verio) for providing hosting for a cvsup mirror for many years, and also for their kind offer to provide ongoing hosting for a machine in their Seattle facility. Since we have no need for additional North America hosting, we have declined their offer. As usual, a number of questions have been raised about code licensing and other matters related to intellectual property. Ed Maste has registered "freebsd" on behalf of the FreeBSD Foundation on the Mastodon social media network. The "Unlicense" is suitable for code being imported into libc. We still have some code published under the old 4-clause style BSD license, where the extra clause refers specifically to the University of California. While UC has generally approved removing that clause, we need to check with all copyright holders before changing any remaining 4-clause licensing. Core, along with the Security Team, are monitoring developments concerning the "Stack Clash" vulnerability that hit the headlines during June. Changes to the stack-guard mitigation system are underway as a response to the proof-of-concept published by Qualys. __________________________________________________________________ The FreeBSD Foundation Links FreeBSD Foundation Website URL: https://www.FreeBSDFoundation.org/ FreeBSD Foundation Quarterly Newsletter URL: https://www.FreeBSDfoundation.org/wp-content/uploads/2017/06/FreeB= SD-Foundation-Q2-2017-Update.pdf Contact: Deb Goodkin Last quarter the Foundation was busy supporting the FreeBSD Project in so many ways! We brought on two interns from the University of Waterloo who were extremely productive, from working on a continuous integration project to adding MSDOS FAT filesystem support to makefs. We continued helping to accelerate OS changes with our internal staff of software developers, as well as funding outside software development projects, and continued promoting FreeBSD by participating in technology conferences around the world. To encourage more commercial users to donate to the Foundation, we launched a new partnership program. The FreeBSD 11.1 release effort has been led by a full-time Foundation employee, to continue keeping releases timely and reliable. Finally, we led the effort to celebrate the newly declared FreeBSD Day, to help raise awareness of FreeBSD around the world! Below, you can read some of the highlights from our Q2 newsletter, and find writeups throughout this status report from Foundation staff members including Ed Maste, Kostik Belousov, and Glen Barber. Don't forget, we are 100% funded by donations. Please take a moment to donate now, so we can continue supporting the FreeBSD Project and community worldwide! Q2 Development Projects Summary Our hard work continues into the 2nd quarter of 2017. Please take a look at the highlights from our more recent Development Projects summaries. April: FreeBSD USB Mass Storage Target Project Update The Foundation awarded a project grant to Edward Tomasz Napiera=C5=82a to develop a USB mass storage target driver, using the FreeBSD CAM Target Layer (CTL) as a backend. This project allows FreeBSD on an embedded platform, such as a BeagleBone Black or Raspberry Pi Zero, to emulate a USB mass storage target, commonly known as a USB flash stick. Read more at https://www.FreeBSDfoundation.org/blog/april-2017-development-project= s-update/. May: Foundation Brings on Co-Op Students At the beginning of May we embarked on a new path in the FreeBSD Foundation, with the hiring of co-operative education (co-op) students from the University of Waterloo. The University of Waterloo is a pioneer and leader in co-operative education, with 100% of Engineering students and a majority of Computer Science students participating in co-op programs. Read more at https://www.FreeBSDfoundation.org/blog/may-2017-development-projects-upd= ate/. June: FreeBSD Foundation 2017 Project Proposal Solicitation (contributed by Ed Maste) One of the ways the Foundation supports FreeBSD is by providing development grants for work on individual projects. These allow developers to propose projects they would like to undertake to improve FreeBSD and request funding to perform that work. The Foundation is always willing to receive proposals, but will occasionally issue a call for proposals to highlight specific areas of focus and to be able to collect and evaluate a group of proposals. The proposal submission deadline was July 14, 2017, but as mentioned above, people are welcome to submit proposals at any time. Although proposals may address any FreeBSD subsystem or infrastructure, we are particularly interested in receiving proposals related to: * Improvements to the security of FreeBSD itself, or of applications running on FreeBSD * New test cases, improved test infrastructure, and quality assurance * Improved software development tools * Projects to improve community collaboration and communication * Improving the FreeBSD "out of the box" experience for new users on various hardware platforms * Establishing FreeBSD as a leader in advancing projects of shared interest (such as ZFS, LLVM, or libarchive) More details can be found at https://www.FreeBSDfoundation.org/blog/FreeBSD-foundation-2017-project-p= roposal-solicitation/. The full project proposal submission guidelines can be found at http://cts.vresp.com/c/?FreeBSDFoundation/d364934d4d/TEST/1b229d9af7. Please do not hesitate to contact proposals@FreeBSDfoundation.org with any questions. Announcing the New Partnership Program (contributed by Deb Goodkin) I'm excited to announce our new FreeBSD Foundation Partnership Program! Our work is 100% supported by donations from individuals and organizations. With a spending budget of $1,500,000, we rely on large donations from our commercial users to help us sustain and increase our support. Recognizing the value of these donations, and putting together a sustainable funding model, we wanted to institute benefits that highlighted this support, and recognize these donors in productive ways. Partnerships are an avenue to assist commercial users by helping them get on board more quickly with FreeBSD, share their needs with the community, and facilitate collaboration with FreeBSD developers. We believe that building these relationships with commercial users will contribute to keeping FreeBSD relevant and help provide a sustainable and healthy ecosystem. You can check out our updated donor pages to see how we are acknowledging our Partners at https://www.FreeBSDfoundation.org/donors/. You can also find out more about this new program at https://www.FreeBSDfoundation.org/FreeBSD-foundation-partnership-program= /. When I was in China last week, I had a chance to talk to a few companies about our new partnership program, and it definitely generated more interest in supporting our efforts. We are continuing to reach out to commercial users for help that will enable us to provide more outreach and support for FreeBSD. This includes funding more projects to improve FreeBSD, providing FreeBSD education and training, and recruiting more contributors to the Project. We can only provide the above support with your donations, and we need your help to connect us with your companies. Please consider notifying your organization about our new Partnership Program and helping to connect us with the appropriate contacts at your company. Your donations will help us: * Accelerate improvements and add new features to FreeBSD * Support release engineering efforts full-time * Create and provide FreeBSD educational and training material * Provide face-to-face opportunities for developers to work together * Improve and support FreeBSD infrastructure We need your support to continue improving FreeBSD. Q2 2017 Conference Recaps From sponsoring events to attending conferences, the Foundation continued its mission of advocacy in the second quarter of 2017. Over the past few weeks, members of the Foundation team represented the Project and the Foundation at events around the world. Below are just a few of the conference recaps. FOSSASIA 2017 (contributed by Philip Paeps) The Foundation kindly funded part of my travel from Tokyo to Singapore to attend FOSSASIA. I gave the "FreeBSD is not a Linux Distribution" presentation that Foundation board member George Neville-Neil wrote for Open Source China in December. My presentation was well-attended, and I got a lot of good questions from the primarily Linux-oriented audience. Read more at https://www.FreeBSDfoundation.org/blog/fossasia-2017-trip-report-philip-= paeps/. OSCON 2017 (contributed by Ed Maste) I represented the FreeBSD Foundation at OSCON 2017, which took place May 8-11, 2017, in Austin, TX: https://conferences.oreilly.com/oscon/oscon-tx . The Foundation booth was also staffed by FreeBSD committer Brad Davis and Doug Mcintire from Netgate. We met up Wednesday morning to set up the table. We were part of a "nonprofit pavilion" which consisted of eight or so tables, located between Open Camps and Operation Code. To help attract booth traffic, I brought a Raspberry Pi 3, with a small LCD display attached. As a demo, the Raspberry Pi showed a video of a Gource rendering of changes to the FreeBSD source tree over time (see example at https://www.youtube.com/watch?v=3DvZ8Sspua0Ks). Read more at https://www.FreeBSDfoundation.org/blog/conference-recap-oscon-2017/. Rootconf 2017 (contributed by Philip Paeps) In mid-May I presented at Rootconf 2017 in Bangalore. Rootconf is India's principal conference where systems and operations engineers share real-world knowledge about building reliable systems: https://rootconf.in/2017/. As always, it was interesting to hear the difficulties people face trying to run reliable systems on less reliable platforms. While many of the presentations were very Linux-specific and not very exciting to me, a couple of talks did catch my eye. I particularly enjoyed the talk by Aruna Sankaranarayanan (https://www.youtube.com/watch?v=3DXQJ7YhVoSWI&feature=3Dyoutu.be) explaining how Mapbox takes advantage of Amazon's "spot pricing" mechanism by spawning and shutting down machines at different price points to optimize for cost without compromising availability. Their spotswap https://github.com/mapbox/spotswap/ software has been released under a BSD license. It sounds as though it should be possible to port this to FreeBSD with minimal effort. Read more at https://www.FreeBSDfoundation.org/blog/rootconf-2017-trip-report-philip-= paeps/. BSDCan 2017/FreeBSD Developers Summit (contributed by Deb Goodkin) One of our initiatives is to assist in providing face-to-face knowledge sharing and development opportunities around the world. One way we do this is by sponsoring BSD-related conferences and FreeBSD Developer and Vendor Summits. We recently sponsored both BSDCan 2017 and the FreeBSD Developer and Vendor Summit in Ottawa, Ontario, Canada, which took place June 7-10, 2017. Many of our board and staff members attended the summit and conference to run tutorials, give presentations, lead sessions, work with developers, give demos, and share knowledge. In addition, this year we were pleased to bring our new University of Waterloo interns to the conference where they had the opportunity to demonstrate some of their projects at the Foundation table. Read more at https://www.FreeBSDfoundation.org/blog/conference-recap-bsdcan-2017Fr= eeBSD-developers-summit/. Open Travel Grant Applications The Foundation recognizes the importance of bringing members of the FreeBSD community face-to-face to both further development of the Project and spread the word about FreeBSD. Travel grants are available to community members who need assistance with travel expenses for attending conferences related to FreeBSD development and advocacy. Please note: the travel grant policy has been recently updated. Please carefully review it before submitting your application. More information about travel grants is available at: https://www.FreeBSDfoundation.org/what-we-do/grants/travel-grants/. FreeBSD Day was June 19! (contributed by Anne Dickison) June 19th was declared FreeBSD Day! Thank you to everyone who joined us in honoring the FreeBSD Project's pioneering legacy and continuing impact on technology. Find out more about FreeBSD Day and how we celebrated here at https://www.FreeBSDfoundation.org/blog/happy-FreeBSD-day/. Upcoming Events Find out about upcoming Foundation events at https://www.FreeBSDfoundation.org/news-and-events/upcoming-events/. FreeBSD Journal The May/June 2017 Issue of the FreeBSD Journal is now available. Don't miss articles on FreeBSD's Firewall Feast, CADETS: Blending Tracing and Security on FreeBSD, Toward Oblivious Sandboxing with Capsicum, and more. (https://www.FreeBSDfoundation.org/past-issues/security/) Did you miss the March/April issue? Check out articles on CFEngine, Puppet on FreeBSD, Vagrant, and more! (https://www.FreeBSDfoundation.org/past-issues/configuration-management/) As a recent addition of functionality, browser-based subscribers now have the ability to download and share PDFs of the articles! Sample Issue! If you've ever wanted to read through an entire issue of the FreeBSD Journal, now's your chance. Download the sample issue from https://mydigitalpublication.com/publication/?i=3D296880#{"issue_id":296= 880,"numpages":1,"page":1} and be sure to share with your friends and colleagues. Not a subscriber? Sign up today at https://www.FreeBSDfoundation.org/journal/. More information about the Foundation's doings and goings-on can be found in our own quarterly newsletter, linked above. __________________________________________________________________ The Postmaster Team Links The Postmaster Team URL: https://www.FreeBSD.org/administration.html#t-postmaster Contact: David Wolfskill Contact: Larry Rosenman Contact: Ryan Steinmetz Contact: Eygene Ryabinkin Contact: Remko Lodder Contact: Kurt Jaeger Postmaster handles the mail flow for the FreeBSD project. Clusteradm provides us with four jails: mailman, mailarchive, mx1, and mx2. In addition, there is some part of the setup running on freefall.FreeBSD.org. The system uses postfix, mailman, spamassassin, and some other tools from the ports tree to handle the mail flow. We use a very small, non-public Subversion repository for parts of the configuration. During Q2, Larry Rosenman, Kurt Jaeger, Eygene Ryabinkin, Remko Lodder and Ryan Steinmetz joined the Postmaster Team, and Florian Smeets left the Postmaster Team. Thanks to Florian for his long service in that role! David Wolfskill is planning to leave the role as soon as the new team members are settled. Vsevolod Stakhov plans to provide us with support to integrate rspamd into the setup, as well. The workload for the Postmaster Team is not high, but the complexity of the setup has its own demands. Open tasks: 1. We need to improve our internal documentation of workflows and processes. 2. We should consider adding some monitoring to provide quarterly numbers on the mail flow. __________________________________________________________________ Projects 64-bit Inode Numbers Links Phabricator Review URL: https://reviews.FreeBSD.org/D10439 Contact: Gleb Kurtsou Contact: Konstantin Belousov Contact: Kirk McKusick The 64-bit inode project was completed and merged into FreeBSD 12 on May 23, 2017. It extends the ino_t, dev_t, and nlink_t types to be 64-bit integers. It modifies the struct dirent layout to add a d_off field, increases the size of d_fileno to 64 bits, increases the size of d_namlen to 16 bits, and changes the required alignment of the structure. It increases the struct statfs f_mntfromname[] and f_mntonname[] array lengths from MNAMELEN to 1024. ABI breakage is mitigated by providing compatibility using versioned symbols, ingenious use of the existing padding in structures, and employing various other tricks. Unfortunately, not everything can be fixed, especially outside the base system. For instance, third-party APIs which pass struct stat as parameters are broken in backward- and forward-incompatible ways. The ABI for kinfo-consuming sysctl MIBs is changed in a backward-compatible way, but there is no general mechanism to handle other sysctl MIBS which return structures where the layout has changed. In our consideration, this breakage is either in management interfaces, where we usually allow ABI slippage, or is not important. The layout of struct xvnode changed, and no compatibility shims are provided. For struct xtty, the dev_t tty device member was reduced to be just uint32_t. It was decided that maintaining ABI compatability in this case is more useful than reporting a 64-bit dev_t value, for the sake of pstat. Updating note: strictly follow the instructions in UPDATING. Build and install the new kernel with the COMPAT_FREEBSD11 option enabled, then reboot, and only then install the new world. Credits: The 64-bit inode project, also known as ino64, started life many years ago as a project by Gleb Kurtsou (gleb). Kirk McKusick (mckusick) then picked up and updated the patch, and acted as a flag-waver. Feedback, suggestions, and discussions were carried out by Ed Maste (emaste), John Baldwin (jhb), Jilles Tjoelker (jilles), and Rick Macklem (rmacklem). Kris Moore (kris) performed an initial ports investigation followed by an exp-run by Antoine Brodin (antoine). Essential and all-embracing testing was done by Peter Holm (pho). The heavy lifting of coordinating all these efforts and bringing the project to completion were done by Konstantin Belousov (kib). This project was sponsored by The FreeBSD Foundation (emaste, kib). __________________________________________________________________ Capability-Based Network Communication for Capsicum/CloudABI Links ARPC: GRPC-Like RPC Library That Supports File Descriptor Passing URL: https://github.com/NuxiNL/arpc Flower: A Label-Based Network Backplane URL: https://github.com/NuxiNL/flower Contact: Ed Schouten One of the weaknesses of Capsicum and CloudABI is that it is not easy to develop applications that need to make outgoing network connections, since system calls like connect() and sendto() are disabled. Though we can sometimes work around this by ensuring that the sandboxed process already possesses socket file descriptors on startup, this does not allow the destination process to be restarted, moved to a different network address, be load balanced, etc.. Coming up with a solution for this is quite important for me, as I am currently working on making CloudABI work on top of Kubernetes, Google's open source cluster management suite. The idea is that Kubernetes will schedule CloudABI processes instead of Docker containers. All of these CloudABI processes will have their dependencies on other services in the cluster injected explicitly, making internal communication very secure. All of this is intended to work on FreeBSD as well, of course! To solve this problem, I've been working on a daemon called Flower (read: flow-er) that allows software to register services and connect to them. Servers are identified by a set of labels with values (e.g., {datacenter: 'frankfurt', service: 'mysql'}). Clients can connect these servers by providing the corresponding label(s). Flower's security model is capability-based, just like Capsicum. The ability to bind and connect can be limited by permanently constraining labels to certain values. Flower has been designed not to act as a proxy. It does not copy any data. It merely forwards existing socket file descriptors or creates UNIX socket pairs and hands these out to its clients and servers. To realize this, processes communicate with Flower using an RPC library called ARPC. ARPC is a very simple clone of Google's GRPC, with the special feature that messages (Protobufs) can have file descriptors attached. This project was sponsored by Nuxi, the Netherlands. Open tasks: 1. Finish implementing the Flower code. 2. Integrate Flower with the Kubernetes/CloudABI runtime. 3. Release the Kubernetes/CloudABI runtime as open source software. __________________________________________________________________ Ceph on FreeBSD Links Ceph Main Site URL: http://ceph.com Main Repository URL: https://github.com/ceph/ceph My FreeBSD Fork URL: https://github.com/wjwithagen/ceph Contact: Willem Jan Withagen Ceph is a distributed object store and file system designed to provide excellent performance, reliability and scalability. * Object Storage Ceph provides seamless access to objects using native language bindings or radosgw, a REST interface that is compatible with applications written for S3 and Swift. * Block Storage Ceph's RADOS Block Device (RBD) provides access to block device images that are striped and replicated across the entire storage cluster. * File System Ceph provides a POSIX-compliant network file system that aims for high performance, large data storage, and maximum compatibility with legacy applications. I started looking into Ceph because the HAST solution with CARP and ggate did not really do what I was looking for. I aim to run a Ceph storage cluster of storage nodes that are running ZFS, with user workstations running bhyve on RBD disks that are stored in Ceph. Compiling for FreeBSD will now build most of the tools available in Ceph. The most important changes since the last report are: * Ceph has released release candidate v12.1.0 (aka Luminous); the corresponding packaging is sitting in my tree waiting for Luminous to be actually released. * ceph-fuse works, and allows mounting of cephfs filesystems. The speed is not impressive, but it does work. * rbd-ggate is available to create a Ceph rbd backed device. rbd-ggate was submitted by Mykola Golub. It works in a rather simple fashion: once a cluster is functioning, rbd import and rbd-ggate map are used to create ggate-like devices backed by the Ceph cluster. Other improvements since the previous report: * Some bugs in the init-ceph code (needed for rc.d) are being fixed. * RBD and rados are functioning. * The needed compatability code was written so that FreeBSD and Linux daemons can operate together in a single cluster. * More of the awkward dependancies on Linux-isms are deleted -- only /bin/bash is there to stay. The next forthcoming official release of Ceph is called Luminous (v12.1.0). As soon as it is available from upstream, a port will be provided for FreeBSD. To get things running on a FreeBSD system, run pkg install net/ceph-devel or clone https://github.com/wjwithagen/ceph, check out the wip.freebsd.201707 branch, and build manually by running ./do_freebsd.sh in the checkout root. Parts not (yet) included: * KRBD -- but rbd-ggate is usable in its stead. * BlueStore -- FreeBSD and Linux have different AIO APIs, and that incompatibility needs to be resolved somehow. Additionally, there is discussion in FreeBSD about aio_cancel not working for all device types. Open tasks: 1. Run integration tests to see if the FreeBSD daemons will work with a Linux Ceph platform. 2. Investigate the keystore, which can be embedded in the kernel on Linux and currently prevents building Cephfs and some other parts. The first question is whether it is really required, or if only KRBD requires it. 3. Scheduler information is not used at the moment, because the schedulers work rather differently between Linux and FreeBSD. But at a certain point in time, this will need some attention (in src/common/Thread.cc). 4. Improve the FreeBSD init scripts in the Ceph stack, both for testing purposes and for running Ceph on production machines. Work on ceph-disk and ceph-deploy to make it more FreeBSD- and ZFS-compatible. 5. Build a test cluster and start running some of the teuthology integration tests on it. Teuthology wants to build its own libvirt, and that does not quite work with all the packages FreeBSD already has in place. There are many details to work out here. 6. Design a virtual disk implementation that can be used with bhyve and attached to an RBD image. __________________________________________________________________ DTS Updates Contact: Emmanuel Vadot DTS (Device Tree Source) files provide a human-readable source description of the hardware resources for a given computer system (such as ARM- or MIPS-based embedded boards). The DTS source representation must be compiled into a binary format in order to be linked into the kernel and used to locate devices at runtime. The DTS files in FreeBSD were updated to match the versions from Linux 4.11, to represent more modern devices and provide more accurate representations. __________________________________________________________________ Kernel Coda revival Links GitHub Repository URL: https://github.com/trasz/FreeBSD/tree/coda Contact: Edward Tomasz Napiera=C5=82a Coda is a distributed file system developed as a research project at Carnegie Mellon University, descended from a older version of the Andrew File System. It got dropped from FreeBSD some five years ago, due to not having been adopted for a MPSAFE world. The focus for this current project is to bring it back into sufficiently workable shape that it could return to the kernel. It is currently in a working condition. Work is underway to test it better, fix whatever issues are found, and commit it to 12-CURRENT. This project was sponsored by Chalmers University of Technology. Open tasks: 1. Additional testing. 2. Update the userspace components (net/coda_client and net/coda_server). __________________________________________________________________ FreeBSD Driver for the Annapurna Labs ENA Links Enhanced Networking Guide URL: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/enhanced-networ= king.html Contact: Marcin Wojtas Contact: Michal/ Krawczyk The ENA (Elastic Network Adapter) is a 25G SmartNIC developed by Annapurna Labs and is based on a custom ARMv8 chip. This is a high-performance networking card available in the AWS offerings. It introduces enhancements in network utilization scalability on EC2 machines under the control of various operating systems, in particular FreeBSD. The goal of FreeBSD enablement is to provide top performance and a wide range of monitoring and management features such as: * multiple queue modes * hardware offloads (rx and tx checksum) * an admin queue * asynchronous notifications * robust hardware access * a scalable number of MSI-X vectors * hardware counters * watchdog mechanism * LRO * RSS The driver is available in the kernel source tree as of r318647. This project was sponsored by Annapurna Labs -- an Amazon company. Open tasks: 1. Add RSS configuration from userspace (via sysctls). 2. Add support for LLQ mechanisms. __________________________________________________________________ Intel 10G Driver Update Links Commit Adding X553 ix/ixv Support URL: https://reviews.FreeBSD.org/D11232 Contact: Chris Galazka Contact: Jeb Cramer The ix and ixv network interface drivers support a variety of Intel network interfaces, with line speeds at 10 Gbit/second. This quarter, the drivers gained support for the X553 network interface, which is found on System-on-a-Chip devices based on the Denverton platform. This update should allow FreeBSD to be more useful on a new class of hardware platform. Work is also underway to convert these drivers to use the iflib network driver library, which should ease future maintenance of the drivers, as well as the network subsystem as a whole. __________________________________________________________________ pNFS Server Plan B Links Testing Instructions URL: http://people.FreeBSD.org/~rmacklem/pnfs-planb-setup.txt Contact: Rick Macklem Parallel NFS (pNFS) is an extension to the NFSv4 protocol that allows for file accesses within a single logical mount to be performed against multiple file servers, with the potential for data access to occur in parallel. The pNFS "layout" in use specifies how the division occurs, with metadata operations occurring against the main server, and bulk data operations (read/write/setattr/etc.) occurring via a layout-specific scheme between the client and the data servers. My first attempt at a pNFS server using GlusterFS was a dud. It worked, but performance was so poor that it was not usable. This attempt that I call "Plan B", only uses FreeBSD, with one FreeBSD server handling the metadata operations and multiple FreeBSD servers configured to serve data, is now ready for third-party testing. If testing by third parties goes well, I anticipate the code will be merged into FreeBSD head in time for FreeBSD 12. Fairly recent FreeBSD or Linux systems should be usable as pNFS clients for testing. This server supports the File Layout, which is supported by both of these clients. There is no support for the Flex Files Layout or mirroring at this time. I hope to use the Flex Files Layout to add mirroring support over the next year or so. Striping is not supported, and I have no plans for implementing this at the moment. The patched FreeBSD sources may now be accessed for testing via either Subversion or download of a gzipped tarball. They consist of a patched kernel and nfsd and can be used on any FreeBSD 11 or later system. Open tasks: 1. Testing by others will be needed, now that the code is available. __________________________________________________________________ Architectures FreeBSD on Marvell Armada38x Contact: Marcin Wojtas Contact: Zbigniew Bodek Work proceeds to finalize the process of bringing support for the Marvell Armada38x platform into FreeBSD head. The most important parts of the recent effort are: * Add the network driver (NETA) * Enable coherent busdma operation for all ARMv7 SoCs * Add various low-level optimizations, such as L1 cache prefetch and MBUS quirks * Enable PL310 L2 cache controller * Add SDHCI support * Fixes for the e6000sw driver and a rework of its PHY handling * Support multi-port PCIe operation * Various fixes and enhancements of the common Marvell code * Fix and enable support for performance counters (HWPMC) This project was sponsored by Stormshield, Semihalf, and Netgate. __________________________________________________________________ FreeBSD/arm64 Links FreeBSD arm64 Wiki Page URL: https://wiki.FreeBSD.org/arm64 Contact: Andrew Turner Support for the Privilege Access Never (PAN) feature was added. This stops the kernel from accessing userspace memory, except through specific instructions. This helps security by only allowing access to userspace via the correct accessor functions. This is enabled on all supported CPUs that implement ARMv8.1 or later. The pmap code now supports the Unprivileged execute-never (UXN) and Privileged execute-never (PXN) bits in the page tables. These bits stop userspace and the kernel, respectively, from executing instructions on any marked page. The performance of the pmap layer has been improved. Many of the cache handling function calls have been removed. Some were needed early on to work around other bugs that have now been fixed. The removal of these calls has led to a large performance improvement. The kernel now uses crc32c instructions where appropriate. These are an optional set of instructions to perform crc32c checksumming quickly without using a lookup table.c The VM_MEMATTR_WRITE_THROUGH memory attribute is now supported. This is used to allocate memory for the framebuffer. Previously, the kernel would use cached memory; however, this leads to visual artifacts. The write-through flag fixes these by writing data out to RAM. The default linker on arm64 is now lld. This means that FreeBSD is able to build itself with just the components in the base system, a big milestone! __________________________________________________________________ Userland Programs DTC Contact: Emmanuel Vadot The in-tree DTC (Device Tree Compiler) was switched to use the BSD-licensed version by default. (The previous default DTC is licensed under the GPL.) The current version supports overlays and is able to compile every DTS (Device Tree Source) used by the FreeBSD arm releases. The ports GPL version was updated to the latest release (1.4.4). The in-tree GPL version is still present but the goal is to remove it before FreeBSD 12.0. __________________________________________________________________ Using LLVM's LLD Linker as FreeBSD's System Linker Links FreeBSD lld Wiki Page URL: https://wiki.FreeBSD.org/LLD FreeBSD/LLD Tracking PR (LLVM Bugzilla) URL: http://llvm.org/pr23214 Exp-Run Request Using lld as /usr/bin/ld URL: https://bugs.FreeBSD.org/214864 Contact: Rafael Esp=C3=ADndola Contact: Ed Maste LLD is the linker in the LLVM family of projects. It is a high-performance linker that supports the ELF, COFF and Mach-O object formats. It is broadly compatible with the common linkers used for each file format. For ELF this is the GNU Binary File Descriptor (BFD) ld and GNU gold. However, LLD's authors are not constrained by strict compatibility where it would hamper performance or desired functionality. LLD is now used as the default system linker for FreeBSD/arm64 and can link a working kernel, kernel modules, and userland for FreeBSD/amd64. LLD can also link a working kernel and modules (but not userland) for FreeBSD/arm and FreeBSD/i386. Work is ongoing to address ports that do not build with LLD as the system linker (either by fixing the port, or configuring the port to be linked by GNU ld). For FreeBSD 12.0 we expect to use LLD as the system linker for the same set of architectures that use Clang by default: 32- and 64-bit arm and x86. This project was sponsored by The FreeBSD Foundation. Open tasks: 1. Fix libtool to detect LLD and pass the same command line arguments as for GNU ld and gold. 2. Investigate the remaining amd64 and arm64 port build failures. 3. Investigate and improve LLD on i386 and arm, before the creation of the stable/12 branch. 4. Investigate and improve LLD on all other architectures. 5. Extensive testing. __________________________________________________________________ Ports A New USES Macro for Porting Cargo-Based Rust Applications Links Rust Homepage URL: https://www.rust-lang.org/ Cargo Homepage URL: https://crates.io/ Alacritty Homepage URL: https://github.com/jwilm/alacritty Exa Homepage URL: https://the.exa.website/ Ripgrep Homepage URL: https://github.com/BurntSushi/ripgrep Short Screencast About How to Use the USES=3Dcargo Macro URL: https://asciinema.org/a/SM2sOLi6iBUOmGWrxn5W1QI8U Contact: Tobias Kortkamp Support in the Ports Collection for applications written in the Rust programming language that use Rust's package manager Cargo was added, via a new USES=3Dcargo setting. The work is based on the cargo module from the OpenBSD ports tree. This should significantly ease the porting of Rust applications, as previously porters had to create their own tarball of the application's dependencies or find other manual ways of bringing them in. Several new ports were added that use it, for example: * Alacritty, a GPU-accelerated terminal emulator * Exa, a modern replacement for ls * Ripgrep, a line-oriented search tool that combines the usability of The Silver Searcher with the raw speed of GNU grep Open tasks: 1. Add documentation for the new feature. __________________________________________________________________ GCC (GNU Compiler Collection) Links GCC Homepage URL: https://gcc.gnu.org Issue Tracker Entry for the Update to GCC 6 URL: https://bugs.FreeBSD.org/bugzilla/show_bug.cgi?id=3D219275 GCC 5 Changelog URL: https://gcc.gnu.org/gcc-5/changes.html GCC 5 Porting Issues URL: https://gcc.gnu.org/gcc-5/porting_to.html Contact: Gerald Pfeifer Contact: Andreas Tobler The default version of GCC in the Ports Collection (the one requested by USE_GCC=3Dyes and various USES=3Dcompiler invocations) has been updat= ed from GCC 4.9.4 to GCC 5.4. This new major version brings many new capabilities and improvements, as well as some changes that may require adjustments. The latter category includes many new compiler warnings, significant improvements to inter-procedural optimizations, and link-time optimization. The default mode for C is now -std=3Dgnu11 instead of -std=3Dgnu89. The = C++ front end has full C++14 language support, including C++14 variable templates, C++14 aggregates with non-static data member initializers, C++14 extended constexpr, and more. The Standard C++ Library (libstdc++) has full C++11 support and experimental full C++14 support. It uses a new ABI by default. The lang/gcc port now is a meta-port that pulls in the respective lang/gccX port (based on the setting of $GCC_DEFAULT) and defines gcc, g++, and gfortran as symlinks to the respective versioned binaries. This is the end of a long journey establishing this infrastructure, which is now similar that used by the python ports, for example. Having the new infrastructure makes upgrading the default, as well as locally adjusting the default version, a lot easier. gcc8-devel has been added, and armv6hf support removed, and we made adjustments for newer versions of FreeBSD. Also of note are various cleanups and changes to improve the robustness of our packages and the addition of support for aarch64 to many ports. Thanks to dim@, jbeich@, tijl@, mat@, miwi@, linimon@ for assisting with this work. Open tasks: 1. The update of the default version of GCC from GCC 5.4 to GCC 6.4 is stalled, unfortunately. The work on the GCC and insfrastructure sides is complete, but unfortunately there are a number of broken ports that need to be adjusted/fixed. Any help is very appreciated; see PR 219275 for details. __________________________________________________________________ GNOME on FreeBSD Links FreeBSD GNOME Website URL: http://www.FreeBSD.org/gnome Development Repository URL: https://github.com/FreeBSD/FreeBSD-ports-gnome Upstream Build Bot URL: https://wiki.gnome.org/Projects/Jhbuild/FreeBSD USE_GNOME Porter's Handbook Chapter URL: https://www.FreeBSD.org/doc/en_US.ISO8859-1/books/porters-handbook= /using-gnome.html Contact: FreeBSD GNOME Team The FreeBSD GNOME Team maintains the GNOME, MATE, and CINNAMON desktop environments and graphical user interfaces for FreeBSD. GNOME 3 is part of the GNU Project. MATE is a fork of the GNOME 2 desktop. CINNAMON is a desktop environment using GNOME 3 technologies but with a GNOME 2 look and feel. After a period of not much activity, this quarter we started a little experiment in how we merge ports from the development repo to the FreeBSD Ports Collection. Instead of merging everything in one big commit, we have been updating the GNOME ports one at a time or in small groups. For example, the GTK+ stack and the Evolution Suite were updated as groups, and all the gnome-games components were done in one commit. It might be a bit more work preparing and testing the updates, but on the plus side, it easy to keep track of what is going on, and allows us to pay attention to the details. It should also make it easier to commit smaller changes. This quarter started with the update of GTK+ 3 to 3.22.15, and the underlying libraries to their latest stable versions. After the GTK+ update, work started on getting newer versions of other GNOME applications updated. The webkit2-gtk3 port was first updated to the 2.14 series and later to 2.16.3, which is the latest stable version. This step was needed because 2.16 couldn't be built on FreeBSD 10.3 without some required framework changes. harfbuzz-icu was split off from the main harfbuzz port. This drops the heavy icu dependency from the main harfbuzz port. A longstanding GLib/gio bug was fixed that had previously caused crashes of gnome-shell and other applications when share/applications was modified, as happens on pkg install or deinstall. Many of these updates are based on work previously done in the Gnome development branch by Ruslan Makhmatkhanov, Gustau Perez and Koop Mast. Open tasks: 1. Porting of Mutter/Gnome-shell/GDM 3.24 is complete. Unfortunately, GDM is blocking the update because of a "handoff" bug to the session after login. 2. Fix the printer submenu in gnome-control-center. As a workaround, system-config-printer can be used to configure printers. 3. MATE 1.18 is being QA tested and should arrive in early July. __________________________________________________________________ KDE on FreeBSD Links KDE on FreeBSD Website URL: https://FreeBSD.kde.org/ KDE Ports Staging Area URL: https://FreeBSD.kde.org/area51.php KDE on FreeBSD Wiki URL: https://wiki.FreeBSD.org/KDE KDE/FreeBSD Mailing List URL: https://mail.kde.org/mailman/listinfo/kde-FreeBSD Development Repository URL: https://github.com/FreeBSD/FreeBSD-ports-kde KDE's Continous Integration Dashboard URL: https://build.kde.org Blog Post on Using the Ninja CMake Generator URL: https://euroquis.nl/bobulate/?p=3D1600 Contact: KDE on FreeBSD Team The KDE on FreeBSD team focuses on packaging KDE and Qt, and making sure that their experience on FreeBSD is as good as possible. This quarter, in addition to the regular updates to the KDE, Qt, and related ports, there have also been some changes behind the scenes: our development repository has moved to GitHub, and FreeBSD is now part of KDE's official continuous integration (CI infrastructure). After the X.Org and GNOME ports teams, the KDE on FreeBSD team has moved its development repository to GitHub. This should make it easier for others to collaborate with us via pull requests, and by basing all our changes on top of the official ports tree we also hope this reduces the amount of conflicts and churn we need to deal with when landing big updates across the tree. We would like to thank iXsystems for hosting and supporting our area51 Subversion repository for many years. FreeBSD has finally joined KDE's CI (Continuous Integration) system as a tier-1 platform. KDE CI builds all the KDE sources -- 70 frameworks, the KDE Plasma Desktop and a plethora of KDE Applications -- continuously, straight from KDE's git repositories. There is strong commitment from upstream and the downstream KDE-FreeBSD team to reduce the amount of patching in the KDE ports to as little as possible. The first effects are being felt in expanding the set of unit tests to include FreeBSD-specific situations, and in extending Qt to handle FreeBSD filesystems better. In addition to the KDE sysadmins, we would also like to extend our thanks to Adriaan de Groot, who is both a KDE committer and part of our KDE on FreeBSD team, for spearheading these efforts. The following big updates landed in the ports tree this quarter: * CMake was updated to 3.8.0 and 3.8.2 * KDE Frameworks was updated to 5.33, 5.34 and 5.35 * The Calligra office suite was updated to 3.0.1, the first release in the ports tree to be based on KDE Frameworks 5, and the latest stable release upstream * The Konversation IRC client was updated to 1.7.2, the latest upstream release and the first ports version based on KDE Frameworks 5 * KchmViewer was updated to 7.7, which is based on KDE Frameworks 5 * LabPlot was updated to 2.3.0 and 2.4.0, and is now based on KDE Frameworks 5 * QtCreator was upated to 4.2.2 and subsequently to 4.3.0 * py-sip was updated to 4.19.2, PyQt4 to 4.12 and PyQt5 to 5.7.1 * Several fixes for ARMv6 landed in the Qt4 and Qt5 ports -- thanks to Mika=C3=ABl Urankar After several review rounds and exp-runs, Tobias Berner (tcberner@) finally made the Ninja generator the default for CMake-based ports, so that devel/ninja is used instead of (g)make in most cases. This should make most builds faster, even if only by a small margin. Adriaan de Groot also wrote a blog post about the change. __________________________________________________________________ New Port: FRRouting Links FRRouting Home Page URL: https://frrouting.org/ Contact: Olivier Cochard-Labb=C3=A9 FRRouting (FRR), a Quagga fork, is an IP routing protocol suite for Linux and Unix platforms which includes protocol daemons for BGP, IS-IS, OSPF and RIP (LPD and PIM support need to be fixed on FreeBSD). FRR is a Linux Foundation Collaborative Project with contributors including 6WIND, Architecture Technology Corporation, Big Switch Networks, Cumulus Networks, LabN Consulting, NetDEF (OpenSourceRouting), Orange, Volta Networks, and other companies. This project was sponsored by Orange. __________________________________________________________________ PHP Ports: Help Improving QA Links My Patreon Page URL: https://www.patreon.com/TorstenZuehlsdorff Contact: Torsten Z=C3=BChlsdorff As maintainer of the PHP ports, I first want to thank you all for the great feedback and patches I receive, in many forms. You keep my life interesting! In the past few months I learned a lot about various configurations, settings and bugs. Also, sadly, there are always PRs, patches and emails left unanswered, because of missing time on my side. I want to improve the situation by adding more automatic QA testing, but I need help to do so. Please send me your non-standard PHP-configurations or describe your exotic setups! These can be as simple as changed default versions, like LibreSSL instead of OpenSSL or the GCC version used for compiling. I, for example, always use another PostgreSQL-version than the default (and always PHP 7.1). Of course, this also covers port options set in an non-default way or setups that change variables to allow for multiple PHP installations, etc.. I plan to test on all supported FreeBSD versions, so you only need to mention if you are using an unsupported version. Note: Since PHP 7.2 is coming (hopefully on schedule), I will test PHP 7.2 from the onset with all the provided configurations, too. Open tasks: 1. Document the various configurations to be tested. 2. Setup the automatic QA infrastructure. __________________________________________________________________ Rust Links Wiki Portal URL: https://wiki.FreeBSD.org/Rust Guide to Bootstrap Rust on FreeBSD URL: https://gist.github.com/dumbbell/b587da50ef014078da9e732a4331ebad Bug Report to Track Progress on Bootstrapping URL: https://bugs.FreeBSD.org/bugzilla/show_bug.cgi?id=3D216143 Upstream Discussion of API/ABI-Breaking Changes URL: https://internals.rust-lang.org/t/pre-rfc-target-extension-dealing= -with-breaking-changes-at-os-level/5289 Contact: FreeBSD Rust team Rust was updated to 1.18.0 and Cargo to 0.19.0, the latest versions at the time of this writing. lang/rust was enabled on FreeBSD/aarch64 and work has continued on devel/cargo to achieve the same. We are also making slow progress to add support for even more platforms. Discussion has started upstream to support API/ABI-breaking changes between major releases of operating systems. For instance, this is required to be able to target both FreeBSD 11.x and 12.x, which have ABI changes involving important structures. Once support is added upstream, it will be possible to target a specific ABI and do cross-compilation. lang/rust-nightly was marked as broken for now. We need to revisit how the port is built so we can use the x.py script as recommended by upstream. Tobias Kortkamp (tobik@) created the USES=3Dcargo setting to make it easy to add Rust applications to the Ports Collection. This is further detailed in a separate entry in this quarterly status report. The compiler, rustc, is crashing sometimes when there is a compilation error. Therefore, there is a bit of work to do to improve its stability. There is some code duplication between the lang/rust* and devel/cargo Makefiles. These all deserve a bit of cleanup, and it might be useful to create a USES=3Drust Makefile helper. Open tasks: 1. Bootstrap Rust on more platforms. 2. Investigate compiler crashes. 3. Investigate how to speed up lang/rust* compilation times. __________________________________________________________________ sndio Support in the FreeBSD Ports Collection Links Sndio Homepage URL: http://www.sndio.org Sndio Paper URL: https://www.openbsd.org/papers/asiabsdcon2010_sndio.pdf Comprehensive and Biased Comparison of OpenBSD and FreeBSD (Section 17) URL: https://www.bsdfrog.org/pub/events/my_bsd_sucks_less_than_yours-As= iaBSDCon2017-paper.pdf Contact: Tobias Kortkamp sndio is a small audio and MIDI framework that is part of the OpenBSD project. It provides a lightweight audio and MIDI server, sndiod. It currently supports OpenBSD, FreeBSD, DragonFly BSD, and Linux. The porting effort to FreeBSD and OSS started last year and the sndio backend support in the FreeBSD Ports Collection can now be considered good enough for daily use. Sndio offers network transparency through sndiod, which provides an easy way to share your audio devices with other machines/VMs/jails on your network. However, applications and libraries need to support playing and recording through it. To that end, I submitted several patches to various ports over the course of the last year. Here's a short selection of ports that now support sndio in the FreeBSD Ports Collection: * Most games, via audio/openal-soft, devel/sdl12, and devel/sdl20. * GStreamer-based applications and WebKit-based browsers through two new GStreamer plugins (audio/gstreamer1-plugins-sndio and audio/gstreamer-plugins-sndio). * Firefox, Firefox ESR, Seamonkey, Chromium, and Iridium. The browsers currently lack or have a non-functional OSS backend. Sndio support provides a BSD-native alternative to the ALSA and PulseAudio backends. * Video players like VLC, Totem, mpv, mplayer, etc.. * Audio players like Clementine, cmus, mpd, mpg123, siren, xmp, etc.. * SoX. * Shairport Sync, through a newly implemented backend. * JACK. * PulseAudio, through audio/pulseaudio-module-sndio. Open tasks: 1. Commit a backport of Kodi's new sndio backend to the Ports Collection. 2. If you maintain or use an audio-related port, consider checking whether it includes an sndio backend, and adding an SNDIO option. Thanks to the OpenBSD developers, several open-source projects already include one, so adding it might be very easy to do. __________________________________________________________________ TensorFlow Links TensorFlow PR URL: https://bugs.FreeBSD.org/bugzilla/show_bug.cgi?id=3D219609 Phabricator Review URL: https://reviews.FreeBSD.org/D11194 Prebuilt Packages URL: https://github.com/amutu/tf-FreeBSD-pkg TensorFlow Upstream URL: https://www.tensorflow.org Contact: Jov As described on its website, "TensorFlow(TM) is an open source software library for numerical computation using data flow graphs. Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) communicated between them. The flexible architecture allows you to deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device with a single API. TensorFlow was originally developed by researchers and engineers working on the Google Brain Team within Google's Machine Intelligence research organization for the purposes of conducting machine learning and deep neural networks research, but the system is general enough to be applicable in a wide variety of other domains as well." TensorFlow now is the most popular platform/library for machine learning and AI. There are official binaries for Linux, Mac, Windows, and Android, but no official support for FreeBSD. For the last several months, I have done some work to make TensorFlow available on FreeBSD. Some notable items include: * bazel was patched to not depend on /proc at build time. bazel is a build tool made by Google. It uses /proc to get path-to-self when building C++ code, but mounting /proc is usually not allowed when building as an unprivileged user. * TensorFlow can now be built on FreeBSD 10.x by using clang38 as the default bazel cross-build tool. * Patch the bazel workspace files to allow TensorFlow to be built using offline third-party dependencies. This work is needed because the FreeBSD Ports framework does not allow network access except during the fetch stage. * Fix the build on FreeBSD i386. * Make TensorFlow build with either Python 2 or Python 3. * Update to the latest version, which is tensorflow-1.2.0. TensorFlow can now be run on FreeBSD in CPU-only mode. Some functional tests have been performed on some combinations of FreeBSD 10.3-RELEASE and 11.0-RELEASE, amd64 and i386, and Python 2.7 and Python 3.6. This port would not be possible without substantial assistance from bapt@, lwhsu@, mat@, and koobs@ -- thank you for your advice, review, and help! You are very nice and I learned a lot about FreeBSD and the Ports framework from you. Open tasks: 1. Review, test, comment, and most importantly, commit to the Ports Collection. 2. Fix OpenCL (GPU acceleration) support on FreeBSD. 3. Port tensorflow-serving, which is a flexible, high-performance serving system for machine learning models produced by TensorFlow. 4. Set up a CI for TensorFlow on FreeBSD and give early notice to upstream when they break TensorFlow on FreeBSD. __________________________________________________________________ Updating Port Metadata for non-x86 Architectures Links aarch64 Poudriere Machine URL: http://thunderx1.nyi.FreeBSD.org/jail.html?mastername=3D110arm64-d= efault armv6 Poudriere Machine URL: http://beefy8.nyi.FreeBSD.org/jail.html?mastername=3Dhead-armv6-de= fault Contact: Mark Linimon I have been analyzing the error logs from ports builds for all non-x86 architectures, including both the logs published on the package build cluster and also other builds of powerpc64 and sparc64. From this analysis, I have marked almost all the failing ports as either BROKEN or NOT_FOR/ONLY_FOR, as appropriate. The intent of this work is not to make life harder for anyone, but rather, in fact, the opposite. With these definitions in place, it is possible to scan the poudriere bulk build output (the "Ignored ports" portion, in particular) and see quickly what ports are failing to build and why. Previously, finding the exact reason why a build failed needed some research (portsmon only analyzes failure messages on amd64). Additionally, it is extremely difficult to work through several hundred logs that simply say "failed to compile", "failed to link", and so forth. This is part of an effort to identify where we need further work to bring sufficient Ports Collection support to, e.g., armv6 and aarch64 to bring them closer to true Tier-1 status. To further facilitate locating patterns in the Poudriere output, I have begun reworking some existing BROKEN/NOT_FOR/ONLY_FOR messages so that they will sort more easily. This includes sorting the order in which architectures appear in the lists. Many people have been doing great work on fixing the individual ports. I hope that my work makes their jobs somewhat easier. __________________________________________________________________ Xfce on FreeBSD Links FreeBSD Xfce Project URL: https://wiki.FreeBSD.org/Xfce Ports Development Repository URL: https://www.assembla.com/spaces/xfce4/subversion/source Contact: FreeBSD Xfce Team Contact: Olivier Duchateau Xfce is a free software desktop environment for Unix and Unix-like platforms such as FreeBSD. It aims to be fast and lightweight, while still being visually appealing and easy to use. During this quarter, we have kept these applications up-to-date: * audio/xfce4-pulseaudio-plugin (0.2.5, PR219357) * deskutils/xfce4-tumbler (0.1.32, PR219848) * deskutils/xfce4-xkb-plugin (0.8.0, PR220071) * sysutils/garcon (0.6.1, PR219928, and PR219334 for Mk/Uses/xfce.mk) * textproc/xfce4-dict-plugin (0.8.0, PR220266) * x11/xfce4-terminal (0.8.5.1, PR219312) * x11/xfce4-whiskermenu-plugin (1.7.2, PR219347) * x11-wm/xfce4-desktop (4.12.4, PR220290) We have created a new Subversion tag (4.13) in order to follow the unstable releases. The separate tag was necessary in order to support changes in the USES=3Dxfce infrastucture, and due to some incompatible changes to the xfconf API. Ports following the unstable release are: * deskutils/xfce4-tumbler (0.1.92.1) * multimedia/xfce4-parole (0.9.2) * sysutils/xfce4-settings (4.13.1) * x11/libexo (0.11.3) * x11/libxfce4menu (4.13.2) * x11/libxfce4util (4.13.1) * x11/xfce4-conf (4.13.2) * x11/xfce4-dashboard (0.7.2) * x11/xfce4-screenshooter (1.9.1) * x11/xfce4-whiskermenu-plugin (2.1.2) * x11-wm/xfce4-desktop (4.13.1) * x11-wm/xfce4-panel (4.13.0) * x11-wm/xfce4-session (4.13.0) * x11-wm/xfce4-wm (4.13.0) Open tasks: 1. Make the transition to Gtk3 smoother for end users. __________________________________________________________________ Documentation Absolute FreeBSD, 3rd Edition Links Status as of 30 June URL: https://blather.michaelwlucas.com/archives/2972 Second Edition URL: https://www.michaelwlucas.com/os/af2e Trivial Updates URL: https://twitter.com/search?q=3D%23af3e&src=3Dtypd Contact: Michael Lucas I'm working on a third edition of Absolute FreeBSD. This will be a nearly complete rewrite, thanks to the addition of little details like ZFS, GPT, dma, GELI, new boot procedures, disk labeling, pkg(8), blacklistd, jails, etc.. My current (delusional) plan is to have a first draft finished by the end of October 2017, so we can have print copies for BSDCan 2018. Open tasks: 1. Write the remaining 75% of the book. __________________________________________________________________ Doc Version Strings Improved by Their Absence Links FreeBSD Documentation Project Primer URL: https://www.freebsd.org/doc/en_US.ISO8859-1/books/fdp-primer/ Get Version Information from Subversion Metadata URL: https://svnweb.freebsd.org/doc/head/share/mk/doc.docbook.mk?r1=3D5= 0233&r2=3D50232&pathrev=3D50233 Contact: Warren Block In retrospect, our $FreeBSD$ strings in source files are kind of weird, like a vestigial tail. The version control system stores all of that information in metadata. Yet here we are, not only allowing the version control system to alter our source files on every commit, but forcing it to do so. The reason for doing so is that the previous version control system did it. Really. Version control strings are a headache for translators using the new PO toolchain. It is an ever-changing string that offers nothing to the translation, yet can cause conflicts with earlier versions of itself. We also had complaints about how the Handbook was always months out of date. It was not, of course... but looking at just the version string in the main, rarely-changing book.xml file gave that impression. We fixed that problem last year, so the build system checks all the source files for the latest commit, but it seems easier to not have to fix the problem at all. Of course, that was really only one aspect of an ongoing problem. Our documentation build system was checking the version string in the source file, not the metadata. In 1973, metadata, like cars not composed chiefly of rust, had not yet been invented. I modified the build system to extract the information from the metadata (and noted, with some surprise, that this is a task at which Git is much better than Subversion). The next step was to remove the $FreeBSD$ strings from the source files and remove the FreeBSD=3D%H property that forces Subversion, against its better judgement, to substitute text in the actual contents of the file. The version information is not lost. It lives in the metadata, so retrieving it is as simple as svn info -- it does not need to be in the source at all. However, as with anything that touches code or processes which have not been touched in living memory, there was some debate over this. At that point, I offered to remove the version strings from the FreeBSD Documentation Project Primer book as a test. The change allowed the zh_TW translation team to turn off the FreeBSD=3D%H property on their translation and continue their work without fighting with the version strings. Rendered versions of the book still display the name of the last committer and the date and revision number of the last commit, but all of that information comes from metadata. As such, it is also more likely to be correct. Since the change, there have not been any complaints, at least not to me. In fairness, the removal of version strings from the FDP Primer alone is a small change in a tiny corner of the project. Looking at it another way, it might be that some things that seem to be necessary are more about the comfort of familiarity than actual utility. At present, this is strictly a change to the documentation build toolchain and a single documentation book. However, there do not appear to be any reason why it could not be extended to the rest of the documents. It might even serve as tiny test of whether the expansion of $FreeBSD$ tags is needed throughout the rest of the FreeBSD tree. __________________________________________________________________ New Xen Handbook Section Links Handbook Section About FreeBSD as a Xen Host URL: https://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/virtual= ization-host-xen.html Original Phabricator Review URL: https://reviews.freebsd.org/D10774 Contact: Benedict Reuschling FreeBSD supports the Xen hypervisor, with DomU (guest) support since FreeBSD 8.0 and Dom0 (host) available since FreeBSD 11.0. The FreeBSD Handbook was lacking instructions on how to run a Xen host and VMs. The steps were outlined in the FreeBSD wiki, but needed some extra bits of text from the upstream Xen wiki in order to form a complete guide. The new handbook section briefly explains what Xen is, how it differs from other hypervisors, and what features are currently available in FreeBSD. It then goes on to describe how to set up the Dom0, as well as detailing the guest VM support known as DomU. Reviewers Nikolai Lifanov, Roger Pau Monn=C3=A9, and Warren Block provid= ed valuable feedback on the initial version in Phabricator. Additional corrections were made by Bj=C3=B6rn Heidotting while translating the sec= tion into German. Open tasks: 1. More options for the Dom0 and DomU could be provided. 2. People should test these instructions on their hardware and provide feedback. This would also help us get better testing of the Xen port for FreeBSD. __________________________________________________________________ Miscellaneous BSD Meetups at Rennes (France) Links First Event URL: https://www.meetup.com/fr-FR/Meetup-BSD-Rennes/events/239248155/ Second Event URL: https://www.meetup.com/fr-FR/Meetup-BSD-Rennes/events/240202297/ Contact: Mathieu Kerjouan Two meetups dedicated to BSD systems were held in Rennes, France. The first one was hosted in the OVH office in Rennes and included presentations on multiple subjects: the non-technical history of FreeNAS (presented by olivier@), how OVH is using ZFS, an introduction to jails, and a use case for BGP/bird on FreeBSD. The second meetup, also hosted in the OVH office, presented these subjects: how to create a FreeBSD port (presented by jadawin@), how OVH is using Finite State Machines for managing their storage system, network high-availability with FreeBSD, and a jail tutorial by means of a demonstration running 200 OSPF (using net/bird) routers using jails and vnets on a small PC Engines APU2 system with only 4 CPU cores (1Ghz AMD) and 4GB RAM). This project was sponsored by OVH. __________________________________________________________________ Third-Party Projects Many projects build upon FreeBSD or incorporate components of FreeBSD into their project. As these projects may be of interest to the broader FreeBSD community, we sometimes include brief updates submitted by these projects in our quarterly report. The FreeBSD project makes no representation as to the accuracy or veracity of any claims in these submissions. HardenedBSD Links HardenedBSD Homepage URL: https://hardenedbsd.org/ SafeStack URL: http://clang.llvm.org/docs/SafeStack.html HardenedBSD Tor Hidden Service URL: http://t3a73imee26zfb3d.onion/ Projects HardenedBSD Would Like Help With URL: https://github.com/HardenedBSD/hardenedBSD/issues?q=3Dis%3Aissue+i= s%3Aopen+label%3A%22help+wanted%22 Contact: Shawn Webb Contact: Oliver Pinter HardenedBSD is a derivative of FreeBSD that gives special attention to security-related enhancements and exploit-mitigation technologies. From an initial focus on Address Space Layout Randomization (ASLR), it has now branched out to explore additional exploit mitigation techniques. It has been a long while since HardenedBSD's last entry in a quarterly status report, back in 2015Q4. The intervening year saw HardenedBSD gain new developers Bernard Spil and Franco Fichtner, import LibreSSL and OpenNTPd into base as the default crypto library and NTP client, respectively, and introduce the hbsd-update binary update mechanism for the base system. The secadm application got a rewrite and Trusted Path Execution (TPE). PIE is now enabled for the base system for arm64 and amd64 as well as the bulk of the ports tree, and the ports tree also gained RELRO and BIND_NOW. Integriforce (similar to NetBSD's verified exec, veriexec) was introduced for the base system, as well as SafeStack, a technology for protection against stack-based buffer overflows that's developed by the Clang/LLVM community. SafeStack relies and builds on top of Address Space Layout Randomization (ASLR), and is strengthened by the presence of PaX NOEXEC. Certain high-profile ports also have SafeStack enabled. Extremely generous hardware donations from G2, Inc. have provided for dedicated package building and binary update servers, as well as development and test servers. In March of 2017, we added Control Flow Integrity (CFI) to the base system. CFI is an exploit mitigation technique that helps prevent attackers from modifying the behavior of a program and jumping to undefined or arbitrary memory locations. This type of technique is gaining adoption across the industry -- Microsoft has implemented a variant of CFI, which they term Control Flow Guard, or CFG, and the PaX team has spent the last few years perfecting their Reuse Attack Protector, RAP. Of these, RAP is the most complete and effective implementation, followed by Clang's CFI. RAP would be a great addition to HardenedBSD; however, it requires a GPLv3 toolchain and is patent-pending. CFI can be implemented either on a per-DSO basis, or across all DSOs in a process. Currently only the former is implemented, but we are working hard to enable cross-DSO CFI. As is the case for SafeStack, cross-DSO CFI requires both ASLR and PaX NOEXEC in order to be effective. If an attacker knows the memory layout of an application, the attacker might be able to craft a data-only attack, modifying the CFI control data. The behavior of several system control (sysctl) nodes has been tighened up, limiting write access and introducing additional safety checks for write accesses. Kernel module APIs received a similar treatment. HardenedBSD's PaX SEGVGUARD implementation received a few updates to make it more stable and performant. As of March 2017, HardenedBSD is now accessible through a Tor hidden service. The main website, binary updates, and package distribution are all available over the hidden service. We now maintain our own version of the drm-next branch for updated graphics support. Binary updates are also provided for this branch. HardenedBSD would like to thank all those who have generously donated time, money, or other resources to the project. This project was sponsored by SoldierX, and G2, Inc. Open tasks: 1. Port SafeStack to arm64. 2. Integrate Cross-DSO CFI. 3. Add documentation to the HardenedBSD Handbook. 4. Start porting grsecurity's RBAC. __________________________________________________________________ --HeFlAV5LIbMFYYuh Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQG3BAABCgAdFiEE2WGV4E2ARf9BYP0XKNmm82TrdRIFAlnMLcwACgkQKNmm82Tr dRKXUAwgzueor3CCu0VPI+jG5v5NVEewwGB3Q3GMKaykJbRbVQusrHzR1mc3CTLq fIE4UWL/bvnV7IPSbfmX4qhVHU5f/zl100wbke88RT1Og4yzWjgQV/kYVmhrnXPV wFxYuuSoW6XckjHvVEXgSyGKgBSwjy/8/1KCM2tUTaxP1zR+11dFure5BsXKLnvo IVONjg4zQS2qdoB4f+/D6rG4pP+tK/VoMQx8NNPQXuO8L+PzEbs23F8l6WG67I6o eLdVX5q5vgEVZIaCRn0tlipNHTvyJuFaw5dlSU970LOwoZ3ZaUHzHtVehbvB5Jck lgG6gZxbI1ROwy9niyhsWixJ7oAyWt0koo4LxxyT7K3nyzv/r5stHiKeijr5JsRR dDFP95br/4rJ+d41TuU+FAiCeKRI9rywusT4qGJdT5nG8Y2ZbFxHLaBj8+SHx2+j C1zMdnCwv7pPGzmYyeIw1m9CoTf8WdDbdWx/07xXFJ6Pp7Tkn1B+kU4PHNmiUYv3 UzLmeRO29rfdZA== =n+Q+ -----END PGP SIGNATURE----- --HeFlAV5LIbMFYYuh-- From owner-freebsd-stable@freebsd.org Wed Sep 27 23:20:07 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7462BE15275 for ; Wed, 27 Sep 2017 23:20:07 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (turbocat.net [IPv6:2a01:4f8:c17:6c4b::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 08BB4652B8; Wed, 27 Sep 2017 23:20:07 +0000 (UTC) (envelope-from hps@selasky.org) Received: from hps2016.home.selasky.org (unknown [62.141.128.70]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 4DF742601AB; Thu, 28 Sep 2017 01:20:04 +0200 (CEST) Subject: Re: [Asterisk-bsd] Asterisk13 coredump on freebsd 11.1 From: Hans Petter Selasky To: Tao Zhou , freebsd-stable , Konstantin Belousov , David Wetzel , Ed Maste Reply-To: Asterisk on BSD discussion References: <30f177e2-3fd7-37e7-2f77-4b43a56c6713@ish.com.au> <25f05b1c-34e5-aa88-39cc-55c9a7b15616@selasky.org> Message-ID: <81116454-105e-f72a-5251-a45aac100c22@selasky.org> Date: Thu, 28 Sep 2017 01:17:24 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <25f05b1c-34e5-aa88-39cc-55c9a7b15616@selasky.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Sep 2017 23:20:07 -0000 Hi, I just upgraded and hit these SEGFAULTs too. First of all you need to install GDB 8.0 from ports to get the right backtrace (important). This leads straight into LibUnwind in libgcc: (gdb) bt #0 uw_frame_state_for (context=context@entry=0x7fffdf3bbe20, fs=fs@entry=0x7fffdf3bbb70) at /wrkdirs/usr/ports/lang/gcc6/work/gcc-6.4.0/libgcc/unwind-dw2.c:1249 #1 0x0000000802cc8ffb in _Unwind_ForcedUnwind_Phase2 (exc=exc@entry=0x804427230, context=context@entry=0x7fffdf3bbe20) at /wrkdirs/usr/ports/lang/gcc6/work/gcc-6.4.0/libgcc/unwind.inc:155 #2 0x0000000802cc9334 in _Unwind_ForcedUnwind (exc=0x804427230, stop=0x8024d5450 , stop_argument=) at /wrkdirs/usr/ports/lang/gcc6/work/gcc-6.4.0/libgcc/unwind.inc:207 #3 0x00000008024d52b3 in _Unwind_ForcedUnwind (ex=, stop_func=0x7fffdf3bb948, stop_arg=0x804427000) at /usr/img/freebsd.11/lib/libthr/thread/thr_exit.c:106 #4 thread_unwind () at /usr/img/freebsd.11/lib/libthr/thread/thr_exit.c:172 #5 _pthread_exit_mask (status=, mask=) at /usr/img/freebsd.11/lib/libthr/thread/thr_exit.c:257 #6 0x00000008024d50db in _pthread_exit (status=0x804427000) at /usr/img/freebsd.11/lib/libthr/thread/thr_exit.c:206 #7 0x00000008024c7c0d in thread_start (curthread=0x804427000) at /usr/img/freebsd.11/lib/libthr/thread/thr_create.c:289 #8 0x00007fffdf340000 in ?? () Backtrace stopped: Cannot access memory at address 0x7fffdf3bc000 libgcc uses this format which is OK: (gdb) ptype struct _Unwind_Context type = struct _Unwind_Context { _Unwind_Context_Reg_Val reg[18]; void *cfa; void *ra; void *lsda; struct dwarf_eh_bases bases; _Unwind_Word flags; _Unwind_Word version; _Unwind_Word args_size; char by_value[18]; } > x86_64_freebsd_fallback_frame_state > (struct _Unwind_Context *context, _Unwind_FrameState *fs) > { > struct sigframe *sf; > long new_cfa; > > /* Prior to FreeBSD 9, the signal trampoline was located immediately > before the ps_strings. To support non-executable stacks on AMD64, > the sigtramp was moved to a shared page for FreeBSD 9. Unfortunately > this means looking frame patterns again (sys/amd64/amd64/sigtramp.S) > rather than using the robust and convenient KERN_PS_STRINGS trick. > > : lea 0x10(%rsp),%rdi > : pushq $0x0 > : mov $0x1a1,%rax > : syscall > > If we can't find this pattern, we're at the end of the stack. > */ > > if (!( *(unsigned int *)(context->ra) == 0x247c8d48 ^^^^ fault is triggered by this read access on the stack > && *(unsigned int *)(context->ra + 4) == 0x48006a10 > && *(unsigned int *)(context->ra + 8) == 0x01a1c0c7 > && *(unsigned int *)(context->ra + 12) == 0x050f0000 )) > return _URC_END_OF_STACK; > The code in question is trying to access the return address of the caller on the stack which apparently I think is caught by the recently added MAP_GUARD feature: https://svnweb.freebsd.org/changeset/base/320763 I think this feature can be disabled by setting: sysctl security.bsd.stack_guard_page=0 And then restart Asterisk. Not sure if it helps, currently testing. This my best guess why Asterisk started segfaulting when upgrading to 11.1. --HPS From owner-freebsd-stable@freebsd.org Thu Sep 28 00:01:12 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AB533E1F32A for ; Thu, 28 Sep 2017 00:01:12 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from bat.yew.relay.mailchannels.net (bat.yew.relay.mailchannels.net [23.83.220.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F360A667E8 for ; Thu, 28 Sep 2017 00:01:09 +0000 (UTC) (envelope-from ian@freebsd.org) X-Sender-Id: _forwarded-from|73.78.92.27 Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 6863B20604D for ; Thu, 28 Sep 2017 00:00:59 +0000 (UTC) Received: from outbound1a.eu.mailhop.org (unknown [100.96.131.244]) (Authenticated sender: duocircle) by relay.mailchannels.net (Postfix) with ESMTPA id C824E208891 for ; Thu, 28 Sep 2017 00:00:58 +0000 (UTC) X-Sender-Id: _forwarded-from|73.78.92.27 Received: from outbound1a.eu.mailhop.org (outbound1a.eu.mailhop.org [100.96.131.1]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384) by 0.0.0.0:2500 (trex/5.9.14); Thu, 28 Sep 2017 00:00:59 +0000 X-MC-Relay: Junk X-MailChannels-SenderId: _forwarded-from|73.78.92.27 X-MailChannels-Auth-Id: duocircle X-Illustrious-Wiry: 5e57cca34d19d403_1506556859251_805839455 X-MC-Loop-Signature: 1506556859250:2009194927 X-MC-Ingress-Time: 1506556859250 X-MHO-User: 15597574-a3e0-11e7-a893-25625093991c X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 73.78.92.27 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [73.78.92.27]) by outbound1.eu.mailhop.org (Halon) with ESMTPSA id 15597574-a3e0-11e7-a893-25625093991c; Thu, 28 Sep 2017 00:00:51 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id v8S00khc002842; Wed, 27 Sep 2017 18:00:46 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: <1506556846.31939.15.camel@freebsd.org> Subject: Re: [Asterisk-bsd] Asterisk13 coredump on freebsd 11.1 From: Ian Lepore To: Asterisk on BSD discussion , Tao Zhou , freebsd-stable , Konstantin Belousov , David Wetzel , Ed Maste Date: Wed, 27 Sep 2017 18:00:46 -0600 In-Reply-To: <81116454-105e-f72a-5251-a45aac100c22@selasky.org> References: <30f177e2-3fd7-37e7-2f77-4b43a56c6713@ish.com.au> <25f05b1c-34e5-aa88-39cc-55c9a7b15616@selasky.org> <81116454-105e-f72a-5251-a45aac100c22@selasky.org> Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 00:01:12 -0000 On Thu, 2017-09-28 at 01:17 +0200, Hans Petter Selasky wrote: > Hi, >=20 > I just upgraded and hit these SEGFAULTs too. First of all you need > to=A0 > install GDB 8.0 from ports to get the right backtrace (important). > This=A0 > leads straight into LibUnwind in libgcc: >=20 > (gdb) bt > #0=A0=A0uw_frame_state_for (context=3Dcontext@entry=3D0x7fffdf3bbe20,=A0 > fs=3Dfs@entry=3D0x7fffdf3bbb70) > =A0=A0=A0=A0=A0at /wrkdirs/usr/ports/lang/gcc6/work/gcc-6.4.0/libgcc/un= wind- > dw2.c:1249 > #1=A0=A00x0000000802cc8ffb in _Unwind_ForcedUnwind_Phase2=A0 > (exc=3Dexc@entry=3D0x804427230, > =A0=A0=A0=A0=A0context=3Dcontext@entry=3D0x7fffdf3bbe20) at=A0 > /wrkdirs/usr/ports/lang/gcc6/work/gcc-6.4.0/libgcc/unwind.inc:155 > #2=A0=A00x0000000802cc9334 in _Unwind_ForcedUnwind (exc=3D0x804427230,=A0 > stop=3D0x8024d5450 , > =A0=A0=A0=A0=A0stop_argument=3D) at=A0 > /wrkdirs/usr/ports/lang/gcc6/work/gcc-6.4.0/libgcc/unwind.inc:207 > #3=A0=A00x00000008024d52b3 in _Unwind_ForcedUnwind (ex=3D,=A0 > stop_func=3D0x7fffdf3bb948, stop_arg=3D0x804427000) > =A0=A0=A0=A0=A0at /usr/img/freebsd.11/lib/libthr/thread/thr_exit.c:106 > #4=A0=A0thread_unwind () at > /usr/img/freebsd.11/lib/libthr/thread/thr_exit.c:172 > #5=A0=A0_pthread_exit_mask (status=3D, mask=3D) > =A0=A0=A0=A0=A0at /usr/img/freebsd.11/lib/libthr/thread/thr_exit.c:257 > #6=A0=A00x00000008024d50db in _pthread_exit (status=3D0x804427000) at=A0 > /usr/img/freebsd.11/lib/libthr/thread/thr_exit.c:206 > #7=A0=A00x00000008024c7c0d in thread_start (curthread=3D0x804427000) > =A0=A0=A0=A0=A0at /usr/img/freebsd.11/lib/libthr/thread/thr_create.c:28= 9 > #8=A0=A00x00007fffdf340000 in ?? () > Backtrace stopped: Cannot access memory at address 0x7fffdf3bc000 >=20 > libgcc uses this format which is OK: >=20 > (gdb) ptype struct _Unwind_Context > type =3D struct _Unwind_Context { > =A0=A0=A0=A0=A0_Unwind_Context_Reg_Val reg[18]; > =A0=A0=A0=A0=A0void *cfa; > =A0=A0=A0=A0=A0void *ra; > =A0=A0=A0=A0=A0void *lsda; > =A0=A0=A0=A0=A0struct dwarf_eh_bases bases; > =A0=A0=A0=A0=A0_Unwind_Word flags; > =A0=A0=A0=A0=A0_Unwind_Word version; > =A0=A0=A0=A0=A0_Unwind_Word args_size; > =A0=A0=A0=A0=A0char by_value[18]; > } >=20 > >=20 > > x86_64_freebsd_fallback_frame_state > > (struct _Unwind_Context *context, _Unwind_FrameState *fs) > > { > > =A0 struct sigframe *sf; > > =A0 long new_cfa; > >=20 > > =A0 /* Prior to FreeBSD 9, the signal trampoline was located > > immediately > > =A0=A0=A0=A0=A0before the ps_strings.=A0=A0To support non-executable = stacks on > > AMD64, > > =A0=A0=A0=A0=A0the sigtramp was moved to a shared page for FreeBSD > > 9.=A0=A0Unfortunately > > =A0=A0=A0=A0=A0this means looking frame patterns again > > (sys/amd64/amd64/sigtramp.S) > > =A0=A0=A0=A0=A0rather than using the robust and convenient KERN_PS_ST= RINGS > > trick. > >=20 > > =A0=A0=A0=A0=A0:=A0=A0lea=A0=A0=A0=A0=A00x10(%rsp),%rdi > > =A0=A0=A0=A0=A0:=A0=A0pushq=A0=A0=A0$0x0 > > =A0=A0=A0=A0=A0:=A0=A0mov=A0=A0=A0=A0=A0$0x1a1,%rax > > =A0=A0=A0=A0=A0:=A0=A0syscall > >=20 > > =A0=A0=A0=A0=A0If we can't find this pattern, we're at the end of the= stack. > > =A0 */ > >=20 > > =A0 if (!(=A0=A0=A0*(unsigned int *)(context->ra)=A0=A0=A0=A0=A0=A0=3D= =3D 0x247c8d48 > =A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0=A0^^^^ fault is triggered by th= is read access on the > stack > >=20 > > =A0=A0=A0=A0=A0=A0=A0=A0&& *(unsigned int *)(context->ra +=A0=A04) =3D= =3D 0x48006a10 > > =A0=A0=A0=A0=A0=A0=A0=A0&& *(unsigned int *)(context->ra +=A0=A08) =3D= =3D 0x01a1c0c7 > > =A0=A0=A0=A0=A0=A0=A0=A0&& *(unsigned int *)(context->ra + 12) =3D=3D= 0x050f0000 )) > > =A0=A0=A0=A0return _URC_END_OF_STACK; > >=20 > The code in question is trying to access the return address of the=A0 > caller on the stack which apparently I think is caught by the > recently=A0 > added MAP_GUARD feature: >=20 > https://svnweb.freebsd.org/changeset/base/320763 >=20 > I think this feature can be disabled by setting: > sysctl security.bsd.stack_guard_page=3D0 >=20 > And then restart Asterisk. Not sure if it helps, currently testing. > This my best guess why Asterisk started segfaulting when upgrading to > 11.1. >=20 > --HPS In 12-current we've switched to the unwind code from the llvm project. =A0I wonder if that can be MFC'd to 11? There are other problems in the contrib/gcc unwind code in 11 right now. =A0For example, I've been chasing what appears to be a clang codegen bug that prevents returning a value from a function that contains a call to __builtin_eh_return(). =A0That leads to a bogus return value getting misinterpretted and eventually abort() gets called when std::terminate() should be called instead due to an uncaught exception. -- Ian From owner-freebsd-stable@freebsd.org Thu Sep 28 07:32:30 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 81305E2B530 for ; Thu, 28 Sep 2017 07:32:30 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0FBA073834; Thu, 28 Sep 2017 07:32:29 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id v8S7WJue095846 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 28 Sep 2017 10:32:19 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua v8S7WJue095846 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id v8S7WHZN095844; Thu, 28 Sep 2017 10:32:17 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 28 Sep 2017 10:32:17 +0300 From: Konstantin Belousov To: Asterisk on BSD discussion Cc: Tao Zhou , freebsd-stable , Konstantin Belousov , David Wetzel , Ed Maste Subject: Re: [Asterisk-bsd] Asterisk13 coredump on freebsd 11.1 Message-ID: <20170928073217.GN2271@kib.kiev.ua> References: <30f177e2-3fd7-37e7-2f77-4b43a56c6713@ish.com.au> <25f05b1c-34e5-aa88-39cc-55c9a7b15616@selasky.org> <81116454-105e-f72a-5251-a45aac100c22@selasky.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <81116454-105e-f72a-5251-a45aac100c22@selasky.org> User-Agent: Mutt/1.9.0 (2017-09-02) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 07:32:30 -0000 On Thu, Sep 28, 2017 at 01:17:24AM +0200, Hans Petter Selasky wrote: > Hi, > > I just upgraded and hit these SEGFAULTs too. First of all you need to > install GDB 8.0 from ports to get the right backtrace (important). This > leads straight into LibUnwind in libgcc: > > (gdb) bt > #0 uw_frame_state_for (context=context@entry=0x7fffdf3bbe20, > fs=fs@entry=0x7fffdf3bbb70) > at /wrkdirs/usr/ports/lang/gcc6/work/gcc-6.4.0/libgcc/unwind-dw2.c:1249 > #1 0x0000000802cc8ffb in _Unwind_ForcedUnwind_Phase2 > (exc=exc@entry=0x804427230, > context=context@entry=0x7fffdf3bbe20) at > /wrkdirs/usr/ports/lang/gcc6/work/gcc-6.4.0/libgcc/unwind.inc:155 > #2 0x0000000802cc9334 in _Unwind_ForcedUnwind (exc=0x804427230, > stop=0x8024d5450 , > stop_argument=) at > /wrkdirs/usr/ports/lang/gcc6/work/gcc-6.4.0/libgcc/unwind.inc:207 > #3 0x00000008024d52b3 in _Unwind_ForcedUnwind (ex=, > stop_func=0x7fffdf3bb948, stop_arg=0x804427000) > at /usr/img/freebsd.11/lib/libthr/thread/thr_exit.c:106 > #4 thread_unwind () at /usr/img/freebsd.11/lib/libthr/thread/thr_exit.c:172 > #5 _pthread_exit_mask (status=, mask=) > at /usr/img/freebsd.11/lib/libthr/thread/thr_exit.c:257 > #6 0x00000008024d50db in _pthread_exit (status=0x804427000) at > /usr/img/freebsd.11/lib/libthr/thread/thr_exit.c:206 > #7 0x00000008024c7c0d in thread_start (curthread=0x804427000) > at /usr/img/freebsd.11/lib/libthr/thread/thr_create.c:289 > #8 0x00007fffdf340000 in ?? () > Backtrace stopped: Cannot access memory at address 0x7fffdf3bc000 > > libgcc uses this format which is OK: > > (gdb) ptype struct _Unwind_Context > type = struct _Unwind_Context { > _Unwind_Context_Reg_Val reg[18]; > void *cfa; > void *ra; > void *lsda; > struct dwarf_eh_bases bases; > _Unwind_Word flags; > _Unwind_Word version; > _Unwind_Word args_size; > char by_value[18]; > } > > > x86_64_freebsd_fallback_frame_state > > (struct _Unwind_Context *context, _Unwind_FrameState *fs) > > { > > struct sigframe *sf; > > long new_cfa; > > > > /* Prior to FreeBSD 9, the signal trampoline was located immediately > > before the ps_strings. To support non-executable stacks on AMD64, > > the sigtramp was moved to a shared page for FreeBSD 9. Unfortunately > > this means looking frame patterns again (sys/amd64/amd64/sigtramp.S) > > rather than using the robust and convenient KERN_PS_STRINGS trick. > > > > : lea 0x10(%rsp),%rdi > > : pushq $0x0 > > : mov $0x1a1,%rax > > : syscall > > > > If we can't find this pattern, we're at the end of the stack. > > */ > > > > if (!( *(unsigned int *)(context->ra) == 0x247c8d48 > ^^^^ fault is triggered by this read access on the stack > > && *(unsigned int *)(context->ra + 4) == 0x48006a10 > > && *(unsigned int *)(context->ra + 8) == 0x01a1c0c7 > > && *(unsigned int *)(context->ra + 12) == 0x050f0000 )) > > return _URC_END_OF_STACK; > > > > The code in question is trying to access the return address of the > caller on the stack which apparently I think is caught by the recently > added MAP_GUARD feature: > > https://svnweb.freebsd.org/changeset/base/320763 > > I think this feature can be disabled by setting: > sysctl security.bsd.stack_guard_page=0 > > And then restart Asterisk. Not sure if it helps, currently testing. > This my best guess why Asterisk started segfaulting when upgrading to 11.1. See this thread on current https://lists.freebsd.org/pipermail/freebsd-current/2017-August/066855.html which contained at least two variants of the supposed improvements. From owner-freebsd-stable@freebsd.org Thu Sep 28 14:20:52 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1F38FE32E0B for ; Thu, 28 Sep 2017 14:20:52 +0000 (UTC) (envelope-from chris@vindaloo.com) Received: from yavin.vindaloo.com (yavin.vindaloo.com [173.199.117.73]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "yavin.vindaloo.com", Issuer "Vindaloo Sign CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id F11D6EB2; Thu, 28 Sep 2017 14:20:51 +0000 (UTC) (envelope-from chris@vindaloo.com) Received: from anza.vindaloo.com (ool-45714982.dyn.optonline.net [69.113.73.130]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "smtp.vindaloo.com", Issuer "Vindaloo Sign CA" (verified OK)) by yavin.vindaloo.com (Postfix) with ESMTPS id 2DEEED7A87; Thu, 28 Sep 2017 10:20:50 -0400 (EDT) Received: from csh-desktop-vm00.loopone.com (h4.82.141.40.ip.windstream.net [40.141.82.4]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client did not present a certificate) by anza.vindaloo.com (Postfix) with ESMTPSA id A05E71019B; Thu, 28 Sep 2017 10:20:49 -0400 (EDT) Date: Thu, 28 Sep 2017 10:20:47 -0400 From: Christopher Sean Hilton To: Dimitry Andric Cc: freebsd-stable@freebsd.org Subject: Re: Bind9 + TCP_FASTOPEN => no rndc Message-ID: <20170928142047.dgzji5mdic632u7w@csh-desktop-vm00.loopone.com> References: <20170927173525.bspia3tpcu35yng3@kessel.vindaloo.com> <5CF82983-FDA1-4F83-9D47-D36845A12B97@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5CF82983-FDA1-4F83-9D47-D36845A12B97@FreeBSD.org> User-Agent: NeoMutt/20170914 (1.9.0) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 14:20:52 -0000 On Wed, Sep 27, 2017 at 09:17:29PM +0000, Dimitry Andric wrote: > On 27 Sep 2017, at 19:35, Christopher Sean Hilton wrote: > > > > I'm trying to configure bind 9.11 as a nameserver on FreeBSD > > 11-STABLE. When the bind9 port compile it enables TCP_FASTOPEN but the > > changes haven't yet been baked into the GENERIC Kernel. I can't find a > > way to disable the use of TCP_FASTOPEN in bind at startup. Is the only > > way to fix this problem to build a new kernel with TCP_FASTOPEN > > enabled? > > It looks like bind enables use of TCP_FASTOPEN whenever its configure > script finds the define in the system headers. But it does not check > whether the functionality actually works with setsockopt. > > In any case, the message is harmless noise, as any errors are ignored: > > #if defined(ISC_PLATFORM_HAVETFO) && defined(TCP_FASTOPEN) > #ifdef __APPLE__ > backlog = 1; > #else > backlog = backlog / 2; > if (backlog == 0) > backlog = 1; > #endif > if (setsockopt(sock->fd, IPPROTO_TCP, TCP_FASTOPEN, > (void *)&backlog, sizeof(backlog)) < 0) { > isc__strerror(errno, strbuf, sizeof(strbuf)); > UNEXPECTED_ERROR(__FILE__, __LINE__, > "setsockopt(%d, TCP_FASTOPEN) failed with %s", > sock->fd, strbuf); > /* TCP_FASTOPEN is experimental so ignore failures */ > } > #endif > Great, I assumed that the FASTOPEN failure was related to the inablity to open the rndc socket. I'll have to debug the rndc socket seperately. Thanks for help! -- Chris -- Chris __o "All I was trying to do was get home from work." _`\<,_ -Rosa Parks ___(*)/_(*)____.___o____..___..o...________ooO..._____________________ Christopher Sean Hilton [chris/at/vindaloo/dot/com] From owner-freebsd-stable@freebsd.org Thu Sep 28 15:28:23 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 00C24E01346 for ; Thu, 28 Sep 2017 15:28:22 +0000 (UTC) (envelope-from chris@vindaloo.com) Received: from yavin.vindaloo.com (yavin.vindaloo.com [173.199.117.73]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "yavin.vindaloo.com", Issuer "Vindaloo Sign CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id C59EA35AA; Thu, 28 Sep 2017 15:28:22 +0000 (UTC) (envelope-from chris@vindaloo.com) Received: from anza.vindaloo.com (ool-45714982.dyn.optonline.net [69.113.73.130]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "smtp.vindaloo.com", Issuer "Vindaloo Sign CA" (verified OK)) by yavin.vindaloo.com (Postfix) with ESMTPS id 03458D7A88; Thu, 28 Sep 2017 11:28:20 -0400 (EDT) Received: from csh-desktop-vm00.loopone.com (h4.82.141.40.ip.windstream.net [40.141.82.4]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client did not present a certificate) by anza.vindaloo.com (Postfix) with ESMTPSA id 77D79101E0; Thu, 28 Sep 2017 11:28:19 -0400 (EDT) Date: Thu, 28 Sep 2017 11:28:17 -0400 From: Christopher Sean Hilton To: Dimitry Andric Cc: freebsd-stable@freebsd.org Subject: Re: Bind9 + TCP_FASTOPEN => no rndc Message-ID: <20170928152817.onsil7mtmnlaobpq@csh-desktop-vm00.loopone.com> References: <20170927173525.bspia3tpcu35yng3@kessel.vindaloo.com> <5CF82983-FDA1-4F83-9D47-D36845A12B97@FreeBSD.org> <20170928142047.dgzji5mdic632u7w@csh-desktop-vm00.loopone.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170928142047.dgzji5mdic632u7w@csh-desktop-vm00.loopone.com> User-Agent: NeoMutt/20170914 (1.9.0) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 15:28:23 -0000 On Thu, Sep 28, 2017 at 02:20:47PM +0000, Christopher Sean Hilton wrote: > Great, > > I assumed that the FASTOPEN failure was related to the inablity to > open the rndc socket. I'll have to debug the rndc socket seperately. > > > Thanks for help! > This had nothing to do with FASTOPEN and everything to do with moving named from the base system to ports. The solution is to explicitly configure the location of the rndc.key file in named.conf. Otherwise named looks in the wrong directory to find it and then fails. Simple when you know what's wrong. -- Chris __o "All I was trying to do was get home from work." _`\<,_ -Rosa Parks ___(*)/_(*)____.___o____..___..o...________ooO..._____________________ Christopher Sean Hilton [chris/at/vindaloo/dot/com] From owner-freebsd-stable@freebsd.org Thu Sep 28 15:31:58 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 429B3E014D2 for ; Thu, 28 Sep 2017 15:31:58 +0000 (UTC) (envelope-from chris@vindaloo.com) Received: from yavin.vindaloo.com (yavin.vindaloo.com [173.199.117.73]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "yavin.vindaloo.com", Issuer "Vindaloo Sign CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 1FE483938; Thu, 28 Sep 2017 15:31:57 +0000 (UTC) (envelope-from chris@vindaloo.com) Received: from anza.vindaloo.com (ool-45714982.dyn.optonline.net [69.113.73.130]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "smtp.vindaloo.com", Issuer "Vindaloo Sign CA" (verified OK)) by yavin.vindaloo.com (Postfix) with ESMTPS id CA0C7D7A88; Thu, 28 Sep 2017 11:31:56 -0400 (EDT) Received: from csh-desktop-vm00.loopone.com (h4.82.141.40.ip.windstream.net [40.141.82.4]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client did not present a certificate) by anza.vindaloo.com (Postfix) with ESMTPSA id 5149A10111; Thu, 28 Sep 2017 11:31:56 -0400 (EDT) Date: Thu, 28 Sep 2017 11:31:54 -0400 From: Christopher Sean Hilton To: Dimitry Andric Cc: freebsd-stable@freebsd.org Subject: [Solved] Bind9 + TCP_FASTOPEN => no rndc Message-ID: <20170928153154.ibi3elq6fedis3g5@csh-desktop-vm00.loopone.com> References: <20170927173525.bspia3tpcu35yng3@kessel.vindaloo.com> <5CF82983-FDA1-4F83-9D47-D36845A12B97@FreeBSD.org> <20170928142047.dgzji5mdic632u7w@csh-desktop-vm00.loopone.com> <20170928152817.onsil7mtmnlaobpq@csh-desktop-vm00.loopone.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170928152817.onsil7mtmnlaobpq@csh-desktop-vm00.loopone.com> User-Agent: NeoMutt/20170914 (1.9.0) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2017 15:31:58 -0000 On Thu, Sep 28, 2017 at 03:28:17PM +0000, Christopher Sean Hilton wrote: > On Thu, Sep 28, 2017 at 02:20:47PM +0000, Christopher Sean Hilton wrote: > > Great, > > > > I assumed that the FASTOPEN failure was related to the inablity to > > open the rndc socket. I'll have to debug the rndc socket seperately. > > > > > > Thanks for help! > > > Just a note for future readers to say that the fix from the previous message solved this problem. -- Chris __o "All I was trying to do was get home from work." _`\<,_ -Rosa Parks ___(*)/_(*)____.___o____..___..o...________ooO..._____________________ Christopher Sean Hilton [chris/at/vindaloo/dot/com] From owner-freebsd-stable@freebsd.org Fri Sep 29 04:46:58 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BEC09E22BAB for ; Fri, 29 Sep 2017 04:46:58 +0000 (UTC) (envelope-from paul.koch@akips.com) Received: from mail.akips.com (mail.akips.com [45.32.79.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6CAA97D64D; Fri, 29 Sep 2017 04:46:58 +0000 (UTC) (envelope-from paul.koch@akips.com) Date: Fri, 29 Sep 2017 14:36:59 +1000 From: Paul Koch To: Sepherosa Ziehau Cc: Julian Elischer , freebsd-stable@freebsd.org, Hongjiang Zhang Subject: Re: 11.1 running on HyperV hn interface hangs Message-ID: <20170929143659.7fe1a4f6@akips.com> In-Reply-To: References: <20170906193309.796c79ed@akips.com> <3f96c7d0-4fbd-26cb-5c84-8868d12eb427@ingresso.co.uk> <14997bba-aac2-947b-9b78-04af41ea29b0@freebsd.org> <20170907150711.025e4e41@akips.com> <20170908010400.0fb81bc4@akips.com> Organization: AKIPS MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2017 04:46:58 -0000 On Thu, 14 Sep 2017 09:54:56 +0800 Sepherosa Ziehau wrote: > If you have any updates on this, please let me know. There is still > time for 10.4. We are still playing around with this in the lab... Running similar setup as the customer Microsoft Windows Server 2012 R2 Datacentre (6.3.9600) Revision 16384 Hyper-V 2012 Two VM guests - 11.0-RELEASE - 11.1-p1 We can not get the Hyper-V hn interface to lock up like the customer can though. We can get the VMs to hang/stall regularly if the guests run ntpd - approx every 15 mins, but no real obvious pattern to it. Disabling ntpd fixes it. Paul. -- Paul Koch | Founder | CEO AKIPS Network Monitor | akips.com Brisbane, Australia From owner-freebsd-stable@freebsd.org Fri Sep 29 05:31:24 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2546DE23961 for ; Fri, 29 Sep 2017 05:31:24 +0000 (UTC) (envelope-from sepherosa@gmail.com) Received: from mail-qk0-x232.google.com (mail-qk0-x232.google.com [IPv6:2607:f8b0:400d:c09::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C87357E5B5; Fri, 29 Sep 2017 05:31:23 +0000 (UTC) (envelope-from sepherosa@gmail.com) Received: by mail-qk0-x232.google.com with SMTP id z143so225001qkb.3; Thu, 28 Sep 2017 22:31:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=VuiU9ok61Tetur9pLazEkKPTj6VkROqVXFQUSps067I=; b=rolGv45S5TDlJx9t+TJbW9CNKcXmuf6JG2B13M1NNt0XXqKfqGI2fbccbGLoxqReN1 H3AXJBmsdSD6wqoTMDyU7ZRX4ZZpuoq4K2xlKogYs9bm/RdgAmb3UKRPR/9oAS2sNbvE HDNOXwcQTPvNteUnM9Oxbuacq73gvOm5STTJRXTM0abO2+gke3xZC8nZyeR8IM6n7SP2 +LwVVWQifb5CIpcugr8RBCuvsPjCTlttHCg7LDgz0Y4SNrsGxih0L80tC2Z6cKHf1iju Y3qjKK1Qqh/25Zlw8VH0QnGjOD9aJ88eSFGfRSd7rdkbuNgjkYFAmIp0KwsO4w/JgVrt CMJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=VuiU9ok61Tetur9pLazEkKPTj6VkROqVXFQUSps067I=; b=mC657sLOdRW49Hb06SmwvyTCIIDP5jSI3bZemP6mV/ufik2nfMMiGjUbNEzI9YAg1L LAM49W/MGAVqFHp8UBGD91vfGayOZNc0rlL1zj9w+Iu3Exe+os/eF02z4JGRTqMjiEkZ ALQO3PpcN8bn8eLQCrOZljndLM6dyQcE+5I4shioVMTZVdY1Ga+S+oPAq3qZUgmdBgPz k2wrT8I7I0r8ONDA+jyJ+pB53WlNXvCmWV44xC1DN88Iai1d+PBMU6ZB7t3ApgT8WJme sgirqgKixzMEniYMpdCZnYeCHmw9zxxtx+SJ+t8bKbDE1zqw10T6lFH/tT+wtu/ghUoh hTQQ== X-Gm-Message-State: AMCzsaXCGCWTLzA5l5gwauog8yOZVPI1gDX0j/jWyNAiMOQLdq3BX//C WBnW0K4AhYRONA/syQcnzft7TdgX8ymWP2LGIw== X-Google-Smtp-Source: AOwi7QCOpb7mGIMcy77trv2mI0+LJ9OaPh9Ko5hHjwolmZJ5+vzqAkvhqk5g006IadC9t4ch5clrbyljgRnB/xE7jC4= X-Received: by 10.55.101.77 with SMTP id z74mr1651426qkb.52.1506663082826; Thu, 28 Sep 2017 22:31:22 -0700 (PDT) MIME-Version: 1.0 Sender: sepherosa@gmail.com Received: by 10.140.21.20 with HTTP; Thu, 28 Sep 2017 22:31:22 -0700 (PDT) In-Reply-To: <20170929143659.7fe1a4f6@akips.com> References: <20170906193309.796c79ed@akips.com> <3f96c7d0-4fbd-26cb-5c84-8868d12eb427@ingresso.co.uk> <14997bba-aac2-947b-9b78-04af41ea29b0@freebsd.org> <20170907150711.025e4e41@akips.com> <20170908010400.0fb81bc4@akips.com> <20170929143659.7fe1a4f6@akips.com> From: Sepherosa Ziehau Date: Fri, 29 Sep 2017 13:31:22 +0800 X-Google-Sender-Auth: tktHpb9TmLRiPx5Xbc5VnGn88eY Message-ID: Subject: Re: 11.1 running on HyperV hn interface hangs To: Paul Koch Cc: Julian Elischer , freebsd-stable@freebsd.org, Hongjiang Zhang Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2017 05:31:24 -0000 On Fri, Sep 29, 2017 at 12:36 PM, Paul Koch wrote: > On Thu, 14 Sep 2017 09:54:56 +0800 > Sepherosa Ziehau wrote: > >> If you have any updates on this, please let me know. There is still >> time for 10.4. > > We are still playing around with this in the lab... > > Running similar setup as the customer > Microsoft Windows Server 2012 R2 Datacentre (6.3.9600) Revision 16384 > Hyper-V 2012 > > Two VM guests > - 11.0-RELEASE > - 11.1-p1 > > We can not get the Hyper-V hn interface to lock up like the customer can > though. > > We can get the VMs to hang/stall regularly if the guests run ntpd - approx > every 15 mins, but no real obvious pattern to it. Disabling ntpd fixes it. Hmm, by ntpd I think you mean ntp client? You will have to disable timesync if you run ntp client: sysctl hw.hvtimesync.sample_thresh=-1 sysctl hw.hvtimesync.ignore_sync=1 They interfere w/ each other. Or do you mean the network hanging triggered by "RXBUF ack failed"? Thanks, sephe -- Tomorrow Will Never Die From owner-freebsd-stable@freebsd.org Fri Sep 29 05:42:32 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 90852E23E33 for ; Fri, 29 Sep 2017 05:42:32 +0000 (UTC) (envelope-from paul.koch@akips.com) Received: from mail.akips.com (mail.akips.com [45.32.79.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7E5217EC7A; Fri, 29 Sep 2017 05:42:32 +0000 (UTC) (envelope-from paul.koch@akips.com) Date: Fri, 29 Sep 2017 15:42:25 +1000 From: Paul Koch To: Sepherosa Ziehau Cc: Julian Elischer , freebsd-stable@freebsd.org, Hongjiang Zhang Subject: Re: 11.1 running on HyperV hn interface hangs Message-ID: <20170929154225.6a187f47@akips.com> In-Reply-To: References: <20170906193309.796c79ed@akips.com> <3f96c7d0-4fbd-26cb-5c84-8868d12eb427@ingresso.co.uk> <14997bba-aac2-947b-9b78-04af41ea29b0@freebsd.org> <20170907150711.025e4e41@akips.com> <20170908010400.0fb81bc4@akips.com> <20170929143659.7fe1a4f6@akips.com> Organization: AKIPS MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2017 05:42:32 -0000 On Fri, 29 Sep 2017 13:31:22 +0800 Sepherosa Ziehau wrote: > On Fri, Sep 29, 2017 at 12:36 PM, Paul Koch wrote: > > On Thu, 14 Sep 2017 09:54:56 +0800 > > Sepherosa Ziehau wrote: > > > >> If you have any updates on this, please let me know. There is still > >> time for 10.4. > > > > We are still playing around with this in the lab... > > > > Running similar setup as the customer > > Microsoft Windows Server 2012 R2 Datacentre (6.3.9600) Revision 16384 > > Hyper-V 2012 > > > > Two VM guests > > - 11.0-RELEASE > > - 11.1-p1 > > > > We can not get the Hyper-V hn interface to lock up like the customer can > > though. > > > > We can get the VMs to hang/stall regularly if the guests run ntpd - approx > > every 15 mins, but no real obvious pattern to it. Disabling ntpd fixes > > it. > > Hmm, by ntpd I think you mean ntp client? You will have to disable > timesync if you run ntp client: > sysctl hw.hvtimesync.sample_thresh=-1 > sysctl hw.hvtimesync.ignore_sync=1 > > They interfere w/ each other. > > Or do you mean the network hanging triggered by "RXBUF ack failed"? > > Thanks, > sephe Yes, ntp client running on the VM guest. After finding it was unstable, we concluded that there must be some type of interference. Best that we automatically force ntp off when our software detects it is running in Hyper-V. We haven't been able to trigger the hn to hang like our customer can with the RXBUF problem though. Different underlying hardware probably. Paul. -- Paul Koch | Founder | CEO AKIPS Network Monitor | akips.com Brisbane, Australia From owner-freebsd-stable@freebsd.org Fri Sep 29 12:23:13 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 45E27E2C1CB for ; Fri, 29 Sep 2017 12:23:13 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from constantine.ingresso.co.uk (unknown [IPv6:2a02:b90:3002:411::3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0EF1564A49; Fri, 29 Sep 2017 12:23:12 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from dilbert.ingresso.co.uk ([2a02:b90:3002:411::6]) by constantine.ingresso.co.uk with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89 (FreeBSD)) (envelope-from ) id 1dxuJx-000K4y-4S; Fri, 29 Sep 2017 12:23:09 +0000 Received: from petefrench by dilbert.ingresso.co.uk with local (Exim 4.89 (FreeBSD)) (envelope-from ) id 1dxuJw-0008cc-QR; Fri, 29 Sep 2017 13:23:08 +0100 To: paul.koch@akips.com, sephe@freebsd.org Subject: Re: 11.1 running on HyperV hn interface hangs Cc: freebsd-stable@freebsd.org, honzhan@microsoft.com, julian@freebsd.org In-Reply-To: Message-Id: From: Pete French Date: Fri, 29 Sep 2017 13:23:08 +0100 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Sep 2017 12:23:13 -0000 > Hmm, by ntpd I think you mean ntp client? You will have to disable > timesync if you run ntp client: > sysctl hw.hvtimesync.sample_thresh=-1 > sysctl hw.hvtimesync.ignore_sync=1 > > They interfere w/ each other. Oh! Does this apply to machines in Azure Hyper-V as well as on standalone installations ? I wasn't aware of this, and have ntp on as standard (which I wil now turn off pending a reply to this email, as sods law says that now I am aware of it then I will get lockups over the weekend) :-) -pete. From owner-freebsd-stable@freebsd.org Sat Sep 30 03:01:05 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D88CCE07EA2 for ; Sat, 30 Sep 2017 03:01:05 +0000 (UTC) (envelope-from sepherosa@gmail.com) Received: from mail-qk0-x22b.google.com (mail-qk0-x22b.google.com [IPv6:2607:f8b0:400d:c09::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 82B2182EF4; Sat, 30 Sep 2017 03:01:05 +0000 (UTC) (envelope-from sepherosa@gmail.com) Received: by mail-qk0-x22b.google.com with SMTP id u67so1313404qkg.6; Fri, 29 Sep 2017 20:01:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=zUFRb3wVr6HPXyWsgQWqEMFr1sEertsCFmCw4hF+EKo=; b=oIhVimcpOMp0WGnGL1TtEh5ZvP9ekGVr4ti1EN38LUc3Ho6zRHydbqTg0loyTUKNV1 nXPoH4saNVNEw0Fg9q/ikwEQaZuW+Jx2PJBfijc8Q358ZHkfX6MuSyaP380w0psdwTvQ N40zGLkFMxAPm7H9IBUyhpQ0G2M/u7l8d9dPA6w500nZt9obThxYyu/FBkVesx/0x3+N S20FUN6T+hLjsSC9fBcNm3ZYWGK/yrpoPA9Mcb/2XPuZadrdI1TyCAhKsukFIILNIq7q vIgmQlgTO8iiqf5wIW9zGFOM27B1KuiuShB3Gg8jCHS5TpSzwkCqNwibKg4+SKyB6Lar mo6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=zUFRb3wVr6HPXyWsgQWqEMFr1sEertsCFmCw4hF+EKo=; b=m6YaMWOgkxXYkxoV7nGLZJEHgDIMzLiVg4xWgMQiz46Z5q816T/r3wWSLSd4bJbPD5 1IEiOyOHsljw11X2M4Ahz9bAMLFkF604s8r4cz7VZ117bloK6vyJ5jIh8yAyADZr76mY 3S1xVtlH4YsZ6vsH8dv5M8bBkB1VTbtIfL3atUacUNcjcQ7pLtalruT4uTAt5VwOupra xWPRYYuA+/aG+4GixAIRbYixjUg1cZps+xDw1NHX0IcUExD2xFE72Rw5Ai1y7cw3ARSh V9QzzcAaHbz6b2J2Sxa++VDphaEWUatybxFDkIR1WM8plGYMb1HhOuygMBo5J4JP8KRs 4uNQ== X-Gm-Message-State: AMCzsaVHWxDqzL1UcMzzbCALzhDVKLgrP1U17W1okBkvBO33g+PRDqos oGBxlRDIY9CTjC7+Z617vh6Nn2feRLJ5hiwJ3w== X-Google-Smtp-Source: AOwi7QDHqFGaO/Mm4t/fvXs/9P/Hk2cvf4z5HWxyTjneZuSZbeyp9nuxO8rekDXWKkR8thDxrdUc6LLEL8V6env5gjI= X-Received: by 10.55.198.152 with SMTP id s24mr6359301qkl.3.1506740461531; Fri, 29 Sep 2017 20:01:01 -0700 (PDT) MIME-Version: 1.0 Sender: sepherosa@gmail.com Received: by 10.140.42.137 with HTTP; Fri, 29 Sep 2017 20:01:00 -0700 (PDT) In-Reply-To: References: From: Sepherosa Ziehau Date: Sat, 30 Sep 2017 11:01:00 +0800 X-Google-Sender-Auth: kYmEXJEn284Mn8nHsXOH2QSaTJs Message-ID: Subject: Re: 11.1 running on HyperV hn interface hangs To: Pete French Cc: Paul Koch , freebsd-stable@freebsd.org, Hongjiang Zhang , Julian Elischer Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Sep 2017 03:01:06 -0000 On Fri, Sep 29, 2017 at 8:23 PM, Pete French wrote: >> Hmm, by ntpd I think you mean ntp client? You will have to disable >> timesync if you run ntp client: >> sysctl hw.hvtimesync.sample_thresh=-1 >> sysctl hw.hvtimesync.ignore_sync=1 >> >> They interfere w/ each other. > > Oh! Does this apply to machines in Azure Hyper-V as well as on standalone > installations ? I wasn't aware of this, and have ntp on as standard (which > I wil now turn off pending a reply to this email, as sods law says that now > I am aware of it then I will get lockups over the weekend) :-) It applies to both Azure and standalone Hyper-V. In addition to the sysctls I mentioned, you can also add tunable hint.hvtimesync.0.disabled=1 to /boot/loader.conf to disable timesync, if you want to use ntp. Thanks, sephe -- Tomorrow Will Never Die From owner-freebsd-stable@freebsd.org Sat Sep 30 12:53:07 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E1850E27774 for ; Sat, 30 Sep 2017 12:53:07 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 767836F84F for ; Sat, 30 Sep 2017 12:53:07 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (ezra.dcm1.omnilan.net [IPv6:2a00:e10:2800::a135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id v8UCr4uP051551 for ; Sat, 30 Sep 2017 14:53:04 +0200 (CEST) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (s1.omnilan.de [217.91.127.234]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id 2F226BE9; Sat, 30 Sep 2017 14:53:04 +0200 (CEST) Message-ID: <59CF93AF.7000804@omnilan.de> Date: Sat, 30 Sep 2017 14:53:03 +0200 From: Harry Schmalzbauer Organization: OmniLAN User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; de-DE; rv:1.9.2.8) Gecko/20100906 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: freebsd-stable@freebsd.org Subject: find(1)'s "newer" primary expression gives wrong results with symbolic links Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]); Sat, 30 Sep 2017 14:53:04 +0200 (CEST) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: ; Sender-helo: mh0.gentlemail.de; ) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Sep 2017 12:53:08 -0000 Hello, utilizing find(1)'s 'newer' primary expression is broken with symbolic links (for a very long time). Anyone who is using find for timestamp comparings should pay special attention regarding symbolic links. The man page states for "-P" (which ist the default), that »the file information and file type (see stat(2)) returned for each symbolic link to be those of the link itself« That's not the case, -P doesn't work as expected, see https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=222698 for details. Hope someone with better C skills will find the root of the problem as soon as possible. I'm trying to find myself, but I'm happyly proven not to be the fastest ;-) -harry From owner-freebsd-stable@freebsd.org Sat Sep 30 16:30:34 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 933BDE2A4DD for ; Sat, 30 Sep 2017 16:30:34 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 31E45741E3 for ; Sat, 30 Sep 2017 16:30:33 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (mh0.gentlemail.de [IPv6:2a00:e10:2800::a135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id v8UGUVpo053249 for ; Sat, 30 Sep 2017 18:30:31 +0200 (CEST) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (s1.omnilan.de [217.91.127.234]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id 680CBBFF; Sat, 30 Sep 2017 18:30:31 +0200 (CEST) Message-ID: <59CFC6A6.6030600@omnilan.de> Date: Sat, 30 Sep 2017 18:30:30 +0200 From: Harry Schmalzbauer Organization: OmniLAN User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; de-DE; rv:1.9.2.8) Gecko/20100906 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: freebsd-stable@freebsd.org Subject: panic: Solaris(panic): blkptr invalid CHECKSUM1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]); Sat, 30 Sep 2017 18:30:31 +0200 (CEST) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: ; Sender-helo: mh0.gentlemail.de; ) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Sep 2017 16:30:34 -0000 Bad surprise. Most likely I forgot to stop a PCIe-Passthrough NIC before shutting down that (byhve(8)) guest – jhb@ helped my identifying this as the root cause for sever memory corruptions I regularly had (on stable-11). Now this time, corruption affected ZFS's RAM area, obviously. What I haven't expected is the panic. The machine has memory disk as root, so luckily I still can boot (from ZFS, –> mdpreload rootfs) into single user mode, but early rc stage (most likely mounting ZFS datasets) leads to the following panic: Trying to mount root from ufs:/dev/ufs/cetusROOT []... panic: Solaris(panic): blkptr at 0xfffffe0005b6b000 has invalid CHECKSUM 1 cpuid = 1 KDB: stack backtrace: #0 0xffffffff805e3837 at kdb_backtrace+0x67 #1 0xffffffff805a2286 at vpanic+0x186 #2 0xffffffff805a20f3 at panic+0x43 #3 0xffffffff81570192 at vcmn_err+0xc2 #4 0xffffffff812d7dda at zfs_panic_recover+0x5a #5 0xffffffff812ff49b at zfs_blkptr_verify+0x8b #6 0xffffffff812ff72c at zio_read+0x2c #7 0xffffffff812761de at arc_read+0x6de #8 0xffffffff81298b4d at traverse_prefetch_metadata+0xbd #9 0xffffffff812980ed at traverse_visitbp+0x39d #10 0xffffffff81298c27 at traverse_dnode+0xc7 #11 0xffffffff812984a3 at traverse_visitbp+0x753 #12 0xffffffff8129788b at traverse_impl+0x22b #13 0xffffffff81297afc at traverse_pool+0x5c #14 0xffffffff812cce06 at spa_load+0x1c06 #15 0xffffffff812cc302 at spa_load+0x1102 #16 0xffffffff812cac6e at spa_load_best+0x6e #17 0xffffffff812c73a1 at spa_open_common+0x101 Uptime: 37s Dumping 1082 out of 15733 MB:..2%..… Dump complete mps0: Sending StopUnit: path (xpt0:mps0:0:2:ffffffff): handle 12 mps0: Incrementing SSU count … Haven't done any scrub attempts yet – expectation is to get all datasets of the striped mirror pool back... Any hints highly appreciated. -harry From owner-freebsd-stable@freebsd.org Sat Sep 30 17:25:17 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1DD93E2B60F for ; Sat, 30 Sep 2017 17:25:17 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A76FF758D6 for ; Sat, 30 Sep 2017 17:25:16 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (ezra.dcm1.omnilan.net [IPv6:2a00:e10:2800::a135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id v8UHPEhR053652 for ; Sat, 30 Sep 2017 19:25:14 +0200 (CEST) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (s1.omnilan.de [217.91.127.234]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id 74A7FC0B; Sat, 30 Sep 2017 19:25:14 +0200 (CEST) Message-ID: <59CFD37A.8080009@omnilan.de> Date: Sat, 30 Sep 2017 19:25:14 +0200 From: Harry Schmalzbauer Organization: OmniLAN User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; de-DE; rv:1.9.2.8) Gecko/20100906 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: freebsd-stable@freebsd.org Subject: Re: panic: Solaris(panic): blkptr invalid CHECKSUM1 References: <59CFC6A6.6030600@omnilan.de> In-Reply-To: <59CFC6A6.6030600@omnilan.de> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]); Sat, 30 Sep 2017 19:25:14 +0200 (CEST) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: ; Sender-helo: mh0.gentlemail.de; ) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Sep 2017 17:25:17 -0000 Bezüglich Harry Schmalzbauer's Nachricht vom 30.09.2017 18:30 (localtime): > Bad surprise. > Most likely I forgot to stop a PCIe-Passthrough NIC before shutting down > that (byhve(8)) guest – jhb@ helped my identifying this as the root > cause for sever memory corruptions I regularly had (on stable-11). > > Now this time, corruption affected ZFS's RAM area, obviously. > > What I haven't expected is the panic. > The machine has memory disk as root, so luckily I still can boot (from > ZFS, –> mdpreload rootfs) into single user mode, but early rc stage > (most likely mounting ZFS datasets) leads to the following panic: > > Trying to mount root from ufs:/dev/ufs/cetusROOT []... > panic: Solaris(panic): blkptr at 0xfffffe0005b6b000 has invalid CHECKSUM 1 > cpuid = 1 > KDB: stack backtrace: > #0 0xffffffff805e3837 at kdb_backtrace+0x67 > #1 0xffffffff805a2286 at vpanic+0x186 > #2 0xffffffff805a20f3 at panic+0x43 > #3 0xffffffff81570192 at vcmn_err+0xc2 > #4 0xffffffff812d7dda at zfs_panic_recover+0x5a > #5 0xffffffff812ff49b at zfs_blkptr_verify+0x8b > #6 0xffffffff812ff72c at zio_read+0x2c > #7 0xffffffff812761de at arc_read+0x6de > #8 0xffffffff81298b4d at traverse_prefetch_metadata+0xbd > #9 0xffffffff812980ed at traverse_visitbp+0x39d > #10 0xffffffff81298c27 at traverse_dnode+0xc7 > #11 0xffffffff812984a3 at traverse_visitbp+0x753 > #12 0xffffffff8129788b at traverse_impl+0x22b > #13 0xffffffff81297afc at traverse_pool+0x5c > #14 0xffffffff812cce06 at spa_load+0x1c06 > #15 0xffffffff812cc302 at spa_load+0x1102 > #16 0xffffffff812cac6e at spa_load_best+0x6e > #17 0xffffffff812c73a1 at spa_open_common+0x101 > Uptime: 37s > Dumping 1082 out of 15733 MB:..2%..… > Dump complete > mps0: Sending StopUnit: path (xpt0:mps0:0:2:ffffffff): handle 12 > mps0: Incrementing SSU count > … > > Haven't done any scrub attempts yet – expectation is to get all datasets > of the striped mirror pool back... > > Any hints highly appreciated. Now it seems I'm in really big trouble. Regular import doesn't work (also not if booted from cd9660). I get all pools listed, but trying to import (unmounted) leads to the same panic as initialy reported – because rc is just doning the same. I booted into single user mode (which works since the bootpool isn't affected and root is a memory disk from the bootpool) and set vfs.zfs.recover=1. But this time I don't even get the list of pools to import 'zpool' import instantaniously leads to that panic: Solaris: WARNING: blkptr at 0xfffffe0005a8e000 has invalid CHECKSUM 1 Solaris: WARNING: blkptr at 0xfffffe0005a8e000 has invalid COMPRESS 0 Solaris: WARNING: blkptr at 0xfffffe0005a8e000 DVA 0 has invalid VDEV 2337865727 Solaris: WARNING: blkptr at 0xfffffe0005a8e000 DVA 1 has invalid VDEV 289407040 Solaris: WARNING: blkptr at 0xfffffe0005a8e000 DVA 2 has invalid VDEV 3959586324 Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x50 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff812de904 stack pointer = 0x28:0xfffffe043f6bcbc0 frame pointer = 0x28:0xfffffe043f6bcbc0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 44 (zpool) trap number = 12 panic: page fault cpuid = 0 KDB: stack backtrace: #0 0xffffffff805e3837 at kdb_backtrace+0x67 #1 0xffffffff805a2286 at vpanic+0x186 #2 0xffffffff805a20f3 at panic+0x43 #3 0xffffffff808a4922 at trap_fatal+0x322 #4 0xffffffff808a4979 at trap_pfault+0x49 #5 0xffffffff808a41f8 at trap+0x298 #6 0xffffffff80889fb1 at calltrap+0x8 #7 0xffffffff812e58a3 at vdev_mirror_child_select+0x53 #8 0xffffffff812e535e at vdev_mirror_io_start+0x2ee #9 0xffffffff81303aa1 at zio_vdev_io_start+0x161 #10 0xffffffff8130054c at zio_execute+0xac #11 0xffffffff812ffe7b at zio_nowait+0xcb #12 0xffffffff812761f3 at arc_read+0x6f3 #13 0xffffffff81298b4d at traverse_prefetch_metadata+0xbd #14 0xffffffff812980ed at traverse_visitbp+0x39d #15 0xffffffff81298c27 at traverse_dnode+0xc7 #16 0xffffffff812984a3 at traverse_visitbp+0x753 #17 0xffffffff8129788b at traverse_impl+0x22b Now I hope any ZFS guru can help me out. Needless to mention that the bits on this mirrored pool are important for me – no productive data, but lots of intermediate... Thanks, -harry From owner-freebsd-stable@freebsd.org Sat Sep 30 21:38:49 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 266A1E2FDA1 for ; Sat, 30 Sep 2017 21:38:49 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 975FF81AEF for ; Sat, 30 Sep 2017 21:38:47 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (mh0.gentlemail.de [78.138.80.135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id v8ULckOA055699 for ; Sat, 30 Sep 2017 23:38:46 +0200 (CEST) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (s1.omnilan.de [217.91.127.234]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id BDB3FC51; Sat, 30 Sep 2017 23:38:45 +0200 (CEST) Message-ID: <59D00EE5.7090701@omnilan.de> Date: Sat, 30 Sep 2017 23:38:45 +0200 From: Harry Schmalzbauer Organization: OmniLAN User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; de-DE; rv:1.9.2.8) Gecko/20100906 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: freebsd-stable@freebsd.org Subject: Re: panic: Solaris(panic): blkptr invalid CHECKSUM1 References: <59CFC6A6.6030600@omnilan.de> <59CFD37A.8080009@omnilan.de> In-Reply-To: <59CFD37A.8080009@omnilan.de> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Greylist: ACL 129 matched, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [78.138.80.130]); Sat, 30 Sep 2017 23:38:46 +0200 (CEST) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: 78.138.80.135; Sender-helo: mh0.gentlemail.de; ) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Sep 2017 21:38:49 -0000 Bezüglich Harry Schmalzbauer's Nachricht vom 30.09.2017 19:25 (localtime): > Bezüglich Harry Schmalzbauer's Nachricht vom 30.09.2017 18:30 (localtime): >> Bad surprise. >> Most likely I forgot to stop a PCIe-Passthrough NIC before shutting down >> that (byhve(8)) guest – jhb@ helped my identifying this as the root >> cause for sever memory corruptions I regularly had (on stable-11). >> >> Now this time, corruption affected ZFS's RAM area, obviously. >> >> What I haven't expected is the panic. >> The machine has memory disk as root, so luckily I still can boot (from >> ZFS, –> mdpreload rootfs) into single user mode, but early rc stage >> (most likely mounting ZFS datasets) leads to the following panic: >> >> Trying to mount root from ufs:/dev/ufs/cetusROOT []... >> panic: Solaris(panic): blkptr at 0xfffffe0005b6b000 has invalid CHECKSUM 1 >> cpuid = 1 >> KDB: stack backtrace: >> #0 0xffffffff805e3837 at kdb_backtrace+0x67 >> #1 0xffffffff805a2286 at vpanic+0x186 >> #2 0xffffffff805a20f3 at panic+0x43 >> #3 0xffffffff81570192 at vcmn_err+0xc2 >> #4 0xffffffff812d7dda at zfs_panic_recover+0x5a >> #5 0xffffffff812ff49b at zfs_blkptr_verify+0x8b >> #6 0xffffffff812ff72c at zio_read+0x2c >> #7 0xffffffff812761de at arc_read+0x6de >> #8 0xffffffff81298b4d at traverse_prefetch_metadata+0xbd >> #9 0xffffffff812980ed at traverse_visitbp+0x39d >> #10 0xffffffff81298c27 at traverse_dnode+0xc7 >> #11 0xffffffff812984a3 at traverse_visitbp+0x753 >> #12 0xffffffff8129788b at traverse_impl+0x22b >> #13 0xffffffff81297afc at traverse_pool+0x5c >> #14 0xffffffff812cce06 at spa_load+0x1c06 >> #15 0xffffffff812cc302 at spa_load+0x1102 >> #16 0xffffffff812cac6e at spa_load_best+0x6e >> #17 0xffffffff812c73a1 at spa_open_common+0x101 >> Uptime: 37s >> Dumping 1082 out of 15733 MB:..2%..… >> Dump complete >> mps0: Sending StopUnit: path (xpt0:mps0:0:2:ffffffff): handle 12 >> mps0: Incrementing SSU count >> … >> >> Haven't done any scrub attempts yet – expectation is to get all datasets >> of the striped mirror pool back... >> >> Any hints highly appreciated. > Now it seems I'm in really big trouble. > Regular import doesn't work (also not if booted from cd9660). > I get all pools listed, but trying to import (unmounted) leads to the > same panic as initialy reported – because rc is just doning the same. > > I booted into single user mode (which works since the bootpool isn't > affected and root is a memory disk from the bootpool) > and set vfs.zfs.recover=1. > But this time I don't even get the list of pools to import 'zpool' > import instantaniously leads to that panic: > > Solaris: WARNING: blkptr at 0xfffffe0005a8e000 has invalid CHECKSUM 1 > Solaris: WARNING: blkptr at 0xfffffe0005a8e000 has invalid COMPRESS 0 > Solaris: WARNING: blkptr at 0xfffffe0005a8e000 DVA 0 has invalid VDEV > 2337865727 > Solaris: WARNING: blkptr at 0xfffffe0005a8e000 DVA 1 has invalid VDEV > 289407040 > Solaris: WARNING: blkptr at 0xfffffe0005a8e000 DVA 2 has invalid VDEV > 3959586324 > > > Fatal trap 12: page fault while in kernel mode > cpuid = 0; apic id = 00 > fault virtual address = 0x50 > fault code = supervisor read data, page not present > instruction pointer = 0x20:0xffffffff812de904 > stack pointer = 0x28:0xfffffe043f6bcbc0 > frame pointer = 0x28:0xfffffe043f6bcbc0 > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags = interrupt enabled, resume, IOPL = 0 > current process = 44 (zpool) > trap number = 12 > panic: page fault > cpuid = 0 … OpenIndiana also panics at regular import. Unfortunately I don't know the aequivalent of vfs.zfs.recover in OI. panic[cpu1]/thread=ffffff06dafe8be0: blkptr at ffffff06dbe63000 has invalid CHECKSUM 1 Warning - stack not written to the dump buffer ffffff001f67f070 genunix:vcmn_err+42 () ffffff001f67f0e0 zfs:zfs_panic_recover+51 () ffffff001f67f140 zfs:zfs_blkptr_verify+8d () ffffff001f67f220 zfs:zio_read+55 () ffffff001f67f310 zfs:arc_read+662 () ffffff001f67f370 zfs:traverse_prefetch_metadata+b5 () ffffff001f67f450 zfs:traverse_visitbp+1c3 () ffffff001f67f4e0 zfs:traverse_dnode+af () ffffff001f67f5c0 zfs:traverse_visitbp+6dd () ffffff001f67f720 zfs:traverse_impl+1a6 () ffffff001f67f830 zfs:traverse_pool+9f () ffffff001f67f8a0 zfs:spa_load_verify+1e6 () ffffff001f67f990 zfs:spa_load_impl+e1c () ffffff001f67fa30 zfs:spa_load+14e () ffffff001f67fad0 zfs:spa_load_best+7a () ffffff001f67fb90 zfs:spa_import+1b0 () ffffff001f67fbe0 zfs:zfs_ioc_pool_import+10f () ffffff001f67fc80 zfs:zfsdev_ioctl+4b7 () ffffff001f67fcc0 genunix:cdev_ioctl+39 () ffffff001f67fd10 specfs:spec_ioctl+60 () ffffff001f67fda0 genunix:fop_ioctl+55 () ffffff001f67fec0 genunix:ioctl+9b () ffffff001f67ff10 unix:brand_sys_sysenter+1c9 () This is a important lesson. My impression was that it's not possible to corrupt a complete pool, but there's always a way to recover healthy/redundant data. Now my striped mirror has all 4 devices healthy available, but all datasets seem to be lost. No problem for 450G (99,9_%), but there's a 80M dataset which I'm really missing :-( Unfortunately I don't know the DVA and blkptr internals, so I won't write a zfs_fsck(8) soon ;-) Does it make sense to dump the disks for further analysis? I need to recreate the pool because I need the machine's resources... :-( Any help highly appreciated! -harry