From owner-freebsd-stable@freebsd.org Sun Aug 16 07:10:56 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1C84F9B9DD0 for ; Sun, 16 Aug 2015 07:10:56 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from smtp.infracaninophile.co.uk (smtp.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3cd3:cd67:fafa:3d78]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.infracaninophile.co.uk", Issuer "infracaninophile.co.uk" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B062616A6 for ; Sun, 16 Aug 2015 07:10:55 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from liminal.local ([192.168.100.2]) (authenticated bits=0) by smtp.infracaninophile.co.uk (8.15.2/8.15.2) with ESMTPSA id t7G7AniV004248 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 16 Aug 2015 08:10:50 +0100 (BST) (envelope-from matthew@FreeBSD.org) Authentication-Results: smtp.infracaninophile.co.uk; dmarc=none header.from=FreeBSD.org DKIM-Filter: OpenDKIM Filter v2.10.3 smtp.infracaninophile.co.uk t7G7AniV004248 Authentication-Results: smtp.infracaninophile.co.uk/t7G7AniV004248; dkim=none; dkim-atps=neutral X-Authentication-Warning: lucid-nonsense.infracaninophile.co.uk: Host [192.168.100.2] claimed to be liminal.local Subject: Re: 10.2: ntp update breaks DCF77 clock To: freebsd-stable@freebsd.org References: From: Matthew Seaman X-Enigmail-Draft-Status: N1110 Message-ID: <55D03771.9000605@FreeBSD.org> Date: Sun, 16 Aug 2015 08:10:41 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="D4rco389VCFWDJxRvUfDkph6wRknH9mLL" X-Virus-Scanned: clamav-milter 0.98.7 at lucid-nonsense.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on lucid-nonsense.infracaninophile.co.uk X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Aug 2015 07:10:56 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --D4rco389VCFWDJxRvUfDkph6wRknH9mLL Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 15/08/2015 16:46, Christian Weisgerber wrote: > The ntp code is not very transparent, but I think the root cause > are the ntp/config.h changes that came with the 4.2.8p3 update. A > number of previously disabled obscure clock drivers were enabled, > but crucially CLOCK_RAWDCF was disabled, and this is the PARSE > subdriver needed to use the popular DCF77 serial receivers. >=20 > Frankly, it looks like we used to have a carefully considered > selection of clock drivers which has been blindly splattered with > the upstream defaults in the last update. Hmmm.... I suggest raising a PR with patches to revert the changes in the set of enabled clock drivers (or merge with the current list). It's not going to get you a working DCF77 receiver in a -RELEASE version any time soon, I'm afraid, as you'll have to wait until the next release for the changes to percolate down, but having a sensible list of enabled clock drivers in base is definitely a good move. For a more timely solution[*], it looks like the ports is your best option. By default the net/ntp port disables all of the clock drivers, but allows you to configure the port to enable whatever drivers you want. If you built your own package it would be simple to get the right support compiled in. However, that won't help if you're determined to use pre-built packages only, in which case there would need to be a slave port with enabled clock drivers. That's something you could certainly argue for; This is a symptom of the current state of the ports tree -- we've switched over to pkg(8), but we're still working through a lot of changes to fully enable pkg capabilities. A lot of the functionality still only really works if you build your own ports. There are changes planned, like sub-packages and package flavours which should help, but in the case of net/ntp where clock drivers are compiled into the main binary unfortunately those won't apply. If NTP clock drivers were implemented as loadable modules it would be a lot easier... Cheers, Matthew [*] Pun unintentional. --D4rco389VCFWDJxRvUfDkph6wRknH9mLL Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.20 (Darwin) iQJ8BAEBCgBmBQJV0Dd4XxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ2NTNBNjhCOTEzQTRFNkNGM0UxRTEzMjZC QjIzQUY1MThFMUE0MDEzAAoJELsjr1GOGkAT1bcP/2KEvDKrEAFN/viTlvPFysFB awox39wLxuR9Goil8EXMOI61+vFV88TKna6M2KbwX+KiyBJiFxMvJxrAe4wps0TV faX1lYEvN/INTXyqBqe0xd1VNIWtUomLt17ikgd7hPf26L/RA5S7UMeN22A2yS9j VFt0Z/pGnRmuMyWXspX03s+rn/UyTd9F/XxupcLcudjDIDy4fZ3KoggB0m1yWQ9e HipaJcVrOa0xHTpIQX2DU0mLtm622uNeUvb+V5amgp8emRMng+eMnu9QTNc/q8Do M5XcjyU+jHzkyVJ+a1ITFkghIQvQ1dGXTpsKKVeqt8SNErpU0JbjLpXwNGPse0q/ vCJLYuzClADhbBTICBPCSvmuNajm8SKDIMEm5JaKeTa4Ls+mgGaKlA0+zO2TQd/+ Ijm8BKhfipgcl/058CN1bs9yx6cTWYydiqJIJ0gBZPr/ob+Imcuoms83GRJAEWMK p40Dj+XestMZYhHCu4/EHn2RNHOUmkt2gDixkpH/Md4jbZHqP0do9DH0WF4J1xBd 2zYQHQbzM5ajPBOHposcQ+FIrEOQKRh2ktcsMtQyqobzKs8+knfvfi/aQTU/OJ7H 30uexP9eTWn3V5YsiyKaEeQKuc7hYLvufiO6hwzQluZ0KVWy0wKF67ocdY/IBea+ dzfmjSdpnY7r5V4fyi+r =17Iy -----END PGP SIGNATURE----- --D4rco389VCFWDJxRvUfDkph6wRknH9mLL-- From owner-freebsd-stable@freebsd.org Sun Aug 16 10:40:57 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B8EA99BAC08 for ; Sun, 16 Aug 2015 10:40:57 +0000 (UTC) (envelope-from florian.ermisch@alumni.tu-berlin.de) Received: from mail-2.alumni.tu-berlin.de (mail-2.alumni.tu-berlin.de [130.149.5.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7B6071144 for ; Sun, 16 Aug 2015 10:40:56 +0000 (UTC) (envelope-from florian.ermisch@alumni.tu-berlin.de) X-tubIT-Incoming-IP: 78.52.166.150 Received: from f052166150.adsl.alicedsl.de ([78.52.166.150] helo=android-a12318b22af) by mailbox.alumni.tu-berlin.de (exim-4.76) with esmtpsa [UNKNOWN:AES256-SHA:256] for id 1ZQvMu-0001jO-2X; Sun, 16 Aug 2015 12:40:48 +0200 User-Agent: K-9 Mail for Android In-Reply-To: <55D03771.9000605@FreeBSD.org> References: <55D03771.9000605@FreeBSD.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8 Subject: Re: 10.2: ntp update breaks DCF77 clock From: Florian Ermisch Date: Sun, 16 Aug 2015 12:40:40 +0200 To: freebsd-stable@freebsd.org Message-ID: <70807946-3B0F-4D8B-B496-F032F76C6F43@alumni.tu-berlin.de> X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Aug 2015 10:40:57 -0000 Am 16. August 2015 09:10:41 MESZ, schrieb Matthew Seaman : >On 15/08/2015 16:46, Christian Weisgerber wrote: >> The ntp code is not very transparent, but I think the root cause >> are the ntp/config.h changes that came with the 4.2.8p3 update. A >> number of previously disabled obscure clock drivers were enabled, >> but crucially CLOCK_RAWDCF was disabled, and this is the PARSE >> subdriver needed to use the popular DCF77 serial receivers. >> […] > >[…] >For a more timely solution[*], it looks like the ports is your best >option. By default the net/ntp port disables all of the clock drivers, >but allows you to configure the port to enable whatever drivers you >want. >[…] You could also check the PCBSD/TrueOS pkg repos. I Kris Moore mentioned on BSDnow he'll happily enable/change build options when people need them. When ntpd is the only pkg you'll install anyway this might be the most hassle free option. Regards, Florian From owner-freebsd-stable@freebsd.org Sun Aug 16 12:08:47 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2E9729B9015 for ; Sun, 16 Aug 2015 12:08:47 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-pa0-x229.google.com (mail-pa0-x229.google.com [IPv6:2607:f8b0:400e:c03::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F403316AF for ; Sun, 16 Aug 2015 12:08:46 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: by pacgr6 with SMTP id gr6so88950649pac.2 for ; Sun, 16 Aug 2015 05:08:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=8WyMS0+bTOQ8m57RntKUGpXFmn5QMNiSelYq9sOy140=; b=HSqkpv2Z52tweSiqhwY4O+ajZeLUfhUV9bBrlT5ZFsk1batYpn2rdr7RqXdyAM8gcw /3OoM/8szXoyk/pbW/P/PkJgpR+Mj/j34JZGzohP1UeeNbZ6wWPpAkuNHjPqrDGWJK1V psIMmi8MDK4swm9AEaxvD7Bo5Fsj23cSVitLXvrCariGdcVR6DIhyNBWuqGCwO1V8jCD LFpUOGP+eTWh64tDhvHCvIanTY7C7HwYd587UZi9jalesLOVb6pdt4139T1We1NE7Mzh 4Xnc3XoMUocUIPnOtxbdVeucfaErtsjnsgkznWkLwbilejM+VxbODi381OhVrGqdFatl vrgg== X-Received: by 10.66.183.164 with SMTP id en4mr93937228pac.24.1439726926683; Sun, 16 Aug 2015 05:08:46 -0700 (PDT) Received: from pyunyh@gmail.com ([106.247.248.2]) by smtp.gmail.com with ESMTPSA id kx11sm11320582pab.13.2015.08.16.05.08.37 (version=TLSv1 cipher=RC4-SHA bits=128/128); Sun, 16 Aug 2015 05:08:40 -0700 (PDT) From: Yonghyeon PYUN X-Google-Original-From: "Yonghyeon PYUN" Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Sun, 16 Aug 2015 21:08:31 +0900 Date: Sun, 16 Aug 2015 21:08:31 +0900 To: Roosevelt Littleton Cc: freebsd-stable@freebsd.org Subject: Re: msk msk0 watchdog timeout freeze hang lock stop problem Message-ID: <20150816120831.GB1288@michelle.fasterthan.com> Reply-To: pyunyh@gmail.com References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Aug 2015 12:08:47 -0000 On Wed, Aug 12, 2015 at 09:44:06AM -0400, Roosevelt Littleton wrote: > Hi, > So, I can confirm with the attached patch. I have a working msk0 that > hasn't failed for the past month. I considered this problem fix for me. > Since, I have went a long time without any problems. Thanks! I'm not sure which patch you used. Given that users reported 10.2-RELEASE works, it would be great if you revert local patch and try it again on 10.2-RELEASE. From owner-freebsd-stable@freebsd.org Sun Aug 16 12:52:05 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D49E69B98BC for ; Sun, 16 Aug 2015 12:52:05 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Received: from smarthost1.greenhost.nl (smarthost1.greenhost.nl [195.190.28.81]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9A550115F for ; Sun, 16 Aug 2015 12:52:05 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Received: from smtp.greenhost.nl ([213.108.104.138]) by smarthost1.greenhost.nl with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.72) (envelope-from ) id 1ZQxPi-0006Y4-QH for freebsd-stable@freebsd.org; Sun, 16 Aug 2015 14:51:56 +0200 Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes To: freebsd-stable@freebsd.org Subject: Re: Swap Questions References: <55CDE9EE.5090603@tundraware.com> <55CF8E0B.7050608@tundraware.com> Date: Sun, 16 Aug 2015 14:51:49 +0200 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: "Ronald Klop" Message-ID: In-Reply-To: <55CF8E0B.7050608@tundraware.com> User-Agent: Opera Mail/12.16 (FreeBSD) X-Authenticated-As-Hash: 398f5522cb258ce43cb679602f8cfe8b62a256d1 X-Virus-Scanned: by clamav at smarthost1.samage.net X-Spam-Level: / X-Spam-Score: -0.2 X-Spam-Status: No, score=-0.2 required=5.0 tests=ALL_TRUSTED, BAYES_50, URIBL_BLOCKED autolearn=disabled version=3.3.1 X-Scan-Signature: 3ced5df4177ef3a93a84b902ce7c160e X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Aug 2015 12:52:05 -0000 On Sat, 15 Aug 2015 21:07:55 +0200, Tim Daneliuk wrote: > On 08/14/2015 12:39 PM, Warren Block wrote: >> On Fri, 14 Aug 2015, Tim Daneliuk wrote: >> >>> I just built a 10.2 machine on a cloud-based VPS (Digital Ocean) that >>> has >>> 512M of memory and 1G of swap partition. I am seeing a ton of errors >>> like >>> this: >>> >>> Aug 14 00:01:22 myhost kernel: swap_pager_getswapspace(10): failed >>> Aug 14 00:01:22 myhost kernel: swap_pager_getswapspace(14): failed >>> Aug 14 00:01:22 myhost kernel: swap_pager_getswapspace(11): failed >>> Aug 14 00:01:22 myhost kernel: swap_pager_getswapspace(6): failed >>> Aug 14 00:01:22 myhost kernel: swap_pager_getswapspace(7): failed >>> Aug 14 00:01:22 myhost last message repeated 2 times >>> >>> >>> So, I added this to fstab (after creating /usr/swap0): >>> >>> md99 none swap sw,file=/usr/swap0 0 0 >>> >>> And then did this: >>> >>> swapon -aq >>> >>> >>> But, when I do a swapinfo, all I can see is the "disk" swap partition >>> that comes standard with the VPS: >>> >>> >>> Device 1K-blocks Used Avail Capacity >>> /dev/gpt/swapfs 1048576 456572 592004 44% >> >> Add the -L (late) option to swapon. How this works might differ >> between 10-Release, 10-Stable, and 11. >> >> Incidentally, md99 does not have to be literal, it's just meant to get >> the md device number up out of the way of common interactive usage of >> mdconfig. > > > So -L does fix the problem - sort of. The machine picks up the file as > additional swap on boot just fine. HWOEVER, when I try to reboot or shut > down the host, I get a panic telling me some noise about not being able > to shutdown swap for some reason. It helps if you provide the exact text of the panic. People regularly don't get to see these inside there crystal ball. ;-) You call it noise. Others might get an helpful hint from it to help you. Regards, Ronald. > So ... I decided to just add a second disk partition for swap and - > for some reason - it works fine interactively, but upon reboot, > the newly created swap partition no longer exists and gpart shows > the space as free again. I tried a gpart commit, but get > "operation not permitted". So now I am trying to figure out > how to make gpart changes stick. This may be an artifact of > the way Digital Ocean droplets are set up .... > > Grrrrrrrr From owner-freebsd-stable@freebsd.org Sun Aug 16 13:02:39 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C4E749B9B10 for ; Sun, 16 Aug 2015 13:02:39 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Received: from smarthost1.greenhost.nl (smarthost1.greenhost.nl [195.190.28.81]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 898E8186E for ; Sun, 16 Aug 2015 13:02:38 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Received: from smtp.greenhost.nl ([213.108.104.138]) by smarthost1.greenhost.nl with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.72) (envelope-from ) id 1ZQxa3-0007yd-HA for freebsd-stable@freebsd.org; Sun, 16 Aug 2015 15:02:37 +0200 Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes To: freebsd-stable@freebsd.org Subject: Re: Swap Questions References: <55CDE9EE.5090603@tundraware.com> <55CF8E0B.7050608@tundraware.com> Date: Sun, 16 Aug 2015 15:02:30 +0200 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: "Ronald Klop" Message-ID: In-Reply-To: User-Agent: Opera Mail/12.16 (FreeBSD) X-Authenticated-As-Hash: 398f5522cb258ce43cb679602f8cfe8b62a256d1 X-Virus-Scanned: by clamav at smarthost1.samage.net X-Spam-Level: / X-Spam-Score: -0.2 X-Spam-Status: No, score=-0.2 required=5.0 tests=ALL_TRUSTED, BAYES_50, URIBL_BLOCKED autolearn=disabled version=3.3.2 X-Scan-Signature: 98cd051f671ee36aeaf0d6c34a549736 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Aug 2015 13:02:40 -0000 On Sun, 16 Aug 2015 14:51:49 +0200, Ronald Klop wrote: > On Sat, 15 Aug 2015 21:07:55 +0200, Tim Daneliuk > wrote: > >> On 08/14/2015 12:39 PM, Warren Block wrote: >>> On Fri, 14 Aug 2015, Tim Daneliuk wrote: >>> >>>> I just built a 10.2 machine on a cloud-based VPS (Digital Ocean) that >>>> has >>>> 512M of memory and 1G of swap partition. I am seeing a ton of errors >>>> like >>>> this: >>>> >>>> Aug 14 00:01:22 myhost kernel: swap_pager_getswapspace(10): failed >>>> Aug 14 00:01:22 myhost kernel: swap_pager_getswapspace(14): failed >>>> Aug 14 00:01:22 myhost kernel: swap_pager_getswapspace(11): failed >>>> Aug 14 00:01:22 myhost kernel: swap_pager_getswapspace(6): failed >>>> Aug 14 00:01:22 myhost kernel: swap_pager_getswapspace(7): failed >>>> Aug 14 00:01:22 myhost last message repeated 2 times >>>> >>>> >>>> So, I added this to fstab (after creating /usr/swap0): >>>> >>>> md99 none swap sw,file=/usr/swap0 0 0 >>>> >>>> And then did this: >>>> >>>> swapon -aq >>>> >>>> >>>> But, when I do a swapinfo, all I can see is the "disk" swap partition >>>> that comes standard with the VPS: >>>> >>>> >>>> Device 1K-blocks Used Avail Capacity >>>> /dev/gpt/swapfs 1048576 456572 592004 44% >>> >>> Add the -L (late) option to swapon. How this works might differ >>> between 10-Release, 10-Stable, and 11. >>> >>> Incidentally, md99 does not have to be literal, it's just meant to get >>> the md device number up out of the way of common interactive usage of >>> mdconfig. >> >> >> So -L does fix the problem - sort of. The machine picks up the file as >> additional swap on boot just fine. HWOEVER, when I try to reboot or >> shut >> down the host, I get a panic telling me some noise about not being able >> to shutdown swap for some reason. > > It helps if you provide the exact text of the panic. People regularly > don't get to see these inside there crystal ball. ;-) You call it noise. > Others might get an helpful hint from it to help you. Maybe you already knew, but adding dumpdev="AUTO" in /etc/rc.conf can provide a kernel dump on panic which can be analyzed after reboot. Ronald. > Regards, > Ronald. > > >> So ... I decided to just add a second disk partition for swap and - >> for some reason - it works fine interactively, but upon reboot, >> the newly created swap partition no longer exists and gpart shows >> the space as free again. I tried a gpart commit, but get >> "operation not permitted". So now I am trying to figure out >> how to make gpart changes stick. This may be an artifact of >> the way Digital Ocean droplets are set up .... >> >> Grrrrrrrr > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Sun Aug 16 16:57:09 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 06FAF9BBDF2 for ; Sun, 16 Aug 2015 16:57:09 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from pmta2.delivery6.ore.mailhop.org (pmta2.delivery6.ore.mailhop.org [54.200.129.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D1036179B for ; Sun, 16 Aug 2015 16:57:08 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from ilsoft.org (unknown [73.34.117.227]) by outbound2.ore.mailhop.org (Halon Mail Gateway) with ESMTPSA; Sun, 16 Aug 2015 16:58:25 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.14.9/8.14.9) with ESMTP id t7GGv0Bc009167; Sun, 16 Aug 2015 10:57:00 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: <1439744220.242.87.camel@freebsd.org> Subject: Re: 10.2: ntp update breaks DCF77 clock From: Ian Lepore To: Matthew Seaman Cc: freebsd-stable@freebsd.org, Cy Schubert Date: Sun, 16 Aug 2015 10:57:00 -0600 In-Reply-To: <55D03771.9000605@FreeBSD.org> References: <55D03771.9000605@FreeBSD.org> X-Mailer: Evolution 3.12.10 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Aug 2015 16:57:09 -0000 On Sun, 2015-08-16 at 08:10 +0100, Matthew Seaman wrote: > On 15/08/2015 16:46, Christian Weisgerber wrote: > > The ntp code is not very transparent, but I think the root cause > > are the ntp/config.h changes that came with the 4.2.8p3 update. A > > number of previously disabled obscure clock drivers were enabled, > > but crucially CLOCK_RAWDCF was disabled, and this is the PARSE > > subdriver needed to use the popular DCF77 serial receivers. > > > > Frankly, it looks like we used to have a carefully considered > > selection of clock drivers which has been blindly splattered with > > the upstream defaults in the last update. > > Hmmm.... I suggest raising a PR with patches to revert the changes in > the set of enabled clock drivers (or merge with the current list). It's > not going to get you a working DCF77 receiver in a -RELEASE version any > time soon, I'm afraid, as you'll have to wait until the next release for > the changes to percolate down, but having a sensible list of enabled > clock drivers in base is definitely a good move. > I wonder: is there a reason to not enable all (or most of) the refclocks in base and in ports? Well, at least all the ones that build on freebsd... a disturbing number of them fail to compile because they include linux-specific header files. Hmm, I just noticed that we actually compile most of the refclocks, but we don't enable them in the config. It looks like the cost of enabling all the refclocks that compile properly is pretty small... doing so increased the size of ntpd from 745K to 801K for me. I'll attach the diff just to save someone else the trouble of iteratively figuring out which ones won't build, but I think there may be a more-proper way to generate this config by tweaking the autotools stuff. -- Ian From owner-freebsd-stable@freebsd.org Sun Aug 16 17:16:11 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E4DBC9BB14A for ; Sun, 16 Aug 2015 17:16:10 +0000 (UTC) (envelope-from ck-lists@cksoft.de) Received: from mx1.cksoft.de (mx1.cksoft.de [IPv6:2001:67c:24f8:1::25:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.cksoft.de", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 79709180 for ; Sun, 16 Aug 2015 17:16:10 +0000 (UTC) (envelope-from ck-lists@cksoft.de) Received: from m.cksoft.de (asa1.cksoft.de [212.17.240.82]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.cksoft.de (Postfix) with ESMTPSA id CE3601E9E6E for ; Sun, 16 Aug 2015 19:16:06 +0200 (CEST) Received: from amavis.cksoft.de (unknown [IPv6:2a01:170:1110:8001::25:a1]) by m.cksoft.de (Postfix) with ESMTP id 429D262F88 for ; Sun, 16 Aug 2015 19:15:05 +0200 (CEST) X-Virus-Scanned: amavisd-new at cksoft.de Received: from m.cksoft.de ([192.168.64.93]) by amavis.cksoft.de (amavis.cksoft.de [192.168.64.94]) (amavisd-new, port 10041) with ESMTP id 3jhK0Zfim31T for ; Sun, 16 Aug 2015 19:15:03 +0200 (CEST) Received: from noc1.cksoft.de (noc1.cksoft.de [IPv6:2a01:170:1110:8001::53:1]) by m.cksoft.de (Postfix) with ESMTP id D528262ECF for ; Sun, 16 Aug 2015 19:15:02 +0200 (CEST) Received: by noc1.cksoft.de (Postfix, from userid 1000) id EB5FD13BCA; Sun, 16 Aug 2015 19:16:03 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by noc1.cksoft.de (Postfix) with ESMTP id E2F9113B9B for ; Sun, 16 Aug 2015 19:16:03 +0200 (CEST) Date: Sun, 16 Aug 2015 19:16:03 +0200 (CEST) From: Christian Kratzer X-X-Sender: ck@noc1.cksoft.de Reply-To: Christian Kratzer To: freebsd-stable@freebsd.org Subject: freebsd-update to 10.2-RELEASE broken ? Message-ID: User-Agent: Alpine 2.20 (BSF 67 2015-01-07) X-Spammer-Kill-Ratio: 75% MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset=US-ASCII Content-Transfer-Encoding: 8BIT X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Aug 2015 17:16:11 -0000 Hi, I have been trying to update several of my FreeBSD 10.1 amd64 VM to 10.2-RELEASE with freebsd-update and have been failing with an incorrect hash error. This is what happens with a plain vanilla 10.1-RELEASE vm when I try to update to 10.2-RELEASE --snipp-- root@test10:~ck # uname -a FreeBSD test10.cksoft.de 10.1-RELEASE FreeBSD 10.1-RELEASE #0 r274401: Tue Nov 11 21:02:49 UTC 2014 root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 root@test10:~ck # freebsd-update upgrade -r 10.2-RELEASE Looking up update.FreeBSD.org mirrors... none found. Fetching metadata signature for 10.1-RELEASE from update.FreeBSD.org... done. Fetching metadata index... done. Inspecting system... done. The following components of FreeBSD seem to be installed: kernel/generic world/base world/doc world/games world/lib32 The following components of FreeBSD do not seem to be installed: src/src Does this look reasonable (y/n)? y Fetching metadata signature for 10.2-RELEASE from update.FreeBSD.org... done. Fetching metadata index... done. Fetching 1 metadata patches. done. Applying metadata patches... done. Fetching 1 metadata files... done. Inspecting system... done. Fetching files from 10.1-RELEASE for merging... done. Preparing to download files... done. Fetching 11120 patches.....10....20....30....40....50....60....70....80....90....100....110....120....130....140....150....160....170....180....190....200....210....220....230....240....250....260....270....280....290....300....310....320....330....340....350....360....370....380....390....400....410....420....430....440....450....460....470....480....490....500....510....520....530....540....550....560....570....580....590....600....610....620....630....640....650....660....670....680....690....700....710....720....730....740....750....760....770....780....790....800....810....820....830....840....850....860....870....880....890....900....910....920....930....940....950....960....970....980....990....1000....1010....1020....1030....1040....1050....1060....1070....1080....1090....1100....1110....1120....1130....1140....1150....1160....1170....1180....1190....1200....1210....1220....1230....1240....1250....1260....1270....1280....1290....1300....1310....1320....1330....1340....1350... .1360....1370....1380....1390....1400....1410....1420....1430....1440....1450....1460....1470....1480....1490....1500....1510....1520....1530....1540....1550....1560....1570....1580....1590....1600....1610....1620....1630....1640....1650....1660....1670....1680....1690....1700....1710....1720....1730....1740....1750....1760....1770....1780....1790....1800....1810....1820....1830....1840....1850....1860....1870....1880....1890....1900....1910....1920....1930....1940....1950....1960....1970....1980....1990....2000....2010....2020....2030....2040....2050....2060....2070....2080....2090....2100....2110....2120....2130....2140....2150....2160....2170....2180....2190....2200....2210....2220....2230....2240....2250....2260....2270....2280....2290....2300....2310....2320....2330....2340....2350....2360....2370....2380....2390....2400....2410....2420....2430....2440....2450....2460....2470....2480....2490....2500....2510....2520....2530....2540....2550....2560....2570....2580....2590....2600 ....2610....2620....2630....2640....2650....2660....2670....2680....2690....2700....2710....2720....2730....2740....2750....2760....2770....2780....2790....2800....2810....2820....2830....2840....2850....2860....2870....2880....2890....2900....2910....2920....2930....2940....2950....2960....2970....2980....2990....3000....3010....3020....3030....3040....3050....3060....3070....3080....3090....3100....3110....3120....3130....3140....3150....3160....3170....3180....3190....3200....3210....3220....3230....3240....3250....3260....3270....3280....3290....3300....3310....3320....3330....3340....3350....3360....3370....3380....3390....3400....3410....3420....3430....3440....3450....3460....3470....3480....3490....3500....3510....3520....3530....3540....3550....3560....3570....3580....3590....3600....3610....3620....3630....3640....3650....3660....3670....3680....3690....3700....3710....3720....3730....3740....3750....3760....3770....3780....3790....3800....3810....3820....3830....3840....3 850....3860....3870....3880....3890....3900....3910....3920....3930....3940....3950....3960....3970....3980....3990....4000....4010....4020....4030....4040....4050....4060....4070....4080....4090....4100....4110....4120....4130....4140....4150....4160....4170....4180....4190....4200....4210....4220....4230....4240....4250....4260....4270....4280....4290....4300....4310....4320....4330....4340....4350....4360....4370....4380....4390....4400....4410....4420....4430....4440....4450....4460....4470....4480....4490....4500....4510....4520....4530....4540....4550....4560....4570....4580....4590....4600....4610....4620....4630....4640....4650....4660....4670....4680....4690....4700....4710....4720....4730....4740....4750....4760....4770....4780....4790....4800....4810....4820....4830....4840....4850....4860....4870....4880....4890....4900....4910....4920....4930....4940....4950....4960....4970....4980....4990....5000....5010....5020....5030....5040....5050....5060....5070....5080....5090.. ..5100....5110....5120....5130....5140....5150....5160....5170....5180....5190....5200....5210....5220....5230....5240....5250....5260....5270....5280....5290....5300....5310....5320....5330....5340....5350....5360....5370....5380....5390....5400....5410....5420....5430....5440....5450....5460....5470....5480....5490....5500....5510....5520....5530....5540....5550....5560....5570....5580....5590....5600....5610....5620....5630....5640....5650....5660....5670....5680....5690....5700....5710....5720....5730....5740....5750....5760....5770....5780....5790....5800....5810....5820....5830....5840....5850....5860....5870....5880....5890....5900....5910....5920....5930....5940....5950....5960....5970....5980....5990....6000....6010....6020....6030....6040....6050....6060....6070....6080....6090....6100....6110....6120....6130....6140....6150....6160....6170....6180....6190....6200....6210....6220....6230....6240....6250....6260....6270....6280....6290....6300....6310....6320....6330....634 0....6350....6360....6370....6380....6390....6400....6410....6420....6430....6440....6450....6460....6470....6480....6490....6500....6510....6520....6530....6540....6550....6560....6570....6580....6590....6600....6610....6620....6630....6640....6650....6660....6670....6680....6690....6700....6710....6720....6730....6740....6750....6760....6770....6780....6790....6800....6810....6820....6830....6840....6850....6860....6870....6880....6890....6900....6910....6920....6930....6940....6950....6960....6970....6980....6990....7000....7010....7020....7030....7040....7050....7060....7070....7080....7090....7100....7110....7120....7130....7140....7150....7160....7170....7180....7190....7200....7210....7220....7230....7240....7250....7260....7270....7280....7290....7300....7310....7320....7330....7340....7350....7360....7370....7380....7390....7400....7410....7420....7430....7440....7450....7460....7470....7480....7490....7500....7510....7520....7530....7540....7550....7560....7570....7580.... 7590....7600....7610....7620....7630....7640....7650....7660....7670....7680....7690....7700....7710....7720....7730....7740....7750....7760....7770....7780....7790....7800....7810....7820....7830....7840....7850....7860....7870....7880....7890....7900....7910....7920....7930....7940....7950....7960....7970....7980....7990....8000....8010....8020....8030....8040....8050....8060....8070....8080....8090....8100....8110....8120....8130....8140....8150....8160....8170....8180....8190....8200....8210....8220....8230....8240....8250....8260....8270....8280....8290....8300....8310....8320....8330....8340....8350....8360....8370....8380....8390....8400....8410....8420....8430....8440....8450....8460....8470....8480....8490....8500....8510....8520....8530....8540....8550....8560....8570....8580....8590....8600....8610....8620....8630....8640....8650....8660....8670....8680....8690....8700....8710....8720....8730....8740....8750....8760....8770....8780....8790....8800....8810....8820....8830. ...8840....8850....8860....8870....8880....8890....8900....8910....8920....8930....8940....8950....8960....8970....8980....8990....9000....9010....9020....9030....9040....9050....9060....9070....9080....9090....9100....9110....9120....9130....9140....9150....9160....9170....9180....9190....9200....9210....9220....9230....9240....9250....9260....9270....9280....9290....9300....9310....9320....9330....9340....9350....9360....9370....9380....9390....9400....9410....9420....9430....9440....9450....9460....9470....9480....9490....9500....9510....9520....9530....9540....9550....9560....9570....9580....9590....9600....9610....9620....9630....9640....9650....9660....9670....9680....9690....9700....9710....9720....9730....9740....9750....9760....9770....9780....9790....9800....9810....9820....9830....9840....9850....9860....9870....9880....9890....9900....9910....9920....9930....9940....9950....9960....9970....9980....9990....10000....10010....10020....10030....10040....10050....10060....100 70....10080....10090....10100....10110....10120....10130....10140....10150....10160....10170....10180....10190....10200....10210....10220....10230....10240....10250....10260....10270....10280....10290....10300....10310....10320....10330....10340....10350....10360....10370....10380....10390....10400....10410....10420....10430....10440....10450....10460....10470....10480....10490....10500....10510....10520....10530....10540....10550....10560....10570....10580....10590....10600....10610....10620....10630....10640....10650....10660....10670....10680....10690....10700....10710....10720....10730....10740....10750....10760....10770....10780....10790....10800....10810....10820....10830....10840....10850....10860....10870....10880....10890....10900....10910....10920....10930....10940....10950....10960....10970....10980....10990....11000....11010....11020....11030....11040....11050....11060....11070....11080....11090....11100....11110....11120 done. Applying patches... done. Fetching 2356 files... 068eb594e5f6b97554750a8321892e4c229a6f26455f40e4d8e4e7a79577c673 has incorrect hash. root@test10:~ck # --snipp-- I get the is on all kinds of VM with different patchlevels of 10.1-RELEASE. Some of them have /usr/src, some of them don't. The hash seems to be different every time round. Could this be an issue with one of the mirrors ? Has anybody had success yet with an update to 10.2-RELEASE using freebsd-update ? Greetings Christian -- Christian Kratzer CK Software GmbH Email: ck@cksoft.de Wildberger Weg 24/2 Phone: +49 7032 893 997 - 0 D-71126 Gaeufelden Fax: +49 7032 893 997 - 9 HRB 245288, Amtsgericht Stuttgart Mobile: +49 171 1947 843 Geschaeftsfuehrer: Christian Kratzer Web: http://www.cksoft.de/ From owner-freebsd-stable@freebsd.org Sun Aug 16 17:23:47 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C45299BB28E for ; Sun, 16 Aug 2015 17:23:47 +0000 (UTC) (envelope-from rb@gid.co.uk) Received: from mx0.gid.co.uk (mx0.gid.co.uk [194.32.164.250]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 70BC07E6 for ; Sun, 16 Aug 2015 17:23:47 +0000 (UTC) (envelope-from rb@gid.co.uk) Received: from [194.32.164.24] (80-46-130-69.static.dsl.as9105.com [80.46.130.69]) by mx0.gid.co.uk (8.14.2/8.14.2) with ESMTP id t7GHNADW066942; Sun, 16 Aug 2015 18:23:10 +0100 (BST) (envelope-from rb@gid.co.uk) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) Subject: Re: freebsd-update to 10.2-RELEASE broken ? From: Bob Bishop In-Reply-To: Date: Sun, 16 Aug 2015 18:23:05 +0100 Cc: freebsd-stable@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <2C3CC22D-749A-4B92-885C-D73311997050@gid.co.uk> References: To: Christian Kratzer X-Mailer: Apple Mail (2.2104) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Aug 2015 17:23:47 -0000 Hi, > On 16 Aug 2015, at 18:16, Christian Kratzer = wrote: >=20 > Hi, >=20 > I have been trying to update several of my FreeBSD 10.1 amd64 VM to = 10.2-RELEASE with freebsd-update and have been failing with an incorrect = hash error. >=20 > This is what happens with a plain vanilla 10.1-RELEASE vm when I try = to update to 10.2-RELEASE >=20 > --snipp=E2=80=94 [=E2=80=A6] > Applying patches... done. > Fetching 2356 files... = 068eb594e5f6b97554750a8321892e4c229a6f26455f40e4d8e4e7a79577c673 has = incorrect hash. > root@test10:~ck # > --snipp-- >=20 > I get the is on all kinds of VM with different patchlevels of = 10.1-RELEASE. Some of them have /usr/src, some of them don't. >=20 > The hash seems to be different every time round. >=20 > Could this be an issue with one of the mirrors ? >=20 > Has anybody had success yet with an update to 10.2-RELEASE using = freebsd-update ? FWIW I had the same issue yesterday on a couple of systems. Repeating = freebsd-update worked after two or three goes. > Greetings > Christian >=20 > --=20 > Christian Kratzer CK Software GmbH > Email: ck@cksoft.de Wildberger Weg 24/2 > Phone: +49 7032 893 997 - 0 D-71126 Gaeufelden > Fax: +49 7032 893 997 - 9 HRB 245288, Amtsgericht Stuttgart > Mobile: +49 171 1947 843 Geschaeftsfuehrer: Christian = Kratzer > Web: http://www.cksoft.de/ > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to = "freebsd-stable-unsubscribe@freebsd.org" >=20 -- Bob Bishop rb@gid.co.uk From owner-freebsd-stable@freebsd.org Sun Aug 16 17:41:16 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BDF949BB562 for ; Sun, 16 Aug 2015 17:41:16 +0000 (UTC) (envelope-from me@janh.de) Received: from mout.kundenserver.de (mout.kundenserver.de [212.227.17.24]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mout.kundenserver.de", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 33C86D67 for ; Sun, 16 Aug 2015 17:41:15 +0000 (UTC) (envelope-from me@janh.de) Received: from [192.168.178.36] ([87.174.244.233]) by mrelayeu.kundenserver.de (mreue103) with ESMTPSA (Nemesis) id 0LfGuG-1YyL0X2Blw-00opWd; Sun, 16 Aug 2015 19:41:01 +0200 Subject: Re: libopie problems after upgrade to 10.2 References: To: Chris Anderson Cc: stable-list freebsd From: Jan Henrik Sylvester X-Enigmail-Draft-Status: N1110 Message-ID: <55D0CB2B.2010309@janh.de> Date: Sun, 16 Aug 2015 19:40:59 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.1.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:TukSm9fK72+wEzDBgR4vki+QQtSInpeftXrND9nf9Y1Gkix4kB7 IBMTLiJbV9zVheHgr85LJXUWYBU6Q1IT9p9vbq6EBBeO7S1lI5A0cDxSWIGrhoZcVWENcIq h6eBjiEvEprbkVryq0JPoJOKVAx7VeDPvla1B2hABJz5IsTtuTM0fqh2O32WmnnOhBNG1su xMrvzUSLSTG6zJkIzQJ+w== X-UI-Out-Filterresults: notjunk:1;V01:K0:vMTJ61lt2uw=:dBToAd5BqONcop5Q+etsXY Zb0LAzC+knGZL7CORskTA4VHsD0ytxgSZpIgeSiavi4N7VkI0MgDZf0DrKZufXx3ZJYiqXy10 qIcTZmbuaL9XhXWvm1ayOkf/GBWBpjcQIWZH+iLz94BfYNnC1ukma41vUCPzNLmFxNAWbeNrn upzJXyXPfaGV0pa9lVdXnbX9MzUuUb+THeunf8xgfBbJuYIdtq23XQnUh5cwnvc1lcVN3iSc/ WvLaAuavLhXhpUgDvX0xcw84/DEu1tX7+hnU3F+4FlK+/05iJERoAuUjujB15tNdeFyZ0rGKz AJ6ZNOabXf8X9gv9Q4zv/4kLuQOaI3wby/YfQIPY69ejFDiA7r9UNZg1SeWrRZP9jYWRLcNyt NFoEwp0PpiV39DIb8rcDpRzRNbWh4mHL/96SUTn3tPUrbpcdC4GWDaDzL2zTCR1v/ye6z8kbT a45hkpwOTe+AwFHZ71W0rGZIsXqIcbB105MP1fUClFa/h++08HFBT4/Zpsz6nCCDObF+NDpmf yZb35I7pIZZ9SMaxlTVOJkFdYxhQJo88u2+ALbUHallJbSU7IbuyzciaMwOrpwZmw3FGFRwRr f7jjw/qMfBuoGYXwmR9ICyaS8zMzzVKYUx7cwLGD8u6qcUTsnzvKCl5f1mCnpwnbdcay1sr9z 1OjLmJ5OxguWIWLORQXJfjo41BnTdGXCeZB5DFZ1CJCYUzUdDYkdp1zcwvrAXWqxxIgUN/bqi TqWd4kK/UkKpHiEa X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Aug 2015 17:41:16 -0000 On 08/15/2015 20:47, Chris Anderson wrote: > just upgraded from 10.1-RELEASE-p16 to 10.2-RELEASE using freebsd-update. > > after the upgrade, I began getting errors because pam_opie.so.5 has an > unsatisfied link to libopie.so.7 (my system only has libopie.so.8). > > I notice a fresh install of 10.2-RELEASE does indeed contain libopie.so.7, > so I'm curious how I managed to get into this state in the first place and > whether it is anything I should worry about. This machine has only been > upgraded using freebsd-update and I'm pretty sure it started from > 10.0-RELEASE. I did the same update using freebsd-update and I do not have libopie.so.8 that should not be in any 10.X-RELEASE. libopie.so.8 was in stable/10 shortly after 10.0-RELEASE, but was set back again to libopie.so.7 between 10.1-RC1 and 10.1-RELEASE: https://svnweb.freebsd.org/base/releng/10.1/lib/libopie/Makefile?view=log&pathrev=273169 Your problem was probably not introduced during the 10.1-RELEASE to 10.2-RELEASE upgrade but earlier. I have a system that had just about every BETA, RC, and RELEASE starting from 9.0-RC1 using freebsd-update binary upgrades only, including some BETA or RC of 10.1 with libopie.so.8... that system has only libopie.so.7 now as it should have. Maybe you forgot the "removing of old libraries" step of "freebsd-update install" after "freebsd-update upgrade" around 10.1-RC3, because you did not expect it on a stable branch? Cheers, Jan Henrik From owner-freebsd-stable@freebsd.org Sun Aug 16 17:47:08 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DDA209BB63F for ; Sun, 16 Aug 2015 17:47:08 +0000 (UTC) (envelope-from kpaasial@gmail.com) Received: from mail-io0-x230.google.com (mail-io0-x230.google.com [IPv6:2607:f8b0:4001:c06::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AA98A103F for ; Sun, 16 Aug 2015 17:47:08 +0000 (UTC) (envelope-from kpaasial@gmail.com) Received: by iodv127 with SMTP id v127so113531983iod.3 for ; Sun, 16 Aug 2015 10:47:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=PeBP9JuEUsmzWK8zV6TwXQqgu1gKpYQpVXWdR7G3GRo=; b=kC64O4414Cdbn+bEWd1MLhfIFO+Q9LnrOMNODIVete931XuNZCSJNbFV3BsTopgcQ1 XeWuQBdd4e+KlGaCYf1YNIryQywYnS5yaCx7iXDPGOVB6xzqYcJRBT9PDBWZzoNC+LCU a4beb6Zm7pkou9i75Q9GMSVSHaty0Mqbsscsoc2YSAl7i4IGIuX3WvrbZ3zL2JjifJ9T hl+yBunRQJsB9K1q2lCwpUjVa0Dhur0LAYjuS1BkRYG6KmXx4jqmJVXgSrp0HrOayPTt sV0mpbrnM9qWSgjy4qcbSVFJZIAIxAUA5fz62NoT4Bzu/S6Q8PRXHM229io/WPhtU+kV nXLw== MIME-Version: 1.0 X-Received: by 10.107.164.41 with SMTP id n41mr476420ioe.197.1439747228073; Sun, 16 Aug 2015 10:47:08 -0700 (PDT) Received: by 10.107.131.163 with HTTP; Sun, 16 Aug 2015 10:47:08 -0700 (PDT) In-Reply-To: <55D0CB2B.2010309@janh.de> References: <55D0CB2B.2010309@janh.de> Date: Sun, 16 Aug 2015 20:47:08 +0300 Message-ID: Subject: Re: libopie problems after upgrade to 10.2 From: Kimmo Paasiala To: Jan Henrik Sylvester Cc: Chris Anderson , stable-list freebsd Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Aug 2015 17:47:09 -0000 On Sun, Aug 16, 2015 at 8:40 PM, Jan Henrik Sylvester wrote: > On 08/15/2015 20:47, Chris Anderson wrote: >> just upgraded from 10.1-RELEASE-p16 to 10.2-RELEASE using freebsd-update. >> >> after the upgrade, I began getting errors because pam_opie.so.5 has an >> unsatisfied link to libopie.so.7 (my system only has libopie.so.8). >> >> I notice a fresh install of 10.2-RELEASE does indeed contain libopie.so.7, >> so I'm curious how I managed to get into this state in the first place and >> whether it is anything I should worry about. This machine has only been >> upgraded using freebsd-update and I'm pretty sure it started from >> 10.0-RELEASE. > > I did the same update using freebsd-update and I do not have > libopie.so.8 that should not be in any 10.X-RELEASE. > > libopie.so.8 was in stable/10 shortly after 10.0-RELEASE, but was set > back again to libopie.so.7 between 10.1-RC1 and 10.1-RELEASE: > > https://svnweb.freebsd.org/base/releng/10.1/lib/libopie/Makefile?view=log&pathrev=273169 > > Your problem was probably not introduced during the 10.1-RELEASE to > 10.2-RELEASE upgrade but earlier. > > I have a system that had just about every BETA, RC, and RELEASE starting > from 9.0-RC1 using freebsd-update binary upgrades only, including some > BETA or RC of 10.1 with libopie.so.8... that system has only > libopie.so.7 now as it should have. Maybe you forgot the "removing of > old libraries" step of "freebsd-update install" after "freebsd-update > upgrade" around 10.1-RC3, because you did not expect it on a stable branch? > > Cheers, > Jan Henrik As far as I know freebsd-update(8) should handle the obsolete files automatically, it's only when you're building from source you have to remember to do 'make delete-old delete-old-libs'. If freebsd-update(8) fails to delete the obsolete files it's a flaw in it that should be reported with a PR. -Kimmo From owner-freebsd-stable@freebsd.org Sun Aug 16 18:05:03 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 73EC09BB966 for ; Sun, 16 Aug 2015 18:05:03 +0000 (UTC) (envelope-from me@janh.de) Received: from mout.kundenserver.de (mout.kundenserver.de [212.227.17.13]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mout.kundenserver.de", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2049F19A5 for ; Sun, 16 Aug 2015 18:05:02 +0000 (UTC) (envelope-from me@janh.de) Received: from [192.168.178.36] ([87.174.244.233]) by mrelayeu.kundenserver.de (mreue104) with ESMTPSA (Nemesis) id 0LuLSB-1YknaQ25KB-011jK9; Sun, 16 Aug 2015 20:04:49 +0200 Subject: Re: libopie problems after upgrade to 10.2 To: Kimmo Paasiala , Chris Anderson References: <55D0CB2B.2010309@janh.de> Cc: stable-list freebsd From: Jan Henrik Sylvester X-Enigmail-Draft-Status: N1110 Message-ID: <55D0D0C0.2000605@janh.de> Date: Sun, 16 Aug 2015 20:04:48 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.1.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:ijHnHlNSx0QN/LHGlKhUcY4zW1P4NaAhLKT1Y4rYRIoXjiuiVOD liIzq1Ev9UXEUVnnAsJJZBu6gyg8m7WORAQIbt71rXmJcnNvh9bJuzTJa5DavU9dZQOLgIE BH9wT4B5LEwEEjREAd70SYWnq52l3olgDpocusjUs5dGD4ZrqGDaRw3Eo8/kIYxYRRfpBS1 aciUZgCc46RqG/HRTOO/w== X-UI-Out-Filterresults: notjunk:1;V01:K0:lbqeY1s2e7M=:5wlZpQzL2GI4PJUh1dmC3m eVv75HcFDijAAso2AXFTJHXm9Yth+xzq7qjXxTaAdVWbp2YO9dOnC75A/FEcv+xA/bQiyK5vk T6DUEWoDJDNE6J2gPOcx5R/IoBlTON9HzdljlvtwOXm1jO3wAgMag3s/1FgwCq5zcp4sQlSfT DK9XhfRU74HnIh8HNuO+5No5EgU9pVtMQeZGdbEHQSrfyuQwKQd0HWoV0lIwZ6kiQkCyyAIb+ zJRu0yOFWh+u6ckVPyuGmZGUiNx0E8r/QLN0AbK5ek/AqIX5XtUQp3UsTPxaw2LcoUtoLweV5 3jJo1VPgr+mMgix3YXTdutfKyBZunRoddjdO5QmHdvnMhtPbue58cKyF+uJ4XA6aNj7idyYb8 Ynev+NrsU3rJ79lzYP21dC0/DxXyc4xE7SvzSICDBHGCbuq7ihvb+Ia3Y2d7B1noeLNafhK+O dHUDSVCePXxfn0bOECatx7QzWgfU3O7DhGF96pZdmtUV1mBcU/oTbmjJnHkT5R+d3i6tszoE1 mcyTHeasJIBAui7KeAGs/BJKcbwE5wuvfKnF0IA00Ot6enPpjZ93ScKxnBVM+MnGU5PV1kLjJ eGhQRDFYCfCZmcNbv15fUJpzTPGnxx9sr4vxfLbhOHqmO1x6FI7BxV21TdjF7ZAaAJ+EiBPHa tJUDz9SeHX5JvgtruG3sSOKMCbBucN9/VCyf+NsrAthEeoB1ae/I5hFQgs7E/Vf+VDQwtsVgt 9rYIDZPp5SQ4coYGaKqnrtEi1NZSWEWZE2la+dfGcrXH6ImDqw8mpjKyRU91gIh7OKWzQu9IP dwSOEtzEIvVpg9zfvWNWuwNfcSDQ9gg7WaJ1Ssd/qO09ra15RN5bxFkvS3HMptD8u0rNX8w+X Z7HagbHKc5WsvCexQLPe1vmqa3DbUuQYx31SlKnpc= X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Aug 2015 18:05:03 -0000 On 08/16/2015 19:47, Kimmo Paasiala wrote: > On Sun, Aug 16, 2015 at 8:40 PM, Jan Henrik Sylvester wrote: >> On 08/15/2015 20:47, Chris Anderson wrote: >>> just upgraded from 10.1-RELEASE-p16 to 10.2-RELEASE using freebsd-update. >>> >>> after the upgrade, I began getting errors because pam_opie.so.5 has an >>> unsatisfied link to libopie.so.7 (my system only has libopie.so.8). >>> >>> I notice a fresh install of 10.2-RELEASE does indeed contain libopie.so.7, >>> so I'm curious how I managed to get into this state in the first place and >>> whether it is anything I should worry about. This machine has only been >>> upgraded using freebsd-update and I'm pretty sure it started from >>> 10.0-RELEASE. >> >> I did the same update using freebsd-update and I do not have >> libopie.so.8 that should not be in any 10.X-RELEASE. >> >> libopie.so.8 was in stable/10 shortly after 10.0-RELEASE, but was set >> back again to libopie.so.7 between 10.1-RC1 and 10.1-RELEASE: >> >> https://svnweb.freebsd.org/base/releng/10.1/lib/libopie/Makefile?view=log&pathrev=273169 >> >> Your problem was probably not introduced during the 10.1-RELEASE to >> 10.2-RELEASE upgrade but earlier. >> >> I have a system that had just about every BETA, RC, and RELEASE starting >> from 9.0-RC1 using freebsd-update binary upgrades only, including some >> BETA or RC of 10.1 with libopie.so.8... that system has only >> libopie.so.7 now as it should have. Maybe you forgot the "removing of >> old libraries" step of "freebsd-update install" after "freebsd-update >> upgrade" around 10.1-RC3, because you did not expect it on a stable branch? >> >> Cheers, >> Jan Henrik > > > As far as I know freebsd-update(8) should handle the obsolete files > automatically, it's only when you're building from source you have to > remember to do 'make delete-old delete-old-libs'. If freebsd-update(8) > fails to delete the obsolete files it's a flaw in it that should be > reported with a PR. In general, you need 3 runs of "freebsd-update install", one for installing the kernel, one for installing the userland, and one for removing obsolete shared libraries (see install_files in freebsd-update). Of course, the third run is usually not needed during minor version upgrades, but it was needed between 10.1-RC2 and 10.1-RC3. Although you get a message about the necessary third run, you may forget about it anyhow. That was my speculation. Rethinking the problem, that is probably not it, since both libopie.so.7 and libopie.so.8 would be present in that case. Cheers, Jan Henrik From owner-freebsd-stable@freebsd.org Sun Aug 16 18:07:16 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0D23A9BB9D7 for ; Sun, 16 Aug 2015 18:07:16 +0000 (UTC) (envelope-from pi@opsec.eu) Received: from home.opsec.eu (home.opsec.eu [IPv6:2001:14f8:200::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C3DF51AC8 for ; Sun, 16 Aug 2015 18:07:15 +0000 (UTC) (envelope-from pi@opsec.eu) Received: from pi by home.opsec.eu with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1ZR2Kx-000JOf-LV; Sun, 16 Aug 2015 20:07:15 +0200 Date: Sun, 16 Aug 2015 20:07:15 +0200 From: Kurt Jaeger To: Bob Bishop Cc: Christian Kratzer , freebsd-stable@freebsd.org Subject: Re: freebsd-update to 10.2-RELEASE broken ? Message-ID: <20150816180715.GM40589@home.opsec.eu> References: <2C3CC22D-749A-4B92-885C-D73311997050@gid.co.uk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2C3CC22D-749A-4B92-885C-D73311997050@gid.co.uk> X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Aug 2015 18:07:16 -0000 Hi! [bob wrote] > [ck wrote] > > I have been trying to update several of my FreeBSD 10.1 amd64 > > VM to 10.2-RELEASE with freebsd-update and have been failing with > > an incorrect hash error. > FWIW I had the same issue yesterday on a couple of systems. > Repeating freebsd-update worked after two or three goes. I've seen the same problem on several hosts and discussed it by mail with gjb@. We assumed that I have a DNS problem because of this line: > Looking up update.FreeBSD.org mirrors... none found. This happens with this query inside the freebsd-update script, at line 950: host -t srv _http._tcp.update.FreeBSD.org If you prime your DNS cache with manual queries, then freebsd-update will sometimes find the hosts and will report that it found some hosts. But, I just tried to reproduce this and failed, the problem persists. So, yes, it looks like a real issue. -- pi@opsec.eu +49 171 3101372 5 years to go ! From owner-freebsd-stable@freebsd.org Sun Aug 16 19:07:53 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6EA7E9BB203 for ; Sun, 16 Aug 2015 19:07:53 +0000 (UTC) (envelope-from ck-lists@cksoft.de) Received: from mx1.cksoft.de (mx1.cksoft.de [IPv6:2001:67c:24f8:1::25:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.cksoft.de", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 2A6C417CF for ; Sun, 16 Aug 2015 19:07:53 +0000 (UTC) (envelope-from ck-lists@cksoft.de) Received: from m.cksoft.de (unknown [IPv6:2a01:170:1110:8001::25:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.cksoft.de (Postfix) with ESMTPSA id 2D5D41E9EB4; Sun, 16 Aug 2015 21:07:51 +0200 (CEST) Received: from amavis.cksoft.de (unknown [IPv6:2a01:170:1110:8001::25:a1]) by m.cksoft.de (Postfix) with ESMTP id 920DE631CF; Sun, 16 Aug 2015 21:06:49 +0200 (CEST) X-Virus-Scanned: amavisd-new at cksoft.de Received: from m.cksoft.de ([192.168.64.93]) by amavis.cksoft.de (amavis.cksoft.de [192.168.64.94]) (amavisd-new, port 10041) with ESMTP id D8uqCyq6Z47k; Sun, 16 Aug 2015 21:06:49 +0200 (CEST) Received: from noc1.cksoft.de (noc1.cksoft.de [IPv6:2a01:170:1110:8001::53:1]) by m.cksoft.de (Postfix) with ESMTP id B337762F88; Sun, 16 Aug 2015 21:06:48 +0200 (CEST) Received: by noc1.cksoft.de (Postfix, from userid 1000) id D823013BC6; Sun, 16 Aug 2015 21:07:49 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by noc1.cksoft.de (Postfix) with ESMTP id AB33513B9B; Sun, 16 Aug 2015 21:07:49 +0200 (CEST) Date: Sun, 16 Aug 2015 21:07:49 +0200 (CEST) From: Christian Kratzer X-X-Sender: ck@noc1.cksoft.de Reply-To: Christian Kratzer To: Kurt Jaeger cc: Bob Bishop , freebsd-stable@freebsd.org Subject: Re: freebsd-update to 10.2-RELEASE broken ? In-Reply-To: <20150816180715.GM40589@home.opsec.eu> Message-ID: References: <2C3CC22D-749A-4B92-885C-D73311997050@gid.co.uk> <20150816180715.GM40589@home.opsec.eu> User-Agent: Alpine 2.20 (BSF 67 2015-01-07) X-Spammer-Kill-Ratio: 75% MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Aug 2015 19:07:53 -0000 Hi, On Sun, 16 Aug 2015, Kurt Jaeger wrote: > We assumed that I have a DNS problem because of this line: > >> Looking up update.FreeBSD.org mirrors... none found. > > This happens with this query inside the freebsd-update script, at > line 950: > > host -t srv _http._tcp.update.FreeBSD.org > > If you prime your DNS cache with manual queries, then freebsd-update > will sometimes find the hosts and will report that it found some hosts. > > But, I just tried to reproduce this and failed, the problem persists. > > So, yes, it looks like a real issue. hmmm. Thanks for pointing me at the dns issue. I actually did not see that message as it seems to only appear on subsequent rounds of running freebsd-update. I always deleted /var/db/freebsd-update and thus always started clean. I was able to complete the freebsd-update upgrade when I manually specified on of the mirrors as in: freebsd-update -s update2.freebsd.org -r 10.2-RELEASE upgrade So this does seem to be a dns related issue. It could also be the related to parsing the results of the dns lookup. Anyway seems we have a workaround if we specify the mirrors manually. Greetings Christian -- Christian Kratzer CK Software GmbH Email: ck@cksoft.de Wildberger Weg 24/2 Phone: +49 7032 893 997 - 0 D-71126 Gaeufelden Fax: +49 7032 893 997 - 9 HRB 245288, Amtsgericht Stuttgart Mobile: +49 171 1947 843 Geschaeftsfuehrer: Christian Kratzer Web: http://www.cksoft.de/ From owner-freebsd-stable@freebsd.org Sun Aug 16 19:15:08 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A75199BB386 for ; Sun, 16 Aug 2015 19:15:08 +0000 (UTC) (envelope-from news@mips.inka.de) Received: from mail.inka.de (quechua.inka.de [IPv6:2001:7c0:407:1001:217:a4ff:fe3b:e77c]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6BEC81CA3 for ; Sun, 16 Aug 2015 19:15:08 +0000 (UTC) (envelope-from news@mips.inka.de) Received: from mips.inka.de (news@[127.0.0.1]) by mail.inka.de with uucp (rmailwrap 0.5) id 1ZR3Ob-000083-5m; Sun, 16 Aug 2015 21:15:05 +0200 Received: from lorvorc.mips.inka.de (localhost [127.0.0.1]) by lorvorc.mips.inka.de (8.15.2/8.15.2) with ESMTP id t7GJA75e024874 for ; Sun, 16 Aug 2015 21:10:07 +0200 (CEST) (envelope-from news@lorvorc.mips.inka.de) Received: (from news@localhost) by lorvorc.mips.inka.de (8.15.2/8.15.2/Submit) id t7GJA7ud024873 for freebsd-stable@freebsd.org; Sun, 16 Aug 2015 21:10:07 +0200 (CEST) (envelope-from news) To: freebsd-stable@freebsd.org From: Christian Weisgerber Newsgroups: list.freebsd.stable Subject: Re: 10.2: ntp update breaks DCF77 clock Date: Sun, 16 Aug 2015 19:10:07 +0000 (UTC) Lines: 17 Message-ID: References: <55D03771.9000605@FreeBSD.org> X-Trace: lorvorc.mips.inka.de 1439752207 24504 ::1 (16 Aug 2015 19:10:07 GMT) X-Complaints-To: usenet@mips.inka.de User-Agent: slrn/1.0.2 (FreeBSD) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Aug 2015 19:15:08 -0000 On 2015-08-16, Matthew Seaman wrote: > Hmmm.... I suggest raising a PR with patches to revert the changes in > the set of enabled clock drivers (or merge with the current list). It's Yes. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=202362 > not going to get you a working DCF77 receiver in a -RELEASE version any > time soon, I'm afraid, as you'll have to wait until the next release for > the changes to percolate down, I finally had an occasion to try "freebsd-update rollback" and went back to 10.1. -- Christian "naddy" Weisgerber naddy@mips.inka.de From owner-freebsd-stable@freebsd.org Sun Aug 16 19:20:08 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EEB799BB431 for ; Sun, 16 Aug 2015 19:20:08 +0000 (UTC) (envelope-from kpaasial@gmail.com) Received: from mail-ig0-x229.google.com (mail-ig0-x229.google.com [IPv6:2607:f8b0:4001:c05::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B98B21DE1 for ; Sun, 16 Aug 2015 19:20:08 +0000 (UTC) (envelope-from kpaasial@gmail.com) Received: by igbjg10 with SMTP id jg10so42098364igb.0 for ; Sun, 16 Aug 2015 12:20:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Lh84QqbIx5Qr5D4Z+Sv0Sy1zOa4h3GjtAT1avwubLkk=; b=sJ6xnU0BBj3WI0VJc668Q9DT70/vUQyVEftKzpeXOKmkP/fPpYj7TgMjhL9wqUQafs UhJ/FUJ+SWTtpD3rUUMNV1FQ9olw80hNQkpqwWu9wF+McNIKetEuV+6k4mApbpv/WMEO YXExb63U9+iPvGOjHbGwO4F2Y/BD1CSJ/UntuAZKxkGMkSJimqcYAEMH1Vm2PxFJplKI Fw8pitYjSxnVcdwXZI+ea8FBg4Pr2VXarKVAMBfzE7fu2DUKSWG+kJhBO2sS9sjOqd4m dWg1H/NT/URWZ0V27PwQSbMgoYcyoYA8YVizQZiutRz08s07MBhsrN0/SSm2q+rdZBu5 Ekkw== MIME-Version: 1.0 X-Received: by 10.50.97.71 with SMTP id dy7mr13443954igb.76.1439752808225; Sun, 16 Aug 2015 12:20:08 -0700 (PDT) Received: by 10.107.131.163 with HTTP; Sun, 16 Aug 2015 12:20:08 -0700 (PDT) In-Reply-To: References: <2C3CC22D-749A-4B92-885C-D73311997050@gid.co.uk> <20150816180715.GM40589@home.opsec.eu> Date: Sun, 16 Aug 2015 22:20:08 +0300 Message-ID: Subject: Re: freebsd-update to 10.2-RELEASE broken ? From: Kimmo Paasiala To: Christian Kratzer Cc: Kurt Jaeger , "freebsd-stable@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Aug 2015 19:20:09 -0000 On Sun, Aug 16, 2015 at 10:07 PM, Christian Kratzer wrote: > Hi, > > On Sun, 16 Aug 2015, Kurt Jaeger wrote: > >> >> We assumed that I have a DNS problem because of this line: >> >>> Looking up update.FreeBSD.org mirrors... none found. >> >> >> This happens with this query inside the freebsd-update script, at >> line 950: >> >> host -t srv _http._tcp.update.FreeBSD.org >> >> If you prime your DNS cache with manual queries, then freebsd-update >> will sometimes find the hosts and will report that it found some hosts. >> >> But, I just tried to reproduce this and failed, the problem persists. >> >> So, yes, it looks like a real issue. > > > hmmm. Thanks for pointing me at the dns issue. > > I actually did not see that message as it seems to only appear on > subsequent rounds of running freebsd-update. I always deleted > /var/db/freebsd-update and thus always started clean. > > I was able to complete the freebsd-update upgrade when I manually > specified on of the mirrors as in: > > freebsd-update -s update2.freebsd.org -r 10.2-RELEASE upgrade > > So this does seem to be a dns related issue. It could also be the > related to parsing the results of the dns lookup. > > Anyway seems we have a workaround if we specify the mirrors manually. > > Greetings > Christian > > -- It could be the classic fall back to TCP on SRV records problem on your upstream DNS forwarder if you're using one: http://lists.freebsd.org/pipermail/freebsd-ports/2012-May/074801.html The cure would be to use your own caching DNS resolver (configured to query the authoritative name servers directly) such as dns/unbound. -Kimmo From owner-freebsd-stable@freebsd.org Sun Aug 16 19:27:29 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F406F9BB575 for ; Sun, 16 Aug 2015 19:27:28 +0000 (UTC) (envelope-from ck-lists@cksoft.de) Received: from mx1.cksoft.de (mx1.cksoft.de [IPv6:2001:67c:24f8:1::25:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.cksoft.de", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id AF9C71171 for ; Sun, 16 Aug 2015 19:27:28 +0000 (UTC) (envelope-from ck-lists@cksoft.de) Received: from m.cksoft.de (unknown [IPv6:2a01:170:1110:8001::25:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.cksoft.de (Postfix) with ESMTPSA id CD36C1E9E6E; Sun, 16 Aug 2015 21:27:26 +0200 (CEST) Received: from amavis.cksoft.de (unknown [IPv6:2a01:170:1110:8001::25:a1]) by m.cksoft.de (Postfix) with ESMTP id 333C662F88; Sun, 16 Aug 2015 21:26:25 +0200 (CEST) X-Virus-Scanned: amavisd-new at cksoft.de Received: from m.cksoft.de ([IPv6:2a01:170:1110:8001::25:1]) by amavis.cksoft.de (amavis.cksoft.de [IPv6:2a01:170:1110:8001::25:a1]) (amavisd-new, port 10041) with ESMTP id MvdMN2KCGNQU; Sun, 16 Aug 2015 21:26:21 +0200 (CEST) Received: from noc1.cksoft.de (noc1.cksoft.de [IPv6:2a01:170:1110:8001::53:1]) by m.cksoft.de (Postfix) with ESMTP id 0F95562ECF; Sun, 16 Aug 2015 21:26:20 +0200 (CEST) Received: by noc1.cksoft.de (Postfix, from userid 1000) id 1B7EA13BC6; Sun, 16 Aug 2015 21:27:22 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by noc1.cksoft.de (Postfix) with ESMTP id 1482513B13; Sun, 16 Aug 2015 21:27:22 +0200 (CEST) Date: Sun, 16 Aug 2015 21:27:22 +0200 (CEST) From: Christian Kratzer X-X-Sender: ck@noc1.cksoft.de Reply-To: Christian Kratzer To: Kimmo Paasiala cc: Kurt Jaeger , "freebsd-stable@freebsd.org" Subject: Re: freebsd-update to 10.2-RELEASE broken ? In-Reply-To: Message-ID: References: <2C3CC22D-749A-4B92-885C-D73311997050@gid.co.uk> <20150816180715.GM40589@home.opsec.eu> User-Agent: Alpine 2.20 (BSF 67 2015-01-07) X-Spammer-Kill-Ratio: 75% MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Aug 2015 19:27:29 -0000 Hi, On Sun, 16 Aug 2015, Kimmo Paasiala wrote: > It could be the classic fall back to TCP on SRV records problem on > your upstream DNS forwarder if you're using one: > > http://lists.freebsd.org/pipermail/freebsd-ports/2012-May/074801.html > > The cure would be to use your own caching DNS resolver (configured to > query the authoritative name servers directly) such as dns/unbound. I run my own bind9 resolvers on freebsd 10 at both sites. I never particurlarly like the concept of an "upstream" resolver. All my resolvers are behind firewalls although different kinds. ASA at one site and freebsd pf at the other. I will investigate though. Thanks for the tip. Greetings Christian -- Christian Kratzer CK Software GmbH Email: ck@cksoft.de Wildberger Weg 24/2 Phone: +49 7032 893 997 - 0 D-71126 Gaeufelden Fax: +49 7032 893 997 - 9 HRB 245288, Amtsgericht Stuttgart Mobile: +49 171 1947 843 Geschaeftsfuehrer: Christian Kratzer Web: http://www.cksoft.de/ From owner-freebsd-stable@freebsd.org Sun Aug 16 20:07:02 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 30C779BBB5C for ; Sun, 16 Aug 2015 20:07:02 +0000 (UTC) (envelope-from lists@opsec.eu) Received: from home.opsec.eu (home.opsec.eu [IPv6:2001:14f8:200::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E68F0149C for ; Sun, 16 Aug 2015 20:07:01 +0000 (UTC) (envelope-from lists@opsec.eu) Received: from pi by home.opsec.eu with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1ZR4Ct-000JcB-Ba; Sun, 16 Aug 2015 22:07:03 +0200 Date: Sun, 16 Aug 2015 22:07:03 +0200 From: Kurt Jaeger To: Kimmo Paasiala Cc: Christian Kratzer , "freebsd-stable@freebsd.org" Subject: Re: freebsd-update to 10.2-RELEASE broken ? Message-ID: <20150816200703.GN40589@home.opsec.eu> References: <2C3CC22D-749A-4B92-885C-D73311997050@gid.co.uk> <20150816180715.GM40589@home.opsec.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Aug 2015 20:07:02 -0000 Hi! > It could be the classic fall back to TCP on SRV records problem on > your upstream DNS forwarder if you're using one: > > http://lists.freebsd.org/pipermail/freebsd-ports/2012-May/074801.html If I query that same DNS resolver using the line from the script, it works every time. It's a 10.1p16 host with a very recent ports build, and directly connected (no NAT, no fw etc). If that would be the problem, how could I diagnose it in depth ? -- pi@opsec.eu +49 171 3101372 5 years to go ! From owner-freebsd-stable@freebsd.org Sun Aug 16 21:17:00 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CA0369BB5FB for ; Sun, 16 Aug 2015 21:17:00 +0000 (UTC) (envelope-from ck-lists@cksoft.de) Received: from mx1.cksoft.de (mx1.cksoft.de [IPv6:2001:67c:24f8:1::25:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.cksoft.de", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 7E4851370 for ; Sun, 16 Aug 2015 21:17:00 +0000 (UTC) (envelope-from ck-lists@cksoft.de) Received: from m.cksoft.de (unknown [IPv6:2a01:170:1110:8001::25:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.cksoft.de (Postfix) with ESMTPSA id 72B6C1E9E6E; Sun, 16 Aug 2015 23:16:58 +0200 (CEST) Received: from amavis.cksoft.de (unknown [IPv6:2a01:170:1110:8001::25:a1]) by m.cksoft.de (Postfix) with ESMTP id A303E62F88; Sun, 16 Aug 2015 23:15:56 +0200 (CEST) X-Virus-Scanned: amavisd-new at cksoft.de Received: from m.cksoft.de ([192.168.64.93]) by amavis.cksoft.de (amavis.cksoft.de [192.168.64.94]) (amavisd-new, port 10041) with ESMTP id 7Xx4QarYvAtS; Sun, 16 Aug 2015 23:15:56 +0200 (CEST) Received: from noc1.cksoft.de (noc1.cksoft.de [IPv6:2a01:170:1110:8001::53:1]) by m.cksoft.de (Postfix) with ESMTP id CAADB62ECF; Sun, 16 Aug 2015 23:15:55 +0200 (CEST) Received: by noc1.cksoft.de (Postfix, from userid 1000) id E103C13BC9; Sun, 16 Aug 2015 23:16:56 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by noc1.cksoft.de (Postfix) with ESMTP id DA0D713A0B; Sun, 16 Aug 2015 23:16:56 +0200 (CEST) Date: Sun, 16 Aug 2015 23:16:56 +0200 (CEST) From: Christian Kratzer X-X-Sender: ck@noc1.cksoft.de Reply-To: Christian Kratzer To: Kurt Jaeger cc: Kimmo Paasiala , "freebsd-stable@freebsd.org" Subject: Re: freebsd-update to 10.2-RELEASE broken ? In-Reply-To: <20150816200703.GN40589@home.opsec.eu> Message-ID: References: <2C3CC22D-749A-4B92-885C-D73311997050@gid.co.uk> <20150816180715.GM40589@home.opsec.eu> <20150816200703.GN40589@home.opsec.eu> User-Agent: Alpine 2.20 (BSF 67 2015-01-07) X-Spammer-Kill-Ratio: 75% MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Aug 2015 21:17:00 -0000 Hi, On Sun, 16 Aug 2015, Kurt Jaeger wrote: > Hi! > >> It could be the classic fall back to TCP on SRV records problem on >> your upstream DNS forwarder if you're using one: >> >> http://lists.freebsd.org/pipermail/freebsd-ports/2012-May/074801.html > > If I query that same DNS resolver using the line from > the script, it works every time. It's a 10.1p16 host with > a very recent ports build, and directly connected (no NAT, no > fw etc). > > If that would be the problem, how could I diagnose it in depth ? freebsd-update upgrade just failed on 3 other vm even when I explicitly specified the server using freebsd-update -s. I had success on another vm when I changed to using google dns. I am not aware that anything would be blocking tcp dns in my setups. Must be something else dns related. Perhaps I will run a local resolver in a vm and logg all queries and dns traffic. Greetings Christian -- Christian Kratzer CK Software GmbH Email: ck@cksoft.de Wildberger Weg 24/2 Phone: +49 7032 893 997 - 0 D-71126 Gaeufelden Fax: +49 7032 893 997 - 9 HRB 245288, Amtsgericht Stuttgart Mobile: +49 171 1947 843 Geschaeftsfuehrer: Christian Kratzer Web: http://www.cksoft.de/ From owner-freebsd-stable@freebsd.org Sun Aug 16 22:09:04 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 279A89BBF7E for ; Sun, 16 Aug 2015 22:09:04 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:130:3ffc::401:25]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D6D111328 for ; Sun, 16 Aug 2015 22:09:03 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id DC4FA25D3892; Sun, 16 Aug 2015 22:08:59 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 35451C7704A; Sun, 16 Aug 2015 22:08:59 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id 9of2IMoj1HbX; Sun, 16 Aug 2015 22:08:57 +0000 (UTC) Received: from [IPv6:fde9:577b:c1a9:4410:c13c:1353:3ca0:53b6] (unknown [IPv6:fde9:577b:c1a9:4410:c13c:1353:3ca0:53b6]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id 99993C76FCE; Sun, 16 Aug 2015 22:08:56 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2102\)) Subject: Re: freebsd-update to 10.2-RELEASE broken ? From: "Bjoern A. Zeeb" In-Reply-To: Date: Sun, 16 Aug 2015 22:08:53 +0000 Cc: "freebsd-stable@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: References: <2C3CC22D-749A-4B92-885C-D73311997050@gid.co.uk> <20150816180715.GM40589@home.opsec.eu> <20150816200703.GN40589@home.opsec.eu> To: Christian Kratzer X-Mailer: Apple Mail (2.2102) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Aug 2015 22:09:04 -0000 > On 16 Aug 2015, at 21:16 , Christian Kratzer = wrote: >=20 > Hi, >=20 > On Sun, 16 Aug 2015, Kurt Jaeger wrote: >> Hi! >>=20 >>> It could be the classic fall back to TCP on SRV records problem on >>> your upstream DNS forwarder if you're using one: >>>=20 >>> = http://lists.freebsd.org/pipermail/freebsd-ports/2012-May/074801.html >>=20 >> If I query that same DNS resolver using the line from >> the script, it works every time. It's a 10.1p16 host with >> a very recent ports build, and directly connected (no NAT, no >> fw etc). >>=20 >> If that would be the problem, how could I diagnose it in depth ? >=20 > freebsd-update upgrade just failed on 3 other vm even when I = explicitly > specified the server using freebsd-update -s. >=20 > I had success on another vm when I changed to using google dns. >=20 > I am not aware that anything would be blocking tcp dns in my setups. >=20 > Must be something else dns related. >=20 > Perhaps I will run a local resolver in a vm and logg all queries and = dns > traffic. Or run tcpdump for port 53; also curious if it might be an IPv4 vs. = IPv6 issue? =E2=80=94=20 Bjoern A. Zeeb Charles Haddon Spurgeon: "Friendship is one of the sweetest joys of life. Many might have failed beneath the bitterness of their trial had they not found a friend." From owner-freebsd-stable@freebsd.org Sun Aug 16 22:16:28 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 319079B9125 for ; Sun, 16 Aug 2015 22:16:28 +0000 (UTC) (envelope-from alnis.m@mail.com) Received: from mout.gmx.com (mout.gmx.com [74.208.4.201]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E9A7918FB for ; Sun, 16 Aug 2015 22:16:27 +0000 (UTC) (envelope-from alnis.m@mail.com) Received: from [192.168.2.192] ([78.84.244.14]) by mail.gmx.com (mrgmxus001) with ESMTPSA (Nemesis) id 0M8lr8-1ZdEAj03Cu-00C8cz for ; Mon, 17 Aug 2015 00:11:17 +0200 Message-ID: <55D10A82.8060607@mail.com> Date: Mon, 17 Aug 2015 01:11:14 +0300 From: Alnis Morics User-Agent: Mozilla/5.0 (X11; Linux i686; rv:31.0) Gecko/20100101 Icedove/31.7.0 MIME-Version: 1.0 To: freebsd-stable@freebsd.org Subject: Re: help me understand latest->quarterly pkg.conf switch References: <20150724212757.GE84931@FreeBSD.org> <20150725020440.GJ84931@FreeBSD.org> In-Reply-To: <20150725020440.GJ84931@FreeBSD.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K0:sVhNTG5FdNnf05m5UHQLPjACPYPheMAjQ2YzIDmy4JII97f2z0j kx0qtR0dIyY9RrYu9DQz3/k+ssDkrId3Es1BtmRrmkrpRAeEwuRu8oTU4bAsiYzPbn34a80 FMgvgsYAYHcgi25mheN/WqV5dKan7+mpYqo7zoMAfwW9M5HkuC1ARhxp6d9nz42K31Bh6cA StqB72CZ4T25tB+V1kHLw== X-UI-Out-Filterresults: notjunk:1;V01:K0:1BxItyFpAfY=:UVf4JcEkY29GDkT5NpbTBx iMs+XBZ3evTkuvHFLutZhMvotAPB25dWWgeY9scsPhpsbCmiYxbt3sPBjVNvDMuNw8HXCe3xE gzHTrwsq0Js+8JNfq1eoJ9AS41ZwDUus29eGrHnaQwPPUmg05eYvKDrJ6gLXByAj+FYwELha7 Y5xrSohLIuRUXQFhmx+sDeDK9Gk8H1nof9Mqk1LzaY0BcM2MkCNMqYKMnQkiN23tgWnB4XM8Y QjnoRzPQy8gT7FJwlz4QtwynuV1qYvaKDHL8I0D615c35IrxwAe8Ry/rd5J9cfLPswuX+3nZt beXovTDVoppEFVM+z20KH8xC3W/i07EmqDIXgmMujrqLK1p6Shm93FZRlMez5gD2zzhTk/EaA KwpN26n6m9qYew6IGKWHr5JRdK7VEozLRIuZswk7NUHL6W3RJiJYs2mGeS8MZAaFS4w2m5emQ J/QX0PfkrwUnuAY+xo43Cqm10lSGtbH0ZAehh+7qL1/DBv0mvACOanMOmSzIeTe10NyTdJaiW ttxP/UtWp5YHeCKDwrxqFf9X3EvYV8Ro1AXyzmHAgz3VUHO6usgyoybmmkKyKe1cGTHsVd4PT 0/MuwkHGmZbSRxcAv5SEnWvBvHDgJN75NDMearJv3tvzucl9YjVllzX5C9X+Eo8T5DCpqNHGh D7gEvLTIxSsyuy/iOa51CiQtK+7/YD8X+qRtzMZyQVlaL/aoEB50GcrSXY/4U5aO7pM8XIrsT SjFAGEhQKdSODeze4YAQ8c13ZhMWyhXKZkmfSw== X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Aug 2015 22:16:28 -0000 On 07/25/2015 05:04 AM, Glen Barber wrote: > On Fri, Jul 24, 2015 at 09:23:12PM -0400, Nikolai Lifanov wrote: >> On 2015-07-24 17:27, Glen Barber wrote: >>> On Fri, Jul 24, 2015 at 05:15:52PM -0400, Nikolai Lifanov wrote: >>>> I noticed that in stable/10, /etc/pkg/FreeBSD.conf was switched from >>>> using >>>> latest package set to whichever one that is "quarterly" word is pointing >>>> to >>>> at the moment. What is the motivation for this change? >>>> >>> This was not done in the stable/10 branch, it was done in releng/10.2. >>> >>>> Quarterly package sets are useful if the end-user is able to pick which >>>> one >>>> to pull from and there is some amount of time of support overlap so that >>>> the >>>> user has time to validate the new package set and switch his systems to >>>> it >>>> (like what is done with pkgsrc). As-is, "quarterly" works just like >>>> "latest" >>> >from end-user perspective, but for most of the lifecycle packages are >>>> outdated and there is a massive update bomb four times per year. >>>> >>>> Port branches are still valuable to those building their own packages, >>>> since >>>> they can support the previous (unsupported by the project) branch, >>>> backporting fixes manually, while validating and upgrading to the new >>>> one. >>>> But, what is the value of the quarterly package set as-is and what is >>>> the >>>> value of switching to this set by default? >>>> >>> In general, the quarterly package set is less prone to having build >>> failures, since the changes in the branch are (by intent) less >>> intrusive, while still receiving security updates. It is analogous to >>> the stable or releng branches in src, and how they compare to head. >>> >>> (This will be noted in the final 10.2-RELEASE announcement, as well as >>> the release notes, and will also include instructions on how to switch >>> to the 'latest' branch if that is what is desired.) >>> >>> Glen >> Cool, thanks for the explanation! >> > You're welcome. (The 'quarterly' branch is admittedly under-documented, > which is a problem.) > >> What would be really amazing to see are quarterly branches that the end user >> can switch between, like pkg.freebsd.org/${ABI}/2015Q3 -> >> pkg.freebsd.org/${ABI}/2015Q4 when he is ready, with at least a little bit >> of overlap. >> > I agree this would be a "nice to see" feature, but as I have insight > into how the pkg(8) mirrors operate, this is an unfortunately > non-trivial thing to implement. > > Glen > Does this mean that quarterly pkg are synchronous with ports quarterly branches? Say, at the moment, packages that we install with the default ("quarterly") setting in /etc/pkg/FreeBSD.conf, are built from the 2015Q3 branch? -Alnis From owner-freebsd-stable@freebsd.org Mon Aug 17 05:50:09 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 382FF9B92D6 for ; Mon, 17 Aug 2015 05:50:09 +0000 (UTC) (envelope-from cy.schubert@komquats.com) Received: from smtp-out-no.shaw.ca (smtp-out-no.shaw.ca [64.59.134.9]) by mx1.freebsd.org (Postfix) with ESMTP id C723A12E7; Mon, 17 Aug 2015 05:50:08 +0000 (UTC) (envelope-from cy.schubert@komquats.com) Received: from spqr.komquats.com ([96.50.22.10]) by shaw.ca with SMTP id RDIzZU33G4kukRDJ1ZBSmT; Sun, 16 Aug 2015 23:50:01 -0600 X-Authority-Analysis: v=2.1 cv=T55GNa+Q c=1 sm=1 tr=0 a=jvE2nwUzI0ECrNeyr98KWA==:117 a=jvE2nwUzI0ECrNeyr98KWA==:17 a=BWvPGDcYAAAA:8 a=VxmjJ2MpAAAA:8 a=kj9zAlcOel0A:10 a=uRRa74qj2VoA:10 a=6I5d2MoRAAAA:8 a=85N1-lAfAAAA:8 a=YxBL1-UpAAAA:8 a=14Cs1veZNTOHoESaBl4A:9 a=LdHzSKrN11lvHwWH:21 a=nkdFCgSthkO_pVVD:21 a=CjuIK1q_8ugA:10 Received: from slippy.cwsent.com (slippy [10.1.1.91]) by spqr.komquats.com (Postfix) with ESMTP id BAE489BF9; Sun, 16 Aug 2015 22:49:57 -0700 (PDT) Received: from slippy (localhost [127.0.0.1]) by slippy.cwsent.com (8.15.2/8.15.2) with ESMTP id t7H5nvxD009140; Sun, 16 Aug 2015 22:49:57 -0700 (PDT) (envelope-from Cy.Schubert@komquats.com) Message-Id: <201508170549.t7H5nvxD009140@slippy.cwsent.com> X-Mailer: exmh version 2.8.0 04/21/2012 with nmh-1.6 Reply-to: Cy Schubert From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.komquats.com/ To: Ian Lepore cc: Matthew Seaman , freebsd-stable@freebsd.org, Cy Schubert , roberto@freebd.org, phk@freebsd.org, delphij@freebsd.org Subject: Re: 10.2: ntp update breaks DCF77 clock In-Reply-To: Message from Ian Lepore of "Sun, 16 Aug 2015 10:57:00 -0600." <1439744220.242.87.camel@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sun, 16 Aug 2015 22:49:57 -0700 X-CMAE-Envelope: MS4wfPSXLjH992U3JBwr9w8vDJeGRxUCafZSwiZjvT7IHiJXwc4aiuQ0rZbF7tejZwd+O8RFDKZBXMicNPVRl7yxsHEvxn0J089iBFoh8fcZd4QICN77/mkLlniWeFqxRgnD5SympHAYyrikqP+QrBYQuXYMFk+4svDe8+lQPThOwVwh/kiyzo0HCDHEiDey0Ie1UJwuoe3D9NK2Xb/bbY8c0v3aseKfk+A2z7ve/rh7NVCWJug6mBgruTahD6BRzNv1B5fqSA3rMqw3bPfvtnDffWk7PvZqJmZFpdBIdwDOHgXqH4ofLMATaNHJdJKW0uO0fg== X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2015 05:50:09 -0000 In message <1439744220.242.87.camel@freebsd.org>, Ian Lepore writes: > > > --=-yOSDvPzQIQnw2oRARoLp > Content-Type: text/plain; charset="us-ascii" > Content-Transfer-Encoding: 7bit > > On Sun, 2015-08-16 at 08:10 +0100, Matthew Seaman wrote: > > On 15/08/2015 16:46, Christian Weisgerber wrote: > > > The ntp code is not very transparent, but I think the root cause > > > are the ntp/config.h changes that came with the 4.2.8p3 update. A > > > number of previously disabled obscure clock drivers were enabled, > > > but crucially CLOCK_RAWDCF was disabled, and this is the PARSE > > > subdriver needed to use the popular DCF77 serial receivers. > > > > > > Frankly, it looks like we used to have a carefully considered > > > selection of clock drivers which has been blindly splattered with > > > the upstream defaults in the last update. > > > > Hmmm.... I suggest raising a PR with patches to revert the changes in > > the set of enabled clock drivers (or merge with the current list). It's > > not going to get you a working DCF77 receiver in a -RELEASE version any > > time soon, I'm afraid, as you'll have to wait until the next release for > > the changes to percolate down, but having a sensible list of enabled > > clock drivers in base is definitely a good move. > > > > I wonder: is there a reason to not enable all (or most of) the refclocks > in base and in ports? Well, at least all the ones that build on > freebsd... a disturbing number of them fail to compile because they > include linux-specific header files. Hmm, I just noticed that we > actually compile most of the refclocks, but we don't enable them in the > config. The reason not to include all drivers is fourfold. 1. Support of all drivers does increase the risk out-of-the-box breakage and security exposure. Not often used drivers may contain additional security exposures. 2. IMO, a client-only configuration should be supported. I've been told via email (which I was cc'd) exchanged between a person on the project and our NTP upline (at ntp.org) that a timekeeping client able of understanding multiple protocols (e.g. PTP) is in the works and will someday (soon?) replace ntp in base. Avoiding too much function creep may avoid POLA when that occurs. 3. Some drivers may interfere with each other. This is why ntp as distributed by our upline does not turn them all on, and which is why the port allows a user to install them as required. I've heard of occasional conflicts over the years. Turning them all on could be a risk for the project, as in someone may have to needlessly fix something that has not been tested or supported by our upline. With the port OTOH, it's easy to tell a user to turn off whatever they don't need. 4. Also IMO, that's what the port is for. If a person wants to use ntp as a server, e.g. to serve time to other computers, i.e. not simply use ntp as a consumer, then the port is always available. (Also juxtapositioned to this point, the port is updated before base because testing and commit [commit to vendor branch, MFV, build tinderbox, run for a while, MFC] is much simpler with the port than in base.) As to why, historically, not all drivers have been built and installed is something I cannot answer, though I hazard to guess it may be due to one or more of the above. IMO I'd prefer to keep ntp minimal within base and direct people who want more to one of the ports. (The ports collection has a -rc incarnation when release candidates are available.) > > It looks like the cost of enabling all the refclocks that compile > properly is pretty small... doing so increased the size of ntpd from > 745K to 801K for me. I'll attach the diff just to save someone else the > trouble of iteratively figuring out which ones won't build, but I think > there may be a more-proper way to generate this config by tweaking the > autotools stuff. Size of the binary is not an issue. I'll be working on generating config.h. The issue here is that config.h is different for each supported FreeBSD platform (32bit, 64bit, little and big endian, etc.). The plan is to make config the port on each supported platform using qemu-sbruno and steal ^W borrow the config.h and import into base, using a master config.h (actually called config.h) to include via #ifdef definitions the appropriate config.h.. Qemu (and qemu-sbruno) doesn't support all our supported platforms, especially the multitude of ARM platforms, so holes in our auto-generated config.h support will exist. Thanks for the diff. I think the place to start is to enable them in a (new) dependent port (ntp-all and ntp-devel-all for the lack of better names) and see what fallout comes from it. -- Cheers, Cy Schubert or FreeBSD UNIX: Web: http://www.FreeBSD.org The need of the many outweighs the greed of the few. From owner-freebsd-stable@freebsd.org Mon Aug 17 06:20:36 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2C9829B9967 for ; Mon, 17 Aug 2015 06:20:36 +0000 (UTC) (envelope-from lists@opsec.eu) Received: from home.opsec.eu (home.opsec.eu [IPv6:2001:14f8:200::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E19CE12B0 for ; Mon, 17 Aug 2015 06:20:35 +0000 (UTC) (envelope-from lists@opsec.eu) Received: from pi by home.opsec.eu with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1ZRDmZ-000MI7-Iu; Mon, 17 Aug 2015 08:20:31 +0200 Date: Mon, 17 Aug 2015 08:20:31 +0200 From: Kurt Jaeger To: "Bjoern A. Zeeb" Cc: Christian Kratzer , "freebsd-stable@freebsd.org" Subject: Re: freebsd-update to 10.2-RELEASE broken ? Message-ID: <20150817062031.GO40589@home.opsec.eu> References: <2C3CC22D-749A-4B92-885C-D73311997050@gid.co.uk> <20150816180715.GM40589@home.opsec.eu> <20150816200703.GN40589@home.opsec.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2015 06:20:36 -0000 Hi! > Or run tcpdump for port 53; also curious if it might be an IPv4 > vs. IPv6 issue? I did run tcpdump on port 53 on both the client and the dns server for this, everything looked normal. It was no IPv6 issue. I'll retest with more detail this evening. -- pi@opsec.eu +49 171 3101372 5 years to go ! From owner-freebsd-stable@freebsd.org Mon Aug 17 07:33:29 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A3769BA9EA; Mon, 17 Aug 2015 07:33:29 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.116.210]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 54ACA1131; Mon, 17 Aug 2015 07:33:28 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from mbpro-w.cs.huji.ac.il ([132.65.80.91]) by kabab.cs.huji.ac.il with esmtp id 1ZREpZ-000LVA-N9; Mon, 17 Aug 2015 10:27:41 +0300 From: Daniel Braniss Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: ix(intel) vs mlxen(mellanox) 10Gb performance Date: Mon, 17 Aug 2015 10:27:41 +0300 Message-Id: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> Cc: FreeBSD Net To: FreeBSD stable Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2102\)) X-Mailer: Apple Mail (2.2102) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2015 07:33:29 -0000 hi, I have a host (Dell R730) with both cards, connected to an = HP8200 switch at 10Gb. when writing to the same storage (netapp) this is what I get: ix0: ~130MGB/s mlxen0 ~330MGB/s this is via nfs/tcpv3 I can get similar (bad) performance with the mellanox if I = increase the file size to 512MGB. so at face value, it seems the mlxen does a better use of = resources than the intel. Any ideas how to improve ix/intel=E2=80=99s performance? cheers, dnny From owner-freebsd-stable@freebsd.org Mon Aug 17 09:41:55 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 05B179BBBDF; Mon, 17 Aug 2015 09:41:55 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B7E051C15; Mon, 17 Aug 2015 09:41:54 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.84 (FreeBSD)) (envelope-from ) id 1ZRGvJ-000HwX-Ji; Mon, 17 Aug 2015 12:41:45 +0300 Date: Mon, 17 Aug 2015 12:41:45 +0300 From: Slawa Olhovchenkov To: Daniel Braniss Cc: FreeBSD stable , FreeBSD Net Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance Message-ID: <20150817094145.GB3158@zxy.spb.ru> References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2015 09:41:55 -0000 On Mon, Aug 17, 2015 at 10:27:41AM +0300, Daniel Braniss wrote: > hi, > I have a host (Dell R730) with both cards, connected to an HP8200 switch at 10Gb. > when writing to the same storage (netapp) this is what I get: > ix0: ~130MGB/s > mlxen0 ~330MGB/s > this is via nfs/tcpv3 > > I can get similar (bad) performance with the mellanox if I increase the file size > to 512MGB. Look like mellanox have internal beffer for caching and do ACK acclerating. > so at face value, it seems the mlxen does a better use of resources than the intel. > Any ideas how to improve ix/intel's performance? Are you sure about netapp performance? From owner-freebsd-stable@freebsd.org Mon Aug 17 10:35:14 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1B52C9BBA94; Mon, 17 Aug 2015 10:35:14 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.116.210]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C77E11812; Mon, 17 Aug 2015 10:35:13 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from mbpro-w.cs.huji.ac.il ([132.65.80.91]) by kabab.cs.huji.ac.il with esmtp id 1ZRHkw-000Pr5-D5; Mon, 17 Aug 2015 13:35:06 +0300 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2102\)) Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance From: Daniel Braniss In-Reply-To: <20150817094145.GB3158@zxy.spb.ru> Date: Mon, 17 Aug 2015 13:35:06 +0300 Cc: FreeBSD stable , FreeBSD Net Content-Transfer-Encoding: quoted-printable Message-Id: <197995E2-0C11-43A2-AB30-FBB0FB8CE2C5@cs.huji.ac.il> References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <20150817094145.GB3158@zxy.spb.ru> To: Slawa Olhovchenkov X-Mailer: Apple Mail (2.2102) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2015 10:35:14 -0000 > On Aug 17, 2015, at 12:41 PM, Slawa Olhovchenkov = wrote: >=20 > On Mon, Aug 17, 2015 at 10:27:41AM +0300, Daniel Braniss wrote: >=20 >> hi, >> I have a host (Dell R730) with both cards, connected to an = HP8200 switch at 10Gb. >> when writing to the same storage (netapp) this is what I get: >> ix0: ~130MGB/s >> mlxen0 ~330MGB/s >> this is via nfs/tcpv3 >>=20 >> I can get similar (bad) performance with the mellanox if I = increase the file size >> to 512MGB. >=20 > Look like mellanox have internal beffer for caching and do ACK = acclerating. what ever they are doing, it=E2=80=99s impressive :-) >=20 >> so at face value, it seems the mlxen does a better use of = resources than the intel. >> Any ideas how to improve ix/intel's performance? >=20 > Are you sure about netapp performance? yes, and why should it act differently if the request is coming from the = same host? in any case the numbers are quiet consistent since I have measured it from several = hosts, and at different times. danny From owner-freebsd-stable@freebsd.org Mon Aug 17 10:41:54 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3DD409BBD5E; Mon, 17 Aug 2015 10:41:54 +0000 (UTC) (envelope-from csforgeron@gmail.com) Received: from mail-io0-x22d.google.com (mail-io0-x22d.google.com [IPv6:2607:f8b0:4001:c06::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0A3171CBC; Mon, 17 Aug 2015 10:41:54 +0000 (UTC) (envelope-from csforgeron@gmail.com) Received: by iodt126 with SMTP id t126so146649979iod.2; Mon, 17 Aug 2015 03:41:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=+vFQ/mA59UR3Ela9yMxgaI2A+vddpfy13o0C64efML4=; b=PA62iMuS76ELRZCw5Z+OKwSBhg5yW0f4KL4io6R6F1J3CsPqAHRsMq5dGMwRfz5N5C ugakzYcXPr030nLuwLC6fnyHlYCWCXidjQLPcMfJrAbVE8Exs3Au6Zzf/U1+tzG3qC1U cne/0fdY5Fn/ig9cWyPbDIFnt0AsCxBfHztdn7p/Ci9WwWcPooMhag8DBIazJoxYS7Bf ItWhnlKLXrTSYMTilxcgp2CpZVrxt7xZ2/ATrmrX5DYUaSErIAJJrG+ewYnRBDFF8cuR vevpmROUtPiXiJz4qznsS/hQp31wtyVuFiINuu12YY4aHTbTlx98hxy3NfZZQpnqzUgB KMtA== MIME-Version: 1.0 X-Received: by 10.107.14.84 with SMTP id 81mr603388ioo.188.1439808113168; Mon, 17 Aug 2015 03:41:53 -0700 (PDT) Received: by 10.36.34.77 with HTTP; Mon, 17 Aug 2015 03:41:53 -0700 (PDT) In-Reply-To: <20150817094145.GB3158@zxy.spb.ru> References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <20150817094145.GB3158@zxy.spb.ru> Date: Mon, 17 Aug 2015 07:41:53 -0300 Message-ID: Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance From: Christopher Forgeron To: Slawa Olhovchenkov Cc: Daniel Braniss , FreeBSD Net , FreeBSD stable Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2015 10:41:54 -0000 FYI, I can regularly hit 9.3 Gib/s with my Intel X520-DA2's and FreeBSD 10.1. Before 10.1 it was less. I used to tweak the card settings, but now it's just stock. You may want to check your settings, the Mellanox may just have better defaults for your switch. On Mon, Aug 17, 2015 at 6:41 AM, Slawa Olhovchenkov wrote: > On Mon, Aug 17, 2015 at 10:27:41AM +0300, Daniel Braniss wrote: > > > hi, > > I have a host (Dell R730) with both cards, connected to an HP8200 > switch at 10Gb. > > when writing to the same storage (netapp) this is what I get: > > ix0: ~130MGB/s > > mlxen0 ~330MGB/s > > this is via nfs/tcpv3 > > > > I can get similar (bad) performance with the mellanox if I > increase the file size > > to 512MGB. > > Look like mellanox have internal beffer for caching and do ACK acclerating. > > > so at face value, it seems the mlxen does a better use of > resources than the intel. > > Any ideas how to improve ix/intel's performance? > > Are you sure about netapp performance? > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-stable@freebsd.org Mon Aug 17 10:51:51 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C21519BBF07; Mon, 17 Aug 2015 10:51:51 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.116.210]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7591613CA; Mon, 17 Aug 2015 10:51:50 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from mbpro-w.cs.huji.ac.il ([132.65.80.91]) by kabab.cs.huji.ac.il with esmtp id 1ZRI13-00005x-IM; Mon, 17 Aug 2015 13:51:45 +0300 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2102\)) Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance From: Daniel Braniss In-Reply-To: Date: Mon, 17 Aug 2015 13:51:45 +0300 Cc: Slawa Olhovchenkov , FreeBSD Net , FreeBSD stable Message-Id: <17871443-E105-4434-80B1-6939306A865F@cs.huji.ac.il> References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <20150817094145.GB3158@zxy.spb.ru> To: Christopher Forgeron X-Mailer: Apple Mail (2.2102) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2015 10:51:52 -0000 > On Aug 17, 2015, at 1:41 PM, Christopher Forgeron = wrote: >=20 > FYI, I can regularly hit 9.3 Gib/s with my Intel X520-DA2's and = FreeBSD 10.1. Before 10.1 it was less. >=20 this is NOT iperf/3 where i do get close to wire speed, it=E2=80=99s NFS writes, i.e., almost real work :-) > I used to tweak the card settings, but now it's just stock. You may = want to check your settings, the Mellanox may just have better defaults = for your switch.=20 >=20 > On Mon, Aug 17, 2015 at 6:41 AM, Slawa Olhovchenkov > wrote: > On Mon, Aug 17, 2015 at 10:27:41AM +0300, Daniel Braniss wrote: >=20 > > hi, > > I have a host (Dell R730) with both cards, connected to an = HP8200 switch at 10Gb. > > when writing to the same storage (netapp) this is what I get: > > ix0: ~130MGB/s > > mlxen0 ~330MGB/s > > this is via nfs/tcpv3 > > > > I can get similar (bad) performance with the mellanox if I = increase the file size > > to 512MGB. >=20 > Look like mellanox have internal beffer for caching and do ACK = acclerating. >=20 > > so at face value, it seems the mlxen does a better use of = resources than the intel. > > Any ideas how to improve ix/intel's performance? >=20 > Are you sure about netapp performance? > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net = > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org = " >=20 From owner-freebsd-stable@freebsd.org Mon Aug 17 11:39:27 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4908C9BB901; Mon, 17 Aug 2015 11:39:27 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F0F8F16A8; Mon, 17 Aug 2015 11:39:26 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.84 (FreeBSD)) (envelope-from ) id 1ZRIl9-000K58-9v; Mon, 17 Aug 2015 14:39:23 +0300 Date: Mon, 17 Aug 2015 14:39:23 +0300 From: Slawa Olhovchenkov To: Daniel Braniss Cc: FreeBSD stable , FreeBSD Net Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance Message-ID: <20150817113923.GK1872@zxy.spb.ru> References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <20150817094145.GB3158@zxy.spb.ru> <197995E2-0C11-43A2-AB30-FBB0FB8CE2C5@cs.huji.ac.il> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <197995E2-0C11-43A2-AB30-FBB0FB8CE2C5@cs.huji.ac.il> User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2015 11:39:27 -0000 On Mon, Aug 17, 2015 at 01:35:06PM +0300, Daniel Braniss wrote: > > > On Aug 17, 2015, at 12:41 PM, Slawa Olhovchenkov wrote: > > > > On Mon, Aug 17, 2015 at 10:27:41AM +0300, Daniel Braniss wrote: > > > >> hi, > >> I have a host (Dell R730) with both cards, connected to an HP8200 switch at 10Gb. > >> when writing to the same storage (netapp) this is what I get: > >> ix0: ~130MGB/s > >> mlxen0 ~330MGB/s > >> this is via nfs/tcpv3 > >> > >> I can get similar (bad) performance with the mellanox if I increase the file size > >> to 512MGB. > > > > Look like mellanox have internal beffer for caching and do ACK acclerating. > what ever they are doing, it's impressive :-) > > > > >> so at face value, it seems the mlxen does a better use of resources than the intel. > >> Any ideas how to improve ix/intel's performance? > > > > Are you sure about netapp performance? > > yes, and why should it act differently if the request is coming from the same host? in any case > the numbers are quiet consistent since I have measured it from several hosts, and at different times. In any case, for 10Gb expect about 1200MGB/s. I see lesser speed. What netapp maximum performance? From other hosts, or local, any? From owner-freebsd-stable@freebsd.org Mon Aug 17 11:49:28 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 572029BBB27; Mon, 17 Aug 2015 11:49:28 +0000 (UTC) (envelope-from haramrae@gmail.com) Received: from mail-io0-x236.google.com (mail-io0-x236.google.com [IPv6:2607:f8b0:4001:c06::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 243B31B23; Mon, 17 Aug 2015 11:49:28 +0000 (UTC) (envelope-from haramrae@gmail.com) Received: by iodt126 with SMTP id t126so148269275iod.2; Mon, 17 Aug 2015 04:49:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=9GXDOBqwjOHp180geY81O4hOqReO4F4gNo+V2nRhhSY=; b=EkpHRuSOXM/xXwBY11wO0UGT2r81htP3dcxr2O8GbA598NImLx9ONkxtloVjIr8yjQ y3KpPD3BdBhlcj7yz7fHnNndNEHdrRc+LaobiJ4VzSYTM+QGWqIYrnyc3/J8FvCIF+6E uckkCU66O4t2pLuUrYKXa7QkNMe/pxbyYmXYO+t8jSAmZIpyXRM/1qmHhDWuvLddNT9M CDxkJL3Xv0RbNlKUKWt9/SQBBTxLjweFGG3azq63wvD4MJ6JE0ONLOTfISDR896Cr5VZ V0cncw2VA/uG7I7alTMF8sdcaoZbzXVHG3a1/PfUGbtLP53K+ZfzEKgJB1ih60np1CD9 FlgQ== MIME-Version: 1.0 X-Received: by 10.107.169.215 with SMTP id f84mr1071456ioj.42.1439812167503; Mon, 17 Aug 2015 04:49:27 -0700 (PDT) Received: by 10.64.80.197 with HTTP; Mon, 17 Aug 2015 04:49:27 -0700 (PDT) In-Reply-To: <20150817113923.GK1872@zxy.spb.ru> References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <20150817094145.GB3158@zxy.spb.ru> <197995E2-0C11-43A2-AB30-FBB0FB8CE2C5@cs.huji.ac.il> <20150817113923.GK1872@zxy.spb.ru> Date: Mon, 17 Aug 2015 13:49:27 +0200 Message-ID: Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance From: Alban Hertroys To: Slawa Olhovchenkov Cc: Daniel Braniss , FreeBSD Net , FreeBSD stable Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2015 11:49:28 -0000 On 17 August 2015 at 13:39, Slawa Olhovchenkov wrote: > In any case, for 10Gb expect about 1200MGB/s. Your usage of units is confusing. Above you claim you expect 1200 million gigabytes per second, or 1.2 * 10^18 Bytes/s. I don't think any known network interface can do that, including highly experimental ones. I suspect you intended to claim that you expect 1.2GB/s (Gigabytes per second) over that 10Gb/s (Gigabits per second) network. That's still on the high side of what's possible. On TCP/IP there is some TCP overhead, so 1.0 GB/s is probably more realistic. WRT the actual problem you're trying to solve, I'm no help there. -- If you can't see the forest for the trees, Cut the trees and you'll see there is no forest. From owner-freebsd-stable@freebsd.org Mon Aug 17 11:54:08 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 809BD9BBCF5; Mon, 17 Aug 2015 11:54:08 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 39F6F1F3D; Mon, 17 Aug 2015 11:54:08 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.84 (FreeBSD)) (envelope-from ) id 1ZRIzN-000KLs-HV; Mon, 17 Aug 2015 14:54:05 +0300 Date: Mon, 17 Aug 2015 14:54:05 +0300 From: Slawa Olhovchenkov To: Alban Hertroys Cc: Daniel Braniss , FreeBSD Net , FreeBSD stable Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance Message-ID: <20150817115405.GL1872@zxy.spb.ru> References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <20150817094145.GB3158@zxy.spb.ru> <197995E2-0C11-43A2-AB30-FBB0FB8CE2C5@cs.huji.ac.il> <20150817113923.GK1872@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2015 11:54:08 -0000 On Mon, Aug 17, 2015 at 01:49:27PM +0200, Alban Hertroys wrote: > On 17 August 2015 at 13:39, Slawa Olhovchenkov wrote: > > > In any case, for 10Gb expect about 1200MGB/s. > > Your usage of units is confusing. Above you claim you expect 1200 I am use as topic starter and expect MeGaBytes per second > million gigabytes per second, or 1.2 * 10^18 Bytes/s. I don't think > any known network interface can do that, including highly experimental > ones. > > I suspect you intended to claim that you expect 1.2GB/s (Gigabytes per > second) over that 10Gb/s (Gigabits per second) network. > That's still on the high side of what's possible. On TCP/IP there is > some TCP overhead, so 1.0 GB/s is probably more realistic. TCP give 5-7% overhead (include retrasmits). 10^9/8*0.97 = 1.2125 From owner-freebsd-stable@freebsd.org Mon Aug 17 12:11:03 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9F8DB9B9510 for ; Mon, 17 Aug 2015 12:11:03 +0000 (UTC) (envelope-from rb@gid.co.uk) Received: from mx0.gid.co.uk (mx0.gid.co.uk [194.32.164.250]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2E0F5190F for ; Mon, 17 Aug 2015 12:11:02 +0000 (UTC) (envelope-from rb@gid.co.uk) Received: from [194.32.164.24] (80-46-130-69.static.dsl.as9105.com [80.46.130.69]) by mx0.gid.co.uk (8.14.2/8.14.2) with ESMTP id t7HCAuBi010123; Mon, 17 Aug 2015 13:10:57 +0100 (BST) (envelope-from rb@gid.co.uk) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) Subject: Re: freebsd-update to 10.2-RELEASE broken ? From: Bob Bishop In-Reply-To: Date: Mon, 17 Aug 2015 13:10:51 +0100 Cc: Christian Kratzer , "freebsd-stable@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <37B04206-8CC2-4FD5-BF11-19073BAAC36C@gid.co.uk> References: <2C3CC22D-749A-4B92-885C-D73311997050@gid.co.uk> <20150816180715.GM40589@home.opsec.eu> <20150816200703.GN40589@home.opsec.eu> To: "Bjoern A. Zeeb" X-Mailer: Apple Mail (2.2104) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2015 12:11:03 -0000 > On 16 Aug 2015, at 23:08, Bjoern A. Zeeb = wrote: >=20 >=20 >> On 16 Aug 2015, at 21:16 , Christian Kratzer = wrote: >>=20 >> Hi, >>=20 >> On Sun, 16 Aug 2015, Kurt Jaeger wrote: >>> Hi! >>>=20 >>>> It could be the classic fall back to TCP on SRV records problem on >>>> your upstream DNS forwarder if you're using one: >>>>=20 >>>> = http://lists.freebsd.org/pipermail/freebsd-ports/2012-May/074801.html >>>=20 >>> If I query that same DNS resolver using the line from >>> the script, it works every time. It's a 10.1p16 host with >>> a very recent ports build, and directly connected (no NAT, no >>> fw etc). >>>=20 >>> If that would be the problem, how could I diagnose it in depth ? >>=20 >> freebsd-update upgrade just failed on 3 other vm even when I = explicitly >> specified the server using freebsd-update -s. >>=20 >> I had success on another vm when I changed to using google dns. >>=20 >> I am not aware that anything would be blocking tcp dns in my setups. >>=20 >> Must be something else dns related. >>=20 >> Perhaps I will run a local resolver in a vm and logg all queries and = dns >> traffic. >=20 > Or run tcpdump for port 53; also curious if it might be an IPv4 vs. = IPv6 issue? I saw the issue on machines with IPv4/IPv6 and IPv4 only. > =E2=80=94=20 > Bjoern A. Zeeb Charles Haddon = Spurgeon: > "Friendship is one of the sweetest joys of life. Many might have = failed > beneath the bitterness of their trial had they not found a friend." >=20 > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to = "freebsd-stable-unsubscribe@freebsd.org" -- Bob Bishop +44 (0)118 940 1243 rb@gid.co.uk fax +44 (0)118 940 1295 mobile +44 (0)783 626 4518 From owner-freebsd-stable@freebsd.org Mon Aug 17 12:15:39 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 354E09B960B for ; Mon, 17 Aug 2015 12:15:39 +0000 (UTC) (envelope-from ck-lists@cksoft.de) Received: from mx1.cksoft.de (mx1.cksoft.de [IPv6:2001:67c:24f8:1::25:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.cksoft.de", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id E3F341CD1 for ; Mon, 17 Aug 2015 12:15:38 +0000 (UTC) (envelope-from ck-lists@cksoft.de) Received: from m.cksoft.de (unknown [IPv6:2a01:170:1110:8001::25:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.cksoft.de (Postfix) with ESMTPSA id 706F21E9E6E; Mon, 17 Aug 2015 14:15:35 +0200 (CEST) Received: from amavis.cksoft.de (unknown [IPv6:2a01:170:1110:8001::25:a1]) by m.cksoft.de (Postfix) with ESMTP id 5150262F88; Mon, 17 Aug 2015 14:14:33 +0200 (CEST) X-Virus-Scanned: amavisd-new at cksoft.de Received: from m.cksoft.de ([192.168.64.93]) by amavis.cksoft.de (amavis.cksoft.de [192.168.64.94]) (amavisd-new, port 10041) with ESMTP id as8lBfmAOYck; Mon, 17 Aug 2015 14:14:32 +0200 (CEST) Received: from noc1.cksoft.de (noc1.cksoft.de [IPv6:2a01:170:1110:8001::53:1]) by m.cksoft.de (Postfix) with ESMTP id 46C0662ECF; Mon, 17 Aug 2015 14:14:32 +0200 (CEST) Received: by noc1.cksoft.de (Postfix, from userid 1000) id BC91213BBF; Mon, 17 Aug 2015 14:15:33 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by noc1.cksoft.de (Postfix) with ESMTP id 87FE313B1F; Mon, 17 Aug 2015 14:15:33 +0200 (CEST) Date: Mon, 17 Aug 2015 14:15:33 +0200 (CEST) From: Christian Kratzer X-X-Sender: ck@noc1.cksoft.de Reply-To: Christian Kratzer To: Bob Bishop cc: "Bjoern A. Zeeb" , "freebsd-stable@freebsd.org" Subject: Re: freebsd-update to 10.2-RELEASE broken ? In-Reply-To: <37B04206-8CC2-4FD5-BF11-19073BAAC36C@gid.co.uk> Message-ID: References: <2C3CC22D-749A-4B92-885C-D73311997050@gid.co.uk> <20150816180715.GM40589@home.opsec.eu> <20150816200703.GN40589@home.opsec.eu> <37B04206-8CC2-4FD5-BF11-19073BAAC36C@gid.co.uk> User-Agent: Alpine 2.20 (BSF 67 2015-01-07) X-Spammer-Kill-Ratio: 75% MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2015 12:15:39 -0000 Hi, On Mon, 17 Aug 2015, Bob Bishop wrote: >> Or run tcpdump for port 53; also curious if it might be an IPv4 vs. IPv6 issue? > > I saw the issue on machines with IPv4/IPv6 and IPv4 only. i have tried with preference set to ipv4 and ipv6. Both fail similarly. Have not tried ipv4 or ipv6 only though. I had success with using google dns 8.8.8.8 as revolver on two machines/VM. But this hack has also since failed on other machines/VM. Greetings Christian -- Christian Kratzer CK Software GmbH Email: ck@cksoft.de Wildberger Weg 24/2 Phone: +49 7032 893 997 - 0 D-71126 Gaeufelden Fax: +49 7032 893 997 - 9 HRB 245288, Amtsgericht Stuttgart Mobile: +49 171 1947 843 Geschaeftsfuehrer: Christian Kratzer Web: http://www.cksoft.de/ From owner-freebsd-stable@freebsd.org Mon Aug 17 12:21:21 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CADDD9B982E; Mon, 17 Aug 2015 12:21:21 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 6F4F410BD; Mon, 17 Aug 2015 12:21:20 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:YM7nTxamSOtZlVGynGzpRfT/LSx+4OfEezUN459isYplN5qZpMi7bnLW6fgltlLVR4KTs6sC0LqN9fy6EjBbqb+681k8M7V0HycfjssXmwFySOWkMmbcaMDQUiohAc5ZX0Vk9XzoeWJcGcL5ekGA6ibqtW1aJBzzOEJPK/jvHcaK1oLsh7v0p8eYP14ArQH+SI0xBS3+lR/WuMgSjNkqAYcK4TyNnEF1ff9Lz3hjP1OZkkW0zM6x+Jl+73YY4Kp5pIZoGJ/3dKUgTLFeEC9ucyVsvJWq5lH+SxCS7C4cTnkOiUgPRAzE9w3hGJnrvybwreY73zOVesj/TLQxUDLl66ZwVB7uhiBAOSQ0/WvMholrkKtRpB/ymxsq74fSYYyRfNBkd6XcZshSEWZIWMBAfydaRIOhbYpJBuFHPOIO/KfnoF5blxq1BkGJDejszjJNzivs2KQx0OAsFCnb2wM9EtYWsDLfpYOmZ+8pTempwfyQnn34ZPRM1GK4sdCQfw== X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2BFAgDX0NFV/61jaINdDoNhaQaDHrpFAQmBawqFL0oCgWYUAQEBAQEBAQGBCYIdggYBAQEDAQEBASArIAsQAgEIDgoCAg0WAwICIQYBCRURAgQIBwQBHASHeAMKCA26CI9pDYVXAQEBAQEBBAEBAQEBARgEgSKKMIJPgWgBAQcVATMHgmmBQwWHIo17hQSFBnWDN5Eng0+DZQImgz9aIjMHfwgXI4EEAQEB X-IronPort-AV: E=Sophos;i="5.15,694,1432612800"; d="scan'208";a="231258543" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 17 Aug 2015 08:21:14 -0400 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 3283815F577; Mon, 17 Aug 2015 08:21:14 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id PbWpkVNngp3n; Mon, 17 Aug 2015 08:21:13 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 9000215F578; Mon, 17 Aug 2015 08:21:13 -0400 (EDT) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id N6GgvCEtRE1Y; Mon, 17 Aug 2015 08:21:13 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 6426915F577; Mon, 17 Aug 2015 08:21:13 -0400 (EDT) Date: Mon, 17 Aug 2015 08:21:12 -0400 (EDT) From: Rick Macklem To: Daniel Braniss Cc: Christopher Forgeron , FreeBSD Net , FreeBSD stable , Slawa Olhovchenkov Message-ID: <473274181.23263108.1439814072514.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <17871443-E105-4434-80B1-6939306A865F@cs.huji.ac.il> References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <20150817094145.GB3158@zxy.spb.ru> <17871443-E105-4434-80B1-6939306A865F@cs.huji.ac.il> Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [172.17.95.10] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF34 (Win)/8.0.9_GA_6191) Thread-Topic: ix(intel) vs mlxen(mellanox) 10Gb performance Thread-Index: tmoW4T+6Z7dNXU5bN4I24LLlQtGA7w== X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2015 12:21:21 -0000 Daniel Braniss wrote: >=20 > > On Aug 17, 2015, at 1:41 PM, Christopher Forgeron > > wrote: > >=20 > > FYI, I can regularly hit 9.3 Gib/s with my Intel X520-DA2's and FreeBSD > > 10.1. Before 10.1 it was less. > >=20 >=20 > this is NOT iperf/3 where i do get close to wire speed, > it=E2=80=99s NFS writes, i.e., almost real work :-) >=20 > > I used to tweak the card settings, but now it's just stock. You may wan= t to > > check your settings, the Mellanox may just have better defaults for you= r > > switch. > >=20 Have you tried disabling TSO for the Intel? With TSO enabled, it will be co= pying every transmitted mbuf chain to a new chain of mbuf clusters via. m_defrag(= ) when TSO is enabled. (Assuming you aren't an 82598 chip. Most seem to be the 825= 99 chip these days?) This has been fixed in the driver very recently, but those fixes won't be i= n 10.1. rick ps: If you could test with 10.2, it would be interesting to see how the ix = does with the current driver fixes in it? > > On Mon, Aug 17, 2015 at 6:41 AM, Slawa Olhovchenkov > > wrote: > > On Mon, Aug 17, 2015 at 10:27:41AM +0300, Daniel Braniss wrote: > >=20 > > > hi, > > > I have a host (Dell R730) with both cards, connected to an HP82= 00 > > > switch at 10Gb. > > > when writing to the same storage (netapp) this is what I get: > > > ix0: ~130MGB/s > > > mlxen0 ~330MGB/s > > > this is via nfs/tcpv3 > > > > > > I can get similar (bad) performance with the mellanox if I incr= ease > > > the file size > > > to 512MGB. > >=20 > > Look like mellanox have internal beffer for caching and do ACK acclerat= ing. > >=20 > > > so at face value, it seems the mlxen does a better use of resou= rces > > > than the intel. > > > Any ideas how to improve ix/intel's performance? > >=20 > > Are you sure about netapp performance? > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-net > > > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org > > " > >=20 >=20 > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Mon Aug 17 12:28:38 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 621A49B998E; Mon, 17 Aug 2015 12:28:38 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1FA7116AE; Mon, 17 Aug 2015 12:28:38 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.84 (FreeBSD)) (envelope-from ) id 1ZRJWm-000Kyn-7S; Mon, 17 Aug 2015 15:28:36 +0300 Date: Mon, 17 Aug 2015 15:28:36 +0300 From: Slawa Olhovchenkov To: Daniel Braniss Cc: FreeBSD stable , FreeBSD Net Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance Message-ID: <20150817122836.GC3158@zxy.spb.ru> References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2015 12:28:38 -0000 On Mon, Aug 17, 2015 at 10:27:41AM +0300, Daniel Braniss wrote: > hi, > I have a host (Dell R730) with both cards, connected to an HP8200 switch at 10Gb. > when writing to the same storage (netapp) this is what I get: > ix0: ~130MGB/s > mlxen0 ~330MGB/s > this is via nfs/tcpv3 > > I can get similar (bad) performance with the mellanox if I increase the file size > to 512MGB. > so at face value, it seems the mlxen does a better use of resources than the intel. > Any ideas how to improve ix/intel's performance? Any way, please show OS version /var/run/dmesg.boot What's tuning perfomed (loader.conf, sysctl.conf)? top -PHS in both cases ifconfig -a in both cases netstat -rn in both cases I am don't know netapp -- what is hardware configuration (disks and etc) and software tuning (MTU?). From owner-freebsd-stable@freebsd.org Mon Aug 17 13:27:45 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D537E9BB336 for ; Mon, 17 Aug 2015 13:27:45 +0000 (UTC) (envelope-from Mark.Martinec@ijs.si) Received: from mail.ijs.si (mail.ijs.si [IPv6:2001:1470:ff80::25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8883C1635 for ; Mon, 17 Aug 2015 13:27:45 +0000 (UTC) (envelope-from Mark.Martinec@ijs.si) Received: from amavis-ori.ijs.si (localhost [IPv6:::1]) by mail.ijs.si (Postfix) with ESMTP id 3mvx4t1hlbz120; Mon, 17 Aug 2015 15:27:42 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ijs.si; h= user-agent:message-id:references:in-reply-to:organization :subject:subject:from:from:date:date:content-transfer-encoding :content-type:content-type:mime-version:received:received :received:received; s=jakla4; t=1439818059; x=1442410060; bh=Cbt 7081f2ZWeGIjIjfLNHxaWJJ5HgNzFIKnax2GbKJY=; b=dLtV5nDc1tuSoHiZpo0 RWNjdOxLvI/dCmSEPgTQuoKSfeJiHQqjSrGJTl71j4AudvoYtNEedh1P8XJopnQn Hm/RGdtNnyv69ofKxxU9vbGzC20zojXJ65mVq9PtqjzHJL6kkohX1cOkVxMoIl8E g/2yPX/nQZgA4PrDVmDEkJiQ= X-Virus-Scanned: amavisd-new at ijs.si Received: from mail.ijs.si ([IPv6:::1]) by amavis-ori.ijs.si (mail.ijs.si [IPv6:::1]) (amavisd-new, port 10026) with LMTP id C4ZbmLotii6c; Mon, 17 Aug 2015 15:27:39 +0200 (CEST) Received: from mildred.ijs.si (mailbox.ijs.si [IPv6:2001:1470:ff80::143:1]) by mail.ijs.si (Postfix) with ESMTP id 3mvx4q1z5tz11y; Mon, 17 Aug 2015 15:27:39 +0200 (CEST) Received: from nabiralnik.ijs.si (mailbox.ijs.si [IPv6:2001:1470:ff80::143:1]) by mildred.ijs.si (Postfix) with ESMTP id 3mvx4q09wpz1Q; Mon, 17 Aug 2015 15:27:39 +0200 (CEST) Received: from sleepy.ijs.si ([2001:1470:ff80:e001::1:1]) by nabiralnik.ijs.si with HTTP (HTTP/1.1 POST); Mon, 17 Aug 2015 15:27:39 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Mon, 17 Aug 2015 15:27:39 +0200 From: Mark Martinec To: Christian Kratzer Cc: freebsd-stable@freebsd.org Subject: Re: freebsd-update to 10.2-RELEASE broken ? Organization: Jozef Stefan Institute In-Reply-To: References: <2C3CC22D-749A-4B92-885C-D73311997050@gid.co.uk> <20150816180715.GM40589@home.opsec.eu> Message-ID: <9c6d132a96b08934c440300a43cf3ef4@mailbox.ijs.si> X-Sender: Mark.Martinec@ijs.si User-Agent: Roundcube Webmail/1.1.2 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2015 13:27:45 -0000 > On Sun, 16 Aug 2015, Kimmo Paasiala wrote: >> It could be the classic fall back to TCP on SRV records problem on >> your upstream DNS forwarder if you're using one: >> http://lists.freebsd.org/pipermail/freebsd-ports/2012-May/074801.html >> >> The cure would be to use your own caching DNS resolver (configured to >> query the authoritative name servers directly) such as dns/unbound. 2015-08-16 Christian Kratzer wrote: > I run my own bind9 resolvers on freebsd 10 at both sites. I never > particurlarly like the concept of an "upstream" resolver. > > All my resolvers are behind firewalls although different kinds. > ASA at one site and freebsd pf at the other. > > I will investigate though. Thanks for the tip. ASA firewall has a nasty setting to *discard* DNS UDP packets with UDP message size over 512 bytes, i.e. it does not allow EDNS0 option. Check that you have this DNS deep packet inspection misfeature turned off. Check also the firewall log. This would affect UDP DNS responses to a SRV query _http._tcp.update.FreeBSD.org, which comes close to the size limit (possibly depending on geolocation). Using google's public DNS server may avoid the problem by stripping nonessential records from the DNS reply (like the ADDITIONAL SECTION). Mark From owner-freebsd-stable@freebsd.org Mon Aug 17 13:28:30 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E43039BB379 for ; Mon, 17 Aug 2015 13:28:29 +0000 (UTC) (envelope-from Mark.Martinec+freebsd@ijs.si) Received: from mail.ijs.si (mail.ijs.si [IPv6:2001:1470:ff80::25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 957841741 for ; Mon, 17 Aug 2015 13:28:29 +0000 (UTC) (envelope-from Mark.Martinec+freebsd@ijs.si) Received: from amavis-ori.ijs.si (localhost [IPv6:::1]) by mail.ijs.si (Postfix) with ESMTP id 3mvx5l5gfmz12L for ; Mon, 17 Aug 2015 15:28:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ijs.si; h= user-agent:message-id:references:in-reply-to:organization :subject:subject:from:from:date:date:content-transfer-encoding :content-type:content-type:mime-version:received:received :received:received; s=jakla4; t=1439818103; x=1442410104; bh=Cbt 7081f2ZWeGIjIjfLNHxaWJJ5HgNzFIKnax2GbKJY=; b=jhx7TKecP8UPEcpQGgE dFZmZ0kA6laxdYGwwmXB5JzXfEYDcOupa/ePtT9HfAFkZNpiRgE0C9XlFEdprhIw EnNEA0/bQqVJwgqitejsjZKh9zusZZQnKJ8gEoetjuxeC7PHUP4Zy0iClX+19f7R Tr0EZv0O2jyuEJoDZM3nDROM= X-Virus-Scanned: amavisd-new at ijs.si Received: from mail.ijs.si ([IPv6:::1]) by amavis-ori.ijs.si (mail.ijs.si [IPv6:::1]) (amavisd-new, port 10026) with LMTP id AXHll9A4wLns for ; Mon, 17 Aug 2015 15:28:23 +0200 (CEST) Received: from mildred.ijs.si (mailbox.ijs.si [IPv6:2001:1470:ff80::143:1]) by mail.ijs.si (Postfix) with ESMTP id 3mvx5g0Ht6z12G for ; Mon, 17 Aug 2015 15:28:23 +0200 (CEST) Received: from nabiralnik.ijs.si (mailbox.ijs.si [IPv6:2001:1470:ff80::143:1]) by mildred.ijs.si (Postfix) with ESMTP id 3mvx5f6XFsz1g for ; Mon, 17 Aug 2015 15:28:22 +0200 (CEST) Received: from sleepy.ijs.si ([2001:1470:ff80:e001::1:1]) by nabiralnik.ijs.si with HTTP (HTTP/1.1 POST); Mon, 17 Aug 2015 15:28:22 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Mon, 17 Aug 2015 15:28:22 +0200 From: Mark Martinec To: freebsd-stable@freebsd.org Subject: Re: freebsd-update to 10.2-RELEASE broken ? Organization: Jozef Stefan Institute In-Reply-To: References: <2C3CC22D-749A-4B92-885C-D73311997050@gid.co.uk> <20150816180715.GM40589@home.opsec.eu> Message-ID: <11b6542dbdfdb5ee7eefcba48fb07e16@mailbox.ijs.si> X-Sender: Mark.Martinec+freebsd@ijs.si User-Agent: Roundcube Webmail/1.1.2 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2015 13:28:30 -0000 > On Sun, 16 Aug 2015, Kimmo Paasiala wrote: >> It could be the classic fall back to TCP on SRV records problem on >> your upstream DNS forwarder if you're using one: >> http://lists.freebsd.org/pipermail/freebsd-ports/2012-May/074801.html >> >> The cure would be to use your own caching DNS resolver (configured to >> query the authoritative name servers directly) such as dns/unbound. 2015-08-16 Christian Kratzer wrote: > I run my own bind9 resolvers on freebsd 10 at both sites. I never > particurlarly like the concept of an "upstream" resolver. > > All my resolvers are behind firewalls although different kinds. > ASA at one site and freebsd pf at the other. > > I will investigate though. Thanks for the tip. ASA firewall has a nasty setting to *discard* DNS UDP packets with UDP message size over 512 bytes, i.e. it does not allow EDNS0 option. Check that you have this DNS deep packet inspection misfeature turned off. Check also the firewall log. This would affect UDP DNS responses to a SRV query _http._tcp.update.FreeBSD.org, which comes close to the size limit (possibly depending on geolocation). Using google's public DNS server may avoid the problem by stripping nonessential records from the DNS reply (like the ADDITIONAL SECTION). Mark From owner-freebsd-stable@freebsd.org Mon Aug 17 13:29:24 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2C6359BB3EF; Mon, 17 Aug 2015 13:29:24 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.116.210]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D417E1881; Mon, 17 Aug 2015 13:29:23 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from mbpro-w.cs.huji.ac.il ([132.65.80.91]) by kabab.cs.huji.ac.il with esmtp id 1ZRKTY-0002fM-G3; Mon, 17 Aug 2015 16:29:20 +0300 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2102\)) Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance From: Daniel Braniss In-Reply-To: <473274181.23263108.1439814072514.JavaMail.zimbra@uoguelph.ca> Date: Mon, 17 Aug 2015 16:29:20 +0300 Cc: Christopher Forgeron , FreeBSD Net , FreeBSD stable , Slawa Olhovchenkov Content-Transfer-Encoding: quoted-printable Message-Id: <7F892C70-9C04-4468-9514-EDBFE75CF2C6@cs.huji.ac.il> References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <20150817094145.GB3158@zxy.spb.ru> <17871443-E105-4434-80B1-6939306A865F@cs.huji.ac.il> <473274181.23263108.1439814072514.JavaMail.zimbra@uoguelph.ca> To: Rick Macklem X-Mailer: Apple Mail (2.2102) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2015 13:29:24 -0000 > On Aug 17, 2015, at 3:21 PM, Rick Macklem = wrote: >=20 > Daniel Braniss wrote: >>=20 >>> On Aug 17, 2015, at 1:41 PM, Christopher Forgeron = >>> wrote: >>>=20 >>> FYI, I can regularly hit 9.3 Gib/s with my Intel X520-DA2's and = FreeBSD >>> 10.1. Before 10.1 it was less. >>>=20 >>=20 >> this is NOT iperf/3 where i do get close to wire speed, >> it=E2=80=99s NFS writes, i.e., almost real work :-) >>=20 >>> I used to tweak the card settings, but now it's just stock. You may = want to >>> check your settings, the Mellanox may just have better defaults for = your >>> switch. >>>=20 > Have you tried disabling TSO for the Intel? With TSO enabled, it will = be copying > every transmitted mbuf chain to a new chain of mbuf clusters via. = m_defrag() when > TSO is enabled. (Assuming you aren't an 82598 chip. Most seem to be = the 82599 chip > these days?) >=20 hi Rick how can i check the chip? > This has been fixed in the driver very recently, but those fixes won't = be in 10.1. >=20 > rick > ps: If you could test with 10.2, it would be interesting to see how = the ix does with > the current driver fixes in it? I new TSO was involved!=20 ok, firstly, it=E2=80=99s 10.2 stable. with TSO enabled, ix is bad, around 64MGB/s. disabling TSO it=E2=80=99s better, around 130 still, mlxen0 is about 250! with and without TSO >=20 >>> On Mon, Aug 17, 2015 at 6:41 AM, Slawa Olhovchenkov >> > wrote: >>> On Mon, Aug 17, 2015 at 10:27:41AM +0300, Daniel Braniss wrote: >>>=20 >>>> hi, >>>> I have a host (Dell R730) with both cards, connected to an = HP8200 >>>> switch at 10Gb. >>>> when writing to the same storage (netapp) this is what I get: >>>> ix0: ~130MGB/s >>>> mlxen0 ~330MGB/s >>>> this is via nfs/tcpv3 >>>>=20 >>>> I can get similar (bad) performance with the mellanox if I = increase >>>> the file size >>>> to 512MGB. >>>=20 >>> Look like mellanox have internal beffer for caching and do ACK = acclerating. >>>=20 >>>> so at face value, it seems the mlxen does a better use of = resources >>>> than the intel. >>>> Any ideas how to improve ix/intel's performance? >>>=20 >>> Are you sure about netapp performance? >>> _______________________________________________ >>> freebsd-net@freebsd.org mailing = list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-net >>> >>> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org >>> " >>>=20 >>=20 >> _______________________________________________ >> freebsd-stable@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to = "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Mon Aug 17 15:44:38 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 33D049BBFC5; Mon, 17 Aug 2015 15:44:38 +0000 (UTC) (envelope-from haramrae@gmail.com) Received: from mail-io0-x233.google.com (mail-io0-x233.google.com [IPv6:2607:f8b0:4001:c06::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EF46F1647; Mon, 17 Aug 2015 15:44:37 +0000 (UTC) (envelope-from haramrae@gmail.com) Received: by iods203 with SMTP id s203so156015012iod.0; Mon, 17 Aug 2015 08:44:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=j/eS1ITatBCtcp/wsRl5hVp7ISn540RkaGwxK3hNflo=; b=VglJYEwaPEp3ZXnd/a6DatLJOosZE4TVCTgnRRnDRmiymDOwtP+csGGhprqQKap0yJ eVejujHCBVKa5F+BnHeUw6242XVbqTXkLrPenwGwnOZHRcrJPvb/jt4TIQW+4zYG0SDv jI5yquL8Mt5EAbUcZRoegYQXA2XFk0g4/8PTUDD9Pl3IfB3iR8XfuBk+Vd96QFjYnEAc p04Np8CdfuwQtpz4Y4pY+TUi69GuZRgxcmvAaXeAn1hvkLNDldltngUojoMS6oJKyhcr BocNqkNzOL4pRdPFI6+ka33rch4h5OnwXP4B5QMnR7v7h6HzlV8n44opaT0JgeQGqiBD OOxg== MIME-Version: 1.0 X-Received: by 10.107.169.215 with SMTP id f84mr2330734ioj.42.1439826277490; Mon, 17 Aug 2015 08:44:37 -0700 (PDT) Received: by 10.64.80.197 with HTTP; Mon, 17 Aug 2015 08:44:37 -0700 (PDT) In-Reply-To: <20150817115405.GL1872@zxy.spb.ru> References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <20150817094145.GB3158@zxy.spb.ru> <197995E2-0C11-43A2-AB30-FBB0FB8CE2C5@cs.huji.ac.il> <20150817113923.GK1872@zxy.spb.ru> <20150817115405.GL1872@zxy.spb.ru> Date: Mon, 17 Aug 2015 17:44:37 +0200 Message-ID: Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance From: Alban Hertroys To: Slawa Olhovchenkov Cc: Daniel Braniss , FreeBSD Net , FreeBSD stable Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2015 15:44:38 -0000 On 17 August 2015 at 13:54, Slawa Olhovchenkov wrote: > On Mon, Aug 17, 2015 at 01:49:27PM +0200, Alban Hertroys wrote: > >> On 17 August 2015 at 13:39, Slawa Olhovchenkov wrote: >> >> > In any case, for 10Gb expect about 1200MGB/s. >> >> Your usage of units is confusing. Above you claim you expect 1200 > > I am use as topic starter and expect MeGaBytes per second That's a highly unusual way of writing MB/s. There are standards for unit prefixes: k means kilo, M means Mega, G means Giga, etc. See: https://en.wikipedia.org/wiki/International_System_of_Units#Prefixes >> million gigabytes per second, or 1.2 * 10^18 Bytes/s. I don't think >> any known network interface can do that, including highly experimental >> ones. >> >> I suspect you intended to claim that you expect 1.2GB/s (Gigabytes per >> second) over that 10Gb/s (Gigabits per second) network. >> That's still on the high side of what's possible. On TCP/IP there is >> some TCP overhead, so 1.0 GB/s is probably more realistic. > > TCP give 5-7% overhead (include retrasmits). > 10^9/8*0.97 = 1.2125 In information science, Bytes are counted in multiples of 2, not 10. A kb is 1024 bits or 2^10 b. So 10 Gb is 10 * 2^30 bits. It's also not unusual to be more specific about that 2-base and use kib, Mib and Gib instead. Apparently you didn't know that... Also, if you take 5% off, you are left with (0.95 * 10 * 2^30) / 8 = 1.1875 B/s, not 0.97 * ... Your calculations were a bit optimistic. Now I have to admit I'm used to use a factor of 10 to convert from b/s to B/s (that's 20%!), but that's probably no longer correct, what with jumbo frames and all. -- If you can't see the forest for the trees, Cut the trees and you'll see there is no forest. From owner-freebsd-stable@freebsd.org Mon Aug 17 15:50:26 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8F2669BB192 for ; Mon, 17 Aug 2015 15:50:26 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4D96418AE for ; Mon, 17 Aug 2015 15:50:26 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.84 (FreeBSD)) (envelope-from ) id 1ZRMg2-0003Oh-WE; Mon, 17 Aug 2015 18:50:23 +0300 Date: Mon, 17 Aug 2015 18:50:22 +0300 From: Slawa Olhovchenkov To: Christian Kratzer Cc: freebsd-stable@freebsd.org Subject: Re: freebsd-update to 10.2-RELEASE broken ? Message-ID: <20150817155022.GD3158@zxy.spb.ru> References: MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2015 15:50:26 -0000 On Sun, Aug 16, 2015 at 07:16:03PM +0200, Christian Kratzer wrote: > Hi, > > I have been trying to update several of my FreeBSD 10.1 amd64 VM to 10.2-RELEASE with freebsd-update and have been failing with an incorrect hash error. > > This is what happens with a plain vanilla 10.1-RELEASE vm when I try to update to 10.2-RELEASE > > Fetching 2356 files... 068eb594e5f6b97554750a8321892e4c229a6f26455f40e4d8e4e7a79577c673 has incorrect hash. > root@test10:~ck # > --snipp-- > > I get the is on all kinds of VM with different patchlevels of 10.1-RELEASE. Some of them have /usr/src, some of them don't. > > The hash seems to be different every time round. > > Could this be an issue with one of the mirrors ? > > Has anybody had success yet with an update to 10.2-RELEASE using freebsd-update ? This is like update.FreeBSD.org in some time give random files. For examle: ============= Applying patches... done. Fetching 10832 files... 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7 has incorrect hash. [00:09:19] ====>> Error: Fail to upgrade system # gzip -cd /poudriere/jails/i386/var/db/freebsd-update/136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.gz > 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.bad # fetch http://update.FreeBSD.org/10.2-RELEASE/i386/f/136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.gz # gzip -cd 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.gz > 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.god # ls -ltr 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7* -rw-r--r-- 1 root wheel 713 Aug 13 03:24 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.gz -rw-r--r-- 1 root wheel 4960 Aug 17 18:37 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.bad -rw-r--r-- 1 root wheel 1278 Aug 17 18:38 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.god # cat 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.god /* * Copyright 2013 Eukr'ea Electromatique * * This program is free software; you can redistribute it and/or * modify it under the terms of the GNU General Public License * as published by the Free Software Foundation; either version 2 * of the License, or (at your option) any later version. * This program is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. */ #include "imx25-eukrea-mbimxsd25-baseboard.dts" / { model = "Eukrea MBIMXSD25 with the DVI-VGA Display"; [...] # cat 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.bad /* * Copyright (C) 2014 Thomas Petazzoni * Copyright (C) 2009 Simon Guinot * * This file is licensed under the terms of the GNU General Public * License version 2. This program is licensed "as is" without any * warranty of any kind, whether express or implied. */ /dts-v1/; #include #include #include #include "orion5x-mv88f5182.dtsi" / { model = "LaCie d2 Network"; compatible = "lacie,d2-network", "marvell,orion5x-88f5182", "marvell,orion5x"; [...] ============== secteam? admins? webmaster? From owner-freebsd-stable@freebsd.org Mon Aug 17 15:54:37 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D3E029BB33B; Mon, 17 Aug 2015 15:54:37 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id C0E2D1C58; Mon, 17 Aug 2015 15:54:37 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by freefall.freebsd.org (Postfix) with ESMTP id 25B03107A; Mon, 17 Aug 2015 15:54:37 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Date: Mon, 17 Aug 2015 15:54:34 +0000 From: Glen Barber To: Slawa Olhovchenkov Cc: Christian Kratzer , freebsd-stable@freebsd.org, FreeBSD Security Team Subject: Re: freebsd-update to 10.2-RELEASE broken ? Message-ID: <20150817155434.GT24069@FreeBSD.org> References: <20150817155022.GD3158@zxy.spb.ru> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="n1q7a47AT2c4WQJe" Content-Disposition: inline In-Reply-To: <20150817155022.GD3158@zxy.spb.ru> X-Operating-System: FreeBSD 11.0-CURRENT amd64 X-SCUD-Definition: Sudden Completely Unexpected Dataloss X-SULE-Definition: Sudden Unexpected Learning Event X-PEKBAC-Definition: Problem Exists, Keyboard Between Admin/Computer User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2015 15:54:37 -0000 --n1q7a47AT2c4WQJe Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Aug 17, 2015 at 06:50:22PM +0300, Slawa Olhovchenkov wrote: > On Sun, Aug 16, 2015 at 07:16:03PM +0200, Christian Kratzer wrote: >=20 > > Hi, > >=20 > > I have been trying to update several of my FreeBSD 10.1 amd64 VM to 10.= 2-RELEASE with freebsd-update and have been failing with an incorrect hash = error. > >=20 > > This is what happens with a plain vanilla 10.1-RELEASE vm when I try to= update to 10.2-RELEASE > >=20 > > Fetching 2356 files... 068eb594e5f6b97554750a8321892e4c229a6f26455f40e4= d8e4e7a79577c673 has incorrect hash. > > root@test10:~ck # > > --snipp-- > >=20 > > I get the is on all kinds of VM with different patchlevels of 10.1-RELE= ASE. Some of them have /usr/src, some of them don't. > >=20 > > The hash seems to be different every time round. > >=20 > > Could this be an issue with one of the mirrors ? > >=20 > > Has anybody had success yet with an update to 10.2-RELEASE using freebs= d-update ? >=20 > This is like update.FreeBSD.org in some time give random files. > For examle: >=20 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > Applying patches... done. > Fetching 10832 files... 136fcae80caaf0143057e16e29a946b62232686b54efd56a2= 848a4e7547017f7 has incorrect hash. > [00:09:19] =3D=3D=3D=3D>> Error: Fail to upgrade system >=20 > # gzip -cd /poudriere/jails/i386/var/db/freebsd-update/136fcae80caaf01430= 57e16e29a946b62232686b54efd56a2848a4e7547017f7.gz > 136fcae80caaf0143057e16= e29a946b62232686b54efd56a2848a4e7547017f7.bad > # fetch http://update.FreeBSD.org/10.2-RELEASE/i386/f/136fcae80caaf014305= 7e16e29a946b62232686b54efd56a2848a4e7547017f7.gz > # gzip -cd 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017= f7.gz > 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.god > # ls -ltr 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f= 7* > -rw-r--r-- 1 root wheel 713 Aug 13 03:24 136fcae80caaf0143057e16e29a9= 46b62232686b54efd56a2848a4e7547017f7.gz > -rw-r--r-- 1 root wheel 4960 Aug 17 18:37 136fcae80caaf0143057e16e29a9= 46b62232686b54efd56a2848a4e7547017f7.bad > -rw-r--r-- 1 root wheel 1278 Aug 17 18:38 136fcae80caaf0143057e16e29a9= 46b62232686b54efd56a2848a4e7547017f7.god > # cat 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.god > /* > * Copyright 2013 Eukr'ea Electromatique > * > * This program is free software; you can redistribute it and/or > * modify it under the terms of the GNU General Public License > * as published by the Free Software Foundation; either version 2 > * of the License, or (at your option) any later version. > * This program is distributed in the hope that it will be useful, > * but WITHOUT ANY WARRANTY; without even the implied warranty of > * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > * GNU General Public License for more details. > */ >=20 > #include "imx25-eukrea-mbimxsd25-baseboard.dts" >=20 > / { > model =3D "Eukrea MBIMXSD25 with the DVI-VGA Display"; > [...] > # cat 136fcae80caaf0143057e16e29a946b62232686b54efd56a2848a4e7547017f7.bad > /* > * Copyright (C) 2014 Thomas Petazzoni > * Copyright (C) 2009 Simon Guinot > * > * This file is licensed under the terms of the GNU General Public > * License version 2. This program is licensed "as is" without any > * warranty of any kind, whether express or implied. > */ >=20 > /dts-v1/; >=20 > #include > #include > #include > #include "orion5x-mv88f5182.dtsi" >=20 > / { > model =3D "LaCie d2 Network"; > compatible =3D "lacie,d2-network", "marvell,orion5x-88f5182", "ma= rvell,orion5x"; > [...] > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > secteam? > admins? > webmaster? Secteam. I've cc'd them. Glen --n1q7a47AT2c4WQJe Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJV0gO1AAoJEAMUWKVHj+KTHlwP/1qXP0E7+TRUkNdArWNmLN0E Ww3TVRrgVMKXxOXc97Zo7pRFouAbzkTrJnDJIwKdloD6Nul4uIpadF/f1VIgH/lv jnjsKHZRLRNGfC+81+A1hPQiE0Qqw2fp9G3K9yAwLDUaLB8WViT2ghwX+ZMNZyCF bya+oIfcRtxMEL8Z160fayJN80j6rPL6b/lzO4EP7iEfr//l/2r7+q0EJEdJQVtr eMD3Vv772hN9vTP0fru7JQljXN3mi/IWvJVRn2FdJ53Pu7TMuz1dswT8xgtJo1ZS phd5Go2K8TlltU6CR44653kyLWCzrLEKPpfmZJSThiTFNRtK0evS2Z8M45ZtWGoq kseC4UzlAi691hGO0Q7QcMwk3Epwjr1H6DV2Y4BzI/cEqz3XqoXuQKzaxm8n/qXa muJK+8geXAn/979zqSD2GG9Ee6z+DmeiF8wRka9Jujpiip5NNifxX52C3ZNBABC9 oV/CknSf57Ed36WtgGDmc5bz9jvzoZph+yFeO6HYsEwvfIDOccJAs/FYuUS34IHX SpHFCtn3WfNQDn6lN3B1GDqw99hWtU0Z0fw0hZncXhHW3LIrQJYFu2bg1BEnaKoY INjUIMaRIn132kGCY5Nowdqy7+0uZHKkRjv2tCIV5uYNIw92zHs3bD1a4mKlVa7u 7maZwkc+GLh9LYz6JGa/ =hCN2 -----END PGP SIGNATURE----- --n1q7a47AT2c4WQJe-- From owner-freebsd-stable@freebsd.org Mon Aug 17 16:01:49 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C4EE59BB506; Mon, 17 Aug 2015 16:01:49 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7F4C1104D; Mon, 17 Aug 2015 16:01:49 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.84 (FreeBSD)) (envelope-from ) id 1ZRMr3-0003cS-JL; Mon, 17 Aug 2015 19:01:45 +0300 Date: Mon, 17 Aug 2015 19:01:45 +0300 From: Slawa Olhovchenkov To: Alban Hertroys Cc: FreeBSD Net , FreeBSD stable Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance Message-ID: <20150817160145.GE3158@zxy.spb.ru> References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <20150817094145.GB3158@zxy.spb.ru> <197995E2-0C11-43A2-AB30-FBB0FB8CE2C5@cs.huji.ac.il> <20150817113923.GK1872@zxy.spb.ru> <20150817115405.GL1872@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2015 16:01:49 -0000 On Mon, Aug 17, 2015 at 05:44:37PM +0200, Alban Hertroys wrote: > On 17 August 2015 at 13:54, Slawa Olhovchenkov wrote: > > On Mon, Aug 17, 2015 at 01:49:27PM +0200, Alban Hertroys wrote: > > > >> On 17 August 2015 at 13:39, Slawa Olhovchenkov wrote: > >> > >> > In any case, for 10Gb expect about 1200MGB/s. > >> > >> Your usage of units is confusing. Above you claim you expect 1200 > > > > I am use as topic starter and expect MeGaBytes per second > > That's a highly unusual way of writing MB/s. I am know. This is do not care for me. > There are standards for unit prefixes: k means kilo, M means Mega, G > means Giga, etc. See: > https://en.wikipedia.org/wiki/International_System_of_Units#Prefixes > > >> million gigabytes per second, or 1.2 * 10^18 Bytes/s. I don't think > >> any known network interface can do that, including highly experimental > >> ones. > >> > >> I suspect you intended to claim that you expect 1.2GB/s (Gigabytes per > >> second) over that 10Gb/s (Gigabits per second) network. > >> That's still on the high side of what's possible. On TCP/IP there is > >> some TCP overhead, so 1.0 GB/s is probably more realistic. > > > > TCP give 5-7% overhead (include retrasmits). > > 10^9/8*0.97 = 1.2125 > > In information science, Bytes are counted in multiples of 2, not 10. A > kb is 1024 bits or 2^10 b. So 10 Gb is 10 * 2^30 bits. Interface speeds counted in multile of 10. 10Mbit ethernet have speed 10^7 bit/s. 64Kbit ISDN have speed 64000, not 65536. > It's also not unusual to be more specific about that 2-base and use > kib, Mib and Gib instead. > > Apparently you didn't know that... > > Also, if you take 5% off, you are left with (0.95 * 10 * 2^30) / 8 = > 1.1875 B/s, not 0.97 * ... Your calculations were a bit optimistic. May bug. 10^10/8*0.93 = 1162500000 = 1162.5 > Now I have to admit I'm used to use a factor of 10 to convert from b/s > to B/s (that's 20%!), but that's probably no longer correct, what with > jumbo frames and all. Ok, may be topic started use software metered speed with MGBs as 1048576 per second. 1162500000/1048576 = 1108.64 From owner-freebsd-stable@freebsd.org Mon Aug 17 16:08:53 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BD4AE9BB64B for ; Mon, 17 Aug 2015 16:08:53 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-lb0-f180.google.com (mail-lb0-f180.google.com [209.85.217.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4ED81186F for ; Mon, 17 Aug 2015 16:08:52 +0000 (UTC) (envelope-from ml@my.gd) Received: by lbbsx3 with SMTP id sx3so85318122lbb.0 for ; Mon, 17 Aug 2015 09:08:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=a5Fe2a+ScrzVGplWZOaLvxKGwafuEp2xTWVIlNSwV5Y=; b=JFnwtLvLkVTDXFcPWmpEt+b9a2dk66ySiwvUJKmiU/oGjeuwxu9lylHv5m0Yi0O8A0 MTUpTQcdi2IIQPL/nJfhL/kCzAwKIwmcFJ0JoVNsghKeXgROq5kOlT0RJUkaYK+rL2lw aSakHnbyBkykRNce9gDhFF8em4Au27QJsVB3zs369nbBHSfNRIOaaCp9QFwpaKDpys6t qnEuTLx9dpsZavcTrCNDnHoyMUGOElsbBNLzKbEBlhMwdNgaQEHlJGHaS/lwgQ5Pdh4v 0YTdHE3H0edP7Fj+CGl7p4vou5NBqWHSyXBII40uIqXUwi3lPOw9eFj0Y4mpOAmDDPdE spSA== X-Gm-Message-State: ALoCoQktYS+QXJiZ0T8fKzZ/quHRZYgARLUdq9E7okB70FPc2HZ2kT7K7Fe0DekzrvbThUwWOVky MIME-Version: 1.0 X-Received: by 10.112.138.170 with SMTP id qr10mr1732700lbb.14.1439827725225; Mon, 17 Aug 2015 09:08:45 -0700 (PDT) Received: by 10.112.60.34 with HTTP; Mon, 17 Aug 2015 09:08:45 -0700 (PDT) In-Reply-To: References: Date: Mon, 17 Aug 2015 18:08:45 +0200 Message-ID: Subject: Re: CARp comatibility between 9 and 10 From: Damien Fleuriot To: Pete French Cc: rainer@ultra-secure.de, "freebsd-stable@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2015 16:08:53 -0000 On 8 June 2015 at 12:13, Pete French wrote: > > I did this a while ago - and it actually worked. > > (CARP between 9 and 10). > > > > I think I have each IP assigned its own VHID, though. > > Thanks for this, someone else has confimred that 10 orks with multiple IP's > per VHID, so I am good to go. > > Do I still need to compile a separate kernel with pfsync and carp in > it, or can these now be loaded as modules ? I cant remember when I started > doing that, but I suspect it may no longer be necessary. > > OK I'm late to reply to this, but a word of caution to those that would upgrade from 8 to 10 or 9 to 10. While CARP maintains compatibility between the two different OS releases, you will lose pfsync capability. That means when you switch over your CARP master, PF will drop all these sessions that existed at host A but were not synched over to host B. From owner-freebsd-stable@freebsd.org Mon Aug 17 16:22:17 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A633C9BB9CF for ; Mon, 17 Aug 2015 16:22:17 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-la0-f41.google.com (mail-la0-f41.google.com [209.85.215.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3A147104B for ; Mon, 17 Aug 2015 16:22:16 +0000 (UTC) (envelope-from ml@my.gd) Received: by labd1 with SMTP id d1so82901121lab.1 for ; Mon, 17 Aug 2015 09:22:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to:cc :content-type; bh=9R+Le/geo/klSKQOmPRzSIYtQyEVv3anqCEYR5yDd2o=; b=AI3vZLtZBqvxNRrct6UfAfli4nBjC95gc1T45d5Ha8yKCgsCIbe2O78PWYlLECvGTV ns3uiBfcr8NT8fFcENGHxGZpqDXlqSBRYL7tX6mgpQccf5RPSVzwUr8geAIh3BZ60Ie+ zWD4B1tveRiqPEgUXkBBdSTBQS1adua1ykzy0okqLKRTHAKV6z/qzAgIEF/5jdh+FFzF YLWUjXvYp2srF6rkmjaqmsdWlkaq6q/3P3N75ravAErkLvw57zoa09M9+vTu0Xwm8Kxq MUlroKxob2MATkfPOWentCiV0Py2bL25G9x0vkHW4AiqN/5s59ZFNUv2x/gHBF08QEGR jz2g== X-Gm-Message-State: ALoCoQn4nI0DQPQzQElAL6k7vc+fdbPGmjBw+oC5BUw052XPPBle2+4SZegsv6iEBBv4vFIJOEtu MIME-Version: 1.0 X-Received: by 10.112.145.1 with SMTP id sq1mr1791329lbb.54.1439828529305; Mon, 17 Aug 2015 09:22:09 -0700 (PDT) Received: by 10.112.60.34 with HTTP; Mon, 17 Aug 2015 09:22:09 -0700 (PDT) Date: Mon, 17 Aug 2015 18:22:09 +0200 Message-ID: Subject: [POSSIBLE BUG] 10-STABLE CARP erroneously becomes master on boot From: Damien Fleuriot To: "freebsd-stable@freebsd.org" Cc: Damien Fleuriot Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2015 16:22:17 -0000 Hello list, I'm seeing this very peculiar behaviour between 2 10-STABLE boxes. Host A is CARP Master with advskew 20 and runs 10.2-BETA1 from 10/07 Host B is CARP Backup with advskew 150 and runs 10.2-PRERELEASE from 12/08 When I configure CARP in rc.conf on host B, it becomes Master on boot, and host A remains Master as well. When I force a state change on host B (ifconfig vlanx vhid y state backup), it transitions to Backup then again to Master. When I comment out the CARP configuration in rc.conf , and configure CARP manually on host B's interfaces after it boots, it correctly becomes and remains Backup. Below is the excerpt from rc.conf pertaining to CARP configuration, the only difference between the 2 hosts being their advskew. Host A == BEGIN ifconfig_vlan410_alias0="vhid 110 pass passhere advskew 20 alias 10.104.10.251/32" == END Host B == BEGIN ifconfig_vlan410_alias0="vhid 110 pass passhere advskew 150 alias 10.104.10.251/32" == END Additionally, the sysctls for CARP for both boxes : == BEGIN net.inet.carp.ifdown_demotion_factor: 240 net.inet.carp.senderr_demotion_factor: 240 net.inet.carp.demotion: 0 net.inet.carp.log: 1 net.inet.carp.preempt: 1 net.inet.carp.allow: 1 net.pfsync.carp_demotion_factor: 240 == END The hosts can see each other correctly, as evidenced by manually configuring the CARP aliases on Host B after it boots, and it retaining Backup status. Has anyone experienced the same problem ? I'm thinking of upgrading Host A to 10.2-PRERELEASE so they're on the same revision level, but I'm afraid it may break CARP on boot as on Host B. From owner-freebsd-stable@freebsd.org Mon Aug 17 16:32:25 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 613DE9BBBBB for ; Mon, 17 Aug 2015 16:32:25 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-ob0-x229.google.com (mail-ob0-x229.google.com [IPv6:2607:f8b0:4003:c01::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 26FA6181D for ; Mon, 17 Aug 2015 16:32:25 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by obbfr1 with SMTP id fr1so116748768obb.1 for ; Mon, 17 Aug 2015 09:32:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=lqlYHieMMDoqkT1V604KGkzCDWQ0YJZwhX9AhoqeVno=; b=mVVUgo5LFR2OK0yzFsUIDPuDt0SkmnesjZn0QcNkvpjf8KV1z6ta8QTWZaXYTlAAMm /xr6nPJb6O6n3Pb5pDQbxsvkbz1GFHoUZfFjG88IAXNLEuITaCpzTbPqek9BtRerizep xqvgYeV+4mpxywZRKR6OM4tQAwJt+r7OnjJJRSNFPetGVBRnYCtMfGaVa//az+2QV5I7 /23rbut7Ip02A+lE5DFY18PbFPiWHZMVBFQZkQz89g0mFBt9OznjWCAe06sFjaXjDJwd /Kl+XmgbLkHX0pN7KFg/Lkrv6PfT4hjSqP4P51pRpjvlMqTcBOea52hF9uX1Gobsfi/V la8g== MIME-Version: 1.0 X-Received: by 10.60.133.50 with SMTP id oz18mr1894024oeb.64.1439829144464; Mon, 17 Aug 2015 09:32:24 -0700 (PDT) Received: by 10.76.81.100 with HTTP; Mon, 17 Aug 2015 09:32:23 -0700 (PDT) Received: by 10.76.81.100 with HTTP; Mon, 17 Aug 2015 09:32:23 -0700 (PDT) In-Reply-To: References: Date: Mon, 17 Aug 2015 09:32:23 -0700 Message-ID: Subject: Re: [POSSIBLE BUG] 10-STABLE CARP erroneously becomes master on boot From: Freddie Cash To: Damien Fleuriot Cc: FreeBSD Stable , Damien Fleuriot Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2015 16:32:25 -0000 On Aug 17, 2015 9:22 AM, "Damien Fleuriot" wrote: > > Hello list, > > > > I'm seeing this very peculiar behaviour between 2 10-STABLE boxes. > > Host A is CARP Master with advskew 20 and runs 10.2-BETA1 from 10/07 > Host B is CARP Backup with advskew 150 and runs 10.2-PRERELEASE from 12/08 > > > When I configure CARP in rc.conf on host B, it becomes Master on boot, and > host A remains Master as well. > When I force a state change on host B (ifconfig vlanx vhid y state backup), > it transitions to Backup then again to Master. > > When I comment out the CARP configuration in rc.conf , and configure CARP > manually on host B's interfaces after it boots, it correctly becomes and > remains Backup. > > > > Below is the excerpt from rc.conf pertaining to CARP configuration, the > only difference between the 2 hosts being their advskew. > > Host A > == BEGIN > > ifconfig_vlan410_alias0="vhid 110 pass passhere advskew 20 alias > 10.104.10.251/32" > > == END > > Host B > == BEGIN > > ifconfig_vlan410_alias0="vhid 110 pass passhere advskew 150 alias > 10.104.10.251/32" > > == END Put the IP first, and the vhid stuff last in rc.conf for things to work the most reliably. And drop the extra alias. ifconfig_vlan410_alias0="inet 10.104.10.251/32 vhid 110 pass passhere advskew 150" CARP requires that all IPs on an interface that are part of the same vhid to be listed (added) in the exact same order for the vhid to be considered "the same". That one trips me up all the time when manually adding an IP to a CARP pair, and then later rebooting one box as they both think they're master for that interface, while being a mix of master/backup for the other interfaces. Cheers, Freddie (running CARP on 2 10-CURRENT boxes and 2 10.1-p13 boxes) From owner-freebsd-stable@freebsd.org Mon Aug 17 17:04:42 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A75479BB364 for ; Mon, 17 Aug 2015 17:04:42 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-la0-f44.google.com (mail-la0-f44.google.com [209.85.215.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3808F17AD for ; Mon, 17 Aug 2015 17:04:41 +0000 (UTC) (envelope-from ml@my.gd) Received: by lahi9 with SMTP id i9so82876962lah.2 for ; Mon, 17 Aug 2015 10:04:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=7Kb9SOWcYfe3Ma5/AzVEEPJILs9fmN34jsPBz5iH820=; b=PEQwZJ8U5AMwLMPneRhwbvfvJjVV16wtN1MuHwButYiLMzTnygV7hwduQ1dk6RMgsI U4zUjcbj6nAG4+niI+m3up2wcgRcBSSL7C1SCX2y/ga3Kx5fai068RjIKym7+thzk2a7 8btJ0OCc+BkGmjE1+uaflCj2+bobCamAi2BEzxyFVb3OjiVMhOMQt+OFjNS0hqJnmWsZ 5rSfIXWQyd6HyGsGWVWJy3aDk1kZgN0AXEajkk6U+TwPfI/8PElD1yncBBraB2M637pb iSrhnLbYwFl65XgJIbB3omoOUxMXNFeenfnnPSQcgdqCx5eXlCX9SmPFNesUVzqlXSD9 aRPA== X-Gm-Message-State: ALoCoQlpAoUQPUy9W/tPHga1QQk4kjb7Z7Tmzl+jSica9H/CoFgmx9ZRC00xs4hc/1IWANP+VYPI MIME-Version: 1.0 X-Received: by 10.112.151.178 with SMTP id ur18mr1826093lbb.59.1439829526419; Mon, 17 Aug 2015 09:38:46 -0700 (PDT) Received: by 10.112.60.34 with HTTP; Mon, 17 Aug 2015 09:38:46 -0700 (PDT) In-Reply-To: References: Date: Mon, 17 Aug 2015 18:38:46 +0200 Message-ID: Subject: Re: [POSSIBLE BUG] 10-STABLE CARP erroneously becomes master on boot From: Damien Fleuriot To: Freddie Cash Cc: FreeBSD Stable , Damien Fleuriot Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2015 17:04:42 -0000 On 17 August 2015 at 18:32, Freddie Cash wrote: > > On Aug 17, 2015 9:22 AM, "Damien Fleuriot" wrote: > > > > Hello list, > > > > > > > > I'm seeing this very peculiar behaviour between 2 10-STABLE boxes. > > > > Host A is CARP Master with advskew 20 and runs 10.2-BETA1 from 10/07 > > Host B is CARP Backup with advskew 150 and runs 10.2-PRERELEASE from > 12/08 > > > > > > When I configure CARP in rc.conf on host B, it becomes Master on boot, > and > > host A remains Master as well. > > When I force a state change on host B (ifconfig vlanx vhid y state > backup), > > it transitions to Backup then again to Master. > > > > When I comment out the CARP configuration in rc.conf , and configure CARP > > manually on host B's interfaces after it boots, it correctly becomes and > > remains Backup. > > > > > > > > Below is the excerpt from rc.conf pertaining to CARP configuration, the > > only difference between the 2 hosts being their advskew. > > > > Host A > > == BEGIN > > > > ifconfig_vlan410_alias0="vhid 110 pass passhere advskew 20 alias > > 10.104.10.251/32" > > > > == END > > > > Host B > > == BEGIN > > > > ifconfig_vlan410_alias0="vhid 110 pass passhere advskew 150 alias > > 10.104.10.251/32" > > > > == END > > Put the IP first, and the vhid stuff last in rc.conf for things to work > the most reliably. And drop the extra alias. > > ifconfig_vlan410_alias0="inet 10.104.10.251/32 vhid 110 pass passhere > advskew 150" > > CARP requires that all IPs on an interface that are part of the same vhid > to be listed (added) in the exact same order for the vhid to be considered > "the same". That one trips me up all the time when manually adding an IP to > a CARP pair, and then later rebooting one box as they both think they're > master for that interface, while being a mix of master/backup for the other > interfaces. > > Cheers, > Freddie > (running CARP on 2 10-CURRENT boxes and 2 10.1-p13 boxes) > Cheers Freddie, will try and keep the thread up to date on the results. From owner-freebsd-stable@freebsd.org Mon Aug 17 17:57:47 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EADBA9BBCEB for ; Mon, 17 Aug 2015 17:57:47 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7CBC71EBA; Mon, 17 Aug 2015 17:57:47 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id t7HHva5b086063 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Mon, 17 Aug 2015 20:57:37 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua t7HHva5b086063 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id t7HHvaXt086062; Mon, 17 Aug 2015 20:57:36 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 17 Aug 2015 20:57:36 +0300 From: Konstantin Belousov To: Cy Schubert Cc: Ian Lepore , freebsd-stable@freebsd.org, roberto@freebd.org, delphij@freebsd.org, phk@freebsd.org, Cy Schubert , Matthew Seaman Subject: Re: 10.2: ntp update breaks DCF77 clock Message-ID: <20150817175736.GA2072@kib.kiev.ua> References: <1439744220.242.87.camel@freebsd.org> <201508170549.t7H5nvxD009140@slippy.cwsent.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201508170549.t7H5nvxD009140@slippy.cwsent.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2015 17:57:48 -0000 On Sun, Aug 16, 2015 at 10:49:57PM -0700, Cy Schubert wrote: > qemu-sbruno) doesn't support all our supported platforms, especially the > multitude of ARM platforms, so holes in our auto-generated config.h support > will exist. I believe that the userspace arm ABI is not that variable. There could be little/bit endian variants, and we seems to get hw-floating point ABI. This is definitely much less than the variations of the platform each of which requires specific kernel. From owner-freebsd-stable@freebsd.org Mon Aug 17 17:59:50 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2E40D9BBD9E for ; Mon, 17 Aug 2015 17:59:50 +0000 (UTC) (envelope-from uspensky@x-art.ru) Received: from x-art.ru (charibdis.x-art.ru [80.70.228.55]) by mx1.freebsd.org (Postfix) with ESMTP id E7A241FF2 for ; Mon, 17 Aug 2015 17:59:49 +0000 (UTC) (envelope-from uspensky@x-art.ru) Received: from gw-old.x-art.ru (gw-old.x-art.ru [192.168.172.252]) by mta.x-art.ru (Postfix) with ESMTP id 823D51BF254; Mon, 17 Aug 2015 20:53:12 +0300 (MSK) Date: Mon, 17 Aug 2015 20:53:11 +0300 (MSK) From: Antony Uspensky X-X-Sender: aiu@gw-old.x-art.ru To: Tim Daneliuk cc: FreeBSD Stable Maling List Subject: Re: Swap Questions In-Reply-To: <55CF8E0B.7050608@tundraware.com> Message-ID: References: <55CDE9EE.5090603@tundraware.com> <55CF8E0B.7050608@tundraware.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2015 17:59:50 -0000 On Sat, 15 Aug 2015, Tim Daneliuk wrote: > So -L does fix the problem - sort of. The machine picks up the file as > additional swap on boot just fine. HWOEVER, when I try to reboot or shut > down the host, I get a panic telling me some noise about not being able > to shutdown swap for some reason. Try to swapoff (by hands) before shutdown. Shutdown sequence, I think, unmounts carrying disk before swapping off a carried file. If I am right, -L should be processed on shutdown also. Just a guess. From owner-freebsd-stable@freebsd.org Mon Aug 17 18:00:08 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 756789BBDE1 for ; Mon, 17 Aug 2015 18:00:08 +0000 (UTC) (envelope-from news@mips.inka.de) Received: from mail.inka.de (quechua.inka.de [IPv6:2001:7c0:407:1001:217:a4ff:fe3b:e77c]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3D2D010F2 for ; Mon, 17 Aug 2015 18:00:08 +0000 (UTC) (envelope-from news@mips.inka.de) Received: from mips.inka.de (news@[127.0.0.1]) by mail.inka.de with uucp (rmailwrap 0.5) id 1ZROhZ-0003iM-HU; Mon, 17 Aug 2015 20:00:05 +0200 Received: from lorvorc.mips.inka.de (localhost [127.0.0.1]) by lorvorc.mips.inka.de (8.15.2/8.15.2) with ESMTP id t7HHxM7R055388 for ; Mon, 17 Aug 2015 19:59:22 +0200 (CEST) (envelope-from news@lorvorc.mips.inka.de) Received: (from news@localhost) by lorvorc.mips.inka.de (8.15.2/8.15.2/Submit) id t7HHxMm9055387 for freebsd-stable@freebsd.org; Mon, 17 Aug 2015 19:59:22 +0200 (CEST) (envelope-from news) To: freebsd-stable@freebsd.org From: Christian Weisgerber Newsgroups: list.freebsd.stable Subject: Re: 10.2: ntp update breaks DCF77 clock Date: Mon, 17 Aug 2015 17:59:22 +0000 (UTC) Lines: 18 Message-ID: References: <55D03771.9000605@FreeBSD.org> <1439744220.242.87.camel@freebsd.org> X-Trace: lorvorc.mips.inka.de 1439834362 55102 ::1 (17 Aug 2015 17:59:22 GMT) X-Complaints-To: usenet@mips.inka.de User-Agent: slrn/1.0.2 (FreeBSD) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2015 18:00:08 -0000 On 2015-08-16, Ian Lepore wrote: > I wonder: is there a reason to not enable all (or most of) the refclocks > in base and in ports? I guess it isn't very elegant to enable clock drivers if nobody knows whether they work or whether the hardware was ever produced in series or whether external services they rely on still exist. As a rough guess I'd say NMEA GPS clocks are the most popular, maybe also by way of gpsd and SHM, some RAWDCF here in Europe, a few institutional or corporate users with MEINBERG clocks, and then things are getting mighty thin on the ground... Not coincidentally this looks quite a bit like the list of clock drivers originally enabled (r268351). -- Christian "naddy" Weisgerber naddy@mips.inka.de From owner-freebsd-stable@freebsd.org Mon Aug 17 19:31:07 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 46CD19BC490 for ; Mon, 17 Aug 2015 19:31:07 +0000 (UTC) (envelope-from tundra@tundraware.com) Received: from oceanview.tundraware.com (oceanview.tundraware.com [45.55.60.57]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "oceanview.tundraware.com", Issuer "oceanview.tundraware.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 0ADBA1636 for ; Mon, 17 Aug 2015 19:31:06 +0000 (UTC) (envelope-from tundra@tundraware.com) Received: from [192.168.0.2] (ozzie.tundraware.com [75.145.138.73]) (authenticated bits=0) by oceanview.tundraware.com (8.15.2/8.15.2) with ESMTPSA id t7HJLCEn006634 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Mon, 17 Aug 2015 19:21:12 GMT (envelope-from tundra@tundraware.com) Message-ID: <55D23423.8090101@tundraware.com> Date: Mon, 17 Aug 2015 14:21:07 -0500 From: Tim Daneliuk User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.8.0 MIME-Version: 1.0 CC: FreeBSD Stable Maling List Subject: Re: Swap Questions References: <55CDE9EE.5090603@tundraware.com> <55CF8E0B.7050608@tundraware.com> In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (oceanview.tundraware.com [45.55.60.57]); Mon, 17 Aug 2015 19:21:12 +0000 (UTC) X-TundraWare-MailScanner-Information: Please contact the ISP for more information X-TundraWare-MailScanner-ID: t7HJLCEn006634 X-TundraWare-MailScanner: Found to be clean X-TundraWare-MailScanner-From: tundra@tundraware.com X-Spam-Status: No X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2015 19:31:07 -0000 On 08/17/2015 12:53 PM, Antony Uspensky wrote: > On Sat, 15 Aug 2015, Tim Daneliuk wrote: > >> So -L does fix the problem - sort of. The machine picks up the file as >> additional swap on boot just fine. HWOEVER, when I try to reboot or shut >> down the host, I get a panic telling me some noise about not being able >> to shutdown swap for some reason. > > Try to swapoff (by hands) before shutdown. > Shutdown sequence, I think, unmounts carrying disk before swapping off a carried file. If I am right, -L should be processed on shutdown also. > Just a guess. Yes, that did it. But, isn't this kind of an operational bug? Shouldn't the shutdown logic do the swapoff before the unmount if it sees files being used for swap? i.e. Should I enter this as a bug report? The only reason this matters - and it's a pretty big reason - is for production servers when someone logs in remotely, becomes root, and issued "reboot". The machine hangs at the panic and never comes back ... something you do not see unless you are in a console of some sort ... -- ---------------------------------------------------------------------------- Tim Daneliuk tundra@tundraware.com PGP Key: http://www.tundraware.com/PGP/ From owner-freebsd-stable@freebsd.org Mon Aug 17 21:50:22 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C9F8D9BB836; Mon, 17 Aug 2015 21:50:22 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 6384511F2; Mon, 17 Aug 2015 21:50:21 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:oguZsh8vk9kE7f9uRHKM819IXTAuvvDOBiVQ1KB82uMcTK2v8tzYMVDF4r011RmSDd6dt6kMotGVmp6jcFRI2YyGvnEGfc4EfD4+ouJSoTYdBtWYA1bwNv/gYn9yNs1DUFh44yPzahANS47AblHf6ke/8SQVUk2mc1ElfaKpQcb7tIee6aObw9XreQJGhT6wM/tZDS6dikHvjPQQmpZoMa0ryxHE8TNicuVSwn50dxrIx06vru/5xpNo8jxRtvQ97IYAFPyiJ+VrBYFeFyksZmAp+NXw516ESQqU+mBaXH8bnxBTD07C9h69W57wti7zsK152TKGPMv4Svc6Qzmv5bxnDQT0gS0DOm0E9nrKgJlwkL5Du0Dm4Bh+2JLPJo+POfd0Za+beskVAm9IX8JUXioGBoKnc4oJAe1GM/xVooPmqx4VsRK0AQT/OOS65jZOh3LylYcg2uIgChqOiAApGdQfmH/P6tXoNqZUWOvzza2enhvZaPYD4zb268DtexsipfyJFeZqdMPayk0iEivYiVqNpIj9P3We37Je4CCg8+N8WLf32CYcoAZrr23qn590hw== X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2BFAgBUGNJV/61jaINdDoNhaQaDHrpKAQmBawqFL0oCgWgUAQEBAQEBAQGBCYIdggYBAQEDAQEBASAEJyALBQsCAQgOCgICDRYDAgIhBgEJFRECBAgHBAEcBId4AwoIDbsukB0NhVcBAQEBAQEEAQEBAQEBGASBIoowgk+BaAEBBxUBMweCaYFDBYcijXuFBIUGdYM3kSeDT4NlAiaDP1oiMwd/CBcjgQQBAQE X-IronPort-AV: E=Sophos;i="5.15,697,1432612800"; d="scan'208";a="233068245" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-annu.net.uoguelph.ca with ESMTP; 17 Aug 2015 17:49:11 -0400 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id DD2B815F565; Mon, 17 Aug 2015 17:49:11 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id jAkikwheNEJB; Mon, 17 Aug 2015 17:49:11 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 05CB815F56D; Mon, 17 Aug 2015 17:49:11 -0400 (EDT) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 6mbRortKRG-f; Mon, 17 Aug 2015 17:49:10 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id D914115F565; Mon, 17 Aug 2015 17:49:10 -0400 (EDT) Date: Mon, 17 Aug 2015 17:49:10 -0400 (EDT) From: Rick Macklem To: Daniel Braniss Cc: FreeBSD Net , Slawa Olhovchenkov , FreeBSD stable , Christopher Forgeron Message-ID: <805850043.24018217.1439848150695.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <7F892C70-9C04-4468-9514-EDBFE75CF2C6@cs.huji.ac.il> References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <20150817094145.GB3158@zxy.spb.ru> <17871443-E105-4434-80B1-6939306A865F@cs.huji.ac.il> <473274181.23263108.1439814072514.JavaMail.zimbra@uoguelph.ca> <7F892C70-9C04-4468-9514-EDBFE75CF2C6@cs.huji.ac.il> Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [172.17.95.11] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF34 (Win)/8.0.9_GA_6191) Thread-Topic: ix(intel) vs mlxen(mellanox) 10Gb performance Thread-Index: eHOLT35UpIGat4XSkuNV2057nk2L2g== X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Aug 2015 21:50:23 -0000 Daniel Braniss wrote: >=20 > > On Aug 17, 2015, at 3:21 PM, Rick Macklem wrote: > >=20 > > Daniel Braniss wrote: > >>=20 > >>> On Aug 17, 2015, at 1:41 PM, Christopher Forgeron > >>> wrote: > >>>=20 > >>> FYI, I can regularly hit 9.3 Gib/s with my Intel X520-DA2's and FreeB= SD > >>> 10.1. Before 10.1 it was less. > >>>=20 > >>=20 > >> this is NOT iperf/3 where i do get close to wire speed, > >> it=E2=80=99s NFS writes, i.e., almost real work :-) > >>=20 > >>> I used to tweak the card settings, but now it's just stock. You may w= ant > >>> to > >>> check your settings, the Mellanox may just have better defaults for y= our > >>> switch. > >>>=20 > > Have you tried disabling TSO for the Intel? With TSO enabled, it will b= e > > copying > > every transmitted mbuf chain to a new chain of mbuf clusters via. > > m_defrag() when > > TSO is enabled. (Assuming you aren't an 82598 chip. Most seem to be the > > 82599 chip > > these days?) > >=20 >=20 > hi Rick >=20 > how can i check the chip? >=20 Haven't a clue. Does "dmesg" tell you? (To be honest, since disabling TSO h= elped, I'll bet you don't have a 82598.) > > This has been fixed in the driver very recently, but those fixes won't = be > > in 10.1. > >=20 > > rick > > ps: If you could test with 10.2, it would be interesting to see how the= ix > > does with > > the current driver fixes in it? >=20 > I new TSO was involved! > ok, firstly, it=E2=80=99s 10.2 stable. > with TSO enabled, ix is bad, around 64MGB/s. > disabling TSO it=E2=80=99s better, around 130 >=20 Hmm, could you check to see of these lines are in sys/dev/ixgbe/if_ix.c at = around line#2500? /* TSO parameters */ 2572 =09 =09 ifp->if_hw_tsomax =3D 65518; 2573 =09 =09 ifp->if_hw_tsomaxsegcount =3D IXGBE_82599_SCATTER; 2574 =09 =09 ifp->if_hw_tsomaxsegsize =3D 2048; They are in stable/10. I didn't look at releng/10.2. (And if they're in a #= ifdef for FreeBSD11, take the #ifdef away.) If they are there and not ifdef'd, I can't explain why disabling TSO would = help. Once TSO is fixed so that it handles the 64K transmit segments without copy= ing all the mbufs, I suspect you might get better perf. with it enabled? Good luck with it, rick > still, mlxen0 is about 250! with and without TSO >=20 >=20 > >=20 > >>> On Mon, Aug 17, 2015 at 6:41 AM, Slawa Olhovchenkov >>> > wrote: > >>> On Mon, Aug 17, 2015 at 10:27:41AM +0300, Daniel Braniss wrote: > >>>=20 > >>>> hi, > >>>> I have a host (Dell R730) with both cards, connected to an HP82= 00 > >>>> switch at 10Gb. > >>>> when writing to the same storage (netapp) this is what I get: > >>>> ix0: ~130MGB/s > >>>> mlxen0 ~330MGB/s > >>>> this is via nfs/tcpv3 > >>>>=20 > >>>> I can get similar (bad) performance with the mellanox if I incr= ease > >>>> the file size > >>>> to 512MGB. > >>>=20 > >>> Look like mellanox have internal beffer for caching and do ACK > >>> acclerating. > >>>=20 > >>>> so at face value, it seems the mlxen does a better use of resou= rces > >>>> than the intel. > >>>> Any ideas how to improve ix/intel's performance? > >>>=20 > >>> Are you sure about netapp performance? > >>> _______________________________________________ > >>> freebsd-net@freebsd.org mailing list > >>> https://lists.freebsd.org/mailman/listinfo/freebsd-net > >>> > >>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org > >>> " > >>>=20 > >>=20 > >> _______________________________________________ > >> freebsd-stable@freebsd.org mailing list > >> https://lists.freebsd.org/mailman/listinfo/freebsd-stable > >> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.o= rg" >=20 > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Tue Aug 18 07:17:32 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BAE3D9A0522; Tue, 18 Aug 2015 07:17:32 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.116.210]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 44F84616; Tue, 18 Aug 2015 07:17:31 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from mbpro-w.cs.huji.ac.il ([132.65.80.91]) by kabab.cs.huji.ac.il with esmtp id 1ZRb99-0000T0-QC; Tue, 18 Aug 2015 10:17:23 +0300 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2102\)) Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance From: Daniel Braniss In-Reply-To: <805850043.24018217.1439848150695.JavaMail.zimbra@uoguelph.ca> Date: Tue, 18 Aug 2015 10:07:23 +0300 Cc: FreeBSD Net , Slawa Olhovchenkov , FreeBSD stable , Christopher Forgeron Message-Id: <9D8B0503-E8FA-43CA-88F0-01F184F84D9B@cs.huji.ac.il> References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <20150817094145.GB3158@zxy.spb.ru> <17871443-E105-4434-80B1-6939306A865F@cs.huji.ac.il> <473274181.23263108.1439814072514.JavaMail.zimbra@uoguelph.ca> <7F892C70-9C04-4468-9514-EDBFE75CF2C6@cs.huji.ac.il> <805850043.24018217.1439848150695.JavaMail.zimbra@uoguelph.ca> To: Rick Macklem X-Mailer: Apple Mail (2.2102) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2015 07:17:32 -0000 > On Aug 18, 2015, at 12:49 AM, Rick Macklem = wrote: >=20 > Daniel Braniss wrote: >>=20 >>> On Aug 17, 2015, at 3:21 PM, Rick Macklem = wrote: >>>=20 >>> Daniel Braniss wrote: >>>>=20 >>>>> On Aug 17, 2015, at 1:41 PM, Christopher Forgeron = >>>>> wrote: >>>>>=20 >>>>> FYI, I can regularly hit 9.3 Gib/s with my Intel X520-DA2's and = FreeBSD >>>>> 10.1. Before 10.1 it was less. >>>>>=20 >>>>=20 >>>> this is NOT iperf/3 where i do get close to wire speed, >>>> it=E2=80=99s NFS writes, i.e., almost real work :-) >>>>=20 >>>>> I used to tweak the card settings, but now it's just stock. You = may want >>>>> to >>>>> check your settings, the Mellanox may just have better defaults = for your >>>>> switch. >>>>>=20 >>> Have you tried disabling TSO for the Intel? With TSO enabled, it = will be >>> copying >>> every transmitted mbuf chain to a new chain of mbuf clusters via. >>> m_defrag() when >>> TSO is enabled. (Assuming you aren't an 82598 chip. Most seem to be = the >>> 82599 chip >>> these days?) >>>=20 >>=20 >> hi Rick >>=20 >> how can i check the chip? >>=20 > Haven't a clue. Does "dmesg" tell you? (To be honest, since disabling = TSO helped, > I'll bet you don't have a 82598.) >=20 >>> This has been fixed in the driver very recently, but those fixes = won't be >>> in 10.1. >>>=20 >>> rick >>> ps: If you could test with 10.2, it would be interesting to see how = the ix >>> does with >>> the current driver fixes in it? >>=20 >> I new TSO was involved! >> ok, firstly, it=E2=80=99s 10.2 stable. >> with TSO enabled, ix is bad, around 64MGB/s. >> disabling TSO it=E2=80=99s better, around 130 >>=20 > Hmm, could you check to see of these lines are in = sys/dev/ixgbe/if_ix.c at around > line#2500? > /* TSO parameters */ > 2572 ifp->if_hw_tsomax =3D 65518; > 2573 ifp->if_hw_tsomaxsegcount =3D = IXGBE_82599_SCATTER; > 2574 ifp->if_hw_tsomaxsegsize =3D 2048; >=20 > They are in stable/10. I didn't look at releng/10.2. (And if they're = in a #ifdef > for FreeBSD11, take the #ifdef away.) > If they are there and not ifdef'd, I can't explain why disabling TSO = would help. > Once TSO is fixed so that it handles the 64K transmit segments without = copying all > the mbufs, I suspect you might get better perf. with it enabled? >=20 this is 10.2 : they are on lines 2509-2511 and I don=E2=80=99t see any #ifdefs around = it. the plot thickens :-) danny > Good luck with it, rick >=20 >> still, mlxen0 is about 250! with and without TSO >>=20 >>=20 >>>=20 >>>>> On Mon, Aug 17, 2015 at 6:41 AM, Slawa Olhovchenkov = >>>> > wrote: >>>>> On Mon, Aug 17, 2015 at 10:27:41AM +0300, Daniel Braniss wrote: >>>>>=20 >>>>>> hi, >>>>>> I have a host (Dell R730) with both cards, connected to an = HP8200 >>>>>> switch at 10Gb. >>>>>> when writing to the same storage (netapp) this is what I get: >>>>>> ix0: ~130MGB/s >>>>>> mlxen0 ~330MGB/s >>>>>> this is via nfs/tcpv3 >>>>>>=20 >>>>>> I can get similar (bad) performance with the mellanox if I = increase >>>>>> the file size >>>>>> to 512MGB. >>>>>=20 >>>>> Look like mellanox have internal beffer for caching and do ACK >>>>> acclerating. >>>>>=20 >>>>>> so at face value, it seems the mlxen does a better use of = resources >>>>>> than the intel. >>>>>> Any ideas how to improve ix/intel's performance? >>>>>=20 >>>>> Are you sure about netapp performance? >>>>> _______________________________________________ >>>>> freebsd-net@freebsd.org mailing = list >>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-net >>>>> >>>>> To unsubscribe, send any mail to = "freebsd-net-unsubscribe@freebsd.org >>>>> " >>>>>=20 >>>>=20 >>>> _______________________________________________ >>>> freebsd-stable@freebsd.org mailing list >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-stable >>>> To unsubscribe, send any mail to = "freebsd-stable-unsubscribe@freebsd.org" >>=20 >> _______________________________________________ >> freebsd-stable@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to = "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Tue Aug 18 12:53:07 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0927A9BCD30; Tue, 18 Aug 2015 12:53:07 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 8F812A62; Tue, 18 Aug 2015 12:53:06 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:12uXCRYSEKe6zyw54jqPidD/LSx+4OfEezUN459isYplN5qZo8q6bnLW6fgltlLVR4KTs6sC0LqN9fu+EUU7or+/81k6OKRWUBEEjchE1ycBO+WiTXPBEfjxciYhF95DXlI2t1uyMExSBdqsLwaK+i760zceF13FOBZvIaytQ8iJ35/xjL760qaQSjsLrQL1Wal1IhSyoFeZnegtqqwmFJwMzADUqGBDYeVcyDAgD1uSmxHh+pX4p8Y7oGx48sgs/M9YUKj8Y79wDfkBVGxnYCgI4tb2v0zDUReX/SlbFWEXiQZTRQbf4RzwRZu3tTH18e902S2fNMuxSbEvRTWk4aAsRgXlhS0cO3si7GjdjsEjsaRAvRj0pwBj25WGJ8aRNeFiZeXTZ94XT3FNGMFLWGtEC4K4aoIJSO4AJvpZqYf64FUUoBa0HgXpH//mwDtF1ULwxrAwhuQ9DRndjktnG9MVrG+Sos/4Oa0JXaay1qaPyDzCa/Zf33D56ZPUcxYvpraCR799e9HdjE8iC1D5iQC8oIrkMjfd/P4EtWmA9KI0WeupjX8PoBo3oiWtx4Elgc/IgtRG5ErD8HBDwY02bfixQ01/bNvsRIFVviqZM4Zzat4lTHxlvD46jLYP783oNBMWwYgqkkaMI8eMdJKFt1e6DL6c X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2AcAgBFKtNV/61jaINdDoNhaQaDHrpgAQmBbAqFL0oCgWUUAQEBAQEBAQGBCYIdggYBAQEDAQEBASAEJyALBQsCAQgOCgICDRYDAgIhBgEJFRECBAgHBAEcBId4AwoIDbpvkE8NhVcBAQEBAQEEAQEBAQEBGASBIoowgk+BaAEBBxUBMweCaYFDBYcijX6FBIUGdYM3kS+DT4NlAiaCDhyBFVoiMwd/CBcjgQQBAQE X-IronPort-AV: E=Sophos;i="5.15,701,1432612800"; d="scan'208";a="233131262" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-annu.net.uoguelph.ca with ESMTP; 18 Aug 2015 08:53:02 -0400 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 09B1F15F565; Tue, 18 Aug 2015 08:53:03 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id sT5Pw9aDTzj7; Tue, 18 Aug 2015 08:53:02 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 0844315F56D; Tue, 18 Aug 2015 08:53:02 -0400 (EDT) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 3okoUCjgvaiq; Tue, 18 Aug 2015 08:53:01 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id DA91F15F565; Tue, 18 Aug 2015 08:53:01 -0400 (EDT) Date: Tue, 18 Aug 2015 08:53:01 -0400 (EDT) From: Rick Macklem To: Daniel Braniss Cc: FreeBSD Net , Christopher Forgeron , FreeBSD stable , Slawa Olhovchenkov Message-ID: <1721122651.24481798.1439902381663.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <9D8B0503-E8FA-43CA-88F0-01F184F84D9B@cs.huji.ac.il> References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <20150817094145.GB3158@zxy.spb.ru> <17871443-E105-4434-80B1-6939306A865F@cs.huji.ac.il> <473274181.23263108.1439814072514.JavaMail.zimbra@uoguelph.ca> <7F892C70-9C04-4468-9514-EDBFE75CF2C6@cs.huji.ac.il> <805850043.24018217.1439848150695.JavaMail.zimbra@uoguelph.ca> <9D8B0503-E8FA-43CA-88F0-01F184F84D9B@cs.huji.ac.il> Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [172.17.95.10] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF34 (Win)/8.0.9_GA_6191) Thread-Topic: ix(intel) vs mlxen(mellanox) 10Gb performance Thread-Index: HIzeCIwOEaU79f03Ql6Ay7y4PoXZQw== X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2015 12:53:07 -0000 Daniel Braniss wrote: >=20 > > On Aug 18, 2015, at 12:49 AM, Rick Macklem wrote= : > >=20 > > Daniel Braniss wrote: > >>=20 > >>> On Aug 17, 2015, at 3:21 PM, Rick Macklem wrot= e: > >>>=20 > >>> Daniel Braniss wrote: > >>>>=20 > >>>>> On Aug 17, 2015, at 1:41 PM, Christopher Forgeron > >>>>> > >>>>> wrote: > >>>>>=20 > >>>>> FYI, I can regularly hit 9.3 Gib/s with my Intel X520-DA2's and Fre= eBSD > >>>>> 10.1. Before 10.1 it was less. > >>>>>=20 > >>>>=20 > >>>> this is NOT iperf/3 where i do get close to wire speed, > >>>> it=E2=80=99s NFS writes, i.e., almost real work :-) > >>>>=20 > >>>>> I used to tweak the card settings, but now it's just stock. You may > >>>>> want > >>>>> to > >>>>> check your settings, the Mellanox may just have better defaults for > >>>>> your > >>>>> switch. > >>>>>=20 > >>> Have you tried disabling TSO for the Intel? With TSO enabled, it will= be > >>> copying > >>> every transmitted mbuf chain to a new chain of mbuf clusters via. > >>> m_defrag() when > >>> TSO is enabled. (Assuming you aren't an 82598 chip. Most seem to be t= he > >>> 82599 chip > >>> these days?) > >>>=20 > >>=20 > >> hi Rick > >>=20 > >> how can i check the chip? > >>=20 > > Haven't a clue. Does "dmesg" tell you? (To be honest, since disabling T= SO > > helped, > > I'll bet you don't have a 82598.) > >=20 > >>> This has been fixed in the driver very recently, but those fixes won'= t be > >>> in 10.1. > >>>=20 > >>> rick > >>> ps: If you could test with 10.2, it would be interesting to see how t= he > >>> ix > >>> does with > >>> the current driver fixes in it? > >>=20 > >> I new TSO was involved! > >> ok, firstly, it=E2=80=99s 10.2 stable. > >> with TSO enabled, ix is bad, around 64MGB/s. > >> disabling TSO it=E2=80=99s better, around 130 > >>=20 > > Hmm, could you check to see of these lines are in sys/dev/ixgbe/if_ix.c= at > > around > > line#2500? > > /* TSO parameters */ > > 2572 =09 =09 ifp->if_hw_tsomax =3D 65518; > > 2573 =09 =09 ifp->if_hw_tsomaxsegcount =3D IXGBE_82599_SCATTER= ; > > 2574 =09 =09 ifp->if_hw_tsomaxsegsize =3D 2048; > >=20 > > They are in stable/10. I didn't look at releng/10.2. (And if they're in= a > > #ifdef > > for FreeBSD11, take the #ifdef away.) > > If they are there and not ifdef'd, I can't explain why disabling TSO wo= uld > > help. > > Once TSO is fixed so that it handles the 64K transmit segments without > > copying all > > the mbufs, I suspect you might get better perf. with it enabled? > >=20 >=20 > this is 10.2 : > they are on lines 2509-2511 and I don=E2=80=99t see any #ifdefs around i= t. >=20 > the plot thickens :-) >=20 If this is just a test machine, maybe you could test with these lines (at a= bout #880) in sys/netinet/tcp_output.c commented out? (It looks to me like this will d= isable TSO for almost all the NFS writes.) - around line #880 in sys/netinet/tcp_output.c: =09=09=09/* =09=09=09 * In case there are too many small fragments =09=09=09 * don't use TSO: =09=09=09 */ =09=09=09if (len <=3D max_len) { =09=09=09=09len =3D max_len; =09=09=09=09sendalot =3D 1; =09=09=09=09tso =3D 0; =09=09=09} This was added along with the other stuff that did the if_hw_tsomaxsegcount= , etc and I never noticed it until now (not my patch). rick > danny >=20 > > Good luck with it, rick > >=20 > >> still, mlxen0 is about 250! with and without TSO > >>=20 > >>=20 > >>>=20 > >>>>> On Mon, Aug 17, 2015 at 6:41 AM, Slawa Olhovchenkov >>>>> > wrote: > >>>>> On Mon, Aug 17, 2015 at 10:27:41AM +0300, Daniel Braniss wrote: > >>>>>=20 > >>>>>> hi, > >>>>>> I have a host (Dell R730) with both cards, connected to an HP8= 200 > >>>>>> switch at 10Gb. > >>>>>> when writing to the same storage (netapp) this is what I get: > >>>>>> ix0: ~130MGB/s > >>>>>> mlxen0 ~330MGB/s > >>>>>> this is via nfs/tcpv3 > >>>>>>=20 > >>>>>> I can get similar (bad) performance with the mellanox if I > >>>>>> increase > >>>>>> the file size > >>>>>> to 512MGB. > >>>>>=20 > >>>>> Look like mellanox have internal beffer for caching and do ACK > >>>>> acclerating. > >>>>>=20 > >>>>>> so at face value, it seems the mlxen does a better use of > >>>>>> resources > >>>>>> than the intel. > >>>>>> Any ideas how to improve ix/intel's performance? > >>>>>=20 > >>>>> Are you sure about netapp performance? > >>>>> _______________________________________________ > >>>>> freebsd-net@freebsd.org mailing li= st > >>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-net > >>>>> > >>>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.o= rg > >>>>> " > >>>>>=20 > >>>>=20 > >>>> _______________________________________________ > >>>> freebsd-stable@freebsd.org mailing list > >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-stable > >>>> To unsubscribe, send any mail to > >>>> "freebsd-stable-unsubscribe@freebsd.org" > >>=20 > >> _______________________________________________ > >> freebsd-stable@freebsd.org mailing list > >> https://lists.freebsd.org/mailman/listinfo/freebsd-stable > >> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.o= rg" >=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Tue Aug 18 13:09:30 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 58F7F9BB2E9 for ; Tue, 18 Aug 2015 13:09:30 +0000 (UTC) (envelope-from marko.cupac@mimar.rs) Received: from smtp.mimar.rs (smtp.mimar.rs [193.53.106.135]) by mx1.freebsd.org (Postfix) with ESMTP id 0AA6516D0 for ; Tue, 18 Aug 2015 13:09:29 +0000 (UTC) (envelope-from marko.cupac@mimar.rs) Received: from vscan.mimar.rs (vscan.mimar.rs [193.53.106.134]) by smtp.mimar.rs (Postfix) with ESMTP id 7A44A89A82 for ; Tue, 18 Aug 2015 15:09:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=mimar.rs; h= content-transfer-encoding:content-type:content-type:mime-version :x-mailer:organization:message-id:subject:subject:from:from:date :date:received:received; s=mimar-0901; t=1439903367; x= 1441717768; bh=bsZ8xcncUV3JssErIbMfYVhEdBwIo4PTLQBPvohHFlg=; b=H 48teT8iKXMaO+VNR/+Rt2dw3q/6gN5sVep+9M1Z4fqB6ung1jnp3AX2G2/QiiRS9 P/VD+rp6cwLPHVj9gAi9mutloqPcQHuk/AwZ0nw/kBgFedHw31XH266kcReQe095 0EkSKLRw0GnfZWEGlr6eFKKH9wR+RoDbbzlYvoJEBM= X-Virus-Scanned: amavisd-new at mimar.rs Received: from smtp.mimar.rs ([193.53.106.135]) by vscan.mimar.rs (vscan.mimar.rs [193.53.106.134]) (amavisd-new, port 10026) with ESMTP id ioS1FEh9ufuw for ; Tue, 18 Aug 2015 15:09:27 +0200 (CEST) Received: from efreet (unknown [193.53.106.34]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: marko.cupac@mimar.rs) by smtp.mimar.rs (Postfix) with ESMTPSA id B3E76898DA for ; Tue, 18 Aug 2015 15:09:27 +0200 (CEST) Date: Tue, 18 Aug 2015 15:09:24 +0200 From: Marko =?UTF-8?B?Q3VwYcSH?= To: FreeBSD Stable Subject: ping from web application Message-ID: <20150818150924.5e9bef04@efreet> Organization: mimar X-Mailer: Claws Mail 3.9.3 (GTK+ 2.24.23; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2015 13:09:30 -0000 Hi, I use web applicaton (net-mgmt/phpipam) which should have the ability to check hosts' availability via ping. I can even specify path to ping executable. This functionality does not work on FreeBSD by default, and suggested workaround is to set setuid bit on /sbin/ping. I don't like to modify permissions of base files. Is there an alternative solution? e.g. a port? Thank you in advance, --=20 Marko Cupa=C4=87 https://www.mimar.rs/ From owner-freebsd-stable@freebsd.org Tue Aug 18 13:21:23 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 02CC69BB8C1; Tue, 18 Aug 2015 13:21:23 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B7BE86AF; Tue, 18 Aug 2015 13:21:22 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id B062F1FE023; Tue, 18 Aug 2015 15:21:20 +0200 (CEST) Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance To: Rick Macklem , Daniel Braniss References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <20150817094145.GB3158@zxy.spb.ru> <17871443-E105-4434-80B1-6939306A865F@cs.huji.ac.il> <473274181.23263108.1439814072514.JavaMail.zimbra@uoguelph.ca> <7F892C70-9C04-4468-9514-EDBFE75CF2C6@cs.huji.ac.il> <805850043.24018217.1439848150695.JavaMail.zimbra@uoguelph.ca> <9D8B0503-E8FA-43CA-88F0-01F184F84D9B@cs.huji.ac.il> <1721122651.24481798.1439902381663.JavaMail.zimbra@uoguelph.ca> Cc: FreeBSD Net , Slawa Olhovchenkov , FreeBSD stable , Christopher Forgeron From: Hans Petter Selasky Message-ID: <55D331A5.9050601@selasky.org> Date: Tue, 18 Aug 2015 15:22:45 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.1.0 MIME-Version: 1.0 In-Reply-To: <1721122651.24481798.1439902381663.JavaMail.zimbra@uoguelph.ca> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2015 13:21:23 -0000 On 08/18/15 14:53, Rick Macklem wrote: > If this is just a test machine, maybe you could test with these lines (at about #880) > in sys/netinet/tcp_output.c commented out? (It looks to me like this will disable TSO > for almost all the NFS writes.) > - around line #880 in sys/netinet/tcp_output.c: > /* > * In case there are too many small fragments > * don't use TSO: > */ > if (len <= max_len) { > len = max_len; > sendalot = 1; > tso = 0; > } > > This was added along with the other stuff that did the if_hw_tsomaxsegcount, etc and I > never noticed it until now (not my patch). FYI: These lines are needed by other hardware, like the mlxen driver. If you remove them mlxen will start doing m_defrag(). I believe if you set the correct parameters in the "struct ifnet" for the TSO size/count limits this problem will go away. If you print the "len" and "max_len" and also the cases where TSO limits are reached, you'll see what parameter is triggering it and needs to be increased. --HPS From owner-freebsd-stable@freebsd.org Tue Aug 18 13:30:44 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2992F9BBBD8; Tue, 18 Aug 2015 13:30:44 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DF251FD1; Tue, 18 Aug 2015 13:30:43 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 92BB21FE023; Tue, 18 Aug 2015 15:30:41 +0200 (CEST) Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance To: Rick Macklem , Daniel Braniss References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <20150817094145.GB3158@zxy.spb.ru> <17871443-E105-4434-80B1-6939306A865F@cs.huji.ac.il> <473274181.23263108.1439814072514.JavaMail.zimbra@uoguelph.ca> <7F892C70-9C04-4468-9514-EDBFE75CF2C6@cs.huji.ac.il> <805850043.24018217.1439848150695.JavaMail.zimbra@uoguelph.ca> <9D8B0503-E8FA-43CA-88F0-01F184F84D9B@cs.huji.ac.il> <1721122651.24481798.1439902381663.JavaMail.zimbra@uoguelph.ca> Cc: FreeBSD Net , Slawa Olhovchenkov , FreeBSD stable , Christopher Forgeron From: Hans Petter Selasky Message-ID: <55D333D6.5040102@selasky.org> Date: Tue, 18 Aug 2015 15:32:06 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.1.0 MIME-Version: 1.0 In-Reply-To: <1721122651.24481798.1439902381663.JavaMail.zimbra@uoguelph.ca> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2015 13:30:44 -0000 On 08/18/15 14:53, Rick Macklem wrote: > 2572 ifp->if_hw_tsomax = 65518; >> >2573 ifp->if_hw_tsomaxsegcount = IXGBE_82599_SCATTER; >> >2574 ifp->if_hw_tsomaxsegsize = 2048; Hi, If IXGBE_82599_SCATTER is the maximum scatter/gather entries the hardware can do, remember to subtract one fragment for the TCP/IP-header mbuf! I think there is an off-by-one here: ifp->if_hw_tsomax = 65518; ifp->if_hw_tsomaxsegcount = IXGBE_82599_SCATTER - 1; ifp->if_hw_tsomaxsegsize = 2048; Refer to: > * > * NOTE: The TSO limits only apply to the data payload part of > * a TCP/IP packet. That means there is no need to subtract > * space for ethernet-, vlan-, IP- or TCP- headers from the > * TSO limits unless the hardware driver in question requires > * so. In sys/net/if_var.h Thank you! --HPS From owner-freebsd-stable@freebsd.org Tue Aug 18 14:09:50 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 218D39BC532; Tue, 18 Aug 2015 14:09:50 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.116.210]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C6EE6118; Tue, 18 Aug 2015 14:09:48 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from chamsa.cs.huji.ac.il ([132.65.80.19]) by kabab.cs.huji.ac.il with esmtp id 1ZRhaA-000Ac9-LX; Tue, 18 Aug 2015 17:09:42 +0300 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance From: Daniel Braniss In-Reply-To: <55D333D6.5040102@selasky.org> Date: Tue, 18 Aug 2015 17:09:41 +0300 Cc: Rick Macklem , FreeBSD Net , Slawa Olhovchenkov , FreeBSD stable , Christopher Forgeron Content-Transfer-Encoding: quoted-printable Message-Id: <47EC9292-082C-4801-B52F-4BD6B8310F99@cs.huji.ac.il> References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <20150817094145.GB3158@zxy.spb.ru> <17871443-E105-4434-80B1-6939306A865F@cs.huji.ac.il> <473274181.23263108.1439814072514.JavaMail.zimbra@uoguelph.ca> <7F892C70-9C04-4468-9514-EDBFE75CF2C6@cs.huji.ac.il> <805850043.24018217.1439848150695.JavaMail.zimbra@uoguelph.ca> <9D8B0503-E8FA-43CA-88F0-01F184F84D9B@cs.huji.ac.il> <1721122651.24481798.1439902381663.JavaMail.zimbra@uoguelph.ca> <55D333D6.5040102@selasky.org> To: Hans Petter Selasky X-Mailer: Apple Mail (2.2104) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2015 14:09:50 -0000 sorry, it=E2=80=99s been a tough day, we had a major meltdown, caused by = a faulty gbic :-( anyways, could you tell me what to do? comment out, fix the off by one? the machine is not yet production. thanks, danny > On 18 Aug 2015, at 16:32, Hans Petter Selasky wrote: >=20 > On 08/18/15 14:53, Rick Macklem wrote: >> 2572 ifp->if_hw_tsomax =3D 65518; >>> >2573 ifp->if_hw_tsomaxsegcount =3D = IXGBE_82599_SCATTER; >>> >2574 ifp->if_hw_tsomaxsegsize =3D 2048; >=20 > Hi, >=20 > If IXGBE_82599_SCATTER is the maximum scatter/gather entries the = hardware can do, remember to subtract one fragment for the TCP/IP-header = mbuf! >=20 > I think there is an off-by-one here: >=20 > ifp->if_hw_tsomax =3D 65518; > ifp->if_hw_tsomaxsegcount =3D IXGBE_82599_SCATTER - 1; > ifp->if_hw_tsomaxsegsize =3D 2048; >=20 > Refer to: >=20 >> * >> * NOTE: The TSO limits only apply to the data payload part of >> * a TCP/IP packet. That means there is no need to subtract >> * space for ethernet-, vlan-, IP- or TCP- headers from the >> * TSO limits unless the hardware driver in question requires >> * so. >=20 > In sys/net/if_var.h >=20 > Thank you! >=20 > --HPS >=20 From owner-freebsd-stable@freebsd.org Tue Aug 18 14:13:38 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C06ED9BC781 for ; Tue, 18 Aug 2015 14:13:38 +0000 (UTC) (envelope-from peixotocassiano@gmail.com) Received: from mail-io0-x236.google.com (mail-io0-x236.google.com [IPv6:2607:f8b0:4001:c06::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8D0C1A96 for ; Tue, 18 Aug 2015 14:13:38 +0000 (UTC) (envelope-from peixotocassiano@gmail.com) Received: by iodv127 with SMTP id v127so174647183iod.3 for ; Tue, 18 Aug 2015 07:13:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=Oau+ckg7waaSQ68rYLT1869oAuVbK7G2H0GbJeewKKk=; b=UvVmnbnLTeTL6rWDVCoTblbyRHAW4LYAggoQ2Kf2p814188tQqvgTBlInltc5lGfCk X6+Oia3klLRzU7AU8SCtXJEoo8fl+OpuZ4JwicLgYwmZGGSKLUcTWYaYFEhTuaP21Lbh C5jjW0FAI5+QTvlMrxA8wmtUZGG85SFuxGCEbsYZ4jnkXjmVwq1i9rvrHNtJ6SQD+tmU pdMbaYpQiqaYTWLh0QrzgokJRhWeiB3Bc7TALt3YdvRmxJs8NO3eyGr2WkQuSkcU6Duh i7LYJNQySiY1EpXXwjKWmpDnNRVLkcsEtKzzdZo5UUowS6DxLtkigs8yt0O+H8xBOfEm COww== MIME-Version: 1.0 X-Received: by 10.107.133.34 with SMTP id h34mr7315685iod.1.1439907217892; Tue, 18 Aug 2015 07:13:37 -0700 (PDT) Received: by 10.36.104.139 with HTTP; Tue, 18 Aug 2015 07:13:37 -0700 (PDT) Date: Tue, 18 Aug 2015 11:13:37 -0300 Message-ID: Subject: Issue with /boot/menu.rc.local after 10.2 update From: Cassiano Peixoto To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2015 14:13:38 -0000 Hi guys, I've been using /boot/menu.rc.local for years to my custom menu option. But after 10.2 update, it's not working anymore. I always get the message "toggle_menuitem not found" on beastie menu. Here is my /boot/menu.rc.local: set optionsmenu_init[7]="init_console" set optionsmenu_caption[7]="[C]onsole..... Off" set optionstoggled_text[7]="[C]onsole..... On" set optionsmenu_command[7]="toggle_console" set optionsmenu_keycode[7]=118 set optionsansi_caption[7]="^[[1mC^[[37monsole..... ^[[34;1mOff^[[37m" set optionstoggled_ansi[7]="^[[1mC^[[37monsole..... ^[[32;7mOn^[[0;37m" \ \ Console Boot \ : console_enabled? ( -- flag ) s" boot_single" getenv -1 <> dup if drop ( c-addr flag -- flag ) then ; : console_enable ( -- ) s" set console=comconsole,vidconsole" evaluate ; : console_disable ( -- ) s" set console=vidconsole,comconsole" evaluate ; : toggle_console ( N -- N TRUE ) toggle_menuitem menu-redraw \ Now we're going to make the change effective dup toggle_stateN @ 0= if console_disable else console_enable then TRUE \ loop menu again ; So what has changed? What should i do to make it works again? Thanks. From owner-freebsd-stable@freebsd.org Tue Aug 18 14:15:26 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 43E0A9BC80A for ; Tue, 18 Aug 2015 14:15:26 +0000 (UTC) (envelope-from freebsd-stable-local@be-well.ilk.org) Received: from be-well.ilk.org (be-well.ilk.org [23.30.133.173]) by mx1.freebsd.org (Postfix) with ESMTP id 1F69CBD1 for ; Tue, 18 Aug 2015 14:15:25 +0000 (UTC) (envelope-from freebsd-stable-local@be-well.ilk.org) Received: from lowell-desk.lan (router.lan [172.30.250.2]) by be-well.ilk.org (Postfix) with ESMTP id 1483A33C1E; Tue, 18 Aug 2015 10:15:20 -0400 (EDT) Received: by lowell-desk.lan (Postfix, from userid 1147) id E54A839819; Tue, 18 Aug 2015 10:15:18 -0400 (EDT) From: Lowell Gilbert To: Marko =?utf-8?Q?Cupa=C4=87?= Cc: FreeBSD Stable Subject: Re: ping from web application References: <20150818150924.5e9bef04@efreet> Reply-To: FreeBSD Stable Date: Tue, 18 Aug 2015 10:15:18 -0400 In-Reply-To: <20150818150924.5e9bef04@efreet> ("Marko =?utf-8?Q?Cupa=C4=87?= =?utf-8?Q?=22's?= message of "Tue, 18 Aug 2015 15:09:24 +0200") Message-ID: <444mjwisy1.fsf@lowell-desk.lan> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2015 14:15:26 -0000 Marko Cupa=C4=87 writes: > I use web applicaton (net-mgmt/phpipam) which should have the ability > to check hosts' availability via ping. I can even specify path to ping > executable. > > This functionality does not work on FreeBSD by default, and suggested > workaround is to set setuid bit on /sbin/ping. > > I don't like to modify permissions of base files. Is there an > alternative solution? e.g. a port? In what way does ping(8) not work? A look at its error output should tell you what the problem is. Additionally, the standard permissions on /sbin/ping *are* suid root. It certainly won't work if you've changed that, so just change it back. And yes, there are other ping programs present, including some with pretty graphical web page UIs. But there's no reason that PHP should have trouble calling /sbin/ping. From owner-freebsd-stable@freebsd.org Tue Aug 18 14:18:51 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1ADF79BC901; Tue, 18 Aug 2015 14:18:51 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C2ACCD7D; Tue, 18 Aug 2015 14:18:50 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.84 (FreeBSD)) (envelope-from ) id 1ZRhio-0002d4-G8; Tue, 18 Aug 2015 17:18:38 +0300 Date: Tue, 18 Aug 2015 17:18:38 +0300 From: Slawa Olhovchenkov To: Daniel Braniss Cc: Hans Petter Selasky , Rick Macklem , FreeBSD Net , FreeBSD stable , Christopher Forgeron Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance Message-ID: <20150818141838.GO1872@zxy.spb.ru> References: <20150817094145.GB3158@zxy.spb.ru> <17871443-E105-4434-80B1-6939306A865F@cs.huji.ac.il> <473274181.23263108.1439814072514.JavaMail.zimbra@uoguelph.ca> <7F892C70-9C04-4468-9514-EDBFE75CF2C6@cs.huji.ac.il> <805850043.24018217.1439848150695.JavaMail.zimbra@uoguelph.ca> <9D8B0503-E8FA-43CA-88F0-01F184F84D9B@cs.huji.ac.il> <1721122651.24481798.1439902381663.JavaMail.zimbra@uoguelph.ca> <55D333D6.5040102@selasky.org> <47EC9292-082C-4801-B52F-4BD6B8310F99@cs.huji.ac.il> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <47EC9292-082C-4801-B52F-4BD6B8310F99@cs.huji.ac.il> User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2015 14:18:51 -0000 On Tue, Aug 18, 2015 at 05:09:41PM +0300, Daniel Braniss wrote: > sorry, it's been a tough day, we had a major meltdown, caused by a faulty gbic :-( > anyways, could you tell me what to do? > comment out, fix the off by one? > > the machine is not yet production. Can you collect this information? https://lists.freebsd.org/pipermail/freebsd-stable/2015-August/083113.html And 'show interface' (or equivalent: error/collsion/events counters) from both ports from HP8200. From owner-freebsd-stable@freebsd.org Tue Aug 18 17:29:27 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 481EE9BC991 for ; Tue, 18 Aug 2015 17:29:27 +0000 (UTC) (envelope-from uspensky@x-art.ru) Received: from x-art.ru (charibdis.x-art.ru [80.70.228.55]) by mx1.freebsd.org (Postfix) with ESMTP id 0BAA01B08 for ; Tue, 18 Aug 2015 17:29:26 +0000 (UTC) (envelope-from uspensky@x-art.ru) Received: from gw-old.x-art.ru (gw-old.x-art.ru [192.168.172.252]) by mta.x-art.ru (Postfix) with ESMTP id 28B451BF430; Tue, 18 Aug 2015 20:29:23 +0300 (MSK) Date: Tue, 18 Aug 2015 20:29:23 +0300 (MSK) From: Antony Uspensky X-X-Sender: aiu@gw-old.x-art.ru To: Tim Daneliuk cc: FreeBSD Stable Maling List Subject: Re: Swap Questions In-Reply-To: <55D23423.8090101@tundraware.com> Message-ID: References: <55CDE9EE.5090603@tundraware.com> <55CF8E0B.7050608@tundraware.com> <55D23423.8090101@tundraware.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2015 17:29:27 -0000 On Mon, 17 Aug 2015, Tim Daneliuk wrote: > On 08/17/2015 12:53 PM, Antony Uspensky wrote: >> On Sat, 15 Aug 2015, Tim Daneliuk wrote: >> >>> So -L does fix the problem - sort of. The machine picks up the file as >>> additional swap on boot just fine. HWOEVER, when I try to reboot or shut >>> down the host, I get a panic telling me some noise about not being able >>> to shutdown swap for some reason. >> >> Try to swapoff (by hands) before shutdown. >> Shutdown sequence, I think, unmounts carrying disk before swapping off a carried file. If I am right, -L should be processed on shutdown also. >> Just a guess. > > Yes, that did it. > > But, isn't this kind of an operational bug? Shouldn't the shutdown logic > do the swapoff before the unmount if it sees files being used for swap? Yes. Must. > i.e. Should I enter this as a bug report? Yes, please. > The only reason this matters - and it's a pretty big reason - is for production > servers when someone logs in remotely, becomes root, and issued "reboot". The > machine hangs at the panic and never comes back ... something you do not see > unless you are in a console of some sort ... From owner-freebsd-stable@freebsd.org Tue Aug 18 17:35:23 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 247D29BCCE6 for ; Tue, 18 Aug 2015 17:35:23 +0000 (UTC) (envelope-from tundra@tundraware.com) Received: from oceanview.tundraware.com (oceanview.tundraware.com [45.55.60.57]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "oceanview.tundraware.com", Issuer "oceanview.tundraware.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D6DEC1FE1 for ; Tue, 18 Aug 2015 17:35:22 +0000 (UTC) (envelope-from tundra@tundraware.com) Received: from [192.168.0.2] (ozzie.tundraware.com [75.145.138.73]) (authenticated bits=0) by oceanview.tundraware.com (8.15.2/8.15.2) with ESMTPSA id t7IHYqtj004952 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Tue, 18 Aug 2015 12:34:53 -0500 (CDT) (envelope-from tundra@tundraware.com) Message-ID: <55D36CB7.30201@tundraware.com> Date: Tue, 18 Aug 2015 12:34:47 -0500 From: Tim Daneliuk User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.8.0 MIME-Version: 1.0 To: FreeBSD Stable Maling List Subject: Re: Swap Questions References: <55CDE9EE.5090603@tundraware.com> <55CF8E0B.7050608@tundraware.com> <55D23423.8090101@tundraware.com> In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (oceanview.tundraware.com [45.55.60.57]); Tue, 18 Aug 2015 12:34:53 -0500 (CDT) X-TundraWare-MailScanner-Information: Please contact the ISP for more information X-TundraWare-MailScanner-ID: t7IHYqtj004952 X-TundraWare-MailScanner: Found to be clean X-TundraWare-MailScanner-From: tundra@tundraware.com X-Spam-Status: No X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2015 17:35:23 -0000 On 08/18/2015 12:29 PM, Antony Uspensky wrote: > On Mon, 17 Aug 2015, Tim Daneliuk wrote: > >> On 08/17/2015 12:53 PM, Antony Uspensky wrote: >>> On Sat, 15 Aug 2015, Tim Daneliuk wrote: >>> >>>> So -L does fix the problem - sort of. The machine picks up the file as >>>> additional swap on boot just fine. HWOEVER, when I try to reboot or shut >>>> down the host, I get a panic telling me some noise about not being able >>>> to shutdown swap for some reason. >>> >>> Try to swapoff (by hands) before shutdown. >>> Shutdown sequence, I think, unmounts carrying disk before swapping off a carried file. If I am right, -L should be processed on shutdown also. >>> Just a guess. >> >> Yes, that did it. >> >> But, isn't this kind of an operational bug? Shouldn't the shutdown logic >> do the swapoff before the unmount if it sees files being used for swap? > > Yes. Must. > >> i.e. Should I enter this as a bug report? > > Yes, please. > >> The only reason this matters - and it's a pretty big reason - is for production >> servers when someone logs in remotely, becomes root, and issued "reboot". The >> machine hangs at the panic and never comes back ... something you do not see >> unless you are in a console of some sort ... > Done. Bug report #202420 -- ---------------------------------------------------------------------------- Tim Daneliuk tundra@tundraware.com PGP Key: http://www.tundraware.com/PGP/ From owner-freebsd-stable@freebsd.org Tue Aug 18 18:50:43 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DEF7A9BDDD8 for ; Tue, 18 Aug 2015 18:50:43 +0000 (UTC) (envelope-from tom@samplonius.org) Received: from mail-pa0-f43.google.com (mail-pa0-f43.google.com [209.85.220.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B951A1B1D for ; Tue, 18 Aug 2015 18:50:43 +0000 (UTC) (envelope-from tom@samplonius.org) Received: by pabyb7 with SMTP id yb7so138491998pab.0 for ; Tue, 18 Aug 2015 11:50:37 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=D1HmliL4RG+zTX2nXVlnoFBs67CQeNNUj2RvXiOJaIU=; b=gl9soKXq0dGgfYTdbYT2IN7LWDbDJFDDW+TJl8mg58u8bpHQjBU6IyDn7X0WsAmKhH VikBB+RSLj1bwoSJtnBnD9IuHrkxNarryOkfVaWvZLKa10f3WrBz+dGOdCmCg/iD+sDD lTXSQdjTv8GI9tm524jhzG9VN75/UULxrM645K1RTY/KGryi4iNNHFOZfRz345GFiuQW DyHn8NMm30TphyIU5orCkfemHT7jYLlWMR3A9NRSgDHRLRx3lnCA6Ksc+GGyrAzum3Fr kL1UMhqmSk/K6dQ6bBu6ZnCswqG1RDdjzpsSvovxUZ139pcIXJwqv88Y1vKF7RriSXxc 7vvw== X-Gm-Message-State: ALoCoQn1efOMBcgPf0nXahkUj7R/00yMGhvtk97wfhXOCzp5dc9XE0/RE7elKcUvv1fCYI5E4mV1 X-Received: by 10.68.135.66 with SMTP id pq2mr16195527pbb.29.1439923837198; Tue, 18 Aug 2015 11:50:37 -0700 (PDT) Received: from [192.168.10.51] (vpn.unet.ca. [97.107.176.59]) by smtp.gmail.com with ESMTPSA id qn6sm18885986pbc.22.2015.08.18.11.50.36 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 18 Aug 2015 11:50:36 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) Subject: Re: ping from web application From: Tom Samplonius In-Reply-To: <444mjwisy1.fsf@lowell-desk.lan> Date: Tue, 18 Aug 2015 11:50:34 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <4FD40952-8DE5-4800-9BC3-C099E09C36AE@samplonius.org> References: <20150818150924.5e9bef04@efreet> <444mjwisy1.fsf@lowell-desk.lan> To: FreeBSD Stable X-Mailer: Apple Mail (2.2104) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2015 18:50:44 -0000 > On Aug 18, 2015, at 7:15 AM, Lowell Gilbert = wrote: >=20 > Marko Cupa=C4=87 writes: >=20 >> I use web applicaton (net-mgmt/phpipam) which should have the ability >> to check hosts' availability via ping. I can even specify path to = ping >> executable. >>=20 >> This functionality does not work on FreeBSD by default, and suggested >> workaround is to set setuid bit on /sbin/ping. >>=20 >> I don't like to modify permissions of base files. Is there an >> alternative solution? e.g. a port? >=20 > In what way does ping(8) not work? A look at its error output should > tell you what the problem is. >=20 > Additionally, the standard permissions on /sbin/ping *are* suid root. > It certainly won't work if you've changed that, so just change it = back. >=20 > And yes, there are other ping programs present, including some with > pretty graphical web page UIs. But there's no reason that PHP should > have trouble calling /sbin/ping. It is a pretty standard issue: only apps running as root can send = ICMP directly, as ping does. PHP runs in Apache, and to prevent = security issues with privilege escalation setuid programs are forced to = run as an unprivileged user. I would check to see how =E2=80=9Cfping=E2=80=9D in Nagios solved this = issue. Tom From owner-freebsd-stable@freebsd.org Tue Aug 18 19:06:21 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 554179BC1E3 for ; Tue, 18 Aug 2015 19:06:21 +0000 (UTC) (envelope-from peixotocassiano@gmail.com) Received: from mail-io0-x22f.google.com (mail-io0-x22f.google.com [IPv6:2607:f8b0:4001:c06::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 21042895 for ; Tue, 18 Aug 2015 19:06:21 +0000 (UTC) (envelope-from peixotocassiano@gmail.com) Received: by iodb91 with SMTP id b91so200263645iod.1 for ; Tue, 18 Aug 2015 12:06:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=1bHbRTMPbr2aUHvBVp7PzBp3BIcEFcr7TvpBiNQul9s=; b=yDLlIGW9Xdc2gj+SPhDaiLj1PJfPX+rzrzqbmhdqmDsO5xQUV+on5jFjRm8WE15tNQ AnPpifvn4AK0WCiTUk9QGSBJhaSS8BSObP6xs8EtpFUwPtKm8ySU+bl47umTjFpgS4mB 4pA6pGAa3pm2vzMj73DnSHbMSVgr6O+juHyb2Dy1ryZB76Hb5sd0Qf/v0x0EQWUHbsYA zEOywC8ggx+MJ72xk1jukXa3Hhh8cdKWhcXxlo/5dAOBY7tAd3spX67ERBRDlrxwHQ9t 0CK9Ly+/2vZM8Uin3lyC8u402IO7mIKgUvFn824FYvcwiXfAlryqWz94AvYwdjyvf2N0 tTJA== MIME-Version: 1.0 X-Received: by 10.107.166.72 with SMTP id p69mr10229082ioe.65.1439924780188; Tue, 18 Aug 2015 12:06:20 -0700 (PDT) Received: by 10.36.104.139 with HTTP; Tue, 18 Aug 2015 12:06:20 -0700 (PDT) In-Reply-To: References: Date: Tue, 18 Aug 2015 16:06:20 -0300 Message-ID: Subject: Re: Issue with /boot/menu.rc.local after 10.2 update From: Cassiano Peixoto To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2015 19:06:21 -0000 I just found out the problem. I don't know who added the following phrase to /boot/menu-commands.4th: only forth definitions Line 31 and 354. It just broke /boot/menu.rc.local as i said before. After removing that everything worked again. Hope it helps someone else. I'm going to open a PR anyway. Thanks. On Tue, Aug 18, 2015 at 11:13 AM, Cassiano Peixoto < peixotocassiano@gmail.com> wrote: > Hi guys, > > I've been using /boot/menu.rc.local for years to my custom menu option. > But after 10.2 update, it's not working anymore. I always get the message > "toggle_menuitem not found" on beastie menu. > > Here is my /boot/menu.rc.local: > > set optionsmenu_init[7]="init_console" > set optionsmenu_caption[7]="[C]onsole..... Off" > set optionstoggled_text[7]="[C]onsole..... On" > set optionsmenu_command[7]="toggle_console" > set optionsmenu_keycode[7]=118 > set optionsansi_caption[7]="^[[1mC^[[37monsole..... ^[[34;1mOff^[[37m" > set optionstoggled_ansi[7]="^[[1mC^[[37monsole..... ^[[32;7mOn^[[0;37m" > > \ > \ Console Boot > \ > > : console_enabled? ( -- flag ) > s" boot_single" getenv -1 <> dup if > drop ( c-addr flag -- flag ) > then > ; > > : console_enable ( -- ) > s" set console=comconsole,vidconsole" evaluate > ; > > : console_disable ( -- ) > s" set console=vidconsole,comconsole" evaluate > ; > > : toggle_console ( N -- N TRUE ) > toggle_menuitem > menu-redraw > > \ Now we're going to make the change effective > > dup toggle_stateN @ 0= if > console_disable > else > console_enable > then > > TRUE \ loop menu again > ; > > So what has changed? What should i do to make it works again? > > Thanks. > > > > From owner-freebsd-stable@freebsd.org Tue Aug 18 20:25:58 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 775EC9BDFF7 for ; Tue, 18 Aug 2015 20:25:58 +0000 (UTC) (envelope-from pgollucci@taximagic.com) Received: from mail-ob0-f177.google.com (mail-ob0-f177.google.com [209.85.214.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 454911FF6 for ; Tue, 18 Aug 2015 20:25:58 +0000 (UTC) (envelope-from pgollucci@taximagic.com) Received: by obbop1 with SMTP id op1so151881302obb.2 for ; Tue, 18 Aug 2015 13:25:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=EPJstEA41Yr848f6HIJIID0GKLehYE7GlrJ/2kLqvdA=; b=MiQa43VpjmI0glDBo1xKIXQ0MzqXpvIAGPQSmy+a0Fe1V6GhXKZGneRvtvrmbGpw6Y 2wLiSlNnhTws1umTqAaT/CIwTxxYix8PsoEHhGrBxcXlUCVKSXj7ZQzO9ymtm+cJ3SS9 YpyjdVaa+u3KrVqj9OJTHwAQ14EtdvL9C0Ph7KRPGQJcUz7tqLFm6Ipk8lbe2hPmvx3m o3Z/HMwLhV9+OM3LpnocSTcQ1VoHLbXFn8/Gv73SN9WTs4+YrJZd+u+Gh42Nvqjnlrw0 9qJ4Vo0MGAsQCVbmLE3vLgjlCoeDzf2mhqK/D0tmxJFVisksBy9WHDyEnpaKTrZ/VdR5 sS9g== X-Gm-Message-State: ALoCoQlyBvPE5/5g2LUoit4S7YmFZhxbkN9bVRjh9+myeP+FTEimnnvjluv38gkFn7IYIiZnTMlI X-Received: by 10.182.20.141 with SMTP id n13mr7930647obe.26.1439929557135; Tue, 18 Aug 2015 13:25:57 -0700 (PDT) MIME-Version: 1.0 Received: by 10.202.197.81 with HTTP; Tue, 18 Aug 2015 13:25:37 -0700 (PDT) In-Reply-To: <20150818201541.C15361893@freefall.freebsd.org> References: <20150818201541.C15361893@freefall.freebsd.org> From: Philip Gollucci Date: Tue, 18 Aug 2015 16:25:37 -0400 Message-ID: Subject: Re: [FreeBSD-Announce] FreeBSD Errata Notice FreeBSD-EN-15:12.netstat To: freebsd-stable@freebsd.org Cc: FreeBSD Errata Notices Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2015 20:25:58 -0000 We do not run 32bit. On Tue, Aug 18, 2015 at 4:15 PM, FreeBSD Errata Notices < errata-notices@freebsd.org> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA512 > > > ============================================================================= > FreeBSD-EN-15:12.netstat Errata > Notice > The FreeBSD > Project > > Topic: Incorrect netstat(1) data handling on 32-bit systems > > Category: core > Module: netstat > Announced: 2015-08-18 > Credits: Mark Johnston > Affects: FreeBSD 10.2-RELEASE > Corrected: 2015-07-31 00:21:41 UTC (stable/10, 10.2-STABLE) > 2015-08-18 19:30:17 UTC (releng/10.2, 10.2-RC3-p1) > 2015-08-18 19:30:17 UTC (releng/10.2, 10.2-RELEASE-p1) > > For general information regarding FreeBSD Errata Notices and Security > Advisories, including descriptions of the fields above, security > branches, and the following sections, please visit > . > > I. Background > > The netstat(1) utility displays the contents of various network related > data > structures. > > II. Problem Description > > The netstat(1) utility incorrectly handles reported values on 32-bit > systems. > > III. Impact > > Due to how netstat(1) processes IPSEC counters, the utility may produce > incorrect output on 32-bit systems. > > IV. Workaround > > No workaround is available, however systems without IPSEC compiled into the > kernel are not affected. > > V. Solution > > Perform one of the following: > > 1) Upgrade your system to a supported FreeBSD stable or release / security > branch (releng) dated after the correction date. > > 2) To update your present system via a binary patch: > > Systems running a RELEASE version of FreeBSD on the i386 or amd64 > platforms can be updated via the freebsd-update(8) utility: > > # freebsd-update fetch > # freebsd-update install > > 3) To update your present system via a source code patch: > > The following patches have been verified to apply to the applicable > FreeBSD release branches. > > a) Download the relevant patch from the location below, and verify the > detached PGP signature using your PGP utility. > > # fetch https://security.FreeBSD.org/patches/EN-15:12/netstat.patch > # fetch https://security.FreeBSD.org/patches/EN-15:12/netstat.patch.asc > # gpg --verify netstat.patch.asc > > b) Apply the patch. Execute the following commands as root: > > # cd /usr/src > # patch < /path/to/patch > > c) Recompile the operating system using buildworld and installworld as > described in . > > VI. Correction details > > The following list contains the correction revision numbers for each > affected branch. > > Branch/path Revision > - ------------------------------------------------------------------------- > stable/10/ r286099 > releng/10.2/ r286901 > - ------------------------------------------------------------------------- > > To see which files were modified by a particular revision, run the > following command, replacing NNNNNN with the revision number, on a > machine with Subversion installed: > > # svn diff -cNNNNNN --summarize svn://svn.freebsd.org/base > > Or visit the following URL, replacing NNNNNN with the revision number: > > > > VII. References > > > > The latest revision of this Errata Notice is available at > https://security.FreeBSD.org/advisories/FreeBSD-EN-15:12.netstat.asc > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.1.7 (FreeBSD) > > iQIcBAEBCgAGBQJV05A1AAoJEO1n7NZdz2rnAzIQANyLdOOQhe9dHyAV4N5YKM3B > Z/86dY/KIIViVb1uzkBASBNnkHlG+mCMOQpzX2x8yCPF4i7bIEfPa4r2Bzw9pvCF > RWRKvZXERESh/RndQFhxcmJMAyYPq7MdK0IZzG53vinlMoSz2WTKj2vSR7t2jfo+ > ObTfgdkqN/PZs/W+AQY8a4DMdxCLg1KCZiSpQRO7ea+4AxsI8qNgoytvG6HRno/z > uGe6Ad82ZfysKgqe9JO4gvRTR77ebQAVSSr3qylQcOGHohy9tFHcI2FEAAqLJrQY > b5DDLOawLRsQm0hwkLCTOZX2QvIFgz0gGRpvPcN9ZKValMc5DKQv36z3hOByK+3i > dDHFG/Diy2JNP0tsKtW8IyyLvW2DAUoTs1nVaWMvLKkMUr+loOYvoaLdGT0xQP2d > M6UT40mRMznfH/Gq/0DJFArsYcyB9YRl7rD0dy1HhqApogHQrTjsT+1vtBtpaTmv > LHA77tHyzI0TxOvmx3hglj/z4BLZDPU6ydXr3zeOYBpLz5p02GKxHUc+JrmWBfOV > Jep0+Fr2fYST5bGVtExNQV6cTlBZPnGR4JxJEUQA6a+FdyJcDuzOTNcs0YzwjuSC > dIk5pdxI3nkc+zf9GZLXUdLcxXfo6jBUy0fSWkzirGzBfo0wxseE6cbxxTH7vumx > CLGGmHiqxVuF/nP4ScHi > =3aK1 > -----END PGP SIGNATURE----- > _______________________________________________ > freebsd-announce@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-announce > To unsubscribe, send any mail to "freebsd-announce-unsubscribe@freebsd.org > " > -- *PHILIP M. GOLLUCCI* Sr. Director of IT | *Curb* *w.* 703-579-6947 *m.* 703-336-9354 @gocurb Enter code* p6magic* for $10 off your first ride From owner-freebsd-stable@freebsd.org Tue Aug 18 20:27:03 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C0F0B9BC09E for ; Tue, 18 Aug 2015 20:27:03 +0000 (UTC) (envelope-from pgollucci@taximagic.com) Received: from mail-oi0-f46.google.com (mail-oi0-f46.google.com [209.85.218.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8CD0317C for ; Tue, 18 Aug 2015 20:27:03 +0000 (UTC) (envelope-from pgollucci@taximagic.com) Received: by oiew67 with SMTP id w67so89050354oie.2 for ; Tue, 18 Aug 2015 13:26:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=qQ8Xwop/gfBbYRfv9eOJLDX3d5mrvBizMEcAywCG5oc=; b=gvoaBUyaF1pmggD4xPGa3A/3FUpf2lVpzw3UxaT77diS8w2Q/TAPkQg7BI/7jGKyR3 KHznfEeFoiTXAISEy9aBk6QrT5Cfg55fUi0P/nc8g+01f4MAx8GVToQat2D8feFC4HgC zXY/eYwfi5iUcYBLBd/v2jBNgQ2zYTyNkf+20oX+GoE81ThASOVl3aVodiIGeS3eNFVe l2ZxAbkhkKoKC7yc4dJVrfSDMxMjxDoeACJSAijCB/0T3W6xCk6rTVdtrzk0wRVtb20v WIbNTv21p7fU+b1Qgw4xDXXK6uzhAbW3ELHNYqekBg4091hA88ZpJBfJaSQJLj2ncXlG Zdpw== X-Gm-Message-State: ALoCoQkWweGuut1JaAHiGEtfT40vx4YBtDutgZ1QuFPNAmW/KRVKvwOM8YRRHZEFJ0zxW1pq2wT6 X-Received: by 10.202.189.5 with SMTP id n5mr6580303oif.17.1439929616966; Tue, 18 Aug 2015 13:26:56 -0700 (PDT) MIME-Version: 1.0 Received: by 10.202.197.81 with HTTP; Tue, 18 Aug 2015 13:26:37 -0700 (PDT) In-Reply-To: References: <20150818201541.C15361893@freefall.freebsd.org> From: Philip Gollucci Date: Tue, 18 Aug 2015 16:26:37 -0400 Message-ID: Subject: Re: [FreeBSD-Announce] FreeBSD Errata Notice FreeBSD-EN-15:12.netstat To: freebsd-stable@freebsd.org Cc: FreeBSD Errata Notices Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2015 20:27:03 -0000 sorry, wrong cc. On Tue, Aug 18, 2015 at 4:25 PM, Philip Gollucci wrote: > We do not run 32bit. > > On Tue, Aug 18, 2015 at 4:15 PM, FreeBSD Errata Notices < > errata-notices@freebsd.org> wrote: > >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA512 >> >> >> ============================================================================= >> FreeBSD-EN-15:12.netstat Errata >> Notice >> The FreeBSD >> Project >> >> Topic: Incorrect netstat(1) data handling on 32-bit systems >> >> Category: core >> Module: netstat >> Announced: 2015-08-18 >> Credits: Mark Johnston >> Affects: FreeBSD 10.2-RELEASE >> Corrected: 2015-07-31 00:21:41 UTC (stable/10, 10.2-STABLE) >> 2015-08-18 19:30:17 UTC (releng/10.2, 10.2-RC3-p1) >> 2015-08-18 19:30:17 UTC (releng/10.2, 10.2-RELEASE-p1) >> >> For general information regarding FreeBSD Errata Notices and Security >> Advisories, including descriptions of the fields above, security >> branches, and the following sections, please visit >> . >> >> I. Background >> >> The netstat(1) utility displays the contents of various network related >> data >> structures. >> >> II. Problem Description >> >> The netstat(1) utility incorrectly handles reported values on 32-bit >> systems. >> >> III. Impact >> >> Due to how netstat(1) processes IPSEC counters, the utility may produce >> incorrect output on 32-bit systems. >> >> IV. Workaround >> >> No workaround is available, however systems without IPSEC compiled into >> the >> kernel are not affected. >> >> V. Solution >> >> Perform one of the following: >> >> 1) Upgrade your system to a supported FreeBSD stable or release / security >> branch (releng) dated after the correction date. >> >> 2) To update your present system via a binary patch: >> >> Systems running a RELEASE version of FreeBSD on the i386 or amd64 >> platforms can be updated via the freebsd-update(8) utility: >> >> # freebsd-update fetch >> # freebsd-update install >> >> 3) To update your present system via a source code patch: >> >> The following patches have been verified to apply to the applicable >> FreeBSD release branches. >> >> a) Download the relevant patch from the location below, and verify the >> detached PGP signature using your PGP utility. >> >> # fetch https://security.FreeBSD.org/patches/EN-15:12/netstat.patch >> # fetch https://security.FreeBSD.org/patches/EN-15:12/netstat.patch.asc >> # gpg --verify netstat.patch.asc >> >> b) Apply the patch. Execute the following commands as root: >> >> # cd /usr/src >> # patch < /path/to/patch >> >> c) Recompile the operating system using buildworld and installworld as >> described in . >> >> VI. Correction details >> >> The following list contains the correction revision numbers for each >> affected branch. >> >> Branch/path Revision >> - >> ------------------------------------------------------------------------- >> stable/10/ r286099 >> releng/10.2/ r286901 >> - >> ------------------------------------------------------------------------- >> >> To see which files were modified by a particular revision, run the >> following command, replacing NNNNNN with the revision number, on a >> machine with Subversion installed: >> >> # svn diff -cNNNNNN --summarize svn://svn.freebsd.org/base >> >> Or visit the following URL, replacing NNNNNN with the revision number: >> >> >> >> VII. References >> >> >> >> The latest revision of this Errata Notice is available at >> https://security.FreeBSD.org/advisories/FreeBSD-EN-15:12.netstat.asc >> >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v2.1.7 (FreeBSD) >> >> iQIcBAEBCgAGBQJV05A1AAoJEO1n7NZdz2rnAzIQANyLdOOQhe9dHyAV4N5YKM3B >> Z/86dY/KIIViVb1uzkBASBNnkHlG+mCMOQpzX2x8yCPF4i7bIEfPa4r2Bzw9pvCF >> RWRKvZXERESh/RndQFhxcmJMAyYPq7MdK0IZzG53vinlMoSz2WTKj2vSR7t2jfo+ >> ObTfgdkqN/PZs/W+AQY8a4DMdxCLg1KCZiSpQRO7ea+4AxsI8qNgoytvG6HRno/z >> uGe6Ad82ZfysKgqe9JO4gvRTR77ebQAVSSr3qylQcOGHohy9tFHcI2FEAAqLJrQY >> b5DDLOawLRsQm0hwkLCTOZX2QvIFgz0gGRpvPcN9ZKValMc5DKQv36z3hOByK+3i >> dDHFG/Diy2JNP0tsKtW8IyyLvW2DAUoTs1nVaWMvLKkMUr+loOYvoaLdGT0xQP2d >> M6UT40mRMznfH/Gq/0DJFArsYcyB9YRl7rD0dy1HhqApogHQrTjsT+1vtBtpaTmv >> LHA77tHyzI0TxOvmx3hglj/z4BLZDPU6ydXr3zeOYBpLz5p02GKxHUc+JrmWBfOV >> Jep0+Fr2fYST5bGVtExNQV6cTlBZPnGR4JxJEUQA6a+FdyJcDuzOTNcs0YzwjuSC >> dIk5pdxI3nkc+zf9GZLXUdLcxXfo6jBUy0fSWkzirGzBfo0wxseE6cbxxTH7vumx >> CLGGmHiqxVuF/nP4ScHi >> =3aK1 >> -----END PGP SIGNATURE----- >> _______________________________________________ >> freebsd-announce@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-announce >> To unsubscribe, send any mail to " >> freebsd-announce-unsubscribe@freebsd.org" >> > > > > -- > *PHILIP M. GOLLUCCI* > Sr. Director of IT | *Curb* > *w.* 703-579-6947 > *m.* 703-336-9354 > @gocurb > > Enter code* p6magic* for $10 off your first ride > -- *PHILIP M. GOLLUCCI* Sr. Director of IT | *Curb* *w.* 703-579-6947 *m.* 703-336-9354 @gocurb Enter code* p6magic* for $10 off your first ride From owner-freebsd-stable@freebsd.org Tue Aug 18 21:54:49 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C14229BB92C; Tue, 18 Aug 2015 21:54:49 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 4CC721DA6; Tue, 18 Aug 2015 21:54:48 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:LX178BFLw6jt9mxB25wmPZ1GYnF86YWxBRYc798ds5kLTJ75oMSwAkXT6L1XgUPTWs2DsrQf27GQ7/2rAT1IyK3CmU5BWaQEbwUCh8QSkl5oK+++Imq/EsTXaTcnFt9JTl5v8iLzG0FUHMHjew+a+SXqvnYsExnyfTB4Ov7yUtaLyZ/njKbuptaLMk1hv3mUX/BbFF2OtwLft80b08NJC50a7V/3mEZOYPlc3mhyJFiezF7W78a0+4N/oWwL46pyv+YJa6jxfrw5QLpEF3xmdjltvIy4/SXEGCuG4GBUamgKjhdSSzPI6BjhXYa55ivircJm1S2TJs7nC7cuVmLxwb1sTUrSiSwEfxsw+2LTh8k42LheqRmioxF665PTb5yYMOJ+OKjUK4BJDVFdV9pcAnQSSri3aJECWq9YZb5V X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2BAAgBNqdNV/61jaINdg29pBoMeumQBCYFtCoUxSgKBcxQBAQEBAQEBAYEJgh2CBgEBAQMBAQEBIAQnIAsFCwIBCBgCAg0ZAgInAQkmAgQIBwQBHASIBQgNu2yWHwEBAQEBAQEBAQEBAQEBAQEBARYEgSKKMYQyBgEBHDQHgmmBQwWVIYUEhQadDwImhBkiMwd/CBcjgQQBAQE X-IronPort-AV: E=Sophos;i="5.15,705,1432612800"; d="scan'208";a="233260254" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-annu.net.uoguelph.ca with ESMTP; 18 Aug 2015 17:54:09 -0400 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 3C93B15F565; Tue, 18 Aug 2015 17:54:09 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id YF57cG3-UtGe; Tue, 18 Aug 2015 17:54:08 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 74D0115F56D; Tue, 18 Aug 2015 17:54:08 -0400 (EDT) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id e_V1IkBsZs1R; Tue, 18 Aug 2015 17:54:08 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 53ACE15F565; Tue, 18 Aug 2015 17:54:08 -0400 (EDT) Date: Tue, 18 Aug 2015 17:54:08 -0400 (EDT) From: Rick Macklem To: Hans Petter Selasky Cc: Daniel Braniss , FreeBSD Net , Christopher Forgeron , FreeBSD stable , Slawa Olhovchenkov Message-ID: <1325951625.25292515.1439934848268.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <55D333D6.5040102@selasky.org> References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <17871443-E105-4434-80B1-6939306A865F@cs.huji.ac.il> <473274181.23263108.1439814072514.JavaMail.zimbra@uoguelph.ca> <7F892C70-9C04-4468-9514-EDBFE75CF2C6@cs.huji.ac.il> <805850043.24018217.1439848150695.JavaMail.zimbra@uoguelph.ca> <9D8B0503-E8FA-43CA-88F0-01F184F84D9B@cs.huji.ac.il> <1721122651.24481798.1439902381663.JavaMail.zimbra@uoguelph.ca> <55D333D6.5040102@selasky.org> Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.10] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF34 (Win)/8.0.9_GA_6191) Thread-Topic: ix(intel) vs mlxen(mellanox) 10Gb performance Thread-Index: 2kCu0WEXLG1Xa/qUNDzhyfV3vrvVUQ== X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2015 21:54:49 -0000 Hans Petter Selasky wrote: > On 08/18/15 14:53, Rick Macklem wrote: > > 2572 ifp->if_hw_tsomax = 65518; > >> >2573 ifp->if_hw_tsomaxsegcount = IXGBE_82599_SCATTER; > >> >2574 ifp->if_hw_tsomaxsegsize = 2048; > > Hi, > > If IXGBE_82599_SCATTER is the maximum scatter/gather entries the > hardware can do, remember to subtract one fragment for the TCP/IP-header > mbuf! > Ouch! Yes, I now see that the code that counts the # of mbufs is before the code that adds the tcp/ip header mbuf. In my opinion, this should be fixed by setting if_hw_tsomaxsegcount to whatever the driver provides - 1. It is not the driver's responsibility to know if a tcp/ip header mbuf will be added and is a lot less confusing that expecting the driver author to know to subtract one. (I had mistakenly thought that tcp_output() had added the tc/ip header mbuf before the loop that counts mbufs in the list. Btw, this tcp/ip header mbuf also has leading space for the MAC layer header.) > I think there is an off-by-one here: > > ifp->if_hw_tsomax = 65518; > ifp->if_hw_tsomaxsegcount = IXGBE_82599_SCATTER - 1; > ifp->if_hw_tsomaxsegsize = 2048; > > Refer to: > > > * > > * NOTE: The TSO limits only apply to the data payload part of > > * a TCP/IP packet. That means there is no need to subtract > > * space for ethernet-, vlan-, IP- or TCP- headers from the > > * TSO limits unless the hardware driver in question requires > > * so. > This comment suggests that the driver author doesn't need to do this. However, unless this is fixed in tcp_output(), the above patch should be applied to the driver. > In sys/net/if_var.h > > Thank you! > > --HPS > The problem I see is that, after doing the calculation of how many mbufs can be in the TSO segment, the code in tcp_output() will have calculated a value for "len" that will always be less that "tp->t_maxopd - optlen" when the if_hw_tsosegcount limit has been hit (see where it does a "break;" out of the while loop). --> This does not imply "too many small fragments" for NFS, just that the driver's transmit segment limit has been reached, where most of them are mbuf clusters, but not the first ones. As such the code: /* * In case there are too many small fragments * don't use TSO: */ if (len <= max_len) { len = max_len; sendalot = 1; tso = 0; } Will always happen for this case and "tso" gets set to 0. Not what we want to happen, imho. The above code block was what I suggested should be commented out or deleted for the test. It appears you should also add the "- 1" in the driver sys/dev/ixgbe/if_ix.c. rick > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-stable@freebsd.org Tue Aug 18 22:04:33 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 37FB59BBB99; Tue, 18 Aug 2015 22:04:33 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id C848C3E8; Tue, 18 Aug 2015 22:04:32 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:25oK8xU/RUE9GnOdjyMBtBKbuiPV8LGtZVwlr6E/grcLSJyIuqrYZhOPt8tkgFKBZ4jH8fUM07OQ6PC7HzBdqs7Q+Fk5M7VyFDY9wf0MmAIhBMPXQWbaF9XNKxIAIcJZSVV+9Gu6O0UGUOz3ZlnVv2HgpWVKQka3CwN5K6zPF5LIiIzvjqbpq8aVP1gD3Gv1SIgxBSv1hD2ZjtMRj4pmJ/R54TryiVwMRd5rw3h1L0mYhRf265T41pdi9yNNp6BprJYYAu3SNp41Rr1ADTkgL3t9pIiy7UGCHkOz4S48W2MN2iJFHxTI9lnBU5P4qSjr/r59wDKyJsDyRKs3SHKl9ag9GzHyjyJSDT8y8ynyg8dziK9e6Ea7ohV0wIrZZamIM/Vjc6fFfZURTDwSDY5qSyVdD9bkPMM0BO0bMLMd9tGlqg== X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2A9AgDBqtNV/61jaINdhF6DHrpkAQmFGoJYAoFzFAEBAQEBAQEBgQmCHYIHAQEEIwRSEAIBCBgCAg0ZAgJXAgSIQbt/lh8BAQEBAQEBAwEBAQEBARyBIooxhFY0B4JpgUMFlSGnGQImgg4cgW8igXuBBAEBAQ X-IronPort-AV: E=Sophos;i="5.15,705,1432612800"; d="scan'208";a="231546692" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 18 Aug 2015 18:04:25 -0400 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id D505315F565; Tue, 18 Aug 2015 18:04:25 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id KC3hLXnbHEvU; Tue, 18 Aug 2015 18:04:25 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 489E015F56D; Tue, 18 Aug 2015 18:04:25 -0400 (EDT) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id Ezlh0gWCO7WZ; Tue, 18 Aug 2015 18:04:25 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 276D315F565; Tue, 18 Aug 2015 18:04:25 -0400 (EDT) Date: Tue, 18 Aug 2015 18:04:25 -0400 (EDT) From: Rick Macklem To: Hans Petter Selasky Cc: Daniel Braniss , FreeBSD Net , Slawa Olhovchenkov , FreeBSD stable , Christopher Forgeron Message-ID: <805386587.25297673.1439935465127.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <55D331A5.9050601@selasky.org> References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <17871443-E105-4434-80B1-6939306A865F@cs.huji.ac.il> <473274181.23263108.1439814072514.JavaMail.zimbra@uoguelph.ca> <7F892C70-9C04-4468-9514-EDBFE75CF2C6@cs.huji.ac.il> <805850043.24018217.1439848150695.JavaMail.zimbra@uoguelph.ca> <9D8B0503-E8FA-43CA-88F0-01F184F84D9B@cs.huji.ac.il> <1721122651.24481798.1439902381663.JavaMail.zimbra@uoguelph.ca> <55D331A5.9050601@selasky.org> Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.11] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF34 (Win)/8.0.9_GA_6191) Thread-Topic: ix(intel) vs mlxen(mellanox) 10Gb performance Thread-Index: Jou6U2JAhtGQyrGph78WF7y8CruWvQ== X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2015 22:04:33 -0000 Hans Petter Selasky wrote: > On 08/18/15 14:53, Rick Macklem wrote: > > If this is just a test machine, maybe you could test with these lines (at > > about #880) > > in sys/netinet/tcp_output.c commented out? (It looks to me like this will > > disable TSO > > for almost all the NFS writes.) > > - around line #880 in sys/netinet/tcp_output.c: > > /* > > * In case there are too many small fragments > > * don't use TSO: > > */ > > if (len <= max_len) { > > len = max_len; > > sendalot = 1; > > tso = 0; > > } > > > > This was added along with the other stuff that did the > > if_hw_tsomaxsegcount, etc and I > > never noticed it until now (not my patch). > > FYI: > > These lines are needed by other hardware, like the mlxen driver. If you > remove them mlxen will start doing m_defrag(). I believe if you set the > correct parameters in the "struct ifnet" for the TSO size/count limits > this problem will go away. If you print the "len" and "max_len" and also > the cases where TSO limits are reached, you'll see what parameter is > triggering it and needs to be increased. > Well, if the driver isn't setting if_hw_tsomaxsegcount correctly, then it is the driver that needs to be fixed. Having the above code block disable TSO for all of the NFS writes, including the ones that set if_hw_tsomaxsegcount correctly doesn't make sense to me. If the driver authors don't set these, the drivers do lots of m_defrag() calls. I have posted more than once to freebsd-net@ asking the driver authors to set these and some now have. (I can't do it, because I don't have the hardware to test it with.) I do think that most/all of them don't subtract 1 for the tcp/ip header and I don't think they should be expected to, since the driver isn't supposed to worry about the protocol at that level. --> I think tcp_output() should subtract one from the if_hw_tsomaxsegcount provided by the driver to handle this, since it chooses to count mbufs (the while() loop at around line #825 in sys/netinet/tcp_output.c.) before it prepends the tcp/ip header mbuf. rick > --HPS > From owner-freebsd-stable@freebsd.org Tue Aug 18 23:20:15 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1E0269BD9AF for ; Tue, 18 Aug 2015 23:20:15 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 018949B1 for ; Tue, 18 Aug 2015 23:20:15 +0000 (UTC) (envelope-from david@catwhisker.org) Received: by mailman.ysv.freebsd.org (Postfix) id F12809BD9AD; Tue, 18 Aug 2015 23:20:14 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D7C019BD9AB; Tue, 18 Aug 2015 23:20:14 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (mx.catwhisker.org [198.144.209.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F0E199A9; Tue, 18 Aug 2015 23:20:13 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.15.2/8.15.2) with ESMTP id t7INK86g058120; Tue, 18 Aug 2015 16:20:08 -0700 (PDT) (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.15.2/8.15.2/Submit) id t7INK7w9058119; Tue, 18 Aug 2015 16:20:07 -0700 (PDT) (envelope-from david) Date: Tue, 18 Aug 2015 16:20:07 -0700 From: David Wolfskill To: stable@freebsd.org, net@freebsd.org Subject: Panic [page fault] in _ieee80211_crypto_delkey(): stable/10/amd64 @r286878 Message-ID: <20150818232007.GN1189@albert.catwhisker.org> Mail-Followup-To: David Wolfskill , stable@freebsd.org, net@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="V2Kn1ZfDiPlyQ6Qk" Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2015 23:20:15 -0000 --V2Kn1ZfDiPlyQ6Qk Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable I was minding my own business in a staff meeting this afternoon, and my laptop rebooted; seems it got a panic. I've copied the core.txt.0 file to , along with a verbose dmesg.boot from this morning and output of "pciconf -l -v". This was running: FreeBSD localhost 10.2-STABLE FreeBSD 10.2-STABLE #122 r286878M/286880:100= 2500: Tue Aug 18 04:06:33 PDT 2015 root@g1-252.catwhisker.org:/common/S= 1/obj/usr/src/sys/CANARY amd64 Excerpts from core.txt.0: panic: page fault =2E.. Unread portion of the kernel message buffer: panic: page fault cpuid =3D 2 KDB: stack backtrace: #0 0xffffffff80946e00 at kdb_backtrace+0x60 #1 0xffffffff8090a9e6 at vpanic+0x126 #2 0xffffffff8090a8b3 at panic+0x43 #3 0xffffffff80c8467b at trap_fatal+0x36b #4 0xffffffff80c8497d at trap_pfault+0x2ed #5 0xffffffff80c8401a at trap+0x47a #6 0xffffffff80c6a1b2 at calltrap+0x8 #7 0xffffffff809eff5e at ieee80211_crypto_delkey+0x1e #8 0xffffffff80a04d45 at ieee80211_ioctl_delkey+0x65 #9 0xffffffff80a03bd2 at ieee80211_ioctl_set80211+0x572 #10 0xffffffff80a2c323 at in_control+0x203 #11 0xffffffff809cd57b at ifioctl+0x15eb #12 0xffffffff8095ecf5 at kern_ioctl+0x255 #13 0xffffffff8095e9f0 at sys_ioctl+0x140 #14 0xffffffff80c84f97 at amd64_syscall+0x357 #15 0xffffffff80c6a49b at Xfast_syscall+0xfb Uptime: 9h45m0s =2E.. Loaded symbols for /usr/local/modules/rtc.ko #0 doadump (textdump=3D) at pcpu.h:219 219 pcpu.h: No such file or directory. in pcpu.h (kgdb) #0 doadump (textdump=3D) at pcpu.h:219 #1 0xffffffff8090a642 in kern_reboot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c:451 #2 0xffffffff8090aa25 in vpanic (fmt=3D,=20 ap=3D) at /usr/src/sys/kern/kern_shutdown.c:758 #3 0xffffffff8090a8b3 in panic (fmt=3D0x0) at /usr/src/sys/kern/kern_shutdown.c:687 #4 0xffffffff80c8467b in trap_fatal (frame=3D,=20 eva=3D) at /usr/src/sys/amd64/amd64/trap.c:851 #5 0xffffffff80c8497d in trap_pfault (frame=3D0xfffffe060d88b510,=20 usermode=3D) at /usr/src/sys/amd64/amd64/trap.c:674 #6 0xffffffff80c8401a in trap (frame=3D0xfffffe060d88b510) at /usr/src/sys/amd64/amd64/trap.c:440 #7 0xffffffff80c6a1b2 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:236 #8 0xffffffff809f003a in _ieee80211_crypto_delkey () at /usr/src/sys/net80211/ieee80211_crypto.c:105 #9 0xffffffff809eff5e in ieee80211_crypto_delkey (vap=3D0xfffffe03d9070000= ,=20 key=3D0xfffffe03d9070800) at /usr/src/sys/net80211/ieee80211_crypto.c:4= 61 #10 0xffffffff80a04d45 in ieee80211_ioctl_delkey (vap=3D0xfffffe03d9070000,= =20 ireq=3D) at /usr/src/sys/net80211/ieee80211_ioctl.c:1252 #11 0xffffffff80a03bd2 in ieee80211_ioctl_set80211 () at /usr/src/sys/net80211/ieee80211_ioctl.c:2814 #12 0xffffffff80a2c323 in in_control (so=3D,=20 cmd=3D9214790412651315593, data=3D0xfffffe060d88bb80 "", ifp=3D0x3,=20 td=3D) at /usr/src/sys/netinet/in.c:308 #13 0xffffffff809cd57b in ifioctl (so=3D0xfffffe03d9070800, cmd=3D214960791= 4,=20 data=3D0xfffffe060d88b8e0 "wlan0", td=3D0xfffff80170abb940) at /usr/src/sys/net/if.c:2770 #14 0xffffffff8095ecf5 in kern_ioctl (td=3D0xfffff80170abb940,=20 fd=3D, com=3D18446741891212314624) at file.h:320 #15 0xffffffff8095e9f0 in sys_ioctl (td=3D0xfffff80170abb940,=20 uap=3D0xfffffe060d88ba40) at /usr/src/sys/kern/sys_generic.c:718 #16 0xffffffff80c84f97 in amd64_syscall (td=3D0xfffff80170abb940, traced=3D= 0) at subr_syscall.c:134 #17 0xffffffff80c6a49b in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:396 #18 0x00000008012a2f9a in ?? () Previous frame inner to this frame (corrupt stack?) Current language: auto; currently minimal (kgdb)=20 Physical 802.11 hardware is iwn(4). I can copy the vmcore.0 file itself after I get home -- it's ~625MB, and I'd rather not try to get that through over a WAN before I need to catch the shuttle to get home. :-} Peace, david --=20 David H. Wolfskill david@catwhisker.org Those who would murder in the name of God or prophet are blasphemous coward= s. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --V2Kn1ZfDiPlyQ6Qk Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQJ8BAEBCgBmBQJV072nXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ4RThEMDY4QTIxMjc1MDZFRDIzODYzRTc4 QTY3RjlDOERFRjQxOTNCAAoJEIpn+cje9Bk76qkP/jFmvBCveCG+FQCQw1CCux/k HDsAlV4C55R24EuTS3PC63WWH7nRl7soQkrwx2DJqpEAQOnV9DFRsfs+jwGLFAHF kflanGr/XXsT3PFV6dQkmfQhRVV2v4QYPVo0/5FtbqkwHucxKrH2RswoReRve3bD qmcrv80OHGsqnWApq7FetD4qDyeWoI6OIS4WZw87CQhqAv8fEvhrk/aeIvjUz2SC 9Bw2WcRGsyYs0MIL8pS5iaA9XUh7I2DER/odoIipPD/ZjhGqJG9+gM6yBJ27gidX pfcy33vApR6rT5tQlNvVubmEnSm/rH9K/xRSOvLvCeqgAKOWjhXtnLXqzfhq1JlM U0SmNNHZTOMAzNg4dwPgjxavjZAFSsGuBU3mHOF6P+ymSXi2vkYFJhmHr7Wvaqh+ dRhKQ8NjwhEMI5BUti5UwY3zAjD5phxOzqUnHaetPYTZcEJuob+5Fs3n0V1ji3Jx cGy8RMnl2Ly2vfD7f77domWd7SHTRf/1PEyKu1NnPaLImLyI1FNcZN6xjyiIxrrz hEhPRPgnfiyr3uGPxpk+QgxXfGysS089RIgGpagbXUIttAe5EjM0QI+tL++lD/tF M2fWm6rQ6WjDxKBmDac0ZtWgGHBcsYVv/YicH+gSCRgB7t5THwLlxJckIjh/tT7s SDDi4t5PwrrbabGur6m+ =ylI8 -----END PGP SIGNATURE----- --V2Kn1ZfDiPlyQ6Qk-- From owner-freebsd-stable@freebsd.org Wed Aug 19 02:42:01 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EB1929BD78A; Wed, 19 Aug 2015 02:42:01 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 74DB3191C; Wed, 19 Aug 2015 02:42:01 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:eIbFqhDlnn1Crzv6DF2DUyQJP3N1i/DPJgcQr6AfoPdwSPn6p8bcNUDSrc9gkEXOFd2CrakU0KyK7uu+AiQp2tWojjMrSNR0TRgLiMEbzUQLIfWuLgnFFsPsdDEwB89YVVVorDmROElRH9viNRWJ+iXhpQAbFhi3DwdpPOO9QteU1JTskbzvsMOIKyxzxxODIppKZC2sqgvQssREyaBDEY0WjiXzn31TZu5NznlpL1/A1zz158O34YIxu38I46Fp34d6XK77Z6U1S6BDRHRjajhtpZ6jiB/YUAHa5mcASn5E1V1MAhPZ91f0RJr8uDD28O1n126fNMzySLkyHjCj9LtqThHvzykdOjMz622SkdB5hqZW8y+nvAF1lo7IfJmOZr05eqLGYchcS3BMU8xKW2pGGIz7aoIOC+8IO6FcrpLhpl0AqlywHwShDvjjjyRUj3Xy0P4H1f88G1TGwBA4BIBJ93DVt8nucqkIXO2/16WOyi/MKPZf2DP44Y6PdhE6vfCKU7U3f9DcxEM0G0bDg0nDlYuwEzqT1+kJ+0KB5uxhTvnn32IurQdgijO0gMcxiIiPj4lTy1SSsW1ZyYAubeW1VFJ2e5afHZ9ZrCKLf992Wc4mSnprqQ400LALs4W3Oi8Qx8J06QTYbqm9coOLqjfqX+WVLDIw0Ghgcbm8gxu32VWnxfDxUtG0ll1D+HkW2uLQv2wAgkSAovOMTeFwqwL4gW6C X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2A6AgBW7NNV/61jaINdDoNhaQaDH7pfAQmBbQqFMUoCgXYUAQEBAQEBAQGBCYIdggcBAQQBAQEgBCcgCxACAQgOCgICDRYDAgIhBgEJFRECBAEHBwQBHASHeAMSDbp5kDQNhVcBAQEBAQEBAwEBAQEBAQEXBIEiijGCT4FiAQYBAQcVATMHgmmBQwWHI41/hQSFBnWDN5Evg0+DZQImgz9aIjMHfgEIFyOBBAEBAQ X-IronPort-AV: E=Sophos;i="5.15,706,1432612800"; d="scan'208";a="231577093" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 18 Aug 2015 22:41:59 -0400 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id A1A3F15F55D; Tue, 18 Aug 2015 22:41:59 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id wmVkoLeVSGkc; Tue, 18 Aug 2015 22:41:58 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 7FE1B15F571; Tue, 18 Aug 2015 22:41:58 -0400 (EDT) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id ZSabU6gBtopC; Tue, 18 Aug 2015 22:41:58 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 6054715F55D; Tue, 18 Aug 2015 22:41:58 -0400 (EDT) Date: Tue, 18 Aug 2015 22:41:58 -0400 (EDT) From: Rick Macklem To: Daniel Braniss , Hans Petter Selasky Cc: FreeBSD Net , Christopher Forgeron , FreeBSD stable , Slawa Olhovchenkov Message-ID: <333280926.25456572.1439952118371.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <9D8B0503-E8FA-43CA-88F0-01F184F84D9B@cs.huji.ac.il> References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <20150817094145.GB3158@zxy.spb.ru> <17871443-E105-4434-80B1-6939306A865F@cs.huji.ac.il> <473274181.23263108.1439814072514.JavaMail.zimbra@uoguelph.ca> <7F892C70-9C04-4468-9514-EDBFE75CF2C6@cs.huji.ac.il> <805850043.24018217.1439848150695.JavaMail.zimbra@uoguelph.ca> <9D8B0503-E8FA-43CA-88F0-01F184F84D9B@cs.huji.ac.il> Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [172.17.95.12] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF34 (Win)/8.0.9_GA_6191) Thread-Topic: ix(intel) vs mlxen(mellanox) 10Gb performance Thread-Index: zLdb+6CG04LhJxMghkR/pohBzDKCLQ== X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2015 02:42:02 -0000 Daniel Braniss wrote: >=20 > > On Aug 18, 2015, at 12:49 AM, Rick Macklem wrote= : > >=20 > > Daniel Braniss wrote: > >>=20 > >>> On Aug 17, 2015, at 3:21 PM, Rick Macklem wrot= e: > >>>=20 > >>> Daniel Braniss wrote: > >>>>=20 > >>>>> On Aug 17, 2015, at 1:41 PM, Christopher Forgeron > >>>>> > >>>>> wrote: > >>>>>=20 > >>>>> FYI, I can regularly hit 9.3 Gib/s with my Intel X520-DA2's and Fre= eBSD > >>>>> 10.1. Before 10.1 it was less. > >>>>>=20 > >>>>=20 > >>>> this is NOT iperf/3 where i do get close to wire speed, > >>>> it=E2=80=99s NFS writes, i.e., almost real work :-) > >>>>=20 > >>>>> I used to tweak the card settings, but now it's just stock. You may > >>>>> want > >>>>> to > >>>>> check your settings, the Mellanox may just have better defaults for > >>>>> your > >>>>> switch. > >>>>>=20 > >>> Have you tried disabling TSO for the Intel? With TSO enabled, it will= be > >>> copying > >>> every transmitted mbuf chain to a new chain of mbuf clusters via. > >>> m_defrag() when > >>> TSO is enabled. (Assuming you aren't an 82598 chip. Most seem to be t= he > >>> 82599 chip > >>> these days?) > >>>=20 Oops, I think I screwed up. It looks like t_maxopd is limited to somewhat l= ess than the mtu. If that is the case, the code block wouldn't do what I thought it would do. However, if_hw_tsomaxsegcount does need to be one less than the limit for t= he driver, since the tcp/ip header isn't yet prepended when it is counted. I think the code in tcp_output() should subtract 1, but you can change it i= n the driver to test this. Thanks for doing this, rick > >>=20 > >> hi Rick > >>=20 > >> how can i check the chip? > >>=20 > > Haven't a clue. Does "dmesg" tell you? (To be honest, since disabling T= SO > > helped, > > I'll bet you don't have a 82598.) > >=20 > >>> This has been fixed in the driver very recently, but those fixes won'= t be > >>> in 10.1. > >>>=20 > >>> rick > >>> ps: If you could test with 10.2, it would be interesting to see how t= he > >>> ix > >>> does with > >>> the current driver fixes in it? > >>=20 > >> I new TSO was involved! > >> ok, firstly, it=E2=80=99s 10.2 stable. > >> with TSO enabled, ix is bad, around 64MGB/s. > >> disabling TSO it=E2=80=99s better, around 130 > >>=20 > > Hmm, could you check to see of these lines are in sys/dev/ixgbe/if_ix.c= at > > around > > line#2500? > > /* TSO parameters */ > > 2572 =09 =09 ifp->if_hw_tsomax =3D 65518; > > 2573 =09 =09 ifp->if_hw_tsomaxsegcount =3D IXGBE_82599_SCATTER= ; > > 2574 =09 =09 ifp->if_hw_tsomaxsegsize =3D 2048; > >=20 > > They are in stable/10. I didn't look at releng/10.2. (And if they're in= a > > #ifdef > > for FreeBSD11, take the #ifdef away.) > > If they are there and not ifdef'd, I can't explain why disabling TSO wo= uld > > help. > > Once TSO is fixed so that it handles the 64K transmit segments without > > copying all > > the mbufs, I suspect you might get better perf. with it enabled? > >=20 >=20 > this is 10.2 : > they are on lines 2509-2511 and I don=E2=80=99t see any #ifdefs around i= t. >=20 > the plot thickens :-) >=20 > danny >=20 > > Good luck with it, rick > >=20 > >> still, mlxen0 is about 250! with and without TSO > >>=20 > >>=20 > >>>=20 > >>>>> On Mon, Aug 17, 2015 at 6:41 AM, Slawa Olhovchenkov >>>>> > wrote: > >>>>> On Mon, Aug 17, 2015 at 10:27:41AM +0300, Daniel Braniss wrote: > >>>>>=20 > >>>>>> hi, > >>>>>> I have a host (Dell R730) with both cards, connected to an HP8= 200 > >>>>>> switch at 10Gb. > >>>>>> when writing to the same storage (netapp) this is what I get: > >>>>>> ix0: ~130MGB/s > >>>>>> mlxen0 ~330MGB/s > >>>>>> this is via nfs/tcpv3 > >>>>>>=20 > >>>>>> I can get similar (bad) performance with the mellanox if I > >>>>>> increase > >>>>>> the file size > >>>>>> to 512MGB. > >>>>>=20 > >>>>> Look like mellanox have internal beffer for caching and do ACK > >>>>> acclerating. > >>>>>=20 > >>>>>> so at face value, it seems the mlxen does a better use of > >>>>>> resources > >>>>>> than the intel. > >>>>>> Any ideas how to improve ix/intel's performance? > >>>>>=20 > >>>>> Are you sure about netapp performance? > >>>>> _______________________________________________ > >>>>> freebsd-net@freebsd.org mailing li= st > >>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-net > >>>>> > >>>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.o= rg > >>>>> " > >>>>>=20 > >>>>=20 > >>>> _______________________________________________ > >>>> freebsd-stable@freebsd.org mailing list > >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-stable > >>>> To unsubscribe, send any mail to > >>>> "freebsd-stable-unsubscribe@freebsd.org" > >>=20 > >> _______________________________________________ > >> freebsd-stable@freebsd.org mailing list > >> https://lists.freebsd.org/mailman/listinfo/freebsd-stable > >> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.o= rg" >=20 > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Wed Aug 19 06:59:30 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ABB269BD956; Wed, 19 Aug 2015 06:59:30 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 67F8F1681; Wed, 19 Aug 2015 06:59:30 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 183721FE023; Wed, 19 Aug 2015 08:59:28 +0200 (CEST) Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance To: Rick Macklem References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <17871443-E105-4434-80B1-6939306A865F@cs.huji.ac.il> <473274181.23263108.1439814072514.JavaMail.zimbra@uoguelph.ca> <7F892C70-9C04-4468-9514-EDBFE75CF2C6@cs.huji.ac.il> <805850043.24018217.1439848150695.JavaMail.zimbra@uoguelph.ca> <9D8B0503-E8FA-43CA-88F0-01F184F84D9B@cs.huji.ac.il> <1721122651.24481798.1439902381663.JavaMail.zimbra@uoguelph.ca> <55D333D6.5040102@selasky.org> <1325951625.25292515.1439934848268.JavaMail.zimbra@uoguelph.ca> Cc: Daniel Braniss , FreeBSD Net , Christopher Forgeron , FreeBSD stable , Slawa Olhovchenkov From: Hans Petter Selasky Message-ID: <55D429A4.3010407@selasky.org> Date: Wed, 19 Aug 2015 09:00:52 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.1.0 MIME-Version: 1.0 In-Reply-To: <1325951625.25292515.1439934848268.JavaMail.zimbra@uoguelph.ca> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2015 06:59:30 -0000 On 08/18/15 23:54, Rick Macklem wrote: > Ouch! Yes, I now see that the code that counts the # of mbufs is before the > code that adds the tcp/ip header mbuf. > > In my opinion, this should be fixed by setting if_hw_tsomaxsegcount to whatever > the driver provides - 1. It is not the driver's responsibility to know if a tcp/ip > header mbuf will be added and is a lot less confusing that expecting the driver > author to know to subtract one. (I had mistakenly thought that tcp_output() had > added the tc/ip header mbuf before the loop that counts mbufs in the list. Btw, > this tcp/ip header mbuf also has leading space for the MAC layer header.) > Hi Rick, Your question is good. With the Mellanox hardware we have separate so-called inline data space for the TCP/IP headers, so if the TCP stack subtracts something, then we would need to add something to the limit, because then the scatter gather list is only used for the data part. Maybe it can be controlled by some kind of flag, if all the three TSO limits should include the TCP/IP/ethernet headers too. I'm pretty sure we want both versions. --HPS From owner-freebsd-stable@freebsd.org Wed Aug 19 07:30:23 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 86E7D9BD560; Wed, 19 Aug 2015 07:30:23 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-pd0-x229.google.com (mail-pd0-x229.google.com [IPv6:2607:f8b0:400e:c02::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 51A5E1AB8; Wed, 19 Aug 2015 07:30:23 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: by pdob1 with SMTP id b1so22219525pdo.2; Wed, 19 Aug 2015 00:30:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=qogDhQRuOFK/8Y4NwWfjhcQYE5bkkvb9vv4hdO8b688=; b=uBtsrv+nlh9+jpt1qEuc/lfvyHHI1MeApda4eX6HpBMMavTPwtPe+aC/Ir8H07nAsA agR/lb+IWyL/MDRccbRCzKE6pab6UX4MHU2xVLFlJ1MbBSYlRF6pDsF0/fjEHr/Ofymx 0iHNveprCGI2PeiKuT9b0Dya3CD5AAN2RwIq2ABSJ5zIFG56pMCTnB9q0NDtVWhh7nGB Zemq9Nm7e5V38NhSAApf7q5H1De9vW9G+L2/1YJQWIEb7AdJgCoJduCNE+vm8Zj/lQaA OzPmFzXpnkFssRIwSnEM9Bwmfjwi3xJEuussCv/U9vVF47ymT4k4YbQVFGMGySOow5/l ubdw== X-Received: by 10.70.44.228 with SMTP id h4mr21767238pdm.45.1439969422805; Wed, 19 Aug 2015 00:30:22 -0700 (PDT) Received: from pyunyh@gmail.com ([106.247.248.2]) by smtp.gmail.com with ESMTPSA id gw3sm20428044pbc.46.2015.08.19.00.30.17 (version=TLSv1 cipher=RC4-SHA bits=128/128); Wed, 19 Aug 2015 00:30:21 -0700 (PDT) From: Yonghyeon PYUN X-Google-Original-From: "Yonghyeon PYUN" Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Wed, 19 Aug 2015 16:30:10 +0900 Date: Wed, 19 Aug 2015 16:30:10 +0900 To: Rick Macklem Cc: Hans Petter Selasky , Daniel Braniss , FreeBSD Net , Christopher Forgeron , FreeBSD stable , Slawa Olhovchenkov Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance Message-ID: <20150819073010.GA964@michelle.fasterthan.com> Reply-To: pyunyh@gmail.com References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <17871443-E105-4434-80B1-6939306A865F@cs.huji.ac.il> <473274181.23263108.1439814072514.JavaMail.zimbra@uoguelph.ca> <7F892C70-9C04-4468-9514-EDBFE75CF2C6@cs.huji.ac.il> <805850043.24018217.1439848150695.JavaMail.zimbra@uoguelph.ca> <9D8B0503-E8FA-43CA-88F0-01F184F84D9B@cs.huji.ac.il> <1721122651.24481798.1439902381663.JavaMail.zimbra@uoguelph.ca> <55D331A5.9050601@selasky.org> <805386587.25297673.1439935465127.JavaMail.zimbra@uoguelph.ca> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <805386587.25297673.1439935465127.JavaMail.zimbra@uoguelph.ca> User-Agent: Mutt/1.4.2.3i X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2015 07:30:23 -0000 On Tue, Aug 18, 2015 at 06:04:25PM -0400, Rick Macklem wrote: > Hans Petter Selasky wrote: > > On 08/18/15 14:53, Rick Macklem wrote: > > > If this is just a test machine, maybe you could test with these lines (at > > > about #880) > > > in sys/netinet/tcp_output.c commented out? (It looks to me like this will > > > disable TSO > > > for almost all the NFS writes.) > > > - around line #880 in sys/netinet/tcp_output.c: > > > /* > > > * In case there are too many small fragments > > > * don't use TSO: > > > */ > > > if (len <= max_len) { > > > len = max_len; > > > sendalot = 1; > > > tso = 0; > > > } > > > > > > This was added along with the other stuff that did the > > > if_hw_tsomaxsegcount, etc and I > > > never noticed it until now (not my patch). > > > > FYI: > > > > These lines are needed by other hardware, like the mlxen driver. If you > > remove them mlxen will start doing m_defrag(). I believe if you set the > > correct parameters in the "struct ifnet" for the TSO size/count limits > > this problem will go away. If you print the "len" and "max_len" and also > > the cases where TSO limits are reached, you'll see what parameter is > > triggering it and needs to be increased. > > > Well, if the driver isn't setting if_hw_tsomaxsegcount correctly, then it > is the driver that needs to be fixed. > Having the above code block disable TSO for all of the NFS writes, including > the ones that set if_hw_tsomaxsegcount correctly doesn't make sense to me. > If the driver authors don't set these, the drivers do lots of m_defrag() > calls. I have posted more than once to freebsd-net@ asking the driver authors > to set these and some now have. (I can't do it, because I don't have the > hardware to test it with.) > Thanks for reminder. I have generated a diff against HEAD. https://people.freebsd.org/~yongari/tso.param.diff The diff restores optimal TSO parameters which were lost in r271946 for drivers that relied on sane default values. I'll commit it after some testing. > I do think that most/all of them don't subtract 1 for the tcp/ip header and > I don't think they should be expected to, since the driver isn't supposed to > worry about the protocol at that level. I agree. > --> I think tcp_output() should subtract one from the if_hw_tsomaxsegcount > provided by the driver to handle this, since it chooses to count mbufs > (the while() loop at around line #825 in sys/netinet/tcp_output.c.) > before it prepends the tcp/ip header mbuf. > > rick > > > --HPS From owner-freebsd-stable@freebsd.org Wed Aug 19 07:42:25 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 019E09BDAF2; Wed, 19 Aug 2015 07:42:25 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-pa0-x22d.google.com (mail-pa0-x22d.google.com [IPv6:2607:f8b0:400e:c03::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C08CBE09; Wed, 19 Aug 2015 07:42:24 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: by paccq16 with SMTP id cq16so108134730pac.1; Wed, 19 Aug 2015 00:42:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=S0ZZmoyRQiijWzna4W8TD4lopx6A6yFnDneAQwWxdZ0=; b=c6MrsZlVkTXk65TXNgwoj0qpu1Vn8yZQi7Iwh5AuPlHY12Fvjt+lc3qdrCzcBX5Z75 V8HHCQDe4XVxqluirLR3sOsREh5CITl1IaCdCmr6dQrmQDxRqkt+y5OPBYZWReMLyjq+ 7vCUP14XLg5z7GzZvx1+uZFQg+PyTUEGWTh1zQ85zzAxlhUn2xKURMlH6xcrhu9ku+oe +OE4qqeTy1FjqtgPkNc8H8jie6D2affav8JZFODIwK80OWG3LIpdDI0FFmFWckv0yt+L WuebY3qhtVuxBbzn+PjEn3tOzO8+0LyWw/e1/l0TGCWmjAqoG7s/4YEuB4gI0GjDIrDn pSVQ== X-Received: by 10.69.0.166 with SMTP id az6mr21689907pbd.168.1439970144299; Wed, 19 Aug 2015 00:42:24 -0700 (PDT) Received: from pyunyh@gmail.com ([106.247.248.2]) by smtp.gmail.com with ESMTPSA id r1sm2419947pdm.31.2015.08.19.00.42.19 (version=TLSv1 cipher=RC4-SHA bits=128/128); Wed, 19 Aug 2015 00:42:23 -0700 (PDT) From: Yonghyeon PYUN X-Google-Original-From: "Yonghyeon PYUN" Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Wed, 19 Aug 2015 16:42:12 +0900 Date: Wed, 19 Aug 2015 16:42:12 +0900 To: Hans Petter Selasky Cc: Rick Macklem , Daniel Braniss , FreeBSD Net , Slawa Olhovchenkov , FreeBSD stable , Christopher Forgeron Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance Message-ID: <20150819074212.GB964@michelle.fasterthan.com> Reply-To: pyunyh@gmail.com References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <17871443-E105-4434-80B1-6939306A865F@cs.huji.ac.il> <473274181.23263108.1439814072514.JavaMail.zimbra@uoguelph.ca> <7F892C70-9C04-4468-9514-EDBFE75CF2C6@cs.huji.ac.il> <805850043.24018217.1439848150695.JavaMail.zimbra@uoguelph.ca> <9D8B0503-E8FA-43CA-88F0-01F184F84D9B@cs.huji.ac.il> <1721122651.24481798.1439902381663.JavaMail.zimbra@uoguelph.ca> <55D333D6.5040102@selasky.org> <1325951625.25292515.1439934848268.JavaMail.zimbra@uoguelph.ca> <55D429A4.3010407@selasky.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <55D429A4.3010407@selasky.org> User-Agent: Mutt/1.4.2.3i X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2015 07:42:25 -0000 On Wed, Aug 19, 2015 at 09:00:52AM +0200, Hans Petter Selasky wrote: > On 08/18/15 23:54, Rick Macklem wrote: > >Ouch! Yes, I now see that the code that counts the # of mbufs is before the > >code that adds the tcp/ip header mbuf. > > > >In my opinion, this should be fixed by setting if_hw_tsomaxsegcount to > >whatever > >the driver provides - 1. It is not the driver's responsibility to know if > >a tcp/ip > >header mbuf will be added and is a lot less confusing that expecting the > >driver > >author to know to subtract one. (I had mistakenly thought that > >tcp_output() had > >added the tc/ip header mbuf before the loop that counts mbufs in the list. > >Btw, > >this tcp/ip header mbuf also has leading space for the MAC layer header.) > > > > Hi Rick, > > Your question is good. With the Mellanox hardware we have separate > so-called inline data space for the TCP/IP headers, so if the TCP stack > subtracts something, then we would need to add something to the limit, > because then the scatter gather list is only used for the data part. > I think all drivers in tree don't subtract 1 for if_hw_tsomaxsegcount. Probably touching Mellanox driver would be simpler than fixing all other drivers in tree. > Maybe it can be controlled by some kind of flag, if all the three TSO > limits should include the TCP/IP/ethernet headers too. I'm pretty sure > we want both versions. > Hmm, I'm afraid it's already complex. Drivers have to tell almost the same information to both bus_dma(9) and network stack. From owner-freebsd-stable@freebsd.org Wed Aug 19 07:50:23 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 264AF9BDE74; Wed, 19 Aug 2015 07:50:23 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D8AEBA2C; Wed, 19 Aug 2015 07:50:22 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 995EE1FE023; Wed, 19 Aug 2015 09:50:19 +0200 (CEST) Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance To: pyunyh@gmail.com References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <17871443-E105-4434-80B1-6939306A865F@cs.huji.ac.il> <473274181.23263108.1439814072514.JavaMail.zimbra@uoguelph.ca> <7F892C70-9C04-4468-9514-EDBFE75CF2C6@cs.huji.ac.il> <805850043.24018217.1439848150695.JavaMail.zimbra@uoguelph.ca> <9D8B0503-E8FA-43CA-88F0-01F184F84D9B@cs.huji.ac.il> <1721122651.24481798.1439902381663.JavaMail.zimbra@uoguelph.ca> <55D333D6.5040102@selasky.org> <1325951625.25292515.1439934848268.JavaMail.zimbra@uoguelph.ca> <55D429A4.3010407@selasky.org> <20150819074212.GB964@michelle.fasterthan.com> Cc: Rick Macklem , FreeBSD stable , FreeBSD Net , Slawa Olhovchenkov , Christopher Forgeron , Daniel Braniss From: Hans Petter Selasky Message-ID: <55D43590.8050508@selasky.org> Date: Wed, 19 Aug 2015 09:51:44 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.1.0 MIME-Version: 1.0 In-Reply-To: <20150819074212.GB964@michelle.fasterthan.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2015 07:50:23 -0000 On 08/19/15 09:42, Yonghyeon PYUN wrote: > On Wed, Aug 19, 2015 at 09:00:52AM +0200, Hans Petter Selasky wrote: >> On 08/18/15 23:54, Rick Macklem wrote: >>> Ouch! Yes, I now see that the code that counts the # of mbufs is before the >>> code that adds the tcp/ip header mbuf. >>> >>> In my opinion, this should be fixed by setting if_hw_tsomaxsegcount to >>> whatever >>> the driver provides - 1. It is not the driver's responsibility to know if >>> a tcp/ip >>> header mbuf will be added and is a lot less confusing that expecting the >>> driver >>> author to know to subtract one. (I had mistakenly thought that >>> tcp_output() had >>> added the tc/ip header mbuf before the loop that counts mbufs in the list. >>> Btw, >>> this tcp/ip header mbuf also has leading space for the MAC layer header.) >>> >> >> Hi Rick, >> >> Your question is good. With the Mellanox hardware we have separate >> so-called inline data space for the TCP/IP headers, so if the TCP stack >> subtracts something, then we would need to add something to the limit, >> because then the scatter gather list is only used for the data part. >> > > I think all drivers in tree don't subtract 1 for > if_hw_tsomaxsegcount. Probably touching Mellanox driver would be > simpler than fixing all other drivers in tree. Hi, If you change the behaviour don't forget to update and/or add comments describing it. Maybe the amount of subtraction could be defined by some macro? Then drivers which inline the headers can subtract it? Your suggestion is fine by me. The initial TSO limits were tried to be preserved, and I believe that TSO limits never accounted for IP/TCP/ETHERNET/VLAN headers! > >> Maybe it can be controlled by some kind of flag, if all the three TSO >> limits should include the TCP/IP/ethernet headers too. I'm pretty sure >> we want both versions. >> > > Hmm, I'm afraid it's already complex. Drivers have to tell almost > the same information to both bus_dma(9) and network stack. You're right it's complicated. Not sure if bus_dma can provide an API for this though. --HPS From owner-freebsd-stable@freebsd.org Wed Aug 19 07:52:34 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EB7139BDF9E; Wed, 19 Aug 2015 07:52:34 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A967BF8E; Wed, 19 Aug 2015 07:52:34 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 6EE091FE023; Wed, 19 Aug 2015 09:52:32 +0200 (CEST) Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance To: pyunyh@gmail.com References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <17871443-E105-4434-80B1-6939306A865F@cs.huji.ac.il> <473274181.23263108.1439814072514.JavaMail.zimbra@uoguelph.ca> <7F892C70-9C04-4468-9514-EDBFE75CF2C6@cs.huji.ac.il> <805850043.24018217.1439848150695.JavaMail.zimbra@uoguelph.ca> <9D8B0503-E8FA-43CA-88F0-01F184F84D9B@cs.huji.ac.il> <1721122651.24481798.1439902381663.JavaMail.zimbra@uoguelph.ca> <55D333D6.5040102@selasky.org> <1325951625.25292515.1439934848268.JavaMail.zimbra@uoguelph.ca> <55D429A4.3010407@selasky.org> <20150819074212.GB964@michelle.fasterthan.com> Cc: Rick Macklem , FreeBSD stable , FreeBSD Net , Slawa Olhovchenkov , Christopher Forgeron , Daniel Braniss From: Hans Petter Selasky Message-ID: <55D43615.1030401@selasky.org> Date: Wed, 19 Aug 2015 09:53:57 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.1.0 MIME-Version: 1.0 In-Reply-To: <20150819074212.GB964@michelle.fasterthan.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2015 07:52:35 -0000 On 08/19/15 09:42, Yonghyeon PYUN wrote: > On Wed, Aug 19, 2015 at 09:00:52AM +0200, Hans Petter Selasky wrote: >> On 08/18/15 23:54, Rick Macklem wrote: >>> Ouch! Yes, I now see that the code that counts the # of mbufs is before the >>> code that adds the tcp/ip header mbuf. >>> >>> In my opinion, this should be fixed by setting if_hw_tsomaxsegcount to >>> whatever >>> the driver provides - 1. It is not the driver's responsibility to know if >>> a tcp/ip >>> header mbuf will be added and is a lot less confusing that expecting the >>> driver >>> author to know to subtract one. (I had mistakenly thought that >>> tcp_output() had >>> added the tc/ip header mbuf before the loop that counts mbufs in the list. >>> Btw, >>> this tcp/ip header mbuf also has leading space for the MAC layer header.) >>> >> >> Hi Rick, >> >> Your question is good. With the Mellanox hardware we have separate >> so-called inline data space for the TCP/IP headers, so if the TCP stack >> subtracts something, then we would need to add something to the limit, >> because then the scatter gather list is only used for the data part. >> > > I think all drivers in tree don't subtract 1 for > if_hw_tsomaxsegcount. Probably touching Mellanox driver would be > simpler than fixing all other drivers in tree. > >> Maybe it can be controlled by some kind of flag, if all the three TSO >> limits should include the TCP/IP/ethernet headers too. I'm pretty sure >> we want both versions. >> > > Hmm, I'm afraid it's already complex. Drivers have to tell almost > the same information to both bus_dma(9) and network stack. Don't forget that not all drivers in the tree set the TSO limits before if_attach(), so possibly the subtraction of one TSO fragment needs to go into ip_output() .... --HPS From owner-freebsd-stable@freebsd.org Wed Aug 19 08:13:21 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7A9E59BD96A; Wed, 19 Aug 2015 08:13:21 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-pa0-x22c.google.com (mail-pa0-x22c.google.com [IPv6:2607:f8b0:400e:c03::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 456BA24B; Wed, 19 Aug 2015 08:13:21 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: by pawq9 with SMTP id q9so55586183paw.3; Wed, 19 Aug 2015 01:13:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=CLkjp+SqMahrtwi6FLVeLyQrvmoXwkpvvLRb8VLmBnQ=; b=zRQlmWuUhdPEueQTUHWN6PlbAi3ALpkq1vpKv1dReKcTFrdBFEpwOHJe2MMWnO7hbD 5QRPq8nWkMVD90DbgF9CcIigUZwPK1/y1yV1cUWolyzJwT8BdrzrhC11CzxoDIxo9JC3 4L5KYMttH3TjFAFLmxenMGzRLse0u0ud9/RZqC8RuViyagib7OZqhi+4Vp7sGuHu3Cfg cE2AlxlyJaj7ZAgM3pgIR8Ny0SkB4gt5r3rrR0jiiWAcfW2T4FVNnsfpt6P8L7jaHq9u rH1f7Bg/5BFbiq4u9pBC9WxUs8QjU1tqQVTdvtAqcv2teKf2wm01IjlFod17o+4nrRRP O9dg== X-Received: by 10.68.250.98 with SMTP id zb2mr22368291pbc.40.1439972000644; Wed, 19 Aug 2015 01:13:20 -0700 (PDT) Received: from pyunyh@gmail.com ([106.247.248.2]) by smtp.gmail.com with ESMTPSA id d5sm20648024pdn.74.2015.08.19.01.13.15 (version=TLSv1 cipher=RC4-SHA bits=128/128); Wed, 19 Aug 2015 01:13:19 -0700 (PDT) From: Yonghyeon PYUN X-Google-Original-From: "Yonghyeon PYUN" Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Wed, 19 Aug 2015 17:13:08 +0900 Date: Wed, 19 Aug 2015 17:13:08 +0900 To: Hans Petter Selasky Cc: Rick Macklem , FreeBSD stable , FreeBSD Net , Slawa Olhovchenkov , Christopher Forgeron , Daniel Braniss Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance Message-ID: <20150819081308.GC964@michelle.fasterthan.com> Reply-To: pyunyh@gmail.com References: <473274181.23263108.1439814072514.JavaMail.zimbra@uoguelph.ca> <7F892C70-9C04-4468-9514-EDBFE75CF2C6@cs.huji.ac.il> <805850043.24018217.1439848150695.JavaMail.zimbra@uoguelph.ca> <9D8B0503-E8FA-43CA-88F0-01F184F84D9B@cs.huji.ac.il> <1721122651.24481798.1439902381663.JavaMail.zimbra@uoguelph.ca> <55D333D6.5040102@selasky.org> <1325951625.25292515.1439934848268.JavaMail.zimbra@uoguelph.ca> <55D429A4.3010407@selasky.org> <20150819074212.GB964@michelle.fasterthan.com> <55D43590.8050508@selasky.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <55D43590.8050508@selasky.org> User-Agent: Mutt/1.4.2.3i X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2015 08:13:21 -0000 On Wed, Aug 19, 2015 at 09:51:44AM +0200, Hans Petter Selasky wrote: > On 08/19/15 09:42, Yonghyeon PYUN wrote: > >On Wed, Aug 19, 2015 at 09:00:52AM +0200, Hans Petter Selasky wrote: > >>On 08/18/15 23:54, Rick Macklem wrote: > >>>Ouch! Yes, I now see that the code that counts the # of mbufs is before > >>>the > >>>code that adds the tcp/ip header mbuf. > >>> > >>>In my opinion, this should be fixed by setting if_hw_tsomaxsegcount to > >>>whatever > >>>the driver provides - 1. It is not the driver's responsibility to know if > >>>a tcp/ip > >>>header mbuf will be added and is a lot less confusing that expecting the > >>>driver > >>>author to know to subtract one. (I had mistakenly thought that > >>>tcp_output() had > >>>added the tc/ip header mbuf before the loop that counts mbufs in the > >>>list. > >>>Btw, > >>>this tcp/ip header mbuf also has leading space for the MAC layer header.) > >>> > >> > >>Hi Rick, > >> > >>Your question is good. With the Mellanox hardware we have separate > >>so-called inline data space for the TCP/IP headers, so if the TCP stack > >>subtracts something, then we would need to add something to the limit, > >>because then the scatter gather list is only used for the data part. > >> > > > >I think all drivers in tree don't subtract 1 for > >if_hw_tsomaxsegcount. Probably touching Mellanox driver would be > >simpler than fixing all other drivers in tree. > > Hi, > > If you change the behaviour don't forget to update and/or add comments > describing it. Maybe the amount of subtraction could be defined by some > macro? Then drivers which inline the headers can subtract it? > I'm also ok with your suggestion. > Your suggestion is fine by me. > > The initial TSO limits were tried to be preserved, and I believe that > TSO limits never accounted for IP/TCP/ETHERNET/VLAN headers! > I guess FreeBSD used to follow MS LSOv1 specification with minor exception in pseudo checksum computation. If I recall correctly the specification says upper stack can generate up to IP_MAXPACKET sized packet. Other L2 headers like ethernet/vlan header size is not included in the packet and it's drivers responsibility to allocate additional DMA buffers/segments for L2 headers. > > > >>Maybe it can be controlled by some kind of flag, if all the three TSO > >>limits should include the TCP/IP/ethernet headers too. I'm pretty sure > >>we want both versions. > >> > > > >Hmm, I'm afraid it's already complex. Drivers have to tell almost > >the same information to both bus_dma(9) and network stack. > > You're right it's complicated. Not sure if bus_dma can provide an API > for this though. > > --HPS From owner-freebsd-stable@freebsd.org Wed Aug 19 08:37:51 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 72BF89BD233 for ; Wed, 19 Aug 2015 08:37:51 +0000 (UTC) (envelope-from marko.cupac@mimar.rs) Received: from smtp.mimar.rs (smtp.mimar.rs [193.53.106.135]) by mx1.freebsd.org (Postfix) with ESMTP id 26735A6A for ; Wed, 19 Aug 2015 08:37:50 +0000 (UTC) (envelope-from marko.cupac@mimar.rs) Received: from vscan.mimar.rs (vscan.mimar.rs [193.53.106.134]) by smtp.mimar.rs (Postfix) with ESMTP id BBB7D89926 for ; Wed, 19 Aug 2015 10:37:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=mimar.rs; h= content-transfer-encoding:content-type:content-type:mime-version :x-mailer:organization:message-id:subject:subject:from:from:date :date:received:received; s=mimar-0901; t=1439973468; x= 1441787869; bh=Khm0nxqWhLlmEZr/c183gyEEqPr3awza0EMsgjolCHo=; b=j ydYX1K8Mq0ap3JtU7d4dMePfBh43pznLLQb4V0x6Nrtd7urrho1Hf+mzElNnBAzy sHhI+EZ89SOT2cvIK3vP2ZBNBi9zqd0TgFsoQYjoaWMf3MDCF5hvPZ8ctz/nEtjB Zi3V1qZfKlg9SyaMwXHToqzDsoCVj8XSeGrf7y5eiQ= X-Virus-Scanned: amavisd-new at mimar.rs Received: from smtp.mimar.rs ([193.53.106.135]) by vscan.mimar.rs (vscan.mimar.rs [193.53.106.134]) (amavisd-new, port 10026) with ESMTP id 4ERGjXi1aZ3I for ; Wed, 19 Aug 2015 10:37:48 +0200 (CEST) Received: from efreet (unknown [193.53.106.34]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: marko.cupac@mimar.rs) by smtp.mimar.rs (Postfix) with ESMTPSA id EA413898CD for ; Wed, 19 Aug 2015 10:37:47 +0200 (CEST) Date: Wed, 19 Aug 2015 10:37:47 +0200 From: Marko =?UTF-8?B?Q3VwYcSH?= To: FreeBSD stable Subject: best way to restart process Message-ID: <20150819103747.5c2ef288@efreet> Organization: mimar X-Mailer: Claws Mail 3.9.3 (GTK+ 2.24.23; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2015 08:37:51 -0000 Hi, I am trying to setup diskless client whose only purpose is logging of one user with permanent password into single RDP server via xfreerdp. I know how to setup autostart of xfreerdp, but I don't know how to restart if it exits for some reason (network problems etc.) What is the best way to check if process is running and restart it if it exits? Thank you in advance, --=20 Marko Cupa=C4=87 https://www.mimar.rs/ From owner-freebsd-stable@freebsd.org Wed Aug 19 09:05:09 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 31D3A9BDDAC for ; Wed, 19 Aug 2015 09:05:09 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E3B6E2F for ; Wed, 19 Aug 2015 09:05:08 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.84 (FreeBSD)) (envelope-from ) id 1ZRzIw-0004o0-AP; Wed, 19 Aug 2015 12:05:06 +0300 Date: Wed, 19 Aug 2015 12:05:06 +0300 From: Slawa Olhovchenkov To: Marko =?koi8-r?Q?Cupa'c?= Cc: FreeBSD stable Subject: Re: best way to restart process Message-ID: <20150819090506.GH3158@zxy.spb.ru> References: <20150819103747.5c2ef288@efreet> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <20150819103747.5c2ef288@efreet> User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2015 09:05:09 -0000 On Wed, Aug 19, 2015 at 10:37:47AM +0200, Marko Cupa'c wrote: > Hi, > > I am trying to setup diskless client whose only purpose is logging > of one user with permanent password into single RDP server via xfreerdp. > I know how to setup autostart of xfreerdp, but I don't know how to > restart if it exits for some reason (network problems etc.) > > What is the best way to check if process is running and restart it if > it exits? DAEMON(8) FreeBSD System Manager's Manual DAEMON(8) NAME daemon - run detached from the controlling terminal SYNOPSIS daemon [-cfr] [-p child_pidfile] [-P supervisor_pidfile] [-u user] command arguments ... DESCRIPTION [...] -r Supervise and restart the program if it has been terminated. From owner-freebsd-stable@freebsd.org Wed Aug 19 09:30:00 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C6DC69BC2B2 for ; Wed, 19 Aug 2015 09:30:00 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-la0-f54.google.com (mail-la0-f54.google.com [209.85.215.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3D8F7CDA for ; Wed, 19 Aug 2015 09:29:59 +0000 (UTC) (envelope-from ml@my.gd) Received: by lahi9 with SMTP id i9so115328969lah.2 for ; Wed, 19 Aug 2015 02:29:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=wlCzwRhfgXsM00mvRRXIEx4PpNUCoBxoIkCxz+5bL6c=; b=UDYWTqNiiFImKrELV7JZSKBCm4P/wNcdjTixNGO6duhoGG0UXHVcFixyu5sI2jlYnP nlrqERoXz24VUwFw9W2+/bQLIwXjubNUUT+O6oG58qUIFjKFhg4GrsA1zkP5kdVZ0T62 Eo/pOdKwfAtrhnKuZwUg702xu+5cKgtuOInD+Px1YmsDcSNJ7MmG4F9BJ2ZNXoPXI0Cb AzAGw3wkfkWhYWqkaq052NvvqYEgu/UcNt4hmSeWZv/ALOi9Caihe+/mhr1HtSNyfaz0 mzAefCMNoTJUlesTINalSgI40oVTUYaDYGKW/bhrJ19WIqLXYPnpaRhmSZjwwjcjjfX/ okVg== X-Gm-Message-State: ALoCoQnurfLmIq0QIPJE12G/F++GGl56K9j+KkBvsqrMY7B8ZFvWkHtRbzZat9Rxn65cxXxM4buB MIME-Version: 1.0 X-Received: by 10.112.151.178 with SMTP id ur18mr10523235lbb.59.1439976592389; Wed, 19 Aug 2015 02:29:52 -0700 (PDT) Received: by 10.112.60.34 with HTTP; Wed, 19 Aug 2015 02:29:52 -0700 (PDT) In-Reply-To: References: Date: Wed, 19 Aug 2015 11:29:52 +0200 Message-ID: Subject: Re: [POSSIBLE BUG] 10-STABLE CARP erroneously becomes master on boot From: Damien Fleuriot To: Freddie Cash Cc: FreeBSD Stable , Damien Fleuriot Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2015 09:30:00 -0000 Hello list, Freddie, I've been able to run extensive tests, the results of which I'm pasting below. First of all, I've been unable to replicate the problem in our preproduction and QA environments. The differences between our production and pre/QA envs are as follows : - we use link aggregation and VLAN tagging in production , we cannot replicate this in pre/QA because of limitations with KVM guests. - we use multiple CARP addresses with the same VHID on our public VLAN in production, we COULD replicate that in pre/QA if required. The context remains the following : - host A is supposed to be CARP MASTER, has advskew 20 and preempt - host B is supposed to be CARP BACKUP, has advskew 150 and preempt - host B assumes mastership if the CARPs are configured from rc.conf , doesn't if they're set up manually after boot Note that these 2 boxes were upgraded from 8-STABLE to 10-STABLE. Host A runs 10.2-BETA1 Host B runs 10.2-PRERELEASE I used the exact same versions in our pre/QA environments (one BETA1, one PRERELEASE from the same build) and couldn't replicate the issue. Now, on to the tests themselves. A/ Create a new CARP address with a new VHID, configure it in rc.conf and see if we get double MASTERS - on Host A : CARP created manually - on Host B : CARP created manually Host A is MASTER and Host B is BACKUP - on Host B : setup the new CARP in rc.conf , reboot Host A is MASTER and Host B is BACKUP , problem not replicated B/ Try with only one of our production addresses - on Host B : uncomment the production CARP address from rc.conf , reboot Host A is MASTER and Host B is MASTER Host A shows net.inet.carp.demotion=0 Host B shows net.inet.carp.demotion=240 C/ Try with the new CARP address + one of our production addresses , different VHIDs - on Host B : uncomment the new CARP address from rc.conf , reboot Host A is MASTER and Host B is MASTER Host A shows net.inet.carp.demotion=0 Host B shows net.inet.carp.demotion=240 D/ Try the new syntax in rc.conf , as per Freddie's suggestion - on Host B : change the rc.conf syntax , reboot Host A is MASTER and Host B is MASTER Host A shows net.inet.carp.demotion=0 Host B shows net.inet.carp.demotion=240 E/ Try, just for the sake of it, to remove old files and libs on host B - on Host B : cd /usr/src ; yes | make delete-old ; yes | make delete-old-libs ; reboot Host A is MASTER and Host B is MASTER Host A shows net.inet.carp.demotion=0 Host B shows net.inet.carp.demotion=240 F/ Edit sysctls to disable CARP demotion on advertisement send errors - on Host A : sysctl net.inet.carp.senderr_demotion_factor=0 - on Host B : set "net.inet.carp.senderr_demotion_factor=0" in sysctl.conf , reboot Host A is MASTER and Host B is MASTER Host A shows net.inet.carp.demotion=0 Host B shows net.inet.carp.demotion=240 Now after this F/ test, I'm thinking there's some voodoo going on here and it sure shows up : - on Host A, 'pfctl -si' shows 3k states - on Host B, 'pfctl -si' shows 800 states Now that would explain why my CARP gets demoted on Host B (as per man 4 carp, pfsync failures result in a -240 demotion). It doesn't, however, explain why the demoted CARP chooses to remain in a MASTER state, or assumed MASTERship in the first place. Surely enough, I can find some CARP errors in-between my reboots : messages:2420:2015-08-19T03:51:38.273600+00:00 pf1-gs kernel: carp: demoted by -240 to 0 (pfsync bulk fail) messages:2429:2015-08-19T03:56:37.178575+00:00 pf1-gs kernel: carp: VHID 110@vlan410: INIT -> BACKUP messages:2430:2015-08-19T03:56:40.664568+00:00 pf1-gs kernel: carp: VHID 110@vlan410: BACKUP -> MASTER (master down) messages:2637:2015-08-19T04:00:22.482071+00:00 pf1-gs kernel: carp: VHID 111@vlan410: BACKUP -> MASTER (master down) messages:2857:2015-08-19T04:04:02.330167+00:00 pf1-gs kernel: carp: VHID 110@vlan410: BACKUP -> MASTER (master down) messages:2877:2015-08-19T04:05:03.288199+00:00 pf1-gs kernel: carp: demoted by -240 to 0 (pfsync bulk fail) messages:3088:2015-08-19T04:08:48.961985+00:00 pf1-gs kernel: carp: VHID 110@vlan410: BACKUP -> MASTER (master down) Things I have not tested yet : - use /24 CARPs instead of /32s - switch my CARPs to all use different VHIDs Things I CANNOT test : - set up a *dedicated* pfsync link between the firewalls , they're in different DCs - set up a *dedicated* VLAN for pfsync , that would entail huge changes in our PCI-DSS environment After all these tests, I find myself in a situation where : - manually set up CARPs on Host B work fine , and pfsync works - CARPs from rc.conf on Host B result in MASTER-MASTER , and pfsync fails I must say I'm stuck here. The "master down" message is very confusing, when the firewalls *can* see each other. The "pfsync bulk fail" is rather interesting as well, since it doesn't occur when the CARPs are unconfigured, or set up manually without rc.conf. tcpdump -nei pflog0 does not show any dropped packet. PF is configured to "pass in quick" pfsync and CARP packets. I'm afraid a STFW hasn't helped overmuch here, although I'll try some more. If anyone's got a pointer, I'll bite. Cheers On 17 August 2015 at 18:38, Damien Fleuriot wrote: > > On 17 August 2015 at 18:32, Freddie Cash wrote: > >> >> On Aug 17, 2015 9:22 AM, "Damien Fleuriot" wrote: >> > >> > Hello list, >> > >> > >> > >> > I'm seeing this very peculiar behaviour between 2 10-STABLE boxes. >> > >> > Host A is CARP Master with advskew 20 and runs 10.2-BETA1 from 10/07 >> > Host B is CARP Backup with advskew 150 and runs 10.2-PRERELEASE from >> 12/08 >> > >> > >> > When I configure CARP in rc.conf on host B, it becomes Master on boot, >> and >> > host A remains Master as well. >> > When I force a state change on host B (ifconfig vlanx vhid y state >> backup), >> > it transitions to Backup then again to Master. >> > >> > When I comment out the CARP configuration in rc.conf , and configure >> CARP >> > manually on host B's interfaces after it boots, it correctly becomes and >> > remains Backup. >> > >> > >> > >> > Below is the excerpt from rc.conf pertaining to CARP configuration, the >> > only difference between the 2 hosts being their advskew. >> > >> > Host A >> > == BEGIN >> > >> > ifconfig_vlan410_alias0="vhid 110 pass passhere advskew 20 alias >> > 10.104.10.251/32" >> > >> > == END >> > >> > Host B >> > == BEGIN >> > >> > ifconfig_vlan410_alias0="vhid 110 pass passhere advskew 150 alias >> > 10.104.10.251/32" >> > >> > == END >> >> Put the IP first, and the vhid stuff last in rc.conf for things to work >> the most reliably. And drop the extra alias. >> >> ifconfig_vlan410_alias0="inet 10.104.10.251/32 vhid 110 pass passhere >> advskew 150" >> >> CARP requires that all IPs on an interface that are part of the same vhid >> to be listed (added) in the exact same order for the vhid to be considered >> "the same". That one trips me up all the time when manually adding an IP to >> a CARP pair, and then later rebooting one box as they both think they're >> master for that interface, while being a mix of master/backup for the other >> interfaces. >> >> Cheers, >> Freddie >> (running CARP on 2 10-CURRENT boxes and 2 10.1-p13 boxes) >> > > Cheers Freddie, will try and keep the thread up to date on the results. > > From owner-freebsd-stable@freebsd.org Wed Aug 19 10:56:00 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B63B39BC0B0 for ; Wed, 19 Aug 2015 10:56:00 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 493B6113B for ; Wed, 19 Aug 2015 10:55:59 +0000 (UTC) (envelope-from ml@my.gd) Received: by lbbtg9 with SMTP id tg9so573010lbb.1 for ; Wed, 19 Aug 2015 03:55:51 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=3uGx8vNDwFHT58NrXKkYIAn4bLrrHEIECKi31hZIFlg=; b=BNkoL+4AL/vpzsTeKYGaAhCLhZ1f0dfNRQXYXoHA6E3FtqLYfgIQjTZ9bJAg7SpbPL fXtvOOkJG17zBJgAnzAc+l2PgL1bgaduOCdICk7Jm+voyp8Bdvf6XLI3+LrJlc8mh3eH qbFKJIfe/2zr+iXpEQm2rD2Ivo8nMBnblhOxby0uk1zX8hw48PscSS4rwYF23CXI34J2 e/S3+On6Qwn7fAOn7+2eiIOEguhsv+A/V6KsRZ4BwzQ+tRxGnKfqRAMEGbdLYSoLf13t bnrUO5VaGClDdP5hUYGe17GPqaAVkVTZgCW1SQ1YF9mY9jPi5a7m+0+VcLtVSLPZiif8 hFhg== X-Gm-Message-State: ALoCoQmGBS/q99vG4ufF84yg9shioppIbPTU2LcqzR6mfo4CmGZrQeyv+GczQAtSAyuxb874yBDd MIME-Version: 1.0 X-Received: by 10.152.170.130 with SMTP id am2mr10754864lac.54.1439981446596; Wed, 19 Aug 2015 03:50:46 -0700 (PDT) Received: by 10.112.60.34 with HTTP; Wed, 19 Aug 2015 03:50:46 -0700 (PDT) Date: Wed, 19 Aug 2015 12:50:46 +0200 Message-ID: Subject: [patch] [ports] net/relayd config check in rc.d From: Damien Fleuriot To: "freebsd-stable@freebsd.org" Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2015 10:56:00 -0000 Hello list, Does anyone want to take a look at https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D200914 ? This is a patch I submitted for net/relayd 's rc.d script so that it performs a configuration check before start/reload/restart, =C3=A0 la nginx= . The ticket's been assigned to mm@ but it looks like he's not had the time to have a go at it. Note that the "patch" to the script was done on 8-STABLE but the script in 10-STABLE is identical. We've tested this in production and have yet to find issue with it. Cheers From owner-freebsd-stable@freebsd.org Wed Aug 19 12:08:39 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F1A769BDE24 for ; Wed, 19 Aug 2015 12:08:39 +0000 (UTC) (envelope-from estartu@ze.tum.de) Received: from io.ze.tum.de (w3projmail.ze.tum.de [129.187.39.2]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5E3AC30B for ; Wed, 19 Aug 2015 12:08:38 +0000 (UTC) (envelope-from estartu@ze.tum.de) Received: from etustar.ze.tum.de (etustar.ze.tum.de [IPv6:2001:4ca0:2e03::16:1]) (authenticated bits=0) by io.ze.tum.de (8.14.5/8.14.5) with ESMTP id t7JC7vKE083869 for ; Wed, 19 Aug 2015 14:07:58 +0200 (CEST) (envelope-from estartu@ze.tum.de) To: FreeBSD stable From: Gerhard Schmidt Subject: Installing FreeBSD on MacPro X-Enigmail-Draft-Status: N1110 Message-ID: <55D4719D.60709@ze.tum.de> Date: Wed, 19 Aug 2015 14:07:57 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.1.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2015 12:08:40 -0000 Hi, I'm trying to install FreeBSD 10.2 on a MacPro. Installing and Booting works. But a few seconds after login and accessing the Harddisk, I'm getting the following errors. ahcich0: Timeout on slot 13 port 0 ahcich0: is 00000008 cs 00000000 ss 00000000 rs ffffffff tfd 40 serr 00000000 cmd 0000cd17 (ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 00 28 15 40 40 2f 00 00 01 00 00 (ada0:ahcich0:0:0:0): CAM status: Command timeout (ada0:ahcich0:0:0:0): Retrying command ahcich0: Timeout on slot 15 port 0 ahcich0: is 00000008 cs 00000000 ss 00000000 rs ffffffff tfd 40 serr 00000000 cmd 0000cf17 (ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 a0 5f bd 40 13 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: Command timeout (ada0:ahcich0:0:0:0): Retrying command ahcich0: Timeout on slot 2 port 0 ahcich0: is 00000008 cs 00000000 ss 00000000 rs ffffffff tfd 40 serr 00000000 cmd 0000c217 (ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 c0 23 bd 40 13 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: Command timeout (ada0:ahcich0:0:0:0): Retrying command Any Ideas what causes this and what can be done to fix it. the dmesg of the MacPro is the following Copyright (c) 1992-2015 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 10.2-RELEASE #0 r286666: Wed Aug 12 15:26:37 UTC 2015 root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 VT: running with driver "efifb". CPU: Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz (2700.06-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306e4 Family=0x6 Model=0x3e Stepping=4 Features=0xbfebfbff Features2=0x7fbee3ff AMD Features=0x2c100800 AMD Features2=0x1 Structured Extended Features=0x281 XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics real memory = 68719476736 (65536 MB) avail memory = 66655334400 (63567 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 24 CPUs FreeBSD/SMP: 1 package(s) x 12 core(s) x 2 SMT threads cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP): APIC ID: 3 cpu4 (AP): APIC ID: 4 cpu5 (AP): APIC ID: 5 cpu6 (AP): APIC ID: 6 cpu7 (AP): APIC ID: 7 cpu8 (AP): APIC ID: 8 cpu9 (AP): APIC ID: 9 cpu10 (AP): APIC ID: 10 cpu11 (AP): APIC ID: 11 cpu12 (AP): APIC ID: 16 cpu13 (AP): APIC ID: 17 cpu14 (AP): APIC ID: 18 cpu15 (AP): APIC ID: 19 cpu16 (AP): APIC ID: 20 cpu17 (AP): APIC ID: 21 cpu18 (AP): APIC ID: 22 cpu19 (AP): APIC ID: 23 cpu20 (AP): APIC ID: 24 cpu21 (AP): APIC ID: 25 cpu22 (AP): APIC ID: 26 cpu23 (AP): APIC ID: 27 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard lapic0: Forcing LINT1 to edge trigger random: initialized module_register_init: MOD_LOAD (vesa, 0xffffffff80db8eb0, 0) error 19 kbd0 at kbdmux0 acpi0: on motherboard acpi0: Power Button (fixed) unknown: I/O range not supported Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 350 Event timer "HPET1" frequency 14318180 Hz quality 340 Event timer "HPET2" frequency 14318180 Hz quality 340 Event timer "HPET3" frequency 14318180 Hz quality 340 Event timer "HPET4" frequency 14318180 Hz quality 340 Event timer "HPET5" frequency 14318180 Hz quality 340 Event timer "HPET6" frequency 14318180 Hz quality 340 Event timer "HPET7" frequency 14318180 Hz quality 340 cpu0: on acpi0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 cpu4: on acpi0 cpu5: on acpi0 cpu6: on acpi0 cpu7: on acpi0 cpu8: on acpi0 cpu9: on acpi0 cpu10: on acpi0 cpu11: on acpi0 cpu12: on acpi0 cpu13: on acpi0 cpu14: on acpi0 cpu15: on acpi0 cpu16: on acpi0 cpu17: on acpi0 cpu18: on acpi0 cpu19: on acpi0 cpu20: on acpi0 cpu21: on acpi0 cpu22: on acpi0 cpu23: on acpi0 atrtc0: port 0x70-0x77 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 acpi_ec0: port 0x62,0x66 on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: at device 1.0 on pci0 pci16: on pcib1 pcib2: mem 0xbf900000-0xbf93ffff at device 0.0 on pci16 pci17: on pcib2 pcib3: at device 1.0 on pci17 pci18: on pcib3 xhci0: mem 0xa0900000-0xa090ffff,0xa0910000-0xa0910fff,0xa0911000-0xa0911fff at device 0.0 on pci18 xhci0: 32 bytes context size, 64-bit DMA usbus0 on xhci0 pcib4: at device 2.0 on pci17 pci20: on pcib4 pcib5: at device 8.0 on pci17 pci19: on pcib5 pcib6: at device 9.0 on pci17 pci91: on pcib6 pcib7: at device 10.0 on pci17 pci162: on pcib7 pcib8: at device 2.0 on pci0 pci2: on pcib8 vgapci0: port 0x3000-0x30ff mem 0x80000000-0x8fffffff,0xa0700000-0xa073ffff at device 0.0 on pci2 hdac0: mem 0xa0760000-0xa0763fff at device 0.1 on pci2 hdac0: hdac_get_capabilities: Invalid corb size (0) device_attach: hdac0 attach returned 6 pcib9: at device 3.0 on pci0 pci6: on pcib9 vgapci1: port 0x2000-0x20ff mem 0x90000000-0x9fffffff,0xa0600000-0xa063ffff at device 0.0 on pci6 hdac0: mem 0xa0660000-0xa0663fff at device 0.1 on pci6 hdac0: hdac_get_capabilities: Invalid corb size (0) device_attach: hdac0 attach returned 6 pcib10: at device 17.0 on pci0 pci10: on pcib10 pci0: at device 22.0 (no driver attached) hdac0: mem 0xa0820000-0xa0823fff at device 27.0 on pci0 pcib11: at device 28.0 on pci0 pci11: on pcib11 bge0: mem 0xa0300000-0xa030ffff,0xa0310000-0xa031ffff at device 0.0 on pci11 bge0: CHIP ID 0x57766000; ASIC REV 0x57766; CHIP REV 0x577660; PCI-E miibus0: on bge0 brgphy0: PHY 1 on miibus0 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow bge0: Using defaults for TSO: 65518/35/2048 bge0: Ethernet address: 00:3e:e1:c2:1a:e3 pcib12: at device 28.1 on pci0 pci12: on pcib12 bge1: mem 0xa0400000-0xa040ffff,0xa0410000-0xa041ffff at device 0.0 on pci12 bge1: CHIP ID 0x57766000; ASIC REV 0x57766; CHIP REV 0x577660; PCI-E miibus1: on bge1 brgphy1: PHY 1 on miibus1 brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow bge1: Using defaults for TSO: 65518/35/2048 bge1: Ethernet address: 00:3e:e1:c2:1a:e2 pcib13: at device 28.2 on pci0 pci13: on pcib13 pci13: at device 0.0 (no driver attached) pcib14: at device 28.4 on pci0 pci14: on pcib14 ahci0: mem 0xa0500000-0xa0501fff at device 0.0 on pci14 ahci0: AHCI v1.30 with 1 6Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ehci0: mem 0xa0827800-0xa0827bff at device 29.0 on pci0 usbus1: EHCI version 1.0 usbus1 on ehci0 pcib15: at device 30.0 on pci0 pci15: on pcib15 isab0: at device 31.0 on pci0 isa0: on isab0 acpi_button0: on acpi0 acpi_button1: on acpi0 ppc0: cannot reserve I/O port range uart0: <8250 or 16450 or compatible> at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0 est0: on cpu0 est1: on cpu1 est2: on cpu2 est3: on cpu3 est4: on cpu4 est5: on cpu5 est6: on cpu6 est7: on cpu7 est8: on cpu8 est9: on cpu9 est10: on cpu10 est11: on cpu11 est12: on cpu12 est13: on cpu13 est14: on cpu14 est15: on cpu15 est16: on cpu16 est17: on cpu17 est18: on cpu18 est19: on cpu19 est20: on cpu20 est21: on cpu21 est22: on cpu22 est23: on cpu23 random: unblocking device. usbus0: 5.0Gbps Super Speed USB v3.0 Timecounters tick every 1.000 msec hdacc0: at cad 0 on hdac0 hdaa0: at nid 1 on hdacc0 pcm0: at nid 18 and 24 on hdaa0 pcm1: at nid 16 on hdaa0 pcm2: at nid 19 on hdaa0 pcm3: at nid 33 on hdaa0 usbus1: 480Mbps High Speed USB v2.0 ugen0.1: <0x1b73> at usbus0 uhub0: <0x1b73 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0 ugen1.1: at usbus1 uhub1: on usbus1 ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 ada0: ATA8-ACS SATA 3.x device ada0: Serial Number S1FVNYAF903719 ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 954204MB (1954210120 512 byte sectors: 16H 63S/T 16383C) ada0: Previously was known as ad4 lapic1: Forcing LINT1 to edge trigger SMP: AP CPU #1 Launched! lapic6: Forcing LINT1 to edge trigger SMP: AP CPU #6 Launched! lapic7: Forcing LINT1 to edge trigger SMP: AP CPU #7 Launched! lapic3: Forcing LINT1 to edge trigger SMP: AP CPU #3 Launched! lapic19: Forcing LINT1 to edge trigger SMP: AP CPU #15 Launched! lapic22: Forcing LINT1 to edge trigger SMP: AP CPU #18 Launched! lapic26: Forcing LINT1 to edge trigger SMP: AP CPU #22 Launched! lapic23: Forcing LINT1 to edge trigger SMP: AP CPU #19 Launched! lapic9: Forcing LINT1 to edge trigger SMP: AP CPU #9 Launched! lapic27: Forcing LINT1 to edge trigger SMP: AP CPU #23 Launched! lapic24: Forcing LINT1 to edge trigger SMP: AP CPU #20 Launched! lapic25: Forcing LINT1 to edge trigger SMP: AP CPU #21 Launched! lapic10: Forcing LINT1 to edge trigger SMP: AP CPU #10 Launched! lapic4: Forcing LINT1 to edge trigger SMP: AP CPU #4 Launched! lapic5: Forcing LINT1 to edge trigger SMP: AP CPU #5 Launched! lapic18: Forcing LINT1 to edge trigger SMP: AP CPU #14 Launched! lapic8: Forcing LINT1 to edge trigger SMP: AP CPU #8 Launched! lapic11: Forcing LINT1 to edge trigger SMP: AP CPU #11 Launched! lapic17: Forcing LINT1 to edge trigger SMP: AP CPU #13 Launched! lapic2: Forcing LINT1 to edge trigger SMP: AP CPU #2 Launched! lapic16: Forcing LINT1 to edge trigger SMP: AP CPU #12 Launched! lapic21: Forcing LINT1 to edge trigger SMP: AP CPU #17 Launched! lapic20: Forcing LINT1 to edge trigger SMP: AP CPU #16 Launched! Timecounter "TSC-low" frequency 1350029784 Hz quality 1000 Root mount waiting for: usbus1 usbus0 uhub0: 8 ports with 8 removable, self powered uhub1: 2 ports with 2 removable, self powered ugen0.2: at usbus0 Root mount waiting for: usbus1 usbus0 ugen1.2: at usbus1 uhub2: on usbus1 ugen0.3: at usbus0 uhub3: on usbus0 uhub3: 1 port with 1 removable, bus powered Root mount waiting for: usbus1 usbus0 ugen0.4: at usbus0 uhub4: on usbus0 uhub4: MTT enabled uhub2: 8 ports with 7 removable, self powered uhub4: 3 ports with 2 removable, self powered Root mount waiting for: usbus1 usbus0 ugen1.3: at usbus1 uhub5: on usbus1 ugen0.5: at usbus0 hid_get_item: Number of items truncated to 255 uhub5: 3 ports with 0 removable, self powered Root mount waiting for: usbus1 usbus0 ugen1.4: at usbus1 ukbd0: on usbus1 kbd1 at ukbd0 ugen0.6: at usbus0 uhub6: on usbus0 uhub6: 3 ports with 2 removable, bus powered ugen1.5: at usbus1 Root mount waiting for: usbus1 usbus0 ugen0.7: at usbus0 ukbd1: on usbus0 kbd2 at ukbd1 ugen1.6: at usbus1 Trying to mount root from ufs:/dev/ada0p2 [rw]... ums0: on usbus0 ums1: on usbus1 ums1: 3 buttons and [XY] coordinates ID=2 ums0: 3 buttons and [XYZ] coordinates ID=0 hid_get_item: Number of items truncated to 255 hid_get_item: Number of items truncated to 255 uhid0: on usbus0 hid_get_item: Number of items truncated to 255 hid_get_item: Number of items truncated to 255 hid_get_item: Number of items truncated to 255 uhid1: on usbus0 ubt0: on usbus1 WARNING: attempt to domain_add(bluetooth) after domainfinalize() WARNING: attempt to domain_add(netgraph) after domainfinalize() ng_hci_send_command: ubt0hci - could not send HCI command, error=12 ng_hci_process_command_timeout: ubt0hci - unable to complete HCI command OGF=0x3, OCF=0x3. Timeout Regards Estartu -- From owner-freebsd-stable@freebsd.org Wed Aug 19 12:14:02 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B4A109BC02B; Wed, 19 Aug 2015 12:14:02 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 2C9B0979; Wed, 19 Aug 2015 12:14:01 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:GkC7YBd9aSdqXt8Roqe2oc6zlGMj4u6mDksu8pMizoh2WeGdxc6/Yx7h7PlgxGXEQZ/co6odzbGG6Oa8Bydcvt6oizMrTt9lb1c9k8IYnggtUoauKHbQC7rUVRE8B9lIT1R//nu2YgB/Ecf6YEDO8DXptWZBUiv2OQc9HOnpAIma153xjLDpvcGNKFkXzBOGIppMbzyO5T3LsccXhYYwYo0Q8TDu5kVyRuJN2GlzLkiSlRuvru25/Zpk7jgC86l5r50IeezAcq85Vb1VCig9eyBwvZWz9EqLcQza/moBVHQWuhVNCgnBqhr9W8TfqCz/49B80yrSGMT9TrQ5XHz29aJiQxzshSIvKjk27WzTksw2h6sN80HpnAB234OBONLdD/F5ZK6IOIpCHWc= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2AnAgA0cdRV/61jaINdg29pBoMfuiQBCYFtCoUxSgKBehQBAQEBAQEBAYEJgh2CBwEBBAEBASArIAsQAgEIGAICDRkCAicBCSYCDAcEARwEiA0NuUuWGwEBAQEBAQEDAQEBAQEZBIEiijGEMQEGAQEcNAeCaYFDBZUjhQSFB4QskDeESINmAiaCDhyBbyIzB34BCBcjgQQBAQE X-IronPort-AV: E=Sophos;i="5.15,709,1432612800"; d="scan'208";a="233342498" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-annu.net.uoguelph.ca with ESMTP; 19 Aug 2015 08:14:00 -0400 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 75F1715F574; Wed, 19 Aug 2015 08:14:00 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 9c6wl2qMDweJ; Wed, 19 Aug 2015 08:13:59 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id A102F15F577; Wed, 19 Aug 2015 08:13:59 -0400 (EDT) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id p_9FsjkfQATs; Wed, 19 Aug 2015 08:13:59 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 7EE9215F574; Wed, 19 Aug 2015 08:13:59 -0400 (EDT) Date: Wed, 19 Aug 2015 08:13:59 -0400 (EDT) From: Rick Macklem To: pyunyh@gmail.com Cc: Hans Petter Selasky , FreeBSD stable , FreeBSD Net , Slawa Olhovchenkov , Christopher Forgeron , Daniel Braniss Message-ID: <1154739904.25677089.1439986439408.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <20150819081308.GC964@michelle.fasterthan.com> References: <473274181.23263108.1439814072514.JavaMail.zimbra@uoguelph.ca> <1721122651.24481798.1439902381663.JavaMail.zimbra@uoguelph.ca> <55D333D6.5040102@selasky.org> <1325951625.25292515.1439934848268.JavaMail.zimbra@uoguelph.ca> <55D429A4.3010407@selasky.org> <20150819074212.GB964@michelle.fasterthan.com> <55D43590.8050508@selasky.org> <20150819081308.GC964@michelle.fasterthan.com> Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.11] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF34 (Win)/8.0.9_GA_6191) Thread-Topic: ix(intel) vs mlxen(mellanox) 10Gb performance Thread-Index: wv7zo8RPDc9ayJBYkipemnefFyp/cg== X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2015 12:14:02 -0000 Yonghyeon PYUN wrote: > On Wed, Aug 19, 2015 at 09:51:44AM +0200, Hans Petter Selasky wrote: > > On 08/19/15 09:42, Yonghyeon PYUN wrote: > > >On Wed, Aug 19, 2015 at 09:00:52AM +0200, Hans Petter Selasky wrote: > > >>On 08/18/15 23:54, Rick Macklem wrote: > > >>>Ouch! Yes, I now see that the code that counts the # of mbufs is before > > >>>the > > >>>code that adds the tcp/ip header mbuf. > > >>> > > >>>In my opinion, this should be fixed by setting if_hw_tsomaxsegcount to > > >>>whatever > > >>>the driver provides - 1. It is not the driver's responsibility to know > > >>>if > > >>>a tcp/ip > > >>>header mbuf will be added and is a lot less confusing that expecting the > > >>>driver > > >>>author to know to subtract one. (I had mistakenly thought that > > >>>tcp_output() had > > >>>added the tc/ip header mbuf before the loop that counts mbufs in the > > >>>list. > > >>>Btw, > > >>>this tcp/ip header mbuf also has leading space for the MAC layer > > >>>header.) > > >>> > > >> > > >>Hi Rick, > > >> > > >>Your question is good. With the Mellanox hardware we have separate > > >>so-called inline data space for the TCP/IP headers, so if the TCP stack > > >>subtracts something, then we would need to add something to the limit, > > >>because then the scatter gather list is only used for the data part. > > >> > > > > > >I think all drivers in tree don't subtract 1 for > > >if_hw_tsomaxsegcount. Probably touching Mellanox driver would be > > >simpler than fixing all other drivers in tree. > > > > Hi, > > > > If you change the behaviour don't forget to update and/or add comments > > describing it. Maybe the amount of subtraction could be defined by some > > macro? Then drivers which inline the headers can subtract it? > > > > I'm also ok with your suggestion. > > > Your suggestion is fine by me. > > > > > The initial TSO limits were tried to be preserved, and I believe that > > TSO limits never accounted for IP/TCP/ETHERNET/VLAN headers! > > > > I guess FreeBSD used to follow MS LSOv1 specification with minor > exception in pseudo checksum computation. If I recall correctly the > specification says upper stack can generate up to IP_MAXPACKET sized > packet. Other L2 headers like ethernet/vlan header size is not > included in the packet and it's drivers responsibility to allocate > additional DMA buffers/segments for L2 headers. > Yep. The default for if_hw_tsomax was reduced from IP_MAXPACKET to 32 * MCLBYTES - max_ethernet_header_size as a workaround/hack so that devices limited to 32 transmit segments would work (ie. the entire packet, including MAC header would fit in 32 MCLBYTE clusters). This implied that many drivers did end up using m_defrag() to copy the mbuf list to one made up of 32 MCLBYTE clusters. If a driver sets if_hw_tsomaxsegcount correctly, then it can set if_hw_tsomax to whatever it can handle as the largest TSO packet (without MAC header) the hardware can handle. If it can handle > IP_MAXPACKET, then it can set it to that. rick > > > > > >>Maybe it can be controlled by some kind of flag, if all the three TSO > > >>limits should include the TCP/IP/ethernet headers too. I'm pretty sure > > >>we want both versions. > > >> > > > > > >Hmm, I'm afraid it's already complex. Drivers have to tell almost > > >the same information to both bus_dma(9) and network stack. > > > > You're right it's complicated. Not sure if bus_dma can provide an API > > for this though. > > > > --HPS > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-stable@freebsd.org Wed Aug 19 12:22:39 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A83D19BC36F; Wed, 19 Aug 2015 12:22:39 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 0B1A410A9; Wed, 19 Aug 2015 12:22:38 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:Vxk/qxbJTY/CfjipoofHFNv/LSx+4OfEezUN459isYplN5qZpcuybnLW6fgltlLVR4KTs6sC0LqN9fy+EjBfqb+681k8M7V0HycfjssXmwFySOWkMmbcaMDQUiohAc5ZX0Vk9XzoeWJcGcL5ekGA6ibqtW1aJBzzOEJPK/jvHcaK1oLsh7v0p8OYP1oArQH+SI0xBS3+lR/WuMgSjNkqAYcK4TyNnEF1ff9Lz3hjP1OZkkW0zM6x+Jl+73YY4Kp5pIZoGJ/3dKUgTLFeEC9ucyVsvJWq5lH/Sl6v730HGl0bjgZFGUD+4RXzRZTg+n/6rvFVwySeNNb1XPYzQzv0vIlxTxq9siYMNHYc+WrUjsF1xPZBpRuqpBhyxqbJZ46IOf5mfuXWdIVJFiJ6Qs9NWnkZUcuHZIwVAr9EZL4Aog== X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2AnAgCWc9RV/61jaINdg29pBoMfuiQBCYFtCoUxSgKBehQBAQEBAQEBAYEJgh2CBwEBBAEBASArIAsQAgEIGAICDRkCAicBCSYCBAgHBAEcBIgNDblVlhsBAQEBAQEBAQEBAQEBAQEBARcEgSKKMYQyBgEBHDQHgmmBQwWVI4UEhQcxg3sVlGqDZgImhBkiMwd/CBcjgQQBAQE X-IronPort-AV: E=Sophos;i="5.15,709,1432612800"; d="scan'208";a="231624929" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 19 Aug 2015 08:22:37 -0400 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 1DFE315F574; Wed, 19 Aug 2015 08:22:37 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id oGHvcUGKoy3Z; Wed, 19 Aug 2015 08:22:36 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 76F4315F577; Wed, 19 Aug 2015 08:22:36 -0400 (EDT) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id aicDNu_ZdPFR; Wed, 19 Aug 2015 08:22:36 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 5630215F574; Wed, 19 Aug 2015 08:22:36 -0400 (EDT) Date: Wed, 19 Aug 2015 08:22:36 -0400 (EDT) From: Rick Macklem To: Hans Petter Selasky Cc: pyunyh@gmail.com, FreeBSD stable , FreeBSD Net , Slawa Olhovchenkov , Christopher Forgeron Message-ID: <160577762.25683294.1439986956328.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <55D43615.1030401@selasky.org> References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <9D8B0503-E8FA-43CA-88F0-01F184F84D9B@cs.huji.ac.il> <1721122651.24481798.1439902381663.JavaMail.zimbra@uoguelph.ca> <55D333D6.5040102@selasky.org> <1325951625.25292515.1439934848268.JavaMail.zimbra@uoguelph.ca> <55D429A4.3010407@selasky.org> <20150819074212.GB964@michelle.fasterthan.com> <55D43615.1030401@selasky.org> Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.10] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF34 (Win)/8.0.9_GA_6191) Thread-Topic: ix(intel) vs mlxen(mellanox) 10Gb performance Thread-Index: 3tLtiaT//pF56x9z31ShK3tEdfsYmQ== X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2015 12:22:39 -0000 Hans Petter Selasky wrote: > On 08/19/15 09:42, Yonghyeon PYUN wrote: > > On Wed, Aug 19, 2015 at 09:00:52AM +0200, Hans Petter Selasky wrote: > >> On 08/18/15 23:54, Rick Macklem wrote: > >>> Ouch! Yes, I now see that the code that counts the # of mbufs is before > >>> the > >>> code that adds the tcp/ip header mbuf. > >>> > >>> In my opinion, this should be fixed by setting if_hw_tsomaxsegcount to > >>> whatever > >>> the driver provides - 1. It is not the driver's responsibility to know if > >>> a tcp/ip > >>> header mbuf will be added and is a lot less confusing that expecting the > >>> driver > >>> author to know to subtract one. (I had mistakenly thought that > >>> tcp_output() had > >>> added the tc/ip header mbuf before the loop that counts mbufs in the > >>> list. > >>> Btw, > >>> this tcp/ip header mbuf also has leading space for the MAC layer header.) > >>> > >> > >> Hi Rick, > >> > >> Your question is good. With the Mellanox hardware we have separate > >> so-called inline data space for the TCP/IP headers, so if the TCP stack > >> subtracts something, then we would need to add something to the limit, > >> because then the scatter gather list is only used for the data part. > >> > > > > I think all drivers in tree don't subtract 1 for > > if_hw_tsomaxsegcount. Probably touching Mellanox driver would be > > simpler than fixing all other drivers in tree. > > > >> Maybe it can be controlled by some kind of flag, if all the three TSO > >> limits should include the TCP/IP/ethernet headers too. I'm pretty sure > >> we want both versions. > >> > > > > Hmm, I'm afraid it's already complex. Drivers have to tell almost > > the same information to both bus_dma(9) and network stack. > > Don't forget that not all drivers in the tree set the TSO limits before > if_attach(), so possibly the subtraction of one TSO fragment needs to go > into ip_output() .... > I think setting them before a call to ether_ifattach() should be required and any driver that doesn't do that needs to be fixed. Also, I notice that "32 * MCLBYTES - (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN)" is getting written as "65536 - (ETHER_HDR_LEN + ETHER_VLAN_ENCAP_LEN)" which obscures the reason it is the default. It probably isn't the correct default for any driver that sets if_hw_tsomaxsegcount, but is close to IP_MAXPACKET, so the breakage is mostly theoretical. rick > --HPS > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > From owner-freebsd-stable@freebsd.org Wed Aug 19 12:24:14 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D7B559BC47D for ; Wed, 19 Aug 2015 12:24:14 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [78.47.246.247]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 6B6CA1378 for ; Wed, 19 Aug 2015 12:24:13 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221]) by hz.grosbein.net (8.14.9/8.14.9) with ESMTP id t7JCH8md004419 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 19 Aug 2015 14:17:08 +0200 (CEST) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: freebsd-stable@freebsd.org Received: from eg.sd.rdtc.ru (eugen@localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.9/8.14.9) with ESMTP id t7JCH6WL036172; Wed, 19 Aug 2015 19:17:07 +0700 (KRAT) (envelope-from eugen@grosbein.net) Message-ID: <55D473C2.6090604@grosbein.net> Date: Wed, 19 Aug 2015 19:17:06 +0700 From: Eugene Grosbein User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: =?UTF-8?B?TWFya28gQ3VwYcSH?= , FreeBSD Stable Subject: Re: ping from web application References: <20150818150924.5e9bef04@efreet> In-Reply-To: <20150818150924.5e9bef04@efreet> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_00,LOCAL_FROM autolearn=no version=3.3.2 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hz.grosbein.net X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2015 12:24:14 -0000 On 18.08.2015 20:09, Marko Cupać wrote: > Hi, > > I use web applicaton (net-mgmt/phpipam) which should have the ability > to check hosts' availability via ping. I can even specify path to ping > executable. > > This functionality does not work on FreeBSD by default, and suggested > workaround is to set setuid bit on /sbin/ping. /sbin/ping DOES have setuid on FreeBSD by default, so phpipam fails for some other reason, e.g. wrong path to ping executable or wrong (linux-like?) arguments. From owner-freebsd-stable@freebsd.org Wed Aug 19 12:26:22 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 92BD49BC52D; Wed, 19 Aug 2015 12:26:22 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 113DC162B; Wed, 19 Aug 2015 12:26:21 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:62rayR/Tb+yobf9uRHKM819IXTAuvvDOBiVQ1KB91OIcTK2v8tzYMVDF4r011RmSDd6dt6wP17WempujcFJDyK7JiGoFfp1IWk1NouQttCtkPvS4D1bmJuXhdS0wEZcKflZk+3amLRodQ56mNBXsq3G/pQQfBg/4fVIsYL+lQciO0Y/riKibwN76XUZhvHKFe7R8LRG7/036l/I9ps9cEJs30QbDuXBSeu5blitCLFOXmAvgtI/rpMYwuwwZgf8q9tZBXKPmZOx4COUAVHV1e1wyse3iswKLdQaT+nYGGl4blhNTABmNuBHiRb/qvy/zrelsni6AMpulY6ozXGGY7qxoADrhgyQDOjtxpHvSg8dziK9eiA+mqAFyx5bUJoqcYqktNpjBdM8XEDISFv1aUDZMV8blN9MC X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2AnAgCWc9RV/61jaINdg29pBoMfuiQBCYFtCoUxSgKBehQBAQEBAQEBAYEJgh2CBwEBBAEBASArIAsQAgEIGAICDQQVAgInAQkmAgQIBwQBHASIDQ25VZYbAQEBAQEBAQEBAQEBAQEBAQEXBIEiijGEMgYBARw0BwqCX4FDBZUjhQSFB4QslH+DZgImhBkiMwd/CBcjgQQBAQE X-IronPort-AV: E=Sophos;i="5.15,709,1432612800"; d="scan'208";a="231625582" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 19 Aug 2015 08:26:20 -0400 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id AB0E615F574; Wed, 19 Aug 2015 08:26:20 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id zmNABxJOE1Hh; Wed, 19 Aug 2015 08:26:20 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id EF4AC15F578; Wed, 19 Aug 2015 08:26:19 -0400 (EDT) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id A6CIpa0jZlVv; Wed, 19 Aug 2015 08:26:19 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id D0CD715F574; Wed, 19 Aug 2015 08:26:19 -0400 (EDT) Date: Wed, 19 Aug 2015 08:26:19 -0400 (EDT) From: Rick Macklem To: Hans Petter Selasky Cc: pyunyh@gmail.com, FreeBSD stable , FreeBSD Net , Slawa Olhovchenkov , Christopher Forgeron , Daniel Braniss Message-ID: <901585223.25686295.1439987179835.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <55D43615.1030401@selasky.org> References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <9D8B0503-E8FA-43CA-88F0-01F184F84D9B@cs.huji.ac.il> <1721122651.24481798.1439902381663.JavaMail.zimbra@uoguelph.ca> <55D333D6.5040102@selasky.org> <1325951625.25292515.1439934848268.JavaMail.zimbra@uoguelph.ca> <55D429A4.3010407@selasky.org> <20150819074212.GB964@michelle.fasterthan.com> <55D43615.1030401@selasky.org> Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.10] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF34 (Win)/8.0.9_GA_6191) Thread-Topic: ix(intel) vs mlxen(mellanox) 10Gb performance Thread-Index: b7KE9Og530PFxSAdUMOGyYtExjLo2w== X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2015 12:26:22 -0000 Hans Petter Selasky wrote: > On 08/19/15 09:42, Yonghyeon PYUN wrote: > > On Wed, Aug 19, 2015 at 09:00:52AM +0200, Hans Petter Selasky wrote: > >> On 08/18/15 23:54, Rick Macklem wrote: > >>> Ouch! Yes, I now see that the code that counts the # of mbufs is before > >>> the > >>> code that adds the tcp/ip header mbuf. > >>> > >>> In my opinion, this should be fixed by setting if_hw_tsomaxsegcount to > >>> whatever > >>> the driver provides - 1. It is not the driver's responsibility to know if > >>> a tcp/ip > >>> header mbuf will be added and is a lot less confusing that expecting the > >>> driver > >>> author to know to subtract one. (I had mistakenly thought that > >>> tcp_output() had > >>> added the tc/ip header mbuf before the loop that counts mbufs in the > >>> list. > >>> Btw, > >>> this tcp/ip header mbuf also has leading space for the MAC layer header.) > >>> > >> > >> Hi Rick, > >> > >> Your question is good. With the Mellanox hardware we have separate > >> so-called inline data space for the TCP/IP headers, so if the TCP stack > >> subtracts something, then we would need to add something to the limit, > >> because then the scatter gather list is only used for the data part. > >> > > > > I think all drivers in tree don't subtract 1 for > > if_hw_tsomaxsegcount. Probably touching Mellanox driver would be > > simpler than fixing all other drivers in tree. > > > >> Maybe it can be controlled by some kind of flag, if all the three TSO > >> limits should include the TCP/IP/ethernet headers too. I'm pretty sure > >> we want both versions. > >> > > > > Hmm, I'm afraid it's already complex. Drivers have to tell almost > > the same information to both bus_dma(9) and network stack. > > Don't forget that not all drivers in the tree set the TSO limits before > if_attach(), so possibly the subtraction of one TSO fragment needs to go > into ip_output() .... > I don't really care where it gets subtracted, so long as it is subtracted at least by default, so all the drivers that don't subtract it get fixed. However, I might argue that tcp_output() is the correct place, since tcp_output() is where the tcp/ip header mbuf is prepended to the list. The subtraction is just taking into account the mbuf that tcp_output() will be adding to the head of the list and it should count that in the "while()" loop. rick > --HPS > > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-stable@freebsd.org Wed Aug 19 13:00:39 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 493949BCC46; Wed, 19 Aug 2015 13:00:39 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id B555480E; Wed, 19 Aug 2015 13:00:38 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:DMCj8hzucrlBwOjXCy+O+j09IxM/srCxBDY+r6Qd0e0eIJqq85mqBkHD//Il1AaPBtWAra4awLeO+4nbGkU+or+5+EgYd5JNUxJXwe43pCcHRPC/NEvgMfTxZDY7FskRHHVs/nW8LFQHUJ2mPw6anHS+4HYoFwnlMkItf6KuStWU05r8irj60qaQSjsLrQL1Wal1IhSyoFeZnegtqqwmFJwMzADUqGBDYeVcyDAgD1uSmxHh+pX4p8Y7oGx48sgs/M9YUKj8Y79wDfkBVGxnYCgJ45jLvB/YBTOC+mcRSC0tnx5BGAvUpEX6RozZqSb+v/F+yW+dJ8KgHp4uXjH31aZgS1fNgSwEMzM8uDXNj8V7j6ZWpTq8oBNizorMYMeePawtLevmYdoGSD8ZDY5qXCtbD9b5NtNXAg== X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2AnAgAsfdRV/61jaINdg29pBoMfuiQBCYFtCoUxSgKBehQBAQEBAQEBAYEJgh2CBwEBBAEBASAEJyALEAIBCBgCAg0ZAgInAQkmAgQIBwQBGgIEiA0NuX6WHgEBAQEBAQEBAQEBAQEBAQEBFwSBIooxhDIGAQEcNAeCaYFDBZUjhQSFB4Qsh0aIcYRIg2YCJoIOHIFvIjMHfwgXI4EEAQEB X-IronPort-AV: E=Sophos;i="5.15,709,1432612800"; d="scan'208";a="231630411" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 19 Aug 2015 09:00:37 -0400 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id EC41A15F563; Wed, 19 Aug 2015 09:00:36 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id XSfv8pzv90CU; Wed, 19 Aug 2015 09:00:36 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 0C67A15F56D; Wed, 19 Aug 2015 09:00:36 -0400 (EDT) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id HfBl68YG1Ywd; Wed, 19 Aug 2015 09:00:35 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id CC6C415F563; Wed, 19 Aug 2015 09:00:35 -0400 (EDT) Date: Wed, 19 Aug 2015 09:00:35 -0400 (EDT) From: Rick Macklem To: Hans Petter Selasky Cc: pyunyh@gmail.com, FreeBSD stable , FreeBSD Net , Slawa Olhovchenkov , Christopher Forgeron , Daniel Braniss Message-ID: <2013503980.25726607.1439989235806.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <55D43615.1030401@selasky.org> References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <9D8B0503-E8FA-43CA-88F0-01F184F84D9B@cs.huji.ac.il> <1721122651.24481798.1439902381663.JavaMail.zimbra@uoguelph.ca> <55D333D6.5040102@selasky.org> <1325951625.25292515.1439934848268.JavaMail.zimbra@uoguelph.ca> <55D429A4.3010407@selasky.org> <20150819074212.GB964@michelle.fasterthan.com> <55D43615.1030401@selasky.org> Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.11] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF34 (Win)/8.0.9_GA_6191) Thread-Topic: ix(intel) vs mlxen(mellanox) 10Gb performance Thread-Index: W+G5Djot61DfsWhTAw5fqJmL10IusQ== X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2015 13:00:39 -0000 Hans Petter Selasky wrote: > On 08/19/15 09:42, Yonghyeon PYUN wrote: > > On Wed, Aug 19, 2015 at 09:00:52AM +0200, Hans Petter Selasky wrote: > >> On 08/18/15 23:54, Rick Macklem wrote: > >>> Ouch! Yes, I now see that the code that counts the # of mbufs is before > >>> the > >>> code that adds the tcp/ip header mbuf. > >>> > >>> In my opinion, this should be fixed by setting if_hw_tsomaxsegcount to > >>> whatever > >>> the driver provides - 1. It is not the driver's responsibility to know if > >>> a tcp/ip > >>> header mbuf will be added and is a lot less confusing that expecting the > >>> driver > >>> author to know to subtract one. (I had mistakenly thought that > >>> tcp_output() had > >>> added the tc/ip header mbuf before the loop that counts mbufs in the > >>> list. > >>> Btw, > >>> this tcp/ip header mbuf also has leading space for the MAC layer header.) > >>> > >> > >> Hi Rick, > >> > >> Your question is good. With the Mellanox hardware we have separate > >> so-called inline data space for the TCP/IP headers, so if the TCP stack > >> subtracts something, then we would need to add something to the limit, > >> because then the scatter gather list is only used for the data part. > >> > > > > I think all drivers in tree don't subtract 1 for > > if_hw_tsomaxsegcount. Probably touching Mellanox driver would be > > simpler than fixing all other drivers in tree. > > > >> Maybe it can be controlled by some kind of flag, if all the three TSO > >> limits should include the TCP/IP/ethernet headers too. I'm pretty sure > >> we want both versions. > >> > > > > Hmm, I'm afraid it's already complex. Drivers have to tell almost > > the same information to both bus_dma(9) and network stack. > > Don't forget that not all drivers in the tree set the TSO limits before > if_attach(), so possibly the subtraction of one TSO fragment needs to go > into ip_output() .... > Ok, I realized that some drivers may not know the answers before ether_ifattach(), due to the way they are configured/written (I saw the use of if_hw_tsomax_update() in the patch). If it is subtracted as a part of the assignment to if_hw_tsomaxsegcount in tcp_output() at line#791 in tcp_output() like the following, I don't think it should matter if the values are set before ether_ifattach()? /* * Subtract 1 for the tcp/ip header mbuf that * will be prepended to the mbuf chain in this * function in the code below this block. */ if_hw_tsomaxsegcount = tp->t_tsomaxsegcount - 1; I don't have a good solution for the case where a driver doesn't plan on using the tcp/ip header provided by tcp_output() except to say the driver can add one to the setting to compensate for that (and if they fail to do so, it still works, although somewhat suboptimally). When I now read the comment in sys/net/if_var.h it is clear what it means, but for some reason I didn't read it that way before? (I think it was the part that said the driver didn't have to subtract for the headers that confused me?) In any case, we need to try and come up with a clear definition of what they need to be set to. I can now think of two ways to deal with this: 1 - Leave tcp_output() as is, but provide a macro for the device driver authors to use that sets if_hw_tsomaxsegcount with a flag for "driver uses tcp/ip header mbuf", documenting that this flag should normally be true. OR 2 - Change tcp_output() as above, noting that this is a workaround for confusion w.r.t. whether or not if_hw_tsomaxsegcount should include the tcp/ip header mbuf and update the comment in if_var.h to reflect this. Then drivers that don't use the tcp/ip header mbuf can increase their value for if_hw_tsomaxsegcount by 1. (The comment should also mention that a value of 35 or greater is much preferred to 32 if the hardware will support that.) Also, I'd like to apologize for some of my emails getting a little "blunt". I just find it flustrating that this problem is still showing up and is even in 10.2. This is partly my fault for not making it clearer to driver authors what if_hw_tsomaxsegcount should be set to, because I had it incorrect. Hopefully we can come up with a solution that everyone is comfortable with, rick > --HPS > > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-stable@freebsd.org Wed Aug 19 13:19:19 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C60B19BE0B8 for ; Wed, 19 Aug 2015 13:19:19 +0000 (UTC) (envelope-from dot.yet@gmail.com) Received: from mail-ob0-x241.google.com (mail-ob0-x241.google.com [IPv6:2607:f8b0:4003:c01::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8E4D3113B for ; Wed, 19 Aug 2015 13:19:19 +0000 (UTC) (envelope-from dot.yet@gmail.com) Received: by obbry9 with SMTP id ry9so284143obb.1 for ; Wed, 19 Aug 2015 06:19:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:cc :content-type; bh=ufSWh/uoaisokxjhpzMU1VDDSo9izVelX2JoMrQ1+bw=; b=fD3eaVd54g7WbvnmvRpn+4lm8GiIq8A15f9QEMCVBoEOTJlk+4jlULzKCWJAdNb7md /JizuEX1iO++tNJfilCppKL1AhEsdtfETqfmgr6uTlE6tjxoiuugFLY/PJC6q6vN06rA uP8GUiITt+ieMIjRajct67gJBFMEtaRuqhobndde6Zl8t+kATwAh6hyKx7/Gb5M42rRo SGYo79V86iFMeBUIJnRxE9j3Mt+x4S76LonMCCHDPj0NfXYC5A6rO8G6bnt9l2XytCPB q/AHkofwfAKTCk6ZlHnhfpky/XiWumRbkuG+yl1rFXbq7q1QwlsDA6/6v9UV1qv7Nbv7 5CPw== MIME-Version: 1.0 X-Received: by 10.60.69.200 with SMTP id g8mr10434027oeu.40.1439990358662; Wed, 19 Aug 2015 06:19:18 -0700 (PDT) Received: by 10.76.20.105 with HTTP; Wed, 19 Aug 2015 06:19:18 -0700 (PDT) In-Reply-To: <55D4719D.60709@ze.tum.de> References: <55D4719D.60709@ze.tum.de> Date: Wed, 19 Aug 2015 09:19:18 -0400 Message-ID: Subject: Re: Installing FreeBSD on MacPro From: Dot Yet Cc: FreeBSD stable Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2015 13:19:20 -0000 Seems to be related to be related to your SSD drive or the SATA control. do you a secondary SSD or HDD to try on this machine? I've seen this happen on my Intel SSD in the past. Thx . On Wed, Aug 19, 2015 at 8:07 AM, Gerhard Schmidt wrote: > Hi, > > I'm trying to install FreeBSD 10.2 on a MacPro. > > Installing and Booting works. But a few seconds after login and accessing > the Harddisk, I'm getting the following errors. > ahcich0: Timeout on slot 13 port 0 > ahcich0: is 00000008 cs 00000000 ss 00000000 rs ffffffff tfd 40 serr > 00000000 cmd 0000cd17 > (ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 00 28 15 40 40 2f 00 00 > 01 00 00 > (ada0:ahcich0:0:0:0): CAM status: Command timeout > (ada0:ahcich0:0:0:0): Retrying command > ahcich0: Timeout on slot 15 port 0 > ahcich0: is 00000008 cs 00000000 ss 00000000 rs ffffffff tfd 40 serr > 00000000 cmd 0000cf17 > (ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 a0 5f bd 40 13 00 00 > 00 00 00 > (ada0:ahcich0:0:0:0): CAM status: Command timeout > (ada0:ahcich0:0:0:0): Retrying command > ahcich0: Timeout on slot 2 port 0 > ahcich0: is 00000008 cs 00000000 ss 00000000 rs ffffffff tfd 40 serr > 00000000 cmd 0000c217 > (ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 c0 23 bd 40 13 00 00 > 00 00 00 > (ada0:ahcich0:0:0:0): CAM status: Command timeout > (ada0:ahcich0:0:0:0): Retrying command > > Any Ideas what causes this and what can be done to fix it. > > the dmesg of the MacPro is the following > > Copyright (c) 1992-2015 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > The Regents of the University of California. All rights reserved. > FreeBSD is a registered trademark of The FreeBSD Foundation. > FreeBSD 10.2-RELEASE #0 r286666: Wed Aug 12 15:26:37 UTC 2015 > root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 > FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 > VT: running with driver "efifb". > CPU: Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz (2700.06-MHz K8-class CPU) > Origin="GenuineIntel" Id=0x306e4 Family=0x6 Model=0x3e Stepping=4 > > > Features=0xbfebfbff > > > Features2=0x7fbee3ff > AMD Features=0x2c100800 > AMD Features2=0x1 > Structured Extended Features=0x281 > XSAVE Features=0x1 > VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr > TSC: P-state invariant, performance statistics > real memory = 68719476736 (65536 MB) > avail memory = 66655334400 (63567 MB) > Event timer "LAPIC" quality 600 > ACPI APIC Table: > FreeBSD/SMP: Multiprocessor System Detected: 24 CPUs > FreeBSD/SMP: 1 package(s) x 12 core(s) x 2 SMT threads > cpu0 (BSP): APIC ID: 0 > cpu1 (AP): APIC ID: 1 > cpu2 (AP): APIC ID: 2 > cpu3 (AP): APIC ID: 3 > cpu4 (AP): APIC ID: 4 > cpu5 (AP): APIC ID: 5 > cpu6 (AP): APIC ID: 6 > cpu7 (AP): APIC ID: 7 > cpu8 (AP): APIC ID: 8 > cpu9 (AP): APIC ID: 9 > cpu10 (AP): APIC ID: 10 > cpu11 (AP): APIC ID: 11 > cpu12 (AP): APIC ID: 16 > cpu13 (AP): APIC ID: 17 > cpu14 (AP): APIC ID: 18 > cpu15 (AP): APIC ID: 19 > cpu16 (AP): APIC ID: 20 > cpu17 (AP): APIC ID: 21 > cpu18 (AP): APIC ID: 22 > cpu19 (AP): APIC ID: 23 > cpu20 (AP): APIC ID: 24 > cpu21 (AP): APIC ID: 25 > cpu22 (AP): APIC ID: 26 > cpu23 (AP): APIC ID: 27 > ioapic0 irqs 0-23 on motherboard > ioapic1 irqs 24-47 on motherboard > lapic0: Forcing LINT1 to edge trigger > random: initialized > module_register_init: MOD_LOAD (vesa, 0xffffffff80db8eb0, 0) error 19 > kbd0 at kbdmux0 > acpi0: on motherboard > acpi0: Power Button (fixed) > unknown: I/O range not supported > Timecounter "HPET" frequency 14318180 Hz quality 950 > Event timer "HPET" frequency 14318180 Hz quality 350 > Event timer "HPET1" frequency 14318180 Hz quality 340 > Event timer "HPET2" frequency 14318180 Hz quality 340 > Event timer "HPET3" frequency 14318180 Hz quality 340 > Event timer "HPET4" frequency 14318180 Hz quality 340 > Event timer "HPET5" frequency 14318180 Hz quality 340 > Event timer "HPET6" frequency 14318180 Hz quality 340 > Event timer "HPET7" frequency 14318180 Hz quality 340 > cpu0: on acpi0 > cpu1: on acpi0 > cpu2: on acpi0 > cpu3: on acpi0 > cpu4: on acpi0 > cpu5: on acpi0 > cpu6: on acpi0 > cpu7: on acpi0 > cpu8: on acpi0 > cpu9: on acpi0 > cpu10: on acpi0 > cpu11: on acpi0 > cpu12: on acpi0 > cpu13: on acpi0 > cpu14: on acpi0 > cpu15: on acpi0 > cpu16: on acpi0 > cpu17: on acpi0 > cpu18: on acpi0 > cpu19: on acpi0 > cpu20: on acpi0 > cpu21: on acpi0 > cpu22: on acpi0 > cpu23: on acpi0 > atrtc0: port 0x70-0x77 on acpi0 > Event timer "RTC" frequency 32768 Hz quality 0 > attimer0: port 0x40-0x43,0x50-0x53 on acpi0 > Timecounter "i8254" frequency 1193182 Hz quality 0 > Event timer "i8254" frequency 1193182 Hz quality 100 > Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 > acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 > acpi_ec0: port 0x62,0x66 on acpi0 > pcib0: port 0xcf8-0xcff on acpi0 > pci0: on pcib0 > pcib1: at device 1.0 on pci0 > pci16: on pcib1 > pcib2: mem 0xbf900000-0xbf93ffff at device 0.0 on > pci16 > pci17: on pcib2 > pcib3: at device 1.0 on pci17 > pci18: on pcib3 > xhci0: mem > 0xa0900000-0xa090ffff,0xa0910000-0xa0910fff,0xa0911000-0xa0911fff at > device 0.0 on pci18 > xhci0: 32 bytes context size, 64-bit DMA > usbus0 on xhci0 > pcib4: at device 2.0 on pci17 > pci20: on pcib4 > pcib5: at device 8.0 on pci17 > pci19: on pcib5 > pcib6: at device 9.0 on pci17 > pci91: on pcib6 > pcib7: at device 10.0 on pci17 > pci162: on pcib7 > pcib8: at device 2.0 on pci0 > pci2: on pcib8 > vgapci0: port 0x3000-0x30ff mem > 0x80000000-0x8fffffff,0xa0700000-0xa073ffff at device 0.0 on pci2 > hdac0: mem 0xa0760000-0xa0763fff at device 0.1 > on pci2 > hdac0: hdac_get_capabilities: Invalid corb size (0) > device_attach: hdac0 attach returned 6 > pcib9: at device 3.0 on pci0 > pci6: on pcib9 > vgapci1: port 0x2000-0x20ff mem > 0x90000000-0x9fffffff,0xa0600000-0xa063ffff at device 0.0 on pci6 > hdac0: mem 0xa0660000-0xa0663fff at device 0.1 > on pci6 > hdac0: hdac_get_capabilities: Invalid corb size (0) > device_attach: hdac0 attach returned 6 > pcib10: at device 17.0 on pci0 > pci10: on pcib10 > pci0: at device 22.0 (no driver attached) > hdac0: mem 0xa0820000-0xa0823fff at device > 27.0 on pci0 > pcib11: at device 28.0 on pci0 > pci11: on pcib11 > bge0: 0x57766000> mem > 0xa0300000-0xa030ffff,0xa0310000-0xa031ffff at device 0.0 on pci11 > bge0: CHIP ID 0x57766000; ASIC REV 0x57766; CHIP REV 0x577660; PCI-E > miibus0: on bge0 > brgphy0: PHY 1 on miibus0 > brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, > 1000baseT-master, 1000baseT-FDX, > 1000baseT-FDX-master, auto, auto-flow > bge0: Using defaults for TSO: 65518/35/2048 > bge0: Ethernet address: 00:3e:e1:c2:1a:e3 > pcib12: at device 28.1 on pci0 > pci12: on pcib12 > bge1: 0x57766000> mem > 0xa0400000-0xa040ffff,0xa0410000-0xa041ffff at device 0.0 on pci12 > bge1: CHIP ID 0x57766000; ASIC REV 0x57766; CHIP REV 0x577660; PCI-E > miibus1: on bge1 > brgphy1: PHY 1 on miibus1 > brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, > 1000baseT-master, 1000baseT-FDX, > 1000baseT-FDX-master, auto, auto-flow > bge1: Using defaults for TSO: 65518/35/2048 > bge1: Ethernet address: 00:3e:e1:c2:1a:e2 > pcib13: at device 28.2 on pci0 > pci13: on pcib13 > pci13: at device 0.0 (no driver attached) > pcib14: at device 28.4 on pci0 > pci14: on pcib14 > ahci0: mem 0xa0500000-0xa0501fff at device 0.0 on > pci14 > ahci0: AHCI v1.30 with 1 6Gbps ports, Port Multiplier not supported > ahcich0: at channel 0 on ahci0 > ehci0: mem 0xa0827800-0xa0827bff at > device 29.0 on pci0 > usbus1: EHCI version 1.0 > usbus1 on ehci0 > pcib15: at device 30.0 on pci0 > pci15: on pcib15 > isab0: at device 31.0 on pci0 > isa0: on isab0 > acpi_button0: on acpi0 > acpi_button1: on acpi0 > ppc0: cannot reserve I/O port range > uart0: <8250 or 16450 or compatible> at port 0x3f8-0x3ff irq 4 flags 0x10 > on isa0 > est0: on cpu0 > est1: on cpu1 > est2: on cpu2 > est3: on cpu3 > est4: on cpu4 > est5: on cpu5 > est6: on cpu6 > est7: on cpu7 > est8: on cpu8 > est9: on cpu9 > est10: on cpu10 > est11: on cpu11 > est12: on cpu12 > est13: on cpu13 > est14: on cpu14 > est15: on cpu15 > est16: on cpu16 > est17: on cpu17 > est18: on cpu18 > est19: on cpu19 > est20: on cpu20 > est21: on cpu21 > est22: on cpu22 > est23: on cpu23 > random: unblocking device. > usbus0: 5.0Gbps Super Speed USB v3.0 > Timecounters tick every 1.000 msec > hdacc0: at cad 0 on hdac0 > hdaa0: at nid 1 on hdacc0 > pcm0: at nid 18 and 24 on hdaa0 > pcm1: at nid 16 on hdaa0 > pcm2: at nid 19 on hdaa0 > pcm3: at nid 33 on hdaa0 > usbus1: 480Mbps High Speed USB v2.0 > ugen0.1: <0x1b73> at usbus0 > uhub0: <0x1b73 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0 > ugen1.1: at usbus1 > uhub1: on usbus1 > ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 > ada0: ATA8-ACS SATA 3.x device > ada0: Serial Number S1FVNYAF903719 > ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) > ada0: Command Queueing enabled > ada0: 954204MB (1954210120 512 byte sectors: 16H 63S/T 16383C) > ada0: Previously was known as ad4 > lapic1: Forcing LINT1 to edge trigger > SMP: AP CPU #1 Launched! > lapic6: Forcing LINT1 to edge trigger > SMP: AP CPU #6 Launched! > lapic7: Forcing LINT1 to edge trigger > SMP: AP CPU #7 Launched! > lapic3: Forcing LINT1 to edge trigger > SMP: AP CPU #3 Launched! > lapic19: Forcing LINT1 to edge trigger > SMP: AP CPU #15 Launched! > lapic22: Forcing LINT1 to edge trigger > SMP: AP CPU #18 Launched! > lapic26: Forcing LINT1 to edge trigger > SMP: AP CPU #22 Launched! > lapic23: Forcing LINT1 to edge trigger > SMP: AP CPU #19 Launched! > lapic9: Forcing LINT1 to edge trigger > SMP: AP CPU #9 Launched! > lapic27: Forcing LINT1 to edge trigger > SMP: AP CPU #23 Launched! > lapic24: Forcing LINT1 to edge trigger > SMP: AP CPU #20 Launched! > lapic25: Forcing LINT1 to edge trigger > SMP: AP CPU #21 Launched! > lapic10: Forcing LINT1 to edge trigger > SMP: AP CPU #10 Launched! > lapic4: Forcing LINT1 to edge trigger > SMP: AP CPU #4 Launched! > lapic5: Forcing LINT1 to edge trigger > SMP: AP CPU #5 Launched! > lapic18: Forcing LINT1 to edge trigger > SMP: AP CPU #14 Launched! > lapic8: Forcing LINT1 to edge trigger > SMP: AP CPU #8 Launched! > lapic11: Forcing LINT1 to edge trigger > SMP: AP CPU #11 Launched! > lapic17: Forcing LINT1 to edge trigger > SMP: AP CPU #13 Launched! > lapic2: Forcing LINT1 to edge trigger > SMP: AP CPU #2 Launched! > lapic16: Forcing LINT1 to edge trigger > SMP: AP CPU #12 Launched! > lapic21: Forcing LINT1 to edge trigger > SMP: AP CPU #17 Launched! > lapic20: Forcing LINT1 to edge trigger > SMP: AP CPU #16 Launched! > Timecounter "TSC-low" frequency 1350029784 Hz quality 1000 > Root mount waiting for: usbus1 usbus0 > uhub0: 8 ports with 8 removable, self powered > uhub1: 2 ports with 2 removable, self powered > ugen0.2: at usbus0 > Root mount waiting for: usbus1 usbus0 > ugen1.2: at usbus1 > uhub2: on > usbus1 > ugen0.3: at usbus0 > uhub3: on > usbus0 > uhub3: 1 port with 1 removable, bus powered > Root mount waiting for: usbus1 usbus0 > ugen0.4: at usbus0 > uhub4: on > usbus0 > uhub4: MTT enabled > uhub2: 8 ports with 7 removable, self powered > uhub4: 3 ports with 2 removable, self powered > Root mount waiting for: usbus1 usbus0 > ugen1.3: at usbus1 > uhub5: on > usbus1 > ugen0.5: at usbus0 > hid_get_item: Number of items truncated to 255 > uhub5: 3 ports with 0 removable, self powered > Root mount waiting for: usbus1 usbus0 > ugen1.4: at usbus1 > ukbd0: on > usbus1 > kbd1 at ukbd0 > ugen0.6: at usbus0 > uhub6: on > usbus0 > uhub6: 3 ports with 2 removable, bus powered > ugen1.5: at usbus1 > Root mount waiting for: usbus1 usbus0 > ugen0.7: at usbus0 > ukbd1: on > usbus0 > kbd2 at ukbd1 > ugen1.6: at usbus1 > Trying to mount root from ufs:/dev/ada0p2 [rw]... > ums0: on > usbus0 > ums1: on > usbus1 > ums1: 3 buttons and [XY] coordinates ID=2 > ums0: 3 buttons and [XYZ] coordinates ID=0 > hid_get_item: Number of items truncated to 255 > hid_get_item: Number of items truncated to 255 > uhid0: on > usbus0 > hid_get_item: Number of items truncated to 255 > hid_get_item: Number of items truncated to 255 > hid_get_item: Number of items truncated to 255 > uhid1: on > usbus0 > ubt0: on > usbus1 > WARNING: attempt to domain_add(bluetooth) after domainfinalize() > WARNING: attempt to domain_add(netgraph) after domainfinalize() > ng_hci_send_command: ubt0hci - could not send HCI command, error=12 > ng_hci_process_command_timeout: ubt0hci - unable to complete HCI command > OGF=0x3, OCF=0x3. Timeout > > Regards > Estartu > > -- > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > From owner-freebsd-stable@freebsd.org Wed Aug 19 13:20:26 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5C75C9BE12B; Wed, 19 Aug 2015 13:20:26 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.116.210]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A1BB3127F; Wed, 19 Aug 2015 13:20:25 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from chamsa.cs.huji.ac.il ([132.65.80.19]) by kabab.cs.huji.ac.il with esmtp id 1ZS3Hs-000F95-7t; Wed, 19 Aug 2015 16:20:16 +0300 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance From: Daniel Braniss In-Reply-To: <2013503980.25726607.1439989235806.JavaMail.zimbra@uoguelph.ca> Date: Wed, 19 Aug 2015 16:20:15 +0300 Cc: Hans Petter Selasky , pyunyh@gmail.com, FreeBSD stable , FreeBSD Net , Slawa Olhovchenkov , Christopher Forgeron Message-Id: <2BF7FA92-2DDD-452C-822C-534C0DC0B49F@cs.huji.ac.il> References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <9D8B0503-E8FA-43CA-88F0-01F184F84D9B@cs.huji.ac.il> <1721122651.24481798.1439902381663.JavaMail.zimbra@uoguelph.ca> <55D333D6.5040102@selasky.org> <1325951625.25292515.1439934848268.JavaMail.zimbra@uoguelph.ca> <55D429A4.3010407@selasky.org> <20150819074212.GB964@michelle.fasterthan.com> <55D43615.1030401@selasky.org> <2013503980.25726607.1439989235806.JavaMail.zimbra@uoguelph.ca> To: Rick Macklem X-Mailer: Apple Mail (2.2104) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2015 13:20:26 -0000 > On 19 Aug 2015, at 16:00, Rick Macklem wrote: >=20 > Hans Petter Selasky wrote: >> On 08/19/15 09:42, Yonghyeon PYUN wrote: >>> On Wed, Aug 19, 2015 at 09:00:52AM +0200, Hans Petter Selasky wrote: >>>> On 08/18/15 23:54, Rick Macklem wrote: >>>>> Ouch! Yes, I now see that the code that counts the # of mbufs is = before >>>>> the >>>>> code that adds the tcp/ip header mbuf. >>>>>=20 >>>>> In my opinion, this should be fixed by setting = if_hw_tsomaxsegcount to >>>>> whatever >>>>> the driver provides - 1. It is not the driver's responsibility to = know if >>>>> a tcp/ip >>>>> header mbuf will be added and is a lot less confusing that = expecting the >>>>> driver >>>>> author to know to subtract one. (I had mistakenly thought that >>>>> tcp_output() had >>>>> added the tc/ip header mbuf before the loop that counts mbufs in = the >>>>> list. >>>>> Btw, >>>>> this tcp/ip header mbuf also has leading space for the MAC layer = header.) >>>>>=20 >>>>=20 >>>> Hi Rick, >>>>=20 >>>> Your question is good. With the Mellanox hardware we have separate >>>> so-called inline data space for the TCP/IP headers, so if the TCP = stack >>>> subtracts something, then we would need to add something to the = limit, >>>> because then the scatter gather list is only used for the data = part. >>>>=20 >>>=20 >>> I think all drivers in tree don't subtract 1 for >>> if_hw_tsomaxsegcount. Probably touching Mellanox driver would be >>> simpler than fixing all other drivers in tree. >>>=20 >>>> Maybe it can be controlled by some kind of flag, if all the three = TSO >>>> limits should include the TCP/IP/ethernet headers too. I'm pretty = sure >>>> we want both versions. >>>>=20 >>>=20 >>> Hmm, I'm afraid it's already complex. Drivers have to tell almost >>> the same information to both bus_dma(9) and network stack. >>=20 >> Don't forget that not all drivers in the tree set the TSO limits = before >> if_attach(), so possibly the subtraction of one TSO fragment needs to = go >> into ip_output() .... >>=20 > Ok, I realized that some drivers may not know the answers before = ether_ifattach(), > due to the way they are configured/written (I saw the use of = if_hw_tsomax_update() > in the patch). >=20 > If it is subtracted as a part of the assignment to = if_hw_tsomaxsegcount in tcp_output() > at line#791 in tcp_output() like the following, I don't think it = should matter if the > values are set before ether_ifattach()? > /* > * Subtract 1 for the tcp/ip header mbuf that > * will be prepended to the mbuf chain in this > * function in the code below this block. > */ > if_hw_tsomaxsegcount =3D tp->t_tsomaxsegcount - = 1; >=20 > I don't have a good solution for the case where a driver doesn't plan = on using the > tcp/ip header provided by tcp_output() except to say the driver can = add one to the > setting to compensate for that (and if they fail to do so, it still = works, although > somewhat suboptimally). When I now read the comment in = sys/net/if_var.h it is clear > what it means, but for some reason I didn't read it that way before? = (I think it was > the part that said the driver didn't have to subtract for the headers = that confused me?) > In any case, we need to try and come up with a clear definition of = what they need to > be set to. >=20 > I can now think of two ways to deal with this: > 1 - Leave tcp_output() as is, but provide a macro for the device = driver authors to use > that sets if_hw_tsomaxsegcount with a flag for "driver uses tcp/ip = header mbuf", > documenting that this flag should normally be true. > OR > 2 - Change tcp_output() as above, noting that this is a workaround for = confusion w.r.t. > whether or not if_hw_tsomaxsegcount should include the tcp/ip = header mbuf and > update the comment in if_var.h to reflect this. Then drivers that = don't use the > tcp/ip header mbuf can increase their value for = if_hw_tsomaxsegcount by 1. > (The comment should also mention that a value of 35 or greater is = much preferred to > 32 if the hardware will support that.) >=20 > Also, I'd like to apologize for some of my emails getting a little = "blunt". I just find > it flustrating that this problem is still showing up and is even in = 10.2. This is partly > my fault for not making it clearer to driver authors what = if_hw_tsomaxsegcount should be > set to, because I had it incorrect. >=20 > Hopefully we can come up with a solution that everyone is comfortable = with, rick ok guys, when you have some code for me to try just let me know. danny From owner-freebsd-stable@freebsd.org Wed Aug 19 15:03:12 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2312E9BC71B for ; Wed, 19 Aug 2015 15:03:12 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from elf.hq.norma.perm.ru (unknown [IPv6:2a00:7540:1::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Vivat-Trade UNIX Root CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 91D391314 for ; Wed, 19 Aug 2015 15:03:11 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from bsdrookie.norma.com. (perederenko.norma.com [IPv6:fd00::7dd] (may be forged)) by elf.hq.norma.perm.ru (8.14.9/8.14.9) with ESMTP id t7JF33fG038427 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Wed, 19 Aug 2015 20:03:04 +0500 (YEKT) (envelope-from emz@norma.perm.ru) To: FreeBSD stable From: "Eugene M. Zheganin" Subject: ipsec on recent STABLE X-Enigmail-Draft-Status: N1110 Message-ID: <55D49AA6.8020907@norma.perm.ru> Date: Wed, 19 Aug 2015 20:03:02 +0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.1.0 MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (elf.hq.norma.perm.ru [IPv6:fd00::30a]); Wed, 19 Aug 2015 20:03:04 +0500 (YEKT) X-Spam-Status: No hits=-100.2 bayes=0.0001 testhits AWL=0.288,BAYES_00=-1.9, RDNS_NONE=0.793,SPF_SOFTFAIL=0.665,USER_IN_WHITELIST=-100 autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on elf.hq.norma.perm.ru X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2015 15:03:12 -0000 Hi. Recently I built an i386 nanobsd image from a recent STABLE, r285595M (seems like some patch laso wasn't overwritten correctly in my tree), and I cannot get it to work. In the same time same revision on amd64 works fine. Symptoms are - nanobsd sends traffic just fine, it's seen on the remote end, remote end replies, these packets are seen as IPSEC on the external interface, but the gre tunnel sees only the outgoing ones. Since I was the author of the false alarm last time - I want to ask if someone is getting this too, or if someone has an gre+ipsec working on a more stable version, on i386. Or may be this can be related to the last netstat errata ? Thanks. Eugene. From owner-freebsd-stable@freebsd.org Wed Aug 19 16:07:24 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 28CFE9BE8EE for ; Wed, 19 Aug 2015 16:07:24 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 0EB258F3 for ; Wed, 19 Aug 2015 16:07:24 +0000 (UTC) (envelope-from david@catwhisker.org) Received: by mailman.ysv.freebsd.org (Postfix) id 0A6989BE8EC; Wed, 19 Aug 2015 16:07:24 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 097FF9BE8EA; Wed, 19 Aug 2015 16:07:24 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (mx.catwhisker.org [198.144.209.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C2B4B8F1; Wed, 19 Aug 2015 16:07:22 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.15.2/8.15.2) with ESMTP id t7JG7GGk067948; Wed, 19 Aug 2015 09:07:16 -0700 (PDT) (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.15.2/8.15.2/Submit) id t7JG7GU5067947; Wed, 19 Aug 2015 09:07:16 -0700 (PDT) (envelope-from david) Date: Wed, 19 Aug 2015 09:07:16 -0700 From: David Wolfskill To: stable@freebsd.org, net@freebsd.org Subject: Re: Panic [page fault] in _ieee80211_crypto_delkey(): stable/10/amd64 @r286878 Message-ID: <20150819160716.GK63584@albert.catwhisker.org> Mail-Followup-To: David Wolfskill , stable@freebsd.org, net@freebsd.org References: <20150818232007.GN1189@albert.catwhisker.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="LHvWgpbS7VDUdu2f" Content-Disposition: inline In-Reply-To: <20150818232007.GN1189@albert.catwhisker.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2015 16:07:24 -0000 --LHvWgpbS7VDUdu2f Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Aug 18, 2015 at 04:20:07PM -0700, David Wolfskill wrote: > I was minding my own business in a staff meeting this afternoon, and my > laptop rebooted; seems it got a panic. I've copied the core.txt.0 file > to , along with a > verbose dmesg.boot from this morning and output of "pciconf -l -v". >=20 > This was running: > FreeBSD localhost 10.2-STABLE FreeBSD 10.2-STABLE #122 r286878M/286880:1= 002500: Tue Aug 18 04:06:33 PDT 2015 root@g1-252.catwhisker.org:/common= /S1/obj/usr/src/sys/CANARY amd64 > .... And this morning (just after I got in to work, and was trying (and trying) to get re-associated with the AP at work), I had another one. I've copied the resulting core.txt.1 over to http://www.cawhisker.org:~david/FreeBSD/stable_10/ as well; here are excerpts from a unidiff between core.txt.{0,1}: --- core.txt.0 2015-08-18 15:39:05.232251000 -0700 +++ core.txt.1 2015-08-19 08:56:37.686238000 -0700 @@ -1,8 +1,8 @@ -localhost dumped core - see /var/crash/vmcore.0 +localhost dumped core - see /var/crash/vmcore.1 =20 -Tue Aug 18 15:39:02 PDT 2015 +Wed Aug 19 08:56:35 PDT 2015 =20 -FreeBSD localhost 10.2-STABLE FreeBSD 10.2-STABLE #122 r286878M/286880:10= 02500: Tue Aug 18 04:06:33 PDT 2015 root@g1-252.catwhisker.org:/common/= S1/obj/usr/src/sys/CANARY amd64 +FreeBSD localhost 10.2-STABLE FreeBSD 10.2-STABLE #123 r286912M/286918:10= 02500: Wed Aug 19 04:05:06 PDT 2015 root@g1-252.catwhisker.org:/common/= S1/obj/usr/src/sys/CANARY amd64 =20 panic: page fault =20 @@ -16,7 +16,7 @@ =20 Unread portion of the kernel message buffer: panic: page fault -cpuid =3D 2 +cpuid =3D 1 KDB: stack backtrace: #0 0xffffffff80946e00 at kdb_backtrace+0x60 #1 0xffffffff8090a9e6 at vpanic+0x126 @@ -34,8 +34,8 @@ #13 0xffffffff8095e9f0 at sys_ioctl+0x140 #14 0xffffffff80c84f97 at amd64_syscall+0x357 #15 0xffffffff80c6a49b at Xfast_syscall+0xfb -Uptime: 9h45m0s -Dumping 625 out of 8095 MB:..3%..11%..21%..31%..41%..52%..62%..72%..82%..9= 3% +Uptime: 3h16m49s +Dumping 584 out of 8095 MB:..3%..11%..22%..31%..42%..53%..61%..72%..83%..9= 1% =20 Reading symbols from /boot/kernel/geom_eli.ko.symbols...done. Loaded symbols for /boot/kernel/geom_eli.ko.symbols @@ -81,32 +81,32 @@ at /usr/src/sys/kern/kern_shutdown.c:687 #4 0xffffffff80c8467b in trap_fatal (frame=3D,=20 eva=3D) at /usr/src/sys/amd64/amd64/trap.c:851 -#5 0xffffffff80c8497d in trap_pfault (frame=3D0xfffffe060d88b510,=20 +#5 0xffffffff80c8497d in trap_pfault (frame=3D0xfffffe060d5ea510,=20 usermode=3D) at /usr/src/sys/amd64/amd64/trap.c:6= 74 -#6 0xffffffff80c8401a in trap (frame=3D0xfffffe060d88b510) +#6 0xffffffff80c8401a in trap (frame=3D0xfffffe060d5ea510) at /usr/src/sys/amd64/amd64/trap.c:440 #7 0xffffffff80c6a1b2 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:236 #8 0xffffffff809f003a in _ieee80211_crypto_delkey () at /usr/src/sys/net80211/ieee80211_crypto.c:105 -#9 0xffffffff809eff5e in ieee80211_crypto_delkey (vap=3D0xfffffe03d907000= 0,=20 - key=3D0xfffffe03d9070800) at /usr/src/sys/net80211/ieee80211_crypto.c:= 461 -#10 0xffffffff80a04d45 in ieee80211_ioctl_delkey (vap=3D0xfffffe03d9070000= ,=20 +#9 0xffffffff809eff5e in ieee80211_crypto_delkey (vap=3D0xfffffe03dd31a00= 0,=20 + key=3D0xfffffe03dd31a800) at /usr/src/sys/net80211/ieee80211_crypto.c:= 461 +#10 0xffffffff80a04d45 in ieee80211_ioctl_delkey (vap=3D0xfffffe03dd31a000= ,=20 ireq=3D) at /usr/src/sys/net80211/ieee80211_ioctl.c:1252 #11 0xffffffff80a03bd2 in ieee80211_ioctl_set80211 () at /usr/src/sys/net80211/ieee80211_ioctl.c:2814 #12 0xffffffff80a2c323 in in_control (so=3D,=20 - cmd=3D9214790412651315593, data=3D0xfffffe060d88bb80 "", ifp=3D0x3,=20 + cmd=3D9214790412651315593, data=3D0xfffffe060d5eab80 "", ifp=3D0x3,=20 td=3D) at /usr/src/sys/netinet/in.c:308 -#13 0xffffffff809cd57b in ifioctl (so=3D0xfffffe03d9070800, cmd=3D21496079= 14,=20 - data=3D0xfffffe060d88b8e0 "wlan0", td=3D0xfffff80170abb940) +#13 0xffffffff809cd57b in ifioctl (so=3D0xfffffe03dd31a800, cmd=3D21496079= 14,=20 + data=3D0xfffffe060d5ea8e0 "wlan0", td=3D0xfffff800098b5940) at /usr/src/sys/net/if.c:2770 -#14 0xffffffff8095ecf5 in kern_ioctl (td=3D0xfffff80170abb940,=20 - fd=3D, com=3D18446741891212314624) at file.h:320 -#15 0xffffffff8095e9f0 in sys_ioctl (td=3D0xfffff80170abb940,=20 - uap=3D0xfffffe060d88ba40) at /usr/src/sys/kern/sys_generic.c:718 -#16 0xffffffff80c84f97 in amd64_syscall (td=3D0xfffff80170abb940, traced= =3D0) +#14 0xffffffff8095ecf5 in kern_ioctl (td=3D0xfffff800098b5940,=20 + fd=3D, com=3D18446741891282216960) at file.h:320 +#15 0xffffffff8095e9f0 in sys_ioctl (td=3D0xfffff800098b5940,=20 + uap=3D0xfffffe060d5eaa40) at /usr/src/sys/kern/sys_generic.c:718 +#16 0xffffffff80c84f97 in amd64_syscall (td=3D0xfffff800098b5940, traced= =3D0) at subr_syscall.c:134 #17 0xffffffff80c6a49b in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:396 @@ -118,305 +118,301 @@ ------------------------------------------------------------------------ =2E... So it looks to me to be quite similar to the previous one. I've also copied the kernel config file ("CANARY") to the above-cited Web page. Anything else I can do to help nail this? Peace, david --=20 David H. Wolfskill david@catwhisker.org Those who would murder in the name of God or prophet are blasphemous coward= s. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --LHvWgpbS7VDUdu2f Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQJ8BAEBCgBmBQJV1KmzXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ4RThEMDY4QTIxMjc1MDZFRDIzODYzRTc4 QTY3RjlDOERFRjQxOTNCAAoJEIpn+cje9Bk7NQAP/1o6K5jgq/F77e/0pWv/BC7Z VRCJjOtoFVeUgYidHFJU7yRUTAJSmNbKI4h41/nFf5simA4e1NIt6GOl0PUa+G2F JVyG1BFE7WFdQoRbPlLMwdAW1zGmi83F22XS+F47AnzCNhm0RClY6fMQ3JphU9N5 1LkZq0FNxeWGFJlZL4JJCkQAJCFSJckX+OReLzbW8nZjxxUMsymsz40J8aRGFGMu yembRQIw44r9/BCEaWCJW8mTZkbR9z7G3hTQo1luSI5u2GB0812Pjd4RkFd/hoWM l3Hdog+VBIVoIXXJvIBTa4Y40dPIA24cRvarKxZVKOh8M4ZTdQbkwatZGJvxDoFg 8qCJA/IRbBp9n6TZvD4pW5rKGN4dZ+wOCT6NOGkFYmYZ6JEqy0z7IZnHON2hVmrJ lUAvU4fFRPZcDYi0X8+SYN9R/rO+tJxqDd3HsOPbS/m61HO/Q2lTMTtsaDC9uPvj PAZAuVWZHEACdW9HHqumC45/AM+3PYHub66xRalG7yYdiN3oTqCc2B3uV7Sj0BXI w7af5cuFCs+kXIRIlDzbzfp4Rg0/48twEKrrZr0R+44rjv3ERN/4PXfQNAfz1p8s T6Xwrb6cHdGvr8SPnB8529CpvmNAON8nMUT/B2Wx3v88ZKzDumygX61BKrcSe249 YI71hq9IsSBzoNwk5IQ0 =UghK -----END PGP SIGNATURE----- --LHvWgpbS7VDUdu2f-- From owner-freebsd-stable@freebsd.org Wed Aug 19 16:20:21 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BBAD39BEB95 for ; Wed, 19 Aug 2015 16:20:21 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-lb0-f172.google.com (mail-lb0-f172.google.com [209.85.217.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2B600F69 for ; Wed, 19 Aug 2015 16:20:20 +0000 (UTC) (envelope-from ml@my.gd) Received: by lbbtg9 with SMTP id tg9so6738277lbb.1 for ; Wed, 19 Aug 2015 09:20:18 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=9tVo/N3fX6Ucf+SZvDtcKYuTb9T/qs7w/DdTfnV7DXM=; b=Y6OY8oivNc1op3vffDqkvxewfIv2FDTY86vXBVu1B/WJkqz3tugh+d3w/YxVTywsw2 ErW1jXMEXM1SXmjwJO3Jyg0hH1qXiMe8DmKsb59BuIPPTe1AeI75HC0ffPx9ejY3GlvZ 6swaTEHB1FBbH0G2yKCURWTXvsrIeVWbRpJqvE3yoQj8ZgnJ/jJpHW/Zb8qfgPPrVfcn i+KBjKZA2SsryWG7P8BQQWio6C9026x6Qn/T7PjCRWZa6hPUwGCi2o9p/ZBFVPGDdUxi EN4dufTg+OaJ4WsuLTq8Rvvqu0/D/vRcKRsq7+tIQKmUJef6pWuJeESrdQBIJfZTRSDY ZMZw== X-Gm-Message-State: ALoCoQmyrfMymWnTjL3IxAVenjnMASu1LfuSZYBtdIi3JfHm1PWMZBPX1gnTFYbY5swNB2+hoGSx MIME-Version: 1.0 X-Received: by 10.152.22.4 with SMTP id z4mr11954922lae.81.1440001218357; Wed, 19 Aug 2015 09:20:18 -0700 (PDT) Received: by 10.112.60.34 with HTTP; Wed, 19 Aug 2015 09:20:18 -0700 (PDT) In-Reply-To: References: Date: Wed, 19 Aug 2015 18:20:18 +0200 Message-ID: Subject: Re: [POSSIBLE BUG] 10-STABLE CARP erroneously becomes master on boot From: Damien Fleuriot To: Freddie Cash Cc: FreeBSD Stable , Damien Fleuriot Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2015 16:20:21 -0000 On 19 August 2015 at 11:29, Damien Fleuriot wrote: > Hello list, Freddie, > > > I've been able to run extensive tests, the results of which I'm pasting > below. > > > First of all, I've been unable to replicate the problem in our > preproduction and QA environments. > > The differences between our production and pre/QA envs are as follows : > - we use link aggregation and VLAN tagging in production , we cannot > replicate this in pre/QA because of limitations with KVM guests. > - we use multiple CARP addresses with the same VHID on our public VLAN in > production, we COULD replicate that in pre/QA if required. > > > The context remains the following : > - host A is supposed to be CARP MASTER, has advskew 20 and preempt > - host B is supposed to be CARP BACKUP, has advskew 150 and preempt > - host B assumes mastership if the CARPs are configured from rc.conf , > doesn't if they're set up manually after boot > > Note that these 2 boxes were upgraded from 8-STABLE to 10-STABLE. > Host A runs 10.2-BETA1 > Host B runs 10.2-PRERELEASE > > I used the exact same versions in our pre/QA environments (one BETA1, one > PRERELEASE from the same build) and couldn't replicate the issue. > > > > Now, on to the tests themselves. > > > A/ Create a new CARP address with a new VHID, configure it in rc.conf and > see if we get double MASTERS > - on Host A : CARP created manually > - on Host B : CARP created manually > Host A is MASTER and Host B is BACKUP > > - on Host B : setup the new CARP in rc.conf , reboot > Host A is MASTER and Host B is BACKUP , problem not replicated > > > B/ Try with only one of our production addresses > - on Host B : uncomment the production CARP address from rc.conf , reboot > Host A is MASTER and Host B is MASTER > Host A shows net.inet.carp.demotion=0 > Host B shows net.inet.carp.demotion=240 > > > C/ Try with the new CARP address + one of our production addresses , > different VHIDs > - on Host B : uncomment the new CARP address from rc.conf , reboot > Host A is MASTER and Host B is MASTER > Host A shows net.inet.carp.demotion=0 > Host B shows net.inet.carp.demotion=240 > > > D/ Try the new syntax in rc.conf , as per Freddie's suggestion > - on Host B : change the rc.conf syntax , reboot > Host A is MASTER and Host B is MASTER > Host A shows net.inet.carp.demotion=0 > Host B shows net.inet.carp.demotion=240 > > > E/ Try, just for the sake of it, to remove old files and libs on host B > - on Host B : cd /usr/src ; yes | make delete-old ; yes | make > delete-old-libs ; reboot > Host A is MASTER and Host B is MASTER > Host A shows net.inet.carp.demotion=0 > Host B shows net.inet.carp.demotion=240 > > > F/ Edit sysctls to disable CARP demotion on advertisement send errors > - on Host A : sysctl net.inet.carp.senderr_demotion_factor=0 > - on Host B : set "net.inet.carp.senderr_demotion_factor=0" in sysctl.conf > , reboot > Host A is MASTER and Host B is MASTER > Host A shows net.inet.carp.demotion=0 > Host B shows net.inet.carp.demotion=240 > > > > Now after this F/ test, I'm thinking there's some voodoo going on here and > it sure shows up : > - on Host A, 'pfctl -si' shows 3k states > - on Host B, 'pfctl -si' shows 800 states > > Now that would explain why my CARP gets demoted on Host B (as per man 4 > carp, pfsync failures result in a -240 demotion). > It doesn't, however, explain why the demoted CARP chooses to remain in a > MASTER state, or assumed MASTERship in the first place. > > Surely enough, I can find some CARP errors in-between my reboots : > messages:2420:2015-08-19T03:51:38.273600+00:00 pf1-gs kernel: carp: > demoted by -240 to 0 (pfsync bulk fail) > messages:2429:2015-08-19T03:56:37.178575+00:00 pf1-gs kernel: carp: VHID > 110@vlan410: INIT -> BACKUP > messages:2430:2015-08-19T03:56:40.664568+00:00 pf1-gs kernel: carp: VHID > 110@vlan410: BACKUP -> MASTER (master down) > messages:2637:2015-08-19T04:00:22.482071+00:00 pf1-gs kernel: carp: VHID > 111@vlan410: BACKUP -> MASTER (master down) > messages:2857:2015-08-19T04:04:02.330167+00:00 pf1-gs kernel: carp: VHID > 110@vlan410: BACKUP -> MASTER (master down) > messages:2877:2015-08-19T04:05:03.288199+00:00 pf1-gs kernel: carp: > demoted by -240 to 0 (pfsync bulk fail) > messages:3088:2015-08-19T04:08:48.961985+00:00 pf1-gs kernel: carp: VHID > 110@vlan410: BACKUP -> MASTER (master down) > > > Things I have not tested yet : > - use /24 CARPs instead of /32s > - switch my CARPs to all use different VHIDs > > Things I CANNOT test : > - set up a *dedicated* pfsync link between the firewalls , they're in > different DCs > - set up a *dedicated* VLAN for pfsync , that would entail huge changes in > our PCI-DSS environment > > > After all these tests, I find myself in a situation where : > - manually set up CARPs on Host B work fine , and pfsync works > - CARPs from rc.conf on Host B result in MASTER-MASTER , and pfsync fails > > > > > I must say I'm stuck here. > The "master down" message is very confusing, when the firewalls *can* see > each other. > The "pfsync bulk fail" is rather interesting as well, since it doesn't > occur when the CARPs are unconfigured, or set up manually without rc.conf. > tcpdump -nei pflog0 does not show any dropped packet. > PF is configured to "pass in quick" pfsync and CARP packets. > > > > I'm afraid a STFW hasn't helped overmuch here, although I'll try some more. > > > If anyone's got a pointer, I'll bite. > > Cheers > > FWIW , additional testing shows that on Host A (10.2-BETA) CARP advertisements are sent from the CARP IP itself. As in, my Host A has physical IPs X and Y, but sends its CARP announces from CARP IP Z. I have not been able to replicate this behaviour in our preproduction environment where both -BETA and -PRERELEASE send the advertisements from their physical IP. These boxes however, have just the one IP, as opposed to the production environment which has 2 physical IPs. I will get some additional testing done, by swapping over to make the Host B running -PRERELEASE master, and see if it also sends its CARP announcements sourced from its physical IP (would be good) or the CARP itself (would be bad). > > On 17 August 2015 at 18:38, Damien Fleuriot wrote: > >> >> On 17 August 2015 at 18:32, Freddie Cash wrote: >> >>> >>> On Aug 17, 2015 9:22 AM, "Damien Fleuriot" wrote: >>> > >>> > Hello list, >>> > >>> > >>> > >>> > I'm seeing this very peculiar behaviour between 2 10-STABLE boxes. >>> > >>> > Host A is CARP Master with advskew 20 and runs 10.2-BETA1 from 10/07 >>> > Host B is CARP Backup with advskew 150 and runs 10.2-PRERELEASE from >>> 12/08 >>> > >>> > >>> > When I configure CARP in rc.conf on host B, it becomes Master on boot, >>> and >>> > host A remains Master as well. >>> > When I force a state change on host B (ifconfig vlanx vhid y state >>> backup), >>> > it transitions to Backup then again to Master. >>> > >>> > When I comment out the CARP configuration in rc.conf , and configure >>> CARP >>> > manually on host B's interfaces after it boots, it correctly becomes >>> and >>> > remains Backup. >>> > >>> > >>> > >>> > Below is the excerpt from rc.conf pertaining to CARP configuration, the >>> > only difference between the 2 hosts being their advskew. >>> > >>> > Host A >>> > == BEGIN >>> > >>> > ifconfig_vlan410_alias0="vhid 110 pass passhere advskew 20 alias >>> > 10.104.10.251/32" >>> > >>> > == END >>> > >>> > Host B >>> > == BEGIN >>> > >>> > ifconfig_vlan410_alias0="vhid 110 pass passhere advskew 150 alias >>> > 10.104.10.251/32" >>> > >>> > == END >>> >>> Put the IP first, and the vhid stuff last in rc.conf for things to work >>> the most reliably. And drop the extra alias. >>> >>> ifconfig_vlan410_alias0="inet 10.104.10.251/32 vhid 110 pass passhere >>> advskew 150" >>> >>> CARP requires that all IPs on an interface that are part of the same >>> vhid to be listed (added) in the exact same order for the vhid to be >>> considered "the same". That one trips me up all the time when manually >>> adding an IP to a CARP pair, and then later rebooting one box as they both >>> think they're master for that interface, while being a mix of master/backup >>> for the other interfaces. >>> >>> Cheers, >>> Freddie >>> (running CARP on 2 10-CURRENT boxes and 2 10.1-p13 boxes) >>> >> >> Cheers Freddie, will try and keep the thread up to date on the results. >> >> > From owner-freebsd-stable@freebsd.org Wed Aug 19 18:58:57 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6D55C9BEC15 for ; Wed, 19 Aug 2015 18:58:57 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 4E94F1847 for ; Wed, 19 Aug 2015 18:58:57 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 4AEF09BEC13; Wed, 19 Aug 2015 18:58:57 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 303749BEC10; Wed, 19 Aug 2015 18:58:57 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-ig0-x22d.google.com (mail-ig0-x22d.google.com [IPv6:2607:f8b0:4001:c05::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F06A81845; Wed, 19 Aug 2015 18:58:56 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by igxp17 with SMTP id p17so112743126igx.1; Wed, 19 Aug 2015 11:58:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=rOZAtgpPO8xS7uVnQoHJEBkLTFsEgT4USulqOUNeF3Y=; b=BEtCxkJfo8H18LstSn18JcrSYHDE8mm/MjoYU0CZALr932hkAvzlxtUzgbVCPj6far Pn+BRuEjiqKDby1uqXr8Is0h2NP1KYyprNlK95KJwW9jlACZo9vyAoTK2OXOAOmR0wRL FK1nD5BQncEfd+TZDdLV2LWt8DsZDIePp9QlMIu5ZBnHADekycHw9FBMQSWA51IjLOOf Cqj6y3AXnahZx2bLssbxkE783fjB+kb4h/70mwr+pRgM4jz0T3DavkvLVKe6SsPueeUV wNPa6wsGfQnxEchunXOzbQNyzWADntWs741oj+PaHtSiearCnHEkzAnj1aBlQi5hC28x qeHQ== MIME-Version: 1.0 X-Received: by 10.50.43.227 with SMTP id z3mr28127613igl.22.1440010736374; Wed, 19 Aug 2015 11:58:56 -0700 (PDT) Received: by 10.36.38.133 with HTTP; Wed, 19 Aug 2015 11:58:56 -0700 (PDT) In-Reply-To: <20150819160716.GK63584@albert.catwhisker.org> References: <20150818232007.GN1189@albert.catwhisker.org> <20150819160716.GK63584@albert.catwhisker.org> Date: Wed, 19 Aug 2015 11:58:56 -0700 Message-ID: Subject: Re: Panic [page fault] in _ieee80211_crypto_delkey(): stable/10/amd64 @r286878 From: Adrian Chadd To: David Wolfskill , "stable@freebsd.org" , "net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2015 18:58:57 -0000 hi, you'll have to do some debugging. it looks like it's some kind of odd race - line 461 is _ieee80211_crypto_delkey(); line 105 is cipher_detach() and it blows up there. Try "wlandebug +crypto" during your next boot and let's see what it logs for the key. If you can 'print *key' in kgdb on the core at some frame then we should get some useful information. -a On 19 August 2015 at 09:07, David Wolfskill wrote: > On Tue, Aug 18, 2015 at 04:20:07PM -0700, David Wolfskill wrote: >> I was minding my own business in a staff meeting this afternoon, and my >> laptop rebooted; seems it got a panic. I've copied the core.txt.0 file >> to , along with a >> verbose dmesg.boot from this morning and output of "pciconf -l -v". >> >> This was running: >> FreeBSD localhost 10.2-STABLE FreeBSD 10.2-STABLE #122 r286878M/286880:1002500: Tue Aug 18 04:06:33 PDT 2015 root@g1-252.catwhisker.org:/common/S1/obj/usr/src/sys/CANARY amd64 >> .... > > And this morning (just after I got in to work, and was trying (and > trying) to get re-associated with the AP at work), I had another one. > > I've copied the resulting core.txt.1 over to > http://www.cawhisker.org:~david/FreeBSD/stable_10/ as well; here are > excerpts from a unidiff between core.txt.{0,1}: > > --- core.txt.0 2015-08-18 15:39:05.232251000 -0700 > +++ core.txt.1 2015-08-19 08:56:37.686238000 -0700 > @@ -1,8 +1,8 @@ > -localhost dumped core - see /var/crash/vmcore.0 > +localhost dumped core - see /var/crash/vmcore.1 > > -Tue Aug 18 15:39:02 PDT 2015 > +Wed Aug 19 08:56:35 PDT 2015 > > -FreeBSD localhost 10.2-STABLE FreeBSD 10.2-STABLE #122 r286878M/286880:1002500: Tue Aug 18 04:06:33 PDT 2015 root@g1-252.catwhisker.org:/common/S1/obj/usr/src/sys/CANARY amd64 > +FreeBSD localhost 10.2-STABLE FreeBSD 10.2-STABLE #123 r286912M/286918:1002500: Wed Aug 19 04:05:06 PDT 2015 root@g1-252.catwhisker.org:/common/S1/obj/usr/src/sys/CANARY amd64 > > panic: page fault > > @@ -16,7 +16,7 @@ > > Unread portion of the kernel message buffer: > panic: page fault > -cpuid = 2 > +cpuid = 1 > KDB: stack backtrace: > #0 0xffffffff80946e00 at kdb_backtrace+0x60 > #1 0xffffffff8090a9e6 at vpanic+0x126 > @@ -34,8 +34,8 @@ > #13 0xffffffff8095e9f0 at sys_ioctl+0x140 > #14 0xffffffff80c84f97 at amd64_syscall+0x357 > #15 0xffffffff80c6a49b at Xfast_syscall+0xfb > -Uptime: 9h45m0s > -Dumping 625 out of 8095 MB:..3%..11%..21%..31%..41%..52%..62%..72%..82%..93% > +Uptime: 3h16m49s > +Dumping 584 out of 8095 MB:..3%..11%..22%..31%..42%..53%..61%..72%..83%..91% > > Reading symbols from /boot/kernel/geom_eli.ko.symbols...done. > Loaded symbols for /boot/kernel/geom_eli.ko.symbols > @@ -81,32 +81,32 @@ > at /usr/src/sys/kern/kern_shutdown.c:687 > #4 0xffffffff80c8467b in trap_fatal (frame=, > eva=) at /usr/src/sys/amd64/amd64/trap.c:851 > -#5 0xffffffff80c8497d in trap_pfault (frame=0xfffffe060d88b510, > +#5 0xffffffff80c8497d in trap_pfault (frame=0xfffffe060d5ea510, > usermode=) at /usr/src/sys/amd64/amd64/trap.c:674 > -#6 0xffffffff80c8401a in trap (frame=0xfffffe060d88b510) > +#6 0xffffffff80c8401a in trap (frame=0xfffffe060d5ea510) > at /usr/src/sys/amd64/amd64/trap.c:440 > #7 0xffffffff80c6a1b2 in calltrap () > at /usr/src/sys/amd64/amd64/exception.S:236 > #8 0xffffffff809f003a in _ieee80211_crypto_delkey () > at /usr/src/sys/net80211/ieee80211_crypto.c:105 > -#9 0xffffffff809eff5e in ieee80211_crypto_delkey (vap=0xfffffe03d9070000, > - key=0xfffffe03d9070800) at /usr/src/sys/net80211/ieee80211_crypto.c:461 > -#10 0xffffffff80a04d45 in ieee80211_ioctl_delkey (vap=0xfffffe03d9070000, > +#9 0xffffffff809eff5e in ieee80211_crypto_delkey (vap=0xfffffe03dd31a000, > + key=0xfffffe03dd31a800) at /usr/src/sys/net80211/ieee80211_crypto.c:461 > +#10 0xffffffff80a04d45 in ieee80211_ioctl_delkey (vap=0xfffffe03dd31a000, > ireq=) > at /usr/src/sys/net80211/ieee80211_ioctl.c:1252 > #11 0xffffffff80a03bd2 in ieee80211_ioctl_set80211 () > at /usr/src/sys/net80211/ieee80211_ioctl.c:2814 > #12 0xffffffff80a2c323 in in_control (so=, > - cmd=9214790412651315593, data=0xfffffe060d88bb80 "", ifp=0x3, > + cmd=9214790412651315593, data=0xfffffe060d5eab80 "", ifp=0x3, > td=) at /usr/src/sys/netinet/in.c:308 > -#13 0xffffffff809cd57b in ifioctl (so=0xfffffe03d9070800, cmd=2149607914, > - data=0xfffffe060d88b8e0 "wlan0", td=0xfffff80170abb940) > +#13 0xffffffff809cd57b in ifioctl (so=0xfffffe03dd31a800, cmd=2149607914, > + data=0xfffffe060d5ea8e0 "wlan0", td=0xfffff800098b5940) > at /usr/src/sys/net/if.c:2770 > -#14 0xffffffff8095ecf5 in kern_ioctl (td=0xfffff80170abb940, > - fd=, com=18446741891212314624) at file.h:320 > -#15 0xffffffff8095e9f0 in sys_ioctl (td=0xfffff80170abb940, > - uap=0xfffffe060d88ba40) at /usr/src/sys/kern/sys_generic.c:718 > -#16 0xffffffff80c84f97 in amd64_syscall (td=0xfffff80170abb940, traced=0) > +#14 0xffffffff8095ecf5 in kern_ioctl (td=0xfffff800098b5940, > + fd=, com=18446741891282216960) at file.h:320 > +#15 0xffffffff8095e9f0 in sys_ioctl (td=0xfffff800098b5940, > + uap=0xfffffe060d5eaa40) at /usr/src/sys/kern/sys_generic.c:718 > +#16 0xffffffff80c84f97 in amd64_syscall (td=0xfffff800098b5940, traced=0) > at subr_syscall.c:134 > #17 0xffffffff80c6a49b in Xfast_syscall () > at /usr/src/sys/amd64/amd64/exception.S:396 > @@ -118,305 +118,301 @@ > ------------------------------------------------------------------------ > .... > > > So it looks to me to be quite similar to the previous one. > > I've also copied the kernel config file ("CANARY") to the above-cited > Web page. > > Anything else I can do to help nail this? > > Peace, > david > -- > David H. Wolfskill david@catwhisker.org > Those who would murder in the name of God or prophet are blasphemous cowards. > > See http://www.catwhisker.org/~david/publickey.gpg for my public key. From owner-freebsd-stable@freebsd.org Wed Aug 19 20:01:30 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 97B569BE765 for ; Wed, 19 Aug 2015 20:01:30 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 7589A1ADB for ; Wed, 19 Aug 2015 20:01:30 +0000 (UTC) (envelope-from david@catwhisker.org) Received: by mailman.ysv.freebsd.org (Postfix) id 72BC89BE762; Wed, 19 Aug 2015 20:01:30 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 71E709BE760; Wed, 19 Aug 2015 20:01:30 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (mx.catwhisker.org [198.144.209.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 115DF1AD2; Wed, 19 Aug 2015 20:01:26 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.15.2/8.15.2) with ESMTP id t7JK1ONJ069448; Wed, 19 Aug 2015 13:01:24 -0700 (PDT) (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.15.2/8.15.2/Submit) id t7JK1Ohx069447; Wed, 19 Aug 2015 13:01:24 -0700 (PDT) (envelope-from david) Date: Wed, 19 Aug 2015 13:01:24 -0700 From: David Wolfskill To: stable@freebsd.org, net@freebsd.org, wireless@freebsd.org Cc: Adrian Chadd Subject: Re: Panic [page fault] in _ieee80211_crypto_delkey(): stable/10/amd64 @r286878 Message-ID: <20150819200124.GR63584@albert.catwhisker.org> Mail-Followup-To: David Wolfskill , stable@freebsd.org, net@freebsd.org, wireless@freebsd.org, Adrian Chadd References: <20150818232007.GN1189@albert.catwhisker.org> <20150819160716.GK63584@albert.catwhisker.org> <20150819190852.GN63584@albert.catwhisker.org> <20150819192315.GO63584@albert.catwhisker.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="/Isdj7O9hWi8F9Bn" Content-Disposition: inline In-Reply-To: <20150819160716.GK63584@albert.catwhisker.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2015 20:01:30 -0000 --/Isdj7O9hWi8F9Bn Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Aug 19, 2015 at 12:25:38PM -0700, Adrian Chadd wrote: > ... But we definitely ahe enough to put into a PR.. > ... Bug 202494 - Panic [page fault] in _ieee80211_crypto_delkey()=20 Peace, david --=20 David H. Wolfskill david@catwhisker.org Those who would murder in the name of God or prophet are blasphemous coward= s. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --/Isdj7O9hWi8F9Bn Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQJ8BAEBCgBmBQJV1OCUXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ4RThEMDY4QTIxMjc1MDZFRDIzODYzRTc4 QTY3RjlDOERFRjQxOTNCAAoJEIpn+cje9Bk7b50P/084RhNSE/TNlG20ZhnOFh4A jAOGPpM/CSmmXUd0GW22JOEq9q7EsymaRkip2vVN34eugEMW7HrXJXtX1I9gR8ji 7MsAYUo3yBHnuI4vrKeeq0YOOwxeZdZXWaZMCU72YAsK/1GJQHmKAZjwx4x00FqL kGjyZeSNHBmQtCfip36rrrlhcpbQlSl7G0Gocz3fh33eVM21jmfjUyBf/ehHcKFd VJogqTkItR1OEoaxNXKk0jqAKqeFEY2B7OtN9rW3KSb/pEVZ4pjGoTEhLk/5Z7tS sdqHqFWVkjyszqANPSjnpAd2KoiUoPbiEcjyFfRjvLEk/sN7MoJZQyhIhelnDWrl 9OdZfXnsf9M3/pI+3Lj4VBYhBV6THzeo+z/FZJ5Mrm04/3NzYAfYEeNCB0z3AGiH lwjCW1iNWFsU9Mu4WwTZggnxgRXxnC3FmJK9QQacAh5LdEtu7CfPdx7mhft5cwkJ D801WXX6u2pVy28T8GP8AaWRhH4l3u7IB04Wz9g36rGlUaUm72no9x595GBTkorw h7X23Dk2CfLceaCeVeRX5WniB1li82NbnjHTmKgFReeYwZp4tRvxzKUL30v28p87 1QYDRrNNTqPh4m/7ASNuvYROnURAiVYc4H0rcrTxQgLmsyIhgZkKzK/7bebuyxD3 ffwqPaV28gm/o5vasIK5 =iqAq -----END PGP SIGNATURE----- --/Isdj7O9hWi8F9Bn-- From owner-freebsd-stable@freebsd.org Wed Aug 19 20:57:30 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 43C499BE035; Wed, 19 Aug 2015 20:57:30 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id DB090646; Wed, 19 Aug 2015 20:57:29 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:qC6hNR9j2YktH/9uRHKM819IXTAuvvDOBiVQ1KB91OMcTK2v8tzYMVDF4r011RmSDd6dt6wP0reP+4nbGkU+or+5+EgYd5JNUxJXwe43pCcHRPC/NEvgMfTxZDY7FskRHHVs/nW8LFQHUJ2mPw6anHS+4HYoFwnlMkItf6KuStWU05r8jr3rs7ToICx2xxOFKYtoKxu3qQiD/uI3uqBFbpgL9x3Sv3FTcP5Xz247bXianhL7+9vitMU7q3cYk7sb+sVBSaT3ebgjBfwdVWx+cjN92Mq+mRDFTAaLrlEGW2MXiQEAVwTM6hfrdpzq9CvntOs70SLcPMmgHp4uXjH31aZgS1fNgSwEMzM8uDXNj8V7j6ZWpTq8oBNizorMYMeePawtLevmYdoGSD8ZDY5qXCtbD9b5NtNXAg== X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2AoAgCr19RV/61jaINdDoNhaQaDH7o3AQmBbQqFMUoCgX0UAQEBAQEBAQGBCYIdggYBAQEDAQEBASAEJyALEAIBCA4KAgINGQICJwEJJgIECAcEARoCBIgFCA25M5YVAQEBAQEBAQEBAQEBAQEBAQEXBIEiijGEMgYBARw0B4JpgUMFlSSFBIUHhCyHRohxhEiDZgImgg4cgRVaIjMHfwgXI4EEAQEB X-IronPort-AV: E=Sophos;i="5.15,712,1432612800"; d="scan'208";a="231754346" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 19 Aug 2015 16:57:27 -0400 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id E17E715F55D; Wed, 19 Aug 2015 16:57:27 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id gBxRjkkCPWn2; Wed, 19 Aug 2015 16:57:27 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 528DB15F563; Wed, 19 Aug 2015 16:57:27 -0400 (EDT) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id oDXVeFL-4bp7; Wed, 19 Aug 2015 16:57:27 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 3256D15F55D; Wed, 19 Aug 2015 16:57:27 -0400 (EDT) Date: Wed, 19 Aug 2015 16:57:27 -0400 (EDT) From: Rick Macklem To: Daniel Braniss Cc: Hans Petter Selasky , pyunyh@gmail.com, FreeBSD stable , FreeBSD Net , Slawa Olhovchenkov , Christopher Forgeron Message-ID: <796827231.26478408.1440017847125.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <2BF7FA92-2DDD-452C-822C-534C0DC0B49F@cs.huji.ac.il> References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <55D333D6.5040102@selasky.org> <1325951625.25292515.1439934848268.JavaMail.zimbra@uoguelph.ca> <55D429A4.3010407@selasky.org> <20150819074212.GB964@michelle.fasterthan.com> <55D43615.1030401@selasky.org> <2013503980.25726607.1439989235806.JavaMail.zimbra@uoguelph.ca> <2BF7FA92-2DDD-452C-822C-534C0DC0B49F@cs.huji.ac.il> Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.11] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF39 (Win)/8.0.9_GA_6191) Thread-Topic: ix(intel) vs mlxen(mellanox) 10Gb performance Thread-Index: arSbth0Dm/H1DJKuXgkJL3pJQvjzbg== X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Aug 2015 20:57:30 -0000 Daniel Braniss wrote: > > > On 19 Aug 2015, at 16:00, Rick Macklem wrote: > > > > Hans Petter Selasky wrote: > >> On 08/19/15 09:42, Yonghyeon PYUN wrote: > >>> On Wed, Aug 19, 2015 at 09:00:52AM +0200, Hans Petter Selasky wrote: > >>>> On 08/18/15 23:54, Rick Macklem wrote: > >>>>> Ouch! Yes, I now see that the code that counts the # of mbufs is before > >>>>> the > >>>>> code that adds the tcp/ip header mbuf. > >>>>> > >>>>> In my opinion, this should be fixed by setting if_hw_tsomaxsegcount to > >>>>> whatever > >>>>> the driver provides - 1. It is not the driver's responsibility to know > >>>>> if > >>>>> a tcp/ip > >>>>> header mbuf will be added and is a lot less confusing that expecting > >>>>> the > >>>>> driver > >>>>> author to know to subtract one. (I had mistakenly thought that > >>>>> tcp_output() had > >>>>> added the tc/ip header mbuf before the loop that counts mbufs in the > >>>>> list. > >>>>> Btw, > >>>>> this tcp/ip header mbuf also has leading space for the MAC layer > >>>>> header.) > >>>>> > >>>> > >>>> Hi Rick, > >>>> > >>>> Your question is good. With the Mellanox hardware we have separate > >>>> so-called inline data space for the TCP/IP headers, so if the TCP stack > >>>> subtracts something, then we would need to add something to the limit, > >>>> because then the scatter gather list is only used for the data part. > >>>> > >>> > >>> I think all drivers in tree don't subtract 1 for > >>> if_hw_tsomaxsegcount. Probably touching Mellanox driver would be > >>> simpler than fixing all other drivers in tree. > >>> > >>>> Maybe it can be controlled by some kind of flag, if all the three TSO > >>>> limits should include the TCP/IP/ethernet headers too. I'm pretty sure > >>>> we want both versions. > >>>> > >>> > >>> Hmm, I'm afraid it's already complex. Drivers have to tell almost > >>> the same information to both bus_dma(9) and network stack. > >> > >> Don't forget that not all drivers in the tree set the TSO limits before > >> if_attach(), so possibly the subtraction of one TSO fragment needs to go > >> into ip_output() .... > >> > > Ok, I realized that some drivers may not know the answers before > > ether_ifattach(), > > due to the way they are configured/written (I saw the use of > > if_hw_tsomax_update() > > in the patch). > > > > If it is subtracted as a part of the assignment to if_hw_tsomaxsegcount in > > tcp_output() > > at line#791 in tcp_output() like the following, I don't think it should > > matter if the > > values are set before ether_ifattach()? > > /* > > * Subtract 1 for the tcp/ip header mbuf that > > * will be prepended to the mbuf chain in this > > * function in the code below this block. > > */ > > if_hw_tsomaxsegcount = tp->t_tsomaxsegcount - 1; > > Well, you can replace the line in sys/netinet/tcp_output.c that looks like: if_hw_tsomaxsegcount = tp->t_tsomaxsegcount; with the above line (at line #797 in head). Any other patch for this will have the same effect, rick > > I don't have a good solution for the case where a driver doesn't plan on > > using the > > tcp/ip header provided by tcp_output() except to say the driver can add one > > to the > > setting to compensate for that (and if they fail to do so, it still works, > > although > > somewhat suboptimally). When I now read the comment in sys/net/if_var.h it > > is clear > > what it means, but for some reason I didn't read it that way before? (I > > think it was > > the part that said the driver didn't have to subtract for the headers that > > confused me?) > > In any case, we need to try and come up with a clear definition of what > > they need to > > be set to. > > > > I can now think of two ways to deal with this: > > 1 - Leave tcp_output() as is, but provide a macro for the device driver > > authors to use > > that sets if_hw_tsomaxsegcount with a flag for "driver uses tcp/ip > > header mbuf", > > documenting that this flag should normally be true. > > OR > > 2 - Change tcp_output() as above, noting that this is a workaround for > > confusion w.r.t. > > whether or not if_hw_tsomaxsegcount should include the tcp/ip header > > mbuf and > > update the comment in if_var.h to reflect this. Then drivers that don't > > use the > > tcp/ip header mbuf can increase their value for if_hw_tsomaxsegcount by > > 1. > > (The comment should also mention that a value of 35 or greater is much > > preferred to > > 32 if the hardware will support that.) > > > > Also, I'd like to apologize for some of my emails getting a little "blunt". > > I just find > > it flustrating that this problem is still showing up and is even in 10.2. > > This is partly > > my fault for not making it clearer to driver authors what > > if_hw_tsomaxsegcount should be > > set to, because I had it incorrect. > > > > Hopefully we can come up with a solution that everyone is comfortable with, > > rick > > > ok guys, > when you have some code for me to try just let me know. > > danny > > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-stable@freebsd.org Thu Aug 20 02:30:37 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8FE929BD6AD; Thu, 20 Aug 2015 02:30:37 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-pd0-x22f.google.com (mail-pd0-x22f.google.com [IPv6:2607:f8b0:400e:c02::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5650E1D29; Thu, 20 Aug 2015 02:30:37 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: by pdbmi9 with SMTP id mi9so8247316pdb.3; Wed, 19 Aug 2015 19:30:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=eQTimaEialIMMPKihgRhyaqLUInJ7JYcDtq5NJmch/I=; b=JRe0lBHMhWSn+1p95Bnf9eLM6MkepZq9Bqaiz42vniZTCxj4UXD3CsRnS6HxO5P4eA js3PXcW32KC0MX59qkX4T5nmuMnFXOLHrP1wpaqtJUqIKgZUJDArcg5u5Vj7bl3lroJY AkTNO6Ledl6HE8OkwLdbz6HIZ1tCBPsQM+rZKhtPhY0broHi3Ckrcwi2DZfzU3EZHR8I c8c7K8o7DLxr9mZwfjmHlao5mOVJIIEXArzZ+SKWR/GTPZMzV8mVsHf9Lj+xrPamcwjx Ize2seGECskFd5LcGGo7gqJchVN8Vfpg+qDVUA3yXy0WR3R1kUq+hQJ4wpt/ETh9svbh Tiqg== X-Received: by 10.70.103.74 with SMTP id fu10mr1578826pdb.11.1440037836738; Wed, 19 Aug 2015 19:30:36 -0700 (PDT) Received: from pyunyh@gmail.com ([106.247.248.2]) by smtp.gmail.com with ESMTPSA id em1sm2330892pbd.42.2015.08.19.19.30.31 (version=TLSv1 cipher=RC4-SHA bits=128/128); Wed, 19 Aug 2015 19:30:35 -0700 (PDT) From: Yonghyeon PYUN X-Google-Original-From: "Yonghyeon PYUN" Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Thu, 20 Aug 2015 11:30:24 +0900 Date: Thu, 20 Aug 2015 11:30:24 +0900 To: Rick Macklem Cc: Hans Petter Selasky , FreeBSD stable , FreeBSD Net , Slawa Olhovchenkov , Christopher Forgeron , Daniel Braniss , Gleb Smirnoff Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance Message-ID: <20150820023024.GB996@michelle.fasterthan.com> Reply-To: pyunyh@gmail.com References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <9D8B0503-E8FA-43CA-88F0-01F184F84D9B@cs.huji.ac.il> <1721122651.24481798.1439902381663.JavaMail.zimbra@uoguelph.ca> <55D333D6.5040102@selasky.org> <1325951625.25292515.1439934848268.JavaMail.zimbra@uoguelph.ca> <55D429A4.3010407@selasky.org> <20150819074212.GB964@michelle.fasterthan.com> <55D43615.1030401@selasky.org> <2013503980.25726607.1439989235806.JavaMail.zimbra@uoguelph.ca> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2013503980.25726607.1439989235806.JavaMail.zimbra@uoguelph.ca> User-Agent: Mutt/1.4.2.3i X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2015 02:30:37 -0000 On Wed, Aug 19, 2015 at 09:00:35AM -0400, Rick Macklem wrote: > Hans Petter Selasky wrote: > > On 08/19/15 09:42, Yonghyeon PYUN wrote: > > > On Wed, Aug 19, 2015 at 09:00:52AM +0200, Hans Petter Selasky wrote: > > >> On 08/18/15 23:54, Rick Macklem wrote: > > >>> Ouch! Yes, I now see that the code that counts the # of mbufs is before > > >>> the > > >>> code that adds the tcp/ip header mbuf. > > >>> > > >>> In my opinion, this should be fixed by setting if_hw_tsomaxsegcount to > > >>> whatever > > >>> the driver provides - 1. It is not the driver's responsibility to know if > > >>> a tcp/ip > > >>> header mbuf will be added and is a lot less confusing that expecting the > > >>> driver > > >>> author to know to subtract one. (I had mistakenly thought that > > >>> tcp_output() had > > >>> added the tc/ip header mbuf before the loop that counts mbufs in the > > >>> list. > > >>> Btw, > > >>> this tcp/ip header mbuf also has leading space for the MAC layer header.) > > >>> > > >> > > >> Hi Rick, > > >> > > >> Your question is good. With the Mellanox hardware we have separate > > >> so-called inline data space for the TCP/IP headers, so if the TCP stack > > >> subtracts something, then we would need to add something to the limit, > > >> because then the scatter gather list is only used for the data part. > > >> > > > > > > I think all drivers in tree don't subtract 1 for > > > if_hw_tsomaxsegcount. Probably touching Mellanox driver would be > > > simpler than fixing all other drivers in tree. > > > > > >> Maybe it can be controlled by some kind of flag, if all the three TSO > > >> limits should include the TCP/IP/ethernet headers too. I'm pretty sure > > >> we want both versions. > > >> > > > > > > Hmm, I'm afraid it's already complex. Drivers have to tell almost > > > the same information to both bus_dma(9) and network stack. > > > > Don't forget that not all drivers in the tree set the TSO limits before > > if_attach(), so possibly the subtraction of one TSO fragment needs to go > > into ip_output() .... > > > Ok, I realized that some drivers may not know the answers before ether_ifattach(), > due to the way they are configured/written (I saw the use of if_hw_tsomax_update() > in the patch). I was not able to find an interface that configures TSO parameters after if_t conversion. I'm under the impression if_hw_tsomax_update() is not designed to use this way. Probably we need a better one?(CCed to Gleb). > > If it is subtracted as a part of the assignment to if_hw_tsomaxsegcount in tcp_output() > at line#791 in tcp_output() like the following, I don't think it should matter if the > values are set before ether_ifattach()? > /* > * Subtract 1 for the tcp/ip header mbuf that > * will be prepended to the mbuf chain in this > * function in the code below this block. > */ > if_hw_tsomaxsegcount = tp->t_tsomaxsegcount - 1; > > I don't have a good solution for the case where a driver doesn't plan on using the > tcp/ip header provided by tcp_output() except to say the driver can add one to the > setting to compensate for that (and if they fail to do so, it still works, although > somewhat suboptimally). When I now read the comment in sys/net/if_var.h it is clear > what it means, but for some reason I didn't read it that way before? (I think it was > the part that said the driver didn't have to subtract for the headers that confused me?) > In any case, we need to try and come up with a clear definition of what they need to > be set to. > > I can now think of two ways to deal with this: > 1 - Leave tcp_output() as is, but provide a macro for the device driver authors to use > that sets if_hw_tsomaxsegcount with a flag for "driver uses tcp/ip header mbuf", > documenting that this flag should normally be true. > OR > 2 - Change tcp_output() as above, noting that this is a workaround for confusion w.r.t. > whether or not if_hw_tsomaxsegcount should include the tcp/ip header mbuf and > update the comment in if_var.h to reflect this. Then drivers that don't use the > tcp/ip header mbuf can increase their value for if_hw_tsomaxsegcount by 1. > (The comment should also mention that a value of 35 or greater is much preferred to > 32 if the hardware will support that.) > Both works for me. My preference is 2 just because it's very common for most drivers that use tcp/ip header mbuf. From owner-freebsd-stable@freebsd.org Thu Aug 20 04:51:39 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4D6FB9BEDEE; Thu, 20 Aug 2015 04:51:39 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-pd0-x22d.google.com (mail-pd0-x22d.google.com [IPv6:2607:f8b0:400e:c02::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1AF47E22; Thu, 20 Aug 2015 04:51:39 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: by pdbmi9 with SMTP id mi9so9547853pdb.3; Wed, 19 Aug 2015 21:51:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=IjJI9xKS/Ee/lQYdIJZfNB/0g0GYNXctkYnPkN380Kg=; b=f/DCgY01KHz4pNVqTRvx6LkT3cCpnOnx+vNiYHmtEZWPjjC20SHZCeuvP3GeJ1qi1N Qj4Y/zCILAujFnZfxwFs04LZco/Wgl1wywGNeGnjP3NjkviV9kKDmddqVF/hg72Hcd4G aJ8ZFz8wYr3YvgDu+Ui55zYCMjjr25/XHe9kYRNHcRfNtxS3Ehuzfsl2VJFDeewCSe4f 8xbZ1IzdARwpFubECeh3/SIYH5zIM9xQFpN2T7HRt3n0DDP/yhVN2ZplDBh8yOHzcVZj 0otr+GYB3UEZ4jIXGRWyYeMN9oiREkx2eKpZRnYK/an/yA+uFrhi3EoVHufnmnEXIEow Q3Ag== X-Received: by 10.70.90.98 with SMTP id bv2mr2684918pdb.36.1440046298588; Wed, 19 Aug 2015 21:51:38 -0700 (PDT) Received: from pyunyh@gmail.com ([106.247.248.2]) by smtp.gmail.com with ESMTPSA id xp10sm2637722pac.34.2015.08.19.21.51.33 (version=TLSv1 cipher=RC4-SHA bits=128/128); Wed, 19 Aug 2015 21:51:37 -0700 (PDT) From: Yonghyeon PYUN X-Google-Original-From: "Yonghyeon PYUN" Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Thu, 20 Aug 2015 13:51:25 +0900 Date: Thu, 20 Aug 2015 13:51:25 +0900 To: Rick Macklem Cc: Hans Petter Selasky , FreeBSD stable , FreeBSD Net , Slawa Olhovchenkov , Christopher Forgeron , Daniel Braniss Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance Message-ID: <20150820045125.GA982@michelle.fasterthan.com> Reply-To: pyunyh@gmail.com References: <473274181.23263108.1439814072514.JavaMail.zimbra@uoguelph.ca> <1721122651.24481798.1439902381663.JavaMail.zimbra@uoguelph.ca> <55D333D6.5040102@selasky.org> <1325951625.25292515.1439934848268.JavaMail.zimbra@uoguelph.ca> <55D429A4.3010407@selasky.org> <20150819074212.GB964@michelle.fasterthan.com> <55D43590.8050508@selasky.org> <20150819081308.GC964@michelle.fasterthan.com> <1154739904.25677089.1439986439408.JavaMail.zimbra@uoguelph.ca> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1154739904.25677089.1439986439408.JavaMail.zimbra@uoguelph.ca> User-Agent: Mutt/1.4.2.3i X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2015 04:51:39 -0000 On Wed, Aug 19, 2015 at 08:13:59AM -0400, Rick Macklem wrote: > Yonghyeon PYUN wrote: > > On Wed, Aug 19, 2015 at 09:51:44AM +0200, Hans Petter Selasky wrote: > > > On 08/19/15 09:42, Yonghyeon PYUN wrote: > > > >On Wed, Aug 19, 2015 at 09:00:52AM +0200, Hans Petter Selasky wrote: > > > >>On 08/18/15 23:54, Rick Macklem wrote: > > > >>>Ouch! Yes, I now see that the code that counts the # of mbufs is before > > > >>>the > > > >>>code that adds the tcp/ip header mbuf. > > > >>> > > > >>>In my opinion, this should be fixed by setting if_hw_tsomaxsegcount to > > > >>>whatever > > > >>>the driver provides - 1. It is not the driver's responsibility to know > > > >>>if > > > >>>a tcp/ip > > > >>>header mbuf will be added and is a lot less confusing that expecting the > > > >>>driver > > > >>>author to know to subtract one. (I had mistakenly thought that > > > >>>tcp_output() had > > > >>>added the tc/ip header mbuf before the loop that counts mbufs in the > > > >>>list. > > > >>>Btw, > > > >>>this tcp/ip header mbuf also has leading space for the MAC layer > > > >>>header.) > > > >>> > > > >> > > > >>Hi Rick, > > > >> > > > >>Your question is good. With the Mellanox hardware we have separate > > > >>so-called inline data space for the TCP/IP headers, so if the TCP stack > > > >>subtracts something, then we would need to add something to the limit, > > > >>because then the scatter gather list is only used for the data part. > > > >> > > > > > > > >I think all drivers in tree don't subtract 1 for > > > >if_hw_tsomaxsegcount. Probably touching Mellanox driver would be > > > >simpler than fixing all other drivers in tree. > > > > > > Hi, > > > > > > If you change the behaviour don't forget to update and/or add comments > > > describing it. Maybe the amount of subtraction could be defined by some > > > macro? Then drivers which inline the headers can subtract it? > > > > > > > I'm also ok with your suggestion. > > > > > Your suggestion is fine by me. > > > > > > > > The initial TSO limits were tried to be preserved, and I believe that > > > TSO limits never accounted for IP/TCP/ETHERNET/VLAN headers! > > > > > > > I guess FreeBSD used to follow MS LSOv1 specification with minor > > exception in pseudo checksum computation. If I recall correctly the > > specification says upper stack can generate up to IP_MAXPACKET sized > > packet. Other L2 headers like ethernet/vlan header size is not > > included in the packet and it's drivers responsibility to allocate > > additional DMA buffers/segments for L2 headers. > > > Yep. The default for if_hw_tsomax was reduced from IP_MAXPACKET to > 32 * MCLBYTES - max_ethernet_header_size as a workaround/hack so that > devices limited to 32 transmit segments would work (ie. the entire packet, > including MAC header would fit in 32 MCLBYTE clusters). > This implied that many drivers did end up using m_defrag() to copy the mbuf > list to one made up of 32 MCLBYTE clusters. > > If a driver sets if_hw_tsomaxsegcount correctly, then it can set if_hw_tsomax > to whatever it can handle as the largest TSO packet (without MAC header) the > hardware can handle. If it can handle > IP_MAXPACKET, then it can set it to that. > I thought the upper limit was still IP_MAXPACKET. If driver increase it (i.e. > IP_MAXPACKET, the length field in the IP header would overflow which in turn may break firewalls and other packet handling in IPv4/IPv6 code path. If the limit no longer apply to network stack, that's great. Some controllers can handle up to 256KB TCP/UDP segmentation and supporting that feature wouldn't be hard. From owner-freebsd-stable@freebsd.org Thu Aug 20 08:10:34 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8E3029BE82F; Thu, 20 Aug 2015 08:10:34 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebius.int.ru", Issuer "cell.glebius.int.ru" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 1386D1832; Thu, 20 Aug 2015 08:10:33 +0000 (UTC) (envelope-from glebius@FreeBSD.org) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.15.2/8.15.2) with ESMTPS id t7K8ADUE040211 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 20 Aug 2015 11:10:13 +0300 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.15.2/8.15.2/Submit) id t7K8ACxi040210; Thu, 20 Aug 2015 11:10:12 +0300 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Thu, 20 Aug 2015 11:10:12 +0300 From: Gleb Smirnoff To: Yonghyeon PYUN Cc: Rick Macklem , Hans Petter Selasky , FreeBSD stable , FreeBSD Net , Slawa Olhovchenkov , Christopher Forgeron , Daniel Braniss Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance Message-ID: <20150820081012.GY75813@glebius.int.ru> References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <9D8B0503-E8FA-43CA-88F0-01F184F84D9B@cs.huji.ac.il> <1721122651.24481798.1439902381663.JavaMail.zimbra@uoguelph.ca> <55D333D6.5040102@selasky.org> <1325951625.25292515.1439934848268.JavaMail.zimbra@uoguelph.ca> <55D429A4.3010407@selasky.org> <20150819074212.GB964@michelle.fasterthan.com> <55D43615.1030401@selasky.org> <2013503980.25726607.1439989235806.JavaMail.zimbra@uoguelph.ca> <20150820023024.GB996@michelle.fasterthan.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150820023024.GB996@michelle.fasterthan.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2015 08:10:34 -0000 Yonghyeon, On Thu, Aug 20, 2015 at 11:30:24AM +0900, Yonghyeon PYUN wrote: Y> > > >> Maybe it can be controlled by some kind of flag, if all the three TSO Y> > > >> limits should include the TCP/IP/ethernet headers too. I'm pretty sure Y> > > >> we want both versions. Y> > > >> Y> > > > Y> > > > Hmm, I'm afraid it's already complex. Drivers have to tell almost Y> > > > the same information to both bus_dma(9) and network stack. Y> > > Y> > > Don't forget that not all drivers in the tree set the TSO limits before Y> > > if_attach(), so possibly the subtraction of one TSO fragment needs to go Y> > > into ip_output() .... Y> > > Y> > Ok, I realized that some drivers may not know the answers before ether_ifattach(), Y> > due to the way they are configured/written (I saw the use of if_hw_tsomax_update() Y> > in the patch). Y> Y> I was not able to find an interface that configures TSO parameters Y> after if_t conversion. I'm under the impression Y> if_hw_tsomax_update() is not designed to use this way. Probably we Y> need a better one?(CCed to Gleb). Yes. In the projects/ifnet all the TSO stuff is configured differently. I'd really appreciate if other developers look there and review it, try it, give some input. Here is a snippet from net/if.h in projects/ifnet: /* * Structure describing TSO properties of an interface. Known both to ifnet * layer and TCP. Most interfaces point to a static tsomax in ifdriver * definition. However, vlan(4) and lagg(4) require a dynamic tsomax. */ struct iftsomax { uint32_t tsomax_bytes; /* TSO total burst length limit in bytes */ uint32_t tsomax_segcount; /* TSO maximum segment count */ uint32_t tsomax_segsize; /* TSO maximum segment size in bytes */ }; Now closer to your original question. I haven't yet converted lagg(4), so haven't yet worked on if_hw_tsomax_update(). I am convinced that it shouldn't be needed for a regular driver (save lagg(4). A proper driver should first study its hardware and only then call if_attach(). Correct me if am wrong, please. Also, I suppose, that a piece of hardware can't change its TSO maximums at runtime, so I don't see reason for changing them at runtime (save lagg(4)). -- Totus tuus, Glebius. From owner-freebsd-stable@freebsd.org Thu Aug 20 09:51:59 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 237DE9BC3CA for ; Thu, 20 Aug 2015 09:51:59 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-lb0-f175.google.com (mail-lb0-f175.google.com [209.85.217.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 86C571680 for ; Thu, 20 Aug 2015 09:51:57 +0000 (UTC) (envelope-from ml@my.gd) Received: by lbbpu9 with SMTP id pu9so20275267lbb.3 for ; Thu, 20 Aug 2015 02:51:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=0BhzE2v8+GW2BrMqx3Z1WO9wnTU0OQsp60bTu0PjyUQ=; b=Ca2fkPLk39oXpmcmmjfrPJBUKVnXZuIJ+qUBxnTRkyGlgUW7dSpYT+xxj44EaEGYHI Z2fRbSDa4a9T1I6NEPhfmqkVTRmmzl9E+b9N+AcClWV+PfxgFIuq3Mrh8SMo7Sxd7WGx JEvFL0luGsA6EUA7s6tXMVuzO9bzTYRZyqqm8Aba14Ib/anMdpgYHpNUwDVdK5v4xrir 8RvQy2gsmVkAmxxOSu4Xu4lRG/PFyXMFzXtHHCpk6nbNaS4rjnKjKHvbgE7sAhCt/91j GVyADiQpg12vnFpDHVuXTr/50+ZEBUfmrf6jCjjdX+4pPSaUu5HwHvCmdLWCT/2tO3fs RxaQ== X-Gm-Message-State: ALoCoQnY5weOVfeB98RF1ZLrclRVJ4v9Y+hUvkopZiQU0sfDQ8/A7k0/8SYbFnLmRCMPiboYgdT1 MIME-Version: 1.0 X-Received: by 10.152.87.116 with SMTP id w20mr2108399laz.119.1440064309859; Thu, 20 Aug 2015 02:51:49 -0700 (PDT) Received: by 10.112.60.34 with HTTP; Thu, 20 Aug 2015 02:51:49 -0700 (PDT) In-Reply-To: References: Date: Thu, 20 Aug 2015 11:51:49 +0200 Message-ID: Subject: Re: [POSSIBLE BUG] 10-STABLE CARP erroneously becomes master on boot From: Damien Fleuriot To: FreeBSD Stable Cc: Damien Fleuriot , Freddie Cash Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2015 09:51:59 -0000 On 19 August 2015 at 18:20, Damien Fleuriot wrote: > > > On 19 August 2015 at 11:29, Damien Fleuriot wrote: > >> Hello list, Freddie, >> >> >> I've been able to run extensive tests, the results of which I'm pasting >> below. >> >> >> First of all, I've been unable to replicate the problem in our >> preproduction and QA environments. >> >> The differences between our production and pre/QA envs are as follows : >> - we use link aggregation and VLAN tagging in production , we cannot >> replicate this in pre/QA because of limitations with KVM guests. >> - we use multiple CARP addresses with the same VHID on our public VLAN in >> production, we COULD replicate that in pre/QA if required. >> >> >> The context remains the following : >> - host A is supposed to be CARP MASTER, has advskew 20 and preempt >> - host B is supposed to be CARP BACKUP, has advskew 150 and preempt >> - host B assumes mastership if the CARPs are configured from rc.conf , >> doesn't if they're set up manually after boot >> >> Note that these 2 boxes were upgraded from 8-STABLE to 10-STABLE. >> Host A runs 10.2-BETA1 >> Host B runs 10.2-PRERELEASE >> >> I used the exact same versions in our pre/QA environments (one BETA1, one >> PRERELEASE from the same build) and couldn't replicate the issue. >> >> >> >> Now, on to the tests themselves. >> >> >> A/ Create a new CARP address with a new VHID, configure it in rc.conf and >> see if we get double MASTERS >> - on Host A : CARP created manually >> - on Host B : CARP created manually >> Host A is MASTER and Host B is BACKUP >> >> - on Host B : setup the new CARP in rc.conf , reboot >> Host A is MASTER and Host B is BACKUP , problem not replicated >> >> >> B/ Try with only one of our production addresses >> - on Host B : uncomment the production CARP address from rc.conf , reboot >> Host A is MASTER and Host B is MASTER >> Host A shows net.inet.carp.demotion=0 >> Host B shows net.inet.carp.demotion=240 >> >> >> C/ Try with the new CARP address + one of our production addresses , >> different VHIDs >> - on Host B : uncomment the new CARP address from rc.conf , reboot >> Host A is MASTER and Host B is MASTER >> Host A shows net.inet.carp.demotion=0 >> Host B shows net.inet.carp.demotion=240 >> >> >> D/ Try the new syntax in rc.conf , as per Freddie's suggestion >> - on Host B : change the rc.conf syntax , reboot >> Host A is MASTER and Host B is MASTER >> Host A shows net.inet.carp.demotion=0 >> Host B shows net.inet.carp.demotion=240 >> >> >> E/ Try, just for the sake of it, to remove old files and libs on host B >> - on Host B : cd /usr/src ; yes | make delete-old ; yes | make >> delete-old-libs ; reboot >> Host A is MASTER and Host B is MASTER >> Host A shows net.inet.carp.demotion=0 >> Host B shows net.inet.carp.demotion=240 >> >> >> F/ Edit sysctls to disable CARP demotion on advertisement send errors >> - on Host A : sysctl net.inet.carp.senderr_demotion_factor=0 >> - on Host B : set "net.inet.carp.senderr_demotion_factor=0" in >> sysctl.conf , reboot >> Host A is MASTER and Host B is MASTER >> Host A shows net.inet.carp.demotion=0 >> Host B shows net.inet.carp.demotion=240 >> >> >> >> Now after this F/ test, I'm thinking there's some voodoo going on here >> and it sure shows up : >> - on Host A, 'pfctl -si' shows 3k states >> - on Host B, 'pfctl -si' shows 800 states >> >> Now that would explain why my CARP gets demoted on Host B (as per man 4 >> carp, pfsync failures result in a -240 demotion). >> It doesn't, however, explain why the demoted CARP chooses to remain in a >> MASTER state, or assumed MASTERship in the first place. >> >> Surely enough, I can find some CARP errors in-between my reboots : >> messages:2420:2015-08-19T03:51:38.273600+00:00 pf1-gs kernel: carp: >> demoted by -240 to 0 (pfsync bulk fail) >> messages:2429:2015-08-19T03:56:37.178575+00:00 pf1-gs kernel: carp: VHID >> 110@vlan410: INIT -> BACKUP >> messages:2430:2015-08-19T03:56:40.664568+00:00 pf1-gs kernel: carp: VHID >> 110@vlan410: BACKUP -> MASTER (master down) >> messages:2637:2015-08-19T04:00:22.482071+00:00 pf1-gs kernel: carp: VHID >> 111@vlan410: BACKUP -> MASTER (master down) >> messages:2857:2015-08-19T04:04:02.330167+00:00 pf1-gs kernel: carp: VHID >> 110@vlan410: BACKUP -> MASTER (master down) >> messages:2877:2015-08-19T04:05:03.288199+00:00 pf1-gs kernel: carp: >> demoted by -240 to 0 (pfsync bulk fail) >> messages:3088:2015-08-19T04:08:48.961985+00:00 pf1-gs kernel: carp: VHID >> 110@vlan410: BACKUP -> MASTER (master down) >> >> >> Things I have not tested yet : >> - use /24 CARPs instead of /32s >> - switch my CARPs to all use different VHIDs >> >> Things I CANNOT test : >> - set up a *dedicated* pfsync link between the firewalls , they're in >> different DCs >> - set up a *dedicated* VLAN for pfsync , that would entail huge changes >> in our PCI-DSS environment >> >> >> After all these tests, I find myself in a situation where : >> - manually set up CARPs on Host B work fine , and pfsync works >> - CARPs from rc.conf on Host B result in MASTER-MASTER , and pfsync fails >> >> >> >> >> I must say I'm stuck here. >> The "master down" message is very confusing, when the firewalls *can* see >> each other. >> The "pfsync bulk fail" is rather interesting as well, since it doesn't >> occur when the CARPs are unconfigured, or set up manually without rc.conf. >> tcpdump -nei pflog0 does not show any dropped packet. >> PF is configured to "pass in quick" pfsync and CARP packets. >> >> >> >> I'm afraid a STFW hasn't helped overmuch here, although I'll try some >> more. >> >> >> If anyone's got a pointer, I'll bite. >> >> Cheers >> >> > > FWIW , additional testing shows that on Host A (10.2-BETA) CARP > advertisements are sent from the CARP IP itself. > As in, my Host A has physical IPs X and Y, but sends its CARP announces > from CARP IP Z. > > I have not been able to replicate this behaviour in our preproduction > environment where both -BETA and -PRERELEASE send the advertisements from > their physical IP. > These boxes however, have just the one IP, as opposed to the production > environment which has 2 physical IPs. > > > I will get some additional testing done, by swapping over to make the Host > B running -PRERELEASE master, and see if it also sends its CARP > announcements sourced from its physical IP (would be good) or the CARP > itself (would be bad). > Hello list, We've managed to find the source of the bug, if it is indeed a bug. It all comes down to the order in which the IP addresses are assigned to the interface from /etc/rc.conf. When using the following syntax, the physical IP address is configured AFTER the CARPs on the interface, which results in the CARP advertisements being sourced from the CARP IP, triggering the double MASTER situation : ipv4_addrs_int="1.2.3.4/24" ifconfig_int_alias0="1.2.3.6/32 vhid 1 pass test advskew 20" When using either of the following syntaxes, the physical IP address is configured BEFORE the CARPs, which results in the CARP advertisements being sourced from the physical IP and restoring normal functionality : ifconfig_int="inet 1.2.3.4/24" ifconfig_int_alias0="1.2.3.6/32 vhid 1 pass test advskew 20" OR ifconfig_int_alias0="1.2.3.4/24" ifconfig_int_alias1="1.2.3.6/32 vhid 1 pass test advskew 20" I cannot find any reference to this behaviour in UPDATING, I'm not even sure it's actually wanted. I'm going to update the boxes to the latest 10-STABLE to see if the problem still occurs, no point in submitting a bug for something that's fixed. > > > >> >> On 17 August 2015 at 18:38, Damien Fleuriot wrote: >> >>> >>> On 17 August 2015 at 18:32, Freddie Cash wrote: >>> >>>> >>>> On Aug 17, 2015 9:22 AM, "Damien Fleuriot" wrote: >>>> > >>>> > Hello list, >>>> > >>>> > >>>> > >>>> > I'm seeing this very peculiar behaviour between 2 10-STABLE boxes. >>>> > >>>> > Host A is CARP Master with advskew 20 and runs 10.2-BETA1 from 10/07 >>>> > Host B is CARP Backup with advskew 150 and runs 10.2-PRERELEASE from >>>> 12/08 >>>> > >>>> > >>>> > When I configure CARP in rc.conf on host B, it becomes Master on >>>> boot, and >>>> > host A remains Master as well. >>>> > When I force a state change on host B (ifconfig vlanx vhid y state >>>> backup), >>>> > it transitions to Backup then again to Master. >>>> > >>>> > When I comment out the CARP configuration in rc.conf , and configure >>>> CARP >>>> > manually on host B's interfaces after it boots, it correctly becomes >>>> and >>>> > remains Backup. >>>> > >>>> > >>>> > >>>> > Below is the excerpt from rc.conf pertaining to CARP configuration, >>>> the >>>> > only difference between the 2 hosts being their advskew. >>>> > >>>> > Host A >>>> > == BEGIN >>>> > >>>> > ifconfig_vlan410_alias0="vhid 110 pass passhere advskew 20 alias >>>> > 10.104.10.251/32" >>>> > >>>> > == END >>>> > >>>> > Host B >>>> > == BEGIN >>>> > >>>> > ifconfig_vlan410_alias0="vhid 110 pass passhere advskew 150 alias >>>> > 10.104.10.251/32" >>>> > >>>> > == END >>>> >>>> Put the IP first, and the vhid stuff last in rc.conf for things to work >>>> the most reliably. And drop the extra alias. >>>> >>>> ifconfig_vlan410_alias0="inet 10.104.10.251/32 vhid 110 pass passhere >>>> advskew 150" >>>> >>>> CARP requires that all IPs on an interface that are part of the same >>>> vhid to be listed (added) in the exact same order for the vhid to be >>>> considered "the same". That one trips me up all the time when manually >>>> adding an IP to a CARP pair, and then later rebooting one box as they both >>>> think they're master for that interface, while being a mix of master/backup >>>> for the other interfaces. >>>> >>>> Cheers, >>>> Freddie >>>> (running CARP on 2 10-CURRENT boxes and 2 10.1-p13 boxes) >>>> >>> >>> Cheers Freddie, will try and keep the thread up to date on the results. >>> >>> >> > From owner-freebsd-stable@freebsd.org Thu Aug 20 09:57:13 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5E4659BC53A for ; Thu, 20 Aug 2015 09:57:13 +0000 (UTC) (envelope-from matpockuh@gmail.com) Received: from mail-io0-x22b.google.com (mail-io0-x22b.google.com [IPv6:2607:f8b0:4001:c06::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 299601811 for ; Thu, 20 Aug 2015 09:57:13 +0000 (UTC) (envelope-from matpockuh@gmail.com) Received: by iodt126 with SMTP id t126so40128946iod.2 for ; Thu, 20 Aug 2015 02:57:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=s0UKRDJNzQ+YGNzZ9IFDXLw1JFpJrnkJk9w4CW2Sc6w=; b=Jpl9xvi5qNd3DmAzEa8Al4SFhOs8mW5OoVnMHQoyk2cXrWjKkoXImeox70rQVAKLM8 hiR9r/RTsPFm+0QIRBJpF+4MHP8qCROVVCKNFi/jz950YJRT3IL/7rEIneGfLyfy0D4z v8uewxubpbuoTscG3NpBbuGc2PeePzMt6aEWFB9KvInY9l99XoLpy7ZAg+6txGqSJBrK qQUv0fAK1WSXWNj7i2L9M6KAOlVB+5B+EOgSbToECphbMp+sg9h7qTI48VKJZ/QW1/Mx gZbKb3wbex3KLOw8+rDreP1jsQZ2GvPfbERQIvurRaqoHlDI0mEbbeOt4wFB1eVe71zH 8xRQ== MIME-Version: 1.0 X-Received: by 10.107.14.141 with SMTP id 135mr1560191ioo.49.1440064632520; Thu, 20 Aug 2015 02:57:12 -0700 (PDT) Received: by 10.79.38.3 with HTTP; Thu, 20 Aug 2015 02:57:12 -0700 (PDT) Date: Thu, 20 Aug 2015 12:57:12 +0300 Message-ID: Subject: Silent data corruption on em(4) interfaces From: KOT MATPOCKuH To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2015 09:57:13 -0000 Hello! I got silent data corruption when transferring data via em(4) interface on 10.2-STABLE r286912. 1. I got broken large file transferred via ftp (MD5 checksum mismatched); 2. I got disconnects when transferring large data via ssh with messages: Corrupted MAC on input. Disconnecting: Packet corrupt Problem occurs only after few hours of uptime. Immediately after reboot I transferred same file via ftp without any errors. I tried to use: - em0 and em2 interfaces in link aggregation - em1 as "clean" interface But I got same problem in both cases. Also one time when transferring file I got this messages: em0: Interface stopped DISTRIBUTING, possible flapping em0: Watchdog timeout -- resetting em2: Interface stopped DISTRIBUTING, possible flapping em2: Watchdog timeout -- resetting netstat -in does not see any problems: Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll em0 1500 00:14:4f:01:3f:7a 6689452 0 0 146720 0 0 em1 1500 00:14:4f:01:3f:7b 5732168 0 0 2865912 0 0 em2 1500 00:14:4f:01:3f:7c 501817 0 0 3392333 0 0 Network adapters is build in to the Sun Fire X4100 mother board: em0@pci0:1:1:0: class=0x020000 card=0x10118086 chip=0x10108086 rev=0x03 hdr=0x00 vendor = 'Intel Corporation' device = '82546EB Gigabit Ethernet Controller (Copper)' class = network subclass = ethernet TCP_OFFLOAD disabled in kernel's config. Any ideas? -- MATPOCKuH From owner-freebsd-stable@freebsd.org Thu Aug 20 10:29:54 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3BD699BCD23; Thu, 20 Aug 2015 10:29:54 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (jenkins-9.freebsd.org [8.8.178.209]) by mx1.freebsd.org (Postfix) with ESMTP id 29057884; Thu, 20 Aug 2015 10:29:54 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (localhost [127.0.0.1]) by jenkins-9.freebsd.org (Postfix) with ESMTP id 9C0CD528; Thu, 20 Aug 2015 10:29:54 +0000 (UTC) Date: Thu, 20 Aug 2015 10:29:51 +0000 (GMT) From: jenkins-admin@FreeBSD.org To: ed@FreeBSD.org, jenkins-admin@FreeBSD.org, freebsd-stable@FreeBSD.org, freebsd-i386@FreeBSD.org Message-ID: <2072979596.36.1440066594082.JavaMail.jenkins@jenkins-9.freebsd.org> In-Reply-To: <1906952481.23.1440014493257.JavaMail.jenkins@jenkins-9.freebsd.org> References: <1906952481.23.1440014493257.JavaMail.jenkins@jenkins-9.freebsd.org> Subject: FreeBSD_STABLE_10-i386 - Build #385 - Fixed MIME-Version: 1.0 X-Jenkins-Job: FreeBSD_STABLE_10-i386 X-Jenkins-Result: SUCCESS Precedence: bulk Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2015 10:29:54 -0000 FreeBSD_STABLE_10-i386 - Build #385 - Fixed: Build information: https://jenkins.FreeBSD.org/job/FreeBSD_STABLE_10-i386/385/ Full change log: https://jenkins.FreeBSD.org/job/FreeBSD_STABLE_10-i386/385/changes Full build log: https://jenkins.FreeBSD.org/job/FreeBSD_STABLE_10-i386/385/console Change summaries: 286952 by ed: MFC r285742: Unbreak "last reboot". According to the last(1) man page, the "reboot" pseudo-user should print all system reboot entries. This got broken by the utmpx import, as records are typed. Re-add support for "last reboot" by specifically matching against SHUTDOWN_TIME and BOOT_TIME records. PR: 168844 Submitted by: matthew@ From owner-freebsd-stable@freebsd.org Thu Aug 20 11:24:04 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 98AE69BDE59 for ; Thu, 20 Aug 2015 11:24:04 +0000 (UTC) (envelope-from marko.cupac@mimar.rs) Received: from smtp.mimar.rs (smtp.mimar.rs [193.53.106.135]) by mx1.freebsd.org (Postfix) with ESMTP id 4D446B78 for ; Thu, 20 Aug 2015 11:24:04 +0000 (UTC) (envelope-from marko.cupac@mimar.rs) Received: from vscan.mimar.rs (vscan.mimar.rs [193.53.106.134]) by smtp.mimar.rs (Postfix) with ESMTP id A353989901 for ; Thu, 20 Aug 2015 13:23:56 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=mimar.rs; h= content-transfer-encoding:content-type:content-type:mime-version :x-mailer:organization:message-id:subject:subject:from:from:date :date:received:received; s=mimar-0901; t=1440069836; x= 1441884237; bh=CUNbpPrHN3zofRWgK0xVYS+kLVTfBdUymn9ilnIssDo=; b=V fgl1sS8GTwOzsV/P4E2ZEtOebfY1ra5zq6Yv9XsBqE/h1sjKHvbWN+i92fsorsvw UAJSPnSp/VG/Zd01GBtHDDmI1A0D+HrSPpA49DP1zYdIhosr3NI+91GltrhY+X1i V6GnYCWxg0E+SxHKfWexS6/UN6dRJP5za1ag/mLyw4= X-Virus-Scanned: amavisd-new at mimar.rs Received: from smtp.mimar.rs ([193.53.106.135]) by vscan.mimar.rs (vscan.mimar.rs [193.53.106.134]) (amavisd-new, port 10026) with ESMTP id zRiMT3Jbdh7f for ; Thu, 20 Aug 2015 13:23:56 +0200 (CEST) Received: from efreet (unknown [193.53.106.34]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: marko.cupac@mimar.rs) by smtp.mimar.rs (Postfix) with ESMTPSA id BBF49898CB for ; Thu, 20 Aug 2015 13:23:55 +0200 (CEST) Date: Thu, 20 Aug 2015 13:23:53 +0200 From: Marko =?UTF-8?B?Q3VwYcSH?= To: freebsd-stable@FreeBSD.org Subject: vmx and esxi vst mode vlan tagging Message-ID: <20150820132353.2ae19568@efreet> Organization: mimar X-Mailer: Claws Mail 3.9.3 (GTK+ 2.24.23; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2015 11:24:04 -0000 Hi, I have just spent half an hour bashing my head against the wall why my FreeBSD-10.2-RELEASE i386 machine with kernel-included vmx driver and emulators/open-vm-tools installed won't ping machines on the other vlan on the same vSwitch. It DOES ping other virtual machines on the same vSwitch and same vlan (freebsd servers), and it DOES ping physical machines in other vlans. It DOES NOT ping virtual machines on the same vSwitch and different vlan (windows servers). To cut the long story short, it appears that the problem is on FreeBSD side, as another machine (FreeBSD-10.1-RELEASE-p14 amd 64) with vmx3f0 driver and VMWare's vmware-tools installed pings everything as it should. Could it be that FreeBSD's vmx driver does not support all the functions of vmware's vmx3f driver? Regards, --=20 Marko Cupa=C4=87 https://www.mimar.rs/ From owner-freebsd-stable@freebsd.org Thu Aug 20 12:03:49 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C3F769BED08; Thu, 20 Aug 2015 12:03:49 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 4C89C159; Thu, 20 Aug 2015 12:03:48 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:c4mtjRVinxux5MmOj2cbwepSweHV8LGtZVwlr6E/grcLSJyIuqrYZhGAt8tkgFKBZ4jH8fUM07OQ6PC7HzBQqsvZ+Fk5M7VyFDY9wf0MmAIhBMPXQWbaF9XNKxIAIcJZSVV+9Gu6O0UGUOz3ZlnVv2HgpWVKQka3CwN5K6zPF5LIiIzvjqbpq8aVP1UD2WL1SIgxBSv1hD2ZjtMRj4pmJ/R54TryiVwMRd5rw3h1L0mYhRf265T41pdi9yNNp6BprJYYAu3SNp41Rr1ADTkgL3t9pIiy7UGCHkOz4S4EQ3gQgxpgDA3M7RW8VZD04QXgse8o4iiRPoXTRLs3XTmnp/NxTRbjiyMKMhYk927Kh8hojORQqUTy9FRE34fIbdTNZ7JFdaTHcIZfHDIZUw== X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2BYAgBLwdVV/61jaINdg29pBoMfui0BCYFtCoUxSgKBaBQBAQEBAQEBAYEJgh2CBgEBAQMBAQEBICsgCwULAgEIGAICDRkCAicBCSYCDAcEARwEiAUIDbkclgQBAQEBAQEBAwEBAQEBGQSBIooxhDEBBgEBHDQHgmmBQwWVKIUFhQiELJA/hEiDZwImgg4cgW8iMwd+AQgXI4EEAQEB X-IronPort-AV: E=Sophos;i="5.15,714,1432612800"; d="scan'208";a="233568830" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-annu.net.uoguelph.ca with ESMTP; 20 Aug 2015 08:03:41 -0400 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 039F715F55D; Thu, 20 Aug 2015 08:03:41 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id ydQ8_x38jF7C; Thu, 20 Aug 2015 08:03:40 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 2012815F563; Thu, 20 Aug 2015 08:03:40 -0400 (EDT) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id QiqjKPhbu5_J; Thu, 20 Aug 2015 08:03:39 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id EA2C315F55D; Thu, 20 Aug 2015 08:03:39 -0400 (EDT) Date: Thu, 20 Aug 2015 08:03:39 -0400 (EDT) From: Rick Macklem To: pyunyh@gmail.com Cc: Hans Petter Selasky , FreeBSD stable , FreeBSD Net , Slawa Olhovchenkov , Christopher Forgeron Message-ID: <1935256446.26896702.1440072219573.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <20150820045125.GA982@michelle.fasterthan.com> References: <473274181.23263108.1439814072514.JavaMail.zimbra@uoguelph.ca> <1325951625.25292515.1439934848268.JavaMail.zimbra@uoguelph.ca> <55D429A4.3010407@selasky.org> <20150819074212.GB964@michelle.fasterthan.com> <55D43590.8050508@selasky.org> <20150819081308.GC964@michelle.fasterthan.com> <1154739904.25677089.1439986439408.JavaMail.zimbra@uoguelph.ca> <20150820045125.GA982@michelle.fasterthan.com> Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.11] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF34 (Win)/8.0.9_GA_6191) Thread-Topic: ix(intel) vs mlxen(mellanox) 10Gb performance Thread-Index: rwJq7qWbysh2G9XrT4qz/e9qa1lX4w== X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2015 12:03:50 -0000 Yonghyeon PYUN wrote: > On Wed, Aug 19, 2015 at 08:13:59AM -0400, Rick Macklem wrote: > > Yonghyeon PYUN wrote: > > > On Wed, Aug 19, 2015 at 09:51:44AM +0200, Hans Petter Selasky wrote: > > > > On 08/19/15 09:42, Yonghyeon PYUN wrote: > > > > >On Wed, Aug 19, 2015 at 09:00:52AM +0200, Hans Petter Selasky wrote: > > > > >>On 08/18/15 23:54, Rick Macklem wrote: > > > > >>>Ouch! Yes, I now see that the code that counts the # of mbufs is > > > > >>>before > > > > >>>the > > > > >>>code that adds the tcp/ip header mbuf. > > > > >>> > > > > >>>In my opinion, this should be fixed by setting if_hw_tsomaxsegcount > > > > >>>to > > > > >>>whatever > > > > >>>the driver provides - 1. It is not the driver's responsibility to > > > > >>>know > > > > >>>if > > > > >>>a tcp/ip > > > > >>>header mbuf will be added and is a lot less confusing that expecting > > > > >>>the > > > > >>>driver > > > > >>>author to know to subtract one. (I had mistakenly thought that > > > > >>>tcp_output() had > > > > >>>added the tc/ip header mbuf before the loop that counts mbufs in the > > > > >>>list. > > > > >>>Btw, > > > > >>>this tcp/ip header mbuf also has leading space for the MAC layer > > > > >>>header.) > > > > >>> > > > > >> > > > > >>Hi Rick, > > > > >> > > > > >>Your question is good. With the Mellanox hardware we have separate > > > > >>so-called inline data space for the TCP/IP headers, so if the TCP > > > > >>stack > > > > >>subtracts something, then we would need to add something to the > > > > >>limit, > > > > >>because then the scatter gather list is only used for the data part. > > > > >> > > > > > > > > > >I think all drivers in tree don't subtract 1 for > > > > >if_hw_tsomaxsegcount. Probably touching Mellanox driver would be > > > > >simpler than fixing all other drivers in tree. > > > > > > > > Hi, > > > > > > > > If you change the behaviour don't forget to update and/or add comments > > > > describing it. Maybe the amount of subtraction could be defined by some > > > > macro? Then drivers which inline the headers can subtract it? > > > > > > > > > > I'm also ok with your suggestion. > > > > > > > Your suggestion is fine by me. > > > > > > > > > > > The initial TSO limits were tried to be preserved, and I believe that > > > > TSO limits never accounted for IP/TCP/ETHERNET/VLAN headers! > > > > > > > > > > I guess FreeBSD used to follow MS LSOv1 specification with minor > > > exception in pseudo checksum computation. If I recall correctly the > > > specification says upper stack can generate up to IP_MAXPACKET sized > > > packet. Other L2 headers like ethernet/vlan header size is not > > > included in the packet and it's drivers responsibility to allocate > > > additional DMA buffers/segments for L2 headers. > > > > > Yep. The default for if_hw_tsomax was reduced from IP_MAXPACKET to > > 32 * MCLBYTES - max_ethernet_header_size as a workaround/hack so that > > devices limited to 32 transmit segments would work (ie. the entire packet, > > including MAC header would fit in 32 MCLBYTE clusters). > > This implied that many drivers did end up using m_defrag() to copy the mbuf > > list to one made up of 32 MCLBYTE clusters. > > > > If a driver sets if_hw_tsomaxsegcount correctly, then it can set > > if_hw_tsomax > > to whatever it can handle as the largest TSO packet (without MAC header) > > the > > hardware can handle. If it can handle > IP_MAXPACKET, then it can set it to > > that. > > > > I thought the upper limit was still IP_MAXPACKET. If driver > increase it (i.e. > IP_MAXPACKET, the length field in the IP > header would overflow which in turn may break firewalls and other > packet handling in IPv4/IPv6 code path. I have no idea if a bogus value in the ip_len field of the TSO segment would break something in ip_output() or not. This would need to be checked before anyone configures if_hw_tsomax > IP_MAXPACKET. I didn't think of any effect this would have in ip_output(), I just knew that the hardware would be replacing ip_len when it generated the TCP/IP segments from the TSO segment. As you note, I vaguely recall some hardware being able to handle a TSO segment > IP_MAXPACKET (presumably getting the TSO segment's length some other way). It would be nice if this was checked, but yes, the comment should specify an upper bound on if_hw_tsomax of IP_MAXPACKET until then. rick > If the limit no longer apply to network stack, that's great. Some > controllers can handle up to 256KB TCP/UDP segmentation and > supporting that feature wouldn't be hard. > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > From owner-freebsd-stable@freebsd.org Thu Aug 20 22:34:28 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9E2DE9BEA5C for ; Thu, 20 Aug 2015 22:34:28 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from erouter6.ore.mailhop.org (erouter6.ore.mailhop.org [54.187.213.119]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 74ED4181C for ; Thu, 20 Aug 2015 22:34:28 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from ilsoft.org (unknown [73.34.117.227]) by outbound3.ore.mailhop.org (Halon Mail Gateway) with ESMTPSA; Thu, 20 Aug 2015 22:31:17 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.14.9/8.14.9) with ESMTP id t7KMYKc7007279; Thu, 20 Aug 2015 16:34:20 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: <1440110060.242.252.camel@freebsd.org> Subject: Re: Will 10.2 also ship with a very stale NTP? From: Ian Lepore To: Harald Schmalzbauer Cc: freebsd-stable@freebsd.org Date: Thu, 20 Aug 2015 16:34:20 -0600 In-Reply-To: <55B23B4E.1080400@omnilan.de> References: <20150710235810.GA76134@rwpc16.gfn.riverwillow.net.au> <20150712032256.GB19305@satori.lan> <20150712050443.GA22240@server.rulingia.com> <20150712154416.b9f3713893fe28bfab1dd4d7@dec.sakura.ne.jp> <20150712184910.2d8d5f085ae659d5b9a2aba0@dec.sakura.ne.jp> <1436715703.1334.193.camel@freebsd.org> <55B23B4E.1080400@omnilan.de> Content-Type: text/plain; charset="iso-2022-jp" X-Mailer: Evolution 3.12.10 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2015 22:34:28 -0000 On Fri, 2015-07-24 at 15:19 +0200, Harald Schmalzbauer wrote: > Bezglich Ian Lepore's Nachricht vom 12.07.2015 17:41 (localtime): > > And let's all just hope that a week or two of testing is enough when > > jumping a major piece of software forward several years in its > > independent evolution. > $B!D(B > > I wonder how many other such things could be lurking in 4.2.8, waiting > > to be triggered by other peoples' non-stock configurations? We've > $B!D(B > > I'd like to report one, most likely an upstream problem: > > 'restrict' definitions in ntp.conf(5) no longer work with unqualified DNS names. > A line like > "restrict time1 nomodify nopeer noquery notrap" > results in: > ntpd[1913]: line 7 column 7 syntax error, unexpected T_Time1 > ntpd[1913]: syntax error in /etc/ntp.conf line 7, column 7 > > I've always been using unqualified hostnames with 'restrict', and since defining 'server' with unqualified hostname still works, this seems to be a significant bug to me. People are forced to change 'restrict' definitions, but not to also change other unqualified definitions, which potentially leads to misconfigurations, since intentionally matching definitions can now differ easily. > > Has anybody already noticed this problem? And any idea if upstream is aware? I had a quick look at this today. It appears that the problem isn't unqualified names exactly, but rather an unqualified name that exactly matches an ntp.conf keyword will be mistaken by the ntpd config parser as a misplaced keyword token. So most unqualified names should work, but there are about 200 words that won't, many of them very sensible names for ntp servers such as "ntp" and "time1" and "time2". When I look at the ntp_parser.y grammar file it's not clear to me why "server time1" works and "restrict time1" doesn't. I couldn't find any way to trick it into taking a keyword as a hostname following restrict (like using quotes). You might be able to work around it using the new "restrict source" syntax that applies the restrictions to every server association that doesn't have a more-explicit matching restrict line. -- Ian From owner-freebsd-stable@freebsd.org Fri Aug 21 06:47:45 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 800BC9BD0A7; Fri, 21 Aug 2015 06:47:45 +0000 (UTC) (envelope-from andreas@naund.org) Received: from naund.org (172-11-194-172.lightspeed.sntcca.sbcglobal.net [172.11.194.172]) by mx1.freebsd.org (Postfix) with ESMTP id 58171E34; Fri, 21 Aug 2015 06:47:44 +0000 (UTC) (envelope-from andreas@naund.org) Received: (from andreas@localhost) by naund.org (8.11.6/8.11.6-20030329ao) id t7L6kvL23753; Thu, 20 Aug 2015 23:46:57 -0700 Date: Thu, 20 Aug 2015 23:46:57 -0700 From: Andreas Ott To: Glen Barber Cc: Slawa Olhovchenkov , Christian Kratzer , freebsd-stable@freebsd.org, FreeBSD Security Team Subject: Re: freebsd-update to 10.2-RELEASE broken ? Message-ID: <20150820234657.A23228@naund.org> References: <20150817155022.GD3158@zxy.spb.ru> <20150817155434.GT24069@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <20150817155434.GT24069@FreeBSD.org>; from gjb@freebsd.org on Mon, Aug 17, 2015 at 03:54:34PM +0000 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Aug 2015 06:47:45 -0000 Hi, On Mon, Aug 17, 2015 at 03:54:34PM +0000, Glen Barber wrote: [...] > Secteam. I've cc'd them. the issue persists even when forcing to a single update server, update2.freebsd.org is very close to this server. The DNS (?) response of "Looking up update2.freebsd.org mirrors... none found" is also still there. I end up with files where name and hash don't match. It appears to be an issue how the filename is generated from the hash, while the fact that the file can be unzipped from .gz format tells me it is not really corrupted. Or perhaps, how the gzip compression gets handled on small files, with certain content and padding. For many files, the SHA256 over the ascii content after gunzip is equal to the filename. This is not the case on the files that are flagged as mismatch. I have not looked at the code, but I think it will exit after the first mismatch, even if there would be more mismatched files/checksums. This server is starting from 10.1-RELEASE-p18, fully updated. I removed all files in /var/db/freebsd-update/* , rebooted, then ran freebsd-update fetch again, and got the meta files. I observe, that when running the freebsd-update "upgrade" again after the first failure, I end up with less patches, less downloads, presumably because a large portion got patched in the previous round, but the hash issue exists on a different file. I did a simple checksum verification on the 809 *.gz files after the second run # for f in `ls *gz`; do ls -la $f; echo $f; gunzip -c $f |sha256; done and the output is deposited here: https://files.naund.org/andreas/freebsd-update-SHA256-mismatch.txt Eventually, in the third run, the upgrade completed. First run: [root@dev1 /usr/home/andreas]# freebsd-update -s update2.freebsd.org -r 10.2-RELEASE upgrade Looking up update2.freebsd.org mirrors... none found. Fetching metadata signature for 10.1-RELEASE from update2.freebsd.org... done. Fetching metadata index... done. Inspecting system... done. The following components of FreeBSD seem to be installed: kernel/generic src/src world/base world/doc The following components of FreeBSD do not seem to be installed: world/games Does this look reasonable (y/n)? y Fetching metadata signature for 10.2-RELEASE from update2.freebsd.org... done. Fetching metadata index... done. Fetching 1 metadata patches. done. Applying metadata patches... done. Fetching 1 metadata files... done. Inspecting system... done. Fetching files from 10.1-RELEASE for merging... done. Preparing to download files... done. Fetching 41142 patches.....10....20....30....40....50....60....70....80....90....100.... [... you all can count to 41030....] 41040....41050....41060....41070....41080....41090....41100....41110....41120....41130....41140. done. Applying patches... done. Fetching 5820 files... a36091931a81837106764f9afbf977c81c286f9bba476e9bfc77a3f962e84955 has incorrect hash. [root@dev1 /usr/home/andreas]# [root@dev1 /usr/home/andreas]# cd /var/db/freebsd-update/ [root@dev1 /var/db/freebsd-update]# ls -la a36091931a81837106764f9afbf977c81c286f9bba476e9bfc77a3f962e84955* -rw-r--r-- 1 root wheel 151 Aug 21 05:38 a36091931a81837106764f9afbf977c81c286f9bba476e9bfc77a3f962e84955.gz [root@dev1 /var/db/freebsd-update]# gunzip -c a36091931a81837106764f9afbf977c81c286f9bba476e9bfc77a3f962e84955.gz |sha256 a3649107fd11187af3797b596807f82cbab6f0ccae026b26a3eea3669a9223e5 [root@dev1 /var/db/freebsd-update]# [root@dev1 /var/db/freebsd-update]# gunzip -c a36091931a81837106764f9afbf977c81c286f9bba476e9bfc77a3f962e84955.gz .\" $FreeBSD: releng/10.2/tools/build/options/WITHOUT_FILE 279506 2015-03-01 22:07:54Z ngie $ Set to not build .Xr file 1 and related programs. [root@dev1 /var/db/freebsd-update]# Second run: [root@dev1 /var/db/freebsd-update]# date Fri Aug 21 05:52:14 UTC 2015 [root@dev1 /var/db/freebsd-update]# freebsd-update -s update2.freebsd.org -r 10.2-RELEASE upgrade Looking up update2.freebsd.org mirrors... none found. Fetching metadata signature for 10.1-RELEASE from update2.freebsd.org... done. Fetching metadata index... done. Fetching 1 metadata patches. done. Applying metadata patches... done. Fetching 1 metadata files... done. Inspecting system... done. The following components of FreeBSD seem to be installed: kernel/generic src/src world/base world/doc The following components of FreeBSD do not seem to be installed: world/games Does this look reasonable (y/n)? y Fetching metadata signature for 10.2-RELEASE from update2.freebsd.org... done. Fetching metadata index... done. Fetching 1 metadata patches. done. Applying metadata patches... done. Fetching 1 metadata files... done. Inspecting system... done. Fetching files from 10.1-RELEASE for merging... done. Preparing to download files... done. Fetching 354 patches.....10....20....30....40....50....60....70....80....90....100....110....120....130....140....150....160....170....180....190....200....210....220....230....240....250....260....270....280....290....300....310....320....330....340....350.. done. Applying patches... done. Fetching 1810 files... e663aaaca813b1ffebc92189b0f209a413806d0faf5a700bab9c9326e6e5b556 has incorrect hash. [root@dev1 /var/db/freebsd-update]# Third run: [root@dev1 /var/db/freebsd-update]# freebsd-update -s update2.freebsd.org -r 10.2-RELEASE upgrade Looking up update2.freebsd.org mirrors... none found. Fetching metadata signature for 10.1-RELEASE from update2.freebsd.org... done. Fetching metadata index... done. Fetching 1 metadata patches. done. Applying metadata patches... done. Fetching 1 metadata files... done. Inspecting system... done. The following components of FreeBSD seem to be installed: kernel/generic src/src world/base world/doc The following components of FreeBSD do not seem to be installed: world/games Does this look reasonable (y/n)? y Fetching metadata signature for 10.2-RELEASE from update2.freebsd.org... done. Fetching metadata index... done. Fetching 1 metadata patches. done. Applying metadata patches... done. Fetching 1 metadata files... done. Inspecting system... done. Fetching files from 10.1-RELEASE for merging... done. Preparing to download files... done. Fetching 1 patches. done. Applying patches... done. Fetching 521 files... done. Attempting to automatically merge changes in files... done. The following file could not be merged automatically: /etc/ntp.conf Press Enter to edit this file in /usr/bin/vi and resolve the conflicts manually... [manually fix diff and write file], then acknowledge the change log of all updated files, proceed with install (kernel), reboot and one more install (user land). Additional debug output available, just ask for it. I have a second server of the same specs awaiting upgrade as well, and then some more. Thanks, andreas -- Andreas Ott K6OTT +1.408.431.8727 andreas@naund.org From owner-freebsd-stable@freebsd.org Fri Aug 21 06:51:21 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B207F9BD2E4 for ; Fri, 21 Aug 2015 06:51:21 +0000 (UTC) (envelope-from h.schmalzbauer@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 40BB3111B; Fri, 21 Aug 2015 06:51:20 +0000 (UTC) (envelope-from h.schmalzbauer@omnilan.de) Received: from mh0.gentlemail.de (ezra.dcm1.omnilan.net [78.138.80.135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id t7L6pG7l068165; Fri, 21 Aug 2015 08:51:16 +0200 (CEST) (envelope-from h.schmalzbauer@omnilan.de) Received: from titan.inop.mo1.omnilan.net (titan.inop.mo1.omnilan.net [IPv6:2001:a60:f0bb:1::3:1]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id B9941DA3; Fri, 21 Aug 2015 08:51:15 +0200 (CEST) Message-ID: <55D6CA5C.1090905@omnilan.de> Date: Fri, 21 Aug 2015 08:51:08 +0200 From: Harald Schmalzbauer Organization: OmniLAN User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; de-DE; rv:1.9.2.8) Gecko/20100906 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: Ian Lepore CC: freebsd-stable@freebsd.org Subject: Re: Will 10.2 also ship with a very stale NTP? References: <20150710235810.GA76134@rwpc16.gfn.riverwillow.net.au> <20150712032256.GB19305@satori.lan> <20150712050443.GA22240@server.rulingia.com> <20150712154416.b9f3713893fe28bfab1dd4d7@dec.sakura.ne.jp> <20150712184910.2d8d5f085ae659d5b9a2aba0@dec.sakura.ne.jp> <1436715703.1334.193.camel@freebsd.org> <55B23B4E.1080400@omnilan.de> <1440110060.242.252.camel@freebsd.org> In-Reply-To: <1440110060.242.252.camel@freebsd.org> X-Enigmail-Version: 1.1.2 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigCB2977B8C68622885D2EF53B" X-Greylist: ACL 119 matched, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [78.138.80.130]); Fri, 21 Aug 2015 08:51:16 +0200 (CEST) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: 78.138.80.135; Sender-helo: mh0.gentlemail.de; ) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Aug 2015 06:51:21 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigCB2977B8C68622885D2EF53B Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Bez=C3=BCglich Ian Lepore's Nachricht vom 21.08.2015 00:34 (localtime): > On Fri, 2015-07-24 at 15:19 +0200, Harald Schmalzbauer wrote: >> Bezglich Ian Lepore's Nachricht vom 12.07.2015 17:41 (localtime): >>> And let's all just hope that a week or two of testing is enough when >>> jumping a major piece of software forward several years in its >>> independent evolution. >> =E2=80=A6 >>> I wonder how many other such things could be lurking in 4.2.8, waitin= g >>> to be triggered by other peoples' non-stock configurations? We've >> =E2=80=A6 >> >> I'd like to report one, most likely an upstream problem: >> >> 'restrict' definitions in ntp.conf(5) no longer work with unqualified = DNS names. >> A line like >> "restrict time1 nomodify nopeer noquery notrap" >> results in: >> ntpd[1913]: line 7 column 7 syntax error, unexpected T_Time1 >> ntpd[1913]: syntax error in /etc/ntp.conf line 7, column 7 >> >> I've always been using unqualified hostnames with 'restrict', and sinc= e defining 'server' with unqualified hostname still works, this seems to = be a significant bug to me. People are forced to change 'restrict' defini= tions, but not to also change other unqualified definitions, which potent= ially leads to misconfigurations, since intentionally matching definition= s can now differ easily. >> >> Has anybody already noticed this problem? And any idea if upstream is = aware? > I had a quick look at this today. It appears that the problem isn't > unqualified names exactly, but rather an unqualified name that exactly > matches an ntp.conf keyword will be mistaken by the ntpd config parser > as a misplaced keyword token. So most unqualified names should work, > but there are about 200 words that won't, many of them very sensible > names for ntp servers such as "ntp" and "time1" and "time2". > > When I look at the ntp_parser.y grammar file it's not clear to me why > "server time1" works and "restrict time1" doesn't. I couldn't find any= > way to trick it into taking a keyword as a hostname following restrict > (like using quotes). Thank you very much! This is very interesting and exactly matches my tested host names. I wish I had better C skills to find such things myself. Out of curiosity: How much time took it to find the ntp_parser.y route? (and with what =E2=80=9CIDE=E2=80=9D =E2=80=93 I'm stuck with vim) One additional observation was that the reserved-name-collision only happens with CNAME records. I hope I'll find some time to actually do look into sources - which I didn't at first hand because of my lousy C skills :-( But that's the place where to find hints :-) Thanks, -Harry --------------enigCB2977B8C68622885D2EF53B Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAlXWymIACgkQLDqVQ9VXb8izkACgqH7RX8BfsCnk1zAqT0avnqWs TdMAoNBFKTFFkAQY3inDeCBLlQeSmA60 =zVyk -----END PGP SIGNATURE----- --------------enigCB2977B8C68622885D2EF53B-- From owner-freebsd-stable@freebsd.org Fri Aug 21 07:07:06 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2E2CC9BD698 for ; Fri, 21 Aug 2015 07:07:06 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from elf.hq.norma.perm.ru (unknown [IPv6:2a00:7540:1::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Vivat-Trade UNIX Root CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 8055F1B1D for ; Fri, 21 Aug 2015 07:07:04 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from bsdrookie.norma.com. (perederenko.norma.com [IPv6:fd00::7dd] (may be forged)) by elf.hq.norma.perm.ru (8.14.9/8.14.9) with ESMTP id t7L76s9k027172 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Fri, 21 Aug 2015 12:06:54 +0500 (YEKT) (envelope-from emz@norma.perm.ru) Subject: Re: [POSSIBLE BUG] 10-STABLE CARP erroneously becomes master on boot To: freebsd-stable@freebsd.org References: From: "Eugene M. Zheganin" X-Enigmail-Draft-Status: N1110 Message-ID: <55D6CE0D.9030806@norma.perm.ru> Date: Fri, 21 Aug 2015 12:06:53 +0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.1.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (elf.hq.norma.perm.ru [IPv6:fd00::30a]); Fri, 21 Aug 2015 12:06:54 +0500 (YEKT) X-Spam-Status: No hits=-100.2 bayes=0.0000 testhits AWL=0.250,BAYES_00=-1.9, RDNS_NONE=0.793,SPF_SOFTFAIL=0.665,USER_IN_WHITELIST=-100 autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on elf.hq.norma.perm.ru X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Aug 2015 07:07:06 -0000 Hi. On 20.08.2015 14:51, Damien Fleuriot wrote: > > Hello list, > > > > We've managed to find the source of the bug, if it is indeed a bug. > > It all comes down to the order in which the IP addresses are assigned to > the interface from /etc/rc.conf. > > > When using the following syntax, the physical IP address is configured > AFTER the CARPs on the interface, which results in the CARP advertisements > being sourced from the CARP IP, triggering the double MASTER situation : > ipv4_addrs_int="1.2.3.4/24" > ifconfig_int_alias0="1.2.3.6/32 vhid 1 pass test advskew 20" > > When using either of the following syntaxes, the physical IP address is > configured BEFORE the CARPs, which results in the CARP advertisements being > sourced from the physical IP and restoring normal functionality : > ifconfig_int="inet 1.2.3.4/24" > ifconfig_int_alias0="1.2.3.6/32 vhid 1 pass test advskew 20" > OR > ifconfig_int_alias0="1.2.3.4/24" > ifconfig_int_alias1="1.2.3.6/32 vhid 1 pass test advskew 20" > It has been there since carp-ng was commited to the 10-CURRENT 2 years ago. The thing is, carp-ng doesn't need a non-carp address on an interface anymore, both nodes can work fine using only shared address. This isn't comfortable in lots of cases, but still. Thus, kernel sends carp advertisements from a primary address on the interface (which is normal behavior for any known network stack) and for FreeBSD that primary address has always been the first address on an interface for a given AF. Thus, your split-brain carp situation cause lies definitely somewhere else. I'm running carp on FreeBSD for years, including legacy one; if there is a bug - the situation you are describing probably isn't one. Eugene. From owner-freebsd-stable@freebsd.org Fri Aug 21 08:25:13 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D82749BF6AD for ; Fri, 21 Aug 2015 08:25:13 +0000 (UTC) (envelope-from marko.cupac@mimar.rs) Received: from smtp.mimar.rs (smtp.mimar.rs [193.53.106.135]) by mx1.freebsd.org (Postfix) with ESMTP id 88881A17 for ; Fri, 21 Aug 2015 08:25:11 +0000 (UTC) (envelope-from marko.cupac@mimar.rs) Received: from vscan.mimar.rs (vscan.mimar.rs [193.53.106.134]) by smtp.mimar.rs (Postfix) with ESMTP id 5995C89A82 for ; Fri, 21 Aug 2015 10:25:07 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=mimar.rs; h= content-transfer-encoding:content-type:content-type:mime-version :x-mailer:organization:references:in-reply-to:message-id:subject :subject:from:from:date:date:received:received; s=mimar-0901; t= 1440145506; x=1441959907; bh=zHPFrpNNt35A7j/7GJZtuv8f7OpPzUueHf/ P6r20QbE=; b=BbViw0Hx2TgOJ+3XEqV3fz4KyYQR/s0kXdbAYnKvR4dGNRYgiGP 3gQKpgNzUlogouaW7TD0pJDtW5oMKlM/vYeE2hq7QEkXXgWt8C4qq6ttsrtetTok WG+jxig+3pnqG8vhybAajgwju281xhFRkf6uK2RJf1UA9Cfhbvi/eqtA= X-Virus-Scanned: amavisd-new at mimar.rs Received: from smtp.mimar.rs ([193.53.106.135]) by vscan.mimar.rs (vscan.mimar.rs [193.53.106.134]) (amavisd-new, port 10026) with ESMTP id kiyEpJ_op13b for ; Fri, 21 Aug 2015 10:25:06 +0200 (CEST) Received: from efreet (unknown [193.53.106.34]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: marko.cupac@mimar.rs) by smtp.mimar.rs (Postfix) with ESMTPSA id 8F68C898F4 for ; Fri, 21 Aug 2015 10:25:06 +0200 (CEST) Date: Fri, 21 Aug 2015 10:25:05 +0200 From: Marko =?UTF-8?B?Q3VwYcSH?= To: freebsd-stable@freebsd.org Subject: Re: vmx and esxi vst mode vlan tagging Message-ID: <20150821102505.7febb5b0@efreet> In-Reply-To: <20150820132353.2ae19568@efreet> References: <20150820132353.2ae19568@efreet> Organization: mimar X-Mailer: Claws Mail 3.9.3 (GTK+ 2.24.23; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Aug 2015 08:25:13 -0000 On Thu, 20 Aug 2015 13:23:53 +0200 Marko Cupa=C4=87 wrote: > Hi, >=20 > I have just spent half an hour bashing my head against the wall why my > FreeBSD-10.2-RELEASE i386 machine with kernel-included vmx driver and > emulators/open-vm-tools installed won't ping machines on the other > vlan on the same vSwitch. It DOES ping other virtual machines on the > same vSwitch and same vlan (freebsd servers), and it DOES ping > physical machines in other vlans. It DOES NOT ping virtual machines > on the same vSwitch and different vlan (windows servers). >=20 > To cut the long story short, it appears that the problem is on FreeBSD > side, as another machine (FreeBSD-10.1-RELEASE-p14 amd 64) with vmx3f0 > driver and VMWare's vmware-tools installed pings everything as it > should. >=20 > Could it be that FreeBSD's vmx driver does not support all the > functions of vmware's vmx3f driver? I deleted open-vm-tools and installed vmware-tools, but the broblem remains. I'm gonna test if 10.2-RELEASE amd64 has the same issue. That should tell me if problem is related to difference between i386 and amd64 in 10.2-RELEASE. If not, could this be a regression in 10.2? Should I report a bug or do some more testing? Thank you in advance, --=20 Marko Cupa=C4=87 https://www.mimar.rs/ From owner-freebsd-stable@freebsd.org Fri Aug 21 09:59:28 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A66A9BEBF4 for ; Fri, 21 Aug 2015 09:59:28 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 532F2191 for ; Fri, 21 Aug 2015 09:59:28 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.84 (FreeBSD)) (envelope-from ) id 1ZSioo-0002f6-DO; Fri, 21 Aug 2015 12:41:02 +0300 Date: Fri, 21 Aug 2015 12:41:02 +0300 From: Slawa Olhovchenkov To: Marko =?koi8-r?Q?Cupa'c?= Cc: freebsd-stable@freebsd.org Subject: Re: vmx and esxi vst mode vlan tagging Message-ID: <20150821094102.GI3158@zxy.spb.ru> References: <20150820132353.2ae19568@efreet> <20150821102505.7febb5b0@efreet> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <20150821102505.7febb5b0@efreet> User-Agent: Mutt/1.5.23 (2014-03-12) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Aug 2015 09:59:28 -0000 On Fri, Aug 21, 2015 at 10:25:05AM +0200, Marko Cupa'c wrote: > On Thu, 20 Aug 2015 13:23:53 +0200 > Marko Cupa'c wrote: > > > Hi, > > > > I have just spent half an hour bashing my head against the wall why my > > FreeBSD-10.2-RELEASE i386 machine with kernel-included vmx driver and > > emulators/open-vm-tools installed won't ping machines on the other > > vlan on the same vSwitch. It DOES ping other virtual machines on the > > same vSwitch and same vlan (freebsd servers), and it DOES ping > > physical machines in other vlans. It DOES NOT ping virtual machines > > on the same vSwitch and different vlan (windows servers). > > > > To cut the long story short, it appears that the problem is on FreeBSD > > side, as another machine (FreeBSD-10.1-RELEASE-p14 amd 64) with vmx3f0 > > driver and VMWare's vmware-tools installed pings everything as it > > should. > > > > Could it be that FreeBSD's vmx driver does not support all the > > functions of vmware's vmx3f driver? > > I deleted open-vm-tools and installed vmware-tools, but the broblem > remains. I'm gonna test if 10.2-RELEASE amd64 has the same issue. That > should tell me if problem is related to difference between i386 and > amd64 in 10.2-RELEASE. If not, could this be a regression in 10.2? > > Should I report a bug or do some more testing? netstat -nr arp -na ifconfig -a ping's commands and results. From owner-freebsd-stable@freebsd.org Fri Aug 21 10:19:18 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 171519BEF3A for ; Fri, 21 Aug 2015 10:19:18 +0000 (UTC) (envelope-from marko.cupac@mimar.rs) Received: from smtp.mimar.rs (smtp.mimar.rs [193.53.106.135]) by mx1.freebsd.org (Postfix) with ESMTP id BAE96B0C for ; Fri, 21 Aug 2015 10:19:16 +0000 (UTC) (envelope-from marko.cupac@mimar.rs) Received: from vscan.mimar.rs (vscan.mimar.rs [193.53.106.134]) by smtp.mimar.rs (Postfix) with ESMTP id 9825A89A82; Fri, 21 Aug 2015 12:19:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=mimar.rs; h= content-transfer-encoding:content-type:content-type:mime-version :x-mailer:organization:references:in-reply-to:message-id:subject :subject:from:from:date:date:received:received; s=mimar-0901; t= 1440152352; x=1441966753; bh=+Vk2f1gDWAyF1rLbRBhm/Lb/iXDL3V5T4Y/ bodh1IoE=; b=Hh4uh3V7MmzRZXtgTkdnb8hpewWCZiwpos6E/g+GtBgfAhG/Ya0 yP2XlcbX5fwGRuD7UB02xHecJ258waJ953CXbGNCDl4Ro6YbnIRa764OEDpPG6wI V8XgOeoRRouu1QMeR58nmG62QxxMN9/do4yGJLcurDvD3UMV7wKm1SjA= X-Virus-Scanned: amavisd-new at mimar.rs Received: from smtp.mimar.rs ([193.53.106.135]) by vscan.mimar.rs (vscan.mimar.rs [193.53.106.134]) (amavisd-new, port 10026) with ESMTP id JLXY4-JILaWv; Fri, 21 Aug 2015 12:19:12 +0200 (CEST) Received: from efreet (unknown [193.53.106.34]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: marko.cupac@mimar.rs) by smtp.mimar.rs (Postfix) with ESMTPSA id C0DC689691; Fri, 21 Aug 2015 12:19:11 +0200 (CEST) Date: Fri, 21 Aug 2015 12:19:11 +0200 From: Marko =?UTF-8?B?Q3VwYcSH?= To: Slawa Olhovchenkov Cc: freebsd-stable@freebsd.org Subject: Re: vmx and esxi vst mode vlan tagging Message-ID: <20150821121911.47728de0@efreet> In-Reply-To: <20150821094102.GI3158@zxy.spb.ru> References: <20150820132353.2ae19568@efreet> <20150821102505.7febb5b0@efreet> <20150821094102.GI3158@zxy.spb.ru> Organization: mimar X-Mailer: Claws Mail 3.9.3 (GTK+ 2.24.23; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Aug 2015 10:19:18 -0000 On Fri, 21 Aug 2015 12:41:02 +0300 Slawa Olhovchenkov wrote: > On Fri, Aug 21, 2015 at 10:25:05AM +0200, Marko Cupa'c wrote: >=20 > > On Thu, 20 Aug 2015 13:23:53 +0200 > > Marko Cupa'c wrote: > >=20 > > > Hi, > > >=20 > > > I have just spent half an hour bashing my head against the wall > > > why my FreeBSD-10.2-RELEASE i386 machine with kernel-included vmx > > > driver and emulators/open-vm-tools installed won't ping machines > > > on the other vlan on the same vSwitch. It DOES ping other virtual > > > machines on the same vSwitch and same vlan (freebsd servers), and > > > it DOES ping physical machines in other vlans. It DOES NOT ping > > > virtual machines on the same vSwitch and different vlan (windows > > > servers). > > >=20 > > > To cut the long story short, it appears that the problem is on > > > FreeBSD side, as another machine (FreeBSD-10.1-RELEASE-p14 amd > > > 64) with vmx3f0 driver and VMWare's vmware-tools installed pings > > > everything as it should. > > >=20 > > > Could it be that FreeBSD's vmx driver does not support all the > > > functions of vmware's vmx3f driver? > >=20 > > I deleted open-vm-tools and installed vmware-tools, but the broblem > > remains. I'm gonna test if 10.2-RELEASE amd64 has the same issue. > > That should tell me if problem is related to difference between > > i386 and amd64 in 10.2-RELEASE. If not, could this be a regression > > in 10.2? > >=20 > > Should I report a bug or do some more testing? >=20 > netstat -nr > arp -na > ifconfig -a > ping's commands and results. Sorry guys, please dismiss this issue entirely. I was constantly setting /24 subnet mask whereas the correct mask for the vlan is /27. Please consider the fact that my long-needed vacation starts tomorrow :) Regards, --=20 Marko Cupa=C4=87 https://www.mimar.rs/ From owner-freebsd-stable@freebsd.org Fri Aug 21 11:13:01 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CB8309BCA8D for ; Fri, 21 Aug 2015 11:13:01 +0000 (UTC) (envelope-from ml@my.gd) Received: from mail-la0-f48.google.com (mail-la0-f48.google.com [209.85.215.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4703916EA for ; Fri, 21 Aug 2015 11:13:00 +0000 (UTC) (envelope-from ml@my.gd) Received: by lagz9 with SMTP id z9so39182275lag.3 for ; Fri, 21 Aug 2015 04:12:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=lN6IqyCtUiNA3MiX8aCrwnObkBFzzjDrzxK9R7Pt3m8=; b=DbWpzQEV92/1DYzSDp3swblskPdOh7CU1plPO4TXY8dS70654E4ZotP+HA0u3/t/ap Pwg71kvDnqeqMZBhnWXNANrCnuVK66oMuhAx074kS33k60oPOcsUnQK1Hw84Do8s/8JZ JX/yqwiA8wVoycjcvjy76TssGV4EK4pQi0VFq82FxjMWSHm5++wP3Zj1UBG4HvmANDyX LtAgAd5GB5daBJc+hD/45iuKik0nI60hIYbhxb/PyQ8dRJvoU/Io6vDF8Ve/uJhSzu10 C89kafiqjwNSfNCaEUhhTXlZHlCuBXc3X/kupvhwhoTueYsFuIEzjFWVLVTA7uPR9+sz NS7w== X-Gm-Message-State: ALoCoQls8on5YCzk8MHuX1UeFFRO8S6y5kngGP/sZciRhPtBcEdVKFWYqjquPX9ksKXKuWZKug1J MIME-Version: 1.0 X-Received: by 10.152.87.116 with SMTP id w20mr7341620laz.119.1440155573284; Fri, 21 Aug 2015 04:12:53 -0700 (PDT) Received: by 10.112.60.34 with HTTP; Fri, 21 Aug 2015 04:12:53 -0700 (PDT) In-Reply-To: <55D6CE0D.9030806@norma.perm.ru> References: <55D6CE0D.9030806@norma.perm.ru> Date: Fri, 21 Aug 2015 13:12:53 +0200 Message-ID: Subject: Re: [POSSIBLE BUG] 10-STABLE CARP erroneously becomes master on boot From: Damien Fleuriot To: "Eugene M. Zheganin" Cc: "freebsd-stable@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Aug 2015 11:13:01 -0000 On 21 August 2015 at 09:06, Eugene M. Zheganin wrote: > Hi. > > On 20.08.2015 14:51, Damien Fleuriot wrote: > > > > Hello list, > > > > > > > > We've managed to find the source of the bug, if it is indeed a bug. > > > > It all comes down to the order in which the IP addresses are assigned to > > the interface from /etc/rc.conf. > > > > > > When using the following syntax, the physical IP address is configured > > AFTER the CARPs on the interface, which results in the CARP > advertisements > > being sourced from the CARP IP, triggering the double MASTER situation : > > ipv4_addrs_int="1.2.3.4/24" > > ifconfig_int_alias0="1.2.3.6/32 vhid 1 pass test advskew 20" > > > > When using either of the following syntaxes, the physical IP address is > > configured BEFORE the CARPs, which results in the CARP advertisements > being > > sourced from the physical IP and restoring normal functionality : > > ifconfig_int="inet 1.2.3.4/24" > > ifconfig_int_alias0="1.2.3.6/32 vhid 1 pass test advskew 20" > > OR > > ifconfig_int_alias0="1.2.3.4/24" > > ifconfig_int_alias1="1.2.3.6/32 vhid 1 pass test advskew 20" > > > It has been there since carp-ng was commited to the 10-CURRENT 2 years > ago. The thing is, carp-ng doesn't need a non-carp address on an > interface anymore, both nodes can work fine using only shared address. > This isn't comfortable in lots of cases, but still. Thus, kernel sends > carp advertisements from a primary address on the interface (which is > normal behavior for any known network stack) and for FreeBSD that > primary address has always been the first address on an interface for a > given AF. Thus, your split-brain carp situation cause lies definitely > somewhere else. I'm running carp on FreeBSD for years, including legacy > one; if there is a bug - the situation you are describing probably isn't > one. > > Eugene, agree to disagree here. I've also been using CARP for years, both legacy and carp-ng, and while I'm not an expert on its inner workings I do understand how it operates. What you describe WRT the network stack and sourcing the advertisements is correct, I'll give you that. The problem lies not with CARP but with how the IP addresses are assigned by rc.conf It is abnormal that the CARP addresses should be set up first, when using the "ipv4_addrs_int" syntax. Therein lies the problem. From owner-freebsd-stable@freebsd.org Fri Aug 21 14:04:07 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D94BF9BF65C for ; Fri, 21 Aug 2015 14:04:07 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from pmta2.delivery6.ore.mailhop.org (pmta2.delivery6.ore.mailhop.org [54.200.129.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BA6E013CC for ; Fri, 21 Aug 2015 14:04:07 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from ilsoft.org (unknown [73.34.117.227]) by outbound2.ore.mailhop.org (Halon Mail Gateway) with ESMTPSA; Fri, 21 Aug 2015 14:05:30 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.14.9/8.14.9) with ESMTP id t7LE3xJ6008939; Fri, 21 Aug 2015 08:03:59 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: <1440165839.242.260.camel@freebsd.org> Subject: Re: Will 10.2 also ship with a very stale NTP? From: Ian Lepore To: Harald Schmalzbauer Cc: freebsd-stable@freebsd.org Date: Fri, 21 Aug 2015 08:03:59 -0600 In-Reply-To: <55D6CA5C.1090905@omnilan.de> References: <20150710235810.GA76134@rwpc16.gfn.riverwillow.net.au> <20150712032256.GB19305@satori.lan> <20150712050443.GA22240@server.rulingia.com> <20150712154416.b9f3713893fe28bfab1dd4d7@dec.sakura.ne.jp> <20150712184910.2d8d5f085ae659d5b9a2aba0@dec.sakura.ne.jp> <1436715703.1334.193.camel@freebsd.org> <55B23B4E.1080400@omnilan.de> <1440110060.242.252.camel@freebsd.org> <55D6CA5C.1090905@omnilan.de> Content-Type: text/plain; charset="euc-jp" X-Mailer: Evolution 3.12.10 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Aug 2015 14:04:08 -0000 On Fri, 2015-08-21 at 08:51 +0200, Harald Schmalzbauer wrote: > Bezglich Ian Lepore's Nachricht vom 21.08.2015 00:34 (localtime): > > On Fri, 2015-07-24 at 15:19 +0200, Harald Schmalzbauer wrote: > >> Bezglich Ian Lepore's Nachricht vom 12.07.2015 17:41 (localtime): > >>> And let's all just hope that a week or two of testing is enough when > >>> jumping a major piece of software forward several years in its > >>> independent evolution. > >> > >>> I wonder how many other such things could be lurking in 4.2.8, waiting > >>> to be triggered by other peoples' non-stock configurations? We've > >> > >> > >> I'd like to report one, most likely an upstream problem: > >> > >> 'restrict' definitions in ntp.conf(5) no longer work with unqualified DNS names. > >> A line like > >> "restrict time1 nomodify nopeer noquery notrap" > >> results in: > >> ntpd[1913]: line 7 column 7 syntax error, unexpected T_Time1 > >> ntpd[1913]: syntax error in /etc/ntp.conf line 7, column 7 > >> > >> I've always been using unqualified hostnames with 'restrict', and since defining 'server' with unqualified hostname still works, this seems to be a significant bug to me. People are forced to change 'restrict' definitions, but not to also change other unqualified definitions, which potentially leads to misconfigurations, since intentionally matching definitions can now differ easily. > >> > >> Has anybody already noticed this problem? And any idea if upstream is aware? > > I had a quick look at this today. It appears that the problem isn't > > unqualified names exactly, but rather an unqualified name that exactly > > matches an ntp.conf keyword will be mistaken by the ntpd config parser > > as a misplaced keyword token. So most unqualified names should work, > > but there are about 200 words that won't, many of them very sensible > > names for ntp servers such as "ntp" and "time1" and "time2". > > > > When I look at the ntp_parser.y grammar file it's not clear to me why > > "server time1" works and "restrict time1" doesn't. I couldn't find any > > way to trick it into taking a keyword as a hostname following restrict > > (like using quotes). > > Thank you very much! This is very interesting and exactly matches my > tested host names. > I wish I had better C skills to find such things myself. Out of > curiosity: How much time took it to find the ntp_parser.y route? (and > with what IDE I'm stuck with vim) > > One additional observation was that the reserved-name-collision only > happens with CNAME records. > I hope I'll find some time to actually do look into sources - which I > didn't at first hand because of my lousy C skills :-( But that's the > place where to find hints :-) > > Thanks, > I started out pretty sure what I was going to discover, based on the error you reported "syntax error, unexpected T_Time1". That 'T_Time1' just said to me "that's a yacc/bison token constant, this is going to be an error in their grammar (.y) file". The tricky part is that the .y file isn't in the base source code, I had to go find it in the vendor branch. I don't think the CNAME part matters. I tried changing my 'ntp' CNAME to a regular A record and the error still happens if I use it as an unqualified name with restrict. The IDE I use is SlickEdit, running on freebsd under the linuxulator. It's a commercial product worth every penny I've paid for various versions since the 90s. It gets the credit for a lot of my productivity. -- Ian From owner-freebsd-stable@freebsd.org Fri Aug 21 21:46:16 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 66E5A9C0558; Fri, 21 Aug 2015 21:46:16 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id EA31B1F43; Fri, 21 Aug 2015 21:46:15 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:19NhgxYJD9UAAixx11n0lZD/LSx+4OfEezUN459isYplN5qZpcW/bnLW6fgltlLVR4KTs6sC0LqN9f25EjRZqb+681k8M7V0HycfjssXmwFySOWkMmbcaMDQUiohAc5ZX0Vk9XzoeWJcGcL5ekGA6ibqtW1aJBzzOEJPK/jvHcaK1oLsh7v0psSYO1wArQH+SI0xBS3+lR/WuMgSjNkqAYcK4TyNnEF1ff9Lz3hjP1OZkkW0zM6x+Jl+73YY4Kp5pIZoGJ/3dKUgTLFeEC9ucyVsvJWq5lH/Sl6X92YaQ2U+nR9BAgyD5xb/Gt/duy37u+418jOTO8ztVvhgVT2k6bZDQwSuiDoFNngw+yfWjpojorhcpUebphd8i6vda4KROf82KrnYdNgZQWdEdttWWDFMBpu8KYAGWblSdd1EppXw8gNd5SC1AhOhUaa2kmdF X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2BEAgBFm9dV/61jaINdDoNhaQaDH7pEAQmBbQqFMUoCgW0UAQEBAQEBAQGBCYIdggcBAQQBAQEgBCcgCwULAgEIGAICDRkCAicBCSYCBAEHBwQBGgIEiA0NuHWVfwEBAQEBAQEBAQEBAQEBAQEBARYEgSKKMoQyBgEBHDQHgmmBQwWVLYUFhQiELIdKiH6ESYNoAiaCDhyBFVoiMwd/CBcjgQQBAQE X-IronPort-AV: E=Sophos;i="5.15,723,1432612800"; d="scan'208";a="232337900" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 21 Aug 2015 17:46:08 -0400 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id E45C915F55D; Fri, 21 Aug 2015 17:46:08 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id l7J6bMg-1Frh; Fri, 21 Aug 2015 17:46:08 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 25FC715F563; Fri, 21 Aug 2015 17:46:08 -0400 (EDT) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 9cPtlj9JK9l1; Fri, 21 Aug 2015 17:46:08 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 0244415F55D; Fri, 21 Aug 2015 17:46:08 -0400 (EDT) Date: Fri, 21 Aug 2015 17:46:07 -0400 (EDT) From: Rick Macklem To: pyunyh@gmail.com, Daniel Braniss Cc: Hans Petter Selasky , FreeBSD stable , FreeBSD Net , Slawa Olhovchenkov , Gleb Smirnoff , Christopher Forgeron Message-ID: <1153838447.28656490.1440193567940.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <20150820023024.GB996@michelle.fasterthan.com> References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <55D333D6.5040102@selasky.org> <1325951625.25292515.1439934848268.JavaMail.zimbra@uoguelph.ca> <55D429A4.3010407@selasky.org> <20150819074212.GB964@michelle.fasterthan.com> <55D43615.1030401@selasky.org> <2013503980.25726607.1439989235806.JavaMail.zimbra@uoguelph.ca> <20150820023024.GB996@michelle.fasterthan.com> Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.12] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF34 (Win)/8.0.9_GA_6191) Thread-Topic: ix(intel) vs mlxen(mellanox) 10Gb performance Thread-Index: Zq6MViNwCd2Hhr1mTS/9wAk6UKubQA== X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Aug 2015 21:46:16 -0000 Yonghyeon PYUN wrote: > On Wed, Aug 19, 2015 at 09:00:35AM -0400, Rick Macklem wrote: > > Hans Petter Selasky wrote: > > > On 08/19/15 09:42, Yonghyeon PYUN wrote: > > > > On Wed, Aug 19, 2015 at 09:00:52AM +0200, Hans Petter Selasky wrote: > > > >> On 08/18/15 23:54, Rick Macklem wrote: > > > >>> Ouch! Yes, I now see that the code that counts the # of mbufs is > > > >>> before > > > >>> the > > > >>> code that adds the tcp/ip header mbuf. > > > >>> > > > >>> In my opinion, this should be fixed by setting if_hw_tsomaxsegcount > > > >>> to > > > >>> whatever > > > >>> the driver provides - 1. It is not the driver's responsibility to > > > >>> know if > > > >>> a tcp/ip > > > >>> header mbuf will be added and is a lot less confusing that expecting > > > >>> the > > > >>> driver > > > >>> author to know to subtract one. (I had mistakenly thought that > > > >>> tcp_output() had > > > >>> added the tc/ip header mbuf before the loop that counts mbufs in the > > > >>> list. > > > >>> Btw, > > > >>> this tcp/ip header mbuf also has leading space for the MAC layer > > > >>> header.) > > > >>> > > > >> > > > >> Hi Rick, > > > >> > > > >> Your question is good. With the Mellanox hardware we have separate > > > >> so-called inline data space for the TCP/IP headers, so if the TCP > > > >> stack > > > >> subtracts something, then we would need to add something to the limit, > > > >> because then the scatter gather list is only used for the data part. > > > >> > > > > > > > > I think all drivers in tree don't subtract 1 for > > > > if_hw_tsomaxsegcount. Probably touching Mellanox driver would be > > > > simpler than fixing all other drivers in tree. > > > > > > > >> Maybe it can be controlled by some kind of flag, if all the three TSO > > > >> limits should include the TCP/IP/ethernet headers too. I'm pretty sure > > > >> we want both versions. > > > >> > > > > > > > > Hmm, I'm afraid it's already complex. Drivers have to tell almost > > > > the same information to both bus_dma(9) and network stack. > > > > > > Don't forget that not all drivers in the tree set the TSO limits before > > > if_attach(), so possibly the subtraction of one TSO fragment needs to go > > > into ip_output() .... > > > > > Ok, I realized that some drivers may not know the answers before > > ether_ifattach(), > > due to the way they are configured/written (I saw the use of > > if_hw_tsomax_update() > > in the patch). > > I was not able to find an interface that configures TSO parameters > after if_t conversion. I'm under the impression > if_hw_tsomax_update() is not designed to use this way. Probably we > need a better one?(CCed to Gleb). > > > > > If it is subtracted as a part of the assignment to if_hw_tsomaxsegcount in > > tcp_output() > > at line#791 in tcp_output() like the following, I don't think it should > > matter if the > > values are set before ether_ifattach()? > > /* > > * Subtract 1 for the tcp/ip header mbuf that > > * will be prepended to the mbuf chain in this > > * function in the code below this block. > > */ > > if_hw_tsomaxsegcount = tp->t_tsomaxsegcount - 1; > > > > I don't have a good solution for the case where a driver doesn't plan on > > using the > > tcp/ip header provided by tcp_output() except to say the driver can add one > > to the > > setting to compensate for that (and if they fail to do so, it still works, > > although > > somewhat suboptimally). When I now read the comment in sys/net/if_var.h it > > is clear > > what it means, but for some reason I didn't read it that way before? (I > > think it was > > the part that said the driver didn't have to subtract for the headers that > > confused me?) > > In any case, we need to try and come up with a clear definition of what > > they need to > > be set to. > > > > I can now think of two ways to deal with this: > > 1 - Leave tcp_output() as is, but provide a macro for the device driver > > authors to use > > that sets if_hw_tsomaxsegcount with a flag for "driver uses tcp/ip > > header mbuf", > > documenting that this flag should normally be true. > > OR > > 2 - Change tcp_output() as above, noting that this is a workaround for > > confusion w.r.t. > > whether or not if_hw_tsomaxsegcount should include the tcp/ip header > > mbuf and > > update the comment in if_var.h to reflect this. Then drivers that don't > > use the > > tcp/ip header mbuf can increase their value for if_hw_tsomaxsegcount by > > 1. > > (The comment should also mention that a value of 35 or greater is much > > preferred to > > 32 if the hardware will support that.) > > > > Both works for me. My preference is 2 just because it's very > common for most drivers that use tcp/ip header mbuf. Thanks for this comment. I tend to agree, both for the reason you state and also because the patch is simple enough that it might qualify as an errata for 10.2. I am hoping Daniel Braniss will be able to test the patch and let us know if it improves performance with TSO enabled? rick > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > From owner-freebsd-stable@freebsd.org Sat Aug 22 07:28:20 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 461AD9BE683; Sat, 22 Aug 2015 07:28:20 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.116.210]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EB7111697; Sat, 22 Aug 2015 07:28:19 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from mbpro2.bs.cs.huji.ac.il ([132.65.179.20]) by kabab.cs.huji.ac.il with esmtp id 1ZT3Dh-000P8u-6j; Sat, 22 Aug 2015 10:28:05 +0300 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance From: Daniel Braniss In-Reply-To: <1153838447.28656490.1440193567940.JavaMail.zimbra@uoguelph.ca> Date: Sat, 22 Aug 2015 10:28:03 +0300 Cc: pyunyh@gmail.com, Hans Petter Selasky , FreeBSD stable , FreeBSD Net , Slawa Olhovchenkov , Gleb Smirnoff , Christopher Forgeron Content-Transfer-Encoding: quoted-printable Message-Id: <15D19823-08F7-4E55-BBD0-CE230F67D26E@cs.huji.ac.il> References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <55D333D6.5040102@selasky.org> <1325951625.25292515.1439934848268.JavaMail.zimbra@uoguelph.ca> <55D429A4.3010407@selasky.org> <20150819074212.GB964@michelle.fasterthan.com> <55D43615.1030401@selasky.org> <2013503980.25726607.1439989235806.JavaMail.zimbra@uoguelph.ca> <20150820023024.GB996@michelle.fasterthan.com> <1153838447.28656490.1440193567940.JavaMail.zimbra@uoguelph.ca> To: Rick Macklem X-Mailer: Apple Mail (2.2104) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Aug 2015 07:28:20 -0000 > On Aug 22, 2015, at 12:46 AM, Rick Macklem = wrote: >=20 > Yonghyeon PYUN wrote: >> On Wed, Aug 19, 2015 at 09:00:35AM -0400, Rick Macklem wrote: >>> Hans Petter Selasky wrote: >>>> On 08/19/15 09:42, Yonghyeon PYUN wrote: >>>>> On Wed, Aug 19, 2015 at 09:00:52AM +0200, Hans Petter Selasky = wrote: >>>>>> On 08/18/15 23:54, Rick Macklem wrote: >>>>>>> Ouch! Yes, I now see that the code that counts the # of mbufs is >>>>>>> before >>>>>>> the >>>>>>> code that adds the tcp/ip header mbuf. >>>>>>>=20 >>>>>>> In my opinion, this should be fixed by setting = if_hw_tsomaxsegcount >>>>>>> to >>>>>>> whatever >>>>>>> the driver provides - 1. It is not the driver's responsibility = to >>>>>>> know if >>>>>>> a tcp/ip >>>>>>> header mbuf will be added and is a lot less confusing that = expecting >>>>>>> the >>>>>>> driver >>>>>>> author to know to subtract one. (I had mistakenly thought that >>>>>>> tcp_output() had >>>>>>> added the tc/ip header mbuf before the loop that counts mbufs in = the >>>>>>> list. >>>>>>> Btw, >>>>>>> this tcp/ip header mbuf also has leading space for the MAC layer >>>>>>> header.) >>>>>>>=20 >>>>>>=20 >>>>>> Hi Rick, >>>>>>=20 >>>>>> Your question is good. With the Mellanox hardware we have = separate >>>>>> so-called inline data space for the TCP/IP headers, so if the TCP >>>>>> stack >>>>>> subtracts something, then we would need to add something to the = limit, >>>>>> because then the scatter gather list is only used for the data = part. >>>>>>=20 >>>>>=20 >>>>> I think all drivers in tree don't subtract 1 for >>>>> if_hw_tsomaxsegcount. Probably touching Mellanox driver would be >>>>> simpler than fixing all other drivers in tree. >>>>>=20 >>>>>> Maybe it can be controlled by some kind of flag, if all the three = TSO >>>>>> limits should include the TCP/IP/ethernet headers too. I'm pretty = sure >>>>>> we want both versions. >>>>>>=20 >>>>>=20 >>>>> Hmm, I'm afraid it's already complex. Drivers have to tell almost >>>>> the same information to both bus_dma(9) and network stack. >>>>=20 >>>> Don't forget that not all drivers in the tree set the TSO limits = before >>>> if_attach(), so possibly the subtraction of one TSO fragment needs = to go >>>> into ip_output() .... >>>>=20 >>> Ok, I realized that some drivers may not know the answers before >>> ether_ifattach(), >>> due to the way they are configured/written (I saw the use of >>> if_hw_tsomax_update() >>> in the patch). >>=20 >> I was not able to find an interface that configures TSO parameters >> after if_t conversion. I'm under the impression >> if_hw_tsomax_update() is not designed to use this way. Probably we >> need a better one?(CCed to Gleb). >>=20 >>>=20 >>> If it is subtracted as a part of the assignment to = if_hw_tsomaxsegcount in >>> tcp_output() >>> at line#791 in tcp_output() like the following, I don't think it = should >>> matter if the >>> values are set before ether_ifattach()? >>> /* >>> * Subtract 1 for the tcp/ip header mbuf that >>> * will be prepended to the mbuf chain in this >>> * function in the code below this block. >>> */ >>> if_hw_tsomaxsegcount =3D tp->t_tsomaxsegcount - = 1; >>>=20 >>> I don't have a good solution for the case where a driver doesn't = plan on >>> using the >>> tcp/ip header provided by tcp_output() except to say the driver can = add one >>> to the >>> setting to compensate for that (and if they fail to do so, it still = works, >>> although >>> somewhat suboptimally). When I now read the comment in = sys/net/if_var.h it >>> is clear >>> what it means, but for some reason I didn't read it that way before? = (I >>> think it was >>> the part that said the driver didn't have to subtract for the = headers that >>> confused me?) >>> In any case, we need to try and come up with a clear definition of = what >>> they need to >>> be set to. >>>=20 >>> I can now think of two ways to deal with this: >>> 1 - Leave tcp_output() as is, but provide a macro for the device = driver >>> authors to use >>> that sets if_hw_tsomaxsegcount with a flag for "driver uses = tcp/ip >>> header mbuf", >>> documenting that this flag should normally be true. >>> OR >>> 2 - Change tcp_output() as above, noting that this is a workaround = for >>> confusion w.r.t. >>> whether or not if_hw_tsomaxsegcount should include the tcp/ip = header >>> mbuf and >>> update the comment in if_var.h to reflect this. Then drivers that = don't >>> use the >>> tcp/ip header mbuf can increase their value for = if_hw_tsomaxsegcount by >>> 1. >>> (The comment should also mention that a value of 35 or greater is = much >>> preferred to >>> 32 if the hardware will support that.) >>>=20 >>=20 >> Both works for me. My preference is 2 just because it's very >> common for most drivers that use tcp/ip header mbuf. > Thanks for this comment. I tend to agree, both for the reason you = state and also > because the patch is simple enough that it might qualify as an errata = for 10.2. >=20 > I am hoping Daniel Braniss will be able to test the patch and let us = know if it > improves performance with TSO enabled? send me the patch and I=E2=80=99ll test it ASAP. danny >=20 > rick >=20 >> _______________________________________________ >> freebsd-stable@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to = "freebsd-stable-unsubscribe@freebsd.org" >>=20 From owner-freebsd-stable@freebsd.org Sat Aug 22 11:59:21 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 86D239C0207; Sat, 22 Aug 2015 11:59:21 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 020CBAE5; Sat, 22 Aug 2015 11:59:20 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:GxIpMRGQwlqbIKuInsPTQp1GYnF86YWxBRYc798ds5kLTJ74o82wAkXT6L1XgUPTWs2DsrQf27GQ7vqrADBIyK3CmU5BWaQEbwUCh8QSkl5oK+++Imq/EsTXaTcnFt9JTl5v8iLzG0FUHMHjew+a+SXqvnYsExnyfTB4Ov7yUtaLyZ/njKbvodaKP01hv3mUX/BbFF2OtwLft80b08NJC50a7V/3mEZOYPlc3mhyJFiezF7W78a0+4N/oWwL46pyv+YJa6jxfrw5QLpEF3xmdjltvIy4/STFVhaFs3sATn0NwF0PBwne8Aq8UI38vyHhuqx6wibdOMT3SbU9X3Om7rx3SRnmj2AJLTM0+nrbz9dshahfrUGdoElTyojVbYXdHuB3eKLGZptOSWNHWNd5XDcHAp6+bs0GBKwAObALgZP6og40rBC9TSylD+DrxzoA0mXz1KY51+kkORzB0xEtG8oO9n/d+oamfJwOWPy4mfGbhQ7IaOlbjHKksNDF X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2CwAgD1YthV/61jaINeDoNhaQaDH7o3CoFtCoUxSgKBYRQBAQEBAQEBAYEJgh2CBwEBBAEBASAEJyALEAIBCA4KERkCAgIlAQkmAgQIBwQBGgIEiA0NrhOVOwEBAQEBAQEBAQEBAQEBAQEBFwSLV4QyBgEBGwEZFgUHgmmBQwWVNII/gkaFCYQsh0yJBIRJg2gCJoIOHIEVWiIzB38IFyOBBAEBAQ X-IronPort-AV: E=Sophos;i="5.15,728,1432612800"; d="scan'208";a="232435417" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 22 Aug 2015 07:59:18 -0400 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id C562115F55D; Sat, 22 Aug 2015 07:59:18 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id xdpi1x1-IhQx; Sat, 22 Aug 2015 07:59:17 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 7342815F563; Sat, 22 Aug 2015 07:59:17 -0400 (EDT) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id ewYCfPJexIum; Sat, 22 Aug 2015 07:59:17 -0400 (EDT) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 397B115F55D; Sat, 22 Aug 2015 07:59:17 -0400 (EDT) Date: Sat, 22 Aug 2015 07:59:16 -0400 (EDT) From: Rick Macklem To: Daniel Braniss Cc: pyunyh@gmail.com, Hans Petter Selasky , FreeBSD stable , FreeBSD Net , Christopher Forgeron , Gleb Smirnoff , Slawa Olhovchenkov Message-ID: <818666007.28930310.1440244756872.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <15D19823-08F7-4E55-BBD0-CE230F67D26E@cs.huji.ac.il> References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <55D429A4.3010407@selasky.org> <20150819074212.GB964@michelle.fasterthan.com> <55D43615.1030401@selasky.org> <2013503980.25726607.1439989235806.JavaMail.zimbra@uoguelph.ca> <20150820023024.GB996@michelle.fasterthan.com> <1153838447.28656490.1440193567940.JavaMail.zimbra@uoguelph.ca> <15D19823-08F7-4E55-BBD0-CE230F67D26E@cs.huji.ac.il> Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_28930308_990593625.1440244756870" X-Originating-IP: [172.17.95.11] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF34 (Win)/8.0.9_GA_6191) Thread-Topic: ix(intel) vs mlxen(mellanox) 10Gb performance Thread-Index: QZitH8l4Q7RLj+jgyGyTQ4NdEz1bXA== X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Aug 2015 11:59:21 -0000 ------=_Part_28930308_990593625.1440244756870 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Daniel Braniss wrote: >=20 > > On Aug 22, 2015, at 12:46 AM, Rick Macklem wrote= : > >=20 > > Yonghyeon PYUN wrote: > >> On Wed, Aug 19, 2015 at 09:00:35AM -0400, Rick Macklem wrote: > >>> Hans Petter Selasky wrote: > >>>> On 08/19/15 09:42, Yonghyeon PYUN wrote: > >>>>> On Wed, Aug 19, 2015 at 09:00:52AM +0200, Hans Petter Selasky wrote= : > >>>>>> On 08/18/15 23:54, Rick Macklem wrote: > >>>>>>> Ouch! Yes, I now see that the code that counts the # of mbufs is > >>>>>>> before > >>>>>>> the > >>>>>>> code that adds the tcp/ip header mbuf. > >>>>>>>=20 > >>>>>>> In my opinion, this should be fixed by setting if_hw_tsomaxsegcou= nt > >>>>>>> to > >>>>>>> whatever > >>>>>>> the driver provides - 1. It is not the driver's responsibility to > >>>>>>> know if > >>>>>>> a tcp/ip > >>>>>>> header mbuf will be added and is a lot less confusing that expect= ing > >>>>>>> the > >>>>>>> driver > >>>>>>> author to know to subtract one. (I had mistakenly thought that > >>>>>>> tcp_output() had > >>>>>>> added the tc/ip header mbuf before the loop that counts mbufs in = the > >>>>>>> list. > >>>>>>> Btw, > >>>>>>> this tcp/ip header mbuf also has leading space for the MAC layer > >>>>>>> header.) > >>>>>>>=20 > >>>>>>=20 > >>>>>> Hi Rick, > >>>>>>=20 > >>>>>> Your question is good. With the Mellanox hardware we have separate > >>>>>> so-called inline data space for the TCP/IP headers, so if the TCP > >>>>>> stack > >>>>>> subtracts something, then we would need to add something to the li= mit, > >>>>>> because then the scatter gather list is only used for the data par= t. > >>>>>>=20 > >>>>>=20 > >>>>> I think all drivers in tree don't subtract 1 for > >>>>> if_hw_tsomaxsegcount. Probably touching Mellanox driver would be > >>>>> simpler than fixing all other drivers in tree. > >>>>>=20 > >>>>>> Maybe it can be controlled by some kind of flag, if all the three = TSO > >>>>>> limits should include the TCP/IP/ethernet headers too. I'm pretty = sure > >>>>>> we want both versions. > >>>>>>=20 > >>>>>=20 > >>>>> Hmm, I'm afraid it's already complex. Drivers have to tell almost > >>>>> the same information to both bus_dma(9) and network stack. > >>>>=20 > >>>> Don't forget that not all drivers in the tree set the TSO limits bef= ore > >>>> if_attach(), so possibly the subtraction of one TSO fragment needs t= o go > >>>> into ip_output() .... > >>>>=20 > >>> Ok, I realized that some drivers may not know the answers before > >>> ether_ifattach(), > >>> due to the way they are configured/written (I saw the use of > >>> if_hw_tsomax_update() > >>> in the patch). > >>=20 > >> I was not able to find an interface that configures TSO parameters > >> after if_t conversion. I'm under the impression > >> if_hw_tsomax_update() is not designed to use this way. Probably we > >> need a better one?(CCed to Gleb). > >>=20 > >>>=20 > >>> If it is subtracted as a part of the assignment to if_hw_tsomaxsegcou= nt > >>> in > >>> tcp_output() > >>> at line#791 in tcp_output() like the following, I don't think it shou= ld > >>> matter if the > >>> values are set before ether_ifattach()? > >>> =09=09=09/* > >>> =09=09=09 * Subtract 1 for the tcp/ip header mbuf that > >>> =09=09=09 * will be prepended to the mbuf chain in this > >>> =09=09=09 * function in the code below this block. > >>> =09=09=09 */ > >>> =09=09=09if_hw_tsomaxsegcount =3D tp->t_tsomaxsegcount - 1; > >>>=20 > >>> I don't have a good solution for the case where a driver doesn't plan= on > >>> using the > >>> tcp/ip header provided by tcp_output() except to say the driver can a= dd > >>> one > >>> to the > >>> setting to compensate for that (and if they fail to do so, it still > >>> works, > >>> although > >>> somewhat suboptimally). When I now read the comment in sys/net/if_var= .h > >>> it > >>> is clear > >>> what it means, but for some reason I didn't read it that way before? = (I > >>> think it was > >>> the part that said the driver didn't have to subtract for the headers > >>> that > >>> confused me?) > >>> In any case, we need to try and come up with a clear definition of wh= at > >>> they need to > >>> be set to. > >>>=20 > >>> I can now think of two ways to deal with this: > >>> 1 - Leave tcp_output() as is, but provide a macro for the device driv= er > >>> authors to use > >>> that sets if_hw_tsomaxsegcount with a flag for "driver uses tcp/ip > >>> header mbuf", > >>> documenting that this flag should normally be true. > >>> OR > >>> 2 - Change tcp_output() as above, noting that this is a workaround fo= r > >>> confusion w.r.t. > >>> whether or not if_hw_tsomaxsegcount should include the tcp/ip head= er > >>> mbuf and > >>> update the comment in if_var.h to reflect this. Then drivers that > >>> don't > >>> use the > >>> tcp/ip header mbuf can increase their value for if_hw_tsomaxsegcou= nt > >>> by > >>> 1. > >>> (The comment should also mention that a value of 35 or greater is = much > >>> preferred to > >>> 32 if the hardware will support that.) > >>>=20 > >>=20 > >> Both works for me. My preference is 2 just because it's very > >> common for most drivers that use tcp/ip header mbuf. > > Thanks for this comment. I tend to agree, both for the reason you state= and > > also > > because the patch is simple enough that it might qualify as an errata f= or > > 10.2. > >=20 > > I am hoping Daniel Braniss will be able to test the patch and let us kn= ow > > if it > > improves performance with TSO enabled? >=20 > send me the patch and I=E2=80=99ll test it ASAP. > =09danny >=20 Patch is attached. The one for head will also include an update to the comm= ent in sys/net/if_var.h, but that isn't needed for testing. Thanks for testing this, rick > >=20 > > rick > >=20 > >> _______________________________________________ > >> freebsd-stable@freebsd.org mailing list > >> https://lists.freebsd.org/mailman/listinfo/freebsd-stable > >> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.o= rg" > >>=20 >=20 > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" ------=_Part_28930308_990593625.1440244756870 Content-Type: text/x-patch; name=tsooutby1.patch Content-Disposition: attachment; filename=tsooutby1.patch Content-Transfer-Encoding: base64 LS0tIG5ldGluZXQvdGNwX291dHB1dC5jLnNhdgkyMDE1LTA4LTIyIDA3OjQ4OjEyLjAwMDAwMDAw MCAtMDQwMAorKysgbmV0aW5ldC90Y3Bfb3V0cHV0LmMJMjAxNS0wOC0yMiAwNzo1MDo1Mi4wMDAw MDAwMDAgLTA0MDAKQEAgLTc5NCw3ICs3OTQsMTMgQEAgc2VuZDoKIAogCQkJLyogZXh0cmFjdCBU U08gaW5mb3JtYXRpb24gKi8KIAkJCWlmX2h3X3Rzb21heCA9IHRwLT50X3Rzb21heDsKLQkJCWlm X2h3X3Rzb21heHNlZ2NvdW50ID0gdHAtPnRfdHNvbWF4c2VnY291bnQ7CisJCQkvKgorCQkJICog U3VidHJhY3QgMSBmb3IgdGhlIHRjcC9pcCBoZWFkZXIgbWJ1ZiB0aGF0CisJCQkgKiB3aWxsIGJl IHByZXBlbmRlZCB0byB0aGlzIG1idWYgY2hhaW4gYWZ0ZXIKKwkJCSAqIHRoZSBjb2RlIGluIHRo aXMgc2VjdGlvbiBsaW1pdHMgdGhlIG51bWJlciBvZgorCQkJICogbWJ1ZnMgaW4gdGhlIGNoYWlu IHRvIGlmX2h3X3Rzb21heHNlZ2NvdW50LgorCQkJICovCisJCQlpZl9od190c29tYXhzZWdjb3Vu dCA9IHRwLT50X3Rzb21heHNlZ2NvdW50IC0gMTsKIAkJCWlmX2h3X3Rzb21heHNlZ3NpemUgPSB0 cC0+dF90c29tYXhzZWdzaXplOwogCiAJCQkvKgo= ------=_Part_28930308_990593625.1440244756870-- From owner-freebsd-stable@freebsd.org Sat Aug 22 13:32:13 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4D8E99BD899 for ; Sat, 22 Aug 2015 13:32:13 +0000 (UTC) (envelope-from joh.hendriks@gmail.com) Received: from mail-wi0-x229.google.com (mail-wi0-x229.google.com [IPv6:2a00:1450:400c:c05::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D33341289 for ; Sat, 22 Aug 2015 13:32:12 +0000 (UTC) (envelope-from joh.hendriks@gmail.com) Received: by wicja10 with SMTP id ja10so35670051wic.1 for ; Sat, 22 Aug 2015 06:32:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=to:from:subject:message-id:date:user-agent:mime-version :content-type:content-transfer-encoding; bh=kD1wb7q4MiE9g8VowWHy4iwJBx4YBCea7IJiDecJ4BU=; b=f2ZKSn5PcmWkZcEENYvAMep3i/jUsqyOLBAQ6D6Q92r07B/hfJV6QUfKwI9WDNJZiW 8h2ViNo3q7dU9cWwv1r54EUycba+T7jjr3FDdCDmGK0xdNnOAdCNYbwnpMWVCUZJ5hOn f09iZuxXfI0BIiggdEN7IpfXFg1oM43J6d6uU8CQGQ+46pILuuLbUpSMMd5fvHdisUL/ iwFvno79awo+uFhyzHAAsCiijcjLCktmkE816iF8C2LweB11WR/qotOqWucksmbeiyJm JmSu59PKkTs0ps27iG1tZO6ASjA+9SYR79Ct3TDZByT2sDINyizoWJSXkxjl/86Sk8a2 74Dw== X-Received: by 10.194.57.19 with SMTP id e19mr26312223wjq.152.1440250331233; Sat, 22 Aug 2015 06:32:11 -0700 (PDT) Received: from Johans-MacBook-Air.local (92-70-102-130.glasvezel.netexpo.nl. [92.70.102.130]) by smtp.googlemail.com with ESMTPSA id e8sm7408623wiz.0.2015.08.22.06.32.10 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 22 Aug 2015 06:32:10 -0700 (PDT) To: freebsd-stable@freebsd.org From: Johan Hendriks Subject: SSH Chroot FreeBSD 10.1 and 10.2 X-Enigmail-Draft-Status: N1110 Message-ID: <55D879DA.1070407@gmail.com> Date: Sat, 22 Aug 2015 15:32:10 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Aug 2015 13:32:13 -0000 Hello all. I want to use the Chrootdirctory feature of openssh on FreeBSD 10.2 And I tried it on 10.1 but gave up... Whatever I do I can not make it work on 10 without error messages, but I got it working on FreeBSD 8 This is what I have in my /etc/ssh/sshd_config file. # Example of overriding settings on a per-user basis Match User testuser1 ChrootDirectory /restricted/%u X11Forwarding no AllowTcpForwarding no I created the dir /restricted and the directory testuser1 the permissions are set to root owned. I created the directory /bin inside /restricted/testuser1 and put the sh file from /rescue there If I log on to the system I get the following ssh testuser1@192.168.1.14 Password for testuser1@node_1: Last login: Sat Aug 22 17:05:52 2015 from 192.168.1.13 Could not chdir to home directory /restricted/testuser1: No such file or directory Cannot read termcap database; using dumb terminal settings. % >From here I can do ls and so on if I copy ls, mkdir and other programs from /rescue to /restricted/username/bin , and can not escape my home, this is what I want but the error messages are frustrating. If I change to csh in /etc/passwd it gives me the following sh testuser1@192.168.1.14 Password for testuser1@node_1: Last login: Sat Aug 22 17:16:32 2015 from 192.168.1.13 Could not chdir to home directory /restricted/testuser1: No such file or directory csh: Cannot open /etc/termcap. csh: using dumb terminal settings. % I think I followed all the tutorials on the internet, and now I get to the point it gets really frustrating. :D I think I do something wrong, but I can not find it. Is there someone who got this working on FreeBSD 10, I have it working on my linux machines also without problem. Thank you for your time. regards Johan From owner-freebsd-stable@freebsd.org Sat Aug 22 13:45:57 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B2E6A9BDBCC for ; Sat, 22 Aug 2015 13:45:57 +0000 (UTC) (envelope-from allbery.b@gmail.com) Received: from mail-la0-x232.google.com (mail-la0-x232.google.com [IPv6:2a00:1450:4010:c03::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 418CC19C6 for ; Sat, 22 Aug 2015 13:45:57 +0000 (UTC) (envelope-from allbery.b@gmail.com) Received: by lalv9 with SMTP id v9so55238099lal.0 for ; Sat, 22 Aug 2015 06:45:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=3B6rdPSB1SHkoJ7+PP8ImH6zFMpCZkaZZq80VeZGCv4=; b=Cn3wOWa21na+0VirlTBT4CDzrV9v0ByNpjws0UX0R05PUuixptTmpclaKNWY4Ri04L eW8Xg0i6hscQ3bPfyE/z3OPEXnFi0weYNa5MVH3bjnfCYEDXDDj1pwTcPUI4ruGH5tZs DyrOMxrSXI8T+qK0OJe/22FWyaFS6NzUsvI6gH2K50JOUvp9409genQrnvBtGupJqExe 0Tp5/yCE+OLkOhRu8RinZpPxAe+z2YRHRDjU3SSBODDJ3OwSIjCq+FWiDiI5C4FiKzwd HgNWF3cUtxaKN9afpO/SfwtFb4y49GAUfj6Hugy34ron1CnnDUdkA/XVk+ECR5f8Cp5r yL2g== MIME-Version: 1.0 X-Received: by 10.112.167.202 with SMTP id zq10mr12425191lbb.69.1440251155293; Sat, 22 Aug 2015 06:45:55 -0700 (PDT) Received: by 10.25.134.198 with HTTP; Sat, 22 Aug 2015 06:45:55 -0700 (PDT) In-Reply-To: <55D879DA.1070407@gmail.com> References: <55D879DA.1070407@gmail.com> Date: Sat, 22 Aug 2015 09:45:55 -0400 Message-ID: Subject: Re: SSH Chroot FreeBSD 10.1 and 10.2 From: Brandon Allbery To: Johan Hendriks Cc: freebsd-stable Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Aug 2015 13:45:57 -0000 On Sat, Aug 22, 2015 at 9:32 AM, Johan Hendriks wrote: > Last login: Sat Aug 22 17:05:52 2015 from 192.168.1.13 > Could not chdir to home directory /restricted/testuser1: No such file or > directory > Cannot read termcap database; > using dumb terminal settings. > % > From here I can do ls and so on if I copy ls, mkdir and other programs > from /rescue to /restricted/username/bin , and can not escape my home, > this is what I want but the error messages are frustrating. > You have the chroot directory both as a chroot directory and a home directory. This means that the *actual* home directory, as seen from outside the chroot, is /restricted/testuser1/restricted/testuser1. (Home directory is *inside* the chroot directory and therefore relative to it.) The termcap message should be self-explanatory; you're missing /etc/termcap inside the chroot. chroot is what it says on the tin: once set, the specified directory is "/". Every file accessed from that point on MUST be available from a tree in which the specified chroot directory is "/". This includes symlinks --- symlink resolution doesn't get to see outside the specified "/" any more than anything else running in the chroot does, so you cannot simply symlink to a file outside the chroot. (Hard links are fine, since they are actually by inode number; they just have to be on the same partition.) -- brandon s allbery kf8nh sine nomine associates allbery.b@gmail.com ballbery@sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net From owner-freebsd-stable@freebsd.org Sat Aug 22 14:54:24 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2E19E9BF6AA for ; Sat, 22 Aug 2015 14:54:24 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: from mail.ultra-secure.de (mail.ultra-secure.de [88.198.178.88]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 941421710 for ; Sat, 22 Aug 2015 14:54:23 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: (qmail 41340 invoked by uid 89); 22 Aug 2015 14:54:16 -0000 Received: from unknown (HELO ?192.168.1.200?) (rainer@ultra-secure.de@217.71.83.52) by mail.ultra-secure.de with ESMTPA; 22 Aug 2015 14:54:16 -0000 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) Subject: Re: SSH Chroot FreeBSD 10.1 and 10.2 From: Rainer Duffner In-Reply-To: Date: Sat, 22 Aug 2015 16:54:09 +0200 Cc: Johan Hendriks , freebsd-stable Content-Transfer-Encoding: quoted-printable Message-Id: References: <55D879DA.1070407@gmail.com> To: Brandon Allbery X-Mailer: Apple Mail (2.2104) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Aug 2015 14:54:24 -0000 > Am 22.08.2015 um 15:45 schrieb Brandon Allbery : >=20 > On Sat, Aug 22, 2015 at 9:32 AM, Johan Hendriks = > wrote: >=20 > chroot is what it says on the tin: once set, the specified directory = is > "/". Every file accessed from that point on MUST be available from a = tree > in which the specified chroot directory is "/". This includes symlinks = --- > symlink resolution doesn't get to see outside the specified "/" any = more > than anything else running in the chroot does, so you cannot simply = symlink > to a file outside the chroot. (Hard links are fine, since they are = actually > by inode number; they just have to be on the same partition.) I found it=E2=80=99s much easier to have actual chroot=E2=80=99ed ssh = users once the users themselves are in an LDAP-directory. Also, for doing anything useful on that shell, it turned out you need a = some more devices in /dev than the usual chroot (like a chroot=E2=80=99ed = PHP-FPM, that just needs the dev-set of jail(4)). And a couple of symlinks. I=E2=80=99ve done this once for a customer (chroot=E2=80=99ed ssh = accounts) and unless this gets more easier in the future, I=E2=80=99ve = made a note to myself to not do that again any time soon. I hadn=E2=80=99t thought of just using /rescue (I would nullfs-mount it = into your target-directory, else you=E2=80=99ve got to copy it again = every time you run freebsd-update). But in my php-fpm chroots, I also need stuff from packages (ImageMagick, = most notably). I end up nullfs-mounting most of the system (except /sbin directories) = into the various chroots, but I was always looking for a better = approach. It=E2=80=99s all a bit of an hack, with lots of stuff borrowed from = ezjail ;-) The big advantage of using nullfs mounts is that I don=E2=80=99t have to = think about updating the chroots if I update the packages (except = /var/run/ld-elf*). Thinking about this: now that we have pkg - would pkg -c (chroot) also = create the SQLite DB inside the chroot? Regards, Rainer= From owner-freebsd-stable@freebsd.org Sat Aug 22 15:01:58 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5FB7C9BF850 for ; Sat, 22 Aug 2015 15:01:58 +0000 (UTC) (envelope-from allbery.b@gmail.com) Received: from mail-lb0-x22c.google.com (mail-lb0-x22c.google.com [IPv6:2a00:1450:4010:c04::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DBA7B1AA5 for ; Sat, 22 Aug 2015 15:01:57 +0000 (UTC) (envelope-from allbery.b@gmail.com) Received: by lbbsx3 with SMTP id sx3so58634069lbb.0 for ; Sat, 22 Aug 2015 08:01:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Gdaz6Cs8QbU5wz3iYWdXURkxVtZfSLR9HuVgi5kKgwA=; b=j1s+vOZKWCR7rwXg3G14SlK+BK73lTlaFRlIhWr3XR/Ve8qO3TNH1AV9WO4fjTHd/I LWCrOU2doNQ6cqX0xL5QJtyCSo100DuNlWR4T2rtDOGa09xd68iEzRLU4SvunA7VRAdp wJyBTYT3LtIlQLaRLXjq/gwBZZmZ6AJ3kWKh69y4FXtqwgVgWrUnQtroRZykkRCfRl35 HA3I7lVTWk9d76fphLnssKpYnwTY2tLZcuyZRImtJOeRd4XYn+nGV3tkI1Ixh8IKCygR +tpIT6vYmTRO0EjZzMBxuRsjWOMvKIhFAP9ki3L2+oy9m5C+3aqAeAIOy5EiJWj7+dJE ckiQ== MIME-Version: 1.0 X-Received: by 10.112.161.137 with SMTP id xs9mr12884178lbb.4.1440255714844; Sat, 22 Aug 2015 08:01:54 -0700 (PDT) Received: by 10.25.134.198 with HTTP; Sat, 22 Aug 2015 08:01:54 -0700 (PDT) In-Reply-To: References: <55D879DA.1070407@gmail.com> Date: Sat, 22 Aug 2015 11:01:54 -0400 Message-ID: Subject: Re: SSH Chroot FreeBSD 10.1 and 10.2 From: Brandon Allbery To: Rainer Duffner Cc: Johan Hendriks , freebsd-stable Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Aug 2015 15:01:58 -0000 On Sat, Aug 22, 2015 at 10:54 AM, Rainer Duffner wrote: > I found it=E2=80=99s much easier to have actual chroot=E2=80=99ed ssh use= rs once the users > themselves are in an LDAP-directory. > Also, for doing anything useful on that shell, it turned out you need a > some more devices in /dev than the usual chroot (like a chroot=E2=80=99ed= PHP-FPM, > that just needs the dev-set of jail(4)). > And a couple of symlinks. > Yep; chroots are always a pain to deal with. I have seen utilities to manage them, but only for Linux. --=20 brandon s allbery kf8nh sine nomine associate= s allbery.b@gmail.com ballbery@sinenomine.ne= t unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.ne= t From owner-freebsd-stable@freebsd.org Sat Aug 22 15:31:11 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DD33D9BFC0D for ; Sat, 22 Aug 2015 15:31:11 +0000 (UTC) (envelope-from dot.yet@gmail.com) Received: from mail-ob0-x242.google.com (mail-ob0-x242.google.com [IPv6:2607:f8b0:4003:c01::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9A519754 for ; Sat, 22 Aug 2015 15:31:11 +0000 (UTC) (envelope-from dot.yet@gmail.com) Received: by obbfr1 with SMTP id fr1so6359566obb.2 for ; Sat, 22 Aug 2015 08:31:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=X2tIlhs6R6VyoxDLzy8s8+QxjNHaCaLoSJ/Ay5pznz0=; b=x+zKeK3pmnGmBrihfz+rHmrp12CwtXuMiaWDso6oNoY//6BKJI8fmYnuHtwxpnTm8K tbjQVXE9WmkrMVsv5vBecf+Vwq3LqeOTU9kWnxjRTAlfasPsfHBmT1Y7ZB5t0dajmwil MGqruMNNQMIILs5fBrRvx2+VCYNHErEOE7VNqlIB3ud84oU/Emttk6HgGHCLOGCD4bgv V2PbP9hR0dB4n53/Hv+JPgOqLdQrUgDhxD2n82WJ/nPKNkh4yfI36cJtWuL5tBtNdkNz C/5y5x3SE8SpUVr2GvHcCC5k5ynP3SwhYvLkthcQGrVZL7RORhStdCmpsHZXHGO8j5c9 B44A== MIME-Version: 1.0 X-Received: by 10.60.103.7 with SMTP id fs7mr13834877oeb.73.1440257470962; Sat, 22 Aug 2015 08:31:10 -0700 (PDT) Received: by 10.76.20.105 with HTTP; Sat, 22 Aug 2015 08:31:10 -0700 (PDT) In-Reply-To: References: <55D4719D.60709@ze.tum.de> Date: Sat, 22 Aug 2015 11:31:10 -0400 Message-ID: Subject: Re: Installing FreeBSD on MacPro From: Dot Yet To: Dot Yet Cc: FreeBSD stable Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Aug 2015 15:31:12 -0000 you can try one more thing. If there is a cd/dvd drive attached, see if removing the sata connectors for that helps avoid the ahci timeouts. it may not be the ssd at all. Thx. On Wed, Aug 19, 2015 at 9:19 AM, Dot Yet wrote: > Seems to be related to be related to your SSD drive or the SATA control. > do you a secondary SSD or HDD to try on this machine? I've seen this happen > on my Intel SSD in the past. > > Thx > . > > On Wed, Aug 19, 2015 at 8:07 AM, Gerhard Schmidt > wrote: > >> Hi, >> >> I'm trying to install FreeBSD 10.2 on a MacPro. >> >> Installing and Booting works. But a few seconds after login and accessing >> the Harddisk, I'm getting the following errors. >> ahcich0: Timeout on slot 13 port 0 >> ahcich0: is 00000008 cs 00000000 ss 00000000 rs ffffffff tfd 40 serr >> 00000000 cmd 0000cd17 >> (ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 00 28 15 40 40 2f 00 00 >> 01 00 00 >> (ada0:ahcich0:0:0:0): CAM status: Command timeout >> (ada0:ahcich0:0:0:0): Retrying command >> ahcich0: Timeout on slot 15 port 0 >> ahcich0: is 00000008 cs 00000000 ss 00000000 rs ffffffff tfd 40 serr >> 00000000 cmd 0000cf17 >> (ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 a0 5f bd 40 13 00 00 >> 00 00 00 >> (ada0:ahcich0:0:0:0): CAM status: Command timeout >> (ada0:ahcich0:0:0:0): Retrying command >> ahcich0: Timeout on slot 2 port 0 >> ahcich0: is 00000008 cs 00000000 ss 00000000 rs ffffffff tfd 40 serr >> 00000000 cmd 0000c217 >> (ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 c0 23 bd 40 13 00 00 >> 00 00 00 >> (ada0:ahcich0:0:0:0): CAM status: Command timeout >> (ada0:ahcich0:0:0:0): Retrying command >> >> Any Ideas what causes this and what can be done to fix it. >> >> the dmesg of the MacPro is the following >> >> Copyright (c) 1992-2015 The FreeBSD Project. >> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 >> The Regents of the University of California. All rights reserved. >> FreeBSD is a registered trademark of The FreeBSD Foundation. >> FreeBSD 10.2-RELEASE #0 r286666: Wed Aug 12 15:26:37 UTC 2015 >> root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 >> FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 >> VT: running with driver "efifb". >> CPU: Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz (2700.06-MHz K8-class CPU) >> Origin="GenuineIntel" Id=0x306e4 Family=0x6 Model=0x3e Stepping=4 >> >> >> Features=0xbfebfbff >> >> >> Features2=0x7fbee3ff >> AMD Features=0x2c100800 >> AMD Features2=0x1 >> Structured Extended Features=0x281 >> XSAVE Features=0x1 >> VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr >> TSC: P-state invariant, performance statistics >> real memory = 68719476736 (65536 MB) >> avail memory = 66655334400 (63567 MB) >> Event timer "LAPIC" quality 600 >> ACPI APIC Table: >> FreeBSD/SMP: Multiprocessor System Detected: 24 CPUs >> FreeBSD/SMP: 1 package(s) x 12 core(s) x 2 SMT threads >> cpu0 (BSP): APIC ID: 0 >> cpu1 (AP): APIC ID: 1 >> cpu2 (AP): APIC ID: 2 >> cpu3 (AP): APIC ID: 3 >> cpu4 (AP): APIC ID: 4 >> cpu5 (AP): APIC ID: 5 >> cpu6 (AP): APIC ID: 6 >> cpu7 (AP): APIC ID: 7 >> cpu8 (AP): APIC ID: 8 >> cpu9 (AP): APIC ID: 9 >> cpu10 (AP): APIC ID: 10 >> cpu11 (AP): APIC ID: 11 >> cpu12 (AP): APIC ID: 16 >> cpu13 (AP): APIC ID: 17 >> cpu14 (AP): APIC ID: 18 >> cpu15 (AP): APIC ID: 19 >> cpu16 (AP): APIC ID: 20 >> cpu17 (AP): APIC ID: 21 >> cpu18 (AP): APIC ID: 22 >> cpu19 (AP): APIC ID: 23 >> cpu20 (AP): APIC ID: 24 >> cpu21 (AP): APIC ID: 25 >> cpu22 (AP): APIC ID: 26 >> cpu23 (AP): APIC ID: 27 >> ioapic0 irqs 0-23 on motherboard >> ioapic1 irqs 24-47 on motherboard >> lapic0: Forcing LINT1 to edge trigger >> random: initialized >> module_register_init: MOD_LOAD (vesa, 0xffffffff80db8eb0, 0) error 19 >> kbd0 at kbdmux0 >> acpi0: on motherboard >> acpi0: Power Button (fixed) >> unknown: I/O range not supported >> Timecounter "HPET" frequency 14318180 Hz quality 950 >> Event timer "HPET" frequency 14318180 Hz quality 350 >> Event timer "HPET1" frequency 14318180 Hz quality 340 >> Event timer "HPET2" frequency 14318180 Hz quality 340 >> Event timer "HPET3" frequency 14318180 Hz quality 340 >> Event timer "HPET4" frequency 14318180 Hz quality 340 >> Event timer "HPET5" frequency 14318180 Hz quality 340 >> Event timer "HPET6" frequency 14318180 Hz quality 340 >> Event timer "HPET7" frequency 14318180 Hz quality 340 >> cpu0: on acpi0 >> cpu1: on acpi0 >> cpu2: on acpi0 >> cpu3: on acpi0 >> cpu4: on acpi0 >> cpu5: on acpi0 >> cpu6: on acpi0 >> cpu7: on acpi0 >> cpu8: on acpi0 >> cpu9: on acpi0 >> cpu10: on acpi0 >> cpu11: on acpi0 >> cpu12: on acpi0 >> cpu13: on acpi0 >> cpu14: on acpi0 >> cpu15: on acpi0 >> cpu16: on acpi0 >> cpu17: on acpi0 >> cpu18: on acpi0 >> cpu19: on acpi0 >> cpu20: on acpi0 >> cpu21: on acpi0 >> cpu22: on acpi0 >> cpu23: on acpi0 >> atrtc0: port 0x70-0x77 on acpi0 >> Event timer "RTC" frequency 32768 Hz quality 0 >> attimer0: port 0x40-0x43,0x50-0x53 on acpi0 >> Timecounter "i8254" frequency 1193182 Hz quality 0 >> Event timer "i8254" frequency 1193182 Hz quality 100 >> Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 >> acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 >> acpi_ec0: port 0x62,0x66 on acpi0 >> pcib0: port 0xcf8-0xcff on acpi0 >> pci0: on pcib0 >> pcib1: at device 1.0 on pci0 >> pci16: on pcib1 >> pcib2: mem 0xbf900000-0xbf93ffff at device 0.0 on >> pci16 >> pci17: on pcib2 >> pcib3: at device 1.0 on pci17 >> pci18: on pcib3 >> xhci0: mem >> 0xa0900000-0xa090ffff,0xa0910000-0xa0910fff,0xa0911000-0xa0911fff at >> device 0.0 on pci18 >> xhci0: 32 bytes context size, 64-bit DMA >> usbus0 on xhci0 >> pcib4: at device 2.0 on pci17 >> pci20: on pcib4 >> pcib5: at device 8.0 on pci17 >> pci19: on pcib5 >> pcib6: at device 9.0 on pci17 >> pci91: on pcib6 >> pcib7: at device 10.0 on pci17 >> pci162: on pcib7 >> pcib8: at device 2.0 on pci0 >> pci2: on pcib8 >> vgapci0: port 0x3000-0x30ff mem >> 0x80000000-0x8fffffff,0xa0700000-0xa073ffff at device 0.0 on pci2 >> hdac0: mem 0xa0760000-0xa0763fff at device 0.1 >> on pci2 >> hdac0: hdac_get_capabilities: Invalid corb size (0) >> device_attach: hdac0 attach returned 6 >> pcib9: at device 3.0 on pci0 >> pci6: on pcib9 >> vgapci1: port 0x2000-0x20ff mem >> 0x90000000-0x9fffffff,0xa0600000-0xa063ffff at device 0.0 on pci6 >> hdac0: mem 0xa0660000-0xa0663fff at device 0.1 >> on pci6 >> hdac0: hdac_get_capabilities: Invalid corb size (0) >> device_attach: hdac0 attach returned 6 >> pcib10: at device 17.0 on pci0 >> pci10: on pcib10 >> pci0: at device 22.0 (no driver attached) >> hdac0: mem 0xa0820000-0xa0823fff at >> device 27.0 on pci0 >> pcib11: at device 28.0 on pci0 >> pci11: on pcib11 >> bge0: > 0x57766000> mem >> 0xa0300000-0xa030ffff,0xa0310000-0xa031ffff at device 0.0 on pci11 >> bge0: CHIP ID 0x57766000; ASIC REV 0x57766; CHIP REV 0x577660; PCI-E >> miibus0: on bge0 >> brgphy0: PHY 1 on miibus0 >> brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, >> 1000baseT-master, 1000baseT-FDX, >> 1000baseT-FDX-master, auto, auto-flow >> bge0: Using defaults for TSO: 65518/35/2048 >> bge0: Ethernet address: 00:3e:e1:c2:1a:e3 >> pcib12: at device 28.1 on pci0 >> pci12: on pcib12 >> bge1: > 0x57766000> mem >> 0xa0400000-0xa040ffff,0xa0410000-0xa041ffff at device 0.0 on pci12 >> bge1: CHIP ID 0x57766000; ASIC REV 0x57766; CHIP REV 0x577660; PCI-E >> miibus1: on bge1 >> brgphy1: PHY 1 on miibus1 >> brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, >> 1000baseT-master, 1000baseT-FDX, >> 1000baseT-FDX-master, auto, auto-flow >> bge1: Using defaults for TSO: 65518/35/2048 >> bge1: Ethernet address: 00:3e:e1:c2:1a:e2 >> pcib13: at device 28.2 on pci0 >> pci13: on pcib13 >> pci13: at device 0.0 (no driver attached) >> pcib14: at device 28.4 on pci0 >> pci14: on pcib14 >> ahci0: mem 0xa0500000-0xa0501fff at device 0.0 on >> pci14 >> ahci0: AHCI v1.30 with 1 6Gbps ports, Port Multiplier not supported >> ahcich0: at channel 0 on ahci0 >> ehci0: mem 0xa0827800-0xa0827bff at >> device 29.0 on pci0 >> usbus1: EHCI version 1.0 >> usbus1 on ehci0 >> pcib15: at device 30.0 on pci0 >> pci15: on pcib15 >> isab0: at device 31.0 on pci0 >> isa0: on isab0 >> acpi_button0: on acpi0 >> acpi_button1: on acpi0 >> ppc0: cannot reserve I/O port range >> uart0: <8250 or 16450 or compatible> at port 0x3f8-0x3ff irq 4 flags 0x10 >> on isa0 >> est0: on cpu0 >> est1: on cpu1 >> est2: on cpu2 >> est3: on cpu3 >> est4: on cpu4 >> est5: on cpu5 >> est6: on cpu6 >> est7: on cpu7 >> est8: on cpu8 >> est9: on cpu9 >> est10: on cpu10 >> est11: on cpu11 >> est12: on cpu12 >> est13: on cpu13 >> est14: on cpu14 >> est15: on cpu15 >> est16: on cpu16 >> est17: on cpu17 >> est18: on cpu18 >> est19: on cpu19 >> est20: on cpu20 >> est21: on cpu21 >> est22: on cpu22 >> est23: on cpu23 >> random: unblocking device. >> usbus0: 5.0Gbps Super Speed USB v3.0 >> Timecounters tick every 1.000 msec >> hdacc0: at cad 0 on hdac0 >> hdaa0: at nid 1 on hdacc0 >> pcm0: at nid 18 and 24 on hdaa0 >> pcm1: at nid 16 on hdaa0 >> pcm2: at nid 19 on hdaa0 >> pcm3: at nid 33 on hdaa0 >> usbus1: 480Mbps High Speed USB v2.0 >> ugen0.1: <0x1b73> at usbus0 >> uhub0: <0x1b73 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0 >> ugen1.1: at usbus1 >> uhub1: on usbus1 >> ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 >> ada0: ATA8-ACS SATA 3.x device >> ada0: Serial Number S1FVNYAF903719 >> ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes) >> ada0: Command Queueing enabled >> ada0: 954204MB (1954210120 512 byte sectors: 16H 63S/T 16383C) >> ada0: Previously was known as ad4 >> lapic1: Forcing LINT1 to edge trigger >> SMP: AP CPU #1 Launched! >> lapic6: Forcing LINT1 to edge trigger >> SMP: AP CPU #6 Launched! >> lapic7: Forcing LINT1 to edge trigger >> SMP: AP CPU #7 Launched! >> lapic3: Forcing LINT1 to edge trigger >> SMP: AP CPU #3 Launched! >> lapic19: Forcing LINT1 to edge trigger >> SMP: AP CPU #15 Launched! >> lapic22: Forcing LINT1 to edge trigger >> SMP: AP CPU #18 Launched! >> lapic26: Forcing LINT1 to edge trigger >> SMP: AP CPU #22 Launched! >> lapic23: Forcing LINT1 to edge trigger >> SMP: AP CPU #19 Launched! >> lapic9: Forcing LINT1 to edge trigger >> SMP: AP CPU #9 Launched! >> lapic27: Forcing LINT1 to edge trigger >> SMP: AP CPU #23 Launched! >> lapic24: Forcing LINT1 to edge trigger >> SMP: AP CPU #20 Launched! >> lapic25: Forcing LINT1 to edge trigger >> SMP: AP CPU #21 Launched! >> lapic10: Forcing LINT1 to edge trigger >> SMP: AP CPU #10 Launched! >> lapic4: Forcing LINT1 to edge trigger >> SMP: AP CPU #4 Launched! >> lapic5: Forcing LINT1 to edge trigger >> SMP: AP CPU #5 Launched! >> lapic18: Forcing LINT1 to edge trigger >> SMP: AP CPU #14 Launched! >> lapic8: Forcing LINT1 to edge trigger >> SMP: AP CPU #8 Launched! >> lapic11: Forcing LINT1 to edge trigger >> SMP: AP CPU #11 Launched! >> lapic17: Forcing LINT1 to edge trigger >> SMP: AP CPU #13 Launched! >> lapic2: Forcing LINT1 to edge trigger >> SMP: AP CPU #2 Launched! >> lapic16: Forcing LINT1 to edge trigger >> SMP: AP CPU #12 Launched! >> lapic21: Forcing LINT1 to edge trigger >> SMP: AP CPU #17 Launched! >> lapic20: Forcing LINT1 to edge trigger >> SMP: AP CPU #16 Launched! >> Timecounter "TSC-low" frequency 1350029784 Hz quality 1000 >> Root mount waiting for: usbus1 usbus0 >> uhub0: 8 ports with 8 removable, self powered >> uhub1: 2 ports with 2 removable, self powered >> ugen0.2: at usbus0 >> Root mount waiting for: usbus1 usbus0 >> ugen1.2: at usbus1 >> uhub2: >> on usbus1 >> ugen0.3: at usbus0 >> uhub3: >> on usbus0 >> uhub3: 1 port with 1 removable, bus powered >> Root mount waiting for: usbus1 usbus0 >> ugen0.4: at usbus0 >> uhub4: >> on usbus0 >> uhub4: MTT enabled >> uhub2: 8 ports with 7 removable, self powered >> uhub4: 3 ports with 2 removable, self powered >> Root mount waiting for: usbus1 usbus0 >> ugen1.3: at usbus1 >> uhub5: on >> usbus1 >> ugen0.5: at usbus0 >> hid_get_item: Number of items truncated to 255 >> uhub5: 3 ports with 0 removable, self powered >> Root mount waiting for: usbus1 usbus0 >> ugen1.4: at usbus1 >> ukbd0: >> on usbus1 >> kbd1 at ukbd0 >> ugen0.6: at usbus0 >> uhub6: on >> usbus0 >> uhub6: 3 ports with 2 removable, bus powered >> ugen1.5: at usbus1 >> Root mount waiting for: usbus1 usbus0 >> ugen0.7: at usbus0 >> ukbd1: on >> usbus0 >> kbd2 at ukbd1 >> ugen1.6: at usbus1 >> Trying to mount root from ufs:/dev/ada0p2 [rw]... >> ums0: on >> usbus0 >> ums1: on >> usbus1 >> ums1: 3 buttons and [XY] coordinates ID=2 >> ums0: 3 buttons and [XYZ] coordinates ID=0 >> hid_get_item: Number of items truncated to 255 >> hid_get_item: Number of items truncated to 255 >> uhid0: >> on usbus0 >> hid_get_item: Number of items truncated to 255 >> hid_get_item: Number of items truncated to 255 >> hid_get_item: Number of items truncated to 255 >> uhid1: on >> usbus0 >> ubt0: >> on usbus1 >> WARNING: attempt to domain_add(bluetooth) after domainfinalize() >> WARNING: attempt to domain_add(netgraph) after domainfinalize() >> ng_hci_send_command: ubt0hci - could not send HCI command, error=12 >> ng_hci_process_command_timeout: ubt0hci - unable to complete HCI command >> OGF=0x3, OCF=0x3. Timeout >> >> Regards >> Estartu >> >> -- >> _______________________________________________ >> freebsd-stable@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" >> > > From owner-freebsd-stable@freebsd.org Sat Aug 22 15:48:54 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8BB5C9BFFD4 for ; Sat, 22 Aug 2015 15:48:54 +0000 (UTC) (envelope-from rleigh@codelibre.net) Received: from b.painless.aa.net.uk (b.painless.aa.net.uk [IPv6:2001:8b0:0:30:5054:ff:fe5e:1643]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4EA64F4A for ; Sat, 22 Aug 2015 15:48:54 +0000 (UTC) (envelope-from rleigh@codelibre.net) Received: from e.8.e.2.2.0.5.b.2.2.2.8.e.e.0.5.d.b.d.d.0.6.8.0.0.b.8.0.1.0.0.2.ip6.arpa ([2001:8b0:860:ddbd:50ee:8222:b502:2e8e]) by b.painless.aa.net.uk with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.77) (envelope-from ) id 1ZTB2I-0007WB-6d for freebsd-stable@freebsd.org; Sat, 22 Aug 2015 16:48:50 +0100 Subject: Re: SSH Chroot FreeBSD 10.1 and 10.2 To: freebsd-stable@freebsd.org References: <55D879DA.1070407@gmail.com> From: Roger Leigh Message-ID: <55D899C4.30406@codelibre.net> Date: Sat, 22 Aug 2015 15:48:20 +0000 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Aug 2015 15:48:54 -0000 On 22/08/2015 15:01, Brandon Allbery wrote: > On Sat, Aug 22, 2015 at 10:54 AM, Rainer Duffner > wrote: > >> I found it’s much easier to have actual chroot’ed ssh users once the users >> themselves are in an LDAP-directory. >> Also, for doing anything useful on that shell, it turned out you need a >> some more devices in /dev than the usual chroot (like a chroot’ed PHP-FPM, >> that just needs the dev-set of jail(4)). >> And a couple of symlinks. >> > > Yep; chroots are always a pain to deal with. I have seen utilities to > manage them, but only for Linux. For your information, I'm in the process of porting my schroot chroot management tool to FreeBSD. https://github.com/codelibre-net/schroot This was traditionally a Linux (Debian) chroot tool for building source packages, but it's worked on Debian GNU/kFreeBSD for a good while so it already supported nullfs filesystem mounts e.g. of home directories and devices, and now the work to build it on FreeBSD proper is done--I was blocked on toolchain/linker bugs for the last 18 months until 10.2 came out (C++11 nullptr_t was broken) The master branch is current development work, and I got it all building on FreeBSD 10.2-RELEASE just yesterday. It's not yet actually *tested* on FreeBSD other than the unit tests pass. So it might not be production-ready right now, but it should be fairly soon. Now it's building, I'll also look at adding some FreeBSD-specific features to it as well, like ZFS snapshots, jail support, etc. While the compiled binaries should be fine, there may be residual Debianisms/GNU libc-isms in the setup scripts. They are likely trivial to fix though. If anyone wants to give it a try and provide some feedback, or if you have any suggestions or feature requests, please just let me know either by mail or at https://github.com/codelibre-net/schroot/issues Instructions for building on FreeBSD are in the README https://github.com/codelibre-net/schroot/blob/master/README.md Kind regards, Roger