From owner-freebsd-stable@freebsd.org Sun Oct 13 19:35:55 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1C68B13C135 for ; Sun, 13 Oct 2019 19:35:55 +0000 (UTC) (envelope-from ctuffli@gmail.com) Received: from mail-oi1-x230.google.com (mail-oi1-x230.google.com [IPv6:2607:f8b0:4864:20::230]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46rsP62Vgxz4S70 for ; Sun, 13 Oct 2019 19:35:54 +0000 (UTC) (envelope-from ctuffli@gmail.com) Received: by mail-oi1-x230.google.com with SMTP id k25so12071316oiw.13 for ; Sun, 13 Oct 2019 12:35:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=PLFxaJrR2IWzGDJ6WSxC8PLqzuxJaWY3tuHLpaZnV1U=; b=CVs/3LH+mbxwraE2v+dEqM7P51VqmQ+Mf163kmIt3noVSpdzc19qrkTXMNv0LB2BNW d4XdBtRIwS9CdWmI1DzTiEWyU5CAlfAD/MTRBFq1edlANVEbUf4C1zh8QrrAkyC5pw4e IvvYBsb1RX+ZzHEu3Zmy/eurL3RDzIrGDn7Nhs77f8hiNIJ6BTSWp26aaJm6YsFanx6j WUNv0Ci3wir7UU/aV18pR5Lf3WWNYY0HThQEipcKw1rmgI77uro1CF0XRref+/Ysd/C5 /1DvnP+QoF5eVTIWIIaHuSc3SI6njNXXy5r+RRzAwtXtbU8l6Z2VcVswl8Lbu3nQaY7O MYwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=PLFxaJrR2IWzGDJ6WSxC8PLqzuxJaWY3tuHLpaZnV1U=; b=Sup6N4Ae5P9P5I8ldZ9u7zIMZjLl1xbUC6BgPcKUuF15FrTbylOQq3WXaBgyS1Cdty /Ywcw5Qr4pSEOyjaE8PM3w+PUdueWz4hgnWmsQG9+lmhUljIx5j/LZ/sR5i8k5Of7kdx +D7mvkIy2RL4oSap3xJ1OCjSvVV6MApTRsK3oIIrVq27VLw5Hk04IMlaCIKZREfxcgIX A1PwuYRI+Ny4y7PfpxvenBgW1jlTGVW9pQgRm1r7iuFL7hg/oU0b9UWvqPbeCPsvdoPE g7I4P8yt6lzW7+WjpQ3l8hRQIGs1P4yhfJfryBgTVH2T032Z56BAUEAJx7YOF7xPxDiz ET4w== X-Gm-Message-State: APjAAAVlKDWmeAvYj50pHf7qUHV6UN4RAeNRmgjZfzCfUzMp6cOuDThX B2H1wNFewtMuu9QTcpr9SEbm1Lv/yLq04F3OqGVwVkyH X-Google-Smtp-Source: APXvYqxJP+pJ3JGGejFmPzsC8iFWBc2RLENlyoMif/qvw7iDqGqp80F9z9EmRmXCIyf3T0MyGJkTBa2AIQimJqzp/8o= X-Received: by 2002:aca:3001:: with SMTP id w1mr21809007oiw.178.1570995352428; Sun, 13 Oct 2019 12:35:52 -0700 (PDT) MIME-Version: 1.0 From: Chuck Tuffli Date: Sun, 13 Oct 2019 12:35:41 -0700 Message-ID: Subject: Confused about i915kms on 12.1 To: freebsd-stable@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 46rsP62Vgxz4S70 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=CVs/3LH+; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of ctuffli@gmail.com designates 2607:f8b0:4864:20::230 as permitted sender) smtp.mailfrom=ctuffli@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; TO_DN_NONE(0.00)[]; IP_SCORE_FREEMAIL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE(0.00)[ip: (-8.29), ipnet: 2607:f8b0::/32(-2.51), asn: 15169(-2.12), country: US(-0.05)]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[0.3.2.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Oct 2019 19:35:55 -0000 I've been running 12.0-REL on my X1 Carbon 3rd Gen happily with xorg, but after installing 12.1-RC1, the laptop reboots when loading the i915kms driver. If I comment out the line in /etc/rc.conf: kld_list="/boot/modules/i915kms.ko" the laptop boots, albeit without graphics. The above comes from the 12.0R release notes, and I didn't see any updates to this in the 12.1 release notes. I've tried drm-kmod from both ports and packages with the same result. Did something change with 12.1? What should I check? TIA --chuck From owner-freebsd-stable@freebsd.org Sun Oct 13 20:04:16 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9B794B3F09 for ; Sun, 13 Oct 2019 20:04:16 +0000 (UTC) (envelope-from pete@nomadlogic.org) Received: from mail.nomadlogic.org (mail.nomadlogic.org [174.136.98.114]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.nomadlogic.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46rt1p72vPz4V5x for ; Sun, 13 Oct 2019 20:04:14 +0000 (UTC) (envelope-from pete@nomadlogic.org) Received: from [192.168.1.206] (cpe-23-243-162-239.socal.res.rr.com [23.243.162.239]) by mail.nomadlogic.org (OpenSMTPD) with ESMTPSA id e7ba534d TLS version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO; Sun, 13 Oct 2019 20:04:07 +0000 (UTC) Subject: Re: Confused about i915kms on 12.1 To: Chuck Tuffli , freebsd-stable@freebsd.org References: From: Pete Wright Message-ID: Date: Sun, 13 Oct 2019 13:04:06 -0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.1.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 46rt1p72vPz4V5x X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of pete@nomadlogic.org designates 174.136.98.114 as permitted sender) smtp.mailfrom=pete@nomadlogic.org X-Spamd-Result: default: False [-4.91 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[239.162.243.23.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.10]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[nomadlogic.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; IP_SCORE(-2.61)[ip: (-9.23), ipnet: 174.136.96.0/20(-3.44), asn: 25795(-0.32), country: US(-0.05)]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:25795, ipnet:174.136.96.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Oct 2019 20:04:16 -0000 On 10/13/19 12:35 PM, Chuck Tuffli wrote: > I've been running 12.0-REL on my X1 Carbon 3rd Gen happily with xorg, > but after installing 12.1-RC1, the laptop reboots when loading the > i915kms driver. If I comment out the line in /etc/rc.conf: > > kld_list="/boot/modules/i915kms.ko" > > the laptop boots, albeit without graphics. The above comes from the > 12.0R release notes, and I didn't see any updates to this in the 12.1 > release notes. I've tried drm-kmod from both ports and packages with > the same result. Did something change with 12.1? What should I check? > TIA Best thing to do is rebuild the drm-fbsd12.0-kmod package from the ports tree.  i have had this issue as well while testing the BETA and RC builds for 12.1, and there is some breakage in the upstream packages causing this behavior.  removing the package, then doing a "make package" in the graphics/drm-fbsd12.0-kmod directory, then installing the resulting package in the work/pkg directory gets everything working again. -pete -- Pete Wright pete@nomadlogic.org @nomadlogicLA From owner-freebsd-stable@freebsd.org Sun Oct 13 20:12:59 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DE3B9B4417 for ; Sun, 13 Oct 2019 20:12:59 +0000 (UTC) (envelope-from warlock@phouka1.phouka.net) Received: from phouka1.phouka.net (phouka1.phouka.net [107.170.196.116]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "phouka.net", Issuer "Go Daddy Secure Certificate Authority - G2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 46rtCt1WRHz4Vbw for ; Sun, 13 Oct 2019 20:12:58 +0000 (UTC) (envelope-from warlock@phouka1.phouka.net) Received: from phouka1.phouka.net (localhost [127.0.0.1]) by phouka1.phouka.net (8.15.2/8.15.2) with ESMTPS id x9DKBZpp077886 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sun, 13 Oct 2019 13:11:36 -0700 (PDT) (envelope-from warlock@phouka1.phouka.net) Received: (from warlock@localhost) by phouka1.phouka.net (8.15.2/8.15.2/Submit) id x9DKBZcN077882; Sun, 13 Oct 2019 13:11:35 -0700 (PDT) (envelope-from warlock) Date: Sun, 13 Oct 2019 13:11:35 -0700 From: John Kennedy To: Chuck Tuffli Cc: freebsd-stable@freebsd.org Subject: Re: Confused about i915kms on 12.1 Message-ID: <20191013201135.GA91243@phouka1.phouka.net> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.12.2 (2019-09-21) X-Rspamd-Queue-Id: 46rtCt1WRHz4Vbw X-Spamd-Bar: + Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of warlock@phouka1.phouka.net has no SPF policy when checking 107.170.196.116) smtp.mailfrom=warlock@phouka1.phouka.net X-Spamd-Result: default: False [1.36 / 15.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; IP_SCORE(0.36)[ipnet: 107.170.192.0/18(0.14), asn: 14061(1.71), country: US(-0.05)]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[phouka.net]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.02)[0.021,0]; NEURAL_HAM_LONG(-0.22)[-0.224,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[warlock@phouka.net,warlock@phouka1.phouka.net]; FREEMAIL_TO(0.00)[gmail.com]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:14061, ipnet:107.170.192.0/18, country:US]; FROM_NEQ_ENVFROM(0.00)[warlock@phouka.net,warlock@phouka1.phouka.net]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Oct 2019 20:12:59 -0000 On Sun, Oct 13, 2019 at 12:35:41PM -0700, Chuck Tuffli wrote: > I've been running 12.0-REL on my X1 Carbon 3rd Gen happily with xorg, > but after installing 12.1-RC1, the laptop reboots when loading the > i915kms driver. If I comment out the line in /etc/rc.conf: > > kld_list="/boot/modules/i915kms.ko" > > the laptop boots, albeit without graphics. The above comes from the > 12.0R release notes, and I didn't see any updates to this in the 12.1 > release notes. I've tried drm-kmod from both ports and packages with > the same result. Did something change with 12.1? What should I check? > TIA Not sure how your upgraded your box, but make sure that the driver is compiled for the new kernel. KBI breakage can easily behave like that. I won't be in front of a box with that setup until Monday, but from memory, graphics/drm-kmod pulls in graphics/drm-fbsd12.0-kmod (at least on my system with it's build-in Intel video card). If you do something like this, but substitute your .ko file for m4 below, you should be able to validate the package it came from and the kernel version it was targeted at: $ pkg which /usr/local/bin/*m4 /usr/local/bin/gm4 was installed by package m4-1.4.18_1,1 $ pkg info m4 | grep -i version Version : 1.4.18_1,1 FreeBSD_version: 1201000 That 1201000 means that it was compiled for 12.1 in my case. Have you overridden the default package location, leaving it aimed at 12.0? What do you mean by trying it from "ports"? How did you refresh that? (I'm just fishing for something that would have survived the upgrade process and been locked onto 12.0 vs 12.1 and stopping you from getting a good, recompiled version installed.) From owner-freebsd-stable@freebsd.org Sun Oct 13 20:16:34 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 373BAB4668 for ; Sun, 13 Oct 2019 20:16:34 +0000 (UTC) (envelope-from ceeso@vip.163.com) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 46rtJ16flPz4VmQ for ; Sun, 13 Oct 2019 20:16:33 +0000 (UTC) (envelope-from ceeso@vip.163.com) Received: by mailman.nyi.freebsd.org (Postfix) id E46D1B4667; Sun, 13 Oct 2019 20:16:33 +0000 (UTC) Delivered-To: stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E431FB4666 for ; Sun, 13 Oct 2019 20:16:33 +0000 (UTC) (envelope-from ceeso@vip.163.com) Received: from m181-177.vip.163.com (m181-177.vip.163.com [123.58.177.181]) by mx1.freebsd.org (Postfix) with ESMTP id 46rtJ06Cpxz4VmP for ; Sun, 13 Oct 2019 20:16:32 +0000 (UTC) (envelope-from ceeso@vip.163.com) Received: from dell-pc (unknown [120.84.141.27]) by smtp2 (Coremail) with SMTP id oWZ4CgBXq8nnhKNdDN+dKg--.32723S2; Mon, 14 Oct 2019 04:11:20 +0800 (CST) Message-ID: <20191014041600616.ceeso@vip.163.com> MIME-Version: 1.0 From: =?utf-8?Q?Shirley_=EF=BC=A3eeso?= To: "Stable" Reply-To: s@ceesohk.com Date: 14 Oct 2019 04:16:01 +0800 Subject: Stable, How to choose/pick the Knives? X-CM-TRANSID: oWZ4CgBXq8nnhKNdDN+dKg--.32723S2 X-Coremail-Antispam: 1Uf129KBjvdXoW7Jr48XF4kCFyktF4DAF4xJFb_yoWDWwc_uF yjvFn7K347AFnrXr4SgFnI9rWfury2g34qy3ZYqr18C34rWr1UArWvgF1kGr1xK34UZr1k Gw48Z347AwnxujkaLaAFLSUrUUUbzb8apTn2vfkv8UJUUUU8Yxn0WfASr-VFAUDa7-sFnT 9fnUUvcSsGvfC2KfnxnUUI43ZEXa7IU59XoUUUUUU== X-Originating-IP: [120.84.141.27] X-CM-SenderInfo: 5fhh20g6yl1hqrwthudrp/1tbiKQlNBldEL+eFFAAAsh X-Rspamd-Queue-Id: 46rtJ06Cpxz4VmP X-Spamd-Bar: +++++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=none) header.from=vip.163.com; spf=pass (mx1.freebsd.org: domain of ceeso@vip.163.com designates 123.58.177.181 as permitted sender) smtp.mailfrom=ceeso@vip.163.com X-Spamd-Result: default: False [5.27 / 15.00]; HAS_REPLYTO(0.00)[s@ceesohk.com]; HAS_XOIP(0.00)[]; R_SPF_ALLOW(0.00)[+ip4:123.58.177.0/24:c]; FREEMAIL_FROM(0.00)[vip.163.com]; URI_COUNT_ODD(1.00)[7]; TO_DN_ALL(0.00)[]; DMARC_POLICY_ALLOW(0.00)[vip.163.com,none]; RCVD_NO_TLS_LAST(0.10)[]; RECEIVED_SPAMHAUS_PBL(0.00)[27.141.84.120.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.11]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[vip.163.com]; ASN(0.00)[asn:45062, ipnet:123.58.160.0/19, country:CN]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.83)[-0.833,0]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; REPLYTO_DOM_NEQ_FROM_DOM(0.00)[]; IP_SCORE_FREEMAIL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; BAD_REP_POLICIES(0.10)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_VIRUSFREE_BOTNET(2.00)[181.177.58.123.bip.virusfree.cz : 127.0.0.2]; NEURAL_SPAM_LONG(1.00)[1.000,0]; MID_CONTAINS_FROM(1.00)[]; RCVD_IN_DNSWL_NONE(0.00)[181.177.58.123.list.dnswl.org : 127.0.5.0]; SUBJECT_ENDS_QUESTION(1.00)[]; RCVD_COUNT_TWO(0.00)[2]; GREYLIST(0.00)[pass,body]; IP_SCORE(0.00)[ip: (0.09), ipnet: 123.58.160.0/19(0.19), asn: 45062(0.27), country: CN(0.02)] Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Oct 2019 20:16:34 -0000 Dear Stable,=0D=0A=0D=0AHow are you? I hope this mail finds you w= ell.=0D=0A=0D=0ACeeso Limited based in Hong Kong.Over ten years t= o produce hardware blades and accessories for customers.Use the C= raftsman Spirit do every detail well.=0D=0A=0D=0AWebsite:http://w= ww.yhenry.com/.=0D=0AWe are the direct factory with a complete se= t of equipments and machines, all of them are automatic and impor= ted from overseas, which make sure high sharpness, up to now, we = have more than 20000 types of blades, we have all kinds of standa= rd blades that is needed in market such as ZUND, ESKO,Eacom,Atom,= Aristo,Elitron,Teseo,Gerber,Axyz industrial blade http://shop.ce= esohk.com, as well as the utility knife, rotary cutter, PCB blade= ...and etc, also we can customize according to your request. And = since being in this industry for more than 10 years, we have big = advantage in price.=0D=0A=0D=0AKindly feel free to let me know if= you have any request about the blade, we shall offer you the bes= t price from direct factory.=0D=0A=0D=0ABest regards,=0D=0AShrley= Liu=0D=0A10/14/2019 04:16:00=0D=0A=20 From owner-freebsd-stable@freebsd.org Sun Oct 13 21:43:01 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3800CB7353 for ; Sun, 13 Oct 2019 21:43:01 +0000 (UTC) (envelope-from ctuffli@gmail.com) Received: from mail-ot1-x341.google.com (mail-ot1-x341.google.com [IPv6:2607:f8b0:4864:20::341]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46rwCm1JQ4z4bCZ for ; Sun, 13 Oct 2019 21:42:59 +0000 (UTC) (envelope-from ctuffli@gmail.com) Received: by mail-ot1-x341.google.com with SMTP id s22so12235636otr.6 for ; Sun, 13 Oct 2019 14:42:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=1abdtL6/NA5VGo0A135MWWXXvyXVAUyel8nDhApUQ74=; b=rkMoUueuAn6lW5JimdzXUOnkrF1wPtthEP7go1aacIQbGST/8GsXY0RNUcQLkyvNUQ yz+Ersg6M/LFr4CrFQpfuwR72A1gcyRe5Ds+8U/MWKWhIjMrGX5FQ/lJpdtEro5HdZhG brWgJHr3nw92PtBEqbkSu7U3/nXpRFKcaDOlLRy04Vi+C66+eJh+LMLhkvNw6ziEZEYq bcF3Yk3CLMfHAbkw9V8dYLLR5fwDFvw0jW9QNFBRSnR8IX0sudHv2FZS5dQyyXu+Op0+ 2lKhoNdzz+I7bR7eo7t/8fHqM8DkDq4VmfydVmYjWDl8RZpVXgIZlpGC9qMYnlIFQ4OJ yuhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=1abdtL6/NA5VGo0A135MWWXXvyXVAUyel8nDhApUQ74=; b=n5mER0lzrjt2ChD9Y21HOVgLAgPS/EU2U7fewDCQRpey4TtIlwJIM+Z0iSkeKWD2ay 7w4HeP1pEOngwQvyVvAIGt40XZMaSBGo62Wf5nvmxfatuWY08v8iqVMAENxyswvpw7Di nXccjUrays7CJUhOVsagElA2VHLoggv7W+F8Pi3Yq/0mhdp0UompmDKpT5qNRh6JAf9Z Ia2YNMMEGR7ULYIHV8fPAqPsVqL0DGUpSJEQr8HANWO81RcE+JXQvMSBeRSswEn9YPJm D+8vFfNd72P77NPVoozy7QfIWdghsCKqPz2T7SEJ99xN4ufp/gLQrWyEl2Qo1umy7xga jpvw== X-Gm-Message-State: APjAAAWdEvFPCalz86DB/pXn4ZMscnmSyF7qtIkB8ualAge5SoVJgpN+ bE1LSyeQnOH+5Y6Dk0cfMHDaMuQGire7TsCCnlv2c5vd X-Google-Smtp-Source: APXvYqxXKFhY7SjnXOy8TLJ5I5hUhfVwSrie6jYhx3VwnuCvNRcoylCz459FB9tiPAB3uJNGdz8FPpZjGK0IYSC7veo= X-Received: by 2002:a05:6830:1253:: with SMTP id s19mr22339104otp.111.1571002978297; Sun, 13 Oct 2019 14:42:58 -0700 (PDT) MIME-Version: 1.0 References: <20191013201135.GA91243@phouka1.phouka.net> In-Reply-To: <20191013201135.GA91243@phouka1.phouka.net> From: Chuck Tuffli Date: Sun, 13 Oct 2019 14:42:46 -0700 Message-ID: Subject: Re: Confused about i915kms on 12.1 To: John Kennedy Cc: freebsd-stable@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 46rwCm1JQ4z4bCZ X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=rkMoUueu; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of ctuffli@gmail.com designates 2607:f8b0:4864:20::341 as permitted sender) smtp.mailfrom=ctuffli@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[1.4.3.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; IP_SCORE(0.00)[ip: (3.12), ipnet: 2607:f8b0::/32(-2.51), asn: 15169(-2.12), country: US(-0.05)]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Oct 2019 21:43:01 -0000 Many thanks to Pete and John. See inline below On Sun, Oct 13, 2019 at 1:13 PM John Kennedy wrote: > > On Sun, Oct 13, 2019 at 12:35:41PM -0700, Chuck Tuffli wrote: > > I've been running 12.0-REL on my X1 Carbon 3rd Gen happily with xorg, > > but after installing 12.1-RC1, the laptop reboots when loading the > > i915kms driver. If I comment out the line in /etc/rc.conf: > > > > kld_list="/boot/modules/i915kms.ko" > > > > the laptop boots, albeit without graphics. The above comes from the > > 12.0R release notes, and I didn't see any updates to this in the 12.1 > > release notes. I've tried drm-kmod from both ports and packages with > > the same result. Did something change with 12.1? What should I check? > > TIA > > Not sure how your upgraded your box, but make sure that the driver is > compiled for the new kernel. KBI breakage can easily behave like that. I upgraded the box per the release announcement: freebsd-update upgrade -r 12.1-RC1 freebsd-update install with the prescribed reboots and installs. Afterwards, I did a pkg update and pkg upgrade. Note, I don't install anything from ports. > I won't be in front of a box with that setup until Monday, but from > memory, graphics/drm-kmod pulls in graphics/drm-fbsd12.0-kmod (at > least on my system with it's build-in Intel video card). > > If you do something like this, but substitute your .ko file for m4 > below, you should be able to validate the package it came from and the > kernel version it was targeted at: > > $ pkg which /usr/local/bin/*m4 > /usr/local/bin/gm4 was installed by package m4-1.4.18_1,1 > > $ pkg info m4 | grep -i version > Version : 1.4.18_1,1 > FreeBSD_version: 1201000 > > That 1201000 means that it was compiled for 12.1 in my case. $ pkg info drm-fbsd12.0-kmod-4.16.g20190814 | grep -i version Version : 4.16.g20190814 FreeBSD_version: 1200086 This version is for FreeBSD 12.0. > Have you overridden the default package location, leaving it aimed at 12.0? Nope > What do you mean by trying it from "ports"? How did you refresh that? The advice I received was 'you need to build drm-kmod from ports'. I don't typically have a ports tree, so I did: # portsnap fetch # portsnap extract # cd /usr/ports/graphics/drm-kmod # make && make install After doing the makes, my immediate thought was 'that finished too fast'. If, instead, I follow Pete's advice and run make package in /usr/ports/graphics/drm-fbsd12.0-kmod, a) make appears to do something and b) the resulting package allows my laptop to boot to a graphical desktop. And the FreeBSD_version looks correct: $ pkg info drm-fbsd12.0-kmod-4.16.g20190814 | grep -i version Version : 4.16.g20190814 FreeBSD_version: 1201000 This version is for FreeBSD 12.0. I did go back and verify that a pkg install drm-kmod appears to install the same (older) i915kms.ko as pkg install drm-fbsd12.0-kmod. Out of curiosity, is the new world order that I'll always need to compile drm-kmod from ports, or did I hit a window where packages hadn't quite caught up to ports? --chuck From owner-freebsd-stable@freebsd.org Mon Oct 14 00:07:54 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AA813BAE08 for ; Mon, 14 Oct 2019 00:07:54 +0000 (UTC) (envelope-from warlock@phouka1.phouka.net) Received: from phouka1.phouka.net (phouka1.phouka.net [107.170.196.116]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "phouka.net", Issuer "Go Daddy Secure Certificate Authority - G2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 46rzQx3DBKz3DVm for ; Mon, 14 Oct 2019 00:07:53 +0000 (UTC) (envelope-from warlock@phouka1.phouka.net) Received: from phouka1.phouka.net (localhost [127.0.0.1]) by phouka1.phouka.net (8.15.2/8.15.2) with ESMTPS id x9E06a3Y032114 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sun, 13 Oct 2019 17:06:36 -0700 (PDT) (envelope-from warlock@phouka1.phouka.net) Received: (from warlock@localhost) by phouka1.phouka.net (8.15.2/8.15.2/Submit) id x9E06aDM032112; Sun, 13 Oct 2019 17:06:36 -0700 (PDT) (envelope-from warlock) Date: Sun, 13 Oct 2019 17:06:36 -0700 From: John Kennedy To: Chuck Tuffli Cc: freebsd-stable@freebsd.org Subject: Re: Confused about i915kms on 12.1 Message-ID: <20191014000636.GB91243@phouka1.phouka.net> References: <20191013201135.GA91243@phouka1.phouka.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.12.2 (2019-09-21) X-Rspamd-Queue-Id: 46rzQx3DBKz3DVm X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of warlock@phouka1.phouka.net has no SPF policy when checking 107.170.196.116) smtp.mailfrom=warlock@phouka1.phouka.net X-Spamd-Result: default: False [2.68 / 15.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; IP_SCORE(0.36)[ipnet: 107.170.192.0/18(0.13), asn: 14061(1.71), country: US(-0.05)]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[phouka.net]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.29)[0.290,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_SPAM_LONG(0.83)[0.828,0]; R_SPF_NA(0.00)[]; FORGED_SENDER(0.30)[warlock@phouka.net,warlock@phouka1.phouka.net]; FREEMAIL_TO(0.00)[gmail.com]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:14061, ipnet:107.170.192.0/18, country:US]; FROM_NEQ_ENVFROM(0.00)[warlock@phouka.net,warlock@phouka1.phouka.net]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Oct 2019 00:07:54 -0000 On Sun, Oct 13, 2019 at 02:42:46PM -0700, Chuck Tuffli wrote: > Many thanks to Pete and John. See inline below > ... > I upgraded the box per the release announcement: > freebsd-update upgrade -r 12.1-RC1 > freebsd-update install > with the prescribed reboots and installs. Afterwards, I did a pkg > update and pkg upgrade. Note, I don't install anything from ports. The FreeBSD handbook might have the extra bits you were missing: https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/updating-upgrading-freebsdupdate.html Bonus note: the 2nd "freebsd-update install" after the reboot to delete some now-obsolete files. The announcement really doesn't talk about upgrading ports. In 23.2.3.2 ("Upgrading Packages After a Major Version Upgrade") they say to do this (among other things): pkg-static upgrade -f So pkg-static (vs pkg) in case you have shared-library problems with the dynamically-linked pkg, and then -f to "force" the upgrade. That may be the missing piece here in getting rid of your previous-kernel packages even if they might otherwise run just fine. > $ pkg info drm-fbsd12.0-kmod-4.16.g20190814 | grep -i version > Version : 4.16.g20190814 > FreeBSD_version: 1200086 > This version is for FreeBSD 12.0. Clearly the old 12.0-* kernel. > The advice I received was 'you need to build drm-kmod from ports'. I > don't typically have a ports tree, so I did: > # portsnap fetch > # portsnap extract > # cd /usr/ports/graphics/drm-kmod > # make && make install > After doing the makes, my immediate thought was 'that finished too fast'. That is apparently a meta-port that doesn't behave like you'd expect. I had to do a similar thing for myself. But I think the "-f" might have fixed it for you above, if the packages were built on the repo (below). > If, instead, I follow Pete's advice and run make package in > /usr/ports/graphics/drm-fbsd12.0-kmod, a) make appears to do something > and b) the resulting package allows my laptop to boot to a graphical > desktop. And the FreeBSD_version looks correct: > $ pkg info drm-fbsd12.0-kmod-4.16.g20190814 | grep -i version > Version : 4.16.g20190814 > FreeBSD_version: 1201000 > This version is for FreeBSD 12.0. > > I did go back and verify that a pkg install drm-kmod appears to > install the same (older) i915kms.ko as pkg install drm-fbsd12.0-kmod. > Out of curiosity, is the new world order that I'll always need to > compile drm-kmod from ports, or did I hit a window where packages > hadn't quite caught up to ports? I don't know. Finding out which package your .ko file came from would tell us a bit there. I've just got in the habit of build all my packages myself because I can get into the bleeding-edge area and run into problems like that. I suspect that the official answer would be "hell no" and this is the type of experience that the FreeBSD folks like to see for the purposes off adding things to UPDATING and such. But I know that it is going to take a while for their poudraire to grid through all the ports In any case, I'm glad we got you up and running. From owner-freebsd-stable@freebsd.org Mon Oct 14 02:35:21 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7C351BD4CB for ; Mon, 14 Oct 2019 02:35:21 +0000 (UTC) (envelope-from ctuffli@gmail.com) Received: from mail-oi1-x22f.google.com (mail-oi1-x22f.google.com [IPv6:2607:f8b0:4864:20::22f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46s2j45mXGz3L9x for ; Mon, 14 Oct 2019 02:35:20 +0000 (UTC) (envelope-from ctuffli@gmail.com) Received: by mail-oi1-x22f.google.com with SMTP id x3so12556268oig.2 for ; Sun, 13 Oct 2019 19:35:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=sHGcEHTOjrMX93IgXwxng3tBcvwFswEsLZewDm2wtFw=; b=S2t06F9RnH1E8SKPbE57ny43GoYDaGVX6VNvZ6sOpWgLTngN/d4W2NKqMsVT8H03zY a2Hvv3XbKjL59Zz0+X/bf7R9kkNG2p55ttrWXHTI2zowFwdxchvPhrm2iC56WtHbPHkB zWo45tWNVrPGXznQdBaDlf3++aRdbkXgK6YdTMfaX5lV41kwDYV0Qb9d/X/L4DzxziGv zAGgnWVNIEhvkB8CLL0nZgCiOo7mYp/0AO+Fse/VHXIiEteBMbQOuG4tnswBXNptzUwm QheTcuAl8kbPu5p11POGAO9djtbZS3ur8o9jZM0ug/F4MJnGtcUzhIqDtxLH6bIVGn/b b3wA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=sHGcEHTOjrMX93IgXwxng3tBcvwFswEsLZewDm2wtFw=; b=PptFAaACQNIy7h+cFd14IfvhS6ZKTzf95Xh9nKBLmwf2gViqzDFjdDaULf/eJb6C0r wKDgXzz2KqPe5bjhXUkEu1rFKe5+VoNqwhqt67C4mLV3raMJqMnkeyHkbz3aWmL9F7dH HiezxN/G28E4RvKlmNomKLtQ0/cwoo1m3I1BhdWODWMslt4HCh2sCOG7eV9pzhBpto/j 1EamTED/hctZHK3/82Od2bXjH/iz9WtBMPPaxCpMqPlCh8YBy3t7Ma03T+Cwicerog87 rVRshP5gL1LptGKFmDTVGqeRf7jfV9+FMDccmy+ptXmvNNKJPRLd0T+6G/0wZ6lc/5Jp 30TQ== X-Gm-Message-State: APjAAAW6uZ18bBlOkhPFHvF+zEQ9ALLIj/ecVcNWN6wFTO0IYGX3vOFp 9RyfTtAbkSatJhydJ1WDqYjS9C5Q0LiMs/ZuLbUA8g== X-Google-Smtp-Source: APXvYqw1S9V7kpdKRA/q60O2JLKgLyajtR9BRroVBc9b7fdllgpJsgAhnMjxZ8gRZ2qmdC4Cjg61WrlKA/acyd9wMmA= X-Received: by 2002:a05:6808:9ad:: with SMTP id e13mr21336506oig.90.1571020518653; Sun, 13 Oct 2019 19:35:18 -0700 (PDT) MIME-Version: 1.0 References: <20191013201135.GA91243@phouka1.phouka.net> <20191014000636.GB91243@phouka1.phouka.net> In-Reply-To: <20191014000636.GB91243@phouka1.phouka.net> From: Chuck Tuffli Date: Sun, 13 Oct 2019 19:35:07 -0700 Message-ID: Subject: Re: Confused about i915kms on 12.1 To: John Kennedy Cc: freebsd-stable@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 46s2j45mXGz3L9x X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=S2t06F9R; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of ctuffli@gmail.com designates 2607:f8b0:4864:20::22f as permitted sender) smtp.mailfrom=ctuffli@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[f.2.2.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; IP_SCORE(0.00)[ip: (-7.71), ipnet: 2607:f8b0::/32(-2.51), asn: 15169(-2.12), country: US(-0.05)]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Oct 2019 02:35:21 -0000 On Sun, Oct 13, 2019 at 5:08 PM John Kennedy wrote: ... > In 23.2.3.2 ("Upgrading Packages After a Major Version Upgrade") they say > to do this (among other things): > > pkg-static upgrade -f FWIW, I went back and did this but still seem to get the older kmod > In any case, I'm glad we got you up and running. Again, many thanks for your help. --chuck From owner-freebsd-stable@freebsd.org Thu Oct 17 05:37:52 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EB93816EBC8 for ; Thu, 17 Oct 2019 05:37:52 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.116.210]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46tycH28ltz3Kdn for ; Thu, 17 Oct 2019 05:37:50 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=cs.huji.ac.il; s=57791128; h=To:Date:Message-Id:Subject:Mime-Version:Content-Transfer-Encoding:Content-Type:From; bh=k2Nar7u542oGwUbXBWMSTy44Trr60Am43h4MzV5dRnQ=; b=efXZkvhaO0Ge3G0txF1Y+A1eH2IM5bLDmBv+uqRXHLtiBMZ6nrkyBZVkHttaZ5UeipS5hkAzzhPwtL/5A56vFL0DkH5tMaKjx78+LPCmReJhznh6QT4JsXyKb8q6PgEpcjinyTdBl0BF2ZI7peJgi36feeVRuDm62VL2Is2r8fRFS56rushAypDEMi0mECqUgN3N5Kq4DDfvu4Rn0T3Fe9TYSorC8ZW5b9LB7AOeip0ap7jQTGyzCcIREY7OfXEsq4kQu6dXtoitDJiRoIrnBi2hhSZclDXRc+MaT1Zf/alyjNClVkVo6eTG8aWW+E4gotE7CqUKAYjoVzRo1xMXoQ==; Received: from bach.cs.huji.ac.il ([132.65.80.20]) by kabab.cs.huji.ac.il with esmtp id 1iKyTq-000K3D-Mb for freebsd-stable@freebsd.org; Thu, 17 Oct 2019 08:37:46 +0300 From: Daniel Braniss Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Subject: 12.1 weirdness Message-Id: <2AE097DF-E7CA-4A02-B57E-2A490DBC40EC@cs.huji.ac.il> Date: Thu, 17 Oct 2019 08:37:46 +0300 To: FreeBSD-STABLE Mailing List X-Mailer: Apple Mail (2.3445.9.1) X-Rspamd-Queue-Id: 46tycH28ltz3Kdn X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=cs.huji.ac.il header.s=57791128 header.b=efXZkvha; dmarc=none; spf=none (mx1.freebsd.org: domain of danny@cs.huji.ac.il has no SPF policy when checking 132.65.116.210) smtp.mailfrom=danny@cs.huji.ac.il X-Spamd-Result: default: False [-2.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.97)[-0.970,0]; R_DKIM_ALLOW(-0.20)[cs.huji.ac.il:s=57791128]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[huji.ac.il]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; RCPT_COUNT_ONE(0.00)[1]; IP_SCORE(-0.23)[ipnet: 132.64.0.0/13(-0.68), asn: 378(-0.54), country: IL(0.06)]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[cs.huji.ac.il:+]; RCVD_IN_DNSWL_NONE(0.00)[210.116.65.132.list.dnswl.org : 127.0.10.0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:378, ipnet:132.64.0.0/13, country:IL]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Oct 2019 05:37:53 -0000 hi, just trying out 12,1 on a DELL PowerEdge R710, and I see: ... Oct 16 22:52:12 store-08 kernel: bce3: bce_pulse(): Warning: bootcode = thinks driver is absent! (bc_state =3D 0x0023600E) Oct 16 22:52:13 store-08 kernel: Limiting icmp unreach response from = 3244 to 200 packets/sec Oct 16 22:52:13 store-08 kernel: bce3: bce_pulse(): Bootcode found the = driver pulse! (bc_state =3D 0x0003610E) Oct 17 06:04:29 store-08 kernel: Limiting icmp unreach response from 564 = to 200 packets/sec there is nothing connected to bce3, only bce0,=20 rev is r353486, and it=E2=80=99s diskless. BTW: is there a way of knowing which port is being reported as = unreachable?=20 any ideas? thanks, danny= From owner-freebsd-stable@freebsd.org Thu Oct 17 09:45:46 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4448914C475 for ; Thu, 17 Oct 2019 09:45:46 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 46v46K0s1Pz41sP for ; Thu, 17 Oct 2019 09:45:44 +0000 (UTC) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: freebsd-stable@freebsd.org Received: from [10.58.0.4] (188-123-32-240.rdtc.ru [188.123.32.240] (may be forged)) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id x9H9jFAV085829 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Thu, 17 Oct 2019 09:45:16 GMT (envelope-from eugen@grosbein.net) Subject: Re: 12.1 weirdness To: Daniel Braniss , FreeBSD-STABLE Mailing List References: <2AE097DF-E7CA-4A02-B57E-2A490DBC40EC@cs.huji.ac.il> From: Eugene Grosbein Message-ID: <60ee7253-d49b-19d1-6193-ab9f7ac22a8c@grosbein.net> Date: Thu, 17 Oct 2019 16:45:14 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <2AE097DF-E7CA-4A02-B57E-2A490DBC40EC@cs.huji.ac.il> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=0.1 required=5.0 tests=ALL_TRUSTED,BAYES_00, DATE_IN_FUTURE_96_Q,LOCAL_FROM autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 0.8 DATE_IN_FUTURE_96_Q Date: is 4 days to 4 months after Received: * date * 2.6 LOCAL_FROM From my domains X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 46v46K0s1Pz41sP X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=permerror (mx1.freebsd.org: domain of eugen@grosbein.net uses mechanism not recognized by this client) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-3.69 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; R_SPF_PERMFAIL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; IP_SCORE(-1.59)[ip: (-3.99), ipnet: 2a01:4f8::/29(-2.11), asn: 24940(-1.81), country: DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Oct 2019 09:45:46 -0000 17.10.2019 12:37, Daniel Braniss wrote: > just trying out 12,1 on a DELL PowerEdge R710, and I see: > ... > Oct 16 22:52:12 store-08 kernel: bce3: bce_pulse(): Warning: bootcode thinks driver is absent! (bc_state = 0x0023600E) > Oct 16 22:52:13 store-08 kernel: Limiting icmp unreach response from 3244 to 200 packets/sec > Oct 16 22:52:13 store-08 kernel: bce3: bce_pulse(): Bootcode found the driver pulse! (bc_state = 0x0003610E) > Oct 17 06:04:29 store-08 kernel: Limiting icmp unreach response from 564 to 200 packets/sec > > there is nothing connected to bce3, only bce0, > rev is r353486, and it’s diskless. Does it experience some disruption? If not, you may just ignore those. > BTW: is there a way of knowing which port is being reported as unreachable? So many packets should be easily seen with net/trafshow or just tcpdump. From owner-freebsd-stable@freebsd.org Wed Oct 16 20:06:28 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A4C3D14EB22; Wed, 16 Oct 2019 20:06:28 +0000 (UTC) (envelope-from lwhsu@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [96.47.72.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46tjx03tLjz4Cg3; Wed, 16 Oct 2019 20:06:28 +0000 (UTC) (envelope-from lwhsu@freebsd.org) Received: by freefall.freebsd.org (Postfix, from userid 1129) id 64BD1DF4E; Wed, 16 Oct 2019 20:06:28 +0000 (UTC) Date: Wed, 16 Oct 2019 20:06:28 +0000 From: Li-Wen Hsu To: freebsd-testing@freebsd.org Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org Subject: FreeBSD CI Weekly Report 2019-10-13 Message-ID: <20191016200628.GA5475@freefall.freebsd.org> Reply-To: freebsd-testing@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.11.4 (2019-03-13) X-Mailman-Approved-At: Thu, 17 Oct 2019 16:17:50 +0000 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Oct 2019 20:06:28 -0000 (Please send the followup to freebsd-testing@ and note Reply-To is set.) FreeBSD CI Weekly Report 2019-10-13 =================================== Here is a summary of the FreeBSD Continuous Integration results for the period from 2019-10-07 to 2019-10-13. During this period, we have: * 2217 builds (91.1% (-8.3) passed, 8.9% (+8.3) failed) of buildworld and buildkernel (GENERIC and LINT) were executed on aarch64, amd64, armv6, armv7, i386, mips, mips64, powerpc, powerpc64, powerpcspe, riscv64, sparc64 architectures for head, stable/12, stable/11 branches. * 283 test runs (81.3% (-0.4) passed, 8.5% (-9.8) unstable, 10.2% (+10.2) exception) were executed on amd64, i386, riscv64 architectures for head, stable/12, stable/11 branches. * 30 doc builds (100% passed) Test case status (on 2019-10-13 23:59): | Branch/Architecture | Total | Pass | Fail | Skipped | | ------------------- | --------- | --------- | ------ | -------- | | head/amd64 | 7590 (+1) | 7527 (+1) | 0 (0) | 63 (0) | | head/i386 | 7588 (+1) | 7516 (+1) | 0 (0) | 72 (0) | | 12-STABLE/amd64 | 7482 (0) | 7438 (-3) | 0 (0) | 44 (+3) | | 12-STABLE/i386 | 7480 (0) | 7432 (0) | 0 (0) | 48 (0) | | 11-STABLE/amd64 | 6849 (0) | 6808 (0) | 0 (0) | 41 (0) | | 11-STABLE/i386 | 6847 (0) | 6770 (0) | 34 (0) | 43 (0) | (The statistics from experimental jobs are omitted) If any of the issues found by CI are in your area of interest or expertise please investigate the PRs listed below. The latest web version of this report is available at https://hackmd.io/@FreeBSD-CI/report-20191013 and archive is available at https://hackmd.io/@FreeBSD-CI/, any help is welcome. ## News * [FCP 20190401-ci_policy: CI policy](https://github.com/freebsd/fcp/blob/master/fcp-20190401-ci_policy.md) is in "feedback" state, please check and provide comments on -fcp@ and -hackers@ mailing lists. * A new wiki page started at https://wiki.freebsd.org/Jenkins/Debug describes how to reproduce and debug the failing cases. It is welcomed to add more contents. ## Failing Tests * https://ci.freebsd.org/job/FreeBSD-stable-11-i386-test/ * local.kyua.* (31 cases) * local.lutok.* (3 cases) ## Failing and Flaky Tests (from experimental jobs) * https://ci.freebsd.org/job/FreeBSD-head-amd64-dtrace_test/ * cddl.usr.sbin.dtrace.common.misc.t_dtrace_contrib.tst_dynopt_d * https://bugs.freebsd.org/237641 * https://ci.freebsd.org/job/FreeBSD-head-amd64-test_zfs/ * There are ~18 failing and ~97 skipped cases, including flakey ones, see https://ci.freebsd.org/job/FreeBSD-head-amd64-test_zfs/lastCompletedBuild/testReport/ for more details ## Disabled Tests * sys.fs.tmpfs.mount_test.large https://bugs.freebsd.org/212862 * sys.fs.tmpfs.link_test.kqueue https://bugs.freebsd.org/213662 * sys.kqueue.libkqueue.kqueue_test.main https://bugs.freebsd.org/233586 * sys.kern.ptrace_test.ptrace__PT_KILL_competing_stop https://bugs.freebsd.org/220841 * lib.libc.regex.exhaust_test.regcomp_too_big (i386 only) https://bugs.freebsd.org/237450 * sys.netinet.socket_afinet.socket_afinet_bind_zero (new) https://bugs.freebsd.org/238781 * sys.netpfil.pf.names.names * sys.netpfil.pf.synproxy.synproxy https://bugs.freebsd.org/238870 * sys.kern.ptrace_test.ptrace__follow_fork_child_detached_unrelated_debugger https://bugs.freebsd.org/239292 * sys.netpfil.pf.forward.v4 (i386 only) * sys.netpfil.pf.forward.v6 (i386 only) * sys.netpfil.pf.set_tos.v4 (i386 only) https://bugs.freebsd.org/239380 * sys.kern.ptrace_test.ptrace__follow_fork_both_attached_unrelated_debugger https://bugs.freebsd.org/239397 * sys.kern.ptrace_test.ptrace__parent_sees_exit_after_child_debugger https://bugs.freebsd.org/239399 * sys.kern.ptrace_test.ptrace__follow_fork_parent_detached_unrelated_debugger https://bugs.freebsd.org/239425 * lib.libc.gen.getmntinfo_test.getmntinfo_test https://bugs.freebsd.org/240049 * sys.sys.qmath_test.qdivq_s64q https://bugs.freebsd.org/240219 * sys.kern.ptrace_test.ptrace__getppid https://bugs.freebsd.org/240510 * lib.libc.sys.stat_test.stat_socket https://bugs.freebsd.org/240621 * sys.netpfil.common.tos.pf_tos (i386 only) https://bugs.freebsd.org/240086 * lib.libarchive.functional_test.test_write_filter_zstd https://bugs.freebsd.org/240683 ## Issues ### Cause build fails * https://bugs.freebsd.org/233735 Possible build race: genoffset.o /usr/src/sys/sys/types.h: error: machine/endian.h: No such file or directory * https://bugs.freebsd.org/233769 Possible build race: ld: error: unable to find library -lgcc_s ### Cause kernel panics * https://bugs.freebsd.org/238870 sys.netpfil.pf.names.names and sys.netpfil.pf.synproxy.synproxy cause panic Patch exists: * https://reviews.freebsd.org/D20868 * https://reviews.freebsd.org/D20869 ### Open * https://bugs.freebsd.org/237403 Tests in sys/opencrypto should be converted to Python3 * https://bugs.freebsd.org/237641 Flakey test case: common.misc.t_dtrace_contrib.tst_dynopt_d * https://bugs.freebsd.org/237656 "Freed UMA keg (rtentry) was not empty (18 items). Lost 1 pages of memory." seen when running sys/netipsec tests * https://bugs.freebsd.org/238781 sys.netinet.socket_afinet.socket_afinet_bind_zero does not work when mac_portacl(4) loaded * https://bugs.freebsd.org/239292 Flakey test case: sys.kern.ptrace_test.ptrace__follow_fork_child_detached_unrelated_debugger * https://bugs.freebsd.org/239380 sys.netpfil.pf.forward.{v4,v6} and sys.netpfil.pf.set_tos.v4 fail on i386 * https://bugs.freebsd.org/239397 Flakey test case: sys.kern.ptrace_test.ptrace__follow_fork_both_attached_unrelated_debugger * https://bugs.freebsd.org/239399 Flakey test case: sys.kern.ptrace_test.ptrace__parent_sees_exit_after_child_debugger * https://bugs.freebsd.org/239425 Flakey test case: sys.kern.ptrace_test.ptrace__follow_fork_parent_detached_unrelated_debugger * https://bugs.freebsd.org/240085 Failing test: sys.netpfil.common.forward.pf_v4 on i386 * https://bugs.freebsd.org/240086 Failing test: sys.netpfil.common.tos.pf_tos on i386 ### Others * [Tickets related to testing@](https://preview.tinyurl.com/y9maauwg) From owner-freebsd-stable@freebsd.org Fri Oct 18 14:12:39 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B2EBF151F3B for ; Fri, 18 Oct 2019 14:12:39 +0000 (UTC) (envelope-from mike@sentex.net) Received: from pyroxene2a.sentex.ca (unknown [IPv6:2607:f3e0:0:3::19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "pyroxene2.sentex.ca", Issuer "pyroxene2.sentex.ca" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 46vnzp3P68z4chp for ; Fri, 18 Oct 2019 14:12:38 +0000 (UTC) (envelope-from mike@sentex.net) Received: from [192.168.43.29] ([192.168.43.29]) by pyroxene2a.sentex.ca (8.15.2/8.15.2) with ESMTPS id x9IECbvZ043096 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Fri, 18 Oct 2019 10:12:37 -0400 (EDT) (envelope-from mike@sentex.net) To: freebsd-stable@freebsd.org From: mike tancsa Subject: SSH error messages (bug id=234793) ) RELENG_12 Openpgp: preference=signencrypt Autocrypt: addr=mike@sentex.net; keydata= mQENBFywzOMBCACoNFpwi5MeyEREiCeHtbm6pZJI/HnO+wXdCAWtZkS49weOoVyUj5BEXRZP xflV2ib2hflX4nXqhenaNiia4iaZ9ft3I1ebd7GEbGnsWCvAnob5MvDZyStDAuRxPJK1ya/s +6rOvr+eQiXYNVvfBhrCfrtR/esSkitBGxhUkBjOti8QwzD71JVF5YaOjBAs7jZUKyLGj0kW yDg4jUndudWU7G2yc9GwpHJ9aRSUN8e/mWdIogK0v+QBHfv/dsI6zVB7YuxCC9Fx8WPwfhDH VZC4kdYCQWKXrm7yb4TiVdBh5kgvlO9q3js1yYdfR1x8mjK2bH2RSv4bV3zkNmsDCIxjABEB AAG0HW1pa2UgdGFuY3NhIDxtaWtlQHNlbnRleC5uZXQ+iQFUBBMBCAA+FiEEmuvCXT0aY6hs 4SbWeVOEFl5WrMgFAlywzOYCGwMFCQHhM4AFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQ eVOEFl5WrMhnPAf7Bf+ola0V9t4i8rwCMGvzkssGaxY/5zNSZO9BgSgfN0WzgmBEOy/3R4km Yn5KH94NltJYAAE5hqkFmAwK6psOqAR9cxHrRfU+gV2KO8pCDc6K/htkQcd/mclJYpCHp6Eq EVJOiAxcNaYuHZkeMdXDuvvI5Rk82VHk84BGgxIqIrhLlkguoPbXOOa+8c/Mpb1sRAGZEOuX EzKNC49+GS9gKW6ISbanyPsGEcFyP7GKMzcHBPf3cPrewZQZ6gBoNscasL6IJeAQDqzQAxbU GjO0qBSMRgnLXK7+DJlxrYdHGXqNbV6AYsmHJ6c2WWWiuRviFBqXinlgJ2FnYebZPAfWibkB DQRcsMzkAQgA1Dpo/xWS66MaOJLwA28sKNMwkEk1Yjs+okOXDOu1F+0qvgE8sVmrOOPvvWr4 axtKRSG1t2QUiZ/ZkW/x/+t0nrM39EANV1VncuQZ1ceIiwTJFqGZQ8kb0+BNkwuNVFHRgXm1 qzAJweEtRdsCMohB+H7BL5LGCVG5JaU0lqFU9pFP40HxEbyzxjsZgSE8LwkI6wcu0BLv6K6c Lm0EiHPOl5G8kgRi38PS7/6s3R8QDsEtbGsYy6O82k3zSLIjuDBwA9GRaeigGppTxzAHVjf5 o9KKu4O7gC2KKVHPegbXS+GK7DU0fjzX57H5bZ6komE5eY4p3oWT/CwVPSGfPs8jOwARAQAB iQE8BBgBCAAmFiEEmuvCXT0aY6hs4SbWeVOEFl5WrMgFAlywzOQCGwwFCQHhM4AACgkQeVOE Fl5WrMhmjQf/dBCjAVn1J0GzSsHiLvSAQz1cchbdy8LD0Tnpzjgp5KLU7sNojbI8vqt4yKAi cayI88j8+xxNXPMWM4pHELuUuVHS5XTpHa/wwulUtI5w/zyKlUDsIvqTPZLUEwH7DfNBueVM WyNaIjV2kxSmM8rNMC+RkgyfbjGLCkmWsMRVuLIUYpl5D9WHmenUbiErlKU2KvEEXEg/aLKq 3m/AdM9RAYsP9O4l+sAZEfyYoNJzDhTZMzn/9Q0uFPLK9smDQh4WBTFaApveVJPHRKmHPoNF Xxj+yScYdQ4SKH34WnhNSELvnZQ3ulH5tpASmm0w+GxfZqSc8+QCwoKtBRDUxoE56A== Message-ID: <100597e5-4491-f455-d247-59f5374ea6a4@sentex.net> Date: Fri, 18 Oct 2019 10:12:37 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.9.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 46vnzp3P68z4chp X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of mike@sentex.net designates 2607:f3e0:0:3::19 as permitted sender) smtp.mailfrom=mike@sentex.net X-Spamd-Result: default: False [0.78 / 15.00]; ARC_NA(0.00)[]; RDNS_NONE(1.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f3e0::/32]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; HFILTER_HELO_IP_A(1.00)[pyroxene2a.sentex.ca]; DMARC_NA(0.00)[sentex.net]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; IP_SCORE(-1.72)[ipnet: 2607:f3e0::/32(-4.94), asn: 11647(-3.56), country: CA(-0.09)]; HFILTER_HELO_NORES_A_OR_MX(0.30)[pyroxene2a.sentex.ca]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11647, ipnet:2607:f3e0::/32, country:CA]; HFILTER_HOSTNAME_UNKNOWN(2.50)[]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Oct 2019 14:12:39 -0000 Does anyone know what the cause is of this fail message ? (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=234793) its triggered by a normal ssh key'd login, but sshd is running with VERBOSE logging.   sshd[63290]: Failed unknown for testuser1 from 192.168.xx.yyy port 60643 ssh2 ? The user is able to login no problem, but the error message is bubbling up in our HIDS. We had to white list it, but it would be useful to understand exactly why and what is failing.     ---Mike From owner-freebsd-stable@freebsd.org Fri Oct 18 14:28:24 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 08AEF15256A for ; Fri, 18 Oct 2019 14:28:24 +0000 (UTC) (envelope-from matt.garber@gmail.com) Received: from mail-qt1-x82e.google.com (mail-qt1-x82e.google.com [IPv6:2607:f8b0:4864:20::82e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46vpKz0KT9z4dPF for ; Fri, 18 Oct 2019 14:28:22 +0000 (UTC) (envelope-from matt.garber@gmail.com) Received: by mail-qt1-x82e.google.com with SMTP id 3so9391578qta.1 for ; Fri, 18 Oct 2019 07:28:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=JYkQFSAocpGfc2vR7nvmTlD9Na9k/5BRNjbjlHdwdw4=; b=cYeEeGSgo/uNDLxUeMnXyks1Wog737FOKrU9o61ubEdFcODX5nxbqalqGek+iHWtRI ngF65SWWmwYpoSqskRtoZ1s9UVrRMnN+QJcpmsDKJ2YG+7vGFfmOdZjJ0Z8Di2IM/UQe spT7Lr3bSH4+9i0R3ATwvSNRQP7afRSHPHmP84Q4KpWwnGnPVKVqsi70Zd1OYhD0LENA K/NhCefcLkoyckVzlkWiHlOuvDsaCdW6CW1pCh2GGgWuh0G4MMhNZ3AP8XfXpJJStMRd UgtSk+EGzBWGJR2Fk++FjlaiteG3BC91g5YDEM0lnMsASvnDIGT3rWATRNYtElX56SNN NLEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=JYkQFSAocpGfc2vR7nvmTlD9Na9k/5BRNjbjlHdwdw4=; b=HhqBTBdScZBsR5OdMEsgOISWny5EvRccwKiPKK0NrXVngqTpnXsLAiSlx5KHs8ARdP V9JxVUS+/MtWYx/TTSu2ZsczSiui0xmPAIqdKwp0681rcaqmYkFkram37i09lRtp0s77 G2FIkhBNkzoksJngOHhSAcQ/D8ORD9x9DXJOcPsO+/ugpoNtSAaWdJCAmfq1CTx7We9t zUcEn+GqgbNxmtHail78ooWuxAwcFVpgTV3z0k8/nTeaCOs4625HLhmgb9/a7VqoZLM4 NzmGSWjd5GTlIiIPApM24BmvwtIkWRcXksjoCA2t5f61qptjaZj9u/9Z5u6HZ7r9tPQp tOnw== X-Gm-Message-State: APjAAAVbOJdNZRZpqEeGMbup4ii3HnugeOL7rXaUXB9z26yLzsS3IgLa jgo1VL9F0BKN2U/pHQdfwco= X-Google-Smtp-Source: APXvYqydQCyRprw/XcHK3SyVDCZs9QZ78v6GBFnwHcDgwXvKFfs5f28je6z2QXOZj39wiTPiBXtPmg== X-Received: by 2002:ac8:1c5a:: with SMTP id j26mr6647289qtk.351.1571408901953; Fri, 18 Oct 2019 07:28:21 -0700 (PDT) Received: from [10.100.20.3] ([68.183.62.201]) by smtp.gmail.com with ESMTPSA id v139sm2979930qkb.53.2019.10.18.07.28.20 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 18 Oct 2019 07:28:20 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: SSH error messages (bug id=234793) ) RELENG_12 From: Matt Garber In-Reply-To: <100597e5-4491-f455-d247-59f5374ea6a4@sentex.net> Date: Fri, 18 Oct 2019 10:28:19 -0400 Cc: freebsd-stable@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <246561E5-9E57-4CC2-B94C-4CE8C553D972@gmail.com> References: <100597e5-4491-f455-d247-59f5374ea6a4@sentex.net> To: mike tancsa X-Mailer: Apple Mail (2.3445.104.11) X-Rspamd-Queue-Id: 46vpKz0KT9z4dPF X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=cYeEeGSg; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of mattgarber@gmail.com designates 2607:f8b0:4864:20::82e as permitted sender) smtp.mailfrom=mattgarber@gmail.com X-Spamd-Result: default: False [-2.50 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MV_CASE(0.50)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(0.00)[ip: (-9.30), ipnet: 2607:f8b0::/32(-2.46), asn: 15169(-2.09), country: US(-0.05)]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[e.2.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Oct 2019 14:28:24 -0000 > Does anyone know what the cause is of this fail message ? >=20 > (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D234793) >=20 > its triggered by a normal ssh key'd login, but sshd is running with > VERBOSE logging.=20 >=20 > sshd[63290]: Failed unknown for testuser1 from 192.168.xx.yyy port > 60643 ssh2 ? >=20 > The user is able to login no problem, but the error message is = bubbling > up in our HIDS. We had to white list it, but it would be useful to > understand exactly why and what is failing. >=20 > =E2=80=94Mike It=E2=80=99s one of the other SSH authentication types (e.g., GSSAPI, = password, etc.) which is in the processing order before public key. = I=E2=80=99m assuming you=E2=80=99re seeing that =E2=80=98failure=E2=80=99 = immediately before your successful key authentication in auth.log; I = actually had to switch back to INFO for logging because that = =E2=80=98failure=E2=80=99 trips up sshguard which kicks in and blocks = the IP despite the public key auth succeeding right after whichever = other auth type is tried and fails. (Unfortunately, I wasn=E2=80=99t able to determine which specific other = authentication type was being tried first, since moving logging back to = INFO resolved my immediate issue of getting blocked by sshguard before = successfully processing my key.) Thanks, -- Matt Garber From owner-freebsd-stable@freebsd.org Fri Oct 18 14:36:13 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A209C15289D for ; Fri, 18 Oct 2019 14:36:13 +0000 (UTC) (envelope-from matt.garber@gmail.com) Received: from mail-qk1-x72a.google.com (mail-qk1-x72a.google.com [IPv6:2607:f8b0:4864:20::72a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46vpW02x04z4dph for ; Fri, 18 Oct 2019 14:36:11 +0000 (UTC) (envelope-from matt.garber@gmail.com) Received: by mail-qk1-x72a.google.com with SMTP id p4so5490001qkf.5 for ; Fri, 18 Oct 2019 07:36:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=SnLYACLnxzjbP/0TUvih/9JhSOdrU9nCYCpsE2hpihE=; b=Md1eMdGIcVwahTwFgiP7YZqNsj351jntGj+m94ISOpix44BVDwpHYR+iGEiYi3PxyU ejl2L/mS1F7S5og7hFj4ZJxIHFhPOk83YztfT+DmoE5jbJP9v+821dgJzjA6wV3DUDYY NZvIEwbUZiJUgBZTSvxICpi/Hv73RWK8QGXL9iK1z8hOFNrhk2KDVz0hgvMh0vaywVPT CaJOG1rjUZdk6MfSp8mPmmjUi0982TkvDhO5g5STYI3yEdd54l5sUH9NfND6m81AdjXo ZonfsWMI80gBybL3uGv2dRash8VdBJ6+yX4k3irP2zi07lmEDuOp4y/TyAeBgh0TXwRH 1jkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=SnLYACLnxzjbP/0TUvih/9JhSOdrU9nCYCpsE2hpihE=; b=FUNzj3fM+cYpnlEGq1xdRgWeL+J3J2614K1SL8+XyZVcaGL5WtrdC77k8qQJwwnfLt qkpmxFQa/YWnTpjp0k8WtUar5gZgz3rd9EgmW2cUNloCyU/fY2sUV6+rd6ZHja/p1itL fRa83THubcRVUROrWyVwKBNiRq4LCy6Y4NqqCWRXsaFZCXzBFcK5v82zqTwNIZhXSlsY asjlmFivvbnO1K2NQ/3zIslfpogtkK40Dm//cmd72PAPckbglQ7tTy9OIPv4CoQmvxEm t2lOYAl11xY5p7o6cB6rlv7DCPbo1ztF4no77Nu6EDfoPyDM1jqnh091tt2P0zA1lBFw r0lQ== X-Gm-Message-State: APjAAAV3o/5a8dFYkOH0eczmrJ8JvXUMHc+48j4F90Ly4LCUEKlte+/b GfODjduy/6BwOU7ljSIO9RM= X-Google-Smtp-Source: APXvYqzFqu8Wu87A0PRK8UTrIeh3Z0whxD/1QQ6cxHB70sznrUM9cvaSBsOnVkatIMiGDIabeLo26A== X-Received: by 2002:a37:8707:: with SMTP id j7mr8967329qkd.399.1571409370887; Fri, 18 Oct 2019 07:36:10 -0700 (PDT) Received: from [10.100.20.3] ([68.183.62.201]) by smtp.gmail.com with ESMTPSA id i185sm3124000qke.83.2019.10.18.07.36.10 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 18 Oct 2019 07:36:10 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: SSH error messages (bug id=234793) ) RELENG_12 From: Matt Garber In-Reply-To: <246561E5-9E57-4CC2-B94C-4CE8C553D972@gmail.com> Date: Fri, 18 Oct 2019 10:36:09 -0400 Cc: freebsd-stable@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <100597e5-4491-f455-d247-59f5374ea6a4@sentex.net> <246561E5-9E57-4CC2-B94C-4CE8C553D972@gmail.com> To: mike tancsa X-Mailer: Apple Mail (2.3445.104.11) X-Rspamd-Queue-Id: 46vpW02x04z4dph X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=Md1eMdGI; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of mattgarber@gmail.com designates 2607:f8b0:4864:20::72a as permitted sender) smtp.mailfrom=mattgarber@gmail.com X-Spamd-Result: default: False [-2.50 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MV_CASE(0.50)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(0.00)[ip: (-9.36), ipnet: 2607:f8b0::/32(-2.46), asn: 15169(-2.09), country: US(-0.05)]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[a.2.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Oct 2019 14:36:13 -0000 >=20 >> Does anyone know what the cause is of this fail message ? >>=20 >> (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D234793) >>=20 >> its triggered by a normal ssh key'd login, but sshd is running with >> VERBOSE logging.=20 >>=20 >> sshd[63290]: Failed unknown for testuser1 from 192.168.xx.yyy port >> 60643 ssh2 ? >>=20 >> The user is able to login no problem, but the error message is = bubbling >> up in our HIDS. We had to white list it, but it would be useful to >> understand exactly why and what is failing. >>=20 >> =E2=80=94Mike >=20 > It=E2=80=99s one of the other SSH authentication types (e.g., GSSAPI, = password, etc.) which is in the processing order before public key. = I=E2=80=99m assuming you=E2=80=99re seeing that =E2=80=98failure=E2=80=99 = immediately before your successful key authentication in auth.log; I = actually had to switch back to INFO for logging because that = =E2=80=98failure=E2=80=99 trips up sshguard which kicks in and blocks = the IP despite the public key auth succeeding right after whichever = other auth type is tried and fails. >=20 > (Unfortunately, I wasn=E2=80=99t able to determine which specific = other authentication type was being tried first, since moving logging = back to INFO resolved my immediate issue of getting blocked by sshguard = before successfully processing my key.) I=E2=80=99d also like to point out that whatever authentication method = is now being tried first was a change from 11.3-RELEASE, as I didn=E2=80=99= t encounter that ordering issue in my VERBOSE logs triggering sshguard = until after upgrading to 12.0-RELEASE. I always have password auth = disabled (only use public keys), but also tried explicit disable = statements for GSSAPI and the several other auth types I could think of, = but unfortunately wasn=E2=80=99t able to determine which auth type that = log line corresponded to. It could also be an auth type that was = previously used, but sshd in 12.0-RELEASE re-ordered the processing = sequence to try it before public keys. Thanks, -- Matt Garber From owner-freebsd-stable@freebsd.org Fri Oct 18 14:44:42 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 18473152B80 for ; Fri, 18 Oct 2019 14:44:42 +0000 (UTC) (envelope-from mike@sentex.net) Received: from pyroxene2a.sentex.ca (unknown [IPv6:2607:f3e0:0:3::19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "pyroxene2.sentex.ca", Issuer "pyroxene2.sentex.ca" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 46vphn18C0z4fFy for ; Fri, 18 Oct 2019 14:44:41 +0000 (UTC) (envelope-from mike@sentex.net) Received: from [192.168.43.29] ([192.168.43.29]) by pyroxene2a.sentex.ca (8.15.2/8.15.2) with ESMTPS id x9IEieQr045164 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Fri, 18 Oct 2019 10:44:40 -0400 (EDT) (envelope-from mike@sentex.net) Subject: Re: SSH error messages (bug id=234793) ) RELENG_12 To: Matt Garber Cc: freebsd-stable@freebsd.org References: <100597e5-4491-f455-d247-59f5374ea6a4@sentex.net> <246561E5-9E57-4CC2-B94C-4CE8C553D972@gmail.com> From: mike tancsa Openpgp: preference=signencrypt Autocrypt: addr=mike@sentex.net; keydata= mQENBFywzOMBCACoNFpwi5MeyEREiCeHtbm6pZJI/HnO+wXdCAWtZkS49weOoVyUj5BEXRZP xflV2ib2hflX4nXqhenaNiia4iaZ9ft3I1ebd7GEbGnsWCvAnob5MvDZyStDAuRxPJK1ya/s +6rOvr+eQiXYNVvfBhrCfrtR/esSkitBGxhUkBjOti8QwzD71JVF5YaOjBAs7jZUKyLGj0kW yDg4jUndudWU7G2yc9GwpHJ9aRSUN8e/mWdIogK0v+QBHfv/dsI6zVB7YuxCC9Fx8WPwfhDH VZC4kdYCQWKXrm7yb4TiVdBh5kgvlO9q3js1yYdfR1x8mjK2bH2RSv4bV3zkNmsDCIxjABEB AAG0HW1pa2UgdGFuY3NhIDxtaWtlQHNlbnRleC5uZXQ+iQFUBBMBCAA+FiEEmuvCXT0aY6hs 4SbWeVOEFl5WrMgFAlywzOYCGwMFCQHhM4AFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQ eVOEFl5WrMhnPAf7Bf+ola0V9t4i8rwCMGvzkssGaxY/5zNSZO9BgSgfN0WzgmBEOy/3R4km Yn5KH94NltJYAAE5hqkFmAwK6psOqAR9cxHrRfU+gV2KO8pCDc6K/htkQcd/mclJYpCHp6Eq EVJOiAxcNaYuHZkeMdXDuvvI5Rk82VHk84BGgxIqIrhLlkguoPbXOOa+8c/Mpb1sRAGZEOuX EzKNC49+GS9gKW6ISbanyPsGEcFyP7GKMzcHBPf3cPrewZQZ6gBoNscasL6IJeAQDqzQAxbU GjO0qBSMRgnLXK7+DJlxrYdHGXqNbV6AYsmHJ6c2WWWiuRviFBqXinlgJ2FnYebZPAfWibkB DQRcsMzkAQgA1Dpo/xWS66MaOJLwA28sKNMwkEk1Yjs+okOXDOu1F+0qvgE8sVmrOOPvvWr4 axtKRSG1t2QUiZ/ZkW/x/+t0nrM39EANV1VncuQZ1ceIiwTJFqGZQ8kb0+BNkwuNVFHRgXm1 qzAJweEtRdsCMohB+H7BL5LGCVG5JaU0lqFU9pFP40HxEbyzxjsZgSE8LwkI6wcu0BLv6K6c Lm0EiHPOl5G8kgRi38PS7/6s3R8QDsEtbGsYy6O82k3zSLIjuDBwA9GRaeigGppTxzAHVjf5 o9KKu4O7gC2KKVHPegbXS+GK7DU0fjzX57H5bZ6komE5eY4p3oWT/CwVPSGfPs8jOwARAQAB iQE8BBgBCAAmFiEEmuvCXT0aY6hs4SbWeVOEFl5WrMgFAlywzOQCGwwFCQHhM4AACgkQeVOE Fl5WrMhmjQf/dBCjAVn1J0GzSsHiLvSAQz1cchbdy8LD0Tnpzjgp5KLU7sNojbI8vqt4yKAi cayI88j8+xxNXPMWM4pHELuUuVHS5XTpHa/wwulUtI5w/zyKlUDsIvqTPZLUEwH7DfNBueVM WyNaIjV2kxSmM8rNMC+RkgyfbjGLCkmWsMRVuLIUYpl5D9WHmenUbiErlKU2KvEEXEg/aLKq 3m/AdM9RAYsP9O4l+sAZEfyYoNJzDhTZMzn/9Q0uFPLK9smDQh4WBTFaApveVJPHRKmHPoNF Xxj+yScYdQ4SKH34WnhNSELvnZQ3ulH5tpASmm0w+GxfZqSc8+QCwoKtBRDUxoE56A== Message-ID: <07bac044-7506-e4a9-9d6a-f89aade926b4@sentex.net> Date: Fri, 18 Oct 2019 10:44:40 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.9.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 46vphn18C0z4fFy X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of mike@sentex.net designates 2607:f3e0:0:3::19 as permitted sender) smtp.mailfrom=mike@sentex.net X-Spamd-Result: default: False [0.78 / 15.00]; ARC_NA(0.00)[]; RDNS_NONE(1.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f3e0::/32]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TAGGED_RCPT(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[sentex.net]; HFILTER_HELO_IP_A(1.00)[pyroxene2a.sentex.ca]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; HFILTER_HELO_NORES_A_OR_MX(0.30)[pyroxene2a.sentex.ca]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; IP_SCORE(-1.72)[ipnet: 2607:f3e0::/32(-4.93), asn: 11647(-3.56), country: CA(-0.09)]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11647, ipnet:2607:f3e0::/32, country:CA]; HFILTER_HOSTNAME_UNKNOWN(2.50)[]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Oct 2019 14:44:42 -0000 On 10/18/2019 10:36 AM, Matt Garber wrote: >>> Does anyone know what the cause is of this fail message ? >>> >>> (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=234793) >>> >>> its triggered by a normal ssh key'd login, but sshd is running with >>> VERBOSE logging. >>> >>> sshd[63290]: Failed unknown for testuser1 from 192.168.xx.yyy port >>> 60643 ssh2 ? >>> >>> The user is able to login no problem, but the error message is bubbling >>> up in our HIDS. We had to white list it, but it would be useful to >>> understand exactly why and what is failing. >>> >>> —Mike >> It’s one of the other SSH authentication types (e.g., GSSAPI, password, etc.) which is in the processing order before public key. I’m assuming you’re seeing that ‘failure’ immediately before your successful key authentication in auth.log; I actually had to switch back to INFO for logging because that ‘failure’ trips up sshguard which kicks in and blocks the IP despite the public key auth succeeding right after whichever other auth type is tried and fails. >> >> (Unfortunately, I wasn’t able to determine which specific other authentication type was being tried first, since moving logging back to INFO resolved my immediate issue of getting blocked by sshguard before successfully processing my key.) > I’d also like to point out that whatever authentication method is now being tried first was a change from 11.3-RELEASE, as I didn’t encounter that ordering issue in my VERBOSE logs triggering sshguard until after upgrading to 12.0-RELEASE. I always have password auth disabled (only use public keys), but also tried explicit disable statements for GSSAPI and the several other auth types I could think of, but unfortunately wasn’t able to determine which auth type that log line corresponded to. It could also be an auth type that was previously used, but sshd in 12.0-RELEASE re-ordered the processing sequence to try it before public keys. If you crank it up to debug3, its even stranger.  There are a two failed unknowns, and one is after the key'd authentication has been accepted. The client I am using, (SecureCRT) has only Public Key auth and has everything else disabled. Oct 18 10:35:35 ryzen-r12 sshd[63439]: debug1: trying public key file /home/testuser1/.ssh/authorized_keys Oct 18 10:35:35 ryzen-r12 sshd[63439]: debug3: mm_request_send entering: type 51 Oct 18 10:35:35 ryzen-r12 sshd[63439]: debug1: fd 4 clearing O_NONBLOCK Oct 18 10:35:35 ryzen-r12 sshd[63439]: Failed unknown for testuser1 from 192.168.43.29 port 63170 ssh2 Oct 18 10:35:35 ryzen-r12 sshd[63439]: debug1: /home/testuser1/.ssh/authorized_keys:2: matching key found: RSA SHA256:xxxxxx     ---Mike From owner-freebsd-stable@freebsd.org Fri Oct 18 14:55:36 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 61B59152F20 for ; Fri, 18 Oct 2019 14:55:36 +0000 (UTC) (envelope-from matt.garber@gmail.com) Received: from mail-qk1-x732.google.com (mail-qk1-x732.google.com [IPv6:2607:f8b0:4864:20::732]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46vpxM1TbXz4flM for ; Fri, 18 Oct 2019 14:55:34 +0000 (UTC) (envelope-from matt.garber@gmail.com) Received: by mail-qk1-x732.google.com with SMTP id y189so5555643qkc.3 for ; Fri, 18 Oct 2019 07:55:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=aBYi2lx8JplkJuuVIX+Nd1rVhPGWfspJS79fvacHTFA=; b=GS0nkMTcOZ90SLPAgbQmjR9h5hsfFXBjQo1Cwkc376xPZu5ikYgKf3LUmvAdd5vLJQ WU7dtYJRduXpp31GgRA+SIK0eNM9AZdnKWNxU8325D2A5Ggatl6RfENsxOzxAmaFlYKS 5WcYULa78G/QHE0OqB2wOiVkLTLR0UaXhbsv1OJAGeSd2/kqP3tC+8I6e5cM/eJxJKsI +kDcYDUjuJOcNV2AxrIuD88PDpR+EDcOLpW8UpyQGoIe5AnHDXmDNyNenVMi+G+Y892s Cb2AzbzrMgXr8xcZxDTK8UJRGZEPukRZE3zfxvdNAL2MD3qfV/mkxQI82c+Gw+L56R0t 0A0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=aBYi2lx8JplkJuuVIX+Nd1rVhPGWfspJS79fvacHTFA=; b=BuhrxsopLiK7a320Gu0/zmqzb2LozHgUeFKHGm9hQBNeIbhWT/reEx9UfkKzZ3MSyl TNWUuOryYlzHc2FMScaTKSRE/TeTPMcZBtnTdHubbgRI7h3SxHPE766ZF71qbDUrH9RE wXB7W0WgF/gU31J0plQXKhytOY8V0SAYnMTBhdHKqe7GX9tTTsIF4T0+rnLcRQzuBUek dHI3IoI4OzVT5WPT+S1L+d/d73/zdXNXwEgJC9ksZa54K/uOEan3O/txtV/PW+3X/Up5 a2z3odeujPZUlhycfPNwkITDU0cObowyjY+7yiH3rYVtI0ldE7e1vohmKwaTocBr4aVj AACQ== X-Gm-Message-State: APjAAAVhICw4/CZBU1jYCKpLP1xYyLliT+9CcXDDuBr1xrIzCDozX/Nm btXbYvGeSVitrmbJOfKFJi4= X-Google-Smtp-Source: APXvYqzEGG6NQ7mIEzRkM+7PxC/cVcfCnzuOO43xj8uOtmIHFY/zFf+WrIdZARGUdsMWDyCSsV3TPQ== X-Received: by 2002:a05:620a:74b:: with SMTP id i11mr8905261qki.417.1571410533156; Fri, 18 Oct 2019 07:55:33 -0700 (PDT) Received: from [10.100.20.3] ([68.183.62.201]) by smtp.gmail.com with ESMTPSA id t40sm3742412qta.36.2019.10.18.07.55.32 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 18 Oct 2019 07:55:32 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\)) Subject: Re: SSH error messages (bug id=234793) ) RELENG_12 From: Matt Garber In-Reply-To: <07bac044-7506-e4a9-9d6a-f89aade926b4@sentex.net> Date: Fri, 18 Oct 2019 10:55:31 -0400 Cc: freebsd-stable@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <63F56530-DA75-425B-9399-6D41DF0B119E@gmail.com> References: <100597e5-4491-f455-d247-59f5374ea6a4@sentex.net> <246561E5-9E57-4CC2-B94C-4CE8C553D972@gmail.com> <07bac044-7506-e4a9-9d6a-f89aade926b4@sentex.net> To: mike tancsa X-Mailer: Apple Mail (2.3445.104.11) X-Rspamd-Queue-Id: 46vpxM1TbXz4flM X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=GS0nkMTc; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of mattgarber@gmail.com designates 2607:f8b0:4864:20::732 as permitted sender) smtp.mailfrom=mattgarber@gmail.com X-Spamd-Result: default: False [-2.50 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MV_CASE(0.50)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(0.00)[ip: (-9.45), ipnet: 2607:f8b0::/32(-2.46), asn: 15169(-2.09), country: US(-0.05)]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2.3.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Oct 2019 14:55:36 -0000 >>>> Does anyone know what the cause is of this fail message ? >>>>=20 >>>> (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D234793) >>>>=20 >>>> its triggered by a normal ssh key'd login, but sshd is running with >>>> VERBOSE logging.=20 >>>>=20 >>>> sshd[63290]: Failed unknown for testuser1 from 192.168.xx.yyy port >>>> 60643 ssh2 ? >>>>=20 >>>> The user is able to login no problem, but the error message is = bubbling >>>> up in our HIDS. We had to white list it, but it would be useful to >>>> understand exactly why and what is failing. >>>>=20 >>>> =E2=80=94Mike >>> It=E2=80=99s one of the other SSH authentication types (e.g., = GSSAPI, password, etc.) which is in the processing order before public = key. I=E2=80=99m assuming you=E2=80=99re seeing that =E2=80=98failure=E2=80= =99 immediately before your successful key authentication in auth.log; I = actually had to switch back to INFO for logging because that = =E2=80=98failure=E2=80=99 trips up sshguard which kicks in and blocks = the IP despite the public key auth succeeding right after whichever = other auth type is tried and fails. >>>=20 >>> (Unfortunately, I wasn=E2=80=99t able to determine which specific = other authentication type was being tried first, since moving logging = back to INFO resolved my immediate issue of getting blocked by sshguard = before successfully processing my key.) >> I=E2=80=99d also like to point out that whatever authentication = method is now being tried first was a change from 11.3-RELEASE, as I = didn=E2=80=99t encounter that ordering issue in my VERBOSE logs = triggering sshguard until after upgrading to 12.0-RELEASE. I always have = password auth disabled (only use public keys), but also tried explicit = disable statements for GSSAPI and the several other auth types I could = think of, but unfortunately wasn=E2=80=99t able to determine which auth = type that log line corresponded to. It could also be an auth type that = was previously used, but sshd in 12.0-RELEASE re-ordered the processing = sequence to try it before public keys. >=20 > If you crank it up to debug3, its even stranger. There are a two = failed > unknowns, and one is after the key'd authentication has been accepted. > The client I am using, (SecureCRT) has only Public Key auth and has > everything else disabled. >=20 > Oct 18 10:35:35 ryzen-r12 sshd[63439]: debug1: trying public key file > /home/testuser1/.ssh/authorized_keys > Oct 18 10:35:35 ryzen-r12 sshd[63439]: debug3: mm_request_send = entering: > type 51 > Oct 18 10:35:35 ryzen-r12 sshd[63439]: debug1: fd 4 clearing = O_NONBLOCK > Oct 18 10:35:35 ryzen-r12 sshd[63439]: Failed unknown for testuser1 = from > 192.168.43.29 port 63170 ssh2 > Oct 18 10:35:35 ryzen-r12 sshd[63439]: debug1: > /home/testuser1/.ssh/authorized_keys:2: matching key found: RSA > SHA256:xxxxxx I think it must be something that the server is trying even if the = client doesn=E2=80=99t actually send that type, since I also tested with = OpenSSH on the client end (macOS 10.14, OpenSSH_7.9p1, LibreSSL 2.7.3) = only specifying public key authentication =E2=80=93 with all of its = other auth types disabled =E2=80=93 and still had the same problem on my = upgraded 12.0-RELEASE systems. Thanks, -- Matt Garber From owner-freebsd-stable@freebsd.org Fri Oct 18 20:54:01 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1524E15A883; Fri, 18 Oct 2019 20:54:01 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46vytx0FSkz41Pr; Fri, 18 Oct 2019 20:54:01 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by freefall.freebsd.org (Postfix) with ESMTPS id 5AC17149D0; Fri, 18 Oct 2019 20:54:00 +0000 (UTC) (envelope-from gjb@freebsd.org) Date: Fri, 18 Oct 2019 20:53:57 +0000 From: Glen Barber To: freebsd-stable@freebsd.org, freebsd-snapshots@freebsd.org Cc: FreeBSD Release Engineering Team Subject: FreeBSD 12.1-RC2 Now Available Message-ID: <20191018205357.GF32642@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; x-action=pgp-signed User-Agent: Mutt/1.12.1 (2019-06-15) X-Mailman-Approved-At: Fri, 18 Oct 2019 20:57:50 +0000 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Oct 2019 20:54:01 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 The second RC build of the 12.1-RELEASE release cycle is now available. Installation images are available for: o 12.1-RC2 amd64 GENERIC o 12.1-RC2 i386 GENERIC o 12.1-RC2 powerpc GENERIC o 12.1-RC2 powerpc64 GENERIC64 o 12.1-RC2 powerpcspe MPC85XXSPE o 12.1-RC2 sparc64 GENERIC o 12.1-RC2 armv6 RPI-B o 12.1-RC2 armv7 BANANAPI o 12.1-RC2 armv7 BEAGLEBONE o 12.1-RC2 armv7 CUBIEBOARD o 12.1-RC2 armv7 CUBIEBOARD2 o 12.1-RC2 armv7 CUBOX-HUMMINGBOARD o 12.1-RC2 armv7 RPI2 o 12.1-RC2 armv7 PANDABOARD o 12.1-RC2 armv7 WANDBOARD o 12.1-RC2 armv7 GENERICSD o 12.1-RC2 aarch64 GENERIC o 12.1-RC2 aarch64 RPI3 o 12.1-RC2 aarch64 PINE64 o 12.1-RC2 aarch64 PINE64-LTS Note regarding arm SD card images: For convenience for those without console access to the system, a freebsd user with a password of freebsd is available by default for ssh(1) access. Additionally, the root user password is set to root. It is strongly recommended to change the password for both users after gaining access to the system. Installer images and memory stick images are available here: https://download.freebsd.org/ftp/releases/ISO-IMAGES/12.1/ The image checksums follow at the end of this e-mail. If you notice problems you can report them through the Bugzilla PR system or on the -stable mailing list. If you would like to use SVN to do a source based update of an existing system, use the "releng/12.1" branch. A summary of changes since 12.1-RC1 includes: o The loader.efi had been updated to use ioalign for compliance with UEFI specification 2.7A. o A null pointer dereference bug had been fixed. o A fix to SCTP to reset local variables to their initial values had been added. o The ixgbe(4) driver had been updated to prevent a system crash when configuring EEE on X500EM_X devices. o The sdhci(4) driver had been updated to fix a boot issue on Beaglebone SoCs. A list of changes since 12.0-RELEASE is available in the releng/12.1 release notes: https://www.freebsd.org/releases/12.1R/relnotes.html Please note, the release notes page is not yet complete, and will be updated on an ongoing basis as the 12.1-RELEASE cycle progresses. === Virtual Machine Disk Images === VM disk images are available for the amd64, i386, and aarch64 architectures. Disk images may be downloaded from the following URL (or any of the FreeBSD download mirrors): https://download.freebsd.org/ftp/releases/VM-IMAGES/12.1-RC2/ The partition layout is: ~ 16 kB - freebsd-boot GPT partition type (bootfs GPT label) ~ 1 GB - freebsd-swap GPT partition type (swapfs GPT label) ~ 20 GB - freebsd-ufs GPT partition type (rootfs GPT label) The disk images are available in QCOW2, VHD, VMDK, and raw disk image formats. The image download size is approximately 135 MB and 165 MB respectively (amd64/i386), decompressing to a 21 GB sparse image. Note regarding arm64/aarch64 virtual machine images: a modified QEMU EFI loader file is needed for qemu-system-aarch64 to be able to boot the virtual machine images. See this page for more information: https://wiki.freebsd.org/arm64/QEMU To boot the VM image, run: % qemu-system-aarch64 -m 4096M -cpu cortex-a57 -M virt \ -bios QEMU_EFI.fd -serial telnet::4444,server -nographic \ -drive if=none,file=VMDISK,id=hd0 \ -device virtio-blk-device,drive=hd0 \ -device virtio-net-device,netdev=net0 \ -netdev user,id=net0 Be sure to replace "VMDISK" with the path to the virtual machine image. === Amazon EC2 AMI Images === FreeBSD/amd64 EC2 AMIs are available in the following regions: eu-north-1 region: ami-0186d6a5fbc8766f2 ap-south-1 region: ami-0b6bef3551f1b0f70 eu-west-3 region: ami-062495360178ede5e eu-west-2 region: ami-0ccfe49c85e5f8cc0 eu-west-1 region: ami-0e2730782e7462f98 ap-northeast-2 region: ami-053ddd72fc1feb00a ap-northeast-1 region: ami-06cd2e1981334f254 sa-east-1 region: ami-08acf6b9b1df41f34 ca-central-1 region: ami-064249d804369c668 ap-east-1 region: ami-020c406cb2f52030b ap-southeast-1 region: ami-08264f040bf980098 ap-southeast-2 region: ami-0da02f500e46cac8f eu-central-1 region: ami-05458e84d05b820e8 us-east-1 region: ami-06f6cbd134064befb us-east-2 region: ami-0cfe92105f4fee6a8 us-west-1 region: ami-0bb63fac9c5ec153a us-west-2 region: ami-00a29b19544968928 FreeBSD/aarch64 EC2 AMIs are available in the following regions: eu-north-1 region: ami-0ea4448b9b547107c ap-south-1 region: ami-07a9fd713466fe63f eu-west-3 region: ami-02e84241865e90f54 eu-west-2 region: ami-0b707024f9aadb94f eu-west-1 region: ami-0abf12b852be4e776 ap-northeast-2 region: ami-086547036e5a47816 ap-northeast-1 region: ami-038017fcbf85e7669 sa-east-1 region: ami-0da52f30dd7d86ef5 ca-central-1 region: ami-092ee6a89213c15a2 ap-east-1 region: ami-0db28099cf79bf65d ap-southeast-1 region: ami-0852402b94d58adf8 ap-southeast-2 region: ami-01f869cc877cef54f eu-central-1 region: ami-04d008006fdb7e720 us-east-1 region: ami-0411db3e8715d4352 us-east-2 region: ami-01e68c35d7ddcac3e us-west-1 region: ami-02dcdcd99bf7fde1f us-west-2 region: ami-09ce8334b595dff30 === Vagrant Images === FreeBSD/amd64 images are available on the Hashicorp Atlas site, and can be installed by running: % vagrant init freebsd/FreeBSD-12.1-RC2 % vagrant up === Upgrading === The freebsd-update(8) utility supports binary upgrades of amd64 and i386 systems running earlier FreeBSD releases. Systems running earlier FreeBSD releases can upgrade as follows: # freebsd-update upgrade -r 12.1-RC2 During this process, freebsd-update(8) may ask the user to help by merging some configuration files or by confirming that the automatically performed merging was done correctly. # freebsd-update install The system must be rebooted with the newly installed kernel before continuing. # shutdown -r now After rebooting, freebsd-update needs to be run again to install the new userland components: # freebsd-update install It is recommended to rebuild and install all applications if possible, especially if upgrading from an earlier FreeBSD release, for example, FreeBSD 11.x. Alternatively, the user can install misc/compat11x and other compatibility libraries, afterwards the system must be rebooted into the new userland: # shutdown -r now Finally, after rebooting, freebsd-update needs to be run again to remove stale files: # freebsd-update install == ISO CHECKSUMS == o 12.1-RC2 amd64 GENERIC: SHA512 (FreeBSD-12.1-RC2-amd64-bootonly.iso) = 2d77661814b4a332ca0a35b5746170264dc4b6a971260e4be72660f3e39f214605028a51448412888589e924fc68792370cce46b050f3b7aa714ae7532d36a93 SHA512 (FreeBSD-12.1-RC2-amd64-bootonly.iso.xz) = 1d185494ac4db87fa798061b7748c33329f76c598dd27ab5592e373426eba58078e7e3e4dc135130deb9d28e8e758460f66b7f82615e41e0cbfbae9fbe335ed4 SHA512 (FreeBSD-12.1-RC2-amd64-disc1.iso) = 98b33b08cf6834db482f375df47e18b58613d0da5b3cc65a2b1f1c0a2701b87b84101f760e9a7b6d3fc28671b6dfa7b34e53e4393f4640e1de0f72f121e5990b SHA512 (FreeBSD-12.1-RC2-amd64-disc1.iso.xz) = 80162579a7033e71213b2ff6de2e20c08914c3586e6d93023e51f8ddb18b4af8c58b6f1d2347b7fddb8630d7efcedc85e681421512286a77cd21c784277fa073 SHA512 (FreeBSD-12.1-RC2-amd64-dvd1.iso) = dacecb509cbfaca20b9d6802d74941138f21a146f0b4104b39ef10f5a0cf2cf43f65fc9b4da031af01da71153526dc96f6f7b67995bf1d9a612327e48609351f SHA512 (FreeBSD-12.1-RC2-amd64-dvd1.iso.xz) = efa9672958db5cc351a90310243207e4e01f69281d257f891736eb392183eccae9cbbc8532990f1a2014e5580ef41ee239c54d04478349f90f2631438db2a0d8 SHA512 (FreeBSD-12.1-RC2-amd64-memstick.img) = 0e390137ed7356ebde4fadf4ac5ee1921b90fb3a4228cc3e937bcbb248c404929d36fe54e4862708c6d6d5f3cd39c98d2a13b548e1e11d89997205199c2623ab SHA512 (FreeBSD-12.1-RC2-amd64-memstick.img.xz) = b9878eae0869d3e3e5938d64cef893491e85938fdad1bb8e16155a3276324b16593ab733c28ca10a2ec67267e2e7c5c741b4c6ea43c146525b4e1cf0de5cf7a3 SHA512 (FreeBSD-12.1-RC2-amd64-mini-memstick.img) = 79e2770891686b4d0b964aef052bccc46adfff9bed40a6fc1d905b9eddee7158183ed451741a82b301472841ead1f80e0bd29f862a861f07b8dd76aedf89a754 SHA512 (FreeBSD-12.1-RC2-amd64-mini-memstick.img.xz) = 0ab81b6cab12ab09be49fe5cc63e32ddd316c9cdf0548bd826cbc1386ca774af19a376b99dd691532a720c313f2c0ddc19c2dbb9580a3457d5e04b61e32db3ff SHA256 (FreeBSD-12.1-RC2-amd64-bootonly.iso) = d1166c65bb8aadd29324161050f14f528d0f27d5dc6ef8536f2cae4b85b1929f SHA256 (FreeBSD-12.1-RC2-amd64-bootonly.iso.xz) = 9bd8ed5f012ba0fba2027faebba1d36420632c7cacd44f4b4119dcfd3dfb8bc0 SHA256 (FreeBSD-12.1-RC2-amd64-disc1.iso) = c142e71b5658d317f1d837739f0430fcbe99d4cefe246ef999cf7fe2c9abc78e SHA256 (FreeBSD-12.1-RC2-amd64-disc1.iso.xz) = d31945846eaac659987674a35c7007a302234682e7b844bfcc2fec3728d45904 SHA256 (FreeBSD-12.1-RC2-amd64-dvd1.iso) = 70ab17f7882ef528328f29026d58825b87d2a732279a20df4d0fe343804f8675 SHA256 (FreeBSD-12.1-RC2-amd64-dvd1.iso.xz) = 700f32b2472e2bdb6167e439fd19f3ece32bacbf63fc0c9de2b31ba5df9c7b09 SHA256 (FreeBSD-12.1-RC2-amd64-memstick.img) = b44943a6989027823c7a90a6f570348425a3a03db97eab6afec343ffa7d7b021 SHA256 (FreeBSD-12.1-RC2-amd64-memstick.img.xz) = cf32f19bffd884b65df371798b69a16673b942ecb9f19391877d8077feb73889 SHA256 (FreeBSD-12.1-RC2-amd64-mini-memstick.img) = 4c6bc97d9e124ea295c71de2159bfa90a75a41df9c347e21bfe4d01de098650a SHA256 (FreeBSD-12.1-RC2-amd64-mini-memstick.img.xz) = 0a841efbebe056c6568c02c336e727ee7858eee113101627c6d7f5685563ddeb o 12.1-RC2 i386 GENERIC: SHA512 (FreeBSD-12.1-RC2-i386-bootonly.iso) = 2c663c077c4d1bef4a476a93b492db0a65b6d807635454afae523350e52536e3ffa4548dcb6ae000682185044bd3d003cd6c05a7bde1f29bc48c49ee6a96ea43 SHA512 (FreeBSD-12.1-RC2-i386-bootonly.iso.xz) = 97c9496a70ef1876997b2b837d1e47bb975c0d0b615a436927a6ab993b80b98a561fb849ce4bd0196cfbcaaadfd4f57eb25115c788528c71c5ec2214402070d1 SHA512 (FreeBSD-12.1-RC2-i386-disc1.iso) = 6a9500012d8df9ff6221b62357c5404dc82529cfd2cee602a7db416d2507285c5f6f5645f8e40030ea8a0498e54e286fa8faf526c3220f84966608e380af67e1 SHA512 (FreeBSD-12.1-RC2-i386-disc1.iso.xz) = 08539e89fba4f142684b18cd214d21489b65d7de8b812f25471d75375317161abd29072ad53e247989b6eef9b3c9bc675dcade91b8eeb98e15ee80dc16d059d3 SHA512 (FreeBSD-12.1-RC2-i386-dvd1.iso) = d9daea0e279e3a4a5371721893991f2303d50b63c04e571c585923fb188b82933b2d7267f4b15cda4f348db86b52e5668f0dc3b9afe5a5e5128d522cc5e55572 SHA512 (FreeBSD-12.1-RC2-i386-dvd1.iso.xz) = 9bbfb70d68c682262704322f7139bf6a99dafa7cf8c921c87e3de23449a0d5594ce20f06c68fbc70c66fb9a448ff769147054fb961ba20ce1225736b6e4d7e21 SHA512 (FreeBSD-12.1-RC2-i386-memstick.img) = 9e4172c07335ef9a590acdbe31fd4fe53f2cf4ab50a5ee16ee834d7276697bf6b04eabf1711479c9d2f6ac7a4c2bd53a944f66940dae6a7c7728097fa8e2d34d SHA512 (FreeBSD-12.1-RC2-i386-memstick.img.xz) = c374e0a62a3a8d1c99968e4586a7fcd4a6bd152dccb82bd79e18df319979d6eaaeecb340356ee09211e5d486a352bc65b0605a5c7287fb443f1c7cbaed1dbb6e SHA512 (FreeBSD-12.1-RC2-i386-mini-memstick.img) = 8d5699540932d19e57afa5f6bb33ef1ae7b8aced092ffbe1d912c9a29873bc7d6ca1570a999db8969cf4838d5dccd395750ce67b10ecd2d3d29c0fc0bc627c6a SHA512 (FreeBSD-12.1-RC2-i386-mini-memstick.img.xz) = f285c3d734bc51d2dc98b2986bedae3210e22b969fa49faa460ba629911cf5e28e041340deb5eafab6f8659371f060da30b408a798965f2f4b897d24bc1e3b14 SHA256 (FreeBSD-12.1-RC2-i386-bootonly.iso) = 91446694a5bf65f02871cd2c75df24378695493abcf0880ddfbb144c95df72fd SHA256 (FreeBSD-12.1-RC2-i386-bootonly.iso.xz) = b8fe264ecc91079b3b72c3ba9f37f4257b4522032357daf259cf6788bfcc857f SHA256 (FreeBSD-12.1-RC2-i386-disc1.iso) = dbaab1c6579524e675cf22f9e0cd15b5ede240657c1dff11045aff097b91ac0a SHA256 (FreeBSD-12.1-RC2-i386-disc1.iso.xz) = 4a3a79b96676fbd542fe08bf79f7b3e61f3ca69b3ff7b6891064a5e7854e3824 SHA256 (FreeBSD-12.1-RC2-i386-dvd1.iso) = 427325842b032b87cc0f59e700c201c223caabe7060f7fe688f0117da2b1b21e SHA256 (FreeBSD-12.1-RC2-i386-dvd1.iso.xz) = c4979fd5a16f8cf6b750ec4736f5e767f13496edb5460c3076a49149f9b25eb1 SHA256 (FreeBSD-12.1-RC2-i386-memstick.img) = 0b30f1285dce25af9b47ac15bec174890c7684112396a2115abcb69556dd49ab SHA256 (FreeBSD-12.1-RC2-i386-memstick.img.xz) = 618a08408418629d8322e68c018a5951944693e053672cea64aae5d93fe87378 SHA256 (FreeBSD-12.1-RC2-i386-mini-memstick.img) = 0e329d9c5452e6c4059f7f878b5bde58ffbca8ef0768656301bf1d5a040b43b0 SHA256 (FreeBSD-12.1-RC2-i386-mini-memstick.img.xz) = 6a26f54940aaa6b779403662af199e39d374fbee835af69560f67e31a7479e2f o 12.1-RC2 powerpc GENERIC: SHA512 (FreeBSD-12.1-RC2-powerpc-bootonly.iso) = 3170ecfe594dc214b0e04aea1e95b002d2ce4354d72b6fc45857ab2a1d26712e7871fe1e340008f8db5d9647a42fb8f26c0a2f8cf682f5e9dae8353a7b38f36c SHA512 (FreeBSD-12.1-RC2-powerpc-bootonly.iso.xz) = 1fd21b7ef9f5cf6bf5baf619839973d8724bc416dd1b18669d2b26c4e733715ddcc5994f82b25daf164b8a327bfc7d60ae4f1eca6eaa7d01fc0a0b754db9bc86 SHA512 (FreeBSD-12.1-RC2-powerpc-disc1.iso) = c4ae1086f39abab2c3c2867a2deba7332d7d62ea17d6e56adc09d59edf2499b247e07ebec51716515d10ab2b55d8974e3c0a5ffa29773c368873d99fb93ae8c1 SHA512 (FreeBSD-12.1-RC2-powerpc-disc1.iso.xz) = 41b91ae2c36dca46ef94d88557626d48e001ca09c4db9575a8064bd2e2a650124db83d0d9f1527f87d55d0e9701012d64c5bf5f1c011ce781c8d071254552b17 SHA512 (FreeBSD-12.1-RC2-powerpc-dvd1.iso) = f79dbd2391343461071221048604ae985d2a63e4bc6b1c0de564735a98c1693a297a0a611788ddf141689a83d382b39f843b763d058a747a42ffd305798bc008 SHA512 (FreeBSD-12.1-RC2-powerpc-dvd1.iso.xz) = 3d094c0eeff6501d81b21d99634c4d34e436df1e27d9b703ccb1d190bba4dd9f2132cb267601daf5394c535db4a5ca025bdafe32902643637ae6ba9401d0d550 SHA512 (FreeBSD-12.1-RC2-powerpc-memstick.img) = bcce9326f55a82e9643798ff4589d6121459be865c67e052b8665a9e240e1ef648258c01a31197ccd70709a91b1cf0ca0f6c8922397fe7aabd695897498c18f3 SHA512 (FreeBSD-12.1-RC2-powerpc-memstick.img.xz) = b3dd297d6511ee96b7ca6b97af86b00a41e5b102aa89f73d52756a73a52e68d9eadf67d0a81cfca053084c00c510aa89b8890a29d73b8bb5bc277a699ad6447c SHA512 (FreeBSD-12.1-RC2-powerpc-mini-memstick.img) = 2fd1342ba1e1f9b6e1e79b948e4ad03c2d297e0a150e206cbfcfe8860527c21eea405856f08951490268ae7ada4c5c901877e690be5ea91fa1ddd936c422c5e9 SHA512 (FreeBSD-12.1-RC2-powerpc-mini-memstick.img.xz) = 7a42e12da7e38da5271d4dbcae429c80f1f88c71af20f498c2dffe54fa305a5c57769beba7b22ef39e5620aeb349b86f3f7dc3a5c2761fbf500c2b253788383c SHA256 (FreeBSD-12.1-RC2-powerpc-bootonly.iso) = 9c8c031f433410a70bc864f8fa5458d7e71668a0dccdd4a94240ed08cdd0d06c SHA256 (FreeBSD-12.1-RC2-powerpc-bootonly.iso.xz) = 8b6196e4c9b062595569fb38a63310e28e9a5265d455e1c8e4eab188de4a0f01 SHA256 (FreeBSD-12.1-RC2-powerpc-disc1.iso) = 847450af8d8412016bcbd430819dee789872c5e3b4bdc85bb083ba466b7963ae SHA256 (FreeBSD-12.1-RC2-powerpc-disc1.iso.xz) = e8bcbf645c283a48ac4be969dc738adfea68405eff8122982ed262b3eccf4878 SHA256 (FreeBSD-12.1-RC2-powerpc-dvd1.iso) = 6ddd7d123fc08ebe17746fbd893d58dcfb065c44d2f981f98a1a1b29a0d6f968 SHA256 (FreeBSD-12.1-RC2-powerpc-dvd1.iso.xz) = 47d4a0c86ad8717deb1ed2986b036072787935a3dac4852bb4e411574cecb42f SHA256 (FreeBSD-12.1-RC2-powerpc-memstick.img) = d62dedd1dce91322fefd706c766a6ca25be1f5a68e069966f94ebfdf5bddeccc SHA256 (FreeBSD-12.1-RC2-powerpc-memstick.img.xz) = 612b43cec8b5b8a084671ece037a627f14ed2bafa0a2089e85436321a38e543a SHA256 (FreeBSD-12.1-RC2-powerpc-mini-memstick.img) = 7e2119bff1f57209786bc425df94b49272619a1c549d36c4a40718af34dde9fb SHA256 (FreeBSD-12.1-RC2-powerpc-mini-memstick.img.xz) = 3f68bec3736155459833807616794b39bcfd0412b14e28d7aabc6b82b114306d o 12.1-RC2 powerpc64 GENERIC64: SHA512 (FreeBSD-12.1-RC2-powerpc-powerpc64-bootonly.iso) = fa34a82ee683770ff53c22b392a9385d743d2c2e1a51a805618973adfde6594c3759a252feb84443fe5342cba5268bf4e56b318a4f1ec1f4f6aaf7e5a2badc98 SHA512 (FreeBSD-12.1-RC2-powerpc-powerpc64-bootonly.iso.xz) = 2f64d36f36129ddb5aff0c7586c912c7ea142e10f14db2b23d40c8fe6b409668283a8c5ef8cdfb1c18761b246f910bbd3ce15f3e02a4545ca8dcad825f476c48 SHA512 (FreeBSD-12.1-RC2-powerpc-powerpc64-disc1.iso) = a53e78af72c2cb5cb018c15ef454c462b0e2d5a8b4d89a967037984b3b2ed388c4b8ad537e73d944781079fe3c0cff2e210d1e248d984c662eb5db9e96a205aa SHA512 (FreeBSD-12.1-RC2-powerpc-powerpc64-disc1.iso.xz) = a68705ebcb5b3ff5a34849562dd9ea3793fb6fdb6a849b073154cc5b3b9186484a508d46cc3fec970b0710ca006ce10b5034b18f944cae779b531e768995e786 SHA512 (FreeBSD-12.1-RC2-powerpc-powerpc64-dvd1.iso) = 263d09e8a172e8810d07ab5c1d4da838bedf480b359cff4d47b374e790d6db189d655586570ab9c58e53423ca71b6f114e264412f1cac76341039d874761d99b SHA512 (FreeBSD-12.1-RC2-powerpc-powerpc64-dvd1.iso.xz) = 88edd16126de620dd7406e26b3fd49ec82f75f1e5231f40f5b517c0efa0adf7fc23c5ce958553ea1475cc1762af907c65cfb153a40f150399608986d06418c2c SHA512 (FreeBSD-12.1-RC2-powerpc-powerpc64-memstick.img) = c0e749c3c9905524ab9b014f55a4aaf09f888768bc02139c05c119e5cd90f612ae4a8074f8ba4c9e670d0ad2f7112af4c11a18ee62c2490921be999b09998059 SHA512 (FreeBSD-12.1-RC2-powerpc-powerpc64-memstick.img.xz) = 2cef577300db16cccae2b064764fb79c98e9d15fc3224e6a4688b16c7604c67151f830229dff5349cb93d0c9c3ebc67cf987f03dda414a4cef9d294691b988d4 SHA512 (FreeBSD-12.1-RC2-powerpc-powerpc64-mini-memstick.img) = 5ee14b593fde1c69c19d3bd8c76fca6879051d22a4969f54854d25998ed66c46a3fb49805695e53f42d28b51597eb8d4e3c4309bfaf483b3e396aae2ef1314fe SHA512 (FreeBSD-12.1-RC2-powerpc-powerpc64-mini-memstick.img.xz) = 58d4e2753f342ce5821242f25535468a4646116ed439658dae6856dc7df127775c5d9e86dbbed602dd2044c1694e2a4516ad103b33cd21c9b570dd84e7697ad2 SHA256 (FreeBSD-12.1-RC2-powerpc-powerpc64-bootonly.iso) = ff6119e6573e114185dd6155801e3889f270ef0c6abb9977ebf9954810a78b9d SHA256 (FreeBSD-12.1-RC2-powerpc-powerpc64-bootonly.iso.xz) = 972cf57c6a5e9ee0a282040015190ff7631b94a6477f5807039c1ae09c3db474 SHA256 (FreeBSD-12.1-RC2-powerpc-powerpc64-disc1.iso) = f97f50cd85ea421aace53a18d52f3ad2c9b2c38029bc510f5407b86e5d2fd222 SHA256 (FreeBSD-12.1-RC2-powerpc-powerpc64-disc1.iso.xz) = d125b4b0a8fd9f54b45253efb86ffe9945027a97138f1f790eec20b30fbca008 SHA256 (FreeBSD-12.1-RC2-powerpc-powerpc64-dvd1.iso) = 00cf5fe48cdc9b69b8459721b9bbd5a45e0215df18a2b826dd647f6a6c94ffd1 SHA256 (FreeBSD-12.1-RC2-powerpc-powerpc64-dvd1.iso.xz) = 0c50ea19580af2047228e1f939642a08aab4b08649b0cfc984c482a5ae2ca3c7 SHA256 (FreeBSD-12.1-RC2-powerpc-powerpc64-memstick.img) = 847f570204954bcb0bafd7151c6ead76d73d062fa0acb99e7d2ad1d793ecf199 SHA256 (FreeBSD-12.1-RC2-powerpc-powerpc64-memstick.img.xz) = 3f9404c6498ba7f572c327ae7e863697a33b98e832607f72bbafcc57f1a6adea SHA256 (FreeBSD-12.1-RC2-powerpc-powerpc64-mini-memstick.img) = bcb9962f64a31e7dcadcef3fa68db26ffa12649624847a11da55c406c6c991b8 SHA256 (FreeBSD-12.1-RC2-powerpc-powerpc64-mini-memstick.img.xz) = 3f7d42114376a92fa323deca2400fad3ee2e04e46b5acae8ec09e3d2dff97e48 o 12.1-RC2 powerpcspe MPC85XXSPE: SHA512 (FreeBSD-12.1-RC2-powerpc-powerpcspe-bootonly.iso) = aab808638e760ff5ac49db7f7fb18c3fed7ec113642f47c02da1b42a21bceedbfb49621563a511c6f7330a798e337efdb7a4a3f315e8ed91650b9de17a4be3cd SHA512 (FreeBSD-12.1-RC2-powerpc-powerpcspe-bootonly.iso.xz) = c2e9092ec050103ef99301665d713f2acda29fb4223389045134bb6c5c10d8f8f8fd9102d2c9341d1c91191c3517ceafc71976c20f099d4ee98c3289199863ed SHA512 (FreeBSD-12.1-RC2-powerpc-powerpcspe-disc1.iso) = 6d300be302281adc568e71f421561afbbddb4e92fa2a5fbac3e2530845f436ca8f245f669d8e6d60e1c91473126c363aa807a8b1abadd905c332df21539af5d3 SHA512 (FreeBSD-12.1-RC2-powerpc-powerpcspe-disc1.iso.xz) = 2abfb4962941bd04a4d00ec698f9241e18e6a1246a70eff8ef7be26f25bfaf01b2c8977acd2ab3144c3dffcfab1188a5717f59a8968a5fd0c8c579a9f49f75ee SHA512 (FreeBSD-12.1-RC2-powerpc-powerpcspe-dvd1.iso) = cf0cb21923d955f340b3d2868a8f0f1936e627bb3b9b2acf4c7fadeb15e14a306381c99cd6556034d0c870eab4f3fc798c783d575ac758de44a2a7f43769a3c3 SHA512 (FreeBSD-12.1-RC2-powerpc-powerpcspe-dvd1.iso.xz) = 986327c85898b7a3225a56cc1575c57f95e9fb71b462f05cdb6321ff39dc3684b9b59d91f854ef63f70061f41e773e18f81b236a173c712208da262c5f7e2995 SHA512 (FreeBSD-12.1-RC2-powerpc-powerpcspe-memstick.img) = f50f66e978eb539566a014fd20479c280f3ba68812e546b67d5c42591132f037b9eb824aa40e38bee9769e98203a34262c2e786ffb266347dadd508e825020c9 SHA512 (FreeBSD-12.1-RC2-powerpc-powerpcspe-memstick.img.xz) = 2bcb61a91cf65710f3e3ace3607d3b2b98c83a8e51f4af5358b2ac50d6f06cf257a31f8a7dd5fb851004c9b8d32c70dd9c28e4691ac6fcf360cacf12f3697e92 SHA512 (FreeBSD-12.1-RC2-powerpc-powerpcspe-mini-memstick.img) = abbfcde1ca0b1aa0c80a0d890d16894b6e23ce27d81a85d60bb3d938953b990b742121cd1f579db75c095972961aafcb74e4630001156a273abec1f975475cb2 SHA512 (FreeBSD-12.1-RC2-powerpc-powerpcspe-mini-memstick.img.xz) = 03abddeb5cf39060c3cbea277251dc90ae8672244c41933d8c4e7dad4bfb1d7aaf76981c4e90223052f7118402d287974e8263925db0e7c08a39802c32cd5f5d SHA256 (FreeBSD-12.1-RC2-powerpc-powerpcspe-bootonly.iso) = 13c4d03e7e2d37e61380a0168258f1033ca6d1daa0d5642c1a709c71ff9607d9 SHA256 (FreeBSD-12.1-RC2-powerpc-powerpcspe-bootonly.iso.xz) = b2de516a94eddd33212158669ce23087d41396cfdc6bcf86b8930aa00e2cb439 SHA256 (FreeBSD-12.1-RC2-powerpc-powerpcspe-disc1.iso) = ebbc12f35ff08ba8590e6e659725adc501cfee122409c7db62610318894d6a3e SHA256 (FreeBSD-12.1-RC2-powerpc-powerpcspe-disc1.iso.xz) = ff6cb81b3c30b4007675ffe5d1f9ddbdbd1b67f6d38de26546128af59bcf5ffa SHA256 (FreeBSD-12.1-RC2-powerpc-powerpcspe-dvd1.iso) = 3dc66f3447944affd2f187dd3a7126913f36c53e1aacca6f6d8f10014141cca8 SHA256 (FreeBSD-12.1-RC2-powerpc-powerpcspe-dvd1.iso.xz) = a0138d1b1df1f0d3855109ed18420155fe5f43f85dfa1fe30a474e28a88e7a70 SHA256 (FreeBSD-12.1-RC2-powerpc-powerpcspe-memstick.img) = cf611ef2f88b991a3ef3dc306199e224c7d2155a91f1fc917fd0f3a8f549cb7c SHA256 (FreeBSD-12.1-RC2-powerpc-powerpcspe-memstick.img.xz) = 7620b0a940f9dd9572e2bf71d5b0827f1de5aa4d2d4d0650605e9d06048d3140 SHA256 (FreeBSD-12.1-RC2-powerpc-powerpcspe-mini-memstick.img) = 4b03b49643bf9f72f6cd8c4a702104caa4ace5cd523a9f30b43bd46ae4a04ba7 SHA256 (FreeBSD-12.1-RC2-powerpc-powerpcspe-mini-memstick.img.xz) = 93cbe100af778ddc42de1ae5f3370c537da29ad884438bc5625962f5ee0a52f7 o 12.1-RC2 sparc64 GENERIC: SHA512 (FreeBSD-12.1-RC2-sparc64-bootonly.iso) = 5485340aa2c86db8836fbf2d55c31d8a19fd30e385f463ca91e32ed14018a8b764759184adaa482ade903cd00ac2c93587a7c450a15af8e009d0eace15d66635 SHA512 (FreeBSD-12.1-RC2-sparc64-bootonly.iso.xz) = 22a4ab53336865bd3860e84d943b7c2cad5b093576bda8a0c1a79ff5cbd72e909472e7733c6c22a4e16ee0995ed6b0b6ff82390d1b23b147741f07b1f26c693d SHA512 (FreeBSD-12.1-RC2-sparc64-disc1.iso) = dfffcbab2caeb37effcf757a7a7ea739cf93fda35892381e70b1827206713eb4d5e2751e436ab3d4ed2c50721b69e8117d0720c386c8914e5b3f4e6580d0921e SHA512 (FreeBSD-12.1-RC2-sparc64-disc1.iso.xz) = 9a6a4fe8927fd74eee885b82b43e1e639b1bbd5e49ecdf6f653d0e0ba776c5b0d90b3aefaad38bfe467cbd27355b9776571ca6b82aeebac32f8fbf7e04df1a86 SHA512 (FreeBSD-12.1-RC2-sparc64-dvd1.iso) = 58dc7d2c964dcb177fd396de05cbbcf5126723750870a4d850d4fea60beb01423734a201d2b208dd811f13df7ce05116e7d204c8f192874dbe15bf5c970a13d6 SHA512 (FreeBSD-12.1-RC2-sparc64-dvd1.iso.xz) = f1a69d6a0a08db23dec5fab358636545f2b235a5660ecdf7301fc53b1b65b7ccbafcd5b73d172246eae4437b4d42aaa97ded5d1a28052691ccd287135ed808fc SHA256 (FreeBSD-12.1-RC2-sparc64-bootonly.iso) = b878fc485d4cea749cc72e1fc3a86e3b9962c384506df2c98cba14d7b5d18f17 SHA256 (FreeBSD-12.1-RC2-sparc64-bootonly.iso.xz) = 04c06248c05f530cbd42e2a2660c3e98c76a9f0d49fcabcc778543f1a6e81edb SHA256 (FreeBSD-12.1-RC2-sparc64-disc1.iso) = 04993fe35e9ecf51628e23917d7b0a4a7167116a91dda2454fa3767f2237b1e5 SHA256 (FreeBSD-12.1-RC2-sparc64-disc1.iso.xz) = 2c496e6a0a9c1c09f7e1b150515a8a4e67b229420550049daf2568821fbbffda SHA256 (FreeBSD-12.1-RC2-sparc64-dvd1.iso) = b6b9e27a8a442576ad1a731ecd43d7c01dc1c27048355f3cb095fa7270a46705 SHA256 (FreeBSD-12.1-RC2-sparc64-dvd1.iso.xz) = 6134ee3824082117633ca81df703b975b33ed77ed81dab5a7cab98cbc08bcb85 o 12.1-RC2 armv6 RPI-B: SHA512 (FreeBSD-12.1-RC2-arm-armv6-RPI-B.img.xz) = 5133397f85ae0a89cc6c878026f1639937fb2904bb6c3391f9d8283206cac55ba901db55ef042c7840abcef290782256c5be57b91242c542a69973509c982c29 SHA256 (FreeBSD-12.1-RC2-arm-armv6-RPI-B.img.xz) = fd2120b183845916f9072d497eb6ea1abfaf0ea9b57f7459dcd4dfb9c5e7baed o 12.1-RC2 armv7 BANANAPI: SHA512 (FreeBSD-12.1-RC2-arm-armv7-BANANAPI.img.xz) = 5b6f7ec59778edb849921110c0b11d2a0bca335debc3ed5dffd38cdf520748bafbd5b7dfa55834ac635095f8a8cd1d7d5a8db80b57b31ee05377cdcbe309b991 SHA256 (FreeBSD-12.1-RC2-arm-armv7-BANANAPI.img.xz) = 0e17acda10743be82aefe3e569c0243fe3cd6567c88a40803ffd26bec358ea96 o 12.1-RC2 armv7 BEAGLEBONE: SHA512 (FreeBSD-12.1-RC2-arm-armv7-BEAGLEBONE.img.xz) = b158eb69ae2ab1c991b805289850781ca41a6e7c00478543ff5f409a803b1ea69e6d7f10e217770235ec3eb5ec77f1b54071fcce74a7e43336beb96e6fc1fdf4 SHA256 (FreeBSD-12.1-RC2-arm-armv7-BEAGLEBONE.img.xz) = 6b28d0fd3e24f110f41a1b6dda9858e6eb1b2f05c1f29b784101a24def92d981 o 12.1-RC2 armv7 CUBIEBOARD: SHA512 (FreeBSD-12.1-RC2-arm-armv7-CUBIEBOARD.img.xz) = 2c6334ba96dfa4bf1aafb71fcc5830136fa20efccb15252d9f337b5634e5bb869de31e59f264898de2fc298070dc88307a20f4d314c254f642c3a6365f2a1306 SHA256 (FreeBSD-12.1-RC2-arm-armv7-CUBIEBOARD.img.xz) = 4b17ad5ab15658ba9fc87454f72b8da8a60949fe5d58b9804403950e918802bf o 12.1-RC2 armv7 CUBIEBOARD2: SHA512 (FreeBSD-12.1-RC2-arm-armv7-CUBIEBOARD2.img.xz) = 59d8c2d3e7b6e51b92e0c820af62b5d0d387e83f62a76624b8aeb5b38326063f8b4e048944c49897084f20e4f59b65bc00c7d136268730cae839e5d8d510ec20 SHA256 (FreeBSD-12.1-RC2-arm-armv7-CUBIEBOARD2.img.xz) = bc5dfddef63904f9a88feedd6c5e268d604681817de0e66e29bf0d53cc928691 o 12.1-RC2 armv7 CUBOX-HUMMINGBOARD: SHA512 (FreeBSD-12.1-RC2-arm-armv7-CUBOX-HUMMINGBOARD.img.xz) = 33196d08133e72717440388912f6400c0a761dff723e0f84763235ba66a76fd0f1b5af2e20e334e27151c385469e597aaf10dd4bd3502a7d63e746eb094d3c86 SHA256 (FreeBSD-12.1-RC2-arm-armv7-CUBOX-HUMMINGBOARD.img.xz) = 284f83be49f848e31077bacd906a5b01edd47f5648feaf299d95a281ff37b630 o 12.1-RC2 armv7 RPI2: SHA512 (FreeBSD-12.1-RC2-arm-armv7-RPI2.img.xz) = 4558dc64cf918e5950290c605b5e3c756707b0c08e5a7eaa8a47a24e01d9ac7e3faca7bb22bb17abf2b869ae3c52028f3f6d1f1c63cf2493b2fd1e51bc0910eb SHA256 (FreeBSD-12.1-RC2-arm-armv7-RPI2.img.xz) = 22323ac595f1ff83b222393761fa46677b9b6b130c50d4855f3a6dde95cb38ee o 12.1-RC2 armv7 PANDABOARD: SHA512 (FreeBSD-12.1-RC2-arm-armv7-PANDABOARD.img.xz) = de8d7c7153079d8f7e08c53b6bf64fc73329657557330761cda510e241a676778cecab3b70faaaf6b0ea8aa56161255874e961c07e51a76bca9ad9adf0cc115b SHA256 (FreeBSD-12.1-RC2-arm-armv7-PANDABOARD.img.xz) = 17bcd7493cf07b79d86d37ddfc03d03a3e1752af4bc62e2e94e7c1dbab9748fb o 12.1-RC2 armv7 WANDBOARD: SHA512 (FreeBSD-12.1-RC2-arm-armv7-WANDBOARD.img.xz) = ea2c6246f66250c690b7927e977a19e602298fbea1ed74017f5bac2b6e5ee41543eabf8cc01bf689eddd83a5e6de6142d0b297f446da5f2303c0b1d81fab8686 SHA256 (FreeBSD-12.1-RC2-arm-armv7-WANDBOARD.img.xz) = 978343c9fd6e190725cf68f96db3e28be33f2deb24681a4b58533e6ae1d8b941 o 12.1-RC2 armv7 GENERICSD: SHA512 (FreeBSD-12.1-RC2-arm-armv7-GENERICSD.img.xz) = 75d81dff411bd5e9c17d43b489e1ba0470c8527b8502b0b0a47452245542f4411241ff09e30015964b97f60b2009dcbb2251937f7a63abbffc79358ac43b1771 SHA256 (FreeBSD-12.1-RC2-arm-armv7-GENERICSD.img.xz) = 8f8e2a21a2be1369fdf4600694164819720989dfb0d2b217c735915b0cdcf1ca o 12.1-RC2 aarch64 GENERIC: SHA512 (FreeBSD-12.1-RC2-arm64-aarch64-memstick.img) = 1299111c31bb30e683aedba7176dfbe2f64b23c65907474e3453a1a7318f4a36b9d48fb574dc3946c840a0677c8412aeede8057d85029b2caea8c6fc9f2d4279 SHA512 (FreeBSD-12.1-RC2-arm64-aarch64-memstick.img.xz) = 323fd5a3f73b9aa34f78b493fd8356196a2e539255aa9e3b3c4a2c0a4d8a82c1e9dc1d240fa1c796153b44ef1137782343f02980b9fda90b3161b047f9242d05 SHA512 (FreeBSD-12.1-RC2-arm64-aarch64-mini-memstick.img) = 9d7b1ff47bb69bf13ed282750b495ce79f0d4ff4bb9eceb5647ee25e8889901dffb675a85e941ede15e50f5f94e67074904ae92a301b75c3169f8f9e11334011 SHA512 (FreeBSD-12.1-RC2-arm64-aarch64-mini-memstick.img.xz) = 9ce55b6ee1f26bc5206c36282bc0c8b252c4d1dc27b2d79c7461271255a4cb45087be201d991a431512e6f0351c6414765c27ac726f43a0b81d9835c0c39dc55 SHA256 (FreeBSD-12.1-RC2-arm64-aarch64-memstick.img) = 8b4ef84836137adfbb4cdc4489a6e274012c794945d692104f4850f5c42e4cbe SHA256 (FreeBSD-12.1-RC2-arm64-aarch64-memstick.img.xz) = 8ad42c6031e54f25d7f1c10c71b53d28c96281f143eac4d458445df8ccf3236b SHA256 (FreeBSD-12.1-RC2-arm64-aarch64-mini-memstick.img) = 7aef49993f8f3742bb1116ed06327136001338314e0da21ad5f0c0da0e75c6e1 SHA256 (FreeBSD-12.1-RC2-arm64-aarch64-mini-memstick.img.xz) = 485d4f96032ae8284021c92f25e4e0bd4e1cd947d0c499f4afbe733ac0d262c4 o 12.1-RC2 aarch64 RPI3: SHA512 (FreeBSD-12.1-RC2-arm64-aarch64-RPI3.img.xz) = ed04880b2fef89d7830a21e3af2b575bb3c7dd28eee5ec9d384553cfb2fb44d3001f9898da8913763ee89a683d7b44c564e1cdf37c162562faadcd7c4261f50a SHA256 (FreeBSD-12.1-RC2-arm64-aarch64-RPI3.img.xz) = 13e480320e1681e99e409ac824229c5f4c16261bd662d3fb3746b6c533b9289c o 12.1-RC2 aarch64 PINE64: SHA512 (FreeBSD-12.1-RC2-arm64-aarch64-PINE64.img.xz) = ccbea14b0c5fdece3e7d4703ef5e273fa49d3afdab0e0c41504acdace0708fe87291ba4756c2bdc6f06e720d0a1b8506e1e07c382c206cbcb6468067b6beff41 SHA256 (FreeBSD-12.1-RC2-arm64-aarch64-PINE64.img.xz) = 383b703131f72d139d0a0625fdd401fac556585772d9aa1968f14c3bb815e9e3 o 12.1-RC2 aarch64 PINE64-LTS: SHA512 (FreeBSD-12.1-RC2-arm64-aarch64-PINE64-LTS.img.xz) = da96cc346023083e76647028d251fdf73df1500f820276d11b04809d7cbe9182bde821fea9e3a5b00b21fbac4565cc35c60d160b9dbfc4efdf46e015c93d3aa7 SHA256 (FreeBSD-12.1-RC2-arm64-aarch64-PINE64-LTS.img.xz) = 0e6ba4422fc195f9520edea689318bc4814c0ccf0ba945d417fcfd50fafd4424 == VM IMAGE CHECKSUMS == o 12.1-RC2 amd64: SHA512 (FreeBSD-12.1-RC2-amd64.qcow2.xz) = f025208831719fe8cb9fcb44a22048037b5dff08b6811501e68c066c3e43a1725befbcf3e67c18a73141c48c4d979fffad61ea54e141761c5de030bf03dbbd4f SHA512 (FreeBSD-12.1-RC2-amd64.raw.xz) = c87940c57822d530d1742a3866fb6824c58907aca254b3888228ddf6893ebe87159b0dfc63ac6fd3538f107a6518ecd501e2ffdf7393d948040efbc450073450 SHA512 (FreeBSD-12.1-RC2-amd64.vhd.xz) = bb0a31d03348c8279ac42aef8e4603538e85a04e93e878e5d1513d70b3de6a12b32680f898b61857f013968f2011dc6bc6958ed2e8e2952dcc0c00a7fff68c08 SHA512 (FreeBSD-12.1-RC2-amd64.vmdk.xz) = d50750a3f3856a3991ccbc262cb3540dd54cfde56fe45034a1d0e336326ebc482f133589e4135df2a322902501ec2f31803ca0889af6dbd0bad61b013893f6e0 SHA256 (FreeBSD-12.1-RC2-amd64.qcow2.xz) = 88208389ced219ecb15aa8e0f3a05491ebf7789883776208832242a32eb61a86 SHA256 (FreeBSD-12.1-RC2-amd64.raw.xz) = a4141b207b073775bb2da0a8385bd7f8dcc4014ca1dd4bc9a2d7cb5c0ba536e0 SHA256 (FreeBSD-12.1-RC2-amd64.vhd.xz) = 0a92830ce97faae3a9a8defb57a12958db19bf925eb64417136427275c567500 SHA256 (FreeBSD-12.1-RC2-amd64.vmdk.xz) = 4c58a879328b4336e55758eeccae7574f758ead20ab73dba8887ddc5f30d63b7 o 12.1-RC2 i386: SHA512 (FreeBSD-12.1-RC2-i386.qcow2.xz) = edab81247582d0875e13337726ce4a6afce4e4f0070b2d28709e9600098fe4877e27e8e55807afb20c7c8a3b9ec7096a663cd2da5118bc33eb69382ab2525126 SHA512 (FreeBSD-12.1-RC2-i386.raw.xz) = ecf771066418affe94ccf3449e7baf1d1502b63ee5645b3195633131ccbe41963fb2a5a13f5e5ffbb51fad3834ba23a31f83fbb0712239547630b6090f3bbfd2 SHA512 (FreeBSD-12.1-RC2-i386.vhd.xz) = eef45d89453081204cb672b30beda0a13bc13f6940cac9404fa6723a504055e85555677c72d38ffc74e4d2c634b74e4fd8065133f5274cb6817c5b716bb094a6 SHA512 (FreeBSD-12.1-RC2-i386.vmdk.xz) = ca7f79d21c854178ede6027a79eb956f4f96ecd74148418ceb6df66f4a683b4e5dc36367468e37b4dab7db73e3d480091a0ffb5fbb1b20d00318ab95fd06c3cc SHA256 (FreeBSD-12.1-RC2-i386.qcow2.xz) = 2421c616f3b66597791285bb9e33effc53c84fb98a6988c0f0224419f8464689 SHA256 (FreeBSD-12.1-RC2-i386.raw.xz) = 75dbf400863fa81ee510c3c8db3d6c3f846f5ebaeed7386428cbf9c5a9d8ae1d SHA256 (FreeBSD-12.1-RC2-i386.vhd.xz) = ba3a70ea8a209600f5f99a9449894162b31cbed89f2c0d1d1f4a5b9df5428ee6 SHA256 (FreeBSD-12.1-RC2-i386.vmdk.xz) = 1b10c87501ffa351d139e8a22d9c0bcf00e45280756708ed4db305103f692028 o 12.1-RC2 aarch64: SHA512 (FreeBSD-12.1-RC2-arm64-aarch64.qcow2.xz) = 76718f78d349ebda77c3655fc07fe70d87fa35df2ad0669b3588e51d532af47f019aa1e56aebc24fabb03aa610a1acae4b4d6143807fef3adbdace382c70ea05 SHA512 (FreeBSD-12.1-RC2-arm64-aarch64.raw.xz) = 059e83492ae9f5e5af4e71baee97a3220a4799f0553bff513ca7a9fa40cb080c056b105e216547cd5601f77bdf31653f9fde832f4787abafef0ce34c93971c50 SHA512 (FreeBSD-12.1-RC2-arm64-aarch64.vhd.xz) = ab847449b9cd6573a4c08a3b67e963448f92acf94290e1fda1aa181a18220953c83c0741739194026a34d8b86d2581e8cae71519acc4264bbbd4dd2dfcfdd870 SHA512 (FreeBSD-12.1-RC2-arm64-aarch64.vmdk.xz) = 485ec9984d4b4810e4361a873441e3aca1532538e58a7fde713dfbaefcc887f3637603b5621fa8562860ef18566ca35f0002686f3415c9c4dd96571cc1ce3fe8 SHA256 (FreeBSD-12.1-RC2-arm64-aarch64.qcow2.xz) = 556d895c660b4f600b5b244b69f65c95c142035feb9f9c41423df133b8e9af06 SHA256 (FreeBSD-12.1-RC2-arm64-aarch64.raw.xz) = bc60757d9f26369c95bf6a8c8c557debd0e29340ece1426d0ec76da44dd46b56 SHA256 (FreeBSD-12.1-RC2-arm64-aarch64.vhd.xz) = 6461f694e5c5244a5d3cc53c9c1cafd03663102c3b2c5314590e75754f76b70e SHA256 (FreeBSD-12.1-RC2-arm64-aarch64.vmdk.xz) = 475cffe20ee74bf99463e7fa7c8a18f263f37252fc435d1bf53dda0123c30a21 Regards, Glen Brought to you by Rubicon Communications, LLC (Netgate). Love FreeBSD? Support this and future releases with a donation to the FreeBSD Foundation! https://www.freebsdfoundation.org/donate/ -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEjRJAPC5sqwhs9k2jAxRYpUeP4pMFAl2qJmUACgkQAxRYpUeP 4pMI3w/+NfUy2iegZq7SBsEC55/7lSve0cts0HRakAXYyBZ6yf2nPinVTdKAVpUe KUlAa4ODHrKlpJKAIwy/EWZfouUvyqKU/zlNdzA4Txf7OJnqhR3fCGdRteMmcMx6 Qu4jVeHzkJSrS4jNVQNhoECR7z+o9BkgP4Dn15YJTKqV0JavcgdQdiEZf8YtK/S5 YS5TvKvYX92MPZdm32W9qKJONFFtOBR0jTXzntHJ+/gp+h9hK2KriDw9w7/9le0Z bWJu/p/oYoA+ZGLQT/mRS2kW9iZtaj2kZ+ymHdxLdMFB86KYqwog/HFjwsuf4yjZ hrtiO2sTKVxRb0f6CzKEXgjuQKMfGDyNJq+pAvpQCBhFOvTaBppX82VWFEwwOP4U cYxFZGSl1zVBJwOSepTBhILhQmHVhGdhodQDx2tkEYtW8q2PlyQplcWfEu00it6o SkVPKiKKxlkZdKf/HU+1dYuLjyElJpV1hSJE2kO/AL86QeIa+3IDkAr2uxRD3TrF qKhnQzc5/mdwedKQChP64Ca35wqwbEanAjmQhtRGQPXRlgNGuInKWrliW1hfjhu7 iEquvgCdgYLyzuZvXQLXmf1H8iT2H7GAHW9TJ3HCVuVxp8WEvqh5xI+IQH38ypEK 7FscNU1gbu2+YngkKqhz2ek7QnbKb0EMRLeV+fWPhJ3/rUQiz+E= =ccSp -----END PGP SIGNATURE----- From owner-freebsd-stable@freebsd.org Fri Oct 18 12:57:22 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B1915150200 for ; Fri, 18 Oct 2019 12:57:22 +0000 (UTC) (envelope-from devgs@ukr.net) Received: from frv197.fwdcdn.com (frv197.fwdcdn.com [212.42.77.197]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "*.ukr.net", Issuer "Thawte RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46vmJx520jz4XcM for ; Fri, 18 Oct 2019 12:57:21 +0000 (UTC) (envelope-from devgs@ukr.net) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=ffe; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Message-Id:Cc:To :Subject:From:Date:Sender:Reply-To:Content-ID:Content-Description:Resent-Date :Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Id:List-Help:List-Unsubscribe:List-Subscribe:List-Post: List-Owner:List-Archive; bh=Zx7RZ/0YdTpglQGsaRiTbZ9nDlzxAbIvKkU4h+z0ESU=; b=n twNMGFhGwpr2NsDWS+Y1EvpMEWVS/aZlemtCp7yJIMbKeqKhXkklMOQRBfGwziKKCXXz61M9toJmd tPDwROC2DSWPyWjZUTUugyK4ViSqIbISxgdbnzYZMIH1Z+qg/r13X9vrVuq0mtgSaSHOgx6MjDXto v0m+5VhFZHnRvfdg=; Received: from [10.10.10.39] (helo=frv39.fwdcdn.com) by frv197.fwdcdn.com with smtp ID 1iLRof-0007Oo-HB for freebsd-stable@freebsd.org; Fri, 18 Oct 2019 15:57:13 +0300 Date: Fri, 18 Oct 2019 15:57:13 +0300 From: Paul Subject: Network anomalies after update from 11.2 STABLE to 12.1 STABLE To: freebsd-net@freebsd.org, freebsd-stable@freebsd.org Received: from devgs@ukr.net by frv39.fwdcdn.com; Fri, 18 Oct 2019 15:57:13 +0300 Message-Id: <1571398510.796520000.8iwbi4pd@frv39.fwdcdn.com> X-Mailer: mail.ukr.net 5.0 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: binary X-Rspamd-Queue-Id: 46vmJx520jz4XcM X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ukr.net header.s=ffe header.b=n twNMGF; dmarc=pass (policy=none) header.from=ukr.net; spf=pass (mx1.freebsd.org: domain of devgs@ukr.net designates 212.42.77.197 as permitted sender) smtp.mailfrom=devgs@ukr.net X-Spamd-Result: default: False [-4.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[ukr.net:s=ffe]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:212.42.77.0/24]; FREEMAIL_FROM(0.00)[ukr.net]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; DWL_DNSWL_LOW(-1.00)[ukr.net.dwl.dnswl.org : 127.0.5.1]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[ukr.net:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[ukr.net,none]; IP_SCORE(0.00)[ipnet: 212.42.77.0/24(-4.77), asn: 8856(-3.85), country: UA(0.08)]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[ukr.net]; ASN(0.00)[asn:8856, ipnet:212.42.77.0/24, country:UA]; RCVD_TLS_LAST(0.00)[] X-Mailman-Approved-At: Sat, 19 Oct 2019 02:51:27 +0000 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Oct 2019 12:57:22 -0000 Our current version is: FreeBSD 11.2-STABLE #0 r340725 New version that we have problems with: FreeBSD 12.1-STABLE #5 r352893 After update to new version we have started to observe an incredible number of errors in HTTP requests in between various services in our system. This problem appeared on all the servers that were upgraded, and seems to not be specific to concrete network card: we use different models, all are affected. During various tests, we observed a lot of spontaneous TCP stream abortions, including at the establishment stage (SYN) in cases that were 100% issue free on 11.2-STABLE. Concrete test cases will be shown below. We also want to highlight that, on numerous occasions, we have observed random, huge ACK indices in a first response to a SYN packet, instead of 1, as expected. This forces client to abort connection via RST. On the fist glance it looks like races in the kernel, because problem disappears when: * we use `dev.ixl.0.iflib.override_nrxqs=1` and `dev.ixl.0.iflib.override_ntxqs=1` * we use `dev.ixl.0.iflib.override_nrxqs=0` and `dev.ixl.0.iflib.override_ntxqs=0`, but don't issue concurrent TCP streams These are some debug log messages, emitted by 12.1-STABLE: Oct 18 14:59:01 test kernel: TCP: [10.10.10.39]:16304 to [10.10.10.92]:80 tcpflags 0x4; syncache_chkrst: Spurious RST without matching syncache entry (possibly syncookie only), segment ignored Oct 18 14:59:01 test kernel: TCP: [10.10.10.39]:16326 to [10.10.10.92]:80 tcpflags 0x4; syncache_chkrst: Spurious RST without matching syncache entry (possibly syncookie only), segment ignored Oct 18 14:59:01 test kernel: TCP: [10.10.10.39]:16402 to [10.10.10.92]:80 tcpflags 0x4; syncache_chkrst: Spurious RST without matching syncache entry (possibly syncookie only), segment ignored Oct 18 14:59:01 test kernel: TCP: [10.10.10.39]:16652 to [10.10.10.92]:80 tcpflags 0x4; syncache_chkrst: Spurious RST without matching syncache entry (possibly syncookie only), segment ignored Oct 18 14:59:01 test kernel: TCP: [10.10.10.39]:16686 to [10.10.10.92]:80 tcpflags 0x4; syncache_chkrst: Spurious RST without matching syncache entry (possibly syncookie only), segment ignored Oct 18 14:59:01 test kernel: TCP: [10.10.10.39]:18562 to [10.10.10.92]:80 tcpflags 0x4; tcp_do_segment: Timestamp missing, no action Oct 18 14:59:01 test kernel: TCP: [10.10.10.39]:18918 to [10.10.10.92]:80 tcpflags 0x4; syncache_chkrst: Spurious RST without matching syncache entry (possibly syncookie only), segment ignored Oct 18 14:59:01 test kernel: TCP: [10.10.10.39]:19331 to [10.10.10.92]:80 tcpflags 0x4; syncache_chkrst: Spurious RST without matching syncache entry (possibly syncookie only), segment ignored Oct 18 14:59:01 test kernel: TCP: [10.10.10.39]:19340 to [10.10.10.92]:80 tcpflags 0x4; tcp_do_segment: Timestamp missing, no action Oct 18 14:59:01 test kernel: TCP: [10.10.10.39]:19340 to [10.10.10.92]:80 tcpflags 0x4; syncache_chkrst: Spurious RST without matching syncache entry (possibly syncookie only), segment ignored Oct 18 14:59:01 test kernel: TCP: [10.10.10.39]:19340 to [10.10.10.92]:80 tcpflags 0x4; syncache_chkrst: Spurious RST without matching syncache entry (possibly syncookie only), segment ignored Oct 18 14:59:01 test kernel: TCP: [10.10.10.39]:19489 to [10.10.10.92]:80 tcpflags 0x4; syncache_chkrst: Spurious RST without matching syncache entry (possibly syncookie only), segment ignored Oct 18 14:59:01 test kernel: TCP: [10.10.10.39]:19580 to [10.10.10.92]:80 tcpflags 0x4; tcp_do_segment: Timestamp missing, no action Oct 18 14:59:01 test kernel: TCP: [10.10.10.39]:19580 to [10.10.10.92]:80 tcpflags 0x4; syncache_chkrst: Spurious RST without matching syncache entry (possibly syncookie only), segment ignored Oct 18 14:59:01 test kernel: TCP: [10.10.10.39]:19580 to [10.10.10.92]:80 tcpflags 0x4; syncache_chkrst: Spurious RST without matching syncache entry (possibly syncookie only), segment ignored Oct 18 14:59:02 test kernel: TCP: [10.10.10.39]:17705 to [10.10.10.92]:80; syncache_timer: Response timeout, retransmitting (1) SYN|ACK Oct 18 14:59:02 test kernel: TCP: [10.10.10.39]:18066 to [10.10.10.92]:80; syncache_timer: Response timeout, retransmitting (1) SYN|ACK Oct 18 14:59:02 test kernel: TCP: [10.10.10.39]:18066 to [10.10.10.92]:80 tcpflags 0x4; syncache_chkrst: Our SYN|ACK was rejected, connection attempt aborted by remote endpoint Oct 18 14:59:02 test kernel: TCP: [10.10.10.39]:17705 to [10.10.10.92]:80 tcpflags 0x4; syncache_chkrst: Our SYN|ACK was rejected, connection attempt aborted by remote endpoint Here, 10.10.10.92 runs 12.1-STABLE, while 10.10.10.39 is a client that runs 11.2-STABLE. In our test case we use nginx and wrk , with a minimal config, where nginx always returns error page 404. nginx is on the 12.1-STABLE, while wrk is on 11.2-STABLE. We run wrk like so: wrk -c 10 --header "Connection: close" -d 10 -t 1 --latency http://10.10.10.92:80/missing and often see errors like these: Socket errors: connect 12, read 4, write 4, timeout 0 If we reverse the test, by switching two servers places, ie 12.1-STABLE becomes a client and issues requests via wrk, we see no problems at all. Same is true between two between two 11.2-STABLE machines. It seems like issue appears only when the same local port is used for multiple connections on 12.1-STABLE. Currently this is possible only when 12.1-STABLE is a server and accepts connections on port, say 80, as in our case. To confirm, this we made another test. We've configured nginx to listen on 10 different ports, 80 through 89, and then launched 10 different wrk processes, each using only one concurrent connection, meaning that we will have only 10 TCP streams, each having its own unique port on the 12.1-STABLE's side: for I in {0..9}; do wrk -c 1 --header "Connection: close" -d 10 -t 1 --latency http://10.10.10.92:8${I}/missing & ; done Socket errors stopped appearing. We ran this test many many times, errors just don't appear. Though, whenever we repeat a previous test, using a single port: wrk -c 10 --header "Connection: close" -d 10 -t 1 --latency http://10.10.10.92:80/missing errors start appearing again and again: Socket errors: connect 8, read 14, write 9, timeout 0 We've tested different drivers with the same outcome: em driver: em0@pci0:10:0:0: class=0x020000 card=0x000015d9 chip=0x10d38086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = '82574L Gigabit Network Connection' ixl driver: ixl0@pci0:4:0:0: class=0x020000 card=0x00078086 chip=0x15728086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'Ethernet Controller X710 for 10GbE SFP+' Even the driver from ports (/usr/ports/net/intel-ixl-kmod): ixl-1.11.9 Help with this matter would be really appreciated. Best regards, -Paul From owner-freebsd-stable@freebsd.org Sat Oct 19 02:39:26 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 348FB14911B for ; Sat, 19 Oct 2019 02:39:26 +0000 (UTC) (envelope-from dpostolov@yandex.ru) Received: from forward400o.mail.yandex.net (forward400o.mail.yandex.net [37.140.190.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46w6YS2Ypnz4Wtj for ; Sat, 19 Oct 2019 02:39:23 +0000 (UTC) (envelope-from dpostolov@yandex.ru) Received: from mxback9j.mail.yandex.net (mxback9j.mail.yandex.net [IPv6:2a02:6b8:0:1619::112]) by forward400o.mail.yandex.net (Yandex) with ESMTP id D7E0114C0C64 for ; Sat, 19 Oct 2019 05:39:20 +0300 (MSK) Received: from localhost (localhost [::1]) by mxback9j.mail.yandex.net (nwsmtp/Yandex) with ESMTP id IN95Egct2j-dKSSegY8; Sat, 19 Oct 2019 05:39:20 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1571452760; bh=HYGSwXpQpJu9X6lkSAGRpYnYElCrb1oGORC4i4OrS8Y=; h=Message-Id:Date:Subject:To:From; b=GCCXLVYOuzIOSYMRNZDQgTvZFWsHrzqu6PDnlOgl3xKC2ApZKzusUGrrxeViDFzvS dlgcN8RTnXRtYTsUZyqz52uAUt8CniNr1h77JSHDjRAdWNEx34+BvGXtlgIUVH8s5V 47naBfxF0gXzOrnTgAe4n//TxLWcSGiy0pr7rVvI= Received: by iva6-ab1cb434301d.qloud-c.yandex.net with HTTP; Sat, 19 Oct 2019 05:39:20 +0300 From: Dmitry Postolov To: freebsd-stable Subject: XScreenSaver must bu rerun as demon on new login in Xfce 4.14 MIME-Version: 1.0 X-Mailer: Yamail [ http://yandex.ru ] 5.0 Date: Sat, 19 Oct 2019 07:39:20 +0500 Message-Id: <9475141571452760@iva6-ab1cb434301d.qloud-c.yandex.net> Content-Transfer-Encoding: 7bit Content-Type: text/plain X-Rspamd-Queue-Id: 46w6YS2Ypnz4Wtj X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yandex.ru header.s=mail header.b=GCCXLVYO; dmarc=pass (policy=none) header.from=yandex.ru; spf=pass (mx1.freebsd.org: domain of dpostolov@yandex.ru designates 37.140.190.176 as permitted sender) smtp.mailfrom=dpostolov@yandex.ru X-Spamd-Result: default: False [-4.10 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[yandex.ru:s=mail]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:37.140.128.0/18]; FREEMAIL_FROM(0.00)[yandex.ru]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_LOW(-1.00)[yandex.ru.dwl.dnswl.org : 127.0.5.1]; RCVD_COUNT_THREE(0.00)[3]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yandex.ru:+]; DMARC_POLICY_ALLOW(-0.50)[yandex.ru,none]; TO_MATCH_ENVRCPT_ALL(0.00)[]; IP_SCORE(0.00)[ipnet: 37.140.128.0/18(-4.80), asn: 13238(-3.76), country: RU(0.01)]; RCVD_IN_DNSWL_LOW(-0.10)[176.190.140.37.list.dnswl.org : 127.0.5.1]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yandex.ru]; ASN(0.00)[asn:13238, ipnet:37.140.128.0/18, country:RU]; MIME_TRACE(0.00)[0:+] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Oct 2019 02:39:26 -0000 Hi to FreeBSD Community! Sorry for my very bad English... FreeBSD-12.1-RC2-amd64. Xfce 4.14 from the "latest" pkg repository. XScreenSaver 5.43 (09-Jul-2019). XScreenSaver must bu rerun as demon on new login, because in other case it dont run as demon (in first start it was run). The hardware: Intel NUC5PPYH & Intel NUC DN2820FYKH with "scfb" video driver. --- best regards, Dmitry Postolov dpostolov@yandex.ru From owner-freebsd-stable@freebsd.org Sat Oct 19 03:51:48 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 43D4B152813 for ; Sat, 19 Oct 2019 03:51:48 +0000 (UTC) (envelope-from jeremy.m.cox@gmail.com) Received: from mail-io1-xd29.google.com (mail-io1-xd29.google.com [IPv6:2607:f8b0:4864:20::d29]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46w88z18Qbz4dg2 for ; Sat, 19 Oct 2019 03:51:46 +0000 (UTC) (envelope-from jeremy.m.cox@gmail.com) Received: by mail-io1-xd29.google.com with SMTP id c25so9758784iot.12 for ; Fri, 18 Oct 2019 20:51:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=JgsLdAf7vBM2y0XA1+/833Qn/YS+AzcR9PKL+jkvX54=; b=hXTaIC16by0Y8XzUcixJRTajWewenecZ48jtTuqJBLexNYWQr2hlAmHLucyOGUrcl1 g8c0uCvJ4ON66hxcmblTuwyNb5kVkbXh1/vFWBNrbFCvswY8ThblV6f0iJbj2/IoOaP0 gE7Zfj7mRAax/Q4NfSrt2yaDg0vRW0wAMB70sxrCizUvbslYXpThEs9Mu+iA3YwBtT16 G/NP/Adctq18BHvEQ5k1Gr9SHXrCJslS+EVnK0w2+MB9fDNZxQ944LEYb/NZqXbRf/Bg UfsOuXEZBC3J29gW0DgSju6F4OnDfFlsoNolMcciE55KNuEH2vdwL4UmcDnD7eIdvb28 6vOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=JgsLdAf7vBM2y0XA1+/833Qn/YS+AzcR9PKL+jkvX54=; b=eat2nsgojRSOH7+a9fR62BG7QQJiIyW0WHWs6wEecbLa0h7fzhdW3ERheJ3xi+ymlY LF/95d/IEVlXcQbu9P7YD/Fjq4yiml4ZHFvrbP3XjiGp1abWFOwAOXkZB+uWdP+CAIgq T8phqf4zSvQT2yFJTy4P1P2RYdaPuWcp7FM3qQ3so41OtIYPtH/bYbZztyajY0kMTkhD MIOvNKtbMAPw4aqRv0SChJgw6qhJtxj8t22Db5TZxy/Elyngo6gcyDfDyr2n30BCdIQA BJ0h2A6Hgu7qG/10557xmrOzUUOVFvwXZvPcUQITlOJ8DAHQqyoIaNgS3zb+rH6Fh5po T2lQ== X-Gm-Message-State: APjAAAW4+DAk1EIKFJ8mxoqT/sxywabyYZ7cRaxvkgf96NGC6DgSKrCL 1yzZy2MqmZDeRKADDE9bBhMMeajtNTqMdjAYundXFQ== X-Google-Smtp-Source: APXvYqwW9Rww3aaLY1DFvTWxJBgmE4iFQeVbBYXBzsaqiNSPwKNRcEcHLBWPX5g0lHw5+ABMbZtgeIh5k3KkcD8i/RQ= X-Received: by 2002:a5d:8991:: with SMTP id m17mr11900928iol.106.1571457105401; Fri, 18 Oct 2019 20:51:45 -0700 (PDT) MIME-Version: 1.0 From: Jeremy Date: Fri, 18 Oct 2019 22:51:34 -0500 Message-ID: Subject: Is it necessary to bump target triple for lib32 To: freebsd-stable@freebsd.org X-Rspamd-Queue-Id: 46w88z18Qbz4dg2 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=hXTaIC16; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of jeremymcox@gmail.com designates 2607:f8b0:4864:20::d29 as permitted sender) smtp.mailfrom=jeremymcox@gmail.com X-Spamd-Result: default: False [-3.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; TO_DN_NONE(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; IP_SCORE(0.00)[ip: (-5.42), ipnet: 2607:f8b0::/32(-2.46), asn: 15169(-2.09), country: US(-0.05)]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[9.2.d.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] X-Mailman-Approved-At: Sat, 19 Oct 2019 05:15:20 +0000 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Oct 2019 03:51:48 -0000 While I was doing a buildworld for FreeBSD 12 stable (r353745) for amd64, I noticed the target triple was set at x86_64-unknown-freebsd12.0 while it was building the 32 bit libraries. I was wondering if it was necessary to bump it to 12.1 now that 12.1 is getting ready for release, or if it doesn't matter either way. I believe the relevant commit was from (r338268). In any case, buildworld completed with no issues. If it's not necessary, sorry for the noise. Regards, Jeremy Cox From owner-freebsd-stable@freebsd.org Sat Oct 19 03:25:34 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AC67414F7A7 for ; Sat, 19 Oct 2019 03:25:34 +0000 (UTC) (envelope-from dpostolov@yandex.ru) Received: from forward104j.mail.yandex.net (forward104j.mail.yandex.net [5.45.198.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46w7Zh3X5kz4bb7 for ; Sat, 19 Oct 2019 03:25:32 +0000 (UTC) (envelope-from dpostolov@yandex.ru) Received: from mxback17j.mail.yandex.net (mxback17j.mail.yandex.net [IPv6:2a02:6b8:0:1619::93]) by forward104j.mail.yandex.net (Yandex) with ESMTP id 6FA394A0CD2 for ; Sat, 19 Oct 2019 06:25:28 +0300 (MSK) Received: from unknown (unknown [::1]) by mxback17j.mail.yandex.net (nwsmtp/Yandex) with ESMTP id HX2R3Az3fP-PRWe2mR7; Sat, 19 Oct 2019 06:25:27 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1571455527; bh=HYGSwXpQpJu9X6lkSAGRpYnYElCrb1oGORC4i4OrS8Y=; h=Message-Id:Date:Subject:To:From; b=X6K5m+P+tNTGpdKH/+GLDDzNMZVRne4VnSLLKLWUrjGgCShiceW9N9QcMPgtdZQ8m Ugq4l5hP7DTPjzRQGNwReK/rNWJB8UkdbsHSsERpQsmag6SfYp5e8skVB+aheNXkkq nW0g1Hn1foKroGlJ+KAFfcomD13wC+Xil49ttCSA= Received: by iva6-22e79380f52c.qloud-c.yandex.net with HTTP; Sat, 19 Oct 2019 06:25:27 +0300 From: Dmitry Postolov To: freebsd-stable Subject: XScreenSaver must bu rerun as demon on new login in Xfce 4.14 MIME-Version: 1.0 X-Mailer: Yamail [ http://yandex.ru ] 5.0 Date: Sat, 19 Oct 2019 08:25:27 +0500 Message-Id: <15963121571455527@iva6-22e79380f52c.qloud-c.yandex.net> Content-Transfer-Encoding: 7bit Content-Type: text/plain X-Rspamd-Queue-Id: 46w7Zh3X5kz4bb7 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yandex.ru header.s=mail header.b=X6K5m+P+; dmarc=pass (policy=none) header.from=yandex.ru; spf=pass (mx1.freebsd.org: domain of dpostolov@yandex.ru designates 5.45.198.247 as permitted sender) smtp.mailfrom=dpostolov@yandex.ru X-Spamd-Result: default: False [-4.10 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[yandex.ru:s=mail]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:5.45.192.0/19]; FREEMAIL_FROM(0.00)[yandex.ru]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_LOW(-1.00)[yandex.ru.dwl.dnswl.org : 127.0.5.1]; RCVD_COUNT_THREE(0.00)[3]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yandex.ru:+]; DMARC_POLICY_ALLOW(-0.50)[yandex.ru,none]; TO_MATCH_ENVRCPT_ALL(0.00)[]; IP_SCORE(0.00)[ipnet: 5.45.192.0/18(-4.84), asn: 13238(-3.77), country: RU(0.01)]; RCVD_IN_DNSWL_LOW(-0.10)[247.198.45.5.list.dnswl.org : 127.0.5.1]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yandex.ru]; ASN(0.00)[asn:13238, ipnet:5.45.192.0/18, country:RU]; MIME_TRACE(0.00)[0:+] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Oct 2019 03:25:34 -0000 Hi to FreeBSD Community! Sorry for my very bad English... FreeBSD-12.1-RC2-amd64. Xfce 4.14 from the "latest" pkg repository. XScreenSaver 5.43 (09-Jul-2019). XScreenSaver must bu rerun as demon on new login, because in other case it dont run as demon (in first start it was run). The hardware: Intel NUC5PPYH & Intel NUC DN2820FYKH with "scfb" video driver. --- best regards, Dmitry Postolov dpostolov@yandex.ru From owner-freebsd-stable@freebsd.org Sat Oct 19 10:30:12 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B649915FF95 for ; Sat, 19 Oct 2019 10:30:12 +0000 (UTC) (envelope-from mad@madpilot.net) Received: from mail.madpilot.net (vogon.madpilot.net [159.69.1.99]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46wK0g5NZzz44PN for ; Sat, 19 Oct 2019 10:30:11 +0000 (UTC) (envelope-from mad@madpilot.net) Received: from mail (mail [192.168.254.3]) by mail.madpilot.net (Postfix) with ESMTP id 46wK0X2HZcz6dQk; Sat, 19 Oct 2019 12:30:04 +0200 (CEST) Received: from mail.madpilot.net ([192.168.254.3]) by mail (mail.madpilot.net [192.168.254.3]) (amavisd-new, port 10026) with ESMTP id QD1njZ7ZkIjH; Sat, 19 Oct 2019 12:30:02 +0200 (CEST) Subject: Re: XScreenSaver must bu rerun as demon on new login in Xfce 4.14 To: Dmitry Postolov , freebsd-stable References: <15963121571455527@iva6-22e79380f52c.qloud-c.yandex.net> From: Guido Falsi Autocrypt: addr=mad@madpilot.net; keydata= mQENBE+G+l0BCADi/WBQ0aRJfnE7LBPsM0G3m/m3Yx7OPu4iYFvS84xawmRHtCNjWIntsxuX fptkmEo3Rsw816WUrek8dxoUAYdHd+EcpBcnnDzfDH5LW/TZ4gbrFezrHPdRp7wdxi23GN80 qPwHEwXuF0X4Wy5V0OO8B6VT/nA0ADYnBDhXS52HGIJ/GCUjgqJn+phDTdCFLvrSFdmgx4Wl c0W5Z1p5cmDF9l8L/hc959AeyNf7I9dXnjekGM9gVv7UDUYzCifR3U8T0fnfdMmS8NeI9NC+ wuREpRO4lKOkTnj9TtQJRiptlhcHQiAlG1cFqs7EQo57Tqq6cxD1FycZJLuC32bGbgalABEB AAG0Hkd1aWRvIEZhbHNpIDxtYWRAbWFkcGlsb3QubmV0PokBOQQTAQgAIwIbAwIeAQIXgAUL CQgHAwUVCgkICwQWAgMBBQJS79AgAhkBAAoJEBrmhg5Wy9KTc0kH/RO64ORBlTbTHaUaOj8F Je5O5NU2Pt9Cyt5ZWBRvxntr1zPTJGKRPS9ihlIfqT4ZvEngQGp57EUyFbCpI0UWasTerImM tt5WACnGmCzUTB39UXx8Oy4b1EgWeTJQ747e/F1mQLXTNa6ijRBE9fYlTb4gAkPN88/wVV9v 3PZozKLTg16ghBzHM/P7Lk8L7clPEZChX1FTa/6eSt3nvzfCuTMZbBPJF/ph+q1KyPqRgVfh tyhu5dvgMoPz/ni41IfeSrkJTD5RXzdyGR9q4Z1NYeBsLkRjC4LxKAP5KqUsvlOUjKvO1byj ApYdMarol+IGkaSk9e3zVYAJkWKjn/ni8Xa5Ag0EUxB7QQEQAKFhrDceoPdK/IHDSmoj6SQY isvM7VdhcleS7E9DoEAVt7yMbf6HbbMVTTY6ckvwTWQssywLBXNVqxgc4WLJjzfUhgef+WE7 5M3+WFYlOVQLGZY/zEVgma1raYnOHNAOzeHLDmEXjbZP6vGAeDyBbGfQPpE7qGYZ7ubeT3Xw QO+PklcCrvOPj2ZPcAxGNS2xVU/LzONqCrJqLMJSIcCdsbiSP4G5PnDFHtMokaTY6OEr8OEQ fOAerhcHUa/z7Uu8YtmaqKH+QGkE/WEgaRqSiTnv0JOTD+DxehaqvoKPPZ++2NpCZMHB2i6A /xifmQwEiIjEXtcueBRzkNUQkxhqZyS13SrhocL9ydtaVPBzZatAEjUDDEJmAMLVFs45qfyh MiNapHJo2n3MW/E5omqCvEkDdWX/en3P7CK2TemeaDghMsgkNKax/z0wNo5UZCkOPOz0xpNi UilOVbkuezZZNg65741qee2lfXhQIaZ66yT7hphc/N/z3PIAtLeze4u1VR2EXAuZ2sWAdlKC NTlJMsaU/x70BV11Wd/ypnVzM68dfdQIIAj1iMFAD/lXGlEUmKXg5Ov2VQDlTntQoanCYrAg +8CttPzjrydgLZFq3hrtQmfc0se5yv1WHS69+BsUOG09RvvawUDZxUjW19kyeN9THaNRgow3 kSuArUp6zSmJABEBAAGJAR8EGAEIAAkFAlMQe0ECGwwACgkQGuaGDlbL0pMN5wgA4bCkX/qw EVC06ToeR6C2putmSWQMgpDaqrv65Hubo+QGmg2P4ewTYQQ4g6oYWS03qHxqVVWhKz7FjfrV +dH8qbCLfSgIcvdBha7ayGZVrsiuMLKGbw36fcmkZPpSDOfHcP0XH8Z+u9CWj0xUkTxAlZ/7 i6gYSUpG2JWNtdmE/X8VVEyXusCLwy0K0BI60A/4dRTIX3C4QKrJ3ZbUXegz70ynjHf+lQMZ 9IZKASoRMuS5FozPQh6abvmwZEPdf5I9riUElzvHrqJ8Bx0t3Pujdoth+yNHpnBxrtO8LkQd rQ58P0SwcaIX33T2U9pG8bhu5YVR88FQ8OQ0cEsPBpDncrkBDQRPhvpdAQgAsd6mrOq1GSZw lzRscNQa9W2WB/3Tj4ON4PL2e9B+hc9lT/ny2zB3agXu5wbsXTzwxgJpQT7hNHkCSckW98h3 HRjFfhZPNCgInuUGsjcNyVguQh+/47ckhph0s7U+6B4yNuIiqQZk4mo8WgCNj1YIihVmGWEs gDOwMaajbDYZ0r1/3GkKlYjOXeUuT/WgourrSR5oZJVNA/k4X2H7M3JUr1BSc32L7BJt8M7A ntul6k17J0L8GmkvLvTUtQTO+p+DYQMna2ngD3PbAvQRcbEGnkg9ABrdEF0Wp4Gx+gGGWsyF KlHvPdMtgWAy3JsS+rQapG6LoW3yUJpwpEpA86KdBwARAQABiQEfBCgBCAAJBQJTEH0NAh0B AAoJEBrmhg5Wy9KTMZcIAMSsidGF4KpjGcKzhkNK0sEpevcelQ6DzgT7kcXuq6LQ6YOrbof2 /KPgGie9/ToFZfJXH8zE5GefqkKvHZbYssWilFvkI90F9n138kG205NB/2zlaQb74/v9ZMXJ XcipnIx+T2tOMCBgHJU41IMJmB+NfRt5A6CDytJdhWxqppsEo5jjy/7tJM1Nn47G87tAV8qV NUtzbS6zdnbHB4W2BJwCObbVv8epL3hu/L5efV2j2tSbVTmyvK/ClYMBqdtUo3uPX75GF/Ku YDCOP1BTA5zzmzp4PMVd+gmHcMgCZKY6lvcEtdi5FLI0we2kcY8ffPvM2d6MNhFsGLaVI95J 0oqJAR8EGAECAAkFAk+G+l0CGwwACgkQGuaGDlbL0pM18Qf9HTNNhu8N0ISKtmR8lgPhJuu8 9rOEa8KKEatr4fQ7gL+hmYOEqZ/yHLcPQvGxbAlLR7F0SheKvAEk4B1aFwGULPo0SzuO0d/W tVMEbGa95JTm/6mfiymWMlWf8UifD1MDKzzPR7Om0ybeoPM8S/RQTboUU1WLpwd4mg9pVJlK 0xr55GOSHNf4m7S+P1kvl3xgmEj14zVMq9yJBNWFlsQK5ciifh7sFpfuxWdEVbtgIdxpzImK LXSLA0vOroKAvxFTGBrBq3vxV6eUmaKyd5HbbWejmafY1ua5dcnew9lxpWKLdqkC27Vt0Cku +LtTY3325V+BChncwNcJJS7IMmBz6w== Message-ID: Date: Sat, 19 Oct 2019 12:30:01 +0200 In-Reply-To: <15963121571455527@iva6-22e79380f52c.qloud-c.yandex.net> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46wK0g5NZzz44PN X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.84 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[madpilot.net:s=bjowvop61wgh]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(-1.84)[ip: (-8.87), ipnet: 159.69.0.0/16(1.48), asn: 24940(-1.82), country: DE(-0.01)]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[madpilot.net:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[madpilot.net,quarantine]; FREEMAIL_TO(0.00)[yandex.ru]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:24940, ipnet:159.69.0.0/16, country:DE]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Oct 2019 10:30:12 -0000 On 19/10/19 05:25, Dmitry Postolov wrote: > Hi to FreeBSD Community! Sorry for my very bad English... > > FreeBSD-12.1-RC2-amd64. Xfce 4.14 from the "latest" pkg repository. > XScreenSaver 5.43 (09-Jul-2019). > > XScreenSaver must bu rerun as demon on new login, because > in other case it dont run as demon (in first start it was run). > Hi, The new XFCE 4.14 packages install xfce4-screensaver enabling it by default (unless you compile your packages disabling it explicitly, in which case it is not installed). Due to this the xscreensaver startup .desktop file is installed disabled by default to avoid launching two screensavers at the same time which could cause conflicts (this has been reported by one user). Please check the UPDATING entry 20190927 which has steps to address this. In short you should enable xscreensaver in XFCE Session and Startup settings application. -- Guido Falsi From owner-freebsd-stable@freebsd.org Sat Oct 19 16:09:29 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EEB211501A7 for ; Sat, 19 Oct 2019 16:09:29 +0000 (UTC) (envelope-from devgs@ukr.net) Received: from frv199.fwdcdn.com (frv199.fwdcdn.com [212.42.77.199]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "*.ukr.net", Issuer "Thawte RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46wSX86Sb9z4Ngw for ; Sat, 19 Oct 2019 16:09:28 +0000 (UTC) (envelope-from devgs@ukr.net) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=ffe; h=Content-Type:MIME-Version:Message-Id:Cc:To:Subject:From:Date:Sender: Reply-To:Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date :Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Id:List-Help:List-Unsubscribe:List-Subscribe:List-Post: List-Owner:List-Archive; bh=JON/9xKyVPBm8Y00LpL8wHxWH75ZHFo/KDtYa5UG/tA=; b=n o7lW8O9TbpnY+mfdDVTb+TON8y+Mm66DAUMtO4/G3oddnkMiRwalIZ98J1Fd5rsjCYrnO8/RUqKXy h1ke2FrjgtaCLuTgxula0JnZASslMDFKoY2+49JDuyjNi3lObFnHbK2KHg8mvsFggpIY2Yd2PFMph 6u/B6HxnBtWWV7Is=; Received: from [10.10.10.39] (helo=frv39.fwdcdn.com) by frv199.fwdcdn.com with smtp ID 1iLrI8-000KZ4-E6 for freebsd-stable@freebsd.org; Sat, 19 Oct 2019 19:09:20 +0300 Date: Sat, 19 Oct 2019 19:09:20 +0300 From: Paul Subject: Re[2]: Network anomalies after update from 11.2 STABLE to 12.1 STABLE To: michael.tuexen@lurchi.franken.de, freebsd-net@freebsd.org, freebsd-stable@freebsd.org Received: from devgs@ukr.net by frv39.fwdcdn.com; Sat, 19 Oct 2019 19:09:20 +0300 Message-Id: <1571499556.409350000.a1ewtyar@frv39.fwdcdn.com> X-Mailer: mail.ukr.net 5.0 MIME-Version: 1.0 X-Rspamd-Queue-Id: 46wSX86Sb9z4Ngw X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ukr.net header.s=ffe header.b=n o7lW8O; dmarc=pass (policy=none) header.from=ukr.net; spf=pass (mx1.freebsd.org: domain of devgs@ukr.net designates 212.42.77.199 as permitted sender) smtp.mailfrom=devgs@ukr.net X-Spamd-Result: default: False [-3.90 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[ukr.net:s=ffe]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:212.42.77.0/24]; FREEMAIL_FROM(0.00)[ukr.net]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; DWL_DNSWL_LOW(-1.00)[ukr.net.dwl.dnswl.org : 127.0.5.1]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[ukr.net:+]; MIME_BASE64_TEXT(0.10)[]; DMARC_POLICY_ALLOW(-0.50)[ukr.net,none]; IP_SCORE(0.00)[ipnet: 212.42.77.0/24(-4.81), asn: 8856(-3.87), country: UA(0.08)]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[ukr.net]; ASN(0.00)[asn:8856, ipnet:212.42.77.0/24, country:UA]; RCVD_TLS_LAST(0.00)[] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: binary X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Oct 2019 16:09:30 -0000 Hi Michael, Thank you, for taking your time! We use physical machines. We don not have any special `pf` rules. Both sides ran `pfctl -d` before testing. `nginx` config is primitive, no secrets there: ------------------------------------------------------------------- user www; worker_processes auto; error_log /var/log/nginx/error.log warn; events { worker_connections 81920; kqueue_changes 4096; use kqueue; } http { include mime.types; default_type application/octet-stream; sendfile off; keepalive_timeout 65; tcp_nopush on; tcp_nodelay on; # Logging log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $request_length $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_real_ip" "$realip_remote_addr" "$request_completion" "$request_time" ' '"$request_body"'; access_log /var/log/nginx/access.log main; server { listen 80 default; server_name localhost _; location / { return 404; } } } ------------------------------------------------------------------- `wrk` is compiled with a default configuration. We test like this: `wrk -c 10 --header "Connection: close" -d 10 -t 1 --latency http://10.10.10.92:80/missing` Also, it seems that our issue, and the one described in this thread, are identical: https://lists.freebsd.org/pipermail/freebsd-net/2019-June/053667.html We both have the Intel network cards, BTW. Our network cards are these: em0 at pci0:10:0:0: class=0x020000 card=0x000015d9 chip=0x10d38086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = '82574L Gigabit Network Connection' ixl0 at pci0:4:0:0: class=0x020000 card=0x00078086 chip=0x15728086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = 'Ethernet Controller X710 for 10GbE SFP+' ============================== Additional info: During the tests, we have bonded two interfaces into a lagg: ixl0: flags=8843 metric 0 mtu 1500 options=c500b8 ether 3c:fd:fe:aa:60:20 media: Ethernet autoselect (10Gbase-SR ) status: active nd6 options=29 ixl1: flags=8843 metric 0 mtu 1500 options=c500b8 ether 3c:fd:fe:aa:60:20 hwaddr 3c:fd:fe:aa:60:21 media: Ethernet autoselect (10Gbase-SR ) status: active nd6 options=29 lagg0: flags=8843 metric 0 mtu 1500 options=c500b8 ether 3c:fd:fe:aa:60:20 inet 10.10.10.92 netmask 0xffff0000 broadcast 10.10.255.255 laggproto failover lagghash l2,l3,l4 laggport: ixl0 flags=5 laggport: ixl1 flags=0<> groups: lagg media: Ethernet autoselect status: active nd6 options=29 using this config: ifconfig_ixl0="up -lro -tso -rxcsum -txcsum" (tried different options - got the same outcome) ifconfig_ixl1="up -lro -tso -rxcsum -txcsum" ifconfig_lagg0="laggproto failover laggport ixl0 laggport ixl1 10.10.10.92/24" We have randomly picked `ixl0` and restricted number of RX/TX queues to 1: /boot/loader.conf : dev.ixl.0.iflib.override_ntxqs=1 dev.ixl.0.iflib.override_nrxqs=1 leaving `ixl1` with a default number, matching number of cores (6). ixl0: mem 0xf8800000-0xf8ffffff,0xf9808000-0xf980ffff irq 40 at device 0.0 on pci4 ixl0: fw 5.0.40043 api 1.5 nvm 5.05 etid 80002927 oem 1.261.0 ixl0: PF-ID[0]: VFs 64, MSI-X 129, VF MSI-X 5, QPs 768, I2C ixl0: Using 1024 TX descriptors and 1024 RX descriptors ixl0: Using 1 RX queues 1 TX queues ixl0: Using MSI-X interrupts with 2 vectors ixl0: Ethernet address: 3c:fd:fe:aa:60:20 ixl0: Allocating 1 queues for PF LAN VSI; 1 queues active ixl0: PCI Express Bus: Speed 8.0GT/s Width x4 ixl0: SR-IOV ready ixl0: netmap queues/slots: TX 1/1024, RX 1/1024 ixl1: mem 0xf8000000-0xf87fffff,0xf9800000-0xf9807fff irq 40 at device 0.1 on pci4 ixl1: fw 5.0.40043 api 1.5 nvm 5.05 etid 80002927 oem 1.261.0 ixl1: PF-ID[1]: VFs 64, MSI-X 129, VF MSI-X 5, QPs 768, I2C ixl1: Using 1024 TX descriptors and 1024 RX descriptors ixl1: Using 6 RX queues 6 TX queues ixl1: Using MSI-X interrupts with 7 vectors ixl1: Ethernet address: 3c:fd:fe:aa:60:21 ixl1: Allocating 8 queues for PF LAN VSI; 6 queues active ixl1: PCI Express Bus: Speed 8.0GT/s Width x4 ixl1: SR-IOV ready ixl1: netmap queues/slots: TX 6/1024, RX 6/1024 This allowed us easy switch between different configurations without the need to reboot, by simply shutting down one interface or the other: `ifconfig XXX down` When testing `ixl0` that runs only a single queue: ixl0: Using 1 RX queues 1 TX queues ixl0: netmap queues/slots: TX 1/1024, RX 1/1024 we've got these results: `wrk -c 10 --header "Connection: close" -d 10 -t 1 --latency http://10.10.10.92:80/missing` Running 10s test @ http://10.10.10.92:80/missing 1 threads and 10 connections Thread Stats Avg Stdev Max +/- Stdev Latency 281.31us 297.74us 22.66ms 99.70% Req/Sec 19.91k 2.79k 21.25k 97.59% Latency Distribution 50% 266.00us 75% 309.00us 90% 374.00us 99% 490.00us 164440 requests in 10.02s, 47.52MB read Socket errors: read 0, write 0, timeout 0 Non-2xx or 3xx responses: 164440 Requests/sec: 16412.09 Transfer/sec: 4.74MB When testing `ixl1` that runs 6 queues: ixl1: Using 6 RX queues 6 TX queues ixl1: netmap queues/slots: TX 6/1024, RX 6/1024 we've got these results: `wrk -c 10 --header "Connection: close" -d 10 -t 1 --latency http://10.10.10.92:80/missing` Running 10s test @ http://10.10.10.92:80/missing 1 threads and 10 connections Thread Stats Avg Stdev Max +/- Stdev Latency 216.16us 71.97us 511.00us 47.56% Req/Sec 4.34k 2.76k 15.44k 83.17% Latency Distribution 50% 216.00us 75% 276.00us 90% 312.00us 99% 365.00us 43616 requests in 10.10s, 12.60MB read Socket errors: connect 0, read 24, write 8, timeout 0 Non-2xx or 3xx responses: 43616 Requests/sec: 4318.26 Transfer/sec: 1.25MB Do note, that, not only multiple queues cause issues they also dramatically decrease the performance of the network. Using `sysctl -w net.inet.tcp.ts_offset_per_conn=0` didn't help at all. Best regards, -Paul From owner-freebsd-stable@freebsd.org Sat Oct 19 16:24:42 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C5848150AAB; Sat, 19 Oct 2019 16:24:42 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-TO1-obe.outbound.protection.outlook.com (mail-eopbgr670089.outbound.protection.outlook.com [40.107.67.89]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46wSsj1tgRz4PQn; Sat, 19 Oct 2019 16:24:40 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YlbsuPP6MAK89kPftkQR7+nbjOnHbGvl2T1gMrv5ZfULWVhY1cpmfiqbDMGEeWABljUAygZHA3w8rO6wLd1ZVWGXnCRZ0PM10tGuuaC+L9TKvcmNBX0mMvOwGBsKNuyNPxTUUtQt5YSaVaiCCZgMkoWN/X3j2JudSNrBCkpZ9rRbRLgVZz0uYtWgwr9Uqrtxa7u2/NHFn6UtESg/7DXZpNy+64vrYTs8n2LievhlNGEwrhbxJyj8IiV7bkGRzxr/Ik8x/Q2OHtM/jkzWcw9FY7eT93FxHUn7U6zVQPSJCpG0Xe7qjDS/x/qCwUlKZxBBLfzx5eujDk5lq3E6mE6Jpg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nmDu2kTrZ/KZjgSAP/EI6XzLFuwxFtA05QPIF91BrgQ=; b=X6CyavNKzJRTKxDONZevhKe+MSIu5p9xx8HY5Tp0kl5mBAxn/xOxqo7zgocdCRGc09pkzc4G59fLVSug9eXrkJRcGzwkQdS/iM1oCin1BrHv8xBMnw8Qa7NTS6Ot7XxvwwM4iJtWpjSRbQPIFFfXU+py0KMo+eacLnaaG3XegHDJXpj6ogDYINysIGDC5n/vbjcdois4Ja3PIGLTYKNb5m96SPMI3A522ND3Lo5DrVUJAGwgL63ScwEE0LNHLN6tDNzSjz1a5NA52cnNH0uCqOk/80ZU1GgRYOyrNvgdsx6/OKQX6v9dNPzu74SvyQQipBuwJrshvQq3PjBFkooPgg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none Received: from YQBPR0101MB1652.CANPRD01.PROD.OUTLOOK.COM (52.132.66.144) by YQBPR0101MB0754.CANPRD01.PROD.OUTLOOK.COM (52.132.73.21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2347.16; Sat, 19 Oct 2019 16:24:38 +0000 Received: from YQBPR0101MB1652.CANPRD01.PROD.OUTLOOK.COM ([fe80::9051:db7:40f3:2ba3]) by YQBPR0101MB1652.CANPRD01.PROD.OUTLOOK.COM ([fe80::9051:db7:40f3:2ba3%7]) with mapi id 15.20.2347.028; Sat, 19 Oct 2019 16:24:38 +0000 From: Rick Macklem To: Paul , "michael.tuexen@lurchi.franken.de" , "freebsd-net@freebsd.org" , "freebsd-stable@freebsd.org" Subject: Re: Re[2]: Network anomalies after update from 11.2 STABLE to 12.1 STABLE Thread-Topic: Re[2]: Network anomalies after update from 11.2 STABLE to 12.1 STABLE Thread-Index: AQHVhpeb/qQpao8VP0Wq+sk7OrrSy6diJbjH Date: Sat, 19 Oct 2019 16:24:38 +0000 Message-ID: References: <1571499556.409350000.a1ewtyar@frv39.fwdcdn.com> In-Reply-To: <1571499556.409350000.a1ewtyar@frv39.fwdcdn.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 27b834bb-216c-42f1-9ac6-08d754b0d6b1 x-ms-traffictypediagnostic: YQBPR0101MB0754: x-ms-exchange-purlcount: 3 x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:3631; x-forefront-prvs: 01952C6E96 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6019001)(346002)(136003)(39850400004)(396003)(366004)(376002)(51234002)(189003)(199004)(256004)(110136005)(76116006)(99286004)(478600001)(86362001)(14454004)(966005)(14444005)(305945005)(7696005)(8936002)(476003)(2501003)(2420400007)(76176011)(11346002)(53546011)(15650500001)(446003)(102836004)(6506007)(52536014)(186003)(81166006)(81156014)(6306002)(8676002)(46003)(6246003)(55016002)(6436002)(25786009)(2906002)(66946007)(316002)(66476007)(2201001)(71190400001)(64756008)(66556008)(786003)(66446008)(71200400001)(74316002)(486006)(33656002)(7110500001)(9686003)(5660300002)(229853002)(21314003); DIR:OUT; SFP:1101; SCL:1; SRVR:YQBPR0101MB0754; H:YQBPR0101MB1652.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: uoguelph.ca does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: mXvfZLPTclq2Mao7G2rP9zD/VpNiUDG8m4ZhFbkJhZRemA3c8pzz+gyu0dgloYvv6/gQThg0Q0Ig3Zglbj5cxuPb2AYTDJGCf8iojSpZx7L3Ztn/mcQA+s0NIyQAzCP9eI+56Ja+JRmntzQ+qZuCNOgVXbj95RzxxXoeMkETjYR+TkLKOB1vDNd80gNegWK3mkNqhX6WF24eM1flMCUYsxYmMheXIZEKDmijA+EbJn+3Tfv+OPVjNxyfPXpDsN1lo61j6QSLhW/Ex+ltKWyn6rQhhSP8vln1A4j2LS8LcJVLNzqr/iQlpJK6z+Gk6Yp0uj4tu/jo+OSycw2c+eelU0mYCEW7zHQLzcdmvUk6MllBra9gbB40/C0IVU3QPLqcJ1I5Ekx+BdOq9QJIi+IJO7+eVSHWxdLUnEhTg8fHf9FOjzHp3ds4U8nM+5N5GK0iNmDvO59E9YXQDYY2+QOFjA== x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-Network-Message-Id: 27b834bb-216c-42f1-9ac6-08d754b0d6b1 X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Oct 2019 16:24:38.8266 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Re0lB8T9uWkb9SmCHtV7fAmHj4uA451AxB2XXrMWD/JyGY/uAulYZPmfh8+KBy4YAY2kmaBwtr4vGi04Rs2qBQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQBPR0101MB0754 X-Rspamd-Queue-Id: 46wSsj1tgRz4PQn X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 40.107.67.89 as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [-4.63 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip4:40.107.0.0/16]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[uoguelph.ca]; TO_DN_SOME(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[89.67.107.40.list.dnswl.org : 127.0.3.0]; IP_SCORE(-1.33)[ipnet: 40.64.0.0/10(-3.92), asn: 8075(-2.68), country: US(-0.05)]; FREEMAIL_TO(0.00)[ukr.net]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:8075, ipnet:40.64.0.0/10, country:US]; ARC_ALLOW(-1.00)[i=1] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Oct 2019 16:24:42 -0000 Btw, I once ran into a situation where "smart networking" was injecting RSTs into a TCP stream. The packet captures at the client and server machines were identical, except for the RSTs and the problem went away when I connected the two machines with a cable, bypassing the network. Might be worth a try, if you can do it? Good luck with it, rick ________________________________________ From: owner-freebsd-net@freebsd.org on beha= lf of Paul Sent: Saturday, October 19, 2019 12:09 PM To: michael.tuexen@lurchi.franken.de; freebsd-net@freebsd.org; freebsd-stab= le@freebsd.org Subject: Re[2]: Network anomalies after update from 11.2 STABLE to 12.1 STA= BLE Hi Michael, Thank you, for taking your time! We use physical machines. We don not have any special `pf` rules. Both sides ran `pfctl -d` before testing. `nginx` config is primitive, no secrets there: ------------------------------------------------------------------- user www; worker_processes auto; error_log /var/log/nginx/error.log warn; events { worker_connections 81920; kqueue_changes 4096; use kqueue; } http { include mime.types; default_type application/octet-stream; sendfile off; keepalive_timeout 65; tcp_nopush on; tcp_nodelay on; # Logging log_format main '$remote_addr - $remote_user [$time_local] = "$request" ' '$status $request_length $body_bytes_sent "= $http_referer" ' '"$http_user_agent" "$http_x_real_ip" "$rea= lip_remote_addr" "$request_completion" "$request_time" ' '"$request_body"'; access_log /var/log/nginx/access.log main; server { listen 80 default; server_name localhost _; location / { return 404; } } } ------------------------------------------------------------------- `wrk` is compiled with a default configuration. We test like this: `wrk -c 10 --header "Connection: close" -d 10 -t 1 --latency http://10.10.1= 0.92:80/missing` Also, it seems that our issue, and the one described in this thread, are id= entical: https://lists.freebsd.org/pipermail/freebsd-net/2019-June/053667.html We both have the Intel network cards, BTW. Our network cards are these: em0 at pci0:10:0:0: class=3D0x020000 card=3D0x000015d9 chip=3D0x10d3= 8086 rev=3D0x00 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '82574L Gigabit Network Connection' ixl0 at pci0:4:0:0: class=3D0x020000 card=3D0x00078086 chip=3D0x1572= 8086 rev=3D0x01 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D 'Ethernet Controller X710 for 10GbE SFP+' =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D Additional info: During the tests, we have bonded two interfaces into a lagg: ixl0: flags=3D8843 metric 0 mtu 150= 0 options=3Dc500b8 ether 3c:fd:fe:aa:60:20 media: Ethernet autoselect (10Gbase-SR ) status: active nd6 options=3D29 ixl1: flags=3D8843 metric 0 mtu 150= 0 options=3Dc500b8 ether 3c:fd:fe:aa:60:20 hwaddr 3c:fd:fe:aa:60:21 media: Ethernet autoselect (10Gbase-SR ) status: active nd6 options=3D29 lagg0: flags=3D8843 metric 0 mtu 15= 00 options=3Dc500b8 ether 3c:fd:fe:aa:60:20 inet 10.10.10.92 netmask 0xffff0000 broadcast 10.10.255.255 laggproto failover lagghash l2,l3,l4 laggport: ixl0 flags=3D5 laggport: ixl1 flags=3D0<> groups: lagg media: Ethernet autoselect status: active nd6 options=3D29 using this config: ifconfig_ixl0=3D"up -lro -tso -rxcsum -txcsum" (tried different option= s - got the same outcome) ifconfig_ixl1=3D"up -lro -tso -rxcsum -txcsum" ifconfig_lagg0=3D"laggproto failover laggport ixl0 laggport ixl1 10.10.= 10.92/24" We have randomly picked `ixl0` and restricted number of RX/TX queues to 1: /boot/loader.conf : dev.ixl.0.iflib.override_ntxqs=3D1 dev.ixl.0.iflib.override_nrxqs=3D1 leaving `ixl1` with a default number, matching number of cores (6). ixl0: mem = 0xf8800000-0xf8ffffff,0xf9808000-0xf980ffff irq 40 at device 0.0 on pci4 ixl0: fw 5.0.40043 api 1.5 nvm 5.05 etid 80002927 oem 1.261.0 ixl0: PF-ID[0]: VFs 64, MSI-X 129, VF MSI-X 5, QPs 768, I2C ixl0: Using 1024 TX descriptors and 1024 RX descriptors ixl0: Using 1 RX queues 1 TX queues ixl0: Using MSI-X interrupts with 2 vectors ixl0: Ethernet address: 3c:fd:fe:aa:60:20 ixl0: Allocating 1 queues for PF LAN VSI; 1 queues active ixl0: PCI Express Bus: Speed 8.0GT/s Width x4 ixl0: SR-IOV ready ixl0: netmap queues/slots: TX 1/1024, RX 1/1024 ixl1: mem = 0xf8000000-0xf87fffff,0xf9800000-0xf9807fff irq 40 at device 0.1 on pci4 ixl1: fw 5.0.40043 api 1.5 nvm 5.05 etid 80002927 oem 1.261.0 ixl1: PF-ID[1]: VFs 64, MSI-X 129, VF MSI-X 5, QPs 768, I2C ixl1: Using 1024 TX descriptors and 1024 RX descriptors ixl1: Using 6 RX queues 6 TX queues ixl1: Using MSI-X interrupts with 7 vectors ixl1: Ethernet address: 3c:fd:fe:aa:60:21 ixl1: Allocating 8 queues for PF LAN VSI; 6 queues active ixl1: PCI Express Bus: Speed 8.0GT/s Width x4 ixl1: SR-IOV ready ixl1: netmap queues/slots: TX 6/1024, RX 6/1024 This allowed us easy switch between different configurations without the need to reboot, by simply shutting down one interface or the other: `ifconfig XXX down` When testing `ixl0` that runs only a single queue: ixl0: Using 1 RX queues 1 TX queues ixl0: netmap queues/slots: TX 1/1024, RX 1/1024 we've got these results: `wrk -c 10 --header "Connection: close" -d 10 -t 1 --latency http://10.10.1= 0.92:80/missing` Running 10s test @ http://10.10.10.92:80/missing 1 threads and 10 connections Thread Stats Avg Stdev Max +/- Stdev Latency 281.31us 297.74us 22.66ms 99.70% Req/Sec 19.91k 2.79k 21.25k 97.59% Latency Distribution 50% 266.00us 75% 309.00us 90% 374.00us 99% 490.00us 164440 requests in 10.02s, 47.52MB read Socket errors: read 0, write 0, timeout 0 Non-2xx or 3xx responses: 164440 Requests/sec: 16412.09 Transfer/sec: 4.74MB When testing `ixl1` that runs 6 queues: ixl1: Using 6 RX queues 6 TX queues ixl1: netmap queues/slots: TX 6/1024, RX 6/1024 we've got these results: `wrk -c 10 --header "Connection: close" -d 10 -t 1 --latency http://10.10.1= 0.92:80/missing` Running 10s test @ http://10.10.10.92:80/missing 1 threads and 10 connections Thread Stats Avg Stdev Max +/- Stdev Latency 216.16us 71.97us 511.00us 47.56% Req/Sec 4.34k 2.76k 15.44k 83.17% Latency Distribution 50% 216.00us 75% 276.00us 90% 312.00us 99% 365.00us 43616 requests in 10.10s, 12.60MB read Socket errors: connect 0, read 24, write 8, timeout 0 Non-2xx or 3xx responses: 43616 Requests/sec: 4318.26 Transfer/sec: 1.25MB Do note, that, not only multiple queues cause issues they also dramatically decrease the performance of the network. Using `sysctl -w net.inet.tcp.ts_offset_per_conn=3D0` didn't help at all. Best regards, -Paul _______________________________________________ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Sat Oct 19 17:23:30 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E51CB1521F1 for ; Sat, 19 Oct 2019 17:23:30 +0000 (UTC) (envelope-from devgs@ukr.net) Received: from frv190.fwdcdn.com (frv190.fwdcdn.com [212.42.77.190]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "*.ukr.net", Issuer "Thawte RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46wV9Z0GFvz4Rl1 for ; Sat, 19 Oct 2019 17:23:29 +0000 (UTC) (envelope-from devgs@ukr.net) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=ffe; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Message-Id: References:In-Reply-To:Cc:To:Subject:From:Date:Sender:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=DvRKeAf5fANH4zVjuMX05802peHFxaHZejyCrlcodjw=; b=KOtlZIxw5/0mqWqqShIEI0ETDL 1wYdJky4xV2iUgmhnjzsxvO98+vDj13OG28QGZlfKBOT84NosEQ0I5JS2jRBZYIwHIWOLvIQbpCuS w3MKJyI3t2J9/O2YgTDnHAmb1MrIv+04eoGS7i/hvChpqWZvD9zJzaC+owPCbZNCXnFY=; Received: from [10.10.10.39] (helo=frv39.fwdcdn.com) by frv190.fwdcdn.com with smtp ID 1iLsRp-000EQr-RX for freebsd-stable@freebsd.org; Sat, 19 Oct 2019 20:23:25 +0300 Date: Sat, 19 Oct 2019 20:23:25 +0300 From: Paul Subject: Re[2]: Network anomalies after update from 11.2 STABLE to 12.1 STABLE To: Michael Tuexen Cc: freebsd-net@freebsd.org, freebsd-stable@freebsd.org Received: from devgs@ukr.net by frv39.fwdcdn.com; Sat, 19 Oct 2019 20:23:25 +0300 In-Reply-To: References: <1571499556.409350000.a1ewtyar@frv39.fwdcdn.com> X-Reply-Action: reply Message-Id: <1571505335.800858000.sqrselsr@frv39.fwdcdn.com> X-Mailer: mail.ukr.net 5.0 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: binary X-Rspamd-Queue-Id: 46wV9Z0GFvz4Rl1 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ukr.net header.s=ffe header.b=KOtlZIxw; dmarc=pass (policy=none) header.from=ukr.net; spf=pass (mx1.freebsd.org: domain of devgs@ukr.net designates 212.42.77.190 as permitted sender) smtp.mailfrom=devgs@ukr.net X-Spamd-Result: default: False [-4.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[ukr.net:s=ffe]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:212.42.77.0/24:c]; FREEMAIL_FROM(0.00)[ukr.net]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(0.00)[ipnet: 212.42.77.0/24(-4.82), asn: 8856(-3.87), country: UA(0.08)]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_DN_SOME(0.00)[]; DWL_DNSWL_LOW(-1.00)[ukr.net.dwl.dnswl.org : 127.0.5.1]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[ukr.net:+]; DMARC_POLICY_ALLOW(-0.50)[ukr.net,none]; IP_SCORE_FREEMAIL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[ukr.net]; ASN(0.00)[asn:8856, ipnet:212.42.77.0/24, country:UA]; RCVD_TLS_LAST(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Oct 2019 17:23:31 -0000 19 October 2019, 19:35:24, by "Michael Tuexen" : > > On 19. Oct 2019, at 18:09, Paul wrote: > > > > Hi Michael, > > > > Thank you, for taking your time! > > > > We use physical machines. We don not have any special `pf` rules. > > Both sides ran `pfctl -d` before testing. > Hi Paul, > > OK. How are the physical machines connected to each other? We have tested different connections. The old, copper ethernet, cable, as well as optics connection with an identical outcome. Machines are connected through Juniper QFX5100. > > What happens when you don't use a lagg interface, but the physical ones? > > (Trying to localise the problem...) Same thing, lagg does not change anything. Originally, the problem was observed on a regular interface. We have tested a on different hardware. Results are consistently stable on 11.2-STABLE and consistently unstable on 12.1-STABLE. The only unchanged thing is the network card vendor, it's Intel. > > Best regards > Michael > > > > > > `nginx` config is primitive, no secrets there: > > > > ------------------------------------------------------------------- > > user www; > > worker_processes auto; > > > > error_log /var/log/nginx/error.log warn; > > > > events { > > worker_connections 81920; > > kqueue_changes 4096; > > use kqueue; > > } > > > > http { > > include mime.types; > > default_type application/octet-stream; > > > > sendfile off; > > keepalive_timeout 65; > > tcp_nopush on; > > tcp_nodelay on; > > > > # Logging > > log_format main '$remote_addr - $remote_user [$time_local] "$request" ' > > '$status $request_length $body_bytes_sent "$http_referer" ' > > '"$http_user_agent" "$http_x_real_ip" "$realip_remote_addr" "$request_completion" "$request_time" ' > > '"$request_body"'; > > > > access_log /var/log/nginx/access.log main; > > > > server { > > listen 80 default; > > > > server_name localhost _; > > > > location / { > > return 404; > > } > > } > > } > > ------------------------------------------------------------------- > > > > > > `wrk` is compiled with a default configuration. We test like this: > > > > `wrk -c 10 --header "Connection: close" -d 10 -t 1 --latency http://10.10.10.92:80/missing` > > > > > > Also, it seems that our issue, and the one described in this thread, are identical: > > > > https://lists.freebsd.org/pipermail/freebsd-net/2019-June/053667.html > > > > We both have the Intel network cards, BTW. Our network cards are these: > > > > em0 at pci0:10:0:0: class=0x020000 card=0x000015d9 chip=0x10d38086 rev=0x00 hdr=0x00 > > vendor = 'Intel Corporation' > > device = '82574L Gigabit Network Connection' > > > > ixl0 at pci0:4:0:0: class=0x020000 card=0x00078086 chip=0x15728086 rev=0x01 hdr=0x00 > > vendor = 'Intel Corporation' > > device = 'Ethernet Controller X710 for 10GbE SFP+' > > > > > > ============================== > > > > Additional info: > > > > During the tests, we have bonded two interfaces into a lagg: > > > > ixl0: flags=8843 metric 0 mtu 1500 > > options=c500b8 > > ether 3c:fd:fe:aa:60:20 > > media: Ethernet autoselect (10Gbase-SR ) > > status: active > > nd6 options=29 > > ixl1: flags=8843 metric 0 mtu 1500 > > options=c500b8 > > ether 3c:fd:fe:aa:60:20 > > hwaddr 3c:fd:fe:aa:60:21 > > media: Ethernet autoselect (10Gbase-SR ) > > status: active > > nd6 options=29 > > > > > > lagg0: flags=8843 metric 0 mtu 1500 > > options=c500b8 > > ether 3c:fd:fe:aa:60:20 > > inet 10.10.10.92 netmask 0xffff0000 broadcast 10.10.255.255 > > laggproto failover lagghash l2,l3,l4 > > laggport: ixl0 flags=5 > > laggport: ixl1 flags=0<> > > groups: lagg > > media: Ethernet autoselect > > status: active > > nd6 options=29 > > > > using this config: > > > > ifconfig_ixl0="up -lro -tso -rxcsum -txcsum" (tried different options - got the same outcome) > > ifconfig_ixl1="up -lro -tso -rxcsum -txcsum" > > ifconfig_lagg0="laggproto failover laggport ixl0 laggport ixl1 10.10.10.92/24" > > > > > > We have randomly picked `ixl0` and restricted number of RX/TX queues to 1: > > /boot/loader.conf : > > dev.ixl.0.iflib.override_ntxqs=1 > > dev.ixl.0.iflib.override_nrxqs=1 > > > > leaving `ixl1` with a default number, matching number of cores (6). > > > > > > ixl0: mem 0xf8800000-0xf8ffffff,0xf9808000-0xf980ffff irq 40 at device 0.0 on pci4 > > ixl0: fw 5.0.40043 api 1.5 nvm 5.05 etid 80002927 oem 1.261.0 > > ixl0: PF-ID[0]: VFs 64, MSI-X 129, VF MSI-X 5, QPs 768, I2C > > ixl0: Using 1024 TX descriptors and 1024 RX descriptors > > ixl0: Using 1 RX queues 1 TX queues > > ixl0: Using MSI-X interrupts with 2 vectors > > ixl0: Ethernet address: 3c:fd:fe:aa:60:20 > > ixl0: Allocating 1 queues for PF LAN VSI; 1 queues active > > ixl0: PCI Express Bus: Speed 8.0GT/s Width x4 > > ixl0: SR-IOV ready > > ixl0: netmap queues/slots: TX 1/1024, RX 1/1024 > > ixl1: mem 0xf8000000-0xf87fffff,0xf9800000-0xf9807fff irq 40 at device 0.1 on pci4 > > ixl1: fw 5.0.40043 api 1.5 nvm 5.05 etid 80002927 oem 1.261.0 > > ixl1: PF-ID[1]: VFs 64, MSI-X 129, VF MSI-X 5, QPs 768, I2C > > ixl1: Using 1024 TX descriptors and 1024 RX descriptors > > ixl1: Using 6 RX queues 6 TX queues > > ixl1: Using MSI-X interrupts with 7 vectors > > ixl1: Ethernet address: 3c:fd:fe:aa:60:21 > > ixl1: Allocating 8 queues for PF LAN VSI; 6 queues active > > ixl1: PCI Express Bus: Speed 8.0GT/s Width x4 > > ixl1: SR-IOV ready > > ixl1: netmap queues/slots: TX 6/1024, RX 6/1024 > > > > > > This allowed us easy switch between different configurations without > > the need to reboot, by simply shutting down one interface or the other: > > > > `ifconfig XXX down` > > > > When testing `ixl0` that runs only a single queue: > > ixl0: Using 1 RX queues 1 TX queues > > ixl0: netmap queues/slots: TX 1/1024, RX 1/1024 > > > > we've got these results: > > > > `wrk -c 10 --header "Connection: close" -d 10 -t 1 --latency http://10.10.10.92:80/missing` > > Running 10s test @ http://10.10.10.92:80/missing > > 1 threads and 10 connections > > Thread Stats Avg Stdev Max +/- Stdev > > Latency 281.31us 297.74us 22.66ms 99.70% > > Req/Sec 19.91k 2.79k 21.25k 97.59% > > Latency Distribution > > 50% 266.00us > > 75% 309.00us > > 90% 374.00us > > 99% 490.00us > > 164440 requests in 10.02s, 47.52MB read > > Socket errors: read 0, write 0, timeout 0 > > Non-2xx or 3xx responses: 164440 > > Requests/sec: 16412.09 > > Transfer/sec: 4.74MB > > > > > > When testing `ixl1` that runs 6 queues: > > ixl1: Using 6 RX queues 6 TX queues > > ixl1: netmap queues/slots: TX 6/1024, RX 6/1024 > > > > we've got these results: > > > > `wrk -c 10 --header "Connection: close" -d 10 -t 1 --latency http://10.10.10.92:80/missing` > > Running 10s test @ http://10.10.10.92:80/missing > > 1 threads and 10 connections > > Thread Stats Avg Stdev Max +/- Stdev > > Latency 216.16us 71.97us 511.00us 47.56% > > Req/Sec 4.34k 2.76k 15.44k 83.17% > > Latency Distribution > > 50% 216.00us > > 75% 276.00us > > 90% 312.00us > > 99% 365.00us > > 43616 requests in 10.10s, 12.60MB read > > Socket errors: connect 0, read 24, write 8, timeout 0 > > Non-2xx or 3xx responses: 43616 > > Requests/sec: 4318.26 > > Transfer/sec: 1.25MB > > > > Do note, that, not only multiple queues cause issues they also dramatically > > decrease the performance of the network. > > > > Using `sysctl -w net.inet.tcp.ts_offset_per_conn=0` didn't help at all. > > > > Best regards, > > -Paul > > > > > > From owner-freebsd-stable@freebsd.org Sat Oct 19 17:32:25 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 940931525A9 for ; Sat, 19 Oct 2019 17:32:25 +0000 (UTC) (envelope-from devgs@ukr.net) Received: from frv191.fwdcdn.com (frv191.fwdcdn.com [212.42.77.191]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "*.ukr.net", Issuer "Thawte RSA CA 2018" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46wVMr21Gpz4S9t for ; Sat, 19 Oct 2019 17:32:24 +0000 (UTC) (envelope-from devgs@ukr.net) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=ffe; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Message-Id: References:In-Reply-To:Cc:To:Subject:From:Date:Sender:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=bScw/jFGWndC5VwaWImR8YFAx/txUP5TIuTknSYFxN0=; b=nnFwlRattPk02qR9bbd7evuCMS QO4mxTeY7Cm/0v9hgA8PNO4f+2De1h2jo/TRnY1NGzKLq9jlxON4B0Z/uCqRy3P3BgKfTLUsU2HkN 7FYXRmFkkwoOhCey4lt2TeG4eEnCaeizwvhVAv+UC8TYifjXg8EvShGc/Y/DUzGHNDdw=; Received: from [10.10.10.39] (helo=frv39.fwdcdn.com) by frv191.fwdcdn.com with smtp ID 1iLsaS-000M0b-OC for freebsd-stable@freebsd.org; Sat, 19 Oct 2019 20:32:20 +0300 Date: Sat, 19 Oct 2019 20:32:20 +0300 From: Paul Subject: Re[2]: Re[2]: Network anomalies after update from 11.2 STABLE to 12.1 STABLE To: Rick Macklem Cc: =?us-ascii?q?michael=2Etuexen=40lurchi=2Efranken=2Ede?= , =?us-ascii?q?freebsd-net=40freebsd=2Eorg?= , =?us-ascii?q?freebsd-stable=40freebsd=2Eorg?= Received: from devgs@ukr.net by frv39.fwdcdn.com; Sat, 19 Oct 2019 20:32:20 +0300 In-Reply-To: References: <1571499556.409350000.a1ewtyar@frv39.fwdcdn.com> X-Reply-Action: reply Message-Id: <1571505850.986841000.zen2nmth@frv39.fwdcdn.com> X-Mailer: mail.ukr.net 5.0 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: binary X-Rspamd-Queue-Id: 46wVMr21Gpz4S9t X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ukr.net header.s=ffe header.b=nnFwlRat; dmarc=pass (policy=none) header.from=ukr.net; spf=pass (mx1.freebsd.org: domain of devgs@ukr.net designates 212.42.77.191 as permitted sender) smtp.mailfrom=devgs@ukr.net X-Spamd-Result: default: False [-2.80 / 15.00]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; R_DKIM_ALLOW(-0.20)[ukr.net:s=ffe]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip4:212.42.77.0/24:c]; FREEMAIL_FROM(0.00)[ukr.net]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(0.00)[ipnet: 212.42.77.0/24(-4.83), asn: 8856(-3.88), country: UA(0.08)]; CC_EXCESS_QP(1.20)[]; TO_DN_SOME(0.00)[]; DWL_DNSWL_LOW(-1.00)[ukr.net.dwl.dnswl.org : 127.0.5.1]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[ukr.net:+]; DMARC_POLICY_ALLOW(-0.50)[ukr.net,none]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; IP_SCORE_FREEMAIL(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[ukr.net]; ASN(0.00)[asn:8856, ipnet:212.42.77.0/24, country:UA]; RCVD_TLS_LAST(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Oct 2019 17:32:25 -0000 Hi Rick, RST is only one part of a syndrome. Apart from it, we have a ton of different other issues. For example: a lot (50+) of ACK and [FIN, ACK] re-transmissions in cases where they are definitely not needed, as seen in tspdump, unless the packets that we see in the dump are not actually processed by the kernel(?), therefore leading to re-transmissions? It definitely has something to do with races, because issue completely disappears when only single queue is enabled. In other cases, we have observed that 12.1-STABLE has sent FIN, but then, when sending the ACK it didn't actually increment SEQ, as if those two packets FIN an ACK were sent concurrently, though ACK was dispatched later. Also, I want to focus on a weird behavior, as I wrote in the original post: issue also disappears if, multiple TCP streams each use different DST port. It's as if it has anything to do with sharing a port. 19 October 2019, 19:24:43, by "Rick Macklem" : > Btw, I once ran into a situation where "smart networking" was injecting > RSTs into a TCP stream. The packet captures at the client and server > machines were identical, except for the RSTs and the problem went away > when I connected the two machines with a cable, bypassing the network. > Might be worth a try, if you can do it? > > Good luck with it, rick > > ________________________________________ > From: owner-freebsd-net@freebsd.org on behalf of Paul > Sent: Saturday, October 19, 2019 12:09 PM > To: michael.tuexen@lurchi.franken.de; freebsd-net@freebsd.org; freebsd-stable@freebsd.org > Subject: Re[2]: Network anomalies after update from 11.2 STABLE to 12.1 STABLE > > Hi Michael, > > Thank you, for taking your time! > > We use physical machines. We don not have any special `pf` rules. > Both sides ran `pfctl -d` before testing. > > > `nginx` config is primitive, no secrets there: > > ------------------------------------------------------------------- > user www; > worker_processes auto; > > error_log /var/log/nginx/error.log warn; > > events { > worker_connections 81920; > kqueue_changes 4096; > use kqueue; > } > > http { > include mime.types; > default_type application/octet-stream; > > sendfile off; > keepalive_timeout 65; > tcp_nopush on; > tcp_nodelay on; > > # Logging > log_format main '$remote_addr - $remote_user [$time_local] "$request" ' > '$status $request_length $body_bytes_sent "$http_referer" ' > '"$http_user_agent" "$http_x_real_ip" "$realip_remote_addr" "$request_completion" "$request_time" ' > '"$request_body"'; > > access_log /var/log/nginx/access.log main; > > server { > listen 80 default; > > server_name localhost _; > > location / { > return 404; > } > } > } > ------------------------------------------------------------------- > > > `wrk` is compiled with a default configuration. We test like this: > > `wrk -c 10 --header "Connection: close" -d 10 -t 1 --latency http://10.10.10.92:80/missing` > > > Also, it seems that our issue, and the one described in this thread, are identical: > > https://lists.freebsd.org/pipermail/freebsd-net/2019-June/053667.html > > We both have the Intel network cards, BTW. Our network cards are these: > > em0 at pci0:10:0:0: class=0x020000 card=0x000015d9 chip=0x10d38086 rev=0x00 hdr=0x00 > vendor = 'Intel Corporation' > device = '82574L Gigabit Network Connection' > > ixl0 at pci0:4:0:0: class=0x020000 card=0x00078086 chip=0x15728086 rev=0x01 hdr=0x00 > vendor = 'Intel Corporation' > device = 'Ethernet Controller X710 for 10GbE SFP+' > > > ============================== > > Additional info: > > During the tests, we have bonded two interfaces into a lagg: > > ixl0: flags=8843 metric 0 mtu 1500 > options=c500b8 > ether 3c:fd:fe:aa:60:20 > media: Ethernet autoselect (10Gbase-SR ) > status: active > nd6 options=29 > ixl1: flags=8843 metric 0 mtu 1500 > options=c500b8 > ether 3c:fd:fe:aa:60:20 > hwaddr 3c:fd:fe:aa:60:21 > media: Ethernet autoselect (10Gbase-SR ) > status: active > nd6 options=29 > > > lagg0: flags=8843 metric 0 mtu 1500 > options=c500b8 > ether 3c:fd:fe:aa:60:20 > inet 10.10.10.92 netmask 0xffff0000 broadcast 10.10.255.255 > laggproto failover lagghash l2,l3,l4 > laggport: ixl0 flags=5 > laggport: ixl1 flags=0<> > groups: lagg > media: Ethernet autoselect > status: active > nd6 options=29 > > using this config: > > ifconfig_ixl0="up -lro -tso -rxcsum -txcsum" (tried different options - got the same outcome) > ifconfig_ixl1="up -lro -tso -rxcsum -txcsum" > ifconfig_lagg0="laggproto failover laggport ixl0 laggport ixl1 10.10.10.92/24" > > > We have randomly picked `ixl0` and restricted number of RX/TX queues to 1: > /boot/loader.conf : > dev.ixl.0.iflib.override_ntxqs=1 > dev.ixl.0.iflib.override_nrxqs=1 > > leaving `ixl1` with a default number, matching number of cores (6). > > > ixl0: mem 0xf8800000-0xf8ffffff,0xf9808000-0xf980ffff irq 40 at device 0.0 on pci4 > ixl0: fw 5.0.40043 api 1.5 nvm 5.05 etid 80002927 oem 1.261.0 > ixl0: PF-ID[0]: VFs 64, MSI-X 129, VF MSI-X 5, QPs 768, I2C > ixl0: Using 1024 TX descriptors and 1024 RX descriptors > ixl0: Using 1 RX queues 1 TX queues > ixl0: Using MSI-X interrupts with 2 vectors > ixl0: Ethernet address: 3c:fd:fe:aa:60:20 > ixl0: Allocating 1 queues for PF LAN VSI; 1 queues active > ixl0: PCI Express Bus: Speed 8.0GT/s Width x4 > ixl0: SR-IOV ready > ixl0: netmap queues/slots: TX 1/1024, RX 1/1024 > ixl1: mem 0xf8000000-0xf87fffff,0xf9800000-0xf9807fff irq 40 at device 0.1 on pci4 > ixl1: fw 5.0.40043 api 1.5 nvm 5.05 etid 80002927 oem 1.261.0 > ixl1: PF-ID[1]: VFs 64, MSI-X 129, VF MSI-X 5, QPs 768, I2C > ixl1: Using 1024 TX descriptors and 1024 RX descriptors > ixl1: Using 6 RX queues 6 TX queues > ixl1: Using MSI-X interrupts with 7 vectors > ixl1: Ethernet address: 3c:fd:fe:aa:60:21 > ixl1: Allocating 8 queues for PF LAN VSI; 6 queues active > ixl1: PCI Express Bus: Speed 8.0GT/s Width x4 > ixl1: SR-IOV ready > ixl1: netmap queues/slots: TX 6/1024, RX 6/1024 > > > This allowed us easy switch between different configurations without > the need to reboot, by simply shutting down one interface or the other: > > `ifconfig XXX down` > > When testing `ixl0` that runs only a single queue: > ixl0: Using 1 RX queues 1 TX queues > ixl0: netmap queues/slots: TX 1/1024, RX 1/1024 > > we've got these results: > > `wrk -c 10 --header "Connection: close" -d 10 -t 1 --latency http://10.10.10.92:80/missing` > Running 10s test @ http://10.10.10.92:80/missing > 1 threads and 10 connections > Thread Stats Avg Stdev Max +/- Stdev > Latency 281.31us 297.74us 22.66ms 99.70% > Req/Sec 19.91k 2.79k 21.25k 97.59% > Latency Distribution > 50% 266.00us > 75% 309.00us > 90% 374.00us > 99% 490.00us > 164440 requests in 10.02s, 47.52MB read > Socket errors: read 0, write 0, timeout 0 > Non-2xx or 3xx responses: 164440 > Requests/sec: 16412.09 > Transfer/sec: 4.74MB > > > When testing `ixl1` that runs 6 queues: > ixl1: Using 6 RX queues 6 TX queues > ixl1: netmap queues/slots: TX 6/1024, RX 6/1024 > > we've got these results: > > `wrk -c 10 --header "Connection: close" -d 10 -t 1 --latency http://10.10.10.92:80/missing` > Running 10s test @ http://10.10.10.92:80/missing > 1 threads and 10 connections > Thread Stats Avg Stdev Max +/- Stdev > Latency 216.16us 71.97us 511.00us 47.56% > Req/Sec 4.34k 2.76k 15.44k 83.17% > Latency Distribution > 50% 216.00us > 75% 276.00us > 90% 312.00us > 99% 365.00us > 43616 requests in 10.10s, 12.60MB read > Socket errors: connect 0, read 24, write 8, timeout 0 > Non-2xx or 3xx responses: 43616 > Requests/sec: 4318.26 > Transfer/sec: 1.25MB > > Do note, that, not only multiple queues cause issues they also dramatically > decrease the performance of the network. > > Using `sysctl -w net.inet.tcp.ts_offset_per_conn=0` didn't help at all. > > Best regards, > -Paul > > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >