Date: Wed, 15 Apr 2020 15:34:12 +0200 From: "Kristof Provost" <kp@FreeBSD.org> To: freebsd-testing@freebsd.org Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org Subject: Re: FreeBSD CI Weekly Report 2020-04-12 Message-ID: <A2DF53A3-FD86-427D-B1EE-508228B0F4CE@FreeBSD.org> In-Reply-To: <20200414223710.GB33328@freefall.freebsd.org> References: <20200414223710.GB33328@freefall.freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 15 Apr 2020, at 0:37, Li-Wen Hsu wrote: > (Please send the followup to freebsd-testing@ and note Reply-To is > set.) > > FreeBSD CI Weekly Report 2020-04-12 > =================================== > > Here is a summary of the FreeBSD Continuous Integration results for > the period > from 2020-04-06 to 2020-04-12. > > During this period, we have: > > * 1801 builds (94.0% (+0.4) passed, 6.0% (-0.4) failed) of buildworld > and > buildkernel (GENERIC and LINT) were executed on aarch64, amd64, > armv6, > armv7, i386, mips, mips64, powerpc, powerpc64, powerpcspe, riscv64, > sparc64 architectures for head, stable/12, stable/11 branches. > * 288 test runs (25.1% (-24.6) passed, 29.9% (+10.6) unstable, 45.1% > (+14.1) > exception) were executed on amd64, i386, riscv64 architectures for > head, > stable/12, stable/11 branches. > * 30 doc and www builds (83.3% (-1.3) passed, 16.7% (+1.3) failed) > > Test case status (on 2020-04-12 23:59): > | Branch/Architecture | Total | Pass | Fail | Skipped | > | ------------------- | --------- | ---------- | -------- | -------- | > | head/amd64 | 7744 (+4) | 7638 (+19) | 14 (+5) | 92 (-20) | > | head/i386 | 7742 (+4) | 7628 (+15) | 16 (+5) | 98 (-16) | > | 12-STABLE/amd64 | 7508 (0) | 7449 (-3) | 1 (+1) | 58 (+2) | > | 12-STABLE/i386 | 7506 (0) | 7425 (-17) | 2 (+2) | 79 (+15) | > | 11-STABLE/amd64 | 6882 (0) | 6829 (-6) | 1 (+1) | 52 (+5) | > | 11-STABLE/i386 | 6880 (0) | 6749 (-82) | 80 (+80) | 51 (+2) | > > (The statistics from experimental jobs are omitted) > > If any of the issues found by CI are in your area of interest or > expertise > please investigate the PRs listed below. > > The latest web version of this report is available at > https://hackmd.io/@FreeBSD-CI/report-20200412 and archive is available > at > https://hackmd.io/@FreeBSD-CI/ , any help is welcome. > > ## News > > * The test env now loads the required module for firewall tests. > > * New armv7 job is added (to replace armv6 one): > * FreeBSD-head-armv7-testvm > The images are available at https://artifact.ci.freebsd.org > FreeBSD-head-armv7-test is ready but needs test env update. > > ## Failing jobs > > * https://ci.freebsd.org/job/FreeBSD-head-amd64-gcc6_build/ > * See console log for the error details. > > ## Failing tests > > * https://ci.freebsd.org/job/FreeBSD-head-amd64-test/ > * local.kyua.integration.cmd_about_test.topic__authors__installed > * sys.netipsec.tunnel.empty.v4 > * sys.netipsec.tunnel.empty.v6 > * sys.netpfil.common.forward.ipf_v4 > * sys.netpfil.common.forward.ipfw_v4 > * sys.netpfil.common.forward.pf_v4 > * sys.netpfil.common.tos.ipfw_tos > * sys.netpfil.common.tos.pf_tos > * sys.netpfil.pf.forward.v4 I can’t actually reproduce this failure in my test VM, but with the ci test VM I can reproduce the problem. It’s not related to pf, because the sanity check ping we do before we set up pf already fails. Or rather pft_ping.py sends an incorrect packet, because `ping` does get the packet to go where it’s supposed to go. Scapy seems to fail to find the source IP address, so we get this: 12:12:22.152652 IP 0.0.0.0 > 198.51.100.3: ICMP echo request, id 0, seq 0, length 12 I have a vague recollection that we’ve seem this problem before, but I can’t remember what we did about it. In all likelihood most of the other netpfil tests fail for exactly the same reason. Best regards, Kristof From owner-freebsd-current@freebsd.org Wed Apr 15 14:09:57 2020 Return-Path: <owner-freebsd-current@freebsd.org> Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BFF8B2B7FD3; Wed, 15 Apr 2020 14:09:57 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 492PPd53FVz3PW8; Wed, 15 Apr 2020 14:09:57 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [5.9.86.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.codepro.be", Issuer "Let's Encrypt Authority X3" (verified OK)) (Authenticated sender: kp) by smtp.freebsd.org (Postfix) with ESMTPSA id 7A9EE14F66; Wed, 15 Apr 2020 14:09:57 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: by venus.codepro.be (Postfix, authenticated sender kp) id 20857EAAA; Wed, 15 Apr 2020 16:09:56 +0200 (CEST) From: "Kristof Provost" <kp@FreeBSD.org> To: freebsd-testing@freebsd.org Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org, bofh@freebsd.org, "Alexander V. Chernikov" <melifaro@freebsd.org> Subject: Re: FreeBSD CI Weekly Report 2020-04-12 Date: Wed, 15 Apr 2020 16:09:55 +0200 X-Mailer: MailMate (1.13.1r5671) Message-ID: <DCC86D9B-ECF3-4393-B1C6-D76D1AE8BAC2@FreeBSD.org> In-Reply-To: <A2DF53A3-FD86-427D-B1EE-508228B0F4CE@FreeBSD.org> References: <20200414223710.GB33328@freefall.freebsd.org> <A2DF53A3-FD86-427D-B1EE-508228B0F4CE@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current <freebsd-current.freebsd.org> List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>; List-Post: <mailto:freebsd-current@freebsd.org> List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help> List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, <mailto:freebsd-current-request@freebsd.org?subject=subscribe> X-List-Received-Date: Wed, 15 Apr 2020 14:09:57 -0000 On 15 Apr 2020, at 15:34, Kristof Provost wrote: > On 15 Apr 2020, at 0:37, Li-Wen Hsu wrote: >> (Please send the followup to freebsd-testing@ and note Reply-To is >> set.) >> >> FreeBSD CI Weekly Report 2020-04-12 >> =================================== >> >> Here is a summary of the FreeBSD Continuous Integration results for >> the period >> from 2020-04-06 to 2020-04-12. >> >> During this period, we have: >> >> * 1801 builds (94.0% (+0.4) passed, 6.0% (-0.4) failed) of buildworld >> and >> buildkernel (GENERIC and LINT) were executed on aarch64, amd64, >> armv6, >> armv7, i386, mips, mips64, powerpc, powerpc64, powerpcspe, riscv64, >> sparc64 architectures for head, stable/12, stable/11 branches. >> * 288 test runs (25.1% (-24.6) passed, 29.9% (+10.6) unstable, 45.1% >> (+14.1) >> exception) were executed on amd64, i386, riscv64 architectures for >> head, >> stable/12, stable/11 branches. >> * 30 doc and www builds (83.3% (-1.3) passed, 16.7% (+1.3) failed) >> >> Test case status (on 2020-04-12 23:59): >> | Branch/Architecture | Total | Pass | Fail | Skipped >> | >> | ------------------- | --------- | ---------- | -------- | -------- >> | >> | head/amd64 | 7744 (+4) | 7638 (+19) | 14 (+5) | 92 (-20) >> | >> | head/i386 | 7742 (+4) | 7628 (+15) | 16 (+5) | 98 (-16) >> | >> | 12-STABLE/amd64 | 7508 (0) | 7449 (-3) | 1 (+1) | 58 (+2) >> | >> | 12-STABLE/i386 | 7506 (0) | 7425 (-17) | 2 (+2) | 79 (+15) >> | >> | 11-STABLE/amd64 | 6882 (0) | 6829 (-6) | 1 (+1) | 52 (+5) >> | >> | 11-STABLE/i386 | 6880 (0) | 6749 (-82) | 80 (+80) | 51 (+2) >> | >> >> (The statistics from experimental jobs are omitted) >> >> If any of the issues found by CI are in your area of interest or >> expertise >> please investigate the PRs listed below. >> >> The latest web version of this report is available at >> https://hackmd.io/@FreeBSD-CI/report-20200412 and archive is >> available at >> https://hackmd.io/@FreeBSD-CI/ , any help is welcome. >> >> ## News >> >> * The test env now loads the required module for firewall tests. >> >> * New armv7 job is added (to replace armv6 one): >> * FreeBSD-head-armv7-testvm >> The images are available at https://artifact.ci.freebsd.org >> FreeBSD-head-armv7-test is ready but needs test env update. >> >> ## Failing jobs >> >> * https://ci.freebsd.org/job/FreeBSD-head-amd64-gcc6_build/ >> * See console log for the error details. >> >> ## Failing tests >> >> * https://ci.freebsd.org/job/FreeBSD-head-amd64-test/ >> * local.kyua.integration.cmd_about_test.topic__authors__installed >> * sys.netipsec.tunnel.empty.v4 >> * sys.netipsec.tunnel.empty.v6 >> * sys.netpfil.common.forward.ipf_v4 >> * sys.netpfil.common.forward.ipfw_v4 >> * sys.netpfil.common.forward.pf_v4 >> * sys.netpfil.common.tos.ipfw_tos >> * sys.netpfil.common.tos.pf_tos >> * sys.netpfil.pf.forward.v4 > I can’t actually reproduce this failure in my test VM, but with the > ci test VM I can reproduce the problem. > It’s not related to pf, because the sanity check ping we do before > we set up pf already fails. > Or rather pft_ping.py sends an incorrect packet, because `ping` does > get the packet to go where it’s supposed to go. > > Scapy seems to fail to find the source IP address, so we get this: > > 12:12:22.152652 IP 0.0.0.0 > 198.51.100.3: ICMP echo request, id 0, > seq 0, length 12 > > I have a vague recollection that we’ve seem this problem before, but > I can’t remember what we did about it. > > In all likelihood most of the other netpfil tests fail for exactly the > same reason. The problem appears to be that /usr/local/lib/python3.7/site-packages/scapy/arch/unix.py is misparsing the `netstat -rnW` output. For reference, this is the output in the test VM: Routing tables Internet: Destination Gateway Flags Nhop# Mtu Netif Expire 127.0.0.1 link#2 UH 1 16384 lo0 192.0.2.0/24 link#4 U 2 1500 epair0a 192.0.2.1 link#4 UHS 1 16384 lo0 198.51.100.0/24 192.0.2.2 UGS 3 1500 epair0a Internet6: Destination Gateway Flags Nhop# Mtu Netif Expire ::/96 ::1 UGRS 4 16384 lo0 ::1 link#2 UH 1 16384 lo0 ::ffff:0.0.0.0/96 ::1 UGRS 4 16384 lo0 fe80::/10 ::1 UGRS 4 16384 lo0 fe80::%lo0/64 link#2 U 3 16384 lo0 fe80::1%lo0 link#2 UHS 2 16384 lo0 fe80::%epair0a/64 link#4 U 5 1500 epair0a fe80::3d:9dff:fe7c:d70a%epair0a link#4 UHS 1 16384 lo0 fe80::%epair1a/64 link#6 U 6 1500 epair1a fe80::37:9eff:fe03:250a%epair1a link#6 UHS 1 16384 lo0 ff02::/16 ::1 UGRS 4 16384 lo0 The parsing code seems to think that the netif for the 198.51.100.0/24 route is 1500 rather than epair0a. This may be related to the difference in netstat output, because on my VM it looks like this: Routing tables Internet: Destination Gateway Flags Use Mtu Netif Expire default 172.16.2.1 UGS 319 1500 vtnet0 127.0.0.1 link#2 UH 0 16384 lo0 172.16.2.0/24 link#1 U 14 1500 vtnet0 172.16.2.2 link#1 UHS 0 16384 lo0 Internet6: Destination Gateway Flags Use Mtu Netif Expire ::/96 ::1 UGRS 0 16384 lo0 ::1 link#2 UH 0 16384 lo0 ::ffff:0.0.0.0/96 ::1 UGRS 0 16384 lo0 fe80::/10 ::1 UGRS 0 16384 lo0 fe80::%vtnet0/64 link#1 U 0 1500 vtnet0 fe80::5a9c:fcff:fe02:a95e%vtnet0 link#1 UHS 0 16384 lo0 fe80::%lo0/64 link#2 U 0 16384 lo0 fe80::1%lo0 link#2 UHS 0 16384 lo0 ff02::/16 ::1 UGRS 0 16384 lo0 I suspect that this change was introduced in r359823 (Introduce nexthop objects and new routing KPI). Best regards, Kristof From owner-freebsd-current@freebsd.org Wed Apr 15 14:49:44 2020 Return-Path: <owner-freebsd-current@freebsd.org> Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4048E2B8F3B; Wed, 15 Apr 2020 14:49:44 +0000 (UTC) (envelope-from olivier@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 492QHX0xqyz3wt2; Wed, 15 Apr 2020 14:49:44 +0000 (UTC) (envelope-from olivier@freebsd.org) Received: from mail-pg1-f179.google.com (mail-pg1-f179.google.com [209.85.215.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) (Authenticated sender: olivier/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id E6B86170E7; Wed, 15 Apr 2020 14:49:43 +0000 (UTC) (envelope-from olivier@freebsd.org) Received: by mail-pg1-f179.google.com with SMTP id r4so17093pgg.4; Wed, 15 Apr 2020 07:49:43 -0700 (PDT) X-Gm-Message-State: AGi0PuYCsN/5awg9udJTfwL7GvOf1DjTBmnISu7CpUvEVvY6WRFj07ag imRgxkdYAlQg+6ygtL28GIdD2tOMvK/So9jB1CM= X-Google-Smtp-Source: APiQypKecHW77MF5DIXi8FXRktWsuQds0xTOB4APspMaWMrkHF4euHjE8RZvfkDfLkoNh/Z3pJhQk8izB38QA7UpQLQ= X-Received: by 2002:a62:4e0c:: with SMTP id c12mr27056707pfb.87.1586962182720; Wed, 15 Apr 2020 07:49:42 -0700 (PDT) MIME-Version: 1.0 References: <20200414223710.GB33328@freefall.freebsd.org> <A2DF53A3-FD86-427D-B1EE-508228B0F4CE@FreeBSD.org> <DCC86D9B-ECF3-4393-B1C6-D76D1AE8BAC2@FreeBSD.org> In-Reply-To: <DCC86D9B-ECF3-4393-B1C6-D76D1AE8BAC2@FreeBSD.org> From: =?UTF-8?Q?Olivier_Cochard=2DLabb=C3=A9?= <olivier@freebsd.org> Date: Wed, 15 Apr 2020 16:49:31 +0200 X-Gmail-Original-Message-ID: <CA+q+Tcr-B5dzPKV-DSiDYmcueXX=AFgh6+j0G=-xJYB0XBzBpQ@mail.gmail.com> Message-ID: <CA+q+Tcr-B5dzPKV-DSiDYmcueXX=AFgh6+j0G=-xJYB0XBzBpQ@mail.gmail.com> Subject: Re: FreeBSD CI Weekly Report 2020-04-12 To: Kristof Provost <kp@freebsd.org> Cc: freebsd-testing@freebsd.org, freebsd-current <freebsd-current@freebsd.org>, "freebsd-stable@freebsd.org Stable" <freebsd-stable@freebsd.org>, bofh@freebsd.org, "Alexander V. Chernikov" <melifaro@freebsd.org> Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current <freebsd-current.freebsd.org> List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>; List-Post: <mailto:freebsd-current@freebsd.org> List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help> List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, <mailto:freebsd-current-request@freebsd.org?subject=subscribe> X-List-Received-Date: Wed, 15 Apr 2020 14:49:44 -0000 On Wed, Apr 15, 2020 at 4:10 PM Kristof Provost <kp@freebsd.org> wrote: > > The problem appears to be that > /usr/local/lib/python3.7/site-packages/scapy/arch/unix.py is misparsing > the `netstat -rnW` output. > Shouldn't scapy use the libxo output of netstat to mitigate this regression ?
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?A2DF53A3-FD86-427D-B1EE-508228B0F4CE>