Date: Mon, 29 Aug 2022 15:39:25 +0000 (GMT) From: jenkins-admin@FreeBSD.org To: melifaro@FreeBSD.org, dev-ci@FreeBSD.org Cc: jenkins-admin@FreeBSD.org Subject: FreeBSD-main-amd64-test - Build #21970 - Still Failing Message-ID: <2040894382.7101.1661787566399@jenkins.ci.freebsd.org> In-Reply-To: <2041606026.7095.1661774806528@jenkins.ci.freebsd.org> References: <2041606026.7095.1661774806528@jenkins.ci.freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
------=_Part_7100_1598561071.1661787565915 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable FreeBSD-main-amd64-test - Build #21970 (c24a8f19c5d554b828908e7dacf1855cd4e= e21f3) - Still Failing Build information: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/21970= / Full change log: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/21970/c= hanges Full build log: https://ci.FreeBSD.org/job/FreeBSD-main-amd64-test/21970/co= nsole Status explanation: "Failure" - the build is suspected being broken by the following changes "Still Failing" - the build has not been fixed by the following changes and this is a notification to note that these changes have not been fully tested by the CI system Change summaries: (Those commits are likely but not certainly responsible) db4ca19002c05d0bf83e85a5402005c9162aeee9 by melifaro: routing: add ability to store opaque indentifiers in nhops/nhgs c24a8f19c5d554b828908e7dacf1855cd4ee21f3 by melifaro: routing: fix rib_add_route_px() The end of the build log: [...truncated 4.14 MB...] #4 0xffffffff80e02d68 at in6_purgeaddr+0x38 #5 0xffffffff80e029e6 at in6_purgeifaddr+0x16 #6 0xffffffff80d14326 at if_purgeaddrs+0x146 #7 0xffffffff80d14892 at if_detach_internal+0x1c2 #8 0xffffffff80d14501 at if_detach+0x71 #9 0xffffffff8288680b at epair_clone_destroy+0x6b #10 0xffffffff80d1c34c at if_clone_destroyif+0x19c #11 0xffffffff80d1c134 at if_clone_destroy+0x114 #12 0xffffffff80d18e71 at ifioctl+0x741 #13 0xffffffff80c62572 at kern_ioctl+0x202 #14 0xffffffff80c622ba at sys_ioctl+0x12a #15 0xffffffff810ca956 at amd64_syscall+0x6a6 #16 0xffffffff8109d0fb at fast_syscall_common+0xf8 carp: demoted by -240 to 0 (vhid removed) epair0b: promiscuous mode disabled ifa_del_loopback_route: failed to delete interface epair0b route: 3 epair1a: link state changed to DOWN carp: 1@epair1b: BACKUP -> INIT (hardware interface down) carp: demoted by 240 to 240 (interface down) epair1b: link state changed to DOWN carp: demoted by -240 to 0 (vhid removed) epair1b: promiscuous mode disabled ifa_del_loopback_route: failed to delete interface epair1b route: 3 passed [5.420s] sys/netinet/carp:nd6_ns_source_mac -> bridge0: Ethernet address: 58:9c:fc= :00:7f:27 epair0a: Ethernet address: 02:89:07:6c:00:0a epair0b: Ethernet address: 02:89:07:6c:00:0b epair0a: link state changed to UP epair0b: link state changed to UP epair1a: Ethernet address: 02:fe:67:33:41:0a epair1b: Ethernet address: 02:fe:67:33:41:0b epair1a: link state changed to UP epair1b: link state changed to UP bridge0: link state changed to UP epair0a: promiscuous mode enabled epair1a: promiscuous mode enabled epair0b: promiscuous mode enabled carp: 1@epair0b: INIT -> BACKUP (initialization complete) epair1b: promiscuous mode enabled carp: 1@epair1b: INIT -> BACKUP (initialization complete) carp: 1@epair0b: BACKUP -> MASTER (master timed out) add_loopback_route_flags: failed to update interface epair0b route: 17 epair1a: promiscuous mode disabled epair0a: promiscuous mode disabled bridge0: link state changed to DOWN epair0a: link state changed to DOWN carp: 1@epair0b: MASTER -> INIT (hardware interface down) carp: demoted by 240 to 240 (interface down) epair0b: link state changed to DOWN carp: demoted by -240 to 0 (vhid removed) epair0b: promiscuous mode disabled ifa_del_loopback_route: failed to delete interface epair0b route: 3 epair1a: link state changed to DOWN carp: 1@epair1b: BACKUP -> INIT (hardware interface down) carp: demoted by 240 to 240 (interface down) epair1b: link state changed to DOWN carp: demoted by -240 to 0 (vhid removed) epair1b: promiscuous mode disabled ifa_del_loopback_route: failed to delete interface epair1b route: 3 passed [18.476s] sys/netinet/carp:negative_demotion -> epair0a: Ethernet address: 02:98:40= :9a:4c:0a epair0b: Ethernet address: 02:98:40:9a:4c:0b epair0a: link state changed to UP epair0b: link state changed to UP epair0a: promiscuous mode enabled carp: 1@epair0a: INIT -> BACKUP (initialization complete) epair0b: promiscuous mode enabled carp: 1@epair0b: INIT -> BACKUP (initialization complete) carp: 1@epair0a: BACKUP -> MASTER (master timed out) carp: demoted by -1 to -1 (sysctl) carp: 1@epair0a: MASTER -> INIT (hardware interface down) carp: demoted by 240 to 239 (interface down) epair0a: link state changed to DOWN carp: 1@epair0b: BACKUP -> INIT (hardware interface down) carp: demoted by 240 to 240 (interface down) epair0b: link state changed to DOWN ifa_del_loopback_route: failed to delete interface epair0a route: 3 ifa_del_loopback_route: failed to delete interface epair0a route: 3 carp: demoted by -240 to -1 (vhid removed) epair0a: promiscuous mode disabled ifa_del_loopback_route: failed to delete interface epair0b route: 3 ifa_del_loopback_route: failed to delete interface epair0b route: 3 ifa_del_loopback_route: failed to delete interface epair0b route: 3 carp: demoted by -240 to 0 (vhid removed) epair0b: promiscuous mode disabled passed [6.260s] sys/netinet/divert:ipdivert_ip_input_local_success -> epair0a: Ethernet a= ddress: 02:6c:b2:cc:da:0a epair0b: Ethernet address: 02:6c:b2:cc:da:0b epair0a: link state changed to UP epair0b: link state changed to UP epair0b: link state changed to DOWN epair0a: link state changed to DOWN passed [0.971s] sys/netinet/divert:ipdivert_ip_output_remote_success -> epair0a: Ethernet= address: 02:f6:68:04:6b:0a epair0b: Ethernet address: 02:f6:68:04:6b:0b epair0a: link state changed to UP epair0b: link state changed to UP epair0b: link state changed to DOWN epair0a: link state changed to DOWN passed [0.940s] sys/netinet/fibs:fibs_ifroutes1_success -> epair0a: Ethernet address: 02:= 93:5b:59:46:0a epair0b: Ethernet address: 02:93:5b:59:46:0b epair0a: link state changed to UP epair0b: link state changed to UP WARNING: Adding ifaddrs to all fibs has been turned off by default. Conside= r tuning net.add_addr_allfibs if needed epair0a: link state changed to DOWN epair0b: link state changed to DOWN passed [0.110s] sys/netinet/fibs_test:arpresolve_checks_interface_fib -> skipped: firewal= ls interfere with this test [0.035s] sys/netinet/fibs_test:default_route_with_multiple_fibs_on_same_subnet -> = tap0: Ethernet address: 58:9c:fc:00:1d:5e tap1: Ethernet address: 58:9c:fc:10:ff:e1 passed [0.063s] sys/netinet/fibs_test:default_route_with_multiple_fibs_on_same_subnet_inet6= -> tap0: Ethernet address: 58:9c:fc:00:1d:5e tap1: Ethernet address: 58:9c:fc:10:ff:e1 passed [0.061s] sys/netinet/fibs_test:loopback_and_network_routes_on_nondefault_fib -> ta= p0: Ethernet address: 58:9c:fc:00:1d:5e passed [0.043s] sys/netinet/fibs_test:loopback_and_network_routes_on_nondefault_fib_inet6 = -> tap0: Ethernet address: 58:9c:fc:00:1d:5e passed [0.051s] sys/netinet/fibs_test:same_ip_multiple_ifaces -> tap0: Ethernet address: = 58:9c:fc:00:1d:5e tap1: Ethernet address: 58:9c:fc:10:ff:e1 tap2: Ethernet address: 58:9c:fc:00:72:1a tap3: Ethernet address: 58:9c:fc:00:48:22 Kernel page fault with the following non-sleepable locks held: exclusive rm rib head lock (rib head lock) r =3D 0 (0xfffff80003d9a4e0) loc= ked @ /usr/src/sys/net/route/route_ctl.c:798 stack backtrace: #0 0xffffffff80c5cc45 at witness_debugger+0x65 #1 0xffffffff80c5ddaa at witness_warn+0x3fa #2 0xffffffff810c9ab6 at trap_pfault+0x86 #3 0xffffffff8109c7e8 at calltrap+0x8 #4 0xffffffff80d4bfa0 at rib_add_route_px+0x100 #5 0xffffffff80d4fea9 at add_loopback_route_flags+0x109 #6 0xffffffff80d4ff56 at ifa_switch_loopback_route+0x36 #7 0xffffffff80dac1a6 at in_scrubprefix+0x76 #8 0xffffffff80dabbab at in_difaddr_ioctl+0x2fb #9 0xffffffff80daac4c at in_control+0xac #10 0xffffffff80d18f24 at ifioctl+0x7f4 #11 0xffffffff80c62572 at kern_ioctl+0x202 #12 0xffffffff80c622ba at sys_ioctl+0x12a #13 0xffffffff810ca956 at amd64_syscall+0x6a6 #14 0xffffffff8109d0fb at fast_syscall_common+0xf8 Fatal trap 12: page fault while in kernel mode cpuid =3D 0; apic id =3D 00 fault virtual address=09=3D 0x18 fault code=09=09=3D supervisor read data, page not present instruction pointer=09=3D 0x20:0xffffffff80d4c1e7 stack pointer=09 =3D 0x28:0xfffffe00cb196860 frame pointer=09 =3D 0x28:0xfffffe00cb196920 code segment=09=09=3D base 0x0, limit 0xfffff, type 0x1b =09=09=09=3D DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags=09=3D interrupt enabled, resume, IOPL =3D 0 current process=09=09=3D 60923 (ifconfig) rdi: 0 rsi: 8 rdx: 0 rcx: 10000 r8: fffff8023fd81980 r9: ffffffff811e6366 rax: 3 rbx: 0 rbp: fffffe00cb196920 r10: 0 r11: ffffffff81f3c528 r12: fffff80003d9a400 r13: fffff80003d9a400 r14: 14 r15: fffff80003d9a4e0 trap number=09=09=3D 12 panic: page fault cpuid =3D 0 time =3D 1661787533 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00cb196= 620 vpanic() at vpanic+0x151/frame 0xfffffe00cb196670 panic() at panic+0x43/frame 0xfffffe00cb1966d0 trap_fatal() at trap_fatal+0x409/frame 0xfffffe00cb196730 trap_pfault() at trap_pfault+0xab/frame 0xfffffe00cb196790 calltrap() at calltrap+0x8/frame 0xfffffe00cb196790 --- trap 0xc, rip =3D 0xffffffff80d4c1e7, rsp =3D 0xfffffe00cb196860, rbp = =3D 0xfffffe00cb196920 --- add_route_flags() at add_route_flags+0x47/frame 0xfffffe00cb196920 rib_add_route_px() at rib_add_route_px+0x100/frame 0xfffffe00cb196990 add_loopback_route_flags() at add_loopback_route_flags+0x109/frame 0xfffffe= 00cb196a00 ifa_switch_loopback_route() at ifa_switch_loopback_route+0x36/frame 0xfffff= e00cb196a50 in_scrubprefix() at in_scrubprefix+0x76/frame 0xfffffe00cb196ad0 in_difaddr_ioctl() at in_difaddr_ioctl+0x2fb/frame 0xfffffe00cb196b30 in_control() at in_control+0xac/frame 0xfffffe00cb196bd0 ifioctl() at ifioctl+0x7f4/frame 0xfffffe00cb196cc0 kern_ioctl() at kern_ioctl+0x202/frame 0xfffffe00cb196d30 sys_ioctl() at sys_ioctl+0x12a/frame 0xfffffe00cb196e00 amd64_syscall() at amd64_syscall+0x6a6/frame 0xfffffe00cb196f30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00cb196f30 --- syscall (54, FreeBSD ELF64, sys_ioctl), rip =3D 0x2da08365097a, rsp =3D= 0x2da07fb1ac68, rbp =3D 0x2da07fb1acb0 --- KDB: enter: panic [ thread pid 60923 tid 103681 ] Stopped at kdb_enter+0x32: movq $0,0x12a4cf3(%rip) db:0:kdb.enter.panic> show pcpu cpuid =3D 0 dynamic pcpu =3D 0x9c2e00 curthread =3D 0xfffffe00cbf47740: pid 60923 tid 103681 critnest 1 "ifcon= fig" curpcb =3D 0xfffffe00cbf47c60 fpcurthread =3D 0xfffffe00cbf47740: pid 60923 "ifconfig" idlethread =3D 0xfffffe001122c3a0: tid 100003 "idle: cpu0" self =3D 0xffffffff82610000 curpmap =3D 0xfffffe00d06d6130 tssp =3D 0xffffffff82610384 rsp0 =3D 0xfffffe00cb197000 kcr3 =3D 0x8000000032813233 ucr3 =3D 0x80000000306aca33 scr3 =3D 0x306aca33 gs32p =3D 0xffffffff82610404 ldt =3D 0xffffffff82610444 tss =3D 0xffffffff82610434 curvnet =3D 0xfffff800031c1f40 spin locks held: db:0:kdb.enter.panic> reset + rc=3D0 + echo 'bhyve return code =3D 0' bhyve return code =3D 0 + sudo /usr/sbin/bhyvectl '--vm=3Dtestvm-main-amd64-21970' --destroy + sh -ex freebsd-ci/scripts/test/extract-meta.sh + METAOUTDIR=3Dmeta-out + rm -fr meta-out + mkdir meta-out + tar xvf meta.tar -C meta-out x ./ x ./disable-notyet-tests.sh x ./disable-dtrace-tests.sh x ./auto-shutdown x ./run.sh x ./run-kyua.sh x ./disable-zfs-tests.sh + rm -f 'test-report.*' + mv 'meta-out/test-report.*' . mv: rename meta-out/test-report.* to ./test-report.*: No such file or direc= tory + report=3Dtest-report.xml + [ -e freebsd-ci/jobs/FreeBSD-main-amd64-test/xfail-list -a -e test-report= .xml ] + rm -f disk-cam + jot 5 + rm -f disk1 + rm -f disk2 + rm -f disk3 + rm -f disk4 + rm -f disk5 + rm -f disk-test.img [PostBuildScript] - [INFO] Executing post build scripts. [FreeBSD-main-amd64-test] $ /bin/sh -xe /tmp/jenkins8301161217274635403.sh + ./freebsd-ci/artifact/post-link.py Post link: {'job_name': 'FreeBSD-main-amd64-test', 'commit': 'c24a8f19c5d55= 4b828908e7dacf1855cd4ee21f3', 'branch': 'main', 'target': 'amd64', 'target_= arch': 'amd64', 'link_type': 'latest_tested'} "Link created: main/latest_tested/amd64/amd64 -> ../../c24a8f19c5d554b82890= 8e7dacf1855cd4ee21f3/amd64/amd64\n" Recording test results ERROR: Step =E2=80=98Publish JUnit test result report=E2=80=99 failed: No t= est report files were found. Configuration error? Checking for post-build Performing post-build step Checking if email needs to be generated Email was triggered for: Failure - Any Sending email for trigger: Failure - Any Sending mail from default account using System Admin e-mail address ------=_Part_7100_1598561071.1661787565915--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?2040894382.7101.1661787566399>