From nobody Mon Nov 15 07:58:06 2021 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id E4C0F186C60B; Mon, 15 Nov 2021 07:58:21 +0000 (UTC) (envelope-from eugene@zhegan.in) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [128.127.146.8]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mail.norma.perm.ru", Issuer "Let's Encrypt Authority X3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Ht1lc5X8gz4gBl; Mon, 15 Nov 2021 07:58:20 +0000 (UTC) (envelope-from eugene@zhegan.in) Received: from [192.168.243.7] ([192.168.243.7]) by elf.hq.norma.perm.ru (8.16.1/8.15.2) with ESMTP id 1AF7tsjK084117; Mon, 15 Nov 2021 12:55:54 +0500 (+05) (envelope-from eugene@zhegan.in) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=vivat-retail.ru; s=key; t=1636962955; bh=1EI8EMkLYl0nAoMPQdaWbjAh4JZvkoGqp3eF/hdiWxQ=; h=Date:Subject:To:References:From:Cc:In-Reply-To; b=PGraU8u0olOzUbuxPlMSzVuNabmOeediTwtjafFXmbf/YbD8D0XV1ocnFa+oWjkty He4ru3KC2uvROVf5N67JrOz/IMVLsNjQwKGaEDW8De68THheaWBkHRYicCALjo2d93 6rdfOovJcbrYBBz05a/LvkE6c9PUuiaBCOlluqQI= Message-ID: Date: Mon, 15 Nov 2021 12:58:06 +0500 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.3.0 Subject: Re: packet loss between interfaces on the router Content-Language: ru To: stable@freebsd.org References: <216340c2-795d-d7ad-87d8-e07d9336564d@zhegan.in> From: "Eugene M. Zheganin" Cc: freebsd-pf@freebsd.org In-Reply-To: <216340c2-795d-d7ad-87d8-e07d9336564d@zhegan.in> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4Ht1lc5X8gz4gBl X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=fail ("headers rsa verify failed") header.d=vivat-retail.ru header.s=key header.b=PGraU8u0; dmarc=none; spf=pass (mx1.freebsd.org: domain of eugene@zhegan.in designates 128.127.146.8 as permitted sender) smtp.mailfrom=eugene@zhegan.in X-Spamd-Result: default: False [-0.30 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; R_DKIM_REJECT(1.00)[vivat-retail.ru:s=key]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[zhegan.in]; R_SPF_ALLOW(-0.20)[+a]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_SPAM_SHORT(1.00)[1.000]; DKIM_TRACE(0.00)[vivat-retail.ru:-]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:212494, ipnet:128.127.146.0/24, country:RU]; RCVD_COUNT_TWO(0.00)[2]; MID_RHS_MATCH_FROM(0.00)[] X-Spam: Yes X-ThisMailContainsUnwantedMimeParts: N Hello, 15.11.2021 2:14, Eugene M. Zheganin пишет: > [...] > The host is running PF as a packet filter, several dozens of rules. I > disable the scrub on outer interface (since the lost packet wasn'ta  > fragment, I was sceptical about it, and it doesn't help indeed). > [...] > ...and seems like it's a PF problem (so I probably should've started this conversation in pf@) Here's another stalled session with PF debug turned "loud". Below are caprtures on outer and inner interfaces, along with PF debug messages. What is the "3" condition ? I only managed to find that this is some sort of ackskew clashing. Could something be done here via pf configuration ? Outer interface: ===Cut=== [...] 12:16:37.537660 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15260617, win 1652, options [nop,nop,TS val 2720237530 ecr 2777961105], length 0 12:16:37.537670 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15263513, win 1652, options [nop,nop,TS val 2720237530 ecr 2777961105], length 0 12:16:37.537737 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15266409, win 1652, options [nop,nop,TS val 2720237531 ecr 2777961105], length 0 12:16:37.537824 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15276545, win 1652, options [nop,nop,TS val 2720237532 ecr 2777961105], length 0 12:16:37.537940 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15282337, win 1652, options [nop,nop,TS val 2720237532 ecr 2777961105], length 0 12:16:37.538261 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15324329:15325777, ack 150, win 4107, options [nop,nop,TS val 2777961135 ecr 2720237529], length 1448: HTTP 12:16:37.538272 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15325777:15327225, ack 150, win 4107, options [nop,nop,TS val 2777961135 ecr 2720237529], length 1448: HTTP 12:16:37.538278 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15327225:15328673, ack 150, win 4107, options [nop,nop,TS val 2777961135 ecr 2720237530], length 1448: HTTP 12:16:37.538289 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15328673:15330121, ack 150, win 4107, options [nop,nop,TS val 2777961135 ecr 2720237530], length 1448: HTTP 12:16:37.538323 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15330121:15331569, ack 150, win 4107, options [nop,nop,TS val 2777961135 ecr 2720237530], length 1448: HTTP 12:16:37.538335 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15331569:15333017, ack 150, win 4107, options [nop,nop,TS val 2777961135 ecr 2720237530], length 1448: HTTP 12:16:37.544805 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15286681, win 1652, options [nop,nop,TS val 2720237533 ecr 2777961105], length 0 12:16:37.545209 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15291025, win 1652, options [nop,nop,TS val 2720237534 ecr 2777961105], length 0 12:16:37.545278 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15296817, win 1652, options [nop,nop,TS val 2720237534 ecr 2777961105], length 0 12:16:37.545436 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15304057, win 1652, options [nop,nop,TS val 2720237535 ecr 2777961105], length 0 12:16:37.545985 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15308401, win 1652, options [nop,nop,TS val 2720237535 ecr 2777961105], length 0 12:16:37.546086 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15314193, win 1652, options [nop,nop,TS val 2720237536 ecr 2777961105], length 0 12:16:37.563993 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15319985, win 1652, options [nop,nop,TS val 2720237554 ecr 2777961125], length 0 12:16:37.564063 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15324329, win 1652, options [nop,nop,TS val 2720237555 ecr 2777961125], length 0 12:16:37.564445 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15327225, win 1652, options [nop,nop,TS val 2720237556 ecr 2777961135], length 0 12:16:37.564696 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15331569, win 1652, options [nop,nop,TS val 2720237557 ecr 2777961135], length 0 12:16:37.607557 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15333017, win 1652, options [nop,nop,TS val 2720237598 ecr 2777961135], length 0 12:16:37.768865 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777961365 ecr 2720237530], length 1448: HTTP 12:16:37.797072 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15333017, win 1652, options [nop,nop,TS val 2720237787 ecr 2777961135,nop,nop,sack 1 {15260617:15262065}], length 0 12:16:38.026968 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777961625 ecr 2720237530], length 1448: HTTP 12:16:38.044806 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15333017, win 1652, options [nop,nop,TS val 2720238045 ecr 2777961135,nop,nop,sack 1 {15260617:15262065}], length 0 12:16:38.349576 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777961945 ecr 2720237530], length 1448: HTTP 12:16:38.385464 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15333017, win 1652, options [nop,nop,TS val 2720238368 ecr 2777961135,nop,nop,sack 1 {15260617:15262065}], length 0 12:16:38.798092 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777962395 ecr 2720237530], length 1448: HTTP 12:16:38.826845 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15333017, win 1652, options [nop,nop,TS val 2720238816 ecr 2777961135,nop,nop,sack 1 {15260617:15262065}], length 0 12:16:39.477080 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777963075 ecr 2720237530], length 1448: HTTP 12:16:39.510688 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15333017, win 1652, options [nop,nop,TS val 2720239497 ecr 2777961135,nop,nop,sack 1 {15260617:15262065}], length 0 12:16:40.643251 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777964235 ecr 2720237530], length 1448: HTTP 12:16:40.673700 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15333017, win 1652, options [nop,nop,TS val 2720240661 ecr 2777961135,nop,nop,sack 1 {15260617:15262065}], length 0 12:16:42.759423 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777966355 ecr 2720237530], length 1448: HTTP 12:16:42.790404 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15333017, win 1652, options [nop,nop,TS val 2720242777 ecr 2777961135,nop,nop,sack 1 {15260617:15262065}], length 0 12:16:46.809355 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777970405 ecr 2720237530], length 1448: HTTP 12:16:46.840079 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15333017, win 1652, options [nop,nop,TS val 2720246827 ecr 2777961135,nop,nop,sack 1 {15260617:15262065}], length 0 12:16:54.692330 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777978285 ecr 2720237530], length 1448: HTTP ^C ===Cut=== Inner interface: ===Cut=== [...] 12:16:37.537666 IP 62.109.28.82.52982 > 91.206.242.9.8080: Flags [.], ack 15260617, win 1652, options [nop,nop,TS val 2720237530 ecr 2777961105], length 0 12:16:37.538251 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15324329:15325777, ack 150, win 4107, options [nop,nop,TS val 2777961135 ecr 2720237529], length 1448: HTTP 12:16:37.538266 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15325777:15327225, ack 150, win 4107, options [nop,nop,TS val 2777961135 ecr 2720237529], length 1448: HTTP 12:16:37.538274 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15327225:15328673, ack 150, win 4107, options [nop,nop,TS val 2777961135 ecr 2720237530], length 1448: HTTP 12:16:37.538282 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15328673:15330121, ack 150, win 4107, options [nop,nop,TS val 2777961135 ecr 2720237530], length 1448: HTTP 12:16:37.538312 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15330121:15331569, ack 150, win 4107, options [nop,nop,TS val 2777961135 ecr 2720237530], length 1448: HTTP 12:16:37.538328 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15331569:15333017, ack 150, win 4107, options [nop,nop,TS val 2777961135 ecr 2720237530], length 1448: HTTP 12:16:37.768852 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777961365 ecr 2720237530], length 1448: HTTP 12:16:38.026950 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777961625 ecr 2720237530], length 1448: HTTP 12:16:38.349553 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777961945 ecr 2720237530], length 1448: HTTP 12:16:38.798075 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777962395 ecr 2720237530], length 1448: HTTP 12:16:39.477063 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777963075 ecr 2720237530], length 1448: HTTP 12:16:40.643234 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777964235 ecr 2720237530], length 1448: HTTP 12:16:42.759404 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777966355 ecr 2720237530], length 1448: HTTP 12:16:46.809338 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777970405 ecr 2720237530], length 1448: HTTP 12:16:54.692313 IP 91.206.242.9.8080 > 62.109.28.82.52982: Flags [.], seq 15260617:15262065, ack 150, win 4107, options [nop,nop,TS val 2777978285 ecr 2720237530], length 1448: HTTP ^C ===Cut=== PF debug messages for this session: ===Cut=== # grep -A1 62.109.28.82:52982 messages Nov 15 12:16:25 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1424 modulator=0 wscale=7] [lo=782592198 high=782801846 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=782670390 len=0 ackskew=-78192 pkts=394:1710 dir=in,fwd Nov 15 12:16:25 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:25 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1424 modulator=0 wscale=7] [lo=782592198 high=782801846 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=782674734 len=0 ackskew=-82536 pkts=394:1710 dir=in,fwd Nov 15 12:16:25 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:25 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1424 modulator=0 wscale=7] [lo=782592198 high=782801846 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=782684870 len=0 ackskew=-92672 pkts=394:1710 dir=in,fwd Nov 15 12:16:25 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:25 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1424 modulator=0 wscale=7] [lo=782592198 high=782801846 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=782690662 len=0 ackskew=-98464 pkts=394:1710 dir=in,fwd Nov 15 12:16:25 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:25 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1424 modulator=0 wscale=7] [lo=782592198 high=782801846 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=782692110 len=0 ackskew=-99912 pkts=394:1710 dir=in,fwd Nov 15 12:16:25 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:32 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1424 modulator=0 wscale=7] [lo=786691630 high=786937614 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=786766926 len=0 ackskew=-75296 pkts=1069:4605 dir=in,fwd Nov 15 12:16:32 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:32 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1424 modulator=0 wscale=7] [lo=786691630 high=786937614 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=786788646 len=0 ackskew=-97016 pkts=1072:4636 dir=in,fwd Nov 15 12:16:32 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:32 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1424 modulator=0 wscale=7] [lo=786691630 high=786937614 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=786792990 len=0 ackskew=-101360 pkts=1072:4636 dir=in,fwd Nov 15 12:16:32 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:32 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1424 modulator=0 wscale=7] [lo=786691630 high=786937614 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=786804574 len=0 ackskew=-112944 pkts=1072:4636 dir=in,fwd Nov 15 12:16:32 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:32 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1424 modulator=0 wscale=7] [lo=786691630 high=786937614 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=786810366 len=0 ackskew=-118736 pkts=1072:4636 dir=in,fwd Nov 15 12:16:32 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:32 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1424 modulator=0 wscale=7] [lo=786691630 high=786937614 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=786829190 len=0 ackskew=-137560 pkts=1072:4636 dir=in,fwd Nov 15 12:16:32 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:34 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1494 modulator=0 wscale=7] [lo=789374774 high=789626822 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=789444278 len=0 ackskew=-69504 pkts=1494:6459 dir=in,fwd Nov 15 12:16:34 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:37 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795415830 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795483886 len=0 ackskew=-68056 pkts=2452:10662 dir=in,fwd Nov 15 12:16:37 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:37 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795415830 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795486782 len=0 ackskew=-70952 pkts=2452:10662 dir=in,fwd Nov 15 12:16:37 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:37 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795415830 high=795692446 win=4107 modulator=0 wscale=6] 4:4pf: state reuse TCP outA wire:  seq=1741467272 (1741467272) ack=795496918 len=0 ackskew=-81088 pkts=2452:10662 dir=in,fwd Nov 15 12:16:37 gw0 kernel: 192.168.0.247pf: State failure on: 3   | -- Nov 15 12:16:37 gw0 kernel:  wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795415830 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795502710 len=0 ackskew=-86880 pkts=2452:10662 dir=in,fwd Nov 15 12:16:37 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:37 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795415830 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795507054 len=0 ackskew=-91224 pkts=2452:10668 dir=in,fwd Nov 15 12:16:37 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:37 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795415830 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795511398 len=0 ackskew=-95568 pkts=2452:10668 dir=in,fwd Nov 15 12:16:37 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:37 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795415830 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795517190 len=0 ackskew=-101360 pkts=2452:10668 dir=in,fwd Nov 15 12:16:37 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:37 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795415830 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795524430 len=0 ackskew=-108600 pkts=2452:10668 dir=in,fwd Nov 15 12:16:37 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:37 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795415830 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795528774 len=0 ackskew=-112944 pkts=2452:10668 dir=in,fwd Nov 15 12:16:37 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:37 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795415830 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795534566 len=0 ackskew=-118736 pkts=2452:10668 dir=in,fwd Nov 15 12:16:37 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:37 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795415830 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795540358 len=0 ackskew=-124528 pkts=2452:10668 dir=in,fwd Nov 15 12:16:37 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:37 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795415830 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795544702 len=0 ackskew=-128872 pkts=2452:10668 dir=in,fwd Nov 15 12:16:37 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:37 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795415830 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795547598 len=0 ackskew=-131768 pkts=2452:10668 dir=in,fwd Nov 15 12:16:37 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:37 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795415830 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795551942 len=0 ackskew=-136112 pkts=2452:10668 dir=in,fwd Nov 15 12:16:37 gw0 kernel: pf: State failure on:     3   | Nov 15 12:16:37 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795415830 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-137560 pkts=2452:10668 dir=in,fwd Nov 15 12:16:37 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:37 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10669 dir=in,fwd Nov 15 12:16:37 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:38 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10670 dir=in,fwd Nov 15 12:16:38 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:38 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10671 dir=in,fwd Nov 15 12:16:38 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:39 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10672 dir=in,fwd Nov 15 12:16:39 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:39 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10673 dir=in,fwd Nov 15 12:16:39 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:40 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10674 dir=in,fwd Nov 15 12:16:40 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:42 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10675 dir=in,fwd Nov 15 12:16:42 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:46 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10676 dir=in,fwd Nov 15 12:16:46 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:16:54 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10677 dir=in,fwd Nov 15 12:16:54 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:17:09 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 FA seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10677 dir=in,fwd Nov 15 12:17:09 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:17:09 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 FA seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10677 dir=in,fwd Nov 15 12:17:09 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:17:09 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 FA seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10677 dir=in,fwd Nov 15 12:17:09 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:17:10 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 FA seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10677 dir=in,fwd Nov 15 12:17:10 gw0 kernel: pf: State failure on:     3   | Nov 15 12:17:10 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467273) ack=795553390 len=0 ackskew=-70952 pkts=2452:10678 dir=in,fwd Nov 15 12:17:10 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:17:11 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 FA seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10678 dir=in,fwd Nov 15 12:17:11 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:17:13 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 FA seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10678 dir=in,fwd Nov 15 12:17:13 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:17:16 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 FA seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10678 dir=in,fwd Nov 15 12:17:16 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:17:24 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 FA seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10678 dir=in,fwd Nov 15 12:17:24 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:17:25 gw0 kernel: pf: BAD state: pf: OK ICMP 5:1 TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6192.168.0.248 -> 192.168.0.230 state: TCP out wire: 192.168.0.230:3340 192.168.11.4] 4:4 A seq=1741467272 (1741467273) ack=795553390 len=0 ackskew=-70952 pkts=2452:10679 dir=in,fwd Nov 15 12:17:25 gw0 kernel: :59016pf: State failure on:     3   | -- Nov 15 12:17:39 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 FA seq=1741467272 (1741467272) ack=795553390 len=0 ackskew=-70952 pkts=2452:10679 dir=in,fwd Nov 15 12:17:39 gw0 kernel: pf: State failure on:     3   | -- Nov 15 12:17:41 gw0 kernel: pf: BAD state: TCP in wire: 62.109.28.82:52982 91.206.242.9:8080 stack: - [lo=1741467272 high=1741730120 win=1652 modulator=0 wscale=7] [lo=795482438 high=795692446 win=4107 modulator=0 wscale=6] 4:4 A seq=1741467272 (1741467273) ack=795553390 len=0 ackskew=-70952 pkts=2452:10680 dir=in,fwd Nov 15 12:17:41 gw0 kernel: pf: State failure on:     3   | ===Cut=== Eugene. From nobody Mon Nov 15 08:06:11 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id C04BF184B1DA for ; Mon, 15 Nov 2021 08:08:17 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: from smtprelay08.ispgateway.de (smtprelay08.ispgateway.de [134.119.228.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Ht1z4627Gz4lh9 for ; Mon, 15 Nov 2021 08:08:16 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: from [91.20.68.89] (helo=fabiankeil.de) by smtprelay08.ispgateway.de with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1mmX28-0002rp-PX for freebsd-stable@freebsd.org; Mon, 15 Nov 2021 09:08:08 +0100 Date: Mon, 15 Nov 2021 09:06:11 +0100 From: Fabian Keil To: freebsd-stable@freebsd.org Subject: Re: stable/12: jail(2) failures after ca9ab8ea1774 Message-ID: <20211115085227.56bd2255@fabiankeil.de> In-Reply-To: <8ef505b5-2d3b-8f94-7de1-fa9eaa1f2e2f@bruelltuete.com> References: <20211110074613.6b81f85a@fabiankeil.de> <8ef505b5-2d3b-8f94-7de1-fa9eaa1f2e2f@bruelltuete.com> List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; boundary="Sig_/gI62DcKeT=JdXkCo+8aZm60"; protocol="application/pgp-signature"; micalg=pgp-sha1 X-Df-Sender: Nzc1MDY3 X-Rspamd-Queue-Id: 4Ht1z4627Gz4lh9 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of freebsd-listen@fabiankeil.de has no SPF policy when checking 134.119.228.98) smtp.mailfrom=freebsd-listen@fabiankeil.de X-Spamd-Result: default: False [-0.73 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_SPAM_SHORT(1.00)[1.000]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[fabiankeil.de]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; RECEIVED_SPAMHAUS_PBL(0.00)[91.20.68.89:received]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_SPAM_LONG(0.47)[0.467]; RCVD_IN_DNSWL_NONE(0.00)[134.119.228.98:from]; SIGNED_PGP(-2.00)[]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:34011, ipnet:134.119.228.0/24, country:DE]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[134.119.228.98:from] X-ThisMailContainsUnwantedMimeParts: N --Sig_/gI62DcKeT=JdXkCo+8aZm60 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Johannes Totz via freebsd-stable wrote on 2021= -11-11: > On 10/11/2021 06:46, Fabian Keil wrote: > > ElectroBSD's ggate[cd]-related patches are available at: > > > > They should apply cleanly on stable/12 e644c87aa. >=20 > Lots of great stuff in that patch file! Thanks for sharing! You're welcome. As it turned out, jail(2) works as advertised when the forked process closes the pidfile which it doesn't need. I've updated the patch set: > Which ones of those should we back-port to fbsd? Not my call. The security fixes should be uncontroversial so I'd recommend looking at them first. > I've got patches open on ggate too, see > https://reviews.freebsd.org/D31722 > https://reviews.freebsd.org/D31727 > https://reviews.freebsd.org/D31709 Interesting. Thanks for the links. Fabian --Sig_/gI62DcKeT=JdXkCo+8aZm60 Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iF0EARECAB0WIQTKUNd6H/m3+ByGULIFiohV/3dUnQUCYZIU8wAKCRAFiohV/3dU nYJNAJ91Zt/wF/MfLn/XKUDj6qj0LXne9ACfRUQQa/yKpv/P8rxDgDr6WAtnmWk= =X7cP -----END PGP SIGNATURE----- --Sig_/gI62DcKeT=JdXkCo+8aZm60-- From nobody Mon Nov 15 16:47:44 2021 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id A7FDF188E0F4; Mon, 15 Nov 2021 16:47:46 +0000 (UTC) (envelope-from debdrup@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HtFVV3y4vz4Zhw; Mon, 15 Nov 2021 16:47:46 +0000 (UTC) (envelope-from debdrup@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1636994866; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type; bh=m+iAUVB0VdGYXfnHCgBmER8qhWHmOpGg6tJMTEIf3jc=; b=arRU/kkFqnRjt9tH2y+Lch2OMfwK/Yv7qtiSIe8E7EUXWS8rUrJdT0Oi7BKhiKRq57LRuC BAviWpcpw6geH/i88Qq2AsJcVA209EvAAYrWlNH2cO0Bty1yw4K6OnJPXVtIWisKO/qV0J wMNrEhkebmtJIQPg+/li6FXVqDkhvEGJ0tUHinP3B0EnK1lX/x8Hq69k8oXrn9ejUoGxOU LdWYvQ4XAB03HcYir4A6ciJJUxLIfPEuXVOEcxF0MJsspANdDpdMNjCwXyokMJnb7iF2Ug NLW2+Qq5/tcNmHHRbpppFxjq9M3Pukr7XcXRjIdxwjcx++fCx0vAmiGihqGWXA== Received: by freefall.freebsd.org (Postfix, from userid 1471) id 702981BB1D; Mon, 15 Nov 2021 16:47:46 +0000 (UTC) Date: Mon, 15 Nov 2021 17:47:44 +0100 From: Daniel Ebdrup Jensen To: hackers@FreeBSD.org Cc: current@FreeBSD.org, stable@FreeBSD.org Subject: FreeBSD Quarterly Status Report - Third Quarter 2021 Message-ID: <20211115164744.chsesrxkz3bb4wct@nerd-thinkpad.local> Mail-Followup-To: Daniel Ebdrup Jensen , hackers@FreeBSD.org, current@FreeBSD.org, stable@FreeBSD.org List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="66dsezfsnajsewcm" Content-Disposition: inline ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1636994866; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type; bh=m+iAUVB0VdGYXfnHCgBmER8qhWHmOpGg6tJMTEIf3jc=; b=qsbgdTuE4/drg/gfERD1lYv7/TTrdYPRAbv36xHF4Ptp1dxWEeSlAnzFSE4jlta+BK2pXj SXmn+JErgXWhOsM+dBtG8Zjza0XasueA0briFqa9/zh8MOyvOf/6gGCa+9WJQ8WlT3A+k4 Evo0MIHzCi7cnpeOJgbFQI7iFz1nGKkwf6fesSexjgRVvUuINz1WtO86bXkfIALNOChIb/ PFTUSRBUN3qYBifA/BaJcfOEAXx6G06yUBpmQw/Oh/v4gPIfBFMaJInt/+PBcGNmDC2ugD 8iqph3Xp+lnvIZRHOVPmRn8s86bIkEaQRwz4Z8CfPPQmEFGAahwrmDLlrOpWyA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1636994866; a=rsa-sha256; cv=none; b=jWTTTGm7W3kWvtsH+6WaC+NDd4uYReFSQyVxxVLhqWXvO2n3GFcwI8E7Yx7yyOKyl1M2Lc GCsMVKasxl2WruS2tcjeSUjU2VI2XsLjMIDG6iRslFZoyrSJtuE4K1P46gmDpWj3G1k7nJ 8CRtTOmLakqQrJByRcBfd05yGNWET3YJEOo/Rqw9moaaTl1C3rpzIzKBayg29TbdSr9VOY eK5A2Uycwtanztk8B056g7OtmmpeZcgyrDHaami0wkVk5+pByQHVaRI+hq10EE8MIMvMjV 2kURTJ4Bp0wL3Nd6eanRPTpZpYswHC3TsCmrKgJ2r7fhG+uzcixnuEKacYg0/A== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N --66dsezfsnajsewcm Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable FreeBSD Quarterly Status Report 3rd Quarter 2021 This report covers FreeBSD related projects for the period between July and September, and is the third of four planned reports for 2021, and contains = 42 entries. The third quarter of 2021 was quite active in lots of different areas, so t= he report covers a bunch of interesting work including but not limited to boot performance, compile-time analysis, hole-punching support, various drivers,= ZFS raidz expansion, an update to the sound mixer, and many more. Regrettably, the status report got a bit delayed, but we hope that the apho= rism better late than never can apply here. Yours, Daniel Ebdrup Jensen, with status hat on. =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Table of Contents =E2=80=A2 FreeBSD Team Reports =E2=96=A1 FreeBSD Foundation =E2=96=A1 FreeBSD Release Engineering Team =E2=96=A1 Cluster Administration Team =E2=96=A1 Continuous Integration =E2=96=A1 Ports Collection =E2=96=A1 Documentation Engineering Team =E2=96=A1 FreeBSD Website Revamp - WebApps working group =E2=80=A2 Projects =E2=96=A1 Boot Performance Improvements =E2=96=A1 Git Migration Working Group =E2=96=A1 LLDB Debugger Improvements =E2=96=A1 Linux compatibility layer update =E2=96=A1 Sound mixer improvements =E2=96=A1 Base System OpenSSH Update =E2=80=A2 Kernel =E2=96=A1 Arm64 improvements =E2=96=A1 FreeBSD on Microsoft HyperV and Azure =E2=96=A1 Improved amd64 UEFI boot =E2=96=A1 ENA FreeBSD Driver Update =E2=96=A1 Hole-punching support on FreeBSD =E2=96=A1 Intel Networking on FreeBSD =E2=96=A1 Intel Wireless driver support =E2=96=A1 Microchip LAN743x mgb(4) Device Driver =E2=96=A1 Fixes for msdosfs_rename VOP =E2=96=A1 OpenZFS RAIDZ Expansion update =E2=96=A1 RFC1191 support in IPSEC tunnels =E2=96=A1 Realtek rtw88 support =E2=96=A1 Marvell SDHCI improvements and ACPI support =E2=96=A1 Stack gap handling improvements =E2=96=A1 syzkaller on FreeBSD =E2=96=A1 Using OCF in WireGuard =E2=96=A1 Intel Volume Management Device driver update =E2=80=A2 Ports =E2=96=A1 Improve CPE Data in the Ports Collection =E2=96=A1 Why do we need it? =E2=96=A1 How can I help? =E2=96=A1 Open Tasks =E2=96=A1 FreeBSD Erlang Ecosystem Ports update =E2=96=A1 KDE on FreeBSD =E2=96=A1 OpenSearch on FreeBSD =E2=96=A1 Valgrind - Official Support for FreeBSD has Landed =E2=96=A1 Wine on FreeBSD =E2=96=A1 Ztop =E2=80=A2 Miscellaneous =E2=96=A1 -CURRENT compilation time analysis =E2=80=A2 Third-Party Projects =E2=96=A1 Gitlab 14.3 available =E2=96=A1 helloSystem =E2=96=A1 Containers & FreeBSD: Pot, Potluck & Potman =E2=96=A1 WireGuard on FreeBSD Stabilizes, Eyes Upstreaming =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 FreeBSD Team Reports Entries from the various official and semi-official teams, as found in the Administration Page. FreeBSD Foundation Links: FreeBSD Foundation URL: https://www.FreeBSDfoundation.org Technology Roadmap URL: https://freebsdfoundation.org/blog/technology-roadm= ap/ Donate URL: https://www.FreeBSDfoundation.org/donate/ Foundation Partnership Program URL: https://www.FreeBSDfoundation.org/ FreeBSD-foundation-partnership-program FreeBSD Journal URL: https://www.FreeBSDfoundation.org/journal/ Foundation News and Events URL: https://www.FreeBSDfoundation.org/ news-and-events/ Contact: Deb Goodkin The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to supporting and promoting the FreeBSD Project and community worldwide. Donat= ions =66rom individuals and corporations are used to fund and manage software development projects, conferences, and developer summits. We also provide travel grants to FreeBSD contributors, purchase and support hardware to imp= rove and maintain FreeBSD infrastructure, and provide resources to improve secur= ity, quality assurance, and release engineering efforts. We publish marketing material to promote, educate, and advocate for the FreeBSD Project, facilit= ate collaboration between commercial vendors and FreeBSD developers, and finall= y, represent the FreeBSD Project in executing contracts, license agreements, a= nd other legal arrangements that require a recognized legal entity. Here are some highlights of what we did to help FreeBSD last quarter: Fundraising Efforts Fundraising last quarter wasn=E2=80=99t as spectacular as we were hoping. B= ut, then again, people tend to take vacations during the summer months, which makes = it that more difficult for our funding requests to go through the management c= hain for approvals. So far this year we=E2=80=99ve raised $180,000 towards our $2,000,000 spend= ing budget. Why do we need so much money? Well, last year we decided to make more significant software contributions to FreeBSD. In order to do that, we had = to grow our team. We developed a technology roadmap based on input we were receiving from commercial users as well as market trends. Based on the road= map, we identified positions we needed to fill. This year we=E2=80=99ve hired three full-time software developers, one full= -time ARM kernel developer, and one project manager. We also are funding wifi work full-time and some other projects to help with FreeBSD on the desktop. You = can read about this effort to attract new users and contributors to the Project= in individual entries elsewhere in this status report. Our growth wasn=E2=80=99t just in our technology team, but in our advocacy = team too. Here we hired a marketing coordinator and technical writer to provide more educational and informational content. You=E2=80=99ll see in the Advocacy a= nd Education section below all the work we did to promote FreeBSD, provide community engagement, education opportunities, and informative content to help pave t= he path to getting started with FreeBSD. You=E2=80=99ll find out how we used your donations here and in individual e= ntries throughout this status report. We=E2=80=99re passionate about supporting you, the FreeBSD community, but w= e can=E2=80=99t do it without your financial support. Please consider making a donation to help us continue and increase our supp= ort for FreeBSD in 2021: https://www.FreeBSDfoundation.org/donate/. We also have the Partnership Program, to provide more benefits for our larg= er commercial donors. Find out more information at https:// www.FreeBSDfoundation.org/FreeBSD-foundation-partnership-program/ and share with your companies! OS Improvements During the third quarter, Foundation staff and grant recipients committed 4= 20 src tree changes, 24 ports tree changes, and 11 doc tree changes. This represents 38%, 48%, and 16% of src, port, and doc commits which identify a sponsor. You can read about Foundation-sponsored projects in individual quarterly re= port entries: =E2=80=A2 Base System OpenSSH Update =E2=80=A2 Fixes for msdosfs_rename VOP =E2=80=A2 Hole Punching =E2=80=A2 Improved amd64 UEFI boot =E2=80=A2 Intel Wireless driver support =E2=80=A2 LLDB Debugger Improvements =E2=80=A2 Microchip LAN743x mgb(4) Device Driver =E2=80=A2 OpenZFS RAIDZ Expansion update =E2=80=A2 Using OCF in WireGuard =E2=80=A2 syzkaller on FreeBSD Foundation-sponsored arm64 work: =E2=80=A2 Support for booting FreeBSD on the Arm Armv8-A AEM simulator =E2=80=A2 sha256 instruction support to libmd(4) on arm64 =E2=80=A2 Initial work to support RAS, PAC and BTI on arm64 Continuous Integration and Quality Assurance The Foundation provides a full-time staff member and funds projects to impr= ove continuous integration, automated testing, and overall quality assurance efforts for the FreeBSD project. See the separate Continuous Integration entry for details. Supporting FreeBSD Infrastructure The Foundation provides hardware and support for the Project. Last quarter,= we continued supporting the test cluster at Sentex, purchased a few needed dri= ves and spam filtering software for project email. We also set up a new and more efficient hardware request/purchase process for clusteradm to use. Partnerships and Commercial User Support We met virtually with corporate users and started travelling again in late = Q3 for some in-person meetings. The goals of the meetings are to facilitate collaboration between commercial users and FreeBSD developers. We also met = with companies to discuss their needs and share that information with the Projec= t. FreeBSD Advocacy and Education Much of our effort is dedicated to Project advocacy. This may involve highlighting interesting FreeBSD work, producing literature, attending even= ts, or giving presentations. The goal of the literature we produce is to teach people FreeBSD basics and help make their path to adoption or contribution easier. Other than attending and presenting at events, we encourage and help community members run their own FreeBSD events, give presentations, or staff FreeBSD tables. The FreeBSD Foundation sponsors many conferences, events, and summits around the globe. These events can be BSD-related, open source, or technology even= ts geared towards underrepresented groups. We support the FreeBSD-focused even= ts to help provide a venue for sharing knowledge, working together on projects, and facilitating collaboration between developers and commercial users. This all helps provide a healthy ecosystem. We support the non-FreeBSD events to promote and raise awareness of FreeBSD, to increase the use of FreeBSD in different applications, and to recruit more contributors to the Project. We finally made it back to our first in-person meeting with the Open Source Su= mmit in late September. We are also continuing to attend virtual events. In addi= tion to attending and planning virtual events, we are continually working on new training initiatives and updating our selection of how-to guides to facilit= ate getting more folks to try out FreeBSD. Check out some of the advocacy and education work we did last quarter: =E2=80=A2 Participated as an Industry Partner for USENIX ATC and OSD, Jul= y 14-16, 2021 =E2=80=A2 Participated as an Industry Partner for USENIX Security, August= 11-13, 2021 =E2=80=A2 Held a FreeBSD Friday: How to Track FreeBSD Using Git =E2=80=A2 Sponsored and gave a talk at OpenFest 2020 in Sofia, Bulgaria, = August 14-15, 2021 =E2=80=A2 Began planning for the November 2021 FreeBSD Vendor Summit to b= e held online, November 18-19 =E2=80=A2 Presented at APNIC on August 13-16, 2021 =E2=80=A2 Served as a Nonprofit In-Kind Sponsor of OSI=E2=80=99s POSI 202= 1, September 16, 2021 =E2=80=A2 Sponsored EuroBSDcon at the Silver Level. The event took place,= September 17-19, 2021 =E2=80=A2 Presented at Open Source Summit, in Seattle, WA, September 27-3= 0, 2021 =E2=80=A2 Committed to be a Bronze Sponsor for the OpenZFS Summit =E2=80=A2 New blog and video posts: =E2=96=A1 Status of Online Conference Software on FreeBSD =E2=96=A1 Meet the Summer 2021 Foundation Interns =E2=96=A1 A Look at Profiling: FreeBSD Sort =E2=96=A1 Meet the 2021 FreeBSD Google Summer of Code Students =E2=96=A1 A Co-op Term at the FreeBSD Foundation =E2=96=A1 Technology Roadmap =E2=80=A2 Devstyler Interview with Deb Goodkin =E2=80=A2 New Video How-to Guides on installing HelloSystem and installin= g GhostBSD =E2=80=A2 New Quick Start Guide on Printing on FreeBSD =E2=80=A2 Committed to be a Media Sponsor for All Things Open We help educate the world about FreeBSD by publishing the professionally produced FreeBSD Journal. As we mentioned previously, the FreeBSD Journal is now a free publication. Find out more and access the latest issues at https= :// www.FreeBSDfoundation.org/journal/. You can find out more about events we attended and upcoming events at https= :// www.FreeBSDfoundation.org/news-and-events/. Legal/FreeBSD IP The Foundation owns the FreeBSD trademarks, and it is our responsibility to protect them. We also provide legal support for the core team to investigate questions that arise. Go to https://www.FreeBSDfoundation.org to find more about how we support FreeBSD and how we can help you! =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 FreeBSD Release Engineering Team Links: FreeBSD 12.3-RELEASE schedule URL: https://www.freebsd.org/releases/12.3R/ schedule/ FreeBSD releases URL: https://download.freebsd.org/ftp/releases/ISO-IMAGES/ FreeBSD development snapshots URL: https://download.freebsd.org/ftp/snapsho= ts/ ISO-IMAGES/ Contact: FreeBSD Release Engineering Team, The FreeBSD Release Engineering Team is responsible for setting and publish= ing release schedules for official project releases of FreeBSD, announcing code freezes and maintaining the respective branches, among other things. Throughout the third quarter, several development snapshots builds were released for the main, stable/13, and stable/12 branches. More work had been done on various release-related tooling following the conversion from Subversion to Git. Additionally, the team had drafted the proposed schedule= for the upcoming 12.3-RELEASE. Sponsor: Rubicon Communications, LLC ("Netgate") Sponsor: The FreeBSD Foundation =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Cluster Administration Team Links: Cluster Administration Team members URL: https://www.freebsd.org/administra= tion /#t-clusteradm Contact: Cluster Administration Team FreeBSD Cluster Administration Team members are responsible for managing the machines the Project relies on to synchronise its distributed work and communications. In this quarter, the team has worked on the following: =E2=80=A2 Fixed www.FreeBSD.org mirror sync source =E2=80=A2 Improvements to GeoDNS for {download,ftp,pkg}.FreeBSD.org =E2=80=A2 Added IPv6 support =E2=80=A2 More optimal mirror selection in Asia =E2=80=A2 Finished installing a new mirror site in Japan, sponsored by Br= oadband Tower, Inc. =E2=80=A2 Full mirror site (ftp, pkg, git, doc, dns,=E2=80=A6=E2=80=8B) =E2=80=A2 The network and machine resources for this mirror are generousl= y sponsored by the Cloud and SDN Laboratory at BroadBand Tower, Inc., one of the Internet data center service providers in Tokyo, Japan, with 300+ Gbps international IP transit bandwidth =E2=80=A2 Improved the retention script used for the artifact server of C= I cluster to offer a mix of the latest artifacts but also a selection of older artif= acts for comparison, space permitting =E2=80=A2 Ongoing day to day cluster administration =E2=80=A2 Replacing failed disks =E2=80=A2 Babysitting pkgsync Work in progress: =E2=80=A2 Move pkg-master.nyi to new hardware =E2=80=A2 Improve the package building infrastructure =E2=80=A2 Review the service jails and service administrators operation =E2=80=A2 Setup powerpc pkgbuilder/ref/universal machines =E2=80=A2 Search for more providers that can fit the requirements for a g= eneric mirrored layout or a tiny mirror =E2=80=A2 Upgrading public-facing cluster services from 12.2-STABLE to 13= =2E0-STABLE =E2=80=A2 Working with doceng@ to improve https://www.freebsd.org and htt= ps:// docs.freebsd.org =E2=80=A2 Improve the web service architecture =E2=80=A2 Improve the cluster backup plan =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Continuous Integration Links: FreeBSD Jenkins Instance URL: https://ci.FreeBSD.org FreeBSD CI artifact archive URL: https://artifact.ci.FreeBSD.org FreeBSD Jenkins wiki URL: https://wiki.freebsd.org/Jenkins Hosted CI wiki URL: https://wiki.freebsd.org/HostedCI 3rd Party Software CI URL: https://wiki.freebsd.org/3rdPartySoftwareCI Tickets related to freebsd-testing@ URL: https://preview.tinyurl.com/y9maau= wg FreeBSD CI Repository URL: https://github.com/freebsd/freebsd-ci+ dev-ci Mailing List URL: https://lists.freebsd.org/subscription/dev-ci Contact: Jenkins Admin Contact: Li-Wen Hsu Contact: freebsd-testing Mailing List Contact: IRC #freebsd-ci channel on EFNet The FreeBSD CI team maintains the continuous integration system of the Free= BSD project. The CI system checks the committed changes can be successfully bui= lt, then performs various tests and analysis over the newly built results. The artifacts from those builds are archived in the artifact server for further testing and debugging needs. The CI team members examine the failing builds= and unstable tests and work with the experts in that area to fix the code or ad= just test infrastructure. During the third quarter of 2021, we continued working with the contributors and developers in the project to fulfil their testing needs and also keep collaborating with external projects and companies to improve their products and FreeBSD. Important changes: =E2=80=A2 The results of FreeBSD-main-amd64-build and FreeBSD-main-amd64-= test jobs are sent to the dev-ci mailing list New jobs: =E2=80=A2 Run tests with KASAN enabled for main on amd64 =E2=80=A2 Run tests with KMSAN enabled for main on amd64 Retired jobs: =E2=80=A2 The jobs for stable/11 branch were removed after September 30th= per FreeBSD 11.4 end-of-life Work in progress and open tasks: =E2=80=A2 Designing and implementing pre-commit CI building and testing (= to support the workflow working group) =E2=80=A2 Designing and implementing use of CI cluster to build release a= rtifacts as release engineering does =E2=80=A2 Collecting and sorting CI tasks and ideas here =E2=80=A2 Testing and merging pull requests in the FreeBSD-ci repo =E2=80=A2 Reducing the procedures of CI/test environment setting up for c= ontributors and developers =E2=80=A2 Setting up the CI stage environment and putting the experimenta= l jobs on it =E2=80=A2 Setting up public network access for the VM guest running tests =E2=80=A2 Implementing using bare metal hardware to run test suites =E2=80=A2 Adding drm ports building tests against -CURRENT =E2=80=A2 Planning to run ztest tests =E2=80=A2 Adding more external toolchain related jobs =E2=80=A2 Improving maturity of the hardware lab and adding more hardware= under test =E2=80=A2 Helping more software get FreeBSD support in its CI pipeline (W= iki pages: 3rdPartySoftwareCI, HostedCI) =E2=80=A2 Working with hosted CI providers to have better FreeBSD support Please see freebsd-testing@ related tickets for more WIP information, and d= on=E2=80=99t hesitate to join the effort! Sponsor: The FreeBSD Foundation =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Ports Collection Links: About FreeBSD Ports URL: https://www.FreeBSD.org/ports/ Contributing to Ports URL: https://docs.freebsd.org/en/articles/contributin= g/ ports-contributing/ FreeBSD Ports Monitoring URL: http://portsmon.freebsd.org/ Ports Management Team URL: https://www.freebsd.org/portmgr/ Ports Tarball URL: http://ftp.freebsd.org/pub/FreeBSD/ports/ports/ Contact: Ren=C3=A9 Ladan Contact: FreeBSD Ports Management Team The Ports Management Team is responsible for overseeing the overall directi= on of the Ports Tree, building packages, and personnel matters. Below is what happened in the last quarter. There are currently 46,500 ports in the Ports Tree according to FreshPorts.= The open PR count for ports is currently 2,588, of which 608 are unassigned. The last quarter saw 9,833 commits on the "main" branch by 148 committers and 7= 43 commits on the quarterly branch by 54 committers. Compared to 2021q2, activ= ity mostly remained the same, the PR counts increased a bit but there were also more commits to the quarterly branch. During 2021q3, we welcomed Daniel Engberg (diizzy@) and Yasuhiro Kimura (yasu@), and we said goodbye to culot@ and grog@ after their commit bits we= re taken in for safekeeping. Two new uses were introduced: angr to help with testing Python-related ports and mlt to help depending on mlt, a multimedia framework for TV broadcastin= g. Ruby saw minor updates for the 2.6, 2.7, and 3.0 series. The "big" packages were also updated: pkg to 1.17.2, Firefox to 93.0, and Chromium to 92.0.4515.159. As usual, exp-runs were performed by antoine@ but only those of July were recorded. There were 5 exp-runs to test adding CLOCK_\*_COARSE compatibility definitions for Linux and to test ports updates. Furthermore, rene@ gave a presentation "portmgr: behind the scenes" at EuroBSDCon 2021 about how portmgr came to be and its daily activities. =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Documentation Engineering Team Links: FreeBSD Documentation Project URL: https://www.freebsd.org/docproj/ FreeBSD Documentation Project Primer for New Contributors URL: link:https:// docs.freebsd.org/en/books/fdp-primer/ Documentation Engineering Team URL: https://www.freebsd.org/administration/# t-doceng Contact: FreeBSD Doceng Team The doceng@ team is a body to handle some of the meta-project issues associ= ated with the FreeBSD Documentation Project; for more information, see FreeBSD Doceng Team Charter. During the last quarter, Philip Paeps (philip@) and Li-Wen Hsu (lwhsu@), already src and ports committers, were granted documentation commit bits, b= oth now have all commit bit types. Ed Maste (emaste@) who is already a src committer was granted a documentation commit bit. We also said goodbye to Murray Stokely (murray@), G=C3=A1bor K=C3=B6vesd=C3=A1n (gabor@), Warren Bl= ock (wblock@), and Sevan Janiyan (sevan@). G=C3=A1bor K=C3=B6vesd=C3=A1n (gabor@) and Warren B= lock (wblock@) are now former members of the doceng@ team; we thank them for their years of service. Implicit (blanket) approvals were documented in the Committers Guide. That = does not cover all cases, but doceng@ encourages any FreeBSD committer from ports and src to contribute to the doc tree. All ports/packages misc/freebsd-doc-* were updated. They have the entire documentation set from the FreeBSD Documentation Project, like Handbook, FA= Q, articles, and more. =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 FreeBSD Website Revamp - WebApps working group Contact: Sergio Carlavilla Working group in charge of creating the new FreeBSD Documentation Portal and redesigning the FreeBSD main website and its components. FreeBSD developers= can follow and join the working group on the FreeBSD Slack channel #wg-www21. T= he work will be divided into four phases: 1. Redesign of the Documentation Portal Create a new design, responsive and with global search. (Work in progre= ss, almost complete) 2. Redesign of the Manual Pages on web Scripts to generate the HTML pages using mandoc. (Work in progress) 3. Redesign of the Ports page on web Ports scripts to create an applications portal. (Not started) 4. Redesign of the FreeBSD main website New design, responsive and dark theme. (Not started) =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Projects Projects that span multiple categories, from the kernel and userspace to the Ports Collection or external projects. Boot Performance Improvements Links: Wiki page URL: https://wiki.freebsd.org/BootTime OS boot time comparison URL: https://www.daemonology.net/blog/ 2021-08-12-EC2-boot-time-benchmarking.html Contact: Colin Percival Colin Percival is coordinating an effort to speed up the FreeBSD boot proce= ss. For benchmarking purposes, he is using an EC2 c5.xlarge instance as a refer= ence platform and is measuring the time between when the virtual machine enters = the EC2 "running" state and when it is possible to SSH into the instance. This work started in 2017, leading to a conference presentation, "Profiling= the FreeBSD kernel boot", and quickly yielded roughly 4850 ms of improvements (starting from a baseline of about 30 seconds). Since June, another roughly 9790 ms of time has been shaved off the boot process, taking it down to approximately 15 seconds. There is still more wo= rk to be done; in particular, while the loader and kernel have been profiled, = the TSLOG system Colin is using does not currently support userland profiling. Issues are listed on the wiki page as they are identified; the wiki page al= so has instructions for performing profiling. Users are encouraged to profile = the boot process on their own systems, in case they experience delays which don= =E2=80=99t show up on the system Colin is using for testing. This work is supported by Colin=E2=80=99s FreeBSD/EC2 Patreon. Sponsor: https://www.patreon.com/cperciva =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Git Migration Working Group Links: Git for FreeBSD development wiki URL: https://wiki.freebsd.org/Git Git transition wiki URL: https://wiki.freebsd.org/GitTransition doc git repo web URL: https://cgit.FreeBSD.org/doc ports git repo web URL: https://cgit.FreeBSD.org/ports src (base system) git repo web URL: https://cgit.FreeBSD.org/src Committers guide Git primer URL: https://docs.freebsd.org/en/articles/ committers-guide/#git-primer Handbook Using Git appendix URL: https://docs.freebsd.org/en/books/handbook/ mirrors/#git Game of Trees URL: http://gameoftrees.org/ gitup URL: https://github.com/johnmehr/gitup Contact: Li-Wen Hsu Contact: Warner Losh Contact: Ed Maste Contact: FreeBSD-git mailing list Contact: #gitcvt channel on EFnet With the end of the third quarter of 2021, we are approaching the one-year anniversary of the transition of the doc and src repositories from Subversi= on to Git. The 2021Q4 branch of the ports tree has been created, the third quarterly branch created using Git. During 2021Q3, repository hooks have been refined as problems were discover= ed and a few lingering references to Subversion were updated in the Committer= =E2=80=99s Guide. The Working Group continues to track progress on two permissively-licensed Git-compatible tools: Gitup and Game of Trees. Gitup = is a small, dependency-free tool to clone and update git repositories. It is used only to keep a local tree up-to-date, and has no support for local commits. Game of Trees is a version control client that is compatible with Git repositories. It provides a user interface and workflow that is distinct fr= om that of Git. It is in no way intended to be a drop-in replacement for Git, = but can be used to develop software maintained in a Git repository. Gitup and Game of Trees are currently available as ports and packages. Futu= re work will evaluate them as candidates for the base system. Remaining work includes continuing with refinements to Git documentation in= the Handbook, committing new and updated repository hooks for managing branch content and commit mail, and surveying pre-commit hooks to use the CI clust= er to build release artifacts. Priorities have been set for the next working g= roup tasked with refining our Git workflow. The first meeting will be in mid-October. Sponsor: The FreeBSD Foundation (in part) =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 LLDB Debugger Improvements Links: Moritz Systems Project Description URL: link:https://www.moritz.systems/blo= g/ freebsd-kgdb-support-in-lldb/ Progress Report 1 URL: https://www.moritz.systems/blog/ improving-gdb-protocol-compatibility-in-lldb/ Progress Report 2 URL: https://www.moritz.systems/blog/ improving-gdb-register-model-compatibility-in-lldb/ Git Repository URL: https://github.com/moritz-systems/llvm-project Contact: Kamil Rytarowski Contact: Micha=C5=82 G=C3=B3rny According to the upstream description, "LLDB is a next generation, high-performance debugger. It is built as a set of reusable components which highly leverage existing libraries in the larger LLVM Project, such as the Clang expression parser and LLVM disassembler." FreeBSD includes LLDB in the base system. At present, it has some limitatio= ns compared to the GNU GDB debugger, and does not yet provide a complete replacement. This project spans from July 2021 to January 2022 and aims to = make LLDB suitable for debugging FreeBSD kernels. The work so far was focused on improving the compatibility between LLDB and other servers implementing the GDB Remote Protocol. Multiple bugs were fixed that limited LLDB=E2=80=99s functionality when interfacing with GNU GDB=E2= =80=99s gdbserver and QEMU=E2=80=99s gdbserver implementations. Support for debugging executables= running on gdbserver architectures other than amd64 was fixed, and was explicitly test= ed on arm64 and i386. This effort also resulted in adjusting lldb-server=E2=80=99s implementation= to conform better to the standard set by GDB. Thanks to these improvements, the LLDB client needs to provide less divergent code paths depending on the server u= sed, and the single code path used is tested more thoroughly. The work also involved improvements to the LLDB API and code deduplication, that result in reducing the maintenance effort and lowering the bar for fut= ure contributions. The test coverage was improved, increasing the likelihood th= at any future problems will be detected before they affect users. The introduced changes are expected to be shipped with LLDB 14.0. Sponsor: The FreeBSD Foundation =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Linux compatibility layer update Contact: Dmitry Chagin, Contact: Edward Tomasz Napierala, The goal of this project is to improve FreeBSD=E2=80=99s ability to execute= unmodified Linux binaries. The current support status of specific Linux applications is being tracked on the Linux app status Wiki page. The vdso(7) implementation was reworked. The futex(2) support was overhaule= d to make use of FreeBSD=E2=80=99s native umtx mechanism. It now supports priority-inheritance futexes, in addition to fixing several bugs. The rt_sigsuspend(2) and sigaltstack(2) syscalls are now supported on ARM64. The faccessat2(2), clone3(2) system calls are now implemented. The CLONE_CLEAR_RESETHAND option is now supported. The prctl(2) syscall now supports PR_SET_NO_NEW_PRIVS. The ptrace(2) syscall now supports PTRACE_GET_SYSCALL_INFO, which is a prerequisite to support newer strace(1) versions. There is ongoing work to make Linuxulator on arm64 on par with the amd64 on= e; right now it=E2=80=99s good enough for development work. Sponsor: EPSRC (Edward=E2=80=99s work) =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Sound mixer improvements Links: GSoC 2021 project article URL: https://wiki.freebsd.org/ SummerOfCode2021Projects/SoundMixerImprovements Contact: Christos Margiolis Contact: Hans Petter Selasky This project is an attempt to improve the capabilities of the OSS sound mix= er on FreeBSD. This means a new mixer(3) library, a complete rewrite of mixer(= 8), and updates to sound(4). Development started during Google Summer of Code 2021, but will likely cont= inue independently. Applied changes: =E2=80=A2 Implement OSS mixer (un)muting ioctls in sound(4) (commit 0f8da= fb45859). =E2=80=A2 Implement playback/recording mode sysctl (commit ed2196e5df0c). =E2=80=A2 Implement mixer(3) and new version of mixer(8) (commit 903873ce= 1560). Patches, comments and discussion are all welcome. =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Base System OpenSSH Update Links: OpenSSH URL: https://www.openssh.com/ Announcement to freebsd-security@ URL: https://lists.freebsd.org/pipermail/ freebsd-security/2021-September/010473.html Contact: Ed Maste OpenSSH, a suite of remote login and file transfer tools, was updated from version 7.9p1 to 8.7p1 in the FreeBSD base system. FreeBSD base OpenSSH includes a number of local bug fixes, configuration changes, and small features. As part of the work for this update, I submitt= ed some of these upstream and am preparing to do the same with the remaining changes. OpenSSH now supports FIDO/U2F devices, although additional work is required= to enable this in the FreeBSD base system. This includes importing a pair of dependencies: libcbor, and libfido2. Within the next couple of months I exp= ect to import these, enable FIDO/U2F support, and update to OpenSSH version 8.8= p1. NOTE: OpenSSH 8.8p1 will disable the ssh-rsa signature scheme by default, so some additional work is required for this next update. For more information please see the Important note for future FreeBSD base system OpenSSH update mailing list post. Sponsor: The FreeBSD Foundation =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Kernel Updates to kernel subsystems/features, driver support, filesystems, and mor= e. Arm64 improvements Links: Pointer Authentication Review URL: https://reviews.freebsd.org/D31261 Contact: Andrew Turner FreeBSD has been ported to the Arm Armv8-A Architecture Envelope Model (AEM= ), an Armv8 architecture simulator. AEM is useful for Armv8 development and testing, because it supports different configurations, including the abilit= y to enable or disable different extensions. As part of this work, the virtio le= gacy pci driver was fixed and ACPI resource parsing code was updated to work with the ACPI tables that the model provides. Libmd(4) has been updated to use the sha256 instructions when available. Th= is can lead to a large performance improvement when these instructions are available. For example, on the Apple M1 under a hypervisor, the time to calculate the sha256 of a 1GB file has decreased from a median of 3.46s to 0.5s. Using an ifunc (indirect function) in a static binary is now supported. This will allow different implementations of a function to be used depending on which hardware it is being run on. Using an ifunc in dynamic binaries was already supported. The arm64 ID register definitions have been updated to match the Armv8.5 specification and other special registers have been updated to the Armv8.7 spec. There have been numerous cleanups in decoding the arm64 ID registers in the kernel to ensure we provide a consistent view of the hardware to userspace = if it reads the emulated ID register directly, or uses an ELF HWCAP value. There has been early work on supporting the Arm Reliability, Availability, = and Serviceability (RAS) extension. The external aborts it may use are now hand= led in the kernel. Support for the Arm Pointer Authentication extension is under review. This extension allows programs to use new instructions that add a hash to unused bits in a code or data pointer. They can then later check the hash in a way that will either, depending on the CPU, create an invalid pointer or raise = an exception. This can be used to protect the function return address from bei= ng overwritten, making Return Oriented Programming (ROP) attacks difficult. The change supports both userspace and kernel pointer authentication. It is wai= ting on debugger changes to be sent upstream so lldb can clear the hash when nee= ded, e.g., in a stack trace. Work has started on supporting the Branch Target Identification extension. = This adds a flag to the page tables to disallow unintended targets from some bra= nch instructions. When a CPU branches to a new location from a register, the CPU will check if the instruction is correct. This will protect, e.g., against a function pointer being called when it points to the middle of a function. T= he kernel has been built with this feature enabled and tested in the Arm AEM. Sponsor: The FreeBSD Foundation =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 FreeBSD on Microsoft HyperV and Azure Links: Microsoft Azure article on FreeBSD wiki URL: https://wiki.freebsd.org/ MicrosoftAzure Microsoft HyperV article on FreeBSD wiki URL: https://wiki.freebsd.org/Hype= rV Contact: Microsoft FreeBSD Integration Services Team Contact: freebsd-cloud Mailing List Contact: The FreeBSD Azure Release Engineering Team Contact: Wei Hu Contact: Li-Wen Hsu The 13.0-RELEASE image on Azure Marketplace has been published. The changes for building official images on Azure Marketplace, which fulfill the requirements of Azure and FreeBSD cloud images like disk layout and UEFI for Gen2 VM, along with some minor improvements like configurations to spee= d up booting, have been imported. Above tasks are sponsored by The FreeBSD Foundation, with resources provide= d by Microsoft. Microsoft Azure Network Adapter (MANA) is the new network adapter from Microsoft which will be available in the Azure public cloud. It provides SR= -IOV NIC as a Virtual Function (VF) to guest OS running on Hyper-V. Wei has been working on its driver for a while and committed in this quarter. This task is sponsored by Microsoft. Work in progress tasks: =E2=80=A2 Build and publish gen2 vm image to Azure Marketplace =E2=80=A2 Build and publish ZFS-based image to Azure Marketplace =E2=80=A2 Upstream local modifications of Azure agent =E2=80=A2 Update Azure agent port to the latest version Open tasks: =E2=80=A2 Update FreeBSD related doc at https://docs.microsoft.com =E2=80=A2 Support FreeBSD in Azure Pipelines =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Improved amd64 UEFI boot Contact: Konstantin Belousov UEFI BIOS on PC provides a much richer and more streamlined environment for pre-OS programs, but also imposes some requirements on the programs that are executed there, OS loaders in particular. One such requirement is that the loader must coordinate its memory use with the BIOS, only using memory that= was allocated for it. Even after the loader handoff to the operating system ker= nel, there are still some memory regions that are reserved by the BIOS for diffe= rent reasons. Examples are runtime services code and data. On the other hand, legacy/CSM BIOS boot works with memory completely differently; there are known memory regions that hold BIOS data and must be avoided. Otherwise, the memory is considered free for loader and OS to use.= (Of course it is not that straightforward, the definition of known regions is u= p to the vendor and there are a lot of workarounds there.) The BIOS boot puts the kernel and preloaded data (like modules, memory disk, CPU microcode update etc) at the contigous physical memory block starting at 2M. This algorithm goes back to how the i386 kernel boots. Also, when preparing to pass control to the kernel, the loader creates very special temporary mappings, where the low 1G of physical address space is mapped 1:1 into virtual address space, and then repeated for each 1G until = the virtual memory end. The kernel knows about its physical location and the temporary mapping, and constructs kernel page tables assuming that the phys= ical address of the text is at 2M. This mechanism of loader to kernel handoff was left unchanged when the load= er gained support for the UEFI environment. The loader prepares kernel and auxiliary preload data in a so-called staging area while UEFI boot services= are active, and after EFI_BOOT_SERVICES.ExitBootServices(), the temporary mappi= ng is activated and the staging area is copied at 2M. An advantage at that time was that no changes to the kernel were needed. But there are issues; the biggest is that memory at 2M might be not free for re= use. For instance, BIOS runtime code or data might be located there. Or there mi= ght be no memory at 2M at all. Or trampoline page table or code, or even some p= arts of the staging area overlapping with the 2M region where staging area is copied. The outcome was a hard to diagnose boot time failure, typically a h= ard hang when the loader started the kernel. Another limitation is the 1G transient mapping, which due to copying means = that the total size of preloaded data cannot exceed around 400M for everything, including kernel, memory disks, and anything else. Also the code to grow the staging area on demand was quite unflexible, only able to grow the staging = area in place. The work described in this report allows the UEFI loader on amd64 to start = the kernel from the staging area without copying. Kernel assumptions about the hand-off were explicitly identified and documented. The kernel only requires the staging area to be located below 4G together with the trampoline page t= able (this is a consequence of CPU architecture requiring 32bit protected mode to enter long mode), be 2M aligned and the whole low 4G mapped 1:1 at hand-off. The kernel computes its physical address and builds kernel page tables accordingly. Making the kernel boot with staging area in-place required identifying all places where the amd64 kernel had a dependency on its physical location. The most complicated part was application processors startup, which required rewriting initialization code, which we were able to streamline as result. = In particular, when an AP enters paging mode, it does so straight into the cor= rect kernel page table, without loading intermediate trampoline page table. The updated loader automatically detects if the loaded kernel can handle in-place staging area ('non-copying mode'). If needed, this can be overridd= en with the loader=E2=80=99s copy_staging command. For instance, 'copy_staging= enable' tells the loader to unconditionally copy the staging area to 2M regardless = of kernel capabilities (default is 'copy_staging auto'). Also, the code to grow the staging area was made much more robust, allowing it to grow without hand-tuning and recompiling the loader. Sponsor: The FreeBSD Foundation =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 ENA FreeBSD Driver Update Links: ENA README URL: https://github.com/amzn/amzn-drivers/blob/master/kernel/fbs= d/ ena/README Contact: Michal Krawczyk Contact: Artur Rojek Contact: Marcin Wojtas ENA (Elastic Network Adapter) is the smart NIC available in the virtualized environment of Amazon Web Services (AWS). The ENA driver supports multiple transmit and receive queues and can handle up to 100 Gb/s of network traffi= c, depending on the instance type on which it is used. Completed since the last update: =E2=80=A2 Update ENA driver to v2.4.1 =E2=80=A2 Introduce full kernel RSS API support =E2=80=A2 Allow reconfiguration of the RSS indirection table and hash key =E2=80=A2 Support netmap on the c6gn/m6i AWS instance types =E2=80=A2 Fix race between detach function and some of the procedural sys= ctl nodes =E2=80=A2 Add new statistics counters =E2=80=A2 Improve safety of the reset handling routine =E2=80=A2 Avoid mbuf collapse on Tx path for the LLQ condition Work in progress: =E2=80=A2 Prototype the driver port to the iflib framework =E2=80=A2 MFC the ENA v2.4.1 driver to the FreeBSD 11/12/13-STABLE branch= es =E2=80=A2 Add IPv6 L4 checksum offload support to the driver Sponsor: Amazon.com Inc =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Hole-punching support on FreeBSD Links: Commit 0dc332bff200 URL: https://cgit.freebsd.org/src/commit/?id=3D 0dc332bff200c940edc36c4715b629a2e1e9f9ae Commit 9e202d036dd6 URL: https://cgit.freebsd.org/src/commit/?id=3D 9e202d036dd6f38ce0f578aa2086ebc358315bab Commit 5ee2c35751ef URL: https://cgit.freebsd.org/src/commit/?id=3D 5ee2c35751ef5d131222423bf3e25073f997c337 OpenZFS Pull Request #12458 URL: https://github.com/openzfs/zfs/pull/12458 Contact: Ka Ho Ng Hole-punching functionality allows turning a contiguous range of bytes into= a hole for a given file. File systems supporting hole-punching may deallocate= the file system space from the given file. One use case for the functionality i= s to convert TRIM/UNMAP/DEALLOCATE requests from a virtual machine guest to hole-punching calls on the host=E2=80=99s side, thus allows reclamation of = file system space when unneeded by the guest. A set of APIs and KPIs are added that developers can call to invoke hole-punching on a given file, if the underlying file system exposes that functionality. For file systems not supporting hole-punching there is a fallback implementation in the kernel which does zero-filling instead. Besi= des the APIs and KPIs addition, the truncate(1) utility is expanded by adding a= -d flag to support invoking hole-punching as well. At the time of writing this report, OpenZFS and tmpfs are the file systems = that support hole-punching. Sponsor: The FreeBSD Foundation =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Intel Networking on FreeBSD Links: DPDK URL: https://github.com/DPDK/dpdk/tree/main/drivers/net Contact: Intel Contact: Kevin Bowling lem(4)/em(4)/igb(4) and ix(4) were updated with various common code improvements obtained from DPDK. There have also been several bug fixes from the community: =E2=80=A2 lem(4)/em(4)/igb(4) changes =E2=80=A2 ix(4) changes Intel has updated ixl(4) with various improvements: =E2=80=A2 ixl(4): Fix 2.5 and 5G speeds reporting and update shared code =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Intel Wireless driver support Links: iwlwifi status FreeBSD wiki page URL: https://wiki.freebsd.org/WiFi/Iwlwifi Contact: Bjoern A. Zeeb The Intel Wireless driver update project aims to bring support for newer chipsets along with mac80211 LinuxKPI compat code. The dual-licensed Intel driver code was ported in the past for the iwm(4) native driver; using the LinuxKPI compat framework allows us to use the driver directly, with only v= ery minor modificationsi that we hope will be incorporated into the original driver. After the initial snapshot at the end of June, another snapshot was release= d in early September. Thank you to everyone testing on CURRENT and reporting bac= k. While we keep updating the driver and all the compat code needed for that, = the focus now is on stability and adding support for newer 802.11 standards. The driver is currently set to 11a/b/g and 11n will be next before we look at 1= 1ac. We are currently trying to get as much into FreeBSD as is possible and makes sense. Full integration into FreeBSD main is waiting for a policy update. For the latest state of the development or code, please follow the referenc= ed wiki page or the freebsd-wireless mailing list. We expect to release another snapshot soon and hope to also support stable/13 again with that one. Sponsor: The FreeBSD Foundation =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Microchip LAN743x mgb(4) Device Driver Links: Commit adding mgb(4) driver URL: https://cgit.freebsd.org/src/commit/?id=3D 8890ab7758b8a03a68f3fe596f6a5446921631a5 Commit connecting mgb(4) module to the build URL: https://cgit.freebsd.org/= src/ commit/?id=3D543df609072fe49079c36d6bee510e1645edde3a Contact: Ed Maste Microchip=E2=80=99s LAN7430 and LAN7431 are PCIe Gigabit Ethernet interface= ICs, with an integrated PHY (LAN7430) or RGMII interface (LAN7431). In 2019 Gerald ND Aryeetey, a FreeBSD Foundation intern, developed the mgb(= 4) driver for these devices. It was added to the tree but not yet connected to= the build. Since it was committed a number of sweeping iflib changes were made, which included updates to mgb(4). I have addressed some outstanding issues in the driver, and have now added = the module to the build. It is available in -CURRENT snapshot images. The drive= r is functional, although some additional work is still needed. In particular, configuration of the Receive Filtering Engine, and VLAN and RX/TX checksum offload are required. Caveats and notes are described in the man page. I am very interested in feedback and test results. Sponsor: The FreeBSD Foundation =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Fixes for msdosfs_rename VOP Contact: Konstantin Belousov Contact: Peter Holm Our msdosfs(5) implementation is old code and has a relatively large legacy cost. In particular, even though it got fine-grained locking and miscellane= ous bugfixes over time, sometimes a serious issue is found in it. Recently trasz@ found that msdosfs rename can be easily deadlocked. Further examination of rename code revealed a lot of issues with locking, potential= use after free, and filesystem structure corruption. As part of the update, locking in the msdosfs rename code was reworked. We = need to lock up to four vnodes, and check one path to ensure that rename does not create circular parent relations between directories. For that, the locking procedure was copied from UFS rename, where all vnodes except the first are locked in try-mode. Lockless relockup was added to msdosfs and the directory path checker was changed to non-blocking mode. During this work, all known issues were fixed and msdosfs passes the enhanc= ed stress2 suite of tests. Sponsor: The FreeBSD Foundation (kib=E2=80=99s contributions) =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 OpenZFS RAIDZ Expansion update Links: OpenZFS Pull Request URL: https://github.com/openzfs/zfs/pull/12225 video from 2021 FreeBSD Developer Summit URL: https://www.youtube.com/watch= ?v=3D yF2KgQGmUic slides from 2021 FreeBSD Developer Summit URL: https://docs.google.com/ presentation/d/1FeQgEwChrtNQBHfWSNsPK3Y53O5BnPh3Cz5nRa5GAQY/view news article from Ars Technica URL: https://arstechnica.com/gadgets/2021/06/ raidz-expansion-code-lands-in-openzfs-master/ Contact: Matthew Ahrens Project This feature allows disks to be added one at a time to a RAID-Z group, expanding its capacity incrementally. This feature is especially useful for small pools (typically with only one RAID-Z group), where there isn=E2=80= =99t sufficient hardware to add capacity by adding a whole new RAID-Z group (typically doubling the number of disks). FreeBSD=E2=80=99s ZFS implementation comes from the OpenZFS project. This w= ork will be integrated into the OpenZFS repo, with support for FreeBSD and Linux. Status The work is functionally complete, and a pull request has been opened. Remaining Work Remaining work includes some code cleanup, design documentation, addressing test failures. We also need to solicit code reviewers and respond to code review feedback. Sponsor: The FreeBSD Foundation =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 RFC1191 support in IPSEC tunnels Contact: Wojciech Macek The Semihalf team has been working on providing support for RFC1191 in IPSEC tunnels. This allows the PMTUD algorithm to dynamically discover the maximum path MTU the tunnel can handle and update tunnel parameters accordingly. As= a result, a better performance can be observed as there is no need for packet fragmentation. The newly introduced rework includes: =E2=80=A2 NEEDFRAG support for IPSEC commit d9d59bb1af1 =E2=80=A2 PMTUD for IPv4 IPSEC over IPv6 link commit 4f3376951d70 =E2=80=A2 Security fix as per RFC1191 specs commit b4220bf387e6 =E2=80=A2 PMTUD support for IPv6 tunnel commit 9dfc8606eb80 Sponsor: Stormshield =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Realtek rtw88 support Links: Rtw88 status FreeBSD wiki page URL: https://wiki.freebsd.org/WiFi/Rtw88 Contact: Bjoern A. Zeeb This project aims to bring in support for Realtek=E2=80=99s rtw88 chips. With the growing LinuxKPI compatibility code an initial port of the Realtek rtw88 driver was done. This gives us support for a second driver after Intel and helps to validate and grow the LinuxKPI code base. Changes and overall = time needed to get the driver compiling were very little. At the moment we are seeing DMA issues which prevent most people from loading firmware or using = the driver later on. Thanks to everyone who was brave enough to give this initial code drop a tr= y. While this is a leasure time project we would love to better support Realtek wireless devices and will keep working on this. For the latest state of the development or code, please follow the referenc= ed wiki page or the freebsd-wireless mailing list. =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Marvell SDHCI improvements and ACPI support Contact: Bartlomiej Grzesik Contact: Marcin Wojtas The Semihalf team was working on the ACPI support for the Marvell Octeon TX2 CN913x and Armada 7k/8k SoCs' SD/MMC controller, as well as extending and improving the related generic parts of the FreeBSD OS. Applied changes: =E2=80=A2 Add ACPI_GET_PROPERTY to access Device Specific Data (DSD) comm= it b91fc6c43a81 =E2=80=A2 Introduce generic device_get_property/device_has_property, whic= h allow to obtain DT/ACPI properties with the same generic code commit 3f9a00e3b577 =E2=80=A2 Switch mmc_helper to device_* API, to access the controller des= cription from DT/ACPI in a unified way commit 8a8166e5bcfb =E2=80=A2 Add ACPI support for the sdhci_xenon driver commit d78e464d2330= and commit adbce5ff747b =E2=80=A2 Apply other minor improvements and fixes related to properties = parsing in core mmc code and sdhci_xenon driver. Sponsor: Semihalf =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Stack gap handling improvements Contact: Dawid Gorecki Contact: Marcin Wojtas Stack gap is a feature that randomizes the stack address by creating a rand= om sized gap between the top of stack and strings area. The current implementa= tion of this mitigation can cause issues for some applications e.g. Firefox ( PR239873). Until now the only workaround for this problem was disabling the stack gap for those programs, as is done for ntpd. Semihalf team has been working on fixing the root causes of stack gap relat= ed problems. One of the issues could be observed when setting the stack limit to a low v= alue with setrlimit(2). Since the stack gap size can be significant, and the pro= cess had no knowledge of how large the stack gap is, this caused programs to abo= rt immediately after returning from a syscall due to the stack extending past = the limit. The fix involves increasing the soft resource limit (rlim_cur) by the size of stack gap during the setrlimit(2) call. This fixes the issue with n= tpd without disabling the stack gap entirely. The patch is currently under revi= ew. The second identified problem is related to the way the thread stacks are handled. Thread stacks are calculated using the kern.usrstack sysctl, which should point to the beginning of the stack. This sysctl returned a constant value depending on the ABI and the presence of the stack gap was ignored. A= new sysctl was introduced, and the thread library was updated to use it accordingly. Patches D31897 and D31898 are currently under discussion. These fix the issues with Firefox and Thunderbird not starting with the stack gap feature enabled. Sponsor: Stormshield =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 syzkaller on FreeBSD Contact: Mark Johnston Contact: Michael Tuexen < tuexen@FreeBSD.org> syzkaller is a coverage-guided operating system kernel fuzzer. See the syzkaller entry in the 2019q1 quarterly report for an introduction to syzkaller. In the past quarter we made a concerted effort to shrink the backlog of rep= orts =66rom the public syzbot instance. A number of long-standing locking bugs i= n the socket subsystem have been fixed, and the SCTP protocol implementation has = seen many bug fixes as well. Beyond that, many bugs in various other kernel subsystems have been fixed and the backlog has become substantially smaller over the past quarter. As a direct result of this effort, we have been able= to identify regressions more easily and fix bugs closer to the time of introduction. Work is still ongoing to further shrink the backlog. KASAN (Kernel Address SANitizer) was enabled in the default kernel configuration used by syzbot, which has greatly enhanced our ability to root-cause and fix kernel bugs. See the kernel-sanitizers entry in the 2021= q2 quarterly report for an introduction to KASAN and KMSAN. KASAN helps ensure that memory safety bugs manifest more deterministically, improving syzkalle= r=E2=80=99s ability to find reproducers and simplifying debugging. A KMSAN (Kernel Memory SANitizer) port was committed to FreeBSD=E2=80=99s m= ain branch in August. Some initial work has been done to make it usable by syzkaller (mainly, kcov(4) required several small modifications to work in a KMSAN-enabled kernel), and a number of bugs were found and fixed using priv= ate syzkaller instances. Goals for the next several months include: =E2=80=A2 Addition of a KMSAN target in syzbot. =E2=80=A2 Further reduction in the syzbot backlog. =E2=80=A2 Upstreaming syzkaller patches to support fuzzing of the Linuxul= ator. =E2=80=A2 Fuzzing using ZFS-based VM images. Sponsor: The FreeBSD Foundation =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Using OCF in WireGuard Contact: John Baldwin I have looked at updating the data path crypto in the upstream WireGuard dr= iver to use the in-kernel OpenCrypto Framework for the data path. Data packets s= ent over a WireGuard tunnel are encrypted with the Chacha20-Poly1305 AEAD ciphe= r. Unlike TLS and IPsec, Wireguard uses an 8 byte nonce rather than a 12 byte nonce with this cipher. To date, most of the work has focused on updating OCF to better support multiple nonce (and tag/MAC) lengths for a given cipher. I resurrected an o= ld branch I had previously started on for this aimed at supporting all of the AES-CCM NIST KAT vectors many of which use non-default nonce and tag length= s. I have refined the approach to better fit the existing OCF model where nonce = and MAC lengths are properties of a session (similar to key lengths). (My earli= er branch had made the nonce length a property of individual operations instea= d.) Mostly this entailed extending the /dev/crypto interface to permit setting these parameters for a session. Existing tests for OCF run in userland and = use the /dev/crypto interface including both the cryptocheck utility and the NI= ST KAT vector tests. Building on these framework changes, I have extended the existing Chacha20-Poly1305 cipher in OCF to support both 8 and 12 byte nonces includ= ing in the accelerated ossl(4) driver. I also have a patch against the upstream WireGuard FreeBSD driver to make use of this for the dataplane and have verified that it interoperates with the stock WireGuard driver. Future work will focus on upstreaming the OCF changes as well as additional review of the upstream WireGuard driver. Sponsor: The FreeBSD Foundation =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Intel Volume Management Device driver update Links: main branch commit URL: https://cgit.freebsd.org/src/commit/?id=3D 7af4475a6e31202a865b1dd3727018659b44470f Contact: Alexander Motin Intel VMD is used by Intel=E2=80=99s VROC (Virtual RAID on chip) to isolate= parts of PCI tree, including NVMe devices, behind one or several VMD devices. Each V= MD device has 3 memory windows to access config space and memory of its child devices, plus some number (I saw 19 and 33) of MSI-X interrupt vectors to forward their MSI and MSI-X interrupts through. The vmd(4) driver was first introduced in FreeBSD 13.0, but was found to ha= ve a number of problems addressed in this update: - The VMD device was made to l= ook more like a regular PCI-to-PCI bridge, implementing the regular pcib(4) interface and using the standard pci(4) bus driver on top. - Memory and bus numbers resource management was rewritten to properly handle resource reque= sts using memory windows of the VMD device. - Interrupt handling was completely rewritten to distribute child interrupts among available VMD MSI-X vectors, setting them up to be handled via standard OS mechanisms with minimal overh= ead instead of custom dispatching via taskqueue. Due to the limited number of vectors and routing abilities of VMD, children are limited to only one MSI = or three MSI-X vectors per device to avoid sharing. The limits can be tuned, depending on the number of VMD vectors and children. To improve the flexibi= lity the nvme(4) driver was made to work with a limited number of vectors, start= ing =66rom just one MSI/MSI-X if needed. Sponsor: iXsystems, Inc. =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Ports Changes affecting the Ports Collection, whether sweeping changes that touch most of the tree, or individual ports themselves. Improve CPE Data in the Ports Collection Links: chkcpe URL: https://github.com/decke/chkcpe CPE on wiki.freebsd.org URL: https://wiki.freebsd.org/Ports/CPE CPE Dictionary URL: https://nvd.nist.gov/products/cpe Contact: Bernhard Froehlich Common Platform Enumeration (CPE) is an open standard for naming hardware a= nd software components. Originally developed by MITRE, it is now maintained by NIST under the aegis of the National Vulnerability Database. A CPE URI uniq= uely identifies a device or program by its vendor, product name, version, revisi= on, etc. NIST maintains the official CPE dictionary which lists CPE names for a= ll software versions that have ever been mentioned in an advisory. In the FreeBSD Ports Collection it has been possible to annotate CPE data w= ith USES=3Dcpe since 2014 but only around 1000 ports out of 30.000 did use it. = This is why a script called chkcpe was created to validate existing CPE data and find new possible matches automatically. Why do we need it? It allows comparing CPE URIs for installed packages against published vulnerability data and will give us a much better and more complete pkg aud= it. As a side effect we will also have a better overview of vulnerable ports in= the FreeBSD Ports Collection that need to be patched or updated. How can I help? In this phase there is no easy possibility for port maintainers to help. Creating separate PRs only to add CPE data does not make sense because the overhead is very high. This is why I did spend some time on chkcpe to impro= ve the review and commit workflow. Right now review and commit consumes around= 3 minutes per port. If you are a Ports Committer and want to help please get in touch! Open Tasks =E2=80=A2 Review remaining reports (~1800) and update ports when appropri= ate =E2=80=A2 Improve matching quality to find more possible matches =E2=80=A2 Support using CPE data in pkg audit =E2=80=A2 Scan for vulnerable ports in the Ports Collection =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 FreeBSD Erlang Ecosystem Ports update Links: FreeBSD Erlang wiki URL: https://wiki.freebsd.org/Erlang Erlang/OTP language URL: https://erlang.org/ Elixir language URL: https://elixir-lang.org/ Gleam language URL: https://gleam.run/ Contact: FreeBSD Erlang mailing list The Erlang runtime system, commonly known as the BEAM, provides a runtime t= hat is used by a number of programming languages and applications in the FreeBSD ports collection. Earlier this year, both Elixir and Erlang runtimes were brought up to date,= but as separate ports, to enable porters and users to test applications side by side. In Q3, the current runtimes have been brought across as defaults - this mea= ns that lang/elixir and lang/erlang are running as the latest releases of these superb programming languages and runtimes. Older releases of lang/erlang-runtime{21,22,23} are still available as port= s. The very old releases prior to OTP20 have been removed from the ports tree,= as they are no longer supported upstream either. Only newer OTP releases include the updated SSL application that will corre= ctly validate cross-signed certificates, as used in Let=E2=80=99s Encrypt=E2=80= =99s upcoming root certificate deprecations. Further details on these changes are well documented at Erlang/OTP impact of DST Root CA X3 expiration and DST Root CA X3 expiration update All of the NIF driver related ports that pull in other FreeBSD ports tree dependencies have been updated to match the newer lang/erlang release, and a number of ports that are not being updated in their upstream community, have therefore been marked as broken. The Erlang team is planning to: =E2=80=A2 remove the deprecated OTP20 and OTP21 runtimes in 2021Q4 =E2=80=A2 remove ports directly dependent on erlang- and elixir- language= s, where they are more commonly installed via mix and rebar3 tools, the standard community build tool chain. Additional testing and community contributions are welcome; please reach ou= t on the mailing list, especially if you are able to help testing of specific po= rt updates. =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 KDE on FreeBSD Links: KDE FreeBSD URL: https://freebsd.kde.org/ KDE Community FreeBSD URL: https://community.kde.org/FreeBSD Contact: Adriaan de Groot The KDE on FreeBSD project aims to package almost all of the software from = the KDE Community for the FreeBSD ports tree. The software includes a full desk= top environment called KDE Plasma (for both X11 and Wayland) and hundreds of applications that can be used on any FreeBSD machine. The KDE team (kde@) is part of desktop@ and x11@ as well, building the soft= ware stack to make FreeBSD beautiful and usable as a daily-driver graphics-based desktop machine. Q3 is summer in the northern hemisphere: bicycles were biked and mountains = were hiked and the team was psyched to chase the software we like. And there were plenty of things to chase: =E2=80=A2 Three CMake releases landed, ending up with CMake 3.21.3 and li= bpkg support that once again works with CPack to produce FreeBSD packages. =E2=80=A2 Monthly releases of KDE Frameworks, KDE Plasma, and KDE Gear ke= pt the exp-runs going. kde@ would like to thank Antoine for overseeing our many exp-run requests. =E2=80=A2 The KDE Plasma applications for monitoring the state of the system=E2=80=89=E2=80=94=E2=80=89ksysguard, Plasma system monitor, and = supporting libraries=E2=80=89=E2=80=94=E2=80=89received a number of updates. FreeB= SD support code has been merged upstream. =E2=80=A2 Across all of the Qt and KDE packages, an attempt was made to r= educe how "heavy" the dependencies are. In general this means removing developer-= only dependencies, avoiding the installation of test-code, etc. The underlyi= ng idea is to cut down the size of the installation when specific ports are installed, while preserving the "developer batteries included" characte= r of the meta-ports. =E2=80=A2 A runtime-incompatibility was introduced by MySQL 5.7.34, which= affected KDE=E2=80=99s personal-information-management applications and email. T= his was patched in the Qt ports. =E2=80=A2 The mixer application in KDE Plasma now defaults to using pulse= audio. =E2=80=A2 accountsservice was updated, and then patched with code from Op= enBSD. =E2=80=A2 kdiff3 was updated to 1.9.3, now with upstream patches for some= corner cases originally reported on FreeBSD. =E2=80=A2 kimageannotator and ksnip were updated, for nicer screenshots. =E2=80=A2 kraft, the small-business support application, was updated to v= ersion 0.97. =E2=80=A2 krita had an upstream release to address a performance issue, w= hich then landed in FreeBSD. =E2=80=A2 kstars, the interactive planetarium and sky-watching applicatio= n, was updated to 3.5.5. =E2=80=A2 latte-dock, used as an alternative launcher, updated to 0.10.2. =E2=80=A2 libphonenumber, Google=E2=80=99s library for dealing with all t= he ways phone numbers are represented around the world, received multiple updates. =E2=80=A2 maliit-framework, for on-screen keyboards, as added to the port= s tree. =E2=80=A2 pipewire was kept up-to-date. This is a next-generation video-h= andling framework, and is being increasingly used in Wayland environments for efficient passing of video data between applications. =E2=80=A2 poppler was updated to version 21.09, with significant speed im= provements. =E2=80=A2 sddm was made a little more robust when installed on its own, w= ith xinitrc support. =E2=80=A2 math/eigen2 was removed. =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 OpenSearch on FreeBSD Links: OpenSearch website URL: https://opensearch.org/ OpenSearch Community (unofficial) on Discord URL: link:https://discord.gg/ NmtQGAWY Contact: OpenSearch Team OpenSearch is a community-driven, open-source search and analytics suite derived from Apache 2.0 licensed Elasticsearch 7.10.2 & Kibana 7.10.2. It consists of a search engine daemon, OpenSearch, and a visualization and user interface, OpenSearch Dashboards. OpenSearch enables people to easily inges= t, secure, search, aggregate, view, and analyze data. These capabilities are popular for use cases such as application search, log analytics, and more. = With OpenSearch people benefit from having an open-source product they can use, modify, extend, monetize, and resell how they want. After the release of OpenSearch 1.0.0, a FreeBSD OpenSearch team was create= d to coordinate the efforts for porting it to FreeBSD. Because ElasticSearch 7 a= nd Kibana 7 were already in the ports tree, we could rely on these ports to get started quickly (kudos to the elastic@ team) and could focus on the parts t= hat already changed between the previous codebase and the fork. The ports have = been committed to the FreeBSD ports tree as textproc/opensearch and textproc/ opensearch-dashboards, and currently provide the latest 1.0.1 version of OpenSearch. Contributing The ports have been thoroughly tested in our testing environments and on so= me production workloads, but we are actively looking for feedback from users. = Feel free to send us an email to report successes or failures. =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Valgrind - Official Support for FreeBSD has Landed Links Valgrind Home Page URL: https://www.valgrind.org/ Valgrind News URL: https://www.valgrind.org/docs/manual/dist.news.html Contact: Paul Floyd Valgrind is an instrumentation framework for building dynamic analysis tool= s. There are Valgrind tools that can automatically detect many memory manageme= nt and threading bugs, and profile your programs in detail. You can also use Valgrind to build new tools. With the release of version 3.18.0 of Valgrind, official FreeBSD support has landed upstream. The two supported architectures are amd64 and i386 (x86). The next step will be to update the FreeBSD port. This will involve switchi= ng =66rom code that was maintained out-of-tree for over 15 years to using the official upstream tarball. As Valgrind is closely tied to operating system details, obtaining official FreeBSD support was the result of significant effort from dozens of develop= ers. =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Wine on FreeBSD Links Wine home page Contact: Gerald Pfeifer Contact: Damjan Jovanovic < damjan.jov@gmail.com> Wine allows running Windows programs on FreeBSD. This quarter we focused on the wine-devel port that tracks mainline development, followed half a dozen of version updates, simplified the port, removed three options (SDL, VKD3D, VULKAN) which are unconditionally active now, and fixed a number of portability issues. These changes will make their way into the primary Wine port over the coming weeks. After working on our Wine ports for more than 21 years, maintaining for more than 19 years, time has come to hand over the baton. Luckily Damjan has stepped up and is going to look after wine-devel and associated ports - thanks! Help with the following is still very welcome: =E2=80=A2 WineHQ bug 50257 =E2=80=A2 FreeBSD Bugzilla 257533 =E2=80=A2 Maintain daily (or at least regular) test builds of upstream Wi= ne. This quarter this triggered two dozen fixes in support of FreeBSD upstream, though usually it=E2=80=99s quite less effort. =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Ztop Links: Repository Port Contact: Alan Somers ztop is a new utility that displays ZFS datasets' I/O in real time, like to= p, but for ZFS. Unlike the existing zpool iostat, which only displays traffic = at the level of a pool, ztop displays it at the level of individual datasets. Previously, there was no tool that could display this information. The Prometheus Node Exporter can export it into Prometheus, from which it can be viewed after the fact with various other tools, but that requires a large s= tack of software, and isn=E2=80=99t real-time. I started ztop this quarter, and it is now fully functional. However, it has revealed a performance problem within the kernel. In the presence of more t= han a few thousand datasets, the sysctl interface is too slow and ztop becomes sluggish. Future work, if I have the time, will address that problem. Sponsor: Axcient =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Miscellaneous Objects that defy categorization. -CURRENT compilation time analysis Links: Bug 257141 URL: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D257141 Discussion on freebsd-current URL: https://lists.freebsd.org/archives/ freebsd-current/2021-September/index.html#msg511 Visual chart of buildworld by stages URL: https://people.freebsd.org/wosch/ build-time/buildworld/ Contact: Wolfram Schneider Re-building FreeBSD from source takes a lot of CPU resources. Depending on = your machine it takes between 20min and several hours. At the end of `make buildworld' we log the real time and which parameters are used for parallel build. E.g. time make -j $(sysctl -n hw.ncpu) buildworld > buildworld.log 2>&1 tail buildworld.log >>> World build completed on Sat Sep 4 20:58:00 UTC 2021 >>> World built in 7235 seconds, ncpu: 3, make -j3 7235.61 real 20527.30 user 915.88 sys The build process runs in several steps. It would be great to know which st= ep takes most of the time, and what are the effects of special build parameter= s. With a small patch in Aug 2021 we get this information now: egrep '>>> stage| real ' buildworld.log >>> stage 1.1: legacy release compatibility shims 0.28 real 0.18 user 0.10 sys >>> stage 1.2: bootstrap tools 165.99 real 472.58 user 11.56 sys >>> stage 2.1: cleaning up the object tree 21.47 real 36.96 user 14.14 sys 15.87 real 29.14 user 11.87 sys >>> stage 2.3: build tools 2.42 real 3.79 user 0.62 sys >>> stage 3: cross tools 9.92 real 18.49 user 1.75 sys >>> stage 3.1: recording build metadata 0.07 real 0.01 user 0.06 sys >>> stage 4.1: building includes 16.62 real 36.46 user 9.48 sys >>> stage 4.2: building libraries 5440.89 real 15724.60 user 482.58 sys >>> stage 4.3: building lib32 shim libraries 615.91 real 1654.77 user 164.58 sys >>> stage 4.4: building everything 937.23 real 2540.06 user 205.47 sys In this example, we spent most of the time in "stage 4.2: building librarie= s", 77% of the CPU time and 75% of the real time. Now running the buildworld wi= th the parameter WITHOUT_TOOLCHAIN=3Dyes we get a 3.3x faster build. Instead o= f 2h it will be done in 36 minutes! time make -j $(sysctl -n hw.ncpu) WITHOUT_TOOLCHAIN=3Dyes buildworld > buil= dworld.log 2>&1 >>> World build completed on Fri Sep 17 12:31:41 UTC 2021 >>> World built in 2207 seconds, ncpu: 3, make -j3 2207.44 real 5710.83 user 676.16 sys egrep '>>> stage| real ' buildworld.log >>> stage 1.1: legacy release compatibility shims 0.35 real 0.20 user 0.16 sys >>> stage 1.2: bootstrap tools 20.47 real 51.98 user 5.12 sys >>> stage 2.1: cleaning up the object tree 20.92 real 34.45 user 13.57 sys 16.33 real 28.59 user 12.33 sys >>> stage 2.3: build tools 2.59 real 3.90 user 0.86 sys >>> stage 3: cross tools 10.46 real 18.62 user 2.35 sys >>> stage 3.1: recording build metadata 0.07 real 0.03 user 0.05 sys 0.08 real 0.03 user 0.05 sys >>> stage 4.1: building includes 15.50 real 33.03 user 9.29 sys >>> stage 4.2: building libraries 750.31 real 1962.43 user 218.60 sys >>> stage 4.3: building lib32 shim libraries 684.05 real 1780.35 user 213.22 sys >>> stage 4.4: building everything 677.87 real 1787.13 user 186.82 sys Using WITHOUT_TOOLCHAIN=3Dyes is fine as long as there are no major LLVM co= mpiler changes. If you dislike this feature or suspect it is causing trouble you can disabl= e it with the variable TIME_ENV=3D"" Next task: get more detailed timing information for the long-running stages (4.2, 4.3, 4.4) =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Third-Party Projects Many projects build upon FreeBSD or incorporate components of FreeBSD into their project. As these projects may be of interest to the broader FreeBSD community, we sometimes include brief updates submitted by these projects in our quarterly report. The FreeBSD project makes no representation as to the accuracy or veracity of any claims in these submissions. Gitlab 14.3 available Links: Gitlab 14.3 New Features URL: https://about.gitlab.com/releases/2021/09/22/ gitlab-14-3-released/ Contact: Matthias Fechner GitLab is the DevOps Platform. Bring velocity with confidence, security wit= hout sacrifice, and visibility into DevOps success. Version 14.3 now available on FreeBSD. =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 helloSystem Links: Documentation URL: https://hellosystem.github.io/ Contact: Simon Peter Contact: #helloSystem on irc.libera.chat, mirrored to #helloSystem:matrix.o= rg on Matrix What is helloSystem? helloSystem is FreeBSD preconfigured as a desktop operating system with a f= ocus on simplicity, elegance, and usability. Its design follows the =E2=80=9CLes= s, but better=E2=80=9D philosophy. Q3 2021 Status =E2=80=A2 Version 0.6.0 of helloSystem has been published including many = contributed features and bugfixes =E2=96=A1 Improved window management with KWin as the window manager =E2=96=A1 Simplified Filer file manager =E2=80=A2 Contributors have started to port the helloSystem desktop envir= onment, helloDesktop, to the FreeBSD Ports and Packages Collection (see changel= og at https://github.com/helloSystem/ISO/releases/tag/r0.6.0 for details) Installable Live ISO images and a full changelog are available at https:// github.com/helloSystem/ISO/releases/tag/r0.6.0 Contributing Help is wanted in a number of areas, especially in the areas of the FreeBSD core OS and kernel, and Qt/C++. =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 Containers & FreeBSD: Pot, Potluck & Potman Links: Pot on github URL: https://github.com/pizzamig/pot Potluck Repository & Project URL: https://potluck.honeyguide.net/ Potluck on github URL: https://github.com/hny-gd/potluck Potman on github URL: https://github.com/grembo/potman Contact: Luca Pizzamiglio (Pot) Contact: Stephan Lichtenauer (Potluck) Contact: Michael Gmelin (Potman) Pot is a jail management tool that also supports orchestration through Noma= d. In the last quarter, a new release 0.13.0 with a number of major new featur= es was made available. Layered images allow a split into a foundation image component that only changes seldom and that e.g. contains the basic jail and packages and a "le= af" image component on top with potentially small additions that might change m= ore frequently, like software built in a CI pipeline. Since it is not the compl= ete image that has to be rebuilt each time, image creation and distribution can= be sped up immensely. Also a copy-out command has been included where the container state that was initialised from within the container can be made persistent and reinjected into additional containers afterwards. Potluck aims to be to FreeBSD and Pot what Dockerhub is to Linux and Docker= : a repository of Pot flavours and complete container images for usage with Pot= and in many cases Nomad. Here we had a busy quarter. Not only did we commit a number of new images l= ike Nextcloud which can be deployed via Nomad, but the images used to build the foundation of a virtual data center (Nomad, Consul and Vault) themselves ha= ve received major updates. For these images, further improvements for even better security and reliabi= lity will likely be finished in the coming quarter. Also, we are aware that right now the advantages of the container approach = as it is described in Virtual DC Part I/Part II/Part III are not yet obvious a= nd transparent enough and also no longer completely up to date, so we plan to provide additional documentation as soon as we find the time to do so. Potman aims to simplify building Pot images with Vagrant and VirtualBox bas= ed on the Potluck approach, e.g. as part of a DevOps workflow for software development including testing and publishing them to a repository. Here, not too much has happened over the last quarter but the current idea = is to incorporate additional features in the medium to longer term to modulari= se and simplify the image build definitions and then utilise Potman in the Pot= luck library build process. As always, feedback and patches are welcome. =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 WireGuard on FreeBSD Stabilizes, Eyes Upstreaming Links WireGuard URL:https://www.wireguard.com/ wireguard-freebsd codebase URL:https://git.zx2c4.com/wireguard-freebsd/ Contact: Jason A. Donenfeld WireGuard is a secure tunneling protocol that lives in the kernel. For the last quarter, the wireguard-freebsd codebase has been quite stable = and complete. For a while, there were rapid-fire releases fixing issues, and a = lot of effort was made to track down every bug report on bugs.freebsd.org, IRC,= the mailing list, or elsewhere. But by now, the reports have dried up, and most= ly users come to IRC with questions on usage and integration, the usual types = of things associated with a stabler project. We also have automated CI now for each commit, compiling and running a small smoke test on wireguard-freebsd= =E2=80=99s supported releases=E2=80=89=E2=80=94=E2=80=8912.1, 12.2, and 13.0. At some = point, hopefully that small smoke test will expand to include the larger battery of tests from Linux. The wireguard-freebsd repository has been geared around being an out-of-tree kmod, which is distributed in ports. But it is also organized to be eventua= lly upstreamed. To that end, the repository maintains two files: compat.h and support.h. compat.h contains polyfills of code that exists in FreeBSD=E2=80= =99s main branch but does not exist in various older releases, with ifdefs for each of the various releases we support. On the other hand, support.h contains code that is not yet in FreeBSD=E2=80=99s main branch. The goal is to eventually= move all the code from support.h into compat.h, at which point, the repository will = be ready for upstreaming. As of writing, there=E2=80=99s basically only one re= al function left=E2=80=89=E2=80=94=E2=80=89sogetsockaddr=E2=80=89=E2=80=94=E2=80=89and = then two convenience macros that need to be sent upstream for consideration by ConcurrencyKit. A significant aspect that isn=E2=80=99t in support.h, though, is the crypto= graphic primitives that the code uses. The files crypto.c and crypto.h contain bori= ng C "reference implementations" of ChaCha20Poly1305, XChaCha20Poly1305, Blake2s, and X25519 (which is formally verified via MIT=E2=80=99s fiat-crypto projec= t). These four algorithms are used by the handshake path on very small inputs for WireGuard=E2=80=99s key exchange, and will hopefully be making their way to= sys/crypto/ in the FreeBSD tree as just ordinary functions. On the flip side, the datap= ath uses an entry point of ChaCha20Poly1305 that works on mbufs (which might be rather large) and is performance critical. To that end, jhb@ has been impro= ving OCF for WireGuard=E2=80=99s particulars. The next step will then be moving = our current calls from chacha20poly1305_{en,de}crypt_mbuf in wg_noise.c to instead call= out to OCF, submitting crypto_buffer`s of type `CRYPTO_BUF_MBUF. This will automatically benefit from Andy Polyakov=E2=80=99s optimized ChaPoly implem= entations that OCF has long since imported from OpenSSL. When we make the move to OCF, it=E2=80=99s likely that the wireguard-freebs= d repo as-is will become "wireguard-freebsd-compat", which will be explicitly aimed at backports to earlier FreeBSD releases for ports, while a new wireguard-free= bsd repository will be a whole FreeBSD tree, where we can work directly on integration patches for upstream. That repository will also have an imported version of the wg(8) utility from wireguard-tools, which I=E2=80=99ll be re= licensing as MIT. I=E2=80=99m quite excited for the upcoming quarter and seeing how much of wireguard-freebsd we=E2=80=99re able to land upstream. =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2= =94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94= =81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81=E2=94=81= =E2=94=81=E2=94=81=E2=94=81=E2=94=81 --66dsezfsnajsewcm Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQGTBAABCgB9FiEEDonNJPbg/JLIMoS6Ps5hSHzN87oFAmGSjzBfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDBF ODlDRDI0RjZFMEZDOTJDODMyODRCQTNFQ0U2MTQ4N0NDREYzQkEACgkQPs5hSHzN 87rCJQf+L85vcCZetfg8jx9wmIkBX8z1Sy4P0k0JYAlDU64bdSorQfv9iehSapMd jGKLOhD5oQjFht/2u4uM9bBYMDBe53JSSo9NUSdcjPDsHcwLbZPKbxNlSZ6GOX6U UmAGftRiXLTIX2DNTeNQmtUzd/Os66/13RTaUlkAg2L9zHxX6ImXWk27EWpsR6Nn oY4gXT4/px750uBdYPv1HxGxLpHIMy4E46T7UOcrc3UmbhN+lAf6fj+yNIx04ur5 IYgPv8vAFffaTjvF/YzPbPj9L5UJhCzTQed8IE8Xbsl4r+NKKmFRCk0Tb+IIUiWA vCnZ8iOJWV9vWTOZgxMHWD/43oxwyQ== =n3Dy -----END PGP SIGNATURE----- --66dsezfsnajsewcm-- From nobody Tue Nov 16 07:15:31 2021 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id F136718909F1 for ; Tue, 16 Nov 2021 07:15:50 +0000 (UTC) (envelope-from graham@menhennitt.com.au) Received: from bumble.maple.relay.mailchannels.net (bumble.maple.relay.mailchannels.net [23.83.214.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Htcm10Kbfz4Tfb for ; Tue, 16 Nov 2021 07:15:44 +0000 (UTC) (envelope-from graham@menhennitt.com.au) X-Sender-Id: dreamhost|x-authsender|graham@menhennitt.com.au Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id EC54B5427AD for ; Tue, 16 Nov 2021 07:15:34 +0000 (UTC) Received: from pdx1-sub0-mail-a299.dreamhost.com (100-96-133-208.trex.outbound.svc.cluster.local [100.96.133.208]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id 67F59541E82 for ; Tue, 16 Nov 2021 07:15:34 +0000 (UTC) X-Sender-Id: dreamhost|x-authsender|graham@menhennitt.com.au Received: from pdx1-sub0-mail-a299.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384) by 100.96.133.208 (trex/6.4.3); Tue, 16 Nov 2021 07:15:34 +0000 X-MC-Relay: Neutral X-MailChannels-SenderId: dreamhost|x-authsender|graham@menhennitt.com.au X-MailChannels-Auth-Id: dreamhost X-Slimy-Spill: 1e5766690accef09_1637046934661_2029781703 X-MC-Loop-Signature: 1637046934661:1473818357 X-MC-Ingress-Time: 1637046934660 Received: from [203.2.73.68] (14-202-237-130.tpgi.com.au [14.202.237.130]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: graham@menhennitt.com.au) by pdx1-sub0-mail-a299.dreamhost.com (Postfix) with ESMTPSA id 4Htcln5kZ9z3J for ; Mon, 15 Nov 2021 23:15:33 -0800 (PST) Subject: Re: packet loss between interfaces on the router To: stable@freebsd.org References: <216340c2-795d-d7ad-87d8-e07d9336564d@zhegan.in> From: Graham Menhennitt Message-ID: <8c8e724a-d9c2-9465-2e35-4422b736aaf9@menhennitt.com.au> Date: Tue, 16 Nov 2021 18:15:31 +1100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.9.1 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 4Htcm10Kbfz4Tfb X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of graham@menhennitt.com.au designates 23.83.214.25 as permitted sender) smtp.mailfrom=graham@menhennitt.com.au X-Spamd-Result: default: False [0.70 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:23.83.208.0/20]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[stable@freebsd.org]; NEURAL_SPAM_MEDIUM(1.00)[1.000]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000]; DMARC_NA(0.00)[menhennitt.com.au]; NEURAL_SPAM_SHORT(1.00)[0.997]; RCVD_IN_DNSWL_NONE(0.00)[23.83.214.25:from]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:36483, ipnet:23.83.208.0/21, country:CA]; MIME_TRACE(0.00)[0:+]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[14.202.237.130:received] X-ThisMailContainsUnwantedMimeParts: N On 15/11/21 6:58 pm, Eugene M. Zheganin wrote: > Hello, > > 15.11.2021 2:14, Eugene M. Zheganin пишет: >> [...] >> The host is running PF as a packet filter, several dozens of rules. I >> disable the scrub on outer interface (since the lost packet wasn'ta  >> fragment, I was sceptical about it, and it doesn't help indeed). >> [...] >> > ...and seems like it's a PF problem (so I probably should've started > this conversation in pf@) > > Here's another stalled session with PF debug turned "loud". Below are > caprtures on outer and inner interfaces, along with PF debug messages. > What is the "3" condition ? I only managed to find that this is some > sort of ackskew clashing. > > Could something be done here via pf configuration ? > > Outer interface: I've never used pf, so I have no idea if this is relevant, but... Are you doing NAT on this interface? If so, maybe you need to turn off various hardware checksum options in the interface.     ifconfig_igb1="-vlanhwtso -tso4 -txcsum -rxcsum" (in /etc/rc.conf - replace igb1 with your interface name) Maybe not all of those are needed. It fixed problems for me with ipfw. It's worth a try anyway. Good luck,     Graham From nobody Wed Nov 17 18:17:20 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4E6C8188A4F6 for ; Wed, 17 Nov 2021 18:27:44 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.org (uucp.dinoex.org [IPv6:2a0b:f840::12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "uucp.dinoex.sub.de", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HvWct66lBz3PWy for ; Wed, 17 Nov 2021 18:27:42 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.org [185.220.148.12]) by uucp.dinoex.org (8.17.1/8.17.1) with ESMTPS id 1AHIR4AW079095 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Wed, 17 Nov 2021 19:27:04 +0100 (CET) (envelope-from pmc@citylink.dinoex.sub.org) X-MDaemon-Deliver-To: X-Authentication-Warning: uucp.dinoex.sub.de: Host uucp.dinoex.org [185.220.148.12] claimed to be uucp.dinoex.sub.de Received: (from uucp@localhost) by uucp.dinoex.sub.de (8.17.1/8.17.1/Submit) with UUCP id 1AHIR403079092 for freebsd-stable@freebsd.org; Wed, 17 Nov 2021 19:27:04 +0100 (CET) (envelope-from pmc@citylink.dinoex.sub.org) Received: from gate.intra.daemon.contact (gate-e [192.168.98.2]) by citylink.dinoex.sub.de (8.16.1/8.16.1) with ESMTP id 1AHIIbAP017264 for ; Wed, 17 Nov 2021 19:18:37 +0100 (CET) (envelope-from peter@gate.intra.daemon.contact) Received: from gate.intra.daemon.contact (gate-e [192.168.98.2]) by gate.intra.daemon.contact (8.16.1/8.16.1) with ESMTPS id 1AHIHKKN016788 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Wed, 17 Nov 2021 19:17:20 +0100 (CET) (envelope-from peter@gate.intra.daemon.contact) Received: (from peter@localhost) by gate.intra.daemon.contact (8.16.1/8.16.1/Submit) id 1AHIHKZj016787 for freebsd-stable@freebsd.org; Wed, 17 Nov 2021 19:17:20 +0100 (CET) (envelope-from peter) Date: Wed, 17 Nov 2021 19:17:20 +0100 From: Peter To: freebsd-stable@freebsd.org Subject: 12.3-RC1 fails to install pkg Message-ID: List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Milter: Spamilter (Reciever: uucp.dinoex.sub.de; Sender-ip: 185.220.148.12; Sender-helo: uucp.dinoex.sub.de;) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.4 (uucp.dinoex.org [185.220.148.12]); Wed, 17 Nov 2021 19:27:07 +0100 (CET) X-Rspamd-Queue-Id: 4HvWct66lBz3PWy X-Spamd-Bar: + Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of pmc@citylink.dinoex.sub.org has no SPF policy when checking 2a0b:f840::12) smtp.mailfrom=pmc@citylink.dinoex.sub.org X-Spamd-Result: default: False [1.04 / 15.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.95)[-0.949]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; TO_DN_NONE(0.00)[]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[4]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; NEURAL_SPAM_SHORT(1.00)[1.000]; NEURAL_SPAM_MEDIUM(0.09)[0.086]; DMARC_NA(0.00)[sub.org]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:205376, ipnet:2a0b:f840::/32, country:DE]; RCVD_TLS_LAST(0.00)[] X-ThisMailContainsUnwantedMimeParts: N Not sure if this is intended or not - it certainly wasn't the case with 12.2, and it likely will break a few automated deploys: # ls -la pkg* -rw-r--r-- 1 root wheel 4011616 Nov 17 17:52 pkg-1.17.2.pkg # pkg add /tmp/vb/pkg-1.17.2.pkg < /dev/null The package management tool is not yet installed on your system. Please set ASSUME_ALWAYS_YES=yes environment variable [etc.etc.] Okay, lets try that one: # pkg add -y /tmp/vb/pkg-1.17.2.pkg < /dev/null Bootstrapping pkg from https://oper-e.intra.daemon.contact/sysimg/ports/conr/Cur, please wait... pkg: Error fetching https://oper-e.intra.daemon.contact/sysimg/ports/conr/Cur/Latest/pkg.txz: Not Found A pre-built version of pkg could not be found for your system. Consider changing PACKAGESITE or installing it from ports: 'ports-mgmt/pkg'. Not good. Not at all. But then: # pwd /tmp/vb # pkg add pkg-1.17.2.pkg < /dev/null Installing pkg-1.17.2... Extracting pkg-1.17.2: 100% So, it *can* still do it. It just does no longer accept an (absolute or relative) pathname. And I have no idea what that might be good for. Any clues, anybody? cheerio, PMc From nobody Wed Nov 17 18:42:28 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4F8BD1891369 for ; Wed, 17 Nov 2021 18:42:31 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [96.47.72.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HvWxz1vTPz3khC; Wed, 17 Nov 2021 18:42:31 +0000 (UTC) (envelope-from gjb@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1637174551; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=WevrRGB3k+YvPTPBACBA03pRMHnFZLxhb/I8jCUpHbE=; b=Hy3lMm7dv1ZmRc00s26I4J7/Vaj0k9+QDkBWdb9hIbLevXsGRBNLChdBrD55nWSWnwdbht zPrUJoIS/jHe9fLkt63QEz98vFHTy7ehsTwpWVTaNscEOHM4k8yyFuyuQGe1NLDHiYXqOD sYuhAgYjOc4YeSOXUio9lq2dZM829lZ54tgpc+ZV7mVgswYNER/ozMzQcLXKA8JFyQkbZX NRPokYVhJwE1y+G/VD2d8FevyYmyRkaamFt5xmjeaHIu6ooAsUlfbAIAd/dITzqTN2OOXf xKja1mX5T+4qFZiV3QYr2rWGmFGqDYfKjvvEf1v4Nc1Qk5InLnOhYAjIkf+vFg== Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by freefall.freebsd.org (Postfix) with ESMTPS id D829270AB; Wed, 17 Nov 2021 18:42:30 +0000 (UTC) (envelope-from gjb@freebsd.org) Date: Wed, 17 Nov 2021 18:42:28 +0000 From: Glen Barber To: Peter Cc: freebsd-stable@freebsd.org Subject: Re: 12.3-RC1 fails to install pkg Message-ID: <20211117184228.GK97139@FreeBSD.org> References: List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="MzdA25v054BPvyZa" Content-Disposition: inline In-Reply-To: ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1637174551; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=WevrRGB3k+YvPTPBACBA03pRMHnFZLxhb/I8jCUpHbE=; b=MQRM/ILpu66RgkRmZAC5irdyUTYvPmIobNAcU/+JEmZOVZUoWLEb2hoL34OoYuZLEmXMFK jO7HiJ/1BvtnEsdMNQl5YVkHNa3WQlEBmF/YpBPm93RHm+MTsPXdK8GxmSjFJaUJaQQcFO GoLAG9rmDE70gYBlDkqeq0L/rBUekqdXpKQjYvK+6Sj2Bt2O2z7OMyXfZ4QABLkHjpwNWj NOKJcaz0qDpFveW+PSaIvIIK/x/kAZQWMNsORln0qymy2JSVjZYo5K3+Y0oUdBg7x/bo7e oymEjnMezUkfiVPersGgdVpM2tn02rraYeYIwLoGNnAwbDsKRE8qwAVcjoIxNA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1637174551; a=rsa-sha256; cv=none; b=tqK/HNLlU3cdo0mHBQ/AUCpeznC/vLvtJfVl+sfs34EzbrlLrqDqMpCEdqJx0PGdYlc4rO XoUmXjqXfFseTpTmIzLtLgtXZpml6l36P0uAfCnIKaJva2esVHqqe54ULwO+mo2J2GqO6y BpYWwPT9aUTpQeEbGczcIN0mff6Yh0n0CL7xhF+X8DYHroRDJRZHh74Zlxvk3DWcmL8i1t 02bDW+U47rZfiwJaeQkHoqEUiDB4XKB56H0oVG0AfHRiY56ehRDVJmPAZByNfQ8pvm2NPm BlCltd6mvVej1aQgjo+89PquAHhoAHOzBh/RTU/wWAlPAPV4J5jXFayPAhzXhA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N --MzdA25v054BPvyZa Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Nov 17, 2021 at 07:17:20PM +0100, Peter wrote: >=20 > Not sure if this is intended or not - it certainly wasn't the case > with 12.2, and it likely will break a few automated deploys: >=20 >=20 > # ls -la pkg* > -rw-r--r-- 1 root wheel 4011616 Nov 17 17:52 pkg-1.17.2.pkg > # pkg add /tmp/vb/pkg-1.17.2.pkg < /dev/null > The package management tool is not yet installed on your system. > Please set ASSUME_ALWAYS_YES=3Dyes environment variable [etc.etc.] >=20 >=20 > Okay, lets try that one: >=20 > # pkg add -y /tmp/vb/pkg-1.17.2.pkg < /dev/null > Bootstrapping pkg from https://oper-e.intra.daemon.contact/sysimg/ports/c= onr/Cur, please wait... > pkg: Error fetching https://oper-e.intra.daemon.contact/sysimg/ports/conr= /Cur/Latest/pkg.txz: Not Found > A pre-built version of pkg could not be found for your system. > Consider changing PACKAGESITE or installing it from ports: 'ports-mgmt/pk= g'. >=20 >=20 > Not good. Not at all. >=20 > But then: >=20 > # pwd > /tmp/vb > # pkg add pkg-1.17.2.pkg < /dev/null > Installing pkg-1.17.2... > Extracting pkg-1.17.2: 100% >=20 >=20 > So, it *can* still do it. It just does no longer accept an (absolute > or relative) pathname. >=20 > And I have no idea what that might be good for. Any clues, anybody? >=20 Can you try this? # pkg add -y file:///tmp/vb/pkg-1.17.2.pkg Glen --MzdA25v054BPvyZa Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEjRJAPC5sqwhs9k2jAxRYpUeP4pMFAmGVTRQACgkQAxRYpUeP 4pMImA/8DWLmnKqmS/U0s0o3JHt1y9dYNkmRUiCtN1zU4bdHggg1zF50bv7Vo3oJ VJonYsFsLgBRlZmyd7jfEbFJxGNQ3JyNlSVKivRth922FBQCvKlE5WLgI/CrVXyP nqoZCYeWBuJHmO/EnEosFbb65K/GXFkE3IYeBc081WHlROA9sfI9ss7ScZGll3EM 4RWz3cbX/0sB7yOQqC6R7WTSYufI35R7hPGRviFlJ20Ip21aErRdEVFVz1D+RH+n /ubeaTtNZug4cEMh3KjMwf8bq84ED2lQiyo9M+eT4bqaX44UuVHKUduBUlX1vv+W Fg5y+7igK+w/a+MST17NEsf0tcGIHw/Od4zL9C/YFTG0O1CKZ93ovIyBfzUxqcrJ u17U+xET7fMS3aeEIczOLgztovu0Z5FOUe5tI3ckT3SEGmbNr8fvWdVSvx0uoJpG 1ughCo9dnu3TcfQwXWxtvQVtHkV83Fll/Vv+phhqWqj5YyaGR9jZ5nuLlyeYvTpH CX+UBzmGtTGGM6F3mQXtiTZBha1eLfyZVO+I2jUnNvibbNLfwwYVlBmY4gQ6VkwP gO9vAnWWPkdHxWrQD2HSLGFIH9fVpgkCtiI12J9ktRsLsUrm1u/Fv0s8tsMbAXWG djKo8zOrql9SOlvFsbXXrc2Ddh9XNa+lfqpdytESEJr3D8b/PmA= =I0Ok -----END PGP SIGNATURE----- --MzdA25v054BPvyZa-- From nobody Wed Nov 17 19:02:19 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 6AEBA18577D2 for ; Wed, 17 Nov 2021 19:09:37 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.org (uucp.dinoex.org [IPv6:2a0b:f840::12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "uucp.dinoex.sub.de", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HvXYD2G1nz3tJj for ; Wed, 17 Nov 2021 19:09:36 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.org [185.220.148.12]) by uucp.dinoex.org (8.17.1/8.17.1) with ESMTPS id 1AHJ940P006980 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Wed, 17 Nov 2021 20:09:04 +0100 (CET) (envelope-from pmc@citylink.dinoex.sub.org) X-MDaemon-Deliver-To: X-Authentication-Warning: uucp.dinoex.sub.de: Host uucp.dinoex.org [185.220.148.12] claimed to be uucp.dinoex.sub.de Received: (from uucp@localhost) by uucp.dinoex.sub.de (8.17.1/8.17.1/Submit) with UUCP id 1AHJ944o006979 for freebsd-stable@freebsd.org; Wed, 17 Nov 2021 20:09:04 +0100 (CET) (envelope-from pmc@citylink.dinoex.sub.org) Received: from gate.intra.daemon.contact (gate-e [192.168.98.2]) by citylink.dinoex.sub.de (8.16.1/8.16.1) with ESMTP id 1AHJ3cUI028808 for ; Wed, 17 Nov 2021 20:03:38 +0100 (CET) (envelope-from peter@gate.intra.daemon.contact) Received: from gate.intra.daemon.contact (gate-e [192.168.98.2]) by gate.intra.daemon.contact (8.16.1/8.16.1) with ESMTPS id 1AHJ2JQW028645 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Wed, 17 Nov 2021 20:02:19 +0100 (CET) (envelope-from peter@gate.intra.daemon.contact) Received: (from peter@localhost) by gate.intra.daemon.contact (8.16.1/8.16.1/Submit) id 1AHJ2JJP028644 for freebsd-stable@freebsd.org; Wed, 17 Nov 2021 20:02:19 +0100 (CET) (envelope-from peter) Date: Wed, 17 Nov 2021 20:02:19 +0100 From: Peter To: freebsd-stable@freebsd.org Subject: Re: 12.3-RC1 fails to install pkg Message-ID: List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Milter: Spamilter (Reciever: uucp.dinoex.sub.de; Sender-ip: 185.220.148.12; Sender-helo: uucp.dinoex.sub.de;) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.4 (uucp.dinoex.org [185.220.148.12]); Wed, 17 Nov 2021 20:09:07 +0100 (CET) X-Rspamd-Queue-Id: 4HvXYD2G1nz3tJj X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of pmc@citylink.dinoex.sub.org has no SPF policy when checking 2a0b:f840::12) smtp.mailfrom=pmc@citylink.dinoex.sub.org X-Spamd-Result: default: False [2.59 / 15.00]; FAKE_REPLY(1.00)[]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.98)[-0.977]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; HAS_XAW(0.00)[]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[4]; TO_DN_NONE(0.00)[]; NEURAL_SPAM_SHORT(1.00)[1.000]; NEURAL_SPAM_MEDIUM(0.67)[0.668]; DMARC_NA(0.00)[sub.org]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:205376, ipnet:2a0b:f840::/32, country:DE]; RCVD_TLS_LAST(0.00)[] X-ThisMailContainsUnwantedMimeParts: N > Can you try this? > # pkg add -y file:///tmp/vb/pkg-1.17.2.pkg Hi Glen, tried, doesn't help. I found the cause in the meantime, it's #370420 It states: * Chop off the final "-" (version delimiter) and check the name that * precedes it. If we didn't have a version delimiter, it must be the * pkg.$archive short form but we'll check it anyways. What they apparently did forget is to also chop off the leading path name up to and including an '/'. So it is not intentional, it is just the normal first try of an improvement where you always forget something. cheerio, PMc From nobody Fri Nov 19 01:16:45 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 0A4C21897FEB; Fri, 19 Nov 2021 01:16:49 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HwJfS6rDXz4TNw; Fri, 19 Nov 2021 01:16:48 +0000 (UTC) (envelope-from gjb@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1637284609; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type; bh=p3hwrRTKVrdNCSMEWoWPDfePoSa5xdBJzxovS+cySsE=; b=hbGNpWXmR4GXu7sOxCv72Y8c46pvXlpeJTkApnmWMJaewn4ONm7BDm4C5eNgKYNKXpxDoa xrNZJ4lmZbjP6LGrXMhT3CMH2A4hdP7Hy5pThmtDqegf0fJkr2bmKGV6QZtdZfwPXjrc5B pSL0rC8BTH+V/hfTRGo/GAAQgpnf5IE2AfmKA6u07mDR1eY4fpL4TH7sha4CEOlr+6f6AZ gz5CWFZDySzz1CP8MELigkla5944744xmeXmxIEQCf4Ka7/aBowZrTw3XRvW7I3SxcZU/p lMinLASYLWMATge+JEstCDWnvTfu3hBC+jBs2VRLB4EP7rziAlmsAGOrTeXzUA== Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by freefall.freebsd.org (Postfix) with ESMTPS id 730D3D1C5; Fri, 19 Nov 2021 01:16:48 +0000 (UTC) (envelope-from gjb@freebsd.org) Date: Fri, 19 Nov 2021 01:16:45 +0000 From: Glen Barber To: freebsd-snapshots@freebsd.org, freebsd-stable@freebsd.org Cc: FreeBSD Release Engineering Team Subject: FreeBSD 12.3-RC2 Now Available Message-ID: <20211119011645.GP97139@FreeBSD.org> List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; x-action=pgp-signed ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1637284609; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type; bh=p3hwrRTKVrdNCSMEWoWPDfePoSa5xdBJzxovS+cySsE=; b=Vde4svwlGx4bi/p85pOIS9gNWi4xQT89EE/bpjxb55oDupJ6c6VcEQPC9T4zT+4nKyYtt7 62AD1soEr/oqBO2+0gfqC54OaIkWsR088PRB//ClwCCYQE7kH/dSwPK8stsPgdkUuo9rZL jz3N1OYUSU30zT1SIJqqkvYEFzlCyX+DXHH/Dm9Sb40iSTOKI3BnkIJTVWWwDJFxSuEiUE Dp5pTRdbE5KaE1hRqcDoVCo+RCWdWt5boBBndulJeSBYbJCxgBY+CuEcKZHy2mbyeJIb2x TezF9K5Sw5yAHD0TkgV5UeeewuSrTc7y+Whdw1KQYC7WHyQprisU05CPX51k2w== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1637284609; a=rsa-sha256; cv=none; b=TYCibPKykdjZvMmaSH/TeYYFpSRZZISR7R/Vb9vwcKtUIf7bq0mdiS5sIaX+YUYfzRSn0M 16IjMuR53UYeLVLvIVM7irOaf3UHer3CrLZlgbjnGvMNGdnbfglUhLxL0WItQ2ac4CsDAs cVeC4p9UtERXblZLhru2NlmPYhtFHiYlubYRIg+pGFcs6RFP0HN3LQTdqGEGW8j8bMmwzD oL+xdJ85feK8xjeGhnWWPSqy63McGa0xoARQ1c6hyAbS+44I3X7cwrOHpaS9972Deo+HNI z5pD83rNUppGpAdMr1ObPgzaTrLQyD+NA3meCoA1OYH+q0o+FdHfVV2EINhf0w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 The second RC build of the 12.3-RELEASE release cycle is now available. Installation images are available for: o 12.3-RC2 amd64 GENERIC o 12.3-RC2 i386 GENERIC o 12.3-RC2 powerpc GENERIC o 12.3-RC2 powerpc64 GENERIC64 o 12.3-RC2 powerpcspe MPC85XXSPE o 12.3-RC2 sparc64 GENERIC o 12.3-RC2 armv6 RPI-B o 12.3-RC2 armv7 BANANAPI o 12.3-RC2 armv7 CUBIEBOARD o 12.3-RC2 armv7 CUBIEBOARD2 o 12.3-RC2 armv7 CUBOX-HUMMINGBOARD o 12.3-RC2 armv7 RPI2 o 12.3-RC2 armv7 WANDBOARD o 12.3-RC2 armv7 GENERICSD o 12.3-RC2 aarch64 GENERIC o 12.3-RC2 aarch64 PINE64 o 12.3-RC2 aarch64 PINE64-LTS Note regarding arm SD card images: For convenience for those without console access to the system, a freebsd user with a password of freebsd is available by default for ssh(1) access. Additionally, the root user password is set to root. It is strongly recommended to change the password for both users after gaining access to the system. Installer images and memory stick images are available here: https://download.freebsd.org/ftp/releases/ISO-IMAGES/12.3/ The image checksums follow at the end of this e-mail. If you notice problems you can report them through the Bugzilla PR system or on the -stable mailing list. If you would like to use SVN to do a source based update of an existing system, use the "releng/12.3" branch. A summary of changes since 12.3-RC1 includes: o Updates to the igc(4) driver. o BEAGLEBONE and RPI3 SoC images have been removed, due to late discovered issues. A list of changes since 12.2-RELEASE is available in the releng/12.3 release notes: https://www.freebsd.org/releases/12.3R/relnotes/ Please note, the release notes page is not yet complete, and will be updated on an ongoing basis as the 12.3-RELEASE cycle progresses. === Virtual Machine Disk Images === VM disk images are available for the amd64, i386, and aarch64 architectures. Disk images may be downloaded from the following URL (or any of the FreeBSD download mirrors): https://download.freebsd.org/ftp/releases/VM-IMAGES/12.3-RC2/ The partition layout is: ~ 16 kB - freebsd-boot GPT partition type (bootfs GPT label) ~ 1 GB - freebsd-swap GPT partition type (swapfs GPT label) ~ 20 GB - freebsd-ufs GPT partition type (rootfs GPT label) The disk images are available in QCOW2, VHD, VMDK, and raw disk image formats. The image download size is approximately 135 MB and 165 MB respectively (amd64/i386), decompressing to a 21 GB sparse image. Note regarding arm64/aarch64 virtual machine images: a modified QEMU EFI loader file is needed for qemu-system-aarch64 to be able to boot the virtual machine images. See this page for more information: https://wiki.freebsd.org/arm64/QEMU To boot the VM image, run: % qemu-system-aarch64 -m 4096M -cpu cortex-a57 -M virt \ -bios QEMU_EFI.fd -serial telnet::4444,server -nographic \ -drive if=none,file=VMDISK,id=hd0 \ -device virtio-blk-device,drive=hd0 \ -device virtio-net-device,netdev=net0 \ -netdev user,id=net0 Be sure to replace "VMDISK" with the path to the virtual machine image. === Amazon EC2 AMI Images === FreeBSD/amd64 EC2 AMIs are available in the following regions: af-south-1 region: ami-0bfdd5057911d1f1c eu-north-1 region: ami-0f33e63bd78e6b062 ap-south-1 region: ami-0041df9c566011795 eu-west-3 region: ami-0dfc53b4197fea81d eu-west-2 region: ami-0a13e2b35fd52ecf6 eu-south-1 region: ami-00e9473b86e9d896a eu-west-1 region: ami-02554d0aa38624cc3 ap-northeast-3 region: ami-0d822374b0bba27cb ap-northeast-2 region: ami-0ec086284b010a6ce me-south-1 region: ami-02b07b07bb8608746 ap-northeast-1 region: ami-09f0f7b5b09f332a7 sa-east-1 region: ami-0efd81f39f21572f0 ca-central-1 region: ami-01fe2fc86bbd95a99 ap-east-1 region: ami-0dfe6c57e79551c2d ap-southeast-1 region: ami-06768e806e26ee601 ap-southeast-2 region: ami-0baa508ebe21145ad eu-central-1 region: ami-0cf46e7c7a7820ae7 us-east-1 region: ami-0dfda17326c84beac us-east-2 region: ami-093642139e57e78c1 us-west-1 region: ami-04411a5ca108defd6 us-west-2 region: ami-0ea034d59240a7fd7 These AMI IDs can be retrieved from the Systems Manager Parameter Store in each region using the keys: /aws/service/freebsd/amd64/base/ufs/12.3/RC2 FreeBSD/aarch64 EC2 AMIs are available in the following regions: af-south-1 region: ami-04423943ad5cf031c eu-north-1 region: ami-07ad5589a1aab025e ap-south-1 region: ami-0ad0fd2f8e247632c eu-west-3 region: ami-00ccb2ae85faac410 eu-west-2 region: ami-0632dd26791c37b45 eu-south-1 region: ami-038822c888f7e0134 eu-west-1 region: ami-05cee60c841f1ed9e ap-northeast-3 region: ami-0a52fdd39a7c1d957 ap-northeast-2 region: ami-06fd620ecf5dd1af7 me-south-1 region: ami-0764e3a8cc2c72767 ap-northeast-1 region: ami-0f3b9212f54d66014 sa-east-1 region: ami-018ff0ce69f689c73 ca-central-1 region: ami-044c3a7fc183f90ff ap-east-1 region: ami-08faec211ffa6f39f ap-southeast-1 region: ami-06e93f606d9fd8468 ap-southeast-2 region: ami-0602580f2a78942bd eu-central-1 region: ami-0f206372eff0e270e us-east-1 region: ami-0fab37a5116eb6bb0 us-east-2 region: ami-0dfd68ad4460eb4ce us-west-1 region: ami-09458b18f3bb93516 us-west-2 region: ami-0ae5a812e41c82c8a These AMI IDs can be retrieved from the Systems Manager Parameter Store in each region using the keys: /aws/service/freebsd/arm64/base/ufs/12.3/RC2 === Vagrant Images === FreeBSD/amd64 images are available on the Hashicorp Atlas site, and can be installed by running: % vagrant init freebsd/FreeBSD-12.3-RC2 % vagrant up === Upgrading === The freebsd-update(8) utility supports binary upgrades of amd64 and i386 systems running earlier FreeBSD releases. Systems running earlier FreeBSD releases can upgrade as follows: # freebsd-update upgrade -r 12.3-RC2 During this process, freebsd-update(8) may ask the user to help by merging some configuration files or by confirming that the automatically performed merging was done correctly. # freebsd-update install The system must be rebooted with the newly installed kernel before continuing. # shutdown -r now After rebooting, freebsd-update needs to be run again to install the new userland components: # freebsd-update install It is recommended to rebuild and install all applications if possible, especially if upgrading from an earlier FreeBSD release, for example, FreeBSD 11.x. Alternatively, the user can install misc/compat11x and other compatibility libraries, afterwards the system must be rebooted into the new userland: # shutdown -r now Finally, after rebooting, freebsd-update needs to be run again to remove stale files: # freebsd-update install == ISO CHECKSUMS == o 12.3-RC2 amd64 GENERIC: SHA512 (FreeBSD-12.3-RC2-amd64-bootonly.iso) = de32c7a629e5d8a36ddc88438b7463b0d7ac7578b04344c536a80377fbdd920a21535ff9bb1f8a0a22a7f7c7ea022565470317c2b93b23a7bfade2d2a81d36e8 SHA512 (FreeBSD-12.3-RC2-amd64-bootonly.iso.xz) = d5372cba41c26e754508157c319aa2114a8a7e0bb4b6ef0df560797312cc4f67f543b40c6f50281a7bea88999a6f4decaf1380250345796024c2d96a11c43512 SHA512 (FreeBSD-12.3-RC2-amd64-disc1.iso) = 2fc2ee38aa416e40e47a85cb48d7f32987bdb70bbfc228c1ee40b8b085ec6739737a8b0f085377227790831984eded65a3b12e7ba16231048447f33fe5eb9488 SHA512 (FreeBSD-12.3-RC2-amd64-disc1.iso.xz) = ef7014e20de1c6841de4a543b54fa5e5fb12107b7440c6e7265e6322c65526b026066e4a6394dd90819bf9fc13417e7962272c596daf38015dc976039a469471 SHA512 (FreeBSD-12.3-RC2-amd64-dvd1.iso) = 58d098a1d65ce9908d27e0714a417e98f235ba77ec245e0504ca84a64ba9f7439285f967c5dbe4208fcbca331bbbb915db8833b8489bf3f503ddaa334f8d671d SHA512 (FreeBSD-12.3-RC2-amd64-dvd1.iso.xz) = 0f434d06ecdb306f0048059a38cd9cb54aaf1458188f00d8672a1eb1471fc7aa0ab7d283902b970a580175e1f48bdc2486c4f873bf315d58adb6052f31871e4b SHA512 (FreeBSD-12.3-RC2-amd64-memstick.img) = c5607e867763ef165a8a49b86fd2d4f6ab0d36d0bce874e529e2771240681d1afee0b5efa4611a5f959f056cae8b003ee9349a7269156c19e55da470c0189108 SHA512 (FreeBSD-12.3-RC2-amd64-memstick.img.xz) = 824e2100a52a47fddba83b64963d0a86b217b720452d00d567935c22adf47c784fd182e14d81d9be11f1037235ae5494cfab295b651d76d26618123dc046d3b9 SHA512 (FreeBSD-12.3-RC2-amd64-mini-memstick.img) = fe2290ba90257bdfc1328a0280379133ea17c436d83e91290ffb28865af49e4e8d2dde53e7bc0a5979805d1a03ba3f047d20af57b527a45e5086563a3850b9a0 SHA512 (FreeBSD-12.3-RC2-amd64-mini-memstick.img.xz) = eacfda59d3292f752e8743d13edfc281cc93977e746dce3295331cde3d775f6c47afb337d56751eecadadecfbb9c40084f171ed15d15eaa88db2109fc253c330 SHA256 (FreeBSD-12.3-RC2-amd64-bootonly.iso) = 46107b4af1d51ded799b92d65006051e4ab3b498b5c0869d467592fc2716e4c9 SHA256 (FreeBSD-12.3-RC2-amd64-bootonly.iso.xz) = 2403bbf39465c7c07fb1f1d88e69025630044dfbda167b764a8ec737ac37aa35 SHA256 (FreeBSD-12.3-RC2-amd64-disc1.iso) = a38b81845e6e0a83edf042bf9c2f05a8d3f46e5c1495509d5e9a25db04597a6e SHA256 (FreeBSD-12.3-RC2-amd64-disc1.iso.xz) = f34d70466448ebd0949b013bb2c57c190c69ef1130b5cb0958f4e5afb4f473b3 SHA256 (FreeBSD-12.3-RC2-amd64-dvd1.iso) = 974e8c538014c3579ffbeecd7a47ac8b053359c0ab4671d315a602a505721e44 SHA256 (FreeBSD-12.3-RC2-amd64-dvd1.iso.xz) = 5be5e61f460f1b7cd52697bdd69e7ab1ad5f5cff91500e1b32b27c73f1ce24e6 SHA256 (FreeBSD-12.3-RC2-amd64-memstick.img) = 15c3115315d77b65b33cfdea06320bc158574d6b1daa0e8e2806801032833014 SHA256 (FreeBSD-12.3-RC2-amd64-memstick.img.xz) = 8f16ab837835aebaa46fe52a0e80028f546375a29232ab5464fc51889ee53f24 SHA256 (FreeBSD-12.3-RC2-amd64-mini-memstick.img) = a10e59edf70c91a40e85b33970008b9d6f71706a59224ee282cf4b0851f48402 SHA256 (FreeBSD-12.3-RC2-amd64-mini-memstick.img.xz) = 8a18350ebe1a3c85da1e1d7d2b2e30fcc79a7ea8e94bb1ed3fdfc797c308f369 o 12.3-RC2 i386 GENERIC: SHA512 (FreeBSD-12.3-RC2-i386-bootonly.iso) = 9fda08bf84a0ee960ff82d7b33954ea93e1cd2dad36db26ea3c6ecec5d2328f036f05dcfb90a62c9067ee4b8a71f6d987d009017acccc5f79960fe92c55a6852 SHA512 (FreeBSD-12.3-RC2-i386-bootonly.iso.xz) = 9807974d13c6892c268521fba40b6d4bc25d89b3ea5deea354fcde25d5c87cf5dbe45332bf3f82a8b8bd42bb85289799af2188776605b6678c75d3facd63c9cf SHA512 (FreeBSD-12.3-RC2-i386-disc1.iso) = 7cdfbcc018476951d64a5cbf3326727158d2dd53350fd94bbe8ffd85f7e95b1b50cac05a66c534fc93286fb27a1d578de2fbc038087e502997df6863cfca6ef0 SHA512 (FreeBSD-12.3-RC2-i386-disc1.iso.xz) = 4a4150ea8f1020b8e90e33b6f33164e7cca72f23409f950d0fce40dae3c2e6624b02de396af737fc666ff27354e5ef863a47200a79d8b3c83316014301cb74c0 SHA512 (FreeBSD-12.3-RC2-i386-dvd1.iso) = bbb5f937e4050f22fcb468901acfaf216eef1289f88316a60d66271e2ce62b840102a0bf1c73845485b965011bcac4a55dc722673d5518efbe287a97368ff815 SHA512 (FreeBSD-12.3-RC2-i386-dvd1.iso.xz) = 2253cdaf539ffbb2cf4d158dc98fcf8846b435fcf4abc0f09d88a4cfdb3a8d85c5c16e72d146b77b759a669ffb315bb1bea8b2ac16b395fc6f441fb6687c1235 SHA512 (FreeBSD-12.3-RC2-i386-memstick.img) = ab75bcc35eb923c97f9569de526f77044c6212b32c36ca1bc5325c734914e67f6a0d6f80fb43a0e23b5e9b6754b92bf137d46019284b3521b948fb1194ee638d SHA512 (FreeBSD-12.3-RC2-i386-memstick.img.xz) = 156b44461a4511cb2bb37d8f7d66d02d5c2c6ec95738ab7e51e01267a0a0f8c2c39ac41f1c07344f5f4d6ecdef7bb5d69e709e593d77dd2c95c5347f08b9cd98 SHA512 (FreeBSD-12.3-RC2-i386-mini-memstick.img) = e1af1eaf1391336e4e1dad0c648799ebe4caad77362815b15e53ec3403ba6d929dba0bdef72a2e4db8fcd4c82c7c28be9c3836eefdc59f5bfd9592d2cbdac868 SHA512 (FreeBSD-12.3-RC2-i386-mini-memstick.img.xz) = 6986a4ffdeb57162611c33493820516f32e05c5e96b3937d87f1fcb420ef3ca98af6ccdf540865971799b6c49c430c54d247567fa710d64c6cb8cfb895c11de4 SHA256 (FreeBSD-12.3-RC2-i386-bootonly.iso) = cd210ec44fcb6db7cbbad2fd1f8b9b7d2679f0ed4f5486eb3d07e374b4aa5fc3 SHA256 (FreeBSD-12.3-RC2-i386-bootonly.iso.xz) = d798c51d196538d3e9780e9456a8f24f88f0b00bfbd575badd550373d4b43536 SHA256 (FreeBSD-12.3-RC2-i386-disc1.iso) = b639e04b91b8d2f031b4b01a18b3411c4626586af5a547c2fc19df4fd423aecb SHA256 (FreeBSD-12.3-RC2-i386-disc1.iso.xz) = ac115d19252c85ea816f19debd5b0dd614122a734552d351861bf43c75c1b185 SHA256 (FreeBSD-12.3-RC2-i386-dvd1.iso) = a0ad6caffee466c1f47c31515ccff1421f40f75e719fb1ad5453b63a38005479 SHA256 (FreeBSD-12.3-RC2-i386-dvd1.iso.xz) = 4ef2216cc699dcc8ef18da303dbc212b258d375b4f65f00aa9d3ebf608d94aac SHA256 (FreeBSD-12.3-RC2-i386-memstick.img) = b10ffd56220527bfd257db3372c43eed893c07984e819ba6fb5c2b72968338a0 SHA256 (FreeBSD-12.3-RC2-i386-memstick.img.xz) = bfc52e58dff13399c465a07d23e6ed412fe300244c88006baf60cf951c9defb0 SHA256 (FreeBSD-12.3-RC2-i386-mini-memstick.img) = 868891a49862249df6c21668529d45a37f51c96c9dae2dfb1c97cb8cd750096b SHA256 (FreeBSD-12.3-RC2-i386-mini-memstick.img.xz) = 77722cc3144a0d67d2a07be8d5c3e0aa2ba0487d7aeb81d8ff0c46e0c3f8963b o 12.3-RC2 powerpc GENERIC: SHA512 (FreeBSD-12.3-RC2-powerpc-bootonly.iso) = 95f29a9cde36b8bed99ac1298fb3422e4c71d0faf5d6001962adeb0db59b85dd6ce92c01a5ebf0a45398543806fe32d04112ac75fe4fb41834a16cd7229da444 SHA512 (FreeBSD-12.3-RC2-powerpc-bootonly.iso.xz) = 6fcf31e6ff767a9de4856c7299e7217df5b52570e4e2fc89ef58c4c5954269f2fc525423900140fb1c02897446f2423c6070561782ed2c52a48d881de09be549 SHA512 (FreeBSD-12.3-RC2-powerpc-disc1.iso) = 30ead440b4f6b5cea9f330bb891925b3f861790641a22dfd715e838d966062d2106d2aaf1d49a0d8ade71f1f9385b82607f4822264f6a7513bc33309acfffd51 SHA512 (FreeBSD-12.3-RC2-powerpc-disc1.iso.xz) = 0f695fffb28d637b3d956ec82408f2b868c22146bf390e3133433e1aebe3d26a4bc5c5ebd2622e8cdc143eb0c89dd69140115513e2482cd46aa2485d762ab8aa SHA512 (FreeBSD-12.3-RC2-powerpc-dvd1.iso) = ff7be48b903cfe7fd7f4718c228f17c51a8bacf37ebce5524e256025257f1fdb147db4d2d369eac640e3c30750f9764fb733fc1b37a9ee64d7d9c6cd0cb0941f SHA512 (FreeBSD-12.3-RC2-powerpc-dvd1.iso.xz) = c476324f7ef19defbe9b4a51d5c758fd8334e17f0fd234b26e5286b429718a9dcc3a55a4ff38d506088b69158e70ad31f44e3a14f28eebf60e54cce856be9dda SHA512 (FreeBSD-12.3-RC2-powerpc-memstick.img) = bc6edc1f00db0ed057777f8892c53736005656b6a0a8946391bc88dd6f529b54623ba7fd0ef0627381e896ccd5fb969a93fd44f8f832cb476c452fe6b707f0f4 SHA512 (FreeBSD-12.3-RC2-powerpc-memstick.img.xz) = f91e584fb31a29757c927075412c302aff28e39070c731fec22584b744fd040c2a3c1d0ceb8192668d47387ee5d2de834aaeee084979f3c7405e36e502b8c948 SHA512 (FreeBSD-12.3-RC2-powerpc-mini-memstick.img) = eb4089bbdbc4a693ce25bef2ed76d03030a6c3dafc7ee639a72ff9df02aa1b3ae699e8b106e8bc6f07d03310fc254310293cd823ac5710511456578c68faf479 SHA512 (FreeBSD-12.3-RC2-powerpc-mini-memstick.img.xz) = 7077ec7f85b45e9fc872931fff1674fcbd8f2faf68f9833aa9931380df26dc4b128f39b37d57e61ccce3a68bd7aef6d085f95ef19f87abc298146b67431ff982 SHA256 (FreeBSD-12.3-RC2-powerpc-bootonly.iso) = af72ad19b696c6b183d0285d2f1eb43a109f2c3cf0e28e1d03548c688e5b80d9 SHA256 (FreeBSD-12.3-RC2-powerpc-bootonly.iso.xz) = b190921439071ae0b3af7d27d61888e5b005aba54a45fa5a700071445af39506 SHA256 (FreeBSD-12.3-RC2-powerpc-disc1.iso) = d84323bdf582e497f7d135f7d14cb4d017c6daf79ca935522d2e943c707a91d9 SHA256 (FreeBSD-12.3-RC2-powerpc-disc1.iso.xz) = 197da49d5373f8c82fa8920b6af6d051d7be8035e3463df80bc855c81dfbd9e0 SHA256 (FreeBSD-12.3-RC2-powerpc-dvd1.iso) = fbd0c6512c3a84fb2b40af85fd772e27a290c336ea29f9891e82b9230e5b0868 SHA256 (FreeBSD-12.3-RC2-powerpc-dvd1.iso.xz) = 6d60be30f956729cd50e008ac0a3f9beeab39c26bf57e77652982398eef6e8ca SHA256 (FreeBSD-12.3-RC2-powerpc-memstick.img) = 6ec77e6453b37c98a1292b68921ff22873f6b43936308103b8180e62b2cf71c4 SHA256 (FreeBSD-12.3-RC2-powerpc-memstick.img.xz) = b48faf22d1765d4b3c54547fb66205c685206e08f2415d7aeb4ffb1551b939a9 SHA256 (FreeBSD-12.3-RC2-powerpc-mini-memstick.img) = c2012f75b6eb303e3498fe279b2c91809156415c2f43e6daae1391dde86cfc9b SHA256 (FreeBSD-12.3-RC2-powerpc-mini-memstick.img.xz) = 5ac945d419c7b4e29ee40caa978cabc54a465f11aefbbf70edc5c53570e7c9a5 o 12.3-RC2 powerpc64 GENERIC64: SHA512 (FreeBSD-12.3-RC2-powerpc-powerpc64-bootonly.iso) = 1ae8fb706f0461c726a740e8bd092c7784f7760836f2435a3e3214ac765de8544e6eb44d9d739c442113feb61ca677f241448893eb1a26fe142afcf68a67f7c2 SHA512 (FreeBSD-12.3-RC2-powerpc-powerpc64-bootonly.iso.xz) = f2b3d56564bd9da7c6e538bd176ed6c2e588992983a3b3f8198c132a7c0fe73e489f5c358203de143fed2402575bae63aa4c0d29897db85415e1d557fe79d688 SHA512 (FreeBSD-12.3-RC2-powerpc-powerpc64-disc1.iso) = 2cffb9b4567288eaab4feebb971de042bd17ab3d602bfff2fae1a883cdc32be7879e639a5e5130894a2bc7eb8c9781751ea3661c2d3a627ac756bdb2202761f1 SHA512 (FreeBSD-12.3-RC2-powerpc-powerpc64-disc1.iso.xz) = c2034b6cf67f4f988104fe4231a8543faf9679f57c8841bef24213ce53a24ed2f41c506e1b49e53f0d4bce1a60dd963e45b37793226650236b4767ee77d9075e SHA512 (FreeBSD-12.3-RC2-powerpc-powerpc64-dvd1.iso) = 0d49fd3e6705d103a797ffa87b662ae0cb192ce13f3a54f35d8bf9363b511d1dcba1f6dd5a84778387ab336b05bd9f046235039b9360702cde591f6992e713f7 SHA512 (FreeBSD-12.3-RC2-powerpc-powerpc64-dvd1.iso.xz) = f0cbc581462f2c500c268ef9c4d8b9c23e79a60c920f65afe2a3f9c29016284f40a67bd71444608993083f7c99b29c4c3db0819c935eadcbcfebad13ebef4dec SHA512 (FreeBSD-12.3-RC2-powerpc-powerpc64-memstick.img) = f66d50e66cd45d1756cdce10ed7a39260b41c2707df3cd2ff251522632ea659471fe507167b9219173c3910916bf40b0dea24216bc04d32f1527dd1d5981f97a SHA512 (FreeBSD-12.3-RC2-powerpc-powerpc64-memstick.img.xz) = f45580d70ed7604cf1ce9bf3a99ee0448ba769c265373d3ab235a105a695abb1cf20dfa6956441d5a7a1ee552ab77c4e2ec6ae3c33a2237d3c8449706bfeb920 SHA512 (FreeBSD-12.3-RC2-powerpc-powerpc64-mini-memstick.img) = 2beb14ffd75b420180e6bb1cf0931190956f7d5e8dcaa2df98e45d7378aa09c5878fb4fe18f99f69a3a6fde82d8e911dcbff307d703db99fa207f9039f0604b3 SHA512 (FreeBSD-12.3-RC2-powerpc-powerpc64-mini-memstick.img.xz) = 017ef6d661576b326400d74706b4bd81e5705fe96ae106af87e533732ea5b42ca6973a61f5f009fd912d1339d97a8f6902d0a5b3a11cdc8edfd4705490581c1b SHA256 (FreeBSD-12.3-RC2-powerpc-powerpc64-bootonly.iso) = 8e5b00fb9e0add582b3164f8b5e5f276d6b8807776340cbede2a2c8d51fb107d SHA256 (FreeBSD-12.3-RC2-powerpc-powerpc64-bootonly.iso.xz) = 6430cbe2002a410eefdd6481e9f76e60929c044c44630e71b4eb3e3fdb3f77c7 SHA256 (FreeBSD-12.3-RC2-powerpc-powerpc64-disc1.iso) = 840e8d0035024975f58a4052bafca5baa2b9d13653ee273392fbeb4aac2f0c16 SHA256 (FreeBSD-12.3-RC2-powerpc-powerpc64-disc1.iso.xz) = fcaa60249ba66c6c542d29ba13eed68d10468fe2006b0f9871fad3dcb4582c60 SHA256 (FreeBSD-12.3-RC2-powerpc-powerpc64-dvd1.iso) = 90af49b84b3939310e151c5cd73c4b532437e34a6b6247b97c9bfb0aa728074b SHA256 (FreeBSD-12.3-RC2-powerpc-powerpc64-dvd1.iso.xz) = 9f1ee9ed4c2e49e5f56c10f6a4ebe21a2d62723c5ff59c100b6483533cae04ec SHA256 (FreeBSD-12.3-RC2-powerpc-powerpc64-memstick.img) = 585f5a0ac539f8112a3d283ee26034b24ff9e603971f1c2073e50ba3dcc6587b SHA256 (FreeBSD-12.3-RC2-powerpc-powerpc64-memstick.img.xz) = 6cb3e942faa7da93b88d54a55f638e130e0da71854237a52c967e847f5dc707b SHA256 (FreeBSD-12.3-RC2-powerpc-powerpc64-mini-memstick.img) = 0202a035b812e0e2e997131e4a7615555ba1bfe30553e80c9303ec9671a99f37 SHA256 (FreeBSD-12.3-RC2-powerpc-powerpc64-mini-memstick.img.xz) = 0cea44f6925f2c1ff021eebc93c395d34634c663206c41dd050dfb50e83beff9 o 12.3-RC2 powerpcspe MPC85XXSPE: SHA512 (FreeBSD-12.3-RC2-powerpc-powerpcspe-bootonly.iso) = 98dc7588d7d7104143a6443ebb8c2a21cec7f3a50a58dc0a6c1f683ec5c6fa0e19b7ce16556d25e425c82728f0615bb9d1be3e28a46995d7f861814a8757c954 SHA512 (FreeBSD-12.3-RC2-powerpc-powerpcspe-bootonly.iso.xz) = 618650ca8886fdd19c51f8ad60bde2449659a93b78faaff7d96cf06bbab40419d8163d42e2808e0ba6bb235e3a22ec5162910e3efa59a705bc55ec7c5c41fe27 SHA512 (FreeBSD-12.3-RC2-powerpc-powerpcspe-disc1.iso) = 61a4b38633ff667f4c532244e4343ad8f0e9cdfa1281933176f8038566131a419a383a75b806c137f24dc96d763dd60b94c4e06b5b22c8f91fe714e0bfd912b6 SHA512 (FreeBSD-12.3-RC2-powerpc-powerpcspe-disc1.iso.xz) = 2ddffea0e586cc6f14b57277100d107f4263b58db21082f3b21d814aeeab4e0e998e11ae99ccdffe602f3ad2e397f090f70950c3e6de93ca73571261dd2074bd SHA512 (FreeBSD-12.3-RC2-powerpc-powerpcspe-dvd1.iso) = c798420ef3853bf9ed8d3f73f7b081528f598c6ade442dcc100367e7719bf64099e7f17e4601e345df202d8704797b509cd016987c96911aaa5616e4d7ac7c5b SHA512 (FreeBSD-12.3-RC2-powerpc-powerpcspe-dvd1.iso.xz) = 23e716b3b55890a416bfdfe26dceb90d676b22d99cf3b3fd172b13e4bd8bd7c4b4aeb62f9a9d2ec1e0f5c8c9e91f949a4ed8ca4fae7bc056dd3e3091ad0bc62d SHA512 (FreeBSD-12.3-RC2-powerpc-powerpcspe-memstick.img) = d8a31452feab6b68a9daaaee648fd41addfb57c7a8606caa72830f33f5205ad05590e9bf8b256fa9fb1afa75623932365b214a02005a69ef2b852c88c2caceda SHA512 (FreeBSD-12.3-RC2-powerpc-powerpcspe-memstick.img.xz) = a9678538ad26ce5993bb394ee0d8184f9e30a4413f05c0e9b19b147cb9acec5cbb995a16ae56865649f9f7eff54334f3f48d03558dd980d6a24db90902bea149 SHA512 (FreeBSD-12.3-RC2-powerpc-powerpcspe-mini-memstick.img) = bdff7edb868597573d454e98b86a8ad33045b1a71bdeb887a7d366db0b1e804a8194967a16ee8398863a1d4e854dea0881910637bd4bdcfaf38abf85baab5340 SHA512 (FreeBSD-12.3-RC2-powerpc-powerpcspe-mini-memstick.img.xz) = 0308b84107ecdc64b5d357dfb909b8f04494d23528ae0ebb598f8857236f6827abee520dbad6f4c19bb24386cb4150032d5f8c8f95b464ae5d1176681587eb15 SHA256 (FreeBSD-12.3-RC2-powerpc-powerpcspe-bootonly.iso) = b173cbedcf5ee058448ff443a03f6ad3cd1e8b7882900bc929028b754ee57602 SHA256 (FreeBSD-12.3-RC2-powerpc-powerpcspe-bootonly.iso.xz) = 60046d463b60739130daade9c2eb0acd851adf2a765f3d7cf87f5abf33089f8e SHA256 (FreeBSD-12.3-RC2-powerpc-powerpcspe-disc1.iso) = c1c781df395095c0f78526dd49989dfe667b799f661661fab68bd9108d1142a9 SHA256 (FreeBSD-12.3-RC2-powerpc-powerpcspe-disc1.iso.xz) = 03e10e367c40d97679c612c9f7adcb29bcb1154c52f25dcd5815f0be7971a1a5 SHA256 (FreeBSD-12.3-RC2-powerpc-powerpcspe-dvd1.iso) = e17f4402dcd3c84f5b129a4019747745023d767d24ab2d72fef97f29b8e3b75a SHA256 (FreeBSD-12.3-RC2-powerpc-powerpcspe-dvd1.iso.xz) = bd835166d4e4027a7627865c42a9e969e89b79c4f75d390ec80dba60e326651e SHA256 (FreeBSD-12.3-RC2-powerpc-powerpcspe-memstick.img) = 9f631be954f0e62be8af043e0079a21ab349e9c8b180fd49f66c835d7563a9ba SHA256 (FreeBSD-12.3-RC2-powerpc-powerpcspe-memstick.img.xz) = 280ef1ea25d59b20b5e67dd8ae48de8502d2c5999a926da9fd5cfc4a0b03a34a SHA256 (FreeBSD-12.3-RC2-powerpc-powerpcspe-mini-memstick.img) = 1ea858665927851db887450e90f11665900b8e463c24de86dc0a30f47e581598 SHA256 (FreeBSD-12.3-RC2-powerpc-powerpcspe-mini-memstick.img.xz) = 987bdf486f8f064677185fee27e9b902792f97a1e606b02568a86af8a79a795c o 12.3-RC2 sparc64 GENERIC: SHA512 (FreeBSD-12.3-RC2-sparc64-bootonly.iso) = 18105006ec6a41ddf4fa631d61cfebb4987cedc261378e89fb84748f1feacf4cee62fa00ac9b1ba73fb89b297edd2c8b9be3d35afe0fa0d293a9e952b32c52b0 SHA512 (FreeBSD-12.3-RC2-sparc64-bootonly.iso.xz) = b2b32caceb22a38fc9e9bba2af4dad8b20fa841f0bb274e7fe38752d876109cd9547616b3230f182938af0dd64ddfd1c7ca09fde77a5aaafb917b15614485ef0 SHA512 (FreeBSD-12.3-RC2-sparc64-disc1.iso) = f54210d66b139184f80dae6d5821327f3f5cf9662a4be2d7e7fc8b3c885d3611ab6218af5217ca7f574540e6a0dc6fba1980e5900959357d8069eb5ae47aa45c SHA512 (FreeBSD-12.3-RC2-sparc64-disc1.iso.xz) = c83ac8354f43f7bcca5b64081d4981851fc8d2885687e9bf64faa5af4e4fd8d10c4b1cc21e39a88aac6ab9db9e2251f7e20c5f06b30ad3d9aa61ed31b5c6aab6 SHA512 (FreeBSD-12.3-RC2-sparc64-dvd1.iso) = 23a5fbeb88be7d5c364735d61b18af46537fb6178693091df67eb62597cc5be818915e34e4c6f6f3d48da4b3f03e56554ae4f9fa4df7c3287eb748da5d8b9660 SHA512 (FreeBSD-12.3-RC2-sparc64-dvd1.iso.xz) = c645b1a9291bc4cdc65b0419f5e6cd2583a953af2ae24358aed652b20ec3c01c3bdb24ac886eaf4700d8c0c5e806421b95525107968661429088794594bf9faf SHA256 (FreeBSD-12.3-RC2-sparc64-bootonly.iso) = 096f993e03233a0ccce5456df6f8682b689ec2f2b866f41dc51a77b5615a0180 SHA256 (FreeBSD-12.3-RC2-sparc64-bootonly.iso.xz) = c08345d48d5cfe44970855d78adc6df25ec1b3a4c07e8d39a33e33128e8c79a4 SHA256 (FreeBSD-12.3-RC2-sparc64-disc1.iso) = d62fb1da1779d70a7fbf7f62bf3b968c4029afee209a30bb186553acd7452a94 SHA256 (FreeBSD-12.3-RC2-sparc64-disc1.iso.xz) = b746d0cb6352666fd097531d284d5031452fff2d1554f5d27d627ac334e43308 SHA256 (FreeBSD-12.3-RC2-sparc64-dvd1.iso) = 1f372bf74d1222eb09ccf06e33f35435daa8cf7bcd62c804a1137f68c5780870 SHA256 (FreeBSD-12.3-RC2-sparc64-dvd1.iso.xz) = af602b976deff858e1054c0c6a5630df7f634d1e40964a0f39458f3955ca20a7 o 12.3-RC2 armv6 RPI-B: SHA512 (FreeBSD-12.3-RC2-arm-armv6-RPI-B.img.xz) = ec2b89126e07589fcbf66ab9f1b148870399044d36825314a972f5875330141656d2c9a87564390efcfcd9404c6d93671456ba3ed4cb3d8037ba5e0d494be250 SHA256 (FreeBSD-12.3-RC2-arm-armv6-RPI-B.img.xz) = 2558ce7cb62bf792536cbda981c01dc3ef55b61d202729819572ef72b3d3313a o 12.3-RC2 armv7 BANANAPI: SHA512 (FreeBSD-12.3-RC2-arm-armv7-BANANAPI.img.xz) = a95b823d0a6af59b33f8395700d6ab6c5b384ad59756fcd138a1e88810abee8db8253d21ee7b39f62e29dd639990998891b598eb36b246a3ea76737055745c55 SHA256 (FreeBSD-12.3-RC2-arm-armv7-BANANAPI.img.xz) = 91cc48b3adf20399bad2e91c43a3bed26a47214c5b0e75903595d4cd5ce8ad78 o 12.3-RC2 armv7 CUBIEBOARD: SHA512 (FreeBSD-12.3-RC2-arm-armv7-CUBIEBOARD.img.xz) = 4f4bfb2573110e882aaf8449553c86dccc2eb20e37e241f492386827dad2687450a1419ac0ec4eb942b27c1b67f719359b2c98b1088d67fc8d0286b8984a9c7d SHA256 (FreeBSD-12.3-RC2-arm-armv7-CUBIEBOARD.img.xz) = 47a920af07eebb5b23fcf0ba03af8128a92d7e592e74e98722832fdfe553f6a9 o 12.3-RC2 armv7 CUBIEBOARD2: SHA512 (FreeBSD-12.3-RC2-arm-armv7-CUBIEBOARD2.img.xz) = da8eae4c38f9565581e93216654d742eef6e487be56e0054a7b1276ce09888fd3dad3c7dee6f36ed557f755a0d459a3223545898b5e4be1ea6493359923c39e6 SHA256 (FreeBSD-12.3-RC2-arm-armv7-CUBIEBOARD2.img.xz) = 470e7e143984821425b6b34a156bd73a82334c898be2319d70d0c352b6696426 o 12.3-RC2 armv7 CUBOX-HUMMINGBOARD: SHA512 (FreeBSD-12.3-RC2-arm-armv7-CUBOX-HUMMINGBOARD.img.xz) = d1e28629f98130961d55676c31c11d9a3cdc571fbe4d5d25c15f5f0dbc5038f077761f099530aa0c202ee919fa53d85429b1a6deaea18acdf706ad9068f0a1ad SHA256 (FreeBSD-12.3-RC2-arm-armv7-CUBOX-HUMMINGBOARD.img.xz) = 2790ffa5a2d18c369cfdc078e0baeff7a28eeef15d04d1b613bfb82cff3c9033 o 12.3-RC2 armv7 RPI2: SHA512 (FreeBSD-12.3-RC2-arm-armv7-RPI2.img.xz) = 61d101541ae4a0f01c669031d1b6ecac417cb3ffe9db5ef6b88249ae969a998440ce8d15a217796ab889bdf6bbd398e1de4b83a103fcd02ce31c73180d102b9f SHA256 (FreeBSD-12.3-RC2-arm-armv7-RPI2.img.xz) = d87b755990c8ce802e7745ec3732d16dfad1f51890e27183ab89ee6ac2cc919f o 12.3-RC2 armv7 WANDBOARD: SHA512 (FreeBSD-12.3-RC2-arm-armv7-WANDBOARD.img.xz) = d2716ad232d81aa48362e1ff2fef52532b25a60d9e04f3c8daa78025aff6801535cc0683d7b310b7bfe22d7872e81bcd0cadd1e373cf4a3a8acb43de2cd6e920 SHA256 (FreeBSD-12.3-RC2-arm-armv7-WANDBOARD.img.xz) = 7f74f4ff37a2bb1eea5c66119e7c904c4e40957841e2f4dd32c5c291f345b9ca o 12.3-RC2 armv7 GENERICSD: SHA512 (FreeBSD-12.3-RC2-arm-armv7-GENERICSD.img.xz) = df687d6626c53ec9cc38a11e7eb49fb90874e3e89e6532d8d5dde25aeca64e4c122118ac22727db58f057b47fbae0573a90d208165a5702fd889e88f732c0921 SHA256 (FreeBSD-12.3-RC2-arm-armv7-GENERICSD.img.xz) = a56d5cba4227e4c61c4a0e34a22a3f225373f1f76345643b742c6fa07966b09c o 12.3-RC2 aarch64 GENERIC: SHA512 (FreeBSD-12.3-RC2-arm64-aarch64-bootonly.iso) = e8d6b7202f40b5edb64fa97c675df1758fc8337a3600797f021baf8845a75aa781ad1f7952468102f03e6102abdf6d750631801735e5b142a3b74da80d295b23 SHA512 (FreeBSD-12.3-RC2-arm64-aarch64-bootonly.iso.xz) = cac6ca86f1526c66660aab1b4bb2d86eeef2399e5c6464b475f93154f13f636fbd695f87de6d006497fca7e750e7289fc3eb3f32eaf177df582a6e85f3c263b2 SHA512 (FreeBSD-12.3-RC2-arm64-aarch64-disc1.iso) = 04f00358a9cb621c3978182fe9e005b1deec956810acdfc4f2618fcff2429b9f43a9c386ff94f3cc570fffc4d163fd4f85258d62479803ff118307f0634b23f7 SHA512 (FreeBSD-12.3-RC2-arm64-aarch64-disc1.iso.xz) = 21cb29694a617d42d0397ba62246af4c5923be747ef983ea56da2a06d13d6bd53a7243c83115e3c8e86eaa1d23a95f3043d12b701eb08357bb99e228078b4a30 SHA512 (FreeBSD-12.3-RC2-arm64-aarch64-dvd1.iso) = f91ae2b22d8061613d35e11f8aaa22b855c60289dde7b851ccd62301cc14ad4477f05755a70127c741ac026274c3d586268fa1143532343769559af916c0a652 SHA512 (FreeBSD-12.3-RC2-arm64-aarch64-dvd1.iso.xz) = 76d3649d274f85fe79d22d116094e564f15663510bed7229d99dd7b82241e2ecc5587af17263bf9409b8fea769b3786e4f68eee8b1dd1263619fa836356bdcab SHA512 (FreeBSD-12.3-RC2-arm64-aarch64-memstick.img) = e1b6ece72762d80c07cdd25b6c685f43fce7cd0c946f2e0c0e4dfb9c0d7d0934c51ab526e5720627695b4f64403f28dd9a90c000032cc312d4ca555a3e1c23c5 SHA512 (FreeBSD-12.3-RC2-arm64-aarch64-memstick.img.xz) = 2a03dfd2b7a47d1406992256afdcbe4bd896ac4a9605d6289be31d39f13f78301008abd4956353514d50f1fb528501492bcdcbeff0b986adaedf124bf1557052 SHA512 (FreeBSD-12.3-RC2-arm64-aarch64-mini-memstick.img) = 41bf83b4106918fbf176921ae26d30a3f4cd7345b1fbe64f99b66708e8bf2e0bf20092f957d3738b2685f57254ce251edb33744fc3c6866fb8421deb87a17aad SHA512 (FreeBSD-12.3-RC2-arm64-aarch64-mini-memstick.img.xz) = 07b317589bb394206a2e7ba0ea616fa871da39fcf6e4ebac3413fcfe1562718f3f4d117dacec132872c76819f2d987e7f2a0b92a4b1275f13804ad20e013f2c9 SHA256 (FreeBSD-12.3-RC2-arm64-aarch64-bootonly.iso) = 2334f63591cc69d5e38541de50be95e68a53cdb9addcdeef3bc8641beaa9ff01 SHA256 (FreeBSD-12.3-RC2-arm64-aarch64-bootonly.iso.xz) = 318d31a86022f5c8795ca851caff5416a534831ba090b77b34b6e28b0d8268ef SHA256 (FreeBSD-12.3-RC2-arm64-aarch64-disc1.iso) = ab4b8bf36e1032bb989f75f24fea7d86d978cc6a61448278b77619ba31c612f1 SHA256 (FreeBSD-12.3-RC2-arm64-aarch64-disc1.iso.xz) = 46ff4dcaa580c9a5536e41ed6d2870b54cd5509c97915cd037d2df423b6f022f SHA256 (FreeBSD-12.3-RC2-arm64-aarch64-dvd1.iso) = 1b9f260c635cf12b7748a7b9eaf01a7eaa1b119d1dea296e8bb41b4a822e78ba SHA256 (FreeBSD-12.3-RC2-arm64-aarch64-dvd1.iso.xz) = 0050f526b76f477c36f16cbb27febd0c2e7ca0cddd0e0b7ed722e23576ff27c8 SHA256 (FreeBSD-12.3-RC2-arm64-aarch64-memstick.img) = a144850d84b288e1db6726775c9183b5c8815f770d360083efef48211e33dd08 SHA256 (FreeBSD-12.3-RC2-arm64-aarch64-memstick.img.xz) = b8c0b8f607210749ca6268d72bb7b37c02675e31137239ad243af6ccd5aa6b59 SHA256 (FreeBSD-12.3-RC2-arm64-aarch64-mini-memstick.img) = 75bdab9b328681e1c081830f6b661621fd9e98b3f86c9cd83c92c07b5d97c07e SHA256 (FreeBSD-12.3-RC2-arm64-aarch64-mini-memstick.img.xz) = 0294fae8eca6057228d3c9f0d3685db4f82262edb590a216e99518a5b9194cf1 o 12.3-RC2 aarch64 PINE64: SHA512 (FreeBSD-12.3-RC2-arm64-aarch64-PINE64.img.xz) = 3b6c2414908d14a92ef2fa9220cda7d74e3d59ad6eff4538d54518499340b630363f3f770a55b58b864db8e8672d2b940a86b7996370f2b8c5df5bab00bd3d64 SHA256 (FreeBSD-12.3-RC2-arm64-aarch64-PINE64.img.xz) = 4eb23428c4cfe541e3d39bb0bc27624b9694c46902100f8731f42528f514c915 o 12.3-RC2 aarch64 PINE64-LTS: SHA512 (FreeBSD-12.3-RC2-arm64-aarch64-PINE64-LTS.img.xz) = aeaa531bb6cec1b9be49767225020d0b670f0ad6cdf64878e4c7f8cf8e621ef3432f242133dca439f1d1ba067dc98d59856f7d6a213f0d93e1105031654a3ee8 SHA256 (FreeBSD-12.3-RC2-arm64-aarch64-PINE64-LTS.img.xz) = 13e21b9536703498580ce7e4ca8b7f2aebfb1a57cd0e000b82c70101f2a489b8 == VM IMAGE CHECKSUMS == o 12.3-RC2 amd64: SHA512 (FreeBSD-12.3-RC2-amd64.qcow2.xz) = 1e5e636e8a5b6c7cb5c2c0579bcaff32c70080b1e3ff20c1b09ef8bd329ad5f201afcfc31c6c3e799623a6344e5db27fefb8cee154172d0a5269a625e0529d90 SHA512 (FreeBSD-12.3-RC2-amd64.raw.xz) = d9791c0e6efedc0a1fbad1d523a26d04d6f3e01056fe3bdaf47a447c00b98300a240ddcd019a9e87637b8991e13077d435186deb42c1780cabc6f587e58c5ffc SHA512 (FreeBSD-12.3-RC2-amd64.vhd.xz) = 34d3e574d8961c4c6344b203f7c7e3da8e1999d46462cc5a3265843014ae93772238e0e7d5ad7e4a527be78b62803e3d4fc6286d77d9c6c342967a99abc0a5d1 SHA512 (FreeBSD-12.3-RC2-amd64.vmdk.xz) = b92c55d197f946809006d9eb834e5794e537278e0558a7801e035c1612a76193385da761ccf6294feb7b21a77518d115da197b25c6c142d6e02c3709a5cc38ca SHA256 (FreeBSD-12.3-RC2-amd64.qcow2.xz) = e7115c1f9b9f3d350be12c106ed9d3147a973ee11c7fde19e5109a77ade17ce0 SHA256 (FreeBSD-12.3-RC2-amd64.raw.xz) = 98c18d84032a9d8cb53463715a5775b81638254d7ee727cc8c732a7f028af2cb SHA256 (FreeBSD-12.3-RC2-amd64.vhd.xz) = 284892a138117a485c9596c787a50016d0bef85d6bb12cd9dd08fc90f0b9f755 SHA256 (FreeBSD-12.3-RC2-amd64.vmdk.xz) = 7099d8d1a5834ef192236ca3a6f538fd03072c082ff8a37691d307d58c099b17 o 12.3-RC2 i386: SHA512 (FreeBSD-12.3-RC2-i386.qcow2.xz) = 24c8b3871e1912a5e3c72eab7f455b6676da5a7989ecd54dfc47b20f8de147e4346174eb67c82d6fdde93070d8ea9b73e2e43e9e485ad09f2a8289660a2bce20 SHA512 (FreeBSD-12.3-RC2-i386.raw.xz) = 568dbe7d82e1bd4418bbdc3af40eea24c50eb8834a2435257a658a5f4d7a2d086eac8b47b88ca6c845ec0755cddb4f25591b844d3283dd0fa042302da019ba12 SHA512 (FreeBSD-12.3-RC2-i386.vhd.xz) = 52fe1916a85cecb452912c962da8d23da0b4d9771526f3ef35f193e804234ea25a1ed1e7246cc894c55a68e763bcff80a95bc7aba2dcb587fca31d7e60d1a8f6 SHA512 (FreeBSD-12.3-RC2-i386.vmdk.xz) = dfb7970db9923ccd1f21b62bff9e0a9df9613b23881cdd9fee34db5a2ef874d059f3c8071a11cbcf0fd23436b1069c4c34ea66782a972c790e2e02cade4b2f72 SHA256 (FreeBSD-12.3-RC2-i386.qcow2.xz) = 72c9acf63a7b1f6ae784f7939f4d9e89c4477149802185487d7ddbbe075df518 SHA256 (FreeBSD-12.3-RC2-i386.raw.xz) = 184c9d64ea99183a2a4aa895fbd9459859bca65b30ae9e0aaef1dfe239764d09 SHA256 (FreeBSD-12.3-RC2-i386.vhd.xz) = 8a577eb08c99a427bbc21dde9e9b8f6dbe95ff1fa42c4a6be7640c96425439e1 SHA256 (FreeBSD-12.3-RC2-i386.vmdk.xz) = 02005e5d829214b9cce7b3b07061d1c6f36685fe058f7bef8ace1ff5ec05c472 o 12.3-RC2 aarch64: SHA512 (FreeBSD-12.3-RC2-arm64-aarch64.qcow2.xz) = 0a9640f4f9d8cec21258faa64221e6aa843061dbcad575fe055c2b27f1ba6909bed033bd4acd2ec2e1b918d3bc887280e482920b2693fe7287e1630fc7aba848 SHA512 (FreeBSD-12.3-RC2-arm64-aarch64.raw.xz) = 425458f73202278dd57438c0e89c7be0d415233c880845f2443f80278e55fa624e2a031e7ea24a650b3bc0a09feb197b0edbf858083ec6590df7394c4d5d3425 SHA512 (FreeBSD-12.3-RC2-arm64-aarch64.vhd.xz) = 6b743b982ba29808e93f586b4fb7ffff7f28daf461a425c4cbec61e05c515e645c0702ba9e3f80ae82ba510af133293567e5f3247123027cb6c618b043e89cdc SHA512 (FreeBSD-12.3-RC2-arm64-aarch64.vmdk.xz) = a7b0c9d524c888afcafc54433ae21763b3758cc16bb44c323905c73d607a51d42c4e86d909bec2915c595d0140d8683fdaab91fc12cad730d3806d85300c60db SHA256 (FreeBSD-12.3-RC2-arm64-aarch64.qcow2.xz) = a5b2c47f7d7bc22be5ce8e3684b7d88dcec14e84ecdf03764e978661d9fdbb04 SHA256 (FreeBSD-12.3-RC2-arm64-aarch64.raw.xz) = e106e412f36b86c6dbd161408e177fdeba5098458a0b89d7225d74fb751e98e3 SHA256 (FreeBSD-12.3-RC2-arm64-aarch64.vhd.xz) = 77b6cdd195f53ec6c29583281d6261d200b3a31556e467f542e91e9a38cdd390 SHA256 (FreeBSD-12.3-RC2-arm64-aarch64.vmdk.xz) = 8ad9964d7484154efe98ba8bf5179486ffdaeef5e0671031733c16b69ec3e63f Regards, Glen Love FreeBSD? Support this and future releases with a donation to the FreeBSD Foundation! https://www.freebsdfoundation.org/donate/ -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEjRJAPC5sqwhs9k2jAxRYpUeP4pMFAmGW+v0ACgkQAxRYpUeP 4pPdzg//f613ml8hyjf5MgF0MM/Hx+dQQKFXNFlQZ79pE5cY1zcEqinwhMmuulfM QdnKV2Ed6NAoahCYSoIEcw9T6IQ72B+La3EbK6Rxo6ysc8g2r7fWS70hYi01k1S1 KgXu57gKXFWx8gSL4zFf7WW6Y9nqx7eNzcUS1hxNNq7YjfgqnkvK3eW+F6TSeu4x FV10jmU0N3YuoLBT7Eu1oY3HSXh8ULwA/DB0K0PyEPIo20FcyErp+gi2M1+O8N9I e0z9WCdhQdcWEwDSbbpzUXHHvTdl+w8dHS0X98LULoVaG96H0O1quQJgJ3+FguHI EZ8lD90cV9w/xfhkaRW5cUfo5V1ey7akyU/UBJywJpduiuMQmE1N0xWYGwofK5bt kGmVT3Rxns9VVVqqBiZi0a+eRZW8V0PAEgrQAnJ4MRBpTjHPyFylWSncLrHGhBWo biVwBnvd0dWcLzJ5zD2+nF0vg4ltBnJScz1CTl5ujJhXjwPGtVwhEW9XLZvEiRmt XLDWPkYpad4ZiEHzzjSk5g2hMUdBeIadSiMvJxcRkiDOP2kU+/J9povG31SSUJBj JcBAKsdMpHyG6U1GEosZh6ykMqFmeH2GFiR2+yvYSjnr+01kjCfJrdAajYLo06k+ f4e4Gmw4bFRc5slSV1YkktDekBvhT7C+5bUxoHrJIdKF9WSZTTE= =UIqU -----END PGP SIGNATURE----- From nobody Fri Nov 19 23:38:09 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4136C189D6A8 for ; Fri, 19 Nov 2021 23:45:35 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.org (uucp.dinoex.org [IPv6:2a0b:f840::12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "uucp.dinoex.sub.de", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HwtZj6Pn3z3DYR for ; Fri, 19 Nov 2021 23:45:33 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.org [185.220.148.12]) by uucp.dinoex.org (8.17.1/8.17.1) with ESMTPS id 1AJNj4PR096879 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Sat, 20 Nov 2021 00:45:04 +0100 (CET) (envelope-from pmc@citylink.dinoex.sub.org) X-MDaemon-Deliver-To: X-Authentication-Warning: uucp.dinoex.sub.de: Host uucp.dinoex.org [185.220.148.12] claimed to be uucp.dinoex.sub.de Received: (from uucp@localhost) by uucp.dinoex.sub.de (8.17.1/8.17.1/Submit) with UUCP id 1AJNj4Y0096877 for freebsd-stable@freebsd.org; Sat, 20 Nov 2021 00:45:04 +0100 (CET) (envelope-from pmc@citylink.dinoex.sub.org) Received: from gate.intra.daemon.contact (gate-e [192.168.98.2]) by citylink.dinoex.sub.de (8.16.1/8.16.1) with ESMTP id 1AJNdxEP057283 for ; Sat, 20 Nov 2021 00:39:59 +0100 (CET) (envelope-from peter@gate.intra.daemon.contact) Received: from gate.intra.daemon.contact (gate-e [192.168.98.2]) by gate.intra.daemon.contact (8.16.1/8.16.1) with ESMTPS id 1AJNc9o6056040 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Sat, 20 Nov 2021 00:38:09 +0100 (CET) (envelope-from peter@gate.intra.daemon.contact) Received: (from peter@localhost) by gate.intra.daemon.contact (8.16.1/8.16.1/Submit) id 1AJNc9Jl056039 for freebsd-stable@freebsd.org; Sat, 20 Nov 2021 00:38:09 +0100 (CET) (envelope-from peter) Date: Sat, 20 Nov 2021 00:38:09 +0100 From: Peter To: freebsd-stable@freebsd.org Subject: 12.3-RC1 fails to compile perl, tcl86 (dtrace error) Message-ID: List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Milter: Spamilter (Reciever: uucp.dinoex.sub.de; Sender-ip: 185.220.148.12; Sender-helo: uucp.dinoex.sub.de;) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (uucp.dinoex.org [185.220.148.12]); Sat, 20 Nov 2021 00:45:07 +0100 (CET) X-Rspamd-Queue-Id: 4HwtZj6Pn3z3DYR X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of pmc@citylink.dinoex.sub.org designates 2a0b:f840::12 as permitted sender) smtp.mailfrom=pmc@citylink.dinoex.sub.org X-Spamd-Result: default: False [0.70 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[4]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; NEURAL_SPAM_SHORT(1.00)[1.000]; NEURAL_SPAM_LONG(1.00)[1.000]; DMARC_NA(0.00)[sub.org]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:205376, ipnet:2a0b:f840::/32, country:DE]; RCVD_TLS_LAST(0.00)[] X-ThisMailContainsUnwantedMimeParts: N Hija, I get a strange error when trying to build perl or tcl from ports: dtrace: failed to establish error handler: "/usr/lib/dtrace/ipfw.d", line 106: failed to copy type of 'inp': Conflicting type is already defined That file ipfw.d appears to be new in 12.3, but I'm clueless what the error means (and why it happens only to me). cheerio, PMc From nobody Sat Nov 20 00:51:59 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 379881899DF8 for ; Sat, 20 Nov 2021 01:00:31 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.org (uucp.dinoex.org [IPv6:2a0b:f840::12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "uucp.dinoex.sub.de", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HwwFB0Fvqz3rqq for ; Sat, 20 Nov 2021 01:00:29 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.org [185.220.148.12]) by uucp.dinoex.org (8.17.1/8.17.1) with ESMTPS id 1AK1048i053258 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Sat, 20 Nov 2021 02:00:04 +0100 (CET) (envelope-from pmc@citylink.dinoex.sub.org) X-MDaemon-Deliver-To: X-Authentication-Warning: uucp.dinoex.sub.de: Host uucp.dinoex.org [185.220.148.12] claimed to be uucp.dinoex.sub.de Received: (from uucp@localhost) by uucp.dinoex.sub.de (8.17.1/8.17.1/Submit) with UUCP id 1AK104Rb053256 for freebsd-stable@freebsd.org; Sat, 20 Nov 2021 02:00:04 +0100 (CET) (envelope-from pmc@citylink.dinoex.sub.org) Received: from gate.intra.daemon.contact (gate-e [192.168.98.2]) by citylink.dinoex.sub.de (8.16.1/8.16.1) with ESMTP id 1AK0sxD1076813 for ; Sat, 20 Nov 2021 01:54:59 +0100 (CET) (envelope-from peter@gate.intra.daemon.contact) Received: from gate.intra.daemon.contact (gate-e [192.168.98.2]) by gate.intra.daemon.contact (8.16.1/8.16.1) with ESMTPS id 1AK0pxMW076371 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Sat, 20 Nov 2021 01:52:00 +0100 (CET) (envelope-from peter@gate.intra.daemon.contact) Received: (from peter@localhost) by gate.intra.daemon.contact (8.16.1/8.16.1/Submit) id 1AK0pxBE076370 for freebsd-stable@freebsd.org; Sat, 20 Nov 2021 01:51:59 +0100 (CET) (envelope-from peter) Date: Sat, 20 Nov 2021 01:51:59 +0100 From: Peter To: freebsd-stable@freebsd.org Subject: 12.3-RC1 errors on boot and shutdown Message-ID: List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Milter: Spamilter (Reciever: uucp.dinoex.sub.de; Sender-ip: 185.220.148.12; Sender-helo: uucp.dinoex.sub.de;) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (uucp.dinoex.org [185.220.148.12]); Sat, 20 Nov 2021 02:00:07 +0100 (CET) X-Rspamd-Queue-Id: 4HwwFB0Fvqz3rqq X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of pmc@citylink.dinoex.sub.org designates 2a0b:f840::12 as permitted sender) smtp.mailfrom=pmc@citylink.dinoex.sub.org X-Spamd-Result: default: False [0.69 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[4]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; NEURAL_SPAM_SHORT(0.99)[0.986]; NEURAL_SPAM_LONG(1.00)[1.000]; DMARC_NA(0.00)[sub.org]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:205376, ipnet:2a0b:f840::/32, country:DE]; RCVD_TLS_LAST(0.00)[] X-ThisMailContainsUnwantedMimeParts: N Hija, when we're already at it: there are errors reported during startup and shutdown. In startup: > xargs: not found This one comes from /etc/rc.d/rctl. Consequentially, rctl rules will not be loaded. The flaw was always there, but (for whatever reason) /etc/rc.d/rctl is now run very early, *before* mountcritlocal. And xargs lives in /usr/bin - it doesn't exist before mountcritlocal. In shutdown: LOTS of "cannot umount, filesystem busy" These come from change #367546 - we actually try stop ZFS now - but we never bothered to stop syslogd, sendmail, linux-emulation and who knows what else. So these daemons sit in various filesytems, or special filesystems are mounted upon (in the case of linux), and the umounts fail. This has no consequences (as we didn't even try to do it before), but looks as if something were broken and would't cleanly stop. cheerio, PMc From eugen@grosbein.net Sat Nov 20 09:20:01 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 2CF121895150 for ; Sat, 20 Nov 2021 09:20:48 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Hx7LR6jsDz4klN for ; Sat, 20 Nov 2021 09:20:47 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221] (may be forged)) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id 1AK9KF2S010216 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 20 Nov 2021 09:20:15 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: pmc@citylink.dinoex.sub.org Received: from [10.58.0.10] (dadvw [10.58.0.10]) by eg.sd.rdtc.ru (8.16.1/8.16.1) with ESMTPS id 1AK9KCnF094628 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Sat, 20 Nov 2021 16:20:12 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: 12.3-RC1 errors on boot and shutdown To: Peter , freebsd-stable@freebsd.org References: From: Eugene Grosbein Message-ID: <18e95eb9-deef-7102-9349-44a18ffffc35@grosbein.net> Date: Sat, 20 Nov 2021 16:20:01 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,SHORTCIRCUIT autolearn=disabled version=3.4.2 X-Spam-Report: * -0.0 SHORTCIRCUIT No description available. * -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 4Hx7LR6jsDz4klN X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N 20.11.2021 7:51, Peter wrote: > Hija, > > when we're already at it: there are errors reported during startup > and shutdown. > > > In startup: > >> xargs: not found > > This one comes from /etc/rc.d/rctl. Consequentially, rctl rules will > not be loaded. > > The flaw was always there, but (for whatever reason) /etc/rc.d/rctl > is now run very early, *before* mountcritlocal. > And xargs lives in /usr/bin - it doesn't exist before mountcritlocal. I've just fixed rctl part in HEAD: https://cgit.freebsd.org/src/commit/?id=0c54fe172ad365e7e60d6249484a7579c18b7d2d I will commit the fix to stable branches soon, but stable/12 will get it after the release as it is too late. From nobody Sat Nov 20 10:54:20 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 12EBE188BB1D for ; Sat, 20 Nov 2021 10:54:50 +0000 (UTC) (envelope-from pi@freebsd.org) Received: from fc.opsec.eu (fc.opsec.eu [IPv6:2001:14f8:200:4::4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Hx9Qy02lYz3jng for ; Sat, 20 Nov 2021 10:54:50 +0000 (UTC) (envelope-from pi@freebsd.org) Received: from pi by fc.opsec.eu with local (Exim 4.95 (FreeBSD)) (envelope-from ) id 1moO0i-0000zT-Dx; Sat, 20 Nov 2021 11:54:20 +0100 Date: Sat, 20 Nov 2021 11:54:20 +0100 From: Kurt Jaeger To: Eugene Grosbein Cc: Peter , freebsd-stable@freebsd.org Subject: Re: 12.3-RC1 errors on boot and shutdown Message-ID: References: <18e95eb9-deef-7102-9349-44a18ffffc35@grosbein.net> List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <18e95eb9-deef-7102-9349-44a18ffffc35@grosbein.net> X-Rspamd-Queue-Id: 4Hx9Qy02lYz3jng X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N Hi! > >> xargs: not found > > > > This one comes from /etc/rc.d/rctl. Consequentially, rctl rules will > > not be loaded. > > > > The flaw was always there, but (for whatever reason) /etc/rc.d/rctl > > is now run very early, *before* mountcritlocal. > > And xargs lives in /usr/bin - it doesn't exist before mountcritlocal. > > I've just fixed rctl part in HEAD: > https://cgit.freebsd.org/src/commit/?id=0c54fe172ad365e7e60d6249484a7579c18b7d2d > > I will commit the fix to stable branches soon, but stable/12 will get it after the release as it is too late. Why is it too late ? I thought that's what the RCs are for ? -- pi@FreeBSD.org +49 171 3101372 Now what ? From eugen@grosbein.net Sat Nov 20 11:46:39 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 74B82188906E for ; Sat, 20 Nov 2021 11:47:18 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HxBbV157Wz4Wlp; Sat, 20 Nov 2021 11:47:17 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221] (may be forged)) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id 1AKBkpIX011270 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 20 Nov 2021 11:46:51 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: pi@freebsd.org Received: from [10.58.0.10] (dadvw [10.58.0.10]) by eg.sd.rdtc.ru (8.16.1/8.16.1) with ESMTPS id 1AKBkor4096103 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Sat, 20 Nov 2021 18:46:50 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: 12.3-RC1 errors on boot and shutdown To: Kurt Jaeger References: <18e95eb9-deef-7102-9349-44a18ffffc35@grosbein.net> Cc: Peter , freebsd-stable@freebsd.org From: Eugene Grosbein Message-ID: <8d016162-1f7a-3d6b-9995-8a6f88b37d9f@grosbein.net> Date: Sat, 20 Nov 2021 18:46:39 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,SHORTCIRCUIT autolearn=disabled version=3.4.2 X-Spam-Report: * -0.0 SHORTCIRCUIT No description available. * -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 4HxBbV157Wz4Wlp X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-Spam: Yes X-ThisMailContainsUnwantedMimeParts: N 20.11.2021 17:54, Kurt Jaeger wrote: > Hi! > >>>> xargs: not found >>> >>> This one comes from /etc/rc.d/rctl. Consequentially, rctl rules will >>> not be loaded. >>> >>> The flaw was always there, but (for whatever reason) /etc/rc.d/rctl >>> is now run very early, *before* mountcritlocal. >>> And xargs lives in /usr/bin - it doesn't exist before mountcritlocal. >> >> I've just fixed rctl part in HEAD: >> https://cgit.freebsd.org/src/commit/?id=0c54fe172ad365e7e60d6249484a7579c18b7d2d >> >> I will commit the fix to stable branches soon, but stable/12 will get it after the release as it is too late. > > Why is it too late ? I thought that's what the RCs are for ? Releng team announced that only critical fixes are accepted after RCs. This fix is not critical as it's easy to fix the script locally until official update. From nobody Sat Nov 20 19:43:31 2021 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 2A0D3188A6D8 for ; Sat, 20 Nov 2021 19:43:33 +0000 (UTC) (envelope-from ltning-freebsd-stable@anduin.net) Received: from mail.anduin.net (mail.anduin.net [185.42.170.45]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4HxP902Y4Nz3QLs for ; Sat, 20 Nov 2021 19:43:32 +0000 (UTC) (envelope-from ltning-freebsd-stable@anduin.net) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=anduin.net; s=dkim2021; h=Content-Transfer-Encoding:MIME-Version:Content-Type:References: In-Reply-To:Date:To:From:Subject:Message-ID:Sender:Reply-To:Cc:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=nFMAfZYrrVExEn92Y4ueGIkZFJZohOgMAUR96cx/R00=; t=1637437412; x=1638301412; b=qTDBbOVsahZkRz9f2ZcHtu9M4aWaN1o2v4odkMEwrXZSKkCs9/Ng+CPTCSaFwb554tFVfZVqO1+ pjmzurkxT0sjhuiZ/inYPwcuWr8QME+RZuANikJglbxfbRooWpTtsVcuo9Rmf7earC1OX70nGJmI+ p6gt9AOSCVYXXP92Mvroh4uUcA1y1H+22qPIEV2Z0hjx951y11knWzkuB+9xl8fOcl+x00aOSLJde VehdPMiaA+pihOOJieJEW+YutXEdyR2bWinjUP/WQc2Uu3Ppl2whDEN2hfrhPqEfNGukpgoAjh5zw toJX6asHh3/CDSMhtNFo38aZPhoBFhZUHW2g==; Received: by mail.modirum.com with esmtpsa (TLS1.3:TLS_AES_256_GCM_SHA384:256) (Exim 4.94.2 (FreeBSD)) (envelope-from ) id 1moWGp-00050L-CM for stable@freebsd.org; Sat, 20 Nov 2021 19:43:31 +0000 Message-ID: <5d3f822320f1f7d202b64a9ad9aaf6cb1731a36c.camel@anduin.net> Subject: Re: 12.3-RC1 errors on boot and shutdown From: Eirik =?ISO-8859-1?Q?=D8verby?= To: stable@freebsd.org Date: Sat, 20 Nov 2021 20:43:31 +0100 In-Reply-To: <8d016162-1f7a-3d6b-9995-8a6f88b37d9f@grosbein.net> References: <18e95eb9-deef-7102-9349-44a18ffffc35@grosbein.net> <8d016162-1f7a-3d6b-9995-8a6f88b37d9f@grosbein.net> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.42.1 FreeBSD GNOME Team List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-SA-Authenticated: Yes X-Rspamd-Queue-Id: 4HxP902Y4Nz3QLs X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=anduin.net header.s=dkim2021 header.b=qTDBbOVs; dmarc=none; spf=pass (mx1.freebsd.org: domain of ltning-freebsd-stable@anduin.net designates 185.42.170.45 as permitted sender) smtp.mailfrom=ltning-freebsd-stable@anduin.net X-Spamd-Result: default: False [2.50 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[anduin.net:s=dkim2021]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[anduin.net]; NEURAL_SPAM_MEDIUM(1.00)[1.000]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_SPAM_SHORT(1.00)[1.000]; DKIM_TRACE(0.00)[anduin.net:+]; NEURAL_SPAM_LONG(1.00)[1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:62248, ipnet:185.42.170.0/24, country:NO]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On Sat, 2021-11-20 at 18:46 +0700, Eugene Grosbein wrote: > 20.11.2021 17:54, Kurt Jaeger wrote: > > Hi! > > > > > > > xargs: not found > > > > > > > > This one comes from /etc/rc.d/rctl. Consequentially, rctl rules will > > > > not be loaded. > > > > > > > > The flaw was always there, but (for whatever reason) /etc/rc.d/rctl > > > > is now run very early, *before* mountcritlocal. > > > > And xargs lives in /usr/bin - it doesn't exist before mountcritlocal. > > > > > > I've just fixed rctl part in HEAD: > > > https://cgit.freebsd.org/src/commit/?id=0c54fe172ad365e7e60d6249484a7579c18b7d2d > > > > > > I will commit the fix to stable branches soon, but stable/12 will get it after the release as it is too late. > > > > Why is it too late ? I thought that's what the RCs are for ? > > Releng team announced that only critical fixes are accepted after RCs. > This fix is not critical as it's easy to fix the script locally until official update. wait....what? This means we *can not* upgrade our 100+ systems upon release, because we cannot at the same time roll out local changes to basic stuff in /etc/rc.d .. (my 2 cents) /Eirik From eugen@grosbein.net Sat Nov 20 19:57:06 2021 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id C64ED189230B for ; Sat, 20 Nov 2021 19:57:27 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HxPT32d3pz3mJF for ; Sat, 20 Nov 2021 19:57:27 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221] (may be forged)) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id 1AKJvIFT013379 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 20 Nov 2021 19:57:18 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: ltning-freebsd-stable@anduin.net Received: from [10.58.0.10] (dadvw [10.58.0.10]) by eg.sd.rdtc.ru (8.16.1/8.16.1) with ESMTPS id 1AKJvHPc099085 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Sun, 21 Nov 2021 02:57:17 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: 12.3-RC1 errors on boot and shutdown To: =?UTF-8?Q?Eirik_=c3=98verby?= , stable@freebsd.org References: <18e95eb9-deef-7102-9349-44a18ffffc35@grosbein.net> <8d016162-1f7a-3d6b-9995-8a6f88b37d9f@grosbein.net> <5d3f822320f1f7d202b64a9ad9aaf6cb1731a36c.camel@anduin.net> From: Eugene Grosbein Message-ID: Date: Sun, 21 Nov 2021 02:57:06 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 In-Reply-To: <5d3f822320f1f7d202b64a9ad9aaf6cb1731a36c.camel@anduin.net> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,SHORTCIRCUIT autolearn=disabled version=3.4.2 X-Spam-Report: * -0.0 SHORTCIRCUIT No description available. * -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 4HxPT32d3pz3mJF X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N 21.11.2021 2:43, Eirik Øverby wrote: >> Releng team announced that only critical fixes are accepted after RCs. >> This fix is not critical as it's easy to fix the script locally until official update. > > wait....what? > This means we *can not* upgrade our 100+ systems upon release, because > we cannot at the same time roll out local changes to basic stuff in > /etc/rc.d .. You could roll out local changes before upgrade, because the upgrade itself does not touch the script. From eugen@grosbein.net Sat Nov 20 21:41:22 2021 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 2CE7E189C17F for ; Sat, 20 Nov 2021 21:41:43 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HxRnL6hv4z4mfb for ; Sat, 20 Nov 2021 21:41:42 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221] (may be forged)) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id 1AKLfYlM013811 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 20 Nov 2021 21:41:35 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: ltning-freebsd-stable@anduin.net Received: from [10.58.0.10] (dadvw [10.58.0.10]) by eg.sd.rdtc.ru (8.16.1/8.16.1) with ESMTPS id 1AKLfYaU000457 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Sun, 21 Nov 2021 04:41:34 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: 12.3-RC1 errors on boot and shutdown To: =?UTF-8?Q?Eirik_=c3=98verby?= , stable@freebsd.org References: <18e95eb9-deef-7102-9349-44a18ffffc35@grosbein.net> <8d016162-1f7a-3d6b-9995-8a6f88b37d9f@grosbein.net> <5d3f822320f1f7d202b64a9ad9aaf6cb1731a36c.camel@anduin.net> From: Eugene Grosbein Message-ID: Date: Sun, 21 Nov 2021 04:41:22 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 In-Reply-To: <5d3f822320f1f7d202b64a9ad9aaf6cb1731a36c.camel@anduin.net> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,SHORTCIRCUIT autolearn=disabled version=3.4.2 X-Spam-Report: * -0.0 SHORTCIRCUIT No description available. * -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 4HxRnL6hv4z4mfb X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N 21.11.2021 2:43, Eirik Øverby wrote: >>>> I've just fixed rctl part in HEAD: >>>> https://cgit.freebsd.org/src/commit/?id=0c54fe172ad365e7e60d6249484a7579c18b7d2d > wait....what? > This means we *can not* upgrade our 100+ systems upon release, because > we cannot at the same time roll out local changes to basic stuff in > /etc/rc.d .. With approval of re@, the change's merged to stable/12 and will be included in the 12.3-RELEASE, so be cool. From nobody Sun Nov 21 16:27:46 2021 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id C944C18A71BA for ; Sun, 21 Nov 2021 16:27:50 +0000 (UTC) (envelope-from ltning-freebsd-stable@anduin.net) Received: from mail.anduin.net (mail.anduin.net [185.42.170.45]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Hxwmh56yTz3w0W for ; Sun, 21 Nov 2021 16:27:48 +0000 (UTC) (envelope-from ltning-freebsd-stable@anduin.net) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=anduin.net; s=dkim2021; h=Content-Transfer-Encoding:MIME-Version:Content-Type:References: In-Reply-To:Date:To:From:Subject:Message-ID:Sender:Reply-To:Cc:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=sh/Qs9Xx7kG6HygY6ThdsoQ8Kbh3zukfdcIjpIUv4BM=; t=1637512068; x=1638376068; b=iZ1Ua16mWo3hpf1ElNqI0vy3ltACexg8fgvw1FmB2RwGiQE8aZ72yNPu2U+hvEbc4qRs+aV2GHI 1nGo+3T+ubIsv/xc///glb9IUECqRinoOIeex+QaY00A+iXR6WhI5/CRZ01ru4UhgyJGTZVq1f0Ve R4wbsUVbmabBn91Poyo12k91zAOYju5iKawqJfvyiKmwuBGblYaUsj9wQE00EcZi8G8pQOlA4+y6V sR5n6J/wqf8pMI3C5U9V6M3lGfVlcSnUG2drGCXkS4K28h8IXg03DoHxSn1evp7zUrQbS7pyK06ND BRqJ2yefa/FFaDZHSYHQA8yuYJlUOP9hnhlQ==; Received: by mail.modirum.com with esmtpsa (TLS1.3:TLS_AES_256_GCM_SHA384:256) (Exim 4.94.2 (FreeBSD)) (envelope-from ) id 1mopgw-000MZr-Lp for stable@freebsd.org; Sun, 21 Nov 2021 16:27:46 +0000 Message-ID: Subject: Re: 12.3-RC1 errors on boot and shutdown From: Eirik =?ISO-8859-1?Q?=D8verby?= To: stable@freebsd.org Date: Sun, 21 Nov 2021 17:27:46 +0100 In-Reply-To: References: <18e95eb9-deef-7102-9349-44a18ffffc35@grosbein.net> <8d016162-1f7a-3d6b-9995-8a6f88b37d9f@grosbein.net> <5d3f822320f1f7d202b64a9ad9aaf6cb1731a36c.camel@anduin.net> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.42.1 FreeBSD GNOME Team List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SA-Authenticated: Yes X-Rspamd-Queue-Id: 4Hxwmh56yTz3w0W X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=anduin.net header.s=dkim2021 header.b=iZ1Ua16m; dmarc=none; spf=pass (mx1.freebsd.org: domain of ltning-freebsd-stable@anduin.net designates 185.42.170.45 as permitted sender) smtp.mailfrom=ltning-freebsd-stable@anduin.net X-Spamd-Result: default: False [2.50 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[anduin.net:s=dkim2021]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[anduin.net]; NEURAL_SPAM_MEDIUM(1.00)[1.000]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_SPAM_SHORT(1.00)[1.000]; DKIM_TRACE(0.00)[anduin.net:+]; NEURAL_SPAM_LONG(1.00)[1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:62248, ipnet:185.42.170.0/24, country:NO]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On Sun, 2021-11-21 at 04:41 +0700, Eugene Grosbein wrote: > 21.11.2021 2:43, Eirik Øverby wrote: > > > > > > I've just fixed rctl part in HEAD: > > > > > https://cgit.freebsd.org/src/commit/?id=0c54fe172ad365e7e60d6249484a7579c18b7d2d > > > wait....what? > > This means we *can not* upgrade our 100+ systems upon release, because > > we cannot at the same time roll out local changes to basic stuff in > > /etc/rc.d .. > > With approval of re@, the change's merged to stable/12 and will be included in the 12.3-RELEASE, so be cool. Thank you. I was just .. momentarily put off by the additional work it would have entailed. If I came across as uncool, I'm sorry. Thanks again, /Eirik From nobody Sun Nov 21 19:26:25 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 480E2189FC49 for ; Sun, 21 Nov 2021 19:26:39 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic315-55.consmr.mail.gq1.yahoo.com (sonic315-55.consmr.mail.gq1.yahoo.com [98.137.65.31]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Hy0l215k2z4bYD for ; Sun, 21 Nov 2021 19:26:38 +0000 (UTC) (envelope-from marklmi@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1637522791; bh=UQXeru2+VtYhSe62lDjtgn7qwcxkI16gxbDrTs7O/3M=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=jYLcS5yvlrorKav5xt2eaI+IV3ZK2wnUW+wRFCeedeECSkFeScjLvbjr3RE2+TVnwnLMogitnv+JXXFJPcfA94BVRI046Lbcvxu4rhRM4nVv+NdnknTs3Ogc5ASuZkj9jQX6ulqCOtUF5+dPSCdiLTaOoQUyI1HpQxWtWSS0phICqkggH7cMLZHUSi6zUk2BH+NMzyLzP2o2RdMr4PMwmP4TXi+d/fjLX0Jobeexh0W/XNnCovZPzJBouBEpqZztETChy0a47i/sKvSwUImB04Z7p275ELPREzXHfsyRgZ5+ILJMFgrd85V5M1XiPrZnSStjpoCdZU5fPUEUpWG3zg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1637522791; bh=tSPbfGSl5NkfgvWFLMoKCiKuA0Oo9bXR2eKz5SDV0ah=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=fzKxifOaoKO9MLQMb2jIoafcg+X0ZdzIsq5ljRf7tVvGbSzWNpNj/nfqjoZV7Nl9tuzuplNQuH5UBjqh+NvvHIKLXQdG1MBywxJOGhX7uQ368p7KhBLy/cHbXV3FsDpPl2TVUZmYeSNZ5D6PnPeQHnRwnmz+J/2D3n3peUXwC3lxjH0w//705G/M2Q5MIDjT7vjzbeF1tSRChRRlFx1p9pjpVSOgBWM6yKjce4NTn6PqNPLAdWybvSjOE9Ks361EKYTfjcXu1UMVXwtTmQqT52/gTizu+iFuJO+CkP/kwchNOesEzJC1ROJjd2rAZkMvVQgNha9m+gA117HRRBRqhw== X-YMail-OSG: v5DMxQkVM1noRgNNjILvi7QZmFInriymCgyBrPhxjRuZp3ovoJqCqhazA_YI0Kr QoIqaFqF84._6vjET1YxDkpT89RjvKR4EZMlMM5SQ3tKLWPKjROm9SVKGUsZc5FGeHsx6hXiE_Sj L69ZPnVo0H8sBTOF3Y08gwfVXeYSDuK7W9A8kCj02IHRvz8vsIpO7d5DQlIO.FXcQb1pcn1QobhN 3UGOc1l.DZTmzAy_pwZM80i64GrhK85xz5nAFiiSP_TIQO5KgBK7.gWFrxp0Gbxiz4AqAFwotryh 4JhqvZffaPMYuklrhrfhB.wqsk1z5KZYAXgDhnl6gc6NO8Gfznvsifwlz.KphkaeLqp3UgP_rZEd WG2rUdGaJhcsIdmre1MEtnrKwHXr1DLzat5TsGzQ9vjwLz.6X5yx1XHAiv2Fb0eRrGITkk7FiGaE Ko1fnCcnpVlficm4JNGmubp2M3nzPQdiR6uI3opovXD5xcytNqsEuZusmf8WKvR05R6UPKwf8CS_ wy_evR.Zryk5SNB2nWWiWX2GIii9lZ6V2IiU3bFDJ9AhopWR_3Y6onJKzYtPPwKI.g5tNFVvjH8i VN_EcYdN4MV3Ho50O3o48RZgtw2mek7di48mJfN6Tl_k85gGbGIJF7Om6bkdpFDEY7srFVmFcsnF oTKTljBuiKzAxGrRh3cCDvkoDA9_L.efFUN.qs0dbehrftS1HTb.OJud8W6F48vJ8tSKc7mzNkSa Ah8yK4e0CToUr5qFpdgqGhucfL7NaQ2GwrNZpN2rmKirY.oc27qoWWsIdHX_nLZwhnfmBny8lRy9 kvI09sV2tKseZDfyjXTORinC2lgsv6gkkBPoxixlP0Kl.4X_AJxL.6EOgdw0JyTmGh6iLlaXyLH9 BVlCRIxZ3zr7c3ZMCp0dpjHjeyEjWafu7xxnWJlRZBi3.ittlem5vpraRh1wK9Z4IWN.VERUjHr1 x1F3rzovA_E99IjrABfqKO.oor1JVWeyuAijCF8hx1Gps3nAe7pMIA60NGvdPhDMD_Aqz2JyMOEq u_XiMab1YY2Weg7lB.Uzyi6WZ35Ba7qaxEFnEYHyRpWuL7MPOgrm2754VvdpHs69RUhiWJz0kp_k l0SE0YaMHwn2xum1oG1m8MKtqfkptPyOqCf2h20CCWOCTN7EsQ5lXr0uFyeIq0yeGdvBVfAH_30z ZK_eIrNKCujXfBAjijpU75PUMdfw0Xlm5rTp5MzAL1uw8G0Sb6Sd3SFojgpCdX5CMFsYiIOAveqe BCeoVJv9q.PCYt.HMSMMwzHTNhGXWWdJ6nntXq5bChsdlld90zmHhQsaM8uBNJsgNxdxIauE3WxM WpcCLeszbltSsRUtoW1SF.JDTJ9tdef66WsL5pmfkomwEn97NCt1agjpFWfn2TYpSUcKEAv6Pdc1 6V7olqIewK.DQptUn.UpYtt_dKHjVhSvxm9jnnogAH3dAXkOliIL3rJC3b47mgfTcR5DThMgAWBl q7tyAvjcXvxuywCnBvV2uO3UrGoFcA89LiZ6k4bAf7irRsaqAQC8o624Dbeai0V8pZHEA8gdcx9b W21Y3G3hqRxvx1NEV7_SaWHXyW5YF6Gnbjwt7Kv0VJ1y4t0ByxfRBvYYu9egT5DjIxlkg4mK6EA. H_2K27lz181h5MocN._nj4Fr5Zln6wpu5DLR.sor1UJphjOH52ZGo9OaXnGTzwJ0dmikg27DdCZs iDqMw3U3uJDPO0pjmpEDKkiU3Gco2lRR.7jsP7V34yttVsUqBW41iHV3sEtnmKuNAoa5o4fsLkem ku2zG7RJO9fS8SLaImC49VX61rT3JWoMgKIwhJ2hqSDGXYyoaASKtoh3pRaienVjppDBPEInxTDY RsevXAPY4xz9o73YQNROn03c6XzEvlrLlzuF3dF74nGu_CF8RG9NijKum3WOV9wKijKHmquGRKgv UukkPWWzsgg5p4JsDn51Ic2fou98RVRO2pWE8xeBLJm6kYgl_H2iKDr_u.oFcfG4Uv6kHQr.n0P4 ky8q7qvee55Wt38JOM8tK6k.jODoqOHGyuBs74dHux_92pTNoL74YnSKyBcF0fAN2.102e.XxyP1 6G8reheReBLaoD6fTwo45Dm2E1v1zEfLUBC2kIOIMg7Tpbsp.q2kcRbrvqPnT1FOC.omzPgggGsb 6BuiSQcDisgQoe1ofp.cZmOxuVjUspgOfCSrgIdwKiNS26B8on6udzVO1MysAkkUCekisX1Mzm9u OlYlggfPxKao7wdIPHXNp020nNKFkgBri1HeHTg.FvcO6wb.8gqeXqVvebpSSHwS51R7bK8rYoiy OPHql5ZqJf3mTkvOHTFZQiZw- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic315.consmr.mail.gq1.yahoo.com with HTTP; Sun, 21 Nov 2021 19:26:31 +0000 Received: by kubenode503.mail-prod1.omega.bf1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 691f5a682ff34022694d036e6e5adf00; Sun, 21 Nov 2021 19:26:28 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) Subject: FYI: aarch64 boot (HoneyComb): example crash during system checks Message-Id: Date: Sun, 21 Nov 2021 11:26:25 -0800 To: "freebsd-arm@freebsd.org" , FreeBSD-STABLE Mailing List X-Mailer: Apple Mail (2.3654.120.0.1.13) References: X-Rspamd-Queue-Id: 4Hy0l215k2z4bYD X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=jYLcS5yv; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.65.31 as permitted sender) smtp.mailfrom=marklmi@yahoo.com X-Spamd-Result: default: False [-3.50 / 15.00]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.31:from]; FROM_HAS_DN(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.31:from]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_COUNT_TWO(0.00)[2]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim] Reply-To: marklmi@yahoo.com From: Mark Millard via arm X-Original-From: Mark Millard X-ThisMailContainsUnwantedMimeParts: N Starting file system checks: /dev/gpt/CA72opt0EFI: 41 files, 242 MiB free (15469 clusters) FIXED /d x0: ffff000000e43ec8 (blocked_lock + 0) x1: ffff00013efa9f50 x2: ffff00000090e39a (cam_status_table + 1d132) x3: deadc0d8 x4: 0 x5: ffff00000082a138 (data_abort + 0) x6: 5 x7: 601 x8: ffff000000e43ec8 (blocked_lock + 0) x9: deadc0de x10: 0 x11: 3938700 x12: 0 x13: 8000 x14: 1de x15: 81ce x16: 425b9080 x17: 8000 x18: ffff00013efa9f40 x19: ffff000000e43ec8 (blocked_lock + 0) x20: ffffa0001a826000 x21: 0 x22: ffffa0001a826000 x23: 0 x24: ffff000000bed000 (queue_ops + 0) x25: 98967f x26: ffff000000e43ee0 (blocked_lock + 18) x27: 0 x28: 114 x29: ffff00013efa9f40 sp: ffff00013efa9f40 lr: ffff0000004b9028 (thread_lock_flags_ + c0) elr: ffff0000004b9028 (thread_lock_flags_ + c0) spsr: 2c5 far: deadc178 esr: 96000004 timeout stopping cpus panic: data abort in critical section or under mutex cpuid =3D 5 time =3D 1637492224 KDB: stack backtrace: db_trace_self() at db_trace_self_wrapper+0x30 pc =3D 0xffff000000807770 lr =3D 0xffff00000011d9ec sp =3D 0xffff00013efa9990 fp =3D 0xffff00013efa9b90 db_trace_self_wrapper() at vpanic+0x188 pc =3D 0xffff00000011d9ec lr =3D 0xffff0000004e1d10 sp =3D 0xffff00013efa9ba0 fp =3D 0xffff00013efa9c00 vpanic() at panic+0x44 pc =3D 0xffff0000004e1d10 lr =3D 0xffff0000004e1b84 sp =3D 0xffff00013efa9c10 fp =3D 0xffff00013efa9cc0 panic() at data_abort+0x290 pc =3D 0xffff0000004e1b84 lr =3D 0xffff00000082a3c8 sp =3D 0xffff00013efa9cd0 fp =3D 0xffff00013efa9d50 data_abort() at handle_el1h_sync+0x78 pc =3D 0xffff00000082a3c8 lr =3D 0xffff00000080a078 sp =3D 0xffff00013efa9d60 fp =3D 0xffff00013efa9eb0 handle_el1h_sync() at thread_lock_flags_+0xbc pc =3D 0xffff00000080a078 lr =3D 0xffff0000004b9024 sp =3D 0xffff00013efa9ec0 fp =3D 0xffff00013efa9f40 thread_lock_flags_() at thread_lock_flags_+0xbc pc =3D 0xffff0000004b9024 lr =3D 0xffff0000004b9024 sp =3D 0xffff00013efa9f50 fp =3D 0xffff00013efa9f60 thread_lock_flags_() at sleepq_timeout+0x10 pc =3D 0xffff0000004b9024 lr =3D 0xffff00000054b2a8 sp =3D 0xffff00013efa9f70 fp =3D 0xffff00013efa9fb0 sleepq_timeout() at softclock_call_cc+0x14c pc =3D 0xffff00000054b2a8 lr =3D 0xffff000000503134 sp =3D 0xffff00013efa9fc0 fp =3D 0xffff00013efaa020 softclock_call_cc() at callout_process+0x17c pc =3D 0xffff000000503134 lr =3D 0xffff000000502df0 sp =3D 0xffff00013efaa030 fp =3D 0xffff00013efaa0a0 callout_process() at handleevents+0x188 pc =3D 0xffff000000502df0 lr =3D 0xffff00000045b42c sp =3D 0xffff00013efaa0b0 fp =3D 0xffff00013efaa100 handleevents() at timercb+0x304 pc =3D 0xffff00000045b42c lr =3D 0xffff00000045be7c sp =3D 0xffff00013efaa110 fp =3D 0xffff00013efaa170 timercb() at arm_tmr_intr+0x5c pc =3D 0xffff00000045be7c lr =3D 0xffff0000007ff850 sp =3D 0xffff00013efaa180 fp =3D 0xffff00013efaa1d0 arm_tmr_intr() at intr_event_handle+0xac pc =3D 0xffff0000007ff850 lr =3D 0xffff000000493c54 sp =3D 0xffff00013efaa1e0 fp =3D 0xffff00013efaa1e0 intr_event_handle() at intr_isrc_dispatch+0x70 pc =3D 0xffff000000493c54 lr =3D 0xffff0000007fb238 sp =3D 0xffff00013efaa1f0 fp =3D 0xffff00013efaa230 intr_isrc_dispatch() at arm_gic_v3_intr+0x11c pc =3D 0xffff0000007fb238 lr =3D 0xffff00000080ff34 sp =3D 0xffff00013efaa240 fp =3D 0xffff00013efaa250 arm_gic_v3_intr() at intr_irq_handler+0x7c pc =3D 0xffff00000080ff34 lr =3D 0xffff0000007faff0 sp =3D 0xffff00013efaa260 fp =3D 0xffff00013efaa2b0 intr_irq_handler() at handle_el1h_irq+0x74 pc =3D 0xffff0000007faff0 lr =3D 0xffff00000080a140 sp =3D 0xffff00013efaa2c0 fp =3D 0xffff00013efaa3f0 handle_el1h_irq() at handle_el1h_sync+0x78 pc =3D 0xffff00000080a140 lr =3D 0xffff00000080a078 sp =3D 0xffff00013efaa400 fp =3D 0xffff00013efaa500 handle_el1h_sync() at handle_el1h_sync+0x78 pc =3D 0xffff00000080a078 lr =3D 0xffff00000080a078 sp =3D 0xffff00013efaa510 fp =3D 0xffff00013efaa660 handle_el1h_sync() at sched_switch+0x6a8 pc =3D 0xffff00000080a078 lr =3D 0xffff0000005197fc sp =3D 0xffff00013efaa670 fp =3D 0xffff00013efaa6f0 sched_switch() at sched_switch+0x6a8 pc =3D 0xffff0000005197fc lr =3D 0xffff0000005197fc sp =3D 0xffff00013efaa700 fp =3D 0xffff00013efaa790 sched_switch() at mi_switch+0xf4 pc =3D 0xffff0000005197fc lr =3D 0xffff0000004f03a0 sp =3D 0xffff00013efaa7a0 fp =3D 0xffff00013efaa7f0 mi_switch() at sleepq_timedwait+0x28 pc =3D 0xffff0000004f03a0 lr =3D 0xffff00000054bd0c sp =3D 0xffff00013efaa800 fp =3D 0xffff00013efaa830 sleepq_timedwait() at _cv_timedwait_sbt+0x110 pc =3D 0xffff00000054bd0c lr =3D 0xffff00000045e7b0 sp =3D 0xffff00013efaa840 fp =3D 0xffff00013efaa850 _cv_timedwait_sbt() at dbuf_evict_thread+0x410 pc =3D 0xffff00000045e7b0 lr =3D 0xffff0000013ca59c sp =3D 0xffff00013efaa860 fp =3D 0xffff00013efaa8f0 dbuf_evict_thread() at fork_exit+0x94 pc =3D 0xffff0000013ca59c lr =3D 0xffff00000048fbf0 sp =3D 0xffff00013efaa900 fp =3D 0xffff00013efaa950 fork_exit() at fork_trampoline+0x10 pc =3D 0xffff00000048fbf0 lr =3D 0xffff000000828ed8 sp =3D 0xffff00013efaa960 fp =3D 0x0000000000000000 KDB: enter: panic [ thread pid 26 tid 100194 ] Stopped at kdb_enter+0x48: undefined f906411f For reference: # uname -apKU FreeBSD CA72_16Gp_ZFS 13.0-STABLE FreeBSD 13.0-STABLE #13 = stable/13-n248062-109330155000-dirty: Sat Nov 13 23:55:14 PST 2021 = root@CA72_16Gp_ZFS:/usr/obj/BUILDs/13S-CA72-nodbg-clang/usr/13S-src/arm64.= aarch64/sys/GENERIC-NODBG-CA72 arm64 aarch64 1300520 1300520 It is a root-on-ZFS context on Optane media in the PCie slot. I've no clue if this will repeat. I've never gotten this before. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From nobody Sun Nov 21 19:36:08 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id D9FC618A44D5 for ; Sun, 21 Nov 2021 19:36:16 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic301-22.consmr.mail.gq1.yahoo.com (sonic301-22.consmr.mail.gq1.yahoo.com [98.137.64.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Hy0y73Vr1z4fjW for ; Sun, 21 Nov 2021 19:36:15 +0000 (UTC) (envelope-from marklmi@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1637523374; bh=0P97hs2LFopknVvakMpaMypTVHAek2r71RRxDoiAKMM=; h=From:Subject:Date:References:To:In-Reply-To:From:Subject:Reply-To; b=nBiK3Tct6a5z3Hd/JAiJCiS/kRSLDqz83JxP4Y3bvutaTB6l9PttJSWFI79xvG+fvRfig5/SnrYfe3/kgg77ff8qOxvelodNtmJNU27T9w89RsbQyu8t07aC4j9bgkc3xE6HOFgC5LqfxHJtaJ5kSRUm3dYb1AmijZLgFPkbN8zl9jSutGN6fZyq/fpYhbXs8rlZb5bTwKmpLx3n+4W6RnQIVCZ+y9pvH5aqBf5QMlF6gXPZQl5oDnzfHOQkWRTzKM8SUEVV7RlkPzoNA9RWBr4RhchjLAp3jGxout9omD/bJOHcDPzKbSUG5xrdAXCaAiplhLVEq3zHqv5BeSzMgw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1637523374; bh=lGi2VPVOFCy9Hm8DqPbAljjuy+k5FguqCZJaaRiUuJz=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=HXgXlvPBC2ZIJGhCQvxIBfizR5tUnkAZohuT2HC7Lsnp3qyBErlJl6dhiclRAEYq9ouZBXnX3SkTRe98DzxnIDqsW44qPKUss2Kus1JJgzpj/u4KJWKQWPq1ujQaeDbNl5rr9WbbOfmxjTAc8XmgVufKbk+HoEiUdxHyp+98XidJYC+eUeUGi4Q3iwB38ngR/clYRi9Co9JK3yP/ATHyfHNttBEp3PEFbK4JQf0sJKyQmbb/MO8mp12oa08jH2wJ+S1p2sg2BX5DiXMtlNYSfnSEVCK3iM5NsNZ+akyUl3HsabXYrqnvdro/HysM6e/UBf5G5UAGI9EMfe4SgwSuJw== X-YMail-OSG: nFblz9cVM1l2mPmfh2dv6OxJPk900BRd9BXyvj3.ILI9zz_w1OmxSl7f2JKddui vl_4leqGMtWz4JEsEmXvbfnH4feRWgYkyxcuQVgoQXe_ax.kMQvD.X6oGOGi03bUcoWF4bLTbUyB X8n3yqBDKIOhyY0BeWQXSJNUhp2lATMCWHFFUCwdYy3a4mjsbYgtbQ4WsdGHU_FHAs4ozypFVm9W BVfpTH_F808qJ88SwjadYNMUchxSvresekGdBYmULHmO5QTeIuLmcTO_XcoziBZiS6PZzMZbbfNi OAwng_jeKhEdyPoIOgw_bRsJtekhLh5hhX9tQr65gEgrVHKxHneD9syqZOZUSdr6tzBWpK3RIHio pE3MFFpHgaLJuJjrQSAXkRr6CrcC8XJoz65MrMz_oJJiQ2zUukzR7JPXfLSB4AWFnwQPPYz4fNNC 9ChLvdRk20cDe32rzd7q4EW02EvD.WYl_g2iz3YnA17z2V9_9Gozj6fiZG98aYxSpnqgDzJcWL7m M0Q1uP2u3QJBYvrlccCxRso_HYhqLLv1HDnn9QEq1lDKe..KF6R_ZdMncDL6Jie6W4hKUtEnf_og 0xzeQRSZsWlbv7WeN_9_1g6qiSf49jjeozcrj1eI8qRke5TwJQ.YqRVLJDoq89HpQbsqxRclkaih C78YJBdvm9a5a4grHTLwYDeZLB1q8yBJzTAB64CdQDFy8Y92vm.j3wXAOIamt75irOd6r_DnpEtH k7WkjdGjiwAbifT2yNgyFL60RD5d7G2vqZRsWmWY_wCPZrFm5Vwe_IBWZFzg_77OQfEefy9d8MA4 1gYg_yKmuFhba_aM_WT8qSE2YfXYUVkjSAHZo5ZXygBgOdjbY2T2JCOBDHQstbgNxoo2gi_fuu2p 16DtY480CHVMCP3nvlcUJz72cl9.nI6Fj1Habare5AHMlYc3eLuqmKAhBuLdWz6UQQ.WWuROQre1 LO3uaZeqcsPuGfoIGIw2Fj8VZvGZh27k2GuXsG5xxLQQPk88JCenzCjTWhONf5ANCjbBrTnbisZ9 3iejqmw7Bvmfo3n5W.LoSZdOkrFM.mBGCYFKblb2qmFF0VEN9I8tN3AftII2ESAUwfs9Z9zmeUW7 0GwOQsMl8UPiqtBUdd9Dw4HHQE9H_C99t0KSpCz1wVHjgNDc0kGrLBSTKKiEew9DuSgKb_gC0PFs xQDlzfludRrmZuHMWQddYh.zIogyBOjmV7qDCimBoFVftaR8BxFm0m53YXJPL0o17sMpvKt8QXX6 RSRWlVc3fRx5Or2XmEAV0KHiiyOd0tc1GryFjA.tpsrYi7BRBFZC0B9lY2DCX2DadG9hkUlruVqD H1mYWf_Jw13ue4oyMhr8h08OfNTZ2gr3_YAls9s1LB6zwutV_hSz9g_xYTvyUPKOfaeorATP7jEw UahOfabkwh6MnZEllBLPFNQX2wsSZK488bUk9CsK0CJaU0vKgDYGJNzuQHO8utql6kP5vY2gsA1l OJ2t.5AmqRmAJFT3doKRlXAtngmkrw20Agw2RzxdKgUnQbj0t00y6l3fjWMsTxuAUaiuOVWI0ojt XjBRjm3b.msRLTzL4JgKczGSWyfX00AO.GjO93wgxK_.pdZlzNyuoawR06HZhxPDS3M1JLQYHTTY G6oIBnmQE5w_pSSjsyoNalVTOx_dq02OWHlI2hZWPfVnO2mRKdTB4PJdIyOndT6TNes8SSi4IFdN z4oqMMJbV3VOZydKNwFkAd1v7wcMFGH6qR8yI_7csFrhkiF54_C31_lkYaxyuyScHNUkjQ6xiCxT kIoDYX_BUBg2361R9vPmjMWzTns5fX38n9V0Xqi7dtEMQfC7lGttt3k2nOnLG2gmz9eM4OMPY_L7 2cIfMvMitk2bvy8xTwMxJh34KNn.YzKobcLKKNiq9ZF2v5WH.ph7LcZhW3KAPIiXC_YuqpU0TqZ2 iinM3Qcm8Zkr4cgTQHdIW7htiQtYMXKZuJrI3UyDYSV4UXIt_W1J2w_tz1uoG0sedweNsgPuTa6W 4XQuI9HzaDm3Djie26Wp4b8OwU5dz_.FFORNKvTFF8Ul.aRoLRhcZ0UW.Rc0UbGWUk57hR2.vhkc zifW.aUQVnn7GLl.mRR5YNM8QZOc8ekryEPVBTH16cERry0Aw6DPvOECbFX5yeIWpX96AhMxMbXY 2w_wI12QTNvDcgHgFJDFp5fok80Xa_S6D833d45.KEp21h8wgoedGgy42T5cvtxiKJ6OcqI5OUSY O5fwFBdyjGsrInFd29WGLFBREXlVh.87EQGlEMv5hmKSS X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic301.consmr.mail.gq1.yahoo.com with HTTP; Sun, 21 Nov 2021 19:36:14 +0000 Received: by kubenode527.mail-prod1.omega.bf1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID b8c654c278341bc118ca9cf770e6c092; Sun, 21 Nov 2021 19:36:10 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) Subject: Re: FYI: aarch64 boot (HoneyComb): example crash during system checks (power-off/power-on form of reboot still fails) Date: Sun, 21 Nov 2021 11:36:08 -0800 References: To: "freebsd-arm@freebsd.org" , FreeBSD-STABLE Mailing List In-Reply-To: Message-Id: X-Mailer: Apple Mail (2.3654.120.0.1.13) X-Rspamd-Queue-Id: 4Hy0y73Vr1z4fjW X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=nBiK3Tct; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.64.148 as permitted sender) smtp.mailfrom=marklmi@yahoo.com X-Spamd-Result: default: False [-3.50 / 15.00]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.148:from]; FROM_HAS_DN(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.148:from]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_COUNT_TWO(0.00)[2]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim] Reply-To: marklmi@yahoo.com From: Mark Millard via arm X-Original-From: Mark Millard X-ThisMailContainsUnwantedMimeParts: N On 2021-Nov-21, at 11:26, Mark Millard wrote: > Starting file system checks: > /dev/gpt/CA72opt0EFI: 41 files, 242 MiB free (15469 clusters) > FIXED > /d x0: ffff000000e43ec8 (blocked_lock + 0) > x1: ffff00013efa9f50 > x2: ffff00000090e39a (cam_status_table + 1d132) > x3: deadc0d8 > x4: 0 > x5: ffff00000082a138 (data_abort + 0) > x6: 5 > x7: 601 > x8: ffff000000e43ec8 (blocked_lock + 0) > x9: deadc0de > x10: 0 > x11: 3938700 > x12: 0 > x13: 8000 > x14: 1de > x15: 81ce > x16: 425b9080 > x17: 8000 > x18: ffff00013efa9f40 > x19: ffff000000e43ec8 (blocked_lock + 0) > x20: ffffa0001a826000 > x21: 0 > x22: ffffa0001a826000 > x23: 0 > x24: ffff000000bed000 (queue_ops + 0) > x25: 98967f > x26: ffff000000e43ee0 (blocked_lock + 18) > x27: 0 > x28: 114 > x29: ffff00013efa9f40 > sp: ffff00013efa9f40 > lr: ffff0000004b9028 (thread_lock_flags_ + c0) > elr: ffff0000004b9028 (thread_lock_flags_ + c0) > spsr: 2c5 > far: deadc178 > esr: 96000004 > timeout stopping cpus > panic: data abort in critical section or under mutex > cpuid =3D 5 > time =3D 1637492224 > KDB: stack backtrace: > db_trace_self() at db_trace_self_wrapper+0x30 > pc =3D 0xffff000000807770 lr =3D 0xffff00000011d9ec > sp =3D 0xffff00013efa9990 fp =3D 0xffff00013efa9b90 >=20 > db_trace_self_wrapper() at vpanic+0x188 > pc =3D 0xffff00000011d9ec lr =3D 0xffff0000004e1d10 > sp =3D 0xffff00013efa9ba0 fp =3D 0xffff00013efa9c00 >=20 > vpanic() at panic+0x44 > pc =3D 0xffff0000004e1d10 lr =3D 0xffff0000004e1b84 > sp =3D 0xffff00013efa9c10 fp =3D 0xffff00013efa9cc0 >=20 > panic() at data_abort+0x290 > pc =3D 0xffff0000004e1b84 lr =3D 0xffff00000082a3c8 > sp =3D 0xffff00013efa9cd0 fp =3D 0xffff00013efa9d50 >=20 > data_abort() at handle_el1h_sync+0x78 > pc =3D 0xffff00000082a3c8 lr =3D 0xffff00000080a078 > sp =3D 0xffff00013efa9d60 fp =3D 0xffff00013efa9eb0 >=20 > handle_el1h_sync() at thread_lock_flags_+0xbc > pc =3D 0xffff00000080a078 lr =3D 0xffff0000004b9024 > sp =3D 0xffff00013efa9ec0 fp =3D 0xffff00013efa9f40 >=20 > thread_lock_flags_() at thread_lock_flags_+0xbc > pc =3D 0xffff0000004b9024 lr =3D 0xffff0000004b9024 > sp =3D 0xffff00013efa9f50 fp =3D 0xffff00013efa9f60 >=20 > thread_lock_flags_() at sleepq_timeout+0x10 > pc =3D 0xffff0000004b9024 lr =3D 0xffff00000054b2a8 > sp =3D 0xffff00013efa9f70 fp =3D 0xffff00013efa9fb0 >=20 > sleepq_timeout() at softclock_call_cc+0x14c > pc =3D 0xffff00000054b2a8 lr =3D 0xffff000000503134 > sp =3D 0xffff00013efa9fc0 fp =3D 0xffff00013efaa020 >=20 > softclock_call_cc() at callout_process+0x17c > pc =3D 0xffff000000503134 lr =3D 0xffff000000502df0 > sp =3D 0xffff00013efaa030 fp =3D 0xffff00013efaa0a0 >=20 > callout_process() at handleevents+0x188 > pc =3D 0xffff000000502df0 lr =3D 0xffff00000045b42c > sp =3D 0xffff00013efaa0b0 fp =3D 0xffff00013efaa100 >=20 > handleevents() at timercb+0x304 > pc =3D 0xffff00000045b42c lr =3D 0xffff00000045be7c > sp =3D 0xffff00013efaa110 fp =3D 0xffff00013efaa170 >=20 > timercb() at arm_tmr_intr+0x5c > pc =3D 0xffff00000045be7c lr =3D 0xffff0000007ff850 > sp =3D 0xffff00013efaa180 fp =3D 0xffff00013efaa1d0 >=20 > arm_tmr_intr() at intr_event_handle+0xac > pc =3D 0xffff0000007ff850 lr =3D 0xffff000000493c54 > sp =3D 0xffff00013efaa1e0 fp =3D 0xffff00013efaa1e0 >=20 > intr_event_handle() at intr_isrc_dispatch+0x70 > pc =3D 0xffff000000493c54 lr =3D 0xffff0000007fb238 > sp =3D 0xffff00013efaa1f0 fp =3D 0xffff00013efaa230 >=20 > intr_isrc_dispatch() at arm_gic_v3_intr+0x11c > pc =3D 0xffff0000007fb238 lr =3D 0xffff00000080ff34 > sp =3D 0xffff00013efaa240 fp =3D 0xffff00013efaa250 >=20 > arm_gic_v3_intr() at intr_irq_handler+0x7c > pc =3D 0xffff00000080ff34 lr =3D 0xffff0000007faff0 > sp =3D 0xffff00013efaa260 fp =3D 0xffff00013efaa2b0 >=20 > intr_irq_handler() at handle_el1h_irq+0x74 > pc =3D 0xffff0000007faff0 lr =3D 0xffff00000080a140 > sp =3D 0xffff00013efaa2c0 fp =3D 0xffff00013efaa3f0 >=20 > handle_el1h_irq() at handle_el1h_sync+0x78 > pc =3D 0xffff00000080a140 lr =3D 0xffff00000080a078 > sp =3D 0xffff00013efaa400 fp =3D 0xffff00013efaa500 >=20 > handle_el1h_sync() at handle_el1h_sync+0x78 > pc =3D 0xffff00000080a078 lr =3D 0xffff00000080a078 > sp =3D 0xffff00013efaa510 fp =3D 0xffff00013efaa660 >=20 > handle_el1h_sync() at sched_switch+0x6a8 > pc =3D 0xffff00000080a078 lr =3D 0xffff0000005197fc > sp =3D 0xffff00013efaa670 fp =3D 0xffff00013efaa6f0 >=20 > sched_switch() at sched_switch+0x6a8 > pc =3D 0xffff0000005197fc lr =3D 0xffff0000005197fc > sp =3D 0xffff00013efaa700 fp =3D 0xffff00013efaa790 >=20 > sched_switch() at mi_switch+0xf4 > pc =3D 0xffff0000005197fc lr =3D 0xffff0000004f03a0 > sp =3D 0xffff00013efaa7a0 fp =3D 0xffff00013efaa7f0 >=20 > mi_switch() at sleepq_timedwait+0x28 > pc =3D 0xffff0000004f03a0 lr =3D 0xffff00000054bd0c > sp =3D 0xffff00013efaa800 fp =3D 0xffff00013efaa830 >=20 > sleepq_timedwait() at _cv_timedwait_sbt+0x110 > pc =3D 0xffff00000054bd0c lr =3D 0xffff00000045e7b0 > sp =3D 0xffff00013efaa840 fp =3D 0xffff00013efaa850 >=20 > _cv_timedwait_sbt() at dbuf_evict_thread+0x410 > pc =3D 0xffff00000045e7b0 lr =3D 0xffff0000013ca59c > sp =3D 0xffff00013efaa860 fp =3D 0xffff00013efaa8f0 >=20 > dbuf_evict_thread() at fork_exit+0x94 > pc =3D 0xffff0000013ca59c lr =3D 0xffff00000048fbf0 > sp =3D 0xffff00013efaa900 fp =3D 0xffff00013efaa950 >=20 > fork_exit() at fork_trampoline+0x10 > pc =3D 0xffff00000048fbf0 lr =3D 0xffff000000828ed8 > sp =3D 0xffff00013efaa960 fp =3D 0x0000000000000000 >=20 > KDB: enter: panic > [ thread pid 26 tid 100194 ] > Stopped at kdb_enter+0x48: undefined f906411f >=20 > For reference: >=20 > # uname -apKU > FreeBSD CA72_16Gp_ZFS 13.0-STABLE FreeBSD 13.0-STABLE #13 = stable/13-n248062-109330155000-dirty: Sat Nov 13 23:55:14 PST 2021 = root@CA72_16Gp_ZFS:/usr/obj/BUILDs/13S-CA72-nodbg-clang/usr/13S-src/arm64.= aarch64/sys/GENERIC-NODBG-CA72 arm64 aarch64 1300520 1300520 >=20 > It is a root-on-ZFS context on Optane media in the PCie slot. >=20 > I've no clue if this will repeat. I've never gotten > this before. The reboot attempt got the following, involving zthr_procedure instead of dbuf_evict_thread . Starting file system checks: /dev/gpt/CA72opt0EFI: FILESYSTEM CLEAN; SKIPPING CHECKS x0: ffff000000e43ec8 (blocked_lock + 0) x1: ffff00013ef9ff90 x2: ffff00000090e39a (cam_status_table + 1d132) x3: deadc0d8 x4: 0 x5: ffff00000082a138 (data_abort + 0) x6: 9 x7: 601 x8: ffff000000e43ec8 (blocked_lock + 0) x9: deadc0de x10: 0 x11: 3938700 x12: 1 x13: 8000 x14: 1ee x15: 81cd x16: 425b9080 x17: 8000 x18: ffff00013ef9ff80 x19: ffff000000e43ec8 (blocked_lock + 0) x20: ffffa0000c011000 x21: 0 x22: ffffa0000c011000 x23: 0 x24: ffff000000bed000 (queue_ops + 0) x25: 98967f x26: ffff000000e43ee0 (blocked_lock + 18) x27: 0 x28: 114 x29: ffff00013ef9ff80 sp: ffff00013ef9ff80 lr: ffff0000004b9028 (thread_lock_flags_ + c0) elr: ffff0000004b9028 (thread_lock_flags_ + c0) spsr: 2c5 far: deadc178 esr: 96000004 timeout stopping cpus panic: data abort in critical section or under mutex cpuid =3D 9 time =3D 1637492224 KDB: stack backtrace: db_trace_self() at db_trace_self_wrapper+0x30 pc =3D 0xffff000000807770 lr =3D 0xffff00000011d9ec sp =3D 0xffff00013ef9f9d0 fp =3D 0xffff00013ef9fbd0 db_trace_self_wrapper() at vpanic+0x188 pc =3D 0xffff00000011d9ec lr =3D 0xffff0000004e1d10 sp =3D 0xffff00013ef9fbe0 fp =3D 0xffff00013ef9fc40 vpanic() at panic+0x44 pc =3D 0xffff0000004e1d10 lr =3D 0xffff0000004e1b84 sp =3D 0xffff00013ef9fc50 fp =3D 0xffff00013ef9fd00 panic() at data_abort+0x290 pc =3D 0xffff0000004e1b84 lr =3D 0xffff00000082a3c8 sp =3D 0xffff00013ef9fd10 fp =3D 0xffff00013ef9fd90 data_abort() at handle_el1h_sync+0x78 pc =3D 0xffff00000082a3c8 lr =3D 0xffff00000080a078 sp =3D 0xffff00013ef9fda0 fp =3D 0xffff00013ef9fef0 handle_el1h_sync() at thread_lock_flags_+0xbc pc =3D 0xffff00000080a078 lr =3D 0xffff0000004b9024 sp =3D 0xffff00013ef9ff00 fp =3D 0xffff00013ef9ff80 thread_lock_flags_() at thread_lock_flags_+0xbc pc =3D 0xffff0000004b9024 lr =3D 0xffff0000004b9024 sp =3D 0xffff00013ef9ff90 fp =3D 0xffff00013ef9ffa0 thread_lock_flags_() at sleepq_timeout+0x10 pc =3D 0xffff0000004b9024 lr =3D 0xffff00000054b2a8 sp =3D 0xffff00013ef9ffb0 fp =3D 0xffff00013ef9fff0 sleepq_timeout() at softclock_call_cc+0x14c pc =3D 0xffff00000054b2a8 lr =3D 0xffff000000503134 sp =3D 0xffff00013efa0000 fp =3D 0xffff00013efa0060 softclock_call_cc() at callout_process+0x17c pc =3D 0xffff000000503134 lr =3D 0xffff000000502df0 sp =3D 0xffff00013efa0070 fp =3D 0xffff00013efa00e0 callout_process() at handleevents+0x188 pc =3D 0xffff000000502df0 lr =3D 0xffff00000045b42c sp =3D 0xffff00013efa00f0 fp =3D 0xffff00013efa0140 handleevents() at timercb+0x304 pc =3D 0xffff00000045b42c lr =3D 0xffff00000045be7c sp =3D 0xffff00013efa0150 fp =3D 0xffff00013efa01b0 timercb() at arm_tmr_intr+0x5c pc =3D 0xffff00000045be7c lr =3D 0xffff0000007ff850 sp =3D 0xffff00013efa01c0 fp =3D 0xffff00013efa0210 arm_tmr_intr() at intr_event_handle+0xac pc =3D 0xffff0000007ff850 lr =3D 0xffff000000493c54 sp =3D 0xffff00013efa0220 fp =3D 0xffff00013efa0220 intr_event_handle() at intr_isrc_dispatch+0x70 pc =3D 0xffff000000493c54 lr =3D 0xffff0000007fb238 sp =3D 0xffff00013efa0230 fp =3D 0xffff00013efa0270 intr_isrc_dispatch() at arm_gic_v3_intr+0x11c pc =3D 0xffff0000007fb238 lr =3D 0xffff00000080ff34 sp =3D 0xffff00013efa0280 fp =3D 0xffff00013efa0290 arm_gic_v3_intr() at intr_irq_handler+0x7c pc =3D 0xffff00000080ff34 lr =3D 0xffff0000007faff0 sp =3D 0xffff00013efa02a0 fp =3D 0xffff00013efa02f0 intr_irq_handler() at handle_el1h_irq+0x74 pc =3D 0xffff0000007faff0 lr =3D 0xffff00000080a140 sp =3D 0xffff00013efa0300 fp =3D 0xffff00013efa0430 handle_el1h_irq() at handle_el1h_sync+0x78 pc =3D 0xffff00000080a140 lr =3D 0xffff00000080a078 sp =3D 0xffff00013efa0440 fp =3D 0xffff00013efa0540 handle_el1h_sync() at handle_el1h_sync+0x78 pc =3D 0xffff00000080a078 lr =3D 0xffff00000080a078 sp =3D 0xffff00013efa0550 fp =3D 0xffff00013efa06a0 handle_el1h_sync() at sched_switch+0x6a8 pc =3D 0xffff00000080a078 lr =3D 0xffff0000005197fc sp =3D 0xffff00013efa06b0 fp =3D 0xffff00013efa0730 sched_switch() at sched_switch+0x6a8 pc =3D 0xffff0000005197fc lr =3D 0xffff0000005197fc sp =3D 0xffff00013efa0740 fp =3D 0xffff00013efa07d0 sched_switch() at mi_switch+0xf4 pc =3D 0xffff0000005197fc lr =3D 0xffff0000004f03a0 sp =3D 0xffff00013efa07e0 fp =3D 0xffff00013efa0830 mi_switch() at sleepq_timedwait+0x28 pc =3D 0xffff0000004f03a0 lr =3D 0xffff00000054bd0c sp =3D 0xffff00013efa0840 fp =3D 0xffff00013efa0870 sleepq_timedwait() at _cv_timedwait_sbt+0x110 pc =3D 0xffff00000054bd0c lr =3D 0xffff00000045e7b0 sp =3D 0xffff00013efa0880 fp =3D 0xffff00013efa0890 _cv_timedwait_sbt() at zthr_procedure+0x20c pc =3D 0xffff00000045e7b0 lr =3D 0xffff0000014e2fb0 sp =3D 0xffff00013efa08a0 fp =3D 0xffff00013efa08f0 zthr_procedure() at fork_exit+0x94 pc =3D 0xffff0000014e2fb0 lr =3D 0xffff00000048fbf0 sp =3D 0xffff00013efa0900 fp =3D 0xffff00013efa0950 fork_exit() at fork_trampoline+0x10 pc =3D 0xffff00000048fbf0 lr =3D 0xffff000000828ed8 sp =3D 0xffff00013efa0960 fp =3D 0x0000000000000000 KDB: enter: panic [ thread pid 26 tid 100192 ] Stopped at kdb_enter+0x48: undefined f906411f Note: All this started after a stress based I/O hangup test that required a forced reboot. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From nobody Sun Nov 21 20:09:35 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id D35DF188D5BD for ; Sun, 21 Nov 2021 20:10:49 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic306-20.consmr.mail.gq1.yahoo.com (sonic306-20.consmr.mail.gq1.yahoo.com [98.137.68.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Hy1k01Fxpz4s51 for ; Sun, 21 Nov 2021 20:10:48 +0000 (UTC) (envelope-from marklmi@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1637525440; bh=PkWcy2dtb5VpjuCKQ6TprLl2ogwQvyH59OHLljTezco=; h=From:Subject:Date:References:To:In-Reply-To:From:Subject:Reply-To; b=pWdn2kvRh5r0VYf5DKSXDTTDvl2WCdR7JNNVFPPOyqT5+s/stN91L8tmNyKj6HT8+tTMItBoTmNZMF0iRvkfpFP3wkZynxJMR8IX0i7QSgRHLDtjRhpaElZ0E1m4ZKa/gWCMe2AJaXAAGA8sGhd0R4xR9UC+r7+ey1beh4IjIL7MkMSn36U2qb7qgYXNgaDQoplSlhJc0EqIeBSs5NaLkrNXa2VX2AGpMZzn2GXa37yPVYct6gSeY5d4Con2ffEDrh6MzLYKuvq6aeDLud4IlUHd/kP6oC8/F925xD3urDgzO9GF3d7+gj/IU//tBmFer1ky0TvAnamBY/BEkfKSUg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1637525440; bh=kYLQ7mRrGnHtzZpQ6z9VnO7Sc3cAtixFp8+KWz25hvl=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=kbdCKgbBWyvWgJ7mswCTufeCZxTII7cGE4N2xKeUuoN0R26oO8FgSYxBzzZof8bTO3vKE0Sri6vbC7QFyuuyLnRFtZCFXe/TdVK3q4VriK15xihGX09EHW14Akld3vTk1LR7p/Dk21ft89jAJ50IkYjaqdrZa2tGNZTFFwLXrMUb31wNFaMeQo3h9GzAKimHV1WQ1Kx7bvN/6kQ6PhrjCqC9k1y30jRhPRZnyPdoBAXdGQvFtYgu5RFgu3QkEJVv+m3nzdeb1M2amPhloa3oO1Y+QNq6+0dY4gXg5kHuSBJs4W2DJSxT7tZprU46QkzqzJFbkXc4i0od18Fs//U85A== X-YMail-OSG: VRDWmcwVM1mHMTgaPGaKlbZ4dJPtaF3HNwSduGP8QOBCRzpmG0twKjSWywWPFSY xjDeTrTIs1mstwI04ZhtCbWKRsElH.QqRaeRBKsm6z2K6.JSxcSCOvSWITsvsBEJXnIUsvpwDpA. Ph1NA2i22Nh4FDe9t4NpdN9P5jCoExjEpNLnx0WbeFhfRYtVI13B5tfJiD4xFk0FAQDbaJInP1E1 tARj2WzBEMpQz3490iEzyLlY7S39tolGEr6Znn_fofEYOD9U7CDtUlxD7BHAAQbNwPi70FUkIjz. 9Uir5FCS02DNSMCwShcgV7U3IzoOiW6oRQOAEYOjYhMuP9ex_iwQmTY.onXGcHGXRX0SuqbD5ASD mWYggkd34Ixwuy.D.V9c.6vpSN021EoTYeLxunMwNhmYDDhIBuOFYQnQJyzW9L7DXwPdcgDVbvGP rGqQxnf6jEGQ6UfpzoWqxxenz1DubaXi4BtDMu.QDLlr0uNSwf_MpAB0ofdaRp2Wd3cNf.gRthe8 jzGF5ypxP0mgKOYxuTIuptC.BuRZ8Rf6pt5MKhYNVgvZs12u1gEMUFOFSNSsLOCaig7qJpzU0O0d FWr2.hwkOzeLdAPlIOUD.U2lfrhLgIIxwHFQxIxvFKGBYwsGL4UOLtrraNmQWwLiM.zOx9yvQCz_ BF5EU9w6ZfW.D3r1dq0D2KNGYfY82TE.3lV1UroY5RYYw5D1iLeC.sQSA6yQiy14zAs5sQC9ePEU ScZHAdS_VRW89RzxBQu14mwDkaVE2pE5h_dDYNNH8EMx_2OR25FQ4K70ZqJvM7pEr.rTIgDdmLSL brAm5xhWZMYDjxZXEB_5RmW4JlfUW_OY_rm_7ry3nR6sZhWkDfyUsma7C6qN5bmNWiR.Cz2kE2Cj 5jH6wnqXpKY9imTCIJue5yXP9G4PVfJf6ftgwmKJvUzW70sGBAl2gfn0.7047ZqINyy_dS2EB96O TlKOr6YT4k9vg_TcHPvgdwLael6jqSq.BkXuprAgfoJ8Zvqc6Ml3aRLTSS8739BYctBsUwbSUpoc YHPrcM91_cLfAgDxLn.V..bgJ32Skg8w8.jfvb9C0uKuslZs0GXwOHHcAviUmzwdWrsPAXZN1YzK eDNO0YDAAoGCdjW5C0vPsfqxtxikTLl4s_O02bw9EayhWMpkh9LIT6l6fZ9aK5GUkPNGWW3gMKHM gV0c0ZQQhp9RR.Zy9hWP6z0pToo.aQe4bRb_DqEGMLyofNMhW7aM6anwjmmKTgeo1ZGWN5zUDWDF 5DIM58VJq9kvqkh4.2OkZ.cnCPaCUWSJjzxZqsH7vpVXKEp32ETF1TBwbGZ57wZQgSiAjYmuiPsL LV_k9hojiK44CM0mLkfFXSRr6J_5FEr4wAXTC73B0GgmrfcJepMhvEjyr68A1h1P4cpesF6N6kKH RDEiMuB4Nnhej3e.0NLLd3EOqb.efinh78R91yP0TkE12RyYnLur8v_w_6wjqNOJFaZNDbdZqU_k 2EWTbY7EDPPN_ye.UzZsVUy0BQjZ9U9tjw3XKxUWRyMWGDUNUmurBTIWwpBHpPryzEw3uNJbMKNU .GrEQytFgwu6PkApzhGewJ0UrOcfz5Ulj8XNbenD0SXxRicubsrI5NFQtk8LcFx89eTdw0vJyUpk Y6k6SSJ8oa0UO8dOW59Wpj.EXnfgVcsLYY4GyLtZPSrw3e7tjgm.BPcPryGkFPSimw5dEp60RT8W jvqwxW4v5Ij7ggYVW0PWyHTXQk_OY0_K4SG09ZE1TaPDlLqoQ9ygH0TuXTRBmhjDQzfl5dCJzmQo iud7WU.1MISY0VUTI8AUptiXqzlR22_0trZtaZ3HjcPYHBDk45S5FoGzXAYZh7SpTiJCUHjB2V3G zqlrk4Xur3dNl7DXcBOUvzjtINd86tHaWS50SDOFIUMsnPNaBbJe7DzkhzGohDsLz6jmWU262zRH jnrpqUR3KhVwyejA2XNq7fE157IzpmqS08v8OFoxKWHsK_1iJpd4xI8p663VT_AOgfHC6jEANX0I 4WRLCg36I8SdzT4WF70zs2b6QRhprOygKdfRFLq_dSdhBq8.kQODWufkuheLaw4GrPQjUkGQlUlH ybuaAs0MoZXCxczMMhliH3tWwhtVdDQj3CQZ9Pfz1LUsuHTazReos5OnFWUoaiLJhQxygjy5RZJH 5LWuIe6Frlt4MCwjgGvCsgTwo3lEUmOcFwZJrh3Pzt_UCCVfi4j8hfVFbroOsQ_OdI0b1ivAR0fw JPhmaF9ChN1hx4VO5h4P02RcVOgChg5cp0Gh1dRKZTBILag-- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic306.consmr.mail.gq1.yahoo.com with HTTP; Sun, 21 Nov 2021 20:10:40 +0000 Received: by kubenode545.mail-prod1.omega.gq1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 9488c875f50d8690f753f7afb465ae80; Sun, 21 Nov 2021 20:09:36 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) Subject: Re: FYI: aarch64 boot (HoneyComb): example crash during system checks (power-off/power-on form of reboot still fails) Date: Sun, 21 Nov 2021 12:09:35 -0800 References: To: "freebsd-arm@freebsd.org" , FreeBSD-STABLE Mailing List In-Reply-To: Message-Id: X-Mailer: Apple Mail (2.3654.120.0.1.13) X-Rspamd-Queue-Id: 4Hy1k01Fxpz4s51 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=pWdn2kvR; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.68.83 as permitted sender) smtp.mailfrom=marklmi@yahoo.com X-Spamd-Result: default: False [-3.50 / 15.00]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.83:from]; FROM_HAS_DN(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.83:from]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_COUNT_TWO(0.00)[2]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim] Reply-To: marklmi@yahoo.com From: Mark Millard via arm X-Original-From: Mark Millard X-ThisMailContainsUnwantedMimeParts: N On 2021-Nov-21, at 11:36, Mark Millard wrote: > On 2021-Nov-21, at 11:26, Mark Millard wrote: >=20 >> Starting file system checks: >> /dev/gpt/CA72opt0EFI: 41 files, 242 MiB free (15469 clusters) >> FIXED >> /d x0: ffff000000e43ec8 (blocked_lock + 0) >> x1: ffff00013efa9f50 >> x2: ffff00000090e39a (cam_status_table + 1d132) >> x3: deadc0d8 >> x4: 0 >> x5: ffff00000082a138 (data_abort + 0) >> x6: 5 >> x7: 601 >> x8: ffff000000e43ec8 (blocked_lock + 0) >> x9: deadc0de >> x10: 0 >> x11: 3938700 >> x12: 0 >> x13: 8000 >> x14: 1de >> x15: 81ce >> x16: 425b9080 >> x17: 8000 >> x18: ffff00013efa9f40 >> x19: ffff000000e43ec8 (blocked_lock + 0) >> x20: ffffa0001a826000 >> x21: 0 >> x22: ffffa0001a826000 >> x23: 0 >> x24: ffff000000bed000 (queue_ops + 0) >> x25: 98967f >> x26: ffff000000e43ee0 (blocked_lock + 18) >> x27: 0 >> x28: 114 >> x29: ffff00013efa9f40 >> sp: ffff00013efa9f40 >> lr: ffff0000004b9028 (thread_lock_flags_ + c0) >> elr: ffff0000004b9028 (thread_lock_flags_ + c0) >> spsr: 2c5 >> far: deadc178 >> esr: 96000004 >> timeout stopping cpus >> panic: data abort in critical section or under mutex >> cpuid =3D 5 >> time =3D 1637492224 >> KDB: stack backtrace: >> db_trace_self() at db_trace_self_wrapper+0x30 >> pc =3D 0xffff000000807770 lr =3D 0xffff00000011d9ec >> sp =3D 0xffff00013efa9990 fp =3D 0xffff00013efa9b90 >>=20 >> db_trace_self_wrapper() at vpanic+0x188 >> pc =3D 0xffff00000011d9ec lr =3D 0xffff0000004e1d10 >> sp =3D 0xffff00013efa9ba0 fp =3D 0xffff00013efa9c00 >>=20 >> vpanic() at panic+0x44 >> pc =3D 0xffff0000004e1d10 lr =3D 0xffff0000004e1b84 >> sp =3D 0xffff00013efa9c10 fp =3D 0xffff00013efa9cc0 >>=20 >> panic() at data_abort+0x290 >> pc =3D 0xffff0000004e1b84 lr =3D 0xffff00000082a3c8 >> sp =3D 0xffff00013efa9cd0 fp =3D 0xffff00013efa9d50 >>=20 >> data_abort() at handle_el1h_sync+0x78 >> pc =3D 0xffff00000082a3c8 lr =3D 0xffff00000080a078 >> sp =3D 0xffff00013efa9d60 fp =3D 0xffff00013efa9eb0 >>=20 >> handle_el1h_sync() at thread_lock_flags_+0xbc >> pc =3D 0xffff00000080a078 lr =3D 0xffff0000004b9024 >> sp =3D 0xffff00013efa9ec0 fp =3D 0xffff00013efa9f40 >>=20 >> thread_lock_flags_() at thread_lock_flags_+0xbc >> pc =3D 0xffff0000004b9024 lr =3D 0xffff0000004b9024 >> sp =3D 0xffff00013efa9f50 fp =3D 0xffff00013efa9f60 >>=20 >> thread_lock_flags_() at sleepq_timeout+0x10 >> pc =3D 0xffff0000004b9024 lr =3D 0xffff00000054b2a8 >> sp =3D 0xffff00013efa9f70 fp =3D 0xffff00013efa9fb0 >>=20 >> sleepq_timeout() at softclock_call_cc+0x14c >> pc =3D 0xffff00000054b2a8 lr =3D 0xffff000000503134 >> sp =3D 0xffff00013efa9fc0 fp =3D 0xffff00013efaa020 >>=20 >> softclock_call_cc() at callout_process+0x17c >> pc =3D 0xffff000000503134 lr =3D 0xffff000000502df0 >> sp =3D 0xffff00013efaa030 fp =3D 0xffff00013efaa0a0 >>=20 >> callout_process() at handleevents+0x188 >> pc =3D 0xffff000000502df0 lr =3D 0xffff00000045b42c >> sp =3D 0xffff00013efaa0b0 fp =3D 0xffff00013efaa100 >>=20 >> handleevents() at timercb+0x304 >> pc =3D 0xffff00000045b42c lr =3D 0xffff00000045be7c >> sp =3D 0xffff00013efaa110 fp =3D 0xffff00013efaa170 >>=20 >> timercb() at arm_tmr_intr+0x5c >> pc =3D 0xffff00000045be7c lr =3D 0xffff0000007ff850 >> sp =3D 0xffff00013efaa180 fp =3D 0xffff00013efaa1d0 >>=20 >> arm_tmr_intr() at intr_event_handle+0xac >> pc =3D 0xffff0000007ff850 lr =3D 0xffff000000493c54 >> sp =3D 0xffff00013efaa1e0 fp =3D 0xffff00013efaa1e0 >>=20 >> intr_event_handle() at intr_isrc_dispatch+0x70 >> pc =3D 0xffff000000493c54 lr =3D 0xffff0000007fb238 >> sp =3D 0xffff00013efaa1f0 fp =3D 0xffff00013efaa230 >>=20 >> intr_isrc_dispatch() at arm_gic_v3_intr+0x11c >> pc =3D 0xffff0000007fb238 lr =3D 0xffff00000080ff34 >> sp =3D 0xffff00013efaa240 fp =3D 0xffff00013efaa250 >>=20 >> arm_gic_v3_intr() at intr_irq_handler+0x7c >> pc =3D 0xffff00000080ff34 lr =3D 0xffff0000007faff0 >> sp =3D 0xffff00013efaa260 fp =3D 0xffff00013efaa2b0 >>=20 >> intr_irq_handler() at handle_el1h_irq+0x74 >> pc =3D 0xffff0000007faff0 lr =3D 0xffff00000080a140 >> sp =3D 0xffff00013efaa2c0 fp =3D 0xffff00013efaa3f0 >>=20 >> handle_el1h_irq() at handle_el1h_sync+0x78 >> pc =3D 0xffff00000080a140 lr =3D 0xffff00000080a078 >> sp =3D 0xffff00013efaa400 fp =3D 0xffff00013efaa500 >>=20 >> handle_el1h_sync() at handle_el1h_sync+0x78 >> pc =3D 0xffff00000080a078 lr =3D 0xffff00000080a078 >> sp =3D 0xffff00013efaa510 fp =3D 0xffff00013efaa660 >>=20 >> handle_el1h_sync() at sched_switch+0x6a8 >> pc =3D 0xffff00000080a078 lr =3D 0xffff0000005197fc >> sp =3D 0xffff00013efaa670 fp =3D 0xffff00013efaa6f0 >>=20 >> sched_switch() at sched_switch+0x6a8 >> pc =3D 0xffff0000005197fc lr =3D 0xffff0000005197fc >> sp =3D 0xffff00013efaa700 fp =3D 0xffff00013efaa790 >>=20 >> sched_switch() at mi_switch+0xf4 >> pc =3D 0xffff0000005197fc lr =3D 0xffff0000004f03a0 >> sp =3D 0xffff00013efaa7a0 fp =3D 0xffff00013efaa7f0 >>=20 >> mi_switch() at sleepq_timedwait+0x28 >> pc =3D 0xffff0000004f03a0 lr =3D 0xffff00000054bd0c >> sp =3D 0xffff00013efaa800 fp =3D 0xffff00013efaa830 >>=20 >> sleepq_timedwait() at _cv_timedwait_sbt+0x110 >> pc =3D 0xffff00000054bd0c lr =3D 0xffff00000045e7b0 >> sp =3D 0xffff00013efaa840 fp =3D 0xffff00013efaa850 >>=20 >> _cv_timedwait_sbt() at dbuf_evict_thread+0x410 >> pc =3D 0xffff00000045e7b0 lr =3D 0xffff0000013ca59c >> sp =3D 0xffff00013efaa860 fp =3D 0xffff00013efaa8f0 >>=20 >> dbuf_evict_thread() at fork_exit+0x94 >> pc =3D 0xffff0000013ca59c lr =3D 0xffff00000048fbf0 >> sp =3D 0xffff00013efaa900 fp =3D 0xffff00013efaa950 >>=20 >> fork_exit() at fork_trampoline+0x10 >> pc =3D 0xffff00000048fbf0 lr =3D 0xffff000000828ed8 >> sp =3D 0xffff00013efaa960 fp =3D 0x0000000000000000 >>=20 >> KDB: enter: panic >> [ thread pid 26 tid 100194 ] >> Stopped at kdb_enter+0x48: undefined f906411f >>=20 >> For reference: >>=20 >> # uname -apKU >> FreeBSD CA72_16Gp_ZFS 13.0-STABLE FreeBSD 13.0-STABLE #13 = stable/13-n248062-109330155000-dirty: Sat Nov 13 23:55:14 PST 2021 = root@CA72_16Gp_ZFS:/usr/obj/BUILDs/13S-CA72-nodbg-clang/usr/13S-src/arm64.= aarch64/sys/GENERIC-NODBG-CA72 arm64 aarch64 1300520 1300520 >>=20 >> It is a root-on-ZFS context on Optane media in the PCie slot. >>=20 >> I've no clue if this will repeat. I've never gotten >> this before. >=20 >=20 > The reboot attempt got the following, involving > zthr_procedure instead of dbuf_evict_thread . >=20 > Starting file system checks: > /dev/gpt/CA72opt0EFI: FILESYSTEM CLEAN; SKIPPING CHECKS > x0: ffff000000e43ec8 (blocked_lock + 0) > x1: ffff00013ef9ff90 > x2: ffff00000090e39a (cam_status_table + 1d132) > x3: deadc0d8 > x4: 0 > x5: ffff00000082a138 (data_abort + 0) > x6: 9 > x7: 601 > x8: ffff000000e43ec8 (blocked_lock + 0) > x9: deadc0de > x10: 0 > x11: 3938700 > x12: 1 > x13: 8000 > x14: 1ee > x15: 81cd > x16: 425b9080 > x17: 8000 > x18: ffff00013ef9ff80 > x19: ffff000000e43ec8 (blocked_lock + 0) > x20: ffffa0000c011000 > x21: 0 > x22: ffffa0000c011000 > x23: 0 > x24: ffff000000bed000 (queue_ops + 0) > x25: 98967f > x26: ffff000000e43ee0 (blocked_lock + 18) > x27: 0 > x28: 114 > x29: ffff00013ef9ff80 > sp: ffff00013ef9ff80 > lr: ffff0000004b9028 (thread_lock_flags_ + c0) > elr: ffff0000004b9028 (thread_lock_flags_ + c0) > spsr: 2c5 > far: deadc178 > esr: 96000004 > timeout stopping cpus > panic: data abort in critical section or under mutex > cpuid =3D 9 > time =3D 1637492224 > KDB: stack backtrace: > db_trace_self() at db_trace_self_wrapper+0x30 > pc =3D 0xffff000000807770 lr =3D 0xffff00000011d9ec > sp =3D 0xffff00013ef9f9d0 fp =3D 0xffff00013ef9fbd0 >=20 > db_trace_self_wrapper() at vpanic+0x188 > pc =3D 0xffff00000011d9ec lr =3D 0xffff0000004e1d10 > sp =3D 0xffff00013ef9fbe0 fp =3D 0xffff00013ef9fc40 >=20 > vpanic() at panic+0x44 > pc =3D 0xffff0000004e1d10 lr =3D 0xffff0000004e1b84 > sp =3D 0xffff00013ef9fc50 fp =3D 0xffff00013ef9fd00 >=20 > panic() at data_abort+0x290 > pc =3D 0xffff0000004e1b84 lr =3D 0xffff00000082a3c8 > sp =3D 0xffff00013ef9fd10 fp =3D 0xffff00013ef9fd90 >=20 > data_abort() at handle_el1h_sync+0x78 > pc =3D 0xffff00000082a3c8 lr =3D 0xffff00000080a078 > sp =3D 0xffff00013ef9fda0 fp =3D 0xffff00013ef9fef0 >=20 > handle_el1h_sync() at thread_lock_flags_+0xbc > pc =3D 0xffff00000080a078 lr =3D 0xffff0000004b9024 > sp =3D 0xffff00013ef9ff00 fp =3D 0xffff00013ef9ff80 >=20 > thread_lock_flags_() at thread_lock_flags_+0xbc > pc =3D 0xffff0000004b9024 lr =3D 0xffff0000004b9024 > sp =3D 0xffff00013ef9ff90 fp =3D 0xffff00013ef9ffa0 >=20 > thread_lock_flags_() at sleepq_timeout+0x10 > pc =3D 0xffff0000004b9024 lr =3D 0xffff00000054b2a8 > sp =3D 0xffff00013ef9ffb0 fp =3D 0xffff00013ef9fff0 >=20 > sleepq_timeout() at softclock_call_cc+0x14c > pc =3D 0xffff00000054b2a8 lr =3D 0xffff000000503134 > sp =3D 0xffff00013efa0000 fp =3D 0xffff00013efa0060 >=20 > softclock_call_cc() at callout_process+0x17c > pc =3D 0xffff000000503134 lr =3D 0xffff000000502df0 > sp =3D 0xffff00013efa0070 fp =3D 0xffff00013efa00e0 >=20 > callout_process() at handleevents+0x188 > pc =3D 0xffff000000502df0 lr =3D 0xffff00000045b42c > sp =3D 0xffff00013efa00f0 fp =3D 0xffff00013efa0140 >=20 > handleevents() at timercb+0x304 > pc =3D 0xffff00000045b42c lr =3D 0xffff00000045be7c > sp =3D 0xffff00013efa0150 fp =3D 0xffff00013efa01b0 >=20 > timercb() at arm_tmr_intr+0x5c > pc =3D 0xffff00000045be7c lr =3D 0xffff0000007ff850 > sp =3D 0xffff00013efa01c0 fp =3D 0xffff00013efa0210 >=20 > arm_tmr_intr() at intr_event_handle+0xac > pc =3D 0xffff0000007ff850 lr =3D 0xffff000000493c54 > sp =3D 0xffff00013efa0220 fp =3D 0xffff00013efa0220 >=20 > intr_event_handle() at intr_isrc_dispatch+0x70 > pc =3D 0xffff000000493c54 lr =3D 0xffff0000007fb238 > sp =3D 0xffff00013efa0230 fp =3D 0xffff00013efa0270 >=20 > intr_isrc_dispatch() at arm_gic_v3_intr+0x11c > pc =3D 0xffff0000007fb238 lr =3D 0xffff00000080ff34 > sp =3D 0xffff00013efa0280 fp =3D 0xffff00013efa0290 >=20 > arm_gic_v3_intr() at intr_irq_handler+0x7c > pc =3D 0xffff00000080ff34 lr =3D 0xffff0000007faff0 > sp =3D 0xffff00013efa02a0 fp =3D 0xffff00013efa02f0 >=20 > intr_irq_handler() at handle_el1h_irq+0x74 > pc =3D 0xffff0000007faff0 lr =3D 0xffff00000080a140 > sp =3D 0xffff00013efa0300 fp =3D 0xffff00013efa0430 >=20 > handle_el1h_irq() at handle_el1h_sync+0x78 > pc =3D 0xffff00000080a140 lr =3D 0xffff00000080a078 > sp =3D 0xffff00013efa0440 fp =3D 0xffff00013efa0540 >=20 > handle_el1h_sync() at handle_el1h_sync+0x78 > pc =3D 0xffff00000080a078 lr =3D 0xffff00000080a078 > sp =3D 0xffff00013efa0550 fp =3D 0xffff00013efa06a0 >=20 > handle_el1h_sync() at sched_switch+0x6a8 > pc =3D 0xffff00000080a078 lr =3D 0xffff0000005197fc > sp =3D 0xffff00013efa06b0 fp =3D 0xffff00013efa0730 >=20 > sched_switch() at sched_switch+0x6a8 > pc =3D 0xffff0000005197fc lr =3D 0xffff0000005197fc > sp =3D 0xffff00013efa0740 fp =3D 0xffff00013efa07d0 >=20 > sched_switch() at mi_switch+0xf4 > pc =3D 0xffff0000005197fc lr =3D 0xffff0000004f03a0 > sp =3D 0xffff00013efa07e0 fp =3D 0xffff00013efa0830 >=20 > mi_switch() at sleepq_timedwait+0x28 > pc =3D 0xffff0000004f03a0 lr =3D 0xffff00000054bd0c > sp =3D 0xffff00013efa0840 fp =3D 0xffff00013efa0870 >=20 > sleepq_timedwait() at _cv_timedwait_sbt+0x110 > pc =3D 0xffff00000054bd0c lr =3D 0xffff00000045e7b0 > sp =3D 0xffff00013efa0880 fp =3D 0xffff00013efa0890 >=20 > _cv_timedwait_sbt() at zthr_procedure+0x20c > pc =3D 0xffff00000045e7b0 lr =3D 0xffff0000014e2fb0 > sp =3D 0xffff00013efa08a0 fp =3D 0xffff00013efa08f0 >=20 > zthr_procedure() at fork_exit+0x94 > pc =3D 0xffff0000014e2fb0 lr =3D 0xffff00000048fbf0 > sp =3D 0xffff00013efa0900 fp =3D 0xffff00013efa0950 >=20 > fork_exit() at fork_trampoline+0x10 > pc =3D 0xffff00000048fbf0 lr =3D 0xffff000000828ed8 > sp =3D 0xffff00013efa0960 fp =3D 0x0000000000000000 >=20 > KDB: enter: panic > [ thread pid 26 tid 100192 ] > Stopped at kdb_enter+0x48: undefined f906411f >=20 >=20 > Note: >=20 > All this started after a stress based I/O hangup test > that required a forced reboot. It is a bectl environment for that Optane: # bectl list BE Active Mountpoint Space Created 13S-CA72-nodbg R - 6.38G 2021-09-29 00:57 13_0R-CA72-nodbg N / 3.57G 2021-09-29 00:45 main-CA72-nodbg - - 7.99G 2021-09-29 00:42 13_0R-CA72-nodbg and main-CA72-nodbg boot fine, just 13S-CA72-nodbg has the problem. For reference: =3D> 40 1875384928 nda1 GPT (894G) 40 532480 nda1p1 CA72opt0EFI (260M) 532520 2008 - free - (1.0M) 534528 515899392 nda1p2 CA72opt0SWP (246G) 516433920 20971520 - free - (10G) 537405440 1337979528 nda1p3 CA72opt0ZFS (638G) Another attempt at booting have gotten reports of . . . lots of waiting notices . . . Root mount waiting for: CAM usbus1 Root mount waiting for: CAM usbus1 Root mount waiting for: CAM usbus1 Root mount waiting for: CAM usbus1 Root mount waiting for: CAM usbus1 x0: ffff000000e43ec8 (blocked_lock + 0) x1: ffff00013efa9f60 x2: ffff00000090e39a (cam_status_table + 1d132) x3: deadc0d8 x4: 0 x5: ffff00000082a138 (data_abort + 0) x6: 4 x7: 601 x8: ffff000000e43ec8 (blocked_lock + 0) x9: deadc0de x10: 0 x11: 3938700 x12: 0 x13: 8000 x14: 1dd x15: 81cd x16: 0 x17: 1 x18: ffff00013efa9f50 x19: ffff000000e43ec8 (blocked_lock + 0) x20: ffffa0000c806000 x21: 0 x22: ffffa0000c806000 x23: 0 x24: ffff000000bed000 (queue_ops + 0) x25: 98967f x26: ffff000000e43ee0 (blocked_lock + 18) x27: 0 x28: 114 x29: ffff00013efa9f50 sp: ffff00013efa9f50 lr: ffff0000004b9028 (thread_lock_flags_ + c0) elr: ffff0000004b9028 (thread_lock_flags_ + c0) spin lock 0xffff00004317b100 (sched lock 4) held by 0xffffa0000c806000 = (tid 100194) too long spin lock 0xffff00004317b100 (sched lock 4) held by 0xffffa0000c806000 = (tid 100194) too long spin lock 0xffff00004317b100 (sched lock 4) held by 0xffffa0000c806000 = (tid 100194) too long spsr: 2c5 far: deadc178 esr: 96000004 timeout stopping cpus panic: spin lock held too long cpuid =3D 9 time =3D 56 KDB: stack backtrace: db_trace_self() at db_trace_self_wrapper+0x30 pc =3D 0xffff000000807770 lr =3D 0xffff00000011d9ec sp =3D 0xffff000156eea430 fp =3D 0xffff000156eea630 db_trace_self_wrapper() at vpanic+0x188 pc =3D 0xffff00000011d9ec lr =3D 0xffff0000004e1d10 sp =3D 0xffff000156eea640 fp =3D 0xffff000156eea6a0 vpanic() at panic+0x44 pc =3D 0xffff0000004e1d10 lr =3D 0xffff0000004e1b84 sp =3D 0xffff000156eea6b0 fp =3D 0xffff000156eea760 panic() at _mtx_lock_indefinite_check+0x8c pc =3D 0xffff0000004e1b84 lr =3D 0xffff0000004b8ea4 sp =3D 0xffff000156eea770 fp =3D 0xffff000156eea770 _mtx_lock_indefinite_check() at _mtx_lock_spin_cookie+0xb8 pc =3D 0xffff0000004b8ea4 lr =3D 0xffff0000004b8a0c sp =3D 0xffff000156eea780 fp =3D 0xffff000156eea790 _mtx_lock_spin_cookie() at sched_add+0x400 pc =3D 0xffff0000004b8a0c lr =3D 0xffff00000051ac94 sp =3D 0xffff000156eea7a0 fp =3D 0xffff000156eea7f0 sched_add() at kthread_add+0x238 pc =3D 0xffff00000051ac94 lr =3D 0xffff0000004a01e0 sp =3D 0xffff000156eea800 fp =3D 0xffff000156eea880 kthread_add() at vm_pageout+0x1b8 pc =3D 0xffff0000004a01e0 lr =3D 0xffff0000007e5a18 sp =3D 0xffff000156eea890 fp =3D 0xffff000156eea8f0 . . . Garbled . . . sp =3D 0xffff00013efa9bb0 fp =3D 0xffff00013efa9c10 vpanic() at panic+0x44 pc =3D 0xffff0000004e1d10 lr =3D 0xffff0000004e1b84 sp =3D 0xffff00013efa9c20 fp =3D 0xffff00013efa9cd0 panic() at data_abort+0x290 pc =3D 0xffff0000004e1b84 lr =3D 0xffff00000082a3c8 sp =3D 0xffff00013efa9ce0 fp =3D 0xffff00013efa9d60 data_abort() at handle_el1h_sync+0x78 pc =3D 0xffff00000082a3c8 lr =3D 0xffff00000080a078 sp =3D 0xffff00013efa9d70 fp =3D 0xffff00013efa9ec0 handle_el1h_sync() at thread_lock_flags_+0xbc pc =3D 0xffff00000080a078 lr =3D 0xffff0000004b9024 sp =3D 0xffff00013efa9ed0 fp =3D 0xffff00013efa9f50 thread_lock_flags_() at thread_lock_flags_+0xbc pc =3D 0xffff0000004b9024 lr =3D 0xffff0000004b9024 . . . Garbled . . . But after that the next attempt booted just fine. zpool scrub got: # zpool status pool: zopt0 state: ONLINE scan: scrub repaired 0B in 00:02:05 with 0 errors on Sun Nov 21 = 12:00:09 2021 config: NAME STATE READ WRITE CKSUM zopt0 ONLINE 0 0 0 nda1p3 ONLINE 0 0 0 errors: No known data errors Further reboots into 13S-CA72-nodbg have worked fine. For reference: The HoneyComb is a 16 Cortex-A72 system. I've no clue if releng/13 (-p5) or main could be induced to get similar behavior to what 13S-CA72-nodbg showed for the Optane media. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From nobody Sun Nov 21 21:46:30 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 1B83918A2147 for ; Sun, 21 Nov 2021 21:54:29 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.org (uucp.dinoex.org [IPv6:2a0b:f840::12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "uucp.dinoex.sub.de", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Hy41b6Hdxz4gZr for ; Sun, 21 Nov 2021 21:54:27 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.org [185.220.148.12]) by uucp.dinoex.org (8.17.1/8.17.1) with ESMTPS id 1ALLs40I017559 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Sun, 21 Nov 2021 22:54:05 +0100 (CET) (envelope-from pmc@citylink.dinoex.sub.org) X-MDaemon-Deliver-To: X-Authentication-Warning: uucp.dinoex.org: Host uucp.dinoex.org [185.220.148.12] claimed to be uucp.dinoex.sub.de Received: (from uucp@localhost) by uucp.dinoex.sub.de (8.17.1/8.17.1/Submit) with UUCP id 1ALLs4Ou017558 for freebsd-stable@freebsd.org; Sun, 21 Nov 2021 22:54:04 +0100 (CET) (envelope-from pmc@citylink.dinoex.sub.org) Received: from gate.intra.daemon.contact (gate-e [192.168.98.2]) by citylink.dinoex.sub.de (8.16.1/8.16.1) with ESMTP id 1ALLn06j056805 for ; Sun, 21 Nov 2021 22:49:00 +0100 (CET) (envelope-from peter@gate.intra.daemon.contact) Received: from gate.intra.daemon.contact (gate-e [192.168.98.2]) by gate.intra.daemon.contact (8.16.1/8.16.1) with ESMTPS id 1ALLkU9R056127 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Sun, 21 Nov 2021 22:46:30 +0100 (CET) (envelope-from peter@gate.intra.daemon.contact) Received: (from peter@localhost) by gate.intra.daemon.contact (8.16.1/8.16.1/Submit) id 1ALLkUnH056124 for freebsd-stable@freebsd.org; Sun, 21 Nov 2021 22:46:30 +0100 (CET) (envelope-from peter) Date: Sun, 21 Nov 2021 22:46:30 +0100 From: Peter To: freebsd-stable@freebsd.org Subject: Re: 12.3-RC1 fails to compile perl,tcl86 (ipfw dtrace issue) Message-ID: List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Milter: Spamilter (Reciever: uucp.dinoex.sub.de; Sender-ip: 185.220.148.12; Sender-helo: uucp.dinoex.sub.de;) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (uucp.dinoex.org [185.220.148.12]); Sun, 21 Nov 2021 22:54:07 +0100 (CET) X-Rspamd-Queue-Id: 4Hy41b6Hdxz4gZr X-Spamd-Bar: + Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of pmc@citylink.dinoex.sub.org designates 2a0b:f840::12 as permitted sender) smtp.mailfrom=pmc@citylink.dinoex.sub.org X-Spamd-Result: default: False [1.35 / 15.00]; FAKE_REPLY(1.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.96)[-0.955]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[4]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; NEURAL_SPAM_SHORT(0.60)[0.605]; NEURAL_SPAM_LONG(1.00)[1.000]; DMARC_NA(0.00)[sub.org]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:205376, ipnet:2a0b:f840::/32, country:DE]; RCVD_TLS_LAST(0.00)[] X-ThisMailContainsUnwantedMimeParts: N ## this seems not have arrived on the list at first send ## > dtrace: failed to establish error handler: "/usr/lib/dtrace/ipfw.d", > line 106: failed to copy type of 'inp': Conflicting type is already defined > That file ipfw.d appears to be new in 12.3, but I'm clueless what > the error means (and why it happens only to me). I figured out why - I have "device dtraceall" in my kernel. This is reproducible: > root@y12y:/ # dtrace -h > dtrace: -h requires one or more scripts or enabling options > root@y12y:/ # kldload dtraceall > root@y12y:/ # dtrace -h > dtrace: failed to establish error handler: "/usr/lib/dtrace/ipfw.d", line 106: failed to copy type of 'inp': Conflicting type is already defined > root@y12y:/ # kldunload dtraceall > root@y12y:/ # dtrace -h > dtrace: -h requires one or more scripts or enabling options But we do already have a bug (#254483) for this error. This bug was closed as duplicate to bug #258763, and the latter one was closed as solved with a fix as stated here: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=258763#c7 But the fancy is: 1. that fix appears to have missed the releng/12.3 branch by three days, so it is not in the code. But also 2. if applied, (*surprize*) that fix does NOT help! More than one thing is wrong here, and bug #254483 is *not* a duplicate to #258763. The failure does NOT come from code covered by Mark Johnstons fix. It comes from a neighboring section where Integers are compared, and it fails with a type conflict 8bit vs. 32bit. The problem must be within /usr/lib/dtrace/ipfw.d - and indeed it is (irrelevant parts stripped away): > typedef struct ipfw_match_info { > struct mbuf *m; > } ipfw_match_info_t; > > translator ipfw_match_info_t < struct ip_fw_args *p > { > m = p->m; > }; This does not work. Within the 'struct mbuf' definition is a construct, that looks like this: > uint32_t m_type:8, /* type of data in this mbuf */ > m_flags:24; /* flags; see below */ And it seems that is somehow the cause for the integer size conflict (not implemented?) In the neighboring file /usr/lib/dtrace/mbuf.d this is done distinctively: > translator mbufinfo_t < struct mbuf *p > { > mbuf_addr = (uintptr_t)p; > m_data = p->m_data; > m_len = p->m_len; > m_type = p->m_type & 0xff000000; > m_flags = p->m_type & 0x00ffffff; >}; So probably we should just duplicate that approach for ipfw. Or, could that definition be directly included and called? Does dtrace allow one tranlation to call another? I can't answer that, have never been that deep in dtrace - but I really love the idea that we now get means to look into ipfw. Comes in handy and at the right time. :) cheerio, PMc From nobody Sun Nov 21 22:13:10 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 665251889FF3 for ; Sun, 21 Nov 2021 22:13:23 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic313-20.consmr.mail.gq1.yahoo.com (sonic313-20.consmr.mail.gq1.yahoo.com [98.137.65.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Hy4RQ3M8Sz4m4F for ; Sun, 21 Nov 2021 22:13:22 +0000 (UTC) (envelope-from marklmi@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1637532795; bh=cG1gAy5HWwOfZn+Hcsx5HXHuuvTCZbU+hMWK+n+YiUo=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=s5Nv+nnLLERybDe0+aIpoqwsFLOvhJ8I4xWydxQDuk9U+zLKizdE9u/i3hQBgQToXwsNk2cM3ki9eUOP7sHg1fyQ0OydxOGyVQ4tzhJgLWKr9fJi4P3dM24ASAlZa9GK39OgIWGJy/YZowxsFErDj66MR7Gp4dlsFlqD/ZfMFsqlV5lXWzgAygnsYwrDL8rDA3YEN/IF1qxmBfDu9DLpBy34LAm+qCgkyxFj41w+mBElPEJf9ZKnVh2cQxISPiLAbZozyRgKUBbl4tZNIq7GxchxZwezkxEiNKrJPgzIAejiWO+raisHDlLWVVj/eBEfFGWxf9owGgb/F9J0LZAYtA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1637532795; bh=Ant+mWuHt0q4y4YSdmAMEI+sp/zhIRL74Fqy6zlbkII=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=EZ/ozT4LW7NeqyravQgarXnTnu3IaQz9T5hbre8fQTWXahIh3tXmcfswFVUuxVcV59CrmBz6B4Aj0E2fzdGb5su9xQII7dWf3QfUCuJe43r1VshegUj5v51NNS8amCD9QdrjFuZIpXiKDAhqlEVdQyvYAwldJtQHK/BnY8z9/I+ZkKnxUCZQ0sWRfEz3OQCEsuqbLN/qNcp26b4Ykl+3Ge+eUDQ5xzl+lO/WFMQnHh0EdEHsCEHN6uD3GCE/AhvT6IUC31JqgfI6QcLVLJwM0d/5j7VOrBWPO094chXPSxPOafdwNqRl7kxFBEvktIFMWU4av65MlkGtd29kf8IwTA== X-YMail-OSG: 9n3.fQIVM1nVIDVEmkFkHOfcojNuaSRhH0IP2GsYrTZJjaCsKnPk_EUGOiJ.kMe ifd5SXawihDm_Lm8AnkeFzpV3yJk9Ctm197OzivlTZB3qC6GER3gu77NKVhEYwnLlxtOJA731yJq 764U9BtVUN9.UV2fp7H0BHSnYfbZIxsGharOuS2H69gc5smjBFVn4uZHT_RdFSniSVbskG7eYx3t B6TdAjTtsSNf6rVJRgPNLYV3W4xTtsascWD2h59lPwR0Upkb6CzpgEyoBAu1L2kVyl41CjIV1zlQ mC2ePIAMhTD1P71hP1Hajcj5gAqMUvxe.ql.2VOZE1EyVCdOGwdjAfT3e0wsM.e_kNvCpO5C5Bc1 bQzBXXgnsf3JMx7fFoLT8fuK0_lstTYvF91ee2DAy6_pZKHivS7o_unDMT7EmI2nqEiE9OCkYH2W xOSOvyRfu8kdGDMpv8pqw2pTcQeZ1wnTmvHoxv5Hk61iSSWsYKmga8ZFDi2O52blvWWiY40j5lL. yI7E.V78UfvEV_utimHiFdcGtRFMZTT9IWUoDG8Gd4cvFDuBaxQxGMMYGGIItk7freHd77x9gYoC 8WL6Jy421VcrN5Y3BudCxNF4LTsc1dINIEog70OCiWEAGFwfGhot1580ELqf42ViKDiGaUuVlunY 2fBX0efp06kznXNoi6dFJ9ZUVzO75ALUYWq7e8GvtlDH.M6La69eR0yvaBJ.GqH0v5SN3bvF8iKO OZbn5eVkukIdprFZ4ValWS1BlfWd6P9XAdy.tWl7fHskoqLv8X22EvbF3BFQgA_tQyShPofKp9kx V3EV2fQ0AjS5rfS0GgQ.7DfDRyEvcNJkEizpN9qop2QG5HGm.GRld4YfKqsaof9KCcgQYf30MuNV 7xSV3HBxsJ7RyROQsOmveJFlqAfliUcAELLZ5b8XJE8MPSaa1MwMdKJYskqOfQ1qZMMiscQhzLJ4 secxPRZCst62LA7_eKqYSY3eaAqCf9UQ9G7s9sDJtCEcXbdSJydk.UKk5KWKY2ch50MrrwXksUWb tsfbeuQcBUIkDYlAdAmTAVrtujhY9O7Q2sAq3luv5HLU9cir7hrzuAs3_YEtgNnnc522AnMzT1wH 4XKXU.iU4HtH1rpkX9eVdUr3LHFcOZNsNWCSxsxXoucd5a79rnVCtz0Vehg69oLfPL0E9m89xJGp 5r2byJ1u2DhW3DTAf6U_enVLAADSbiDIlnt6d21oBBAdDFyWvxofzpCQKUmAHwtqRuPmLUdiNVG2 7myCmskTP3tAv5oyJYtHFOjRPRmgSlMRMW4UVPOXwjU0mbOrsfNLzo0Umef5FVeFNZbpLxadsXqC vIdt7dx5Kol62QzcZBrScBu0c69A.v76ZdhfCKEmccf3V7Ja9ROkQA8.xa3QRS9LOjxO5DTdBzd4 cIxH6f9cw2jEreOaH1zyPWeevi3mTjmvmjMM3gpFckaHjAklizBAYQm5HKPxfmIcC86LSTc_1YGs MmOvGXhhsc3sMkXdky82nJq4gGyy8bENlos857rYMwUDUuEjoQ7LUkIftTWSdTYGpTqE.mgQyafR nFXFZpAl9cYWwXKiWORQJzfqF_6cZZE58iOw.GjeLSbzu4MQre7H31FY5lH1vdsGjndonPKoOFMo p.Fhowtiz05YrRFMa56WSNfDA040oFbdP9_AJre_u8lGJRcdTpCjSRuZf7xwF6z7plnaQQCSrLp7 yEBF52ukG9LnTox0HXEab5ZJrckNiNdWCBTQ7LTzCsMm0jtDnX32IqrKR7OEg1KBOmeWYtVs5UFD oSQQ9_FOJrzLgBx3UdsQcq8_v.PQr6152IXObfxS8VtXQjF_FU_XvLtbXZPoz8_WqdMDPsY01Tn2 52nm1MaG99fP0fqTLpBoTRBGvKGxdMLnuMcw0blaRpoO336gj1X08tVguI1JiC.fHgaO.M70pywZ vpWslNWa4wcXUUAjlgEPLuedlV.hO20rhTXUqxIhsrQh0bhMmFJokDADLskasZ6WkqRRp3ZSD4Sw uEMq_h.KoaoCwXu9SkeKht.RnAu.dpE205jSzroWC15Ew_34l0dj4KipwstNoZaPUoyXlz8QtqdR WqV08_9CrD0JkN2jRMblmjjAgwgm2KWvE8Wo7SLVb7O.yH1XK51s8ODl2mGLU96hX2au3ZzMrEsw eP6RLyOHqG81hncgrz.3Mq4P.oA7PNJjSrYeSTzVErnBJWeomM7TzGAns1_dbThz.8l1G6k62mmH c62lmmJUEnQRzSfyVgxoipSj192u7Gj3vcW1_W35zq4Q_GF4- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic313.consmr.mail.gq1.yahoo.com with HTTP; Sun, 21 Nov 2021 22:13:15 +0000 Received: by kubenode524.mail-prod1.omega.gq1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 505bee3b87c130c37f93dca6c4ef992e; Sun, 21 Nov 2021 22:13:11 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) Subject: tmpfs use and "stress --hdd ? --hdd-bytes ??G" can lead to hung up system (16 Cortex-A72's system example) Message-Id: Date: Sun, 21 Nov 2021 14:13:10 -0800 To: FreeBSD-STABLE Mailing List , "freebsd-arm@freebsd.org" X-Mailer: Apple Mail (2.3654.120.0.1.13) References: X-Rspamd-Queue-Id: 4Hy4RQ3M8Sz4m4F X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=s5Nv+nnL; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.65.83 as permitted sender) smtp.mailfrom=marklmi@yahoo.com X-Spamd-Result: default: False [-2.55 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-0.05)[-0.053]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; SUBJECT_HAS_QUESTION(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.83:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.83:from]; RCVD_COUNT_TWO(0.00)[2] Reply-To: marklmi@yahoo.com From: Mark Millard via freebsd-stable X-Original-From: Mark Millard X-ThisMailContainsUnwantedMimeParts: N Context: # uname -apKU FreeBSD CA72_16Gp_ZFS 13.0-STABLE FreeBSD 13.0-STABLE #13 = stable/13-n248062-109330155000-dirty: Sat Nov 13 23:55:14 PST 2021 = root@CA72_16Gp_ZFS:/usr/obj/BUILDs/13S-CA72-nodbg-clang/usr/13S-src/arm64.= aarch64/sys/GENERIC-NODBG-CA72 arm64 aarch64 1300520 1300520 (It is a non-debug kernel build, but with symbols.) 64 GiBytes of RAM Swap: 251904Mi Total 16-core Cortex-A72 system (HoneyComb) root-on-ZFS on Optane media in the PCIe slot The following sort of sequence using tmpfs tends to hangs up the system long before the maximum swap usage would happen: # mount -ttmpfs tmpfs /mnt # cd /mnt # stress --hdd 16 --hdd-bytes 16G The details of processes seeming to hang up does vary. Sometimes ^C, ^T, and the like do not seem to work (or echo) at all. Top can stop updating for the likes of: # stress --hdd 12 --hdd-bytes 22G until ^C leads to stress exiting. Based on what top was reporting at the point its updates stopped, there still was lots of swap-space and no apparent need to have swapped out the kernel stacks for the top process. For example: 22% used. Another issue is that having systat -swap going as well leads to partial hang ups even for the likes of: # stress --hdd 8 --hdd-bytes 32G "Partial" in that some things stop but I've never had to force a reboot to get control: ^C for the stress run eventually works. I do not seem to have problems like any of the above for the likes of: # stress --hdd 1 --hdd-bytes 270G For this, "systat -swap" does progress the like of: Device/Path Size Used |0% /10 /20 /30 /40 / 60\ 70\ 80\ = 90\ 100| gpt/CA72opt0SWP 246G 154G XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX But not with the larger ---hdd figures above. I've not yet tried main [so: 14]. Note: These experiments are from attmpting to reproduce hangups seen during pourdiere-devel builds that have USE_TMPFS=3Dall and builders that are generating indefinitely growing log files. (This was actually under main targeting main .) =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) From nobody Sun Nov 21 22:18:29 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id C9EAB18918BA for ; Sun, 21 Nov 2021 22:27:25 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.org (uucp.dinoex.org [IPv6:2a0b:f840::12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "uucp.dinoex.sub.de", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Hy4ld3hF1z4rXV for ; Sun, 21 Nov 2021 22:27:25 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.org [185.220.148.12]) by uucp.dinoex.org (8.17.1/8.17.1) with ESMTPS id 1ALMR48g043584 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sun, 21 Nov 2021 23:27:04 +0100 (CET) (envelope-from pmc@citylink.dinoex.sub.org) X-Authentication-Warning: uucp.dinoex.org: Host uucp.dinoex.org [185.220.148.12] claimed to be uucp.dinoex.sub.de Received: (from uucp@localhost) by uucp.dinoex.sub.de (8.17.1/8.17.1/Submit) with UUCP id 1ALMR426043583; Sun, 21 Nov 2021 23:27:04 +0100 (CET) (envelope-from pmc@citylink.dinoex.sub.org) Received: from gate.intra.daemon.contact (gate-e [192.168.98.2]) by citylink.dinoex.sub.de (8.16.1/8.16.1) with ESMTP id 1ALMJ1nk065549; Sun, 21 Nov 2021 23:19:01 +0100 (CET) (envelope-from peter@gate.intra.daemon.contact) Received: from gate.intra.daemon.contact (gate-e [192.168.98.2]) by gate.intra.daemon.contact (8.16.1/8.16.1) with ESMTPS id 1ALMIUL5065195 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sun, 21 Nov 2021 23:18:30 +0100 (CET) (envelope-from peter@gate.intra.daemon.contact) Received: (from peter@localhost) by gate.intra.daemon.contact (8.16.1/8.16.1/Submit) id 1ALMIUBP065194; Sun, 21 Nov 2021 23:18:30 +0100 (CET) (envelope-from peter) Date: Sun, 21 Nov 2021 23:18:29 +0100 From: Peter To: freebsd-stable@freebsd.org Cc: ltning-freebsd-stable@anduin.net Subject: Re: 12.3-RC1 errors on boot and shutdown Message-ID: List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: X-Milter: Spamilter (Reciever: uucp.dinoex.sub.de; Sender-ip: 185.220.148.12; Sender-helo: uucp.dinoex.sub.de;) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (uucp.dinoex.org [185.220.148.12]); Sun, 21 Nov 2021 23:27:07 +0100 (CET) X-Rspamd-Queue-Id: 4Hy4ld3hF1z4rXV X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N Eirik =D8verby wrote; > > > wait....what? > > > This means we *can not* upgrade our 100+ systems upon release, because > > > we cannot at the same time roll out local changes to basic stuff in > > > /etc/rc.d .. > >=20 > > With approval of re_at_, the change's merged to stable/12 and will be i= ncluded in the 12.3-RELEASE, so be cool. > Thank you. I was just .. momentarily put off by the additional work it > would have entailed. Ups, You're indeed serious - I was imagining You were nitpicking. So, next time you run into such an issue, you just roll out something like this one, at any time before: https://daemon.contact/gitr/tools/tree/rc.d/RCTLLATE This is a twister. It orders the sequence of startup. I never change things in /etc/rc.d. These twisters go into local rc.d directory. And anyway, this issue would hit You only when You have separate filesystems for / and /usr. Sure You do? @Eugene: Thanks for fixing! :)) cheerio, PMc From nobody Mon Nov 22 15:45:09 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 0DCF31898039 for ; Mon, 22 Nov 2021 15:45:13 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-il1-x12d.google.com (mail-il1-x12d.google.com [IPv6:2607:f8b0:4864:20::12d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HyWn46gZ6z3Cwj for ; Mon, 22 Nov 2021 15:45:12 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-il1-x12d.google.com with SMTP id e8so18554380ilu.9 for ; Mon, 22 Nov 2021 07:45:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=PtqcqwHz00OKqaLTKAZqyso2hvajGVTxS3tzAKZ04fg=; b=hBWwwKmBDLVEohU/139Iddf+rtA64DzEIFsWH+IvR/g8kppkYGaXVcAvWk9zGp4Jmr xLui5ab/+74G27ASGZq/Jeb3l0rRjsP50NuZ0y/iBSOB5i+P18xl5b4UYUGESb7tVfeM vOTNVPlzIcZASrqdhRk+XqcGV7en1oPeHf1xYlCTUHIe4NlxhdEKBOT86MjMWoI8Nm/k AgXO8G6FJHdP5FVG7U8lbWChJgv59Yiuc9n+Si/hxExfOQP4VM1yDYZpLmRnEi9mC9g8 nipve9odGcoIguwDIITiXvTWka+R1W189tilKYh6ZFx+3IhqmdNShz2RXvfC6a6SPeay a1Hw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to; bh=PtqcqwHz00OKqaLTKAZqyso2hvajGVTxS3tzAKZ04fg=; b=pYZw6n7vcwtx4YPtavh8O8dd+tR5OPZtwpnM/h3hqwPOaThmuOosATz7crFsSKK9AV JBpn3q+kuC3S1nPjmQm3CH67o9Ye6qpm+f+VdjmflIRbpFp+/EayuvsbyJXq6B8wUXQV vFZI3F94z9klqBwb/gTwsQXdE3BXvOwUvvYwNLPVpgu6SedBTWS2/Fy67P3nEBVeQz9c xmQWL8ZE1lRJg4nqYHB+N4Qn9p/LfrfyqEXUcieVusiEx4jwJQT49b8c3P154RdBXYvG Zx1FSgsos1053Oxpqd5pxsRTrPUWbuHta1uhZQb0/51OwgAJbtCF4HPCLfaNg8wfsacc zHpw== X-Gm-Message-State: AOAM530J3bY5XMlornD35423HgIMZNAJ8UBbGWTFGvCJ1VU8vSw2SHrX JUgOHvNW05+14n3kRkm6Gte75xQvvtw= X-Google-Smtp-Source: ABdhPJzNme+Le0rcbfeq7RGLhu7VdoqqUsTetBFly025+XhccmOMl+/LIxRMKhOPda3AdkklhWJk7Q== X-Received: by 2002:a92:c74a:: with SMTP id y10mr20271752ilp.122.1637595912118; Mon, 22 Nov 2021 07:45:12 -0800 (PST) Received: from nuc ([142.126.186.191]) by smtp.gmail.com with ESMTPSA id x15sm6721011ill.20.2021.11.22.07.45.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Nov 2021 07:45:11 -0800 (PST) Date: Mon, 22 Nov 2021 10:45:09 -0500 From: Mark Johnston To: Peter Cc: freebsd-stable@freebsd.org Subject: Re: 12.3-RC1 fails to compile perl,tcl86 (ipfw dtrace issue) Message-ID: References: List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 4HyWn46gZ6z3Cwj X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On Sun, Nov 21, 2021 at 10:46:30PM +0100, Peter wrote: > ## this seems not have arrived on the list at first send ## > > > dtrace: failed to establish error handler: "/usr/lib/dtrace/ipfw.d", > > line 106: failed to copy type of 'inp': Conflicting type is already defined > > > That file ipfw.d appears to be new in 12.3, but I'm clueless what > > the error means (and why it happens only to me). > > > I figured out why - I have "device dtraceall" in my kernel. This is > reproducible: > > > root@y12y:/ # dtrace -h > > dtrace: -h requires one or more scripts or enabling options > > root@y12y:/ # kldload dtraceall > > root@y12y:/ # dtrace -h > > dtrace: failed to establish error handler: "/usr/lib/dtrace/ipfw.d", line 106: failed to copy type of 'inp': Conflicting type is already defined > > root@y12y:/ # kldunload dtraceall > > root@y12y:/ # dtrace -h > > dtrace: -h requires one or more scripts or enabling options > > > But we do already have a bug (#254483) for this error. > > This bug was closed as duplicate to bug #258763, and the latter one > was closed as solved with a fix as stated here: > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=258763#c7 > > > But the fancy is: > > 1. that fix appears to have missed the releng/12.3 branch by three > days, so it is not in the code. But also > > 2. if applied, (*surprize*) that fix does NOT help! Hmm, that is indeed surprising. I'm able to reproduce the problem locally. > More than one thing is wrong here, and bug #254483 is *not* a > duplicate to #258763. > > The failure does NOT come from code covered by Mark Johnstons fix. > > It comes from a neighboring section where Integers are compared, and > it fails with a type conflict 8bit vs. 32bit. > > > The problem must be within /usr/lib/dtrace/ipfw.d - and indeed > it is (irrelevant parts stripped away): > > > typedef struct ipfw_match_info { > > struct mbuf *m; > > } ipfw_match_info_t; > > > > translator ipfw_match_info_t < struct ip_fw_args *p > { > > m = p->m; > > }; > > This does not work. Within the 'struct mbuf' definition is a > construct, that looks like this: > > > uint32_t m_type:8, /* type of data in this mbuf */ > > m_flags:24; /* flags; see below */ > > And it seems that is somehow the cause for the integer size conflict > (not implemented?) How did you come to that conclusion? > In the neighboring file /usr/lib/dtrace/mbuf.d this is done > distinctively: > > > translator mbufinfo_t < struct mbuf *p > { > > mbuf_addr = (uintptr_t)p; > > m_data = p->m_data; > > m_len = p->m_len; > > m_type = p->m_type & 0xff000000; > > m_flags = p->m_type & 0x00ffffff; > >}; > > So probably we should just duplicate that approach for ipfw. > > Or, could that definition be directly included and called? Does dtrace > allow one tranlation to call another? > I can't answer that, have never been that deep in dtrace - but I > really love the idea that we now get means to look into ipfw. Comes in > handy and at the right time. :) From nobody Mon Nov 22 16:05:43 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4AC5E18A30E6 for ; Mon, 22 Nov 2021 16:05:47 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-io1-xd2b.google.com (mail-io1-xd2b.google.com [IPv6:2607:f8b0:4864:20::d2b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HyXDp44jLz3LZl for ; Mon, 22 Nov 2021 16:05:46 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-io1-xd2b.google.com with SMTP id x6so2293411iol.13 for ; Mon, 22 Nov 2021 08:05:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=3gwf24ZYZ8OyvBGj6l28VI3KCXPMGSKpTHuUOTqFK2k=; b=c5tMa21o+bz82VRsJYMS2XuoT+xaPXftLE6FZyMAtoRGmj4rxYi4FOcRZJFDBje0xO UKa9QV0b4Sw8zBgMkMcn+sZRDODcYSRfQ2DeSeHmXWkQn46D1L8esMmRjBNIs0toA1Nn qKUUqK6co7BnyAc6L98DbMUQ1U8SiWQ6A2ShRfUyXhTVBy4wl6s6dzSgVFXysIPlbv/k 9qVh2BzY9tWtgw+G9InkIR21SLJALgssX83ND91WYSZ9xgskPyzVi3Rtt0HIiHKU99oQ N3lSsM0RNae6FV+klh0/IByp787cW38uHJIpDC5eMRVtCUMirq34Stxx330o84KSfB34 jpNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to; bh=3gwf24ZYZ8OyvBGj6l28VI3KCXPMGSKpTHuUOTqFK2k=; b=wpVAMGFwzUuU9aGwKV/qvkPe1Fu7eqViFM4yGqJnUqKeUWjkSiYDcGexzGzqbb5kLn AkD9P9KMah7zGdjkPEl/3i0agbDNRpVjRVtP+xWC2bEHmyVCf2oSS1De3kiEjDa2RUhR XQrmqXMBkP/qQmPw8ARtqM4BmIsp7h0eyS/3zPru0X7q3ScVIIhZ+HbBdNnCeR7RPaG7 IFLSVWrKxapOYZaLoc9X36KAn5W52/ZZiawFPxIF5+GHKCWj524NPu+tU2h6GqgrsyM3 ieuJGLBdZVrtWpCI1NEONyTSKu8OS04jgx1K78FVIvu40m7UuXAxoJbBq774fErBAtCC QWPg== X-Gm-Message-State: AOAM531BniPbyMtJPjzmwJ7wno/g5nDJ+7gt8welK5DKCgxJb7AIbmkP l7A7Ub26cthtrmtWBoenEiLd72q8b3Q= X-Google-Smtp-Source: ABdhPJzs3wB/UXyQw9AE1usBN18DlPEuAFqdtr7fXU5M75+5L/dSleLVuaz0JLQsrAXDFM4Hi86U5w== X-Received: by 2002:a05:6602:1581:: with SMTP id e1mr23439495iow.64.1637597145829; Mon, 22 Nov 2021 08:05:45 -0800 (PST) Received: from nuc ([142.126.186.191]) by smtp.gmail.com with ESMTPSA id t2sm5237591iob.1.2021.11.22.08.05.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Nov 2021 08:05:45 -0800 (PST) Date: Mon, 22 Nov 2021 11:05:43 -0500 From: Mark Johnston To: Peter Cc: freebsd-stable@freebsd.org Subject: Re: 12.3-RC1 fails to compile perl,tcl86 (ipfw dtrace issue) Message-ID: References: List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 4HyXDp44jLz3LZl X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=c5tMa21o; dmarc=none; spf=pass (mx1.freebsd.org: domain of markjdb@gmail.com designates 2607:f8b0:4864:20::d2b as permitted sender) smtp.mailfrom=markjdb@gmail.com X-Spamd-Result: default: False [-2.70 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::d2b:from]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MID_RHS_NOT_FQDN(0.50)[]; FORGED_SENDER(0.30)[markj@freebsd.org,markjdb@gmail.com]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[markj@freebsd.org,markjdb@gmail.com]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-ThisMailContainsUnwantedMimeParts: N On Mon, Nov 22, 2021 at 10:45:09AM -0500, Mark Johnston wrote: > On Sun, Nov 21, 2021 at 10:46:30PM +0100, Peter wrote: > > ## this seems not have arrived on the list at first send ## > > > > > dtrace: failed to establish error handler: "/usr/lib/dtrace/ipfw.d", > > > line 106: failed to copy type of 'inp': Conflicting type is already defined > > > > > That file ipfw.d appears to be new in 12.3, but I'm clueless what > > > the error means (and why it happens only to me). > > > > > > I figured out why - I have "device dtraceall" in my kernel. This is > > reproducible: > > > > > root@y12y:/ # dtrace -h > > > dtrace: -h requires one or more scripts or enabling options > > > root@y12y:/ # kldload dtraceall > > > root@y12y:/ # dtrace -h > > > dtrace: failed to establish error handler: "/usr/lib/dtrace/ipfw.d", line 106: failed to copy type of 'inp': Conflicting type is already defined > > > root@y12y:/ # kldunload dtraceall > > > root@y12y:/ # dtrace -h > > > dtrace: -h requires one or more scripts or enabling options > > > > > > But we do already have a bug (#254483) for this error. > > > > This bug was closed as duplicate to bug #258763, and the latter one > > was closed as solved with a fix as stated here: > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=258763#c7 > > > > > > But the fancy is: > > > > 1. that fix appears to have missed the releng/12.3 branch by three > > days, so it is not in the code. But also > > > > 2. if applied, (*surprize*) that fix does NOT help! > > Hmm, that is indeed surprising. I'm able to reproduce the problem > locally. > > > More than one thing is wrong here, and bug #254483 is *not* a > > duplicate to #258763. > > > > The failure does NOT come from code covered by Mark Johnstons fix. > > > > It comes from a neighboring section where Integers are compared, and > > it fails with a type conflict 8bit vs. 32bit. > > > > > > The problem must be within /usr/lib/dtrace/ipfw.d - and indeed > > it is (irrelevant parts stripped away): > > > > > typedef struct ipfw_match_info { > > > struct mbuf *m; > > > } ipfw_match_info_t; > > > > > > translator ipfw_match_info_t < struct ip_fw_args *p > { > > > m = p->m; > > > }; > > > > This does not work. Within the 'struct mbuf' definition is a > > construct, that looks like this: > > > > > uint32_t m_type:8, /* type of data in this mbuf */ > > > m_flags:24; /* flags; see below */ > > > > And it seems that is somehow the cause for the integer size conflict > > (not implemented?) > > How did you come to that conclusion? Indeed, the problem had to do with handling of bitfield types in libctf. The problem you found is fixed by https://cgit.freebsd.org/src/commit/?id=3cbb4cc200f8a0ad7ed08233425ea54524a21f1c but that change was not merged to 12. I just merged it to stable/12 and will try to get both libctf changes into 12.3, as both are required for dtrace to work properly while ipfw.ko is loaded. From nobody Mon Nov 22 20:52:38 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id E65471888127 for ; Mon, 22 Nov 2021 21:00:19 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.org (uucp.dinoex.org [IPv6:2a0b:f840::12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "uucp.dinoex.sub.de", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Hyfmg4YDkz3HPm; Mon, 22 Nov 2021 21:00:19 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.org [185.220.148.12]) by uucp.dinoex.org (8.17.1/8.17.1) with ESMTPS id 1AML04Xs081873 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 22 Nov 2021 22:00:04 +0100 (CET) (envelope-from pmc@citylink.dinoex.sub.org) X-Authentication-Warning: uucp.dinoex.org: Host uucp.dinoex.org [185.220.148.12] claimed to be uucp.dinoex.sub.de Received: (from uucp@localhost) by uucp.dinoex.sub.de (8.17.1/8.17.1/Submit) with UUCP id 1AML04EI081872; Mon, 22 Nov 2021 22:00:04 +0100 (CET) (envelope-from pmc@citylink.dinoex.sub.org) Received: from gate.intra.daemon.contact (gate-e [192.168.98.2]) by citylink.dinoex.sub.de (8.16.1/8.16.1) with ESMTP id 1AMKrKtM037906; Mon, 22 Nov 2021 21:53:20 +0100 (CET) (envelope-from peter@gate.intra.daemon.contact) Received: from gate.intra.daemon.contact (gate-e [192.168.98.2]) by gate.intra.daemon.contact (8.16.1/8.16.1) with ESMTPS id 1AMKqcUm037806 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 22 Nov 2021 21:52:38 +0100 (CET) (envelope-from peter@gate.intra.daemon.contact) Received: (from peter@localhost) by gate.intra.daemon.contact (8.16.1/8.16.1/Submit) id 1AMKqcCC037805; Mon, 22 Nov 2021 21:52:38 +0100 (CET) (envelope-from peter) Date: Mon, 22 Nov 2021 21:52:38 +0100 From: Peter To: Mark Johnston Cc: freebsd-stable@freebsd.org Subject: Re: 12.3-RC1 fails to compile perl,tcl86 (ipfw dtrace issue) Message-ID: References: List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Milter: Spamilter (Reciever: uucp.dinoex.sub.de; Sender-ip: 185.220.148.12; Sender-helo: uucp.dinoex.sub.de;) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (uucp.dinoex.org [185.220.148.12]); Mon, 22 Nov 2021 22:00:07 +0100 (CET) X-Rspamd-Queue-Id: 4Hyfmg4YDkz3HPm X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On Mon, Nov 22, 2021 at 10:45:09AM -0500, Mark Johnston wrote: ! On Sun, Nov 21, 2021 at 10:46:30PM +0100, Peter wrote: ! > ## this seems not have arrived on the list at first send ## ! > ! > > dtrace: failed to establish error handler: "/usr/lib/dtrace/ipfw.d", ! > > line 106: failed to copy type of 'inp': Conflicting type is already defined ! > ! > > That file ipfw.d appears to be new in 12.3, but I'm clueless what ! > > the error means (and why it happens only to me). ! > ! > ! > I figured out why - I have "device dtraceall" in my kernel. This is ! > reproducible: ! > ! > > root@y12y:/ # dtrace -h ! > > dtrace: -h requires one or more scripts or enabling options ! > > root@y12y:/ # kldload dtraceall ! > > root@y12y:/ # dtrace -h ! > > dtrace: failed to establish error handler: "/usr/lib/dtrace/ipfw.d", line 106: failed to copy type of 'inp': Conflicting type is already defined ! > > root@y12y:/ # kldunload dtraceall ! > > root@y12y:/ # dtrace -h ! > > dtrace: -h requires one or more scripts or enabling options ! > ! > ! > But we do already have a bug (#254483) for this error. ! > ! > This bug was closed as duplicate to bug #258763, and the latter one ! > was closed as solved with a fix as stated here: ! > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=258763#c7 ! > ! > ! > But the fancy is: ! > ! > 1. that fix appears to have missed the releng/12.3 branch by three ! > days, so it is not in the code. But also ! > ! > 2. if applied, (*surprize*) that fix does NOT help! ! ! Hmm, that is indeed surprising. I'm able to reproduce the problem ! locally. ! ! > More than one thing is wrong here, and bug #254483 is *not* a ! > duplicate to #258763. ! > ! > The failure does NOT come from code covered by Mark Johnstons fix. ! > ! > It comes from a neighboring section where Integers are compared, and ! > it fails with a type conflict 8bit vs. 32bit. ! > ! > ! > The problem must be within /usr/lib/dtrace/ipfw.d - and indeed ! > it is (irrelevant parts stripped away): ! > ! > > typedef struct ipfw_match_info { ! > > struct mbuf *m; ! > > } ipfw_match_info_t; ! > > ! > > translator ipfw_match_info_t < struct ip_fw_args *p > { ! > > m = p->m; ! > > }; ! > ! > This does not work. Within the 'struct mbuf' definition is a ! > construct, that looks like this: ! > ! > > uint32_t m_type:8, /* type of data in this mbuf */ ! > > m_flags:24; /* flags; see below */ ! > ! > And it seems that is somehow the cause for the integer size conflict ! > (not implemented?) ! ! How did you come to that conclusion? Well, that's my trade secret :) | Indeed, the problem had to do with handling of bitfield types in libctf. | The problem you found is fixed by | https://cgit.freebsd.org/src/commit/?id=3cbb4cc200f8a0ad7ed08233425ea54524a21f1c | but that change was not merged to 12. I just merged it to stable/12 and | will try to get both libctf changes into 12.3, as both are required for | dtrace to work properly while ipfw.ko is loaded. That would be great, but ... I gave that patch a quick try, and doesn't yet seem to work, it is now a different error message: > dtrace: failed to establish error handler: "/usr/lib/dtrace/ipfw.d", line 106: failed to copy type of 'inp': Invalid type identifier Cheerio, PMc From nobody Mon Nov 22 21:09:43 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 05790188C0D3 for ; Mon, 22 Nov 2021 21:09:53 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-io1-xd33.google.com (mail-io1-xd33.google.com [IPv6:2607:f8b0:4864:20::d33]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Hyfzh6QGSz3L59 for ; Mon, 22 Nov 2021 21:09:52 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-io1-xd33.google.com with SMTP id k21so25201841ioh.4 for ; Mon, 22 Nov 2021 13:09:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=plrii865tq8mljU3gg6Re9KcoG0PnDrwh/QRnp1g5wY=; b=gS3ok98n2cr+DYkC8egU94U59plAcaNdqaIsPoqZ+bFVAlh7M1eg2Y6eoolTNSkHuz oqGozPeBwvtva4p8ofxrUKuqU31ID2NFBnOU0AxyG487CeDGyf8SIMXJ4nMI9YKslq2U 1HjQZBPu1lkMwtK5YeLe1mGF3dn5ixg3DN1fQz3wzCCyLAI+KNZzGkuXHnQGf3TOneOE Xu78MGngZ3YFHtzYuKmcF7TNwK9ZaK18wbylQvfGIovTggbvF/puGldjErGTLzcpQS1n oCbmRnz7Fa17iytWmyIWn8iZt+tz/BKoveS4XQJwsZN5sta/bN9S4OtQC9xR4jHaQdKZ nTyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to; bh=plrii865tq8mljU3gg6Re9KcoG0PnDrwh/QRnp1g5wY=; b=zZZtGS04cmQN/Q7Tp7l2ponCsDOUG+4wSI/dXIPYwb9+3uL3fil9MNvXCC97LIYRVx URGqm3OT9dG0Dr8/rSmzVCdLjxwuwzzlmkmOum6V9GI0i20INS5qMeGXzoDyBZ/IDLNN VGWtP5QXB3wnoJTT1/5nibKmu1Noa6pGcIjvQ0ROQPHEbTOkS3HDTmpaRnKhJ9TgKx6B 3/MPZqIhI/3lca8brHonAwVpcW+8KsGQWncq3mZ4RZjaF3HvqsMcnvm+FBGjXs9RYUXn HYDW1iQiBn2XrTAVNZyVjxK/v+35oMic0koPRbN7EGN3z5dAtxYwd37F7iJMAc0M1UBf DSuw== X-Gm-Message-State: AOAM531aSDIiz9F6XR5BQ19xmKgaDKxVjjMrsHVBOPjIXeTVvBKesAke 7kBaBMPIXhIV4mqwpBuj3e3EmhDkqI0= X-Google-Smtp-Source: ABdhPJyjOtjqe30PYcHaAVwZ+MMoUQgyxV9FDmewNXOOWbQ06Ym8kfAbOlf4O6DG2ThlrfHsF5zOTA== X-Received: by 2002:a05:6638:2487:: with SMTP id x7mr52908282jat.94.1637615385733; Mon, 22 Nov 2021 13:09:45 -0800 (PST) Received: from nuc ([142.126.186.191]) by smtp.gmail.com with ESMTPSA id w19sm3831264iov.12.2021.11.22.13.09.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 22 Nov 2021 13:09:45 -0800 (PST) Date: Mon, 22 Nov 2021 16:09:43 -0500 From: Mark Johnston To: Peter Cc: freebsd-stable@freebsd.org Subject: Re: 12.3-RC1 fails to compile perl,tcl86 (ipfw dtrace issue) Message-ID: References: List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 4Hyfzh6QGSz3L59 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On Mon, Nov 22, 2021 at 09:52:38PM +0100, Peter wrote: > On Mon, Nov 22, 2021 at 10:45:09AM -0500, Mark Johnston wrote: > ! On Sun, Nov 21, 2021 at 10:46:30PM +0100, Peter wrote: > ! > ## this seems not have arrived on the list at first send ## > ! > > ! > > dtrace: failed to establish error handler: "/usr/lib/dtrace/ipfw.d", > ! > > line 106: failed to copy type of 'inp': Conflicting type is already defined > ! > > ! > > That file ipfw.d appears to be new in 12.3, but I'm clueless what > ! > > the error means (and why it happens only to me). > ! > > ! > > ! > I figured out why - I have "device dtraceall" in my kernel. This is > ! > reproducible: > ! > > ! > > root@y12y:/ # dtrace -h > ! > > dtrace: -h requires one or more scripts or enabling options > ! > > root@y12y:/ # kldload dtraceall > ! > > root@y12y:/ # dtrace -h > ! > > dtrace: failed to establish error handler: "/usr/lib/dtrace/ipfw.d", line 106: failed to copy type of 'inp': Conflicting type is already defined > ! > > root@y12y:/ # kldunload dtraceall > ! > > root@y12y:/ # dtrace -h > ! > > dtrace: -h requires one or more scripts or enabling options > ! > > ! > > ! > But we do already have a bug (#254483) for this error. > ! > > ! > This bug was closed as duplicate to bug #258763, and the latter one > ! > was closed as solved with a fix as stated here: > ! > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=258763#c7 > ! > > ! > > ! > But the fancy is: > ! > > ! > 1. that fix appears to have missed the releng/12.3 branch by three > ! > days, so it is not in the code. But also > ! > > ! > 2. if applied, (*surprize*) that fix does NOT help! > ! > ! Hmm, that is indeed surprising. I'm able to reproduce the problem > ! locally. > ! > ! > More than one thing is wrong here, and bug #254483 is *not* a > ! > duplicate to #258763. > ! > > ! > The failure does NOT come from code covered by Mark Johnstons fix. > ! > > ! > It comes from a neighboring section where Integers are compared, and > ! > it fails with a type conflict 8bit vs. 32bit. > ! > > ! > > ! > The problem must be within /usr/lib/dtrace/ipfw.d - and indeed > ! > it is (irrelevant parts stripped away): > ! > > ! > > typedef struct ipfw_match_info { > ! > > struct mbuf *m; > ! > > } ipfw_match_info_t; > ! > > > ! > > translator ipfw_match_info_t < struct ip_fw_args *p > { > ! > > m = p->m; > ! > > }; > ! > > ! > This does not work. Within the 'struct mbuf' definition is a > ! > construct, that looks like this: > ! > > ! > > uint32_t m_type:8, /* type of data in this mbuf */ > ! > > m_flags:24; /* flags; see below */ > ! > > ! > And it seems that is somehow the cause for the integer size conflict > ! > (not implemented?) > ! > ! How did you come to that conclusion? > > Well, that's my trade secret :) > > | Indeed, the problem had to do with handling of bitfield types in libctf. > | The problem you found is fixed by > | https://cgit.freebsd.org/src/commit/?id=3cbb4cc200f8a0ad7ed08233425ea54524a21f1c > | but that change was not merged to 12. I just merged it to stable/12 and > | will try to get both libctf changes into 12.3, as both are required for > | dtrace to work properly while ipfw.ko is loaded. > > That would be great, but ... I gave that patch a quick try, and > doesn't yet seem to work, it is now a different error message: > > > dtrace: failed to establish error handler: "/usr/lib/dtrace/ipfw.d", line 106: failed to copy type of 'inp': Invalid type identifier Hmm. Just to be clear, this is with both libctf patches applied, libctf.so reinstalled, and with a pristine ipfw.d? If so, I'm not able to reproduce the problem. From nobody Tue Nov 23 00:07:22 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id ACABB18AEA5B for ; Tue, 23 Nov 2021 00:18:30 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.org (uucp.dinoex.org [IPv6:2a0b:f840::12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "uucp.dinoex.sub.de", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Hyl9L3Dhpz3sL9; Tue, 23 Nov 2021 00:18:30 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.org [185.220.148.12]) by uucp.dinoex.org (8.17.1/8.17.1) with ESMTPS id 1AN0I4hA032969 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Tue, 23 Nov 2021 01:18:04 +0100 (CET) (envelope-from pmc@citylink.dinoex.sub.org) X-Authentication-Warning: uucp.dinoex.org: Host uucp.dinoex.org [185.220.148.12] claimed to be uucp.dinoex.sub.de Received: (from uucp@localhost) by uucp.dinoex.sub.de (8.17.1/8.17.1/Submit) with UUCP id 1AN0I4M2032968; Tue, 23 Nov 2021 01:18:04 +0100 (CET) (envelope-from pmc@citylink.dinoex.sub.org) Received: from gate.intra.daemon.contact (gate-e [192.168.98.2]) by citylink.dinoex.sub.de (8.16.1/8.16.1) with ESMTP id 1AN08KrB097180; Tue, 23 Nov 2021 01:08:20 +0100 (CET) (envelope-from peter@gate.intra.daemon.contact) Received: from gate.intra.daemon.contact (gate-e [192.168.98.2]) by gate.intra.daemon.contact (8.16.1/8.16.1) with ESMTPS id 1AN07MXS096725 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Tue, 23 Nov 2021 01:07:22 +0100 (CET) (envelope-from peter@gate.intra.daemon.contact) Received: (from peter@localhost) by gate.intra.daemon.contact (8.16.1/8.16.1/Submit) id 1AN07M0J096724; Tue, 23 Nov 2021 01:07:22 +0100 (CET) (envelope-from peter) Date: Tue, 23 Nov 2021 01:07:22 +0100 From: Peter To: Mark Johnston Cc: freebsd-stable@freebsd.org Subject: Re: 12.3-RC1 fails to compile perl,tcl86 (ipfw dtrace issue) Message-ID: References: List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Milter: Spamilter (Reciever: uucp.dinoex.sub.de; Sender-ip: 185.220.148.12; Sender-helo: uucp.dinoex.sub.de;) X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (uucp.dinoex.org [185.220.148.12]); Tue, 23 Nov 2021 01:18:07 +0100 (CET) X-Rspamd-Queue-Id: 4Hyl9L3Dhpz3sL9 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On Mon, Nov 22, 2021 at 04:09:43PM -0500, Mark Johnston wrote: ! On Mon, Nov 22, 2021 at 09:52:38PM +0100, Peter wrote: ! > On Mon, Nov 22, 2021 at 10:45:09AM -0500, Mark Johnston wrote: ! > ! On Sun, Nov 21, 2021 at 10:46:30PM +0100, Peter wrote: ! > ! > ## this seems not have arrived on the list at first send ## ! > ! > ! > ! > > dtrace: failed to establish error handler: "/usr/lib/dtrace/ipfw.d", ! > ! > > line 106: failed to copy type of 'inp': Conflicting type is already defined ! > ! > ! > ! > > That file ipfw.d appears to be new in 12.3, but I'm clueless what ! > ! > > the error means (and why it happens only to me). ! > ! > ! > ! > ! > ! > I figured out why - I have "device dtraceall" in my kernel. This is ! > ! > reproducible: ! > ! > ! > ! > > root@y12y:/ # dtrace -h ! > ! > > dtrace: -h requires one or more scripts or enabling options ! > ! > > root@y12y:/ # kldload dtraceall ! > ! > > root@y12y:/ # dtrace -h ! > ! > > dtrace: failed to establish error handler: "/usr/lib/dtrace/ipfw.d", line 106: failed to copy type of 'inp': Conflicting type is already defined ! > ! > > root@y12y:/ # kldunload dtraceall ! > ! > > root@y12y:/ # dtrace -h ! > ! > > dtrace: -h requires one or more scripts or enabling options ! > ! > ! > ! > ! > ! > But we do already have a bug (#254483) for this error. ! > ! > ! > ! > This bug was closed as duplicate to bug #258763, and the latter one ! > ! > was closed as solved with a fix as stated here: ! > ! > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=258763#c7 ! > ! > ! > ! > ! > ! > But the fancy is: ! > ! > ! > ! > 1. that fix appears to have missed the releng/12.3 branch by three ! > ! > days, so it is not in the code. But also ! > ! > ! > ! > 2. if applied, (*surprize*) that fix does NOT help! ! > ! ! > ! Hmm, that is indeed surprising. I'm able to reproduce the problem ! > ! locally. ! > ! ! > ! > More than one thing is wrong here, and bug #254483 is *not* a ! > ! > duplicate to #258763. ! > ! > ! > ! > The failure does NOT come from code covered by Mark Johnstons fix. ! > ! > ! > ! > It comes from a neighboring section where Integers are compared, and ! > ! > it fails with a type conflict 8bit vs. 32bit. ! > ! > ! > ! > ! > ! > The problem must be within /usr/lib/dtrace/ipfw.d - and indeed ! > ! > it is (irrelevant parts stripped away): ! > ! > ! > ! > > typedef struct ipfw_match_info { ! > ! > > struct mbuf *m; ! > ! > > } ipfw_match_info_t; ! > ! > > ! > ! > > translator ipfw_match_info_t < struct ip_fw_args *p > { ! > ! > > m = p->m; ! > ! > > }; ! > ! > ! > ! > This does not work. Within the 'struct mbuf' definition is a ! > ! > construct, that looks like this: ! > ! > ! > ! > > uint32_t m_type:8, /* type of data in this mbuf */ ! > ! > > m_flags:24; /* flags; see below */ ! > ! > ! > ! > And it seems that is somehow the cause for the integer size conflict ! > ! > (not implemented?) ! > ! ! > ! How did you come to that conclusion? ! > ! > Well, that's my trade secret :) ! > ! > | Indeed, the problem had to do with handling of bitfield types in libctf. ! > | The problem you found is fixed by ! > | https://cgit.freebsd.org/src/commit/?id=3cbb4cc200f8a0ad7ed08233425ea54524a21f1c ! > | but that change was not merged to 12. I just merged it to stable/12 and ! > | will try to get both libctf changes into 12.3, as both are required for ! > | dtrace to work properly while ipfw.ko is loaded. ! > ! > That would be great, but ... I gave that patch a quick try, and ! > doesn't yet seem to work, it is now a different error message: ! > ! > > dtrace: failed to establish error handler: "/usr/lib/dtrace/ipfw.d", line 106: failed to copy type of 'inp': Invalid type identifier ! ! Hmm. Just to be clear, this is with both libctf patches applied, Ahh, My mistake, I had the other patch put away as no apparent effect at first try. So now I know it has an effect. ;) Very well, it does work now. Cheerio, PMc From nobody Wed Nov 24 06:54:57 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id E77A3188DBEA for ; Wed, 24 Nov 2021 06:55:38 +0000 (UTC) (envelope-from darius@dons.net.au) Received: from midget.dons.net.au (2403-5800-5200-4700-225-90ff-fe47-39b4.ip6.aussiebb.net [IPv6:2403:5800:5200:4700:225:90ff:fe47:39b4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "dons.net.au", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HzWx44Xcbz4tqw for ; Wed, 24 Nov 2021 06:55:36 +0000 (UTC) (envelope-from darius@dons.net.au) Received: from midget.dons.net.au (localhost [127.0.0.1]) by midget.dons.net.au (8.16.1/8.16.1) with ESMTPS id 1AO6tDQ0076837 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Wed, 24 Nov 2021 17:25:17 +1030 (ACDT) (envelope-from darius@dons.net.au) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=dons.net.au; s=default; t=1637736921; bh=M5I95HddDi/63l0EXVipKvONeqbI5+1J6GYkpFB9IXo=; h=From:Date:Subject:To; b=jCb3j3kFXlYUHCa42R/ldldMvVjiuYsSTfwS5ZpfNLbuH3+fJ9s86/pFPl9hBHIwc K+26rvEye+1FKxF2htI6Yn81T+R+KGixGbxt4mWK+HbfH86/OVR/k+1ej+6ixtzrsV 5oVh3lCGBX0u+pINEGQndBbHUlU6hG0nHl+kRaz8= Received: (from mailnull@localhost) by midget.dons.net.au (8.16.1/8.16.1/Submit) id 1AO6swIX074049 for ; Wed, 24 Nov 2021 17:24:58 +1030 (ACDT) (envelope-from darius@dons.net.au) X-MIMEDefang-Relay-f0f0b4ff001831caa5b8ac39868c4c7e9b4d12fc: 2001:44b8:1d2:8900:e4c9:2cd0:5678:7ddb Received: from smtpclient.apple ([IPv6:2001:44b8:1d2:8900:e4c9:2cd0:5678:7ddb] [2001:44b8:1d2:8900:e4c9:2cd0:5678:7ddb]) by 2403-5800-5200-4700-225-90ff-fe47-39b4.ip6.aussiebb.net (envelope-sender ) (MIMEDefang) with ESMTP id 1AO6svEF074046; Wed, 24 Nov 2021 17:24:58 +1030 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) Date: Wed, 24 Nov 2021 17:24:57 +1030 Subject: Poor USB performance on ASUS 520 motherboard (no IRQ?) Message-Id: <1E968920-8819-4C2B-9572-38CC1002DC89@dons.net.au> To: freebsd-stable X-Mailer: Apple Mail (2.3654.120.0.1.13) X-Spam-Score: 1.3 (*) No, score=1.3 required=5.0 tests=RDNS_NONE,SPF_HELO_NONE, T_SPF_PERMERROR autolearn=no autolearn_force=no version=3.4.4 X-Scanned-By: MIMEDefang 2.83 on 10.0.2.1 X-Rspamd-Queue-Id: 4HzWx44Xcbz4tqw X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=dons.net.au header.s=default header.b=jCb3j3kF; dmarc=pass (policy=quarantine) header.from=dons.net.au; spf=pass (mx1.freebsd.org: domain of darius@dons.net.au designates 2403:5800:5200:4700:225:90ff:fe47:39b4 as permitted sender) smtp.mailfrom=darius@dons.net.au X-Spamd-Result: default: False [-3.44 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[dons.net.au:s=default]; FROM_HAS_DN(0.00)[]; MV_CASE(0.50)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[dons.net.au:+]; DMARC_POLICY_ALLOW(-0.50)[dons.net.au,quarantine]; NEURAL_HAM_SHORT(-0.94)[-0.935]; R_SPF_ALLOW(-0.20)[+mx]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:4764, ipnet:2403:5800:5000::/36, country:AU]; MID_RHS_MATCH_FROM(0.00)[]; SUBJECT_HAS_QUESTION(0.00)[] Reply-To: darius@dons.net.au From: Daniel O'Connor via freebsd-stable X-Original-From: Daniel O'Connor X-ThisMailContainsUnwantedMimeParts: N Hi, We bought a few low end motherboards to drive our hardware which is = controlled by a custom USB (Cypress FX2 interface) board. I found that the Gigabyte GA-A320M-H one works fine but the ASUS = A520M-A/CSM one is significantly slower for tests where there are many = back and forth messages (streaming data seems fine). I've tried updating the BIOS and tested FreeBSD 12 (same as the = Gigabyte) and FreeBSD 13 with no change. One thing I did notice is this dmesg output: xhci0: mem 0xfcfa0000-0xfcfa7fff at = device 0.0 on pci1 xhci0: 32 bytes context size, 64-bit DMAxhci1: mem 0xfcb00000-0xfcbfffff at = device 0.3 on pci7 xhci1: 64 bytes context size, 64-bit DMA .. xhci2: mem 0xfca00000-0xfcafffff at = device 0.4 on pci7 xhci2: 64 bytes context size, 64-bit DMA vs the working one: xhci0: mem 0xfcea0000-0xfcea7fff irq = 28 at device 0.0 on pci1 xhci0: 32 bytes context size, 64-bit DMA ... xhci1: mem 0xfca00000-0xfcafffff irq = 55 at device 0.3 on pci7 xhci1: 64 bytes context size, 64-bit DMA xhci1: Unable to map MSI-X table ... xhci2: mem 0xfc900000-0xfc9fffff irq = 52 at device 0.4 on pci7 xhci2: 64 bytes context size, 64-bit DMA xhci2: Unable to map MSI-X table I think the MSI-X warning is a difference between 12 & 13 but the lack = of IRQ appears to be common to the brokenness. I tried setting these with no change: hw.pci.honor_msi_blacklist=3D"0" hw.pci.msix_rewrite_table=3D"1" And also without success: hw.pci.enable_msix=3D"0" hw.pci.enable_msi=3D"0" pciconf -lcvb shows: xhci0@pci0:1:0:0: class=3D0x0c0330 card=3D0x11421b21 = chip=3D0x43ec1022 rev=3D0x00 hdr=3D0x00 vendor =3D 'Advanced Micro Devices, Inc. [AMD]' class =3D serial bus subclass =3D USB bar [10] =3D type Memory, range 64, base 0xfcfa0000, size 32768, = enabled cap 05[50] =3D MSI supports 8 messages, 64 bit cap 11[68] =3D MSI-X supports 8 messages Table in map 0x10[0x2000], PBA in map 0x10[0x2080] cap 01[78] =3D powerspec 3 supports D0 D3 current D0 cap 10[80] =3D PCI-Express 2 legacy endpoint max data 128(512) RO NS link x4(x4) speed 8.0(8.0) ASPM disabled(L0s/L1) ecap 0001[100] =3D AER 1 0 fatal 0 non-fatal 1 corrected ecap 0019[200] =3D PCIe Sec 1 lane errors 0 ecap 0018[300] =3D LTR 1 xhci1@pci0:7:0:3: class=3D0x0c0330 card=3D0x876b1043 = chip=3D0x15e01022 rev=3D0x00 hdr=3D0x00 vendor =3D 'Advanced Micro Devices, Inc. [AMD]' device =3D 'Raven USB 3.1' class =3D serial bus subclass =3D USB bar [10] =3D type Memory, range 64, base 0xfcb00000, size 1048576, = enabled cap 09[48] =3D vendor (length 8) cap 01[50] =3D powerspec 3 supports D0 D3 current D0 cap 10[64] =3D PCI-Express 2 endpoint max data 256(256) RO NS link x16(x16) speed 8.0(8.0) ASPM disabled(L0s/L1) cap 05[a0] =3D MSI supports 8 messages, 64 bit cap 11[c0] =3D MSI-X supports 8 messages Table in map 0x10[0xfe000], PBA in map 0x10[0xff000] ecap 000b[100] =3D Vendor 1 ID 1 xhci2@pci0:7:0:4: class=3D0x0c0330 card=3D0x876b1043 = chip=3D0x15e11022 rev=3D0x00 hdr=3D0x00 vendor =3D 'Advanced Micro Devices, Inc. [AMD]' device =3D 'Raven USB 3.1' class =3D serial bus subclass =3D USB bar [10] =3D type Memory, range 64, base 0xfca00000, size 1048576, = enabled cap 09[48] =3D vendor (length 8) cap 01[50] =3D powerspec 3 supports D0 D3 current D0 cap 10[64] =3D PCI-Express 2 endpoint max data 256(256) RO NS link x16(x16) speed 8.0(8.0) ASPM disabled(L0s/L1) cap 05[a0] =3D MSI supports 8 messages, 64 bit cap 11[c0] =3D MSI-X supports 8 messages Table in map 0x10[0xfe000], PBA in map 0x10[0xff000] ecap 000b[100] =3D Vendor 1 ID 1 I'm happy to try a newer kernel or patches if anyone has suggestions :) Thanks. -- Daniel O'Connor "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum From nobody Thu Nov 25 03:49:18 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 55CB2189BDDE for ; Thu, 25 Nov 2021 03:50:01 +0000 (UTC) (envelope-from darius@dons.net.au) Received: from midget.dons.net.au (2403-5800-5200-4700-225-90ff-fe47-39b4.ip6.aussiebb.net [IPv6:2403:5800:5200:4700:225:90ff:fe47:39b4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "dons.net.au", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4J03mR290Vz4b19 for ; Thu, 25 Nov 2021 03:49:57 +0000 (UTC) (envelope-from darius@dons.net.au) Received: from midget.dons.net.au (localhost [127.0.0.1]) by midget.dons.net.au (8.16.1/8.16.1) with ESMTPS id 1AP3nklO081122 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Thu, 25 Nov 2021 14:19:46 +1030 (ACDT) (envelope-from darius@dons.net.au) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=dons.net.au; s=default; t=1637812190; bh=keKhBoIN9wJX1vZtGgFBVSR4t+EhINij8w/Wy2equJU=; h=Subject:From:In-Reply-To:Date:References:To; b=Z5d8dQ/mwW0MiHYt88cWU+bo1+yv7cMx6WGqgPZ+XKQjHWeoi5d9duAoXORYAwlS/ /OSTkNZzFCnqLi208bdGGnMQzXAwfCfBgAT3JerbeGJE4jsNhYK4+sqcY5yszHXAj6 m2J2O2HMY7a6vTcIc0UoV8NPr+e4FTsx5wubLgkw= Received: (from mailnull@localhost) by midget.dons.net.au (8.16.1/8.16.1/Submit) id 1AP3nJqv081115 for ; Thu, 25 Nov 2021 14:19:19 +1030 (ACDT) (envelope-from darius@dons.net.au) X-MIMEDefang-Relay-f0f0b4ff001831caa5b8ac39868c4c7e9b4d12fc: 2001:44b8:1d2:8900:2c89:af97:2781:e690 Received: from smtpclient.apple ([IPv6:2001:44b8:1d2:8900:2c89:af97:2781:e690] [2001:44b8:1d2:8900:2c89:af97:2781:e690]) by 2403-5800-5200-4700-225-90ff-fe47-39b4.ip6.aussiebb.net (envelope-sender ) (MIMEDefang) with ESMTP id 1AP3nJKa081112; Thu, 25 Nov 2021 14:19:19 +1030 Content-Type: text/plain; charset=us-ascii List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) Subject: Re: Poor USB performance on ASUS 520 motherboard (no IRQ?) In-Reply-To: <1E968920-8819-4C2B-9572-38CC1002DC89@dons.net.au> Date: Thu, 25 Nov 2021 14:19:18 +1030 Content-Transfer-Encoding: quoted-printable Message-Id: <1279E19E-50E4-41E6-8090-466A0AC8CFD3@dons.net.au> References: <1E968920-8819-4C2B-9572-38CC1002DC89@dons.net.au> To: freebsd-stable X-Mailer: Apple Mail (2.3654.120.0.1.13) X-Spam-Score: 1.3 (*) No, score=1.3 required=5.0 tests=RDNS_NONE,SPF_HELO_NONE, T_SPF_PERMERROR autolearn=no autolearn_force=no version=3.4.4 X-Scanned-By: MIMEDefang 2.83 on 10.0.2.1 X-Rspamd-Queue-Id: 4J03mR290Vz4b19 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=dons.net.au header.s=default header.b="Z5d8dQ/m"; dmarc=pass (policy=quarantine) header.from=dons.net.au; spf=pass (mx1.freebsd.org: domain of darius@dons.net.au designates 2403:5800:5200:4700:225:90ff:fe47:39b4 as permitted sender) smtp.mailfrom=darius@dons.net.au X-Spamd-Result: default: False [-3.40 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[dons.net.au:s=default]; FROM_HAS_DN(0.00)[]; MV_CASE(0.50)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[dons.net.au:+]; DMARC_POLICY_ALLOW(-0.50)[dons.net.au,quarantine]; NEURAL_HAM_SHORT(-0.90)[-0.897]; R_SPF_ALLOW(-0.20)[+mx]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:4764, ipnet:2403:5800:5000::/36, country:AU]; MID_RHS_MATCH_FROM(0.00)[]; SUBJECT_HAS_QUESTION(0.00)[] Reply-To: darius@dons.net.au From: Daniel O'Connor via freebsd-stable X-Original-From: Daniel O'Connor X-ThisMailContainsUnwantedMimeParts: N > On 24 Nov 2021, at 17:24, Daniel O'Connor wrote: > I've tried updating the BIOS and tested FreeBSD 12 (same as the = Gigabyte) and FreeBSD 13 with no change. >=20 > One thing I did notice is this dmesg output: > xhci0: mem 0xfcfa0000-0xfcfa7fff = at device 0.0 on pci1 > xhci0: 32 bytes context size, 64-bit DMAxhci1: ... > xhci1: mem 0xfcb00000-0xfcbfffff = at device 0.3 on pci7 > xhci1: 64 bytes context size, 64-bit DMA > .. > xhci2: mem 0xfca00000-0xfcafffff = at device 0.4 on pci7 > xhci2: 64 bytes context size, 64-bit DMA It seems this is a bit of a red herring as vmstat -i does show it = getting IRQs, not sure why they don't show up in dmesg though. Some = discussion on IRC suggests it is because the system has no legacy PCI = IRQs. However on the system in question the IRQ rate tops out at 1kHz and on = the other it is 8kHz which I think explains my problem. I'm going to have a look for BIOS settings today but getting a bit = desperate for ideas.. -- Daniel O'Connor "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum From nobody Thu Nov 25 05:50:30 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 2216318B3182 for ; Thu, 25 Nov 2021 05:51:08 +0000 (UTC) (envelope-from darius@dons.net.au) Received: from midget.dons.net.au (2403-5800-5200-4700-225-90ff-fe47-39b4.ip6.aussiebb.net [IPv6:2403:5800:5200:4700:225:90ff:fe47:39b4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "dons.net.au", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4J06S95zLgz3mKP for ; Thu, 25 Nov 2021 05:51:05 +0000 (UTC) (envelope-from darius@dons.net.au) Received: from midget.dons.net.au (localhost [127.0.0.1]) by midget.dons.net.au (8.16.1/8.16.1) with ESMTPS id 1AP5ok3X070423 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Thu, 25 Nov 2021 16:20:50 +1030 (ACDT) (envelope-from darius@dons.net.au) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=dons.net.au; s=default; t=1637819453; bh=O0N72XlkWPvg3s7HO38FzU55Rzv4u+FnxAbxxrwU+WI=; h=Subject:From:In-Reply-To:Date:References:To; b=LQRg0GZaNzeaUGGhtmJt+jdmBFHot8HGxONQQexuhXIgucocu58hKgzefErG2giSP PlGZuadL5BVAfra+jbtyp2FxQT4hOX6WheJ8jPYWX84jylQBrHiN6+d2URHalNSi2W XcaX8u+MzfpxOFQ2Or1JFN58xvrTMQsALnIrZiTI= Received: (from mailnull@localhost) by midget.dons.net.au (8.16.1/8.16.1/Submit) id 1AP5oWPC070419 for ; Thu, 25 Nov 2021 16:20:32 +1030 (ACDT) (envelope-from darius@dons.net.au) X-MIMEDefang-Relay-f0f0b4ff001831caa5b8ac39868c4c7e9b4d12fc: 2001:44b8:1d2:8900:2c89:af97:2781:e690 Received: from smtpclient.apple ([IPv6:2001:44b8:1d2:8900:2c89:af97:2781:e690] [2001:44b8:1d2:8900:2c89:af97:2781:e690]) by 2403-5800-5200-4700-225-90ff-fe47-39b4.ip6.aussiebb.net (envelope-sender ) (MIMEDefang) with ESMTP id 1AP5oVc4070415; Thu, 25 Nov 2021 16:20:32 +1030 Content-Type: text/plain; charset=us-ascii List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) Subject: Re: Poor USB performance on ASUS 520 motherboard (no IRQ?) In-Reply-To: <1279E19E-50E4-41E6-8090-466A0AC8CFD3@dons.net.au> Date: Thu, 25 Nov 2021 16:20:30 +1030 Content-Transfer-Encoding: quoted-printable Message-Id: <2DB18886-CC19-4061-9D70-E56FEBCB1F1F@dons.net.au> References: <1E968920-8819-4C2B-9572-38CC1002DC89@dons.net.au> <1279E19E-50E4-41E6-8090-466A0AC8CFD3@dons.net.au> To: freebsd-stable X-Mailer: Apple Mail (2.3654.120.0.1.13) X-Spam-Score: 1.3 (*) No, score=1.3 required=5.0 tests=RDNS_NONE,SPF_HELO_NONE, T_SPF_PERMERROR autolearn=no autolearn_force=no version=3.4.4 X-Scanned-By: MIMEDefang 2.83 on 10.0.2.1 X-Rspamd-Queue-Id: 4J06S95zLgz3mKP X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=dons.net.au header.s=default header.b=LQRg0GZa; dmarc=pass (policy=quarantine) header.from=dons.net.au; spf=pass (mx1.freebsd.org: domain of darius@dons.net.au designates 2403:5800:5200:4700:225:90ff:fe47:39b4 as permitted sender) smtp.mailfrom=darius@dons.net.au X-Spamd-Result: default: False [-3.50 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[dons.net.au:s=default]; FROM_HAS_DN(0.00)[]; MV_CASE(0.50)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[dons.net.au:+]; DMARC_POLICY_ALLOW(-0.50)[dons.net.au,quarantine]; NEURAL_HAM_SHORT(-1.00)[-1.000]; R_SPF_ALLOW(-0.20)[+mx]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:4764, ipnet:2403:5800:5000::/36, country:AU]; MID_RHS_MATCH_FROM(0.00)[]; SUBJECT_HAS_QUESTION(0.00)[] Reply-To: darius@dons.net.au From: Daniel O'Connor via freebsd-stable X-Original-From: Daniel O'Connor X-ThisMailContainsUnwantedMimeParts: N > On 25 Nov 2021, at 14:19, Daniel O'Connor wrote: >> One thing I did notice is this dmesg output: >> xhci0: mem 0xfcfa0000-0xfcfa7fff = at device 0.0 on pci1 >> xhci0: 32 bytes context size, 64-bit DMAxhci1: > ... >> xhci1: mem 0xfcb00000-0xfcbfffff = at device 0.3 on pci7 >> xhci1: 64 bytes context size, 64-bit DMA >> .. >> xhci2: mem 0xfca00000-0xfcafffff = at device 0.4 on pci7 >> xhci2: 64 bytes context size, 64-bit DMA >=20 > It seems this is a bit of a red herring as vmstat -i does show it = getting IRQs, not sure why they don't show up in dmesg though. Some = discussion on IRC suggests it is because the system has no legacy PCI = IRQs. >=20 > However on the system in question the IRQ rate tops out at 1kHz and on = the other it is 8kHz which I think explains my problem. >=20 > I'm going to have a look for BIOS settings today but getting a bit = desperate for ideas.. I had a PCIe USB card lying around: xhci1@pci0:3:0:0: class=3D0x0c0330 rev=3D0x01 hdr=3D0x00 = vendor=3D0x1106 device=3D0x3483 subvendor=3D0x1106 subdevice=3D0x3483 vendor =3D 'VIA Technologies, Inc.' device =3D 'VL805/806 xHCI USB 3.0 Controller' class =3D serial bus subclass =3D USB bar [10] =3D type Memory, range 64, base 0xfce00000, size 4096, = enabled And connecting my device to that fixes the speed problem (and shows = >8000 IRQ/sec). The difference between the A320 and A520 motherboards seem strange, = since I would expect their USB controllers to be identical. -- Daniel O'Connor "The nice thing about standards is that there are so many of them to choose from." -- Andrew Tanenbaum From eugen@grosbein.net Sun Nov 28 05:53:54 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id AF27618BA95A; Sun, 28 Nov 2021 05:54:08 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4J1yNH2Hdqz58m1; Sun, 28 Nov 2021 05:54:07 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221] (may be forged)) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id 1AS5s0Z9063807 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 28 Nov 2021 05:54:01 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: cross+freebsd@distal.com Received: from [10.58.0.10] (dadvw [10.58.0.10]) by eg.sd.rdtc.ru (8.16.1/8.16.1) with ESMTPS id 1AS5rxqk086545 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Sun, 28 Nov 2021 12:53:59 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: swap_pager: cannot allocate bio To: Chris Ross , freebsd-fs References: <9FE99EEF-37C5-43D1-AC9D-17F3EDA19606@distal.com> <09989390-FED9-45A6-A866-4605D3766DFE@distal.com> Cc: freebsd-stable@freebsd.org From: Eugene Grosbein Message-ID: <25012f1b-fb02-1e36-3414-e2696d080999@grosbein.net> Date: Sun, 28 Nov 2021 12:53:54 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 In-Reply-To: <09989390-FED9-45A6-A866-4605D3766DFE@distal.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,SHORTCIRCUIT autolearn=disabled version=3.4.2 X-Spam-Report: * -0.0 SHORTCIRCUIT No description available. * -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 4J1yNH2Hdqz58m1 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=fail (mx1.freebsd.org: domain of eugen@grosbein.net does not designate 2a01:4f8:c2c:26d8::2 as permitted sender) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-0.21 / 15.00]; ARC_NA(0.00)[]; R_SPF_FAIL(1.00)[-all]; FREEFALL_USER(0.00)[eugen]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-0.38)[-0.379]; TAGGED_RCPT(0.00)[freebsd]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; NEURAL_SPAM_SHORT(0.27)[0.272]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/32, country:DE]; RCVD_TLS_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Spam: Yes X-ThisMailContainsUnwantedMimeParts: N 11.11.2021 11:43, Chris Ross wrote: >> On Nov 10, 2021, at 23:35, Chris Ross wrote: >> >> Hey all. I have a system that I’m trying to do some intensive CPU and I/O on. FreeBSD 13.0-RELEASE, amd64, 128GB RAM, hardware RAID1 OS volume, and a large (40TB) zpool where most of the I/O is happening. >> >> Initially, it was failing for me because it was running out of swap space. It had only the normal small (4-8G) swap partition, so I resized the filesystems on the root disk and now have 400+GB swap. The system had frozen up and I wasn’t able to log in. When I go to the console, I find a long list of: >> >> swap_pager: cannot allocate bio >> >> lines. I was able to log into the console as root and pstat -s shows the swap minimally used (7.5GB used). Attempting a “zpool status” at that point locked up. I don’t know if the problem is the memory subsystem, or zfs. >> >> But, based on the error, is there perhaps some kernel parameter I can tune that might prevent the swap pager from encountering that error? > > Moving to freebsd-fs. More information makes it looks more like a ZFS problem than anything else. > > I am able to log into another root virtual console, and I can run ps (shows many things, including dozens of "cron: running job (cron)” jobs, in D state), and I’m able to wander around the root disk (3T ufs filesystem) without trouble. But, as mentioned above the “zpool status” is hung, and I suspect if I tried to access anything in that filesystem it would hang to. Those cron jobs, which aren’t anything I added, I assume are just system “check around the system” cron jobs that are getting stuck there. > > So, if anyone has any suggestions. I can leave this system stuck like this for a little while, but I’ll probably want to bring it back before the end of the day tomorrow. (I’m US EST, so it’s almost midnight here. I’ll check in on email for suggestions or ideas in the morning.) > > Thanks all. If you have debugging symbols for your kernel and DDB in the kernel, you could drop to DDB with Ctrla-Alt-ESC and try to "call doadump" in the debugger to produce crashdump before reboot for post-mortem analysis. Are you sure you did not enable file-backed swap? From nobody Wed Dec 1 18:15:14 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 140DF18C245F for ; Wed, 1 Dec 2021 18:15:33 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-oi1-f182.google.com (mail-oi1-f182.google.com [209.85.167.182]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4J46hM6XX8z3GbH for ; Wed, 1 Dec 2021 18:15:31 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-oi1-f182.google.com with SMTP id s139so50218179oie.13 for ; Wed, 01 Dec 2021 10:15:31 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=6yfDtFy8wspL7fZXjsOujF/4/m/SEIXbVts4OdTo+wk=; b=jxJBFOX44yhDPQTxVz8GCJaiOyNwOxrK9yVFZwuq4RXydGL1GdNdCDhixkiWwzOXAq iyYGAu4bqGlc8H9Odm+BO3682+eLJ5LYhjSwELluccTCwzJhTlk5k60vg0yL9AUj2gLa J4nQ/Ub15DKWnHyCeqm9EHC5+jMYKARMc+9Uf1kYnzn+VDFNU0hYZlXnEX2WQPyrx1oW jXs+QoWrAk7gvINxRH9sJFQj/RA2rqX19i9Q4MEmGezKMvyMOYPTrqayTdhr/iCldegU jq40/ChZIJnFwC4jkJfr6ZuVlhSPK2ZcVvRw23pWH0L3nfAydGBnyQWol/JrZ7wnO918 1tog== X-Gm-Message-State: AOAM530E5NihLDi8XWoj2CgI3t/O6Ump36xkC+F4dDV6O7fDGUc5qHtl 070Cx5sJNY0de2SRjYIWdeONHP1J58E6GHP4M57xPtRT X-Google-Smtp-Source: ABdhPJznvwVNnKYsn2GxbMXS4EHEV2fYTpPE4OzKpASlGw4xtcEqGgsmBqsOQ4D8mVIJYzK4Tc+JKck0MONfTVeV+8E= X-Received: by 2002:a05:6808:ec3:: with SMTP id q3mr7692800oiv.57.1638382525410; Wed, 01 Dec 2021 10:15:25 -0800 (PST) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 From: Alan Somers Date: Wed, 1 Dec 2021 11:15:14 -0700 Message-ID: Subject: ZFS deadlocks triggered by HDD timeouts To: FreeBSD Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4J46hM6XX8z3GbH X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.167.182 as permitted sender) smtp.mailfrom=asomers@gmail.com X-Spamd-Result: default: False [0.80 / 15.00]; ARC_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FREEFALL_USER(0.00)[asomers]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; NEURAL_SPAM_MEDIUM(0.86)[0.860]; RCPT_COUNT_ONE(0.00)[1]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-0.95)[-0.951]; RCVD_TLS_ALL(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[209.85.167.182:from]; NEURAL_SPAM_LONG(0.89)[0.890]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.167.182:from]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; FREEMAIL_ENVFROM(0.00)[gmail.com]; TO_DOM_EQ_FROM_DOM(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On a stable/13 build from 16-Sep-2021 I see frequent ZFS deadlocks triggered by HDD timeouts. The timeouts are probably caused by genuine hardware faults, but they didn't lead to deadlocks in 12.2-RELEASE or 13.0-RELEASE. Unfortunately I don't have much additional information. ZFS's stack traces aren't very informative, and dmesg doesn't show anything besides the usual information about the disk timeout. I don't see anything obviously related in the commit history for that time range, either. Has anybody else observed this phenomenon? Or does anybody have a good way to deliberately inject timeouts? CAM makes it easy enough to inject an error, but not a timeout. If it did, then I could bisect the problem. As it is I can only reproduce it on production servers. -Alan From nobody Wed Dec 1 18:24:57 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id BD25F18C5D42 for ; Wed, 1 Dec 2021 18:25:10 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ua1-x929.google.com (mail-ua1-x929.google.com [IPv6:2607:f8b0:4864:20::929]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4J46vV4p09z3KXL for ; Wed, 1 Dec 2021 18:25:10 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-ua1-x929.google.com with SMTP id p37so50904777uae.8 for ; Wed, 01 Dec 2021 10:25:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20210112.gappssmtp.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=NUjV1tmSsC8FxFsNNxlUqvAkpdp68+36svNmMW3doRM=; b=V3QCE/BWWB+xHYYl6TjhmP7SGLFvMR+LAxvSjnkw35izD3poiyj8K6aHOueZxzVb+o wZOjk0d4j84Rfnoqlbr4Tdt+1xUCERM0NoZDfnaO7C+0QiaCPZofNQYRl4KTR1JhzhaG q4LLOf63IKKEXVemE5HSsymV7jdsGLNyZW0BFUYCxgvOMuFyNVGZIu8yH5LRCS2QMJlN 378aIE9iDg4CWaZTnW7Agb6Ak552VjwR8WVFEVoZoMpRV6ciz1IM5W1ExdbJx9YfDzqE LgVE6aYtvEX01KNGB+h8Og6C+36BYC32DmKE1j04L0RkOrbrt4HwWxrSlWiiEFBcbWHH tTDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=NUjV1tmSsC8FxFsNNxlUqvAkpdp68+36svNmMW3doRM=; b=r1DqYYdgQUSqZzgQPSc1aTiNDOaRbBkfeVEqUkFLdkBk4FdtMdxR7ZhgYaTRqzaZRl 9r9RZAfc+Zb9ADU4g8q2OAhiDsWMzzhzcQCJ53mwL9kQVXS5/vD26GprIDG9x7wZjv+I yI8wiNCsNViF1K9iy4XHvqkwitF2wAekPRzg22nymAtY+lWqSWBGG+81XwtAwVuchh/i v3IFqnZEBahRXStnRzGzB/SUuaGtJ7MO8fmceb8/h/asW/9yc+htT0kPlllEsxvam1v9 LzOBYf38eL1BEDOWdB7Csv55BW72DLhtgMZqO9OuaVdDCNotKoWyB+KiHaptIXXVts/B IECA== X-Gm-Message-State: AOAM532l/skMhNU27nBiDgXCVgsRo94ptEM4b//dAKkn55FgCNj1ADLs id2aJCos3tkB/TRLtBmxF19c7V7faR8MjwLD3gKC3tJBhf2M9A== X-Google-Smtp-Source: ABdhPJxGaz9VEgflI887HoqFNw1ZxKn21ktHOTf5tK7YNkvF0yMkoaGjAhOvIfCyRGkemI+y3C9X3gKEtxowjDKmDxs= X-Received: by 2002:ab0:6f47:: with SMTP id r7mr9624222uat.85.1638383110088; Wed, 01 Dec 2021 10:25:10 -0800 (PST) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Wed, 1 Dec 2021 11:24:57 -0700 Message-ID: Subject: Re: ZFS deadlocks triggered by HDD timeouts To: Alan Somers Cc: FreeBSD Content-Type: multipart/alternative; boundary="0000000000004878e205d219cb42" X-Rspamd-Queue-Id: 4J46vV4p09z3KXL X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: Y --0000000000004878e205d219cb42 Content-Type: text/plain; charset="UTF-8" On Wed, Dec 1, 2021, 11:16 AM Alan Somers wrote: > On a stable/13 build from 16-Sep-2021 I see frequent ZFS deadlocks > triggered by HDD timeouts. The timeouts are probably caused by > genuine hardware faults, but they didn't lead to deadlocks in > 12.2-RELEASE or 13.0-RELEASE. Unfortunately I don't have much > additional information. ZFS's stack traces aren't very informative, > and dmesg doesn't show anything besides the usual information about > the disk timeout. I don't see anything obviously related in the > commit history for that time range, either. > > Has anybody else observed this phenomenon? Or does anybody have a > good way to deliberately inject timeouts? CAM makes it easy enough to > inject an error, but not a timeout. If it did, then I could bisect > the problem. As it is I can only reproduce it on production servers. > What SIM? Timeouts are tricky because they have many sources, some of which are nonlocal... Warner > --0000000000004878e205d219cb42-- From nobody Wed Dec 1 20:28:14 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 1347918B5EEC for ; Wed, 1 Dec 2021 20:28:26 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ot1-f46.google.com (mail-ot1-f46.google.com [209.85.210.46]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4J49dj6sbqz4gGF for ; Wed, 1 Dec 2021 20:28:25 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-ot1-f46.google.com with SMTP id n17-20020a9d64d1000000b00579cf677301so36924624otl.8 for ; Wed, 01 Dec 2021 12:28:25 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=wtn9dWYfZpUH4jUGqoFBEs5io46YnT5zRacr2qJ49gc=; b=V0/DSiiZsXDuiOw25ovA3P8LXJZYr87NjWpNHLWliDikw4/UFm8OuXZtHF2TsY5QmS Drcm2yj9MYvxVNTr4roGtdoBi/T2PeNeyIgfHT1NPzWG57mU9jBJWQQ/akGzIb94pkA1 eAL5yLWqU8o/tLEvcuU5daLxeumQIsJKrwj1NlrK8bHDj4EHBITRpNIqyXTD89xJ0Tyn 9HHvZ4WAJ2T0mZxp9Nkn6bsEDpN1EDCKIiHG5I1m92FeQEXVj8q42s45UlKa5j0XtE4L qyvZSABZY1rjtOukbpSBuyM7uQ3c/T7LFi7Phc47CHEQVkGkNHVft55L5v9/9+ZgDVSm 9flA== X-Gm-Message-State: AOAM533spxx5S44IbByCC9E13VkbPgD16FpqqsZWRBL/p2lZ4PdB3mCl egmxDN7Cj2HGajE/aXYw62PWzRqsr1ByfW6DZBSGd+XI X-Google-Smtp-Source: ABdhPJy3GCBgWXMUlGGCzLZjmySclT2ELid6yYGkKVO7zmILK84J6lnj/1fMEtPrNBBVJ7UbUiXQmmAW/LfM5NFqMgQ= X-Received: by 2002:a9d:6d98:: with SMTP id x24mr7472569otp.371.1638390505339; Wed, 01 Dec 2021 12:28:25 -0800 (PST) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Wed, 1 Dec 2021 13:28:14 -0700 Message-ID: Subject: Re: ZFS deadlocks triggered by HDD timeouts To: Warner Losh Cc: FreeBSD Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4J49dj6sbqz4gGF X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On Wed, Dec 1, 2021 at 11:25 AM Warner Losh wrote: > > > > On Wed, Dec 1, 2021, 11:16 AM Alan Somers wrote: >> >> On a stable/13 build from 16-Sep-2021 I see frequent ZFS deadlocks >> triggered by HDD timeouts. The timeouts are probably caused by >> genuine hardware faults, but they didn't lead to deadlocks in >> 12.2-RELEASE or 13.0-RELEASE. Unfortunately I don't have much >> additional information. ZFS's stack traces aren't very informative, >> and dmesg doesn't show anything besides the usual information about >> the disk timeout. I don't see anything obviously related in the >> commit history for that time range, either. >> >> Has anybody else observed this phenomenon? Or does anybody have a >> good way to deliberately inject timeouts? CAM makes it easy enough to >> inject an error, but not a timeout. If it did, then I could bisect >> the problem. As it is I can only reproduce it on production servers. > > > What SIM? Timeouts are tricky because they have many sources, some of which are nonlocal... > > Warner mpr(4) From nobody Wed Dec 1 20:36:58 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 5B36018B99C2 for ; Wed, 1 Dec 2021 20:37:10 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ua1-x92b.google.com (mail-ua1-x92b.google.com [IPv6:2607:f8b0:4864:20::92b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4J49qp1v1yz4jf3 for ; Wed, 1 Dec 2021 20:37:10 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-ua1-x92b.google.com with SMTP id ay21so51664529uab.12 for ; Wed, 01 Dec 2021 12:37:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20210112.gappssmtp.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=I0QCHX9PSFT2UkbY8tsMX8C9DevFxwoZNz/X6THiHmU=; b=WLdbDoRWifel3LUfT/Pr6Vu/MbxXP6uCA7w41++Sl9idZM2CeRaYmdBAsR84XjoMJf Ijp5dPbzLhiQOBbR7XhhNxpmEUpF4yyFrhNXQeufetBGoTeNt6ldRDjnsT4HEakSM29J qUsbBKwkzr2RoKVKf8dzD42flGoQNqTjvBY37rqMRItBto7IVZH8ObgZfvgS2BOL7u1c 4UTLC9ECJKc0t4y6dmVxUy5bLJcMNix+ixKvLkBPt/ZWI+1FxhTZOTTPxEl4CFwzcj7i AyOmTVf4h9oT4/FxTuiDjoEBmep1ehB4IzKjTKgPDurPI3SFr1Xvy/QuDrMSkM/Wh5ox GEmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=I0QCHX9PSFT2UkbY8tsMX8C9DevFxwoZNz/X6THiHmU=; b=ZInj+k7IeO65yZDMXXo1cVzcVyANsU1L/zFP7zqROtb/DkDxsV9omtQQlVDsiK711z BacX+dqBac+Ii6HwNjMZzpGUz/zLdhk5BldFN9BpjSLUXtAZicUOyO4DOENRxWJKKDd9 2d7nDMC048kHJPt3t1talJZCYCmOodnBrjD+29oQ2kiA7ioc+jwzDy4XSKOtZokpMVxZ d6D56qQYX+xaymFIaddY+CZ6zPpnx4m7iRhytbMt2ICq1kwVJVL5BOzEkaJgL4AAIMS8 gSxYw+Cpc88lNER7ktAP9dvcOXeqaBNPgaF3A8lLcAVjq/AOeTJxuVdmNcMhmOk7BQ/6 LFsw== X-Gm-Message-State: AOAM533II++FVdSlk0AEJvLnI+uQGAqjvzhs3OM6TKtwpbl97ql+Pqzq K4tiO6l1i0x3cj4K0t2GaMojstzZ6WfijclZtl5y04pNITXjSQ== X-Google-Smtp-Source: ABdhPJwtxMz5Izopqc+2OGLCVcd+evQyqmps3TVFlKKC7j4QEfjbeuFo3Ziz/SyDtQepX2nsRgs/e4eA03Nf+tyc4BI= X-Received: by 2002:a67:f950:: with SMTP id u16mr10729394vsq.68.1638391029740; Wed, 01 Dec 2021 12:37:09 -0800 (PST) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Wed, 1 Dec 2021 13:36:58 -0700 Message-ID: Subject: Re: ZFS deadlocks triggered by HDD timeouts To: Alan Somers Cc: FreeBSD Content-Type: multipart/alternative; boundary="00000000000054c93905d21ba341" X-Rspamd-Queue-Id: 4J49qp1v1yz4jf3 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: Y --00000000000054c93905d21ba341 Content-Type: text/plain; charset="UTF-8" On Wed, Dec 1, 2021 at 1:28 PM Alan Somers wrote: > On Wed, Dec 1, 2021 at 11:25 AM Warner Losh wrote: > > > > > > > > On Wed, Dec 1, 2021, 11:16 AM Alan Somers wrote: > >> > >> On a stable/13 build from 16-Sep-2021 I see frequent ZFS deadlocks > >> triggered by HDD timeouts. The timeouts are probably caused by > >> genuine hardware faults, but they didn't lead to deadlocks in > >> 12.2-RELEASE or 13.0-RELEASE. Unfortunately I don't have much > >> additional information. ZFS's stack traces aren't very informative, > >> and dmesg doesn't show anything besides the usual information about > >> the disk timeout. I don't see anything obviously related in the > >> commit history for that time range, either. > >> > >> Has anybody else observed this phenomenon? Or does anybody have a > >> good way to deliberately inject timeouts? CAM makes it easy enough to > >> inject an error, but not a timeout. If it did, then I could bisect > >> the problem. As it is I can only reproduce it on production servers. > > > > > > What SIM? Timeouts are tricky because they have many sources, some of > which are nonlocal... > > > > Warner > > mpr(4) > Is this just a single drive that's acting up, or is the controller initialized as part of the error recovery? If a single drive, are there multiple timeouts that happen at the same time such that we timeout a request while we're waiting for the abort command we send to the firmware to be acknowledged? Would you be able to run a kgdb script to see if you're hitting a situation that I fixed in mpr that would cause I/O to never complete in this rather odd circumstance? If you can, and if it is, then there's a change I can MFC :). Warner --00000000000054c93905d21ba341-- From nobody Wed Dec 1 20:47:01 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id AD06318BF8BF for ; Wed, 1 Dec 2021 20:47:19 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-oi1-f179.google.com (mail-oi1-f179.google.com [209.85.167.179]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4J4B3W4Q81z4nSD for ; Wed, 1 Dec 2021 20:47:19 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-oi1-f179.google.com with SMTP id bf8so51169389oib.6 for ; Wed, 01 Dec 2021 12:47:19 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=zNQlH2oS2bKZlKX14dqHJz1ibYd4zSgkk2GleEU+CoQ=; b=Eng971scrHs1DsAl7VlR+xI57uVvUFGt5R7oXqTMlgZEdLoKIYJJJ9nnujjoK9VUqH G0aUs0EfLthn+tqL42h+zXpGKFNxVebJrqx+EyL3wpAdsH4e4gKpOy3jJ/ei/AOgzDos d8wr4WpwJv+Sqc3LvzJ1QurvXUs77dREr1IJvum6Wtd2y8RkT4FIt/7GfkGOMyqE/+tr vsy7AMGpY1ec+09OfsMRU0WD/4HBNy1JmhIt3HNvKSlS0hBUWar6fO/Zue4W8DLABP0U WBGHgHqtRPeW8Aew0BuJ7KMb/4NNEtL1h8v2ChIdIBeynDon2j/W/ITSQ6KMClx12wqg 7EyQ== X-Gm-Message-State: AOAM533qX7/6UKiy9sUX4LzTqBh1LEsBZSyXB3m3136tiaJmy8leDdh+ x6Q1iqlP0Rx4Hw62TZHVQMyKxX8x0eoX4a4ChXHgp5CH X-Google-Smtp-Source: ABdhPJyVkHQ1cXYsh8YylRCrL8PmcgblSRWUSXf2HOLsOyHW40uhYKo6lOGT4MTkUXT/Zr4wfPtgAFSsrsj8ng+5E2E= X-Received: by 2002:a05:6808:ec3:: with SMTP id q3mr604148oiv.57.1638391633119; Wed, 01 Dec 2021 12:47:13 -0800 (PST) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Wed, 1 Dec 2021 13:47:01 -0700 Message-ID: Subject: Re: ZFS deadlocks triggered by HDD timeouts To: Warner Losh Cc: FreeBSD Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4J4B3W4Q81z4nSD X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On Wed, Dec 1, 2021 at 1:37 PM Warner Losh wrote: > > > > On Wed, Dec 1, 2021 at 1:28 PM Alan Somers wrote: >> >> On Wed, Dec 1, 2021 at 11:25 AM Warner Losh wrote: >> > >> > >> > >> > On Wed, Dec 1, 2021, 11:16 AM Alan Somers wrote: >> >> >> >> On a stable/13 build from 16-Sep-2021 I see frequent ZFS deadlocks >> >> triggered by HDD timeouts. The timeouts are probably caused by >> >> genuine hardware faults, but they didn't lead to deadlocks in >> >> 12.2-RELEASE or 13.0-RELEASE. Unfortunately I don't have much >> >> additional information. ZFS's stack traces aren't very informative, >> >> and dmesg doesn't show anything besides the usual information about >> >> the disk timeout. I don't see anything obviously related in the >> >> commit history for that time range, either. >> >> >> >> Has anybody else observed this phenomenon? Or does anybody have a >> >> good way to deliberately inject timeouts? CAM makes it easy enough to >> >> inject an error, but not a timeout. If it did, then I could bisect >> >> the problem. As it is I can only reproduce it on production servers. >> > >> > >> > What SIM? Timeouts are tricky because they have many sources, some of which are nonlocal... >> > >> > Warner >> >> mpr(4) > > > Is this just a single drive that's acting up, or is the controller initialized as part of the error recovery? I'm not doing anything fancy with mprutil or sas3flash, if that's what you're asking. > If a single drive, > are there multiple timeouts that happen at the same time such that we timeout a request while we're waiting for > the abort command we send to the firmware to be acknowledged? I don't know. > Would you be able to run a kgdb script to see > if you're hitting a situation that I fixed in mpr that would cause I/O to never complete in this rather odd circumstance? > If you can, and if it is, then there's a change I can MFC :). Possibly. When would I run this kgdb script? Before ZFS locks up, after, or while the problematic timeout happens? > > Warner From nobody Wed Dec 1 20:56:33 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 53EA318C34D8 for ; Wed, 1 Dec 2021 20:56:45 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ua1-x92f.google.com (mail-ua1-x92f.google.com [IPv6:2607:f8b0:4864:20::92f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4J4BGP1hpLz4qyS for ; Wed, 1 Dec 2021 20:56:45 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-ua1-x92f.google.com with SMTP id j14so51625632uan.10 for ; Wed, 01 Dec 2021 12:56:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20210112.gappssmtp.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=JgRg2xtDcuGc6Zpd3FyuuuRndcYvANkauJJu6b59udw=; b=gi+yg5QQMTWOPYKTTBtXXPX/jTr66biRNO6lYGZTLa2E7+KDPJCd/G5Qpn6LFRreej tm9G8rtPfyVL4m06xG/hAG8JunfUrIrsA4J9Ju8RJsIWoes51YK/dklf5VIuc+umGpXv bwAwB9bVTkYz5ptScnJeKlHS8HWv2xfl/6p920VB4DKG1Ef+Xxzdh1f8yJnXoOECLDhx SJJ864hFRNom1E1dszjnltk2Ap5q2mshX7u+VDhjGegZG3qde9h9iVl7HZHHbAgcZzss rQpJBjHqHl7ZhfwlSNgno2qrmPcQiBNYDQtTti8URbgN1fJrqLzDOzTsrD4LVZ8NeK2V 0Z1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=JgRg2xtDcuGc6Zpd3FyuuuRndcYvANkauJJu6b59udw=; b=oxuShif3D3SKmfSt7LnTMf7J3m3YMXqT23oBh1b5V2Jx8gzfnd9g11kYBI+A7a0CPv DPnoc4jhTS+EsqDp4X1tHcpZ8uyXlzuPQvuoXRua3s3XmJ0VtE15ZRgLpuoLmoCZcb0g bA+kXHqFtZqmtMhFab+UqWU0p78unpWwiCtfnjQblcHOvZaj4yW1JhXkYtEhtUKr0S6k chmR5ykgxj3RJ4ZUbpUuFzH2z7sghwvIHDTntt8JqrwgujgU6HBkceMjOortSVqt49pv cv3ZaEAZWithPxZ/qHQsim4k8fIBEJRUPjY+OMFtDMSh56SmoizrABiGhV6lDt2pQzVj agAA== X-Gm-Message-State: AOAM530A4GaNnDKrXnuP9EDhmv13GZRTkMIvU5A/VK/+eZpMjuzm3WW1 oXtJMCz4TsDM6doOYLmhIRtkThNbXqoYHbc2+S4l2dTSYXuXie9T X-Google-Smtp-Source: ABdhPJxNpmkxjtqcN1jpv5z0TO8Pfb5k5B+zJTgyuU1uOzjXOASZJlR6vQemhnQj7kyEcs5s+pQYHytn0wWpc62yuSQ= X-Received: by 2002:a05:6102:3e95:: with SMTP id m21mr10762043vsv.77.1638392204557; Wed, 01 Dec 2021 12:56:44 -0800 (PST) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Wed, 1 Dec 2021 13:56:33 -0700 Message-ID: Subject: Re: ZFS deadlocks triggered by HDD timeouts To: Alan Somers Cc: FreeBSD Content-Type: multipart/alternative; boundary="0000000000005b14f905d21be9fc" X-Rspamd-Queue-Id: 4J4BGP1hpLz4qyS X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: Y --0000000000005b14f905d21be9fc Content-Type: text/plain; charset="UTF-8" On Wed, Dec 1, 2021 at 1:47 PM Alan Somers wrote: > On Wed, Dec 1, 2021 at 1:37 PM Warner Losh wrote: > > > > > > > > On Wed, Dec 1, 2021 at 1:28 PM Alan Somers wrote: > >> > >> On Wed, Dec 1, 2021 at 11:25 AM Warner Losh wrote: > >> > > >> > > >> > > >> > On Wed, Dec 1, 2021, 11:16 AM Alan Somers > wrote: > >> >> > >> >> On a stable/13 build from 16-Sep-2021 I see frequent ZFS deadlocks > >> >> triggered by HDD timeouts. The timeouts are probably caused by > >> >> genuine hardware faults, but they didn't lead to deadlocks in > >> >> 12.2-RELEASE or 13.0-RELEASE. Unfortunately I don't have much > >> >> additional information. ZFS's stack traces aren't very informative, > >> >> and dmesg doesn't show anything besides the usual information about > >> >> the disk timeout. I don't see anything obviously related in the > >> >> commit history for that time range, either. > >> >> > >> >> Has anybody else observed this phenomenon? Or does anybody have a > >> >> good way to deliberately inject timeouts? CAM makes it easy enough > to > >> >> inject an error, but not a timeout. If it did, then I could bisect > >> >> the problem. As it is I can only reproduce it on production servers. > >> > > >> > > >> > What SIM? Timeouts are tricky because they have many sources, some of > which are nonlocal... > >> > > >> > Warner > >> > >> mpr(4) > > > > > > Is this just a single drive that's acting up, or is the controller > initialized as part of the error recovery? > > I'm not doing anything fancy with mprutil or sas3flash, if that's what > you're asking. > No. I'm asking if you've enabled debugging on the recovery messages and see that we enter any kind of controller reset when the timeouts occur. > > If a single drive, > > are there multiple timeouts that happen at the same time such that we > timeout a request while we're waiting for > > the abort command we send to the firmware to be acknowledged? > > I don't know. > OK. > > Would you be able to run a kgdb script to see > > if you're hitting a situation that I fixed in mpr that would cause I/O > to never complete in this rather odd circumstance? > > If you can, and if it is, then there's a change I can MFC :). > > Possibly. When would I run this kgdb script? Before ZFS locks up, > after, or while the problematic timeout happens? > After the timeouts. I've been doing 'kgdb' followed by 'source mpr-hang.gdb' to run this. What you are looking for is anything with a qfrozen_cnt > 0.. The script is imperfect and racy with normal operations (but not in a bad way), so you may need to run it a couple of times to get consistent data. On my systems, there'd be one or two devices with a frozen count > 1 and no I/O happened on those drives and processes hung. That might not be any different than a deadlock :) Warner P.S. here's the mpr-hang.gdb script. Not sure if I can make an attachment survive the mailing lists :) define cam_path set $path=(struct cam_path *)$arg0 printf " Periph: %p\n", $path->periph printf " Bus: %p\n", $path->bus printf " Target: %p\n", $path->target printf " Device: %p\n", $path->device end define periph set $periph = (struct cam_periph *)$arg0 printf "%s%d:\n", $periph->periph_name, $periph->unit_number printf "softc: %p\n", $periph->softc printf "sim: %p\n", $periph->sim printf "flags: 0x%x\n", $periph->flags cam_path $periph->path printf "priority: sched %d immed %d\n", $periph->scheduled_priority, $periph->immediate_priority printf "allocated %d allocating %d\n", $periph->periph_allocated, $periph->periph_allocating printf "refcount: %d\n", $periph->refcount printf "qfrozen_cnt: %d\n", $periph->path->device->ccbq.queue.qfrozen_cnt end define periphunits set $count = 0 set $driver = $arg0 set $periph = $driver.units.tqh_first while ($periph != 0) if $periph->periph_allocated != 0 || $periph->periph_allocating != 0 || $periph->path->device->ccbq.queue.qfrozen_cnt != 0 periph $periph set $count = $count + 1 end set $periph = $periph->unit_links.tqe_next end if ($count == 0) printf "No problems found for periph %s\n", $driver->driver_name end end define periphs set $i = 0 while (periph_drivers[$i] != 0) set $p = periph_drivers[$i] periphunits $p set $i = $i + 1 end end periphs Warner --0000000000005b14f905d21be9fc-- From nobody Wed Dec 1 21:35:50 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 6F92918AC734 for ; Wed, 1 Dec 2021 21:36:02 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ot1-f45.google.com (mail-ot1-f45.google.com [209.85.210.45]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4J4C7k2cN3z3Jm0 for ; Wed, 1 Dec 2021 21:36:02 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-ot1-f45.google.com with SMTP id i5-20020a05683033e500b0057a369ac614so15732420otu.10 for ; Wed, 01 Dec 2021 13:36:02 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=I623DG4OO94ZGrxXmIYt4H1MiMc1Ex4G41ONZZko2r8=; b=cqWlU/AzFbM8LdSpc4rCkkoDiiRuLGktjt9DhSsN+IrZB9LrsBi2KBuVtmhF0OGGCq lj4UmTiBvxLBnT+rAILm1zRLF+qiJLbWHyddoFX/IP+teRyPlVd+hEZd4dEODwLpHlP7 CLwa5t+07PWRhGGLGVEFB9iT8vQwGvCxLYVlf3ky1qyJAqxxf1dmONKCzCI98Obwf0b3 wyZTy9Tk+OqpO30GJahoQQa0ruX8XM+9f7JYeusGB8aMN6PXOU1Bsa6XoGhaT+gsVGLG +UabUSLtB0Pbn5fRKT0T513H9vJyJRbj40nB9PtpQWG2IWxPCLT5RBowVyyCE8w071i8 aiAQ== X-Gm-Message-State: AOAM532Te50L4YcwYlS/x/MOftX9Tg7ZujCRPx/1QekylqzCWQxqxGp0 XjCH90KIT7kVH9n7NojW8OboUjxjtlquS9LinV3XmfUi X-Google-Smtp-Source: ABdhPJx474MGs+vdLzDzVkfBv1KwUT3f5wdRX0+6DiBkAnzwfErR8OXIiXrWDwaf8qiSc+WZKy5BMifgwGJiKFh8tx0= X-Received: by 2002:a9d:7cce:: with SMTP id r14mr7905695otn.114.1638394561700; Wed, 01 Dec 2021 13:36:01 -0800 (PST) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Wed, 1 Dec 2021 14:35:50 -0700 Message-ID: Subject: Re: ZFS deadlocks triggered by HDD timeouts To: Warner Losh Cc: FreeBSD Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4J4C7k2cN3z3Jm0 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On Wed, Dec 1, 2021 at 1:56 PM Warner Losh wrote: > > > > On Wed, Dec 1, 2021 at 1:47 PM Alan Somers wrote: >> >> On Wed, Dec 1, 2021 at 1:37 PM Warner Losh wrote: >> > >> > >> > >> > On Wed, Dec 1, 2021 at 1:28 PM Alan Somers wrote: >> >> >> >> On Wed, Dec 1, 2021 at 11:25 AM Warner Losh wrote: >> >> > >> >> > >> >> > >> >> > On Wed, Dec 1, 2021, 11:16 AM Alan Somers wrote: >> >> >> >> >> >> On a stable/13 build from 16-Sep-2021 I see frequent ZFS deadlocks >> >> >> triggered by HDD timeouts. The timeouts are probably caused by >> >> >> genuine hardware faults, but they didn't lead to deadlocks in >> >> >> 12.2-RELEASE or 13.0-RELEASE. Unfortunately I don't have much >> >> >> additional information. ZFS's stack traces aren't very informative, >> >> >> and dmesg doesn't show anything besides the usual information about >> >> >> the disk timeout. I don't see anything obviously related in the >> >> >> commit history for that time range, either. >> >> >> >> >> >> Has anybody else observed this phenomenon? Or does anybody have a >> >> >> good way to deliberately inject timeouts? CAM makes it easy enough to >> >> >> inject an error, but not a timeout. If it did, then I could bisect >> >> >> the problem. As it is I can only reproduce it on production servers. >> >> > >> >> > >> >> > What SIM? Timeouts are tricky because they have many sources, some of which are nonlocal... >> >> > >> >> > Warner >> >> >> >> mpr(4) >> > >> > >> > Is this just a single drive that's acting up, or is the controller initialized as part of the error recovery? >> >> I'm not doing anything fancy with mprutil or sas3flash, if that's what >> you're asking. > > > No. I'm asking if you've enabled debugging on the recovery messages and see that we enter any kind of > controller reset when the timeouts occur. No. My CAM setup is the default except that I enabled CAM_IO_STATS and changed the following two sysctls: kern.cam.da.retry_count=2 kern.cam.da.default_timeout=10 > >> >> > If a single drive, >> > are there multiple timeouts that happen at the same time such that we timeout a request while we're waiting for >> > the abort command we send to the firmware to be acknowledged? >> >> I don't know. > > > OK. > >> >> > Would you be able to run a kgdb script to see >> > if you're hitting a situation that I fixed in mpr that would cause I/O to never complete in this rather odd circumstance? >> > If you can, and if it is, then there's a change I can MFC :). >> >> Possibly. When would I run this kgdb script? Before ZFS locks up, >> after, or while the problematic timeout happens? > > > After the timeouts. I've been doing 'kgdb' followed by 'source mpr-hang.gdb' to run this. > > What you are looking for is anything with a qfrozen_cnt > 0.. The script is imperfect and racy > with normal operations (but not in a bad way), so you may need to run it a couple of times > to get consistent data. On my systems, there'd be one or two devices with a frozen count > 1 > and no I/O happened on those drives and processes hung. That might not be any different than > a deadlock :) > > Warner > > P.S. here's the mpr-hang.gdb script. Not sure if I can make an attachment survive the mailing lists :) Thanks, I'll try that. If this is the problem, do you have any idea why it wouldn't happen on 12.2-RELEASE (I haven't seen it on 13.0-RELEASE, but maybe I just don't have enough runtime on that version). > > Warner From nobody Wed Dec 1 21:45:52 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id ECFA918B3D1F for ; Wed, 1 Dec 2021 21:46:04 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ua1-x936.google.com (mail-ua1-x936.google.com [IPv6:2607:f8b0:4864:20::936]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4J4CMJ2PCWz3PTd for ; Wed, 1 Dec 2021 21:46:04 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-ua1-x936.google.com with SMTP id p37so51988375uae.8 for ; Wed, 01 Dec 2021 13:46:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20210112.gappssmtp.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=VY5myeNRupeTkhKJCPm4lkwNcAzPNnpfmgtrF9Y4R3U=; b=fDffibUchlA4nabjlP7nNFYBhI+YdJQAtbS4kbDcqiOPJjGOOmlAOgX0jTlyP4qEkJ og3A5uRKf+L7l46msp2DZDQXE4ojnFj+a6e0GcGpfVKK8Mvy5q8KADX38raotfFLrH4V 4or94RfYu4OlDiKUJ0ioHCKC22CGpIeRWLDLG/sG5qZKZssbQFoKCU665oY+2T/5/UHM S86G8HJEO/FkeTjRyCa+DZDIqFsndproeDOcQrQGOEipaToiKpsgRZg9Oinh75aIufsT RdCMTMgB4c8ueY76bER7NBmYIJCdx0nONmjkHxWjkEXxrepe+HjK1jqiW4G+cDh/VApu SAGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=VY5myeNRupeTkhKJCPm4lkwNcAzPNnpfmgtrF9Y4R3U=; b=69v+r6E2MRFWZILK6HUDtm1ni99MzVpbZjVGY15q4jpzR4a8D0/DZbegabuqY/u8DJ CMKUCd1KfHyB5Gyn1fQGrg5wGJCrb/+UvuUas5Mmx2CyIHZbidKsJQBoWOzcvGKMHpkA wZHALyQg5bI0ZQpTMCpxoKisMwPkYOjbg7+QP2Es7eFCsqxNzVLUvnEqHsl43U/wjEvH mMQKofi/UHxzNj9ZBpsm1uBUA1+O7jY16Ys9PoFmS1fVc//lcaTUPQblRNiU8xcEg2VY nn1M6GF0qdeTXqtcgU4aUYa8KgurbM7ED90fJHXF/hiMV6MdNhVOgwm4fU4IF1I5QPd9 ISMw== X-Gm-Message-State: AOAM5318zfjU5w4QRkI8BLqFTGbPiP5DdircxWrPF5+AOA3uMcxIMQ9E eDtgT7RnP17OJDWQbWuYBfo2MCsS8pX4WVxFCaRNcdH9cus10tAt X-Google-Smtp-Source: ABdhPJwQiF8kiAWLNkjeSiFoYJtdf0t+6STjBExsW07sC2AplOcLnZdbdGof3T3zcGiguYYz4WEyd8cDKhrxqjkzcrA= X-Received: by 2002:a67:d508:: with SMTP id l8mr10523177vsj.42.1638395163285; Wed, 01 Dec 2021 13:46:03 -0800 (PST) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Wed, 1 Dec 2021 14:45:52 -0700 Message-ID: Subject: Re: ZFS deadlocks triggered by HDD timeouts To: Alan Somers Cc: FreeBSD Content-Type: multipart/alternative; boundary="000000000000b5ad3605d21c996f" X-Rspamd-Queue-Id: 4J4CMJ2PCWz3PTd X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: Y --000000000000b5ad3605d21c996f Content-Type: text/plain; charset="UTF-8" On Wed, Dec 1, 2021, 2:36 PM Alan Somers wrote: > On Wed, Dec 1, 2021 at 1:56 PM Warner Losh wrote: > > > > > > > > On Wed, Dec 1, 2021 at 1:47 PM Alan Somers wrote: > >> > >> On Wed, Dec 1, 2021 at 1:37 PM Warner Losh wrote: > >> > > >> > > >> > > >> > On Wed, Dec 1, 2021 at 1:28 PM Alan Somers > wrote: > >> >> > >> >> On Wed, Dec 1, 2021 at 11:25 AM Warner Losh wrote: > >> >> > > >> >> > > >> >> > > >> >> > On Wed, Dec 1, 2021, 11:16 AM Alan Somers > wrote: > >> >> >> > >> >> >> On a stable/13 build from 16-Sep-2021 I see frequent ZFS deadlocks > >> >> >> triggered by HDD timeouts. The timeouts are probably caused by > >> >> >> genuine hardware faults, but they didn't lead to deadlocks in > >> >> >> 12.2-RELEASE or 13.0-RELEASE. Unfortunately I don't have much > >> >> >> additional information. ZFS's stack traces aren't very > informative, > >> >> >> and dmesg doesn't show anything besides the usual information > about > >> >> >> the disk timeout. I don't see anything obviously related in the > >> >> >> commit history for that time range, either. > >> >> >> > >> >> >> Has anybody else observed this phenomenon? Or does anybody have a > >> >> >> good way to deliberately inject timeouts? CAM makes it easy > enough to > >> >> >> inject an error, but not a timeout. If it did, then I could > bisect > >> >> >> the problem. As it is I can only reproduce it on production > servers. > >> >> > > >> >> > > >> >> > What SIM? Timeouts are tricky because they have many sources, some > of which are nonlocal... > >> >> > > >> >> > Warner > >> >> > >> >> mpr(4) > >> > > >> > > >> > Is this just a single drive that's acting up, or is the controller > initialized as part of the error recovery? > >> > >> I'm not doing anything fancy with mprutil or sas3flash, if that's what > >> you're asking. > > > > > > No. I'm asking if you've enabled debugging on the recovery messages and > see that we enter any kind of > > controller reset when the timeouts occur. > > No. My CAM setup is the default except that I enabled CAM_IO_STATS > and changed the following two sysctls: > kern.cam.da.retry_count=2 > kern.cam.da.default_timeout=10 > > > > > >> > >> > If a single drive, > >> > are there multiple timeouts that happen at the same time such that we > timeout a request while we're waiting for > >> > the abort command we send to the firmware to be acknowledged? > >> > >> I don't know. > > > > > > OK. > > > >> > >> > Would you be able to run a kgdb script to see > >> > if you're hitting a situation that I fixed in mpr that would cause > I/O to never complete in this rather odd circumstance? > >> > If you can, and if it is, then there's a change I can MFC :). > >> > >> Possibly. When would I run this kgdb script? Before ZFS locks up, > >> after, or while the problematic timeout happens? > > > > > > After the timeouts. I've been doing 'kgdb' followed by 'source > mpr-hang.gdb' to run this. > > > > What you are looking for is anything with a qfrozen_cnt > 0.. The script > is imperfect and racy > > with normal operations (but not in a bad way), so you may need to run it > a couple of times > > to get consistent data. On my systems, there'd be one or two devices > with a frozen count > 1 > > and no I/O happened on those drives and processes hung. That might not > be any different than > > a deadlock :) > > > > Warner > > > > P.S. here's the mpr-hang.gdb script. Not sure if I can make an > attachment survive the mailing lists :) > > Thanks, I'll try that. If this is the problem, do you have any idea > why it wouldn't happen on 12.2-RELEASE (I haven't seen it on > 13.0-RELEASE, but maybe I just don't have enough runtime on that > version). > 9781c28c6d63 was merged to stable/13 as a996b55ab34c on Sept 2nd. I fixed a bug with that version in current as a8837c77efd0, but haven't merged it. I kinda expect that this might be the cause of the problem. But in Netflix's fleet we've seen this maybe a couple of times a week over many thousands of machines, so I've been a little cautious in merging it to make sure that it's really fixed. So far, the jury is out. Warner --000000000000b5ad3605d21c996f-- From nobody Wed Dec 1 22:36:03 2021 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 78B5E18ABA82 for ; Wed, 1 Dec 2021 22:36:15 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ot1-f49.google.com (mail-ot1-f49.google.com [209.85.210.49]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4J4DTC2pxTz4STB for ; Wed, 1 Dec 2021 22:36:15 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-ot1-f49.google.com with SMTP id 47-20020a9d0332000000b005798ac20d72so37324819otv.9 for ; Wed, 01 Dec 2021 14:36:15 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=mmOuDe2dgCGoHdsjifBc8V82UOsXS1RLhq8EPKB/Vi4=; b=jFbfaCV0FmDR5dM5vwH7A60MrfCA1UhMDx5NzLAYSiinYnRXjF0PPgHUBTvyvY8jbl CPYeKCWmJuIema1mKWjm/Kv8S5MPCcLT25LnAh0MCEdyQISr587qd4hCgDX2agG6hFHl NG8qwdHKNKu523SLfHOytSQxmpBfQUCtGasj2v3EbRxho8fcJ04fJByqYeZovxWN8jxz YBBvDqxl3ES6bgw3d14/rbsQ7xRC7ZNrSz2xDBL25u3e7s4GGqsaRC58MBPPkwZyd2wQ qTiT0yJbcisAlg+ZXPgpVYpzRx2wcIWTpshS0Tg/QxA4zvghlRP2t2bHUPBuaiRHp+hq 4pSw== X-Gm-Message-State: AOAM531AdgS9/v8bcigsYcAODlVAVHe1+rVTe28IayWWEmZvMnEiQ3db abu8XjfvId4SvjymOytlFUYlpM4LpvWX5N/7KZg= X-Google-Smtp-Source: ABdhPJykIPv3fRQ0pbblKiPA9d6C468Bxlk9ftypkboavZRs9Aoit8FJF1sUUysCps25XUakNpu1l23NjkTZ6+POtAM= X-Received: by 2002:a05:6830:19c8:: with SMTP id p8mr8497376otp.111.1638398174519; Wed, 01 Dec 2021 14:36:14 -0800 (PST) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Wed, 1 Dec 2021 15:36:03 -0700 Message-ID: Subject: Re: ZFS deadlocks triggered by HDD timeouts To: Warner Losh Cc: FreeBSD Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4J4DTC2pxTz4STB X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On Wed, Dec 1, 2021 at 2:46 PM Warner Losh wrote: > > > > > On Wed, Dec 1, 2021, 2:36 PM Alan Somers wrote: >> >> On Wed, Dec 1, 2021 at 1:56 PM Warner Losh wrote: >> > >> > >> > >> > On Wed, Dec 1, 2021 at 1:47 PM Alan Somers wrote: >> >> >> >> On Wed, Dec 1, 2021 at 1:37 PM Warner Losh wrote: >> >> > >> >> > >> >> > >> >> > On Wed, Dec 1, 2021 at 1:28 PM Alan Somers wrote: >> >> >> >> >> >> On Wed, Dec 1, 2021 at 11:25 AM Warner Losh wrote: >> >> >> > >> >> >> > >> >> >> > >> >> >> > On Wed, Dec 1, 2021, 11:16 AM Alan Somers wrote: >> >> >> >> >> >> >> >> On a stable/13 build from 16-Sep-2021 I see frequent ZFS deadlocks >> >> >> >> triggered by HDD timeouts. The timeouts are probably caused by >> >> >> >> genuine hardware faults, but they didn't lead to deadlocks in >> >> >> >> 12.2-RELEASE or 13.0-RELEASE. Unfortunately I don't have much >> >> >> >> additional information. ZFS's stack traces aren't very informative, >> >> >> >> and dmesg doesn't show anything besides the usual information about >> >> >> >> the disk timeout. I don't see anything obviously related in the >> >> >> >> commit history for that time range, either. >> >> >> >> >> >> >> >> Has anybody else observed this phenomenon? Or does anybody have a >> >> >> >> good way to deliberately inject timeouts? CAM makes it easy enough to >> >> >> >> inject an error, but not a timeout. If it did, then I could bisect >> >> >> >> the problem. As it is I can only reproduce it on production servers. >> >> >> > >> >> >> > >> >> >> > What SIM? Timeouts are tricky because they have many sources, some of which are nonlocal... >> >> >> > >> >> >> > Warner >> >> >> >> >> >> mpr(4) >> >> > >> >> > >> >> > Is this just a single drive that's acting up, or is the controller initialized as part of the error recovery? >> >> >> >> I'm not doing anything fancy with mprutil or sas3flash, if that's what >> >> you're asking. >> > >> > >> > No. I'm asking if you've enabled debugging on the recovery messages and see that we enter any kind of >> > controller reset when the timeouts occur. >> >> No. My CAM setup is the default except that I enabled CAM_IO_STATS >> and changed the following two sysctls: >> kern.cam.da.retry_count=2 >> kern.cam.da.default_timeout=10 >> >> >> > >> >> >> >> > If a single drive, >> >> > are there multiple timeouts that happen at the same time such that we timeout a request while we're waiting for >> >> > the abort command we send to the firmware to be acknowledged? >> >> >> >> I don't know. >> > >> > >> > OK. >> > >> >> >> >> > Would you be able to run a kgdb script to see >> >> > if you're hitting a situation that I fixed in mpr that would cause I/O to never complete in this rather odd circumstance? >> >> > If you can, and if it is, then there's a change I can MFC :). >> >> >> >> Possibly. When would I run this kgdb script? Before ZFS locks up, >> >> after, or while the problematic timeout happens? >> > >> > >> > After the timeouts. I've been doing 'kgdb' followed by 'source mpr-hang.gdb' to run this. >> > >> > What you are looking for is anything with a qfrozen_cnt > 0.. The script is imperfect and racy >> > with normal operations (but not in a bad way), so you may need to run it a couple of times >> > to get consistent data. On my systems, there'd be one or two devices with a frozen count > 1 >> > and no I/O happened on those drives and processes hung. That might not be any different than >> > a deadlock :) >> > >> > Warner >> > >> > P.S. here's the mpr-hang.gdb script. Not sure if I can make an attachment survive the mailing lists :) >> >> Thanks, I'll try that. If this is the problem, do you have any idea >> why it wouldn't happen on 12.2-RELEASE (I haven't seen it on >> 13.0-RELEASE, but maybe I just don't have enough runtime on that >> version). > > > 9781c28c6d63 was merged to stable/13 as a996b55ab34c on Sept 2nd. I fixed a bug > with that version in current as a8837c77efd0, but haven't merged it. I kinda expect that > this might be the cause of the problem. But in Netflix's fleet we've seen this maybe a > couple of times a week over many thousands of machines, so I've been a little cautious > in merging it to make sure that it's really fixed. So far, the jury is out. > > Warner Well, I'm experiencing this error much more frequently than you then. I've seen it on about 10% of similarly-configured servers and they've only been running that release for 1 week. -Alan