From nobody Sun Apr 9 00:58:53 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PvDJb604sz455QM for ; Sun, 9 Apr 2023 00:59:11 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PvDJZ4G7Hz3wlR; Sun, 9 Apr 2023 00:59:10 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Authentication-Results: mx1.freebsd.org; none Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id 3390wr4x020758; Sat, 8 Apr 2023 17:58:53 -0700 (PDT) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: (from freebsd-rwg@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id 3390wrE1020757; Sat, 8 Apr 2023 17:58:53 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <202304090058.3390wrE1020757@gndrsh.dnsmgr.net> Subject: Re: low TCP speed, wrong rtt measurement In-Reply-To: To: Richard Perini Date: Sat, 8 Apr 2023 17:58:53 -0700 (PDT) CC: freebsd-hackers@FreeBSD.org, rscheff@FreeBSD.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 4PvDJZ4G7Hz3wlR X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:13868, ipnet:69.59.192.0/19, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N > On Tue, Apr 04, 2023 at 02:46:34PM -0000, Peter 'PMc' Much wrote: > > ** maybe this should rather go the -net list, but then > > ** there are only bug messages > > > > Hi, > > I'm trying to transfer backup data via WAN; the link bandwidth is > > only ~2 Mbit, but this can well run for days and just saturate the spare > > bandwidth. > > > > The problem is, it doesn't saturate the bandwidth. > > > > I found that the backup application opens the socket in this way: > > if ((fd = socket(ipaddr->GetFamily(), SOCK_STREAM, 0)) < 0) { > > > > Apparently that doesn't work well. So I patched the application to do > > it this way: > > - if ((fd = socket(ipaddr->GetFamily(), SOCK_STREAM, 0)) < 0) { > > + if ((fd = socket(ipaddr->GetFamily(), SOCK_STREAM, IPPROTO_TCP)) < 0) { > > > > The result, observed with tcpdump, was now noticeably different, but > > rather worse than better. > > > > I tried various cc algorithms, all behaved very bad with the exception > > of cc_vegas. Vegas, after tuning the alpha and beta, gave satisfying > > results with less than 1% tradeoff. > > > > But only for a time. After transferring for a couple of hours the > > throughput went bad again: > > > > # netstat -aC > > Proto Recv-Q Send-Q Local Address Foreign Address (state) CC cwin ssthresh MSS ECN > > tcp6 0 57351 edge-jo.26996 pole-n.22 ESTABLISHED vegas 22203 10392 1311 off > > tcp4 0 106305 edge-e.62275 pole-n.bacula-sd ESTABLISHED vegas 11943 5276 1331 off > > > > The first connection is freshly created. The second one runs for a day > > already , and it is obviousely hosed - it doesn't recover. > > > > # sysctl net.inet.tcp.cc.vegas > > net.inet.tcp.cc.vegas.beta: 14 > > net.inet.tcp.cc.vegas.alpha: 8 > > > > 8 (alpha) x 1331 (mss) = 10648 > > > > The cwin is adjusted to precisely one tick above the alpha, and > > doesn't rise further. (Increasing the alpha further does solve the > > issue for this connection - but that is not how things are supposed to > > work.) > > > > Now I tried to look into the data that vegas would use for it's > > decisions, and found this: > > > > # dtrace -n 'fbt:kernel:vegas_ack_received:entry { printf("%s %u %d %d %d %d", execname,\ > > (*((struct tcpcb **)(arg0+24)))->snd_cwnd,\ > > ((struct ertt *)((*((struct tcpcb **)(arg0+24)))->osd->osd_slots[0]))->minrtt,\ > > ((struct ertt *)((*((struct tcpcb **)(arg0+24)))->osd->osd_slots[0]))->marked_snd_cwnd,\ > > ((struct ertt *)((*((struct tcpcb **)(arg0+24)))->osd->osd_slots[0]))->bytes_tx_in_marked_rtt,\ > > ((struct ertt *)((*((struct tcpcb **)(arg0+24)))->osd->osd_slots[0]))->markedpkt_rtt);\ > > }' > > CPU ID FUNCTION:NAME > > 6 17478 vegas_ack_received:entry ng_queue 11943 1 11943 10552 131 > > 17 17478 vegas_ack_received:entry ng_queue 22203 56 22203 20784 261 > > 17 17478 vegas_ack_received:entry ng_queue 22203 56 22203 20784 261 > > 3 17478 vegas_ack_received:entry ng_queue 11943 1 11943 10552 131 > > 5 17478 vegas_ack_received:entry ng_queue 22203 56 22203 20784 261 > > 17 17478 vegas_ack_received:entry ng_queue 11943 1 11943 10552 131 > > 11 17478 vegas_ack_received:entry ng_queue 11943 1 11943 10552 106 > > 15 17478 vegas_ack_received:entry ng_queue 22203 56 22203 20784 261 > > 13 17478 vegas_ack_received:entry ng_queue 22203 56 22203 20784 261 > > 16 17478 vegas_ack_received:entry ng_queue 11943 1 11943 10552 106 > > 3 17478 vegas_ack_received:entry ng_queue 22203 56 22203 20784 261 > > > > One can see that the "minrtt" value for the freshly created connection > > is 56 (which is very plausible). > > But the old and hosed connection shows minrtt = 1, which explains the > > observed cwin. > > > > The minrtt gets calculated in sys/netinet/khelp/h_ertt.c: > > e_t->rtt = tcp_ts_getticks() - txsi->tx_ts + 1; > > There is a "+1", so this was apparently zero. > > > > But source and destination are at least 1000 km apart. So either we > > have had one of the rare occasions of hyperspace tunnelling, or > > something is going wrong in the ertt measurement code. > > > > For now this is a one-time observation, but it might also explain why > > the other cc algorithms behaved badly. These algorithms are widely in > > use and should work - the ertt measurement however is the same for all of > > them. > > I can confirm I am seeing similar problems transferring files to our various > production sites around Australia. Various types/sizes of links and bandwidths. > I can saturate the nearby links, but the link utilisation/saturation decreases > with distance. > > I've tried various transfer protocols: ftp, scp, rcp, http: results are > similar for all. Ping times for the closest WAN link is 2.3ms, furthest is > 60ms. On the furthest link, we get around 15% utilisation. Transfer between > 2 Windows hosts on the furthest link yields ~80% utilisation. Windows should be using cc_cubic, you say above you had tried all the congestion algorithims, and only cc_vegas after tuning gave good results. > > FreeBSD versions involved are 12.1 and 12.2. I wonder if cc_cubic is broken in 12.X, it should give similiar results to windows if things are working correctly. I am adding Richard Scheffenegger as he is the most recent expect on the congestion control code in FreeBSD. > -- > Richard Perini > Ramico Australia Pty Ltd Sydney, Australia rpp@ci.com.au +61 2 9552 5500 > ----------------------------------------------------------------------------- > "The difference between theory and practice is that in theory there is no > difference, but in practice there is" -- Rod Grimes rgrimes@freebsd.org From nobody Sun Apr 9 09:40:29 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PvRtB2Dnnz449gc; Sun, 9 Apr 2023 09:40:34 +0000 (UTC) (envelope-from rscheff@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PvRtB1hyzz4PQs; Sun, 9 Apr 2023 09:40:34 +0000 (UTC) (envelope-from rscheff@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681033234; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type:autocrypt:autocrypt; bh=mz+IdMGqHwhlJ4wMSWAerfNac2njWCiQAVdFRjpHc9Y=; b=CeciY2ZL3vn8TxS9/c8Vgxsn8QPzt+jISOeYcJDGJ3AA50MLJGPBZytiOia8nwqmbOhY5s oywhDxoJE2FoPpHoIDuXp16owJtPcNnG+n+C88bGXZ99MAyYXOIew4enu4/7webWJ1GkZu nr241SEa+sB21i6s/3tpSV66tPAMSWur3f+c6T4u4Jqe9oaY7RxXnVjPHil80mW1O2Xnz7 OCmbl0thQRNInhtO2tXKrFvc8zvLIfL7317h1q6trCt+6UlOi0oWJkhPFuZFYAiceIJvRJ IPfRcGSlQiPcpt9Fi/3IRuxQgfsc1tK8axQI2wK3TMRzknaNJ1PtK4mMM2o5Hg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681033234; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type:autocrypt:autocrypt; bh=mz+IdMGqHwhlJ4wMSWAerfNac2njWCiQAVdFRjpHc9Y=; b=igMBi26TGyoa0hrkvMhXTJKydwmq8Bpfq29lpdBoeSDQD+PZI5u3mMB5adbA1KxOorOIdj 5DCUhFtrOwYlNIJkVSc+mJR1kI6FpzbqQGTiLBPWjllj4pB7mrQI/zerCnfvp+A7ZtQy1q vFlpEB+SdWwv84f+C5H/YEW911NIQKdM25cKqZOXPaQypsiLJ/kP3sIgN1/hLDagFV25IG FvCqDG9ud0NCloaBt0VsZPRrKqo2wfUjb2BczjeyB6XvWF6q3URmFI9SEyXF8feQHKKiVA +nB3glbmLq+ETLp+bS1Rm8qo673MhhZcKoYRw8uiDF1IbDzN5jQqdpmwvKyT7w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1681033234; a=rsa-sha256; cv=none; b=BOAu6SddzSt31m+uL4Dtu+sawwSfbRAnslw/DAYdt9fa3BSNT8D82VpU3q0h7BYbgMBmIn NpO2VtXZuaAwd9M9fu+k5WsAeWjVQE31DoSQ1Creiaa+ZPQEKFacO1ipJdsaE7jOcOKynQ wcMmvvTnY/xO7wv7dYKEhRNBAm/oGXNExgbk6cfY9oPAYK6ErdIoZKW/AM8FADy2rPGmyU JzCotnE4/hjXj4344lduHIArMoNcrFMXsPuHoOWkYU07e/b4qrsv0EWQFxEAldmuSum8sr g77okzl2UdDbYO5mws0Cx79qcJD+s5l5WqZuKOh0NXHXCbNEWzPTLZwl2xd8RQ== Received: from [192.168.233.124] (unknown [185.236.167.136]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: rscheff/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4PvRt959Jzz10Gb; Sun, 9 Apr 2023 09:40:33 +0000 (UTC) (envelope-from rscheff@freebsd.org) Message-ID: <31e31ac3-f336-bbba-59c2-ab2cda918fb9@freebsd.org> Date: Sun, 9 Apr 2023 11:40:29 +0200 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.9.1 To: freebsd-hackers@FreeBSD.org, FreeBSD Transport From: "Scheffenegger, Richard" Subject: Re: low TCP speed, wrong rtt measurement Autocrypt: addr=rscheff@freebsd.org; keydata= xjMEY/i74RYJKwYBBAHaRw8BAQdAwtnvjlFVnnzNXO9hjHtB6MPGSY19L/BHh/iziPF0FzrN K1JpY2hhcmQgU2NoZWZmZW5lZ2dlciA8cnNjaGVmZkBmcmVlYnNkLm9yZz7CmgQTFgoAQhYh BDZLt5msg0Ras820cRe+WJngsUObBQJj+LvhAhsDBQkJZgGABQsJCAcCAyICAQYVCgkICwIE FgIDAQIeBwIXgAAKCRAXvliZ4LFDm4ylAQCSw2/nvht8kExJ31M+3qpjOqdVypMp+/Ojvh5Z lsk96QEA5HCBkteJcrohwRA7llZvLH3m25hcJdzmDh39mc0cSgPOOARj+LvhEgorBgEEAZdV AQUBAQdA1Dim8ZWpXRS5i9hb3O4RNHub8XvqTTkYyiZ2lSkXDwYDAQgHwn4EGBYKACYWIQQ2 S7eZrINEWrPNtHEXvliZ4LFDmwUCY/i74QIbDAUJCWYBgAAKCRAXvliZ4LFDm2TGAQDcg+bA EPqOH+JCIND8wZ62MwnjFyXFv73qevXkUHHNSgEApUgpHW9f6UaIAQpc3R185xjz6tk8XXBx eYpxKgIAeQ8= Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="------------8AtF0NY2J9HdWBu02vsHsObm" X-ThisMailContainsUnwantedMimeParts: N This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --------------8AtF0NY2J9HdWBu02vsHsObm Content-Type: multipart/mixed; boundary="------------Bq78VnA842pdinJqyHvdZMz9"; protected-headers="v1" From: "Scheffenegger, Richard" To: freebsd-hackers@FreeBSD.org, FreeBSD Transport Message-ID: <31e31ac3-f336-bbba-59c2-ab2cda918fb9@freebsd.org> Subject: Re: low TCP speed, wrong rtt measurement --------------Bq78VnA842pdinJqyHvdZMz9 Content-Type: multipart/mixed; boundary="------------STuGrT2Dl9lxrSEoScRORSa6" --------------STuGrT2Dl9lxrSEoScRORSa6 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: base64 W3Jlc2VuZGluZyB0byB0aGUgbGlzdHNdDQoNCkhpLA0KDQpBZGRpbmcgZmJzZC10cmFuc3Bv cnQgdG9vLg0KDQpGb3Igc3RhYmxlLTEyLCBJIGJlbGlldmUgYWxsIHJlbGV2YW50IChhbGdv cml0aG0pIGltcHJvdmVtZW50cyB3ZW50IGluLg0KDQpIb3dldmVyLCAxMi4yIGlzIG1pc3Np bmcgRDI2ODA3IGFuZCBEMjY4MDggLSBpbXByb3ZlbWVudHMgaW4gQ3ViaWMgdG8gDQpyZXRy YW5zbWlzc2lvbiB0aW1lb3V0cyAoYnV0IHRoZXNlIGFyZSBub3QgbWF0ZXJpYWwpDQoNCldo aWxlIDEyLjEuIGhhcyBub25lIG9mIHRoZSBpbXByb3ZlbWVudHMgZG9uZSBpbiAyMDIwIHRv IHRoZSBDdWJpYyANCm1vZHVsZSAtIEQxODk1NCwgRDE4OTgyLCBEMTkxMTgsIEQyMzM1Mywg RDIzNjU1LCBEMjUwNjUsIEQyNTEzMywgRDI1NzQ0LCANCkQyNDY1NywgRDI1NzQ2LCBEMjU5 NzYsIEQyNjA2MCwgRDI2ODA3LCBEMjY4MDguDQoNClRoZXNlIHNob3VsZCBmaXggbnVtZXJv dXMgaXNzdWVzIGluIGN1YmljLCB3aGljaCB3b3VsZCB2ZXJ5IGxpa2VseSBtYWtlIA0KaXQg cGVyZm9ybSBwb29ybHkgcGFydGljdWxhcmx5IG9uIGxvbmdlciBkdXJhdGlvbiBzZXNzaW9u cy4NCg0KSG93ZXZlciwgQ3ViaWMgaXMgaGVhdmlseSByZWxpYW50IG9uIGEgdmFsaWQgbWVh c3VyZW1lbnQgb2YgUlRUIGFuZCB0aGUgDQplcG9jaCBzaW5jZSB0aGUgbGFzdCBjb25nZXN0 aW9uIHJlc3BvbnNlIChtZWFzdXJlZCBpbiB1bml0cyBvZiBSVFQpLiBBbiANCmlzc3VlIGlu IGdldHRpbmcgUlRUIG1lYXN1cmVkIHByb3Blcmx5IHdvdWxkIGRlcmFpbCBjdWJpYyBmb3Ig c3VyZSAobW9zdCANCmxpa2VseSBjdWJpYyB3b3VsZCBpbmZsYXRlIGN3bmQgbXVjaCBmYXN0 ZXIsIHRoZW4gcnVubmluZyBpbnRvIA0Kc2lnbmlmaWNhbnQgcGFja2V0IGxvc3MsIHZlcnkg bGlrZWx5IGxvc3Mgb2YgcmV0cmFuc21pc3Npb25zLCBmb2xsb3dlZCANCmJ5IHJldHJhbnNt aXNzaW9uIHRpbWVvdXRzLCBhbmQgc2hyaW5raW5nIG9mIHRoZSBzc3RocmVzaCB0byBzbWFs bCB2YWx1ZXMuDQoNCg0KSSBoYXZlbid0IGxvb2tlZCBpbnRvIGNjX3ZlZ2FzIG9yIHRoZSBl cnR0IG1vZHVsZSB0aG91Z2guDQoNCk9uZSBtb3JlIGluaXRpYWwgcXVlc3Rpb246IEFyZSB5 b3UgdXNpbmcgdGltZXN0YW1wcyBvbiB0aGF0IGxvbmcsIHRoaW4gDQpwaXBlIC0gb3IgaXMg bmV0LmluZXQudGNwLnJmYzEzMjMgZGlzYWJsZWQgKG1vcmUgcmVjZW50IHZlcnNpb25zIGFs bG93IA0KdGhlIHNlbGVjdGl2ZSBlbmFibGVtZW50L2Rpc2FibGluZyBvZiB3aW5kb3cgc2Nh bGluZyBhbmQgdGltZXN0YW1wcyANCmluZGVwZW50ZW5kIG9mIGVhY2ggb3RoZXIsIGJ1dCBJ IGRvbid0IHRoaW5rIHRoaXMgaXMgaW4gYW5kIDEyIHJlbGVhc2UuIA0KKEQzNjg2Myk/DQoN CkZpbmFsbHksIHlvdSBjb3VsZCBiZSB1c2luZyBTSUZUUiB0byB0cmFjayB0aGUgZXZvbHV0 aW9uIG9mIHRoZSBtaW5ydHQgDQp2YWx1ZSBvdmVyIHRoZSBjb3Vyc2Ugb2YgdGhlIHNlc3Np b24uDQoNCkFsdGhvdWdoIEkgc3VzcGVjdCB1bHRpbWF0ZWx5IGEgdGNwZHVtcCBpbmNsdWRp bmcgdGhlIHRjcCBoZWFkZXIgKC1zIDgwKSANCiwgYW5kIHRoZSBzaWZ0ZXIgaW50ZXJuYWwg c3RhdGUgZXZvbHV0aW9uIHdvdWxkIGJlIG9wdGltYWwgdG8gDQp1bmRlcnN0YW5kaW5nIHdo ZW4gYW5kIHdoeSB0aGUgUlRUIHZhbHVlcyBnbyBvZmYgdGhlIHJhaWxzLg0KDQoNCkF0IGZp cnN0IGdsYW5jZSwgdGhlIGVydHQgbW9kdWxlIG1heSBiZSBwcm9uZSB0byBtaXNjYWxjdWxh dGlvbnMsIHdoZW4gDQpyZXRyYW5zbWlzc2lvbnMgYXJlIGluIHBsYXkgLSBubyBzcGVjaWFs IHByZWNhdXRpb25zIGFwcGVhciB0byBiZSANCnByZXNlbnQsIHRvIGRpc3Rpbmd1aXNoIGJl dHdlZW4gdGhlIG9yaWdpbmFsbHkgc2VudCBwYWNrZXQsIGFuZCBhbnkgDQpyZXRyYW5zbWlz c2lvbiwgbm9yIGFueSBmaWx0ZXJpbmcgb2YgQUNLcyB3aGljaCBjb21lIGluIGFzIGR1cGxp Y2F0ZXMuIA0KVGh1cyB0aGVyZSBjb3VsZCBiZSBhIHNjZW5hcmlvLCB3aGVyZSBhbiBBQ0sg Zm9yIGEgc3B1cmlvdXMgDQpyZXRyYW5zbWlzc2lvbiwgZS5nLiBkdWUgdG8gcmVvcmRlcmlu ZywgY291bGQgbGVhZCB0byBhIHdyb25nIGJhc2VsaW5lIA0KUlRUIG1lYXN1cmVtZW50LCB3 aGljaCBpcyBwaHlzaWNhbGx5IGltcG9zc2libGUgb24gc3VjaCBhIGxvbmcgZGlzdGFuY2Ug DQpjb25uZWN0aW9uLi4uDQoNCkJ1dCBhZ2FpbiwgSSBoYXZlbid0IGxvb2tlZCBpbnRvIHRo ZSBlcnR0IG1vZHVsZSBzbyBmYXIgYXQgYWxsLg0KDQpIb3cgYXJlIHRoZSBiYXNlIHN0YWNr IFJUVCByZWxhdGVkIHZhbHVlcyBsb29rIG9uIHRoZXNlIG1pc2JlaGF2aW5nIA0Kc2Vzc2lv bnM/DQpUY3BjYi0+IHRfcnR0bWluLCB0X3NydHQsIHRfcnR0dmFyLCB0X3J4dGN1ciwgdF9y dHR0aW1lLCB0X3J0c2VxLA0KVGNwY2ItPiB0X3J0dGxvdywgdF9ydHR1cGRhdGVkDQoNCkJl c3QgcmVnYXJkcywNCiAgIFJpY2hhcmQNCg== --------------STuGrT2Dl9lxrSEoScRORSa6 Content-Type: application/pgp-keys; name="OpenPGP_0x17BE5899E0B1439B.asc" Content-Disposition: attachment; filename="OpenPGP_0x17BE5899E0B1439B.asc" Content-Description: OpenPGP public key Content-Transfer-Encoding: quoted-printable -----BEGIN PGP PUBLIC KEY BLOCK----- xjMEY/i74RYJKwYBBAHaRw8BAQdAwtnvjlFVnnzNXO9hjHtB6MPGSY19L/BHh/iz iPF0FzrNK1JpY2hhcmQgU2NoZWZmZW5lZ2dlciA8cnNjaGVmZkBmcmVlYnNkLm9y Zz7CmgQTFgoAQhYhBDZLt5msg0Ras820cRe+WJngsUObBQJj+LvhAhsDBQkJZgGA BQsJCAcCAyICAQYVCgkICwIEFgIDAQIeBwIXgAAKCRAXvliZ4LFDm4ylAQCSw2/n vht8kExJ31M+3qpjOqdVypMp+/Ojvh5Zlsk96QEA5HCBkteJcrohwRA7llZvLH3m 25hcJdzmDh39mc0cSgPOOARj+LvhEgorBgEEAZdVAQUBAQdA1Dim8ZWpXRS5i9hb 3O4RNHub8XvqTTkYyiZ2lSkXDwYDAQgHwn4EGBYKACYWIQQ2S7eZrINEWrPNtHEX vliZ4LFDmwUCY/i74QIbDAUJCWYBgAAKCRAXvliZ4LFDm2TGAQDcg+bAEPqOH+JC IND8wZ62MwnjFyXFv73qevXkUHHNSgEApUgpHW9f6UaIAQpc3R185xjz6tk8XXBx eYpxKgIAeQ8=3D =3DBwxS -----END PGP PUBLIC KEY BLOCK----- --------------STuGrT2Dl9lxrSEoScRORSa6-- --------------Bq78VnA842pdinJqyHvdZMz9-- --------------8AtF0NY2J9HdWBu02vsHsObm Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wnsEABYIACMWIQQ2S7eZrINEWrPNtHEXvliZ4LFDmwUCZDKIDQUDAAAAAAAKCRAXvliZ4LFDm8pe AP4i0CQXuKbxCyOv24wa6tzhF8HtTugQFX9M2CXw0G2QYwEAlSeMC6FvKXtHut5AMYjEPAYaPyAl HJuAOTCZ2MDIxgI= =L8Uq -----END PGP SIGNATURE----- --------------8AtF0NY2J9HdWBu02vsHsObm-- From nobody Sun Apr 9 14:58:43 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PvZxb6nKTz44Z1f; Sun, 9 Apr 2023 14:58:59 +0000 (UTC) (envelope-from ccfreebsd@gmail.com) Received: from mail-lf1-f51.google.com (mail-lf1-f51.google.com [209.85.167.51]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PvZxZ6ppqz3R1W; Sun, 9 Apr 2023 14:58:58 +0000 (UTC) (envelope-from ccfreebsd@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-lf1-f51.google.com with SMTP id r27so3758797lfe.0; Sun, 09 Apr 2023 07:58:58 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1681052336; x=1683644336; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=+b+O2Yom3wqEl+9wsw4DjgDCVc0AlQOWJqCq59vBQdo=; b=M4hy9fpxTpFLxjg1htENSulZgxxG1AxVSyS3tumc9So/omA/g11qNvU+HB9H6BT9MO 9dY53/y9SuVlgyGit1V60vzsabIpImwZRYxLHwEVyLa7n8X7f1bHqWaLVS97cHFAUwyD P5xJY+oBHmWyF6iXJQftjhjh6e5y6t4wnPSN2b597AZwYdj1fcLGt+K0JZZ6+J/KcQ22 ntQte05dZnrxHhxof523o9P9bYR6eG2+Li5zYCQRY+tnj4EoFFp2VQwawtoDL/rlj+8g Cz5np9nzFi4mw4oBx0edeS2vuwaZgzop1elNl0WeFrD7nL2B4FtHQBEHxEpK851eA4jI Sr+w== X-Gm-Message-State: AAQBX9fKWRGnK4zxPgwcVwIjND4aDz1rtMM8cSU0+yoSCtYz0ZNCHYXF ApXNz2Plc05y7xONI4gqScNrYITtsFbtAQ== X-Google-Smtp-Source: AKy350Y2CpYhPpQz3TelUCAyG6aqu0pLZ5oLF1EiHQRxm/HZvMToTqhNKxYetu5I1tMJ+V0ffoZm3A== X-Received: by 2002:ac2:5231:0:b0:4e8:3ea0:cf37 with SMTP id i17-20020ac25231000000b004e83ea0cf37mr2851116lfl.34.1681052335540; Sun, 09 Apr 2023 07:58:55 -0700 (PDT) Received: from mail-lf1-f43.google.com (mail-lf1-f43.google.com. [209.85.167.43]) by smtp.gmail.com with ESMTPSA id a4-20020ac25204000000b004d86808fd33sm1656826lfl.15.2023.04.09.07.58.54 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 09 Apr 2023 07:58:55 -0700 (PDT) Received: by mail-lf1-f43.google.com with SMTP id h11so52841708lfu.8; Sun, 09 Apr 2023 07:58:54 -0700 (PDT) X-Received: by 2002:ac2:5385:0:b0:4e9:8c46:32ad with SMTP id g5-20020ac25385000000b004e98c4632admr2189475lfh.9.1681052334533; Sun, 09 Apr 2023 07:58:54 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: <202304090058.3390wrE1020757@gndrsh.dnsmgr.net> In-Reply-To: From: Cheng Cui Date: Sun, 9 Apr 2023 10:58:43 -0400 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: low TCP speed, wrong rtt measurement To: "Scheffenegger, Richard" Cc: "Rodney W. Grimes" , Richard Perini , "freebsd-hackers@FreeBSD.org" , "rscheff@FreeBSD.org" , "tuexen@freebsd.org" , "" Content-Type: multipart/alternative; boundary="0000000000003fb16405f8e87f20" X-Rspamd-Queue-Id: 4PvZxZ6ppqz3R1W X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --0000000000003fb16405f8e87f20 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable First of all, we need to make sure there are TCP retransmissions that are caused by packet loss. Otherwise, TCP congestion control or cwnd is irrelevant. Some tests like below from iperf3 or "netstat -s" can report TCP retransmissions. For example, over a 20ms link, the theoretical max cwnd size is determined by the Bandwidth Delay Product (BDP): 20ms x 10Mb/s =3D 25000 Bytes (around 25KB) cc@s1:~ % ping -c 3 r1 PING r1-link1 (10.1.1.3): 56 data bytes 64 bytes from 10.1.1.3: icmp_seq=3D0 ttl=3D64 time=3D19.807 ms 64 bytes from 10.1.1.3: icmp_seq=3D1 ttl=3D64 time=3D19.387 ms 64 bytes from 10.1.1.3: icmp_seq=3D2 ttl=3D64 time=3D19.488 ms --- r1-link1 ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/stddev =3D 19.387/19.561/19.807/0.179 ms before test: cc@s1:~ % netstat -sp tcp | egrep "tcp:|retrans|SACK" tcp: 0 data packets (0 bytes) retransmitted 0 data packets unnecessarily retransmitted 0 retransmit timeouts 0 retransmitted 0 SACK recovery episodes 0 segment rexmits in SACK recovery episodes 0 byte rexmits in SACK recovery episodes 0 SACK options (SACK blocks) received 0 SACK options (SACK blocks) sent 0 SACK retransmissions lost 0 SACK scoreboard overflow cc@s1:~ % iperf3 -c r1 -t 5 -i 1 Connecting to host r1, port 5201 [ 5] local 10.1.1.2 port 49487 connected to 10.1.1.3 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 2.58 MBytes 21.7 Mbits/sec 7 11.3 KBytes [ 5] 1.00-2.00 sec 1.39 MBytes 11.7 Mbits/sec 2 31.0 KBytes [ 5] 2.00-3.00 sec 1.14 MBytes 9.59 Mbits/sec 4 24.1 KBytes [ 5] 3.00-4.00 sec 1.01 MBytes 8.48 Mbits/sec 3 30.4 KBytes [ 5] 4.00-5.00 sec 1.33 MBytes 11.2 Mbits/sec 4 23.0 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-5.00 sec 7.46 MBytes 12.5 Mbits/sec 20 sende= r [ 5] 0.00-5.02 sec 7.23 MBytes 12.1 Mbits/sec receiver iperf Done. after test: cc@s1:~ % netstat -sp tcp | egrep "tcp:|retrans|SACK" tcp: 20 data packets (28960 bytes) retransmitted << 0 data packets unnecessarily retransmitted 0 retransmit timeouts 0 retransmitted 18 SACK recovery episodes 20 segment rexmits in SACK recovery episodes << 28960 byte rexmits in SACK recovery episodes 598 SACK options (SACK blocks) received 0 SACK options (SACK blocks) sent 0 SACK retransmissions lost 0 SACK scoreboard overflow > I've tried various transfer protocols: ftp, scp, rcp, http: results > are similar for all. Ping times for the closest WAN link is 2.3ms, > furthest is 60ms. On the furthest link, we get around 15% > utilisation. Transfer between > 2 Windows hosts on the furthest link yields ~80% utilisation. Thus, theoretical max cwnd the sender can grow up to is: 2.3ms x 2Mb/s =3D 575 Byts 60ms x 2Mb/s =3D 15000 Bytes (around 15KB) Best Regards, Cheng Cui On Sun, Apr 9, 2023 at 5:31=E2=80=AFAM Scheffenegger, Richard < Richard.Scheffenegger@netapp.com> wrote: > Hi, > > Adding fbsd-transport too. > > For stable-12, I believe all relevant (algorithm) improvements went in. > > However, 12.2 is missing D26807 and D26808 - improvements in Cubic to > retransmission timeouts (but these are not material) > > While 12.1. has none of the improvements done in 2020 to the Cubic module > - D18954, D18982, D19118, D23353, D23655, D25065, D25133, D25744, D24657, > D25746, D25976, D26060, D26807, D26808. > > These should fix numerous issues in cubic, which would very likely make i= t > perform poorly particularly on longer duration sessions. > > However, Cubic is heavily reliant on a valid measurement of RTT and the > epoch since the last congestion response (measured in units of RTT). An > issue in getting RTT measured properly would derail cubic for sure (most > likely cubic would inflate cwnd much faster, then running into significan= t > packet loss, very likely loss of retransmissions, followed by > retransmission timeouts, and shrinking of the ssthresh to small values. > > > I haven't looked into cc_vegas or the ertt module though. > > One more initial question: Are you using timestamps on that long, thin > pipe - or is net.inet.tcp.rfc1323 disabled (more recent versions allow th= e > selective enablement/disabling of window scaling and timestamps indepente= nd > of each other, but I don't think this is in and 12 release. (D36863)? > > Finally, you could be using SIFTR to track the evolution of the minrtt > value over the course of the session. > > Although I suspect ultimately a tcpdump including the tcp header (-s 80) = , > and the sifter internal state evolution would be optimal to understanding > when and why the RTT values go off the rails. > > > At first glance, the ertt module may be prone to miscalculations, when > retransmissions are in play - no special precautions appear to be present= , > to distinguish between the originally sent packet, and any retransmission= , > nor any filtering of ACKs which come in as duplicates. Thus there could b= e > a scenario, where an ACK for a spurious retransmission, e.g. due to > reordering, could lead to a wrong baseline RTT measurement, which is > physically impossible on such a long distance connection... > > But again, I haven't looked into the ertt module so far at all. > > How are the base stack RTT related values look on these misbehaving > sessions? > Tcpcb-> t_rttmin, t_srtt, t_rttvar, t_rxtcur, t_rtttime, t_rtseq, > t_rttlow, t_rttupdated > > Best regards, > Richard > > > > > -----Original Message----- > From: Rodney W. Grimes > Sent: Sonntag, 9. April 2023 02:59 > To: Richard Perini > Cc: freebsd-hackers@FreeBSD.org; rscheff@FreeBSD.org > Subject: Re: low TCP speed, wrong rtt measurement > > NetApp Security WARNING: This is an external email. Do not click links or > open attachments unless you recognize the sender and know the content is > safe. > > > > > > On Tue, Apr 04, 2023 at 02:46:34PM -0000, Peter 'PMc' Much wrote: > > > ** maybe this should rather go the -net list, but then > > > ** there are only bug messages > > > > > > Hi, > > > I'm trying to transfer backup data via WAN; the link bandwidth is > > > only ~2 Mbit, but this can well run for days and just saturate the > > > spare bandwidth. > > > > > > The problem is, it doesn't saturate the bandwidth. > > > > > > I found that the backup application opens the socket in this way: > > > if ((fd =3D socket(ipaddr->GetFamily(), SOCK_STREAM, 0)) < 0) { > > > > > > Apparently that doesn't work well. So I patched the application to > > > do it this way: > > > - if ((fd =3D socket(ipaddr->GetFamily(), SOCK_STREAM, 0)) < 0) = { > > > + if ((fd =3D socket(ipaddr->GetFamily(), SOCK_STREAM, > > > + IPPROTO_TCP)) < 0) { > > > > > > The result, observed with tcpdump, was now noticeably different, but > > > rather worse than better. > > > > > > I tried various cc algorithms, all behaved very bad with the > > > exception of cc_vegas. Vegas, after tuning the alpha and beta, gave > > > satisfying results with less than 1% tradeoff. > > > > > > But only for a time. After transferring for a couple of hours the > > > throughput went bad again: > > > > > > # netstat -aC > > > Proto Recv-Q Send-Q Local Address Foreign Address > (state) CC cwin ssthresh MSS ECN > > > tcp6 0 57351 edge-jo.26996 pole-n.22 > ESTABLISHED vegas 22203 10392 1311 off > > > tcp4 0 106305 edge-e.62275 pole-n.bacula-sd > ESTABLISHED vegas 11943 5276 1331 off > > > > > > The first connection is freshly created. The second one runs for a > > > day already , and it is obviousely hosed - it doesn't recover. > > > > > > # sysctl net.inet.tcp.cc.vegas > > > net.inet.tcp.cc.vegas.beta: 14 > > > net.inet.tcp.cc.vegas.alpha: 8 > > > > > > 8 (alpha) x 1331 (mss) =3D 10648 > > > > > > The cwin is adjusted to precisely one tick above the alpha, and > > > doesn't rise further. (Increasing the alpha further does solve the > > > issue for this connection - but that is not how things are supposed > > > to > > > work.) > > > > > > Now I tried to look into the data that vegas would use for it's > > > decisions, and found this: > > > > > > # dtrace -n 'fbt:kernel:vegas_ack_received:entry { printf("%s %u %d > > > %d %d %d", execname,\ (*((struct tcpcb **)(arg0+24)))->snd_cwnd,\ > > > ((struct ertt *)((*((struct tcpcb > > > **)(arg0+24)))->osd->osd_slots[0]))->minrtt,\ > > > ((struct ertt *)((*((struct tcpcb > > > **)(arg0+24)))->osd->osd_slots[0]))->marked_snd_cwnd,\ > > > ((struct ertt *)((*((struct tcpcb > > > **)(arg0+24)))->osd->osd_slots[0]))->bytes_tx_in_marked_rtt,\ > > > ((struct ertt *)((*((struct tcpcb > > > **)(arg0+24)))->osd->osd_slots[0]))->markedpkt_rtt);\ > > > }' > > > CPU ID FUNCTION:NAME > > > 6 17478 vegas_ack_received:entry ng_queue 11943 1 11943 > 10552 131 > > > 17 17478 vegas_ack_received:entry ng_queue 22203 56 22203 > 20784 261 > > > 17 17478 vegas_ack_received:entry ng_queue 22203 56 22203 > 20784 261 > > > 3 17478 vegas_ack_received:entry ng_queue 11943 1 11943 > 10552 131 > > > 5 17478 vegas_ack_received:entry ng_queue 22203 56 22203 > 20784 261 > > > 17 17478 vegas_ack_received:entry ng_queue 11943 1 11943 > 10552 131 > > > 11 17478 vegas_ack_received:entry ng_queue 11943 1 11943 > 10552 106 > > > 15 17478 vegas_ack_received:entry ng_queue 22203 56 22203 > 20784 261 > > > 13 17478 vegas_ack_received:entry ng_queue 22203 56 22203 > 20784 261 > > > 16 17478 vegas_ack_received:entry ng_queue 11943 1 11943 > 10552 106 > > > 3 17478 vegas_ack_received:entry ng_queue 22203 56 22203 > 20784 261 > > > > > > One can see that the "minrtt" value for the freshly created > > > connection is 56 (which is very plausible). > > > But the old and hosed connection shows minrtt =3D 1, which explains > > > the observed cwin. > > > > > > The minrtt gets calculated in sys/netinet/khelp/h_ertt.c: > > > e_t->rtt =3D tcp_ts_getticks() - txsi->tx_ts + 1; There > > > is a "+1", so this was apparently zero. > > > > > > But source and destination are at least 1000 km apart. So either we > > > have had one of the rare occasions of hyperspace tunnelling, or > > > something is going wrong in the ertt measurement code. > > > > > > For now this is a one-time observation, but it might also explain > > > why the other cc algorithms behaved badly. These algorithms are > > > widely in use and should work - the ertt measurement however is the > > > same for all of them. > > > > I can confirm I am seeing similar problems transferring files to our > > various production sites around Australia. Various types/sizes of links > and bandwidths. > > I can saturate the nearby links, but the link utilisation/saturation > > decreases with distance. > > > > I've tried various transfer protocols: ftp, scp, rcp, http: results > > are similar for all. Ping times for the closest WAN link is 2.3ms, > > furthest is 60ms. On the furthest link, we get around 15% > > utilisation. Transfer between > > 2 Windows hosts on the furthest link yields ~80% utilisation. > > Windows should be using cc_cubic, you say above you had tried all the > congestion algorithims, and only cc_vegas after tuning gave good results. > > > > > FreeBSD versions involved are 12.1 and 12.2. > > I wonder if cc_cubic is broken in 12.X, it should give similiar results t= o > windows if things are working correctly. > > I am adding Richard Scheffenegger as he is the most recent expect on the > congestion control code in FreeBSD. > > > -- > > Richard Perini > > Ramico Australia Pty Ltd Sydney, Australia rpp@ci.com.au +61 2 > 9552 5500 > > ---------------------------------------------------------------------- > > ------- "The difference between theory and practice is that in theory > > there is no difference, but in practice there is" > > -- > Rod Grimes > rgrimes@freebsd.org > --0000000000003fb16405f8e87f20 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
First of all, we need to make sure there are TCP retransmi= ssions that are caused by packet loss.
Otherwise, TCP congestion control= or cwnd is irrelevant.

Some tests like below from iperf3 or "n= etstat -s" can report TCP retransmissions.

For example, over a = 20ms link, the theoretical max cwnd size is determined by the
Bandwidth = Delay Product (BDP):
20ms x 10Mb/s =3D 25000 Bytes (around 25KB)

cc@s1:~ % ping -c 3 r1
PING r1-link1 (10.1.1.3): = 56 data bytes
64 bytes from 10.1.1.3: ic= mp_seq=3D0 ttl=3D64 time=3D19.807 ms
64 bytes from 10.1.1.3: icmp_seq=3D1 ttl=3D64 time=3D19.387 ms
64 bytes fro= m 10.1.1.3: icmp_seq=3D2 ttl=3D64 time=3D19= .488 ms

--- r1-link1 ping statistics ---
3 packets transmitted, 3= packets received, 0.0% packet loss
round-trip min/avg/max/stddev =3D 19= .387/19.561/19.807/0.179 ms

before test:
cc@s1:~ % netstat -sp tc= p | egrep "tcp:|retrans|SACK"
tcp:
0 data packets (0 byte= s) retransmitted
0 data packets unnecessarily retransmitted
0 retr= ansmit timeouts
0 retransmitted
0 SACK recovery episodes
0 seg= ment rexmits in SACK recovery episodes
0 byte rexmits in SACK recovery = episodes
0 SACK options (SACK blocks) received
0 SACK options (SACK= blocks) sent
0 SACK retransmissions lost
0 SACK scoreboard overflo= w

cc@s1:~ % iperf3 -c r1 -t 5 -i 1
Connecting to host r1, port 5= 201
[ =C2=A05] local 10.1.1.2 port 49487 connected to 10.1.1.3 port 5201=
[ ID] Interval =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Transfer =C2=A0 =C2= =A0 Bitrate =C2=A0 =C2=A0 =C2=A0 =C2=A0 Retr =C2=A0Cwnd
[ =C2=A05] =C2= =A0 0.00-1.00 =C2=A0 sec =C2=A02.58 MBytes =C2=A021.7 Mbits/sec =C2=A0 =C2= =A07 =C2=A0 11.3 KBytes =C2=A0 =C2=A0 =C2=A0
[ =C2=A05] =C2=A0 1.00-2.0= 0 =C2=A0 sec =C2=A01.39 MBytes =C2=A011.7 Mbits/sec =C2=A0 =C2=A02 =C2=A0 3= 1.0 KBytes =C2=A0 =C2=A0 =C2=A0
[ =C2=A05] =C2=A0 2.00-3.00 =C2=A0 sec = =C2=A01.14 MBytes =C2=A09.59 Mbits/sec =C2=A0 =C2=A04 =C2=A0 24.1 KBytes =C2=A0 =C2=A0 =C2=A0=
[ =C2=A05] =C2=A0 3.00-4.00 =C2=A0 sec =C2=A01.01 MBytes =C2=A08.48 Mb= its/sec =C2=A0 =C2=A03 =C2=A0 30.4 KBytes =C2=A0 =C2=A0 =C2=A0
[ =C2=A0= 5] =C2=A0 4.00-5.00 =C2=A0 sec =C2=A01.33 MBytes =C2=A011.2 Mbits/sec =C2= =A0 =C2=A04 =C2=A0 23.0 KBytes =C2=A0 =C2=A0 =C2=A0
- - - - - - - - - -= - - - - - - - - - - - - - - -
[ ID] Interval =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 Transfer =C2=A0 =C2=A0 Bitrate =C2=A0 =C2=A0 =C2=A0 =C2=A0 Retr<= br>[ =C2=A05] =C2=A0 0.00-5.00 =C2=A0 sec =C2=A07.46 MBytes =C2=A012.5 Mbit= s/sec =C2=A0 20 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sender
[ =C2= =A05] =C2=A0 0.00-5.02 =C2=A0 sec =C2=A07.23 MBytes =C2=A012.1 Mbits/sec = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0receiver
<= br>iperf Done.

after test:
cc@s1:~ % netstat -sp tcp | egrep &quo= t;tcp:|retrans|SACK"
tcp:
20 data packets (28960 bytes) retran= smitted =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<<
0 data packe= ts unnecessarily retransmitted
0 retransmit timeouts
0 retransmitt= ed
18 SACK recovery episodes
20 segment rexmits in SACK recovery epis= odes =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 <<= ;
28960 byte rexmits in SACK recovery episodes
598 SACK options (SACK= blocks) received
0 SACK options (SACK blocks) sent
0 SACK retransmis= sions lost
0 SACK scoreboard overflow

> I've tried various transfer protocols: ftp, scp,= rcp, http: results
> are similar for all.=C2=A0 Ping times for the c= losest WAN link is 2.3ms,
> furthest is 60ms.=C2=A0 On the furthest l= ink, we get around 15%
> utilisation. Transfer between
> 2 Wind= ows hosts on the furthest link yields ~80% utilisation.

Thus, theore= tical max cwnd the sender can grow up to is:
2.3ms x 2Mb/s =3D 575 Byts<= br>60ms =C2=A0x 2Mb/s =3D 15000 Bytes (around 15KB)

Best Regards,
Cheng Cui


On Sun, Apr 9, 2023= at 5:31=E2=80=AFAM Scheffenegger, Richard <Richard.Scheffenegger@netapp.com> wrote:
=
Hi,

Adding fbsd-transport too.

For stable-12, I believe all relevant (algorithm) improvements went in.

However, 12.2 is missing D26807 and D26808 - improvements in Cubic to retra= nsmission timeouts (but these are not material)

While 12.1. has none of the improvements done in 2020 to the Cubic module -= D18954, D18982, D19118, D23353, D23655, D25065, D25133, D25744, D24657, D2= 5746, D25976, D26060, D26807, D26808.

These should fix numerous issues in cubic, which would very likely make it = perform poorly particularly on longer duration sessions.

However, Cubic is heavily reliant on a valid measurement of RTT and the epo= ch since the last congestion response (measured in units of RTT). An issue = in getting RTT measured properly would derail cubic for sure (most likely c= ubic would inflate cwnd much faster, then running into significant packet l= oss, very likely loss of retransmissions, followed by retransmission timeou= ts, and shrinking of the ssthresh to small values.


I haven't looked into cc_vegas or the ertt module though.

One more initial question: Are you using timestamps on that long, thin pipe= - or is net.inet.tcp.rfc1323 disabled (more recent versions allow the sele= ctive enablement/disabling of window scaling and timestamps indepentend of = each other, but I don't think this is in and 12 release. (D36863)?

Finally, you could be using SIFTR to track the evolution of the minrtt valu= e over the course of the session.

Although I suspect ultimately a tcpdump including the tcp header (-s 80) , = and the sifter internal state evolution would be optimal to understanding w= hen and why the RTT values go off the rails.


At first glance, the ertt module may be prone to miscalculations, when retr= ansmissions are in play - no special precautions appear to be present, to d= istinguish between the originally sent packet, and any retransmission, nor = any filtering of ACKs which come in as duplicates. Thus there could be a sc= enario, where an ACK for a spurious retransmission, e.g. due to reordering,= could lead to a wrong baseline RTT measurement, which is physically imposs= ible on such a long distance connection...

But again, I haven't looked into the ertt module so far at all.

How are the base stack RTT related values look on these misbehaving session= s?
Tcpcb-> t_rttmin, t_srtt, t_rttvar, t_rxtcur, t_rtttime, t_rtseq, t_rttl= ow, t_rttupdated

Best regards,
=C2=A0 Richard




-----Original Message-----
From: Rodney W. Grimes <freebsd-rwg@gndrsh.dnsmgr.net>
Sent: Sonntag, 9. April 2023 02:59
To: Richard Perini <r= pp@ci.com.au>
Cc: freebsd-hackers@FreeBSD.org; rscheff@FreeBSD.org
Subject: Re: low TCP speed, wrong rtt measurement

NetApp Security WARNING: This is an external email. Do not click links or o= pen attachments unless you recognize the sender and know the content is saf= e.




> On Tue, Apr 04, 2023 at 02:46:34PM -0000, Peter 'PMc' Much wro= te:
> > ** maybe this should rather go the -net list, but then
> > ** there are only bug messages
> >
> > Hi,
> >=C2=A0 =C2=A0I'm trying to transfer backup data via WAN; the l= ink bandwidth is
> > only ~2 Mbit, but this can well run for days and just saturate th= e
> > spare bandwidth.
> >
> > The problem is, it doesn't saturate the bandwidth.
> >
> > I found that the backup application opens the socket in this way:=
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0if ((fd =3D socket(ipaddr->GetFamily= (), SOCK_STREAM, 0)) < 0) {
> >
> > Apparently that doesn't work well. So I patched the applicati= on to
> > do it this way:
> > -=C2=A0 =C2=A0 =C2=A0 if ((fd =3D socket(ipaddr->GetFamily(), = SOCK_STREAM, 0)) < 0) {
> > +=C2=A0 =C2=A0 =C2=A0 if ((fd =3D socket(ipaddr->GetFamily(), = SOCK_STREAM,
> > + IPPROTO_TCP)) < 0) {
> >
> > The result, observed with tcpdump, was now noticeably different, = but
> > rather worse than better.
> >
> > I tried various cc algorithms, all behaved very bad with the
> > exception of cc_vegas. Vegas, after tuning the alpha and beta, ga= ve
> > satisfying results with less than 1% tradeoff.
> >
> > But only for a time. After transferring for a couple of hours the=
> > throughput went bad again:
> >
> > # netstat -aC
> > Proto Recv-Q Send-Q Local Address=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 Foreign Address=C2=A0 =C2=A0 =C2=A0 =C2=A0 (state)=C2=A0 =C2=A0 =C2=A0C= C=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 cwin=C2=A0 =C2=A0ssthresh=C2=A0 =C2=A0M= SS ECN
> > tcp6=C2=A0 =C2=A0 =C2=A0 =C2=A00=C2=A0 57351 edge-jo.26996=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 pole-n.22=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 ESTABLISHED vegas=C2=A0 =C2=A0 =C2=A0 22203=C2=A0 =C2=A0 =C2=A0 = 10392=C2=A0 1311 off
> > tcp4=C2=A0 =C2=A0 =C2=A0 =C2=A00 106305 edge-e.62275=C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0pole-n.bacula-sd=C2=A0 =C2=A0 =C2=A0 =C2=A0ESTA= BLISHED vegas=C2=A0 =C2=A0 =C2=A0 11943=C2=A0 =C2=A0 =C2=A0 =C2=A05276=C2= =A0 1331 off
> >
> > The first connection is freshly created. The second one runs for = a
> > day already , and it is obviousely hosed - it doesn't recover= .
> >
> > # sysctl net.inet.tcp.cc.vegas
> > net.inet.tcp.cc.vegas.beta: 14
> > net.inet.tcp.cc.vegas.alpha: 8
> >
> > 8 (alpha) x 1331 (mss) =3D 10648
> >
> > The cwin is adjusted to precisely one tick above the alpha, and <= br> > > doesn't rise further. (Increasing the alpha further does solv= e the
> > issue for this connection - but that is not how things are suppos= ed
> > to
> > work.)
> >
> > Now I tried to look into the data that vegas would use for it'= ;s
> > decisions, and found this:
> >
> > # dtrace -n 'fbt:kernel:vegas_ack_received:entry { printf(&qu= ot;%s %u %d
> > %d %d %d", execname,\ (*((struct tcpcb **)(arg0+24)))->sn= d_cwnd,\
> > ((struct ertt *)((*((struct tcpcb
> > **)(arg0+24)))->osd->osd_slots[0]))->minrtt,\
> > ((struct ertt *)((*((struct tcpcb
> > **)(arg0+24)))->osd->osd_slots[0]))->marked_snd_cwnd,\ > > ((struct ertt *)((*((struct tcpcb
> > **)(arg0+24)))->osd->osd_slots[0]))->bytes_tx_in_marked_= rtt,\
> > ((struct ertt *)((*((struct tcpcb
> > **)(arg0+24)))->osd->osd_slots[0]))->markedpkt_rtt);\ > > }'
> > CPU=C2=A0 =C2=A0 =C2=A0ID=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 FUNCTION:NAME
> >=C2=A0 =C2=A06=C2=A0 17478=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0vegas_= ack_received:entry ng_queue 11943 1 11943 10552 131
> >=C2=A0 17=C2=A0 17478=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0vegas_ack_r= eceived:entry ng_queue 22203 56 22203 20784 261
> >=C2=A0 17=C2=A0 17478=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0vegas_ack_r= eceived:entry ng_queue 22203 56 22203 20784 261
> >=C2=A0 =C2=A03=C2=A0 17478=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0vegas_= ack_received:entry ng_queue 11943 1 11943 10552 131
> >=C2=A0 =C2=A05=C2=A0 17478=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0vegas_= ack_received:entry ng_queue 22203 56 22203 20784 261
> >=C2=A0 17=C2=A0 17478=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0vegas_ack_r= eceived:entry ng_queue 11943 1 11943 10552 131
> >=C2=A0 11=C2=A0 17478=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0vegas_ack_r= eceived:entry ng_queue 11943 1 11943 10552 106
> >=C2=A0 15=C2=A0 17478=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0vegas_ack_r= eceived:entry ng_queue 22203 56 22203 20784 261
> >=C2=A0 13=C2=A0 17478=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0vegas_ack_r= eceived:entry ng_queue 22203 56 22203 20784 261
> >=C2=A0 16=C2=A0 17478=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0vegas_ack_r= eceived:entry ng_queue 11943 1 11943 10552 106
> >=C2=A0 =C2=A03=C2=A0 17478=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0vegas_= ack_received:entry ng_queue 22203 56 22203 20784 261
> >
> > One can see that the "minrtt" value for the freshly cre= ated
> > connection is 56 (which is very plausible).
> > But the old and hosed connection shows minrtt =3D 1, which explai= ns
> > the observed cwin.
> >
> > The minrtt gets calculated in sys/netinet/khelp/h_ertt.c:
> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0e_t->rtt= =3D tcp_ts_getticks() - txsi->tx_ts + 1; There
> > is a "+1", so this was apparently zero.
> >
> > But source and destination are at least 1000 km apart. So either = we
> > have had one of the rare occasions of hyperspace tunnelling, or <= br> > > something is going wrong in the ertt measurement code.
> >
> > For now this is a one-time observation, but it might also explain=
> > why the other cc algorithms behaved badly. These algorithms are <= br> > > widely in use and should work - the ertt measurement however is t= he
> > same for all of them.
>
> I can confirm I am seeing similar problems transferring files to our <= br> > various production sites around Australia. Various types/sizes of link= s and bandwidths.
> I can saturate the nearby links, but the link utilisation/saturation <= br> > decreases with distance.
>
> I've tried various transfer protocols: ftp, scp, rcp, http: result= s
> are similar for all.=C2=A0 Ping times for the closest WAN link is 2.3m= s,
> furthest is 60ms.=C2=A0 On the furthest link, we get around 15%
> utilisation. Transfer between
> 2 Windows hosts on the furthest link yields ~80% utilisation.

Windows should be using cc_cubic, you say above you had tried all the conge= stion algorithims, and only cc_vegas after tuning gave good results.

>
> FreeBSD versions involved are 12.1 and 12.2.

I wonder if cc_cubic is broken in 12.X, it should give similiar results to = windows if things are working correctly.

I am adding Richard Scheffenegger as he is the most recent expect on the co= ngestion control code in FreeBSD.

> --
> Richard Perini
> Ramico Australia Pty Ltd=C2=A0 =C2=A0Sydney, Australia=C2=A0 =C2=A0rpp@ci.com.au=C2=A0 +61= 2 9552 5500
> ----------------------------------------------------------------------=
> ------- "The difference between theory and practice is that in th= eory
> there is no=C2=A0 difference, but in practice there is"

--
Rod Grimes=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0rgrimes@freebsd.org
--0000000000003fb16405f8e87f20-- From nobody Mon Apr 10 07:00:45 2023 X-Original-To: hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Pw0Hb3rXJz44tCk for ; Mon, 10 Apr 2023 07:00:59 +0000 (UTC) (envelope-from crb@chrisbowman.com) Received: from mail-pj1-x1035.google.com (mail-pj1-x1035.google.com [IPv6:2607:f8b0:4864:20::1035]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Pw0HZ5YfLz3nsD for ; Mon, 10 Apr 2023 07:00:58 +0000 (UTC) (envelope-from crb@chrisbowman.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=chrisbowman-com.20210112.gappssmtp.com header.s=20210112 header.b=xT6R5sYM; spf=none (mx1.freebsd.org: domain of crb@chrisbowman.com has no SPF policy when checking 2607:f8b0:4864:20::1035) smtp.mailfrom=crb@chrisbowman.com; dmarc=none Received: by mail-pj1-x1035.google.com with SMTP id h24-20020a17090a9c1800b002404be7920aso3546795pjp.5 for ; Mon, 10 Apr 2023 00:00:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chrisbowman-com.20210112.gappssmtp.com; s=20210112; t=1681110057; h=to:date:message-id:subject:mime-version:content-transfer-encoding :from:from:to:cc:subject:date:message-id:reply-to; bh=jKwvWVP3OzQeONKER1QWNCXXZEwlo7mfINzvskwyKGU=; b=xT6R5sYMjC8lBCEIFIuDwLtCqvTBwfujLgm9KSsi/vqLX8R1OwAuCPeLEjdHbVgX62 wXmQ8ciPaxn/3wv8YHeAXlGrsYB6BkcIaMCkh24z1HA8CNYBaenL0dtZKjjIAJrmUgTq S1nKWAKIMjgy7gY2YA7I8LiKHRGHb+z/M9G+008tTl1jaiu8xChfTNEKWXMOALu4bJ+B FuhvklNxxQs0ZaXpY7u+fYhzGw5C2SRiu3HfSMEJDckApz79fPEsQ3Ai8c4d4S2zbRlR jVmdk3ZRv+wuudT7DXfrudY92BrPzUE//M1Jj84cOG+/7voOF3rPiiBQt+Rv5GToOGe/ BqBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1681110057; h=to:date:message-id:subject:mime-version:content-transfer-encoding :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=jKwvWVP3OzQeONKER1QWNCXXZEwlo7mfINzvskwyKGU=; b=OjYk2oUfh7eccMr6Q9ToayjxdTglJcBJNYmHLFiLtnj3zkwgqmWLk4pJWnFKy0BgKX IBddhj30LdFP37fcEBXtIgfjZP64g272uOEo0fZtjMgBqnra0t9YrCFZU40FBHfPHOzl RyrZ+hxH+EhcjA8TGjEwv+KpZHSA4/yANppzIaKFAphqg87X0X0ZZUtxJByYh8Z8tco/ ud7iJwq5Rh9w4P8Vr+M6VEgTU+fqhnrdODde5KxZL/xQ3u9C5hNzQgVQrUmio4n4+y3e uFvZACC4CmrSNwT9tc/Apywj2qV2gy+uB/dfmsGK3CqZCT12QVkHmM26CbOz+qZWHvaD /UMg== X-Gm-Message-State: AAQBX9eoDs1LsbrSQNaQ4x7axKgY63iJ/UBUC/Wjrt/ZEiqS7Yb1601K GghHxqwU/9XKhPMBRV6Wfkt5IA/x7uxxnWrl5R+3Mw== X-Google-Smtp-Source: AKy350agNbITt516zOAPd/wOU8mJHmeelUBIHAxbQdD3WBySLVpyP74wpc4KKKCt9gFbgyuuAwyXBg== X-Received: by 2002:a17:90b:4ac5:b0:246:9374:febb with SMTP id mh5-20020a17090b4ac500b002469374febbmr5146150pjb.19.1681110056672; Mon, 10 Apr 2023 00:00:56 -0700 (PDT) Received: from smtpclient.apple ([2600:1700:5430:10b1:b13f:887c:f72a:a199]) by smtp.gmail.com with ESMTPSA id k91-20020a17090a4ce400b00246b1b4a3ffsm882537pjh.0.2023.04.10.00.00.56 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 10 Apr 2023 00:00:56 -0700 (PDT) From: Christopher Bowman Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.500.231\)) Subject: make installworld -DNO_ROOT Message-Id: Date: Mon, 10 Apr 2023 00:00:45 -0700 To: hackers@freebsd.org X-Mailer: Apple Mail (2.3731.500.231) X-Spamd-Result: default: False [-2.80 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[chrisbowman-com.20210112.gappssmtp.com:s=20210112]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[hackers@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::1035:from]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; DKIM_TRACE(0.00)[chrisbowman-com.20210112.gappssmtp.com:+]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[chrisbowman.com]; RCVD_VIA_SMTP_AUTH(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[hackers@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Queue-Id: 4Pw0HZ5YfLz3nsD X-Spamd-Bar: -- X-ThisMailContainsUnwantedMimeParts: N Id like to build some images and I=E2=80=99d like do that as an ordinary = user NOT as root I=E2=80=99m experimenting with =E2=80=9Cmake installworld -DNO_ROOT = TARGET_ARCH=3Darmv7=E2=80=9D When I do that or =E2=80=9Cmake distribution -DNO_ROOT = TARGET_ARCH=3Darmv7=E2=80=9D I get the following error messages during the make run: make[4] warning: /etc: Permission denied. make[4] warning: /lib: Permission denied. make[5] warning: /lib/geom: Permission denied. make[4] warning: /libexec: Permission denied. make[4] warning: /bin: Permission denied. make[4] warning: /rescue: Permission denied. make[4] warning: /sbin: Permission denied. make[4] warning: /etc: Permission denied. make[2] warning: /etc: Permission denied. make[3] warning: /etc/gss: Permission denied. make[3] warning: /etc/mtree: Permission denied. make[3] warning: /lib: Permission denied. make[4] warning: /lib/geom: Permission denied. make[3] warning: /libexec: Permission denied. make[3] warning: /bin: Permission denied. make[3] warning: /rescue: Permission denied. make[3] warning: /sbin: Permission denied. make[3] warning: /etc: Permission denied. I think as a results either the recursive makes are failing or not doing = all they are supposed to. It=E2=80=99s possible I=E2=80=99m using -DNO_ROOT wrong but if anyone = has some suggestions about what=E2=80=99s happening that would be very = helpful. In the alternative I=E2=80=99m open to suggestions as to how to debug = the hugs set of make invocations. I find reading and debug make files = hard enough but on huge treee like this with all the invocations and = arguments I=E2=80=99m kind lost as to even which command within the make = files are causing this. Thanks Christopher= From nobody Mon Apr 10 07:18:46 2023 X-Original-To: hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Pw0hF3XNsz44txh for ; Mon, 10 Apr 2023 07:18:53 +0000 (UTC) (envelope-from schakrabarti@microsoft.com) Received: from APC01-TYZ-obe.outbound.protection.outlook.com (mail-tyzapc01on2070c.outbound.protection.outlook.com [IPv6:2a01:111:f403:704b::70c]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "DigiCert Cloud Services CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Pw0hD3jrbz4GSP for ; Mon, 10 Apr 2023 07:18:52 +0000 (UTC) (envelope-from schakrabarti@microsoft.com) Authentication-Results: mx1.freebsd.org; none ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=L6gBSQ14iBVzjYJOE2dST6eIskfEdqDh/14GZdIvYPzBDrEgM8EHjjSdrjDsDYizdzPqCawFvLBbYsj7RoenW4p4VtzuYOOpHRpwCclNZIm8NFUmgs0kDWai/Dbl0YgnAeK9cTjmrR8YsXbsM5JTSqm8Qn5vkvdFEbVXjIxU2Gixj40lJLaUd0JtYRmySKVcyJgEcW6smOoP56S5LQPkvLamCdFBbulMZpKwWK+Vw9xP5X5FAKrl6viLKWZlKElQmX57d7r+pCoVCTWx8efwNKGBWOKMLAqqR9fdDZgFVhta5wft/IUlnxygdig4Kwk0ePz77gUu5Zq/7+2TMZeRAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=vZxz5Vu2mfnEWtIOLIQ0zJ3qxDdNq3pRYpI5QG5HYtg=; b=kCLjg9d9JIxaQwbQWCkpFr96aoFB0wsaZcRwUhQRZABXE/U3kRCOZdDNqMavIw7jnVWsepyqySEh8tloAVAoU5L0HKsbbCbap/afKILVfaPvpGQK0ueMDgsHlGxVn9W74JNLSRcnxunJ/QLcaIdAArQorCMQ2L0493Vq5u3NkKgJhe2jYsdGUhV7qcyVU59SRiB/yaoMqHc9lziQjqX69Hen8BG199sLbu0O+Nb4dWJaOf9E0SXk/14kU4LTNqksHXQ1uXYOrTUdYB1gWWYO+tfqJbQLwDveGGpRXqNEONHRyF9yzxpZ6Z2TReHfvnc4cABc0D5mAtXOb14JGFDtug== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=microsoft.com; dmarc=pass action=none header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vZxz5Vu2mfnEWtIOLIQ0zJ3qxDdNq3pRYpI5QG5HYtg=; b=Mr063KRM9uMghItcUU/kqngHOLeF8HvOzw4AJnI48GJCCUb2nFX+W36AsIMSB3ZTz+ORMfEizjuyg0qd59eZ2TfMSS7AWtxh46TzzWjUFIdI+DIyyunL+HrmzFGb9XeSe5NJM5CHNBqaix35KcATW9CVc+pjVUbFhC6eDEGCWzU= Received: from PSAP153MB0536.APCP153.PROD.OUTLOOK.COM (2603:1096:301:75::14) by SI2P153MB0703.APCP153.PROD.OUTLOOK.COM (2603:1096:4:19e::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6319.2; Mon, 10 Apr 2023 07:18:46 +0000 Received: from PSAP153MB0536.APCP153.PROD.OUTLOOK.COM ([fe80::912c:f20a:859:d034]) by PSAP153MB0536.APCP153.PROD.OUTLOOK.COM ([fe80::912c:f20a:859:d034%3]) with mapi id 15.20.6319.002; Mon, 10 Apr 2023 07:18:46 +0000 From: Souradeep Chakrabarti To: Christopher Bowman , "hackers@freebsd.org" Subject: Re: [EXTERNAL] make installworld -DNO_ROOT Thread-Topic: [EXTERNAL] make installworld -DNO_ROOT Thread-Index: AQHZa3pCcHGLHnveEUKO+kvtHsMLUq8kIkOc Date: Mon, 10 Apr 2023 07:18:46 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=True;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2023-04-10T07:18:47.222Z;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=General;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0;MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: PSAP153MB0536:EE_|SI2P153MB0703:EE_ x-ms-office365-filtering-correlation-id: e6175e4c-283b-496f-358d-08db3993d2bf x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: 7fG9l4mbJ2nRKABBw/G4bktAvhmywXKMWDmhmPyXrW5CtP/kgY2PC1TffgNK4ISpaXug8WE0UDmaUqsSHmeJPArXdL79arrBm+5Ri68i7xcyBqMPm0NDwHiSQAy1puUMrRutw2If3Uyt3KOcEbA3AZ6vCS9OLA2Le5C4hkKMMXf+zbYjAphyshV3ube3srEyXni05vi/ekMgk9ZUXZ8ofN6Uo3D4yTws2SQXhV+zU9HNmvPMA+N3AxPdZGIXl2hFGO7vLJ8BVO31QFZCYfZw7FIWxVyBPzqvmwpUaA/+DsCCia1WxsoVU2LlZMxucoCn1E6oJ4vxKVUiQmG6aS3vbH4g2CbGDsN9aFcaWgBI7qdMfySnFNkigpBCQPSw5PLgt72jAm+msGWejid/wd5MV1BmaUqHCgV/nx6zPyO8sjnO16bPTtG57wGZ2MHBA7NfF7rlK3uJpVEdhHn0gK/Z5VtCcqNrVkIArdlVGQrh/ACK9vE6MqQyaj59p/xXDI/ZyOR8t+ti9c24/LuLYijcPSb2Kw8ovfZ+PgWWULRCdz/GFrtp+ZKLZZ0ONK5HoyIBLx94etMMWLAw/lTsXD9aHhZ4RXayvf00ue+dI+tIG6F+Ruy4E8YfNyxg8A5Qd2dj0x1OJYqkYkzLjVJfefi3Y1zvxCdxpozhsxAay/34coc= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PSAP153MB0536.APCP153.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230028)(4636009)(376002)(39860400002)(346002)(136003)(396003)(366004)(451199021)(55016003)(83380400001)(76116006)(91956017)(71200400001)(478600001)(10290500003)(7696005)(186003)(110136005)(26005)(9686003)(6506007)(53546011)(8990500004)(2906002)(52536014)(33656002)(5660300002)(38100700002)(122000001)(66476007)(66556008)(66946007)(64756008)(82960400001)(82950400001)(41300700001)(66446008)(8676002)(8936002)(38070700005)(316002)(786003)(86362001)(19627405001);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?Windows-1252?Q?NLBkQQ5ns0JpdE5+nLX7jFyGOf7pNHV7RAC1i3HokR5lDoB+/bOnNxha?= =?Windows-1252?Q?J4w9pRHILjBeCXC4vOBl4EJgt7WPodEatTw+fvId1iPS5t/zBBqu5oDp?= =?Windows-1252?Q?fDlwwdRkUSEQD0/JjFOko9HiY7lNxzqxaoFiykRxV9WCutjaVmA/BOoY?= =?Windows-1252?Q?onptedFGYG/qf2w4FWGlCQ3komA9dnFyWFlG9nNnXxirIhrqewDbbn9i?= =?Windows-1252?Q?G1f3SkfMrBfJ02iH9VJme/uPPn0JzKUjoCbfBHvaTiF2flMgX0c1gzFc?= =?Windows-1252?Q?up0+mbNU/XjNAvPnuKnIUC96Y3pT5Vvi+lox9txkujQTcT5V6sdpzT8U?= =?Windows-1252?Q?oJyU/XPSesCTdyNoGz3pi+aJeQ4bcG/+uUR09sVs4fLaxSqVuvHla8s0?= =?Windows-1252?Q?+m9P/2xpxfuLdec6a9mENx46KdUT/hZN5bUNOfR5LAhlZoqBhA+xRxSQ?= =?Windows-1252?Q?MhVHbAfxkzfPtccI9zGptOl3L8VqwqtiQD2o9iiuqEG+e4cg9S+xuSEJ?= =?Windows-1252?Q?DmRYkOd4oYo1SORnXpXJ6Y/fulhsNhTeWN8++3nV7hxIikjYnbKZcVjz?= =?Windows-1252?Q?XN7a0IUucTfgyBiENw/k6wmDGUHyjShMFxYmu22QwwhoVjJCNSY+X30a?= =?Windows-1252?Q?gWMfSrx2GKAmudGr32rUdITdU8NPjV9VnUUtnH1pTz54sn8woQlXQo2d?= =?Windows-1252?Q?JyMbIunNlWUxKmbb3M5fs8dlv+NkRcaqqysl01qHegs+2GoYKTMiEYrA?= =?Windows-1252?Q?olb/sDEx+ZO7ag6sVBJOFRg+d6lMuiEubIrxqS875fnLgsofrF26YP9m?= =?Windows-1252?Q?bfrI1aqqV8e3GvduV0JOBDoD2Ms6EJ/XIoVA9ocoPg10H2TWeBWWuP6L?= =?Windows-1252?Q?pQOhHyBNzuw/HDIcybgkzrwGJbBPeIHndJJGNjPOGxJuCA1xBeU7YUTa?= =?Windows-1252?Q?YZQ2CPrEh8jwp84bsNIBh+87tXDrqghThbPO18vkoFGFjaKGO00Doe3/?= =?Windows-1252?Q?n+p1SuhzpWzFWp2XvapN1MK28vdAdttsj3VLrJDrVT991kZPu70m6cX+?= =?Windows-1252?Q?cemE8xiAdscXsUn39t5Rs22T+CbnPLul53zCdFHU6wEa0Vw2E/HdcB8Z?= =?Windows-1252?Q?vtb7dl/6tUtfbXEwxCx0CSS85/7NeNldpAlX6NeLlJoVizQaoiHXhbeV?= =?Windows-1252?Q?xg1He0lNk+HV+cFi8w9SMrwtKpM17Kk5nYukYCJN3L7XwryD8HAiEbrd?= =?Windows-1252?Q?SIo+WQG1gSUqnoatKA8TrJUSHckk5s6NQuPl32wv7jGKwutwj/zxlRq+?= =?Windows-1252?Q?kwaKOTqy6p7HTY3U4CPGUVWOI6rihihjGHarEXi5A/c5kQ4BqpZjzhXW?= =?Windows-1252?Q?q2QxFnhJ2bci3pJRReLeniL0AQxHCTnHBl9G/XDRABdX9TCRyYxt6DZy?= =?Windows-1252?Q?QfzHC+LYRxYshH15nkV30503fUM0euYXS1tdHLopfx/LeiLZb3fAJlYd?= =?Windows-1252?Q?MmqVewcLOQy3tCfRP6NANvLlZowXdMz/Z1p3HPKpD0QqYZ2tyUvqvbsD?= =?Windows-1252?Q?7z4Ziwmx3P/2S2NScTIhXnDrCgSn4aGBnU0BBcsVnWVrye8EvwKpXlSC?= =?Windows-1252?Q?Ido+cMva3sAfXzZFmXXdCFHB?= Content-Type: multipart/alternative; boundary="_000_PSAP153MB0536D554620251C222A1D72BCC959PSAP153MB0536APCP_" List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: PSAP153MB0536.APCP153.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: e6175e4c-283b-496f-358d-08db3993d2bf X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Apr 2023 07:18:46.1782 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: LRcWXng6gLPeZry9eZ9tYwE/f6eKBeVglPj9dBQqMMRDx5CwQEE/kdkLpCp1LAeYaFu2O+pdDtJ46X9OjoT025/eBL+TSb8JoD564/fq3AA= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SI2P153MB0703 X-Rspamd-Queue-Id: 4Pw0hD3jrbz4GSP X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:8075, ipnet:2a01:111:f000::/36, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --_000_PSAP153MB0536D554620251C222A1D72BCC959PSAP153MB0536APCP_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Have you tried to use export MAKEOBJDIRPREFIX ? Thanks & Regards, Souradeep ________________________________ From: owner-freebsd-hackers@freebsd.org = on behalf of Christopher Bowman Sent: Monday, April 10, 2023 12:30 PM To: hackers@freebsd.org Subject: [EXTERNAL] make installworld -DNO_ROOT Id like to build some images and I=92d like do that as an ordinary user NOT= as root I=92m experimenting with =93make installworld -DNO_ROOT TARGET_ARCH=3Darmv7= =94 When I do that or =93make distribution -DNO_ROOT TARGET_ARCH=3Darmv7=94 I get the following error messages during the make run: make[4] warning: /etc: Permission denied. make[4] warning: /lib: Permission denied. make[5] warning: /lib/geom: Permission denied. make[4] warning: /libexec: Permission denied. make[4] warning: /bin: Permission denied. make[4] warning: /rescue: Permission denied. make[4] warning: /sbin: Permission denied. make[4] warning: /etc: Permission denied. make[2] warning: /etc: Permission denied. make[3] warning: /etc/gss: Permission denied. make[3] warning: /etc/mtree: Permission denied. make[3] warning: /lib: Permission denied. make[4] warning: /lib/geom: Permission denied. make[3] warning: /libexec: Permission denied. make[3] warning: /bin: Permission denied. make[3] warning: /rescue: Permission denied. make[3] warning: /sbin: Permission denied. make[3] warning: /etc: Permission denied. I think as a results either the recursive makes are failing or not doing al= l they are supposed to. It=92s possible I=92m using -DNO_ROOT wrong but if anyone has some suggesti= ons about what=92s happening that would be very helpful. In the alternative I=92m open to suggestions as to how to debug the hugs se= t of make invocations. I find reading and debug make files hard enough but= on huge treee like this with all the invocations and arguments I=92m kind = lost as to even which command within the make files are causing this. Thanks Christopher --_000_PSAP153MB0536D554620251C222A1D72BCC959PSAP153MB0536APCP_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable
Have you tried to use = ; export MAKEOBJDIRPREFIX ?


Thanks & Regards,
 Souradeep

From: owner-freebsd-hackers= @freebsd.org <owner-freebsd-hackers@freebsd.org> on behalf of Christo= pher Bowman <crb@chrisbowman.com>
Sent: Monday, April 10, 2023 12:30 PM
To: hackers@freebsd.org <hackers@freebsd.org>
Subject: [EXTERNAL] make installworld -DNO_ROOT
 
Id like to build some images and I=92d like do tha= t as an ordinary user NOT as root

I=92m experimenting with =93make installworld -DNO_ROOT TARGET_ARCH=3Darmv7= =94

When I do that or  =93make distribution -DNO_ROOT TARGET_ARCH=3Darmv7= =94

I get the following error messages during the make run:

make[4] warning: /etc: Permission denied.
make[4] warning: /lib: Permission denied.
make[5] warning: /lib/geom: Permission denied.
make[4] warning: /libexec: Permission denied.
make[4] warning: /bin: Permission denied.
make[4] warning: /rescue: Permission denied.
make[4] warning: /sbin: Permission denied.
make[4] warning: /etc: Permission denied.
make[2] warning: /etc: Permission denied.
make[3] warning: /etc/gss: Permission denied.
make[3] warning: /etc/mtree: Permission denied.
make[3] warning: /lib: Permission denied.
make[4] warning: /lib/geom: Permission denied.
make[3] warning: /libexec: Permission denied.
make[3] warning: /bin: Permission denied.
make[3] warning: /rescue: Permission denied.
make[3] warning: /sbin: Permission denied.
make[3] warning: /etc: Permission denied.

I think as a results either the recursive makes are failing or not doing al= l they are supposed to.

It=92s possible I=92m using -DNO_ROOT wrong but if anyone has some suggesti= ons about what=92s happening that would be very helpful.

In the alternative I=92m open to suggestions as to how to debug the hugs se= t of make invocations.  I find reading and debug make files hard enoug= h but on huge treee like this with all the invocations and arguments I=92m = kind lost as to even which command within the make files are causing this.

Thanks
Christopher
--_000_PSAP153MB0536D554620251C222A1D72BCC959PSAP153MB0536APCP_-- From nobody Mon Apr 10 07:20:01 2023 X-Original-To: hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Pw0jp4SQMz44tld for ; Mon, 10 Apr 2023 07:20:14 +0000 (UTC) (envelope-from crb@chrisbowman.com) Received: from mail-pj1-x102a.google.com (mail-pj1-x102a.google.com [IPv6:2607:f8b0:4864:20::102a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Pw0jp2lT7z4JnK for ; Mon, 10 Apr 2023 07:20:14 +0000 (UTC) (envelope-from crb@chrisbowman.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pj1-x102a.google.com with SMTP id 90-20020a17090a0fe300b0023b4bcf0727so3733146pjz.0 for ; Mon, 10 Apr 2023 00:20:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chrisbowman-com.20210112.gappssmtp.com; s=20210112; t=1681111213; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:from:to:cc:subject:date:message-id:reply-to; bh=+KfNCiCH7wYvD6Li+wF6mPz3njt0Bl3E/uO6q/Tvazo=; b=rxSd24bv12le6Vxl6CifyDpthvlnGh10KSE7FZfWDviOyHCk1k6mhW4W6VuCLg3bq/ jgvYlfBVtHiwvXpTqcEvMw11xyb6oGDJsDTvujBraiKcTozL5VTGH5lYa9yPPihNmqO+ ETIYD4+zxMKtgJN9JeqX9XpSrfz6co989b6GlQHZW094ekGJLywmonWVOigeZra+P12q 8jXDoqd+77PGyDe1Ti1V98iMc2a6IkIeZiIjt41cVZwsnNsH6k7FjkvaJm/TbLBjOkyE rMscpsAcndc9zNOj7v3SPEmPGZCNK2ctGePiowTYAkyCi0w4QZh3UM1MJHG6tyzcI0xv bR7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1681111213; h=references:to:cc:in-reply-to:date:subject:mime-version:message-id :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=+KfNCiCH7wYvD6Li+wF6mPz3njt0Bl3E/uO6q/Tvazo=; b=t1oZNWtVN/Utywar3rNR5a5GSjk8LQPbcMgN5NZqII7Ob09bV1sNSyP7jponylnQG4 7nWuT9FvPN0qqpOMTTFk1x8/A2AZ9apLEw7q+d32G7V+TpXwgjWSqEveuotSXjtvcZr1 31mHx2IZxaptk/TkkIDUaVLhIIVJdJlw7fl7sgitE/NbQeW0Y5TJhZHgjg8tkDcP709t LLFON5UA+Jm6dqYJMLzMFbzRKwcnhrxZ7ZvNzpx9nQ0zInM9JxZr7otmkFPdO5YUBwea LjRG+RKHorljTwZZLGKajh59xU6RQD0B47GV2POu4xe+y/lHCxTDvrll96CuNhJmNtuA EIxw== X-Gm-Message-State: AAQBX9dKwUELYT3wCVoIuM3CCXsjk52ZLBfiztPEiJ9jg0GcdrvWl/NQ yRZGjbncuUcVX1dnwLX8B5UQjUwZuS4OcwXGfsV3nQ== X-Google-Smtp-Source: AKy350YjWy4VsBSuyvAohr76n3PY9n7tNWaxgC94XU5lykKHhBOBncLeJN8R7FWlaA8zR9pxz6lX/w== X-Received: by 2002:a17:903:68b:b0:1a6:4541:73c with SMTP id ki11-20020a170903068b00b001a64541073cmr1412634plb.33.1681111212725; Mon, 10 Apr 2023 00:20:12 -0700 (PDT) Received: from smtpclient.apple ([2600:1700:5430:10b1:1525:e2a6:94d0:9d3]) by smtp.gmail.com with ESMTPSA id ja3-20020a170902efc300b001a525b082a5sm3838878plb.199.2023.04.10.00.20.12 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 10 Apr 2023 00:20:12 -0700 (PDT) From: Christopher Bowman Message-Id: <6F51AE22-6892-4D06-A345-F645007E45B2@chrisbowman.com> Content-Type: multipart/alternative; boundary="Apple-Mail=_B7BBCC59-CBAB-4708-B839-FBCBFDDF4650" List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.500.231\)) Subject: Re: [EXTERNAL] make installworld -DNO_ROOT Date: Mon, 10 Apr 2023 00:20:01 -0700 In-Reply-To: Cc: "hackers@freebsd.org" To: Souradeep Chakrabarti References: X-Mailer: Apple Mail (2.3731.500.231) X-Rspamd-Queue-Id: 4Pw0jp2lT7z4JnK X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --Apple-Mail=_B7BBCC59-CBAB-4708-B839-FBCBFDDF4650 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii No, I am using DESTDIR and I apologize for having forgotten that > On Apr 10, 2023, at 12:18 AM, Souradeep Chakrabarti = wrote: >=20 > MAKEOBJDIRPREFIX --Apple-Mail=_B7BBCC59-CBAB-4708-B839-FBCBFDDF4650 Content-Transfer-Encoding: 7bit Content-Type: text/html; charset=us-ascii No, I am using DESTDIR and I apologize for having forgotten that

On Apr 10, 2023, at 12:18 AM, Souradeep Chakrabarti <schakrabarti@microsoft.com> wrote:

MAKEOBJDIRPREFIX

--Apple-Mail=_B7BBCC59-CBAB-4708-B839-FBCBFDDF4650-- From nobody Tue Apr 11 11:45:54 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PwkYx3S3Rz44jx5 for ; Tue, 11 Apr 2023 11:45:57 +0000 (UTC) (envelope-from Lee.MATTHEWS.external@stormshield.eu) Received: from mail.stormshield.eu (mail.stormshield.eu [91.212.116.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.stormshield.eu", Issuer "Stormshield Servers CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PwkYw47nwz3NT5 for ; Tue, 11 Apr 2023 11:45:56 +0000 (UTC) (envelope-from Lee.MATTHEWS.external@stormshield.eu) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=stormshield.eu header.s=signer header.b=Zukkzv7W; spf=pass (mx1.freebsd.org: domain of Lee.MATTHEWS.external@stormshield.eu designates 91.212.116.25 as permitted sender) smtp.mailfrom=Lee.MATTHEWS.external@stormshield.eu; dmarc=pass (policy=quarantine) header.from=stormshield.eu DKIM-Signature: v=1; a=rsa-sha256; d=stormshield.eu; s=signer; c=simple/simple; t=1681213555; h=from:subject:to:date:message-id; bh=/6/GZpPr1d7fjz2jdQch8vOJAmXOqtoSS4YOtuksSdA=; b=Zukkzv7WQFNp6Zjcg2IG8RnR6Nvu1psDnMJ5aOEq6ZY/GnS/nFf99B/fGozDv/jdQNreyoDb5G9 SIWAYC6gmTmj2ggXOLwiOZyWueRwj4AhHr6IHnY/JcXh44EvYq0XDh1piwuxNiW/lascekHqKSLLX FwA9LatUPeEuUhNCOAW7Cssnh5Bc9b312tSqnLApJgIZHFBywC6JWG9hzGG4gHjUaqAPeI1NpCB08 cxztX+4PA/d4QwgArg6T1CZxfeifuLaRh73L0TGwCkGn2ZtO6JOXvtXu5JnMbp9cEN9ZxNfzK5+5k 5so/+rILnHM26QodysR3WJf8hdCD4N2Hw4/A== Received: from ICTDCCEXCH001.one.local (10.180.4.1) by ICTDCCEXCH001.one.local (10.180.4.1) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.26; Tue, 11 Apr 2023 13:45:54 +0200 Received: from ICTDCCEXCH001.one.local ([::1]) by ICTDCCEXCH001.one.local ([::1]) with mapi id 15.02.1118.026; Tue, 11 Apr 2023 13:45:54 +0200 From: Lee MATTHEWS To: "freebsd-hackers@FreeBSD.org" Subject: BINIT and BERR signals in MCA Thread-Topic: BINIT and BERR signals in MCA Thread-Index: AQHZbGp/DawEN79MiUet3ycHaMLakw== Date: Tue, 11 Apr 2023 11:45:54 +0000 Message-ID: <4bd3e1017a104598ab92e658f25b5367@stormshield.eu> Accept-Language: en-GB, fr-FR, en-US Content-Language: en-GB X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.100.17.62] Content-Type: multipart/alternative; boundary="_000_4bd3e1017a104598ab92e658f25b5367stormshieldeu_" List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.997]; SUBJECT_ENDS_SPACES(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[stormshield.eu,quarantine]; R_SPF_ALLOW(-0.20)[+ip4:91.212.116.25]; R_DKIM_ALLOW(-0.20)[stormshield.eu:s=signer]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; MIME_TRACE(0.00)[0:+,1:+,2:~]; TO_DN_EQ_ADDR_ALL(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@FreeBSD.org]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; ASN(0.00)[asn:49068, ipnet:91.212.116.0/24, country:FR]; HAS_XOIP(0.00)[]; DKIM_TRACE(0.00)[stormshield.eu:+]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Queue-Id: 4PwkYw47nwz3NT5 X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N --_000_4bd3e1017a104598ab92e658f25b5367stormshieldeu_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hello, One of our clients is experiencing problems using one of our products. It r= uns FreeBSD 11.3 on an Intel Atom Apollo Lake E3930 two core SoC processor. Occasionally, under very light load, the kernel will panic. I've managed to= get a couple of vmcores and I notice via the backtrace that the MCA interr= upt is called. I've managed to recover two vmcores and I notice in both of them that the I= nter-Processor Interrupts are not being transferred from one CPU to the oth= er. I've also noticed that the structure mca_internal contains information = concerning the state of the MCA status register (value : 0x9000000020000003= ) for bank 0. >From Intel's software architecture document, the MCA Error Code is 0x0003 "= The BINIT# from another processor caused this processor to enter machine ch= eck." and the Model Specific Error Code is 0x2000 "1 if BERR is driven." The Intel document is not clear; could anyone please explain what the BINIT= and BERR signals mean? They appear to be related to a bus, but I'm not sur= e which one. A bus external to the Atom SoC or one of the internal buses wi= thin the Atom SoC? Do you have any ideas of what could generate this type of error? Is it like= ly a hardware or a software issue? Thanks in advance. Best wishes, Lee Matthews --_000_4bd3e1017a104598ab92e658f25b5367stormshieldeu_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Hello,

One of our clients is experiencing problems using one of our products. It r= uns FreeBSD 11.3 on an Intel Atom Apollo Lake E3930 two core SoC processor.=

Occasionally, under very light load, the kernel will panic. I've managed to= get a couple of vmcores and I notice via the backtrace that the MCA interr= upt is called.

I've managed to recover two vmcores and I notice in both of them that the I= nter-Processor Interrupts are not being transferred from one CPU to the oth= er. I've also noticed that the structure mca_internal contains information = concerning the state of the MCA status register (value : 0x9000000020000003) for bank 0.

>From Intel's software architecture document, the MCA Error Code is 0x0003 &= quot;The BINIT# from another processor caused this processor to enter machi= ne check." and the Model Specific Error Code is 0x2000 "1 if BERR= is driven."

The Intel document is not clear; could anyone please explain what the BINIT= and BERR signals mean? They appear to be related to a bus, but I'm not sur= e which one. A bus external to the Atom SoC or one of the internal buses wi= thin the Atom SoC?

Do you have any ideas of what could generate this type of error? Is it like= ly a hardware or a software issue?

Thanks in advance.

Best wishes,
Lee Matthews

--_000_4bd3e1017a104598ab92e658f25b5367stormshieldeu_-- From eugen@grosbein.net Tue Apr 11 11:59:08 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PwksT424bz44kwy for ; Tue, 11 Apr 2023 11:59:25 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PwksS0Q5Rz436r for ; Tue, 11 Apr 2023 11:59:23 +0000 (UTC) (envelope-from eugen@grosbein.net) Authentication-Results: mx1.freebsd.org; none Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221] (may be forged)) by hz.grosbein.net (8.17.1/8.17.1) with ESMTPS id 33BBxEM3086388 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 11 Apr 2023 11:59:15 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: Lee.MATTHEWS.external@stormshield.eu Received: from [10.58.0.11] (dadvw [10.58.0.11] (may be forged)) by eg.sd.rdtc.ru (8.16.1/8.16.1) with ESMTPS id 33BBxEEs001191 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 11 Apr 2023 18:59:14 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: BINIT and BERR signals in MCA To: Lee MATTHEWS , "freebsd-hackers@FreeBSD.org" References: <4bd3e1017a104598ab92e658f25b5367@stormshield.eu> From: Eugene Grosbein Message-ID: <24a51bf0-71de-f596-ef8b-785da4a27fd7@grosbein.net> Date: Tue, 11 Apr 2023 18:59:08 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 In-Reply-To: <4bd3e1017a104598ab92e658f25b5367@stormshield.eu> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,SHORTCIRCUIT autolearn=disabled version=3.4.6 X-Spam-Report: * -0.0 SHORTCIRCUIT No description available. * -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on hz.grosbein.net X-Rspamd-Queue-Id: 4PwksS0Q5Rz436r X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/32, country:DE] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N 11.04.2023 18:45, Lee MATTHEWS wrote: > Hello, > > One of our clients is experiencing problems using one of our products. It runs FreeBSD 11.3 on an Intel Atom Apollo Lake E3930 two core SoC processor. > > Occasionally, under very light load, the kernel will panic. I've managed to get a couple of vmcores and I notice via the backtrace that the MCA interrupt is called. > > I've managed to recover two vmcores and I notice in both of them that the Inter-Processor Interrupts are not being transferred from one CPU to the other. I've also noticed that the structure mca_internal contains information concerning the state of the MCA status register (value : 0x9000000020000003) for bank 0. > >>>From Intel's software architecture document, the MCA Error Code is 0x0003 "The BINIT# from another processor caused this processor to enter machine check." and the Model Specific Error Code is 0x2000 "1 if BERR is driven." > > The Intel document is not clear; could anyone please explain what the BINIT and BERR signals mean? They appear to be related to a bus, but I'm not sure which one. A bus external to the Atom SoC or one of the internal buses within the Atom SoC? > > Do you have any ideas of what could generate this type of error? Is it likely a hardware or a software issue? > > Thanks in advance. > > Best wishes, > Lee Matthews I believe this is some hardware issue, probably over-heating. Did you check for thermal sensor values? From nobody Tue Apr 11 13:13:11 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PwmVf2vqzz44r9t for ; Tue, 11 Apr 2023 13:13:14 +0000 (UTC) (envelope-from grahamperrin@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PwmVf1wWlz3PZr for ; Tue, 11 Apr 2023 13:13:14 +0000 (UTC) (envelope-from grahamperrin@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681218794; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type; bh=xzQ1asr1XuhWiHjAr+le0oHsw7wCX1Lpi8y4YEpZaZI=; b=yIEdHXjCLRBMpI+PdWzl0JSUzKhySmrIzcOtli3uOXaQu0OLjr2jphRdjqhu9A9MdOSbZK ryG4UlY+ai8QbbCTDNWURWRn3Ebvqq/Duu4svfC0Nr6qBMcRGBrpHIbWQoSENmQvMUh+5Z DeT4+fty5qqlP0SCEKfFnz5qSOdY97miJQjHHtZiL04OeCzwCIBDbpR9txwQLsUalD90ae rYXcbe1AroBioDGMGc3ewGtgpfbNC81T5waA3bPnDwttY/lCRWfLvixrYqc1W4qQSmFZbr WoBMvc+aIpY7gUJDNR6RdEYRwjU19wDQHByYapuLVaWJvuxKRZVffANu1B8NCw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681218794; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type; bh=xzQ1asr1XuhWiHjAr+le0oHsw7wCX1Lpi8y4YEpZaZI=; b=U2Vgej4VtadQ/Le9FQP7ToKzDJjMNP/CQqsxa1693dVwSMM1/SSVIjUcprG4hGx2sjYGuP vCEn44bJoFvYOeBIqX/3XnnmjamZgsamNjSRCo6z9ELNF4VjXh6tmXT8FeWfGPY8uDCF8y RLJd3F7JU/mDvoTPx1foUXyK2ov02Of69wj0wfsnmc6calokgu0JbiGyAFSFTGll6oIMMz +MlMHz1VTgSz+rGxhohDXkCDLRPTxN4GCT+p9WCMEYSNN7x1GS0VnpgUI3hMuNPtGHYtD1 9o+wV2i0Wf7RNCFSgH7CRtoe6xTMU7+km5FDEFgrDDmEpPdsRu0Jn7Hwg5QIdg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1681218794; a=rsa-sha256; cv=none; b=kXRrLsVOCxqMIrzTC4pqEiRx3uKDa1HWoWJLurg9ovdLdy0T+tak0p/cyOpF5PQ4akAUWJ PecbBaKlpOw06eAZ3jdTEdI4tgUEjqk8fuLxWPaZY6iVdkxG/z2k1YV1nvKl3fjlTHCGm/ MBHL9KXIdOcrDKnutzqu6aJvDVFjjV8Gjc5bL1RnuyXy5ZERseK4kipSlHSTXzVomKdtUn o17R90ScC8rH3CG7Omp52DnR40xB2y+3HMmpzR5vbIN0VKEIOmSARoozQjxtyTwheVpBXa kCR3DAEhLQuTlq0AFcjewtLLJ/WEroqGGzXCbhblQqK0683BozDFJKDXyEdiYQ== Received: from [194.81.204.27] (oa-mowa-01-194.81.204.27.brighton.ac.uk [194.81.204.27]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: grahamperrin) by smtp.freebsd.org (Postfix) with ESMTPSA id 4PwmVd5JZbz11XT for ; Tue, 11 Apr 2023 13:13:13 +0000 (UTC) (envelope-from grahamperrin@freebsd.org) Message-ID: <64e88636-1e03-7ff5-7d3f-e8a36117cdbe@freebsd.org> Date: Tue, 11 Apr 2023 14:13:11 +0100 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 To: freebsd-hackers@FreeBSD.org Content-Language: en-US From: Graham Perrin Subject: gpart destroy, in depth Organization: FreeBSD Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="------------EoQ9wzbjQc5HTA58jX0bMv05" X-ThisMailContainsUnwantedMimeParts: N This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --------------EoQ9wzbjQc5HTA58jX0bMv05 Content-Type: multipart/mixed; boundary="------------Shkh89dLjSpuxonrydbOC5D8"; protected-headers="v1" From: Graham Perrin To: freebsd-hackers@FreeBSD.org Message-ID: <64e88636-1e03-7ff5-7d3f-e8a36117cdbe@freebsd.org> Subject: gpart destroy, in depth --------------Shkh89dLjSpuxonrydbOC5D8 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: base64 UGxlYXNlOiB3aGF0LCBleGFjdGx5LCBvY2N1cnMgd2hlbiBhIHBhcnRpdGlvbiB0YWJsZSBp cyBkZXN0cm95ZWQgYnkgDQpncGFydCg4KT8NCg0KQmFja2dyb3VuZDogYSBjdXN0b21lciB3 YXMgMTAwJSBjZXJ0YWluIHRoYXQgbm8gZGF0YSB3YXMgcmVxdWlyZWQgZnJvbSBhIA0KaGFy ZCBkaXNrIGluIGFuIGlNYWMgdGhhdCB3aWxsIGJlIGRpc3Bvc2VkIG9mLiBJIGJvb3RlZCBm cm9tIGEgVVNCIGRyaXZlIA0Kd2l0aCBGcmVlQlNEIDE0LjAtQ1VSUkVOVCwgdGhlbiBsYXpp bHkgcmFuIGdwYXJ0IGRlc3Ryb3kgZm9yIHRoZSANCmludGVybmFsIGRpc2sgKHByb2JhYmx5 IGFkYTApLiBBIGZldyBob3VycyBsYXRlciwgdGhlIGN1c3RvbWVyIHJlYWxpc2VkIA0KdGhh dCB0aGUgZGlzayBwcm9iYWJseSBoZWxkIGEgZmV3IHllYXJzJyB3b3J0aCBvZiBpbXBvcnRh bnQgZW1haWxzLg0KDQpJIGRvbid0IGludGVuZCB0byBhdHRlbXB0IHJlY292ZXJ5LCBvciBy ZWNyZWF0aW9uIG9mIHRoZSBwYXJ0aXRpb24gDQp0YWJsZSwgYnV0IEknZCBsaWtlIHRvIHBy b3ZpZGUgZW5vdWdoIHRlY2huaWNhbCBpbmZvcm1hdGlvbiBmb3IgYSB0aGlyZCANCnBhcnR5 IHRvIGFkdmlzZSB0aGUgY3VzdG9tZXIgd2hldGhlciBkb2luZyBzbyB3aWxsIGJlIGVjb25v bWljYWxseSB2aWFibGUuDQoNCihJIHNlZSBmaWxlcyBzdWNoIGFzIGxpYi9nZW9tL3BhcnQv Z2VvbV9wYXJ0LmMgYW5kIA0Kc3lzL2dlb20vcGFydC9nX3BhcnQuYywgYnV0IEkgaGF2ZSBu byBpZGVhIGhvdyB0byBpbnRlcnByZXQgdGhlbS4pDQoNClRoYW5rcw0KDQo8aHR0cHM6Ly9n aXRodWIuY29tL2ZyZWVic2QvZnJlZWJzZC1zcmMvYmxvYi9tYWluL2xpYi9nZW9tL3BhcnQv Z2VvbV9wYXJ0LmM+DQoNCjxodHRwczovL2dpdGh1Yi5jb20vZnJlZWJzZC9mcmVlYnNkLXNy Yy9ibG9iL21haW4vc3lzL2dlb20vcGFydC9nX3BhcnQuYz4NCg0K --------------Shkh89dLjSpuxonrydbOC5D8-- --------------EoQ9wzbjQc5HTA58jX0bMv05 Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wsF5BAABCAAjFiEEWT/lssMHB+28ly8Kt2dIb0oY1AsFAmQ1XOcFAwAAAAAACgkQt2dIb0oY1AvO BA/9E3B1rHs8qnQmWt4e4W6pKsc4rd9vFnFXXzU6AY57C6lq86CQUeF1srD/tR++kmw1t4+Ganet 4jo/qqmTbSqtrW6LdUsuW/GcxH6aV6cg6BsMgQMIADCwY6hyzkZH+TK4tgHy7oTSu9Op4L5tlt0+ 6aiGNI18Zlt5cp9tO+3MkDsdFXkKsYzsR8c6/FTIPBh33u0GHlk+Bg2Pp2UNGxN8UezhXyHFIQ6i kXsLXRcE0N6PBNZ9ydOiXVkFFfLIjFedAP7sC3aB9jyfh0otto6B9bL+r6gUmBugSv7ustiNfkM5 YphVnvyV1eF1IOnnfO9hoB3xIdCTXhwZnE0Un4n1AV2p3f4Lx7zteBeF8gY5zDsmzQkHe2OIJCH7 5EllCv67PzmM1T+RJbO6u61VGj77z6wbv5OctsIT/oqvPemDT1Vvub6k8MJISFAfYu1Zxak5qM5R IZEr4mkkqBdevOhCANCZv695vEEDp5bkis2nEoebb+Q7nKXo28krUNzez6+oUAx/70mvr3b9wuVA Sg0LWSD6Ji6Z0SpT37X4aBRNTeNFPxkGmKjnjGDwg6a+ku6GeZbRF7Ld3vEp1sHg1LEsKA4bp29V J5hbXnBkAf4aeECNwTM8b+WFIJAxe9AyNkVs/KtHS7H0cPHWPqA6Dl4PDDLjjNd6TZDR1LyUdqpa 9e8= =uYva -----END PGP SIGNATURE----- --------------EoQ9wzbjQc5HTA58jX0bMv05-- From nobody Tue Apr 11 13:21:00 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Pwmgw5ZBJz44rW4; Tue, 11 Apr 2023 13:21:16 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-lj1-f173.google.com (mail-lj1-f173.google.com [209.85.208.173]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Pwmgv38dGz3sQv; Tue, 11 Apr 2023 13:21:15 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of carpeddiem@gmail.com designates 209.85.208.173 as permitted sender) smtp.mailfrom=carpeddiem@gmail.com; dmarc=none Received: by mail-lj1-f173.google.com with SMTP id x31so4429947ljq.10; Tue, 11 Apr 2023 06:21:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1681219273; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=v60PDIV+Tl3Kp0UgE7QzazOLzaPN2C6jSjCEFiH7Q8c=; b=yARSs3bMQuq+/vU+cJRXzYRc8vKgjqpSYpBUQkBDhG1wvGFS3ZFI+MibkMtlY0yQ9Z KmhdCnimvQ93dXAXJdMLe9YG1WloVwBXa/CSYeBHu/aOtJwufBXnHekvJjS2ZZ+Rdjf2 Fd2B7yKyROibREoqKuW4m1ls6ehyA1rr9OWDC8xC8gR+qDhJY3U7rhXnrRzaoFSXxMqY aLQYifhLeIF5zFWJI2uxWfiDTh2cbjlTxtWJsXQdKUztwZ03vcDB4xRLn95mQLW2Jq43 nNYZoOP0lQMBK0RG2kMqegtbGZBTLJyB8+SJGOVQWNDmu6BPjW89TmzGEs0bdM8Dq6W5 PMfA== X-Gm-Message-State: AAQBX9di2HrsNdOP31RVwLw83mmuVVM7KyIdmjCEc26cU5Pkp1KeQByC JhjmQv5zablNHd9k1Qli0ATIcDVUaUACqBxK3wYG6r254mc= X-Google-Smtp-Source: AKy350Zv/25hw9DM0qZ8lyeRCtPAw+YRhYCBum9uQogUqrlZ8PP45cEzehm3y/fgCm+oK+b8GSKu8N0XJvwlNQGC+FY= X-Received: by 2002:a2e:800e:0:b0:2a7:83fa:2392 with SMTP id j14-20020a2e800e000000b002a783fa2392mr1358841ljg.8.1681219272614; Tue, 11 Apr 2023 06:21:12 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Ed Maste Date: Tue, 11 Apr 2023 09:21:00 -0400 Message-ID: Subject: Re: Migrating to LLVM binutils tools (ar, nm, addr2line, etc.) To: "freebsd-toolchain@FreeBSD.org" , FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" X-Spamd-Result: default: False [-2.36 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-0.94)[-0.943]; NEURAL_HAM_SHORT(-0.41)[-0.413]; FORGED_SENDER(0.30)[emaste@freebsd.org,carpeddiem@gmail.com]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_NONE(0.00)[209.85.208.173:from]; MLMMJ_DEST(0.00)[freebsd-toolchain@freebsd.org,freebsd-hackers@freebsd.org]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.208.173:from]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCPT_COUNT_TWO(0.00)[2]; ARC_NA(0.00)[]; DMARC_NA(0.00)[freebsd.org]; FREEFALL_USER(0.00)[carpeddiem]; TO_DN_EQ_ADDR_SOME(0.00)[]; FROM_NEQ_ENVFROM(0.00)[emaste@freebsd.org,carpeddiem@gmail.com]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4Pwmgv38dGz3sQv X-Spamd-Bar: -- X-ThisMailContainsUnwantedMimeParts: N On Mon, 5 Jul 2021 at 11:09, Ed Maste wrote: > > FreeBSD migrated from GNU binutils to versions from ELF Tool Chain, > starting in 2014. At that time there were no usable LLVM versions of > those tools, but they have been developing rapidly since then. Now I > think it may be prudent to migrate to the LLVM tools where they exist, > for both functionality and maintainability reasons. Functionality arguments still stand, but ELF Tool Chain's maintainability situation has likely improved in the time since I wrote the original email. > I'd like to allow use of link-time optimization (LTO) in the FreeBSD > base system. LTO runs optimization passes over the entire executable > (or library) at link time and thus allows for more effective > optimization than when performed on individual compilation units. ... > Potential next steps are: > - Introduce new build knob Alex introduced WITH_LLVM_BINUTILS in 021385aba562. > - Iterate on exp-runs and call for testing The exp-run is open in 258872. Perl initially failed to build, which caused thousands of ports to be skipped. That issue was fixed by markj in 27f35b7dd418. The next issue is both tcl86 and postgresql13-client failing to build, resulting in a little over 5000 skipped ports. The error is of the form: strip: error: 'libtclstub86.a': not stripping symbol '.L.str.1' because it is named in a relocation tcl's build is requesting to discard all non-global symbols (-x) but has a local symbol referenced by a relocation entry. Both ELF Tool Chain and GNU strip leave the symbol intact in this case, and llvm-strip likely needs to do the same. There's an LLVM ticket for this open at https://github.com/llvm/llvm-project/issues/47468. When switching to LLVM's lld linker we introduced an LLD_UNSAFE tag which had the port fall back to using GNU ld. As a short term workaround for LLVM strip issues we could use a similar approach here. However, it looks like ports USE_BINUTILS infrastructure does not currently override strip, and that would first need to be fixed. See PR270663. > - Switch to LLVM tools by default Given the open ports issues we are not yet ready to entertain this change, and I am not sure there's enough time in advance of FreeBSD 14 to do so. From nobody Tue Apr 11 13:24:57 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PwmmR3gBMz44s3b for ; Tue, 11 Apr 2023 13:25:11 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com [IPv6:2a00:1450:4864:20::635]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PwmmR2x5Cz41kn for ; Tue, 11 Apr 2023 13:25:11 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ej1-x635.google.com with SMTP id kt17so8128054ejb.11 for ; Tue, 11 Apr 2023 06:25:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20210112.gappssmtp.com; s=20210112; t=1681219508; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=fqXCZdOTsDfw6BZ9Dz00UdchOQaCUOa9UvG/6tl8zrg=; b=aSMVX/9+xAaQXJa3te4CIKE2Ur71zZe4Q4AGWH3og3gghOtLRKTe4uhRp+aIxOrZjq PrGAoIi+B+nB4IgalsVG9AUiZEuUtefyp45Naam90qokQdcmpokQf+aNwOPBKCHeH7Zi 7QpRBp6wGUQDXlyQy+PFAQzx2nfi+Gg2CjSnGG5BGxPsrZ+GRHQKe+jdQgmMGuvLjfzq G33cyTNmSoco4Y19GtbgkN/tajJEVCj4wixOcNDQNTSmErGz1fBQvYQjZ4m81QEmWPP9 1n0sJy9XyxKm/ZkpFYHIXPgF2UgE+/puS28r+t9G4Gk0HZWR1OfxLPJByde6xCuKwSXp QqAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1681219508; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=fqXCZdOTsDfw6BZ9Dz00UdchOQaCUOa9UvG/6tl8zrg=; b=YKPklITwl/B5PZeiPMspGRy37zJlji4v05rgNoOFivR36MO5AaaLxAvtgkD3Yj/N1i gamOxgc9xFMK+51VinHoMKsX5nusP3PLdHgTIfanQK6Ehy87pSC4cSFmmL4xUAvpB4yY kIzSqxY9U84l9WrNq7UTDeD6Rqss2u7A8O7WYvEHUBWfAY4psld+QuGsmOQ84M4DLURt 2HCCodCeVFeTAYcevRt00oTMbmKhaUphCwdf6VFmZSjnPvgyyBTUIs13x20oh5UVFZK3 cWAVw7MHT1xbY2S4Qj3T59mXL9hzMEYW9QX8MAIg+N00feB0jMfMhp3wso+36QZ+02nQ Ugjw== X-Gm-Message-State: AAQBX9dYafmvtHPCcURMDj7hr3BtL+A0EjU+QGrKVn0E8BJ6dm/gkz9n YDlXjZ1eEQknkANGaCG/vZ6l6AyoDZUrHOytxCS32NPDs6+YcmB4 X-Google-Smtp-Source: AKy350YGC5zmi3cQ6S1c92DU2Wwtrui3eMKY4W59bb7/HgMh9G513Z2j6BLRC5EM9RHbmaP8TNYR1KfJg9jktEBTGlc= X-Received: by 2002:a17:906:d78a:b0:93d:a14f:c9b4 with SMTP id pj10-20020a170906d78a00b0093da14fc9b4mr4412327ejb.2.1681219508338; Tue, 11 Apr 2023 06:25:08 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: <64e88636-1e03-7ff5-7d3f-e8a36117cdbe@freebsd.org> In-Reply-To: <64e88636-1e03-7ff5-7d3f-e8a36117cdbe@freebsd.org> From: Warner Losh Date: Tue, 11 Apr 2023 07:24:57 -0600 Message-ID: Subject: Re: gpart destroy, in depth To: Graham Perrin Cc: FreeBSD Hackers Content-Type: multipart/alternative; boundary="00000000000095986605f90f6be8" X-Rspamd-Queue-Id: 4PwmmR2x5Cz41kn X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --00000000000095986605f90f6be8 Content-Type: text/plain; charset="UTF-8" It writes zeros to the partition table. Warner On Tue, Apr 11, 2023, 7:13 AM Graham Perrin wrote: > Please: what, exactly, occurs when a partition table is destroyed by > gpart(8)? > > Background: a customer was 100% certain that no data was required from a > hard disk in an iMac that will be disposed of. I booted from a USB drive > with FreeBSD 14.0-CURRENT, then lazily ran gpart destroy for the > internal disk (probably ada0). A few hours later, the customer realised > that the disk probably held a few years' worth of important emails. > > I don't intend to attempt recovery, or recreation of the partition > table, but I'd like to provide enough technical information for a third > party to advise the customer whether doing so will be economically viable. > > (I see files such as lib/geom/part/geom_part.c and > sys/geom/part/g_part.c, but I have no idea how to interpret them.) > > Thanks > > < > https://github.com/freebsd/freebsd-src/blob/main/lib/geom/part/geom_part.c > > > > > > --00000000000095986605f90f6be8 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
It writes zeros to the partition table.
=
Warner

On Tue, Apr 11, 2023, 7:13 AM Graha= m Perrin <grahamperrin@freeb= sd.org> wrote:
Please: what,= exactly, occurs when a partition table is destroyed by
gpart(8)?

Background: a customer was 100% certain that no data was required from a hard disk in an iMac that will be disposed of. I booted from a USB drive with FreeBSD 14.0-CURRENT, then lazily ran gpart destroy for the
internal disk (probably ada0). A few hours later, the customer realised that the disk probably held a few years' worth of important emails.

I don't intend to attempt recovery, or recreation of the partition
table, but I'd like to provide enough technical information for a third=
party to advise the customer whether doing so will be economically viable.<= br>
(I see files such as lib/geom/part/geom_part.c and
sys/geom/part/g_part.c, but I have no idea how to interpret them.)

Thanks

<https://git= hub.com/freebsd/freebsd-src/blob/main/lib/geom/part/geom_part.c>

<https://github= .com/freebsd/freebsd-src/blob/main/sys/geom/part/g_part.c>

--00000000000095986605f90f6be8-- From nobody Tue Apr 11 13:32:47 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PwmxN33Nwz44sd4 for ; Tue, 11 Apr 2023 13:32:56 +0000 (UTC) (envelope-from marck@rinet.ru) Received: from woozle.rinet.ru (woozle.rinet.ru [195.54.192.68]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PwmxM4G9Cz4HGD; Tue, 11 Apr 2023 13:32:55 +0000 (UTC) (envelope-from marck@rinet.ru) Authentication-Results: mx1.freebsd.org; none Received: from localhost (localhost [127.0.0.1]) by woozle.rinet.ru (8.14.5/8.14.5) with ESMTP id 33BDWlDB008481; Tue, 11 Apr 2023 16:32:47 +0300 (MSK) (envelope-from marck@rinet.ru) Date: Tue, 11 Apr 2023 16:32:47 +0300 (MSK) From: Dmitry Morozovsky To: Graham Perrin cc: freebsd-hackers@FreeBSD.org Subject: Re: gpart destroy, in depth In-Reply-To: <64e88636-1e03-7ff5-7d3f-e8a36117cdbe@freebsd.org> Message-ID: References: <64e88636-1e03-7ff5-7d3f-e8a36117cdbe@freebsd.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) X-OpenPGP-Key-ID: 6B691B03 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (woozle.rinet.ru [0.0.0.0]); Tue, 11 Apr 2023 16:32:47 +0300 (MSK) X-Rspamd-Queue-Id: 4PwmxM4G9Cz4HGD X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:8331, ipnet:195.54.192.0/19, country:RU] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Tue, 11 Apr 2023, Graham Perrin wrote: > Please: what, exactly, occurs when a partition table is destroyed by gpart(8)? > > Background: a customer was 100% certain that no data was required from a hard > disk in an iMac that will be disposed of. I booted from a USB drive with > FreeBSD 14.0-CURRENT, then lazily ran gpart destroy for the internal disk > (probably ada0). A few hours later, the customer realised that the disk > probably held a few years' worth of important emails. > > I don't intend to attempt recovery, or recreation of the partition table, but > I'd like to provide enough technical information for a third party to advise > the customer whether doing so will be economically viable. > > (I see files such as lib/geom/part/geom_part.c and sys/geom/part/g_part.c, but > I have no idea how to interpret them.) > > Thanks > > > > I suppose sysutils/testdisk will be useful for recovering, as most of actual file systems are left intact -- Sincerely, D.Marck [DM5020, MCK-RIPE, DM3-RIPN] [ FreeBSD committer: marck@FreeBSD.org ] --------------------------------------------------------------------------- *** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- woozle@woozle.net *** --------------------------------------------------------------------------- From nobody Tue Apr 11 14:10:11 2023 X-Original-To: hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PwnmV39tvz44w57 for ; Tue, 11 Apr 2023 14:10:18 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: from spindle.one-eyed-alien.net (spindle.one-eyed-alien.net [199.48.129.229]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PwnmT5l7vz4Js1 for ; Tue, 11 Apr 2023 14:10:17 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Authentication-Results: mx1.freebsd.org; dkim=none; spf=none (mx1.freebsd.org: domain of brooks@spindle.one-eyed-alien.net has no SPF policy when checking 199.48.129.229) smtp.mailfrom=brooks@spindle.one-eyed-alien.net; dmarc=none Received: by spindle.one-eyed-alien.net (Postfix, from userid 3001) id 295ED3C0199; Tue, 11 Apr 2023 14:10:11 +0000 (UTC) Date: Tue, 11 Apr 2023 14:10:11 +0000 From: Brooks Davis To: Christopher Bowman Cc: hackers@freebsd.org Subject: Re: make installworld -DNO_ROOT Message-ID: References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spamd-Result: default: False [-1.33 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; AUTH_NA(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.53)[-0.534]; FORGED_SENDER(0.30)[brooks@freebsd.org,brooks@spindle.one-eyed-alien.net]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[hackers@freebsd.org]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; R_SPF_NA(0.00)[no SPF record]; RCVD_COUNT_TWO(0.00)[2]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DMARC_NA(0.00)[freebsd.org]; ARC_NA(0.00)[]; ASN(0.00)[asn:36236, ipnet:199.48.128.0/22, country:US]; FREEFALL_USER(0.00)[brooks]; TO_DN_SOME(0.00)[]; FROM_HAS_DN(0.00)[]; FROM_NEQ_ENVFROM(0.00)[brooks@freebsd.org,brooks@spindle.one-eyed-alien.net] X-Rspamd-Queue-Id: 4PwnmT5l7vz4Js1 X-Spamd-Bar: - X-ThisMailContainsUnwantedMimeParts: N These warnings are annoying, but mostly harmless. A stalled and seemingly not quite right review to address the issue is at https://reviews.freebsd.org/D30990 -- Brooks On Mon, Apr 10, 2023 at 12:00:45AM -0700, Christopher Bowman wrote: > Id like to build some images and I???d like do that as an ordinary user NOT as root > > I???m experimenting with ???make installworld -DNO_ROOT TARGET_ARCH=armv7??? > > When I do that or ???make distribution -DNO_ROOT TARGET_ARCH=armv7??? > > I get the following error messages during the make run: > > make[4] warning: /etc: Permission denied. > make[4] warning: /lib: Permission denied. > make[5] warning: /lib/geom: Permission denied. > make[4] warning: /libexec: Permission denied. > make[4] warning: /bin: Permission denied. > make[4] warning: /rescue: Permission denied. > make[4] warning: /sbin: Permission denied. > make[4] warning: /etc: Permission denied. > make[2] warning: /etc: Permission denied. > make[3] warning: /etc/gss: Permission denied. > make[3] warning: /etc/mtree: Permission denied. > make[3] warning: /lib: Permission denied. > make[4] warning: /lib/geom: Permission denied. > make[3] warning: /libexec: Permission denied. > make[3] warning: /bin: Permission denied. > make[3] warning: /rescue: Permission denied. > make[3] warning: /sbin: Permission denied. > make[3] warning: /etc: Permission denied. > > I think as a results either the recursive makes are failing or not doing all they are supposed to. > > It???s possible I???m using -DNO_ROOT wrong but if anyone has some suggestions about what???s happening that would be very helpful. > > In the alternative I???m open to suggestions as to how to debug the hugs set of make invocations. I find reading and debug make files hard enough but on huge treee like this with all the invocations and arguments I???m kind lost as to even which command within the make files are causing this. > > Thanks > Christopher From nobody Tue Apr 11 14:28:35 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Pwp9g3d8yz44xYD for ; Tue, 11 Apr 2023 14:28:39 +0000 (UTC) (envelope-from Lee.MATTHEWS.external@stormshield.eu) Received: from mail.stormshield.eu (mail.stormshield.eu [91.212.116.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.stormshield.eu", Issuer "Stormshield Servers CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Pwp9f5Vwtz3N7B for ; Tue, 11 Apr 2023 14:28:38 +0000 (UTC) (envelope-from Lee.MATTHEWS.external@stormshield.eu) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; d=stormshield.eu; s=signer; c=simple/simple; t=1681223316; h=from:subject:to:date:message-id; bh=N7shFkf+gNzD4h55Avpw/vemyd4e4MhSfHZ69BMjG9Y=; b=DXQY5Aibr5WVrQwyNIa/Jv3NYFdBDBWFO3clM75RSX7plKxA0SRn02bicl/Sv2waq+x/Vfjo1ww EGe8atMGPqiU5JFQASNRYWCDT0CSqzbistAYilOXyJxG7FomY6ahndeN6UY5SRXpSEUcCp+/YV1RD 5ZxxuzYur/N3J8zMyMjJgz/iiA57szDFw7FXnfpRzEzwpgjB24pQ/EVO6f+lgEb2uhdnoJmGitG14 uUrxizAw2NiZ1nlLkaJPxMG0YnyeHNIZAh2UqMBr97DTcIz/VjIxRiH5808mkLL9fXREe0HZo09hX gj+nfFSBnLjGdjjadBi5GIChzBEZzASXQPBw== Received: from ICTDCCEXCH001.one.local (10.180.4.1) by ICTDCCEXCH001.one.local (10.180.4.1) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.26; Tue, 11 Apr 2023 16:28:36 +0200 Received: from ICTDCCEXCH001.one.local ([::1]) by ICTDCCEXCH001.one.local ([::1]) with mapi id 15.02.1118.026; Tue, 11 Apr 2023 16:28:36 +0200 From: Lee MATTHEWS To: Eugene Grosbein , "freebsd-hackers@FreeBSD.org" Subject: Re: BINIT and BERR signals in MCA Thread-Topic: BINIT and BERR signals in MCA Thread-Index: AQHZbGp/DawEN79MiUet3ycHaMLak68l364AgABJGqk= Date: Tue, 11 Apr 2023 14:28:35 +0000 Message-ID: References: <4bd3e1017a104598ab92e658f25b5367@stormshield.eu>,<24a51bf0-71de-f596-ef8b-785da4a27fd7@grosbein.net> In-Reply-To: <24a51bf0-71de-f596-ef8b-785da4a27fd7@grosbein.net> Accept-Language: en-GB, fr-FR, en-US Content-Language: en-GB X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.100.17.62] Content-Type: multipart/alternative; boundary="_000_fed099e1e6c2448e96883ebb6496347dstormshieldeu_" List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 X-Rspamd-Queue-Id: 4Pwp9f5Vwtz3N7B X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:49068, ipnet:91.212.116.0/24, country:FR] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --_000_fed099e1e6c2448e96883ebb6496347dstormshieldeu_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Thanks for getting back to me Eugene. On the two cores that I've received, they seem to die at the same point : #4 0xffffffff8049a9e3 in panic (fmt=3D) at ../../../kern/kern= _shutdown.c:714 #5 0xffffffff80780a2b in mca_intr () at ../../../x86/x86/mca.c:1193 #6 #7 smp_rendezvous_action () at ../../../kern/subr_smp.c:417 #8 0xffffffff804e5f79 in smp_rendezvous_cpus (map=3D..., setup_func=3D0xffffffff804e5e40 , action_func=3D0xffffffff80496730 , teardown_func=3D0xffffffff804e5e40 , arg=3D0= xffffffff80cb5048 ) at ../../../kern/subr_smp.c:554 #9 0xffffffff80496639 in _rm_wlock (rm=3D0xffffffff80cb5048 ) at ../../../kern/kern_rmlock.c:551 Do you think the temperature could still be an issue? If it were temperatur= e related, could one not expect the MCA interrupt to occur during other fun= ction calls? I've asked for a log of the CPU temperatures, I'll write back when I get th= em. Lee ________________________________ From: Eugene Grosbein Sent: 11 April 2023 13:59:08 To: Lee MATTHEWS; freebsd-hackers@FreeBSD.org Subject: Re: BINIT and BERR signals in MCA 11.04.2023 18:45, Lee MATTHEWS wrote: > Hello, > > One of our clients is experiencing problems using one of our products. It= runs FreeBSD 11.3 on an Intel Atom Apollo Lake E3930 two core SoC processo= r. > > Occasionally, under very light load, the kernel will panic. I've managed = to get a couple of vmcores and I notice via the backtrace that the MCA inte= rrupt is called. > > I've managed to recover two vmcores and I notice in both of them that the= Inter-Processor Interrupts are not being transferred from one CPU to the o= ther. I've also noticed that the structure mca_internal contains informatio= n concerning the state of the MCA status register (value : 0x90000000200000= 03) for bank 0. > >>>From Intel's software architecture document, the MCA Error Code is 0x0003= "The BINIT# from another processor caused this processor to enter machine = check." and the Model Specific Error Code is 0x2000 "1 if BERR is driven." > > The Intel document is not clear; could anyone please explain what the BIN= IT and BERR signals mean? They appear to be related to a bus, but I'm not s= ure which one. A bus external to the Atom SoC or one of the internal buses = within the Atom SoC? > > Do you have any ideas of what could generate this type of error? Is it li= kely a hardware or a software issue? > > Thanks in advance. > > Best wishes, > Lee Matthews I believe this is some hardware issue, probably over-heating. Did you check= for thermal sensor values? --_000_fed099e1e6c2448e96883ebb6496347dstormshieldeu_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable

Thanks for getting back to me Eugene.


On the two cores that I've received, they seem to die at the same point = :


#4  0xffffffff8049a9e3 in panic (fmt=3D<unavailable>) at ..= /../../kern/kern_shutdown.c:714
#5  0xffffffff80780a2b in mca_intr () at ../../../x86/x86/mca.c:1193 #6  <signal handler called>
#7  smp_rendezvous_action () at ../../../kern/subr_smp.c:417
#8  0xffffffff804e5f79 in smp_rendezvous_cpus (map=3D...,
    setup_func=3D0xffffffff804e5e40 <smp_no_rendezvous_ba= rrier>,
    action_func=3D0xffffffff80496730 <rm_cleanIPI>,     teardown_func=3D0xffffffff804e5e40 <smp_no_rendezvous= _barrier>, arg=3D0xffffffff80cb5048 <g_conf_lock>)
    at ../../../kern/subr_smp.c:554
#9  0xffffffff80496639 in _rm_wlock (rm=3D0xffffffff80cb5048 <g_con= f_lock>)
    at ../../../kern/kern_rmlock.c:551

Do you think the temperature could still be an issue? If it were tempera= ture related, could one not expect the MCA interrupt to occur during other = function calls?


I've asked for a log of the CPU temperatures, I'll write back when I get= them.


Lee

From: Eugene Grosbein <= ;eugen@grosbein.net>
Sent: 11 April 2023 13:59:08
To: Lee MATTHEWS; freebsd-hackers@FreeBSD.org
Subject: Re: BINIT and BERR signals in MCA
 
11.04.2023 18:45, Lee MATTHEWS wrote:

> Hello,
>
> One of our clients is experiencing problems using one of our products.= It runs FreeBSD 11.3 on an Intel Atom Apollo Lake E3930 two core SoC proce= ssor.
>
> Occasionally, under very light load, the kernel will panic. I've manag= ed to get a couple of vmcores and I notice via the backtrace that the MCA i= nterrupt is called.
>
> I've managed to recover two vmcores and I notice in both of them that = the Inter-Processor Interrupts are not being transferred from one CPU to th= e other. I've also noticed that the structure mca_internal contains informa= tion concerning the state of the MCA status register (value : 0x9000000020000003) for bank 0.
>
>>From Intel's software architecture document, the MCA Error Code is = 0x0003 "The BINIT# from another processor caused this processor to ent= er machine check." and the Model Specific Error Code is 0x2000 "1= if BERR is driven."
>
> The Intel document is not clear; could anyone please explain what the = BINIT and BERR signals mean? They appear to be related to a bus, but I'm no= t sure which one. A bus external to the Atom SoC or one of the internal bus= es within the Atom SoC?
>
> Do you have any ideas of what could generate this type of error? Is it= likely a hardware or a software issue?
>
> Thanks in advance.
>
> Best wishes,
> Lee Matthews

I believe this is some hardware issue, probably over-heating. Did you check= for thermal sensor values?


 --_000_fed099e1e6c2448e96883ebb6496347dstormshieldeu_-- From nobody Tue Apr 11 17:35:40 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PwtKV1QfHz44FZJ for ; Tue, 11 Apr 2023 17:35:42 +0000 (UTC) (envelope-from grahamperrin@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PwtKT6YZgz3Nk4; Tue, 11 Apr 2023 17:35:41 +0000 (UTC) (envelope-from grahamperrin@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681234541; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=kGE6ErXs6VLGVhwj8YtrOWa/LoXUsaQMbVeqUeL3pLY=; b=oiAG441Msww6ppXK+1/Iel6JYkGQlJVKCCsWkvuHxesGFF05nXkjmnHNrq+VjCh8CHc6na wheTvoq/7t97bHnX6kOHpJRUds2Jkhr7yMh3uyCPZRnfY61ge9BEW9sWL7ISCUF4qnQnwG 69ltdJrXxjW2dzxblD9+27YhirREaRAuKaoSie67O6bI95TzIH6ddS2gdFRk7ml4n6XzvP a+FwPW3jSAaMaZBXYGdb0x1vvL/C5HzfLkHQ8q4ZzuSB8X7UdMjEmRfPYnB9Da+ME0/77t lzfEqjD9wLLFXU2I1epwD6nkf+9JNarqa/gXsofbxu5HeaTNMZXZ757EyK/PBw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681234541; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=kGE6ErXs6VLGVhwj8YtrOWa/LoXUsaQMbVeqUeL3pLY=; b=pOL3/dzE/LDe2gm71G213VKBz90ueVs1q26jNAnXxtgO5RUIzw36eXvjusP+PaDfCr5C53 gG7ZFjZdcdhNnchwwx+vJy6+3JKWXRynyfZDHkzSSpMxWoYyZ2mqBJgxPK5cIcqvHErTgz g/qUVsTXZ3NDT0s7LFBadgQTUYYxu2PcrshXbel5nXoI1yOcFfoIRo9NpR3S9PLBqjRcFK lcFVgJaOUfyncAHUTPHVCQz8bpx8whu/yPPXCPwsNTvEmfunDH4wFHjVl4Ah0XCmMFhMFy K31ZLu68hmANhlbXh9qakDIfhc1+noGQwQc0k2jMq0uJeWBnhWKM6yaCL9tdYw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1681234541; a=rsa-sha256; cv=none; b=kNPvht3ayCcXQNxdP0YMzZyNslkb6Gej3rIrlQbVoPYPhLQrm39gglhIF5Jr+KFWj5Khnf l61zrnwShWwWr5LsNDUcbllpX5YFJhudacFpfyFSpR4naADsgjY78F3iSmVUfncZscjmn+ WmWR/CkufJjjXEl1wkNQnViIHufx4YpWdPPeEgZNtqH+h+QQBGKBCIYSXjn66YEZhAQJdL TmqA7q+dUuqYG5Zxh1YJx6/ZUXyjS3POOe0wDJv0s2Wb5kIcn9u2cPNH7WYNmc1Tpa60Gd IryNUFEJl/tyuzBxgf1QlzZGkIyixh0CzmNMRWpsHU1+ak92Mxt8ZfzYpNmuJA== Received: from [IPV6:2001:470:1f1c:a0::2] (tunnel642390-pt.tunnel.tserv1.lon2.ipv6.he.net [IPv6:2001:470:1f1c:a0::2]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) (Authenticated sender: grahamperrin) by smtp.freebsd.org (Postfix) with ESMTPSA id 4PwtKT3Kzyz15Zb; Tue, 11 Apr 2023 17:35:41 +0000 (UTC) (envelope-from grahamperrin@freebsd.org) Message-ID: <519b5387-4561-94a1-f2e1-6fe6c4a4f35e@freebsd.org> Date: Tue, 11 Apr 2023 18:35:40 +0100 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: gpart destroy, in depth Content-Language: en-US To: Warner Losh Cc: FreeBSD Hackers References: <64e88636-1e03-7ff5-7d3f-e8a36117cdbe@freebsd.org> From: Graham Perrin Organization: FreeBSD In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="------------esgQ6W4xJg61g3ZGvhVjs5XT" X-ThisMailContainsUnwantedMimeParts: N This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --------------esgQ6W4xJg61g3ZGvhVjs5XT Content-Type: multipart/mixed; boundary="------------Ev00wvEXl3XbZCKAeeyKdE0x"; protected-headers="v1" From: Graham Perrin To: Warner Losh Cc: FreeBSD Hackers Message-ID: <519b5387-4561-94a1-f2e1-6fe6c4a4f35e@freebsd.org> Subject: Re: gpart destroy, in depth References: <64e88636-1e03-7ff5-7d3f-e8a36117cdbe@freebsd.org> In-Reply-To: --------------Ev00wvEXl3XbZCKAeeyKdE0x Content-Type: multipart/alternative; boundary="------------mXhEaQsKUoxGdwtVYyQh0wmE" --------------mXhEaQsKUoxGdwtVYyQh0wmE Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: base64 T24gMTEvMDQvMjAyMyAxNDoyNCwgV2FybmVyIExvc2ggd3JvdGU6DQoNCj4gSXQgd3JpdGVz IHplcm9zIHRvIHRoZSBwYXJ0aXRpb24gdGFibGUuDQo+DQo+IFdhcm5lcg0KDQoNClRoYW5r cy4NCg0KSXMgaXQgcHJlY2lzZSBhYm91dCB3aGVyZSBpdCBjZWFzZXMgd3JpdGluZyB6ZXJv cyBpbiB0aGUgYXJlYXMgDQpwcmV2aW91c2x5IG9jY3VwaWVkIGJ5IHRoZSB0YWJsZSBhbmQg YmFja3VwIHRhYmxlPw0KDQpPciBkb2VzIGl0IChmb3Igd2FudCBvZiBhIGJldHRlciBleHBy ZXNzaW9uKSBzcGlsbCBvdmVyLCBhIGxpdHRsZT8NCg0KPg0KPiBPbiBUdWUsIEFwciAxMSwg MjAyMywgNzoxMyBBTSBHcmFoYW0gUGVycmluIDxncmFoYW1wZXJyaW5AZnJlZWJzZC5vcmc+ IA0KPiB3cm90ZToNCj4NCj4gICAgIFBsZWFzZTogd2hhdCwgZXhhY3RseSwgb2NjdXJzIHdo ZW4gYSBwYXJ0aXRpb24gdGFibGUgaXMgZGVzdHJveWVkIGJ5DQo+ICAgICBncGFydCg4KT8N Cj4NCj4gICAgIEJhY2tncm91bmQ6IOKApg0KPg0K --------------mXhEaQsKUoxGdwtVYyQh0wmE Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

On 11/04/2023 14:24, Warner Losh wrote:

It writes zeros to the partition table.

Warner


Thanks.

Is it precise about where it ceases writing zeros in the areas previously occupied by the table and backup table?

Or does it (for want of a better expression) spill over, a little?


On Tue, Apr 11, 2023, 7:13 = AM Graham Perrin <grah= amperrin@freebsd.org> wrote:
Please: what, exactly, occurs when a partition table is destroyed by gpart(8)?

Background: =E2=80=A6
--------------mXhEaQsKUoxGdwtVYyQh0wmE-- --------------Ev00wvEXl3XbZCKAeeyKdE0x-- --------------esgQ6W4xJg61g3ZGvhVjs5XT Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wsF5BAABCAAjFiEEWT/lssMHB+28ly8Kt2dIb0oY1AsFAmQ1mmwFAwAAAAAACgkQt2dIb0oY1AuS 3BAAjmwJW00rhzXC2vypwP42dXn3FaPWEy2mqDFxy4PLMm5gmDNm1PFizDpSOYAdmieCm+OWJ2hS msaDJLeawwXi6sNx6ih39yCxANWrrzReHuhQ/KkCT2xCLY9moD5dvgF9Cbr2jqIRpHOyEyoRNceH n+NBwds4Mx0L1bWiCYOOGaKr06/Qt/GenmkYCFd8onOZ/1HGiyUl4iWtvH4UpByGrW26yN5gPevo wA4EgSLe11o2JhEzkHlYa1lW/iOVAaGXLdiULYn10sIr6l8ejLo8XWnxaYW1UChT48cjTS+fSfu6 Zfqbkmy37tnCa6AyWuYlECIyROV2QMTwZ3uxVLYhtNQ7erBJ3y/m8DPcMPlX19wmyGkRV0188u/i OqfUvjDNeHTYz9HGmsvzLjaO3mMr6siQHqSiPD1HXPSQFP4NL1XDUV38Ey10TUIUG/Tf2KsWtMvf 1W2gEVMBfdzDv2ckJX+HbFLfSNmj0t9qQpbR6hBBFvyFeBXTEdHWOkCsoJDpO/gLHDBNcADWt4tm wEdZqGl5cTx4sj6LTYFpiGlGD6QRZNoRSA4luJkjLiQvBNj5Oa/TEqUPR0FDbwIt8BhQxPGKFuwd pLd+CEY0XXqJvStfkN1m45FEDUUd9SIGZ58JuN0UOe9S4B0FJpp81hsVf0fI0IGqCtTaMYMxsXU+ oZM= =YrrA -----END PGP SIGNATURE----- --------------esgQ6W4xJg61g3ZGvhVjs5XT-- From nobody Tue Apr 11 17:46:48 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PwtZM56pgz44GNf for ; Tue, 11 Apr 2023 17:46:51 +0000 (UTC) (envelope-from grahamperrin@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PwtZM4Rdqz3yKw; Tue, 11 Apr 2023 17:46:51 +0000 (UTC) (envelope-from grahamperrin@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681235211; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=WAmi9bozpOXoMjKD/rlH9AupTKl/OPrGZUDposijR9A=; b=vxRotY16bP9ge75dBHA3e+zbAHIiSAjELBatYhSOjEHrtBTU5RlAoV1tZ0M5ZX4fEDQ3Z+ h8V1fYxp/H98G0hQbAHO5brxjyQKrN/pEB/FmAp8dlERJruS7OyAK0h4yevsSJC6ztDWrT cVLPChMVsmp3MS5Vr08zbP52zKvYsNkkMk2KNFU7uNZ6JAmz5H2BUyS7vcNY16fiB8jczX 1gIchzK/Z96qzLjDA5c0vt3fNkusHzNWm0+BzyZpk2CXYfk5PcrMGMhY1m0IrqpB2QIS/w ibCDP8PnjhjJW4ZHjAjCKmPqnmmHg9T+9RVsx0teJQbk8yqJZHDq1GwKT0L5Rw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681235211; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=WAmi9bozpOXoMjKD/rlH9AupTKl/OPrGZUDposijR9A=; b=HcGiG1WrACm/AKDc5qs/hh0FXj7FVRKJI55YwIp4Uu8SzLBL6dKZw5BxUDiHkVGF3vnhYJ wksQJcw7pmLXRtsOPvzqjP3C8+MmI1Hk5vX7J6UIIweXlleARoxv4yIJCCPEd0OmyAsCBt sYLmkYfrscGzncGyojQx12RMgh6BkBVJnABdv8Y+puI+2uiNOe4FD9SJszm9r1cW8KuEpe ireYOTEMImmJ609QWbkHnuzktGnW0f1vEvWwsJ8e2+FhQPm/X4ePv+8lJPFy9zTiLblODF 2acceTnVr4x+x09pF+lvIpWDKeS8D87d58uaW91oOeac5StGP5kU35XUmhw8qw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1681235211; a=rsa-sha256; cv=none; b=ijgLh4US6LIqD+bbjI2uZexjxa3LvDU/uUS7z4en7ma1XbBnBnIuSkGhLfY4NoocZ2wKvl +n5mtpjRrt5mojugTAlBtya5CI/xzyJTb3DzRmDaz/M8G4o4W4+ahzXk2Cn71aZBdzGs7b HkFKzCipgvfExmWFxC59oblqIS5YUKTS2xH5aYwJAJajtYlLPEofK98QHNaatKDSF5kZQF FF69yEFArhs6b5kFhOHOQfCPItHpsXQo4czCXoJQ3JNEjHy5gWrFJmHfy5CxohwCIlymdc h2jBUZFNYAg6sYu40jSIBG1HcBOhdt35izxsZI/NIbxTIH2iHITW1C+bFiVToQ== Received: from [IPV6:2001:470:1f1c:a0::2] (tunnel642390-pt.tunnel.tserv1.lon2.ipv6.he.net [IPv6:2001:470:1f1c:a0::2]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: grahamperrin) by smtp.freebsd.org (Postfix) with ESMTPSA id 4PwtZM0BSxz14wY; Tue, 11 Apr 2023 17:46:50 +0000 (UTC) (envelope-from grahamperrin@freebsd.org) Message-ID: <29390cd8-56cd-aced-990a-f2b25a00aaed@freebsd.org> Date: Tue, 11 Apr 2023 18:46:48 +0100 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: gpart destroy, in depth Content-Language: en-US To: Dmitry Morozovsky Cc: freebsd-hackers@FreeBSD.org References: <64e88636-1e03-7ff5-7d3f-e8a36117cdbe@freebsd.org> From: Graham Perrin Organization: FreeBSD In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="------------5l42PBLi9Kl0KfR0LWijebST" X-ThisMailContainsUnwantedMimeParts: N This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --------------5l42PBLi9Kl0KfR0LWijebST Content-Type: multipart/mixed; boundary="------------UZoFeYqD9tSuRBHj0Z7OEk76"; protected-headers="v1" From: Graham Perrin To: Dmitry Morozovsky Cc: freebsd-hackers@FreeBSD.org Message-ID: <29390cd8-56cd-aced-990a-f2b25a00aaed@freebsd.org> Subject: Re: gpart destroy, in depth References: <64e88636-1e03-7ff5-7d3f-e8a36117cdbe@freebsd.org> In-Reply-To: --------------UZoFeYqD9tSuRBHj0Z7OEk76 Content-Type: multipart/alternative; boundary="------------40jmnQhciEob1sS5JL2ZPdkD" --------------40jmnQhciEob1sS5JL2ZPdkD Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: base64 T24gMTEvMDQvMjAyMyAxNDozMiwgRG1pdHJ5IE1vcm96b3Zza3kgd3JvdGU6DQo+PiDigKYN Cj4gSSBzdXBwb3NlIHN5c3V0aWxzL3Rlc3RkaXNrIHdpbGwgYmUgdXNlZnVsIGZvciByZWNv dmVyaW5nLCBhcyBtb3N0IG9mIGFjdHVhbA0KPiBmaWxlIHN5c3RlbXMgYXJlIGxlZnQgaW50 YWN0DQoNCg0KVGhhbmtzLCBJIGZvcmdvdCB0aGF0IHRoZSB1dGlsaXR5IGhhcyB0aGlzIGNh cGFiaWxpdHkuDQoNClRoaXMgcmVpZ25pdGVzIGhvcGUgb2YgcmVjb3Zlcnkgd2l0aG91dCBh IHRoaXJkIHBhcnR5Lg0KDQpSZTogbXkgcXVlc3Rpb24gdG8gV2FybmVyIGFib3V0IHByZWNp c2VuZXNzLCBub3cgSSByZWFsaXNlLCB0aGUgYW5zd2VyIA0KaXMgYWxtb3N0IGNlcnRhaW5s eSAiWWVzIiwgYmVjYXVzZSB0aGUgbWFudWFsIHBhZ2UgZm9yIGdwYXJ0KDgpIG5vdGVzIA0K dGhhdCB0aGUgJ3Jlc3RvcmUnIGFyZ3VtZW50L2NvbW1hbmQsIHdoaWNoIGRvZXMgbm90IGFm ZmVjdCB0aGUgY29udGVudCANCm9mIHBhcnRpdGlvbnMsIGNhbiBiZSBwcmVjZWRlZCBieSBk ZXN0cnVjdGlvbiAoLUYpLg0KDQpBbHNvLCB0aGFua3MgdG8gdGhlIHBlb3BsZSB3aG8gcmVz cG9uZGVkIHByaXZhdGVseS4NCg0K --------------40jmnQhciEob1sS5JL2ZPdkD Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
On 11/04/2023 14:32, Dmitry Morozovsky= wrote:
=E2=80=A6 
I suppose sysutils/testdisk =
will be useful for recovering, as most of actual=20
file systems are left intact


Thanks, I forgot that the utility has this capability.

This reignites hope of recovery without a third party.

Re: my question to Warner about preciseness, now I realise, the answer is almost certainly "Yes", because the manual page for gpart(8) notes that the 'restore' argument/command, which does not affect the content of partitions, can be preceded by destruction (-F).=C2=A0

Also, thanks to the people who responded privately.

--------------40jmnQhciEob1sS5JL2ZPdkD-- --------------UZoFeYqD9tSuRBHj0Z7OEk76-- --------------5l42PBLi9Kl0KfR0LWijebST Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wsF5BAABCAAjFiEEWT/lssMHB+28ly8Kt2dIb0oY1AsFAmQ1nQkFAwAAAAAACgkQt2dIb0oY1AuI 6g/9H/udi9+eqtO2idhKa3WKYT6M47XCRUmgJuM0VTaldEl8ANHP85ZaGbJ764q10f2mFAVCXGrQ h5McaoC4am1Trj4Eqf75wTLMbwWc38rq/P5HUPjavDn/90EBuJlqDiHjE04X02Ty3umbeJSCWgA9 31yB+/yYn7ycy6ffkT6Yagu51Iwfir1e5BFXijyBuwP4SlmJCZAw91lV4FQ7Lg5eVMbbiimmf0cp QVhOttofGwMcsytwG0o9XWtMk5/OqhKZi+xnJmL0VYF/xrWl+Tm0l0uesIT6DjlrfHN+GyDqL/ON nzx9Qm07c+JzDQxufnqp2FjYLxdA4RyuAjGRhCP+AR3i6deCd+PZ/hDfq0EvoDYN957lhUZzV6Hw 5w9nSbx1lG9anQ8zbMPshn4neigSzEOHtk8+B7nKAJVJJNt3X7Wi2Q/D8roCvvWXf3ecRwfZjzvM koEH/HdDoDS0+IP7hAQH+GFG2hmFZEryCCUGljpnYzSJAQoBZsWvvFbzuWMv1qJ2A2EwqrazC2Rt zdm4pPB9s3kPe7cl0pH8jgDXe2PZtKHUHdCUbg98k60EZOB9QoQJe5eqLwh15y7q2IXEEuW3RbwU kLr3SttDoTzMWpZEqeEcISb1Gmwt3a419hEOqsoS3OV6VHesd3l6ctO/dYi5zitWpn+QnaZ7TsM7 fio= =vDkx -----END PGP SIGNATURE----- --------------5l42PBLi9Kl0KfR0LWijebST-- From nobody Tue Apr 11 18:03:56 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PwtyT02TKz44J2y for ; Tue, 11 Apr 2023 18:04:17 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from omta001.cacentral1.a.cloudfilter.net (omta001.cacentral1.a.cloudfilter.net [3.97.99.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PwtyS5Jf7z3CZd; Tue, 11 Apr 2023 18:04:16 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Authentication-Results: mx1.freebsd.org; none Received: from shw-obgw-4003a.ext.cloudfilter.net ([10.228.9.183]) by cmsmtp with ESMTP id mAWCp0MVRuZMSmILnpIamk; Tue, 11 Apr 2023 18:04:15 +0000 Received: from spqr.komquats.com ([70.66.148.124]) by cmsmtp with ESMTPA id mILmpu6H0cyvumILnp0t3U; Tue, 11 Apr 2023 18:04:15 +0000 X-Authority-Analysis: v=2.4 cv=VbHkgXl9 c=1 sm=1 tr=0 ts=6435a11f a=Cwc3rblV8FOMdVN/wOAqyQ==:117 a=Cwc3rblV8FOMdVN/wOAqyQ==:17 a=IkcTkHD0fZMA:10 a=dKHAf1wccvYA:10 a=6I5d2MoRAAAA:8 a=YxBL1-UpAAAA:8 a=EkcXrb_YAAAA:8 a=-K33PcvFBo8K_hxmGGEA:9 a=QEXdDO2ut3YA:10 a=IjZwj45LgO3ly-622nXo:22 a=Ia-lj3WSrqcvXOmTRaiG:22 a=LK5xJRSDVpKd5WXXoEvA:22 Received: from [127.0.0.1] (S0106002401cb186f.gv.shawcable.net [70.67.229.168]) by spqr.komquats.com (Postfix) with ESMTPSA id 8EB7A1F5; Tue, 11 Apr 2023 11:04:13 -0700 (PDT) Date: Tue, 11 Apr 2023 11:03:56 -0700 From: Cy Schubert To: freebsd-hackers@freebsd.org, Graham Perrin , Warner Losh CC: FreeBSD Hackers Subject: Re: gpart destroy, in depth In-Reply-To: <519b5387-4561-94a1-f2e1-6fe6c4a4f35e@freebsd.org> References: <64e88636-1e03-7ff5-7d3f-e8a36117cdbe@freebsd.org> <519b5387-4561-94a1-f2e1-6fe6c4a4f35e@freebsd.org> Message-ID: <372DCA30-5F1D-48E7-9D12-159E33EC1878@cschubert.com> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-CMAE-Envelope: MS4xfB4ThsSJdrqCsJW1rNicQzXcATR3uH9QUSVfWTJbhDZDqwxHCHcvfqiGNd5dn8fOm4Rnhn3ePNjC0OnZob60U2kqD66SfSRxJMzzjf0+Cw+1IjcZ4jiN 42Zhb2gD0i+gwUVuY/rgzJtXFD1c23GyicYPn1GLIPPs9EtZfp2pjvHSr/547L3Arau9PwTXiabJxBjqJGSwBs+gV2tIEdNmzZd18EmYrebJWuTh5eobnM1f tiNmvyVmI0kvGeH/63xbRjzlzAzhuOGVmZDvYK+t8cA= X-Rspamd-Queue-Id: 4PwtyS5Jf7z3CZd X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:16509, ipnet:3.96.0.0/15, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On April 11, 2023 10:35:40 AM PDT, Graham Perrin wrote: >On 11/04/2023 14:24, Warner Losh wrote: > >> It writes zeros to the partition table=2E >>=20 >> Warner > > >Thanks=2E > >Is it precise about where it ceases writing zeros in the areas previously= occupied by the table and backup table? > >Or does it (for want of a better expression) spill over, a little? > >>=20 >> On Tue, Apr 11, 2023, 7:13 AM Graham Perrin wrote: >>=20 >> Please: what, exactly, occurs when a partition table is destroyed b= y >> gpart(8)? >>=20 >> Background: =E2=80=A6 >>=20 You should be able to simply recreate the partitions as before=2E The data= will be untouched, unless the disk has been reused=2E --=20 Cheers, Cy Schubert FreeBSD UNIX: Web: https://FreeBSD=2Eorg NTP: Web: https://nwtime=2Eorg e^(i*pi)+1=3D0 Pardon the typos=2E Small keyboard in use=2E From eugen@grosbein.net Wed Apr 12 03:14:52 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Px79r5JYJz456YZ for ; Wed, 12 Apr 2023 03:14:56 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Px79r00KGz4L9P for ; Wed, 12 Apr 2023 03:14:55 +0000 (UTC) (envelope-from eugen@grosbein.net) Authentication-Results: mx1.freebsd.org; none Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221] (may be forged)) by hz.grosbein.net (8.17.1/8.17.1) with ESMTPS id 33C3Er2V096423 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 12 Apr 2023 03:14:53 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: Lee.MATTHEWS.external@stormshield.eu Received: from [10.58.0.11] (dadvw [10.58.0.11] (may be forged)) by eg.sd.rdtc.ru (8.16.1/8.16.1) with ESMTPS id 33C3EqeI011097 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Wed, 12 Apr 2023 10:14:52 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: BINIT and BERR signals in MCA To: Lee MATTHEWS , "freebsd-hackers@FreeBSD.org" References: <4bd3e1017a104598ab92e658f25b5367@stormshield.eu> <24a51bf0-71de-f596-ef8b-785da4a27fd7@grosbein.net> From: Eugene Grosbein Message-ID: Date: Wed, 12 Apr 2023 10:14:52 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,SHORTCIRCUIT autolearn=disabled version=3.4.6 X-Spam-Report: * -0.0 SHORTCIRCUIT No description available. * -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on hz.grosbein.net X-Rspamd-Queue-Id: 4Px79r00KGz4L9P X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/32, country:DE] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N 11.04.2023 21:28, Lee MATTHEWS wrote: > Thanks for getting back to me Eugene. > > > On the two cores that I've received, they seem to die at the same point : > > > #4 0xffffffff8049a9e3 in panic (fmt=) at ../../../kern/kern_shutdown.c:714 > #5 0xffffffff80780a2b in mca_intr () at ../../../x86/x86/mca.c:1193 > #6 > #7 smp_rendezvous_action () at ../../../kern/subr_smp.c:417 > #8 0xffffffff804e5f79 in smp_rendezvous_cpus (map=..., > setup_func=0xffffffff804e5e40 , > action_func=0xffffffff80496730 , > teardown_func=0xffffffff804e5e40 , arg=0xffffffff80cb5048 ) > at ../../../kern/subr_smp.c:554 > #9 0xffffffff80496639 in _rm_wlock (rm=0xffffffff80cb5048 ) > at ../../../kern/kern_rmlock.c:551 > > Do you think the temperature could still be an issue? If it were temperature related, > could one not expect the MCA interrupt to occur during other function calls? It depends on how much time it spends waiting for events vs. doing other things. From eugen@grosbein.net Wed Apr 12 03:19:24 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Px7H53z0wz456y4 for ; Wed, 12 Apr 2023 03:19:29 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Px7H447qVz4SNW; Wed, 12 Apr 2023 03:19:28 +0000 (UTC) (envelope-from eugen@grosbein.net) Authentication-Results: mx1.freebsd.org; none Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221] (may be forged)) by hz.grosbein.net (8.17.1/8.17.1) with ESMTPS id 33C3JPk4096443 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 12 Apr 2023 03:19:26 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: grahamperrin@freebsd.org Received: from [10.58.0.11] (dadvw [10.58.0.11] (may be forged)) by eg.sd.rdtc.ru (8.16.1/8.16.1) with ESMTPS id 33C3JOVZ011148 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Wed, 12 Apr 2023 10:19:24 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: gpart destroy, in depth To: Graham Perrin , Dmitry Morozovsky References: <64e88636-1e03-7ff5-7d3f-e8a36117cdbe@freebsd.org> <29390cd8-56cd-aced-990a-f2b25a00aaed@freebsd.org> Cc: freebsd-hackers@FreeBSD.org From: Eugene Grosbein Message-ID: <3942716f-076b-6d06-26fe-0d44087ac779@grosbein.net> Date: Wed, 12 Apr 2023 10:19:24 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 In-Reply-To: <29390cd8-56cd-aced-990a-f2b25a00aaed@freebsd.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,SHORTCIRCUIT autolearn=disabled version=3.4.6 X-Spam-Report: * -0.0 SHORTCIRCUIT No description available. * -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on hz.grosbein.net X-Rspamd-Queue-Id: 4Px7H447qVz4SNW X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/32, country:DE] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N 12.04.2023 0:46, Graham Perrin wrote: > On 11/04/2023 14:32, Dmitry Morozovsky wrote: >>> … >> I suppose sysutils/testdisk will be useful for recovering, as most of actual >> file systems are left intact > > > Thanks, I forgot that the utility has this capability. > > This reignites hope of recovery without a third party. > > Re: my question to Warner about preciseness, now I realise, the answer is almost certainly "Yes", because the manual page for gpart(8) notes that the 'restore' argument/command, which does not affect the content of partitions, can be preceded by destruction (-F). > > Also, thanks to the people who responded privately. I did it previously. Just create same partition at same offset and same length and you get file system back. From nobody Wed Apr 12 03:51:53 2023 X-Original-To: hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Px80l3S35z4596f for ; Wed, 12 Apr 2023 03:52:07 +0000 (UTC) (envelope-from crb@chrisbowman.com) Received: from mail-pl1-x629.google.com (mail-pl1-x629.google.com [IPv6:2607:f8b0:4864:20::629]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Px80l1k6vz4F7C for ; Wed, 12 Apr 2023 03:52:07 +0000 (UTC) (envelope-from crb@chrisbowman.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pl1-x629.google.com with SMTP id ke16so9998955plb.6 for ; Tue, 11 Apr 2023 20:52:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chrisbowman-com.20210112.gappssmtp.com; s=20210112; t=1681271525; x=1683863525; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=PFfNPe0pqFOD6Hys9v/+zza1fDI3BcJONHxXUIYXcPI=; b=t0odJ5emFL/aBquog7deLYquoxXwsbEKJjsuG02zQiaGgV8Uq+aE+d11FsEHMtpWj4 CqKfPisCuT71V4QGLWQBBxbhFJz4cYtQ6swH5yUCv4eP+u0+Q8/G1ZCEye1NaxMKFA2F gB3SGOWglRxpOlP4GhtIzqScrKadBAW+MQ2rtT4VKG9nIynLYVkzIwhM+8S2foQzIqxF hqPxBKiiKVn1S+zBo5saDATQHE9LQu/hUcvY1TL4m3hrf0v9QFl8VLtN3xbCqcFdKENw ejkqOsyW1nSs8QFhtQvtwrvhTPPux6+8PsdPF0yUf7AyC3NtY5qsGcUFih5JcRGVtKX1 m+BA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1681271525; x=1683863525; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PFfNPe0pqFOD6Hys9v/+zza1fDI3BcJONHxXUIYXcPI=; b=F1RZhNa7PSp04lGGGUR9nYxrdV/RBxMLFjO82W3Bqj9UdMiZj7pSN4xfsMcfbtD0Ot HaYjIhs/vg0deAjzfmI2e4EvgFSXxm0CsBZ09Pe0h5ye9XzpwamD70fpUeTHLF2xe4/D 8NGxd6Q48Fod91obn6n298LZP3ABclhnDLzlvxwruU1wsPIHzbD9qzpn7OJ1a5VJdLIm aUbnVjoa4ldNz+DwPTvTvNRdDjIcNAHa95RiVy0BYiNaC/zZ9tRwqdxteGchvbuWAcZM qGGyFP+IeVYAGZ3W/r2oJNG53aLRDOsbiufoVVRidBR/k0S8o+9Nyomcr9TyogfJA7W2 /fhw== X-Gm-Message-State: AAQBX9cKS2es5JqKInvXCLtieKjoa5h/NarEnpnOgknntnO3UZIDPh4c yiFzQKAkNXxOw+0DrhHSwAtWow== X-Google-Smtp-Source: AKy350Z+JiFigYPMzd1x4zmMGCCOBEAgxxweg6fayQ5woulwjZsvf0uA32e6CcguaAgn+B873MsBzw== X-Received: by 2002:a17:90b:380f:b0:237:30ef:e593 with SMTP id mq15-20020a17090b380f00b0023730efe593mr19316128pjb.24.1681271524882; Tue, 11 Apr 2023 20:52:04 -0700 (PDT) Received: from smtpclient.apple ([2600:1700:5430:10b1:4ce3:bf44:f929:f0f9]) by smtp.gmail.com with ESMTPSA id o12-20020a1709026b0c00b00199203a4fa3sm10358810plk.203.2023.04.11.20.52.04 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 11 Apr 2023 20:52:04 -0700 (PDT) Content-Type: text/plain; charset=utf-8 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.500.231\)) Subject: Re: make installworld -DNO_ROOT From: Christopher Bowman In-Reply-To: Date: Tue, 11 Apr 2023 20:51:53 -0700 Cc: hackers@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <3E739B8C-9416-43B4-B8C7-6671FF5C028C@chrisbowman.com> References: To: Brooks Davis X-Mailer: Apple Mail (2.3731.500.231) X-Rspamd-Queue-Id: 4Px80l1k6vz4F7C X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N > On Apr 11, 2023, at 7:10 AM, Brooks Davis wrote: >=20 > These warnings are annoying, but mostly harmless. A stalled > and seemingly not quite right review to address the issue is at > https://reviews.freebsd.org/D30990 >=20 > -- Brooks >=20 > On Mon, Apr 10, 2023 at 12:00:45AM -0700, Christopher Bowman wrote: >> Id like to build some images and I???d like do that as an ordinary = user NOT as root >>=20 >> I???m experimenting with ???make installworld -DNO_ROOT = TARGET_ARCH=3Darmv7??? >>=20 >> When I do that or ???make distribution -DNO_ROOT = TARGET_ARCH=3Darmv7??? >>=20 >> I get the following error messages during the make run: >>=20 >> make[4] warning: /etc: Permission denied. >> make[4] warning: /lib: Permission denied. >> make[5] warning: /lib/geom: Permission denied. >> make[4] warning: /libexec: Permission denied. >> make[4] warning: /bin: Permission denied. >> make[4] warning: /rescue: Permission denied. >> make[4] warning: /sbin: Permission denied. >> make[4] warning: /etc: Permission denied. >> make[2] warning: /etc: Permission denied. >> make[3] warning: /etc/gss: Permission denied. >> make[3] warning: /etc/mtree: Permission denied. >> make[3] warning: /lib: Permission denied. >> make[4] warning: /lib/geom: Permission denied. >> make[3] warning: /libexec: Permission denied. >> make[3] warning: /bin: Permission denied. >> make[3] warning: /rescue: Permission denied. >> make[3] warning: /sbin: Permission denied. >> make[3] warning: /etc: Permission denied. >>=20 >> I think as a results either the recursive makes are failing or not = doing all they are supposed to. >>=20 >> It???s possible I???m using -DNO_ROOT wrong but if anyone has some = suggestions about what???s happening that would be very helpful. >>=20 >> In the alternative I???m open to suggestions as to how to debug the = hugs set of make invocations. I find reading and debug make files hard = enough but on huge treee like this with all the invocations and = arguments I???m kind lost as to even which command within the make files = are causing this. >>=20 >> Thanks >> Christopher It=E2=80=99s possible but I=E2=80=99m not so sure. I=E2=80=99m trying to build images and I=E2=80=99m using -DNO_ROOT to = create a file system image without root privileges. Then later I use = makefs to build the filesystems images passing the METALOG file from the = installworld/installkernel/distribution so that the files get the = correct permission in the built filesystem. My METLOG file doesn=E2=80=99t have an entry for /etc/fstab so even = though I had build such a file and place it in my image, it doesn=E2=80=99= t get copied into my filesystem by makefs because there is no METLOG = entry. So I=E2=80=99m of two minds. On the one hand, this is probably = ok for /boot/loader.conf which I add to my image, after all it=E2=80=99s = not strictly necessary that /boot/loader.conf exist in all systems, but = I=E2=80=99m not getting an /etc/fstab even though I hand create the = proper fstab because I have no METALOG entry for it, and this is = absolutely an essential file. What I=E2=80=99m not sure of is how this = is supposed to work and I=E2=80=99d appreciate anyones feedback. 1. I suppose it=E2=80=99s possible that fstab indeed isn=E2=80=99t = created by make distribution and so I should hand create a METALOG entry = for it, but this leaves me wondering what other files are important but = also don=E2=80=99t get copied because they don=E2=80=99t have a METALOG = entry. Perhaps there is a switch to makefs that would warn me of files = in the image without METALOG entries? 2. The /etc/fstab should have a METALOG entry and I=E2=80=99m doing = something wrong, perhaps I=E2=80=99m missing a switch or options = somewhere. In any even I welcome any feedback and thank you for the response = you=E2=80=99ve already sent. Regards, Christopher=09 From nobody Wed Apr 12 14:49:21 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PxQbB1Fc9z44tbM for ; Wed, 12 Apr 2023 14:49:26 +0000 (UTC) (envelope-from steffen@sdaoden.eu) Received: from sdaoden.eu (sdaoden.eu [217.144.132.164]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PxQb91hyHz4Vpg for ; Wed, 12 Apr 2023 14:49:25 +0000 (UTC) (envelope-from steffen@sdaoden.eu) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of steffen@sdaoden.eu designates 217.144.132.164 as permitted sender) smtp.mailfrom=steffen@sdaoden.eu; dmarc=none Date: Wed, 12 Apr 2023 16:49:21 +0200 Author: Steffen Nurpmeso From: Steffen Nurpmeso To: freebsd-hackers@freebsd.org Subject: capsicum(4): .. and SIGTRAP causing syscall really is in siginfo_t.si_errno? Message-ID: <20230412144921.8plun%steffen@sdaoden.eu> User-Agent: s-nail v14.9.24-443-g95b3cc3d07 OpenPGP: id=EE19E1C1F2F7054F8D3954D8308964B51883A0DD; url=https://ftp.sdaoden.eu/steffen.asc; preference=signencrypt BlahBlahBlah: Any stupid boy can crush a beetle. But all the professors in the world can make no bugs. X-Spamd-Result: default: False [2.13 / 15.00]; MID_CONTAINS_FROM(1.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_SPAM_MEDIUM(0.94)[0.937]; NEURAL_HAM_LONG(-0.88)[-0.879]; NEURAL_SPAM_SHORT(0.37)[0.373]; R_SPF_ALLOW(-0.20)[+a]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; R_DKIM_NA(0.00)[]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; FROM_HAS_DN(0.00)[]; ASN(0.00)[asn:15987, ipnet:217.144.128.0/20, country:DE]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DMARC_NA(0.00)[sdaoden.eu]; TO_DN_NONE(0.00)[]; MIME_TRACE(0.00)[0:+] X-Rspamd-Queue-Id: 4PxQb91hyHz4Vpg X-Spamd-Bar: ++ X-ThisMailContainsUnwantedMimeParts: N List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Hello. I am trying to capsicumize a simple daemon (for learning purposes as that runs only in the second line behind postfix), and i have a hard time as that thing is not designed for that (for said reasons). And want to say OpenBSD pledge/unveil was very easy, Linux seccomp required design split with dedicated syslog logger process as C libraries are a block box etc etc. (That is needed for FreeBSD, too, and it keeps the casper out. One would really think "just pack it in a ip netns + unshare + capsh or whatever container, or a jail, and do not do anything regarding such restrictions in a daemon, my code blow is about 30 percent by now.) Anyhow. Regardless of 13.1-i386 or 12.2-amd64 (despite no_new_privs) i only see capsicum(4) violation (syscall 93, 4, 5, 0); please report this bug for sip->si_errno, sip->si_code, sip->si_signo, sip->si_status); Mind you (anything but si_errno a sign of despair), i also saw capsicum(4) violation (syscall 94, 4, 5, 0); please report this bug I only ever saw 93 (and the never-existed-it-seems 94), regardless of whatever syscall was missing still (read(2), the false unlink(2), fsync(2), .. and what not). If only realpath(3->2!) would be accessible, i should have placed the configuration file evaluation in its own process, that would make reloading much easier. But that is my problem, sigh. Not insult desired, just interested $ git show origin/main:sbin | grep /\$ | wc -l 84 $ git grep -lE caph?_enter origin/main -- sbin|wc -l 8 $ git show origin/main:usr.sbin | grep /\$ | wc -l 224 $ git grep -lE caph?_enter origin/main -- usr.sbin|wc -l 10 $ git show origin/main:bin | grep /\$ | wc -l 41 $ git grep -lE caph?_enter origin/main -- bin|wc -l 5 $ git show origin/main:usr.bin | grep /\$ | wc -l 275 $ git grep -lE caph?_enter origin/main -- usr.bin|wc -l 42 to see how hard to put it onto existing code. Luckily i test with that simple thing, so a possibly happening different one can be designed a bit more conforming from scratch. But hey, i read This takes the usual shortcut of only sandboxing the last input file. It's a first cut and this program will be easy to adapt to sandbox all files in the future from a December 2016 commit message, and i like the word "easy". Ciao, --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) From nobody Wed Apr 12 18:36:13 2023 X-Original-To: hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PxWd02Dzwz44WMG for ; Wed, 12 Apr 2023 18:36:20 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: from spindle.one-eyed-alien.net (spindle.one-eyed-alien.net [199.48.129.229]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PxWcz4bVjz4JYS for ; Wed, 12 Apr 2023 18:36:19 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Authentication-Results: mx1.freebsd.org; none Received: by spindle.one-eyed-alien.net (Postfix, from userid 3001) id CE5693C0199; Wed, 12 Apr 2023 18:36:13 +0000 (UTC) Date: Wed, 12 Apr 2023 18:36:13 +0000 From: Brooks Davis To: Christopher Bowman Cc: hackers@freebsd.org Subject: Re: make installworld -DNO_ROOT Message-ID: References: <3E739B8C-9416-43B4-B8C7-6671FF5C028C@chrisbowman.com> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: <3E739B8C-9416-43B4-B8C7-6671FF5C028C@chrisbowman.com> X-Rspamd-Queue-Id: 4PxWcz4bVjz4JYS X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36236, ipnet:199.48.128.0/22, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Tue, Apr 11, 2023 at 08:51:53PM -0700, Christopher Bowman wrote: >=20 > > On Apr 11, 2023, at 7:10 AM, Brooks Davis wrote: > >=20 > > These warnings are annoying, but mostly harmless. A stalled > > and seemingly not quite right review to address the issue is at > > https://reviews.freebsd.org/D30990 > >=20 > > -- Brooks > >=20 > > On Mon, Apr 10, 2023 at 12:00:45AM -0700, Christopher Bowman wrote: > >> Id like to build some images and I???d like do that as an ordinary use= r NOT as root > >>=20 > >> I???m experimenting with ???make installworld -DNO_ROOT TARGET_ARCH=3D= armv7??? > >>=20 > >> When I do that or ???make distribution -DNO_ROOT TARGET_ARCH=3Darmv7?= ?? > >>=20 > >> I get the following error messages during the make run: > >>=20 > >> make[4] warning: /etc: Permission denied. > >> make[4] warning: /lib: Permission denied. > >> make[5] warning: /lib/geom: Permission denied. > >> make[4] warning: /libexec: Permission denied. > >> make[4] warning: /bin: Permission denied. > >> make[4] warning: /rescue: Permission denied. > >> make[4] warning: /sbin: Permission denied. > >> make[4] warning: /etc: Permission denied. > >> make[2] warning: /etc: Permission denied. > >> make[3] warning: /etc/gss: Permission denied. > >> make[3] warning: /etc/mtree: Permission denied. > >> make[3] warning: /lib: Permission denied. > >> make[4] warning: /lib/geom: Permission denied. > >> make[3] warning: /libexec: Permission denied. > >> make[3] warning: /bin: Permission denied. > >> make[3] warning: /rescue: Permission denied. > >> make[3] warning: /sbin: Permission denied. > >> make[3] warning: /etc: Permission denied. > >>=20 > >> I think as a results either the recursive makes are failing or not doi= ng all they are supposed to. > >>=20 > >> It???s possible I???m using -DNO_ROOT wrong but if anyone has some sug= gestions about what???s happening that would be very helpful. > >>=20 > >> In the alternative I???m open to suggestions as to how to debug the hu= gs set of make invocations. I find reading and debug make files hard enoug= h but on huge treee like this with all the invocations and arguments I???m = kind lost as to even which command within the make files are causing this. > >>=20 > >> Thanks > >> Christopher >=20 > It???s possible but I???m not so sure. > I???m trying to build images and I???m using -DNO_ROOT to create a file s= ystem image without root privileges. Then later I use makefs to build the = filesystems images passing the METALOG file from the installworld/installke= rnel/distribution so that the files get the correct permission in the built= filesystem. > My METLOG file doesn???t have an entry for /etc/fstab so even though I ha= d build such a file and place it in my image, it doesn???t get copied into = my filesystem by makefs because there is no METLOG entry. So I???m of two = minds. On the one hand, this is probably ok for /boot/loader.conf which I = add to my image, after all it???s not strictly necessary that /boot/loader.= conf exist in all systems, but I???m not getting an /etc/fstab even though = I hand create the proper fstab because I have no METALOG entry for it, and = this is absolutely an essential file. What I???m not sure of is how this i= s supposed to work and I???d appreciate anyones feedback. >=20 > 1. I suppose it???s possible that fstab indeed isn???t created by make di= stribution and so I should hand create a METALOG entry for it, but this lea= ves me wondering what other files are important but also don???t get copied= because they don???t have a METALOG entry. Perhaps there is a switch to m= akefs that would warn me of files in the image without METALOG entries? /etc/fstab is normally created by the installer. If you want to add one you need to add a METALOG entry (METALOG use is all or nothing). Note that you can choose to store it outside DESTDIR and use contents=3D keyword to specify an absolute path to it if you prefer. I'm not sure if there's an easy way to check for stray files in DESTDIR, probably mtree can do it. -- Brooks >=20 > 2. The /etc/fstab should have a METALOG entry and I???m doing something w= rong, perhaps I???m missing a switch or options somewhere. >=20 > In any even I welcome any feedback and thank you for the response you???v= e already sent. > Regards, > Christopher=09 >=20 From nobody Wed Apr 12 18:59:21 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PxX7q5JBsz44Xvp for ; Wed, 12 Apr 2023 18:59:35 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-lf1-f53.google.com (mail-lf1-f53.google.com [209.85.167.53]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PxX7q3jY6z3RDn for ; Wed, 12 Apr 2023 18:59:35 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-lf1-f53.google.com with SMTP id a23so16517463lfk.4 for ; Wed, 12 Apr 2023 11:59:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681325974; x=1683917974; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=EP1EYp6wOD7iDBv/or/GtVqjg+hBf/sLNi5MRbA9Pt0=; b=gRbCE3BSw5BL2TXmXeRL653Kw632hS26NQAtiNbG/aazF5xRcfqiSiL4MR00IZKuhf ov76aNxlz+86eCjtf930f7tsl9UuUZjPwyQwbdafRagqD3cHWaMLrZcOzO0nQsiZkMtk dFBw/mElakCvgP937uN00BSOcqUhkJsjaD0+/WDxs5CahgxWy1v5LlYK0N0iwV7jVQzq MdpQyacnug9GBgarzRcWcdXpeRtE3uxpkTWpLwcfRjXuN67O0pJT3S7i+ilJTa5F4ZTp GlE/26lzB4c683/dbXTIlIwQYAb8eQ6wxc6FAo/iH2kg6ZH7hwQzJETd+hUSfR193oLg akLA== X-Gm-Message-State: AAQBX9c1U9LdRCZOxNP7LvTO2g1bKAAB+YBwtVaCt5L732VQsPyHbDs2 WFtt21+dkNZpwyhaQ3BbVXZVn3S/rvKHDc3Kpa2uMRfLIcY= X-Google-Smtp-Source: AKy350YbHr1bWlpAiSsHz2SxhSQgv17KgujcU3jZOay+9jZAFUGWjU3sDhri6JxE/5TneDcG3KwFtKaDMgyrh/zN9Zo= X-Received: by 2002:ac2:5681:0:b0:4eb:3f84:8082 with SMTP id 1-20020ac25681000000b004eb3f848082mr4457916lfr.5.1681325973636; Wed, 12 Apr 2023 11:59:33 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: <20230412144921.8plun%steffen@sdaoden.eu> In-Reply-To: <20230412144921.8plun%steffen@sdaoden.eu> From: Ed Maste Date: Wed, 12 Apr 2023 14:59:21 -0400 Message-ID: Subject: Re: capsicum(4): .. and SIGTRAP causing syscall really is in siginfo_t.si_errno? To: Steffen Nurpmeso Cc: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4PxX7q3jY6z3RDn X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Wed, 12 Apr 2023 at 10:49, Steffen Nurpmeso wrote: > > Hello. > > I am trying to capsicumize a simple daemon (for learning purposes > as that runs only in the second line behind postfix), and i have > a hard time as that thing is not designed for that (for said > reasons). And want to say OpenBSD pledge/unveil was very easy, Excellent, always happy to see folks exploring Capsicum. Keep in mind that Capsicum and pledge/unvil are not equivalent, so comparing the ease of applying one or the other isn't particularly meaningful. Achieving similar security properties with pledge/unveil as with Capsicum requires similar effort in decomposing and refactoring existing applications. > Anyhow. Regardless of 13.1-i386 or 12.2-amd64 (despite > no_new_privs) i only see > > capsicum(4) violation (syscall 93, 4, 5, 0); please report this bug I'm not sure what you mean in the subject with respect to the syscall in siginfo_t.si_errno. It looks like this is ENOTCAPABLE, which means an attempt to perform an operation on an fd that you are not allowed to do - for example, calling write() on an fd which has had cap_rights_limit() applied without CAP_WRITE. errno 94 is ECAPMODE. This could be for example trying to use open() in capability mode, which is just not permitted (openat() is). > This takes the usual shortcut of only sandboxing the last input file. > It's a first cut and this program will be easy to adapt to sandbox all > files in the future > > from a December 2016 commit message, and i like the word "easy". cap_fileargs() didn't exist in December 2016 and there was not yet a straightforward, performant and desirable way to apply Capsicum to existing applications that operate on a list of files provided on the commandline. For a more recent change that makes use of cap_fileargs a good example commit is: commit 802c2095b5a6dcf0f63c473cbba1e40445e9052a Author: Mark Johnston Date: Thu Aug 1 18:57:08 2019 +0000 Capsicumize readelf(1). Reviewed by: oshogbo Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D21108 From nobody Wed Apr 12 20:34:38 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PxZFb1ntRz44j2C for ; Wed, 12 Apr 2023 20:34:43 +0000 (UTC) (envelope-from steffen@sdaoden.eu) Received: from sdaoden.eu (sdaoden.eu [217.144.132.164]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PxZFZ4vwZz4MCt; Wed, 12 Apr 2023 20:34:42 +0000 (UTC) (envelope-from steffen@sdaoden.eu) Authentication-Results: mx1.freebsd.org; none Date: Wed, 12 Apr 2023 22:34:38 +0200 Author: Steffen Nurpmeso From: Steffen Nurpmeso To: Ed Maste Cc: freebsd-hackers@freebsd.org Subject: Re: capsicum(4): .. and SIGTRAP causing syscall really is in siginfo_t.si_errno? Message-ID: <20230412203438.IcwD7%steffen@sdaoden.eu> In-Reply-To: References: <20230412144921.8plun%steffen@sdaoden.eu> User-Agent: s-nail v14.9.24-443-g95b3cc3d07 OpenPGP: id=EE19E1C1F2F7054F8D3954D8308964B51883A0DD; url=https://ftp.sdaoden.eu/steffen.asc; preference=signencrypt BlahBlahBlah: Any stupid boy can crush a beetle. But all the professors in the world can make no bugs. X-Rspamd-Queue-Id: 4PxZFZ4vwZz4MCt X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15987, ipnet:217.144.128.0/20, country:DE] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Hello! Ah, oh!! Ed Maste wrote in : |On Wed, 12 Apr 2023 at 10:49, Steffen Nurpmeso wrote: |> I am trying to capsicumize a simple daemon (for learning purposes |> as that runs only in the second line behind postfix), and i have ... |Excellent, always happy to see folks exploring Capsicum. | |Keep in mind that Capsicum and pledge/unvil are not equivalent, so |comparing the ease of applying one or the other isn't particularly |meaningful. Achieving similar security properties with pledge/unveil |as with Capsicum requires similar effort in decomposing and |refactoring existing applications. Luckily not this simple thing. (With unveil together pledge seems pretty good, despite the many system calls i get, and of course the path fixation that does not allow users to add new paths when they reload configurations .. the way the program is designed; i like that new syslog system call which avoids all the things GNU C lib for example does and potentially needs, later maybe more. I think capsicum is very, very smart and capable, like CloudABI was. Yet very hard to work with as it needs so many new *at(), needs to have hooks to modify descriptors after accept(), and openat(), etc. And needs user-path <> realpath(3) mappings .. the way i do it.) As i am very new with this -- am i correct assuming that once a capability was set on a directory or listening socket, opened / accepted FDs inherit the capability of "their parent"? |> Anyhow. Regardless of 13.1-i386 or 12.2-amd64 (despite |> no_new_privs) i only see |> |> capsicum(4) violation (syscall 93, 4, 5, 0); please report this bug | |I'm not sure what you mean in the subject with respect to the syscall |in siginfo_t.si_errno. It looks like this is ENOTCAPABLE, which means This is a misunderstanding!! I *thought* PROC_TRAPCAP_CTL_ENABLE saying "the si_errno member of the siginfo signal handler parameter is set to the syscall error value" meant the actual "syscall number"! And since git head now has that _capsicum._syscall member i thought *that* would now be an explicit thing "to detangle that". It really is an error number! I did not even think about that! So .. the actual syscall number is not available in that siginfo_t before FreeBSD 14? I guess you guys simply write one of those dtrace snippets to get over that. (You know i did not even think, because the Linux seccomp(2) thing i did like that, though there it is SIGSYS and the syscall is in si_syscall. The capsicum(4) and rights(4) etc manuals are complete, but for someone without any real foreknowledge they miss some small hints, here and there. Not that Linux does that better. Or OpenBSD, where you need at least one unveil with "some meat" in order to apply it, even if you simply want no paths at all. .. I think.) |an attempt to perform an operation on an fd that you are not allowed |to do - for example, calling write() on an fd which has had |cap_rights_limit() applied without CAP_WRITE. errno 94 is ECAPMODE. Ah yes! Not a thought on error values. |This could be for example trying to use open() in capability mode, |which is just not permitted (openat() is). Yes. I have had real problems with that, or rather that FDCWD is not possible. (And realpath did cause violations, in at least 12.2, .. though yesterday evening the program was in terrible state on FreeBSD.) |> This takes the usual shortcut of only sandboxing the last input file. |> It's a first cut and this program will be easy to adapt to sandbox \ |> all |> files in the future |> |> from a December 2016 commit message, and i like the word "easy". | |cap_fileargs() didn't exist in December 2016 and there was not yet a |straightforward, performant and desirable way to apply Capsicum to |existing applications that operate on a list of files provided on the |commandline. | |For a more recent change that makes use of cap_fileargs a good example |commit is: | |commit 802c2095b5a6dcf0f63c473cbba1e40445e9052a |Author: Mark Johnston |Date: Thu Aug 1 18:57:08 2019 +0000 | | Capsicumize readelf(1). | | Reviewed by: oshogbo | Sponsored by: The FreeBSD Foundation | Differential Revision: https://reviews.freebsd.org/D21108 I had the impression that casper uses a supervising process. You know, then i thought i better do it myself: this allows the Linux seccomp(2) program for the clients and the server to be streamlined; not only for this small one, where that bystanding process only logs; ie, i simply sliced that into the server, and the server then forks again so that logger actually can synchronize on the server via SIGCLD (etc etc etc), and thus can inherit file locks, naturally, etc etc. --End of Thank you. --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) From nobody Wed Apr 12 20:45:27 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PxZV33Hwtz44jZd for ; Wed, 12 Apr 2023 20:45:31 +0000 (UTC) (envelope-from dumbbell@FreeBSD.org) Received: from smtp-190c.mail.infomaniak.ch (smtp-190c.mail.infomaniak.ch [185.125.25.12]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "relay.mail.infomaniak.ch", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PxZV21gQGz3D9p for ; Wed, 12 Apr 2023 20:45:30 +0000 (UTC) (envelope-from dumbbell@FreeBSD.org) Authentication-Results: mx1.freebsd.org; dkim=none; spf=softfail (mx1.freebsd.org: 185.125.25.12 is neither permitted nor denied by domain of dumbbell@FreeBSD.org) smtp.mailfrom=dumbbell@FreeBSD.org; dmarc=none Received: from smtp-2-0001.mail.infomaniak.ch (unknown [10.5.36.108]) by smtp-3-3000.mail.infomaniak.ch (Postfix) with ESMTPS id 4PxZTz4LLYzMqCDm for ; Wed, 12 Apr 2023 22:45:27 +0200 (CEST) Received: from unknown by smtp-2-0001.mail.infomaniak.ch (Postfix) with ESMTPA id 4PxZTz2NM1zMpsd1 for ; Wed, 12 Apr 2023 22:45:27 +0200 (CEST) Message-ID: <4ed85151-09e8-db3e-0e0b-d0a8f3bb937c@FreeBSD.org> Date: Wed, 12 Apr 2023 22:45:27 +0200 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 To: freebsd-hackers@freebsd.org Content-Language: fr, en-US From: =?UTF-8?Q?Jean-S=c3=a9bastien_P=c3=a9dron?= Subject: Handling panics inside vt(4) callbacks Organization: The FreeBSD Project Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Infomaniak-Routing: alpha X-Spamd-Result: default: False [-2.08 / 15.00]; R_MIXED_CHARSET(1.11)[subject]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.993]; RCVD_IN_DNSWL_LOW(-0.10)[185.125.25.12:from]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[freebsd.org]; FROM_HAS_DN(0.00)[]; FREEFALL_USER(0.00)[dumbbell]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+]; TO_DOM_EQ_FROM_DOM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; HAS_ORG_HEADER(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all:c]; TO_DN_NONE(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; R_DKIM_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:29222, ipnet:185.125.24.0/22, country:CH]; RWL_MAILSPIKE_POSSIBLE(0.00)[185.125.25.12:from] X-Rspamd-Queue-Id: 4PxZV21gQGz3D9p X-Spamd-Bar: -- X-ThisMailContainsUnwantedMimeParts: N Hi! While working on the DRM drivers, I don't always get a kernel core dump in case of a panic. My hypothesis is that if the DRM driver code called by vt(4) panics, then the panic code might not go through successfully. The reason is because panic(9) prints the reason, a stacktrace and possibly some progress to the console, which calls vt(4) and the DRM driver code again. I played with the following patch: https://gist.github.com/dumbbell/88d77789bfeb38869268c84c40575f49 The idea is that before calling "vt_flush()" in "vtterm_done()", I set a global flag to true to indicate that vt(4) is called as part of kdb or a panic. If another panic occurs inside vt_flush(), typically the underlying DRM driver code, "vtterm_done()" is called recursively and "vt_flush()" might trigger the same panic again. If the flag is set, the entire function is skipped instead. I test the patch by adding a panic(9) just before "vt_flush()" and I trigger the initial panic with debug.kdb.panic=1. I don't even load a DRM driver. My problem is that in this case, the laptop reboots immediately. However, if I replace panic(9) with a simple printf(9), it works as expected and I get a kernel dump. I could not find something in panic(9) code that would reboot the computer in case of a nested panic. Previous versions of the patch called doadump() and rebooted the computer explicitly if the flag was set, but it didn't work either and I thought I could simplify that patch and let panic(9) handle recursion. In other words, I just want to skip most of vt(4) code if vt(4) or DRM crash. Does someone spot something wrong in my hypothesis or methodology? -- Jean-Sébastien Pédron The FreeBSD Project From nobody Wed Apr 12 21:33:22 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PxbYV4NLNz44p2G for ; Wed, 12 Apr 2023 21:33:34 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PxbYV3McZz3nxf; Wed, 12 Apr 2023 21:33:34 +0000 (UTC) (envelope-from kevans@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681335214; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4W4+d0gJ4EIHKJJWed4h9OrlVpagUacm6HwLC95+bi8=; b=n8uWBqKzOF1nG0wRthE3y7oSJKwDSOrfk0WxF08b3I0ZAO3UrVK/91E7plWpPgsI8Q0xmC 7QZCB7X23hKuc3xbc230NrkeNv/TsMGokMIBMhE1ydvalNAPaY6qN9C1NmJLIxxKrjGtjn m9ir1FN3lHK8Rh3kNJwSxbhEwYrdSYkooa3BCoITANRR06DIEIEYPefP8htG/hzY1/HBjA nFMn2WNdX1uOwMTo4+ITDPL1BWG5m+QwUG+IvePdnUNMiFcKLOiEb/ny2YEPcPpu+p0X8J SAkzcWbEGCHQdoInuUa3NxpkLkG8ISUgbVjmI1hjiLMsY2IJw3htVWE4A7VC+A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681335214; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4W4+d0gJ4EIHKJJWed4h9OrlVpagUacm6HwLC95+bi8=; b=De+OYmjocNbnraZXJIQoa5WMuxqvOD0llVUOFxdupqrU3PJIgzSIyAcUlFwiv98taMNmXO rzzBVTNSSJbYKP3Q3LRmVFxXUVkRhLesXzmlsUMgBvFL7RLCxKRo5CsIPNDy+0nDa8iqGP os6wcGQ7G1zUZqf97OIW2GK80GPrZxM5wribKx+dHcwH7Gd54YyYyC5J1Xe5ssd9UCGDzN FtM+LohCPC8Qr1DtBbS5JTsw/bAl3MIoObEtxgIUEgjl5oTPvGGAFcSDuF8B7CScPhAelD PZ4ardGWk3/w1ddKgZj8oyY+X6PDlfvqlQDo29ke2ltQBsW+1hHyI+3pxK5LYg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1681335214; a=rsa-sha256; cv=none; b=LPVjlivPtMrHQg2a88Diu+aBanwQpAJZobxF5Vh5f/mOdldKSn0JheGiHZwzDCQYPCCYDq 1XOfhkIC7MCUhLhEd6yELVApOE+cfdIQ1i10Y3xO1CrmqZwcIw3ukZ8tnmYIxdBHrBsod7 K9p7auAK31glVC37/amiwRCH+Ty+FX/MhYQLdAirxnFwFqPgjeuSr4m/FNbrw5shvICYx/ ufzj9JciGQgc5Wcu3Ot8ObyGu8ogGfNZfgDvLTKv8ZYTILvU1IIr6hvz+f/fWNesSQwc/F GXpmfeYa7twmxBws2BeaP0oa4fyeN5k51HAg20ANGatn1YjTY39QMGyNL7YyOQ== Received: from mail-qk1-f173.google.com (mail-qk1-f173.google.com [209.85.222.173]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) (Authenticated sender: kevans) by smtp.freebsd.org (Postfix) with ESMTPSA id 4PxbYV2HZBzNY6; Wed, 12 Apr 2023 21:33:34 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: by mail-qk1-f173.google.com with SMTP id bi39so11503267qkb.13; Wed, 12 Apr 2023 14:33:34 -0700 (PDT) X-Gm-Message-State: AAQBX9dBFFYMZqVEKHBghFQsVAwGr01bSX2DxrIVnS0CZhIQZWLB9KAY Nkujg9d4xKlzHn3ZO+vz3b7SaL4H5WdE+sDVlAE= X-Google-Smtp-Source: AKy350YmhwpZlV++jJBg/LaNFk92ewQJ2ciMYtGTsjoALe7CYMA2WWVQAFxp6xrEFso1AM4HbEUBM0nHYa9BBMi5IPk= X-Received: by 2002:a05:620a:4082:b0:748:5b93:ec65 with SMTP id f2-20020a05620a408200b007485b93ec65mr1333623qko.13.1681335213784; Wed, 12 Apr 2023 14:33:33 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: <4ed85151-09e8-db3e-0e0b-d0a8f3bb937c@FreeBSD.org> In-Reply-To: <4ed85151-09e8-db3e-0e0b-d0a8f3bb937c@FreeBSD.org> From: Kyle Evans Date: Wed, 12 Apr 2023 16:33:22 -0500 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Handling panics inside vt(4) callbacks To: =?UTF-8?B?SmVhbi1Tw6liYXN0aWVuIFDDqWRyb24=?= Cc: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-ThisMailContainsUnwantedMimeParts: N On Wed, Apr 12, 2023 at 3:45=E2=80=AFPM Jean-S=C3=A9bastien P=C3=A9dron wrote: > > Hi! > > While working on the DRM drivers, I don't always get a kernel core dump > in case of a panic. > > My hypothesis is that if the DRM driver code called by vt(4) panics, > then the panic code might not go through successfully. The reason is > because panic(9) prints the reason, a stacktrace and possibly some > progress to the console, which calls vt(4) and the DRM driver code again. > > I played with the following patch: > https://gist.github.com/dumbbell/88d77789bfeb38869268c84c40575f49 > > The idea is that before calling "vt_flush()" in "vtterm_done()", I set a > global flag to true to indicate that vt(4) is called as part of kdb or a > panic. If another panic occurs inside vt_flush(), typically the > underlying DRM driver code, "vtterm_done()" is called recursively and > "vt_flush()" might trigger the same panic again. If the flag is set, the > entire function is skipped instead. > > I test the patch by adding a panic(9) just before "vt_flush()" and I > trigger the initial panic with debug.kdb.panic=3D1. I don't even load a > DRM driver. My problem is that in this case, the laptop reboots > immediately. However, if I replace panic(9) with a simple printf(9), it > works as expected and I get a kernel dump. > > I could not find something in panic(9) code that would reboot the > computer in case of a nested panic. > > Previous versions of the patch called doadump() and rebooted the > computer explicitly if the flag was set, but it didn't work either and I > thought I could simplify that patch and let panic(9) handle recursion. > In other words, I just want to skip most of vt(4) code if vt(4) or DRM > crash. > > Does someone spot something wrong in my hypothesis or methodology? > FWIW, I have a related patch that I maintain in my tree that I simply haven't found time to try and upstream. When the system panics, it tries to switch back to ttyv0, but it calls into vd_postswitch() with the vt lock still held. In my case, with i915kms, the vd_postswitch implementation attempts to sleep with the lock still held and everything goes off the rails. See below. Thanks, Kyle Evans diff --git a/sys/dev/vt/vt_core.c b/sys/dev/vt/vt_core.c index 267dd7bf2489..b57370beae7b 100644 --- a/sys/dev/vt/vt_core.c +++ b/sys/dev/vt/vt_core.c @@ -592,10 +592,10 @@ vt_window_switch(struct vt_window *vw) * switch to console mode when panicking, making sure the p= anic * is readable (even when a GUI was using ttyv0). */ + VT_UNLOCK(vd); if ((kdb_active || KERNEL_PANICKED()) && vd->vd_driver->vd_postswitch) vd->vd_driver->vd_postswitch(vd); - VT_UNLOCK(vd); return (0); } if (!(vw->vw_flags & (VWF_OPENED|VWF_CONSOLE))) { From nobody Thu Apr 13 06:39:52 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Pxqgw0wn4z45Zb3 for ; Thu, 13 Apr 2023 06:39:56 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-ot1-x336.google.com (mail-ot1-x336.google.com [IPv6:2607:f8b0:4864:20::336]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Pxqgv0JbWz3R8l for ; Thu, 13 Apr 2023 06:39:54 +0000 (UTC) (envelope-from mjguzik@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20221208 header.b=n3Fu5oe8; spf=pass (mx1.freebsd.org: domain of mjguzik@gmail.com designates 2607:f8b0:4864:20::336 as permitted sender) smtp.mailfrom=mjguzik@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-ot1-x336.google.com with SMTP id bu14-20020a0568300d0e00b0069fab3f4cafso6679241otb.9 for ; Wed, 12 Apr 2023 23:39:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1681367993; x=1683959993; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=f1VrafUhHvWnBi/zYomClFmJ3MY8KbdB7Ma7xOaMfxs=; b=n3Fu5oe8ras2dV5cuLqJDpeiJ3QLygqDAw8dS/j53pXtmw/d1o/acLs5faANuPl+on DgcWtUty4uPJ7BJQKkhdQ6h1VDk6NOed5Zg71aS1853Kv8GINEWv1WmUhM5QCd/j5jSY rVRxnTtgagqmheinzhg1lWjFsJ12d+IdMMvVA7Bdt+tJb0m1Cbw6MBq/oO8qxKM7LMpj ROG/2YqKGGA574SCd06ACN3BYtjG2sKVJbY8yF8ABv6eTUOBX91covU0dRH6jM4DQU6F KKGAGnL1NK30b2jEojAtOk2yAijwTEdtrIGvnNYPNQOMLDExXFHCQvBKARhJLFq3pWmg J8Cg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681367993; x=1683959993; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=f1VrafUhHvWnBi/zYomClFmJ3MY8KbdB7Ma7xOaMfxs=; b=a6Fy9UivHCoxZc7AmDYLKiJP2wpkPKBboqwE63WRW8lli9UWkXnwXrk0j59+1kZk1y irN6DY/9qGQ60AurRbV/q99Gb0FwsJAnGsfyaoEALotzfHbSDxYbDyyENcEgjQNqZZz/ n3BRyubbERVEFlvGCV5aA8N+E6c/C6D1MGQpGDqgX/aRtQTW/PfYMnSvttpPx84c79dy /Y9gR6YCjz5mQa9l2mud12mipAxaJSzZweS50zDScAPzOIqkz08XAdQqYNRTP5W5aaiy Of/lpGka2cIVbJ+eA2Puu3Q0x+YerNzfs5kP2ZhuqSJVrs0ZgaFptZmWfqkY4smw/slr oVVw== X-Gm-Message-State: AAQBX9fTZVPUSY9s11VlrzBo7heoxBskIyGOctt+q+my6AS8c9zShUOP Gqda17ODwVdZAM0YwzMGGbMUY29lIhANilp5gFHLTFYy X-Google-Smtp-Source: AKy350ZambnEfymz1b7edoRnHGPJNcyiyT7N+pdpYjHEStnNoaNEHpcjdbMAeiA+RDrqFRrynGkSGsAWchPgA/liHQs= X-Received: by 2002:a05:6830:18f6:b0:69f:573:6113 with SMTP id d22-20020a05683018f600b0069f05736113mr251649otf.2.1681367993362; Wed, 12 Apr 2023 23:39:53 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Received: by 2002:a8a:28c:0:b0:49c:b071:b1e3 with HTTP; Wed, 12 Apr 2023 23:39:52 -0700 (PDT) In-Reply-To: References: From: Mateusz Guzik Date: Thu, 13 Apr 2023 08:39:52 +0200 Message-ID: Subject: Re: ULE process to resolution To: "Peter 'PMc' Much" Cc: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20221208]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::336:from]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCPT_COUNT_TWO(0.00)[2]; MID_RHS_MATCH_FROMTLD(0.00)[]; RCVD_TLS_LAST(0.00)[]; TO_DN_SOME(0.00)[]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; FREEMAIL_FROM(0.00)[gmail.com]; RCVD_COUNT_THREE(0.00)[3]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-Rspamd-Queue-Id: 4Pxqgv0JbWz3R8l X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N On 4/5/23, Peter 'PMc' Much wrote: > On 2023-04-04, Mateusz Guzik wrote: >> All that said, I'm buggering off the subject. My interest in it was >> limited to the nice problem, since I have pretty good reasons to >> suspect this is what is causing pathological total real time instances >> for package builds. >> >> Have fun, > > Wow. So, that was Groundhog Day, again. > If this ultimately yields no response, I'm going to cook something up in time for 14.0 -- Mateusz Guzik From nobody Thu Apr 13 07:47:14 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Pxs9p71GCz45gkT for ; Thu, 13 Apr 2023 07:47:26 +0000 (UTC) (envelope-from theraven@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Pxs9p6SW5z3GZj; Thu, 13 Apr 2023 07:47:26 +0000 (UTC) (envelope-from theraven@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681372046; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eL8RwBeCb3i2LS/4U6QTX+KlS163s4l5DCyxGUeAJCc=; b=tkx8+9NTz/Zd6EJmkzHBHvq/OCR0siRrGN2e8F00T0zWY9Ef5T4kQsPcPbpxPbTxzAbgY3 UIQJfLFETvwdyjzq8dufNpX1lP8c96c2QLXkNh4xveOPCsxp0pi+3FwpAZy+msECstAfK5 jF0RniGtMhyu/Fho3sseYRnYnkeGoi6lj0eQgp0yUok8UPqMZxrtKngtik1QcSp5FA8WGn 4QGuyo3VYdXycxST0A62YSKSqfThaZI/xneUJBBVLPIdnb6FsbFFFe47eangeB7hUEbHCr XaSFEM7u+7wCosPsyn16rP6Pws/Di6x/V4U0nyGMGrMrBon2mnHqL/rePCZ8iQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681372046; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eL8RwBeCb3i2LS/4U6QTX+KlS163s4l5DCyxGUeAJCc=; b=kUiMwmC4duFW0wjQMKXgo35AHJrrP5bWiDFYjReSGqu5X1hKSZNvGgEiUXsNYQCiUc/3GM 4KObfE2E2XB5eisd4VrdBXgL3SjAOjxgqrNjHp12uYSNtHsO/2dmuumpUoT6BAfmaoq24I t9/wO5C+nHFZgRKICjBzzVW8Cu7V57gg0L+p9s7cX+BfgDACwBVhFdUA28n5dG0yiw9plW K30ohfbcr3GIkVFTXCpPdf4Gv23VPD2wEIS/oFs9q6kuP+qy5Y0NEhC68E08ezVDkPoYwV YcNa2zIf8HXl63xAEdt94hmms7vE7QaLDLLEaFJLj9fqT3G5uBNA36oOh5Owog== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1681372046; a=rsa-sha256; cv=none; b=HScFzt5hLbqt1YPK4mwGqDVV6L3r/5i1vW/vBFHWWJPPQc04tZuFFspvwF8v138QDl/JWl 1/CJnB0Q9zmnbsQ6bmIS5oJ6i4BLfAxzniZI5GUPImE+M7t3wF66pZsB6ZX4APyZXqDoSD 2cjmu+FVLBjHZV0vkbgYvE1WzmohaWmfRARNeRdNI68anYbcbCat9XSK4xobyJqCHK2Z30 6ke3Y78i0dPuZZgdJaGRWN1dni6NCdRLlM6C5j33oY1s0OKnFC699UHPdZ77fPsXaogdXn GAUtqr6UmrGnL1sFnUGpMqQfyTSfCiUQn4uFcPoRZZWdUMbpjD79/R6MH5lyGg== Received: from smtp.theravensnest.org (smtp.theravensnest.org [45.77.103.195]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: theraven) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Pxs9p5NvQzZ5L; Thu, 13 Apr 2023 07:47:26 +0000 (UTC) (envelope-from theraven@freebsd.org) Received: from smtpclient.apple (host86-162-255-125.range86-162.btcentralplus.com [86.162.255.125]) by smtp.theravensnest.org (Postfix) with ESMTPSA id 1DF2110C28; Thu, 13 Apr 2023 08:47:26 +0100 (BST) Content-Type: multipart/alternative; boundary=Apple-Mail-F52579F3-1153-45FC-AE27-EC214D603731 Content-Transfer-Encoding: 7bit From: David Chisnall List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (1.0) Subject: Re: capsicum(4): .. and SIGTRAP causing syscall really is in siginfo_t.si_errno? Date: Thu, 13 Apr 2023 08:47:14 +0100 Message-Id: References: <20230412203438.IcwD7%steffen@sdaoden.eu> Cc: Ed Maste , freebsd-hackers@freebsd.org In-Reply-To: <20230412203438.IcwD7%steffen@sdaoden.eu> To: Steffen Nurpmeso X-Mailer: iPad Mail (20E252) X-ThisMailContainsUnwantedMimeParts: N --Apple-Mail-F52579F3-1153-45FC-AE27-EC214D603731 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi, I added the siginfo member that passes the system call number (si_syscall). = The problem that it solves is the syscall system call. For normal system ca= lls, you can extract the system call number from the register frame, since i= t will be in rax. Unfortunately, for the syscall system call, this value is c= lobbered and you have no way of usefully recovering it. You might want to take a look at the Verona Sandbox code for inspiration (it= works correctly without si_syscall for all system calls except syscall): https://github.com/microsoft/verona-sandbox This was my project that required this functionality, since it needed to int= ercept system calls and convert them to RPCs. It provides a simple mechanism= for loading a .so in an unprivileged child process and handling all system= calls that touch a global namespace (open, bind, getaddrinfo) via RPC into t= he parent, with some easy-to-use abstractions for filesystem and network acc= ess. It works on Linux with seccomp-bpf and on FreeBSD with Capsicum. The Fre= eBSD version was significantly easier to write for a variety of reasons (Lin= ux doesn=E2=80=99t support strongly aligned allocation in mmap, Linux can=E2= =80=99t kill ld process when the parent process exits, only the parent threa= d, seccomp-bpf policies are amazingly fragile and require an entire library d= ependency to get right). I have a patch under review that adds a SIGCAP as an alternative to SIGTRAP,= which avoids painful interaction with the debugger. I=E2=80=99d love to get= that merged before 14 but haven=E2=80=99t had time to address the last roun= d of review comments. I=E2=80=99ve been running with it locally for a year o= r so. David > On 12 Apr 2023, at 21:35, Steffen Nurpmeso wrote: >=20 > =EF=BB=BFHello! >=20 > Ah, oh!! >=20 > Ed Maste wrote in > : > |On Wed, 12 Apr 2023 at 10:49, Steffen Nurpmeso wrote= : > |> I am trying to capsicumize a simple daemon (for learning purposes > |> as that runs only in the second line behind postfix), and i have > ... > |Excellent, always happy to see folks exploring Capsicum. > | > |Keep in mind that Capsicum and pledge/unvil are not equivalent, so > |comparing the ease of applying one or the other isn't particularly > |meaningful. Achieving similar security properties with pledge/unveil > |as with Capsicum requires similar effort in decomposing and > |refactoring existing applications. >=20 > Luckily not this simple thing. (With unveil together pledge seems > pretty good, despite the many system calls i get, and of course > the path fixation that does not allow users to add new paths when > they reload configurations .. the way the program is designed; > i like that new syslog system call which avoids all the things GNU > C lib for example does and potentially needs, later maybe more. > I think capsicum is very, very smart and capable, like CloudABI > was. Yet very hard to work with as it needs so many new *at(), > needs to have hooks to modify descriptors after accept(), and > openat(), etc. And needs user-path <> realpath(3) mappings .. the > way i do it.) >=20 > As i am very new with this -- am i correct assuming that once > a capability was set on a directory or listening socket, opened > / accepted FDs inherit the capability of "their parent"? >=20 > |> Anyhow. Regardless of 13.1-i386 or 12.2-amd64 (despite > |> no_new_privs) i only see > |> > |> capsicum(4) violation (syscall 93, 4, 5, 0); please report this bug > | > |I'm not sure what you mean in the subject with respect to the syscall > |in siginfo_t.si_errno. It looks like this is ENOTCAPABLE, which means >=20 > This is a misunderstanding!! I *thought* PROC_TRAPCAP_CTL_ENABLE > saying "the si_errno member of the siginfo signal handler > parameter is set to the syscall error value" meant the actual > "syscall number"! And since git head now has that > _capsicum._syscall member i thought *that* would now be an > explicit thing "to detangle that". > It really is an error number! > I did not even think about that! > So .. the actual syscall number is not available in that siginfo_t > before FreeBSD 14? I guess you guys simply write one of those > dtrace snippets to get over that. > (You know i did not even think, because the Linux seccomp(2) thing > i did like that, though there it is SIGSYS and the syscall is in > si_syscall. The capsicum(4) and rights(4) etc manuals are > complete, but for someone without any real foreknowledge they miss > some small hints, here and there. Not that Linux does that > better. Or OpenBSD, where you need at least one unveil with "some > meat" in order to apply it, even if you simply want no paths at > all. .. I think.) >=20 > |an attempt to perform an operation on an fd that you are not allowed > |to do - for example, calling write() on an fd which has had > |cap_rights_limit() applied without CAP_WRITE. errno 94 is ECAPMODE. >=20 > Ah yes! Not a thought on error values. >=20 > |This could be for example trying to use open() in capability mode, > |which is just not permitted (openat() is). >=20 > Yes. I have had real problems with that, or rather that FDCWD is > not possible. (And realpath did cause violations, in at least > 12.2, .. though yesterday evening the program was in terrible > state on FreeBSD.) >=20 > |> This takes the usual shortcut of only sandboxing the last input fil= e. > |> It's a first cut and this program will be easy to adapt to sandbox \= > |> all > |> files in the future > |> > |> from a December 2016 commit message, and i like the word "easy". > | > |cap_fileargs() didn't exist in December 2016 and there was not yet a > |straightforward, performant and desirable way to apply Capsicum to > |existing applications that operate on a list of files provided on the > |commandline. > | > |For a more recent change that makes use of cap_fileargs a good example > |commit is: > | > |commit 802c2095b5a6dcf0f63c473cbba1e40445e9052a > |Author: Mark Johnston > |Date: Thu Aug 1 18:57:08 2019 +0000 > | > | Capsicumize readelf(1). > | > | Reviewed by: oshogbo > | Sponsored by: The FreeBSD Foundation > | Differential Revision: https://reviews.freebsd.org/D21108 >=20 > I had the impression that casper uses a supervising process. You > know, then i thought i better do it myself: this allows the Linux > seccomp(2) program for the clients and the server to be > streamlined; not only for this small one, where that bystanding > process only logs; ie, i simply sliced that into the server, and > the server then forks again so that logger actually can > synchronize on the server via SIGCLD (etc etc etc), and thus can > inherit file locks, naturally, etc etc. >=20 > --End of .com> >=20 > Thank you. >=20 > --steffen > | > |Der Kragenbaer, The moon bear, > |der holt sich munter he cheerfully and one by one > |einen nach dem anderen runter wa.ks himself off > |(By Robert Gernhardt) >=20 --Apple-Mail-F52579F3-1153-45FC-AE27-EC214D603731 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
Hi,=

I added the siginfo member= that passes the system call number (si_syscall).  The problem that it s= olves is the syscall system call. For normal system calls, you can extract t= he system call number from the register frame, since it will be in rax. Unfo= rtunately, for the syscall system call, this value is clobbered and you have= no way of usefully recovering it.

You might want to take a look at the Verona Sandbox code for inspirati= on (it works correctly without si_syscall for all system calls except syscal= l):


This was my project th= at required this functionality, since it needed to intercept system calls an= d convert them to RPCs. It provides a simple mechanism for loading a .so in &= nbsp;an unprivileged child process and handling all system calls that touch a= global namespace (open, bind, getaddrinfo) via RPC into the parent, with so= me easy-to-use abstractions for filesystem and network access. It works on L= inux with seccomp-bpf and on FreeBSD with Capsicum. The FreeBSD version was s= ignificantly easier to write for a variety of reasons (Linux doesn=E2=80=99t= support strongly aligned allocation in mmap, Linux can=E2=80=99t kill ld pr= ocess when the parent process exits, only the parent thread, seccomp-bpf pol= icies are amazingly fragile and require an entire library dependency to get r= ight).

I have a patch under= review that adds a SIGCAP as an alternative to SIGTRAP, which avoids painfu= l interaction with the debugger. I=E2=80=99d love to get that merged before 1= 4 but haven=E2=80=99t had time to address the last round of review comments.= I=E2=80=99ve been running with it locally for a year or so.

David


On 12 Apr 20= 23, at 21:35, Steffen Nurpmeso <steffen@sdaoden.eu> wrote:

=EF=BB=BFHel= lo!

Ah, oh!!

Ed Maste wrote in
<CAPyFy2Do80xZmNFdtG=3DxbRuscKaQQM7= rQ5ir5TVZENX3UfyKtg@mail.gmail.com>:
|On Wed, 12 Apr 202= 3 at 10:49, Steffen Nurpmeso <steffen@sdaoden.eu> wrote:
|> I am trying to capsicumize a simple daemon (for learning purposes<= /span>
|> as that runs only in the second line behind postfix),= and i have
...
|Excellent, always happy t= o see folks exploring Capsicum.
|
|Keep in= mind that Capsicum and pledge/unvil are not equivalent, so
= |comparing the ease of applying one or the other isn't particularly<= br> |meaningful. Achieving similar security properties with pledge/unv= eil
|as with Capsicum requires similar effort in decomposin= g and
|refactoring existing applications.
<= /span>
Luckily not this simple thing.  (With unveil together p= ledge seems
pretty good, despite the many system calls i get= , and of course
the path fixation that does not allow users t= o add new paths when
they reload configurations .. the way t= he program is designed;
i like that new syslog system call w= hich avoids all the things GNU
C lib for example does and po= tentially needs, later maybe more.
I think capsicum is very,= very smart and capable, like CloudABI
was.  Yet very h= ard to work with as it needs so many new *at(),
needs to hav= e hooks to modify descriptors after accept(), and
openat(), e= tc.  And needs user-path <> realpath(3) mappings .. theway i do it.)

As i am very new with t= his -- am i correct assuming that once
a capability was set o= n a directory or listening socket, opened
/ accepted FDs inh= erit the capability of "their parent"?

|&g= t; Anyhow.  Regardless of 13.1-i386 or 12.2-amd64 (despite
|> no_new_privs) i only see
|>
|= >   capsicum(4) violation (syscall 93, 4, 5, 0); please report t= his bug
|
|I'm not sure what you mean in t= he subject with respect to the syscall
|in siginfo_t.si_err= no. It looks like this is ENOTCAPABLE, which means
This is a misunderstanding!!  I *thought* PROC_TRAPCAP_CTL_ENAB= LE
saying "the si_errno member of the siginfo signal handler=
parameter is set to the syscall error value" meant the actu= al
"syscall number"!  And since git head now has that
_capsicum._syscall member i thought *that* would now be an
explicit thing "to detangle that".
It really i= s an error number!
I did not even think about that!So .. the actual syscall number is not available in that siginfo_t
before FreeBSD 14?  I guess you guys simply write one of= those
dtrace snippets to get over that.
(Yo= u know i did not even think, because the Linux seccomp(2) thing
i did like that, though there it is SIGSYS and the syscall is in<= br>si_syscall.  The capsicum(4) and rights(4) etc manuals are
complete, but for someone without any real foreknowledge they m= iss
some small hints, here and there.  Not that Linux d= oes that
better.  Or OpenBSD, where you need at least o= ne unveil with "some
meat" in order to apply it, even if you= simply want no paths at
all.  .. I think.)
<= span>
|an attempt to perform an operation on an fd that you= are not allowed
|to do - for example, calling write() on a= n fd which has had
|cap_rights_limit() applied without CAP_= WRITE. errno 94 is ECAPMODE.

Ah yes!  = Not a thought on error values.

|This could= be for example trying to use open() in capability mode,
|w= hich is just not permitted (openat() is).

Y= es.  I have had real problems with that, or rather that FDCWD is=
not possible.  (And realpath did cause violations, in at leas= t
12.2, .. though yesterday evening the program was in terri= ble
state on FreeBSD.)

|&g= t;     This takes the usual shortcut of only sandboxing t= he last input file.
|>     It's a fi= rst cut and this program will be easy to adapt to sandbox \
= |>     all
|>    =  files in the future
|>
|> from= a December 2016 commit message, and i like the word "easy".
|
|cap_fileargs() didn't exist in December 2016 and there= was not yet a
|straightforward, performant and desirable w= ay to apply Capsicum to
|existing applications that operate= on a list of files provided on the
|commandline. |
|For a more recent change that makes use of cap_f= ileargs a good example
|commit is:
|
|commit 802c2095b5a6dcf0f63c473cbba1e40445e9052a
|Author: Mark Johnston <markj@FreeBSD.org>
|Date: &= nbsp; Thu Aug 1 18:57:08 2019 +0000
|
= |    Capsicumize readelf(1).
|
|    Reviewed by:    oshogbo
|    Sponsored by:   The FreeBSD Foundation
|    Differential Revision:  https://revi= ews.freebsd.org/D21108

I had the impression= that casper uses a supervising process.  You
know, the= n i thought i better do it myself: this allows the Linux
sec= comp(2) program for the clients and the server to be
streaml= ined; not only for this small one, where that bystanding
pro= cess only logs; ie, i simply sliced that into the server, and
the server then forks again so that logger actually can
sy= nchronize on the server via SIGCLD (etc etc etc), and thus can
inherit file locks, naturally, etc etc.

= --End of <CAPyFy2Do80xZmNFdtG=3DxbRuscKaQQM7rQ5ir5TVZENX3UfyKtg@mail.gma= il\
.com>

Thank you.

--steffen
|
|= Der Kragenbaer,           =      The moon bear,
|der holt sich m= unter           he cheerfu= lly and one by one
|einen nach dem anderen runter  wa.k= s himself off
|(By Robert Gernhardt)
=
= --Apple-Mail-F52579F3-1153-45FC-AE27-EC214D603731-- From nobody Thu Apr 13 11:25:36 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Pxy1c5KcJz44pBf; Thu, 13 Apr 2023 11:25:40 +0000 (UTC) (envelope-from egoitz@ramattack.net) Received: from cu1208c.smtpx.saremail.com (cu1208c.smtpx.saremail.com [195.16.148.183]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Pxy1b3YKyz49hW; Thu, 13 Apr 2023 11:25:38 +0000 (UTC) (envelope-from egoitz@ramattack.net) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of egoitz@ramattack.net designates 195.16.148.183 as permitted sender) smtp.mailfrom=egoitz@ramattack.net; dmarc=pass (policy=reject) header.from=ramattack.net Received: from www.saremail.com (unknown [194.30.0.183]) by sieve-smtp-backend02.sarenet.es (Postfix) with ESMTPA id 652DB60C234; Thu, 13 Apr 2023 13:25:36 +0200 (CEST) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="=_88fd3998bafc93f27a15061a2f6d1095" Date: Thu, 13 Apr 2023 13:25:36 +0200 From: egoitz@ramattack.net To: Freebsd fs , Freebsd hackers , freebsd-hardware@freebsd.org Subject: M2 NVME support Message-ID: X-Sender: egoitz@ramattack.net User-Agent: Saremail webmail X-Spamd-Result: default: False [0.70 / 15.00]; NEURAL_SPAM_LONG(1.00)[1.000]; NEURAL_SPAM_SHORT(0.61)[0.605]; DMARC_POLICY_ALLOW(-0.50)[ramattack.net,reject]; R_SPF_ALLOW(-0.20)[+ip4:195.16.148.0/24]; NEURAL_HAM_MEDIUM(-0.11)[-0.115]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; XM_UA_NO_VERSION(0.01)[]; ASN(0.00)[asn:3262, ipnet:195.16.128.0/19, country:ES]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org,freebsd-hackers@freebsd.org,freebsd-hardware@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; R_DKIM_NA(0.00)[]; FROM_NO_DN(0.00)[]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCPT_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Queue-Id: 4Pxy1b3YKyz49hW X-Spamd-Bar: / X-ThisMailContainsUnwantedMimeParts: N --=_88fd3998bafc93f27a15061a2f6d1095 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII Hi!, We are in the process of buying new hardware for use with FreeBSD and ZFS. We are planning whether to buy M2 NVME disks or just SATA SSD disks (probably Samsung PM* ones). How is you experience with them?. Do you recommend one over the another?. Is perhaps better support from some of them from a specificic version to newer?. Or do they perhaps work better with some specific disk controller?. Best regards, --=_88fd3998bafc93f27a15061a2f6d1095 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=UTF-8

Hi!,


We are in the process of buying new hardware for use with FreeBSD and ZF= S. We are planning whether to buy M2 NVME disks or just SATA SSD disks (pro= bably Samsung PM* ones). How is you experience with them?. Do you recommend= one over the another?. Is perhaps better support from some of them from a = specificic version to newer?. Or do they perhaps work better with some spec= ific disk controller?.


Best regards,

--=_88fd3998bafc93f27a15061a2f6d1095-- From nobody Thu Apr 13 11:35:52 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PxyFY3Mrqz44qFM; Thu, 13 Apr 2023 11:36:01 +0000 (UTC) (envelope-from thierry@pompo.net) Received: from edna.lautre.net (edna.lautre.net [80.67.160.88]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "lautre.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PxyFX6zSCz3Dvh; Thu, 13 Apr 2023 11:36:00 +0000 (UTC) (envelope-from thierry@pompo.net) Authentication-Results: mx1.freebsd.org; none Received: from graf.pompo.net (graf.pompo.net [82.66.0.218]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by edna.lautre.net (Postfix) with ESMTPSA id D29FD10B8EE; Thu, 13 Apr 2023 13:35:52 +0200 (CEST) Received: by graf.pompo.net (Postfix, from userid 1001) id 73E0D587ED7; Thu, 13 Apr 2023 13:35:52 +0200 (CEST) Date: Thu, 13 Apr 2023 13:35:52 +0200 From: Thierry Thomas To: egoitz@ramattack.net Cc: Freebsd fs , Freebsd hackers , freebsd-hardware@freebsd.org Subject: Re: M2 NVME support Message-ID: Mail-Followup-To: egoitz@ramattack.net, Freebsd fs , Freebsd hackers , freebsd-hardware@freebsd.org References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="bjt3ILRrrR97TDbB" Content-Disposition: inline In-Reply-To: X-Operating-System: FreeBSD 13.1-STABLE amd64 Organization: Kabbale Eros X-Face: (hRbQnK~Pt7$ct`!fupO(`y_WL4^-Iwn4@ly-.,[4xC4xc;y=\ipKMNm<1J>lv@PP~7Z<.tKjAnXLs: X-PGP: 0xF1C516B3C8359753 X-Rspamd-Queue-Id: 4PxyFX6zSCz3Dvh X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:20766, ipnet:80.67.160.0/19, country:FR] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --bjt3ILRrrR97TDbB Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Le jeu. 13 avr. 23 =E0 13:25:36 +0200, egoitz@ramattack.net =E9crivait=A0: > Hi!,=20 Hello, > We are in the process of buying new hardware for use with FreeBSD and > ZFS. We are planning whether to buy M2 NVME disks or just SATA SSD disks > (probably Samsung PM* ones). How is you experience with them?. Do you > recommend one over the another?. Is perhaps better support from some of > them from a specificic version to newer?. Or do they perhaps work better > with some specific disk controller?.=20 I have a bad experience, the problem seems to be caused by the driver. See . Regards. --=20 Th. Thomas. --bjt3ILRrrR97TDbB Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQJ8BAEBCgBmBQJkN+kYXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRFNTM2QkU4NTM4NTM5OUQwMEI2RkFBNzZG MUM1MTZCM0M4MzU5NzUzAAoJEPHFFrPINZdTYWIP/3zxEIk+TVYh+rpMaFGBm7fh BrkPAOrKVfdfVuwpqQAaxwkHX5C5ISSSg8QMY+DrzpwEjX0wzDhgDaZnG3f1Glen 14e/yHxyCY1R5QzzM5C2aZV+I/tI+AVWRgCaHVsNVT1+u5yE4XDyUP5Qosb6EaR1 4QTYD8nxHUkyAH31GcshLcnJVBbLSVPRkoAoZOrs0a5r2J8E8plkwnwrF6R7t+L8 rZ4Iw9PuZi6LIjkkwqRYQjGFQkJ8oLkRQBMKvGltnwXhhumsMrFe4UEkD3hC+rzd XDUyMhFPrPL9LNS3uikPYuowTJGxmhBxYIuwmrd3p/B2vi2McdQxJ+oH82CfgPzk Kq+yjnUap28mjrl0MhI3AkjoPXQsXHTJ8JHta1BhWxJdCeU1sGtbcUsvJOWk789F ih4FW31HfEduNg9GakUIGnGLyxb3pzYc1nCZ/oYSrwKDIboOFqehMauqFyWdr+Ts Iouxv8pwReKa/J54E27lZBuRUlNsv8pXcfMJ75qH0Fpg5crFmJLyEs00a7EyOTSH hOYY1ZCyXnh2R5uJMsUbthGr+bk7qeP/UxDXAERwagdp3T2/VJEnqZmZVviouKq5 LTWe3m0FjWUqXX5IDuDav5EshGKuKDnWvy6io4w7mu2DLkKqnArenIjQLTp8rco9 J92LBA+tgAvlDtGZ1FY/ =OBnS -----END PGP SIGNATURE----- --bjt3ILRrrR97TDbB-- From nobody Thu Apr 13 11:43:32 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PxyQK6zZ6z44qmB; Thu, 13 Apr 2023 11:43:37 +0000 (UTC) (envelope-from pi@freebsd.org) Received: from fc.opsec.eu (fc.opsec.eu [IPv6:2001:14f8:200:4::4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PxyQK5LT7z3kPv; Thu, 13 Apr 2023 11:43:37 +0000 (UTC) (envelope-from pi@freebsd.org) Authentication-Results: mx1.freebsd.org; none Received: from pi by fc.opsec.eu with local (Exim 4.96 (FreeBSD)) (envelope-from ) id 1pmvMS-000Hca-19; Thu, 13 Apr 2023 13:43:32 +0200 Date: Thu, 13 Apr 2023 13:43:32 +0200 From: Kurt Jaeger To: egoitz@ramattack.net Cc: Freebsd fs , Freebsd hackers , freebsd-hardware@freebsd.org Subject: Re: M2 NVME support Message-ID: References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 4PxyQK5LT7z3kPv X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:12502, ipnet:2001:14f8::/32, country:DE] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N Hi! > We are in the process of buying new hardware for use with FreeBSD and > ZFS. We are planning whether to buy M2 NVME disks or just SATA SSD disks > (probably Samsung PM* ones). How is you experience with them? One box with two NVMe (my packet-builder host), running CURRENT. Model Number: Samsung SSD 970 PRO 1TB Works fine, no problems. I have not measured throughput, but I guess NVMe is faster than SSD, due to the SATA bottleneck etc. Several boxes with SSD Sata (mostly Samsung SSD 850/960 and some with Intel S4510/S4610/S4500/S4600 Series SSDs), all work fine. There were issues with Samsung SSDs and AMD EPYC 3251 CPUs. That's when we brought in the Intel SSDs... > Do you recommend one over the another?. Is perhaps better support from > some of them from a specificic version to newer?. > Or do they perhaps work better > with some specific disk controller?. -- pi@FreeBSD.org +49 171 3101372 Now what ? From nobody Thu Apr 13 11:47:27 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PxyW159Ybz44rJF for ; Thu, 13 Apr 2023 11:47:41 +0000 (UTC) (envelope-from tomek@cedro.info) Received: from mail-yw1-x1136.google.com (mail-yw1-x1136.google.com [IPv6:2607:f8b0:4864:20::1136]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PxyW03hx4z3s7l for ; Thu, 13 Apr 2023 11:47:40 +0000 (UTC) (envelope-from tomek@cedro.info) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=cedro.info header.s=google header.b=HHrJ7BBg; spf=none (mx1.freebsd.org: domain of tomek@cedro.info has no SPF policy when checking 2607:f8b0:4864:20::1136) smtp.mailfrom=tomek@cedro.info; dmarc=none Received: by mail-yw1-x1136.google.com with SMTP id 00721157ae682-54fb615ac3dso63460687b3.2 for ; Thu, 13 Apr 2023 04:47:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cedro.info; s=google; t=1681386459; x=1683978459; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :from:to:cc:subject:date:message-id:reply-to; bh=ihV8HoluZxy2lJxSKGH+0a12aS1o68Y3MTJ5s8iMGn0=; b=HHrJ7BBg99esj1DT16PrqRZzuarehUbnwKEIUJRJ/xXY/rc36cx7tgElUAR5IGnt/8 TfU3hcjGlL2L+CN9a4q/0i2PCMpBfgnj0Bs/+cOB4pnan4eN8kHQhnpLYgceGPkBIzwW 9ukX1CZjvSnl85XtZvSkNSZwpgBWMaEIhllnqF01FDzyFAEoW7D9qUKbsn6rE/YxnOJg C2KVc609ZV8PSDzEk6Vr4lMSIiI3gLrF3NkT1WMp9Miz1Hl43KioMiWg01RlwhGyXULg Uq+EdTwQ1jEsbgIjaI/79N3+SDxUmtdbNgNw4HEj59m++x4BnP5LKM80VttBTb8bJJU7 zyCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681386459; x=1683978459; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ihV8HoluZxy2lJxSKGH+0a12aS1o68Y3MTJ5s8iMGn0=; b=GpvsMFdKKXrHPBDLZvPd9WgTUOnHScavLcxViDMQ3pUvkBo1s2HWmiQv9TtCMupHPz X9aB2uR2cZWOeayJCP4v+Y7pjAd5CHj35sUAWictmf9EmOfOWiqiB9LuvaFe3WtNdk4S WeUD3V+OwYqxKR/5+Ih45InWMrO2xfH+O5oNdGTFqCms5XerMDfProR2tOxgLQlzweDt VGE2frNnZJj4A9/ElERtyWtifu9HZlCXMXwCnufBfvEImR2hAViyWTG9UCCXg6ybiEgs ZeTylCzK7Bdt4y2g6ypgO6sCiUhbHQcinV6S/lt6/BhKsd9fSebuzf7Ypr9kfa/VaSwl Jyaw== X-Gm-Message-State: AAQBX9eTMhrncCDYntxthr2AbDw+jXJtqIJyTRcFQxj24inWssVgu6S1 1e0SIsjl+HLARcvxQJ8ukhGmvA== X-Google-Smtp-Source: AKy350a2qAO0klQPL8G1rib7DRhxjGdmGzKD3BmQzsz4OzXyEz3qZ/uWwQGeSgp9d1BpNa9v9kbbmA== X-Received: by 2002:a0d:d982:0:b0:54f:8b2b:adec with SMTP id b124-20020a0dd982000000b0054f8b2badecmr1625943ywe.33.1681386459735; Thu, 13 Apr 2023 04:47:39 -0700 (PDT) Received: from mail-yb1-f173.google.com (mail-yb1-f173.google.com. [209.85.219.173]) by smtp.gmail.com with ESMTPSA id 141-20020a810393000000b0054fba955474sm401373ywd.17.2023.04.13.04.47.39 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 13 Apr 2023 04:47:39 -0700 (PDT) Received: by mail-yb1-f173.google.com with SMTP id h198so21486974ybg.12; Thu, 13 Apr 2023 04:47:39 -0700 (PDT) X-Received: by 2002:a25:77cb:0:b0:b21:5fb4:c6e6 with SMTP id s194-20020a2577cb000000b00b215fb4c6e6mr985861ybc.11.1681386458841; Thu, 13 Apr 2023 04:47:38 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Tomek CEDRO Date: Thu, 13 Apr 2023 13:47:27 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: M2 NVME support To: egoitz@ramattack.net, Freebsd fs , Freebsd hackers , freebsd-hardware@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Spamd-Result: default: False [-3.30 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[cedro.info:s=google]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::1136:from,209.85.219.173:received]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[cedro.info:+]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_LAST(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; ARC_NA(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; FROM_HAS_DN(0.00)[]; DMARC_NA(0.00)[cedro.info]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4PxyW03hx4z3s7l X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N Sorry, I missed the initial post. I am using Samsung 980 1TB NVME M2 SSD on a ICY BOX PCI-E 4.0x4 1xM.2 NVMe with ZFS as my root pool, works like a charm on old 2013 mainboard where only SATA is available.. just note that maximum chipset efficiency will be the bottleneck but with a good chipset it may be faster than SATA :-) -- CeDeROM, SQ7MHZ, http://www.tomek.cedro.info From nobody Thu Apr 13 12:56:21 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Py02L6Q5bz44xWb; Thu, 13 Apr 2023 12:56:26 +0000 (UTC) (envelope-from ml@netfence.it) Received: from soth.netfence.it (mailserver.netfence.it [78.134.96.152]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mailserver.netfence.it", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Py02L3F9Bz3PDP; Thu, 13 Apr 2023 12:56:26 +0000 (UTC) (envelope-from ml@netfence.it) Authentication-Results: mx1.freebsd.org; none Received: from [10.1.2.18] (mailserver.netfence.it [78.134.96.152]) (authenticated bits=0) by soth.netfence.it (8.17.1/8.17.1) with ESMTPSA id 33DCuLgA090917 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO); Thu, 13 Apr 2023 14:56:21 +0200 (CEST) (envelope-from ml@netfence.it) X-Authentication-Warning: soth.netfence.it: Host mailserver.netfence.it [78.134.96.152] claimed to be [10.1.2.18] Message-ID: Date: Thu, 13 Apr 2023 14:56:21 +0200 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: M2 NVME support Content-Language: en-US To: egoitz@ramattack.net References: Cc: Freebsd fs , freebsd-hardware@freebsd.org, Freebsd hackers From: Andrea Venturoli In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.86 X-Rspamd-Queue-Id: 4Py02L3F9Bz3PDP X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:35612, ipnet:78.134.0.0/17, country:IT] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On 4/13/23 13:25, egoitz@ramattack.net wrote: > We are in the process of buying new hardware for use with FreeBSD and > ZFS. We are planning whether to buy M2 NVME disks or just SATA SSD disks > (probably Samsung PM* ones). How is you experience with them? I've never had any trouble with any SATA disks. As for M.2, I only have a RAID-1 root pool on one box: # nvmecontrol devlist nvme0: SPCC M.2 PCIe SSD nvme0ns1 (244198MB) nvme1: SPCC M.2 PCIe SSD nvme1ns1 (244198MB) So far, so good, but it's quite new. > Do you recommend one over the another? M.2 should be a lot faster than SATA. A good SATA RAID might be fast enough, however, depending on your needs. In the low-end market I would go with M.2, provided you have two slots on the MB. In the high-end, I'm not so sure... SATA enterprise SSDs are available, M.2, AFAIK, not. The new U.2 standard looks promising, but I'm not so sure it will catch on, while SATA will eventually disappear, but not any time soon. > Or do they perhaps work better with some specific disk controller?. You mean NVMe? There is really no controller (as it is intended in SATA/SCSI/SAS/...), they just appear on the PCI bus. bye av. From nobody Thu Apr 13 15:18:51 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Py3Bm41hfz45GDb for ; Thu, 13 Apr 2023 15:18:56 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-qt1-x833.google.com (mail-qt1-x833.google.com [IPv6:2607:f8b0:4864:20::833]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Py3Bm10kbz4bKh; Thu, 13 Apr 2023 15:18:56 +0000 (UTC) (envelope-from markjdb@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-qt1-x833.google.com with SMTP id l16so3649618qtv.1; Thu, 13 Apr 2023 08:18:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1681399134; x=1683991134; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:sender :from:to:cc:subject:date:message-id:reply-to; bh=cXjtt2fUtoQ9Kxr2xpE89qU6ZlyZDLaFXW3xDYps9qE=; b=fAUodTfCKs/t7+yaelkZZuvHLchN4iYoYqP9y0eVUC/MehO8ZRzgHStCNrsMkC998/ CnDbXDN0RMwIWfOQInImFxPK00S3SKOk6t7k2BtGOyHF/X/x3umGJESkc+vP9NDUR/m+ QhpLckmi//Y0Df6CIDNPo6X6E07UbkaHNoVUOSZbO5k3Oy8k1bXc4ikHJkevOI8OaOUb 3PBrHqBYH+nZY2trPnyFsf12pMKnATfjXEhVJWRII76N82BRRbMT6pwXY3zeHktph+iU zpucNzlGmanR/YXSG+Ibtziel5QShavshDBjXmqAEjCXXPj/ajERScQHIrXHmFZsHQsg 8+VA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681399134; x=1683991134; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:sender :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=cXjtt2fUtoQ9Kxr2xpE89qU6ZlyZDLaFXW3xDYps9qE=; b=ROhpMdBzb7ZZS0J5ns3yJFxu3U5lZ73KKf11E2eVYBSduyr6WQVwj5VM37l7usO1VI 6fJWiSfaHJ72RA942Eau0RMz/CGOowGqfcFN0tPJUC10cXJl1AnlCv1bvXEsRlM/hr2W OtbBSIZmcb6FD0+ZwKCkm9eRRX26U9iPYpPH1Lq/vC9dPth1Uw72nw5kGo/GyEp/96us 36M5Ry0XqaLSC5n3QVvRqMKNFnW1z2zAwHE2WMEvYElednavFmiiBLVKlfOT7HDWah+I wH8UVg4SJo1BBfrJq6HvDdg3nyyygNxsoXwUylC0EuV8yvSv2NLSeMlXoSrZw364/CWb Zyjg== X-Gm-Message-State: AAQBX9d2oX/zUxcDUduy3jcfItFaguezX9COCZh0YnzCl4aODqufmcm4 sp8dgUank4k14/d8b0Mpm5paxREt7js= X-Google-Smtp-Source: AKy350aXVNQLITmWgGCni7wXqDeBFmFk4MreQiTAO3D6rCvacdm3kwTCw5LkmX2EXCj0C7964GqsYg== X-Received: by 2002:ac8:5b53:0:b0:3db:786f:d91a with SMTP id n19-20020ac85b53000000b003db786fd91amr2655672qtw.57.1681399134611; Thu, 13 Apr 2023 08:18:54 -0700 (PDT) Received: from nuc (192-0-220-237.cpe.teksavvy.com. [192.0.220.237]) by smtp.gmail.com with ESMTPSA id dw20-20020a05620a601400b0074269db4699sm541833qkb.46.2023.04.13.08.18.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Apr 2023 08:18:53 -0700 (PDT) Date: Thu, 13 Apr 2023 11:18:51 -0400 From: Mark Johnston To: =?iso-8859-1?Q?Jean-S=E9bastien_P=E9dron?= Cc: freebsd-hackers@freebsd.org Subject: Re: Handling panics inside vt(4) callbacks Message-ID: References: <4ed85151-09e8-db3e-0e0b-d0a8f3bb937c@FreeBSD.org> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <4ed85151-09e8-db3e-0e0b-d0a8f3bb937c@FreeBSD.org> X-Rspamd-Queue-Id: 4Py3Bm10kbz4bKh X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Wed, Apr 12, 2023 at 10:45:27PM +0200, Jean-Sébastien Pédron wrote: > Hi! > > While working on the DRM drivers, I don't always get a kernel core dump > in case of a panic. > > My hypothesis is that if the DRM driver code called by vt(4) panics, > then the panic code might not go through successfully. The reason is > because panic(9) prints the reason, a stacktrace and possibly some > progress to the console, which calls vt(4) and the DRM driver code again. > > I played with the following patch: > https://gist.github.com/dumbbell/88d77789bfeb38869268c84c40575f49 > > The idea is that before calling "vt_flush()" in "vtterm_done()", I set a > global flag to true to indicate that vt(4) is called as part of kdb or a > panic. If another panic occurs inside vt_flush(), typically the > underlying DRM driver code, "vtterm_done()" is called recursively and > "vt_flush()" might trigger the same panic again. If the flag is set, the > entire function is skipped instead. > > I test the patch by adding a panic(9) just before "vt_flush()" and I > trigger the initial panic with debug.kdb.panic=1. I don't even load a > DRM driver. My problem is that in this case, the laptop reboots > immediately. However, if I replace panic(9) with a simple printf(9), it > works as expected and I get a kernel dump. > > I could not find something in panic(9) code that would reboot the > computer in case of a nested panic. In the case of a nested panic, vpanic() will not set RB_DUMP when it calls kern_reboot(), so it won't write a kernel dump. And, if debug.debugger_on_recursive_panic is not set, the kernel will not try to re-enter the debugger. So the kernel will simply reboot. > Previous versions of the patch called doadump() and rebooted the > computer explicitly if the flag was set, but it didn't work either and I > thought I could simplify that patch and let panic(9) handle recursion. > In other words, I just want to skip most of vt(4) code if vt(4) or DRM > crash. Perhaps we should set RB_DUMP in the case of a recursive panic so long as dumping == 0, i.e., we did not panic again while trying to dump core. In fact, kern_reboot() already checks this. > Does someone spot something wrong in my hypothesis or methodology? > > -- > Jean-Sébastien Pédron > The FreeBSD Project > From nobody Thu Apr 13 16:43:31 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Py54c3MrYz45NXK; Thu, 13 Apr 2023 16:43:44 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-yw1-x1130.google.com (mail-yw1-x1130.google.com [IPv6:2607:f8b0:4864:20::1130]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Py54b5DQqz3vW6; Thu, 13 Apr 2023 16:43:43 +0000 (UTC) (envelope-from fjwcash@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20221208 header.b=puBahsTb; spf=pass (mx1.freebsd.org: domain of fjwcash@gmail.com designates 2607:f8b0:4864:20::1130 as permitted sender) smtp.mailfrom=fjwcash@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-yw1-x1130.google.com with SMTP id 00721157ae682-54c12009c30so400506777b3.9; Thu, 13 Apr 2023 09:43:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1681404222; x=1683996222; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :from:to:cc:subject:date:message-id:reply-to; bh=n3qu6bzZzuNavcQ7l7EKMS7ut+Xkc+eGIrL5iAOrC6U=; b=puBahsTb/jRUh30TcPePpEnWlVKZ/+188FMqPipO4PhkFlpuE5rZ7COC6VzVGRps7P gV0b5lAQDAz+ILzX8603Wwnlah1+iS4ih40ajwf/X6dWIXCqmuz6vqVNk+wLAA5OPw+D x8DTxQGlWbsLhTi6/Vkbln6BYQ3LoijB2Pjw605v1Q1IQJmJyBt6PPldoKQxUwei4oq0 7w3CM2gfSRgyA/i1s70GKcTT1aL+rC/fuAQMLFxvbdl6iMNX/+y2v6CPMH23A01yVvmW woNno1XMX0D9HLxuCuxe5V9/atlb8pLxRoH0plV7Nju5LIqt8TLbim3MN38j0YHhS2El W95w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681404222; x=1683996222; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=n3qu6bzZzuNavcQ7l7EKMS7ut+Xkc+eGIrL5iAOrC6U=; b=dNDoDJTj37BopyVOXFLVjVgSSEfxMrVtG9eo2DUUiVw1m+iI73eyJ4MDUb9A38kBGJ Qo21zKN0B/ZrHH8UdTTs5EmlvN1WlNGdlB9UAcAD5YNvHANh28UM/SJZvWXVW+LIHqMT h04Ofpp0fBRFTNtWWwAyuT3/k4XtelGIT6XnJxbrbIoxlVMvIr7+rRO9Uzw488KGrV/V HPd+6KaPMkSO27KIk80NJJ5Ohn5NJ/37JOX1dH70eqbRQXoB3gtiMZcCijNfpyQYB0pA l7TU3K6ge0MZdvoykGHsgjlME5qeD3RJw/xDrBOIL2ukGjDycYAGOZ5qQvCSWGnJYcrX 8b2A== X-Gm-Message-State: AAQBX9dBRJgsxa2M9Oc9Vo+dr3C5vZfEKVWwFjQRn86eFIrt40josvB/ pW73ZRdtP7yJ/H7ym86+HsNcoOm5Fvb0ZVM8g/gBNr90L8E= X-Google-Smtp-Source: AKy350YMS14mw2UmkHALhmbiHU9DrSMvyqXwLk+lmfAGqfOhKKQMGK7HOsozUdJnjsySSfU1hnP+upqnmHNO1xuI4ZQ= X-Received: by 2002:a05:690c:b85:b0:54f:6aa5:7c4f with SMTP id ck5-20020a05690c0b8500b0054f6aa57c4fmr5199296ywb.3.1681404222603; Thu, 13 Apr 2023 09:43:42 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Freddie Cash Date: Thu, 13 Apr 2023 09:43:31 -0700 Message-ID: Subject: Re: M2 NVME support To: egoitz@ramattack.net, Freebsd fs , Freebsd hackers , freebsd-hardware@freebsd.org Content-Type: multipart/alternative; boundary="00000000000069924a05f93a6d46" X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20221208]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org,freebsd-hackers@freebsd.org,freebsd-hardware@freebsd.org]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::1130:from]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-Rspamd-Queue-Id: 4Py54b5DQqz3vW6 X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N --00000000000069924a05f93a6d46 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable > > Le jeu. 13 avr. 23 =C3=A0 13:25:36 +0200, egoitz@ramattack.net < > egoitz@ramattack.net> > > We are in the process of buying new hardware for use with FreeBSD and > > ZFS. We are planning whether to buy M2 NVME disks or just SATA SSD disk= s > > (probably Samsung PM* ones). How is you experience with them?. Do you > > recommend one over the another?. Is perhaps better support from some of > > them from a specificic version to newer?. Or do they perhaps work bette= r > > with some specific disk controller?. > There were issues in the past where NVMe drives were "too fast" for ZFS, and various bottlenecks were uncovered. Most (all?) of those have been fixed over the past couple years. These issues were found on pools using all NVMe drives in various configurations for data storage (multiple raidz vdevs; multiple mirror vdevs). This was back when PCIe 3.0 NVMe drives were all the rage, or maybe when PCIe 4.0 drives first started appearing? If you're running a recent release of FreeBSD (13.x) with the newer versions of OpenZFS 2.x, then you shouldn't have any issues using NVMe drives. The hard part will be finding drives with MLC or 3D TLC NAND chips in multiple channels, with a large SLC cache, and lots of RAM onboard using good controllers, in order to get consistent, strong performance during writes. Especially when the drive is near full. Too many drives are moving to QLC NAND, or using DRAM-less controllers (using system RAM as a buffer) in order to reduce the cost. You'll want to do your research into the technology used on the drive before buying any specific drive. SATA SSDs will perform better than hard drives, but will be limited by the SATA bus to around 550 MBps of read/write throughput. NVMe drives will provide multiple GBps of read/write throughput (depending on the drive and PCIe bus version). Finding a motherboard that supports more than 2 M.2 slots will be very hard. If you want more than 2 drives, you'll have to look into PCIe add-in boards with M.2 slots. Really expensive ones will include PCIe switches onboard so they'll work in pretty much any motherboard with spare x16 slots (and maybe x8 slots, with reduced performance?). Less expensive add-in boards require PCIe bifurcation support in the BIOS, and will only work in specific slots on the motherboard. My home ZFS server uses an ASUS motherboard with PCIe bifurcation support, has an ASUS Hyper M.2 expansion card in the second PCIe x16 slot, with 2 WD Blue M.2 SSDs installed (card supports 4 M.2 drives). These are used to create a root pool using a single mirror vdev. /, /usr, and /var are mounted from there. There's 6x hard drives in a separate data pool using multiple mirror vdevs, with /home mounted from there (this pool has been migrated from IDE drives to SATA, from FreeBSD to Linux, and from raidz to mirror vdevs at various points in the past, without losing any data so far; yay ZFS!). At work, all our ZFS servers use 2.5" SATA SSDs for the root pool, and for separate L2ARC/SLOG devices, with 24-90 SATA hard drives for the storage pool. These are all running FreeBSD 13.x. If you want the best performance, and money isn't a restriction, then you'll want to look into servers that have U.2 (or whatever the next-gen small form factor interface name is) slots and backplanes. The drives cost a lot more than regular M.2 SSDs, but provide a lot more performance. Especially in AMD EPYC servers with 128 PCIe lanes to play with. :) --=20 Freddie Cash fjwcash@gmail.com --00000000000069924a05f93a6d46 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Le jeu. 13 avr. 23 =C3=A0 13:25:36 +0200, egoitz@ramattack.net <egoitz@ramattack.ne= t>
> We are in the process of buying new hardware for use with= FreeBSD and
> ZFS. We are planning whether to buy M2 NVME disks or just SATA SSD dis= ks
> (probably Samsung PM* ones). How is you experience with them?. Do you<= br> > recommend one over the another?. Is perhaps better support from some o= f
> them from a specificic version to newer?. Or do they perhaps work bett= er
> with some specific disk controller?.

There were issues in the past where NVMe drives were "too fast"= ; for ZFS, and various bottlenecks were uncovered.=C2=A0 Most (all?) of tho= se have been fixed over the past couple years.=C2=A0 These issues were foun= d on pools using all NVMe drives in various configurations for data storage= (multiple raidz vdevs; multiple mirror vdevs).=C2=A0 This was back when PC= Ie 3.0 NVMe drives were all the rage, or maybe when PCIe 4.0 drives first s= tarted appearing?

If you're running a recent r= elease of FreeBSD (13.x) with the newer versions of OpenZFS 2.x, then you s= houldn't have any issues using NVMe drives.=C2=A0 The hard part will be= finding drives with MLC or 3D TLC NAND chips in multiple channels, with a = large SLC cache, and lots of RAM onboard using good controllers, in order t= o get consistent, strong performance during writes.=C2=A0 Especially when t= he drive is near full.=C2=A0 Too many drives are moving to QLC NAND, or usi= ng DRAM-less controllers (using system RAM as a buffer) in order to reduce = the cost.=C2=A0 You'll want to do your research into the technology use= d on the drive before buying any specific drive.

S= ATA SSDs will perform better than hard drives, but will be limited by the S= ATA bus to around 550 MBps of read/write throughput.=C2=A0 NVMe drives will= provide multiple GBps of read/write throughput (depending on the drive and= PCIe bus version).=C2=A0 Finding a motherboard that supports more than 2 M= .2 slots will be very hard.=C2=A0 If you want more than 2 drives, you'l= l have to look into PCIe add-in boards with M.2 slots.=C2=A0 Really expensi= ve ones will include PCIe switches onboard so they'll work in pretty mu= ch any motherboard with spare x16 slots (and maybe x8 slots, with reduced p= erformance?).=C2=A0 Less expensive add-in boards require PCIe bifurcation s= upport in the BIOS, and will only work in specific slots on the motherboard= .

My home ZFS server uses an ASUS motherboard with= PCIe bifurcation support, has an ASUS Hyper M.2 expansion card in the seco= nd PCIe x16 slot, with 2 WD Blue M.2 SSDs installed (card supports 4 M.2 dr= ives).=C2=A0 These are used to create a root pool using a single mirror vde= v.=C2=A0 /, /usr, and /var are mounted from there.=C2=A0 There's 6x har= d drives in a separate data pool using multiple mirror vdevs, with /home mo= unted from there (this pool has been migrated from IDE drives to SATA, from= FreeBSD to Linux, and from raidz to mirror vdevs at various points in the = past, without losing any data so far; yay ZFS!).

A= t work, all our ZFS servers use 2.5" SATA SSDs for the root pool, and = for separate L2ARC/SLOG devices, with 24-90 SATA hard drives for the storag= e pool.=C2=A0 These are all running FreeBSD 13.x.

= If you want the best performance, and money isn't a restriction, then y= ou'll want to look into servers that have U.2 (or whatever the next-gen= small form factor interface name is) slots and backplanes.=C2=A0 The drive= s=C2=A0cost a lot more than regular M.2 SSDs, but provide a lot more perfor= mance.=C2=A0 Especially in AMD EPYC servers with 128 PCIe lanes to play wit= h.=C2=A0 :)

--
Freddie Cash
= fjwcash@gmail.com
--00000000000069924a05f93a6d46-- From nobody Thu Apr 13 18:19:10 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Py7C02tv5z45ZTm for ; Thu, 13 Apr 2023 18:19:24 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-lf1-f54.google.com (mail-lf1-f54.google.com [209.85.167.54]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Py7C00t81z4N2J for ; Thu, 13 Apr 2023 18:19:24 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-lf1-f54.google.com with SMTP id o1so20349307lfc.2 for ; Thu, 13 Apr 2023 11:19:24 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681409962; x=1684001962; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=evzTCIHQx/DLt0yGx5q62S8Br0DvUO15UnJzkpWBUlI=; b=Bvmkw3ZgAqkBaBu1lkGH2m05ZW1UCRD8XwlyBuWjPfG2QrEzNzFuyMhKmGK5D+O5Zz iDbDuzPHC0EWZFHDwvxuuNgAAJHg0gz/2byFmdHAmM8BXy58nKfqt0KkhHVZT2zujcrT cgV+kCL0lWxzbpKjlOqU6893qNswpW/A5+1gL4K6rXIcl+3YxH/IC9Tc5Bvs+ngSmHbX FbcPoSCe3EKD8Se197JwX9XfTxYLhna85Ay5RJYh16+/PTKCo0Je1gQm5se1YD7FUNUq nuZAWfgJQStOER9bWqdp+cNyONqu2Zln/uQa2WUB6JS1l9g9lRMrXQSP3fWuyB1KDwGI x1Dg== X-Gm-Message-State: AAQBX9eqmcIpCcVM/0xN3ahhGXw/two9zLPVt7wOGl6qPDtsg5U9L6eS 1UZkTREqRUsNCoGAbc62lEc9LhYCkQFtso86Pi6Z2TWjJ5M= X-Google-Smtp-Source: AKy350Z1ponsA8b6BTh0azWIct7cYXbcoLF5L22teJCnvOkzizzOakakpGZSFCn3VLVSlsdMyxlCuSL7T5o7Ae6yGSE= X-Received: by 2002:ac2:522e:0:b0:4eb:a8c:5f22 with SMTP id i14-20020ac2522e000000b004eb0a8c5f22mr1048940lfl.5.1681409962038; Thu, 13 Apr 2023 11:19:22 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: <20230412144921.8plun%steffen@sdaoden.eu> <20230412203438.IcwD7%steffen@sdaoden.eu> In-Reply-To: <20230412203438.IcwD7%steffen@sdaoden.eu> From: Ed Maste Date: Thu, 13 Apr 2023 14:19:10 -0400 Message-ID: Subject: Re: capsicum(4): .. and SIGTRAP causing syscall really is in siginfo_t.si_errno? To: Steffen Nurpmeso Cc: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4Py7C00t81z4N2J X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Wed, 12 Apr 2023 at 16:34, Steffen Nurpmeso wrote: > > I think capsicum is very, very smart and capable, like CloudABI > was. Yet very hard to work with as it needs so many new *at(), > needs to have hooks to modify descriptors after accept(), and > openat(), etc. And needs user-path <> realpath(3) mappings .. the > way i do it.) Indeed. These limitations all stem from the properties that give Capsicum its security properties (no access to global namespaces or ambient authority). It is "relatively" straightforward to use when incorporated into new software from the beginning, but certainly harder to retrofit. realpath is an interesting case, as in capability mode there is no absolute root directory. All paths are necessarily relative to a directory fd. The Casper fileargs service provides a realpath replacement, but I've also started looking into what a Capsicum-native realpathat would be. > As i am very new with this -- am i correct assuming that once > a capability was set on a directory or listening socket, opened > / accepted FDs inherit the capability of "their parent"? In general, yes. Rights can be removed or subsetted but not added. > The capsicum(4) and rights(4) etc manuals are > complete, but for someone without any real foreknowledge they miss > some small hints, here and there. Not that Linux does that > better. Or OpenBSD, where you need at least one unveil with "some > meat" in order to apply it, even if you simply want no paths at > all. .. I think.) I think capsicum(4) in particular could use more information, but man pages are always going to be closer to reference material. The initial Capsicum paper is a good read to understand Capsicum's design and approaches that can be taken to refactor existing software: https://www.cl.cam.ac.uk/research/security/capsicum/papers/2010usenix-security-capsicum-website.pdf > I had the impression that casper uses a supervising process. I wouldn't describe it as a "supervising process", but Casper does indeed involve additional process(es). In essence Casper services are performing the same work that an explicitly privilege-separated process would do. With cap_fileargs for example you have one process (that is not operating in capability mode) with access to the filesystem (global namespace). That process opens files, and passes file descriptors to the capability-mode process that performs the work. From nobody Thu Apr 13 19:26:25 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Py8ht4TrZz45h6R for ; Thu, 13 Apr 2023 19:26:54 +0000 (UTC) (envelope-from steffen@sdaoden.eu) Received: from sdaoden.eu (sdaoden.eu [217.144.132.164]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Py8ht1Gphz44mS; Thu, 13 Apr 2023 19:26:54 +0000 (UTC) (envelope-from steffen@sdaoden.eu) Authentication-Results: mx1.freebsd.org; none Date: Thu, 13 Apr 2023 21:26:25 +0200 Author: Steffen Nurpmeso From: Steffen Nurpmeso To: David Chisnall Cc: Ed Maste , freebsd-hackers@freebsd.org Subject: Re: capsicum(4): .. and SIGTRAP causing syscall really is in siginfo_t.si_errno? Message-ID: <20230413192625.mUQ_T%steffen@sdaoden.eu> In-Reply-To: References: <20230412203438.IcwD7%steffen@sdaoden.eu> User-Agent: s-nail v14.9.24-443-gb34bfaf135 OpenPGP: id=EE19E1C1F2F7054F8D3954D8308964B51883A0DD; url=https://ftp.sdaoden.eu/steffen.asc; preference=signencrypt BlahBlahBlah: Any stupid boy can crush a beetle. But all the professors in the world can make no bugs. List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4Py8ht1Gphz44mS X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15987, ipnet:217.144.128.0/20, country:DE] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N Hello. David Chisnall wrote in : |I added the siginfo member that passes the system call number (si_syscal\ |l). The problem that it solves is the syscall system call. For normal \ |system calls, you can extract the system call number from the register \ |frame, since it will be in rax. Unfortunately, for the syscall system \ |call, this value is clobbered and you have no way of usefully recovering \ |it. I am too amd64 bound for sure. Well, to be honest, i had written a test-strace test target which runs the entire machinery (hmm..), and then generates the list of necessary system calls for the client and the server. It, however, includes all system calls, including all the ugly pre-sandbox things. I am in the lucky position not to run debuggers, as well as being too stupid to handle them, anyway (step and stepi, and break, that is all i know). But good that someone did that. (Having said that i wrote that test target after i already had started the seccomp(2) implementation, and the SIGSYS thing is a regular build target, that i use.) |You might want to take a look at the Verona Sandbox code for inspiration \ |(it works correctly without si_syscall for all system calls except \ |syscall): | |https://github.com/microsoft/verona-sandbox | |This was my project that required this functionality, since it needed \ |to intercept system calls and convert them to RPCs. It provides a simple \ |mechanism for loading a .so in an unprivileged child process and handlin= g \ |all system calls that touch a global namespace (open, bind, getaddrinfo) \ |via RPC into the parent, with some easy-to-use abstractions for filesyste= m \ |and network access. It works on Linux with seccomp-bpf and on FreeBSD \ |with Capsicum. The FreeBSD version was significantly easier to write \ |for a variety of reasons (Linux doesn=E2=80=99t support strongly aligned = alloc\ |ation in mmap, Linux can=E2=80=99t kill ld process when the parent proces= s \ |exits, only the parent thread, seccomp-bpf policies are amazingly fragile= \ |and require an entire library dependency to get right). This sounds like a very impressive project, especially compared to my little and primitive thing. BPF for seccomp(2) seems to be very different than what the new epbf is capable to do; I watched a LWN-linked presentation on what BPF can do "some years" ago, with live modification / tracing / inspection of the kernel etc. (But *i* dreamed of "a syscall bitset in front" (like capsicum seems to have), and then executable snippets to do the rest, including checks against real in-use descriptors, as opposed to only compile-time constants. Or complicated runtime program generation. And then, running a program for any systemcall is tough.) I think capsicum is likely the smartest thing and so nicely reflects the UNIX "everything is a file". But really, my setup for my simple client/server is tremendous(ly complicated). I see from looking that the FreeBSD kernel now supports realpathat(2), yet not for users ([main] as of 03-31). And this would be so really important to have! I mean, i can evaluate configuration in a/the "super-capable" base process, and then simply fork off a new server which then inherits the new configuration (after the old has been told to die), but that is a real mess. Also because, you know, so i opened a directory FD for / (the way i do it: do this, use realpath(3) on all paths, and then simply openat(2) "rootfd,&[1]" to not openat(2) an absolute path..), but this is only for the sandboxed process. So if someone would mount some filesystem over / (i presumed that is the reason why AT_FDCWD and plain open(2) and openat(2) with absolute paths are forbidden), then this will affect the "super-capable" process which reloads the configuration and from which the new sandboxed server instance is spawned. That does not make sense. I could open the / descriptor already in that process, on the other hand; hmm. But still ugly. So my thinking would be that there *must* be a realpathat(2) so that the capsicum(4)ized server can simply reload the configuration itself, while allowing the user full flexibility. (My current approach is rather identical to what OpenBSDs unveil(2) thing ends up with, ... yet relative to the opened / file descriptor, of course. Because .. what else could i do? So users have to use the _very same_ file names, or the thing fails. realpath(3) cannot be used. I need to implement some purely string-only path canonicalization to make this a bit better. Lesser files the user may use, but new ones not at all.) |I have a patch under review that adds a SIGCAP as an alternative to \ |SIGTRAP, which avoids painful interaction with the debugger. I=E2=80=99d = love \ |to get that merged before 14 but haven=E2=80=99t had time to address the = last \ |round of review comments. I=E2=80=99ve been running with it locally for a= year \ |or so. So good luck for get this going! --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) From nobody Thu Apr 13 21:20:16 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PyCCp2VSnz44fQY; Thu, 13 Apr 2023 21:20:22 +0000 (UTC) (envelope-from yuri@aetern.org) Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PyCCp0K7Pz442h; Thu, 13 Apr 2023 21:20:22 +0000 (UTC) (envelope-from yuri@aetern.org) Authentication-Results: mx1.freebsd.org; none Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id 8EE845C0163; Thu, 13 Apr 2023 17:20:20 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Thu, 13 Apr 2023 17:20:20 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=aetern.org; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1681420820; x=1681507220; bh=dcAci4eppcE6CtbW52vyUfa1M4EBtQjHWmC zKLRJlhs=; b=WEkk00xmzGXDgTUVzt7FbrRLaHmspC6RXbRW4OQFxqX7sEPjfk+ eDgfEpgsf+01aQBuiewFN9h6kWRklc6SdQYCsplKPuu78HAnnJ9HQ33zoTnJuvYY NBRhCpW7NnK/F+UqHDZE/9MJjW3pHTJeN1TeidL/uUU7lx4aD2G8tEgpDKhyeyGg qC34Xx2IOMOO0m7XLtCJBOQXTp4lq6yYYtHaJwh3uxjpwLt4I9Jl+gxcW7j+Qd0z ROhtit5VYDhcrWwZXC084rVks6dDZWI+KxXIH/HSUIC2B3MfYOtTmAWVObOoBOd4 O1dm6cB9DbyRMbd5qj055a2/4VwspLr6+Cg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1681420820; x=1681507220; bh=dcAci4eppcE6CtbW52vyUfa1M4EBtQjHWmC zKLRJlhs=; b=DQ26q9YwM2lPGd/JL6kPKlw2wlxaHxyY9Re0Xb3NtjRVZgERrQr 9y3pWGEexWC0f6y8CRpTLK046r5a/vpdnm99OLBpxq8748NF+mIkjIawd+XdxmPu FZe6YNf+T8WPpwpBo7WN90x5RU5qfvYZKxatC74BWYxY8o/MshceQbgQoNCB+4QJ BnsM+yZtLpZ2S4kSY984mjy8t5Fj4D0SanOR/uonIegYfUUv2S2pv3Lai78pfiE9 yuhOKUcd8EvM9J1c3MD+Jhzs8NNIXpm4ltEJfBO75z2wlDTtl5bD7DmjQqAEPjoT G2EdWu+roELDZXl20blvf+10kSnGIWxRzNA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrvdekkedgudeivdcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefkffggfgfuvfevfhfhjggtgfesthekredttdefjeenucfhrhhomhepjghu rhhiuceohihurhhisegrvghtvghrnhdrohhrgheqnecuggftrfgrthhtvghrnhepvdektd fhkeeuueevleefudethffgkeeigfetveduieeggfejueegheffjeefgffhnecuffhomhgr ihhnpehfrhgvvggsshgurdhorhhgnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrg hmpehmrghilhhfrhhomhephihurhhisegrvghtvghrnhdrohhrgh X-ME-Proxy: Feedback-ID: i0d79475b:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 13 Apr 2023 17:20:18 -0400 (EDT) Message-ID: <26febe50-a854-08df-13f9-d1629436c1f3@aetern.org> Date: Thu, 13 Apr 2023 23:20:16 +0200 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.9.1 Subject: Re: M2 NVME support Content-Language: en-US To: andy thomas , Thierry Thomas Cc: egoitz@ramattack.net, Freebsd fs , Freebsd hackers , freebsd-hardware@freebsd.org References: From: Yuri In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4PyCCp0K7Pz442h X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N andy thomas wrote: > On Thu, 13 Apr 2023, Thierry Thomas wrote: > >> Le jeu. 13 avr. 23 ? 13:25:36 +0200, egoitz@ramattack.net >> >> ?crivait : >> >>> Hi!, >> >> Hello, >> >>> We are in the process of buying new hardware for use with FreeBSD and >>> ZFS. We are planning whether to buy M2 NVME disks or just SATA SSD disks >>> (probably Samsung PM* ones). How is you experience with them?. Do you >>> recommend one over the another?. Is perhaps better support from some of >>> them from a specificic version to newer?. Or do they perhaps work better >>> with some specific disk controller?. >> >> I have a bad experience, the problem seems to be caused by the driver. >> >> See . > > I thought I would mention I have been using a WD Blue SA510 250GB M2 > NVME SSD mounted on a PCIe to NVME adapter card under FBSD 13.1 for some > months now with no problems or any special drivers being installed. Only it looks like it's actually SATA and not NVMe, based on the output? > The server is a Dell PowerEdge 430 1U server fitted with four SATA disks > in a ZFS pool, with a 4 GB partition on each disk being used for the > swap (since using SSDs for swap space can wear them out quite quickly). > > Attached is a screenshot showing FreeBSD version, the NVME SSD type, > mounted SSD partitions, the server make & model and some info about the > spinning disks too (there is a known issue with the built-in Megaraid > driver in the FreeBSD 13.1 kernel not working with the camcontrol > utility nor converting disk device names like /dev/mfisyspd0 to > traditional names such as /dev/ada0 but these controllers & disks do > work with FBSD 13.1 as you cna see from the zpool output). From nobody Thu Apr 13 20:41:17 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PyCKH5hLlz44f7j for ; Thu, 13 Apr 2023 21:25:07 +0000 (UTC) (envelope-from steffen@sdaoden.eu) Received: from sdaoden.eu (sdaoden.eu [217.144.132.164]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PyCKH10TPz4GFc; Thu, 13 Apr 2023 21:25:06 +0000 (UTC) (envelope-from steffen@sdaoden.eu) Authentication-Results: mx1.freebsd.org; none Date: Thu, 13 Apr 2023 22:41:17 +0200 Author: Steffen Nurpmeso From: Steffen Nurpmeso To: Ed Maste Cc: freebsd-hackers@freebsd.org Subject: Re: capsicum(4): .. and SIGTRAP causing syscall really is in siginfo_t.si_errno? Message-ID: <20230413204117.T906W%steffen@sdaoden.eu> In-Reply-To: References: <20230412144921.8plun%steffen@sdaoden.eu> <20230412203438.IcwD7%steffen@sdaoden.eu> User-Agent: s-nail v14.9.24-443-gb34bfaf135 OpenPGP: id=EE19E1C1F2F7054F8D3954D8308964B51883A0DD; url=https://ftp.sdaoden.eu/steffen.asc; preference=signencrypt BlahBlahBlah: Any stupid boy can crush a beetle. But all the professors in the world can make no bugs. X-Rspamd-Queue-Id: 4PyCKH10TPz4GFc X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15987, ipnet:217.144.128.0/20, country:DE] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Hello. Ed Maste wrote in : |On Wed, 12 Apr 2023 at 16:34, Steffen Nurpmeso wrote: |> I think capsicum is very, very smart and capable, like CloudABI |> was. Yet very hard to work with as it needs so many new *at(), |> needs to have hooks to modify descriptors after accept(), and |> openat(), etc. And needs user-path <> realpath(3) mappings .. the |> way i do it.) | |Indeed. These limitations all stem from the properties that give |Capsicum its security properties (no access to global namespaces or |ambient authority). It is "relatively" straightforward to use when |incorporated into new software from the beginning, but certainly |harder to retrofit. | |realpath is an interesting case, as in capability mode there is no |absolute root directory. All paths are necessarily relative to a |directory fd. The Casper fileargs service provides a realpath |replacement, but I've also started looking into what a Capsicum-native |realpathat would be. Yes! I do not know how it can work without actually, like i had written in the other email. I mean, if the service shall be able to re-evaluate its configuration at runtime, with users having the full flexibility of simply giving a path (in my case files with allow or block lists, and resource-file chains), then it must be able to realpath since openat(2) does not allow relative paths nor absolute paths. And .. outsourcing that "realpath" thing (as you say), or performing the entire re-configuration outside the sandbox, seems a target for those attacks that the capsicum(2) designers had in mind when designing it that way. realpathat(2) would be great. |> As i am very new with this -- am i correct assuming that once |> a capability was set on a directory or listening socket, opened |> / accepted FDs inherit the capability of "their parent"? | |In general, yes. Rights can be removed or subsetted but not added. Thank you, i had tested that a bit yesterday after the thing did run, by removing rights from the root FD, .. and yes it worked. |> The capsicum(4) and rights(4) etc manuals are |> complete, but for someone without any real foreknowledge they miss |> some small hints, here and there. Not that Linux does that |> better. Or OpenBSD, where you need at least one unveil with "some |> meat" in order to apply it, even if you simply want no paths at |> all. .. I think.) | |I think capsicum(4) in particular could use more information, but man |pages are always going to be closer to reference material. The initial |Capsicum paper is a good read to understand Capsicum's design and |approaches that can be taken to refactor existing software: |https://www.cl.cam.ac.uk/research/security/capsicum/papers/2010usenix-se\ |curity-capsicum-website.pdf 2010 even, it came to my mind seeing all these 2013 copyrights, .. but the anniversary passed aleady. 'Will download it when i am online again. |> I had the impression that casper uses a supervising process. | |I wouldn't describe it as a "supervising process", but Casper does |indeed involve additional process(es). In essence Casper services are |performing the same work that an explicitly privilege-separated |process would do. With cap_fileargs for example you have one process |(that is not operating in capability mode) with access to the |filesystem (global namespace). That process opens files, and passes |file descriptors to the capability-mode process that performs the |work. Yes. Ok here it is actually supervising since i need some easy and portable way to ensure gracefully the logger process terminates when the actual server process terminates. (I mean, without an additional communication pipe or something. The logger simply does blocking reads, and then blocking writes. Also, like this, he shares the file lock with the actual server, so no shutdown/startup synchronization problems ever can occur, portable and easily. You know .. even though it now has the special --startup, --status, --shutdown modes, the original idea was, and that is how i personally use it, that one simply asks it a question by starting the client, and the client automatically starts the single instance server if that is not running, and that configurably times out after so and so much time without any client.. etc etc etc. That made me painfully realize that unfortunately UNIX domain sockets fail their bind(2) with EADDRINUSE if the path exists, even though if there is no active server bound to them, actually requiring that additional reassurance lock file the descriptor of which now is shared, as above.) I wanted to avoid such very expensive things, like your casper example, for such a small thing. I mean, before the sandboxing, this secondary server took about 800 KB RAM and consumed 0.00 seconds CPU time for over a week of work. I hate that on Linux and FreeBSD log messages now go X->OS->logger process out of sandbox->OS->syslogd->OS and what do i know. But i never grasped such also for sound processing, i still worship the FreeBSD sound device muxer from about twenty years ago, i could record music with about ~1.5 percent CPU usage on a Cyrix 166+, and easily play music as some frequency and format conversions were built into the kernel, simplemost and cheap user space work to get this done. (While doing work and compile sessions, without stutter etc, on a single processor box with slow hard disk!) --End of Thank you, and Ciao! --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) From nobody Thu Apr 13 22:10:55 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PyDLG5vSTz44kfx for ; Thu, 13 Apr 2023 22:11:02 +0000 (UTC) (envelope-from dumbbell@FreeBSD.org) Received: from smtp-bc08.mail.infomaniak.ch (smtp-bc08.mail.infomaniak.ch [IPv6:2001:1600:4:17::bc08]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "relay.mail.infomaniak.ch", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PyDLG0sVJz3H4s for ; Thu, 13 Apr 2023 22:11:02 +0000 (UTC) (envelope-from dumbbell@FreeBSD.org) Authentication-Results: mx1.freebsd.org; dkim=none; spf=softfail (mx1.freebsd.org: 2001:1600:4:17::bc08 is neither permitted nor denied by domain of dumbbell@FreeBSD.org) smtp.mailfrom=dumbbell@FreeBSD.org; dmarc=none Received: from smtp-2-0000.mail.infomaniak.ch (unknown [10.5.36.107]) by smtp-3-3000.mail.infomaniak.ch (Postfix) with ESMTPS id 4PyDL80KbHzMqP1C for ; Fri, 14 Apr 2023 00:10:56 +0200 (CEST) Received: from unknown by smtp-2-0000.mail.infomaniak.ch (Postfix) with ESMTPA id 4PyDL75fJwzMppDV for ; Fri, 14 Apr 2023 00:10:55 +0200 (CEST) Message-ID: Date: Fri, 14 Apr 2023 00:10:55 +0200 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: Handling panics inside vt(4) callbacks To: freebsd-hackers@freebsd.org References: <4ed85151-09e8-db3e-0e0b-d0a8f3bb937c@FreeBSD.org> Content-Language: fr, en-US From: =?UTF-8?Q?Jean-S=c3=a9bastien_P=c3=a9dron?= Organization: The FreeBSD Project In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Infomaniak-Routing: alpha X-Spamd-Result: default: False [-1.39 / 15.00]; R_MIXED_CHARSET(1.00)[subject]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.29)[-0.290]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[freebsd.org]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; ASN(0.00)[asn:29222, ipnet:2001:1600::/32, country:CH]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; HAS_ORG_HEADER(0.00)[]; FROM_HAS_DN(0.00)[]; FREEFALL_USER(0.00)[dumbbell]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; R_SPF_SOFTFAIL(0.00)[~all:c]; TO_DOM_EQ_FROM_DOM(0.00)[] X-Rspamd-Queue-Id: 4PyDLG0sVJz3H4s X-Spamd-Bar: - X-ThisMailContainsUnwantedMimeParts: N On 13/04/2023 17:18, Mark Johnston wrote: > On Wed, Apr 12, 2023 at 10:45:27PM +0200, Jean-Sébastien Pédron wrote: >> I could not find something in panic(9) code that would reboot the >> computer in case of a nested panic. > > In the case of a nested panic, vpanic() will not set RB_DUMP when it > calls kern_reboot(), so it won't write a kernel dump. And, if > debug.debugger_on_recursive_panic is not set, the kernel will not try to > re-enter the debugger. So the kernel will simply reboot. That was exactly the problem and I missed it even though I read that function many times... Thank you very much! If I always set RB_DUMP, it does what I want in the context of that forced nested panic. I will continue to test with the DRM driver and the actual issues I'm currently chasing. Once done, I will prepare a patch. > Perhaps we should set RB_DUMP in the case of a recursive panic so long > as dumping == 0, i.e., we did not panic again while trying to dump core. > In fact, kern_reboot() already checks this. In fact, "kern_reboot()" already verify if "dumping" is non-zero. I think "vpanic()" could set the flag regardless of that variable. Thank you! -- Jean-Sébastien Pédron The FreeBSD Project From nobody Thu Apr 13 22:12:54 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PyDNV2Y9Kz44lGr for ; Thu, 13 Apr 2023 22:12:58 +0000 (UTC) (envelope-from dumbbell@FreeBSD.org) Received: from smtp-42ab.mail.infomaniak.ch (smtp-42ab.mail.infomaniak.ch [84.16.66.171]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "relay.mail.infomaniak.ch", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PyDNT1sWWz3M0k for ; Thu, 13 Apr 2023 22:12:57 +0000 (UTC) (envelope-from dumbbell@FreeBSD.org) Authentication-Results: mx1.freebsd.org; dkim=none; spf=softfail (mx1.freebsd.org: 84.16.66.171 is neither permitted nor denied by domain of dumbbell@FreeBSD.org) smtp.mailfrom=dumbbell@FreeBSD.org; dmarc=none Received: from smtp-2-0001.mail.infomaniak.ch (unknown [10.5.36.108]) by smtp-2-3000.mail.infomaniak.ch (Postfix) with ESMTPS id 4PyDNQ6bp6zMsHFd for ; Fri, 14 Apr 2023 00:12:54 +0200 (CEST) Received: from unknown by smtp-2-0001.mail.infomaniak.ch (Postfix) with ESMTPA id 4PyDNQ4cFnzMpx5T for ; Fri, 14 Apr 2023 00:12:54 +0200 (CEST) Message-ID: Date: Fri, 14 Apr 2023 00:12:54 +0200 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: Handling panics inside vt(4) callbacks Content-Language: fr, en-US To: freebsd-hackers@freebsd.org References: <4ed85151-09e8-db3e-0e0b-d0a8f3bb937c@FreeBSD.org> From: =?UTF-8?Q?Jean-S=c3=a9bastien_P=c3=a9dron?= Organization: The FreeBSD Project In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Infomaniak-Routing: alpha X-Spamd-Result: default: False [-1.11 / 15.00]; R_MIXED_CHARSET(1.00)[subject]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-0.93)[-0.930]; RCVD_IN_DNSWL_LOW(-0.10)[84.16.66.171:from]; MIME_GOOD(-0.10)[text/plain]; NEURAL_SPAM_SHORT(0.02)[0.020]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; FREEFALL_USER(0.00)[dumbbell]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_ONE(0.00)[1]; TO_DOM_EQ_FROM_DOM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; HAS_ORG_HEADER(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; R_SPF_SOFTFAIL(0.00)[~all:c]; TO_DN_NONE(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:29222, ipnet:84.16.64.0/19, country:CH]; R_DKIM_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[84.16.66.171:from] X-Rspamd-Queue-Id: 4PyDNT1sWWz3M0k X-Spamd-Bar: - X-ThisMailContainsUnwantedMimeParts: N On 12/04/2023 23:33, Kyle Evans wrote: > FWIW, I have a related patch that I maintain in my tree that I simply > haven't found time to try and upstream. When the system panics, it > tries to switch back to ttyv0, but it calls into vd_postswitch() with > the vt lock still held. In my case, with i915kms, the vd_postswitch > implementation attempts to sleep with the lock still held and > everything goes off the rails. See below. Indeed, there are several locking issues in the DRM driver callbacks related to vt(4) because of the different contraints in Linux. I'm currently working on revisiting the way we integrate the DRM drivers in vt(4), hopefully this work will address the problem you hit. -- Jean-Sébastien Pédron The FreeBSD Project From nobody Fri Apr 14 06:07:20 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PyQw354dwz44Wy5; Fri, 14 Apr 2023 06:07:31 +0000 (UTC) (envelope-from ml@netfence.it) Received: from soth.netfence.it (mailserver.netfence.it [78.134.96.152]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mailserver.netfence.it", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PyQw31w0Fz3x0d; Fri, 14 Apr 2023 06:07:31 +0000 (UTC) (envelope-from ml@netfence.it) Authentication-Results: mx1.freebsd.org; none Received: from [10.1.2.18] (mailserver.netfence.it [78.134.96.152]) (authenticated bits=0) by soth.netfence.it (8.17.1/8.17.1) with ESMTPSA id 33E67K81017479 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO); Fri, 14 Apr 2023 08:07:21 +0200 (CEST) (envelope-from ml@netfence.it) X-Authentication-Warning: soth.netfence.it: Host mailserver.netfence.it [78.134.96.152] claimed to be [10.1.2.18] Message-ID: <70dd5c7a-d37d-2463-ea34-e05c081ab325@netfence.it> Date: Fri, 14 Apr 2023 08:07:20 +0200 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: M2 NVME support Content-Language: en-US To: Freddie Cash , egoitz@ramattack.net, Freebsd fs , Freebsd hackers , freebsd-hardware@freebsd.org References: From: Andrea Venturoli In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.86 X-Rspamd-Queue-Id: 4PyQw31w0Fz3x0d X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:35612, ipnet:78.134.0.0/17, country:IT] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On 4/13/23 18:43, Freddie Cash wrote: > If you want the best performance, and money isn't a restriction, then > you'll want to look into servers that have U.2 (or whatever the next-gen > small form factor interface name is) slots and backplanes.  The > drives cost a lot more than regular M.2 SSDs But, being "enterprise" drive, this is perhaps justified. Actually they can even cost less than (slower) SATA "enterprise" drives. Of course you'll need to add the cost of a controller, which you have almost always "for free" for SATA. bye av. From nobody Fri Apr 14 23:57:08 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Pytg30S5qz45557 for ; Fri, 14 Apr 2023 23:57:51 +0000 (UTC) (envelope-from steffen@sdaoden.eu) Received: from sdaoden.eu (sdaoden.eu [217.144.132.164]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Pytg10rbwz3Fb6; Fri, 14 Apr 2023 23:57:49 +0000 (UTC) (envelope-from steffen@sdaoden.eu) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of steffen@sdaoden.eu designates 217.144.132.164 as permitted sender) smtp.mailfrom=steffen@sdaoden.eu; dmarc=none Date: Sat, 15 Apr 2023 01:57:08 +0200 Author: Steffen Nurpmeso From: Steffen Nurpmeso To: Steffen Nurpmeso Cc: Ed Maste , freebsd-hackers@freebsd.org Subject: Re: capsicum(4): .. and SIGTRAP causing syscall really is in siginfo_t.si_errno? Message-ID: <20230414235708.gNy-c%steffen@sdaoden.eu> In-Reply-To: <20230413204117.T906W%steffen@sdaoden.eu> References: <20230412144921.8plun%steffen@sdaoden.eu> <20230412203438.IcwD7%steffen@sdaoden.eu> <20230413204117.T906W%steffen@sdaoden.eu> User-Agent: s-nail v14.9.24-443-gb34bfaf135 OpenPGP: id=EE19E1C1F2F7054F8D3954D8308964B51883A0DD; url=https://ftp.sdaoden.eu/steffen.asc; preference=signencrypt BlahBlahBlah: Any stupid boy can crush a beetle. But all the professors in the world can make no bugs. X-Spamd-Result: default: False [2.36 / 15.00]; MID_CONTAINS_FROM(1.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_SPAM_MEDIUM(0.94)[0.944]; NEURAL_HAM_LONG(-0.89)[-0.890]; NEURAL_SPAM_SHORT(0.60)[0.603]; R_SPF_ALLOW(-0.20)[+a]; MIME_GOOD(-0.10)[text/plain]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_ZERO(0.00)[0]; R_DKIM_NA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; ARC_NA(0.00)[]; ASN(0.00)[asn:15987, ipnet:217.144.128.0/20, country:DE]; RCPT_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_SOME(0.00)[]; DMARC_NA(0.00)[sdaoden.eu]; FROM_HAS_DN(0.00)[]; MIME_TRACE(0.00)[0:+] X-Rspamd-Queue-Id: 4Pytg10rbwz3Fb6 X-Spamd-Bar: ++ X-ThisMailContainsUnwantedMimeParts: N List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org P.S.: as an addendum. Steffen Nurpmeso wrote in <20230413204117.T906W%steffen@sdaoden.eu>: |Ed Maste wrote in | : ||On Wed, 12 Apr 2023 at 16:34, Steffen Nurpmeso wrote: ... ||Indeed. These limitations all stem from the properties that give ||Capsicum its security properties (no access to global namespaces or ||ambient authority). It is "relatively" straightforward to use when ||incorporated into new software from the beginning, but certainly ||harder to retrofit. || ||realpath is an interesting case, as in capability mode there is no ||absolute root directory. All paths are necessarily relative to a ||directory fd. The Casper fileargs service provides a realpath ||replacement, but I've also started looking into what a Capsicum-native ||realpathat would be. ... I changed the program in that, on FreeBSD, configuration reload after SIGHUP is only performed in --untamed mode (no OS sandbox, only generic setrlimit(2)). Like this i can tighten the capsicum(4) sandbox in that (instead) only a descriptor to our --store-path is open, an actual "chroot" thus (we chdir(2) to that on startup). 'Allowed to get rid of all realpath(3) etc (of course), all special path treatment is gone. P.P.S.: i learned quite a bit the last weeks. postfix for example does not use anything such (except a bit setrlimit), but i trust it. I do not know. I think next time i have to bite the bullet and design it with such a supercapable process from the start. Ah, is all that expensive! Compared to strict manager / worker thread scheme with almost no shared data, thread-specific memory etc etc. I was really wondering how the usage of TLS aka OpenSSL fits this, if workers are capsicum(4)ized; they need to access CA files / directories, and, hm, private keys. I saw ressl commit messages fly by with "in-memory-images", years ago, but i never really looked. That is the thing that i wonder of, all the libraries that i have to use, how does this fit such a security scheme. All black boxes, all/most evolving targets that can change at any time. Ciao, and a nice weekend i wish! And thanks again. --steffen | |Der Kragenbaer, The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) From nobody Sat Apr 15 00:13:08 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Pyv0m0b4Pz456m2 for ; Sat, 15 Apr 2023 00:13:12 +0000 (UTC) (envelope-from void@f-m.fm) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Pyv0l1G6Tz3mJm for ; Sat, 15 Apr 2023 00:13:11 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm3 header.b=FRLNxuPV; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=RBlclxHt; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 66.111.4.25 as permitted sender) smtp.mailfrom=void@f-m.fm; dmarc=pass (policy=none) header.from=f-m.fm Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id 9D3D45C01AC for ; Fri, 14 Apr 2023 20:13:10 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Fri, 14 Apr 2023 20:13:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-type:content-type:date:date:from:from:in-reply-to :message-id:mime-version:reply-to:sender:subject:subject:to:to; s=fm3; t=1681517590; x=1681603990; bh=180GwhEHRglG5i+TgdP7bKw8u 4kWi4vHxaOW7U4IQgI=; b=FRLNxuPVU5YUXKdJq8AnW5P7RL52FzvPcNk5rQXPz 1WUyrY/M0OJiLLxr/ZAT2T3lnpPFps0Zlb6r21XkaUwsDjHA1kZCN67C1PxNKOx4 XZQvItpPWL7X/9h1zOYf7dc2oC0mg+Pb9sHnWkK148gy72UB8LKH8cSWljFpDZiO 1SCQ7H1vp+W5YpFs/Gq7UGW4wb4Wii1mh2NzJRX9wSr9eAaPo8QAsG7AxrqXGMw+ B7shTHrYIxBII1/YF3kOdJ9dRoYL3ZuDWpyTWTt6hn6ZHgtEL/P+SCKR4Ng7IJu/ YI0E02XA5kmLy0tO0rniN04KA+oaTUfSkWOreGznI10wg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:message-id :mime-version:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1681517590; x=1681603990; bh=180GwhEHRglG5i+TgdP7bKw8u4kWi4vHxaO W7U4IQgI=; b=RBlclxHtJarbBsnMA4lpaxKk9Y5LpT/6bXZ7hkKJh41Cr3HVMB3 Fd3jI1xsfOe/oKCzYuxzlHq8d5zwbcnUZGdzIxFHYdfsaESbp+ik9rUZ2Fdkg5i6 up6Ryf4LW3jcjz7zHx20/rGTriR0rCqreYQhaOTKiM5mV0gOqf72LM3lxVMDUAv3 /pzVY0ERBdnlL29Z3nc9qswfGdCHpXjcJC3TqG+Gf6O8CK2ncn4H+T+PWbnVigiL PJVGpJHi/E8v1a2tRqYJpjrBQDtU1TZdOwkz53D3LALFD+X9tUH/s38W4VYxE4Iz dhSvgZFY3qMAiHqhCSWmd0k9WUWa+gKBYvg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrvdeluddgfeefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkgggtugesthdtredttd dtvdenucfhrhhomhepvhhoihguuceovhhoihgusehfqdhmrdhfmheqnecuggftrfgrthht vghrnhepveduffeivdfffffghfegfeejfefftdeiteehteekfefhvdefgfettdeuheegff eunecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepvhho ihgusehfqdhmrdhfmh X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Fri, 14 Apr 2023 20:13:10 -0400 (EDT) Date: Sat, 15 Apr 2023 01:13:08 +0100 From: void To: freebsd-hackers@freebsd.org Subject: sccache and sccache-overlay Message-ID: Mail-Followup-To: freebsd-hackers@freebsd.org List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline X-Spamd-Result: default: False [-4.70 / 15.00]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-0.998]; NEURAL_HAM_SHORT(-1.00)[-0.998]; MID_RHS_NOT_FQDN(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm3,messagingengine.com:s=fm3]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.25:c]; MIME_GOOD(-0.10)[text/plain]; RWL_MAILSPIKE_GOOD(-0.10)[66.111.4.25:from]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.25:from]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; ARC_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US]; RCVD_COUNT_THREE(0.00)[4]; TO_DN_NONE(0.00)[]; FREEMAIL_FROM(0.00)[f-m.fm]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Rspamd-Queue-Id: 4Pyv0l1G6Tz3mJm X-Spamd-Bar: ---- X-ThisMailContainsUnwantedMimeParts: N Hello hackers@ I'm trying to work out why sccache/sccache-overlay will sometimes fail for a pkg in poudriere-devel like this: ### [00:00:13] ===> Configuring for icu-73.1,1 [00:00:13] ==> Starting sccache [00:00:13] sccache: Starting the server... [00:00:23] sccache: error: Timed out waiting for server startup [00:00:23] sccache: error: No such file or directory (os error 2) [00:00:23] *** Error code 2 [00:00:23] [00:00:23] Stop. [00:00:23] make: stopped in /usr/ports/devel/icu [00:00:25] =>> Cleaning up wrkdir [00:00:25] ===> Cleaning for icu-73.1,1 ### Running poudriere again, and the problem is gone and the pkg builds fine. sccache-overlay is installed as per 'pkg info -D sccache-overlay' There's no man page yet for either sccache-overlay or sccache, so am at a loss as to how to debug. thanks in advance for any clue -- From nobody Sat Apr 15 08:45:54 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Pz6NP5f3vz44td5 for ; Sat, 15 Apr 2023 08:45:57 +0000 (UTC) (envelope-from grahamperrin@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Pz6NP54N9z3G4l for ; Sat, 15 Apr 2023 08:45:57 +0000 (UTC) (envelope-from grahamperrin@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681548357; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Zx8Szb4e4BWyuO77O+/5ocuQctt0UtGnoWE5/NQV3S0=; b=m7ruTkmo2nX5VaWPw01uum6PP26lYvgA8AN9Yj76uMG9e8I5J0H4g0JQdwi/YXJz+nxFPM 6k3uOQMuWZcV8FqlZ3vL+LD7VuRsWzfljy8tRvt3AjrYANtFGKvd8RhSIAMWi/KBMd17g3 gr+qtGuLZXgRiLJrr7Q1inzBKcWQBDCmxQaiKgXlsFnnjdqElEChwbZQJ7+ZOx0pJPIcoF Ybs+Q2bkUlwcfgoFXsduqOFYDktdUD6mMEEtE05Zblbh/n9p7Oz9Wvu1i/cDMzzUbW6eqL 0FtogUEMiD/PgGnXdHRA7pOz4Aulm7qi350+35TBNK7pFsbtA+ab1jSvfn8rMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681548357; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Zx8Szb4e4BWyuO77O+/5ocuQctt0UtGnoWE5/NQV3S0=; b=sViyGXwFmen9dsUkKXUpgiZQsXYX1nCi3243zyqckpZZInH7hhTooL+6FYRh+IuOuPMJrA PeQ83wDh3AlZO30ebRY1LG1mVwSIad3bcnGMxM3ZJPg0ASKZnCeRvoJ/pyq94etTybjX2h uSlxaVQFAja2+CQXeyjO8GdbwtzCuw081DQRF3fyopGW9sUX1iz/PBilnM3POwnF49L4su lwqlWcRyTzo5Wws4Hs1+kpj2ItfIqTLzfh8FRzDR5QC08al94VhPZt02BT3sHk75dT+iQQ mDjPS7TKRi+u6pIBLwZwSh3E4jRUkUJiEWVWAJUAMjQGHXneFijauAVy5drKbQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1681548357; a=rsa-sha256; cv=none; b=rdmUNcGsmzGv8lqZgs4HlHpUZ2KZdunSbSZeQBtpNit3rx0pu/PtDPplsvVzxToKYPWUMH G0ZMlm+er8V1I2FwUG3eYdnDO1R8IXgGL9NiVw8EP50AksbYSKVZew8/DekvU5SUTz9yG8 OjtHPmWt9zLEctmzSLvU1b+GXxDdmG0Wua+TK4yxl6ac88ah9WIMJ5hhqxT/2QReIF9Xsi cNMVee/gAcDRIo6nMwEyeGuvu0c6lXcWA2fs10diEF7W4hB6DFiL+A+SFwV8MWnNGPyWeE 0iBzQQuUlRQk2UqbLq++uytBBSkYYetR3qQKdeTqO9bGRnbQ1rspNv0/qTYM0g== Received: from [IPV6:2001:470:1f1c:a0::2] (tunnel642390-pt.tunnel.tserv1.lon2.ipv6.he.net [IPv6:2001:470:1f1c:a0::2]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: grahamperrin) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Pz6NP2BvYzXSQ for ; Sat, 15 Apr 2023 08:45:57 +0000 (UTC) (envelope-from grahamperrin@freebsd.org) Message-ID: <26c69307-5e32-16f4-898e-5ad5747ff1d7@freebsd.org> Date: Sat, 15 Apr 2023 09:45:54 +0100 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: sccache and sccache-overlay Content-Language: en-US To: freebsd-hackers@freebsd.org References: From: Graham Perrin Organization: FreeBSD In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="------------w1KKls0799r0ygThEZ80skCC" X-ThisMailContainsUnwantedMimeParts: N This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --------------w1KKls0799r0ygThEZ80skCC Content-Type: multipart/mixed; boundary="------------IVIRvwgkVrzC9yZIr0wYY2QI"; protected-headers="v1" From: Graham Perrin To: freebsd-hackers@freebsd.org Message-ID: <26c69307-5e32-16f4-898e-5ad5747ff1d7@freebsd.org> Subject: Re: sccache and sccache-overlay References: In-Reply-To: --------------IVIRvwgkVrzC9yZIr0wYY2QI Content-Type: multipart/alternative; boundary="------------ALmRYXzwqKncAfpGqxfj2nxu" --------------ALmRYXzwqKncAfpGqxfj2nxu Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: base64 T24gMTUvMDQvMjAyMyAwMToxMywgdm9pZCB3cm90ZToNCg0KPiDigKYgc29tZXRpbWVzIOKA pg0KDQo+IFJ1bm5pbmcgcG91ZHJpZXJlIGFnYWluLCBhbmQgdGhlIHByb2JsZW0gaXMgZ29u ZSDigKYNCg0KV2hpY2ggdmVyc2lvbnMgb2YgRnJlZUJTRCAoamFpbCBhbmQgaG9zdCksIGV4 YWN0bHk/DQoNCg== --------------ALmRYXzwqKncAfpGqxfj2nxu Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

On 15/04/2023 01:13, void wrote:

=E2=80= =A6 sometimes =E2=80=A6

Runnin= g poudriere again, and the problem is gone =E2=80=A6

Which versions of FreeBSD (jail and host), exactly?

--------------ALmRYXzwqKncAfpGqxfj2nxu-- --------------IVIRvwgkVrzC9yZIr0wYY2QI-- --------------w1KKls0799r0ygThEZ80skCC Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wsF5BAABCAAjFiEEWT/lssMHB+28ly8Kt2dIb0oY1AsFAmQ6ZEIFAwAAAAAACgkQt2dIb0oY1AtR cQ//UrYQuj5zPtd2IdtOMVoozDROA68ea127vbGFan0PXCd/kAnsg9BCjbR279TUqfRKo4Ix1mM7 kKZ7uiMW8xnR16AmsRzmcEo7iYkO19Kw8K/Q7zD5szgLjU8JekYhElA4tW1Cg3cnr8+ysViq7SGW aWT5jHG7jt12sLvSxMLL1g3vA4XeA9mvQ698UwzveRg19TcEgde+AcUzzuXd4Tzs6ZS5mhsHR09p NUscG8v7EGavR8QKudPna1ePzazxOuLF5pPmGqyY4GOESakgR4X3fWKKmRwkX0iS8MjWtTeKqB+A pZCNJ04vlLHDPcNCx6viRNf+BS1Nwm6OHTGX9sRrbNIWFdaqr95W7kQf/kTBo8TdgaGORFfq8qq5 vaRXd8BLxdnoNiVOkuLgZ+w8f+1k6DofiYo5Jkl4FSt+rajxEUjQrZeyeyp1XnFodxvkuvBPT5Yc roIDg8KsE7DlElNBogVnM4vzegA/e6jpO/PJWWaqjv6e+X0Xl+ETmTCYZOjWzoGgoi+/xZdpxrbr ds6JUJPitugctMIKHYXHKTNhLmv5+GI5iAmfB0StlG4gxlphP1i/EjVckXlYSJq9mxE569tqggpB nzvuEoz1YW+L/tmFhp19JWGpP94AJWgJK7Usm0gGjl3nqxz0EN8NYF7IRpNsw/JHFwT1r7PA1khn HXo= =SFbA -----END PGP SIGNATURE----- --------------w1KKls0799r0ygThEZ80skCC-- From nobody Sat Apr 15 10:39:03 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Pz8v01Nzmz455LJ for ; Sat, 15 Apr 2023 10:39:08 +0000 (UTC) (envelope-from void@f-m.fm) Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Pz8tz2bxSz3Jx9 for ; Sat, 15 Apr 2023 10:39:07 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm3 header.b=YY9EHwXO; dkim=pass header.d=messagingengine.com header.s=fm3 header.b="S ReRMmD"; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 64.147.123.19 as permitted sender) smtp.mailfrom=void@f-m.fm; dmarc=pass (policy=none) header.from=f-m.fm Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id DD4643200312 for ; Sat, 15 Apr 2023 06:39:05 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Sat, 15 Apr 2023 06:39:06 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm3; t= 1681555145; x=1681641545; bh=LmRIgJvMbq7g8WziHotxrORy78CBhoa00LM 4HN5rxV0=; b=YY9EHwXOq6x/cRAf1PCDhlCmJcSCsFZ/9dVm4WnJeGgLTYlXVZm Lt8b+QpS8hwPM8KSLvJn1746azBOOPbBHYC51t4iKwMGd51/DN0ZN9Lqjtp75p3d qCNq8eHhW5k5rYRzFFYdknyY9VvPuRdtqqw5ltdeN4Q4NDbSY1/iWsL/61Coti4t xBB4WS16zyxUUBgFjjiYNHZ1Hx0CvIvPD9r2bnmX5///zy2Cfk9WKTo2RGD3ujuS RDD2kwI4/TSgYVwbpYSF7YmKToDNlWnJOLF6wHvpHv9xgSR/tkGA3GLPK3tHpjhc yLUukujz2K8V0khlcrl95NrtocBIlMazVXA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1681555145; x= 1681641545; bh=LmRIgJvMbq7g8WziHotxrORy78CBhoa00LM4HN5rxV0=; b=S ReRMmDP97uFAAdx9VNQ/y5+nSIKCUAnaGOsZ9eaAYH90DaxGOcXRvAucVuBgPVnn 1Z9eNadMQSyBnXyOJBlMpare2p/EJDXX8sJST4fSxFJW25N4Art8PG4QYtYJ2kf6 jOLJOTXCv59Y4jGPenjZw25msiS/JwuA/ZbJc8T2vrvnIf4eeADPHZlNiJ3Y5oGg Qiv9zOYKZm8Tx+cObtclIFQ5RqOPDGpSElwUt9XRHLpiUM3b9urcViZhPXRRlo4u AkDMdLo2ObGZN7cDHTjR+yyjwM4++YOGo7V+gXsQHgCB4p07Xqy2EbUg/5vZNIuG lHqkouFIYdaOhZt/0sdrA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrvdelvddgvdekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkfhggtggugfgjsehtke ertddttdejnecuhfhrohhmpehvohhiugcuoehvohhiugesfhdqmhdrfhhmqeenucggtffr rghtthgvrhhnpeduteevgeeggeevieetvdduudetfffffffhteejgffggeevjeffiedute eftedvkeenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhm pehvohhiugesfhdqmhdrfhhm X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Sat, 15 Apr 2023 06:39:04 -0400 (EDT) Date: Sat, 15 Apr 2023 11:39:03 +0100 From: void To: freebsd-hackers@freebsd.org Subject: Re: sccache and sccache-overlay Message-ID: Mail-Followup-To: freebsd-hackers@freebsd.org References: <26c69307-5e32-16f4-898e-5ad5747ff1d7@freebsd.org> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <26c69307-5e32-16f4-898e-5ad5747ff1d7@freebsd.org> X-Spamd-Result: default: False [-4.60 / 15.00]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; MID_RHS_NOT_FQDN(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.19]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm3,messagingengine.com:s=fm3]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[64.147.123.19:from]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; TO_MATCH_ENVRCPT_ALL(0.00)[]; ARC_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:29838, ipnet:64.147.123.0/24, country:US]; RCVD_COUNT_THREE(0.00)[4]; TO_DN_NONE(0.00)[]; FREEMAIL_FROM(0.00)[f-m.fm]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Rspamd-Queue-Id: 4Pz8tz2bxSz3Jx9 X-Spamd-Bar: ---- X-ThisMailContainsUnwantedMimeParts: N On Sat, Apr 15, 2023 at 09:45:54AM +0100, Graham Perrin wrote: >On 15/04/2023 01:13, void wrote: > >> … sometimes … > >> Running poudriere again, and the problem is gone … > >Which versions of FreeBSD (jail and host), exactly? > host is main-n261613-2b4b3789f877 amd64 1400083 1400083 built 19th March. poudriere jail is 13.2-STABLE 1302504 amd64 built 3rd March thanks, --