From nobody Sat Nov 11 14:41:23 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SSJLG4P5Cz512tR for ; Sat, 11 Nov 2023 14:41:58 +0000 (UTC) (envelope-from yasu@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SSJLG3ypNz3Z78; Sat, 11 Nov 2023 14:41:58 +0000 (UTC) (envelope-from yasu@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1699713718; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=m5VS/iyKnyk9ufJ9JnF5R5+6nZHJ9/+JhASjEWq/Z4Q=; b=h1+uDYIhVwT93iqf57jpSDJvyN3wXDUM+n4Ig/0ih9ycDJfZdDzslNabQ5915r56ezAsUq DUx7RnIZUuFjmkJIQefQ5E5wV1vaVGhPuVaRmcEnW6vWkMK7darHr8wW90WITlNvjHPNgY DA4vnEAXCOi6FkW3wVga2Ls2Iw3HLb/SJEFPGA5JtGL/Xnypbcl5RIiAwkjh7D1fHyhFTa a12Qo8tm4evGPBCbg57Mskkhz0BfccMLpYFt97L+vn+37JVjWgb/ycOORDMVVlC9d6Hg5/ rlOhgOWOsKlyDroN9Y8BBT0fPcUJuX989Wr0wpWEW23XFoPY2Q/k/ZsvOPI8bw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1699713718; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=m5VS/iyKnyk9ufJ9JnF5R5+6nZHJ9/+JhASjEWq/Z4Q=; b=sGC2mNREbaRGQHdX5fCYVV9YFDRalzrVpmp2+qIn1y72u2Dukkxwd7/Bnyul8hYG5aSdP2 11bwG8utPZ8W40JeXbHa9OgPcl+u1T1CxsD7qnW2sGIPS+vL0LRHtx21Nw5BDlWXVt4JNz yMKggFCS+y8kW6aVZEm7Ls2FWtrbe83CA30iOuJZzuGYQBbCyvWdbGcgAMom8uTGnE//hQ xRzkz85p5IyFxMSut11UAGDxFZs3VWlDTcHK2UgsCtLtY+beo/5gCpmefwij+QKtDCvBWX NfvzXXY83ztg2bM8jJSpgWVvbSrcRzH+hgLrlXDPVqARmx8jkheOUlOahMc9tw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1699713718; a=rsa-sha256; cv=none; b=WkKQSf2BVEgYX2xLzWI6ZQam3G0wTxD9nYLS3UMZBIzfhuGTS0et+lDfa4j/+lDBD29TP/ XeGRjOHV5VGG+TNiXqE4IrmyVYhkpojicOHILVKjTmN8CNnikkYv5IejE5Sc+3zdMYyQcF S0jL75WeuABV78McLbteHXmBah+uecioGJoROBMXAboNbkTZI5O4AmbwdSSO6Phtnfjuct 0M7oTjoGFBhXuWxFHKfQN2Vixu7tvoXeKYWAeaO/nI6JULIa+qXzpDZ+BjM7pQShNWjvuP r+g9UtHKSkqI3gE3TtyhwZJoL8oXBZIVMyFLNeYiWmpZmCCGqxBOEXjFLOehTQ== Received: from localhost (unknown [IPv6:240b:11:220:fe00::174:11]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: yasu/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4SSJLF6JtTz78q; Sat, 11 Nov 2023 14:41:57 +0000 (UTC) (envelope-from yasu@FreeBSD.org) Date: Sat, 11 Nov 2023 23:41:23 +0900 (JST) Message-Id: <20231111.234123.775590883419791978.yasu@FreeBSD.org> To: freebsd-current@freebsd.org Subject: Re: Updating motherboard BIOS without MS Windows From: Yasuhiro Kimura In-Reply-To: References: X-Mailer: Mew version 6.9 on Emacs 30.0.50 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit From: Nuno Teixeira Subject: Updating motherboard BIOS without MS Windows Date: Sat, 11 Nov 2023 10:56:58 +0000 > Hello all, > > Maybe not the best mailing to ask it but... > > How do you update BIOS without Windows since most brands only have bios software to windows? > > I'm about to buy a new pc without windows license and I'm looking for brands that have bios-cli that work in FreeBSD. > > Thanks, I always buy PC parts and assemble them by myself. Currently I have 3 PCs that use motherboard of different brands. And BIOS of all 3 motherbords includes the way to update itself without the help of OS and/or utility program. Typically it works as following. 1. Enter BIOS menu. 2. Invoke the way to update BIOS. 3. Read input file from the media formatted with FAT file system. 4. Start updating BIOS. 5. After finished PC is rebooted. On the other hand, new versions of BIOS are provided on the web site of each brand as zip archive. So BIOS of motherboard can be updated with following steps. 1. Download zip archive of BIOS file from web site of motherboard brand. 2. Extract BIOS file from the archive with `unzip`. 3. Insert USB memstick. 4. Format the memstick with `newfs_msdos`. 5. Mount the partion with `mount -t msdosfs` and copy BIOS file to it. 6. Reboot PC and enter BIOS menu. 7. Invoke the way to update BIOS. 8. Select BIOS file in the memstick as input. 9. Start update of BIOS. Though I haven't check motherboards of eatch brand in detail, I think it rather common for recent motherboards to provide similar functionality. It seems you are going to buy already assembled PC. And I'm not sure if it's easy to know the brand and model of the motherboard used with the PC. But if it isn't diffcult there is good chance you can select and buy the PC that can update BIOS with similar way as above. HTH. --- Yasuhiro Kimura From nobody Sun Nov 12 16:51:40 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SSzBq4phLz50H7p for ; Sun, 12 Nov 2023 16:52:51 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-yw1-x112d.google.com (mail-yw1-x112d.google.com [IPv6:2607:f8b0:4864:20::112d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SSzBp6nPhz4mcS for ; Sun, 12 Nov 2023 16:52:50 +0000 (UTC) (envelope-from mavbsd@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=WZ7KPEqj; spf=pass (mx1.freebsd.org: domain of mavbsd@gmail.com designates 2607:f8b0:4864:20::112d as permitted sender) smtp.mailfrom=mavbsd@gmail.com; dmarc=none Received: by mail-yw1-x112d.google.com with SMTP id 00721157ae682-5a84204e7aeso42830197b3.0 for ; Sun, 12 Nov 2023 08:52:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1699807970; x=1700412770; darn=freebsd.org; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=eQpKBmFIcakM2IGrHJcFnJcPmjrbVDdv3xdprJ/BGsU=; b=WZ7KPEqjmf7tx+DbCM9gN6LA9OFD3vlt8Dgiqe1Hja1PJfU9nOpT6sdM0W8V/FrP5H OiZ3xat02VtBtlJclH37j6hmAV+oN7VUPduBf9cjuwRUMsuKXJGqutXT2tdNq5VOyCL1 lfnCHLyr4XmPMNm/A6oBS55ZFeMsjscS4RjQ/D+wCW8kAnp7zJNAAjclbAP7hlCHkcKd REWPJb73eMCYp1So/I0XeSR+MOm5iKf32Cv9AHG7jh4fJ79S5qMCIcA7UVfURLh+Z/I/ 4+WjvkZNM82Y/H5Sfcf5kdDY/MXl4JhiIdAw3Ei39BTtndjDog06q1+WanV+mXuP28rF WFJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699807970; x=1700412770; h=content-transfer-encoding:in-reply-to:from:references:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=eQpKBmFIcakM2IGrHJcFnJcPmjrbVDdv3xdprJ/BGsU=; b=efbvSf5NqpqQ7ft2w29iNyNMk2oQGQ/+XoWGhjOpFXqY8hSAGDhGnSZD27Y7DL7I+X QIzDnE8z6SduCttfPzcTwG4fz8VQD40+9jKSKiQkcbfDCr5VSM7QVfPpehU+fxliu80W F7kgPxJNQ+AKOjbJnYTO7NqLxCr43MGBTDFDM04813E0CCDGp0WzkNtFCeVrHL/HGaV7 1w9J1M8uy8LRPijU8MBbpWddUPFL0H7Kzo8nMk68HDgWM5qO75gWl0ZoFpUGSqEDgdgj WS99zSfYKj9CMTMpDFlyDs3RxZ6E5jr/KrpLWCcFjsPjmxIQ0YWTNvP77QRdGacH4K6E 9K6A== X-Gm-Message-State: AOJu0Yz28u+yXy6wGQn6yr6CMHDJJg1WRlauIUs8WMG1HTylcka8zRxo KPwvfNbDWiQ/lCwQMiPnIvm7e6Jp8kE= X-Google-Smtp-Source: AGHT+IHRToxzUZHBfq1pJ7bm2vSH6NoSnQFM+yQWqhE8PfNmEt5BBfOX56uWoTP08lL1tacYijVnpA== X-Received: by 2002:a0d:cc53:0:b0:5af:b0ca:6950 with SMTP id o80-20020a0dcc53000000b005afb0ca6950mr4817399ywd.42.1699807969545; Sun, 12 Nov 2023 08:52:49 -0800 (PST) Received: from [10.230.45.5] ([38.32.73.2]) by smtp.gmail.com with ESMTPSA id j10-20020a81920a000000b005a8eadbadbesm1284671ywg.19.2023.11.12.08.52.48 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 12 Nov 2023 08:52:48 -0800 (PST) Message-ID: Date: Sun, 12 Nov 2023 11:51:40 -0500 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.11.0 Subject: Re: crash zfs_clone_range() Content-Language: en-US To: Ronald Klop , current@freebsd.org References: <349700057.3452.1699611152405@localhost> From: Alexander Motin In-Reply-To: <349700057.3452.1699611152405@localhost> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spamd-Result: default: False [-3.20 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FORGED_SENDER(0.30)[mav@FreeBSD.org,mavbsd@gmail.com]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; DMARC_NA(0.00)[freebsd.org]; PREVIOUSLY_DELIVERED(0.00)[current@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::112d:from]; TO_MATCH_ENVRCPT_SOME(0.00)[]; ARC_NA(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; MLMMJ_DEST(0.00)[current@freebsd.org]; RCPT_COUNT_TWO(0.00)[2]; DKIM_TRACE(0.00)[gmail.com:+]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MIME_TRACE(0.00)[0:+]; FROM_NEQ_ENVFROM(0.00)[mav@FreeBSD.org,mavbsd@gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4SSzBp6nPhz4mcS X-Spamd-Bar: --- Hi Ronald, As I can see, the clone request to ZFS came through nullfs, and it crashed immediately on enter. I've never been a VFS layer expert, but to me it may be a nullfs problem, not zfs. Is there chance you was (un-)mounting something when this happened? On 10.11.2023 05:12, Ronald Klop wrote: > Hi, > > Had this crash today on RPI4/15-CURRENT. > > FreeBSD rpi4 15.0-CURRENT FreeBSD 15.0-CURRENT #19 > main-b0203aaa46-dirty: Sat Nov  4 11:48:33 CET 2023 > ronald@rpi4:/home/ronald/dev/freebsd/obj/home/ronald/dev/freebsd/src/arm64.aarch64/sys/GENERIC-NODEBUG arm64 > > $ sysctl -a | grep bclon > vfs.zfs.bclone_enabled: 1 > > I started a jail with poudriere to build a package. The jail uses null > mounts over ZFS. > > [root]# cu -s 115200 -l /dev/cuaU0 > Connected > > db> bt > Tracing pid 95213 tid 100438 td 0xffff0000e1e97900 > db_trace_self() at db_trace_self > db_stack_trace() at db_stack_trace+0x120 > db_command() at db_command+0x2e4 > db_command_loop() at db_command_loop+0x58 > db_trap() at db_trap+0x100 > kdb_trap() at kdb_trap+0x334 > handle_el1h_sync() at handle_el1h_sync+0x18 > --- exception, esr 0xf2000000 > kdb_enter() at kdb_enter+0x48 > vpanic() at vpanic+0x1dc > panic() at panic+0x48 > data_abort() at data_abort+0x2fc > handle_el1h_sync() at handle_el1h_sync+0x18 > --- exception, esr 0x96000004 > rms_rlock() at rms_rlock+0x1c > zfs_clone_range() at zfs_clone_range+0x68 > zfs_freebsd_copy_file_range() at zfs_freebsd_copy_file_range+0x19c > null_bypass() at null_bypass+0x118 > vn_copy_file_range() at vn_copy_file_range+0x18c > kern_copy_file_range() at kern_copy_file_range+0x36c > sys_copy_file_range() at sys_copy_file_range+0x8c > do_el0_sync() at do_el0_sync+0x634 > handle_el0_sync() at handle_el0_sync+0x48 > --- exception, esr 0x56000000 > > > Oh.. While typing this I rebooted the machine and it happened again. I > didn't start anything in particular although the machine runs some jails. > > x0: 0x00000000000000e0 >   x1: 0xffffa00090317a48 >   x2: 0xffffa000f79d4f00 >   x3: 0xffffa000c61a44a8 >   x4: 0xffff0000deefe460 ($d.2 + 0xdd776560) >   x5: 0xffffa001250e4c00 >   x6: 0xffff0000e54025b5 ($d.5 + 0xc) >   x7: 0x000000000000030a >   x8: 0xffff0000e1559000 ($d.2 + 0xdfdd1100) >   x9: 0x0000000000000001 >  x10: 0x0000000000000000 >  x11: 0x0000000000000001 >  x12: 0x0000000000000002 >  x13: 0x0000000000000000 >  x14: 0x0000000000000001 >  x15: 0x0000000000000000 >  x16: 0xffff0000016dce88 (__stop_set_modmetadata_set + 0x1310) >  x17: 0xffff0000004e0d44 (rms_rlock + 0x0) >  x18: 0xffff0000deefe280 ($d.2 + 0xdd776380) >  x19: 0x0000000000000000 >  x20: 0xffff0000deefe460 ($d.2 + 0xdd776560) >  x21: 0x7fffffffffffffff >  x22: 0xffffa00090317a48 >  x23: 0xffffa000f79d4f00 >  x24: 0xffffa001067ef910 >  x25: 0x00000000000000e0 >  x26: 0xffffa000158a8000 >  x27: 0x0000000000000000 >  x28: 0xffffa000158a8000 >  x29: 0xffff0000deefe280 ($d.2 + 0xdd776380) >   sp: 0xffff0000deefe280 >   lr: 0xffff000001623564 (zfs_clone_range + 0x6c) >  elr: 0xffff0000004e0d60 (rms_rlock + 0x1c) > spsr: 0x00000000a0000045 >  far: 0x0000000000000108 >  esr: 0x0000000096000004 > panic: data abort in critical section or under mutex > cpuid = 1 > time = 1699610885 > KDB: stack backtrace: > db_trace_self() at db_trace_self > db_trace_self_wrapper() at db_trace_self_wrapper+0x38 > vpanic() at vpanic+0x1a0 > panic() at panic+0x48 > data_abort() at data_abort+0x2fc > handle_el1h_sync() at handle_el1h_sync+0x18 > --- exception, esr 0x96000004 > rms_rlock() at rms_rlock+0x1c > zfs_clone_range() at zfs_clone_range+0x68 > zfs_freebsd_copy_file_range() at zfs_freebsd_copy_file_range+0x19c > null_bypass() at null_bypass+0x118 > vn_copy_file_range() at vn_copy_file_range+0x18c > kern_copy_file_range() at kern_copy_file_range+0x36c > sys_copy_file_range() at sys_copy_file_range+0x8c > do_el0_sync() at do_el0_sync+0x634 > handle_el0_sync() at handle_el0_sync+0x48 > --- exception, esr 0x56000000 > KDB: enter: panic > [ thread pid 3792 tid 100394 ] > Stopped at      kdb_enter+0x48: str     xzr, [x19, #768] > db> > > I'll keep the debugger open for a while. Can I type something for > additional info? > > Regards, > Ronald. -- Alexander Motin From nobody Sun Nov 12 18:47:36 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4ST1lW1wJQz50ktR for ; Sun, 12 Nov 2023 18:47:51 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4ST1lV2vyfz3L4d; Sun, 12 Nov 2023 18:47:50 +0000 (UTC) (envelope-from kostikbel@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: from tom.home (kib@localhost [127.0.0.1] (may be forged)) by kib.kiev.ua (8.17.1/8.17.1) with ESMTP id 3ACIlatp075977; Sun, 12 Nov 2023 20:47:39 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 3ACIlatp075977 Received: (from kostik@localhost) by tom.home (8.17.1/8.17.1/Submit) id 3ACIlahJ075976; Sun, 12 Nov 2023 20:47:36 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 12 Nov 2023 20:47:36 +0200 From: Konstantin Belousov To: Alexander Motin Cc: Ronald Klop , current@freebsd.org Subject: Re: crash zfs_clone_range() Message-ID: References: <349700057.3452.1699611152405@localhost> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM,LOTS_OF_MONEY, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-14) on tom.home X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US] X-Rspamd-Queue-Id: 4ST1lV2vyfz3L4d On Sun, Nov 12, 2023 at 11:51:40AM -0500, Alexander Motin wrote: > Hi Ronald, > > As I can see, the clone request to ZFS came through nullfs, and it crashed > immediately on enter. I've never been a VFS layer expert, but to me it may > be a nullfs problem, not zfs. Is there chance you was (un-)mounting > something when this happened? It is not nullfs issue, I believe, but the lack of the busy reference on the upper mount. I think https://reviews.freebsd.org/D42554 should cover it. > > On 10.11.2023 05:12, Ronald Klop wrote: > > Hi, > > > > Had this crash today on RPI4/15-CURRENT. > > > > FreeBSD rpi4 15.0-CURRENT FreeBSD 15.0-CURRENT #19 > > main-b0203aaa46-dirty: Sat Nov  4 11:48:33 CET 2023 ronald@rpi4:/home/ronald/dev/freebsd/obj/home/ronald/dev/freebsd/src/arm64.aarch64/sys/GENERIC-NODEBUG > > arm64 > > > > $ sysctl -a | grep bclon > > vfs.zfs.bclone_enabled: 1 > > > > I started a jail with poudriere to build a package. The jail uses null > > mounts over ZFS. > > > > [root]# cu -s 115200 -l /dev/cuaU0 > > Connected > > > > db> bt > > Tracing pid 95213 tid 100438 td 0xffff0000e1e97900 > > db_trace_self() at db_trace_self > > db_stack_trace() at db_stack_trace+0x120 > > db_command() at db_command+0x2e4 > > db_command_loop() at db_command_loop+0x58 > > db_trap() at db_trap+0x100 > > kdb_trap() at kdb_trap+0x334 > > handle_el1h_sync() at handle_el1h_sync+0x18 > > --- exception, esr 0xf2000000 > > kdb_enter() at kdb_enter+0x48 > > vpanic() at vpanic+0x1dc > > panic() at panic+0x48 > > data_abort() at data_abort+0x2fc > > handle_el1h_sync() at handle_el1h_sync+0x18 > > --- exception, esr 0x96000004 > > rms_rlock() at rms_rlock+0x1c > > zfs_clone_range() at zfs_clone_range+0x68 > > zfs_freebsd_copy_file_range() at zfs_freebsd_copy_file_range+0x19c > > null_bypass() at null_bypass+0x118 > > vn_copy_file_range() at vn_copy_file_range+0x18c > > kern_copy_file_range() at kern_copy_file_range+0x36c > > sys_copy_file_range() at sys_copy_file_range+0x8c > > do_el0_sync() at do_el0_sync+0x634 > > handle_el0_sync() at handle_el0_sync+0x48 > > --- exception, esr 0x56000000 > > > > > > Oh.. While typing this I rebooted the machine and it happened again. I > > didn't start anything in particular although the machine runs some > > jails. > > > > x0: 0x00000000000000e0 > >   x1: 0xffffa00090317a48 > >   x2: 0xffffa000f79d4f00 > >   x3: 0xffffa000c61a44a8 > >   x4: 0xffff0000deefe460 ($d.2 + 0xdd776560) > >   x5: 0xffffa001250e4c00 > >   x6: 0xffff0000e54025b5 ($d.5 + 0xc) > >   x7: 0x000000000000030a > >   x8: 0xffff0000e1559000 ($d.2 + 0xdfdd1100) > >   x9: 0x0000000000000001 > >  x10: 0x0000000000000000 > >  x11: 0x0000000000000001 > >  x12: 0x0000000000000002 > >  x13: 0x0000000000000000 > >  x14: 0x0000000000000001 > >  x15: 0x0000000000000000 > >  x16: 0xffff0000016dce88 (__stop_set_modmetadata_set + 0x1310) > >  x17: 0xffff0000004e0d44 (rms_rlock + 0x0) > >  x18: 0xffff0000deefe280 ($d.2 + 0xdd776380) > >  x19: 0x0000000000000000 > >  x20: 0xffff0000deefe460 ($d.2 + 0xdd776560) > >  x21: 0x7fffffffffffffff > >  x22: 0xffffa00090317a48 > >  x23: 0xffffa000f79d4f00 > >  x24: 0xffffa001067ef910 > >  x25: 0x00000000000000e0 > >  x26: 0xffffa000158a8000 > >  x27: 0x0000000000000000 > >  x28: 0xffffa000158a8000 > >  x29: 0xffff0000deefe280 ($d.2 + 0xdd776380) > >   sp: 0xffff0000deefe280 > >   lr: 0xffff000001623564 (zfs_clone_range + 0x6c) > >  elr: 0xffff0000004e0d60 (rms_rlock + 0x1c) > > spsr: 0x00000000a0000045 > >  far: 0x0000000000000108 > >  esr: 0x0000000096000004 > > panic: data abort in critical section or under mutex > > cpuid = 1 > > time = 1699610885 > > KDB: stack backtrace: > > db_trace_self() at db_trace_self > > db_trace_self_wrapper() at db_trace_self_wrapper+0x38 > > vpanic() at vpanic+0x1a0 > > panic() at panic+0x48 > > data_abort() at data_abort+0x2fc > > handle_el1h_sync() at handle_el1h_sync+0x18 > > --- exception, esr 0x96000004 > > rms_rlock() at rms_rlock+0x1c > > zfs_clone_range() at zfs_clone_range+0x68 > > zfs_freebsd_copy_file_range() at zfs_freebsd_copy_file_range+0x19c > > null_bypass() at null_bypass+0x118 > > vn_copy_file_range() at vn_copy_file_range+0x18c > > kern_copy_file_range() at kern_copy_file_range+0x36c > > sys_copy_file_range() at sys_copy_file_range+0x8c > > do_el0_sync() at do_el0_sync+0x634 > > handle_el0_sync() at handle_el0_sync+0x48 > > --- exception, esr 0x56000000 > > KDB: enter: panic > > [ thread pid 3792 tid 100394 ] > > Stopped at      kdb_enter+0x48: str     xzr, [x19, #768] > > db> > > > > I'll keep the debugger open for a while. Can I type something for > > additional info? > > > > Regards, > > Ronald. > > -- > Alexander Motin From nobody Mon Nov 13 04:11:03 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4STGFQ165Sz50nBk for ; Mon, 13 Nov 2023 04:11:06 +0000 (UTC) (envelope-from gbe@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4STGFQ0g1wz4J8n; Mon, 13 Nov 2023 04:11:06 +0000 (UTC) (envelope-from gbe@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1699848666; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=acbVcmOoBPBHguQCQ0cRVJjfW/tD2Wfat1qantJOUxA=; b=orEFxTTH9movuCNTzzT8pLitigoygYe9+z/GKNvbjafnVC9+MxDeafm0V+FXKY8+T9yUcQ sx1b1jpRfd5cIzMB9QdHymS1ZWacwTJE5FMyaigbLe//V9T5iGDSpBdKoch8hjsqICTY7o suFVIIddFSkydPk50g9UQOrtuac3g2/qgnx0+MWTWzj9FD3/kHZvF4XEKWU8uqs6A+dk7e sqZreTMSN56dqQs0e252wgseqFtb5x3/DyQ6Jhcm5IV9e046HrVZGKMzaVgNepIu24uXav nk2dZSPKMjXY+/9++8vTLc8X+FrVUGOhHo1IfIiBaCz/0Ws/J/DzYhbzpFjWAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1699848666; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=acbVcmOoBPBHguQCQ0cRVJjfW/tD2Wfat1qantJOUxA=; b=OuKutXizh/+kQQZWrveDnorSrc7cDkJYzsziIdNpcdRCC2T570vu64V3yACC+AXpiXaWRw hgmsv2pxC1eeJk0JLX+ca+miVX9Aw6Ge+yC7OE7wQs9Hz+htR397m/qENj9zXOQdmFiuu2 ZdU/TES2xK0bt3+2co3xk7+emRAdY8OOwJrzsoEaCDssVmkdbpIAydFJsrdG9qcjWE/oSh xmlXy0/BJLbzKs10cFo/ld41UKdP+dNTOSglH2q3pywO2pJS3byYOfiW+5rjQrib6y58g7 d9n+5UnB9O3MaC9gl+8uc4Nb84Fsyo6e3MVvkhVJRON9WgUyRxhWYqLwczSDBQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1699848666; a=rsa-sha256; cv=none; b=P0cqDSefFQYmPOX4AgfIh7sWJdCi3+CDUPykzYUFQhpyTLWFaPQn+2sMGQY/g949s2Bu7x l0cs5RS0gLxiRUKmVNUhfhYfN9P5hpMi3waFQbM001wIHN63zlD8PeSeJdeW3htcDSmvDs bwrr8PPR3tB5BBaHGdKirizbY3kX8n8U/Nwsxn9WFlDgNft6caluCfhCbZbvMXG8hDedbu YF5fXCkGw4UebGTCOKGQ3VU/Mb5PiMsGLmIgqHwqf/CZ+vY+Dfzc3w+3kOUM3qYvjj4DMF 8+iw3Kaii9QBhRapTbJcXxUN32Ks/DuAjjecksXLIkycY14wWocddMUEWgK2cQ== Received: from localhost (p200300cb870909e461899cc7c6a0fa86.dip0.t-ipconnect.de [IPv6:2003:cb:8709:9e4:6189:9cc7:c6a0:fa86]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: gbe) by smtp.freebsd.org (Postfix) with ESMTPSA id 4STGFP3bW0z1M8p; Mon, 13 Nov 2023 04:11:05 +0000 (UTC) (envelope-from gbe@freebsd.org) Date: Mon, 13 Nov 2023 05:11:03 +0100 From: Gordon Bergling To: Zhenlei Huang Cc: Rick Macklem , John Baldwin , FreeBSD Current , gallatin@freebsd.org Subject: Re: KTLS thread on 14.0-RC3 Message-ID: References: <53AC8651-141E-4950-84D9-FD94E8B353FD@FreeBSD.org> <77952261-2fe0-428e-b72c-d805f0273a76@FreeBSD.org> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Url: X-Operating-System: FreeBSD 14.0-RELEASE amd64 X-Host-Uptime: 5:07AM up 2 days, 10:07, 3 users, load averages: 0.51, 0.87, 0.61 Hi, On Wed, Nov 01, 2023 at 09:01:32AM +0800, Zhenlei Huang wrote: > > On Nov 1, 2023, at 8:37 AM, Rick Macklem wrote: > > On Tue, Oct 31, 2023 at 10:06 AM John Baldwin > wrote: > >> On 10/30/23 3:41 AM, Zhenlei Huang wrote: > >>>> On Oct 30, 2023, at 12:09 PM, Zhenlei Huang wrote: > >>>>> On Oct 29, 2023, at 5:43 PM, Gordon Bergling wrote: > >>>>> > >>>>> Hi, > >>>>> > >>>>> I am currently building a new system, which should be based on 14.0-RELEASE. > >>>>> Therefor I am tracking releng/14.0 since its creation and updating it currently > >>>>> via the usualy buildworld steps. > >>>>> > >>>>> What I have noticed recently is, that the [KTLS] is missing. I have a stable/13 > >>>>> system which shows the [KTLS] thread and a very recent -CURRENT that also shows > >>>>> the [KTLS] thread. > >>>>> > >>>>> The stable/13 and releng/14.0 systems both use the GENERIC kernel, without any > >>>>> custom modifications. > >>>>> > >>>>> Loaded KLDs are also the same. > >>>>> > >>>>> Did I miss something, or is there something in releng/14.0 missing, which > >>>>> is currenlty enabled in stable/13? > >>>> > >>>> KTLS shall still work as intended, the creation of it threads is deferred. Thanks for the information, I wasn't aware of this change. > >>>> See a72ee355646c (ktls: Defer creation of threads and zones until first use) > >>>>> Run ktls_init() when the first KTLS session is created rather than > >>>>> unconditionally during boot. This avoids creating unused threads and > >>>>> allocating unused resources on systems which do not use KTLS. > >>>> > >>>> ``` > >>>> -SYSINIT(ktls, SI_SUB_SMP + 1, SI_ORDER_ANY, ktls_init, NULL); > >>>> ``` > >>> > >>> Seems 14.0 only create one KTLS thread. > >>> > >>> IIRC 13.2 create one thread per core. > >> > >> That part should not be different. There should always be one thread per core. > > Just fyi, I see one thread/core. > > Did you happen to do something like "ps ax" instead of "ps axHl"? > > Yes, I typed "ps auxx". `ps axHl` is the right way to get kernel threads. > Sorry for the noise. > > > > > rick > > ps: I also see a reclaim_0 thread. -- Gordon From nobody Mon Nov 13 07:00:00 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4STL0c1tFhz51Mmy for ; Mon, 13 Nov 2023 07:00:16 +0000 (UTC) (envelope-from kiri@truefc.org) Received: from kx.truefc.org (1.212.52.36.ap.yournet.ne.jp [36.52.212.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp", Issuer "smtp" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4STL0Z2JYFz3CGj for ; Mon, 13 Nov 2023 07:00:13 +0000 (UTC) (envelope-from kiri@truefc.org) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of kiri@truefc.org designates 36.52.212.1 as permitted sender) smtp.mailfrom=kiri@truefc.org; dmarc=none Received: from kx.truefc.org (kx.truefc.org [36.52.212.1]) by kx.truefc.org (8.17.1/8.17.1) with ESMTP id 3AD700uN016693 for ; Mon, 13 Nov 2023 16:00:03 +0900 (JST) (envelope-from kiri@kx.truefc.org) Message-Id: <202311130700.3AD700uN016693@kx.truefc.org> Date: Mon, 13 Nov 2023 16:00:00 +0900 From: KIRIYAMA Kazuhiko To: FreeBSD Current Subject: bsdinstall/scriptedpart could not run ;-( User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI/1.14.6 (Maruoka) FLIM/1.14.9 (=?ISO-8859-4?Q?Goj=F2?=) APEL/10.8 MULE XEmacs/21.4 (patch 24) (Standard C) (amd64--freebsd) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII X-Spamd-Result: default: False [-3.09 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.998]; NEURAL_HAM_MEDIUM(-1.00)[-0.995]; R_SPF_ALLOW(-0.20)[+ip4:36.52.212.0/29]; MIME_GOOD(-0.10)[text/plain]; RCVD_NO_TLS_LAST(0.10)[]; ONCE_RECEIVED(0.10)[]; PREVIOUSLY_DELIVERED(0.00)[current@freebsd.org]; FREEFALL_USER(0.00)[kiri]; RCVD_COUNT_ONE(0.00)[1]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MLMMJ_DEST(0.00)[current@freebsd.org]; TO_DN_ALL(0.00)[]; DMARC_NA(0.00)[truefc.org]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MID_RHS_MATCH_FROMTLD(0.00)[]; ASN(0.00)[asn:10013, ipnet:36.52.208.0/21, country:JP]; ARC_NA(0.00)[] X-Rspamd-Queue-Id: 4STL0Z2JYFz3CGj X-Spamd-Bar: --- Hi, all I usually run bsdinstall by instllerconfig, but bsdinstall/scriptedpart could not run ;-( My installerconfig is: PARTITIONS='nda0 gpt { 200M efi, 804G freebsd-ufs /, 128G freebsd-swap }' DISTRIBUTIONS='base.txz kernel-dbg.txz kernel.txz lib32.txz tests.txz' ZFSBOOT_DISKS="" #!/bin/sh /bin/mkdir -p /.dake /bin/cp /usr/share/zoneinfo/Asia/Tokyo /etc/localtime /bin/cp /root/.cshrc /root/.cshrc.org /bin/cat <> /etc/fstab 192.168.1.17:/.dake /.dake nfs rw 0 0 EOF sed -i".bak" -Ee '/^#BDS_install.sh_added:start_line$/,/^#BDS_install.sh_added:end_line$/d' /root/.cshrc /bin/cat <<'EOF' >> /root/.cshrc #BDS_install.sh_added:start_line setenv PATH ${PATH}:/.dake/bin setenv MGRHOME /usr/home/admin setenv OPENTOOLSDIR /.dake setenv DAKEDIR /.dake #BDS_install.sh_added:end_line EOF : (snip) : I investigated in bsdinstall script and found scriptedpart which acutually run partedit with scriptedpart would not be destroy existing partition. In fact scriptedpart -> partedit changed in script as follows, then parttion editor run at terminal. root@:~ # diff -u /usr/libexec/bsdinstall/script.bak /usr/libexec/bsdinstall/script --- /usr/libexec/bsdinstall/script.bak 2023-11-02 01:35:14.728717000 +0000 +++ /usr/libexec/bsdinstall/script 2023-11-08 07:24:01.799149000 +0000 @@ -86,11 +86,13 @@ shift f_dprintf "Began Installation at %s" "$( date )" +echo "Split installerconfig" >> /tmp/bsdinstall_log rm -rf $BSDINSTALL_TMPETC mkdir $BSDINSTALL_TMPETC split -a 2 -p '^#!.*' "$SCRIPT" $TMPDIR/bsdinstall-installscript- +echo "Set DISTRIBUTIONS" >> /tmp/bsdinstall_log . $TMPDIR/bsdinstall-installscript-aa : ${DISTRIBUTIONS="kernel.txz base.txz"}; export DISTRIBUTIONS export BSDINSTALL_DISTDIR @@ -107,10 +109,19 @@ rm -f $PATH_FSTAB touch $PATH_FSTAB if [ "$ZFSBOOT_DISKS" ]; then + echo "Zfsboot" >> /tmp/bsdinstall_log + f_dprintf "Zfsboot" bsdinstall zfsboot else - bsdinstall scriptedpart "$PARTITIONS" + echo "Scriptedpart" >> /tmp/bsdinstall_log + echo "PARTITIONS=\"$PARTITIONS\"" >> /tmp/bsdinstall_log + f_dprintf "Scriptedpart with %s" "$PARTITIONS" +# bsdinstall scriptedpart "$PARTITIONS" + bsdinstall partedit fi +exit 1 +echo "Mount" >> /tmp/bsdinstall_log +f_dprintf "Mount" Maybe gpart_destroy(gpart) in part_config in parse_disk_config in scripted_editor would not execute. Target machine is as follows: root@:~ # uname -a FreeBSD 15.0-CURRENT FreeBSD 15.0-CURRENT #0 n265729-9b03a5de73d4: Thu Oct 5 21:59:06 JST 2023 root@tbedfc:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 root@:~ # gpart show nda0 => 40 1953525088 nda0 GPT (932G) 40 409600 1 efi (200M) 409640 1686110208 2 freebsd-ufs (804G) 1686519848 267005280 3 freebsd-swap (127G) root@:~ # cat /var/run/dmesg.boot ---<>--- Copyright (c) 1992-2023 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 15.0-CURRENT #0 n265729-9b03a5de73d4: Thu Oct 5 21:59:06 JST 2023 root@tbedfc:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 FreeBSD clang version 16.0.6 (https://github.com/llvm/llvm-project.git llvmorg-16.0.6-0-g7cbf1a259152) WARNING: WITNESS option enabled, expect reduced performance. VT(efifb): resolution 800x600 CPU: Intel(R) Core(TM) i5-8259U CPU @ 2.30GHz (2300.00-MHz K8-class CPU) Origin="GenuineIntel" Id=0x806ea Family=0x6 Model=0x8e Stepping=10 Features=0xbfebfbff Features2=0x7ffafbbf AMD Features=0x2c100800 AMD Features2=0x121 Structured Extended Features=0x29c67af Structured Extended Features3=0x9c002400 XSAVE Features=0xf VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID TSC: P-state invariant, performance statistics real memory = 42948624384 (40959 MB) avail memory = 41573068800 (39647 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 hardware threads random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" random: unblocking device. ioapic0 irqs 0-119 Launching APs: 1 5 2 6 7 3 4 random: entropy device external interface kbd1 at kbdmux0 efirtc0: efirtc0: registered as a time-of-day clock, resolution 1.000000s smbios0: at iomem 0x8c9b5000-0x8c9b501e smbios0: Version: 3.2, BCD Revision: 3.2 aesni0: acpi0: acpi0: Power Button (fixed) ACPI Error: No handler for Region [ECF2] (0xfffff800022f8580) [EmbeddedControl] (20221020/evregion-292) ACPI Error: Region EmbeddedControl (ID=3) has no handler (20221020/exfldio-428) ACPI Error: Aborting method \134_SB.PCI0.LPCB.H_EC.LID0._STA due to previous error (AE_NOT_EXIST) (20221020/psparse-689) cpu0: on acpi0 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 24000000 Hz quality 950 Event timer "HPET" frequency 24000000 Hz quality 550 atrtc0: port 0x70-0x77 irq 8 on acpi0 atrtc0: Warning: Couldn't map I/O. atrtc0: registered as a time-of-day clock, resolution 1.000000s Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x43,0x50-0x53 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1808-0x180b on acpi0 acpi_ec0: port 0x62,0x66 on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 vgapci0: port 0x3000-0x303f mem 0xa0000000-0xa0ffffff,0x90000000-0x9fffffff at device 2.0 on pci0 vgapci0: Boot video device xhci0: mem 0xa1300000-0xa130ffff at device 20.0 on pci0 xhci0: 32 bytes context size, 64-bit DMA usbus0 on xhci0 usbus0: 5.0Gbps Super Speed USB v3.0 pci0: at device 20.2 (no driver attached) sdhci_pci0: mem 0xa132c000-0xa132cfff at device 20.5 on pci0 sdhci_pci0: 1 slot(s) allocated pci0: at device 21.0 (no driver attached) pci0: at device 21.1 (no driver attached) pci0: at device 21.2 (no driver attached) pci0: at device 22.0 (no driver attached) ahci0: port 0x3090-0x3097,0x3080-0x3083,0x3060-0x307f mem 0xa131c000-0xa131dfff,0xa1327000-0xa13270ff,0xa1326000-0xa13267ff at device 23.0 on pci0 ahci0: AHCI v1.31 with 2 6Gbps ports, Port Multiplier not supported ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 pci0: at device 25.0 (no driver attached) sdhci_pci1: mem 0xa1324000-0xa1324fff at device 26.0 on pci0 sdhci_pci1: 1 slot(s) allocated mmc0: on sdhci_pci1 pcib1: at device 28.0 on pci0 pci1: on pcib1 pci1: at device 0.0 (no driver attached) pcib2: at device 29.0 on pci0 pci2: on pcib2 nvme0: mem 0xa1100000-0xa1103fff,0xa1104000-0xa11040ff at device 0.0 on pci2 pci0: at device 30.0 (no driver attached) pci0: at device 30.3 (no driver attached) isab0: at device 31.0 on pci0 isa0: on isab0 hdac0: mem 0xa1318000-0xa131bfff,0xa1000000-0xa10fffff at device 31.3 on pci0 pci0: at device 31.5 (no driver attached) acpi_button0: on acpi0 acpi_button1: on acpi0 acpi_tz0: on acpi0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] acpi_syscontainer0: on acpi0 acpi_acad0: on acpi0 battery0: on acpi0 acpi_lid0: on acpi0 hwpstate_intel0: on cpu0 hwpstate_intel1: on cpu1 hwpstate_intel2: on cpu2 hwpstate_intel3: on cpu3 hwpstate_intel4: on cpu4 hwpstate_intel5: on cpu5 hwpstate_intel6: on cpu6 hwpstate_intel7: on cpu7 Timecounter "TSC-low" frequency 1152000116 Hz quality 1000 Timecounters tick every 1.000 msec ugen0.1: at usbus0 uhub0 on usbus0 uhub0: on usbus0 mmc0: No compatible cards found on bus hdacc0: at cad 0 on hdac0 hdaa0: at nid 1 on hdacc0 pcm0: at nid 20 and 18 on hdaa0 pcm1: at nid 27 on hdaa0 hdacc1: at cad 2 on hdac0 hdaa1: at nid 1 on hdacc1 pcm2: at nid 3 on hdaa1 nda0 at nvme0 bus 0 scbus2 target 0 lun 1 nda0: nda0: Serial Number 2144HW440501 nda0: nvme version 1.3 nda0: 953869MB (1953525168 512 byte sectors) Trying to mount root from ufs:/dev/ufs/FreeBSD_Install [ro,noatime]... WARNING: WITNESS option enabled, expect reduced performance. uhub0: 18 ports with 18 removable, self powered Root mount waiting for: usbus0 ugen0.2: at usbus0 ugen0.3: at usbus0 uhub1 on uhub0 uhub1: on usbus0 Root mount waiting for: usbus0 uhub1: 4 ports with 4 removable, self powered Root mount waiting for: usbus0 usb_msc_auto_quirk: UQ_MSC_NO_GETMAXLUN set for USB mass storage device ELECOM MF-HMU2 (0x056e:0x6006) usb_msc_auto_quirk: UQ_MSC_NO_PREVENT_ALLOW set for USB mass storage device ELECOM MF-HMU2 (0x056e:0x6006) ugen0.4: at usbus0 umass0 on uhub1 umass0: on usbus0 umass0: SCSI over Bulk-Only; quirks = 0x8100 umass0:3:0: Attached to scbus3 da0 at umass-sim0 bus 0 scbus3 target 0 lun 0 da0: Removable Direct Access SCSI device da0: Serial Number 07BA1B0820ACB40B da0: 40.000MB/s transfers da0: 1912MB (3915776 512 byte sectors) da0: quirks=0x2 Root mount waiting for: usbus0 ugen0.5: at usbus0 ukbd0 on uhub1 ukbd0: on usbus0 kbd2 at ukbd0 ugen0.6: at usbus0 ugen0.7: at usbus0 Root mount waiting for: usbus0 ugen0.8: at usbus0 Intel(R) Wireless WiFi based driver for FreeBSD pchtherm0: mem 0xa132e000-0xa132efff at device 18.0 on pci0 ig4iic0: at device 21.0 on pci0 ig4iic0: Using MSI iicbus0: on ig4iic0 iicbus0: at addr 0x10 iicbus0: at addr 0x2c ig4iic1: at device 21.1 on pci0 ig4iic1: Using MSI iicbus1: on ig4iic1 ig4iic2: at device 21.2 on pci0 ig4iic2: Using MSI iicbus2: on ig4iic2 ig4iic3: at device 25.0 on pci0 ig4iic3: Using MSI iicbus3: on ig4iic3 iwm0: mem 0xa1200000-0xa1201fff at device 0.0 on pci1 iwm0: hw rev 0x210, fw ver 22.361476.0, address 8c:c6:81:0a:6f:81 acpi_wmi0: on acpi0 acpi_wmi0: Embedded MOF found ACPI: \134_SB.WFDE.WQCC: 1 arguments were passed to a non-method ACPI object (Buffer) (20221020/nsarguments-361) acpi_wmi1: on acpi0 acpi_wmi1: Embedded MOF found ACPI: \134_SB.WFTE.WQCC: 1 arguments were passed to a non-method ACPI object (Buffer) (20221020/nsarguments-361) lo0: link state changed to UP ums0 on uhub1 ums0: on usbus0 ums0: 5 buttons and [XYZT] coordinates ID=3 ums1 on uhub1 ums1: on usbus0 ums1: 5 buttons and [XYZ] coordinates ID=1 ure0 on uhub1 ure0: on usbus0 ugen0.8: at usbus0 (disconnected) miibus0: on ure0 rgephy0: PHY 0 on miibus0 rgephy0: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT-FDX, 1000baseT-FDX-master, auto ue0: on ure0 ue0: Ethernet address: a0:ce:c8:de:9c:95 ue0: link state changed to DOWN iichid0: at addr 0x2c on iicbus0 iichid0: Interrupt setup failed. Fallback to sampling hidbus0: on iichid0 ugen0.8: at usbus0 hms0: on hidbus0 hms0: 2 buttons and [XY] coordinates ID=2 hmt0: on hidbus0 hconf0: on hidbus0 hmt0: Multitouch touchpad with 2 external buttons, click-pad hmt0: 5 contacts with [C] properties. Report range [0:0] - [1707:1060] ng_ubt: HCI command 0x0c03 timed out ng_ubt: HCI command 0x0c03 timed out ue0: link state changed to UP ue0: link state changed to DOWN ue0: link state changed to UP And bsdinstall log is as follows: DEBUG: Running installation step: script /etc/installerconfig DEBUG: dialog.subr: DEBUG_SELF_INITIALIZE=[] DEBUG: UNAME_S=[FreeBSD] UNAME_P=[amd64] UNAME_R=[15.0-CURRENT] DEBUG: common.subr: Successfully loaded. DEBUG: /usr/libexec/bsdinstall/script: loading includes... DEBUG: f_include: file=[/usr/share/bsdconfig/dialog.subr] DEBUG: dialog.subr: loading includes... DEBUG: f_include: file=[/usr/share/bsdconfig/strings.subr] DEBUG: strings.subr: Successfully loaded. DEBUG: f_include: file=[/usr/share/bsdconfig/variable.subr] DEBUG: variable.subr: loading includes... DEBUG: f_include: file=[/usr/share/bsdconfig/dialog.subr] DEBUG: f_include: file=[/usr/share/bsdconfig/strings.subr] DEBUG: variable.subr: New variable VAR_CONFIG_FILE -> configFile DEBUG: variable.subr: New variable VAR_DEBUG -> debug DEBUG: variable.subr: New variable VAR_DEBUG_FILE -> debugFile DEBUG: variable.subr: New variable VAR_DIRECTORY_PATH -> _directoryPath DEBUG: variable.subr: New variable VAR_DOMAINNAME -> domainname DEBUG: variable.subr: New variable VAR_EDITOR -> editor DEBUG: variable.subr: New variable VAR_EXTRAS -> ifconfig_ DEBUG: variable.subr: New variable VAR_FTP_DIR -> ftpDirectory DEBUG: variable.subr: New variable VAR_FTP_HOST -> ftpHost DEBUG: variable.subr: New variable VAR_FTP_PASS -> ftpPass DEBUG: variable.subr: New variable VAR_FTP_PATH -> _ftpPath DEBUG: variable.subr: New variable VAR_FTP_PORT -> ftpPort DEBUG: variable.subr: New variable VAR_FTP_STATE -> ftpState DEBUG: variable.subr: New variable VAR_FTP_USER -> ftpUser DEBUG: variable.subr: New variable VAR_GATEWAY -> defaultrouter DEBUG: variable.subr: New variable VAR_GROUP -> group DEBUG: variable.subr: New variable VAR_GROUP_GID -> groupGid DEBUG: variable.subr: New variable VAR_GROUP_MEMBERS -> groupMembers DEBUG: variable.subr: New variable VAR_GROUP_PASSWORD -> groupPassword DEBUG: variable.subr: New variable VAR_HOSTNAME -> hostname DEBUG: variable.subr: New variable VAR_HTTP_DIR -> httpDirectory DEBUG: variable.subr: New variable VAR_HTTP_FTP_MODE -> httpFtpMode DEBUG: variable.subr: New variable VAR_HTTP_HOST -> httpHost DEBUG: variable.subr: New variable VAR_HTTP_PATH -> _httpPath DEBUG: variable.subr: New variable VAR_HTTP_PORT -> httpPort DEBUG: variable.subr: New variable VAR_HTTP_PROXY -> httpProxy DEBUG: variable.subr: New variable VAR_HTTP_PROXY_HOST -> httpProxyHost DEBUG: variable.subr: New variable VAR_HTTP_PROXY_PATH -> _httpProxyPath DEBUG: variable.subr: New variable VAR_HTTP_PROXY_PORT -> httpProxyPort DEBUG: variable.subr: New variable VAR_IFCONFIG -> ifconfig_ DEBUG: variable.subr: New variable VAR_IPADDR -> ipaddr DEBUG: variable.subr: New variable VAR_IPV6ADDR -> ipv6addr DEBUG: variable.subr: New variable VAR_IPV6_ENABLE -> ipv6_activate_all_interfaces DEBUG: variable.subr: New variable VAR_KEYMAP -> keymap DEBUG: variable.subr: New variable VAR_MEDIA_TIMEOUT -> MEDIA_TIMEOUT DEBUG: variable.subr: New variable VAR_MEDIA_TYPE -> mediaType DEBUG: variable.subr: New variable VAR_NAMESERVER -> nameserver DEBUG: variable.subr: New variable VAR_NETINTERACTIVE -> netInteractive DEBUG: variable.subr: New variable VAR_NETMASK -> netmask DEBUG: variable.subr: New variable VAR_NETWORK_DEVICE -> netDev DEBUG: variable.subr: New variable VAR_NFS_HOST -> nfsHost DEBUG: variable.subr: New variable VAR_NFS_PATH -> nfsPath DEBUG: variable.subr: New variable VAR_NFS_SECURE -> nfs_reserved_port_only DEBUG: variable.subr: New variable VAR_NFS_TCP -> nfs_use_tcp DEBUG: variable.subr: New variable VAR_NFS_V3 -> nfs_use_v3 DEBUG: variable.subr: New variable VAR_NONINTERACTIVE -> nonInteractive DEBUG: variable.subr: New variable VAR_NO_CONFIRM -> noConfirm DEBUG: variable.subr: New variable VAR_NO_ERROR -> noError DEBUG: variable.subr: New variable VAR_NO_INET6 -> noInet6 DEBUG: variable.subr: New variable VAR_PACKAGE -> package DEBUG: variable.subr: New variable VAR_PKG_TMPDIR -> PKG_TMPDIR DEBUG: variable.subr: New variable VAR_PORTS_PATH -> ports DEBUG: variable.subr: New variable VAR_RELNAME -> releaseName DEBUG: variable.subr: New variable VAR_SLOW_ETHER -> slowEthernetCard DEBUG: variable.subr: New variable VAR_TRY_DHCP -> tryDHCP DEBUG: variable.subr: New variable VAR_TRY_RTSOL -> tryRTSOL DEBUG: variable.subr: New variable VAR_UFS_PATH -> ufs DEBUG: variable.subr: New variable VAR_USER -> user DEBUG: variable.subr: New variable VAR_USER_ACCOUNT_EXPIRE -> userAccountExpire DEBUG: variable.subr: New variable VAR_USER_DOTFILES_CREATE -> userDotfilesCreate DEBUG: variable.subr: New variable VAR_USER_GECOS -> userGecos DEBUG: variable.subr: New variable VAR_USER_GID -> userGid DEBUG: variable.subr: New variable VAR_USER_GROUPS -> userGroups DEBUG: variable.subr: New variable VAR_USER_GROUP_DELETE -> userGroupDelete DEBUG: variable.subr: New variable VAR_USER_HOME -> userHome DEBUG: variable.subr: New variable VAR_USER_HOME_CREATE -> userHomeCreate DEBUG: variable.subr: New variable VAR_USER_HOME_DELETE -> userHomeDelete DEBUG: variable.subr: New variable VAR_USER_LOGIN_CLASS -> userLoginClass DEBUG: variable.subr: New variable VAR_USER_PASSWORD -> userPassword DEBUG: variable.subr: New variable VAR_USER_PASSWORD_EXPIRE -> userPasswordExpire DEBUG: variable.subr: New variable VAR_USER_SHELL -> userShell DEBUG: variable.subr: New variable VAR_USER_UID -> userUid DEBUG: variable.subr: New variable VAR_ZFSINTERACTIVE -> zfsInteractive DEBUG: variable.subr: VARIABLE_SELF_INITIALIZE=[1] DEBUG: f_variable_set_defaults: Initializing defaults... DEBUG: f_getvar: var=[debug] value=[1] r=0 DEBUG: f_getvar: var=[editor] value=[/usr/bin/ee] r=0 DEBUG: f_getvar: var=[ftpState] value=[auto] r=0 DEBUG: f_getvar: var=[ftpUser] value=[ftp] r=0 DEBUG: f_getvar: var=[hostname] value=[] r=0 DEBUG: f_getvar: var=[MEDIA_TIMEOUT] value=[300] r=0 DEBUG: f_getvar: var=[nfs_reserved_port_only] value=[NO] r=0 DEBUG: f_getvar: var=[nfs_use_tcp] value=[NO] r=0 DEBUG: f_getvar: var=[nfs_use_v3] value=[YES] r=0 DEBUG: f_getvar: var=[PKG_TMPDIR] value=[/var/tmp] r=0 DEBUG: f_getvar: var=[releaseName] value=[15.0-CURRENT] r=0 DEBUG: f_variable_set_defaults: Defaults initialized. DEBUG: variable.subr: Successfully loaded. DEBUG: f_include_lang: file=[/usr/libexec/bsdconfig/include/messages.subr] lang=[C.UTF-8] DEBUG: dialog.subr: DIALOG_SELF_INITIALIZE=[1] DEBUG: f_dialog_init: ARGV=[/etc/installerconfig] GETOPTS_STDARGS=[dD:SX] DEBUG: f_dialog_init: SECURE=[] USE_XDIALOG=[] DEBUG: f_dialog_init: dialog(1) API initialized. DEBUG: dialog.subr: Successfully loaded. DEBUG: f_include: file=[/usr/share/bsdconfig/variable.subr] DEBUG: Began Installation at Mon Nov 13 06:36:11 UTC 2023 Split installerconfig Set DISTRIBUTIONS Scriptedpart PARTITIONS="nda0 gpt { 200M efi, 804G freebsd-ufs /, 128G freebsd-swap }" Best regards --- Kazuhiko Kiriyama From nobody Tue Nov 14 12:59:56 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SV5xL6WgGz50stT for ; Tue, 14 Nov 2023 13:00:06 +0000 (UTC) (envelope-from SRS0=Pv4d=G3=klop.ws=ronald-lists@realworks.nl) Received: from smtp-relay-int-backup.realworks.nl (smtp-relay-int-backup.realworks.nl [87.255.56.188]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SV5xK2y2Tz3GmR; Tue, 14 Nov 2023 13:00:05 +0000 (UTC) (envelope-from SRS0=Pv4d=G3=klop.ws=ronald-lists@realworks.nl) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=klop.ws header.s=rw2 header.b=NmHHrZmL; spf=pass (mx1.freebsd.org: domain of "SRS0=Pv4d=G3=klop.ws=ronald-lists@realworks.nl" designates 87.255.56.188 as permitted sender) smtp.mailfrom="SRS0=Pv4d=G3=klop.ws=ronald-lists@realworks.nl"; dmarc=pass (policy=quarantine) header.from=klop.ws Received: from rwvirtual46.colo.realworks.nl (rwvirtual46.colo.realworks.nl [10.0.10.46]) by mailrelayint1.colo2.realworks.nl (Postfix) with ESMTP id 4SV5x85rTYz1cF; Tue, 14 Nov 2023 13:59:56 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=klop.ws; s=rw2; t=1699966796; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=VaX2QakZF6smWaSkXY6r5Np7hfLM7C5i/lWEfCHcJAw=; b=NmHHrZmLbk412F7Y2yI4l06TmhKhzzo9VfFTmMnQwDLvisoCoWmBpYEHO/WyDOFeKoMro4 6Cw+pfKxolOyJDF/7OZ0QZ5XEg55SCXh8KKkjfkGO0bol69O6PdUXtCgrKmmHBYJ4wqyJS ivCKh3Rg0bL1fddkR2rZaCFhvlm6aungLjtFgCK6y1DoWI8u2X4JLNkNmX8hfzO7fzIo08 6y1eeXY+0NNSE9wjLi+fz6ZDCFRkWiqVJ0vj/axWif0Se3jzUkGFK/dgC99q6hQSyyGruF nJ919uBdkLSLRFKY0LIRIiUl7ngBEG9rCxk2D85HglA+1Tlehlfubo1j3XIi5Q== Received: from rwvirtual46.colo.realworks.nl (localhost [127.0.0.1]) by rwvirtual46.colo.realworks.nl (Postfix) with ESMTP id AD41324051D; Tue, 14 Nov 2023 13:59:56 +0100 (CET) Date: Tue, 14 Nov 2023 13:59:56 +0100 (CET) From: Ronald Klop To: Konstantin Belousov Cc: Alexander Motin , current@freebsd.org Message-ID: <1900239445.5968.1699966796547@localhost> In-Reply-To: References: <349700057.3452.1699611152405@localhost> Subject: Re: crash zfs_clone_range() List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_5967_382656657.1699966796420" X-Mailer: Realworks (679.16) X-Originating-Host: from (84-105-120-103.cable.dynamic.v4.ziggo.nl [84.105.120.103]) by rwvirtual46 [10.0.10.46] with HTTP; Tue, 14 Nov 2023 13:59:56 +0100 Importance: Normal X-Priority: 3 (Normal) X-Originating-User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/119.0 X-Spamd-Result: default: False [-3.20 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[klop.ws,quarantine]; MID_RHS_NOT_FQDN(0.50)[]; FORGED_SENDER(0.30)[ronald-lists@klop.ws,SRS0=Pv4d=G3=klop.ws=ronald-lists@realworks.nl]; R_DKIM_ALLOW(-0.20)[klop.ws:s=rw2]; R_SPF_ALLOW(-0.20)[+ip4:87.255.56.128/26]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; MLMMJ_DEST(0.00)[current@freebsd.org]; FREEMAIL_TO(0.00)[gmail.com]; ASN(0.00)[asn:38930, ipnet:87.255.32.0/19, country:NL]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; DKIM_TRACE(0.00)[klop.ws:+]; FROM_NEQ_ENVFROM(0.00)[ronald-lists@klop.ws,SRS0=Pv4d=G3=klop.ws=ronald-lists@realworks.nl]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; HAS_X_PRIO_THREE(0.00)[3]; RCPT_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4SV5xK2y2Tz3GmR X-Spamd-Bar: --- ------=_Part_5967_382656657.1699966796420 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Response below Van: Konstantin Belousov Datum: zondag, 12 november 2023 19:47 Aan: Alexander Motin CC: Ronald Klop , current@freebsd.org Onderwerp: Re: crash zfs_clone_range() > > On Sun, Nov 12, 2023 at 11:51:40AM -0500, Alexander Motin wrote: > > Hi Ronald, > > > > As I can see, the clone request to ZFS came through nullfs, and it crashed > > immediately on enter. I've never been a VFS layer expert, but to me it may > > be a nullfs problem, not zfs. Is there chance you was (un-)mounting > > something when this happened? > It is not nullfs issue, I believe, but the lack of the busy reference on the > upper mount. I think https://reviews.freebsd.org/D42554 should cover it. > > > > > On 10.11.2023 05:12, Ronald Klop wrote: > > > Hi, > > > > > > Had this crash today on RPI4/15-CURRENT. > > > > > > FreeBSD rpi4 15.0-CURRENT FreeBSD 15.0-CURRENT #19 > > > main-b0203aaa46-dirty: Sat Nov 4 11:48:33 CET 2023 ronald@rpi4:/home/ronald/dev/freebsd/obj/home/ronald/dev/freebsd/src/arm64.aarch64/sys/GENERIC-NODEBUG > > > arm64 > > > > > > $ sysctl -a | grep bclon > > > vfs.zfs.bclone_enabled: 1 > > > > > > I started a jail with poudriere to build a package. The jail uses null > > > mounts over ZFS. > > > > > > [root]# cu -s 115200 -l /dev/cuaU0 > > > Connected > > > > > > db> bt > > > Tracing pid 95213 tid 100438 td 0xffff0000e1e97900 > > > db_trace_self() at db_trace_self > > > db_stack_trace() at db_stack_trace+0x120 > > > db_command() at db_command+0x2e4 > > > db_command_loop() at db_command_loop+0x58 > > > db_trap() at db_trap+0x100 > > > kdb_trap() at kdb_trap+0x334 > > > handle_el1h_sync() at handle_el1h_sync+0x18 > > > --- exception, esr 0xf2000000 > > > kdb_enter() at kdb_enter+0x48 > > > vpanic() at vpanic+0x1dc > > > panic() at panic+0x48 > > > data_abort() at data_abort+0x2fc > > > handle_el1h_sync() at handle_el1h_sync+0x18 > > > --- exception, esr 0x96000004 > > > rms_rlock() at rms_rlock+0x1c > > > zfs_clone_range() at zfs_clone_range+0x68 > > > zfs_freebsd_copy_file_range() at zfs_freebsd_copy_file_range+0x19c > > > null_bypass() at null_bypass+0x118 > > > vn_copy_file_range() at vn_copy_file_range+0x18c > > > kern_copy_file_range() at kern_copy_file_range+0x36c > > > sys_copy_file_range() at sys_copy_file_range+0x8c > > > do_el0_sync() at do_el0_sync+0x634 > > > handle_el0_sync() at handle_el0_sync+0x48 > > > --- exception, esr 0x56000000 > > > > > > > > > Oh.. While typing this I rebooted the machine and it happened again. I > > > didn't start anything in particular although the machine runs some > > > jails. > > > > > > x0: 0x00000000000000e0 > > > x1: 0xffffa00090317a48 > > > x2: 0xffffa000f79d4f00 > > > x3: 0xffffa000c61a44a8 > > > x4: 0xffff0000deefe460 ($d.2 + 0xdd776560) > > > x5: 0xffffa001250e4c00 > > > x6: 0xffff0000e54025b5 ($d.5 + 0xc) > > > x7: 0x000000000000030a > > > x8: 0xffff0000e1559000 ($d.2 + 0xdfdd1100) > > > x9: 0x0000000000000001 > > > x10: 0x0000000000000000 > > > x11: 0x0000000000000001 > > > x12: 0x0000000000000002 > > > x13: 0x0000000000000000 > > > x14: 0x0000000000000001 > > > x15: 0x0000000000000000 > > > x16: 0xffff0000016dce88 (__stop_set_modmetadata_set + 0x1310) > > > x17: 0xffff0000004e0d44 (rms_rlock + 0x0) > > > x18: 0xffff0000deefe280 ($d.2 + 0xdd776380) > > > x19: 0x0000000000000000 > > > x20: 0xffff0000deefe460 ($d.2 + 0xdd776560) > > > x21: 0x7fffffffffffffff > > > x22: 0xffffa00090317a48 > > > x23: 0xffffa000f79d4f00 > > > x24: 0xffffa001067ef910 > > > x25: 0x00000000000000e0 > > > x26: 0xffffa000158a8000 > > > x27: 0x0000000000000000 > > > x28: 0xffffa000158a8000 > > > x29: 0xffff0000deefe280 ($d.2 + 0xdd776380) > > > sp: 0xffff0000deefe280 > > > lr: 0xffff000001623564 (zfs_clone_range + 0x6c) > > > elr: 0xffff0000004e0d60 (rms_rlock + 0x1c) > > > spsr: 0x00000000a0000045 > > > far: 0x0000000000000108 > > > esr: 0x0000000096000004 > > > panic: data abort in critical section or under mutex > > > cpuid = 1 > > > time = 1699610885 > > > KDB: stack backtrace: > > > db_trace_self() at db_trace_self > > > db_trace_self_wrapper() at db_trace_self_wrapper+0x38 > > > vpanic() at vpanic+0x1a0 > > > panic() at panic+0x48 > > > data_abort() at data_abort+0x2fc > > > handle_el1h_sync() at handle_el1h_sync+0x18 > > > --- exception, esr 0x96000004 > > > rms_rlock() at rms_rlock+0x1c > > > zfs_clone_range() at zfs_clone_range+0x68 > > > zfs_freebsd_copy_file_range() at zfs_freebsd_copy_file_range+0x19c > > > null_bypass() at null_bypass+0x118 > > > vn_copy_file_range() at vn_copy_file_range+0x18c > > > kern_copy_file_range() at kern_copy_file_range+0x36c > > > sys_copy_file_range() at sys_copy_file_range+0x8c > > > do_el0_sync() at do_el0_sync+0x634 > > > handle_el0_sync() at handle_el0_sync+0x48 > > > --- exception, esr 0x56000000 > > > KDB: enter: panic > > > [ thread pid 3792 tid 100394 ] > > > Stopped at kdb_enter+0x48: str xzr, [x19, #768] > > > db> > > > > > > I'll keep the debugger open for a while. Can I type something for > > > additional info? > > > > > > Regards, > > > Ronald. > > > > -- > > Alexander Motin > > > Hi, Build a new kernel today. FreeBSD rpi4 15.0-CURRENT FreeBSD 15.0-CURRENT #20 main-051d69d6f8: Tue Nov 14 12:16:28 CET 2023 ronald@rpi4:/home/ronald/dev/freebsd/obj/home/ronald/dev/freebsd/src/arm64.aarch64/sys/GENERIC-NODEBUG arm64 x0: 0x00000000000008e0 x1: 0xffffa0006ce1fb38 x2: 0xffffa0006837a400 x3: 0xffffa0012c503a48 x4: 0xffff0000eb0ef430 (next_index + 0x815d790) x5: 0xffffa00152636300 x6: 0xffff0000e2e025b5 ($d.5 + 0xc) x7: 0x000000000000030a x8: 0xffff0000eb3212c0 (next_index + 0x838f620) x9: 0x0000000000000001 x10: 0x0000000000000000 x11: 0x0000000000000001 x12: 0x0000000000000002 x13: 0x0000000000000000 x14: 0x0000000000000001 x15: 0x0000000000000000 x16: 0xffff0000016e5b58 (__stop_set_modmetadata_set + 0x1328) x17: 0xffff0000004e0c28 (rms_rlock + 0x0) x18: 0xffff0000eb0ef250 (next_index + 0x815d5b0) x19: 0x0000000000000800 x20: 0xffff0000eb0ef430 (next_index + 0x815d790) x21: 0x7fffffffffffffff x22: 0xffffa0006ce1fb38 x23: 0xffffa0006837a400 x24: 0xffffa001ee486000 x25: 0x00000000000008e0 x26: 0xffffa000135ca000 x27: 0x0000000000000800 x28: 0xffffa000135ca000 x29: 0xffff0000eb0ef250 (next_index + 0x815d5b0) sp: 0xffff0000eb0ef250 lr: 0xffff00000162bee8 (zfs_clone_range + 0x6c) elr: 0xffff0000004e0c44 (rms_rlock + 0x1c) spsr: 0x00000000a0000045 far: 0x0000000000000908 esr: 0x0000000096000004 panic: data abort in critical section or under mutex cpuid = 2 time = 1699966486 KDB: stack backtrace: db_trace_self() at db_trace_self db_trace_self_wrapper() at db_trace_self_wrapper+0x38 vpanic() at vpanic+0x1a0 panic() at panic+0x48 data_abort() at data_abort+0x2fc handle_el1h_sync() at handle_el1h_sync+0x18 --- exception, esr 0x96000004 rms_rlock() at rms_rlock+0x1c zfs_clone_range() at zfs_clone_range+0x68 zfs_freebsd_copy_file_range() at zfs_freebsd_copy_file_range+0x19c null_bypass() at null_bypass+0x118 vn_copy_file_range() at vn_copy_file_range+0x1c0 kern_copy_file_range() at kern_copy_file_range+0x36c sys_copy_file_range() at sys_copy_file_range+0x8c do_el0_sync() at do_el0_sync+0x634 handle_el0_sync() at handle_el0_sync+0x48 --- exception, esr 0x56000000 KDB: enter: panic [ thread pid 3620 tid 100911 ] Stopped at kdb_enter+0x48: str xzr, [x19, #768] db> This happens as soon as I start poudriere in a jenkins-agent jail. AFAIK this includes the two recent vn_copy_file_range changes of Konstantin. Next I will install a GENERIC kernel instead of GENERIC-NODEBUG. Regards, Ronald. ------=_Part_5967_382656657.1699966796420 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit Response below
 

Van: Konstantin Belousov <kostikbel@gmail.com>
Datum: zondag, 12 november 2023 19:47
Aan: Alexander Motin <mav@freebsd.org>
CC: Ronald Klop <ronald-lists@klop.ws>, current@freebsd.org
Onderwerp: Re: crash zfs_clone_range()

On Sun, Nov 12, 2023 at 11:51:40AM -0500, Alexander Motin wrote:
> Hi Ronald,
>
> As I can see, the clone request to ZFS came through nullfs, and it crashed
> immediately on enter.  I've never been a VFS layer expert, but to me it may
> be a nullfs problem, not zfs.  Is there chance you was (un-)mounting
> something when this happened?
It is not nullfs issue, I believe, but the lack of the busy reference on the
upper mount.  I think https://reviews.freebsd.org/D42554 should cover it.

>
> On 10.11.2023 05:12, Ronald Klop wrote:
> > Hi,
> >
> > Had this crash today on RPI4/15-CURRENT.
> >
> > FreeBSD rpi4 15.0-CURRENT FreeBSD 15.0-CURRENT #19
> > main-b0203aaa46-dirty: Sat Nov  4 11:48:33 CET 2023     ronald@rpi4:/home/ronald/dev/freebsd/obj/home/ronald/dev/freebsd/src/arm64.aarch64/sys/GENERIC-NODEBUG
> > arm64
> >
> > $ sysctl -a | grep bclon
> > vfs.zfs.bclone_enabled: 1
> >
> > I started a jail with poudriere to build a package. The jail uses null
> > mounts over ZFS.
> >
> > [root]# cu -s 115200 -l /dev/cuaU0
> > Connected
> >
> > db> bt
> > Tracing pid 95213 tid 100438 td 0xffff0000e1e97900
> > db_trace_self() at db_trace_self
> > db_stack_trace() at db_stack_trace+0x120
> > db_command() at db_command+0x2e4
> > db_command_loop() at db_command_loop+0x58
> > db_trap() at db_trap+0x100
> > kdb_trap() at kdb_trap+0x334
> > handle_el1h_sync() at handle_el1h_sync+0x18
> > --- exception, esr 0xf2000000
> > kdb_enter() at kdb_enter+0x48
> > vpanic() at vpanic+0x1dc
> > panic() at panic+0x48
> > data_abort() at data_abort+0x2fc
> > handle_el1h_sync() at handle_el1h_sync+0x18
> > --- exception, esr 0x96000004
> > rms_rlock() at rms_rlock+0x1c
> > zfs_clone_range() at zfs_clone_range+0x68
> > zfs_freebsd_copy_file_range() at zfs_freebsd_copy_file_range+0x19c
> > null_bypass() at null_bypass+0x118
> > vn_copy_file_range() at vn_copy_file_range+0x18c
> > kern_copy_file_range() at kern_copy_file_range+0x36c
> > sys_copy_file_range() at sys_copy_file_range+0x8c
> > do_el0_sync() at do_el0_sync+0x634
> > handle_el0_sync() at handle_el0_sync+0x48
> > --- exception, esr 0x56000000
> >
> >
> > Oh.. While typing this I rebooted the machine and it happened again. I
> > didn't start anything in particular although the machine runs some
> > jails.
> >
> > x0: 0x00000000000000e0
> >    x1: 0xffffa00090317a48
> >    x2: 0xffffa000f79d4f00
> >    x3: 0xffffa000c61a44a8
> >    x4: 0xffff0000deefe460 ($d.2 + 0xdd776560)
> >    x5: 0xffffa001250e4c00
> >    x6: 0xffff0000e54025b5 ($d.5 + 0xc)
> >    x7: 0x000000000000030a
> >    x8: 0xffff0000e1559000 ($d.2 + 0xdfdd1100)
> >    x9: 0x0000000000000001
> >   x10: 0x0000000000000000
> >   x11: 0x0000000000000001
> >   x12: 0x0000000000000002
> >   x13: 0x0000000000000000
> >   x14: 0x0000000000000001
> >   x15: 0x0000000000000000
> >   x16: 0xffff0000016dce88 (__stop_set_modmetadata_set + 0x1310)
> >   x17: 0xffff0000004e0d44 (rms_rlock + 0x0)
> >   x18: 0xffff0000deefe280 ($d.2 + 0xdd776380)
> >   x19: 0x0000000000000000
> >   x20: 0xffff0000deefe460 ($d.2 + 0xdd776560)
> >   x21: 0x7fffffffffffffff
> >   x22: 0xffffa00090317a48
> >   x23: 0xffffa000f79d4f00
> >   x24: 0xffffa001067ef910
> >   x25: 0x00000000000000e0
> >   x26: 0xffffa000158a8000
> >   x27: 0x0000000000000000
> >   x28: 0xffffa000158a8000
> >   x29: 0xffff0000deefe280 ($d.2 + 0xdd776380)
> >    sp: 0xffff0000deefe280
> >    lr: 0xffff000001623564 (zfs_clone_range + 0x6c)
> >   elr: 0xffff0000004e0d60 (rms_rlock + 0x1c)
> > spsr: 0x00000000a0000045
> >   far: 0x0000000000000108
> >   esr: 0x0000000096000004
> > panic: data abort in critical section or under mutex
> > cpuid = 1
> > time = 1699610885
> > KDB: stack backtrace:
> > db_trace_self() at db_trace_self
> > db_trace_self_wrapper() at db_trace_self_wrapper+0x38
> > vpanic() at vpanic+0x1a0
> > panic() at panic+0x48
> > data_abort() at data_abort+0x2fc
> > handle_el1h_sync() at handle_el1h_sync+0x18
> > --- exception, esr 0x96000004
> > rms_rlock() at rms_rlock+0x1c
> > zfs_clone_range() at zfs_clone_range+0x68
> > zfs_freebsd_copy_file_range() at zfs_freebsd_copy_file_range+0x19c
> > null_bypass() at null_bypass+0x118
> > vn_copy_file_range() at vn_copy_file_range+0x18c
> > kern_copy_file_range() at kern_copy_file_range+0x36c
> > sys_copy_file_range() at sys_copy_file_range+0x8c
> > do_el0_sync() at do_el0_sync+0x634
> > handle_el0_sync() at handle_el0_sync+0x48
> > --- exception, esr 0x56000000
> > KDB: enter: panic
> > [ thread pid 3792 tid 100394 ]
> > Stopped at      kdb_enter+0x48: str     xzr, [x19, #768]
> > db>
> >
> > I'll keep the debugger open for a while. Can I type something for
> > additional info?
> >
> > Regards,
> > Ronald.
>
> --
> Alexander Motin


Hi,

Build a new kernel today.
FreeBSD rpi4 15.0-CURRENT FreeBSD 15.0-CURRENT #20 main-051d69d6f8: Tue Nov 14 12:16:28 CET 2023     ronald@rpi4:/home/ronald/dev/freebsd/obj/home/ronald/dev/freebsd/src/arm64.aarch64/sys/GENERIC-NODEBUG arm64

x0: 0x00000000000008e0                                                                                                         
  x1: 0xffffa0006ce1fb38                                                                                                                
  x2: 0xffffa0006837a400                                                                                                                
  x3: 0xffffa0012c503a48                                                                                                                
  x4: 0xffff0000eb0ef430 (next_index + 0x815d790)                                                                                       
  x5: 0xffffa00152636300                                                                                                                
  x6: 0xffff0000e2e025b5 ($d.5 + 0xc)                                                                                                   
  x7: 0x000000000000030a                                                                                                                
  x8: 0xffff0000eb3212c0 (next_index + 0x838f620)                                                                                       
  x9: 0x0000000000000001                                                                                                                
 x10: 0x0000000000000000                                                                                                                
 x11: 0x0000000000000001                                                                                                                
 x12: 0x0000000000000002                                                                                                                
 x13: 0x0000000000000000                                                                                                                
 x14: 0x0000000000000001                                                                                                                
 x15: 0x0000000000000000
 x16: 0xffff0000016e5b58 (__stop_set_modmetadata_set + 0x1328)
 x17: 0xffff0000004e0c28 (rms_rlock + 0x0)
 x18: 0xffff0000eb0ef250 (next_index + 0x815d5b0)
 x19: 0x0000000000000800
 x20: 0xffff0000eb0ef430 (next_index + 0x815d790)
 x21: 0x7fffffffffffffff
 x22: 0xffffa0006ce1fb38
 x23: 0xffffa0006837a400
 x24: 0xffffa001ee486000
 x25: 0x00000000000008e0
 x26: 0xffffa000135ca000
 x27: 0x0000000000000800
 x28: 0xffffa000135ca000
 x29: 0xffff0000eb0ef250 (next_index + 0x815d5b0)
  sp: 0xffff0000eb0ef250
  lr: 0xffff00000162bee8 (zfs_clone_range + 0x6c)
 elr: 0xffff0000004e0c44 (rms_rlock + 0x1c)
spsr: 0x00000000a0000045
 far: 0x0000000000000908
 esr: 0x0000000096000004
panic: data abort in critical section or under mutex
cpuid = 2
time = 1699966486
KDB: stack backtrace:
db_trace_self() at db_trace_self
db_trace_self_wrapper() at db_trace_self_wrapper+0x38
vpanic() at vpanic+0x1a0
panic() at panic+0x48
data_abort() at data_abort+0x2fc
handle_el1h_sync() at handle_el1h_sync+0x18
--- exception, esr 0x96000004
rms_rlock() at rms_rlock+0x1c
zfs_clone_range() at zfs_clone_range+0x68
zfs_freebsd_copy_file_range() at zfs_freebsd_copy_file_range+0x19c
null_bypass() at null_bypass+0x118
vn_copy_file_range() at vn_copy_file_range+0x1c0
kern_copy_file_range() at kern_copy_file_range+0x36c
sys_copy_file_range() at sys_copy_file_range+0x8c
do_el0_sync() at do_el0_sync+0x634
handle_el0_sync() at handle_el0_sync+0x48
--- exception, esr 0x56000000
KDB: enter: panic
[ thread pid 3620 tid 100911 ]
Stopped at      kdb_enter+0x48: str     xzr, [x19, #768]
db>

This happens as soon as I start poudriere in a jenkins-agent jail.

AFAIK this includes the two recent vn_copy_file_range changes of Konstantin.

Next I will install a GENERIC kernel instead of GENERIC-NODEBUG.

Regards,
Ronald.
  ------=_Part_5967_382656657.1699966796420-- From nobody Tue Nov 14 15:21:45 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SV94z2fSsz51J0X for ; Tue, 14 Nov 2023 15:21:55 +0000 (UTC) (envelope-from SRS0=Pv4d=G3=klop.ws=ronald-lists@realworks.nl) Received: from smtp-relay-int.realworks.nl (smtp-relay-int.realworks.nl [194.109.157.24]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SV94y6F80z3Zw7; Tue, 14 Nov 2023 15:21:54 +0000 (UTC) (envelope-from SRS0=Pv4d=G3=klop.ws=ronald-lists@realworks.nl) Authentication-Results: mx1.freebsd.org; none Date: Tue, 14 Nov 2023 16:21:45 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=klop.ws; s=rw2; t=1699975306; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=pSS2kU5o/7lJjo5gS6AKRqy9ppWor89irnz+D+3QkEs=; b=D0j75Sd7hS1DPVEyz3Dw3ZV8zhaMIfKg9k7bi7B4o+QlOgsH+1XQPgRhxlbuKILhrpPnXB dm9f0nXi81ZroWRtA9kfQflPUaBmhj9/i3m9SWrfgkbZ0O3kLn1n3CVUbkL2JK+9Jaevuc ocgYuoA/1zWdXP7nv0def5QcvHjaVX9mZwgx3nRLctMqus1xjn9mp8fGu0l9UpCS5y0WVw yyxrQ4D4xUFnffCaglnmOJo+5CdZTCtpLAlXK5fBFtNA7vOM5uRFm4HybNRQdAnr5WRKj/ Gws+A5dDC93xnumLL7OhZ4woiu69oOJXAzZeBN7BMYUll3rKPjXU4s3DtHbRow== From: Ronald Klop To: Ronald Klop Cc: Alexander Motin , current@freebsd.org, Konstantin Belousov Message-ID: <1099306217.8124.1699975305932@localhost> In-Reply-To: <1900239445.5968.1699966796547@localhost> References: <349700057.3452.1699611152405@localhost> <1900239445.5968.1699966796547@localhost> Subject: Re: crash zfs_clone_range() List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_8123_18781866.1699975305923" X-Mailer: Realworks (679.16) Importance: Normal X-Priority: 3 (Normal) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:3265, ipnet:194.109.0.0/16, country:NL] X-Rspamd-Queue-Id: 4SV94y6F80z3Zw7 ------=_Part_8123_18781866.1699975305923 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Van: Ronald Klop Datum: dinsdag, 14 november 2023 13:59 Aan: Konstantin Belousov CC: Alexander Motin , current@freebsd.org Onderwerp: Re: crash zfs_clone_range() > > Response below > > Van: Konstantin Belousov > Datum: zondag, 12 november 2023 19:47 > Aan: Alexander Motin > CC: Ronald Klop , current@freebsd.org > Onderwerp: Re: crash zfs_clone_range() >> >> On Sun, Nov 12, 2023 at 11:51:40AM -0500, Alexander Motin wrote: >> > Hi Ronald, >> > >> > As I can see, the clone request to ZFS came through nullfs, and it crashed >> > immediately on enter. I've never been a VFS layer expert, but to me it may >> > be a nullfs problem, not zfs. Is there chance you was (un-)mounting >> > something when this happened? >> It is not nullfs issue, I believe, but the lack of the busy reference on the >> upper mount. I think https://reviews.freebsd.org/D42554 should cover it. >> >> > >> > On 10.11.2023 05:12, Ronald Klop wrote: >> > > Hi, >> > > >> > > Had this crash today on RPI4/15-CURRENT. >> > > >> > > FreeBSD rpi4 15.0-CURRENT FreeBSD 15.0-CURRENT #19 >> > > main-b0203aaa46-dirty: Sat Nov 4 11:48:33 CET 2023 ronald@rpi4:/home/ronald/dev/freebsd/obj/home/ronald/dev/freebsd/src/arm64.aarch64/sys/GENERIC-NODEBUG >> > > arm64 >> > > >> > > $ sysctl -a | grep bclon >> > > vfs.zfs.bclone_enabled: 1 >> > > >> > > I started a jail with poudriere to build a package. The jail uses null >> > > mounts over ZFS. >> > > >> > > [root]# cu -s 115200 -l /dev/cuaU0 >> > > Connected >> > > >> > > db> bt >> > > Tracing pid 95213 tid 100438 td 0xffff0000e1e97900 >> > > db_trace_self() at db_trace_self >> > > db_stack_trace() at db_stack_trace+0x120 >> > > db_command() at db_command+0x2e4 >> > > db_command_loop() at db_command_loop+0x58 >> > > db_trap() at db_trap+0x100 >> > > kdb_trap() at kdb_trap+0x334 >> > > handle_el1h_sync() at handle_el1h_sync+0x18 >> > > --- exception, esr 0xf2000000 >> > > kdb_enter() at kdb_enter+0x48 >> > > vpanic() at vpanic+0x1dc >> > > panic() at panic+0x48 >> > > data_abort() at data_abort+0x2fc >> > > handle_el1h_sync() at handle_el1h_sync+0x18 >> > > --- exception, esr 0x96000004 >> > > rms_rlock() at rms_rlock+0x1c >> > > zfs_clone_range() at zfs_clone_range+0x68 >> > > zfs_freebsd_copy_file_range() at zfs_freebsd_copy_file_range+0x19c >> > > null_bypass() at null_bypass+0x118 >> > > vn_copy_file_range() at vn_copy_file_range+0x18c >> > > kern_copy_file_range() at kern_copy_file_range+0x36c >> > > sys_copy_file_range() at sys_copy_file_range+0x8c >> > > do_el0_sync() at do_el0_sync+0x634 >> > > handle_el0_sync() at handle_el0_sync+0x48 >> > > --- exception, esr 0x56000000 >> > > >> > > >> > > Oh.. While typing this I rebooted the machine and it happened again. I >> > > didn't start anything in particular although the machine runs some >> > > jails. >> > > >> > > x0: 0x00000000000000e0 >> > > x1: 0xffffa00090317a48 >> > > x2: 0xffffa000f79d4f00 >> > > x3: 0xffffa000c61a44a8 >> > > x4: 0xffff0000deefe460 ($d.2 + 0xdd776560) >> > > x5: 0xffffa001250e4c00 >> > > x6: 0xffff0000e54025b5 ($d.5 + 0xc) >> > > x7: 0x000000000000030a >> > > x8: 0xffff0000e1559000 ($d.2 + 0xdfdd1100) >> > > x9: 0x0000000000000001 >> > > x10: 0x0000000000000000 >> > > x11: 0x0000000000000001 >> > > x12: 0x0000000000000002 >> > > x13: 0x0000000000000000 >> > > x14: 0x0000000000000001 >> > > x15: 0x0000000000000000 >> > > x16: 0xffff0000016dce88 (__stop_set_modmetadata_set + 0x1310) >> > > x17: 0xffff0000004e0d44 (rms_rlock + 0x0) >> > > x18: 0xffff0000deefe280 ($d.2 + 0xdd776380) >> > > x19: 0x0000000000000000 >> > > x20: 0xffff0000deefe460 ($d.2 + 0xdd776560) >> > > x21: 0x7fffffffffffffff >> > > x22: 0xffffa00090317a48 >> > > x23: 0xffffa000f79d4f00 >> > > x24: 0xffffa001067ef910 >> > > x25: 0x00000000000000e0 >> > > x26: 0xffffa000158a8000 >> > > x27: 0x0000000000000000 >> > > x28: 0xffffa000158a8000 >> > > x29: 0xffff0000deefe280 ($d.2 + 0xdd776380) >> > > sp: 0xffff0000deefe280 >> > > lr: 0xffff000001623564 (zfs_clone_range + 0x6c) >> > > elr: 0xffff0000004e0d60 (rms_rlock + 0x1c) >> > > spsr: 0x00000000a0000045 >> > > far: 0x0000000000000108 >> > > esr: 0x0000000096000004 >> > > panic: data abort in critical section or under mutex >> > > cpuid = 1 >> > > time = 1699610885 >> > > KDB: stack backtrace: >> > > db_trace_self() at db_trace_self >> > > db_trace_self_wrapper() at db_trace_self_wrapper+0x38 >> > > vpanic() at vpanic+0x1a0 >> > > panic() at panic+0x48 >> > > data_abort() at data_abort+0x2fc >> > > handle_el1h_sync() at handle_el1h_sync+0x18 >> > > --- exception, esr 0x96000004 >> > > rms_rlock() at rms_rlock+0x1c >> > > zfs_clone_range() at zfs_clone_range+0x68 >> > > zfs_freebsd_copy_file_range() at zfs_freebsd_copy_file_range+0x19c >> > > null_bypass() at null_bypass+0x118 >> > > vn_copy_file_range() at vn_copy_file_range+0x18c >> > > kern_copy_file_range() at kern_copy_file_range+0x36c >> > > sys_copy_file_range() at sys_copy_file_range+0x8c >> > > do_el0_sync() at do_el0_sync+0x634 >> > > handle_el0_sync() at handle_el0_sync+0x48 >> > > --- exception, esr 0x56000000 >> > > KDB: enter: panic >> > > [ thread pid 3792 tid 100394 ] >> > > Stopped at kdb_enter+0x48: str xzr, [x19, #768] >> > > db> >> > > >> > > I'll keep the debugger open for a while. Can I type something for >> > > additional info? >> > > >> > > Regards, >> > > Ronald. >> > >> > -- >> > Alexander Motin >> >> >> > > > Hi, > > Build a new kernel today. > FreeBSD rpi4 15.0-CURRENT FreeBSD 15.0-CURRENT #20 main-051d69d6f8: Tue Nov 14 12:16:28 CET 2023 ronald@rpi4:/home/ronald/dev/freebsd/obj/home/ronald/dev/freebsd/src/arm64.aarch64/sys/GENERIC-NODEBUG arm64 > > x0: 0x00000000000008e0 > x1: 0xffffa0006ce1fb38 > x2: 0xffffa0006837a400 > x3: 0xffffa0012c503a48 > x4: 0xffff0000eb0ef430 (next_index + 0x815d790) > x5: 0xffffa00152636300 > x6: 0xffff0000e2e025b5 ($d.5 + 0xc) > x7: 0x000000000000030a > x8: 0xffff0000eb3212c0 (next_index + 0x838f620) > x9: 0x0000000000000001 > x10: 0x0000000000000000 > x11: 0x0000000000000001 > x12: 0x0000000000000002 > x13: 0x0000000000000000 > x14: 0x0000000000000001 > x15: 0x0000000000000000 > x16: 0xffff0000016e5b58 (__stop_set_modmetadata_set + 0x1328) > x17: 0xffff0000004e0c28 (rms_rlock + 0x0) > x18: 0xffff0000eb0ef250 (next_index + 0x815d5b0) > x19: 0x0000000000000800 > x20: 0xffff0000eb0ef430 (next_index + 0x815d790) > x21: 0x7fffffffffffffff > x22: 0xffffa0006ce1fb38 > x23: 0xffffa0006837a400 > x24: 0xffffa001ee486000 > x25: 0x00000000000008e0 > x26: 0xffffa000135ca000 > x27: 0x0000000000000800 > x28: 0xffffa000135ca000 > x29: 0xffff0000eb0ef250 (next_index + 0x815d5b0) > sp: 0xffff0000eb0ef250 > lr: 0xffff00000162bee8 (zfs_clone_range + 0x6c) > elr: 0xffff0000004e0c44 (rms_rlock + 0x1c) > spsr: 0x00000000a0000045 > far: 0x0000000000000908 > esr: 0x0000000096000004 > panic: data abort in critical section or under mutex > cpuid = 2 > time = 1699966486 > KDB: stack backtrace: > db_trace_self() at db_trace_self > db_trace_self_wrapper() at db_trace_self_wrapper+0x38 > vpanic() at vpanic+0x1a0 > panic() at panic+0x48 > data_abort() at data_abort+0x2fc > handle_el1h_sync() at handle_el1h_sync+0x18 > --- exception, esr 0x96000004 > rms_rlock() at rms_rlock+0x1c > zfs_clone_range() at zfs_clone_range+0x68 > zfs_freebsd_copy_file_range() at zfs_freebsd_copy_file_range+0x19c > null_bypass() at null_bypass+0x118 > vn_copy_file_range() at vn_copy_file_range+0x1c0 > kern_copy_file_range() at kern_copy_file_range+0x36c > sys_copy_file_range() at sys_copy_file_range+0x8c > do_el0_sync() at do_el0_sync+0x634 > handle_el0_sync() at handle_el0_sync+0x48 > --- exception, esr 0x56000000 > KDB: enter: panic > [ thread pid 3620 tid 100911 ] > Stopped at kdb_enter+0x48: str xzr, [x19, #768] > db> > > This happens as soon as I start poudriere in a jenkins-agent jail. > > AFAIK this includes the two recent vn_copy_file_range changes of Konstantin. > > Next I will install a GENERIC kernel instead of GENERIC-NODEBUG. > > Regards, > Ronald. > This is while running a DEBUG kernel. FreeBSD 15.0-CURRENT #5 main-20e1f207cc: Tue Nov 14 15:44:40 CET 2023 ronald@rpi4:/home/ronald/dev/freebsd/obj/home/ronald/dev/freebsd/src/arm64.aarch64/sys/GENERIC arm64 acquiring duplicate lock of same type: "vnode interlock" 1st vnode interlock @ /home/ronald/dev/freebsd/src/sys/fs/nullfs/null_vnops.c:370 2nd vnode interlock @ /home/ronald/dev/freebsd/src/sys/fs/nullfs/null_vnops.c:370 stack backtrace: #0 0xffff00000050798c at witness_debugger+0x60 #1 0xffff00000046c25c at __mtx_lock_flags+0xac #2 0xffff0000e3014f14 at null_add_writecount+0x40 #3 0xffff0000e3014f44 at null_add_writecount+0x70 #4 0xffff0000005a08b4 at vn_open_vnode+0x278 #5 0xffff0000005a019c at vn_open_cred+0x484 #6 0xffff000000596cac at kern_openat+0x24c #7 0xffff0000008443f4 at do_el0_sync+0x59c #8 0xffff00000081e91c at handle_el0_sync+0x48 x0: 0x0000000000000000 x1: 0xffff000000a2dfeb (console_pausestr + 0xab0) x2: 0xffff0000009715fb (cam_status_table + 0x14a13) x3: 0xffffa000ba780d18 x4: 0xffff0000deef8430 (ucom_cons_softc + 0xdd66faa0) x5: 0xffffa001200d4900 x6: 0x0000808080808080 x7: 0xffffa001200d4900 x8: 0x0000000000000000 x9: 0x0000000000000000 x10: 0x0000000000000001 x11: 0x0000000000000001 x12: 0x0000000000000002 x13: 0x0000000000000000 x14: 0x0000000000010000 x15: 0x0000000000000001 x16: 0xffff00000172b258 (__stop_set_modmetadata_set + 0x1360) x17: 0xffff00000048ba24 (rms_rlock + 0x0) x18: 0xffff0000deef8240 (ucom_cons_softc + 0xdd66f8b0) x19: 0xffff0000e2ef9fd0 (ucom_cons_softc + 0xe1671640) x20: 0xffff0000deef8430 (ucom_cons_softc + 0xdd66faa0) x21: 0x7fffffffffffffff x22: 0xffffa000b9a169a8 x23: 0xffffa0018c7e4d00 x24: 0xffffa001a377acb0 x25: 0xffff0000e2ef9fd0 (ucom_cons_softc + 0xe1671640) x26: 0xffffa0001480f000 x27: 0xffff0000e2ef9ef0 (ucom_cons_softc + 0xe1671560) x28: 0xffffa0001480f000 x29: 0xffff0000deef8240 (ucom_cons_softc + 0xdd66f8b0) sp: 0xffff0000deef8240 lr: 0xffff00000048ba4c (rms_rlock + 0x28) elr: 0xffff00000048ba90 (rms_rlock + 0x6c) spsr: 0x0000000000000045 far: 0x0000000000000000 esr: 0x0000000096000004 panic: data abort in critical section or under mutex cpuid = 0 time = 1699974087 KDB: stack backtrace: db_trace_self() at db_trace_self db_trace_self_wrapper() at db_trace_self_wrapper+0x38 vpanic() at vpanic+0x1a0 panic() at panic+0x48 data_abort() at data_abort+0x31c handle_el1h_sync() at handle_el1h_sync+0x18 --- exception, esr 0x96000004 rms_rlock() at rms_rlock+0x6c zfs_clone_range() at zfs_clone_range+0x68 zfs_freebsd_copy_file_range() at zfs_freebsd_copy_file_range+0x1bc null_bypass() at null_bypass+0x118 vn_copy_file_range() at vn_copy_file_range+0x1b0 kern_copy_file_range() at kern_copy_file_range+0x37c sys_copy_file_range() at sys_copy_file_range+0x8c do_el0_sync() at do_el0_sync+0x59c handle_el0_sync() at handle_el0_sync+0x48 --- exception, esr 0x56000000 KDB: enter: panic [ thread pid 3703 tid 100223 ] Stopped at kdb_enter+0x48: str xzr, [x19, #896] db> I have coredump/textdump files and can DM if needed. BTW: while typing this the RPI rebooted poudriere was restarted by Jenkins and it now logs this message (not a panic): acquiring duplicate lock of same type: "vnode interlock" 1st vnode interlock @ /home/ronald/dev/freebsd/src/sys/fs/nullfs/null_vnops.c:370 2nd vnode interlock @ /home/ronald/dev/freebsd/src/sys/fs/nullfs/null_vnops.c:370 stack backtrace: #0 0xffff00000050798c at witness_debugger+0x60 #1 0xffff00000046c25c at __mtx_lock_flags+0xac #2 0xffff0000e5214f14 at null_add_writecount+0x40 #3 0xffff0000e5214f44 at null_add_writecount+0x70 #4 0xffff0000005a08b4 at vn_open_vnode+0x278 #5 0xffff0000005a019c at vn_open_cred+0x484 #6 0xffff000000596cac at kern_openat+0x24c #7 0xffff0000008443f4 at do_el0_sync+0x59c #8 0xffff00000081e91c at handle_el0_sync+0x48 The last duplicate lock log is with "vfs.zfs.bclone_enabled: 0". The panics are with "vfs.zfs.bclone_enabled: 1". Regards, Ronald. ------=_Part_8123_18781866.1699975305923 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit

Van: Ronald Klop <ronald-lists@klop.ws>
Datum: dinsdag, 14 november 2023 13:59
Aan: Konstantin Belousov <kostikbel@gmail.com>
CC: Alexander Motin <mav@freebsd.org>, current@freebsd.org
Onderwerp: Re: crash zfs_clone_range()

Response below
 

Van: Konstantin Belousov <kostikbel@gmail.com>
Datum: zondag, 12 november 2023 19:47
Aan: Alexander Motin <mav@freebsd.org>
CC: Ronald Klop <ronald-lists@klop.ws>, current@freebsd.org
Onderwerp: Re: crash zfs_clone_range()

On Sun, Nov 12, 2023 at 11:51:40AM -0500, Alexander Motin wrote:
> Hi Ronald,
>
> As I can see, the clone request to ZFS came through nullfs, and it crashed
> immediately on enter.  I've never been a VFS layer expert, but to me it may
> be a nullfs problem, not zfs.  Is there chance you was (un-)mounting
> something when this happened?
It is not nullfs issue, I believe, but the lack of the busy reference on the
upper mount.  I think https://reviews.freebsd.org/D42554 should cover it.

>
> On 10.11.2023 05:12, Ronald Klop wrote:
> > Hi,
> >
> > Had this crash today on RPI4/15-CURRENT.
> >
> > FreeBSD rpi4 15.0-CURRENT FreeBSD 15.0-CURRENT #19
> > main-b0203aaa46-dirty: Sat Nov  4 11:48:33 CET 2023     ronald@rpi4:/home/ronald/dev/freebsd/obj/home/ronald/dev/freebsd/src/arm64.aarch64/sys/GENERIC-NODEBUG
> > arm64
> >
> > $ sysctl -a | grep bclon
> > vfs.zfs.bclone_enabled: 1
> >
> > I started a jail with poudriere to build a package. The jail uses null
> > mounts over ZFS.
> >
> > [root]# cu -s 115200 -l /dev/cuaU0
> > Connected
> >
> > db> bt
> > Tracing pid 95213 tid 100438 td 0xffff0000e1e97900
> > db_trace_self() at db_trace_self
> > db_stack_trace() at db_stack_trace+0x120
> > db_command() at db_command+0x2e4
> > db_command_loop() at db_command_loop+0x58
> > db_trap() at db_trap+0x100
> > kdb_trap() at kdb_trap+0x334
> > handle_el1h_sync() at handle_el1h_sync+0x18
> > --- exception, esr 0xf2000000
> > kdb_enter() at kdb_enter+0x48
> > vpanic() at vpanic+0x1dc
> > panic() at panic+0x48
> > data_abort() at data_abort+0x2fc
> > handle_el1h_sync() at handle_el1h_sync+0x18
> > --- exception, esr 0x96000004
> > rms_rlock() at rms_rlock+0x1c
> > zfs_clone_range() at zfs_clone_range+0x68
> > zfs_freebsd_copy_file_range() at zfs_freebsd_copy_file_range+0x19c
> > null_bypass() at null_bypass+0x118
> > vn_copy_file_range() at vn_copy_file_range+0x18c
> > kern_copy_file_range() at kern_copy_file_range+0x36c
> > sys_copy_file_range() at sys_copy_file_range+0x8c
> > do_el0_sync() at do_el0_sync+0x634
> > handle_el0_sync() at handle_el0_sync+0x48
> > --- exception, esr 0x56000000
> >
> >
> > Oh.. While typing this I rebooted the machine and it happened again. I
> > didn't start anything in particular although the machine runs some
> > jails.
> >
> > x0: 0x00000000000000e0
> >    x1: 0xffffa00090317a48
> >    x2: 0xffffa000f79d4f00
> >    x3: 0xffffa000c61a44a8
> >    x4: 0xffff0000deefe460 ($d.2 + 0xdd776560)
> >    x5: 0xffffa001250e4c00
> >    x6: 0xffff0000e54025b5 ($d.5 + 0xc)
> >    x7: 0x000000000000030a
> >    x8: 0xffff0000e1559000 ($d.2 + 0xdfdd1100)
> >    x9: 0x0000000000000001
> >   x10: 0x0000000000000000
> >   x11: 0x0000000000000001
> >   x12: 0x0000000000000002
> >   x13: 0x0000000000000000
> >   x14: 0x0000000000000001
> >   x15: 0x0000000000000000
> >   x16: 0xffff0000016dce88 (__stop_set_modmetadata_set + 0x1310)
> >   x17: 0xffff0000004e0d44 (rms_rlock + 0x0)
> >   x18: 0xffff0000deefe280 ($d.2 + 0xdd776380)
> >   x19: 0x0000000000000000
> >   x20: 0xffff0000deefe460 ($d.2 + 0xdd776560)
> >   x21: 0x7fffffffffffffff
> >   x22: 0xffffa00090317a48
> >   x23: 0xffffa000f79d4f00
> >   x24: 0xffffa001067ef910
> >   x25: 0x00000000000000e0
> >   x26: 0xffffa000158a8000
> >   x27: 0x0000000000000000
> >   x28: 0xffffa000158a8000
> >   x29: 0xffff0000deefe280 ($d.2 + 0xdd776380)
> >    sp: 0xffff0000deefe280
> >    lr: 0xffff000001623564 (zfs_clone_range + 0x6c)
> >   elr: 0xffff0000004e0d60 (rms_rlock + 0x1c)
> > spsr: 0x00000000a0000045
> >   far: 0x0000000000000108
> >   esr: 0x0000000096000004
> > panic: data abort in critical section or under mutex
> > cpuid = 1
> > time = 1699610885
> > KDB: stack backtrace:
> > db_trace_self() at db_trace_self
> > db_trace_self_wrapper() at db_trace_self_wrapper+0x38
> > vpanic() at vpanic+0x1a0
> > panic() at panic+0x48
> > data_abort() at data_abort+0x2fc
> > handle_el1h_sync() at handle_el1h_sync+0x18
> > --- exception, esr 0x96000004
> > rms_rlock() at rms_rlock+0x1c
> > zfs_clone_range() at zfs_clone_range+0x68
> > zfs_freebsd_copy_file_range() at zfs_freebsd_copy_file_range+0x19c
> > null_bypass() at null_bypass+0x118
> > vn_copy_file_range() at vn_copy_file_range+0x18c
> > kern_copy_file_range() at kern_copy_file_range+0x36c
> > sys_copy_file_range() at sys_copy_file_range+0x8c
> > do_el0_sync() at do_el0_sync+0x634
> > handle_el0_sync() at handle_el0_sync+0x48
> > --- exception, esr 0x56000000
> > KDB: enter: panic
> > [ thread pid 3792 tid 100394 ]
> > Stopped at      kdb_enter+0x48: str     xzr, [x19, #768]
> > db>
> >
> > I'll keep the debugger open for a while. Can I type something for
> > additional info?
> >
> > Regards,
> > Ronald.
>
> --
> Alexander Motin


Hi,

Build a new kernel today.
FreeBSD rpi4 15.0-CURRENT FreeBSD 15.0-CURRENT #20 main-051d69d6f8: Tue Nov 14 12:16:28 CET 2023     ronald@rpi4:/home/ronald/dev/freebsd/obj/home/ronald/dev/freebsd/src/arm64.aarch64/sys/GENERIC-NODEBUG arm64

x0: 0x00000000000008e0                                                                                                         
  x1: 0xffffa0006ce1fb38                                                                                                                
  x2: 0xffffa0006837a400                                                                                                                
  x3: 0xffffa0012c503a48                                                                                                                
  x4: 0xffff0000eb0ef430 (next_index + 0x815d790)                                                                                       
  x5: 0xffffa00152636300                                                                                                                
  x6: 0xffff0000e2e025b5 ($d.5 + 0xc)                                                                                                   
  x7: 0x000000000000030a                                                                                                                
  x8: 0xffff0000eb3212c0 (next_index + 0x838f620)                                                                                       
  x9: 0x0000000000000001                                                                                                                
 x10: 0x0000000000000000                                                                                                                
 x11: 0x0000000000000001                                                                                                                
 x12: 0x0000000000000002                                                                                                                
 x13: 0x0000000000000000                                                                                                                
 x14: 0x0000000000000001                                                                                                                
 x15: 0x0000000000000000
 x16: 0xffff0000016e5b58 (__stop_set_modmetadata_set + 0x1328)
 x17: 0xffff0000004e0c28 (rms_rlock + 0x0)
 x18: 0xffff0000eb0ef250 (next_index + 0x815d5b0)
 x19: 0x0000000000000800
 x20: 0xffff0000eb0ef430 (next_index + 0x815d790)
 x21: 0x7fffffffffffffff
 x22: 0xffffa0006ce1fb38
 x23: 0xffffa0006837a400
 x24: 0xffffa001ee486000
 x25: 0x00000000000008e0
 x26: 0xffffa000135ca000
 x27: 0x0000000000000800
 x28: 0xffffa000135ca000
 x29: 0xffff0000eb0ef250 (next_index + 0x815d5b0)
  sp: 0xffff0000eb0ef250
  lr: 0xffff00000162bee8 (zfs_clone_range + 0x6c)
 elr: 0xffff0000004e0c44 (rms_rlock + 0x1c)
spsr: 0x00000000a0000045
 far: 0x0000000000000908
 esr: 0x0000000096000004
panic: data abort in critical section or under mutex
cpuid = 2
time = 1699966486
KDB: stack backtrace:
db_trace_self() at db_trace_self
db_trace_self_wrapper() at db_trace_self_wrapper+0x38
vpanic() at vpanic+0x1a0
panic() at panic+0x48
data_abort() at data_abort+0x2fc
handle_el1h_sync() at handle_el1h_sync+0x18
--- exception, esr 0x96000004
rms_rlock() at rms_rlock+0x1c
zfs_clone_range() at zfs_clone_range+0x68
zfs_freebsd_copy_file_range() at zfs_freebsd_copy_file_range+0x19c
null_bypass() at null_bypass+0x118
vn_copy_file_range() at vn_copy_file_range+0x1c0
kern_copy_file_range() at kern_copy_file_range+0x36c
sys_copy_file_range() at sys_copy_file_range+0x8c
do_el0_sync() at do_el0_sync+0x634
handle_el0_sync() at handle_el0_sync+0x48
--- exception, esr 0x56000000
KDB: enter: panic
[ thread pid 3620 tid 100911 ]
Stopped at      kdb_enter+0x48: str     xzr, [x19, #768]
db>

This happens as soon as I start poudriere in a jenkins-agent jail.

AFAIK this includes the two recent vn_copy_file_range changes of Konstantin.

Next I will install a GENERIC kernel instead of GENERIC-NODEBUG.

Regards,
Ronald.
 


This is while running a DEBUG kernel.
FreeBSD 15.0-CURRENT #5 main-20e1f207cc: Tue Nov 14 15:44:40 CET 2023
    ronald@rpi4:/home/ronald/dev/freebsd/obj/home/ronald/dev/freebsd/src/arm64.aarch64/sys/GENERIC arm64

acquiring duplicate lock of same type: "vnode interlock"                                                                         
 1st vnode interlock @ /home/ronald/dev/freebsd/src/sys/fs/nullfs/null_vnops.c:370                                                      
 2nd vnode interlock @ /home/ronald/dev/freebsd/src/sys/fs/nullfs/null_vnops.c:370                                                      
stack backtrace:                                                                                                                        
#0 0xffff00000050798c at witness_debugger+0x60                                                                                          
#1 0xffff00000046c25c at __mtx_lock_flags+0xac                                                                                          
#2 0xffff0000e3014f14 at null_add_writecount+0x40                                                                                       
#3 0xffff0000e3014f44 at null_add_writecount+0x70                                                                                       
#4 0xffff0000005a08b4 at vn_open_vnode+0x278                                                                                            
#5 0xffff0000005a019c at vn_open_cred+0x484                                                                                             
#6 0xffff000000596cac at kern_openat+0x24c                                                                                              
#7 0xffff0000008443f4 at do_el0_sync+0x59c                                                                                              
#8 0xffff00000081e91c at handle_el0_sync+0x48                                                                                           
  x0: 0x0000000000000000                                                                                                                
  x1: 0xffff000000a2dfeb (console_pausestr + 0xab0)                                                                                     
  x2: 0xffff0000009715fb (cam_status_table + 0x14a13)                                                                                   
  x3: 0xffffa000ba780d18                                                                                                                
  x4: 0xffff0000deef8430 (ucom_cons_softc + 0xdd66faa0)                                                                                 
  x5: 0xffffa001200d4900                                                                                                                
  x6: 0x0000808080808080                                                                                                                
  x7: 0xffffa001200d4900                                                                                                                
  x8: 0x0000000000000000                                                                                                                
  x9: 0x0000000000000000                                                                                                                
 x10: 0x0000000000000001                                                                                                                
 x11: 0x0000000000000001                                                                                                                
 x12: 0x0000000000000002                                                                                                                
 x13: 0x0000000000000000                                                                                                                
 x14: 0x0000000000010000                                                                                                                
 x15: 0x0000000000000001
 x16: 0xffff00000172b258 (__stop_set_modmetadata_set + 0x1360)
 x17: 0xffff00000048ba24 (rms_rlock + 0x0)
 x18: 0xffff0000deef8240 (ucom_cons_softc + 0xdd66f8b0)
 x19: 0xffff0000e2ef9fd0 (ucom_cons_softc + 0xe1671640)
 x20: 0xffff0000deef8430 (ucom_cons_softc + 0xdd66faa0)
 x21: 0x7fffffffffffffff
 x22: 0xffffa000b9a169a8
 x23: 0xffffa0018c7e4d00
 x24: 0xffffa001a377acb0
 x25: 0xffff0000e2ef9fd0 (ucom_cons_softc + 0xe1671640)
 x26: 0xffffa0001480f000
 x27: 0xffff0000e2ef9ef0 (ucom_cons_softc + 0xe1671560)
 x28: 0xffffa0001480f000
 x29: 0xffff0000deef8240 (ucom_cons_softc + 0xdd66f8b0)
  sp: 0xffff0000deef8240
  lr: 0xffff00000048ba4c (rms_rlock + 0x28)
 elr: 0xffff00000048ba90 (rms_rlock + 0x6c)
spsr: 0x0000000000000045
 far: 0x0000000000000000
 esr: 0x0000000096000004
panic: data abort in critical section or under mutex
cpuid = 0
time = 1699974087
KDB: stack backtrace:
db_trace_self() at db_trace_self
db_trace_self_wrapper() at db_trace_self_wrapper+0x38
vpanic() at vpanic+0x1a0
panic() at panic+0x48
data_abort() at data_abort+0x31c
handle_el1h_sync() at handle_el1h_sync+0x18
--- exception, esr 0x96000004
rms_rlock() at rms_rlock+0x6c
zfs_clone_range() at zfs_clone_range+0x68
zfs_freebsd_copy_file_range() at zfs_freebsd_copy_file_range+0x1bc
null_bypass() at null_bypass+0x118
vn_copy_file_range() at vn_copy_file_range+0x1b0
kern_copy_file_range() at kern_copy_file_range+0x37c
sys_copy_file_range() at sys_copy_file_range+0x8c
do_el0_sync() at do_el0_sync+0x59c
handle_el0_sync() at handle_el0_sync+0x48
--- exception, esr 0x56000000
KDB: enter: panic
[ thread pid 3703 tid 100223 ]
Stopped at      kdb_enter+0x48: str     xzr, [x19, #896]
db>


I have coredump/textdump files and can DM if needed.

BTW: while typing this the RPI rebooted poudriere was restarted by Jenkins and it now logs this message (not a panic):
acquiring duplicate lock of same type: "vnode interlock"
 1st vnode interlock @ /home/ronald/dev/freebsd/src/sys/fs/nullfs/null_vnops.c:370
 2nd vnode interlock @ /home/ronald/dev/freebsd/src/sys/fs/nullfs/null_vnops.c:370
stack backtrace:
#0 0xffff00000050798c at witness_debugger+0x60
#1 0xffff00000046c25c at __mtx_lock_flags+0xac
#2 0xffff0000e5214f14 at null_add_writecount+0x40
#3 0xffff0000e5214f44 at null_add_writecount+0x70
#4 0xffff0000005a08b4 at vn_open_vnode+0x278
#5 0xffff0000005a019c at vn_open_cred+0x484
#6 0xffff000000596cac at kern_openat+0x24c
#7 0xffff0000008443f4 at do_el0_sync+0x59c
#8 0xffff00000081e91c at handle_el0_sync+0x48

The last duplicate lock log is with "vfs.zfs.bclone_enabled: 0".

The panics are with "vfs.zfs.bclone_enabled: 1".

Regards,
Ronald.

  ------=_Part_8123_18781866.1699975305923-- From nobody Tue Nov 14 17:39:31 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SVD7p4gFhz51cX4 for ; Tue, 14 Nov 2023 17:39:34 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-oo1-xc2d.google.com (mail-oo1-xc2d.google.com [IPv6:2607:f8b0:4864:20::c2d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SVD7p2DNbz3XFZ; Tue, 14 Nov 2023 17:39:34 +0000 (UTC) (envelope-from mjguzik@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-oo1-xc2d.google.com with SMTP id 006d021491bc7-586ad15f9aaso3001168eaf.2; Tue, 14 Nov 2023 09:39:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1699983572; x=1700588372; darn=freebsd.org; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=dNZP2p4/JVoYw6TKkLaqHCaFa0CV8rJGX2CphEKzJMw=; b=dzXYbeNjQb7Xyd0C3AssMtkon2JipPCZE1zlO9CHKPiGTpAhimp7e/oUk/uwif7Iji l0klVCyqDZPBNXRzAqK7XuidsRUiHWTFTk4ourdyUy5VCscyt2mNUB3mscTkctKufGaj 8y1+T65c5wa5vkKHeszpzLQZjAKqAm37GrJb8x9IqUdbaUzUpSzyt4cbG0EvIYV7rWlQ /m1IcxRyJUZL4sq4XPt01txMwWioMQCnGh97WRjtgWUfxMlu4i3FJ/AKSU139Ideejbt QyxpdnhZZFhwUZvPRVNYxwbPXDgnCBOdVnpf81wqEz/4925OMPoAPWal7+yOQcoqEexN ZVZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699983572; x=1700588372; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=dNZP2p4/JVoYw6TKkLaqHCaFa0CV8rJGX2CphEKzJMw=; b=CHlvExuUDfaucnQ5aOARE7grhD1Mn6qUnkRBsZDQKNxJTIWf2xOxlpfcIN191sdU8L mitAzEyw0BJJ4uQHa3L6cdUJF5To8/ln95ISDKK2MzejTIYUV1+T6yeqRC0LgPmk1fLC UfGMaOGPvEcQrbD+bTtyKhWDkO0+75DQbpIcqq79kjGbn20/E+tCWCAHDaot/OCUnV1l ZGTo57tWdXHZ24YetGAMzCwxLAmUGE0I9Fg2FDR0enSqm54XAK+uBX58+MwLhPQ4/wHX GaVsC5wbCVsUGVieq5Po3YD/f9mZS4Vhdt99+4fVxWxc2egMcpVOouE3Nt+Mkfdm4bdq IFWg== X-Gm-Message-State: AOJu0Yxka7EUpQZn8a36oI83GlT5Eq5+jlX2cLK7oAh7e42/BjBvTerz XEWzZ3F4wlbENL1pcKBtvKMd1823046dG7Ihle8= X-Google-Smtp-Source: AGHT+IHdZHUTCnL0bhlN6jZEuiX5DzBvq3JtBJLkPp8jgDbRadkmHYL09pFcBUakhjw0BxSrQLw05LHYmfAubuAFS/s= X-Received: by 2002:a4a:bb0f:0:b0:581:f2de:25f8 with SMTP id f15-20020a4abb0f000000b00581f2de25f8mr2003093oop.0.1699983572491; Tue, 14 Nov 2023 09:39:32 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Received: by 2002:ac9:57d4:0:b0:4f0:1250:dd51 with HTTP; Tue, 14 Nov 2023 09:39:31 -0800 (PST) In-Reply-To: <1900239445.5968.1699966796547@localhost> References: <349700057.3452.1699611152405@localhost> <1900239445.5968.1699966796547@localhost> From: Mateusz Guzik Date: Tue, 14 Nov 2023 18:39:31 +0100 Message-ID: Subject: Re: crash zfs_clone_range() To: Ronald Klop Cc: Konstantin Belousov , Alexander Motin , current@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4SVD7p2DNbz3XFZ On 11/14/23, Ronald Klop wrote: > Response below > > Van: Konstantin Belousov > Datum: zondag, 12 november 2023 19:47 > Aan: Alexander Motin > CC: Ronald Klop , current@freebsd.org > Onderwerp: Re: crash zfs_clone_range() >> >> On Sun, Nov 12, 2023 at 11:51:40AM -0500, Alexander Motin wrote: >> > Hi Ronald, >> > >> > As I can see, the clone request to ZFS came through nullfs, and it >> > crashed >> > immediately on enter. I've never been a VFS layer expert, but to me it >> > may >> > be a nullfs problem, not zfs. Is there chance you was (un-)mounting >> > something when this happened? >> It is not nullfs issue, I believe, but the lack of the busy reference on >> the >> upper mount. I think https://reviews.freebsd.org/D42554 should cover it. >> >> > >> > On 10.11.2023 05:12, Ronald Klop wrote: >> > > Hi, >> > > >> > > Had this crash today on RPI4/15-CURRENT. >> > > >> > > FreeBSD rpi4 15.0-CURRENT FreeBSD 15.0-CURRENT #19 >> > > main-b0203aaa46-dirty: Sat Nov 4 11:48:33 CET 2023 >> > > ronald@rpi4:/home/ronald/dev/freebsd/obj/home/ronald/dev/freebsd/src/arm64.aarch64/sys/GENERIC-NODEBUG >> > > arm64 >> > > >> > > $ sysctl -a | grep bclon >> > > vfs.zfs.bclone_enabled: 1 >> > > >> > > I started a jail with poudriere to build a package. The jail uses >> > > null >> > > mounts over ZFS. >> > > >> > > [root]# cu -s 115200 -l /dev/cuaU0 >> > > Connected >> > > >> > > db> bt >> > > Tracing pid 95213 tid 100438 td 0xffff0000e1e97900 >> > > db_trace_self() at db_trace_self >> > > db_stack_trace() at db_stack_trace+0x120 >> > > db_command() at db_command+0x2e4 >> > > db_command_loop() at db_command_loop+0x58 >> > > db_trap() at db_trap+0x100 >> > > kdb_trap() at kdb_trap+0x334 >> > > handle_el1h_sync() at handle_el1h_sync+0x18 >> > > --- exception, esr 0xf2000000 >> > > kdb_enter() at kdb_enter+0x48 >> > > vpanic() at vpanic+0x1dc >> > > panic() at panic+0x48 >> > > data_abort() at data_abort+0x2fc >> > > handle_el1h_sync() at handle_el1h_sync+0x18 >> > > --- exception, esr 0x96000004 >> > > rms_rlock() at rms_rlock+0x1c >> > > zfs_clone_range() at zfs_clone_range+0x68 >> > > zfs_freebsd_copy_file_range() at zfs_freebsd_copy_file_range+0x19c >> > > null_bypass() at null_bypass+0x118 >> > > vn_copy_file_range() at vn_copy_file_range+0x18c >> > > kern_copy_file_range() at kern_copy_file_range+0x36c >> > > sys_copy_file_range() at sys_copy_file_range+0x8c >> > > do_el0_sync() at do_el0_sync+0x634 >> > > handle_el0_sync() at handle_el0_sync+0x48 >> > > --- exception, esr 0x56000000 >> > > >> > > >> > > Oh.. While typing this I rebooted the machine and it happened again. >> > > I >> > > didn't start anything in particular although the machine runs some >> > > jails. >> > > >> > > x0: 0x00000000000000e0 >> > > x1: 0xffffa00090317a48 >> > > x2: 0xffffa000f79d4f00 >> > > x3: 0xffffa000c61a44a8 >> > > x4: 0xffff0000deefe460 ($d.2 + 0xdd776560) >> > > x5: 0xffffa001250e4c00 >> > > x6: 0xffff0000e54025b5 ($d.5 + 0xc) >> > > x7: 0x000000000000030a >> > > x8: 0xffff0000e1559000 ($d.2 + 0xdfdd1100) >> > > x9: 0x0000000000000001 >> > > x10: 0x0000000000000000 >> > > x11: 0x0000000000000001 >> > > x12: 0x0000000000000002 >> > > x13: 0x0000000000000000 >> > > x14: 0x0000000000000001 >> > > x15: 0x0000000000000000 >> > > x16: 0xffff0000016dce88 (__stop_set_modmetadata_set + 0x1310) >> > > x17: 0xffff0000004e0d44 (rms_rlock + 0x0) >> > > x18: 0xffff0000deefe280 ($d.2 + 0xdd776380) >> > > x19: 0x0000000000000000 >> > > x20: 0xffff0000deefe460 ($d.2 + 0xdd776560) >> > > x21: 0x7fffffffffffffff >> > > x22: 0xffffa00090317a48 >> > > x23: 0xffffa000f79d4f00 >> > > x24: 0xffffa001067ef910 >> > > x25: 0x00000000000000e0 >> > > x26: 0xffffa000158a8000 >> > > x27: 0x0000000000000000 >> > > x28: 0xffffa000158a8000 >> > > x29: 0xffff0000deefe280 ($d.2 + 0xdd776380) >> > > sp: 0xffff0000deefe280 >> > > lr: 0xffff000001623564 (zfs_clone_range + 0x6c) >> > > elr: 0xffff0000004e0d60 (rms_rlock + 0x1c) >> > > spsr: 0x00000000a0000045 >> > > far: 0x0000000000000108 >> > > esr: 0x0000000096000004 >> > > panic: data abort in critical section or under mutex >> > > cpuid = 1 >> > > time = 1699610885 >> > > KDB: stack backtrace: >> > > db_trace_self() at db_trace_self >> > > db_trace_self_wrapper() at db_trace_self_wrapper+0x38 >> > > vpanic() at vpanic+0x1a0 >> > > panic() at panic+0x48 >> > > data_abort() at data_abort+0x2fc >> > > handle_el1h_sync() at handle_el1h_sync+0x18 >> > > --- exception, esr 0x96000004 >> > > rms_rlock() at rms_rlock+0x1c >> > > zfs_clone_range() at zfs_clone_range+0x68 >> > > zfs_freebsd_copy_file_range() at zfs_freebsd_copy_file_range+0x19c >> > > null_bypass() at null_bypass+0x118 >> > > vn_copy_file_range() at vn_copy_file_range+0x18c >> > > kern_copy_file_range() at kern_copy_file_range+0x36c >> > > sys_copy_file_range() at sys_copy_file_range+0x8c >> > > do_el0_sync() at do_el0_sync+0x634 >> > > handle_el0_sync() at handle_el0_sync+0x48 >> > > --- exception, esr 0x56000000 >> > > KDB: enter: panic >> > > [ thread pid 3792 tid 100394 ] >> > > Stopped at kdb_enter+0x48: str xzr, [x19, #768] >> > > db> >> > > >> > > I'll keep the debugger open for a while. Can I type something for >> > > additional info? >> > > >> > > Regards, >> > > Ronald. >> > >> > -- >> > Alexander Motin >> >> >> > > > Hi, > > Build a new kernel today. > FreeBSD rpi4 15.0-CURRENT FreeBSD 15.0-CURRENT #20 main-051d69d6f8: Tue Nov > 14 12:16:28 CET 2023 > ronald@rpi4:/home/ronald/dev/freebsd/obj/home/ronald/dev/freebsd/src/arm64.aarch64/sys/GENERIC-NODEBUG > arm64 > > x0: 0x00000000000008e0 > > x1: 0xffffa0006ce1fb38 > > x2: 0xffffa0006837a400 > > x3: 0xffffa0012c503a48 > > x4: 0xffff0000eb0ef430 (next_index + 0x815d790) > > x5: 0xffffa00152636300 > > x6: 0xffff0000e2e025b5 ($d.5 + 0xc) > > x7: 0x000000000000030a > > x8: 0xffff0000eb3212c0 (next_index + 0x838f620) > > x9: 0x0000000000000001 > > x10: 0x0000000000000000 > > x11: 0x0000000000000001 > > x12: 0x0000000000000002 > > x13: 0x0000000000000000 > > x14: 0x0000000000000001 > > x15: 0x0000000000000000 > x16: 0xffff0000016e5b58 (__stop_set_modmetadata_set + 0x1328) > x17: 0xffff0000004e0c28 (rms_rlock + 0x0) > x18: 0xffff0000eb0ef250 (next_index + 0x815d5b0) > x19: 0x0000000000000800 > x20: 0xffff0000eb0ef430 (next_index + 0x815d790) > x21: 0x7fffffffffffffff > x22: 0xffffa0006ce1fb38 > x23: 0xffffa0006837a400 > x24: 0xffffa001ee486000 > x25: 0x00000000000008e0 > x26: 0xffffa000135ca000 > x27: 0x0000000000000800 > x28: 0xffffa000135ca000 > x29: 0xffff0000eb0ef250 (next_index + 0x815d5b0) > sp: 0xffff0000eb0ef250 > lr: 0xffff00000162bee8 (zfs_clone_range + 0x6c) > elr: 0xffff0000004e0c44 (rms_rlock + 0x1c) > spsr: 0x00000000a0000045 > far: 0x0000000000000908 > esr: 0x0000000096000004 > panic: data abort in critical section or under mutex > cpuid = 2 > time = 1699966486 > KDB: stack backtrace: > db_trace_self() at db_trace_self > db_trace_self_wrapper() at db_trace_self_wrapper+0x38 > vpanic() at vpanic+0x1a0 > panic() at panic+0x48 > data_abort() at data_abort+0x2fc > handle_el1h_sync() at handle_el1h_sync+0x18 > --- exception, esr 0x96000004 > rms_rlock() at rms_rlock+0x1c > zfs_clone_range() at zfs_clone_range+0x68 > zfs_freebsd_copy_file_range() at zfs_freebsd_copy_file_range+0x19c > null_bypass() at null_bypass+0x118 > vn_copy_file_range() at vn_copy_file_range+0x1c0 > kern_copy_file_range() at kern_copy_file_range+0x36c > sys_copy_file_range() at sys_copy_file_range+0x8c > do_el0_sync() at do_el0_sync+0x634 > handle_el0_sync() at handle_el0_sync+0x48 > --- exception, esr 0x56000000 > KDB: enter: panic > [ thread pid 3620 tid 100911 ] > Stopped at kdb_enter+0x48: str xzr, [x19, #768] > db> > > This happens as soon as I start poudriere in a jenkins-agent jail. > > AFAIK this includes the two recent vn_copy_file_range changes of > Konstantin. > > Next I will install a GENERIC kernel instead of GENERIC-NODEBUG. > One of the vnodes is probably not zfs, I suspect this will do it (untested): diff --git a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c index 107cd69c756c..e799a7091b8e 100644 --- a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c +++ b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c @@ -6270,6 +6270,11 @@ zfs_freebsd_copy_file_range(struct vop_copy_file_range_args *ap) goto bad_write_fallback; } } + + if (invp->v_mount->mnt_vfc != outvp->v_mount->mnt_vfc) { + goto bad_write_fallback; + } + if (invp == outvp) { if (vn_lock(outvp, LK_EXCLUSIVE) != 0) { goto bad_write_fallback; -- Mateusz Guzik From nobody Tue Nov 14 17:44:58 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SVDG65516z50PYb for ; Tue, 14 Nov 2023 17:45:02 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-yw1-x1129.google.com (mail-yw1-x1129.google.com [IPv6:2607:f8b0:4864:20::1129]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SVDG6353Dz3ZcF for ; Tue, 14 Nov 2023 17:45:02 +0000 (UTC) (envelope-from mavbsd@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-yw1-x1129.google.com with SMTP id 00721157ae682-5b383b4184fso69416867b3.1 for ; Tue, 14 Nov 2023 09:45:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1699983901; x=1700588701; darn=freebsd.org; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:from:to:cc:subject:date:message-id:reply-to; bh=XIHs5jWKTP3bMasydA+O8p2OIhF9eoav3hqv1Nmtky0=; b=AL8oR04/rL338DMYEeTnlC4W0hKCUQdAFPe/JaOZ9og8koouyVe4eKzIdbjJYnnBsC uMwWeFx32lMt8jbbbknacR0QKXmlVHjzBBa/d+SlxY84wIruF2ZvC1jz6+ORI1ffdKK2 Mt4bHFaNybEJ4yh96SRsLemLfGNpbkb3CJlfkCvBcQzIiJy8zYehqOM2LgsPZegNPpww n3V3RMDaabJ++nUO+SyCChvZgbznSgnd3M2ncqJA2jb+pum3IL6Bw1UxoQqYLyGXxZLo +bXgRsI1OungPMuVuu6fGxokv++qeRqOPqXdQmw+YVqvMyx6289Tw2ojKA5eZHOt1tgz 96KA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699983901; x=1700588701; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :sender:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=XIHs5jWKTP3bMasydA+O8p2OIhF9eoav3hqv1Nmtky0=; b=UVPddsGy6dah1MMV9MG5AgQwwxH1WYz6/fydXCdt/QuqEl2Pb7qVoBNgTE1kWE4Rrg sbdVNDvSYdIZyWVvI2QeTxcy36vmE3/4PEC6gA9G6XLmTCrP7TZym1WRpTYxuhlGIquO phIWi04G2zkEIOP66Ud7K6CHnM5G63HigaOIvtMLMOxTGk3wfk7j2lb7Z8MrSurlw9oF dWdveb7VhS60CwP5UXP3QYzrDQQVwqLod3bZQyXlK8Lw6Uf/LJFOPH6aYIGHuSvWBoR0 waPpnrfWxzrEX6rQT2/7XsWwSiAVzQy7vJwJhPBijZ1LtwRylJfpojN69+oOMqZQNKRa GbGQ== X-Gm-Message-State: AOJu0YzmWim2zJHOtAdqeO84o3bPcIizhEs1HY4Nl2ugUMDY2vIsYl/n UNcX9AmWoDXCfnFrR+qz2lQ= X-Google-Smtp-Source: AGHT+IF0IVnOKXU/nENKAm08oQXEtO47s0zn2olq5zftiXti84SC4j69gQ7y//0+DVoXTdJk0N1wpg== X-Received: by 2002:a81:49c2:0:b0:5b3:23f7:4254 with SMTP id w185-20020a8149c2000000b005b323f74254mr10973969ywa.25.1699983901027; Tue, 14 Nov 2023 09:45:01 -0800 (PST) Received: from [10.230.45.5] ([38.32.73.2]) by smtp.gmail.com with ESMTPSA id w137-20020a0dd48f000000b0059f802fad40sm2451445ywd.22.2023.11.14.09.44.59 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 14 Nov 2023 09:45:00 -0800 (PST) Message-ID: Date: Tue, 14 Nov 2023 12:44:58 -0500 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.11.0 Subject: Re: crash zfs_clone_range() Content-Language: en-US To: Mateusz Guzik , Ronald Klop Cc: Konstantin Belousov , current@freebsd.org References: <349700057.3452.1699611152405@localhost> <1900239445.5968.1699966796547@localhost> From: Alexander Motin In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4SVDG6353Dz3ZcF On 14.11.2023 12:39, Mateusz Guzik wrote: > One of the vnodes is probably not zfs, I suspect this will do it (untested): > > diff --git a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > index 107cd69c756c..e799a7091b8e 100644 > --- a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > +++ b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > @@ -6270,6 +6270,11 @@ zfs_freebsd_copy_file_range(struct > vop_copy_file_range_args *ap) > goto bad_write_fallback; > } > } > + > + if (invp->v_mount->mnt_vfc != outvp->v_mount->mnt_vfc) { > + goto bad_write_fallback; > + } > + > if (invp == outvp) { > if (vn_lock(outvp, LK_EXCLUSIVE) != 0) { > goto bad_write_fallback; > vn_copy_file_range() verifies for that: /* * If the two vnodes are for the same file system type, call * VOP_COPY_FILE_RANGE(), otherwise call vn_generic_copy_file_range() * which can handle copies across multiple file system types. */ *lenp = len; if (inmp == outmp || strcmp(inmp->mnt_vfc->vfc_name, outmp->mnt_vfc->vfc_name) == 0) error = VOP_COPY_FILE_RANGE(invp, inoffp, outvp, outoffp, lenp, flags, incred, outcred, fsize_td); else error = vn_generic_copy_file_range(invp, inoffp, outvp, outoffp, lenp, flags, incred, outcred, fsize_td); -- Alexander Motin From nobody Tue Nov 14 17:47:46 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SVDKJ3Jbjz50Q4Q for ; Tue, 14 Nov 2023 17:47:48 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-ot1-x333.google.com (mail-ot1-x333.google.com [IPv6:2607:f8b0:4864:20::333]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SVDKJ1SxCz3bp7; Tue, 14 Nov 2023 17:47:48 +0000 (UTC) (envelope-from mjguzik@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ot1-x333.google.com with SMTP id 46e09a7af769-6ce29d1db6eso305a34.1; Tue, 14 Nov 2023 09:47:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1699984067; x=1700588867; darn=freebsd.org; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=dWPV1qHe+GNzGSS8/W4DoZiyrJya5tsWYRS5IPiYtSs=; b=Tuuz4L1DuSZ/0c2monm9ifhQapMa1sg2vaJMSj62DsSpySTRcKZMUWvDexBrPAvC3V TzzYWl0VaklUA1GAkAfo+lEFfzJ4u7IzzsQG/cWXCL4PLIyvYUYYe4l4Le0M1GooCNHr M/oGB47O5swEmMpCwfMamACKg3MJRUdNt+BRs+za2T7We8lSjMP4c0ZjBv+R8x2lyk1Y IlMqy1bejGfm8NV8Q5ZSOLi/aO6h+/ae8Dux9AgnUtewmGiBenBEmnOuFNb+vJ7aScZ2 UkM4bfhpio0Nt+j0QQxqi72RRh0GJtlWoO13jsbx1dWjoCL8ixzYMgO9XHwXtu7rcnvr 33hA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699984067; x=1700588867; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=dWPV1qHe+GNzGSS8/W4DoZiyrJya5tsWYRS5IPiYtSs=; b=HaqsNiIyeRb6keyI3aP2hVL+Vosmlbr3dSV3wiQFeSgsbplYpDa6U5oJKcTuOFeg6L bHH28adWWfIzDIU85l11QiN06v5bErmxobzHVCQdnzZu6dQrFUG6tAk6qzj80xGfAKOQ QUWrmZRzOegdCywVm/S0UPBpFbCRPb6O5i2Q9ahu80vF147kt8E5ULpPmhtUMYgF69hD vhYrZy9eu5SM2j948pdIat9Bdo0LDMR3ywGggAc5AfqdtjlZ5AnrkDE44n0HWBaKAZel 4vMClwtIM6DrcyX/ELoudI0+HED9JCjS/I2PJJWn4Q/DJvR+vBtJCx+2QlcKxczTS2qm d9Rw== X-Gm-Message-State: AOJu0Yya3cRWS5xI27aJS/ZbBK6qwubC/DJn5PUQ3tzC7YhcgUMTSfZ0 Hi2P3fKwk0kKhfVFVyH7P5lllz+B40gOqb859MzRF3FKNf4= X-Google-Smtp-Source: AGHT+IH6I9NZ6BU0iLp5Wc7K8ct4U1xurbN396QNpoBV++XREMf8BmpoXe0vpQPYmJL2ITwF2O17fYIXvJGd8iOndfg= X-Received: by 2002:a05:6830:922:b0:6bd:9e1c:93a6 with SMTP id v34-20020a056830092200b006bd9e1c93a6mr1443421ott.0.1699984066724; Tue, 14 Nov 2023 09:47:46 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Received: by 2002:ac9:57d4:0:b0:4f0:1250:dd51 with HTTP; Tue, 14 Nov 2023 09:47:46 -0800 (PST) In-Reply-To: References: <349700057.3452.1699611152405@localhost> <1900239445.5968.1699966796547@localhost> From: Mateusz Guzik Date: Tue, 14 Nov 2023 18:47:46 +0100 Message-ID: Subject: Re: crash zfs_clone_range() To: Alexander Motin Cc: Ronald Klop , Konstantin Belousov , current@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4SVDKJ1SxCz3bp7 On 11/14/23, Alexander Motin wrote: > On 14.11.2023 12:39, Mateusz Guzik wrote: >> One of the vnodes is probably not zfs, I suspect this will do it >> (untested): >> >> diff --git a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c >> b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c >> index 107cd69c756c..e799a7091b8e 100644 >> --- a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c >> +++ b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c >> @@ -6270,6 +6270,11 @@ zfs_freebsd_copy_file_range(struct >> vop_copy_file_range_args *ap) >> goto bad_write_fallback; >> } >> } >> + >> + if (invp->v_mount->mnt_vfc != outvp->v_mount->mnt_vfc) { >> + goto bad_write_fallback; >> + } >> + >> if (invp == outvp) { >> if (vn_lock(outvp, LK_EXCLUSIVE) != 0) { >> goto bad_write_fallback; >> > > vn_copy_file_range() verifies for that: > > /* > * If the two vnodes are for the same file system type, call > * VOP_COPY_FILE_RANGE(), otherwise call > vn_generic_copy_file_range() > * which can handle copies across multiple file system types. > */ > *lenp = len; > if (inmp == outmp || strcmp(inmp->mnt_vfc->vfc_name, > outmp->mnt_vfc->vfc_name) == 0) > error = VOP_COPY_FILE_RANGE(invp, inoffp, outvp, outoffp, > lenp, flags, incred, outcred, fsize_td); > else > error = vn_generic_copy_file_range(invp, inoffp, outvp, > outoffp, lenp, flags, incred, outcred, fsize_td); > > The crash at hand comes from nullfs. If "outward" vnodes are both nullfs, but only one underlying vnode is zfs, you get the above. -- Mateusz Guzik From nobody Tue Nov 14 18:15:24 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SVDxR0MCDz50T6s for ; Tue, 14 Nov 2023 18:15:39 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SVDxQ4YsSz3gJZ; Tue, 14 Nov 2023 18:15:38 +0000 (UTC) (envelope-from kostikbel@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: from tom.home (kib@localhost [127.0.0.1] (may be forged)) by kib.kiev.ua (8.17.1/8.17.1) with ESMTP id 3AEIFOCb085895; Tue, 14 Nov 2023 20:15:27 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 3AEIFOCb085895 Received: (from kostik@localhost) by tom.home (8.17.1/8.17.1/Submit) id 3AEIFOKI085894; Tue, 14 Nov 2023 20:15:24 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 14 Nov 2023 20:15:24 +0200 From: Konstantin Belousov To: Mateusz Guzik Cc: Alexander Motin , Ronald Klop , current@freebsd.org Subject: Re: crash zfs_clone_range() Message-ID: References: <349700057.3452.1699611152405@localhost> <1900239445.5968.1699966796547@localhost> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-14) on tom.home X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US] X-Rspamd-Queue-Id: 4SVDxQ4YsSz3gJZ On Tue, Nov 14, 2023 at 06:47:46PM +0100, Mateusz Guzik wrote: > On 11/14/23, Alexander Motin wrote: > > On 14.11.2023 12:39, Mateusz Guzik wrote: > >> One of the vnodes is probably not zfs, I suspect this will do it > >> (untested): > >> > >> diff --git a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > >> b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > >> index 107cd69c756c..e799a7091b8e 100644 > >> --- a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > >> +++ b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > >> @@ -6270,6 +6270,11 @@ zfs_freebsd_copy_file_range(struct > >> vop_copy_file_range_args *ap) > >> goto bad_write_fallback; > >> } > >> } > >> + > >> + if (invp->v_mount->mnt_vfc != outvp->v_mount->mnt_vfc) { > >> + goto bad_write_fallback; > >> + } > >> + > >> if (invp == outvp) { > >> if (vn_lock(outvp, LK_EXCLUSIVE) != 0) { > >> goto bad_write_fallback; > >> > > > > vn_copy_file_range() verifies for that: > > > > /* > > * If the two vnodes are for the same file system type, call > > * VOP_COPY_FILE_RANGE(), otherwise call > > vn_generic_copy_file_range() > > * which can handle copies across multiple file system types. > > */ > > *lenp = len; > > if (inmp == outmp || strcmp(inmp->mnt_vfc->vfc_name, > > outmp->mnt_vfc->vfc_name) == 0) > > error = VOP_COPY_FILE_RANGE(invp, inoffp, outvp, outoffp, > > lenp, flags, incred, outcred, fsize_td); > > else > > error = vn_generic_copy_file_range(invp, inoffp, outvp, > > outoffp, lenp, flags, incred, outcred, fsize_td); > > > > > > The crash at hand comes from nullfs. If "outward" vnodes are both > nullfs, but only one underlying vnode is zfs, you get the above. If this is the reason, the check must be done by nullfs bypass for vop_copy_file_range(). From nobody Tue Nov 14 18:45:55 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SVFcR5Tpnz50XW5 for ; Tue, 14 Nov 2023 18:45:59 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-yw1-x1130.google.com (mail-yw1-x1130.google.com [IPv6:2607:f8b0:4864:20::1130]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SVFcQ5pK6z4J3N for ; Tue, 14 Nov 2023 18:45:58 +0000 (UTC) (envelope-from mavbsd@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=jubGvThe; spf=pass (mx1.freebsd.org: domain of mavbsd@gmail.com designates 2607:f8b0:4864:20::1130 as permitted sender) smtp.mailfrom=mavbsd@gmail.com; dmarc=none Received: by mail-yw1-x1130.google.com with SMTP id 00721157ae682-5af6c445e9eso69368847b3.0 for ; Tue, 14 Nov 2023 10:45:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1699987558; x=1700592358; darn=freebsd.org; h=content-transfer-encoding:in-reply-to:subject:references:cc:to:from :content-language:user-agent:mime-version:date:message-id:sender :from:to:cc:subject:date:message-id:reply-to; bh=W/oazbLClpx202d58N+PDp7Ezl+dwm6+DPJLLyM87ME=; b=jubGvTheYhjuVQridkF/Mq59+hC9NqPnDsD7z9nMbEf+XBe7hgVzfZf92TcicNCXr7 PkM3fIiyNcCYjIVTw91dM7Umoh1TqEehh0SdBxH7XcnnWL9WUFZBxQNyEeezukMxnsjB CbQH66wnAA3ByGfmOVqtWNLYnv9VyTEHCViy1vbHo9AwzU0T0e1x9VW9r3L+f+6FDxqa MSB/oPVVLvKEG2P8MOido28gCChyWcMpvfqy7xhCU2kpXsYem1gmnwOxg62GK5Qi13Z2 kFO/9RyfZ0az/Nd+xjMNf8go6BxFoObK3Cvk2IjiF0sOJIpS+d7rffw3FVgtcYCL8Pnl b3oQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699987558; x=1700592358; h=content-transfer-encoding:in-reply-to:subject:references:cc:to:from :content-language:user-agent:mime-version:date:message-id:sender :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=W/oazbLClpx202d58N+PDp7Ezl+dwm6+DPJLLyM87ME=; b=QH//HDleEey/yEqgCtRNZHECVjY2SG7Zla1lTQ/MSHa2N/wCgK8+3uW5N3A+s0i6Ko ab7atm2kqgPGzEdUxL6xAaWzu4HCzSSKdhh0FHAjwbsTUxEeoD59qdARPq3noyQEiysI Og1jEr/9uGNTqPyUdRRyMvSBhk2P5PKtbh+mefAJo0yQfPD6Y2/1oXQ6wet5M+gbTQ14 r8+mSZdoUVyWOpyQfUZzPp8AZdhwhxd9eVc3QS4QLBTLaW2sWlHn41WaSPVZcWw+4YLW nTE/CRYWjg0/xpFelUdjaa6a5dUff7p9cT9VG5qdrz1F0o1wzNO5FdsL0hvcdOK6J9QR 28Lg== X-Gm-Message-State: AOJu0Yz+htIN30p0Zc16SM58UnaKGrecFyGtiNBq3m3i6ORTndskNPbT nXm+z83YX33yxuEzcwJ+YOTOEro99vA= X-Google-Smtp-Source: AGHT+IGjefWNadU0cmKr0BfxABeFbthZeeYVF5NgYxfqzGc2sKV/zsltCVjZ4upjh8qe/uf3rE9lcg== X-Received: by 2002:a0d:d9c5:0:b0:5a7:b942:c3fe with SMTP id b188-20020a0dd9c5000000b005a7b942c3femr10524572ywe.16.1699987557792; Tue, 14 Nov 2023 10:45:57 -0800 (PST) Received: from [10.230.45.5] ([38.32.73.2]) by smtp.gmail.com with ESMTPSA id b6-20020a0dc006000000b005b37c6e01f9sm2545685ywd.90.2023.11.14.10.45.57 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 14 Nov 2023 10:45:57 -0800 (PST) Message-ID: Date: Tue, 14 Nov 2023 13:45:55 -0500 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.11.0 Content-Language: en-US From: Alexander Motin To: Mateusz Guzik , Ronald Klop Cc: Konstantin Belousov , current@freebsd.org References: <349700057.3452.1699611152405@localhost> <1900239445.5968.1699966796547@localhost> Subject: Re: crash zfs_clone_range() In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spamd-Result: default: False [-3.08 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.88)[-0.877]; FORGED_SENDER(0.30)[mav@FreeBSD.org,mavbsd@gmail.com]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_THREE(0.00)[4]; FROM_HAS_DN(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[current@freebsd.org]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DMARC_NA(0.00)[freebsd.org]; MLMMJ_DEST(0.00)[current@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::1130:from]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FREEMAIL_TO(0.00)[gmail.com,klop.ws]; DKIM_TRACE(0.00)[gmail.com:+]; TO_DN_SOME(0.00)[]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MIME_TRACE(0.00)[0:+]; FROM_NEQ_ENVFROM(0.00)[mav@FreeBSD.org,mavbsd@gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4SVFcQ5pK6z4J3N X-Spamd-Bar: --- On 14.11.2023 12:44, Alexander Motin wrote: > On 14.11.2023 12:39, Mateusz Guzik wrote: >> One of the vnodes is probably not zfs, I suspect this will do it >> (untested): >> >> diff --git a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c >> b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c >> index 107cd69c756c..e799a7091b8e 100644 >> --- a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c >> +++ b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c >> @@ -6270,6 +6270,11 @@ zfs_freebsd_copy_file_range(struct >> vop_copy_file_range_args *ap) >>                          goto bad_write_fallback; >>                  } >>          } >> + >> +       if (invp->v_mount->mnt_vfc != outvp->v_mount->mnt_vfc) { >> +               goto bad_write_fallback; >> +       } >> + >>          if (invp == outvp) { >>                  if (vn_lock(outvp, LK_EXCLUSIVE) != 0) { >>                          goto bad_write_fallback; >> > > vn_copy_file_range() verifies for that: > >         /* >          * If the two vnodes are for the same file system type, call >          * VOP_COPY_FILE_RANGE(), otherwise call > vn_generic_copy_file_range() >          * which can handle copies across multiple file system types. >          */ >         *lenp = len; >         if (inmp == outmp || strcmp(inmp->mnt_vfc->vfc_name, >             outmp->mnt_vfc->vfc_name) == 0) >                 error = VOP_COPY_FILE_RANGE(invp, inoffp, outvp, outoffp, >                     lenp, flags, incred, outcred, fsize_td); >         else >                 error = vn_generic_copy_file_range(invp, inoffp, outvp, >                     outoffp, lenp, flags, incred, outcred, fsize_td); Thinking again, what happen if there are two nullfs mounts on top of two different file systems, one of which is indeed not ZFS? Do we need to add those checks to all ZFS, NFS and FUSE, implementing VOP_COPY_FILE_RANGE, or it is responsibility of nullfs or VFS? -- Alexander Motin From nobody Tue Nov 14 18:51:39 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SVFl161Vfz50Y3B for ; Tue, 14 Nov 2023 18:51:41 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-oi1-x235.google.com (mail-oi1-x235.google.com [IPv6:2607:f8b0:4864:20::235]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SVFl14MSyz4KVK; Tue, 14 Nov 2023 18:51:41 +0000 (UTC) (envelope-from mjguzik@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-oi1-x235.google.com with SMTP id 5614622812f47-3b566ee5f1dso3490041b6e.0; Tue, 14 Nov 2023 10:51:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1699987900; x=1700592700; darn=freebsd.org; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=4v7QXHe5aoXi30g0I7JM4laOLDYhIU6ZkjpeUvyr5sY=; b=FIiete8SupyBOKYfr93TcPKv7f+peRzmdIsYYnNACe7lNmBI7t3OIZKsZzC2qJTaF+ 6+ZhdbSRCfmvGD9Ygfsp912BxHz78PuKHNQw76duQlHKN7buof9jhQWnhdI01t0FNxoC lcE0ZAnVdkBrH566YhAJic1oXV8IxsHwXTsCmZL1h0Sj9OmSb8yl/Ai/AgD6aVnRWwhS 6A+rJeW/TPjooJM4YL9UhgNy/60cf9hLW6L2qBVfdIuOZcZoz/IL9omcpfw2IF+JYiNr zAFiMtpW8PVnZPYo7w6Ub9G0BgoCjUgSMfb/KWAJdAVWPHGeOxfZy8eNU7Kq9kaXq/+B 7Ahg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699987900; x=1700592700; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=4v7QXHe5aoXi30g0I7JM4laOLDYhIU6ZkjpeUvyr5sY=; b=Hk2xA000TIQG41IoM5Xca9wjLSZR7DOXpLL5sA4rFwmtkVTy+sBjrv38jvY4l86UvO aNVA4oDSvezr1P4WZgWQC6cOl0aakS8u2Fa2JVctDiJ4A2fcnLgtM1m8q4VeKzRZ5ZlF s4ngEKA0HVhrHOqyOn1Z0pwZDF97thj7EZdYA/TVjTjOT3/0np9WZHxTWVM/8n8WVpRC AuQLPTjnskowLDYXxB3yQh/tzyrLBJt4JkWgqXOAqhsxvRLJxg0bwDjdSWWTkNDSPesI roE66G/wnIboTDlbuFmabO2Lpa0ECxuythPCpFulwYznbWKiIkRK9AGVNIME7Uip4MNK 2tlg== X-Gm-Message-State: AOJu0YzDFDlX315nep9PRdqUs0aG875d598jT4fYg+Vd4JoJiHskg/uJ cbiCjGroY0mQ+7JuAICraQ7zKk/kyaAFM0elu1FDjSdTo4k= X-Google-Smtp-Source: AGHT+IFQ4jfyVSskwjzLiafsVfuNUvRI1KjKAbJfNMoFl7V7pgickP9jKpgdgj5z696mtNbdNeaJbyZcyLsOEO4E1wk= X-Received: by 2002:a05:6808:2a0b:b0:3a7:b4e8:563e with SMTP id ez11-20020a0568082a0b00b003a7b4e8563emr12290950oib.38.1699987900306; Tue, 14 Nov 2023 10:51:40 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Received: by 2002:ac9:57d4:0:b0:4f0:1250:dd51 with HTTP; Tue, 14 Nov 2023 10:51:39 -0800 (PST) In-Reply-To: References: <349700057.3452.1699611152405@localhost> <1900239445.5968.1699966796547@localhost> From: Mateusz Guzik Date: Tue, 14 Nov 2023 19:51:39 +0100 Message-ID: Subject: Re: crash zfs_clone_range() To: Alexander Motin Cc: Ronald Klop , Konstantin Belousov , current@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4SVFl14MSyz4KVK On 11/14/23, Alexander Motin wrote: > On 14.11.2023 12:44, Alexander Motin wrote: >> On 14.11.2023 12:39, Mateusz Guzik wrote: >>> One of the vnodes is probably not zfs, I suspect this will do it >>> (untested): >>> >>> diff --git a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c >>> b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c >>> index 107cd69c756c..e799a7091b8e 100644 >>> --- a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c >>> +++ b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c >>> @@ -6270,6 +6270,11 @@ zfs_freebsd_copy_file_range(struct >>> vop_copy_file_range_args *ap) >>> goto bad_write_fallback; >>> } >>> } >>> + >>> + if (invp->v_mount->mnt_vfc != outvp->v_mount->mnt_vfc) { >>> + goto bad_write_fallback; >>> + } >>> + >>> if (invp == outvp) { >>> if (vn_lock(outvp, LK_EXCLUSIVE) != 0) { >>> goto bad_write_fallback; >>> >> >> vn_copy_file_range() verifies for that: >> >> /* >> * If the two vnodes are for the same file system type, call >> * VOP_COPY_FILE_RANGE(), otherwise call >> vn_generic_copy_file_range() >> * which can handle copies across multiple file system types. >> */ >> *lenp = len; >> if (inmp == outmp || strcmp(inmp->mnt_vfc->vfc_name, >> outmp->mnt_vfc->vfc_name) == 0) >> error = VOP_COPY_FILE_RANGE(invp, inoffp, outvp, >> outoffp, >> lenp, flags, incred, outcred, fsize_td); >> else >> error = vn_generic_copy_file_range(invp, inoffp, outvp, >> outoffp, lenp, flags, incred, outcred, fsize_td); > > Thinking again, what happen if there are two nullfs mounts on top of two > different file systems, one of which is indeed not ZFS? Do we need to > add those checks to all ZFS, NFS and FUSE, implementing > VOP_COPY_FILE_RANGE, or it is responsibility of nullfs or VFS? > I already advocated for not trying to guess for filesystems what they can or cannot handle internally. That is to say vn_copy_file_range should call VOP_COPY_FILE_RANGE, that can try to figure out what to do and if it got nothing punt to a fallback. This already happens for some of the cases. -- Mateusz Guzik From nobody Tue Nov 14 20:13:48 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SVHZ03PY2z50kg1 for ; Tue, 14 Nov 2023 20:14:00 +0000 (UTC) (envelope-from kib@freebsd.org) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SVHZ006ggz4WgK; Tue, 14 Nov 2023 20:13:59 +0000 (UTC) (envelope-from kib@freebsd.org) Authentication-Results: mx1.freebsd.org; none Received: from tom.home (kib@localhost [127.0.0.1] (may be forged)) by kib.kiev.ua (8.17.1/8.17.1) with ESMTP id 3AEKDmeZ015220; Tue, 14 Nov 2023 22:13:51 +0200 (EET) (envelope-from kib@freebsd.org) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 3AEKDmeZ015220 Received: (from kostik@localhost) by tom.home (8.17.1/8.17.1/Submit) id 3AEKDmDm015219; Tue, 14 Nov 2023 22:13:48 +0200 (EET) (envelope-from kib@freebsd.org) X-Authentication-Warning: tom.home: kostik set sender to kib@freebsd.org using -f Date: Tue, 14 Nov 2023 22:13:48 +0200 From: Konstantin Belousov To: Mateusz Guzik Cc: Alexander Motin , Ronald Klop , current@freebsd.org Subject: Re: crash zfs_clone_range() Message-ID: References: <349700057.3452.1699611152405@localhost> <1900239445.5968.1699966796547@localhost> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-14) on tom.home X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US] X-Rspamd-Queue-Id: 4SVHZ006ggz4WgK On Tue, Nov 14, 2023 at 07:51:39PM +0100, Mateusz Guzik wrote: > On 11/14/23, Alexander Motin wrote: > > On 14.11.2023 12:44, Alexander Motin wrote: > >> On 14.11.2023 12:39, Mateusz Guzik wrote: > >>> One of the vnodes is probably not zfs, I suspect this will do it > >>> (untested): > >>> > >>> diff --git a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > >>> b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > >>> index 107cd69c756c..e799a7091b8e 100644 > >>> --- a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > >>> +++ b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > >>> @@ -6270,6 +6270,11 @@ zfs_freebsd_copy_file_range(struct > >>> vop_copy_file_range_args *ap) > >>> goto bad_write_fallback; > >>> } > >>> } > >>> + > >>> + if (invp->v_mount->mnt_vfc != outvp->v_mount->mnt_vfc) { > >>> + goto bad_write_fallback; > >>> + } > >>> + > >>> if (invp == outvp) { > >>> if (vn_lock(outvp, LK_EXCLUSIVE) != 0) { > >>> goto bad_write_fallback; > >>> > >> > >> vn_copy_file_range() verifies for that: > >> > >> /* > >> * If the two vnodes are for the same file system type, call > >> * VOP_COPY_FILE_RANGE(), otherwise call > >> vn_generic_copy_file_range() > >> * which can handle copies across multiple file system types. > >> */ > >> *lenp = len; > >> if (inmp == outmp || strcmp(inmp->mnt_vfc->vfc_name, > >> outmp->mnt_vfc->vfc_name) == 0) > >> error = VOP_COPY_FILE_RANGE(invp, inoffp, outvp, > >> outoffp, > >> lenp, flags, incred, outcred, fsize_td); > >> else > >> error = vn_generic_copy_file_range(invp, inoffp, outvp, > >> outoffp, lenp, flags, incred, outcred, fsize_td); > > > > Thinking again, what happen if there are two nullfs mounts on top of two > > different file systems, one of which is indeed not ZFS? Do we need to > > add those checks to all ZFS, NFS and FUSE, implementing > > VOP_COPY_FILE_RANGE, or it is responsibility of nullfs or VFS? > > > > I already advocated for not trying to guess for filesystems what they > can or cannot handle internally. > > That is to say vn_copy_file_range should call VOP_COPY_FILE_RANGE, > that can try to figure out what to do and if it got nothing punt to a > fallback. This already happens for some of the cases. > It is nullfs that is to blame there. See https://reviews.freebsd.org/D42603 From nobody Tue Nov 14 20:36:54 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SVJ4T4Jhvz50n27; Tue, 14 Nov 2023 20:36:57 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SVJ4T3ltlz4YwV; Tue, 14 Nov 2023 20:36:57 +0000 (UTC) (envelope-from gjb@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1699994217; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type; bh=qykkcsPOg4pmdCCUZGGgb9Sh/ouRz+KGeynLXW1uxmg=; b=nlxZgW/Mthu5snCOy9w1WNZ47P5gmMO0HFBKLjM28fM8/CCxUO6mFUvrCjwI26b4kMtG4r K6uZyPT4RBpC+1VdzYDtc+SMDB6PhNOVpQ5G1YwOzY2v1sKXFDk6QWKX7bhndJEQNYZjUq cAb80J6NvQpljiQVeUoIYthevSeqSXhyoTDrumYYkyAEzKsVNWO6DltaTnT4pQXf3zVFsn vb2Pxgwne74JGgXvCNTYmvYzhPzB+QyAekwevNYjNG3Cg2n8JsPXM35olGga4/syQcnM4M ELoA5ksdIcR96+ZA/N+0kWyo7OQkIaGKIYBmcOVMV7uu3e8erTs8Mkz+ZAhS9w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1699994217; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type; bh=qykkcsPOg4pmdCCUZGGgb9Sh/ouRz+KGeynLXW1uxmg=; b=hyDUXZ0m4eIg0O+KGUpwu9xeCl+kUKufi10R5Ovlf7I2Jqjz7bh1UcKISYbfeWU0na21/d gvAmH41L29EUuGwunrhH5ZbZBKNDCr9OfXB7700pbhIHm5cQcsq7gvur0M1QKE155IGXnH eG+aaaMTHXxlSFJr/OJH3TYa/42DbnAGxylF2ivxg2rLGdxxymPxU81ykq+p3seyVvgwMb ZO4Vljcxf6l310vJwShO7XLD1vL8Pmnqvd7mkdVQfyGBYEwnSlP05QBYzoe74e15gdLob1 e+iTMb5H8VjEt+NwwsUrZPxuQ8G9PahPqvhzJN40YTXqUUwQFJMYqMybu4aj3w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1699994217; a=rsa-sha256; cv=none; b=X8ZMuxDMUpG0kuCDwGcpCHS/lX+CYht8J4ZgEuJhB59UidCFa15yagPdBQ69KHXrgSHYya ACSPDAXmR5b89hief/aaooNkOEgR2Yh8RBIh+tUdcheqh9k62dwRwzY5WY+oVUVGtVDp21 ubyalmo9plLoH+RAGNY+nySTq7nyH4wArmSdZCjzsgcWOyim8gAkhiY+DyMfXetF3rHQel ymI7+sV8sEchiJssHj2iWQJ4EhMyQPw0PGbZBj5DNc+ndsAqnMJpQCN8TYwx6OfhJmnUcw QLXiEK7tXgdaODFJzCWWPD2ek//7EQpsHjHvoFZ5FjnpMMHToL0CVpSWHGpvmQ== Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by freefall.freebsd.org (Postfix) with ESMTPS id 117D81146B; Tue, 14 Nov 2023 20:36:57 +0000 (UTC) (envelope-from gjb@freebsd.org) Date: Tue, 14 Nov 2023 20:36:54 +0000 From: Glen Barber To: freebsd-current@freebsd.org, freebsd-stable@freebsd.org Cc: FreeBSD Release Engineering Team Subject: [HEADS-UP] Quick update to 14.0-RELEASE schedule Message-ID: <20231114203654.GB52320@FreeBSD.org> Reply-To: FreeBSD Release Engineering Team List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="l76fUT7nc3MelDdI" Content-Disposition: inline --l76fUT7nc3MelDdI Content-Type: text/plain; charset=us-ascii Content-Disposition: inline We are still waiting for a few (non-critical) things to complete before the announcement of 14.0-RELEASE will be ready. It should only be another day or so before these things complete. Thank you for your understanding. Glen On behalf of: re@ --l76fUT7nc3MelDdI Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEjRJAPC5sqwhs9k2jAxRYpUeP4pMFAmVT2mEACgkQAxRYpUeP 4pP/8w//YMcMGO5wL1Yo6Vk5rBxPf19w6fM654dgnKwqaUuUmZdhp2FMO9Omjyrn N57wihYQocEeUnTMwiwDClR1+14sGIqwy1FLEZTzZlBXYNIrv4yTEHHUrsYmAMMX 8QDbMbZzEXtLkzAsN7FifVrXHsAv+Rpbks/bbOxi7exEAsH+gHJ/oFxFWX0JY3AE EFFICDAPgxy7E49qeam24/YTZw73OsISA5DMx7wvoJLUfX+5DRX+hoqJ+Lw8fLbV OVlR3/t/M61SQtj8FdBoLpoQ1upoh1TB/670i4fsrY9DHlX99snU9Z15QFo9dmUS v1IgmGQ6S8H+MmYV3ltxywjJMl2Z3BzI+PDmyMwIoGaQsbh6zZQxqYWLGifAspfR rvZf8ryshgeqGYVSAjHPDy9SANdqDmEoJR2isx3ygmLofTmodnOyWmrOAYhQ9lhA Yixtp/wPXx2h5j3dF9jRRDjxtN5wAAgcHP+MIxcqFM4m6kOuic0sUC13xWsNkp8Y rHMwy9xpnksUlFu8t+NHNMw51sM1qzifFyDFBcLzzRtxcnG6YWX07qd123lxqItO 0tIbBxCzHZCNNUHZW3yh55X0A2B08w/oMgQcpcv5PS5mpVeV5SEegVSDf+TGN2O6 iYy4DIkQXOvnT6mzaM3t0NNNavpCX9IzLxyuu4oSA+7BlG9ovUc= =wVV2 -----END PGP SIGNATURE----- --l76fUT7nc3MelDdI-- From nobody Tue Nov 14 21:10:14 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SVJq74BTbz50rpR for ; Tue, 14 Nov 2023 21:10:27 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Received: from mail-pg1-x531.google.com (mail-pg1-x531.google.com [IPv6:2607:f8b0:4864:20::531]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SVJq63ZWTz4fHm; Tue, 14 Nov 2023 21:10:26 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pg1-x531.google.com with SMTP id 41be03b00d2f7-5bde80aad05so4889071a12.2; Tue, 14 Nov 2023 13:10:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1699996225; x=1700601025; darn=freebsd.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=c8sYlcqU5uErCmBPFA2HRUfpGnY0BiSsvcgVrmFhNWo=; b=gtd00OyH6K7uRtkD+4DpFgtQA4XKeEK7A/m2548+GPlFSwOdtFPdYNdW8njSfKj1zp 8LdUl1UT/uAzeDYOknlgSTL+4E2pIb5bmL9dw41g6iAWL5Nv6An9NfYBSM4J9sfQUyAM EhhIIvS3dKOorXmHF+5jh+yCwitdvPpuXzJ8Yu/hk6vsoGETQ8/63nVi4uUM0ItKcMDf AoLKodOx9k153BT4ZvQQ/485UARFfP+qqAHzWKTIo/HEdaqsVIaJgs8faDIzxQTpYECd nQmwD/fW1c8+WWyu5/FoPVOVMZaY9+5OZhH57oPZmnRxMCgL8GET8F62UuceuiF6WwpA dDcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699996225; x=1700601025; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=c8sYlcqU5uErCmBPFA2HRUfpGnY0BiSsvcgVrmFhNWo=; b=MLoB9eyiDYQwvgCsMDyR3LfPmAIFdVb2PkdG6/pTLE6EipX7LIyOwMBp3tAtu6MC2G 9J3mw9wuIiQJJYHrSS1wNRCog3sqpQAHoW4JFb6eYmeQShklcI0/bN5BXDuRolWd/ewz QB75rzFsqjgNXvsL6B0hQoA6GeQ11OzbrseL7MqfwM6fzzDzzXdzB+ONeORD0OZ6OPXm IHMddHRJPD6PfermiDfZUWbGDMOq7rX7pr4DWvc+ISubptLrky8KhIB4uALnGK4cpirt Tr6+oUBlrcL6/T3BJT/RUemdThydcAKrb8Qgk67I3CmQtKpzpJdb+aHsQxVOXaheOvIP bCNQ== X-Gm-Message-State: AOJu0Yzg7FzAGnG37+uzLIm8EnBNBfnh6mrOeBARfxOwoZZEWWsIWIFL FvYm7sTQmqnzI+rniinfn3wBBMo8X9XFc9Kg2oofXNvN0w== X-Google-Smtp-Source: AGHT+IFSx5uX5GoHlq760NOG2GSvNPT1BDMwbpZB7R5PWiksIK/IFqoyRNtoP7vfyvFsOljlVKih916uDZGLyH2cRTM= X-Received: by 2002:a17:90b:4f4e:b0:281:3a4:2220 with SMTP id pj14-20020a17090b4f4e00b0028103a42220mr10386882pjb.36.1699996225124; Tue, 14 Nov 2023 13:10:25 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: <349700057.3452.1699611152405@localhost> <1900239445.5968.1699966796547@localhost> In-Reply-To: From: Rick Macklem Date: Tue, 14 Nov 2023 13:10:14 -0800 Message-ID: Subject: Re: crash zfs_clone_range() To: Alexander Motin Cc: Mateusz Guzik , Ronald Klop , Konstantin Belousov , current@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[] X-Rspamd-Queue-Id: 4SVJq63ZWTz4fHm On Tue, Nov 14, 2023 at 10:46=E2=80=AFAM Alexander Motin = wrote: > > On 14.11.2023 12:44, Alexander Motin wrote: > > On 14.11.2023 12:39, Mateusz Guzik wrote: > >> One of the vnodes is probably not zfs, I suspect this will do it > >> (untested): > >> > >> diff --git a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > >> b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > >> index 107cd69c756c..e799a7091b8e 100644 > >> --- a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > >> +++ b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > >> @@ -6270,6 +6270,11 @@ zfs_freebsd_copy_file_range(struct > >> vop_copy_file_range_args *ap) > >> goto bad_write_fallback; > >> } > >> } > >> + > >> + if (invp->v_mount->mnt_vfc !=3D outvp->v_mount->mnt_vfc) { > >> + goto bad_write_fallback; > >> + } > >> + > >> if (invp =3D=3D outvp) { > >> if (vn_lock(outvp, LK_EXCLUSIVE) !=3D 0) { > >> goto bad_write_fallback; > >> > > > > vn_copy_file_range() verifies for that: > > > > /* > > * If the two vnodes are for the same file system type, call > > * VOP_COPY_FILE_RANGE(), otherwise call > > vn_generic_copy_file_range() > > * which can handle copies across multiple file system types. > > */ > > *lenp =3D len; > > if (inmp =3D=3D outmp || strcmp(inmp->mnt_vfc->vfc_name, > > outmp->mnt_vfc->vfc_name) =3D=3D 0) > > error =3D VOP_COPY_FILE_RANGE(invp, inoffp, outvp, out= offp, > > lenp, flags, incred, outcred, fsize_td); > > else > > error =3D vn_generic_copy_file_range(invp, inoffp, out= vp, > > outoffp, lenp, flags, incred, outcred, fsize_td); > > Thinking again, what happen if there are two nullfs mounts on top of two > different file systems, one of which is indeed not ZFS? Do we need to > add those checks to all ZFS, NFS and FUSE, implementing > VOP_COPY_FILE_RANGE, or it is responsibility of nullfs or VFS? Although it would be nice to do the check before the VOP call, I don't see an easy way to do that. It looks like the simple solution is to add a check in each of the VOP_COPY_FILE_RANGE() calls, such as mjg@ has proposed for ZFS. At this point there is only the three and I can easily do the NFS one. rick > > -- > Alexander Motin > From nobody Tue Nov 14 21:15:34 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SVJx45yRWz50sVC for ; Tue, 14 Nov 2023 21:15:36 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-oo1-xc30.google.com (mail-oo1-xc30.google.com [IPv6:2607:f8b0:4864:20::c30]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SVJx41fXyz3D1D; Tue, 14 Nov 2023 21:15:36 +0000 (UTC) (envelope-from mjguzik@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=K8Y1xnO9; spf=pass (mx1.freebsd.org: domain of mjguzik@gmail.com designates 2607:f8b0:4864:20::c30 as permitted sender) smtp.mailfrom=mjguzik@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-oo1-xc30.google.com with SMTP id 006d021491bc7-5844bc378feso3453898eaf.0; Tue, 14 Nov 2023 13:15:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1699996535; x=1700601335; darn=freebsd.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :references:in-reply-to:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=wSLD480hnKCj21lK15I1ejbrxRBXYmYErW+mqpuWpd0=; b=K8Y1xnO9hi1AGfMS59JoopAv92c+y1wizkLs3IHlHiIxdKVy5TWxAJuiSe/yta9Qd9 7nFZFN9k7pAkDI6Vw9JvWafHQQWtv1MK3xNAMQ/IYt76SkeCCk/skarvM81fcfG9yAol EX/X6KGeXEp6SqQ6AkCPMGwL0rH/X6MzVa9OqWPKFyIzcxgpcV7GpYRafzfWkYRgDvyq 75bqG0pXCZUwFwLETZ71m+9mgs8faTcmcVxvxpP17HSdWsNhIxTcWwshDTK2VN2GDRVv HxJd3enVqsAdmhS/jrIE7Bel/x1yrBC2ADTV7lTHfnWYd7GBEkX8cdUW+LJ37JytXtgQ sFQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699996535; x=1700601335; h=content-transfer-encoding:cc:to:subject:message-id:date:from :references:in-reply-to:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wSLD480hnKCj21lK15I1ejbrxRBXYmYErW+mqpuWpd0=; b=wgAf0Rja+hMIJsR1DgLDtB0J7cLMCqYRP/tIo2rawu1RHgGgX2hl/h9rR/rrREw4fP tgVrudQHw3n25ytpOsVIpSyBRsIGEi5XqwjmWt+b7uesT1q5NXHHuGaJbP6NtAhWrr4K NfbEONRYPqCV1IaBFPlIOVpzeWzGUmsOe17eLj4MJKSh7Z3SR25R58DcMys88b93J/7N VasNwpNXMXAgtRiJ8sHCC86664MOo0GklA308oG11FzTwHd+0Y7o+7GcDxlTZr9M4J2z ZTGGS4JfCISSyT44x0InmVnHCKTfi+iIY2Pey/Z70vm988glbLDfa8TbegN0V6Kxz8bi RpKw== X-Gm-Message-State: AOJu0Yy9CSpMFdjLfTFthK98RD6JmKluyyLdqs6wLSeLxmhaAWp1BX/D mZASkzuZ9GdNX4jcDp0S/ykgauhkgEJmfV/WTvs= X-Google-Smtp-Source: AGHT+IEE2yS+CedPbNYOyigdKbZbeV0ldVtFG56832RWAgyjWsMo9An7PAs22SXzpxYNHec6kRfb83QaVZw0NaYfc/8= X-Received: by 2002:a4a:9c51:0:b0:586:8c18:ddd9 with SMTP id c17-20020a4a9c51000000b005868c18ddd9mr10790774ook.9.1699996534744; Tue, 14 Nov 2023 13:15:34 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Received: by 2002:ac9:7510:0:b0:4f0:1250:dd51 with HTTP; Tue, 14 Nov 2023 13:15:34 -0800 (PST) In-Reply-To: References: <349700057.3452.1699611152405@localhost> <1900239445.5968.1699966796547@localhost> From: Mateusz Guzik Date: Tue, 14 Nov 2023 22:15:34 +0100 Message-ID: Subject: Re: crash zfs_clone_range() To: Rick Macklem Cc: Alexander Motin , Ronald Klop , Konstantin Belousov , current@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-2.49 / 15.00]; SUSPICIOUS_RECIPS(1.50)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.994]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::c30:from]; FROM_HAS_DN(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; TAGGED_RCPT(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MLMMJ_DEST(0.00)[current@freebsd.org]; ARC_NA(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FREEMAIL_CC(0.00)[freebsd.org,klop.ws,gmail.com]; MID_RHS_MATCH_FROMTLD(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_FIVE(0.00)[5]; FREEMAIL_FROM(0.00)[gmail.com]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[] X-Rspamd-Queue-Id: 4SVJx41fXyz3D1D X-Spamd-Bar: -- On 11/14/23, Rick Macklem wrote: > On Tue, Nov 14, 2023 at 10:46=E2=80=AFAM Alexander Motin wrote: >> >> On 14.11.2023 12:44, Alexander Motin wrote: >> > On 14.11.2023 12:39, Mateusz Guzik wrote: >> >> One of the vnodes is probably not zfs, I suspect this will do it >> >> (untested): >> >> >> >> diff --git a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c >> >> b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c >> >> index 107cd69c756c..e799a7091b8e 100644 >> >> --- a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c >> >> +++ b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c >> >> @@ -6270,6 +6270,11 @@ zfs_freebsd_copy_file_range(struct >> >> vop_copy_file_range_args *ap) >> >> goto bad_write_fallback; >> >> } >> >> } >> >> + >> >> + if (invp->v_mount->mnt_vfc !=3D outvp->v_mount->mnt_vfc) { >> >> + goto bad_write_fallback; >> >> + } >> >> + >> >> if (invp =3D=3D outvp) { >> >> if (vn_lock(outvp, LK_EXCLUSIVE) !=3D 0) { >> >> goto bad_write_fallback; >> >> >> > >> > vn_copy_file_range() verifies for that: >> > >> > /* >> > * If the two vnodes are for the same file system type, call >> > * VOP_COPY_FILE_RANGE(), otherwise call >> > vn_generic_copy_file_range() >> > * which can handle copies across multiple file system types. >> > */ >> > *lenp =3D len; >> > if (inmp =3D=3D outmp || strcmp(inmp->mnt_vfc->vfc_name, >> > outmp->mnt_vfc->vfc_name) =3D=3D 0) >> > error =3D VOP_COPY_FILE_RANGE(invp, inoffp, outvp, >> > outoffp, >> > lenp, flags, incred, outcred, fsize_td); >> > else >> > error =3D vn_generic_copy_file_range(invp, inoffp, >> > outvp, >> > outoffp, lenp, flags, incred, outcred, fsize_td); >> >> Thinking again, what happen if there are two nullfs mounts on top of two >> different file systems, one of which is indeed not ZFS? Do we need to >> add those checks to all ZFS, NFS and FUSE, implementing >> VOP_COPY_FILE_RANGE, or it is responsibility of nullfs or VFS? > Although it would be nice to do the check before the VOP call, I don't > see an easy way to do that. > > It looks like the simple solution is to add a check in each of the > VOP_COPY_FILE_RANGE() calls, such as mjg@ has proposed > for ZFS. At this point there is only the three and I can easily do the > NFS one. > All filesystems except for zfs are already covered because they check for mismatched mount. --=20 Mateusz Guzik From nobody Tue Nov 14 21:25:40 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SVK8z0Tymz50v1d for ; Tue, 14 Nov 2023 21:25:55 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Received: from mail-pf1-x434.google.com (mail-pf1-x434.google.com [IPv6:2607:f8b0:4864:20::434]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SVK8x0n51z3L4D; Tue, 14 Nov 2023 21:25:53 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pf1-x434.google.com with SMTP id d2e1a72fcca58-6c4d06b6ddaso4245494b3a.3; Tue, 14 Nov 2023 13:25:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1699997151; x=1700601951; darn=freebsd.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=72mr8112Mv6S9k5sb6qCalrEI5j9TALC8lqXQPRB0xY=; b=ISOhmbp4hAjYjWpn4kIWog3wuAGCYtYWvB//ziy7XGMi0EuH8vh5PYhihOA2dhr15T 5cVEmiiwM9QT6jHmTIhKScEsN78Jz3onrXS02UVvbylhnsdMv8s9ujSdGzvzwfPUzf8R lRQfsdkicvy72K6ZUX3WiFs/QxAsH/5Kg9BoC4SIYfZ+hbjUsJ+uFYW/SNgQhBjtVoSP 6xE8Jf85N7FS9tc0/9DEJlngtJ3NuJcOS3XUVUtTKHW5GA1wPoUii8MMGqtiJmo1s/WS 5Zhk6OuTyb7Aacm3A2RUOFkMcq6214I5nMmqMLUxK/5v9Y7rhJajPsDMCTdhGDs1iolC NaDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699997151; x=1700601951; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=72mr8112Mv6S9k5sb6qCalrEI5j9TALC8lqXQPRB0xY=; b=eqlO2o1XCWbcySVB5TXZsemE9uIm50Nl5qR+WufTkhA4jtX0H4yl3US4whdXU+1MZc R7d5AmT8MmaljP5HRmBbvKi89SclsPr6ZW6COS73IHUo8vrvIW8DMhkgTVFpCRKsWcxD OLr9AVDUPxPhrZznw0KFJUwisCh5LU/CTlDWq8xKzFFIe1hFB7OxtZl985jhMHRLVtAY kkK0qwW76ivGUqceRLADNsLNe3slgArUynDoDkagfwi0pYcXe9zIckmJhQd4eZ/0WFSv aJAFYtkNE8Q8E2wZdUK61J49senmQYqkPJi2dSLcnIJ8kE+c6EssjqYYoDmsqK2ZRmRC x7/w== X-Gm-Message-State: AOJu0YzROvZlvPFMGTkrbV+GEnZh/CUDoOeMvn5zFEElmZ3HWF/Ov94w Lm3L8REtZ6+Qs73X72SSp7a2bR9PllehB9+DGA== X-Google-Smtp-Source: AGHT+IHlBQGTNGQZY9YYsYsvbbKjkEOthfIJ5SG5pt4ru7wOjoThYw382tdVV4UIpSJlJc9PsjjZ4/2NbIlvfAa6pIM= X-Received: by 2002:a05:6a20:c901:b0:186:603b:6b4a with SMTP id gx1-20020a056a20c90100b00186603b6b4amr7228555pzb.32.1699997151134; Tue, 14 Nov 2023 13:25:51 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: <349700057.3452.1699611152405@localhost> <1900239445.5968.1699966796547@localhost> In-Reply-To: From: Rick Macklem Date: Tue, 14 Nov 2023 13:25:40 -0800 Message-ID: Subject: Re: crash zfs_clone_range() To: Mateusz Guzik Cc: Alexander Motin , Ronald Klop , Konstantin Belousov , current@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[] X-Rspamd-Queue-Id: 4SVK8x0n51z3L4D On Tue, Nov 14, 2023 at 1:15=E2=80=AFPM Mateusz Guzik w= rote: > > On 11/14/23, Rick Macklem wrote: > > On Tue, Nov 14, 2023 at 10:46=E2=80=AFAM Alexander Motin wrote: > >> > >> On 14.11.2023 12:44, Alexander Motin wrote: > >> > On 14.11.2023 12:39, Mateusz Guzik wrote: > >> >> One of the vnodes is probably not zfs, I suspect this will do it > >> >> (untested): > >> >> > >> >> diff --git a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os= .c > >> >> b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > >> >> index 107cd69c756c..e799a7091b8e 100644 > >> >> --- a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > >> >> +++ b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > >> >> @@ -6270,6 +6270,11 @@ zfs_freebsd_copy_file_range(struct > >> >> vop_copy_file_range_args *ap) > >> >> goto bad_write_fallback; > >> >> } > >> >> } > >> >> + > >> >> + if (invp->v_mount->mnt_vfc !=3D outvp->v_mount->mnt_vfc) { > >> >> + goto bad_write_fallback; > >> >> + } > >> >> + > >> >> if (invp =3D=3D outvp) { > >> >> if (vn_lock(outvp, LK_EXCLUSIVE) !=3D 0) { > >> >> goto bad_write_fallback; > >> >> > >> > > >> > vn_copy_file_range() verifies for that: > >> > > >> > /* > >> > * If the two vnodes are for the same file system type, cal= l > >> > * VOP_COPY_FILE_RANGE(), otherwise call > >> > vn_generic_copy_file_range() > >> > * which can handle copies across multiple file system type= s. > >> > */ > >> > *lenp =3D len; > >> > if (inmp =3D=3D outmp || strcmp(inmp->mnt_vfc->vfc_name, > >> > outmp->mnt_vfc->vfc_name) =3D=3D 0) > >> > error =3D VOP_COPY_FILE_RANGE(invp, inoffp, outvp, > >> > outoffp, > >> > lenp, flags, incred, outcred, fsize_td); > >> > else > >> > error =3D vn_generic_copy_file_range(invp, inoffp, > >> > outvp, > >> > outoffp, lenp, flags, incred, outcred, fsize_td= ); > >> > >> Thinking again, what happen if there are two nullfs mounts on top of t= wo > >> different file systems, one of which is indeed not ZFS? Do we need to > >> add those checks to all ZFS, NFS and FUSE, implementing > >> VOP_COPY_FILE_RANGE, or it is responsibility of nullfs or VFS? > > Although it would be nice to do the check before the VOP call, I don't > > see an easy way to do that. > > > > It looks like the simple solution is to add a check in each of the > > VOP_COPY_FILE_RANGE() calls, such as mjg@ has proposed > > for ZFS. At this point there is only the three and I can easily do the > > NFS one. > > > > All filesystems except for zfs are already covered because they check > for mismatched mount. Yes, now that the mount point(s) are busied. The NFS check is before the vnodes were locked, so it is unsafe without busying the mount points. (That was not my patch, but I missed the problem during review.) rick > > -- > Mateusz Guzik From nobody Tue Nov 14 21:30:25 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SVKGQ5BQyz50vFY for ; Tue, 14 Nov 2023 21:30:38 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Received: from mail-pj1-x1029.google.com (mail-pj1-x1029.google.com [IPv6:2607:f8b0:4864:20::1029]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SVKGQ3Mpfz3MRS; Tue, 14 Nov 2023 21:30:38 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pj1-x1029.google.com with SMTP id 98e67ed59e1d1-2809fb0027cso4866865a91.2; Tue, 14 Nov 2023 13:30:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1699997436; x=1700602236; darn=freebsd.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=ynpJJEgI87HbRiPj/s+rARS1qW0KMOY9dtNYc3x3LDY=; b=ClU/EJmL0iDJ6Cl57UzMRyzKt05/c404W7nBldjjU7f90z1C96AzeJw5lAAY/yhhus qgop35hCR/JUnsUZUwzLPvctaZjjByZwlCAAcoymwZm0i8A/uMqXLj/aMo85o7xn9qgP gABJEuRwvvvMBPgtcPyO+WI6kiL6JQhaX8F4pdvYK1omSfX4x7HDshLkC+NK8HAYmCBl bRdgEqjlvJeZPRupLJtE+hOfdybCgRwHeKl85XZoxKNBxWexNsSmowMthDxs3xc3Qfc3 xpLBANwBCschtdWBSRYmf/keSEZMpTr7nNyQtfFv7ul+wHdyCrRhbiuHUuE5CDj1319x Ilag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699997436; x=1700602236; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ynpJJEgI87HbRiPj/s+rARS1qW0KMOY9dtNYc3x3LDY=; b=ae99vg9jaCC/Q1YNWseAFfx62iCDLbkbLYkf/lQnbk1LDR8xj3zwVyMWV0bac6Qw8A B/11RpBttmQHDO+E91Z5sKk+pzs0HxeLCTaJ7Fx2/KffiyPpnpZGBCSB2MkeezrQ7dq7 wzRGJDAMnRJvT8vmCqzcPZCzSZyvq2kI5TXrpmUW7mIUe6Xasay+L0qMXxrMvSK06YXk M5LHjrLGsnSHduZkIXV8SB/cU9e3RMrrOjwvybZnPcJrZp3BFgNh4pLNtfgyEySZxkMT rlpextGAOPHpK0HYbshjhPSN7Dx6TUOgzzYqhVYOZ0uJMYGN8GQiuqIAZZQX78/gZcam u9oA== X-Gm-Message-State: AOJu0YwQUn8czGX6smMkjZP54GE74OSWulJqmjxBJFhnDKxDcybrAvl9 54qaDiY413WMALAmWHPA5h/BKV7iuvdRj1pC6g== X-Google-Smtp-Source: AGHT+IFzuQ+JasTzl9WKgl5prHr+cWpJNnHLT5NCaamn8ln+h3bx398yfQNv7sxFSM1Pak2SqEl4QTpUriYBqOsw8T8= X-Received: by 2002:a17:90a:e7c7:b0:26d:17da:5e9f with SMTP id kb7-20020a17090ae7c700b0026d17da5e9fmr8993577pjb.1.1699997435892; Tue, 14 Nov 2023 13:30:35 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: <349700057.3452.1699611152405@localhost> <1900239445.5968.1699966796547@localhost> In-Reply-To: From: Rick Macklem Date: Tue, 14 Nov 2023 13:30:25 -0800 Message-ID: Subject: Re: crash zfs_clone_range() To: Konstantin Belousov Cc: Mateusz Guzik , Alexander Motin , Ronald Klop , current@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4SVKGQ3Mpfz3MRS On Tue, Nov 14, 2023 at 1:20=E2=80=AFPM Konstantin Belousov wrote: > > On Tue, Nov 14, 2023 at 06:47:46PM +0100, Mateusz Guzik wrote: > > On 11/14/23, Alexander Motin wrote: > > > On 14.11.2023 12:39, Mateusz Guzik wrote: > > >> One of the vnodes is probably not zfs, I suspect this will do it > > >> (untested): > > >> > > >> diff --git a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.= c > > >> b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > > >> index 107cd69c756c..e799a7091b8e 100644 > > >> --- a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > > >> +++ b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > > >> @@ -6270,6 +6270,11 @@ zfs_freebsd_copy_file_range(struct > > >> vop_copy_file_range_args *ap) > > >> goto bad_write_fallback; > > >> } > > >> } > > >> + > > >> + if (invp->v_mount->mnt_vfc !=3D outvp->v_mount->mnt_vfc) { > > >> + goto bad_write_fallback; > > >> + } > > >> + > > >> if (invp =3D=3D outvp) { > > >> if (vn_lock(outvp, LK_EXCLUSIVE) !=3D 0) { > > >> goto bad_write_fallback; > > >> > > > > > > vn_copy_file_range() verifies for that: > > > > > > /* > > > * If the two vnodes are for the same file system type, call > > > * VOP_COPY_FILE_RANGE(), otherwise call > > > vn_generic_copy_file_range() > > > * which can handle copies across multiple file system types= . > > > */ > > > *lenp =3D len; > > > if (inmp =3D=3D outmp || strcmp(inmp->mnt_vfc->vfc_name, > > > outmp->mnt_vfc->vfc_name) =3D=3D 0) > > > error =3D VOP_COPY_FILE_RANGE(invp, inoffp, outvp, o= utoffp, > > > lenp, flags, incred, outcred, fsize_td); > > > else > > > error =3D vn_generic_copy_file_range(invp, inoffp, o= utvp, > > > outoffp, lenp, flags, incred, outcred, fsize_td)= ; > > > > > > > > > > The crash at hand comes from nullfs. If "outward" vnodes are both > > nullfs, but only one underlying vnode is zfs, you get the above. > > If this is the reason, the check must be done by nullfs bypass for > vop_copy_file_range(). I suppose this is a reasonable alternative, although it means that all stacked file systems will need the check. It just seems easier to do it in the actual VOPs, but it is up to others. Btw, the stuff above the VOP_COPY_FILE_RANGE() that busies the mounts and checks mnt_vfc being the same could be dropped, if the VOP_COPY_FILE_RANGE() calls like NFS were careful to lock the vnodes before doing a "same fs type or same mount" check. (I suppose that would be a subtle change in VOP semantics that is arguably not allowed for a minor version.) Anyhow, I am happy with whatever others decide. rick > From nobody Tue Nov 14 22:06:19 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SVL3l4r9gz50yxs for ; Tue, 14 Nov 2023 22:06:27 +0000 (UTC) (envelope-from kib@freebsd.org) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SVL3k6mw6z3TVC; Tue, 14 Nov 2023 22:06:26 +0000 (UTC) (envelope-from kib@freebsd.org) Authentication-Results: mx1.freebsd.org; dkim=none; spf=softfail (mx1.freebsd.org: 2001:470:d5e7:1::1 is neither permitted nor denied by domain of kib@freebsd.org) smtp.mailfrom=kib@freebsd.org; dmarc=none Received: from tom.home (kib@localhost [127.0.0.1] (may be forged)) by kib.kiev.ua (8.17.1/8.17.1) with ESMTP id 3AEM6JJE043497; Wed, 15 Nov 2023 00:06:22 +0200 (EET) (envelope-from kib@freebsd.org) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 3AEM6JJE043497 Received: (from kostik@localhost) by tom.home (8.17.1/8.17.1/Submit) id 3AEM6JXB043496; Wed, 15 Nov 2023 00:06:19 +0200 (EET) (envelope-from kib@freebsd.org) X-Authentication-Warning: tom.home: kostik set sender to kib@freebsd.org using -f Date: Wed, 15 Nov 2023 00:06:19 +0200 From: Konstantin Belousov To: Rick Macklem Cc: Mateusz Guzik , Alexander Motin , Ronald Klop , current@freebsd.org Subject: Re: crash zfs_clone_range() Message-ID: References: <349700057.3452.1699611152405@localhost> <1900239445.5968.1699966796547@localhost> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-14) on tom.home X-Spamd-Result: default: False [-3.10 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; FREEMAIL_TO(0.00)[gmail.com]; MLMMJ_DEST(0.00)[current@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; MIME_TRACE(0.00)[0:+]; R_SPF_SOFTFAIL(0.00)[~all]; RCPT_COUNT_FIVE(0.00)[5]; DMARC_NA(0.00)[freebsd.org]; FREEFALL_USER(0.00)[kib]; ARC_NA(0.00)[]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org,klop.ws]; FROM_HAS_DN(0.00)[]; TAGGED_RCPT(0.00)[]; TO_DN_SOME(0.00)[]; HAS_XAW(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4SVL3k6mw6z3TVC X-Spamd-Bar: --- On Tue, Nov 14, 2023 at 01:30:25PM -0800, Rick Macklem wrote: > On Tue, Nov 14, 2023 at 1:20 PM Konstantin Belousov wrote: > > > > On Tue, Nov 14, 2023 at 06:47:46PM +0100, Mateusz Guzik wrote: > > > On 11/14/23, Alexander Motin wrote: > > > > On 14.11.2023 12:39, Mateusz Guzik wrote: > > > >> One of the vnodes is probably not zfs, I suspect this will do it > > > >> (untested): > > > >> > > > >> diff --git a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > > > >> b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > > > >> index 107cd69c756c..e799a7091b8e 100644 > > > >> --- a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > > > >> +++ b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > > > >> @@ -6270,6 +6270,11 @@ zfs_freebsd_copy_file_range(struct > > > >> vop_copy_file_range_args *ap) > > > >> goto bad_write_fallback; > > > >> } > > > >> } > > > >> + > > > >> + if (invp->v_mount->mnt_vfc != outvp->v_mount->mnt_vfc) { > > > >> + goto bad_write_fallback; > > > >> + } > > > >> + > > > >> if (invp == outvp) { > > > >> if (vn_lock(outvp, LK_EXCLUSIVE) != 0) { > > > >> goto bad_write_fallback; > > > >> > > > > > > > > vn_copy_file_range() verifies for that: > > > > > > > > /* > > > > * If the two vnodes are for the same file system type, call > > > > * VOP_COPY_FILE_RANGE(), otherwise call > > > > vn_generic_copy_file_range() > > > > * which can handle copies across multiple file system types. > > > > */ > > > > *lenp = len; > > > > if (inmp == outmp || strcmp(inmp->mnt_vfc->vfc_name, > > > > outmp->mnt_vfc->vfc_name) == 0) > > > > error = VOP_COPY_FILE_RANGE(invp, inoffp, outvp, outoffp, > > > > lenp, flags, incred, outcred, fsize_td); > > > > else > > > > error = vn_generic_copy_file_range(invp, inoffp, outvp, > > > > outoffp, lenp, flags, incred, outcred, fsize_td); > > > > > > > > > > > > > > The crash at hand comes from nullfs. If "outward" vnodes are both > > > nullfs, but only one underlying vnode is zfs, you get the above. > > > > If this is the reason, the check must be done by nullfs bypass for > > vop_copy_file_range(). > I suppose this is a reasonable alternative, although it means that > all stacked file systems will need the check. > It just seems easier to do it in the actual VOPs, but it is up to others. In theory, eventually we can have much more implementations for VOP than VOP' callers. I.e. fixing all stacked fs means adding unionfs to my patch. Forcing this requirements on all future VOP_COPY_FILE_RANGE implementations is IMO not good. BTW, we already have zfs, nfs, and fuse implementing the VOP. > > Btw, the stuff above the VOP_COPY_FILE_RANGE() that busies the > mounts and checks mnt_vfc being the same could be dropped, if the > VOP_COPY_FILE_RANGE() calls like NFS were careful to lock the > vnodes before doing a "same fs type or same mount" check. > (I suppose that would be a subtle change in VOP semantics that > is arguably not allowed for a minor version.) > > Anyhow, I am happy with whatever others decide. > > rick > > > From nobody Wed Nov 15 00:15:48 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SVQFn71lDz51M2J; Wed, 15 Nov 2023 01:15:25 +0000 (UTC) (envelope-from doctor@doctor.nl2k.ab.ca) Received: from doctor.nl2k.ab.ca (doctor.nl2k.ab.ca [204.209.81.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SVQFn4MgGz4LxD; Wed, 15 Nov 2023 01:15:25 +0000 (UTC) (envelope-from doctor@doctor.nl2k.ab.ca) Authentication-Results: mx1.freebsd.org; none Received: from doctor by doctor.nl2k.ab.ca with local (Exim 4.97 (FreeBSD)) (envelope-from ) id 1r33ZM-00000000GoC-3qVJ; Tue, 14 Nov 2023 17:15:48 -0700 Date: Tue, 14 Nov 2023 17:15:48 -0700 From: The Doctor To: FreeBSD Release Engineering Team Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org Subject: Re: [HEADS-UP] Quick update to 14.0-RELEASE schedule Message-ID: References: <20231114203654.GB52320@FreeBSD.org> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20231114203654.GB52320@FreeBSD.org> X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:6171, ipnet:204.209.81.0/24, country:CA] X-Rspamd-Queue-Id: 4SVQFn4MgGz4LxD On Tue, Nov 14, 2023 at 08:36:54PM +0000, Glen Barber wrote: > We are still waiting for a few (non-critical) things to complete before > the announcement of 14.0-RELEASE will be ready. > > It should only be another day or so before these things complete. > > Thank you for your understanding. > I always just installed my copy. > Glen > On behalf of: re@ > -- Member - Liberal International This is doctor@nk.ca Ici doctor@nk.ca Yahweh, King & country!Never Satan President Republic!Beware AntiChrist rising! Look at Psalms 14 and 53 on Atheism ; unsubscribe from Google Groups to be seen Lest we forget 11 Nov 2023 Beware https://mindspring.com From nobody Wed Nov 15 02:27:01 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SVRrS6371z50W3s; Wed, 15 Nov 2023 02:27:04 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [96.47.72.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SVRrS5bgfz4TdJ; Wed, 15 Nov 2023 02:27:04 +0000 (UTC) (envelope-from gjb@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700015224; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=vgxJumFBFtGigime4v/APx4P69gUEZz1URaSzAGBz50=; b=nQzcZRhwXL3+LKini3ajwH8oX4hIxTnTVcPiKIeW9vZlgQXcOasFuTN5Cgukw/D048CCMF HFbfshDpYiczX6VceguBpqDiXV3wQf8QfEV3R2OqpPT3VyZve6+kwN2oVrtOGPcokNnaQS 0MUlyd/t4dx7abCV1U/1z4npAD+4VFA77suFt9DSnEJ8nogbMehZiMHucKYBQZ86fvzzb3 rSNWTuOQXurdPliWkffXhFtrCv6otbSj/ZTqv+oDx05vhg2UuEmr8IFsrKTz0q9ReE15Hr Tl9PCJCctk6HhDa75SAmoDHbsRYGNeA6amFfgVG7mBT+FK8l6x3K83r8OizTEQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700015224; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=vgxJumFBFtGigime4v/APx4P69gUEZz1URaSzAGBz50=; b=e9hGV2++KbbuSAq+nFS89b3CanIbJM7PN6KJdAIa4FyW6ukYoLAubpPju/m5sJzk5n83p6 vxSn+3d4gsz5HK+EjVK0uF5nJx4pgX/iLTwZhOUIKJm2neSW7UC5ei8xrVSbok7SW0risU 4Xb5UZGvvM5zSbfkRWxuIqiT9VjIIFbPJV17EsRFEonquYvBtHkzWaGDIXBZbXvHqNCWLS 9CP+Hhsusyu9Ieeu5AytvRgtqIKXKEnhk2lyG3Gm4rBy4Qdk1Gh+qTPUn5P5Ry3n6t5j2O CkpjFufOrlu2OZGewfSsEyUuGWdMOhOmaFQc/w9lMdwAb4qUJSDOYyHuXH8Svw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1700015224; a=rsa-sha256; cv=none; b=ADDngzQey8GD/8EgHdPUmsTmtPbYsknWvzSmqv+UpUa2LeUSuqfRAsxh0pzVdaoAnN1+lU sOqK0/geP29niCzEXoNHy2liOr3ZCOrP0SWI0XT2tOVplcGoMocns2lLJ65dwlVnJyalwJ cH8mq2MBr5ftpjjRybAcKe3Hiaekt5WHNfhP/cr7HHOlSx7h0m2AYTtoDMoeHfZccdtQb0 OySwnQ6nqDIhgA04r1Mcjm6O+rJ0U5i8eU1JBy1YVnUhoX8vV+ArPSV7lUSG75CDYX9qVW hcp06D1LU6IE93r/Fn5iosXarTfidpy0XTimDq8wdga4/mREQknRgdgDYJBDpQ== Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by freefall.freebsd.org (Postfix) with ESMTPS id 25F8D1300A; Wed, 15 Nov 2023 02:27:04 +0000 (UTC) (envelope-from gjb@freebsd.org) Date: Wed, 15 Nov 2023 02:27:01 +0000 From: Glen Barber To: The Doctor Cc: FreeBSD Release Engineering Team , freebsd-current@freebsd.org, freebsd-stable@freebsd.org Subject: Re: [HEADS-UP] Quick update to 14.0-RELEASE schedule Message-ID: <20231115022701.GM1307@FreeBSD.org> References: <20231114203654.GB52320@FreeBSD.org> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="AnoCzHCzU/Mgm7Oo" Content-Disposition: inline In-Reply-To: --AnoCzHCzU/Mgm7Oo Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Nov 14, 2023 at 05:15:48PM -0700, The Doctor wrote: > On Tue, Nov 14, 2023 at 08:36:54PM +0000, Glen Barber wrote: > > We are still waiting for a few (non-critical) things to complete before > > the announcement of 14.0-RELEASE will be ready. > >=20 > > It should only be another day or so before these things complete. > >=20 > > Thank you for your understanding. > > >=20 > I always just installed my copy. >=20 Ok. I do not know what exactly is your point, but releases are never official until there is a PGP-signed email sent. The email is intended for the general public of consumers of official releases, not "yeah, but"s. Glen --AnoCzHCzU/Mgm7Oo Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEjRJAPC5sqwhs9k2jAxRYpUeP4pMFAmVULG4ACgkQAxRYpUeP 4pPP4xAAhZJddCTp1QTAxhkEtGd+6fLjdcdZ3GM0SjmrIHYmVHqi2Le3CubzoGHN J3k0Aoo/vNVbYVDL2vVXYmw3xeU24yDdkjvSPQNoATprfPr/88r9lWKbrv1hQ0Vs Zuj7j4YDYyrPGeak4gP21BqoVs4/gR1jnQaIQIQLnYfV2qdE7kKhcDlT/oXGPwgB mbzhLYgBjrW2wk2NbKG22WwRCXYMVFC6OyUfEB0kAiLnrrD8QL6oL0Wdk5DR73x/ VwZQOiTjdKoBWh2HI9OLdarnRPiRCMF2i7MOjqmc2XPYEYpzWVPRHghtuUzZ00VM b+oqkl0KcbnjHe05XExY5BlTZfFSdAj45TmddCbkdXEnbq+sfml5vh7MLV5FEWqf 1pwq2A2HjpBTAmi7Yd5EVQLf9racxXSLeSt2JRs2Ibjmt5x8hhkDzpiBVaZ0ZM0n 41q0OgHnt5F9B6V0XrtlZyouvQCerzdo4JZIMER3rmdun19yd1xliLbHMEuoyWpo 7j4D2T7L9ETsliXh4CAjm1t0/84hFwGgWC46QNTRIsgdOyjtmu24MrOWlIFXL6aM bb4qvu0qpubwLTVYSJQojclUxzdB+OkeAwG9tyWxYBr7sMRhC14EHUu/k1JALcFR sjqjJyFMzXIjuQ6rWwYxXG/HlbKM0eT6QEH4msOG0qgMlid6Oqo= =9Ndh -----END PGP SIGNATURE----- --AnoCzHCzU/Mgm7Oo-- From nobody Wed Nov 15 03:10:23 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SVVFk1SPkz50kbm; Wed, 15 Nov 2023 04:15:38 +0000 (UTC) (envelope-from doctor@doctor.nl2k.ab.ca) Received: from doctor.nl2k.ab.ca (doctor.nl2k.ab.ca [204.209.81.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SVVFj6rtBz4kqr; Wed, 15 Nov 2023 04:15:37 +0000 (UTC) (envelope-from doctor@doctor.nl2k.ab.ca) Authentication-Results: mx1.freebsd.org; none Received: from doctor by doctor.nl2k.ab.ca with local (Exim 4.97 (FreeBSD)) (envelope-from ) id 1r36IJ-00000000GpY-2R0x; Tue, 14 Nov 2023 20:10:23 -0700 Date: Tue, 14 Nov 2023 20:10:23 -0700 From: The Doctor To: Glen Barber Cc: FreeBSD Release Engineering Team , freebsd-current@freebsd.org, freebsd-stable@freebsd.org Subject: Re: [HEADS-UP] Quick update to 14.0-RELEASE schedule Message-ID: References: <20231114203654.GB52320@FreeBSD.org> <20231115022701.GM1307@FreeBSD.org> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20231115022701.GM1307@FreeBSD.org> X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:6171, ipnet:204.209.81.0/24, country:CA] X-Rspamd-Queue-Id: 4SVVFj6rtBz4kqr On Wed, Nov 15, 2023 at 02:27:01AM +0000, Glen Barber wrote: > On Tue, Nov 14, 2023 at 05:15:48PM -0700, The Doctor wrote: > > On Tue, Nov 14, 2023 at 08:36:54PM +0000, Glen Barber wrote: > > > We are still waiting for a few (non-critical) things to complete before > > > the announcement of 14.0-RELEASE will be ready. > > > > > > It should only be another day or so before these things complete. > > > > > > Thank you for your understanding. > > > > > > > I always just installed my copy. > > > > Ok. I do not know what exactly is your point, but releases are never > official until there is a PGP-signed email sent. The email is intended > for the general public of consumers of official releases, not "yeah, > but"s. > Howver if you do a freebsd-update upgrade, you can upgrade. Is that suppose to happen? > Glen > -- Member - Liberal International This is doctor@nk.ca Ici doctor@nk.ca Yahweh, King & country!Never Satan President Republic!Beware AntiChrist rising! Look at Psalms 14 and 53 on Atheism ; unsubscribe from Google Groups to be seen Lest we forget 11 Nov 2023 Beware https://mindspring.com From nobody Wed Nov 15 04:52:31 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SVW4M0blbz50p6Z; Wed, 15 Nov 2023 04:52:35 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [96.47.72.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SVW4M06Lvz4pRT; Wed, 15 Nov 2023 04:52:35 +0000 (UTC) (envelope-from gjb@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700023955; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=mX4GQf6FmvbBRuMX+aZI1rFBmzbcHYQQnSk1dJPRaJg=; b=d04La5ESRXq9JReFvdaQ2e1Xq+tjc3I19rx3vjZ1yN7mE0teWdczJBENPVefMiz6vum/f8 XguGwJAKfNBp459ID1ZsXAmtdVlxe7A7fWKvgBuCuGmlQ6ZoWcE6AxJIJ0dsB3XjBI7nmX 7HkEnjajwkT3lkfGK3+4tBUmjSBvlRhAkZbX1bRRzC+vNPYAE3KOUsrwUrowQYMfSsVj/8 a8DlPJBZpMFO3a0ZB/oMF+VMaJz2t0Wqr51PvMhpO6odE4DtRq1G2VsT+sGNQcM0eGclP1 EMiqYq4Wgd7ZHzdir3XYFNdTZPBz3aHlLVyrej3Q976FAtTczMMkmg3f/trbbg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700023955; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=mX4GQf6FmvbBRuMX+aZI1rFBmzbcHYQQnSk1dJPRaJg=; b=SooxsbdLaMJfs5OqgjvxZ1mHG6AWDdeLNgXtjp6LGl/0lLtM8jvRAVmbb25cphkI1JH773 pirPABEhJjEeBOAKqlUdWxDK1uzhCEjC8bBgaxUhGPSEO3tSYKbJnjy41dLiD4Iw9q5OAY 3gs3yM65AvICX5kxCTK0Hu5FDA/fO9Pjhla4uVQm5dzLDinf8WPnzHrEw3k2REn/zw5cgy IvIR4qg+qTYK7oH39/L2Ui1zgviN46TUnWTj6nTM0rjxdlh4rEHHUhUB2pVOgaLhTbuo+t w6YbIh3CGV+4P8Gb4f3m9je+2iDrMhq9f2OmRuVBinvYG+Em1o8AJW1RlC2exg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1700023955; a=rsa-sha256; cv=none; b=hmmdVvkb6C9Ad9Au+C6kp9wCGFEsJaxnujjT652uV23jzRaxhyI68V7uAIIaPpoDlK5Gex iGXbwtE+OlqPueIw7vmS/8MOgu5c3bXs/s6JwMiHlgcy/mtBM600Mq9KxXDSkOIlyGlgHD BVokD71KIBORBkG8K7i5qctdoctcTq9bPT6z3iP/vW/26fjN4eyAdUSR98OZjXrsv0Yat6 5zbA4+KcatsWyyDwUd/NbYmgDurIAl1nVd9vIrkb1s1I81I8G7138gyCHpuAEyi1+2QhCX stDixy6S+cPPfGb7s7xgfxMX1kLvP3poH69ypf0MJnq2znh5dZhtnZThS5Yegg== Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by freefall.freebsd.org (Postfix) with ESMTPS id 9C54413660; Wed, 15 Nov 2023 04:52:34 +0000 (UTC) (envelope-from gjb@freebsd.org) Date: Wed, 15 Nov 2023 04:52:31 +0000 From: Glen Barber To: The Doctor Cc: FreeBSD Release Engineering Team , freebsd-current@freebsd.org, freebsd-stable@freebsd.org Subject: Re: [HEADS-UP] Quick update to 14.0-RELEASE schedule Message-ID: <20231115045231.GN1307@FreeBSD.org> References: <20231114203654.GB52320@FreeBSD.org> <20231115022701.GM1307@FreeBSD.org> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="OmEpep6L+R/25PJj" Content-Disposition: inline In-Reply-To: --OmEpep6L+R/25PJj Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Nov 14, 2023 at 08:10:23PM -0700, The Doctor wrote: > On Wed, Nov 15, 2023 at 02:27:01AM +0000, Glen Barber wrote: > > On Tue, Nov 14, 2023 at 05:15:48PM -0700, The Doctor wrote: > > > On Tue, Nov 14, 2023 at 08:36:54PM +0000, Glen Barber wrote: > > > > We are still waiting for a few (non-critical) things to complete be= fore > > > > the announcement of 14.0-RELEASE will be ready. > > > >=20 > > > > It should only be another day or so before these things complete. > > > >=20 > > > > Thank you for your understanding. > > > > > > >=20 > > > I always just installed my copy. > > >=20 > >=20 > > Ok. I do not know what exactly is your point, but releases are never > > official until there is a PGP-signed email sent. The email is intended > > for the general public of consumers of official releases, not "yeah, > > but"s. > > >=20 > Howver if you do a freebsd-update upgrade, you can upgrade. >=20 > Is that suppose to happen? >=20 That does not say that the freebsd-update bits will not change *until* the official release announcement has been sent. In my past 15 years involved in the Project, I think we have been very clear on that. A RELEASE IS NOT FINAL UNTIL THE PGP-SIGNED ANNOUNCEMENT IS SENT. I mean, c'mon, dude. We really, seriously, for all intents and purposes, cannot be any more clear than that. So, yes, *IF* an update necessitates a new freebsd-update build, what you are running is *NOT* official. For at least 15 years, we have all said the same entire thing. Glen --OmEpep6L+R/25PJj Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEjRJAPC5sqwhs9k2jAxRYpUeP4pMFAmVUTooACgkQAxRYpUeP 4pOmIBAAmMewpb30LVILPpfUpQuspylK2S0wQ6wdwiolWoi8XW07PAa8tn3g47S0 TVvjbn0I144HaIwKIW6td7mwfZMJJkUFvaXdAsBhExzBcNx35rxBu0sF7KjWaNSX cFcWr+Yrd8WtpE6PZDbVXQ/jM/cgzqI5PioZFQJZse3yW/8aZp5weamnJRPIJnIx 0nq8J32+zcKE3B/Z9e65kGtdsTkL4KjHEp4+FPmKADHMr52Indilk9xE23hpWhVJ Mv64ApoOJi1uE1HqezI8MvYoKnr1Mk4j8nUljGO95TBXxo6XUtWmndl1YHCS+PGP 8U/DaSNWZcXDNqBojO53lIBt+5AozLpQgsFnSoMjjPk95wO4j7JDYOyg7uTABXJy iW4KtMcfUp27oRZ6LiY6RI+NZbzikPw4jeonCGcnbi//vR6qxpU+EXCiKByaUDUc fccNEy+4J9+IxQOJwq4Jzri8HTxuiJsfqGHRrioP+JIcBz/+Lyau6vUc7BP5VkL5 CzwaWBJWkfihiFFd3FMfKhHC+P4tWNsY/mNkphzFOtjS5z7SB/gtqULsL/9kBtMN wAoJxwNqp75Vh44TvyIz3lLtdGl8ryR3V4kSlv4HxMD6kTmDX8wA7CycbaiSTuTO pF+XANyWnenCSVkckbv2KhAxsBCg0tUCKlTxmZZ3V49bvnipc9M= =aOaR -----END PGP SIGNATURE----- --OmEpep6L+R/25PJj-- From nobody Wed Nov 15 07:09:39 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SVZ6m219fz515HR for ; Wed, 15 Nov 2023 07:09:52 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Received: from mail-io1-xd2d.google.com (mail-io1-xd2d.google.com [IPv6:2607:f8b0:4864:20::d2d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SVZ6l4Gq1z3MDH for ; Wed, 15 Nov 2023 07:09:51 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b="gx8f/GIU"; spf=pass (mx1.freebsd.org: domain of grahamperrin@gmail.com designates 2607:f8b0:4864:20::d2d as permitted sender) smtp.mailfrom=grahamperrin@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-io1-xd2d.google.com with SMTP id ca18e2360f4ac-7a6907e9aa8so254692239f.1 for ; Tue, 14 Nov 2023 23:09:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700032190; x=1700636990; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=W0qpbHUzhWDzgOTE+a9GSkWprzaA8hbIBhsv1toUJv0=; b=gx8f/GIUx0eyRG+HYNXzVVCpuChZVq10FtPeINQo/3QCaXTKG9iiqQrVnNdzkT1L9F e1YMWm2OdlFL9X1b+apzNc0ovruTjpdtUsZNh8NqX7NXy4IImeeqzZ2M3k2SrKCK1Jnz ONk6qE0FBNd1LLIwjaBS/M0pWSIa2RHIe4mwaD/Q6H7o13MUweXotnE1KJY00ygr8nSP wMLqEgHcwraMaQUBZqpGWa1UkNsRCDZ7KZnE+LDk/ouQsANGWDDs0i+wdRcexBS56AQL evlWnS4fnqE46mx6fREZkMGEyqTazqkOVoaSepq90kk0iKBWMcnS61oW4qpA+0IST7c3 bK0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700032190; x=1700636990; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=W0qpbHUzhWDzgOTE+a9GSkWprzaA8hbIBhsv1toUJv0=; b=ShcvWjEpvXEie6DNO/zXiEPn2YlFFQWm9dra7vWXDJn4Nq87kaNXB4IOQbj4Vhqu8b m7SXCswuP0QCURTemw/55fJY+Pj11VSHZbO4Est98W5FVPPfiGVY6xYOe1Ktbrg+ER/l 5I41yzmo4CK4/AMIUmkSH1rRnTH5DZBUY2kLnWTW9EaFZs14/BgXOhMSXvehmD0HfEpg 2zKwFdu0iQquxuaOKKg8ZEt+5vov/PMlOZDw3tP0x+e6DT49LZLlWqCHPtIrOZGS9KEP /fyZiIFQmmQ89e6y1vBCYVBO6N/yxlEuBo+lf161kOVs29pT+p4iywncCK2aoX7MtIhk hCYA== X-Gm-Message-State: AOJu0YycVovU8nDV1ZLoh4cTKoayTqDceGqRmQpEKWybnhVN6PM31pLq 6fWq+sROajYfTDv+xnr7Ns2jmDUIqAPpGbx2hUs= X-Google-Smtp-Source: AGHT+IH3MwqRVRDQSmdCKqRhjYgUjHuLGWKp9WbQjBssmOy7NaBAPClsUIhv9X3bB0/IsqjJSMVexVvslpf/v8el2aU= X-Received: by 2002:a05:6602:1550:b0:790:f866:d71b with SMTP id h16-20020a056602155000b00790f866d71bmr17780657iow.13.1700032190552; Tue, 14 Nov 2023 23:09:50 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Graham Perrin Date: Wed, 15 Nov 2023 07:09:39 +0000 Message-ID: Subject: Re: uname -KU 1500001 1500000 To: Mark Millard Cc: Current FreeBSD Content-Type: text/plain; charset="UTF-8" X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[grahamperrin]; FREEMAIL_FROM(0.00)[gmail.com]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FREEMAIL_ENVFROM(0.00)[gmail.com]; DKIM_TRACE(0.00)[gmail.com:+]; TO_DN_ALL(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::d2d:from]; RCPT_COUNT_TWO(0.00)[2]; FREEMAIL_TO(0.00)[yahoo.com]; RCVD_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org] X-Rspamd-Queue-Id: 4SVZ6l4Gq1z3MDH X-Spamd-Bar: --- On Mon, 2 Oct 2023 at 08:33, Mark Millard wrote: > Or file and ls results for the amd64.amd64/usr.bin/uname/uname.o ? > (Just checking for if there are surprises.) Please, where would I find that file? I'm lost From nobody Wed Nov 15 07:14:43 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SVZDf2y4bz516BX for ; Wed, 15 Nov 2023 07:14:58 +0000 (UTC) (envelope-from otis@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SVZDf1KWCz3QBj; Wed, 15 Nov 2023 07:14:58 +0000 (UTC) (envelope-from otis@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700032498; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=532HvUschD3kAYuuo2Td3V/YTmcQw2qSUOgeT6vGiT0=; b=sq0Wly6WG4pD2SKyoE/hFLYEkHy+k4cMvxp5GAtEZDyQwcSmPbcIaCo5mU2LeHKar6zVQo xtAzKrbczLCJSji0iX+rufCSmFrmpXb8acBF6+s125aX2AS3ZhsXbQZJlPd2Psf31nP8x1 +tknaqUXDcKg9gXUxR3ff+y3923W4YIZs8GR1Hq0fM1H4tQ/n5b/rGE0NtuUJwlTcM8vjH Q0hcbLtaLbzQp2J1aPRc6pDAuvQIlMg7v8ShWcqgMiWaZubJUpZzOFEQrnZ8k65TNolW2H 1VkbsyMYNS/4eD2DrctYiFgW4WIB7b5TrUukAIpYYt1Kkzl/bl0zwjkyvb5Ybg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700032498; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=532HvUschD3kAYuuo2Td3V/YTmcQw2qSUOgeT6vGiT0=; b=Op429T2Ynmv7hgim16QXrtQ6c97iodHbU9k9lS5oVtxfWV1PBtRMy2e8Vt6p9U+3GqQYOS R4wWkZA0WR29PGS7JVbk09C4xOHJiA/HZEPWhSd89PbYzbG85npXAfRnDsfZ9KjTkgSJsZ umwqaE9MlnLliWwpe7tspeeOlVyFKRNXX+Q6z05yYxMdChewc1+gGJlNuHclgrpQzZ8ea6 CCBV3P8hdVnJgx2GcmsZ445CSQQkw/rVNuBpx05FuN7pHb3xwJyDhrJkMDc8OCt6qYF4wU eaOcZNCX3WRcNHnKqMi3Vnzcv0FJ39poyLIeejjG15JxGjDGTKHozuntFSV3SQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1700032498; a=rsa-sha256; cv=none; b=d/5FfBntlVYdxgwn43Osphm1zCNiVbO6qS26m2qA4l68t4pO21VjwhCyeRHU0Vzel50q4C ME2+aaM8F2rooCwbCMcBU9DaHoH4JsBx31nM9oakHeGM7VXV88P/C3UaER3QxtSIgS7qGs lcb8AnB6UDKo/z9vFG31VZ+y4iD453GFgyytto2W0fAJwqtn1/IBVH1Jpdd7f8yjh87El+ g3FJQTn1OosNnMfaJK801YMuWRlnsyNhjWnPXCOpEkNP5npMVsOgSebEUR9xb9zKKMuFQQ 0hWgjO99LcAH45j0D3roCQdh7P2gdb47vI3PruMEAIqgaJlvpCM21h1JecUKlQ== Received: from ns2.wilbury.net (ns2.wilbury.net [92.60.51.55]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "svc.wilbury.net", Issuer "R3" (verified OK)) (Authenticated sender: otis) by smtp.freebsd.org (Postfix) with ESMTPSA id 4SVZDd6lYtz1MHF; Wed, 15 Nov 2023 07:14:57 +0000 (UTC) (envelope-from otis@FreeBSD.org) Received: from smtpclient.apple (gw-upc.owhome.net [188.167.168.254]) (Authenticated sender: juraj@lutter.sk) by svc.wilbury.net (Postfix) with ESMTPSA id 6D24961F85; Wed, 15 Nov 2023 08:14:54 +0100 (CET) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.100.2.1.4\)) Subject: Re: uname -KU 1500001 1500000 From: Juraj Lutter In-Reply-To: Date: Wed, 15 Nov 2023 08:14:43 +0100 Cc: Mark Millard , Current FreeBSD Content-Transfer-Encoding: quoted-printable Message-Id: <9EF1572B-7750-4B05-A873-A57BDCC9B722@FreeBSD.org> References: To: Graham Perrin X-Mailer: Apple Mail (2.3774.100.2.1.4) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-14) on ns2.wilbury.net > On 15 Nov 2023, at 08:09, Graham Perrin = wrote: >=20 > On Mon, 2 Oct 2023 at 08:33, Mark Millard wrote: >=20 >> Or file and ls results for the amd64.amd64/usr.bin/uname/uname.o ? >> (Just checking for if there are surprises.) >=20 > Please, where would I find that file? I'm lost Under ${MAKEOBJDIRPREFIX}/$(SRCDIR) (in case you do the build yourself). MAKEOBJDIRPREFIX is normally /usr/obj and SRCDIR is /usr/src (in case = you don=E2=80=99t use any other). =E2=80=94 Juraj Lutter otis@FreeBSD.org From nobody Wed Nov 15 07:17:02 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SVZHH31sWz516TL for ; Wed, 15 Nov 2023 07:17:15 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Received: from mail-io1-xd35.google.com (mail-io1-xd35.google.com [IPv6:2607:f8b0:4864:20::d35]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SVZHG5kHGz3R5j for ; Wed, 15 Nov 2023 07:17:14 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=EXRdStn+; spf=pass (mx1.freebsd.org: domain of grahamperrin@gmail.com designates 2607:f8b0:4864:20::d35 as permitted sender) smtp.mailfrom=grahamperrin@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-io1-xd35.google.com with SMTP id ca18e2360f4ac-7affff20d38so201601339f.0 for ; Tue, 14 Nov 2023 23:17:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700032633; x=1700637433; darn=freebsd.org; h=content-transfer-encoding:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=EtBMWPY3nDlELsldHHn4t92zUGkPwK88HFsImxZ8qws=; b=EXRdStn+YXLGW02WCqTIcro4JOCK/ychZrx4GdIopO83Z8bClXwyhtVkqa3g8OULDe EEuPXyMw8vcG6qneUA2ng1zH02lB4+2sYm5md7Cqu0++zwmdt0FDreE0c0SGLEmWb3yz csEkrJOoiW+vGBW4HRD2tsf4Fomr14VopY3+KswRb3EaYFeoUAXLazayyzJd+mCM4Q2N tcRdiTMR6w4r3DyyhCHPO8gozU+OzGWjnnwXBVMsjahPOWIV/T9VpCqBAmKGQqZzKFHI jVRcBxez7sp/3th7UBQGLT88354Rflfmj9l7M+gWr/041NH51OrqJexNhnA7TjaeFnrN HrRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700032633; x=1700637433; h=content-transfer-encoding:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EtBMWPY3nDlELsldHHn4t92zUGkPwK88HFsImxZ8qws=; b=eaRajBfpJAb/aKm/oguvGTKZSEicHCZ1NGlpY0a6k8VGlXzsJ7ozxq774j/krQDPc7 KxXFV6vdTAcybsChHNojYtmn7vFyD8PfeVbl2Rlu1HVhA8E2U+fKhAEsbCFV6mnJaFa2 qYDhgbq3YhIQQpHCqwqqGKPII/Bxuat4+1lV/fCC4FRihXXg6bhu9F3PJEFxU6FsVRJU XRW+8jLQ6ATfuFl+MZAl7NseIUhnBc9CQdVJCQu/mLgnfn4RGd0sSKf/wIJx5V1Ex0vO a0hLEFNqXc3VVAFKLZBkrWvCYyhWvHpqYyN2P4uXHYSPGl3HnNlnevEqPa0PjpDLj85M 9RoQ== X-Gm-Message-State: AOJu0YxKqKkJ4gC+S3GNW0PcwqAMiKdfZjqWE0LexJRAKm5XpSYINxmx HZsJv/qWK2vzjQ+zliSjRS1G1+oo1giJAylBDzsbG8+SCHYYAw== X-Google-Smtp-Source: AGHT+IH895HYZp/E5fTe0Bs+E8qlF6yCXJnR8g9ODbzzCH6cMa3sDgahs6El6yPANkcz1NYa7i7vSMQEM87yIrRhwfY= X-Received: by 2002:a5d:9287:0:b0:7a9:77ac:5446 with SMTP id s7-20020a5d9287000000b007a977ac5446mr12902686iom.21.1700032633176; Tue, 14 Nov 2023 23:17:13 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: <96f478a2-a455-4f5b-8a5f-cf5441b3e7df@gmail.com> In-Reply-To: <96f478a2-a455-4f5b-8a5f-cf5441b3e7df@gmail.com> From: Graham Perrin Date: Wed, 15 Nov 2023 07:17:02 +0000 Message-ID: Subject: FreeBSD user environment repeatedly inferior to kernel (was: uname -KU 1500001 1500000) To: FreeBSD current Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-3.99 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.995]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; MIME_GOOD(-0.10)[text/plain]; FREEFALL_USER(0.00)[grahamperrin]; FROM_HAS_DN(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; ARC_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::d35:from]; DKIM_TRACE(0.00)[gmail.com:+]; TO_DN_ALL(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; RCVD_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org] X-Rspamd-Queue-Id: 4SVZHG5kHGz3R5j X-Spamd-Bar: --- On Sun, 1 Oct 2023 at 20:28, Graham Perrin wrote: > User environment (1500000) inferior to kernel (1500001). > > =E2=80=A6 I discovered a comparable mismatch from build and installation on 9th November. Any ideas? I use devel/ccache4 4.8, if that's relevant. Copied from : % uname -KU 1500003 1500001 % uname -a FreeBSD mowa219-gjp4-8570p-freebsd 15.0-CURRENT FreeBSD 15.0-CURRENT #3 main-n266317-f5b3e686292b-dirty: Thu Nov 9 12:49:29 GMT 2023 grahamperrin@mowa219-gjp4-8570p-freebsd:/usr/obj/usr/src/amd64.amd64/sys/GE= NERIC-NODEBUG amd64 % grep -C 2 completed\ on /var/log/buildworld.log -------------------------------------------------------------- >>> World build completed on Thu Nov 9 10:31:19 GMT 2023 >>> World built in 3208 seconds, ncpu: 8, make -j4 -------------------------------------------------------------- % grep -C 2 completed\ on /var/log/buildkernel.log 4356.16 real 2766.18 user 363.33 sys -------------------------------------------------------------- >>> Kernel build for GENERIC completed on Thu Nov 9 11:43:59 GMT 2023 -------------------------------------------------------------- -------------------------------------------------------------- -- 4002.48 real 2817.89 user 367.08 sys -------------------------------------------------------------- >>> Kernel build for GENERIC-NODEBUG completed on Thu Nov 9 12:50:48 GMT 2= 023 -------------------------------------------------------------- >>> Kernel(s) GENERIC GENERIC-NODEBUG built in 8369 seconds, ncpu: 8, make= -j4 % grep -C 1 completed\ on /var/log/installkernel.log -------------------------------------------------------------- >>> Installing kernel GENERIC completed on Thu Nov 9 20:22:06 GMT 2023 -------------------------------------------------------------- -- -------------------------------------------------------------- >>> Installing kernel GENERIC-NODEBUG completed on Thu Nov 9 20:23:48 GMT = 2023 -------------------------------------------------------------- % grep -C 2 everything\ completed\ on /var/log/installworld.log 28.98 real 16.25 user 12.06 sys -------------------------------------------------------------- >>> Installing everything completed on Fri Nov 10 00:36:46 GMT 2023 -------------------------------------------------------------- 287.91 real 111.51 user 171.26 sys % From nobody Wed Nov 15 07:22:23 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SVZPK03wPz516qg for ; Wed, 15 Nov 2023 07:22:29 +0000 (UTC) (envelope-from garyj@gmx.de) Received: from mout.gmx.net (mout.gmx.net [212.227.15.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mout.gmx.net", Issuer "Telekom Security ServerID OV Class 2 CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SVZPJ3g3Vz3SkW for ; Wed, 15 Nov 2023 07:22:28 +0000 (UTC) (envelope-from garyj@gmx.de) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.de; s=s31663417; t=1700032943; x=1700637743; i=garyj@gmx.de; bh=RKCYpDmv29zXD5D9pI9XuFV0pI+mKm1jCQJxeKsFy9s=; h=X-UI-Sender-Class:Date:From:To:Cc:Subject:In-Reply-To:References: Reply-To; b=GhdNdNXN1YNDLXDeefEwLZWb1iGboB/pkkqLLifjsZfVLSpFYM+0P06EpyyOFgIc 6wMS41FGp/OFgp5uyYp+h1rR/LQpcnrf4guncBI/SFAEeFF8fngkJXheQCbxMDCWY i7OakqoIvb34n48vxSF19/0RWGYV8Cw0hmleSVFKHXOBdt3RVgS9/F6qHAi9MybRN K+ZJgGgwqHWMl4IxKqFIS4zTqQD7qwuonomA7LN64Ix7s0d1NXz4TXgb/rVO58DY4 yEOZTr/SoZJRWEnVRMuj9EaJm9BmtnE0NogIEbRNoivmMeY6nRUfPUcfWVvqkYlgy BEpHqBQ5evWocBTf1A== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from ernst.home ([217.226.57.134]) by mail.gmx.net (mrgmx005 [212.227.17.190]) with ESMTPSA (Nemesis) id 1N1fn0-1rVnx13NNI-011yTi; Wed, 15 Nov 2023 08:22:23 +0100 Date: Wed, 15 Nov 2023 07:22:23 +0000 From: Gary Jennejohn To: Graham Perrin Cc: Mark Millard , Current FreeBSD Subject: Re: uname -KU 1500001 1500000 Message-ID: <20231115082223.6e6fa7ca@ernst.home> In-Reply-To: References: Reply-To: garyj@gmx.de X-Mailer: Claws Mail 3.19.1 (GTK+ 2.24.33; amd64-portbld-freebsd14.0) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable X-Provags-ID: V03:K1:t5X1Yikb6dg96mNdSWsGzWIGlhsUCTK+txoGbbse+4nHnJiN5KU kNSbm0MWzxumYyUI4P/+lk0yYaKYe+Cr+ciAp5/PwN+O1IyMNnTEBi3m8455J+uMx52F94m sdX3wxURR/78AaupML9iJiSKmBpteM0uNLiqJ7dJAhmA6IFaqMOD2t8zu8rVkHTS/lX+Ij5 CTLvhQMt4dybbEAZgnCMQ== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:mP34ZzsNpz0=;Kbb+TPJtH2wV4vAQdAAkIv0QN4F AuMRlluT0Kwq3UbWcLB1tbQDNe+WNZ7DhPTgADAFFOhoCugYx/h/Unwjhqnqrlvr7Lbsdk1lS r7mcR6lSpmeCfCxBuoXsh0fk8KhpzxS//2V8fQtS5rLr0QbYuE6/UkK8j3iRAvGFgLDthYbbH uvSgPV/WrI1ILzBc9KUZgkzk2+FIqE7n+kQb0GvY/5EFDC2xPbXgQ67Az4gLunbR+fXOamZME JHHHpjdn5807wC6KkltWfl6drO2uYwKYsQsrZQxcGSxC22BlMYIcdzEZk+Yd2GWc2R4KKpfd6 v0MiYLkk64FsYODnQznUG5EYUzlYZHVjV8cJoP8r0v2VV2oGAyjRvqWZJqT8C3HikhbidrnZx rHNdMJy0wW/RrW6CpWZTvUTvhF0snScsS6eUPIcA6+C6tlbobkYvhFAtZSaWpkwn5AUtooPU+ 0wi2Eoi7T78ArsCETrPTm4jrNjCbpbOopwh/JaDxhkc5LC7Sra3hFLxhi3sT91GxziMRrPafb BJQfx5eD0Cud5rk3cj4i1RtJIAdGq8L2X6By+obrbRWbSeC6vgyXLhukmv+YkQd3PPk6GXfSK z0pnttWHhDXfbp7HNqZxU6em4BH4xrJQuvLNGCYWsMyr0w14w4Q8FY5AYi+ung4jCK/7hj/CF vFYtcrHALbnbKdZBryzGHW7qFob4KuCDke8UllBsKK+0WD4RZiOnqvzy1isq6yP8Btg8IrmWd 6y87tXkox9D/IHANTPAXI8isTM0MfLXFRPgUcLK25siWYGbGyYiULV5lH2fjLCWnT0cuwDg3Q rgXHBiQLu7kQraZ6DRa9RcoBmeyhOfFvhWWN1MsdT9sAOCqwPHmsTz6vZb+0sNMhfCSCsVIJ5 yEoRlzdtG7cCNktpdVaNnSdsUPvcqgnbw+zNEA/xiVf1CNcS422vUrxhD6fy9MOKYiFtxZmEJ Z6Sg4Ymxqxu3mvoRduQVTtn5slQ= X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:8560, ipnet:212.227.0.0/16, country:DE] X-Rspamd-Queue-Id: 4SVZPJ3g3Vz3SkW On Wed, 15 Nov 2023 07:09:39 +0000 Graham Perrin wrote: > On Mon, 2 Oct 2023 at 08:33, Mark Millard wrote: > > > Or file and ls results for the amd64.amd64/usr.bin/uname/uname.o ? > > (Just checking for if there are surprises.) > > Please, where would I find that file? I'm lost > Assuming that you have a /usr/obj and are compiling a AMD64 buildworld it should be here: /usr/obj/usr/src/amd64.amd64/usr.bin/uname/uname.o Otherwise replace amd64 with whatever architecture you're using. =2D- Gary Jennejohn From nobody Wed Nov 15 07:36:38 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SVZjx2tLzz517TD for ; Wed, 15 Nov 2023 07:36:53 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Received: from mail-il1-x136.google.com (mail-il1-x136.google.com [IPv6:2607:f8b0:4864:20::136]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SVZjw3lcTz3W8q for ; Wed, 15 Nov 2023 07:36:52 +0000 (UTC) (envelope-from grahamperrin@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=VDcraT0C; spf=pass (mx1.freebsd.org: domain of grahamperrin@gmail.com designates 2607:f8b0:4864:20::136 as permitted sender) smtp.mailfrom=grahamperrin@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-il1-x136.google.com with SMTP id e9e14a558f8ab-359472f74c5so26327395ab.0 for ; Tue, 14 Nov 2023 23:36:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700033810; x=1700638610; darn=freebsd.org; h=content-transfer-encoding:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=xQX8cRjE9k4YrOKfjbzFHJJrIAp3bdNxOBueEkGUfsQ=; b=VDcraT0CFluma42LDvvF/yE7AtV1hS0gBf9MLmV6iaiYERJYoY8De6xPfCtCv+CY3t Aq6yLavNN+Zo8csOx46byEtlhIj93JizWseCg4QnJ7T2edgoEuGO0qGtFFuU5hiQbBIg fWnNipsllp1bVRMIh3UdZh+ropQBtODqa2V8RKSZmGjyJ4ISAVIhuCEFZ0ukKtm9NfYi 1tCe+ugYOxOZFRH5gFGUwq6nrtCEM8Y+xJ1DvjeBhWcbS+ItXAjtPVOYWzmewx50jvpY sV/dOu7MNrATkTtkNkJCvcm/yy7s/vvfbbmFqdT0E43ESzx7RnDkIisGpNaSNy2aM2/Y Y44g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700033810; x=1700638610; h=content-transfer-encoding:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xQX8cRjE9k4YrOKfjbzFHJJrIAp3bdNxOBueEkGUfsQ=; b=nGauAklw4hQznGkEyQkNosMwP2589fZPWVu3Ff5W5EtVPHdCUjpuputbAb0rxD2WGA McTNjstRiNNQ/mOqTg0dXDa0AyB/lJO3nmz0LMQd2LA8mOFNa/Xvr03mYkvlyr0+WHeF QpLnFISQo54tR/SZ4HGpZWXDtEUDVI+6i9CvmU7aRQIvCKuS/OdYSUB30KRgsJqE4X5X vPql3xvKTfGCstJi6qV9ojLB5S4LSW3tvDQajbem57DDufbpU+Ih/4Oy5Np/EWBblTnU As1K7rSj2+pOT1RzVgttCbYMMR/i4dUvt+36qDoy/q562wmEVWCelOIN+ZJRfQmCSRfq CNPQ== X-Gm-Message-State: AOJu0YxPn5xjlWo9E7DQDxxYaXCJWVQkAHG4PBYlib9S7h5/tSML1mkt p0unZ8obaUzrSuQfYRRIhI5xvJo8fAy++APm3Kdemslu58Eqag== X-Google-Smtp-Source: AGHT+IEfRpiwX+Iipu8DDJujoTGCx9H4Y48GlavBnWK/4K1JJnl3rlURpr3Zj5pd7C4ExtBaSf7D9ZanNnAsBpvFaAE= X-Received: by 2002:a05:6e02:214b:b0:359:4048:38df with SMTP id d11-20020a056e02214b00b00359404838dfmr13094712ilv.7.1700033810378; Tue, 14 Nov 2023 23:36:50 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: <96f478a2-a455-4f5b-8a5f-cf5441b3e7df@gmail.com> In-Reply-To: From: Graham Perrin Date: Wed, 15 Nov 2023 07:36:38 +0000 Message-ID: Subject: Re: FreeBSD user environment repeatedly inferior to kernel (was: uname -KU 1500001 1500000) To: FreeBSD-CURRENT Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; MIME_GOOD(-0.10)[text/plain]; FREEFALL_USER(0.00)[grahamperrin]; FROM_HAS_DN(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; ARC_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::136:from]; DKIM_TRACE(0.00)[gmail.com:+]; TO_DN_ALL(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; RCVD_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org] X-Rspamd-Queue-Id: 4SVZjw3lcTz3W8q X-Spamd-Bar: --- On Wed, 15 Nov 2023 at 07:17, Graham Perrin wrote: > > On Sun, 1 Oct 2023 at 20:28, Graham Perrin wrote= : > > > User environment (1500000) inferior to kernel (1500001). > > > > =E2=80=A6 > > I discovered a comparable mismatch from build and installation on 9th > November. Any ideas? uname.o % ls -hl /usr/obj/usr/src/amd64.amd64/usr.bin/uname/uname.o -rw-r--r-- 1 grahamperrin wheel 36K Oct 9 12:51 /usr/obj/usr/src/amd64.amd64/usr.bin/uname/uname.o % file /usr/obj/usr/src/amd64.amd64/usr.bin/uname/uname.o /usr/obj/usr/src/amd64.amd64/usr.bin/uname/uname.o: ELF 64-bit LSB relocatable, x86-64, version 1 (FreeBSD), with debug_info, not stripped % > I use devel/ccache4 4.8, if that's relevant. > > Copied from : > > % uname -KU > 1500003 1500001 > % uname -a > FreeBSD mowa219-gjp4-8570p-freebsd 15.0-CURRENT FreeBSD 15.0-CURRENT > #3 main-n266317-f5b3e686292b-dirty: Thu Nov 9 12:49:29 GMT 2023 > grahamperrin@mowa219-gjp4-8570p-freebsd:/usr/obj/usr/src/amd64.amd64/sys/= GENERIC-NODEBUG > amd64 > % grep -C 2 completed\ on /var/log/buildworld.log > > -------------------------------------------------------------- > >>> World build completed on Thu Nov 9 10:31:19 GMT 2023 > >>> World built in 3208 seconds, ncpu: 8, make -j4 > -------------------------------------------------------------- > % grep -C 2 completed\ on /var/log/buildkernel.log > 4356.16 real 2766.18 user 363.33 sys > -------------------------------------------------------------- > >>> Kernel build for GENERIC completed on Thu Nov 9 11:43:59 GMT 2023 > -------------------------------------------------------------- > -------------------------------------------------------------- > -- > 4002.48 real 2817.89 user 367.08 sys > -------------------------------------------------------------- > >>> Kernel build for GENERIC-NODEBUG completed on Thu Nov 9 12:50:48 GMT= 2023 > -------------------------------------------------------------- > >>> Kernel(s) GENERIC GENERIC-NODEBUG built in 8369 seconds, ncpu: 8, ma= ke -j4 > % grep -C 1 completed\ on /var/log/installkernel.log > -------------------------------------------------------------- > >>> Installing kernel GENERIC completed on Thu Nov 9 20:22:06 GMT 2023 > -------------------------------------------------------------- > -- > -------------------------------------------------------------- > >>> Installing kernel GENERIC-NODEBUG completed on Thu Nov 9 20:23:48 GM= T 2023 > -------------------------------------------------------------- > % grep -C 2 everything\ completed\ on /var/log/installworld.log > 28.98 real 16.25 user 12.06 sys > -------------------------------------------------------------- > >>> Installing everything completed on Fri Nov 10 00:36:46 GMT 2023 > -------------------------------------------------------------- > 287.91 real 111.51 user 171.26 sys > % From nobody Wed Nov 15 07:38:57 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SVZmj5RmWz517S8 for ; Wed, 15 Nov 2023 07:39:17 +0000 (UTC) (envelope-from mattik@gwsit.com.au) Received: from m4.out4.mxs.au (m4.out4.mxs.au [110.232.143.188]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SVZmh36XWz3WsD for ; Wed, 15 Nov 2023 07:39:15 +0000 (UTC) (envelope-from mattik@gwsit.com.au) Authentication-Results: mx1.freebsd.org; none Received: from s02ad.syd2.hostingplatform.net.au (s02ad.syd2.hostingplatform.net.au [103.27.32.38]) by out4.mxs.au (Halon) with ESMTPS (TLSv1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 id 0b7ddf84-838a-11ee-9cb2-00163c87da3f; Wed, 15 Nov 2023 18:39:01 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gwsit.com.au; s=default; h=Content-Transfer-Encoding:Content-Type: MIME-Version:References:In-Reply-To:Message-ID:Subject:Cc:To:From:Date:Sender :Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help: List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=wnVD7Efi0UjeRf7Xh57uVKzmlcU7sbw097LAgbl+y7o=; b=SyuZzuHE6i47YJRn9NzrG6oRSX 7gPcP4dqCPfjf2QOTpf2rEl0ZLDuHfy4cjyiblOUwdNAw5opVpwMPpgGGY5HWpquSPaVSWw3fCZiS hoKOXxRFxT16ITc3+rdI5+u+soXp8JT+EmTybKnjnPWW50lAD80a6tL8K5p0rKtsfb8/OcOf+BUMB d5PVQvbWdIaC6i+s03IVgNd3qZIx5POQJHgUsq9NLIsyaWwf4k+bC7Gvyw5+7EjKl14nAb8Y2nX4F 9v06fVXt9WGa1c994XTfqHQImXFII/grANcH0tVbxWmF3De2B85coFm/em6aLriefu+BMJfP6wt3C 2gFEb1sA==; Received: from 180-150-31-87.b4961f.syd.static.aussiebb.net ([180.150.31.87]:52025 helo=ws1.wobblyboot.net) by s02ad.syd2.hostingplatform.net.au with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.96.2) (envelope-from ) id 1r3AUH-001OGv-1t; Wed, 15 Nov 2023 18:39:01 +1100 Date: Wed, 15 Nov 2023 18:38:57 +1100 From: matti k To: Glen Barber Cc: FreeBSD Release Engineering Team , freebsd-current@freebsd.org, freebsd-stable@freebsd.org Subject: Re: [HEADS-UP] Quick update to 14.0-RELEASE schedule Message-ID: <20231115183857.2c728fa5@ws1.wobblyboot.net> In-Reply-To: <20231115045231.GN1307@FreeBSD.org> References: <20231114203654.GB52320@FreeBSD.org> <20231115022701.GM1307@FreeBSD.org> <20231115045231.GN1307@FreeBSD.org> X-Mailer: Claws Mail 3.19.1 (GTK+ 2.24.33; amd64-portbld-freebsd13.2) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - s02ad.syd2.hostingplatform.net.au X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - gwsit.com.au X-Get-Message-Sender-Via: s02ad.syd2.hostingplatform.net.au: authenticated_id: mattik@gwsit.com.au X-Authenticated-Sender: s02ad.syd2.hostingplatform.net.au: mattik@gwsit.com.au X-Source: X-Source-Args: X-Source-Dir: X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:45638, ipnet:110.232.143.0/24, country:AU] X-Rspamd-Queue-Id: 4SVZmh36XWz3WsD On Wed, 15 Nov 2023 04:52:31 +0000 Glen Barber wrote: > > > Ok. I do not know what exactly is your point, but releases are > > > never official until there is a PGP-signed email sent. The email > > > is intended for the general public of consumers of official > > > releases, not "yeah, but"s. > > > > That does not say that the freebsd-update bits will not change *until* > the official release announcement has been sent. > > In my past 15 years involved in the Project, I think we have been very > clear on that. > > A RELEASE IS NOT FINAL UNTIL THE PGP-SIGNED ANNOUNCEMENT IS SENT. > > I mean, c'mon, dude. > > We really, seriously, for all intents and purposes, cannot be any more > clear than that. > > So, yes, *IF* an update necessitates a new freebsd-update build, what > you are running is *NOT* official. > > For at least 15 years, we have all said the same entire thing. Here I have been tracking 14.0 release candidates Reminds me of when slashdot would announce a new release days before it was actually released and then someone would remind them of that I am using an Australian mirror so will probably need to wait an extra day to be on the safe side :-) From nobody Wed Nov 15 08:27:08 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SVbr05vRVz519bm; Wed, 15 Nov 2023 08:27:12 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from mail0.glenbarber.us (mail0.glenbarber.us [IPv6:2607:fc50:1:2300:1001:1001:1001:face]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail0.glenbarber.us", Issuer "Gandi Standard SSL CA 2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SVbr05TFKz3bv4; Wed, 15 Nov 2023 08:27:12 +0000 (UTC) (envelope-from gjb@freebsd.org) Authentication-Results: mx1.freebsd.org; none Received: from smtpclient.apple (unknown [IPv6:2607:fb90:e94b:905a:c8b8:3487:dbbc:d826]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: gjb) by mail0.glenbarber.us (Postfix) with ESMTPSA id 84129D5DB; Wed, 15 Nov 2023 08:27:10 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.10.3 mail0.glenbarber.us 84129D5DB Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (1.0) Subject: Re: [HEADS-UP] Quick update to 14.0-RELEASE schedule From: Glen Barber In-Reply-To: <20231115183857.2c728fa5@ws1.wobblyboot.net> Date: Wed, 15 Nov 2023 03:27:08 -0500 Cc: FreeBSD Release Engineering Team , freebsd-current@freebsd.org, freebsd-stable@freebsd.org Message-Id: References: <20231115183857.2c728fa5@ws1.wobblyboot.net> To: matti k X-Mailer: iPhone Mail (20G81) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36236, ipnet:2607:fc50::/36, country:US] X-Rspamd-Queue-Id: 4SVbr05TFKz3bv4 Well, today is your yesterday, so we might not be ready for another more tom= orrows your time, yesterdays, my time. Or something. Glen Sent from my phone. Please excuse my brevity and/or typos. > On Nov 15, 2023, at 2:39 AM, matti k wrote: >=20 > =EF=BB=BFOn Wed, 15 Nov 2023 04:52:31 +0000 > Glen Barber wrote: >=20 >>>> Ok. I do not know what exactly is your point, but releases are >>>> never official until there is a PGP-signed email sent. The email >>>> is intended for the general public of consumers of official >>>> releases, not "yeah, but"s. >>>>=20 >=20 >> That does not say that the freebsd-update bits will not change *until* >> the official release announcement has been sent. >>=20 >> In my past 15 years involved in the Project, I think we have been very >> clear on that. >>=20 >> A RELEASE IS NOT FINAL UNTIL THE PGP-SIGNED ANNOUNCEMENT IS SENT. >>=20 >> I mean, c'mon, dude. >>=20 >> We really, seriously, for all intents and purposes, cannot be any more >> clear than that. >>=20 >> So, yes, *IF* an update necessitates a new freebsd-update build, what >> you are running is *NOT* official. >>=20 >> For at least 15 years, we have all said the same entire thing. >=20 > Here I have been tracking 14.0 release candidates >=20 > Reminds me of when slashdot would announce a new release days before it > was actually released and then someone would remind them of that >=20 > I am using an Australian mirror so will probably need to wait an extra > day to be on the safe side :-) >=20 >=20 >=20 >=20 From nobody Wed Nov 15 08:59:02 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SVcXt3bDMz51C5d for ; Wed, 15 Nov 2023 08:59:10 +0000 (UTC) (envelope-from garyj@gmx.de) Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mout.gmx.net", Issuer "Telekom Security ServerID OV Class 2 CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SVcXs4CX7z4F30 for ; Wed, 15 Nov 2023 08:59:09 +0000 (UTC) (envelope-from garyj@gmx.de) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmx.de header.s=s31663417 header.b=Jl+ZHiMD; spf=pass (mx1.freebsd.org: domain of garyj@gmx.de designates 212.227.17.20 as permitted sender) smtp.mailfrom=garyj@gmx.de; dmarc=pass (policy=none) header.from=gmx.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.de; s=s31663417; t=1700038747; x=1700643547; i=garyj@gmx.de; bh=FU8dTHdP8hke3sFSB8/D7ww/aVNt5UjY3FfYrGAwJi0=; h=X-UI-Sender-Class:Date:From:To:Subject:In-Reply-To:References: Reply-To; b=Jl+ZHiMDCfIeasoLIr4X+XCS7qZcVdErHJvMcfpV/Qdoz/ZEv4xT+OFzBN960TLU saT+7/LO7l5XiQkGeBGJT6LbXa5UXywEnYxPjZxcZVgoIA0Zq6CpH0Fsbvt227BNJ C25cyXAFTSIAv0DlZDQzqjxzuCcy9ptgpRdIjS1iTN8mrOESuF9AdSHjkX459Vm7+ clPS9Q+GpJVBhTxksPYFxIA59ghuOwgA4DcBXPtd+xiXSI/KDd3+A11lWTNXJF+ad QZXQy8edT9gWIwDLRU8ANICYNosA53wbYmSBRObZ5GnxJ2QW+qWEUWQgzZtF+IJRr +CaYW86b2vcfYi2J/g== X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a Received: from ernst.home ([217.226.57.134]) by mail.gmx.net (mrgmx105 [212.227.17.168]) with ESMTPSA (Nemesis) id 1MFsUp-1r9Yzk2W7T-00HMnz for ; Wed, 15 Nov 2023 09:59:07 +0100 Date: Wed, 15 Nov 2023 08:59:02 +0000 From: Gary Jennejohn To: freebsd-current@freebsd.org Subject: Re: FreeBSD user environment repeatedly inferior to kernel (was: uname -KU 1500001 1500000) Message-ID: <20231115095902.4c43f02e@ernst.home> In-Reply-To: References: <96f478a2-a455-4f5b-8a5f-cf5441b3e7df@gmail.com> Reply-To: garyj@gmx.de X-Mailer: Claws Mail 3.19.1 (GTK+ 2.24.33; amd64-portbld-freebsd14.0) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Provags-ID: V03:K1:qODN2qXzBjF9GcZJMdLZTnTDNifxr0y6j2jRhHNPUIt78pD2Hqt RCblS0nLh0I86GyZZfrBCPNkzvI1fQj5TMBK9nD9/5Ckwhuojppg33tSkEpzIX63ZWEZscQ 48bcTyBHtDSTC1E15IvxaKli/b4JVnVGtQYABGxAdd7lmgewAZPAFldr9zxmk9O9ncahRjx qbvdP+xBjbMe0B1/CQi/w== X-Spam-Flag: NO UI-OutboundReport: notjunk:1;M01:P0:MvXnas4f0h4=;sIAQSLa1/A3C8TqnhSe/4MrGs8R kf/toC5sgSl6L6aKUp2xZiVLvlbwVhQ7HgQxDGgFKbe0HKYRU6oq52eLJKHX8HsqJpukqSvd3 apHgJBCkh/alhJMjl+W1STox+q7fUZHynlOi4lxVzDDDHxP3wmsVTsgN4cytJthn2owL2+6JR IPPkqp5rXzcUeLJn8nF+x/jeUtOgKBcSXrpt6Z7wgJHxY4CBoP75rd4Cp94f6fQb9HshJaJBa 37wmBhK4V3E2N5mtPndKDUoKJdLDBYcDVWZxqhmqE1JV5xQxP3LdDhSdzQA6oAEovxOyZ3UxZ XIhpVgdc+lcP1a1AJlkHahaZJRUed/lddM/VXWnyIX1kLZQbj21kWVdTnzodxXzTeb0XOc/qG x/oxtGy5egV0qxs++k+RtHM2ajMQG5x8tlbkqIePpOzUrMAjhd4o2T5PZNKD9/JeRMT0UGaQG VcrgN0+N6z3onFXcLQytuAzc3SLeA73H9NIh5GllIaiRewKoL9+i4kr6c1r4qXfBH4Eo9FIv1 qoso4L2R8pm3OqqZxYqjwgLvm2Cf0fZ1Q1R5WyWZBmA3OSRRdHfXKCms+uG4DskzOW7nh0YhN 9vQSwKdSYluCQ9iWWlqy07czokRTCrN/j5RZp5BAB6v5wQtYJjFAd+YYqC/RwTGCKkIlokDQU r2iXsdzciQEM6WhcZKQigTLBLBaOuXw3PSVh/u0MAN4dr52/Xqe8GTbnXf9Uvc52TrwemwAx5 lBJ162zYjfeHyfCZGaA5j3c+VPDINhjlbK5x7LZRzOLoUP0DP4E/0XQX88EKHChql8lyO78Wk TV8x+N1npVpjGxKJSDUhkZ1NZrEBK2x6+WtwucKkpHdLdAqPryaAedSR8U6OUI6kHronNSSCm jBHq4REjZOKOPa1RUf5ICXMPD1g11lNknjqnoZesth1ZTRWFd4itUhKAGRHpJ6kGK5ce2PHbx gibdNxAacNpCOYmw8h63T74Ct/M= X-Spamd-Result: default: False [-3.39 / 15.00]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[gmx.de,none]; R_DKIM_ALLOW(-0.20)[gmx.de:s=s31663417]; RWL_MAILSPIKE_VERYGOOD(-0.20)[212.227.17.20:from]; R_SPF_ALLOW(-0.20)[+ip4:212.227.17.0/27]; NEURAL_HAM_LONG(-0.19)[-0.191]; ONCE_RECEIVED(0.10)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[212.227.17.20:from]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; FREEMAIL_REPLYTO(0.00)[gmx.de]; HAS_REPLYTO(0.00)[garyj@gmx.de]; ASN(0.00)[asn:8560, ipnet:212.227.0.0/16, country:DE]; FREEMAIL_ENVFROM(0.00)[gmx.de]; REPLYTO_ADDR_EQ_FROM(0.00)[]; FREEMAIL_FROM(0.00)[gmx.de]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; DKIM_TRACE(0.00)[gmx.de:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Queue-Id: 4SVcXs4CX7z4F30 X-Spamd-Bar: --- On Wed, 15 Nov 2023 07:36:38 +0000 Graham Perrin wrote: > On Wed, 15 Nov 2023 at 07:17, Graham Perrin wro= te: > > > > On Sun, 1 Oct 2023 at 20:28, Graham Perrin wr= ote: > > > > > User environment (1500000) inferior to kernel (1500001). > > > > > > ? > > > > I discovered a comparable mismatch from build and installation on 9th > > November. Any ideas? > > uname.o > > % ls -hl /usr/obj/usr/src/amd64.amd64/usr.bin/uname/uname.o > -rw-r--r-- 1 grahamperrin wheel 36K Oct 9 12:51 > /usr/obj/usr/src/amd64.amd64/usr.bin/uname/uname.o > % file /usr/obj/usr/src/amd64.amd64/usr.bin/uname/uname.o > /usr/obj/usr/src/amd64.amd64/usr.bin/uname/uname.o: ELF 64-bit LSB > relocatable, x86-64, version 1 (FreeBSD), with debug_info, not > stripped > % > > > I use devel/ccache4 4.8, if that's relevant. > > Might be. You should probably run make clean in /usr/src and then run buildworld again to make sure that you're getting .o's from the current source. > > Copied from : > > > > % uname -KU > > 1500003 1500001 > > % uname -a > > FreeBSD mowa219-gjp4-8570p-freebsd 15.0-CURRENT FreeBSD 15.0-CURRENT > > #3 main-n266317-f5b3e686292b-dirty: Thu Nov 9 12:49:29 GMT 2023 > > grahamperrin@mowa219-gjp4-8570p-freebsd:/usr/obj/usr/src/amd64.amd64/s= ys/GENERIC-NODEBUG > > amd64 > > % grep -C 2 completed\ on /var/log/buildworld.log > > > > -------------------------------------------------------------- > > >>> World build completed on Thu Nov 9 10:31:19 GMT 2023 > > >>> World built in 3208 seconds, ncpu: 8, make -j4 > > -------------------------------------------------------------- > > % grep -C 2 completed\ on /var/log/buildkernel.log > > 4356.16 real 2766.18 user 363.33 sys > > -------------------------------------------------------------- > > >>> Kernel build for GENERIC completed on Thu Nov 9 11:43:59 GMT 2023 > > -------------------------------------------------------------- > > -------------------------------------------------------------- > > -- > > 4002.48 real 2817.89 user 367.08 sys > > -------------------------------------------------------------- > > >>> Kernel build for GENERIC-NODEBUG completed on Thu Nov 9 12:50:48 = GMT 2023 > > -------------------------------------------------------------- > > >>> Kernel(s) GENERIC GENERIC-NODEBUG built in 8369 seconds, ncpu: 8,= make -j4 > > % grep -C 1 completed\ on /var/log/installkernel.log > > -------------------------------------------------------------- > > >>> Installing kernel GENERIC completed on Thu Nov 9 20:22:06 GMT 202= 3 > > -------------------------------------------------------------- > > -- > > -------------------------------------------------------------- > > >>> Installing kernel GENERIC-NODEBUG completed on Thu Nov 9 20:23:48= GMT 2023 > > -------------------------------------------------------------- > > % grep -C 2 everything\ completed\ on /var/log/installworld.log > > 28.98 real 16.25 user 12.06 sys > > -------------------------------------------------------------- > > >>> Installing everything completed on Fri Nov 10 00:36:46 GMT 2023 > > -------------------------------------------------------------- > > 287.91 real 111.51 user 171.26 sys > > % > =2D- Gary Jennejohn From nobody Wed Nov 15 15:57:35 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SVnqj2r6Pz50Rg8; Wed, 15 Nov 2023 15:57:37 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SVnqj2JS9z3TNt; Wed, 15 Nov 2023 15:57:37 +0000 (UTC) (envelope-from jhb@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700063857; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9bVWED4ZEd0X14ls1AIhVMMhDtdXCH4+40nJD6M151A=; b=ooh7WGv2hn5UOm5RJp06jPpgS9Yr2R6wti36ehd8vi3skzQd7EMTSQVUXbqfgLo4kj/FrX +8Ar0etjNgFSDrhKLsqVhNUlHjctkkl7zMqzV9VLNryvzfg6PEmZBVH6IBUV+IydJ/lIzg X14YHk27UEb75zePy9XXiL5cMCIP1a0Clpx/k7zmjURD1DUtG2H146S51ozV4E8/2ugrRO GOEnu4RC0fESE7Do17m+nKwKVk19daSj1f1BYpW3JquIA/cfAWQMRmDclZrNaw9zTWBPNj WOt4AoB3XvdUlAiPJXRCh/DFqxAwSuC9T93QKKN7DpQsIc0A/ua/SYFNn1oHtQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700063857; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9bVWED4ZEd0X14ls1AIhVMMhDtdXCH4+40nJD6M151A=; b=pxNhKW1v6RJmPA2upf+rJjctsRiu4G+MjlsnijltRgrZDQiTRjiQ5Xq/wwgAgpx6akmjiY 0JpAm9ULJ1WWVnEiBWzH80yCBqkBvL/N3jxI2EXZsFDoK9lXaAg/AbHumHOGbGelXH5rcg t19pO1x4h+ndRv5aK4ZpNoLI2TnCD8FyxHW7SHTZmMVNAKXstGXpq7k5EMa+e222pcQ9LX eh9KewREaUaU4MB0KbnH0EbcpBUQVz9U+wDOZmz+HBIsvUNmdyqIy+44kCN/kPoU7efvfx 9jbnFSELkrT2aYOxyVtF32r1lD15FhajzG4IPiNeoMZQJ04pbWy+IWRao2gEOQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1700063857; a=rsa-sha256; cv=none; b=CG+gWrLgMwGdowaWjgwdEchQgHZwW2zgFh9wNyKUa+nYNXUZegC5hEc6H+R1654APLA7TG D5pty0/cfaz6QOqLQk+Y7oPxgYerV3dhTnZHsRTxr9SujLSrHqOizzvEbYWsS0WR5UvBhZ OqQmRr5r10nilUOh5uXQLE0y/I8cH170m6dJTPGhWOBRyBCsnGBiEqjv9go0h+eD15nCP0 b+2chUNJaLxu41QE3c8yhQwqVavpMtURVfaELhi1+mBQGhoqxj97M28AHvy2TRwv2lEvGa GkoX+1o63d28e3FQFeiGo3B/DtXQlUMgduN8ETiClDXdcoi3D9VONDljaDcBLQ== Received: from [IPV6:2601:648:8384:fd00:7c48:55a7:81e0:3a0a] (unknown [IPv6:2601:648:8384:fd00:7c48:55a7:81e0:3a0a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: jhb) by smtp.freebsd.org (Postfix) with ESMTPSA id 4SVnqh5snlz37M; Wed, 15 Nov 2023 15:57:36 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Message-ID: Date: Wed, 15 Nov 2023 07:57:35 -0800 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: bhyve -G Content-Language: en-US To: Bakul Shah , FreeBSD CURRENT Cc: virtualization@freebsd.org References: From: John Baldwin In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 10/9/23 5:21 PM, Bakul Shah wrote: > Any hints on how to use bhyve's -G option to debug a VM > kernel? I can connect to it from gdb with "target remote :" > & bhyve stops the VM initially but beyond that I am not sure. > Ideally this should work just like an in-circuit-emulator, not > requiring anything special in the VM or kernel itself. step only works on Intel CPUs currently (and is a bit fragile anyway due to interrupts firing while you try to step, but that happens for me in QEMU as well). Breakpoints should work fine. I tend to use 'until' to do stepping (basically stepping via temporary breakpoints) when debugging the kernel this way. -- John Baldwin From nobody Wed Nov 15 16:17:31 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SVpGj6GvBz50SLf for ; Wed, 15 Nov 2023 16:17:33 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SVpGj5lx4z3WsJ; Wed, 15 Nov 2023 16:17:33 +0000 (UTC) (envelope-from jhb@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700065053; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6YZp8h00QTzL41QkxLbkol7za0044WY9JBjibH5Sddw=; b=e21FUCjwcGvCl1mN3xkdfa5lP89oAaueKbeUskYWf4a/2ZXS4PXUtLKJq0Bx6bRn1iS6YD 1wCuTa3oWjgqh82qusYF8r29nRCrOVgQ/iyqb5MqGehKu3JpdFwcl8YMsYSncxYdypSN7Z QAs66TiklCQAc+t9lAqOQOiZdki+U+KUGzfqMQZ9EZiJ5ufN1cVJ/kpprhyuCfasPO9ROh RLZF65RGXowyXKW5Zq1HX/6QlPmzSqm1WIFXrMKmkTwDgxZz0gql5KaAiIHTkqJFoAKQ5K 16ryCvRO1v1/HyHvEieUfWaOYCHOn50Bcx49NZuFFBXaIdqa4WiyUnNcIdLsMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700065053; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6YZp8h00QTzL41QkxLbkol7za0044WY9JBjibH5Sddw=; b=XFU7QJ4MrcmX1HcnBtAmc2aoq+mzLr/0Vp+F4lnKrWk6YoM1bWQ4cpVymbxTEEOKJll2HA 7Fh7fvTVCr0FKZ78ZK5MA0xOE9NSVR6qi/QuSls1hS3Pg0HP4Bxef+BEYV2MGmWmvS9Hig 4mTF4v74/l9XaxAF69kiZYSOS7FfzSQpmE/FHb44WRHy5t3Re4C7fxrkrXDnObUnLwvuzx NWuO6iiFl/0jiApvGBA8uYPEaczim47G2TJj57Ui6Vwy4bSMT3OUsBQikjn9Wehk7sl9Fj 0CNjRrsasRQZd+Yr4qxz6x5FVgabIu7+klZcudKhPZQ9roy8kW3BsxxgEt43Ig== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1700065053; a=rsa-sha256; cv=none; b=WHXi4rIUR/GVnILN8sWuHwCotVGHXQT7YVRChbJlZuZVE6wxPimMsowvq4Yj9G115auwSs FON8Hb0YgNQ/t3Hfi3hgSNWUlIUBMBhHnEhmAN6fq5WSHNyp/fvG9KFXZ7PxQq/+WtWj+B 1Te+HUgJ2suF6IUORwaemNB5TJlD8X8WvOW8Dp+RxNwf0uDSONjJpSmJBsiHg2D+9/P6Qn SVSIBspSILHzHig4pJkMOTzIU7f5MUpiJSgheTz7f87PT/h8O6hd36Ux9TGQ+QSqoFfS3N 9kTn73dHif6h2Yc0H9YJtGCqYGtw+lNWQelq/jw+WFRaBjjewYxcecKX9Rp7Ug== Received: from [IPV6:2601:648:8384:fd00:7c48:55a7:81e0:3a0a] (unknown [IPv6:2601:648:8384:fd00:7c48:55a7:81e0:3a0a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: jhb) by smtp.freebsd.org (Postfix) with ESMTPSA id 4SVpGj2x74z3R8; Wed, 15 Nov 2023 16:17:33 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Message-ID: <7baa3bd9-029c-440a-82a9-60a5098a9a1c@FreeBSD.org> Date: Wed, 15 Nov 2023 08:17:31 -0800 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: bsdinstall/scriptedpart could not run ;-( Content-Language: en-US To: KIRIYAMA Kazuhiko , FreeBSD Current References: <202311130700.3AD700uN016693@kx.truefc.org> From: John Baldwin In-Reply-To: <202311130700.3AD700uN016693@kx.truefc.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 11/12/23 11:00 PM, KIRIYAMA Kazuhiko wrote: > Hi, all > > I usually run bsdinstall by instllerconfig, but > bsdinstall/scriptedpart could not run ;-( > > My installerconfig is: > > PARTITIONS='nda0 gpt { 200M efi, 804G freebsd-ufs /, 128G freebsd-swap }' > DISTRIBUTIONS='base.txz kernel-dbg.txz kernel.txz lib32.txz tests.txz' > ZFSBOOT_DISKS="" > > #!/bin/sh > /bin/mkdir -p /.dake > /bin/cp /usr/share/zoneinfo/Asia/Tokyo /etc/localtime > /bin/cp /root/.cshrc /root/.cshrc.org > /bin/cat <> /etc/fstab > 192.168.1.17:/.dake /.dake nfs rw 0 0 > EOF > sed -i".bak" -Ee '/^#BDS_install.sh_added:start_line$/,/^#BDS_install.sh_added:end_line$/d' /root/.cshrc > /bin/cat <<'EOF' >> /root/.cshrc > #BDS_install.sh_added:start_line > setenv PATH ${PATH}:/.dake/bin > setenv MGRHOME /usr/home/admin > setenv OPENTOOLSDIR /.dake > setenv DAKEDIR /.dake > #BDS_install.sh_added:end_line > EOF > : > (snip) > : > > I investigated in bsdinstall script and found scriptedpart > which acutually run partedit with scriptedpart would not be > destroy existing partition. In fact scriptedpart -> partedit > changed in script as follows, then parttion editor run at > terminal. My guess is something to do with commit 23099099196548550461ba427dcf09dcfb01878d, though I don't see how it could work any differently in this case as the only change to part_config there was to return if if geom_gettree fails, and if it fails, provider_for_name would presumably have failed anyway. -- John Baldwin From nobody Wed Nov 15 16:39:39 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SVpmF4LSPz50Tfg; Wed, 15 Nov 2023 16:39:41 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SVpmF2n1xz3Zy5; Wed, 15 Nov 2023 16:39:41 +0000 (UTC) (envelope-from jhb@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700066381; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hVXpbp1vUeVAkC/E+/QE+QVp17+SylL5fR/o6A07Mao=; b=uQnXS7S8XklGRxHmamoUCiM4L48GKcVSuL8rVLs3+LtTXmblZD5e70mIhbiFDeoPhJfpP4 xwesWgv7ZEo9MElKg1kcYTQ4FRbJ/28BD78GprOzgycl47GqKtKVnmOR/tA6njSwJ3p/Gd c/kU4h6MY04Oisdf/D5hp0l5vFzyXgzwxax8ZSAHQKRbQUGSqjsaXL8SWn6ybnxskqYcAy RvD6I0yiBuIqTDfAIfbMIsoxyoCWTY15l/PU3j+5A31iyuoaKRQ+vX0FTQH9iuc9Qv3d7n RKsjzIrrGTaCONNYWoq1o1cWXfU9aFaL1crFoo16f9XG0mB73cJ3gbxkobE3aA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700066381; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hVXpbp1vUeVAkC/E+/QE+QVp17+SylL5fR/o6A07Mao=; b=hgjCLrfx21TYeBjaet+Ypn4yOpBwEFovvn04kUQMY8jleyDigRFrbKb/lLw1GvtDti8QLW v8ymEyiP8wLzviFBE3vMszopO+4IFTrqTpNt2EQcDaq4lI0ds2IPUqOP6XizoJeUUPyRsu uRiPkCXK5kPthQzYzC4U5mu5fujWnx/g/ewxtcrTTDjdEeyaU/8+pKPwpRMaH4zsLZ7oo5 LJk6YVVHJ9BN7MjtbEn+2UoXhwQp+yBL9h48dlt+skHYz/maHOqBzHEI+v3YLYLJc6eX6a NFlumBaEXKsPoRIb1Df60yLdZXxFO8rsHWIYveDoWXdHn/k9IJI7T38WZUfN5Q== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1700066381; a=rsa-sha256; cv=none; b=VHbGbzkTF1cH47ZdogjftyzaRgpL8Thy3N4YIwZ+TltkmBg5y3GGoalBdtKf4l5nREeCao QQwoAgqZt2PSA7a6pnArl5IAKZIocAgMVZsPExeDDs6YcYGQgM0DWRyE+VBSr0j+1UAXd7 SiXDkPZHN6nBMs8pFIg/XYQ6BlAliapDNdrMng8GO9roamrmOFdCcXpTcYTJacMYC6zmcV 4R4YLIOOHwg7tr4CuS3VCL/JCTlFh7EZKPzSkzihMVy4mH6ExPvHqPfMKRoR/qWQGf05N2 2cZwf/MdO2QYxwdZn+C5w2b4qdp+BYSEHs+KPFVm6xoIjfWRPWDqJREMFfZcbQ== Received: from [IPV6:2601:648:8384:fd00:7c48:55a7:81e0:3a0a] (unknown [IPv6:2601:648:8384:fd00:7c48:55a7:81e0:3a0a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: jhb) by smtp.freebsd.org (Postfix) with ESMTPSA id 4SVpmD5GGwz3sw; Wed, 15 Nov 2023 16:39:40 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Message-ID: Date: Wed, 15 Nov 2023 08:39:39 -0800 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [HEADS-UP] Quick update to 14.0-RELEASE schedule Content-Language: en-US To: Glen Barber , The Doctor Cc: FreeBSD Release Engineering Team , freebsd-current@freebsd.org, freebsd-stable@freebsd.org References: <20231114203654.GB52320@FreeBSD.org> <20231115022701.GM1307@FreeBSD.org> <20231115045231.GN1307@FreeBSD.org> From: John Baldwin In-Reply-To: <20231115045231.GN1307@FreeBSD.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 11/14/23 8:52 PM, Glen Barber wrote: > On Tue, Nov 14, 2023 at 08:10:23PM -0700, The Doctor wrote: >> On Wed, Nov 15, 2023 at 02:27:01AM +0000, Glen Barber wrote: >>> On Tue, Nov 14, 2023 at 05:15:48PM -0700, The Doctor wrote: >>>> On Tue, Nov 14, 2023 at 08:36:54PM +0000, Glen Barber wrote: >>>>> We are still waiting for a few (non-critical) things to complete before >>>>> the announcement of 14.0-RELEASE will be ready. >>>>> >>>>> It should only be another day or so before these things complete. >>>>> >>>>> Thank you for your understanding. >>>>> >>>> >>>> I always just installed my copy. >>>> >>> >>> Ok. I do not know what exactly is your point, but releases are never >>> official until there is a PGP-signed email sent. The email is intended >>> for the general public of consumers of official releases, not "yeah, >>> but"s. >>> >> >> Howver if you do a freebsd-update upgrade, you can upgrade. >> >> Is that suppose to happen? >> > > That does not say that the freebsd-update bits will not change *until* > the official release announcement has been sent. > > In my past 15 years involved in the Project, I think we have been very > clear on that. > > A RELEASE IS NOT FINAL UNTIL THE PGP-SIGNED ANNOUNCEMENT IS SENT. > > I mean, c'mon, dude. > > We really, seriously, for all intents and purposes, cannot be any more > clear than that. > > So, yes, *IF* an update necessitates a new freebsd-update build, what > you are running is *NOT* official. > > For at least 15 years, we have all said the same entire thing. Yes, but, if at this point we had to rebuild, it would have to be 14.0.1 or something (which we have done a few times in the past). It would be too confusing otherwise once the bits are built and published (where published means "uploaded to our CDN"). It is the 14.0 release bits, the only question is if for some reason we had a dire emergency that meant we had to pull it at the last minute and publish different bits (under a different release name). Realistically, once the bits are available, we can't prevent people from using them, it's just at their own risk to do so until the project says "yes, we believe these are good". Granted, they are under the same risk if they are still running the last RC. The best way to minimize that risk going forward is to add more automation of testing/CI to go along with the process of building release bits so that the build artifacts from the release build run through CI and are only published if the CI is green as that would give us greater confidence of "we believe these are good" before they are uploaded for publishing. -- John Baldwin From nobody Wed Nov 15 23:06:06 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SVzLN0P3kz50rTF for ; Wed, 15 Nov 2023 23:06:20 +0000 (UTC) (envelope-from bakul@iitbombay.org) Received: from mail-pf1-x432.google.com (mail-pf1-x432.google.com [IPv6:2607:f8b0:4864:20::432]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SVzLM53dwz3LBG for ; Wed, 15 Nov 2023 23:06:19 +0000 (UTC) (envelope-from bakul@iitbombay.org) Authentication-Results: mx1.freebsd.org; none Received: by mail-pf1-x432.google.com with SMTP id d2e1a72fcca58-6c34e87b571so158495b3a.3 for ; Wed, 15 Nov 2023 15:06:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iitbombay-org.20230601.gappssmtp.com; s=20230601; t=1700089578; x=1700694378; darn=freebsd.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=6FN7Ng6ZbdaYJJ/x6VeGgtvxVTHDF4I3JujhNUyTdA4=; b=RwXIDcI9tK1QfcE7XsdXJcPkJqX2hj3LWxkMtcYSl+zYSp4HsaPZsCaMZtBGtkTZFX 9P7OxLY122l3whQZyuSmPydmHIWooRtYtIW3DieBXCUTKye8bThKXKrAVzh7uzj/R0Dh Ow31vcJN18RiOK54GpfwpijV3e/0ggBxTBU63ZuE+k7E6nLL0OOWJB5q9GTnc1HmGqlu eFVvqYr592PUq0AsUYa3XZnrgMgXrypxANkYYDxYyYucWqmaAEe830qJG8HEgary9f47 h7hhAIvVtZfDBiRMsDJqv1JoyPGcC9FJUIi1yX70IYdAb0G/5BGtEQfneP8HBmncYd5x /X+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700089578; x=1700694378; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6FN7Ng6ZbdaYJJ/x6VeGgtvxVTHDF4I3JujhNUyTdA4=; b=MawMz3VWnqR7GrMVUckg61AivAvg75wtyL/9Q5O6FnNbswlmybSy4425jX9qtxV1NT C5a82rd48LD37xWvFpGr7le/eISaRzb+EfbcmL5ZDbXWxotJuVov6rX76nVkI70t7oS2 lA2lGs9qwsWBXtJC5VkV3nkV2KjtZh+nLn4SU1O9IvthmZPu4tDbQk7AMCYrojQsaYmb M6AF/aEoQDyODYUPfywKzEhGTkMyRAUl/7i0wGvmTDxRhsq2vOZtJnsVlZa+I5ZIHz/5 6fZ2XfcrS9gFEIGzTrYuBaAgVGOTAckS5NN5Ma3/398n2UbwMTe+Hmdl3xD9QeXtNPQG AU6A== X-Gm-Message-State: AOJu0YzYLdnl5B6X7YdGnZXVsh0KkSdg0qSawqbSCzYxQ1QT5EOj/hqv xb6iA1oaa7VTBdFsXtyLwb3IFw== X-Google-Smtp-Source: AGHT+IHd9vUTenDNbrghIm71SVtsqUeH2/cGvB/N7cCDfcfnfEA0tNkl0LYI1FQ1g7/3UIQnm3v+bg== X-Received: by 2002:a05:6a20:7353:b0:186:c0fe:b842 with SMTP id v19-20020a056a20735300b00186c0feb842mr9064057pzc.2.1700089578377; Wed, 15 Nov 2023 15:06:18 -0800 (PST) Received: from smtpclient.apple (107-215-223-229.lightspeed.sntcca.sbcglobal.net. [107.215.223.229]) by smtp.gmail.com with ESMTPSA id jd7-20020a170903260700b001ca222edc16sm7853803plb.135.2023.11.15.15.06.17 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 15 Nov 2023 15:06:17 -0800 (PST) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.200.91.1.1\)) Subject: Re: bhyve -G From: Bakul Shah In-Reply-To: Date: Wed, 15 Nov 2023 15:06:06 -0800 Cc: FreeBSD CURRENT , virtualization@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <234F558A-FB6A-4DE2-A99F-4B4F96C86EF0@iitbombay.org> References: To: John Baldwin X-Mailer: Apple Mail (2.3774.200.91.1.1) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4SVzLM53dwz3LBG > On Nov 15, 2023, at 7:57=E2=80=AFAM, John Baldwin = wrote: >=20 > On 10/9/23 5:21 PM, Bakul Shah wrote: >> Any hints on how to use bhyve's -G option to debug a VM >> kernel? I can connect to it from gdb with "target remote :" >> & bhyve stops the VM initially but beyond that I am not sure. >> Ideally this should work just like an in-circuit-emulator, not >> requiring anything special in the VM or kernel itself. >=20 > step only works on Intel CPUs currently (and is a bit fragile > anyway due to interrupts firing while you try to step, but that > happens for me in QEMU as well). Breakpoints should work fine. > I tend to use 'until' to do stepping (basically stepping via > temporary breakpoints) when debugging the kernel this way. Thanks for your response! I can ^C to stop the VM, examine the stack, set breakpoints, continue etc. but when the breakpoint is hit, kgdb doesn't regain control -- instead I get the usual db> ... prompt on the console. I guess I have to set some sysctl for this? Thanks, Bakul= From nobody Wed Nov 15 23:55:37 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SW0RH614jz50tqb for ; Wed, 15 Nov 2023 23:55:39 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SW0RH5XBQz3Yfx for ; Wed, 15 Nov 2023 23:55:39 +0000 (UTC) (envelope-from jhb@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700092539; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MtFyzTAuEnYyEJiboap03y8hiOyGqPn8jiUinO9prHs=; b=xU+k/CdXbbo35rs1i9f2hnP4BqfkxpuYklOEkGp6Bqjk+m6yf3IObE02rgYTWfrdHm3J5e bR1QnQlzpx+vTkyaJftNonxiKa2xHCpdswOUo7VDzm2D/NBWAOIt2BSIG0zxekxJWZ4Avp a4C/y7pEnym+dFsa+68A5RDih8th9JLTEU7KM7FZyMZhvHiaGJuWX9Ele5JpbiXNBYUJNp kq+NvmtNt9opQ70dRzeyLVcog/nhku9YLnLWaj2TtOwlOKfNX64WEq1NwKiZ3YFUonyOly bXjsuSKq6j7uGHmidSwkSrIqQNymFBJOBVNVIyoKWpPQBQ30UHJL+ih9r+nDMg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700092539; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MtFyzTAuEnYyEJiboap03y8hiOyGqPn8jiUinO9prHs=; b=wquzM15UXQpc2/BVLd+VPCM3xCUa23yGPgrwP14vqfET7N+Epyo+yejss54Ho/jwBQRp4C lrdqEeCrhjJNqwx8wSOJfJCD++C3RDijbdiwLn6gmi5mc8eyMVH40SzxdI5nolvOyqFv0d k4IP/flpWLZNBhm7qVM4TLPrT4NEPPs9/vfCokSEcvGazQsQ5h6qP7ZTCoLUO/ClVl5Sdp cOtGOFu/z7vs9bSvzliLTtc4gXlFOfEB7Wn74XK8eDInMGbDsSK6Eq03l65vvrijRXWKsf 5e0UCBsMJTgDu/NkW/05tIAgIyJJdW1u4WPeFN/isD5Ve5KH2EQyGWNfn22syw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1700092539; a=rsa-sha256; cv=none; b=Wi5WxQMWezON0SH7kXHmtdJOzmVnjysj2g9Hw5elU4AZVBloNfOdgiIw6Xw0KlhQfUtjUg rWn/QHiAR19PLc372MQVNgFeXGP1osl3oVWYXdiZnTzVrE7v0MvHfEcRJMw0dktH4RsV2s mHHurygwrXB48GJQCPY5A8Y0tCmRtlfb0T8eVfOjXIoFgFYqU5d9NChjanmyn64Akts2i+ dDg9qPGzg5bqft7Qw587c6yfnyNPolaKp3NcPektBV/S6tqoBI2erFrAnyzQOZX0EF/APg gCT5hD6yQsS+Lonm8rkeE0Q0sTwP4BOBj8d0rJ0ta/jAqaySz3ogx41oF2OYFg== Received: from [IPV6:2601:648:8384:fd00:7c48:55a7:81e0:3a0a] (unknown [IPv6:2601:648:8384:fd00:7c48:55a7:81e0:3a0a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: jhb) by smtp.freebsd.org (Postfix) with ESMTPSA id 4SW0RH3BF8zCGt for ; Wed, 15 Nov 2023 23:55:39 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Message-ID: <56b1a46c-db8c-4c0f-8d3e-2cbae7e8015c@FreeBSD.org> Date: Wed, 15 Nov 2023 15:55:37 -0800 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: bhyve -G Content-Language: en-US To: freebsd-current@freebsd.org References: <234F558A-FB6A-4DE2-A99F-4B4F96C86EF0@iitbombay.org> From: John Baldwin In-Reply-To: <234F558A-FB6A-4DE2-A99F-4B4F96C86EF0@iitbombay.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 11/15/23 3:06 PM, Bakul Shah wrote: > > >> On Nov 15, 2023, at 7:57 AM, John Baldwin wrote: >> >> On 10/9/23 5:21 PM, Bakul Shah wrote: >>> Any hints on how to use bhyve's -G option to debug a VM >>> kernel? I can connect to it from gdb with "target remote :" >>> & bhyve stops the VM initially but beyond that I am not sure. >>> Ideally this should work just like an in-circuit-emulator, not >>> requiring anything special in the VM or kernel itself. >> >> step only works on Intel CPUs currently (and is a bit fragile >> anyway due to interrupts firing while you try to step, but that >> happens for me in QEMU as well). Breakpoints should work fine. >> I tend to use 'until' to do stepping (basically stepping via >> temporary breakpoints) when debugging the kernel this way. > > Thanks for your response! > > I can ^C to stop the VM, examine the stack, set breakpoints, > continue etc. but when the breakpoint is hit, kgdb doesn't > regain control -- instead I get the usual > > db> ... > > prompt on the console. I guess I have to set some sysctl for > this? Hmm, no, it shouldn't be breaking into DDB in the guest as the breakpoint exception should be intercepted by the stub and never made visible to the guest. -- John Baldwin From nobody Thu Nov 16 00:30:30 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SW1CX5TLZz50w8f; Thu, 16 Nov 2023 00:30:32 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [96.47.72.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SW1CX4wFPz3csZ; Thu, 16 Nov 2023 00:30:32 +0000 (UTC) (envelope-from gjb@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700094632; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=v9VEuB/9ATJ7sc2r8hGcwu5OUWS1VYVO6p1jK5+z4mg=; b=ZQAUexmgcouMFYOR3MRyT8bXPi/sjajcrd0lB14c8eglvHsBzfgLODEIAuoj4b3w7R1rdn blOQbLx1+bOJ+/CaUfeb+STjsk+A2EQERLC7M/YTovwDEQR04JTCYbD/75Vk0v0ji8GJaR yf66vF8nK9rozucA2mssGH4EuP7N1GyDJWBMV5IEzz4vgFm23kU6ZxyYLB+ew17BM35irH a04sd9imBN4twKm4dzjqUaxeMwfGFiAmANGq+0zGCWW6CMQN/K6ExcITBJnkGc0TdI1cJG Iuy1ChnDcZNjNsk3SxEQzfjO/AIuNiplNTQya3PBhBdPxGTYQ0IWhXY+XnQ2Tw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700094632; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=v9VEuB/9ATJ7sc2r8hGcwu5OUWS1VYVO6p1jK5+z4mg=; b=FdJCi3zi4dVTyKVdbLxjkm5J6DUqJVtQkDxb5LMTa7GiTm14p5QJ3JBCN9Yg1NNxPQPF+1 qqRgW8mzw0cCsPsRZR0L8S+wk4fDrcLWC52EfyueVpcnpE8kmnuJ1vcQBiC5ZaGeb+dmBX LMZNlpXVXaCf65V/B56Y81QrkkCjj0DEzst/XbWFL53O072cXRsiCnmjIKcycSkJxeVq7E AlMR4YkIbstO5uvdXmZPOPkJ6j44YnVvGV0sVf9Wm392Swu2GUiO/B1wttfqFcjwl3NhT/ R0lghRTCjpuouDNARnL6maKoNPGDyNjh6qS4ON4NOX4EWjskicTlYdCAoKKIDw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1700094632; a=rsa-sha256; cv=none; b=s7oHqIoZ4QdKw7kL8/5XM+WMbuDJCILQIY3slJv+WmFVRx4KRT/724U/21EHOS77s8+r2R IvAQ7Nnmsf1h3Ows8h/UrFpZrZnLFnntPL8mB7UktdgGcugLP7gInjoSOTaPX8pZAlsF8b UWrc5rFE+tWN5lCfGu20NFhKuvGud8/V9BJZ3dkatAI4WrFYTeIW11zvJDtJR/RrucuRgL T3D+mw7urJ8NeoF5ijeQREWHxQ3p6+dbA8BRscX/XUO4EbnzZLFw/fmlg1pL+BMZb/Gnr9 xt0oWosjtPx92XwqhY5zIZxMP+g1+g0e5P3VDdzQFcvbfdgJHAaoBM4r3KT8Xg== Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by freefall.freebsd.org (Postfix) with ESMTPS id 4A53916228; Thu, 16 Nov 2023 00:30:32 +0000 (UTC) (envelope-from gjb@freebsd.org) Date: Thu, 16 Nov 2023 00:30:30 +0000 From: Glen Barber To: John Baldwin Cc: The Doctor , FreeBSD Release Engineering Team , freebsd-current@freebsd.org, freebsd-stable@freebsd.org Subject: Re: [HEADS-UP] Quick update to 14.0-RELEASE schedule Message-ID: <20231116003030.GO1307@FreeBSD.org> References: <20231114203654.GB52320@FreeBSD.org> <20231115022701.GM1307@FreeBSD.org> <20231115045231.GN1307@FreeBSD.org> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="iYlQhtYNd1Y0HGg2" Content-Disposition: inline In-Reply-To: --iYlQhtYNd1Y0HGg2 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Nov 15, 2023 at 08:39:39AM -0800, John Baldwin wrote: > On 11/14/23 8:52 PM, Glen Barber wrote: > > On Tue, Nov 14, 2023 at 08:10:23PM -0700, The Doctor wrote: > > > On Wed, Nov 15, 2023 at 02:27:01AM +0000, Glen Barber wrote: > > > > On Tue, Nov 14, 2023 at 05:15:48PM -0700, The Doctor wrote: > > > > > On Tue, Nov 14, 2023 at 08:36:54PM +0000, Glen Barber wrote: > > > > > > We are still waiting for a few (non-critical) things to complet= e before > > > > > > the announcement of 14.0-RELEASE will be ready. > > > > > >=20 > > > > > > It should only be another day or so before these things complet= e. > > > > > >=20 > > > > > > Thank you for your understanding. > > > > > >=20 > > > > >=20 > > > > > I always just installed my copy. > > > > >=20 > > > >=20 > > > > Ok. I do not know what exactly is your point, but releases are nev= er > > > > official until there is a PGP-signed email sent. The email is inte= nded > > > > for the general public of consumers of official releases, not "yeah, > > > > but"s. > > > >=20 > > >=20 > > > Howver if you do a freebsd-update upgrade, you can upgrade. > > >=20 > > > Is that suppose to happen? > > >=20 > >=20 > > That does not say that the freebsd-update bits will not change *until* > > the official release announcement has been sent. > >=20 > > In my past 15 years involved in the Project, I think we have been very > > clear on that. > >=20 > > A RELEASE IS NOT FINAL UNTIL THE PGP-SIGNED ANNOUNCEMENT IS SENT. > >=20 > > I mean, c'mon, dude. > >=20 > > We really, seriously, for all intents and purposes, cannot be any more > > clear than that. > >=20 > > So, yes, *IF* an update necessitates a new freebsd-update build, what > > you are running is *NOT* official. > >=20 > > For at least 15 years, we have all said the same entire thing. >=20 > Yes, but, if at this point we had to rebuild, it would have to be 14.0.1 > or something (which we have done a few times in the past). It would be > too confusing otherwise once the bits are built and published (where > published means "uploaded to our CDN"). It is the 14.0 release bits, > the only question is if for some reason we had a dire emergency that > meant we had to pull it at the last minute and publish different bits > (under a different release name). >=20 > Realistically, once the bits are available, we can't prevent people from > using them, it's just at their own risk to do so until the project says > "yes, we believe these are good". Granted, they are under the same risk > if they are still running the last RC. The best way to minimize that > risk going forward is to add more automation of testing/CI to go along > with the process of building release bits so that the build artifacts > from the release build run through CI and are only published if the CI > is green as that would give us greater confidence of "we believe these > are good" before they are uploaded for publishing. >=20 You are correct on all points. If there were a need to re-roll 14.0, it would indeed necessitate a release/14.0.1 tag. Note, release/14.0.0 has not yet been tagged, and I find it extremely unlikely that it will be necessary to rebuild from a release/14.0.1 tag. I also agree we cannot prevent people from downloading the images, installers, whatever before the announcement. That is the lovely race condition with which we have to live at the moment. My email was intended to be informative. Period. There were no suggestions that 14.0-RELEASE was not yet final. And to be fair, I had to personally deal with the fallout of a release "too soon", notably 11.0-RELEASE, where I thought a critical issue had been addressed, but I was wrong. My only point, in being overly open to the public on the delay, is that we (the Release Engineering Team) are not yet ready to rubber-stamp this as complete, as there are some outstanding items that are pending that have not yet completed. The alternative would be to say nothing at all. Either way, it is a productivity, communication drain. It is a lose-lose situation no matter how one looks at it given the above context. We either get chastised for being "too open" into insights delaying an official announcement, or for being "not open enough" when there is silence from RE when a release does not meet its scheduled announcement date. Glen --iYlQhtYNd1Y0HGg2 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEjRJAPC5sqwhs9k2jAxRYpUeP4pMFAmVVYqAACgkQAxRYpUeP 4pPB7g/+PRsNVNMwdBIu3ex8Tjp/J6uEfSlo3iUTOFIXQ98NxcdPVHPtYFr+dKWo TYGgHMuOIJWM6mtRIEDCVazYTkPr0si4lCnReXxq/splwrwyMoNa6GxVL7m8KVfH FozYvRqb+UFlkdUaZepVZSXq8ADtpUH2ocwRZaQjehiW3Cv9lHSH88ey3OVwg2li JX64AUvvJ/BBQxr0bYL8kTP1oJTmZzf4Fj3P64KAVnOXfkCR8o07uqqlrqQzMXIG SSKhuySq9PSAETS3UZ/QoH8srh4N7HrlyrPYjrG2EW8jogJ08GaDkrUozVFub+WR ikK3tNF8KXIZUi4mQ+CfwUfyls1GDrJsiZ79Hqs3vtEtAXd+lwntzLz3vdMr2Odx UDgAOK9qaDJDIhV70mcgmfkmWlindF0qm0HxRnhDPJzt8iMuTULHKNCK06vJ3/z0 ewZ2HnD9sbXV/9F6ETz5l2lSuLh4T7jSLaQOW5SDaxsDyoU/Lz/g0VvqEOjnQcah CsoH9LkYDS3Cda2x2RZ2FwTxVgh/v4t16B9K8a1p0Xw22aCQ+Dt4YW1nFzJCZEAO AOXk0co8fetv+k5oeydOINwsbEOV24Fx3Fea7yYXSpCk05/MBKZKVPbMGJI8zQQm qZb7t9fHUI1mpc06rDHk9IbwCtp84YzMpRHCNwHeuiWoqRQwC5c= =Ltbk -----END PGP SIGNATURE----- --iYlQhtYNd1Y0HGg2-- From nobody Thu Nov 16 03:19:39 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SW4ys0jCwz514Jj; Thu, 16 Nov 2023 03:19:49 +0000 (UTC) (envelope-from pete@nomadlogic.org) Received: from mail.nomadlogic.org (mail.nomadlogic.org [66.165.241.226]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SW4yr2ZSGz4QRx; Thu, 16 Nov 2023 03:19:48 +0000 (UTC) (envelope-from pete@nomadlogic.org) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nomadlogic.org; s=04242021; t=1700104780; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=wBPCnY43OxQjHdlKWYkNJqBjcGi+0dLfVovlYlYnuCI=; b=4kh5jj4kkthq84iFtXiXpwoJtqaLAiqpav8VCFwMbXEbThaXhy7kbv9jaz2U8mDOFSUw3v /hO6xgvpOF0EJgtfnT/atbMCxiantHwGevqMF2Dmz18S+ClzGoJVepYhpEKY/eH5f7zfa/ GC5DYr4wfvOnnqlgPo4Fl2aKqq8V3NE= Received: from [192.168.1.223] (cpe-24-24-168-214.socal.res.rr.com [24.24.168.214]) by mail.nomadlogic.org (OpenSMTPD) with ESMTPSA id 3d9d9019 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Thu, 16 Nov 2023 03:19:39 +0000 (UTC) Content-Type: multipart/alternative; boundary="------------koVe1rSEAnHTQ7IVSnZIv0gx" Message-ID: <23a4bd83-dca4-4e76-a513-9df6671d930d@nomadlogic.org> Date: Wed, 15 Nov 2023 19:19:39 -0800 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [HEADS-UP] Quick update to 14.0-RELEASE schedule To: Glen Barber , John Baldwin Cc: The Doctor , FreeBSD Release Engineering Team , freebsd-current@freebsd.org, freebsd-stable@freebsd.org References: <20231114203654.GB52320@FreeBSD.org> <20231115022701.GM1307@FreeBSD.org> <20231115045231.GN1307@FreeBSD.org> <20231116003030.GO1307@FreeBSD.org> Content-Language: en-US From: Pete Wright In-Reply-To: <20231116003030.GO1307@FreeBSD.org> X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:29802, ipnet:66.165.240.0/22, country:US] X-Rspamd-Queue-Id: 4SW4yr2ZSGz4QRx This is a multi-part message in MIME format. --------------koVe1rSEAnHTQ7IVSnZIv0gx Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 11/15/23 16:30, Glen Barber wrote: > The alternative would be to say nothing at all. > > Either way, it is a productivity, communication drain. It is > a lose-lose situation no matter how one looks at it given the above > context. We either get chastised for being "too open" into insights > delaying an official announcement, or for being "not open enough" when > there is silence from RE when a release does not meet its scheduled > announcement date. Glen I appreciate your transparency and the efforts you have done to keep everyone in the loop.  I think people get excited about new releases, which is probably a good thing.  IMHO i feel you've been striking a good balance. As an operator these updates are really helpful for me as it allows me to adjust timelines on my end for updating my fleet of servers. You and the RE team do an incredible job - and personally I am thankful for the caution and common sense you all bring to this complex process. Cheers, -pete -- Pete Wright pete@nomadlogic.org @nomadlogicLA --------------koVe1rSEAnHTQ7IVSnZIv0gx Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit

On 11/15/23 16:30, Glen Barber wrote:
The alternative would be to say nothing at all.

Either way, it is a productivity, communication drain.  It is
a lose-lose situation no matter how one looks at it given the above
context.  We either get chastised for being "too open" into insights
delaying an official announcement, or for being "not open enough" when
there is silence from RE when a release does not meet its scheduled
announcement date.
Glen I appreciate your transparency and the efforts you have done to keep everyone in the loop.  I think people get excited about new releases, which is probably a good thing.  IMHO i feel you've been striking a good balance.

As an operator these updates are really helpful for me as it allows me to adjust timelines on my end for updating my fleet of servers.  You and the RE team do an incredible job - and personally I am thankful for the caution and common sense you all bring to this complex process.

Cheers,
-pete 

-- 
Pete Wright
pete@nomadlogic.org
@nomadlogicLA
--------------koVe1rSEAnHTQ7IVSnZIv0gx-- From nobody Thu Nov 16 03:31:08 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SW5Cy5B0Kz515S2; Thu, 16 Nov 2023 03:31:10 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [96.47.72.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SW5Cy4h5tz4TDL; Thu, 16 Nov 2023 03:31:10 +0000 (UTC) (envelope-from gjb@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700105470; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=zYsjPfX7A6+EJmjurvlHMHk6xNK53VeYV70cWZaz5MI=; b=jwwnoPb5/bAe1b0yHnHJf2t5P2Swe/l4FKcDco/2Gk2ri8lWkmfxQ7cTVy1o57V2589Uod uDqHfbBQSa1H0N14CJSSInOov9cjIkJf/cAyDjc2EWD/bHlOjJSft7sgBS0tN5lRZCLPeK CgdSDCUkirzsP97vshCAXCPsAyAi3qjpSvXBF/K8yu/c5Zq5EguO4Gt99Pa9h+jVUVrU8T N07y/9xgC/ZhO7HjlYwMwL1ntJ/F8UtVtPHSq1U9E7R/rfkbleneOGfEtVtexl2azKPlkF IxTgn6l7PlhKZc1221a19XkfDmLjEP3ZjT2XSXcqF3klv6Ra/8ceuNw0wDq2wA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700105470; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=zYsjPfX7A6+EJmjurvlHMHk6xNK53VeYV70cWZaz5MI=; b=pIQpryeQ6MJDIcKJLOe8ZSF4GdDwS+xGQJphuUYoTjFBEBF5gohlM5NY9+LviyFJte397u uemS9s/a4KkjMMt/wWqQS9e0qzWGAAHB3X31sUY/2YDddC+Sgg5+74CUKXHrrOe0zl/dyo otsKaq43YvWSug0v7LGBApWXUt8Zt7C4pDf0YqaNstEiPK/WowlhcVXS++i7Diwo9v1Pn7 UmLmpgYH9ek+wMchMxXvmHbidxQzxG/MenCFFnrygTpSISfMlKhTG1LKKJnNceG4Ub9qc1 Q2ZHT1AhuUeJ6U8RI5MOZPvkyqdn2ZmRgyXthvaWQGRO41O7fWBUBlTq3xfnrA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1700105470; a=rsa-sha256; cv=none; b=AV4pbHwXEkPkvaEYm/qx4fbC5AzdJCbq6wr26ZvbfHT3bNoctNAjKICPlmEQJGb5MyFUJ5 xcqEjnTLnnxjxp+girAceJx9tQEvSpJuxfM9O6IOENVLS0k+bmoX6WBeO0zCMD+rnqyFG8 bD7Me1TfiRAEXvuyaYzhX3/8cpTbX2p/nfwRFUnt8525NoaHiZC2TjW+0fVJHkDQZdr3Qi ppdDGcjt9ATQA3hfFWIqUf6pO5Dzofp6aCYeAYxjlK2/uFu1bciFB1WyYU6UCiBa1vGLOf 2dMbqLrPm7P5Ayj4TPjwLa16VD3ZegOI8/8RuU3YFIYDlPswjgOGeD9xdjZ3gw== Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by freefall.freebsd.org (Postfix) with ESMTPS id 15DF516843; Thu, 16 Nov 2023 03:31:10 +0000 (UTC) (envelope-from gjb@freebsd.org) Date: Thu, 16 Nov 2023 03:31:08 +0000 From: Glen Barber To: Pete Wright Cc: John Baldwin , The Doctor , FreeBSD Release Engineering Team , freebsd-current@freebsd.org, freebsd-stable@freebsd.org Subject: Re: [HEADS-UP] Quick update to 14.0-RELEASE schedule Message-ID: <20231116033108.GP1307@FreeBSD.org> References: <20231114203654.GB52320@FreeBSD.org> <20231115022701.GM1307@FreeBSD.org> <20231115045231.GN1307@FreeBSD.org> <20231116003030.GO1307@FreeBSD.org> <23a4bd83-dca4-4e76-a513-9df6671d930d@nomadlogic.org> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="BC0PqexbI+QDQ83Q" Content-Disposition: inline In-Reply-To: <23a4bd83-dca4-4e76-a513-9df6671d930d@nomadlogic.org> --BC0PqexbI+QDQ83Q Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Nov 15, 2023 at 07:19:39PM -0800, Pete Wright wrote: >=20 >=20 > On 11/15/23 16:30, Glen Barber wrote: > > The alternative would be to say nothing at all. > >=20 > > Either way, it is a productivity, communication drain. It is > > a lose-lose situation no matter how one looks at it given the above > > context. We either get chastised for being "too open" into insights > > delaying an official announcement, or for being "not open enough" when > > there is silence from RE when a release does not meet its scheduled > > announcement date. > Glen I appreciate your transparency and the efforts you have done to keep > everyone in the loop.=A0 I think people get excited about new releases, w= hich is > probably a good thing.=A0 IMHO i feel you've been striking a good balance. >=20 > As an operator these updates are really helpful for me as it allows me to > adjust timelines on my end for updating my fleet of servers. You and the = RE > team do an incredible job - and personally I am thankful for the caution = and > common sense you all bring to this complex process. >=20 Thank you very much, Pete, for your email on this. It is appreciated by all of us involved in FreeBSD releases. Best regards, Glen --BC0PqexbI+QDQ83Q Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEjRJAPC5sqwhs9k2jAxRYpUeP4pMFAmVVjPwACgkQAxRYpUeP 4pPPmw//TYnmbxII8IogAFd+a/3RcmxiltsGVWWlbBX6/bN4qEHLrVzNANkryZR0 bwXBT/CIHL4cdP8U7/MT0TrCeXf+ngK3+RcXPg16OFAv7img0su7+Ru/G9uHnirb 0QRwzHRsAq3ZzHqsyOujcuYGJGIQPAMF7n3JkM6paXzPYwR09S2mGxZhn/U9kzQN LN91BaMllY1ewDH3SdldCvuPKfXDmCFcInTKREl3E+jKbBlkqW1FaqB1XvdOpavi B6Vl5Zw3cM/lnoq6D5ugmjqkF/e0EW0cPhVbAM2P8vTGCi1UIcNAemhaKWkZnySr Qwo0qOXyk0nCcdI4C/AVdVLAL34FYXr6XKyuqmpezm76IY9f2SiUZnVJTGLc+yvm Ym9nawmP7Fz0dcNO7heVv0raLHlHty9mN7F2SOejjGlINTqD/8WYt8gmYRWbh0Qz qrAosM0eVHpfAGZUt/Z+t5eLrqnhK3PKvqfW9bLQOGaW5+K+bUKY/FTin4eVuFe4 SPjEAN/eUO/YfUQzRbA5xTAppDbrfAgn+gDbqW/lwGkHr04pp533CxNLorCMPaV7 M5VN5U4ppydn7d8RIMwnxzK5fqG+3snU35zIONjj6ZHiP6lKTMwrqEO1HeI2URHM tkA/oFO3lVGkItBjX3qf8bUSgPNjbh8czCCZjkhThQh0l+Zb3Vw= =EkWh -----END PGP SIGNATURE----- --BC0PqexbI+QDQ83Q-- From nobody Thu Nov 16 04:01:56 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SW5vq2MKyz516h1 for ; Thu, 16 Nov 2023 04:02:15 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic316-8.consmr.mail.gq1.yahoo.com (sonic316-8.consmr.mail.gq1.yahoo.com [98.137.69.32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SW5vp0xhsz4XtZ for ; Thu, 16 Nov 2023 04:02:13 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=rdr44bze; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.32 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1700107331; bh=owsnKU8MSAZGABLiz3ZpTnehUZ1AkdwjQwyO+O0hVaA=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=rdr44bzet59/2Zb2JN00xqwume37agiXguapUSMSvJCpgLFDC2bSd00zrE4veRlaavpqB0RHLoP2XLKWBIBo3y2QOgkv0nwXguqEowgTj4N83bJR01WJGVH8YiGbKDa2cTe/zmlguWFvXgtHJvRxklXIocjAHQHSULMDh/Vum6khS+O81O75fvVVSnQIHbHgPrpof47f3FGdxqjqXQxPPUxx/2WQylBF5G5CLnDlVJNbQdrhHsk/A+YAj4uQsDh4q++LLfTAHIkPPe4OYEG3VmqUXQ7Y+GN4lAguPIhKgEPuJyXWIurrPUdtmsk5QiKccjtItB2W7T28cZQcWbDqGA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1700107331; bh=NTrE5+v+Pky56wPPO2lAixCmD3f+4lQ7Jv6bBdlTwyH=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=IfC9+Gv5Nv7w6czt36K8dOIFs/ieWR6HsePH+0B/8BRda85I4PIzTBS7GJnvl06ovhicWvaWwatzKizo5htqjWMY0FVGH2LaICIypCIDLzmMHReCQgw5VI4GoRvsw8QpbW1Uf3vomwF7Ke+Kd4EqGAhRNpTaMGYEnjKr9XlL5XfoGcFKQexLKLsBI6wyEoucXqqyOc/qKXT0GQhz+TmiMgKJ72rp+R8WIDNztZ4jFDBO6sRy2Drm4EKhhwwvq4AtVG58uuvs6Jm9WoGf0F3A2m4s3TpJ10rUaKfw9FyS/V8Uxw0BrxeMyAP9JBAGthw2ar1mV6yfaLRKXs/7BSBinw== X-YMail-OSG: 9zCa6tMVM1m5AJS8lTcm71OlW1lsZn_sKAZ3r8HTCME09L52YFi8T5yoUHAFKHb LiMRl3O8O_Ephd1XZKaXqr8EjuINIXYbjo9KKwrii6mz5NeCfgTnzpc2EfuQeIbenhzQ536FoJlZ sQ2XGOO8D.MkGGmY5Ds.jCgoiKNRBljFyHTUUN6EYjPgiQahWcI8V7WV8iI_f3ixQ7deRiPUXoHR 20c2Y_7ouB.AFe1_F5ljDzI0US9cIAUZH9JuY9aYSpKhuS8J35TuunY9t95RhB4PsApNpc7sZSc. kYKza1.1D37b0UEBfNUpMXNQbdjt7UDH84W6wn3nNhEvuhdgnt8YFFE8o5HQIsl9o.O.tpxQjdgo B5A7m7o6pKFxE3tHW_aiYaX39m0d8Hq8YxM8QJYest_cGe2wRGRClRKGK94uYxEpHhKTB1nvNhEy 2avhB0y6kaSsPbTEPyZoWIJVTGK_YmQs7nBsJcuFUAP4oHCVoHdyW4XdwT3qYIkbViWGOLzN4swT u1xE7EyNpR0Verp5foRWlVZYgSAGrOZHz9zP5m5DiqW_FpfMohGz..CD_Sg38QcnwwHqJckJsnxv Cu54cugT7RC8_8kEfDcFEEu_RUkf7Fi7d4mlje72NMbqBWqgpmczFJMq7qa5NauHOeti1b1iW_1V r8LPtwre3y0xDfY5ct.texW25clVVM2VoFlHQkuEuUL3zHtmQPPkmznPS1TpRRndPrFGVhpRg_eV vKmu32mKh0TBTQJwH0vtZjMnQNMwK.OpjUJEk1pxVqzWYjRyYAcrHmnZBpQ8vOnKB9mpEfWglosy ImhQ2HpqXtLO.DJIdfvgmDnnti3qJ9p0.ThhjMd3xsJFDMk9s6TYMJyiasR7v6_C09EBoUCbRo3Y SwSTaeZ5yzWsU9Q0WOl0SBXw4nGyENeII5vu5woA5bPGWYdgzMMV0QGApzrLoAwy8LKJNcJ5QbCA ktth5tpggstf2qkziT5fbStADIqvx3xkjkTM4JZUvMaE_vSEWwgInHDXRBwRehkNqDrAQAEvfADa NVXyCORrJi_KKjy7kvYQlt5s1TYrW1RJMTHosGK1VFMddylLZuiD2AdWxH9T_7F4L9BR8iyesrzQ _rJmqKGBY0.GJu60TOHd5IABsQsuWm7tWb7LHxb1qfEetnmf1dWya.EGfsTAkmP8XEdYl1kg0a7y yGa9762fYw5QuGX.GccXZMhaiW2C1O1Z64GhfGwTHLv1Zg98ZbwB9It7U3nwt88E20s8jUnd6T6j ATuynvJ1_r_Hh7jYdp_r.AA4J3T52KUJGjbeQpXL_rSn6oYLRD6MhocTMAkPziFKXkrmGQVOsKiz AtCCXXh9q9wjGufvknkdtiuiKvyitAZh.Vk4j9nvVpljEyu6Sm4PcoIqH4_Y_Yzy0mb2ubG.NNmY inruvFjt_uyP9om4OsMVRM6YSouKvchOnzJUXM.dBzsa_E7XVEmwzaq.XOSN16Mg34qipRdvxLhC Ti.Cg3VKItiwUJ7TJD3358JjjadBikKnLua3QdhyxJMvq3FiP72ShWlGmev.YbtjdMDrrJyQn8hM nDN.AWaJzizAE8.M3pCcyK99r_0D0BjhW8Bg5_IWQrNUx.G1iDRhxOu5QQL2LIzU0YSYUmokLikc Ayi_3gU8mDXSrQ7.KC0HIry0p1.FjfKD2ncxeEduD8q_I682_ji0yZgr_5EWQBZUQ3_l3eW2hqY1 EXuUI2WCC4cGIRAITWYLgSMXO7Tr0tMglLYaNZRi2Vq5hlUhcg_EIG9DJmODak0z8uMwj6HRV1OG dsezOmG5WpnVizfN.E3CmU4bqI__3.hoMlIcdjqCJSyPSwTcfzNNi7b4BYTpVSM6HQPy88ClZ_8r gOsz7t1mOqYNHoH7XUjvuWt4QmG7tVOm3JZAHFmf7VYb4_v4G9GWcRIakVaFoArhKKKr5DFmb43n yLL8Ic4L3gQ6L2hrdAZOHaaJx._YI0_5vLNcqdSw6V1XjxWndm8Tui2v9bsBcF7ckvBHPyJ.XfVf 5V0lBD7yvfEz7yRsU5_f6Jtp9HpiX3ldK0XEHMdet2nal1te8mDBTeheR_VaJuPuH_pq3OEnQuKD hropc.Wp_JC.uosryQGU3F9ZOlVJnElGXkPY_w1M7j53Q97cpBgR2ils1kD2SEHH3YaqtYvPtn0B ZckGexDmSv2nZx7L4.aTxrJl4nAocULnhKWcdkyMQPHV0PMNR7wOPl1Zjjd12gcYLHjz.S1yyzIz 0bQvk0aCU4u6DOaGSVMkvIdckGrb56GZ0.3cVerTze1H0EYU57EHy1mkhvIVrYc2QS.vjmh5C33I 5 X-Sonic-MF: X-Sonic-ID: 3a4d4e7a-1535-4141-81e1-b3f7da2025da Received: from sonic.gate.mail.ne1.yahoo.com by sonic316.consmr.mail.gq1.yahoo.com with HTTP; Thu, 16 Nov 2023 04:02:11 +0000 Received: by hermes--production-gq1-59b5df67b6-srhh5 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 6a1e3410b172dda9bebc4be4ea26bb79; Thu, 16 Nov 2023 04:02:06 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.200.91.1.1\)) Subject: FYI: poudriere-devel builder got stuck in kqread during from scratch bulk -a Message-Id: <9BBA5DFF-50ED-4007-8C90-EC5099AB4C7C@yahoo.com> Date: Wed, 15 Nov 2023 20:01:56 -0800 To: FreeBSD Mailing List , Current FreeBSD X-Mailer: Apple Mail (2.3774.200.91.1.1) References: <9BBA5DFF-50ED-4007-8C90-EC5099AB4C7C.ref@yahoo.com> X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; ARC_NA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.32:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.32:from]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MID_RHS_MATCH_FROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4SW5vp0xhsz4XtZ X-Spamd-Bar: --- Tail of the poudriere-devel Building and Finished notices just before = only the stuck builder was left "running": . . . [85:21:50] [27] [02:06:01] Finished databases/mongodb60 | = mongodb60-6.0.11: Success [85:34:00] [28] [03:23:06] Finished biology/ncbi-cxx-toolkit | = ncbi-cxx-toolkit-27.0.0_1: Success [85:46:31] [30] [08:19:30] Finished cad/kicad-library-packages3d | = kicad-library-packages3d-7.0.2_2: Success [87:07:02] [03] [13:00:45] Finished emulators/libretro-mame | = libretro-mame-20220124_1: Success # poudriere status -b [main-amd64-bulk_a-default] [2023-11-11_17h59m25s] [parallel_build:] = Queued: 34683 Built: 33807 Failed: 173 Skipped: 382 Ignored: 320 = Fetched: 0 Tobuild: 1 Time: 88:17:59 ID TOTAL ORIGIN PKGNAME PHASE PHASE TMPFS = CPU% MEM% [05] 17:27:25 ftp/curlie | curlie-1.6.7_15 check-sanity 17:27:15 1.28 = GiB =20 =3D>> Logs: = /usr/local/poudriere/data/logs/bulk/main-amd64-bulk_a-default/2023-11-11_1= 7h59m25s # ps -oetime -alxdww # but left vs. right part split out, left side = first (with 2 notes) . . . . . . 3-16:18:11 0 83812 1765 8 20 0 22128 6856 select I+ 0 = 4:33.84 | 17:27:35 0 48809 83812 26 68 0 22128 3044 kqread I 0 = 0:00.05 | (<<=3D=3Dbuild_pkg) 17:27:25 0 50788 48809 25 88 0 0 0 - Z 0 = 0:00.03 | (<<=3D=3Ddefunct) 3-16:16:40 0 76609 83812 19 34 0 18544 3608 piperd I+ 0 = 16:45.55 | 3-16:18:09 0 84173 83812 13 68 0 15472 2864 nanslp S+ 0 = 184:47.04 | . . . Then right side . . . . . . | `-- /usr/local/libexec/poudriere/sh -e = /usr/local/share/poudriere/bulk.sh -jmain-amd64-bulk_a -a | |-- sh: poudriere[main-amd64-bulk_a-default][05]: build_pkg = (curlie-1.6.7_15) (sh) | | `-- | |-- /usr/local/libexec/poudriere/sh -e = /usr/local/share/poudriere/bulk.sh -jmain-amd64-bulk_a -a | `-- /usr/local/libexec/poudriere/sh -e = /usr/local/share/poudriere/bulk.sh -jmain-amd64-bulk_a -a . . . Later forcing a failure of the builder that was stuck for curlie: . . . [main-amd64-bulk_a-default] [2023-11-11_17h59m25s] [committing:] Queued: = 34683 Built: 33807 Failed: 174 Skipped: 382 Ignored: 320 Fetched: = 0 Tobuild: 0 Time: 88:46:04 [88:46:05] Logs: = /usr/local/poudriere/data/logs/bulk/main-amd64-bulk_a-default/2023-11-11_1= 7h59m25s [88:46:07] Cleaning up [88:46:07] Unmounting file systems Context, for reference: # uname -apKU FreeBSD amd64-ZFS 15.0-CURRENT FreeBSD 15.0-CURRENT #126 = main-n266130-d521abdff236-dirty: Tue Oct 24 18:17:40 PDT 2023 = root@amd64-ZFS:/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.a= md64/sys/GENERIC-NODBG amd64 amd64 1500002 1500002 # ~/fbsd-based-on-what-commit.sh -C /usr/ports 6ec8e3450b29 (HEAD -> main, freebsd/main, freebsd/HEAD) devel/sdts++: = Mark DEPRECATED Author: Muhammad Moinur Rahman Commit: Muhammad Moinur Rahman CommitDate: 2023-10-21 19:01:38 +0000 branch: main merge-base: 6ec8e3450b29462a590d09fb0b07ed214d456bd5 merge-base: CommitDate: 2023-10-21 19:01:38 +0000 n637598 (--first-parent --count for merge-base) ThreadRipper 1950X system, 32 hardware threads, 128 GiBytes of RAM, ZFS. USE_TMPFS=3Dall in use. The style of bulk -a build has high load averages for much of the bulk -a compared to the hardware thread count. Largest swap space use observed was 123513 MiBytes, observed via a hacked top that tracks some "Maximum Observed" figures for some of top's monitoring data. For reference: 245564Mi MaxObs(Act+Wir+Lndry+SwapUsed) Note: max(A+B) <=3D max(A) + max(B) =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Thu Nov 16 03:38:13 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SW6k46qF7z518KR; Thu, 16 Nov 2023 04:38:52 +0000 (UTC) (envelope-from doctor@doctor.nl2k.ab.ca) Received: from doctor.nl2k.ab.ca (doctor.nl2k.ab.ca [204.209.81.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SW6k44fDhz4dyZ; Thu, 16 Nov 2023 04:38:52 +0000 (UTC) (envelope-from doctor@doctor.nl2k.ab.ca) Authentication-Results: mx1.freebsd.org; none Received: from doctor by doctor.nl2k.ab.ca with local (Exim 4.97 (FreeBSD)) (envelope-from ) id 1r3TCn-00000000J12-48Nx; Wed, 15 Nov 2023 20:38:13 -0700 Date: Wed, 15 Nov 2023 20:38:13 -0700 From: The Doctor To: Glen Barber Cc: John Baldwin , FreeBSD Release Engineering Team , freebsd-current@freebsd.org, freebsd-stable@freebsd.org Subject: Re: [HEADS-UP] Quick update to 14.0-RELEASE schedule Message-ID: References: <20231114203654.GB52320@FreeBSD.org> <20231115022701.GM1307@FreeBSD.org> <20231115045231.GN1307@FreeBSD.org> <20231116003030.GO1307@FreeBSD.org> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20231116003030.GO1307@FreeBSD.org> X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:6171, ipnet:204.209.81.0/24, country:CA] X-Rspamd-Queue-Id: 4SW6k44fDhz4dyZ On Thu, Nov 16, 2023 at 12:30:30AM +0000, Glen Barber wrote: > On Wed, Nov 15, 2023 at 08:39:39AM -0800, John Baldwin wrote: > > On 11/14/23 8:52 PM, Glen Barber wrote: > > > On Tue, Nov 14, 2023 at 08:10:23PM -0700, The Doctor wrote: > > > > On Wed, Nov 15, 2023 at 02:27:01AM +0000, Glen Barber wrote: > > > > > On Tue, Nov 14, 2023 at 05:15:48PM -0700, The Doctor wrote: > > > > > > On Tue, Nov 14, 2023 at 08:36:54PM +0000, Glen Barber wrote: > > > > > > > We are still waiting for a few (non-critical) things to complete before > > > > > > > the announcement of 14.0-RELEASE will be ready. > > > > > > > > > > > > > > It should only be another day or so before these things complete. > > > > > > > > > > > > > > Thank you for your understanding. > > > > > > > > > > > > > > > > > > > I always just installed my copy. > > > > > > > > > > > > > > > > Ok. I do not know what exactly is your point, but releases are never > > > > > official until there is a PGP-signed email sent. The email is intended > > > > > for the general public of consumers of official releases, not "yeah, > > > > > but"s. > > > > > > > > > > > > > Howver if you do a freebsd-update upgrade, you can upgrade. > > > > > > > > Is that suppose to happen? > > > > > > > > > > That does not say that the freebsd-update bits will not change *until* > > > the official release announcement has been sent. > > > > > > In my past 15 years involved in the Project, I think we have been very > > > clear on that. > > > > > > A RELEASE IS NOT FINAL UNTIL THE PGP-SIGNED ANNOUNCEMENT IS SENT. > > > > > > I mean, c'mon, dude. > > > > > > We really, seriously, for all intents and purposes, cannot be any more > > > clear than that. > > > > > > So, yes, *IF* an update necessitates a new freebsd-update build, what > > > you are running is *NOT* official. > > > > > > For at least 15 years, we have all said the same entire thing. > > > > Yes, but, if at this point we had to rebuild, it would have to be 14.0.1 > > or something (which we have done a few times in the past). It would be > > too confusing otherwise once the bits are built and published (where > > published means "uploaded to our CDN"). It is the 14.0 release bits, > > the only question is if for some reason we had a dire emergency that > > meant we had to pull it at the last minute and publish different bits > > (under a different release name). > > > > Realistically, once the bits are available, we can't prevent people from > > using them, it's just at their own risk to do so until the project says > > "yes, we believe these are good". Granted, they are under the same risk > > if they are still running the last RC. The best way to minimize that > > risk going forward is to add more automation of testing/CI to go along > > with the process of building release bits so that the build artifacts > > from the release build run through CI and are only published if the CI > > is green as that would give us greater confidence of "we believe these > > are good" before they are uploaded for publishing. > > > > You are correct on all points. If there were a need to re-roll 14.0, it > would indeed necessitate a release/14.0.1 tag. Note, release/14.0.0 has > not yet been tagged, and I find it extremely unlikely that it will be > necessary to rebuild from a release/14.0.1 tag. > > I also agree we cannot prevent people from downloading the images, > installers, whatever before the announcement. That is the lovely race > condition with which we have to live at the moment. > > My email was intended to be informative. Period. > > There were no suggestions that 14.0-RELEASE was not yet final. And to > be fair, I had to personally deal with the fallout of a release "too > soon", notably 11.0-RELEASE, where I thought a critical issue had been > addressed, but I was wrong. > > My only point, in being overly open to the public on the delay, is that > we (the Release Engineering Team) are not yet ready to rubber-stamp this > as complete, as there are some outstanding items that are pending that > have not yet completed. > > The alternative would be to say nothing at all. > > Either way, it is a productivity, communication drain. It is > a lose-lose situation no matter how one looks at it given the above > context. We either get chastised for being "too open" into insights > delaying an official announcement, or for being "not open enough" when > there is silence from RE when a release does not meet its scheduled > announcement date. > > Glen > > What ahappened was that I inititated the freebsd-upgrade RELEASE upgrade command. question should that RELEASe been released? -- Member - Liberal International This is doctor@nk.ca Ici doctor@nk.ca Yahweh, King & country!Never Satan President Republic!Beware AntiChrist rising! Look at Psalms 14 and 53 on Atheism ; unsubscribe from Google Groups to be seen Lest we forget 11 Nov 2023 Beware https://mindspring.com From nobody Thu Nov 16 05:06:01 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SW7KR2Rznz519fc; Thu, 16 Nov 2023 05:06:03 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from mail0.glenbarber.us (mail0.glenbarber.us [IPv6:2607:fc50:1:2300:1001:1001:1001:face]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail0.glenbarber.us", Issuer "Gandi Standard SSL CA 2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SW7KR1sn1z3FXy; Thu, 16 Nov 2023 05:06:03 +0000 (UTC) (envelope-from gjb@freebsd.org) Authentication-Results: mx1.freebsd.org; none Received: from smtpclient.apple (50.29.233.174.res-cmts.swb2.ptd.net [50.29.233.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: gjb) by mail0.glenbarber.us (Postfix) with ESMTPSA id 94007D535; Thu, 16 Nov 2023 05:06:02 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.10.3 mail0.glenbarber.us 94007D535 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (1.0) Subject: Re: [HEADS-UP] Quick update to 14.0-RELEASE schedule From: Glen Barber In-Reply-To: Date: Thu, 16 Nov 2023 00:06:01 -0500 Cc: FreeBSD Release Engineering Team , freebsd-current@freebsd.org, freebsd-stable@freebsd.org Message-Id: <9D0AE587-C9AE-4751-9B08-B4AF6811B633@freebsd.org> References: To: Rich Reynolds X-Mailer: iPhone Mail (20G81) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36236, ipnet:2607:fc50::/36, country:US] X-Rspamd-Queue-Id: 4SW7KR1sn1z3FXy Thank you for that. Glen Sent from my phone. Please excuse my brevity and/or typos. > On Nov 15, 2023, at 11:17 PM, Rich Reynolds wrote= : >=20 > =EF=BB=BFOn 11/15/23 20:19, Pete Wright wrote: >>> On 11/15/23 16:30, Glen Barber wrote: >>> The alternative would be to say nothing at all. >>>=20 >>> Either way, it is a productivity, communication drain. It is >>> a lose-lose situation no matter how one looks at it given the above >>> context. We either get chastised for being "too open" into insights >>> delaying an official announcement, or for being "not open enough" when >>> there is silence from RE when a release does not meet its scheduled >>> announcement date. >> Glen I appreciate your transparency and the efforts you have done to keep= everyone in the loop. I think people get excited about new releases, which= is probably a good thing. IMHO i feel you've been striking a good balance.= >> As an operator these updates are really helpful for me as it allows me to= adjust timelines on my end for updating my fleet of servers. You and the RE= team do an incredible job - and personally I am thankful for the caution an= d common sense you all bring to this complex process. >> Cheers, >> -pete >> --=20 >> Pete Wright >> pete@nomadlogic.org >> @nomadlogicLA > +1 from a retired RE manager > yet another hard thankless job in the development chain... >=20 > --=20 > When you beleve in things. that you don't understand, > then you suffer, superstition ain't the way. > Stevie Wonder - 1972 >=20 From nobody Thu Nov 16 05:07:36 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SW7MF67Slz519j0; Thu, 16 Nov 2023 05:07:37 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from mail0.glenbarber.us (mail0.glenbarber.us [IPv6:2607:fc50:1:2300:1001:1001:1001:face]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail0.glenbarber.us", Issuer "Gandi Standard SSL CA 2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SW7MF3Fhwz3HYY; Thu, 16 Nov 2023 05:07:37 +0000 (UTC) (envelope-from gjb@freebsd.org) Authentication-Results: mx1.freebsd.org; none Received: from smtpclient.apple (50.29.233.174.res-cmts.swb2.ptd.net [50.29.233.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: gjb) by mail0.glenbarber.us (Postfix) with ESMTPSA id BDE4CD544; Thu, 16 Nov 2023 05:07:36 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.10.3 mail0.glenbarber.us BDE4CD544 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (1.0) Subject: Re: [HEADS-UP] Quick update to 14.0-RELEASE schedule From: Glen Barber In-Reply-To: Date: Thu, 16 Nov 2023 00:07:36 -0500 Cc: John Baldwin , FreeBSD Release Engineering Team , freebsd-current@freebsd.org, freebsd-stable@freebsd.org Message-Id: <5F24BC3A-88D9-4295-AA94-435C11A1F445@freebsd.org> References: To: The Doctor X-Mailer: iPhone Mail (20G81) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36236, ipnet:2607:fc50::/36, country:US] X-Rspamd-Queue-Id: 4SW7MF3Fhwz3HYY And if there is a reason to reissue a release, the update will reflect such.= So to answer your latter question, yes. Unless it needs to be replaced. Glen Sent from my phone. Please excuse my brevity and/or typos. > On Nov 15, 2023, at 11:38 PM, The Doctor wrote:= >=20 > =EF=BB=BFOn Thu, Nov 16, 2023 at 12:30:30AM +0000, Glen Barber wrote: >>> On Wed, Nov 15, 2023 at 08:39:39AM -0800, John Baldwin wrote: >>> On 11/14/23 8:52 PM, Glen Barber wrote: >>>> On Tue, Nov 14, 2023 at 08:10:23PM -0700, The Doctor wrote: >>>>> On Wed, Nov 15, 2023 at 02:27:01AM +0000, Glen Barber wrote: >>>>>> On Tue, Nov 14, 2023 at 05:15:48PM -0700, The Doctor wrote: >>>>>>> On Tue, Nov 14, 2023 at 08:36:54PM +0000, Glen Barber wrote: >>>>>>>> We are still waiting for a few (non-critical) things to complete be= fore >>>>>>>> the announcement of 14.0-RELEASE will be ready. >>>>>>>>=20 >>>>>>>> It should only be another day or so before these things complete. >>>>>>>>=20 >>>>>>>> Thank you for your understanding. >>>>>>>>=20 >>>>>>>=20 >>>>>>> I always just installed my copy. >>>>>>>=20 >>>>>>=20 >>>>>> Ok. I do not know what exactly is your point, but releases are never= >>>>>> official until there is a PGP-signed email sent. The email is intend= ed >>>>>> for the general public of consumers of official releases, not "yeah, >>>>>> but"s. >>>>>>=20 >>>>>=20 >>>>> Howver if you do a freebsd-update upgrade, you can upgrade. >>>>>=20 >>>>> Is that suppose to happen? >>>>>=20 >>>>=20 >>>> That does not say that the freebsd-update bits will not change *until* >>>> the official release announcement has been sent. >>>>=20 >>>> In my past 15 years involved in the Project, I think we have been very >>>> clear on that. >>>>=20 >>>> A RELEASE IS NOT FINAL UNTIL THE PGP-SIGNED ANNOUNCEMENT IS SENT. >>>>=20 >>>> I mean, c'mon, dude. >>>>=20 >>>> We really, seriously, for all intents and purposes, cannot be any more >>>> clear than that. >>>>=20 >>>> So, yes, *IF* an update necessitates a new freebsd-update build, what >>>> you are running is *NOT* official. >>>>=20 >>>> For at least 15 years, we have all said the same entire thing. >>>=20 >>> Yes, but, if at this point we had to rebuild, it would have to be 14.0.1= >>> or something (which we have done a few times in the past). It would be >>> too confusing otherwise once the bits are built and published (where >>> published means "uploaded to our CDN"). It is the 14.0 release bits, >>> the only question is if for some reason we had a dire emergency that >>> meant we had to pull it at the last minute and publish different bits >>> (under a different release name). >>>=20 >>> Realistically, once the bits are available, we can't prevent people from= >>> using them, it's just at their own risk to do so until the project says >>> "yes, we believe these are good". Granted, they are under the same risk= >>> if they are still running the last RC. The best way to minimize that >>> risk going forward is to add more automation of testing/CI to go along >>> with the process of building release bits so that the build artifacts >>> from the release build run through CI and are only published if the CI >>> is green as that would give us greater confidence of "we believe these >>> are good" before they are uploaded for publishing. >>>=20 >>=20 >> You are correct on all points. If there were a need to re-roll 14.0, it >> would indeed necessitate a release/14.0.1 tag. Note, release/14.0.0 has >> not yet been tagged, and I find it extremely unlikely that it will be >> necessary to rebuild from a release/14.0.1 tag. >>=20 >> I also agree we cannot prevent people from downloading the images, >> installers, whatever before the announcement. That is the lovely race >> condition with which we have to live at the moment. >>=20 >> My email was intended to be informative. Period. >>=20 >> There were no suggestions that 14.0-RELEASE was not yet final. And to >> be fair, I had to personally deal with the fallout of a release "too >> soon", notably 11.0-RELEASE, where I thought a critical issue had been >> addressed, but I was wrong. >>=20 >> My only point, in being overly open to the public on the delay, is that >> we (the Release Engineering Team) are not yet ready to rubber-stamp this >> as complete, as there are some outstanding items that are pending that >> have not yet completed. >>=20 >> The alternative would be to say nothing at all. >>=20 >> Either way, it is a productivity, communication drain. It is >> a lose-lose situation no matter how one looks at it given the above >> context. We either get chastised for being "too open" into insights >> delaying an official announcement, or for being "not open enough" when >> there is silence from RE when a release does not meet its scheduled >> announcement date. >>=20 >> Glen >>=20 >>=20 >=20 > What ahappened was that I inititated the freebsd-upgrade RELEASE upgrade c= ommand. >=20 > question should that RELEASe been released? >=20 > --=20 > Member - Liberal International This is doctor@nk.ca Ici doctor@nk.ca > Yahweh, King & country!Never Satan President Republic!Beware AntiChrist ri= sing! > Look at Psalms 14 and 53 on Atheism ; unsubscribe from Google Groups to be= seen > Lest we forget 11 Nov 2023 Beware https://mindspring.com >=20 From nobody Thu Nov 16 08:44:33 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SWD9d2lkzz51ML6 for ; Thu, 16 Nov 2023 08:44:37 +0000 (UTC) (envelope-from fluffy@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SWD9d0nz1z3ch8 for ; Thu, 16 Nov 2023 08:44:37 +0000 (UTC) (envelope-from fluffy@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700124277; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type; bh=4RxDY8z3K4U/dMMaCUXebF7cWZWrtD6bWONGXzE/ANA=; b=FuqZSHqU5i22j2Gy0Qp7q1vBUOdrvo67jXMbFg1hLhNMbYJVxQvf26aIf4IAWS5nt0Gk84 XeNRYHEvOZa8QzIqW+vV2bi0XeexgE8qLGPYAfjRMJIRv1mnlyWl552tr1Af0caGqMb+zq xdavAXR8Djq0CFLOvpaJo19L8+1eoSn6KRdLxNOIF5xO20J/zoaAD92RBz4sDqp16v+8uh lpjZ7RtjyPgZojlrIn+47jRa13SYC9TwweTCy1iloCg6BoDNY8Gtac4vD4XRoIE5edSmfO afQl6B7/nOfWgBSAE4bnKwSyPxKwQb1XwpnraAborU3nzovABfRyc+/DQZhglg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700124277; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type; bh=4RxDY8z3K4U/dMMaCUXebF7cWZWrtD6bWONGXzE/ANA=; b=DaCBhYUnheL4i4W80MErZrOOmCbfPem1+3hQbiJgL0UDkCzXIdb5FYqhf/9wa+JOzGWZgF cFCTrPkw87Nz7G7piV0GWP8kTeHVjuPnbnE/vzJiBB3or+MrQFSCE+YcarwI288XwY+IyU irK8Bt4TQ/1JPmuXiUuC2z12g/QCzxo12ZDD3HFLoRgSDAf3pTg10S3/FJNL2EJc9jzcm+ JaWbO1MuapU9jthk5VZT4yuxdsGKVaeX8/j/sRxUOcG+Oe4BBlq+X6LBOeEbja41iTDV+O xiTBqwz6ggzNJrc51PCAdvmhQYZYC8IA6ZPzGtyVJridta5y/s5HwUTxJGIPdQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1700124277; a=rsa-sha256; cv=none; b=UtNeGn/hAtdLEejwGgOQUH/xcWWfj1rTVGgo2kc0WuveKllcP8mGJ385pdG864i3ZL1XWQ XuBvA5E1RI49UmJjap43Ey/s2a+MC/9VhX8YWrsfYCSwDeamI2hyLf8RzcP7BzTH0fR56A RIsq4VcFNRfjHjZb4jjDa3p+0oQo0mk4uJBOuG0TsPjquDqC/uFCb9n4CtlKxhuU0uyDmp MjHmH9utt8I/3d/JEgzwWjhV9xDcoKOfJ4jakDPMBs0z1lKtsFylvVhzTap8T17NNNvJUI 2km0hEIGyhOjKF3aiw7vsKCIvt825j6FfpJfZxAV+BkqLc66E58+ZB6rDPT2Cg== Received: from [10.216.0.102] (unknown [188.243.165.67]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: fluffy) by smtp.freebsd.org (Postfix) with ESMTPSA id 4SWD9c540GzsB4 for ; Thu, 16 Nov 2023 08:44:36 +0000 (UTC) (envelope-from fluffy@FreeBSD.org) Message-ID: Date: Thu, 16 Nov 2023 11:44:33 +0300 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: ru, en-GB, en-US To: FreeBSD CURRENT From: Dima Panov Subject: something got wrong with 32bit ld between 1500002 and 1500003 Organization: FreeBSD.org Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="------------bB3ks7vcM9wxUcwhLQw5vUbt" This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --------------bB3ks7vcM9wxUcwhLQw5vUbt Content-Type: multipart/mixed; boundary="------------V00vjW5TZrl5Wc3UYK0QnLcv"; protected-headers="v1" From: Dima Panov To: FreeBSD CURRENT Message-ID: Subject: something got wrong with 32bit ld between 1500002 and 1500003 --------------V00vjW5TZrl5Wc3UYK0QnLcv Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: base64 TW9pbi1tb2luIQ0KDQpBZnRlciB1cGdyYWRlIHRvIHJlY2VudCAtY3VycmVudCAoMTUwMDAw MykgSSd2ZSBnb3QgYSBsZCBlcnJvciBmb3IgMzJiaXQgYW4gYWFyY2g2NCBtYWNoaW5lOigN Cg0KRUxGIGJpbmFyeSB0eXBlICI5IiBub3Qga25vd24uDQpldmFsOiAvbGliZXhlYy9sZC1l bGYzMi5zby4xOiBFeGVjIGZvcm1hdCBlcnJvcg0KDQpTbmFwc2hvdCBmcm9tIDIwMjMtMTAt MTMgd2FzIGZpbmUsIG5vIGVycm9ycw0KDQoxNS4wLUNVUlJFTlQgIzAgbWFpbi02ZjM4ZDJl N2IwOiBUaHUgTm92IDE2IDAzOjI5OjAzIE1TSyAyMDIzIGlzIHN0aWxsIGJyb2tlbg0KDQoN Ci0tIA0KU2luY2VyZWx5LA0KRGltYSAoZmx1ZmZ5QEZyZWVCU0Qub3JnLCBodHRwczovL3Qu bWUvRmx1ZmZ5QlNEKQ0KKGRlc2t0b3AsIGtkZSwgeDExLCBvZmZpY2UsIHBvcnRzLXNlY3Rl YW0pQEZyZWVCU0QgdGVhbQ0K --------------V00vjW5TZrl5Wc3UYK0QnLcv-- --------------bB3ks7vcM9wxUcwhLQw5vUbt Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature.asc" -----BEGIN PGP SIGNATURE----- wsF5BAABCAAjFiEELTAsy5mEEwxvh7r8+4ugndU5jykFAmVV1nEFAwAAAAAACgkQ+4ugndU5jymj Jg//WszUUr+u22J8yKjwVPTzyx1zaMCVq4r40UQIN97y4IcLBT2bvN7g/s39iyVsOVlBZJpK74q1 gU3BL2uNQNEamxBfYInTR2HaHWu41HP+AE70dMw44OEGAB5YLR6LAa9VFazNjrMMFEjr3ymL0XYy apxPWWSkc2qc0CjMUnfDA0bdrTTpwtzbmQuhZr24MihP+lUwA4OF6pZ/CAGppt7suPRw/UQBTmIa y7SfBUU5rERdasf36yISopdy5PqUM6na4Y1W0YULLb0fNdZ2IXzIPSNmLCUXslxlBb6TpMLMiU8W 7p/X8jUnNlJOFbqLP9vxIOYiZEXkEcjKur57+odhybMSSQZDmzcNeORfLTd9L2VNYqq+j0j99mzK kyav2kwqbw5wN+RrwRo5y0FxoaZucIPIIUcUTA1rNPyIUlGl2Yn6LAmkU7sEu+ts266QBX35V2au 7rVKldnIeL5cyQGL1YC+qKwjF6j14+abp4LbEGczk+AgLfhb/xxZAOfxKiqbHclgfv2AFKa6ZvuJ zBMswm+/BBAljGnVJy3TkeW49XtISipJCyv7GdcJRfVJzkNs0qMI/L2nltjJKc5lZqtUQtlXuznn qERQ/ZCFPE0hly3aND70mkHcRIK5L4X3uI87eiQsTsY3dat68aOf6oIvXYxbQ9tHho/UL9c7RKlH xqo= =ySXo -----END PGP SIGNATURE----- --------------bB3ks7vcM9wxUcwhLQw5vUbt-- From nobody Thu Nov 16 09:13:05 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SWDpY5d35z51Ngf; Thu, 16 Nov 2023 09:13:09 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from drew.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.franken.de", Issuer "Sectigo RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SWDpY0l45z3gQw; Thu, 16 Nov 2023 09:13:09 +0000 (UTC) (envelope-from tuexen@freebsd.org) Authentication-Results: mx1.freebsd.org; dkim=none; spf=softfail (mx1.freebsd.org: 2001:638:a02:a001:20e:cff:fe4a:feaa is neither permitted nor denied by domain of tuexen@freebsd.org) smtp.mailfrom=tuexen@freebsd.org; dmarc=none Received: from smtpclient.apple (unknown [IPv6:2003:a:e03:d412:855f:9ce5:7233:4d51]) (Authenticated sender: micmac) by mail-n.franken.de (Postfix) with ESMTPSA id 5D5D3721E2817; Thu, 16 Nov 2023 10:13:06 +0100 (CET) From: tuexen@freebsd.org Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.200.91.1.1\)) Subject: Request for Testing: TCP RACK Message-Id: <42C327BD-6CE4-43AA-A1AE-3BEC08D623DB@freebsd.org> Date: Thu, 16 Nov 2023 10:13:05 +0100 To: current@freebsd.org, net@freebsd.org X-Mailer: Apple Mail (2.3774.200.91.1.1) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, T_SCC_BODY_TEXT_LINE autolearn=disabled version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail-n.franken.de X-Spamd-Result: default: False [-2.44 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.94)[-0.935]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; ONCE_RECEIVED(0.10)[]; TO_DN_NONE(0.00)[]; R_DKIM_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_ALL(0.00)[]; ASN(0.00)[asn:680, ipnet:2001:638::/32, country:DE]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MLMMJ_DEST(0.00)[current@freebsd.org,net@freebsd.org]; RCPT_COUNT_TWO(0.00)[2]; R_SPF_SOFTFAIL(0.00)[~all:c]; DMARC_NA(0.00)[freebsd.org]; FROM_NO_DN(0.00)[]; FREEFALL_USER(0.00)[tuexen]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[] X-Rspamd-Queue-Id: 4SWDpY0l45z3gQw X-Spamd-Bar: -- Dear all, recently the main branch was changed to build the TCP RACK stack which is a loadable kernel module, by default: = https://cgit.FreeBSD.org/src/commit/?id=3D3a338c534154164504005beb00a3c6fe= b03756cc As discussed on the bi-weekly transport call, it would be great if = people could test the RACK stack for their workload. Please report any problems = to the net@ mailing list or open an issue in the bug tracker and drop me a note = via email. This includes regressions in CPU usage, regressions in performance or = any other unexpected change you observe. You can load the kernel module using kldload tcp_rack You can make the RACK stack the default stack using sysctl net.inet.tcp.functions_default=3Drack Based on the feedback we get, the default stack might be switched to the RACK stack. Please let me know if you have any questions. Best regards Michael From nobody Thu Nov 16 12:06:03 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SWJfG5vrnz50ZBl; Thu, 16 Nov 2023 12:06:14 +0000 (UTC) (envelope-from herbert@gojira.at) Received: from mail.bsd4all.net (mail.bsd4all.net [94.130.200.20]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "mail.bsd4all.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SWJfG3X2tz4Vk6; Thu, 16 Nov 2023 12:06:14 +0000 (UTC) (envelope-from herbert@gojira.at) Authentication-Results: mx1.freebsd.org; none Date: Thu, 16 Nov 2023 13:06:03 +0100 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=gojira.at; s=mail202005; t=1700136367; bh=zNCwYmCwyO2OGk2JRakaq6NUNJF3LB7WdKPvfQzVU1w=; h=Date:Message-ID:From:To:Cc:Subject:MIME-Version:Content-Type; b=LJHxjvdD2T3jt4Aw5Iqi3Yu8E58NGLvgg5c35lze8ET2EY3EoaCxNCsuKPHNmDiL/ EmC9SnD7IPXbXbDWP2CApeT51qs+QeR9Q30kwBx2oN6wb0RlJwY1s7P36DDeLT7Ar1 7WEF5Nx8+8en0suzrjeEZDba6oRQdakd56EOKlaTGqCE3umdi0EvKtw0SoVF9WtoK9 GUKjFH5CkUZ7ao4mhdSh6FTkuklQ0HKDl4C1/6WSkOfAKwOZHvxtkgOSBnCwHSYozX 8Le78lntnGCk1Kdfzw6qoB0LNH8OBGZH47Vfk0CrRiOXYJk1baEHOcqx2kBRUpcTo1 Qj4dyANbz1CLQ== Message-ID: <87pm09ykb8.wl-herbert@gojira.at> From: "Herbert J. Skuhra" To: tuexen@freebsd.org Cc: current@freebsd.org, net@freebsd.org Subject: Re: Request for Testing: TCP RACK In-Reply-To: <42C327BD-6CE4-43AA-A1AE-3BEC08D623DB@freebsd.org> References: <42C327BD-6CE4-43AA-A1AE-3BEC08D623DB@freebsd.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) Emacs/29.1 Mule/6.0 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:24940, ipnet:94.130.0.0/16, country:DE] X-Rspamd-Queue-Id: 4SWJfG3X2tz4Vk6 Hi, On Thu, 16 Nov 2023 10:13:05 +0100, tuexen@freebsd.org wrote: > > Dear all, > > recently the main branch was changed to build the TCP RACK stack > which is a loadable kernel module, by default: > https://cgit.FreeBSD.org/src/commit/?id=3a338c534154164504005beb00a3c6feb03756cc > > As discussed on the bi-weekly transport call, it would be great if people > could test the RACK stack for their workload. Please report any problems to the > net@ mailing list or open an issue in the bug tracker and drop me a note via email. > This includes regressions in CPU usage, regressions in performance or any other > unexpected change you observe. > > You can load the kernel module using > kldload tcp_rack > > You can make the RACK stack the default stack using > sysctl net.inet.tcp.functions_default=rack > > Based on the feedback we get, the default stack might be switched to the > RACK stack. > > Please let me know if you have any questions. I am running main-n266450-a592812327de with a GENERIC-NODEBUG kernel. # kldload tcp_rack kldload: an error occurred while loading module tcp_rack. Please check dmesg(8) for more details. In dmesg: KLD tcp_rack.ko: depends on tcphpts - not available or version mismatch linker_load_file: /boot/kernel/tcp_rack.ko - unsupported file type So you have to build a kernel with "options TCPHPTS" first? -- Herbert From nobody Thu Nov 16 11:19:35 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SWJn749Ngz50Zbh; Thu, 16 Nov 2023 12:12:11 +0000 (UTC) (envelope-from doctor@doctor.nl2k.ab.ca) Received: from doctor.nl2k.ab.ca (doctor.nl2k.ab.ca [204.209.81.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SWJn72VrYz4Y5M; Thu, 16 Nov 2023 12:12:11 +0000 (UTC) (envelope-from doctor@doctor.nl2k.ab.ca) Authentication-Results: mx1.freebsd.org; none Received: from doctor by doctor.nl2k.ab.ca with local (Exim 4.97 (FreeBSD)) (envelope-from ) id 1r3aPH-000000007FZ-1E0a; Thu, 16 Nov 2023 04:19:35 -0700 Date: Thu, 16 Nov 2023 04:19:35 -0700 From: The Doctor To: Glen Barber Cc: John Baldwin , FreeBSD Release Engineering Team , freebsd-current@freebsd.org, freebsd-stable@freebsd.org Subject: Re: [HEADS-UP] Quick update to 14.0-RELEASE schedule Message-ID: References: <5F24BC3A-88D9-4295-AA94-435C11A1F445@freebsd.org> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5F24BC3A-88D9-4295-AA94-435C11A1F445@freebsd.org> X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:6171, ipnet:204.209.81.0/24, country:CA] X-Rspamd-Queue-Id: 4SWJn72VrYz4Y5M On Thu, Nov 16, 2023 at 12:07:36AM -0500, Glen Barber wrote: > And if there is a reason to reissue a release, the update will reflect such. > > So to answer your latter question, yes. Unless it needs to be replaced. > > Glen > Sent from my phone. > Please excuse my brevity and/or typos. > > > On Nov 15, 2023, at 11:38 PM, The Doctor wrote: > > > > ???On Thu, Nov 16, 2023 at 12:30:30AM +0000, Glen Barber wrote: > >>> On Wed, Nov 15, 2023 at 08:39:39AM -0800, John Baldwin wrote: > >>> On 11/14/23 8:52 PM, Glen Barber wrote: > >>>> On Tue, Nov 14, 2023 at 08:10:23PM -0700, The Doctor wrote: > >>>>> On Wed, Nov 15, 2023 at 02:27:01AM +0000, Glen Barber wrote: > >>>>>> On Tue, Nov 14, 2023 at 05:15:48PM -0700, The Doctor wrote: > >>>>>>> On Tue, Nov 14, 2023 at 08:36:54PM +0000, Glen Barber wrote: > >>>>>>>> We are still waiting for a few (non-critical) things to complete before > >>>>>>>> the announcement of 14.0-RELEASE will be ready. > >>>>>>>> > >>>>>>>> It should only be another day or so before these things complete. > >>>>>>>> > >>>>>>>> Thank you for your understanding. > >>>>>>>> > >>>>>>> > >>>>>>> I always just installed my copy. > >>>>>>> > >>>>>> > >>>>>> Ok. I do not know what exactly is your point, but releases are never > >>>>>> official until there is a PGP-signed email sent. The email is intended > >>>>>> for the general public of consumers of official releases, not "yeah, > >>>>>> but"s. > >>>>>> > >>>>> > >>>>> Howver if you do a freebsd-update upgrade, you can upgrade. > >>>>> > >>>>> Is that suppose to happen? > >>>>> > >>>> > >>>> That does not say that the freebsd-update bits will not change *until* > >>>> the official release announcement has been sent. > >>>> > >>>> In my past 15 years involved in the Project, I think we have been very > >>>> clear on that. > >>>> > >>>> A RELEASE IS NOT FINAL UNTIL THE PGP-SIGNED ANNOUNCEMENT IS SENT. > >>>> > >>>> I mean, c'mon, dude. > >>>> > >>>> We really, seriously, for all intents and purposes, cannot be any more > >>>> clear than that. > >>>> > >>>> So, yes, *IF* an update necessitates a new freebsd-update build, what > >>>> you are running is *NOT* official. > >>>> > >>>> For at least 15 years, we have all said the same entire thing. > >>> > >>> Yes, but, if at this point we had to rebuild, it would have to be 14.0.1 > >>> or something (which we have done a few times in the past). It would be > >>> too confusing otherwise once the bits are built and published (where > >>> published means "uploaded to our CDN"). It is the 14.0 release bits, > >>> the only question is if for some reason we had a dire emergency that > >>> meant we had to pull it at the last minute and publish different bits > >>> (under a different release name). > >>> > >>> Realistically, once the bits are available, we can't prevent people from > >>> using them, it's just at their own risk to do so until the project says > >>> "yes, we believe these are good". Granted, they are under the same risk > >>> if they are still running the last RC. The best way to minimize that > >>> risk going forward is to add more automation of testing/CI to go along > >>> with the process of building release bits so that the build artifacts > >>> from the release build run through CI and are only published if the CI > >>> is green as that would give us greater confidence of "we believe these > >>> are good" before they are uploaded for publishing. > >>> > >> > >> You are correct on all points. If there were a need to re-roll 14.0, it > >> would indeed necessitate a release/14.0.1 tag. Note, release/14.0.0 has > >> not yet been tagged, and I find it extremely unlikely that it will be > >> necessary to rebuild from a release/14.0.1 tag. > >> > >> I also agree we cannot prevent people from downloading the images, > >> installers, whatever before the announcement. That is the lovely race > >> condition with which we have to live at the moment. > >> > >> My email was intended to be informative. Period. > >> > >> There were no suggestions that 14.0-RELEASE was not yet final. And to > >> be fair, I had to personally deal with the fallout of a release "too > >> soon", notably 11.0-RELEASE, where I thought a critical issue had been > >> addressed, but I was wrong. > >> > >> My only point, in being overly open to the public on the delay, is that > >> we (the Release Engineering Team) are not yet ready to rubber-stamp this > >> as complete, as there are some outstanding items that are pending that > >> have not yet completed. > >> > >> The alternative would be to say nothing at all. > >> > >> Either way, it is a productivity, communication drain. It is > >> a lose-lose situation no matter how one looks at it given the above > >> context. We either get chastised for being "too open" into insights > >> delaying an official announcement, or for being "not open enough" when > >> there is silence from RE when a release does not meet its scheduled > >> announcement date. > >> > >> Glen > >> > >> > > > > What ahappened was that I inititated the freebsd-upgrade RELEASE upgrade command. > > > > question should that RELEASe been released? > > Here is what happened. Since Openssl 1.X is no longer supported I switch to 14.0-RC4 and found for the most part that RC4 is 98 % there. Given the web site saying expect a release notice around 10 Nov, I deciced to try to see if 14.0 RELEASE was ready. Since then I found 1, port openvpn is not stable 2, wireguard FrreBSD client to FreeBSD serve is not working 3) Dovecot is acting glitchy. -- Member - Liberal International This is doctor@nk.ca Ici doctor@nk.ca Yahweh, King & country!Never Satan President Republic!Beware AntiChrist rising! Look at Psalms 14 and 53 on Atheism ; unsubscribe from Google Groups to be seen Lest we forget 11 Nov 2023 Beware https://mindspring.com From nobody Thu Nov 16 13:05:10 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SWKyP4KpTz50dfG; Thu, 16 Nov 2023 13:05:17 +0000 (UTC) (envelope-from herbert@gojira.at) Received: from mail.bsd4all.net (mail.bsd4all.net [94.130.200.20]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "mail.bsd4all.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SWKyN2n2Wz4gjn; Thu, 16 Nov 2023 13:05:16 +0000 (UTC) (envelope-from herbert@gojira.at) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gojira.at header.s=mail202005 header.b=d8tFpNAc; spf=pass (mx1.freebsd.org: domain of herbert@gojira.at designates 94.130.200.20 as permitted sender) smtp.mailfrom=herbert@gojira.at; dmarc=none Date: Thu, 16 Nov 2023 14:05:10 +0100 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=gojira.at; s=mail202005; t=1700139914; bh=ehatBrGLPd5gYOcfokV0Uy/wnUplq7NmM0dipGzbmcw=; h=Date:Message-ID:From:To:Cc:Subject:MIME-Version:Content-Type; b=d8tFpNAcsxBPfTcpmW5icsUYNutI7ll2B3vDg0U2oQU9LqaCMMuSJWDFBfvj4mjdO r5YoMCOnetuZUHBlK0Pwz9e6q9fJCVotoE7mx9YIpKMUk0HvLRykxyCvUo80zWPYoY nXMK0OAP2ZmaZQJVXUsNqL4HMksac97P1J+1sPdmUuEVEUUE2zV7kJtpvHwVCY9GOx N2/penyu9cftQuc+/swa3trDET3h038dLZ3dxoX00NDT3v+/+j6gJZPnh0X0LGms/p e9rMjJf8rPbCQNARYK/HUn/30oa2e143YuEe6Ft3BnkgkxkpULd1Q+jG4c7XYOLC9c GN+Ib7SIvINFw== Message-ID: <87o7ftyhkp.wl-herbert@gojira.at> From: "Herbert J. Skuhra" To: tuexen@freebsd.org Cc: current@freebsd.org, net@freebsd.org Subject: Re: Request for Testing: TCP RACK In-Reply-To: <87pm09ykb8.wl-herbert@gojira.at> References: <42C327BD-6CE4-43AA-A1AE-3BEC08D623DB@freebsd.org> <87pm09ykb8.wl-herbert@gojira.at> User-Agent: Wanderlust/2.15.9 (Almost Unreal) Emacs/29.1 Mule/6.0 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-Spamd-Result: default: False [-2.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_SHORT(-1.00)[-0.996]; R_SPF_ALLOW(-0.20)[+ip4:94.130.200.20]; R_DKIM_ALLOW(-0.20)[gojira.at:s=mail202005]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; MLMMJ_DEST(0.00)[current@freebsd.org,net@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_ZERO(0.00)[0]; DKIM_TRACE(0.00)[gojira.at:+]; DMARC_NA(0.00)[gojira.at]; TO_DN_NONE(0.00)[]; FROM_HAS_DN(0.00)[]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_ALL(0.00)[]; ASN(0.00)[asn:24940, ipnet:94.130.0.0/16, country:DE] X-Rspamd-Queue-Id: 4SWKyN2n2Wz4gjn X-Spamd-Bar: -- On Thu, 16 Nov 2023 13:06:03 +0100, "Herbert J. Skuhra" wrote: > > Hi, > > On Thu, 16 Nov 2023 10:13:05 +0100, tuexen@freebsd.org wrote: > > > > Dear all, > > > > recently the main branch was changed to build the TCP RACK stack > > which is a loadable kernel module, by default: > > https://cgit.FreeBSD.org/src/commit/?id=3a338c534154164504005beb00a3c6feb03756cc > > > > As discussed on the bi-weekly transport call, it would be great if people > > could test the RACK stack for their workload. Please report any problems to the > > net@ mailing list or open an issue in the bug tracker and drop me a note via email. > > This includes regressions in CPU usage, regressions in performance or any other > > unexpected change you observe. > > > > You can load the kernel module using > > kldload tcp_rack > > > > You can make the RACK stack the default stack using > > sysctl net.inet.tcp.functions_default=rack > > > > Based on the feedback we get, the default stack might be switched to the > > RACK stack. > > > > Please let me know if you have any questions. > > I am running main-n266450-a592812327de with a GENERIC-NODEBUG kernel. > > # kldload tcp_rack > kldload: an error occurred while loading module tcp_rack. Please check > dmesg(8) for more details. > > In dmesg: > KLD tcp_rack.ko: depends on tcphpts - not available or version mismatch > linker_load_file: /boot/kernel/tcp_rack.ko - unsupported file type > > So you have to build a kernel with "options TCPHPTS" first? OK, I am now running GENERIC-NODEBUG + "options TCPHPTS". After setting "sysctl net.inet.tcp.functions_default=rack" git no longer works: # sysctl net.inet.tcp.functions_default=rack $ git pull client_loop: send disconnect: Broken pipe fatal: the remote end hung up upon initial contact # sysctl net.inet.tcp.functions_default=freebsd $ git pull Already up to date. -- Herbert From nobody Thu Nov 16 16:50:27 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SWQyT3kn0z50rkG for ; Thu, 16 Nov 2023 16:50:41 +0000 (UTC) (envelope-from zarychtam@plan-b.pwste.edu.pl) Received: from plan-b.pwste.edu.pl (plan-b.pwste.edu.pl [IPv6:2001:678:618::40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "plan-b.pwste.edu.pl", Issuer "GEANT OV RSA CA 4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SWQyS27n6z3HKp for ; Thu, 16 Nov 2023 16:50:39 +0000 (UTC) (envelope-from zarychtam@plan-b.pwste.edu.pl) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=plan-b.pwste.edu.pl header.s=plan-b-mailer header.b=j9PRLc1X; spf=none (mx1.freebsd.org: domain of zarychtam@plan-b.pwste.edu.pl has no SPF policy when checking 2001:678:618::40) smtp.mailfrom=zarychtam@plan-b.pwste.edu.pl; dmarc=pass (policy=none) header.from=plan-b.pwste.edu.pl Received: from [IPV6:2a02:22e0:cf00:1ff:63f0:2fc0:7fca:98ab] (mzar@[IPv6:2a02:22e0:cf00:1ff:63f0:2fc0:7fca:98ab]) (authenticated bits=0) by plan-b.pwste.edu.pl (8.17.2/8.17.2) with ESMTPSA id 3AGGoSGL016331 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO) for ; Thu, 16 Nov 2023 17:50:28 +0100 (CET) (envelope-from zarychtam@plan-b.pwste.edu.pl) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=plan-b.pwste.edu.pl; s=plan-b-mailer; t=1700153429; bh=BRuNgNAO4unMMp4jZs1WB4rzqt9QT4JOzgSDK/wNoes=; h=Date:Subject:To:References:From:In-Reply-To; b=j9PRLc1X4uyMeVnzbhh8v90KsvsE5Q7GBprixrwFSqrVjSMwntvZHind/hPtomVYQ c2SDt37iQzQudbT+a7bB6lhUWtjj4MBJNlB+E4GugE6pWyR6wnURUjLLuXnyM3/PBu dkdl69NSe6kRXLDmBcONzsdtSqqaYJ+G8WGib7sAsrox/wwYCIQKUnFKJp4jU1UnCv Bt8p86MEMxZWZFA9MbHg+2PXbNmADVJaR3wkYxH1UuFWierQagv1J6tb/vMi8mh49d R21PrVyZmE99O/SKw7WXCB9CyfWeYmbnBfPSQgNe/TEgy+2yFIHlYPl1ZmA+xKM+q9 dRinA9SjNfqRQ== Content-Type: multipart/alternative; boundary="------------SPA0eaGCKAFQsaHKAxSFlbVA" Message-ID: Date: Thu, 16 Nov 2023 17:50:27 +0100 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Request for Testing: TCP RACK Content-Language: en-US To: freebsd-current@freebsd.org References: <42C327BD-6CE4-43AA-A1AE-3BEC08D623DB@freebsd.org> From: Marek Zarychta Autocrypt: addr=zarychtam@plan-b.pwste.edu.pl; keydata= xsBNBFfi3cMBCADLecMTFXad4uDXqv3eRuB4qJJ8G9tzzFezeRnnwxOsPdytW5ES2z1ibSrR IsiImx6+PTqrAmXpTInxAi7yiZGdSiONRI4CCxKY9d1YFiNYT/2WyNXCekm9x29YeIU7x0JB Llbz0f/9HC+styBIu2H+PY/X98Clzm110CS+n/b9l1AtiGxTiVFj7/uavYAKxH6LNWnbkuc5 v8EVNc7NkEcl5h7Z9X5NEtzDxTOiBIFQ/kOT7LAtkYUPo1lqLeOM2DtWSXTXQgXl0zJI4iP1 OAu4qQYm2nXwq4b2AH9peknelvnt1mpfgDCGSKnhc26q6ibTfMwydp+tvUtQIQYpA6b9ABEB AAHNN01hcmVrIFphcnljaHRhIChQbGFuLWIpIDx6YXJ5Y2h0YW1AcGxhbi1iLnB3c3RlLmVk dS5wbD7CwHcEEwEIACEFAlfi4LkCGwMFCwkIBwIGFQgJCgsCBBYCAwECHgECF4AACgkQHZW8 vIFppoJXdgf8D9X3VRFSNaR9lthSx/+uqas17J3FJKBo1xMQsC2a+44vzNvYJSuPGLLJ+LW2 HPVazjP/BWZJbxOYpliY4zxNRU0YCp0BLIVLibc//yax+mE42FND/+NiIZhqJscl6MLPrSwo sIwXec4XYkldkyqW/xBbBYXoIkBqdKB9j5j42Npy1IV/RizOSdmvTWY27ir8e/yGMR1RLr4F 8P5K3OWTdlGy2H2F/3J8bIPBLG6FpaIyLQw4dHSx8V02PYqDxK1cNo2kAOnU8PnZL/AGuMOH iv3MN1VYL8ehcmpBBsrZGebQJxrjY2/5IaTSgp9xHYT70kshuU6Qb97vk1mOjNZxgc7ATQRX 4t3DAQgA10h6RCXuBLMHxq5B8X/ZIlj9sgLoeyfRdDZEc9rT2KUeUJVHDsbvOFf4/7F1ovWY hJbA6GK/LUZeHHTjnbZcH1uDYQeHly4UOLxeEvhGoz4JhS2C7JzN/uRnwbdOAUbJr8rUj/IY a7gk906rktsc/Ldrxrxh7O6WO0JCh2XO/p4pDfEwwB37g4xHprSab28ECYJ9JMbtA8Sy4M55 g3+GQ28FvSlGnx48OoGXU2BZdc1vZKSQmNOlikB+9/hDX8zdYWVfDaX1TLQ8Ib4+xTUmapza mV/bxIsaZRBw+jFjLQHhTbIMfPEU+4mxFDvTdbKPruKPqVf1ydgMnPZWngowdwARAQABwsBf BBgBCAAJBQJX4t3DAhsMAAoJEB2VvLyBaaaC6qkIAJs9sDPqrqW0bYoRfzY6XjDWQ59p9tJi v8aogxacQNCfAu+WkJ8PNVUtC1dlVcG5NnZ80gXzd1rc8ueIvXlvdanUt/jZd8jbb3gaDbK3 wh1yMCGBl/1fOJTyEGYv1CRojv97KK89KP5+r8x1P1iHcSrunlDNqGxTMydNCwBH23QcOM+m u4spKnJ/s0VRBkw3xoKBZfZza6fTQ4gTpAipjyk7ldOGBV+PvkKATdhK2yLwuWXhKbg/GRlD 1r5P0gxzSqfV4My+KJuc2EDcrqp1y0wOpE1m9iZqCcd0fup5f7HDsYlLWshr7NQl28f6+fQb sylq/j672BHXsdeqf/Ip9V4= In-Reply-To: <42C327BD-6CE4-43AA-A1AE-3BEC08D623DB@freebsd.org> X-Spamd-Result: default: False [-3.69 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[plan-b.pwste.edu.pl,none]; R_DKIM_ALLOW(-0.20)[plan-b.pwste.edu.pl:s=plan-b-mailer]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; ONCE_RECEIVED(0.10)[]; XM_UA_NO_VERSION(0.01)[]; RCVD_TLS_ALL(0.00)[]; ASN(0.00)[asn:206006, ipnet:2001:678:618::/48, country:PL]; MIME_TRACE(0.00)[0:+,1:+,2:~]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_COUNT_ONE(0.00)[1]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; DKIM_TRACE(0.00)[plan-b.pwste.edu.pl:+]; R_SPF_NA(0.00)[no SPF record]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Rspamd-Queue-Id: 4SWQyS27n6z3HKp X-Spamd-Bar: --- This is a multi-part message in MIME format. --------------SPA0eaGCKAFQsaHKAxSFlbVA Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit W dniu 16.11.2023 o 10:13, tuexen@freebsd.org pisze: > Dear all, > > recently the main branch was changed to build the TCP RACK stack > which is a loadable kernel module, by default: > https://cgit.FreeBSD.org/src/commit/?id=3a338c534154164504005beb00a3c6feb03756cc That's really good news and long-awaited change. Thank you. > As discussed on the bi-weekly transport call, it would be great if people > could test the RACK stack for their workload. Please report any problems to the > net@ mailing list or open an issue in the bug tracker and drop me a note via email. > This includes regressions in CPU usage, regressions in performance or any other > unexpected change you observe. > > You can load the kernel module using > kldload tcp_rack > > You can make the RACK stack the default stack using > sysctl net.inet.tcp.functions_default=rack > > Based on the feedback we get, the default stack might be switched to the > RACK stack. Yes, please do it, at least for CURRENT. Last problem I have spotted with RACK was fixed in June: it was missing support TCP-MD5: https://cgit.freebsd.org/src/commit/?id=02b885b09d1e90574162a1442b9ede06cef2b13a We switched to RACK since upgrading to early stable/13 and genuinely appreciate this gift from Netflix. The performance improvement is invaluable, both in a lossy LAN and on a long-haul overseas TCP connection. > Please let me know if you have any questions. Are any MFCs planned, especially to stable/14 ? Now, when stable/14 is almost in sync with main aka CURRENT it's an optimal time for such a MFC. When the change hits stable/14, it would be possible to test it extensively and have it fully functional in 14.1-RELEASE. Best regards -- Marek Zarychta --------------SPA0eaGCKAFQsaHKAxSFlbVA Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit
W dniu 16.11.2023 o 10:13, tuexen@freebsd.org pisze:
Dear all,

recently the main branch was changed to build the TCP RACK stack
which is a loadable kernel module, by default:
https://cgit.FreeBSD.org/src/commit/?id=3a338c534154164504005beb00a3c6feb03756cc
That's really good news and long-awaited change. Thank you. 
As discussed on the bi-weekly transport call, it would be great if people
could test the RACK stack for their workload. Please report any problems to the
net@ mailing list or open an issue in the bug tracker and drop me a note via email.
This includes regressions in CPU usage, regressions in performance or any other
unexpected change you observe.

You can load the kernel module using
kldload tcp_rack

You can make the RACK stack the default stack using
sysctl net.inet.tcp.functions_default=rack

Based on the feedback we get, the default stack might be switched to the
RACK stack.

Yes, please do it, at least for CURRENT. Last problem I have spotted with RACK was fixed in June: it was missing support TCP-MD5:

https://cgit.freebsd.org/src/commit/?id=02b885b09d1e90574162a1442b9ede06cef2b13a

We switched to RACK since upgrading to early stable/13 and genuinely appreciate this gift from Netflix. The performance improvement is invaluable, both in a lossy LAN and on a long-haul overseas TCP connection. 

Please let me know if you have any questions.

Are any MFCs planned, especially to stable/14 ? Now, when stable/14 is almost in sync with main aka CURRENT it's an optimal time for such a MFC. When the change hits stable/14, it would be possible to test it extensively and have it fully functional in 14.1-RELEASE.

Best regards

-- 
Marek Zarychta
--------------SPA0eaGCKAFQsaHKAxSFlbVA-- From nobody Thu Nov 16 18:07:29 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SWSgL4G2rz50wTN; Thu, 16 Nov 2023 18:07:42 +0000 (UTC) (envelope-from olivier@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SWSgL3c5Jz3Q3B; Thu, 16 Nov 2023 18:07:42 +0000 (UTC) (envelope-from olivier@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700158062; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=ugnB03lttSSHGgDOWD1U8zuoZY2sAn4cbixlthR5NnA=; b=w3joNOU5YAilY9o55rdAXkYA4JHizIY++b9HzYJmsD4hammvLd0ni8vYtqvO+1NQaVAvVV Kbd7doQ3T5o5ZfxZD560ZMgtyXca8szvjX3kmlnP4NCg6U7KFoSbkFf3lwQ7BVIeX6MfIm +OasvAuxteVDugntHW3r7zX5S6Q8V+dTyKC5OBqFoQWjL4jp3rWuBN+AZkc+XWcsUqb7yC V1TVxIHAkqw3adeU5otCcu0me2/T+BVLWRBJulJCoKGRUaLkNyyL52nUqzCeqCKBVFKV2R wbQjAt8m44RFUsvZvMCZ+cqFYQLqlX7g4bqHmYEtBWk4wYy6/5Mt1fOiTk26JQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700158062; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=ugnB03lttSSHGgDOWD1U8zuoZY2sAn4cbixlthR5NnA=; b=UI4kAF7u+FdCoPCkJ9xUL2QcxXPPMVIZGmrWsr5fVJxSRc8hQ/2D9VjZanki2rgVlH9ZDa zqe8PQxdEgTXSQpAtdWYyTq6CUe2TLO4U+lgLUV6sjCpexgeeWL0s9/nINXLgH7Ooa5AY9 zpNGzW9I0Cw7STEU4THd7Cy9xQpGGnoS8PY+Y+bL4dnasYDvOJZiGwRpijkC+GDM3Iwnib GLLskgkOV7xfoHPwh3I8REfDj0/8mpq2ycH0K13qjrpriXBqzdswDFNd2ppM9fHOCLWtYN SG8ZMT8h4C6EfQB7rSJAkGgIgrRUeiXMAdUCKzykNTT08A8s89SLpPgzGs+H3Q== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1700158062; a=rsa-sha256; cv=none; b=cqtcMd3XWsAjZBKndT7vj6XAIORhgVF/OHPXj13J0wjIYXl2V266mIJDifj2JO+Ajk2ZQz t1Hc8Vdh5lg9nDI0pldtn/kLQ8LfYz58sIiDVRWiiJB/F+mrU29/c82HbWrDpnyc1YJT7K KpRTQaTUA3dOwWWFazSb63GtuegzumJqm4sDukcK/ZnzxWfgNj9HRPk6ZOALeq6+gQx1bJ NateccCvEs634ugYbAUjfm84EP05OGWwNQfsFK8+Bf07ATLpevqZhyQh9ZIFJO00OuI9Kp 2xm2uwUge95df/+rU1iuKAf/6CEJx8N9v3Cg71Y5PIk2v1a3emVdsa7X+SL/6Q== Received: from mail-qv1-f41.google.com (mail-qv1-f41.google.com [209.85.219.41]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) (Authenticated sender: olivier/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4SWSgL2f90z13rS; Thu, 16 Nov 2023 18:07:42 +0000 (UTC) (envelope-from olivier@freebsd.org) Received: by mail-qv1-f41.google.com with SMTP id 6a1803df08f44-66d0760cd20so9506256d6.0; Thu, 16 Nov 2023 10:07:42 -0800 (PST) X-Gm-Message-State: AOJu0YyA+HqavVw+ZQJD/kKJMdfzemOqDgxGm9tERvx7x50bPILv3YNH aNbiDgLbEWqdEa3I99T66p/DDZiRKWzIinRP52U= X-Google-Smtp-Source: AGHT+IFJ6U1iZTv43XkLkJBYZcnvaHPEaHhuSCWIL8pt7tNtCF4CgCE5QmyokBq1sxJBSs98bKinWo79dqq4FMK0vmU= X-Received: by 2002:ad4:5444:0:b0:63d:580:9c68 with SMTP id h4-20020ad45444000000b0063d05809c68mr3045497qvt.32.1700158061351; Thu, 16 Nov 2023 10:07:41 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: <42C327BD-6CE4-43AA-A1AE-3BEC08D623DB@freebsd.org> <87pm09ykb8.wl-herbert@gojira.at> <87o7ftyhkp.wl-herbert@gojira.at> In-Reply-To: <87o7ftyhkp.wl-herbert@gojira.at> From: =?UTF-8?Q?Olivier_Cochard=2DLabb=C3=A9?= Date: Thu, 16 Nov 2023 19:07:29 +0100 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Request for Testing: TCP RACK To: "Herbert J. Skuhra" Cc: tuexen@freebsd.org, current@freebsd.org, net@freebsd.org Content-Type: multipart/alternative; boundary="0000000000004f1a7d060a48e517" --0000000000004f1a7d060a48e517 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Nov 16, 2023 at 5:10=E2=80=AFPM Herbert J. Skuhra wrote: > > OK, I am now running GENERIC-NODEBUG + "options TCPHPTS". > > After setting "sysctl net.inet.tcp.functions_default=3Drack" git no > longer works: > > Are you using a fresh 15 head or a specific network setup ? Because I'm not able to reproduce your problem on my system: $ uname -a FreeBSD bigone 15.0-CURRENT FreeBSD 15.0-CURRENT #0 main-n266452-070d9e3540e6: Thu Nov 16 17:53:15 CET 2023 root@bigone:/usr/obj/usr/src/amd64.amd64/sys/TCPHPTS amd64 $ cat /usr/src/sys/amd64/conf/TCPHPTS include GENERIC-NODEBUG ident TCPHPTS options TCPHPTS $ sysctl net.inet.tcp.functions_default net.inet.tcp.functions_default: rack $ git clone -q git@github.com:freebsd/freebsd-src.git && echo working working $ --0000000000004f1a7d060a48e517 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Thu, Nov 16, 2023 at 5:10=E2=80=AFPM Herbert J. = Skuhra <herbert@gojira.at> w= rote:

OK, I am now running GENERIC-NODEBUG + "options TCPHPTS".

After setting "sysctl net.inet.tcp.functions_default=3Drack" git = no
longer works:


Are you using a = fresh 15 head or a specific network setup ?

Because I'm not able to reproduce your problem on my system:
<= div class=3D"gmail_default" style=3D"font-family:"courier new",mo= nospace">
$ uname -a
FreeBSD bigone 15.0-CURRENT Free= BSD 15.0-CURRENT #0 main-n266452-070d9e3540e6: Thu Nov 16 17:53:15 CET 2023= =C2=A0 =C2=A0 root@bigone:/usr/obj/usr/src/amd64.amd64/sys/TCPHPTS amd64$ cat /usr/src/sys/amd64/conf/TCPHPTS
include GENERIC-NODEBUG
ident= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 TCPHPTS
= options =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 TCPHPTS
=
$ sysctl net.inet.tcp.functions_default
= net.inet.tcp.functions_default: rack
$ git clone -q git@github.com:freeb= sd/freebsd-src.git && echo working
working
$
--0000000000004f1a7d060a48e517-- From nobody Thu Nov 16 19:06:07 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SWTz32cSrz50yvh; Thu, 16 Nov 2023 19:06:23 +0000 (UTC) (envelope-from herbert@gojira.at) Received: from mail.bsd4all.net (mail.bsd4all.net [IPv6:2a01:4f8:13b:240c::25]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "mail.bsd4all.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SWTz30Zywz3XGG; Thu, 16 Nov 2023 19:06:22 +0000 (UTC) (envelope-from herbert@gojira.at) Authentication-Results: mx1.freebsd.org; none Date: Thu, 16 Nov 2023 20:06:07 +0100 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=gojira.at; s=mail202005; t=1700161574; bh=OrdUVnoDm+OrOFROYDtbyD6jkvhSSnuWHxqFAo1wSUo=; h=Date:Message-ID:From:To:Cc:Subject:MIME-Version:Content-Type; b=cW8FfLx/ug6TXiFdEC6fNNWjKd074BFbTlE3AOmOgRcvj/+tVrcVHnTZxbtu4CkPQ VfdDhsBhBHCs9edX2Wx79u+gLlcFQBQrDojwy/EW5nA0sWwXX7znvqqxzT9KfO40Tn peg84lKe/NBWDV7MWKAUiP9PBlHbNmxkNY/Gw3id4TaqNcKvwwSwcX8gAur1nE7Gwf dokophyTc2bX9ZODbAGgB+xJxxCrJsVu36wldUDmn02i/fD7RLCq5ZM1eFnUsgQKvi eKl2AQ/sak4Gld0pTmzeh55/lOndD2+bl8XHMZVZJgTc861mhPMWnrC53OS4CWX3AR Beq7NAueZm94Q== Message-ID: <87msvdy0v4.wl-herbert@gojira.at> From: "Herbert J. Skuhra" To: Olivier =?ISO-8859-1?Q?Cochard-Labb=E9?= Cc: tuexen@freebsd.org, current@freebsd.org, net@freebsd.org Subject: Re: Request for Testing: TCP RACK In-Reply-To: References: <42C327BD-6CE4-43AA-A1AE-3BEC08D623DB@freebsd.org> <87pm09ykb8.wl-herbert@gojira.at> <87o7ftyhkp.wl-herbert@gojira.at> User-Agent: Wanderlust/2.15.9 (Almost Unreal) Emacs/29.1 Mule/6.0 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/32, country:DE] X-Rspamd-Queue-Id: 4SWTz30Zywz3XGG On Thu, 16 Nov 2023 19:07:29 +0100, Olivier Cochard-Labb=C3=A9 wrote: >=20 > On Thu, Nov 16, 2023 at 5:10=E2=80=AFPM Herbert J. Skuhra wrote: >=20 > > > > OK, I am now running GENERIC-NODEBUG + "options TCPHPTS". > > > > After setting "sysctl net.inet.tcp.functions_default=3Drack" git no > > longer works: > > > > > Are you using a fresh 15 head or a specific network setup ? >=20 > Because I'm not able to reproduce your problem on my system: >=20 > $ uname -a > FreeBSD bigone 15.0-CURRENT FreeBSD 15.0-CURRENT #0 > main-n266452-070d9e3540e6: Thu Nov 16 17:53:15 CET 2023 > root@bigone:/usr/obj/usr/src/amd64.amd64/sys/TCPHPTS > amd64 > $ cat /usr/src/sys/amd64/conf/TCPHPTS > include GENERIC-NODEBUG > ident TCPHPTS > options TCPHPTS > $ sysctl net.inet.tcp.functions_default > net.inet.tcp.functions_default: rack > $ git clone -q git@github.com:freebsd/freebsd-src.git && echo working > working > $ OK, (g)it works if I disable pf. Do you use pf? -- Herbert From nobody Thu Nov 16 21:22:52 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SWY3J70Mdz516mV; Thu, 16 Nov 2023 21:25:16 +0000 (UTC) (envelope-from jamie@catflap.org) Received: from donotpassgo.dyslexicfish.net (donotpassgo.dyslexicfish.net [IPv6:2001:19f0:7400:8808:123::1]) by mx1.freebsd.org (Postfix) with ESMTP id 4SWY3H6rJ7z4N5h; Thu, 16 Nov 2023 21:25:15 +0000 (UTC) (envelope-from jamie@catflap.org) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of jamie@catflap.org designates 2001:19f0:7400:8808:123::1 as permitted sender) smtp.mailfrom=jamie@catflap.org; dmarc=pass (policy=none) header.from=catflap.org X-Catflap-Envelope-From: X-Catflap-Envelope-To: freebsd-current@FreeBSD.org Received: from donotpassgo.dyslexicfish.net (donotpassgo.dyslexicfish.net [209.250.224.51]) by donotpassgo.dyslexicfish.net (8.14.5/8.14.5) with ESMTP id 3AGLMrso086012; Thu, 16 Nov 2023 21:22:53 GMT (envelope-from jamie@donotpassgo.dyslexicfish.net) Received: (from jamie@localhost) by donotpassgo.dyslexicfish.net (8.14.5/8.14.5/Submit) id 3AGLMqsN086011; Thu, 16 Nov 2023 21:22:52 GMT (envelope-from jamie) From: Jamie Landeg-Jones Message-Id: <202311162122.3AGLMqsN086011@donotpassgo.dyslexicfish.net> Date: Thu, 16 Nov 2023 21:22:52 +0000 Organization: Dyslexic Fish To: gjb@FreeBSD.org, doctor@doctor.nl2k.ab.ca Cc: re@FreeBSD.org, freebsd-stable@FreeBSD.org, freebsd-current@FreeBSD.org Subject: Re: [HEADS-UP] Quick update to 14.0-RELEASE schedule References: <20231114203654.GB52320@FreeBSD.org> <20231115022701.GM1307@FreeBSD.org> In-Reply-To: <20231115022701.GM1307@FreeBSD.org> User-Agent: Heirloom mailx 12.4 7/29/08 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (donotpassgo.dyslexicfish.net [209.250.224.51]); Thu, 16 Nov 2023 21:22:53 +0000 (GMT) X-Spamd-Result: default: False [-3.69 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.986]; DMARC_POLICY_ALLOW(-0.50)[catflap.org,none]; R_SPF_ALLOW(-0.20)[+mx:dyslexicfish.net]; MIME_GOOD(-0.10)[text/plain]; RCVD_NO_TLS_LAST(0.10)[]; MLMMJ_DEST(0.00)[freebsd-current@FreeBSD.org,freebsd-stable@FreeBSD.org]; R_DKIM_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; HAS_ORG_HEADER(0.00)[]; ASN(0.00)[asn:20473, ipnet:2001:19f0:7400::/38, country:US]; FREEFALL_USER(0.00)[jamie]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_FIVE(0.00)[5]; TO_DN_NONE(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4SWY3H6rJ7z4N5h X-Spamd-Bar: --- > Ok. I do not know what exactly is your point, but releases are never > official until there is a PGP-signed email sent. The email is intended > for the general public of consumers of official releases, not "yeah, > but"s. Foe a recent new build, I just went to the ftp site to grab the latest rc, only to see 14.0-release there, so I grabbed and installed that, many hours before your original mail went out. No biggy, I generally track stable, but does this mean we can't rely on the download sites without checking out the lists first? It's not like I was plaing sneaky by doing "guess the URL" or anything like that. Jamie From nobody Thu Nov 16 22:21:38 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SWZJY1QwCz518kf for ; Thu, 16 Nov 2023 22:21:49 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mx1.sbone.de (mx1.sbone.de [IPv6:2a01:4f8:13b:39f::9f:25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE Root Certificate Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SWZJX1MWGz4bJx for ; Thu, 16 Nov 2023 22:21:48 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of bzeeb-lists@lists.zabbadoz.net designates 2a01:4f8:13b:39f::9f:25 as permitted sender) smtp.mailfrom=bzeeb-lists@lists.zabbadoz.net; dmarc=none Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:4902:0:7404:2:1025]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id 39E818D4A217 for ; Thu, 16 Nov 2023 22:21:40 +0000 (UTC) Received: from content-filter.t4-02.sbone.de (content-filter.t4-02.sbone.de [IPv6:fde9:577b:c1a9:4902:0:7404:2:2742]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id DD2312D029D6 for ; Thu, 16 Nov 2023 22:21:39 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:4902:0:7404:2:1025]) by content-filter.t4-02.sbone.de (content-filter.t4-02.sbone.de [IPv6:fde9:577b:c1a9:4902:0:7404:2:2742]) (amavisd-new, port 10024) with ESMTP id NhA8eWRB2ZzU for ; Thu, 16 Nov 2023 22:21:39 +0000 (UTC) Received: from strong-iwl0.sbone.de (strong-iwl0.sbone.de [IPv6:fde9:577b:c1a9:4902:b66b:fcff:fef3:e3d2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id DE7842D029D2 for ; Thu, 16 Nov 2023 22:21:38 +0000 (UTC) Date: Thu, 16 Nov 2023 22:21:38 +0000 (UTC) From: "Bjoern A. Zeeb" To: current@freebsd.org Subject: db> reset -> panic: acquiring blockable sleep lock with spinlock or critical section held ... Message-ID: <0q526858-7s33-27qo-6q80-8o1q10415q2s@yvfgf.mnoonqbm.arg> X-OpenPGP-Key-Id: 0x14003F198FEFA3E77207EE8D2B58B8F83CCF1842 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset=US-ASCII X-Spamd-Result: default: False [-3.27 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-0.99)[-0.990]; NEURAL_HAM_SHORT(-0.98)[-0.984]; R_SPF_ALLOW(-0.20)[+ip6:2a01:4f8:13b:39f::9f:25]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[zabbadoz.net]; MLMMJ_DEST(0.00)[current@freebsd.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_THREE(0.00)[4]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[current@freebsd.org]; ARC_NA(0.00)[]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/32, country:DE]; FROM_HAS_DN(0.00)[]; TO_DN_NONE(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[] X-Rspamd-Queue-Id: 4SWZJX1MWGz4bJx X-Spamd-Bar: --- Hi, I seem to remember changes related to that a while ago but my cache is miss for the actual change. Are we suppoed to handle this case? It would be nice if "reset" would reset again the first time ... KDB: enter: Break to debugger [ thread pid 11 tid 100005 ] Stopped at kdb_alt_break_internal+0x180: str xzr, [x19, #896] db> reset panic: acquiring blockable sleep lock with spinlock or critical section held (sleep mutex) eventhandler @ /usr/src/sys/kern/subr_eventhandler.c:269 cpuid = 2 time = 307 KDB: stack backtrace: db_trace_self() at db_trace_self db_trace_self_wrapper() at db_trace_self_wrapper+0x38 vpanic() at vpanic+0x1a0 panic() at panic+0x48 witness_checkorder() at witness_checkorder+0xb4c __mtx_lock_flags() at __mtx_lock_flags+0xac eventhandler_find_list() at eventhandler_find_list+0x44 kern_reboot() at kern_reboot+0x284 db_reset() at db_reset+0xd8 db_command() at db_command+0x2e4 db_command_loop() at db_command_loop+0x58 db_trap() at db_trap+0x100 kdb_trap() at kdb_trap+0x364 handle_el1h_sync() at handle_el1h_sync+0x18 --- exception, esr 0xf2000000 kdb_alt_break_internal() at kdb_alt_break_internal+0x180 kdb_alt_break() at kdb_alt_break+0x10 uart_intr_rxready() at uart_intr_rxready+0x8c uart_intr() at uart_intr+0x120 intr_event_handle() at intr_event_handle+0xf4 intr_isrc_dispatch() at intr_isrc_dispatch+0x78 arm_gic_v3_intr() at arm_gic_v3_intr+0x120 intr_irq_handler() at intr_irq_handler+0x88 handle_el1h_irq() at handle_el1h_irq+0x14 --- interrupt cpu_idle() at cpu_idle+0x78 sched_idletd() at sched_idletd+0x4a0 fork_exit() at fork_exit+0x78 fork_trampoline() at fork_trampoline+0x18 KDB: enter: panic [ thread pid 11 tid 100005 ] Stopped at kdb_enter+0x48: str xzr, [x19, #896] db> reset Uptime: 5m7s INFO: PSCI Power Domain Map: -- Bjoern A. Zeeb r15:7 From nobody Thu Nov 16 22:52:46 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SWb0L0Nfqz51Bcb; Thu, 16 Nov 2023 22:52:50 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SWb0K6xv3z4dHC; Thu, 16 Nov 2023 22:52:49 +0000 (UTC) (envelope-from gjb@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700175170; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=0bxwCocVwBUbhsOH1iGGqsBO/61zQFJIHWKZoYNiJDc=; b=TtLUV4VzKktLSSPw8n+EohxuaAwmbs/T5U2PZcbo1X/L39wIvpy/vnOoKKB7IB//5VPlLs 2EApXphITJSQQfAwdSCFkgPtWeUYzTeEn4e41JII7hOiGvJaHrMnwCIIQRSv81QkwLj2/G djoYlfIZpjGei/TeFEicof2EUr9yz2HZ/kS03Qm1QK94AadZHKrveqeJRLsS3TW6MStoXd KNBibaXk5aYwSr8+VQtk1ED3Jtc9hHZ4mSaNobOPSJmVzyBxGIaqGN7MTYy1bsMjHuBT7l QYqRrzy62e+YOWQ8Vi52OeqtkF8N/TSM0CgG813t85gMPvgk2iU3WUMwRvfs+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700175170; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=0bxwCocVwBUbhsOH1iGGqsBO/61zQFJIHWKZoYNiJDc=; b=vlk5l/qQb8DPok1IkbpE9paKGgI8+Cl2FaYZ3gjPCwGpkzUB4ZZVIqS8wV+KP87WMkhqnD l/56+AZQhiM4bxYqqnAGwEYKvcq/7ow7lxc8mGPlx/lT+/NksV8cwcTht3540XfbItLiXs Ntka2Zm/LkzdfU5DsUovVBuHZC00YfZ8YHI+XqAtTvUG2nu8j/WbkgshohFQo0iFQwXBCq 3YGVdxsgNeMHa8x5JWRHMU8xN44zqk02TSmsvNQNMV4dR6RWX1mDQ5C6s1k63naQezZdqe LdSBZ/5u/hVlKTNDyKYWLAoniGnARlfpInCtMFFoH910UJ2cmYcNCoWfUalzaA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1700175170; a=rsa-sha256; cv=none; b=xpV2sV1GwoHPegnX0pLR9llLM0er8n8OKSTtZ+q8733FKPcDA+VGyidwKlIyD3h9AbpaYB JvAj9u4RUOd6lqV7sTmaKR0kvVmBvLLFzY6LgM64Dwy2y/UIplmrWkSTcexlsacHU4wGv2 4noZGpjxqo4jB/COq7jgDEzixf8pKoGYSgRvZLeTpOq8UQJ+S5UrYpribKwai0dHvXAD20 5kH/OgaZTQlGxeyF/8rmUb0+xKR/AV5C08TYB6l+2XJDCzMvp9sE6O4AonGY7MTMn90/vB A1Q2iU2AHSjU5NiasIwqLTv+GM/NEQOTOrPPsmwx3F8sJLkDMLvQReaYVLq9KA== Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by freefall.freebsd.org (Postfix) with ESMTPS id 8F437193C1; Thu, 16 Nov 2023 22:52:49 +0000 (UTC) (envelope-from gjb@freebsd.org) Date: Thu, 16 Nov 2023 22:52:46 +0000 From: Glen Barber To: Jamie Landeg-Jones Cc: doctor@doctor.nl2k.ab.ca, re@freebsd.org, freebsd-stable@freebsd.org, freebsd-current@freebsd.org Subject: Re: [HEADS-UP] Quick update to 14.0-RELEASE schedule Message-ID: <20231116225246.GQ1307@FreeBSD.org> References: <20231114203654.GB52320@FreeBSD.org> <20231115022701.GM1307@FreeBSD.org> <202311162122.3AGLMqsN086011@donotpassgo.dyslexicfish.net> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="UI7gKp310iCrxvPC" Content-Disposition: inline In-Reply-To: <202311162122.3AGLMqsN086011@donotpassgo.dyslexicfish.net> --UI7gKp310iCrxvPC Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Nov 16, 2023 at 09:22:52PM +0000, Jamie Landeg-Jones wrote: > > Ok. I do not know what exactly is your point, but releases are never > > official until there is a PGP-signed email sent. The email is intended > > for the general public of consumers of official releases, not "yeah, > > but"s. >=20 > Foe a recent new build, I just went to the ftp site to grab the latest rc, > only to see 14.0-release there, so I grabbed and installed that, many > hours before your original mail went out. >=20 > No biggy, I generally track stable, but does this mean we can't rely > on the download sites without checking out the lists first? It's not > like I was plaing sneaky by doing "guess the URL" or anything like that. >=20 No. It merely suggests the release is not officially official yet. Glen --UI7gKp310iCrxvPC Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEjRJAPC5sqwhs9k2jAxRYpUeP4pMFAmVWnTkACgkQAxRYpUeP 4pOitg//SJu3XzG9RZdYoyxo8CVjXD17xvqWRiiR+y2QHCjShn+CS7kPv9Ic6ZcB Zzl05Igh2AfkWEL85hh17pHlK4QzTcdNQ8DJkyE50MKQh2pfZ7jsDdpbpOc1jzNk Gj7BI3hJwjkwIHs8CoB8BGVQp40gqmAK+xJOVnps+hwTTN9VstdkFp5vMLRFt7pp j3Ah8u4mYc1cuxvdCXOHbTnBHMPvpPqZdS5GwdiKiC3hc13pfkeo3RNXv8YumQfs Ju6opx/Tcocmt1X3msUfThxvTQOqk357E+YdxZLAprYnRludyEUucpC84p4/CrXp R+bN4YR0XhRP67BpeneUa4jwYUHAyhOF5QmNvLqU4Oc6uIGMctgSrQHdHK5w+tij DcE2RFO5SkpRZC60156G7iSuKRBBcFrpm/mwyzC7rLHdfZGpknoT+lIcpHmQjisT 4UQfwljWWUd0s1BTqI1NzIZ1y8PsgIxDjn/w3f3Sl+dFjBOdWiy7N3W6hd+vU9fs UmkphQc6ibTepB/wu+1d0sl0Rt2oA7RnIoAjgeZIxpG3UWvVuPKr8b1fvqRN3BfU SvZmOVFBc4R1OVG8dcEUhMxN6ode6Ecg0uYUdvAEJvco8GGIJs3/a2EK4OTIdgke xUs5ToJxTEV73upqA5EcJ+G55z6MUqs/LPEhJAZTYLcaZFxYGAE= =vQrP -----END PGP SIGNATURE----- --UI7gKp310iCrxvPC-- From nobody Thu Nov 16 23:08:58 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SWbM55Fnqz51C5c; Thu, 16 Nov 2023 23:09:05 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from drew.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.franken.de", Issuer "Sectigo RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SWbM44DD2z3CyP; Thu, 16 Nov 2023 23:09:04 +0000 (UTC) (envelope-from tuexen@freebsd.org) Authentication-Results: mx1.freebsd.org; none Received: from smtpclient.apple (unknown [IPv6:2a02:8109:1140:c3d:69c4:8554:a779:b3aa]) (Authenticated sender: micmac) by mail-n.franken.de (Postfix) with ESMTPSA id 77270721E280D; Fri, 17 Nov 2023 00:08:59 +0100 (CET) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.200.91.1.1\)) Subject: Re: Request for Testing: TCP RACK From: tuexen@freebsd.org In-Reply-To: <87pm09ykb8.wl-herbert@gojira.at> Date: Fri, 17 Nov 2023 00:08:58 +0100 Cc: current@freebsd.org, net@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <42C327BD-6CE4-43AA-A1AE-3BEC08D623DB@freebsd.org> <87pm09ykb8.wl-herbert@gojira.at> To: "Herbert J. Skuhra" X-Mailer: Apple Mail (2.3774.200.91.1.1) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, T_SCC_BODY_TEXT_LINE autolearn=disabled version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail-n.franken.de X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:680, ipnet:2001:638::/32, country:DE] X-Rspamd-Queue-Id: 4SWbM44DD2z3CyP > On Nov 16, 2023, at 13:06, Herbert J. Skuhra = wrote: >=20 > Hi, >=20 > On Thu, 16 Nov 2023 10:13:05 +0100, tuexen@freebsd.org wrote: >>=20 >> Dear all, >>=20 >> recently the main branch was changed to build the TCP RACK stack >> which is a loadable kernel module, by default: >> = https://cgit.FreeBSD.org/src/commit/?id=3D3a338c534154164504005beb00a3c6fe= b03756cc >>=20 >> As discussed on the bi-weekly transport call, it would be great if = people >> could test the RACK stack for their workload. Please report any = problems to the >> net@ mailing list or open an issue in the bug tracker and drop me a = note via email. >> This includes regressions in CPU usage, regressions in performance or = any other >> unexpected change you observe. >>=20 >> You can load the kernel module using >> kldload tcp_rack >>=20 >> You can make the RACK stack the default stack using >> sysctl net.inet.tcp.functions_default=3Drack >>=20 >> Based on the feedback we get, the default stack might be switched to = the >> RACK stack. >>=20 >> Please let me know if you have any questions. >=20 > I am running main-n266450-a592812327de with a GENERIC-NODEBUG kernel. >=20 > # kldload tcp_rack > kldload: an error occurred while loading module tcp_rack. Please check > dmesg(8) for more details. >=20 > In dmesg: > KLD tcp_rack.ko: depends on tcphpts - not available or version = mismatch > linker_load_file: /boot/kernel/tcp_rack.ko - unsupported file type >=20 > So you have to build a kernel with "options TCPHPTS" first? Hi Herbert, yes this is correct. For whatever reason I was assuming the TCPHPTS is = already enabled in all configs, but this is not correct. Will put up an review to do this tomorrow. Thanks for reporting. Best regards Michael >=20 > -- > Herbert From nobody Thu Nov 16 23:12:35 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SWbRD1wmXz51CgW; Thu, 16 Nov 2023 23:12:40 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from drew.franken.de (mail-n.franken.de [193.175.24.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.franken.de", Issuer "Sectigo RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SWbRC3Pdhz3Fkm; Thu, 16 Nov 2023 23:12:39 +0000 (UTC) (envelope-from tuexen@freebsd.org) Authentication-Results: mx1.freebsd.org; none Received: from smtpclient.apple (unknown [IPv6:2a02:8109:1140:c3d:69c4:8554:a779:b3aa]) (Authenticated sender: micmac) by mail-n.franken.de (Postfix) with ESMTPSA id 18874721E280D; Fri, 17 Nov 2023 00:12:36 +0100 (CET) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.200.91.1.1\)) Subject: Re: Request for Testing: TCP RACK From: tuexen@freebsd.org In-Reply-To: <87o7ftyhkp.wl-herbert@gojira.at> Date: Fri, 17 Nov 2023 00:12:35 +0100 Cc: current@freebsd.org, net@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <42C327BD-6CE4-43AA-A1AE-3BEC08D623DB@freebsd.org> <87pm09ykb8.wl-herbert@gojira.at> <87o7ftyhkp.wl-herbert@gojira.at> To: "Herbert J. Skuhra" X-Mailer: Apple Mail (2.3774.200.91.1.1) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, T_SCC_BODY_TEXT_LINE autolearn=disabled version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail-n.franken.de X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:680, ipnet:193.174.0.0/15, country:DE] X-Rspamd-Queue-Id: 4SWbRC3Pdhz3Fkm > On Nov 16, 2023, at 14:05, Herbert J. Skuhra = wrote: >=20 > On Thu, 16 Nov 2023 13:06:03 +0100, "Herbert J. Skuhra" wrote: >>=20 >> Hi, >>=20 >> On Thu, 16 Nov 2023 10:13:05 +0100, tuexen@freebsd.org wrote: >>>=20 >>> Dear all, >>>=20 >>> recently the main branch was changed to build the TCP RACK stack >>> which is a loadable kernel module, by default: >>> = https://cgit.FreeBSD.org/src/commit/?id=3D3a338c534154164504005beb00a3c6fe= b03756cc >>>=20 >>> As discussed on the bi-weekly transport call, it would be great if = people >>> could test the RACK stack for their workload. Please report any = problems to the >>> net@ mailing list or open an issue in the bug tracker and drop me a = note via email. >>> This includes regressions in CPU usage, regressions in performance = or any other >>> unexpected change you observe. >>>=20 >>> You can load the kernel module using >>> kldload tcp_rack >>>=20 >>> You can make the RACK stack the default stack using >>> sysctl net.inet.tcp.functions_default=3Drack >>>=20 >>> Based on the feedback we get, the default stack might be switched to = the >>> RACK stack. >>>=20 >>> Please let me know if you have any questions. >>=20 >> I am running main-n266450-a592812327de with a GENERIC-NODEBUG kernel. >>=20 >> # kldload tcp_rack >> kldload: an error occurred while loading module tcp_rack. Please = check >> dmesg(8) for more details. >>=20 >> In dmesg: >> KLD tcp_rack.ko: depends on tcphpts - not available or version = mismatch >> linker_load_file: /boot/kernel/tcp_rack.ko - unsupported file type >>=20 >> So you have to build a kernel with "options TCPHPTS" first? >=20 > OK, I am now running GENERIC-NODEBUG + "options TCPHPTS". >=20 > After setting "sysctl net.inet.tcp.functions_default=3Drack" git no > longer works: >=20 > # sysctl net.inet.tcp.functions_default=3Drack > $ git pull > client_loop: send disconnect: Broken pipe > fatal: the remote end hung up upon initial contact > # sysctl net.inet.tcp.functions_default=3Dfreebsd > $ git pull > Already up to date. Interesting. It works on my development system: tuexen@head:~/freebsd-src % sysctl net.inet.tcp.functions_default net.inet.tcp.functions_default: rack tuexen@head:~/freebsd-src % git pull Already up to date. tuexen@head:~/freebsd-src % Could you provide a .pcapng file covering the `git pull` command? Best regards Michael >=20 > -- > Herbert From nobody Thu Nov 16 23:15:13 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SWbVF1fkrz51DF3; Thu, 16 Nov 2023 23:15:17 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from drew.franken.de (mail-n.franken.de [193.175.24.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.franken.de", Issuer "Sectigo RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SWbVF0Npyz3HV0; Thu, 16 Nov 2023 23:15:17 +0000 (UTC) (envelope-from tuexen@freebsd.org) Authentication-Results: mx1.freebsd.org; none Received: from smtpclient.apple (unknown [IPv6:2a02:8109:1140:c3d:69c4:8554:a779:b3aa]) (Authenticated sender: micmac) by mail-n.franken.de (Postfix) with ESMTPSA id 4A1EB721E280D; Fri, 17 Nov 2023 00:15:14 +0100 (CET) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.200.91.1.1\)) Subject: Re: Request for Testing: TCP RACK From: tuexen@freebsd.org In-Reply-To: <87msvdy0v4.wl-herbert@gojira.at> Date: Fri, 17 Nov 2023 00:15:13 +0100 Cc: =?utf-8?Q?Olivier_Cochard-Labb=C3=A9?= , current@freebsd.org, net@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <867DC224-F0B1-4C9C-8EB7-C9F7BE1C029A@freebsd.org> References: <42C327BD-6CE4-43AA-A1AE-3BEC08D623DB@freebsd.org> <87pm09ykb8.wl-herbert@gojira.at> <87o7ftyhkp.wl-herbert@gojira.at> <87msvdy0v4.wl-herbert@gojira.at> To: "Herbert J. Skuhra" X-Mailer: Apple Mail (2.3774.200.91.1.1) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, T_SCC_BODY_TEXT_LINE autolearn=disabled version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail-n.franken.de X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:680, ipnet:193.174.0.0/15, country:DE] X-Rspamd-Queue-Id: 4SWbVF0Npyz3HV0 > On Nov 16, 2023, at 20:06, Herbert J. Skuhra = wrote: >=20 > On Thu, 16 Nov 2023 19:07:29 +0100, Olivier Cochard-Labb=C3=A9 wrote: >>=20 >> On Thu, Nov 16, 2023 at 5:10=E2=80=AFPM Herbert J. Skuhra = wrote: >>=20 >>>=20 >>> OK, I am now running GENERIC-NODEBUG + "options TCPHPTS". >>>=20 >>> After setting "sysctl net.inet.tcp.functions_default=3Drack" git no >>> longer works: >>>=20 >>>=20 >> Are you using a fresh 15 head or a specific network setup ? >>=20 >> Because I'm not able to reproduce your problem on my system: >>=20 >> $ uname -a >> FreeBSD bigone 15.0-CURRENT FreeBSD 15.0-CURRENT #0 >> main-n266452-070d9e3540e6: Thu Nov 16 17:53:15 CET 2023 >> root@bigone:/usr/obj/usr/src/amd64.amd64/sys/TCPHPTS >> amd64 >> $ cat /usr/src/sys/amd64/conf/TCPHPTS >> include GENERIC-NODEBUG >> ident TCPHPTS >> options TCPHPTS >> $ sysctl net.inet.tcp.functions_default >> net.inet.tcp.functions_default: rack >> $ git clone -q git@github.com:freebsd/freebsd-src.git && echo working >> working >> $ >=20 > OK, (g)it works if I disable pf. Do you use pf? Can you share your pf config such that I can reproduce the problem = locally? Best regards Michael >=20 > -- > Herbert From nobody Fri Nov 17 10:00:07 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SWspX2kxpz50mNt; Fri, 17 Nov 2023 10:00:20 +0000 (UTC) (envelope-from olivier.freebsd@free.fr) Received: from smtp2-g21.free.fr (smtp2-g21.free.fr [IPv6:2a01:e0c:1:1599::11]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SWspV4JMwz4dvG; Fri, 17 Nov 2023 10:00:18 +0000 (UTC) (envelope-from olivier.freebsd@free.fr) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=free.fr header.s=smtp-20201208 header.b=sTctf6KI; spf=pass (mx1.freebsd.org: domain of olivier.freebsd@free.fr designates 2a01:e0c:1:1599::11 as permitted sender) smtp.mailfrom=olivier.freebsd@free.fr; dmarc=pass (policy=none) header.from=free.fr Received: from ravel.localnet (unknown [90.118.140.172]) (Authenticated sender: olivier.freebsd@free.fr) by smtp2-g21.free.fr (Postfix) with ESMTPSA id 7FF0D2005A2; Fri, 17 Nov 2023 11:00:08 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=free.fr; s=smtp-20201208; t=1700215214; bh=D2ROkOy91N8soXi5RJniZ6QapYj1cqR5PTXbvlx4dqE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sTctf6KIypwVWVxgJOv/H7imeteTfihuN0lS7xnTw1Y8WKQ9afqDjRBVbVOx03J4G Inw4ldSGbLeLcExp15axZcSA0M77158VVaO2fsUPNJHOiiyTrBi19ZPjzrHXyVXSVh Ja+LtnuGylc89iivRP9a/GTO72P9sSFb7n2va5+xDLKG0DyulGd/HxjRTs7GlqysTy mYvngn7SFH2M0aaiJEu3D4SgsnoVbKeAOlQaw2QBvEtt29cZI26FtI8gwjRBIjiM3s zQ++FGZqKyYSYnQIHRoCqiOkuBk1GeP36R30S/1IkME8DeVUpPOnKjSi/NEPbIf9gQ /SLIWhxzy245w== From: Olivier Certner To: The Doctor , Glen Barber Cc: John Baldwin , FreeBSD Release Engineering Team , freebsd-current@freebsd.org, freebsd-stable@freebsd.org Subject: Re: [HEADS-UP] Quick update to 14.0-RELEASE schedule Date: Fri, 17 Nov 2023 11:00:07 +0100 Message-ID: <1965344.ruP0FAUDoQ@ravel> In-Reply-To: <20231116003030.GO1307@FreeBSD.org> References: <20231114203654.GB52320@FreeBSD.org> <20231116003030.GO1307@FreeBSD.org> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="UTF-8" X-Spamd-Result: default: False [-1.04 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_SPAM_SHORT(0.86)[0.860]; DMARC_POLICY_ALLOW(-0.50)[free.fr,none]; CTE_CASE(0.50)[]; MID_RHS_NOT_FQDN(0.50)[]; R_DKIM_ALLOW(-0.20)[free.fr:s=smtp-20201208]; R_SPF_ALLOW(-0.20)[+ip6:2a01:e0c:1:1599::11]; ONCE_RECEIVED(0.10)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_HAS_DN(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org,freebsd-stable@freebsd.org]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[free.fr]; ASN(0.00)[asn:12322, ipnet:2a01:e00::/26, country:FR]; RCPT_COUNT_FIVE(0.00)[6]; FREEMAIL_FROM(0.00)[free.fr]; TO_DN_SOME(0.00)[]; DWL_DNSWL_NONE(0.00)[free.fr:dkim]; DKIM_TRACE(0.00)[free.fr:+]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Queue-Id: 4SWspV4JMwz4dvG X-Spamd-Bar: - Hi Glen, > I also agree we cannot prevent people from downloading the images, > installers, whatever before the announcement. That is the lovely race > condition with which we have to live at the moment. Yes, and given that, I don't think you did anything wrong. It seems that the race is the same for freebsd-update(8), according to "The Doctor". But maybe in this case it is easier to fix, perhaps if installing the metadata signaling the new release to it can be delayed, by contrast with big files containing the actual data (I don't know how freebsd-update(8) works, so this is just a guess). > The alternative would be to say nothing at all. I really hope you and re@ will never choose this way. > Either way, it is a productivity, communication drain. It is > a lose-lose situation no matter how one looks at it given the above > context. We either get chastised for being "too open" into insights > delaying an official announcement, or for being "not open enough" when > there is silence from RE when a release does not meet its scheduled > announcement date. IMHO, a delay should always be announced (perhaps unless it's only a few days). If it is too hard to decide on the right level of details, then only mention that some (potential or verified) problem is being looked upon, but don't let that prevent you from communicating. As for people saying that they have already installed the "release", I'd suggest to have some text on the website (https://www.freebsd.org/releng/ perhaps) explaining that images on mirrors can appear in advance of release announcements and that they should not be considered as official until the official mail is sent, and just kindly point them to it. I hope this can contribute to at least attenuating the drain you're experiencing. Regards. -- Olivier Certner From nobody Fri Nov 17 13:29:31 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SWyS20RMXz50ynd for ; Fri, 17 Nov 2023 13:29:38 +0000 (UTC) (envelope-from void@f-m.fm) Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SWyS10R4bz3V3T for ; Fri, 17 Nov 2023 13:29:36 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm1 header.b=ZYgrGUl0; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=tj4NEsbC; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 66.111.4.26 as permitted sender) smtp.mailfrom=void@f-m.fm; dmarc=pass (policy=none) header.from=f-m.fm Received: from compute7.internal (compute7.nyi.internal [10.202.2.48]) by mailout.nyi.internal (Postfix) with ESMTP id 763DF5C0231 for ; Fri, 17 Nov 2023 08:29:35 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute7.internal (MEProxy); Fri, 17 Nov 2023 08:29:35 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm1; t=1700227775; x=1700314175; bh=n2 RV0yqh8d3r9bP7dHAb0wXq+veA7FLQNwjsUM5fg1g=; b=ZYgrGUl0K0qPZMU08B xxesMSF8Y31xtixUyRCpk0APaqBG55Wycf3tny/QgduRsQbPt2URRAuTjV9dkivI 4nfewLqlFh7c7cvKjBMOkHpS210Q4KagowNIjN3fEWAiLim+ZfyfPmxoZCzuQRUK HV/dp7klOErwv9YEWfcoJucoD0hEnOPYRpR8VCnhBH5Vy1haFdThW3HoyrWY3h3h 74BX6C7jIctkvu6c/ej74nH0LrxqJx39wg62gp4dJvCx0OAuZHC6ipPQvMu9kDQZ DOT1V/2gn1RBSl6aYNw4vA10ulRImltOOc159OdlPH+0KMMS0qJFsGGIdU5orP42 9UuQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; t=1700227775; x=1700314175; bh=n2RV0yqh8d3r9 bP7dHAb0wXq+veA7FLQNwjsUM5fg1g=; b=tj4NEsbC2zJ2TYHxdkiSSE8qFm1kG E8tw8nFzGsWQNDAON4dVdolkYu9YP49I5qjanjfDfnFJL2OIn1o7uklGC9qoXFN+ J+aiL0NGT882XbJWBdRYlhxCQbVaxM5emaBdRjlhjsdcYj0gO9GCMXPDPrBaWYjM MQUTp1SZqN8T6o80wMHnp0hTqL+kH/G88h+RnyZZ51M9coF3E4+Igko7bqyNNYsR cwTvcdrX5DPxSVVyiD357e4i1n7uTsXXhRBy2aLNoeMfdv4Cl1ttV+2rGi+WEK9A JV0Cicq24eyWXZ6ItTT8RAYQaLzSw/p3NmMamkxzrMWlaIZw6wGOdlueQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrudegtddgheefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkfhggtggujgesthdtro dttddtvdenucfhrhhomhepvhhoihguuceovhhoihgusehfqdhmrdhfmheqnecuggftrfgr thhtvghrnhephfekffeigeehgffghfdvveeiudejueelffefvdelgedtkeeigeefkeffhe fffeeinecuffhomhgrihhnpehklhgrrhgrshihshhtvghmshdrtghomhenucevlhhushht vghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehvohhiugesfhdqmhdrfh hm X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Fri, 17 Nov 2023 08:29:34 -0500 (EST) Date: Fri, 17 Nov 2023 13:29:31 +0000 From: void To: freebsd-current@freebsd.org Subject: Re: Request for Testing: TCP RACK Message-ID: Mail-Followup-To: freebsd-current@freebsd.org References: <42C327BD-6CE4-43AA-A1AE-3BEC08D623DB@freebsd.org> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <42C327BD-6CE4-43AA-A1AE-3BEC08D623DB@freebsd.org> X-Spamd-Result: default: False [-4.70 / 15.00]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MID_RHS_NOT_FQDN(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm1,messagingengine.com:s=fm1]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.26]; MIME_GOOD(-0.10)[text/plain]; RWL_MAILSPIKE_GOOD(-0.10)[66.111.4.26:from]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.26:from]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; ARC_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_NONE(0.00)[]; FREEMAIL_FROM(0.00)[f-m.fm]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Rspamd-Queue-Id: 4SWyS10R4bz3V3T X-Spamd-Bar: ---- On Thu, Nov 16, 2023 at 10:13:05AM +0100, tuexen@freebsd.org wrote: >You can load the kernel module using >kldload tcp_rack > >You can make the RACK stack the default stack using >sysctl net.inet.tcp.functions_default=rack Hi, thank you for this. https://klarasystems.com/articles/using-the-freebsd-rack-tcp-stack/ mentions this needs to be set in /etc/src.conf : WITH_EXTRA_TCP_STACKS=1 Is this still the case? Context here is -current both in a vm and bare metal, on various machines, on various connections, from DSL to 10Gb. Is there a method (yet) for enabling this functionality in various -RELENG maybe where one can compile in a vm built for that purpose, then transferring to the production vm? Would it be expected to work on arm64? What testing would you like doing? -- From nobody Fri Nov 17 13:31:02 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SWyTl4Nhfz50ysK; Fri, 17 Nov 2023 13:31:07 +0000 (UTC) (envelope-from herbert@gojira.at) Received: from mail.bsd4all.net (mail.bsd4all.net [IPv6:2a01:4f8:13b:240c::25]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "mail.bsd4all.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SWyTl11lcz3W03; Fri, 17 Nov 2023 13:31:07 +0000 (UTC) (envelope-from herbert@gojira.at) Authentication-Results: mx1.freebsd.org; none Date: Fri, 17 Nov 2023 14:31:02 +0100 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=gojira.at; s=mail202005; t=1700227863; bh=+BnxFnFBkQh5y0sFmW59LrvMuULs3MQFYqoPGlEzWb0=; h=Date:Message-ID:From:To:Cc:Subject:MIME-Version:Content-Type; b=fJDxbqPnA9aSHXtPx+j+geVhWLl0/wt8FsjTlRo1BFbXIzz16n4m9zUHF3g4sj9Qj tAQ1qpJDbc7lP/qhU7/fMd0FDhswnG736OKUqO08bpK4vQANWqaz3ayVkxidOukZwA L52a3orpe3le91RglHMnlMZx4oKHj/Qf6VWPII4fC80Fwb1XpU/9SAK1nhh6eX4MvU gKMgiSkJ/kgZLMTWEamH1Si203QQmUfmegK/O8LwiMcoOhm+HbGCzyNFv/0arsELXn 4P/Hf6actLMpzfDG/4ZUvL3lJwUpGcLdC25CPj8IxOwkCk9z2WDYdExl5shjGnc+mB JebA1VaqjMjYg== Message-ID: <87edgo33s9.wl-herbert@gojira.at> From: "Herbert J. Skuhra" To: tuexen@freebsd.org Cc: Olivier =?ISO-8859-1?Q?Cochard-Labb=E9?= , current@freebsd.org, net@freebsd.org Subject: Re: Request for Testing: TCP RACK In-Reply-To: <867DC224-F0B1-4C9C-8EB7-C9F7BE1C029A@freebsd.org> References: <42C327BD-6CE4-43AA-A1AE-3BEC08D623DB@freebsd.org> <87pm09ykb8.wl-herbert@gojira.at> <87o7ftyhkp.wl-herbert@gojira.at> <87msvdy0v4.wl-herbert@gojira.at> <867DC224-F0B1-4C9C-8EB7-C9F7BE1C029A@freebsd.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) Emacs/29.1 Mule/6.0 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/32, country:DE] X-Rspamd-Queue-Id: 4SWyTl11lcz3W03 Hi, On Fri, 17 Nov 2023 00:15:13 +0100, tuexen@freebsd.org wrote: >=20 > > On Nov 16, 2023, at 20:06, Herbert J. Skuhra wrote: > >=20 > > On Thu, 16 Nov 2023 19:07:29 +0100, Olivier Cochard-Labb=C3=A9 wrote: > >>=20 > >> On Thu, Nov 16, 2023 at 5:10=E2=80=AFPM Herbert J. Skuhra wrote: > >>=20 > >>>=20 > >>> OK, I am now running GENERIC-NODEBUG + "options TCPHPTS". > >>>=20 > >>> After setting "sysctl net.inet.tcp.functions_default=3Drack" git no > >>> longer works: > >>>=20 > >>>=20 > >> Are you using a fresh 15 head or a specific network setup ? > >>=20 > >> Because I'm not able to reproduce your problem on my system: > >>=20 > >> $ uname -a > >> FreeBSD bigone 15.0-CURRENT FreeBSD 15.0-CURRENT #0 > >> main-n266452-070d9e3540e6: Thu Nov 16 17:53:15 CET 2023 > >> root@bigone:/usr/obj/usr/src/amd64.amd64/sys/TCPHPTS > >> amd64 > >> $ cat /usr/src/sys/amd64/conf/TCPHPTS > >> include GENERIC-NODEBUG > >> ident TCPHPTS > >> options TCPHPTS > >> $ sysctl net.inet.tcp.functions_default > >> net.inet.tcp.functions_default: rack > >> $ git clone -q git@github.com:freebsd/freebsd-src.git && echo working > >> working > >> $ > >=20 > > OK, (g)it works if I disable pf. Do you use pf? > Can you share your pf config such that I can reproduce the problem locall= y? 1. It even fails with a simple pf.conf: pass in all pass out all 2. Fetching port distfiles also failed. 3. If I disable rxcsum on the ethernet adapter (igb0) it works. Thanks. -- Herbert From nobody Fri Nov 17 13:29:19 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SWyVG0Cn0z50yLL; Fri, 17 Nov 2023 13:31:34 +0000 (UTC) (envelope-from jamie@catflap.org) Received: from donotpassgo.dyslexicfish.net (donotpassgo.dyslexicfish.net [IPv6:2001:19f0:7400:8808:123::1]) by mx1.freebsd.org (Postfix) with ESMTP id 4SWyVF500qz3X4y; Fri, 17 Nov 2023 13:31:33 +0000 (UTC) (envelope-from jamie@catflap.org) Authentication-Results: mx1.freebsd.org; none X-Catflap-Envelope-From: Received: from donotpassgo.dyslexicfish.net (donotpassgo.dyslexicfish.net [209.250.224.51]) by donotpassgo.dyslexicfish.net (8.14.5/8.14.5) with ESMTP id 3AHDTLk6021700; Fri, 17 Nov 2023 13:29:21 GMT (envelope-from jamie@donotpassgo.dyslexicfish.net) Received: (from jamie@localhost) by donotpassgo.dyslexicfish.net (8.14.5/8.14.5/Submit) id 3AHDTJhH021698; Fri, 17 Nov 2023 13:29:19 GMT (envelope-from jamie) From: Jamie Landeg-Jones Message-Id: <202311171329.3AHDTJhH021698@donotpassgo.dyslexicfish.net> Date: Fri, 17 Nov 2023 13:29:19 +0000 Organization: Dyslexic Fish To: jamie@catflap.org, gjb@freebsd.org Cc: re@freebsd.org, freebsd-stable@freebsd.org, freebsd-current@freebsd.org, doctor@doctor.nl2k.ab.ca Subject: Re: [HEADS-UP] Quick update to 14.0-RELEASE schedule References: <20231114203654.GB52320@FreeBSD.org> <20231115022701.GM1307@FreeBSD.org> <202311162122.3AGLMqsN086011@donotpassgo.dyslexicfish.net> <20231116225246.GQ1307@FreeBSD.org> In-Reply-To: <20231116225246.GQ1307@FreeBSD.org> User-Agent: Heirloom mailx 12.4 7/29/08 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (donotpassgo.dyslexicfish.net [209.250.224.51]); Fri, 17 Nov 2023 13:29:21 +0000 (GMT) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:20473, ipnet:2001:19f0:7400::/38, country:US] X-Rspamd-Queue-Id: 4SWyVF500qz3X4y Glen Barber wrote: > No. It merely suggests the release is not officially official yet. Ok. Thanks for the clarification. Jamie. From nobody Fri Nov 17 13:46:52 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SWyr910BDz510b4; Fri, 17 Nov 2023 13:47:05 +0000 (UTC) (envelope-from olivier@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SWyr90T6mz3Zyh; Fri, 17 Nov 2023 13:47:05 +0000 (UTC) (envelope-from olivier@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700228825; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=ernILEH42RQVApuJux4nyM/OC7sjtY9XktmhN1xwQHo=; b=PzSBDwUJoYKNSrrxODCc/3EfgW7dCmXaIu1FbFQQSam2E1QoBUJ5/fbTMjcpNOMmflZiCT Gra3TY0r3V5C2XBDN5fuDRtBaGXMJLg4nrueusKGzacIk/1D8rHfP+007ELNN9CkAkVAZ5 i/sx4yvUI4Ubult7lSJFdTwOxH601Bj4ZSQ2woX8CxptiRAmZu32J2+dRYpM44lWs7A+4E njc4qjjPxBofk6cD8dIqBigrCLGuajukrPhO9C2wPAqRsdWUZBlonSQhP81MxlqLa4Aguz gVYpwyLJIH2C65JkioKOUjikqSrAR5UBuZ0B5FvpGrjRZ3nYeAYS1kASa1VG1w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700228825; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=ernILEH42RQVApuJux4nyM/OC7sjtY9XktmhN1xwQHo=; b=hFX9xWTRvpiFcP5vJFolwUVeSJ5RneQYnGNoRY/q2erIRxFR/QdmTt6E48JFTd0rjo+dsX UyDvqMHTKUcNbJ9yR55FyDRSJxS94fYsT7OpmO9bN8Dk466pTfioSJQTlSQjUqUsw43HQS dRMFDHjM7jzOzTGQ4P9XaUKqerDgyRNZLm+E+Wvby9pDCs7ShOCGSC42X+rA2phFQYFbML O2IK1nWmWXts9aDETjaqfnqMAWNOkXa4U5uBbIOc0x9/Kqo0e7vGKcOiXcqxLCsmA29Rir 2TAcCyeTxvPikHoi0qyOBsTaYMVQuD+zwKKRlxY4nWBEvZrv4avvQ2EeAbjlDQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1700228825; a=rsa-sha256; cv=none; b=rfS+oSm7rpqJ9kaf59ZcUnR1dEFwkY9qnioWyyacDf/WwmHcTovEQlSsoWcsmnwDIzd9m1 eO8xOEcGXSXTfUqBs4cyBAxPMkyJPsS5PPtGk38T7VAvwJNrDCEJEGp6lniwC97rTyBzrd EqInJVHUJhFf9+r0iRkWE1Xt8SB4dj6VUymDgYBmXGwPKm/FwTKgIlQG4dL8gtsJdb4o7c a63F4+dY53CYbxKOjq1v7Ur48k/HGW4+vKxgWAqfV8JZbeZSJROD1fFcs48uXyuHtnUH+e 6cF5hCRdYH1QFLJ2VmCgN6ho0e40RgTpxlhB0HPN7DRruVuY1WGxdvW9Hlc3Rw== Received: from mail-oi1-f172.google.com (mail-oi1-f172.google.com [209.85.167.172]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) (Authenticated sender: olivier/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4SWyr86XWtz1Q7j; Fri, 17 Nov 2023 13:47:04 +0000 (UTC) (envelope-from olivier@freebsd.org) Received: by mail-oi1-f172.google.com with SMTP id 5614622812f47-3b2ec5ee2e4so1221995b6e.3; Fri, 17 Nov 2023 05:47:04 -0800 (PST) X-Gm-Message-State: AOJu0YwBfJFXzDn0pJvWEzJm7++53IGJ/YMITCoC5W3DXGhAT0dsmlRW 2ix0pbftY5qPfp+JLMQpj0+Ow7fWjrFu1X4qmIg= X-Google-Smtp-Source: AGHT+IHsf/TzHHs7bPHo7jhsBBxfNDnQSgcrdbLXa/AqOryzKvIzoK9z9dDRAwk69gLckxXgqp5F59gtp7sK/PrT4BU= X-Received: by 2002:a05:6359:1a4e:b0:16d:a0d3:d285 with SMTP id ru14-20020a0563591a4e00b0016da0d3d285mr3609943rwb.18.1700228824019; Fri, 17 Nov 2023 05:47:04 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: <42C327BD-6CE4-43AA-A1AE-3BEC08D623DB@freebsd.org> <87pm09ykb8.wl-herbert@gojira.at> <87o7ftyhkp.wl-herbert@gojira.at> <87msvdy0v4.wl-herbert@gojira.at> <867DC224-F0B1-4C9C-8EB7-C9F7BE1C029A@freebsd.org> <87edgo33s9.wl-herbert@gojira.at> In-Reply-To: <87edgo33s9.wl-herbert@gojira.at> From: =?UTF-8?Q?Olivier_Cochard=2DLabb=C3=A9?= Date: Fri, 17 Nov 2023 14:46:52 +0100 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Request for Testing: TCP RACK To: "Herbert J. Skuhra" Cc: tuexen@freebsd.org, current@freebsd.org, net@freebsd.org Content-Type: multipart/alternative; boundary="00000000000017b87a060a595f61" --00000000000017b87a060a595f61 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, Nov 17, 2023 at 2:31=E2=80=AFPM Herbert J. Skuhra wrote: > > 1. It even fails with a simple pf.conf: > pass in all > pass out all > > 2. Fetching port distfiles also failed. > > 3. If I disable rxcsum on the ethernet adapter (igb0) it works. > > > I can't reproduce it with pfctl too (same igb drivers with default RXCSUM enabled). $ cat /etc/pf.conf pass in all pass out all $ service pf onestart Enabling pf . $ pfctl -sr pass in all flags S/SA keep state pass out all flags S/SA keep state $ sysctl net.inet.tcp.functions_default net.inet.tcp.functions_default: rack $ ifconfig igb0 | grep option options=3D4e523bb nd6 options=3D21 $ git clone -q git@github.com:freebsd/freebsd-src.git && echo working working What is your igb chipset exactly ? (pciconf -lv | grep -B 3 -F "network") What is your netstat -ss output ? --00000000000017b87a060a595f61 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Fri, Nov 17, 2023 at 2:31=E2=80=AFPM Herbert J. = Skuhra <herbert@gojira.at> w= rote:

1. It even fails with a simple pf.conf:
=C2=A0 =C2=A0pass in all
=C2=A0 =C2=A0pass out all

2. Fetching port distfiles also failed.

3. If I disable rxcsum on the ethernet adapter (igb0) it works.


I can't reproduce it with= pfctl too (same igb drivers with default RXCSUM enabled).

$ cat /et= c/pf.conf
pass in all
pass out all
$ service pf onestart
Enabli= ng pf
.
$ pfctl -sr
pass in all flags S/SA keep state
pass out = all flags S/SA keep state
$ sysctl net.inet.tcp.functions_default
net= .inet.tcp.functions_default: rack
$ ifconfig igb0 | grep option
=C2= =A0 =C2=A0 =C2=A0 =C2=A0 options=3D4e523bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_H= WTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO= ,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
=C2=A0 =C2=A0 =C2=A0 =C2=A0 = nd6 options=3D21<PERFORMNUD,AUTO_LINKLOCAL>

$ git clone -q git= @github.com:freebsd/freebsd-src.git && echo working
working
<= br>What is your igb chipset exactly =C2=A0? (pciconf -lv | grep -B 3 -F &qu= ot;network")
What is your netstat -ss output ?

--00000000000017b87a060a595f61-- From nobody Fri Nov 17 14:06:03 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SWzG55q9Cz511pf; Fri, 17 Nov 2023 14:06:05 +0000 (UTC) (envelope-from herbert@gojira.at) Received: from mail.bsd4all.net (mail.bsd4all.net [IPv6:2a01:4f8:13b:240c::25]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "mail.bsd4all.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SWzG54lRSz3dhP; Fri, 17 Nov 2023 14:06:05 +0000 (UTC) (envelope-from herbert@gojira.at) Authentication-Results: mx1.freebsd.org; none Date: Fri, 17 Nov 2023 15:06:03 +0100 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=gojira.at; s=mail202005; t=1700229963; bh=ptivQN0zZWgC8co5omvlBVlkepZstdRrzf++V56Rfgk=; h=Date:From:To:Cc:Subject:Message-ID:MIME-Version:Content-Type; b=H/wu7ez2YStFLupRxGkKNirMjbVjReWvlyAIyVJOsgyjgBPmF/G63NunnqLIK+oz3 7fwHUTXJIX8v++5RVjdwLYwzPjcKp3zyPYHC99IwifF4S9yYxU64MKG9OFi/JIdPW2 rjREBIoAFRtSbWGD6MpODrk12UG+r+KqqKJWaSrLT4qULkqcGhNoU18SbUpU1rII30 WlCJHqI1UKsYGERSVPOxrypBbGvYBVUZU3h0bDcgYUPyeqhEye4zv/4K3ENTOVVTrw e0jL3KduRhSbzpNep5bOMIsEsITOY2uOEJ+3xkn7UCSbzDUpnZq3OK4ZjSmGa02G+3 AUo+J6HWUjErA== From: "Herbert J. Skuhra" To: Olivier =?iso-8859-1?Q?Cochard-Labb=E9?= Cc: tuexen@freebsd.org, current@freebsd.org, net@freebsd.org Subject: Re: Request for Testing: TCP RACK Message-ID: References: <42C327BD-6CE4-43AA-A1AE-3BEC08D623DB@freebsd.org> <87pm09ykb8.wl-herbert@gojira.at> <87o7ftyhkp.wl-herbert@gojira.at> <87msvdy0v4.wl-herbert@gojira.at> <867DC224-F0B1-4C9C-8EB7-C9F7BE1C029A@freebsd.org> <87edgo33s9.wl-herbert@gojira.at> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/32, country:DE] X-Rspamd-Queue-Id: 4SWzG54lRSz3dhP Hi, On Fri, Nov 17, 2023 at 02:46:52PM +0100, Olivier Cochard-Labbé wrote: > On Fri, Nov 17, 2023 at 2:31 PM Herbert J. Skuhra wrote: > > > > > 1. It even fails with a simple pf.conf: > > pass in all > > pass out all > > > > 2. Fetching port distfiles also failed. > > > > 3. If I disable rxcsum on the ethernet adapter (igb0) it works. > > > > > I can't reproduce it with pfctl too (same igb drivers with default RXCSUM > enabled). > > $ cat /etc/pf.conf > pass in all > pass out all > $ service pf onestart > Enabling pf > . > $ pfctl -sr > pass in all flags S/SA keep state > pass out all flags S/SA keep state > $ sysctl net.inet.tcp.functions_default > net.inet.tcp.functions_default: rack > $ ifconfig igb0 | grep option > > options=4e523bb > nd6 options=21 > > $ git clone -q git@github.com:freebsd/freebsd-src.git && echo working > working > > What is your igb chipset exactly ? (pciconf -lv | grep -B 3 -F "network") igb0@pci0:41:0:0: class=0x020000 rev=0x03 hdr=0x00 vendor=0x8086 device=0x1533 subvendor=0x1849 subdevice=0x1533 vendor = 'Intel Corporation' device = 'I210 Gigabit Network Connection' class = network > What is your netstat -ss output ? tcp: 946 packets sent 933 data packets (41681 bytes) 7946 ack-only packets (2 delayed) 1 control packet 999 packets received 860 acks (for 41681 bytes) 910 packets (118790 bytes) received in-sequence 12 completely duplicate packets (17136 bytes) 1 connection request 1 connection established (including accepts) 1 time used RTT from hostcache 1 time used RTT variance from hostcache 1 connection closed (including 1 drop) 1 connection updated cached RTT on close 1 connection updated cached RTT variance on close 862 segments updated rtt (of 847 attempts) 71 correct ACK header predictions 124 correct data packet header predictions 7910 SACK options (SACK blocks) sent TCP connection count by state: 7 connections in LISTEN state 1 connection in ESTABLISHED state udp: 39 datagrams received 39 delivered 39 datagrams output ip: 80 total packets received 23 packets for this host 23 packets sent from this host icmp: ICMP address mask responses are disabled igmp: arp: ip6: 933 total packets received 931 packets for this host 8891 packets sent from this host Input histogram: TCP: 915 UDP: 16 ICMP6: 2 Mbuf statistics: 870 one mbuf 63 one ext mbuf 0 two or more ext mbuf source addresses on an outgoing I/F 2 link-locals 3 globals source addresses of same scope 2 link-locals 3 globals Source addresses selection rule applied: 5 first candidate 3 appropriate scope icmp6: Output histogram: neighbor solicitation: 2 Input histogram: neighbor advertisement: 2 Histogram of error messages to be generated: rip6: Thanks. -- Herbert From nobody Fri Nov 17 14:32:24 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SWzs34wylz512pW for ; Fri, 17 Nov 2023 14:32:55 +0000 (UTC) (envelope-from Alexander@Leidinger.net) Received: from mailgate.Leidinger.net (mailgate.leidinger.net [IPv6:2a00:1828:2000:313::1:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "mailgate.leidinger.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SWzs24J2Gz4CZJ for ; Fri, 17 Nov 2023 14:32:54 +0000 (UTC) (envelope-from Alexander@Leidinger.net) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=leidinger.net header.s=outgoing-alex header.b=fhFcrrhl; spf=pass (mx1.freebsd.org: domain of Alexander@Leidinger.net designates 2a00:1828:2000:313::1:5 as permitted sender) smtp.mailfrom=Alexander@Leidinger.net; dmarc=pass (policy=quarantine) header.from=leidinger.net List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=leidinger.net; s=outgoing-alex; t=1700231562; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=fCphztVuW1IuWczZrMXr9l7rY8X3EG0NR4+ohItWqi8=; b=fhFcrrhlbth6sdzo0t4c97a2OBhb5w9hP8cr4ubFJofIa/4tSRNHoecghDZ69z3k8sDKYs duj4FTYkcT/dXfUYsORQJvtIcXW4X48rS1MdQxYbuGx698PGBz+BDUUrrNgICE+MGS+LZN 9m8Tkk2xsFL2wl2E0407TvKN+1F+98Z5wqwKc4omi7PGO+Nvdm0LSE63F0JqjQZvRYbcQt dT+yFVHztKT+wVpjM9hWQmaidGUQz8jORugVLUbQ5U0LcY1ShdUXFJN8GJgnZwxClg1JHv fr/5SSlV6oEUNALMG/a/HnUWFm0EoStLtfyVIzjylmJz6XzG5TldYs/JD73O1g== Date: Fri, 17 Nov 2023 15:32:24 +0100 From: Alexander Leidinger To: freebsd-current@freebsd.org Subject: Re: Request for Testing: TCP RACK In-Reply-To: References: <42C327BD-6CE4-43AA-A1AE-3BEC08D623DB@freebsd.org> Message-ID: X-Sender: Alexander@Leidinger.net Organization: No organization, this is a private message. Content-Type: multipart/signed; protocol="application/pgp-signature"; boundary="=_b27818d7618f8125ddb95a78259314df"; micalg=pgp-sha256 X-Spamd-Result: default: False [-6.06 / 15.00]; SIGNED_PGP(-2.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.96)[-0.963]; DMARC_POLICY_ALLOW(-0.50)[leidinger.net,quarantine]; R_DKIM_ALLOW(-0.20)[leidinger.net:s=outgoing-alex]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; R_SPF_ALLOW(-0.20)[+mx]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; HAS_ORG_HEADER(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[leidinger.net:+]; HAS_ATTACHMENT(0.00)[]; ASN(0.00)[asn:34240, ipnet:2a00:1828::/32, country:DE]; TO_DN_NONE(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Queue-Id: 4SWzs24J2Gz4CZJ X-Spamd-Bar: ------ This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --=_b27818d7618f8125ddb95a78259314df Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; format=flowed Am 2023-11-17 14:29, schrieb void: > On Thu, Nov 16, 2023 at 10:13:05AM +0100, tuexen@freebsd.org wrote: > >> You can load the kernel module using >> kldload tcp_rack >> >> You can make the RACK stack the default stack using >> sysctl net.inet.tcp.functions_default=rack > > Hi, thank you for this. > > https://klarasystems.com/articles/using-the-freebsd-rack-tcp-stack/ > mentions > this needs to be set in /etc/src.conf : > > WITH_EXTRA_TCP_STACKS=1 > > Is this still the case? Context here is -current both in a vm and bare > metal, on various machines, on various connections, from DSL to 10Gb. On a recent -current: this is not needed anymore, it is part of the defaults now. But you may still compile the kernel with "option TCPHPTS" (until it's added to the defaults too). > Is there a method (yet) for enabling this functionality in various > -RELENG > maybe where one can compile in a vm built for that purpose, then > transferring to the production vm? Copy the kernel which was build according to the acticle from klara systems to your target VM. > Would it be expected to work on arm64? Yes (I use it on an ampere VM in the cloud). Bye, Alexander. -- http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF --=_b27818d7618f8125ddb95a78259314df Content-Type: application/pgp-signature; name=signature.asc Content-Disposition: attachment; filename=signature.asc; size=833 Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEER9UlYXp1PSd08nWXEg2wmwP42IYFAmVXeYYACgkQEg2wmwP4 2IZOrA/+K9jEUlov3PU5HoaRF5+CL/II8p+3S8hcU4Gwuw4sN6EO6/StaT6RNbce uEMc6VNp2mNKlNgLGKvkszARxkdC8nSkO3wWKHJOiqMsmilNY7suFC1k/uEXJ2Fm D9SKstFn/RbN1TeGbMg/sY41xjMi5C2dlePo07gM5qJtsOv8d12GN+otFbW05aqq U9fNd3g2rc15iPWotHTWTx8lGdFR5zU4RutgTpNXYdvOuOA82HLfkV7Nyvm7CyEo 9aVJUt2cwuze6FLb2RdIbdnb4/J9CqirUQHmj+5RMMDFFVjnYm1HwFyVwm+sdEu/ 1Cbmj3psJAcXGskKzEIGQetFb6nVjwipwccU7gxGDTDiUFjBOf8a72YU52Xgza3Q lxBLqn1hQmz00FVPXEqL0FJGhnGdgcNLhTLv6P227jD4Il0ElzIJPh3KXMw2oKoH 5A7hXFzGuUzfLE4yvX/pDfxFE8X1kD76AoPwrcVo1HwaeCplO1AALs/3UDvbXbiy 1pbPP9W3HqDIvaBI8MWEVpDdAaRHdPdQgoOBrknOcUOmpeVW3q0aLAvsz2LIz1LP HYjMOb0r5cyAb+waZ//P5/dijLTrjznk2ykmKGVwVRd0MWCV96LIcHYyxL+XthJi E2gBL2ou9YO7IjtdykKhA5Axb/Guzx8jlEwdxd1fS8YnL4mCgas= =VuQ7 -----END PGP SIGNATURE----- --=_b27818d7618f8125ddb95a78259314df-- From nobody Fri Nov 17 16:06:47 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SX1xS284hz516H9 for ; Fri, 17 Nov 2023 16:06:52 +0000 (UTC) (envelope-from joh.hendriks@gmail.com) Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com [IPv6:2a00:1450:4864:20::52c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SX1xR1fq0z4LWg for ; Fri, 17 Nov 2023 16:06:51 +0000 (UTC) (envelope-from joh.hendriks@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=e+vb5nqT; spf=pass (mx1.freebsd.org: domain of joh.hendriks@gmail.com designates 2a00:1450:4864:20::52c as permitted sender) smtp.mailfrom=joh.hendriks@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-ed1-x52c.google.com with SMTP id 4fb4d7f45d1cf-5482df11e73so1471886a12.0 for ; Fri, 17 Nov 2023 08:06:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700237209; x=1700842009; darn=freebsd.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:to:subject:user-agent:mime-version:date:message-id:from :to:cc:subject:date:message-id:reply-to; bh=5An1cY82Vv/0cNV5ULzNNSH41RbIWEyoEEDU8TI4llA=; b=e+vb5nqTIUh2lwyqjgDa3B1USDjORc1iK89frqBiGsWKXPOP0dbZFNPlRKxT0jI1FD 0UxttxzoXK+Oy38dVuwDFMEbFl/nBFcnH2v3Zw2J3qZTERDfzU7IcpvcBE8JN9LGdwLA 7/ipeQxhe2ebKkV4UiRgdVlbArASA9DOwfbCyzEmW7BVYH39fszBTPxNU62/NoFMD1du leq8Not7FSGzDLoHxTN7XmNXPB82v3WvN1yWQzNwCsip+qZj9HbFxE/8tEsmpgZVCdQi CIMnEeGffmm9uGSXPyvLuAQ7/I0HeDPezZx0pWdY25W8WQ/gfmXCysMti5pklVPCupdV //7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700237209; x=1700842009; h=content-transfer-encoding:in-reply-to:from:content-language :references:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=5An1cY82Vv/0cNV5ULzNNSH41RbIWEyoEEDU8TI4llA=; b=H25iuGt/MLx0jKi1UlkDJIxDfVhVsN0HZSCxZq80ZlsJSf4dnnnpd1Sj5hSWBlnMyJ NSabjbHB14aY95rRv78kiFZcj47KeXmvZJ3ZRwqIpXgCEkhdCO5lkEduQ+h6kek9xAxg KWxhEWOUDF+Py2P6IBWniP/bXmDhjHwJTt3L1TscXRqsObaHjyv08vjEk5ngGjFoXleX Upunw9HL/4a3xbU1MNtXsmt9UK3BsyFlshEOwS9PsvTSXqTR7XTd6lwiJ6aCz7gvhNqc flH8btMnAracmxwst4HiZUhtJf5GxoLqKwR7zSIyO2BhqLQaEfm77Kmo48taV1nCSlRI rOZg== X-Gm-Message-State: AOJu0Yxio5K67mO/y9X4axYNhS5nH6Bx0mqO1mw1+/BMDnI9BamuEmIc KW980gF8pXWxmgkKYashVnS22PRZZyY= X-Google-Smtp-Source: AGHT+IEynsAy+paPlfVAK/ueqcMI1/AKb+UFdyBviExoj6LjyeGmGdrlRH3Myt8tmjb5GTFjAgTjUA== X-Received: by 2002:a17:906:2dda:b0:9ae:82b4:e306 with SMTP id h26-20020a1709062dda00b009ae82b4e306mr13825100eji.62.1700237209158; Fri, 17 Nov 2023 08:06:49 -0800 (PST) Received: from ?IPV6:2001:1c04:1337:b400:945:2f4b:b0e1:d0fa? (2001-1c04-1337-b400-0945-2f4b-b0e1-d0fa.cable.dynamic.v6.ziggo.nl. [2001:1c04:1337:b400:945:2f4b:b0e1:d0fa]) by smtp.gmail.com with ESMTPSA id i22-20020a1709063c5600b009e6b6681da7sm925951ejg.94.2023.11.17.08.06.48 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 17 Nov 2023 08:06:48 -0800 (PST) Message-ID: <2bbfcf68-3249-45c7-a8e5-af7115a1ccfe@gmail.com> Date: Fri, 17 Nov 2023 17:06:47 +0100 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Request for Testing: TCP RACK To: freebsd-current@freebsd.org References: <42C327BD-6CE4-43AA-A1AE-3BEC08D623DB@freebsd.org> Content-Language: nl From: Johan Hendriks In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spamd-Result: default: False [-3.88 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.89)[-0.893]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[text/plain]; XM_UA_NO_VERSION(0.01)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; ARC_NA(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::52c:from]; RCPT_COUNT_ONE(0.00)[1]; MID_RHS_MATCH_FROM(0.00)[]; TAGGED_FROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; DKIM_TRACE(0.00)[gmail.com:+]; TO_DN_NONE(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4SX1xR1fq0z4LWg X-Spamd-Bar: --- I am running the rack stack for quiet some time now on a baremetal machiene and never had problems. Also use pf.  This is a test machine so not a lot happening on it. Are there any thing we can test? Do we have some test scripts we can run? From nobody Fri Nov 17 17:51:05 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SX4Fq2zgJz51Cqy for ; Fri, 17 Nov 2023 17:51:11 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from drew.franken.de (mail-n.franken.de [193.175.24.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.franken.de", Issuer "Sectigo RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SX4Fp3RbKz4X6s for ; Fri, 17 Nov 2023 17:51:10 +0000 (UTC) (envelope-from tuexen@freebsd.org) Authentication-Results: mx1.freebsd.org; dkim=none; spf=softfail (mx1.freebsd.org: 193.175.24.27 is neither permitted nor denied by domain of tuexen@freebsd.org) smtp.mailfrom=tuexen@freebsd.org; dmarc=none Received: from smtpclient.apple (unknown [IPv6:2a02:8109:1140:c3d:ecd2:bf6:77d5:2c9f]) (Authenticated sender: micmac) by mail-n.franken.de (Postfix) with ESMTPSA id 426BF721E280D; Fri, 17 Nov 2023 18:51:06 +0100 (CET) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.200.91.1.1\)) Subject: Re: Request for Testing: TCP RACK From: tuexen@freebsd.org In-Reply-To: <2bbfcf68-3249-45c7-a8e5-af7115a1ccfe@gmail.com> Date: Fri, 17 Nov 2023 18:51:05 +0100 Cc: freebsd-current@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <42C327BD-6CE4-43AA-A1AE-3BEC08D623DB@freebsd.org> <2bbfcf68-3249-45c7-a8e5-af7115a1ccfe@gmail.com> To: Johan Hendriks X-Mailer: Apple Mail (2.3774.200.91.1.1) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, T_SCC_BODY_TEXT_LINE autolearn=disabled version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail-n.franken.de X-Spamd-Result: default: False [-2.70 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[193.175.24.27:from]; TAGGED_RCPT(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:680, ipnet:193.174.0.0/15, country:DE]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; DMARC_NA(0.00)[freebsd.org]; FREEFALL_USER(0.00)[tuexen]; FROM_NO_DN(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all:c]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[] X-Rspamd-Queue-Id: 4SX4Fp3RbKz4X6s X-Spamd-Bar: -- > On Nov 17, 2023, at 17:06, Johan Hendriks = wrote: >=20 > I am running the rack stack for quiet some time now on a baremetal = machiene and never had problems. > Also use pf. This is a test machine so not a lot happening on it. >=20 > Are there any thing we can test? Do we have some test scripts we can = run? We are actually interested in feedback about using the stack in whatever use case you use TCP for. The stack has been tested with the Netflix use case, but not so much with others. That is why we ask for broader = testing. Best regards Michael >=20 >=20 >=20 >=20 From nobody Fri Nov 17 16:11:35 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SX6YT2QTHz51JcF; Fri, 17 Nov 2023 19:34:53 +0000 (UTC) (envelope-from doctor@doctor.nl2k.ab.ca) Received: from doctor.nl2k.ab.ca (doctor.nl2k.ab.ca [204.209.81.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SX6YS6Vb6z3PNL; Fri, 17 Nov 2023 19:34:52 +0000 (UTC) (envelope-from doctor@doctor.nl2k.ab.ca) Authentication-Results: mx1.freebsd.org; none Received: from doctor by doctor.nl2k.ab.ca with local (Exim 4.97 (FreeBSD)) (envelope-from ) id 1r41RP-00000000EBT-10HC; Fri, 17 Nov 2023 09:11:35 -0700 Date: Fri, 17 Nov 2023 09:11:35 -0700 From: The Doctor To: Olivier Certner Cc: Glen Barber , John Baldwin , FreeBSD Release Engineering Team , freebsd-current@freebsd.org, freebsd-stable@freebsd.org Subject: Re: [HEADS-UP] Quick update to 14.0-RELEASE schedule Message-ID: References: <20231114203654.GB52320@FreeBSD.org> <20231116003030.GO1307@FreeBSD.org> <1965344.ruP0FAUDoQ@ravel> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1965344.ruP0FAUDoQ@ravel> X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:6171, ipnet:204.209.81.0/24, country:CA] X-Rspamd-Queue-Id: 4SX6YS6Vb6z3PNL On Fri, Nov 17, 2023 at 11:00:07AM +0100, Olivier Certner wrote: > Hi Glen, > > > I also agree we cannot prevent people from downloading the images, > > installers, whatever before the announcement. That is the lovely race > > condition with which we have to live at the moment. > > Yes, and given that, I don't think you did anything wrong. > > It seems that the race is the same for freebsd-update(8), according to "The Doctor". But maybe in this case it is easier to fix, perhaps if installing the metadata signaling the new release to it can be delayed, by contrast with big files containing the actual data (I don't know how freebsd-update(8) works, so this is just a guess). > > > The alternative would be to say nothing at all. > > I really hope you and re@ will never choose this way. > > > Either way, it is a productivity, communication drain. It is > > a lose-lose situation no matter how one looks at it given the above > > context. We either get chastised for being "too open" into insights > > delaying an official announcement, or for being "not open enough" when > > there is silence from RE when a release does not meet its scheduled > > announcement date. > > IMHO, a delay should always be announced (perhaps unless it's only a few days). If it is too hard to decide on the right level of details, then only mention that some (potential or verified) problem is being looked upon, but don't let that prevent you from communicating. > > As for people saying that they have already installed the "release", I'd suggest to have some text on the website (https://www.freebsd.org/releng/ perhaps) explaining that images on mirrors can appear in advance of release announcements and that they should not be considered as official until the official mail is sent, and just kindly point them to it. I hope this can contribute to at least attenuating the drain you're experiencing. > > Regards. > > -- > Olivier Certner > > just a suggestion for the future is that to have some sort of gpg/gpg check in freebsd-update . -- Member - Liberal International This is doctor@nk.ca Ici doctor@nk.ca Yahweh, King & country!Never Satan President Republic!Beware AntiChrist rising! Look at Psalms 14 and 53 on Atheism ; unsubscribe from Google Groups to be seen Lest we forget 11 Nov 2023 Beware https://mindspring.com From nobody Fri Nov 17 23:37:04 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SXCx810Ztz51VrX for ; Fri, 17 Nov 2023 23:37:16 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from www121.sakura.ne.jp (www121.sakura.ne.jp [153.125.133.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SXCx73z75z4QNr; Fri, 17 Nov 2023 23:37:14 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Authentication-Results: mx1.freebsd.org; none Received: from kalamity.joker.local (123-1-22-158.area1b.commufa.jp [123.1.22.158]) (authenticated bits=0) by www121.sakura.ne.jp (8.16.1/8.16.1/[SAKURA-WEB]/20201212) with ESMTPA id 3AHNb5Jm057462; Sat, 18 Nov 2023 08:37:05 +0900 (JST) (envelope-from junchoon@dec.sakura.ne.jp) Date: Sat, 18 Nov 2023 08:37:04 +0900 From: Tomoaki AOKI To: tuexen@freebsd.org Cc: freebsd-current@freebsd.org Subject: Re: Request for Testing: TCP RACK Message-Id: <20231118083704.3ebc17eb8f392ceae33f65f7@dec.sakura.ne.jp> In-Reply-To: References: <42C327BD-6CE4-43AA-A1AE-3BEC08D623DB@freebsd.org> <2bbfcf68-3249-45c7-a8e5-af7115a1ccfe@gmail.com> Organization: Junchoon corps X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; amd64-portbld-freebsd14.0) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:7684, ipnet:153.125.128.0/18, country:JP] X-Rspamd-Queue-Id: 4SXCx73z75z4QNr On Fri, 17 Nov 2023 18:51:05 +0100 tuexen@freebsd.org wrote: > > On Nov 17, 2023, at 17:06, Johan Hendriks wrote: > > > > I am running the rack stack for quiet some time now on a baremetal machiene and never had problems. > > Also use pf. This is a test machine so not a lot happening on it. > > > > Are there any thing we can test? Do we have some test scripts we can run? > We are actually interested in feedback about using the stack in whatever > use case you use TCP for. The stack has been tested with the Netflix use > case, but not so much with others. That is why we ask for broader testing. > > Best regards > Michael Are there any difference regarding with performance between main and stable/14? If so, please ignore below. I have stable/14 environment which is configured to be able to switch to TCP RACK and experienced huge performance loss when writing a large file to smbfs share on commercial NAS. CC is cubic. Testing large archive on the smbfs share doesn't seem to be affected. Comparison between default (freebsd) and rack TCP stack using sysutils/clone on stable/14 at commit 7d1321288ad9, amd64. Last 3 lines of outputs from clone (upload to NAS) are shown. Before switching to rack: 1 item copied, 2342.4 MB in 39.12 s -- 59.9 MB/s Leaked memory: 0 bytes No errors occured. Unmount the smbfs share, switch to rack, and after remount: 1 item copied, 2342.4 MB in 926.59 s -- 2.5 MB/s Leaked memory: 0 bytes No errors occured. Switch back to freebsd (default) without unmounting: 1 item copied, 2342.4 MB in 906.94 s -- 2.6 MB/s Leaked memory: 0 bytes No errors occured. Unmount and remount the smbfs share: 1 item copied, 2342.4 MB in 39.12 s -- 59.9 MB/s Leaked memory: 0 bytes No errors occured. Regards. -- Tomoaki AOKI From nobody Fri Nov 17 23:52:40 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SXDGz5838z51VtW; Fri, 17 Nov 2023 23:52:43 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SXDGz4S7Lz4SYx; Fri, 17 Nov 2023 23:52:43 +0000 (UTC) (envelope-from gjb@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700265163; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type; bh=/Q9jgqSr5bRBuaAOWVa7n1djeU2LlLT7lM5DCiGGlIs=; b=QmH9vWdXbHKvViwUVwOhdeNIK7TMiuOZVaGaWd6NmMf6xO8n29tgAUzJIv4PNem8pyo1jy 8T079GN2S6OdA5bzrJNvEmkWYhuVqnEHcvNnnBqAA5XV4qpmarzI5ShOwz12ExJmPX7/qi XC4IKZ79WbQG+iYmUQN+LDO0lCiqTq24+Lo6vMh+E/MtLsnukOjupKQnCArVsjcgtBs3BG U20FSZMoMZu3QN4GxNlXC5QWI0Nn71ivFZhuLrvt25cfH+DY3xFOehootPc7LhSlaQ0pas nNek6xrqb5CELX/a5dKMmLimcLLUTGbRF4Bf3DZ+o/j+cDxrKQitLTrYhTy4kw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700265163; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type; bh=/Q9jgqSr5bRBuaAOWVa7n1djeU2LlLT7lM5DCiGGlIs=; b=VyVrYK1v26Z7revO5QUuz/6epukfpQqHkz0d4wDp9EYmxnQC3E5dB0ze46oGl8axgx1LOG YB7S1P5yAhg7FQFmm8VtQShnQPxaEMJ7VGwXUKE+ORBKiAMv47jj3IEsSzhkIohDnjsDaY d8RMmFBbbQ+LO4i8A7l/UMotPSzV9r3MxnfW6Ruiv8/1FvP3uXaNLVQPcQigS5UprQNcBO YS+IWCe42vupk2bvXoMutDy25wPP39+gfXsEcVZBcSoebPha+lIGYGpRVnHbb6IwVzbck6 jY6jdVDdypkMRDtXv2CYf0ewhniAYwt44DZj52aQLYrS/2OJlipJSaSLxjsAuA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1700265163; a=rsa-sha256; cv=none; b=tFjYOeP4zjyGFcTVBPsxenN4LQS2kphKgCO657TmMORDYTKM0I6mqHUSkvM17zwIziOGeV gN8LX4XBzGlPfhiwgMhyJpSiFInOpS7iRnY5qWq3Jbj26d44ksJ80z4FvFIFnyrJdXTM1G PX9tFHHTG03TrvpP5ekGyfiQaQu+YSnfQmbCEvkZ4lutKtyWirFV6s4O6+xgSIv5wk8/Wc XKIEJnqp2euoiKGth5kP9fIQ/gItE4jiPEUXS/zVMBwwEUyaXSJV7elmK/iF6kevpO55Lc +2QsZlBhioY8ESEOXT05XphF4WiQyL+0fJ8+q8o4lj/AIDZmkMcWaQ4GvgAzmg== Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by freefall.freebsd.org (Postfix) with ESMTPS id 2D0C71C273; Fri, 17 Nov 2023 23:52:43 +0000 (UTC) (envelope-from gjb@freebsd.org) Date: Fri, 17 Nov 2023 23:52:40 +0000 From: Glen Barber To: freebsd-current@freebsd.org, freebsd-stable@freebsd.org Cc: FreeBSD Release Engineering Team Subject: Update to the Release Engineering Team Roster Message-ID: <20231117235240.GU1307@FreeBSD.org> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; x-action=pgp-signed -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 FreeBSD community, It is with a heavy heart that I have decided to resign my role as the Release Engineering Team Lead. Colin Percival will take over this role, and I will act as his Deputy Lead unless and until he finds a more suitable person to fill this role. Regardless, I do not have any immediate plans to resign from the RE Team as a whole, and though Colin has proved during the 13.2-RELEASE cycle that he is fully capable of overtaking this role, I will remain active on the team and will be available to him to provide any guidance or insights as needed. I personally want to thank Colin for accepting this role change. I have worked with Colin for several years directly, and indirectly, much longer than I can remember. As I am sure many of you all are aware, Colin served the Project as the Security Officer for several years, and authored important utilities used by many within the community, such as portsnap and freebsd-update. I have sent and received acknowledgement from the Core Team of this decision to step aside, which was well received and no objection to me selecting Colin to be my successor had been posed. Colin, thank you, again, for overtaking this role, and as we discussed, I will be here to provide any needed guidance, insight, or information as you need. As I had mentioned to you, I do not have any immediate plans to leave the Team, nor do I have any question in my mind that you are the obvious successor to this role. I am proud to be able to hand over the proverbial torch to you, and look very much forward to your successes as the Release Engineering Team Lead. It has been an absolute pleasure to be able to serve you all to the best of my abilities over the past several years. With my very best regards to all involved in the Project, and nothing but the best intentions of the Project in mind. Glen -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEjRJAPC5sqwhs9k2jAxRYpUeP4pMFAmVX/MgACgkQAxRYpUeP 4pPLVg/9HW6nhw6EXTGqkrdibCkThurdasCqftkLSGrsABDj1AsErHbBnbO3W04G KSVZ6Jw28JQNGUsprJwcAsJre8PXgL2iKOkIZTL31CYSBvQbfGEzIWTC+18oTQ6Q d1uxqGcqoMcpPfFxSry8wnfxO7hYGabv38lPy2KipskCqq/3zVqsdVb6UxGdo+5Q 0sis6OVh8KhNdcgOu+RTZr194KJogsdqqGll4pOQNMv/Qpy5e9JOqHgVXueseUC8 6GDHa990VLF22+DLgq3qzVS0xGLIOM5MuC0EjWJEjaq5bANEzQX0SvMEKQEbGLrI MjBXGoJGTreBMr0VbCewhuDIiYy7DU5dtohvXvCx+UoBryT9qU1RKmRyRxxp3hQ4 dcZUyZ3RarwmZfJBTs7ib2s9LztUlpyIRAU7gJ2vRKcegVTH/Fe1FuRZHOFHZf46 kzHeIygER3nitPtJj9ePQgq97F2WBjRDZ5MSzwNWsOMknsbUKekIB9xshbhxlUw8 5ZN61k3frKM/dOcnLulsNHFaMGJaodDsxMuTKhJCEHvHWLXOin66WrfuHKbeqzVf LDgMCWiHcM0GwrtzXR/aITVZmwyg096gZw0NLXtlNadY+PyQrg8R83DmOKTk0fWo MYaMnWBcZM78DviTaBPhcblMtB5/IqrbJzqnJqsnCBEW2Z29qgs= =sYrP -----END PGP SIGNATURE----- From nobody Sat Nov 18 03:47:47 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SXKVH3BrRz51jdb for ; Sat, 18 Nov 2023 03:47:51 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from omta001.cacentral1.a.cloudfilter.net (omta001.cacentral1.a.cloudfilter.net [3.97.99.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SXKVG4dR0z4qmp for ; Sat, 18 Nov 2023 03:47:50 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Authentication-Results: mx1.freebsd.org; dkim=none; spf=none (mx1.freebsd.org: domain of cy.schubert@cschubert.com has no SPF policy when checking 3.97.99.32) smtp.mailfrom=cy.schubert@cschubert.com; dmarc=none Received: from shw-obgw-4002a.ext.cloudfilter.net ([10.228.9.250]) by cmsmtp with ESMTPS id 3z5ariQ3u8jpT4CJBrlOPW; Sat, 18 Nov 2023 03:47:49 +0000 Received: from spqr.komquats.com ([70.66.152.170]) by cmsmtp with ESMTPSA id 4CJArvD5OnCF04CJBrevT7; Sat, 18 Nov 2023 03:47:49 +0000 X-Authority-Analysis: v=2.4 cv=MPFzJeVl c=1 sm=1 tr=0 ts=655833e5 a=y8EK/9tc/U6QY+pUhnbtgQ==:117 a=y8EK/9tc/U6QY+pUhnbtgQ==:17 a=kj9zAlcOel0A:10 a=BNY50KLci1gA:10 a=YxBL1-UpAAAA:8 a=6I5d2MoRAAAA:8 a=EkcXrb_YAAAA:8 a=KWo1sG5w0cvzccTm5TYA:9 a=CjuIK1q_8ugA:10 a=Ia-lj3WSrqcvXOmTRaiG:22 a=IjZwj45LgO3ly-622nXo:22 a=LK5xJRSDVpKd5WXXoEvA:22 Received: from slippy.cwsent.com (slippy [10.1.1.91]) by spqr.komquats.com (Postfix) with ESMTP id E504DA8 for ; Fri, 17 Nov 2023 19:47:47 -0800 (PST) Received: by slippy.cwsent.com (Postfix, from userid 1000) id B044210F; Fri, 17 Nov 2023 19:47:47 -0800 (PST) X-Mailer: exmh version 2.9.0 11/07/2018 with nmh-1.8+dev Reply-to: Cy Schubert From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.cschubert.com/ To: freebsd-current@freebsd.org Subject: autofs -hosts maps List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 17 Nov 2023 19:47:47 -0800 Message-Id: <20231118034747.B044210F@slippy.cwsent.com> X-CMAE-Envelope: MS4xfF0ZSYRwa2XMNqN++HOUyiZKYqiYHwt1nKnrC3ol8pRqH0GEJMzkErPfTgiSGEfgcC5KIl9vz3V3gLGsBMk2u/kSg7ZEiIIjzL3El7mwPanVm1D/0ZJa W+haMX/LPJnIsFi5dH5at7ShfX1JWDP7g5PNmms7oC10ZaBWMt5NFe5PCbFmh+CReDDn+KXEzHhNzPuVaUfuURRCp4F0UpcMRzM= X-Spamd-Result: default: False [-1.79 / 15.00]; AUTH_NA(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.985]; MV_CASE(0.50)[]; RCVD_IN_DNSWL_LOW(-0.10)[3.97.99.32:from]; MIME_GOOD(-0.10)[text/plain]; RWL_MAILSPIKE_GOOD(-0.10)[3.97.99.32:from]; FROM_HAS_DN(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; REPLYTO_EQ_FROM(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DMARC_NA(0.00)[cschubert.com]; RCPT_COUNT_ONE(0.00)[1]; HAS_REPLYTO(0.00)[Cy.Schubert@cschubert.com]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_COUNT_THREE(0.00)[4]; TO_DN_NONE(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:16509, ipnet:3.96.0.0/15, country:US]; R_SPF_NA(0.00)[no SPF record] X-Rspamd-Queue-Id: 4SXKVG4dR0z4qmp X-Spamd-Bar: - Hi, The discussion about NFS exports of ZFS snapshots prompted me to play around with -hosts maps on my network. -hosts maps are mounted on /net. I've discovered that -hosts maps don't work with most shares but do with others. I've only played with this for a few minutes so I don't fully understand why some maps work and others not. Some of underlying directories that don't work are ZFS while others are UFS. Yet, auto_home maps mounting the same directories does work. And mounting the shares by hand (using mount_nfs) also works. Just putting this out there should someone else have noticed this. I'll play around with this a little over the weekend. -- Cheers, Cy Schubert FreeBSD UNIX: Web: https://FreeBSD.org NTP: Web: https://nwtime.org e^(i*pi)+1=0 From nobody Sat Nov 18 06:13:31 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SXNl02Lxnz51qM6 for ; Sat, 18 Nov 2023 06:14:04 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.116.210]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SXNky6KwGz3KNw for ; Sat, 18 Nov 2023 06:14:02 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=cs.huji.ac.il; s=57791128; h=References:To:Cc:In-Reply-To:Date:Subject:Mime-Version:Content-Type:Message-Id:From; bh=DoXgNjUy2yuJgy5WJzddj4SnLWmsoqPbcQWr/WOMDn8=; b=oatpGvsaJeqa+WWM1K522Te+PyY+zVUkp4UNEqYIOLOd+Wz9jSBtut+/BBekfuv5BkFYm3+nh/F0PQb9McVapAwm6M5LPsZxYeoyOdQG+O/hi83DaIpsj5FJyqhBQbfKwfz4hCcSKrxYHbiWW2kKS23rR1zvFQnwtaZ+ixKF32mK27jED1FfbAJS1AfxMw7/Kc6O9A4CwBRyd3MLp9gDxcFOBEHQTqf9d4aAeHnVnNBwSWg7hbsSJg071MMT0J/bytTE4BZZ1qoAk8A/iQho9b2M3hCEVdR1FhjKY55J2guhJ2Xa/pND7KdAGa+rAvh/ZegS+wFYsYkOloLO1rrmow==; Received: from imac.bk.cs.huji.ac.il ([132.65.179.42] helo=smtpclient.apple) by kabab.cs.huji.ac.il with esmtp id 1r4EaV-000N1Q-NI; Sat, 18 Nov 2023 08:13:51 +0200 From: Daniel Braniss Message-Id: <0837C3D3-AD50-4AA9-996E-2111C28CA2C9@cs.huji.ac.il> Content-Type: multipart/alternative; boundary="Apple-Mail=_5922A834-C00B-4838-9891-4B576339DA31" List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.600.7\)) Subject: Re: autofs -hosts maps Date: Sat, 18 Nov 2023 08:13:31 +0200 In-Reply-To: <20231118034747.B044210F@slippy.cwsent.com> Cc: freebsd-current@freebsd.org To: Cy Schubert References: <20231118034747.B044210F@slippy.cwsent.com> X-Mailer: Apple Mail (2.3731.600.7) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:378, ipnet:132.64.0.0/15, country:IL] X-Rspamd-Queue-Id: 4SXNky6KwGz3KNw --Apple-Mail=_5922A834-C00B-4838-9891-4B576339DA31 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On 18 Nov 2023, at 5:47, Cy Schubert = wrote: >=20 > Hi, >=20 > The discussion about NFS exports of ZFS snapshots prompted me to play=20= > around with -hosts maps on my network. -hosts maps are mounted on = /net. >=20 > I've discovered that -hosts maps don't work with most shares but do = with=20 > others. I've only played with this for a few minutes so I don't fully=20= > understand why some maps work and others not. Some of underlying=20 > directories that don't work are ZFS while others are UFS. >=20 > Yet, auto_home maps mounting the same directories does work. And = mounting=20 > the shares by hand (using mount_nfs) also works. >=20 > Just putting this out there should someone else have noticed this. >=20 > I'll play around with this a little over the weekend. >=20 >=20 it=E2=80=99s subdirectories that don=E2=80=99t work? if so it=E2=80=99s a bug/feature of -hosts. danny >=20 > --=20 > Cheers, > Cy Schubert > FreeBSD UNIX: Web: https://FreeBSD.org > NTP: Web: https://nwtime.org >=20 > e^(i*pi)+1=3D0 >=20 >=20 > =C2=80=C2=80=C2=80=C2=80=C2=80=C2=80=C2=80=C2=80 >=20 Daniel Braniss danny@cs.huji.ac.il --Apple-Mail=_5922A834-C00B-4838-9891-4B576339DA31 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8

On 18 = Nov 2023, at 5:47, Cy Schubert <Cy.Schubert@cschubert.com> = wrote:

Hi,

The discussion = about NFS exports of ZFS snapshots prompted me to play
around with = -hosts maps on my network. -hosts maps are mounted on /net.

I've = discovered that -hosts maps don't work with most shares but do with =
others. I've only played with this for a few minutes so I don't = fully
understand why some maps work and others not. Some of = underlying
directories that don't work are ZFS while others are = UFS.

Yet, auto_home maps mounting the same directories does work. = And mounting
the shares by hand (using mount_nfs) also = works.

Just putting this out there should someone else have = noticed this.

I'll play around with this a little over the = weekend.



it=E2=80=99s = subdirectories that don=E2=80=99t work?
if so it=E2=80=99s a = bug/feature of = -hosts.

danny


--
Cheers,
Cy Schubert = <Cy.Schubert@cschubert.com>
FreeBSD UNIX: =  <cy@FreeBSD.org>   Web: =  https://FreeBSD.org
NTP: =           <cy@nwtime.= org>    Web:  https://nwtime.org

= e^(i*pi)+1=3D0


=C2=80=C2=80=C2=80=C2=80=C2=80=C2=80=C2=80= =C2=80


Daniel = Braniss
danny@cs.huji.ac.il



= --Apple-Mail=_5922A834-C00B-4838-9891-4B576339DA31-- From nobody Sat Nov 18 08:50:43 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SXSCs60ZTz50knw for ; Sat, 18 Nov 2023 08:50:49 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from drew.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.franken.de", Issuer "Sectigo RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SXSCs3tYnz3Vr8 for ; Sat, 18 Nov 2023 08:50:49 +0000 (UTC) (envelope-from tuexen@freebsd.org) Authentication-Results: mx1.freebsd.org; none Received: from smtpclient.apple (unknown [IPv6:2a00:20:7049:7b36:f457:6ae2:608a:64d5]) (Authenticated sender: micmac) by mail-n.franken.de (Postfix) with ESMTPSA id 5D79D721E280D; Sat, 18 Nov 2023 09:50:44 +0100 (CET) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.200.91.1.1\)) Subject: Re: Request for Testing: TCP RACK From: tuexen@freebsd.org In-Reply-To: <20231118083704.3ebc17eb8f392ceae33f65f7@dec.sakura.ne.jp> Date: Sat, 18 Nov 2023 09:50:43 +0100 Cc: freebsd-current@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <42C327BD-6CE4-43AA-A1AE-3BEC08D623DB@freebsd.org> <2bbfcf68-3249-45c7-a8e5-af7115a1ccfe@gmail.com> <20231118083704.3ebc17eb8f392ceae33f65f7@dec.sakura.ne.jp> To: Tomoaki AOKI X-Mailer: Apple Mail (2.3774.200.91.1.1) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, T_SCC_BODY_TEXT_LINE autolearn=disabled version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail-n.franken.de X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:680, ipnet:2001:638::/32, country:DE] X-Rspamd-Queue-Id: 4SXSCs3tYnz3Vr8 > On Nov 18, 2023, at 00:37, Tomoaki AOKI = wrote: >=20 > On Fri, 17 Nov 2023 18:51:05 +0100 > tuexen@freebsd.org wrote: >=20 >>> On Nov 17, 2023, at 17:06, Johan Hendriks = wrote: >>>=20 >>> I am running the rack stack for quiet some time now on a baremetal = machiene and never had problems. >>> Also use pf. This is a test machine so not a lot happening on it. >>>=20 >>> Are there any thing we can test? Do we have some test scripts we can = run? >> We are actually interested in feedback about using the stack in = whatever >> use case you use TCP for. The stack has been tested with the Netflix = use >> case, but not so much with others. That is why we ask for broader = testing. >>=20 >> Best regards >> Michael >=20 > Are there any difference regarding with performance between main and > stable/14? If so, please ignore below. >=20 > I have stable/14 environment which is configured to be able to switch > to TCP RACK and experienced huge performance loss when writing > a large file to smbfs share on commercial NAS. CC is cubic. > Testing large archive on the smbfs share doesn't seem to be > affected. >=20 > Comparison between default (freebsd) and rack TCP stack using > sysutils/clone on stable/14 at commit 7d1321288ad9, amd64. > Last 3 lines of outputs from clone (upload to NAS) are shown. Thank you very much for testing. This is what we are looking for. Would it be possible to repeat the test using NewReno as the CC? Best regards Michael >=20 > Before switching to rack: > 1 item copied, 2342.4 MB in 39.12 s -- 59.9 MB/s > Leaked memory: 0 bytes > No errors occured. >=20 > Unmount the smbfs share, switch to rack, and after remount: > 1 item copied, 2342.4 MB in 926.59 s -- 2.5 MB/s > Leaked memory: 0 bytes > No errors occured. >=20 > Switch back to freebsd (default) without unmounting: > 1 item copied, 2342.4 MB in 906.94 s -- 2.6 MB/s > Leaked memory: 0 bytes > No errors occured. >=20 > Unmount and remount the smbfs share: > 1 item copied, 2342.4 MB in 39.12 s -- 59.9 MB/s > Leaked memory: 0 bytes > No errors occured. >=20 >=20 > Regards. >=20 > --=20 > Tomoaki AOKI From nobody Sat Nov 18 10:48:20 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SXVqX53Lzz50wcc for ; Sat, 18 Nov 2023 10:48:24 +0000 (UTC) (envelope-from hans@beastielabs.net) Received: from ewsoutbound.kpnmail.nl (ewsoutbound.kpnmail.nl [195.121.94.185]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SXVqW2ln0z4KSn for ; Sat, 18 Nov 2023 10:48:23 +0000 (UTC) (envelope-from hans@beastielabs.net) Authentication-Results: mx1.freebsd.org; none X-KPN-MessageId: eb29f208-85ff-11ee-abd3-005056999439 Received: from smtp.kpnmail.nl (unknown [10.31.155.6]) by ewsoutbound.so.kpn.org (Halon) with ESMTPS id eb29f208-85ff-11ee-abd3-005056999439; Sat, 18 Nov 2023 11:47:50 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kpnmail.nl; s=kpnmail01; h=content-type:from:to:subject:mime-version:date:message-id; bh=7JlSjf+Zthlidriz1W7e3LJwR+whWqRWgWNr7SodaPo=; b=gPQziTfjSItXRimbx+F7KsDlYiMTJc9jtK/PTk0T5CfLqoE1q7R0sjQ+AdnHk6oCLAkR8oBy0h27e ZBwEAwiJQwOHXOzbpuE9SjOGQZWqyEN/ZF3Jy07wPaaZSYgloiggqwyQ03KbXaEojG09A/kVcaQoK4 WK02RcKkoFMSJ/4M= X-KPN-MID: 33|UwOMSy4BXKmS5bcldId59wmFgqdZW6pS1KrRGS5oSJUuY4U5uOkGd8hiWzyC42k Tcge7etac1aLNlkLqR0w+vx360IlqvKkVWh/n6KK8y2o= X-KPN-VerifiedSender: No X-CMASSUN: 33|x2oJNuEDxzY7Yp5Ko4k938ezosjQyOEYotNpkWJMA7rGnCHgPhs13t1okPiRUpI dNc9pcQOJ9W4MuamxdWf9qw== X-Originating-IP: 77.171.212.158 Received: from [192.168.66.163] (77-171-212-158.fixed.kpn.net [77.171.212.158]) by smtp.xs4all.nl (Halon) with ESMTPSA id fd1d8830-85ff-11ee-9dc8-00505699772e; Sat, 18 Nov 2023 11:48:20 +0100 (CET) Message-ID: <42673c74-c76c-4eef-be95-47a910eb95a8@beastielabs.net> Date: Sat, 18 Nov 2023 11:48:20 +0100 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: autofs -hosts maps Content-Language: en-US To: Daniel Braniss References: <20231118034747.B044210F@slippy.cwsent.com> <0837C3D3-AD50-4AA9-996E-2111C28CA2C9@cs.huji.ac.il> From: Hans Ottevanger Cc: Current FreeBSD , Cy.Schubert@cschubert.com In-Reply-To: <0837C3D3-AD50-4AA9-996E-2111C28CA2C9@cs.huji.ac.il> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:8737, ipnet:195.121.64.0/18, country:NL] X-Rspamd-Queue-Id: 4SXVqW2ln0z4KSn On 11/18/23 07:13, Daniel Braniss wrote: > > >> On 18 Nov 2023, at 5:47, Cy Schubert wrote: >> >> Hi, >> >> The discussion about NFS exports of ZFS snapshots prompted me to play >> around with -hosts maps on my network. -hosts maps are mounted on /net. >> >> I've discovered that -hosts maps don't work with most shares but do with >> others. I've only played with this for a few minutes so I don't fully >> understand why some maps work and others not. Some of underlying >> directories that don't work are ZFS while others are UFS. >> >> Yet, auto_home maps mounting the same directories does work. And mounting >> the shares by hand (using mount_nfs) also works. >> >> Just putting this out there should someone else have noticed this. >> >> I'll play around with this a little over the weekend. >> >> > > it’s subdirectories that don’t work? > if so it’s a bug/feature of -hosts. > > danny > >> >> -- >> Cheers, >> Cy Schubert >> FreeBSD UNIX:     Web:  https://FreeBSD.org >> NTP:               Web:  https://nwtime.org >> >> e^(i*pi)+1=0 >> >> >> €€€€€€€€ >> > > Daniel Braniss > danny@cs.huji.ac.il > > > Hi, Could it be related to the issue in the (old) discussion starting with this message: https://lists.freebsd.org/pipermail/freebsd-current/2014-September/051969.html -- Kind regards, Hans Ottevanger Eindhoven, Netherlands hans@beastielabs.net www.beastielabs.net From nobody Sat Nov 18 16:09:34 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SXdyM4NcGz51J0c for ; Sat, 18 Nov 2023 16:09:47 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Received: from mail-pj1-x1033.google.com (mail-pj1-x1033.google.com [IPv6:2607:f8b0:4864:20::1033]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SXdyL3SjDz3MCN; Sat, 18 Nov 2023 16:09:46 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=GCgNzbV3; spf=pass (mx1.freebsd.org: domain of rick.macklem@gmail.com designates 2607:f8b0:4864:20::1033 as permitted sender) smtp.mailfrom=rick.macklem@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pj1-x1033.google.com with SMTP id 98e67ed59e1d1-2802e5ae23bso2613125a91.2; Sat, 18 Nov 2023 08:09:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700323785; x=1700928585; darn=freebsd.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=4opcVmtRK/B77y0nsISDw1smlbvyYKU4ccDLJgR+TXw=; b=GCgNzbV33aY8aJpo0yocB0ISg9ovL1uMVdBvpomuQJ4oBldO0jfw9TEfkhVNB0xtil SlgeaPKiyJp9pdwOD0ejjaPPgxjYN6HcGPhWNbaFhxa03jWZwvHwkvpy5WpCo4xrt97f IsV73faP63/VfhwVCNNYlMkFTxnRq4dIPvKrDJXK+Z7gOsGKyG9+2jGRcnQsheyISkxl V8vVudgY5FC/4neP9pVSumRoczRJulSz6YcrkqiZKNVI/MzYLGDCYI+YUk6DNtgkBxKj kVP/G+OHhnvaUjkrgXMp8GI0RommdFR3Nq7iWQOJ6osMzS6B1soIECt9NtGB54eDLTEx CxkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700323785; x=1700928585; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4opcVmtRK/B77y0nsISDw1smlbvyYKU4ccDLJgR+TXw=; b=cddRDcaVOrIPvnd3xar55hOy8cDSeqCAOJ7akCz4ueVJax1o23iVxfF/CucfCZywBv IFzz8uNBw54ZeaQyboFr2d/NmDmviYvA/suZ1xdSHapPLInMWJorpPhj7Y3z4Hlc6pHl BCfX5QXtmPRGSeFerlEBS3v0grJmFTlt6Z0Xw9KUo4FIsBd283qJy0jaVWVSCcCBspHw 1fbJDgkkZj8oqIRKDoEPK2G4LnzVuSN+YbxkkF+ndUeJFLn5myqC0bGiMADWILAaUq8W VmaL0SQT4wlSA8h7W3CT4KT+MTVmSDdX3c5hRCKy2deZ4vCW17tULbnHc/Ev69WiB1jP J1DQ== X-Gm-Message-State: AOJu0YzifGxNtBdah7HtrEKoDySjXEBkaoJqi9ZZ/bw+pzURP0RCe231 3l35BBbIyiMv11RbQMB+OV1HbniHpNFWmiUDoIG/QtQ= X-Google-Smtp-Source: AGHT+IGXH4atTi4Ef+gVb7PekLMg9Euwr2M1XEqkjwORLAgsvUxxjxflYHvtE9hnJ5gQT0FZz9tP/acOCAgj//8Jywc= X-Received: by 2002:a17:90b:1c07:b0:280:235:19d with SMTP id oc7-20020a17090b1c0700b002800235019dmr2720118pjb.36.1700323784865; Sat, 18 Nov 2023 08:09:44 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: <25943.60056.880614.452966@hergotha.csail.mit.edu> In-Reply-To: From: Rick Macklem Date: Sat, 18 Nov 2023 08:09:34 -0800 Message-ID: Subject: Re: NFS exports of ZFS snapshots broken To: Mike Karels , FreeBSD CURRENT Cc: Rick Macklem , Garrett Wollman , Alexander Motin , Martin Matuska Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_COUNT_ONE(0.00)[1]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::1033:from]; DKIM_TRACE(0.00)[gmail.com:+]; TAGGED_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_FIVE(0.00)[6]; TO_DN_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-Rspamd-Queue-Id: 4SXdyL3SjDz3MCN X-Spamd-Bar: --- On Fri, Nov 17, 2023 at 8:19=E2=80=AFPM Mike Karels wrote= : > > On 17 Nov 2023, at 22:14, Mike Karels wrote: > > > On 17 Nov 2023, at 21:24, Rick Macklem wrote: > > > >> Most of the changes in stable/13 that are not in releng/13.2 > >> are the "make it work in a jail" stuff. Unfortunately, they are > >> a large # of changes (mostly trivial edits adding vnet macros), > >> but it also includes export check changes. > >> > >> I have attached a trivial patch that I think disables the export > >> checks for jails. If either of you can try it and see if it fixes > >> the problem, that would be great. > >> (Note that this is only for testing, although it probably does not > >> matter unless you are running nfsd(8) in vnet jails.) > > > > Yes, I can see snapshots with the patch. This system is just a test > > system that doesn't normally run ZFS or NFS, so no problem messing > > with permissions. It's a bhyve VM, so I just added a small disk and > > enabled ZFS for testing. > > btw, you might try to get mm@ or maybe mav@ to help out from the ZFS > side. It must be doing something differently inside a snapshot than > outside, maybe with file handles or something like that. Yes. I've added freebsd-current@ (although Garrett is not on it, he is cc'd) and these guys specifically... So, here's what appears to be the problem... Commit 88175af (in main and stable/13, but not 13.2) added checks for nfsd(8) running in jails by filling in mnt_exjail with a reference to the c= red used when the file system is exported. When mnt_exjail is found NULL, the current nfsd code assumes that there is no access allowed for the mount. My vague understanding is that when a ZFS snapshot is accessed, it is "pseudo-mounted" by zfsctl_snapdir_lookup() and I am guessing that mnt_exjail is NULL as a result. Since I do not know the ZFS code and don't even have an easy way to test this (thankfully Mike can test easily), I do not know what to do from here? Is there a "struct mount" constructed for this pseudo mount (or it actually appears to be the lookup of ".." that fails, so it might be the parent of the snapshot subdir?)? One thought is that I can check to see if the mount pointer is in the mountlist (I don't think the snapshot's mount is in the mountlist) and avoid the jail test for this case. This would assume that snapshots are always within the file system(s) exported via that jail (which includes the case of prison0, of course), so that they do not need a separate jail check. If this doesn't work, there will need to be some sort of messing about in ZFS to set mnt_exjail for these. I will try and get a test setup going here, which leads me to.. how do I create a ZFS snapshot? (I do have a simple ZFS pool running on a test machine, but I've never done a snapshot.) Although this problem is not in 13.2, it will have shipped in 14.0. Any help with be appreciated, rick > > Mike > > > >> rick > >> > >> On Fri, Nov 17, 2023 at 6:14=E2=80=AFPM Mike Karels = wrote: > >>> > >>> CAUTION: This email originated from outside of the University of Guel= ph. Do not click links or open attachments unless you recognize the sender = and know the content is safe. If in doubt, forward suspicious emails to ITh= elp@uoguelph.ca. > >>> > >>> > >>> Rick, have you been following this thread on freebsd-stable? I have = been able > >>> to reproduce this using a 13-stable server from Oct 7 and a 15-curren= t system > >>> that is up to date using NFSv3. I did not reproduce with a 13.2 serv= er. The > >>> client was running 13.2. Any ideas? A full bisect seems fairly pain= ful, but > >>> maybe you have an idea of points to try. Fortunately, these are all = test > >>> systems that I can reboot at will. > >>> > >>> Mike > >>> > >>> Forwarded message: > >>> > >>>> From: Garrett Wollman > >>>> To: Mike Karels > >>>> Cc: freebsd-stable@freebsd.org > >>>> Subject: Re: NFS exports of ZFS snapshots broken > >>>> Date: Fri, 17 Nov 2023 17:35:04 -0500 > >>>> > >>>> < = said: > >>>> > >>>>> I have not run into this, so I tried it just now. I had no problem= . > >>>>> The server is 13.2, fully patched, the client is up-to-date -curren= t, > >>>>> and the mount is v4. > >>>> > >>>> On my 13.2 client and 13-stable server, I see: > >>>> > >>>> 25034 ls CALL open(0x237d32f9a000,0x120004) > >>>> 25034 ls NAMI "/mnt/tools/.zfs/snapshot/weekly-2023-45" > >>>> 25034 ls RET open 4 > >>>> 25034 ls CALL fcntl(0x4,F_ISUNIONSTACK,0x0) > >>>> 25034 ls RET fcntl 0 > >>>> 25034 ls CALL getdirentries(0x4,0x237d32faa000,0x1000,0x237d= 32fa7028) > >>>> 25034 ls RET getdirentries -1 errno 5 Input/output error > >>>> 25034 ls CALL close(0x4) > >>>> 25034 ls RET close 0 > >>>> 25034 ls CALL exit(0) > >>>> > >>>> Certainly a libc bug here that getdirentries(2) returning [EIO] > >>>> results in ls(1) returning EXIT_SUCCESS, but the [EIO] error is > >>>> consistent across both FreeBSD and Linux clients. > >>>> > >>>> Looking at this from the RPC side: > >>>> > >>>> (PUTFH, GETATTR, LOOKUP(snapshotname), GETFH, GETATTR) > >>>> [NFS4_OK for all ops] > >>>> (PUTFH, GETATTR) > >>>> [NFS4_OK, NFS4_OK] > >>>> (PUTFH, ACCESS(0x3f), GETATTR) > >>>> [NFS4_OK, NFS4_OK, rights =3D 0x03, NFS4_OK] > >>>> (PUTFH, GETATTR, LOOKUPP, GETFH, GETATTR) > >>>> [NFS4_OK, NFS4_OK, NFS4ERR_NOFILEHANDLE] > >>>> > >>>> and at this point the [EIO] is returned. > >>>> > >>>> It seems that clients always do a LOOKUPP before calling READDIR, an= d > >>>> this is failing when the subject file handle is the snapshot. The > >>>> client is perfectly able to *traverse into* the snapshot: if I try t= o > >>>> list a subdirectory I know exists in the snapshot, the client is abl= e to > >>>> LOOKUP(dirname) just fine, but LOOKUPP still fails with > >>>> NFS4ERR_NOFILEHANDLE *on the subndirectory*. > >>>> > >>>> -GAWollman > >>> From nobody Sat Nov 18 18:35:16 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SXjBQ6tXXz51S6S; Sat, 18 Nov 2023 18:35:26 +0000 (UTC) (envelope-from zlei@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SXjBQ6PSdz3cMB; Sat, 18 Nov 2023 18:35:26 +0000 (UTC) (envelope-from zlei@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700332526; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Nm5hXl77sQqNNSlm4+XGZ7JYr8GOwZJC3kEyC3oFaFk=; b=okZ1BkSGOPZxpzLyn7PY99ezlYFHWQV8tx4FzVmvt2uq+G6avlqKgn1v6B2xzC6HENyTUF U0PWDOXv4I0UIHIoIqFbZ7LyZHdqyrIV4iFjvIeB5iI9qsPVpv6Ray5FDQVk5/wW56vT+u beJFHri4O+iZPHKy6JsMyWvy1R8RqaYbBzvR+k+n8qRLPRl8jkJpDPpIDdfUBgbBmeXifc zbdHdE5g7nYRlxwNJfg2CdMnxeAYKs9b+7NNJji4tYVfqQXptX4+TXOU1E0DB7IJc5+2il ExdFagDF6xJiphK8CEpW9Grrg1ZIIOye6fOre4pfdGbTECTvftbUWYU/E/vg5g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700332526; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Nm5hXl77sQqNNSlm4+XGZ7JYr8GOwZJC3kEyC3oFaFk=; b=ZyrewlmDvOSg2CzNoCNBxgYtlLgADhiLGguCHhn/gKgnZEgMPpb+pN6n7/taGzJ0v5umQI fdullKUElO0DpYqgGRxSHhbQmPlPUObCYuJNkeWrvK3UVkKAgUWKISPwFxaKat3AJUDeQz nUjuNNcjltwfT+evu0oRjF3mOHaHALhObakzaBjalNxCdM1FaXB+/XfAV64tFFNm5OBwvQ Jx1UfZyWCcQK+9wYX8SfaXCE7oK1vFERDDO3lF9V30XglPnb+LZHYQGQpDL5bHw7l7HdqK jwuGTZ1pgdgnbQ7l0DgMz48mcDQ7CYC+a3P4jzLP2RYczpDZTdfXa1uxPvV3zg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1700332526; a=rsa-sha256; cv=none; b=Tpg6AGzNQUbwiqJ45fspgZS4t8GJDeRnQlIK7+RPkpnm1NapZy+mVj9KIUHEHZXSyG0DJZ pwMqcuRn08h1YG5VC6em2nXKGc2fR06L+/shFBa2PGMC/GroYhbpYSOGOfWoGFv5LRm8fT 4wrMrKmB68+z5UxTkZuFfyiCDYMGeqdabdQTDug//4AG1IGnWm2D//LDfB65nh1KO2ZkC6 UHKaAg7s6amjl+DvJxTLVex+CKmcOt1XeU9wJXWrP+PMtaSR+DpldjtuO5GH0AYFFSpRSL V3QRbCb4FcB/aP/vBeSn9NtBpySmzUYGTD4chGj5avBtJwdzovwmdbvEqYcdPg== Received: from smtpclient.apple (ns1.oxydns.net [45.32.91.63]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: zlei/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4SXjBP4rKVz10tK; Sat, 18 Nov 2023 18:35:25 +0000 (UTC) (envelope-from zlei@FreeBSD.org) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.4\)) Subject: Re: Request for Testing: TCP RACK From: Zhenlei Huang In-Reply-To: <42C327BD-6CE4-43AA-A1AE-3BEC08D623DB@freebsd.org> Date: Sun, 19 Nov 2023 02:35:16 +0800 Cc: current@freebsd.org, net@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <941203D6-3A72-48A4-8C2C-F59C964199A9@FreeBSD.org> References: <42C327BD-6CE4-43AA-A1AE-3BEC08D623DB@freebsd.org> To: tuexen@freebsd.org X-Mailer: Apple Mail (2.3696.120.41.1.4) > On Nov 16, 2023, at 5:13 PM, tuexen@freebsd.org wrote: >=20 > Dear all, >=20 > recently the main branch was changed to build the TCP RACK stack > which is a loadable kernel module, by default: > = https://cgit.FreeBSD.org/src/commit/?id=3D3a338c534154164504005beb00a3c6fe= b03756cc >=20 > As discussed on the bi-weekly transport call, it would be great if = people > could test the RACK stack for their workload. Please report any = problems to the > net@ mailing list or open an issue in the bug tracker and drop me a = note via email. > This includes regressions in CPU usage, regressions in performance or = any other > unexpected change you observe. I see some performance regression with the rack stack. This is iperf3 test on local host (bare metal, an old i5 2 cores 4 = threads MBP). freebsd: 16.1 Gbits/sec rack: 12.3 Gbits/sec The congestion control algorithm is cubic. Note this is a quick test with DEBUG options enabled. I'll try with no debug options and report later. >=20 > You can load the kernel module using > kldload tcp_rack >=20 > You can make the RACK stack the default stack using > sysctl net.inet.tcp.functions_default=3Drack >=20 > Based on the feedback we get, the default stack might be switched to = the > RACK stack. >=20 > Please let me know if you have any questions. >=20 > Best regards > Michael >=20 >=20 >=20 Best regards, Zhenlei From nobody Sat Nov 18 18:46:25 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SXjRT1Mvpz51Sg5 for ; Sat, 18 Nov 2023 18:46:45 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic316-54.consmr.mail.gq1.yahoo.com (sonic316-54.consmr.mail.gq1.yahoo.com [98.137.69.30]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SXjRS0lxlz3gcZ for ; Sat, 18 Nov 2023 18:46:44 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=Wx+9yLLD; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.30 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1700333199; bh=NgD1bpEbkY0keLCtMk9l9mJ8Ll95McafwEpTTIZvK7c=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=Wx+9yLLDZuciLRnFahMaaI0XIgjx9sR5On4LEc6RWibL/eqNF4BrKE+DEIla7wgGqhNAXzAx4Uyb9OKg0vs3tg+z+0St1loNSGDX5gWQ2IJw5ud1yW+WNeEOONYeOxwGnydkPZov5h5R4xgo2wN4CRmPpDRIc6b8U65nkLXx7ez51xvvlt2/xqtbYvqOgyE5MTJ/BJ9pjInMWxOk8CwRcUKvWTcOjgd7k8WQGmYzHOQx5/LCchhJwB1WEwlEdSxc7ujfinH5PgjruAugIXerAxk4fJci7Y3b6uaBVcQ4Jd9OqvdkkdPVeJOTuYzqfzJv3NRSgrSHimwa1AZzne2ELg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1700333199; bh=i+LPKROUvhza6AD2Wcn4q+3fok4F/JAmJ8KOCfkJYlz=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=bcSEddf+ajUJONXMc7MYXZxHt3jL1UHR0Tb8hzhwWO+03h6VE0Tt05CBg7C5Glppt0jIBKwV2NwSbzAkIL+hxHa8richavTQinx2QLs7W/bvdb+cDyBEG/DRgAsD4T9DJhBrKrPzvamUioXb44mH+GwDtoMFZY4hgmO0/7VV9MG0d0lJMNXs5Q8p8chp4VRYn2OGti++hrq7cBK5Ha/YdBVxPzIbHaxQJhWMa5QNza32nkhgnSfh6UhUMcD8hQvuOjqVWDs4Blio4+T+e7uF6k0FO/gsAbbKhrX38oH0L4FX3ZdWUaaUAPqATkfUpOGofhXsP0hhGRQVUDLiwAIN3w== X-YMail-OSG: y1rpDmIVM1neYXQ8Wt17ZyzT48zqsgv0nR0gdj1cScyFvorP8paLSggd0h.bUWX 9Yifwvcg12utT1KiknU_xH1x4kbN1xl8ijQ1jjmnRAIl_4nnr8SOt3oJ00YCVqTEIhoaWHy4SIo. tgDk_i2IaHZ7c3FCvjfhl4UZKyrJbYxvRzK8K9Btz2KJrMkeAYjecarSJ5iTQdOTHfkP2OCMyn5W M6yQ1yvL8sNCaEmSOXbvAvo9ieRZNQ4ELoyhIkv.5GNb.GhJRk9R6Mm51TmNRAraZcMEAbI8hBgb tl3jL0prqSslSVYlJtPmrCJIQ5T48TxbGc60LOgMLHZDQFkrmrMDs9F87t.1FsyOFgSCtr6asIbK sYI4OtMNeAMGSnUgcYUHuMDx4pqsfB0KGB20CbqzYt3tTErz_.kOHbGc1nvKnOTAZpC62PePFvnC J7Yn2Jojvhjza9TtKtesvB3eISkhuqnOu8zhBzk0BP7vQwUfSmQLgi5KjuM0dHUHbhjmnxdoLRm_ aVrFI2HdH47vMvlPCiP2hldkMaSQhy_cuLjZG51LHOe7I59RyymwdgXQ1nwFF_Q30Y.USFAm5l.2 DpOjINL0RfI3FANAXiCzI.J9wDUuoDZ9TrItzH0OPkLqMEBMWEIP9xd65Fd2Dwnge111wUTPaXbl OOvFVnMi7FejHxENJSR.KSjRtLQdlXJeo4o06psie4Zumd88ZeJiowkGeGuOnTliZcey6Pa4FT4H _cPTZwhZXf8opWPyHt3N6N_A_hbNs6gmng0l5eQwBZLPk5gSdhskLF2Kz8Ha05qmfmmY0a4SkgvW 3xHwP3x2AmTYg6mrwyOXt4HGHl6oIS3jhQr8uZjMtQrpZRoYP3gXxd54e3dOFpVm_s2ujSFW04Ox WxffE4EtQMdZTNbznsEOKnOz5agXdcWUmOtJ3UU97oHJHtfaViSHUJieJQLV_WW11XuyhGvS_nd8 0zeV4u0Ergv3AsKXEZrk89K8ogngpBoBj1jDi1Je4_zaHDNCgXbITwfA3t0TW8HOmXVxnOXK3VB_ sS58TCqmx0ML_WO24wnofkWv3Dy3vgI6jrm0wxX0eSriWhDSOT_i_K9E26AIxUHAXEogjPJuZKbm RruwBpOhcovBD33.r95kLhomdACjnUy2bTX93sUuIMaWpaFrWsaZhHFeB3t5wMAnMbiMdPR0V.tS MW0WMo5ZrVFr_SyYW9JI6QcCL6ghwLM9A9UK.MK9Tm6CLZp8Xuu2Yem6igARJbqCC4LibdAUUORq gIb146Txh0yYCGlOKZMlUf9OFxqFtCciVlp9nw9lY8ch.mQs6lOBWh4F2e0FA1zKiQvtg2bpW5c6 V2Fo8KeFN4SC_3IyDaKLP5py69ljr4kN6e9gaSdS0._erqHOBuXJSCUy2FPT.I.GZ8OxuVLQ9MMz OOJqkYx_tQEtoawybW8b2Hric90sBOIjE48FH1Wy_ghkSB2cXeh2_r544cn642cAdoFso.LNT96h iGiyMjdGqcob.IyJNuQXAe8LeCOG.NJBzQ.sDtcpzOpjniDydOCB_5OpJiLBya7u0oHPCICa8eB6 hAOIL_JSu0VKluk6ytFtZYFI57B1vEh7DJRBmtB3oxaUqp0wpJ2WJTF3txTKsfulVIh0J5Xk0F42 .x0cpsvR472IAAUzsFr_akc8605cBHTcd7fwNVc5xw60z5.M0iRuKk5.6XRA9KsF4E4_LgHLB3xJ qudyKIEHlzOv4jIxKiw2GRtU.BAR_F2o2TMBsVX.PyNgOl9ea2yxjGZIarFyKoWQBvlBco3Nb1ok 6cr6P._4JXWPMMW5BtzAmJm2xLwaddO8TvSvQ2JY8M0TKWBUlJtYzPkNLbSFFbheEdVEcoQzU1Ug buRdGYA8RaUEO2mLnmCF16mQyXUbvTmosEffprjjUAxW8nrV1WoXh0VA4M7HHs2dURgStzJQHJta 4wQEjFkjSLqTwISJ_.GzBcv482HREdGzeEAaNXZmHvVUuJK.jMzjdNFXS2eWsRU.CgeDxWYRg9FC jq4DXLckE49pAoUDxc3MskCVsR.gFRhtYftVRxlUQuvYWu5GmSKlNarCjARLbc93Tj9kRC9z1frb ASiE_x91aTinvy_q7kvNHLT7JOYIbLwFFcl_A6c0vsSZhONhP3qfMGMZlDndEgSBonrLzv1._qMp vJ_fwee1KBVsRwkjZXdU3_GoX45AnX_WmAaTKaBVjhVHXe7MJeGG_sk7agjEVLpVOyGl9owQj.C5 Tx_mVaPdfUP5PtOQNlOFjJuMiz31kWbF_JyN_itMGeC6N7ewIU6Gv3fW2L2sWXPojEAId X-Sonic-MF: X-Sonic-ID: a97a1d1c-00d1-4685-82a7-2f36ea761fe1 Received: from sonic.gate.mail.ne1.yahoo.com by sonic316.consmr.mail.gq1.yahoo.com with HTTP; Sat, 18 Nov 2023 18:46:39 +0000 Received: by hermes--production-gq1-59b5df67b6-ffk59 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 1804ebae6e91fe9e5f2823fefbf5048e; Sat, 18 Nov 2023 18:46:36 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.200.91.1.1\)) Subject: Re: NFS exports of ZFS snapshots broken Message-Id: <4D7CDA6F-F92D-4A36-B4B6-1D6A87FD1E3F@yahoo.com> Date: Sat, 18 Nov 2023 10:46:25 -0800 To: rick.macklem@gmail.com, Current FreeBSD X-Mailer: Apple Mail (2.3774.200.91.1.1) References: <4D7CDA6F-F92D-4A36-B4B6-1D6A87FD1E3F.ref@yahoo.com> X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; TAGGED_RCPT(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; ARC_NA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.30:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.30:from]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FREEMAIL_TO(0.00)[gmail.com,freebsd.org]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4SXjRS0lxlz3gcZ X-Spamd-Bar: --- Rick Macklem wrote on Date: Sat, 18 Nov 2023 16:09:34 UTC: > . . . > I will try and get a test setup going here, which leads me to.. > how do I create a ZFS snapshot? (I do have a simple ZFS pool running > on a test machine, but I've never done a snapshot.) > . . . There is: "man zfs-snapshot" for "zfs snapshot" commands. I mention this in part because it references also using "zfs promote" to swap around the status of what is a snapshot vs. what is not. There is also the man page, accessible via: "man zfs-promote" .=20 man zfs-snapshot also mentions destroying snapshots via "zfs destroy". The man pages have examples, such as: Example 1: Creating a ZFS Snapshot The following command creates a snapshot named yesterday. This = snapshot is mounted on demand in the .zfs/snapshot directory at the root of = the pool/home/bob file system. # zfs snapshot pool/home/bob@yesterday Example 3 involves all 3 operations (snapshot, promote, destroy) that I've referenced: Example 3: Promoting a ZFS Clone The following commands illustrate how to test out changes to a file system, and then replace the original file system with the changed = one, using clones, clone promotion, and renaming: # zfs create pool/project/production populate /pool/project/production with data # zfs snapshot pool/project/production@today # zfs clone pool/project/production@today pool/project/beta make changes to /pool/project/beta and test them # zfs promote pool/project/beta # zfs rename pool/project/production pool/project/legacy # zfs rename pool/project/beta pool/project/production once the legacy version is no longer needed, it can be = destroyed # zfs destroy pool/project/legacy The description of "zfs promote" is: DESCRIPTION The zfs promote command makes it possible to destroy the dataset = that the clone was created from. The clone parent-child dependency = relationship is reversed, so that the origin dataset becomes a clone of the = specified dataset. The snapshot that was cloned, and any snapshots previous to this snapshot, are now owned by the promoted clone. The space they use = moves from the origin dataset to the promoted clone, so enough space must = be available to accommodate these snapshots. No new space is consumed = by this operation, but the space accounting is adjusted. The promoted = clone must not have any conflicting snapshot names of its own. The zfs = rename subcommand can be used to rename any conflicting snapshots. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sat Nov 18 18:47:13 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SXjS51tT0z51SZ2 for ; Sat, 18 Nov 2023 18:47:17 +0000 (UTC) (envelope-from ronald@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SXjS456Vkz4CwV; Sat, 18 Nov 2023 18:47:16 +0000 (UTC) (envelope-from ronald@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700333236; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IVtC6tUIOI8/p4hxmgEQIGohGVMZJs1bdyh/rVkrvm4=; b=P3oelTtrMuWPPDlyf0yPZYZrHQh8m/kodgq65Foy8NfqcM6I7MdCM/Bawmg/7BFzW4moP2 N+dlq81Cj7/4IaKNeO3vtqiQCWZYLqftUUQks9EmjfQ2FdEgTtVaz7Z/dTRMmTmqOaBA/I HYYGrn0VpQ7JtHQklVl8gebO+c2BfweDArIGO2cVGhMFekUFtQRSkfhf0TkBxUs/kb8TjM k73HgyBMpI7KyqcQcChp9s3MpSPtNYZ0R+ATPc9zyZsm4uxdxIx501Ak4v30mMelbFuWrL /gMId5Oi9EsCQ3gNproSL0dasY930HhEwKo8e2T/eCNWkKdTNiGI579X8JzJOA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700333236; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IVtC6tUIOI8/p4hxmgEQIGohGVMZJs1bdyh/rVkrvm4=; b=EIUpwQ8iKM5NJT7yx1XOVkAzilYblC12j5Mra0GZORnYrPIi76YuqN0cIk0/jaMRVlEBrp nVcinN39MP9n8SP48n+axFtMJKK/5tRfWIfqW1vOCh2VjrbeYHE1dEnShGGK8q8MW9yqJa VjlWvWP0rbY9m8iJe2XhlQ1ypPoNSxxyzdrZMf3qnUZ5369Mo/YpH0bZaLknqb6tw1UjUn yGvOFjWqu43aIEKTv7tJzoLfMvj98rMjq7bDwQOoR+sm4B8nOJEasZ34ADdyLCAZOIWgCL P6zaPiWKLMYTYDw9D0S0j1r/uvEZUuZTsQV+VMSl7e9m3Ypu6Air2azTW/l7mA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1700333236; a=rsa-sha256; cv=none; b=A3y1VdMw6R0K+JZCNJ5VLgNAuwgXgbhqj6I6L54aW9ziOanGu4cSyQTAmFBY/tJctfZqMF VCdTpe7b6Nbury0CnKUpdViZy8IH9bCejsj6JN2xpv2xva58Kh6GcUaYQc9e3lCJ7m0LTM pvt5Rbah3CQ08tJjYewu+4JDkxqbISOhmpO6DFy/fMBu5Gzbib4WdeUfvH55F/c5WA5+p6 xAHKHdRPtohc69BeHe8cLJG3sLscZVWWJcE8kk4hw2mtRTmPQulWOf20R5gwPn35VMfw8X 2BKy/XubjkHMVlpUpF1bLnVjTusaPfa2k1BtVShvOmYk9b/8X7N40cH4uJoviA== Received: from [192.168.1.109] (84-105-120-103.cable.dynamic.v4.ziggo.nl [84.105.120.103]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: ronald/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4SXjS367BRz105n; Sat, 18 Nov 2023 18:47:15 +0000 (UTC) (envelope-from ronald@FreeBSD.org) Message-ID: <805acb16-ee6f-4d09-b01b-39d9ae3f8c86@FreeBSD.org> Date: Sat, 18 Nov 2023 19:47:13 +0100 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: NFS exports of ZFS snapshots broken Content-Language: en-US To: FreeBSD CURRENT , Rick Macklem References: <25943.60056.880614.452966@hergotha.csail.mit.edu> From: Ronald Klop In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 11/18/23 17:09, Rick Macklem wrote: > On Fri, Nov 17, 2023 at 8:19 PM Mike Karels wrote: >> >> On 17 Nov 2023, at 22:14, Mike Karels wrote: >> >>> On 17 Nov 2023, at 21:24, Rick Macklem wrote: >>> >>>> Most of the changes in stable/13 that are not in releng/13.2 >>>> are the "make it work in a jail" stuff. Unfortunately, they are >>>> a large # of changes (mostly trivial edits adding vnet macros), >>>> but it also includes export check changes. >>>> >>>> I have attached a trivial patch that I think disables the export >>>> checks for jails. If either of you can try it and see if it fixes >>>> the problem, that would be great. >>>> (Note that this is only for testing, although it probably does not >>>> matter unless you are running nfsd(8) in vnet jails.) >>> >>> Yes, I can see snapshots with the patch. This system is just a test >>> system that doesn't normally run ZFS or NFS, so no problem messing >>> with permissions. It's a bhyve VM, so I just added a small disk and >>> enabled ZFS for testing. >> >> btw, you might try to get mm@ or maybe mav@ to help out from the ZFS >> side. It must be doing something differently inside a snapshot than >> outside, maybe with file handles or something like that. > Yes. I've added freebsd-current@ (although Garrett is not on it, he is > cc'd) and these guys specifically... > > So, here's what appears to be the problem... > Commit 88175af (in main and stable/13, but not 13.2) added checks for > nfsd(8) running in jails by filling in mnt_exjail with a reference to the cred > used when the file system is exported. > When mnt_exjail is found NULL, the current nfsd code assumes that there > is no access allowed for the mount. > > My vague understanding is that when a ZFS snapshot is accessed, it is > "pseudo-mounted" by zfsctl_snapdir_lookup() and I am guessing that > mnt_exjail is NULL as a result. > Since I do not know the ZFS code and don't even have an easy way to > test this (thankfully Mike can test easily), I do not know what to do from > here? > > Is there a "struct mount" constructed for this pseudo mount > (or it actually appears to be the lookup of ".." that fails, so it > might be the parent of the snapshot subdir?)? > > One thought is that I can check to see if the mount pointer is in the > mountlist (I don't think the snapshot's mount is in the mountlist) and > avoid the jail test for this case. This would assume that snapshots are > always within the file system(s) exported via that jail (which includes > the case of prison0, of course), so that they do not need a separate > jail check. > > If this doesn't work, there will need to be some sort of messing about > in ZFS to set mnt_exjail for these. > > I will try and get a test setup going here, which leads me to.. > how do I create a ZFS snapshot? (I do have a simple ZFS pool running > on a test machine, but I've never done a snapshot.) # zfs list ... zroot/usr/local 4.59G 27.5G 2.76G /usr/local zroot/usr/ports 1.03G 27.5G 952M /usr/ports ... # zfs snapshot zroot/usr/local@myfirstsnapshot -- to view them # zfs list -t snapshot zroot/usr/local -- and to remove it: # zfs destroy zroot/usr/local@myfirstsnapshot -- more info # man zfs-snapshot If you get used to this you are going to love it. :-) Regards and happy hacking, Ronald. > > Although this problem is not in 13.2, it will have shipped in 14.0. > > Any help with be appreciated, rick > >> >> Mike >>> >>>> rick >>>> >>>> On Fri, Nov 17, 2023 at 6:14 PM Mike Karels wrote: >>>>> >>>>> CAUTION: This email originated from outside of the University of Guelph. Do not click links or open attachments unless you recognize the sender and know the content is safe. If in doubt, forward suspicious emails to IThelp@uoguelph.ca. >>>>> >>>>> >>>>> Rick, have you been following this thread on freebsd-stable? I have been able >>>>> to reproduce this using a 13-stable server from Oct 7 and a 15-current system >>>>> that is up to date using NFSv3. I did not reproduce with a 13.2 server. The >>>>> client was running 13.2. Any ideas? A full bisect seems fairly painful, but >>>>> maybe you have an idea of points to try. Fortunately, these are all test >>>>> systems that I can reboot at will. >>>>> >>>>> Mike >>>>> >>>>> Forwarded message: >>>>> >>>>>> From: Garrett Wollman >>>>>> To: Mike Karels >>>>>> Cc: freebsd-stable@freebsd.org >>>>>> Subject: Re: NFS exports of ZFS snapshots broken >>>>>> Date: Fri, 17 Nov 2023 17:35:04 -0500 >>>>>> >>>>>> < said: >>>>>> >>>>>>> I have not run into this, so I tried it just now. I had no problem. >>>>>>> The server is 13.2, fully patched, the client is up-to-date -current, >>>>>>> and the mount is v4. >>>>>> >>>>>> On my 13.2 client and 13-stable server, I see: >>>>>> >>>>>> 25034 ls CALL open(0x237d32f9a000,0x120004) >>>>>> 25034 ls NAMI "/mnt/tools/.zfs/snapshot/weekly-2023-45" >>>>>> 25034 ls RET open 4 >>>>>> 25034 ls CALL fcntl(0x4,F_ISUNIONSTACK,0x0) >>>>>> 25034 ls RET fcntl 0 >>>>>> 25034 ls CALL getdirentries(0x4,0x237d32faa000,0x1000,0x237d32fa7028) >>>>>> 25034 ls RET getdirentries -1 errno 5 Input/output error >>>>>> 25034 ls CALL close(0x4) >>>>>> 25034 ls RET close 0 >>>>>> 25034 ls CALL exit(0) >>>>>> >>>>>> Certainly a libc bug here that getdirentries(2) returning [EIO] >>>>>> results in ls(1) returning EXIT_SUCCESS, but the [EIO] error is >>>>>> consistent across both FreeBSD and Linux clients. >>>>>> >>>>>> Looking at this from the RPC side: >>>>>> >>>>>> (PUTFH, GETATTR, LOOKUP(snapshotname), GETFH, GETATTR) >>>>>> [NFS4_OK for all ops] >>>>>> (PUTFH, GETATTR) >>>>>> [NFS4_OK, NFS4_OK] >>>>>> (PUTFH, ACCESS(0x3f), GETATTR) >>>>>> [NFS4_OK, NFS4_OK, rights = 0x03, NFS4_OK] >>>>>> (PUTFH, GETATTR, LOOKUPP, GETFH, GETATTR) >>>>>> [NFS4_OK, NFS4_OK, NFS4ERR_NOFILEHANDLE] >>>>>> >>>>>> and at this point the [EIO] is returned. >>>>>> >>>>>> It seems that clients always do a LOOKUPP before calling READDIR, and >>>>>> this is failing when the subject file handle is the snapshot. The >>>>>> client is perfectly able to *traverse into* the snapshot: if I try to >>>>>> list a subdirectory I know exists in the snapshot, the client is able to >>>>>> LOOKUP(dirname) just fine, but LOOKUPP still fails with >>>>>> NFS4ERR_NOFILEHANDLE *on the subndirectory*. >>>>>> >>>>>> -GAWollman >>>>> > From nobody Sat Nov 18 19:01:01 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SXjm41mFFz51TRw for ; Sat, 18 Nov 2023 19:01:08 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from www121.sakura.ne.jp (www121.sakura.ne.jp [153.125.133.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SXjm34Wkrz4GTk; Sat, 18 Nov 2023 19:01:07 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Authentication-Results: mx1.freebsd.org; none Received: from kalamity.joker.local (123-1-22-158.area1b.commufa.jp [123.1.22.158]) (authenticated bits=0) by www121.sakura.ne.jp (8.16.1/8.16.1/[SAKURA-WEB]/20201212) with ESMTPA id 3AIJ11RE009571; Sun, 19 Nov 2023 04:01:01 +0900 (JST) (envelope-from junchoon@dec.sakura.ne.jp) Date: Sun, 19 Nov 2023 04:01:01 +0900 From: Tomoaki AOKI To: tuexen@freebsd.org Cc: freebsd-current@freebsd.org Subject: Re: Request for Testing: TCP RACK Message-Id: <20231119040101.503d44475182e9721081179e@dec.sakura.ne.jp> In-Reply-To: References: <42C327BD-6CE4-43AA-A1AE-3BEC08D623DB@freebsd.org> <2bbfcf68-3249-45c7-a8e5-af7115a1ccfe@gmail.com> <20231118083704.3ebc17eb8f392ceae33f65f7@dec.sakura.ne.jp> Organization: Junchoon corps X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; amd64-portbld-freebsd14.0) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:7684, ipnet:153.125.128.0/18, country:JP] X-Rspamd-Queue-Id: 4SXjm34Wkrz4GTk On Sat, 18 Nov 2023 09:50:43 +0100 tuexen@freebsd.org wrote: > > On Nov 18, 2023, at 00:37, Tomoaki AOKI wrote: > > > > On Fri, 17 Nov 2023 18:51:05 +0100 > > tuexen@freebsd.org wrote: > > > >>> On Nov 17, 2023, at 17:06, Johan Hendriks wrote: > >>> > >>> I am running the rack stack for quiet some time now on a baremetal machiene and never had problems. > >>> Also use pf. This is a test machine so not a lot happening on it. > >>> > >>> Are there any thing we can test? Do we have some test scripts we can run? > >> We are actually interested in feedback about using the stack in whatever > >> use case you use TCP for. The stack has been tested with the Netflix use > >> case, but not so much with others. That is why we ask for broader testing. > >> > >> Best regards > >> Michael > > > > Are there any difference regarding with performance between main and > > stable/14? If so, please ignore below. > > > > I have stable/14 environment which is configured to be able to switch > > to TCP RACK and experienced huge performance loss when writing > > a large file to smbfs share on commercial NAS. CC is cubic. > > Testing large archive on the smbfs share doesn't seem to be > > affected. > > > > Comparison between default (freebsd) and rack TCP stack using > > sysutils/clone on stable/14 at commit 7d1321288ad9, amd64. > > Last 3 lines of outputs from clone (upload to NAS) are shown. > Thank you very much for testing. This is what we are looking > for. Would it be possible to repeat the test using NewReno as > the CC? > > Best regards > Michael Sure. Here we go! sysctl net.inet.tcp.functions_default net.inet.tcp.functions_default: freebsd sysctl net.inet.tcp.cc.algorithm net.inet.tcp.cc.algorithm: newreno Umounted and remounted smbfs share. 1 item copied, 2343.1 MB in 37.65 s -- 62.2 MB/s Leaked memory: 0 bytes No errors occured. sysctl net.inet.tcp.functions_default net.inet.tcp.functions_default: rack Umounted and remounted smbfs share. 1 item copied, 2343.1 MB in 905.17 s -- 2.6 MB/s Leaked memory: 0 bytes No errors occured. sysctl net.inet.tcp.functions_default net.inet.tcp.functions_default: freebsd Without umount and remount to reproduce previous oddness, maybe caused by keep-alive. 1 item copied, 2343.1 MB in 897.67 s -- 2.6 MB/s Leaked memory: 0 bytes No errors occured. Umounted and remounted, without change for CC and TCP stack. 1 item copied, 2343.1 MB in 37.43 s -- 62.6 MB/s Leaked memory: 0 bytes No errors occured. All test are proceeded simultaneously. So the last one is with CC=newreno and TCP stack=freebsd. Not exactly recorded, but testing transferred file by diff -u -p -N was roughly 30MB/sec, while roughly 25MB/sec with CC=cubic. > > > > Before switching to rack: > > 1 item copied, 2342.4 MB in 39.12 s -- 59.9 MB/s > > Leaked memory: 0 bytes > > No errors occured. > > > > Unmount the smbfs share, switch to rack, and after remount: > > 1 item copied, 2342.4 MB in 926.59 s -- 2.5 MB/s > > Leaked memory: 0 bytes > > No errors occured. > > > > Switch back to freebsd (default) without unmounting: > > 1 item copied, 2342.4 MB in 906.94 s -- 2.6 MB/s > > Leaked memory: 0 bytes > > No errors occured. > > > > Unmount and remount the smbfs share: > > 1 item copied, 2342.4 MB in 39.12 s -- 59.9 MB/s > > Leaked memory: 0 bytes > > No errors occured. > > > > > > Regards. > > > > -- > > Tomoaki AOKI -- Tomoaki AOKI From nobody Sat Nov 18 19:11:57 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SXk0h6fqwz51V5B; Sat, 18 Nov 2023 19:12:04 +0000 (UTC) (envelope-from zlei@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SXk0h6Qs4z4KG4; Sat, 18 Nov 2023 19:12:04 +0000 (UTC) (envelope-from zlei@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700334724; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=R/ioEw3Sxqkpn+Ru5uizaBz//B8St8P2BbwF/hiv4eE=; b=LvKl7VREC10oz23HMemjjiCXp/da73p5c3va/zEIvKuw71enUy2OdMRDRkWj8iHsip641m obYXjmSsZyLJGPpEER9bp6Y1UCQE4YQAMEttR7gdBA++4Qe1MONfXuxVrTJ7EwLxU1+2lN c2rPj5wYCoG9A+1F4rNN+GiAbnhuvelfMpdvLAlCqP8/v3vCDXSd8KeTsx4Wc8VVQle72V 1YUvBxk4y0Ooec/UslH47qiewgoULngAsMNzJyGS21evAzbmVQL6U1KXVDiaCnyII+39jr Ng7gFnMRQyvpx34aCTBc419D+QYoA1AClMU50kBh5d7TQy9m1IkcJtVzOmv03w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1700334724; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=R/ioEw3Sxqkpn+Ru5uizaBz//B8St8P2BbwF/hiv4eE=; b=J3lOISKkZIn4RY0amqRJKSDht3HKYS4vRezvAu8SrNo2woQyoNspJmUi+DW9OwO4CJv7m1 qcwmGBcXzApdLhu2liLx0h/kY4E6x/kNiemHkmrdDKe3UCOXF7avmTH+p7qoFRChTdxTdT nXXohqqJNjtZEFOdkcP58cI1XVvGce3Y4oEebhN00aRJSOr1k5+qu1/LUHWeBuxkEu/r1J SWYQeBzF+iomXLiOkYfgCWZ/mC1osyhOqOjA9/gyCuKDWmE46oTmYxv9iyx/GfAHDBvAjC SJooUUqnFmiihyOLpA3qRKwEUEhKF6tU83ZQ0m5ayTpDE3E84sxsF7UZW/aa7Q== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1700334724; a=rsa-sha256; cv=none; b=gP/BCroEqq5Z6RUatqTGeZvrXWsGUHnbgyCHBbzXIgVZuL2UVb6+fs3fRFQN82NG7fSNZF l2Ql74sNUw+cm4p/VDe+mNvMKx1Rir3wGg4R91VkKevVVJDr7GUxPyRT1/ok9FfXrEK5Lh y8Y8zmLZHf1WpADFKph7R4G6J0hF6IHpj55/56hWdtOXyU/zAb+4p5r9dD4Q5QqMKjCKAF jDDNkQ1IFavzA282liZrArT6cwcQtDx2Woi47uQSnL+TTpXXRnLnTBo7/9DjIWH+OolbI5 fjz12hi8lokzSK9Vo5aNpcJ5TYeB/C2/rnuRNpXyyPaQv1H/5XyJhE0ljKZpbg== Received: from smtpclient.apple (ns1.oxydns.net [45.32.91.63]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: zlei/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4SXk0g2g4Qz106M; Sat, 18 Nov 2023 19:12:03 +0000 (UTC) (envelope-from zlei@FreeBSD.org) From: Zhenlei Huang Message-Id: <64F46D36-7327-44E9-A0AD-59669452217B@FreeBSD.org> Content-Type: multipart/alternative; boundary="Apple-Mail=_0CD24F9E-1A69-4822-9C27-CFB1FF17C237" List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.4\)) Subject: Re: Request for Testing: TCP RACK Date: Sun, 19 Nov 2023 03:11:57 +0800 In-Reply-To: <941203D6-3A72-48A4-8C2C-F59C964199A9@FreeBSD.org> Cc: FreeBSD Current , net@freebsd.org To: tuexen@freebsd.org References: <42C327BD-6CE4-43AA-A1AE-3BEC08D623DB@freebsd.org> <941203D6-3A72-48A4-8C2C-F59C964199A9@FreeBSD.org> X-Mailer: Apple Mail (2.3696.120.41.1.4) --Apple-Mail=_0CD24F9E-1A69-4822-9C27-CFB1FF17C237 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On Nov 19, 2023, at 2:35 AM, Zhenlei Huang wrote: >=20 >=20 >=20 >> On Nov 16, 2023, at 5:13 PM, tuexen@freebsd.org wrote: >>=20 >> Dear all, >>=20 >> recently the main branch was changed to build the TCP RACK stack >> which is a loadable kernel module, by default: >> = https://cgit.FreeBSD.org/src/commit/?id=3D3a338c534154164504005beb00a3c6fe= b03756cc >>=20 >> As discussed on the bi-weekly transport call, it would be great if = people >> could test the RACK stack for their workload. Please report any = problems to the >> net@ mailing list or open an issue in the bug tracker and drop me a = note via email. >> This includes regressions in CPU usage, regressions in performance or = any other >> unexpected change you observe. >=20 > I see some performance regression with the rack stack. >=20 > This is iperf3 test on local host (bare metal, an old i5 2 cores 4 = threads MBP). >=20 > freebsd: 16.1 Gbits/sec > rack: 12.3 Gbits/sec >=20 > The congestion control algorithm is cubic. >=20 > Note this is a quick test with DEBUG options enabled. >=20 > I'll try with no debug options and report later. ** UPDATE ** With no debug options: freebsd: 37.2 Gbits/sec rack: 27.9 Gbits/sec >=20 >>=20 >> You can load the kernel module using >> kldload tcp_rack >>=20 >> You can make the RACK stack the default stack using >> sysctl net.inet.tcp.functions_default=3Drack >>=20 >> Based on the feedback we get, the default stack might be switched to = the >> RACK stack. >>=20 >> Please let me know if you have any questions. >>=20 >> Best regards >> Michael >>=20 >>=20 >>=20 >=20 > Best regards, > Zhenlei --Apple-Mail=_0CD24F9E-1A69-4822-9C27-CFB1FF17C237 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii

On Nov 19, 2023, at 2:35 AM, Zhenlei Huang <zlei@FreeBSD.org> = wrote:



On = Nov 16, 2023, at 5:13 PM, tuexen@freebsd.org wrote:

Dear= all,

recently the main branch was changed = to build the TCP RACK stack
which is a loadable kernel = module, by default:
https://cgit.FreeBSD.org/src/commit/?id=3D3a338c534154164504005= beb00a3c6feb03756cc

As discussed on the = bi-weekly transport call, it would be great if people
could = test the RACK stack for their workload. Please report any problems to = the
net@ mailing list or open an issue in the bug tracker = and drop me a note via email.
This includes regressions in = CPU usage, regressions in performance or any other
unexpected change you observe.

I see some = performance regression with the rack stack.

This is = iperf3 test on local host (bare metal, an old i5 2 cores 4 threads = MBP).

freebsd: = 16.1 = Gbits/sec
rack: = 12.3 = Gbits/sec

The = congestion control algorithm is cubic.

Note this is a quick test with DEBUG options = enabled.

I'll try with = no debug options and report later.

** UPDATE **

With no debug options:

freebsd: 37.2 = Gbits/sec
rack: 27.9 = Gbits/sec



You can load the = kernel module using
kldload tcp_rack

You can make the RACK stack the default stack using
sysctl net.inet.tcp.functions_default=3Drack

Based on the feedback we get, the default stack might be = switched to the
RACK stack.

Please let me know if you have any questions.
Best regards
Michael




Best = regards,
Zhenlei



= --Apple-Mail=_0CD24F9E-1A69-4822-9C27-CFB1FF17C237-- From nobody Sat Nov 18 21:58:20 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SXnhm21TNz50hfR for ; Sat, 18 Nov 2023 21:58:32 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Received: from mail-oa1-x35.google.com (mail-oa1-x35.google.com [IPv6:2001:4860:4864:20::35]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SXnhl32jzz4cNb; Sat, 18 Nov 2023 21:58:31 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=kguMvSgQ; spf=pass (mx1.freebsd.org: domain of rick.macklem@gmail.com designates 2001:4860:4864:20::35 as permitted sender) smtp.mailfrom=rick.macklem@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-oa1-x35.google.com with SMTP id 586e51a60fabf-1f00b95dc43so1803923fac.3; Sat, 18 Nov 2023 13:58:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700344710; x=1700949510; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=HjMlC5JAhisItwQUfd8YDopK5d2PPwiiK4qvs+2jZ3I=; b=kguMvSgQh2UavZFu5HlXGACbkrQSXW2x5Fv2gmYj9HJVimkd+iZzNltRTqV5AGwXXL 3zVKQ6Ogb+dAhH2jCjEfSqu9p+psExrYU8msacoLDKCq+FgYDRGCAIn4FihBXuhO2z8H tVyU4uDza0g9oq7xnZUPPtxWtSsUA54vPcnI9bPKwA+MvLchnx3C9T3QxYi+mbQPnMfz fdOr4Y+kXPNmwQGkUFP2fHp/uwS1yWNlC4vNKLRdeipIRV8KFuPMap5cOJ/gxwBBr5J/ SNuWsV++fMmEjHXJYw5Hd+uWVdAiuM7xVK2RkBmDFCBG68PTFSc3WZ9RxVMBkSBJzpSt 2zog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700344710; x=1700949510; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=HjMlC5JAhisItwQUfd8YDopK5d2PPwiiK4qvs+2jZ3I=; b=LGht0/QDZA+/dQldBPK4TZNNpne/dEA5helLsc2Z8irGIVBMXokO08NUnLhj3kBaZ2 N+4YAu2we9AcmZV3BtqU00RzjqN5jgs7Hj6ojGpiOZQ+/zr8ndSGes1wE4f8pfsVsDhz ixrsEouRBc2ZIb5x5BLmb7oaW0xt1I4GNCcmhKYczYVXChFD9p2nqroxSZAw9rpIiBXy 2U9PLt5LDVmtB1Ff26AItysNh4aApRLdo5gkEm1dXtB9cBFrNkPwDcFZcqBvTjJT08tb E8Z91CeW3EKrv2D/r5XBOxRPsUe9e5oZBq8uh/xeTcK/K3Ybt6t5/we9isZs02uy/ynO ivog== X-Gm-Message-State: AOJu0YxFUytH3A6HCafwlAMWEKO80Kkdxadta+8BsPGJolypyzAWRvJC /DAAQMJRYVwfPlTG5nXlPqf1ODg3LWMMYSB+JQ5LMPo= X-Google-Smtp-Source: AGHT+IEBKKWrrJUSvpQVPzVkhZMHrUC6uOFPSPbiyeFtsAKyQCoUPgqBm9FRvUDv1KIiERUb/bc4Qg3tzUnNV6evJcE= X-Received: by 2002:a05:6870:a40b:b0:1e9:cde8:6db0 with SMTP id m11-20020a056870a40b00b001e9cde86db0mr4047109oal.50.1700344710075; Sat, 18 Nov 2023 13:58:30 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: <25943.60056.880614.452966@hergotha.csail.mit.edu> In-Reply-To: From: Rick Macklem Date: Sat, 18 Nov 2023 13:58:20 -0800 Message-ID: Subject: Re: NFS exports of ZFS snapshots broken To: Mike Karels , FreeBSD CURRENT Cc: Rick Macklem , Garrett Wollman , Alexander Motin , Martin Matuska Content-Type: multipart/mixed; boundary="00000000000072469f060a745a22" X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; R_SPF_ALLOW(-0.20)[+ip6:2001:4860:4000::/36]; MIME_GOOD(-0.10)[multipart/mixed,text/plain]; ARC_NA(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; FREEMAIL_FROM(0.00)[gmail.com]; FROM_HAS_DN(0.00)[]; TAGGED_FROM(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; ASN(0.00)[asn:15169, ipnet:2001:4860:4864::/48, country:US]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_IN_DNSWL_NONE(0.00)[2001:4860:4864:20::35:from]; MID_RHS_MATCH_FROMTLD(0.00)[]; RCPT_COUNT_FIVE(0.00)[6]; HAS_ATTACHMENT(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; TO_DN_ALL(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; DKIM_TRACE(0.00)[gmail.com:+]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org] X-Rspamd-Queue-Id: 4SXnhl32jzz4cNb X-Spamd-Bar: --- --00000000000072469f060a745a22 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sat, Nov 18, 2023 at 8:09=E2=80=AFAM Rick Macklem wrote: > > On Fri, Nov 17, 2023 at 8:19=E2=80=AFPM Mike Karels wro= te: > > > > On 17 Nov 2023, at 22:14, Mike Karels wrote: > > > > > On 17 Nov 2023, at 21:24, Rick Macklem wrote: > > > > > >> Most of the changes in stable/13 that are not in releng/13.2 > > >> are the "make it work in a jail" stuff. Unfortunately, they are > > >> a large # of changes (mostly trivial edits adding vnet macros), > > >> but it also includes export check changes. > > >> > > >> I have attached a trivial patch that I think disables the export > > >> checks for jails. If either of you can try it and see if it fixes > > >> the problem, that would be great. > > >> (Note that this is only for testing, although it probably does not > > >> matter unless you are running nfsd(8) in vnet jails.) > > > > > > Yes, I can see snapshots with the patch. This system is just a test > > > system that doesn't normally run ZFS or NFS, so no problem messing > > > with permissions. It's a bhyve VM, so I just added a small disk and > > > enabled ZFS for testing. > > > > btw, you might try to get mm@ or maybe mav@ to help out from the ZFS > > side. It must be doing something differently inside a snapshot than > > outside, maybe with file handles or something like that. > Yes. I've added freebsd-current@ (although Garrett is not on it, he is > cc'd) and these guys specifically... > > So, here's what appears to be the problem... > Commit 88175af (in main and stable/13, but not 13.2) added checks for > nfsd(8) running in jails by filling in mnt_exjail with a reference to the= cred > used when the file system is exported. > When mnt_exjail is found NULL, the current nfsd code assumes that there > is no access allowed for the mount. > > My vague understanding is that when a ZFS snapshot is accessed, it is > "pseudo-mounted" by zfsctl_snapdir_lookup() and I am guessing that > mnt_exjail is NULL as a result. > Since I do not know the ZFS code and don't even have an easy way to > test this (thankfully Mike can test easily), I do not know what to do fro= m > here? > > Is there a "struct mount" constructed for this pseudo mount > (or it actually appears to be the lookup of ".." that fails, so it > might be the parent of the snapshot subdir?)? > > One thought is that I can check to see if the mount pointer is in the > mountlist (I don't think the snapshot's mount is in the mountlist) and > avoid the jail test for this case. This would assume that snapshots are > always within the file system(s) exported via that jail (which includes > the case of prison0, of course), so that they do not need a separate > jail check. > > If this doesn't work, there will need to be some sort of messing about > in ZFS to set mnt_exjail for these. Ok, so now onto the hard part... Thanks to Mike and others, I did create a snapshot under .zfs and I can see the problem. It is that mnt_exjail =3D=3D NULL. Now, is there a way that this "struct mount" can be recognized as "special" for snapshots, so I can avoid the mnt_exjail =3D=3D NULL test? (I had hoped that "mp->mnt_list.tqe_prev" would be NULL, but that is not the case.) Do I need to search mountlist for it? rick ps: The hack patch attached should fix the problem, but can only be safely used if mountd/nfsd are not run in any jails. > > I will try and get a test setup going here, which leads me to.. > how do I create a ZFS snapshot? (I do have a simple ZFS pool running > on a test machine, but I've never done a snapshot.) > > Although this problem is not in 13.2, it will have shipped in 14.0. > > Any help with be appreciated, rick > > > > > Mike > > > > > >> rick > > >> > > >> On Fri, Nov 17, 2023 at 6:14=E2=80=AFPM Mike Karels wrote: > > >>> > > >>> CAUTION: This email originated from outside of the University of Gu= elph. Do not click links or open attachments unless you recognize the sende= r and know the content is safe. If in doubt, forward suspicious emails to I= Thelp@uoguelph.ca. > > >>> > > >>> > > >>> Rick, have you been following this thread on freebsd-stable? I hav= e been able > > >>> to reproduce this using a 13-stable server from Oct 7 and a 15-curr= ent system > > >>> that is up to date using NFSv3. I did not reproduce with a 13.2 se= rver. The > > >>> client was running 13.2. Any ideas? A full bisect seems fairly pa= inful, but > > >>> maybe you have an idea of points to try. Fortunately, these are al= l test > > >>> systems that I can reboot at will. > > >>> > > >>> Mike > > >>> > > >>> Forwarded message: > > >>> > > >>>> From: Garrett Wollman > > >>>> To: Mike Karels > > >>>> Cc: freebsd-stable@freebsd.org > > >>>> Subject: Re: NFS exports of ZFS snapshots broken > > >>>> Date: Fri, 17 Nov 2023 17:35:04 -0500 > > >>>> > > >>>> < said: > > >>>> > > >>>>> I have not run into this, so I tried it just now. I had no probl= em. > > >>>>> The server is 13.2, fully patched, the client is up-to-date -curr= ent, > > >>>>> and the mount is v4. > > >>>> > > >>>> On my 13.2 client and 13-stable server, I see: > > >>>> > > >>>> 25034 ls CALL open(0x237d32f9a000,0x120004) > > >>>> 25034 ls NAMI "/mnt/tools/.zfs/snapshot/weekly-2023-45" > > >>>> 25034 ls RET open 4 > > >>>> 25034 ls CALL fcntl(0x4,F_ISUNIONSTACK,0x0) > > >>>> 25034 ls RET fcntl 0 > > >>>> 25034 ls CALL getdirentries(0x4,0x237d32faa000,0x1000,0x23= 7d32fa7028) > > >>>> 25034 ls RET getdirentries -1 errno 5 Input/output error > > >>>> 25034 ls CALL close(0x4) > > >>>> 25034 ls RET close 0 > > >>>> 25034 ls CALL exit(0) > > >>>> > > >>>> Certainly a libc bug here that getdirentries(2) returning [EIO] > > >>>> results in ls(1) returning EXIT_SUCCESS, but the [EIO] error is > > >>>> consistent across both FreeBSD and Linux clients. > > >>>> > > >>>> Looking at this from the RPC side: > > >>>> > > >>>> (PUTFH, GETATTR, LOOKUP(snapshotname), GETFH, GETATTR) > > >>>> [NFS4_OK for all ops] > > >>>> (PUTFH, GETATTR) > > >>>> [NFS4_OK, NFS4_OK] > > >>>> (PUTFH, ACCESS(0x3f), GETATTR) > > >>>> [NFS4_OK, NFS4_OK, rights =3D 0x03, NFS4_OK] > > >>>> (PUTFH, GETATTR, LOOKUPP, GETFH, GETATTR) > > >>>> [NFS4_OK, NFS4_OK, NFS4ERR_NOFILEHANDLE] > > >>>> > > >>>> and at this point the [EIO] is returned. > > >>>> > > >>>> It seems that clients always do a LOOKUPP before calling READDIR, = and > > >>>> this is failing when the subject file handle is the snapshot. The > > >>>> client is perfectly able to *traverse into* the snapshot: if I try= to > > >>>> list a subdirectory I know exists in the snapshot, the client is a= ble to > > >>>> LOOKUP(dirname) just fine, but LOOKUPP still fails with > > >>>> NFS4ERR_NOFILEHANDLE *on the subndirectory*. > > >>>> > > >>>> -GAWollman > > >>> --00000000000072469f060a745a22 Content-Type: application/octet-stream; name="zfsjail.patch" Content-Disposition: attachment; filename="zfsjail.patch" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_lp4lbu7d0 LS0tIHN5cy9mcy9uZnNzZXJ2ZXIvbmZzX25mc2Rwb3J0LmMuamFpbAkyMDIzLTExLTE3IDE5OjEw OjEyLjg3MTYxOTAwMCAtMDgwMAorKysgc3lzL2ZzL25mc3NlcnZlci9uZnNfbmZzZHBvcnQuYwky MDIzLTExLTE3IDE5OjExOjM4Ljk0MjY3NDAwMCAtMDgwMApAQCAtMzI4MiwxMSArMzI4MiwxMyBA QCBuZnN2bm9fY2hlY2tleHAoc3RydWN0IG1vdW50ICptcCwgc3RydWN0IHNvY2thZGRyICpuYW0K IAogCWVycm9yID0gMDsKIAkqY3JlZHAgPSBOVUxMOworI2lmZGVmIG5vdG5vdwogCU1OVF9JTE9D SyhtcCk7CiAJaWYgKG1wLT5tbnRfZXhqYWlsID09IE5VTEwgfHwKIAkgICAgbXAtPm1udF9leGph aWwtPmNyX3ByaXNvbiAhPSBjdXJ0aHJlYWQtPnRkX3VjcmVkLT5jcl9wcmlzb24pCiAJCWVycm9y ID0gRUFDQ0VTOwogCU1OVF9JVU5MT0NLKG1wKTsKKyNlbmRpZgogCWlmIChlcnJvciA9PSAwKQog CQllcnJvciA9IFZGU19DSEVDS0VYUChtcCwgbmFtLCAmZXhwLT5uZXNfZXhmbGFnLCBjcmVkcCwK IAkJICAgICZleHAtPm5lc19udW1zZWNmbGF2b3IsIGV4cC0+bmVzX3NlY2ZsYXZvcnMpOwpAQCAt MzMyMywxMSArMzMyNSwxMyBAQCBuZnN2bm9fZmh0b3ZwKHN0cnVjdCBtb3VudCAqbXAsIGZoYW5k bGVfdCAqZmhwLCBzdHJ1Y3QKIAkJLyogTWFrZSBzdXJlIHRoZSBzZXJ2ZXIgcmVwbGllcyBFU1RB TEUgdG8gdGhlIGNsaWVudC4gKi8KIAkJZXJyb3IgPSBFU1RBTEU7CiAJaWYgKG5hbSAmJiAhZXJy b3IpIHsKKyNpZmRlZiBub3Rub3cKIAkJTU5UX0lMT0NLKG1wKTsKIAkJaWYgKG1wLT5tbnRfZXhq YWlsID09IE5VTEwgfHwKIAkJICAgIG1wLT5tbnRfZXhqYWlsLT5jcl9wcmlzb24gIT0gY3VydGhy ZWFkLT50ZF91Y3JlZC0+Y3JfcHJpc29uKQogCQkJZXJyb3IgPSBFQUNDRVM7CiAJCU1OVF9JVU5M T0NLKG1wKTsKKyNlbmRpZgogCQlpZiAoZXJyb3IgPT0gMCkKIAkJCWVycm9yID0gVkZTX0NIRUNL RVhQKG1wLCBuYW0sICZleHAtPm5lc19leGZsYWcsIGNyZWRwLAogCQkJICAgICZleHAtPm5lc19u dW1zZWNmbGF2b3IsIGV4cC0+bmVzX3NlY2ZsYXZvcnMpOwo= --00000000000072469f060a745a22-- From nobody Sat Nov 18 22:03:11 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SXnpL2r0fz50jPc for ; Sat, 18 Nov 2023 22:03:22 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Received: from mail-oa1-x2b.google.com (mail-oa1-x2b.google.com [IPv6:2001:4860:4864:20::2b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SXnpL18MPz4djQ; Sat, 18 Nov 2023 22:03:22 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-oa1-x2b.google.com with SMTP id 586e51a60fabf-1f5da5df68eso334888fac.2; Sat, 18 Nov 2023 14:03:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1700345001; x=1700949801; darn=freebsd.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=ex9QfF4KngHHDOYCOOBXCsmXGbjOktgKYHNzrV5gT98=; b=QISNpbAvacgVvP0v/XW+IPCe26EP5AyvLV2cZWEBct8J1Xe1C6fg4d9tMHJ9fu7Xqj aOp/XyHSAy0uXMziW3YmB5dbA7QzRUuMx3yFQe1hX1U+RnjFZjo+YSOM/UofjcT8ZMfS +LQ3EGeyAiXWKpHQ5DwYzYElF1kcOzoCSHzeN+D7CBEl6ruL0OI4/LPKxhbFgY1M4v8a YOaQ++NDAtfyWxdPmQwjIjhmPLrRM+jv98IqGZMJxl7+pgj5o/3VPZX6qdaA4rUU8Q6p SBDe7qVa6Mw0tXYAV1TPPfTNkXYbddQW3xEMqmfcdkBPpb4oPhxvEkZSmdeJ/Ig7Z58m FivQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700345001; x=1700949801; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ex9QfF4KngHHDOYCOOBXCsmXGbjOktgKYHNzrV5gT98=; b=G4hHON2ZlUIAZFgw2GibVxKndBY6MEw9jmP4WzKxRTPo4k+ByCicbYe6cnghD99Q65 oRVi7FN7GulgobNKug54//OJyZfaobg5HpoPfpmOP+t01IC7/BuMhgx+KC5R/cMB8MaJ I/p/wi1QKAPhTs6imZw0GIdyfsxJrLtc4njE460V3/iZiCZ1rz5e81HtBYyqZxk5NQFP RKB1Q2JU8CfuVBQ/OtcH906NGz1o7xl9P/6UxGmExjpujtNxUYNk3kLwT/LmAysYGlF8 jVRo0xCxASfwdwgNZDSkTkqOmHORkf/lNFdZ/7ttZ1wdaPwPbHhM7ivDKFd46qdTdv3J 7t1Q== X-Gm-Message-State: AOJu0Yx8wiG7X+Sq2Cgb9c6SprR9N4unAHfbSsiG52lzfjdmn4MwHo+W orB5Ko+HU2EtllZY4PfpuvolWXJXveteWpYc59EM+lk= X-Google-Smtp-Source: AGHT+IEeuWE9DHSNOf/mMjOcAG+39VnYGUrQ5jMnZtP+sqThNfLscf9tn9h07Ew1z20uFMZ+ZmrZLddU5iJzR25HQsM= X-Received: by 2002:a05:6870:460a:b0:1f4:a631:f5fc with SMTP id z10-20020a056870460a00b001f4a631f5fcmr4193296oao.33.1700345000786; Sat, 18 Nov 2023 14:03:20 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: <25943.60056.880614.452966@hergotha.csail.mit.edu> <805acb16-ee6f-4d09-b01b-39d9ae3f8c86@FreeBSD.org> In-Reply-To: <805acb16-ee6f-4d09-b01b-39d9ae3f8c86@FreeBSD.org> From: Rick Macklem Date: Sat, 18 Nov 2023 14:03:11 -0800 Message-ID: Subject: Re: NFS exports of ZFS snapshots broken To: Ronald Klop Cc: FreeBSD CURRENT , Rick Macklem Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2001:4860:4864::/48, country:US]; TAGGED_FROM(0.00)[] X-Rspamd-Queue-Id: 4SXnpL18MPz4djQ On Sat, Nov 18, 2023 at 10:47=E2=80=AFAM Ronald Klop w= rote: > > CAUTION: This email originated from outside of the University of Guelph. = Do not click links or open attachments unless you recognize the sender and = know the content is safe. If in doubt, forward suspicious emails to IThelp@= uoguelph.ca. > > > On 11/18/23 17:09, Rick Macklem wrote: > > On Fri, Nov 17, 2023 at 8:19=E2=80=AFPM Mike Karels w= rote: > >> > >> On 17 Nov 2023, at 22:14, Mike Karels wrote: > >> > >>> On 17 Nov 2023, at 21:24, Rick Macklem wrote: > >>> > >>>> Most of the changes in stable/13 that are not in releng/13.2 > >>>> are the "make it work in a jail" stuff. Unfortunately, they are > >>>> a large # of changes (mostly trivial edits adding vnet macros), > >>>> but it also includes export check changes. > >>>> > >>>> I have attached a trivial patch that I think disables the export > >>>> checks for jails. If either of you can try it and see if it fixes > >>>> the problem, that would be great. > >>>> (Note that this is only for testing, although it probably does not > >>>> matter unless you are running nfsd(8) in vnet jails.) > >>> > >>> Yes, I can see snapshots with the patch. This system is just a test > >>> system that doesn't normally run ZFS or NFS, so no problem messing > >>> with permissions. It's a bhyve VM, so I just added a small disk and > >>> enabled ZFS for testing. > >> > >> btw, you might try to get mm@ or maybe mav@ to help out from the ZFS > >> side. It must be doing something differently inside a snapshot than > >> outside, maybe with file handles or something like that. > > Yes. I've added freebsd-current@ (although Garrett is not on it, he is > > cc'd) and these guys specifically... > > > > So, here's what appears to be the problem... > > Commit 88175af (in main and stable/13, but not 13.2) added checks for > > nfsd(8) running in jails by filling in mnt_exjail with a reference to t= he cred > > used when the file system is exported. > > When mnt_exjail is found NULL, the current nfsd code assumes that there > > is no access allowed for the mount. > > > > My vague understanding is that when a ZFS snapshot is accessed, it is > > "pseudo-mounted" by zfsctl_snapdir_lookup() and I am guessing that > > mnt_exjail is NULL as a result. > > Since I do not know the ZFS code and don't even have an easy way to > > test this (thankfully Mike can test easily), I do not know what to do f= rom > > here? > > > > Is there a "struct mount" constructed for this pseudo mount > > (or it actually appears to be the lookup of ".." that fails, so it > > might be the parent of the snapshot subdir?)? > > > > One thought is that I can check to see if the mount pointer is in the > > mountlist (I don't think the snapshot's mount is in the mountlist) and > > avoid the jail test for this case. This would assume that snapshots ar= e > > always within the file system(s) exported via that jail (which includes > > the case of prison0, of course), so that they do not need a separate > > jail check. > > > > If this doesn't work, there will need to be some sort of messing about > > in ZFS to set mnt_exjail for these. > > > > I will try and get a test setup going here, which leads me to.. > > how do I create a ZFS snapshot? (I do have a simple ZFS pool running > > on a test machine, but I've never done a snapshot.) > > # zfs list > ... > zroot/usr/local 4.59G 27.5G 2.76G /usr/loca= l > zroot/usr/ports 1.03G 27.5G 952M /usr/port= s > ... > > # zfs snapshot zroot/usr/local@myfirstsnapshot > -- to view them > # zfs list -t snapshot zroot/usr/local > -- and to remove it: > # zfs destroy zroot/usr/local@myfirstsnapshot > -- more info > # man zfs-snapshot > > If you get used to this you are going to love it. :-) Not likely. My test systems are old laptops and don't need fancy storage. However, I do have one very simple setup for testing. To give you a clue, it's called /example because the description I found to do it used "example" and I didn't even realize it would end up as the name of the mount. However, thanks to Mike and others, I do now have a snapshot on it. Now, as noted in my other post, comes the hard part. I hope I can identify that the "mount is special and refers to a snapshot" from the *mp, so I can avoid the mp->mnt_exjail =3D=3D NULL check for this case. I'd like to avoid having to get ZFS set mnt_exjail for the snapshots. Thanks, rick > > Regards and happy hacking, > Ronald. > > > > > > Although this problem is not in 13.2, it will have shipped in 14.0. > > > > Any help with be appreciated, rick > > > >> > >> Mike > >>> > >>>> rick > >>>> > >>>> On Fri, Nov 17, 2023 at 6:14=E2=80=AFPM Mike Karels wrote: > >>>>> > >>>>> CAUTION: This email originated from outside of the University of Gu= elph. Do not click links or open attachments unless you recognize the sende= r and know the content is safe. If in doubt, forward suspicious emails to I= Thelp@uoguelph.ca. > >>>>> > >>>>> > >>>>> Rick, have you been following this thread on freebsd-stable? I hav= e been able > >>>>> to reproduce this using a 13-stable server from Oct 7 and a 15-curr= ent system > >>>>> that is up to date using NFSv3. I did not reproduce with a 13.2 se= rver. The > >>>>> client was running 13.2. Any ideas? A full bisect seems fairly pa= inful, but > >>>>> maybe you have an idea of points to try. Fortunately, these are al= l test > >>>>> systems that I can reboot at will. > >>>>> > >>>>> Mike > >>>>> > >>>>> Forwarded message: > >>>>> > >>>>>> From: Garrett Wollman > >>>>>> To: Mike Karels > >>>>>> Cc: freebsd-stable@freebsd.org > >>>>>> Subject: Re: NFS exports of ZFS snapshots broken > >>>>>> Date: Fri, 17 Nov 2023 17:35:04 -0500 > >>>>>> > >>>>>> < said: > >>>>>> > >>>>>>> I have not run into this, so I tried it just now. I had no probl= em. > >>>>>>> The server is 13.2, fully patched, the client is up-to-date -curr= ent, > >>>>>>> and the mount is v4. > >>>>>> > >>>>>> On my 13.2 client and 13-stable server, I see: > >>>>>> > >>>>>> 25034 ls CALL open(0x237d32f9a000,0x120004) > >>>>>> 25034 ls NAMI "/mnt/tools/.zfs/snapshot/weekly-2023-45" > >>>>>> 25034 ls RET open 4 > >>>>>> 25034 ls CALL fcntl(0x4,F_ISUNIONSTACK,0x0) > >>>>>> 25034 ls RET fcntl 0 > >>>>>> 25034 ls CALL getdirentries(0x4,0x237d32faa000,0x1000,0x2= 37d32fa7028) > >>>>>> 25034 ls RET getdirentries -1 errno 5 Input/output error > >>>>>> 25034 ls CALL close(0x4) > >>>>>> 25034 ls RET close 0 > >>>>>> 25034 ls CALL exit(0) > >>>>>> > >>>>>> Certainly a libc bug here that getdirentries(2) returning [EIO] > >>>>>> results in ls(1) returning EXIT_SUCCESS, but the [EIO] error is > >>>>>> consistent across both FreeBSD and Linux clients. > >>>>>> > >>>>>> Looking at this from the RPC side: > >>>>>> > >>>>>> (PUTFH, GETATTR, LOOKUP(snapshotname), GETFH, GETATTR) > >>>>>> [NFS4_OK for all ops] > >>>>>> (PUTFH, GETATTR) > >>>>>> [NFS4_OK, NFS4_OK] > >>>>>> (PUTFH, ACCESS(0x3f), GETATTR) > >>>>>> [NFS4_OK, NFS4_OK, rights =3D 0x03, NFS4_OK] > >>>>>> (PUTFH, GETATTR, LOOKUPP, GETFH, GETATTR) > >>>>>> [NFS4_OK, NFS4_OK, NFS4ERR_NOFILEHANDLE] > >>>>>> > >>>>>> and at this point the [EIO] is returned. > >>>>>> > >>>>>> It seems that clients always do a LOOKUPP before calling READDIR, = and > >>>>>> this is failing when the subject file handle is the snapshot. The > >>>>>> client is perfectly able to *traverse into* the snapshot: if I try= to > >>>>>> list a subdirectory I know exists in the snapshot, the client is a= ble to > >>>>>> LOOKUP(dirname) just fine, but LOOKUPP still fails with > >>>>>> NFS4ERR_NOFILEHANDLE *on the subndirectory*. > >>>>>> > >>>>>> -GAWollman > >>>>> > > >