From nobody Mon Aug 9 14:58:08 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id A7E511241F87 for ; Mon, 9 Aug 2021 14:58:18 +0000 (UTC) (envelope-from alfadev@protonmail.com) Received: from mail-40132.protonmail.ch (mail-40132.protonmail.ch [185.70.40.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GjzjP6fXqz4d2V for ; Mon, 9 Aug 2021 14:58:17 +0000 (UTC) (envelope-from alfadev@protonmail.com) Date: Mon, 09 Aug 2021 14:58:08 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail; t=1628521090; bh=SIY8Axam+bgLZ4U8CZAOI8VGadgbPgKLCns/P3EgUeo=; h=Date:To:From:Reply-To:Subject:From; b=nQe2kzKIyx5pOG6e5/IF7L00kYWRy/niKECrT1yay8vmkOfAqkDjcLlhBjL1TkWPO Mt7Ua4sTLs4Oom8ds5N1B0FFvaflRSBvBMknUnkGyBLhn2AeC5j/u5/KK/vEla5Lpn vWDGwVCVrpAwE2tFsWNm8AAaShLsvNJ2SyiO0SMo= To: "freebsd-hackers@FreeBSD.org" , "freebsd-ipfw@FreeBSD.org" Reply-To: alfadev Subject: Throughput extremely decreases when IPFW 7000 mac based rules activated Message-ID: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="b1_bKHJdBxPYo44UDTJdRsGhXPG1UKtvcBbgPnzviZpg" X-Spam-Status: No, score=-1.2 required=10.0 tests=ALL_TRUSTED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM,HTML_MESSAGE shortcircuit=no autolearn=disabled version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on mailout.protonmail.ch X-Rspamd-Queue-Id: 4GjzjP6fXqz4d2V X-Spamd-Bar: + Authentication-Results: mx1.freebsd.org; dkim=pass header.d=protonmail.com header.s=protonmail header.b=nQe2kzKI; dmarc=pass (policy=quarantine) header.from=protonmail.com; spf=pass (mx1.freebsd.org: domain of alfadev@protonmail.com designates 185.70.40.132 as permitted sender) smtp.mailfrom=alfadev@protonmail.com X-Spamd-Result: default: False [1.08 / 15.00]; HAS_REPLYTO(0.00)[alfadev@protonmail.com]; R_SPF_ALLOW(-0.20)[+ip4:185.70.40.0/24:c]; FREEMAIL_FROM(0.00)[protonmail.com]; MIME_BASE64_TEXT_BOGUS(1.00)[]; DKIM_TRACE(0.00)[protonmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[protonmail.com,quarantine]; MIME_BASE64_TEXT(0.10)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[protonmail.com]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[protonmail.com:s=protonmail]; REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; NEURAL_SPAM_SHORT(0.98)[0.983]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; FREEMAIL_REPLYTO(0.00)[protonmail.com]; HAS_PHPMAILER_SIG(0.00)[]; NEURAL_SPAM_MEDIUM(1.00)[0.996]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_EQ_ADDR_ALL(0.00)[] Reply-To: alfadev@protonmail.com From: alfadev via freebsd-hackers X-Original-From: alfadev X-ThisMailContainsUnwantedMimeParts: Y This is a multi-part message in MIME format. --b1_bKHJdBxPYo44UDTJdRsGhXPG1UKtvcBbgPnzviZpg Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 SGksIEkgaGF2ZSBmcmVlYnNkIDExLjIgc2VydmVyIHdpdGggSVBGVyBmaXJld2FsbAoKODcwTWJp dHMgRmliZXIgTmV0IGV4aXN0IGluIG15IGRhdGEgY2VudGVyCgpUaGVyZSBhcmUgNzAwMCBkZWZp bmVkIG1hYyBiYXNlZCBydWxlcyBvbiBJUEZXIGFuZCAzMDAwIG9mIHRoZW0gYWN0aXZlIGNsaWVu dCAuIFRoZXJlIGlzIG5vIHByb2JsZW0gYmVmb3JlIElQRlcgcnVsZXMgbG9hZGluZyBidXQgd2hl biBpIGxvYWQgSVBGVyBydWxlcywKCnRocm91Z2hwdXQgZXh0cmVtZWx5IGRlY3JlYXNlcyB1cCB0 byA4ME1iaXRzLiBUaGVyZSBhcmUgbm90IGFueSBlcnJvciBsb2dzLiBJIGNvdWxkIG5vdCBmaW5k IHdoYXQgaXMgdGhlIHByb2JsZW0uCgpBbnkgaGVscCB3b3VsZCBiZSBhcHByZWNpYXRlZCBhdCB0 aGlzIHBvaW50Lg== --b1_bKHJdBxPYo44UDTJdRsGhXPG1UKtvcBbgPnzviZpg-- From nobody Mon Aug 9 22:10:07 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 3A95917574C0 for ; Mon, 9 Aug 2021 22:10:09 +0000 (UTC) (envelope-from jwdevel@gmail.com) Received: from mail-ed1-x533.google.com (mail-ed1-x533.google.com [IPv6:2a00:1450:4864:20::533]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Gk9Hh5Q8pz3qVm for ; Mon, 9 Aug 2021 22:10:08 +0000 (UTC) (envelope-from jwdevel@gmail.com) Received: by mail-ed1-x533.google.com with SMTP id d6so26911394edt.7 for ; Mon, 09 Aug 2021 15:10:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=FkZDQFn6BCnVmHwWjMFiI70RXv5nRO5Of4K37HkGKWs=; b=eu9Jv8l8ft5HYz/qKFHsctG6CZDnK+ay6EtwPUku3li6nH1iQ1UlPNm1E8LvOueeAE a5H6e8DSj9ClY28tO83zqJpvxY7U2hYPhimfpJ/7xPI97EdzqiGGsuPx02qmroUEgT2B oJUGkP3s0MZFJMNGE/DlAfZ77GNybioTJ83pBNoArsz28mA7H+xOQL1C2r9qQ/YruvLU hw8N7kSUAch8O2FTqiWsv1ITwEedLzWxhtqQgm5A0wYful8FSkRTrV+8hxORPQI4Yxva RMIvUCRFw8WrkoRJhmMTAQGUxKvkRhatPII7F5m+b1FKtcKzIJFlulX1USZKvhGitVqs DyNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=FkZDQFn6BCnVmHwWjMFiI70RXv5nRO5Of4K37HkGKWs=; b=TpyZ4a5CSc9Jpu2ihSNm/cOSgsVgBPzy9/WcDEPHUF0x2CnupTS9/buXmcq7fC5k+c v1yzmP03wlzPnVjzwrTUVcGHufybCrnKjQO08EJJZvm78A5CdTumshmzTylAJIrGUt1z HXa8Kyp1MWljJPTzziQ+qC2aIC+ighXTj4Egifm9C1VCslpT1RUssFsNFzQYV0VUf8hU qorwkg36sZ6jM3gUfDUB6Hth1CHCYbVKZ2aV0Q7gaoNMS3Ap5lgyxLyFCCkGHGoqcNNr 76zqBn8UBC14UTleGHKQ+64mv42n6haf4iRF0tyoNuiZMHocjkmAgBHF4LK0LDzhAbpv 5Mpg== X-Gm-Message-State: AOAM531KBwq9JzgBFB+oPK+vTksRoSOmFNm/pU2k8tDm5hVFWeLAbYEQ SCYy3mBqbECvn45VsHsaPxKnHFFA/C+XWJWCM8T57EKO X-Google-Smtp-Source: ABdhPJzu2J/SJbPWxcYcTgogCfSK/bzkIuv8B4RCjgrMGPeTZ9xwo8ckYCnTnNENDIIlYsNpPAlsAD0H40o1E7bXMoY= X-Received: by 2002:a05:6402:4412:: with SMTP id y18mr628957eda.1.1628547007723; Mon, 09 Aug 2021 15:10:07 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Received: by 2002:a17:907:704d:0:0:0:0 with HTTP; Mon, 9 Aug 2021 15:10:07 -0700 (PDT) From: John W Date: Mon, 9 Aug 2021 15:10:07 -0700 Message-ID: Subject: Tracking down BTX behavior change. To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4Gk9Hh5Q8pz3qVm X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=eu9Jv8l8; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of jwdevel@gmail.com designates 2a00:1450:4864:20::533 as permitted sender) smtp.mailfrom=jwdevel@gmail.com X-Spamd-Result: default: False [-4.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_RHS_MATCH_FROMTLD(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::533:from]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-ThisMailContainsUnwantedMimeParts: N I have a problem on my system where 11.3 boots fine, but any supported release (11.4, 12.2, 13.0) fail during the BTX stage. There's a bug report if you're interested[1] But my question is: how can I track down differences in BTX code? I have the 11.3 and 11.4 SVN sources checked out, and so far I am not able to find any differences. So far, I am looking in: src/stand/i386/btx/ and src/usr.sbin/btxld I am on amd64, and I do not see any /amd64/ directory, so I am just guessing that /i386/ is what I want. Also: given that the 11.3 boot code works fine, I wonder: what are the ramifications of using it on an 11.4 system? I am not really experienced with the boot loader process, so am not sure what the dangers involved are. Certainly, I am able to boot in to the new (11.4) kernel with the old userland installed. But my woes happened when I did 'make installworld', which presumably updated the boot code. Would it be reasonable to manually roll back the boot code to 11.3, but keep the 11.4 userland? Again, I'm really not sure the ramifications, just trying to make my system usable again. Thanks -John From nobody Mon Aug 9 22:19:42 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 5168811F8825 for ; Mon, 9 Aug 2021 22:19:45 +0000 (UTC) (envelope-from jwdevel@gmail.com) Received: from mail-ej1-x632.google.com (mail-ej1-x632.google.com [IPv6:2a00:1450:4864:20::632]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Gk9Vm4nQPz3s92 for ; Mon, 9 Aug 2021 22:19:44 +0000 (UTC) (envelope-from jwdevel@gmail.com) Received: by mail-ej1-x632.google.com with SMTP id d11so7239393eja.8 for ; Mon, 09 Aug 2021 15:19:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=IyhQWGxEAYUvo8pT9E12N0Okvddpg7gCCoJFijXgSmg=; b=DKQqqg+P05RBjPy9sGsSHoHteMZzdLvNv5Iz5zxIqIhh0BCJ8FaTCu9wzKav5Bp5bb IyG7F0c5kOGKWegL7K5F2VT5n95SVwAYnGsJordYkh8I0VQHRMlkMnpSLsPN8mtwP1EK zCjqsu4hIZCmSPqXc1f92J+mxbSsl7TXydfsww0xRSAJTSH+xXCNylfXbaQbXpaWCCQp wzPZtlvw5FkAH/9McIsX549Pu6eIUCBCpwf36b8K9ZSVRPC7tk7FWZeI2cM3NR5kiCg+ +7Ayy9st3rzNvFJbddZCPY6djL/YjQp2O2+1tRLn9yIQoRJZLA8c9mF7WWXQf0J6j5I+ vBRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=IyhQWGxEAYUvo8pT9E12N0Okvddpg7gCCoJFijXgSmg=; b=ZtFWsmR+U1RLNdujQ758LBEmNxeAPlRDlO97io5OQdMMjz7oM08PTzcdydoGDkiasU wIStn9l1Lj2vQi0p5YqVheQPkeAOokz9vrzN2LKoYkjM6lEuzF1LEgPJoteub8yx01+i linxx5NFZaIWY6+wOPhElf5jV1I20vTUMdm2Kejc4PPdj7h1Kek5AuKxaGCkSIVz0krq wH1bTglW4budLcpB/qE9EjNOk4ekD5+p/3D8zGtgFIT8SYyrdE6WcFA+U2u6TmplJ4LR 6U3Q/90jfbG7OIKXaLd8K912wrOxL7ryPU6VyVHqxd2CCSTN5OKdmC+8dOdGSnHJ9O0g EYgA== X-Gm-Message-State: AOAM530Jdi71YsUnyTVl1rGVfGEYh/EqZbWXl9WrAaKRhRh5d7YLCA6U XfhPf+qQdpOxuYuguf5btEQ3Wclx+6FzdrlwVyRkYpT2 X-Google-Smtp-Source: ABdhPJyw18O6el7ekz7aHjj7mJ915st+NM5GvYohISs2VwtZCx12/lKBK5t5flfzw5Ssg/FyzCwXmnzyBvTrUGXgwsw= X-Received: by 2002:a17:906:9c84:: with SMTP id fj4mr24959669ejc.356.1628547582827; Mon, 09 Aug 2021 15:19:42 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Received: by 2002:a17:907:704d:0:0:0:0 with HTTP; Mon, 9 Aug 2021 15:19:42 -0700 (PDT) In-Reply-To: References: From: John W Date: Mon, 9 Aug 2021 15:19:42 -0700 Message-ID: Subject: Re: Tracking down BTX behavior change. To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4Gk9Vm4nQPz3s92 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=DKQqqg+P; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of jwdevel@gmail.com designates 2a00:1450:4864:20::632 as permitted sender) smtp.mailfrom=jwdevel@gmail.com X-Spamd-Result: default: False [-2.00 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000]; MID_RHS_MATCH_FROMTLD(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::632:from]; NEURAL_SPAM_SHORT(1.00)[1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-ThisMailContainsUnwantedMimeParts: N Ah, forgot the bug link: [1] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=257722 On 8/9/21, John W wrote: > I have a problem on my system where 11.3 boots fine, but any supported > release (11.4, 12.2, 13.0) fail during the BTX stage. There's a bug > report if you're interested[1] > > But my question is: how can I track down differences in BTX code? I > have the 11.3 and 11.4 SVN sources checked out, and so far I am not > able to find any differences. > > So far, I am looking in: src/stand/i386/btx/ and src/usr.sbin/btxld > > I am on amd64, and I do not see any /amd64/ directory, so I am just > guessing that /i386/ is what I want. > > Also: given that the 11.3 boot code works fine, I wonder: what are the > ramifications of using it on an 11.4 system? > > I am not really experienced with the boot loader process, so am not > sure what the dangers involved are. > Certainly, I am able to boot in to the new (11.4) kernel with the old > userland installed. But my woes happened when I did 'make > installworld', which presumably updated the boot code. > > Would it be reasonable to manually roll back the boot code to 11.3, > but keep the 11.4 userland? Again, I'm really not sure the > ramifications, just trying to make my system usable again. > > Thanks > -John > From nobody Mon Aug 9 23:31:34 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 663E611FD911 for ; Mon, 9 Aug 2021 23:31:43 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound2r.ore.mailhop.org (outbound2r.ore.mailhop.org [54.200.129.228]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GkC5q12J9z4S96 for ; Mon, 9 Aug 2021 23:31:43 +0000 (UTC) (envelope-from ian@freebsd.org) ARC-Seal: i=1; a=rsa-sha256; t=1628551896; cv=none; d=outbound.mailhop.org; s=arc-outbound20181012; b=DaxwPD/nPYhgb6GdT04uGQ2eawvOUdJ9WssIxqqbzp8QoX6vrz8y5rm5h1D7emKrhY3NSU3GCRKag rWZ6WZghufLLmr+5o/XIz+9rqmndId3cSMCnIv+D/pL37CDut34FQ8nMx+GNcOuqgMmBMZW1KUEa43 ouFJWJ7BpCH6/+I7Rh0Jl2YX8f0ciyPhTI32wXxucClpx/Jes0bCquXooNIo4iWjc0AOqx8/DGhH53 A1kTxhZIPVtWm8e5YXbhmpHwTtpiy42Bd74A21UKTil/5Hk2gpEcIRIymp4KWaRCdUzsij0w2JU1k9 UItTMMsld4G44voAyixLg9qCzVeei+A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=arc-outbound20181012; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:to:from:subject:message-id:dkim-signature:from; bh=4GhUNPSV1J+h2A2kxr/D1x6tXoF5ePM1tckQPCQ/ClU=; b=LbQnGPMXRg7XbxnzY00O+BU6VX5QBKJ0ffNGNwceFoIDASGyO2BcIUDkxd4Whhtzjv/8rdUTyBKrH 2bPsyxsfbprtbIVNGdOv7ouxdj2BsG3R6OqDWl8OYFCG3tkEg8WbxgLCQ+8octfrwT+yzNwgPlLkLH Zc328H6geHdLJ+5XUHJJoF8QmyedoxW+4eKpvWt1UWo9mSIz0+V1ZkzM3emwQLAxB45rS5+sYdUf/6 gUBalZK2SmlDlLGahgI1yiIsp9n4ulXo5ynRZQyYCIALIoI49I8nANmjQAfBXwAUeyjT2JE7cYqna+ TZwZWkvniDfjvSndLI073O0cmIg18hA== ARC-Authentication-Results: i=1; outbound4.ore.mailhop.org; spf=softfail smtp.mailfrom=freebsd.org smtp.remote-ip=67.177.211.60; dmarc=none header.from=freebsd.org; arc=none header.oldest-pass=0; DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outbound.mailhop.org; s=dkim-high; h=content-transfer-encoding:mime-version:content-type:references:in-reply-to: date:to:from:subject:message-id:from; bh=4GhUNPSV1J+h2A2kxr/D1x6tXoF5ePM1tckQPCQ/ClU=; b=BOwdCfwneu1altS060K5i19g+Dkb3PtavRgM79d5x4+XfiMs2euzOuPmItif4yxfmTodkwGcJEMuD DsC0I0ElgEq8aHRfnYJd+n+fa3vXaoWAAMGVNITP7b6o+vmRtlAUVdKd7t/gtno2d20lzQWX40faQy R1jYla1BgPHbSjF9Qw/aAyT3AGnc6fsGVS9O5VWCEMnXS1uDNmf5+viYAdGD29eut1blKDjXgweJge VgvzrvNiSWX6ycTpNQe/xpHSDke9wHh7J1rpaSHkXnqVo78DzGYd4fUs69CS4W2SbUy1CTcAkALs5G Ms1EYmyUC1ZckoqKVo5e/aHDAOR6m7g== X-Originating-IP: 67.177.211.60 X-MHO-RoutePath: aGlwcGll X-MHO-User: ef34534e-f969-11eb-a658-89389772cfc7 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (c-67-177-211-60.hsd1.co.comcast.net [67.177.211.60]) by outbound4.ore.mailhop.org (Halon) with ESMTPSA id ef34534e-f969-11eb-a658-89389772cfc7; Mon, 09 Aug 2021 23:31:35 +0000 (UTC) Received: from [172.22.42.84] (rev2.hippie.lan [172.22.42.84]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id 179NVYcj000911; Mon, 9 Aug 2021 17:31:34 -0600 (MDT) (envelope-from ian@freebsd.org) X-Authentication-Warning: paranoia.hippie.lan: Host rev2.hippie.lan [172.22.42.84] claimed to be [172.22.42.84] Message-ID: Subject: Re: Tracking down BTX behavior change. From: Ian Lepore To: John W , freebsd-hackers@freebsd.org Date: Mon, 09 Aug 2021 17:31:34 -0600 In-Reply-To: References: Content-Type: text/plain; charset="ISO-8859-1" User-Agent: Evolution 3.40.3 FreeBSD GNOME Team List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4GkC5q12J9z4S96 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On Mon, 2021-08-09 at 15:19 -0700, John W wrote: > Ah, forgot the bug link: > > [1]  https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=257722 > > On 8/9/21, John W wrote: > > I have a problem on my system where 11.3 boots fine, but any > > supported > > release (11.4, 12.2, 13.0) fail during the BTX stage. There's a bug > > report if you're interested[1] > > > > But my question is: how can I track down differences in BTX code? I > > have the 11.3 and 11.4 SVN sources checked out, and so far I am not > > able to find any differences. > > > > So far, I am looking in: src/stand/i386/btx/ and src/usr.sbin/btxld > > Don't focus solely on the stand/i386/btx directory. All the code under src/stand is part of the loader. -- Ian From nobody Mon Aug 9 23:41:56 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 2AD5F11FF1E7 for ; Mon, 9 Aug 2021 23:42:08 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qt1-x836.google.com (mail-qt1-x836.google.com [IPv6:2607:f8b0:4864:20::836]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GkCKr0WTqz4TJ9 for ; Mon, 9 Aug 2021 23:42:07 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qt1-x836.google.com with SMTP id c5so13999431qtp.13 for ; Mon, 09 Aug 2021 16:42:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=y7x6ld1eOj2cNyLTbLH60ysIslrnrHGdRGaehhWKLp0=; b=1DiDF0IWkp0gdwEiKADvtbObMgPI6qdShn69WZPn1XSIkHmcb+YAwxQeZudNnldgnV UntTZWju5iHfwwtBpFaxTurKmlEQ8MpSSrTKOZbyahJ5FPYU9aDByJ655+Higf6hRvd4 R7ytebbGboz3nnHvs3rzb1t61zsUTwr1+747kgc0gpNSLlHkPOTn3L2FRugdASG0/pmH GZ/8VVXe8oRQLe/PM2K9DRZqJ2tLKgEjrSKJOXk0PT+pvK9YIV96Ttgw7sEuDAH+Afd5 A6gs/umEl8i8j0Wpc53NRpmz0tOu4V8cWPBp79EQ7t4O9YArGLk4xHdi4VmQZPyTvKGl dpag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=y7x6ld1eOj2cNyLTbLH60ysIslrnrHGdRGaehhWKLp0=; b=MOKKHbtW7gPV4NDWRWaz1l9pK4BKaqMIAf1iUpVJL2CNfK78ubi5beU3aSr3xBqAXK ikuMmbshwGM5i/edhmpGeqGkN6uyQFkDaEdsIaKAXl8m2UMMQBeSQKP62qj4LO4HFEIK bR5UCXLE5wTwM2EJpS2Ol6ncxIW4kuwE7RZdMkT+9mgElgph/R+t6MZuGf4hwK0FQTGE d4q8TFndXO3aBMSUN/JWZ+HV4bH8J9mayuoWOMTnIaZw+WCquDxJS68J5ZkCf7IItJuy BiN8Ykir57URy0RcMj25Yh7UvpklNZ/+BVN8rn3u52TZgDxQ3NcEpaWnhdh4hmJ5IKSu WHZw== X-Gm-Message-State: AOAM532W4oNS2zV3o02uqXHnAvw+1C56x/HExAHd4G5vR06BeLU32Kse EYQO0r1pQh05vI9rK/L21ocZe+sHXDc1EVJtePNFLw== X-Google-Smtp-Source: ABdhPJwcsAAdIY0VrBiVyCw021cp+N4KyQyaDjg77K+9Nydl3G3f75DLWGoLD4chdubYzH7+ZyYM5JgvoL6tS+lamto= X-Received: by 2002:ac8:4c8b:: with SMTP id j11mr14656020qtv.244.1628552527318; Mon, 09 Aug 2021 16:42:07 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Mon, 9 Aug 2021 17:41:56 -0600 Message-ID: Subject: Re: Tracking down BTX behavior change. To: Ian Lepore Cc: John W , FreeBSD Hackers Content-Type: multipart/alternative; boundary="000000000000e3a41805c928ee87" X-Rspamd-Queue-Id: 4GkCKr0WTqz4TJ9 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: Y --000000000000e3a41805c928ee87 Content-Type: text/plain; charset="UTF-8" On Mon, Aug 9, 2021 at 5:33 PM Ian Lepore wrote: > On Mon, 2021-08-09 at 15:19 -0700, John W wrote: > > Ah, forgot the bug link: > > > > [1] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=257722 > > > > On 8/9/21, John W wrote: > > > I have a problem on my system where 11.3 boots fine, but any > > > supported > > > release (11.4, 12.2, 13.0) fail during the BTX stage. There's a bug > > > report if you're interested[1] > > > > > > But my question is: how can I track down differences in BTX code? I > > > have the 11.3 and 11.4 SVN sources checked out, and so far I am not > > > able to find any differences. > > > > > > So far, I am looking in: src/stand/i386/btx/ and src/usr.sbin/btxld > > > > > Don't focus solely on the stand/i386/btx directory. All the code under > src/stand is part of the loader. > And BIOS boot on amd64 is a 32-bit i386 binary, so all its source lives under stand/i386/btx... But it's not BTX that's dying, per se, rather it's the thing btx is loading / is running is hitting a problem. The EIP is weird as well, at 0x1011f002 which is at 269,611,010 and that's well above where the ~500k loader should be and well below where I'd expect the BIOS routines to live... Not much was merged between 11.3 and 11.4. Can you describe your system in terms of CPU, RAM, etc? But the interface between the different parts of the boot system didn't change, so it's safe to use 11.3 gptboot pmbr with an 11.4 /boot/loader and kernel. The bug posted shows what I think is gptboot dying. At least I think so, the bug didn't say if it was UFS or ZFS... My guess is that things are a bit bigger, and an allocation is failing and the boot code isn't coping as well as it should with a nice error, but it could always be some other rare, edge case bug that you're running into. It may also make sense to check out the stable/13 branch to see where, exactly on that branch it fails to give some clue about what the root cause might be. Warner --000000000000e3a41805c928ee87-- From nobody Tue Aug 10 00:01:34 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 69E3613792E8 for ; Tue, 10 Aug 2021 00:01:36 +0000 (UTC) (envelope-from jwdevel@gmail.com) Received: from mail-ed1-x531.google.com (mail-ed1-x531.google.com [IPv6:2a00:1450:4864:20::531]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GkCmJ2G3pz4XRT; Tue, 10 Aug 2021 00:01:36 +0000 (UTC) (envelope-from jwdevel@gmail.com) Received: by mail-ed1-x531.google.com with SMTP id x14so4808514edr.12; Mon, 09 Aug 2021 17:01:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=oFIOr1Fr2aZTw5JoynaI71YHFr3U2Gyq1rIH1Cu3bc4=; b=gJpvWQMYMFni3KV+wQYP9M49i7vBn39JplURhbBMa1e02o06aN/0bGakjIRETI1kbI +zF+uiPsZwtD6hHahlFyzCFt4x45N0K8+kuoj6puiso7Rs3m9vX1uIUBHw2JEKk54jjV MQR379LN83L/Iup62jWbEfOCDQiFI1HkjqG3zgYlHn1Cx3FEtxblhJfWgMuv8P+QqiQI 9MskNEL0+re97W423RwKy0sojh9ko5dNMP27tE7c5aXmKEAmKwAKXgd+FEPweNu7s4aJ boWSb/MH0QNkj/Yg8/rSdZSIHAyYaqerYIHGm65Lc+lD5gaXWQ2GWR6m9sC7YbN5wPLg mr2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=oFIOr1Fr2aZTw5JoynaI71YHFr3U2Gyq1rIH1Cu3bc4=; b=NOpIPTt1GgtIyduKmMs/jXgZwn42grSu+t5goDVl42u795eOAQ8IIGm2LacMBwx1Tx 2RArCXWT8r3+jCAIGAmqEtd8fEZKtVCf9BVAGkniAOjhEH4PyDDrgrf3f5/2+z6pr9Ay qk4C86YF3GNz8idSTTC4GAWXAuXTl8pze85p8i5/XIWzzXoN0kSkAJ+4na8sF9BlSkbH 5S39TuW5IdWf+ySAg/ZC8mTz6UZQwtesahg1eiXnBJ/273K7iIpc2Jk9yJjTeQFjp4Ts Hb3Mj5jaweuSRpENvGiKb/V0MDMuIO8WJibtENxbdUVkRo1yZmDbhb/Zy2C6iToRS7Ct Ymyw== X-Gm-Message-State: AOAM530O7+pmRP8ncgBAkidO6dZ4UG5zGTuXtbKxY4uVg0tYFFGMscAx xU0Q3trI3Zuua6pcfFdlR9Svq42ENjEkcSAffZBh5j3e X-Google-Smtp-Source: ABdhPJz9W1x31N4Icr+PWlWdp/e2ln/31s+9eLMtvzfPOikK56ZDMGau2r0e4TLvZr53eDoYKNUR3gx3NTTn+OoVuT0= X-Received: by 2002:a05:6402:4412:: with SMTP id y18mr1199195eda.1.1628553695439; Mon, 09 Aug 2021 17:01:35 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Received: by 2002:a17:907:704d:0:0:0:0 with HTTP; Mon, 9 Aug 2021 17:01:34 -0700 (PDT) In-Reply-To: References: From: John W Date: Mon, 9 Aug 2021 17:01:34 -0700 Message-ID: Subject: Re: Tracking down BTX behavior change. To: Warner Losh Cc: Ian Lepore , FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4GkCmJ2G3pz4XRT X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On 8/9/21, Warner Losh wrote: > Can you describe your system in terms of CPU, RAM, etc? Yes, I put info in the bug. I guess I'll keep the discussion there, to avoid it getting fragmented. Thank you for the help -John From eugen@grosbein.net Tue Aug 10 05:08:15 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 91724175436B; Tue, 10 Aug 2021 05:08:49 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GkLZm49SSz3CdL; Tue, 10 Aug 2021 05:08:48 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221] (may be forged)) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id 17A58YFA086207 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 10 Aug 2021 05:08:34 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: alfadev@protonmail.com Received: from [10.58.0.10] (dadvw [10.58.0.10]) by eg.sd.rdtc.ru (8.16.1/8.16.1) with ESMTPS id 17A58Xgs057667 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 10 Aug 2021 12:08:33 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: Throughput extremely decreases when IPFW 7000 mac based rules activated To: alfadev , "freebsd-hackers@FreeBSD.org" , "freebsd-ipfw@FreeBSD.org" , "Alexander V. Chernikov" , "Andrey V. Elsukov" References: From: Eugene Grosbein Message-ID: <7a737f3d-e291-e8a1-b629-09365a99c937@grosbein.net> Date: Tue, 10 Aug 2021 12:08:15 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=4.0 required=5.0 tests=BAYES_05,LOCAL_FROM, NICE_REPLY_A,RDNS_NONE,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -0.5 BAYES_05 BODY: Bayes spam probability is 1 to 5% * [score: 0.0257] * 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record * -0.0 SPF_PASS SPF: sender matches SPF record * 2.6 LOCAL_FROM From my domains * 1.9 RDNS_NONE Delivered to internal network by a host with no rDNS * -0.0 NICE_REPLY_A Looks like a legit reply (A) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Spam-Level: *** X-Rspamd-Queue-Id: 4GkLZm49SSz3CdL X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=fail (mx1.freebsd.org: domain of eugen@grosbein.net does not designate 2a01:4f8:c2c:26d8::2 as permitted sender) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-1.65 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; RCPT_COUNT_FIVE(0.00)[5]; RCVD_COUNT_THREE(0.00)[3]; NEURAL_HAM_SHORT(-0.55)[-0.549]; FREEMAIL_TO(0.00)[protonmail.com,FreeBSD.org,freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/32, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; R_SPF_FAIL(1.00)[-all]; FREEFALL_USER(0.00)[eugen]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RBL_AMI_RCVD_FAIL(0.00)[62.231.161.221:server fail]; RCVD_TLS_ALL(0.00)[] X-Spam: Yes X-ThisMailContainsUnwantedMimeParts: N CC'ing more knowledgeable eyes that may have something to add. 09.08.2021 21:58, alfadev via freebsd-hackers wrote: > Hi, I have freebsd 11.2 server with IPFW firewall > > 870Mbits Fiber Net exist in my data center > > There are 7000 defined mac based rules on IPFW and 3000 of them active client . There is no problem before IPFW rules loading but when i load IPFW rules, > > throughput extremely decreases up to 80Mbits. There are not any error logs. I could not find what is the problem. > > Any help would be appreciated at this point. The search over ipfw rules is linear, so no wonder it decreases drastically when the list grows so big. Also, layer-2 frames and then layer-3 packets may pass over ipfw matching process upto four times\ unless you carefully create your ruleset like this: ipfw add 10 skipto 1000 not layer2 # do not MAC-filter layer3 packets that have L2 header stripped already anyway ipfw add 20 allow out # do not MAC-filter packets leaving the firewall, MAC-filter incoming only ipfw add 30 ... # start MAC-filtering here ... ipfw add 1000 ... # firewall part for layer3 packets Also, if you do filtering bridge, you should carefully read if_bridge(4) manual page, section PACKET FILTERING and disable extra passes over packet filters such as: sysctl net.link.bridge.pfil_member=0 # disable extra passes over ipfw ruleset for bridge members, filter the bridge itself only Such ruleset could decrease filtering overhead several times but I'm afraid that ipfw is not right tool for this task at the moment. ipfw has "tables" to optimize large list matching and they perform great but for layer3 IP matching, not for layer2 MAC matching. Also, your 11.2 version is quite old and you may need to upgrade to 11.4-STABLE at least to catch up with bugfixes and/or optimizations. From nobody Tue Aug 10 12:14:46 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 5593D174EA22 for ; Tue, 10 Aug 2021 12:15:02 +0000 (UTC) (envelope-from alfadev@protonmail.com) Received: from mail4.protonmail.ch (mail4.protonmail.ch [185.70.40.27]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GkX2Z0XGPz4T03 for ; Tue, 10 Aug 2021 12:14:59 +0000 (UTC) (envelope-from alfadev@protonmail.com) Date: Tue, 10 Aug 2021 12:14:46 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail; t=1628597689; bh=qmrMxTKwbO8N5aI2u068D2jQTGp3Gk8txVerrAwKdsE=; h=Date:To:From:Reply-To:Subject:In-Reply-To:References:From; b=T2V68NP8XMDWi1PKK2bfQtjIUhkaWZUw0STs2uw7DK7Qsy1jcmKYClQD4y2KLt9dz GC6RCYFxuf+0fPPEte94AfMj7l5irXJ11AsWEqMgMQgFCuNTK2G8JxjxSX/T8REm7r Wd/MDK5qX0R82Z9Tprj54lPqVCHZuzW7L7/PPadI= To: Eugene Grosbein , "freebsd-hackers@FreeBSD.org" , "frebsd-ipfw@FreeBSD.org" , "melifaro@freebsd.org" , "ae@FreeBSD.org" Reply-To: alfadev Subject: Re: Throughput extremely decreases when IPFW 7000 mac based rules activated Message-ID: In-Reply-To: <7a737f3d-e291-e8a1-b629-09365a99c937@grosbein.net> References: <7a737f3d-e291-e8a1-b629-09365a99c937@grosbein.net> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-1.2 required=10.0 tests=ALL_TRUSTED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM shortcircuit=no autolearn=disabled version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on mailout.protonmail.ch X-Rspamd-Queue-Id: 4GkX2Z0XGPz4T03 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Reply-To: alfadev@protonmail.com From: alfadev via freebsd-hackers X-Original-From: alfadev X-ThisMailContainsUnwantedMimeParts: N Thanks! > ipfw add 10 skipto 1000 not layer2 # do not MAC-filter layer3 packets tha= t have L2 header stripped already anyway > ipfw add 20 allow out # do not MAC-filter packets leaving the firewall, M= AC-filter incoming only > Also, your 11.2 version is quite old and you may need to upgrade to 11.4-= STABLE at least to catch up with bugfixes and/or optimizations. > Also, if you do filtering bridge, you should carefully read if_bridge(4) = manual page, * I have added skipto rule for layer3 * I have to add both in and out allow pipe rules for each MAC Address to as= sign bandwidth per MAC * I also tried this configuration on FreeBSD 12.2 but no luck same problem = occurs. * I have no bridge configuration > that ipfw is not right tool for this task at the moment. * How can i overcome this problem without using IPFW? Thanks for any help .. Here is my configuration: ################################################# ipfw -q -f flush ipfw pipe 2 config bw 500000Kbit mask dst-ip 0xffffffff ipfw pipe 1002 config bw 500000Kbit mask src-ip 0xffffffff ipfw pipe 4 config bw 1024Kbit mask dst-ip 0xffffffff ipfw pipe 1004 config bw 1024Kbit mask src-ip 0xffffffff # Loopback allow ipfw -q add 1 allow all from any to any out via lo0 ipfw -q add 2 allow all from any to any in via lo0 # WAN Allow ipfw -q add 3 allow ip from any to any MAC any any via em0 ipfw -q add 4 allow ip from any to any via em0 # Layer2 em1 enable arp traffic ipfw -q add 5 allow ip from any to any layer2 mac-type arp via em1 ipfw -q add 6 skipto 64000 all from any to any not layer2 # Layer2 blocked mac ipfw -q add 1189 deny ip from any to any MAC 1c:cc:d6:42:5e:xx any via em1 ipfw -q add 2189 deny ip from any to any MAC any 1c:cc:d6:42:5e:xx via em1 ipfw -q add 1190 deny ip from any to any MAC 3c:dc:bc:ab:56:yy any via em1 ipfw -q add 2190 deny ip from any to any MAC any 3c:dc:bc:ab:56:yy via em1 ipfw -q add 1193 deny ip from any to any MAC 02:93:ca:4a:24:ab any via em1 ipfw -q add 5004 pipe 2 tag 1 ip from any to any MAC 78:67:d7:23:14:zz any = via em1 ipfw -q add 5005 pipe 1002 tag 1 ip from any to any MAC any 78:67:d7:23:14:= zz via em1 ... ... ... ... sample added mac address allow and pipe rules ... ... TOTAL 2500-3000 mac address in and out allow pipe rules ... ... ipfw -q add 12004 pipe 4 tag 1 ip from any to any MAC b8:37:e7:53:e4:qq any= via em1 ipfw -q add 12005 pipe 1004 tag 1 ip from any to any MAC any b8:37:e7:53:e4= :qq via em1 ipfw -q add 60000 allow ip from any to any MAC any any via em1 ... ... NOT tagged Mac address redirected block page ... ipfw -q add 65534 allow all from any to any ################################################# Sent with ProtonMail Secure Email. =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90 Original Me= ssage =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90 On Tuesday, August 10th, 2021 at 8:08 AM, Eugene Grosbein wrote: > CC'ing more knowledgeable eyes that may have something to add. > > 09.08.2021 21:58, alfadev via freebsd-hackers wrote: > > > Hi, I have freebsd 11.2 server with IPFW firewall > > > > 870Mbits Fiber Net exist in my data center > > > > There are 7000 defined mac based rules on IPFW and 3000 of them active = client . There is no problem before IPFW rules loading but when i load IPFW= rules, > > > > throughput extremely decreases up to 80Mbits. There are not any error l= ogs. I could not find what is the problem. > > > > Any help would be appreciated at this point. > > The search over ipfw rules is linear, so no wonder it decreases drastical= ly when the list grows so big. > > Also, layer-2 frames and then layer-3 packets may pass over ipfw matching= process upto four times\ > > unless you carefully create your ruleset like this: > > ipfw add 10 skipto 1000 not layer2 # do not MAC-filter layer3 packets tha= t have L2 header stripped already anyway > > ipfw add 20 allow out # do not MAC-filter packets leaving the firewall, M= AC-filter incoming only > > ipfw add 30 ... # start MAC-filtering here > > ... > > ipfw add 1000 ... # firewall part for layer3 packets > > Also, if you do filtering bridge, you should carefully read if_bridge(4) = manual page, > > section PACKET FILTERING and disable extra passes over packet filters suc= h as: > > sysctl net.link.bridge.pfil_member=3D0 # disable extra passes over ipfw r= uleset for bridge members, filter the bridge itself only > > Such ruleset could decrease filtering overhead several times but I'm afra= id > > that ipfw is not right tool for this task at the moment. > > ipfw has "tables" to optimize large list matching and they perform great = but for layer3 IP matching, not for layer2 MAC matching. > > Also, your 11.2 version is quite old and you may need to upgrade to 11.4-= STABLE at least to catch up with bugfixes and/or optimizations. From eugen@grosbein.net Tue Aug 10 13:07:13 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 001DB1752D61 for ; Tue, 10 Aug 2021 13:07:37 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GkYCD6b8Xz4Zcl for ; Tue, 10 Aug 2021 13:07:36 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221] (may be forged)) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id 17AD7WeD090691 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 10 Aug 2021 13:07:32 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: alfadev@protonmail.com Received: from [10.58.0.10] (dadvw [10.58.0.10]) by eg.sd.rdtc.ru (8.16.1/8.16.1) with ESMTPS id 17AD7Vjm062381 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 10 Aug 2021 20:07:31 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: Throughput extremely decreases when IPFW 7000 mac based rules activated To: alfadev , "freebsd-hackers@FreeBSD.org" References: <7a737f3d-e291-e8a1-b629-09365a99c937@grosbein.net> From: Eugene Grosbein Message-ID: <15f7c7da-a426-f4ff-48c0-1cfd0d64ebf2@grosbein.net> Date: Tue, 10 Aug 2021 20:07:13 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-102.3 required=5.0 tests=BAYES_00,SHORTCIRCUIT autolearn=disabled version=3.4.2 X-Spam-Report: * -100 SHORTCIRCUIT No description available. * [score: 0.0000] * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Rspamd-Queue-Id: 4GkYCD6b8Xz4Zcl X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=fail (mx1.freebsd.org: domain of eugen@grosbein.net does not designate 2a01:4f8:c2c:26d8::2 as permitted sender) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-2.10 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; R_SPF_FAIL(1.00)[-all]; FREEFALL_USER(0.00)[eugen]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_TWO(0.00)[2]; FREEMAIL_TO(0.00)[protonmail.com,FreeBSD.org]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/32, country:DE]; RCVD_TLS_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RBL_AMI_RCVD_FAIL(0.00)[62.231.161.221:server fail] X-ThisMailContainsUnwantedMimeParts: N Trimming CC: list (I'm not in ipfw@) 10.08.2021 19:14, alfadev wrote: >> ipfw add 10 skipto 1000 not layer2 # do not MAC-filter layer3 packets that have L2 header stripped already anyway >> ipfw add 20 allow out # do not MAC-filter packets leaving the firewall, MAC-filter incoming only >> Also, your 11.2 version is quite old and you may need to upgrade to 11.4-STABLE at least to catch up with bugfixes and/or optimizations. >> Also, if you do filtering bridge, you should carefully read if_bridge(4) manual page, > > * I have added skipto rule for layer3 > * I have to add both in and out allow pipe rules for each MAC Address to assign bandwidth per MAC Why do you assign bandwidth per MAC instead of per IP or (set of) subnet(s)? ipfw is designed to process multiple sets of IP networks but not MACs. > * I also tried this configuration on FreeBSD 12.2 but no luck same problem occurs. FreeBSD 12 is distinct major branch and may have its own performance problems, I did not propose using latest version. I proposed to try 11.4-STABLE that is most tested of supported versions to the moment. > * I have no bridge configuration > >> that ipfw is not right tool for this task at the moment. > * How can i overcome this problem without using IPFW? Cannot tell for sure. You need something with hashed or tree-based lookup, not linear search. Some comments on your ruleset follow. > Thanks for any help .. > > Here is my configuration: > ################################################# > > ipfw -q -f flush > > ipfw pipe 2 config bw 500000Kbit mask dst-ip 0xffffffff > ipfw pipe 1002 config bw 500000Kbit mask src-ip 0xffffffff > > ipfw pipe 4 config bw 1024Kbit mask dst-ip 0xffffffff > ipfw pipe 1004 config bw 1024Kbit mask src-ip 0xffffffff > > > # Loopback allow > ipfw -q add 1 allow all from any to any out via lo0 > ipfw -q add 2 allow all from any to any in via lo0 You need not two different rules for loopback. Singe rule will work with less overhead: allow all from any to any via lo0 > > # WAN Allow > ipfw -q add 3 allow ip from any to any MAC any any via em0 > ipfw -q add 4 allow ip from any to any via em0 Same here, single rule is sufficient: allow ip from any to any via em0 > # Layer2 em1 enable arp traffic > ipfw -q add 5 allow ip from any to any layer2 mac-type arp via em1 > ipfw -q add 6 skipto 64000 all from any to any not layer2 > > # Layer2 blocked mac > ipfw -q add 1189 deny ip from any to any MAC 1c:cc:d6:42:5e:xx any via em1 > ipfw -q add 2189 deny ip from any to any MAC any 1c:cc:d6:42:5e:xx via em1 This is inefficient as each incoming and outgoing frame will be matched against both rules but only one match should be performed for incoming and one for outgoing frame. You should first check only incoming frames: add 1100 skipto (NNN+1) out # filter outgoing frames later add 1110 skipto 12000 in not recv em1 # do not filter frames coming from other interfaces here add 1189 deny MAC any 1c:cc:d6:42:5e:xx # less attributes, faster matching (do not match for em1 again and again) add 1190 deny MAC any 3c:dc:bc:ab:56:yy ... add NNN skipto 12000 # end of MAC-filter for incoming frames So outgoing frames will not pass over whole list of rules for "MAC xxx any" they never match anyway. Then same for another direction: add (NNN+1) skipto 12000 out not xmit em1 # do not filter frames transmitted over other interfaces here add (NNN+2) deny MAC 1c:cc:d6:42:5e:xx any # again, optimize out match for interface name, check MAC only ... You should be VERY careful writing rules for expensive filtering. From nobody Tue Aug 10 20:56:56 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 784E4137B3DD; Tue, 10 Aug 2021 20:56:58 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: from spindle.one-eyed-alien.net (spindle.one-eyed-alien.net [199.48.129.229]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Gklcn2K2Xz5414; Tue, 10 Aug 2021 20:56:57 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: by spindle.one-eyed-alien.net (Postfix, from userid 3001) id 7BAF03C0199; Tue, 10 Aug 2021 20:56:56 +0000 (UTC) Date: Tue, 10 Aug 2021 20:56:56 +0000 From: Brooks Davis To: Ed Maste Cc: "freebsd-toolchain@FreeBSD.org" , FreeBSD Hackers Subject: Re: Compressed debug info sections and big-endian targets Message-ID: <20210810205656.GD4352@spindle.one-eyed-alien.net> References: List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="+g7M9IMkV8truYOl" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.4 (2018-02-28) X-Rspamd-Queue-Id: 4Gklcn2K2Xz5414 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of brooks@spindle.one-eyed-alien.net has no SPF policy when checking 199.48.129.229) smtp.mailfrom=brooks@spindle.one-eyed-alien.net X-Spamd-Result: default: False [-2.78 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; SIGNED_PGP(-2.00)[]; FORGED_SENDER(0.30)[brooks@freebsd.org,brooks@spindle.one-eyed-alien.net]; RCVD_COUNT_ZERO(0.00)[0]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:36236, ipnet:199.48.128.0/22, country:US]; FROM_NEQ_ENVFROM(0.00)[brooks@freebsd.org,brooks@spindle.one-eyed-alien.net]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FREEFALL_USER(0.00)[brooks]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; DMARC_NA(0.00)[freebsd.org]; AUTH_NA(1.00)[]; NEURAL_SPAM_SHORT(0.12)[0.117]; R_SPF_NA(0.00)[no SPF record] X-ThisMailContainsUnwantedMimeParts: N --+g7M9IMkV8truYOl Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Aug 07, 2021 at 12:07:39PM -0400, Ed Maste wrote: > GCC and Clang/LLVM support coimpressed debug info, which compresses > the .debug_* ELF sections in objects, archives, libraries, and > executables. I recently committed build infrastructure changes to turn > this on (c910570e7573) but it broke the build on big-endian targets > (mips, powerpc) and so disabled it again (89ed2ecb14ce). PR257638 has > more details. >=20 > The lld bug is now fixed in main thanks to Simon Atanasyan upstream > and dim@ for merging it over (d69d07569ee2), and I would like to > enable it again. I've committed the fix to devel/llvm12 and will merge to quarterly. I'll follow up with an update to llvm-devel. I've not investigated if it can be merged to older ports. > An outstanding issue is that the bug is triggered by the linker's > input, and so this will occur if we attempt to link against a base > system .a archive using a buggy lld. In the short term I think we have > no choice but to leave compressed debug disabled on BE targets, until > fixed lld versions are available in ports/packages. >=20 > I have a review open to enable it for LE targets only: > https://reviews.freebsd.org/D31454. >=20 > I'd like to apply that change for now, but would like to enable > compressed debug across all targets in the future. This would break, > on big-endian targets, any port that has a build-dependency on an > older lld and links against a base system archive. Such a port could > be fixed by switching to linking with binutils ld, or lld from the > base system or a newer package. What do big-endian mips or powerpc > users think? I'd be surprised if it was significant for mips given the generally sorry state of the port. Might be more exciting for powerpc where X11 works. (As an aside, I suspect we'll have updated CHERI-LLVM to ~13 before we've caught CheriBSD up to this change and we may even manage to give mips the boot before then.) -- Brooks --+g7M9IMkV8truYOl Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJhEugXAAoJEKzQXbSebgfAEJMH/RhfpD7pn/OXZBby78CtPcDu 1og1+UJsnhrjiu6gIFRiCQ6WR0f6n/I+PKkbyOCYPLLXX8oaKotOIvwDRQrIoy3b KThFRJOFsHOV4V+QX3BhqohW5VvvLiMtyzpTokrFaD1zagZGfwDmULMRWIJwkiK3 Ady/uy8+HVLBctIoADkipsxMreX7CYndOU/u0a20g5JP6WUk9l/vXwI3nq6PSYmT iY8urxXb+muRa/hvrm49h+5U/TUq+9RVzpS8Ike3nPOd0uy3/bMdMw+AAyfWbedM U7texYoxU99G+1muIv6grHy+8lA+1qYSFYiYhW4ldMBll3RAix0detrzierJPVA= =/dJt -----END PGP SIGNATURE----- --+g7M9IMkV8truYOl-- From nobody Wed Aug 11 10:30:06 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id A7E071750D97 for ; Wed, 11 Aug 2021 10:30:19 +0000 (UTC) (envelope-from patrascu.naina14@gmail.com) Received: from mail-pl1-x62d.google.com (mail-pl1-x62d.google.com [IPv6:2607:f8b0:4864:20::62d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Gl5gG52GSz4mCC for ; Wed, 11 Aug 2021 10:30:18 +0000 (UTC) (envelope-from patrascu.naina14@gmail.com) Received: by mail-pl1-x62d.google.com with SMTP id e19so2043834pla.10 for ; Wed, 11 Aug 2021 03:30:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to:cc; bh=z43TzR/5xF5Jd7ylXbuvMMsWYKAE8k/V9iblZk5iffY=; b=IvPTE07H6ZyqmDnsCxjY3As5lc/8uLGV/PQ/LI9ZsCdT5C7J9PHnvQ79Zdius7krrE wrTeLjNk1gD8cwo9yyvFH6/YRxeE9pvoxXoSGgavEMKYch1z4EbOPnmDb0f4ULQ6yTXX g6z4a4tTCxOHUlx5R8Xw7M/1L08tIG89OogmeV+MPzqISCSThWk6e4sBqnsMMnWNzwUo pso/kFbluj/NVnlI8LCCHWatxmazZdi/pDmmjYYWfr8jTtUCjm7Xbrfalqb9kLpB3EAh cdNgILiBnwYeX17CtzDDEs+9WBiei58RwDTaveWQTy7xgSam+xCivEiT0qrUQW3Rwhcx Atjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=z43TzR/5xF5Jd7ylXbuvMMsWYKAE8k/V9iblZk5iffY=; b=Wg6vkkdIs08ssFRsNEiH3/kp+Q+qL5dTC/8WKKK/0tVYH828UgrLyRxAvH8UGXO8vo Xi8XwULdP21qi1y7LeWfGTqS/wfgekqrqixpolZnhk1ZPQmOP3NfgWg2DUC9fXubOzdS jNjhsoZwQV9REdyVN+pDzWjTYsUlejbsAskNWSkM9CZWMy3mIg93nPb4MJUhTur57IBh lSbOpFJUEWXUoBbmMjIpWXdP2bggUIHhheL7qEEjFwwxCN3exGayPlql5CEykoCifRwu Iwo586OjHj0vhJzUxukLESVLDQTle7i6A7mZGlN+3ZkgkyXyH83IxwP+f5AVPYJU1ckX C7mg== X-Gm-Message-State: AOAM532+tjepDG4EB7coP3+QHtPGYkScW29ceyrVE7v0Zb8lhKxJ++XG GZXi0FysPYypHwTBdytpepmHuQSFAiynV1Durqx0SpSZ X-Google-Smtp-Source: ABdhPJxhXZYnH/oKs3/xx+ha6b9lwl8Cy6kOdk5R84cCnrVLMwrvPFBlQkEccm/6Puxi9W3ZT4QTIwGR2yAn3anBA90= X-Received: by 2002:a17:90a:c3:: with SMTP id v3mr9907071pjd.76.1628677817452; Wed, 11 Aug 2021 03:30:17 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 From: Naina Patrascu Date: Wed, 11 Aug 2021 13:30:06 +0300 Message-ID: Subject: USB passthrough in bhyve, USB keyboard disconnected To: freebsd-hackers@freebsd.org Cc: lucian_ioan.popescu@stud.acs.upb.ro, Elena Mihailescu Content-Type: multipart/alternative; boundary="000000000000c34b2a05c9461a16" X-Rspamd-Queue-Id: 4Gl5gG52GSz4mCC X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=IvPTE07H; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of patrascunaina14@gmail.com designates 2607:f8b0:4864:20::62d as permitted sender) smtp.mailfrom=patrascunaina14@gmail.com X-Spamd-Result: default: False [-3.80 / 15.00]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MID_RHS_MATCH_FROMTLD(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; NEURAL_HAM_SHORT(-0.80)[-0.795]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::62d:from]; RCVD_COUNT_TWO(0.00)[2]; FREEMAIL_CC(0.00)[stud.acs.upb.ro,gmail.com]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: Y --000000000000c34b2a05c9461a16 Content-Type: text/plain; charset="UTF-8" Hello, We are working on the USB passthrough functionality for bhyve and we are currently focusing on the passthrough of a USB keyboard. We implemented the enumeration process and it seems to work properly, as we can see the information about the USB keyboard inside the virtual machine through the 'usbconfig' command: (guest) $ usbconfig (guest) ugen0.2: at usbus0, cfg=0 md=HOST spd=LOW (1.5Mbps) pwr=ON (100mA) But at some point during the boot of the guest virtual machine, the USB keyboard is powered off: any LED from the keyboard is turned off and pressing any key does not generate logs in the guest or in the host. We tried multiple commands to display the information about the USB device: usbconfig, devinfo. We compared the outputs from the host and guest and the attachment to the guest looks correct. To inspect the transfers, we used the command below, but nothing is displayed on the guest: usbdump -i usbusX -f Y -s 65536 -vvv In the logs we see the steps of the enumeration successfully executed, and then some timeout errors, as we cannot execute any transfer with the keyboard, which seems disconnected. Could you help us find out why this disconnection happens? Could you give us a path for investigating this problem? We used multiple work environments: - the FreeBSD host started in QEMU kvm; nested virtualization enabled to create guest machine - FreeBSD host directly on a PC The behavior of our feature is the same. If you want to take a look, [1] is the repo containing our implementation; here [2] you can find iso files from our sources, and here [3] is a tutorial, describing the steps we use to create, run and destroy a virtual machine. Thank you! [1] https://github.com/FreeBSD-UPB/freebsd-src/tree/projects/bhyve_usb_passthrough [2] https://drive.google.com/drive/folders/17xUlbWSZ-wn6xT2ouv-M6miHV_LDeIEw?usp=sharing [3] https://github.com/FreeBSD-UPB/freebsd-src/wiki/Working-with-virtual-machines-in-bhyve-for-USB-passthrough --000000000000c34b2a05c9461a16-- From nobody Sat Aug 14 19:59:54 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 546981755D0C for ; Sat, 14 Aug 2021 19:59:57 +0000 (UTC) (envelope-from dan@langille.org) Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GnB983pQGz4j7G for ; Sat, 14 Aug 2021 19:59:56 +0000 (UTC) (envelope-from dan@langille.org) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id 6DD575C00B3 for ; Sat, 14 Aug 2021 15:59:56 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Sat, 14 Aug 2021 15:59:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=langille.org; h= to:from:subject:message-id:date:mime-version:content-type :content-transfer-encoding; s=fm2; bh=h4d9B/uMCV6LSgPAUlYYrIid99 Pt70V7cqDMNF7n5/A=; b=EB9bPb4Fe9NiS1IgpC5p+uvJaRfkA+K4UlkOYdf5ar j+u8aXMTtn+fgCm982h/+Ov3vZ9P0dJjabLamwyz274zLewe0pBI+zz7KG2of3dY 6CRRdxMjejgT4ug1SByEbeG7gM/QRhzaE2b3As8lPuHb3XlIY1wRhIAHfWKjxrAh AwISj5KjFXbunSKPZVCsVshDVtJUw+TDRVDS1LK2wkj8RQUmY7cBpStnEP8qyX2R GeOgLmITyfOcp+iA/EbjcSnlV2UWMSLZpgGVOYyIvnCzLlTgWoRGmLjjJZh5K4EH zWJBKZOVHOjO/mijeZXEdkWw2vbCGFhsb8/VaOu5clEw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:message-id:mime-version:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=h4d9B/ uMCV6LSgPAUlYYrIid99Pt70V7cqDMNF7n5/A=; b=hDKxBXeDopu0X3kMu/Kl5U HvDdzVq54woVe4iGlnPNh8YRKGvgzpk/lJwr+2Xuxs2QI2aBi+YxSN0gawycTa8D zHarWgs3k4i7EgmTVUViDEWd4fXYIp7dvyy7D1SbeO2vd2xNQ0IHh+ai+Ee7DHBm w2zg+LjmwrfIF4qFimRLaDs0wNBphWqGcKXscX/+IAB46MCopuVeWHZ5nzHvC8x8 u7yixkhvxS7zmfeNPOyMPUyrw8GZ0shhMpLTOzMogUbuEXt5gsO73tXsmN52RUQE 1Y9ZJPiqHUvo10FtHF5TjvbP1lfdxnxImKMxO3HXzP83k0FIFx7SmlMZWH5FI3hg == X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvtddrkeejgddugeegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefvhffukffffgggtgfgsehtkeertd dtfeejnecuhfhrohhmpeffrghnucfnrghnghhilhhlvgcuoegurghnsehlrghnghhilhhl vgdrohhrgheqnecuggftrfgrthhtvghrnhepgfevteffvdegleduvdejgeffhfevfffghe fhffeivdehjeeikeehfeejudffjedtnecuvehluhhsthgvrhfuihiivgeptdenucfrrghr rghmpehmrghilhhfrhhomhepuggrnheslhgrnhhgihhllhgvrdhorhhg X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Sat, 14 Aug 2021 15:59:55 -0400 (EDT) To: freebsd-hackers@freebsd.org From: Dan Langille Subject: starting jails within jails using rc Message-ID: <60ecf265-b308-738d-ec2f-64e76b625a38@langille.org> Date: Sat, 14 Aug 2021 15:59:54 -0400 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:52.0) Gecko/20100101 PostboxApp/7.0.48 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 4GnB983pQGz4j7G X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=langille.org header.s=fm2 header.b=EB9bPb4F; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=hDKxBXeD; dmarc=pass (policy=none) header.from=langille.org; spf=pass (mx1.freebsd.org: domain of dan@langille.org designates 66.111.4.26 as permitted sender) smtp.mailfrom=dan@langille.org X-Spamd-Result: default: False [-5.10 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[langille.org:s=fm2,messagingengine.com:s=fm3]; FREEFALL_USER(0.00)[dan]; FROM_HAS_DN(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[66.111.4.26:from]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.26:c]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; RCVD_COUNT_THREE(0.00)[4]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; DKIM_TRACE(0.00)[langille.org:+,messagingengine.com:+]; DMARC_POLICY_ALLOW(-0.50)[langille.org,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:11403, ipnet:66.111.0.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.26:from] X-ThisMailContainsUnwantedMimeParts: N Hello, Background information: Each FreshPorts instance runs two jails: ingress & web.  The ingress jail pulls data from both git & the repos in order to populate the database. Until recently, the ingress jail used a chroot to isolate itself from the packages installed within the jail. That can taint the information pulled out of the repo.  Recently work has moved from using a chroot to using a child jail. The chroot (jail), is used to run various commands (e.g make -V) on a ports tree contained within the chroot (jail). This extracts the information which is then loaded into the database. Bonus: changing all the commands from chroot to jexec was pretty easy. The conversion required only trivial changes. In short, each FreshPorts ingress jail will have a child jail containing a copy of the ports repo. The problem: The parent jail cannot automatically start the child jail. The child jail can be started manually. Running this command in the parent child succeeds: service jail start freshports Why? I think it's because /etc/rc.d/jail contains: # KEYWORD: nojail shutdown This tells the rc system not to run the jail script if the host is a jail. How can I trick it? My two ideas so far: * remove the keyword from the script (I've tested this; it works) * duplicate the script, removing the keyword from the script * mangle security.jail.jailed in the parent jail it thinks it's not in a jail and runs it anyway The downsides to these: * the first two require I keep up to date with the jail script. * the last one will have unintended consequences I'm sure, many which I most likely would not like. Do you have other ideas please? Thank you -- Dan Langille dan@langille.org From nobody Sun Aug 15 16:29:59 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 6A8591766428 for ; Sun, 15 Aug 2021 16:30:07 +0000 (UTC) (envelope-from jamie@freebsd.org) Received: from gritton.org (gritton.org [199.192.165.131]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GnjSb1rz9z3jY7 for ; Sun, 15 Aug 2021 16:30:07 +0000 (UTC) (envelope-from jamie@freebsd.org) Received: from gritton.org ([127.0.0.131]) (authenticated bits=0) by gritton.org (8.15.2/8.15.2) with ESMTPA id 17FGTxWE049359; Sun, 15 Aug 2021 09:29:59 -0700 (PDT) (envelope-from jamie@freebsd.org) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Sun, 15 Aug 2021 09:29:59 -0700 From: James Gritton To: freebsd-hackers@freebsd.org Cc: Dan Langille Subject: Re: starting jails within jails using rc In-Reply-To: <60ecf265-b308-738d-ec2f-64e76b625a38@langille.org> References: <60ecf265-b308-738d-ec2f-64e76b625a38@langille.org> User-Agent: Roundcube Webmail/1.4.1 Message-ID: X-Sender: jamie@freebsd.org X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.6.2 (gritton.org [127.0.0.131]); Sun, 15 Aug 2021 09:29:59 -0700 (PDT) X-Rspamd-Queue-Id: 4GnjSb1rz9z3jY7 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On 2021-08-14 12:59, Dan Langille wrote: > The problem: > > The parent jail cannot automatically start the child jail. The child > jail can be started manually. > > Running this command in the parent child succeeds: service jail start > freshports > > Why? I think it's because /etc/rc.d/jail contains: > > # KEYWORD: nojail shutdown > > This tells the rc system not to run the jail script if the host is a > jail. > > How can I trick it? > > My two ideas so far: > > * remove the keyword from the script (I've tested this; it works) > * duplicate the script, removing the keyword from the script > * mangle security.jail.jailed in the parent jail it thinks it's not in > a jail and runs it anyway > > The downsides to these: > > * the first two require I keep up to date with the jail script. > * the last one will have unintended consequences I'm sure, many which > I most likely would not like. Since jails with jails is a supported (though not defaulted) feature, I see no reason why simply removing the "nojail" keyword from the script shouldn't be the default. The only cost is typical jail startup having to run the script to no effect, but the rc system is already built of dozens of such seldom-used scripts. - Jamie From nobody Sun Aug 15 16:56:33 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 41DA41768341 for ; Sun, 15 Aug 2021 16:56:42 +0000 (UTC) (envelope-from dan@langille.org) Received: from wout4-smtp.messagingengine.com (wout4-smtp.messagingengine.com [64.147.123.20]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Gnk3G0gjTz3m3l; Sun, 15 Aug 2021 16:56:42 +0000 (UTC) (envelope-from dan@langille.org) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id 7C3A83200657; Sun, 15 Aug 2021 12:56:35 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Sun, 15 Aug 2021 12:56:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=langille.org; h= subject:to:cc:references:from:message-id:date:mime-version :in-reply-to:content-type:content-transfer-encoding; s=fm2; bh=e 1o3+XbCAjjuZRYnsEFIa/nwQugWdhaeAElBkIOB8g4=; b=c+DrHyZedC/5NWamj CNq5vHIP70tMP6OxvgXSYOabGyBwiv3f4wbjyaB/BjC9zAEjDX7yjkPnVz2Pf+JP 9y1EuyOa48+vQzlpoxb3Z+NKlux0mNefSwSIVLsaOzFBdE38GZWOnXjoRaXKP2ys NS6dP13BFKeAkO5vj+S4vq/STqpUpgKtUClTxszHtquw6/PpOsyO+j9rMNLVkNLg 8G3Ly0MRKj+QLKU8onnQ38480Qdxo8pv7ugHXKe1xk+T642/VsbH28SJtLfIFMNP dg/KtOFl6H/KpH5DgcCvmBC1Cy80gb0AqEB2AHCJVi5yRxmnpcZi17Y+9MB48hkA su7zw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=e1o3+XbCAjjuZRYnsEFIa/nwQugWdhaeAElBkIOB8 g4=; b=GF+GAxEsNVikCwuhruciW/VoEHstPCur4ynry5UugemhEie8ClrXNx1Ql EepFJSuLjy+knrsmIzrYpa6G7NPcEfHzAYduQDh1idhokgyq7ztZhCC8f/Fqosd8 zx3/Pf37khB9wOoDhyKnfC3MAi5oHpahaxI8i7nrkkdnBLhijoE8hX1GbAf5/Mg7 bwnpDj99hdHxzczZGKDO5q2GghsRucczRg4xhZ6TmIujPc+TD++gXeddJN9maP56 LXbaFM15d456J4gy8mDXVWXgmSrkenA0W/QShYa6wESyYZwLM1uOGelxOHVoLTNA NeSaOCNHnmVUoe+XP5MXmtaQTzI5Q== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvtddrkeelgddutdeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefuvfhfhffkffgfgggjtgfgsehtke ertddtfeejnecuhfhrohhmpeffrghnucfnrghnghhilhhlvgcuoegurghnsehlrghnghhi lhhlvgdrohhrgheqnecuggftrfgrthhtvghrnhepvedttdelffetfefgjeefvedthffgie egudekleetfeehudejjeekleduvdevffegnecuffhomhgrihhnpehgihhthhhusgdrtgho mhdplhgrnhhgihhllhgvrdhorhhgnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrg hmpehmrghilhhfrhhomhepuggrnheslhgrnhhgihhllhgvrdhorhhg X-ME-Proxy: Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sun, 15 Aug 2021 12:56:34 -0400 (EDT) Subject: Re: starting jails within jails using rc To: James Gritton Cc: freebsd-hackers@freebsd.org References: <60ecf265-b308-738d-ec2f-64e76b625a38@langille.org> From: Dan Langille Message-ID: <2fde54a8-1f19-28e0-46b2-74b5ef2c2e65@langille.org> Date: Sun, 15 Aug 2021 12:56:33 -0400 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:52.0) Gecko/20100101 PostboxApp/7.0.48 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Rspamd-Queue-Id: 4Gnk3G0gjTz3m3l X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N James Gritton wrote on 8/15/21 12:29 PM: > On 2021-08-14 12:59, Dan Langille wrote: >> The problem: >> >> The parent jail cannot automatically start the child jail. The child >> jail can be started manually. >> >> Running this command in the parent child succeeds: service jail start >> freshports >> >> Why? I think it's because /etc/rc.d/jail contains: >> >> # KEYWORD: nojail shutdown >> >> This tells the rc system not to run the jail script if the host is a >> jail. >> >> How can I trick it? >> >> My two ideas so far: >> >> * remove the keyword from the script (I've tested this; it works) >> * duplicate the script, removing the keyword from the script >> * mangle security.jail.jailed in the parent jail it thinks it's not in >> a jail and runs it anyway >> >> The downsides to these: >> >> * the first two require I keep up to date with the jail script. >> * the last one will have unintended consequences I'm sure, many which >> I most likely would not like. > > Since jails with jails is a supported (though not defaulted) feature, > I see no reason why simply removing the "nojail" keyword from the > script shouldn't be the default.  The only cost is typical jail > startup having to run the script to no effect, but the rc system is > already built of dozens of such seldom-used scripts. Wow. I had not considered a patch until now. Submitted. https://github.com/freebsd/freebsd-src/pull/525 -- Dan Langille - dan@langille.org https://langille.org/