From nobody Mon Mar 18 11:42:48 2024 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TytJh2x8Lz5DYVH; Mon, 18 Mar 2024 11:43:00 +0000 (UTC) (envelope-from eduardo@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TytJh2WPWz41V2; Mon, 18 Mar 2024 11:43:00 +0000 (UTC) (envelope-from eduardo@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1710762180; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nRW1zKiuHdfkRBaVz8VhI700iA151KX5T6T7h+C7OmI=; b=VplVI3hsHi/DAQ9CC+Az/nlItLz3U0KGpRO27aVAuz6/SB3mh9cKKCbEC36smoNFW2KoaU TFquuXzGjOtYMztqtPVNDwBAf5XfbHz/4Lsr8fBKJxVgCrZ1APyPztytc2u3ThbzQaPYcS 7zIrSeh2ZRybSJqnIN7n2+m6BqTKfL4wPkiB7MMtGasIz7+10iYzbfT8EvV6b2HfF1FmPk AFJWkjwaijwM7yhgqO/enyyykHcgWrIxTrWkOBtns4KQgZCMJYXUwVNS7X+YNyNLNDv9q3 z6FKeGv0Jr5909JA9pQrD8Jy5XbGfROHftN02DnaoZPcaNH+QTmXtdNZ/fiECQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1710762180; a=rsa-sha256; cv=none; b=LqUEI6u1lQiGBldQlundAvbdBcePuAR2V7262iwhZ6XNqP7F5gP6fSET31gwcHcaj83UEH jKtTXl9clEKSLo3f8mG7+qPb3r8Gm83bQ5CxcM28SW8bpCjBXffWhRU/oh/4ysgBrlXUdl E5eqbxTxT8JaT5Fxy+fnpFfRVbIEhAbXZD13D8IP1NdG+A4nUkGM4QQkNtwkmMl6maJnM6 Agz3/6/3PK3KiT59/7/5BsDPE+k90jsQuZnsYjasoJaH7IcLmr1ObOJN84YXxfVv5ojh8h BoPl7ywvgRsIrqalP2Jh3V2/Kc2yM9xU1eSucKc4dNh2E9ER6nJhTW+UNm02wg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1710762180; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nRW1zKiuHdfkRBaVz8VhI700iA151KX5T6T7h+C7OmI=; b=ToFI5GV7ky7ztM2bGkvR8ZF7KyXakTW0HzT6i51uG6XIOJYzCiYgrz04C/Xq7newcUt2I8 HmPGOy3ysCxk3TO46MKxuL+9CUr8HU2N1gnhJcSbWYQHgKdD1OKE+LdjRUTU52iEz1tTUN allbzkayumN2dII/yxIvPQxi6LDtahATjtemT5uVjenDnWSTFJ8lR1G1bXzaLw+zxF6IbV /qw1Co3gEqDt7N7brKsh9XmL8cyXPGrTuclgKLvilPFvbGVC+VHZAbCi3U0+F/bOISofQJ VTIHFPY6bWKHU1wuVKNX0Zl1S75heaBb2wFQK27SJ+meJZnxCtCgWPtPC56KYg== Received: from mail-yb1-f170.google.com (mail-yb1-f170.google.com [209.85.219.170]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) (Authenticated sender: eduardo) by smtp.freebsd.org (Postfix) with ESMTPSA id 4TytJh2045zFcY; Mon, 18 Mar 2024 11:43:00 +0000 (UTC) (envelope-from eduardo@freebsd.org) Received: by mail-yb1-f170.google.com with SMTP id 3f1490d57ef6-dd161eb03afso3508443276.0; Mon, 18 Mar 2024 04:43:00 -0700 (PDT) X-Forwarded-Encrypted: i=1; AJvYcCWI+9L6oc6rkCRNCb2F5fT/iFW29yVs5SwDG2ZQbk2TmR2bWP+KK619nQZz+0X3VRaDrOw32x8Pp7TtAV6/Z4KQssnXRu69s1UMB5aENTO9BWAfOpyheblBVAetk4CC3NJyBfculA== X-Gm-Message-State: AOJu0Yz8o6MY5hoO0xu67Pd8Ld2CG01nxu0rvBmaiFPv9N2kJGxMxuhg Vfsa2lJ5H3oW8zlOldF74JnlEk3IJ83acymQp7SYQYUE6VMGZxZWe+dVsd+ftFmLD3ymV1pn0nx naOelOEusoycdVqXLY7YsME0mhMA= X-Google-Smtp-Source: AGHT+IEVGcLQaZvLj2F2msTlyRw8wgpGp7EpaTC93rNd7cTVEkp6hphPXf+/wSRuTrGZlmLIeMSrEpvJ3QNahO+lP8Q= X-Received: by 2002:a5b:cc2:0:b0:dd1:37ff:696 with SMTP id e2-20020a5b0cc2000000b00dd137ff0696mr8777489ybr.59.1710762179517; Mon, 18 Mar 2024 04:42:59 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: <42C327BD-6CE4-43AA-A1AE-3BEC08D623DB@freebsd.org> <486915F0-456B-4B09-A8BC-93BBA79C4CA1@freebsd.org> <20240313080624.6c73908c@ernst.home> <508E3B47-8E1B-469F-97B1-2171A3098888@freebsd.org> <86a5n1i0xg.fsf@ltc.des.dev> <78D1FF09-71A3-4486-B934-D8332F54B237@freebsd.org> <20240316104053.20bef8c2@ernst.home> <20240316115128.33d11f7b@ernst.home> <7367F29A-D52B-4828-B79A-AA2667E81E7D@freebsd.org> <4FF534F6-B35D-4596-8D1E-226AD1347AC8@freebsd.org> <6e795e9c-8de4-4e02-9a96-8fabfaa4e66f@app.fastmail.com> In-Reply-To: From: Nuno Teixeira Date: Mon, 18 Mar 2024 11:42:48 +0000 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Request for Testing: TCP RACK To: Drew Gallatin Cc: tuexen@freebsd.org, garyj@gmx.de, current@freebsd.org, net@freebsd.org, Randall Stewart Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hello all! It works just fine! System performance is OK. Using patch on main-n268841-b0aaf8beb126(-dirty). --- net.inet.tcp.functions_available: Stack D Alias PCB coun= t freebsd freebsd 0 rack * rack 38 --- It would be so nice that we can have a sysctl tunnable for this patch so we could do more tests without recompiling kernel. Thanks all! Really happy here :) Cheers, Nuno Teixeira escreveu (domingo, 17/03/2024 =C3=A0(s)= 20:26): > > Hello, > > > I don't have the full context, but it seems like the complaint is a per= formance regression in bonnie++ and perhaps other things when tcp_hpts is l= oaded, even when it is not used. Is that correct? > > > > If so, I suspect its because we drive the tcp_hpts_softclock() routine = from userret(), in order to avoid tons of timer interrupts and context swit= ches. To test this theory, you could apply a patch like: > > It's affecting overall system performance, bonnie was just a way to > get some numbers to compare. > > Tomorrow I will test patch. > > Thanks! > > -- > Nuno Teixeira > FreeBSD Committer (ports) --=20 Nuno Teixeira FreeBSD Committer (ports) From nobody Mon Mar 18 12:04:40 2024 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Tytns5zP5z5Dbj5; Mon, 18 Mar 2024 12:04:49 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from drew.franken.de (drew.ipv6.franken.de [IPv6:2001:638:a02:a001:20e:cff:fe4a:feaa]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.franken.de", Issuer "Sectigo RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Tytns1Q0pz44Xs; Mon, 18 Mar 2024 12:04:49 +0000 (UTC) (envelope-from tuexen@freebsd.org) Authentication-Results: mx1.freebsd.org; none Received: from smtpclient.apple (unknown [IPv6:2a02:c6a0:3052:202:550a:8b4c:ffef:3f6f]) (Authenticated sender: micmac) by drew.franken.de (Postfix) with ESMTPSA id 6DD94721E2806; Mon, 18 Mar 2024 13:04:41 +0100 (CET) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.500.171.1.1\)) Subject: Re: Request for Testing: TCP RACK From: tuexen@freebsd.org In-Reply-To: Date: Mon, 18 Mar 2024 13:04:40 +0100 Cc: garyj@gmx.de, current@freebsd.org, net@freebsd.org, Randall Stewart Content-Transfer-Encoding: quoted-printable Message-Id: References: <42C327BD-6CE4-43AA-A1AE-3BEC08D623DB@freebsd.org> <486915F0-456B-4B09-A8BC-93BBA79C4CA1@freebsd.org> <20240313080624.6c73908c@ernst.home> <508E3B47-8E1B-469F-97B1-2171A3098888@freebsd.org> <86a5n1i0xg.fsf@ltc.des.dev> <78D1FF09-71A3-4486-B934-D8332F54B237@freebsd.org> <20240316104053.20bef8c2@ernst.home> <20240316115128.33d11f7b@ernst.home> <7367F29A-D52B-4828-B79A-AA2667E81E7D@freebsd.org> <4FF534F6-B35D-4596-8D1E-226AD1347AC8@freebsd.org> <6e795e9c-8de4-4e02-9a96-8fabfaa4e66f@app.fastmail.com> To: Nuno Teixeira , Drew Gallatin X-Mailer: Apple Mail (2.3774.500.171.1.1) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, T_SCC_BODY_TEXT_LINE autolearn=disabled version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail-n.franken.de X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:680, ipnet:2001:638::/32, country:DE] X-Rspamd-Queue-Id: 4Tytns1Q0pz44Xs > On 18. Mar 2024, at 12:42, Nuno Teixeira wrote: >=20 > Hello all! >=20 > It works just fine! > System performance is OK. > Using patch on main-n268841-b0aaf8beb126(-dirty). >=20 > --- > net.inet.tcp.functions_available: > Stack D Alias PCB = count > freebsd freebsd 0 > rack * rack 38 > --- >=20 > It would be so nice that we can have a sysctl tunnable for this patch > so we could do more tests without recompiling kernel. Thanks for testing! @gallatin: can you come up with a patch that is acceptable for Netflix and allows to mitigate the performance regression. Best regards Michael >=20 > Thanks all! > Really happy here :) >=20 > Cheers, >=20 > Nuno Teixeira escreveu (domingo, 17/03/2024 = =C3=A0(s) 20:26): >>=20 >> Hello, >>=20 >>> I don't have the full context, but it seems like the complaint is a = performance regression in bonnie++ and perhaps other things when = tcp_hpts is loaded, even when it is not used. Is that correct? >>>=20 >>> If so, I suspect its because we drive the tcp_hpts_softclock() = routine from userret(), in order to avoid tons of timer interrupts and = context switches. To test this theory, you could apply a patch like: >>=20 >> It's affecting overall system performance, bonnie was just a way to >> get some numbers to compare. >>=20 >> Tomorrow I will test patch. >>=20 >> Thanks! >>=20 >> -- >> Nuno Teixeira >> FreeBSD Committer (ports) >=20 >=20 >=20 > --=20 > Nuno Teixeira > FreeBSD Committer (ports) From nobody Mon Mar 18 12:26:10 2024 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TyvGg3NcDz5DdcB; Mon, 18 Mar 2024 12:26:19 +0000 (UTC) (envelope-from mike@karels.net) Received: from mail2.karels.net (mail2.karels.net [3.19.118.201]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "freebsd", Issuer "freebsd" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TyvGf65Zgz47fH; Mon, 18 Mar 2024 12:26:18 +0000 (UTC) (envelope-from mike@karels.net) Authentication-Results: mx1.freebsd.org; none Received: from mail2.karels.net (localhost [IPv6:0:0:0:0:0:0:0:1]) by mail2.karels.net (8.18.1/8.18.1) with ESMTP id 42ICQBBv054697; Mon, 18 Mar 2024 07:26:11 -0500 (CDT) (envelope-from mike@karels.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=karels.net; s=mail2; t=1710764772; bh=ewZYGbxI4QWGlXbwewAilCFxYmXLkdBwORFYqLEWzNw=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=ap7WZBuT+KXXggGCehJWapm7X7UfcU/kFTSJVhf/RurKXnmp+WBJ8yTl/TiymFtv/ ++p8/jmHeGEhduWXotDAOnZkRZ9iok3d4UGEUxWTJ5jc/3U8X0qAnndHEuByEp7tfk Uub65E/PjBb0eqfAvqn1ioGJI1JourWHg41XD1OJotfdDNxAQMmNw731cGiCjkcRP0 P6GdaV1nkWGjwGTfgoYurljaqSfuDt3V6Ot8YSQlzEyQ0UM0Q+qKMVqcqZs71jRJIR GKwE2yY4VWX7e/a75qeCaZiVvVGAQ4O9AHk+6flLT0l6LFzkXMqswnbGpCgkV1QgPA wUQHc309o9frg== Received: from [10.0.2.130] ([73.62.165.147]) by mail2.karels.net with ESMTPSA id jWyrL+My+GWn1QAAs/W3XQ (envelope-from ); Mon, 18 Mar 2024 07:26:11 -0500 From: Mike Karels To: tuexen@freebsd.org Cc: Nuno Teixeira , Drew Gallatin , garyj@gmx.de, current@freebsd.org, net@freebsd.org, Randall Stewart Subject: Re: Request for Testing: TCP RACK Date: Mon, 18 Mar 2024 07:26:10 -0500 X-Mailer: MailMate (1.14r6015) Message-ID: <6047C8EF-B1B0-4286-93FA-AA38F8A18656@karels.net> In-Reply-To: References: <42C327BD-6CE4-43AA-A1AE-3BEC08D623DB@freebsd.org> <486915F0-456B-4B09-A8BC-93BBA79C4CA1@freebsd.org> <20240313080624.6c73908c@ernst.home> <508E3B47-8E1B-469F-97B1-2171A3098888@freebsd.org> <86a5n1i0xg.fsf@ltc.des.dev> <78D1FF09-71A3-4486-B934-D8332F54B237@freebsd.org> <20240316104053.20bef8c2@ernst.home> <20240316115128.33d11f7b@ernst.home> <7367F29A-D52B-4828-B79A-AA2667E81E7D@freebsd.org> <4FF534F6-B35D-4596-8D1E-226AD1347AC8@freebsd.org> <6e795e9c-8de4-4e02-9a96-8fabfaa4e66f@app.fastmail.com> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:16509, ipnet:3.16.0.0/14, country:US] X-Rspamd-Queue-Id: 4TyvGf65Zgz47fH On 18 Mar 2024, at 7:04, tuexen@freebsd.org wrote: >> On 18. Mar 2024, at 12:42, Nuno Teixeira wrote: >> >> Hello all! >> >> It works just fine! >> System performance is OK. >> Using patch on main-n268841-b0aaf8beb126(-dirty). >> >> --- >> net.inet.tcp.functions_available: >> Stack D Alias PCB= count >> freebsd freebsd 0 >> rack * rack 38 >> --- >> >> It would be so nice that we can have a sysctl tunnable for this patch >> so we could do more tests without recompiling kernel. > Thanks for testing! > > @gallatin: can you come up with a patch that is acceptable for Netflix > and allows to mitigate the performance regression. Ideally, tcphpts could enable this automatically when it starts to be used (enough?), but a sysctl could select auto/on/off. Mike > Best regards > Michael >> >> Thanks all! >> Really happy here :) >> >> Cheers, >> >> Nuno Teixeira escreveu (domingo, 17/03/2024 =C3=A0= (s) 20:26): >>> >>> Hello, >>> >>>> I don't have the full context, but it seems like the complaint is a = performance regression in bonnie++ and perhaps other things when tcp_hpts= is loaded, even when it is not used. Is that correct? >>>> >>>> If so, I suspect its because we drive the tcp_hpts_softclock() routi= ne from userret(), in order to avoid tons of timer interrupts and context= switches. To test this theory, you could apply a patch like: >>> >>> It's affecting overall system performance, bonnie was just a way to >>> get some numbers to compare. >>> >>> Tomorrow I will test patch. >>> >>> Thanks! >>> >>> -- >>> Nuno Teixeira >>> FreeBSD Committer (ports) >> >> >> >> -- = >> Nuno Teixeira >> FreeBSD Committer (ports) From nobody Mon Mar 18 12:33:11 2024 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TyvQm4QVqz5Df6m for ; Mon, 18 Mar 2024 12:33:20 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from mx.catwhisker.org (mx.catwhisker.org [107.204.234.170]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TyvQm12TMz4DBX; Mon, 18 Mar 2024 12:33:19 +0000 (UTC) (envelope-from david@catwhisker.org) Authentication-Results: mx1.freebsd.org; none Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.18.1/8.18.1) with ESMTP id 42ICXBeq020711; Mon, 18 Mar 2024 12:33:11 GMT (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.18.1/8.18.1/Submit) id 42ICXBW3020710; Mon, 18 Mar 2024 05:33:11 -0700 (PDT) (envelope-from david) Date: Mon, 18 Mar 2024 05:33:11 -0700 From: David Wolfskill To: Mike Karels Cc: tuexen@freebsd.org, Nuno Teixeira , Drew Gallatin , garyj@gmx.de, current@freebsd.org, net@freebsd.org, Randall Stewart Subject: Re: Request for Testing: TCP RACK Message-ID: Mail-Followup-To: David Wolfskill , Mike Karels , tuexen@freebsd.org, Nuno Teixeira , Drew Gallatin , garyj@gmx.de, current@freebsd.org, net@freebsd.org, Randall Stewart References: <4FF534F6-B35D-4596-8D1E-226AD1347AC8@freebsd.org> <6e795e9c-8de4-4e02-9a96-8fabfaa4e66f@app.fastmail.com> <6047C8EF-B1B0-4286-93FA-AA38F8A18656@karels.net> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="ZW+1ZJTfz339vpLs" Content-Disposition: inline In-Reply-To: <6047C8EF-B1B0-4286-93FA-AA38F8A18656@karels.net> X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:7018, ipnet:107.192.0.0/12, country:US] X-Rspamd-Queue-Id: 4TyvQm12TMz4DBX --ZW+1ZJTfz339vpLs Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Mar 18, 2024 at 07:26:10AM -0500, Mike Karels wrote: > ... > >> It would be so nice that we can have a sysctl tunnable for this patch > >> so we could do more tests without recompiling kernel. > > Thanks for testing! > > > > @gallatin: can you come up with a patch that is acceptable for Netflix > > and allows to mitigate the performance regression. >=20 > Ideally, tcphpts could enable this automatically when it starts to be > used (enough?), but a sysctl could select auto/on/off. >=20 > Mike > .... Or (at some risk of over-{complicat,engineer}ing things), perhaps sysctl/tunable for high- and low-water marks? Peace, david (who is quite unlikely to be writing the code, so "grain of salt") --=20 David H. Wolfskill david@catwhisker.org Alexey Navalny was a courageous man; Putin has made him a martyr. See https://www.catwhisker.org/~david/publickey.gpg for my public key. --ZW+1ZJTfz339vpLs Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iNUEARYKAH0WIQSTLzOSbomIK53fjFliipiWhXYx5QUCZfg0hl8UgAAAAAAuAChp c3N1ZXItZnByQG5vdGF0aW9ucy5vcGVucGdwLmZpZnRoaG9yc2VtYW4ubmV0OTMy RjMzOTI2RTg5ODgyQjlEREY4QzU5NjI4QTk4OTY4NTc2MzFFNQAKCRBiipiWhXYx 5VO3AQDpWSTmo4IwVnPuV8nRxio+oU11e/Q4gNJwn2l73ZZeYgD/V6UxqL1hl7f3 mgPM5118Xkj38cZmGpAsWhb9sZEWEwI= =Y8dn -----END PGP SIGNATURE----- --ZW+1ZJTfz339vpLs-- From nobody Mon Mar 18 16:26:31 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Tz0cB1hghz5F2j2 for ; Mon, 18 Mar 2024 16:26:50 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic313-19.consmr.mail.gq1.yahoo.com (sonic313-19.consmr.mail.gq1.yahoo.com [98.137.65.82]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Tz0c832Hcz4j45 for ; Mon, 18 Mar 2024 16:26:48 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=JHevevRr; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.65.82 as permitted sender) smtp.mailfrom=marklmi@yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1710779205; bh=RGZYNe5UsD6Yb0eDpQf2P3Pt9z4c102QJKqrOsxFg84=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=JHevevRrmcDhXpOny6e2lF9k/ymxQ7iq77L27wzJxa8m8EfTr6sVKoMLIbztcuKelQcF1L+QZvhPV/VWYbPqqYr8D7o/oaY4MxXn+ZFjjRTEkg63AVYKSLX+CmMSSpRGNVn2dJyehiZlMNSc9t3E/cPfTacToB2R++ICFvmXNrArQZhzLmIIplmQ9/ADc7GyAyOhcjn6YKKxrWWE1gjpS2bmVyFpH/m8yZa5kdEgIyaj2KzgD72u8YxEc1ZmAQNS8+PR5vEpTQzQgNn/+6BiOl5Wi5HAbiIIMzlGeggfIqMgVZ3b0erbNrDSibb2jszaRC8AnFph55AiXYEAhk8lAQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1710779205; bh=By1ZgUQ3D5TWmaKyWpB9IHXJmFDWZ1U72qiOgADg7+0=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=uf3YJY9UwvEKoldjF+fAAWjlDKM9sBGTelKCrqkpa4M2GkNGoXw5rbc1ofGPz42zUfeeMRpgDEUqLEkNWGtif8/RW0OyLYAXSKZS8R32WhKqRn08K+ppPkNDLe2kTUOos/DnO/qRwrCqzLVVEOztGXFXpWFzD2fSOPq8JWa5OEq8OTpEbrYiPVVtjOkzZoKQ9FBB5xqlGkm44G3newr6/RvRA7UIRPYzBNWW2d5DSADCRfZj6P/qeGPeQ8ZBk6+XqSUsSjv00jYoHle9A3gYdiKvVca8rRk8feinz4LZVFGKYISHLORaS7x2HFEKMZvWWYTPN38p6iNk1EESDtitig== X-YMail-OSG: CwLid6MVM1ntdFFbJgR6QrrePzmWt95KPIUwOnFR1sZgByk_yC1RJQQUiGijVJZ aw75DlDMSudjXei5MZN5o3oT_d3C.g2JPZpZsVK11Nv9.mPbfvIke1.hfupikW30OaH5mzDoKoZh PWRWekeDxgg.xi5zeu73CCm1gv7Bn7Q6QQjOnE2AtvWWkSJ0B3eqyV.DrC4a_j61jQnF7yWjdOw3 gUm3XvaRAVSeVvkVBShoUXv67Othz62zMjwLWfZf1vOAP1AWZUBM8JbjZAFNcffqpI.1UqWFeCyg dWlqooaXZ7jbn8bLE.yOQmr2Fsr9J3STu1HyPbqTP08H9fTGEe.vUjzuIOpoqiGlKxIGBQYo78NT oq4oTcktDB3HjzoGB0LcSc7pv0icILCOigZaOdpKyV9uOHDRXAiuVCwsDm3g8uNC7mHvvn2z0gmn v1uMonT2wBp4jdxwFXQf0Lzf3bF38I3SUVof3xls9cJaHtKNACshj3904ywaRSCUkfRugRoLv3og YFJwPWjuds75DHNlHLE_dVkvqvay_2jJnAqop2KcZphW0ybzuf9opOCfSQzJnUGLpwpOUhx9Q.E7 RDSUUT7exaBG8mIv8NfcFQRh4tHYpVzkfZmqN0LXAt_BMyDaPMN9tbTeMjlWx_.loX1kCtODXTvU 5oyYtEdjDasT8.2AcmrB4vnLJcyT7BdUYHl_R3U1mfxjTBWkr0V0LyAT8hFrA797VMERgzP5k_aZ PWIhdieHiZ.uS11M7LiAcCIp1hcK8wP6qO0U9mM2RiL5cREBty9EzohH0ygQVdQq7rwBpUsNNs6Y 3kRMHDrv4.KZ2QDOqSU25lZ8vSO9q_wyYgEIxtxhXHKiKtXw7Is10ncde2.G4Py4hlqSi3JuQ_nh gDA37e8XTRb2yysDjVSa1AEZAIPFG605XEfy2lsmA_GVG83aMvbalvtxQ9rStGBwin7lgyU5KKtp VFLWZom7oJXwB9AbL51OfXL2W0uYH1mJeNwLJ35gtpZ_yjmHwwseIo6euhZw0_9f3vWo1qUyrMyC EiRKbCe1IO5Rr09CL6vqfENBJg5eIM8p8WtWoTC4T6pmxGblz4fRb4wQdigGLLa6MbK5HPNCbxnS e_6pl2SGZngcsVVEhl_VPgijKdf0NOwKcfBAkbAVKzB_KAOLEIoiJpptuzgEe_VANPwKJAs8eDik nUJP0ilrhoVjj18TZmbEXUmK8xrYumRM9RzicmUSzoo0NRN8uaoUYf8VbgHBddzYk12BDu9J7CfR TYR2oxb.12BrmZLyAJ.OEbv2OfpxUZSbC74ulvUGZ.Uq1B1vWll3gJRAaOGYVo65_B1uh7pQP8jJ qTJQF4sqYTt_iB61TKWWVtkgTnql5hJCeiMkvkSb6_F_wxwIAmSSyARPMqWa3WjqtOtrasDB_AiA .5S_bnnhiKnbAnwI8S2PeF6CGM08.BUNy4RfUGj1D8l.JRZfb11ww0gBYaFmx90EE_BkXJj2u2TC 3DkhVA1ser0Tu5cuzX5Rm8xz0uJN65KEdPs6N0dKJLo9Qt5Bw_dRUDEpfm0LZ9r_6hjSZisSqvFz 53AOhrJjg38GiYzE5khTBxLFi92eqKOwnTdEyLyldQeerFlB4Hv_app4d2BnGEbqtPDgshPWtviM 6x9.SkLRkiV9.AzlUmPRyfzxAispnFdK.VzCujB2FUvXf43A6.Z5XdIYDSH1eULAfLADWVbWQD.P J5N.R7uyC2on9cpYIuh39QkltqnfweBzyua.1T.r3Ci9u9VpwVfZWOgbS_tffyjQtGFwg1kMFwMD 3MZm9cx9MIdHWbQpNHtMZBucpwvhCk0DcCum5UTc5ZQyp8Q59LGeVRF8I6eFaHvFBrpl6Mv0sNZ1 B_y1Lq6Ni52AcfX65ydMZggd39Opg6Rp7Ub6jnCNoDmL4HFa.XTwNWnxg8amITWlIGYTthywlmQ. QfgGP59IwJJa1hROm2WC37vfI5czzeP9QqVAsnKydZu.vZQHQ50uIQkZpAl4sIdnyk532KAbM5D6 n3rx3GRmBgIj_q5FkOcx44vOOIjTcdkbMEFls6vg1sX8eOP33w0tl8L1u.yxgIUbyIdiWwT6j7fn JasdC6rk4lVyBCvk9hEJjJ7NVnVNQ_dzn5XVEkFah6chRhLZNY0rqFIaedMiCwQtNLPAikGb4onn vT09uzjSYFdIEUBftEOo8bfRf6lnXQmh7vkhXLMVYdJT_wLIhvCtdkVeewJS3Wxu_mOslUN.mZmv eSzAn2mvb_ldwSfFh0XpIbfGnDDbNn2Mmo8GhNsIzbDIoiVyWcypn5rjquQWGA7B6Z.iorvLfYg- - X-Sonic-MF: X-Sonic-ID: 870b4af2-9d52-4da5-87de-f498f25d59cf Received: from sonic.gate.mail.ne1.yahoo.com by sonic313.consmr.mail.gq1.yahoo.com with HTTP; Mon, 18 Mar 2024 16:26:45 +0000 Received: by hermes--production-gq1-5c57879fdf-bmngc (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 50e65e86d8118963b9854ec3713e1182; Mon, 18 Mar 2024 16:26:42 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\)) Subject: kernel= , kern.module_path= , and loader menu selections: how to have it always find the matching, say, nfsd.ko (same directory as the kernel) Message-Id: Date: Mon, 18 Mar 2024 09:26:31 -0700 To: Warner Losh , Current FreeBSD X-Mailer: Apple Mail (2.3774.400.31) References: X-Spamd-Bar: --- X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; TO_DN_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; FREEMAIL_FROM(0.00)[yahoo.com]; DKIM_TRACE(0.00)[yahoo.com:+]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; APPLE_MAILER_COMMON(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.82:from]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.82:from] X-Rspamd-Queue-Id: 4Tz0c832Hcz4j45 I'm using an aarch64 environment as the example context for this note. I have instances of the various PkgBase kernels and one or more personal kernel builds. Currently: # ls -Tlod /boot/kernel*/nfsd* -r--r--r-- 1 root wheel - 401600 Mar 17 18:45:35 2024 = /boot/kernel.CA76-NODBG.good/nfsd.ko -r--r--r-- 1 root wheel - 401600 Mar 17 18:45:35 2024 = /boot/kernel.CA76-NODBG/nfsd.ko -r--r--r-- 1 root wheel - 388624 Mar 15 19:18:47 2024 = /boot/kernel.GENERIC-DEBUG.good/nfsd.ko -r--r--r-- 1 root wheel - 388624 Mar 15 19:18:47 2024 = /boot/kernel.GENERIC-MMCCAM/nfsd.ko -r--r--r-- 1 root wheel - 425968 Mar 15 19:18:47 2024 = /boot/kernel.GENERIC-NODEBUG.good/nfsd.ko -r--r--r-- 1 root wheel - 425968 Mar 15 19:18:47 2024 = /boot/kernel.GENERIC-NODEBUG/nfsd.ko -r--r--r-- 1 root wheel - 388624 Mar 15 19:18:47 2024 = /boot/kernel/nfsd.ko The kernel.CA76-*/ ones are my tailored builds. The others are from PkgBase. (amd64 also has a PkgBase kernel.MINIMAL/ .) I originally used loader.conf lines like the following to control what loaded by default: #kernel=3D"kernel" #kernel=3D"kernel.GENERIC-DEBUG.good" kernel=3D"kernel.GENERIC-NODEBUG" #kernel=3D"kernel.GENERIC-NODEBUG.good" #kernel=3D"kernel.GENERIC-MMCCAM" # #kernel=3D"kernel.CA76-DBG" #kernel=3D"kernel.CA76-NODBG" However, I found that the /etc/rc.conf lines like: rpcbind_enable=3D"YES" nfs_server_enable=3D"YES" mountd_flags=3D"-r" nfs_client_enable=3D"YES" did not cause nfsd to load (for example). I worked around this by also having an assignment to kern.module_path : #kernel=3D"kernel.GENERIC-DEBUG.good" #kern.module_path=3D"/boot/kernel.GENERIC-DEBUG.good" kernel=3D"kernel.GENERIC-NODEBUG" kern.module_path=3D"/boot/kernel.GENERIC-NODEBUG" #kernel=3D"kernel.GENERIC-NODEBUG.good" #kern.module_path=3D"/boot/kernel.GENERIC-NODEBUG.good" #kernel=3D"kernel.GENERIC-MMCCAM" #kern.module_path=3D"/boot/kernel.GENERIC-MMCCAM" # #kernel=3D"kernel.CA76-DBG" #kern.module_path=3D"/boot/kernel.CA76-DBG" #kernel=3D"kernel.CA76-NODBG" #kern.module_path=3D"/boot/kernel.CA76-NODBG" (That suggests that kern.module_path did not automatically track the kernel=3D assignment.) That worked for loading the loader.conf specified default kernel and the matching nfsd . But, if, in the loader menu, I pick a different kernel, the problem returns, likely from still trying the kern.module_path from the loader.conf (?). Note: My keeping a little library of kernels around is new. I've no history to know how long the behavior that I've seen has been this way. Is there a way for it to automatically use the likes of the nfsd.ko from the same directory as the kernel in all cases, where I pick the default kernel place via loader.conf ? Do I need to do more manual steps in the loader, not just use the menu selections to override the defaults for this sufficiently to have the likes of the matching nfsd.ko load? =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Mon Mar 18 18:33:16 2024 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Tz3QQ4Dk8z5FDcr for ; Mon, 18 Mar 2024 18:33:34 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Tz3QQ0j0fz3yfr; Mon, 18 Mar 2024 18:33:33 +0000 (UTC) (envelope-from kostikbel@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: from tom.home (kib@localhost [127.0.0.1] (may be forged)) by kib.kiev.ua (8.18.1/8.18.1) with ESMTP id 42IIXHW2082887; Mon, 18 Mar 2024 20:33:20 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 42IIXHW2082887 Received: (from kostik@localhost) by tom.home (8.18.1/8.18.1/Submit) id 42IIXH90082886; Mon, 18 Mar 2024 20:33:17 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 18 Mar 2024 20:33:16 +0200 From: Konstantin Belousov To: Mike Karels Cc: tuexen@freebsd.org, Nuno Teixeira , Drew Gallatin , garyj@gmx.de, current@freebsd.org, net@freebsd.org, Randall Stewart Subject: Re: Request for Testing: TCP RACK Message-ID: References: <4FF534F6-B35D-4596-8D1E-226AD1347AC8@freebsd.org> <6e795e9c-8de4-4e02-9a96-8fabfaa4e66f@app.fastmail.com> <6047C8EF-B1B0-4286-93FA-AA38F8A18656@karels.net> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <6047C8EF-B1B0-4286-93FA-AA38F8A18656@karels.net> X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-14) on tom.home X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US] X-Rspamd-Queue-Id: 4Tz3QQ0j0fz3yfr On Mon, Mar 18, 2024 at 07:26:10AM -0500, Mike Karels wrote: > On 18 Mar 2024, at 7:04, tuexen@freebsd.org wrote: > > >> On 18. Mar 2024, at 12:42, Nuno Teixeira wrote: > >> > >> Hello all! > >> > >> It works just fine! > >> System performance is OK. > >> Using patch on main-n268841-b0aaf8beb126(-dirty). > >> > >> --- > >> net.inet.tcp.functions_available: > >> Stack D Alias PCB count > >> freebsd freebsd 0 > >> rack * rack 38 > >> --- > >> > >> It would be so nice that we can have a sysctl tunnable for this patch > >> so we could do more tests without recompiling kernel. > > Thanks for testing! > > > > @gallatin: can you come up with a patch that is acceptable for Netflix > > and allows to mitigate the performance regression. > > Ideally, tcphpts could enable this automatically when it starts to be > used (enough?), but a sysctl could select auto/on/off. There is already a well-known mechanism to request execution of the specific function on return to userspace, namely AST. The difference with the current hack is that the execution is requested for one callback in the context of the specific thread. Still, it might be worth a try to use it; what is the reason to hit a thread that does not do networking, with TCP processing? > > Mike > > > Best regards > > Michael > >> > >> Thanks all! > >> Really happy here :) > >> > >> Cheers, > >> > >> Nuno Teixeira escreveu (domingo, 17/03/2024 à(s) 20:26): > >>> > >>> Hello, > >>> > >>>> I don't have the full context, but it seems like the complaint is a performance regression in bonnie++ and perhaps other things when tcp_hpts is loaded, even when it is not used. Is that correct? > >>>> > >>>> If so, I suspect its because we drive the tcp_hpts_softclock() routine from userret(), in order to avoid tons of timer interrupts and context switches. To test this theory, you could apply a patch like: > >>> > >>> It's affecting overall system performance, bonnie was just a way to > >>> get some numbers to compare. > >>> > >>> Tomorrow I will test patch. > >>> > >>> Thanks! > >>> > >>> -- > >>> Nuno Teixeira > >>> FreeBSD Committer (ports) > >> > >> > >> > >> -- > >> Nuno Teixeira > >> FreeBSD Committer (ports) > From nobody Mon Mar 18 19:13:11 2024 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Tz4JY0z7Mz5Dnb8; Mon, 18 Mar 2024 19:13:33 +0000 (UTC) (envelope-from gallatin@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Tz4JY0PwGz44Jt; Mon, 18 Mar 2024 19:13:33 +0000 (UTC) (envelope-from gallatin@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1710789213; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=LBPHA0DV2V4AA/r8GnMiwoXOA7ru9W10xJUHU6dWmyY=; b=xkXtsWbWGBgK/RI4ZpWEISyJ0SYG1afTzfwLf8Xakj9LWXPsQT8EXUybqhx2BFxjZqukEY RiCWlenfTqDhQdbNIybGUldF7VceMrwgOSPZt/nAr2wg2mKtoArbzKFNV171QoJACp7T0X ERVWXY5xCxaPWThDS/CpTu2XRoU9Cu4TmyWse4uSopKDjGymnoAT7+kcP3m5gLjlscIvwL EA1eF8wX8ohqwhm5YVx41eDQwwiftbkIKD7mlBBVahj0DxFYkTmCtq3pEaQzsOOhBo4Gss 2aH5+Et8yYCD/oeHGz4UnEfsKj/sIN2nwI27eNclrGzFT54rNOdrodPvqOpL7A== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1710789213; a=rsa-sha256; cv=none; b=VU2CMXmlKcoQJJUz5zJr0RorQiuNX8f3v3ZMBO7M+vrL+6yYnjzT80f28dUu7zA2tZN025 TA5bSGAm2mu9Sc7N0LvSGh+hg4F6FuNFgtjTCjQhVopFWPiymczMNCtvOgPRF3FPDlrllx mDhfnx1b4knaJcmSBo3SwjJXACoUV8fa2AU4nnk3yiE+rDL9pRB0c3uNb7fFmOC9TkwbDn Y3npys97HLxtv2YNYRVGQXFYPsQLAqXuT4f6Uf7FKpcYYdh0NhwKmr+5Vpkua6qiPnovuk Av7Jf0Ijzb3syEPvvAlQGbE3D87+dGUc6y6ELf48T/OSo0Zbo4LH6R1awTBHkg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1710789213; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=LBPHA0DV2V4AA/r8GnMiwoXOA7ru9W10xJUHU6dWmyY=; b=XBW4GC6kX1lbQ+rQfFGVWzDBDTBds4bQUgSAuwffCu3TnnHKtqc3hkoF06k3JwM3W0N2La oBE09Kcf+2mkaIioKVSInuubXfhbpdfw7YSLrMqSk+0o6OyAiNRuKmwwhvWk/SQE/Tvepi Om3llYeP4WdLR78NwUalGGjauCz4y6vsa19I5msuZj1DO9+D8ioAHRx4V/pXg7m5aqNVoO A7KnyQR/GTFwwmjVeNDw3W9oa6iyTwPvy10sMeiDNuAjLO4tEmeOlrVbnWXFMUDvnZ6AyZ BTn+gxKrhdYywZtxQt0rTp55kEiZpNg2+gDetu2XbINgWhQYfNNCMjJc1V+Xyw== Received: from fauth2-smtp.messagingengine.com (fauth2-smtp.messagingengine.com [103.168.172.201]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: gallatin) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Tz4JX4pn9zPyv; Mon, 18 Mar 2024 19:13:32 +0000 (UTC) (envelope-from gallatin@freebsd.org) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailfauth.nyi.internal (Postfix) with ESMTP id 8E7E71200066; Mon, 18 Mar 2024 15:13:31 -0400 (EDT) Received: from imap53 ([10.202.2.103]) by compute5.internal (MEProxy); Mon, 18 Mar 2024 15:13:31 -0400 X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrkeejgdduvddvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepofgfggfkjghffffhvfevufgtsegrtderreerreejnecuhfhrohhmpedfffhr vgifucfirghllhgrthhinhdfuceoghgrlhhlrghtihhnsehfrhgvvggsshgurdhorhhgqe enucggtffrrghtthgvrhhnpefgteeluefggeevheefheejgedtvdelheejfffhgeeuhfel keevheeiieeuhfefieenucffohhmrghinhepmhhpihdqshifshdrohhrghenucevlhhush htvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehgrghllhgrthhinhdo mhgvshhmthhprghuthhhphgvrhhsohhnrghlihhthidqudeffeehledvvdduiedqvdelhe dtgedukeegqdhgrghllhgrthhinheppehfrhgvvggsshgurdhorhhgsehfrghsthhmrghi lhdrtghomh X-ME-Proxy: Feedback-ID: i41414658:Fastmail Received: by mailuser.nyi.internal (Postfix, from userid 501) id 52F5C36400BD; Mon, 18 Mar 2024 15:13:31 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.11.0-alpha0-300-gdee1775a43-fm-20240315.001-gdee1775a List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Message-Id: <8031cd99-ded8-4b06-93b3-11cc729a8b2c@app.fastmail.com> In-Reply-To: References: <4FF534F6-B35D-4596-8D1E-226AD1347AC8@freebsd.org> <6e795e9c-8de4-4e02-9a96-8fabfaa4e66f@app.fastmail.com> <6047C8EF-B1B0-4286-93FA-AA38F8A18656@karels.net> Date: Mon, 18 Mar 2024 15:13:11 -0400 From: "Drew Gallatin" To: "Konstantin Belousov" , "Mike Karels" Cc: tuexen , "Nuno Teixeira" , garyj@gmx.de, current@freebsd.org, net@freebsd.org, "Randall Stewart" Subject: Re: Request for Testing: TCP RACK Content-Type: multipart/alternative; boundary=01b96c257b37417295d61c17eb06343b --01b96c257b37417295d61c17eb06343b Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: quoted-printable I got the idea from https://people.mpi-sws.org/~druschel/publications/so= ft-timers-tocs.pdf The gist is that the TCP pacing stuff needs to run f= requently, and rather than run it out of a clock interrupt, its more eff= icient to run it out of a system call context at just the point where we= return to userspace and the cache is trashed anyway. The current impl= ementation is fine for our workload, but probably not idea for a generic= system. Especially one where something is banging on system calls. =20 Ast's could be the right tool for this, but I'm super unfamiliar with th= em, and I can't find any docs on them.=20 Would ast_register(0, ASTR_UNCOND, 0, func) be roughly equivalent to wha= t's happening here? Drew On Mon, Mar 18, 2024, at 2:33 PM, Konstantin Belousov wrote: > On Mon, Mar 18, 2024 at 07:26:10AM -0500, Mike Karels wrote: > > On 18 Mar 2024, at 7:04, tuexen@freebsd.org wrote: > >=20 > > >> On 18. Mar 2024, at 12:42, Nuno Teixeira wr= ote: > > >> > > >> Hello all! > > >> > > >> It works just fine! > > >> System performance is OK. > > >> Using patch on main-n268841-b0aaf8beb126(-dirty). > > >> > > >> --- > > >> net.inet.tcp.functions_available: > > >> Stack D Alias = PCB count > > >> freebsd freebsd = 0 > > >> rack * rack = 38 > > >> --- > > >> > > >> It would be so nice that we can have a sysctl tunnable for this p= atch > > >> so we could do more tests without recompiling kernel. > > > Thanks for testing! > > > > > > @gallatin: can you come up with a patch that is acceptable for Net= flix > > > and allows to mitigate the performance regression. > >=20 > > Ideally, tcphpts could enable this automatically when it starts to be > > used (enough?), but a sysctl could select auto/on/off. > There is already a well-known mechanism to request execution of the > specific function on return to userspace, namely AST. The difference > with the current hack is that the execution is requested for one callb= ack > in the context of the specific thread. >=20 > Still, it might be worth a try to use it; what is the reason to hit a = thread > that does not do networking, with TCP processing? >=20 > >=20 > > Mike > >=20 > > > Best regards > > > Michael > > >> > > >> Thanks all! > > >> Really happy here :) > > >> > > >> Cheers, > > >> > > >> Nuno Teixeira escreveu (domingo, 17/03/2024= =C3=A0(s) 20:26): > > >>> > > >>> Hello, > > >>> > > >>>> I don't have the full context, but it seems like the complaint = is a performance regression in bonnie++ and perhaps other things when tc= p_hpts is loaded, even when it is not used. Is that correct? > > >>>> > > >>>> If so, I suspect its because we drive the tcp_hpts_softclock() = routine from userret(), in order to avoid tons of timer interrupts and c= ontext switches. To test this theory, you could apply a patch like: > > >>> > > >>> It's affecting overall system performance, bonnie was just a way= to > > >>> get some numbers to compare. > > >>> > > >>> Tomorrow I will test patch. > > >>> > > >>> Thanks! > > >>> > > >>> -- > > >>> Nuno Teixeira > > >>> FreeBSD Committer (ports) > > >> > > >> > > >> > > >> --=20 > > >> Nuno Teixeira > > >> FreeBSD Committer (ports) > >=20 >=20 --01b96c257b37417295d61c17eb06343b Content-Type: text/html;charset=utf-8 Content-Transfer-Encoding: quoted-printable
I got the idea = from https://people.mpi-sws.org/~druschel/publications/s= oft-timers-tocs.pdf  The gist is that the TCP pacing stuff need= s to run frequently, and rather than run it out of a clock interrupt, it= s more efficient to run it out of a system call context at just the poin= t where we return to userspace and the cache is trashed anyway. &nb= sp; The current implementation is fine for our workload, but probably no= t idea for a generic system.  Especially one where something is ban= ging on system calls. 

Ast's could be= the right tool for this, but I'm super unfamiliar with them, and I can'= t find any docs on them.

Would ast_registe= r(0, ASTR_UNCOND, 0, func) be roughly equivalent to what's happening her= e?

Drew

On Mon= , Mar 18, 2024, at 2:33 PM, Konstantin Belousov wrote:
On Mon, Mar 18, 2024 at 07:2= 6:10AM -0500, Mike Karels wrote:
> On 18 Mar 2024, at 7= :04, tuexen@freebsd.org w= rote:
> 
> >> On 18. Mar 20= 24, at 12:42, Nuno Teixeira <e= duardo@freebsd.org> wrote:
> >>
<= div>> >> Hello all!
> >>
&= gt; >> It works just fine!
> >> System perf= ormance is OK.
> >> Using patch on main-n268841-b= 0aaf8beb126(-dirty).
> >>
> >= > ---
> >> net.inet.tcp.functions_available:
> >> Stack      &nbs= p;           &nbs= p;        D Alias   &n= bsp;           &n= bsp;            P= CB count
> >> freebsd    &nbs= p;           &nbs= p;          freebsd &n= bsp;           &n= bsp;            0=
> >> rack      &nb= sp;           &nb= sp;         * rack  &n= bsp;           &n= bsp;           &n= bsp;  38
> >> ---
> >>= ;
> >> It would be so nice that we can have a sys= ctl tunnable for this patch
> >> so we could do m= ore tests without recompiling kernel.
> > Thanks for= testing!
> >
> > @gallatin: can= you come up with a patch that is acceptable for Netflix
&= gt; > and allows to mitigate the performance regression.
> 
> Ideally, tcphpts could enable this autom= atically when it starts to be
> used (enough?), but a s= ysctl could select auto/on/off.
There is already a well-kn= own mechanism to request execution of the
specific functio= n on return to userspace, namely AST.  The difference
with the current hack is that the execution is requested for one callba= ck
in the context of the specific thread.
Still, it might be worth a try to use it; what is the reaso= n to hit a thread
that does not do networking, with TCP pr= ocessing?

> 
> M= ike
> 
> > Best regards
> > Michael
> >>
>= >> Thanks all!
> >> Really happy here :)
> >>
> >> Cheers,
=
> >>
> >> Nuno Teixeira <eduardo@freebsd.org> escreveu (do= mingo, 17/03/2024 =C3=A0(s) 20:26):
> >>>
<= /div>
> >>> Hello,
> >>>
> >>>> I don't have the full context, but it see= ms like the complaint is a performance regression in bonnie++ and perhap= s other things when tcp_hpts is loaded, even when it is not used.  = Is that correct?
> >>>>
> = >>>> If so, I suspect its because we drive the tcp_hpts_soft= clock() routine from userret(), in order to avoid tons of timer interrup= ts and context switches.  To test this theory,  you could appl= y a patch like:
> >>>
> >&= gt;> It's affecting overall system performance, bonnie was just a way= to
> >>> get some numbers to compare.
> >>>
> >>> Tomorrow I will= test patch.
> >>>
> >>= > Thanks!
> >>>
> >>= > --
> >>> Nuno Teixeira
>= >>> FreeBSD Committer (ports)
> >>
<= /div>
> >>
> >>
> &= gt;> -- 
> >> Nuno Teixeira
> >> FreeBSD Committer (ports)
> 


--01b96c257b37417295d61c17eb06343b-- From nobody Mon Mar 18 19:41:47 2024 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Tz4xM0w25z5DrFT; Mon, 18 Mar 2024 19:41:59 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Tz4xL64Xyz47lj; Mon, 18 Mar 2024 19:41:58 +0000 (UTC) (envelope-from kostikbel@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: from tom.home (kib@localhost [127.0.0.1] (may be forged)) by kib.kiev.ua (8.18.1/8.18.1) with ESMTP id 42IJflGB000230; Mon, 18 Mar 2024 21:41:50 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 42IJflGB000230 Received: (from kostik@localhost) by tom.home (8.18.1/8.18.1/Submit) id 42IJflUr000229; Mon, 18 Mar 2024 21:41:47 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 18 Mar 2024 21:41:47 +0200 From: Konstantin Belousov To: Drew Gallatin Cc: Mike Karels , tuexen , Nuno Teixeira , garyj@gmx.de, current@freebsd.org, net@freebsd.org, Randall Stewart Subject: Re: Request for Testing: TCP RACK Message-ID: References: <6e795e9c-8de4-4e02-9a96-8fabfaa4e66f@app.fastmail.com> <6047C8EF-B1B0-4286-93FA-AA38F8A18656@karels.net> <8031cd99-ded8-4b06-93b3-11cc729a8b2c@app.fastmail.com> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <8031cd99-ded8-4b06-93b3-11cc729a8b2c@app.fastmail.com> X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-14) on tom.home X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US] X-Rspamd-Queue-Id: 4Tz4xL64Xyz47lj On Mon, Mar 18, 2024 at 03:13:11PM -0400, Drew Gallatin wrote: > I got the idea from > https://people.mpi-sws.org/~druschel/publications/soft-timers-tocs.pdf > The gist is that the TCP pacing stuff needs to run frequently, and > rather than run it out of a clock interrupt, its more efficient to run > it out of a system call context at just the point where we return to > userspace and the cache is trashed anyway. The current implementation > is fine for our workload, but probably not idea for a generic system. > Especially one where something is banging on system calls. > > Ast's could be the right tool for this, but I'm super unfamiliar with > them, and I can't find any docs on them. > > Would ast_register(0, ASTR_UNCOND, 0, func) be roughly equivalent to > what's happening here? This call would need some AST number added, and then it registers the ast to run on next return to userspace, for the current thread. Is it enough? > > Drew > > On Mon, Mar 18, 2024, at 2:33 PM, Konstantin Belousov wrote: > > On Mon, Mar 18, 2024 at 07:26:10AM -0500, Mike Karels wrote: > > > On 18 Mar 2024, at 7:04, tuexen@freebsd.org wrote: > > > > > > >> On 18. Mar 2024, at 12:42, Nuno Teixeira wrote: > > > >> > > > >> Hello all! > > > >> > > > >> It works just fine! > > > >> System performance is OK. > > > >> Using patch on main-n268841-b0aaf8beb126(-dirty). > > > >> > > > >> --- > > > >> net.inet.tcp.functions_available: > > > >> Stack D Alias PCB count > > > >> freebsd freebsd 0 > > > >> rack * rack 38 > > > >> --- > > > >> > > > >> It would be so nice that we can have a sysctl tunnable for this patch > > > >> so we could do more tests without recompiling kernel. > > > > Thanks for testing! > > > > > > > > @gallatin: can you come up with a patch that is acceptable for Netflix > > > > and allows to mitigate the performance regression. > > > > > > Ideally, tcphpts could enable this automatically when it starts to be > > > used (enough?), but a sysctl could select auto/on/off. > > There is already a well-known mechanism to request execution of the > > specific function on return to userspace, namely AST. The difference > > with the current hack is that the execution is requested for one callback > > in the context of the specific thread. > > > > Still, it might be worth a try to use it; what is the reason to hit a thread > > that does not do networking, with TCP processing? > > > > > > > > Mike > > > > > > > Best regards > > > > Michael > > > >> > > > >> Thanks all! > > > >> Really happy here :) > > > >> > > > >> Cheers, > > > >> > > > >> Nuno Teixeira escreveu (domingo, 17/03/2024 à(s) 20:26): > > > >>> > > > >>> Hello, > > > >>> > > > >>>> I don't have the full context, but it seems like the complaint is a performance regression in bonnie++ and perhaps other things when tcp_hpts is loaded, even when it is not used. Is that correct? > > > >>>> > > > >>>> If so, I suspect its because we drive the tcp_hpts_softclock() routine from userret(), in order to avoid tons of timer interrupts and context switches. To test this theory, you could apply a patch like: > > > >>> > > > >>> It's affecting overall system performance, bonnie was just a way to > > > >>> get some numbers to compare. > > > >>> > > > >>> Tomorrow I will test patch. > > > >>> > > > >>> Thanks! > > > >>> > > > >>> -- > > > >>> Nuno Teixeira > > > >>> FreeBSD Committer (ports) > > > >> > > > >> > > > >> > > > >> -- > > > >> Nuno Teixeira > > > >> FreeBSD Committer (ports) > > > > > From nobody Mon Mar 18 19:42:42 2024 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Tz4yc6SHbz5DrRX; Mon, 18 Mar 2024 19:43:04 +0000 (UTC) (envelope-from gallatin@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Tz4yc5mjGz4CNN; Mon, 18 Mar 2024 19:43:04 +0000 (UTC) (envelope-from gallatin@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1710790984; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Q09t9S3BPqCd2J+f0CbsV0jK2W3DmzGtvs+ffQMpKac=; b=I8/m2ozO2d3OtmpIjXgFPfpAXa1bybZ/KDOMR3BDiQ0s0+SO1lPyi+Nis4yMOCFY4uV/6/ YB9ZVyw9kTqwLGDE6lKFkKS0l+i+8b1rsNAn044Q83chS6epxhITsHCeyVAUAixKA2fTfR OMt49rBtpQb/cKjL2KSdt/obCVsNWNS7rt9RntxRuKBftXgDNTkB2bvqnXqld9H/ZOK0E/ j231SyP7PnySE/vgZIB9q7BU/JU34dw8Un6n6+x0En28KGMw4m6M6cCRv5q4mRWFHmiexN ObZHQk+3rvZZyC9LN2vxj2ugGGfV78QcRt7Wt+neFcSfdUWQNmBUo5+GsdDgRg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1710790984; a=rsa-sha256; cv=none; b=DBJIJL55XWpi2xUKQfZYFCS4khZ5/b4tol0FnKSbOQbsYEzC2vCJ06GJLVRiD0itRMINh7 rTsTkk1zHjlAoBnVsQ4e1AY35PEvB9BzU/hiqzRyzy614rfOH0DrEel81kj7/wd32R8wWX BxVYUmW6N0qQmgdVNfB8yqlU/42LJA/AO5fn9e9HKl/3z7R0hDtRQN/CMYlZP+5XpLC8T7 XlkKa8bMAmJQ8G6DMUi99N7cyGF0ogv20Ib9fXoWgroyFga9Cd9LsY/lMxIdceJ0Ub4jgH 9HjG/Q+CVv/trSLsW27BPsU0QWCX2o7SIEBumaObp651kpCyDICJ6C08z51oeQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1710790984; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Q09t9S3BPqCd2J+f0CbsV0jK2W3DmzGtvs+ffQMpKac=; b=SmUyykqshjgTYeZ6APoFuG3Y8GOC7pYqtNFBA5cQkXUZh1RoOiJ+Qy2iQiYOaRb1lHAZr8 Ouf5N5FwtRbS2FeCEDZwETwJpe0I/Z2q697w2gmGeUIFMgQ8FSoByuvmOvMgaRy/EwYQzu rYmoj2YpErBUieEQ1DsXquNevI5V4vB3KBJLpYw4Job7gXOepvVJLfS/78r8Ka4PgpO/II 5Kp0TUVHX6e2PdFVB2Mc6kJdKDDz16rPDufea3ZgdlmT6sLMMsKzP/OKyZGduCuUn2Rpd1 Qr1vSSmc7tP/8PZjcP/LsW5jVp5u5SeY+6VL2iI9nhRDQ8k5dKDeYfAIIYLE1w== Received: from fauth1-smtp.messagingengine.com (fauth1-smtp.messagingengine.com [103.168.172.200]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: gallatin) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Tz4yc3kdxzPj4; Mon, 18 Mar 2024 19:43:04 +0000 (UTC) (envelope-from gallatin@freebsd.org) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailfauth.nyi.internal (Postfix) with ESMTP id 027CB120006B; Mon, 18 Mar 2024 15:43:03 -0400 (EDT) Received: from imap53 ([10.202.2.103]) by compute5.internal (MEProxy); Mon, 18 Mar 2024 15:43:04 -0400 X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrkeejgdduvdekucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepofgfggfkjghffffhvfevufgtsegrtderreerreejnecuhfhrohhmpedfffhr vgifucfirghllhgrthhinhdfuceoghgrlhhlrghtihhnsehfrhgvvggsshgurdhorhhgqe enucggtffrrghtthgvrhhnpefgteeluefggeevheefheejgedtvdelheejfffhgeeuhfel keevheeiieeuhfefieenucffohhmrghinhepmhhpihdqshifshdrohhrghenucevlhhush htvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehgrghllhgrthhinhdo mhgvshhmthhprghuthhhphgvrhhsohhnrghlihhthidqudeffeehledvvdduiedqvdelhe dtgedukeegqdhgrghllhgrthhinheppehfrhgvvggsshgurdhorhhgsehfrghsthhmrghi lhdrtghomh X-ME-Proxy: Feedback-ID: i41414658:Fastmail Received: by mailuser.nyi.internal (Postfix, from userid 501) id B879D36400BB; Mon, 18 Mar 2024 15:43:03 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.11.0-alpha0-300-gdee1775a43-fm-20240315.001-gdee1775a List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Message-Id: <38c54399-6c96-44d8-a3a2-3cc1bfbe50c2@app.fastmail.com> In-Reply-To: References: <6e795e9c-8de4-4e02-9a96-8fabfaa4e66f@app.fastmail.com> <6047C8EF-B1B0-4286-93FA-AA38F8A18656@karels.net> <8031cd99-ded8-4b06-93b3-11cc729a8b2c@app.fastmail.com> Date: Mon, 18 Mar 2024 15:42:42 -0400 From: "Drew Gallatin" To: "Konstantin Belousov" Cc: "Mike Karels" , tuexen , "Nuno Teixeira" , garyj@gmx.de, current@freebsd.org, net@freebsd.org, "Randall Stewart" Subject: Re: Request for Testing: TCP RACK Content-Type: multipart/alternative; boundary=49341a7599d5444a8bbcc6b7abbe0677 --49341a7599d5444a8bbcc6b7abbe0677 Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: quoted-printable No. The goal is to run on every return to userspace for every thread. Drew On Mon, Mar 18, 2024, at 3:41 PM, Konstantin Belousov wrote: > On Mon, Mar 18, 2024 at 03:13:11PM -0400, Drew Gallatin wrote: > > I got the idea from > > https://people.mpi-sws.org/~druschel/publications/soft-timers-tocs.p= df > > The gist is that the TCP pacing stuff needs to run frequently, and > > rather than run it out of a clock interrupt, its more efficient to r= un > > it out of a system call context at just the point where we return to > > userspace and the cache is trashed anyway. The current implementation > > is fine for our workload, but probably not idea for a generic system. > > Especially one where something is banging on system calls. > > > > Ast's could be the right tool for this, but I'm super unfamiliar with > > them, and I can't find any docs on them. > > > > Would ast_register(0, ASTR_UNCOND, 0, func) be roughly equivalent to > > what's happening here? > This call would need some AST number added, and then it registers the > ast to run on next return to userspace, for the current thread. >=20 > Is it enough? > > > > Drew >=20 > >=20 > > On Mon, Mar 18, 2024, at 2:33 PM, Konstantin Belousov wrote: > > > On Mon, Mar 18, 2024 at 07:26:10AM -0500, Mike Karels wrote: > > > > On 18 Mar 2024, at 7:04, tuexen@freebsd.org wrote: > > > >=20 > > > > >> On 18. Mar 2024, at 12:42, Nuno Teixeira wrote: > > > > >> > > > > >> Hello all! > > > > >> > > > > >> It works just fine! > > > > >> System performance is OK. > > > > >> Using patch on main-n268841-b0aaf8beb126(-dirty). > > > > >> > > > > >> --- > > > > >> net.inet.tcp.functions_available: > > > > >> Stack D Alias = PCB count > > > > >> freebsd freebsd = 0 > > > > >> rack * rack = 38 > > > > >> --- > > > > >> > > > > >> It would be so nice that we can have a sysctl tunnable for th= is patch > > > > >> so we could do more tests without recompiling kernel. > > > > > Thanks for testing! > > > > > > > > > > @gallatin: can you come up with a patch that is acceptable for= Netflix > > > > > and allows to mitigate the performance regression. > > > >=20 > > > > Ideally, tcphpts could enable this automatically when it starts = to be > > > > used (enough?), but a sysctl could select auto/on/off. > > > There is already a well-known mechanism to request execution of the > > > specific function on return to userspace, namely AST. The differe= nce > > > with the current hack is that the execution is requested for one c= allback > > > in the context of the specific thread. > > >=20 > > > Still, it might be worth a try to use it; what is the reason to hi= t a thread > > > that does not do networking, with TCP processing? > > >=20 > > > >=20 > > > > Mike > > > >=20 > > > > > Best regards > > > > > Michael > > > > >> > > > > >> Thanks all! > > > > >> Really happy here :) > > > > >> > > > > >> Cheers, > > > > >> > > > > >> Nuno Teixeira escreveu (domingo, 17/03/= 2024 =C3=A0(s) 20:26): > > > > >>> > > > > >>> Hello, > > > > >>> > > > > >>>> I don't have the full context, but it seems like the compla= int is a performance regression in bonnie++ and perhaps other things whe= n tcp_hpts is loaded, even when it is not used. Is that correct? > > > > >>>> > > > > >>>> If so, I suspect its because we drive the tcp_hpts_softcloc= k() routine from userret(), in order to avoid tons of timer interrupts a= nd context switches. To test this theory, you could apply a patch like: > > > > >>> > > > > >>> It's affecting overall system performance, bonnie was just a= way to > > > > >>> get some numbers to compare. > > > > >>> > > > > >>> Tomorrow I will test patch. > > > > >>> > > > > >>> Thanks! > > > > >>> > > > > >>> -- > > > > >>> Nuno Teixeira > > > > >>> FreeBSD Committer (ports) > > > > >> > > > > >> > > > > >> > > > > >> --=20 > > > > >> Nuno Teixeira > > > > >> FreeBSD Committer (ports) > > > >=20 > > >=20 >=20 --49341a7599d5444a8bbcc6b7abbe0677 Content-Type: text/html;charset=utf-8 Content-Transfer-Encoding: quoted-printable
No.  The g= oal is to run on every return to userspace for every thread.

Drew

On Mon, Mar 18, 2024= , at 3:41 PM, Konstantin Belousov wrote:
On Mon, Mar 18, 2024 at 03:13:11PM -0400, = Drew Gallatin wrote:
> I got the idea from
> https://people.mpi-sws.org/~druschel/publications= /soft-timers-tocs.pdf
> The gist is that the TCP pa= cing stuff needs to run frequently, and
> rather than r= un it out of a clock interrupt, its more efficient to run
= > it out of a system call context at just the point where we return t= o
> userspace and the cache is trashed anyway. The curr= ent implementation
> is fine for our workload, but prob= ably not idea for a generic system.
> Especially one wh= ere something is banging on system calls.
>
> Ast's could be the right tool for this, but I'm super unfamiliar= with
> them, and I can't find any docs on them.
>
> Would ast_register(0, ASTR_UNCOND, 0, fu= nc) be roughly equivalent to
> what's happening here?
This call would need some AST number added, and then it reg= isters the
ast to run on next return to userspace, for the= current thread.

Is it enough?
>
> Drew

> 
> On Mon, Mar 18, 2024, at 2:33 PM, Konstantin Belousov = wrote:
> > On Mon, Mar 18, 2024 at 07:26:10AM -0500,= Mike Karels wrote:
> > > On 18 Mar 2024, at 7:04= , tuexen@freebsd.org wrot= e:
> > > 
> > > >&= gt; On 18. Mar 2024, at 12:42, Nuno Teixeira <eduardo@freebsd.org> wrote:
> &g= t; > >>
> > > >> Hello all!
> > > >>
> > > >> It= works just fine!
> > > >> System performan= ce is OK.
> > > >> Using patch on main-n268= 841-b0aaf8beb126(-dirty).
> > > >>
> > > >> ---
> > > >> = net.inet.tcp.functions_available:
> > > >> = Stack           &= nbsp;           &= nbsp;   D Alias        = ;            = ;        PCB count
>= > > >> freebsd       &nb= sp;           &nb= sp;       freebsd    &= nbsp;           &= nbsp;         0
&g= t; > > >> rack       &nbs= p;           &nbs= p;        * rack   &nb= sp;           &nb= sp;           &nb= sp; 38
> > > >> ---
> >= > >>
> > > >> It would be so nice= that we can have a sysctl tunnable for this patch
> &g= t; > >> so we could do more tests without recompiling kernel.
> > > > Thanks for testing!
> = > > >
> > > > @gallatin: can you come= up with a patch that is acceptable for Netflix
> > = > > and allows to mitigate the performance regression.
> > > 
> > > Ideally, tcphpts co= uld enable this automatically when it starts to be
> &g= t; > used (enough?), but a sysctl could select auto/on/off.
=
> > There is already a well-known mechanism to request execut= ion of the
> > specific function on return to usersp= ace, namely AST.  The difference
> > with the c= urrent hack is that the execution is requested for one callback
> > in the context of the specific thread.
>= ; > 
> > Still, it might be worth a try to u= se it; what is the reason to hit a thread
> > that d= oes not do networking, with TCP processing?
> > = ;
> > > 
> > > Mike
> > > 
> > > > Best= regards
> > > > Michael
> &g= t; > >>
> > > >> Thanks all!
> > > >> Really happy here :)
>= > > >>
> > > >> Cheers,
> > > >>
> > > >> Nu= no Teixeira <eduardo@freebsd.o= rg> escreveu (domingo, 17/03/2024 =C3=A0(s) 20:26):
> > > >>>
> > > >>> H= ello,
> > > >>>
> > = > >>>> I don't have the full context, but it seems like t= he complaint is a performance regression in bonnie++ and perhaps other t= hings when tcp_hpts is loaded, even when it is not used.  Is that c= orrect?
> > > >>>>
>= > > >>>> If so, I suspect its because we drive the tc= p_hpts_softclock() routine from userret(), in order to avoid tons of tim= er interrupts and context switches.  To test this theory,  you= could apply a patch like:
> > > >>>
=
> > > >>> It's affecting overall system per= formance, bonnie was just a way to
> > > >>= > get some numbers to compare.
> > > >>&= gt;
> > > >>> Tomorrow I will test patch= .
> > > >>>
> > >= >>> Thanks!
> > > >>>
> > > >>> --
> > > >&g= t;> Nuno Teixeira
> > > >>> FreeBSD C= ommitter (ports)
> > > >>
>= ; > > >>
> > > >>
> > > >> -- 
> > > >> = Nuno Teixeira
> > > >> FreeBSD Committer (p= orts)
> > > 
> > 


--49341a7599d5444a8bbcc6b7abbe0677-- From nobody Mon Mar 18 21:01:28 2024 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Tz6j70tPnz5F0jt; Mon, 18 Mar 2024 21:01:31 +0000 (UTC) (envelope-from glebius@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Tz6j70C1fz4P0K; Mon, 18 Mar 2024 21:01:31 +0000 (UTC) (envelope-from glebius@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1710795691; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=gg1RjIp8mwiSaxQ/2AAJtWTaI4RJI/I+/aEogie+sdQ=; b=pdJ5gAcHXxjvvjdCQkCDLFe+ZyHuBgWi3NKnO6HXkP/VRE+WuX3VZb6r1ZKjVWh9pu3UVl mRzTEe8aoRk+y2Y7QqieZg5Iknw+CwG2C4/E6t1PL+xOYGNtjzaFGleu1/O43GRzuvTCdl TPnu1N0OptcJuuSIXPKo+Yy3c97SbBfYuR19e/8mLr1NkZF8gso0magySUZv3rEuQ8KBMi iZdJ8lDv39VxGf98EL314IUAdc348ewYsRz/eWIG97GLn9miy/HhkMlgCD+VB/U6gujGG5 6zJmEEFt4x60PWi0GDPb+VnmWxNZVckH4fmmOZq2NW8j9tQ92Ggv17YxETu37w== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1710795691; a=rsa-sha256; cv=none; b=o+Ijj9d64kXee8jgUaMvamrP9Sjzp1hdOnwEfWJLvcsLLTuZYMOSHvo2o0alQTbqsYOipk caAEoxWd0pikcyRDs4Zl36P239i/RU2x+yo210OM6i0TLrwPCWdBBskETd3z2gKS8B7AoK 69trVsc0y9FTCs6eErb57omNyto0CZdUtI7GjTw+sC2dsz9wdG9GVm/PruFky+/Ep5Z5Jz Zrnme+uJoQp5sZsIMQcBxvVq2cgLzyY8nmYprrZttwJM/CzeO9FdrI2Ugm8g8KLeHUMQnh hoLjxOx2O5jsbbIzwjzuUxlcC16aOEW+TxwsD3JW4/ZhfWdGPxb/0sh3iLrWoQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1710795691; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=gg1RjIp8mwiSaxQ/2AAJtWTaI4RJI/I+/aEogie+sdQ=; b=unISR5GF2XLZEG4RxBm04xhgeQuARiDZzL3XNpWfLKCMB+8D/5k/GBSNR3xuojtdLm/fZu CyYcSxYudfVR4U70vRbQAM3GubDALsf6LQmW1Fwzi9TPaZYD5N6P2HiRuAF0RnIBJhGFt/ erkK9cFM2OipVqvaZSuGrbO4Hl2RoJpRK5eZg0qSCfqu7tFrobGBdzV3huFOYcqfoiYZSr GcxpRVNabIslhpShnjTOTeeF+VT+KU1qEFdn76j3jEAY2OMtIKcaRXTsm2Kcd1RdNwh1nj qehrOP3wpidfz47cmpPsf71FsMEaUIYwe6JhfrBF6urfbA4ZlZCFgX9fb8FxYA== Received: from cell.glebi.us (glebi.us [162.251.186.162]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: glebius) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Tz6j64JpZzRcY; Mon, 18 Mar 2024 21:01:30 +0000 (UTC) (envelope-from glebius@freebsd.org) Date: Mon, 18 Mar 2024 14:01:28 -0700 From: Gleb Smirnoff To: dev-commits-src-main@freebsd.org, current@freebsd.org Subject: Re: git: e34ea0196f44 - main - tcp: clear all TCP timers in tcp_timer_stop() when in callout Message-ID: References: <202403182057.42IKvAad009676@gitrepo.freebsd.org> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <202403182057.42IKvAad009676@gitrepo.freebsd.org> Hi! This commit is supposed to fix a problem we do not have a reproducer for. The problem was that sometimes a TCP connection enters tcp_discardcb() with a scheduled timer. A temporary patch 57e27ff07aff was committed to mask the problem. This patch is supposed to fix the root cause. However, again, we were not able to reproduce the problem reliably and thus were not able to test that the just committed patch indeed fixes the root cause. But the assertion is put back to tcp_discardcb(). In case you got a panic in tcp_discardcb(), please email me ASAP. On Mon, Mar 18, 2024 at 08:57:10PM +0000, Gleb Smirnoff wrote: T> The branch main has been updated by glebius: T> T> URL: https://cgit.FreeBSD.org/src/commit/?id=e34ea0196f4497d3f3939025aff3a89ee77b652e T> T> commit e34ea0196f4497d3f3939025aff3a89ee77b652e T> Author: Gleb Smirnoff T> AuthorDate: 2024-03-18 20:57:00 +0000 T> Commit: Gleb Smirnoff T> CommitDate: 2024-03-18 20:57:00 +0000 T> T> tcp: clear all TCP timers in tcp_timer_stop() when in callout T> T> When a TCP callout decides to disable self, e.g. tcp_timer_2msl() calling T> tcp_close(), we must also clear all other possible timers. Otherwise, T> upon return, the callout would be scheduled again in tcp_timer_enter(). T> T> Revert 57e27ff07aff, which was a temporary partial revert of otherwise T> correct 62d47d73b7eb, that exposed the problem being fixed now. Add an T> extra assertion in tcp_timer_enter() to check we aren't arming callout for T> a closed connection. T> T> Reviewed by: rscheff T> --- T> sys/netinet/tcp_subr.c | 3 +-- T> sys/netinet/tcp_timer.c | 6 ++++-- T> 2 files changed, 5 insertions(+), 4 deletions(-) T> T> diff --git a/sys/netinet/tcp_subr.c b/sys/netinet/tcp_subr.c T> index f618bc1ba04b..a6f84c297688 100644 T> --- a/sys/netinet/tcp_subr.c T> +++ b/sys/netinet/tcp_subr.c T> @@ -2395,10 +2395,9 @@ tcp_discardcb(struct tcpcb *tp) T> #endif T> T> INP_WLOCK_ASSERT(inp); T> + MPASS(!callout_active(&tp->t_callout)); T> MPASS(TAILQ_EMPTY(&tp->snd_holes)); T> T> - tcp_timer_stop(tp); T> - T> /* free the reassembly queue, if any */ T> tcp_reass_flush(tp); T> T> diff --git a/sys/netinet/tcp_timer.c b/sys/netinet/tcp_timer.c T> index ed50659abf8e..785f68be5621 100644 T> --- a/sys/netinet/tcp_timer.c T> +++ b/sys/netinet/tcp_timer.c T> @@ -881,6 +881,7 @@ tcp_timer_enter(void *xtp) T> if (tp_valid) { T> tcp_bblog_timer(tp, which, TT_PROCESSED, 0); T> if ((which = tcp_timer_next(tp, &precision)) != TT_N) { T> + MPASS(tp->t_state > TCPS_CLOSED); T> callout_reset_sbt_on(&tp->t_callout, T> tp->t_timers[which], precision, tcp_timer_enter, T> tp, inp_to_cpuid(inp), C_ABSOLUTE); T> @@ -939,8 +940,7 @@ tcp_timer_active(struct tcpcb *tp, tt_which which) T> /* T> * Stop all timers associated with tcpcb. T> * T> - * Called only on tcpcb destruction. The tcpcb shall already be dropped from T> - * the pcb lookup database and socket is not losing the last reference. T> + * Called when tcpcb moves to TCPS_CLOSED. T> * T> * XXXGL: unfortunately our callout(9) is not able to fully stop a locked T> * callout even when only two threads are involved: the callout itself and the T> @@ -963,6 +963,8 @@ tcp_timer_stop(struct tcpcb *tp) T> T> stopped = callout_stop(&tp->t_callout); T> MPASS(stopped == 0); T> + for (tt_which i = 0; i < TT_N; i++) T> + tp->t_timers[i] = SBT_MAX; T> } else while(__predict_false(callout_stop(&tp->t_callout) == 0)) { T> INP_WUNLOCK(inp); T> kern_yield(PRI_UNCHANGED); -- Gleb Smirnoff From nobody Tue Mar 19 06:54:27 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TzMsR6lKTz5Dh4w for ; Tue, 19 Mar 2024 06:54:35 +0000 (UTC) (envelope-from agh@riseup.net) Received: from mx0.riseup.net (mx0.riseup.net [198.252.153.6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mx0.riseup.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TzMsR0hJZz4CkF for ; Tue, 19 Mar 2024 06:54:35 +0000 (UTC) (envelope-from agh@riseup.net) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=riseup.net header.s=squak header.b="Abc/8Akm"; dmarc=pass (policy=none) header.from=riseup.net; spf=pass (mx1.freebsd.org: domain of agh@riseup.net designates 198.252.153.6 as permitted sender) smtp.mailfrom=agh@riseup.net Received: from fews01-sea.riseup.net (fews01-sea-pn.riseup.net [10.0.1.109]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx0.riseup.net (Postfix) with ESMTPS id 4TzMsJ062jz9sVn for ; Tue, 19 Mar 2024 06:54:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=riseup.net; s=squak; t=1710831273; bh=AhtWYtV47OgBwy6pwOQeKr/6b35OjY2jaczR/4CXm/w=; h=Date:From:To:Subject:From; b=Abc/8Akmkv4MWEQ0afQ7JBUHaFjKWkkReXEOBusgMLIzV0HCPhL3vQ6EWIC4ahj59 c3qd5TshulLygRLrfDHmoURgEaaUQke+5wjA1lCI68WkJlDBFburT2VaUiJYMU5Hfk sCmdqQuo2gmCNaDUzI1XjAzmI4QBetntOUjNNH+8= X-Riseup-User-ID: 78E8CDA1ECAA00B56C4DAEDC70F6179C02FB118D27ABE9BA1C1D8752841559CD Received: from [127.0.0.1] (localhost [127.0.0.1]) by fews01-sea.riseup.net (Postfix) with ESMTPSA id 4TzMsH67VvzJqhR for ; Tue, 19 Mar 2024 06:54:27 +0000 (UTC) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Date: Tue, 19 Mar 2024 06:54:27 +0000 From: Alastair Hogge To: Freebsd Current Subject: sysutils/pam_xdg: Cancelled on -CURRENT Message-ID: <4e4a5f033f4169dd07f4afdd7b5f6976@riseup.net> Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spamd-Bar: ----- X-Spamd-Result: default: False [-5.10 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; DWL_DNSWL_LOW(-1.00)[riseup.net:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[riseup.net,none]; R_DKIM_ALLOW(-0.20)[riseup.net:s=squak]; R_SPF_ALLOW(-0.20)[+a:mx0.riseup.net]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[198.252.153.6:from]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MISSING_XM_UA(0.00)[]; ASN(0.00)[asn:16652, ipnet:198.252.153.0/24, country:US]; RCVD_TLS_ALL(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[riseup.net:+] X-Rspamd-Queue-Id: 4TzMsR0hJZz4CkF Hello, Recently a similar module (PAM) mentioned in the subject was committed to base[1]. The module in base masks the currently installed Port, the man page can be accessed with man -M /usr/local/share/man 8 pam_xdg, however, I can now no longer build the Port. I noticed that the base module has no WITHOUT_ option, tho, that might be extreme for one module, but then again, the base module masks a more feature full module. What is the practice to enable use of the Port again? At the moment I am updating my host, and testing the following: diff --git a/lib/libpam/modules/modules.inc b/lib/libpam/modules/modules.inc index f3ab65333f4f..ddbb326f0312 100644 --- a/lib/libpam/modules/modules.inc +++ b/lib/libpam/modules/modules.inc @@ -30,4 +30,3 @@ MODULES += pam_ssh .endif MODULES += pam_tacplus MODULES += pam_unix -MODULES += pam_xdg \ No newline at end of file 1: https://cgit.freebsd.org/src/commit/?id=6e69612d5df1c1d5bd86990ea4d9a170c030b292 Thanks. From nobody Tue Mar 19 07:23:06 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TzNVb2zZrz5DkVj for ; Tue, 19 Mar 2024 07:23:19 +0000 (UTC) (envelope-from manu@bidouilliste.com) Received: from mx.blih.net (mx.blih.net [212.83.155.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TzNVY73lGz4Gr4 for ; Tue, 19 Mar 2024 07:23:17 +0000 (UTC) (envelope-from manu@bidouilliste.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bidouilliste.com; s=mx; t=1710832990; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dnFBJs90vF4RefTxcBYy11+MATAKaBvU4s6WMIh1+wE=; b=FQHmWJ+2H7CL1U49jzWGHoLqAiTU+CClylSv4DEyAxWvFFFhe7Z8q7j7alPgYrwp4wElZp NA1shQeL4AM+g6zz0XoPrpsfyc1XW/dZXtOOWh5s0GkkYXF62bEMg6nxdoKXk/zN5CkwtS alCOlkot7cgCVO0FxYbGaeCGVSNHuHk= Received: from skull.home.blih.net (lfbn-lyo-1-2174-135.w90-66.abo.wanadoo.fr [90.66.97.135]) by mx.blih.net (OpenSMTPD) with ESMTPSA id 93692e73 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 19 Mar 2024 07:23:09 +0000 (UTC) Date: Tue, 19 Mar 2024 08:23:06 +0100 From: Emmanuel Vadot To: Alastair Hogge Cc: Freebsd Current Subject: Re: sysutils/pam_xdg: Cancelled on -CURRENT Message-Id: <20240319082306.f4ffef050d8439be07b10962@bidouilliste.com> In-Reply-To: <4e4a5f033f4169dd07f4afdd7b5f6976@riseup.net> References: <4e4a5f033f4169dd07f4afdd7b5f6976@riseup.net> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; amd64-portbld-freebsd15.0) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:12876, ipnet:212.83.128.0/19, country:FR] X-Rspamd-Queue-Id: 4TzNVY73lGz4Gr4 Hi, On Tue, 19 Mar 2024 06:54:27 +0000 Alastair Hogge wrote: > Hello, > > Recently a similar module (PAM) mentioned in the subject was committed > to base[1]. The module in base masks the currently installed Port, the > man page can be accessed with man -M /usr/local/share/man 8 pam_xdg, > however, I can now no longer build the Port. I noticed that the base > module has no WITHOUT_ option, tho, that might be extreme for one > module, but then again, the base module masks a more feature full > module. What is the practice to enable use of the Port again? At the > moment I am updating my host, and testing the following: > > diff --git a/lib/libpam/modules/modules.inc > b/lib/libpam/modules/modules.inc > index f3ab65333f4f..ddbb326f0312 100644 > --- a/lib/libpam/modules/modules.inc > +++ b/lib/libpam/modules/modules.inc > @@ -30,4 +30,3 @@ MODULES += pam_ssh > .endif > MODULES += pam_tacplus > MODULES += pam_unix > -MODULES += pam_xdg > \ No newline at end of file > > 1: > https://cgit.freebsd.org/src/commit/?id=6e69612d5df1c1d5bd86990ea4d9a170c030b292 > > Thanks. > I don't see why you can't build the ports. Using would be a problem but why do you want to use it now that we have one in base ? Do you have any problems with the one in base ? Cheers, -- Emmanuel Vadot From nobody Tue Mar 19 07:55:15 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TzPCV0ZMLz5DmpK for ; Tue, 19 Mar 2024 07:55:18 +0000 (UTC) (envelope-from agh@riseup.net) Received: from mx0.riseup.net (mx0.riseup.net [198.252.153.6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mx0.riseup.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TzPCT5lZgz4LGn for ; Tue, 19 Mar 2024 07:55:17 +0000 (UTC) (envelope-from agh@riseup.net) Authentication-Results: mx1.freebsd.org; none Received: from fews01-sea.riseup.net (fews01-sea-pn.riseup.net [10.0.1.109]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx0.riseup.net (Postfix) with ESMTPS id 4TzPCS0rjBz9sSK; Tue, 19 Mar 2024 07:55:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=riseup.net; s=squak; t=1710834916; bh=nhPX1faseCUwrZlMN+Soocscrll1TRDI62d0y58PIgY=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=ZqTr+cVke3PpAofX6t9G5I6mML2zgsJqZ3tK+94LwOMdhlaZh0mDYooFZbZzP9zuK MYrS0hHARC/gRCcsubEXcfzH5GR29RRV5J0LWO8DR1f9vxAIxyghagHSZUnZSfbK5q iQQgbvhtYHUrEqLq3StCKnP1dHaLX6sZmnQNgR7k= X-Riseup-User-ID: C6A1F203F78B3A4EE97F9BB0E99FA628786C2671FC97417AF9EDA06A390CB206 Received: from [127.0.0.1] (localhost [127.0.0.1]) by fews01-sea.riseup.net (Postfix) with ESMTPSA id 4TzPCR6BgKzJssH; Tue, 19 Mar 2024 07:55:15 +0000 (UTC) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Date: Tue, 19 Mar 2024 07:55:15 +0000 From: Alastair Hogge To: Emmanuel Vadot Cc: Freebsd Current Subject: Re: sysutils/pam_xdg: Cancelled on -CURRENT In-Reply-To: <20240319082306.f4ffef050d8439be07b10962@bidouilliste.com> References: <4e4a5f033f4169dd07f4afdd7b5f6976@riseup.net> <20240319082306.f4ffef050d8439be07b10962@bidouilliste.com> Message-ID: Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:16652, ipnet:198.252.153.0/24, country:US] X-Rspamd-Queue-Id: 4TzPCT5lZgz4LGn On 2024-03-19 15:23, Emmanuel Vadot wrote: > Hi, Hey Emmanuel, > On Tue, 19 Mar 2024 06:54:27 +0000 > Alastair Hogge wrote: > >> Hello, >> >> Recently a similar module (PAM) mentioned in the subject was committed >> to base[1]. The module in base masks the currently installed Port, the >> man page can be accessed with man -M /usr/local/share/man 8 pam_xdg, >> however, I can now no longer build the Port. I noticed that the base >> module has no WITHOUT_ option, tho, that might be extreme for one >> module, but then again, the base module masks a more feature full >> module. What is the practice to enable use of the Port again? At the >> moment I am updating my host, and testing the following: >> >> diff --git a/lib/libpam/modules/modules.inc >> b/lib/libpam/modules/modules.inc >> index f3ab65333f4f..ddbb326f0312 100644 >> --- a/lib/libpam/modules/modules.inc >> +++ b/lib/libpam/modules/modules.inc >> @@ -30,4 +30,3 @@ MODULES += pam_ssh >> .endif >> MODULES += pam_tacplus >> MODULES += pam_unix >> -MODULES += pam_xdg >> \ No newline at end of file >> >> 1: >> https://cgit.freebsd.org/src/commit/?id=6e69612d5df1c1d5bd86990ea4d9a170c030b292 >> >> Thanks. >> > > I don't see why you can't build the ports. >From sysutils/pam_xdg[2]: .if exists(/usr/lib/pam_xdg.so) IGNORE= module name conflict with a different implementation in base system .endif > Using would be a problem but why do you want to use it now that we > have one in base ? > Do you have any problems with the one in base ? I would like to continue using sysutils/pam_xdg because it handles all ${XDG_*_HOME}, and local name spaces. 2: https://cgit.freebsd.org/ports/tree/sysutils/pam_xdg/Makefile#n16 Thanks. From nobody Tue Mar 19 08:02:30 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TzPMw6BDqz5DnMx for ; Tue, 19 Mar 2024 08:02:36 +0000 (UTC) (envelope-from manu@bidouilliste.com) Received: from mx.blih.net (mx.blih.net [212.83.155.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TzPMw34p6z4MNN for ; Tue, 19 Mar 2024 08:02:36 +0000 (UTC) (envelope-from manu@bidouilliste.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bidouilliste.com; s=mx; t=1710835351; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aBPZ4HcrBN0kwBV1bQB1BbCsVyo7LeKeuCdeu7C64Bo=; b=ZY5/8eHo2tJabYMNyyOPwfk1emLvlEXTy1U8k0OY88N/otF3J+dN45r9XI1+K75FyEYd/S 4eh+qgZu/tZDoMB+nR/Q2mqxh3ax2zgomfzN9DDD9Whhmlf3/TdNoaGD+FqmH69Uj0FDWp tla6hMfy+jLdlfUG8HGLyqjJDhSCp5E= Received: from skull.home.blih.net (lfbn-lyo-1-2174-135.w90-66.abo.wanadoo.fr [90.66.97.135]) by mx.blih.net (OpenSMTPD) with ESMTPSA id ed22357d (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 19 Mar 2024 08:02:31 +0000 (UTC) Date: Tue, 19 Mar 2024 09:02:30 +0100 From: Emmanuel Vadot To: Alastair Hogge Cc: Freebsd Current Subject: Re: sysutils/pam_xdg: Cancelled on -CURRENT Message-Id: <20240319090230.3a1e7409578f8f4a373a341e@bidouilliste.com> In-Reply-To: References: <4e4a5f033f4169dd07f4afdd7b5f6976@riseup.net> <20240319082306.f4ffef050d8439be07b10962@bidouilliste.com> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; amd64-portbld-freebsd15.0) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:12876, ipnet:212.83.128.0/19, country:FR] X-Rspamd-Queue-Id: 4TzPMw34p6z4MNN On Tue, 19 Mar 2024 07:55:15 +0000 Alastair Hogge wrote: > On 2024-03-19 15:23, Emmanuel Vadot wrote: > > Hi, > > Hey Emmanuel, > > > On Tue, 19 Mar 2024 06:54:27 +0000 > > Alastair Hogge wrote: > > > >> Hello, > >> > >> Recently a similar module (PAM) mentioned in the subject was committed > >> to base[1]. The module in base masks the currently installed Port, the > >> man page can be accessed with man -M /usr/local/share/man 8 pam_xdg, > >> however, I can now no longer build the Port. I noticed that the base > >> module has no WITHOUT_ option, tho, that might be extreme for one > >> module, but then again, the base module masks a more feature full > >> module. What is the practice to enable use of the Port again? At the > >> moment I am updating my host, and testing the following: > >> > >> diff --git a/lib/libpam/modules/modules.inc > >> b/lib/libpam/modules/modules.inc > >> index f3ab65333f4f..ddbb326f0312 100644 > >> --- a/lib/libpam/modules/modules.inc > >> +++ b/lib/libpam/modules/modules.inc > >> @@ -30,4 +30,3 @@ MODULES += pam_ssh > >> .endif > >> MODULES += pam_tacplus > >> MODULES += pam_unix > >> -MODULES += pam_xdg > >> \ No newline at end of file > >> > >> 1: > >> https://cgit.freebsd.org/src/commit/?id=6e69612d5df1c1d5bd86990ea4d9a170c030b292 > >> > >> Thanks. > >> > > > > I don't see why you can't build the ports. > > From sysutils/pam_xdg[2]: > > if exists(/usr/lib/pam_xdg.so) > IGNORE= module name conflict with a different implementation in > base system > endif Ah yes, I've missed this :) > > Using would be a problem but why do you want to use it now that we > > have one in base ? > > Do you have any problems with the one in base ? > > I would like to continue using sysutils/pam_xdg because it handles all > ${XDG_*_HOME}, and local name spaces. XDG_*_HOME variables aren't needed, all applications must have a fallback to the base directories in the spec and sysutils/pam_xdg doesn't offer to use other directories so that's why I didn't implement those in the base one. What do you mean by "local name spaces" ? > 2: https://cgit.freebsd.org/ports/tree/sysutils/pam_xdg/Makefile#n16 > > Thanks. > Cheers, -- Emmanuel Vadot From nobody Wed Mar 20 01:03:27 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Tzr1y3nCsz5FxJW for ; Wed, 20 Mar 2024 01:03:34 +0000 (UTC) (envelope-from agh@riseup.net) Received: from mx0.riseup.net (mx0.riseup.net [198.252.153.6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mx0.riseup.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Tzr1y0qFkz4WZ4 for ; Wed, 20 Mar 2024 01:03:34 +0000 (UTC) (envelope-from agh@riseup.net) Authentication-Results: mx1.freebsd.org; none Received: from fews01-sea.riseup.net (fews01-sea-pn.riseup.net [10.0.1.109]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx0.riseup.net (Postfix) with ESMTPS id 4Tzr1q6ygRz9sM5; Wed, 20 Mar 2024 01:03:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=riseup.net; s=squak; t=1710896608; bh=wK5XCmiI2jrdZBNpXwuguZy8EpUyCUU20EgVCCOtstE=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=bu1GnOeF2UJz1olNge3nNLu6edp2rsgCp3TPynnlqlQJllGRaiRFjiB93gLRNQV5F pMmNMuLYm+ugItgNV0yfR6SrZnRL21Vw6lvpYGDyXzWFSDuUEuYtf4ijj8E1yhLGlX 1L11mNESr+svwN8MPqeesIPnYsgxZGp+DLa8aUGw= X-Riseup-User-ID: 70FC29C05C2F6E305D5F103D9E0D8B78FB8D90086D39A20851DBA9D954839F85 Received: from [127.0.0.1] (localhost [127.0.0.1]) by fews01-sea.riseup.net (Postfix) with ESMTPSA id 4Tzr1q5pCzzJpZq; Wed, 20 Mar 2024 01:03:27 +0000 (UTC) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Date: Wed, 20 Mar 2024 01:03:27 +0000 From: Alastair Hogge To: Emmanuel Vadot Cc: Freebsd Current Subject: Re: sysutils/pam_xdg: Cancelled on -CURRENT In-Reply-To: <20240319090230.3a1e7409578f8f4a373a341e@bidouilliste.com> References: <4e4a5f033f4169dd07f4afdd7b5f6976@riseup.net> <20240319082306.f4ffef050d8439be07b10962@bidouilliste.com> <20240319090230.3a1e7409578f8f4a373a341e@bidouilliste.com> Message-ID: <544b57564b9c5a2749765cf2007c7153@riseup.net> Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:16652, ipnet:198.252.153.0/24, country:US] X-Rspamd-Queue-Id: 4Tzr1y0qFkz4WZ4 On 2024-03-19 16:02, Emmanuel Vadot wrote: > On Tue, 19 Mar 2024 07:55:15 +0000 > Alastair Hogge wrote: > >> On 2024-03-19 15:23, Emmanuel Vadot wrote: >> > Hi, >> >> Hey Emmanuel, >> >> > On Tue, 19 Mar 2024 06:54:27 +0000 >> > Alastair Hogge wrote: >> > >> >> Hello, >> >> >> >> Recently a similar module (PAM) mentioned in the subject was committed >> >> to base[1]. The module in base masks the currently installed Port, the >> >> man page can be accessed with man -M /usr/local/share/man 8 pam_xdg, >> >> however, I can now no longer build the Port. I noticed that the base >> >> module has no WITHOUT_ option, tho, that might be extreme for one >> >> module, but then again, the base module masks a more feature full >> >> module. What is the practice to enable use of the Port again? At the >> >> moment I am updating my host, and testing the following: >> >> >> >> diff --git a/lib/libpam/modules/modules.inc >> >> b/lib/libpam/modules/modules.inc >> >> index f3ab65333f4f..ddbb326f0312 100644 >> >> --- a/lib/libpam/modules/modules.inc >> >> +++ b/lib/libpam/modules/modules.inc >> >> @@ -30,4 +30,3 @@ MODULES += pam_ssh >> >> .endif >> >> MODULES += pam_tacplus >> >> MODULES += pam_unix >> >> -MODULES += pam_xdg >> >> \ No newline at end of file >> >> >> >> 1: >> >> https://cgit.freebsd.org/src/commit/?id=6e69612d5df1c1d5bd86990ea4d9a170c030b292 >> >> >> >> Thanks. >> >> >> > >> > I don't see why you can't build the ports. >> >> From sysutils/pam_xdg[2]: >> >> if exists(/usr/lib/pam_xdg.so) >> IGNORE= module name conflict with a different implementation in >> base system >> endif > > Ah yes, I've missed this :) > >> > Using would be a problem but why do you want to use it now that we >> > have one in base ? >> > Do you have any problems with the one in base ? >> >> I would like to continue using sysutils/pam_xdg because it handles all >> ${XDG_*_HOME}, and local name spaces. > > XDG_*_HOME variables aren't needed, all applications must have a > fallback to the base directories in the spec and sysutils/pam_xdg > doesn't offer to use other directories so that's why I didn't implement > those in the base one. > What do you mean by "local name spaces" ? I meant all the other ${XDG_FU} excluding ${XDG_*_HOME}. Anyways, turns out incredibly mistaken. I deployed another corporate craptop from the dumpster today, and the User's homedir was not populated with XDG dirs. I was sure I was using sysutils/pam_xdg for that, but will now have to find my older scripts that predate using sysutils/pam_xdg, to achieve that. Sorry for the noise. Thanks, Alastair From nobody Wed Mar 20 20:44:13 2024 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4V0LDL0pRdz5Dtyk for ; Wed, 20 Mar 2024 20:44:18 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from drew.franken.de (mail-n.franken.de [193.175.24.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.franken.de", Issuer "Sectigo RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4V0LDK1z0fz4R5d for ; Wed, 20 Mar 2024 20:44:17 +0000 (UTC) (envelope-from tuexen@freebsd.org) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=freebsd.org (policy=none); spf=softfail (mx1.freebsd.org: 193.175.24.27 is neither permitted nor denied by domain of tuexen@freebsd.org) smtp.mailfrom=tuexen@freebsd.org Received: from smtpclient.apple (unknown [IPv6:2a02:8109:1140:c3d:499b:2fb4:e4d2:c5fd]) (Authenticated sender: micmac) by drew.franken.de (Postfix) with ESMTPSA id 8E11B721E2806 for ; Wed, 20 Mar 2024 21:44:13 +0100 (CET) From: tuexen@freebsd.org Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.500.171.1.1\)) Subject: Problem with make installworld Message-Id: <3DCD0639-057D-4BC9-96B7-FDD6F05E6BB5@freebsd.org> Date: Wed, 20 Mar 2024 21:44:13 +0100 To: current@freebsd.org X-Mailer: Apple Mail (2.3774.500.171.1.1) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, T_SCC_BODY_TEXT_LINE autolearn=disabled version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail-n.franken.de X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.37 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.77)[-0.775]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_POLICY_SOFTFAIL(0.10)[freebsd.org : No valid SPF, No valid DKIM,none]; RCVD_IN_DNSWL_LOW(-0.10)[193.175.24.27:from]; MIME_TRACE(0.00)[0:+]; RCVD_VIA_SMTP_AUTH(0.00)[]; FREEFALL_USER(0.00)[tuexen]; ARC_NA(0.00)[]; FROM_NO_DN(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:680, ipnet:193.174.0.0/15, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; TO_DN_NONE(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all:c]; FROM_EQ_ENVFROM(0.00)[]; TO_DOM_EQ_FROM_DOM(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MLMMJ_DEST(0.00)[current@freebsd.org]; PREVIOUSLY_DELIVERED(0.00)[current@freebsd.org]; R_DKIM_NA(0.00)[]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Queue-Id: 4V0LDK1z0fz4R5d Dear all, I'm trying to run make buildworld / make installworld on a recent main = branch (some days old). The problem is related to lib/libc/tests/ssp/Makefile which contains: _libclang_rt_ubsan=3D = ${SYSROOT}${SANITIZER_LIBDIR}/libclang_rt.ubsan_standalone-${CRTARCH}.a .if exists(${_libclang_rt_ubsan}) PROGS+=3D h_raw LDADD.h_raw+=3D ${SANITIZER_LDFLAGS} When running make buildworld, we have ${SYSROOT} =3D = /usr/obj/usr/home/tuexen/freebsd-src/powerpc.powerpc64/tmp ${SANITIZER_LIBDIR} =3D /usr/lib/clang/17/lib/freebsd and so the script is looking for = /usr/obj/usr/home/tuexen/freebsd-src/powerpc.powerpc64/tmp/usr/lib/clang/1= 7/lib/freebsd/libclang_rt.ubsan_standalone-powerpc64.a which does not exist: tuexen@blackbird:~ % ls -l = /usr/obj/usr/home/tuexen/freebsd-src/powerpc.powerpc64/tmp/usr/lib/clang/1= 7/lib/freebsd/ total 652 -r--r--r-- 1 root wheel 284316 Mar 20 18:03 = libclang_rt.profile-powerpc.a -r--r--r-- 1 root wheel 380704 Mar 20 17:41 = libclang_rt.profile-powerpc64.a Therefore, h_raw to NOT built. However, when make installworld runs, we have ${SYSROOT} =3D ${SANITIZER_LIBDIR} =3D /usr/lib/clang/17/lib/freebsd and so the script is looking for /usr/lib/clang/17/lib/freebsd/libclang_rt.ubsan_standalone-powerpc64.a which does exist: tuexen@blackbird:~ % ls -l /usr/lib/clang/17/lib/freebsd/ total 47320 -r--r--r-- 1 root wheel 14485032 Dec 24 22:48 = libclang_rt.asan-powerpc64.a -r--r--r-- 1 root wheel 1249352 Dec 24 22:48 = libclang_rt.asan-powerpc64.so -r--r--r-- 1 root wheel 9820 Dec 24 22:48 = libclang_rt.asan-preinit-powerpc64.a -r--r--r-- 1 root wheel 176354 Dec 24 22:48 = libclang_rt.asan_cxx-powerpc64.a -r--r--r-- 1 root wheel 10154 Dec 24 22:48 = libclang_rt.asan_static-powerpc64.a -r--r--r-- 1 root wheel 8261052 Dec 24 22:48 = libclang_rt.msan-powerpc64.a -r--r--r-- 1 root wheel 166924 Dec 24 22:48 = libclang_rt.msan_cxx-powerpc64.a -r--r--r-- 1 root wheel 284316 Dec 24 22:51 = libclang_rt.profile-powerpc.a -r--r--r-- 1 root wheel 380704 Dec 24 22:48 = libclang_rt.profile-powerpc64.a -r--r--r-- 1 root wheel 3925468 Dec 24 22:48 = libclang_rt.stats-powerpc64.a -r--r--r-- 1 root wheel 9770 Dec 24 22:48 = libclang_rt.stats_client-powerpc64.a -r--r--r-- 1 root wheel 14144552 Dec 24 22:48 = libclang_rt.tsan-powerpc64.a -r--r--r-- 1 root wheel 295650 Dec 24 22:48 = libclang_rt.tsan_cxx-powerpc64.a -r--r--r-- 1 root wheel 64462 Dec 24 22:48 = libclang_rt.ubsan_minimal-powerpc64.a -r--r--r-- 1 root wheel 4550190 Dec 24 22:48 = libclang_rt.ubsan_standalone-powerpc64.a -r--r--r-- 1 root wheel 113638 Dec 24 22:48 = libclang_rt.ubsan_standalone_cxx-powerpc64.a Therefore, h_raw is tried to be installed, which fails since it wasn't = built. Is it intended that ${SYSROOT} is different during make installworld and = make buildworld? This is on a Power9 system, but I guess this is not relevant... But = maybe I'm wrong. Best regards Michael From nobody Wed Mar 20 23:27:22 2024 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4V0PrX06QRz5FDBK for ; Wed, 20 Mar 2024 23:27:24 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4V0PrW6Txmz4nkX; Wed, 20 Mar 2024 23:27:23 +0000 (UTC) (envelope-from dim@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1710977243; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0c6r/aSTrs/b9a2iVLNXjWY4sEYBMayb4FH9wB9knWc=; b=PhKI+VC0Fw6Se2jUqbYT9LI4+ZeNsEyTr72G2Wq1zM6/xtG6qq5giz06bIzffKasYXINhJ rlJhWksSUbjSzqQ9wg0Op31/L+WkENzDNmXrjTS973mrw8DA2xpoUy/85TjI1IDU85F66n 0A7TJyGRlcpzRTw/mGV3CpvqKpKWeexQWkJ6kMR1loBHmrpB0dQmbdL+HzfkN916TPgNaB 4/uqnm7op+IbJ+c1Df1z6p1InF8yig7Mgx79CkAyQs2KWfcJUZt2Iu9l1IfVJINS+QejUQ HH26y2tGpt6BJ9gAT6peNhiJotW4rK41u5rzfwbv1VNvT9wFX1XfKr6w95p1zg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1710977243; a=rsa-sha256; cv=none; b=yxAXCsyAxkQ0Z7ed5MFp0LEzZAmztK5Ap7FXi7NQcCuCn2nk7qdGyI8w/0EihoMoQ2PwCn wlHCtCgyCUjzzuYpUQvDG+Bph3SlH55M8uQMePYCDQsZNs6YZZ9aeKrMrxLrBn9upD0m6T +TfDQMM9ohtv4O3nEya6JlCCQGzk+7DXp5yFmxIe+0m71kW4LDmX/N//o2sPf9QuTBXFn5 5D+4uRZekWcK02eFN6Zleil7lYz/UFnhgiumncrEQZyt3rwTWzAPjY2gXHKv3OwaAXEjLe 8rknXBMTekF5cz/yJq7exFw/jquaEpnbdJJaqejZHvJS5/22PxsDFgGIMApPZg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1710977243; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0c6r/aSTrs/b9a2iVLNXjWY4sEYBMayb4FH9wB9knWc=; b=U+MEoqJqbVcP0s3X/uhFqkT0ilWk8OJSj2vQIc2Lgs21hQs2p/1PHmZPpZ2OGFmYpasLxg E9PDO7JyN+NwD95Ll8PZnL3y5paGDRZ+eHKeKrMRGV9BphBzfqPEv+XfYoFf7m1nXEjPy0 MXVBydRbr0UL9/KVoHm8XfPfkGSKoPrSo6gf0xKBsvvMWCbkP12eCVZRsY66WfjUs8qZct VjrwpjCp8Ok+0KwXbpm62CSXHPxD19oySLvdKgN1zb0SixW2146W8RdSe+80lCVueZEKgw l9GxhXyolPfb4XL4F+RaM4uIT95QmyLxztFRt4e0L28IltI1OVcNIiWu6+flow== Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "tensor.andric.com", Issuer "R3" (verified OK)) (Authenticated sender: dim) by smtp.freebsd.org (Postfix) with ESMTPSA id 4V0PrW5M2zzTNJ; Wed, 20 Mar 2024 23:27:23 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtpclient.apple (longrow.home.andric.com [192.168.0.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id BDCCE2005E; Thu, 21 Mar 2024 00:27:22 +0100 (CET) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6.1.1\)) Subject: Re: Problem with make installworld From: Dimitry Andric In-Reply-To: <3DCD0639-057D-4BC9-96B7-FDD6F05E6BB5@freebsd.org> Date: Thu, 21 Mar 2024 00:27:22 +0100 Cc: current@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <3DCD0639-057D-4BC9-96B7-FDD6F05E6BB5@freebsd.org> To: tuexen@freebsd.org X-Mailer: Apple Mail (2.3731.700.6.1.1) On 20 Mar 2024, at 21:44, tuexen@freebsd.org wrote: >=20 > I'm trying to run make buildworld / make installworld on a recent main = branch > (some days old). >=20 > The problem is related to lib/libc/tests/ssp/Makefile > which contains: > _libclang_rt_ubsan=3D = ${SYSROOT}${SANITIZER_LIBDIR}/libclang_rt.ubsan_standalone-${CRTARCH}.a > if exists(${_libclang_rt_ubsan}) > PROGS+=3D h_raw > LDADD.h_raw+=3D ${SANITIZER_LDFLAGS} >=20 > When running make buildworld, we have > ${SYSROOT} =3D = /usr/obj/usr/home/tuexen/freebsd-src/powerpc.powerpc64/tmp > ${SANITIZER_LIBDIR} =3D /usr/lib/clang/17/lib/freebsd > and so the script is looking for > = /usr/obj/usr/home/tuexen/freebsd-src/powerpc.powerpc64/tmp/usr/lib/clang/1= 7/lib/freebsd/libclang_rt.ubsan_standalone-powerpc64.a > which does not exist: > tuexen@blackbird:~ % ls -l = /usr/obj/usr/home/tuexen/freebsd-src/powerpc.powerpc64/tmp/usr/lib/clang/1= 7/lib/freebsd/ > total 652 > -r--r--r-- 1 root wheel 284316 Mar 20 18:03 = libclang_rt.profile-powerpc.a > -r--r--r-- 1 root wheel 380704 Mar 20 17:41 = libclang_rt.profile-powerpc64.a >=20 > Therefore, h_raw to NOT built. As far as I can see, for powerpc64 it should have been built somewhere = during the libraries stage. So it's a bit strange that you don't have = the file. Did you use any special options to build? -Dimitry From nobody Thu Mar 21 00:12:40 2024 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4V0Qrq6Qjpz5FJpW for ; Thu, 21 Mar 2024 00:12:43 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from drew.franken.de (mail-n.franken.de [193.175.24.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.franken.de", Issuer "Sectigo RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4V0Qrq31jkz4tG1; Thu, 21 Mar 2024 00:12:43 +0000 (UTC) (envelope-from tuexen@freebsd.org) Authentication-Results: mx1.freebsd.org; none Received: from smtpclient.apple (unknown [IPv6:2a02:8109:1140:c3d:499b:2fb4:e4d2:c5fd]) (Authenticated sender: micmac) by drew.franken.de (Postfix) with ESMTPSA id A021A721E2806; Thu, 21 Mar 2024 01:12:40 +0100 (CET) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.500.171.1.1\)) Subject: Re: Problem with make installworld From: tuexen@freebsd.org In-Reply-To: Date: Thu, 21 Mar 2024 01:12:40 +0100 Cc: current@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <3DCD0639-057D-4BC9-96B7-FDD6F05E6BB5@freebsd.org> To: Dimitry Andric X-Mailer: Apple Mail (2.3774.500.171.1.1) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, T_SCC_BODY_TEXT_LINE autolearn=disabled version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail-n.franken.de X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:680, ipnet:193.174.0.0/15, country:DE] X-Rspamd-Queue-Id: 4V0Qrq31jkz4tG1 > On 21. Mar 2024, at 00:27, Dimitry Andric wrote: >=20 > On 20 Mar 2024, at 21:44, tuexen@freebsd.org wrote: >>=20 >> I'm trying to run make buildworld / make installworld on a recent = main branch >> (some days old). >>=20 >> The problem is related to lib/libc/tests/ssp/Makefile >> which contains: >> _libclang_rt_ubsan=3D = ${SYSROOT}${SANITIZER_LIBDIR}/libclang_rt.ubsan_standalone-${CRTARCH}.a >> if exists(${_libclang_rt_ubsan}) >> PROGS+=3D h_raw >> LDADD.h_raw+=3D ${SANITIZER_LDFLAGS} >>=20 >> When running make buildworld, we have >> ${SYSROOT} =3D = /usr/obj/usr/home/tuexen/freebsd-src/powerpc.powerpc64/tmp >> ${SANITIZER_LIBDIR} =3D /usr/lib/clang/17/lib/freebsd >> and so the script is looking for >> = /usr/obj/usr/home/tuexen/freebsd-src/powerpc.powerpc64/tmp/usr/lib/clang/1= 7/lib/freebsd/libclang_rt.ubsan_standalone-powerpc64.a >> which does not exist: >> tuexen@blackbird:~ % ls -l = /usr/obj/usr/home/tuexen/freebsd-src/powerpc.powerpc64/tmp/usr/lib/clang/1= 7/lib/freebsd/ >> total 652 >> -r--r--r-- 1 root wheel 284316 Mar 20 18:03 = libclang_rt.profile-powerpc.a >> -r--r--r-- 1 root wheel 380704 Mar 20 17:41 = libclang_rt.profile-powerpc64.a >>=20 >> Therefore, h_raw to NOT built. >=20 > As far as I can see, for powerpc64 it should have been built somewhere = during the libraries stage. So it's a bit strange that you don't have = the file. Did you use any special options to build? No, not any I'm aware of. I can run tests or provide further = information. Best regards Michael >=20 > -Dimitry >=20 From nobody Thu Mar 21 00:17:37 2024 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4V0Qyk3njdz5FK8q for ; Thu, 21 Mar 2024 00:17:50 +0000 (UTC) (envelope-from kib@freebsd.org) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4V0Qyj4fgNz4vSD; Thu, 21 Mar 2024 00:17:49 +0000 (UTC) (envelope-from kib@freebsd.org) Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=freebsd.org (policy=none); spf=softfail (mx1.freebsd.org: 2001:470:d5e7:1::1 is neither permitted nor denied by domain of kib@freebsd.org) smtp.mailfrom=kib@freebsd.org Received: from tom.home (kib@localhost [127.0.0.1] (may be forged)) by kib.kiev.ua (8.18.1/8.18.1) with ESMTP id 42L0HcYZ082994; Thu, 21 Mar 2024 02:17:41 +0200 (EET) (envelope-from kib@freebsd.org) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 42L0HcYZ082994 Received: (from kostik@localhost) by tom.home (8.18.1/8.18.1/Submit) id 42L0Hb4s082987; Thu, 21 Mar 2024 02:17:37 +0200 (EET) (envelope-from kib@freebsd.org) X-Authentication-Warning: tom.home: kostik set sender to kib@freebsd.org using -f Date: Thu, 21 Mar 2024 02:17:37 +0200 From: Konstantin Belousov To: rrs Cc: Drew Gallatin , Mike Karels , tuexen , Nuno Teixeira , garyj@gmx.de, current@freebsd.org, net@freebsd.org, Randall Stewart Subject: Re: Request for Testing: TCP RACK Message-ID: References: <6e795e9c-8de4-4e02-9a96-8fabfaa4e66f@app.fastmail.com> <6047C8EF-B1B0-4286-93FA-AA38F8A18656@karels.net> <8031cd99-ded8-4b06-93b3-11cc729a8b2c@app.fastmail.com> <38c54399-6c96-44d8-a3a2-3cc1bfbe50c2@app.fastmail.com> <27d8144f-0658-46f6-b8f3-35eb60061644@lakerest.net> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <27d8144f-0658-46f6-b8f3-35eb60061644@lakerest.net> X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-14) on tom.home X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.97 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.97)[-0.965]; DMARC_POLICY_SOFTFAIL(0.10)[freebsd.org : No valid SPF, No valid DKIM,none]; MIME_GOOD(-0.10)[text/plain]; R_SPF_SOFTFAIL(0.00)[~all]; HAS_XAW(0.00)[]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[kib]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_SEVEN(0.00)[9]; MLMMJ_DEST(0.00)[current@freebsd.org]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_CC(0.00)[freebsd.org,karels.net,gmx.de]; R_DKIM_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MISSING_XM_UA(0.00)[] X-Rspamd-Queue-Id: 4V0Qyj4fgNz4vSD On Tue, Mar 19, 2024 at 06:19:52AM -0400, rrs wrote: > Ok I have created > > https://reviews.freebsd.org/D44420 > > > To address the issue. I also attach a short version of the patch that Nuno > can try and validate > > it works. Drew you may want to try this and validate the optimization does > kick in since I can > > only now test that it does not on my local box :) The patch still causes access to all cpu's cachelines on each userret. It would be much better to inc/check the threshold and only schedule the call when exceeded. Then the call can occur in some dedicated context, like per-CPU thread, instead of userret. > > > R > > > > On 3/18/24 3:42 PM, Drew Gallatin wrote: > > No.  The goal is to run on every return to userspace for every thread. > > > > Drew > > > > On Mon, Mar 18, 2024, at 3:41 PM, Konstantin Belousov wrote: > > > On Mon, Mar 18, 2024 at 03:13:11PM -0400, Drew Gallatin wrote: > > > > I got the idea from > > > > https://people.mpi-sws.org/~druschel/publications/soft-timers-tocs.pdf > > > > The gist is that the TCP pacing stuff needs to run frequently, and > > > > rather than run it out of a clock interrupt, its more efficient to run > > > > it out of a system call context at just the point where we return to > > > > userspace and the cache is trashed anyway. The current implementation > > > > is fine for our workload, but probably not idea for a generic system. > > > > Especially one where something is banging on system calls. > > > > > > > > Ast's could be the right tool for this, but I'm super unfamiliar with > > > > them, and I can't find any docs on them. > > > > > > > > Would ast_register(0, ASTR_UNCOND, 0, func) be roughly equivalent to > > > > what's happening here? > > > This call would need some AST number added, and then it registers the > > > ast to run on next return to userspace, for the current thread. > > > > > > Is it enough? > > > > > > > > Drew > > > > > > > > > > > On Mon, Mar 18, 2024, at 2:33 PM, Konstantin Belousov wrote: > > > > > On Mon, Mar 18, 2024 at 07:26:10AM -0500, Mike Karels wrote: > > > > > > On 18 Mar 2024, at 7:04, tuexen@freebsd.org wrote: > > > > > > > > > > > > >> On 18. Mar 2024, at 12:42, Nuno Teixeira > > > wrote: > > > > > > >> > > > > > > >> Hello all! > > > > > > >> > > > > > > >> It works just fine! > > > > > > >> System performance is OK. > > > > > > >> Using patch on main-n268841-b0aaf8beb126(-dirty). > > > > > > >> > > > > > > >> --- > > > > > > >> net.inet.tcp.functions_available: > > > > > > >> Stack                           D > > > Alias                            PCB count > > > > > > >> freebsd freebsd                          0 > > > > > > >> rack                            * > > > rack                             38 > > > > > > >> --- > > > > > > >> > > > > > > >> It would be so nice that we can have a sysctl tunnable for > > > this patch > > > > > > >> so we could do more tests without recompiling kernel. > > > > > > > Thanks for testing! > > > > > > > > > > > > > > @gallatin: can you come up with a patch that is acceptable > > > for Netflix > > > > > > > and allows to mitigate the performance regression. > > > > > > > > > > > > Ideally, tcphpts could enable this automatically when it > > > starts to be > > > > > > used (enough?), but a sysctl could select auto/on/off. > > > > > There is already a well-known mechanism to request execution of the > > > > > specific function on return to userspace, namely AST.  The difference > > > > > with the current hack is that the execution is requested for one > > > callback > > > > > in the context of the specific thread. > > > > > > > > > > Still, it might be worth a try to use it; what is the reason to > > > hit a thread > > > > > that does not do networking, with TCP processing? > > > > > > > > > > > > > > > > > Mike > > > > > > > > > > > > > Best regards > > > > > > > Michael > > > > > > >> > > > > > > >> Thanks all! > > > > > > >> Really happy here :) > > > > > > >> > > > > > > >> Cheers, > > > > > > >> > > > > > > >> Nuno Teixeira escreveu (domingo, > > > 17/03/2024 à(s) 20:26): > > > > > > >>> > > > > > > >>> Hello, > > > > > > >>> > > > > > > >>>> I don't have the full context, but it seems like the > > > complaint is a performance regression in bonnie++ and perhaps other > > > things when tcp_hpts is loaded, even when it is not used.  Is that > > > correct? > > > > > > >>>> > > > > > > >>>> If so, I suspect its because we drive the > > > tcp_hpts_softclock() routine from userret(), in order to avoid tons > > > of timer interrupts and context switches.  To test this theory,  you > > > could apply a patch like: > > > > > > >>> > > > > > > >>> It's affecting overall system performance, bonnie was just > > > a way to > > > > > > >>> get some numbers to compare. > > > > > > >>> > > > > > > >>> Tomorrow I will test patch. > > > > > > >>> > > > > > > >>> Thanks! > > > > > > >>> > > > > > > >>> -- > > > > > > >>> Nuno Teixeira > > > > > > >>> FreeBSD Committer (ports) > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> -- > > > > > > >> Nuno Teixeira > > > > > > >> FreeBSD Committer (ports) > > > > > > > > > > > > > > > > > diff --git a/sys/netinet/tcp_hpts.c b/sys/netinet/tcp_hpts.c > index 8c4d2d41a3eb..eadbee19f69c 100644 > --- a/sys/netinet/tcp_hpts.c > +++ b/sys/netinet/tcp_hpts.c > @@ -216,6 +216,7 @@ struct tcp_hpts_entry { > void *ie_cookie; > uint16_t p_num; /* The hpts number one per cpu */ > uint16_t p_cpu; /* The hpts CPU */ > + uint8_t hit_callout_thresh; > /* There is extra space in here */ > /* Cache line 0x100 */ > struct callout co __aligned(CACHE_LINE_SIZE); > @@ -269,6 +270,11 @@ static struct hpts_domain_info { > int cpu[MAXCPU]; > } hpts_domains[MAXMEMDOM]; > > +counter_u64_t hpts_that_need_softclock; > +SYSCTL_COUNTER_U64(_net_inet_tcp_hpts_stats, OID_AUTO, needsoftclock, CTLFLAG_RD, > + &hpts_that_need_softclock, > + "Number of hpts threads that need softclock"); > + > counter_u64_t hpts_hopelessly_behind; > > SYSCTL_COUNTER_U64(_net_inet_tcp_hpts_stats, OID_AUTO, hopeless, CTLFLAG_RD, > @@ -334,7 +340,7 @@ SYSCTL_INT(_net_inet_tcp_hpts, OID_AUTO, precision, CTLFLAG_RW, > &tcp_hpts_precision, 120, > "Value for PRE() precision of callout"); > SYSCTL_INT(_net_inet_tcp_hpts, OID_AUTO, cnt_thresh, CTLFLAG_RW, > - &conn_cnt_thresh, 0, > + &conn_cnt_thresh, DEFAULT_CONNECTION_THESHOLD, > "How many connections (below) make us use the callout based mechanism"); > SYSCTL_INT(_net_inet_tcp_hpts, OID_AUTO, logging, CTLFLAG_RW, > &hpts_does_tp_logging, 0, > @@ -1548,6 +1554,9 @@ __tcp_run_hpts(void) > struct tcp_hpts_entry *hpts; > int ticks_ran; > > + if (counter_u64_fetch(hpts_that_need_softclock) == 0) > + return; > + > hpts = tcp_choose_hpts_to_run(); > > if (hpts->p_hpts_active) { > @@ -1683,6 +1692,13 @@ tcp_hpts_thread(void *ctx) > ticks_ran = tcp_hptsi(hpts, 1); > tv.tv_sec = 0; > tv.tv_usec = hpts->p_hpts_sleep_time * HPTS_TICKS_PER_SLOT; > + if ((hpts->p_on_queue_cnt > conn_cnt_thresh) && (hpts->hit_callout_thresh == 0)) { > + hpts->hit_callout_thresh = 1; > + counter_u64_add(hpts_that_need_softclock, 1); > + } else if ((hpts->p_on_queue_cnt <= conn_cnt_thresh) && (hpts->hit_callout_thresh == 1)) { > + hpts->hit_callout_thresh = 0; > + counter_u64_add(hpts_that_need_softclock, -1); > + } > if (hpts->p_on_queue_cnt >= conn_cnt_thresh) { > if(hpts->p_direct_wake == 0) { > /* > @@ -1818,6 +1834,7 @@ tcp_hpts_mod_load(void) > cpu_top = NULL; > #endif > tcp_pace.rp_num_hptss = ncpus; > + hpts_that_need_softclock = counter_u64_alloc(M_WAITOK); > hpts_hopelessly_behind = counter_u64_alloc(M_WAITOK); > hpts_loops = counter_u64_alloc(M_WAITOK); > back_tosleep = counter_u64_alloc(M_WAITOK); > @@ -2042,6 +2059,7 @@ tcp_hpts_mod_unload(void) > free(tcp_pace.grps, M_TCPHPTS); > #endif > > + counter_u64_free(hpts_that_need_softclock); > counter_u64_free(hpts_hopelessly_behind); > counter_u64_free(hpts_loops); > counter_u64_free(back_tosleep); From nobody Thu Mar 21 12:57:44 2024 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4V0lqy484gz5DYjn; Thu, 21 Mar 2024 12:58:06 +0000 (UTC) (envelope-from gallatin@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4V0lqy14d6z4Pvg; Thu, 21 Mar 2024 12:58:06 +0000 (UTC) (envelope-from gallatin@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1711025886; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=VDngckJsj6YwsGVPKf4TVi02lPK/AhReZS1mJGoCgQY=; b=SLdD4qCxkcNyxCM641BomckfhnCmHmOUf+Qu2DB2ox5hdqhn006Jg5xDp+OmAN6cf4B1vg DHfBECa/vScysiMHAk67BMmJbY7ru8En7Xmqz5bYPd/mJ20S/uJJyJNYKfCBCaIacNY56D VQbLsyqPYxhHZW5DWIVHx8+HD90w2FN5Ai9Xg/qRrfLLDBSP5j1t7vpWeOvQJTbRssMZje 5GYSyXxZc9Uudl9CWFqFPvz59+oaSfQMj9wVYSW0UzwEj6oXItzxQZBFSv90oTe7fTrnwq 5LOqAP82hTK13Tk6ErDrE/Akwi1HfWlMnYpZgtRpqfx+elIO9HpWcPqoBwCAug== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1711025886; a=rsa-sha256; cv=none; b=jXlRKPpla4YXlUMnXqETuVj9PTgL8DlGh4CZX3T4eTYDYFpruyw30kRynwTW8/1l1qZ6hK EefV0LDY87QJUk0sPITzUNvKI4QgfZHVnHbvHaR8NEpCW0jJwgpt4Q2vCTE+vXBns0KBQs dboTuZ5z6H8hxl59JVk3xSkTnctgjM6OlfnJ1lG/+yspZWd1cTkSN/sc6cXHUPjvoWdOPo 5yJScXooAMXP8Qn8qlvXRvzIj5fa0Cyd3ML0EyQx/5ZrGhl8sUYPG0a3RFVZOVkJs/PF2V rfsA2gVO/kv51UXX19UZYVP5MFQoK3AQInyPG8vCuzFl89PMFU4tshfP6Uqgzw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1711025886; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=VDngckJsj6YwsGVPKf4TVi02lPK/AhReZS1mJGoCgQY=; b=m0bnOnL3qhWofFV4sboanFG6WwMDYk/7Oh3aJxcVj6qYWLKLwCFXZPs5kEbtbjdhv+TgLf dz1vJfRUjcu5cvzEyzfBr/5MdMLNGcV5hAlQmT2RNzgGmAYDyVE+2XkeBT59l7+pGpibce 1nEka2r0AqOY9KlZn49nvAB1oM6Qpjh/hwIL/+87PlxjEZ/IsvVW0jYMGlAOwEeJ/8Stsd vNm58qTR0ikCLfwQfCHlpxNNPM3e9dves0fM92Wfh3PIh25OmYB28PO0hUjNtMRBTctAcA vt5ujcjSpCw8YeIaqOUX4CUpJ4LMhzStafluAGS+Y/UGd8WhauZwHyojEvL9og== Received: from fauth2-smtp.messagingengine.com (fauth2-smtp.messagingengine.com [103.168.172.201]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: gallatin) by smtp.freebsd.org (Postfix) with ESMTPSA id 4V0lqx6pz5zjjD; Thu, 21 Mar 2024 12:58:05 +0000 (UTC) (envelope-from gallatin@freebsd.org) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailfauth.nyi.internal (Postfix) with ESMTP id 448621200066; Thu, 21 Mar 2024 08:58:05 -0400 (EDT) Received: from imap53 ([10.202.2.103]) by compute5.internal (MEProxy); Thu, 21 Mar 2024 08:58:05 -0400 X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrleeigdegiecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefofgggkfgjfhffhffvvefutgesrgdtreerreerjeenucfhrhhomhepfdffrhgv ficuifgrlhhlrghtihhnfdcuoehgrghllhgrthhinhesfhhrvggvsghsugdrohhrgheqne cuggftrfgrthhtvghrnhepudehheeiffefudeuteeluddtgeeijedtffehjeeufeeiteei vdegvdeiiefgtddvnecuffhomhgrihhnpehfrhgvvggsshgurdhorhhgpdhmphhiqdhsfi hsrdhorhhgnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhho mhepghgrlhhlrghtihhnodhmvghsmhhtphgruhhthhhpvghrshhonhgrlhhithihqddufe efheelvddvudeiqddvleehtdegudekgedqghgrlhhlrghtihhnpeepfhhrvggvsghsugdr ohhrghesfhgrshhtmhgrihhlrdgtohhm X-ME-Proxy: Feedback-ID: i41414658:Fastmail Received: by mailuser.nyi.internal (Postfix, from userid 501) id E0755364006F; Thu, 21 Mar 2024 08:58:04 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.11.0-alpha0-332-gdeb4194079-fm-20240319.002-gdeb41940 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Message-Id: In-Reply-To: References: <6e795e9c-8de4-4e02-9a96-8fabfaa4e66f@app.fastmail.com> <6047C8EF-B1B0-4286-93FA-AA38F8A18656@karels.net> <8031cd99-ded8-4b06-93b3-11cc729a8b2c@app.fastmail.com> <38c54399-6c96-44d8-a3a2-3cc1bfbe50c2@app.fastmail.com> <27d8144f-0658-46f6-b8f3-35eb60061644@lakerest.net> Date: Thu, 21 Mar 2024 08:57:44 -0400 From: "Drew Gallatin" To: "Konstantin Belousov" , rrs Cc: "Mike Karels" , tuexen , "Nuno Teixeira" , garyj@gmx.de, current@freebsd.org, net@freebsd.org, "Randall Stewart" Subject: Re: Request for Testing: TCP RACK Content-Type: multipart/alternative; boundary=3b39556bddae44fcbfa6d30004956a6c --3b39556bddae44fcbfa6d30004956a6c Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: quoted-printable The entire point is to *NOT* go through the overhead of scheduling somet= hing asynchronously, but to take advantage of the fact that a user/kerne= l transition is going to trash the cache anyway. In the common case of a system which has less than the threshold number= of connections , we access the tcp_hpts_softclock function pointer, mak= e one function call, and access hpts_that_need_softclock, and then retur= n. So that's 2 variables and a function call. I think it would be preferable to avoid that call, and to move the decla= ration of tcp_hpts_softclock and hpts_that_need_softclock so that they a= re in the same cacheline. Then we'd be hitting just a single line in th= e common case. (I've made comments on the review to that effect). Also, I wonder if the threshold could get higher by default, so that hpt= s is never called in this context unless we're to the point where we're = scheduling thousands of runs of the hpts thread (and taking all those cl= ock interrupts). Drew On Wed, Mar 20, 2024, at 8:17 PM, Konstantin Belousov wrote: > On Tue, Mar 19, 2024 at 06:19:52AM -0400, rrs wrote: > > Ok I have created > >=20 > > https://reviews.freebsd.org/D44420 > >=20 > >=20 > > To address the issue. I also attach a short version of the patch tha= t Nuno > > can try and validate > >=20 > > it works. Drew you may want to try this and validate the optimizatio= n does > > kick in since I can > >=20 > > only now test that it does not on my local box :) > The patch still causes access to all cpu's cachelines on each userret. > It would be much better to inc/check the threshold and only schedule t= he > call when exceeded. Then the call can occur in some dedicated context, > like per-CPU thread, instead of userret. >=20 > >=20 > >=20 > > R > >=20 > >=20 > >=20 > > On 3/18/24 3:42 PM, Drew Gallatin wrote: > > > No. The goal is to run on every return to userspace for every thr= ead. > > >=20 > > > Drew > > >=20 > > > On Mon, Mar 18, 2024, at 3:41 PM, Konstantin Belousov wrote: > > > > On Mon, Mar 18, 2024 at 03:13:11PM -0400, Drew Gallatin wrote: > > > > > I got the idea from > > > > > https://people.mpi-sws.org/~druschel/publications/soft-timers-= tocs.pdf > > > > > The gist is that the TCP pacing stuff needs to run frequently,= and > > > > > rather than run it out of a clock interrupt, its more efficien= t to run > > > > > it out of a system call context at just the point where we ret= urn to > > > > > userspace and the cache is trashed anyway. The current impleme= ntation > > > > > is fine for our workload, but probably not idea for a generic = system. > > > > > Especially one where something is banging on system calls. > > > > > > > > > > Ast's could be the right tool for this, but I'm super unfamili= ar with > > > > > them, and I can't find any docs on them. > > > > > > > > > > Would ast_register(0, ASTR_UNCOND, 0, func) be roughly equival= ent to > > > > > what's happening here? > > > > This call would need some AST number added, and then it register= s the > > > > ast to run on next return to userspace, for the current thread. > > > >=20 > > > > Is it enough? > > > > > > > > > > Drew > > > >=20 > > > > > > > > > > On Mon, Mar 18, 2024, at 2:33 PM, Konstantin Belousov wrote: > > > > > > On Mon, Mar 18, 2024 at 07:26:10AM -0500, Mike Karels wrote: > > > > > > > On 18 Mar 2024, at 7:04, tuexen@freebsd.org wrote: > > > > > > > > > > > > > > >> On 18. Mar 2024, at 12:42, Nuno Teixeira > > > > wrote: > > > > > > > >> > > > > > > > >> Hello all! > > > > > > > >> > > > > > > > >> It works just fine! > > > > > > > >> System performance is OK. > > > > > > > >> Using patch on main-n268841-b0aaf8beb126(-dirty). > > > > > > > >> > > > > > > > >> --- > > > > > > > >> net.inet.tcp.functions_available: > > > > > > > >> Stack D > > > > Alias PCB count > > > > > > > >> freebsd freebsd 0 > > > > > > > >> rack * > > > > rack 38 > > > > > > > >> --- > > > > > > > >> > > > > > > > >> It would be so nice that we can have a sysctl tunnable = for > > > > this patch > > > > > > > >> so we could do more tests without recompiling kernel. > > > > > > > > Thanks for testing! > > > > > > > > > > > > > > > > @gallatin: can you come up with a patch that is acceptab= le > > > > for Netflix > > > > > > > > and allows to mitigate the performance regression. > > > > > > > > > > > > > > Ideally, tcphpts could enable this automatically when it > > > > starts to be > > > > > > > used (enough?), but a sysctl could select auto/on/off. > > > > > > There is already a well-known mechanism to request execution= of the > > > > > > specific function on return to userspace, namely AST. The d= ifference > > > > > > with the current hack is that the execution is requested for= one > > > > callback > > > > > > in the context of the specific thread. > > > > > > > > > > > > Still, it might be worth a try to use it; what is the reason= to > > > > hit a thread > > > > > > that does not do networking, with TCP processing? > > > > > > > > > > > > > > > > > > > > Mike > > > > > > > > > > > > > > > Best regards > > > > > > > > Michael > > > > > > > >> > > > > > > > >> Thanks all! > > > > > > > >> Really happy here :) > > > > > > > >> > > > > > > > >> Cheers, > > > > > > > >> > > > > > > > >> Nuno Teixeira escreveu (domingo, > > > > 17/03/2024 =C3=A0(s) 20:26): > > > > > > > >>> > > > > > > > >>> Hello, > > > > > > > >>> > > > > > > > >>>> I don't have the full context, but it seems like the > > > > complaint is a performance regression in bonnie++ and perhaps ot= her > > > > things when tcp_hpts is loaded, even when it is not used. Is th= at > > > > correct? > > > > > > > >>>> > > > > > > > >>>> If so, I suspect its because we drive the > > > > tcp_hpts_softclock() routine from userret(), in order to avoid t= ons > > > > of timer interrupts and context switches. To test this theory, = you > > > > could apply a patch like: > > > > > > > >>> > > > > > > > >>> It's affecting overall system performance, bonnie was = just > > > > a way to > > > > > > > >>> get some numbers to compare. > > > > > > > >>> > > > > > > > >>> Tomorrow I will test patch. > > > > > > > >>> > > > > > > > >>> Thanks! > > > > > > > >>> > > > > > > > >>> -- > > > > > > > >>> Nuno Teixeira > > > > > > > >>> FreeBSD Committer (ports) > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> -- > > > > > > > >> Nuno Teixeira > > > > > > > >> FreeBSD Committer (ports) > > > > > > > > > > > > > > > > >=20 > > >=20 >=20 > > diff --git a/sys/netinet/tcp_hpts.c b/sys/netinet/tcp_hpts.c > > index 8c4d2d41a3eb..eadbee19f69c 100644 > > --- a/sys/netinet/tcp_hpts.c > > +++ b/sys/netinet/tcp_hpts.c > > @@ -216,6 +216,7 @@ struct tcp_hpts_entry { > > void *ie_cookie; > > uint16_t p_num; /* The hpts number one per cpu */ > > uint16_t p_cpu; /* The hpts CPU */ > > + uint8_t hit_callout_thresh; > > /* There is extra space in here */ > > /* Cache line 0x100 */ > > struct callout co __aligned(CACHE_LINE_SIZE); > > @@ -269,6 +270,11 @@ static struct hpts_domain_info { > > int cpu[MAXCPU]; > > } hpts_domains[MAXMEMDOM]; > > =20 > > +counter_u64_t hpts_that_need_softclock; > > +SYSCTL_COUNTER_U64(_net_inet_tcp_hpts_stats, OID_AUTO, needsoftcloc= k, CTLFLAG_RD, > > + &hpts_that_need_softclock, > > + "Number of hpts threads that need softclock"); > > + > > counter_u64_t hpts_hopelessly_behind; > > =20 > > SYSCTL_COUNTER_U64(_net_inet_tcp_hpts_stats, OID_AUTO, hopeless, CT= LFLAG_RD, > > @@ -334,7 +340,7 @@ SYSCTL_INT(_net_inet_tcp_hpts, OID_AUTO, precisi= on, CTLFLAG_RW, > > &tcp_hpts_precision, 120, > > "Value for PRE() precision of callout"); > > SYSCTL_INT(_net_inet_tcp_hpts, OID_AUTO, cnt_thresh, CTLFLAG_RW, > > - &conn_cnt_thresh, 0, > > + &conn_cnt_thresh, DEFAULT_CONNECTION_THESHOLD, > > "How many connections (below) make us use the callout based mec= hanism"); > > SYSCTL_INT(_net_inet_tcp_hpts, OID_AUTO, logging, CTLFLAG_RW, > > &hpts_does_tp_logging, 0, > > @@ -1548,6 +1554,9 @@ __tcp_run_hpts(void) > > struct tcp_hpts_entry *hpts; > > int ticks_ran; > > =20 > > + if (counter_u64_fetch(hpts_that_need_softclock) =3D=3D 0) > > + return; > > + > > hpts =3D tcp_choose_hpts_to_run(); > > =20 > > if (hpts->p_hpts_active) { > > @@ -1683,6 +1692,13 @@ tcp_hpts_thread(void *ctx) > > ticks_ran =3D tcp_hptsi(hpts, 1); > > tv.tv_sec =3D 0; > > tv.tv_usec =3D hpts->p_hpts_sleep_time * HPTS_TICKS_PER_SLOT; > > + if ((hpts->p_on_queue_cnt > conn_cnt_thresh) && (hpts->hit_callout= _thresh =3D=3D 0)) { > > + hpts->hit_callout_thresh =3D 1; > > + counter_u64_add(hpts_that_need_softclock, 1); > > + } else if ((hpts->p_on_queue_cnt <=3D conn_cnt_thresh) && (hpts->h= it_callout_thresh =3D=3D 1)) { > > + hpts->hit_callout_thresh =3D 0; > > + counter_u64_add(hpts_that_need_softclock, -1); > > + } > > if (hpts->p_on_queue_cnt >=3D conn_cnt_thresh) { > > if(hpts->p_direct_wake =3D=3D 0) { > > /* > > @@ -1818,6 +1834,7 @@ tcp_hpts_mod_load(void) > > cpu_top =3D NULL; > > #endif > > tcp_pace.rp_num_hptss =3D ncpus; > > + hpts_that_need_softclock =3D counter_u64_alloc(M_WAITOK); > > hpts_hopelessly_behind =3D counter_u64_alloc(M_WAITOK); > > hpts_loops =3D counter_u64_alloc(M_WAITOK); > > back_tosleep =3D counter_u64_alloc(M_WAITOK); > > @@ -2042,6 +2059,7 @@ tcp_hpts_mod_unload(void) > > free(tcp_pace.grps, M_TCPHPTS); > > #endif > > =20 > > + counter_u64_free(hpts_that_need_softclock); > > counter_u64_free(hpts_hopelessly_behind); > > counter_u64_free(hpts_loops); > > counter_u64_free(back_tosleep); >=20 >=20 --3b39556bddae44fcbfa6d30004956a6c Content-Type: text/html;charset=utf-8 Content-Transfer-Encoding: quoted-printable
The entire poin= t is to *NOT* go through the overhead of scheduling something asynchrono= usly, but to take advantage of the fact that a user/kernel transition is= going to trash the cache anyway.

In the co= mmon case of a system which has less than the threshold  number of = connections , we access the tcp_hpts_softclock function pointer, make on= e function call, and access hpts_that_need_softclock, and then return.&n= bsp; So that's 2 variables and a function call.

=
I think it would be preferable to avoid that call, and to move the = declaration of tcp_hpts_softclock and hpts_that_need_softclock so that t= hey are in the same cacheline.  Then we'd be hitting just a single = line in the common case.  (I've made comments on the review to that= effect).

Also, I wonder if the threshold c= ould get higher by default, so that hpts is never called in this context= unless we're to the point where we're scheduling thousands of runs of t= he hpts thread (and taking all those clock interrupts).
Drew

On Wed, Mar 20, 2024, at = 8:17 PM, Konstantin Belousov wrote:
On Tue, Mar 19, 2024 at 06:19:52AM -0400, rrs w= rote:
> Ok I have created
> To address the issue. I also attach a sho= rt version of the patch that Nuno
> can try and validat= e
> 
> it works. Drew you may wan= t to try this and validate the optimization does
> kick= in since I can
> 
> only now tes= t that it does not on my local box :)
The patch still caus= es access to all cpu's cachelines on each userret.
It woul= d be much better to inc/check the threshold and only schedule the
call when exceeded.  Then the call can occur in some dedica= ted context,
like per-CPU thread, instead of userret.
<= /div>

> 
> 
> R
> 
> 
> 
> On 3/18/24 3:42 PM, Drew Gallatin wrote:=
> > No.  The goal is to run on every return to= userspace for every thread.
> > 
> > Drew
> > 
> > On= Mon, Mar 18, 2024, at 3:41 PM, Konstantin Belousov wrote:
> > > On Mon, Mar 18, 2024 at 03:13:11PM -0400, Drew Gallatin = wrote:
> > > > I got the idea from
> > > > https://people.mpi-sws.org/~drusc= hel/publications/soft-timers-tocs.pdf
> > > &= gt; The gist is that the TCP pacing stuff needs to run frequently, and
> > > > rather than run it out of a clock inter= rupt, its more efficient to run
> > > > it out= of a system call context at just the point where we return to
=
> > > > userspace and the cache is trashed anyway. The = current implementation
> > > > is fine for our= workload, but probably not idea for a generic system.
>= ; > > > Especially one where something is banging on system cal= ls.
> > > >
> > > > = Ast's could be the right tool for this, but I'm super unfamiliar with
> > > > them, and I can't find any docs on them.=
> > > >
> > > > Wou= ld ast_register(0, ASTR_UNCOND, 0, func) be roughly equivalent to
> > > > what's happening here?
> &g= t; > This call would need some AST number added, and then it register= s the
> > > ast to run on next return to userspac= e, for the current thread.
> > > 
<= div>> > > Is it enough?
> > > >
> > > > Drew
> > > 
=
> > > >
> > > > On Mon,= Mar 18, 2024, at 2:33 PM, Konstantin Belousov wrote:
>= > > > > On Mon, Mar 18, 2024 at 07:26:10AM -0500, Mike Kare= ls wrote:
> > > > > > On 18 Mar 2024, at= 7:04, tuexen@freebsd.org= wrote:
> > > > > >
> &= gt; > > > > >> On 18. Mar 2024, at 12:42, Nuno Teixeir= a
> > > <eduardo@freebsd.org> wrote:
> > > > &= gt; > >>
> > > > > > >> H= ello all!
> > > > > > >>
=
> > > > > > >> It works just fine!
> > > > > > >> System performance is OK.
> > > > > > >> Using patch on main-= n268841-b0aaf8beb126(-dirty).
> > > > > >= ; >>
> > > > > > >> ---
<= /div>
> > > > > > >> net.inet.tcp.functions_= available:
> > > > > > >> Stack&nb= sp;           &nb= sp;           &nb= sp;  D
> > > Alias    &n= bsp;           &n= bsp;           PCB cou= nt
> > > > > > >> freebsd freebsd&= nbsp;           &= nbsp;           &= nbsp; 0
> > > > > > >> rack &= nbsp;           &= nbsp;           &= nbsp;  *
> > > rack    &= nbsp;           &= nbsp;            = 38
> > > > > > >> ---
> > > > > > >>
> > > &g= t; > > >> It would be so nice that we can have a sysctl tunn= able for
> > > this patch
> >= > > > > >> so we could do more tests without recompil= ing kernel.
> > > > > > > Thanks for = testing!
> > > > > > >
= > > > > > > > @gallatin: can you come up with a pat= ch that is acceptable
> > > for Netflix
=
> > > > > > > and allows to mitigate the perfo= rmance regression.
> > > > > >
=
> > > > > > Ideally, tcphpts could enable this au= tomatically when it
> > > starts to be
<= div>> > > > > > used (enough?), but a sysctl could sel= ect auto/on/off.
> > > > > There is already= a well-known mechanism to request execution of the
> &= gt; > > > specific function on return to userspace, namely AST.=   The difference
> > > > > with the cu= rrent hack is that the execution is requested for one
>= > > callback
> > > > > in the contex= t of the specific thread.
> > > > >
> > > > > Still, it might be worth a try to use it= ; what is the reason to
> > > hit a thread
> > > > > that does not do networking, with TCP p= rocessing?
> > > > >
> >= ; > > > >
> > > > > > Mike
> > > > > >
> > > = > > > > Best regards
> > > > > = > > Michael
> > > > > > >>
> > > > > > >> Thanks all!
> > > > > > >> Really happy here :)
> > > > > > >>
> > &= gt; > > > >> Cheers,
> > > > &g= t; > >>
> > > > > > >> Nu= no Teixeira <eduardo@freebsd.o= rg> escreveu (domingo,
> > > 17/03/2024 =C3= =A0(s) 20:26):
> > > > > > >>><= br>
> > > > > > >>> Hello,
> > > > > > >>>
> >= > > > > >>>> I don't have the full context, but= it seems like the
> > > complaint is a performan= ce regression in bonnie++ and perhaps other
> > >= things when tcp_hpts is loaded, even when it is not used.  Is that=
> > > correct?
> > > >= > > >>>>
> > > > > > = >>>> If so, I suspect its because we drive the
> > > tcp_hpts_softclock() routine from userret(), in order to= avoid tons
> > > of timer interrupts and context= switches.  To test this theory,  you
> > = > could apply a patch like:
> > > > > &g= t; >>>
> > > > > > >>>= It's affecting overall system performance, bonnie was just
> > > a way to
> > > > > > &g= t;>> get some numbers to compare.
> > > >= ; > > >>>
> > > > > > >= ;>> Tomorrow I will test patch.
> > > > = > > >>>
> > > > > > >&= gt;> Thanks!
> > > > > > >>>=
> > > > > > >>> --
> > > > > > >>> Nuno Teixeira
> > > > > > >>> FreeBSD Committer (ports)
> > > > > > >>
> &= gt; > > > > >>
> > > > > = > >>
> > > > > > >> --
> > > > > > >> Nuno Teixeira
> > > > > > >> FreeBSD Committer (ports)<= br>
> > > > > >
> > >= > >
> > > 
> >&nb= sp;

> diff --git a/sys/netinet/tcp_hpts.= c b/sys/netinet/tcp_hpts.c
> index 8c4d2d41a3eb..eadbee= 19f69c 100644
> --- a/sys/netinet/tcp_hpts.c
<= div>> +++ b/sys/netinet/tcp_hpts.c
> @@ -216,6 +216,= 7 @@ struct tcp_hpts_entry {
>  void *ie_cookie;<= br>
>  uint16_t p_num; /* The hpts number one per cp= u */
>  uint16_t p_cpu; /* The hpts CPU */
> + uint8_t hit_callout_thresh;
>  /*= There is extra space in here */
>  /* Cache line= 0x100 */
>  struct callout co __aligned(CACHE_LI= NE_SIZE);
> @@ -269,6 +270,11 @@ static struct hpts_dom= ain_info {
>  int cpu[MAXCPU];
>=   } hpts_domains[MAXMEMDOM];
>  
> +counter_u64_t hpts_that_need_softclock;
> = +SYSCTL_COUNTER_U64(_net_inet_tcp_hpts_stats, OID_AUTO, needsoftclock, C= TLFLAG_RD,
> +    &hpts_that_need_so= ftclock,
> +    "Number of hpts threads = that need softclock");
> +
>  cou= nter_u64_t hpts_hopelessly_behind;
>  
>  SYSCTL_COUNTER_U64(_net_inet_tcp_hpts_stats, OID_AUTO= , hopeless, CTLFLAG_RD,
> @@ -334,7 +340,7 @@ SYSCTL_IN= T(_net_inet_tcp_hpts, OID_AUTO, precision, CTLFLAG_RW,
>= ;      &tcp_hpts_precision, 120,
<= div>>      "Value for PRE() precision of cal= lout");
>  SYSCTL_INT(_net_inet_tcp_hpts, OID_AUTO= , cnt_thresh, CTLFLAG_RW,
> -    &co= nn_cnt_thresh, 0,
> +    &conn_cnt_t= hresh, DEFAULT_CONNECTION_THESHOLD,
>   =    "How many connections (below) make us use the callout based= mechanism");
>  SYSCTL_INT(_net_inet_tcp_hpts, OI= D_AUTO, logging, CTLFLAG_RW,
>    &= nbsp; &hpts_does_tp_logging, 0,
> @@ -1548,6 +1554,= 9 @@ __tcp_run_hpts(void)
>  struct tcp_hpts_entr= y *hpts;
>  int ticks_ran;
>&nbs= p; 
> + if (counter_u64_fetch(hpts_that_need_softc= lock) =3D=3D 0)
> + return;
> +
>  hpts =3D tcp_choose_hpts_to_run();
&g= t;  
>  if (hpts->p_hpts_active) {
> @@ -1683,6 +1692,13 @@ tcp_hpts_thread(void *ctx)
<= /div>
>  ticks_ran =3D tcp_hptsi(hpts, 1);
&g= t;  tv.tv_sec =3D 0;
>  tv.tv_usec =3D hpts= ->p_hpts_sleep_time * HPTS_TICKS_PER_SLOT;
> + if ((= hpts->p_on_queue_cnt > conn_cnt_thresh) && (hpts->hit_c= allout_thresh =3D=3D 0)) {
> + hpts->hit_callout_th= resh =3D 1;
> + counter_u64_add(hpts_that_need_softclo= ck, 1);
> + } else if ((hpts->p_on_queue_cnt <=3D= conn_cnt_thresh) && (hpts->hit_callout_thresh =3D=3D 1)) {
> + hpts->hit_callout_thresh =3D 0;
&g= t; + counter_u64_add(hpts_that_need_softclock, -1);
> = + }
>  if (hpts->p_on_queue_cnt >=3D conn_c= nt_thresh) {
>  if(hpts->p_direct_wake =3D=3D= 0) {
>  /*
> @@ -1818,6 +1834= ,7 @@ tcp_hpts_mod_load(void)
>  cpu_top =3D NULL= ;
>  #endif
>  tcp_pace.rp_= num_hptss =3D ncpus;
> + hpts_that_need_softclock =3D c= ounter_u64_alloc(M_WAITOK);
>  hpts_hopelessly_be= hind =3D counter_u64_alloc(M_WAITOK);
>  hpts_loo= ps =3D counter_u64_alloc(M_WAITOK);
>  back_tosle= ep =3D counter_u64_alloc(M_WAITOK);
> @@ -2042,6 +2059,= 7 @@ tcp_hpts_mod_unload(void)
>  free(tcp_pace.g= rps, M_TCPHPTS);
>  #endif
> = ; 
> + counter_u64_free(hpts_that_need_softclock);=
>  counter_u64_free(hpts_hopelessly_behind);
=
>  counter_u64_free(hpts_loops);
>&= nbsp; counter_u64_free(back_tosleep);


=

--3b39556bddae44fcbfa6d30004956a6c-- From nobody Thu Mar 21 17:12:21 2024 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4V0sTN6Z3Rz5F5QJ for ; Thu, 21 Mar 2024 17:12:24 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4V0sTM6w2tz40WP; Thu, 21 Mar 2024 17:12:23 +0000 (UTC) (envelope-from dim@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1711041144; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Zw4iV44QY8RjhkCgoMeti3ht5uIFNqJt7bVTU4/g7G8=; b=C8oi04yjpprCrz3Hr0FCNZU/XuuIAGOB8tCAJjZpbutfi+wo8HeNe5rE7KeePyCTTl3uV1 GuAaFUdjAD1orMG8cMzvHb7mYFefPfU8deTCy2w+ioB6P8qx0wkjkJ8UpTN3EO68SawEL5 LEiAFg3cUtJoAW5hZd89RWBc5aZsD999pVyTYy9wRPY5q6mDsr7wuAmj2xbkDPyVz1L05O gFwUb9KztCgkJ2Il+ZQyMITDD4kUEUWB4L8dHDlviMCV1toXOKlD3kjlrOBTCtYHEB3L9A QQeH/9lJS1fYkFc9O+pZmFE++FvrMe7wjyYiR6VyjQuRfEhgOHoPGG94h2OmBw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1711041144; a=rsa-sha256; cv=none; b=Ib8v1KNqHGGf8IW8lSvJzkMVNGDHczTJFRWJsJ0vwJDFM3aRcQlQy3q3t4Qvr2YtOzfZds EiVwBirmvMnIJovyTqeEC7tQjV7c2XsmAOiWUqlgqho7Q1zx5XDGZEMAD2DKBqvYimMvSM +1yB1Nm6AmuDGJQ4RFWyaDKYcqOP0sPoZy1c9Sz5PfDpcHwLpqq0WV2hMiJJjza70ltBpA w0wtOKz931cq2QaNCK8e0qxxdcYiHO/ZG/PfwQ3ryBGUKlJBrZ7aDxWr5iUUG3AkH0y+4A L5DWcwQGlPkuzQfPEsgJDXJJCz1RScG71iSdDcSL6WYnr7Q6+Awli+tRSHlwsw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1711041144; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Zw4iV44QY8RjhkCgoMeti3ht5uIFNqJt7bVTU4/g7G8=; b=yznXI4J0dcFVtvqr1o9HImDfFDeBi2S/so9BrHYDHj9nx5pdXwRU4zukPv6tzVjC8x6u+f LZIIxY31d49wryGC3NhNPij6DBKZwiVNd3Q7XkCe3HjoqOleLWZDm9Bqx5ZvvSVmXwi+sJ lS9t+uCSU8F5v5mZ1Be8YODJmeOgvp+jvMCI/81mKvriLnyPJQxDtCBjEME7uiO2qC6mjb ZUIX6Q1o66B8SXEKPGYwEF5fRX6rgHs78Ts1T+/H8O52uQlKgoe7SalzMCFJDVft7o8ytR 4IL6T3ScY94+48vxkmuL7JrgrO5Jye9Mnv1BK3eFX0u6/I+4QuQDXoMgNONZ7w== Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "tensor.andric.com", Issuer "R3" (verified OK)) (Authenticated sender: dim) by smtp.freebsd.org (Postfix) with ESMTPSA id 4V0sTL5brrz15Bb; Thu, 21 Mar 2024 17:12:22 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtpclient.apple (longrow.home.andric.com [192.168.0.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 93CCD2A206; Thu, 21 Mar 2024 18:12:21 +0100 (CET) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6.1.1\)) Subject: Re: Problem with make installworld From: Dimitry Andric In-Reply-To: Date: Thu, 21 Mar 2024 18:12:21 +0100 Cc: current@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <3DCD0639-057D-4BC9-96B7-FDD6F05E6BB5@freebsd.org> To: tuexen@freebsd.org X-Mailer: Apple Mail (2.3731.700.6.1.1) On 21 Mar 2024, at 01:12, tuexen@freebsd.org wrote: >=20 >> On 21. Mar 2024, at 00:27, Dimitry Andric wrote: >>=20 >> On 20 Mar 2024, at 21:44, tuexen@freebsd.org wrote: >>>=20 >>> I'm trying to run make buildworld / make installworld on a recent = main branch >>> (some days old). >>>=20 >>> The problem is related to lib/libc/tests/ssp/Makefile >>> which contains: >>> _libclang_rt_ubsan=3D = ${SYSROOT}${SANITIZER_LIBDIR}/libclang_rt.ubsan_standalone-${CRTARCH}.a >>> if exists(${_libclang_rt_ubsan}) >>> PROGS+=3D h_raw >>> LDADD.h_raw+=3D ${SANITIZER_LDFLAGS} >>>=20 >>> When running make buildworld, we have >>> ${SYSROOT} =3D = /usr/obj/usr/home/tuexen/freebsd-src/powerpc.powerpc64/tmp >>> ${SANITIZER_LIBDIR} =3D /usr/lib/clang/17/lib/freebsd >>> and so the script is looking for >>> = /usr/obj/usr/home/tuexen/freebsd-src/powerpc.powerpc64/tmp/usr/lib/clang/1= 7/lib/freebsd/libclang_rt.ubsan_standalone-powerpc64.a >>> which does not exist: >>> tuexen@blackbird:~ % ls -l = /usr/obj/usr/home/tuexen/freebsd-src/powerpc.powerpc64/tmp/usr/lib/clang/1= 7/lib/freebsd/ >>> total 652 >>> -r--r--r-- 1 root wheel 284316 Mar 20 18:03 = libclang_rt.profile-powerpc.a >>> -r--r--r-- 1 root wheel 380704 Mar 20 17:41 = libclang_rt.profile-powerpc64.a >>>=20 >>> Therefore, h_raw to NOT built. >>=20 >> As far as I can see, for powerpc64 it should have been built = somewhere during the libraries stage. So it's a bit strange that you = don't have the file. Did you use any special options to build? > No, not any I'm aware of. I can run tests or provide further = information. This was my mistake: I recently refactored lib/libclang_rt/Makefile to = make it more readable and maintainable, but it accidentally broke = building of most of the libclang_rt*.a files for powerpc64. It should = now be fixed by https://cgit.freebsd.org/src/commit/?id=3Df0620ceeccf0 . -Dimitry From nobody Thu Mar 21 20:13:36 2024 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4V0xVX2wBlz5FPxQ for ; Thu, 21 Mar 2024 20:13:40 +0000 (UTC) (envelope-from tuexen@freebsd.org) Received: from drew.franken.de (mail-n.franken.de [193.175.24.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.franken.de", Issuer "Sectigo RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4V0xVX0clSz4PkL; Thu, 21 Mar 2024 20:13:40 +0000 (UTC) (envelope-from tuexen@freebsd.org) Authentication-Results: mx1.freebsd.org; none Received: from smtpclient.apple (unknown [IPv6:2a02:8109:1140:c3d:499b:2fb4:e4d2:c5fd]) (Authenticated sender: micmac) by drew.franken.de (Postfix) with ESMTPSA id 94F62721E2806; Thu, 21 Mar 2024 21:13:36 +0100 (CET) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.500.171.1.1\)) Subject: Re: Problem with make installworld From: tuexen@freebsd.org In-Reply-To: Date: Thu, 21 Mar 2024 21:13:36 +0100 Cc: current@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <5C76C958-3BBB-4887-96C8-84F663AC2D78@freebsd.org> References: <3DCD0639-057D-4BC9-96B7-FDD6F05E6BB5@freebsd.org> To: Dimitry Andric X-Mailer: Apple Mail (2.3774.500.171.1.1) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=disabled version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail-n.franken.de X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:680, ipnet:193.174.0.0/15, country:DE] X-Rspamd-Queue-Id: 4V0xVX0clSz4PkL > On 21. Mar 2024, at 18:12, Dimitry Andric wrote: >=20 > On 21 Mar 2024, at 01:12, tuexen@freebsd.org wrote: >>=20 >>> On 21. Mar 2024, at 00:27, Dimitry Andric wrote: >>>=20 >>> On 20 Mar 2024, at 21:44, tuexen@freebsd.org wrote: >>>>=20 >>>> I'm trying to run make buildworld / make installworld on a recent = main branch >>>> (some days old). >>>>=20 >>>> The problem is related to lib/libc/tests/ssp/Makefile >>>> which contains: >>>> _libclang_rt_ubsan=3D = ${SYSROOT}${SANITIZER_LIBDIR}/libclang_rt.ubsan_standalone-${CRTARCH}.a >>>> if exists(${_libclang_rt_ubsan}) >>>> PROGS+=3D h_raw >>>> LDADD.h_raw+=3D ${SANITIZER_LDFLAGS} >>>>=20 >>>> When running make buildworld, we have >>>> ${SYSROOT} =3D = /usr/obj/usr/home/tuexen/freebsd-src/powerpc.powerpc64/tmp >>>> ${SANITIZER_LIBDIR} =3D /usr/lib/clang/17/lib/freebsd >>>> and so the script is looking for >>>> = /usr/obj/usr/home/tuexen/freebsd-src/powerpc.powerpc64/tmp/usr/lib/clang/1= 7/lib/freebsd/libclang_rt.ubsan_standalone-powerpc64.a >>>> which does not exist: >>>> tuexen@blackbird:~ % ls -l = /usr/obj/usr/home/tuexen/freebsd-src/powerpc.powerpc64/tmp/usr/lib/clang/1= 7/lib/freebsd/ >>>> total 652 >>>> -r--r--r-- 1 root wheel 284316 Mar 20 18:03 = libclang_rt.profile-powerpc.a >>>> -r--r--r-- 1 root wheel 380704 Mar 20 17:41 = libclang_rt.profile-powerpc64.a >>>>=20 >>>> Therefore, h_raw to NOT built. >>>=20 >>> As far as I can see, for powerpc64 it should have been built = somewhere during the libraries stage. So it's a bit strange that you = don't have the file. Did you use any special options to build? >> No, not any I'm aware of. I can run tests or provide further = information. >=20 > This was my mistake: I recently refactored lib/libclang_rt/Makefile to = make it more readable and maintainable, but it accidentally broke = building of most of the libclang_rt*.a files for powerpc64. It should = now be fixed by https://cgit.freebsd.org/src/commit/?id=3Df0620ceeccf0 . Hi Dimitry, I tested the main branch with your fix and I can confirm that the problem is fixed. Thank you very much for the quick fix! Best regards Michael >=20 > -Dimitry >=20 From nobody Fri Mar 22 00:34:30 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4V13Hn6PC3z5FBB3 for ; Fri, 22 Mar 2024 00:34:45 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic315-55.consmr.mail.gq1.yahoo.com (sonic315-55.consmr.mail.gq1.yahoo.com [98.137.65.31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4V13Hl5HQDz4vlj for ; Fri, 22 Mar 2024 00:34:43 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=mC6T0g0u; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.65.31 as permitted sender) smtp.mailfrom=marklmi@yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1711067681; bh=iLYWG2ZvoimYjH0Olva8QsVECL6yMZrW4x8SP2BqNck=; h=From:Subject:Date:Cc:To:References:From:Subject:Reply-To; b=mC6T0g0u2u8KVRYqT0pBQrxomgjVzIprD4jAeg+JDiEOJDUG2pwBz/FwVTphfar4MEZd9/gENiKlJ3ZMZhQjy5+cxOAkN2i7yn9iWnfrrxOcN2PEl6WtOOzb6nvFVOPkAZtxM4J0zYdTCGAfrL3IroLgoSBopw3AfDU5hgC0gBqsddk9aczbnyWdnerUU8m26+09cMTczjh/yFSsxw8BpMs7RjChk98tgxExweLdbwt00XhVI2YiV7VTQXotRBlQis8jQ4EtqfkTzl4CM9F3ra37DzGKarqlEqtKt0ISTU4LUBKanqDXNUv55GVxVvBYh0/kkJTu6E1ZlemO1ByPoQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1711067681; bh=WEM5P7p/1MD/c70lna/+5UUQDkHnj9SeANUJRoTYQcs=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=mC3y/J7x7udQ3hFQ2BJyLiBlsicwyh1Obnfu5IwHd9zTzzxCXp33M1Um4xpT619UV7b+soilcAqibMcPedQTAKgQAW2QYXmOx0cjFaHRzGB5t0h3trM8ULf3jsrnOPC/R8tI6ozV7sbvRqmrYYupWh1x388GBHmGQvGYjeHNVkK7ioNUC70lANa37bl7AHQZBYII6HXpq+wRuLEimx+rrBWLboRmE7aoAtnbsHRjTX7m1X8zdYPheiSOt/i+Q+GA869Wl66/4LUMQgvlvJFaL70HraobrJQ6SHYgT5RRBROZTS5G9z8jOsLe4eYCTtRVmMMcPUXSnCkd0QMbiMb5Qw== X-YMail-OSG: LBPUFCkVM1kO8lzBGY_W5SliZihxXT2aIe3DvXEsqESAa69VVMxLVEo3LiEz_Kw Ydn_GtUveYUicm6nzsmfh7XHn.8UZQzPZe1vVldBkZkWXCXNBgv1cbF5PMnOW97MiN_m3_zF1mgj hIoGnQUlBxSJQR0KOW3dtR2wNk4hhq3UOITYC8jKxBGbdBM1RU1rYwsRBr4bRoMSyIV6KgccPZbk 6O0T8Rn4HpoFUjtD1VTqQSARPbSGXGUwzl.ectgOpqtm.vR3cdKCtoI4L3uCFlStr5xO44kPpKx1 tGTo9gERUNOM5QFZ3.xWifrFEapjfOtMxqCQlQLWVQF.qwV_v_x0b529QOfYkeL25w.zJgXI8efc Jml3ug3tQE_TLa8.MGSdvfJDgCuWWB1VqfjcN5Xot1Koq0oyyeZG0_gkdZpQQSFYTLp4QpMrOQs5 y4VHUYFQSJpEuWchROpV4pey5nh8Q60dLyAiJaeWaNtMpGKL168uE_PgmibxvhFBvrwyBgDwj.1_ YS_udTqyHIbhapK8hemJgeXC5WGk.wK3QZ2Ker.mho_i.4fdstSYBFegtSKADjd9F6KQjlEqpPIX Z8PTacl4mTvBDZeaJQ0EBAJbKeKw54CAwyYR1FDhZSx7G48Dr33_a21t_0fHDylqtf5ha4mxB4fa S00CXu1KAGt.yS8fvTxBvGP4wPqCXDihps.3t0lSJh0dO4ZJVvY8HumIm9i.5DPt6SdLr9g7NCRo TpTOGKaTrGT3N7HnP9vyk13h2sLHp1S.oiCEcm3BDFWzg7Cc_VJv.eci6iYOk3xnRpoZj4LOsQB0 w4lMp_I9Rzwo75o_mkRvitahl6GLAsDyPi_q.UfCnmEBOd6WhukrkVdV54xBVtDj6HzWMDBxzRxY zdwjPuAkgq916kocJk0BFZghwrfD83FXczayd8wBIYpfoPYZuvRbzlqicrrhFFJt_IwzdxEzHkzU YkkNoddYqDPckr73gdCBdwqrxjcwApBEbazqvtFKl30_d9nccmrwOn9nxK_Bs60BlRQWSXxa81GT L0c.j1TMxIFn8o7jFkikWYn4RO9r9tKOp8U4V5byFHCrV.u7Na4D.N3IwTAdoJGgCV9zaqftv_3i lfkSp9zuNAWHUQMFMoD24wPofJ5sRCPnTRwnMVKYzipokPQl.1DnNfsYveLSmRdCfEhp.w2wkj5A pncQObbLmMELPe5EZFlCE5vDnPS1CPeLWwiRLv3nuLPJRbNCpdKuHN7mi2upAHBSoiu2BgRhycpl 8n_duiLyEQeXqO_t5nKxpTKjyb1YI.cEHEp_in6av7SGWrWM00I3H56ElDXKgtFbU77IkfDz2nyk jNgn15_U0LN5FeQS5fMjfl4UhSE8YHg6LS7JlSX6NwdEEmcehjSYdbo0OPAfez3XTY_WF6Cn63Nk iI1xKgH2mYckFO5FV3MG0a9GWM4yg4McyVKE0KFcN6XNlDMHpzKlRS4ZYdQtNSDVdYasCj97xdT1 t5A1M95GtZKMEkvP31VkzyPvMn7WXXosRJRl47SJnyWu6B9fiHPzhF3c8vBWEOrC6oeQlLis4zfz PWtyzXjVTNw1IUfm3rBu.S_qV9z5_J94LCL4x.v3BT4JZtvH4AU3A5RvfmyJ5IhPY4MQUpY3RrCm 3o3dlhyv.QD1rimF8sIE7h2zhwEXTNIciWl8G56jCChrEhEDgkDn7bxdMhqpfDHc2QEFBbXSxgp8 49e94wO9k06UWB_Wwo0EmtMEVFMgyQ1bgijm1t6GDEI4qL17koFldkcb0WR_Lzpzd7S0IYn116uH LdJBE6jFEt1IcxMjE7GFReLdwi9A5nDdit4GSy6uCaeoRME2KW6.PD5IukKQcuhqd47.GG42nUHk gFna8dgnsT4TFUPkmvU.bwvSc.VAkSj0OXkB9TxukVhCF0fb7xvq3sj8f09dJNttrdRPnsURCoqe VUVsAsN3GIsTLUh99YzbS9kzc74mtaS9YYtQ76UnKYtLBbdN7CVePtT1Zz_srG9inOMRXg5pcU_N dZhbzRdn46J_eUjEUPUybiUdtnDElloobXYgTI8xX_SvLbhxGNu07WnP78QAqhCnK699VAFN8XS. 5cTBUxX6GP2wapQ8uJrBFiBSOCkwwtY45xBv9wCihcG6w0TfzbIjEiRZnVT4P7KVd3NDgOdaB2aO B3NzWwuLzyZ2RyU1CWxBHd5qwWz_mQRj9DItvwhjzvBE8uCHh.6WPg_xdcxdPLwhnBtH_WEvmfvu qREaGwQRueIHzwWZF17Ra_FHYyyyMRk8SQ8Dz.5XWLWp0DCRYPFC7QyXSJS3dbJ05xIK_89l.teu 7 X-Sonic-MF: X-Sonic-ID: a765e4d7-d6c9-4cc9-8e6f-51fdcd32e5ac Received: from sonic.gate.mail.ne1.yahoo.com by sonic315.consmr.mail.gq1.yahoo.com with HTTP; Fri, 22 Mar 2024 00:34:41 +0000 Received: by hermes--production-gq1-5c57879fdf-bmngc (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 27290ee8c32ec341c97ed35d6a69c07e; Fri, 22 Mar 2024 00:34:41 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\)) Subject: main aarch64: poudriere-devel [UFS context] cpdup stuck in pgnslp state Message-Id: Date: Thu, 21 Mar 2024 17:34:30 -0700 Cc: FreeBSD Mailing List To: Current FreeBSD , FreeBSD ARM List X-Mailer: Apple Mail (2.3774.400.31) References: X-Spamd-Bar: --- X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.997]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCPT_COUNT_THREE(0.00)[3]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; APPLE_MAILER_COMMON(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.31:from]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.31:from] X-Rspamd-Queue-Id: 4V13Hl5HQDz4vlj Note, more recent process creations towards top, older ones towards = bottom: PID JID USERNAME PRI NICE SIZE RES STATE C TIME = CPU COMMAND . . . 33693 19 root 68 0 6524Ki 3252Ki wait 3 0:00 = 0.00% /usr/bin/make -C /usr/ports/lang/gcc13 build 33692 0 root 68 0 15728Ki 3552Ki wait 0 0:00 = 0.00% sh: poudriere[main-CA7-default][02]: build_pkg (gcc13-13.2.0_4) = (sh) 30174 0 root 68 0 15728Ki 3564Ki select 3 0:00 = 0.00% sh: poudriere[main-CA7-default][02]: build_pkg (gcc13-13.2.0_4) = (sh) 26338 0 root 66 0 17740Ki 5044Ki pgnslp 0 0:01 = 0.00% cpdup -i0 -s0 -f -x ref 01 26308 0 root 68 0 15728Ki 3556Ki wait 0 0:00 = 0.00% sh: poudriere[main-CA7-default][01]: build_pkg (boost-libs-1.84.0) = (sh) 33592 0 root 26 0 15728Ki 3388Ki piperd 2 0:01 = 0.00% sh: poudriere[main-CA7-default]: pkg_cacher_main (sh) 29205 0 root 68 0 15728Ki 3392Ki nanslp 2 1:52 = 0.14% sh: poudriere[main-CA7-default]: html_json_main (sh) 28834 0 root 20 0 15728Ki 3548Ki select 3 0:01 = 0.00% /usr/local/libexec/poudriere/sh -e = /usr/local/share/poudriere/bulk.sh -jmain-CA7 -c -f = /root/origins/CA7-origins.txt 28833 0 root 20 0 13560Ki 1924Ki wait 3 0:00 = 0.00% /bin/sh /root/build-ports-main-CA7.sh -c . . . pgnslp seems to be from: vm_page_acquire_unlocked in sys/vm/vm_page.c . That in turn looks to be using vm_page_grab_sleep : if (!vm_page_grab_sleep(object, m, pindex, "pgnslp", allocflags, false)) return (false); and: /* * vm_page_grab_sleep * * Sleep for busy according to VM_ALLOC_ parameters. Returns true * if the caller should retry and false otherwise. * * If the object is locked on entry the object will be unlocked = with * false returns and still locked but possibly having been dropped * with true returns. */ static bool vm_page_grab_sleep(vm_object_t object, vm_page_t m, vm_pindex_t pindex, const char *wmesg, int allocflags, bool locked) { =20 if ((allocflags & VM_ALLOC_NOWAIT) !=3D 0) return (false); =20 /* * Reference the page before unlocking and sleeping so that * the page daemon is less likely to reclaim it. */ if (locked && (allocflags & VM_ALLOC_NOCREAT) =3D=3D 0) vm_page_reference(m); =20 if (_vm_page_busy_sleep(object, m, pindex, wmesg, allocflags, = locked) && locked) VM_OBJECT_WLOCK(object); if ((allocflags & VM_ALLOC_WAITFAIL) !=3D 0) return (false); return (true); } . . . [10:08:06] [01] [00:00:00] Building devel/boost-libs | boost-libs-1.84.0 . . . # poudriere status -b [main-CA7-default] [2024-03-21_06h23m31s] [parallel_build] Queued: 265 = Built: 213 Failed: 0 Skipped: 0 Ignored: 0 Fetched: 0 Tobuild: = 52 Time: 10:50:40 ID TOTAL ORIGIN PKGNAME = PHASE TIME TMPFS CPU% MEM% [01] 00:42:40 devel/boost-libs | boost-libs-1.84.0 = starting 00:42:40 951.54 MiB =20 . . . Unfortunately: A) The booted kernel is my personal build based on -mcpu=3Dcortex-a76 and LSE_ATOMICS . (It is in use on a RPi5 booted via EDK2.) B) The booted world is a PkgBase world. C) The poudriere jail's world directory tree is my personal armv7 world build based on -mcpu=3Dcortex-a7 . All are based on: main-n268827-75464941dc17 . (Well, PkgBase commit identification/verification for world does not exist. I happened to update PkgBase during a long lull for commits to main. In the context, the boot-world seems unlikely to be involved here.) The boot media is a U2 Optane 960 GB used via a USB3 adaptor. I've done bunches of builds in the (A)-(C) context on the RPi5 and have not seen this before, so: does not look to be readily repeatable. (Unfortunately, the purpose of the build was to find out how long the particular build configuration took to finish building the 265 packages from scratch, for comparison to other builds.) I may wait for the system to become fairly idle and then see about forcing a crash dump. It may be a while before the poudriere bulk runs out of packages it can build, absent building boost-libs . Side note: As far as I can tell, how to identify a context that allows identification of what commit vintage a PkgBase world is based on is unspecified so far. For a PkgBase kernel uname -apKU may well report the kernel-commit identification well. (Hard to verify.) =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Fri Mar 22 13:00:53 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4V1Mrv6Dvfz5Dsfh for ; Fri, 22 Mar 2024 13:01:03 +0000 (UTC) (envelope-from imb@protected-networks.net) Received: from mail.protected-networks.net (mail.protected-networks.net [202.12.127.228]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mail.protected-networks.net", Issuer "R3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4V1Mrv1YnZz3y21 for ; Fri, 22 Mar 2024 13:01:03 +0000 (UTC) (envelope-from imb@protected-networks.net) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=protected-networks.net header.s=201508 header.b="J zMaAux"; dmarc=pass (policy=reject) header.from=protected-networks.net; spf=pass (mx1.freebsd.org: domain of imb@protected-networks.net designates 202.12.127.228 as permitted sender) smtp.mailfrom=imb@protected-networks.net DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d= protected-networks.net; h=content-transfer-encoding:content-type :content-type:subject:subject:from:from:content-language :user-agent:mime-version:date:date:message-id; s=201508; t= 1711112454; bh=bcolCCZPoiq37lfRTPFxp6A84kh/66yWEUlvLqOxfIU=; b=J zMaAuxSiN6TWL7pyQTsIFAkjPFvUT/as/C0eCjd/uXsVjvkfLEdCCtBK+cRkoLTF dpHR7xiorLJBzvxJlNpUQ7TAnyCKqnE15O2p+ep4RROShdTew2r0rfVS+NKWVnvN 01GruoPQUsKcv/GKxN11ON1wHBwCouslPd5EL7y0Ug= Received: from [IPV6:2600:4040:53d9:8200:4ed7:17ff:fe3c:c98] (unknown [IPv6:2600:4040:53d9:8200:4ed7:17ff:fe3c:c98]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: imb@mail.protected-networks.net) by mail.protected-networks.net (Postfix) with ESMTPSA id 759032CED4 for ; Fri, 22 Mar 2024 09:00:54 -0400 (EDT) Message-ID: <157c12ab-a3f9-4349-94cb-e2228ff5bad3@protected-networks.net> Date: Fri, 22 Mar 2024 09:00:53 -0400 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-NZ To: freebsd-current From: Michael Butler Subject: kernel build w/o DDB broken after c21bc6f Autocrypt: addr=imb@protected-networks.net; keydata= xsDiBETHZAURBACJicNaIbVVVZahtQcdJeogtTLjCYAdj4kFMpy6Y3Ac19UNWDM+TrD4yFPi 5nc/pp9M/5Q4RNBr6a97fTYroTaq+vDwWdklOHwD2ZXs7FqwWOtVSIPT/rev5fUvwEF2VFYE sNDbpE5HHpP/oFUw5scEJZVyOBJSGvYb1IhV55NWswCgzkUGbG8A3s+oZXkHqTCYGW/seukD +wTo/L835xLpbTJxoxEKeGA3aWifSsRvpWWHyXye6sTkSN3SmtE9A8Pqmdb1dBEO0eOms6GD RamvCFgdvg2HesAv9l7L/7Mm9iKJs6uTAa+taIQslpumGh4PRc94IepVFzAa4Ef/FA4mWx9w P/EqNsKUPE2U5HI1decbopkxH/d/A/9Hupc10lPsXVMACd54/YZRsSTTcArheekm8qE/f8Hl 1Q7At+yuFgfMll4QPAhefnrLUanXF1bWtxG5PmaJktDYp3HOmy43giZgacgt+a3TVd6vu8Gs DnI4FOfYllq7mZFezMIulCWUYtnkMEXEeyzp39dygi7blPIjckWlQ2sc380rTWljaGFlbCBC dXRsZXIgPGltYkBwcm90ZWN0ZWQtbmV0d29ya3MubmV0PsJgBBMRAgAgBQJEx2QFAhsDBgsJ CAcDAgQVAggDBBYCAwECHgECF4AACgkQQv9rrgRC1JL7mgCdEnPeo22kqT/bES+D78QSGhNR r8cAn2xOMeu6pBrc2tDY8Ky/70HBctmjzsFNBETHZA4QCACKbm/PMn4QcyDEvIn4MF+t2E1A zgiBAkPCMtWT1CcqeUj13OwNM8qJD/mBWjCZCnr1hKVbvzOmgKaM4uDCWIcSCdoDTJx1DqMx abr+EpHz1fL6aagEOKHz5sCYOkDXt3zzZ/5RBMdkEJwunXYtAbu5e68oty+d0DFzAM3pBp6l GC0TE3VutmFR/KK66rf0KB83YQBf/IAtyqsRIQPP9t0SLfJ+kqKXf73nvAUFEtb21gZSzhTm QP87QKyQvenE8o4PQ2tEslq2jICB7pGcqIrwP4o3Hl4V+HXi3lA26MMJ5rakQB2sKKWroPVQ BiRXO+W8Qf+0oQFq38oMXR5sPOs/AAMFB/sEKcjzvkwviZOsDElthxtgrmqUNKC9G/4Fw0tK k6fMynv+bcKz85k2uWOIfefUKBFoQ0SCphU4jquJENqqy6BPTkXePlIJok2/GkF7xtHm2FPq tTTuYmoBrGsls28Z9dn2LcBwFHz59SSWM9JFPIvFr9HCkKtp6zPUsJd5b02+0wgzDubTMQS9 M2LwGSh9xK6xl4MGgngl22b0TZDh5qHwmsywOX6SbGsQfeNpkptJ4gPjShypusFyF+pevnCM wTfUPCBd/AFbu2fHFQjA8sgkr5IqXuc4PoiIBXc9upoFpDqGkYssAKbzGcRsK94a8hRROJV9 bzPyYempIWaPXr2EwkkEGBECAAkFAkTHZA4CGwwACgkQQv9rrgRC1JKqhwCeOov6gTo8eWte es3gbLr2n2b5AXMAoItSlajet574lkouzY3u3scSRfiE Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.99 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[protected-networks.net,reject]; R_SPF_ALLOW(-0.20)[+mx]; R_DKIM_ALLOW(-0.20)[protected-networks.net:s=201508]; MIME_GOOD(-0.10)[text/plain]; XM_UA_NO_VERSION(0.01)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_VIA_SMTP_AUTH(0.00)[]; ASN(0.00)[asn:5716, ipnet:202.12.127.0/24, country:US]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_ONE(0.00)[1]; RCVD_TLS_ALL(0.00)[]; TO_DN_ALL(0.00)[]; ARC_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DKIM_TRACE(0.00)[protected-networks.net:+] X-Rspamd-Queue-Id: 4V1Mrv1YnZz3y21 When a kernel is built without KDB and DDB, it fails with: --- kernel.full --- linking kernel.full ld: error: undefined symbol: db_ctf_lookup_typename >>> referenced by kern_ctf.c:329 (/usr/src/sys/kern/kern_ctf.c:329) >>> link_elf_obj.o:(link_elf_ctf_lookup_typename) >>> referenced by kern_ctf.c:329 (/usr/src/sys/kern/kern_ctf.c:329) >>> link_elf.o:(link_elf_ctf_lookup_typename) *** [kernel.full] Error code 1 From nobody Sun Mar 24 00:01:07 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4V2GS35bfHz5G9GW; Sun, 24 Mar 2024 00:01:07 +0000 (UTC) (envelope-from salvadore@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [96.47.72.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4V2GS349KJz4ZkB; Sun, 24 Mar 2024 00:01:07 +0000 (UTC) (envelope-from salvadore@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1711238467; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc; bh=wQRuhnGxjGDxWCKVSidOba/8XIYV/+IJAobT8GnlIo8=; b=l705n5EqX81nOBVKZL4NfF8L7RCuZ9TMlkrIjxfDMjaQ3HVfGUw2UCZJVlZlTMjzsxP3rV jiCqHOuDdF++RjZeIsg0f9um7P+dok0ovw+C4uvxwnrOWZ6Ra7ARrYwqyupe5vhLtnFoaT kNMYaSgQfYu6X8WZ/JWD3TUifO70pyBpBI+Vg9fE9m6TPAnr2EShzJomJCad9YyP+QtpQ5 U2PFLvD7pkGhAi74i+z4ARp1Igaygd424/1jgwqU5SIdt/AU7JoKJd6L6v2IWd3dknxC8d rf29nBZUk6l/1wEpq2YHCgMkGbMJrkotajHtJRBfmxxIV/gmvh9fcJlgu0sKZw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1711238467; a=rsa-sha256; cv=none; b=ORVlEG7lh+aI4jsbmx9ZB1PhQYV9pOEARSYg7p3NEbpMsQpR+m3rd9qwACjyHGqZWbr3Qe 1CmIRHXNQNpbV1hkeDVDHYjZhCLM3I5M0ZNFsjU19OdV6L74yorgg+spbuOJA4UOS4mcfy kWgFusm/qNY/sFWsyFwLpG+3iuyLCy5QBvSPJhKOTKextfH0DitmkShhERprD8NfbKimRA P1Ebj994fN6IdEKF46fVJPEwN4K1Re3624BZHbsCeWW1Yi/i+MtfMGqPx2ywi248eBoWUz a9QaH6jPk+GNqdaekB+VCOrVrOVjApPmuTLKeel/a/pPlfev9aogxnYiby9R4A== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1711238467; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc; bh=wQRuhnGxjGDxWCKVSidOba/8XIYV/+IJAobT8GnlIo8=; b=hCSd4T0MKI8RXoBfdSBHQpYQ3/yUMYpQazByHi/pae144UBatGN8DmTDCXEIWFHMueie4A 6NFRwk5vvKZqbv5JbHZb24mKar9rjcz43HsiMJuJou8ZtvitlXPe1f7aZFtGRVaPVl3Xcm qVySzFNBB1IvP6HXvygohYWE2tT4z7IV36xGdZl54JcIcan2Rt3jS3IlljJXRPyNveXJa7 7iLSHfiwJL+DI46hGCSN7gEbxPpxvEpbijIPHoUudvWBawxnASNmAalozLk4ySxG3uKzfX Fdm9Zv/xwzjHYBQQNCVFSUtAAYSGCjuipd1JeYGUFSJrXM8ZFF+bofhQI2AOLA== Received: by freefall.freebsd.org (Postfix, from userid 1472) id 6BADF172E9; Sun, 24 Mar 2024 00:01:07 +0000 (UTC) To: freebsd-status-calls@FreeBSD.org Subject: [LAST OFFICIAL REMINDER] Call for 2024Q1 status reports Cc: freebsd-current@FreeBSD.org,freebsd-hackers@FreeBSD.org,devsummit@FreeBSD.org,secretary@asiabsdcon.org Message-Id: <20240324000107.6BADF172E9@freefall.freebsd.org> Date: Sun, 24 Mar 2024 00:01:07 +0000 (UTC) From: Lorenzo Salvadore List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Dear FreeBSD Community, The deadline for the next FreeBSD Status Report update is March, 31st 2024 for work done since the last round of quarterly reports: January 2024 - March 2024. I would like to remind you that reports are published on a quarterly basis and are usually collected during the last month of each quarter, You are also welcome to submit them even earlier if you want, and the earlier you submit them, the more time we have for reviewing. Status report submissions do not need to be very long. They may be about anything happening in the FreeBSD project and community, and they provide a great way to inform FreeBSD users and developers about work that is underway or has been completed. Report submissions are not limited to committers; anyone doing anything interesting and FreeBSD related can -- and should -- write one! The following methods are available to submit your reports: * submit a review on Phabricator and add the group "status" to the reviewers list. You should put your reports in the directory doc/website/content/en/status/report-2024-01-2024-03/ (create it if it is missing); * submit a pull request at . You should put your reports in the directory doc/website/content/en/status/report-2024-01-2024-03/ (create it if it is missing); * send an email to status-submissions@FreeBSD.org including your report. An AsciiDoc template is available at . We look forward to seeing your 2024Q1 reports! Thanks, Lorenzo Salvadore (on behalf of status@) From nobody Sun Mar 24 13:02:56 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4V2bpT1cHGz5FdZ3 for ; Sun, 24 Mar 2024 13:03:13 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic316-54.consmr.mail.gq1.yahoo.com (sonic316-54.consmr.mail.gq1.yahoo.com [98.137.69.30]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4V2bpS1mTPz4nt5 for ; Sun, 24 Mar 2024 13:03:12 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=ZUQ0IE0T; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.30 as permitted sender) smtp.mailfrom=marklmi@yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1711285387; bh=ATNQilRnSx2KbJ3wqDUHmhJDomEos+lyCkrxzJoAiW8=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=ZUQ0IE0TGRsx2qxLQIlokT27lI7kj9IjUY6vHawG0ddXI+pvfz0kWdA4cGagGtF1efc+46uMbHdr2btTZ22ZrJW/T7sFyKLKltswDaNFVN0VjoHN90XlKc6ID2Nk9UyMe6+izIYLo40aTujkQPBX4JLYAOPxNk1p1iE7Dgu176jCATM+cloihYM5D/crm4Uo76mJI/PlqgHtDpfjx9yuikvAYTuvcuXZ+CHzsUjO8/+LDEnS2/m6YmPqTmxqTpKkwhrQ4t404DAf6aBitpXzu5R/4zxjFxP4f4rJ8wuJCksxkCd73CcpRnsXl3x7UxOLZjp1K7nq9jikuj9wdVMfXg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1711285387; bh=1wEV9vzgw27ipxcop5oANOK+zRU92oAjA2Q5r3xE9Gv=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=pZqFDVnEZmOQmLFmjZuMn6aPGGCo2TmhB1+Ns2TE3QRs+wdXld4woBKdQUNIs4TC6OZGF9Gi+wnHSd41tNtF2fIcbouIOwHbpC7qrctbnsbeIXS9m38Jd5mQRYhMI5NwIUhFnwsLhyeZzWBs1L6SoASC2VNRYDlsUhunOOcIOPvMZub+MoFx3dUPk2B0Vblwc7zdBzbQQWUzRWOjqDKfA7ku/jkwFSuVNmfoYGnyr6cvOXaPA/Avi43FnJOGYjJgdIMQTom0bawoM8IE26oQj9jT9sHZw+iMM6mkJ+1mKzh37/3U2AuVV5gWahuUEdhmrohJHfY+h1HwN6gEb2hVvA== X-YMail-OSG: Qy5eVoYVM1mTHUbOcF0cfgGWY1YIOymBWE0LyHPMaV8wvXG0ljQWeZLCJ2LEzQ2 dKyBDcxZJnjsq0lexbuH1yrblY64hlZlhbUrLURIV.ncyRsqdilbIwTQovZWxP_4UsN1PdFigGiH PRKEpJekqbJCekjaQommk6JPCHxtfY73KWKueqfPJvQMMYruNIBQdv7yYsURX_otZVvdrmjsCvT_ Z_4xwkg8HM9QHx4a50fXSwOgO_wr09mXNC3nGrJ2FYXRi2_f4BKC9H16R7lzvlTTlCJuPeoZWAdf Z8pDsLCocWjOe4u8Xu7EN3LZAddr8z8mhUMd6963.ojoYmXukVv4H6MD.1CJICuEeXTgrwR6Xj6r KzAPsVs40brebAGy5LbjZp2LPp13jLfvEfk_67ltcJetS5s8qDp.V5.tVVlp_C.zPkrPP2xILnIY A0yLYMMC_eOhim9i_bWXWsOK8PYVFxsMyuLyJZOnpSZhxeo0uVVmV5HSmB10SffOG6W2Q2A_QJdn mKXZy4Z0F6y48jwTlGYCkEK8S8yANBcTDwnXsjLVJdhHmtz39stWi8Ea_qNj.s7fuZDHg6yPznPx gIJuaXEQ6Rimi5fHLq1BKdHqyiB7q2mbYM_Me4shxV3f6lKLpEOZu1rIqWSNnRPTvMn2b_SBkB0f 7gOboPQZuWva712FStItfs7O_5JgoMvHN3M1eL2eCcNQTNLnE_1qmrP.70w_sPCIIvIllCaltTmk UtC2bQThmjughA9oVk68dJdeJBC4Wx6PddVhP7q45E4g3z2FF9J8uCG_N_GO_FnbtGZlkrRTfEjZ Lwaemhl2gl1GogM98ggLMfgfmzLLYU0hwe86VZsR9AsMCEP3t2ECGdhL8oFbBK.FO5899cK3FJS6 0yaogIYXA82Zd.nq8WyFWbStutOUq8Q4gLzVUXu7Ah35v0YkHRfWlhbRQEF5RneRzIljyujv8CNr eoPw28N_fnLP.Os_nHO_CjfJRail__zeV7rcesS.dQ4JlyXqgGJuUKdW2KJSmYlBGjO.BDuM49EW j4_Imw8blhJb9MuMpeEh2PE79CxeHJmEZGzU6wwqxsq_7_x3UcIsPPGxfPdjcaGpmpb0_iCqt1vy hslly1PouLsJsynjPoDqM1R9urWNCAx14OBHVowPSMQl.8TkDt5MYhUXjZ10HhWi8zRvFtuSx59h iPXKBeLi0OY51RAsfMz7cvxu5owxr3.faUu__FAMFCg1zRxGa84zHQsKIFe_9aGRedlX3AZe5NQx wl4AWd7aAKno3NV6bdrq8tqqdHzEijD4Ki9UZLQd.DWswPOSxoZiCKmF01ZaRblSBtr2Y9I2QVM4 Z96rvlC0saBMFEU5rwvkmN.T1onoNy9TEflocgg0ZaL079zEA9.c_im34n0LFkgwG6xh2OrU6BDD 3H8oxF4QTAh0DKMlP3dtPkGV29d1CNppRyzN8YZI6HwRQX6GIM8Wayubvt48hL63Q4jKq8tDsYcK KPX.kqYQ62cB6YeFP0P5AFuwLKLbZCZEBStef8dLkAv53h12q1m8XgcTGfXzoPlUMVsiDhnWzYcG RU7cQv5lndKhyxvMNKWEHWRvWhlOfyKXKTfKkrS9WtHeli7EXD6vGlgQ9DqSz.r_Xz5Hse5Ds_13 FDAMAJzUJUuJryw43w27ulZpMKNKegZFyndV0P5CZVD2RCHDxo2A31WCSx6.buTzsBo7vQpFJAkd aoDwnafPMZOVqeUs5cR3yyQaoHeM8.R1CJdYI21On5tqoLnY_om5VQZJqPCPOS7krmEZOyLMJn65 WVA3GpH2fdva2H.v1lTuQtlFP2KVWv8RltKlK2VbFcEoZfD2pcFvsaUJbxskPmX4CKk58mE6ZRqL mvf7Fz7lUoFWt5WmthxzB0CQ7H5JdvmMzBSannwKXmSkkLI071C8LSac.89iVfkFLSwrZxNozTOl yr5q4O1R4741_yT8aYTVWGrfSmhhC3dLzvZCy4.Z2r0sqTYRVSgC5ZWSPnJKWD4ncFZ4y0hOf4Fw zfUolmp6w7Yk90ZEQxy6CBcGCS6dlEfysqIBfSrBMY2pJaM62umw0yXX.zjI3H9MTSkFEze1ky50 vwQeCcb4V2ijhWQ8sp0v_JdJCUm0_qXPd13e89UvDReX5tgC7z3CsSWZ.dzHFOQkmiB1u.pa59Az IPhMNpCV9pTqD07Ias_vXJXIWxzmzh84hZdwcZcowwzh7UIW6_FH..5.E6CbtqEX51MuRHQHR_LG FjGABYVOcVQPinQgk6r_nZTZH3x_PZfMK3W..sawiBTVtRNOQtQ1SWAz1YnmtrfB8oeIhzwknegs - X-Sonic-MF: X-Sonic-ID: 6152f479-ee9e-4421-b321-b114fc3807c6 Received: from sonic.gate.mail.ne1.yahoo.com by sonic316.consmr.mail.gq1.yahoo.com with HTTP; Sun, 24 Mar 2024 13:03:07 +0000 Received: by hermes--production-gq1-5c57879fdf-c7xks (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID c9e64e2394b8cc9f57a38c0e1c1d50cd; Sun, 24 Mar 2024 13:03:07 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.400.31\)) Subject: FYI: main-n268827-75464941dc17 GENERIC-NODEBUG UFS-based poudriere bulk context got 4 "swap_pager: cannot allocate bio" Message-Id: Date: Sun, 24 Mar 2024 06:02:56 -0700 To: FreeBSD ARM List , Current FreeBSD X-Mailer: Apple Mail (2.3774.400.31) References: X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.99 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.994]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; TO_DN_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; FREEMAIL_FROM(0.00)[yahoo.com]; DKIM_TRACE(0.00)[yahoo.com:+]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; APPLE_MAILER_COMMON(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.30:from]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.30:from] X-Rspamd-Queue-Id: 4V2bpS1mTPz4nt5 Context: I'm deliberately testing building example poudriere-devel configurations with only 2 GiBytes of RAM and RAM+SWAP =3D=3D 8.5 = GiBytes, SWAP on a partition of its own. # tail -3 /var/log/messages Mar 23 16:39:38 aarch64-main-pkgs kernel: pid 37137 (conftest), jid 11, = uid 0: exited on signal 11 (core dumped) Mar 24 04:51:50 aarch64-main-pkgs kernel: swap_pager: cannot allocate = bio Mar 24 04:51:50 aarch64-main-pkgs syslogd: last message repeated 3 times It is the first time I've seen such a message. I've a modified top that tracks some "MaxObs" (Maximum Observed) figures. They happened to show: Mem: . . . 1473Mi MaxObsActive, 477304Ki MaxObsWired, 1908Mi = MaxObs(Act+Wir+Lndry) Swap: . . . 3101Mi MaxObsUsed, 4456Mi MaxObs(Act+Lndry+SwapUsed), 4887Mi = MaxObs(A+Wir+L+SU), 4933Mi (A+W+L+SU+InAct) (The 4933Mi (A+W+L+SU+InAct) is from when 4887Mi MaxObs(A+Wir+L+SU) was live but is not a MaxObs figure itself.) It was paging significantly at the time, of course. It seems to have survived okay and everything continued to build. It got past the peak RAM+SWAP use activity. For reference: /usr/local/etc/poudriere.conf has . . . NO_ZFS=3Dyes USE_TMPFS=3Ddata PARALLEL_JOBS=3D2 ALLOW_MAKE_JOBS=3Dyes MAX_EXECUTION_TIME=3D432000 NOHANG_TIME=3D432000 MAX_EXECUTION_TIME_EXTRACT=3D14400 MAX_EXECUTION_TIME_INSTALL=3D14400 MAX_EXECUTION_TIME_PACKAGE=3D57600 MAX_EXECUTION_TIME_DEINSTALL=3D14400 /usr/local/etc/poudriere.d/make.conf has . . . MAKE_JOBS_NUMBER_LIMIT=3D2 /boot/loader.conf has . . . vm.pageout_oom_seq=3D120 rust and llvm18 were building at the time. The from-scratch bulk build is of 271 packages. 143 had already built. llvm18 was using more RAM+SWAP than rust and was working on some llvm-tblgen runs for AMDGPU at the time. The swap partition in use was/is: =3D> 34 1875384941 da0 GPT (894G) . . . 523773952 13631488 da0p10 freebsd-swap (6.5G) . . . # uname -apKU FreeBSD aarch64-main-pkgs 15.0-CURRENT FreeBSD 15.0-CURRENT = main-n268827-75464941dc17 GENERIC-NODEBUG arm64 aarch64 1500015 1500015 That is a PkgBase kernel and PkgBase world combination. The poudriere bulk jail is also 75464941dc17 but was via the artifact build materials. The 2 GiBytes is via the RPi4B config.txt having total_mem=3D2048 . (I've no access to an aarch64 with 2 GiBytes of actual RAM.) The media is USB3. It happens to be U.2 Optane 960GB media via a USB3 adapter for U.2 . The UFS partition was/is: 537405440 1337979528 da0p9 freebsd-ufs (638G) =3D=3D=3D Mark Millard marklmi at yahoo.com