From nobody Sun Dec 1 04:38:47 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Y1DhM6wvFz5gCHL for ; Sun, 01 Dec 2024 04:38:59 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Y1DhM269Cz464Q for ; Sun, 1 Dec 2024 04:38:59 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pf1-x433.google.com with SMTP id d2e1a72fcca58-724fee568aaso3016351b3a.1 for ; Sat, 30 Nov 2024 20:38:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1733027937; x=1733632737; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=cz/N7jaK/7vzf4dqJad3ugKlG0on7Twj2L+jQ2wAU2U=; b=tQZs6OT5Bd/NKk7QdZZtU5VU3TzEUL3Z/9J7nsP7nGnGK3VAuiPifyVBJLEK0XXuJd 5VwhIDq9jLE50pwEaL3FFILVBNahXUwSzhstQfp4O6YY3+orpZEGaoOLNMOQtPyJTsVN w5uyj4LhEVdhqT9QPt2sfQ8yqphFgBFuSwscpXT47ioF4s+ISwD9BQCiMyOjYb9VDmUE ismX2O/kdD7XvlmBwpDCD8EaBk6eZHKtyz1WcKj9ldwBTUuPBpiZUgO1pln/qBt/V9on d1BkmFgGvrams8gZ6E3Si4ZKMxwov7I7hjTLc6cyCmK3OUAxHCBxfmGzZqSkDguIsw/Q Vu8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733027937; x=1733632737; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=cz/N7jaK/7vzf4dqJad3ugKlG0on7Twj2L+jQ2wAU2U=; b=HlWVsn3NB2dXH8lCI++LUQ++pThwYQW68/dZEI9pBp6etXxod9tLuvK8Z6K1yVl4jd r/tb9Q8asSEl7nlTwH1OFm0bUxIuPy7YHV2QilQt+BIpZ95L+mStvLDmPTg9RUFKvKpD G1VVBPPKt5h1SJJBPMea1eyvxgKbCQVsk36InpraM6W/zMU8M5QIvjBSBB8RTWrUjWZf a7o6EU3dUU7fIfmdIarQxqMIJGyXmZ3DfjJDVoCRb/7NJS4wR1vZ6GwIVNz5IpbRlvE8 roGFZNX0SctdrysNijaIplyxHbRo10c4pw8nmy6OnjR4f0Bzm/xUdh4GsGxzLdsyMBOk OGTA== X-Forwarded-Encrypted: i=1; AJvYcCVcHL4K1Dwj+CkL1ljzxpqlHbnlEAF22JBLu1z+99HRJRsIRL8j8AueAopCGg5QPiVaLn0gDFMKi5e7LEOAy2c=@freebsd.org X-Gm-Message-State: AOJu0YxrEQBnRDqC/01HHHgapxaYBFE30XSnt7iK/v+YprIHBPweTpus 0GJD5v1PS5Ln5WhF90HQ09uXpXiblHjTWz1Ks+gZvJsMSJJ9IWDu03RMCYOIPAwx3svSKCMraEM nzIV7v/7BwtxZ0DYUTdrZqiGmplGti7n+e8mmuA== X-Gm-Gg: ASbGnctJ1MQ9I9k2wUEN/6oF6vD5K+ih5E6Pq9SmhnOsh3ONoVdkBK0fdDqupKq2rlV L1R2y+RS9B5Z8xupr6WRZBmU1x61LVsTWCP7YifNeAiqYen/8TIsEPfXYvkZZQQ== X-Google-Smtp-Source: AGHT+IGVzOHpCFn4aZ9afwV8gXJKyg+FSj58gLoZDESZfX1VW0t3Pd+NZsRPI3A4iNATb0zn36sjKTJMlG1q09PCX9Q= X-Received: by 2002:a17:90b:3a90:b0:2ee:ab11:fab2 with SMTP id 98e67ed59e1d1-2eeab11fca1mr1465631a91.22.1733027937444; Sat, 30 Nov 2024 20:38:57 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Sat, 30 Nov 2024 21:38:47 -0700 Message-ID: Subject: Re: Long time outdated jemalloc To: Minsoo Choo Cc: cglogic , FreeBSD CURRENT Content-Type: multipart/alternative; boundary="00000000000098fea706282e028f" X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4Y1DhM269Cz464Q X-Spamd-Bar: ---- --00000000000098fea706282e028f Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Yea, I need to get a copy of jemalloc 5.3.0 and 5.2.1 to try to 'bootstrap' the vendor branch. Then I need to bootstrap it... I just did the same with edk2 (which had a vendor branch, but hadn't been updated since svn times). However, jemalloc doesn't have a vendor branch yet, so I'll have to create that, but I'll start with the current version rather than doing full history... So I'll start there. I also just did awk and lua, so once I have things bootstrapped, I'll be able to add 5.3.0 and then layer Minsoo's on top of that and then start testing it somehow. Malloc makes me nervous to touch, honestly, but I'll give it a go and test boot on my system and maybe see if we can survive a workload at work w/o regressions... But I can't do a full test with lots of machines until after the first of the year (though I can do a couple for a few days before then). So my next step is to bootstrap the vendor branch... I'll give that a try tonight. Warner On Sat, Nov 30, 2024 at 8:26=E2=80=AFPM Minsoo Choo wrote: > I have already submitted PR on github ( > https://github.com/freebsd/freebsd-src/pull/1337) and phabricator ( > https://reviews.freebsd.org/D41421). I don't have access (commit bit) to > freebsd git repo, so there is nothing I can do at this point since vendor > import and landing patches requires commit bit. > On Saturday, November 30th, 2024 at 1:42 PM, cglogic < > cglogic@protonmail.com> wrote: > > I see, it happens. > Maybe another committer will volunteer to do the update. > I hope it will make its way into 15.0 release. > > Thanks. > On Friday, November 29th, 2024 at 9:38 PM, Warner Losh > wrote: > > I've been swamped. we need to bootstrap the vendor branch, and the way > prior updates were done > isn't so great. > > Warner > > On Mon, Nov 25, 2024 at 2:21=E2=80=AFAM cglogic = wrote: > >> Hello guys, >> >> How the update of jemalloc is going? It's November now. >> >> Thanks. >> On Monday, July 22nd, 2024 at 7:02 PM, Minsoo Choo < >> minsoochoo0122@proton.me> wrote: >> >> First, sorry for late response. >> >> cglogic, thank you for bringing up this issue again since I nearly forgo= t >> that this issue was still open. >> >> Warner, as I can't access to my FreeBSD instance until the end of August= , >> but I can still edit and push the code through my Arm Mac. This means th= at >> I can't test the updated code on my machine, but I can join the review >> process and listen to change proposals. >> >> I'll open a Github PR in a few hours. (The phabricator review will stay >> opened just in case) >> On Monday, July 22nd, 2024 at 5:08 AM, Warner Losh >> wrote: >> >> >> >> On Sun, Jul 21, 2024 at 2:03=E2=80=AFPM cglogic = wrote: >> >>> >>> On Sunday, July 21st, 2024 at 6:54 AM, Warner Losh >>> wrote: >>> >>> >>> >>> On Sat, Jul 20, 2024 at 1:59=E2=80=AFAM cglogic wrote: >>> >>>> Hello FreeBSD community, >>>> >>>> After Jason Evans stepped aside from maintaining jemalloc in FreeBSD, >>>> it's not updating in time anymore. >>>> Version 5.3.0 was released May 6, 2022 and FreeBSD still not imported >>>> it into the tree. >>>> >>>> There is a pending review https://reviews.freebsd.org/D41421 from Aug >>>> 11, 2023. >>>> I'm successfully running FreeBSD/amd64 system with D41421 applied for = 8 >>>> months, as well as many other people. >>>> >>>> Can it be reviewed and committed to CURRENT? >>>> Or, if there is no committers willing to do it, can commit bit be give= n >>>> to submitter or another person willing to do this? >>>> >>>> It's very disappointing when users spend their time to fill such gaps >>>> and their efforts just ignored by the developers. >>>> Every year FreeBSD Community Survey asking about user experience in >>>> contributing to FreeBSD. >>>> Here you can see an example of such contributing. >>>> >>>> >>> First, thank you for being persistent and continuing to bring it up. >>> It's important to do that to make sure this (and your many other) >>> contribution doesn't fall on the floor. >>> >>> And to be fair, we're only 3 months since the last update. Still, quite >>> a bit longer than you should have to wait, but not nearly the year the >>> original date suggests. >>> >>> And this is a perfect storm of "how the project is bad at accepting >>> contributions": >>> (1) The original submission was close to the 14 branch creation time. >>> This meant that we weren't well prepared to look at it since it is such= an >>> invasive change (at least on its surface). It also slowed the initial >>> response... >>> (2) There was a number of back and forth requests for changes, which >>> took time to sort out... >>> (3) The size of this is huge, well beyond the capacity of Phabricator t= o >>> review accurately... >>> (4) It's a vendor import. That means we can't just drop the Phabricator >>> review into the tree... >>> (5) It's phabricator: this is a great tool for developers, but we have = a >>> terrible track record of using it for intake from new contributors. We >>> don't have any oversight at all over this tool, at there's at best tepi= d >>> and luke warm attempts to look for drop balls. >>> >>> All of these things are a terrible experience. I can only apologize. >>> These days, we might steer this towards github, but the 'vendor import' >>> means you really need someone on the inside, or you need to be on the >>> inside to make that work. >>> >>> So, how to move forward? Well, I'd like to propose the following: >>> (1) submit all the other Phabricator reviews you have open (they are >>> mostly good, or close to good) to github. Github is being actively mana= ged >>> and will make it faster to get things it. It's a much better tool for n= ew >>> contributors (and even frequent contributors of smallish things). >>> (2) I should do an vendor import of 5.3.0 from github, and do the merge >>> to a branch and push that to github. You can then layer on your changes= and >>> those can be reviewed more closely as a pull request against the branch= I >>> push. I suspect that most of the issues are sorted out already >>> (3) I'll land it via that route... >>> >>> And, if the sum of the other pull requests and this are good (and I >>> suspect they will be), then we can talk about commit bits and such. >>> >>> It's experiences like this which is why I'm trying to stand up github >>> pull requests as a reliable way to get things and and the best place to >>> send people... >>> >>> Thanks again for persisting, and also for expressing this criticism tha= t >>> we (hopefully) can use to make it better. >>> >>> Warner >>> >>> >>> Hello. >>> >>> I'm not the author of D41421. Just applied the patch to test it 8 >>> months ago. And recently discovered that it's still not committed. >>> I can't copy your message to Phabricator because don't have an account.= Please, >>> if you have time, help the author in D41421. >>> >> >> Ah yes. I've been in touch with the author for other things, and somehow >> thought it was you.... I'll reach out to him via other means... >> >> Warner >> >> >> >> > > --00000000000098fea706282e028f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Yea, I need to get a copy of jemalloc 5.3.0 and 5.2.1 to t= ry to 'bootstrap' the vendor branch.
Then I need to bootstrap i= t...

I just did the same with edk2 (which had a ve= ndor branch, but hadn't been updated since svn times).
Howeve= r, jemalloc doesn't have a vendor branch yet, so I'll have to creat= e that, but I'll start with the
current version rather than d= oing full history...=C2=A0 So I'll start there.
I also just d= id awk and lua, so once I have things bootstrapped, I'll be able to add= 5.3.0 and then layer Minsoo's=C2=A0 on
top of that and then = start testing it somehow.

Malloc makes me nervous = to touch, honestly, but I'll give it a go and test boot on my system an= d
maybe see if we can survive a workload at work w/o regressions.= .. But I can't do a full test with lots
of machines until= after the first of the year (though I can do a couple for a few days befor= e then).

So my next step is to bootstrap the vendo= r branch... I'll give that a try tonight.

Warn= er

On Sat, Nov 30, 2024 at 8:26=E2=80=AFPM Minso= o Choo <minsoochoo0122@proto= n.me> wrote:
I have already submitted PR on githu= b (https://github.com/freebs= d/freebsd-src/pull/1337) and phabricator (https://reviews.freebsd.org/D41421). I don't hav= e access (commit bit) to freebsd git repo, so there is nothing I can do at = this point since vendor import and landing patches requires commit bit.
=
On Friday, November 29th, 2024 at 9:38 PM, Warner Losh <imp@bsdimp.com> wrote:<= br>
I've been swamped. we need to bootstrap th= e vendor branch, and the way prior updates were done
isn't so great= .

Warner

On Mon, Nov 25, 2024 at 2:21=E2=80= =AFAM cglogic <cglogic@protonmail.com> wrot= e:
Hello guys,

How the update of jemalloc is go= ing? It's November now.

Thanks.
On Monday, July 22nd, 2024 at 7:02 PM, Minsoo Choo <minsoochoo0122@proton.me> wrote:
Firs= t, sorry for late response.

cglogic, thank you for bringing up this issue again since I near= ly forgot that this issue was still open.

Warner, as I can't access to my FreeBSD instan= ce until the end of August, but I can still edit and push the code through = my Arm Mac. This means that I can't test the updated code on my machine= , but I can join the review process and listen to change proposals.

I'll open a Github P= R in a few hours. (The phabricator review will stay opened just in case)
On Monday, July 22nd, 2024 at 5:08 AM, Warner Losh <imp@bsdimp.com> wrote:


On Sun, Jul 21, 2024 at 2= :03=E2=80=AFPM cglogic <cglogic@protonmail.com= > wrote:

On Sunday, July 21st, 2024 at 6:54 AM, Warner Losh <imp@bsdimp.com> wrote:


On Sat, Jul 20, 2024 at 1= :59=E2=80=AFAM cglogic <cglogic@protonmail.com= > wrote:
Hello FreeBSD community,

After Jason Evans stepped aside from = maintaining jemalloc in FreeBSD, it's not updating in time anymore.
Version 5.3.0 was released = May 6, 2022 and FreeBSD still not imported it into the tree.

There is a pending review https://reviews.freebsd.org/D41421 from Aug 11, 2023.
I'm succes= sfully running FreeBSD/amd64 system with D41421 applied for 8 months, as we= ll as many other people.

= Can it be reviewed and committed to CURRENT?
Or, if there is no committe= rs willing to do it, can commit bit be given to submitter or another person= willing to do this?
=
It's very disappointing when users spend their time to fi= ll such gaps and their efforts just ignored by the developers.
Every year FreeBSD Community Survey asking about user experience in contr= ibuting to FreeBSD.
Here you can see an example of su= ch contributing.


Firs= t, thank you for being persistent and continuing to bring it up. It's i= mportant to do that to make sure this (and your many other) contribution do= esn't fall on the floor.

And to be fair, w= e're only 3 months since the last update. Still, quite a bit longer tha= n you should have to wait, but not nearly the year the original date sugges= ts.

And this is a perfect storm of "how t= he project is bad at accepting contributions":
(1) The origi= nal submission was close to the 14 branch creation time. This meant that we= weren't well prepared to look at it since it is such an invasive chang= e (at least on its surface). It also slowed the initial response...
(2) There was a number of back and forth requests for changes, which= took time to sort out...
(3) The size of this is huge, well beyo= nd the capacity of Phabricator to review accurately...
(4) It'= ;s a vendor import. That means we can't just drop the Phabricator revie= w into the tree...
(5) It's phabricator: this is a great tool= for developers, but we have a terrible track record of using it for intake= from new contributors. We don't have any oversight at all over this to= ol, at there's at best tepid and luke warm attempts to look for drop ba= lls.

All of these things are a terrible experience= . I can only apologize. These days, we might steer this towards github, but= the 'vendor import' means you really need someone on the inside, o= r you need to be on the inside to make that work.

= So, how to move forward? Well, I'd like to propose the following:
=
(1) submit all the other Phabricator reviews you have open (they are m= ostly good, or close to good) to github. Github is being actively managed a= nd will make it faster to get things it. It's a much better tool for ne= w contributors (and even frequent contributors of smallish things).
(2) I should do an vendor import of 5.3.0 from github, and do the merge = to a branch and push that to github. You can then layer on your changes and= those can be reviewed more closely as a pull request against the branch I = push. I suspect that most of the issues are sorted out already
(3) I'll land it via that route...

And, if = the sum of the other pull requests and this are good (and I suspect they wi= ll be), then we can talk about commit bits and such.

It's experiences like this which is why I'm trying to stand up g= ithub pull requests as a reliable way to get things and and the best place = to send people...

Thanks again for persistin= g, and also for expressing this criticism that we (hopefully) can use to ma= ke it better.

Warner

Hello.

I'm not the author of = D41421. Just applied the patch to test it 8 months ago. And recently = discovered that it's still not committed.
I can't copy your message to Phabricator becaus= e don't have an account. Please, if you have time, help the auth= or in D41421.

Ah yes. I've been i= n touch with the author for other things, and somehow thought it was you...= . I'll reach out to him via other means...

Wa= rner




--00000000000098fea706282e028f--